This PR defines contributors to a service node as a small contributor if
they contribute <25% of the nodes locked contributions. If a contributor
is small then they are prevented from unlocking their stake for 30 days
after the node is registered. This was in response to operators getting
frustrated with small contributors immediately unlocking after a node
was registered.
If you imagine the following node contribution structure:
Operator: 25%
Contributor 1: 65%
Contributor 2: 10%
Total: 100%
Node is registered on day 0. On Day 1 both the operator and Contributor
1 will be able to unlock their stake and the node begins its 15 day
unlock period.
Contributor 2 however will be prevented from unlocking until day 30.
At this point they will be able to unlock and begin the nodes 15 day
unlock period.
This allows a user to query the daemon and ask for the accrued balance
of oxen rewards that have been batched but not yet paid out.
The endpoint takes in an array of addresses (Strings) and returns an
array of atomic oxen amounts (integers) that reflect the current
database balances for those requested addresses.
For example an address has accrued 2x batch payments of 16.5 oxen but
has not yet been paid out will show 33000000000 being the atomic amount
in the database.
- Replace all cryptonote_config macros with constexpr variables. Some
become integer types, some become chrono types.
- generally this involved removing a "CRYPTONOTE_" prefix since the
values are now in the `cryptonote` namespace
- some constants are grouped into sub-namespaces (e.g.
cryptonote::p2p)
- deprecated constants (i.e. for old HFs) are in the `cryptonote::old`
namespace.
- all the magic hash key domain separating strings are now in
cryptonote::hashkey::WHATEVER.
- Move some economy-related constants to oxen_economy.h instead
- Replaced the BLOCKS_EXPECTED_IN_DAYS constexpr functions with more
straightforward `BLOCKS_PER_DAY` value (i.e. old
`BLOCKS_EXPECTED_IN_DAYS(10)` is now `BLOCKS_PER_DAY * 10`.
- Replaced `network_version` unscoped enum with a scoped enum
`cryptonote::hf`, replacing all the raw uint8_t values where it was
currently accepted with the new `hf` type.
- Made `network_type` a scoped enum so that it now has to be qualified
(network_type::TESTNET) and can't be arbitrarily/unintentionally
converted to/from an int.
- HARDFORK_WHATEVER macros have become cryptonote::feature::WHATEVER
constexpr hf values.
- Add `revision` to rpc hard_fork_info response
- Don't build trezor code at all (previously we were pointlessly
building an empty dummy lib).
A fixed 3750 amount won't work on testnet (and also was set to 3750
atomic units instead of 3750 coins).
This refactor the math a bit so that the amount falls out of the portion
constant when making a registration.
This requires the operator to still contribute 25% of the service node
but another 9 nodes will be allowed to stake to the node makeing 10
contributors total rather than our previous 4.
This updates the coinbase transactions to reward service nodes
periodically rather than every block. If you recieve a service node
reward this reward will be delayed x blocks, if you receive another
reward to the same wallet before those blocks have been completed it
will be added to your total and all will be paid out after those x
blocks has passed.
For example if our batching interval is 2 blocks:
Block 1 - Address A receives reward of 10 oxen - added to batch
Block 2 - Address A receives reward of 10 oxen - added to batch
Block 3 - Address A is paid out 20 oxen.
Batching accumulates a small reward for all nodes every block
The batching of service node rewards allows us to drip feed rewards
to service nodes. Rather than accruing each service node 16.5 oxen every
time they are pulse block leader we now reward every node the 16.5 /
num_service_nodes every block and pay each wallet the full amount that
has been accrued after a period of time (Likely 3.5 days).
To spread each payment evenly we now pay the rewards based on the
address of the recipient. This modulus of their address determines which
block the address should be paid and by setting the interval to our
service_node_batching interval we can guarantee they will be paid out
regularly and evenly distribute the payments for all wallets over this
We could get a rounding error in the contributor spots that would make
the registration fail when passed to oxend. This fixes it by restoring
the missing logic that was accidentally dropped in the
prepare_registration refactor.
Testnet has shown that it is virtually impossible to route around a
checkpoint for a block that had to be rendered unacceptable because it
accepts the future checkpoint which then prevents an alternate block
from being generated at all.
This mitigates it in two ways:
- Don't accept new checkpoint votes before we receive the block
- Delay checkpoint voting & vote distribution by one block (e.g. nodes
will vote on block 12345 only once 12346 is received).
The first allows a manual pop_blocks to remove checkpoints that won't
get re-added (e.g. if the checkpointed block has become unacceptable, as
occurs sometimes in testnet/devnet, but could theoretically occur on
mainnet if we have to do a block reversal).
The second minimizes the number of rejected votes that will occur since
nodes will almost always have the block now when the checkpoint arrives.
Reorders the way questions are asked to flow better. Previously the way
questions was asked was sort of awkward.
The old order, for instance staking a partially open node:
- entire stake?
- fee?
- reserve more?
- how many more?
- operator address?
- operator stake size?
- contributor address?
- how much for contributor?
- more contributors, if any
- not filled okay?
- confirm everything?
where there are related values (e.g. operator addr & stake) intermingled
with around other things (e.g. fee & number of contributors).
With this commit the order is rewritten to:
- operator address?
- entire stake?
- operator stake size?
- fee?
- reserve more?
- how many more?
- contributor 1 addr
- contributor 1 amnt
- more contributors, if any
- not filled okay?
- confirm everything?
Additional changes:
- substantially DRY out the code. There was a huge amount of repeated
code in this section.
- highlight OXEN amounts in cyan.
- I rewrote much of the language to make it clearer and more grammatical
- reformatted the summary table to look a bit better
- the fee in the final summary is highlighted yellow.
- if there are multiple open spots this is now more clearly indicated in
the output and the final summary screen.
Example:
```
$ oxend prepare_registration
2022-04-15 20:11:29.211 I Oxen 'Audacious Aurochs' (v9.2.0-0b38bd982-dev)
Current staking requirement: 15000 OXEN
Enter the OXEN address of the Service Node operator (C/Cancel):
LDoptfyQB3YHbS9cnt2wHdTTj2wtZGPuM48evCFwZpomVajQw4eJ6mDCpXeUNTxsqbTiYytnqEDQNin3XGwp3nReMooMaWG
Will the operator contribute the entire stake? (Y/Yes/N/No/B/Back/C/Cancel): n
The operator contribution must be between 3750 OXEN and 15000 OXEN to meet the staking requirements.
How much OXEN does the operator want to stake? (B/Back/C/Cancel): 5555
Enter operator fee as a percentage of earned rewards [0-100]% (B/Back/C/Cancel): 4.567
Do you want to reserve stakes for specific contributors? (Y/Yes/N/No/B/Back/C/Cancel): y
Number of additional contributors [1-3] (B/Back/C/Cancel): 2
Enter the OXEN address of contributor 1 (B/Back/C/Cancel):
L6JasonXTGW6juHrJgsXTxXH1Jh9u94H8NQ9m4rMwz5a9SZGr2e1cXw9MqnVH6Qx5bTgL5wf7qfvHeNPdwyC63AzMixU5ad
The next contribution must be between 3148.333333333 OXEN and 9445 OXEN to meet the staking requirements.
How much OXEN does contributor 1 want to stake? (B/Back/C/Cancel): 5555
Enter the OXEN address of contributor 2 (B/Back/C/Cancel):
L7jC35E9MnQZu2dS1n8wCqbvVBFpGHJt7iNHgAuEy5is57dzX2uQZDASv4KvwzruxXARxa4omafN6i19QBY7KAcSH1Xz7ZL
The next contribution must be between 1945 OXEN and 3890 OXEN to meet the staking requirements.
How much OXEN does contributor 2 want to stake? (B/Back/C/Cancel): 3000
Total reserved contributions: 14110 OXEN
The total reserved amount (14110 OXEN) is less than the required full stake (15000 OXEN).
The remaining stake (890 OXEN) will be open to contribution from 1 public contributor.
The Service Node will not activate until the full contribution has been filled.
Is this acceptable? (Y/Yes/N/No/B/Back/C/Cancel): y
Registration Summary:
Operator fee (as % of Service Node rewards): 4.567%
Contributor Address Contribution Contr. %
_____________ _____________ _________________ ________
Operator LDoptfyQB..WG 5555.000000000 37.03%
Contributor 1 L6JasonXT..ad 5555.000000000 37.03%
Contributor 2 L7jC35E9M..ZL 3000.000000000 20.00%
(open) (any) 890.000000000 5.93%
Is the staking information above correct? (Y/Yes/N/No/B/Back/C/Cancel):
```
For a open node with 3 open spots the summary table looks like this:
```
Contributor Address Contribution Contr. %
_____________ _____________ _________________ ________
Operator L6JasonXT..ad 6789.000000000 45.26%
(open) (any) >=2737.000000000 >=18.25%
(open) (any)
(open) (any)
```
- The decimal point argument isn't used anywhere.
- change `get_unit()` to return `OXEN` instead of `oxen`
- Add a "strip_zeros" option to print_money that lets you remove
trailing insignificant 0's from the returned amount.
- Add format_money that does print_money (with 0s stripped by default) +
get_unit(). E.g. it returns `"45.67 OXEN"`.
When I issue a `prepare_registration` command the cursor gets positioned
at the beginning of the line instead of the end of the line, so typing
things in results in overwriting the text that I was asked.
E.g.:
```
Current staking requirement: 15000.000000000 oxen
yill the operator contribute the entire stake? (Y/Yes/N/No/C/Cancel):
`-- the y I typed replaced the "W"
```
or:
```
Enter the oxen address for the solo staker (B/Back/C/Cancel):
```
and because the cursor is positioned at the beginning, my address
overwrites the prompt as I type/paste it:
```
L6JasonXTGoxen address for the solo staker (B/Back/C/Cancel):
```
This is caused because we were suspending the readline buffer *after*
printing things out, and something (perhaps local config, or perhaps an
update to libreadline) repositions to cursor at column 0.
This commit suspends the readline handler *before* we print the message,
which makes it sane again:
```
Enter the oxen address for the solo staker (B/Back/C/Cancel): L6JasonXTG
```
Remove misc_language.h: Half of it is unused, half of it is crap doesn't
need to be used, and the two useful things (median calculator and a
scope exit caller) were poorly written.
Rewrote median from scratch and moved it out of epee.
Simplified the scope exit handler and moved it to its own small header
in epee.
openssl is a miserable dependency to fight with, especially for
iOS/Android, and we use it for very little:
- it gets (mis)used to write base64 data to a file in wallet2 for things
like multisig data. However this mode is *not* enabled by default,
basically completely unknown, completely unused, only exists in the
cli wallet, and is just dumb. (Honestly the justification given in
the PR is that "Windows users might want it", presupposing that there
exists Windows users who are capable of generating a multisig wallet
in a CLI-only application and yet are incapable of dealing with binary
files).
- it's a dependency of unbound (in order to do dnssec, I believe).
Unbound itself is fairly useless for Oxen, so I've removed it too:
- it does OpenAlias lookups, which are a Monero thing that has never
been used outside Monero, doesn't work reliably (because it fails
if the result isn't DNSSEC validated) and is pointless when we
have ONS.
- it does DNS resolution on seed nodes, but we have never set seed
nodes by name and just use seed node IPs instead (which seems a
bit better anyway since the DNS lookup leaks some metadata).
- it *was* being used for sha256, but an earlier commit in this PR
already replaced that with libsodium (entirely coincidentally).
- for static deps, it enables HTTPS support for the wallet. However
only the CLI wallet actually supports this (the GUI and mobile wallets
don't), and since oxend hasn't support this for a while I have strong
doubts it is being used anywhere. (Plus wallet3 will do everything
encrypted using zmq/libsodium, so doesn't need this to be secure).
Note here that it is *only* removed by this commit for static builds:
if doing a system build that links to libcurl supporting HTTPS then
HTTPS support will still work.
Libexpat is also gone because it was only there for libunbound.
- Aside from converting code, this commit also:
- Cleans out a bunch of epee garbage that this code touches.
- Removes some of the wipeable_string usage from wallet2 added by
upstream Monero, because that usage was worse than useless: every
instance removed uses a wipeable_string but then *copies the value
into a std::string*, which makes the entire thing useless.
It is, however, *worse* than useless because it is deceptive: a casual
observer would think that the values are being wiped, when in
actuality it only added an extra intermediate buffer that gets wiped
while the final destination doesn't. This deception is worse than
nothing, and so I've removed it.
- Removes epee md5 code. It is entirely unused, and was depending on
the removed hex code.
- Removes a bunch of useless functions from
epee/storages/parserse_base_utils.h: in particular the
exception-to-bool wrappers were only being used in the test suite
(which was dumb: the test suite is perfectly capable of a "does not
throw" assertion).
- De-templatizes the parsing code in parserse_base_utils.h that was only
ever called with a `const char*` "It" (except for one place in the
test suite was which easily fixed), and moved the implementation into
a .cpp file.
We have disabled uPnP by default for nearly a year now (#1423) with no
adverse affects; this commit finishes it off entirely, removing it from
the codebase.
Fixes#1419 -- see discussion in that issue for why UPnP really doesn't
add much utility to Oxen.
All the encoding parts move to oxen-encoding recently; this updates to
the latest version of oxen-mq, adds oxen-encoding, and converts
everything to use oxenc headers rather than the oxenmq compatibility
shims.
This is currently returning all contributions in all service nodes you
are staked to, which causes the mobile wallet to list everyone's stakes
in multi-contribution service nodes as your own.
The wallet needs to be fed a single output plus a sufficient number of
decoys. This transaction was run on the testnet and details were copied
into the test file.
This goes through the remaining code in wallet2 that needed updating to
utilise our newly defined json_rpc methods rather than epee
serialisation to call rpc methods.
The keyring structure contains functions that will take a pending
transaction and fill out the cryptonote::transaction member with
the relevant signature fields ready for serialization.
When building the pending transaction we can call GetFee() to calculate
how much the transaction will cost. It takes a single parameter for the
number of inputs because we will want to specify how many when
estimating.
We then build a list of the potential fees for up to 300 inputs and pass
that to our output selection function which will use it to determine if
the outputs selected will be sufficient to cover the fees. This allows
us to know in advance how much the fees will be rather than trial and
error.
The Transaction Constructor will be used to generate pending
transactions, depending on the type of transaction the pending
transaction class will allow the user to modify items such as tx type
(ONS for example) add additional data to the tx extra field, modify burn
amount etc before finalising the transaction which will select the
necessary outputs, mixins and calculate fees and change amounts. Once
finalised the pending transaction will be forwarded to a signing
structure for further action.
tx scanning for basic transactions working
- TODO: subadresses. The scanning code is there, but it does not
currently know/care about any subaddresses.
daemon comms for basic syncing working
(multi-)wallet sync more or less working properly
- seem to have a dangling shared_ptr somewhere when removing a wallet from
daemon comms, so not working perfectly yet.
Lots of TODOs and cleanup needed, as well as further features of course.
This communicates with the Ledger over TCP, which is what the ledger
emulator requires.
To use, specify:
--hw-device LedgerTCP --hw-device-address localhost:9999
to the wallet command-line arguments.
- Clean up a bunch of not very nice C and older C++ code in the
`device/` tree.
- Rename device/device_io_* source files to io_*, and remove their `_io`
suffix from the class names (since they are *already* inside an `io`
namespace).
The public rpc port code from Monero is not used on Oxen (we have no
ability to enable it because we didn't want it), but still carried
around and printed pointlessly. This removes it entirely.
Along the way I ran into some really nasty p2p code using templates for
absolutely no reason at all, so I fixed that crap, and moved some p2p
serialization out of headers into a .cpp file. (This only scratches the
surface, but as we're going to replace the p2p code entirely eventually
I don't want to waste time trying to polish a turd).
Updates GET_PEER_LIST to new RPC.
Removes GET_PUBLIC_NODES because we don't use this option at all in
oxend. (At the point where we need a decentralized public node
repository we'll almost certainly want to use the service node network
to do it, not the p2p layer).
We have no need for any of this, so just remove it rather than convert
it.
(This doesn't drop block generation support, which we can still do via
start/stop mining calls, but rather just the ability to hook up an
external miner, which we are unlikely to ever need again).
This endpoint has bizarre inputs and outputs, so this breaks
compatibility to make them sane.
Old:
"height": [h]
where h must be exactly 1 height (yes, that 1 single height must be in a
list), and output is the raw hash as a string (no dict, etc. around it).
New in:
"heights": [123, 456, ...]
Result:
{
"123": "hash123",
"456": "hash456",
"status": "OK"
}
- Rename endpoint from "send_raw_transaction" to "submit_transaction"
and rename the RPC struct similarly.
- Rename tx_as_hex to just tx, and start accepting it as hex, base64, or
raw bytes. (Transactions formats are consistent enough that we can
easily tell which one we have from the first couple bytes).
- Drop `do_not_relay` parameter as I can see no purpose to ever
submitting a transaction to a daemon that shouldn't be relayed. (The
only use I can see would be to try to exploit a PoW network with
double-spends).
- Drop `do_sanity_checks` parameter; the checks done here should be (and
are!) done by the wallet, not the oxend receiving the completed
transaction.
Not only is this endpoint not used anywhere, but it returns (invalid)
cancerous json output when invoked in current stable oxend:
{
"jsonrpc": "2.0",
"id": "0",
"result": {
"backlog": "l\u0007\u0000\u0000\u0000\u0000\u0000\u0000�\u001f�\u0000\u0000\u0000\u0000\u0000Y�������",
"status": "OK",
"untrusted": false
}
}
where that cancer in the backlog field is the raw bytes copied out from
under a struct with 3 uint64_t's. (It's also invalid json because those
unprintable characters are \xff, which isn't valid UTF-8 encoding, and
thus isn't valid JSON).
Just nuke the whole thing since nothing uses it (probably because it is
such cancer).
Updates to new RPC interface.
Removes distinct `limit`, `limit_up`, `limit_down` command and makes
them all go through `limit` by allowing limit to take the down and up
parameters.
limit_up 123
limit_down 456
becomes
limit 0 123
limit 456 0
or to set both at once:
limit 456 123
Also improves documentation, and allows "default" to be specified for
either limit to return the limit to its default. (-1 is also accepted
for this, but is a pain because on the command line it produces an
invalid option error unless you also prefix the command with `--` to
stop further arg parsing).
Also makes the command actually print an error (instead of nothing at
all) when given invalid arguments. Whoa such innovation!
This drops the dedicated GET_TRANSACTION_POOL endpoint entirely and
folds the txpool fetching into GET_TRANSACTIONS via a new `memory_pool`
parameter. (This is backwards incompatible, of course, but the results
may not be too harmful: the wallet doesn't actually use this endpoint,
so it mainly affects internal oxend commands (fixed here) and the block
explorer (which is fairly easily fixed).
This also removes some of the useless GET_TRANSACTIONS bits such as
decode_as_json.
`fee` and `burned` are now always provided in the return (for both
specific requested transactions and mempool transactions).
As a side effect, this code ventured deep into core/blockchain in terms
of dealing with missed transactions: previously they were always
returned (via lvalue ref) as a vector, and always had to be specified.
This changes to take via pointer to an unordered_set, which can be null
if you don't care about the missed ones. This reduces several calls
that indeed didn't care about collecting the missed values, and improved
virtually all the places that *did* care which need to query which ones
were missed rather than needing a specific order.
- Take the "include_unrelayed" as a parameter instead of switching on on
the admin context; there isn't likely to be anything sensitive there,
and it makes little sense for a wallet to get *different* results just
because it happens to be talking to a localhost oxend.
- Change the histogram code so that it always returns a fixed size
- Various improvements to histogram output; most notably printing a more
useful indication of what the histogram outputs are (e.g. instead of:
Age Txes Bytes
now 0 0
8.0 minutes ago 0 0
16.1 minutes ago 0 0
24.1 minutes ago 0 0
32.2 minutes ago 0 0
40.2 minutes ago 0 0
48.3 minutes ago 0 0
56.3 minutes ago 0 0
64.3 minutes ago 0 0
72.4 minutes ago 2 4559
it now displays the ranges:
Age Txes Bytes
now - 10.6min ago 0 0
10.6min - 21.3min ago 0 0
21.3min - 32.0min ago 0 0
32.0min - 42.6min ago 0 0
42.6min - 53.2min ago 0 0
53.2min - 63.9min ago 0 0
63.9min - 74.6min ago 0 0
74.6min - 85.2min ago 0 0
85.2min - 1.6hr ago 0 0
1.6hr - 1.8hr ago 2 4559
Rather than serializing in the `invoke` call, we now return a variant
(bt, json, or string) from invoke and let the RPC server deal with it.
This significantly simplifies the JSON RPC interface because it lets us
move json objects around, which is particularly useful for building a
json_rpc response and lets us avoid the vector of strings hack that was
needed for epee json serialization.
This also lets the RPC caller decide what it wants to accept -- so the
OMQ RPC can deal with anything (bt, json, binary), while the http
json_rpc handler only accepts json, and the http/legacy binary handlers
accepts json or strings, but not bt_value.
This also fixes a but in the json_rpc handler that failed when given an
"id" value that wasn't a string (such as a number).
- add documentation
- change download/upload speed values to B/s instead of KiB/s
- return times as milliseconds instead of seconds, and rename those
fields from ..._time to ..._ms.
- Code cleanup of state strings
- remove completely unused peer state "idle"
It makes no sense for the definition to be there (except that it was
for epee serialization purposes), so move it into tx_memory_pool where
it belongs, and convert it to json on the fly for the rpc method that
wants it.
Autodetect is not that useful in the first place, even if we weren't
PoS, but is especially useless now.
miner_conf.json is this probably-never-used feature to load some extra
crap into the tx_extra. Just kill it.
Remove the ability (and RPC endpoint) to print the hashrate periodically
to stdout because that is pretty dumb.
Removes a bunch of epee garbage and replaces it with std::chrono.
In particular:
- TIME_MEASURE_* macros are gone because they are garbage.
- epee::get_tick_count() is gone because it is garbage.
- epee::get_ns_count() is gone because it is garbage.
- TIME_MEASURE_NS_* macros are gone because they are unused and garbage.
- PROFILE_FUNC_* macros are gone because they are unused and garbage.
- profile_tools::local_call_account is gone because it is unused and
garbage.
- various places passing around ints (which could be seconds,
milliseconds, or nanoseconds) changed to pass std::chrono duration
types.
- Make it compile/work
- Take outputs as a list of output indices. We never need "amount" in
Oxen (it will always be 0).
- Add backwards compatibility to still take outputs as
[{"amount":0,"index":n},...] when making a json request.
- Add an "as_tuple" bool parameter: if specified then you get results in
a much denser tuple form rather than a dict, which can make a huge
difference in the response size when requesting many outputs.
- Supports loading into tuples, std::arrays, std::pairs from a list of
the correct size.
- Supports loading into vectors of arbitrary types (include vector,
tuples, binary types, etc.)
- Removes system_clock type as we aren't using it (and because it won't
be recursively handled nicely when nested, especially for json).
Use `- \p paramname Description` rather than `\arg \c paramname
Description`.
\arg is for compatibility with some particular doc format and is
equivalent to - except that - can be nested while \arg cannot.
\c and \p are equivalent, but \p is explicitly intended for parameters.
Updates the internal callers of get_service_nodes to properly deal with
the new request format/fields.
Note that this also introduces some significant changes to the
`print_sn` output:
- contributor formats are more compact (one line per contributor), show
reserved amounts (if != contributed amouont), and show the stake %.
- reworded "Operator Cut (% of Fee Reward)" to "Operator Fee"
- checkpoint/pulse vote format is completely overhauled to list voted
and missed heights separately.
- pulse votes now show the pulse round for non-zero round results.
- timestamp checks renamed to Quorumnet tests
- qnet & timesync results now just print the number of successes/fails
rather than a list of true/false values.
Converts request parsing and result generation for get_service_nodes and
get_service_node_status.
Note that there are some significant changes here (due to no longer
having to be under epee's output constraints) that significantly reduce
the size of an all-service-nodes request by nearly half, most notably in
a significantly more compact representation of vote/participation
results.
- added a "decomm_reasons" which decomposes the reason codes into string
values such as:
"decomm_reasons": {
"all": ["uptime"],
"some": ["pulse", "lokinet"]
}
- reworded/fixed/expanded various field descriptions
- Various Loki currency references changed to OXEN
- added "operator_fee" which is a rounded operator fee expressed as
thousanths of a percent (e.g. 100000 means 100%, 4567 means 4.567%).
This is inaccurate but much easier to deal with than the raw
portions_for_operator value (especially because the value *usually*
exceeds what can be stored in a double, and thus is not safe with
various JSON parsers).
- added "swarm" which is a hex string representation of the swarm_id,
because swarm_id also isn't double-storage safe.
- reserved and total_reserved fields are now only included if they are
actually different from the amount/total_contributed values.
- various uptime-proof-dependent values (like versions, public ip,
ports) are now omitted if we haven't received an uptime proof yet.
- Flipped inverted descriptions of timestamp participation / synced
values in the RPC documentation.
- checkpoint/pulse/timecheck/timesync participations data is completely
redesigned to be less bloated; the names are now changed to
"checkpoint_votes" and "pulse_votes" and information is returned much
more compactly using lists of lists and/or plain values rather than
lists of dicts that repeat a ton of fields.
- Renamed timestamp_participation in the RPC to quorumnet_tests since,
although we're doing the test for time checking, the main purpose of
the participation is to see that we can reach them. As above, the
result value is significantly more compact: [SUCCESS,FAILS] instead of
an array of dicts.
- Similarly renamed timesync_status to timesync_tests ("status" was
fine, but changing the value completely to [SUCCESS,FAILS] warrants a
name change).
- dropped the "as_json" field because it is asking oxend to
double-encode json, and that is retarded.
- also applies all the same refactoring to get_service_node_status
(which is really just a shortcut for getting the current node's status
out of get_service_nodes).
- overhauled how "fields" works to take a list of field names (the dict
is name: bool values is still supported for backwards compatibility).
Also extended fields to handle "locked_contributions" as a supported
field (so that you can easily omit it, since it often isn't cared
about).
At MERROR they don't print at the default --log-level=0, so you get
output such as:
2021-08-06 18:04:08.073 I Starting up oxend services...
2021-08-06 18:04:08.073 I Starting core
2021-08-06 18:04:08.073 F Failed to start core
2021-08-06 18:04:08.173 I Deinitializing daemon objects...
with no actual error displayed as to why it failed to start. With this
change you get:
2021-08-06 18:04:14.004 F Please specify an IPv4 public address which the service node & storage server is accessible from with: '--service-node-public-ip <ip address>'
2021-08-06 18:04:14.004 F IMPORTANT: One or more required service node-related configuration settings/options were omitted or invalid; please fix them and restart oxend.
2021-08-06 18:04:14.004 F Failed to start core
- Moves the json binary handling code into new rpc_binary.h
- Adds support for storing/loading vectors of binary types
(crypto::hash, etc.), and vectors of strings.
*Nothing* calls `print_money()` with a decimal place value, for good
reason: it's a stupid thing to want to do. Similarly nothing gives a
decimal places argument to `get_unit()`.
This removes the dumb unused decimal place argument.
- make checkpoint and pulse participation distinct types
- make the container act a little more like an stl container
- replace check_participation() with a failures() method that just
returns the number of failures (so the caller can decide whether that
is bad or not).
- As a consequence of the above, failures now trigger on N+1 failures,
while previously they required N+1 failures *and* 8 total responses.
There is no need to wait for all 8 as far as I can see since we will
be failing no matter what the next 3 responses are.
- Removed the serialization code (hopefully this doesn't break
everything outside the RPC code).
- Simplify/DRY the code that records participation
The bt-encoding, in particular, is aimed at more efficient wallet3 rpc
interactions with a daemon.
This removes the rigidity from RPC request responses, making it easier
to return useful constructs that the current epee code doesn't support
(like lists of lists), and also adds support for bt-encoded RPC requests
(intended to eventually replace the ".bin" endpoints with Monero NIH
binary encoding).
Instead endpoints now set a nlohmann::json response object with whatever
arbitrary values they want without having to add a bunch of limited epee
serialization macro hell.
So, for example, to set the value "foo" to 20, you use:
rpc.response["height"] = 20;
Binary values (for things like hashes) are handled by going through a proxy object:
rpc.response_hex["hash"] = hash; // hash is a crypto::hash
which, for json, is equivalent to:
rpc.response["hash"] = tools::type_to_hex(hash);
but when the response is to be bt-encoded it leaves it as binary for
more efficient transfers.
There is also a `response_b64` that converts json-destined values to
base64 instead of hex (and again leaves bt-destined values as binary).
Parsing of incoming requests now moves to a new
core_rpc_server_command_parser file, which does free-form parsing from a
nlohmann::json or bt_dict_consumer, and has various templated helper
functions to make this easier.
This is preliminary: this builds the basic infrastucture for handling
requests, and converts three endpoints:
- get_info
- get_height
- ons_resolve
Currently only `make daemon` builds (there is code in the wallet that
hasn't been updated yet), and the other RPC interfaces are disabled by
this commit (until they get converted to the new style).
It doesn't compiler and I can't be arsed to fix it; if someone really
needs 32-bit they probably don't care about performance of turtle
hashing in oxend anyway, so the portable one is fine.
The cn_turtle_hash code makes GCC misoptimization in multiple ways under
LTO, so just force it off for GCC-11 (Hopefully the bugginess gets fixed
by GCC-12).
gcc-8's unique_ptr apparently isn't okay with a forward declared type.
This reverses how proof and service_node_list include/fwd declare each
other to get around it.
This aren't used or supported by anyone in Oxen land (as far as I know)
and have not been maintained since the beginning, so hide them behind
disabled-by-default cmake options.
This cuts about 10% off the wallet2 compilation time and 3% off the
simplewallet compilation time, plus gets rid of the page of mms crap in
the cli wallet's "help" screen.
This code is bitrotting, doesn't compile, and isn't being maintained
anymore.
The integration test suite was an interesting idea, in early Loki days,
but is no longer being maintained and is quite cumbersome to run (for
instance, it is not possible to run it via CI because it depends on
xterm to actually run). The code to actually run it (in doy-lee's
loki-integration-testing repository) is also a large burden of "janky"
code that isn't worth maintaining.
Remove this from the code; if someone wants to pick it back up in the
future reverting this commit shouldn't be too difficult (though I'd
suggest that a much better approach to integration testing would be to
run different daemons/wallets via rpc commands, as the network-tests do,
rather than trying to feed stdin and parse stdout from running
individual oxends/wallets).
The get earliest version returns HF 18.1 and this means that our blink
fees that were enabled in HF18 are no longer accepted. Push through to
allow the fees to be used 4000 blocks earlier (current height)
Wallet API is really messy with threads -- the refresh thread can run at
any time and change internal wallet state which breaks just about
everything.
This puts all non-trivial wallet access from wallet_api behind a lock of
the refresh thread to lock it out from refresh while other operations
are underway.
It seems that the `m_tinfo` can be null, sometimes, when `m_cursors ==
&m_wcursors` is true, and the upstream Monero code (which is pure macro)
doesn't touch the bool in such a case.
For some reason this started segfaulting now, only on macos, only on a
release build because of the access into `m_tinfo`.
The workaround (which is indeed a correct fix) appears to avoid the
segfault, but the segfault could retrigger if that invariant doesn't
hold (and it isn't immediately obvious why that invariant *should*
hold).
This, like pretty much all of the LMDB code, is garbage.
Though this statement seems dubious since "The Oxen Project" is no a
legal entity. Should perhaps be "Oxen Privacy Tech Foundation"? Or
alternatively we could have a statement somewhere that "The Oxen
Project" refers to the OPTF + code contributed by Oxen community members
through github, etc.
Okay enough copyright law.
Remove support for old (non-bt) proofs with the 9.2.0 snode revision
block (I'm not 100% sure on what to call this; "snode revision"? "soft
fork"? "spork"?).
Also bumps the working version to 9.2.0; this likely isn't release
ready, but allows for testing of this on testnet.
Snode revisions are a secondary version that let us put out a mandatory
update for snodes that isn't a hardfork (and so isn't mandatory for
wallets/exchanges/etc.).
The main point of this is to let us make a 9.2.0 release that includes
new mandatory minimums of future versions of storage server (2.2.0) and
lokinet (0.9.4) to bring upgrades to the network.
This slightly changes the HF7 blocks to 0 (instead of 1) because,
apparently, we weren't properly checking the HF value of the
pre-first-hf genesis block at all before. (In practice this changes
nothing because genesis blocks are v7 anyway).
This also changes (slightly) how we check for hard forks: now if we skip
some hard forks then we still want to know the height when a hard fork
triggers. For example, if the hf tables contains {7,14} then we still
need to know that the HF14 block height also is the height that
activates HF9, 10, etc.
This makes reachability testing activate at HF19. We probably want to
come back and update this before HF19, but for now we just check but
don't enforce lokinet reachability.
It works just like storage server testing.
Renames the report_peer_storage_server_status to report_peer_status, and
repurposes the code to handle both SS and lokinet.
This *doesn't* need a HF by design because the reason bit field was
deliberately designed so that we can add reason fields (older clients
will just ignore unknown bits).
When doing a lookup by owner in simplewallet (`ons_by_owner`) the wallet
was always writing "Owner: ..." for whatever owner row was requested,
*not* the actual Owner row returned.
In particular, when we get back a record where WalletX is the backup
owner and WalletA is the primary owner then when looking up ons records
by owner for WalletX it would print:
Owner: WalletX
BackupOwner: WalletX
Instead of properly showing the record owner details of:
Owner: WalletA
BackupOwner: WalletX
This fixes it.
When looking up the ONS names by owner (e.g. ons_by_owner in the cli
wallet) the oxend RPC method returns an error for any records that get
returned where the looked up owner is a backup owner, because it tries
to map (only) the primary owner to a response index and fails if it got
back a record from the database that wasn't in the list of requested
addresses.
This fixes the RPC method to properly check both owner and backup owner.
This index was being created on a column that doesn't exist, and
apparently because it was quoted (until the commit earlier in this PR),
sqlite was apparently treating it as a string literal.
Cursed AF.
Drop the old name and recreate the index.
This index was only getting created on upgraded (from v7, I think)
databases but not new ones.
Also adds a comment about what the "prune" queries actually do (because
the `>=` seems really counterintuitive to me until I figured out that
they are only meant for handling a rollback).
* Make preparing registration fail if lokinet or the storage server have not received a ping yet
* Add prepare_registration boolean default
* Add signature override for get_human_time_ago to accept an optional uint64. Add stricter comparisons to ping checks
* Remove duplicate logic
* Remove method override and assign/cast last_lokinet_ping/last_storage_server_ping
* Use auto definition and static_cast res.last_lokinet_ping
This option is incredibly misguided: exceptions are a normal part of C++
error handling that are used *as intended* in lots of places in the
code. Spewing massive amounts of output every time any exception is
thrown anywhere (even when caught!) is terrible.
More than that, we don't ever build with it enabled (for the above
reasons) so this is all just unused code.
We dropped the contrib/depends build system quite a while ago (because
it was nasty), but there are still various DEPENDS checks scattered
through cmake that are just dead code now. This removes them.
bt-encoded proofs have a bug for nodes with different legacy/ed25519
pubkeys that isn't easily solvable (it needs a fix on *both* sender and
receiver ends). That fix is in here (in uptime_proof.cpp) but that
isn't enough to solve it immediately.
This works around the issue by submitting old-style proofs if we are a
node with different legacy/ed25519 pubkeys.
The timestamp inside the proof is only for signature validation, but we
were using it in some places as the uptime proof time, but not updating
it everywhere we needed to. This fixes it by using our own timestamp
for all local timed events (e.g. when we received it, when the node is
not sending proofs, etc.) to fix the issue.
The displaying of ONS transactions in show_transfers and
export_transfers previously showed the dummy destination address which
contained zero bytes and converted to a nonsense address. Created a new
ONS type for the display functions and removed the displaying of the destination
address.
In addition refactored how we determine that a transaction was either a
staking, ons or output transaction as we were previously parsing the
tx_extra where the data was already in the
cryptonote::transaction::type.
Finally renamed the wording for staking rewards. Previously "miner" now
"reward"
A couple people have messaged me now because they tried transferring
keys and used the `restore` command on their legacy `key` file, but it
restored an ed25519 key.
This adds a red warning if attempting to restore an ed key to a filename
ending in `/key` with a note about probably wanting the restore-legacy
command instead.
PR #1433 getting merged changed the fees within HF18 on testnet, which
broke syncing/ONS rescanning because the per-merged testnet has higher
fee ONS txes on it.
This adds a hack to allow wrong fees for blocks before yesterday.
Clang warns about http_server having a non-virtual destructor; we aren't
actually doing anything that would cause problems, but the warning is
legit and a correct thing to fix.
macOS's system_clock apparently only has microsecond resolution while
steady_clock has nanosecond, so the conversion here was failing (because
time_point conversions are only implicit when converting to a more
precise type).
The important part here is removing this line:
if (swarm_to_snodes.size() == 1) return MAX_ID / 2;
because, if we end up in a case where we have only one swarm and it
*already* has that ID (e.g. create 2, which will be [MAX/2,0] then drop
0) then this returns a swarm_id that already exists, which is bad
because then we fail to insert the new swarm, a service node gets left
with an unassigned swarm id, and that then causes issues in SS because
that node thinks it is deactivated because it doesn't have a swarm id
(yet it *is* in the active nodes list, so other network members still
try to talk to it).
This moves all the responsibility of ping testing (deciding when it's
unreachable, etc.) into oxend, allowing for better reporting on SS ping
results and eliminating some edge cases that can lead to oxend and
storage server getting "stuck" thinking each is in a different state.
The reason behind the limit is that the burn amount was supposed to be
encoded using varint encoding and therefore the limit was to make sure
that once we figured out the final burn amount and put it in, we were
guaranteed not to be making the TX extra any bigger (just in case that
could end up making the overall tx get a couple bytes bigger and break
the tx size limit).
However, it never actually *used* varint encoding: instead it is encoded
as a raw, full size uint64_t value of 8 bytes regardless of the value,
so this check is not actually doing anything. (And if we changed it
to a varint we'd break the protocol, so just leave it).
It also turns out that this comment was wrong:
This value (~4398 OXEN) was chosen because it's unlikely to ever be
needed to be burned in a single transaction
Also I hear that some users really do need more than 640kB RAM. ;-)
Jason Rhinelander
Sean
Thomas Winget
javabudd