This updates the coinbase transactions to reward service nodes
periodically rather than every block. If you recieve a service node
reward this reward will be delayed x blocks, if you receive another
reward to the same wallet before those blocks have been completed it
will be added to your total and all will be paid out after those x
blocks has passed.
For example if our batching interval is 2 blocks:
Block 1 - Address A receives reward of 10 oxen - added to batch
Block 2 - Address A receives reward of 10 oxen - added to batch
Block 3 - Address A is paid out 20 oxen.
Batching accumulates a small reward for all nodes every block
The batching of service node rewards allows us to drip feed rewards
to service nodes. Rather than accruing each service node 16.5 oxen every
time they are pulse block leader we now reward every node the 16.5 /
num_service_nodes every block and pay each wallet the full amount that
has been accrued after a period of time (Likely 3.5 days).
To spread each payment evenly we now pay the rewards based on the
address of the recipient. This modulus of their address determines which
block the address should be paid and by setting the interval to our
service_node_batching interval we can guarantee they will be paid out
regularly and evenly distribute the payments for all wallets over this
We could get a rounding error in the contributor spots that would make
the registration fail when passed to oxend. This fixes it by restoring
the missing logic that was accidentally dropped in the
prepare_registration refactor.
Reorders the way questions are asked to flow better. Previously the way
questions was asked was sort of awkward.
The old order, for instance staking a partially open node:
- entire stake?
- fee?
- reserve more?
- how many more?
- operator address?
- operator stake size?
- contributor address?
- how much for contributor?
- more contributors, if any
- not filled okay?
- confirm everything?
where there are related values (e.g. operator addr & stake) intermingled
with around other things (e.g. fee & number of contributors).
With this commit the order is rewritten to:
- operator address?
- entire stake?
- operator stake size?
- fee?
- reserve more?
- how many more?
- contributor 1 addr
- contributor 1 amnt
- more contributors, if any
- not filled okay?
- confirm everything?
Additional changes:
- substantially DRY out the code. There was a huge amount of repeated
code in this section.
- highlight OXEN amounts in cyan.
- I rewrote much of the language to make it clearer and more grammatical
- reformatted the summary table to look a bit better
- the fee in the final summary is highlighted yellow.
- if there are multiple open spots this is now more clearly indicated in
the output and the final summary screen.
Example:
```
$ oxend prepare_registration
2022-04-15 20:11:29.211 I Oxen 'Audacious Aurochs' (v9.2.0-0b38bd982-dev)
Current staking requirement: 15000 OXEN
Enter the OXEN address of the Service Node operator (C/Cancel):
LDoptfyQB3YHbS9cnt2wHdTTj2wtZGPuM48evCFwZpomVajQw4eJ6mDCpXeUNTxsqbTiYytnqEDQNin3XGwp3nReMooMaWG
Will the operator contribute the entire stake? (Y/Yes/N/No/B/Back/C/Cancel): n
The operator contribution must be between 3750 OXEN and 15000 OXEN to meet the staking requirements.
How much OXEN does the operator want to stake? (B/Back/C/Cancel): 5555
Enter operator fee as a percentage of earned rewards [0-100]% (B/Back/C/Cancel): 4.567
Do you want to reserve stakes for specific contributors? (Y/Yes/N/No/B/Back/C/Cancel): y
Number of additional contributors [1-3] (B/Back/C/Cancel): 2
Enter the OXEN address of contributor 1 (B/Back/C/Cancel):
L6JasonXTGW6juHrJgsXTxXH1Jh9u94H8NQ9m4rMwz5a9SZGr2e1cXw9MqnVH6Qx5bTgL5wf7qfvHeNPdwyC63AzMixU5ad
The next contribution must be between 3148.333333333 OXEN and 9445 OXEN to meet the staking requirements.
How much OXEN does contributor 1 want to stake? (B/Back/C/Cancel): 5555
Enter the OXEN address of contributor 2 (B/Back/C/Cancel):
L7jC35E9MnQZu2dS1n8wCqbvVBFpGHJt7iNHgAuEy5is57dzX2uQZDASv4KvwzruxXARxa4omafN6i19QBY7KAcSH1Xz7ZL
The next contribution must be between 1945 OXEN and 3890 OXEN to meet the staking requirements.
How much OXEN does contributor 2 want to stake? (B/Back/C/Cancel): 3000
Total reserved contributions: 14110 OXEN
The total reserved amount (14110 OXEN) is less than the required full stake (15000 OXEN).
The remaining stake (890 OXEN) will be open to contribution from 1 public contributor.
The Service Node will not activate until the full contribution has been filled.
Is this acceptable? (Y/Yes/N/No/B/Back/C/Cancel): y
Registration Summary:
Operator fee (as % of Service Node rewards): 4.567%
Contributor Address Contribution Contr. %
_____________ _____________ _________________ ________
Operator LDoptfyQB..WG 5555.000000000 37.03%
Contributor 1 L6JasonXT..ad 5555.000000000 37.03%
Contributor 2 L7jC35E9M..ZL 3000.000000000 20.00%
(open) (any) 890.000000000 5.93%
Is the staking information above correct? (Y/Yes/N/No/B/Back/C/Cancel):
```
For a open node with 3 open spots the summary table looks like this:
```
Contributor Address Contribution Contr. %
_____________ _____________ _________________ ________
Operator L6JasonXT..ad 6789.000000000 45.26%
(open) (any) >=2737.000000000 >=18.25%
(open) (any)
(open) (any)
```
When I issue a `prepare_registration` command the cursor gets positioned
at the beginning of the line instead of the end of the line, so typing
things in results in overwriting the text that I was asked.
E.g.:
```
Current staking requirement: 15000.000000000 oxen
yill the operator contribute the entire stake? (Y/Yes/N/No/C/Cancel):
`-- the y I typed replaced the "W"
```
or:
```
Enter the oxen address for the solo staker (B/Back/C/Cancel):
```
and because the cursor is positioned at the beginning, my address
overwrites the prompt as I type/paste it:
```
L6JasonXTGoxen address for the solo staker (B/Back/C/Cancel):
```
This is caused because we were suspending the readline buffer *after*
printing things out, and something (perhaps local config, or perhaps an
update to libreadline) repositions to cursor at column 0.
This commit suspends the readline handler *before* we print the message,
which makes it sane again:
```
Enter the oxen address for the solo staker (B/Back/C/Cancel): L6JasonXTG
```
Remove misc_language.h: Half of it is unused, half of it is crap doesn't
need to be used, and the two useful things (median calculator and a
scope exit caller) were poorly written.
Rewrote median from scratch and moved it out of epee.
Simplified the scope exit handler and moved it to its own small header
in epee.
All the encoding parts move to oxen-encoding recently; this updates to
the latest version of oxen-mq, adds oxen-encoding, and converts
everything to use oxenc headers rather than the oxenmq compatibility
shims.
This code is bitrotting, doesn't compile, and isn't being maintained
anymore.
The integration test suite was an interesting idea, in early Loki days,
but is no longer being maintained and is quite cumbersome to run (for
instance, it is not possible to run it via CI because it depends on
xterm to actually run). The code to actually run it (in doy-lee's
loki-integration-testing repository) is also a large burden of "janky"
code that isn't worth maintaining.
Remove this from the code; if someone wants to pick it back up in the
future reverting this commit shouldn't be too difficult (though I'd
suggest that a much better approach to integration testing would be to
run different daemons/wallets via rpc commands, as the network-tests do,
rather than trying to feed stdin and parse stdout from running
individual oxends/wallets).
Snode revisions are a secondary version that let us put out a mandatory
update for snodes that isn't a hardfork (and so isn't mandatory for
wallets/exchanges/etc.).
The main point of this is to let us make a 9.2.0 release that includes
new mandatory minimums of future versions of storage server (2.2.0) and
lokinet (0.9.4) to bring upgrades to the network.
This slightly changes the HF7 blocks to 0 (instead of 1) because,
apparently, we weren't properly checking the HF value of the
pre-first-hf genesis block at all before. (In practice this changes
nothing because genesis blocks are v7 anyway).
This also changes (slightly) how we check for hard forks: now if we skip
some hard forks then we still want to know the height when a hard fork
triggers. For example, if the hf tables contains {7,14} then we still
need to know that the HF14 block height also is the height that
activates HF9, 10, etc.
It works just like storage server testing.
Renames the report_peer_storage_server_status to report_peer_status, and
repurposes the code to handle both SS and lokinet.
This *doesn't* need a HF by design because the reason bit field was
deliberately designed so that we can add reason fields (older clients
will just ignore unknown bits).
* Make preparing registration fail if lokinet or the storage server have not received a ping yet
* Add prepare_registration boolean default
* Add signature override for get_human_time_ago to accept an optional uint64. Add stricter comparisons to ping checks
* Remove duplicate logic
* Remove method override and assign/cast last_lokinet_ping/last_storage_server_ping
* Use auto definition and static_cast res.last_lokinet_ping
This moves all the responsibility of ping testing (deciding when it's
unreachable, etc.) into oxend, allowing for better reporting on SS ping
results and eliminating some edge cases that can lead to oxend and
storage server getting "stuck" thinking each is in a different state.
Improves the oxend<->storage server communications protocol:
- pass storage server HTTPS port as part of the storage server ping
(which already carries the also-required OMQ port) rather than needing
to provide it when starting up oxend. --storage-server-port is now
obsolete (and ignored, if specified).
- Fix up the internal API to use `storage_https_port` and
`storage_omq_port` rather than `storage_port` and `storage_lmq_port`.
- Redo and the SS ping RPC endpoint so that it is less verbose and more
closely matches the lokinet endpoint; instead of:
{ "version_major": 2, "version_minor": 0, "version_patch": 9, "storage_lmq_port": 22222 }
we now expect:
{ "version": [2,0,9], "https_port": 11111, "omq_port": 22222 }
- Tweaks the (not-yet-released) SS proof key names: "s"->"shp" and "slp"->"sop"
The status doesn't have a nice way to tell it is active or awaiting
contributions, so add it.
If "all" reasons don't give any results then try showing the "any"
reasons, and if that doesn't work, show a "reason(s) not available"
message.
- Put SS/lokinet version on same line
- make checkpoint/pulse/timestamp/timesync each take one line instead of
three
- instead of [height,round,vote] for pulse just print [height,vote] and
add a '+R' on height if for a pulse round > 0.
- remove the space after the , in the checkpoint/pulse/etc. lines to
save a little
random sampling of service nodes, call timestamp lmq message
checks timestamp of 5 service nodes, if local time is 30 seconds different from 80% of the nodes tested then warn user
tracks external timesync status and timestamp participation of service nodes
clean up includes
new template struct for participation history, individual types for participation entry
refactor checking participation
update select_randomly, move the testing for variance overflow
version locks, bump to 8.1.5
explicit casting for mac & clang
note to remove after hard fork
timestamp debugging log messages
debugging messages for before timesync - before message sent
logging errord with compiling
print version and change add_command to add_request_command
log if statement test
std::to_string replaced with tools::view_guts for x25519 key
check if my sn is active before sending timestamp requests
logging the failures
checking if statement for success of message
more logging, if guards arn't passing
more logging, successfully tests if service node might be out of sync
more tests before we decide we are out of sync
logging output if sn isn't passing tests
if check_participation fails then disconnect
print timestamp status
remove saving variance from the participation history
reduce MIN_TIME_IN_S_BEFORE_VOTING
reset participation history on recommission
undo reduction in startup time
reduce log levels
Set hardfork time in testnet
- Remove some useless epee functions, and add deprecated markers to ones
that have good replacements already.
- Don't use boost::lexical_cast when std::to_string or direct stream
output can be used just as well.
- Get rid of dumb epee "include_base_utils.h" header
The existing boost::format was a monstrosity, so rewrote it to build a
stringstream instead.
- No longer print anything about mining on HF16 unless we are actually
mining.
- Don't print net hash for HF16+ since it is not valid.
- don't show incoming/outgoing connections count if response looks
restricted.
- Print "next fork in x.x hours" starting at 24.0 hours instead of 12.0
hours.
- Replace ridiculous integer-math-done-on-floats with actual integer
math.
- Instead of the Pulse quorums validators recording participation
between each other- so failures may not manifest in a decommission until
the several common nodes align and agree to vote off the node.
Voting now occurs when blocks arrives, validators participating in the
generation of the block are marked. This is shared information between
all nodes syncing the chain so decommissions are more readily agreeable
and acted upon in the Obligation quorums immediately.
- Allow retrieving multiple blocks headers by height in the same request
(matching the API already available to do that for by-hash lookups).
- make pow_hash an optional so that it gets omitted entirely if not
requested
- Likewise for block header responses when using the array arguments
instead of the single value argument.
Otherwise RPCing into the wallet and triggering a refresh doesn't really
work, i.e.
loki-wallet-cli --wallet-file file --password '' refresh
loki-wallet-cli --wallet-file file --password '' balance <-- this returns 0
- Add opt-in tx-extra parsing to `get_transactions`; this now lets you
get most tx details (including service node tx info) via the RPC.
- Add ability for RPC block header requests to include a list of tx
hashes contained in the block.
- remove long-deprecated txs_as_hex from `get_transactions`: this
essentially doubled the length of requests since txs[i].as_hex has the
exact same data. Despite this being the "new" format for several
years, wallet code was still relying on the "old" format. Since 8.x
already breaks the RPC, this seems a good time to remove it.
- Significantly cleaned up and documented how pruned transactions
results are returned in GET_TRANSACTIONS. Previously it was fairly
uncertain when you'd get a pruned or unpruned transaction. (See
comments added in core_rpc_server_commands_defs.h)
- Stop returning the transaction data hex values when you ask for
`decode_as_json`. The documentation already implied you don't get
it, but you do, and this is virtually always just wasted extra data
when you're asking for the json-decoded value.
- fix miner txes being treated in the RPC as pruned transactions (they
aren't: they just have no prunable data).
- Made a bunch of `get_transactions` fields (including virtually all of
the new parsed tx-extras fields) into std::optional's so that the keys
aren't included when the values are empty.
- Fix bug in `get_transactions` that raised an exception if *some* (but
not all) of the requested transactions were missed. Previously you
could only get all-missed, all-found, or an exception.
In short: epee's http client is garbage, standard violating, and
unreliable.
This completely removes the epee http client support and replaces it
with cpr, a curl-based C++ wrapper. rpc/http_client.h wraps cpr for RPC
requests specifically, but it is also usable directly.
This replacement has a number of advantages:
- requests are considerably more reliable. The epee http client code
assumes that a connection will be kept alive forever, and returns a
failure if a connection is ever closed. This results in some very
annoying things: for example, preparing a transaction and then waiting
a long tim before confirming it will usually result in an error
communication with the daemon. This is just terribly behaviour: the
right thing to do on a connection failure is to resubmit the request.
- epee's http client is broken in lots of other ways: for example, it
tries throwing SSL at the port to see if it is HTTPS, but this is
protocol violating and just breaks (with a several second timeout) on
anything that *isn't* epee http server (for example, when lokid is
behind a proxying server).
- even when it isn't doing the above, the client breaks in other ways:
for example, there is a comment (replaced in this PR) in the Trezor PR
code that forces a connection close after every request because epee's
http client doesn't do proper keep-alive request handling.
- it seems noticeably faster to me in practical use in this PR; both
simple requests (for example, when running `lokid status`) and
wallet<->daemon connections are faster, probably because of crappy
code in epee. (I think this is also related to the throw-ssl-at-it
junk above: the epee client always generates an ssl certificate during
static initialization because it might need one at some point).
- significantly reduces the amount of code we have to maintain.
- removes all the epee ssl option code: curl can handle all of that just
fine.
- removes the epee socks proxy code; curl can handle that just fine.
(And can do more: it also supports using HTTP/HTTPS proxies).
- When a cli wallet connection fails we know show why it failed (which
now is an error message from curl), which could have all sorts of
reasons like hostname resolution failure, bad ssl certificate, etc.
Previously you just got a useless generic error that tells you
nothing.
Other related changes in this PR:
- Drops the check-for-update and download-update code. To the best of
my knowledge these have never been supported in loki-core and so it
didn't seem worth the trouble to convert them to use cpr for the
requests.
- Cleaned up node_rpc_proxy return values: there was an inconsistent mix
of ways to return errors and how the returned strings were handled.
Instead this cleans it up to return a pair<bool, val>, which (with
C++17) can be transparently captured as:
auto [success, val] = node.whatever(req);
This drops the failure message string, but it was almost always set to
something fairly useless (if we want to resurrect it we could easily
change the first element to be a custom type with a bool operator for
success, and a `.error` attribute containing some error string, but
for the most part the current code wasn't doing much useful with the
failure string).
- changed local detection (for automatic trusted daemon determination)
to just look for localhost, and to not try to resolve anything.
Trusting non-public IPs does not work well (e.g. with lokinet where
all .loki addresses resolve to a local IP).
- ssl fingerprint option is removed; this isn't supported by curl
(because it is essentially just duplicating what a custom cainfo
bundle does)
- --daemon-ssl-allow-chained is removed; it wasn't a useful option (if
you don't want chaining, don't specify a cainfo chain).
- --daemon-address is now a URL instead of just host:port. (If you omit
the protocol, http:// is prepended).
- --daemon-host and --daemon-port are now deprecated and produce a
warning (in simplewallet) if used; the replacement is to use
--daemon-address.
- --daemon-ssl is deprecated; specify --daemon-address=https://whatever
instead.
- the above three are now hidden from --help
- reordered the wallet connection options to make more logical sense.
Sean Darcy
Jason Rhinelander
javabudd
Sean Darcy
Doyle