This adds a new table to the batching schema to copy the accrued
balances every so often. This means that we can jump back to a
previous point without popping blocks.
The archiving is triggered in sql every 100 blocks and added to the
archive table, then pruned from the archive table at a later time to
ensure the size is kept small. Rebuilding 100 blocks is pretty
reasonable and should be less than 10s.
For longer distance pop_blocks and blockchain detached every 10k blocks
is kept in the archiving table. This takes longer to rebuild but is
better than rebuilding from scratch.
The blockchain detached function is also added to our regular blockchain
detached hooks so that it gets called every time the blockchain
detaches. Which appears to have caused some issues previously when some
of the modules would detach but batching would be stuck in an advanced
state.
- Remove implicit `operator bool` from ec_point/public_key/etc. which
was causing all sorts of implicit conversion mess and bugs.
- Change ec_point/public_key/etc. to use a `std::array<unsigned char,
32>` (via a base type) rather than a C-array of char that has to be
reinterpret_cast<>'ed all over the place.
- Add methods to ec_point/public_key/etc. that make it work more like a
container of bytes (`.data()`, `.size()`, `operator[]`, `begin()`,
`end()`).
- Make a generic `crypto::null<T>` that is a constexpr all-0 `T`, rather
than the mishmash `crypto::null_hash`, crypto::null_pkey,
crypto:#️⃣:null(), and so on.
- Replace three metric tons of `crypto::hash blahblah =
crypto::null_hash;` with the much simpler `crypto::hash blahblah{};`,
because there's no need to make a copy of a null hash in all these
cases. (Likewise for a few other null_whatevers).
- Remove a whole bunch of `if (blahblah == crypto::null_hash)` and `if
(blahblah != crypto::null_hash)` with the more concise `if
(!blahblah)` and `if (blahblah)` (which are fine via the newly
*explicit* bool conversion operators).
- `crypto::signature` becomes a 64-byte container (as above) but with
`c()` and `r()` to get the c() and r() data pointers. (Previously
`.c` and `.r` were `ec_scalar`s).
- Delete with great prejudice CRYPTO_MAKE_COMPARABLE and
CRYPTO_MAKE_HASHABLE and all the other utter trash in
`crypto/generic-ops.h`.
- De-inline functions in very common crypto/*.h files so that they don't
have to get compiled 300 times.
- Remove the disgusting include-a-C-header-inside-a-C++-namespace
garbage from some crypto headers trying to be both a C and *different*
C++ header at once.
- Remove the toxic, disgusting, shameful `operator&` on ec_scalar, etc.
that replace `&x` with `reinterpret_cast x into an unsigned char*`.
This was pure toxic waste.
- changed some `<<` outputs to fmt
- Random other small changes encountered while fixing everything that
cascaded out of the above changes.
oxen::log::info(...), etc. are a bit too verbose; this simplifies them
to just `log::info(...)`, etc. by aliasing the `oxen::log` namespace
into most of the common namespaces we use in core.
This result is usage that is shorter but also reads better:
oxen::log::info(logcat, "blah: {}", 42);
log::info(logcat, "blah: {}", 42);
This replaces the current epee logging system with our oxen::log
library. It replaces the easylogging library with spdlog, removes the
macros and replaces with functions and standardises how we call the
logs.
Fixes a few merge conflicts, several compilation errors, and
some behavioral incorrectness. Still a few bugs with wallet3
but as far as I can tell wallet2 and daemon etc. should be working
correctly.
The checking if the service node contributers are small miscalculated
atomic oxen. This brings in the correct figure for HF20 and in the
interim brings measures into the wallet and blink to prevent small
contributors from being to unlock in HF19.
This hook *isn't* called after a block is added, but rather it is part
of the block addition process and can abort the whole thing by raising
an exception (or returning false, prior to this PR).
This is unintuitive and causes bugs if using it as a "block has been
added" hook.
bool returns suck in general, but in most cases here they are also a
pain in the ass because *each* place that returns false is also issuing
a log statement. If only there were a way to return error information
to the common caller to have the common caller handle it... oh wait,
there is!
- Instead of inheriting from a pure virtual base class, we know just use
lambdas for hook callbacks.
- The signature of hooks also changes to take an info struct rather than
a list of parameters, so that you don't have to change everything to
ignore unwanted parameters if someone adds something to a hook.
Adds two slightly different versions of prepare_registration: one that
works with the existing 4-person registrations (used up to the hard
fork), and one that produces HF19 amount-based registrations.
This also overhauls the questions to be a bit nicer to input;
specifically:
- allow "max" and "min" as stake amount options, and make "max" the
default.
- eliminate the "do you want to contribute the whole stake?" question
entirely. We instantly figure that out if you choose "max" (or enter
the 15000 manually).
- eliminate the "how many contributors?" question. Instead we just keep
taking additional contributors until you stop entering them.
- move the fee to later, after you've provided contributor info (if not
a full stake).
There is, unfortunately, a *huge* amount of duplication here: I copy and
pasted the entire HF18 registration code and just eliminated the
portions parts of it. This is temporary: as soon as we are into HF19 we
can eliminate the HF18 version entirely.
This adds a new tx registration interpretation for HF19+ by repurposing
the fields of the registration:
- `expiration` becomes `hf_or_expiration`; for "new" registrations it
contains the hardfork number (e.g. 19 for HF19), but the data layout
on chain doesn't change: essentially we determine whether it's a new
registration based on whether the field is <= 255 (interpret as HF) or
not (interpret as expiration).
Changes in "new" registrations:
- stake amounts are atomic OXEN rather than portions. This lets us skip
a whole bunch of fiddling around with amounts that was necessary to
deal with integer truncation when converting between amounts and
portions.
- the fee in the registration data is now a value out of 10000 instead
of a portion (i.e. value out of 2^64-4). This limits fee precision to
a percentage with two decimal places instead of ~17 decimal places.
Internally we still convert this to a portion when processing the
registration for service_node_states, but this makes the registration
itself much simpler and easier to work with (as a human).
- HF19+ registrations no longer have an expiry timestamp (though they do
depend on the hardfork, so they "expire" whenever the next hard fork).
The expiry timestamp was really only there to avoid a registration
amount decreasing too much from the dropping staking requirement.
- Both types are registration are still permitted for HF19, but because
registrations with more than 4 contributors expose bugs in the portion
transfer code (that results in registrations become invalid),
old-style registrations are still limited to 4 contributors.
- HF19 will allow both old and new registrations, so that registrations
generated before the HF will still work, and so that we don't break
testnet which has various "old" registrations on it.
This PR defines contributors to a service node as a small contributor if
they contribute <25% of the nodes locked contributions. If a contributor
is small then they are prevented from unlocking their stake for 30 days
after the node is registered. This was in response to operators getting
frustrated with small contributors immediately unlocking after a node
was registered.
If you imagine the following node contribution structure:
Operator: 25%
Contributor 1: 65%
Contributor 2: 10%
Total: 100%
Node is registered on day 0. On Day 1 both the operator and Contributor
1 will be able to unlock their stake and the node begins its 15 day
unlock period.
Contributor 2 however will be prevented from unlocking until day 30.
At this point they will be able to unlock and begin the nodes 15 day
unlock period.
- Replace all cryptonote_config macros with constexpr variables. Some
become integer types, some become chrono types.
- generally this involved removing a "CRYPTONOTE_" prefix since the
values are now in the `cryptonote` namespace
- some constants are grouped into sub-namespaces (e.g.
cryptonote::p2p)
- deprecated constants (i.e. for old HFs) are in the `cryptonote::old`
namespace.
- all the magic hash key domain separating strings are now in
cryptonote::hashkey::WHATEVER.
- Move some economy-related constants to oxen_economy.h instead
- Replaced the BLOCKS_EXPECTED_IN_DAYS constexpr functions with more
straightforward `BLOCKS_PER_DAY` value (i.e. old
`BLOCKS_EXPECTED_IN_DAYS(10)` is now `BLOCKS_PER_DAY * 10`.
- Replaced `network_version` unscoped enum with a scoped enum
`cryptonote::hf`, replacing all the raw uint8_t values where it was
currently accepted with the new `hf` type.
- Made `network_type` a scoped enum so that it now has to be qualified
(network_type::TESTNET) and can't be arbitrarily/unintentionally
converted to/from an int.
- HARDFORK_WHATEVER macros have become cryptonote::feature::WHATEVER
constexpr hf values.
- Add `revision` to rpc hard_fork_info response
- Don't build trezor code at all (previously we were pointlessly
building an empty dummy lib).
This requires the operator to still contribute 25% of the service node
but another 9 nodes will be allowed to stake to the node makeing 10
contributors total rather than our previous 4.
This updates the coinbase transactions to reward service nodes
periodically rather than every block. If you recieve a service node
reward this reward will be delayed x blocks, if you receive another
reward to the same wallet before those blocks have been completed it
will be added to your total and all will be paid out after those x
blocks has passed.
For example if our batching interval is 2 blocks:
Block 1 - Address A receives reward of 10 oxen - added to batch
Block 2 - Address A receives reward of 10 oxen - added to batch
Block 3 - Address A is paid out 20 oxen.
Batching accumulates a small reward for all nodes every block
The batching of service node rewards allows us to drip feed rewards
to service nodes. Rather than accruing each service node 16.5 oxen every
time they are pulse block leader we now reward every node the 16.5 /
num_service_nodes every block and pay each wallet the full amount that
has been accrued after a period of time (Likely 3.5 days).
To spread each payment evenly we now pay the rewards based on the
address of the recipient. This modulus of their address determines which
block the address should be paid and by setting the interval to our
service_node_batching interval we can guarantee they will be paid out
regularly and evenly distribute the payments for all wallets over this
All the encoding parts move to oxen-encoding recently; this updates to
the latest version of oxen-mq, adds oxen-encoding, and converts
everything to use oxenc headers rather than the oxenmq compatibility
shims.
- make checkpoint and pulse participation distinct types
- make the container act a little more like an stl container
- replace check_participation() with a failures() method that just
returns the number of failures (so the caller can decide whether that
is bad or not).
- As a consequence of the above, failures now trigger on N+1 failures,
while previously they required N+1 failures *and* 8 total responses.
There is no need to wait for all 8 as far as I can see since we will
be failing no matter what the next 3 responses are.
- Removed the serialization code (hopefully this doesn't break
everything outside the RPC code).
- Simplify/DRY the code that records participation
Remove support for old (non-bt) proofs with the 9.2.0 snode revision
block (I'm not 100% sure on what to call this; "snode revision"? "soft
fork"? "spork"?).
Also bumps the working version to 9.2.0; this likely isn't release
ready, but allows for testing of this on testnet.
Snode revisions are a secondary version that let us put out a mandatory
update for snodes that isn't a hardfork (and so isn't mandatory for
wallets/exchanges/etc.).
The main point of this is to let us make a 9.2.0 release that includes
new mandatory minimums of future versions of storage server (2.2.0) and
lokinet (0.9.4) to bring upgrades to the network.
This slightly changes the HF7 blocks to 0 (instead of 1) because,
apparently, we weren't properly checking the HF value of the
pre-first-hf genesis block at all before. (In practice this changes
nothing because genesis blocks are v7 anyway).
This also changes (slightly) how we check for hard forks: now if we skip
some hard forks then we still want to know the height when a hard fork
triggers. For example, if the hf tables contains {7,14} then we still
need to know that the HF14 block height also is the height that
activates HF9, 10, etc.
It works just like storage server testing.
Renames the report_peer_storage_server_status to report_peer_status, and
repurposes the code to handle both SS and lokinet.
This *doesn't* need a HF by design because the reason bit field was
deliberately designed so that we can add reason fields (older clients
will just ignore unknown bits).
The timestamp inside the proof is only for signature validation, but we
were using it in some places as the uptime proof time, but not updating
it everywhere we needed to. This fixes it by using our own timestamp
for all local timed events (e.g. when we received it, when the node is
not sending proofs, etc.) to fix the issue.
This moves all the responsibility of ping testing (deciding when it's
unreachable, etc.) into oxend, allowing for better reporting on SS ping
results and eliminating some edge cases that can lead to oxend and
storage server getting "stuck" thinking each is in a different state.
Sean Darcy
Jason Rhinelander
Thomas Winget