- Remove implicit `operator bool` from ec_point/public_key/etc. which
was causing all sorts of implicit conversion mess and bugs.
- Change ec_point/public_key/etc. to use a `std::array<unsigned char,
32>` (via a base type) rather than a C-array of char that has to be
reinterpret_cast<>'ed all over the place.
- Add methods to ec_point/public_key/etc. that make it work more like a
container of bytes (`.data()`, `.size()`, `operator[]`, `begin()`,
`end()`).
- Make a generic `crypto::null<T>` that is a constexpr all-0 `T`, rather
than the mishmash `crypto::null_hash`, crypto::null_pkey,
crypto:#️⃣:null(), and so on.
- Replace three metric tons of `crypto::hash blahblah =
crypto::null_hash;` with the much simpler `crypto::hash blahblah{};`,
because there's no need to make a copy of a null hash in all these
cases. (Likewise for a few other null_whatevers).
- Remove a whole bunch of `if (blahblah == crypto::null_hash)` and `if
(blahblah != crypto::null_hash)` with the more concise `if
(!blahblah)` and `if (blahblah)` (which are fine via the newly
*explicit* bool conversion operators).
- `crypto::signature` becomes a 64-byte container (as above) but with
`c()` and `r()` to get the c() and r() data pointers. (Previously
`.c` and `.r` were `ec_scalar`s).
- Delete with great prejudice CRYPTO_MAKE_COMPARABLE and
CRYPTO_MAKE_HASHABLE and all the other utter trash in
`crypto/generic-ops.h`.
- De-inline functions in very common crypto/*.h files so that they don't
have to get compiled 300 times.
- Remove the disgusting include-a-C-header-inside-a-C++-namespace
garbage from some crypto headers trying to be both a C and *different*
C++ header at once.
- Remove the toxic, disgusting, shameful `operator&` on ec_scalar, etc.
that replace `&x` with `reinterpret_cast x into an unsigned char*`.
This was pure toxic waste.
- changed some `<<` outputs to fmt
- Random other small changes encountered while fixing everything that
cascaded out of the above changes.
oxen::log::info(...), etc. are a bit too verbose; this simplifies them
to just `log::info(...)`, etc. by aliasing the `oxen::log` namespace
into most of the common namespaces we use in core.
This result is usage that is shorter but also reads better:
oxen::log::info(logcat, "blah: {}", 42);
log::info(logcat, "blah: {}", 42);
This replaces the current epee logging system with our oxen::log
library. It replaces the easylogging library with spdlog, removes the
macros and replaces with functions and standardises how we call the
logs.
Converting between internal addresses and string wallet addresses is
surprisingly expensive (partly because a keccak hash is involved);
normally on the blockchain that doesn't matter because string
conversions are rare, but batching exposed some code paths that do the
conversion a lot.
This makes two optimizations to considerably speed up batching
processing time:
1. Add an address string hash for batch addresses. We don't see that
many, but we tend to see the same addresses for *every* block. By
caching it we can considerably reduce the time needed to do the
actual conversions.
2. In many places we don't really need strings at all: we can work just
with the address_infos (which are cheap). This eliminates some of
the places we use addresses and pushes address_infos further through
the code.
In total, this plus the previous optimization drop batch processing time
by this cuts batch processing CPU time by about 85%, roughly half from
this optimization and roughly half from the store-offsets optimization.
On a debug build, timing mainnet resyncing from just before HF19 to
current:
- neither optimization: 223.60s spent processing batch jobs (of 282s
total recalculating subsystems).
- just offsets: 131s spent in batch processing
- both optimizations: 33s spent in batch processing (down to 60s total
recalculating).
On a release build:
- before: 107.35s (4.73s snl; 95.55s batch)
- after: 28.27s (4.66s snl; 0.00s ons; 21.72s batch)
We don't do anything differently for a nullopt versus an empty vector,
so just return an empty vector for both not-at-the-hf and no-payments
cases.
(This also fixes a segfault and/or chainsync bug in the previous commit
because the optional is dereferenced without checking when on hf19).
- Replace all cryptonote_config macros with constexpr variables. Some
become integer types, some become chrono types.
- generally this involved removing a "CRYPTONOTE_" prefix since the
values are now in the `cryptonote` namespace
- some constants are grouped into sub-namespaces (e.g.
cryptonote::p2p)
- deprecated constants (i.e. for old HFs) are in the `cryptonote::old`
namespace.
- all the magic hash key domain separating strings are now in
cryptonote::hashkey::WHATEVER.
- Move some economy-related constants to oxen_economy.h instead
- Replaced the BLOCKS_EXPECTED_IN_DAYS constexpr functions with more
straightforward `BLOCKS_PER_DAY` value (i.e. old
`BLOCKS_EXPECTED_IN_DAYS(10)` is now `BLOCKS_PER_DAY * 10`.
- Replaced `network_version` unscoped enum with a scoped enum
`cryptonote::hf`, replacing all the raw uint8_t values where it was
currently accepted with the new `hf` type.
- Made `network_type` a scoped enum so that it now has to be qualified
(network_type::TESTNET) and can't be arbitrarily/unintentionally
converted to/from an int.
- HARDFORK_WHATEVER macros have become cryptonote::feature::WHATEVER
constexpr hf values.
- Add `revision` to rpc hard_fork_info response
- Don't build trezor code at all (previously we were pointlessly
building an empty dummy lib).
This requires the operator to still contribute 25% of the service node
but another 9 nodes will be allowed to stake to the node makeing 10
contributors total rather than our previous 4.
This updates the coinbase transactions to reward service nodes
periodically rather than every block. If you recieve a service node
reward this reward will be delayed x blocks, if you receive another
reward to the same wallet before those blocks have been completed it
will be added to your total and all will be paid out after those x
blocks has passed.
For example if our batching interval is 2 blocks:
Block 1 - Address A receives reward of 10 oxen - added to batch
Block 2 - Address A receives reward of 10 oxen - added to batch
Block 3 - Address A is paid out 20 oxen.
Batching accumulates a small reward for all nodes every block
The batching of service node rewards allows us to drip feed rewards
to service nodes. Rather than accruing each service node 16.5 oxen every
time they are pulse block leader we now reward every node the 16.5 /
num_service_nodes every block and pay each wallet the full amount that
has been accrued after a period of time (Likely 3.5 days).
To spread each payment evenly we now pay the rewards based on the
address of the recipient. This modulus of their address determines which
block the address should be paid and by setting the interval to our
service_node_batching interval we can guarantee they will be paid out
regularly and evenly distribute the payments for all wallets over this
All the encoding parts move to oxen-encoding recently; this updates to
the latest version of oxen-mq, adds oxen-encoding, and converts
everything to use oxenc headers rather than the oxenmq compatibility
shims.
This code is bitrotting, doesn't compile, and isn't being maintained
anymore.
The integration test suite was an interesting idea, in early Loki days,
but is no longer being maintained and is quite cumbersome to run (for
instance, it is not possible to run it via CI because it depends on
xterm to actually run). The code to actually run it (in doy-lee's
loki-integration-testing repository) is also a large burden of "janky"
code that isn't worth maintaining.
Remove this from the code; if someone wants to pick it back up in the
future reverting this commit shouldn't be too difficult (though I'd
suggest that a much better approach to integration testing would be to
run different daemons/wallets via rpc commands, as the network-tests do,
rather than trying to feed stdin and parse stdout from running
individual oxends/wallets).
Governance reward calculations were hard-coded for == HF17 rather than
>= 17, so for HF18 it was falling back to the old "add up all the
values" method that we used to use. Updated it to support HF18, and add
a static_assert that will fail to compile (without a fix) when we add
HF19.
Also some minor cleanups (mostly indent changes for unnecessary blocks
-- ignore whitespace when looking at the diff).
Network validation expects N outputs when there are N contributors, but
if any of the received contribution amounts was 0 we were skipping it,
leading to a block that failed validation.
This happened at blocks 739994 and 740010 when recently registered SNs
*with* a contributor reached the top of the reward list with a 100% fee;
this caused the second SN reward recipient amount to be 0, which then
got left off the block and then failed block validation.
This doesn't require a hard fork to fix -- it just requires block
producers to update to start including the 0 payout in such cases which
makes the network happy with the block.
(This may result in some stalls when we hit those SNs again, but as long
as enough of the network has upgraded we should unstall when an upgraded
node gets randomly selected to produce a backup block).
We can't call cryptonote::add_tx_secret_key_to_tx_extra from `device`
code because that isn't necessarily available in `device` (though for
some odd reason this only actually showed up on the i386 build).
This amends the call to just get the secret key, leaving the actual job
of adding it to tx.extra to the caller (which is a cleaner way to do it
anyway).
Core tests were breaking because of the removal of pre-CLSAG
transaction generation support. This fixes it by allowing and using
CLSAG transactions before HF16 (which is safe to do now that we are well
past the hard fork).
We add the tx secret key to the tx_extra in staking transactions so that
values can be decoded, but the tx secret key value that we have on hand
is encrypted and so we can't access it.
This moves the call that adds the secret key into the device code so
that devices can provide this. It also adds the tx version/type earlier
in the process (into `open_tx`) so that the device can know early on
that this is a stake transaction and therefore that leaking the tx
secret key is okay (and can also apply other stake-specific behaviour).
We use generate_ring_signature and check_ring_signature somewhat
inappropriately to sign and check a signature of a single key image.
While it works for that, the full ring signature algorithm adds quite a
bit of complexity that we don't need (and simply doesn't run) for the
key image proof included in stake transactions and exported key images
from the wallet.
This splits it up, makes the key image interface considerably simpler,
and adds annotation comments through it (and also adds comments into the
"main" signature code).
This is a necessary step to getting stake transactions and key image
exports working with Ledger, without implementing the full ring
signature (because that is quite involved, and not needed for most of
these cases).
Also remove unused gen/check_ring_signatures interfaces: The raw pointer
code is never called, except through the vector version and one place in
the test suite, so just remove it and make the vector version the main
implementation.
Converts all use of boost::filesystem to std::filesystem.
For macos and potentially other exotic systems where std::filesystem
isn't available, we use ghc::filesystem instead (which is a drop-in
replacement for std::filesystem, unlike boost::filesystem).
This also greatly changes how we handle filenames internally by holding
them in filesystem::path objects as soon as possible (using
fs::u8path()), rather than strings, which avoids a ton of issues around
unicode filenames. As a result this lets us drop the boost::locale
dependency on Windows along with a bunch of messy Windows ifdef code,
and avoids the need for doing gross boost locale codecvt calls.
When targetting macos <10.14 macos won't allow use of anything from
C++17 that throws, such as:
- std::get on a variant
- std::visit
- std::optional::value()
- std::any_cast
This avoids all of these.
For std::get, we either replace with std::get_if (where appropriate), or
else use a `var::get` implementation of std::get added to lokimq (also
updated here). (This `var` namespace is just an `std` alias everywhere
*except* old target macos).
For std::visit, likewise lokimq adds an var::visit implementation for
old macos that we use.
std::optional::value() uses weren't useful anyway as everywhere it calls
them we've already checked that the option has a value, in which case we
can use `*opt` (which doesn't check for contents and throw).
std::any just has to be avoided as far as I can tell, but the one place
we used it is only ever a block, so I just replaced it with a `const
block*`.
- On some of the SN's who are queued for payout, requesting a block
template with it fails because we end up paying less than the total
amount allocated.
This was previously solved by having a `service_node_paid` in
`reward_parts` that I removed to simplify `reward_parts`.
This is only a client side sanity check and does not affect consensus.
The rest of the codebase uses the original calculate_sums_of_portions on
the total allocated reward (which when it does the division out, the
remainder is not included in the payouts and the verifying code expects
that, so everything should continue to work fine).
- We also remove the distinction between service_node_total and
service_node_paid. Previously the portions payout may still have some
remainder dust loki that was not distributed, and or distributed to the
miner as the difference between the pay out and the base reward was
added to the miner.
This is still the case for backwards compatibility before HF16. After
HF16, dust is not allowed and everything is split with the remainder
dust going to the 1st contributor in the Service Node (the operator).
- The wrong assumption was made when get_loki_block_reward is called.
1. It is called when figuring out what the best list of TX's to include
to get maximal fee. In this case, we don't want to fail out when the
penalty exceeds the block because we use this as a marker as to not
include the next TX in the block to maximise on fee.
2. It is also called in retrospect when syncing the chain to check that
the block weight + the fees in the block match what we expect. In this
case it'd be sensible to fail out, but, we don't necessarily have to.
The rewards in the block vs the rewards we calculated have to match
anyway.
All block producers (Miner or Service Node) share in common the tx
fees as part of the reward they receive. The block penalty is applied on
this amount.
The reward pre-hf16 is removed from the miner as they are the block
producer. After Pulse, the block producer changes to the Service Node.
Here we subtract the penalty from the Service Node Leader (the node at
the top of the Service Node List irrespective of round or even after PoW
the fallback).
https://github.com/loki-project/loki-core/issues/1267
- 0 amount outputs can still be generated if the portions was too low
for splitting the reward amount. An example where this can happen is on
alternative rounds, splitting the TX fee to the alternative quorum, if
the portions is too low we could potentially get a 0 output which would
be interpreted as RCT and cause some bugs on the chain.
The fix here is always check each output is > 0 before generating the
output in the miner tx.
- Also explicitly check the miner output amount when validating miner
transactions. Previously we only check the sum of the outputs match the
total expected amounts, and the service node outputs match our expected
rewards (and so i.e. no explicit checking on miner amount at any point).
- Avoid generating any outputs on an alternative round, for the
alternative block producer if the total tx fee was 0.
- Remove the 1 atomic loki awarded to the miner in HF16 (previously kept
to 1 for making sure tests work, whilst implementing the new block
reward split)
