Various code improvements to LMDB code (no logic changes):
- compile less code by moving the cursor macros (which are used often)
into local functions
- removes the TXN_POSTFIX_RDONLY macro because it does exactly nothing
- remove useless `inline` from various functions in C++ only has one
purpose: to tell the compiler that it's okay to have multiple
definitions (and just throw the extra away when linking). It is not
used by any modern C++ compiler to actually decide when to inline, and
so has no use in a .cpp file. (Suspect this was written by a C
programmer who doesn't fully understand C++).
- Remove the C `typedef struct blah { ... } blah;` idiom which has no
purpose in C++.
- Use inheritance on the nearly-duplicate `mdb_block_info_{1,2,3}
structs rather than repeating all the field in each one. This also
removed a completely unneeded offsetof into them that was being used
to avoid having to learn C++. (Also adds a static_assert in case
anyone ever changes some contained type and breaks things).
- Fix bi_weight comment about why it is really uin64_t
- Add a missing method `override`.
- Replace "22" buried in the code with a constant indicating what it is
for. (Oh, reader, you wanted to know what it's used for? It's the
number of different databases allowed in the LMDB)
- Move `m_cur_whatever` macros into the .cpp file, and avoid using them
in any code we've added in loki. These are incredibly useless
macros: they are basically just defined to slightly shorten a name.
I'm guessing that, at some point, these members got moved and rather
than fix the remaining code someone defined a bunch of macros to deal
with the rename. Yuck.
(Getting rid of *all* of them (i.e. replacing all `m_cur_whatever`
with `m_cursors->whatever` would be preferable, but would make future
merge conflicts more painful, so I left the macros as is for the
upstream databases but removed our service_node_data one.)
- Rename `mdb_txn_cursors` members to drop the leading `m_txc_` prefix.
When the struct is nothing more than a POD collection of values the
prefix makes no sense at all. This allows us the write
`m_cursors->service_node_data` instead of
`m_cursors->m_txc_service_node_data` (or the horrible macro, above,
which seems designed to just macro around this naming horror).
This allows fetching multiple tx heights in a single tx rather than
needing to call multiple times and incur the overhead of a transaction.
This devirtualizes the singular version and reimplements it as a simple
wrapper around the vector version.
These structures will always end up packed anyway (everything in them is
8-byte aligned and a multiple of 8 bytes), but the pack pragma also
changes their alignment which isn't good (and causes warnings). This
just static_asserts that they are padding-free instead, which is the
intention.
Silences:
src/blockchain_db/lmdb/db_lmdb.cpp: In member function ‘virtual void cryptonote::BlockchainLMDB::open(const string&, cryptonote::network_type, int)’:
src/blockchain_db/lmdb/db_lmdb.cpp:1473:63: error: type qualifiers ignored on cast result type [-Werror=ignored-qualifiers]
1473 | if (db_version > static_cast<decltype(db_version)>(VERSION))
| ^
src/blockchain_db/lmdb/db_lmdb.cpp:1480:68: error: type qualifiers ignored on cast result type [-Werror=ignored-qualifiers]
1480 | else if (db_version < static_cast<decltype(db_version)>(VERSION))
| ^
which happens because the `decltype()` here is const, and gcc apparently
now apparently warns on a cast to a const cast result.
While here I also noticed and fixed what looks like an unaligned access
that could be a potential problem on non-x86 architectures.
Add migratory code for new alt_block_data_t.checkpointed field
Code review
Redo switch to alt chain logic to handle PoW properly
We always switch to the chain with prevailing checkpoints. If an old
chain reappears with more checkpoints it will be rebroadcasted and at
that point we will switch over then. So here we restore the old
behaviour regarding keeping alt chains around or not depending on the
scenario that caused the chain switch
* Make immutable checkpoint function return the checkpoint
* Update test db immutable checkpoints prototype
* Handle the empty checkpoints_range case, remove unused variable
* Code review: simplify immutable detection logic
* Switch back to holding checkpoint ptrs to avoid copies
Increase the range to mitigate frequent rescanning when reorganising,
technically with checkpointing we only need up to the 2nd oldest
checkpoint, but I'm sure there's some case we're forgetting that would
benefit from a larger rollback range.
Also considering there were only 160 events in the month with a mass
deregistration- keeping 2 days worth of rollbacks around is not going to
be too expensive.
Start storing service node list state for past blocks
Separate quorums and reduce storage to 1 days worth
Don't use pointer, declare m_state by value
Fix rebase to dev
Fix double serialise of service node state and expensive blob copy
std::move quorums into storage, reserve vector size
Don't +1 checkpoint finality height
Code review
Code review 2
* Recalculate difficulty on reorganisation workaround
* Gate difficulty workaround to only be on mainnet
* Make difficulty changes emit logs on diff delta only
* Improved print_checkpoints
* Flesh out print checkpoints and associated RPC call
* Remove debug print checkpoints
* Update help text for print_checkpoints
* Rewrite to fix num_checkpoints != heights as a unit of measurement
* Use GET_ALL_CHECKPOINTS defined value in get_checkpoints_range
* Let T be deduced in parse_if_present, json_rpc_request not rpc_request
We're going to be coming down off a ~300MH/s high, so add a difficulty
cap of 30MH/s for the first 60 blocks so that we don't end with
extremely long blocks.
Fixes some potential architecture issues with the checkpoint lmdb:
- size_t is not a fixed size type (so 32-bit architectures and 64-bit
architectures would store a different amount of data here)
- add endian conversions to the integer fields before/after loading
(these are no-ops on little-endian architectures, i.e. in the vast
majority of the time).
This converts the transaction type and version to scoped enum, giving
type safety and making the tx type assignment less error prone because
there is no implicit conversion or comparison with raw integers that has
to be worried about.
This ends up converting any use of `cryptonote::transaction::type_xyz`
to `cryptonote::transaction::txtype::xyz`. For version, names like
`transaction::version_v4` become `cryptonote::txversion::v4_tx_types`.
This also allows/includes various other simplifications related to or
enabled by this change:
- handle `is_deregister` dynamically in serialization code (setting
`type::standard` or `type::deregister` rather than using a
version-determined union)
- `get_type()` is no longer needed with the above change: it is now
much simpler to directly access `type` which will always have the
correct value (even for v2 or v3 transaction types). And though there
was an assertion on the enum value, `get_type()` was being used only
sporadically: many places accessed `.type` directly.
- the old unscoped enum didn't have a type but was assumed castable
to/from `uint16_t`, which technically meant there was potential
undefined behaviour when deserializing any type values >= 8.
- tx type range checks weren't being done in all serialization paths;
they are now. Because `get_type()` was not used everywhere (lots of
places simply accessed `.type` directory) these might not have been
caught.
- `set_type()` is not needed; it was only being used in a single place
(wallet2.cpp) and only for v4 txes, so the version protection code was
never doing anything.
- added a std::ostream << operator for the enum types so that they can be
output with `<< tx_type <<` rather than needing to wrap it in
`type_to_string(tx_type)` everywhere. For the versions, you get the
annotated version string (e.g. 4_tx_types) rather than just the number
4.
* Initial updates to allow syncing of checkpoints in protocol_handler
* Handle checkpoints in prepare_handle_incoming_blocks
* Debug changes for testing, cancel changes later
* Add checkpoint pruning code
* Reduce DB checkpoint accesses, sync votes for up to 60 blocks
* Remove duplicate lifetime variable
* Move parsing checkpoints to above early return for consistency
* Various integration test fixes
- Don't try participate in quorums at startup if you are not a service node
- Add lock for when potentially writing to the DB
- Pre-existing batch can be open whilst updating checkpoint so can't use
DB guards
- Temporarily emit the ban message using msgwriter instead of logs
* Integration mode bug fixes
- Always deserialize empty quorums so get_testing_quorum only fails when
an actual critical error has occurred
- Cache the last culled height so we don't needlessly query the DB over
the same heights trying to delete already-deleted checkpoints
- Submit votes when new blocks arrive for more reliable vote
transportation
* Undo debug changes for testing
* Update incorrect DB message and stale comment
* Unify checkpointing and uptime quorums
* Begin making checkpoints cull old votes/checkpoints
* Begin rehaul of service node code out of core, to assist checkpoints
* Begin overhaul of votes to move resposibility into quorum_cop
* Update testing suite to work with the new system
* Remove vote culling from checkpoints and into voting_pool
* Fix bugs making integration deregistration fail
* Votes don't always specify an index in the validators
* Update tests for validator index member change
* Rename deregister to voting, fix subtle hashing bug
Update the deregister hash derivation to use uint32_t as originally set
not uint64_t otherwise this affects the result and produces different
results.
* Remove un-needed nettype from vote pool
* PR review, use <algorithms>
* Rename uptime_deregister/uptime quorums to just deregister quorums
* Remove unused add_deregister_vote, move side effect out of macro
* Beging adding functions to recalculate difficulty
* Add command line args to utility for recalculating difficulty
* Exception safety for recalculating difficulty
* Update help text for recalc difficulty
* Add recalc flag on the daemon
* Make context be const, signify intent for var++ to 1
* Add functions for storing checkpoints to the DB
* Allocate the DB entry on the stack instead of heap
* Add virtual overrides for new checkpoint functions
* Clean up for pull request, simplify some logic
* Revise API to include height in checkpoint header
* Move log to top of function even if early exit
* Begin moving checkpoints to db
* Allow storing of checkpoints to DB
* Cleanup for code reviewer, fix unit tests
* Fix tests, fix casting issues
* Don't use DUPSORT, use height->checkpoint mapping in DB
* Remove if 0 disabling checkpoint vote, we already check HF12
* Fix unit test infinite loop
* Update db schemas to match blk_checkpoint_header
* Code review
The db txn in add_block ending caused the entire overarching
batch txn to stop.
Also add a new guard class so a db txn can be stopped in the
face of exceptions.
Also use a read only db txn in init when the db itself is
read only, and do not save the max tx size in that case.
