Converts all use of boost::filesystem to std::filesystem.
For macos and potentially other exotic systems where std::filesystem
isn't available, we use ghc::filesystem instead (which is a drop-in
replacement for std::filesystem, unlike boost::filesystem).
This also greatly changes how we handle filenames internally by holding
them in filesystem::path objects as soon as possible (using
fs::u8path()), rather than strings, which avoids a ton of issues around
unicode filenames. As a result this lets us drop the boost::locale
dependency on Windows along with a bunch of messy Windows ifdef code,
and avoids the need for doing gross boost locale codecvt calls.
Removes all "using namespace epee;" and "using namespace std;" from the
code and fixes up the various crappy places where unnamespaced types
were being used.
Also removes the ENDL macro (which was defined to be `std::endl`)
because it is retarded, and because even using std::endl instead of a
plain "\n" is usually a mistake (`<< std::endl` is equivalent to `<<
"\n" << std::flush`, and that explicit flush is rarely desirable).
The blockchain prunes seven eighths of prunable tx data.
This saves about two thirds of the blockchain size, while
keeping the node useful as a sync source for an eighth
of the blockchain.
No other data is currently pruned.
There are three ways to prune a blockchain:
- run monerod with --prune-blockchain
- run "prune_blockchain" in the monerod console
- run the monero-blockchain-prune utility
The first two will prune in place. Due to how LMDB works, this
will not reduce the blockchain size on disk. Instead, it will
mark parts of the file as free, so that future data will use
that free space, causing the file to not grow until free space
grows scarce.
The third way will create a second database, a pruned copy of
the original one. Since this is a new file, this one will be
smaller than the original one.
Once the database is pruned, it will stay pruned as it syncs.
That is, there is no need to use --prune-blockchain again, etc.
* Retrieve quorum list from height, reviewed
* Setup data structures for de/register TX
* Submit and validate partial/full deregisters
* Add P2P relaying of partial deregistration votes
* Code review adjustments for deregistration part 1
- Fix check_tx_semantic
- Remove signature_pod as votes are now stored as blobs. Serialization
overrides don't intefere with crypto::signature anymore.
* deregistration_vote_pool - changed sign/verify interface and removed repeated code
* Misc review, fix sign/verify api, vote threshold
* Deregister/tx edge case handling for combinatoric votes
* core, service_node_list: separated address from service node pubkey
* Retrieve quorum list from height, reviewed
* Setup data structures for de/register TX
* Submit and validate partial/full deregisters
* Add P2P relaying of partial deregistration votes
* Code review adjustments for deregistration part 1
- Fix check_tx_semantic
- Remove signature_pod as votes are now stored as blobs. Serialization
overrides don't intefere with crypto::signature anymore.
* deregistration_vote_pool - changed sign/verify interface and removed repeated code
* Misc review, fix sign/verify api, vote threshold
* Deregister/tx edge case handling for combinatoric votes
* Store service node lists for the duration of deregister lifetimes
* Quorum min/max bug, sort node list, fix node to test list
* Change quorum to store acc pub address, fix oob bug
* Code review for expiring votes, acc keys to pub_key, improve err msgs
* Add early out for is_deregistration_tx and protect against quorum changes
* Remove debug code, fix segfault
* Remove irrelevant check for tx v3 in blockchain, fix >= height for pruning quorum states
Incorrect assumption that a transaction can be kept in the chain if it could
eventually become invalid, because if it were the chain would be split and
eventually these transaction would be dropped. But also that we should not
override the pre-existing logic which handles this case anyway.
Hide DB types from db_types.h - no reason to recompile dependencies
when DB types change.
Also remove lingering in-memory DB references, they've been
obsolete since 9e82b694da
This replaces the epee and data_loggers logging systems with
a single one, and also adds filename:line and explicit severity
levels. Categories may be defined, and logging severity set
by category (or set of categories). epee style 0-4 log level
maps to a sensible severity configuration. Log files now also
rotate when reaching 100 MB.
To select which logs to output, use the MONERO_LOGS environment
variable, with a comma separated list of categories (globs are
supported), with their requested severity level after a colon.
If a log matches more than one such setting, the last one in
the configuration string applies. A few examples:
This one is (mostly) silent, only outputting fatal errors:
MONERO_LOGS=*:FATAL
This one is very verbose:
MONERO_LOGS=*:TRACE
This one is totally silent (logwise):
MONERO_LOGS=""
This one outputs all errors and warnings, except for the
"verify" category, which prints just fatal errors (the verify
category is used for logs about incoming transactions and
blocks, and it is expected that some/many will fail to verify,
hence we don't want the spam):
MONERO_LOGS=*:WARNING,verify:FATAL
Log levels are, in decreasing order of priority:
FATAL, ERROR, WARNING, INFO, DEBUG, TRACE
Subcategories may be added using prefixes and globs. This
example will output net.p2p logs at the TRACE level, but all
other net* logs only at INFO:
MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE
Logs which are intended for the user (which Monero was using
a lot through epee, but really isn't a nice way to go things)
should use the "global" category. There are a few helper macros
for using this category, eg: MGINFO("this shows up by default")
or MGINFO_RED("this is red"), to try to keep a similar look
and feel for now.
Existing epee log macros still exist, and map to the new log
levels, but since they're used as a "user facing" UI element
as much as a logging system, they often don't map well to log
severities (ie, a log level 0 log may be an error, or may be
something we want the user to see, such as an important info).
In those cases, I tried to use the new macros. In other cases,
I left the existing macros in. When modifying logs, it is
probably best to switch to the new macros with explicit levels.
The --log-level options and set_log commands now also accept
category settings, in addition to the epee style log levels.
TEST:
blockchain_export -h
This should show "berkeley" as an available option to --database.
With an existing BerkeleyDB database, run:
blockchain_export --database berkeley
There are various locale related bugs in various versions of boost,
where exceptions are thrown in boost::filesystem APIs when the
current locale is not to boost's liking. It's not clear what "not
to boost's liking" means in detail, though "en" and "en_US.UTF-8"
are not to its liking.
Fix it by running a test function that's known to throw in such
a case, and resetting LANG and LC_ALL to C if an exception is
thrown. In simplewallet, the locale is queried before that so the
correct translations will still be used.