This replaces the current epee logging system with our oxen::log
library. It replaces the easylogging library with spdlog, removes the
macros and replaces with functions and standardises how we call the
logs.
This updates the coinbase transactions to reward service nodes
periodically rather than every block. If you recieve a service node
reward this reward will be delayed x blocks, if you receive another
reward to the same wallet before those blocks have been completed it
will be added to your total and all will be paid out after those x
blocks has passed.
For example if our batching interval is 2 blocks:
Block 1 - Address A receives reward of 10 oxen - added to batch
Block 2 - Address A receives reward of 10 oxen - added to batch
Block 3 - Address A is paid out 20 oxen.
Batching accumulates a small reward for all nodes every block
The batching of service node rewards allows us to drip feed rewards
to service nodes. Rather than accruing each service node 16.5 oxen every
time they are pulse block leader we now reward every node the 16.5 /
num_service_nodes every block and pay each wallet the full amount that
has been accrued after a period of time (Likely 3.5 days).
To spread each payment evenly we now pay the rewards based on the
address of the recipient. This modulus of their address determines which
block the address should be paid and by setting the interval to our
service_node_batching interval we can guarantee they will be paid out
regularly and evenly distribute the payments for all wallets over this
The archaic (i.e. decade old) cmake usage here really got in the way of
trying to properly use newer libraries (like lokimq), so this undertakes
overhauling it considerably to make it much more sane (and significantly
reduce the size).
I left more of the architecture-specific bits in the top-level
CMakeLists.txt intact; most of the efforts here are about properly
loading dependencies, specifying dependencies and avoiding a whole pile
of cmake antipatterns.
This bumps the required cmake version to 3.5, which is what xenial comes
with.
- extensive use of interface libraries to include libraries,
definitions, and include paths
- use Boost::whatever instead of ${Boost_WHATEVER_LIBRARY}. The
interface targets are (again) much better as they also give you any
needed include or linking flags without needing to worry about them.
- don't list header files when building things. This has *never* been
correct cmake usage (cmake has always known how to wallet_rpc_headers
the headers that .cpp files include to know about build changes).
- remove the loki_add_library monstrosity; it breaks target names and
makes compiling less efficient because the author couldn't figure out
how to link things together.
- make loki_add_executable take the output filename, and set the output
path to bin/ and install to bin because *every single usage* of
loki_add_executable was immediately followed by setting the output
filename and setting the output path to bin/ and installing to bin.
- move a bunch of crap that is only used in one particular
src/whatever/CMakeLists.txt into that particular CMakeLists.txt instead
of the top level CMakeLists.txt (or src/CMakeLists.txt).
- Remove a bunch of redundant dependencies; most of them look like they
were just copy-and-pasted in, and many more aren't needed (since they
are implied by the PUBLIC linking of other dependencies).
- Removed `die` since it just does a FATAL_ERROR, but adds color (which
is useless since CMake already makes FATAL_ERRORs perfectly visible).
- Change the way LOKI_DAEMON_AND_WALLET_ONLY works to just change the
make targets to daemon and simplewallet rather than changing the build
process (this should make it faster, too, since there are various other
things that will be excluded).
This enables optional support for systemd notification which allows
lokid to be run via `Type=notify`, allowing it to better signal status
to systemd and enables systemd watchdog handling to restart if something
goes wrong.
Enabled here are:
- systemd watchdog ping every 10s
- systemd status update every 10s, so that `systemctl status loki-node`
gives you a status line such as:
Status: "Height: 450085, SN: active, proof: 15m12s, storage: 3m7s, lokinet: 27s"
- initialization notification so that systemd can wait for
and report on initialization status rather than just that the process
has launched.
- shutdown notification
All of these require changing the service type to `Type=notify` in the
`[Service]` section of the systemd service file; enabling the watchdog
also requires adding a `WatchdogSec=5min` line in the `[Service]`
section.
The systemd support is optional and requires the libsystemd-dev package
to be built (and is probably not feasible at all for a static build).
* Unify checkpointing and uptime quorums
* Begin making checkpoints cull old votes/checkpoints
* Begin rehaul of service node code out of core, to assist checkpoints
* Begin overhaul of votes to move resposibility into quorum_cop
* Update testing suite to work with the new system
* Remove vote culling from checkpoints and into voting_pool
* Fix bugs making integration deregistration fail
* Votes don't always specify an index in the validators
* Update tests for validator index member change
* Rename deregister to voting, fix subtle hashing bug
Update the deregister hash derivation to use uint32_t as originally set
not uint64_t otherwise this affects the result and produces different
results.
* Remove un-needed nettype from vote pool
* PR review, use <algorithms>
* Rename uptime_deregister/uptime quorums to just deregister quorums
* Remove unused add_deregister_vote, move side effect out of macro
* unit tests and core tests for new queueless swarm assignment logic
* new bufferless swarm assignment logic
* new lines
* Address reviews
* Improve node stealing logic
* unit tests for additional functions
* get_excess_pool to return early if the threshold is lower than MIN_SWARM_SIZE
* Address reviews
* another round of reviews
* Move new_swarm_vector inside loop
* Rebase changes: moved get_new_swarm_id to service_node_swarm.cpp
* reserve vector size for all_ids in get_new_swarm_id
* Enable staking in GUI wallet
* Move more helper functions into service_node_rules.h
* Move validation of arguments for staking into wallet2
* call stake validation from within create_stake_tx
* Retrieve quorum list from height, reviewed
* Setup data structures for de/register TX
* Submit and validate partial/full deregisters
* Add P2P relaying of partial deregistration votes
* Code review adjustments for deregistration part 1
- Fix check_tx_semantic
- Remove signature_pod as votes are now stored as blobs. Serialization
overrides don't intefere with crypto::signature anymore.
* deregistration_vote_pool - changed sign/verify interface and removed repeated code
* Misc review, fix sign/verify api, vote threshold
* Deregister/tx edge case handling for combinatoric votes
* core, service_node_list: separated address from service node pubkey
* Retrieve quorum list from height, reviewed
* Setup data structures for de/register TX
* Submit and validate partial/full deregisters
* Add P2P relaying of partial deregistration votes
* Code review adjustments for deregistration part 1
- Fix check_tx_semantic
- Remove signature_pod as votes are now stored as blobs. Serialization
overrides don't intefere with crypto::signature anymore.
* deregistration_vote_pool - changed sign/verify interface and removed repeated code
* Misc review, fix sign/verify api, vote threshold
* Deregister/tx edge case handling for combinatoric votes
* Store service node lists for the duration of deregister lifetimes
* Quorum min/max bug, sort node list, fix node to test list
* Change quorum to store acc pub address, fix oob bug
* Code review for expiring votes, acc keys to pub_key, improve err msgs
* Add early out for is_deregistration_tx and protect against quorum changes
* Remove debug code, fix segfault
* Remove irrelevant check for tx v3 in blockchain, fix >= height for pruning quorum states
Incorrect assumption that a transaction can be kept in the chain if it could
eventually become invalid, because if it were the chain would be split and
eventually these transaction would be dropped. But also that we should not
override the pre-existing logic which handles this case anyway.
The basic approach it to delegate all sensitive data (master key, secret
ephemeral key, key derivation, ....) and related operations to the device.
As device has low memory, it does not keep itself the values
(except for view/spend keys) but once computed there are encrypted (with AES
are equivalent) and return back to monero-wallet-cli. When they need to be
manipulated by the device, they are decrypted on receive.
Moreover, using the client for storing the value in encrypted form limits
the modification in the client code. Those values are transfered from one
C-structure to another one as previously.
The code modification has been done with the wishes to be open to any
other hardware wallet. To achieve that a C++ class hw::Device has been
introduced. Two initial implementations are provided: the "default", which
remaps all calls to initial Monero code, and the "Ledger", which delegates
all calls to Ledger device.
This replaces the epee and data_loggers logging systems with
a single one, and also adds filename:line and explicit severity
levels. Categories may be defined, and logging severity set
by category (or set of categories). epee style 0-4 log level
maps to a sensible severity configuration. Log files now also
rotate when reaching 100 MB.
To select which logs to output, use the MONERO_LOGS environment
variable, with a comma separated list of categories (globs are
supported), with their requested severity level after a colon.
If a log matches more than one such setting, the last one in
the configuration string applies. A few examples:
This one is (mostly) silent, only outputting fatal errors:
MONERO_LOGS=*:FATAL
This one is very verbose:
MONERO_LOGS=*:TRACE
This one is totally silent (logwise):
MONERO_LOGS=""
This one outputs all errors and warnings, except for the
"verify" category, which prints just fatal errors (the verify
category is used for logs about incoming transactions and
blocks, and it is expected that some/many will fail to verify,
hence we don't want the spam):
MONERO_LOGS=*:WARNING,verify:FATAL
Log levels are, in decreasing order of priority:
FATAL, ERROR, WARNING, INFO, DEBUG, TRACE
Subcategories may be added using prefixes and globs. This
example will output net.p2p logs at the TRACE level, but all
other net* logs only at INFO:
MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE
Logs which are intended for the user (which Monero was using
a lot through epee, but really isn't a nice way to go things)
should use the "global" category. There are a few helper macros
for using this category, eg: MGINFO("this shows up by default")
or MGINFO_RED("this is red"), to try to keep a similar look
and feel for now.
Existing epee log macros still exist, and map to the new log
levels, but since they're used as a "user facing" UI element
as much as a logging system, they often don't map well to log
severities (ie, a log level 0 log may be an error, or may be
something we want the user to see, such as an important info).
In those cases, I tried to use the new macros. In other cases,
I left the existing macros in. When modifying logs, it is
probably best to switch to the new macros with explicit levels.
The --log-level options and set_log commands now also accept
category settings, in addition to the epee style log levels.
