Improves the oxend<->storage server communications protocol:
- pass storage server HTTPS port as part of the storage server ping
(which already carries the also-required OMQ port) rather than needing
to provide it when starting up oxend. --storage-server-port is now
obsolete (and ignored, if specified).
- Fix up the internal API to use `storage_https_port` and
`storage_omq_port` rather than `storage_port` and `storage_lmq_port`.
- Redo and the SS ping RPC endpoint so that it is less verbose and more
closely matches the lokinet endpoint; instead of:
{ "version_major": 2, "version_minor": 0, "version_patch": 9, "storage_lmq_port": 22222 }
we now expect:
{ "version": [2,0,9], "https_port": 11111, "omq_port": 22222 }
- Tweaks the (not-yet-released) SS proof key names: "s"->"shp" and "slp"->"sop"
This makes uptime proof times network-dependent, and tweaks them a bit.
Also converts the times to type-safe std::chrono types rather than
macros.
Mainnet/testnet/devnet:
- Send the first proof 30s after startup rather than waiting 2 minutes.
- Check for whether we need to send a proof every 30s rather than every
5mins.
Mainnet:
Other times unchanged.
Testnet/devnet:
- Send proofs every 10min instead of 1h, and consider nodes to be down
after 21m instead of 2h5m.
Fakechain:
- Send 5s after startup, check every 5s, and send every 1min.
- Expiry after 2min5s
Also remove the cmake debug option for short proofs since the fakechain
changes above basically incorporate what it did.
random sampling of service nodes, call timestamp lmq message
checks timestamp of 5 service nodes, if local time is 30 seconds different from 80% of the nodes tested then warn user
tracks external timesync status and timestamp participation of service nodes
clean up includes
new template struct for participation history, individual types for participation entry
refactor checking participation
update select_randomly, move the testing for variance overflow
version locks, bump to 8.1.5
explicit casting for mac & clang
note to remove after hard fork
timestamp debugging log messages
debugging messages for before timesync - before message sent
logging errord with compiling
print version and change add_command to add_request_command
log if statement test
std::to_string replaced with tools::view_guts for x25519 key
check if my sn is active before sending timestamp requests
logging the failures
checking if statement for success of message
more logging, if guards arn't passing
more logging, successfully tests if service node might be out of sync
more tests before we decide we are out of sync
logging output if sn isn't passing tests
if check_participation fails then disconnect
print timestamp status
remove saving variance from the participation history
reduce MIN_TIME_IN_S_BEFORE_VOTING
reset participation history on recommission
undo reduction in startup time
reduce log levels
Set hardfork time in testnet
There is no reason at all to sign a *different* message in every stake
unlock; signatures already have their own nonce.
Having something that serves no purpose is worse than not having it
(because it leads to questions about why such a thing is there), so this
commit removes it by always using 0 as a nonce and comments about it.
Removing this from the broadcast tx would require a new tx extra field
so that isn't worth doing for now (but can be done in the future if we
change the tx extra structure for unlocks).
This also simplifies the nonce-to-hash code and fixes an endian bug in
it.
We use generate_ring_signature and check_ring_signature somewhat
inappropriately to sign and check a signature of a single key image.
While it works for that, the full ring signature algorithm adds quite a
bit of complexity that we don't need (and simply doesn't run) for the
key image proof included in stake transactions and exported key images
from the wallet.
This splits it up, makes the key image interface considerably simpler,
and adds annotation comments through it (and also adds comments into the
"main" signature code).
This is a necessary step to getting stake transactions and key image
exports working with Ledger, without implementing the full ring
signature (because that is quite involved, and not needed for most of
these cases).
Also remove unused gen/check_ring_signatures interfaces: The raw pointer
code is never called, except through the vector version and one place in
the test suite, so just remove it and make the vector version the main
implementation.
When targetting macos <10.14 macos won't allow use of anything from
C++17 that throws, such as:
- std::get on a variant
- std::visit
- std::optional::value()
- std::any_cast
This avoids all of these.
For std::get, we either replace with std::get_if (where appropriate), or
else use a `var::get` implementation of std::get added to lokimq (also
updated here). (This `var` namespace is just an `std` alias everywhere
*except* old target macos).
For std::visit, likewise lokimq adds an var::visit implementation for
old macos that we use.
std::optional::value() uses weren't useful anyway as everywhere it calls
them we've already checked that the option has a value, in which case we
can use `*opt` (which doesn't check for contents and throw).
std::any just has to be avoided as far as I can tell, but the one place
we used it is only ever a block, so I just replaced it with a `const
block*`.
- Currently nodes that have updated over time will have seeded their
sort key to a different value than everyone else, causing everone to
generate Pulse quorums that are inconsistent with each other.
Instead, set the sort key to 0. Ties in the sort list are currently
dealt with by sorting on the public key itself.
This changes the `get_quorum_state` RPC endpoint in two ways:
- If requesting a pulse quorum for a height that includes the current
height (i.e. top block + 1), this returns the current height, round-0
pulse quorum.
- When requesting the latest quorum state (i.e. the no-argument request)
you now get back the latest available quorum of each requested type,
instead of the quorum for the latest available block. Thus requesting
all types will now give you:
- the current top-of-the-chain round-0 pulse quorum
- the top block obligations quorum (no change)
- the top divisible-by-4 block for checkpoint quorums
- the top divisible-by-5 block for blink quorums
Previously you would just get whatever quorums existing for the top
height, which often meant just the one-old pulse quorum + top block
obligations quorum, only only checkpoint 25% of the time and blink 20%
of the time.
- Also updated the RPC comments for GET_QUORUM_STATE, both to reflect
the above and to update some parts which were a bit stale.
- We also remove the distinction between service_node_total and
service_node_paid. Previously the portions payout may still have some
remainder dust loki that was not distributed, and or distributed to the
miner as the difference between the pay out and the base reward was
added to the miner.
This is still the case for backwards compatibility before HF16. After
HF16, dust is not allowed and everything is split with the remainder
dust going to the 1st contributor in the Service Node (the operator).
- Avoid situations where rescanning/syncing the chain pre records the
node participation when it may not be relevant anymore (i.e. someone
restarting their node would re-record historical non-participation in
blocks and prime them for re-voting off a particular node).
- Instead of the Pulse quorums validators recording participation
between each other- so failures may not manifest in a decommission until
the several common nodes align and agree to vote off the node.
Voting now occurs when blocks arrives, validators participating in the
generation of the block are marked. This is shared information between
all nodes syncing the chain so decommissions are more readily agreeable
and acted upon in the Obligation quorums immediately.
The reward pre-hf16 is removed from the miner as they are the block
producer. After Pulse, the block producer changes to the Service Node.
Here we subtract the penalty from the Service Node Leader (the node at
the top of the Service Node List irrespective of round or even after PoW
the fallback).
https://github.com/loki-project/loki-core/issues/1267
The miner reward includes atomic dust resulting from truncation errors
of the other amounts, so this test for exact miner amounts failed (on
current mainnet blocks, particularly ones with lots of SN reward
recipients where such dust is likely).
- 0 amount outputs can still be generated if the portions was too low
for splitting the reward amount. An example where this can happen is on
alternative rounds, splitting the TX fee to the alternative quorum, if
the portions is too low we could potentially get a 0 output which would
be interpreted as RCT and cause some bugs on the chain.
The fix here is always check each output is > 0 before generating the
output in the miner tx.
- Also explicitly check the miner output amount when validating miner
transactions. Previously we only check the sum of the outputs match the
total expected amounts, and the service node outputs match our expected
rewards (and so i.e. no explicit checking on miner amount at any point).
- Avoid generating any outputs on an alternative round, for the
alternative block producer if the total tx fee was 0.
- Remove the 1 atomic loki awarded to the miner in HF16 (previously kept
to 1 for making sure tests work, whilst implementing the new block
reward split)
- Alternative pulse blocks must be verified against the quorum they belong to.
This updates alt_block_added hook in Service Node List to check the new Pulse
invariants and on passing allow the alt block to be stored into the DB until
enough blocks have been checkpointed.
- New reorganization behaviour for the Pulse hard fork. Currently reorganization
rules work by preferring chains with greater cumulative difficulty and or
a chain with more checkpoints. Pulse blocks introduces a 'fake' difficulty to
allow falling back to PoW and continuing the chain with reasonable difficulty.
If we fall into a position where we have an alt chain of mixed Pulse blocks
and PoW blocks, difficulty is no longer a valid metric to compare blocks (a
completely PoW chain could have much higher cumulative difficulty if hash
power is thrown at it vs Pulse chain with fixed difficulty).
So starting in HF16 we only reorganize when 2 consecutive checkpoints prevail
on one chain. This aligns with the idea of a PoS network that is
governed by the Service Nodes. The chain doesn't essentially recover until
Pulse is re-enabled and Service Nodes on that chain checkpoint the chain
again, causing the PoW chain to switch over.
- Generating Pulse Entropy no longer does a confusing +-1 to the height dance
and always begins from the top block. It now takes a block instead of a height
since the blocks may be on an alternative chain or the main chain. In the
former case, we have to query the alternative DB table to grab the blocks to
work.
- Removes the developer debug hashes in code for entropy.
- Adds core tests to check reorganization works
- Fixes a edge case that is prevalent in testing when adding nodes to
the network. Presuming an emty SN network, with Pulse once N nodes are
added to the network, the network switches from PoW blocks to Pulse
Blocks which require SN to SN communication.
SN's can't communicate with each other until their uptime proofs are
propagated with the IP bundled and currently LMQ only updates said
communication info when blocks are added, meaning Pulse can never
proceed.
So instead on receipt of an uptime proof we add the comm data to LMQ if
it was accepted and said Service Node is active.
Changes naming scheme
- Block Producer: The Service Node/Miner reponsible for generating the block
- Block Leader: The Service Node that is at the top of the Service Node
List queue. They are also by default on pulse round 0, the block producer.
If that round fails, then the block producer is changed as per the LIP-6
with multi-block seeding.
This commit also re-writes validate miner TX to repeat some of the
verification code. This appears to be clearer than having multiple
variables that change the verification loop in small ways which was more
error prone than I'd like to tolerate.
With Service Node List now generating 2 coinbase rewards, block_winner
is ambiguous, so, distinguish the reward generated for nodes at the top
of the queued list as- queued_winner.
- Remove PoW/Difficulty information for Pulse Block printout
- Use unary + to co-erce char's in ostreams
- Avoid epee::string_tools::pod_to_hex for lokimq equivalent
- Express validator bitmask in terms of PULSE_QUORUM_NUM_VALIDATORS
- Fix bug in generating Pulse Quorum, avoiding the use of the 0th
service node in pulse_candidates
- Switch to std::any for void * context
- By merging the quorum verification with pre-existing checkpointing
code, checkpoints votes are currently being sorted by the vote index
order. This was also enforced on the pulse signatures.
- Previously we generated quorums based off the previous quorum. To
incorporate the round, we can no longer pre-compute the quorum.
Instead it must be calculated when the block arrives, just before the
Service Node List is changed.
- Add tests to check round changing works
Once a block is added, the next leader is going to generate the Pulse
data based on the Service Node List at that point. (List won't change
until a block is added anyway).
This was failing before because of entropy being generated from the
wrong height not that the quorum was chosen incorrectly.
- Entropy is taken from the blockchain height, (i.e. if top block is block
100, then the blockchain height is 101). When the Core Tests is generating
a block in the tests, we haven't added the block to the DB yet, so when
db.height() is called in the tests, the height returned is 100.
Whereas when the Blockchain is receiving a block and on validation,
block 100, has been added to the DB already (added to DB, then we do
Service Node List validation). So the DB returns a blockchain height of
101. The 2 inconsistencies causes differences in the source entropy
for the blockchain.
- Begin moving code to allow creating blocks with custom pulse data in
tests by introducing loki_create_block_params. The workflow is to get
loki_chain_generator to return you the parameters to form the next
valid block. The user can modify the parameters to other valid
blocks/invalid blocks.
- Change the stable sort to a normal sort to avoid requiring the caller
side of the API needing to implicitly know that the list given must
be sorted lexicographically.
We don't want to choose from [0, (partition_index - running_it)) from
the active Service Node list. What we desire is a sliding window that
always considers half of the list.
Once we swap a Service Node into place, 1 service node from the second
half of the list will enter the first half. Otherwise, if we have
exactly PULSE_QUORUM_SIZE nodes to choose from, we would never be able
to fulfill a quorum as there are only half of the nodes to choose from,
(the other half would be in the 2nd half of the list and never
be considered) in the old scheme.
- Keep none for blocks
- Remove HF_VERSION_PULSE and prefer network_version_16 for big features
- Keep HF_VERSION_... for smaller feature flags
- Add operator< for pulse_sort_key
- Avoid erase() in validators gen loop for pulse
Add stable sort to handle ties via public keys. The quorum_index reduces
bias in Service Nodes choosing a lexigraphical closer public key. On
launch all Service Nodes will have this set to 0, but eventually it
will hold a unique value per Service Node.
With a null winner block the `if ((miner_tx.vout.size() - 1) <
winner.payouts.size())` fires, when it shouldn't. Bypass checks (after
verifying that a null winner is correct) for such a block.
Jason Rhinelander
Sean Darcy
Sean Darcy
Doyle
Doyle
moneromooo-monero