The PN server potentially needs to subscribe to many addresses at once;
this allows it to do so in a single request rather than needing many
separate small requests.
This PR adds endpoints to storage server that allows someone to maintain
a oxenmq connection to one or more swarm members and receive pushed
notifications through that connection when new messages are delivered.
This is authenticated: subscribing requires a signature from the mailbox
owner signed within the past 14 days, and connections have to be
refreshed at least once/hour to keep the push notifications alive.
The immediate use for this will be for more efficient push notifications
for mobile clients using the push notification server, but this
mechanism will eventually also allow clients (over lokinet) to get
messages pushed to them rather than having to frequently poll.
This adds a new endpoint to the storage server to revoke a subkey. The
storage server will keep track of previously revoked subkeys and will
now check that a message authenticated by a subkey does not use one of
these revoked subkeys.
The nearest-swarm logic was rather broken when handling ids in the
wrapping space (i.e. > max swarm id or < min swarm id). This fixes the
issues, as well as optimizing it.
Issues:
- all the wrapped-around values trying to compute distances were off by
*2*:
- The first off-by-one was because, even though swarm ids cannot be ==
max-u64, swarm space values (from pubkeys) *can* be, so all the
logic trying to make it wrap at MAX_SWARM_ID was invalid.
- The wrapping at MAX_SWARM_ID was *also* wrong because, even ignoring
the above point, distances are % (MAX_SWARM_ID+1), not %
(MAX_SWARM_ID).
- Because of the above, a session ID that maps to 0xff..ff in swarm
space would compute the distance to swarm_id 0 as max-u64 and thus
always go to the top swarm_id, even when 0xff..fe would map to
swarm_id 0.
- Values in the space between 0 and the minimum swarm id were rounding
to the *right* side node, while everything else rounds to the left
edge.
All of this is due to bad logic and insufficient test cases that
unfortunately was not properly reviewed when originally added to storage
server.
This commit changes the behaviour. This means old and new nodes *will*
slightly disagree for a couple of edge cases:
- we *do* have a swarm 0 on mainnet, so a session id that maps to
exactly 0xff..ff will break while the network has mixed versions.
This, however, is extremely rare (~1 in 2^64), and so is unlikely to
matter in practice.
- Values in the [0, lowest_swarm_id) that are exactly at the midpoint
change to now prefer the left node (highest_swarm_id) rather than the
right (lowest_swarm_id). This makes it consistent with the other
cases. On mainnet, however, this case doesn't actually exist between
lowest_swarm_id equals 0.
This also changes the code to use a binary search, rather than linear
scan, because swarms are guaranteed to be sorted. This also
significantly simplifies the code. The linear scan was just
unoptimized.
Swarm replacement had a bug in updating IPs: it would try to *preserve*
the IP from the old data when loading new swarm data.
This was completely wrong: it meant we would refuse IP changes, which
would almost certainly lead to storage server connectivity problems when
an IP changes.
This fixes it to only preserve IP/port data *if* the new data has it as
0s (which could happen if, for example, oxend is resyncing and hasn't
received proofs yet, but we got an initial data from a bootstrap node).
The code path was also rather inefficient with multiple unnecessary
vector copies, and an oversived map allocation.
There was yet another oddity where we were returning a reference to a
dummy value, rather than returning a nullptr on failure; this changes it
to be a nullable pointer instead (which also cascaded into returning an
optional swarm where that pointer was being used).
Allows subkey authentication for the `expire` endpoint; if using a
subkey, expiries can be extended but not shortened (so that you can't
misuse expiry to delete messages prematurely).
The `load_fields` calls using a ton of vertical space because of the
long list of type names; this adds some short typedefs to significantly
shorten them and make the formatted code much smaller.
Adding `"required": true` to the delete_msgs request tells SS to return
a 404 if no swarm members found anything to delete, allowing chaining
into a sequence to only proceed if a message got deleted.
storage-server has a bug where the git tag is only regenerated when
cmake runs, but not when the git index changes.
Import the code to do this properly from lokinet, and rename it from
SHORT_HASH -> VERSIONTAG to be consistent.
- Allow an external OXENSS_SHORT_HASH to override the version (which
lets us drop a patch to do this from the debs)
- Remove the build time from the version.cpp because that is a bad idea
in general that makes builds non-reproducible.
This will allow the oxend to verify the pubkey when pings are received
and reject the ping if they are wrong.
The reason this causes issues is that storage server only receives the
snode keys on startup and if oxend is restarted without restarting
storage server then it could end up running with the wrong keys.
If oxend instead rejects the pings and stops sending uptime proofs then
the operator will know that something needs attention and can restart
the storage server.
This removes the restriction on updating message expiries that only
allowed shortening, but not extending, expiries.
With this change, a client can renew its own messages indefinitely as
long as it continually re-renews within 14 days of the last renewal.
