This code is bitrotting, doesn't compile, and isn't being maintained
anymore.
The integration test suite was an interesting idea, in early Loki days,
but is no longer being maintained and is quite cumbersome to run (for
instance, it is not possible to run it via CI because it depends on
xterm to actually run). The code to actually run it (in doy-lee's
loki-integration-testing repository) is also a large burden of "janky"
code that isn't worth maintaining.
Remove this from the code; if someone wants to pick it back up in the
future reverting this commit shouldn't be too difficult (though I'd
suggest that a much better approach to integration testing would be to
run different daemons/wallets via rpc commands, as the network-tests do,
rather than trying to feed stdin and parse stdout from running
individual oxends/wallets).
The displaying of ONS transactions in show_transfers and
export_transfers previously showed the dummy destination address which
contained zero bytes and converted to a nonsense address. Created a new
ONS type for the display functions and removed the displaying of the destination
address.
In addition refactored how we determine that a transaction was either a
staking, ons or output transaction as we were previously parsing the
tx_extra where the data was already in the
cryptonote::transaction::type.
Finally renamed the wording for staking rewards. Previously "miner" now
"reward"
The wallet can occassionally become "out of sync" when a new block is
added so it becomes one block behind the chain height. This results in
errors in the wallet which are unnecessary because the user immediately
calls the function after it fails and it will succeed.
There is no reason at all to sign a *different* message in every stake
unlock; signatures already have their own nonce.
Having something that serves no purpose is worse than not having it
(because it leads to questions about why such a thing is there), so this
commit removes it by always using 0 as a nonce and comments about it.
Removing this from the broadcast tx would require a new tx extra field
so that isn't worth doing for now (but can be done in the future if we
change the tx extra structure for unlocks).
This also simplifies the nonce-to-hash code and fixes an endian bug in
it.
- Adds "create_wallet" rpc arguments to allow creating a hardware wallet
- Adds a "create-hwdev" option to both rpc and cli wallets that creates
a "[wallet].hwdev.txt" file containing an optional comment to flag the
wallet as a hardware wallet. Although this data is present in the
wallet file itself, that isn't available without a password description
and the the GUI wallet needs to know this *before* it opens the wallet
to properly display a wallet list with some indicators for hardware
wallets. The comment also lets the user record some optional comment
such as "Jason's Ledger Nano S" to serve as a reminder in the GUI wallet
list.
- Fixes some unicode EN DASHes that got added to documentation comments
in place of the intended hyphens.
Tx prefix communication was missing some needed information on the tx
type, and was a little inefficient. This redoes the protocol to send
the tx type info and then the entire prefix (rather than starting from a
few bytes in). It also changes how we number requests and signal the
final piece of a multi-piece transmission.
The cli wallet was correctly refusing to sign, but the RPC wallet (and
wallet2 internally) allowed it and would sign with the all-0s address,
which of course is invalid and fails to verify. This now throws if
attempting to do so to get an error returned to the rpc endpoint.
Other minor changes here:
- make wallet2::verify static (verification doesn't depend on the wallet
instance at all).
- Add documentation for wallet2 sign/verify
- Slight DRY of wallet2::sign
Converts all use of boost::filesystem to std::filesystem.
For macos and potentially other exotic systems where std::filesystem
isn't available, we use ghc::filesystem instead (which is a drop-in
replacement for std::filesystem, unlike boost::filesystem).
This also greatly changes how we handle filenames internally by holding
them in filesystem::path objects as soon as possible (using
fs::u8path()), rather than strings, which avoids a ton of issues around
unicode filenames. As a result this lets us drop the boost::locale
dependency on Windows along with a bunch of messy Windows ifdef code,
and avoids the need for doing gross boost locale codecvt calls.
When targetting macos <10.14 macos won't allow use of anything from
C++17 that throws, such as:
- std::get on a variant
- std::visit
- std::optional::value()
- std::any_cast
This avoids all of these.
For std::get, we either replace with std::get_if (where appropriate), or
else use a `var::get` implementation of std::get added to lokimq (also
updated here). (This `var` namespace is just an `std` alias everywhere
*except* old target macos).
For std::visit, likewise lokimq adds an var::visit implementation for
old macos that we use.
std::optional::value() uses weren't useful anyway as everywhere it calls
them we've already checked that the option has a value, in which case we
can use `*opt` (which doesn't check for contents and throw).
std::any just has to be avoided as far as I can tell, but the one place
we used it is only ever a block, so I just replaced it with a `const
block*`.
Our cached value could well be wrong, if another owner (or another copy
of the wallet) has updated it, and so the information we have in the
cache other than the name and type (which don't change) can be
unreliable.
Drop the cached value/owner fields and instead use queried values for
encrypted/owner/backup, and decrypt value on the fly when needed.
- Add a wallet function to convert the type string into a type value
- Remove the lns buy/update/renew overloads that take a string; callers
should use the above instead to convert (and check) the type.
- Fix rpc wallet cache entries to set the proper type in the lns cache
instead of always inserting session type.
The lns records are stored on the blockchain in a hashed state so the
name and value are not immediately accessable. This creates a store in
the wallet so the unhashed details are saved and can be viewed at a
later date.
Revamps how .loki LNS registrations work:
- Enable lokinet registrations beginning at HF16.
- rework renewal so that you can renew at any time and it simply adds to the end of the current
expiry. Previously there was only a window in which you could renew.
- Renewals are a new type of LNS transaction, distinct from buys and updates. (Internally it is an
update with no fields, which cannot be produced in the existing code).
- Add optional "type=" parameter to lns commands. Commands default to trying to auto-detect (i.e.
if the name ends with .loki it is lokinet), but the type allows you to be explicit *and* allows
select non-default registration lengths for lokinet buys/renewals.
- change .loki naming requirements: we now require <= 32 chars if it doesn't contain a -, and 63 if
it does. We also reserve names starting "??--" for any ?? other than xn-- (punycode), as this is
a DNS restriction. "loki.loki" and "snode.loki" are also now reserved (just in case someone
sticks .loki as a DNS search domain).
- Tweak LNS registration times to consider "a year" to be 368 days worth of blocks (to allow for
leap years and some minor block time drift).
- Overhaul how LNS registrations are displayed in the cli wallet. For example:
[wallet L6QPcD]: lns_print_name_to_owners jasonv.loki jason.loki jasonz.loki
Error: jasonv.loki not found
Name: jason.loki
Type: lokinet
Value: azfoj73snr9f3neh5c6sf7rtbaeabyxhr1m4un5aydsmsrxo964o.loki
Owner: L6QPcDVp6Fu7HwtXrXjtfvWvgBPvvMQ9FiyquMWn2BBEDsk2vydwu1A3BrK2uQcCo94G7HA5xiKvpZ4CMQva6pxW2GXkCG9
Last updated height: 46
Expiration height: 75
Encrypted value: 870e42cd172a(snip)
Error: jasonz.loki not found
- Add an RPC end-point to do simple LNS resolution; you can get the same info out of
names-to-owners, but the new lns_resolve end-point is considerably simpler for doing simple
lookups (e.g. for lokinet), and allows for a simpler SQL query + processing.
Code changes:
- Rename mapping_type::lokinet_1year to mapping_type::lokinet (see next point).
- Don't store lokinet_2y, etc. in the database, but instead always store as type=2/::lokinet. The
LNS extra data can still specify different lengths, but this now just affects the
expiration_height value that we set.
- Reworked some binding code to use std::variant's and add a bind_container() to simplify passing in
a variable list of bind parameters of different types.
- Accept both base64 and hex inputs for binary LNS parameters in the RPC interface.
- This commit adds some (incomplete) expiry adjustment code, but ignore it as it all gets replaced
with the following commit to overhaul record updating.
- Updated a bunch of test suite code, mainly related to lokinet.
- Some random C++17 niceties (string_view, variant, structured binding returns) in the related code.
- Tweaked the test suite to generate a bit fewer blocks in some cases where we just need to
confirm/unlock a transfers rather than a coinbase tx.
boost::serialization 1.74 has a missing header in some of the container
serialization types (list, set). This adds a workaround in blockchain.h,
and reorders the headers in wallet2.h to implicitly get the required
include.
In short: epee's http client is garbage, standard violating, and
unreliable.
This completely removes the epee http client support and replaces it
with cpr, a curl-based C++ wrapper. rpc/http_client.h wraps cpr for RPC
requests specifically, but it is also usable directly.
This replacement has a number of advantages:
- requests are considerably more reliable. The epee http client code
assumes that a connection will be kept alive forever, and returns a
failure if a connection is ever closed. This results in some very
annoying things: for example, preparing a transaction and then waiting
a long tim before confirming it will usually result in an error
communication with the daemon. This is just terribly behaviour: the
right thing to do on a connection failure is to resubmit the request.
- epee's http client is broken in lots of other ways: for example, it
tries throwing SSL at the port to see if it is HTTPS, but this is
protocol violating and just breaks (with a several second timeout) on
anything that *isn't* epee http server (for example, when lokid is
behind a proxying server).
- even when it isn't doing the above, the client breaks in other ways:
for example, there is a comment (replaced in this PR) in the Trezor PR
code that forces a connection close after every request because epee's
http client doesn't do proper keep-alive request handling.
- it seems noticeably faster to me in practical use in this PR; both
simple requests (for example, when running `lokid status`) and
wallet<->daemon connections are faster, probably because of crappy
code in epee. (I think this is also related to the throw-ssl-at-it
junk above: the epee client always generates an ssl certificate during
static initialization because it might need one at some point).
- significantly reduces the amount of code we have to maintain.
- removes all the epee ssl option code: curl can handle all of that just
fine.
- removes the epee socks proxy code; curl can handle that just fine.
(And can do more: it also supports using HTTP/HTTPS proxies).
- When a cli wallet connection fails we know show why it failed (which
now is an error message from curl), which could have all sorts of
reasons like hostname resolution failure, bad ssl certificate, etc.
Previously you just got a useless generic error that tells you
nothing.
Other related changes in this PR:
- Drops the check-for-update and download-update code. To the best of
my knowledge these have never been supported in loki-core and so it
didn't seem worth the trouble to convert them to use cpr for the
requests.
- Cleaned up node_rpc_proxy return values: there was an inconsistent mix
of ways to return errors and how the returned strings were handled.
Instead this cleans it up to return a pair<bool, val>, which (with
C++17) can be transparently captured as:
auto [success, val] = node.whatever(req);
This drops the failure message string, but it was almost always set to
something fairly useless (if we want to resurrect it we could easily
change the first element to be a custom type with a bool operator for
success, and a `.error` attribute containing some error string, but
for the most part the current code wasn't doing much useful with the
failure string).
- changed local detection (for automatic trusted daemon determination)
to just look for localhost, and to not try to resolve anything.
Trusting non-public IPs does not work well (e.g. with lokinet where
all .loki addresses resolve to a local IP).
- ssl fingerprint option is removed; this isn't supported by curl
(because it is essentially just duplicating what a custom cainfo
bundle does)
- --daemon-ssl-allow-chained is removed; it wasn't a useful option (if
you don't want chaining, don't specify a cainfo chain).
- --daemon-address is now a URL instead of just host:port. (If you omit
the protocol, http:// is prepended).
- --daemon-host and --daemon-port are now deprecated and produce a
warning (in simplewallet) if used; the replacement is to use
--daemon-address.
- --daemon-ssl is deprecated; specify --daemon-address=https://whatever
instead.
- the above three are now hidden from --help
- reordered the wallet connection options to make more logical sense.
This moves the wallet structs that are needed elsewhere (for instance,
in the device drivers) from `tools::wallet2::whatever` to
`wallet::whatever`, allowing them to be included individually via
wallet/whatever.h without needing to include the entire wallet2.h.
What was particularly problematic here is that you couldn't even forward
declare them because they were nested inside the wallet2 class, but
instead had to include the full wallet2.h.
Jason Rhinelander
Sean Darcy
Sean Darcy
Doyle