When targetting macos <10.14 macos won't allow use of anything from
C++17 that throws, such as:
- std::get on a variant
- std::visit
- std::optional::value()
- std::any_cast
This avoids all of these.
For std::get, we either replace with std::get_if (where appropriate), or
else use a `var::get` implementation of std::get added to lokimq (also
updated here). (This `var` namespace is just an `std` alias everywhere
*except* old target macos).
For std::visit, likewise lokimq adds an var::visit implementation for
old macos that we use.
std::optional::value() uses weren't useful anyway as everywhere it calls
them we've already checked that the option has a value, in which case we
can use `*opt` (which doesn't check for contents and throw).
std::any just has to be avoided as far as I can tell, but the one place
we used it is only ever a block, so I just replaced it with a `const
block*`.
- make tagged and stable branch builds not add git tag to the tar.xz
filename
- copy apt-get tweaks (to make it less noisy) from deb branches
- do a faster shallow submodule clone
- remove exclusion for removed is_hdd test
- remove deb_builder (deb building drone code is in its own branches)
- fix upload dir for a tagged build
In short: epee's http client is garbage, standard violating, and
unreliable.
This completely removes the epee http client support and replaces it
with cpr, a curl-based C++ wrapper. rpc/http_client.h wraps cpr for RPC
requests specifically, but it is also usable directly.
This replacement has a number of advantages:
- requests are considerably more reliable. The epee http client code
assumes that a connection will be kept alive forever, and returns a
failure if a connection is ever closed. This results in some very
annoying things: for example, preparing a transaction and then waiting
a long tim before confirming it will usually result in an error
communication with the daemon. This is just terribly behaviour: the
right thing to do on a connection failure is to resubmit the request.
- epee's http client is broken in lots of other ways: for example, it
tries throwing SSL at the port to see if it is HTTPS, but this is
protocol violating and just breaks (with a several second timeout) on
anything that *isn't* epee http server (for example, when lokid is
behind a proxying server).
- even when it isn't doing the above, the client breaks in other ways:
for example, there is a comment (replaced in this PR) in the Trezor PR
code that forces a connection close after every request because epee's
http client doesn't do proper keep-alive request handling.
- it seems noticeably faster to me in practical use in this PR; both
simple requests (for example, when running `lokid status`) and
wallet<->daemon connections are faster, probably because of crappy
code in epee. (I think this is also related to the throw-ssl-at-it
junk above: the epee client always generates an ssl certificate during
static initialization because it might need one at some point).
- significantly reduces the amount of code we have to maintain.
- removes all the epee ssl option code: curl can handle all of that just
fine.
- removes the epee socks proxy code; curl can handle that just fine.
(And can do more: it also supports using HTTP/HTTPS proxies).
- When a cli wallet connection fails we know show why it failed (which
now is an error message from curl), which could have all sorts of
reasons like hostname resolution failure, bad ssl certificate, etc.
Previously you just got a useless generic error that tells you
nothing.
Other related changes in this PR:
- Drops the check-for-update and download-update code. To the best of
my knowledge these have never been supported in loki-core and so it
didn't seem worth the trouble to convert them to use cpr for the
requests.
- Cleaned up node_rpc_proxy return values: there was an inconsistent mix
of ways to return errors and how the returned strings were handled.
Instead this cleans it up to return a pair<bool, val>, which (with
C++17) can be transparently captured as:
auto [success, val] = node.whatever(req);
This drops the failure message string, but it was almost always set to
something fairly useless (if we want to resurrect it we could easily
change the first element to be a custom type with a bool operator for
success, and a `.error` attribute containing some error string, but
for the most part the current code wasn't doing much useful with the
failure string).
- changed local detection (for automatic trusted daemon determination)
to just look for localhost, and to not try to resolve anything.
Trusting non-public IPs does not work well (e.g. with lokinet where
all .loki addresses resolve to a local IP).
- ssl fingerprint option is removed; this isn't supported by curl
(because it is essentially just duplicating what a custom cainfo
bundle does)
- --daemon-ssl-allow-chained is removed; it wasn't a useful option (if
you don't want chaining, don't specify a cainfo chain).
- --daemon-address is now a URL instead of just host:port. (If you omit
the protocol, http:// is prepended).
- --daemon-host and --daemon-port are now deprecated and produce a
warning (in simplewallet) if used; the replacement is to use
--daemon-address.
- --daemon-ssl is deprecated; specify --daemon-address=https://whatever
instead.
- the above three are now hidden from --help
- reordered the wallet connection options to make more logical sense.
MacOS build-in compiler doesn't like std::variant when targetting
anything earlier than 10.14.
(We can deal with this for releases by making a build for older macs
that uses a full clang stack, but will worry about that later).
This uses EXCLUDE_FROM_ALL when BUILD_DEBUG_UTILS is not on so that they
don't get built but they *can* be built with a `make cn_deserialize` (or
whatever target you want) from a build dir.
Also adds them to a couple drone builds to make sure they build.
This makes it a little clearer what failed when something fails.
Ideally we'd also be able to distinguish between build and test, but
currently that's not practical to do with drone: between each stage of
the build the docker image gets removed and replaced with the next
stage's image, which means we'd have to build up a new docker image with
dependencies before running the tests.
(Being able to have multiple stages in one docker image is a drone
upstream wishlist item).
I've uploaded the various deps to https://builds.lokinet.dev/deps so as
not to hit the upstream mirrors so much (also reduces the reliance on
github, which is sometimes flakey).
This is prefixed onto the existing URLs so that if it isn't found we
fall back to the upstream URL.
Adds static drone builds for linux (built on bionic), mac, and Windows
(built with mingw32).
The builds get uploaded to https://builds.lokinet.dev
The linux and mac builds use LTO (which takes longer, but significantly
reduces binary size). The mingw32 build can probably also get there,
but currently fails with LTO when unbound tries linking against openssl
(it probably just needs a small patch to add magic -lwsock2 dep in the
right place in unbound).
The Mac binaries are built using a 10.13 deployment target. (I'm not
100% sure that this is sufficient -- it's possible we might have to also
push the magic mac deployment flag to the built dependencies).
This makes three big changes to how translation files are generated:
- use Qt5 cmake built-in commands to do the translations rather than
calling lrelease directly. lrelease is often not in the path, while Qt5
cmake knows how to find and invoke it.
- Slam the resulting files into a C++ file using a cmake script rather
than needing to compile a .c file to generate C++ file. This is
simpler, but more importantly avoids the mess needed when cross
compiling of having to import a cmake script from an external native
build.
- In the actual generated files, use an unordered_map rather than a
massive list of static variable pointers.
CMake 3.9+ has generic LTO enabling code, switch to it.
Update required cmake version to 3.10 (3.9 is probably sufficient, but
3.10 is bionic's version that we're actually testing).
Adds various builds and does a basic offline startup test on all, plus
runs the full test suite on a debian and a mac build.
This has the ability to do an all-static build and to build debs, and
upload both to builds.lokinet.dev, but it is not yet enabled and will
likely take some additional work.
Jason Rhinelander