Adds support for building Lokinet as a system extension, and fixes
various problems in the macos implementation found during development of
the system extension support.
Using constructor inheritance DRYs the code, but unfortunately confuses
GCC as to where the proper "required from here" location is, which makes
debugging formatting errors very difficult. Avoid it (and update
oxen-logging to avoid it there as well).
Replaces custom logging system with spdlog-based oxen logging. This
commit mainly replaces the backend logging with the spdlog-based system,
but doesn't (yet) convert all the existing LogWarn, etc. to use the new
format-based logging.
New logging statements will look like:
llarp::log::warning(cat, "blah: {}", val);
where `cat` should be set up in each .cpp or cluster of .cpp files, as
described in the oxen-logging README.
As part of spdlog we get fmt, which gives us nice format strings, where
are applied generously in this commit.
Making types printable now requires two steps:
- add a ToString() method
- add this specialization:
template <>
constexpr inline bool llarp::IsToStringFormattable<llarp::Whatever> = true;
This will then allow the type to be printed as a "{}" value in a
fmt::format string. This is applied to all our printable types here,
and all of the `operator<<` are removed.
This commit also:
- replaces various uses of `operator<<` to ToString()
- replaces various uses of std::stringstream with either fmt::format or
plain std::string
- Rename some to_string and toString() methods to ToString() for
consistency (and to work with fmt)
- Replace `stringify(...)` and `make_exception` usage with fmt::format
(and remove stringify/make_exception from util/str.hpp).
Fixes a bug on older cmake linking against oxenmq (older cmake hates the
direct oxenmq::oxenmq -> PkgConfig::OXENMQ alias), and also makes it
easier to handle things like nlohmann::json (which we can use from the
system *or* submodule).
Borrowed from oxen-ss/oxen-core.
bump submodules
Fix dynamic version generation
- GIT_FOUND OR Git_FOUND wasn't available because we hadn't done a
find_package(Git) yet.
- required version variables weren't being passed through to the cmake
script
ngtcp2's top-level CMakeLists.txt is not friendly to being used as a
subdirectory (for instance, it always adds a `check` target when cunit
is installed), so stop using it in favour of skipping directly into the
ngtcp2/lib subdir.
This requires some hackery to set up a couple of the things the
top-level cmake does, but it isn't *too* painful.
- Move uvw target to external instead of llarp/CMakeLists.txt
- Add sqlite_orm headers via interface library rather than shoving it
into the global include search path.
- Find sqlite3 as part of the sqlite_orm target deps rather than linking
it separately. (Also don't link it into lokinet-util, which doesn't
appear to actually need it).
- removes all the llarp_ev_* functions, replacing with methods/classes/functions in the llarp
namespace.
- banish ev/ev.h to the void
- Passes various things by const lvalue ref, especially shared_ptr's that don't need to be copied
(to avoid an atomic refcount increment/decrement).
- Add a llarp::UDPHandle abstract class for UDP handling
- Removes the UDP tick handler; code that needs tick can just do a separate handler on the event
loop outside the UDP socket.
- Adds an "OwnedBuffer" which owns its own memory but is implicitly convertible to a llarp_buffer_t.
This is mostly needed to take over ownership of buffers from uvw without copying them as,
currently, uvw does its own allocation (pending some open upstream issues/PRs).
- Logic:
- add `make_caller`/`call_forever`/`call_every` utility functions to abstract Call wrapping and
dependent timed tasks.
- Add inLogicThread() so that code can tell its inside the logic thread (typically for
debugging assertions).
- get rid of janky integer returns and dealing with cancellations on call_later: the other methods
added here and the event loop code remove the need for them.
- Event loop:
- redo everything with uvw instead of libuv
- rename EventLoopWakeup::Wakeup to EventLoopWakeup::Trigger to better reflect what it does.
- add EventLoopRepeater for repeated events, and replace the code that reschedules itself every
time it is called with a repeater.
- Split up `EventLoop::run()` into a non-virtual base method and abstract `run_loop()` methods;
the base method does a couple extra setup/teardown things that don't need to be in the derived class.
- udp_listen is replaced with ev->udp(...) which returns a new UDPHandle object rather that
needing gross C-style-but-not-actually-C-compatible structs.
- Remove unused register_poll_fd_(un)readable
- Use shared_ptr for EventLoopWakeup rather than returning a raw pointer; uvw lets us not have to
worry about having the event loop class maintain ownership of it.
- Add factory EventLoop::create() function to create a default (uvw-based) event loop (previously
this was one of the llarp_ev_blahblah unnamespaced functions).
- ev_libuv: this is mostly rewritten; all of the glue code/structs, in particular, are gone as
they are no longer needed with uvw.
- DNS:
- Rename DnsHandler to DnsInterceptor to better describe what it does (this is the code that
intercepts all DNS to the tun IP range for Android).
- endpoint:
- remove unused "isolated network" code
- remove distinct (but actually always the same) variables for router/endpoint logic objects
- llarp_buffer_t
- make constructors type-safe against being called with points to non-size-1 values
- tun packet reading:
- read all available packets off the device/file descriptor; previously we were reading one packet
at a time then returning to the event loop to poll again.
- ReadNextPacket() now returns a 0-size packet if the read would block (so that we can implement
the previous point).
- ReadNextPacket() now throws on I/O error
- Miscellaneous code cleanups/simplifications
* Update how we build libuv
- Update submoduled libuv to latest stable (1.40.0)
- Don't look for a system libuv if we're under BUILD_STATIC_DEPS
- Add a libuv interface library rather than using globals
- Make the windows build fall back to the submodule if not explicitly
given a LIBUV_ROOT
* Replace ${LIBS} global with `base_libs` interface
This simplifies linking and include handling a bit.
* Remove unneeded header
* Add missing csignal header
(This was previously being pulled in incredibly indirectly via some
stuff that eventually includes some other stuff that eventually included
uv.h)
* Use GNUInstallDirs to get lib dir instead of hard-coding lib
Fixes#1429
This is relatively painless for lokinet as it already had workarounds
during 0.8 dev work for the things macos hated in 10.13.
Dependencies, however, were not being built with the proper macos target
junk, so this fixes that.
* initial work for RC SRVs.
Needs tests for new RC format.
Needs SRVs added to new RC, and associated tests.
* convert rc sign/verify test to catch2, add test for new rc
Also fixes a mistake in new rc serialization
* bump loki-mq submodule
need support for viewing bt deserialize consumer buffer so we
know how much it has consumed.
* fix some behavior errors
need to advance llarp_buffer_t after consuming data from it
only rewind and modify size of llarp_buffer_t in owning context.
* Add test for router contact (de-)serialization
Adds a test that makes a list of RouterContact with mixed
versions and ensures it serializes and deserializes correctly.
