Commit graph

231 commits

Author SHA1 Message Date
Jason Rhinelander 06a7148515 Fix macos no-dynamic-libs check 2022-11-30 23:00:58 -04:00
Jason Rhinelander 72a356798d
apple now links everything to libobjc.dylib now, hurray for great success! 2022-10-17 13:45:23 -03:00
Thomas Winget 1311a20e9f merge dev branch with RPC/wallet3 changes
Incomplete, many things to fix, some annotated with
a comment MERGEFIX
2022-07-11 20:40:50 -04:00
Sean dce39ec8ee
Merge branch 'dev' into batching-rpc-endpoints 2022-05-24 11:29:49 +10:00
Sean Darcy f45d0588ae Adds RPC endpoint for accrued batching balances
This allows a user to query the daemon and ask for the accrued balance
of oxen rewards that have been batched but not yet paid out.

The endpoint takes in an array of addresses (Strings) and returns an
array of atomic oxen amounts (integers) that reflect the current
database balances for those requested addresses.

For example an address has accrued 2x batch payments of 16.5 oxen but
has not yet been paid out will show 33000000000 being the atomic amount
in the database.
2022-05-23 08:47:18 +10:00
Sean Darcy 9edc2bb52d
Increase max number of contributors to 10
This requires the operator to still contribute 25% of the service node
but another 9 nodes will be allowed to stake to the node makeing 10
contributors total rather than our previous 4.
2022-05-10 18:48:20 -03:00
Sean Darcy 866691d9d8 Batching of service node rewards
This updates the coinbase transactions to reward service nodes
periodically rather than every block. If you recieve a service node
reward this reward will be delayed x blocks, if you receive another
reward to the same wallet before those blocks have been completed it
will be added to your total and all will be paid out after those x
blocks has passed.

For example if our batching interval is 2 blocks:

Block 1 - Address A receives reward of 10 oxen - added to batch
Block 2 - Address A receives reward of 10 oxen - added to batch
Block 3 - Address A is paid out 20 oxen.

Batching accumulates a small reward for all nodes every block

The batching of service node rewards allows us to drip feed rewards
to service nodes. Rather than accruing each service node 16.5 oxen every
time they are pulse block leader we now reward every node the 16.5 /
num_service_nodes every block and pay each wallet the full amount that
has been accrued after a period of time (Likely 3.5 days).

To spread each payment evenly we now pay the rewards based on the
address of the recipient. This modulus of their address determines which
block the address should be paid and by setting the interval to our
service_node_batching interval we can guarantee they will be paid out
regularly and evenly distribute the payments for all wallets over this
2022-04-29 09:51:14 +10:00
Jason Rhinelander 0638aea2fb
Fix apple static libs check 2022-04-15 14:32:06 -03:00
Jason Rhinelander 3cfbf0fb66
Windows build fixes
- bump libusb version
- build hidapi using cmake (autoconf is deprecated, and doesn't properly
  set up the required linking on Windows)
- pass through toolchain
- zmq/win32 compilation fix
2022-04-14 14:36:59 -03:00
Jason Rhinelander 5a5c9bf450
Update static build deps to latest versions 2022-04-14 14:32:02 -03:00
Jason Rhinelander 3e3da24351 lmq-rpc.py: curve:// url support and binary bt printing
- Make curve://HOST/PUBKEY URLs work (equivalent to specifying two
  arguments, tcp://HOST PUBKEY, on the command line).

- When we get a response that looks bt-encoded (starts with d/l/i and
  ends with e) then write it as binary rather than having Python display
  the bytes representation.  This allows piping it to lokinet's
  `contrib/bencode-dump.py` to dump the contents.
2021-11-01 14:53:10 -04:00
Jason Rhinelander 4a8023ade6 Make RPC more flexible & add bt-encoding
The bt-encoding, in particular, is aimed at more efficient wallet3 rpc
interactions with a daemon.

This removes the rigidity from RPC request responses, making it easier
to return useful constructs that the current epee code doesn't support
(like lists of lists), and also adds support for bt-encoded RPC requests
(intended to eventually replace the ".bin" endpoints with Monero NIH
binary encoding).

Instead endpoints now set a nlohmann::json response object with whatever
arbitrary values they want without having to add a bunch of limited epee
serialization macro hell.

So, for example, to set the value "foo" to 20, you use:

    rpc.response["height"] = 20;

Binary values (for things like hashes) are handled by going through a proxy object:

    rpc.response_hex["hash"] = hash; // hash is a crypto::hash

which, for json, is equivalent to:

    rpc.response["hash"] = tools::type_to_hex(hash);

but when the response is to be bt-encoded it leaves it as binary for
more efficient transfers.

There is also a `response_b64` that converts json-destined values to
base64 instead of hex (and again leaves bt-destined values as binary).

Parsing of incoming requests now moves to a new
core_rpc_server_command_parser file, which does free-form parsing from a
nlohmann::json or bt_dict_consumer, and has various templated helper
functions to make this easier.

This is preliminary: this builds the basic infrastucture for handling
requests, and converts three endpoints:

- get_info
- get_height
- ons_resolve

Currently only `make daemon` builds (there is code in the wallet that
hasn't been updated yet), and the other RPC interfaces are disabled by
this commit (until they get converted to the new style).
2021-11-01 14:53:05 -04:00
Jason Rhinelander 14cb314f70 Add patch for hidapi + autoconf 2.70+
Fixes the mac build, and will fix future linux build once distros start
shipping 2.70+.
2021-07-06 20:07:52 -03:00
Sean Darcy 2ddf644844 local devnet scripts 2021-05-18 09:50:17 +10:00
Jason Rhinelander f3325f9ec8 Clean up unused/unmaintained contrib and utils
Everything removed here is unsupported and doesn't work (leftover stuff
from Monero).
2021-03-28 22:08:45 -03:00
Jason Rhinelander 91a4e7f2df Make ios static lib match android static lib name 2021-01-08 18:16:06 -04:00
Jason Rhinelander 37159b2ba0 Export symbols and strip debug symbols from android static lib
This massively reduces the size.
2021-01-08 18:15:25 -04:00
Jason Rhinelander d19d8dd0ee Update default ipc socket in lmq-rpc.py 2021-01-07 18:40:22 -04:00
Sean Darcy d37a6e88c9 Drone static upload still looking for loki executables 2021-01-07 18:40:22 -04:00
Sean Darcy 2822747a62 lokinet revert 2021-01-07 18:40:22 -04:00
Jason Rhinelander 5c9ed9b61d Fix android tarfile creation 2021-01-07 18:37:52 -04:00
Jason Rhinelander d3b08c4a5e builds.lokinet.dev -> oxen.rocks 2021-01-07 18:37:52 -04:00
Jason Rhinelander 2d2e7c53ac Add 'include' & 'lib' dirs into android deps archive
This makes it directly extractable into what the new android wallet
expects (and matches the ios archive, aside from the arch subdirs).
2021-01-07 18:37:52 -04:00
Jason Rhinelander cdf62b2377 Remove all but the wallet2_api header for ios
The new ios wallet now uses the wallet2_api.h rather than directly
including core loki headers which greatly simplifies what we have to
package.
2021-01-07 18:37:52 -04:00
Jason Rhinelander 952543da76 Update electron wallet filename 2021-01-07 18:35:21 -04:00
Jason Rhinelander 9dd56bdc43 Print to stderr to not break | jq . 2020-11-26 23:55:04 -04:00
Jason Rhinelander 8e049e563a Fix encrypted connections in utils/lmq-rpc.py
- Added client key generation if no curve pubkey is specified
- Fixed missing import causing an error if you tried to specify an
x25519 privkey
2020-11-26 23:52:16 -04:00
Jason Rhinelander 3dd6bbf2ba Add snapshot gui wallet builds + uploads 2020-11-19 22:54:42 -04:00
Jason Rhinelander 8175e5e810 Run lokid test under gdb
Scripts for running under gdb + dumping a stack trace on problems from
lokinet.
2020-11-06 18:04:20 -04:00
Jason Rhinelander 7f83c58c63 Merge arm/sim libs and collect headers in ios deps package 2020-11-06 00:41:28 -04:00
Jason Rhinelander 7d0eeeff06 Also build simulator library 2020-11-06 00:41:28 -04:00
Jason Rhinelander ca1a89ded0 Add ios build & upload
Also renames the android upload script to drone-android-blahblah to be
more consistent.
2020-11-06 00:41:28 -04:00
Jason Rhinelander daaa8dbefd Lots of build hacks for iOS 2020-11-04 15:06:59 -04:00
Jason Rhinelander 0bc97684a2 Package merged wallet_api rather than plain wallet_api 2020-11-04 12:11:20 -04:00
Jason Rhinelander 7a533e19a3 Add android build to drone steps 2020-11-04 12:11:20 -04:00
Jason Rhinelander f6c3adfb8e Small tweaks to build system
- make tagged and stable branch builds not add git tag to the tar.xz
  filename
- copy apt-get tweaks (to make it less noisy) from deb branches
- do a faster shallow submodule clone
- remove exclusion for removed is_hdd test
- remove deb_builder (deb building drone code is in its own branches)
- fix upload dir for a tagged build
2020-10-03 23:24:08 -03:00
Jason Rhinelander 48e28d4c91 Improve wording on deb conversion script 2020-09-18 20:53:04 -03:00
Jason Rhinelander 5ba6197d59 Update zeromq patches for mingw 2020-09-16 13:14:24 -03:00
Jason Rhinelander 68e84c3c63 lmq-rpc.py: fix missing data parts 2020-08-17 02:44:17 -03:00
Jason Rhinelander 0a49ce5a51
Added loki-launcher-to-debs conversion script (#1203) 2020-08-17 13:12:33 +10:00
Jason Rhinelander b7a7dfc080 Add zlib + curl to static deps 2020-08-07 17:14:03 -03:00
Jason Rhinelander 300667e5f2
Add support for N=11 and N=12 quorum comms (#1178)
Previously we supported N=7 through N=10 and N=13 through N=20; with
pulse we need N=11 and N=12, so fill in the gaps so we now have a matrix
for everything from N=7 to N=20.

Connections are generated from utils/generate-quorum-matrix.py using
that same criteria as the existing connection matrices (that is, finding
a set of connections to ensure that every node is connected to every
other node twice with direct or two-hop connections).  The missing added
connection details are as follows:

┌────┬───────────────────────────────┬─────────────┬────────────────────┐
│ i  │ Connections (1 = out, x = in) │ ≤2 paths    │ Connectivity:      │
├────┼───────────────────────────────┼─────────────┼────────────────────┤
│ 0  │ -100xx1x001                   │ -3433343332 │ 6 (= 3 out + 3 in) │
│ 1  │ x-10x0011x0                   │ 3-333344322 │ 6 (= 3 out + 3 in) │
│ 2  │ 0x-11x01000                   │ 43-33232322 │ 5 (= 3 out + 2 in) │
│ 3  │ 00x-110010x                   │ 333-2322223 │ 5 (= 3 out + 2 in) │
│ 4  │ 11xx-010000                   │ 3332-424222 │ 5 (= 3 out + 2 in) │
│ 5  │ 101x0-100xx                   │ 33234-33323 │ 6 (= 3 out + 3 in) │
│ 6  │ x000xx-1010                   │ 443223-2222 │ 5 (= 2 out + 3 in) │
│ 7  │ 1xx000x-100                   │ 3422432-232 │ 5 (= 2 out + 3 in) │
│ 8  │ 0x0x000x-11                   │ 33322322-22 │ 5 (= 2 out + 3 in) │
│ 9  │ 010001x0x-0                   │ 322222232-2 │ 4 (= 2 out + 2 in) │
│ 10 │ x0010100x0-                   │ 2223232222- │ 4 (= 2 out + 2 in) │
└────┴───────────────────────────────┴─────────────┴────────────────────┘

┌────┬───────────────────────────────┬──────────────┬────────────────────┐
│ i  │ Connections (1 = out, x = in) │ ≤2 paths     │ Connectivity:      │
├────┼───────────────────────────────┼──────────────┼────────────────────┤
│ 0  │ -10xx0x00010                  │ -23332332222 │ 5 (= 2 out + 3 in) │
│ 1  │ x-110x001000                  │ 2-3422232222 │ 5 (= 3 out + 2 in) │
│ 2  │ 0x-10100001x                  │ 33-222233222 │ 5 (= 3 out + 2 in) │
│ 3  │ 1xx-1001x000                  │ 342-23323233 │ 6 (= 3 out + 3 in) │
│ 4  │ 100x-010010x                  │ 3222-2323332 │ 5 (= 3 out + 2 in) │
│ 5  │ 01x00-110x00                  │ 22232-323232 │ 5 (= 3 out + 2 in) │
│ 6  │ 1000xx-101x0                  │ 322333-32333 │ 6 (= 3 out + 3 in) │
│ 7  │ 000x0xx-1010                  │ 3332223-2322 │ 5 (= 2 out + 3 in) │
│ 8  │ 0x01000x-101                  │ 22333322-222 │ 5 (= 3 out + 2 in) │
│ 9  │ 0000x1x0x-01                  │ 222232332-23 │ 5 (= 2 out + 3 in) │
│ 10 │ x0x0001x00-1                  │ 2223333222-2 │ 5 (= 2 out + 3 in) │
│ 11 │ 00101000xxx-                  │ 22232232232- │ 5 (= 2 out + 3 in) │
└────┴───────────────────────────────┴──────────────┴────────────────────┘
2020-06-22 11:10:52 +10:00
Jason Rhinelander 5da3d2f7ef Drone: add static builds for mac/linux/windows
Adds static drone builds for linux (built on bionic), mac, and Windows
(built with mingw32).

The builds get uploaded to https://builds.lokinet.dev

The linux and mac builds use LTO (which takes longer, but significantly
reduces binary size).  The mingw32 build can probably also get there,
but currently fails with LTO when unbound tries linking against openssl
(it probably just needs a small patch to add magic -lwsock2 dep in the
right place in unbound).

The Mac binaries are built using a 10.13 deployment target.  (I'm not
100% sure that this is sufficient -- it's possible we might have to also
push the magic mac deployment flag to the built dependencies).
2020-06-15 12:49:34 -03:00
Jason Rhinelander 70b9fed4fd Static builds: make usable binaries from cmake
This adds a static dependency script for libraries like boost, unbound,
etc. to cmake, invokable with:

    cmake .. -DBUILD_STATIC_DEPS=ON

which downloads and builds static versions of all our required
dependencies (boost, unbound, openssl, ncurses, etc.).  It also implies
-DSTATIC=ON to build other vendored deps (like miniupnpc, lokimq) as
static as well.

Unlike the contrib/depends system, this is easier to maintain (one
script using nicer cmake with functions instead of raw Makefile
spaghetti code), and isn't concerned with reproducible builds -- this
doesn't rebuild the compiler, for instance.  It also works with the
existing build system so that it is simply another way to invoke the
cmake build scripts but doesn't require any external tooling.

This works on Linux, Mac, and Windows.

Some random comments on this commit (for preserving history):

- Don't use target_link_libraries on imported targets.  Newer cmake is
fine with it, but Bionic's cmake doesn't like it but seems okay with
setting the properties directly.

- This rebuilds libzmq and libsodium, even though there is some
provision already within loki-core to do so: however, the existing
embedded libzmq fails with the static deps because it uses libzmq's
cmake build script, which relies on pkg-config to find libsodium which
ends up finding the system one (or not finding any), rather than the one
we build with DownloadLibSodium.  Since both libsodium and libzmq are
faily simple builds it seemed easiest to just add them to the cmake
static build rather than trying to shoehorn the current code into the
static build script.

- Half of the protobuf build system ignores CC/CXX just because Google,
and there's no documentation anywhere except for a random closed bug
report about needing to set these other variables (CC_FOR_BUILD,
CXX_FOR_BUILD) instead, but you need to.  Thanks Google.

- The boost build is set to output very little because even the minimum
-d1 output level spams ~15k lines of output just for the headers it
installs.
2020-06-15 12:49:33 -03:00
Doyle 7c07a3e8f3 Merge commit '8136bf37e2c0a76851c0bb6482b6e4c2b653f5d7' into MergeUpstream3 2020-05-27 15:40:27 +10:00
Doyle a6db0e1baf Merge commit 'beb815a' into MergeUpstream3 2020-05-27 15:03:44 +10:00
Doyle 334f94af62 Merge commit 'c39ea48' into MergeUpstream3 2020-05-27 14:43:35 +10:00
Doyle f2f48f8815 Merge commit 'b4e1dc83d275f8ee9a8c12615cf952f05161c7a3' into MergeUpstream3 2020-05-27 14:25:14 +10:00
Doyle 836d51b7c9 Merge commit '3b0a43a' into MergeUpstream3 2020-05-25 15:35:00 +10:00
Doyle 0f818c929b Merge commit 'f253bf3' into MergeUpstream3 2020-05-25 15:17:47 +10:00