Commit graph

9477 commits

Author SHA1 Message Date
Jason Rhinelander
2b568f02d2 Add support for loading from various container types
- Supports loading into tuples, std::arrays, std::pairs from a list of
  the correct size.
- Supports loading into vectors of arbitrary types (include vector,
  tuples, binary types, etc.)
- Removes system_clock type as we aren't using it (and because it won't
  be recursively handled nicely when nested, especially for json).
2021-11-01 14:53:10 -04:00
Sean Darcy
71fb298b3a parse params of get_outputs 2021-11-01 14:53:10 -04:00
Sean Darcy
533b319fff define parsers, returning correct types 2021-11-01 14:53:10 -04:00
Jason Rhinelander
a8b36efd4c Update doc formatting style
Use `- \p paramname Description` rather than `\arg \c paramname
Description`.

\arg is for compatibility with some particular doc format and is
equivalent to - except that - can be nested while \arg cannot.

\c and \p are equivalent, but \p is explicitly intended for parameters.
2021-11-01 14:53:10 -04:00
Jason Rhinelander
9a0e23ca05 get_service_node *client* updates
Updates the internal callers of get_service_nodes to properly deal with
the new request format/fields.

Note that this also introduces some significant changes to the
`print_sn` output:

- contributor formats are more compact (one line per contributor), show
  reserved amounts (if != contributed amouont), and show the stake %.
- reworded "Operator Cut (% of Fee Reward)" to "Operator Fee"
- checkpoint/pulse vote format is completely overhauled to list voted
  and missed heights separately.
- pulse votes now show the pulse round for non-zero round results.
- timestamp checks renamed to Quorumnet tests
- qnet & timesync results now just print the number of successes/fails
  rather than a list of true/false values.
2021-11-01 14:53:10 -04:00
Jason Rhinelander
ebc8bc79e5 RPC: GET_SERVICE_NODES & GET_SERVICE_NODE_STATUS
Converts request parsing and result generation for get_service_nodes and
get_service_node_status.

Note that there are some significant changes here (due to no longer
having to be under epee's output constraints) that significantly reduce
the size of an all-service-nodes request by nearly half, most notably in
a significantly more compact representation of vote/participation
results.

- added a "decomm_reasons" which decomposes the reason codes into string
  values such as:

    "decomm_reasons": {
        "all": ["uptime"],
        "some": ["pulse", "lokinet"]
    }
- reworded/fixed/expanded various field descriptions
- Various Loki currency references changed to OXEN
- added "operator_fee" which is a rounded operator fee expressed as
  thousanths of a percent (e.g.  100000 means 100%, 4567 means 4.567%).
  This is inaccurate but much easier to deal with than the raw
  portions_for_operator value (especially because the value *usually*
  exceeds what can be stored in a double, and thus is not safe with
  various JSON parsers).
- added "swarm" which is a hex string representation of the swarm_id,
  because swarm_id also isn't double-storage safe.
- reserved and total_reserved fields are now only included if they are
  actually different from the amount/total_contributed values.
- various uptime-proof-dependent values (like versions, public ip,
  ports) are now omitted if we haven't received an uptime proof yet.
- Flipped inverted descriptions of timestamp participation / synced
  values in the RPC documentation.
- checkpoint/pulse/timecheck/timesync participations data is completely
  redesigned to be less bloated; the names are now changed to
  "checkpoint_votes" and "pulse_votes" and information is returned much
  more compactly using lists of lists and/or plain values rather than
  lists of dicts that repeat a ton of fields.
- Renamed timestamp_participation in the RPC to quorumnet_tests since,
  although we're doing the test for time checking, the main purpose of
  the participation is to see that we can reach them.  As above, the
  result value is significantly more compact: [SUCCESS,FAILS] instead of
  an array of dicts.
- Similarly renamed timesync_status to timesync_tests ("status" was
  fine, but changing the value completely to [SUCCESS,FAILS] warrants a
  name change).
- dropped the "as_json" field because it is asking oxend to
  double-encode json, and that is retarded.
- also applies all the same refactoring to get_service_node_status
  (which is really just a shortcut for getting the current node's status
  out of get_service_nodes).
- overhauled how "fields" works to take a list of field names (the dict
  is name: bool values is still supported for backwards compatibility).
  Also extended fields to handle "locked_contributions" as a supported
  field (so that you can easily omit it, since it often isn't cared
  about).
2021-11-01 14:53:10 -04:00
Jason Rhinelander
4317feb7d6 Promote critical startup errors to MFATAL
At MERROR they don't print at the default --log-level=0, so you get
output such as:

    2021-08-06 18:04:08.073	I Starting up oxend services...
    2021-08-06 18:04:08.073	I Starting core
    2021-08-06 18:04:08.073	F Failed to start core
    2021-08-06 18:04:08.173	I Deinitializing daemon objects...

with no actual error displayed as to why it failed to start.  With this
change you get:

    2021-08-06 18:04:14.004	F Please specify an IPv4 public address which the service node & storage server is accessible from with: '--service-node-public-ip <ip address>'
    2021-08-06 18:04:14.004	F IMPORTANT: One or more required service node-related configuration settings/options were omitted or invalid; please fix them and restart oxend.
    2021-08-06 18:04:14.004	F Failed to start core
2021-11-01 14:53:10 -04:00
Jason Rhinelander
de3f9a103f Move json binary handling code; teach it about vectors
- Moves the json binary handling code into new rpc_binary.h

- Adds support for storing/loading vectors of binary types
  (crypto::hash, etc.), and vectors of strings.
2021-11-01 14:53:10 -04:00
Jason Rhinelander
e8cbf6352d using nlohmann::json
Simplify things by not require full qualification everywhere.
2021-11-01 14:53:10 -04:00
Jason Rhinelander
1ea9791ca4 Add a version of tools::join that also transforms
Also add a missing <cassert> header needed for elsewhere in the header.
2021-11-01 14:53:10 -04:00
Jason Rhinelander
ea8292703f print_money: allow trimming unused decimals
This lets the caller get a value of 15000 or 4.56 instead of
15000.000000000 or 4.560000000 by passing a true second argument to
print_money().
2021-11-01 14:53:10 -04:00
Jason Rhinelander
5330069421 Remove dumb configurable decimal place support
*Nothing* calls `print_money()` with a decimal place value, for good
reason: it's a stupid thing to want to do.  Similarly nothing gives a
decimal places argument to `get_unit()`.

This removes the dumb unused decimal place argument.
2021-11-01 14:53:10 -04:00
Jason Rhinelander
a3547c8467 Simplify participation structures
- make checkpoint and pulse participation distinct types
- make the container act a little more like an stl container
- replace check_participation() with a failures() method that just
  returns the number of failures (so the caller can decide whether that
  is bad or not).
- As a consequence of the above, failures now trigger on N+1 failures,
  while previously they required N+1 failures *and* 8 total responses.
  There is no need to wait for all 8 as far as I can see since we will
  be failing no matter what the next 3 responses are.
- Removed the serialization code (hopefully this doesn't break
  everything outside the RPC code).
- Simplify/DRY the code that records participation
2021-11-01 14:53:10 -04:00
Jason Rhinelander
41c01efadf Add int-to-string converter (from storage server) 2021-11-01 14:53:10 -04:00
Sean Darcy
c7ab69b4f9 Convert RPC functions to new format:
MINING_STATUS, GET_TRANSACTION_POOL_HASHES, GET_TRANSACTION_POOL_BACKLOG, GET_TRANSACTION_POOL_STATS, GET_CONNECTIONS
2021-11-01 14:53:10 -04:00
Sean Darcy
434ca35c7e New RPC formats for STOP_MINING, SAVE_BC, STOP_DAEMON, GETBLOCKCOUNT 2021-11-01 14:53:10 -04:00
Sean Darcy
ced01dfb16 START_MINING and GET_OUTPUTS to new RPC format 2021-11-01 14:53:09 -04:00
Sean Darcy
b3f125bfdf GET_NET_STATS - To new RPC format 2021-11-01 14:53:09 -04:00
Jason Rhinelander
d350e2a664 Remove old code 2021-11-01 14:53:09 -04:00
Jason Rhinelander
4a8023ade6 Make RPC more flexible & add bt-encoding
The bt-encoding, in particular, is aimed at more efficient wallet3 rpc
interactions with a daemon.

This removes the rigidity from RPC request responses, making it easier
to return useful constructs that the current epee code doesn't support
(like lists of lists), and also adds support for bt-encoded RPC requests
(intended to eventually replace the ".bin" endpoints with Monero NIH
binary encoding).

Instead endpoints now set a nlohmann::json response object with whatever
arbitrary values they want without having to add a bunch of limited epee
serialization macro hell.

So, for example, to set the value "foo" to 20, you use:

    rpc.response["height"] = 20;

Binary values (for things like hashes) are handled by going through a proxy object:

    rpc.response_hex["hash"] = hash; // hash is a crypto::hash

which, for json, is equivalent to:

    rpc.response["hash"] = tools::type_to_hex(hash);

but when the response is to be bt-encoded it leaves it as binary for
more efficient transfers.

There is also a `response_b64` that converts json-destined values to
base64 instead of hex (and again leaves bt-destined values as binary).

Parsing of incoming requests now moves to a new
core_rpc_server_command_parser file, which does free-form parsing from a
nlohmann::json or bt_dict_consumer, and has various templated helper
functions to make this easier.

This is preliminary: this builds the basic infrastucture for handling
requests, and converts three endpoints:

- get_info
- get_height
- ons_resolve

Currently only `make daemon` builds (there is code in the wallet that
hasn't been updated yet), and the other RPC interfaces are disabled by
this commit (until they get converted to the new style).
2021-11-01 14:53:05 -04:00
Jason Rhinelander
d66e72fcea friendly_duration: show 0s for exactly 0 rather than 0ns 2021-11-01 14:19:27 -04:00
Jason Rhinelander
0ff6d3f491 Minor api improvement: return pair instead of two output params 2021-11-01 14:19:27 -04:00
Jason Rhinelander
c3c90a48f7 Code simplification 2021-11-01 14:19:27 -04:00
Jason Rhinelander
e312e30f12
Merge pull request #1494 from tewinget/sqlitedb
fix bugs in initial implementation of sqlitedb abstraction
2021-10-19 12:34:33 -03:00
Jason Rhinelander
5a73ec194a
Merge pull request #1495 from jagerman/remove-uselessness
Remove useless atomic calculations
2021-10-19 12:33:35 -03:00
Jason Rhinelander
340c99b524 Set up aes flags properly for cn-turtle compilation 2021-10-18 22:23:01 -03:00
Thomas Winget
a255889286 minor fixups in sqlitedb wrapper 2021-10-16 22:32:32 -04:00
Jason Rhinelander
ce6050c3d9 And no optimization for apple, too.
It also doesn't compile, because who knows BS with how apple handles
aes, and I hate this cn turtle code.
2021-10-15 20:57:18 -03:00
Jason Rhinelander
290984257f Disable broken intel-optimized build on i386
It doesn't compiler and I can't be arsed to fix it; if someone really
needs 32-bit they probably don't care about performance of turtle
hashing in oxend anyway, so the portable one is fine.
2021-10-15 20:54:12 -03:00
Jason Rhinelander
45e03a4575 Disable LTO for cncrypto on gcc-11
The cn_turtle_hash code makes GCC misoptimization in multiple ways under
LTO, so just force it off for GCC-11 (Hopefully the bugginess gets fixed
by GCC-12).
2021-10-15 20:42:11 -03:00
Jason Rhinelander
62ef3f64b1 Remove useless functions 2021-10-15 19:42:19 -03:00
Thomas Winget
e12f8dc518 fix some bugs in sqlitedb abstraction.
also set BUSY timeout for internal sqlite lib to 5 seconds
2021-10-15 17:21:44 -04:00
Jason Rhinelander
ee62a23550 Fix x86 arch detection
We were *always* falling back to the software implementation because
this define isn't set here.
2021-10-15 15:19:49 -03:00
Jason Rhinelander
7a70914e15 Split up turtle arch branches
Try to make the code a little more followable.
2021-10-15 14:52:23 -03:00
Jason Rhinelander
69ad37daee Remove useless atomic calculations
These are useless and not used anywhere.  Hurray!
2021-10-15 12:46:32 -03:00
Thomas Winget
640adb7aca fix missing include in sqlitedb
also add "maybe get" function to Database class
2021-10-13 20:29:35 -04:00
Thomas Winget
73e787a9ec sqlitecpp cmake nonsense 2021-10-06 19:36:46 -04:00
Thomas Winget
df07ae3e40 require sqlite foreign_key ON 2021-10-04 22:26:33 -04:00
Thomas Winget
435c189a9b sqlitecpp 2021-10-04 20:33:24 -04:00
Sean
578c1c235f
Merge pull request #1484 from jagerman/disable-mms-light
Disable MMS and light wallet support by default
2021-09-06 13:45:16 +10:00
Jason Rhinelander
03add6ebe3 Move operator== into namespace
Argument-dependent lookup will find these if in the same namespace as
Proof itself.
2021-09-03 02:17:07 -03:00
Jason Rhinelander
e3999ccb26 Rearrange proof/service node forward declarations to help gcc-8
gcc-8's unique_ptr apparently isn't okay with a forward declared type.
This reverses how proof and service_node_list include/fwd declare each
other to get around it.
2021-09-03 02:15:37 -03:00
Jason Rhinelander
9782155c81
Merge pull request #1481 from jagerman/drop-integration-tests
Drop unmaintained integration test code
2021-09-03 00:28:49 -03:00
Jason Rhinelander
e2a7391fb3
Merge pull request #1482 from darcys22/initialise-uptime-proof-info
Initialise uptime proof info
2021-09-03 00:28:30 -03:00
Jason Rhinelander
5759c41291
Merge pull request #1480 from jagerman/logger-rebrand-updates
Fix missing logger rebrand updates
2021-09-03 00:26:34 -03:00
Jason Rhinelander
eefd42f93f Add fmtlib::fmt backport of C++20 std::format
Because I'm tired of waiting for std::format.
2021-09-02 17:25:13 -03:00
Jason Rhinelander
ff8e2dd09d Disable MMS and light wallet support by default
This aren't used or supported by anyone in Oxen land (as far as I know)
and have not been maintained since the beginning, so hide them behind
disabled-by-default cmake options.

This cuts about 10% off the wallet2 compilation time and 3% off the
simplewallet compilation time, plus gets rid of the page of mms crap in
the cli wallet's "help" screen.
2021-09-02 14:03:19 -03:00
Sean Darcy
36a86af3fc more uptime proof initialisations 2021-08-20 15:42:24 +10:00
Sean Darcy
302259ce69 Uninitialised variables 2021-08-20 15:04:07 +10:00
Jason Rhinelander
5f1bd2f1e4 Drop integration test code.
This code is bitrotting, doesn't compile, and isn't being maintained
anymore.

The integration test suite was an interesting idea, in early Loki days,
but is no longer being maintained and is quite cumbersome to run (for
instance, it is not possible to run it via CI because it depends on
xterm to actually run).  The code to actually run it (in doy-lee's
loki-integration-testing repository) is also a large burden of "janky"
code that isn't worth maintaining.

Remove this from the code; if someone wants to pick it back up in the
future reverting this commit shouldn't be too difficult (though I'd
suggest that a much better approach to integration testing would be to
run different daemons/wallets via rpc commands, as the network-tests do,
rather than trying to feed stdin and parse stdout from running
individual oxends/wallets).
2021-08-19 16:42:15 -03:00