Commit graph

9896 commits

Author SHA1 Message Date
Jason Rhinelander
965e4f05eb
Documentation generation using docsify
- Adds script to extract markdown from the rpc comments to build a docs
  page suitable for docsify.
- Split up daemon rpc docs logically into sections
- Reformat (mostly rewrapping) various docs entries
- Add examples, many of which are live-fetched at documentation
  generation time.
- Mark various required options as `required{...}` so that they produce
  an error if not provided in a request.
2023-01-11 14:27:26 -04:00
Jason Rhinelander
6122164735
Change ["key"] -> .at("key")
So that we throw when the key doesn't exist.
2023-01-10 23:14:34 -04:00
Jason Rhinelander
bb111c3959
Fix get_service_nodes rpc node proxy query
Also fixes the wrong error message being shown when it fails.
2023-01-10 22:43:31 -04:00
Jason Rhinelander
78192856ab
Small string_view optimizations
Using string_view is slightly faster as it lets us get a view directly
into nlohmann's string value, rather than making an intermediate copy
into a temporary string.
2023-01-10 22:26:28 -04:00
Jason Rhinelander
64fbdb8c7f
Fix locked_contributions not included in cached SN list
Not having it broke wallet2 that was expecting it to exist; this fixes
it and also changes the `[]` usage to `.at()` to throw on non-existent
keys.
2023-01-10 22:25:03 -04:00
Jason Rhinelander
31fff4cb0a
Fix get_service_node_blacklisted_key_images
- the endpoint wasn't publicly accessible
- fixed typo in wallet2 miscalling it
2023-01-10 22:23:39 -04:00
Jason Rhinelander
c64c1c329a
Fix get_accrued_batched_earnings missing "status": "OK"
Not having it broke wallet2.
2023-01-10 22:22:44 -04:00
Jason Rhinelander
cf9f1c0823
Bump minor network revision for testnet 2023-01-10 16:12:40 -04:00
Jason Rhinelander
8b54c010d9
Fix error when pow_hash missing 2022-12-21 19:08:21 -04:00
Jason Rhinelander
10a45781cd
Fix coinbase_payouts in block rpc output, continued 2022-12-21 19:01:08 -04:00
Jason Rhinelander
3963bb8f5b
Move nlohmann_json dependency to 'common' target 2022-12-21 17:04:07 -04:00
Jason Rhinelander
883658fbda
Fix segfault in get_block
Only `.reserve()` was called here, so assigning into `.begin()` isn't
valid (we either need to `.resize()`, or use `std::back_inserter`).
2022-12-21 15:21:59 -04:00
Jason Rhinelander
165b545975
Remove json_archiver debugging 2022-12-21 15:14:32 -04:00
Jason Rhinelander
9496a3b6e9
Fix coinbase_payouts in block rpc output 2022-12-21 14:31:16 -04:00
Jason Rhinelander
eaf8bd39f2
Redo json_archiver in nlohmann::json
Manually creating json the way we were doing is dumb.  This redoes it
using nlohmann::json, which in turn lets us merge the serialized json
into rpc results.
2022-12-20 21:34:04 -04:00
Jason Rhinelander
fb3b414118
Re-add get_transaction_pool
We removed it at some point during the RPC refactor because its purpose
is replaced by get_transactions' new `memory_pool` parameter, but we
also need a wrapper to do that for older clients making direct
`get_transaction_pool` rpc requests.
2022-12-19 16:41:18 -04:00
Jason Rhinelander
2a8b0733be
Log levels: case-insensitive levels; add abbreviations 2022-12-19 16:41:18 -04:00
Jason Rhinelander
2b3a71527b
Debug log bad request body 2022-12-19 16:00:22 -04:00
Jason Rhinelander
402aa450c9
Rename LMQ -> OMQ in code
(This doesn't rename the `--lmq-...` cli options because that would
break backwards compatibility.)
2022-12-19 13:22:02 -04:00
Jason Rhinelander
d3d0a4460d
Fix string construction when no pow_hash
This was throwing when there was no pow_hash (can't construct a string
from a nullptr).

It doesn't make much sense to stick an empty string there at all,
though, so this amends it to only set the `pow_hash` key when there is
something to set (rather than setting it to an empty string).
2022-12-19 12:30:09 -04:00
Jason Rhinelander
af37b44427
Reformatting/cleanup 2022-12-19 12:27:10 -04:00
Jason Rhinelander
b128f373d3
More RPC fixes 2022-12-16 22:42:20 -04:00
Jason Rhinelander
3179a97b32
Make get_sn_state_changes admin-only
This call can make the node do a full tx scan for state change
transactions, which can be expensive.  It is currently only used in the
oxend CLI interface, so seems fine to make admin-restricted.
2022-12-16 19:22:21 -04:00
Jason Rhinelander
2c7a4ab841
Activate various RPC endpoints
These all appear to be converted, but weren't moved into the active
endpoint type list.

Includes a rename of SETBANS/GETBANS to SET_BANS/GET_BANS
2022-12-16 16:10:14 -04:00
Jason Rhinelander
47107e6e37
Fix out-of-order argument parsing
Incoming RPC arguments *must* be parsed in alphabetical order because
our parsing is consuming a bt_dict in order, when the parameters are
bt-encoded.
2022-12-16 16:07:23 -04:00
Jason Rhinelander
f71b848b6a
:sort 2022-12-16 15:55:38 -04:00
Jason Rhinelander
5bd678010f
Fix binary local pubkey setting
The ed/x pubkeys were not being properly set via the json proxy, and so
were throwing when use.  (This only triggered when calling on a service
node as the non-service-node path did the wrong thing).
2022-12-16 14:51:46 -04:00
Jason Rhinelander
7f66923ab4
Respond with "id" as-provided
It isn't necessarily a string; for JSON RPC we're required to return it
back as whatever it came in as.
2022-12-15 20:45:48 -04:00
Jason Rhinelander
7bd70b3f80
Fix get_service_keys/get_service_privkeys arg parsing 2022-12-15 20:42:56 -04:00
Jason Rhinelander
81ef0a4924
:sort 2022-12-15 20:42:56 -04:00
Jason Rhinelander
64b39b33d8
Activate get_service_keys/get_service_privkeys rpc endpoints 2022-12-15 20:42:56 -04:00
Jason Rhinelander
40898c1670 Add support for skipping empty string fields
These should never have been specified, but are in wallet2 and backwards
compat requires us to deal with this.
2022-12-06 08:10:38 +11:00
Sean Darcy
4c3588ad4d enable ONS_OWNERS_TO_NAMES 2022-12-06 08:10:38 +11:00
Sean Darcy
e8962da52c move blacklisted key images to completed 2022-12-06 08:10:38 +11:00
Sean Darcy
172093fbbf wallet3 cli
This introduces wallet3 cli to the codebase, build using python and
wraps the c++ wallet3 core using pybind.
2022-12-06 08:10:31 +11:00
Sean
f8307ee34e
Merge pull request #1598 from darcys22/wallet3-test-fixes
fixes wallet3 test segfault, first test passing
2022-10-26 09:51:39 +11:00
Sean Darcy
c2a33cf188 Creates archiving for reward batching
This adds a new table to the batching schema to copy the accrued
balances every so often. This means that we can jump back to a
previous point without popping blocks.

The archiving is triggered in sql every 100 blocks and added to the
archive table, then pruned from the archive table at a later time to
ensure the size is kept small. Rebuilding 100 blocks is pretty
reasonable and should be less than 10s.

For longer distance pop_blocks and blockchain detached every 10k blocks
is kept in the archiving table. This takes longer to rebuild but is
better than rebuilding from scratch.

The blockchain detached function is also added to our regular blockchain
detached hooks so that it gets called every time the blockchain
detaches. Which appears to have caused some issues previously when some
of the modules would detach but batching would be stuck in an advanced
state.
2022-10-25 10:56:49 +11:00
Jason Rhinelander
9792b637fb
Remove broken formattable::to_string(T) support
- This was infinitely recursing because it was picking up fmt::to_string
  (on macos) rather than formattable::to_string, which led to infinite
  recursion and then a crash.

- Add a custom ghc::filesystem::path formatter (because it just seems
  wrong to add a function into ghc::filesystem) instead of using the
  to_string formatting.  Same for boost formatting crap in simplewallet.

- Move the formatter structs into the `formattable` namespace so that
  this sort of recursive error can't happen again.
2022-10-18 11:23:28 -03:00
Jason Rhinelander
6aa9db9538
Overhaul and fix crypto::{public_key,ec_point,etc.} types
- Remove implicit `operator bool` from ec_point/public_key/etc. which
  was causing all sorts of implicit conversion mess and bugs.
- Change ec_point/public_key/etc. to use a `std::array<unsigned char,
  32>` (via a base type) rather than a C-array of char that has to be
  reinterpret_cast<>'ed all over the place.
- Add methods to ec_point/public_key/etc. that make it work more like a
  container of bytes (`.data()`, `.size()`, `operator[]`, `begin()`,
  `end()`).
- Make a generic `crypto::null<T>` that is a constexpr all-0 `T`, rather
  than the mishmash `crypto::null_hash`, crypto::null_pkey,
  crypto:#️⃣:null(), and so on.
- Replace three metric tons of `crypto::hash blahblah =
  crypto::null_hash;` with the much simpler `crypto::hash blahblah{};`,
  because there's no need to make a copy of a null hash in all these
  cases.  (Likewise for a few other null_whatevers).
- Remove a whole bunch of `if (blahblah == crypto::null_hash)` and `if
  (blahblah != crypto::null_hash)` with the more concise `if
  (!blahblah)` and `if (blahblah)` (which are fine via the newly
  *explicit* bool conversion operators).
- `crypto::signature` becomes a 64-byte container (as above) but with
  `c()` and `r()` to get the c() and r() data pointers.  (Previously
  `.c` and `.r` were `ec_scalar`s).
- Delete with great prejudice CRYPTO_MAKE_COMPARABLE and
  CRYPTO_MAKE_HASHABLE and all the other utter trash in
  `crypto/generic-ops.h`.
- De-inline functions in very common crypto/*.h files so that they don't
  have to get compiled 300 times.
- Remove the disgusting include-a-C-header-inside-a-C++-namespace
  garbage from some crypto headers trying to be both a C and *different*
  C++ header at once.
- Remove the toxic, disgusting, shameful `operator&` on ec_scalar, etc.
  that replace `&x` with `reinterpret_cast x into an unsigned char*`.
  This was pure toxic waste.
- changed some `<<` outputs to fmt
- Random other small changes encountered while fixing everything that
  cascaded out of the above changes.
2022-10-17 22:20:54 -03:00
Jason Rhinelander
5b4a60b412
Update variant header from oxenmq to oxenc
The latest oxenmq dropped the long-deprecated compat header.
2022-10-17 13:45:24 -03:00
Jason Rhinelander
43343f3b90
Fix ec_point sub-types not being formattable; fix crappy test code
Fixed a bug where ec_point subclasses were not getting formatted via
to_string(ec_point).

Also fixed the crappy test code that was incapable of reporting the
actual test failure.
2022-10-17 13:45:24 -03:00
Jason Rhinelander
cd38683619
Add missing header 2022-10-17 13:45:24 -03:00
Jason Rhinelander
661a0c089c
Fix slurp_file potentially reading in non-binary mode
I'm not sure if this gets used on Windows, but if it does, this could do
windows linefeed conversions, which it isn't supposed to.
2022-10-17 13:45:24 -03:00
Jason Rhinelander
463590ad5c
Eliminate most << output operators
Replace (nearly) everything with fmt formatting.  Some crap in wallet2
remains that I'm not going to bother with.
2022-10-17 13:45:24 -03:00
Jason Rhinelander
c4f4bfc87e
Fix deprecated oxenmq::* usage
Some of this is just obsolete, some slipped in recent feature branches.
2022-10-17 13:45:24 -03:00
Jason Rhinelander
0cf55f3945
Add garbage to make it work on a garbage OS
- Don't touch <fmt/std.h> because it touches std::filesystem which makes
  macOS throw a hissy fit and refuse to compile.
- int_to_string is broken on macOS because it uses std::to_chars which
  makes macos throw a hissy fit like a cranky old female cat seeing a
  kitten if it sees it.
- wallet3 was using std::filesystem and std::visit, both of which make
  macos throw a hissy fit.  (There is a pattern here).  Apply the dumb
  fs::path and var::visit workarounds needed to appease this garbage OS.
- use var::get (from oxenc/variant.h) instead of std::get because, oh
  yeah, we need to support a garbage OS that Apple themselves don't even
  properly support.  Yay!
2022-10-17 13:45:24 -03:00
Jason Rhinelander
6b5a05d3f6
Remove unused variable 2022-10-17 13:45:23 -03:00
Jason Rhinelander
9e4e4976fa
Remove unnecessary move
gcc is (rightly) warning: the value here is already a temporary, so
moving doesn't do anything useful (but also prevents copy elision).
2022-10-17 13:45:23 -03:00
Jason Rhinelander
dbf6c2dd45
Make ghc::filesystem fmt-formattable 2022-10-17 13:45:23 -03:00
Sean Darcy
23dc129dc7
silence unused variable error for if constexpr 2022-10-17 13:45:22 -03:00
Sean Darcy
9d093f1c5a
update windows builds 2022-10-17 13:45:22 -03:00
Sean Darcy
9461cfc674
wallet_api updated to use new json_rpc and logging 2022-10-17 13:45:22 -03:00
Jason Rhinelander
4f701feaea
Remove nasty print-on-destruction wrapper; more fmt 2022-10-17 13:45:22 -03:00
Jason Rhinelander
cec9d5db1c
Simplify arg printing; move <iostream> out of header 2022-10-17 13:45:22 -03:00
Jason Rhinelander
a91774f2fc
Add formattable helper header
This allows types to more easily opt-in to being to_string formattable
without having to do a full fmt specialization.
2022-10-17 13:45:22 -03:00
Jason Rhinelander
5806fd7825
Change simple fmt::format calls to new "..."_format(...) 2022-10-17 13:45:22 -03:00
Jason Rhinelander
085e9ba963
Unwrap fmt::format with a style argument
oxen::logging handles a style argument now (it is also safer e.g. if the
final formatted string itself intentionally contains {}).
2022-10-17 13:41:45 -03:00
Jason Rhinelander
c9934b9f5f
Change most oxen::log::whatever to log::whatever
oxen::log::info(...), etc. are a bit too verbose; this simplifies them
to just `log::info(...)`, etc. by aliasing the `oxen::log` namespace
into most of the common namespaces we use in core.

This result is usage that is shorter but also reads better:

    oxen::log::info(logcat, "blah: {}", 42);

    log::info(logcat, "blah: {}", 42);
2022-10-17 13:41:43 -03:00
Jason Rhinelander
980237a95b
Fix missing oxen logger include when no HIDAPI
oxen_logger was getting included indirectly via something
device_ledger.hpp includes, but that doesn't get included at all if
HIDAPI headers weren't found.
2022-10-17 13:41:12 -03:00
Sean Darcy
d7992b5940
Logging Refactor
This replaces the current epee logging system with our oxen::log
library. It replaces the easylogging library with spdlog, removes the
macros and replaces with functions and standardises how we call the
logs.
2022-10-17 13:41:10 -03:00
Jason Rhinelander
0f85299e1f
Merge pull request #1601 from darcys22/virtual-destructor-wallet3
create virtual destructor on wallet class and define destructor on child
2022-10-17 13:39:37 -03:00
Jason Rhinelander
7427fb051b
Merge pull request #1607 from jagerman/aux-ping-errors-dev
Allow lokinet/ss to send an error ping
2022-10-17 13:39:03 -03:00
Jason Rhinelander
867b87dc42
Merge pull request #1604 from jagerman/sn-self-info-dev
Put current info for ourself in service node list
2022-10-17 13:38:41 -03:00
Sean Darcy
31b45ce96c
change naked throws to throw exceptions 2022-10-17 13:37:53 -03:00
Sean Darcy
c33b024037 create virtual destructor on wallet class and define destructor on child 2022-10-17 07:13:43 +11:00
Jason Rhinelander
46fd75fa90 Allow lokinet/ss to send an error ping
This lets lokinet or SS signal to oxend that something is known to be
wrong, and so oxend should not send a proof so that, hopefully, the
problem gets noticed and fixed.
2022-10-14 21:42:40 -03:00
Jason Rhinelander
06e3713dfc
Put current info for ourself in service node list
Currently we're putting the last-uptime-received data in the service
node list info, even when we have more updated info that hasn't yet been
accepted by the network.

This causes problems for lokinet, in particular, because it ignores any
SNs in the list without ed25519 pubkeys, including itself, which can
result in lokinet thinking it is deregistered for a while after the SN
becomes registered.

This updates `get_service_nodes` to always include current info (rather
than latest proof info) for itself when querying a service node.
2022-10-14 14:12:49 -03:00
Sean Darcy
75410dd675 transaction fees updated for blink, mock decoy selector was not being used previously 2022-09-29 16:47:47 +10:00
Sean Darcy
6146a88acd fixes wallet3 test segfault, first test passing 2022-09-28 16:52:29 +10:00
Jason Rhinelander
e1c3d95dad
Fix get_accrued_batched_earnings being non-public 2022-09-22 12:48:21 -03:00
Sean Darcy
6e6f3bf5b4 tests rely on false returned here 2022-09-20 08:52:49 +10:00
Thomas Winget
4b56bf4e4b all tests building now, not all passing 2022-09-07 19:34:52 -04:00
Sean Darcy
230a8185aa start of compiler errors 2022-09-02 15:38:21 -04:00
Jason Rhinelander
1250a11452
Fix automining by default
wallet3 had changes to miner.cpp frankenstein merge-resolved, which was
resulting in various fields in miner.h being uninitialized.  This fixes
the conflict (and fixes the auto-mine-at-startup from the wallet3 branch
merge).

This reapplies all of the changes from
b9755af4b4 to fix it.
2022-09-02 15:16:33 -03:00
Jason Rhinelander
2bd874e0b5
Merge remote-tracking branch 'tewinget/wallet3' into dev 2022-09-01 18:56:14 -03:00
Jason Rhinelander
f1ee71a1d4
Merge pull request #1591 from darcys22/atomic-sn-reg-cmd
Atomic values for the get_service_node_registration_cmd endpoint
2022-08-30 23:25:32 -03:00
Sean Darcy
872210608b use new basis points function in rpc_command_executor 2022-08-31 10:02:46 +10:00
Jason Rhinelander
3662cb21fb Remove pre-HF19 registration commands from oxend 2022-08-31 10:01:21 +10:00
Thomas Winget
ebbc0e5644 wallet3 changes and demo fixup
demo is vaguely presentable now.  syncing and basic tx send work.
change output was being sent to void (change address was unimplemented)
but that works now as well.
2022-08-30 19:58:51 -04:00
Sean Darcy
a9c83bd0b2 Atomic values for the get_service_node_registration_cmd endpoint 2022-08-31 09:58:15 +10:00
Jason Rhinelander
c3a00f57a2
Oxen 10.2 version bumps & mandatory upgrade date
- 10.2.0 oxend version
- keep lokinet at 0.9.9
- require storage server 2.4.0
- schedule 10.2.0 mandatory date as 14 September 00:00 UTC (10am in
  Australia Eastern Time; 8pm (on the 13th) in US Eastern time.
2022-08-30 19:06:58 -03:00
Thomas Winget
e3726f520d wallet3 demo code and syncing bugfix 2022-08-25 17:48:08 -04:00
Jason Rhinelander
2f1e2557ff
Fix pulse not avoiding small contributor unlocks 2022-08-25 11:24:26 -03:00
Thomas Winget
d4b6f967fd Merge most recent dev and wallet3 branches
Fixes a few merge conflicts, several compilation errors, and
some behavioral incorrectness.  Still a few bugs with wallet3
but as far as I can tell wallet2 and daemon etc. should be working
correctly.
2022-08-22 19:25:49 -04:00
Jason Rhinelander
d955607306
Merge pull request #1588 from darcys22/create-hwdev-file-by-default-for-hw-devices
Create HWDEV file by default for hardware wallets
2022-08-22 17:27:11 -03:00
Jason Rhinelander
5f338c2492
Merge pull request #1585 from jagerman/fix-block-notify-race
Fix block notify race condition, and drain some of the swamp
2022-08-22 17:25:21 -03:00
Jason Rhinelander
e730b2fcbb
Merge pull request #1583 from darcys22/small-contributor-atomic-amount-fix
fix for small contributor
2022-08-22 17:24:04 -03:00
Sean Darcy
9ae474eb76 Create HWDEV file by default for hardware wallets
This modifies the current behaviour from only creating the hwdev.txt
file, when a new wallet is created if content is specified as an
argument to always creating the file. This file is used by the GUI
wallet to detect if the device is a hardware device and the CLI wallet
behaviour is kept the same to keep behavious consistent between CLI and
RPC wallets.
2022-08-19 15:28:17 +10:00
Sean Darcy
d6b8b2d036 fix for small contributor unlocks
The checking if the service node contributers are small miscalculated
atomic oxen. This brings in the correct figure for HF20 and in the
interim brings measures into the wallet and blink to prevent small
contributors from being to unlock in HF19.
2022-08-11 15:09:04 +10:00
Sean Darcy
415098f9e9 Wallet RPC calls for Get Balance, Get Address and Export Keys 2022-08-01 17:21:25 -04:00
Sean Darcy
cf6c3724cb Config Struct for wallet3
This brings a simple config structure into the wallet3 that all of the
wallet submodules should be able to access. The first module included in
this PR is the daemon comms module which has its own substruct on the
config and the submodule is now constructed with a reference to the
config.
2022-08-01 17:21:01 -04:00
Sean Darcy
90433d0b71 rpc for get_version 2022-08-01 17:16:29 -04:00
Thomas Winget
9a7f7d6704 wallet3 send/receive works, more or less 2022-08-01 17:12:24 -04:00
Jason Rhinelander
0bc3c2f880
Rewrite trash tools::Notify
This was nasty.

Remove --block-rate-notify entirely; it's nearly useless on Oxen.

block and reorg notify remain, but now use a post-block hook rather than
shoving toxic Notify crap into cryptonote_core headers.
2022-07-27 17:25:48 -03:00
Jason Rhinelander
edbce1bf8c
Add and use post-block-added hooks; fix block notify race
Add hooks that are called *after* a block is successfully added to the
blockchain.

This also fixes a race condition with OMQ notify.block subscriptions:
because the notification was firing during (instead of after) block
addition, the lokinet/storage server receiving the notify would race to
fetch the block info, but that request could happen *before* the block
addition is finished, ending up with lokinet/SS sometimes getting a
stale block.
2022-07-27 17:25:12 -03:00
Jason Rhinelander
59f90dcf75
Rename block_added hook to block_add
This hook *isn't* called after a block is added, but rather it is part
of the block addition process and can abort the whole thing by raising
an exception (or returning false, prior to this PR).

This is unintuitive and causes bugs if using it as a "block has been
added" hook.
2022-07-27 15:20:35 -03:00
Jason Rhinelander
9c9552380d
Make hooks use exceptions on error rather than bool returns
bool returns suck in general, but in most cases here they are also a
pain in the ass because *each* place that returns false is also issuing
a log statement.  If only there were a way to return error information
to the common caller to have the common caller handle it... oh wait,
there is!
2022-07-27 15:10:52 -03:00
Jason Rhinelander
f382c1971e
Replace hook virtual calls with std::function
- Instead of inheriting from a pure virtual base class, we know just use
  lambdas for hook callbacks.
- The signature of hooks also changes to take an info struct rather than
  a list of parameters, so that you don't have to change everything to
  ignore unwanted parameters if someone adds something to a hook.
2022-07-27 14:20:29 -03:00
Thomas Winget
0b17bb9adb fix build after merge
still need to make tests build and fix up some behavior
correctness around RPC (and maybe other areas)
2022-07-25 22:22:49 -04:00
Jason Rhinelander
76908d9d5d
Add missing header
This was previously being included via oaes_lib.h.
2022-07-22 14:18:24 -03:00