Commit graph

289 commits

Author SHA1 Message Date
Doyle 71447e7877
Add flag to recalculate difficulty per block (#598)
* Beging adding functions to recalculate difficulty

* Add command line args to utility for recalculating difficulty

* Exception safety for recalculating difficulty

* Update help text for recalc difficulty

* Add recalc flag on the daemon

* Make context be const, signify intent for var++ to 1
2019-05-12 09:40:57 +10:00
Doyle 0c014138a7
Don't attempt an a db append for blacklisted outputs if empty (#450) 2019-02-27 12:36:15 +11:00
moneromooo-monero 0b0caba86a db_lmdb: fix missing mdb_dbi_close in migration
Fixed by hyc
2019-02-21 15:49:58 +11:00
moneromooo-monero 617eb6e74b Fix v3/v4 db conversion 2019-02-21 15:47:26 +11:00
moneromooo-monero 6cc35100f9 ArticMine's new block weight algorithm
This curbs runaway growth while still allowing substantial
spikes in block weight

Original specification from ArticMine:

here is the scaling proposal
Define: LongTermBlockWeight
Before fork:
LongTermBlockWeight = BlockWeight
At or after fork:
LongTermBlockWeight = min(BlockWeight, 1.4*LongTermEffectiveMedianBlockWeight)
Note: To avoid possible consensus issues over rounding the LongTermBlockWeight for a given block should be calculated to the nearest byte, and stored as a integer in the block itself. The stored LongTermBlockWeight is then used for future calculations of the LongTermEffectiveMedianBlockWeight and not recalculated each time.
Define:   LongTermEffectiveMedianBlockWeight
LongTermEffectiveMedianBlockWeight = max(300000, MedianOverPrevious100000Blocks(LongTermBlockWeight))
Change Definition of EffectiveMedianBlockWeight
From (current definition)
EffectiveMedianBlockWeight  = max(300000, MedianOverPrevious100Blocks(BlockWeight))
To (proposed definition)
EffectiveMedianBlockWeight  = min(max(300000, MedianOverPrevious100Blocks(BlockWeight)), 50*LongTermEffectiveMedianBlockWeight)
Notes:
1) There are no other changes to the existing penalty formula, median calculation, fees etc.
2) There is the requirement to store the LongTermBlockWeight of a block unencrypted in the block itself. This  is to avoid possible consensus issues over rounding and also to prevent the calculations from becoming unwieldy as we move away from the fork.
3) When the  EffectiveMedianBlockWeight cap is reached it is still possible to mine blocks up to 2x the EffectiveMedianBlockWeight by paying the corresponding penalty.
2019-02-20 18:57:58 +11:00
Doyle 43436b081d
Infinite Staking Part 2 (#406)
* Cleanup and undoing some protocol breakages

* Simplify expiration of nodes

* Request unlock schedules entire node for expiration

* Fix off by one in expiring nodes

* Undo expiring code for pre v10 nodes

* Fix RPC returning register as unlock height and not checking 0

* Rename key image unlock height const

* Undo testnet hardfork debug changes

* Remove is_type for get_type, fix missing var rename

* Move serialisable data into public namespace

* Serialise tx types properly

* Fix typo in no service node known msg

* Code review

* Fix == to >= on serialising tx type

* Code review 2

* Fix tests and key image unlock

* Add command to print locked key images

* Update ui to display lock stakes, query in print cmd blacklist

* Modify print stakes to be less slow

* Remove autostaking code

* Refactor staking into sweep functions

It appears staking was derived off stake_main written separately at
implementation at the beginning. This merges them back into a common
code path, after removing autostake there's only some minor differences.

It also makes sure that any changes to sweeping upstream are going to be
considered in the staking process which we want.

* Display unlock height for stakes

* Begin creating output blacklist

* Make blacklist output a migration step

* Implement get_output_blacklist for lmdb

* In wallet output selection ignore blacklisted outputs

* Apply blacklisted outputs to output selection

* Fix broken tests, switch key image unlock

* Fix broken unit_tests

* Begin change to limit locked key images to 4 globally

* Revamp prepare registration for new min contribution rules

* Fix up old back case in prepare registration

* Remove debug code

* Cleanup debug code and some unecessary changes

* Fix migration step on mainnet db

* Fix blacklist outputs for pre-existing DB's

* Remove irrelevant note

* Tweak scanning addresses for locked stakes

Since we only now allow contributions from the primary address we can
skip checking all subaddress + lookahead to speed up wallet scanning

* Define macro for SCNu64 for Mingw

* Fix failure on empty DB

* Add missing error msg, remove contributor from stake

* Improve staking messages

* Flush prompt to always display

* Return the msg from stake failure and fix stake parsing error

* Tweak fork rules for smaller bulletproofs

* Tweak pooled nodes minimum amounts

* Fix crash on exit, there's no need to store on destructor

Since all information about service nodes is derived from the blockchain
and we store state every time we receive a block, storing in the
destructor is redundant as there is no new information to store.

* Make prompt be consistent with CLI

* Check max number of key images from per user to node

* Implement error message on get_output_blacklist failure

* Remove resolved TODO's/comments

* Handle infinite staking in print_sn

* Atoi->strtol, fix prepare_registration, virtual override, stale msgs
2019-02-14 12:12:57 +11:00
doy-lee ac4d0469bf Merge commit '4a0e4c7' into LokiMergeUpstream 2019-01-30 12:04:20 +11:00
doy-lee d0bab8e6f5 Merge commit '13b1297' into LokiMergeUpstream 2019-01-29 20:11:17 +11:00
moneromooo-monero b750fb27b0
Pruning
The blockchain prunes seven eighths of prunable tx data.
This saves about two thirds of the blockchain size, while
keeping the node useful as a sync source for an eighth
of the blockchain.

No other data is currently pruned.

There are three ways to prune a blockchain:

- run monerod with --prune-blockchain
- run "prune_blockchain" in the monerod console
- run the monero-blockchain-prune utility

The first two will prune in place. Due to how LMDB works, this
will not reduce the blockchain size on disk. Instead, it will
mark parts of the file as free, so that future data will use
that free space, causing the file to not grow until free space
grows scarce.

The third way will create a second database, a pruned copy of
the original one. Since this is a new file, this one will be
smaller than the original one.

Once the database is pruned, it will stay pruned as it syncs.
That is, there is no need to use --prune-blockchain again, etc.
2019-01-22 20:30:51 +00:00
doy-lee 074ed7680f Merge commit '3e9bb96' into LokiMergeUpstream 2019-01-18 10:35:18 +11:00
doy-lee 9eefb869f8 Merge commit '69e8567' into LokiMergeUpstream 2019-01-17 15:57:30 +11:00
moneromooo-monero a6216d1ac2
blockchain_db: allow getting output keys without commitment
Since the commitment has to be calculated for non rct outputs,
it slows down a lot unnecessarily if we don't need it
2019-01-16 20:13:08 +00:00
Riccardo Spagni 9d57ee9ff6
Merge pull request #4984
008647d7 blockchain_db: speedup tx output gathering (moneromooo-monero)
2019-01-16 19:06:07 +02:00
moneroexamples ec68cf822f Make get_output_key method const
get_output_key method is commonly used when working with txs and their key images. Because the method is not const, passing blockchain object though const& or pointers to const is not possible in this context. This is especially problematic in external projects (e.g., projects in moneroexamples) that use monero C++ api to operate on the blockchain and txs.

Thus, having get_output_key method will simplify moving blockchain object around through const references and pointers to const objects.
2019-01-07 05:53:14 +08:00
luigi1111 69e8567c0e
Merge pull request #4946
6644b9b blockchain_db: remove a couple unused functions (moneromooo-monero)
ce594f5 blockchain_db: allocate known size vector only once (moneromooo-monero)
8332698 db_lmdb: inline check_open, it's trivial and called everywhere (moneromooo-monero)
5511563 db_lmdb: avoid pointless division (moneromooo-monero)
d1efe3d cryptonote: set tx hash on newly parsed txes when known (moneromooo-monero)
9cc68a2 tx_pool: add a few std::move where it can make a difference (moneromooo-monero)
2018-12-31 16:06:50 -06:00
doy-lee f5b17fc86e Merge commit '6bc0c7e' into LokiMergeUpstream 2018-12-19 12:25:48 +11:00
moneromooo-monero 008647d7eb
blockchain_db: speedup tx output gathering
We know all the data we'll want for getblocks.bin is contiguous
2018-12-18 11:55:21 +00:00
Riccardo Spagni d855f9bb92
Merge pull request #4901
a48f2dab blockchain_prune_known_spent_data: blackball file is now optional (moneromooo-monero)
17b45725 Outputs where all amounts are known spent can now be pruned (moneromooo-monero)
2018-12-12 11:57:31 +02:00
doy-lee d5beaee824 Merge commit 'a605c0d' into LokiMergeUpstream 2018-12-10 19:32:50 +11:00
doy-lee a07c5d9cba Merge commit '58ce16d' into LokiMergeUpstream 2018-12-10 14:54:27 +11:00
moneromooo-monero 5511563e92
db_lmdb: avoid pointless division 2018-12-05 21:13:24 +00:00
moneromooo-monero 833269834d
db_lmdb: inline check_open, it's trivial and called everywhere 2018-12-05 21:13:21 +00:00
moneromooo-monero 6644b9b7b7
blockchain_db: remove a couple unused functions 2018-12-05 21:13:13 +00:00
Riccardo Spagni a605c0da08
Merge pull request #4872
fc99f177 lmdb: fix gcc 7.3.0 'implicit-fallthrough' warning (xiphon)
2018-12-04 17:13:21 +02:00
Riccardo Spagni c00ac446fd
Merge pull request #4854
bd98e99c Removed a lot of unnecessary includes (Martijn Otto)
2018-12-04 17:08:42 +02:00
moneromooo-monero 17b45725af
Outputs where all amounts are known spent can now be pruned
Only for pre rct for obvious reasons.

Note: DO NOT use a known spent list which includes outputs
which are not known spent. If the list includes any output
that's just strongly thought to be spent, but not provably
so, you risk finding yourself unable to sync past the point
where that output is spent.

I estimate only 200 MB saved on current mainnet though,
unless the new blackballing rule unearths a good amount of
large-amount-set extra spent outs.
2018-11-27 15:45:42 +00:00
Riccardo Spagni 58ce16d4d9
Merge pull request #4821
fc98f7a0 rpc: speedup get_outs.bin (moneromooo-monero)
2018-11-26 21:51:27 +02:00
moneromooo-monero fc98f7a0a1
rpc: speedup get_outs.bin 2018-11-26 18:56:23 +00:00
Riccardo Spagni c4fd8ce97e
Merge pull request #4830
dffec258 db_lmdb: error out if the db needs migration in read only mode (moneromooo-monero)
2018-11-26 20:28:49 +02:00
doy-lee e36737a525 Merge commit 'd0c4123' into LokiMergeUpstream 2018-11-22 13:27:45 +11:00
xiphon fc99f177f3 lmdb: fix gcc 7.3.0 'implicit-fallthrough' warning 2018-11-19 22:36:10 +00:00
Martijn Otto bd98e99c80
Removed a lot of unnecessary includes 2018-11-15 17:29:34 +01:00
Riccardo Spagni 0cc3fc3756
Merge pull request #4769
5808530f blockchain: remove unused output_scan_worker parameter (moneromooo-monero)
1426209a blockchain: don't run threads if we have just one function to run (moneromooo-monero)
6f7a5fd4 db_lmdb: slight speedup getting array data from the blockchain (moneromooo-monero)
99fbe100 db_lmdb: save some string copies for readonly db keys/values (moneromooo-monero)
bf31447e tx_pool: speed up take_tx for transactions from blocks (moneromooo-monero)
4f005a77 tx_pool: remove unnecessary get_transaction_hash (moneromooo-monero)
593ef598 perf_timer: call reserve on new timer array (moneromooo-monero)
6ecc99ad core: avoid unnecessary tx/blob conversions (moneromooo-monero)
00cc1a16 unit_tests: notify test special case for the usual weirdo (moneromooo-monero)
2018-11-14 21:35:03 +02:00
doy-lee 8f03bb3fd7 Merge commit '8534f71' into LokiMergeUpstream 2018-11-12 14:55:42 +11:00
moneromooo-monero dffec2587f
db_lmdb: error out if the db needs migration in read only mode 2018-11-09 09:39:22 +00:00
doy-lee 1bb2fb8721 Merge commit '9a4b15c' into LokiMergeUpstream 2018-11-05 11:17:33 +11:00
moneromooo-monero 6f7a5fd4f7
db_lmdb: slight speedup getting array data from the blockchain 2018-11-04 13:23:00 +00:00
moneromooo-monero 97cad7e979
db_lmdb: log which output was not found in get_output_key 2018-11-02 11:49:39 +00:00
Riccardo Spagni 9a4b15c631
Merge pull request #4664
61304151 db_lmdb: use MDB_MULTIPLE_NEXT where possible for some speedup (moneromooo-monero)
2018-11-01 22:34:20 +02:00
moneromooo-monero 99fbe1008b
db_lmdb: save some string copies for readonly db keys/values 2018-11-01 17:26:09 +00:00
moneromooo-monero 6ecc99ad1f
core: avoid unnecessary tx/blob conversions 2018-11-01 15:51:19 +00:00
moneromooo-monero 61304151b4
db_lmdb: use MDB_MULTIPLE_NEXT where possible for some speedup 2018-10-19 16:47:29 +00:00
Cactii1 ec3013903b
Update db_lmdb.cpp 2018-10-17 21:36:10 +02:00
doy-lee 339f96b0cc Merge commit 'c23b6f8' into LokiMergeUpstreamUntil_20181010_77e1ebf 2018-10-10 15:07:16 +11:00
doy-lee c73da116ae Merge commit '4a652d6' into LokiMergeUpstreamUntil_20180911_e6d36c1 2018-10-10 13:48:01 +11:00
doy-lee c1da4f690d Merge commit '5ffb2ff' into LokiMergeUpstreamUntil_20180911_e6d36c1 2018-10-09 12:31:08 +11:00
doy-lee 12b8ffaee3 Merge commit 'f34e2e2' into LokiMergeUpstreamUntil_20180911_e6d36c1 2018-10-04 14:30:26 +10:00
Howard Chu 7c790f11f2
Fix rtxn usage in BlockchainLMDB::get_estimated_batch_size
Should only stop the rtxn if we actually started it
Fixes Coverity 184960
2018-10-02 16:14:20 +01:00
Riccardo Spagni effcbf2060
Merge pull request #4459
bcf3f6af fuzz_tests: catch unhandled exceptions (moneromooo-monero)
3ebd05d4 miner: restore stream flags after changing them (moneromooo-monero)
a093092e levin_protocol_handler_async: do not propagate exception through dtor (moneromooo-monero)
1eebb82b net_helper: do not propagate exceptions through dtor (moneromooo-monero)
fb6a3630 miner: do not propagate exceptions through dtor (moneromooo-monero)
2e2139ff epee: do not propagate exception through dtor (moneromooo-monero)
0749a8bd db_lmdb: do not propagate exceptions in dtor (moneromooo-monero)
1b0afeeb wallet_rpc_server: exit cleanly on unhandled exceptions (moneromooo-monero)
418a9936 unit_tests: catch unhandled exceptions (moneromooo-monero)
ea7f9543 threadpool: do not propagate exceptions through the dtor (moneromooo-monero)
6e855422 gen_multisig: nice exit on unhandled exception (moneromooo-monero)
53df2deb db_lmdb: catch error in mdb_stat calls during migration (moneromooo-monero)
e67016dd blockchain_blackball: catch failure to commit db transaction (moneromooo-monero)
661439f4 mlog: don't remove old logs if we failed to rename the current file (moneromooo-monero)
5fdcda50 easylogging++: test for NULL before dereference (moneromooo-monero)
7ece1550 performance_test: fix bad last argument calling add_arg (moneromooo-monero)
a085da32 unit_tests: add check for page size > 0 before dividing (moneromooo-monero)
d8b1ec8b unit_tests: use std::shared_ptr to shut coverity up about leaks (moneromooo-monero)
02563bf4 simplewallet: top level exception catcher to print nicer messages (moneromooo-monero)
c57a65b2 blockchain_blackball: fix shift range for 32 bit archs (moneromooo-monero)
2018-09-29 22:20:38 +02:00
Riccardo Spagni 09fb2bb5f8
Merge pull request #4445
fe125647 Fixup RENAME_DB() macro (Howard Chu)
2018-09-29 22:20:12 +02:00