Commit Graph

55 Commits

Author SHA1 Message Date
Amine Khaldi 7e50f5e0b0
Merge commit '03943a0e51413f1f7fbd1f9ff852fe7259c0658e' into checkpoint/main_from_release_1.8.2_03943a0e51413f1f7fbd1f9ff852fe7259c0658e 2023-06-27 20:18:36 +01:00
Kyle Altendorf 03943a0e51
allow tests/tools/test_full_sync.py::test_full_sync_test to run on 3.7 (#15570)
* allow tests/tools/test_full_sync.py::test_full_sync_test to run on 3.7

* drop dirs_exist_ok=

* more

* less
2023-06-21 18:14:34 -05:00
dustinface 5e0b8bb1ec
util: Drop `chia.util.chunks` and use `chia.util.to_batches` instead (#15418) 2023-06-09 16:35:31 -05:00
Kyle Altendorf 2ed75d47f3
more mypy platforms in ci (#12299)
* more mypy platforms in ci

* check not checks

* oh yeah, activate...

* add 3.11

* adjust job names

* actually use the right python version for install

* adjust for 3.7 hinting support

* exclude macos 3.7 and macos arm 3.8

* back to 3.8 for now

* major_dot_minor

* oops

* move ignore comments for 3.7

* and more
2023-04-27 09:51:18 -05:00
Arvid Norberg 2b62f168f6
extend mempool-benchmark (#15038) 2023-04-17 15:44:06 -05:00
Kyle Altendorf b839dfa0c1
update black to 23.3.0 (#14537)
* unpin black

* Update setup.py

* Update .pre-commit-config.yaml

* black

* black

* 23.3.0

* cleanup
2023-04-04 12:50:58 -05:00
Arvid Norberg 39d5283f1a
add benchmark that tests the mempool over a long time (#14728)
add benchmark that tests the mempool over a long time, churning transactions
2023-03-20 13:16:21 -05:00
Arvid Norberg 4ed31c50d5
Bump chia rs 0.2.4 (#14758)
* fix bug in make_test_conds() test utility

* bump chia_rs to 0.2.4, which preserves assert_seconds_relative 0 in parsing conditions. This allows for the 1.8.0 soft-fork to make the existing time-lock conditions stricter, > instead of >=. This is to match the existing ASSERT_HEIGHT_RELATIVE, which already is >

* fixup separating ENABLE_ASSERT_BEFORE from MEMPOOL_MODE

* Use a low value for SOFT_FORK2_HEIGHT during tests and cover the case before soft-fork2

---------

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
2023-03-07 15:29:54 -06:00
Kyle Altendorf 2d44e510b6
hint_benchmarks (#14738) 2023-03-06 11:50:44 -06:00
Arvid Norberg 9e8dae67b2
Bump chia_rs to 0.2.3. and ASSERT_MY_BIRTH_* (#14720)
* bump chia_rs to version 0.2.3

* add new error codes for assert_my_birth_*

* add new condition codes for ASSERT_MY_BIRTH_*

* add logic for ASSERT_MY_BIRTH_* conditions
2023-03-03 15:12:37 -06:00
Kyle Altendorf ff5ef6e073
remove leading newlines from indented code blocks (#14653)
* remove leading newlines from indented code blocks

* catch another one
2023-02-23 21:53:48 -06:00
Amine Khaldi b019fe5962
Introduce BlockRecordProtocol as a subset of BlockRecord that the mempool manager uses for peak (#14628)
* Introduce BlockRecordProtocol as a subset of BlockRecord that the mempool manager uses for peak.

* Create BenchBlockRecord and use it for benchmarks/mempool.py.

* PR #14611 didn't land yet (keep 3.7 support for now).

* We don't need this guidance anymore.
2023-02-23 01:04:47 -06:00
Arvid Norberg 1a722be39b
extend and improve the mempool benchmark (#14635) 2023-02-22 12:12:08 -06:00
Arvid Norberg 68c9aafc0a
move SerializedProgram to its own file (#14391)
* move SerializedProgram into its own file (to fixup type annotations)

* fix type annotations for SerializedProgram

* fixup import statements for new SerializedProgram module
2023-01-27 17:20:23 -06:00
Amine Khaldi cd4874c584
create_bundle_from_mempool() doesn't need to be async (#14114)
create_bundle_from_mempool() doesn't need to be async.
2022-12-13 17:11:12 -06:00
Adam Kelly 30feeaa45e
Avoid calling spend_bundle.name() twice (#14109) 2022-12-13 00:32:55 -06:00
Amine Khaldi 0cfd19f38d
Simplify the mempool manager instantiation by not making it depend on a coin store (#14015)
* Explore simplifying the mempool manager instantiation.

* Address reviews.
2022-11-29 12:18:09 -06:00
Adam Kelly 5cbc415589
Preserve correct `MempoolItem` block height when rebuilding mempool (#13951) 2022-11-21 16:28:48 -06:00
Adam Kelly 73e9d15cf7
Fee estimation. (#13645) 2022-10-18 09:12:22 -05:00
Kyle Altendorf 3b084a165b
configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Mariano Sorgente 87a3e93cc4
Ms.mypy enable (#13320)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Enable mypy for a bunch of files

* Update after merge with main

* 99/260 remaining

* Address comments

* Enable more mypy

* Merge conflict mypy file

* More fixes

* pytest.ini and cb-gui

* One more flake8 fix

* Flake8 and tests

* More test fixes

* isort

* Make LGTM happy

* Update chia/full_node/block_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/full_node/full_node_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/wallet/util/peer_request_cache.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Apply all suggestions from PR comments

* Revert cb-gui

* 100 back to 1000, minor tweaks

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-09-08 12:57:15 -05:00
Mariano Sorgente 8ef4c980b3
Ms.mempool simplify (#13314)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Update after merge with main

* Address comments
2022-09-06 12:32:03 -05:00
Arvid Norberg f295d8899b
add benchmark for MempoolManager.add_spendbundle (#12780)
* add benchmark for MempoolManager.add_spendbundle

* log warning if pre_validate_spendbundle takes too long
2022-08-09 18:29:00 -05:00
Arvid Norberg 5ae87189c4
Rust coin (#11807)
* use rust implementation of Coin class, from chia_rs

* bump chia_rs dependency
2022-06-13 07:21:04 -05:00
Arvid Norberg afae2ad5fe
make recurse_jsonify() work directly on types (#11537)
* make recurse_jsonify() work directly on types, circumventing the dataclasses.asdict() step. This enables simpler integration of non dataclasses into the Streamable and JSON protocols

* add benchmark for Streamable.to_json_dict()
2022-05-18 09:15:03 -07:00
Mariano Sorgente d1e445fac2
Ms.mypy consensus (#11403)
* Mypy for consensus dir (Except blockchain.py)

* Finish up consensus folder mypy

* Lint

* isort

* lint

* awaitable over coroutine

* less Any

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-05-12 15:21:47 -07:00
Kyle Altendorf 34eed9a3bc
more isort (#11401) 2022-05-05 08:17:38 -07:00
dustinface a48fd43100
streamable: Simplify and force correct usage (#10509)
* streamable: Merge `strictdataclass` into `Streamable` class

* tests: Test not supported streamable types

* streamable: Reorder decorators

* streamable: Simplify streamable decorator and force correct usage/syntax

* streamable: Just move some stuff around in the file

* streamable: Improve syntax error messages

* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion

* streamable: Use cached fields instead of `__annotations__`

This is now possible after merging `__post_init__` into `Streamable`

* Introduce `DefinitionError` as `StreamableError`

* `/t` -> `    `
2022-04-08 18:29:32 -07:00
Mariano Sorgente 890c7d3754
Fix remaining linting issues (#10962)
* FIx remaining linting issues

* Revert type:ignore

* Revert token_bytes change
2022-04-02 13:22:55 -07:00
Arvid Norberg a100dda37e
new DBWrapper supporting concurrent readers (#10166)
* new DBWrapper supporting concurrent readers

* adress review comments

* fixup default database version, when file doesn't exist

* remove unused argument
2022-03-28 11:58:00 -07:00
Arvid Norberg 656d7d94d8
fix type annotations for FullBlock.header_hash and FullBlock.prev_header_hash (#10909) 2022-03-27 12:20:30 -07:00
dustinface 0909fe7672
benchmarks: Implement streamable data comparison (#10433) 2022-03-04 09:51:39 -08:00
dustinface cdf48f7b29
benchmarks: Implement benchmarks for streamable (#10388)
* benchmarks: Implement benchmarks for streamable

* benchmarks: Collect iterations per time instead of time per iterations

* benchmarks: Add standard deviation to streamable benchs

* benchmarks: Add ns/iteration to streamable benchs

* benchmarks: Move object creation out or the runs loop

* benchmarks: Use `click.Choice` for `--data` and `--mode`

* benchmarks: Its µ

* benchmarks: Improve logging

* benchmarks: Drop unused code

* benchmarks: Use `process_time` as clock

* benchmarks: Add stdev `us/iterations %` + more precission

* benchmarks: Add `--live/--no-live` option to enable live results
2022-02-24 10:19:57 -08:00
Arvid Norberg 34159d3529
optimize get_block_generator() (#10104)
* optimize get_block_generator()

* add a v1 compatible get_generator() to speed up get_block_generator() with v1 databases. Add test. Add error log in case generator_from_block() fails.

* speed up test_full_block_utils
2022-02-17 07:32:29 -08:00
Arvid Norberg 0464265734
use monotonic clock in benchmarks (#10242) 2022-02-15 14:42:51 -08:00
dustinface 98971de9d5
pre-commit: Add a new hook to run `isort` (#8827)
* pre-commit: Add a new hook to run `isort`

* contributing: Add hint about `isort`

* add isort to dev deps, ignore existing .py files, use black profile

* long list to lines not comma delimited

* isort: Update and sort ignore list to match latest `main`

* add bash command line to generate isort extend skip list (#3)

* add bash command line to generate extend skip list

* tidy

* isort: More files to ignore after rebase

* tests: Fix `test_wallet_user_store.py` after rebase

* Some fixes after rebase

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-02-15 07:28:53 -08:00
Kyle Altendorf d404d0e92b
just put __init__.py files where we have .py files (plus subdirectories) (#10129)
* just put __init__.py files where we have .py files

* and the rest

* remove unused ignore
2022-02-07 13:16:13 -08:00
Arvid Norberg 6872e75a48
benchmark for blockchain.get_block_generator() (#9999) 2022-02-03 16:09:04 -08:00
Arvid Norberg 7fb7a916ee
Simplify FullNode._replace_proof (#10049)
* simplify updating of proofs (when compacting them). This also saves two database lookups before the update

* fixup

* insert or ignore duplicate blocks

* when replacing a weight proof with a compact version, we do so by height. Rather than asking for all blocks at the specific height followed by a linear search for the one with the correct header hash, we can simply lookup the correct header hash and just look up that block. This simplifies the logic by dropping the looping over blocks.

* pass in header_hash to _replace_proof() to avoid the height-to-hash lookup
2022-02-01 22:31:47 -08:00
Mariano Sorgente 6e37b74742
Change compact block (#10011)
* Change compact block

* Also recent blocks

* Add bill's rollback code

* Fix v2 issue

* Fix v2 issue blockchain

* Fix tests
2022-01-28 19:19:10 -08:00
Arvid Norberg 93d967525b
revert allow-upgrades configuration option. It makes it significantly more complicated to do proper database migrations (#9768) 2022-01-13 14:53:27 -08:00
Arvid Norberg f1dcd36c2a
In main chain (#9550)
* add in_main_chain column to full_blocks table

* add test for block_store in_main_chain

* test block_store invariant
2021-12-17 14:37:45 -08:00
Arvid Norberg 35cab7cea2
Block store benchmark (#9520)
* include SubEpochSummary objects

* make header_hashes authentic. include benchmarks of lookups on block_store as well
2021-12-09 15:38:59 -08:00
Arvid Norberg cd80f67d74
[v2] use binary fields in coin_store table (#9455)
* use binary fields on coin_store table

* fix coin_store benchmark format string
2021-12-06 11:22:12 -08:00
Arvid Norberg 546011381d
fix mypy issues in benchmark/block_store.py (#9457) 2021-12-05 09:36:52 -08:00
Arvid Norberg 70d31a8c68
V2 database framework (#9442)
* fix block_store benchmark linting issues

* basic support for database versioning

* update benchmark to run v1 and v2
2021-12-03 13:52:10 -08:00
Kyle Altendorf 8291f0221a
Make the sized bytes types hint compatible (#9369)
* Rework sized bytes for type hinting compatibility

* add a bunch of type: ignores

* this will be handled elsewhere

* noqa E501 instead of changing code

* normalize comment plurality

* @classmethod

* Revert "@classmethod"

This reverts commit 95db80e339.

* add ignore in benchmarks

* just E501 again...

* add some new type: ignores
2021-12-02 09:43:39 -08:00
Arvid Norberg a75b639adf
fix typo in block_store benchmark (and missing clvm generator field) (#9433) 2021-12-01 11:31:31 -08:00
Arvid Norberg fdaadc1ff9
add benchmark for block_store (#9425)
* add benchmark for block_store

* use random instead of secrets
2021-11-30 15:56:11 -08:00
Arvid Norberg 7f2be11da7
print database size in CoinStore benchmark (#9419) 2021-11-29 07:48:42 -08:00