Commit graph

485 commits

Author SHA1 Message Date
Kyle Altendorf
2daa524aaf
add long_lived/** and release/** to (almost) all workflow push triggers (#11439) 2022-05-05 08:51:36 -06:00
Arvid Norberg
1b2d119cc3
optimize hash_coin_list() by sorting pure coin_ids, rather than Coins that need to compute its name multiple times during sorting, and also by sorting the actual binary coin names, not hexadecimal strings (#11421) 2022-05-04 07:24:55 -07:00
Kyle Altendorf
d7f39e48c0
Add Windows to wheel check matrix (#11388) 2022-05-02 13:01:08 -07:00
Kyle Altendorf
795f199597
Correct naming of Windows installer artifacts (the GitHub Actions ones) (#11368)
The wrong artifact was renamed in https://github.com/Chia-Network/chia-blockchain/pull/11096.  This corrects that.
2022-04-29 18:36:05 -07:00
dustinface
984c442b5e
util: Implement Paginator class as interface to access a list by pages (#11247)
* util: Implement `Paginator` class as interface to access a list by pages

* Be less restrictive about page sizes and refactor tests

* Make the pages based of 0 instead of 1 and some more test refactoring

* More tests

* Adjust workflows after rebase

* Introduce `Paginator.create`

* `<=` instead of `- 1`
2022-04-29 18:35:47 -07:00
Kyle Altendorf
fd9ff115e3
tidy installer building (#11077)
* tidy installer building

* -d

* fix rpm build

* try for windows

* Install.ps1 -d

* move the install.ps1 call out to the workflow

* Update build_windows.ps1

* Update build_windows.ps1

* debug non-editable install on windows

* platlib and purelib in pyinstaller pathex

* Revert "debug non-editable install on windows"

This reverts commit 7c04168969.

* Revert "platlib and purelib in pyinstaller pathex"

This reverts commit c5c5cdcd81.

* debug

* debug not-editable

* Revert "debug not-editable"

This reverts commit d4373d8e82.

* a bit more similar

* git -C

* find dlls in site-packages?

* oops

* Revert "debug"

This reverts commit 88090f689f.

* consistently name installer github actions artifact zips

* leave SCM_VERSION here

* Update build-windows-installer.yml
2022-04-29 18:34:48 -07:00
Justin England
518bc3966d
adding workflow for checking for conflicts (#11370)
* adding workflow for checking for conflicts

* proper version number

* proper version number syntax?

* not sure tbh

* trying 2.x

* using newly created labels

* adding static 2.0.0 version

* update based on feedback

* removing label removal

* merge_conflict
2022-04-29 19:33:08 -06:00
Chris Marslender
ad3259e8fd
Mark workspace safe for arm installers (#11339) 2022-04-27 16:10:41 -07:00
Kyle Altendorf
a813e5906c
Match plurality between macOS and Ubuntu test job names (#11317)
* Match plurality between macOS and Ubuntu test job names

With this, searching for `test /` will match both groups of jobs.

* rebuild workflows
2022-04-27 11:40:31 -07:00
Chris Marslender
3abcbf1dfa
Build installers on push to release branches (#11321) 2022-04-26 16:18:16 -05:00
Kyle Altendorf
7892148bdc
Support for Python 3.10 (#9930)
* Support for Python 3.10

* Update install.sh to block Python 3.11

* websockets to 10.1

* Update workflows for Python 3.10

* single quote 3.10

* Enable fedora:35 (py3.10) installer script testing

* rebuild workflows

* fixup test-install-scripts.yml

* add ignore for distutils deprecation in tests for now

* asyncio.get_event_loop().run_until_complete() -> asyncio.run()

* aiohttp==3.8.1 for python 3.10 support

* use ssl.Purpose.CLIENT_AUTH for ssl_context_for_server()

* rebuild workflows

* use ssl_context_for_client() in BlockTools.get_daemon_ssl_context()

* create a client context for the RpcServer to connect to the daemon

* go back to asyncio.get_event_loop().run_until_complete() for now to recover 3.7

* ignore:There is no current event loop:DeprecationWarning

* Ms.plot load perf2 (#10978)

* 2.7 seconds -> 0.45 seconds

* Merge

* Work on create_plots refactor

* Try to fix tests

* Try to fix tests

* Use new functions

* Fix block_tools by adding dir

* Extra argument

* Try to fix cyclic import

* isort

* Drop warning

* Some cleanups around `exclude_final_dir` and directory adding

* Cleanup `min_mainnet_k_size` checks

* Drop unrelated changes

* Fixes after rebase

* Fix cyclic import

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* remove 3.10 avoidance step from debian:bookworm installer testing

* add 3.10 to wheel availability check workflow

* add 3.10 to Install.ps1 supported Python versions for Windows

* add jammy jellyfish to the install script test matrix

* correct ubuntu:jammy job name

* add 22.04 with Python 3.10 to install.sh

Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-04-26 12:37:01 -07:00
Chris Marslender
a983c4a90b
Upload CLI RPMS to s3, create checksums, etc (#11316) 2022-04-26 13:27:14 -05:00
Kyle Altendorf
f976c07c54
also m1 and arm64 for wheel checks (#11277)
* also m1 and arm64 for wheel checks

* account for self-hosted and pre-setup-python of m1 and arm64 runners

* &&

* python3

* report python version

* use docker on arm64 to get multiple python versions

* flush

* report more system information

* except pycryptodome for now

* more variables, simpler logic

* corrections

* switch to [macos, arm64]

* add python version to job name

* separate os and arch matrix axes

* reorder matrixing

* drop workflow name from job name

* oops

* skip python setup in docker cases

* drop the containers

* Update check_dependency_artifacts.py
2022-04-26 10:48:04 -07:00
Chris Marslender
fd7c3c6363
Update names of installer workflows so they get grouped together in the workflow listing (#11310) 2022-04-26 09:01:21 -06:00
Chris Marslender
d750c901b9
Fix targeting for arm64 to not land on native arm64 mac runners (#11309) 2022-04-25 16:50:38 -05:00
Kyle Altendorf
e584513c70
Add Linux Mint to test matrix (#11295)
* add Linux Mint to test matrix

* : -> / for linux mint dockers

* Prepare Linux Mint

* stop testing linux mint 21 since it reports 20.3

* names, comment, and add 19.2

* mint

* manually install requests

* trailing whitespace
2022-04-25 09:20:08 -07:00
Kyle Altendorf
8460a35466
Debian bookworm now uses 3.10, update the comment (#11278) 2022-04-23 20:11:09 -07:00
Kyle Altendorf
3c48224551
check dependency artifacts (#11243)
* check dependency artifacts

* remove commented out code

* find root path relative to script

* fixup stringy pathy mixup

* Update check_dependency_artifacts.py
2022-04-21 17:00:28 -07:00
Kyle Altendorf
aaf3fbc160
generate file lists for workflows instead of wildcard patterns (#11117) 2022-04-21 16:58:07 -07:00
Chris Marslender
527b30e917
Add cli only rpm (#11236)
* Add cli only rpm

* Ensure rvm (fpm) is loaded before running fpm

* Use full path to fpm, since GHA seems to mess up the PATH in the container

* Add back source and add use ruby-3

* Call rpm script with bash, to see if the rvm script works

* Add --depends for libcrypt.so.1
2022-04-20 21:39:51 -07:00
dustinface
25ab0c90cb
github: Fix plot_sync workflows after #10940 merge (#11241) 2022-04-20 15:27:59 -07:00
Gene Hoffman
797a2fb5fe
Remove 3.9.11 for Windows Installer (#11226)
And replace with just 3.9.x
2022-04-20 11:44:52 -07:00
dependabot[bot]
efbac6a05f
Bump actions/upload-artifact from 2 to 3 (#11144)
* Bump actions/upload-artifact from 2 to 3

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Also update runner_templates

* Mark the github workspace as safe (#11159)

* Mark the github workspace as safe

* Move the git config step after git is installed in the test containers

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
2022-04-20 11:40:51 -07:00
Kyle Altendorf
5b044ea497
locate pyinstaller in dev deps rather than build scripts (#11118) 2022-04-20 11:16:53 -07:00
Kyle Altendorf
93da3b727b
Up Windows installer build timeout to 50 minutes (#11107)
Recent runs are mostly low 30s but some are close to 40 and one just timed out.
2022-04-20 11:14:20 -07:00
Kyle Altendorf
b9c9727b75
--module pytest (#10940) 2022-04-20 11:11:08 -07:00
Chris Marslender
7a123afddb
Fix install.sh test for bookworm (#11227) 2022-04-19 15:43:06 -07:00
Chris Marslender
337dbd4e5c
Revert "Revert "Pin mac intel installer to 10.15 (#11209)" (#11210)" (#11211)
This reverts commit d3e73a75ab.
2022-04-19 12:53:59 -06:00
Chris Marslender
d3e73a75ab
Revert "Pin mac intel installer to 10.15 (#11209)" (#11210)
This reverts commit 93a61eece1.
2022-04-18 15:42:09 -06:00
Chris Marslender
93a61eece1
Pin mac intel installer to 10.15 (#11209) 2022-04-18 15:34:27 -06:00
Chris Marslender
7567b4ea9c
Fix filename of latest intel dev installer (#11203) 2022-04-18 10:18:59 -05:00
Chris Marslender
2fbe062e78
Build cli only version of debs (#11166)
* Build cli only version of debs

* Export the vars needed by j2

* Fix paths

* Add symlink to chia in /usr/local/bin/

* Upload the cli only debs to s3

* Add init.py

* Ensure SHA is on the dev build for amd64
2022-04-14 19:32:26 -05:00
Chris Marslender
3ea3a4dc0c
Use get latest release endpoint for plotters, so that we ignore any pre-releases that could be returned by listReleases (#11165) 2022-04-13 18:06:50 -05:00
Chris Marslender
b3a9459fea
Mark the github workspace as safe (#11159)
* Mark the github workspace as safe

* Move the git config step after git is installed in the test containers
2022-04-13 09:35:42 -06:00
Kyle Altendorf
2d4045b9d2
only check the version once in installer build workflows (#11099) 2022-04-08 13:49:21 -07:00
Kyle Altendorf
e6f60c572e
git -C and consistent activation in installer builds (#11098) 2022-04-08 09:59:59 -07:00
Kyle Altendorf
13512be2b2
consistently name installer github actions artifact zips (#11096) 2022-04-08 09:59:36 -07:00
Kyle Altendorf
415236bf67
can we get by without dead snakes? (#11070)
* can we get by without dead snakes?

* Update install-timelord.sh

* Revert "Update install-timelord.sh"

This reverts commit cba3250b09.

* do not install python dev package for timelords build in ci

it is already there...

* more quotes for sh
2022-04-08 09:58:46 -07:00
Mariano Sorgente
2f9e718073
Ms.fast test blockchain (#11051)
* more work on test blockchain

* Optimize test_blockchain.py

* Fix weight proof bug

* Rename variable

* first rc_sub_slot hash bug

* New plots

* try with a new ID

* Run without cache

* Address test blocks and plots preparation.

* Update constant in test_compact_protocol().

* Update this constant too.

* Revert accidental altering of the gui submodule in ae7e3295f280a591e76c4dffdea75fb74ea5de6f.

* Fix benchmark test

* Revert mozilla-ca change

* Rebase on main

Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
2022-04-08 09:57:59 -07:00
Kyle Altendorf
02a880fbac
rebuild workflows (#11092) 2022-04-07 20:43:27 -07:00
Chris Marslender
c7a89fc425
Add wallentx as additional assignee on mozilla CA update PRs (#11089) 2022-04-07 19:43:14 -05:00
dustinface
ded9f68583
chia|tests|github: Implement, integrate and test plot sync protocol (#9695)
* protocols|server: Define new harvester plot refreshing protocol messages

* protocols: Bump `protocol_version` to `0.0.34`

* tests: Introduce `setup_farmer_multi_harvester`

Allows to run a test setup with 1 farmer and mutiple harvesters.

* plotting: Add an initial plot loading indication to `PlotManager`

* plotting|tests: Don't add removed duplicates to `total_result.removed`

`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.

* harvester: Reset `PlotManager` on shutdown

* plot_sync: Implement plot sync protocol

* farmer|harvester: Integrate and enable plot sync

* tests: Implement tests for the plot sync protocol

* farmer|tests: Drop obsolete harvester caching code

* setup: Add `chia.plot_sync` to packages

* plot_sync: Type hints in `DeltaType`

* plot_sync: Drop parameters in `super()` calls

* plot_sync: Introduce `send_response` helper in `Receiver._process`

* plot_sync: Add some parentheses

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* plot_sync: Additional hint for a `Receiver.process_path_list` parameter

* plot_sync: Force named parameters in `Receiver.process_path_list`

* test: Fix fixtures after rebase

* tests: Fix sorting after rebase

* tests: Return type hint for `plot_sync_setup`

* tests: Rename `WSChiaConnection` and move it in the outer scope

* tests|plot_sync: More type hints

* tests: Rework some delta tests

* tests: Drop a `range` and iterate over the list directly

* tests: Use the proper flags to overwrite

* test: More missing duplicates tests

* tests: Drop `ExpectedResult.reset`

* tests: Reduce some asserts

* tests: Add messages to some `assert False` statements

* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`

* tests: Use `secrects` instead of `Crypto.Random`

* Fixes after rebase

* Import from `typing_extensions` to support python 3.7

* Drop task name to support python 3.7

* Introduce `Sender.syncing`, `Sender.connected` and a log about the task

* Add `tests/plot_sync/config.py`

* Align the multi harvester fixture with what we do in other places

* Update the workflows

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-07 17:10:44 -07:00
Kyle Altendorf
c0d346d428
Remove dead snakes usage from benchmark tests (#11053) 2022-04-07 09:18:15 -07:00
dependabot[bot]
42245d74eb
Bump github/super-linter from 4.9.1 to 4.9.2 (#11067)
Bumps [github/super-linter](https://github.com/github/super-linter) from 4.9.1 to 4.9.2.
- [Release notes](https://github.com/github/super-linter/releases)
- [Changelog](https://github.com/github/super-linter/blob/main/docs/release-process.md)
- [Commits](https://github.com/github/super-linter/compare/v4.9.1...v4.9.2)

---
updated-dependencies:
- dependency-name: github/super-linter
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-07 09:17:53 -07:00
Amine Khaldi
6026d734cf
Significantly speedup preparing test blocks and plots by opting for a release download instead of a shallow git clone, and also by putting a caching layer on top of that. (#11065) 2022-04-06 20:15:10 -07:00
Kyle Altendorf
45e4c7c156
Checkout test blocks and plots for benchmarks workflow (#11068) 2022-04-05 17:16:38 -05:00
William Allen
6ed61bfbe5
Adding clean-workspace step to benchmarks (#11063) 2022-04-05 15:28:33 -05:00
Amine Khaldi
a2fa8dda01
Prepare test blocks and plots only for tests that need them. This saves us a couple more hours of CI running time. (#10975) 2022-04-05 10:34:57 -07:00
dustinface
4a4b14b78b
github: Drop unused BUILD_VDF_CLIENT variables (#11050)
From my understanding this is only used by `chiavdf` source builds which 
happen only if `install-timelord.sh` gets called but it doesn't in the 
addressed cases.
2022-04-04 18:43:51 -07:00
Adam Kelly
22edd73331
Force apt to install the things we asked it to (#11047)
* Force apt to install the things we asked it to

* Update .github/workflows/benchmarks.yml

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-04-04 17:43:31 -07:00