Commit Graph

1152 Commits

Author SHA1 Message Date
Ludovic Courtès 87a6d6ea7a
remote-worker: Correctly compute parallelism per worker.
Fixes <https://issues.guix.gnu.org/67502>.

Since ‘run-fibers’ binds each thread to one core by default,
‘current-processor-count’ when called from within ‘run-fibers’ on a
machine with 4 or fewer cores would always return 1.

* src/cuirass/scripts/remote-worker.scm (worker-management-thunk): Add
‘cpu-count’ parameter and use it instead of calling
‘current-processor-count’.
(cuirass-remote-worker): Call ‘current-processor-count’ outside
‘run-fibers’ and pass the result to ‘worker-management-thunk’.
2023-11-28 15:51:38 +01:00
Ludovic Courtès 3a6abc17f9
remote-worker: Add ‘--user’.
* src/cuirass/scripts/remote-worker.scm (show-help, %options): Add ‘--user’.
(cuirass-remote-worker): Honor it.
* doc/cuirass.texi (Invocation): Document it.
2023-11-27 15:26:43 +01:00
Ludovic Courtès 6c9e9fc26f
remote-server: Warn when running as root.
* src/cuirass/scripts/remote-server.scm (cuirass-remote-server): Warn
when running as root.
2023-11-27 15:22:28 +01:00
Ludovic Courtès a68b3d7122
utils: Add ‘gather-user-privileges’.
* src/cuirass/scripts/remote-server.scm (gather-user-privileges): Move to…
* src/cuirass/utils.scm (gather-user-privileges): … here.
2023-11-27 15:19:33 +01:00
Ludovic Courtès bdc1f9f304
templates: Use plural wisely in “new failures” button.
* src/cuirass/templates.scm (build-eval-table): Adjust plural for
failure button.
2023-11-22 11:07:14 +01:00
Ludovic Courtès ef5437e051
templates: Choose “danger” badge when there are actual new failures.
Fixes a bug in 0daae8bfc9 whereby a
“danger” badge would sometimes be displayed even though the “new”
failures were not introduced by this particular evaluation.

* src/cuirass/templates.scm (evaluation-badges): Use
‘build-summary-newly-failed’ rather than
‘evaluation-summary-newly-failed’ to choose the type of badge.
2023-11-22 10:49:55 +01:00
Ludovic Courtès 7ab2107688
tests: Check value of ‘evaluation-summary-newly-failed’.
This is a followup to 7f042dbc37.

* tests/database.scm ("db-get-evaluation-absolute-summary")
("db-get-evaluations-absolute-summary"): Check
‘evaluation-summary-newly-failed’.
2023-11-22 10:48:42 +01:00
Ludovic Courtès 66167c1a29
database: Add ‘newly-failed’ field to <build-summary>.
* src/cuirass/database.scm (<build-summary>)[newly-failed]: New field.
(db-get-evaluations-build-summary): Set it.
* tests/database.scm ("db-get-evaluations-build-summary"): Test it.
2023-11-22 10:47:36 +01:00
Ludovic Courtès 8cb3db46f1
database: Use ‘exec-query/bind’ yet some more.
* src/cuirass/database.scm (db-get-evaluations-build-summary): Use
‘exec-query/bind’ instead of ‘exec-query/bind-params’.
2023-11-21 23:27:53 +01:00
Ludovic Courtès ea4ddfb7f6
database: Remove <build-summary> default values.
* src/cuirass/database.scm (<build-summary>): Remove default values.
2023-11-19 12:01:01 +01:00
Ludovic Courtès ac4f5c0a7e
templates: Build table can be limited to new failures.
* src/cuirass/http.scm (evaluation-html-page): Adjust ‘db-get-builds’
call for “newly-failed” STATUS.
* src/cuirass/templates.scm (build-eval-table): Add #:summary.  Add
button when STATUS is “failed” or “newly-failed”.
(evaluation-build-table): Handle STATUS as “newly-failed”.
Pass #:summary to ‘build-eval-table’.
2023-11-18 19:09:30 +01:00
Ludovic Courtès 1c2e4f0f70
database: Allow filtering out new failures.
* src/cuirass/database.scm (db-get-builds): Add support for
‘new-failure’.
(db-get-builds-min, db-get-builds-max): Add support for ‘newly-failed’.
2023-11-18 19:01:25 +01:00
Ludovic Courtès 9fdfd27c4f
database: Use ‘exec-query/bind’ some more.
* src/cuirass/database.scm (db-get-builds-min, db-get-builds-max):
Rewrite using ‘exec-query/bind’ instead of ‘exec-query/bind-params’.
2023-11-18 18:03:38 +01:00
Ludovic Courtès 0daae8bfc9
templates: Distinguish failure counter when there are new failures.
The failed-build counter on /eval/ID pages is now orange when there are
no new build failures, red when there are new build failures.

* src/cuirass/templates.scm (newly-failed-build-badge): New procedure.
(specifications-table): Use it instead of ‘failed-build-badge’.
(evaluation-badges): Use ‘newly-failed-build-badge’ when ABSOLUTE has
a positive ‘new-failed’.
2023-11-18 12:46:19 +01:00
Ludovic Courtès 7f042dbc37
database: Add ‘newly-failed’ field to <evaluation-summary>.
* src/cuirass/database.scm (<evaluation-summary>)[newly-failed]: New
field.
(db-get-evaluation-summary, db-get-evaluation-absolute-summary):
Initialize it.
2023-11-18 12:38:44 +01:00
Ludovic Courtès 7c697ad7f1
utils: ‘with-resource-from-pool’ does not suspend from exception handler.
Fixes <https://issues.guix.gnu.org/67041>.

* src/cuirass/utils.scm (call-with-resource-from-pool): Rewrite to avoid
calling ‘put-message’ from the exception handler.
* tests/utils.scm ("resource pool, exception thrown"): New test.
2023-11-16 23:20:00 +01:00
Ludovic Courtès bdcbf01fa5
utils: Add tests for the resource pool.
* tests/utils.scm: New file.
* Makefile.am (TESTS): Add it.
2023-11-16 22:16:27 +01:00
Ludovic Courtès 85bcc32bcf
base: Fix typo in ‘exception-reporter’.
* src/cuirass/base.scm (exception-reporter): Fix (> depth 0) case.

Reported-by: Ricardo Wurmus <rekado@elephly.net>
2023-11-16 16:15:16 +01:00
Ludovic Courtès a969ca8390
remote-server: Ignore unrecognized messages.
* src/cuirass/scripts/remote-server.scm (serve-build-requests): Add
catch-all clause for unrecognized messages.
2023-11-16 14:20:00 +01:00
Ludovic Courtès 38d864defc
remote-server: Catch ZeroMQ errors when replying to workers.
This fixes a bug whereby EHOSTUNREACH would cause ‘cuirass
remote-server’ to exit.

Fixes <https://issues.guix.gnu.org/67224>.

* src/cuirass/scripts/remote-server.scm (serve-build-requests): Catch
‘zmq-error’ when calling ‘reply-worker’.
2023-11-16 12:13:52 +01:00
Ludovic Courtès dadd43a04c
remove-server: Remove misplaced function decomposition.
* src/cuirass/scripts/remote-server.scm (read-worker-exp)
(need-fetching?): Fold into…
(serve-build-requests): … this.  Use quasiquote for ‘match’ patterns.
2023-11-16 10:19:11 +01:00
Ludovic Courtès ba3ea7947e
remote-worker: Better guard against server disconnections.
Fixes <https://issues.guix.gnu.org/66980>.

Fixes a bug whereby a worker fiber would exit due to an uncaught
exception, typically when calling ‘finish’.

* src/cuirass/scripts/remote-worker.scm (run-build): Add ‘else’ clause
in ‘guard’ form.
2023-11-16 08:44:42 +01:00
Ludovic Courtès eb3f539dc9
build: Delete ‘cuirass-remote-worker.service’ upon ‘make clean’.
* Makefile.am (MOSTLYCLEANFILES): Add ‘etc/cuirass-remote-worker.service’.
2023-10-29 23:02:25 +01:00
Ludovic Courtès c949436462
build: Add missing test files to the distribution.
* Makefile.am (EXTRA_DIST): Add ‘tests/common.scm’ and ‘tests/mail.sh’.
* tests/database.scm (mailer): Use ‘search-path’ to locate ‘mail.sh’
relative to $top_srcdir.
2023-10-29 22:58:14 +01:00
Ludovic Courtès 5ca1147af3
Update NEWS. 2023-10-29 22:39:07 +01:00
Ludovic Courtès 0b1e1ed44f
build: Distribute ‘new-client-cert.scm’.
* Makefile.am (noinst_SCRIPTS): Move ‘etc/new-client-cert.scm’ to…
(dist_noinst_SCRIPTS): … here.  New variable.
2023-10-29 22:37:25 +01:00
Ludovic Courtès dfab507438
Update NEWS. 2023-10-29 20:58:46 +01:00
Ludovic Courtès 299e95a7b6
templates: Factorize the success rate bar.
* src/cuirass/templates.scm (success-rate-bar): New procedure.
(specifications-table): Use it.
2023-10-29 17:34:10 +01:00
Ludovic Courtès 286fef2ebb
templates: Move RSS button to the right.
* src/cuirass/templates.scm (specifications-table): Make “RSS events”
the rightmost button.
(evaluation-info-table): Likewise.
2023-10-29 17:26:55 +01:00
Ludovic Courtès 6611151b7f
templates: Improve toggle button on evaluation and spec pages.
The contrast icon was arguably misleading.

* src/cuirass/templates.scm (specifications-table): Use ‘oi-dashboard’
rather than ‘oi-contrast’ for the toggle button; clarify ‘title’.
(evaluation-info-table): Likewise.
2023-10-29 17:20:42 +01:00
Ludovic Courtès b80461b75e
build-log.js: Tweak icon of the phase folding button.
* src/static/js/build-log.js: Change the class of the phase
collapse/expand button.  Give it an ID and change its icon as a function
of OPENNESS.
2023-10-28 23:01:03 +02:00
Ludovic Courtès dabc19e3f0
templates: Adjust styling for link to raw build log.
* src/cuirass/templates.scm (pretty-build-log): Add spacing after the
‘oi-external-link’ <span> tag.
2023-10-28 23:00:14 +02:00
Ludovic Courtès 57902ecb2d
templates: Clarify evaluation page for states other than ‘succeeded’.
* src/cuirass/templates.scm (evaluation-build-table): Special-case the
situation where EVALUATION is not in ‘succeeded’ state.  Do not emit
build table when BUILDS is empty and EVALUATION is not in ‘succeeded’
state.
2023-10-28 22:48:46 +02:00
Ludovic Courtès 0a9776e57d
remote-worker: Use a separate GC root directory.
* src/cuirass/scripts/remote-worker.scm (cuirass-remote-worker): Change
‘%gc-root-directory’.
2023-10-25 20:11:18 +02:00
Ludovic Courtès 476324286b
remote-worker: Periodically delete old GC roots.
With commit 55af0f70c0, GC roots created
by ‘cuirass remote-worker’ would no longer be deleted (unless it’s
running on the same machine as ‘cuirass remote-server’).

* src/cuirass/scripts/remote-worker.scm (cuirass-remote-worker): Add
call to ‘spawn-gc-root-cleaner’.
* src/cuirass/base.scm (delete-old-gc-roots): Add #:check-database? and
honor it.
(spawn-gc-root-cleaner): Likewise.
2023-10-25 20:10:39 +02:00
Ludovic Courtès 3cbaa5674f
remote-server: Tweak message.
* src/cuirass/scripts/remote-server.scm (spawn-periodic-updates-fiber):
Clarify log message.
2023-10-25 19:15:12 +02:00
Ludovic Courtès 505ce3f8dc
Switch from SRFI-11 to SRFI-71.
* src/cuirass/base.scm, src/cuirass/http.scm,
src/cuirass/scripts/remote-worker.scm, src/cuirass/zabbix.scm: Use
SRFI-71 instead of SRFI-11.
2023-10-25 19:09:40 +02:00
Ludovic Courtès 2eb3e13580
remote-worker: Discard log anytime ‘send-log’ throws.
* src/cuirass/scripts/remote-worker.scm (run-build): Discard build logs
when ‘send-log’ throws, no matter which exception is thrown.  Improve
logging.
2023-10-25 18:29:21 +02:00
Ludovic Courtès 4cc37f540c
remote-worker: Ignore the return value of ‘build-derivations&’.
* src/cuirass/scripts/remote-worker.scm (run-build): Ignore the return
value of ‘build-derivations&’ since it’s always #t.
2023-10-25 18:28:20 +02:00
Ludovic Courtès 3bbb5c8447
store: ‘build-derivations&’ enforces synchronization with the build.
Previously, it was possible for the user to call the returned thunk
before ‘build-derivations’ had completed, thereby getting #f (the
initial value of ‘result’).  This made no sense because
‘build-derivations’ always returns #t or raises an exception.

This situation could happen in ‘cuirass remote-worker’ if the
corresponding ‘cuirass remote-server’ process disappeared in the middle
of a build, because ‘send-logs’ would return early, leading the finish
thunk of ‘build-derivations&’ to be called before build completion.

This change uses a channel to enforce synchronization with the thread
that calls ‘build-derivations’.

Partly fixes <https://issues.guix.gnu.org/66692>.

* src/cuirass/store.scm (build-derivations&): Use
‘with-exception-handler’ and ‘put-message’ in the thread.  Use
‘get-message’ instead of ‘atomic-box-ref’ in the finalization procedure.

Reported-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
2023-10-25 18:22:27 +02:00
Ludovic Courtès 159cadca3f
remote: Improve error messages in ‘send-log’.
* src/cuirass/remote.scm (sockaddr->string): New procedure, moved from…
(receive-logs): … here.
(send-log): Print the zlib error when it occurs.  Print the peer’s
address upon invalid handshake.
2023-10-25 18:06:52 +02:00
Ludovic Courtès c5ecec38ce
remote-server: Slightly improve ‘ensure-path*’ error messages.
* src/cuirass/scripts/remote-server.scm (ensure-path*): Tweak error
messages.
2023-10-25 18:06:42 +02:00
Ludovic Courtès 966505d22e
Update TODO. 2023-10-25 00:43:36 +02:00
Ludovic Courtès 623beb5288
remote-server: Tweak error message.
* src/cuirass/scripts/remote-server.scm (cuirass-remote-server): Adjust
error message.
2023-10-25 00:41:41 +02:00
Ludovic Courtès ba3278866f
templates: Link to /build/ID/log.
* src/cuirass/templates.scm (build-details): Add link to /build/ID/log.
(build-eval-table, build-search-results-table, running-builds-table):
Link to /build/ID/log.
2023-10-25 00:14:29 +02:00
Ludovic Courtès 2ab4df3fdd
templates: Add /build/ID/log, with client-side build log highlighting.
* src/static/js/build-log.js: New file.
* Makefile.am (dist_js_DATA): Add it.
* src/cuirass/http.scm (%file-white-list): Add it.
(url-handler): Add “/build/ID/log” handler.
* src/cuirass/templates.scm (pretty-build-log): New procedure.
2023-10-25 00:06:22 +02:00
Ludovic Courtès 18337a7608
templates: For new failures, link to a view of the commit range.
* src/cuirass/templates.scm (%vcs-web-commit-range-views): New variable.
(commit-range-hyperlink): New procedure.
(checkout-change-table): Use it.
2023-10-21 22:01:01 +02:00
Ludovic Courtès b5ea2cb620
templates: Show which checkouts have changed on evaluation pages.
* src/cuirass/http.scm (evaluation-html-page): Define ‘checkout-changes’
and pass it to ‘evaluation-build-table’.
* src/cuirass/templates.scm (checkout-table): Add #:changes parameter
and honor it.
(evaluation-build-table): Add #:checkout-changes parameter and pass it
on to ‘checkout-table.
(evaluation-dashboard): Define ‘checkout-changes’ and pass it to
‘checkout-table’.
2023-10-21 19:05:12 +02:00
Ludovic Courtès ab3265bad0
store: Remove ‘%gc-root-ttl’ parameter.
This is a followup to 55af0f70c0.

* src/cuirass/store.scm (%gc-root-ttl): Remove.
* src/cuirass/scripts/register.scm (cuirass-register): Remove references
to ‘%gc-root-ttl’.
* src/cuirass/scripts/remote-server.scm (%options): Warn about ‘--ttl’
having no effect.  Remove reference to ‘%gc-root-ttl’.
* src/cuirass/scripts/remote-worker.scm (%options): Warn about ‘--ttl’
having no effect.  Remove reference to ‘%gc-root-ttl’.
2023-10-21 18:44:30 +02:00
Ludovic Courtès 1ee143c0c0
maint: Build against the latest ‘guile-fibers’.
Fibers 1.1 has bugs that have long been fixed in 1.3.

* guix.scm: Use ‘guile-fibers’ instead of ‘guile-fibers-1.1’.
2023-10-19 08:51:49 +02:00