So that Cuirass specifications used to build 'guix-modular' can be named
differently than "guix" and "guix-modular" (see Guix's
build-aux/hydra/guix-modular.scm).
The name is used as a primary key, so before that commit, it was also
impossible to have several such specifications.
This is a workaround, Cuirass should normally support several inputs per
specification, as Hydra does. The specification's name would then only be
used to identify it. See <https://bugs.gnu.org/31813>.
* bin/evaluate.in (main): Replace custom NAME (passed to PROC) with 'guix'.
Co-authored-by: Mathieu Othacehe <m.othacehe@gmail.com>
Fixes a regression introduced in
074b9d02f1 whereby we'd attempt to build
all of DRV at once.
* src/cuirass/base.scm (spawn-builds): Pass BATCH, not DRV, to
'build-derivations&' and to 'update-build-statuses!'.
Previously we'd systematically add the first argument to the search
path. When that first argument was the empty string (as with the
'guix-modular' jobset), we'd thus add the current directory to the
search path, even if the intent was to leave the load path unchanged.
Furthermore, a3a7c09b06 changed load-path
handling to be explicit in Scheme.
* bin/evaluate.in: Remove GUILE_LOAD_PATH assignment.
This avoids a situation whereby, when evaluating from a Guix checkout,
we'd have already loaded slightly different and incompatible (guix …)
modules.
Hydra's 'hydra-eval-guile-jobs' implemented the same solution as in this
patch already.
* bin/evaluate.in: Remove use of (cuirass …) and (guix …) modules.
(ref): New procedure.
(with-directory-excursion): New macro.
(main): Use 'ref'. Remove uses of Guix or Cuirass modules.
The previous load path was potentially incorrect since commit
2fe7ff87e2.
* src/cuirass/base.scm (evaluate)[tokenize, load-path]: New variables.
Assume #:load-path is colon-separated. Pass LOAD-PATH as the second
argument to 'evaluate'.
* doc/cuirass.texi (Database): Adjust documentation.
Previously we would make a SQL query that would return many build jobs,
and then call 'delete-duplicates' on that. This was extremely wasteful
because the list of returned by the query was huge leading to a heap of
several tens of GiB on a big database, and 'delete-duplicates' would
lead to more GC and it would take ages.
Furthermore, since 'delete-duplicates' is written in C as of Guile
2.2.3, it is uninterruptible from Fiber's viewpoint. Consequently, the
kernel thread running the 'restart-builds' fiber would never schedule
other fibers, which could lead to deadlocks--e.g., since fibers are
scheduled on a circular shuffled list of kernel threads, once every N
times, a web server fiber would be sent to that kernel thread and not be
serviced.
* src/cuirass/base.scm (shuffle-jobs): Remove.
(shuffle-derivations): New procedure.
(spawn-builds): Take a list of derivations instead of a list of jobs.
(restart-builds): Remove 'builds' parameter. Remove 'delete-duplicates'
call. Remove done/remaining partitioning.
(build-packages): Adjust to pass 'spawn-builds' a list of derivations.
* bin/cuirass.in (main): Remove computation of PENDING. Remove second
parameter in call to 'restart-builds'.
* bin/evaluate.in (main): Always pass an alist as the arguments to PROC,
containing at least 'file-name' and 'revision'.
* examples/random-jobs.scm (make-random-jobs): Display 'file-name' and
'revision' from ARGUMENTS.
This avoids copying things back and forth.
* src/cuirass/base.scm (fetch-repository): Add #:writable-copy?
parameter. Call 'make-writable-copy' when it's true.
(copy-repository-cache): Remove.
(make-writable-copy): New procedure.
(evaluate): Add 'source' parameter and pass it to the 'evaluate' program.
(process-specs): Define 'compile?'. Pass #:writable-copy? to
'fetch-repository'. Remove call to 'copy-repository-cache'. Remove
computation of the checkout directory name. Pass CHECKOUT to 'evaluate'.
* bin/evaluate.in (main): Replace 'cachedir' with 'source'. Remove
computation of the checkout directory name.
Fixes a regression introduced in
1bab5c4e56 whereby the
'status+submission-time' order would no longer be honored.
As a result, /api/queue would return the queue ordered by build IDs,
making it largely useless.
* src/cuirass/database.scm (db-get-builds): Remove 'order' and rename
'order-column-name' to 'order'. Add case for 'status+submission-time'.
* tests/database.scm ("database")["db-get-builds"]: Move below
"db-update-build-status!" test. Add case for the
'status+submission-time' order.
This reinstates c47dfdf82b and fixes the
issues that led to the revert in
b71f0cdca5.
Before that, 'run-server' would force sequential processing of client
requests one after another.
* src/cuirass/http.scm (run-cuirass-server): Rewrite to use its own loop
instead of 'run-server'. Spawn a database fiber.
(with-database-access): New macro.
(handle-build-request): Expect 'db-channel' and use 'with-database-access'.
(handle-builds-request): Likewise.
(url-handler): Likewise.
Fixes a bug whereby some fibers would get a SQLITE_BUSY exception while
accessing the database: see
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30644#26>.
Suggested by Danny Milosavljevic <dannym@scratchpost.org>.
* src/cuirass/database.scm (wal-mode): Rename to...
(set-db-options): ... this. Add call to 'sqlite-exec' for
'busy_timeout'.
Fixes a regression introduced in
f083282fd3, whereby we'd return 0 as the
timestamp for everything in /api/queue.
* src/cuirass/http.scm (build->hydra-build): Make 'finished?' a
Boolean. Move 'bool->int' call in #:finished definition.
Previously, when an exception was raised from
'handle-build-event' (e.g., a "database is locked" error), we'd throw,
thereby leaving PORT open and we'd never read from it again. Thus, the
corresponding 'guix-daemon' process would eventually get stuck in a
'write' call to that socket, and its build processes would stall.
* src/cuirass/base.scm (exception-reporter): New procedure.
(spawn-builds): Use it.
Fixes <https://bugs.gnu.org/30618>.
Reported by Andreas Enge <andreas@enge.fr>.
* bin/evaluate.in (fill-job): Remove.
(main): Remove 'database' command-line argument. Remove DB and its
uses. Write an (evaluation EVAL JOBS) sexp.
* src/cuirass/base.scm (evaluate)[augment-job]: New procedure.
Use it. Adjust to read (evaluation EVAL JOBS) sexp. Call
'db-add-evaluation' and 'db-add-derivation'.
Fixes a regression introduced in
593cb7be10.
* src/cuirass/http.scm (request-parameters): Fix fallback case in
'match' form. Previously it would return a procedure in this case, as
returned by (const param), leading to a failure down the road in
'sqlite-bind-arguments' as could be seen by running tests/http.scm.
I've seen 'scandir' report #f once, even though that's theoretically
impossible.
* src/cuirass/logging.scm (log-monitoring-stats): Return '() if
'scandir' returns #f.
Previously it would be installed under cuirass/.
* Makefile.am (webmoduledir, webobjectdir): New variables.
(dist_pkgmodule_DATA): Remove src/web/server/fiberized.scm.
(dist_webmodule_DATA, nodist_webobject_DATA): New variables.
(CLEANFILES): Add $(nodist_webobject_DATA).
* examples/random-jobs.scm (random-derivation): Add 'suffix' parameter
and honor it.
(make-random-jobs): Pass a non-empty suffix to 'random-derivation'.
Clément Lassieur
Mathieu Othacehe
Ludovic Courtès
Ludovic Courtès
Danny Milosavljevic