* src/cuirass/database.scm (sqlite-exec): Use (ice-9 format).
(db-update-build-status!): Add #:log-file parameter and honor it.
* tests/database.scm ("database")["db-update-build-status!"]: Test it.
Fixes a regression introduced in
d7306a4f48 whereby 'register' would pass
the empty list as #:outputs to 'db-add-build'.
* src/cuirass/base.scm (build-packages)[register]: Always include
outputs.
Previously STORE would be closed on the first context switch, before
'restart-builds' has completed, leading to an error (writing to a closed
port).
* src/cuirass/base.scm (restart-builds): Remove 'store' parameter. Wrap
body in 'with-store'.
* bin/cuirass.in (main): Adjust accordingly.
* src/cuirass/http.scm (url-handler): Add /api/queue handler.
* tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename
BUILD and DERIVATION accordingly.
("/build/2", "/build/2/log/raw"): Rename to /42.
("/api/queue?nr=100"): New test.
Now the database is updated as things are built, rather than after the
whole batch of derivation builds has completed.
* src/cuirass/base.scm (handle-build-event): Call
'db-update-build-status!'.
(build-packages)[register]: Set #:starttime and #:stoptime to 0.
Set #:status to 'scheduled'.
Define 'build-ids' and use it to determine the overall result.
* src/cuirass/database.scm (build-status): Add 'scheduled' and
'started'.
(db-add-build): Make sure #:timestamp, #:starttime, #:stoptime, and
#:status are integers.
(db-update-build-status!): New procedure.
* tests/database.scm ("database")["db-update-build-status!"]: New test.
* src/cuirass/utils.scm (define-enumeration): New macro.
* src/cuirass/database.scm (build-status): New macro.
* src/cuirass/base.scm (build-packages)[hydra-build-status]: Remove.
Use the 'build-status' macro instead.
* README: Mark Fibers as required.
* configure.ac: Check for Guile 2.2 only. Check for (fibers).
* bin/cuirass.in (main): Use (fibers). Run 'process-specs' and web
server in separate fibers.
* src/cuirass/base.scm (with-store): New macro.
(non-blocking-port): New procedure.
(evaluate): Use 'non-blocking-port'. Use 'read-string' followed by 'read'.
(process-specs): Move 'db-add-stamp' right after 'string=?' comparison.
Run evaluation and subsequent builds in a separate fiber.
* src/cuirass/http.scm (run-cuirass-server): Pass 'fibers as the second
argument to 'run-server'. Use 'log-message' instead of 'format'.
* src/cuirass/database.scm (with-database): Remove 'dynamic-wind'.
* src/cuirass/logging.scm: New file.
* Makefile.am (dist_pkgmodule_DATA): Add it.
* src/cuirass/base.scm (handle-build-event): Use 'log-message' instead
of 'log'.
* src/cuirass/base.scm (prepare-git): New procedure. Body moved from...
(process-specs): ... here. Remove it.
* bin/cuirass.in (main): Call 'prepare-git'.
* src/cuirass/base.scm (%newline): New variable.
(build-event-output-port, handle-build-event): New procedures.
(build-packages): Use 'handle-build-event'.
This moves log handling responsibility to 'guix publish'.
* src/cuirass/http.scm (handle-log-request): Remove.
(url-handler): Change /log/raw URI handler to return 302 to /log/OUTPUT.
* tests/http.scm (log-file-name): Remove, and remove code to create and
delete it.
("fill-db"): Change #:log value.
("/build/1/log/raw"): Expect 302.
* src/cuirass/database.scm (db-get-builds): Make 'order' a separate
filter. Add 'format-limit-clause'. Reverse OUTPUTS.
* tests/database.scm (make-dummy-eval, make-dummy-derivation)
(make-dummy-build): New procedures.
(with-temporary-database): New macro.
("database"): Use 'make-dummy-build'.
("db-get-builds"): New test.
'GIT_SSL_CAINFO' designates a single-file certificate bundle, not a
directory; thus it must be passed as the second argument to
'set-tls-certificate-locations!'.
* src/cuirass/base.scm (process-specs): Pass $SSL_CERT_DIR as the first
argument to 'set-tls-certificate-locations!', and $GIT_SSL_CAINFO or
$SSL_CERT_FILE as the second argument.
* doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API.
* src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it
object->json-string.
(object->json-scm): Move it utils.scm.
(handle-*-request): New helpers procedures.
(request-parameters): New procedure to parse a request query.
(url-handler): Add new API's.
* src/cuirass/utils.scm (object->json-scm, object->json-string): Exported
procedures moved from http.scm.
* tests/http.scm: Add various tests on new HTTP API.
integration.
* bin/evaluate.in (fill-job): New procedure.
(main): Use it to fill informations (nix-name, system) that will later be
added to database.
* doc/cuirass.texi (Database)[Derivation]: Add system and nix_name fields.
(Database)[Builds]: Add id, status, timestamp, starttime and stoptime
fields. Remove output field.
(Database)[Outputs]: New table describing the build outputs.
* src/cuirass/base.scm (build-packages): Add new fields to build object before
adding it to database.
* src/cuirass/database.scm (db-get-build, db-get-builds): New procedures to get
a build by id from database and a list of builds using filter parameters
respectively.
* src/schema.sql (Outputs) : New table.
(Derivations): Add system and nix_name columns.
(Builds): Remove output column and add id, status, timestamp, starttime and
stoptime columns.
When current-processor-count is used without (ice-9 threads) being used, Guile
complains with the following warning:
Import (ice-9 threads) to have access to `current-processor-count'.
* src/cuirass/base.scm: Use (ice-9 threads).
* src/cuirass/base.scm (report-git-error): New procedure.
(with-git-error-handling): New macro.
(process-specs): Use with-git-error-handling to catch and report git errors.
* build-aux/guix.scm (package)[inputs]: Add guile-git.
* configure.ac: Check for (git) module. Also check that (git) exports
git-error-message procedure.
* src/cuirass/base.scm (&evaluation-error): New condition type.
(evaluate): Report an &evaluation-error if eof-object? is true on
data read from port. Otherwise, suppose that data are correct and keep things
going.
(process-specs): Catch &evaluation-error, report the error in the log and keep
going.
* src/cuirass/database.scm (db-get-stamp, db-add-stamp): Replace #:id
by #:name. There is no #:id in specifications. Plus, the primary key
of Specifications is #:name.
* src/cuirass/base.scm (copy-repository-cache) : New procedure.
(fetch-repository): Use latest-repository-commit to fetch git
repository instead of raw git system commands.
(process-specs): Use fetch-repository to get a store directory
containing the repository described in SPEC, add copy it to cache with
"copy-repository-cache".
* configure.ac: Check (guix git) module presence.
* src/cuirass/utils.scm (mkdir-p, make-user-module,
call-with-temporary-directory, with-directory-excursion): Remove because
already defined in guix.
* tests/utils (with-directory-excursion): Remove associated test.
* src/cuirass/base.scm: Use (guix build utils) to provide procedure removed
from (cuirass utils).
* bin/evaluate.in: Ditto.
* bin/cuirass.in: Use "make-user-module" provided by (guix ui).
* src/cuirass/repo.scm (git-repo): Remove it.
* tests/repo.scm: Remove related tests.
git-repo is not used and it's usecases will be covered by (guix git).
Fixes a regression introduced in
a42cf16fec whereby build logs would go the
Cuirass' stderr.
* src/cuirass/base.scm (build-packages): Parameterize
'current-build-output-port' to a void output port.
This improves parallelism when 'max-jobs' is large.
* src/cuirass/base.scm (build-packages): Rewrite to pass the complete
list of derivations to 'build-derivations' at once. Handle multiple outputs.
* src/schema.sql (Builds): Make "output" part of the primary key.
Co-authored-by: Mathieu Lirzin <mthl@gnu.org>
