Reported by Danny Milosavljevic.
* src/cuirass/http.scm (url-handler) <"jobsets"> Return the result of
'db-get-specifications' as-is, not just the car.
Works around <https://github.com/wingo/fibers/issues/19>.
The effect in practice would be that we'd usually not reach the
'close-pipe' call in 'evaluate', leaving zombie processes behind us,
never executing the continuation, and additionally spinning fast on a
sequence of epoll_wait/epoll_ctl calls.
* src/cuirass/base.scm <top level>: Monkey-patch (fibers internal).
We used to have 'build-derivations' write to the custom binary port
returned by 'build-event-output-port'. However, custom binary ports
constitute continuation barriers, thereby preventing fibers from being
suspended.
To make build log processing non-blocking, we therefore invert this
inversion of control and use a suspendable I/O procedure,
'read-line/non-blocking', when reading the build log.
* src/cuirass/base.scm (read-line/non-blocking, process-build-log)
(build-derivations&): New procedures.
(%newline, build-event-output-port): Remove.
(spawn-builds): Use 'build-derivations&' instead of 'build-derivations'
with 'build-event-output-port'.
This should placate 'guix-hydra-queued-builds', for instance, which
checks these values to choose between "Running" and "Scheduled".
* src/cuirass/http.scm (build->hydra-build): Provide the correct value
for #:finished and #:busy.
Fixes a bug whereby 'build-event-output-port' would sometimes read more
than COUNT from BV, which would usually result in a 'decoding-error'
exception from 'utf8->string'.
* src/cuirass/base.scm (build-event-output-port)[write!]: Use
'bytevector-range' to honor COUNT and OFFSET.
Previously these calls would effectively suspend execution of all the
fibers until they had completed.
* src/cuirass/base.scm (process-specs): Wrap 'fetch-repository' and
'compile' calls in 'non-blocking'.
* src/cuirass/logging.scm (call-with-time-logging): New procedure.
(with-time-logging): New macro.
* src/cuirass/http.scm (handle-builds-request): Use it.
* src/cuirass/base.scm (handle-build-event)[valid?]: New procedure.
Use it when handling 'build-started', 'build-succeeded', and
'build-failed' events.
* src/cuirass/database.scm (db-get-builds)[format-order-clause]: Add
'order' clauses.
* src/cuirass/http.scm (url-handler): Default to (order finish-time) for
/latestbuilds and (order submission-time) for /queue.
This works around a scalability issue in guix-daemon when passing a long
list of derivations to 'build-derivations'.
* src/cuirass/base.scm (spawn-builds): New procedure.
(restart-builds, build-packages): Use it.
* src/cuirass/database.scm (sqlite-exec): Use (ice-9 format).
(db-update-build-status!): Add #:log-file parameter and honor it.
* tests/database.scm ("database")["db-update-build-status!"]: Test it.
Fixes a regression introduced in
d7306a4f48 whereby 'register' would pass
the empty list as #:outputs to 'db-add-build'.
* src/cuirass/base.scm (build-packages)[register]: Always include
outputs.
Previously STORE would be closed on the first context switch, before
'restart-builds' has completed, leading to an error (writing to a closed
port).
* src/cuirass/base.scm (restart-builds): Remove 'store' parameter. Wrap
body in 'with-store'.
* bin/cuirass.in (main): Adjust accordingly.
* src/cuirass/http.scm (url-handler): Add /api/queue handler.
* tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename
BUILD and DERIVATION accordingly.
("/build/2", "/build/2/log/raw"): Rename to /42.
("/api/queue?nr=100"): New test.
Now the database is updated as things are built, rather than after the
whole batch of derivation builds has completed.
* src/cuirass/base.scm (handle-build-event): Call
'db-update-build-status!'.
(build-packages)[register]: Set #:starttime and #:stoptime to 0.
Set #:status to 'scheduled'.
Define 'build-ids' and use it to determine the overall result.
* src/cuirass/database.scm (build-status): Add 'scheduled' and
'started'.
(db-add-build): Make sure #:timestamp, #:starttime, #:stoptime, and
#:status are integers.
(db-update-build-status!): New procedure.
* tests/database.scm ("database")["db-update-build-status!"]: New test.
* src/cuirass/utils.scm (define-enumeration): New macro.
* src/cuirass/database.scm (build-status): New macro.
* src/cuirass/base.scm (build-packages)[hydra-build-status]: Remove.
Use the 'build-status' macro instead.
* README: Mark Fibers as required.
* configure.ac: Check for Guile 2.2 only. Check for (fibers).
* bin/cuirass.in (main): Use (fibers). Run 'process-specs' and web
server in separate fibers.
* src/cuirass/base.scm (with-store): New macro.
(non-blocking-port): New procedure.
(evaluate): Use 'non-blocking-port'. Use 'read-string' followed by 'read'.
(process-specs): Move 'db-add-stamp' right after 'string=?' comparison.
Run evaluation and subsequent builds in a separate fiber.
* src/cuirass/http.scm (run-cuirass-server): Pass 'fibers as the second
argument to 'run-server'. Use 'log-message' instead of 'format'.
* src/cuirass/database.scm (with-database): Remove 'dynamic-wind'.
