This turns 'cuirass remote-worker' into a fiberized program instead of a
multi-process program (previously 'cuirass remote-worker' would create
one child process per actual "worker").
* src/cuirass/remote.scm (send-log): Pass SOCK_CLOEXEC | SOCK_NONBLOCK
to 'socket'. Remove 'select' call.
* src/cuirass/scripts/remote-worker.scm (spawn-worker-ping): Replace
'call-with-new-thread' by 'spawn-fiber'.
(start-worker): Replace 'primitive-fork' by 'spawn-fiber'.
(worker-management-thunk): New procedure.
(%worker-pids, add-to-worker-pids!): Remove.
(signal-handler): Adjust accordingly.
(cuirass-remote-worker): Define 'management-channel'. Spawn
a fiber running 'worker-management-thunk'. Create workers by sending
message to MANAGEMENT-CHANNEL.
This turns 'cuirass remote-server' into a fiberized program, as opposed
to a heavy-handed multi-threaded program.
* src/cuirass/remote.scm (receive-logs): Rewrite in Fibers style,
assuming non-blocking I/O calls and using 'spawn-fiber' instead of
'call-with-new-thread'.
(zmq-socket->port): New procedure.
(receive-message)[wait]: New procedure.
Call it upfront.
* src/cuirass/notification.scm (start-notification-thread): Rename to...
(spawn-notification-fiber): ... this. Use 'spawn-fiber' instead of
'call-with-new-thread'. Use (@ (fibers) sleep) instead of 'sleep'.
* src/cuirass/scripts/remote-server.scm (ensure-non-blocking-store-connection):
New procedure.
(add-to-store): Use it.
(start-fetch-worker): Use 'spawn-fiber' instead of
'call-with-new-thread'. Remove 'set-thread-name' call.
(start-periodic-updates-thread): Rename to...
(spawn-periodic-updates-fiber): ... this. Use 'spawn-fiber' instead of
'call-with-new-thread'. Remove 'set-thread-name' call.
(zmq-start-proxy): Remove 'socket-ready?', '%loop-timeout', and
'poll-items'. Use 'spawn-fiber' for the worker list update. Remove use
of 'zmq-poll*' in the main loop.
(terminate-helper-processes): New procedure.
(signal-handler): Use it instead of inline code. Call 'primitive-exit'
rather than 'exit'.
(cuirass-remote-server): Wrap body in 'run-fibers'.
* tests/database.scm ("database")["db-init"]: Remove
'start-notification-thread' call.
["mail notification", "mail notification, broken job"]: Add call to
'spawn-notification-fiber'.
This hides serialization/deserialization, assembly of message parts, and
the actual send/receive operation behind 'send-message' and
'receive-message'.
* src/cuirass/remote.scm (zmq-remote-address)
(zmq-message-string, zmq-read-message): Remove.
(send-message, receive-message): New procedures.
* src/cuirass/remote.scm (build-request-message):
(no-build-message, build-started-message)
(build-failed-message, build-succeeded-message)
(worker-ping, worker-ready-message)
(worker-request-work-message)
(worker-request-info-message, server-info-message): Remove 'format'
call and return an sexp instead.
* src/cuirass/scripts/remote-server.scm (read-worker-exp):
Add #:peer-address. Change 'msg' to 'sexp'.
(need-fetching?): Remove call to 'zmq-read-message'. Remove
inappropriate use of 'else' keyword.
(run-fetch): Remove call to 'zmq-read-message'. Use 'receive-message'
instead of 'zmq-message-receive*' & co.
(zmq-start-proxy): Use 'receive-message' and 'send-message' instead of
'zmq-message-receive*', 'zmq-message-send' & co. Pass #:peer-address to
'read-worker-exp'.
* src/cuirass/scripts/remote-worker.scm (run-command): Remove call to
'zmq-read-message'.
(spawn-worker-ping)[ping]: Use 'send-message'.
(start-worker): Use 'send-message' and 'receive-message' instead of
the whole shebang.
This partially reverts bb7579acc0, which
introduced an incorrect membership test with 'memq'.
* src/cuirass/scripts/remote-server.scm (zmq-start-proxy)[socket-ready?]:
New procedure.
Use it when checking the result of 'zmq-poll*'.
* src/cuirass/scripts/remote-server.scm (%options): Fix program name
passed as a 'show-version-and-exit' argument.
* src/cuirass/scripts/remote-worker.scm (%options): Likewise.
This is a followup to c7db7f88a2, which
would only show, say, 'guix-past' and not 'guix' (the latter being a
dependency of the former).
* src/cuirass/http.scm (dashboard-page): Call 'latest-checkouts' and
pass #:checkouts to 'evaluation-dashboard'.
* src/cuirass/templates.scm (evaluation-dashboard): Add #:checkouts and
honor it.
* src/cuirass/http.scm (url-handler)[redirect]: New procedure.
Add clause for /eval/latest/dashboard.
* src/cuirass/templates.scm (specifications-table): Link to it.
* src/cuirass/base.scm (spawn-builds): Add 'set-build-options' call.
Previously, we'd build with #:keep-going? #false, which made Cuirass
build very little when not using the "remote worker" mechanism.
Previously, an error such as 'system-error would be uncaught, which
would (presumably) lead 'cuirass register' to exit right away, via
'essential-task'.
* src/cuirass/base.scm (process-specs): Use 'with-exception-handler' +
'let/ec' instead of 'catch'. Report 'system-error' exceptions in
detail. Print other exceptions as well.
Fixes a regression introduced in
f1f0489ed7 where 'error-level messages
would never be logged.
* src/cuirass/logging.scm (log-message): Always log errors.
These primitives are no longer used. They're also antithetical to the
"communicating sequential processes" (CSP) model where communication is
synchronous and lack of a recipient or sender can lead to lockups.
* src/cuirass/utils.scm (with-timeout)
(put-message-with-timeout, get-message-with-timeout): Remove.
With 'exec-query' from (squee) no longer blocking, we no longer need to
carry out 'exec-query' calls in a dedicated worker thread. However, it
is useful to have a pool of database connections at hand to allow for
concurrent queries.
Thanks to Christopher Baines for suggesting the use of a connection pool.
* src/cuirass/database.scm (%db-channel): Remove.
(%db-connection-pool, %db-connection-pool-size): New variables.
(with-database): Rewrite to use 'make-resource-pool' when in a fiber
context.
(current-db): New variable.
(with-db-worker-thread): Rewrite to use 'current-db' or
'with-resource-from-pool' when appropriate, and plain 'db-open'
otherwise.
* tests/common.scm (test-init-db!): Remove reference to '%db-channel'.
* tests/database.scm (with-fibers): New macro.
Use it throughout.
* configure.ac: Check whether Guile-Squee is recent enough.
