guix-cuirass/doc/cuirass.texi

\input texinfo
@setfilename cuirass.info
@documentencoding UTF-8
@include version.texi
@settitle Cuirass Reference Manual
@setchapternewpage odd

This manual is for Cuirass version @value{VERSION}, a build automation
server.

@copying
Copyright @copyright{} 2016, 2017 Mathieu Lirzin@*
Copyright @copyright{} 2017, 2020, 2021 Mathieu Othacehe@*
Copyright @copyright{} 2018 Ludovic Courtès@*
Copyright @copyright{} 2018 Clément Lassieur

@quotation
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A
copy of the license is included in the section entitled ``GNU Free
Documentation License''.
@end quotation
@end copying

@dircategory Software development
@direntry
* Cuirass: (cuirass).       Build automation server.
@end direntry

@titlepage
@title Cuirass Reference Manual
@subtitle Build automation server
@subtitle for version @value{VERSION}, @value{UPDATED}
@author The Cuirass Developers

@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents

@ifnottex
@node Top
@top Cuirass
@insertcopying
@end ifnottex

@c *********************************************************************
@menu
* Introduction::                What is Cuirass about?
* Specifications::              Writing Cuirass specifications.
* Notifications::               Build notifications.
* Parameters::                  Cuirass parameters.
* Build modes::                 Build modes.
* Invocation::                  How to run Cuirass.
* Web API::                     Description of the Web API.
* Database::                    About the database schema.

* Contributing::                Your help needed!
* GNU Free Documentation License::  The license of this manual.
* Concept Index::               Concepts.

@end menu

@c *********************************************************************
@node Introduction
@unnumbered Introduction
@cindex introduction

@dfn{Cuirass} is a general-purpose build automation server that checks
out source files from @acronym{VCS, Version Control System}
repositories, executes build jobs, and stores build results in a
database.  It provides a web interface to monitor the build results,
as well as an HTTP API. Cuirass is also able to send build
notifications using different mechanisms such as RSS and email.

Cuirass is inspired by the @url{https://nixos.org/hydra/, Hydra}
continuous build system. Unlike Hydra, it is built on top of the
@url{https://www.gnu.org/software/guix/, GNU Guix} functional package
manager.

The goal of Cuirass is to prevent software regressions by building a
set of package definitions, system images and running periodical tests
for various architectures.  Cuirass is also responsible for GNU Guix
binary substitutes production (@pxref{Substitutes, Substitutes,, guix,
Guix}).

Cuirass is deployed on the GNU Guix build farm at
@url{https://ci.guix.gnu.org}.  It is also common for Guix users to
run their own Cuirass instance to build different sources, using
different priorities (@pxref{Continuous Integration, Continuous
Integration,, guix, Guix}).

@c *********************************************************************
@node Specifications
@chapter Specifications
@cindex cuirass specifications

The main Cuirass argument is the @var{specification} file.  It
describes the repositories that must be used, the build jobs and their
priorities between other things.

@deftp {Data Type} specification
@table @asis
@item @code{name}
The specification name as a Scheme symbol.

@item @code{build} (default: @code{all})
The packages to be built by Cuirass.  It defaults to @code{all}, which
means that all the discovered packages in the subsequent @code{channels}
field are to be selected.

It is also possible to set this field to:

@itemize
@item @code{core}
Build only the core packages such as @code{gcc}, @code{guile} and
@code{glibc}.

@item @code{guix}
Build only the Guix modules that are involved in the @command{guix
pull} command.

@item @code{hello}
Build only the hello package.

@item @code{images}
Build only the Guix System images.

@item @code{system-tests}
Build only the Guix System tests.

@item @code{tarball}
Build only the Guix binary tarball.

@item @code{(channels . list)}
Build only the packages that are part of the given channel
@code{list}.  For instance, @code{(channels my-channel)} will only
build the packages that are part of @code{my-channel} channel.

@item @code{(packages . list)}
Build only the specified packages in @code{list}.  For instance,
@code{(packages "strace" "perf")} will only build the packages
@code{strace} and @code{perf}.

@item @code{(manifests . list)}
Build only the packages that are part of the manifests @code{list}.
For instance, @code{(manifests "etc/manifest")} will only build the
packages that are part of the @code{etc/manifest} file.  This file
must be provided by exactly one of the channels defined below.

@end itemize

@item @code{channels} (default: @code{(list %default-guix-channel)})
The channels to be fetched by Cuirass (@pxref{Channels, Channels,,
guix, Guix}).

@item @code{build-outputs} (default: @code{()})
The build artifacts that must be saved and proposed to download in the
web interface as a list of @code{build-outputs} records.

@deftp {Data Type} build-outputs
@table @asis
@item @code{job}
Save the build outputs of the build jobs which names match the
@code{job} regexp.

@item @code{type}
The build output type as a string.  It is only used to describe the
build output in the web interface.

@item @code{output} (default: @code{("out")})
The job output if it has multiple outputs (@pxref{Packages with
Multiple Outputs, Packages with Multiple Outputs,, guix, Guix}).

@item @code{path}
The build output path within the job, as a string.

@end table
@end deftp

For instance, let's consider the @code{binary-tarball.x86_64-linux}
job which produces the following output:
@code{/gnu/store/xxx-guix-binary.tar.xz}.

The build output definition below will save the root element
(@code{""}) of the @code{"out"} output of the
@code{"binary-tarball.x86_64-linux"} job---i.e., the
@code{"xxx-guix-binary.tar.xz"} file.

@lisp
(build-output
 (job "binary-tarball*")
 (type "archive")
 (output "out")
 (path ""))
@end lisp


@item @code{notifications} (default: @code{()})
The list of build notifications that must be sent.  For instance:

@lisp
(list (email
       (from "build@@cuirass.org")
       (to "notification@@myself.org")
       (server "sendmail:///etc/my-mailer.sh")))
@end lisp

will send build notifications emails from @code{build@@cuirass.org} to
@code{notifications@@myself.org}, using
@code{"sendmail:///etc/my-mailer.sh"} mailer.

The different notification types are described in the
@ref{Notifications} section.

@item @code{period} (default: @code{0})
When @code{period} is strictly superior to zero new evaluations will
only be triggered if the latest evaluation occured more than
@code{period} seconds ago.  Otherwise, the specification will be
evaluated for each new commit in the declared @code{channels}.

@item @code{priority} (default: @code{9})
The specification priority relatively to the other specifications, as
an integer ranging from 0 to 9 where 0 is the higher priority and 9
the lowest.

@item @code{systems} (default: @code{(list (%current-system))})
Build every job for each system in this list.  By default only the
current system is selected.

@end table
@end deftp

@c *********************************************************************
@node Notifications
@chapter Notifications
@cindex cuirass notifications

Cuirass supports different build notifications types, that can be
passed in the @code{notifications} field of the specification record,
see @ref{Specifications}.

Cuirass sends build notifications each time a build is broken or
fixed.

@section Email
Email build notifications can be enabled using the following record.

@deftp {Data Type} email
@table @asis
@item @code{from}
The email @code{From} field, as a string.

@item @code{to}
The email @code{To} field, as a string.

@item @code{server}
The mail server connection string.  Cuirass uses the @code{mailutils}
package.  Hence the server can be specified as a remote SMTP mailbox
(@pxref{SMTP Mailboxes, SMTP Mailboxes,, mailutils, Mailutils}) or as
a program mailbox (@pxref{Program Mailboxes, Program Mailboxes,,
mailutils, Mailutils}).

@end table
@end deftp

@section Mastodon
Mastodon build notifications can be enabled using the following record.

@deftp {Data Type} mastodon
@end deftp

The Mastodon credentials must be defined as Cuirass parameters, see
@ref{Parameters}.

@section RSS
Cuirass is proposing a build notification RSS feed at the following
URL:

@itemize
@item @url{http://cuirass-url/events/rss[?specification=spec]}
By default build notifications are sent for all specifications.  If
the @code{specification} argument is passed, they can be restricted to
the @var{spec} specification.

@end itemize

@c *********************************************************************
@node Parameters
@chapter Parameters
@cindex cuirass parameters

Cuirass is able to connect to different external services such as
@code{postgresql} for the database, @code{zabbix} for machine
monitoring and @code{mastodon} for build notifications.  As those
services often require using secret credentials, Cuirass can be passed
a parameter file.

The parameters file can be passed using the @code{parameters} command
line argument, see @ref{Invocation}.

Here's an example parameter file:

@example
(%cuirass-url "https://ci.guix.gnu.org")
(%zabbix-url "http://127.0.0.1:15412/api_jsonrpc.php")
(%mastodon-instance-name "My Instance")
(%mastodon-instance-url "https://instance.org")
(%mastodon-instance-token "secret-token")
@end example

@deftp {Parameters} parameters
@table @asis
@item @code{%cuirass-database} (default: @code{"cuirass"})
The Cuirass PostgreSQL database name.

@item @code{%cuirass-host} (default: @code{"/var/run/postgresql"})
The Cuirass PostgreSQL database host.

@item @code{%cuirass-url} (default: @code{#f})
The URL of the Cuirass web server.  This is useful to send absolute
links within notifications.

@item @code{%zabbix-url} (default: @code{#f})
The URL of the Zabbix monitoring server providing the workers status,
if supported.

@item @code{%zabbix-user} (default: @code{"Admin"})
The user for Zabbix API authentication.

@item @code{%zabbix-password} (default: @code{"zabbix"})
The password for Zabbix API authentication.

@item @code{%mastodon-instance-name} (default: @code{#f})
The name of the Mastodon instance used to send build notifications.

@item @code{%mastodon-instance-url} (default: @code{#f})
The URL of the Mastodon instance.

@item @code{%mastodon-instance-token} (default: @code{#f})
The token used to authenticate on the Mastodon instance.

@end table
@end deftp

@c *********************************************************************
@node Build modes
@chapter Build modes
@cindex cuirass build modes

Cuirass supports two mechanisms to build derivations.

@section With the local Guix daemon
This is the default build mechanism.  Once the build jobs are
evaluated, they are sent to the local Guix daemon.  Cuirass then
listens to the Guix daemon output to detect the various build events.

While this mode doesn't require any particular configuration, it
doesn't scale well.  The scheduling decisions of the Guix daemon are
opaque and often suboptimal.

When Cuirass is used to build a large amount of jobs, the remote build
mechanism described below should be preferred.

@section With the remote build mechanism.

This mode is harder to setup but scales way better. This is the build
mode that is used on the GNU Guix build farm at
@url{https://ci.guix.gnu.org}.  The build jobs are not submitted to
the local Guix daemon.  Instead, a remote server dispatches build
requests to the connect remote workers, according to the build
priorities.

The remote server and the connected workers communicate using ZMQ over
TCP.  The workers are able to discover the remote server using Avahi.

The built items are exchanged as substitutes (@pxref{Substitutes,
Substitutes,, guix, Guix}) by spawning Guix publish servers both on
the remote server and on each connected remote worker.

It can be enabled this way:

@itemize
@item
Start the @code{cuirass register} process with the @code{build-remote}
command line argument, see @ref{Invocation}.  This way, the
registration process does not submit the new build jobs to the local
Guix daemon.

@item
Start the @code{cuirass remote-server} process to dispatch the build
jobs to the connected workers.

@item
Start at least one @code{cuirass remote-worker} process on any machine
of the local network to actually perform the builds and report their
status.

@end itemize

Note that some Cuirass features are only available when using this
build mode. That's the case for:

@itemize
@item
The build priority support.

@item
The notification mechanism, see @ref{Notifications}.

@item
The transmission of @code{timeout} and @code{max-silent-time} package
properties to the Guix daemon.

@item
The live build log mechanism of the Web interface.

@item
The workers status page of the Web inferface accessible at
@url{http://cuirass-url/workers}.

@end itemize

The easiest way to setup such an infrastructure is to rely on the GNU
Guix Cuirass services definitions (@pxref{Continuous Integration,
Continuous Integration,, guix, Guix}).

@c *********************************************************************
@node Invocation
@chapter Invocation

@section Invoking cuirass register
@cindex register

The usual way to invoke @code{cuirass} registration process is as follows:

@example
cuirass register --specifications @var{specs}
@end example

This starts a Cuirass registration instance building @var{specs} and
storing the results using the default PostgreSQL database.

Additionally the following options can be used.

@table @code
@item --one-shot
Instead of executing @code{cuirass} as a daemon looping over the jobs.
Only evaluate and build the specifications once.

@item --cache-directory=@var{directory}
@var{directory} is the place where the VCS repositories used by the jobs
are stored.

@item --specifications=@var{specifications-file}
@itemx -S @var{specifications-file}
Add the specifications defined in @var{specifications-file} in the job
database before launching the evaluation and build processes.

@item --database=@var{database}
@itemx -D @var{database}
Use @var{database} as the database containing the jobs and the past
build results. Since Cuirass uses PostgreSQL as a database engine,
@var{database} must be a string such as @code{"dbname=cuirass
host=localhost"}.  By default, Cuirass uses the following connection
string: @code{dbname=cuirass host=/var/run/postgresql"}.

@item --parameters=@var{parameters-file}
@itemx -P @var{parameters-file}
Read parameters from the given @var{parameters-file}.  The supported
parameters are described here (@pxref{Parameters}).

@item --ttl=@var{duration}
Cuirass registers build results as garbage collector (GC) roots, thereby
preventing them from being deleted by the GC.  The @option{--ttl} option
instructs it to keep those GC roots live for at least @var{duration}---e.g.,
@code{1m} for one month, @code{2w} for two weeks, and so on.  The default is
30 days.

Those GC roots are typically stored in
@file{/var/guix/gcroots/profiles/per-user/@var{user}/cuirass}, where @var{user} is the
user under which Cuirass is running.

@item --interval=@var{n}
@itemx -I @var{n}
Wait @var{n} seconds between each poll.

@item --use-substitutes
This can be useful when you are not interested in building the
dependencies of a particular job.

@item --threads=@var{n}
Use up to @var{n} kernel threads.

@var{n} should be lower than or equal to the number of CPU cores on the
machine.  In general though, having a large @var{n} is not very useful
since the work of Cuirass is primarily I/O-bound---on the contrary,
large values of @var{n} may increase overhead.  The default value should
be appropriate for most cases.

@item --version
@itemx -V
Display the actual version of @code{cuirass}.

@item --help
@itemx -h
Display an help message that summarize all the options provided.
@end table

@section Invoking cuirass web
@cindex web

The usual way to invoke the @code{cuirass} web server is as follows:

@example
cuirass web
@end example

This starts a Cuirass web server on the default port. Additionally the
following options can be used.

@table @code
@item --database=@var{database}
@itemx -D @var{database}
Use @var{database} as the database containing the jobs and the past
build results. Since Cuirass uses PostgreSQL as a database engine,
@var{database} must be a string such as @code{"dbname=cuirass
host=localhost"}.  By default, Cuirass uses the following connection
string: @code{dbname=cuirass host=/var/run/postgresql"}.

@item --parameters=@var{parameters-file}
@itemx -P @var{parameters-file}
Read parameters from the given @var{parameters-file}.  The supported
parameters are described here (@pxref{Parameters}).

@item --port=@var{num}
@itemx -p @var{num}
Make the HTTP interface listen on port @var{num}.  Use port 8080 by
default.

@item --listen=@var{host}
Make the HTTP interface listen on network interface for @var{host}.  Use
localhost by default.

@item --version
@itemx -V
Display the actual version of @code{cuirass}.

@item --help
@itemx -h
Display an help message that summarize all the options provided.
@end table

@section Invoking cuirass remote-server
@cindex remote-server

The @code{remote-server} command starts a daemon that is able to
communicate with @code{remote-worker} processes.  Its role is to
answer build requests from the workers, by sending back derivations
that must be built.

On build completion it updates the database accordingly and possibly
fetches build substitutes.  The @code{remote-server} and
@code{remote-worker} processes communicate using ZMQ over TCP.

Additionally the following options can be used.

@table @code
@item --backend-port=@var{port}
The TCP port for communicating with @code{remote-worker} processes
using ZMQ.  It defaults to @code{5555}.

@item --log-port=@var{port}
The TCP port of the log server.  It defaults to @code{5556}.

@item --publish-port=@var{port}
The TCP port of the publish server.  It defaults to @code{5557}.

@item --parameters=@var{parameters-file}
@itemx -P @var{parameters-file}
Read parameters from the given @var{parameters-file}.  The supported
parameters are described here (@pxref{Parameters}).

@item --database=@var{database}
@itemx -D @var{database}
Use @var{database} PostgreSQL connection string.

@item --cache=@var{directory}
Use @var{directory} to cache build log files.

@item --trigger-substitute-url=@var{URL}
Once a substitute is successfully fetched, trigger substitute baking
at @var{URL}.

@item --user=@var{user}
Change privileges to @var{user} as soon as possible---i.e., once the
signing key has been read.

@item --public-key=@var{file}
@itemx --private-key=@var{file}
Use the specific @var{file}s as the public/private key pair used to sign
the store items being published.

@item --version
@itemx -V
Display the actual version of @code{cuirass}.

@item --help
@itemx -h
Display an help message that summarize all the options provided.
@end table

@section Invoking cuirass remote-worker
@cindex remote-worker

The @code{remote-worker} command starts a daemon that is able to
communicate with a @code{remote-server} process.  Its role is to
request builds to the @code{remote-server}, perform them and report
their status.

The @code{remote-worker} is able to discover a @code{remote-server}
process on the local network using Avahi and connect to it.

Additionally the following options can be used.

@table @code
@item --workers=@var{count}
Start @var{count} parallel workers.  It defaults to @code{1}.

@item --publish-port=@var{port}
The TCP port of the publish server.  It defaults to @code{5558}.

@item --server=@var{ip-address}
Do not use Avahi discovery and connect to the given
@code{remote-server} IP address.

@item --systems=@var{systems}
Only request builds for the given @var{systems}.  It defaults to
@code{(list (%current-system))}.

@item --public-key=@var{file}
@itemx --private-key=@var{file}
Use the specific @var{file}s as the public/private key pair used to sign
the store items being published.

@item --version
@itemx -V
Display the actual version of @code{cuirass}.

@item --help
@itemx -h
Display an help message that summarize all the options provided.
@end table

@c *********************************************************************
@node Web API
@chapter Web API
@cindex web api

The Cuirass web API is inspired from the Hydra one.

@section API description
@cindex description, json

@subsection Evaluation information
@subsubheading Single evaluation

It is possible to query the Cuirass web server for evaluation
information. The dedicated API is "/api/evaluation?id=@var{eval-id}"
where @var{eval-id} is the unique id associated to the evaluation in
database.

For instance, querying a local Cuirass web server can be done with
@code{curl} and @code{jq} to format the JSON response :

@example
$ curl -s "http://localhost:8080/api/evaluation?id=1" | jq

@{
  "id": 1,
  "specification": "guix-master",
  "status": 0,
  "timestamp": 1615289011,
  "checkouttime": 1615289011,
  "evaltime": 1615289655,
  "checkouts": [
    @{
      "commit": "bd311f5a6ccbd4696ac77f0426a036bb375a2f10",
      "channel": "guix",
      "directory": "/gnu/store/6978xw9vs4ybg2pc3g736p1dba2056yl-guix-bd311f5"
    @}
  ]
@}
@end example

The nominal output is a JSON object whose fields are described
hereafter.

@table @code
@item id
The unique build id.

@item specification
The associated specification name, as a string.

@item status
The evaluation status, as an integer. Possible values are :

@example
-1 -> started
0 -> succeeded
1 -> failed
2 -> aborted
@end example

@item checkouttime
The timestamp after channel checkout.

@item evaltime
The timestamp after evaluation completion.

@item checkouts
The evaluation checkouts as a JSON object.

@end table

@subsubheading Multiple evaluations

The latest evaluations list can be obtained with the API
"/api/evaluations".  The output is a JSON array of
evaluations. Evaluations are represented as in the
"/api/evaluation?id=@var{eval-id}" API.

This request accepts a mandatory parameter.
@table @code
@item nr
Limit query result to nr elements. This parameter is @emph{mandatory}.
@end table

@subsection Build information

It is possible to query Cuirass web server for build informations. The
dedicated API is "/build/@var{build-id}" where @var{build-id} is the
unique id associated to the build in database.

The build information can also be queried by output. For example,
@samp{/output/kg9mirg6xbvzcp0a98v7326n1nvvwgsj-hello-2.10} will return
the details of the output, along with the build if available.

@example
$ curl -s "http://localhost:8080/build/2" | jq

@{
  "id": 2,
  "jobset": "guix",
  "job": "acpica-20150410-job",
  "timestamp": 1501347493,
  "starttime": 1501347493,
  "stoptime": 1501347493,
  "buildoutputs": @{
    "out": @{
      "path": "/gnu/store/6g3njhfzqpdm335s7qhvmwvs5l7gcbq1-acpica-20150410"
    @}
  @},
  "system": "x86_64-linux",
  "nixname": "acpica-20150410",
  "buildstatus": 0,
  "weather": 0,
  "busy": 0,
  "priority": 0,
  "finished": 1,
  "buildproducts": null
@}
@end example

If requested @var{build-id} is not known, the HTTP code 404 is
answered with a JSON error message. For example:

@example
$ curl -s "http://localhost:8080/build/fff"

@{"error" : "Build with ID fff doesn't exist."@}
@end example

The nominal output is a JSON object whose fields are described
hereafter.

@table @code
@item id
The unique build id.

@item jobset
The associated specification name, as a string.

@item job
The associated job-name, as a string.

@item timestamp
Timestamp taken at build creation time.

@item starttime
Timestamp taken at build start time.

@item stoptime
Timestamp taken at build stop time.

@item buildoutputs
Build outputs as a JSON object. The keys names are referring to the
eventual output names. The associated value is another JSON object which
only key is @code{path}. @code{path} value is the output directory in
store as a string.

@item system
System name of the build, as a string.

@item nixname
Derivation name, as a string.

@item buildstatus
Build status, as an integer. Possible values are :

@example
0 -> succeeded
1 -> failed
2 -> failed dependency
3 -> failed other
4 -> cancelled
@end example

@item weather
Build weather, as an integer.

@example
-1 -> unknown
0 -> new-success
1 -> new-failure
2 -> still-succeeding
3 -> still-failing
@end example

@item busy
Whether the build is pending, as an integer.

@item priority
Build priority, as an integer.

@item finished
Build finished, as an integer.

@item buildproducts
Build products in store as a JSON object.

@end table

@subsection Build raw log output

It is possible to ask Cuirass for the raw build output log with the API
"/build/@var{build-id}/log/raw" where @var{build-id} is the
unique id associated to the build in database.

The output is a raw text, for example:

@example
$ curl http://localhost:8080/build/2/log/raw

starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
...
@end example

If requested @var{build-id} is not known, the HTTP code 404 is
answered with a JSON error message. For example:

@example
$ curl -s "http://localhost:8080/build/fff/log/raw"

@{"error" : "Build with ID fff doesn't exist."@}
@end example

@subsection Jobs

The list of jobs associated with a given evaluation can be obtained
with the API "/api/jobs".  The output is a JSON array of
jobs.

This request accepts a mandatory parameter and multiple optional ones.

@table @code
@item evaluation
The evaluation id. This parameter is @emph{mandatory}.

@item names
Filter query result to jobs which names are part of the given
@code{names} list, a comma separated list of job names.

@item system
Filter query result to jobs with the given @code{system}.

@end table

For example, to ask for the jobs of evaluation @code{12} for
@code{x86_64-linux}:

@example
$ curl "http://localhost:8080/api/jobs?evaluation=12&system=x86_64-linux"
@end example

or the @code{emacs} and @code{emacs-minimal} jobs of evaluation
@code{12} for @code{x86_64-linux}:

@example
$ curl "http://localhost:8080/api/jobs?evaluation=12&names=emacs.x86_64-linux,emacs-minimal.x86_64-linux"
@end example

The nominal output is a JSON object whose fields are described
hereafter.

@table @code
@item build
The unique build id associated with the job.

@item status
The build status, as an integer.

@item name
The job name, as a string.

@end table

@subsection Latest builds

The list of latest builds can be obtained with the API
"/api/latestbuilds".  The output is a JSON array of builds. Builds are
represented as in the "/build/@var{build-id}" API.

This request accepts a mandatory parameter and multiple optional ones.

@table @code
@item nr
Limit query result to nr elements. This parameter is @emph{mandatory}.

@item jobset
Filter query result to builds with the given @code{jobset}.

@item job
Filter query result to builds with the given @code{job} name.

@item system
Filter query result to builds with the given @code{system}.

@end table

For example, to ask for the ten last builds:

@example
$ curl "http://localhost:8080/api/latestbuilds?nr=10"
@end example

or the five last builds where jobset ``guix'':

@example
$ curl "http://localhost:8080/api/latestbuilds?nr=5&jobset=guix"
@end example

If no builds matching given parameters are found, an empty JSON array is
returned.

@subsection Queued builds

The list of queued builds can be obtained with the API
"/api/queue".  The output is a JSON array of builds. Builds are
represented as in the "/build/@var{build-id}" API.

This request accepts a mandatory parameter.
@table @code
@item nr
Limit query result to nr elements. This parameter is @emph{mandatory}.
@end table

@c *********************************************************************
@node Database
@chapter Database schema
@cindex cuirass database
@cindex postgresql database
@cindex persistent configuration

Cuirass uses a PostgreSQL database to store information about jobs and
past build results, but also to coordinate the execution of jobs.

The database contains the following tables: @code{Specifications},
@code{Checkouts}, @code{Evaluations}, @code{Builds}, @code{Outputs},
@code{Metrics}, @code{BuildProducts}, @code{Events} and
@code{Workers}.  The purpose of each of these tables is explained
below.

@section Specifications
@cindex specifications, database

This table stores specifications describing the repositories from whence
Cuirass fetches code and the environment in which it will be processed.
Entries in this table must have values for the following text fields:

@table @code
@item name
This field holds the name of the specification.  This field is also the
primary key of this table.

@item channels
The channels to be fetched by Cuirass as an SEXP string.

@item build_ouputs
The build outputs to be saved by Cuirass as an SEXP string.

@item notifications
The build notifications to be sent by Cuirass as an SEXP string.

@item period
The specification periodicity as an integer.

@item priority
The specification priority relatively to the other specifications, as
an integer ranging from 0 to 9 where 0 is the higher priority and 9
the lowest.

@item systems
The systems for which build jobs must be evaluated, as a comma
separated list.

@end table

@section Checkouts
@cindex checkouts, database

When a specification is processed, the repositories must be downloaded at a
certain revision as specified.  The download is called a checkout.  The
@code{Checkouts} table stores the new checkouts for every specification when
it is being processed.

The @code{Checkouts} table has the following columns:

@table @code
@item specification
The specification associated with the checkout.

@item revision
The revision of the checkout. Within the same specification, two checkouts
can't be identical: they can't have the same revision.

@item evaluation
The evaluation that was triggered by the addition of that new checkout.

@item channel
The channel associated with the checkout.

@item directory
The directory into which the checkout was extracted.

@item timestamp
The checkout insertion timestamp.

@end table

@section Evaluations
@cindex evaluations, database

An evaluation relates a specification with the revision of the repository
specified therein.  Builds (see below) belong to a specific evaluation.

The @code{Evaluations} table has the following columns:

@table @code
@item id
This is an automatically incrementing numeric identifier.

@item specification
This field holds the @code{name} of a specification from the
@code{Specifications} table.

@item status
This integer field hold the evaluation status.  Possible values are:
@itemize
@item started (@code{-1})
@item succeeded (@code{0})
@item failed (@code{1})
@item aborted (@code{2})
@end itemize

@item timestamp
The timestamp at evaluation insertion.

@item checkout
The timestamp after channel checkout.

@item evaltime
The timestamp after evaluation completion.

@end table

@section Jobs
@cindex jobs, database

This table contains all the jobs associated with a given evaluation.
If a new job produces build outputs that are not already stored inside
the @code{Outputs} table then, it is added to the @code{Builds} table.

@table @code
@item name
This text field holds the job name.

@item evaluation
This integer field references the evaluation identifier from the
@code{Evaluations} table, indicating to which evaluation this job
belongs.

@item builds
This integer field references the build id from the @code{Builds}
table, corresponding to this job.

@item system
This text field holds the system name of the derivation.

@end table

@section Builds
@cindex builds, database

This table holds records of the derivations and their build status.  Note that
a job will be registered here only if its derivation doesn't already exist.

@table @code
@item derivation
This text field holds the absolute name of the derivation file that
resulted in this build.

@item evaluation
This integer field references the evaluation identifier from the
@code{Evaluations} table, indicating to which evaluation this build
belongs.

@item job_name
This text field holds the name of the job.

@item system
This text field holds the system name of the derivation.

@item nix_name
This text field holds the name of the derivation ---e.g.,
@code{coreutils-8.24}.

@item worker
This text field references the name of worker performing the build
from the @code{Workers} table.

@item log
This text field holds the absolute file name of the build log file.

@item status
This integer field holds the build status of the derivation.

@item last_status
This integer field holds the build status of the previous job
evaluation.

@item weather
This integer field holds the weather of the build. Possible values
are:

@itemize
@item unknown (@code{-1})
@item new-success (@code{0})
@item new-failure (@code{1})
@item still-succeeding (@code{2})
@item still-failing (@code{3})
@end itemize

@item priority
The build priority relatively to the other builds with the same
@code{job_name}, as an integer ranging from 0 to 99 where 0 is the
higher priority and 99 the lowest.

@item max_silent
This integer field holds the number of seconds of silence after which
a build process times out.

@item timeout
This integer field holds the number of seconds of activity after which
a build process times out.

@item timestamp
This integer field holds a timestamp taken at build creation time.

@item starttime
This integer field holds a timestamp taken at build start time.
Currently, it has the same value as the @code{timestamp} above.

@item stoptime
This integer field holds a timestamp taken at build stop time.
Currently, it has the same value as the @code{timestamp} above.

@end table

@section Outputs
@cindex outputs, database

This table keep tracks for every eventual build outputs. Each build
stored in @code{Builds} table may have zero (if it has failed), one or
multiple outputs.

@table @code
@item derivation
This field holds the @code{derivation} of a build from the @code{Builds}
table.

@item name
This text field holds the name of the output.

@item path
This text field holds the path of the output.

@end table

@section Metrics
@cindex metrics, database

This table contains several metrics that are recorded by the
@code{metrics} fiber periodically.

@table @code
@item field
This text field holds the application field of the metric.

@item type
This integer field holds the type of the metric.

@item path
This float field holds the value of the metric.

@item evaltime
The metric insertion timestamp.

@end table

@section BuildProducts
@cindex buildproducts, database

This table contains the saved build products, that are proposed to
download through the web interface.

@table @code
@item build
This integer field holds a reference to the build @code{id} from the
@code{Builds} table, the build product belongs to.

@item type
This text field holds the build product type.

@item file_size
This integer field holds build product size in bytes.

@item checksum
This text field holds the build product checksum.

@item path
This text field holds the build product absolute store path.

@end table

@section Notifications
@cindex notifications, database

This table contains the notifications that are queued for sending.

@table @code
@item id
This is an automatically incrementing numeric identifier.

@item type
This text field holds the SEXP representation of the notification.

@item build
This integer fields references the build id associated with the
notification.

@end table

@section Workers
@cindex workers, database

This table contains the registered workers when Cuirass is using the
remote building mechanism.

@table @code
@item name
This text field holds the worker name.  This field is also the primary
key of this table.

@item address
This text field holds the worker IP address.

@item machine
This text field holds the worker machine name.

@item systems
This text field holds the systems that are supported by the worker, as
a comma separated list of systems.

@item last_seen
This integer field holds the timestamp of the last communication with
the worker.

@end table

@c *********************************************************************

@node Contributing
@chapter Contributing

Everyone is welcome to contribute to Cuirass.  You can report bugs, send
patches and share your ideas with others by sending emails the
@email{guix-devel@@gnu.org, mailing list}.

Development is done using the Git distributed version control system.
Thus, access to the repository is not strictly necessary.  We welcome
contributions in the form of patches as produced by @code{git
format-patch}.  Please write commit logs in the ChangeLog format
(@pxref{Change Logs,,, standards, GNU Coding Standards}); you can check
the commit history for examples.

When posting a patch to the mailing list, use @samp{[PATCH] @dots{}} as
a subject.  You may use your email client or the @command{git
send-email} command.  We prefer to get patches in plain text messages,
either inline or as MIME attachments.  You are advised to pay attention
if your email client changes anything like line breaks or indentation
which could potentially break the patches.

@c *********************************************************************
@node GNU Free Documentation License
@appendix GNU Free Documentation License
@cindex license, GNU Free Documentation License
@include fdl-1.3.texi

@c *********************************************************************
@node Concept Index
@unnumbered Concept Index
@printindex cp

@bye