mirror of
git://git.savannah.gnu.org/guix/guix-cuirass.git
synced 2023-12-14 06:03:04 +01:00
2bd6e80482
* doc/cuirass.texi (Web API): Update it. * src/cuirass/http.scm (build->hydra-build): Update it accordingly.
1150 lines
32 KiB
Text
1150 lines
32 KiB
Text
\input texinfo
|
|
@setfilename cuirass.info
|
|
@documentencoding UTF-8
|
|
@include version.texi
|
|
@settitle Cuirass Reference Manual
|
|
@setchapternewpage odd
|
|
|
|
This manual is for Cuirass version @value{VERSION}, a build automation
|
|
server.
|
|
|
|
@copying
|
|
Copyright @copyright{} 2016, 2017 Mathieu Lirzin@*
|
|
Copyright @copyright{} 2017, 2020, 2021 Mathieu Othacehe@*
|
|
Copyright @copyright{} 2018 Ludovic Courtès@*
|
|
Copyright @copyright{} 2018 Clément Lassieur
|
|
|
|
@quotation
|
|
Permission is granted to copy, distribute and/or modify this document
|
|
under the terms of the GNU Free Documentation License, Version 1.3 or
|
|
any later version published by the Free Software Foundation; with no
|
|
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
|
|
copy of the license is included in the section entitled ``GNU Free
|
|
Documentation License''.
|
|
@end quotation
|
|
@end copying
|
|
|
|
@dircategory Software development
|
|
@direntry
|
|
* Cuirass: (cuirass). Build automation server.
|
|
@end direntry
|
|
|
|
@titlepage
|
|
@title Cuirass Reference Manual
|
|
@subtitle Build automation server
|
|
@subtitle for version @value{VERSION}, @value{UPDATED}
|
|
@author The Cuirass Developers
|
|
|
|
@page
|
|
@vskip 0pt plus 1filll
|
|
@insertcopying
|
|
@end titlepage
|
|
|
|
@contents
|
|
|
|
@ifnottex
|
|
@node Top
|
|
@top Cuirass
|
|
@insertcopying
|
|
@end ifnottex
|
|
|
|
@c *********************************************************************
|
|
@menu
|
|
* Introduction:: What is Cuirass about?
|
|
* Specifications:: Writing Cuirass specifications.
|
|
* Notifications:: Build notifications.
|
|
* Parameters:: Cuirass parameters.
|
|
* Build modes:: Build modes.
|
|
* Invocation:: How to run Cuirass.
|
|
* Web API:: Description of the Web API.
|
|
* Database:: About the database schema.
|
|
|
|
* Contributing:: Your help needed!
|
|
* GNU Free Documentation License:: The license of this manual.
|
|
* Concept Index:: Concepts.
|
|
|
|
@end menu
|
|
|
|
@c *********************************************************************
|
|
@node Introduction
|
|
@unnumbered Introduction
|
|
@cindex introduction
|
|
|
|
@dfn{Cuirass} is a general-purpose build automation server that checks
|
|
out source files from @acronym{VCS, Version Control System}
|
|
repositories, executes build jobs, and stores build results in a
|
|
database. It provides a web interface to monitor the build results,
|
|
as well as an HTTP API. Cuirass is also able to send build
|
|
notifications using different mechanisms such as RSS and email.
|
|
|
|
Cuirass is inspired by the @url{https://nixos.org/hydra/, Hydra}
|
|
continuous build system. Unlike Hydra, it is built on top of the
|
|
@url{https://www.gnu.org/software/guix/, GNU Guix} functional package
|
|
manager.
|
|
|
|
The goal of Cuirass is to prevent software regressions by building a
|
|
set of package definitions, system images and running periodical tests
|
|
for various architectures. Cuirass is also responsible for GNU Guix
|
|
binary substitutes production (@pxref{Substitutes, Substitutes,, guix,
|
|
Guix}).
|
|
|
|
Cuirass is deployed on the GNU Guix build farm at
|
|
@url{https://ci.guix.gnu.org}. It is also common for Guix users to
|
|
run their own Cuirass instance to build different sources, using
|
|
different priorities (@pxref{Continuous Integration, Continuous
|
|
Integration,, guix, Guix}).
|
|
|
|
@c *********************************************************************
|
|
@node Specifications
|
|
@chapter Specifications
|
|
@cindex cuirass specifications
|
|
|
|
The main Cuirass argument is the @var{specification} file. It
|
|
describes the repositories that must be used, the build jobs and their
|
|
priorities between other things.
|
|
|
|
@deftp {Data Type} specification
|
|
@table @asis
|
|
@item @code{name}
|
|
The specification name as a Scheme symbol.
|
|
|
|
@item @code{build} (default: @code{all})
|
|
The packages to be built by Cuirass. It defaults to @code{all}, which
|
|
means that all the discovered packages in the subsequent @code{channels}
|
|
field are to be selected.
|
|
|
|
It is also possible to set this field to:
|
|
|
|
@itemize
|
|
@item @code{core}
|
|
Build only the core packages such as @code{gcc}, @code{guile} and
|
|
@code{glibc}.
|
|
|
|
@item @code{guix}
|
|
Build only the Guix modules that are involved in the @command{guix
|
|
pull} command.
|
|
|
|
@item @code{hello}
|
|
Build only the hello package.
|
|
|
|
@item @code{(packages . list)}
|
|
Build only the specified packages in @code{list}. For instance,
|
|
@code{(packages "strace" "perf")} will only build the packages
|
|
@code{strace} and @code{perf}.
|
|
|
|
@item @code{(manifests . list)}
|
|
Build only the packages that are part of the manifest @code{list}.
|
|
For instance, @code{(manifests . ((my-channel . "etc/manifest")))}
|
|
will only build the packages that are part of the @code{etc/manifest}
|
|
file in the @code{my-channel} channel.
|
|
|
|
@end itemize
|
|
|
|
@item @code{channels} (default: @code{(list %default-guix-channel)})
|
|
The channels to be fetched by Cuirass (@pxref{Channels, Channels,,
|
|
guix, Guix}).
|
|
|
|
@item @code{build-outputs} (default: @code{()})
|
|
The build artifacts that must be saved and proposed to download in the
|
|
web interface as a list of @code{build-outputs} records.
|
|
|
|
@deftp {Data Type} build-outputs
|
|
@table @asis
|
|
@item @code{job}
|
|
Save the build outputs of the build jobs which names match the
|
|
@code{job} regexp.
|
|
|
|
@item @code{type}
|
|
The build output type as a string. It is only used to describe the
|
|
build output in the web interface.
|
|
|
|
@item @code{output} (default: @code{("out")})
|
|
The job output if it has multiple outputs (@pxref{Packages with
|
|
Multiple Outputs, Packages with Multiple Outputs,, guix, Guix}).
|
|
|
|
@item @code{path}
|
|
The build output path within the job, as a string.
|
|
|
|
@end table
|
|
@end deftp
|
|
|
|
For instance,
|
|
|
|
@lisp
|
|
(build-output
|
|
(job "binary-tarball*")
|
|
(type "archive")
|
|
(output "out")
|
|
(path ""))
|
|
@end lisp
|
|
|
|
let's consider the @code{binary-tarball.x86_64-linux} job which
|
|
produces the following output:
|
|
@code{/gnu/store/xxx-guix-binary.tar.xz}.
|
|
|
|
The build output definition above will save the root element
|
|
(@code{""}) of the @code{"out"} output of the
|
|
@code{"binary-tarball.x86_64-linux"} job---i.e., the
|
|
@code{"xxx-guix-binary.tar.xz"} file.
|
|
|
|
@item @code{notifications} (default: @code{()})
|
|
The list of build notifications that must be sent. For instance:
|
|
|
|
@lisp
|
|
(list (email
|
|
(from "build@@cuirass.org")
|
|
(to "notification@@myself.org")
|
|
(server "sendmail:///etc/my-mailer.sh")))
|
|
@end lisp
|
|
|
|
will send build notifications emails from @code{build@@cuirass.org} to
|
|
@code{notifications@@myself.org}, using
|
|
@code{"sendmail:///etc/my-mailer.sh"} mailer.
|
|
|
|
The different notification types are described in the
|
|
@ref{Notifications} section.
|
|
|
|
@item @code{priority} (default: @code{9})
|
|
The specification priority relatively to the other specifications, as
|
|
an integer ranging from 0 to 9 where 0 is the higher priority and 9
|
|
the lowest.
|
|
|
|
@item @code{systems} (default: @code{(list (%current-system))})
|
|
Build every job for each system in this list. By default only the
|
|
current system is selected.
|
|
|
|
@end table
|
|
@end deftp
|
|
|
|
@c *********************************************************************
|
|
@node Notifications
|
|
@chapter Notifications
|
|
@cindex cuirass notifications
|
|
|
|
Cuirass supports different build notifications types, that can be
|
|
passed in the @code{notifications} field of the specification record,
|
|
see @ref{Specifications}.
|
|
|
|
Cuirass sends build notifications each time a build is broken or
|
|
fixed.
|
|
|
|
@section Email
|
|
Email build notifications can be enabled using the following record.
|
|
|
|
@deftp {Data Type} email
|
|
@table @asis
|
|
@item @code{from}
|
|
The email @code{From} field, as a string.
|
|
|
|
@item @code{to}
|
|
The email @code{To} field, as a string.
|
|
|
|
@item @code{server}
|
|
The mail server connection string. Cuirass uses the @code{mailutils}
|
|
package. Hence the server can be specified as a remote SMTP mailbox
|
|
(@pxref{SMTP Mailboxes, SMTP Mailboxes,, mailutils, Mailutils}) or as
|
|
a program mailbox (@pxref{Program Mailboxes, Program Mailboxes,,
|
|
mailutils, Mailutils}).
|
|
|
|
@end table
|
|
@end deftp
|
|
|
|
@section Mastodon
|
|
Mastodon build notifications can be enabled using the following record.
|
|
|
|
@deftp {Data Type} mastodon
|
|
@end deftp
|
|
|
|
The Mastodon credentials must be defined as Cuirass parameters, see
|
|
@ref{Parameters}.
|
|
|
|
@section RSS
|
|
Cuirass is proposing a build notification RSS feed at the following
|
|
URL:
|
|
|
|
@itemize
|
|
@item @url{http://cuirass-url/events/rss[?specification=spec]}
|
|
By default build notifications are sent for all specifications. If
|
|
the @code{specification} argument is passed, they can be restricted to
|
|
the @var{spec} specification.
|
|
|
|
@end itemize
|
|
|
|
@c *********************************************************************
|
|
@node Parameters
|
|
@chapter Parameters
|
|
@cindex cuirass parameters
|
|
|
|
Cuirass is able to connect to different external services such as
|
|
@code{postgresql} for the database, @code{zabbix} for machine
|
|
monitoring and @code{mastodon} for build notifications. As those
|
|
services often require using secret credentials, Cuirass can be passed
|
|
a parameter file.
|
|
|
|
The parameters file can be passed using the @code{parameters} command
|
|
line argument, see @ref{Invocation}.
|
|
|
|
Here's an example parameter file:
|
|
|
|
@example
|
|
(%cuirass-url "https://ci.guix.gnu.org")
|
|
(%zabbix-url "http://127.0.0.1:15412/api_jsonrpc.php")
|
|
(%mastodon-instance-name "My Instance")
|
|
(%mastodon-instance-url "https://instance.org")
|
|
(%mastodon-instance-token "secret-token")
|
|
@end example
|
|
|
|
@deftp {Parameters} parameters
|
|
@table @asis
|
|
@item @code{%cuirass-database} (default: @code{"cuirass"})
|
|
The Cuirass PostgreSQL database name.
|
|
|
|
@item @code{%cuirass-host} (default: @code{"/var/run/postgresql"})
|
|
The Cuirass PostgreSQL database host.
|
|
|
|
@item @code{%cuirass-url} (default: @code{#f})
|
|
The URL of the Cuirass web server. This is useful to send absolute
|
|
links within notifications.
|
|
|
|
@item @code{%zabbix-url} (default: @code{#f})
|
|
The URL of the Zabbix monitoring server providing the workers status,
|
|
if supported.
|
|
|
|
@item @code{%zabbix-user} (default: @code{"Admin"})
|
|
The user for Zabbix API authentication.
|
|
|
|
@item @code{%zabbix-password} (default: @code{"zabbix"})
|
|
The password for Zabbix API authentication.
|
|
|
|
@item @code{%mastodon-instance-name} (default: @code{#f})
|
|
The name of the Mastodon instance used to send build notifications.
|
|
|
|
@item @code{%mastodon-instance-url} (default: @code{#f})
|
|
The URL of the Mastodon instance.
|
|
|
|
@item @code{%mastodon-instance-token} (default: @code{#f})
|
|
The token used to authenticate on the Mastodon instance.
|
|
|
|
@end table
|
|
@end deftp
|
|
|
|
@c *********************************************************************
|
|
@node Build modes
|
|
@chapter Build modes
|
|
@cindex cuirass build modes
|
|
|
|
Cuirass supports two mechanisms to build derivations.
|
|
|
|
@section With the local Guix daemon
|
|
This is the default build mechanism. Once the build jobs are
|
|
evaluated, they are sent to the local Guix daemon. Cuirass then
|
|
listens to the Guix daemon output to detect the various build events.
|
|
|
|
While this mode doesn't require any particular configuration, it
|
|
doesn't scale well. The scheduling decisions of the Guix daemon are
|
|
opaque and often suboptimal.
|
|
|
|
When Cuirass is used to build a large amount of jobs, the remote build
|
|
mechanism described below should be preferred.
|
|
|
|
@section With the remote build mechanism.
|
|
|
|
This mode is harder to setup but scales way better. This is the build
|
|
mode that is used on the GNU Guix build farm at
|
|
@url{https://ci.guix.gnu.org}.
|
|
|
|
It can be enabled by passing the @code{build-remote} command line
|
|
argument, see @ref{Invocation}.
|
|
|
|
@c *********************************************************************
|
|
@node Invocation
|
|
@chapter Invocation
|
|
@section Invoking cuirass
|
|
@cindex invoking cuirass
|
|
|
|
The usual way to invoke @code{cuirass} is as follows:
|
|
|
|
@example
|
|
cuirass --specifications @var{specs} &
|
|
cuirass --web &
|
|
@end example
|
|
|
|
This starts a Cuirass instance building @var{specs} and storing the
|
|
results using the default PostgreSQL database. The second command
|
|
starts a Cuirass web server, on the default port, using the same
|
|
database.
|
|
|
|
Additionally the following options can be used.
|
|
|
|
@table @code
|
|
@item --one-shot
|
|
Instead of executing @code{cuirass} as a daemon looping over the jobs.
|
|
Only evaluate and build the specifications once.
|
|
|
|
@item --cache-directory=@var{directory}
|
|
@var{directory} is the place where the VCS repositories used by the jobs
|
|
are stored.
|
|
|
|
@item --specifications=@var{specifications-file}
|
|
@itemx -S @var{specifications-file}
|
|
Add the specifications defined in @var{specifications-file} in the job
|
|
database before launching the evaluation and build processes.
|
|
|
|
@item --database=@var{database}
|
|
@itemx -D @var{database}
|
|
Use @var{database} as the database containing the jobs and the past
|
|
build results. Since Cuirass uses PostgreSQL as a database engine,
|
|
@var{database} must be a string such as @code{"dbname=cuirass
|
|
host=localhost"}. By default, Cuirass uses the following connection
|
|
string: @code{dbname=cuirass host=/var/run/postgresql"}.
|
|
|
|
@item --parameters=@var{parameters-file}
|
|
@itemx -P @var{parameters-file}
|
|
Read parameters from the given @var{parameters-file}. The supported
|
|
parameters are described here (@pxref{Parameters}).
|
|
|
|
@item --ttl=@var{duration}
|
|
Cuirass registers build results as garbage collector (GC) roots, thereby
|
|
preventing them from being deleted by the GC. The @option{--ttl} option
|
|
instructs it to keep those GC roots live for at least @var{duration}---e.g.,
|
|
@code{1m} for one month, @code{2w} for two weeks, and so on. The default is
|
|
30 days.
|
|
|
|
Those GC roots are typically stored in
|
|
@file{/var/guix/gcroots/profiles/per-user/@var{user}/cuirass}, where @var{user} is the
|
|
user under which Cuirass is running.
|
|
|
|
@item --port=@var{num}
|
|
@itemx -p @var{num}
|
|
Make the HTTP interface listen on port @var{num}. Use port 8080 by
|
|
default.
|
|
|
|
@item --listen=@var{host}
|
|
Make the HTTP interface listen on network interface for @var{host}. Use
|
|
localhost by default.
|
|
|
|
@item --interval=@var{n}
|
|
@itemx -I @var{n}
|
|
Wait @var{n} seconds between each poll.
|
|
|
|
@item --use-substitutes
|
|
This can be useful when you are not interested in building the
|
|
dependencies of a particular job.
|
|
|
|
@item --threads=@var{n}
|
|
Use up to @var{n} kernel threads.
|
|
|
|
@var{n} should be lower than or equal to the number of CPU cores on the
|
|
machine. In general though, having a large @var{n} is not very useful
|
|
since the work of Cuirass is primarily I/O-bound---on the contrary,
|
|
large values of @var{n} may increase overhead. The default value should
|
|
be appropriate for most cases.
|
|
|
|
@item --version
|
|
@itemx -V
|
|
Display the actual version of @code{cuirass}.
|
|
|
|
@item --help
|
|
@itemx -h
|
|
Display an help message that summarize all the options provided.
|
|
@end table
|
|
|
|
@section Invoking remote-server
|
|
@cindex remote-server
|
|
|
|
The @code{remote-server} command starts a daemon that is able to
|
|
communicate with @code{remote-worker} processes. Its role is to
|
|
answer build requests from the workers, by sending back derivations
|
|
that must be built.
|
|
|
|
On build completion it updates the database accordingly and possibly
|
|
fetches build substitutes. The @code{remote-server} and
|
|
@code{remote-worker} processes communicate using ZMQ over TCP.
|
|
|
|
Additionally the following options can be used.
|
|
|
|
@table @code
|
|
@item --backend-port=@var{port}
|
|
The TCP port for communicating with @code{remote-worker} processes
|
|
using ZMQ. It defaults to @code{5555}.
|
|
|
|
@item --log-port=@var{port}
|
|
The TCP port of the log server. It defaults to @code{5556}.
|
|
|
|
@item --publish-port=@var{port}
|
|
The TCP port of the publish server. It defaults to @code{5557}.
|
|
|
|
@item --parameters=@var{parameters-file}
|
|
@itemx -P @var{parameters-file}
|
|
Read parameters from the given @var{parameters-file}. The supported
|
|
parameters are described here (@pxref{Parameters}).
|
|
|
|
@item --database=@var{database}
|
|
@itemx -D @var{database}
|
|
Use @var{database} PostgreSQL connection string.
|
|
|
|
@item --cache=@var{directory}
|
|
Use @var{directory} to cache build log files.
|
|
|
|
@item --trigger-substitute-url=@var{URL}
|
|
Once a substitute is successfully fetched, trigger substitute baking
|
|
at @var{URL}.
|
|
|
|
@item --user=@var{user}
|
|
Change privileges to @var{user} as soon as possible---i.e., once the
|
|
signing key has been read.
|
|
|
|
@item --public-key=@var{file}
|
|
@itemx --private-key=@var{file}
|
|
Use the specific @var{file}s as the public/private key pair used to sign
|
|
the store items being published.
|
|
|
|
@item --version
|
|
@itemx -V
|
|
Display the actual version of @code{cuirass}.
|
|
|
|
@item --help
|
|
@itemx -h
|
|
Display an help message that summarize all the options provided.
|
|
@end table
|
|
|
|
@section Invoking remote-worker
|
|
@cindex remote-worker
|
|
|
|
The @code{remote-worker} command starts a daemon that is able to
|
|
communicate with a @code{remote-server} process. Its role is to
|
|
request builds to the @code{remote-server}, perform them and report
|
|
their status.
|
|
|
|
The @code{remote-worker} is able to discover a @code{remote-server}
|
|
process on the local network using Avahi and connect to it.
|
|
|
|
Additionally the following options can be used.
|
|
|
|
@table @code
|
|
@item --workers=@var{count}
|
|
Start @var{count} parallel workers. It defaults to @code{1}.
|
|
|
|
@item --publish-port=@var{port}
|
|
The TCP port of the publish server. It defaults to @code{5558}.
|
|
|
|
@item --server=@var{ip-address}
|
|
Do not use Avahi discovery and connect to the given
|
|
@code{remote-server} IP address.
|
|
|
|
@item --systems=@var{systems}
|
|
Only request builds for the given @var{systems}. It defaults to
|
|
@code{(list (%current-system))}.
|
|
|
|
@item --public-key=@var{file}
|
|
@itemx --private-key=@var{file}
|
|
Use the specific @var{file}s as the public/private key pair used to sign
|
|
the store items being published.
|
|
|
|
@item --version
|
|
@itemx -V
|
|
Display the actual version of @code{cuirass}.
|
|
|
|
@item --help
|
|
@itemx -h
|
|
Display an help message that summarize all the options provided.
|
|
@end table
|
|
|
|
@c *********************************************************************
|
|
@node Web API
|
|
@chapter Web API
|
|
@cindex web api
|
|
|
|
The Cuirass web API is inspired from the Hydra one.
|
|
|
|
@section API description
|
|
@cindex description, json
|
|
|
|
@subsection Evaluation information
|
|
@subsubheading Single evaluation
|
|
|
|
It is possible to query the Cuirass web server for evaluation
|
|
information. The dedicated API is "/api/evaluation?id=@var{eval-id}"
|
|
where @var{eval-id} is the unique id associated to the evaluation in
|
|
database.
|
|
|
|
For instance, querying a local Cuirass web server can be done with
|
|
@code{curl} and @code{jq} to format the JSON response :
|
|
|
|
@example
|
|
$ curl -s "http://localhost:8080/api/evaluation?id=1" | jq
|
|
|
|
@{
|
|
"id": 1,
|
|
"specification": "guix-master",
|
|
"status": 0,
|
|
"timestamp": 1615289011,
|
|
"checkouttime": 1615289011,
|
|
"evaltime": 1615289655,
|
|
"checkouts": [
|
|
@{
|
|
"commit": "bd311f5a6ccbd4696ac77f0426a036bb375a2f10",
|
|
"channel": "guix",
|
|
"directory": "/gnu/store/6978xw9vs4ybg2pc3g736p1dba2056yl-guix-bd311f5"
|
|
@}
|
|
]
|
|
@}
|
|
@end example
|
|
|
|
The nominal output is a JSON object whose fields are described
|
|
hereafter.
|
|
|
|
@table @code
|
|
@item id
|
|
The unique build id.
|
|
|
|
@item specification
|
|
The associated specification name, as a string.
|
|
|
|
@item status
|
|
The evaluation status, as an integer. Possible values are :
|
|
|
|
@example
|
|
-1 -> started
|
|
0 -> succeeded
|
|
1 -> failed
|
|
2 -> aborted
|
|
@end example
|
|
|
|
@item checkouttime
|
|
The timestamp after channel checkout.
|
|
|
|
@item evaltime
|
|
The timestamp after evaluation completion.
|
|
|
|
@item checkouts
|
|
The evaluation checkouts as a JSON object.
|
|
|
|
@end table
|
|
|
|
@subsubheading Multiple evaluations
|
|
|
|
The latest evaluations list can be obtained with the API
|
|
"/api/evaluations". The output is a JSON array of
|
|
evaluations. Evaluations are represented as in the
|
|
"/api/evaluation?id=@var{eval-id}" API.
|
|
|
|
This request accepts a mandatory parameter.
|
|
@table @code
|
|
@item nr
|
|
Limit query result to nr elements. This parameter is @emph{mandatory}.
|
|
@end table
|
|
|
|
@subsection Build information
|
|
|
|
It is possible to query Cuirass web server for build informations. The
|
|
dedicated API is "/build/@var{build-id}" where @var{build-id} is the
|
|
unique id associated to the build in database.
|
|
|
|
The build information can also be queried by output. For example,
|
|
@samp{/output/kg9mirg6xbvzcp0a98v7326n1nvvwgsj-hello-2.10} will return
|
|
the details of the output, along with the build if available.
|
|
|
|
@example
|
|
$ curl -s "http://localhost:8080/build/2" | jq
|
|
|
|
@{
|
|
"id": 2,
|
|
"jobset": "guix",
|
|
"job": "acpica-20150410-job",
|
|
"timestamp": 1501347493,
|
|
"starttime": 1501347493,
|
|
"stoptime": 1501347493,
|
|
"buildoutputs": @{
|
|
"out": @{
|
|
"path": "/gnu/store/6g3njhfzqpdm335s7qhvmwvs5l7gcbq1-acpica-20150410"
|
|
@}
|
|
@},
|
|
"system": "x86_64-linux",
|
|
"nixname": "acpica-20150410",
|
|
"buildstatus": 0,
|
|
"weather": 0,
|
|
"busy": 0,
|
|
"priority": 0,
|
|
"finished": 1,
|
|
"buildproducts": null
|
|
@}
|
|
@end example
|
|
|
|
If requested @var{build-id} is not known, the HTTP code 404 is
|
|
answered with a JSON error message. For example:
|
|
|
|
@example
|
|
$ curl -s "http://localhost:8080/build/fff"
|
|
|
|
@{"error" : "Build with ID fff doesn't exist."@}
|
|
@end example
|
|
|
|
The nominal output is a JSON object whose fields are described
|
|
hereafter.
|
|
|
|
@table @code
|
|
@item id
|
|
The unique build id.
|
|
|
|
@item jobset
|
|
The associated specification name, as a string.
|
|
|
|
@item job
|
|
The associated job-name, as a string.
|
|
|
|
@item timestamp
|
|
Timestamp taken at build creation time.
|
|
|
|
@item starttime
|
|
Timestamp taken at build start time.
|
|
|
|
@item stoptime
|
|
Timestamp taken at build stop time.
|
|
|
|
@item buildoutputs
|
|
Build outputs as a JSON object. The keys names are referring to the
|
|
eventual output names. The associated value is another JSON object which
|
|
only key is @code{path}. @code{path} value is the output directory in
|
|
store as a string.
|
|
|
|
@item system
|
|
System name of the build, as a string.
|
|
|
|
@item nixname
|
|
Derivation name, as a string.
|
|
|
|
@item buildstatus
|
|
Build status, as an integer. Possible values are :
|
|
|
|
@example
|
|
0 -> succeeded
|
|
1 -> failed
|
|
2 -> failed dependency
|
|
3 -> failed other
|
|
4 -> cancelled
|
|
@end example
|
|
|
|
@item weather
|
|
Build weather, as an integer.
|
|
|
|
@example
|
|
-1 -> unknown
|
|
0 -> new-success
|
|
1 -> new-failure
|
|
2 -> still-succeeding
|
|
3 -> still-failing
|
|
@end example
|
|
|
|
@item busy
|
|
Whether the build is pending, as an integer.
|
|
|
|
@item priority
|
|
Build priority, as an integer.
|
|
|
|
@item finished
|
|
Build finished, as an integer.
|
|
|
|
@item buildproducts
|
|
Build products in store as a JSON object.
|
|
|
|
@end table
|
|
|
|
@subsection Build raw log output
|
|
|
|
It is possible to ask Cuirass for the raw build output log with the API
|
|
"/build/@var{build-id}/log/raw" where @var{build-id} is the
|
|
unique id associated to the build in database.
|
|
|
|
The output is a raw text, for example:
|
|
|
|
@example
|
|
$ curl http://localhost:8080/build/2/log/raw
|
|
|
|
starting phase `set-SOURCE-DATE-EPOCH'
|
|
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
|
|
starting phase `set-paths'
|
|
...
|
|
@end example
|
|
|
|
If requested @var{build-id} is not known, the HTTP code 404 is
|
|
answered with a JSON error message. For example:
|
|
|
|
@example
|
|
$ curl -s "http://localhost:8080/build/fff/log/raw"
|
|
|
|
@{"error" : "Build with ID fff doesn't exist."@}
|
|
@end example
|
|
|
|
@subsection Latest builds
|
|
|
|
The list of latest builds can be obtained with the API
|
|
"/api/latestbuilds". The output is a JSON array of builds. Builds are
|
|
represented as in the "/build/@var{build-id}" API.
|
|
|
|
This request accepts a mandatory parameter and multiple optional ones.
|
|
|
|
@table @code
|
|
@item nr
|
|
Limit query result to nr elements. This parameter is @emph{mandatory}.
|
|
|
|
@item jobset
|
|
Filter query result to builds with the given @code{jobset}.
|
|
|
|
@item job
|
|
Filter query result to builds with the given @code{job} name.
|
|
|
|
@item system
|
|
Filter query result to builds with the given @code{system}.
|
|
|
|
@end table
|
|
|
|
For example, to ask for the ten last builds:
|
|
|
|
@example
|
|
$ curl "http://localhost:8080/api/latestbuilds?nr=10"
|
|
@end example
|
|
|
|
or the five last builds where jobset ``guix'':
|
|
|
|
@example
|
|
$ curl "http://localhost:8080/api/latestbuilds?nr=5&jobset=guix"
|
|
@end example
|
|
|
|
If no builds matching given parameters are found, an empty JSON array is
|
|
returned.
|
|
|
|
@subsection Queued builds
|
|
|
|
The list of queued builds can be obtained with the API
|
|
"/api/queue". The output is a JSON array of builds. Builds are
|
|
represented as in the "/build/@var{build-id}" API.
|
|
|
|
This request accepts a mandatory parameter.
|
|
@table @code
|
|
@item nr
|
|
Limit query result to nr elements. This parameter is @emph{mandatory}.
|
|
@end table
|
|
|
|
@c *********************************************************************
|
|
@node Database
|
|
@chapter Database schema
|
|
@cindex cuirass database
|
|
@cindex postgresql database
|
|
@cindex persistent configuration
|
|
|
|
Cuirass uses a PostgreSQL database to store information about jobs and
|
|
past build results, but also to coordinate the execution of jobs.
|
|
|
|
The database contains the following tables: @code{Specifications},
|
|
@code{Checkouts}, @code{Evaluations}, @code{Builds}, @code{Outputs},
|
|
@code{Metrics}, @code{BuildProducts}, @code{Events} and
|
|
@code{Workers}. The purpose of each of these tables is explained
|
|
below.
|
|
|
|
@section Specifications
|
|
@cindex specifications, database
|
|
|
|
This table stores specifications describing the repositories from whence
|
|
Cuirass fetches code and the environment in which it will be processed.
|
|
Entries in this table must have values for the following text fields:
|
|
|
|
@table @code
|
|
@item name
|
|
This field holds the name of the specification. This field is also the
|
|
primary key of this table.
|
|
|
|
@item channels
|
|
The channels to be fetched by Cuirass as an SEXP string.
|
|
|
|
@item build_ouputs
|
|
The build outputs to be saved by Cuirass as an SEXP string.
|
|
|
|
@item notifications
|
|
The build notifications to be sent by Cuirass as an SEXP string.
|
|
|
|
@item priority
|
|
The specification priority relatively to the other specifications, as
|
|
an integer ranging from 0 to 9 where 0 is the higher priority and 9
|
|
the lowest.
|
|
|
|
@item systems
|
|
The systems for which build jobs must be evaluated, as a comma
|
|
separated list.
|
|
|
|
@end table
|
|
|
|
@section Checkouts
|
|
@cindex checkouts, database
|
|
|
|
When a specification is processed, the repositories must be downloaded at a
|
|
certain revision as specified. The download is called a checkout. The
|
|
@code{Checkouts} table stores the new checkouts for every specification when
|
|
it is being processed.
|
|
|
|
The @code{Checkouts} table has the following columns:
|
|
|
|
@table @code
|
|
@item specification
|
|
The specification associated with the checkout.
|
|
|
|
@item revision
|
|
The revision of the checkout. Within the same specification, two checkouts
|
|
can't be identical: they can't have the same revision.
|
|
|
|
@item evaluation
|
|
The evaluation that was triggered by the addition of that new checkout.
|
|
|
|
@item channel
|
|
The channel associated with the checkout.
|
|
|
|
@item directory
|
|
The directory into which the checkout was extracted.
|
|
|
|
@item timestamp
|
|
The checkout insertion timestamp.
|
|
|
|
@end table
|
|
|
|
@section Evaluations
|
|
@cindex evaluations, database
|
|
|
|
An evaluation relates a specification with the revision of the repository
|
|
specified therein. Builds (see below) belong to a specific evaluation.
|
|
|
|
The @code{Evaluations} table has the following columns:
|
|
|
|
@table @code
|
|
@item id
|
|
This is an automatically incrementing numeric identifier.
|
|
|
|
@item specification
|
|
This field holds the @code{name} of a specification from the
|
|
@code{Specifications} table.
|
|
|
|
@item status
|
|
This integer field hold the evaluation status. Possible values are:
|
|
@itemize
|
|
@item started (@code{-1})
|
|
@item succeeded (@code{0})
|
|
@item failed (@code{1})
|
|
@item aborted (@code{2})
|
|
@end itemize
|
|
|
|
@item timestamp
|
|
The timestamp at evaluation insertion.
|
|
|
|
@item checkout
|
|
The timestamp after channel checkout.
|
|
|
|
@item evaltime
|
|
The timestamp after evaluation completion.
|
|
|
|
@end table
|
|
|
|
@section Builds
|
|
@cindex builds, database
|
|
|
|
This table holds records of the derivations and their build status. Note that
|
|
a job will be registered here only if its derivation doesn't already exist.
|
|
|
|
@table @code
|
|
@item derivation
|
|
This text field holds the absolute name of the derivation file that
|
|
resulted in this build.
|
|
|
|
@item evaluation
|
|
This integer field references the evaluation identifier from the
|
|
@code{Evaluations} table, indicating to which evaluation this build
|
|
belongs.
|
|
|
|
@item job_name
|
|
This text field holds the name of the job.
|
|
|
|
@item system
|
|
This text field holds the system name of the derivation.
|
|
|
|
@item nix_name
|
|
This text field holds the name of the derivation ---e.g.,
|
|
@code{coreutils-8.24}.
|
|
|
|
@item worker
|
|
This text field references the name of worker performing the build
|
|
from the @code{Workers} table.
|
|
|
|
@item log
|
|
This text field holds the absolute file name of the build log file.
|
|
|
|
@item status
|
|
This integer field holds the build status of the derivation.
|
|
|
|
@item last_status
|
|
This integer field holds the build status of the previous job
|
|
evaluation.
|
|
|
|
@item weather
|
|
This integer field holds the weather of the build. Possible values
|
|
are:
|
|
|
|
@itemize
|
|
@item unknown (@code{-1})
|
|
@item new-success (@code{0})
|
|
@item new-failure (@code{1})
|
|
@item still-succeeding (@code{2})
|
|
@item still-failing (@code{3})
|
|
@end itemize
|
|
|
|
@item priority
|
|
The build priority relatively to the other builds with the same
|
|
@code{job_name}, as an integer ranging from 0 to 99 where 0 is the
|
|
higher priority and 99 the lowest.
|
|
|
|
@item max_silent
|
|
This integer field holds the number of seconds of silence after which
|
|
a build process times out.
|
|
|
|
@item timeout
|
|
This integer field holds the number of seconds of activity after which
|
|
a build process times out.
|
|
|
|
@item timestamp
|
|
This integer field holds a timestamp taken at build creation time.
|
|
|
|
@item starttime
|
|
This integer field holds a timestamp taken at build start time.
|
|
Currently, it has the same value as the @code{timestamp} above.
|
|
|
|
@item stoptime
|
|
This integer field holds a timestamp taken at build stop time.
|
|
Currently, it has the same value as the @code{timestamp} above.
|
|
|
|
@end table
|
|
|
|
@section Outputs
|
|
@cindex outputs, database
|
|
|
|
This table keep tracks for every eventual build outputs. Each build
|
|
stored in @code{Builds} table may have zero (if it has failed), one or
|
|
multiple outputs.
|
|
|
|
@table @code
|
|
@item derivation
|
|
This field holds the @code{derivation} of a build from the @code{Builds}
|
|
table.
|
|
|
|
@item name
|
|
This text field holds the name of the output.
|
|
|
|
@item path
|
|
This text field holds the path of the output.
|
|
|
|
@end table
|
|
|
|
@section Metrics
|
|
@cindex metrics, database
|
|
|
|
This table contains several metrics that are recorded by the
|
|
@code{metrics} fiber periodically.
|
|
|
|
@table @code
|
|
@item field
|
|
This text field holds the application field of the metric.
|
|
|
|
@item type
|
|
This integer field holds the type of the metric.
|
|
|
|
@item path
|
|
This float field holds the value of the metric.
|
|
|
|
@item evaltime
|
|
The metric insertion timestamp.
|
|
|
|
@end table
|
|
|
|
@section BuildProducts
|
|
@cindex buildproducts, database
|
|
|
|
This table contains the saved build products, that are proposed to
|
|
download through the web interface.
|
|
|
|
@table @code
|
|
@item build
|
|
This integer field holds a reference to the build @code{id} from the
|
|
@code{Builds} table, the build product belongs to.
|
|
|
|
@item type
|
|
This text field holds the build product type.
|
|
|
|
@item file_size
|
|
This integer field holds build product size in bytes.
|
|
|
|
@item checksum
|
|
This text field holds the build product checksum.
|
|
|
|
@item path
|
|
This text field holds the build product absolute store path.
|
|
|
|
@end table
|
|
|
|
@section Workers
|
|
@cindex workers, database
|
|
|
|
This table contains the registered workers when Cuirass is using the
|
|
remote building mechanism.
|
|
|
|
@table @code
|
|
@item name
|
|
This text field holds the worker name. This field is also the primary
|
|
key of this table.
|
|
|
|
@item address
|
|
This text field holds the worker IP address.
|
|
|
|
@item machine
|
|
This text field holds the worker machine name.
|
|
|
|
@item systems
|
|
This text field holds the systems that are supported by the worker, as
|
|
a comma separated list of systems.
|
|
|
|
@item last_seen
|
|
This integer field holds the timestamp of the last communication with
|
|
the worker.
|
|
|
|
@end table
|
|
|
|
@c *********************************************************************
|
|
|
|
@node Contributing
|
|
@chapter Contributing
|
|
|
|
Everyone is welcome to contribute to Cuirass. You can report bugs, send
|
|
patches and share your ideas with others by sending emails the
|
|
@email{guix-devel@@gnu.org, mailing list}.
|
|
|
|
Development is done using the Git distributed version control system.
|
|
Thus, access to the repository is not strictly necessary. We welcome
|
|
contributions in the form of patches as produced by @code{git
|
|
format-patch}. Please write commit logs in the ChangeLog format
|
|
(@pxref{Change Logs,,, standards, GNU Coding Standards}); you can check
|
|
the commit history for examples.
|
|
|
|
When posting a patch to the mailing list, use @samp{[PATCH] @dots{}} as
|
|
a subject. You may use your email client or the @command{git
|
|
send-email} command. We prefer to get patches in plain text messages,
|
|
either inline or as MIME attachments. You are advised to pay attention
|
|
if your email client changes anything like line breaks or indentation
|
|
which could potentially break the patches.
|
|
|
|
@c *********************************************************************
|
|
@node GNU Free Documentation License
|
|
@appendix GNU Free Documentation License
|
|
@cindex license, GNU Free Documentation License
|
|
@include fdl-1.3.texi
|
|
|
|
@c *********************************************************************
|
|
@node Concept Index
|
|
@unnumbered Concept Index
|
|
@printindex cp
|
|
|
|
@bye
|