Commit Graph

96 Commits

Author SHA1 Message Date
Ludovic Courtès a97665672a http: Sort /api/latestbuilds and /api/queue by time.
* src/cuirass/database.scm (db-get-builds)[format-order-clause]: Add
'order' clauses.
* src/cuirass/http.scm (url-handler): Default to (order finish-time) for
/latestbuilds and (order submission-time) for /queue.
2018-01-25 10:39:18 +01:00
Ludovic Courtès c4963deca1 database: Turn 'sqlite-exec' into a macro.
* src/cuirass/database.scm (%sqlite-exec): New procedure.
(sqlite-exec): New macro.
2018-01-25 10:38:53 +01:00
Ludovic Courtès 8675d6309b database: 'db-update-build-status!' takes a #:log-file parameter.
* src/cuirass/database.scm (sqlite-exec): Use (ice-9 format).
(db-update-build-status!): Add #:log-file parameter and honor it.
* tests/database.scm ("database")["db-update-build-status!"]: Test it.
2018-01-24 23:41:24 +01:00
Ludovic Courtès dd30a1a25c base: Restart pending builds upfront.
* src/cuirass/database.scm (db-get-builds)[format-where-clause]:
Honor (status pending) filter.
* src/cuirass/base.scm (restart-builds): New procedure.
* bin/cuirass.in (main): Fetch pending builds.  Start fiber that invokes
'restart-builds' on them.
2018-01-23 23:40:07 +01:00
Ludovic Courtès 06b8af00fb http: /api/latestbuilds only returns finished builds.
* src/cuirass/database.scm (db-get-builds)[format-where-clause]:
Honor (status done) filter.
* src/cuirass/http.scm (url-handler): Always add (status done)
to the filters passed to 'handle-builds-request'.
2018-01-23 23:39:13 +01:00
Ludovic Courtès 6fb7118839 database: Use US spelling for "canceled".
* src/cuirass/database.scm (build-status): Use US spelling for
"canceled".
2018-01-23 23:39:13 +01:00
Ludovic Courtès fe0a98315d database: Add 'db-update-build-status!'.
* src/cuirass/database.scm (build-status): Add 'scheduled' and
'started'.
(db-add-build): Make sure #:timestamp, #:starttime, #:stoptime, and
 #:status are integers.
(db-update-build-status!): New procedure.
* tests/database.scm ("database")["db-update-build-status!"]: New test.
2018-01-23 18:17:31 +01:00
Ludovic Courtès b0f1a438ff Add a 'build-status' enumeration.
* src/cuirass/utils.scm (define-enumeration): New macro.
* src/cuirass/database.scm (build-status): New macro.
* src/cuirass/base.scm (build-packages)[hydra-build-status]: Remove.
Use the 'build-status' macro instead.
2018-01-23 17:37:08 +01:00
Ludovic Courtès ee11ba1d93 Introduce concurrency with Fibers.
* README: Mark Fibers as required.
* configure.ac: Check for Guile 2.2 only.  Check for (fibers).
* bin/cuirass.in (main): Use (fibers).  Run 'process-specs' and web
server in separate fibers.
* src/cuirass/base.scm (with-store): New macro.
(non-blocking-port): New procedure.
(evaluate): Use 'non-blocking-port'.  Use 'read-string' followed by 'read'.
(process-specs): Move 'db-add-stamp' right after 'string=?' comparison.
Run evaluation and subsequent builds in a separate fiber.
* src/cuirass/http.scm (run-cuirass-server): Pass 'fibers as the second
argument to 'run-server'.  Use 'log-message' instead of 'format'.
* src/cuirass/database.scm (with-database): Remove 'dynamic-wind'.
2018-01-22 23:39:51 +01:00
Ludovic Courtès f9481e2222 database: Change default file name to /var/run/cuirass/cuirass.db.
* src/cuirass/database.scm (%package-database): Change default file name
from /var/cuirass.db (or similar) to /var/run/cuirass/cuirass.db.
2018-01-22 14:51:14 +01:00
Ludovic Courtès 60b6c6fcc5 database: Extend and test 'db-get-builds'.
* src/cuirass/database.scm (db-get-builds): Make 'order' a separate
filter.  Add 'format-limit-clause'.  Reverse OUTPUTS.
* tests/database.scm (make-dummy-eval, make-dummy-derivation)
(make-dummy-build): New procedures.
(with-temporary-database): New macro.
("database"): Use 'make-dummy-build'.
("db-get-builds"): New test.
2018-01-07 23:59:30 +01:00
Mathieu Othacehe 8a7483a3bb
cuirass: add Hydra compatible HTTP API.
* doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API.
* src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it
  object->json-string.
(object->json-scm): Move it utils.scm.
(handle-*-request): New helpers procedures.
(request-parameters): New procedure to parse a request query.
(url-handler): Add new API's.
* src/cuirass/utils.scm (object->json-scm, object->json-string): Exported
procedures moved from http.scm.
* tests/http.scm: Add various tests on new HTTP API.
2017-09-08 21:15:32 +02:00
Mathieu Othacehe e550cb6a9a
cuirass: Store new information in database to prepare new HTTP API
integration.

* bin/evaluate.in (fill-job): New procedure.
(main): Use it to fill informations (nix-name, system) that will later be
added to database.
* doc/cuirass.texi (Database)[Derivation]: Add system and nix_name fields.
(Database)[Builds]: Add id, status, timestamp, starttime and stoptime
fields. Remove output field.
(Database)[Outputs]: New table describing the build outputs.
* src/cuirass/base.scm (build-packages): Add new fields to build object before
adding it to database.
* src/cuirass/database.scm (db-get-build, db-get-builds): New procedures to get
a build by id from database and a list of builds using filter parameters
respectively.
* src/schema.sql (Outputs) : New table.
(Derivations): Add system and nix_name columns.
(Builds): Remove output column and add id, status, timestamp, starttime and
stoptime columns.
2017-09-08 21:00:57 +02:00
Ludovic Courtès 72f2b6b77c database: 'db-add-build' is now idempotent.
Fixes <https://bugs.gnu.org/28094>.
Reported by Ricardo Wurmus <rekado@elephly.net>.

* src/cuirass/database.scm (SQLITE_CONSTRAINT)
(SQLITE_CONSTRAINT_PRIMARYKEY): New variables.
(db-add-build): Catch 'sqlite-error, and swallow
SQLITE_CONSTRAINT_PRIMARYKEY errors.
* tests/database.scm ("db-add-build"): New test.
2017-08-26 10:45:43 +02:00
Mathieu Othacehe 707b053024
database: Fix typo.
* src/cuirass/database.scm (db-get-stamp, db-add-stamp): Replace #:id
  by #:name. There is no #:id in specifications. Plus, the primary key
  of Specifications is #:name.
2017-07-12 16:28:41 +02:00
Mathieu Othacehe d7dc7cb92c
cuirass: Stop λ/lambda mixing.
* bin/cuirass.in: Replace λ by lambda.
* bin/evaluate.in: Ditto.
* build-aux/guix.scm: Ditto.
* examples/gnu-system.scm: Ditto.
* examples/guix-track-git.scm: Ditto.
* src/cuirass.scm: Ditto.
* src/cuirass/base.scm: Ditto.
* src/cuirass/database.scm: Ditto.
* src/cuirass/ui.scm: Ditto.
* src/cuirass/utils.scm: Remove λ* macro.
2017-07-06 10:03:16 +02:00
Mathieu Lirzin d0a5801e39
database: db-add-derivation: Don't try to add a derivation twice.
This fixes the issue of having multiple identical derivations associated with
an evaluation as reported by Ludovic Courtès <ludo@gnu.org> at
<https://lists.gnu.org/archive/html/guix-devel/2017-01/msg00109.html>.

* src/cuirass/database.scm (db-add-derivation): Ignore if JOB is already
present in DB.
2017-01-09 10:55:58 +01:00
Mathieu Lirzin bfd395c09f
db: Forbid inserting the same specification twice.
* src/schema.sql (Specifications): Make 'repo_name' the primary key.
(Stamps, Evaluations): Adapt.
* src/cuirass/database.scm (db-get-specifications): Likewise.
(db-add-specification): Ignore if SPEC has already been added.
* tests/database.scm (example-spec): Adapt.
2016-11-17 01:19:05 +01:00
Jan Nieuwenhuizen 5ef0701f54
base: Support tracking of a Guix package's git.
* src/schema.sql (Specifications): Add no_compile_p column.
* src/cuirass/database.scm (db-add-specification)
(db-get-specifications): Handle #:no-compile? property.
* tests/database.scm (example-spec): Adapt.
* src/cuirass/base.scm (process-specs): Skip compilation if #:no-compile?.

Signed-off-by: Mathieu Lirzin <mthl@gnu.org>
2016-09-23 16:50:36 +02:00
Mathieu Lirzin ac4512897c
schema: Evaluations: Add 'revision' column.
* src/schema.sql (Evaluations): Add 'revision' column.
* src/cuirass/database.scm (db-add-evaluation): Adapt.
All callers changed.
2016-07-30 23:07:19 +02:00
Mathieu Lirzin 24bfb61e8e
schema: Builds: Add 'evaluation' column.
* src/schema.sql (Builds): Add 'evaluation' column.
* src/cuirass/database.scm (db-add-build): Adapt.
* src/cuirass/base.scm (build-packages): Likewise.
2016-07-27 14:17:10 +02:00
Mathieu Lirzin d493a58823
schema: Separate "Derivations" from "Evaluations".
* src/schema.sql (Derivations): New table.
(Evaluations): Remove 'derivation' and 'job_name' columns.  Add 'id'
column.
* src/cuirass/database.scm (db-add-evaluation): Adapt.
(db-get-derivation, db-add-derivation): New procedures.
(evaluation-exists?, db-get-evaluation): Delete.
* bin/evaluate.in (main): Adapt.
* tests/database.scm ("sqlite-exec"): Likewise.
("db-add-derivation", "db-get-derivation"): New tests.
("db-add-evaluation", "db-get-evaluation"): Delete.
2016-07-27 14:17:09 +02:00
Mathieu Lirzin 719c126863
database: Remove 'db-delete-evaluation' procedure.
* src/cuirass/database.scm (db-delete-evaluation): Delete.
2016-07-27 14:17:09 +02:00
Mathieu Lirzin b8d8c99bd3
database: with-database: Keep the data.
* src/cuirass/database.scm (with-database): Use 'db-open'.
(db-open): Fallback to 'db-init' if DB do not exist.
2016-07-26 11:40:51 +02:00
Mathieu Lirzin f40ac8a535
database: db-open: Add DB optional argument.
* src/cuirass/database.scm (db-open): Add DB optional argument.
2016-07-26 11:21:27 +02:00
Mathieu Lirzin c83d5d4da8
Use stamps to poll repositories. 2016-07-26 00:36:12 +02:00
Mathieu Lirzin e51a755f10
database: Add 'db-add-build' procedure. 2016-07-25 20:32:01 +02:00
Mathieu Lirzin 7292bd5019
database: read-quoted-string: add default value to optional argument. 2016-07-25 20:32:01 +02:00
Mathieu Lirzin a063a2277e
cuirass: Use database to store specifications. 2016-07-25 02:37:00 +02:00
Mathieu Lirzin 5db6894ae2
database: db-init: Add arguments for debugging. 2016-07-25 02:12:42 +02:00
Mathieu Lirzin c7c9e91876
schema: Separate 'Evaluations' from 'Builds'.
Adapt src/cuirass/database.scm and its tests.
2016-07-25 02:12:41 +02:00
Mathieu Lirzin cf7e290dc2
maint: Update file headers. 2016-07-25 02:12:41 +02:00
Mathieu Lirzin a4c8bf0fd4
database: db-get-evaluation: Use 'sqlite-exec'. 2016-07-25 02:12:40 +02:00
Mathieu Lirzin 8ad3ce782f
database: Add 'last-insert-rowid' procedure. 2016-07-25 02:12:40 +02:00
Mathieu Lirzin 447d9f7ba8
database: Add 'assq-refs' procedure. 2016-07-25 02:12:40 +02:00
Mathieu Lirzin b1dd669196
database: sqlite-exec: Return every rows.
Export it.  Add a test.
2016-07-25 02:12:40 +02:00
Mathieu Lirzin 18e42be302
database: Use 'simple-format' in 'sqlite-exec'. 2016-07-25 02:12:40 +02:00
Mathieu Lirzin f9938a3caa
database: Don't catch exceptions from (sqlite3) module. 2016-07-25 02:12:39 +02:00
Mathieu Lirzin 46c9d432ea
database: Use an external SQL schema. 2016-07-25 02:12:39 +02:00
Mathieu Lirzin 0808ab3ecd Remove unnecessary (cuirass job) module. 2016-07-13 14:16:36 +02:00
Mathieu Lirzin 53c12be409 Evaluate derivations in a separate process.
This fixes a bug where different Guix branches gave the same
derivations.
2016-07-13 14:07:24 +02:00
Mathieu Lirzin 13db5aa618 λ all the things! 2016-07-02 15:49:34 +02:00
Mathieu Lirzin 9a9b3fbd45 database: Remove commented code. 2016-06-29 16:32:55 +02:00
Mathieu Lirzin 990c902fcc Store build logs in the database. 2016-06-29 16:16:48 +02:00
Mathieu Lirzin 5ff38984e8 Make %package-database a parameter object.
Move it to (cuirass database).
2016-06-26 22:54:49 +02:00
Mathieu Lirzin b24541e604 Add (cuirass database) module. 2016-06-26 16:59:26 +02:00