guix/guix-cuirass
2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/guix-cuirass.git synced 2023-12-14 06:03:04 +01:00
Code Issues Releases Wiki Activity
guix-cuirass/tests/http.scm

283 lines
9.4 KiB
Scheme
Raw Normal View History

Add (cuirass http) module. * src/cuirass/http.scm: New file. * tests/http.scm: Likewise. * Makefile.am (dist_pkgmodule_DATA, TESTS): Add them. * configure.ac: Check for (json) module.
2016-07-30 23:00:41 +02:00
;;; http.scm -- tests for (cuirass http) module
;;; Copyright © 2016 Mathieu Lirzin <mthl@gnu.org>
http: Add missing import of (ice-9 threads) in test. Using the core binding for 'call-with-new-thread' was deprecated in 2.2 and is removed in 3.0. * tests/http.scm: Use (ice-9 threads).
2020-01-18 15:16:01 +01:00
;;; Copyright © 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
Rewrite evaluation.
2021-03-03 15:25:04 +01:00
;;; Copyright © 2017, 2020, 2021 Mathieu Othacehe <m.othacehe@gmail.com>
database: Call a specification 'jobset' instead of 'project'. This removes the possibility to filter specifications by branch, because branches were previously called 'jobset'. But it doesn't matter because later on, specifications will have as many branches as inputs. And people should filter by specification name instead. * doc/cuirass.texi (Build Information, Latest builds): Remove 'jobset', replace 'project' with 'jobset'. * src/cuirass/http.scm (build->hydra-build): Idem. * tests/database.scm (db-get-builds): Idem. * tests/http.scm (build-query-result, /api/latestbuilds?nr=1&jobset=guix, /api/latestbuilds?nr=1&jobset=gnu): Idem. * src/cuirass/database.scm (db-format-build, db-get-builds): Don't associate builds with branches (which were called 'jobset' afterwards). (db-get-builds): Remove the #:project filter.
2018-07-02 17:25:31 +02:00
;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
Add (cuirass http) module. * src/cuirass/http.scm: New file. * tests/http.scm: Likewise. * Makefile.am (dist_pkgmodule_DATA, TESTS): Add them. * configure.ac: Check for (json) module.
2016-07-30 23:00:41 +02:00
;;;
;;; This file is part of Cuirass.
;;;
;;; Cuirass is free software: you can redistribute it and/or modify
;;; it under the terms of the GNU General Public License as published by
;;; the Free Software Foundation, either version 3 of the License, or
;;; (at your option) any later version.
;;;
;;; Cuirass is distributed in the hope that it will be useful,
;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with Cuirass. If not, see <http://www.gnu.org/licenses/>.
(use-modules (cuirass http)
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(cuirass database)
Rewrite evaluation.
2021-03-03 15:25:04 +01:00
(cuirass specification)
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(cuirass utils)
tests: Add a common module. * tests/common.scm: New file. * tests/database.scm: Use it. * tests/http.scm: Ditto. * tests/metrics.scm: Ditto.
2021-03-09 18:06:57 +01:00
(tests common)
Rewrite evaluation.
2021-03-03 15:25:04 +01:00
(guix channels)
tests: Compare JSON objects instead of JSON strings. This fixes a test failure with Guile 2.2. * tests/http.scm (hash-table-keys, hash-table=?): New procedures. ("spec->json-string"): Use it to compare the JSON objects.
2017-05-10 16:27:45 +02:00
(json)
http: Run test within 'run-fibers'. * tests/http.scm ("http")["cuirass-run"]: Run server with 'run-fibers'.
2018-01-26 14:07:09 +01:00
(fibers)
Switch to PostegreSQL.
2021-01-05 10:20:34 +01:00
(squee)
http: /build/N/log/raw redirects to /log. This moves log handling responsibility to 'guix publish'. * src/cuirass/http.scm (handle-log-request): Remove. (url-handler): Change /log/raw URI handler to return 302 to /log/OUTPUT. * tests/http.scm (log-file-name): Remove, and remove code to create and delete it. ("fill-db"): Change #:log value. ("/build/1/log/raw"): Expect 302.
2018-01-07 12:28:13 +01:00
(web uri)
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(web client)
(web response)
(rnrs bytevectors)
tests: Compare JSON objects instead of JSON strings. This fixes a test failure with Guile 2.2. * tests/http.scm (hash-table-keys, hash-table=?): New procedures. ("spec->json-string"): Use it to compare the JSON objects.
2017-05-10 16:27:45 +02:00
(srfi srfi-1)
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(srfi srfi-64)
http: Add missing import of (ice-9 threads) in test. Using the core binding for 'call-with-new-thread' was deprecated in 2.2 and is removed in 3.0. * tests/http.scm: Use (ice-9 threads).
2020-01-18 15:16:01 +01:00
(ice-9 threads)
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(ice-9 match))
Add (cuirass http) module. * src/cuirass/http.scm: New file. * tests/http.scm: Likewise. * Makefile.am (dist_pkgmodule_DATA, TESTS): Add them. * configure.ac: Check for (json) module.
2016-07-30 23:00:41 +02:00
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(define (http-get-body uri)
(call-with-values (lambda () (http-get uri))
(lambda (response body) body)))
(define (wait-until-ready port)
;; Wait until the server is accepting connections.
(let ((conn (socket PF_INET SOCK_STREAM 0)))
(let loop ()
(unless (false-if-exception
(connect conn AF_INET (inet-pton AF_INET "127.0.0.1") port))
(loop)))))
(define (test-cuirass-uri route)
(string-append "http://localhost:6688" route))
(define build-query-result
'((#:id . 1)
Fix tests. This is a follow-up of ad44a9bb0291d193a553a2ba33d37caa4e34724c. * tests/http.scm (build-query-result): Add "evaluation" field.
2020-12-07 16:27:45 +01:00
(#:evaluation . 1)
database: Call a specification 'jobset' instead of 'project'. This removes the possibility to filter specifications by branch, because branches were previously called 'jobset'. But it doesn't matter because later on, specifications will have as many branches as inputs. And people should filter by specification name instead. * doc/cuirass.texi (Build Information, Latest builds): Remove 'jobset', replace 'project' with 'jobset'. * src/cuirass/http.scm (build->hydra-build): Idem. * tests/database.scm (db-get-builds): Idem. * tests/http.scm (build-query-result, /api/latestbuilds?nr=1&jobset=guix, /api/latestbuilds?nr=1&jobset=gnu): Idem. * src/cuirass/database.scm (db-format-build, db-get-builds): Don't associate builds with branches (which were called 'jobset' afterwards). (db-get-builds): Remove the #:project filter.
2018-07-02 17:25:31 +02:00
(#:jobset . "guix")
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(#:job . "fake-job")
(#:timestamp . 1501347493)
(#:starttime . 1501347493)
(#:stoptime . 1501347493)
http: Adjust test to include #:derivation. This is a followup to commit c89c117c12792cbc4996035feb670c2535dcb105. * tests/http.scm (build-query-result): Add #:derivation.
2018-01-25 09:52:03 +01:00
(#:derivation . "/gnu/store/fake.drv")
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(#:buildoutputs . ((out ("path" . "/gnu/store/fake-1.0"))))
(#:system . "x86_64-linux")
(#:nixname . "fake-1.0")
(#:buildstatus . 0)
Fix weather value. * src/cuirass/database.scm (db-get-builds): Fix weather value. * tests/http.scm (build-query-result): Adapt it.
2021-02-02 14:27:49 +01:00
(#:weather . -1)
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(#:busy . 0)
tests: Add a common module. * tests/common.scm: New file. * tests/database.scm: Use it. * tests/http.scm: Ditto. * tests/metrics.scm: Ditto.
2021-03-09 18:06:57 +01:00
(#:priority . 9)
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(#:finished . 1)
tests: Add a common module. * tests/common.scm: New file. * tests/database.scm: Use it. * tests/http.scm: Ditto. * tests/metrics.scm: Ditto.
2021-03-09 18:06:57 +01:00
(#:buildproducts . #())))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
http: Add /api/evaluations route. * src/cuirass/database.scm (db-get-evaluations): New exported procedure. * src/cuirass/http.scm (url-handler): Add /api/evaluations route. * tests/http.scm ("http"): Add /api/evaluations test route.
2018-04-19 11:17:42 +02:00
(define evaluations-query-result
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
#(((#:id . 2)
database: Add a Checkouts table. It is used to know when a new evaluation must be triggered and to display input changes. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-3.sql'. * bin/cuirass.in (main): Call DB-SET-EVALUATION-DONE at startup to clear 'in-progress' evaluations. * bin/evaluate.in (input-checkout, format-checkouts): Rename '#:name' to '#:input'. * doc/cuirass.texi (Stamps): Remove section. (Checkouts): New section. * src/cuirass/base.scm (fetch-input, fetch-inputs, compile-checkouts): Rename '#:name' to '#:input'. (evaluate): Remove the COMMITS argument. Add an EVAL-ID argument. Don't call DB-ADD-EVALUATION because it was called sooner. Remove the EVAL-ID argument to AUGMENT-JOB because it's a closure. (build-packages): Add an EVAL-ID argument. Call DB-SET-EVALUATION-DONE once all the derivations are registered. (process-specs): Replace the stamping mechanism by the primary key constraint of the Checkouts table: call "evaluate" only when DB-ADD-EVALUATION is true, which means that at least one checkout was added. Change the EVALUATE and BUILD-PACKAGES arguments accordingly. * src/cuirass/database.scm (db-add-stamp, db-get-stamp): Remove procedures. (db-set-evaluations-done, db-set-evaluation-done): New exported procedure. (db-add-checkout): New procedure that returns #f if a checkout with the same revision already exists. (db-add-evaluation): Replace the EVAL argument with a SPEC-NAME and a CHECKOUTS arguments. Insert the evaluation only if at least one checkout was inserted. Return #f otherwise. (db-get-checkouts): New procedure. (db-get-evaluations, db-get-evaluations-build-summary): Handle the 'in_progress' column, remove the 'commits' column. Return the result of DB-GET-CHECKOUTS as part of the evaluation. * src/cuirass/templates.scm (input-changes, evaluation-badges): New procedures. (evaluation-info-table): Rename "Commits" to "Input changes". Use INPUT-CHANGES to display the input changes that triggered the evaluation. Use EVALUATION-BADGES to display a message indicating that the evaluation is in progress. * src/schema.sql (Stamps): Remove table. (Checkouts): New table. (Evaluations): Remove the 'commits' column. Add an 'in_progress' column. * src/sql/upgrade-3.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-eval): Remove procedure. (make-dummy-checkouts): New procedure. ("sqlite-exec"): Remove the 'commits' column. Add the 'in_progress' column. ("db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Update the arguments of DB-ADD-EVALUATION accordingly. * tests/http.scm (hash-table=?): Add support for lists of hash tables. (evaluations-query-result): Replace '#:commits' with '#:checkouts'. Return a list instead of returning one element, for symmetry. ("fill-db"): Add a new input so that the second checkout can refer to it. Replace EVALUATION1 and EVALUATION2 with CHECKOUTS1 and CHECKOUTS2. Update the arguments of DB-ADD-EVALUATION accordingly. ("/api/queue?nr=100"): Take the CAR of the EVALUATIONS-QUERY-RESULT list to make it symmetrical with the other argument of HASH-TABLE=?.
2018-08-11 20:30:11 +02:00
(#:specification . "guix")
Add a status field to Evaluation table. The Evaluation table currently has an 'in_progress' field. Distinction between succeeded and failed evaluations are based on the presence of Builds records for the evaluation. It it also not possible to distinguish aborted evaluations from failed evaluations. Rename 'in_progress' field to 'status'. The 'status' field can be equal to 'started', 'succeeded', 'failed' or 'aborted'. * src/cuirass/database.scm (evaluation-status): New exported enumeration. (db-set-evaluations-done, db-set-evaluation-done): Remove them. (db-abort-pending-evaluations, db-set-evaluation-status): New exported procedures. (db-add-evaluation, db-get-builds, db-get-evaluations, db-get-evaluations-build-summary, db-get-evaluation-summary): Adapt to use 'status' field instead of 'in_progress' field. * src/cuirass/templates.scm (evaluation-badges): Ditto. * src/schema.sql (Evaluations): Rename 'in_progress' field to 'status'. * src/sql/upgrade-10.sql: New file. * bin/cuirass.in (main): Use "db-abort-pending-evaluations" instead of "db-set-evaluations-done". * src/cuirass/base.scm (evaluate): Use "db-set-evaluation-status" instead of "db-set-evaluations-done". (build-packages): Use "db-set-evaluation-status" instead of "db-set-evaluation-done". * tests/database.scm (sqlite-exec): Adapt accordingly. * tests/http.scm (evaluations-query-result): Ditto.
2020-09-10 14:25:19 +02:00
(#:status . -1)
tests: Fix HTTP tests. This is a follow-up of 154232bc767d002f69aa6bb1cdddfd108b98584b. * tests/http.scm (evaluations-query-result): Add 'timestamp', 'checkouttime' and 'evaltime' fields. (fill-db): Pass a matching timestamp to "db-add-evaluation" calls.
2020-09-10 14:37:03 +02:00
(#:timestamp . 1501347493)
(#:checkouttime . 0)
(#:evaltime . 0)
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(#:checkouts . #(((#:commit . "fakesha2")
Rewrite evaluation.
2021-03-03 15:25:04 +01:00
(#:channel . "guix")
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(#:directory . "dir3")))))))
http: Add /api/evaluations route. * src/cuirass/database.scm (db-get-evaluations): New exported procedure. * src/cuirass/http.scm (url-handler): Add /api/evaluations route. * tests/http.scm ("http"): Add /api/evaluations test route.
2018-04-19 11:17:42 +02:00
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(test-group-with-cleanup "http"
(test-assert "object->json-string"
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(lset= equal?
(call-with-input-string
(string-append "{"
"\"boolean\" : false,"
"\"string\" : \"guix\","
"\"alist\" : {\"subset\" : \"hello\"},"
"\"list\" : [1, \"2\", \"three\"],"
"\"symbol\" : \"hydra-jobs\","
"\"number\" : 1"
"}")
json->scm)
(call-with-input-string
(object->json-string '((#:number . 1)
(string . "guix")
("symbol" . hydra-jobs)
(#:alist . ((subset . "hello")))
(list . #(1 "2" #:three))
("boolean" . #f)))
json->scm)))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(test-assert "db-init"
database: Serialize all database accesses in a thread. Fixes <https://bugs.gnu.org/32234>. * bin/cuirass.in (main): Keep only one WITH-DATABASE call around all fibers. Remove all DB arguments. * src/cuirass/base.scm (evaluate, update-build-statuses!, spawn-builds, handle-build-event, build-packages): Remove all DB arguments. (clear-build-queue, cancel-old-builds): Wrap in WITH-DB-CRITICAL-SECTION, remove all DB arguments. (restart-builds): Remove the NON-BLOCKING call, remove all DB arguments. (process-specs): Remove all DB arguments, remove the WITH-DATABASE call. * src/cuirass/database.scm (%db-channel): New parameter. (with-db-critical-section): New macro. (db-add-input, db-add-specification, db-get-inputs, db-get-specifications, db-add-evaluation, db-add-build, db-update-build-status!, db-get-outputs, db-get-builds, db-get-build, db-get-pending-derivations, db-get-stamp, db-add-stamp, db-get-evaluations, db-get-evaluations-build-summary, db-get-evaluations-id-min, db-get-evaluations-id-max, db-get-builds-min, db-get-builds-max): Wrap in WITH-DB-CRITICAL-SECTION, remove all DB arguments. (with-database): Wrap BODY in PARAMETERIZE form that sets %DB-CHANNEL to the channel returned by MAKE-CRITICAL-SECTION. * src/cuirass/http.scm (handle-build-request, handle-builds-request): Remove all DB arguments. (url-handler): Remove all DB arguments, remove the DB-CHANNEL state, remove the WITH-CRITICAL-SECTION calls. (run-cuirass-server): Remove the DB arguments, remove the MAKE-CRITICAL-SECTION call. * src/cuirass/utils.scm (make-critical-section): Replace SPAWN-FIBER with CALL-WITH-NEW-THREAD. Wrap body in PARAMETERIZE form that clears CURRENT-FIBER. * tests/database.scm (with-temporary-database, "db-add-specification", "db-add-build", "db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Remove the DB arguments. ("db-init"): Set the %DB-CHANNEL parameter to the channel returned by MAKE-CRITICAL-SECTION, and return #t. ("database"): Set %DB-CHANNEL to #f during cleanup. * tests/http.scm ("db-init"): Set the %DB-CHANNEL parameter to the channel returned by MAKE-CRITICAL-SECTION, and return #t. ("cuirass-run", "fill-db"): Remove the DB arguments. ("http"): Set %DB-CHANNEL to #f during cleanup.
2018-08-05 13:14:44 +02:00
(begin
tests: Add a common module. * tests/common.scm: New file. * tests/database.scm: Use it. * tests/http.scm: Ditto. * tests/metrics.scm: Ditto.
2021-03-09 18:06:57 +01:00
(test-init-db!)
database: Serialize all database accesses in a thread. Fixes <https://bugs.gnu.org/32234>. * bin/cuirass.in (main): Keep only one WITH-DATABASE call around all fibers. Remove all DB arguments. * src/cuirass/base.scm (evaluate, update-build-statuses!, spawn-builds, handle-build-event, build-packages): Remove all DB arguments. (clear-build-queue, cancel-old-builds): Wrap in WITH-DB-CRITICAL-SECTION, remove all DB arguments. (restart-builds): Remove the NON-BLOCKING call, remove all DB arguments. (process-specs): Remove all DB arguments, remove the WITH-DATABASE call. * src/cuirass/database.scm (%db-channel): New parameter. (with-db-critical-section): New macro. (db-add-input, db-add-specification, db-get-inputs, db-get-specifications, db-add-evaluation, db-add-build, db-update-build-status!, db-get-outputs, db-get-builds, db-get-build, db-get-pending-derivations, db-get-stamp, db-add-stamp, db-get-evaluations, db-get-evaluations-build-summary, db-get-evaluations-id-min, db-get-evaluations-id-max, db-get-builds-min, db-get-builds-max): Wrap in WITH-DB-CRITICAL-SECTION, remove all DB arguments. (with-database): Wrap BODY in PARAMETERIZE form that sets %DB-CHANNEL to the channel returned by MAKE-CRITICAL-SECTION. * src/cuirass/http.scm (handle-build-request, handle-builds-request): Remove all DB arguments. (url-handler): Remove all DB arguments, remove the DB-CHANNEL state, remove the WITH-CRITICAL-SECTION calls. (run-cuirass-server): Remove the DB arguments, remove the MAKE-CRITICAL-SECTION call. * src/cuirass/utils.scm (make-critical-section): Replace SPAWN-FIBER with CALL-WITH-NEW-THREAD. Wrap body in PARAMETERIZE form that clears CURRENT-FIBER. * tests/database.scm (with-temporary-database, "db-add-specification", "db-add-build", "db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Remove the DB arguments. ("db-init"): Set the %DB-CHANNEL parameter to the channel returned by MAKE-CRITICAL-SECTION, and return #t. ("database"): Set %DB-CHANNEL to #f during cleanup. * tests/http.scm ("db-init"): Set the %DB-CHANNEL parameter to the channel returned by MAKE-CRITICAL-SECTION, and return #t. ("cuirass-run", "fill-db"): Remove the DB arguments. ("http"): Set %DB-CHANNEL to #f during cleanup.
2018-08-05 13:14:44 +02:00
#t))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(test-assert "cuirass-run"
(call-with-new-thread
(lambda ()
http: Run test within 'run-fibers'. * tests/http.scm ("http")["cuirass-run"]: Run server with 'run-fibers'.
2018-01-26 14:07:09 +01:00
(run-fibers
(lambda ()
database: Serialize all database accesses in a thread. Fixes <https://bugs.gnu.org/32234>. * bin/cuirass.in (main): Keep only one WITH-DATABASE call around all fibers. Remove all DB arguments. * src/cuirass/base.scm (evaluate, update-build-statuses!, spawn-builds, handle-build-event, build-packages): Remove all DB arguments. (clear-build-queue, cancel-old-builds): Wrap in WITH-DB-CRITICAL-SECTION, remove all DB arguments. (restart-builds): Remove the NON-BLOCKING call, remove all DB arguments. (process-specs): Remove all DB arguments, remove the WITH-DATABASE call. * src/cuirass/database.scm (%db-channel): New parameter. (with-db-critical-section): New macro. (db-add-input, db-add-specification, db-get-inputs, db-get-specifications, db-add-evaluation, db-add-build, db-update-build-status!, db-get-outputs, db-get-builds, db-get-build, db-get-pending-derivations, db-get-stamp, db-add-stamp, db-get-evaluations, db-get-evaluations-build-summary, db-get-evaluations-id-min, db-get-evaluations-id-max, db-get-builds-min, db-get-builds-max): Wrap in WITH-DB-CRITICAL-SECTION, remove all DB arguments. (with-database): Wrap BODY in PARAMETERIZE form that sets %DB-CHANNEL to the channel returned by MAKE-CRITICAL-SECTION. * src/cuirass/http.scm (handle-build-request, handle-builds-request): Remove all DB arguments. (url-handler): Remove all DB arguments, remove the DB-CHANNEL state, remove the WITH-CRITICAL-SECTION calls. (run-cuirass-server): Remove the DB arguments, remove the MAKE-CRITICAL-SECTION call. * src/cuirass/utils.scm (make-critical-section): Replace SPAWN-FIBER with CALL-WITH-NEW-THREAD. Wrap body in PARAMETERIZE form that clears CURRENT-FIBER. * tests/database.scm (with-temporary-database, "db-add-specification", "db-add-build", "db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Remove the DB arguments. ("db-init"): Set the %DB-CHANNEL parameter to the channel returned by MAKE-CRITICAL-SECTION, and return #t. ("database"): Set %DB-CHANNEL to #f during cleanup. * tests/http.scm ("db-init"): Set the %DB-CHANNEL parameter to the channel returned by MAKE-CRITICAL-SECTION, and return #t. ("cuirass-run", "fill-db"): Remove the DB arguments. ("http"): Set %DB-CHANNEL to #f during cleanup.
2018-08-05 13:14:44 +02:00
(run-cuirass-server #:port 6688))
http: Run test within 'run-fibers'. * tests/http.scm ("http")["cuirass-run"]: Run server with 'run-fibers'.
2018-01-26 14:07:09 +01:00
#:drain? #t))))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(test-assert "wait-server"
(wait-until-ready 6688))
(test-assert "fill-db"
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(let* ((build1
`((#:derivation . "/gnu/store/fake.drv")
(#:eval-id . 1)
database: Merge Derivations into Builds table. Fixes <https://bugs.gnu.org/32190>. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-2.sql'. * doc/cuirass.texi (Derivations): Remove section. (Builds): Update accordingly. Add columns from the Derivations table. (Outputs): Replace 'id' with 'derivation'. * src/cuirass/base.scm (evaluate): Don't add jobs to the Derivations table. (build-packages): Add columns that were in the Derivations table before. Only build the derivations that were successfully registered, that is, those that didn't exist in the Builds table. Give a derivation instead of a build id to DB-GET-BUILD. Compute the number of failed jobs based on the derivations that were added to the table, instead of the jobs. * src/cuirass/database.scm (db-add-derivation, db-get-derivation): Remove exported procedures. (db-add-build): Catch SQLITE_CONSTRAINT_PRIMARYKEY error, which means that two jobs produced the same derivation, and return #f in that case. Add columns that were in the Derivations table before. Use 'derivation' as primary key for the Outputs table. (db-get-outputs): Use 'derivation' as identifier, instead of 'build-id'. (filters->order): Replace 'id' with 'rowid'. (db-get-builds): Add a 'derivation' filter. Replace 'id' with 'rowid'. Remove the 'INNER JOIN Derivations'. Replace Derivations with Builds. Return 'derivation' in first position to make it clear that it's the primary key. Pass DERIVATION instead of ID to DB-GET-OUTPUTS. (db-get-build): Allow to take a derivation as argument. Use NUMBER? to differentiate between derivations and ids. (db-get-pending-derivations): Remove the 'SELECT DISTINCT' clause now that derivations are unique. Remove the 'INNER JOIN Builds'. (db-get-evaluations-build-summary, db-get-builds-min, db-get-builds-max): Replace 'id' with 'rowid'. * src/schema.sql (Derivations): Remove table. (Outputs): Replace Builds.id with Builds.derivation. (Builds): Use 'derivation' as primary key. Remove the 'id' column. Add 'job_name', 'system', 'nix_name' columns that were in the Derivations table before. (Builds_Derivations_index): Rename to Builds_index. Update accordingly. (Derivations_index): Remove index. * src/sql/upgrade-2.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-job, make-dummy-derivation): Remove procedures. (make-dummy-build): Add columns that were in MAKE-DUMMY-DERIVATION. Get the DRV parameter to be mandatory because it's a primary key. (%id): Remove parameter. ("db-add-derivation", "db-get-derivation"): Remove tests. ("db-add-build"): Expect #f, because it adds twice the same derivation. Pass the derivation argument to MAKE-DUMMY-BUILD. ("db-update-build-status!"): Rename 'id' to 'derivation'. Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION call. ("db-get-builds", "db-get-pending-derivations"): Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION calls. * tests/http.scm ("fill-db"): Remove DERIVATION1 and DERIVATION2, and put their content in BUILD1 and BUILD2. Remove the DB-ADD-DERIVATION calls.
2018-08-01 00:03:12 +02:00
(#:job-name . "fake-job")
(#:system . "x86_64-linux")
(#:nix-name . "fake-1.0")
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(#:log . "unused so far")
(#:status . ,(build-status succeeded))
(#:outputs . (("out" . "/gnu/store/fake-1.0")))
(#:timestamp . 1501347493)
(#:starttime . 1501347493)
(#:stoptime . 1501347493)))
(build2
`((#:derivation . "/gnu/store/fake2.drv")
(#:eval-id . 1)
database: Merge Derivations into Builds table. Fixes <https://bugs.gnu.org/32190>. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-2.sql'. * doc/cuirass.texi (Derivations): Remove section. (Builds): Update accordingly. Add columns from the Derivations table. (Outputs): Replace 'id' with 'derivation'. * src/cuirass/base.scm (evaluate): Don't add jobs to the Derivations table. (build-packages): Add columns that were in the Derivations table before. Only build the derivations that were successfully registered, that is, those that didn't exist in the Builds table. Give a derivation instead of a build id to DB-GET-BUILD. Compute the number of failed jobs based on the derivations that were added to the table, instead of the jobs. * src/cuirass/database.scm (db-add-derivation, db-get-derivation): Remove exported procedures. (db-add-build): Catch SQLITE_CONSTRAINT_PRIMARYKEY error, which means that two jobs produced the same derivation, and return #f in that case. Add columns that were in the Derivations table before. Use 'derivation' as primary key for the Outputs table. (db-get-outputs): Use 'derivation' as identifier, instead of 'build-id'. (filters->order): Replace 'id' with 'rowid'. (db-get-builds): Add a 'derivation' filter. Replace 'id' with 'rowid'. Remove the 'INNER JOIN Derivations'. Replace Derivations with Builds. Return 'derivation' in first position to make it clear that it's the primary key. Pass DERIVATION instead of ID to DB-GET-OUTPUTS. (db-get-build): Allow to take a derivation as argument. Use NUMBER? to differentiate between derivations and ids. (db-get-pending-derivations): Remove the 'SELECT DISTINCT' clause now that derivations are unique. Remove the 'INNER JOIN Builds'. (db-get-evaluations-build-summary, db-get-builds-min, db-get-builds-max): Replace 'id' with 'rowid'. * src/schema.sql (Derivations): Remove table. (Outputs): Replace Builds.id with Builds.derivation. (Builds): Use 'derivation' as primary key. Remove the 'id' column. Add 'job_name', 'system', 'nix_name' columns that were in the Derivations table before. (Builds_Derivations_index): Rename to Builds_index. Update accordingly. (Derivations_index): Remove index. * src/sql/upgrade-2.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-job, make-dummy-derivation): Remove procedures. (make-dummy-build): Add columns that were in MAKE-DUMMY-DERIVATION. Get the DRV parameter to be mandatory because it's a primary key. (%id): Remove parameter. ("db-add-derivation", "db-get-derivation"): Remove tests. ("db-add-build"): Expect #f, because it adds twice the same derivation. Pass the derivation argument to MAKE-DUMMY-BUILD. ("db-update-build-status!"): Rename 'id' to 'derivation'. Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION call. ("db-get-builds", "db-get-pending-derivations"): Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION calls. * tests/http.scm ("fill-db"): Remove DERIVATION1 and DERIVATION2, and put their content in BUILD1 and BUILD2. Remove the DB-ADD-DERIVATION calls.
2018-08-01 00:03:12 +02:00
(#:job-name . "fake-job")
(#:system . "x86_64-linux")
(#:nix-name . "fake-2.0")
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(#:log . "unused so far")
(#:status . ,(build-status scheduled))
(#:outputs . (("out" . "/gnu/store/fake-2.0")))
(#:timestamp . 1501347493)
(#:starttime . 0)
(#:stoptime . 0)))
Rewrite evaluation.
2021-03-03 15:25:04 +01:00
(spec
(specification
(name "guix")
(build 'hello)
(channels
(list (channel
(name 'guix)
(url "https://gitlab.com/mothacehe/guix.git")
(branch "master"))
(channel
(name 'packages)
(url "https://gitlab.com/mothacehe/guix.git")
(branch "master"))))))
database: Add a Checkouts table. It is used to know when a new evaluation must be triggered and to display input changes. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-3.sql'. * bin/cuirass.in (main): Call DB-SET-EVALUATION-DONE at startup to clear 'in-progress' evaluations. * bin/evaluate.in (input-checkout, format-checkouts): Rename '#:name' to '#:input'. * doc/cuirass.texi (Stamps): Remove section. (Checkouts): New section. * src/cuirass/base.scm (fetch-input, fetch-inputs, compile-checkouts): Rename '#:name' to '#:input'. (evaluate): Remove the COMMITS argument. Add an EVAL-ID argument. Don't call DB-ADD-EVALUATION because it was called sooner. Remove the EVAL-ID argument to AUGMENT-JOB because it's a closure. (build-packages): Add an EVAL-ID argument. Call DB-SET-EVALUATION-DONE once all the derivations are registered. (process-specs): Replace the stamping mechanism by the primary key constraint of the Checkouts table: call "evaluate" only when DB-ADD-EVALUATION is true, which means that at least one checkout was added. Change the EVALUATE and BUILD-PACKAGES arguments accordingly. * src/cuirass/database.scm (db-add-stamp, db-get-stamp): Remove procedures. (db-set-evaluations-done, db-set-evaluation-done): New exported procedure. (db-add-checkout): New procedure that returns #f if a checkout with the same revision already exists. (db-add-evaluation): Replace the EVAL argument with a SPEC-NAME and a CHECKOUTS arguments. Insert the evaluation only if at least one checkout was inserted. Return #f otherwise. (db-get-checkouts): New procedure. (db-get-evaluations, db-get-evaluations-build-summary): Handle the 'in_progress' column, remove the 'commits' column. Return the result of DB-GET-CHECKOUTS as part of the evaluation. * src/cuirass/templates.scm (input-changes, evaluation-badges): New procedures. (evaluation-info-table): Rename "Commits" to "Input changes". Use INPUT-CHANGES to display the input changes that triggered the evaluation. Use EVALUATION-BADGES to display a message indicating that the evaluation is in progress. * src/schema.sql (Stamps): Remove table. (Checkouts): New table. (Evaluations): Remove the 'commits' column. Add an 'in_progress' column. * src/sql/upgrade-3.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-eval): Remove procedure. (make-dummy-checkouts): New procedure. ("sqlite-exec"): Remove the 'commits' column. Add the 'in_progress' column. ("db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Update the arguments of DB-ADD-EVALUATION accordingly. * tests/http.scm (hash-table=?): Add support for lists of hash tables. (evaluations-query-result): Replace '#:commits' with '#:checkouts'. Return a list instead of returning one element, for symmetry. ("fill-db"): Add a new input so that the second checkout can refer to it. Replace EVALUATION1 and EVALUATION2 with CHECKOUTS1 and CHECKOUTS2. Update the arguments of DB-ADD-EVALUATION accordingly. ("/api/queue?nr=100"): Take the CAR of the EVALUATIONS-QUERY-RESULT list to make it symmetrical with the other argument of HASH-TABLE=?.
2018-08-11 20:30:11 +02:00
(checkouts1
Rewrite evaluation.
2021-03-03 15:25:04 +01:00
(list
(checkout->channel-instance "dir1"
#:name 'guix
#:url "url1"
#:commit "fakesha1")
(checkout->channel-instance "dir2"
#:name 'packages
#:url "url2"
#:commit "fakesha3")))
database: Add a Checkouts table. It is used to know when a new evaluation must be triggered and to display input changes. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-3.sql'. * bin/cuirass.in (main): Call DB-SET-EVALUATION-DONE at startup to clear 'in-progress' evaluations. * bin/evaluate.in (input-checkout, format-checkouts): Rename '#:name' to '#:input'. * doc/cuirass.texi (Stamps): Remove section. (Checkouts): New section. * src/cuirass/base.scm (fetch-input, fetch-inputs, compile-checkouts): Rename '#:name' to '#:input'. (evaluate): Remove the COMMITS argument. Add an EVAL-ID argument. Don't call DB-ADD-EVALUATION because it was called sooner. Remove the EVAL-ID argument to AUGMENT-JOB because it's a closure. (build-packages): Add an EVAL-ID argument. Call DB-SET-EVALUATION-DONE once all the derivations are registered. (process-specs): Replace the stamping mechanism by the primary key constraint of the Checkouts table: call "evaluate" only when DB-ADD-EVALUATION is true, which means that at least one checkout was added. Change the EVALUATE and BUILD-PACKAGES arguments accordingly. * src/cuirass/database.scm (db-add-stamp, db-get-stamp): Remove procedures. (db-set-evaluations-done, db-set-evaluation-done): New exported procedure. (db-add-checkout): New procedure that returns #f if a checkout with the same revision already exists. (db-add-evaluation): Replace the EVAL argument with a SPEC-NAME and a CHECKOUTS arguments. Insert the evaluation only if at least one checkout was inserted. Return #f otherwise. (db-get-checkouts): New procedure. (db-get-evaluations, db-get-evaluations-build-summary): Handle the 'in_progress' column, remove the 'commits' column. Return the result of DB-GET-CHECKOUTS as part of the evaluation. * src/cuirass/templates.scm (input-changes, evaluation-badges): New procedures. (evaluation-info-table): Rename "Commits" to "Input changes". Use INPUT-CHANGES to display the input changes that triggered the evaluation. Use EVALUATION-BADGES to display a message indicating that the evaluation is in progress. * src/schema.sql (Stamps): Remove table. (Checkouts): New table. (Evaluations): Remove the 'commits' column. Add an 'in_progress' column. * src/sql/upgrade-3.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-eval): Remove procedure. (make-dummy-checkouts): New procedure. ("sqlite-exec"): Remove the 'commits' column. Add the 'in_progress' column. ("db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Update the arguments of DB-ADD-EVALUATION accordingly. * tests/http.scm (hash-table=?): Add support for lists of hash tables. (evaluations-query-result): Replace '#:commits' with '#:checkouts'. Return a list instead of returning one element, for symmetry. ("fill-db"): Add a new input so that the second checkout can refer to it. Replace EVALUATION1 and EVALUATION2 with CHECKOUTS1 and CHECKOUTS2. Update the arguments of DB-ADD-EVALUATION accordingly. ("/api/queue?nr=100"): Take the CAR of the EVALUATIONS-QUERY-RESULT list to make it symmetrical with the other argument of HASH-TABLE=?.
2018-08-11 20:30:11 +02:00
(checkouts2
Rewrite evaluation.
2021-03-03 15:25:04 +01:00
(list
(checkout->channel-instance "dir3"
#:name 'guix
#:url "dir3"
#:commit "fakesha2")
(checkout->channel-instance "dir4"
#:name 'packages
#:url "dir4"
#:commit "fakesha3"))))
(db-add-specification spec)
tests: Fix HTTP tests. This is a follow-up of 154232bc767d002f69aa6bb1cdddfd108b98584b. * tests/http.scm (evaluations-query-result): Add 'timestamp', 'checkouttime' and 'evaltime' fields. (fill-db): Pass a matching timestamp to "db-add-evaluation" calls.
2020-09-10 14:37:03 +02:00
(db-add-evaluation "guix" checkouts1
#:timestamp 1501347493)
(db-add-evaluation "guix" checkouts2
Switch to PostegreSQL.
2021-01-05 10:20:34 +01:00
#:timestamp 1501347493)
(db-add-build build1)
(db-add-build build2)))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
Fix /specifications route. Fixes <https://issues.guix.gnu.org/43163>. * src/cuirass/http.scm (specification->json-object): New procedure, (url-handler): use it for "/specifications" route to convert specification objects into a representation suitable for json->scm. * tests/http.scm ("/specifications"): Test the above route.
2020-09-02 10:43:22 +02:00
(test-assert "/specifications"
(match (call-with-input-string
(utf8->string
(http-get-body (test-cuirass-uri "/specifications")))
json->scm)
(#(spec)
Rewrite evaluation.
2021-03-03 15:25:04 +01:00
(string=? (assoc-ref spec "name") "guix"))))
Fix /specifications route. Fixes <https://issues.guix.gnu.org/43163>. * src/cuirass/http.scm (specification->json-object): New procedure, (url-handler): use it for "/specifications" route to convert specification objects into a representation suitable for json->scm. * tests/http.scm ("/specifications"): Test the above route.
2020-09-02 10:43:22 +02:00
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(test-assert "/build/1"
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(lset= equal?
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(call-with-input-string
(utf8->string
(http-get-body (test-cuirass-uri "/build/1")))
json->scm)
(call-with-input-string
(object->json-string build-query-result)
json->scm)))
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(test-equal "/build/42"
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
404
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(response-code (http-get (test-cuirass-uri "/build/42"))))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
tests: http: Add some erroneous routes. * tests/http.scm: Add new test cases covering some erroneous routes.
2020-08-03 14:48:45 +02:00
(test-equal "/build/42)"
404
(response-code (http-get (test-cuirass-uri "/build/42)"))))
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(test-equal "/build/42/log/raw"
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
404
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(response-code (http-get (test-cuirass-uri "/build/42/log/raw"))))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
tests: http: Add some erroneous routes. * tests/http.scm: Add new test cases covering some erroneous routes.
2020-08-03 14:48:45 +02:00
(test-equal "/build/42xx/log/raw"
404
(response-code (http-get (test-cuirass-uri "/build/42xx/log/raw"))))
(test-equal "/build/42/details"
404
(response-code (http-get (test-cuirass-uri "/build/42/details"))))
(test-equal "/build/42xx/details"
404
(response-code (http-get (test-cuirass-uri "/build/42xx/details"))))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(test-equal "/api/latestbuilds"
500
(response-code (http-get (test-cuirass-uri "/api/latestbuilds"))))
database: Call a specification 'jobset' instead of 'project'. This removes the possibility to filter specifications by branch, because branches were previously called 'jobset'. But it doesn't matter because later on, specifications will have as many branches as inputs. And people should filter by specification name instead. * doc/cuirass.texi (Build Information, Latest builds): Remove 'jobset', replace 'project' with 'jobset'. * src/cuirass/http.scm (build->hydra-build): Idem. * tests/database.scm (db-get-builds): Idem. * tests/http.scm (build-query-result, /api/latestbuilds?nr=1&jobset=guix, /api/latestbuilds?nr=1&jobset=gnu): Idem. * src/cuirass/database.scm (db-format-build, db-get-builds): Don't associate builds with branches (which were called 'jobset' afterwards). (db-get-builds): Remove the #:project filter.
2018-07-02 17:25:31 +02:00
(test-assert "/api/latestbuilds?nr=1&jobset=guix"
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(match (json-string->scm
(utf8->string
(http-get-body
(test-cuirass-uri
"/api/latestbuilds?nr=1&jobset=guix"))))
(#(build)
tests: Add a common module. * tests/common.scm: New file. * tests/database.scm: Use it. * tests/http.scm: Ditto. * tests/metrics.scm: Ditto.
2021-03-09 18:06:57 +01:00
(lset= equal? (pk build)
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(json-string->scm
(object->json-string build-query-result))))))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(test-equal "/api/latestbuilds?nr=1&jobset=gnu"
#() ;the result should be an empty JSON array
(json-string->scm
(utf8->string
(http-get-body
(test-cuirass-uri
"/api/latestbuilds?nr=1&jobset=gnu")))))
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
http: Handle request parameters with no value. Handle requests such as "/build/?nr" by ignoring parameters without any associated value. * src/cuirass/http.scm (request-parameters): Ignore silently parameters without an associated value. * tests/http.scm: Add a corresponding test case.
2020-08-03 15:27:00 +02:00
(test-equal "/api/latestbuilds?nr&jobset=gnu"
500
(response-code
(http-get
(test-cuirass-uri
"/api/latestbuilds?nr&jobset=gnu"))))
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
(test-equal "/api/queue?nr=100"
`("fake-2.0" ,(build-status scheduled))
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(match (json-string->scm
(utf8->string
(http-get-body
(test-cuirass-uri "/api/queue?nr=100"))))
(#(dictionary)
(list (assoc-ref dictionary "nixname")
(assoc-ref dictionary "buildstatus")))))
http: Implement /api/queue. * src/cuirass/http.scm (url-handler): Add /api/queue handler. * tests/http.scm ("http"): Add a BUILD2 and DERIVATION2, and rename BUILD and DERIVATION accordingly. ("/build/2", "/build/2/log/raw"): Rename to /42. ("/api/queue?nr=100"): New test.
2018-01-23 23:37:13 +01:00
Switch to Guile-JSON 3.x. Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * src/cuirass/http.scm (evaluation->json-object): New procedure. (handle-builds-request): Pass the result through 'list->vector'. (handle-builds-search-request): Likewise. (url-handler): Likewise for /jobsets, /specifications, /api/evaluations, and /build. For /api/evaluations, use 'evaluation->json-object'. * src/cuirass/utils.scm (object->json-scm): Add 'vector?' case. * tests/http.scm (hash-table-keys, hash-table=?): Remove. (evaluations-query-result): Use vectors for JSON arrays. ("object->json-string"): Expects alists instead of hash tables. ("/build/1"): Use 'lset=' instead of 'hash-table=?'. ("/api/latestbuilds?nr=1&jobset=guix"): Likewise, and expect alists instead of hash tables. ("/api/latestbuilds?nr=1&jobset=gnu"): Likewise. ("/api/evaluations?nr=1"): Likewise. * README: Mention Guile-JSON 3.x.
2019-08-17 18:48:34 +02:00
(test-equal "/api/evaluations?nr=1"
(json-string->scm
(object->json-string evaluations-query-result))
(json-string->scm
(utf8->string
(http-get-body (test-cuirass-uri "/api/evaluations?nr=1")))))
http: Add /api/evaluations route. * src/cuirass/database.scm (db-get-evaluations): New exported procedure. * src/cuirass/http.scm (url-handler): Add /api/evaluations route. * tests/http.scm ("http"): Add /api/evaluations test route.
2018-04-19 11:17:42 +02:00
cuirass: add Hydra compatible HTTP API. * doc/cuirass.texi (Sections)[Web API]: New section describing the HTTP API. * src/cuirass/http.scm (spec->json-string): Move it to utils.scm and rename it object->json-string. (object->json-scm): Move it utils.scm. (handle-*-request): New helpers procedures. (request-parameters): New procedure to parse a request query. (url-handler): Add new API's. * src/cuirass/utils.scm (object->json-scm, object->json-string): Exported procedures moved from http.scm. * tests/http.scm: Add various tests on new HTTP API.
2017-07-31 19:27:28 +02:00
(test-assert "db-close"
Switch to PostegreSQL.
2021-01-05 10:20:34 +01:00
(begin
(exec-query (%db) (format #f "DROP OWNED BY CURRENT_USER;"))
(db-close (%db))
#t)))
Copy permalink