2016-06-26 15:13:31 +02:00
|
|
|
;;;; database.scm - tests for (cuirass database) module
|
|
|
|
;;;
|
|
|
|
;;; Copyright © 2016 Mathieu Lirzin <mthl@gnu.org>
|
2018-01-07 10:19:56 +01:00
|
|
|
;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
|
2018-07-02 17:25:31 +02:00
|
|
|
;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
|
2016-06-26 15:13:31 +02:00
|
|
|
;;;
|
|
|
|
;;; This file is part of Cuirass.
|
|
|
|
;;;
|
|
|
|
;;; Cuirass is free software; you can redistribute it and/or modify it
|
|
|
|
;;; under the terms of the GNU General Public License as published by
|
|
|
|
;;; the Free Software Foundation; either version 3 of the License, or (at
|
|
|
|
;;; your option) any later version.
|
|
|
|
;;;
|
|
|
|
;;; Cuirass is distributed in the hope that it will be useful, but
|
|
|
|
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
;;; GNU General Public License for more details.
|
|
|
|
;;;
|
|
|
|
;;; You should have received a copy of the GNU General Public License
|
|
|
|
;;; along with Cuirass. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
|
|
|
|
(use-modules (cuirass database)
|
2018-01-07 10:19:56 +01:00
|
|
|
((guix utils) #:select (call-with-temporary-output-file))
|
2016-06-26 15:13:31 +02:00
|
|
|
(srfi srfi-64))
|
|
|
|
|
2016-07-23 22:05:50 +02:00
|
|
|
(define example-spec
|
2016-11-13 01:54:41 +01:00
|
|
|
'((#:name . "guix")
|
Add support for multiple inputs.
* Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql.
* bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that
was used afterwards as %GUIX-PACKAGE-PATH.
* bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path,
spec-package-path, format-checkouts): New procedures.
(%not-colon): Remove variable.
(main): Take the load path, package path and PROC from the checkouts that
result from the inputs. Format the checkouts before sending them to the
procedure. Remove the LOAD-PATH argument.
* doc/cuirass.texi (Overview, Database schema): Document the changes.
* examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm,
hello-subset.scm, random.scm}: Adapt to the new specification format.
* examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT.
(package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle
the new format of its return value.
* examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT.
Rename the checkout from 'random (which is a specification) to 'cuirass (which
is a checkout resulting from an input).
* src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC
to INPUT. Return a checkout object instead of returning two values.
(evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of
SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the
SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object
instead of getting it from "evaluate".
(compile?, fetch-inputs, compile-checkouts): New procedures.
(process-specs): Fetch all inputs instead of only fetching one repository.
The result of that fetching operation is a list of CHECKOUTS whose COMMITS are
used as a STAMP.
(%guix-package-path, set-guix-package-path): Remove them.
* src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures.
(db-add-specification, db-get-specifications): Adapt to the new specification
format. Add/get all inputs as well.
(db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space
separated commit hashes.
(db-get-builds): Rename REPO_NAME to NAME.
(db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP.
(db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return
value.
(db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS.
* src/cuirass/utils.scm (%non-blocking): Export it.
* src/schema.sql (Inputs): New table that refers to the Specifications table.
(Specifications): Move input related fields to the Inputs table. Rename
REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE.
Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to
the Inputs table.
(Stamps): Rename REPO_NAME to NAME.
(Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS.
(Specifications_index): Replace with Inputs_index.
* src/sql/upgrade-1.sql: New file.
* tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to
the new specifications format. Rename REVISION to COMMITS.
* tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
|
|
|
(#:load-path-inputs . ("savannah"))
|
|
|
|
(#:package-path-inputs . ())
|
|
|
|
(#:proc-input . "savannah")
|
|
|
|
(#:proc-file . "/tmp/gnu-system.scm")
|
2016-07-23 22:05:50 +02:00
|
|
|
(#:proc . hydra-jobs)
|
Add support for multiple inputs.
* Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql.
* bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that
was used afterwards as %GUIX-PACKAGE-PATH.
* bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path,
spec-package-path, format-checkouts): New procedures.
(%not-colon): Remove variable.
(main): Take the load path, package path and PROC from the checkouts that
result from the inputs. Format the checkouts before sending them to the
procedure. Remove the LOAD-PATH argument.
* doc/cuirass.texi (Overview, Database schema): Document the changes.
* examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm,
hello-subset.scm, random.scm}: Adapt to the new specification format.
* examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT.
(package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle
the new format of its return value.
* examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT.
Rename the checkout from 'random (which is a specification) to 'cuirass (which
is a checkout resulting from an input).
* src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC
to INPUT. Return a checkout object instead of returning two values.
(evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of
SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the
SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object
instead of getting it from "evaluate".
(compile?, fetch-inputs, compile-checkouts): New procedures.
(process-specs): Fetch all inputs instead of only fetching one repository.
The result of that fetching operation is a list of CHECKOUTS whose COMMITS are
used as a STAMP.
(%guix-package-path, set-guix-package-path): Remove them.
* src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures.
(db-add-specification, db-get-specifications): Adapt to the new specification
format. Add/get all inputs as well.
(db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space
separated commit hashes.
(db-get-builds): Rename REPO_NAME to NAME.
(db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP.
(db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return
value.
(db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS.
* src/cuirass/utils.scm (%non-blocking): Export it.
* src/schema.sql (Inputs): New table that refers to the Specifications table.
(Specifications): Move input related fields to the Inputs table. Rename
REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE.
Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to
the Inputs table.
(Stamps): Rename REPO_NAME to NAME.
(Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS.
(Specifications_index): Replace with Inputs_index.
* src/sql/upgrade-1.sql: New file.
* tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to
the new specifications format. Rename REVISION to COMMITS.
* tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
|
|
|
(#:proc-args (subset . "hello"))
|
|
|
|
(#:inputs . (((#:name . "savannah")
|
|
|
|
(#:url . "git://git.savannah.gnu.org/guix.git")
|
|
|
|
(#:load-path . ".")
|
|
|
|
(#:branch . "master")
|
|
|
|
(#:tag . #f)
|
|
|
|
(#:commit . #f)
|
|
|
|
(#:no-compile? . #f))
|
|
|
|
((#:name . "maintenance")
|
|
|
|
(#:url . "git://git.savannah.gnu.org/guix/maintenance.git")
|
|
|
|
(#:load-path . ".")
|
|
|
|
(#:branch . "master")
|
|
|
|
(#:tag . #f)
|
|
|
|
(#:commit . #f)
|
|
|
|
(#:no-compile? . #f))))))
|
2016-07-23 22:05:50 +02:00
|
|
|
|
Add support for multiple inputs.
* Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql.
* bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that
was used afterwards as %GUIX-PACKAGE-PATH.
* bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path,
spec-package-path, format-checkouts): New procedures.
(%not-colon): Remove variable.
(main): Take the load path, package path and PROC from the checkouts that
result from the inputs. Format the checkouts before sending them to the
procedure. Remove the LOAD-PATH argument.
* doc/cuirass.texi (Overview, Database schema): Document the changes.
* examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm,
hello-subset.scm, random.scm}: Adapt to the new specification format.
* examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT.
(package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle
the new format of its return value.
* examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT.
Rename the checkout from 'random (which is a specification) to 'cuirass (which
is a checkout resulting from an input).
* src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC
to INPUT. Return a checkout object instead of returning two values.
(evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of
SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the
SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object
instead of getting it from "evaluate".
(compile?, fetch-inputs, compile-checkouts): New procedures.
(process-specs): Fetch all inputs instead of only fetching one repository.
The result of that fetching operation is a list of CHECKOUTS whose COMMITS are
used as a STAMP.
(%guix-package-path, set-guix-package-path): Remove them.
* src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures.
(db-add-specification, db-get-specifications): Adapt to the new specification
format. Add/get all inputs as well.
(db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space
separated commit hashes.
(db-get-builds): Rename REPO_NAME to NAME.
(db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP.
(db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return
value.
(db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS.
* src/cuirass/utils.scm (%non-blocking): Export it.
* src/schema.sql (Inputs): New table that refers to the Specifications table.
(Specifications): Move input related fields to the Inputs table. Rename
REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE.
Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to
the Inputs table.
(Stamps): Rename REPO_NAME to NAME.
(Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS.
(Specifications_index): Replace with Inputs_index.
* src/sql/upgrade-1.sql: New file.
* tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to
the new specifications format. Rename REVISION to COMMITS.
* tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
|
|
|
(define* (make-dummy-eval #:optional (commits '("cabba3e 61730ea")))
|
2018-01-07 10:19:56 +01:00
|
|
|
`((#:specification . "guix")
|
Add support for multiple inputs.
* Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql.
* bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that
was used afterwards as %GUIX-PACKAGE-PATH.
* bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path,
spec-package-path, format-checkouts): New procedures.
(%not-colon): Remove variable.
(main): Take the load path, package path and PROC from the checkouts that
result from the inputs. Format the checkouts before sending them to the
procedure. Remove the LOAD-PATH argument.
* doc/cuirass.texi (Overview, Database schema): Document the changes.
* examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm,
hello-subset.scm, random.scm}: Adapt to the new specification format.
* examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT.
(package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle
the new format of its return value.
* examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT.
Rename the checkout from 'random (which is a specification) to 'cuirass (which
is a checkout resulting from an input).
* src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC
to INPUT. Return a checkout object instead of returning two values.
(evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of
SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the
SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object
instead of getting it from "evaluate".
(compile?, fetch-inputs, compile-checkouts): New procedures.
(process-specs): Fetch all inputs instead of only fetching one repository.
The result of that fetching operation is a list of CHECKOUTS whose COMMITS are
used as a STAMP.
(%guix-package-path, set-guix-package-path): Remove them.
* src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures.
(db-add-specification, db-get-specifications): Adapt to the new specification
format. Add/get all inputs as well.
(db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space
separated commit hashes.
(db-get-builds): Rename REPO_NAME to NAME.
(db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP.
(db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return
value.
(db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS.
* src/cuirass/utils.scm (%non-blocking): Export it.
* src/schema.sql (Inputs): New table that refers to the Specifications table.
(Specifications): Move input related fields to the Inputs table. Rename
REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE.
Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to
the Inputs table.
(Stamps): Rename REPO_NAME to NAME.
(Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS.
(Specifications_index): Replace with Inputs_index.
* src/sql/upgrade-1.sql: New file.
* tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to
the new specifications format. Rename REVISION to COMMITS.
* tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
|
|
|
(#:commits . ,commits)))
|
2018-01-07 10:19:56 +01:00
|
|
|
|
2016-07-12 11:42:20 +02:00
|
|
|
(define* (make-dummy-job #:optional (name "foo"))
|
|
|
|
`((#:name . ,name)
|
database: Use 'sqlite-bind' to avoid SQL injection.
* src/cuirass/database.scm (%sqlite-exec): Remove.
(sqlite-exec): Turn back into a procedure. Use 'sqlite-bind'. Add
'normalize' procedure and use it.
(db-add-specification, db-add-derivation, db-get-derivation)
(db-add-evaluation, db-add-build, db-update-build-status!)
(db-get-build, db-get-stamp, db-add-stamp): Use question marks in SQL
queries.
* src/cuirass/base.scm (build-packages)[register]: Make #:log
non-false.
* tests/database.scm (make-dummy-job): Add #:job-name, #:system,
#:nix-name, and #:eval-id. This is necessary because 'sqlite-bind'
would now translate #f to a real NULL (before it would translate to the
string "#f"...), and would thus report violations of the non-NULL
constraint.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2018-02-08 11:39:45 +01:00
|
|
|
(#:job-name . "job")
|
|
|
|
(#:system . "x86_64-linux")
|
2016-07-23 17:00:38 +02:00
|
|
|
(#:derivation . ,(string-append name ".drv"))
|
database: Use 'sqlite-bind' to avoid SQL injection.
* src/cuirass/database.scm (%sqlite-exec): Remove.
(sqlite-exec): Turn back into a procedure. Use 'sqlite-bind'. Add
'normalize' procedure and use it.
(db-add-specification, db-add-derivation, db-get-derivation)
(db-add-evaluation, db-add-build, db-update-build-status!)
(db-get-build, db-get-stamp, db-add-stamp): Use question marks in SQL
queries.
* src/cuirass/base.scm (build-packages)[register]: Make #:log
non-false.
* tests/database.scm (make-dummy-job): Add #:job-name, #:system,
#:nix-name, and #:eval-id. This is necessary because 'sqlite-bind'
would now translate #f to a real NULL (before it would translate to the
string "#f"...), and would thus report violations of the non-NULL
constraint.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2018-02-08 11:39:45 +01:00
|
|
|
(#:nix-name . "foo")
|
|
|
|
(#:specification 0)
|
|
|
|
(#:eval-id . 42)))
|
2016-06-26 15:13:31 +02:00
|
|
|
|
2018-01-07 10:19:56 +01:00
|
|
|
(define* (make-dummy-derivation drv #:optional (eval-id 0))
|
|
|
|
`((#:derivation . ,drv)
|
|
|
|
(#:job-name . "job")
|
|
|
|
(#:system . "x86_64-linux")
|
|
|
|
(#:nix-name . ,(basename drv ".drv"))
|
|
|
|
(#:eval-id . ,eval-id)))
|
|
|
|
|
|
|
|
(define* (make-dummy-build #:optional (eval-id 42)
|
|
|
|
#:key (drv "/foo.drv")
|
|
|
|
(outputs '(("foo" . "/foo"))))
|
|
|
|
`((#:derivation . ,drv)
|
|
|
|
(#:eval-id . ,eval-id)
|
|
|
|
(#:log . "log")
|
|
|
|
(#:outputs . (("foo" . "/foo")))))
|
|
|
|
|
|
|
|
(define-syntax-rule (with-temporary-database db body ...)
|
|
|
|
(call-with-temporary-output-file
|
|
|
|
(lambda (file port)
|
|
|
|
(parameterize ((%package-database file))
|
|
|
|
(db-init file)
|
|
|
|
(with-database db
|
|
|
|
body ...)))))
|
|
|
|
|
2016-06-26 18:40:31 +02:00
|
|
|
(define %db
|
|
|
|
;; Global Slot for a database object.
|
|
|
|
(make-parameter #t))
|
|
|
|
|
|
|
|
(define %id
|
|
|
|
;; Global Slot for a job ID in the database.
|
|
|
|
(make-parameter #t))
|
|
|
|
|
2016-08-05 15:04:04 +02:00
|
|
|
(define database-name
|
|
|
|
;; Use an empty and temporary database for the tests.
|
|
|
|
(string-append (getcwd) "/" (number->string (getpid)) "-tmp.db"))
|
2016-07-27 13:43:53 +02:00
|
|
|
|
2016-08-05 15:04:04 +02:00
|
|
|
(test-group-with-cleanup "database"
|
|
|
|
(test-assert "db-init"
|
|
|
|
(%db (db-init database-name)))
|
2016-06-26 18:40:31 +02:00
|
|
|
|
2016-08-05 15:04:04 +02:00
|
|
|
(test-assert "sqlite-exec"
|
|
|
|
(begin
|
|
|
|
(sqlite-exec (%db) "\
|
Add support for multiple inputs.
* Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql.
* bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that
was used afterwards as %GUIX-PACKAGE-PATH.
* bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path,
spec-package-path, format-checkouts): New procedures.
(%not-colon): Remove variable.
(main): Take the load path, package path and PROC from the checkouts that
result from the inputs. Format the checkouts before sending them to the
procedure. Remove the LOAD-PATH argument.
* doc/cuirass.texi (Overview, Database schema): Document the changes.
* examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm,
hello-subset.scm, random.scm}: Adapt to the new specification format.
* examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT.
(package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle
the new format of its return value.
* examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT.
Rename the checkout from 'random (which is a specification) to 'cuirass (which
is a checkout resulting from an input).
* src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC
to INPUT. Return a checkout object instead of returning two values.
(evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of
SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the
SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object
instead of getting it from "evaluate".
(compile?, fetch-inputs, compile-checkouts): New procedures.
(process-specs): Fetch all inputs instead of only fetching one repository.
The result of that fetching operation is a list of CHECKOUTS whose COMMITS are
used as a STAMP.
(%guix-package-path, set-guix-package-path): Remove them.
* src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures.
(db-add-specification, db-get-specifications): Adapt to the new specification
format. Add/get all inputs as well.
(db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space
separated commit hashes.
(db-get-builds): Rename REPO_NAME to NAME.
(db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP.
(db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return
value.
(db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS.
* src/cuirass/utils.scm (%non-blocking): Export it.
* src/schema.sql (Inputs): New table that refers to the Specifications table.
(Specifications): Move input related fields to the Inputs table. Rename
REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE.
Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to
the Inputs table.
(Stamps): Rename REPO_NAME to NAME.
(Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS.
(Specifications_index): Replace with Inputs_index.
* src/sql/upgrade-1.sql: New file.
* tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to
the new specifications format. Rename REVISION to COMMITS.
* tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
|
|
|
INSERT INTO Evaluations (specification, commits) VALUES (1, 1);")
|
2016-08-05 15:04:04 +02:00
|
|
|
(sqlite-exec (%db) "\
|
Add support for multiple inputs.
* Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql.
* bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that
was used afterwards as %GUIX-PACKAGE-PATH.
* bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path,
spec-package-path, format-checkouts): New procedures.
(%not-colon): Remove variable.
(main): Take the load path, package path and PROC from the checkouts that
result from the inputs. Format the checkouts before sending them to the
procedure. Remove the LOAD-PATH argument.
* doc/cuirass.texi (Overview, Database schema): Document the changes.
* examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm,
hello-subset.scm, random.scm}: Adapt to the new specification format.
* examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT.
(package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle
the new format of its return value.
* examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT.
Rename the checkout from 'random (which is a specification) to 'cuirass (which
is a checkout resulting from an input).
* src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC
to INPUT. Return a checkout object instead of returning two values.
(evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of
SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the
SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object
instead of getting it from "evaluate".
(compile?, fetch-inputs, compile-checkouts): New procedures.
(process-specs): Fetch all inputs instead of only fetching one repository.
The result of that fetching operation is a list of CHECKOUTS whose COMMITS are
used as a STAMP.
(%guix-package-path, set-guix-package-path): Remove them.
* src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures.
(db-add-specification, db-get-specifications): Adapt to the new specification
format. Add/get all inputs as well.
(db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space
separated commit hashes.
(db-get-builds): Rename REPO_NAME to NAME.
(db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP.
(db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return
value.
(db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS.
* src/cuirass/utils.scm (%non-blocking): Export it.
* src/schema.sql (Inputs): New table that refers to the Specifications table.
(Specifications): Move input related fields to the Inputs table. Rename
REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE.
Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to
the Inputs table.
(Stamps): Rename REPO_NAME to NAME.
(Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS.
(Specifications_index): Replace with Inputs_index.
* src/sql/upgrade-1.sql: New file.
* tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to
the new specifications format. Rename REVISION to COMMITS.
* tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
|
|
|
INSERT INTO Evaluations (specification, commits) VALUES (2, 2);")
|
2016-08-05 15:04:04 +02:00
|
|
|
(sqlite-exec (%db) "\
|
Add support for multiple inputs.
* Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql.
* bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that
was used afterwards as %GUIX-PACKAGE-PATH.
* bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path,
spec-package-path, format-checkouts): New procedures.
(%not-colon): Remove variable.
(main): Take the load path, package path and PROC from the checkouts that
result from the inputs. Format the checkouts before sending them to the
procedure. Remove the LOAD-PATH argument.
* doc/cuirass.texi (Overview, Database schema): Document the changes.
* examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm,
hello-subset.scm, random.scm}: Adapt to the new specification format.
* examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT.
(package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle
the new format of its return value.
* examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT.
Rename the checkout from 'random (which is a specification) to 'cuirass (which
is a checkout resulting from an input).
* src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC
to INPUT. Return a checkout object instead of returning two values.
(evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of
SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the
SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object
instead of getting it from "evaluate".
(compile?, fetch-inputs, compile-checkouts): New procedures.
(process-specs): Fetch all inputs instead of only fetching one repository.
The result of that fetching operation is a list of CHECKOUTS whose COMMITS are
used as a STAMP.
(%guix-package-path, set-guix-package-path): Remove them.
* src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures.
(db-add-specification, db-get-specifications): Adapt to the new specification
format. Add/get all inputs as well.
(db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space
separated commit hashes.
(db-get-builds): Rename REPO_NAME to NAME.
(db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP.
(db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return
value.
(db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS.
* src/cuirass/utils.scm (%non-blocking): Export it.
* src/schema.sql (Inputs): New table that refers to the Specifications table.
(Specifications): Move input related fields to the Inputs table. Rename
REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE.
Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to
the Inputs table.
(Stamps): Rename REPO_NAME to NAME.
(Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS.
(Specifications_index): Replace with Inputs_index.
* src/sql/upgrade-1.sql: New file.
* tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to
the new specifications format. Rename REVISION to COMMITS.
* tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
|
|
|
INSERT INTO Evaluations (specification, commits) VALUES (3, 3);")
|
2016-08-05 15:04:04 +02:00
|
|
|
(sqlite-exec (%db) "SELECT * FROM Evaluations;")))
|
2016-07-22 12:35:09 +02:00
|
|
|
|
2016-08-05 15:04:04 +02:00
|
|
|
(test-equal "db-add-specification"
|
|
|
|
example-spec
|
|
|
|
(begin
|
|
|
|
(db-add-specification (%db) example-spec)
|
|
|
|
(car (db-get-specifications (%db)))))
|
2016-07-23 22:05:50 +02:00
|
|
|
|
2016-08-05 15:04:04 +02:00
|
|
|
(test-assert "db-add-derivation"
|
|
|
|
(let* ((job (make-dummy-job))
|
|
|
|
(key (assq-ref job #:derivation)))
|
|
|
|
(db-add-derivation (%db) job)
|
2018-02-14 16:40:50 +01:00
|
|
|
(db-add-derivation (%db) job) ;idempotent
|
2016-08-05 15:04:04 +02:00
|
|
|
(%id key)))
|
2016-06-26 18:40:31 +02:00
|
|
|
|
2016-08-05 15:04:04 +02:00
|
|
|
(test-assert "db-get-derivation"
|
|
|
|
(db-get-derivation (%db) (%id)))
|
2016-06-26 18:40:31 +02:00
|
|
|
|
2017-08-26 10:42:40 +02:00
|
|
|
(test-assert "db-add-build"
|
2018-01-07 10:19:56 +01:00
|
|
|
(let ((build (make-dummy-build)))
|
2017-08-26 10:42:40 +02:00
|
|
|
(db-add-build (%db) build)
|
|
|
|
|
|
|
|
;; This should be idempotent, see <https://bugs.gnu.org/28094>.
|
|
|
|
(db-add-build (%db) build)))
|
|
|
|
|
2018-01-23 18:15:42 +01:00
|
|
|
(test-equal "db-update-build-status!"
|
|
|
|
(list (build-status scheduled)
|
|
|
|
(build-status started)
|
2018-01-24 23:40:07 +01:00
|
|
|
(build-status succeeded)
|
|
|
|
"/foo.drv.log")
|
2018-01-23 18:15:42 +01:00
|
|
|
(with-temporary-database db
|
|
|
|
(let* ((id (db-add-build
|
|
|
|
db
|
|
|
|
(make-dummy-build 1 #:drv "/foo.drv"
|
|
|
|
#:outputs '(("out" . "/foo")))))
|
|
|
|
(get-status (lambda* (#:optional (key #:status))
|
|
|
|
(assq-ref (db-get-build db id) key))))
|
|
|
|
(db-add-derivation db (make-dummy-derivation "/foo.drv" 1))
|
|
|
|
(db-add-evaluation db (make-dummy-eval))
|
|
|
|
(db-add-specification db example-spec)
|
|
|
|
|
|
|
|
(let ((status0 (get-status)))
|
|
|
|
(db-update-build-status! db "/foo.drv" (build-status started))
|
|
|
|
(let ((status1 (get-status)))
|
2018-01-24 23:40:07 +01:00
|
|
|
(db-update-build-status! db "/foo.drv" (build-status succeeded)
|
|
|
|
#:log-file "/foo.drv.log")
|
2018-02-08 11:59:42 +01:00
|
|
|
|
|
|
|
;; Second call shouldn't make any difference.
|
|
|
|
(db-update-build-status! db "/foo.drv" (build-status succeeded)
|
|
|
|
#:log-file "/foo.drv.log")
|
|
|
|
|
2018-01-23 18:15:42 +01:00
|
|
|
(let ((status2 (get-status))
|
|
|
|
(start (get-status #:starttime))
|
2018-01-24 23:40:07 +01:00
|
|
|
(end (get-status #:stoptime))
|
|
|
|
(log (get-status #:log)))
|
2018-01-23 18:15:42 +01:00
|
|
|
(and (> start 0) (>= end start)
|
2018-01-24 23:40:07 +01:00
|
|
|
(list status0 status1 status2 log))))))))
|
2018-01-23 18:15:42 +01:00
|
|
|
|
2018-03-29 15:30:57 +02:00
|
|
|
(test-equal "db-get-builds"
|
|
|
|
#(((1 "/foo.drv") (2 "/bar.drv") (3 "/baz.drv")) ;ascending order
|
|
|
|
((3 "/baz.drv") (2 "/bar.drv") (1 "/foo.drv")) ;descending order
|
|
|
|
((3 "/baz.drv") (2 "/bar.drv") (1 "/foo.drv")) ;ditto
|
|
|
|
((3 "/baz.drv")) ;nr = 1
|
|
|
|
((2 "/bar.drv") (1 "/foo.drv") (3 "/baz.drv"))) ;status+submission-time
|
|
|
|
(with-temporary-database db
|
|
|
|
;; Populate the 'Builds', 'Derivations', 'Evaluations', and
|
|
|
|
;; 'Specifications' tables in a consistent way, as expected by the
|
|
|
|
;; 'db-get-builds' query.
|
|
|
|
(db-add-build db (make-dummy-build 1 #:drv "/foo.drv"
|
|
|
|
#:outputs `(("out" . "/foo"))))
|
|
|
|
(db-add-build db (make-dummy-build 2 #:drv "/bar.drv"
|
|
|
|
#:outputs `(("out" . "/bar"))))
|
|
|
|
(db-add-build db (make-dummy-build 3 #:drv "/baz.drv"
|
|
|
|
#:outputs `(("out" . "/baz"))))
|
|
|
|
(db-add-derivation db (make-dummy-derivation "/foo.drv" 1))
|
|
|
|
(db-add-derivation db (make-dummy-derivation "/bar.drv" 2))
|
|
|
|
(db-add-derivation db (make-dummy-derivation "/baz.drv" 3))
|
|
|
|
(db-add-evaluation db (make-dummy-eval))
|
|
|
|
(db-add-evaluation db (make-dummy-eval))
|
|
|
|
(db-add-evaluation db (make-dummy-eval))
|
|
|
|
(db-add-specification db example-spec)
|
|
|
|
|
|
|
|
(db-update-build-status! db "/bar.drv" (build-status started)
|
|
|
|
#:log-file "/bar.drv.log")
|
|
|
|
|
|
|
|
(let ((summarize (lambda (alist)
|
|
|
|
(list (assq-ref alist #:id)
|
|
|
|
(assq-ref alist #:derivation)))))
|
|
|
|
(vector (map summarize (db-get-builds db '((nr 3) (order build-id))))
|
|
|
|
(map summarize (db-get-builds db '()))
|
2018-07-02 17:25:31 +02:00
|
|
|
(map summarize (db-get-builds db '((jobset "guix"))))
|
2018-03-29 15:30:57 +02:00
|
|
|
(map summarize (db-get-builds db '((nr 1))))
|
|
|
|
(map summarize
|
|
|
|
(db-get-builds db '((order status+submission-time))))))))
|
|
|
|
|
2018-04-05 22:15:20 +02:00
|
|
|
(test-equal "db-get-pending-derivations"
|
|
|
|
'("/bar.drv" "/foo.drv")
|
|
|
|
(with-temporary-database db
|
|
|
|
;; Populate the 'Builds', 'Derivations', 'Evaluations', and
|
|
|
|
;; 'Specifications' tables. Here, two builds map to the same derivation
|
|
|
|
;; but the result of 'db-get-pending-derivations' must not contain any
|
|
|
|
;; duplicate.
|
|
|
|
(db-add-build db (make-dummy-build 1 #:drv "/foo.drv"
|
|
|
|
#:outputs `(("out" . "/foo"))))
|
|
|
|
(db-add-build db (make-dummy-build 2 #:drv "/bar.drv"
|
|
|
|
#:outputs `(("out" . "/bar"))))
|
|
|
|
(db-add-build db (make-dummy-build 3 #:drv "/foo.drv"
|
|
|
|
#:outputs `(("out" . "/foo"))))
|
|
|
|
(db-add-derivation db (make-dummy-derivation "/foo.drv" 1))
|
|
|
|
(db-add-derivation db (make-dummy-derivation "/bar.drv" 2))
|
|
|
|
(db-add-derivation db (make-dummy-derivation "/foo.drv" 3))
|
|
|
|
(db-add-evaluation db (make-dummy-eval))
|
|
|
|
(db-add-evaluation db (make-dummy-eval))
|
|
|
|
(db-add-evaluation db (make-dummy-eval))
|
|
|
|
(db-add-specification db example-spec)
|
|
|
|
|
|
|
|
(sort (db-get-pending-derivations db) string<?)))
|
|
|
|
|
2016-08-05 15:04:04 +02:00
|
|
|
(test-assert "db-close"
|
|
|
|
(db-close (%db)))
|
2016-07-27 13:43:53 +02:00
|
|
|
|
2016-08-05 15:04:04 +02:00
|
|
|
(delete-file database-name))
|
2018-01-07 10:19:56 +01:00
|
|
|
|
|
|
|
;;; Local Variables:
|
2018-01-23 17:37:56 +01:00
|
|
|
;;; eval: (put 'with-temporary-database 'scheme-indent-function 1)
|
2018-01-07 10:19:56 +01:00
|
|
|
;;; End:
|