guix-cuirass/src/schema.sql

80 lines
2.6 KiB
MySQL
Raw Normal View History

BEGIN TRANSACTION;
CREATE TABLE Specifications (
Add support for multiple inputs. * Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql. * bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that was used afterwards as %GUIX-PACKAGE-PATH. * bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path, spec-package-path, format-checkouts): New procedures. (%not-colon): Remove variable. (main): Take the load path, package path and PROC from the checkouts that result from the inputs. Format the checkouts before sending them to the procedure. Remove the LOAD-PATH argument. * doc/cuirass.texi (Overview, Database schema): Document the changes. * examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm, hello-subset.scm, random.scm}: Adapt to the new specification format. * examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT. (package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle the new format of its return value. * examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT. Rename the checkout from 'random (which is a specification) to 'cuirass (which is a checkout resulting from an input). * src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC to INPUT. Return a checkout object instead of returning two values. (evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object instead of getting it from "evaluate". (compile?, fetch-inputs, compile-checkouts): New procedures. (process-specs): Fetch all inputs instead of only fetching one repository. The result of that fetching operation is a list of CHECKOUTS whose COMMITS are used as a STAMP. (%guix-package-path, set-guix-package-path): Remove them. * src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures. (db-add-specification, db-get-specifications): Adapt to the new specification format. Add/get all inputs as well. (db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space separated commit hashes. (db-get-builds): Rename REPO_NAME to NAME. (db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP. (db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return value. (db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS. * src/cuirass/utils.scm (%non-blocking): Export it. * src/schema.sql (Inputs): New table that refers to the Specifications table. (Specifications): Move input related fields to the Inputs table. Rename REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE. Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to the Inputs table. (Stamps): Rename REPO_NAME to NAME. (Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS. (Specifications_index): Replace with Inputs_index. * src/sql/upgrade-1.sql: New file. * tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to the new specifications format. Rename REVISION to COMMITS. * tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
name TEXT NOT NULL PRIMARY KEY,
load_path_inputs TEXT NOT NULL, -- list of input names whose load path will be in Guile's %load-path
package_path_inputs TEXT NOT NULL, -- list of input names whose load paths will be in GUIX_PACKAGE_PATH
proc_input TEXT NOT NULL, -- name of the input containing the proc that does the evaluation
proc_file TEXT NOT NULL, -- file containing the procedure that does the evaluation, relative to proc_input
proc TEXT NOT NULL, -- defined in proc_file
proc_args TEXT NOT NULL -- passed to proc
);
CREATE TABLE Inputs (
specification TEXT NOT NULL,
name TEXT NOT NULL,
url TEXT NOT NULL,
load_path TEXT NOT NULL,
-- The following columns are optional.
branch TEXT,
tag TEXT,
revision TEXT,
Add support for multiple inputs. * Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql. * bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that was used afterwards as %GUIX-PACKAGE-PATH. * bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path, spec-package-path, format-checkouts): New procedures. (%not-colon): Remove variable. (main): Take the load path, package path and PROC from the checkouts that result from the inputs. Format the checkouts before sending them to the procedure. Remove the LOAD-PATH argument. * doc/cuirass.texi (Overview, Database schema): Document the changes. * examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm, hello-subset.scm, random.scm}: Adapt to the new specification format. * examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT. (package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle the new format of its return value. * examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT. Rename the checkout from 'random (which is a specification) to 'cuirass (which is a checkout resulting from an input). * src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC to INPUT. Return a checkout object instead of returning two values. (evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object instead of getting it from "evaluate". (compile?, fetch-inputs, compile-checkouts): New procedures. (process-specs): Fetch all inputs instead of only fetching one repository. The result of that fetching operation is a list of CHECKOUTS whose COMMITS are used as a STAMP. (%guix-package-path, set-guix-package-path): Remove them. * src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures. (db-add-specification, db-get-specifications): Adapt to the new specification format. Add/get all inputs as well. (db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space separated commit hashes. (db-get-builds): Rename REPO_NAME to NAME. (db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP. (db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return value. (db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS. * src/cuirass/utils.scm (%non-blocking): Export it. * src/schema.sql (Inputs): New table that refers to the Specifications table. (Specifications): Move input related fields to the Inputs table. Rename REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE. Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to the Inputs table. (Stamps): Rename REPO_NAME to NAME. (Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS. (Specifications_index): Replace with Inputs_index. * src/sql/upgrade-1.sql: New file. * tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to the new specifications format. Rename REVISION to COMMITS. * tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
no_compile_p INTEGER,
PRIMARY KEY (specification, name),
FOREIGN KEY (specification) REFERENCES Specifications (name)
);
database: Add a Checkouts table. It is used to know when a new evaluation must be triggered and to display input changes. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-3.sql'. * bin/cuirass.in (main): Call DB-SET-EVALUATION-DONE at startup to clear 'in-progress' evaluations. * bin/evaluate.in (input-checkout, format-checkouts): Rename '#:name' to '#:input'. * doc/cuirass.texi (Stamps): Remove section. (Checkouts): New section. * src/cuirass/base.scm (fetch-input, fetch-inputs, compile-checkouts): Rename '#:name' to '#:input'. (evaluate): Remove the COMMITS argument. Add an EVAL-ID argument. Don't call DB-ADD-EVALUATION because it was called sooner. Remove the EVAL-ID argument to AUGMENT-JOB because it's a closure. (build-packages): Add an EVAL-ID argument. Call DB-SET-EVALUATION-DONE once all the derivations are registered. (process-specs): Replace the stamping mechanism by the primary key constraint of the Checkouts table: call "evaluate" only when DB-ADD-EVALUATION is true, which means that at least one checkout was added. Change the EVALUATE and BUILD-PACKAGES arguments accordingly. * src/cuirass/database.scm (db-add-stamp, db-get-stamp): Remove procedures. (db-set-evaluations-done, db-set-evaluation-done): New exported procedure. (db-add-checkout): New procedure that returns #f if a checkout with the same revision already exists. (db-add-evaluation): Replace the EVAL argument with a SPEC-NAME and a CHECKOUTS arguments. Insert the evaluation only if at least one checkout was inserted. Return #f otherwise. (db-get-checkouts): New procedure. (db-get-evaluations, db-get-evaluations-build-summary): Handle the 'in_progress' column, remove the 'commits' column. Return the result of DB-GET-CHECKOUTS as part of the evaluation. * src/cuirass/templates.scm (input-changes, evaluation-badges): New procedures. (evaluation-info-table): Rename "Commits" to "Input changes". Use INPUT-CHANGES to display the input changes that triggered the evaluation. Use EVALUATION-BADGES to display a message indicating that the evaluation is in progress. * src/schema.sql (Stamps): Remove table. (Checkouts): New table. (Evaluations): Remove the 'commits' column. Add an 'in_progress' column. * src/sql/upgrade-3.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-eval): Remove procedure. (make-dummy-checkouts): New procedure. ("sqlite-exec"): Remove the 'commits' column. Add the 'in_progress' column. ("db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Update the arguments of DB-ADD-EVALUATION accordingly. * tests/http.scm (hash-table=?): Add support for lists of hash tables. (evaluations-query-result): Replace '#:commits' with '#:checkouts'. Return a list instead of returning one element, for symmetry. ("fill-db"): Add a new input so that the second checkout can refer to it. Replace EVALUATION1 and EVALUATION2 with CHECKOUTS1 and CHECKOUTS2. Update the arguments of DB-ADD-EVALUATION accordingly. ("/api/queue?nr=100"): Take the CAR of the EVALUATIONS-QUERY-RESULT list to make it symmetrical with the other argument of HASH-TABLE=?.
2018-08-11 20:30:11 +02:00
CREATE TABLE Checkouts (
specification TEXT NOT NULL,
revision TEXT NOT NULL,
evaluation INTEGER NOT NULL,
input TEXT NOT NULL,
directory TEXT NOT NULL,
PRIMARY KEY (specification, revision),
FOREIGN KEY (evaluation) REFERENCES Evaluations (id),
FOREIGN KEY (specification) REFERENCES Specifications (name),
FOREIGN KEY (input) REFERENCES Inputs (name)
2016-07-26 00:36:12 +02:00
);
CREATE TABLE Evaluations (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
specification TEXT NOT NULL,
database: Add a Checkouts table. It is used to know when a new evaluation must be triggered and to display input changes. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-3.sql'. * bin/cuirass.in (main): Call DB-SET-EVALUATION-DONE at startup to clear 'in-progress' evaluations. * bin/evaluate.in (input-checkout, format-checkouts): Rename '#:name' to '#:input'. * doc/cuirass.texi (Stamps): Remove section. (Checkouts): New section. * src/cuirass/base.scm (fetch-input, fetch-inputs, compile-checkouts): Rename '#:name' to '#:input'. (evaluate): Remove the COMMITS argument. Add an EVAL-ID argument. Don't call DB-ADD-EVALUATION because it was called sooner. Remove the EVAL-ID argument to AUGMENT-JOB because it's a closure. (build-packages): Add an EVAL-ID argument. Call DB-SET-EVALUATION-DONE once all the derivations are registered. (process-specs): Replace the stamping mechanism by the primary key constraint of the Checkouts table: call "evaluate" only when DB-ADD-EVALUATION is true, which means that at least one checkout was added. Change the EVALUATE and BUILD-PACKAGES arguments accordingly. * src/cuirass/database.scm (db-add-stamp, db-get-stamp): Remove procedures. (db-set-evaluations-done, db-set-evaluation-done): New exported procedure. (db-add-checkout): New procedure that returns #f if a checkout with the same revision already exists. (db-add-evaluation): Replace the EVAL argument with a SPEC-NAME and a CHECKOUTS arguments. Insert the evaluation only if at least one checkout was inserted. Return #f otherwise. (db-get-checkouts): New procedure. (db-get-evaluations, db-get-evaluations-build-summary): Handle the 'in_progress' column, remove the 'commits' column. Return the result of DB-GET-CHECKOUTS as part of the evaluation. * src/cuirass/templates.scm (input-changes, evaluation-badges): New procedures. (evaluation-info-table): Rename "Commits" to "Input changes". Use INPUT-CHANGES to display the input changes that triggered the evaluation. Use EVALUATION-BADGES to display a message indicating that the evaluation is in progress. * src/schema.sql (Stamps): Remove table. (Checkouts): New table. (Evaluations): Remove the 'commits' column. Add an 'in_progress' column. * src/sql/upgrade-3.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-eval): Remove procedure. (make-dummy-checkouts): New procedure. ("sqlite-exec"): Remove the 'commits' column. Add the 'in_progress' column. ("db-update-build-status!", "db-get-builds", "db-get-pending-derivations"): Update the arguments of DB-ADD-EVALUATION accordingly. * tests/http.scm (hash-table=?): Add support for lists of hash tables. (evaluations-query-result): Replace '#:commits' with '#:checkouts'. Return a list instead of returning one element, for symmetry. ("fill-db"): Add a new input so that the second checkout can refer to it. Replace EVALUATION1 and EVALUATION2 with CHECKOUTS1 and CHECKOUTS2. Update the arguments of DB-ADD-EVALUATION accordingly. ("/api/queue?nr=100"): Take the CAR of the EVALUATIONS-QUERY-RESULT list to make it symmetrical with the other argument of HASH-TABLE=?.
2018-08-11 20:30:11 +02:00
in_progress INTEGER NOT NULL,
Add support for multiple inputs. * Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql. * bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that was used afterwards as %GUIX-PACKAGE-PATH. * bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path, spec-package-path, format-checkouts): New procedures. (%not-colon): Remove variable. (main): Take the load path, package path and PROC from the checkouts that result from the inputs. Format the checkouts before sending them to the procedure. Remove the LOAD-PATH argument. * doc/cuirass.texi (Overview, Database schema): Document the changes. * examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm, hello-subset.scm, random.scm}: Adapt to the new specification format. * examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT. (package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle the new format of its return value. * examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT. Rename the checkout from 'random (which is a specification) to 'cuirass (which is a checkout resulting from an input). * src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC to INPUT. Return a checkout object instead of returning two values. (evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object instead of getting it from "evaluate". (compile?, fetch-inputs, compile-checkouts): New procedures. (process-specs): Fetch all inputs instead of only fetching one repository. The result of that fetching operation is a list of CHECKOUTS whose COMMITS are used as a STAMP. (%guix-package-path, set-guix-package-path): Remove them. * src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures. (db-add-specification, db-get-specifications): Adapt to the new specification format. Add/get all inputs as well. (db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space separated commit hashes. (db-get-builds): Rename REPO_NAME to NAME. (db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP. (db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return value. (db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS. * src/cuirass/utils.scm (%non-blocking): Export it. * src/schema.sql (Inputs): New table that refers to the Specifications table. (Specifications): Move input related fields to the Inputs table. Rename REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE. Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to the Inputs table. (Stamps): Rename REPO_NAME to NAME. (Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS. (Specifications_index): Replace with Inputs_index. * src/sql/upgrade-1.sql: New file. * tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to the new specifications format. Rename REVISION to COMMITS. * tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
FOREIGN KEY (specification) REFERENCES Specifications (name)
2016-07-16 18:16:39 +02:00
);
CREATE TABLE Outputs (
database: Merge Derivations into Builds table. Fixes <https://bugs.gnu.org/32190>. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-2.sql'. * doc/cuirass.texi (Derivations): Remove section. (Builds): Update accordingly. Add columns from the Derivations table. (Outputs): Replace 'id' with 'derivation'. * src/cuirass/base.scm (evaluate): Don't add jobs to the Derivations table. (build-packages): Add columns that were in the Derivations table before. Only build the derivations that were successfully registered, that is, those that didn't exist in the Builds table. Give a derivation instead of a build id to DB-GET-BUILD. Compute the number of failed jobs based on the derivations that were added to the table, instead of the jobs. * src/cuirass/database.scm (db-add-derivation, db-get-derivation): Remove exported procedures. (db-add-build): Catch SQLITE_CONSTRAINT_PRIMARYKEY error, which means that two jobs produced the same derivation, and return #f in that case. Add columns that were in the Derivations table before. Use 'derivation' as primary key for the Outputs table. (db-get-outputs): Use 'derivation' as identifier, instead of 'build-id'. (filters->order): Replace 'id' with 'rowid'. (db-get-builds): Add a 'derivation' filter. Replace 'id' with 'rowid'. Remove the 'INNER JOIN Derivations'. Replace Derivations with Builds. Return 'derivation' in first position to make it clear that it's the primary key. Pass DERIVATION instead of ID to DB-GET-OUTPUTS. (db-get-build): Allow to take a derivation as argument. Use NUMBER? to differentiate between derivations and ids. (db-get-pending-derivations): Remove the 'SELECT DISTINCT' clause now that derivations are unique. Remove the 'INNER JOIN Builds'. (db-get-evaluations-build-summary, db-get-builds-min, db-get-builds-max): Replace 'id' with 'rowid'. * src/schema.sql (Derivations): Remove table. (Outputs): Replace Builds.id with Builds.derivation. (Builds): Use 'derivation' as primary key. Remove the 'id' column. Add 'job_name', 'system', 'nix_name' columns that were in the Derivations table before. (Builds_Derivations_index): Rename to Builds_index. Update accordingly. (Derivations_index): Remove index. * src/sql/upgrade-2.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-job, make-dummy-derivation): Remove procedures. (make-dummy-build): Add columns that were in MAKE-DUMMY-DERIVATION. Get the DRV parameter to be mandatory because it's a primary key. (%id): Remove parameter. ("db-add-derivation", "db-get-derivation"): Remove tests. ("db-add-build"): Expect #f, because it adds twice the same derivation. Pass the derivation argument to MAKE-DUMMY-BUILD. ("db-update-build-status!"): Rename 'id' to 'derivation'. Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION call. ("db-get-builds", "db-get-pending-derivations"): Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION calls. * tests/http.scm ("fill-db"): Remove DERIVATION1 and DERIVATION2, and put their content in BUILD1 and BUILD2. Remove the DB-ADD-DERIVATION calls.
2018-08-01 00:03:12 +02:00
derivation TEXT NOT NULL,
name TEXT NOT NULL,
path TEXT NOT NULL PRIMARY KEY,
database: Merge Derivations into Builds table. Fixes <https://bugs.gnu.org/32190>. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-2.sql'. * doc/cuirass.texi (Derivations): Remove section. (Builds): Update accordingly. Add columns from the Derivations table. (Outputs): Replace 'id' with 'derivation'. * src/cuirass/base.scm (evaluate): Don't add jobs to the Derivations table. (build-packages): Add columns that were in the Derivations table before. Only build the derivations that were successfully registered, that is, those that didn't exist in the Builds table. Give a derivation instead of a build id to DB-GET-BUILD. Compute the number of failed jobs based on the derivations that were added to the table, instead of the jobs. * src/cuirass/database.scm (db-add-derivation, db-get-derivation): Remove exported procedures. (db-add-build): Catch SQLITE_CONSTRAINT_PRIMARYKEY error, which means that two jobs produced the same derivation, and return #f in that case. Add columns that were in the Derivations table before. Use 'derivation' as primary key for the Outputs table. (db-get-outputs): Use 'derivation' as identifier, instead of 'build-id'. (filters->order): Replace 'id' with 'rowid'. (db-get-builds): Add a 'derivation' filter. Replace 'id' with 'rowid'. Remove the 'INNER JOIN Derivations'. Replace Derivations with Builds. Return 'derivation' in first position to make it clear that it's the primary key. Pass DERIVATION instead of ID to DB-GET-OUTPUTS. (db-get-build): Allow to take a derivation as argument. Use NUMBER? to differentiate between derivations and ids. (db-get-pending-derivations): Remove the 'SELECT DISTINCT' clause now that derivations are unique. Remove the 'INNER JOIN Builds'. (db-get-evaluations-build-summary, db-get-builds-min, db-get-builds-max): Replace 'id' with 'rowid'. * src/schema.sql (Derivations): Remove table. (Outputs): Replace Builds.id with Builds.derivation. (Builds): Use 'derivation' as primary key. Remove the 'id' column. Add 'job_name', 'system', 'nix_name' columns that were in the Derivations table before. (Builds_Derivations_index): Rename to Builds_index. Update accordingly. (Derivations_index): Remove index. * src/sql/upgrade-2.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-job, make-dummy-derivation): Remove procedures. (make-dummy-build): Add columns that were in MAKE-DUMMY-DERIVATION. Get the DRV parameter to be mandatory because it's a primary key. (%id): Remove parameter. ("db-add-derivation", "db-get-derivation"): Remove tests. ("db-add-build"): Expect #f, because it adds twice the same derivation. Pass the derivation argument to MAKE-DUMMY-BUILD. ("db-update-build-status!"): Rename 'id' to 'derivation'. Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION call. ("db-get-builds", "db-get-pending-derivations"): Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION calls. * tests/http.scm ("fill-db"): Remove DERIVATION1 and DERIVATION2, and put their content in BUILD1 and BUILD2. Remove the DB-ADD-DERIVATION calls.
2018-08-01 00:03:12 +02:00
FOREIGN KEY (derivation) REFERENCES Builds (derivation)
);
CREATE TABLE Builds (
database: Merge Derivations into Builds table. Fixes <https://bugs.gnu.org/32190>. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-2.sql'. * doc/cuirass.texi (Derivations): Remove section. (Builds): Update accordingly. Add columns from the Derivations table. (Outputs): Replace 'id' with 'derivation'. * src/cuirass/base.scm (evaluate): Don't add jobs to the Derivations table. (build-packages): Add columns that were in the Derivations table before. Only build the derivations that were successfully registered, that is, those that didn't exist in the Builds table. Give a derivation instead of a build id to DB-GET-BUILD. Compute the number of failed jobs based on the derivations that were added to the table, instead of the jobs. * src/cuirass/database.scm (db-add-derivation, db-get-derivation): Remove exported procedures. (db-add-build): Catch SQLITE_CONSTRAINT_PRIMARYKEY error, which means that two jobs produced the same derivation, and return #f in that case. Add columns that were in the Derivations table before. Use 'derivation' as primary key for the Outputs table. (db-get-outputs): Use 'derivation' as identifier, instead of 'build-id'. (filters->order): Replace 'id' with 'rowid'. (db-get-builds): Add a 'derivation' filter. Replace 'id' with 'rowid'. Remove the 'INNER JOIN Derivations'. Replace Derivations with Builds. Return 'derivation' in first position to make it clear that it's the primary key. Pass DERIVATION instead of ID to DB-GET-OUTPUTS. (db-get-build): Allow to take a derivation as argument. Use NUMBER? to differentiate between derivations and ids. (db-get-pending-derivations): Remove the 'SELECT DISTINCT' clause now that derivations are unique. Remove the 'INNER JOIN Builds'. (db-get-evaluations-build-summary, db-get-builds-min, db-get-builds-max): Replace 'id' with 'rowid'. * src/schema.sql (Derivations): Remove table. (Outputs): Replace Builds.id with Builds.derivation. (Builds): Use 'derivation' as primary key. Remove the 'id' column. Add 'job_name', 'system', 'nix_name' columns that were in the Derivations table before. (Builds_Derivations_index): Rename to Builds_index. Update accordingly. (Derivations_index): Remove index. * src/sql/upgrade-2.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-job, make-dummy-derivation): Remove procedures. (make-dummy-build): Add columns that were in MAKE-DUMMY-DERIVATION. Get the DRV parameter to be mandatory because it's a primary key. (%id): Remove parameter. ("db-add-derivation", "db-get-derivation"): Remove tests. ("db-add-build"): Expect #f, because it adds twice the same derivation. Pass the derivation argument to MAKE-DUMMY-BUILD. ("db-update-build-status!"): Rename 'id' to 'derivation'. Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION call. ("db-get-builds", "db-get-pending-derivations"): Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION calls. * tests/http.scm ("fill-db"): Remove DERIVATION1 and DERIVATION2, and put their content in BUILD1 and BUILD2. Remove the DB-ADD-DERIVATION calls.
2018-08-01 00:03:12 +02:00
derivation TEXT NOT NULL PRIMARY KEY,
evaluation INTEGER NOT NULL,
database: Merge Derivations into Builds table. Fixes <https://bugs.gnu.org/32190>. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-2.sql'. * doc/cuirass.texi (Derivations): Remove section. (Builds): Update accordingly. Add columns from the Derivations table. (Outputs): Replace 'id' with 'derivation'. * src/cuirass/base.scm (evaluate): Don't add jobs to the Derivations table. (build-packages): Add columns that were in the Derivations table before. Only build the derivations that were successfully registered, that is, those that didn't exist in the Builds table. Give a derivation instead of a build id to DB-GET-BUILD. Compute the number of failed jobs based on the derivations that were added to the table, instead of the jobs. * src/cuirass/database.scm (db-add-derivation, db-get-derivation): Remove exported procedures. (db-add-build): Catch SQLITE_CONSTRAINT_PRIMARYKEY error, which means that two jobs produced the same derivation, and return #f in that case. Add columns that were in the Derivations table before. Use 'derivation' as primary key for the Outputs table. (db-get-outputs): Use 'derivation' as identifier, instead of 'build-id'. (filters->order): Replace 'id' with 'rowid'. (db-get-builds): Add a 'derivation' filter. Replace 'id' with 'rowid'. Remove the 'INNER JOIN Derivations'. Replace Derivations with Builds. Return 'derivation' in first position to make it clear that it's the primary key. Pass DERIVATION instead of ID to DB-GET-OUTPUTS. (db-get-build): Allow to take a derivation as argument. Use NUMBER? to differentiate between derivations and ids. (db-get-pending-derivations): Remove the 'SELECT DISTINCT' clause now that derivations are unique. Remove the 'INNER JOIN Builds'. (db-get-evaluations-build-summary, db-get-builds-min, db-get-builds-max): Replace 'id' with 'rowid'. * src/schema.sql (Derivations): Remove table. (Outputs): Replace Builds.id with Builds.derivation. (Builds): Use 'derivation' as primary key. Remove the 'id' column. Add 'job_name', 'system', 'nix_name' columns that were in the Derivations table before. (Builds_Derivations_index): Rename to Builds_index. Update accordingly. (Derivations_index): Remove index. * src/sql/upgrade-2.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-job, make-dummy-derivation): Remove procedures. (make-dummy-build): Add columns that were in MAKE-DUMMY-DERIVATION. Get the DRV parameter to be mandatory because it's a primary key. (%id): Remove parameter. ("db-add-derivation", "db-get-derivation"): Remove tests. ("db-add-build"): Expect #f, because it adds twice the same derivation. Pass the derivation argument to MAKE-DUMMY-BUILD. ("db-update-build-status!"): Rename 'id' to 'derivation'. Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION call. ("db-get-builds", "db-get-pending-derivations"): Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION calls. * tests/http.scm ("fill-db"): Remove DERIVATION1 and DERIVATION2, and put their content in BUILD1 and BUILD2. Remove the DB-ADD-DERIVATION calls.
2018-08-01 00:03:12 +02:00
job_name TEXT NOT NULL,
system TEXT NOT NULL,
nix_name TEXT NOT NULL,
log TEXT NOT NULL,
status INTEGER NOT NULL,
timestamp INTEGER NOT NULL,
starttime INTEGER NOT NULL,
stoptime INTEGER NOT NULL,
FOREIGN KEY (evaluation) REFERENCES Evaluations (id)
2016-07-16 18:16:39 +02:00
);
CREATE TABLE Events (
id INTEGER PRIMARY KEY,
type TEXT NOT NULL,
timestamp INTEGER NOT NULL,
event_json TEXT NOT NULL
);
-- Create indexes to speed up common queries, in particular those
-- corresponding to /api/latestbuilds and /api/queue HTTP requests.
database: Merge Derivations into Builds table. Fixes <https://bugs.gnu.org/32190>. * Makefile.am (dist_sql_DATA): Add 'src/sql/upgrade-2.sql'. * doc/cuirass.texi (Derivations): Remove section. (Builds): Update accordingly. Add columns from the Derivations table. (Outputs): Replace 'id' with 'derivation'. * src/cuirass/base.scm (evaluate): Don't add jobs to the Derivations table. (build-packages): Add columns that were in the Derivations table before. Only build the derivations that were successfully registered, that is, those that didn't exist in the Builds table. Give a derivation instead of a build id to DB-GET-BUILD. Compute the number of failed jobs based on the derivations that were added to the table, instead of the jobs. * src/cuirass/database.scm (db-add-derivation, db-get-derivation): Remove exported procedures. (db-add-build): Catch SQLITE_CONSTRAINT_PRIMARYKEY error, which means that two jobs produced the same derivation, and return #f in that case. Add columns that were in the Derivations table before. Use 'derivation' as primary key for the Outputs table. (db-get-outputs): Use 'derivation' as identifier, instead of 'build-id'. (filters->order): Replace 'id' with 'rowid'. (db-get-builds): Add a 'derivation' filter. Replace 'id' with 'rowid'. Remove the 'INNER JOIN Derivations'. Replace Derivations with Builds. Return 'derivation' in first position to make it clear that it's the primary key. Pass DERIVATION instead of ID to DB-GET-OUTPUTS. (db-get-build): Allow to take a derivation as argument. Use NUMBER? to differentiate between derivations and ids. (db-get-pending-derivations): Remove the 'SELECT DISTINCT' clause now that derivations are unique. Remove the 'INNER JOIN Builds'. (db-get-evaluations-build-summary, db-get-builds-min, db-get-builds-max): Replace 'id' with 'rowid'. * src/schema.sql (Derivations): Remove table. (Outputs): Replace Builds.id with Builds.derivation. (Builds): Use 'derivation' as primary key. Remove the 'id' column. Add 'job_name', 'system', 'nix_name' columns that were in the Derivations table before. (Builds_Derivations_index): Rename to Builds_index. Update accordingly. (Derivations_index): Remove index. * src/sql/upgrade-2.sql: New file with SQL queries to upgrade the database. * tests/database.scm (make-dummy-job, make-dummy-derivation): Remove procedures. (make-dummy-build): Add columns that were in MAKE-DUMMY-DERIVATION. Get the DRV parameter to be mandatory because it's a primary key. (%id): Remove parameter. ("db-add-derivation", "db-get-derivation"): Remove tests. ("db-add-build"): Expect #f, because it adds twice the same derivation. Pass the derivation argument to MAKE-DUMMY-BUILD. ("db-update-build-status!"): Rename 'id' to 'derivation'. Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION call. ("db-get-builds", "db-get-pending-derivations"): Pass the derivation argument to MAKE-DUMMY-BUILD. Remove the DB-ADD-DERIVATION calls. * tests/http.scm ("fill-db"): Remove DERIVATION1 and DERIVATION2, and put their content in BUILD1 and BUILD2. Remove the DB-ADD-DERIVATION calls.
2018-08-01 00:03:12 +02:00
CREATE INDEX Builds_index ON Builds(job_name, system, status ASC, timestamp ASC, derivation, evaluation, stoptime DESC);
Add support for multiple inputs. * Makefile.am (dist_sql_DATA): Add src/sql/upgrade-1.sql. * bin/cuirass.in (show-help, %options, main): Remove the LOAD-PATH option that was used afterwards as %GUIX-PACKAGE-PATH. * bin/evaluate.in (absolutize, input-checkout, spec-source, spec-load-path, spec-package-path, format-checkouts): New procedures. (%not-colon): Remove variable. (main): Take the load path, package path and PROC from the checkouts that result from the inputs. Format the checkouts before sending them to the procedure. Remove the LOAD-PATH argument. * doc/cuirass.texi (Overview, Database schema): Document the changes. * examples/{guix-jobs.scm, hello-git.scm, hello-singleton.scm, hello-subset.scm, random.scm}: Adapt to the new specification format. * examples/guix-track-git.scm (package->spec): Rename to PACKAGE->INPUT. (package->git-tracked): Replace FETCH-REPOSITORY with FETCH-INPUT and handle the new format of its return value. * examples/random-jobs.scm (make-random-jobs): Rename RANDOM to CHECKOUT. Rename the checkout from 'random (which is a specification) to 'cuirass (which is a checkout resulting from an input). * src/cuirass/base.scm (fetch-repository): Rename to fetch-input. Rename SPEC to INPUT. Return a checkout object instead of returning two values. (evaluate): Take a list of CHECKOUTS and COMMITS as arguments, instead of SOURCE. Remove TOKENIZE and LOAD-PATH. Pass the CHECKOUTS instead of the SOURCE to "evaluate". Remove %GUIX-PACKAGE-PATH. Build the EVAL object instead of getting it from "evaluate". (compile?, fetch-inputs, compile-checkouts): New procedures. (process-specs): Fetch all inputs instead of only fetching one repository. The result of that fetching operation is a list of CHECKOUTS whose COMMITS are used as a STAMP. (%guix-package-path, set-guix-package-path): Remove them. * src/cuirass/database.scm (db-add-input, db-get-inputs): New procedures. (db-add-specification, db-get-specifications): Adapt to the new specification format. Add/get all inputs as well. (db-add-evaluation): Rename REVISION to COMMITS. Store COMMITS as space separated commit hashes. (db-get-builds): Rename REPO_NAME to NAME. (db-get-stamp): Rename COMMIT to STAMP. Return #f when there is no STAMP. (db-add-stamp): Rename COMMIT to STAMP. Deal with DB-GET-STAMP's new return value. (db-get-evaluations): Rename REVISION to COMMITS. Tokenize COMMITS. * src/cuirass/utils.scm (%non-blocking): Export it. * src/schema.sql (Inputs): New table that refers to the Specifications table. (Specifications): Move input related fields to the Inputs table. Rename REPO_NAME to NAME. Rename ARGUMENTS to PROC_ARGS. Rename FILE to PROC_FILE. Add LOAD_PATH_INPUTS, PACKAGE_PATH_INPUTS and PROC_INPUT fields that refer to the Inputs table. (Stamps): Rename REPO_NAME to NAME. (Evaluations): Rename REPO_NAME to NAME. Rename REVISION to COMMITS. (Specifications_index): Replace with Inputs_index. * src/sql/upgrade-1.sql: New file. * tests/database.scm (example-spec, make-dummy-eval, sqlite-exec): Adapt to the new specifications format. Rename REVISION to COMMITS. * tests/http.scm (evaluations-query-result, fill-db): Idem.
2018-06-26 11:18:23 +02:00
CREATE INDEX Inputs_index ON Inputs(specification, name, branch);
COMMIT;