2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/data-service.git synced 2023-12-14 03:23:03 +01:00
Commit graph

630 commits

Author SHA1 Message Date
Christopher Baines f13077d087 Remove non-existent export from the derivation module 2019-12-12 20:07:22 +00:00
Christopher Baines 7dc0238436 Add a page to show derivation outputs for a revision 2019-12-12 20:07:22 +00:00
Christopher Baines eecfdeb9e4 Add a function to select the outputs in a revision 2019-12-12 20:07:22 +00:00
Christopher Baines e92d4dbbcf Add a results count to the revision derivations page 2019-12-12 20:07:22 +00:00
Christopher Baines 05d295d6f0 Fix the all-results/limit handling in the revision derivatinos controller 2019-12-12 20:07:22 +00:00
Christopher Baines 544f122ff5 Fix related builds in select-derivations-in-revision
Rather than just counting builds where the derivation matches, select builds
based on the derivation outputs, which accounts for different but equivalent
derivations.
2019-12-12 20:07:22 +00:00
Christopher Baines c8b93cb0d0 Add new derivation_output_details_set_id column to the builds table
As this will hopefully provide a faster way of associating derivations with
builds.
2019-12-12 20:07:22 +00:00
Christopher Baines a965ba1f6a Start storing and identifying sets of derivation outputs
Derivations are effectively equivalent if they produce the same set of
outputs, which is possible because of the equivalence of fixed output
derivations. A fixed output derivation can be different, but equivalent,
because it produces the same fixed output.

To better allow tracking equivalent derivations, primarily to allow working
out what derivations might correspond to a build, store the sets of derivation
outputs, and which derivations they relate to.
2019-12-12 20:07:22 +00:00
Christopher Baines 0e97c55c0c Add some database indexes 2019-12-12 20:07:22 +00:00
Christopher Baines 00bfa5336e Add a page to show the derivations in a revision 2019-12-12 20:07:22 +00:00
Christopher Baines 9a99722643 Add a utility to generate a span for the status of a build 2019-12-12 20:07:22 +00:00
Christopher Baines 0cebb97340 Add a function to select derivations in a revision 2019-12-12 20:07:22 +00:00
Christopher Baines aa3af00e07 Fix and improve fetching outputs to fetch from build servers
The join on derivation_outputs was completely wrong, so fix that.

Also, rather than only selecting outputs where there's a corresponding build,
just eliminate outputs where there is an entry in the nars table, fetched from
the relevant build server.

Also just look at x86_64-linux derivations, because there's no filtering on
relevant derivations for a build server at the moment.
2019-12-12 20:07:22 +00:00
Christopher Baines 9e5cde80b3 Fix query in select-derivations-with-no-known-build
Rather than eliminating any derivation that has a known build, eliminate those
derivations, but also equivalent derivations as well.

For selecting the derivations in a revision, join on the equivalent
derivations, as all the equilalent derivations need checking as well, as it's
unknown which one Cuirass would have associated the build against.

Also, filter for x86_64-linux, to avoid checking for crossbuilt things. This
should be replaced by some way of describing what derivations a build server
has.
2019-12-12 20:07:22 +00:00
Christopher Baines 2e9844e599 Display the build server URL when fetching data 2019-12-12 20:07:22 +00:00
Christopher Baines 1e040010fc Fix the match statement in select-build-by-build-server-and… 2019-12-12 20:07:22 +00:00
Christopher Baines 29e897ce9c Improve variable names in fetch-narinfo-files 2019-12-12 20:07:22 +00:00
Christopher Baines eb2ba599d9 Populate the narinfo_fetch_records table 2019-12-12 20:07:22 +00:00
Christopher Baines 0291bda98a Deduplicate builds and add a unique index
Duplicate builds could creep in if the code to create them ran concurrently. I
didn't exclude them initially, as I was unsure if there should be such a
restriction, but at least for now, Cuirass builds map exactly to a single
derivation, so use the same restriction here.
2019-12-12 20:07:22 +00:00
Christopher Baines 8a5beb7f88 Change nar_urls size to bigint
As some nars can be bigger than the size of an int.
2019-12-12 20:07:22 +00:00
Christopher Baines 5168b38c2a Change nars size to bigint
As some nars can be bigger than the maximum size of an int.
2019-12-12 20:07:22 +00:00
Christopher Baines d2405f6c77 Add a table to record where narinfo files were fetched from
Otherwise it's hard to associated narinfo files to build servers.
2019-12-12 20:07:22 +00:00
Christopher Baines 299f9dc480 Add an index on the derivation_file_name field in the builds table
As this helps when finding builds relating to specific derivations.
2019-12-12 20:07:22 +00:00
Christopher Baines 716eb3c3b6 Move build-status helpers to the html-utils module 2019-12-12 20:07:22 +00:00
Christopher Baines e078b9611d Include nars on the output page 2019-12-12 20:07:22 +00:00
Christopher Baines 8e5f8e4170 Add a new page for the narinfos associated with an output 2019-12-12 20:07:22 +00:00
Christopher Baines 2d87bd6340 Add a page for each signing key 2019-12-12 20:07:22 +00:00
Christopher Baines 8ff27eaa14 Add a new html-utils module
To store code used for rendering HTML across multiple controllers.
2019-12-12 20:07:22 +00:00
Christopher Baines acf1735a0f Stop setting a long max-age for the viewing store items
As as new derivations are loaded, the contents of the page can change.
2019-12-12 20:07:22 +00:00
Christopher Baines a0ce016c02 Support querying builds by build servers 2019-12-12 20:07:22 +00:00
Christopher Baines f7069456b5 Support querying builds by status 2019-12-12 20:01:30 +00:00
Christopher Baines 04e98ed4da Handle multiple invalid values for a single form field 2019-12-12 20:01:26 +00:00
Christopher Baines 7cc5c02cdd Query build servers for narinfo files
As well as making it possible to only query build servers for builds relating
to a specific revision.
2019-11-30 10:59:16 +00:00
Christopher Baines b278065d2a Fix selecting pending builds 2019-11-30 10:58:40 +00:00
Christopher Baines 0aa78e9ef5 Don't create misleading scheduled build status entries
Cuirass provides a timestamp field in build responses, and sometimes this
means when the build was scheduled, but when the build is finished, it's the
stoptime.

So only use the timestamp when the build hasn't finished.
2019-11-30 10:55:16 +00:00
Christopher Baines 0fa747e404 Support handling jsonb in insert-missing-data-and-return-all-ids 2019-11-30 10:53:13 +00:00
Christopher Baines b6194e7b3d Begin to add support for importing narinfo files
This commit adds the tables, as well as code to support extracting data from
narinfo files.
2019-11-30 10:51:58 +00:00
Christopher Baines 20c75e1103 Fix some queries relating to the build_status table changes 2019-11-28 19:32:00 +00:00
Christopher Baines af1324855e Fix the JSON output for the derivation history page 2019-11-26 07:17:24 +00:00
Christopher Baines 32e1fee7ea Add the build-server-token-seed.scm file to Makefile.am 2019-11-26 07:17:09 +00:00
Christopher Baines 2a3ee20a99 Extract the builds page in to it's own set of modules
To allow for expanding it, without cluttering the root controller.
2019-11-24 20:42:37 +00:00
Christopher Baines 33ecb7448e Respond to any method with a 404, if the request is unrecognised 2019-11-24 20:18:08 +00:00
Christopher Baines 78695c46a1 Just update the package derivation entries that need updating
Previously, all the entries for the branch were deleted, but not, only the
entries for the branch, that are present in the revision that was loaded will
be deleted. This is more efficient, as it avoids deleting and recreating
entries in the table that shouldn't have changed.
2019-11-24 20:18:08 +00:00
Christopher Baines 5663235048 Rework the builds and build_status tables as well as related code
Allow for build status information to be submitted by POST request. This
required some changes to the builds and build_status tables, as for example,
the Cuirass build id may not be available, and the derivation may not be know
yet, so just record the derivation file name.
2019-11-24 20:18:08 +00:00
Christopher Baines 0ffd8caeeb Add a script to help manage build servers
This computes and displays the tokens needed to send build events to the Guix
Data Service.
2019-11-23 11:50:51 +00:00
Christopher Baines c3b17c0cb0 Add guile-gcrypt to the guix-dev.scm file 2019-11-23 11:50:33 +00:00
Christopher Baines 79d85f5847 Add a new module to handle build server token seeds
The intent with these tokens is for them to be provided when build events are
submitted, preventing unauthorised submission of build events.

It should be possible to make the database public, so it shouldn't contain any
secret information. To have secret tokens that aren't in the database, store a
token seed, and combine this with some secret data to compute the secret.
2019-11-23 11:33:16 +00:00
Christopher Baines 9a3dbb1d5b Add new table to store token seeds for build servers 2019-11-23 11:30:36 +00:00
Christopher Baines 6fad559b32 Improve select-build-servers to convert strings to appropriate types 2019-11-23 11:23:16 +00:00
Christopher Baines 53652db9ca Add a page to render the channel news entries for a revision 2019-11-22 06:28:10 +00:00