2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/data-service.git synced 2023-12-14 03:23:03 +01:00
Commit graph

140 commits

Author SHA1 Message Date
Christopher Baines
e3d8a23716 Add a function to fetch a derivation by file name hash 2019-12-26 09:34:35 +00:00
Christopher Baines
b504094262 Use the returned file name in select-derivation-by-file-name
They should be the same, but it seems more sensible to return the value from
the database.
2019-12-26 09:34:08 +00:00
Christopher Baines
a92d4d0cf2 Add a function serialize a derivation
This effectively duplicates the behaviour in Guix for serializing derivations,
but this uses the database representation in the Guix Data Service, rather
than the records Guix uses.
2019-12-26 09:04:00 +00:00
Christopher Baines
21e298f68c Ensure output ordering for derivations is correct
This affects the formatted derivation output.
2019-12-26 09:03:38 +00:00
Christopher Baines
dd94f59acf Fix the representation of the fixed output derivation hashes
Previously, they were nix-base32-string encoded, but the representation in the
derivations is base16, so it doesn't make sense to use a different
representation in the database.

Therefore, add some code that runs before the start of each job to convert the
data in the database. It was easier to do this in Guile with the existing
support for working with these bytevector representations. After some
migration period, the code for converting the old hashes can be removed.
2019-12-26 08:53:51 +00:00
Christopher Baines
4eb5a3417c Group derivation input outputs together by derivation
Both in terms of the code fetching the data from the database, as well as the
formatted and detail outputs. This corrects an error in the formatted output
for derivations where inputs would be duplicated.
2019-12-26 08:52:04 +00:00
Christopher Baines
801ebdfa9e Fix the successful builds outputs query where there are no revisions
As it was erroring before.
2019-12-26 08:35:33 +00:00
Christopher Baines
e359cfea2b Fix the derivations list on the builds page 2019-12-23 22:36:53 +00:00
Christopher Baines
5c643e09e5 Fix the unknown build counts on the builds pages 2019-12-23 21:30:36 +00:00
Christopher Baines
d2e0fc53c1 Fix an error when updating the package derivation versions table 2019-12-23 19:27:42 +00:00
Christopher Baines
e45db1cd30 Include builds on the derivation history page 2019-12-22 21:42:49 +00:00
Christopher Baines
2e0c1b4fcb Add a function to rebuild the package_derivations_by_guix_revision
At the moment, some data is missing, but this code will fix that.
2019-12-22 17:37:18 +00:00
Christopher Baines
69d95b133f Extract out some functions from update-package-derivations-table
To allow these to be used independently.
2019-12-22 16:20:08 +00:00
Christopher Baines
639f24bb22 Extract management of package_derivations_by_guix_revision_range
... entries in to a separate module, to split the code up a little further.
2019-12-22 16:09:32 +00:00
Christopher Baines
14b7993636 Allow filtering the revision builds page by system 2019-12-22 14:27:59 +00:00
Christopher Baines
e4a7f221c9 Filter builds on the revision builds page by revision 2019-12-22 14:27:49 +00:00
Christopher Baines
89e34f3df8 Ignore cross built derivations for the build stats 2019-12-22 14:27:49 +00:00
Christopher Baines
4af643136e Allow filtering by system and target on the derivation-outputs page
So that this can match up with the package reproducibility page.
2019-12-16 20:29:28 +00:00
Christopher Baines
5b54259974 Fix matching equivalent derivations in another function 2019-12-16 20:11:57 +00:00
Christopher Baines
96f897cf3b Add a page with charts to show package reproducibility 2019-12-16 19:57:36 +00:00
Christopher Baines
bdcba528fc Fix group-to-alist
To not just append elements to the alist.
2019-12-16 15:19:02 +00:00
Christopher Baines
a41249d8fe Move group-to-alist to the model utils module
So it can be more widely used.
2019-12-16 15:19:02 +00:00
Christopher Baines
5337b171ea Add a function to find the builds required for a build that failed 2019-12-16 15:19:02 +00:00
Christopher Baines
771b01ef80 Link to the build pages from the derivation page 2019-12-16 15:19:02 +00:00
Christopher Baines
308d1d7102 Add a page for each build 2019-12-16 15:18:14 +00:00
Christopher Baines
a59f352046 Split out select-build-by-build-server-and-derivation-file-name
Create select-build-id-by-build-server-and-derivation-file-name, which just
returns the id, and then a variant that returns more fields for making a page
for each build.
2019-12-15 10:32:40 +00:00
Christopher Baines
b693d3e57d Improve select-builds-with-context-by-derivation-file-name
To find builds for equivalent derivations.
2019-12-15 08:51:31 +00:00
Christopher Baines
a84fc954d1 Extend select-build-stats to allow filtering by revision 2019-12-14 20:59:54 +00:00
Christopher Baines
633e757fe8 Break down the build stats by build server 2019-12-14 20:17:53 +00:00
Christopher Baines
9dddf2c606 Implement the reproducibility status filter
On the derivation-outputs page.
2019-12-14 17:10:23 +00:00
Christopher Baines
156b7eea7e Improve the select-build-stats function
Account for equivalent derivations, rather than counting them as unknown.
2019-12-12 20:07:22 +00:00
Christopher Baines
f13077d087 Remove non-existent export from the derivation module 2019-12-12 20:07:22 +00:00
Christopher Baines
eecfdeb9e4 Add a function to select the outputs in a revision 2019-12-12 20:07:22 +00:00
Christopher Baines
544f122ff5 Fix related builds in select-derivations-in-revision
Rather than just counting builds where the derivation matches, select builds
based on the derivation outputs, which accounts for different but equivalent
derivations.
2019-12-12 20:07:22 +00:00
Christopher Baines
c8b93cb0d0 Add new derivation_output_details_set_id column to the builds table
As this will hopefully provide a faster way of associating derivations with
builds.
2019-12-12 20:07:22 +00:00
Christopher Baines
a965ba1f6a Start storing and identifying sets of derivation outputs
Derivations are effectively equivalent if they produce the same set of
outputs, which is possible because of the equivalence of fixed output
derivations. A fixed output derivation can be different, but equivalent,
because it produces the same fixed output.

To better allow tracking equivalent derivations, primarily to allow working
out what derivations might correspond to a build, store the sets of derivation
outputs, and which derivations they relate to.
2019-12-12 20:07:22 +00:00
Christopher Baines
0cebb97340 Add a function to select derivations in a revision 2019-12-12 20:07:22 +00:00
Christopher Baines
aa3af00e07 Fix and improve fetching outputs to fetch from build servers
The join on derivation_outputs was completely wrong, so fix that.

Also, rather than only selecting outputs where there's a corresponding build,
just eliminate outputs where there is an entry in the nars table, fetched from
the relevant build server.

Also just look at x86_64-linux derivations, because there's no filtering on
relevant derivations for a build server at the moment.
2019-12-12 20:07:22 +00:00
Christopher Baines
1e040010fc Fix the match statement in select-build-by-build-server-and… 2019-12-12 20:07:22 +00:00
Christopher Baines
eb2ba599d9 Populate the narinfo_fetch_records table 2019-12-12 20:07:22 +00:00
Christopher Baines
8e5f8e4170 Add a new page for the narinfos associated with an output 2019-12-12 20:07:22 +00:00
Christopher Baines
2d87bd6340 Add a page for each signing key 2019-12-12 20:07:22 +00:00
Christopher Baines
a0ce016c02 Support querying builds by build servers 2019-12-12 20:07:22 +00:00
Christopher Baines
f7069456b5 Support querying builds by status 2019-12-12 20:01:30 +00:00
Christopher Baines
0fa747e404 Support handling jsonb in insert-missing-data-and-return-all-ids 2019-11-30 10:53:13 +00:00
Christopher Baines
b6194e7b3d Begin to add support for importing narinfo files
This commit adds the tables, as well as code to support extracting data from
narinfo files.
2019-11-30 10:51:58 +00:00
Christopher Baines
20c75e1103 Fix some queries relating to the build_status table changes 2019-11-28 19:32:00 +00:00
Christopher Baines
5663235048 Rework the builds and build_status tables as well as related code
Allow for build status information to be submitted by POST request. This
required some changes to the builds and build_status tables, as for example,
the Cuirass build id may not be available, and the derivation may not be know
yet, so just record the derivation file name.
2019-11-24 20:18:08 +00:00
Christopher Baines
79d85f5847 Add a new module to handle build server token seeds
The intent with these tokens is for them to be provided when build events are
submitted, preventing unauthorised submission of build events.

It should be possible to make the database public, so it shouldn't contain any
secret information. To have secret tokens that aren't in the database, store a
token seed, and combine this with some secret data to compute the secret.
2019-11-23 11:33:16 +00:00
Christopher Baines
6fad559b32 Improve select-build-servers to convert strings to appropriate types 2019-11-23 11:23:16 +00:00