guix/data-service
2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/data-service.git synced 2023-12-14 03:23:03 +01:00
Code Issues Releases Wiki Activity
630 commits 1 branch 0 tags 1.8 MiB
Commit graph

183 commits

Author SHA1 Message Date
Christopher Baines c355c42584 Enhance ensure-build-exists 
Support calling it with derivation-output-details-set-id, which can be used
when querying for builds.
 2020-02-15 21:28:09 +00:00
Christopher Baines 617af6c9d3 Properly sort the derivation outputs 2020-02-15 19:54:17 +00:00
Christopher Baines a6e19a4102 Avoid an error when there are no references to insert 2020-02-15 18:23:04 +00:00
Christopher Baines 374dc25440 Improve system and target query parameter handling 2020-02-13 23:11:59 +00:00
Christopher Baines 7306df0a0f Add a function to select channel instances for a revision 2020-02-12 08:37:38 +00:00
Christopher Baines 7326e8b3bb Remove the restriction on cross build derivation querying 
As some build servers do build cross derivations.
 2020-02-11 22:48:12 +00:00
Christopher Baines f5899a5762 Update the cross build filter for the package reproducibility page 2020-02-11 22:48:06 +00:00
Christopher Baines 9be7dbac0b Start storing channel instance derivations 
These are the ones that relate to Guix pull.
 2020-02-11 08:56:24 +00:00
Christopher Baines c31c9575e0 Support excluding and including branches from a repository 
So that you can have the Guix Data Service only pay attention to some of the
branches.
 2020-02-08 12:03:41 +00:00
Christopher Baines bb271c366b Tweak the sort order for derivation counts 
So that the non-cross derivations appear first.
 2020-02-08 11:42:24 +00:00
Christopher Baines f1989e8758 Rework cross derivations support 
Stop using the system values as targets, and remove package derivation entries
where this is the case.

Switch the non-cross derivation case to have a target of "", rather than
matching the system, as this makes more sense, and is more consistent now that
the target values no longer match the system values.

Hardcode some more correct target values, and use these instead. Hopefully
this can be better integrated with Guix in the future.

This commit also includes a migration attempting to shrink some indexes.
 2020-02-08 11:16:08 +00:00
Christopher Baines 77beb59495 Use a temporary table when inserting package metadata entries 
To speed up this step, as there is a lot of data.
 2020-02-04 00:11:47 +00:00
Christopher Baines a3c5f53fcd Support using a temp table in insert-missing-data-and-return-all-ids 
This is more performant when there is a lot of data.
 2020-02-04 00:08:49 +00:00
Christopher Baines 19630014a3 Add a page to show system tests for a revision 2020-02-03 09:58:02 +01:00
Christopher Baines 8f0d5b51b1 Store information about system tests 2020-02-03 09:57:02 +01:00
Christopher Baines 671a85f798 Add a function to check how many repositories have x_git_repo_header 2020-02-01 13:59:41 +01:00
Christopher Baines bd52629674 Fix insert-build when derivation_output_details_set_id is NULL 
I can't figure out how to insert NULL through a param, so just construct a
query to insert the values directly.
 2020-01-16 20:37:02 +00:00
Christopher Baines 54baf32be6 Fix select-build-id-by-build-server-and-derivation-file-name 
The id will be wrapped in a list.
 2020-01-16 20:34:44 +00:00
Christopher Baines 2f36d47b1b Don't hardcode the expected x-git-repo header value 
Rather than expecting it always to be "guix", store the expected value in the
database, and use the value of the header to find the relevant repository.
 2020-01-11 17:25:08 +00:00
Christopher Baines c7e3a1bd0b Use the build_servers_build_config table when querying build servers 
This allows configuing which systems a build server builds.
 2020-01-05 12:48:49 +00:00
Christopher Baines c388f3ee13 Add a basic page for build servers 2020-01-05 12:01:06 +00:00
Christopher Baines 726674486f Add a couple of comments to select-outputs-without-known-nar-entries 2020-01-05 12:01:06 +00:00
Christopher Baines 0759be52de Remove -for-successful-builds from procedure name 
This used to be the case, but now it just selects all outputs, as that's
quicker overall (as querying the build server is quick).
 2020-01-05 12:01:06 +00:00
Christopher Baines e070c20030 Make valid-systems quicker 
In the future, it would be good to pull this from the database again, but in
some way which is really quick.
 2020-01-05 12:01:06 +00:00
Christopher Baines d7036bdd37 Implement the search query input on the derivation outputs page 2020-01-04 22:15:39 +00:00
Christopher Baines ceb1f95a19 Add a function to search for derivations in a revision 2020-01-04 21:45:27 +00:00
Christopher Baines e5aca1f317 Remove the outputs from the query behind the revision derivations 
As that data is now unused.
 2020-01-04 21:20:32 +00:00
Christopher Baines b26ba82dad Add more timing and debugging output to the load process 
To help identify what's taking the most time.
 2020-01-04 11:20:02 +00:00
Christopher Baines d107830300 Add a #:loop? option to backfill-derivation-source-file-nars 2020-01-03 21:42:52 +00:00
Christopher Baines 4ed14007c4 Fix some instability in the lint warnings page results 
The ordering could flip around, so fix it better.
 2020-01-02 21:19:43 +00:00
Christopher Baines a6302a32ef Make it possible to specify the batch size 
In backfill-derivation-source-file-nars.
 2019-12-31 14:58:31 +00:00
Christopher Baines 2cb5309851 Display details for the derivation source files 
If the nar file is available.
 2019-12-30 12:32:20 +00:00
Christopher Baines 7881eec315 Add a function to backfill missing derivation source file nars 
If the file exists in the local store, then read it and add an entry to the
derivation_source_file_nars table. This will help to fill in the missing
entries, as currently entries are only added when the derivation source file
isn't in the database when the load new revision job runs.
 2019-12-30 11:26:23 +00:00
Christopher Baines 7e7cc1c8f1 Rework insert-derivation-sources to use a helper function 
This means it returns ids properly.
 2019-12-29 16:04:45 +00:00
Christopher Baines d15ba4f25c Serve narinfo files for derivation sources 2019-12-29 16:04:45 +00:00
Christopher Baines da3a294496 Serve nar files for the derivation source files 2019-12-29 16:04:45 +00:00
Christopher Baines 67af7e17f0 Generate and store nars for derivation source files 
This'll allow serving nars for these derivation source files.
 2019-12-29 16:04:45 +00:00
Christopher Baines f56935a8a6 Change the language around reproducibility/matching outputs 
It's more accurate to describe the specifics of the relevant data here through
terms like "matching" and "not matching", as a statement that something built
reproducibility needs to be made alongside the test conditions. So just say
that build outputs matched, or didn't match, as this is more descriptive of
the data available.
 2019-12-27 14:05:12 +00:00
Christopher Baines 7654877991 Improve the builds info on the revision package version page 
Show multiple builds, and link to the build page.
 2019-12-27 00:09:07 +00:00
Christopher Baines 0e3c28ffbd Fix more cases where equivalent builds were missed 2019-12-26 23:22:54 +00:00
Christopher Baines 0cea51cc79 Add copyright and license headers to a bunch of source files 
That were missing them.
 2019-12-26 10:16:55 +00:00
Christopher Baines 9ffe884711 Remove a rogue 1 from the (guix-data-service model nar) module 2019-12-26 10:16:16 +00:00
Christopher Baines 120af42c24 Add a function to fetch references for derivations 
This replicates the store item references for the derivation, through looking
up the inputs which the derivation references, and also the sources.
 2019-12-26 09:34:41 +00:00
Christopher Baines e3d8a23716 Add a function to fetch a derivation by file name hash 2019-12-26 09:34:35 +00:00
Christopher Baines b504094262 Use the returned file name in select-derivation-by-file-name 
They should be the same, but it seems more sensible to return the value from
the database.
 2019-12-26 09:34:08 +00:00
Christopher Baines a92d4d0cf2 Add a function serialize a derivation 
This effectively duplicates the behaviour in Guix for serializing derivations,
but this uses the database representation in the Guix Data Service, rather
than the records Guix uses.
 2019-12-26 09:04:00 +00:00
Christopher Baines 21e298f68c Ensure output ordering for derivations is correct 
This affects the formatted derivation output.
 2019-12-26 09:03:38 +00:00
Christopher Baines dd94f59acf Fix the representation of the fixed output derivation hashes 
Previously, they were nix-base32-string encoded, but the representation in the
derivations is base16, so it doesn't make sense to use a different
representation in the database.

Therefore, add some code that runs before the start of each job to convert the
data in the database. It was easier to do this in Guile with the existing
support for working with these bytevector representations. After some
migration period, the code for converting the old hashes can be removed.
 2019-12-26 08:53:51 +00:00
Christopher Baines 4eb5a3417c Group derivation input outputs together by derivation 
Both in terms of the code fetching the data from the database, as well as the
formatted and detail outputs. This corrects an error in the formatted output
for derivations where inputs would be duplicated.
 2019-12-26 08:52:04 +00:00
Christopher Baines 801ebdfa9e Fix the successful builds outputs query where there are no revisions 
As it was erroring before.
 2019-12-26 08:35:33 +00:00