guix/data-service
2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/data-service.git synced 2023-12-14 03:23:03 +01:00
Code Issues Releases Wiki Activity
1228 commits 1 branch 0 tags 1.8 MiB
Commit graph

55 commits

Author SHA1 Message Date
Christopher Baines 1f078e1af0 Fix more things for the systems table introduction 2021-04-23 23:10:28 +01:00
Christopher Baines df2a0a73f1 Update for the changes around (guix scripts substitute) 
There's now a better (guix substitutes) module to use.
 2021-04-09 11:27:19 +01:00
Christopher Baines 375a6a37dc Support not querying pending builds 
As this can take some time.
 2020-11-01 22:52:53 +00:00
Christopher Baines f485423d5a Allow only fetching builds for a specific system 2020-11-01 22:49:49 +00:00
Christopher Baines 2732ba8f68 Support limiting fetching pending builds to specific revisions 2020-11-01 22:34:59 +00:00
Christopher Baines adc1d01525 Only query recent pending builds 
To avoid old pending builds that never leave the scheduled/started state.
 2020-11-01 21:53:19 +00:00
Christopher Baines 5674f8838d Improve verbose output for fetching build information 2020-11-01 21:53:04 +00:00
Christopher Baines 63fc1917f8 Sort the pending builds by timestamp 
So that builds that have been pending for a longer time are fetched first.
 2020-10-21 20:49:58 +01:00
Christopher Baines 2e859ed981 Increase the limit on pending builds to query 
As there are quite a lot of pending builds.
 2020-10-21 20:08:02 +01:00
Christopher Baines 41a7f38f44 Cleanup scheduled and started builds with "0" timestamps 
Not sure how these appeared, but deleting them seems to be sensible.
 2020-10-21 19:25:30 +01:00
Christopher Baines 83f68a8afa Display backtraces when exceptions occur for fetching builds 2020-10-13 19:35:45 +01:00
Christopher Baines e273e22042 Switch queries across to use latest_build_status 2020-10-13 19:35:40 +01:00
Christopher Baines 9192550331 Implement more support for builds from the Guix Build Coordinator 
Builds from the Guix Build Coordinator might not have timestamps, and the id
from the build server is more important, as one build server can build the
same derivation many times.
 2020-07-01 09:35:29 +01:00
Christopher Baines b6754c8a4c Add a lookup_builds field to the build_servers table 
This is to allow for build servers where only the substitutes should be
queried, and it shouldn't be assumed that they're running Cuirass.
 2020-05-24 17:02:53 +01:00
Christopher Baines a21d3d3cf7 Better handle cuirass builds with a stoptime of 0 
Use the timestamp instead for the status. Previously no status was being
associated with these builds.
 2020-05-06 23:09:05 +01:00
Christopher Baines 6baef6ae25 Split out querying of build servers and substitute servers 
These are related things, but somewhat separate. This change should make it
easier to deal with changes regarding querying build servers, and querying
substitute servers.
 2020-05-03 13:23:43 +01:00
Christopher Baines edcf1c1e8f Fetch narinfo files first, because that's faster 
At least than fetching builds.
 2020-04-25 12:46:39 +01:00
Christopher Baines 33958eac79 Increase the batch size for fetching builds/narinfo files 
To increase the likelyhood that all the builds and narinfo files for the
latest revisions are fetched.
 2020-03-14 12:35:34 +00:00
Christopher Baines 24e7df541b Query for more recent derivation outputs by default 
Rather than looking for the oldest unknown outputs, as the new ones are
generally more useful.
 2020-03-12 20:47:43 +00:00
Christopher Baines 885053e107 Remove peek call that snuck in to the builds module 2020-03-02 21:48:14 +00:00
Christopher Baines c407f55c84 Update http-multiple-get 
Update this by copying the code from Guix again.
 2020-03-02 21:18:10 +00:00
Christopher Baines 18eb9dfdcb Improve the builds verbose output 2020-03-01 19:28:50 +00:00
Christopher Baines b19227c086 Order build statuses by id, not timestamp 
Some builds from Cuirass change status at the same timestamp, so use the id
for ordering instead.
 2020-03-01 15:57:35 +00:00
Christopher Baines 9c7310f8e3 Make it possible to query builds servers for specific outputs 2020-02-16 09:54:27 +00:00
Christopher Baines 6f97cec962 Guard against build being missing from the output response 2020-02-16 09:06:12 +00:00
Christopher Baines cc370fae56 Better handle errors when fetching builds by output 2020-02-15 21:58:07 +00:00
Christopher Baines a2387c12d8 Improve error handling in the query-build-servers script 2020-02-15 21:33:47 +00:00
Christopher Baines 2c495fe8f6 Improve associating builds with derivations 
Even without knowing the details of the derivation.
 2020-02-15 21:29:42 +00:00
Christopher Baines 33749786e4 Add verbose output to the query-build-servers script 2020-02-15 19:54:42 +00:00
Christopher Baines 1c34e07e84 Query for builds for channel instance derivations 2020-02-12 08:38:05 +00:00
Christopher Baines 00d9664714 Query builds for system test derivations 2020-02-03 09:57:35 +01:00
Christopher Baines cf407fe830 Switch to querying builds by output, rather than derivation 
This is better, as Cuirass will return a build for the derivation it built to
generate that output. This avoids having to query for multiple derivations
that generate a single output, until the one that Cuirass used is found.
 2020-01-16 20:42:56 +00:00
Christopher Baines c7e3a1bd0b Use the build_servers_build_config table when querying build servers 
This allows configuing which systems a build server builds.
 2020-01-05 12:48:49 +00:00
Christopher Baines 0759be52de Remove -for-successful-builds from procedure name 
This used to be the case, but now it just selects all outputs, as that's
quicker overall (as querying the build server is quick).
 2020-01-05 12:01:06 +00:00
Christopher Baines b877b64eb8 Remove reverse from the copy of http-multiple-get 
This makes assumptions about the return value of the provided procedure, and
fails if it doesn't return a list.
 2019-12-26 23:01:34 +00:00
Christopher Baines 0cea51cc79 Add copyright and license headers to a bunch of source files 
That were missing them.
 2019-12-26 10:16:55 +00:00
Christopher Baines 566f20a03d Use HTTP pipelining when making requests to build servers 
This removes some overhead in making a connection for each request.
 2019-12-26 08:42:44 +00:00
Christopher Baines 31110acc91 Fix a bug with argument handling in the query-build-servers script 2019-12-22 22:35:43 +00:00
Christopher Baines af948de646 Fix the select-derivations-with-no-known-build 
The equivalent_derivations table was an experiment that didn't work, so switch
to using the derivations_by_output_details_set table.

Also take the build server id as input, as this allows selecting derivations
which have no known build for the relevant build server.
 2019-12-12 20:07:22 +00:00
Christopher Baines c90f411e5e Add options to the query-build-servers script 
So you can select to query specific build servers.
 2019-12-12 20:07:22 +00:00
Christopher Baines 9e5cde80b3 Fix query in select-derivations-with-no-known-build 
Rather than eliminating any derivation that has a known build, eliminate those
derivations, but also equivalent derivations as well.

For selecting the derivations in a revision, join on the equivalent
derivations, as all the equilalent derivations need checking as well, as it's
unknown which one Cuirass would have associated the build against.

Also, filter for x86_64-linux, to avoid checking for crossbuilt things. This
should be replaced by some way of describing what derivations a build server
has.
 2019-12-12 20:07:22 +00:00
Christopher Baines 2e9844e599 Display the build server URL when fetching data 2019-12-12 20:07:22 +00:00
Christopher Baines 29e897ce9c Improve variable names in fetch-narinfo-files 2019-12-12 20:07:22 +00:00
Christopher Baines eb2ba599d9 Populate the narinfo_fetch_records table 2019-12-12 20:07:22 +00:00
Christopher Baines 7cc5c02cdd Query build servers for narinfo files 
As well as making it possible to only query build servers for builds relating
to a specific revision.
 2019-11-30 10:59:16 +00:00
Christopher Baines b278065d2a Fix selecting pending builds 2019-11-30 10:58:40 +00:00
Christopher Baines 0aa78e9ef5 Don't create misleading scheduled build status entries 
Cuirass provides a timestamp field in build responses, and sometimes this
means when the build was scheduled, but when the build is finished, it's the
stoptime.

So only use the timestamp when the build hasn't finished.
 2019-11-30 10:55:16 +00:00
Christopher Baines 5663235048 Rework the builds and build_status tables as well as related code 
Allow for build status information to be submitted by POST request. This
required some changes to the builds and build_status tables, as for example,
the Cuirass build id may not be available, and the derivation may not be know
yet, so just record the derivation file name.
 2019-11-24 20:18:08 +00:00
Christopher Baines 6fad559b32 Improve select-build-servers to convert strings to appropriate types 2019-11-23 11:23:16 +00:00
Christopher Baines 5bc0e7d4bf
Add some catch all error handling to fetch-build-for-derivation 2019-03-08 08:22:52 +00:00