Commit Graph

630 Commits

Author SHA1 Message Date
Christopher Baines 27904e8fd7 Fix some comments
Copied over from the Mumi code.
2020-02-16 22:29:11 +00:00
Christopher Baines e5a6680808 Make a constraint deferrable
To allow for deferring it in a transaction when deleting derivations.
2020-02-16 22:28:37 +00:00
Christopher Baines bfa468a325 Add another 0 to the job default character limit
This seems to work fine, and now because of the channel-instances, it's useful
to display more data by default.
2020-02-16 13:53:49 +00:00
Christopher Baines 07b97fcb73 Fix target handling on the package derivation history page 2020-02-16 13:38:07 +00:00
Christopher Baines b087cfca67 Define the code to delete data from non-master branches properly 2020-02-16 10:59:38 +00:00
Christopher Baines 9c7310f8e3 Make it possible to query builds servers for specific outputs 2020-02-16 09:54:27 +00:00
Christopher Baines 6f97cec962 Guard against build being missing from the output response 2020-02-16 09:06:12 +00:00
Christopher Baines cc370fae56 Better handle errors when fetching builds by output 2020-02-15 21:58:07 +00:00
Christopher Baines 119fd801e1 Fix some functions being called with the wrong number of arguments 2020-02-15 21:38:42 +00:00
Christopher Baines a2387c12d8 Improve error handling in the query-build-servers script 2020-02-15 21:33:47 +00:00
Christopher Baines 2c495fe8f6 Improve associating builds with derivations
Even without knowing the details of the derivation.
2020-02-15 21:29:42 +00:00
Christopher Baines c355c42584 Enhance ensure-build-exists
Support calling it with derivation-output-details-set-id, which can be used
when querying for builds.
2020-02-15 21:28:09 +00:00
Christopher Baines 5758795777 Set a default target for the revision builds page
This solves the build stats showing up incorrectly, as without this there was
no filtering on target.
2020-02-15 21:25:02 +00:00
Christopher Baines 33749786e4 Add verbose output to the query-build-servers script 2020-02-15 19:54:42 +00:00
Christopher Baines 617af6c9d3 Properly sort the derivation outputs 2020-02-15 19:54:17 +00:00
Christopher Baines a6e19a4102 Avoid an error when there are no references to insert 2020-02-15 18:23:04 +00:00
Christopher Baines 36254f98e3 Improve the job logging
Switch to using a sequence for the ids in the log parts table, and spawn a
thread to listen for output from the inferior processes, and enter it in to
the database.
2020-02-15 17:42:07 +00:00
Christopher Baines 40f6de27f6 Make deleting the existing log parts optional 2020-02-15 16:49:37 +00:00
Christopher Baines 46c8ce5b82 Define a new parameter for a port used for the inferior error output 2020-02-15 13:41:17 +00:00
Christopher Baines 773e5a9c38 Add a module to handle deleting data
This, along with the way of specifying which branches are processed is a way
to manage the data stored within the Guix Data Service.

This only goes so far, it doesn't delete derivations, but it does delete some
of the information related to a revision.
2020-02-15 11:36:31 +00:00
Christopher Baines f6f51bbe0d Fake the store connection in the tests
So that the tests don't require a store connection.
2020-02-14 09:18:28 +00:00
Christopher Baines 374dc25440 Improve system and target query parameter handling 2020-02-13 23:11:59 +00:00
Christopher Baines dfccbf911c Update the build_servers_build_config values
Change target to '' if it matches system, as this matches the updated way of
storing derivations.
2020-02-13 23:11:36 +00:00
Christopher Baines 5410961f65 Add temporary roots for channel instance derivations 2020-02-13 23:11:36 +00:00
Christopher Baines 0ef3349ed8 Use a more long lived store connection for loading data
As this will enable registering temporary roots, to avoid store items being
garbage collected.
2020-02-13 23:11:36 +00:00
Christopher Baines 32052c45b3 Remove the target query parameter from some links
As the system is no longer a valid value.
2020-02-12 09:00:21 +00:00
Christopher Baines 1c34e07e84 Query for builds for channel instance derivations 2020-02-12 08:38:05 +00:00
Christopher Baines 5cf186e5b1 Add a page for channel instance derivations 2020-02-12 08:37:57 +00:00
Christopher Baines 7306df0a0f Add a function to select channel instances for a revision 2020-02-12 08:37:38 +00:00
Christopher Baines 7326e8b3bb Remove the restriction on cross build derivation querying
As some build servers do build cross derivations.
2020-02-11 22:48:12 +00:00
Christopher Baines f5899a5762 Update the cross build filter for the package reproducibility page 2020-02-11 22:48:06 +00:00
Christopher Baines c8dec34f33 Switch to using more correct targets on the build page 2020-02-11 22:47:30 +00:00
Christopher Baines 9be7dbac0b Start storing channel instance derivations
These are the ones that relate to Guix pull.
2020-02-11 08:56:24 +00:00
Christopher Baines 406aa5e160 Switch using set-current-system to parameterize
As this is neater.
2020-02-11 08:55:34 +00:00
Christopher Baines 876bae182d Update the load-new-revision-job tests 2020-02-11 08:55:16 +00:00
Christopher Baines e0ee3d224b Refactor how channel derivations are handled
Yet again...

This makes the channel-derivations for each system accessible within the
load-new-guix-revision procedure, in preparation for storing them in the
database.
2020-02-09 21:40:56 +00:00
Christopher Baines 60d53f898f Collpase store-item-for-channel in to channel->guix-store-item 2020-02-09 21:06:12 +00:00
Christopher Baines 13b0ebe561 Start computing channel instance derivations for multiple systems
These aren't stored yet, but this is a start.
2020-02-09 21:03:05 +00:00
Christopher Baines beab4babac Further refactor channel->derivation-file-name
To reduce the line length.
2020-02-09 16:44:18 +00:00
Christopher Baines 9c38115c3d Refactor channel->derivation-file-name to reduce line length 2020-02-09 16:39:46 +00:00
Christopher Baines 3ee481110e Shift extracting channel news to later on
To avoid race conditions between jobs, have it happen after the call to
extract-information-from.
2020-02-09 10:20:36 +00:00
Christopher Baines 82b1ae6b75 Improve the branch-updated-emails module line length 2020-02-08 12:05:42 +00:00
Christopher Baines c31c9575e0 Support excluding and including branches from a repository
So that you can have the Guix Data Service only pay attention to some of the
branches.
2020-02-08 12:03:41 +00:00
Christopher Baines bb271c366b Tweak the sort order for derivation counts
So that the non-cross derivations appear first.
2020-02-08 11:42:24 +00:00
Christopher Baines f9871faa88 Remove peek call 2020-02-08 11:18:58 +00:00
Christopher Baines f1989e8758 Rework cross derivations support
Stop using the system values as targets, and remove package derivation entries
where this is the case.

Switch the non-cross derivation case to have a target of "", rather than
matching the system, as this makes more sense, and is more consistent now that
the target values no longer match the system values.

Hardcode some more correct target values, and use these instead. Hopefully
this can be better integrated with Guix in the future.

This commit also includes a migration attempting to shrink some indexes.
2020-02-08 11:16:08 +00:00
Christopher Baines 77beb59495 Use a temporary table when inserting package metadata entries
To speed up this step, as there is a lot of data.
2020-02-04 00:11:47 +00:00
Christopher Baines a3c5f53fcd Support using a temp table in insert-missing-data-and-return-all-ids
This is more performant when there is a lot of data.
2020-02-04 00:08:49 +00:00
Christopher Baines a3d84b1aec Add missing migration 2020-02-03 22:57:57 +00:00
Christopher Baines 19630014a3 Add a page to show system tests for a revision 2020-02-03 09:58:02 +01:00