guix/data-service
2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/data-service.git synced 2023-12-14 03:23:03 +01:00
Code Issues Releases Wiki Activity
1258 commits 1 branch 0 tags 1.8 MiB
Commit graph

1258 commits

This branch
This branch
All branches
Author SHA1 Message Date
Christopher Baines 0796cb3bd3 Add a chunk procedure 
Just a variant of chunk! which doesn't modify the provided list.
 2021-10-03 14:54:10 +01:00
Christopher Baines af0a06d147 Log the time to read missing derivations from the store 2021-10-03 12:59:26 +01:00
Christopher Baines 3627d36d77 Select existing derivations in chunks 
To avoid one massive query.
 2021-10-03 12:59:02 +01:00
Christopher Baines 857b4e32d5 Insert derivation inputs in chunks 
To avoid one massive query.
 2021-10-03 12:56:23 +01:00
Christopher Baines d5ab67000e Add a chunk! untility 
For splitting a list in to multiple chuncks, satisfying some max length.
 2021-10-03 12:55:21 +01:00
Christopher Baines 211da6868f Handle the case where there are no missing file names 
In update-derivation-ids-hash-table!.
 2021-09-25 00:09:08 +01:00
Christopher Baines 3081887b90 Optimise inserting derivation inputs 
Rather than querying for the output ids one by one and then running an insert
query for each derivation, perform the task with a single insert query.
 2021-09-24 18:22:28 +01:00
Christopher Baines abff41f9ae Neaten up formatting in select-derivation-output-id 2021-09-24 17:26:48 +01:00
Christopher Baines 947cabb685 Replace derivation-file-names->vhash 
Rather than creating vhashes, just update the hash table that is used as a
cache, and query that. This should speed things up and reduce memory usage
when loading derivations.
 2021-09-24 17:14:40 +01:00
Christopher Baines 261552bd5e Speed up inserting missing derivation sources 
Split the recursive part of the query from the non-recursive part, since
PostgreSQL doesn't do a great job of estimating the number of rows which will
come back from the recursive part, and thus generates a bad plan.
 2021-09-24 12:55:45 +01:00
Christopher Baines 8b34126d22 Fix the select-source-files-missing-nars query 
Previously, the VALUES list wasn't being generated properly.
 2021-09-16 17:01:34 +01:00
Christopher Baines f2b9663cf2 Make sure to add missing source file nars when inserting derivations 
Source file nars weren't always recorded, so this will help backfill that
data (which enables providing substitutes for those derivations).
 2021-09-14 21:19:48 +01:00
Christopher Baines 52094f5f19 Handle #f package descriptions 
One was added in e81cf4e79a6e297db0ae2a9c39eab495e7e204f0.
 2021-08-09 09:43:03 +01:00
Christopher Baines 5cefa250d8 Add an index on derivation_output_details.id 
Where hash is not null, since this helps with the fixed output package
derivations query.
 2021-07-11 15:25:12 +01:00
Christopher Baines d1a2a7125c Fix a regression with running sqitch 
Introduced in 0dc05982cd.
 2021-07-11 12:40:48 +01:00
Christopher Baines af209170f7 Track package replacements 
Start at least looking for package replacements, and storing the
details (particularly the derivation). I'm looking at doing this so that build
servers using the Guix Data Service can build these derivations.
 2021-07-11 11:57:05 +01:00
Christopher Baines b4188bda9d Run sqitch in the change mode 
Since this rolls back migrations less, which is good when the rollback bit
isn't always implemented.
 2021-07-04 10:43:13 +01:00
Christopher Baines 0dc05982cd Try to adapt the PostgreSQL paramstring to use with sqitch 2021-06-16 13:44:00 +01:00
Christopher Baines 673d762e68 Remove redundant derivation-sources call 2021-06-09 16:38:33 +01:00
Christopher Baines 1a21bc40a8 Pass #:system to channel-instances->manifest 
This is better than setting the %current-system, since more of the process
will run as native code.
 2021-06-09 10:59:31 +01:00
Christopher Baines 8a48960fa7 Use derivations.system_id in select-builds-with-context 
As that table is always included in the query.
 2021-06-09 10:35:29 +01:00
Christopher Baines f421faa790 Fix a uri encoding issue with linking to outputs 2021-06-09 10:28:11 +01:00
Canan Talayhan 9c1b989d20 Add index for package_metadata 
This index speeds up inserting new entries to the package_metadata table.

Signed-off-by: Christopher Baines <mail@cbaines.net>
 2021-05-27 21:26:56 +01:00
Christopher Baines 854f2cbba9 Fix a URI decoding issue 
This caused a problem viewing this derivation
/gnu/store/65r4488732qy6ilrh1ryb8xwmip1qnir-tklib-0.6.tar.gz?uuid=tklib-0-6.drv
 2021-05-27 09:38:41 +01:00
Christopher Baines f4a87ec059 Add some indexes 
This is from reading the slow query log on data.guix.gnu.org.
 2021-05-17 18:40:45 +01:00
Christopher Baines b8bfdcf622 Remove the "derivations using output" data 
From the HTML and JSON responses, since this data isn't very useful, and the
query to collect the small amount which is displayed is very slow. This could
return at some point, but in a proper paginated form which isn't slow.
 2021-05-17 18:35:19 +01:00
Christopher Baines da0ee9dff0 Use filter-map rather than filter and map when processing linters 
I guess this is a good change in general, but this seems to avoid a long
stack, which when a linter crashes, and the inferior tries to return the
exception details, and apparently hang the inferior/client as the reply isn't
written/read.
 2021-05-16 20:54:07 +01:00
Christopher Baines 2a4b16f5e4 Fix incorrect query parameters in a couple of builds queries 
The parameters must be strings rather than numbers.
 2021-05-12 08:28:06 +01:00
Christopher Baines cdb41cfa3c Fix query typo in branches-by-package-version 2021-05-12 08:27:48 +01:00
Christopher Baines d13a7187af Fix request->path-components-and-mime-type 
Previously, it wouldn't pull the mime types out of the request properly, I
think it does a better job of that now.
 2021-05-12 08:26:15 +01:00
Luciana Lima Brito 767e60b2b3 Change data handling when comparing derivations 
comparison.scm: return query data for derivation comparison as an alist,
instead of list.
html.scm: Access derivation differences data using assq-ref.
controller.scm: remove mapping for outputs/inputs/sources.
utils.scm: add group-to-alist/vector function.

Signed-off-by: Christopher Baines <mail@cbaines.net>
 2021-04-27 21:18:51 +01:00
Christopher Baines e38bddcae5 Reformat a couple of package related queries 2021-04-24 06:00:47 +01:00
Christopher Baines 1f078e1af0 Fix more things for the systems table introduction 2021-04-23 23:10:28 +01:00
Christopher Baines c252d1cb30 Actually add the system module 2021-04-23 18:42:40 +01:00
Christopher Baines 869d2522e5 Add the system module to Makefile.am 2021-04-23 18:07:55 +01:00
Canan Talayhan fe97021cd0 Set more informative page titles 
For many pages where previously the title was just "Guix Data Service".

Signed-off-by: Christopher Baines <mail@cbaines.net>
 2021-04-23 12:00:03 +01:00
Christopher Baines 6387f1bc67 Fetch the list of system values from the database 
This removes the need to hardcode some values in the code.
 2021-04-23 11:20:36 +01:00
Christopher Baines b430c632b7 Create a table for systems 
And use it for the systems in the derivations and package derivations tables.

The primary motivation here is to allow quickly working out what systems the
database contains, and having a small table with just the right data seems a
good way to do that.
 2021-04-23 11:14:51 +01:00
Christopher Baines 1cc5accb9e Fix an error with the formatted derivation page 2021-04-23 11:13:04 +01:00
Luciana Brito a498433643 Implement basic JSON output for the derivation comparison page 
Signed-off-by: Christopher Baines <mail@cbaines.net>
 2021-04-19 20:53:46 +01:00
Christopher Baines c2c033b435 Fix a timezone issue 2021-04-10 14:44:10 +01:00
Christopher Baines a8b903f7b9 Only query for substitutes for things built in the last 5 minutes 2021-04-10 10:08:50 +01:00
Christopher Baines f37ae27ca6 Support querying for outputs that have had successful builds 2021-04-10 10:08:25 +01:00
Christopher Baines 28a7cb6291 Better handle filtering narinfos 
This avoids a query error when trying to insert data for 0 narinfos.
 2021-04-10 08:30:48 +01:00
Christopher Baines 91aebb3c4e State where to send patches in the README 2021-04-10 08:23:39 +01:00
Christopher Baines df2a0a73f1 Update for the changes around (guix scripts substitute) 
There's now a better (guix substitutes) module to use.
 2021-04-09 11:27:19 +01:00
Christopher Baines 410f58cb43 Fix load revision jobs not failing if the extraction of data fails 2021-03-25 17:57:48 +00:00
Christopher Baines 0175914f79 Fix a null handling issue for system test derivation comparison 2021-03-25 17:49:44 +00:00
Christopher Baines f47d337a72 Have branches-by-package-version look at latest processed revision 
This means branches aren't omitted if the latest commit hasn't been processed
yet.
 2021-03-15 10:29:36 +00:00
Christopher Baines 663bd1411a Add a /package/NAME page 
This might be useful for working out when a non-master branch contains a newer
version of a package, or someone has sent in a patch for a newer version
already.
 2021-03-14 21:48:43 +00:00