2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/data-service.git synced 2023-12-14 03:23:03 +01:00
Commit graph

473 commits

Author SHA1 Message Date
Christopher Baines
2cb389037b Fix data input for package-metadata
Represent non-string home pages as NULL, and delete duplicates, as package
metadata entries can be duplicated.
2019-09-05 16:07:23 +02:00
Christopher Baines
4c67160b92 Improve error handling for insert-missing-data-and-return-all-ids 2019-09-05 16:07:23 +02:00
Christopher Baines
2b5594e5d4 Support deleting duplicates in insert-missing-data-and-return-all-ids
This is useful when you want to pass in data with some duplicates, and get
back a list of ids, where the duplicate entries are represented by the same
id.
2019-09-05 16:07:23 +02:00
Christopher Baines
1a55022524 Improve the package and package-metadata modules
Add tests around the package module, extract out the use of the
inferior-package record assessors so that they aren't part of the tests, and
switch across the package-metadata module to use
insert-missing-data-and-return-all-ids.
2019-09-05 16:07:23 +02:00
Christopher Baines
f29230e034 Put some comments in the mock-inferior module
About mocking the record file accessors not working :(
2019-09-05 16:07:23 +02:00
Christopher Baines
d3913a14d5 Start handling ids as numbers, rather than strings
squee, returns all data as strings, and expects strings as inputs to
queries. So, keeping the ids as strings was easy initially, but it means that
you can't tell from the type whether it should be quoted, or not...

Therefore, handle ids as strings, converting them to numbers when they're
fetched from the database, and back to strings as part of the queries.
2019-09-05 16:07:23 +02:00
Christopher Baines
6c90fe4324 Reduce code duplication in the package module
By using insert-missing-data-and-return-all-ids.
2019-09-05 16:07:23 +02:00
Christopher Baines
f1a9495727 Reduce code duplication in the package-derivation module
Through using insert-missing-data-and-return-all-ids.
2019-09-04 19:27:48 +02:00
Christopher Baines
fc6bbf3e3c Change license code to use insert-missing-data-and-return-all-ids
As this now supports inserting sets of data.
2019-09-04 19:27:48 +02:00
Christopher Baines
80010a8a1b Improve insert-missing-data-and-return-all-ids
Use exec-query-with-null-handling to distinguish NULL values, change it to
just take a list of fields and remove the handlers. Also, add a sets-of-data?
parameter so that this can be used licenses.
2019-09-04 19:27:48 +02:00
Christopher Baines
1441942200 Hack better NULL support on to (squee)
PQgetvalue used by squee returns null values as empty strings, which are
ambiguous for string fields. Therefore, use PQgetisnull to implement a
serialiser for squee which checks empty strings to see if they're actually a
NULL value, then returns '() in this case.

exec-query-with-null-handling can be used to access this behaviour.
2019-09-04 12:52:12 +02:00
Christopher Baines
902560d56b Change normalise-database-values to normalise-values
As these values are not from the database, which is why they're not strings.
2019-09-02 18:17:01 +02:00
Christopher Baines
939b8b8bb6 Fix a couple of GET's in the controller match expression
These should be symbols, not just placeholders.
2019-09-01 18:48:31 +01:00
Christopher Baines
960eac26d7 Fix the after link on the packages page 2019-09-01 18:47:06 +01:00
Christopher Baines
939b51c80c Add a header for the derivations on the package page 2019-09-01 18:32:09 +01:00
Christopher Baines
d3254dddb6 Add lint warnings to the package page 2019-09-01 18:32:09 +01:00
Christopher Baines
514933db47 Link to the package from the lint warnings page 2019-09-01 18:32:09 +01:00
Christopher Baines
14fab21dc1 Link to the lint warnings page from the revision page 2019-09-01 18:32:09 +01:00
Christopher Baines
be25e487f1 Support filtering lint warnings by linter 2019-09-01 18:32:09 +01:00
Christopher Baines
2bfd81ee1b Add lint-checkers-for-revision to the lint-checker module 2019-09-01 18:32:09 +01:00
Christopher Baines
91a9ba4349 Add a new table guix_revision_lint_checkers
To associate a set of lint checkers with a specific revision. While there is
the association through the lint warnings, that only works for checkers where
there are lint warnings for that revision.

Therefore, to allow finding all the checkers for a specific revision, also
associate them directly with the revision.
2019-09-01 18:32:09 +01:00
Christopher Baines
c1fad22cd1 Handle a change in guix with the <derivation-input> record
The first field is now a <derivation> record, rather than a store path for the
derivation.

It's probably not necessary to cope with two versions, but this is what I've
done currently.
2019-09-01 17:40:16 +01:00
Christopher Baines
3ad100bedc Log the manifest derivation when loading new revisions
As this is useful to know.
2019-09-01 17:40:16 +01:00
Christopher Baines
609c5cf4f0 Add a page to show the lint warnings for a revision 2019-09-01 17:40:16 +01:00
Christopher Baines
3544f9300f Add lint-warnings-for-guix-revision to the lint-warning module 2019-09-01 13:12:10 +01:00
Christopher Baines
07f368a54b Include glibc-locales from the inferior guix in the environment
When loading data from an inferior Guix, first build it's latest version of
glibc-locales, and include that in the environment from the inferior.

This improves locale support, which is currently relevant for extracting lint
warnings.

Without this change, you'd only be able to switch to locales supported by the
glibc-utf8-locales package, assuming that the right version is available.
2019-09-01 13:12:10 +01:00
Christopher Baines
af581784b2 Log the guix store path being used
In the load-new-guix-revision module.
2019-09-01 13:12:10 +01:00
Christopher Baines
f118ed0bf9 Improve handling of errors when changing locale
In the load-new-guix-revision module.
2019-09-01 13:12:10 +01:00
Christopher Baines
3a79449974 Display counts of lint warnings on the revision page 2019-09-01 13:12:10 +01:00
Christopher Baines
75f2c19fc8 Add lint-warning-count-by-lint-checker-for-revision
To return counts of lint warnings, grouped by lint checker for a specific
revision.
2019-09-01 13:12:10 +01:00
Christopher Baines
6b9977f62e Store lint warnings in the database
This commit adds the relevant tables and code to store lint warnings in the
database.

Currently, only lint checkers which don't require access to the network will
be run, as this allows the processing to happen without network access. Also,
this functionality won't work in older versions of Guix which don't expose the
lint warnings in a compatible way.
2019-09-01 13:12:10 +01:00
Christopher Baines
bf469504eb Add buffering to the log handling for jobs
To increase the log output speed, avoiding inserting one character at a time
in to the database.
2019-08-31 12:10:54 +01:00
Christopher Baines
8c741c569b Add insert-missing-data-and-return-all-ids to the model utils module
This should help greatly with populating the database with new entries, and
greatly reduce code duplication.
2019-08-31 12:09:54 +01:00
Christopher Baines
657c72c203 Use $STR$ for better quoting in quote-string 2019-08-31 12:09:31 +01:00
Christopher Baines
c7baf53c3f Add value->sql-boolean to (guix-data-service model utils) 2019-08-31 12:07:58 +01:00
Christopher Baines
66df42ae4f Fix insert-guix-revision to return an id
Rather than an id wrapped in a list.
2019-08-31 12:07:22 +01:00
Christopher Baines
d35cc85d52 Improve error handling for processing emails 2019-08-31 12:06:36 +01:00
Christopher Baines
84ad3656df Alphabetically order the derivation comparison page 2019-08-06 10:33:00 +01:00
Christopher Baines
89b498aad0 Remove the build status filter from the compare derivations page
As there isn't sufficient data to make this work yet.
2019-08-06 10:24:19 +01:00
Christopher Baines
3849bb79c5 Support filtering by systems and targets when comparing derivations 2019-08-06 10:22:57 +01:00
Christopher Baines
36a16d356f Improve derivation comparison to show more changes
In cases where the version is changed for example, the relevant derivations
will now show up, whereas previously they did not.
2019-08-06 09:55:03 +01:00
Christopher Baines
82c3e8942b Only show the Compare button when it'll work
On the branch page.
2019-08-05 21:24:07 +01:00
Christopher Baines
9e80bda4f9 Avoid erroring when processing emails again
These changes allow processing emails again, and just creating job and branch
entries where data is missing.
2019-08-05 20:59:48 +01:00
Christopher Baines
712dd377a1 Remove test data after the tests have ran
The load-new-guix-revision tests aren't transactional, so truncate some tables
after running the tests.
2019-08-05 20:45:40 +01:00
Christopher Baines
f553a7490d Change the git_branches table primary key
To allow for having branches in multiple git repositories.
2019-08-05 20:02:06 +01:00
Christopher Baines
4b59108fdd Properly handle not finding a matching derivation 2019-08-05 19:45:10 +01:00
Christopher Baines
70126f7ed0 Set the Vary header to a sensible default value
As most pages vary based on the Accept header.
2019-08-05 19:43:50 +01:00
Christopher Baines
8f7398b814 Tweak the branch table layout 2019-08-05 11:36:57 +01:00
Christopher Baines
002ae7685c Clear the previous log entry when starting a job 2019-08-04 09:42:21 +01:00
Christopher Baines
84197686ce Fix some duplicated values in tables
The licenses table, along with the package_metadata table had duplicate
values. This could happen as the unique constraints on those tables didn't
properly account for the nullable fields.

The duplicates in those tables also affected the license_sets, packages,
package_derivations tables in a similar way. Finally, the
guix_revision_package_derivations table was also affected.

This commit adds a migration to fix the data, as well as the constraints. THe
code to populate the licenses and package_metadata tables is also updated.
2019-08-04 09:39:40 +01:00