As I'm seeing the inferior process crash with [1] just after fetching the
derivation lint warnings.
This change appears to help, although it's probably just a workaround. When
there's more packages/derivations, the caches might need clearing while
fetching the derivation lint warnings, or this will need to be split across
multiple processes.
1: Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
These cached store connections have caches associated with them, that take up
lots of memory, leading to the inferior crashing. This change seems to help.
To the end of the main revision processing transaction.
Currently, I think there are issues when this query does update some builds,
as those rows in the build table remain locked until the end of the
transaction. This then causes build event submission to hang. Moving this part
of the revision loading process to the end of the transaction should help to
mitigate this.
Previously, duplicates could creep through if the duplicate wasn't exported,
and only found as a replacement. Now they're filtered out.
This isn't ideal, as duplicates aren't always mistakes, it would be useful
still to capture this package, but having multiple entries for the same
name+version causes the comparison functionality to break.
As I think some operations (like the database backup) can block the DROP
SEQUENCE bit, so at least this approach means that the main transaction should
commit and then the sequence is eventually dropped.
This code is a bit tricky, since it should be compatible with old and new guix
revisions. I think these changes stop computing package derivations for
invalid systems, while hopefully not breaking anything.
Start at least looking for package replacements, and storing the
details (particularly the derivation). I'm looking at doing this so that build
servers using the Guix Data Service can build these derivations.
I guess this is a good change in general, but this seems to avoid a long
stack, which when a linter crashes, and the inferior tries to return the
exception details, and apparently hang the inferior/client as the reply isn't
written/read.
This means that these derivations are stored, even if a later part of the
process fails. Having the channel instance derivations stored might help work
out why the failure occurred, or better display information about it.
As each job deletes lots of rows in the table, vacuum it to free up space for
the next job.
I don't think this is something to do widely, but I wanted to try it out for
the tables which have a lot of churn.
From the normalized one, to the one actually contained within glibc. Recent
versions of glibc also contain symlinks linking the normalized codeset to the
locales with the .UTF-8 ending, but older ones do not.
Maybe handling codeset normalisation for queries would be good, but the locale
values ending in .UTF-8 are more compatible and allow the code to be
simplified. For querying, maybe there should be a locales table which handles
different representations.
Use the deduplicated list of packages when fetching lint warnings, to avoid
duplicates. This was first seen in the following Guix commit
843344273c6a587b8e6c84d8406500fd64d8908a.
In recent versions of Guix on the Guix Data Service side, I don't think these
are system-error's, but a different type. For compatibility, just catch any
error.
Maxime Devos
