Previously, we'd spawn 'guix authenticate' once for each item that has
to be signed (when exporting) or authenticated (when importing). Now,
we spawn it once for all and then follow a request/reply protocol. This
reduces the wall-clock time of:
guix archive --export -r $(guix build coreutils -d)
from 30s to 2s.
* guix/scripts/authenticate.scm (sign-with-key): Return the signature
instead of displaying it. Raise a &formatted-message instead of calling
'leave'.
(validate-signature): Likewise.
(read-command): New procedure.
(define-enumerate-type, reply-code): New macros.
(guix-authenticate)[send-reply]: New procedure.
Change to read commands from current-input-port.
* nix/libstore/local-store.cc (runAuthenticationProgram): Remove.
(authenticationAgent, readInteger, readAuthenticateReply): New
functions.
(signHash, verifySignature): Rewrite in terms of the agent.
* tests/store.scm ("import not signed"): Remove 'pk' call.
("import signed by unauthorized key"): Check the error message of C.
* tests/guix-authenticate.sh: Rewrite using the new protocol.
fixlet
* nix/libstore/build.cc (DerivationGoal::tryBuildHook): Add "offload" to
'args' and pass settings.guixProgram as the first argument to
Agent::Agent.
(pathNullDevice, commonChildInit, Agent, Agent::Agent)
(Agent::~Agent): Move to...
* nix/libutil/util.cc: ... here.
* nix/libutil/util.hh (struct Agent, commonChildInit): New
declarations.
* nix/libstore/local-store.cc (signHash, verifySignature): New
functions.
(LocalStore::exportPath): Use 'signHash' instead of inline code.
(LocalStore::importPath): Use 'verifySignature' instead of inline code.
* nix/libstore/build.cc (HookInstance): Rename to...
(Agent): ... this. Rename 'toHook' and 'fromHook' similarly and update
users. Change constructor to require a command and an argument list.
(DerivationGoal::tryBuildHook): Pass arguments to the 'Agent'
constructor.
There's no reason at this point to mimic the calling convention of the
'openssl' command.
* nix/libstore/local-store.cc (LocalStore::exportPath): Add only "sign"
and HASH to ARGS. Remove 'tmpDir' and 'hashFile'.
(LocalStore::importPath): Add only "verify" and SIGNATURE to
* guix/scripts/authenticate.scm (guix-authenticate): Adjust
accordingly; remove the OpenSSL-style clauses.
(read-hash-data): Remove.
(sign-with-key): Replace 'port' with 'sha256' and adjust accordingly.
(validate-signature): Export SIGNATURE to be a canonical sexp.
* tests/guix-authenticate.sh: Adjust tests accordingly.
Suggested by Caleb Ristvedt <caleb.ristvedt@cune.org>.
* nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Save errno
from 'rename' before calling 'unlink'.
Remove .gitignore entries where they match source files that are tracked in
Git. This is relevant to me at least, as some code searching tools use
.gitignore files and will ignore matched files.
Fixes <https://bugs.gnu.org/41607>.
Reported by Stephen Scheck <singularsyntax@gmail.com>.
* nix/libstore/gc.cc (LocalStore::deletePathRecursive): When we try to
move a dead directory into the trashDir using rename(2) but it returns
an EXDEV error, just delete the directory instead. This can happen in a
Docker container when the directory is not on the "top layer".
This follows up on 1a1faa78b0, and avoids
the (non-fatal) error seen in <https://issues.guix.gnu.org/41356>.
/gnu/store will remain writable on new foreign distribution
installations until the next release.
* etc/guix-install.sh (sys_enable_guix_daemon): Check for
‘gnu-store.mount’ presence before trying to cp it.
Update forgotten copyright header.
* etc/gnu-store.mount.in: New file.
* nix/local.mk (nodist_systemdservice_DATA): Add it.
(etc/%.mount): New rule for it.
* etc/guix-install.sh (sys_enable_guix_daemon): Install it.
* doc/guix.texi (Binary Installation): Document it.
* .gitignore: Ignore changes to it.
This allows for native builds on the Hurd, doing
sudo ./pre-inst-env guix-daemon --disable-chroot --build-users-group=guixbuild &
./pre-inst-env guix build hello
* nix/libutil/util.cc (killUser)[__GNU__]: Avoid kill -1 bug; kill only
current process and ignore SIGKILL status in parent.
Co-authored-by: Jan Nieuwenhuizen <janneke@gnu.org>
Checking for CLONE_NEWNS is only needed for using tha Linux specific clone(2),
otherwise we can use fork(2). Using clone on the Hurd needs some work, only
support LINUX for now. See
https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00190.html
* nix/libstore/build.cc (CHROOT_ENABLED): Break into CHROOT_ENABLED
and CLONE_ENABLED.
(DerivationGoal::startBuilder): Replace CHROOT_ENABLED with __linux__.
(DerivationGoal::runChild): Only define pivot_root() if SYS_pivot_root is
defined.
Co-authored-by: Jan Nieuwenhuizen <janneke@gnu.org>
Fixes <https://bugs.gnu.org/39727>.
Reported by Paul Garlick <pgarlick@tourbillion-technology.com>.
* nix/libstore/gc.cc (LocalStore::removeUnusedLinks) [HAVE_STATX]: Add
'statx_flags' static variables. Clear 'AT_STATX_DONT_SYNC' flag from
'statx_flags' when 'statx' returns EINVAL.
Previously, a store item that is a regular file would not be accounted
for in the 'bytesFreed' value computed by 'deletePath' because its
'st_nlink' count would always be >= 2. This commit fixes that.
* nix/libutil/util.hh (deletePath): Add optional 'linkThreshold' argument.
* nix/libutil/util.cc (_deletePath): Add 'linkThreshold' argument and
honor it. Pass it down in recursive call.
(deletePath): Add 'linkThreshold' and honor it.
* nix/libstore/gc.cc (LocalStore::deleteGarbage): Pass 'linkThreshold'
argument to 'deletePath', with a value of 2 when PATH is a store item
and deduplication is on.
This is a followup to bc69ea2d60.
* guix/scripts/build.scm (show-build-options-help): Rename
"--no-build-hook" to "--no-offload".
(%standard-build-options): Likewise, and warn when "--no-build-hook" is
passed.
* nix/nix-daemon/guix-daemon.cc (options): Add "--no-offload" and mark
"--no-build-hook" as hidden.
* guix/scripts/offload.scm: Adjust comment.
* doc/guix.texi (Invoking guix-daemon, Common Build Options): Replace
"--no-build-hook" with "--no-offload".
* etc/completion/fish/guix.fish, etc/completion/zsh/_guix: Adjust
accordingly.
* nix/libstore/gc.cc (LocalStore::deletePathRecursive): Display the
percentage reached relative to 'maxFreed', or the total amount of data
deleted when 'maxFreed' is ULLONG_MAX.
As of GNU libc 2.29, <sys/mount.h> declares all the constants and
functions we need, so there's no use in including <linux/fs.h> anymore.
This silences annoying warnings like this one:
In file included from nix/libstore/local-store.cc:32:0:
/gnu/store/…-linux-libre-headers-4.19.56/include/linux/fs.h:108:0: warning: "MS_RDONLY" redefined
#define MS_RDONLY 1 /* Mount read-only */
In file included from nix/libstore/local-store.cc:28:0:
/gnu/store/…-glibc-2.29/include/sys/mount.h:36:0: note: this is the location of the previous definition
#define MS_RDONLY MS_RDONLY
* config-daemon.ac: Remove check for <linux/fs.h>.
* nix/libstore/build.cc: Remove conditional inclusion of <linux/fs.h>.
* nix/libstore/local-store.cc: Remove "#if HAVE_LINUX_FS_H" and
inclusion of <linux/fs.h>.
Fixes <https://bugs.gnu.org/38062>.
This is a followup to ada9a19a2d.
* nix/libstore/build.cc (DerivationGoal::killChild): Add conditional
call to 'worker.childTerminated' for 'hook->pid'.
Fixes <https://bugs.gnu.org/37744>.
Reported at <https://www.openwall.com/lists/oss-security/2019/10/09/4>.
Based on Nix commit 5a303093dcae1e5ce9212616ef18f2ca51020b0d
by Eelco Dolstra <edolstra@gmail.com>.
* nix/libstore/local-store.cc (LocalStore::LocalStore): Set 'perUserDir'
to #o755 instead of #o1777.
(LocalStore::createUser): New function.
* nix/libstore/local-store.hh (LocalStore): Add it.
* nix/libstore/store-api.hh (StoreAPI): Add it.
* nix/nix-daemon/nix-daemon.cc (performOp): In 'wopSetOptions', add
condition to handle "user-name" property and honor it.
(processConnection): Add 'userId' parameter. Call 'store->createUser'
when userId is not -1.
* guix/profiles.scm (ensure-profile-directory): Note that this is now
handled by the daemon.
* guix/store.scm (current-user-name): New procedure.
(set-build-options): Add #:user-name parameter and pass it to the daemon.
* tests/guix-daemon.sh: Test the creation of 'profiles/per-user' when
listening on a TCP socket.
* tests/store.scm ("profiles/per-user exists and is not writable")
("profiles/per-user/$USER exists"): New tests.
This is a followup to 2e3e5d2198.
* build-aux/test-env.in: Remove mentions of 'NIX_ROOT_FINDER'.
* nix/libstore/gc.cc (LocalStore::collectGarbage): Adjust comment
accordingly.
Until now it was up to 'guix offload' to honor timeouts. Unfortunately
it would sometimes fail to do that, for example due to the libssh bug at
<https://bugs.libssh.org/T33>. With this change, 'guix offload' is
automatically killed by the daemon when one of the timeouts expires.
Thus, data transfers performed by 'guix offload' now count as part of
the timeouts, rather than just actual build time.
* nix/libstore/build.cc (DerivationGoal::tryBuildHook): Pass true as the
'respectTimeouts' argument to 'childStarted'.
The daemon had a mechanism that allows it to handle a list of
substituters and try them sequentially; this removes it.
* nix/scripts/substitute.in: Remove.
* nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove.
* config-daemon.ac: Don't output 'nix/scripts/substitute'.
* nix/libstore/build.cc (SubstitutionGoal)[subs, sub, hasSubstitute]:
Remove.
[tryNext]: Make private.
(SubstitutionGoal::SubstitutionGoal, SubstitutionGoal::init): Remove now
unneeded initializers.
(SubstitutionGoal::tryNext): Adjust to assume a single substituter: call
'amDone' upfront when we couldn't find substitutes.
(SubstitutionGoal::tryToRun): Adjust to run 'guix substitute' via
'settings.guixProgram'.
(SubstitutionGoal::finished): Call 'amDone(ecFailed)' upon failure
instead of setting 'state' to 'tryNext'.
* nix/libstore/globals.hh (Settings)[substituters]: Remove.
* nix/libstore/local-store.cc (LocalStore::~LocalStore): Adjust to
handle a single substituter.
(LocalStore::startSubstituter): Remove 'path' parameter. Adjust to
invoke 'settings.guixProgram'. Don't refer to 'run.program', which no
longer exists.
(LocalStore::querySubstitutablePaths): Adjust for 'runningSubstituters'
being a singleton instead of a list.
(LocalStore::querySubstitutablePathInfos): Likewise, and remove
'substituter' parameter.
* nix/libstore/local-store.hh (RunningSubstituter)[program]: Remove.
(LocalStore)[runningSubstituters]: Remove.
[runningSubstituter]: New field.
[querySubstitutablePathInfos]: Remove 'substituter' parameter.
[startSubstituter]: Remove 'substituter' parameter.
* nix/nix-daemon/guix-daemon.cc (main): Remove references to
'settings.substituters'.
* nix/nix-daemon/nix-daemon.cc (performOp): Ignore the user's
"build-use-substitutes" value when 'settings.useSubstitutes' is false.
Until now, something like "guix build sed -v1 --check" would not get a
'build-succeeded' event, which in turn meant that the spinner would not
be erased upon build completion.
* nix/libstore/build.cc (DerivationGoal::registerOutputs): When
'buildMode' is bmCheck and 'settings.printBuildTrace' emit a
"@ build-succeeded" trace upon success.
* tests/store.scm ("build-succeeded trace in check mode"): New test.
This makes it easier to run the uninstalled daemon.
* nix/local.mk (libstore_a_CPPFLAGS): Append "/guix" to
NIX_LIBEXEC_DIR.
* build-aux/pre-inst-env.in (NIX_LIBEXEC_DIR): Adjust comment.
* nix/libstore/builtins.cc (builtinDownload): Remove SUBDIR and its
use.
* nix/libstore/local-store.cc (runAuthenticationProgram): Ditto.
* nix/libstore/gc.cc (addAdditionalRoots): Remove "/guix" prefix.
* nix/nix-daemon/guix-daemon.cc (main): Ditto.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopSetOptions>: When
'isRemoteConnection' is true, set 'settings.keepFailed' to zero.
* doc/guix.texi (Common Build Options): Document this behavior.
That way it is handled in the same way as other helper scripts.
* nix/scripts/guix-authenticate.in: Rename to...
* nix/scripts/authenticate.in: ... this.
* config-daemon.ac: Adjust accordingly.
* nix/local.mk (libstore_a_CPPFLAGS): Remove -DOPENSSL_PATH.
(nodist_libexec_SCRIPTS): Remove.
(nodist_pkglibexec_SCRIPTS): New variable.
* nix/nix-daemon/guix-daemon.cc (main): Remove 'setenv' call for
"PATH".
* nix/libstore/local-store.cc (runAuthenticationProgram): New function.
(LocalStore::exportPath, LocalStore::importPath): Use it instead of
'runProgram' and OPENSSL_PATH.
This allows clients to tell whether output comes from the daemon or, if
it comes from a builder, from which builder it comes. The latter is
particularly useful when MAX-BUILD-JOBS > 1.
* nix/libstore/build.cc (DerivationGoal::tryBuildHook)
(DerivationGoal::startBuilder): Print the child's PID in "@ build-started"
traces.
(DerivationGoal::handleChildOutput): Define 'prefix', pass it to
'writeToStderr'.
* nix/libstore/globals.cc (Settings:Settings): Initialize
'multiplexedBuildOutput'.
(Settings::update): Likewise.
* nix/libstore/globals.hh (Settings)[multiplexedBuildOutput]: New field.
Update 'printBuildTrace' documentation.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0.163.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopSetOptions>: Special-case
"multiplexed-build-output" and remove "use-ssh-substituter".
* guix/store.scm (set-build-options): Add #:multiplexed-build-output?
and honor it.
(%protocol-version): Bump to #x163.
* tests/store.scm ("multiplexed-build-output"): New test.
fixlet
