3
5
Fork 0
mirror of git://git.savannah.gnu.org/guix.git synced 2023-12-14 03:33:07 +01:00
Commit graph

27 commits

Author SHA1 Message Date
Ludovic Courtès
2718c29c3f
nar: Deduplicate files right as they are restored.
This avoids having to traverse and re-read the files that we have just
restored, thereby reducing I/O.

* guix/serialization.scm (dump-file): New procedure.
(restore-file): Add #:dump-file parameter and honor it.
* guix/store/deduplication.scm (tee, dump-file/deduplicate): New
procedures.
* guix/nar.scm (restore-one-item): Pass #:dump-file to 'restore-file'.
(finalize-store-file): Pass #:deduplicate? #f to 'register-items'.
* tests/nar.scm <top level>: Call 'setenv' to set "NIX_STORE".
2020-12-15 17:32:09 +01:00
Ludovic Courtès
ed7d02f7c1
serialization: 'restore-file' sets canonical timestamp and permissions.
* guix/serialization.scm (restore-file): Set the permissions and mtime
of FILE.
* guix/nar.scm (finalize-store-file): Pass #:reset-timestamps? #f to
'register-items'.
* tests/nar.scm (rm-rf): Add 'chmod' calls to ensure files are writable.
("write-file + restore-file with symlinks"): Ensure every file in OUTPUT
passes 'canonical-file?'.
* tests/guix-archive.sh: Run "chmod -R +w" before "rm -rf".
2020-12-15 17:32:09 +01:00
Ludovic Courtès
465d2cb286
serialization: 'fold-archive' notifies about directory processing completion.
* guix/serialization.scm (fold-archive): Call PROC with a
'directory-complete tag when done with a directory.
(restore-file): Handle it.
* guix/scripts/archive.scm (list-contents): Likewise.
* guix/scripts/challenge.scm (archive-contents): Likewise.
* tests/nar.scm ("write-file-tree + fold-archive"): Adjust accordingly.
2020-12-15 17:32:09 +01:00
Ludovic Courtès
12c1afcdbd
serialization: Add 'fold-archive'.
* guix/serialization.scm (read-contents): Remove.
(read-file-type, fold-archive): New procedures.
(restore-file): Rewrite in terms of 'fold-archive'.
* tests/nar.scm ("write-file-tree + fold-archive")
("write-file-tree + fold-archive, flat file"): New tests.
2019-12-12 17:56:57 +01:00
Ludovic Courtès
9fe3f11398
serialization: 'restore-file' errors out upon non-convertible file names.
Fixes <https://bugs.gnu.org/33603>.
Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>.

* guix/serialization.scm (port-conversion-strategy): New variable.
(restore-file): Parameterize it.
* tests/nar.scm ("restore-file with non-UTF8 locale"): New test.
2019-01-18 17:51:34 +01:00
Ludovic Courtès
f5a2724ae4
deduplication: Restore directory mtime and permissions after deduplication.
Fixes <https://bugs.gnu.org/33361>.

* guix/store/deduplication.scm (replace-with-link): Call 'set-file-time'
and 'chmod' after 'rename-file'.
* tests/nar.scm ("restore-file-set with directories (signed, valid)"):
New test.
2018-11-13 14:59:46 +01:00
Ludovic Courtès
8390869811
tests: Check file canonicalization for 'restore-file-set'.
* guix/tests.scm (canonical-file?): New procedure.
* tests/nar.scm ("restore-file-set (signed, valid)"): Check that every
item of FILES matches 'canonical-file?'.
2018-11-13 14:59:46 +01:00
Ludovic Courtès
3ed56ad09b
tests: Remove check for a feature that appeared in Guile 2.0.10.
* tests/nar.scm: Remove test for 'open-sha256-input-port'.
2018-11-13 14:59:45 +01:00
Ludovic Courtès
ca71942445
Switch to Guile-Gcrypt.
This removes (guix hash) and (guix pk-crypto), which now live as part of
Guile-Gcrypt (version 0.1.0.)

* guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm,
tests/hash.scm, tests/pk-crypto.scm: Remove.
* configure.ac: Test for Guile-Gcrypt.  Remove LIBGCRYPT and
LIBGCRYPT_LIBDIR assignments.
* m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove.
* README: Add Guile-Gcrypt to the dependencies; move libgcrypt as
"required unless --disable-daemon".
* doc/guix.texi (Requirements): Likewise.
* gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm,
guix/git.scm, guix/http-client.scm, guix/import/cpan.scm,
guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm,
guix/import/gnu.scm, guix/import/hackage.scm,
guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm,
guix/pki.scm, guix/scripts/archive.scm,
guix/scripts/authenticate.scm, guix/scripts/download.scm,
guix/scripts/hash.scm, guix/scripts/pack.scm,
guix/scripts/publish.scm, guix/scripts/refresh.scm,
guix/scripts/substitute.scm, guix/store.scm,
guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm,
tests/builders.scm, tests/challenge.scm, tests/cpan.scm,
tests/crate.scm, tests/derivations.scm, tests/gem.scm,
tests/nar.scm, tests/opam.scm, tests/pki.scm,
tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm,
tests/store.scm, tests/substitute.scm: Adjust imports.
* gnu/system/vm.scm: Likewise.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this.  Add GUILE-GCRYPT.
(expression->derivation-in-linux-vm)[config]: Remove.
(iso9660-image)[config]: Remove.
(qemu-image)[config]: Remove.
(system-docker-image)[config]: Remove.
* guix/scripts/pack.scm: Adjust imports.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this.  Add GUILE-GCRYPT.
(self-contained-tarball)[build]: Call 'make-config.scm' without
 #:libgcrypt argument.
(squashfs-image)[libgcrypt]: Remove.
[build]: Call 'make-config.scm' without #:libgcrypt.
(docker-image)[config, json]: Remove.
[build]: Add GUILE-GCRYPT to the extensions  Remove (guix config) from
the imported modules.
* guix/self.scm (specification->package): Remove "libgcrypt", add
"guile-gcrypt".
(compiled-guix): Remove #:libgcrypt.
[guile-gcrypt]: New variable.
[dependencies]: Add it.
[*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call.
Add #:extensions.
[*config*]: Remove #:libgcrypt from 'make-config.scm' call.
(%dependency-variables): Remove %libgcrypt.
(make-config.scm): Remove #:libgcrypt.
* build-aux/build-self.scm (guile-gcrypt): New variable.
(make-config.scm): Remove #:libgcrypt.
(build-program)[fake-gcrypt-hash]: New variable.
Add (gcrypt hash) to the imported modules.  Adjust load path
assignments.
* gnu/packages/package-management.scm (guix)[propagated-inputs]: Add
GUILE-GCRYPT.
[arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search
path.
2018-09-04 17:25:11 +02:00
Ludovic Courtès
b94b698d4e
serialization: Add 'write-file-tree'.
* guix/serialization.scm (write-contents-from-port): New procedure.
(write-contents): Write in terms of 'write-contents-from-port'.
(filter/sort-directory-entries, write-file-tree): New procedures.
(write-file): Rewrite in terms of 'write-file-tree'.
* tests/nar.scm ("write-file-tree + restore-file"): New test.
2018-07-19 11:48:04 +02:00
Ludovic Courtès
36626c556e
build: Require Guile >= 2.0.9.
* configure.ac: Bump requirement to 2.0.9.
* doc/guix.texi (Requirements): Adjust accordingly.
* README (Requirements): Likewise.
* build-aux/download.scm: Remove workaround for <http://bugs.gnu.org/13095>.
* guix/build/download.scm: Likewise.
(http-fetch)[post-2.0.7?]: Remove.  Remove conditional code for not
POST-2.0.7?.
* guix/http-client.scm:  Remove workaround for <http://bugs.gnu.org/13095>.
(http-fetch)[post-2.0.7?]: Remove.  Remove conditional code for not
POST-2.0.7?.
* guix/serialization.scm (read-latin1-string): Remove mention of 2.0.9.
* tests/nar.scm: Use (ice-9 control).
(let/ec): Remove.
2017-03-18 00:39:49 +01:00
Ludovic Courtès
6f38960055
hash: Close or flush sha256 output ports before calling their 'get' procedure.
Slightly different buffering of custom binary output ports in Guile 2.1.7
would lead tests missing 'close-port' or 'force-output' to fail.

* tests/hash.scm ("open-sha256-port, hello"): Add call to 'force-output'
before call to 'get'.
* tests/nar.scm ("write-file puts file in C locale collation order"):
Call 'close-port' before calling 'get-hash'.
2017-03-08 23:42:52 +01:00
Ludovic Courtès
fe585be9aa
serialization: Add #:select? parameter to 'write-file'.
* guix/serialization.scm (write-file): Add #:select? parameter and honor it.
* tests/nar.scm ("write-file #:select? + restore-file"): New test.
2016-06-12 23:55:22 +02:00
Mathieu Lirzin
a9edb211e7 build: Add a Guile custom test driver using SRFI-64.
Before that '.log' files for scheme tests were fragmented and not
included in test-suite.log.  This unifies the semantics of SRFI-64 API
with Automake test suite.

* build-aux/test-driver.scm: New file.
* Makefile.am (SCM_LOG_DRIVER, AM_SCM_LOG_DRIVER_FLAGS): New variables.
(SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
(AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
* test-env.in: Silence guix-daemon.
* doc/guix.texi (Running the Test Suite): Describe how to display the
detailed results.  Bug reports require only 'test-suite.log' file.
* tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
tests/challenge.scm, tests/cpan.scm, tests/cpio.scm, tests/cran.scm,
tests/cve.scm, tests/derivations.scm, tests/elpa.scm,
tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
tests/gnu-maintenance.scm, tests/grafts.scm, tests/graph.scm,
tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
tests/import-utils.scm, tests/lint.scm, tests/monads.scm, tests/nar.scm,
tests/packages.scm, tests/pk-crypto.scm, tests/pki.scm,
tests/profiles.scm, tests/publish.scm, tests/pypi.scm,
tests/records.scm, tests/scripts-build.scm, tests/scripts.scm,
tests/services.scm, tests/sets.scm, tests/size.scm, tests/snix.scm,
tests/store.scm, tests/substitute.scm, tests/syscalls.scm,
tests/system.scm, tests/ui.scm, tests/union.scm, tests/upstream.scm,
tests/utils.scm: Don't exit at the end of test groups.
* tests/containers.scm: Likewise.  Use 'test-skip' instead of exiting
with error code 77.
2016-04-03 14:19:09 +02:00
Ludovic Courtès
fe32241ad5 tests: Work around Btrfs handling of nlink for empty directories.
Fixes <http://bugs.gnu.org/21280>.
Reported by Leo Famulari <leo@famulari.name>.

* tests/nar.scm (file-tree-equal?): Use #t as the initial result.  Btrfs would
  set nlink to 1 instead of 2 for an empty directory, thereby leading
  'file-tree-equal?' to return #f.
2015-08-26 00:43:36 +02:00
Ludovic Courtès
8de3df72bc tests: Move 'file=?' to (guix tests).
* tests/nar.scm (file-tree-equal?)[file=?]: Move to...
* guix/tests.scm (file=?): ... here.  New procedure.
2015-06-24 18:05:03 +02:00
Ludovic Courtès
46b8aadbd6 serialization: Check for EOF and incomplete input conditions.
Fixes <http://bugs.gnu.org/19756>.
Reported by <sleep_walker@suse.cz>.

* guix/serialization.scm (currently-restored-file): New variable.
  (get-bytevector-n*): New procedure.
  (read-int, read-long-long, read-string, read-latin1-string,
  read-contents): Use it instead of 'get-bytevector-n'.
  (restore-file): Parameterize 'currently-restored-file' and set it.
* tests/nar.scm ("restore-file with incomplete input"): New test.
2015-02-07 23:16:55 +01:00
Ludovic Courtès
834ea02ad8 tests: When a store item is to be deleted, make it statistically unique.
Fixes <http://bugs.gnu.org/18935>.
Reported by Mark H Weaver <mhw@netris.org>.

Previously, if tests/nar.scm ran concurrently with tests/gexp.scm, the
store item "foo" containing the string "Hello, world!" could be
deleted (via 'delete-paths') from the tests/nar.scm daemon while the
tests/gexp.scm daemon would still consider it live.

* tests/nar.scm ("restore-file-set (missing signature)"):
  Use (random-text) rather than "Hello, world!", to avoid concurrent
  deletion of store item "foo" used in tests/gexp.scm.
2014-11-04 21:13:28 +01:00
Ludovic Courtès
0363991a25 Break module cycle involving (guix store) and (guix ui).
Before, there was a cycle along the lines of:

  (guix store) -> (guix nar) -> (guix ui) -> (guix store)

This caused problems, as discussed at:

  http://lists.gnu.org/archive/html/guix-devel/2014-10/msg00109.html

This patch removes cycles in the (guix ...) modules.

* guix/nar.scm (&nar-error, &nar-read-error, dump, write-contents,
  read-contents, %archive-version-1, write-file, restore-file): Move to...
* guix/serialization.scm: ... here.
* guix/store.scm: Remove dependency on (guix nar).
* guix/scripts/hash.scm, guix/scripts/offload.scm,
  guix/scripts/substitute-binary.scm, tests/nar.scm, tests/store.scm,
  tests/substitute-binary.scm: Adjust accordingly.
2014-10-09 23:51:19 +02:00
Ludovic Courtès
c1bc358f29 Factorize test suite support in (guix tests).
* guix/tests.scm: New file.
* Makefile.am (noinst_DATA): New variable.
  (GOBJECTS): Add guix/tests.go.
* tests/builders.scm (%store): Use 'open-connection-for-tests'
  from (guix tests).
* tests/derivations.scm: Likewise.
* tests/monads.scm: Likewise.
* tests/packages.scm: Likewise.
* tests/profiles.scm: Likewise.
* tests/union.scm: Likewise.
* tests/gexp.scm: Likewise.
  (guile-for-build): Remove.  Use (%guile-for-build) instead.
* tests/nar.scm (make-random-bytevector, %seed, random-text): Remove.
  (populate-file): Change 'make-random-bytevector' to 'random-bytevector'.
  Use (guix tests).
* tests/store.scm (%seed, random-text): Remove.
  Use (guix tests).
2014-08-23 22:33:04 +02:00
Ludovic Courtès
96c7448f37 nar: Produce archives with files sorted in C collation order.
* guix/nar.scm (write-file) <directory>: Pass 'string<?' as the second
  argument to 'scandir'.
* tests/nar.scm ("write-file puts file in C locale collation order"):
  New test.
2014-02-21 23:49:53 +01:00
Ludovic Courtès
a93e91ff48 nar: 'write-file' can write to non-file ports.
* guix/nar.scm (write-contents): Use 'sendfile' only when P is a file
  port.
* tests/nar.scm ("write-file supports non-file output ports"): New test.
2014-02-21 23:49:52 +01:00
Ludovic Courtès
cd4027fa47 nar: Add 'restore-file-set', for use by build hooks.
* guix/nar.scm (&nar-invalid-hash-error, &nar-signature-error): New
  condition types.
  (&nar-error): Add 'file' and 'port' fields.
  (&nar-read-error): Remove 'port' and 'file' fields.
  (lock-store-file, unlock-store-file, finalize-store-file,
  temporary-store-directory, restore-file-set): New procedures.
* tests/nar.scm (%seed): New variable.
  (random-text): New procedure.
  ("restore-file-set (signed, valid)", "restore-file-set (missing
  signature)", "restore-file-set (corrupt)"): New tests.
* po/Makevars (XGETTEXT_OPTIONS): Add '--keyword=message'.nar fixes
* po/POTFILES.in: Add guix/nar.scm.
2014-01-24 00:01:50 +01:00
Ludovic Courtès
410e38f814 tests: Fix out-of-source builds.
* tests/nar.scm (%test-dir): Compute relative to `pre-inst-env', which
  is in $top_builddir.
2013-05-11 15:43:33 +02:00
Ludovic Courtès
77ffd691bf tests: Remove temporary directory created by nar.scm.
* tests/nar.scm ("write-file + restore-file with symlinks"): Add (rm-rf
  output).
2013-04-12 18:40:22 +02:00
Ludovic Courtès
8f3114b7a4 nar: Add support for symlinks.
* guix/nar.scm (write-file): Add case for type `symlink'.
  (restore-file): Likewise.
* tests/nar.scm (random-file-size, make-file-tree, delete-file-tree,
  with-file-tree, file-tree-equal?, make-random-bytevector,
  populate-file): New procedures.
  (%test-dir): New variable.
  ("write-file + restore-file"): Use `%test-dir' and `file-tree-equal?'.
  ("write-file + restore-file with symlinks"): New test.
2013-04-12 14:42:56 +02:00
Ludovic Courtès
53c63ee937 nar: Implement restoration from Nar.
* guix/nar.scm (&nar-error, &nar-read-error): New condition types.
  (dump): New procedure.
  (write-contents)[dump]: Remove.  Use the one above instead.
  (read-contents, write-file, restore-file): New procedures.
  (%archive-version-1): New variable.
2013-04-08 23:21:38 +02:00