McSinyx/pip
1
1
Fork 0
mirror of https://github.com/pypa/pip synced 2023-12-13 21:30:23 +01:00
Code Issues Releases Activity

Add topic guide: Repeatable Installs

Browse source
This commit is contained in:
Pradyun Gedam 2021-06-25 11:58:02 +01:00
parent 4561b1f182
commit b9d27d6d28
4 changed files with 104 additions and 84 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/html/cli/pip_install.rst
View file

@ -680,9 +680,9 @@ of having the wheel cache disabled is thus extra build time for sdists, and
this can be solved by making sure pre-built wheels are available from the index
server.
Hash-checking mode also works with :ref:`pip download` and :ref:`pip wheel`. A
:ref:`comparison of hash-checking mode with other repeatability strategies
<Repeatability>` is available in the User Guide.
Hash-checking mode also works with :ref:`pip download` and :ref:`pip wheel`.
See :doc:`../topics/repeatable-installs` for a comparison of hash-checking mode
with other repeatability strategies.
.. warning::

1
docs/html/topics/index.md
View file

@ -13,4 +13,5 @@ This section of the documentation is currently being fleshed out. See
authentication
caching
configuration
repeatable-installs
```

98
docs/html/topics/repeatable-installs.md Normal file
View file

@ -0,0 +1,98 @@
# Repeatable Installs
pip can be used to achieve various levels of repeatable environments. This page
walks through increasingly stricter definitions of what "repeatable" means.
## Pinning the package versions
Pinning package versions of your dependencies in the requirements file
protects you from bugs or incompatibilities in newly released versions:
```
SomePackage == 1.2.3
DependencyOfSomePackage == 4.5.6
```
```{note}
Pinning refers to using the `==` operator to require the package to be a
specific version.
```
A requirements file, containing pinned package versions can be generated using
{ref}`pip freeze`. This would not only the top-level packages, but also all of
their transitive dependencies. Performing the installation using
{ref}`--no-deps <install_--no-deps>` would provide an extra dose of insurance
against installing anything not explicitly listed.
This strategy is easy to implement and works across OSes and architectures.
However, it trusts the locations you're fetching the packages from (like PyPI)
and the certificate authority chain. It also relies on those locations not
allowing packages to change without a version increase. (PyPI does protect
against this.)
## Hash-checking
Beyond pinning version numbers, you can add hashes against which to verify
downloaded packages:
```none
FooProject == 1.2 --hash=sha256:2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
```
This protects against a compromise of PyPI or the HTTPS certificate chain. It
also guards against a package changing without its version number changing (on
indexes that allow this). This approach is a good fit for automated server
deployments.
Hash-checking mode is a labour-saving alternative to running a private index
server containing approved packages: it removes the need to upload packages,
maintain ACLs, and keep an audit trail (which a VCS gives you on the
requirements file for free). It can also substitute for a vendored library,
providing easier upgrades and less VCS noise. It does not, of course,
provide the availability benefits of a private index or a vendored library.
[pip-tools] is a package that builds upon pip, and provides a good workflow for
managing and generating requirements files.
[pip-tools]: https://github.com/jazzband/pip-tools#readme
## Using a wheelhouse (AKA Installation Bundles)
{ref}`pip wheel` can be used to generate and package all of a project's
dependencies, with all the compilation performed, into a single directory that
can be converted into a single archive. This archive then allows installation
when index servers are unavailable and avoids time-consuming recompilation.
````{admonition} Example
Creating the bundle, on a modern Unix system:
```
$ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX)
$ python -m pip wheel -r requirements.txt --wheel-dir=$tempdir
$ cwd=`pwd`
$ (cd "$tempdir"; tar -cjvf "$cwd/bundled.tar.bz2" *)
```
Installing from the bundle, on a modern Unix system:
```
$ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX)
$ (cd $tempdir; tar -xvf /path/to/bundled.tar.bz2)
$ python -m pip install --force-reinstall --no-index --no-deps $tempdir/*
```
````
Note that such a wheelhouse contains compiled packages, which are typically
OS and architecture-specific, so these archives are not necessarily portable
across machines.
Hash-checking mode can also be used along with this method (since this uses a
requirements file as well), to ensure that future archives are built with
identical packages.
```{warning}
Beware of the `setup_requires` keyword arg in {file}`setup.py`. The (rare)
packages that use it will cause those dependencies to be downloaded by
setuptools directly, skipping pip's protections. If you need to use such a
package, see {ref}`Controlling setup_requires <controlling-setup-requires>`.
```

83
docs/html/user_guide.rst
View file

@ -122,7 +122,7 @@ installed by pip in any particular order.
In practice, there are 4 common uses of Requirements files:
1. Requirements files are used to hold the result from :ref:`pip freeze` for the
purpose of achieving :ref:`repeatable installations <Repeatability>`. In
purpose of achieving :doc:`topics/repeatable-installs`. In
this case, your requirement file contains a pinned version of everything that
was installed when ``pip freeze`` was run.
@ -762,86 +762,7 @@ is the latest version:
Ensuring Repeatability
======================
pip can achieve various levels of repeatability:
Pinned Version Numbers
----------------------
Pinning the versions of your dependencies in the requirements file
protects you from bugs or incompatibilities in newly released versions::
SomePackage == 1.2.3
DependencyOfSomePackage == 4.5.6
Using :ref:`pip freeze` to generate the requirements file will ensure that not
only the top-level dependencies are included but their sub-dependencies as
well, and so on. Perform the installation using :ref:`--no-deps
<install_--no-deps>` for an extra dose of insurance against installing
anything not explicitly listed.
This strategy is easy to implement and works across OSes and architectures.
However, it trusts PyPI and the certificate authority chain. It
also relies on indices and find-links locations not allowing
packages to change without a version increase. (PyPI does protect
against this.)
Hash-checking Mode
------------------
Beyond pinning version numbers, you can add hashes against which to verify
downloaded packages::
FooProject == 1.2 --hash=sha256:2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
This protects against a compromise of PyPI or the HTTPS
certificate chain. It also guards against a package changing
without its version number changing (on indexes that allow this).
This approach is a good fit for automated server deployments.
Hash-checking mode is a labor-saving alternative to running a private index
server containing approved packages: it removes the need to upload packages,
maintain ACLs, and keep an audit trail (which a VCS gives you on the
requirements file for free). It can also substitute for a vendor library,
providing easier upgrades and less VCS noise. It does not, of course,
provide the availability benefits of a private index or a vendor library.
For more, see
:ref:`pip install\'s discussion of hash-checking mode <hash-checking mode>`.
.. _`Installation Bundle`:
Installation Bundles
--------------------
Using :ref:`pip wheel`, you can bundle up all of a project's dependencies, with
any compilation done, into a single archive. This allows installation when
index servers are unavailable and avoids time-consuming recompilation. Create
an archive like this::
$ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX)
$ python -m pip wheel -r requirements.txt --wheel-dir=$tempdir
$ cwd=`pwd`
$ (cd "$tempdir"; tar -cjvf "$cwd/bundled.tar.bz2" *)
You can then install from the archive like this::
$ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX)
$ (cd $tempdir; tar -xvf /path/to/bundled.tar.bz2)
$ python -m pip install --force-reinstall --ignore-installed --upgrade --no-index --no-deps $tempdir/*
Note that compiled packages are typically OS- and architecture-specific, so
these archives are not necessarily portable across machines.
Hash-checking mode can be used along with this method to ensure that future
archives are built with identical packages.
.. warning::
Finally, beware of the ``setup_requires`` keyword arg in :file:`setup.py`.
The (rare) packages that use it will cause those dependencies to be
downloaded by setuptools directly, skipping pip's protections. If you need
to use such a package, see :ref:`Controlling
setup_requires<controlling-setup-requires>`.
This is now covered in :doc:`../topics/repeatable-installs`.
.. _`Fixing conflicting dependencies`: