Originally we would throw an `AttributeError` if a bad scheme key was
used. After refactoring we would throw a `KeyError`, which isn't much
better. Now we call out the wheel being processed, scheme key we didn't
recognize, and provide a list of the valid scheme keys. This would
likely be useful for people developing/testing the wheel.
Previously our wheel installation process allowed wheels which contained
non-conforming contents in a contained .data directory.
After the refactoring to enable direct-from-wheel installation, pip
throws an exception when encountering these wheels, but does not include
any helpful information to pinpoint the cause.
Now if we encounter such a wheel, we trace an error that includes the
name of the requirement we're trying to install, the path to the wheel
file, the path we didn't understand, and a hint about what we expect.
Since this is the special part of this test. This gives us more
confidence that we're doing the right thing when removing the standalone
wheel file next.
This mainly deals with correctly recording the wheel content in the
RECORD metadata. This metadata file must be written in UTF-8, but the
actual files need to be installed to the filesystem, the encoding of
which is not (always) UTF-8. So we need to carefully handle file name
encoding/decoding when comparing RECORD entries to the actual file.
The fix here makes sure we always use the correct encoding by adding
strict type hints. The entries in RECORD is decoded/encoded with UTF-8
on the read/write boundaries to make sure we always deal with text
types. A type-hint-only type RecordPath is introduced to make sure this
is enforced (because Python 2 "helpfully" coerces str to unicode with
the wrong encoding).
We now extract all metadata files from the wheel directly into memory
and make them available to the wrapping pkg_resources.Distribution via
the DictMetadata introduced earlier.
shared_data avoids copying the entire data directory, so use it in cases
where we know pip won't have any opportunity to edit the data files
(where we're passing tmpdir for --find-links).
Any test using --find-links data.packages is potentially using several
packages. By copying specifically the packages we need, we can more
easily see the packages that each test depends on and avoid using the
`data` fixture.
* Edit subdirs of top-level instead of checking in each directory
Previously, we were checking whether the top of the relative path ended
with .data. Now, we do not recurse into those directories, so there's no
need to check every time.
* Store info_dir in separate variable
Instead of working with a list everywhere, we use the single info_dir.
* Separate variables for info_dir and the destination path
* Use destination .dist-info dir only when needed
By initially storing just the name of the folder we ensure our code is
agnostic to the destination, so it'll be easier to install from a zip
later.
* Use os.listdir instead of os.walk for wheel dir population
Since we only execute any code when basedir == '', we only need the
top-level directories.
* Inline data_dirs calculation
* Inline info_dirs calculation
Previously we were restricting to a single .dist-info directory anywhere
in the unpacked wheel directory. That was incorrect since only a
top-level .dist-info directory indicates a contained "package". Now we
limit our restriction to top-level .dist-info directories.
This aligns with the previous behavior that would have enforced the
found .dist-info directory starting with the name of the package.
We raise UnsupportedWheel because it looks better in output than the
AssertionError (which includes traceback).
Our isolation logic for venv isn't correct and that is causing these
tests to fail. The culprits for this are:
tests/lib/venv.py::VirtualEnvironment.user_site_packages
tests/lib/venv.py::VirtualEnvironment.sitecustomize
Both these together are supposed to create an environment to isolate the
tests. However, they were written for virtualenv and make assumptions
that are not true for environments created with venv. Until we can fix
VirtualEnvironment to properly isolate the test from the underlying test
environment when using venv, these tests will continue to fail.
This is blocking an important bugfix for users facing issues with since
pip is installing packages into `--user` when run in a venv, even when
`--user` isn't visible from that environment.
As a temporary band-aid for this problem, I'm skipping these tests to
unblock us from shipping the bugfix for the aforementioned issue.
The test isolation logic should be fixed to work for venv. Once such a
fix is made, this commit should be reverted.
- cleanup virtualenv creation code
- ensure all testing virtual environments use a recent version
of setuptools / wheel, making it easier to switch to custom
versions of those, as well as reducing network accesses
- reduce size of testing virtual environment, slightly speeding
up the testsuite
Without sorting, the 'installed' hash had entries in random order
that caused output to differ for every run.
See https://reproducible-builds.org/ for why this matters.
Sorting all entries to make testing easier.
This essentially allows me to do an overall check general check by running the tests using pytest's `-k basic` syntax. Given that I like running tests often and that, in general, I make typos more often than changes that break core functionality, I think this will reduce cycle times for me.
Chris Hunt
Stéphane Bidoul
Tzu-ping Chung
gutsytechster
Paul Moore
Devesh Kumar Singh
Pradyun Gedam
Chris Jerdonek
Albert Tugushev
Benoit Pierre
Bernhard M. Wiedemann
Xavier Fernandez
Pradyun S. Gedam
fiber-space
Bruno Oliveira
Donald Stufft
Jeff Barber
Nick Coghlan
Domen Kožar
Robert Collins