This gives us a concrete place to put the new resolver code and
resolver-specific modules (`resolution.resolver`).
The reason for another level of hierarchy compared to other modules
is to allow us to move other modules here as they
become implementation details of the legacy resolver. Examples I
have in mind are: `req.req_set`, `req.req_install`,
`req.constructors`, and `operations.prepare`.
* Make ParsedLine record the type of line
* Split handle_line to allow passing arguments only where needed
* Remove unneeded attributes from ParsedRequirement
This makes the resolver interface simpler by returning a brand new
RequirementSet vs mutating the one that was input to the function, and
will let us specialize RequirementSet for the different use cases.
Previously we were writing a delete marker file which is checked in
InstallRequirement.remove_temporary_source which is only invoked if the
user did not pass --no-clean (and a PreviousBuildDirError was not
raised). Since our TempDirectory machinery now respects these conditions
we can just wrap our source directory in that instead of using this
ad-hoc mechanism for tracking our delete preference.
This will let us clean up a lot of dead code that only existed for this
use case.
As we introduce stricter metadata parsing, we will need to ensure that
the wheel files used in our tests are compliant (except in the specific
way that we're testing against).
Currently we have a large number of test cases relying on undocumented or
under-documented wheel files that have various inconsistencies
(incorrect name, missing fields) that are unrelated to the features
actually under test.
With a new wheel builder helper function, we will be able to replace all
of our instances of pre-built wheel test fixtures with dynamically-generated
files in code that are correct by default.
_should_cache is only called by _get_cache_dir.
In pip install mode, _get_cache_dir is never called when
check_binary_allowed returns False because in that case
should_build_for_install_command has returned False before
and the build was skipped.
In pip wheel mode, check_binary_allowed always returns True
(because it is not passed to the build function).
So _should_cache can use _always_true for check_binary_allowed.
*Alternative*
Alternatively, we could have passed check_binary_allowed
to build in pip wheel mode. The only difference is that wheels built
locally from *legacy* packages would then not be cached,
when pip wheel is used with --no-binary.
This gives us a global toggle that we can use to control whether
temporary directories get deleted from one place (ideally, in the
commands taking --no-clean).
By default, make_archive uses str paths on Python 2, which causes it to
skip files with unencodable names. By passing in a unicode base_dir
explicitly, it is smart enough to use unicode all the way down instead.
pytest (rather py.path.local) does not handle non-ASCII paths properly
on Windows with Python 2, but Python's builtin shutil.rmtree() does.
Co-authored-by: Pradyun Gedam <pradyunsg@gmail.com>
Previously, these were used when the user did not provide an explicit
ABI. Now this is handled internally in `packaging.tags` (by
`_cpython_abi` and `_generic_abi`).
We assume this function improves the status quo over the current
`_cpython_tags`, so we use it when no arguments need to be customized.
Since `packaging.tags` has its own tests and derives defaults
differently than pep425tags, we also remove the applicable tests that
were testing against the result of a plain call to
`pep425tags.get_supported()`.
We assume this function improves on our existing behavior, so use it
as-is. We will customize the arguments over the next few commits.
Since packaging.tags internally calculates platforms when not provided,
we skip the tests which patch functions assuming that manylinux
compatibility determination depends on them.
This is the standard type used by packaging.tags. Making this change
throughout the code lets us start switching over to using its
tag-generating functions in get_supported().
We also get rid of a test, since it was superseded by `__str__` in
packaging.tags.Tag.
This reduces the amount of code we have to manage.
interpreter_name is calculated differently, defaulting to the
long name of the interpreter rather than "cp", but that is more
conformant.
Separately test should_build_for_wheel_command
and should_buid_for_install_command.
The tests are simpler and this open the door
for reasoning about both functions independently.
We now extract all metadata files from the wheel directly into memory
and make them available to the wrapping pkg_resources.Distribution via
the DictMetadata introduced earlier.
pkg_resources.Distribution classes delegate to the IMetadataProvider
implementation provided at construction. This is the one we'll use for
adapting a ZipFile to pkg_resources.DistInfoDistribution.
In order to parse metadata from wheel files directly we want to reuse
parse_wheel. Moving it out helps avoid creating an unnecessary
dependence on operations.install.wheel.
Previously we were copying an existing wheel to a file with a
different distribution name. When using stricter metadata parsing this
would fail, so now we use a more conformant dummy wheel function.
shared_data avoids copying the entire data directory, so use it in cases
where we know pip won't have any opportunity to edit the data files
(where we're passing tmpdir for --find-links).
Any test using --find-links data.packages is potentially using several
packages. By copying specifically the packages we need, we can more
easily see the packages that each test depends on and avoid using the
`data` fixture.
First example of transitioning a directory-aware function to using a
zipfile directly. Since we will not need to maintain the unpacked dir
going forward, we don't need to worry about making wheel_dist_info_dir
"generic", just that the same tests pass for both cases at each commit.
To do this neatly we use pytest.fixture(params=[...]), which
generates a test for each param. Once we've transitioned the
necessary functions we only need to replace the fixture name and remove
the dead code.
Since zips don't typically contain directory entries, we want to
only operate on files. Adding files to the .dist-info tests means we
will be able to reuse them for both cases, while they coexist.
Since retrieval of the .dist-info dir already ensures that a
distribution is found, this reduces responsibility on wheel_metadata and
lets us remove a few tests already covered by the tests for
test_wheel_dist_info_dir_*.
This will let us re-use the wheel_metadata for other parts of
processing, and by parameterizing checks in terms of metadata we will be
able to substitute in metadata derived directly from the zip.
* Raise exception on exception in finding wheel dist
We plan to replace this code with direct extraction from a zip, so no
point catching anything more precise.
* Raise exception if no dist is found in wheel_version
* Catch file read errors when reading WHEEL
get_metadata delegates to the underlying implementation which tries
to locate and read the file, throwing an IOError (Python 2) or OSError
subclass on any errors.
Since the new explicit test checks the same case as brokenwheel in
test_wheel_version we remove the redundant test.
* Check for WHEEL decoding errors explicitly
This was the last error that could be thrown by get_metadata, so we can
also remove the catch-all except block.
* Move WHEEL parsing outside try...except
This API does not raise an exception, but returns any errors on the
message object itself. We are preserving the original behavior, and can
decide later whether to start warning or raising our own exception.
* Raise explicit error if Wheel-Version is missing
`email.message.Message.__getitem__` returns None on missing values, so
we have to check for ourselves explicitly.
* Raise explicit exception on failure to parse Wheel-Version
This is also the last exception that can be raised, so we remove
`except Exception`.
* Remove dead code
Since wheel_version never returns None, this exception will never be
raised.
* Edit subdirs of top-level instead of checking in each directory
Previously, we were checking whether the top of the relative path ended
with .data. Now, we do not recurse into those directories, so there's no
need to check every time.
* Store info_dir in separate variable
Instead of working with a list everywhere, we use the single info_dir.
* Separate variables for info_dir and the destination path
* Use destination .dist-info dir only when needed
By initially storing just the name of the folder we ensure our code is
agnostic to the destination, so it'll be easier to install from a zip
later.
* Use os.listdir instead of os.walk for wheel dir population
Since we only execute any code when basedir == '', we only need the
top-level directories.
* Inline data_dirs calculation
* Inline info_dirs calculation
Previously we were making unguarded calls to non-Windows-only APIs. Mypy
only automatically excludes these from platform-specific checks when
inside conditions.
Previously we were restricting to a single .dist-info directory anywhere
in the unpacked wheel directory. That was incorrect since only a
top-level .dist-info directory indicates a contained "package". Now we
limit our restriction to top-level .dist-info directories.
This aligns with the previous behavior that would have enforced the
found .dist-info directory starting with the name of the package.
We raise UnsupportedWheel because it looks better in output than the
AssertionError (which includes traceback).
This should make everything "just work" with respect to combinations of
PATH, sys.path, and multiple Python installs. Later we can add a warning
here to help guide users to better understanding.
* Convert Windows app data directory values to bytes on Python 2, so the
output type is consistent across platforms (pypa/pip#4000)
* Also look in ~/.config for user config on macOS (pypa/pip#4100)
* Remove pywin32 dependency, only use ctypes and winreg for directory
lookup on Windows (pypa/pip#2467)
* Always use os.path.join() instead of os.sep.join() so cross-platform
tests work as expected (pypa/pip#3275)
* Add option to silence warnings related to deprecation of Python versions
* Move skip_if_python2 and skip_if_not_python2 decorator declaratios to test/lib/__init__.py and use them in test_warning.py
* Add tests to ensure that python version deprecation warning is shown correctly and can be silenced by a flag.
* Add new test to ensure that --no-python-version-warning flag does nothing if python version is not 2
This and the next several changes will uncouple the tests from the
current implementation, allowing us to factor the actual file download
out of `unpack_file_url` and `unpack_http_url`.
This reverts commit bcad1b1cb5, reversing
changes made to f86490317a.
As discussed, we should rethink the interface of this command output as
part of larger CLI usability review. In the interim, the same
functionality can be achieved using straightforward shell commands.
In cases where there is not a clear scope, or where enforcing a scope
and passing a temp directory to callees creates unnecessary coupling
between components, this will let us tie the lifetime of temporary
directories to the lifetime of the application without using e.g.
atexit or finalizers.
This has the benefit of being easier to test and reason about.
When we factor out tests these will be needed in both sets, and it's
easier to refactor tests later if we avoid creating a dependency between
test files.
Pip 20 changes the cache key format to include the
interpreter name. To avoid invalidating all existing caches,
we continue using existing cache entries that were computed
with the legacy algorithm. This should not regress issue #3025
because wheel cached in such legacy entries should have
the python implementation tag set.
Make sure ``pip wheel`` never outputs pure python wheels with a
python implementation tag. Better fix/workaround for
`#3025 <https://github.com/pypa/pip/issues/3025>`_ by
using a per-implementation wheel cache instead of caching pure python
wheels with an implementation tag in their name.
Fixes#7296
This is a more appropriate place for the function, since it is more
related to tags than wheels, and will make it easier to refactor Wheel
into its own module.
Xavier Fernandez
sinscary
Pradyun Gedam
Chris Hunt
Anudit Nagar
Paul Moore
Bhavam Vidyarthi
Deepak Sharma
Stéphane Bidoul (ACSONE)
Tzu-ping Chung
Xavier Fernandez
victorvpaulo
Thomas Kluyver