This tremendously helps understand why a package is being fetched and
can help investigate and fix dependency resolver backtracking issues
when incoherent constraints/package sets are provided or when new
versions of a package trigger a completely different backtracking
strategy, leading to very hard to debug situations.
The PEP 668 expects an override mechanism to ease the transition.
This provides an override.
---------
Co-authored-by: Pradyun Gedam <pradyunsg@gmail.com>
Refactored `_get_index_url()` to get integration tests for the subprocess backend working.
Keyring support via the 'subprocess' provider can only retrieve a password, not a username-password combo. The username therefor MUST come from the URL.
If the URL obtained from the index does not contain a username then the username from a matching index is used. `_get_index_url()` does that matching.
The problem this refactoring solves is that the URL where a wheel or sdist can be downloaded from does not always start with the index url. Azure DevOps Artifacts Feeds are an example since it replaces the friendly name of the Feed with the GUID of the Feed. Causing `url.startswith(prefix)` to evaluate as `False`.
The new behaviour is to return the index which matches the netloc and has the longest common prefix of the `path` property of the value returned by `urllib.parse.urlsplit()`. The behaviour for resolving ties is unspecified.
The fix merged in 3b60e36289 also fixed
the wheel cache; this cherry-picks the test and news from the other pull
request to validate and document the fact.
Commit bad03ef931 introduced the new
link_hash attribute that holds the link's hash info, but that attribute
does the same thing as _hashes, and some existing usages still populate
that old attribute. Since the plural variant covers more use cases (a
file can be hashed with multiple algorithms), we restore the old logic
that uses _hashes before the commit, and consolidate link_hash back into
that attribute.
Windows does not implement LC_MESSAGES, and since PEP 668 is mainly
designed for Linux distributions, we simply take the easier way out
until someone wants an equivalent on Windows.
This turns invalid egg fragments into a soft error, with a scheduled
deprecation period of two releases.
Signed-off-by: William Woodruff <william@trailofbits.com>
This exercises our expectation that egg fragments don't include version
specifiers and are evaluated eagerly.
Signed-off-by: William Woodruff <william@trailofbits.com>
Previously, the special case to generate 'pip' and 'easy_install' entry
points with the correct Python version (e.g. 'pip3.9' on Python 3.9)
only accounted for single-digit version segments, and did not work
correctly on Python 3.10 and up. This was missed when Python 3.10 was
released because we (accidentally) generated wheels that did not need
any such replacements, but was exposed in CPython 3.11.0 since it
bundled pip 22.3 generated against Python 3.10.
The modern virtual environment structure does not allow us to enable
"fake user site" while disabling the global site, so we need to do more
fine-grained configuration to correctly set up test environments for
each test case.
With this done, we can also properly support the stdlib venv ad the test
environment backend, since it basically works identically with modern
virtualenv. The incompatible_with_test_venv is thus removed.
The old INITools tests rely on setup.py, which relies on distutils and
generates a ton of issues. Build fake wheels directly to avoid dealing
with them.
Git 2.38.1 patched CVE-2022-39253 by disaling automated fetch against a
file: repository. This breaks git submodule, which is used by a pip
test. Information on how projects relying on automated fetch should
configure git correctly after this change is lacking, so the test is
disabled for now until someone can come up with a better solution.
The tests still don't run without distutils
because they require virtualenv < 20 (and virtualenv 16 uses distutils),
but at least they don't import distutils directly now.
Fixes https://github.com/pypa/pip/issues/11521
shlex.split, used to split options in requirements.txt files, might
raise a ValueError when the input string is not well formed. Catch the
ValueError and re-raise it as a RequirementsFileParseError instead.
The new functionality serves a use case which was not previously possible with pip configuration files, namely the situation where you have a base Python installation and want to influence the pip configuration for all derivative virtual environments *without* changing the config for all other environments on a machine (global), or for all other environment run by the same user (user). Concretely, this could be used for a centrally managed network mounted filesystem based Python installation, from which multiple users can build virtual environments and inside which a specific pip configuration is needed (e.g. an index URL).
use site.getsitepackages() where available
instead of just purelib/platlib,
which is often insufficient on e.g. System Pythons for Debian/macOS
handle virtualenv < 20 overwriting site.py without getsitepackages() by preserving current behavior.
See #11276
SearchScope was extended with an extra parameter to be able to pass-on the
value of no_index as we do with the other parameters. This allows us to respect
its value regardless of the order in which options are evaluated.
This applies to the new importlib.metadata backend. The legacy
pkg_resources backend already does this (albeit accidentally).
A package inside a wheel is not guaranteed to "work" when directly
imported, so we should not treat it as an installed distribution.
The code to do this already exists in `get_csv_rows_for_installed`, but it's
broken due to inconsistent usage of the `_fs_to_record_path` function. When
we build the dictionary of installed files, we call it with a base
directory, while when build the set of modified files, we call it without a
base directory. As a result, the values of `installed` do not match the
elements of `changed`, and `get_csv_rows_for_installed` fails to identify
the rows that should be updated.
Fix this by ensuring that `_fs_to_record_path` is always called with a base
directory. `_record_to_fs_path` also needs a a base directory parameter to
be able to transform the path back into an absolute path, so add one.
The approach it uses now doesn't work anymore due to 452d7da8.
The installation of `fake_dep` now succeeds whether or not `setuptools`
is installed in the test environment.
Use a different approach instead: try to import `pip` in the `setup.py`
script. If it succeeds, then we are not running in an isolated environment,
and therefore PEP 517 isn't being used.
To add this custom logic to `setup.py`, add a new argument to
`create_basic_sdist_for_package`. Note that to make this work, I had to
switch from f-strings to `str.format`, since the `dedent` has to happen
before formatting.
egg-info distributions may not have the Requires-Dist and Provides-Extra
fields in their metadata. For consistency and to provide an
unsurprising metadata property, we emulate it by reading requires.txt.
This resolves a behavioral different between Python 3.7 and 3.8+, where
a == comparison would incorrectly compare Path against str and cause
incorrect results.
The pip-specific Path implementation has been removed, and all its
usages replaced by pathlib.Path. The tmpdir and tmpdir_factory fixtures
are also removed, and all usages are replaced by tmp_path and
tmp_path_factory, which use pathlib.Path.
The pip() function now also accepts pathlib.Path so we don't need to put
str() everywhere. Path arguments are coerced with os.fspath() into str.
This was the only call-site for this method and, realistically, it is
highly coupled with the legacy resolver's dependency resolution
strategy/approach; so it makes sense for this code to live as part of
the resolver, rather than the container object the various resolvers.
There is no mechanism provided for build requirements to have extras.
It should be acceptable to enforce that any "optional" packages
that are supposed to be conditionally installed based on the presence of
an extra should not be installed in a build environment.
