The resolver collects previously known incompatibilites and sends them
to the provider. But previously the provider does not correctly exclude
the currently-installed candidate if it is present in that
incompatibility list, causing the resolver to enter a loop trying that
same candidate. This patch correctly applies incompat_ids when producing
an AlreadyInstalledCandidate and exclude it if its id() is in the set.
When a requirement is requested multiple times, some via a direct URL
("req @ URL") and some not but with extras ("req[extra] VERSION"), the
resolver previous could not correctly find "req[extra]" if "req" is
available in an index.
This additional logic makes the resolver, when encountering a
requirement with identifier "req[extra]", to also look for explicit
candidates listed under "req", and add them as found matches for
"req[extra]".
The typing module has been available since Python 3.5. Guarding the
import has been unnecessary since dropping Python 2.
Some guards remain to either:
- Avoid circular imports
- Importing objects that are also guarded by typing.TYPE_CHECKING
- Avoid mypy_extensions dependency
The stdlib module has been available since Python 3.5 and the
TYPE_CHECKING constant has been available since 3.5.2.
By using stdlib, this removes the need for pip to maintain its own
Python 2 typing compatibility shim.
Since 41a30089de, Candidate objects prepare their underlying
distribution eagerly on creation, so the error can be caught
deterministically to trigger backtracking.
This however has a negative side-effect. Since dist preparation involves
metadata validation, a remote distribution must be downloaded on
Candidate creation. This means that an sdist will be downloaded,
validated, and discarded (since its version is known to be incompatible)
during backtracking.
This commit moves version deduplication of candidates from indexes to
before the Candidate object is created, to avoid unneeded preparation.
Note that we still need another round of deduplication in FoundCandidates
to remove duplicated candidates when a distribution is already installed.
Use pyupgrade to convert simple string formatting to use f-string
syntax. pyupgrade is intentionally timid and will not create an f-string
if it would make the expression longer or if the substitution parameters
are anything but simple names or dotted names.
Since the "Requirement already satisfied" message is printed during
candidate preparation, instantiating the candidate multiple times result
in excessive logging during intensive backtracking. By caching the
already-installed candidates, each package is only prepared, and thus
only logged once.
This is done by catching InstallationError from the underlying
distribution preparation logic. There are three cases to catch:
1. Candidates from indexes. These are simply ignored since we can
potentially satisfy the requirement with other candidates.
2. Candidates from URLs with a dist name (PEP 508 or #egg=). A new
UnsatisfiableRequirement class is introduced to represent this; it is
like an ExplicitRequirement without an underlying candidate. As the
name suggests, an instance of this can never be satisfied, and will
cause eventual backtracking.
3. Candidates from URLs without a dist name. This is only possible for
top-level user requirements, and no recourse is possible for them. So
we error out eagerly.
The InstallationError raised during distribution preparation is cached
in the factory, like successfully prepared candidates, since we don't
want to repeatedly try to build a candidate if we already know it'd
fail. Plus pip's preparation logic also does not allow packages to be
built multiple times anyway.
Otherwise the test_install_distribution_union_with_versions test can end
up with either:
Cannot install localextras[bar] 0.0.2 and localextras[baz] 0.0.1 because these package versions have conflicting dependencies.
or
Cannot install localextras[baz] 0.0.2 and localextras[bar] 0.0.1 because these package versions have conflicting dependencies.
find_matches() is modified to return a special type that implements
the sequence protocol (instead of a plain list). This special sequence
type tries to use the installed candidate as the first element if
possible, and only access indexes when the installed candidate is
considered unsatisfactory.
This makes it more consistent with how error "summary" lines look.
eg:
IndexError: list index out of range
ModuleNotFoundError: No module named 'notamodule'
This check only applies to explicit requirements since we avoid
downloading the dist from finder altogether when there is a matching
installation (although the check wouldn’t change the behaviour in that
case anyway).
We can do this when we build the `ExplicitRequirement` instead, like how
we did for `SpecifierRequirement`, but that would require us to resolve
the direct requirement’s version eagerly, which I don’t want to.
The implemented approach checks the version only after resolution, at
which point the distribution is already built anyway and the operation
is cheap.
This mirrors the behavior in the legacy resolver. In the future we may
want to backtrack in this situation instead, but I haven't found a clean
way to do this. We may need to introduce an "empty" requirement class.
The `PackageFinder.target_python` interface is also not the most clean.
Maybe we should expose the target Python object instead. Not sure yet.
This specialized class is able to carry more context information than
the previous implementation (which reuses ExplicitRequirement). Error
reports can thus provide better messages by introspecting.
This rewrites how a SpecifierRequirement generates candidates, so it
* Always return an AlreadyInstalledCandidate (as long as the version
satisfies the specifier), even if PackageFinder does not return a
candidate for the same version.
* Always put the AlreadyInstalledCandidate last, so it's preferred over
LinkCandidate, preventing version changes if possible.
The candidate creation logic is further moved into the factory. The
factory would use pkg_resources.get_distribution() to find a matching
distribution for a givan InstallationCandidate. If found, the Candidate
would be created based on that found distribution, instead of the link.
--ignore-installed is implemented as to always use the link to create
candidates, even if an installed distribution is found.
Nguyễn Gia Phong
