Our isolation logic for venv isn't correct and that is causing these
tests to fail. The culprits for this are:
tests/lib/venv.py::VirtualEnvironment.user_site_packages
tests/lib/venv.py::VirtualEnvironment.sitecustomize
Both these together are supposed to create an environment to isolate the
tests. However, they were written for virtualenv and make assumptions
that are not true for environments created with venv. Until we can fix
VirtualEnvironment to properly isolate the test from the underlying test
environment when using venv, these tests will continue to fail.
This is blocking an important bugfix for users facing issues with since
pip is installing packages into `--user` when run in a venv, even when
`--user` isn't visible from that environment.
As a temporary band-aid for this problem, I'm skipping these tests to
unblock us from shipping the bugfix for the aforementioned issue.
The test isolation logic should be fixed to work for venv. Once such a
fix is made, this commit should be reverted.
________________ test_constraints_local_editable_install_pep518 ________________
...
----------------------------- Captured stdout call -----------------------------
Script result: python -m pip download setuptools wheel -d /tmp/pytest-of-mockbuild/pytest-0/test_constraints_local_editabl0/data/packages
return code: 1
-- stderr: --------------------
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fba7fdeb160>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fba7fdeb0b8>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fba7fdeb4e0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fba7fdeb6d8>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fba7fdeb860>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Could not find a version that satisfies the requirement setuptools (from versions: )
No matching distribution found for setuptools
_____________ test_pep517_wheels_are_not_confused_with_other_files _____________
...
-- stdout: --------------------
Processing /tmp/pytest-of-mockbuild/pytest-0/test_pep517_wheels_are_not_con0/data/src/withpyproject
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
Complete output from command /tmp/pytest-of-mockbuild/pytest-0/test_pep517_wheels_are_not_con0/workspace/venv/bin/python /builddir/build/BUILDROOT/python-pip-19.0.2-1.fc30.x86_64/usr/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pytest-of-mockbuild/pytest-0/test_pep517_wheels_are_not_con0/workspace/tmp/pip-build-env-3un7dqu3/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel:
Collecting setuptools
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fb7cf5a8898>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fb7cf5b9588>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fb7cf5b9470>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fb7cf5b9278>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fb7cf5b9208>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
Could not find a version that satisfies the requirement setuptools (from versions: )
No matching distribution found for setuptools
________________________ test_upgrade_argparse_shadowed ________________________
...
----------------------------- Captured stdout call -----------------------------
Script result: python -m pip install argparse==1.3
return code: 1
-- stderr: --------------------
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff07d6db240>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/argparse/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff07d6db358>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/argparse/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff07d6db400>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/argparse/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff07d6db518>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/argparse/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff07d6c7710>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/argparse/
Could not find a version that satisfies the requirement argparse==1.3 (from versions: )
No matching distribution found for argparse==1.3
- cleanup virtualenv creation code
- ensure all testing virtual environments use a recent version
of setuptools / wheel, making it easier to switch to custom
versions of those, as well as reducing network accesses
- reduce size of testing virtual environment, slightly speeding
up the testsuite
Mostly indentation complaints, plus an unused test parameter, fixed.
Leaves one new pylint complaint, which is about a test function naming
issue which affects every test.
The list of install options passed into the setup routine is mutable,
passed by reference, so adding items for "this package" to that list
mutates the options for all subsequent packages.
Isolate the lists before mutating them.
Includes a functional test, which has been confirmed to fail without
this fix.
Fixes#3763Fixes#4453Fixes#5089
The use of environment markers implies that the user expects the
packages to not be installed in some cases (eg depending on version
of Python), so the log output shouldn't be classed as a warning,
particularly since this results in it being sent to `stderr` rather
than `stdout`.
Fixes#4876.
* fix test not using temporary directory
* avoid creating a `cache_dir` directory in the source tree
* drop `data` fixture from tests that don't use it
* tests: fix pytest warning
Also, reduce the number of test cases because these tests take too long.
The issue is that the setup and teardown of the test environment takes a
long time, and in order to reuse the same test environment, we must
perform install/uninstall cycles to test different URL and path formats.
A better approach might be to generate `FSPkg` using a function that
writes the `setup.py` and `__init__.py` files to a new directory. The
test packages would be easy to generate and save the uninstall time.
For now, we reduce the number of URL/path formats tested, since the
previous test coverage was even more limited.
Tests are timing out and also not following requirements/install test
factorization. Split to conform to test organization and to prevent
test timeout.
`pip install -r reqs.txt` was failing when the requirements file includes
an unsupported wheel, regardless of whether it is conditionally removed
by a marker. This patch fixes that issue.
Additionally, this patch makes pip check local file wheels for
compatibility. Previously, a requirements file could include a path
to a valid wheel for any platform and pip would happily install it.
It seems pip distinguishes paths with .. or extra / for constraints.
For example, the following directories are considered different.
/path/to/dir
/path/to//dir
/path/to/dir/subdir/..
It can lead "Could not satisfy constraints for 'xxxx':
installation from path or url cannot be constrained to a version"
error, whose cause is not obvious to users. [1]
This commit tries to normalize the given directory name to avoid
the error.
[1] https://bugs.launchpad.net/devstack/+bug/1542545
Compare extras when checking if a requirement has already been
specified, and take a union of the extras before installation.
Co-Authored-By: Sachi King <nakato@nakato.io>
Closes#3046, #3189
This would occur when, for example, installing from a requirements file that references a certain hashed sdist, a common situation.
As of pip 7, pip always tries to build a wheel for each requirement (if one wasn't provided directly) and installs from that. The way this was implemented, InstallRequirement.link pointed to the cached wheel, which obviously had a different hash than the index-sourced archive, so spurious mismatch errors would result.
Now we no longer read from the wheel cache in hash-checking mode.
Make populate_link(), rather than the `link` setter, responsible for mapping InstallRequirement.link to a cached wheel. populate_link() isn't called until until prepare_files(). At that point, when we've examined all InstallRequirements and their potential --hash options, we know whether we should be requiring hashes and thus whether to use the wheel cache at all.
The only place that sets InstallRequirement.link other than InstallRequirement itself is pip.wheel, which does so long after hashes have been checked, when it's unpacking the wheel it just built, so it won't cause spurious hash mismatches.
When installing an editable from file:///path/to/file, pip currently does
not attempt to determine the name from #egg=NAME, just passing back
None. This causes constraints code to completely ignore this line
resulting in unexpected installation behaviour.
This patch makes '-e file:///path#egg=name' function similarly to
'file:///path#egg=name' and '-e git+URL#egg=name'. If #egg=name is not
defined, it returns None and the package becomes an unamed requirement,
which constraints will not parse but in the case of a requirement will
later be processed and determined.
Closes#3026
Currently if the local package you are trying to install is listed in
the constraint file it will silently fail to install the local package
while installing its dependancies.
This process was caused by the has_requirement returning true when a
constraint with that name was defined, resulting in the local package
not being processed.
Closes#2928
Fixes bug #2683
There are two changes here; one to fix the using-wheels codepath and one
to fix the no-wheels codepath. Two tests are introduced, one to test
each codepath.
If a single package is listed as a constraint; is a dependency of a
package being installed; *and* is already installed, we end up
processing it multiple times. This change adds a new "prepared" flag
which we set the first time the package is processed, to prevent
multiple handling.
Fixes bug #2888
This adds constraints files. Like requirements files constraints files
control what version of a package is installed, but unlike
requirements files this doesn't itself choose to install the package.
This allows things that aren't explicitly desired to be constrained if
and only if they are installed.
Using --install-options, --build-options, --global-options changes
the way that setup.py behaves, and isn't honoured by the wheel code.
The new wheel autobuilding code made this very obvious - disable
the use of wheels when these options are supplied.
"Downloading/unpacking" was misleading and confusing, especially when
the package is never found, as nothing was ever downloaded or unpacked
in this case.
The word "collecting" seems general enough to be accurate and it goes
well with the fact that later on pip will say, "Installing collected
packages".
Fixes: GH-376
Conflicts:
CHANGES.txt
tests/functional/test_install_reqs.py
tests/lib/__init__.py
tests/unit/test_req.py
Additional work - refactored tests to new style.
* Move virtualenv creation out of TestPipEnvironment
* Remove global state and force explicit use of TestPipEnvironment
instances
* Remove "backup" virtualenv copying and instead create new
virtual environments each time.
* Remove the monkeypatched "PyPICache" functionality
* Remove things that were not being used anymore and were dead
weight
* Remove sitecustomize support which was primarily used to
monkeypatch the "PyPICache" but was used in one or two other
tests.
* Better output with the bare asserts we use throughout the tests
* Function fixtures are pretty nice, especially as a way to
start a background server or create an isolated virtualenv
