Commit Graph

2858 Commits

Author SHA1 Message Date
Pradyun Gedam e51a027964
Merge pull request #8659 from uranusjr/fix-get-distribution-dot-in-name
Canonicalize name in check_if_exists
2020-08-02 07:44:24 +05:30
Nguyễn Gia Phong e48a0cb7cb Remove no-longer-used messages 2020-08-01 20:26:44 +07:00
Tzu-ping Chung c3e1a153fd Improve SVN version parser
SVN has multiple distributions on Windows, e.g. SlikSVN, CollabNet. Some
of them suffix the version with a "-{distro}" part, which causes the
previous implementation to fail.

This patch removes that final part and make the version logic work.
2020-07-31 06:38:43 +08:00
Tzu-ping Chung 1fd5098b24 Canonicalize name in check_if_exists
The previous implementation uses pkg_resources.get_distribution(), which
does not canonicalize the package name correctly, and fails when
combined with pip's own get_distribution(), which does canonicalize
names. This makes InstallRequirement.check_if_exists() only use pip's
own canonicalization logic so different package name forms are matched
as expected.
2020-07-30 22:04:51 +08:00
Chris Hunt 3f9b326c11 Provide a reasonable error on invalid scheme keys
Originally we would throw an `AttributeError` if a bad scheme key was
used. After refactoring we would throw a `KeyError`, which isn't much
better. Now we call out the wheel being processed, scheme key we didn't
recognize, and provide a list of the valid scheme keys. This would
likely be useful for people developing/testing the wheel.
2020-07-29 22:24:09 -04:00
Chris Hunt 864f0e0efa Explicitly handle incorrect .data paths during wheel install
Previously our wheel installation process allowed wheels which contained
non-conforming contents in a contained .data directory.

After the refactoring to enable direct-from-wheel installation, pip
throws an exception when encountering these wheels, but does not include
any helpful information to pinpoint the cause.

Now if we encounter such a wheel, we trace an error that includes the
name of the requirement we're trying to install, the path to the wheel
file, the path we didn't understand, and a hint about what we expect.
2020-07-29 18:12:16 -04:00
Pradyun Gedam 31299ee370
Merge pull request #8603 from tekumara/pip-list-ignore-require-venv 2020-07-29 09:28:58 +05:30
Pradyun Gedam a28081c28e
Bump for development 2020-07-29 07:53:18 +05:30
Pradyun Gedam 127acd8c9e
Bump for release 2020-07-29 07:53:17 +05:30
Pradyun Gedam a8edffda1c
Merge pull request #8594 from pradyunsg/improve-install-conflict-warning 2020-07-28 18:22:53 +05:30
Pradyun Gedam 9033824cbc
Python 2 *sigh* 2020-07-28 11:24:44 +05:30
Pradyun Gedam 1b2ae22e7b
Don't print that form link after the end of month. 2020-07-27 19:51:40 +05:30
Pradyun Gedam 4e4951066d
Reverse if statement's condition 2020-07-27 13:52:54 +05:30
Pradyun Gedam 8db354260a
Move the form link to make the linter happy 2020-07-27 13:49:21 +05:30
Pradyun Gedam efdb66ed16
Add messaging variation based on "new resolver" usage 2020-07-27 13:49:21 +05:30
Pradyun Gedam 42c62a08f7
Short circuit when there's nothing to report 2020-07-27 13:49:20 +05:30
Pradyun Gedam d77b5c234c
Refactor the logging calls into a dedicated loop 2020-07-27 13:49:11 +05:30
Pradyun Gedam 7056132f6f
Merge pull request #8601 from rouge8/use-feature-requirements-file 2020-07-27 13:19:51 +05:30
Nguyễn Gia Phong b795c9a7d6 Abstract away AbstractDistribution in higher-level resolver code 2020-07-26 17:13:04 +07:00
Tzu-ping Chung 3ff9ee151d Account for comes_from string 2020-07-26 15:59:42 +08:00
Stéphane Bidoul 38fe3c2f14
Percolate --use-feature from req file upwards
We explicitly propagate --use-feature options from req files upwards.
This is not strictly necessary for the option to be enabled, because
of the default value is a global list, but that implicit behaviour is
certainly accidental, so we make it explicit, with a test.
2020-07-25 11:14:04 +02:00
Pradyun Gedam 89a51a6fef
Merge pull request #8588 from McSinyx/fast-deps
Use lazy wheel to obtain dep info for new resolver
2020-07-24 09:01:56 +05:30
Pradyun Gedam 43485f5ab7
Merge pull request #8617 from pradyunsg/drop-final-20.2-deprecation
Reject setup.py projects that don't generate .egg-info
2020-07-24 08:54:16 +05:30
Pradyun Gedam 3c20d5bfbd
Reject setup.py projects that don't generate .egg-info
This finalizes a deprecation, since we've not recieved any reports from
users in responses to the deprecation.
2020-07-23 22:32:46 +05:30
Pradyun Gedam dcfea6ee8f
Really? Wow.
This is a fix for the sole failing test in the CI for these changes (in
tests/functional/test_debug.py::test_debug__library_versions). The
failure took me a fair bit of time to diagnose, but it looks like the
issue is that we're strictly comparing versions as strings. This is a
bad idea when they're not normalized.
2020-07-22 19:57:20 +05:30
Pradyun Gedam 4272aa7980
Upgrade toml to 0.10.1 2020-07-22 19:57:20 +05:30
Pradyun Gedam afa59d97f6
Upgrade six to 1.15.0 2020-07-22 19:57:19 +05:30
Pradyun Gedam 072b70b9bf
Upgrade urllib3 to 1.25.9 2020-07-22 19:57:19 +05:30
Pradyun Gedam fe7128c662
Upgrade idna to 2.10 2020-07-22 19:57:19 +05:30
Pradyun Gedam 2ac80f4336
Upgrade certifi to 2020.6.20 2020-07-22 19:57:19 +05:30
Pradyun Gedam 28aad200b3
Upgrade requests to 2.24.0 2020-07-22 19:57:19 +05:30
Pradyun Gedam 413968dd17
Upgrade packaging to 20.4 2020-07-22 19:57:19 +05:30
Pradyun Gedam e84d0c1813
Upgrade html5lib to 1.1
Also, drop the no-longer-necessary patch.
2020-07-22 19:57:18 +05:30
Pradyun Gedam 11a64cdae8
Upgrade distlib to 0.3.1 2020-07-22 19:57:18 +05:30
Pradyun Gedam b9b2c18735
Upgrade appdirs to 1.4.4 2020-07-22 19:57:18 +05:30
Nguyễn Gia Phong 5d15291274 Use lazy wheel to obtain dep info for new resolver 2020-07-21 16:00:34 +07:00
Nguyễn Gia Phong 2152a51b8c Give metadata consistency check its own method 2020-07-21 16:00:01 +07:00
Pradyun Gedam 982aac544e
Merge pull request #8584 from McSinyx/range-unsupported-exc 2020-07-21 13:53:20 +05:30
Oliver Mannion 95dfd8b5a7 Ignore require-virtualenv in `pip list` 2020-07-20 16:45:52 +10:00
Andy Freeland d6b0481c8c Add --use-feature to pip freeze requirements parsing 2020-07-19 02:35:50 -07:00
Andy Freeland 3eef588a03 Support '--use-feature' in requirements files
This patch adds support for `--use-feature` in requirements files
so that a project that wants all contributors using the same pip
features can specify it in the requirements file. For example, to ensure
a requirements file uses the new resolver:

```
--use-feature=2020-resolver
boto3
boto3==1.13.13
```

This is a new version of #8293.
2020-07-18 16:08:47 -07:00
Stéphane Bidoul d924b16b0d
Give mypy some love after rebase 2020-07-18 13:46:19 +02:00
Stéphane Bidoul fe5682627a
Quote 'setup.py install' when calling it legacy
We want to make it clear that it is the setup.py install command we consider
legacy, not setup.py itself.
2020-07-18 13:46:19 +02:00
Stéphane Bidoul 462d6ca590
Deprecate install fallback when bdist_wheel fails 2020-07-18 13:46:19 +02:00
Pradyun Gedam 6fa4a9a0a7 Clean up code style changes
Toward minimizing style changes in the overall PR diff, or toward
consistency with the future use of black (in cases where I wasn't sure
of a good way to minimize the diff).
2020-07-17 14:33:19 +07:00
Nguyễn Gia Phong f8b06a3906 Enable flake8-logging-format 2020-07-17 14:33:19 +07:00
Nguyễn Gia Phong cb8d81d135 Nitpick logging calls 2020-07-17 14:33:19 +07:00
Pradyun Gedam b419ca7317
Fix the link on new-resolver dependency conflicts 2020-07-17 06:03:01 +05:30
Pradyun Gedam 2a07a2444c
Merge pull request #8590 from pradyunsg/install-conflict-warning-move
Change when we warn about dependency conflicts during `pip install`
2020-07-17 05:35:53 +05:30
Pradyun Gedam 7ddbcc2e67
Return early for clarity 2020-07-17 03:25:53 +05:30
Pradyun Gedam eafbec5aa6
Move conflict warning to just-before success message
This is a much better location for these errors, since they're in a much
more visible spot. We've had reports in the past of users missing these
messages, and changing where we present these warnings should help
resolve that issue.

We do lose the ability for an advanced user to potentially see the
warning and abort installation before the conflicts are introduced, but
given that we don't even pause for input, I don't think that's a strong
argument and neither do I view this as necessary.
2020-07-16 13:15:39 +05:30
Pradyun Gedam de741fa0dd
Clearly note where code duplication exists
The duplication of this code isn't really that bad, but saying
"pip check" makes it ambigous which file is relevant. Changing to
reference the exact filename makes this clearer.
2020-07-16 13:14:35 +05:30
Pradyun Gedam 67cbd0ca18
Break up pip install's "conflict check" function
Making this into two functions allows for separating the "check"
and "print warnings" step in a follow up commit.
2020-07-16 13:14:29 +05:30
Chris Hunt af89994eb3
Merge pull request #8574 from chrahunt/refactor/simpler-exception-handling
Simplify wheel install entrypoint exception handling
2020-07-15 16:53:25 -04:00
Pradyun Gedam e3e916be8d
Add a dedicated type for check_install_conflicts 2020-07-16 01:38:36 +05:30
Pradyun Gedam fa2fceecf1
Merge pull request #8189 from gutsytechster/remove_raise_for_status 2020-07-15 18:06:27 +05:30
Pradyun Gedam 61d497120f
Merge pull request #8424 from jku/8288-deprecate-nonconforming-extras 2020-07-15 18:03:00 +05:30
Nguyễn Gia Phong db217992bd Use more descriptive exception when range requests are unsupported 2020-07-15 16:28:53 +07:00
Jussi Kukkonen 76b20d738e Deprecate requirements format "base>=1.0[extra]"
This requirements format does not conform to PEP-508. Currently the
extras specified like this work by accident (because _strip_extras()
also parses them). The version checks end up being done with a
misparsed version '1.0[extra]' -- this is not changed in this commit.

Add deprecation warning and fix the corresponding resolver test. Add a
command line test.

Note that we really only check that the Requirement has SpecifierSet
with a specifier that ends in a ']'. A valid version number cannot
contain ']' and no wheels currently on pypi have versions ending in ']'.
2020-07-13 12:33:50 +03:00
Chris Hunt 677b4e7f11 Remove redundant try ... except 2020-07-11 15:34:06 -04:00
Chris Hunt 145b7add7e Move error message construction into MissingCallableSuffix
Now, we are free to throw MissingCallableSuffix from anywhere and don't
have to worry about catching it in the middle of our processing.
2020-07-11 15:34:06 -04:00
Chris Hunt f48c44e203 Add req description to error in install_wheel
Moving this value up from `_install_wheel` means that we do not need to
pass `req_description` anymore. This will also let us move our
entrypoint error handling around without worrying about losing the
context from the previous message.
2020-07-11 15:34:06 -04:00
Nguyễn Gia Phong c5e19c01c4 Clean up Configuration.unset_value and nit __init__ 2020-07-11 15:56:34 +07:00
Chris Hunt 8bf5731b84
Merge pull request #8562 from chrahunt/extract-direct-from-zip-refactoring
Install directly from wheels, without unpacking into an intermediate directory
2020-07-10 21:33:16 -04:00
Chris Hunt 83078470a6
Merge pull request #8566 from deveshks/mypy-configuration
Add type annotations to pip._internal.configuration
2020-07-10 20:09:30 -04:00
Devesh Kumar Singh 4cc731c62b Add type annotations to pip._internal.configuration 2020-07-11 01:38:19 +05:30
Chris Hunt 15b7afbcad
Merge pull request #8266 from gutsytechster/add_logs_for_install
feat(): Add logs for pip environment when installing
2020-07-10 09:36:13 -04:00
Chris Hunt 46bd55f371
Merge pull request #8565 from deveshks/mypy-cache
Add type annotations to "pip._internal.cache"
2020-07-09 21:53:06 -04:00
Chris Hunt e9508591ce
Merge pull request #8556 from chrahunt/maint/fail-on-install-location-options
Disallow explicitly passing install-location-related arguments in --install-options
2020-07-09 21:25:20 -04:00
Chris Hunt d13ec25361 Prevent path traversal when installing wheels directly 2020-07-09 21:21:35 -04:00
Chris Hunt 4605b32c49 Don't unpack wheel before installing 2020-07-09 18:25:11 -04:00
Chris Hunt 483213a318 Drop unused wheeldir parameter 2020-07-09 18:25:11 -04:00
Chris Hunt df92f25079 Don't use wheeldir for pyc record paths 2020-07-09 18:25:11 -04:00
Chris Hunt 4bdb8bcd7e Extract files directly from wheel 2020-07-09 18:25:10 -04:00
Chris Hunt f9432790de Get list of files directly from wheel
One less dependency on the wheel being extracted.
2020-07-09 18:24:33 -04:00
Chris Hunt e0f95f12b6 Get non-root scheme files from paths, not disk
Now we rely solely on the list of RECORD-like paths derived from the
filesystem, and can easily trade out the implementation for one that
comes from the wheel file directly.
2020-07-09 18:24:30 -04:00
Chris Hunt 8221aac105 Get paths and make files separately
At the beginning of our wheel processing we are going to have the list
of contained files. By splitting this into its own function, and
deriving it from disk in the same way it will appear in the zip, we can
incrementally refactor our approach using the same interface that will
be available at that time.

We start with the root-scheme paths (that end up in lib_dir) first.
2020-07-09 18:23:17 -04:00
Chris Hunt d241f0d30a Add partition from itertools recipes
From https://docs.python.org/3/library/itertools.html,
adapted for Python 2 and with types added.

This will be used in the next commit.
2020-07-09 18:23:17 -04:00
Chris Hunt a3b977330a Separate RECORD path from source file path
When we start processing files directly from the wheel, all we will have
are the files with their zip path (which should match a `RECORD`
entry). Separating this from the source file path (used for copying)
and annotating it with our `RecordPath` type makes it clear what the
format of this public property is, and that it should match what is in
`RECORD`.
2020-07-09 18:23:17 -04:00
Chris Hunt 239accb1b6 Inline clobber 2020-07-09 18:23:17 -04:00
Chris Hunt f2239b5488 Combine processing of root- and data-scheme files
With this approach, we can add the rest of our generated files into the
same iterable and they can undergo the same processing.
2020-07-09 18:23:17 -04:00
Chris Hunt e8382871ad Filter files outside of file-finding function
Simplifying the file-finding function will make it easier to drive our
whole wheel installation from a single list of files later.
2020-07-09 18:23:17 -04:00
Chris Hunt f1e906d166 Move script fixing into separate class
This makes `clobber` much simpler, and aligns the interface of
root_scheme files and data_scheme files, so we can process them in the
same way.
2020-07-09 18:23:17 -04:00
Chris Hunt 50f6f8d69b Simplify return type for fix_script
We always pass a file path to this function, so assert as much. We want
the return type to be consistent so we can assign the result to
non-Optional types.
2020-07-09 18:23:17 -04:00
Devesh Kumar Singh dadac2ce03 Add type annotations to pip._internal.cache 2020-07-09 21:46:55 +05:30
gutsytechster 0ccbad8367
feat(): Add logs for pip environment when installing 2020-07-09 18:15:43 +05:30
Prashant Sharma 07e8712677
reformat(pip/_internal): Resolve lint errors 2020-07-09 15:15:01 +05:30
Chris Hunt bf45bd77be Extract "getting files" outside `clobber`
"getting files" is one of the places that requires files to be on disk.
By extracting this out of `clobber` we can make it simpler and then
trade it out for a zip-based implementation.
2020-07-08 21:43:40 -04:00
Chris Hunt aa8dd9cecc Add File class to represent a file to install
Hiding the file-specific implementation we currently use will let us
trade out the implementation for a zip-backed one later. We can also use
this interface to represent the other kinds of files that we have to
generate as part of wheel installation.

We use a Protocol instead of a base class because there's no need for
shared behavior right now, and using Protocol is less verbose.
2020-07-08 20:41:53 -04:00
Chris Hunt 6b26ac911a Derive parent directory from destination path
By removing this dependency of the "file installation" part of `clobber`
on the "file finding" part of `clobber`, we can more easily factor out
the "file installation" part.
2020-07-08 20:41:43 -04:00
Chris Hunt 64dd286d88 Don't unconditionally create destination directory
Dropping the top-level directory creation allows us to make the
processing completely dependent on files to be installed, and not on the
top-level directory they happen to be installed in.

We already create the parent directory in the loop below, so this call
should be redundant for files that get installed.
2020-07-08 17:31:59 -04:00
gutsytechster 8c267e6e39
feat(pip/_internal/*): Use custom raise_for_status method 2020-07-08 18:08:55 +05:30
Tzu-ping Chung a17e5e0a72 Refactor to apply Mypy strict-optional=True 2020-07-08 20:24:29 +08:00
Christopher Hunt 43ca0805ae
Merge pull request #8456 from McSinyx/lint-vendor-readme
Lint src/pip/_vendor/README.rst
2020-07-07 21:40:38 -04:00
Christopher Hunt b27956928f
Merge pull request #8553 from uranusjr/header-location-prefix
Header location prefix
2020-07-07 21:37:50 -04:00
Chris Hunt f878fcdeec Rename function that reject location install options
Since it no longer shows a deprecation warning, the previous name
doesn't fit.
2020-07-07 19:11:11 -04:00
Chris Hunt 89572a7d40 Throw CommandError on any location-related install options 2020-07-07 19:11:11 -04:00
Tzu-ping Chung b85d5026e3 Fix header calculation when prefix is given 2020-07-07 18:53:33 +08:00
Devesh Kumar Singh fb05730898 Align write_output types with logger.info 2020-07-07 15:23:05 +05:30
Devesh Kumar Singh 87249b9850 Assert target_temp_dir instead of if check 2020-07-07 15:06:27 +05:30