McSinyx
/
pip
mirror of https://github.com/pypa/pip
1
1
Fork 0
Code Issues Releases Activity
12,952 Commits 3 Branches 142 Tags 93 MiB
Commit Graph

223 Commits

Author SHA1 Message Date
Christopher Hunt ab12706aaa
Merge pull request #7478 from chrahunt/refactor/make-functional-download-tests 
Make unpack_* unit tests into functional tests
 2019-12-17 07:55:29 +08:00
Chris Hunt 327c295554 Keep path to downloaded archive on InstallRequirement 
Now we'll be able to transition other parts of the code to use
pre-existing archives directly instead of relying on unpacked sources.
 2019-12-14 11:59:42 -05:00
Christopher Hunt fedde5fa2c
Do not cleanup archive download tempdir immediately (#7479) 
* Do not cleanup download tempdir immediately

The previous logic forced us to handle populating the download directory
in this function right next to the download and hash checking. By
extending the lifetime of the directory we can more easily separate the
code.

This also allows for additional optimizations later: by using metadata
from wheels directly instead of unpacking them, we can avoid extracting
wheels unnecessarily. Unpacked files can be easily 3x larger than the
archives themselves, so this should reduce disk utilization and general
IO significantly.
 2019-12-14 23:19:25 +08:00
Chris Hunt d30f406c37 Check file actually used against hashes 
This makes the behavior of this function easier to test, since we can
use a different file to distinguish the already-downloaded case from the
existing-file-hash-failed case.
 2019-12-13 20:30:45 -05:00
Chris Hunt 7dea92e5b3 Replace already_downloaded_path with existence check 
These statements are equivalent, so we exchange them. This will make it
easier to factor the download_dir concerns out of these functions.
 2019-12-13 00:46:44 -05:00
Chris Hunt 6b0a79501c Assert that target file does not exist 
_check_download_dir will only return a falsy value if either:

* the provided path does not exist
* the hash does not match - in which case the file is unlinked

so the file cannot exist at either of these points.
 2019-12-12 21:18:03 -05:00
Maxim Kurnikov b69560661b remove disallow_untyped_defs=False for pip._internal.distributions, pip._internal.operations.prepare 2019-12-12 21:35:37 +01:00
Chris Hunt cf743dd245 Remove unnecessary unlink in unpack_http_url 
The file is downloaded into a TempDirectory which will get cleaned up at
the end of the `with` block, so no need to explicitly unlink the file.
 2019-12-08 22:33:51 +01:00
Chris Hunt 24d2f1e719 Remove unused arguments 2019-12-05 20:29:43 -05:00
Chris Hunt 7db57478a2 Construct Downloader outside RequirementPreparer 
Reduces RequirementPreparer responsibilities, and will let us get rid of
some constructor arguments.
 2019-12-05 20:26:53 -05:00
Chris Hunt 78a221cf71 Move Downloader to network.download 
This will help us move Downloader construction out of
RequirementPreparer, reducing its concerns and making it easier to test
in isolation.
 2019-12-04 18:31:53 -05:00
Chris Hunt 3a2ff979de Move _http_get_download to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt e354b728dc Move _get_http_response_filename to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt 762e4a0817 Move parse_content_disposition to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt 32b0fc23ab Move sanitize_content_filename to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt 3fbc991f0c Move _prepare_download to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt 864f78bd96 Add network.download module 
This will be home to Dowloader, Download, and associated helper
functions. Since this is an abstraction over PipSession, it makes
sense to keep these functions in a separate module.

Also move a helper function here from operations.prepare.
 2019-12-04 18:30:27 -05:00
Chris Hunt aaabdfc2f7 Switch to early-return in _prepare_download 
Makes this function simpler by removing a local variable and reduces
overall lines of code.
 2019-11-29 10:26:37 -05:00
Chris Hunt d279807a0e Extract chunks into separate variable 2019-11-29 10:24:56 -05:00
Chris Hunt 46adcc89a6 Remove outdated comment in operations.prepare 
There are several reasons why we don't trace progress as indicated a few
lines above.
 2019-11-29 10:21:24 -05:00
Xavier Fernandez db1a2754bb Fix logging of cached response 
Cached responses often (always ?) provide a length meaning they were
never logged as such.
 2019-11-25 23:15:00 +01:00
Chris Hunt 59f22068d4 Move Downloader construction to RequirementPreparer.__init__ 
Just one step away from pulling this out of operations.prepare
altogether.
 2019-11-21 19:11:46 -05:00
Chris Hunt 3c8be92c32 Move Downloader out of unpack_url 
This simplifies the work done in the operations.prepare helper functions
and also opens up the door to remove session and progress_bar from
RequirementPreparer itself.
 2019-11-21 19:11:46 -05:00
Chris Hunt 19806574e5 Move Downloader out of unpack_http_url 
Reduces scope of session and progress_bar.
 2019-11-21 19:11:46 -05:00
Chris Hunt 4be0a5445a Move Downloader up out of _download_http_url 
Reduces exposure of session and progress_bar.
 2019-11-21 19:11:46 -05:00
Chris Hunt 3ce317890e Introduce Downloader for progress-showing downloads 2019-11-21 19:05:42 -05:00
Chris Hunt e4d2d34fe1 Represent download as its own type 
This abstraction will let us hide details about the actual downloading
and display of progress from most of operations.prepare.
 2019-11-21 19:04:28 -05:00
Chris Hunt e706af20fe Simplify writing file chunks 
A plain loop is easier to follow than chained generators consumed by
a helper function, and reduces the number of objects being passed around
just to download a file.
 2019-11-19 17:06:18 -05:00
Chris Hunt dca556fdaa Check hashes after download is complete 
Instead of computing hashes on-the-fly we do it after fully downloading
the file. This step will let us move hash checking to a higher-level
function without introducing a lot of complexity.
 2019-11-19 17:06:18 -05:00
Christopher Hunt 9ae2e6440d
Merge pull request #7365 from chrahunt/refactor/cleanup-prepare-2 
Cleanup operations.prepare download functions
 2019-11-20 06:05:25 +08:00
Ahilya 5051c74d5e Redact passwords from index-url during download 
Made changes in _download_url so that it relies on
redact_auth_from_url from misc to redact passwords.

Closes https://github.com/pypa/pip/issues/6783
 2019-11-17 20:54:32 +05:30
Chris Hunt 09b19dbbb8 Move actual session.get call into separate function 
By moving this call (and its long explanation) to a separate function,
diffs for later refactoring will be easier to review.
 2019-11-16 12:37:47 -05:00
Chris Hunt a9997127b9 Don't pass explicit step size to progress indicator 
Now the place we construct the progress indicator doesn't need to know
about our strategy for consuming the response, freeing us to extract the
chunk iterator construction into the caller.
 2019-11-16 12:37:47 -05:00
Chris Hunt 6e68c8d9ba Retrieve HTTP response size in separate function 
Also use `None` as the "no size available" indicator value, which is
already compatible with all of its current uses.
 2019-11-16 12:37:44 -05:00
Chris Hunt dd08330ac3 Construct download name in separate function 
Later, we will use this to make a dedicated object that represents
our download.
 2019-11-16 12:26:54 -05:00
Chris Hunt f5f1761aba Inline content-type retrieval 
This makes it easier to factor out the filename calculation.
 2019-11-16 12:26:14 -05:00
Pradyun Gedam ccd2ced9bc
Add disallow_any_generics = True to mypy configuration (#7110) 2019-11-14 20:46:21 +05:30
Maxim Kurnikov d1452ffabd add global disallow_any_generics=True 2019-11-12 21:08:48 +03:00
Chris Hunt 16ffd1a9c4 Normalize style 2019-11-11 22:36:46 -05:00
Chris Hunt 95375a5213 Make session a required parameter in operations.prepare 2019-11-11 22:34:52 -05:00
Pradyun Gedam 11b3fc2265
Drop use_user_site argument from prepare_editable_requirement 2019-11-10 13:51:36 +05:30
Pradyun Gedam c9606b86e1
Add use_user_site to RequirementPreparer 2019-11-10 13:51:29 +05:30
Pradyun Gedam 0612685e7e
Stop passing require_hashes to Resolver directly 2019-11-10 01:52:03 +05:30
Pradyun Gedam c0afe5c81d
Add require_hashes to RequirementPreparer 2019-11-09 12:13:29 +05:30
Chris Hunt deac2343dc Don't pass PackageFinder to Preparer from Resolver 
Preparer's overall responsibilities align more with having its own
reference to finder, which will help us remove it from resolver later.
 2019-11-05 20:57:14 -05:00
Christopher Hunt a0ff620240
Merge pull request #7291 from chrahunt/refactor/cleanup-prepare-1 
Cleanup operations.prepare
 2019-11-04 19:39:24 +08:00
Chris Hunt 54efa9dbdb Reorder operations.prepare._download_url 2019-11-03 16:26:37 -05:00
Chris Hunt b873df49b0 Factor from_cache check into separate function 2019-11-03 15:29:19 -05:00
Chris Hunt e8976af4e1 Show download size if available 
This simplifies the logic for displaying output.
 2019-11-03 15:25:59 -05:00
Chris Hunt 5da6abf60d Move resp_chunks to network.utils 
This generic piece of code only clutters operations.prepare, making it
harder to refactor.
 2019-11-03 15:16:42 -05:00
Chris Hunt 44cc3aeb0a Use session from RequirePreparer, not Resolver 2019-11-03 14:26:47 -05:00
Christopher Hunt de736c661e
Merge pull request #7178 from Desetude/fix/use-str-format 
Replace %-format with str#format
 2019-11-03 01:34:14 +08:00
Stéphane Bidoul (ACSONE) 18a58815a1
Refactor _get_used_vcs_backend 2019-11-01 12:53:54 +01:00
Pradyun Gedam 2db6f428bf
Update imports to {index -> index.package_finder} 2019-10-19 22:24:01 +05:30
Chris Hunt 6eb83c6d3a Move remaining functions in download to operations.prepare 
The only user of this module is operations.prepare.RequirementPreparer.
Moving the functionality to the single using module means that
refactoring will be easier (since all the mess is in one place). This
also removes a mis-named module from the top-level of the repository.
 2019-10-16 21:29:06 -04:00
Desetude 7e4e8eea40 Replace %-format with str#format 2019-10-12 17:08:10 +01:00
Maxim Kurnikov 3692097cca add per-file disallow_untyped_defs=False, and set it to True globally 2019-09-29 23:31:15 +02:00
Chris Hunt cf7ebdbbc2 Move PipSession to network.session 2019-09-27 01:02:52 -04:00
Chris Hunt 892b58f244 Move helper functions into Link 2019-09-22 00:08:10 -04:00
Pradyun Gedam ab6d550592
Remove link.is_artifact 2019-09-21 14:30:14 +05:30
Chris Hunt ee8cc0e648 Use unpack_file instead of unpack_file_url in wheel 2019-09-19 20:19:24 -04:00
Chris Hunt 59b0a79bde Add Link.file_path property 2019-09-19 20:14:10 -04:00
Chris Jerdonek 3303c11828 Remove index.py and req/constructors.py's dependence on download.py. 2019-09-13 09:57:04 -07:00
Chris Hunt 997c9b71b3 Move non-throwing code outside try/except. 2019-09-06 19:55:57 -04:00
Chris Hunt b72d9f1477 Move autodelete_unpacked to more logical place. 2019-09-06 19:55:57 -04:00
Chris Hunt a20a06cd34 Move unpack_url above irrelevant checks. 2019-09-06 19:55:57 -04:00
Chris Hunt aef01d25cc Remove unused argument from unpack_url. 2019-09-06 19:55:57 -04:00
Chris Hunt 45283a332f Move delete marker file creation from unpack_url to caller. 2019-09-06 19:55:38 -04:00
Pradyun Gedam a217f550e1
Remove duplicate line in comment. (#6965) 2019-09-05 13:03:24 +05:30
Christopher Hunt 0a246b98a9
Update src/pip/_internal/operations/prepare.py 
Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com>
 2019-09-04 02:54:23 -04:00
Chris Hunt efd43fd0fe Minor comment typo. 2019-09-02 19:37:33 -04:00
Chris Hunt b719cc0033 Fix indentation: fix indentation 2019-09-02 19:36:57 -04:00
Chris Hunt b1978bc608 Fix indentation: move expanduser to constructor 2019-09-02 19:36:00 -04:00
Chris Hunt d45a544b47 Remove duplicate line in comment. 2019-09-02 15:52:26 -04:00
Stéphane Bidoul (ACSONE) 2e1dfbef76 consolidate vcs link detection 2019-08-16 08:14:33 +02:00
Pradyun Gedam 476606425a
Add mypy's no-strict-optional configuration inline (#6750) 2019-08-04 21:59:42 +05:30
Pradyun Gedam f18ec526db
Unify calls to prepare_distribution_metadata to one callsite 2019-08-04 11:16:16 +05:30
Pradyun Gedam f377148f6d
Use mypy's inline configuration syntax for opt-outs 2019-07-30 12:39:51 +05:30
Pradyun Gedam b43c71cbc7
Simplify references to req.link 2019-07-29 20:30:55 +05:30
Pradyun Gedam ec725cb1a0
Populate InstallRequirement.link before preparing it 2019-07-29 20:29:34 +05:30
Pradyun Gedam 537b0fca12
Change isort multi_line_output to 3 2019-07-22 10:15:27 +05:30
Pradyun Gedam a0670a9c98
Rename AbstractDistribution.prep_for_dist() 
Use a better name prepare_distribution_metadata()
 2019-06-16 05:29:00 +05:30
Pradyun Gedam 534a102564
Rename make_abstract_dist 
Use a more descriptive name make_distribution_for_install_requirement()
 2019-06-16 05:23:16 +05:30
Pradyun Gedam d43e31802b
Add distributions sub-package for holding build logic 
This moves the DistAbstraction logic into a dedicated 'distributions'
sub-package, with separated modules for the concrete implementations of
various distribution handling logic.
 2019-06-14 23:27:42 +05:30
Pradyun Gedam d3987fe114
Sort imports 2019-06-14 23:27:42 +05:30
Chris Jerdonek 24688ee8e8 Revert "Require --no-use-pep517 if using editable mode with pyproject.toml." 
This reverts commit 71f506e71e.
 2019-04-26 20:19:05 -07:00
Chris Jerdonek 71f506e71e Require --no-use-pep517 if using editable mode with pyproject.toml. 2019-03-31 05:40:08 -07:00
Chris Jerdonek f47d012970 Refactor install_backend_dependencies() out from prep_for_dist(). 2019-03-31 01:11:30 -07:00
Albert Tugushev f0f31998d5 Use is_artifact prop instead of scheme checking 2019-02-23 11:55:37 +03:00
Maxim Kurnikov f77b8ca051 remove #noqa: F401 2019-02-22 14:17:07 +03:00
Pradyun Gedam b7a4d70b25
Don't instantiate NotImplementedError 2019-01-02 17:27:34 +05:30
Pradyun Gedam f9b66cacb3
Stop passing finder to DistAbstraction.dist 2019-01-02 17:27:34 +05:30
Pradyun Gedam 7222cb8fdb
Remove --process-dependency-links and related support code 2019-01-02 17:27:06 +05:30
Maxim Kurnikov 7ee388178a Add type annotations for pip._internal.operations (#6066) 
References https://github.com/pypa/pip/issues/4748, https://github.com/pypa/pip/pull/6038.
 2018-12-17 16:41:47 +05:30
Paul Moore f805ac15a7 Properly wrap all hook calls in our subprocess runner 2018-10-29 23:33:21 +01:00
Paul Moore 41b07c9fb5 Include backend-provided requirements in build environment 2018-10-29 23:33:19 +01:00
Paul Moore 8fbf78d407 Phase 2 - generate metadata using PEP 517 hook 2018-10-29 23:28:23 +01:00
Benoit Pierre 744b8cf96e improve build environment 
- check build requirements for conflicts
- better isolation (ignore system site packages)
- support 2 prefixes: a "normal" one, and an "overlay" one
  (with higher priority over "normal")
 2018-10-29 23:17:51 +01:00
Paul Moore efa5ad31e7 Refactor load_pyproject_toml into a standalone module 2018-08-05 14:52:46 +01:00
Paul Moore 4e018d76e2 Check for assumed requirements once the build environment is created 2018-08-02 20:08:04 +01:00
Paul Moore 8cdf6564fd Stop trying to lazy-load pyproject.toml, it isn't worth it. 2018-08-02 20:07:52 +01:00
Paul Moore c5a6843c08 Corrections to use_pep517 implementation 2018-07-31 16:21:20 +01:00
Paul Moore fc7bbf1b39 Make pyproject.toml values into properties of the requirement 2018-07-31 16:21:19 +01:00
Paul Moore 695be300bc Make args for prep_for_dist consistent 2018-07-30 09:42:11 +01:00
Pradyun Gedam 4fc697847a
Update imports for the moved compat module 2018-07-29 18:32:20 +05:30
Benoit Pierre 43b8ed4945 detect fork-bombs during build dependencies installs 2018-06-26 10:40:23 +02:00
Pradyun Gedam 0eb566fa82
Rework PEP 518 requirement logic 2018-06-19 16:27:07 +05:30
Benoit Pierre 7b4790d0ca cleanup PEP 518 implementation 
- remove unused `BuildEnvironment` parameter: `no_clean`
- move helper to install build requirement to the `BuildEnvironment` class
- simplify a little the 2 different code paths (with and without build isolation)
 2018-05-18 23:45:55 +02:00
Benoit Pierre ca4e2ad6ad improve handling of PEP 518 build requirements 
Offload more work to the underlying pip command used to install the
build requirements, so there's no need to duplicate code to handle
environment markers/extras. This is done by setting the correct options
from the finder passed in argument to `_install_build_reqs`.
 2018-05-18 23:45:55 +02:00
Benoit Pierre 02b678f557 fix PEP 518 support when pip is installed in the user site 
Do not use the build environment isolation during the build dependencies installation.
 2018-04-16 22:58:44 +02:00
Benoit Pierre 6da1d9e47c fix PEP 518 support 
* fix build environment handling when using PyPy
* do use the build environment for all build commands
* allow installing and building a wheel of a PEP 518 enabled
  package without prior installation of setuptools and/or wheels
* fix check for minimum supported requirements for PEP 518 support:
  - correctly handle complex requirements
  - both setuptools and wheels are needed
 2018-04-08 14:06:11 +02:00
Pradyun Gedam 3d7fbb31a9
Merge branch 'master' into resolver/warn-after-resolution 2018-03-27 20:10:56 +05:30
Paul Moore 61eda75a3f
Merge pull request #4989 from pradyunsg/misc/better-pep-518-message 
Improve message when setuptools is missing (PEP 518)
 2018-03-26 14:46:40 +01:00
Paul Moore 17e0d115e8 Fix for issue 5085 (--user set in config causes pip wheel to fail) 2018-03-25 18:22:18 +01:00
Pradyun Gedam 398fea844c
Improve message when setuptools is missing 2018-03-09 15:51:24 +05:30
Pradyun Gedam c08d4cc806
Add --no-build-isolation for disabling build isolation 2018-03-09 00:01:26 +05:30
Pradyun Gedam cdb8d71fed
Merge branch 'master' into resolver/warn-after-resolution 2018-03-01 14:49:00 +05:30
xoviat 163149fff3 Install build requirements before calling setup.py (#4799) 
Setup build environment before first invocation of setup.py and improve test to actually make check that build requirements are installed upon invocation.
 2018-03-01 14:20:06 +05:30
Pradyun Gedam 045e244b7e
Run pip check in pip install 
Also, add --no-warn-conflicts option to allow supressing the error messages generated.
 2018-01-24 18:16:18 +05:30
Pradyun Gedam de7d42feb9
Reduce classes with use_user_site attribute 2018-01-21 16:16:19 +05:30
Pradyun Gedam 353a13ddff
isort: Include trailing commas 2017-11-21 13:23:59 +05:30
Pradyun Gedam 8f68f387f0 Remove Circular dependency between Resolver and Preparer (#4636) 
RequirementPreparer no longer takes Resolver as argument in any of its methods and directly takes the reference of the values it actually needs.
 2017-10-22 22:34:14 +05:30
Donald Stufft 95bcf8c5f6 Move all internal APIs to pip._internal 2017-08-31 14:53:00 -04:00
Renamed from pip/operations/prepare.py (Browse further)