McSinyx
/
pip
mirror of https://github.com/pypa/pip
1
1
Fork 0
Code Issues Releases Activity
10,811 Commits 3 Branches 142 Tags 105 MiB
Commit Graph

188 Commits

Author SHA1 Message Date
Chris Hunt 4acc059cfd Remove unused argument 2020-02-04 21:55:42 -05:00
Chris Hunt c825c53f76 Rename unpack_{file,http}_url since they no longer unpack 2020-02-04 21:55:12 -05:00
Chris Hunt ada2f55f13 Inline file variable 2020-02-04 21:49:42 -05:00
Chris Hunt 135f2ac0ef Move unpacking into unpack_url 2020-02-04 21:49:42 -05:00
Chris Hunt 35a52c1397 Hold file in intermediate variable in unpack_url 2020-02-04 21:49:42 -05:00
Chris Hunt 71eaa4658f Return File from unpack_* functions 
Since we need both the file path and content type to unpack, and we want
to move unpacking out of the lower-level functions, return all the
information needed so it's easier to move the unpacking out.
 2020-02-04 21:49:42 -05:00
Chris Hunt 791725aad9 Move directory handling from unpack_file_url to unpack_url 2020-02-04 21:49:42 -05:00
Chris Hunt 395787bebc Inline link.file_path in unpack_file_url 2020-02-04 21:49:42 -05:00
Chris Hunt 062ccf9dba Use early-return style in unpack_url 
This will make it easier to move directory handling up to unpack_url.
 2020-02-04 21:49:42 -05:00
Chris Hunt 71e0ea19af Assert req.source_dir is None during linked requirement preparation 
req.source_dir is only set by:

1. `InstallRequirement.__init__`
2. `InstallRequirement.ensure_has_source_dir`

`InstallRequirement.__init__` is only called with source_dir for
editable requirements, for which we would not call
`RequirementPreparer.prepare_linked_requirement` (only
`prepare_editable_requirement`).

We will use this assertion for justifying later refactoring.
 2020-01-08 20:11:28 -05:00
Stéphane Bidoul (ACSONE) 68e49b9613 Remove redundant expanduser for download_dir in preparer 
Since
- download_dir is only set by the download command
- download_dir is
normalized at the beginning of the download command
- path normalization includes expanduser
Therefore expanduser in the preparer is redundant
 2020-01-01 18:54:08 +01:00
Stéphane Bidoul (ACSONE) a6d212383b
Remove redundant path normalization in preparer 
wheel_download_dir is normalized at the
beginning of the wheel command
 2019-12-31 05:32:42 +01:00
Chris Hunt ce9ddbb600 Move download file copying out of unpacking functions 
Now our "unpacking" functions aren't also for sometimes populating
the download directory.
 2019-12-26 11:12:17 -05:00
Chris Hunt 5c90145462 Switch conditions 2019-12-24 16:59:24 -05:00
Chris Hunt 554db9230d Move directory requirement download logging out of unpack_file_url 
One less use of `download_dir` in `unpack_file_url`, which will make it
easier to factor out.
 2019-12-24 16:56:24 -05:00
Christopher Hunt ab12706aaa
Merge pull request #7478 from chrahunt/refactor/make-functional-download-tests 
Make unpack_* unit tests into functional tests
 2019-12-17 07:55:29 +08:00
Chris Hunt 327c295554 Keep path to downloaded archive on InstallRequirement 
Now we'll be able to transition other parts of the code to use
pre-existing archives directly instead of relying on unpacked sources.
 2019-12-14 11:59:42 -05:00
Christopher Hunt fedde5fa2c
Do not cleanup archive download tempdir immediately (#7479) 
* Do not cleanup download tempdir immediately

The previous logic forced us to handle populating the download directory
in this function right next to the download and hash checking. By
extending the lifetime of the directory we can more easily separate the
code.

This also allows for additional optimizations later: by using metadata
from wheels directly instead of unpacking them, we can avoid extracting
wheels unnecessarily. Unpacked files can be easily 3x larger than the
archives themselves, so this should reduce disk utilization and general
IO significantly.
 2019-12-14 23:19:25 +08:00
Chris Hunt d30f406c37 Check file actually used against hashes 
This makes the behavior of this function easier to test, since we can
use a different file to distinguish the already-downloaded case from the
existing-file-hash-failed case.
 2019-12-13 20:30:45 -05:00
Chris Hunt 7dea92e5b3 Replace already_downloaded_path with existence check 
These statements are equivalent, so we exchange them. This will make it
easier to factor the download_dir concerns out of these functions.
 2019-12-13 00:46:44 -05:00
Chris Hunt 6b0a79501c Assert that target file does not exist 
_check_download_dir will only return a falsy value if either:

* the provided path does not exist
* the hash does not match - in which case the file is unlinked

so the file cannot exist at either of these points.
 2019-12-12 21:18:03 -05:00
Maxim Kurnikov b69560661b remove disallow_untyped_defs=False for pip._internal.distributions, pip._internal.operations.prepare 2019-12-12 21:35:37 +01:00
Chris Hunt cf743dd245 Remove unnecessary unlink in unpack_http_url 
The file is downloaded into a TempDirectory which will get cleaned up at
the end of the `with` block, so no need to explicitly unlink the file.
 2019-12-08 22:33:51 +01:00
Chris Hunt 24d2f1e719 Remove unused arguments 2019-12-05 20:29:43 -05:00
Chris Hunt 7db57478a2 Construct Downloader outside RequirementPreparer 
Reduces RequirementPreparer responsibilities, and will let us get rid of
some constructor arguments.
 2019-12-05 20:26:53 -05:00
Chris Hunt 78a221cf71 Move Downloader to network.download 
This will help us move Downloader construction out of
RequirementPreparer, reducing its concerns and making it easier to test
in isolation.
 2019-12-04 18:31:53 -05:00
Chris Hunt 3a2ff979de Move _http_get_download to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt e354b728dc Move _get_http_response_filename to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt 762e4a0817 Move parse_content_disposition to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt 32b0fc23ab Move sanitize_content_filename to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt 3fbc991f0c Move _prepare_download to network.download 2019-12-04 18:31:53 -05:00
Chris Hunt 864f78bd96 Add network.download module 
This will be home to Dowloader, Download, and associated helper
functions. Since this is an abstraction over PipSession, it makes
sense to keep these functions in a separate module.

Also move a helper function here from operations.prepare.
 2019-12-04 18:30:27 -05:00
Chris Hunt aaabdfc2f7 Switch to early-return in _prepare_download 
Makes this function simpler by removing a local variable and reduces
overall lines of code.
 2019-11-29 10:26:37 -05:00
Chris Hunt d279807a0e Extract chunks into separate variable 2019-11-29 10:24:56 -05:00
Chris Hunt 46adcc89a6 Remove outdated comment in operations.prepare 
There are several reasons why we don't trace progress as indicated a few
lines above.
 2019-11-29 10:21:24 -05:00
Xavier Fernandez db1a2754bb Fix logging of cached response 
Cached responses often (always ?) provide a length meaning they were
never logged as such.
 2019-11-25 23:15:00 +01:00
Chris Hunt 59f22068d4 Move Downloader construction to RequirementPreparer.__init__ 
Just one step away from pulling this out of operations.prepare
altogether.
 2019-11-21 19:11:46 -05:00
Chris Hunt 3c8be92c32 Move Downloader out of unpack_url 
This simplifies the work done in the operations.prepare helper functions
and also opens up the door to remove session and progress_bar from
RequirementPreparer itself.
 2019-11-21 19:11:46 -05:00
Chris Hunt 19806574e5 Move Downloader out of unpack_http_url 
Reduces scope of session and progress_bar.
 2019-11-21 19:11:46 -05:00
Chris Hunt 4be0a5445a Move Downloader up out of _download_http_url 
Reduces exposure of session and progress_bar.
 2019-11-21 19:11:46 -05:00
Chris Hunt 3ce317890e Introduce Downloader for progress-showing downloads 2019-11-21 19:05:42 -05:00
Chris Hunt e4d2d34fe1 Represent download as its own type 
This abstraction will let us hide details about the actual downloading
and display of progress from most of operations.prepare.
 2019-11-21 19:04:28 -05:00
Chris Hunt e706af20fe Simplify writing file chunks 
A plain loop is easier to follow than chained generators consumed by
a helper function, and reduces the number of objects being passed around
just to download a file.
 2019-11-19 17:06:18 -05:00
Chris Hunt dca556fdaa Check hashes after download is complete 
Instead of computing hashes on-the-fly we do it after fully downloading
the file. This step will let us move hash checking to a higher-level
function without introducing a lot of complexity.
 2019-11-19 17:06:18 -05:00
Christopher Hunt 9ae2e6440d
Merge pull request #7365 from chrahunt/refactor/cleanup-prepare-2 
Cleanup operations.prepare download functions
 2019-11-20 06:05:25 +08:00
Ahilya 5051c74d5e Redact passwords from index-url during download 
Made changes in _download_url so that it relies on
redact_auth_from_url from misc to redact passwords.

Closes https://github.com/pypa/pip/issues/6783
 2019-11-17 20:54:32 +05:30
Chris Hunt 09b19dbbb8 Move actual session.get call into separate function 
By moving this call (and its long explanation) to a separate function,
diffs for later refactoring will be easier to review.
 2019-11-16 12:37:47 -05:00
Chris Hunt a9997127b9 Don't pass explicit step size to progress indicator 
Now the place we construct the progress indicator doesn't need to know
about our strategy for consuming the response, freeing us to extract the
chunk iterator construction into the caller.
 2019-11-16 12:37:47 -05:00
Chris Hunt 6e68c8d9ba Retrieve HTTP response size in separate function 
Also use `None` as the "no size available" indicator value, which is
already compatible with all of its current uses.
 2019-11-16 12:37:44 -05:00
Chris Hunt dd08330ac3 Construct download name in separate function 
Later, we will use this to make a dedicated object that represents
our download.
 2019-11-16 12:26:54 -05:00
Chris Hunt f5f1761aba Inline content-type retrieval 
This makes it easier to factor out the filename calculation.
 2019-11-16 12:26:14 -05:00
Pradyun Gedam ccd2ced9bc
Add disallow_any_generics = True to mypy configuration (#7110) 2019-11-14 20:46:21 +05:30
Maxim Kurnikov d1452ffabd add global disallow_any_generics=True 2019-11-12 21:08:48 +03:00
Chris Hunt 16ffd1a9c4 Normalize style 2019-11-11 22:36:46 -05:00
Chris Hunt 95375a5213 Make session a required parameter in operations.prepare 2019-11-11 22:34:52 -05:00
Pradyun Gedam 11b3fc2265
Drop use_user_site argument from prepare_editable_requirement 2019-11-10 13:51:36 +05:30
Pradyun Gedam c9606b86e1
Add use_user_site to RequirementPreparer 2019-11-10 13:51:29 +05:30
Pradyun Gedam 0612685e7e
Stop passing require_hashes to Resolver directly 2019-11-10 01:52:03 +05:30
Pradyun Gedam c0afe5c81d
Add require_hashes to RequirementPreparer 2019-11-09 12:13:29 +05:30
Chris Hunt deac2343dc Don't pass PackageFinder to Preparer from Resolver 
Preparer's overall responsibilities align more with having its own
reference to finder, which will help us remove it from resolver later.
 2019-11-05 20:57:14 -05:00
Christopher Hunt a0ff620240
Merge pull request #7291 from chrahunt/refactor/cleanup-prepare-1 
Cleanup operations.prepare
 2019-11-04 19:39:24 +08:00
Chris Hunt 54efa9dbdb Reorder operations.prepare._download_url 2019-11-03 16:26:37 -05:00
Chris Hunt b873df49b0 Factor from_cache check into separate function 2019-11-03 15:29:19 -05:00
Chris Hunt e8976af4e1 Show download size if available 
This simplifies the logic for displaying output.
 2019-11-03 15:25:59 -05:00
Chris Hunt 5da6abf60d Move resp_chunks to network.utils 
This generic piece of code only clutters operations.prepare, making it
harder to refactor.
 2019-11-03 15:16:42 -05:00
Chris Hunt 44cc3aeb0a Use session from RequirePreparer, not Resolver 2019-11-03 14:26:47 -05:00
Christopher Hunt de736c661e
Merge pull request #7178 from Desetude/fix/use-str-format 
Replace %-format with str#format
 2019-11-03 01:34:14 +08:00
Stéphane Bidoul (ACSONE) 18a58815a1
Refactor _get_used_vcs_backend 2019-11-01 12:53:54 +01:00
Pradyun Gedam 2db6f428bf
Update imports to {index -> index.package_finder} 2019-10-19 22:24:01 +05:30
Chris Hunt 6eb83c6d3a Move remaining functions in download to operations.prepare 
The only user of this module is operations.prepare.RequirementPreparer.
Moving the functionality to the single using module means that
refactoring will be easier (since all the mess is in one place). This
also removes a mis-named module from the top-level of the repository.
 2019-10-16 21:29:06 -04:00
Desetude 7e4e8eea40 Replace %-format with str#format 2019-10-12 17:08:10 +01:00
Maxim Kurnikov 3692097cca add per-file disallow_untyped_defs=False, and set it to True globally 2019-09-29 23:31:15 +02:00
Chris Hunt cf7ebdbbc2 Move PipSession to network.session 2019-09-27 01:02:52 -04:00
Chris Hunt 892b58f244 Move helper functions into Link 2019-09-22 00:08:10 -04:00
Pradyun Gedam ab6d550592
Remove link.is_artifact 2019-09-21 14:30:14 +05:30
Chris Hunt ee8cc0e648 Use unpack_file instead of unpack_file_url in wheel 2019-09-19 20:19:24 -04:00
Chris Hunt 59b0a79bde Add Link.file_path property 2019-09-19 20:14:10 -04:00
Chris Jerdonek 3303c11828 Remove index.py and req/constructors.py's dependence on download.py. 2019-09-13 09:57:04 -07:00
Chris Hunt 997c9b71b3 Move non-throwing code outside try/except. 2019-09-06 19:55:57 -04:00
Chris Hunt b72d9f1477 Move autodelete_unpacked to more logical place. 2019-09-06 19:55:57 -04:00
Chris Hunt a20a06cd34 Move unpack_url above irrelevant checks. 2019-09-06 19:55:57 -04:00
Chris Hunt aef01d25cc Remove unused argument from unpack_url. 2019-09-06 19:55:57 -04:00
Chris Hunt 45283a332f Move delete marker file creation from unpack_url to caller. 2019-09-06 19:55:38 -04:00
Pradyun Gedam a217f550e1
Remove duplicate line in comment. (#6965) 2019-09-05 13:03:24 +05:30
Christopher Hunt 0a246b98a9
Update src/pip/_internal/operations/prepare.py 
Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com>
 2019-09-04 02:54:23 -04:00
Chris Hunt efd43fd0fe Minor comment typo. 2019-09-02 19:37:33 -04:00
Chris Hunt b719cc0033 Fix indentation: fix indentation 2019-09-02 19:36:57 -04:00
Chris Hunt b1978bc608 Fix indentation: move expanduser to constructor 2019-09-02 19:36:00 -04:00
Chris Hunt d45a544b47 Remove duplicate line in comment. 2019-09-02 15:52:26 -04:00
Stéphane Bidoul (ACSONE) 2e1dfbef76 consolidate vcs link detection 2019-08-16 08:14:33 +02:00
Pradyun Gedam 476606425a
Add mypy's no-strict-optional configuration inline (#6750) 2019-08-04 21:59:42 +05:30
Pradyun Gedam f18ec526db
Unify calls to prepare_distribution_metadata to one callsite 2019-08-04 11:16:16 +05:30
Pradyun Gedam f377148f6d
Use mypy's inline configuration syntax for opt-outs 2019-07-30 12:39:51 +05:30
Pradyun Gedam b43c71cbc7
Simplify references to req.link 2019-07-29 20:30:55 +05:30
Pradyun Gedam ec725cb1a0
Populate InstallRequirement.link before preparing it 2019-07-29 20:29:34 +05:30
Pradyun Gedam 537b0fca12
Change isort multi_line_output to 3 2019-07-22 10:15:27 +05:30
Pradyun Gedam a0670a9c98
Rename AbstractDistribution.prep_for_dist() 
Use a better name prepare_distribution_metadata()
 2019-06-16 05:29:00 +05:30
Pradyun Gedam 534a102564
Rename make_abstract_dist 
Use a more descriptive name make_distribution_for_install_requirement()
 2019-06-16 05:23:16 +05:30
Pradyun Gedam d43e31802b
Add distributions sub-package for holding build logic 
This moves the DistAbstraction logic into a dedicated 'distributions'
sub-package, with separated modules for the concrete implementations of
various distribution handling logic.
 2019-06-14 23:27:42 +05:30
Pradyun Gedam d3987fe114
Sort imports 2019-06-14 23:27:42 +05:30