Commit Graph

2858 Commits

Author SHA1 Message Date
Nguyễn Gia Phong b46576d933 Give batch downloader a separate class 2020-08-12 16:07:27 +07:00
Nguyễn Gia Phong a1aeb4ce01 Check download folder for files to be downloaded in batch 2020-08-12 16:02:28 +07:00
Nguyễn Gia Phong 18c803a413 Check hashes of memoized downloads 2020-08-11 22:40:42 +07:00
Nguyễn Gia Phong 39d296eeb8 Clean up code style and internal interface
Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com>
Co-Authored-By: Chris Hunt <chrahunt@gmail.com>
2020-08-11 22:40:39 +07:00
Nguyễn Gia Phong 078e0effb7 Add memoization mechanism for file download
This is intentionally dependent from caching, which relies on cache dir.
2020-08-11 22:39:42 +07:00
Nguyễn Gia Phong e62f16e969 Make Downloader perform the download 2020-08-11 22:36:08 +07:00
Pradyun Gedam 8eea660e03
Merge pull request #8744 from hroncok/keyring_global_nope
When one keyring attempt fails, don't bother with more
2020-08-11 15:41:02 +05:30
Pradyun Gedam 4c7bbdb4e4
Merge pull request #8702 from uranusjr/get-distribution-looks-for-all 2020-08-10 20:30:55 +05:30
Pradyun Gedam e8f52198a9
Merge pull request #8718 from uranusjr/pyvenv-cfg-encoding
Always use UTF-8 to read pyvenv.cfg
2020-08-07 15:11:32 +05:30
Tzu-ping Chung 4683ad02e3 Allow filtering constraints with markers 2020-08-07 13:55:38 +08:00
Nguyễn Gia Phong 11f7994a66 Revise method fetching metadata using lazy wheels
* Rename it to fit the fact that it no longer handle
  fetching _not_ using lazy wheels
* Use self as the first parameter
* Unnest the checks with additional logs showing reason
  when lazy wheel is not used
2020-08-06 18:43:02 +07:00
Nguyễn Gia Phong 487d00295c Define RequirementPreparer._session 2020-08-06 18:42:59 +07:00
Tzu-ping Chung 810385b971 Always use UTF-8 to read pyvenv.cfg 2020-08-06 11:46:46 +08:00
Nguyễn Gia Phong ffd6a38646 Disable caching for range requests 2020-08-05 22:19:49 +07:00
Tzu-ping Chung c04182893a Work around lax semantics in commands.search 2020-08-05 04:56:00 +08:00
Tzu-ping Chung 8491ce7723 Refactor and clarify get_distribution() behavior
The call to get_installed_distributions() now passes all flags
excplicitly so they are more obvious and less likely to be misunderstood
in the future. The behavior also documented in the function docstring.

The search_distribution() helper function is renamed with a leading
underscore to make it clear that it is intended as a helper function to
get_distribution().
2020-08-05 03:07:07 +08:00
Tzu-ping Chung e459763d81 Also look for non-local when searching for dists
This matches the behavior of pkg_resources.get_distribution(), which
this function intends to replace.
2020-08-05 03:07:07 +08:00
Pradyun Gedam ee4371c386
Merge pull request #8685 from chrahunt/use-preparer-for-lazy-wheels 2020-08-04 18:46:56 +05:30
Pradyun Gedam e7357c71cd
Merge pull request #8681 from McSinyx/faster-deps
[fast-deps] Make range requests closer to chunk size
2020-08-04 07:38:48 +05:30
Pradyun Gedam 70768de0a8
Merge pull request #8684 from uranusjr/zipfile-unicode-path-python2 2020-08-04 06:50:38 +05:30
Pradyun Gedam 0c48ec0696
Merge pull request #8679 from pradyunsg/tweak-resolutionimpossible-message 2020-08-04 06:49:10 +05:30
Pradyun Gedam 4fa31d509f
Merge pull request #8678 from uranusjr/new-resolver-no-deps-extras-install-self 2020-08-04 06:48:11 +05:30
Nguyễn Gia Phong d98ff19c27 [fast-deps] Make range requests closer to chunk size 2020-08-03 21:49:58 +07:00
Miro Hrončok ba062c3ed0 When one keyring attempt fails, don't bother with more
This makes https://github.com/pypa/pip/issues/8090 much less painful.
2020-08-03 12:34:26 +02:00
Tzu-ping Chung a12e2f1479 PEP 427 mandates UTF-8, we don't need the fallback 2020-08-03 15:00:58 +08:00
Tzu-ping Chung d4995cb89e Implement heuristics to get non-ASCII ZIP entries 2020-08-03 09:29:19 +08:00
Chris Hunt 8b838ebb89 Prepare lazy wheels more so they are downloaded
This keeps all knowledge about preparation and types of requirements in
`RequirementPreparer`, so there's one place to look when we're ready to
start breaking it apart later.
2020-08-02 19:36:29 -04:00
Chris Hunt c7ade159d4 Pass link to _fetch_metadata instead of req
Removes dependence on `InstallRequirement`.
2020-08-02 19:36:29 -04:00
Chris Hunt 21db4f3096 Log in one common location
Reduces dependence on `InstallRequirement` being passed to
`_fetch_metadata`.
2020-08-02 19:36:29 -04:00
Chris Hunt 6c4d4f3b78 Move _fetch_metadata to RequirementPreparer
The fact that all of this functionality can be put in terms of the
`RequirementPreparer` indicates that, at least at this point, this is
the cleanest place to put this functionality.
2020-08-02 19:36:29 -04:00
Chris Hunt 4e1bff741d Promote Wheel-related assertions to LinkCandidate constructor
These are things we know will be true because of the existing wheel
processing. In the future we may delegate the extraction of these to the
LinkCandidate itself so it doesn't have to be an assertion.
2020-08-02 19:36:28 -04:00
Chris Hunt 9e463916d0 Extract name and version from Wheel link
We happen to know that this is the same treatment that gave us `_name`
and `_version` for Wheels in the first place (in `LinkEvaluator`). It's not
ideal, however the metadata consistency check that occurs in `Candidate`
after creation of a `Distribution` guards us against any deviation in
the name and version during our processing.

Reduces dependence on Candidate.
2020-08-02 19:36:14 -04:00
Chris Hunt defbf82a8f Use link from InstallRequirement
Since when we generate the InstallRequirement we set the link, these
must be the same.

Reduces dependence on Candidate.
2020-08-02 18:40:11 -04:00
Chris Hunt f4603078cf Pass InstallRequirement to _fetch_metadata
Reduces dependence on Candidate.
2020-08-02 18:40:11 -04:00
Chris Hunt f0d4df10eb Propagate lazy_wheel option through RequirementPreparer
Reduces dependence on Candidate (and Resolver (and Factory)).
2020-08-02 18:40:11 -04:00
Chris Hunt e49dcfdc35 Move lazy_wheel warning out of Resolver
This warning just needs to be traced in one place for all commands,
there's no need for the resolver to know about it. Moving the warning
out of the Resolver will make it easier to change how we provide the
option.
2020-08-02 18:40:11 -04:00
Chris Hunt 45ab317610 Move call to _fetch_metadata next to call to RequirementPreparer
Since wheels can't be editable, we can move this into LinkCandidate,
closer to `RequirementPreparer.prepare_linked_requirement` into which we
want to integrate `_fetch_metadata`.
2020-08-02 18:40:11 -04:00
Chris Hunt ec5b6d7b80 Remove extra metadata consistency check
Instead of an early return, we fall through to the existing check at the
end of this function. This aligns our treatment of `_fetch_metadata` and
`_prepare_distribution`.
2020-08-02 18:40:11 -04:00
Chris Hunt a72d04f734 Move common processing out of _fetch_metadata
Returning a `Distribution` makes `_fetch_metadata` look more like
`_prepare_distribution`, in preparation for moving it there next.
2020-08-02 18:40:11 -04:00
Chris Hunt 8c3c0ade78 Move _fetch_metadata into _prepare
Since `_prepare` is called in two places, we preserve the
`if self._dist is not None` protection above the new call to
`_fetch_metadata`. The second `if` in `_prepare` handles the early
return required when processing a lazy wheel.
2020-08-02 18:40:09 -04:00
Chris Hunt 4d94ae4c40 Move non-lazy req fallback outside of `_fetch_metadata`
No change in behavior, we just want to unify "requirements processing"
and moving this function out is a prereq for moving `_fetch_metadata` in.
2020-08-02 18:39:57 -04:00
Chris Hunt 7a5e043776 Remove unnecessary check for _dist
Since `_prepare` now internally validates that `_dist` isn't set, we
don't need to.
2020-08-02 18:39:15 -04:00
Chris Hunt 7289625734 Remove redundant guard variable
Now that `_dist` is only set on success, we can use it to guard against
repeated execution instead of `_prepared`. As a result there are now only
two possible outcomes for calling `dist`:

1. `_dist` set and returned - lazy and non-lazy req
2. `_dist` not set and exception raised - bad lazy or bad non-lazy req
2020-08-02 18:38:13 -04:00
Chris Hunt d957cc94c8 Don't set _dist until it has been validated
Previously a call to `_fetch_metadata` could result in several possible
outcomes:

1. `_dist` set, `_provided` not set, dist returned - for lazy wheels
2. `_dist` set, `_provided` not set, exception - for bad lazy wheels
3. `_dist` not set, `_provided` not set, exception - for non-lazy req
   exceptions
4. `_dist` set, `_provided` not set, exception - for bad non-lazy reqs
5. `_dist` set, `_provided` set, dist returned - for non-lazy reqs

and probably more.

Our intent is to use `_dist` being set as the indicator of "this
requirement has been fully processed successfully" and discard
`_prepared`, since we don't actually rely on any of the other states
(they simply lead to a failure or in the future a retry).
2020-08-02 18:37:50 -04:00
Tzu-ping Chung 32b5e43c79 Flip the flag with another name 2020-08-03 05:28:10 +08:00
Chris Hunt 89d8cba55b
Merge pull request #8629 from McSinyx/abstract-abstract
Abstract away AbstractDistribution in higher-level resolver code
2020-08-02 14:41:24 -04:00
Pradyun Gedam 7e6ff08aae
Merge pull request #8656 from chrahunt/gracefully-handle-bad-data-paths
Trace a better error message on installation failure due to invalid .data files in wheels
2020-08-02 19:55:40 +05:30
Tzu-ping Chung 3ce63a62d7 Ask candidates for dependencies even on --no-deps
ExtrasCandidate need to provide one dependency on the non-extra-ed self.
2020-08-02 16:12:46 +08:00
Pradyun Gedam c412613efe
Tweak ResolutionImpossible error line
This makes it more consistent with how error "summary" lines look.

eg:
     IndexError: list index out of range
      ModuleNotFoundError: No module named 'notamodule'
2020-08-02 07:55:49 +05:30
Pradyun Gedam b97c199cf7
Point to latest documentation
This allows us to update the content users would see as we get feedback.
2020-08-02 07:53:02 +05:30