Skip to content

Commit 4ad1828

Browse files
authored
Merge pull request #13495 from ichard26/feat/direct-editables
Support Direct URL editable requirements
2 parents 66ded3b + 8a4d049 commit 4ad1828

File tree

8 files changed

+161
-37
lines changed

8 files changed

+161
-37
lines changed

docs/html/cli/pip_install.rst

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -66,8 +66,8 @@ for the name and project version (this is in theory slightly less reliable
6666
than using the ``egg_info`` command, but avoids downloading and processing
6767
unnecessary numbers of files).
6868

69-
Any URL may use the ``#egg=name`` syntax (see :doc:`../topics/vcs-support`) to
70-
explicitly state the project name.
69+
The :ref:`Direct URL requirement syntax <pypug:dependency-specifiers>` can be used
70+
to explicitly state the project name (see :doc:`../topics/vcs-support`).
7171

7272
Satisfying Requirements
7373
-----------------------
@@ -367,21 +367,21 @@ Examples
367367

368368
.. code-block:: shell
369369
370-
python -m pip install -e 'git+https://git.repo/some_pkg.git#egg=SomePackage' # from git
371-
python -m pip install -e 'hg+https://hg.repo/some_pkg.git#egg=SomePackage' # from mercurial
372-
python -m pip install -e 'svn+svn://svn.repo/some_pkg/trunk/#egg=SomePackage' # from svn
373-
python -m pip install -e 'git+https://git.repo/some_pkg.git@feature#egg=SomePackage' # from 'feature' branch
374-
python -m pip install -e 'git+https://git.repo/some_repo.git#egg=subdir&subdirectory=subdir_path' # install a python package from a repo subdirectory
370+
python -m pip install -e 'SomePackage @ git+https://git.repo/some_pkg.git' # from git
371+
python -m pip install -e 'SomePackage @ hg+https://hg.repo/some_pkg.git' # from mercurial
372+
python -m pip install -e 'SomePakcage @ svn+svn://svn.repo/some_pkg/trunk/' # from svn
373+
python -m pip install -e 'SomePackage @ git+https://git.repo/some_pkg.git@feature' # from 'feature' branch
374+
python -m pip install -e 'SomePackage @ git+https://git.repo/some_repo.git#subdirectory=subdir_path' # install a python package from a repo subdirectory
375375
376376
.. tab:: Windows
377377

378378
.. code-block:: shell
379379
380-
py -m pip install -e "git+https://git.repo/some_pkg.git#egg=SomePackage" # from git
381-
py -m pip install -e "hg+https://hg.repo/some_pkg.git#egg=SomePackage" # from mercurial
382-
py -m pip install -e "svn+svn://svn.repo/some_pkg/trunk/#egg=SomePackage" # from svn
383-
py -m pip install -e "git+https://git.repo/some_pkg.git@feature#egg=SomePackage" # from 'feature' branch
384-
py -m pip install -e "git+https://git.repo/some_repo.git#egg=subdir&subdirectory=subdir_path" # install a python package from a repo subdirectory
380+
py -m pip install -e "SomePackage @ git+https://git.repo/some_pkg.git" # from git
381+
py -m pip install -e "SomePackage @ hg+https://hg.repo/some_pkg.git" # from mercurial
382+
py -m pip install -e "SomePackage @ svn+svn://svn.repo/some_pkg/trunk/" # from svn
383+
py -m pip install -e "SomePackage @ git+https://git.repo/some_pkg.git@feature" # from 'feature' branch
384+
py -m pip install -e "SomePackage @ git+https://git.repo/some_repo.git#subdirectory=subdir_path" # install a python package from a repo subdirectory
385385
386386
#. Install a package with extras, i.e., optional dependencies
387387
(:ref:`specification <pypug:dependency-specifiers>`).

docs/html/topics/vcs-support.md

Lines changed: 32 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,26 @@ control system being used). It is used through URL prefixes:
1010
- Subversion -- `svn+`
1111
- Bazaar -- `bzr+`
1212

13+
The general form of a VCS requirement is `ProjectName @ VCS_URL`, e.g.
14+
15+
```none
16+
MyProject @ git+https://git.example.com/MyProject
17+
MyProject[extra] @ git+https:/git.example.com/MyProject
18+
```
19+
20+
This is the {ref}`Direct URL <pypug:dependency-specifiers>` requirement syntax.
21+
It is also permissible to remove `MyProject @` portion is removed and provide
22+
a bare VCS URL.
23+
24+
```none
25+
git+https://git.example.com/MyProject
26+
```
27+
28+
This is a pip specific extension. This form can be used as long as pip does
29+
not need to know the project name in advance. pip is generally able to infer
30+
the project name except in the case of {ref}`editable-vcs-installs`. In
31+
addition, extras cannot be requested using a bare VCS URL.
32+
1333
## Supported VCS
1434

1535
### Git
@@ -81,8 +101,8 @@ MyProject @ svn+ssh://user@svn.example.com/MyProject
81101
You can also give specific revisions to an SVN URL, like so:
82102

83103
```none
84-
-e svn+http://svn.example.com/svn/MyProject/trunk@2019#egg=MyProject
85-
-e svn+http://svn.example.com/svn/MyProject/trunk@{20080101}#egg=MyProject
104+
-e MyProject @ svn+http://svn.example.com/svn/MyProject/trunk@2019
105+
-e MyProject @ svn+http://svn.example.com/svn/MyProject/trunk@{20080101}
86106
```
87107

88108
Note that you need to use [Editable VCS installs](#editable-vcs-installs) for
@@ -115,6 +135,9 @@ MyProject @ bzr+http://bzr.example.com/MyProject/trunk@v1.0
115135
VCS projects can be installed in {ref}`editable mode <editable-installs>` (using
116136
the {ref}`--editable <install_--editable>` option) or not.
117137

138+
In editable mode, the project name must be provided upfront using the Direct URL
139+
(`MyProject @ URL`) form so pip can determine the VCS clone location.
140+
118141
- The default clone location (for editable installs) is:
119142

120143
- `<venv path>/src/SomeProject` in virtual environments
@@ -133,15 +156,16 @@ take on the VCS requirement (not the commit itself).
133156
## URL fragments
134157

135158
pip looks at the `subdirectory` fragments of VCS URLs for specifying the path to the
136-
Python package, when it is not in the root of the VCS directory. eg: `pkg_dir`.
159+
Python package, when it is not in the root of the VCS directory.
137160

138-
pip also looks at the `egg` fragment specifying the "project name". In practice the
139-
`egg` fragment is only required to help pip determine the VCS clone location in editable
140-
mode. In all other circumstances, the `egg` fragment is not necessary and its use is
141-
discouraged.
161+
```{note}
162+
pip also supports an `egg` fragment to specify the "project name". This is a legacy
163+
feature and its use is discouraged in favour of the
164+
{ref}`Direct URL <pypug:dependency-specifiers>` form.
142165
143166
The `egg` fragment **should** be a bare {ref}`project name <pypug:name-normalization>`.
144167
Anything else is not guaranteed to work.
168+
```
145169

146170
````{admonition} Example
147171
If your repository layout is:
@@ -164,6 +188,6 @@ $ pip install "pkg @ vcs+protocol://repo_url/#subdirectory=pkg_dir"
164188
or:
165189
166190
```{pip-cli}
167-
$ pip install -e "vcs+protocol://repo_url/#egg=pkg&subdirectory=pkg_dir"
191+
$ pip install -e "pkg @ vcs+protocol://repo_url/#subdirectory=pkg_dir"
168192
```
169193
````

docs/html/user_guide.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,7 @@ In practice, there are 4 common uses of Requirements files:
183183
``sometag``. You'd reference it in your requirements file with a line like
184184
so::
185185

186-
git+https://myvcs.com/some_dependency@sometag#egg=SomeDependency
186+
SomeDependency @ git+https://myvcs.com/some_dependency@sometag
187187

188188
If ``SomeDependency`` was previously a top-level requirement in your
189189
requirements file, then **replace** that line with the new line. If

news/13495.feature.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Add support installing an editable requirement written as a Direct URL (``PackageName @ URL``).

src/pip/_internal/req/constructors.py

Lines changed: 41 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -86,17 +86,25 @@ def _set_requirement_extras(req: Requirement, new_extras: set[str]) -> Requireme
8686
return get_requirement(f"{pre}{extras}{post}")
8787

8888

89-
def parse_editable(editable_req: str) -> tuple[str | None, str, set[str]]:
90-
"""Parses an editable requirement into:
91-
- a requirement name
92-
- an URL
93-
- extras
94-
- editable options
95-
Accepted requirements:
96-
svn+http://blahblah@rev#egg=Foobar[baz]&subdirectory=version_subdir
97-
.[some_extra]
98-
"""
89+
def _parse_direct_url_editable(editable_req: str) -> tuple[str | None, str, set[str]]:
90+
try:
91+
req = Requirement(editable_req)
92+
except InvalidRequirement:
93+
pass
94+
else:
95+
if req.url:
96+
# Join the marker back into the name part. This will be parsed out
97+
# later into a Requirement again.
98+
if req.marker:
99+
name = f"{req.name} ; {req.marker}"
100+
else:
101+
name = req.name
102+
return (name, req.url, req.extras)
103+
104+
raise ValueError
99105

106+
107+
def _parse_pip_syntax_editable(editable_req: str) -> tuple[str | None, str, set[str]]:
100108
url = editable_req
101109

102110
# If a file path is specified with extras, strip off the extras.
@@ -122,23 +130,41 @@ def parse_editable(editable_req: str) -> tuple[str | None, str, set[str]]:
122130
url = f"{version_control}+{url}"
123131
break
124132

133+
return Link(url).egg_fragment, url, set()
134+
135+
136+
def parse_editable(editable_req: str) -> tuple[str | None, str, set[str]]:
137+
"""Parses an editable requirement into:
138+
- a requirement name with environment markers
139+
- an URL
140+
- extras
141+
Accepted requirements:
142+
- svn+http://blahblah@rev#egg=Foobar[baz]&subdirectory=version_subdir
143+
- local_path[some_extra]
144+
- Foobar[extra] @ svn+http://blahblah@rev#subdirectory=subdir ; markers
145+
"""
146+
try:
147+
package_name, url, extras = _parse_direct_url_editable(editable_req)
148+
except ValueError:
149+
package_name, url, extras = _parse_pip_syntax_editable(editable_req)
150+
125151
link = Link(url)
126152

127-
if not link.is_vcs:
153+
if not link.is_vcs and not link.url.startswith("file:"):
128154
backends = ", ".join(vcs.all_schemes)
129155
raise InstallationError(
130156
f"{editable_req} is not a valid editable requirement. "
131157
f"It should either be a path to a local project or a VCS URL "
132158
f"(beginning with {backends})."
133159
)
134160

135-
package_name = link.egg_fragment
136-
if not package_name:
161+
# The project name can be inferred from local file URIs easily.
162+
if not package_name and not link.url.startswith("file:"):
137163
raise InstallationError(
138164
f"Could not detect requirement name for '{editable_req}', "
139-
"please specify one with #egg=your_package_name"
165+
"please specify one with your_package_name @ URL"
140166
)
141-
return package_name, url, set()
167+
return package_name, url, extras
142168

143169

144170
def check_first_requirement_in_file(filename: str) -> None:

src/pip/_internal/resolution/resolvelib/candidates.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,8 +84,12 @@ def make_install_req_from_editable(
8484
link: Link, template: InstallRequirement
8585
) -> InstallRequirement:
8686
assert template.editable, "template not editable"
87+
if template.name:
88+
req_string = f"{template.name} @ {link.url}"
89+
else:
90+
req_string = link.url
8791
ireq = install_req_from_editable(
88-
link.url,
92+
req_string,
8993
user_supplied=template.user_supplied,
9094
comes_from=template.comes_from,
9195
isolated=template.isolated,

tests/functional/test_install_reqs.py

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
PipTestEnvironment,
1111
ResolverVariant,
1212
TestData,
13+
_create_test_package,
1314
_create_test_package_with_subdirectory,
1415
create_basic_sdist_for_package,
1516
create_basic_wheel_for_package,
@@ -940,3 +941,52 @@ def test_config_settings_local_to_package(
940941
assert "--verbose" not in simple3_args
941942
simple2_args = simple2_sdist.args()
942943
assert "--verbose" not in simple2_args
944+
945+
946+
class TestEditableDirectURL:
947+
def test_install_local_project(
948+
self, script: PipTestEnvironment, data: TestData, common_wheels: Path
949+
) -> None:
950+
uri = (data.src / "simplewheel-2.0").as_uri()
951+
script.pip(
952+
"install", "--no-index", "-e", f"simplewheel @ {uri}", "-f", common_wheels
953+
)
954+
script.assert_installed(simplewheel="2.0")
955+
956+
def test_install_local_project_with_extra(
957+
self, script: PipTestEnvironment, data: TestData, common_wheels: Path
958+
) -> None:
959+
uri = (data.src / "requires_simple_extra").as_uri()
960+
script.pip(
961+
"install",
962+
"--no-index",
963+
"-e",
964+
f"requires-simple-extra[extra] @ {uri}",
965+
"-f",
966+
common_wheels,
967+
"-f",
968+
data.packages,
969+
)
970+
script.assert_installed(requires_simple_extra="0.1")
971+
script.assert_installed(simple="1.0")
972+
973+
def test_install_local_git_repo(
974+
self, script: PipTestEnvironment, common_wheels: Path
975+
) -> None:
976+
repo_path = _create_test_package(script.scratch_path, "simple")
977+
url = "git+" + repo_path.as_uri()
978+
script.pip(
979+
"install", "--no-index", "-e", f"simple @ {url}", "-f", common_wheels
980+
)
981+
script.assert_installed(simple="0.1")
982+
983+
@pytest.mark.network
984+
def test_install_remote_git_repo_with_extra(
985+
self, script: PipTestEnvironment, data: TestData, common_wheels: Path
986+
) -> None:
987+
req = "pip-test-package[extra] @ git+https://github.com/pypa/pip-test-package"
988+
script.pip(
989+
"install", "--no-index", "-e", req, "-f", common_wheels, "-f", data.packages
990+
)
991+
script.assert_installed(pip_test_package="0.1.1")
992+
script.assert_installed(simple="3.0")

tests/unit/test_req.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -849,6 +849,25 @@ def test_install_req_extend_extras(
849849
assert extended.permit_editable_wheels == req.permit_editable_wheels
850850

851851

852+
@pytest.mark.parametrize(
853+
"req_str, expected",
854+
[
855+
(
856+
'foo[extra] @ svn+http://foo ; os_name == "nt"',
857+
('foo ; os_name == "nt"', "svn+http://foo", {"extra"}),
858+
),
859+
(
860+
"foo @ svn+http://foo",
861+
("foo", "svn+http://foo", set()),
862+
),
863+
],
864+
)
865+
def test_parse_editable_pep508(
866+
req_str: str, expected: tuple[str, str, set[str]]
867+
) -> None:
868+
assert parse_editable(req_str) == expected
869+
870+
852871
@mock.patch("pip._internal.req.req_install.os.path.abspath")
853872
@mock.patch("pip._internal.req.req_install.os.path.exists")
854873
@mock.patch("pip._internal.req.req_install.os.path.isdir")

0 commit comments

Comments
 (0)