1

I am looking for a one-liner to install a Python package from GitHub via pip, without having git installed necessarily, and with being able to specify extra dependencies.

This is what I tried (no need to escape [] as I am on Windows):

pip install https://github.com/python/mypy/archive/master.zip[reports]

The following does work, with the first one taken from this answer:

# requires a git client to be installed
pip install -e git+https://github.com/python/mypy.git#egg=mypy[reports]

# omits extra dependencies
pip install https://github.com/python/mypy/archive/master.zip

But they have various restrictions as noted in the comments.

For the initial command, I am getting

Collecting https://github.com/python/mypy/archive/master.zip[reports]
  ERROR: HTTP error 404 while getting https://github.com/python/mypy/archive/master.zip[reports]
  ERROR: Could not install requirement https://github.com/python/mypy/archive/master.zip[reports] because of HTTP error 404 Client Error: Not Found for url: https://github.com/python/mypy/archive/master.zip%5Breports%5D for URL https://github.com/python/mypy/archive/master.zip[reports]

(I am aware that this is not supposed to work for packages using setuptools_scm, e.g., psf/black, due to https://github.com/pypa/setuptools_scm/blob/ca3855ba66fa4cb100f5039eea909932f815a4a4/src/setuptools_scm/__init__.py#L106-L115, but my package does not depend on that and installs fine from a GitHub zip file.)

I have found the rather obscure solution

pip install "mypy[reports] @ https://github.com/python/mypy/archive/master.zip" 

from How to pip install with wheel specifying extras?

Is there anything more intuitive?

3
  • I also found python -m pip install "SomePackage[PDF] @ ... at pip.pypa.io/en/stable/cli/pip_install, so maybe this is the (only) answer. Commented May 5, 2022 at 5:10
  • Does it really need to be a one-liner? I haven:t tried it yet on this package, but a common workaround is downloading the release zip file and manually installing the package locally. Commented May 5, 2022 at 5:18
  • 1
    @GinoMempin well not really really, but it would be easier. For example, to which (portable, writable) location should I download the release zip?. How do I deal with network connection errors, storage space running out, ... - pip will have all of these cases covered. Commented May 5, 2022 at 8:58

1 Answer 1

3

I know you are asking for a one-liner, but let me describe first the common workaround to targeting Git repositories (not just Github) that also avoids requiring git:

  1. Download a copy of the source code
    • Using wget or curl or whichever shell or script-able utility is available
    • From Github, you can target either
  2. Unzip/Untar the .zip/.tar.gz to some temporary directory
  3. From that temporary directory, run
    pip install ./<temp directory>[<extra dependency]
    
    • The same considerations for where pip installs the package applies
    • If using a virtual env, make sure to activate it first
    • Or use the python -m pip install ... syntax to select a specific environment and interpreter

With regular pip install targeting a Git repo like this,

pip install git+https://github.com/python/mypy.git#egg=mypy[reports]
pip install "mypy[reports] @ https://github.com/python/mypy/archive/master.zip"

...it automatically already does steps 1 and 2. If you add a -v/--verbose option, you'll see that it clones a copy of the codes first from a specific commit, and stores them in some temporary directory (which we don't normally care about), then does the regular installation from there.

Sample for mypy:

$ pip install -v git+https://github.com/python/mypy.git#egg=mypy[reports]
...
Collecting mypy[reports]
  Cloning https://github.com/python/mypy.git to /private/var/folders/3h/pdjwtnlx4p13chnw21rvwbtw0000gp/T/pip-install-d6gv_iva/mypy_fe678fe853cb45b68755c0c57dfcb757
...
Cloning into '/private/var/folders/3h/pdjwtnlx4p13chnw21rvwbtw0000gp/T/pip-install-d6gv_iva/mypy_fe678fe853cb45b68755c0c57dfcb757'
...
Building wheels for collected packages: mypy
  Running command Building wheel for mypy (pyproject.toml)
  running bdist_wheel
  running build
  running build_py
  pin_version()
  creating build
  creating build/lib
  creating build/lib/mypy
  creating build/lib/mypyc
...
Successfully built mypy
Installing collected packages: mypy
  Running command git rev-parse HEAD
  4f07c79aea0fef61ab649d6acedf01186f1054eb
  changing mode of /path/to/venv/bin/dmypy to 755
  changing mode of /path/to/venv/bin/mypy to 755
  changing mode of /path/to/venv/bin/mypyc to 755
  changing mode of /path/to/venv/bin/stubgen to 755
  changing mode of /path/to/venvi/bin/stubtest to 755
Successfully installed mypy-0.960+dev.4f07c79aea0fef61ab649d6acedf01186f1054eb

$ python
...
>>> import importlib_metadata
>>> importlib_metadata.metadata('mypy').get_all('Requires-Dist')
[ ... extra == 'reports'"]

So, if you don't have Git, you can't do a git clone, but instead have to download the codes manually. As with the direct pip install, if you don't care about a specific version, you can just download the latest master/main commit:

Sample again for mypy:

$ wget -O mypy-master.zip https://github.com/python/mypy/archive/refs/heads/master.zip
...
2022-05-05 20:29:07 (2.85 MB/s) - ‘mypy-master.zip’ saved [3433301]

$ unzip -o mypy-master.zip
...
inflating: mypy-master/test-data/unit/typexport-basic.test  
inflating: mypy-master/test-requirements.txt  
inflating: mypy-master/tox.ini 

Then the last step would simply be doing pip install, which supports targetting a <local project path> and specifying extras all in one command:

python -m pip install .[PDF]  # project in current directory

So following the downloaded and unpacked mypy-master above, the last step would simply be:

$ pip install -v ./mypy-master[reports]
Processing ./mypy-master
...
Building wheels for collected packages: mypy
  Running command Building wheel for mypy (pyproject.toml)
  running bdist_wheel
  running build
  running build_py
  pin_version()
  creating build
  creating build/lib
  creating build/lib/mypy
  creating build/lib/mypyc
...
Successfully built mypy
Installing collected packages: mypy
  Running command git rev-parse HEAD
  4f07c79aea0fef61ab649d6acedf01186f1054eb
  changing mode of /path/to/venv/bin/dmypy to 755
  changing mode of /path/to/venv/bin/mypy to 755
  changing mode of /path/to/venv/bin/mypyc to 755
  changing mode of /path/to/venv/bin/stubgen to 755
  changing mode of /path/to/venvi/bin/stubtest to 755
Successfully installed mypy-0.960+dev.4f07c79aea0fef61ab649d6acedf01186f1054eb

$ python
...
>>> import importlib_metadata
>>> importlib_metadata.metadata('mypy').get_all('Requires-Dist')
[ ... extra == 'reports'"]

...which shows the same output as the one that pip install-ed directly from git+https. The same considerations with using pip install applies, for example, if you are using a virtual environment, make sure to activate that first so mypy is installed in the same location.

Now, to make it all a one-liner command, you can turn those steps into a script, a shell alias, or just straight-up run multiple commands with && operator:

$ wget -O mypy-master.zip https://github.com/python/mypy/archive/refs/heads/master.zip && unzip -o mypy-master.zip && pip install ./mypy-master[reports] && rm -R mypy-master*

Note that I added removal of the temporary files/folders at the end of the command. If one of the steps fails (ex. network connection error), then the rest of the commands will be aborted. You can add better/more error-handling as needed.

$ wget -O mypy-master.zip https://github.com/python/mypy/archive/refs/heads/master.zip && unzip -o mypy-master.zip && pip install ./mypy-master[reports] && rm -Rf mypy-master*
--2022-05-05 21:09:39--  https://github.com/python/mypy/archive/refs/heads/master.zip
Resolving github.com (github.com)... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address ‘github.com’

I agree though that that isn't an elegant or an intuitive command.

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.