pip ignores dependency_links in setup.py

This answer should help. In a nutshell, you need to specify the version (or "dev") for the #egg=python-s3 so it looks like #egg=python-s3-1.0.0.

Updates based on @Cerin's comment:

• Pip 1.5.x has a flag to enable dependency-links processing: --process-dependency-links. I haven't tested it because I agree with the point below.
• This discussion seems to indicate that using dependency-links for pip is a bad practice. Although this feature was enlisted for deprecation, it's not anymore. There's a valid use case for private packages.

I realize this is an old question, but, just in case you find yourself here like I did, this is what worked for me.

I've got a package on GitHub (not registered with pypi) that relies on other GitHub (non-pypi) packages. I spent an inordinate amount of time trying to figure out how to get pip to handle this correctly. I will include what I did to fix it here.

Putting dependencies in a requirements.txt file is the preferred method of listing dependencies. However, you also need to populate install_requires in setup. It was at this stage that I ran into a roadblock with pip not wanting to install dependencies from GitHub.

Most places, including answers to this question, tell you to populate the dependency_links section of setup. However, you also need to populate the install_requires field with the name of the package referenced in dependency_links.

For example, if your requirements.txt contains the following.

somepackage==1.2.0
https://github.com/user/repo/tarball/master#egg=repo-1.0.0
anotherpackage==4.2.1

Then, your setup call should look like this:

setup(
name='yourpackage',
version='1.7.5',
packages=[],
url='',
license='',
author='',
author_email='',
description='',
install_requires=[
'somepackage==1.2.0',
'repo==1.0.0',
'anotherpackage==4.2.1'
],
dependency_links=[
'https://github.com/user/repo/tarball/master#egg=repo-1.0.0'
]
)

Ok, so now we've got our package configured; installing it is the next task. This is where I spent a lot of time. I could not figure out why specifying dependency_links apparently did nothing. The trick is that in some cases, you need to set the allow-all-external (can be more specific) flag for pip. For example:

pip install git+https://github.com/user/anotherrepo.git
--process-dependency-links --allow-all-external

You're done and it works!

DISCLAIMER: dependency_links and the flags process-dependency-links and allow-all-external are deprecated, so they will be removed soon. In the time I spent, I could not locate a better, prefered method and still have pip function properly.

A couple of notes on some issues I found, in particular for installing from private repos.

Installing from pip & setuptools have some subtle differences; but this way should work for both.

from setuptools import setup
import os
# get deploy key from https://help.github.com/articles/git-automation-with-oauth-tokens/
github_token = os.environ['GITHUB_TOKEN']


setup(
# ...
install_requires='package',
dependency_links = [
'git+https://{github_token}@github.com/user/{package}.git/@{version}#egg={package}-0'
.format(github_token=github_token, package=package, version=master)
]

A couple of notes here:

• For private repos, you need to authenticate with GitHub; the simplest way I found is to create an oauth token, drop that into your environment, and then include it with the URL
• You need to include some version number (here is 0) at the end of the link, even if there's no package on PyPI. This has to be a actual number, not a word.
• You need to preface with git+ to tell setuptools it's to clone the repo, rather than pointing at a zip / tarball
• version can be a branch, a tag, or a commit hash
• You need to supply --process-dependency-links if installing from pip

since pip version 18.1 PEP 508 URL is supported. That means you don't need the deprecated dependency_links anymore. You write the dependency directly in the install_requires list instead. The example from @Chad looks like this:

setup(
name='yourpackage',
version='1.7.5',
packages=[],
url='',
license='',
author='',
author_email='',
description='',
install_requires=[
'somepackage==1.2.0',
'repo @ https://github.com/user/archive/master.zip#egg=repo-1.0.0',
'anotherpackage==4.2.1'
],
)

To install your package you can simply write:

pip install yourpackage

(without --process-dependency-links)

First upgrade your pip version as this is a new syntax.

pip install pip --upgrade

Then do the following:

install_requires=[
'bleualign-git @ https://github.com/rsennrich/Bleualign/archive/<commit-hash or branch-name>.zip#egg=bleualign-git-1.0.0'
]

• Version 1.0.0 and name "bluealign-git" are randomly chosen.
• The version number is necessary.
• We advisedly used the name "bleualign-git" to distinguish it from the main repository version.

Hope this helps.

Comments:

Good answer (upvoted). Also want to mention that the actual result would likely depend on the platform and/or pip version, though. I've seen when it works, or seemingly works but did not really pull the dependency from the specified link, or simply being rejected. So I would probably just use this as a short time workaround only. Stick with the mainstream way whenever possible.

None of the above solutions worked for me in their respective explicit forms. I am adding this as another solution for certain specific cases. OP has a tarball in their remote repo, but I ended up here for a related case: only source code in the remote repo.

I have a dependency that is a remote private repository on Azure DevOps. I would like to use pip to install a local python package and also automatically install the remote dependency. The remote repository is a python package with a setup.py and some source code. The repo doesn't have build artifacts like eggs/wheels/tarballs, just .py files. I am able to manually install the remote dependency with pip install git+https://..., hence I suspected correctly that the remote repository doesn't need to contain eggs/wheels/tarballs.

Here is the relevant line of my setup.py file

setup.py


...


install_requires=["packagename @ git+https://dev.azure.com/.../_git/packagename"]