Issue
I'm trying to build a Docker container that should install a series of python packages from a requirements.txt file. One of the entries is a python package hosted on a private GitHub repository. To install it, I've created a pair of SSH keys and added the public one as a Deploy Key to the GitHub repository.
However, when I'm building the container I'm getting this error:
ERROR: Command errored out with exit status 128: git clone -q 'ssh://****@github.com:organization/my-package' /tmp/pip-install-e81w4wri/my-package Check the logs for full command output.
I've tried to debug the error by changing the pip install command of the docker file with RUN git clone [email protected]:organization/my-package.git and it did work fine.
What does this error mean and how can I solve it? I could clone it and install it with a dedicated command, but if possible I'd like to keep all requirements in a single place. Thanks!
This is the Dockerfile that I'm using:
FROM joyzoursky/python-chromedriver:3.7-alpine3.8 as base
FROM base as builder
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add bash gcc gfortran build-base git wget freetype-dev libpng-dev openblas-dev openssh-client
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
# copy requirements
RUN mkdir /install
WORKDIR /install
COPY ./requirements.txt /var/www/requirements.txt
### GITHUB SSH KEY ###
COPY ./keys/deploy_key_private .
RUN mkdir /root/.ssh && mv deploy_key_private /root/.ssh/id_rsa
RUN eval $(ssh-agent) && \
ssh-add /root/.ssh/id_rsa && \
ssh-keyscan -H github.com >> /etc/ssh/ssh_known_hosts
RUN pip install --upgrade pip && pip install --prefix=/install -r /var/www/requirements.txt --log logs.txt
FROM base
COPY --from=builder /install /usr/local
# KEEP ON BUILDING THE CONTAINER
The package is listed in the requirements.txt as git+ssh://[email protected]:organization/my-package@master#egg=my_package
If relevant, here is the traceback from pip:
Exception information:
2020-07-30T11:56:55,329 Traceback (most recent call last):
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
2020-07-30T11:56:55,329 status = self.run(options, args)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
2020-07-30T11:56:55,329 return func(self, options, args)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run
2020-07-30T11:56:55,329 reqs, check_supported_wheels=not options.target_dir
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
2020-07-30T11:56:55,329 discovered_reqs.extend(self._resolve_one(requirement_set, req))
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
2020-07-30T11:56:55,329 abstract_dist = self._get_abstract_dist_for(req_to_install)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 340, in _get_abstract_dist_for
2020-07-30T11:56:55,329 abstract_dist = self.preparer.prepare_linked_requirement(req)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 469, in prepare_linked_requirement
2020-07-30T11:56:55,329 hashes=self._get_linked_req_hashes(req)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 239, in unpack_url
2020-07-30T11:56:55,329 unpack_vcs_link(link, location)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 99, in unpack_vcs_link
2020-07-30T11:56:55,329 vcs_backend.unpack(location, url=hide_url(link.url))
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 733, in unpack
2020-07-30T11:56:55,329 self.obtain(location, url=url)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 641, in obtain
2020-07-30T11:56:55,329 self.fetch_new(dest, url, rev_options)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/git.py", line 230, in fetch_new
2020-07-30T11:56:55,329 self.run_command(make_command('clone', '-q', url, dest))
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 774, in run_command
2020-07-30T11:56:55,329 log_failed_cmd=log_failed_cmd)
2020-07-30T11:56:55,329 File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 166, in call_subprocess
2020-07-30T11:56:55,329 raise SubProcessError(exc_msg)
2020-07-30T11:56:55,329 pip._internal.exceptions.SubProcessError: Command errored out with exit status 128: git clone -q 'ssh://****@github.com:organization/my-package' /tmp/pip-install-e81w4wri/my-package Check the logs for full command output.
Solution
[email protected]:organization/my-package.git is a valid SSH URL.
ssh://[email protected]:organization/my-package.git is not.
ssh://[email protected]/organization/my-package.git would be.
As in here, you can add GIT_SSH_COMMAND='ssh -v' pip install ... to see exactly what is going on.
You might need:
git config --global url."ssh://[email protected]/".insteadOf ssh://[email protected]:
The OP arabinelli reports in the comments having to use the following line in requirements.txt:
git+ssh://[email protected]/my-organization/my-repo-name@master#egg=my_package_dir
Aug. 2022, Jako adds in the comments:
This worked for me with a private BitBucket repository:
git+ssh://[email protected]/my-organization/my-repo-name@master#egg=my_project&subdirectory=subdir1^^^^^^^I had to specify the subdirectory '
subdir1'
Answered By - VonC Answer Checked By - Dawn Plyler (WPSolving Volunteer)