[SOLVED] Pip upgrade does not remove the old pip and breaks

Issue

Edit: My question is not regarding how to immediately get pip to work. I fixed this by relinking as you can see in the question.

The question is: What is best/pythonic practice on a fresh *NIX system? One would probably want to have the most recent version of both Python2 and especially Python3 with pip and setuptools and so on. A safeguarded virtual environment seems to be the way, but anyway one has to start somewhere.

So what is the recommendation?

From the comments it seems that one should use the get-pip.py script to install and then use pip to install virtualenv. I guess this has to be done both for Python2 and Python3 if one wants to have both.

This is what happened some days ago:

Ubuntu 18.04 and Python 2.7

I am a heavy R user, but use more and more Python. Virtually every time I set up any kind of customized python environment on Linux or OSX, something breaks and has to be fiddled with.... links, binaries, paths, dependencies. Each and every time. How many thousands of people are sitting on a Ubuntu/Debian box right now and do apt install python pip; pip install --upgrade pip and -duh- it breaks? Really?

Specifically right now: I want to install django on a webserver and started with apt install python-pip.

Then I did the recommended pip install --upgrade pip which installed pip-18.0 but gave the message Not uninstalling pip at /usr/lib/python2.7/dist-packages

After that pip --version threw an error

Traceback (most recent call last): File "/usr/bin/pip", line 9, in from pip import main

It turned out that the old pip still resides in /usr/bin/pip while the new one is in /usr/local/bin/pip.

I fixed it brute force with:

rm /usr/bin/pip
ln -s /usr/local/bin/pip /usr/bin/pip

Did I do the right thing or is this the way to doom down the road?

Is there a pythonic or elegant way to handle such issues or to prevent them in the first place?

With best regards and thanks for any kind of suggestions.

Solution

My conclusion after investigating the hints in the comments is:

1. Python conflicts with itself.
2. Do not try to avoid this - cope with it.
3. Use virtual environments to handle the unavoidable multiple Python versions consistently.
4. pyenv seems to work best for my purposes. Use the following to install first pyenv (1), update it (2), install a Pythons version (for this example 3.5.6) with pyenv (3), set your user-global python to the installed version (4), upgrade the Python package mangement tools (5) and install anything inside the virtual Python enviroment (6).
   1. curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
   2. pyenv update
   3. CONFIGURE_OPTS=--enable-shared pyenv install 3.5.6
   4. pyenv global 3.5.6
   5. pip install --upgrade pip setuptools wheel
   6. pip install ipython jupyter snowflake-connector-python seaborn

CONFIGURE_OPTS=--enable-shared in (3) is needed for some python packages which depend on shared libraries. seaborn will not install without it.

Note that the pyenvinstalled versions includes the package manager pip. Now every call python or pip everything references to the correct version.

When referencing python in scripts in MacOS bash and other *NIXes use #!/usr/bin/env python.

It is possible to install pyenv install 2.7.15 the latest python 2.7 and change to it whenever necessary. pyenv shell 2.7.15 switches to Python2.7 only for a particular shell session when needed to run an adhoc script.

Since I use this flow I have no headaches anymore.

The same setup works superfine so far both on a local MacOS and on a remote Ububtu based jupyter notebook server.

Disclaimer: I am not a hardcore Python fullstack developer, but use Python whenever necessary in Data Science pipelines. So for application development in Python this might not be the best solution, but for consistent management of a Data Science environment it seems to work well.

Hope this is helpful.

Answered By - Reinhard Seifert

Answer Checked By - Gilberto Lyons (WPSolving Admin)