[SOLVED] Installing Numba on a Cluster (Compute Canada) using virtualenv

Issue

I am trying to install Numba on a cluster so I can run my Jitted python codes on there. However I keep running into an error with "libllvmlite.so" when I try to import Numba. This is done on the Cedar cluster using virtualenv. Numba is installed using pip (and also tried pip3).

I start off my activating my enviroment using (... is the directory to my enviroment folder) : source ~/.../ENV/bin/activate. Then I use pip to install Numba (I've also tried used pip uninstall to remove and reinstall Numba). However when I load up Python 3.7.0 on my enviroment, I obtain an error whenever I try to import Numba.

I use pip show to check versions: Python 3.7.0, Numba 0.53.1, llvmlite 0.36.0.

Here is the output of my terminal:

(ENV) [04:12:46] patrick7@cedar1 > [~/projects/def-mann/patrick7/diffusioncluster]python
Python 3.7.0 (default, Sep 25 2018, 18:19:16)
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numba
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/patrick7/projects/def-mann/patrick7/diffusioncluster/ENV/lib/python3.7/site-packages/numba/__init__.py", line 19, in <module>
    from numba.core import config
  File "/home/patrick7/projects/def-mann/patrick7/diffusioncluster/ENV/lib/python3.7/site-packages/numba/core/config.py", line 16, in <module>
    import llvmlite.binding as ll
  File "/home/patrick7/projects/def-mann/patrick7/diffusioncluster/ENV/lib/python3.7/site-packages/llvmlite/binding/__init__.py", line 4, in <module>
    from .dylib import *
  File "/home/patrick7/projects/def-mann/patrick7/diffusioncluster/ENV/lib/python3.7/site-packages/llvmlite/binding/dylib.py", line 3, in <module>
    from llvmlite.binding import ffi
  File "/home/patrick7/projects/def-mann/patrick7/diffusioncluster/ENV/lib/python3.7/site-packages/llvmlite/binding/ffi.py", line 191, in <module>
    raise OSError("Could not load shared object file: {}".format(_lib_name))
OSError: Could not load shared object file: libllvmlite.so

I have tried other solutions found online (reinstall, try different versions...), so far none has worked. Any help would be greatly appreciated! Thanks!

Solution

I figured out the solution to my problem.

What I've tried and didn't work:

• Install llvmlite using pip manually
• Downgrade to Numba version 0.43 since its the oldest version compatible with Python 3.7
• Uninstall (llvmlite numpy and numba) and reinstalling Numba (which automatically installs it's dependancies) after upgrading Pip to the latest version

What worked:

• Upgrading Python from 3.7.0 to 3.7.10

It seems like for Python version >= 3.7.3, the default command pip install numba works flawlessly!

Thanks for reading!

Answered By - patrick7

Answer Checked By - Cary Denson (WPSolving Admin)