Issue
What is a best practice to manage a different versions of packages? I know about virtualenv but i am not sure it is suitable in my case.
My problem:
Consider that i have 2 projects (P1, P2) which both use 1 small project (P3). I use git submodules and add P3 to both P1 and P2.
Then i have P4 which uses all projects described above and P4 needs the latest version of P3.
How to deal with it? I want P1, P2, P4 to use their own version of P3. But when i build P4 i have only one version of P3.
Projects structure:
All projects have such structure (some files omitted):
P4 example:
├── project_name (sources are here)
├── Makefile
├── submodules
│ └── P1
│ ├── submodules
│ │ └── P3
│ └── P2
│ ├── submodules
│ │ └── P3
│ └── P3
├── tests
└── setup.py
How subprojects are imported:
No surprises here, all projects import P3 in that way as it was the only one version installed.
So P1 uses:
from P3 import something
And P4 also uses:
from P3 import something
from P1 import something_else
Solution
The general rule is that running multiple versions of the same project within a single Python environment is not supported. Obviously (like any other kind of limitation in software) there are ways to go around it, but they all require quite a bit of work.
From what I can gather from this particular question, I believe vendoring could be a fitting technique to circumvent this limitation while requiring as little modifications as possible to the current state of the project.
1. with git submodules
- Keep P3 in a different git repository.
- Use git submodules to have two different copies of P3 in the main project.
2. with the vendoring tool
- Promote P3 to its own Python project with complete packaging.
- Use the vendoring tool to have two different copies of P3 in the main project. See for example how pip uses it in "Switch to a dedicated tool for vendoring pip's dependencies #7485"
What is a best practice to manage a different versions of packages? I know about virtualenv but i am not sure it is suitable in my case.
It is not. Virtual environments could help for example to work on two different projects using two different versions of the same library. But the two projects would need to live in two different virtual environments. So you couldn't have one single project importing simultaneously from two different versions of a same library.
Answered By - sinoroc