[SOLVED] Can I communicate between two python environments? Call functions from one environment to the next?

Issue

I have a Python app that requires a list of packages with specific versions. The app is incompatible with later versions of packages like numpy, scipy etc or will require significant engineering in order to be compatible. It also utilizes external applications which require specific packages to run.

The second addon to the application requires the more updated versions of these packages. Which means that if I upgrade the one will work but not the other and vice versa.

I have done some reading and I believe I can set up virtual environments with Python (or I can use Miniconda, Anaconda or virtulenv to set up multiple environments).

The question is, can an application running in one environment call a function from a set of code meant to run in a separate environment and receive the reply? So the same way that a function call is created now, I would call the function of the code that is meant to be housed in the separate environment. Or do I need to set up the second environment with python apis and run python app as a service (or use Flask) in order to send requests and responses to the second environment?

The second option is challenging as there are many client machines this needs to be set up on and this would result in more failure points.

Thanks

Solution

No, you can't directly call a function from one Python environment to another because they are isolated from each other, you have 3 solutions to achieve communication between different Python environments

1. You can use the subprocess module to call a Python script in another environment.
2. You can set up socket communication between the two environments, one environment will act as a server and the other as a client.
3. as you said, you can set up a Flask or FastAPI application in one environment and expose functions through an API.

subprocess is the simpler one but if you want something more complex you can look at the other two, here is an example for it:

import subprocess

def call_other_environment(script_path, *args):
    other_env_python = "/path/to/other/environment/python"
    result = subprocess.run([other_env_python, script_path, *args], capture_output=True, text=True)
    return result.stdout

output = call_other_environment("other_env_script.py", "arg1", "arg2")
print(output)

Answered By - Saxtheowl

Answer Checked By - Clifford M. (WPSolving Volunteer)