[SOLVED] Running Python script with temporary environment variables

Issue

I have a Python application and it utilizes environment variables in Kubernetes configuration such as:

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
  namespace: default
data:
  var1: foo
  var2: bar

---

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  namespace: default
spec:
  containers:
    - envFrom:
      - configMapRef:
          name: my-config

So, it is fine when the app is Dockerized and running on a Kubernetes farm.

However, when running the application on a local machine without Docker and Kubernetes but a humble command:

python app.py

I have to make the Python modules find the environment variables with os.getenv('var1') even if there is no ConfigMap or Pod on.

Is it possible to without a need to add extra code in Python modules or adding environment variables to the local machine system?

Solution

You will need to have a "loader" small program that will invoke the second program with the appropriate variables. Of course it is more convenient to have it in Python as well, as everything is already setup - but it could be something along

#!/usr/bin/env python3
import subprocess
import sys

import yaml

config = yaml.load(open(sys.argv[1]))["data"]  
# Instead of "data" here, use the yaml path to the place in the configuration your variables are

cmdline = sys.argv[2:]
if cmdline[0].endswith(".py"):
    cmdline.insert(0, sys.executable)

result = subprocess.run(cmdline, env=config)
exit(result.returncode)

(Of course you need to have installed some 3rd party lib to read the yaml config files - I've used PyYAML to test this)

If you mark the above script as exectuable, you can use it directly in your command line, without prefixing "python3" - otherwise, if you name this file "runner.py" your command line can be

python3 runner.py myconfig.yaml myscript.py parameter1 parameter2 parameter3

If you need to pass the current enviroments and just update then with the variables in the script, then do that before the .run call:

import os
...
config = yaml.load(...)[...]
config = {**os.environ, **config}

Also, see that this will use the same Python interpreter it is running on to execute the script, in a more or less naive way (just checks the literal ".py" file extension) - if you need this to be more robust, it is recommended to perform an stat on the target script and check if it is executable - then call it directly - and only use the current Python interpreter for other files.

or modify your code

Since you have a single script as entry points, and want to simplify running it for 3rdy paties, you could change it instead of having a generic loader - in that case, I'd recomment adding an extra config variable just to indicate the variables are already properly set - so when the script run in its container, it does nothing - otherwise, it loads the data as above, and uptates its os.envrion.

In other words, at the entry point for your code, do something along:

if not os.environ.get("EVERYTHING_SETUP"):
   import yaml
   config = yaml.load(open("myconfig.yaml"))["data"] 
   os.environ.update(config)

Answered By - jsbueno

Answer Checked By - Marilyn (WPSolving Volunteer)