[SOLVED] Permission denied when calling subprocess.call(['.', bash_path])

Issue

When I run the following code

import os
import subprocess

bash_path = os.path.expanduser('~/.bash_profile')
subprocess.call(['.',  bash_path])

I get the following error:

Traceback (most recent call last):
  File "/path/to/my/script/my_script.py", line 4, in my_func
    subprocess.call(['.',  bash_path])
  File "/Users/user/miniconda3/envs/live_auction/lib/python3.7/subprocess.py", line 323, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/Users/jisom/miniconda3/envs/live_auction/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/Users/jisom/miniconda3/envs/live_auction/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: '.'

I have also tried the variation subprocess.call(['source', bash_path]), but I get the same results with source replacing .. It appears that from the python script I don't have permissions to call source or ., but from my terminal I do.

I'm trying to reload my environment variables because my program calls another subprocess before this that updates some configuration variables, however, they're not available until I either re-source them, or restart the terminal.

How can I re-source my .bash_profile from within a python script?

Solution

There are two issues here:

1. . (and its alias, source) are not programs which could be executed by execv (or subprocess in Python). They are shell built-ins.

2. Not coincidentally, . cannot be implemented by an external program, which is why it is built-in to the shell.

The first one explains why you cannot use subprocess.call to execute . or source. subprocess.call can only execute external programs. It cannot execute a shell built-in, because there is no shell; subprocess.call is running inside a Python program, not a shell. So when you try to subprocess.call('.',...), you are trying to execute ., which is a directory. Saying that you don't have permissions to run a directory is technically true, but not very useful as an error message; directories cannot be executed, even by the root user. I would expect subprocess.call(['source', ...]) to produce a "no such file or directory" error, but perhaps you have a file called source (without execution permissions) somewhere in your execution path. (Not a good idea, since source is commonly used as a shell built-in.)

But it is really the second issue which is key. An external program, even one running as a child, cannot reach into the process in which Python is running and retroactively change the process environment variables. (Or, for that matter, the current working directory, which is why you can't subprocess.call the cd command.)

Environment variables are so-called because they are part of the execution environment. The execution environment is created along with a process, or perhaps it would be better to say that part of a process is the execution environment. Most of the execution environment is inherited from the process parent. But that does not mean that a process shares its environment with its children. Rather, the process copies its environment into the new environment created for the child. So environment variables are passed from parent to child, but the child's variables are its own independent variables; changing them does not affect the parent's environment variables, nor the environment variables of its already spawned children.

When you start a new "login shell" (which is the process which negotiates with the OS to allow you to log in), that shell executes your profile script, which customises the shell's execution environment. (If your shell is bash, it will use the bash-specific profile ~/.bash_profile script, if it exists.) From there on, newly created children of that shell -- which includes all the other processes you start, including graphical console sessions -- are started with a copy of this execution environment.

The execution of the profile script must be done with . (or equivalent) because the intent is to change the current execution environment. . can do that, because it is a bash command, not an external command running in a subprocess. It just executes each shell command in the given script as though you had typed it directly, so that it happens inside the current execution environment. But Python does not have any equivalent. Python is not a bash shell, and it has no idea what any bash command line means.

Answered By - rici

Answer Checked By - Dawn Plyler (WPSolving Volunteer)