Execute bash script that defines environment variables using Python

Question

I'm using the next code to execute a bash script that defines a lot of environment variables:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import subprocess

# Code used to get the value of variable before the call to my script
command = "echo $MYDIR"
ps = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
my_dir = ps.communicate()[0]
print "my_dir", my_dir

subprocess.Popen(["/bin/sh", "/home/user/env_file"])

# Code used to get the value of established variable
command = "echo $MYDIR"
ps = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
my_dir = ps.communicate()[0]
print "my_dir", my_dir

The content of my /home/user/env_file is the next:

#!/bin/bash

export MYDIR=/home/user
export MYDIR2=/home/user2
export MYDIR3=/home/user3
export MYDIR4=/home/user4
export MYDIR5=/home/user5
export MYDIR6=/home/user6
export MYDIR7=/home/user7
export MYDIR8=/home/user8
export MYDIR9=/home/user9
export MYDIR10=/home/user10
export MYDIR11=/home/user11
export MYDIR12=/home/user12
export MYDIR13=/home/user13
export MYDIR14=/home/user14
export MYDIR15=/home/user15
export MYDIR16=/home/user16

When I execute the Python code I do not get any value for my_dir variable. How can I perform this?

Best regards.

A program can only change the environment variables of itself and its children, not of the program that started it. Thus, no bash script you start from a Python interpreter can set variables for that Python interpreter. — Charles Duffy
– Charles Duffy, Commented Apr 19, 2017 at 15:56
Since a subprocess can't change its parent environment, you can't execute the script to do it. You need to write Python code that reads the script, parses the assignments, and then assigns to os.environ. — Barmar
– Barmar, Commented Apr 19, 2017 at 15:58
@Barmar, ...wellll. I'm hesitant to encourage someone to write their own shell parser -- lots of pitfalls down that road. I'm currently starting on an answer implemented in shell that sources an arbitrary script and writes all environment variables present at end-of-execution that weren't there at the start to stdout in NUL-delimited form. — Charles Duffy
– Charles Duffy, Commented Apr 19, 2017 at 16:00
Why do you want to do it as an external shell script? Why not just read the names and values for the variables in the python script? Do you actually need all the functions provided by the shell (starting from quoting issues and ending with stuff like command substitutions)? — ilkkachu
– ilkkachu, Commented Apr 19, 2017 at 16:05
@CharlesDuffy My comment only applies if they're literal assignments like in the question. If there's any shell logic, it's a problem. — Barmar
– Barmar, Commented Apr 19, 2017 at 16:34

Hai Vu · Accepted Answer · 2017-04-19 16:24:54Z

The bash script, env_file is executed in its own process, thus changes to the enviroment variables here does not get reflected back to the python script's process. If you can redesign your scripts to avoid this situation, it would be best. Otherwise, we have to resort to some hack and here is one of them.

#!/usr/bin/python
# -*- coding: utf-8 -*-
import os
import subprocess
import tempfile

env_file = "/tmp/env_file"
with open(env_file) as f:
    original_script_contents = f.read()

# Duplicate the env_file and add an echo statement at the end
with tempfile.NamedTemporaryFile() as temp_script:
    temp_script.write(original_script_contents)
    temp_script.write('\necho $MYDIR')
    temp_script.flush()

    my_dir = subprocess.check_output(["/bin/sh", temp_script.name])
    print 'my_dir =', my_dir

This hack involves duplicate the contents of env_file into another temporary file, then add an echo statement at the end of the duplicate file, execute it and collect the output.

Also, my assumption is the original script does not output anything. If it does, you have to parse the output and find what you are looking for.

Charles Duffy · Accepted Answer · 2017-04-19 16:42:58Z

The only way for a process to influence its parent's environment (absent ugly hacks abusing development/debugging facilities) is with that parent process's direct involvement (which is why one runs eval "$(ssh-agent)" -- evaling its output is, in this case, the cooperation needed from the parent to let ssh-agent set environment variables in the process that started it). Consider the following shell wrapper:

#!/usr/bin/env bash
# Save this script as "envvar-extraction-wrapper"
exec {orig_stdout}>&1 # create a backup of stdout
exec >&2              # and then use stderr while sourcing the child script

source "$@"           # run arbitrary script with arbitrary arguments *in this shell*
                      # note that this means that if it runs "exit", we abort prematurely

while read -r varname; do
  printf '%s\0%s\0' "$varname" "${!varname}" >&$orig_stdout
done < <(compgen -e)

...called from the following Python:

import subprocess, itertools, os

ps = subprocess.Popen(['envvar-extraction-wrapper', '/home/user/env_file'],
                      stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
(stdout, stderr) = ps.communicate()
items_iter = iter(stdout.split('\0'))
for (k,v) in itertools.izip(items_iter, items_iter):
    os.environ[k]=v

Collectives™ on Stack Overflow

Execute bash script that defines environment variables using Python

2 Answers 2

Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Linked

Related