.. title: Change and retrieve shell environment in Python
.. slug: change-and-retrieve-bash-environment-in-python
.. date: 2019-08-16 12:32:37 UTC+02:00
.. tags: 
.. category: blog
.. link: 
.. description: 
.. type: text

Python is a very versatile language and it is generally speaking easing to interface it with other languages and also to interact with shell and the system. When you interact with your system, problem is a subshell is launched, meaning that all the environment modifications you may do will be lost and you cannot launch independent shell commands that are linked (through the environment). This problem was why I ask this `Stackoverflow <https://stackoverflow.com/questions/54342237/run-multiple-bash-lines-in-python-and-separetely-check-their-status-and-output>`_ question, that I answered myself in the end.

.. TEASER_END

The problem
===========

A simple working example of what we would like to do is the following (defining a variable, trying then to print it):
   
.. code-block:: python

    from subprocess import getstatusoutput as cmd
    
    stat, out = cmd("export TEST=1")
    stat, out = cmd("echo $TEST")

where we expect to get `1`... but experience shows that `$TEST` is empty:
    
.. code-block:: python

    >>> print(out)
    (0, "")

that can be explained by the fact that each command is launched in a subshell, and the environment was not passed from on to the other.

A solution
===========

The idea, quite simple, is to retrieve the environment... printing it. Then, parsing this output, you can get all the environment variables and communicate them to the next subshell. The solution is only two functions:

- A `launch` function to wrap a `subprocess` command to launch shell commands. Using this, we'll retrieve the environment with a `printenv` at the end of the executed command(s).
- A `get_env` function to parse the output from the last function and stock variables in a dictionary.

.. code-block:: python

    import os
    import subprocess as sp
    
    def launch(cmd_, env=os.environ, get_env=False):
        if get_env: cmd_ += " && printenv"
        load = sp.Popen(cmd_, shell=True, stdout=sp.PIPE, stderr=sp.PIPE, env=env)
        out  = load.communicate()
        err  = load.returncode
        return(err, out)

We can see that `launch` takes three arguments:

- The shell command you intend to launch.
- The environment, that is nothing more than a dictionary containing variables.
- A boolean to specify if we want to retrieve the environment.

.. code-block:: python
    
    def get_env(out, encoding='utf-8'):
        lout = str(out[0], encoding).split('\n')
        new_env = {}
        for line in lout:
            if len(line.split('=')) <= 1:
                pass
            else:
                k = line.split("=")[0]
                v = "=".join(line.split("=")[1:])
                new_env[k] = v
        return new_env

The `get_env` function simply takes as argument the output from the previous function and an encoding (to avoid bad surprises).

In the end, it can be used as followed:

.. code-block:: python
	
    err, out = launch("export TEST=1", get_env=True)
    if not err: new_env = get_env(out)
    err, out = launch("echo $TEST", env=new_env)

And now `$TEST` is known!
    
.. code-block:: python

    >>> print(str(out[0], encoding='utf-8'))
    1

We can write something a bit more complicated, in case of there are functions in the environment. A more complete and complex version can be found on my `GitHub <https://github.com/marcjoos-phd/pymisc/tree/master/getenv>`_.