How to save all the variables in the current python session?

Question

I want to save all the variables in my current python environment. It seems one option is to use the 'pickle' module. However, I don't want to do this for 2 reasons:

1. I have to call pickle.dump() for each variable
2. When I want to retrieve the variables, I must remember the order in which I saved the variables, and then do a pickle.load() to retrieve each variable.

I am looking for some command which would save the entire session, so that when I load this saved session, all my variables are restored. Is this possible?

Edit: I guess I don't mind calling pickle.dump() for each variable that I would like to save, but remembering the exact order in which the variables were saved seems like a big restriction. I want to avoid that.

Answer 1

If you use shelve, you do not have to remember the order in which the objects are pickled, since shelve gives you a dictionary-like object:

To shelve your work:

import shelve

T='Hiya'
val=[1,2,3]

filename='/tmp/shelve.out'
my_shelf = shelve.open(filename,'n') # 'n' for new

for key in dir():
    try:
        my_shelf[key] = globals()[key]
    except TypeError:
        #
        # __builtins__, my_shelf, and imported modules can not be shelved.
        #
        print('ERROR shelving: {0}'.format(key))
my_shelf.close()

To restore:

my_shelf = shelve.open(filename)
for key in my_shelf:
    globals()[key]=my_shelf[key]
my_shelf.close()

print(T)
# Hiya
print(val)
# [1, 2, 3]

Answer 2

Having sat here and failed to save the globals() as a dictionary, I discovered you can pickle a session using the dill library.

This can be done by using:

import dill                            #pip install dill --user
filename = 'globalsave.pkl'
dill.dump_session(filename)

# and to load the session again:
dill.load_session(filename)

Answer 3

One very easy way that might satisfy your needs. For me, it did pretty well:

Simply, click on this icon on the Variable Explorer (right side of Spider):

Answer 4

Here is a way saving the Spyder workspace variables using the spyderlib functions

#%%  Load data from .spydata file
from spyderlib.utils.iofuncs import load_dictionary

globals().update(load_dictionary(fpath)[0])
data = load_dictionary(fpath)



#%% Save data to .spydata file
from spyderlib.utils.iofuncs import save_dictionary
def variablesfilter(d):
    from spyderlib.widgets.dicteditorutils import globalsfilter
    from spyderlib.plugins.variableexplorer import VariableExplorer
    from spyderlib.baseconfig import get_conf_path, get_supported_types

    data = globals()
    settings = VariableExplorer.get_settings()

    get_supported_types()
    data = globalsfilter(data,                   
                         check_all=True,
                         filters=tuple(get_supported_types()['picklable']),
                         exclude_private=settings['exclude_private'],
                         exclude_uppercase=settings['exclude_uppercase'],
                         exclude_capitalized=settings['exclude_capitalized'],
                         exclude_unsupported=settings['exclude_unsupported'],
                         excluded_names=settings['excluded_names']+['settings','In'])
    return data

def saveglobals(filename):
    data = globalsfiltered()
    save_dictionary(data,filename)


#%%

savepath = 'test.spydata'

saveglobals(savepath) 

Let me know if it works for you. David B-H

Answer 5

What you're trying to do is to hibernate your process. This was discussed already. The conclusion is that there are several hard-to-solve problems exist while trying to do so. For example with restoring open file descriptors.

It is better to think about serialization/deserialization subsystem for your program. It is not trivial in many cases, but is far better solution in long-time perspective.

Although if I've exaggerated the problem. You can try to pickle your global variables dict. Use globals() to access the dictionary. Since it is varname-indexed you haven't to bother about the order.

Answer 6

If you want the accepted answer abstracted to function you can use:

    import shelve

    def save_workspace(filename, names_of_spaces_to_save, dict_of_values_to_save):
    '''
        filename = location to save workspace.
        names_of_spaces_to_save = use dir() from parent to save all variables in previous scope.
            -dir() = return the list of names in the current local scope
        dict_of_values_to_save = use globals() or locals() to save all variables.
            -globals() = Return a dictionary representing the current global symbol table.
            This is always the dictionary of the current module (inside a function or method,
            this is the module where it is defined, not the module from which it is called).
            -locals() = Update and return a dictionary representing the current local symbol table.
            Free variables are returned by locals() when it is called in function blocks, but not in class blocks.

        Example of globals and dir():
            >>> x = 3 #note variable value and name bellow
            >>> globals()
            {'__builtins__': <module '__builtin__' (built-in)>, '__name__': '__main__', 'x': 3, '__doc__': None, '__package__': None}
            >>> dir()
            ['__builtins__', '__doc__', '__name__', '__package__', 'x']
    '''
    print 'save_workspace'
    print 'C_hat_bests' in names_of_spaces_to_save
    print dict_of_values_to_save
    my_shelf = shelve.open(filename,'n') # 'n' for new
    for key in names_of_spaces_to_save:
        try:
            my_shelf[key] = dict_of_values_to_save[key]
        except TypeError:
            #
            # __builtins__, my_shelf, and imported modules can not be shelved.
            #
            #print('ERROR shelving: {0}'.format(key))
            pass
    my_shelf.close()

    def load_workspace(filename, parent_globals):
        '''
            filename = location to load workspace.
            parent_globals use globals() to load the workspace saved in filename to current scope.
        '''
        my_shelf = shelve.open(filename)
        for key in my_shelf:
            parent_globals[key]=my_shelf[key]
        my_shelf.close()

an example script of using this:
import my_pkg as mp

x = 3

mp.save_workspace('a', dir(), globals())

to get/load the workspace:

import my_pkg as mp

x=1

mp.load_workspace('a', globals())

print x #print 3 for me

it worked when I ran it. I will admit I don't understand dir() and globals() 100% so I am not sure if there might be some weird caveat, but so far it seems to work. Comments are welcome :)

---

after some more research if you call save_workspace as I suggested with globals and save_workspace is within a function it won't work as expected if you want to save the veriables in a local scope. For that use locals(). This happens because globals takes the globals from the module where the function is defined, not from where it is called would be my guess.

Answer 7

You can save it as a text file or a CVS file. People use Spyder for example to save variables but it has a known issue: for specific data types it fails to import down in the road.