268

I'm working on a documentation (personal) for nested matplotlib (MPL) library, which differs from MPL own provided, by interested submodule packages. I'm writing Python script which I hope will automate document generation from future MPL releases.

I selected interested submodules/packages and want to list their main classes from which I'll generate list and process it with pydoc.

The problem is that I can't find a way to instruct Python to load a submodule from a string. Here is an example of what I tried:

import matplotlib.text as text
x = dir(text)
i = __import__('matplotlib.text')
y = dir(i)
j = __import__('matplotlib')
z = dir(j)

And here is a 3-way comparison of above lists through pprint:

enter image description here

I don't understand what's loaded in y object - it's base matplotlib plus something else, but it lacks information that I wanted and that is main classes from matplotlib.text package. It's the top blue coloured part on screenshot (x list).

3
  • Can you explain why you need to use __import__(str) rather than the standard import statemetn? Commented Jan 3, 2012 at 21:53
  • It's because I'll process lists which items are MPL submodules and get their methods paths Commented Jan 3, 2012 at 22:00
  • 23
    @thesamet - c'mon - there are endless ideas where you'd want this functionality. When you have a textual configuration of libraries, you can load them by name, which wouldn't quite work with the import statement. Here's one example of use: djangosnippets.org/snippets/3048 Commented May 12, 2014 at 4:55

8 Answers 8

362

The __import__ function can be a bit hard to understand.

If you change

i = __import__('matplotlib.text')

to

i = __import__('matplotlib.text', fromlist=[''])

then i will refer to matplotlib.text.

In Python 3.1 or later, you can use importlib:

import importlib

i = importlib.import_module("matplotlib.text")

Some notes

  • If you're trying to import something from a sub-folder e.g. ./feature/email.py, the code will look like importlib.import_module("feature.email")

  • Before Python 3.3 you could not import anything if there was no __init__.py in the folder with file you were trying to import (see caveats before deciding if you want to keep the file for backward compatibility e.g. with pytest).

Sign up to request clarification or add additional context in comments.

5 Comments

importlib should be avaliable on pypi for < python2.7
@mzjn This is for import moduleName where moduleName is string. How about from moduleName import * ?
Just found answer for my question here in case any one needs it stackoverflow.com/a/31306598/248616
I found the answer on how to import from parent directory here.
Due to implicit namespace packages, it is possible to import from folders without an __init__.py.
119

importlib.import_module is what you are looking for. It returns the imported module.

import importlib

# equiv. of your `import matplotlib.text as text`
text = importlib.import_module('matplotlib.text')

You can thereafter access anything in the module as text.myclass, text.myfunction, etc.

1 Comment

6

spent some time trying to import modules from a list, and this is the thread that got me most of the way there - but I didnt grasp the use of ___import____ -

so here's how to import a module from a string, and get the same behavior as just import. And try/except the error case, too. :)

  pipmodules = ['pycurl', 'ansible', 'bad_module_no_beer']
  for module in pipmodules:
      try:
          # because we want to import using a variable, do it this way
          module_obj = __import__(module)
          # create a global object containging our module
          globals()[module] = module_obj
      except ImportError:
          sys.stderr.write("ERROR: missing python module: " + module + "\n")
          sys.exit(1)

and yes, for python 2.7> you have other options - but for 2.6<, this works.

Comments

4

Apart from using the importlib one can also use exec method to import a module from a string variable.

Here I am showing an example of importing the combinations method from itertools package using the exec method:

MODULES = [
    ['itertools','combinations'],
]

for ITEM in MODULES:
    import_str = "from {0} import {1}".format(ITEM[0],', '.join(str(i) for i in ITEM[1:]))
    exec(import_str)

ar = list(combinations([1, 2, 3, 4], 2))
for elements in ar:
    print(elements)

Output:

(1, 2)
(1, 3)
(1, 4)
(2, 3)
(2, 4)
(3, 4)

Comments

3

You can also use exec built-in function that execute any string as a Python code.

In [1]: module = 'pandas'
   ...: function = 'DataFrame'
   ...: alias = 'DF'

In [2]: exec(f"from {module} import {function} as {alias}")

In [3]: DF
Out[3]: pandas.core.frame.DataFrame

For me this was the most readable way to solve my problem.

1 Comment

This is the solution. However, if we need to handle user input, we should not pass it to the exec function, in a such situation, see my answer: stackoverflow.com/a/79414611/1908192
1

Module auto-install & import from list

Below script works fine with both submodules and pseudo submodules.

# PyPI imports
import pkg_resources, subprocess, sys

modules   = {'lxml.etree', 'pandas', 'screeninfo'}
required  = {m.split('.')[0] for m in modules}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing   = required - installed

if missing:
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--upgrade', 'pip'])
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])

for module in set.union(required, modules):
    globals()[module] = __import__(module)

Tests:

print(pandas.__version__)
print(lxml.etree.LXML_VERSION)

Comments

0

I developed these 3 useful functions:

def loadModule(moduleName):
    module = None
    try:
        import sys
        del sys.modules[moduleName]
    except BaseException as err:
        pass
    try:
        import importlib
        module = importlib.import_module(moduleName)
    except BaseException as err:
        serr = str(err)
        print("Error to load the module '" + moduleName + "': " + serr)
    return module

def reloadModule(moduleName):
    module = loadModule(moduleName)
    moduleName, modulePath = str(module).replace("' from '", "||").replace("<module '", '').replace("'>", '').split("||")
    if (modulePath.endswith(".pyc")):
        import os
        os.remove(modulePath)
        module = loadModule(moduleName)
    return module

def getInstance(moduleName, param1, param2, param3):
    module = reloadModule(moduleName)
    instance = eval("module." + moduleName + "(param1, param2, param3)")
    return instance

And everytime I want to reload a new instance I just have to call getInstance() like this:

myInstance = getInstance("MyModule", myParam1, myParam2, myParam3)

Finally I can call all the functions inside the new Instance:

myInstance.aFunction()

The only specificity here is to customize the params list (param1, param2, param3) of your instance.

Comments

0

from <mymodule> import x, y, z

Since I do not see a safe solution mimicking from <mymodule> import x, y, z, where mymodule is given as user input, this is what I suggest:

mymodule = input()

# pseudocode: from <mymodule> import x, y, z
symbols = ['x', 'y', 'z']
mymodule_ = __import__(mymodule, fromlist=symbols)
for s in symbols:
    globals()[s] = eval("mymodule_.%s" % s)

Explanation:

One cannot subscript a module in Python (e.g., mymodule_[symbols[1]]), therefore need for an eval/exec function. We do not want a user input there as a safety measure though. Thus it is done in two steps: first, import the module; second, expose the symbols.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.