Python: Importing everything from a python namespace / package

Question

all.

I'd think that this could be answered easily, but it isn't. As long as I've been searching for an answer, I keep thinking that I'm overlooking something simple.

I have a python workspace with the following package structure:

    MyTestProject
        /src
           /TestProjectNamespace
              __init__.py
              Module_A.py
              Module_B.py

    SecondTestProject
        /src
           /SecondTestProjectNamespace
              __init__.py
              Module_1.py
              Module_2.py
              ...
              Module_10.py

Note that MyTestProjectNamespace has a reference to SecondTestProjectNamespace.

In MyTestProjectNamespace, I need to import everything in SecondTestProjectNamespace. I could import one module at a time with the following statement(s):

    from SecondTestProjectNamespace.Module_A import *
    from SecondTestProjectNamespace.Module_B import *

...but this isn't practical if the SecondTestProject has 50 modules in it.

Does Python support a way to import everything in a namespace / package? Any help would be appreciated.

Thanks in advance.

Answer 1

Yes, you can roll this using pkgutil.

Here's an example that lists all packages under twisted (except tests), and imports them:

# -*- Mode: Python -*-
# vi:si:et:sw=4:sts=4:ts=4

import pkgutil
import twisted

for importer, modname, ispkg in pkgutil.walk_packages(
    path=twisted.__path__,
    prefix=twisted.__name__+'.',
    onerror=lambda x: None):
        # skip tests
        if modname.find('test') > -1: 
            continue
        print(modname)
        # gloss over import errors
        try:
            __import__(modname)
        except:
            print 'Failed importing', modname
            pass

# show that we actually imported all these, by showing one subpackage is imported
print twisted.python

I have to agree with the other posters that star imports are a bad idea.

Answer 2

No. It is possible to set up SecondTestProject to automatically import everything in its submodules, by putting code in __init__.py to do the from ... import * you mention. It's also possible to automate this to some extent using the __import__ function and/or the imp module. But there is no quick and easy way to take a package that isn't set up this way and make it work this way.

It's probably not a good idea anyway. If you have 50 modules, importing everything from all of them into your global namespace is going to cause a proliferation of names, and very likely conflicts among those names.

Answer 3

As other had put it - it might not be a good idea. But there are ways of keeping your namespaces and therefore avoiding naming conflicts - and having all the modules/sub-packages in a module available to the package user with a single import.

Let's suppose I have a package named "pack", within it a module named "a.py" defining some "b" variable. All I want to do is :

>>> import pack
>>> pack.a.b
1

One way of doing this is to put in pack/__init__.py a line that says import a - thus in your case you'd need fifty such lines, and keep them up to date.

Not that bad.

However, the documentation at http://docs.python.org/tutorial/modules.html#importing-from-a-package - says that if you have a string list named __all__ in your __init__.py file, all module/sub-package names in that list are imported when one does from pack import *

That alone would half-work - but would require users of your package to perform the not-recommended "from x import *" form.

But -- you can do the "... import *" inside __init__.py itself, after defining the __all__ variable - so all you have to do is to keep the __all__ up to date:

With the TestProjectNamespace/__init__.py being like this:

__all__ = ["Module_A", "Module_B", ...]
from TestProjectNamespace import *

your users would have TestProjectNamespace.Module_A (and others) available upon import of TestProjectNamespace.

And, of course - you could automate the creation of __all__ - it is just a variable, after all - but I would not recommend that.

Answer 4

Does Python support a way to import everything in a namespace / package?

No. A package is not a super-module -- it's a collection of modules grouped together.

---

At least part of the reason is that it's not trivial to determine what 'everything' means inside a folder: there are problems like network drives, soft links, hard links, ...