Python import all functions from files within a directory into one namespace

Question

I'm trying to make an __init__.py file which will import all functions from *.py files from a directory into a namespace (the directory's name). The logic is shown below:

for f in os.listdir(wd):
    if not f.endswith('.py') or f == '__init__.py':
        continue
    names = get_public_functions(open(wd + f))
    try:
        mod = __import__(f[:-3], fromlist=names)
        for fn in names:
            fun = getattr(mod, fn)
            setattr(sys.modules[__name__], fn, fun)
    except Exception as e: 
        for fn in names:
            setattr(sys.modules[__name__], fn, lambda: str(e))

So, you can observe that if there are syntax error in a file, the functions will still be imported, but they will return the syntax error (as a string).

What's frustrating, is that when there are syntax errors in multiple files, while I'm expecting something like:

mymodule.fn() => error1,
mymodule.fn2() => error1 (these were from the first file),
mymodule.fn3() => error2 etc.

I get only the last error message. I think the error must be in the except block, but I can't figure it. Can anyone help?

possible duplicate of Local variables in Python nested functions — Martijn Pieters
– Martijn Pieters, Commented Mar 11, 2013 at 15:34

Martijn Pieters · Accepted Answer · 2013-03-11 15:30:20Z

1

You need to bind the value of e to the scope of the lambda you create. Or rather, you want to bind the str(e) result to the lambda:

error = str(e)
for fn in names:
    setattr(sys.modules[__name__], fn, lambda error=error: error

Now each lambda has a keyword parameter storing the then-current value of error. Because the error parameter is a keyword parameter with a default value, your lambdas still work when called with arguments.

The alternative would be to create a new function scope:

def errorfunction(error):
    return lambda: error

error = str(e)
for fn in names:
    setattr(sys.modules[__name__], fn, errorfunction(error)

edited Mar 11, 2013 at 15:30

answered Mar 11, 2013 at 15:24

Martijn Pieters

1.1m325 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

kaspersky Over a year ago

Is it possible to make the functions without arguments?

Martijn Pieters Over a year ago

@gg.kaspersky: Not without creating another nested scope. Your function can be called without an argument (it has a default value).

kaspersky Over a year ago

Yes, the default argument solve it. So, was the error because of for? I recall reading something about this...

Martijn Pieters Over a year ago

@gg.kaspersky: The outer loop can assign another exception to e, and you don't call your lambdas until the loop is already done. The lambdas all refer to the same e variable (a cell reference), so they all, in the end, point to the same exception, the last one raised in the loops.

Martijn Pieters Over a year ago

@gg.kaspersky: I've marked your question a dupe where I explained this in more detail; I couldn't find that one at first.

Collectives™ on Stack Overflow

Python import all functions from files within a directory into one namespace

1 Answer 1

5 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Related