Using exec on a file in a different directory causes module import errors

Question

I have 3 very simple scripts. The structure looks like this:

test.py
test_problem_folder
     test_problem_1.py
     test_problem_2.py

test.py:

import os

if __name__ == "__main__":
    filename = "./test_problem_folder/test_problem_1.py"
    exec(compile(open(filename, "rb").read(), filename, 'exec'), globals(), locals())

test_problem_folder/test_problem_1.py:

import test_problem_2
test_problem_2.test()

test_problem_folder/test_problem_2.py:

def test():
    print("Hello")

If I try to run test.py, I get the error:

ModuleNotFoundError: No module named 'test_problem_2'

If I flatten the folder structure so that test_problem_* is the same directory as test.py, I don't get this problem. I figured the paths must be getting screwed up, so I tried os.chdir() to ./test_problem_folder, but that still gets the same error. What am I doing wrong? My real scenario is more complicated and I need to use exec instead of popen.

Also might be that you need to tell it to look in the same dir for test_problem_2 by using import .test_problem_2 (note the leading .) — wpercy
– wpercy, Commented Feb 7, 2017 at 21:57

Haifeng Zhang · Accepted Answer · 2017-02-07 22:21:51Z

I tried your code, if i run python test_problem_1.py under test_problem_folder, everything is working properly. Apparently, the Python path doesnt know anything about test_problem_folder

You can append abs path of test_problem_folder to your python path, then the module can be found, you dont have to have the __init__.py file under test_problem_folder

import os
import sys

if __name__ == "__main__":
    sys.path.append("/path/to/.../test_problem_folder")
    filename = "./test_problem_folder/test_problem_1.py"
    exec(compile(open(filename, "rb").read(), filename, 'exec'), globals(), locals())

Alternatively, you can append the directory of test.py to pythonpath, create __init__.py under test_problem_folder(this makes it as a python package other than directory) and then import test_problem_1 from module test_problem_folder

import os
import sys
sys.path.append(os.path.abspath(os.path.dirname(__file__)))
import test_problem_folder.test_problem_1 as problem1

if __name__ == "__main__":
    pass

That works! Is this the only way though? In reality my code will be executing some python file given as an argument, and after running this a lot I don't want to pollute the path too much
Also I noticed that this works: sys.path.append(os.path.dirname(filename)) while this doesn't: sys.path.append(os.path.abspath(filename))
@user3715648 see my edit. the second method is more often used in python project, you append your project path to pythonpath

Collectives™ on Stack Overflow

Using exec on a file in a different directory causes module import errors

1 Answer 1

3 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Linked

Related