Add application root to path

Question

I am creating a package to be used in our company that adds the root of the project to the system path.

I will add a source_root.config file in our root directory.

add_root_path.py

import sys
import inspect
from pathlib import Path
from inspect import getsourcefile

CUR_FILE = Path(getsourcefile(lambda:0)).name
SOURCE_ROOT_DEFAULT_FILE = 'source_root.config'


def _get_dir_of_importer_file():
    """ the directory of the file importing the package"""
    frames_stack = inspect.stack()
    for frame in frames_stack:
        # first file that's not importlib._bootstrap and not current file
        if 'py' in frame.filename and Path(frame.filename).name != CUR_FILE:  
            file_name = frame.filename
            break

    return Path(file_name).resolve().parent


def find_source_root(source_root_file=SOURCE_ROOT_DEFAULT_FILE):
    importer_dir = _get_dir_of_importer_file()
    while True:  # can go wrong if package is not used correctly
        files_in_dir = map(Path.resolve, importer_dir.glob('*'))
        for file in files_in_dir:
            if file.name == source_root_file:
                return importer_dir
        importer_dir = importer_dir.parent


def append_root_to_path(source_root_file=SOURCE_ROOT_DEFAULT_FILE):
    source_root = find_source_root(source_root_file)
    sys.path.append(source_root.resolve().as_posix())

This is done in order to properly import the files inside the project and because using PYTHONPATH in windows is not that convenient for pytest and alembic (meaning not running the files directly via python.exe).

Answer 1

Your project is setup wrong

Pytest

The simplest solution to most import problems is to just make your project a setuptools package. And install your package.

Whether or not your project is a library or application to fix pytest is really, really simple. You make your project a setuptools package (Which you should already have done if you're making a library). This is by configuring the setup.py file that explains to pip, setuptools, etc how to install your project.

From here you install your package and things just work.

$ pip install .
$ pytest

Applications

Now you might be saying that's cool and all it works for pytest. But now it's broken when I run my program, using python. There are two solutions to that.

• Execute your application as a module. Add a __main__.py to the top level directory which will be your application's entry point. Make sure you move your if __name__ == '__main__' code here. And then just use:

  $ python -m my_project
  

• Setup an entry point for the setuptools package.

  Once you get the above working, then all you need to do is ensure your main guard is only calling a function main. If this is the case, then you can say main is your entry point in your setup.py.

  entry_points={
      'console_scripts': [
          'my_project=my_project.__main__:main',
      ],
  },
  

  Usage is then just:

  $ my_project
  

  This is how the cool kids publish applications to PyPI.

But what about my code? Where's my code review?

Oh yeah, your code is an unneeded hack job. Seriously just make a setuptools package. You get some benefits from it like your project always being on the path, being able to install projects from a private PyPI repository, being able to use tox and nox, having a customizable entry point mapped to a cool name, and not having to use hacks to get your tests to work. I feel I'm biased here, but I really don't see any downsides. Heck I now only use pip to install to Apache.

From here you can just use either of the import strategies you want. I prefer relative imports, but absolute imports might be your jam.

1. Absolute imports.

   from my_project import subpackage
   subpackage.foo()
   

2. Cool relative imports.

   from . import subpackage
   subpackage.foo()
   

MVCE of all of the above

I remember when I was trying to convert to relative imports I Googled and all that I could find was hack jobs. People saying the names of some PEPs that talk about what I'm on about, but only talk about __name__, __files__ and '__main__'. Overall I feel the subjects a shit show mess.

And so if you're like I was and just trying to make some sense of all this nonsense, I have a small MVCE for you. If you copy the files and commands verbatim then you too can have a properly configured Python project.

./src/my_project/__init__.py

./src/my_project/__main__.py (Works with from my_project.foo import FOO too.)

from .foo import FOO


def main():
    print(FOO)


if __name__ == '__main__':
    main()

./src/my_project/foo.py

FOO = 'Bar'

./tests/test_foo.py

import my_project.foo

def test_foo():
    assert my_project.foo.FOO == 'Bar'

./setup.py

from setuptools import setup, find_packages

setup(
    name='my_project',
    packages=find_packages('src'),
    package_dir={'': 'src'},
    entry_points={
        'console_scripts': [
            'my_project=my_project.__main__:main',
        ],
    },
)

Running it

$ cd src
$ python -m my_project
Bar
$ cd ..
$ pip install .
$ pytest
===== 1 passed in 0.05s =====
$ my_project
Bar

In Addition

If you configure your project correctly then your problems should magically disappear. If you still have problems then you should double check you have installed your project.

You may want to use the -e flag when you install the project, so that you don't need to pip install . each time you change a file.

Better yet use this as a stepping stone to upgrade to modern Python development and use tox or nox to run you tests. Not only do these come with the benefit that they test your setuptools package is correctly configured. They also let you, and your coworkers, not fret over having to install a virtual environment and ensure pip install -e . has ran. Just setup a config file and then you all only need to run tox or nox for things to just work^TM.