In a python project, when should you use __init__.py, __main__.py, and just normal .py?

Question

Say I am making a pdf editor app with the following structure:

├── main.py
├── 📂 drawing_functions/
├── 📂 util/
├── 📂 GUI/
└── 📂 document_handling/

Each of these folders have a collection of python code, and more subfolders.

Now there are a few ways to architect this:

1. Each of the folders can have an __init__.py, and treat each folder as a module
2. Each of the folders can have their own main.py, e.g. GUI/gui_main.py
3. Each of the folders can have their own __main__.py
4. The root directory main.py could be __main__.py

But I am not sure how to make these architecture choices. When should you use __init__.py vs __main__.py? I know what they do, literally. __init__.py is for declaring regular packages and are immediately executed on load, __main__.py will allow you to execute from the command line with -m. But if you import GUI.gui_main then the contents will be executed just like as if you imported GUI with an __init__.py. It's also totally possible to architect a project without these. When should you be using them and when should you not?

Answer 1

Understanding what each file does will help you understand when (or if) to use it.

__init__.py is used to define a module when you can't use an ordinary script. You could define a module named foo using either foo/__init__.py or foo.py, but the former is much more convenient when you also want to define submodules like bar in the package foo. (You'd write foo/__init__.py and foo/bar.py, rather than foo.py and foo/bar.py.) Contrary to popular belief, an __init__.py is not necessary to define a package.

__main__.py lets you treat a directory as a script, whether or not that directory is also defining a package. If you have foo/__main__.py, you can treat foo itself as the script and execute python3 foo. This also works with a ZIP file; if __main__.py exists at the top level of the archive, then the ZIP file itself can be treated as a script.

Mentioning something like GUI/gui_main.py suggests that you are incorrectly storing a script in a directory meant to define a package. Either move gui_main.py outside the package hierarchy, or use GUI/__main__.py to allow GUI itself (rather than one of its submodules) as a script.

Answer 2

EDIT: With the concept "package" is intended any directory that contains code files even in its subdirectories.

The problem

Having a so much powerful language (in terms of "freedom of design" capabilities) can be stunning sometimes.

In fact when i faced Python to develop large projects (mostly libraries) i had to make a choice of design, because as you said you can use any of those architectures.

My personal solution

According to me every software system (a big big program, or maybe even a smaller one) should be designed the most decoupled it can be.

With this technique you will end up wanting to have all leaf-packages (packages that directly contains code files, not other packages) to be independent, like mini libraries that performs each one a service or a set of related services.

This makes us naturally think to design every leaf-package as a module, containing an __init__.py file that establishes what to expose and how from the module implementation. The rest of non-leaf packages will then contain other packages and an __init__.py file just to expose those packages in the namespace that is importing them.