The ad-hoc config system that we currently have relies on each deployment (development, development-workers, production, production-workers, CI and maybe some I'm missing) to have their own copies of a credentials.py file in the python source directory/module tree. Additionally, within this system there is a further "split" between PRODUCTION, DEVELOPMENT. and TEST environments. This means that keys are duplicated over and over, and it's unclear which ones are necessary or need to be edited for each environment.

This system creates friction when adding new services or dependencies that need to be configured. It's also error prone when updating workers or CI. Finally, it is nearly impossible to explain succinctly to potential new developers who wish to work on the project.

I propose a migration to dotenv (https://pypi.org/project/python-dotenv/), which is more of an industry standard solution. We can use --env-file .env as an argument to our development docker containers. In CI, we can just load a .env file as a set of standard environment variables. Even in testing, we can manually configure the enviornment by loading a specific .env.test file in the Pytest runtime, to avoid this awful hack.