18

Once in a while I run into a very difficult-to-debug problem: there's a leftover .pyc file somewhere in my $PYTHONPATH, and the matching .py file has been moved to somewhere else that's later in $PYTHONPATH - so when I try to import the module, the "orphaned" .pyc file is used and all changes to the "real" .py file are ignored, leaving me incredibly confused until I figure out that's what's happening.

Is there any way of making python not use "orphaned" .pyc files, or print a warning when using them?
Alternatively, does the fact that I have this problem at all mean I'm doing something wrong, and if so, what?

Improve this question
2
  • 2
    That will NOT remove pyc files from sub directories (shell expands the *.pyc first before the rm command sees the arguments). use find * -name '*.pyc' | xargs rm -f instead. Note 1. I use find * rather than find . so that .git etc isn't searched, 2. I delete all pyc files since python appears to only check the source is older than the compiled files, rather than matches the mod time exactly (since it doesn't make the mtime match when creating pyc files - under linux at least). Commented Nov 21, 2014 at 3:01
  • 1
    find . -name '*pyc' -delete - apparently find has a flag for deleting found files Commented May 6, 2015 at 8:02

2 Answers 2

Reset to default
6

Try this (see here):

PYTHONDONTWRITEBYTECODE

If this is set, Python won’t try to write .pyc or .pyo files on the import of source modules. This is equivalent to specifying the -B option.

New in version 2.6.

But there is one issue associated with it: you will loose the benefit of having .pyc files (thus losing performance). The better, cleaner and friendlier way is to walk through the directory and clean orphaned .pyc files you do not need. You should do it with a script and be sure you do not have .pyc files that are not meant to be associated with .py ones (eg. for some level of obfuscation).

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

This prevents the writing of new .pyc files, but does it prevent python loading them?
This does not prevent Python from loading existing .pyc files. I just tested this with Python 2.6.8 and 2.7.3
I was afraid those were my only options... Good to know about the PYTHONDONTWRITEBYTECODE env variable, though! I may use that for now (after removing all existing .pyc files), until I get around to writing a cleaning script and sticking it in my cron or something.
6

You cannot prevent Python from loading .pyc files if it finds them in it's path, no.

You have a few options:

  1. Import the offending module and figure out what it's path is:

    >>> import borkenmod
>>> import sys
>>> sys.modules[borkenmod.__name__].__file__
'lib/python2.7/borkenmod.pyc'

  2. Use a script to walk your PYTHONPATH and delete stale bytecode files. The linked script removes all .pyc files for which there is no corresponding .py file present.

  3. Wipe all .pyc files in your PYTHONPATH, and have Python regenerate them with the compileall module:

    python -m compileall [path]

    then remove that one file.

Do note that the last two options could result in the removal of legitimate python modules! Some python modules (especially commercially licenced) are distributed as compiled bytecode only.

Improve this answer

1 Comment

Wipe+compileall is probably overkill - I should probably just write something to only wipe the orphaned ones. Thanks for the sys.modules trick! That should come in useful - at least once I figure out the wrong file is being loaded.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.