python memory allocation trace

Question

My code allocates continously memory (~12kb per second). With a runtime of 8 hours its a lot of memory!!

Because of this I want to trace the moments/code lines my python code allocates memory.

Something like you can do with processed code lines with:

python -m trace --count -C ./tmp code.py

this generate a view where you can see how often this line was executed. It looks like:

code.cover

1:     import sys
1:     import os
1534:  while 1:
1534:      print "foo"

I need this for memory allocation. if possible something like

1245 B    import sys
893 B     import os
17.46 KB  import somecode

Answer 1

Looks like this question has already been answered here: Python memory profiler

Maybe this one can help you: http://pypi.python.org/pypi/memory_profiler

From the docs, execute the code passing the option -m memory_profiler to the python interpreter to load the memory_profiler module and print to stdout the line-by-line analysis. If the file name was example.py, this would result in:

$ python -m memory_profiler example.py

Output will follow:

Line #    Mem usage  Increment   Line Contents
==============================================
     3                           @profile
     4      5.97 MB    0.00 MB   def my_func():
     5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
     6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
     7     13.61 MB -152.59 MB       del b
     8     13.61 MB    0.00 MB       return a