Interpreting the output of python memory_profiler

Question

Please excuse this naive question of mine. I am trying to monitor memory usage of my python code, and have come across the promising memory_profiler package. I have a question about interpreting the output generated by @profile decorator.

Here is a sample output that I get by running my dummy code below:

dummy.py

from memory_profiler import profile

@profile
def my_func():
   a = [1] * (10 ** 6)
   b = [2] * (2 * 10 ** 7)
   del b
   return a

if __name__ == '__main__':
   my_func()

Calling dummy.py by "python dummy.py" returns the table below.

Line # Mem usage Increment Line Contents

 3      8.2 MiB      0.0 MiB   @profile
 4                             def my_func():
 5     15.8 MiB      7.6 MiB       a = [1] * (10 ** 6)
 6    168.4 MiB    152.6 MiB       b = [2] * (2 * 10 ** 7)
 7     15.8 MiB   -152.6 MiB       del b
 8     15.8 MiB      0.0 MiB       return a

My question is what does the 8.2 MiB in the first line of the table correspond to. My guess is that it is the initial memory usage by the python interpreter itself; but I am not sure. If that is the case, is there a way to have this baseline usage automatically subtracted from the memory usage of the script?

Many thanks for your time and consideration!

Noushin

Answer 1

According to the docs:

The first column represents the line number of the code that has been profiled, the second column (Mem usage) the memory usage of the Python interpreter after that line has been executed. The third column (Increment) represents the difference in memory of the current line with respect to the last one.

So, that 8.2 MiB is the memory usage after the first line has been executed. That includes the memory needed to start up Python, load your script and all of its imports (including memory_profiler itself), and so on.

There don't appear to be any documented options for removing that from each entry. But it wouldn't be too hard to post-process the results.

Alternatively, do you really need to do that? The third column shows how much additional memory has been used after each line, and either that, or the sum of that across a range of lines, seems more interesting than the difference between each line's second column and the start.

Answer 2

The difference in memory between lines is given in the second column or you could write a small script to process the output.