2

I have recently started using the python memory profiler from here. As a test run, I tried to profile the toy code from here following the instructions therein. I have some naive questions on the outputs I saw.

import time

@profile
def test1():
    n = 10000
    a = [1] * n
    time.sleep(1)
    return a

@profile
def test2():
    n = 100000
    b = [1] * n
    time.sleep(1)
    return b

if __name__ == "__main__":
    test1()
    test2()

This is the output using mprof run and then plot command line options: With function profiling

After removing the @profile lines, I ran the profiler again and obtained the following result: Without function profiling

Except for the brackets for the functions, I was expecting almost identical plots (since the code is simple), but I am seeing some significant differences such as the ending time of the plot, variations within brackets etc.

Can someone please shed light into these differences?

Edit: For small intervals, the plot with function profiling looks like: small intervals

1 Answer 1

2

The differences you are seeing are probably due to the fact that the information stored by @profile is counted within the total memory used by the program. There is also a slight overhead of storing this information, hence the different running times.

Also, you might get slightly different plots in different runs just due to variations in how Python manages memory.

Sign up to request clarification or add additional context in comments.

4 Comments

Thanks for the answer. Just wanted to make sure I am not missing anything significant. Also, just curious: shouldn't there be a line towards the end in both cases, showing the drop in memory usage?
Indeed, if you give it a sufficiently small time interval you should see the drop in memory usage. Try running it for example with -T 0.001: mprof run -T 0.001 test1.py
Yes, I am now seeing the drop in memory on the one with the functions profiled; have added that plot also for future users. BTW, I am getting an error when I try plotting the .dat output without function profiling. I am submitting the details in the github issue tracker.
Fabian: found the problem area. I forced the mem_line_label to an empty string in the function plot_file, and that was creating the error.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.