27 questions
2
votes
0
answers
62
views
How can I dynamically trace and visualize memory allocation patterns of nested Python coroutines in real-time?
I'm trying to understand how memory is used in my async Python application. I have multiple coroutines running, and I want to see how much memory each one is using, especially when they are nested or ...
- 218
0
votes
0
answers
130
views
RuntimeWarning: Enable tracemalloc to get the object allocation traceback?
I am building a Telegram bot. This code is working:
import telebot
bot = telebot.TeleBot("TOKEN")
@bot.message_handler(func=lambda newMessage: True)
def main(newMessage):
print(...
- 13
1
vote
0
answers
254
views
Why does my python app use far more memory than tracemalloc suggests? [duplicate]
I trying to debug a memory leak in my python ML app. When I run it locally, the memory usage on my local machine increases gradually by 700mb. So I put a breakpoint in when memory use is around its ...
- 45
2
votes
0
answers
223
views
Understanding Python's memory allocation
I'm trying to track down a suspected memory leak in a Python application which uses numpy and pandas as two of the main libraries. I can see that the application uses more and more memory over time (...
- 8,252
1
vote
1
answer
1k
views
Tracking Memory Usage in Python
Python code:
def my_function():
my_list = [i for i in range(1_000_000)]
return my_list
my_list = my_function()
Memory usage reported by different libraries:
memory_profiler: 82MB
tracemalloc:...
- 139
0
votes
1
answer
2k
views
Is it correct that tracemalloc does not show the actual memory usage in python?
The official document says, "The tracemalloc module is a debug tool to trace memory blocks allocated by Python."
Python uses memory management as Arena and stores through pools and blocks.
...
- 169
2
votes
1
answer
583
views
ResourceWarning: unclosed socket <zmq.Socket(zmq.PUSH) when Threading
So I have an issue with the import of a driver, which apparently leads to almost all threads that I open to end in a Resourcewarning. (python version 3.9.16 on Windows in Anaconda environment; nidaqmx ...
- 21
1
vote
1
answer
700
views
python3 -- RAM usage (htop and tracemalloc give different values)
I have a code where I need to save RAM usage so I've been tracing RAM usage through tracemalloc.get_traced_memory. However, I have found that what tracemalloc.get_traced_memory gives is very different ...
- 11
1
vote
0
answers
423
views
Can I use tracemalloc to line-by-line profile a subprocess?
I am attempting to write a generic,python script that memory profiles a target script, but I can't figure out how to get line-by-line memory stats from the script being run by subprocess.
# ...
- 11
0
votes
0
answers
505
views
Precision about tracemalloc and memory consumption with python
I don't exactly understand how python use memory, and what Python gives with the tracemalloc builtins.
Let's take this exemple:
import tracemalloc
import gc
L=[1,2,3]
tracemalloc.start()
gc.collect()
...
- 1
0
votes
0
answers
426
views
sort tracemalloc statistics by size
I have this code:
def print_statistics(s: Snapshot, s2: Snapshot):
stats = s2.compare_to(s, 'lineno')
for stat in stats[:20]:
print(stat)
def some_func()
tracemalloc.start(50)
...
- 437
-1
votes
1
answer
2k
views
How to fix memory leak of flask application with tracemalloc
I have a flask application that makes requests to retrieve data and then exports the data as an excel file with openpyxl.
After ~ 50 exported excel files the flask application exceeds the RAM of 8GB ...
- 41
1
vote
1
answer
1k
views
long-running python program ram usage
I am currently working on a project where a python program is supposed to be running for several days, essentially in an endless loop until an user intervenes.
I have observed that the ram usage (as ...
- 31
6
votes
1
answer
1k
views
Does Python not reuse memory here? What does tracemalloc's output mean?
I create a list of a million int objects, then replace each with its negated value. tracemalloc reports 28 MB extra memory (28 bytes per new int object). Why? Does Python not reuse the memory of the ...
- 27.9k
0
votes
0
answers
377
views
How to trace memory allocations in Apache httpd server?
I am running apache benchmark (ab) with server [httpd2.4.52] running locally. I want to track how many memory allocations and what size allocations does the server make.
I run 'valgrind --trace-malloc=...
- 31