Skip to main content
3 of 4
added more useful information related to the question
Y. E.
  • 123
  • 1
  • 1
  • 6

How to troubleshoot what eats memory?

I'm trying to figure out why memory consumption started to constantly increase on my server during the last hours. I've tried to find the cause on the application level, but no success. That is why now looking into possible server cause. I'm not a pro in servers administration, so any help is appreciated. First common memory was eaten, now swap consumption is also constantly increasing.

My server runs on CentOS 7 with the kernel 3.10.0-514.26.2.el7.x86_64

SOLUTION
Finally, the issue was identified to be caused by a recently updated server library. The accepted answer is a good reminder, in the situation when you're stressed out by the memory usage, to trace back what had been changed in your system before the issue appeared.

Some tips I've been looking for and found to be very useful are described in How to find which processes are taking all the memory?

I'm listing below the commands that I used and may help in such situation.

ps auwx --sort rss - processes sorted by memory usage

ps -fu username - processes by a user

htop usage/analysis showed many hung application cron-launched processes in my case. I configured htop to output both PID and PPID, because I needed to correlate PPID to /var/log/cron logged processes.

free -m

    total        used        free      shared  buff/cache   available
Mem: 7565        6525         440          47         599         657
Swap: 8191        2612        5579

cat /proc/meminfo

MemTotal:        7747260 kB
MemFree:          253960 kB
MemAvailable:     498904 kB
Buffers:            6160 kB
Cached:           189076 kB
SwapCached:       467788 kB
Active:          5572588 kB
Inactive:        1258540 kB
Active(anon):    5498664 kB
Inactive(anon):  1185908 kB
Active(file):      73924 kB
Inactive(file):    72632 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       8388604 kB
SwapFree:        5686452 kB
Dirty:               104 kB
Writeback:             0 kB
AnonPages:       6168400 kB
Mapped:            68668 kB
Shmem:             48676 kB
Slab:             456672 kB
SReclaimable:     389064 kB
SUnreclaim:        67608 kB
KernelStack:        7232 kB
PageTables:       106848 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    12262232 kB
Committed_AS:   10244216 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       26276 kB
VmallocChunk:   34359705340 kB
HardwareCorrupted:     0 kB
AnonHugePages:   5191680 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       83968 kB
DirectMap2M:     8300544 kB

How can I proceed to find if there's any issue in how the server is functioning or configured in terms of memory usage?

Sorry if this sounds stupid or a duplicate - I've tried to search for similar questions, but failed to troubleshoot my actual issue.

Thank you.

Y. E.
  • 123
  • 1
  • 1
  • 6