Sungju's Slow Life

There are times the crash had happened because of the lack of memory. Or times that system had hard time because of the memory issue. To check those, you can use below command.

crash> kmem -i
              PAGES        TOTAL      PERCENTAGE
 TOTAL MEM  16464824      62.8 GB         ----
      FREE  3807386      14.5 GB   23% of TOTAL MEM
      USED  12657438      48.3 GB   76% of TOTAL MEM
    SHARED  10203845      38.9 GB   61% of TOTAL MEM
   BUFFERS   110899     433.2 MB    0% of TOTAL MEM
    CACHED  10381430      39.6 GB   63% of TOTAL MEM
      SLAB   471425       1.8 GB    2% of TOTAL MEM

TOTAL HIGH        0            0    0% of TOTAL MEM
 FREE HIGH        0            0    0% of TOTAL HIGH
 TOTAL LOW  16464824      62.8 GB  100% of TOTAL MEM
  FREE LOW  3807386      14.5 GB   23% of TOTAL LOW

TOTAL SWAP  16777882        64 GB         ----
 SWAP USED   129199     504.7 MB    0% of TOTAL SWAP
 SWAP FREE  16648683      63.5 GB   99% of TOTAL SWAP

Above was the system that actually had lock contention, so, the memory consumption wasn’t bad. Only used 76% of memory and 63% of the memory was consumed by ‘caches’ which is normal in most cases.

But, if you want to check which processes actually consumed mostly, you can check it with the following commands.

Check specific process list

crash> ps -g | grep "^PID" | grep "Process name"

Most memory consumers

crash> ps | sed -e "s/^>/ /g" | sort -n -k 8

Total used virtual memory (committed amount)

crash> ps -G | tail -n +2 | cut -b2- | gawk '{mem += $8} END {print "total " mem/1048576 "GB"}'
total 37.2951GB

Checking Total physical memory, overcommitted, swapped amount

crash> pd totalram_pages
totalram_pages = $8 = 16464824
crash> pd vm_committed_space
vm_committed_space = $9 = {
  counter = 10638744
}
crash> pd total_swap_pages
total_swap_pages = $10 = 16777882
crash> !echo "scale=5;(10638744*4096)/2^30"|bc -q
40.58358   !echo "scale=5;(16464824*4096)/2^30"|bc -q
62.80831  <- physical memory amount