smem: how much memory is your linux system really using?

As you might know by now, I was debugging memory related problems on one of my systems recently and concluded that normal output from Linux commands are more or less inaccurate. If you want to know why, take a look at Matt Mackall presentation at ELC2009: Visualizing Process Memory or watch following video:

Convinced? So, hop at smem page, compile user-land part and start really tracking your memory usage, let's compare:

dpavlin@t61p:/rest/cvs/smem$ free
             total       used       free     shared    buffers     cached
Mem:       4081400    3882476     198924          0     142904    2731480
-/+ buffers/cache:    1008092    3073308
Swap:      8209172       7492    8201680

dpavlin@t61p:/rest/cvs/smem$ ./smem -w -t
Area                           Used      Cache   Noncache 
firmware/hardware                 0          0          0 
kernel image                      0          0          0 
kernel dynamic memory       2927016    2845456      81560 
userspace memory             954900     119368     835532 
free memory                  199484     199484          0 
----------------------------------------------------------
                            4081400    3164308     917092 
Just a few quick notes if you didn't watched whole video carefully:
  • needs kernel 2.6.27 or newer
  • it can work on archived data (from cron in my example usage)
  • userspace cache is backed by file on disk
  • it's a python script which requires matplotlib to create graphs so it's for local reporting