Kernel space: How much memory am I really using?

With memory pages shared among applications, sometimes it's hard to tell how much you'll really save by killing that memory hog. Two new memory metrics offer help for busy administrators.

By Jonathan Corbet, LinuxWorld.com, 04/25/07

 Start a discussion    Print article

 Comments (1)    Print article

Anybody who has tried to figure out why a Linux system is running short of memory can attest that the memory usage information made available by the kernel is, at best, difficult to use. Matt Mackall has recently been working on a set of patches aimed at improving this situation. Given the constraints imposed by embedded Linux systems, it is not surprising that Matt chose the Embedded Linux Conference to present his work (which, incidentally, was funded by the Consumer Electronics Linux Forum).

Matt pointed out that the currently-available information is confusing at best. The page cache muddies the situation, and the sharing of pages between applications complicates things even more. The result is that it is hard to say where memory is being used; one can't even get a definitive answer to the question of how big a specific application is. More detailed questions - such as which parts of an application are using the most memory - are even harder to answer. Trying to answer questions of interest to embedded systems developers - how many applications can run on a specific device without pushing it into thrashing, for example - is nearly impossible without simply running a test.

The problem is that the numbers exported by the current kernels are nearly meaningless. The reported virtual size of an application is nearly irrelevant; it says nothing about how much of that virtual space is actually being used. The resident set size (RSS) number is a little better, but there is no information on sharing of pages there. The /proc/pid/smaps file gives a bit of detail, but also lacks sharing information. And the presence of memory pressure can change the situation significantly.

The Linux virtual memory system, in other words, is a black box which provides too little information on what is going on inside. Matt's project is to open up that box and shine some light inside.

The first step is to add a new file (pagemap) in each process's /proc directory. It is a binary file containing the page frame number for each page in the process's address space. The file can be read to see where a process's pages have been placed and, more interestingly, it can be compared between processes to see which pages are being shared. Matt has a little graphical tool which can display this file, showing the patterns of which pages are present in memory and which are not.

Featured Whitepapers

LinuxCast Listen to the latest Linux news.

About Us | Contact Us | Advertising Info | Terms of Service/Privacy

Copyright, 2006-2007 Network World, Inc. All rights reserved.