When looking through the filesystem of a Linux system you may notice a directory named /proc. It's a fascinating directory which exposes many of the internal data for the kernel. I'd like to show some of the interesting information you can get from /proc as well as some practical applications in popular software.
Finding One's Self
One of the more interesting pieces of information you can find is for the current process located in /proc/self:
cwprogram@rpi:/proc $ ls -lah /proc/self/
total 0
dr-xr-xr-x 9 cwprogram cwprogram 0 Apr 22 20:49 .
dr-xr-xr-x 217 root root 0 Dec 31 1969 ..
dr-xr-xr-x 2 cwprogram cwprogram 0 Apr 22 20:49 attr
-rw-r--r-- 1 cwprogram cwprogram 0 Apr 22 20:49 autogroup
-r-------- 1 cwprogram cwprogram 0 Apr 22 20:49 auxv
-r--r--r-- 1 cwprogram cwprogram 0 Apr 22 20:49 cgroup
--w------- 1 cwprogram cwprogram 0 Apr 22 20:49 clear_refs
-r--r--r-- 1 cwprogram cwprogram 0 Apr 22 20:49 cmdline
-rw-r--r-- 1 cwprogram cwprogram 0 Apr 22 20:49 comm
-rw-r--r-- 1 cwprogram cwprogram 0 Apr 22 20:49 coredump_filter
-r--r--r-- 1 cwprogram cwprogram 0 Apr 22 20:49 cpuset
lrwxrwxrwx 1 cwprogram cwprogram 0 Apr 22 20:49 cwd -> /proc
-r-------- 1 cwprogram cwprogram 0 Apr 22 20:49 environ
lrwxrwxrwx 1 cwprogram cwprogram 0 Apr 22 20:49 exe -> /usr/bin/ls
<truncate>
Due to ls being the current process when the listing is run information for it is made available. There's also a cwd which points to the current working directory and exe that points to the executable. There's a status file available with a decent amount of information as well:
cwprogram@rpi:/proc $ cat self/status
Name: cat
Umask: 0002
State: R (running)
Tgid: 9755
Ngid: 0
Pid: 9755
PPid: 9687
You can use this with a bit of basic grep to get names of processes like so:
cwprogram@rpi:/proc $ grep "Name:" */status
102/status:Name: kworker/0:1H-kblockd
1124/status:Name: agetty
1126/status:Name: agetty
11/status:Name: kworker/u16:0-ipv6_addrconf
131/status:Name: kworker/R-mmc_complete
<truncate>
The first part of the path here is the PID itself. So this gives you a somewhat rudimentary process listing. Granted it's certainly not as user friendly as the ps command.
Finding Mounts
Processes also have mount information available in either a mountinfo or mounts file. The former is a bit more detailed:
$ cat mountinfo
20 25 0:19 / /sys rw,nosuid,nodev,noexec,relatime shared:6 - sysfs sysfs rw
21 25 0:20 / /proc rw,relatime shared:11 - proc proc rw
22 25 0:6 / /dev rw,nosuid,relatime shared:2 - devtmpfs udev rw,size=1672900k,nr_inodes=418225,mode=755
And the later may be a more familiar output:
cwprogram@rpi:/proc/self $ cat mounts
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,relatime 0 0
udev /dev devtmpfs rw,nosuid,relatime,size=1672900k,nr_inodes=418225,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=600,ptmxmode=000 0 0
This is for the process itself keeping namespace restrictions into consideration.
Topping It Off
Outside of the standard process information, /proc also has a number of toplevel files with interesting entries in them:
-
devices- Listing of character and block devices -
meminfo- Memory statistics -
mounts- Similar to the process version, except for the top system level -
crypto- Various crypto ciphers available to the system -
stat- Several statistics about the system -
version- Kernel version string -
cmdline- The options given to the kernel at boot -
filesystems- Filesystems available to the kernel -
cgroups- Cgroup information, particularly of use to container based solutions
Many of these contain useful information for the case of debugging fairly stripped down environments commonly found in container operating systems.
The Programmatic Approach
Now this isn't just for operating system debugging. It also has practical uses in modern day software. Take for example kubernetes:
cmdline, err := os.ReadFile(filepath.Join("/proc", entry.Name(), "cmdline"))
if err != nil {
klog.V(4).Infof("Error reading file %s: %+v", filepath.Join("/proc", entry.Name(), "cmdline"), err)
continue
}
In this particular case kubernetes is using the proc system to obtain PIDs which match a specific regex. It does so by getting the command name from cmdline on the process directory level. AWS's firecracker VM which is used to power some of its services such as Lambda also uses /proc to obtain cgroup directory info from /proc/mounts:
// search PROC_MOUNTS for cgroup mount points
let f = File::open(proc_mounts_path)
.map_err(|err| JailerError::FileOpen(PathBuf::from(proc_mounts_path), err))?;
// Regex courtesy of Filippo.
// This will match on each line from /proc/mounts for both v1 and v2 mount points.
//
// /proc/mounts cointains lines that look like this:
// cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0
// cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
//
// This Regex will extract:
// * "/sys/fs/cgroup/unified" in the "dir" capture group.
// * "2" in the "ver" capture group as the cgroup version taken from "cgroup2"; for v1,
// the "ver" capture group will be empty (len = 0).
// * "[...],relatime,cpu,cpuacct" in the "options" capture group; this is used for
// cgroupv1 to determine what controllers are mounted at the location.
let re = Regex::new(
r"^([a-z2]*)[[:space:]](?P<dir>.*)[[:space:]]cgroup(?P<ver>2?)[[:space:]](?P<options>.*)[[:space:]]0[[:space:]]0$",
).map_err(JailerError::RegEx)?;
Cython, which is the official C implementation of the Python programming language also uses /proc for a few things, one of them includes obtaining the parent process ID of a process:
snprintf(stat_path, sizeof(stat_path), "/proc/%d/stat", (int)pid);
The stat file is a pretty cryptic thing to look at which gives somewhat more parser friendly statistics for a process:
9687 (bash) S 9686 9687 9687 34817 9994 4194304 49314 139925 0 3 104 36 299 142 20 0 1 0 43945508 9326592 1490 18446744073709551615 367249391616 367250706224 549003478784 0 0 0 65536 3686404 1266761467 1 0 0 17 3 0 0 0 0 0 367250815728 367250867132 367940014080 549003480647 549003480653 549003480653 549003481070 0
The first entry is the process ID, the second the command, the third entry is the process state, and the fourth entry is the parent process ID of the process. While using C for tokenized parsing such as this is a bit awkward it does get the job done.
Wrapping It Up
This is just a small peak into the usefulness that is the proc filesystem. As mentioned it's great when you need a source of information for debugging a Linux based system. It's also useful for handling certain task programmatically, especially if you're doing any form of container development. I urge you to look around /proc some more to see what other useful things you can find.