One detail that is often overlooked when dealing with embedded (or remote) devices is a key point of vulnerability: the file system.

For non-COW file systems (like ext4 on Linux, FFS, etc.), there are situations where a crash or a power outage could cause corruption, requiring manual intervention. This risk is especially present when using root on SD cards or similar storage media in embedded systems. Over time, these media are destined to fail due to numerous writes.

For certain use cases, it's advisable to set up a read-only root file system, which ensures better reliability in case of system issues. Think of scenarios like a router (critical for network access) or a caching reverse-proxy, such as the one described in my series "Make your own CDN".

While FreeBSD natively supports this configuration and some Linux distributions offer custom solutions (e.g., Alpine Linux), NetBSD stands out as an excellent choice for such devices. It supports nearly all embedded devices, is lightweight, and its stability minimizes the need for frequent updates.

The Key Idea

Although NetBSD doesn't provide native read-only support, it's flexible enough to allow for this configuration. Many years ago, I followed a howto to set up a read-only system, and the idea remains simple and effective: NetBSD writes primarily to specific locations, unlike some Linux distributions that attempt to write to various parts of the file system. On NetBSD, the main write targets are the /tmp directory and /var.

With this in mind, we can configure these directories to reside in memory file systems (mfs) and ensure that /var contains everything necessary for the system to function correctly.

1. Prepare the Environment

Start with a basic NetBSD installation (I'll skip the installation steps as there are many tutorials available, and it's straightforward).

As a first step, clean up the /var directory. After each reboot, its contents will be extracted into the mfs and occupy RAM.

Disable man.db generation and swap usage:

Edit the following configuration files:

# /etc/rc.conf
makemandb=NO
no_swap=YES

# /etc/daily.conf
run_makemandb=NO

Then, remove the current man.db file:

rm /var/db/man.db

Once this is done, create a compressed archive of the current /var contents to be replicated at every boot:

cd /
tar -cvzf var-image.tar.gz var

2. Create a Custom Startup Script

Next, create a file called /etc/rc.d/mount_mfs_fs with the following content:

#!/bin/sh
#
# mount_mfs_fs: mount memory file system for /var
# by roby, 23 jun 2003 - adapted by Stefano - 01 Sep 2024

# PROVIDE: mount_mfs_fs
# REQUIRE: root

. /etc/rc.subr

name="mount_mfs_fs"
start_cmd="mount_mfs_fs_start"
stop_cmd=":"

mount_mfs_fs_start()
{
    # Check if the /var entry is present and uncommented in /etc/fstab
    if grep -q '^tmpfs[[:space:]]\+/var[[:space:]]\+tmpfs[[:space:]]\+rw,-m1777,-sram%25' /etc/fstab; then
        echo "Mounting memory file system: /var"

        # Mount the file system for /var
        mount /var

        # Extract the contents of the tar file into /var
        tar -xvzpf /var-image.tar.gz -C /

        echo "Mounting memory file systems: Done."
    else
        echo "The tmpfs entry for /var is not present or is commented out in /etc/fstab. Skipping mount and extraction."
    fi
    sleep 5
}

load_rc_config $name
run_rc_command "$1"

Make the script executable:

chmod a+rx /etc/rc.d/mount_mfs_fs

3. Modify Boot Process

Now, modify the /etc/rc.d/mountcritlocal file to require this script to run before its execution:

#!/bin/sh
#
# $NetBSD: mountcritlocal,v 1.17 2022/02/20 14:42:07 alnsn Exp $
#

# PROVIDE: mountcritlocal
# REQUIRE: mount_mfs_fs

$_rc_subr_loaded . /etc/rc.subr

name="mountcritlocal"
start_cmd="mountcritlocal_start"
stop_cmd=":"

mountcritlocal_start()
{
    #       Mount critical file systems that are `local'
    #       (as specified in $critical_filesystems_local)
    #       This usually includes /var.
    #
    mount_critical_filesystems local || return $?
    if checkyesno zfs; then
        mount_critical_filesystems_zfs || return $?
    fi
    return 0
}

load_rc_config $name
load_rc_config_var zfs zfs
run_rc_command "$1"

4. Configure /etc/fstab

Edit the /etc/fstab file to mount /tmp and /var in memory:

tmpfs           /tmp        tmpfs   rw,-m1777,-sram%25
tmpfs           /var        tmpfs   rw,-m1777,-sram%25

5. Test the Configuration

You can now reboot the system and check if everything works correctly. Once logged in, the df command should show something like this:

Filesystem      1K-blocks         Used        Avail %Cap Mounted on
/dev/ld0a        18298254       393938     16989404   2% /
tmpfs              524192         4224       519968   0% /var
kernfs                  1            1            0 100% /kern
ptyfs                   1            1            0 100% /dev/pts
procfs                  4            4            0 100% /proc
tmpfs              524192            4       524188   0% /var/shm
tmpfs              524192            4       524188   0% /tmp

6. Set Root to Read-Only

Currently, the system is still running in read-write mode, but /var and /tmp are in memory disks. To switch the root file system to read-only, edit the /etc/fstab file like this:

/dev/ld0a               /       ffs     ro               1 1

On the next reboot, the system will be in pure read-only mode.

7. Perform Updates

To install packages, update the system, or make any changes, you'll need to temporarily switch back to read-write mode. Comment out /var in mfs, change the root file system back to read-write, and reboot:

mount -uw /
vi /etc/fstab
/dev/ld0a               /       ffs     rw               1 1
[...]
#tmpfs           /var    tmpfs   rw,-m1777,-sram%25
[...]

After making the necessary changes, regenerate the /var tarball as done in step 1.