Disk Layout & Impermanence

The server uses a sophisticated disk strategy combining LUKS encryption, BTRFS subvolumes, an ephemeral root filesystem, and a dedicated backup disk.

Physical Disks

Disk	Model	Size	Role
Primary	WD Black SN850X NVMe	2 TB	OS + data
Backup	SATA SSD	512 GB	Restic backup repository

Partition Layout

Primary Disk (NVMe)

/dev/disk/by-id/nvme-WD_BLACK_SN850X_2000GB_24517E4ABM06
├── ESP  (512 MB, EF00, FAT32)     →  /boot
└── luks (remaining, LUKS)
    └── crypted (BTRFS)
        ├── /root     →  /         (ephemeral, wiped on boot)
        ├── /nix      →  /nix      (persistent, zstd compressed)
        └── /persist  →  /persist  (persistent, zstd compressed)

Backup Disk (SATA)

/dev/disk/by-id/ata-512GB_SSD_MQ02W52300963
└── backup (100%, XFS)             →  /backup

Mount Options

Subvolume	Options	Notes
/boot	defaults nosuid nodev noexec umask=0077	EFI partition, maximum restrictions
/ (root)	defaults noatime nodev nosuid noexec	Ephemeral, security-hardened mounts
/nix	compress=zstd noatime	Store benefits from compression
/persist	compress=zstd noatime nodev nosuid noexec	Persistent data with restrictions
/backup	noatime nofail nodev nosuid	nofail prevents boot failure if disk is absent

WARNING

The root filesystem has noexec set. Execution is possible because /nix (which contains all binaries) does not have noexec.

Impermanence

The root BTRFS subvolume is rolled back to a blank snapshot on every boot. This is the core of the impermanence pattern.

How It Works

A systemd service runs during early boot (in the initrd), before the root filesystem is mounted:

boot.initrd.systemd.services.rollback = {
  wantedBy = [ "initrd.target" ];
  after = [ "systemd-cryptsetup@crypted.service" ];
  before = [ "sysroot.mount" ];

  script = ''
    # Mount the raw BTRFS volume
    mount /dev/mapper/crypted /btrfs_tmp

    # Move the current root to old_roots/ with a timestamp
    timestamp=$(date --date="@$(stat -c %Y /btrfs_tmp/root)" "+%Y-%m-%d_%H:%M:%S")
    mv /btrfs_tmp/root "/btrfs_tmp/old_roots/$timestamp"

    # Delete old roots older than 30 days
    for i in $(find /btrfs_tmp/old_roots/ -maxdepth 1 -mtime +30); do
        delete_subvolume_recursively "$i"
    done

    # Create a fresh, empty root subvolume
    btrfs subvolume create /btrfs_tmp/root
  '';
};

Boot Sequence

1. LUKS volume crypted is unlocked (manually or via SSH).
2. The rollback service runs:
   • Archives the current /root subvolume to /old_roots/<timestamp>.
   • Deletes archived roots older than 30 days.
   • Creates a new, empty /root subvolume.
3. The fresh /root is mounted as /.
4. NixOS activation scripts populate / from the Nix store.
5. /persist is bind-mounted to provide persistent state.

What Survives a Reboot

Only paths declared in impermanence.nix:

environment.persistence."/persist" = {
  hideMounts = true;

  directories = [
    "/etc/nixos-containers"
    "/var/lib/nixos"
    "/var/lib/nixos-containers"
  ];

  files = [
    "/etc/machine-id"
    "/etc/ssh/ssh_host_rsa_key"
    "/etc/ssh/ssh_host_rsa_key.pub"
    "/etc/ssh/ssh_host_ed25519_key"
    "/etc/ssh/ssh_host_ed25519_key.pub"
  ];
};

Persistent Path	Purpose
/etc/machine-id	Stable machine identity
/etc/ssh/ssh_host_*	SSH host keys (prevents key change warnings)
/etc/nixos-containers	Container configuration state
/var/lib/nixos	NixOS state (UID/GID mappings)
/var/lib/nixos-containers	Container data volumes

Everything else : logs, temp files, runtime state is discarded on reboot.

Recovery

Old root snapshots are kept for 30 days in /old_roots/. To recover a file:

# Mount the BTRFS volume
mount /dev/mapper/crypted /mnt

# List available snapshots
ls /mnt/old_roots/

# Access a specific snapshot
ls /mnt/old_roots/2026-02-28_14:30:00/

Remote LUKS Unlock

The server can be unlocked remotely via SSH during early boot:

boot.initrd = {
  availableKernelModules = [ "r8169" ];  # Ethernet driver in initrd

  network = {
    enable = true;
    udhcpc.enable = true;
    flushBeforeStage2 = true;

    ssh = {
      port = 22;
      enable = true;
      hostKeys = [ "/etc/ssh/ssh_host_ed25519_key" ];
      authorizedKeys = config.users.users.${username}.openssh.authorizedKeys.keys;
    };

    postCommands = ''echo "cryptsetup-askpass" >> /root/.profile'';
  };
};

Unlock Procedure

ssh root@192.168.1.20
# The shell will prompt for the LUKS passphrase
# After entering it, the server continues booting

Key Details

• The r8169 network driver is loaded in the initrd for early network access.
• udhcpc provides DHCP in the initrd (the server uses static IP in normal operation, but DHCP is simpler in initrd).
• flushBeforeStage2 = true tears down the initrd network before normal boot to avoid conflicts with the main network configuration.
• The postCommands script sets up the cryptsetup-askpass command to run automatically when SSH-ing into the initrd.
• The same SSH host key and authorised keys are used as in normal operation.