diff --git a/CHANGELOG.md b/CHANGELOG.md index d121187..bdedc4d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,31 @@ All notable changes to **felhom-agent** are recorded here. Update on every code change that gets pushed. +## v0.29.0 — OS / Docker-data storage split: golden + provision (2026-06-13) + +Phase 1 of the storage-split slice (Phase 2 = felhom-controller v0.58.0 prevention layer). The +controller guest's OS rootfs and Docker data are carved onto separate `local-lvm` volumes for +RESILIENCE — an isolated OS rootfs stays bootable + agent-recoverable if the Docker volume fills. + +- **`configs/build-golden.sh` — split baked in:** `--rootfs ${ROOTFS_STORAGE}:${OS_SIZE_GB}` (default + **32**, was hardcoded 8) **plus** `--mp0 ${ROOTFS_STORAGE}:${GOLDEN_DOCKER_GB},mp=/var/lib/docker,backup=1` + (default 16). The baked controller + infra images land on the data volume and travel inside the + golden archive (no empty-volume shadowing, no deploy-time pull). `backup=1` is MANDATORY — extra LXC + mountpoints default to `backup=0` = EXCLUDED from vzdump (spike B3), which would drop the images from + the archive entirely. The script now also bakes Docker **log rotation** into `daemon.json` + (`max-size 10m`, `max-file 3` — prevention layer 2D), asserts `/var/lib/docker` is a separate mount, + and **aborts if vzdump excludes mp0**. +- **`internal/reconcile/bringup.go` — sized provision:** `GuestMount` gains `Backup` (emits `,backup=1` + — closes the spike-B3/B5 silent-DB-loss trap at the mount builder). `BringUpSpec` gains + `DataVolGrowGB` + `DataVolMount` (default `mp0`): provision GROWS the golden-carried Docker-data + volume online to the per-customer target (grow-only, spike B4) rather than attaching a fresh empty + volume that would shadow the baked images. Plus `RootfsGrowGB` for the OS rootfs. +- **CLI seam:** `--selftest=bring-up|provision` gain `-rootfs-grow` / `-datavol-grow` / `-datavol-mount` + flags. Per-customer sizing source = flags now, the slice-10 hub storage manifest later. +- **`RUNBOOK-provisioning-storage.md`** (new): the split provisioning procedure + fresh-PVE-install + thin-pool carving knobs (`hdsize`/`maxroot`/`maxvz`, spike B4) + the per-customer sizing seam. +- Tests: `buildBringUpConfig` backup=1 emission; bring-up issues rootfs + data-volume resizes. + ## (no version) — storage OS/data-split spike findings (2026-06-13) Investigation only — **no code changed**. Findings report: `REPORT-storage-split-spike.md` (gates the diff --git a/RUNBOOK-provisioning-storage.md b/RUNBOOK-provisioning-storage.md new file mode 100644 index 0000000..54ad330 --- /dev/null +++ b/RUNBOOK-provisioning-storage.md @@ -0,0 +1,89 @@ +# RUNBOOK — provisioning the OS / Docker-data storage split + +How the controller guest's **OS rootfs** and **Docker data** are carved onto separate `local-lvm` +volumes, and how to size them per node/customer. Grounded by the storage-split spike +(`REPORT-storage-split-spike.md`). Defaults target a ~512 GB SSD: **32 GB OS rootfs + 256 GB +Docker-data**, rest of the thin pool free for other guests. + +## The design (locked) + +- **Split for resilience, not backup size.** An isolated OS rootfs stays bootable + agent-recoverable + if the Docker volume fills. Both volumes ride into PBS regardless (so the backup size is unchanged) — + the win is independent sizing/growth and a smaller, faster-to-restore OS rootfs. +- **One Docker daemon, one data-root, one volume** at `/var/lib/docker` for ALL images + overlay + (controller, traefik, cloudflared, filebrowser, and customer apps). Infra is NOT relocated to the OS + partition — infra/user separation under one daemon isn't cleanly possible and isn't needed. Infra is + protected by the controller's **prevention layer** (reserved buffer + deploy gate + disk alerts + + baked log rotation), not by placement. +- **`backup=1` on the Docker-data volume is non-negotiable.** Extra LXC mountpoints default to + `backup=0` = EXCLUDED from vzdump/PBS (spike B3). Without it, named-volume DBs fall out of the + snapshot AND the volume (with baked images) is excluded from the golden archive. + +## Golden build (per controller-image baseline) + +`configs/build-golden.sh` builds the template guest with the split baked in: + +``` +# defaults shown; override via env +OS_SIZE_GB=32 GOLDEN_DOCKER_GB=16 \ + ./build-golden.sh