felhom-controller/REPORT.md

REPORT — felhom-controller v0.58.0 (infra-protection prevention layer for the OS/Docker-data split)

Phase 2 of the OS/Docker-data storage-split slice (Phase 1 = felhom-agent v0.29.0: golden + provision). The controller guest's OS rootfs and Docker data are now split onto separate local-lvm volumes for resilience; infra (controller/traefik/cloudflared/filebrowser) shares the one Docker data-root and is protected by prevention, not placement. Built, deployed, and live-validated on a freshly re-provisioned guest 9201.

What shipped (v0.58.0)

• Reserved-buffer headroom guard (internal/system/dockervol.go): GetDockerVolumeHeadroom() measures the Docker-data volume via statfs("/") — the controller container's root overlay is the upperdir on the guest's /var/lib/docker volume (true with the overlay2 driver; see the agent report), so / reports the data volume. Reserve floor DockerVolumeReserveGB = max(5 GB, 10%). Fail-open on a measurement error.
• Deploy-time hard gate (internal/api/router.go deployStack): a new deploy is refused (HTTP 507 + Hungarian message) when free space on the Docker-data volume is at/under the reserved buffer.
• Deploy-page surfacing (deploy.html): a new deploy below the buffer shows a Hungarian warning and disables the "Telepítés indítása" button; the API gate is the hard backstop.
• Runtime monitoring (monitor/healthcheck.go): confirmed DiskPercent watches the Docker-data volume (statfs /); warn 80% / crit 90% trip ABOVE the 10%-free buffer, so the customer is warned before the gate engages. Clarifying comment added.
• Log rotation baked into the golden's daemon.json (agent side; max-size 10m, max-file 3).
• Tests: DockerVolumeReserveGB floor/scale.

Live validation (guest 9201, freshly re-provisioned from the split golden)

9201 was destroyed and re-provisioned from the new split golden (32 GB OS rootfs + 256 GB Docker-data volume, backup=1), via felhom-agent --selftest=provision + a reboot. The controller bootstrapped from baked images (no pull), pulled its config from the hub (catalog synced — 55 app defs — hub HTTP 200, CF token configured, hub report pushed). Then:

• Split layout: controller image 0.58.0, Storage Driver overlay2, Docker Root Dir /var/lib/docker; images on the data volume (/var/lib/docker/overlay2 1.7 GB), /var/lib/containerd idle (380 K); df: / 935 MB/32 GB (4%, lean OS rootfs), /var/lib/docker 256 GB.
• Prevention gate (the headline): with ample space the deploy page shows no gate banner; after fallocate-filling the data volume to 99% (3.2 GB free < 25.6 GB reserve), a POST /api/stacks/.../deploy returned HTTP 507 with the Hungarian "Nincs elég szabad tárhely" message — proven on the real 256 GB data volume.
• Regression: /, /stacks, /backups, /monitoring, /stacks/{n}/deploy, /stacks/{n}/backup all HTTP 200; A1 host-storage list still ordered + friendly-labelled (felhom-usb → local-lvm → local → felhom-pbs); A2 Tier-2 panel route serves.
• Deploy path + DB-on-data-volume (step 3): deployed ActualBudget (HTTP 200, container up); its named volume landed at /var/lib/docker/volumes/actualbudget_actualbudget_data = the data volume.
• External access: via Cloudflare the controller returns HTTP 200 for vmid 9201 (tunnel + traefik route healthy). (A local-DNS override on the dev machine points the hostname at a stale LAN IP — a red herring; the real public path works.)

OS isolation (resilience — the reason for the split), proven on the provisioned guest

Filling the Docker-data volume to 100% (239 GB) left the OS rootfs at 4% and fully writable, the guest healthy throughout — the data volume cannot starve the OS.

Outstanding (demo restoration, not slice validation)

• RomM (HDD app) + USB re-enroll: RomM's data is safe on the host USB (/mnt/felhom-usb/felhom-data, untouched by the re-provision). Restoring it is the slice-10 enroll flow (assign → guest-attach → reboot to activate the bind → register storage → deploy). With the split, the USB binds to a free slot (mp1+) since mp0 is now the Docker-data volume — no collision. Documented as the final restore step; not required for slice validation (ActualBudget covered the deploy path; the USB bind was not touched).