57 lines
4.3 KiB
Markdown
57 lines
4.3 KiB
Markdown
# REPORT — felhom-controller v0.58.0 (infra-protection prevention layer for the OS/Docker-data split)
|
|
|
|
Phase 2 of the OS/Docker-data storage-split slice (Phase 1 = felhom-agent v0.29.0: golden + provision).
|
|
The controller guest's OS rootfs and Docker data are now split onto separate `local-lvm` volumes for
|
|
resilience; infra (controller/traefik/cloudflared/filebrowser) shares the one Docker data-root and is
|
|
protected by **prevention, not placement**. Built, deployed, and **live-validated on a freshly
|
|
re-provisioned guest 9201**.
|
|
|
|
## What shipped (v0.58.0)
|
|
- **Reserved-buffer headroom guard** (`internal/system/dockervol.go`): `GetDockerVolumeHeadroom()`
|
|
measures the Docker-data volume via `statfs("/")` — the controller container's root overlay is the
|
|
upperdir on the guest's `/var/lib/docker` volume (true with the **overlay2** driver; see the agent
|
|
report), so `/` reports the data volume. Reserve floor `DockerVolumeReserveGB = max(5 GB, 10%)`.
|
|
Fail-open on a measurement error.
|
|
- **Deploy-time hard gate** (`internal/api/router.go` `deployStack`): a new deploy is **refused (HTTP
|
|
507** + Hungarian message) when free space on the Docker-data volume is at/under the reserved buffer.
|
|
- **Deploy-page surfacing** (`deploy.html`): a new deploy below the buffer shows a Hungarian warning and
|
|
**disables** the "Telepítés indítása" button; the API gate is the hard backstop.
|
|
- **Runtime monitoring** (`monitor/healthcheck.go`): confirmed `DiskPercent` watches the Docker-data
|
|
volume (statfs `/`); warn 80% / crit 90% trip ABOVE the 10%-free buffer, so the customer is warned
|
|
before the gate engages. Clarifying comment added.
|
|
- **Log rotation** baked into the golden's `daemon.json` (agent side; `max-size 10m`, `max-file 3`).
|
|
- Tests: `DockerVolumeReserveGB` floor/scale.
|
|
|
|
## Live validation (guest 9201, freshly re-provisioned from the split golden)
|
|
9201 was **destroyed and re-provisioned** from the new split golden (32 GB OS rootfs + 256 GB Docker-data
|
|
volume, `backup=1`), via `felhom-agent --selftest=provision` + a reboot. The controller bootstrapped
|
|
from baked images (no pull), **pulled its config from the hub** (catalog synced — 55 app defs — hub
|
|
HTTP 200, CF token configured, hub report pushed). Then:
|
|
|
|
- **Split layout:** controller image 0.58.0, **Storage Driver overlay2**, `Docker Root Dir
|
|
/var/lib/docker`; images on the data volume (`/var/lib/docker/overlay2` 1.7 GB), `/var/lib/containerd`
|
|
idle (380 K); `df`: `/` 935 MB/32 GB (4%, lean OS rootfs), `/var/lib/docker` 256 GB.
|
|
- **Prevention gate (the headline):** with ample space the deploy page shows **no** gate banner; after
|
|
`fallocate`-filling the data volume to 99% (3.2 GB free < 25.6 GB reserve), a `POST /api/stacks/.../deploy`
|
|
returned **HTTP 507** with the Hungarian "Nincs elég szabad tárhely" message — proven on the real
|
|
256 GB data volume.
|
|
- **Regression:** `/`, `/stacks`, `/backups`, `/monitoring`, `/stacks/{n}/deploy`, `/stacks/{n}/backup`
|
|
all HTTP 200; A1 host-storage list still ordered + friendly-labelled (felhom-usb → local-lvm → local
|
|
→ felhom-pbs); A2 Tier-2 panel route serves.
|
|
- **Deploy path + DB-on-data-volume (step 3):** deployed ActualBudget (HTTP 200, container up); its named
|
|
volume landed at `/var/lib/docker/volumes/actualbudget_actualbudget_data` = the data volume.
|
|
- **External access:** via Cloudflare the controller returns HTTP 200 for vmid 9201 (tunnel + traefik
|
|
route healthy). (A local-DNS override on the dev machine points the hostname at a stale LAN IP — a
|
|
red herring; the real public path works.)
|
|
|
|
## OS isolation (resilience — the reason for the split), proven on the provisioned guest
|
|
Filling the Docker-data volume to 100% (239 GB) left the OS rootfs at 4% and fully writable, the guest
|
|
healthy throughout — the data volume cannot starve the OS.
|
|
|
|
## Outstanding (demo restoration, not slice validation)
|
|
- **RomM (HDD app) + USB re-enroll:** RomM's data is safe on the host USB (`/mnt/felhom-usb/felhom-data`,
|
|
untouched by the re-provision). Restoring it is the slice-10 enroll flow (assign → guest-attach →
|
|
reboot to activate the bind → register storage → deploy). With the split, the USB binds to a free slot
|
|
(mp1+) since **mp0 is now the Docker-data volume** — no collision. Documented as the final restore step;
|
|
not required for slice validation (ActualBudget covered the deploy path; the USB bind was not touched).
|