test(nomercy): Phase 5 + deliverable summary — campaign complete

Demo restored to healthy baseline (test apps removed, drives at stable, sys_drive
separate, settings UI clean). Deliverable summary: 4 findings (F3 onboot=0 guest
no-auto-start [highest impact], F2 FileBrowser bounce on controller restart, F1
restic dead-weight, F4 stable double-bind), all report-not-fixed. Method per test,
PBS floor (unused), end state, coverage gaps (1.3/1.4/1.8 loop-whitelist, 4.7
operator-gated). No data-loss/path-escape/corruption found.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PsnU2ASocYrvzqE82YDYW
This commit is contained in:
2026-06-23 20:20:50 +02:00
parent 2c1441f5dd
commit 347546e597
@@ -65,7 +65,7 @@ INCONCLUSIVE = could not assert cleanly (reason given).
---
## Phase 1 — stable-vs-raw path model — ⏳ IN PROGRESS
## Phase 1 — stable-vs-raw path model — ✅ COMPLETE (1.3/1.4/1.8 by-design inconclusive)
**Method:** mix of live-UI (claude-in-chrome on `felhom.demo-felhom.eu`) and exact-endpoint (curl to
the in-guest controller `127.0.0.1:8080`). The demo controller has **no password set**
@@ -220,7 +220,7 @@ finding (F1).
- `df /mnt/sys_drive` = **50 GB** (8 golden + 42 grow, matching `mp1 size=50G`). Still a separate device
after the guest reboot — confirmed in **Phase 3.3** (post-restart sys_drive still `disk--2`).
## Phase 3 — server restart + retired banner — ⏳ IN PROGRESS
## Phase 3 — server restart + retired banner — ✅ COMPLETE (F2 found)
### 3.1 — banner gone in every storage state — ✅ PASS
- Rendered `/settings` in the current state (navidrome on flash, radarr on sys_drive, both drives
@@ -281,7 +281,7 @@ finding (F1).
durability win — drives stable, sys_drive separate, apps recreated, registry intact, no placeholders (3.3).
The controller-only restart keeps the guest + customer apps running but **does bounce FileBrowser** (F2).
## Phase 4 — host reboot + chaos — ⏳ IN PROGRESS
## Phase 4 — host reboot + chaos — ✅ COMPLETE (4.7 deferred; F3/F4 found)
Pre-host-reboot capture: `sdb1`=`da9e7089`(usb), `sdc1`=`81a26531`(flash); `onboot: 0`;
`felhom-agent` service `enabled` (will auto-start on host boot). navidrome on flash, radarr on sys_drive.
@@ -368,13 +368,79 @@ drive yank (4.4), and concurrency doesn't corrupt (4.6). Two findings: **F3** (`
auto-start after host reboot — the highest-impact finding) and **F4** (stable-but-present double-bind). The
capstone (4.7) is deferred pending operator credentials.
## Phase 5 — restore to clean baseline — ⏳ PENDING
## Phase 5 — restore to clean baseline — ✅ COMPLETE
- Removed the test apps (navidrome stop+remove, radarr stop+remove) → only infra remains
(felhom-controller, traefik, cloudflared, filebrowser — all healthy), matching the Phase-0 baseline.
- Loop devices `loop0`/`loop1` were already gone (cleared by the host reboot); scratch imgs + dir removed.
All fill files removed (2.2). No decommission soft-markers left (everything reconnected; both intents
`enrolled`).
- **Final assertions (all ✅):** both drives live at **stable** with correct device/size (felhom-flash
`/dev/sdb1` 125.7 GB, felhom-usb `/dev/sdc1` 983.3 GB — letters are post-swap, bound by UUID);
registry = exactly 2 stable paths; `/mnt/sys_drive` separate (`disk--2`, 50 G, `[PASS] System data
path`); dashboard `GET /` 200; settings UI clean — **0** banner occurrences, both "Vezérlő
újraindítása" + "Kiszolgáló újraindítása" cards present. The PBS floor was **not** needed (nothing
wedged). `9201` left at `onboot:0` (original state — F3 is reported, not fixed).
---
## Findings (running)
_None yet beyond the flagged baseline observations (`onboot:0`, leftover absent intents)._
# DELIVERABLE SUMMARY
## Method notes
- User-facing actions: method stated per test (claude-in-chrome vs exact-endpoint). The demo controller
renders `/settings` unauthenticated and is reachable in-guest at `127.0.0.1:8080` (curl in-container).
## 1. Findings (root-caused; report-don't-fix — none implemented)
| ID | Sev | New? | Summary | Evidence / source | Fix direction |
|----|-----|------|---------|-------------------|---------------|
| **F3** | **Moderate** | surfaced this session (pre-existing config) | **Guest `9201` has `onboot:0` → does NOT auto-start after a host reboot.** The whole customer home-server (apps + controller) stays down until a manual `pct start`. | 4.1: host rebooted → `pct status 9201` = `stopped`; `pct config 9201` → `onboot: 0` | Golden/provision should set `onboot:1` (+ boot order) on customer guests; audit all provisioned guests. |
| **F2** | Minor/Mod | surfaced this session (behavior pre-existing) | **"Vezérlő újraindítása" (controller-only restart) also force-recreates the FileBrowser container** every time, even on a no-op sync — contradicts the "your apps keep running" promise. | 3.4: filebrowser `StartedAt` changed on controller restart; `SyncFileBrowserMounts` runs `docker compose up -d --force-recreate` unconditionally — `internal/web/handlers.go:1543`; `sourcesChanged` (`:1504`) only gates the DB reset (`:1529`) | Gate the `--force-recreate` on `sourcesChanged`; skip recreate when config/compose is byte-identical. |
| **F1** | Minor | new discovery (pre-existing) | **`restic` binary ships in the controller image as dead weight.** All restic *code* moved to the agent during de-privileging, but `Dockerfile:58` still `apt install`s it. | 1.11: `/usr/bin/restic` present; no Go code execs it (only "moved to agent" comments + vestigial config/migration) | Drop `restic` from the Dockerfile install list + the dead `Restic*` config fields. |
| **F4** | Minor | pre-existing (documented v0.36.x area) | **Each enrolled drive is double-bound** (2 identical `/proc/mounts` entries). Stable, not leaking; agent logs "normalized to one bind (prior_binds=2)" but steady state is 2. | 4.5: `grep -c` = 2 (guest+host), stable across ticks; brief detach transient in 4.3 | Make normalize-to-one converge to a single bind / stop the duplicate rbind at source. |
**Observations (not findings):** (a) a named-volume app (navidrome) deployed onto a drive creates **no**
recovery unit / appdata namespace on that drive — likely by-design (recovery units key on appdata-bind
apps). (b) The free-form "add storage path" form would accept `/mnt/sys_drive` (a real mountpoint) —
consistent with the SSD-fallback, not the raw-drive bug.
## 2. Method per user-facing test
- **claude-in-chrome (live UI):** 1.5 (disk-card badges, screenshot), 3.3 ("Kiszolgáló újraindítása"
click → guest reboot; confirm() neutralized, page handler invoked), 3.4 ("Vezérlő újraindítása" click),
5 (final settings UI). No F9-style bypass — the real handlers/endpoints ran the full server pipeline.
- **Exact-endpoint (= the UI's own request; demo has no password so CSRF is a no-op):** 1.1, 1.2, 1.6,
1.7, 1.9, 1.10, 1.11, 1.12, 2.x, 3.1/3.2, 4.3, 4.4, 4.6. The drive ops invoked the exact
`/api/storage/*` and `/api/stacks/*` endpoints the buttons call.
- **Host/agent-level (real infrastructure):** 2.2 (fill), 2.3 (vzdump), 4.1/4.2 (host reboot + real
/dev swap), 4.3 (agent restart → ReassertGuestBinds), 4.4 (agent stop + unmount).
## 3. Phase-0 PBS floor
- `felhom-pbs:backup/ct/9201/2026-06-23T17:28:25Z` (datastore `felhom-spike`). **Not used** — nothing
wedged; every destructive step was reversed in-band.
## 4. End state
- **9201 restored to a healthy baseline:** infra-only apps healthy, both drives live at **stable** (correct
device/size), `/mnt/sys_drive` separate + warning-free, registry = 2 stable paths, settings UI clean (no
banner, both restart buttons). Left intentionally at `onboot:0` (original state; F3 reported not fixed).
Two pre-existing absent drive-intents (`277a2179` enrolled-absent, `ce9d1d81` ejected) remain — they
predate the campaign (present in the Phase-0 baseline), not introduced here.
## 5. Coverage gaps
- **1.3 / 1.4 / 1.8 (loop-device init/attach/wipe):** not executable — the agent's block-device whitelist
(`felhom-agent internal/storage/validate.go:124`) refuses non-whitelisted devices (a safety PASS), and no
spare whitelisted disk exists (the 2 real drives hold un-floored data). Stable-registration covered by 1.2
live + unit tests. *Needs a real spare disk in a supervised run.*
- **4.7 (capstone destroy + re-provision):** deferred — operator-gated, needs golden volid + retrieval
passphrase (not provided this run). Compensating coverage via 1.2/3.3/4.1.
- **3.2 auth/CSRF rejection:** can't be shown on the passwordless demo (gate is a no-op); wrapping is
source-verified.
- **4.2 was a REAL /dev swap** (better than the planned code-path fallback) — no gap there.
---
## Campaign result (overview)
- **Phases 05 complete.** The three shipped features hold up: the **stable-vs-raw path model** (v0.780.80)
is correct across register/deregister/disk-card/eject/decommission/migrate/backup (Phase 1); the
**/mnt/sys_drive split** (agent v0.40.0) isolates disk-full, is backed up, and survives reboots (Phase 2);
the **retired banner + standalone server-restart** (v0.81.0) work — full-server restart recovers drives at
stable, sys_drive separate, apps recreated, registry intact (Phase 3); and durability survives a real host
reboot with a genuine sdb↔sdc swap, the intent gate, the mountpoint-gate, and concurrency (Phase 4).
- **4 findings, all reported-not-fixed** (see DELIVERABLE SUMMARY §1): **F3** `onboot:0` (guest won't
auto-start after host reboot — highest impact), **F2** controller restart bounces FileBrowser, **F1**
restic dead-weight binary, **F4** stable double-bind. No data-loss, no path-escape, no silent corruption
found. Demo restored to a healthy baseline.