docs(pilot): GL-6 Phases 0-2 evidence + findings F1-F4

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-08 12:52:07 +02:00
parent 0640aa06ea
commit 4cb70ff88b
+102 -5
View File
@@ -66,9 +66,106 @@ Witness file (for the Phase-1/5 drive-data-intact checks):
## Gate 0 — [Viktor]
- Input 1 (byo caps): <pending>
- Input 2 (Phase-6 repopulation): <pending>
- 0c fingerprint confirmation vs password manager: <pending>
- Verdict: <pending>
- Input 1 (byo caps): **--cores 4 --memory 8192**
- Input 2 (Phase-6 repopulation): **fresh + redeploy apps**
- 0c fingerprint confirmation vs password manager: **CONFIRMED — match**
- Verdict: **GO** — with one pre-Phase-1 STOP + ruling (below)
*(Phases 16 appended as they run.)*
### Gate-0 deviation — the demo customer's empty git credentials (Viktor ruling → v1.11.2)
Pre-destroy check found demo-felhom's `git.username`/`git.token` still EMPTY (the GL-2 finding);
the v1.11.1 installer would die at step 5/8. **Viktor ruled: the artifacts are world-readable by
design — anonymous fetch is the supported path.** All 12 installer fetch targets (2 packages + 10
raw configs) validated HTTP 200 anonymously, then **host-install v1.11.2** shipped mid-drill
(felhom.eu `0640aa06`): empty creds → loud WARN + anonymous fetch (conditional curl auth; the
hub-vouched sha256 stays the integrity root; configured creds still used when present). Harness
26/26 (new GL6-ANON case; GL4-C2 assertion updated for the pinned constants).
## Phase 1 — uninstall #1 (full mode, busy drive staged) — PASS with 2 findings
Real `--uninstall --vmid 9201` (typed confirm, v1.11.2 fetched from the served URL):
- **Busy-drive behavior (the staged prop on felhom-usb):** WARN + "Eject the drive via the
dashboard, or stop the apps using it and re-run" — NO `-l`/`-f`, teardown CONTINUED; the other
two drives unmounted ("data stays on the drive"); the root bind correctly left mounted with its
own guard warn. The KEPT-vs-WIPED statement listed felhom-usb as
"still mounted (busy — stop the apps and retry)" — i.e. the code takes the
statement-lists-it-as-retry path; after releasing the prop a plain manual `umount` completed the
job (no re-run needed).
- **KEPT-vs-WIPED statement:** printed verbatim as designed (guest+volumes / agent / self-update
artifacts / watchdog+OOB / pveum+pool / state = WIPED; drives+data / PBS backups / hub record /
escrow = KEPT). The recovery-credential KEPT line correctly keyed off the state file — which on
THIS box predates the 4b state marker, so the line was absent although a G1-vaulted credential
exists (observation: state-keyed conditionals can under-report on pre-v1.10.0 installs).
- **1b audit:** agent binary+baks, selfupdate wrapper+slots, rollback unit+drop-in, watchdog set,
mkfs wrapper, snippets, dnsmasq drop-ins, sudoers, state dirs, pveum user/roles/ACL/pool — ALL
GONE. PVE printed a benign "ignore invalid acl role 'FelhomAgentStore'" notice during ACL
removal (ordering noise, recorded).
- **Drive data intact:** witness file re-hashed on the still-mounted primary path —
`7b00bc3550b10e3952e0e0965f5a2628` == the Phase-0 value.
### Findings
- **GL6-F1 (MED, security-adjacent):** `/etc/felhom-agent/` SURVIVES the uninstall holding
`agent.json.bak-*` files (incl. the operator's same-day ceremony backup) — old configs with
secret material (pve token now dead; the per-host hub api_key is still live because the hub
record survives). The uninstall removes `agent.json` but not `.bak-*` siblings. Fix candidate:
purge `"$agent_cfg"*` / the dir in 4b (v1.11.3). Files left in place pending Viktor's call
(one is his own backup).
- **GL6-F2 (MED):** the agent-created per-drive systemd mount units
(`mnt-felhom-usb.mount` + flash + teszt_enroll) and their ACTIVE primary mounts at
`/mnt/<name>` survive — the statement's "physically removable now" overclaims (the binds are
gone but the partition is still mounted). Fix candidate: uninstall stops+removes the agent's
`mnt-*` units for enrolled namespaces (data untouched). Left as-is for Phase 2 (the re-enroll
behavior against pre-existing mounts is itself drill-relevant).
## Gate 1 — [Viktor]: **GO** (delete the .bak secrets now)
GL6-F1 residue resolved at the gate: `shred -u /etc/felhom-agent/agent.json.bak-*` + `rmdir`
(incl. Viktor's ceremony backup, confirmed disposable). F1/F2 kept as v1.11.3 fix candidates.
## Phase 2 — byo-mode day-0 (Peti's exact path, on real hardware) — PASS (happy path), 3 findings
Command (Viktor typed the `demo-felhom` hostname ack — interactive consent leg):
`./felhom-host-install.sh --customer-id demo-felhom --mode byo --vmid 9201 --cores 4 --memory 8192
--force-gitea-golden --acl-storages "local local-lvm" --passphrase-file <0600>` (+ `--allow-new-leaf
--resume` after F3/F4, see below).
**End state — all green (verify step, verbatim highlights):**
- disclosure printed; **Viktor's typed-hostname ack accepted** (`byo install acknowledged for host demo-felhom`);
- **NO 4b / chpasswd / recovery-credential** anywhere (grep of the full transcript = 0);
- **anonymous artifact fetch worked** (v1.11.2): agent v0.76.0 sha `9828c5f7…` verified, golden
0.103.0 sha `8481e8a1…` verified, both against the hub manifest;
- installed agent `--version` == **0.76.0**; daemon **active**;
- **`authz signers: 2 (operator-signed self-update armed)`** from first boot — **key validation leg 3 ✓** (config carries operational:felhom-op-1 + recovery:felhom-rec-1);
- pool member + all 4 scoped-ACL grants (user+token) present; **caps on the guest: cores 4 / memory 8192**;
- byo config asserts pass (lan_resolver/wg_tunnel/oob all off);
- **controller Up (healthy) 0.103.0** + traefik + cloudflared up; dashboard `HTTP 200` (Host-header probe); hub config-retrieve OK.
### Findings — the byo install's resume path is fragile (happy single-shot path is clean)
- **GL6-F3 (MED, GL-7-relevant):** on a BYO host carrying the OWNER's (non-Felhom) guests, a FRESH
install trips the populated-host leaf guard (`pct list | grep -q .` counts ANY guest) and dies
demanding `--preserve-state-from`/`--allow-new-leaf` — even though no Felhom controller pin exists
to protect on a fresh install. **Peti's box has his own guests → his day-0 command MUST include
`--allow-new-leaf`** (GL-7). Better fix candidate: narrow the guard to `felhom_guests` so a fresh
byo install on a populated non-Felhom host doesn't trip.
- **GL6-F4 (HIGH, resume-correctness):** `--resume` fully skips completed steps via
`should_skip X || step_X`, but later steps consume the skipped steps' IN-MEMORY outputs, which are
then empty → broken results: **(a)** `enroll` skipped → `HOST_ID`/`HOST_API_KEY` empty →
`step_agent_config` writes a config missing `hub.host_id` (+ empty `proxmox.token` when `token` is
skipped) → **daemon crash-loops** (`hub.host_id is required`); the mint-once token secret is
unrecoverable, so recovery = re-run `token`+`enroll`. **(b)** `grows` skipped →
`ROOTFS_GROW`/… empty → provision passes `-rootfs-grow ""` → **flag parse error**. Note
`step_token` even has an internal resume-guard (`[[ -n "$PVE_TOKEN" ]]`) that the `|| step_token`
dispatch DEFEATS — strong signal the dispatch is the bug (steps whose outputs feed later steps must
re-run, or re-derive from the written config, on resume). **Drill recovery:** dropped the affected
completed marks from `state.json` to force re-run — the install then completed cleanly. The
single-shot happy path (no interruption) is unaffected, which is Peti's path.
- **GL6-F2 (from Phase 1, MED):** the agent's per-drive `mnt-*.mount` units + primary `/mnt/<name>`
mounts survived uninstall #1; on this reinstall the parent bind `/mnt/felhom-drives` is back but the
drives are NOT yet bound under it — re-enrollment is the Phase-2b UI leg.
## Gate 2 — [Viktor]: <pending — interactive bring-to-life legs below>
*(Phases 36 appended as they run.)*