From 1dc86523b53dd780bb39bc1886390e77929a1b46 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Wed, 8 Jul 2026 16:07:51 +0200 Subject: [PATCH] =?UTF-8?q?docs(pilot):=20GL-6=20wrap-up=20=E2=80=94=20gat?= =?UTF-8?q?es=20LIVE-VALIDATED,=20REPORT,=20decision=20log?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit G2/G4/G5/G6/G7 -> LIVE-VALIDATED; G1 -> DONE (armed-key round-trip proven, manifest bump confirmed); G9 -> deferred to GL-7; G10 -> GL-7 prereq. Findings F1/F2/F3/F6/F7 filed for a v1.11.4 installer follow-up (none block Peti). REPORT.md = the GL-6 drill. Next: GL-7. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6 --- REPORT.md | 106 ++++++++++--------------- documentation/pilot/GO-LIVE-PACKAGE.md | 43 ++++++---- 2 files changed, 71 insertions(+), 78 deletions(-) diff --git a/REPORT.md b/REPORT.md index ba2c9c2..99a8eca 100644 --- a/REPORT.md +++ b/REPORT.md @@ -2,79 +2,57 @@ > **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md). -## TASK GL-4 — uninstaller gap-closure + operator-key day-0 fold — host-install v1.11.0 (2026-07-08) +## RUNBOOK GL-6 — supervised go-live drill (demo box, two full cycles) — 2026-07-08 -**Baseline confirmed:** felhom.eu @ `a63cc715`, `SCRIPT_VERSION="1.10.0"` → **`1.11.0`**; -felhom-agent reference @ `4c408467` (read-only — authz schema from `internal/config/config.go` -SignerKey, self-update paths from `configs/felhom-selfupdate-guarded`, drive root from -`internal/storage/{claim,netmount}.go`, bind store `/var/lib/felhom-agent/guest-binds.json` from -`cmd/felhom-agent/main.go:514`). §12 STOP honored: zero live install/uninstall runs anywhere. +Full per-phase record: `documentation/pilot/DRILL-GL6-2026-07-08.md`. Supervised, Viktor gating +every phase; destructive (guest 9201 destroyed + rebuilt twice on felhom-pve). Go-live item **G7** +and the live-validation landing zone for GL-1…GL-5b. Ended with the demo healthy on normal life. -**Part 0 — GO-LIVE-PACKAGE.md was ABSENT AGAIN.** The spec said the operator attaches it; it is -not in the repo, not anywhere under `E:\git`. Per the spec's own fallback: G6/G1 status recorded -in CONTEXT.md, no Commit 1. Third task in a row without the doc — flagging it loudly. +**Baselines:** felhom.eu `bd97bc34` → **`87ba30ae`** (host-install `v1.11.1` → **`v1.11.3`** — two +fixes shipped mid-drill). felhom-agent `7f073936` (v0.76.0). Manifest confirmed vouching agent +0.76.0 + golden 0.103.0. -### Files modified +### Result: DRILL COMPLETE — all gates GO -- `scripts/felhom-host-install.sh` → v1.11.0 (Parts 1–3; feature detail in scripts/CHANGELOG.md) -- `scripts/hostinstall-mode-harness.sh` — +13 static cases + PVE-tier GL4 H-U (extended, not forked) -- `documentation/runbooks/day0-install.md` — §C.5b key-pin section; Part E teardown additions -- `scripts/CHANGELOG.md`, `REUSE.md` (parity-pattern row), `CONTEXT.md`, this file +- **Key-pin chain (GL-1's whole point) — legs 1–4 all green:** box==repo==password-manager + fingerprints; signers ARMED from first boot (byo + appliance); the **pinned operational key + authorized a real signed `agent_update` round-trip** 0.76.0→0.76.1→0.76.0 (verify vs pinned + `felhom-op-1` + nonce, A/B swap, dwell, commit, crash-guard silent), throwaway 0.76.1 deleted. +- **Two full destroy/rebuild cycles + two clean uninstalls.** Uninstall: busy-drive prop handled + (warn+guidance, never `-l/-f`, statement=retry), KEPT/WIPED statement verbatim, host audit clean, + **drive data intact on all three checks** (witness hash matched). byo (Phase 2) + appliance + (Phase 6) installs both green; appliance's 4b vaulted the root@pam recovery credential to the hub. +- **C7 live-validated:** an induced pool-membership drop made `step_verify` `log_error` + + WITH-WARNINGS (read-only); repaired via `--adopt-pool` → green. +- **Drive re-enrollment via the real UI** (Chrome bridge) — additive, pre-drill data reappeared; + restore-test `mount_parity: ok` on both PBS (5m35s) and local tiers. -### Per-scenario results +### Two fixes shipped mid-drill -| Scenario | Result | How | -|---|---|---| -| A — full uninstall dry covers everything | **PASS (live dry)** | GL4 H-U on felhom-pve vs the real guest 9201: selfupdate-artifact removals + kept-vs-wiped statement present, no `umount -l/-f`, no destructive op on any `/mnt/felhom-drives/` path (no drives currently mounted there → per-mount umount lines correctly conditional) | -| B — guest-only | **static** | branch adds `_guest_drive_note` (bind-store best-effort, generic fallback) + guest-scoped statement; not transcript-runnable on felhom-pve (no second Felhom guest) — GL-6 | -| C1 dormant WARN | grep-shape PASS (GL4-C1) + verify code path | | -| C2 keys written | **runtime PASS** (GL4-C2: valid file passes resolution, dies later at preflight, never at key parse) + write-shape in GL4-C4 | | -| C3 malformed file | **runtime PASS ×5** (unknown role / non-key line / missing key_id comment / empty file / missing file — each dies at argv naming the line) | | -| C4 preserve rule | grep-shape PASS (`if signers:` guard + pin-rotation notice) + red-proof RP-2 | | -| C5 file overrides constants | grep-shape PASS (notice + constant reset) — not runtime-testable while the shipped constants are empty (deliberate); the notice fires only when both sources are set | | -| D parity | **PASS** (curated token list over the uninstall section; every disclosure artifact covered by a removal or an explicit KEPT line) | | +1. **host-install v1.11.2 (Gate-0 ruling, G3):** empty customer git creds → **anonymous artifact + fetch** (the Felhom artifacts are world-readable by operator ruling; the hub-vouched sha256 stays + the integrity root). All 12 installer fetch targets 200-anon-verified first. Harness `+GL6-ANON`. +2. **host-install v1.11.3 (finding F4, HIGH — Viktor HOLD):** `--resume` fully skipped completed + producer steps (`should_skip X || step_X`), but token/enroll/grows produce in-memory outputs + later steps consume → a resumed install wrote a config missing `hub.host_id`/`proxmox.token` + (daemon crash-loop) and passed `-rootfs-grow ""` (flag parse error). Fix: those three re-run every + pass (idempotent); golden's volid re-derived from local. **Live-proven** (crafted dry-resume + + the subsequent appliance install ran single-shot). Harness `+GL6-F4`; 27/27; shellcheck clean. -**Red-proofs (run→fail→revert on scratch copies; repo file never mutated):** -RP-1 dropped the 4b4 block → GL4-D FAILED (24/28). RP-2 made the signers write unconditional → -GL4-C4 FAILED. RP-3 dropped the unknown-role die → GL4-C3a FAILED. All reverted (scratch deleted). +### Findings filed (small installer follow-up, none block Peti) -**Gates:** `bash -n` clean; shellcheck 0.10.0 `--severity=warning` clean on both scripts (the two -pre-existing SC2015 infos on untouched v1.9.1 lines remain triaged); harness **25/25 static -locally, 28/28 on felhom-pve** (H-A/H-B still cred-gated by demo-felhom's empty git credentials — -unchanged since GL-2). GL-2's Scenario-A contract re-verified: all v1.10.0 cases still pass. +F1 config `.bak` residue on uninstall · F2 per-drive `mnt-*.mount` residue · F3 byo fresh install on +a populated host needs `--allow-new-leaf` (guard counts any guest) · F6 leftover broken dnsmasq +breaks LAN-client DNS on a byo host · F7 pool re-assert is bring-up-only (no continuous heal; +`--adopt-pool` repairs). (F5 was a self-corrected misdiagnosis — the public DNS wildcard works.) -### Implementation notes / judgment calls +### Deferred to GL-7 (with Peti) -1. **4b4 inventory** (from the guarded script, not spec memory): wrapper + - `felhom-agent.prev` + `felhom-agent.new.*` + `felhom-agent-rollback.service` + the - `felhom-agent-limits.conf` drop-in (+dir). `pending.json` + the staging dir live under - `$AGENT_STATE_DIR` — already removed in step 4 (noted in the block comment). -2. **Root-bind guard**: with a busy child mount, v1.10.0's `run umount /mnt/felhom-drives` would - have DIED mid-teardown (set -e); the root umount is now skipped with a warn when children stayed - busy — the statement lists them as "retry". -3. The statement's conditional lines: PBS (any `pbs`-type storage present), recovery credential - (`_state_has break_glass`, snapshotted BEFORE the state file is deleted); hub record + escrow - always printed (escrow phrased "if one exists" — there is no cheap local detector). -4. Key-file validation happens at argv time (before the passphrase prompt) so all C3 cases run on - any machine; options-prefixed authorized_keys lines (e.g. `command="…"`) are rejected as - "bad key type" — the pin format is deliberately plain ` `. -5. H-U initially FAILED on felhom-pve because the naive assertion flagged the legitimate - `rm -f /usr/local/sbin/felhom-mkfs-guarded` line as "contains mkfs" — fixed to Scenario A's - real invariant (destructive ops on `/mnt/felhom-drives/` paths only). +Dashboard password (operator-set via hub — G9) + geo-restriction; **G10** (public dashboard is open +until the operator sets a password) is a GL-7 prerequisite. Full demo app redeploy + LAN-DNS +alignment (re-provision changed the guest DHCP IP) are Viktor's return-to-normal. -### NOT live-validated — awaiting supervised GL-6 +### Honest leftovers (unchanged) -- The real (non-dry) full uninstall: drive umounts incl. a genuinely busy mount, the statement on - a real teardown, residue re-diff at v1.11.0. -- Guest-only mode on a multi-guest host (Scenario B transcript). -- An armed key-pin install end-to-end (needs the operator's real keys — the pin CEREMONY, incl. - pinning felhom-pve, is the operator's; the constants ship empty). -- C7-class verify-drift firing. - -### Follow-ups - -- **OPERATOR:** the key ceremony (offline keypairs → fill `OPERATOR_KEY_*` or keep a pubkey file); - the GL-1 manifest bump is still pending too (agent 0.74.0 / golden 0.103.0). -- **OPERATOR:** actually attach/commit GO-LIVE-PACKAGE.md (absent for the third task running). -- Observation: `install-v191.sh` (v1.9.1) is what `~/drill` still carries; GL-6 should fetch the - served v1.11.0 from felhom.eu (git-sync auto-deploys this push in ~1–2 min). +S5 customer-data DR drill; G12 parity-on-real-drift + PBS-tier runtime; CGNAT retest; Impl-3; +OOB spike. **Next: GL-7** (Peti's day-0 runbook + tester agreement). diff --git a/documentation/pilot/GO-LIVE-PACKAGE.md b/documentation/pilot/GO-LIVE-PACKAGE.md index 10edf23..206fe18 100644 --- a/documentation/pilot/GO-LIVE-PACKAGE.md +++ b/documentation/pilot/GO-LIVE-PACKAGE.md @@ -9,6 +9,7 @@ catalog `2ebe082a` · felhom.eu `1a1e42ad`/hub v0.35.0 · host-install v1.9.1 · **Updated 2026-07-07 (GL-2 shipped):** felhom.eu `02d63ed0` · host-install **v1.10.0** + `scripts/hostinstall-mode-harness.sh`. **Updated 2026-07-08 (GL-4 + GL-5 shipped):** felhom.eu `f7cc6a72` · host-install **v1.11.0** (uninstall gap-closure + key-pin plumbing) · felhom-agent **v0.75.0** (DR bind overrides, live on felhom-pve). **Updated 2026-07-08 (GL-5b shipped):** felhom-agent **v0.76.0** live + PUBLISHED (`9828c5f7…f50b`) — restore-test full-fidelity + mount parity (new G12); the manifest bump target is now 0.76.0. +**Updated 2026-07-08 (GL-6 DRILL COMPLETE):** felhom.eu `87ba30ae` · host-install **v1.11.3** (F4 resume fix shipped mid-drill; v1.11.2 anon-fetch ruling). Two full destroy/rebuild cycles on felhom-pve; manifest confirmed vouching agent 0.76.0 + golden 0.103.0. G2/G4/G5/G6/G7 LIVE-VALIDATED; G1 DONE; key-pin legs 1–4 proven. Record: `DRILL-GL6-2026-07-08.md`. --- @@ -58,15 +59,15 @@ Status legend: ✔ DONE · ◐ PARTIAL · ○ OPEN · ⚠ BLOCKER | ID | Item | Status | Evidence / gap | |----|------|--------|----------------| -| **G1** | **Publish + vouch chain current & self-update alive.** ARTIFACTS PUBLISHED (RUNBOOK GL-1, felhom.eu `a63cc715`): agent **0.74.0** sha `1ec3f588…76af05` (the exact live-proven felhom-pve bytes, hash independently re-verified anonymously) and golden **0.103.0** sha `8481e8a1…6026e` (clean-room bake on the virgin drill VM, controller 0.103.0 baked, both split volumes in the vzdump). **Key-pin day-0 PLUMBING SHIPPED in GL-4** (v1.11.0: `OPERATOR_KEY_*` constants empty-by-default + `--operator-pubkey-file` → `authz.signers`; preserved-signers never clobbered; verify reports armed/dormant). Remaining: (a) **operator manifest bump — target agent 0.76.0** (published 2026-07-08, sha `9828c5f7…f50b`, the exact live felhom-pve bytes; 0.74.0's published artifact has broken guest-loss DR, 0.75.0 was superseded UNPUBLISHED); until the bump fresh installs still land 0.63.0/0.98.3; (b) the **key CEREMONY** (real keypairs + felhom-pve pin) = operator. | ◐ | RUNBOOK-GL1-publish-2026-07-07.md; host-install v1.11.0 | -| **G2** | **BYO-host install profile.** SHIPPED as `--mode appliance\|byo` (mandatory, no default; the flag was repurposed from the retired `provision\|dr` — `dr` stub seam preserved). byo: break-glass gated OFF at its single call site (:2040–2044 — root@pam never touched, nothing vaulted), caps mandatory, `--enable-oob`/`--rotate-recovery` refused, host-mutation disclosure + typed-hostname ack, post-write config asserts (`lan_resolver`/`wg_tunnel`/`oob` must be off — asserts run BEFORE `_state_mark agent_config`, so `--resume` re-asserts). Red-proofs RP-1..3 run→fail→revert. Appliance flow unchanged (regression cases green). | ◐ | host-install v1.10.0 @ `02d63ed0`; verified at file:line 2026-07-07. **Remaining: GL-6 live drill** (C7 verify-assert firing, interactive ack) | +| **G1** | **Publish + vouch chain current & self-update alive.** ARTIFACTS PUBLISHED (RUNBOOK GL-1, felhom.eu `a63cc715`): agent **0.74.0** sha `1ec3f588…76af05` (the exact live-proven felhom-pve bytes, hash independently re-verified anonymously) and golden **0.103.0** sha `8481e8a1…6026e` (clean-room bake on the virgin drill VM, controller 0.103.0 baked, both split volumes in the vzdump). **Key-pin day-0 PLUMBING SHIPPED in GL-4** (v1.11.0: `OPERATOR_KEY_*` constants empty-by-default + `--operator-pubkey-file` → `authz.signers`; preserved-signers never clobbered; verify reports armed/dormant). **DONE (GL-6):** hub Day-0 manifest confirmed vouching agent **0.76.0** + golden **0.103.0** (fetched via the box API at Phase 0); the key CEREMONY is complete (real op/rec keys pinned in v1.11.1, fingerprints box==repo==manager, ARMED from first boot); and the **pinned operational key authorized a real signed `agent_update` round-trip** end-to-end (0.76.0→0.76.1→0.76.0). Published chain consumed live via anonymous fetch (v1.11.2 ruling). | ✔ | DRILL-GL6-2026-07-08.md §Phase 3; RUNBOOK-GL1 | +| **G2** | **BYO-host install profile.** SHIPPED as `--mode appliance\|byo` (mandatory, no default; the flag was repurposed from the retired `provision\|dr` — `dr` stub seam preserved). byo: break-glass gated OFF at its single call site (:2040–2044 — root@pam never touched, nothing vaulted), caps mandatory, `--enable-oob`/`--rotate-recovery` refused, host-mutation disclosure + typed-hostname ack, post-write config asserts (`lan_resolver`/`wg_tunnel`/`oob` must be off — asserts run BEFORE `_state_mark agent_config`, so `--resume` re-asserts). Red-proofs RP-1..3 run→fail→revert. Appliance flow unchanged. **LIVE-VALIDATED (GL-6):** byo install on real hardware — disclosure printed, Viktor typed the hostname ack, NO 4b/chpasswd/vault, config asserts held, signers armed. | ✔ | DRILL-GL6 §Phase 2 | | **G3** | **`git.token` scope-down + rotate — UNBLOCKED, and now coupled to a visibility ruling.** Gate 3c (GL-1): both generic packages are **anonymously world-readable**, so rotating the customer token to read-only cannot break fresh-install fetches; the empty-creds die on demo-felhom is the installer's `resolve_git_creds` precondition (:326–332), not a Gitea requirement. **NEW FINDING (project-Claude probe, 2026-07-08): the ENTIRE Gitea instance is anonymously readable — all nine repos `private=False`, including `homelab-manifests` (k3s manifests — audit for raw Secret objects) and personal repos (`work`, `revfulop-calendar`). Only the geo-fence (HU/DE/US) limits reach.** Coupling warning: if the reaction is to lock the instance ("require sign-in"), the anonymous day-0 package fetch dies and read-scoped tokens become MANDATORY — retest the fetch path with a token before any visibility change. | ○ ⚠ | GL-1 record §Gate 3c; anonymous `repos/search` probe 2026-07-08 | -| **G4** | **Preflight guards (refuse-to-provision gate).** SHIPPED in v1.10.0: byo dies on non-9.x PVE (unless `--force`) and on missing `--acl-storages` entries (named); `:53` DNS info line + existing-guest count; `--preflight-only` runs the full mode-aware checks with a PASS/FAIL verdict and **zero state marks** (live-proven on felhom-pve, incl. the FAIL verdict on a bogus storage). Verify step now asserts pool membership + all scoped-ACL grants (user+token) in BOTH modes — R2's lesson made structural. | ◐ | host-install v1.10.0 @ `02d63ed0` (verify asserts read at source). **Remaining: GL-6** (asserts firing on real drift) | -| **G5** | **LXC caps mandatory on BYO hosts.** SHIPPED: byo refuses to run without explicit `--cores` AND `--memory` (:947, fires before the passphrase prompt); existing cap-sanity warns retained. Appliance unchanged (optional, golden default). Cap VALUES for Peti's guest still to be chosen (§6). | ◐ | host-install v1.10.0 :947. **Remaining: pick values + GL-6** | -| **G6** | **Uninstaller gap-closure (customer offboarding).** SHIPPED in host-install v1.11.0 (GL-4, `f7cc6a72`): 4b4 self-update-artifact removal (wrapper, .prev/.new.* slots, rollback unit, limits drop-in — derived from the guarded script itself), per-drive unmounts under `/mnt/felhom-drives` (plain umount ONLY, busy = warn+guidance, root-bind guarded), the **KEPT-vs-WIPED statement** in BOTH modes (drives/PBS/hub record/escrow/vaulted recovery credential named as living on), guest-only drive note from the bind store. Harness: disclosure↔uninstall parity check (GL4-D) + full-uninstall DRY transcript vs live 9201; red-proofs RP-1..3. | ◐ | host-install v1.11.0 @ `f7cc6a72`. **Remaining: GL-6** (real teardown incl. a busy drive) | -| **G7** | **Demo-box drill: provision → uninstall → re-provision** (runbook, supervised). Proves G2/G4/G5/G6 end-to-end before they touch Peti's hardware. | ○ | D4 | +| **G4** | **Preflight guards (refuse-to-provision gate).** SHIPPED in v1.10.0: byo dies on non-9.x PVE (unless `--force`) and on missing `--acl-storages` entries (named); `:53` DNS info line + existing-guest count; `--preflight-only` runs the full mode-aware checks with a PASS/FAIL verdict and **zero state marks** (live-proven on felhom-pve, incl. the FAIL verdict on a bogus storage). Verify asserts pool membership + all scoped-ACL grants in BOTH modes. **LIVE-VALIDATED (GL-6): C7 fired** — an induced pool-membership drop made `step_verify` `log_error` + WITH-WARNINGS, read-only; `--preflight-only` also proven live. | ✔ | DRILL-GL6 §Phase 4 (C7) | +| **G5** | **LXC caps mandatory on BYO hosts.** SHIPPED: byo refuses to run without explicit `--cores` AND `--memory` (:947, fires before the passphrase prompt); existing cap-sanity warns retained. Appliance unchanged (optional). **LIVE-VALIDATED (GL-6):** byo install ran with `--cores 4 --memory 8192` (Viktor's N100 values); the guest's `pct config` shows cores 4 / memory 8192. | ✔ | DRILL-GL6 §Phase 2 | +| **G6** | **Uninstaller gap-closure (customer offboarding).** SHIPPED in host-install v1.11.0 (GL-4, `f7cc6a72`): 4b4 self-update-artifact removal (wrapper, .prev/.new.* slots, rollback unit, limits drop-in — derived from the guarded script itself), per-drive unmounts under `/mnt/felhom-drives` (plain umount ONLY, busy = warn+guidance, root-bind guarded), the **KEPT-vs-WIPED statement** in BOTH modes (drives/PBS/hub record/escrow/vaulted recovery credential named as living on), guest-only drive note from the bind store. **LIVE-VALIDATED (GL-6): two real uninstalls** — busy-drive prop handled (warn+guidance, no `-l/-f`, statement=retry), KEPT/WIPED statement verbatim, host audit clean, **drive data intact ×3** (witness hash matched every time). Findings F1 (config `.bak` residue) + F2 (mount-unit residue) filed. | ✔ | DRILL-GL6 §Phase 1/5 | +| **G7** | **Demo-box drill** — **DONE 2026-07-08.** Two full destroy/rebuild cycles (byo + appliance) + two uninstalls on felhom-pve, Viktor gating every phase; F4 found+fixed+proven mid-drill (v1.11.3), C7 + armed-key chain live-validated, data intact ×3. | ✔ | DRILL-GL6-2026-07-08.md | | **G8** | **DR bring-up bind-override fix** (agent). SHIPPED as **v0.75.0** (GL-5, `b3446213`, LIVE on felhom-pve) + **scratch-DR live-validated end-to-end** (9310 from a real 9201 archive: mp0 200G + mp1 50G restored with content, real mp8/mp9 binds, zero unusedN, clean auto-teardown; the same op failed outright on v0.74.0). **TWO live-discovered PVE rules beyond the spike:** explicit-params restore requires an explicit rootfs AND silently drops unlisted mountpoints — the full param set now derives from the archive's embedded config (`ExtractArchiveConfig`, 200 under the scoped token; bind LAYOUT stays the known constants). | ◐ | agent v0.75.0 @ `b3446213`; REPORT.md. **Remaining: the full customer-data DR drill (GL-6/S5). (The publish follow-up landed as 0.76.0 — GL-5b/G12.)** | -| **G9** | **Auth-on onboarding.** Auth + CSRF proven in campaign 2; remaining work is procedural: Peti's onboarding sets the dashboard password via the REAL settings pipeline (anti-F9) and verifies login. Runbook step in GL-7. | ◐ | controller v0.103.0 (F-C2-1 live-validated) | +| **G9** | **Auth-on onboarding.** Auth + CSRF proven in campaign 2; remaining work is procedural: GL-6 confirmed the dashboard password is **operator-set via the hub config** (anti-F9 pipeline), NOT a customer-dashboard field — until set the public dashboard is OPEN (**G10 exposure**). Rehearsed the location; **password-set + geo-restriction deferred to GL-7 with Peti** (a GL-7 prerequisite). | ◐ | DRILL-GL6 §Phase 2b | | **G10** | **Local-API posture documented + spot-verified.** Token→vmid binding is structural (server.go:23, :53–55, :153 — verified this session); write it into the security notes, including the vmbr0-reachable surface and its TLS+token defense. One CC spot-check: a token minted for guest A is refused for a guest-B-scoped op (test exists? verify; add if hollow). | ◐ | localapi/server.go as cited | | **G12** | **Restore-test full-fidelity verification** (GL-5b, agent **v0.76.0**). The restore-test had GL-5 finding #2's mirror image: its live-source-config override path tripped PVE's drop-unlisted-mountpoints rule, so it boot-verified scratch guests WITHOUT their storage mpN. Now: params derive from the ARCHIVE's embedded config (`drRestoreOverrides`, same as DR) + a **mount-parity assert** (restored mpN set vs the archive's; missing/mispathed/undersized/extra = FAIL naming the delta) so the rule can never regress into a green light. `MountParity`+`MountInventory` ride the hub wire record (additive). Honest cost, measured: **3m4s** on the local tier (the extraction adds ~2min over data-less; cheaper than the DR-derived ~7m estimate; PBS tier will run longer). Live-proven on felhom-pve: parity ok, inventory mp0 200G + mp1 50G + 2 throwaways; a rotated-out archive volid refuses up front. | ◐ | agent v0.76.0 (published `9828c5f7…f50b`); agent REPORT.md. Remaining: parity-on-real-drift (GL-6 family) + PBS-tier runtime | | **G11** | **Pilot backup statement.** Assert local backups green on Peti's box at onboarding; the tester agreement states plainly: *no offsite copy yet — a dead/stolen box or dead drive without a second local target loses data*. Honesty is the sovereignty pitch. | ○ | D3; agent CONTEXT.md v0.66/0.67 (retarget reverted; Tier-1/2 split pending) | @@ -87,8 +88,8 @@ BUNDLE cert/key/token migration item; hub-floor auto-update. | **GL-3** git.token rotate + scope-down | Operator (small) | Gitea admin + config template | G3 | — | | **GL-4** Uninstaller gap-closure + key-pin fold | ✔ **DONE 2026-07-08** (host-install v1.11.0, `f7cc6a72`) | felhom.eu/scripts | G6 + G1-plumbing | GL-2 | | **GL-5** DR bind-override | ✔ **DONE 2026-07-08** (agent v0.75.0, `b3446213`, live-validated) | felhom-agent | G8 | — | -| **GL-6** Demo-box drill | RUNBOOK, supervised | — | G7 | GL-1, GL-2, GL-4 | -| **GL-7** Peti day-0 runbook + onboarding | RUNBOOK, supervised | felhom.eu/documentation/pilot | G9, G10, G11 | GL-6 green | +| **GL-6** Demo-box drill | ✔ **DONE 2026-07-08** (`DRILL-GL6-2026-07-08.md`; F4 fix v1.11.3 mid-drill) | felhom.eu/scripts (v1.11.3) | G7 | GL-1, GL-2, GL-4 | +| **GL-7** Peti day-0 runbook + onboarding | **NEXT** (GL-6 green) | felhom.eu/documentation/pilot | G9, G10, G11 + findings F1/F2/F3/F6/F7 | GL-6 green | Recommended order: **GL-1 and GL-3 immediately** (operator-heavy, unblock everything), GL-2 next (the biggest CC task), GL-5 in parallel (independent repo), then GL-4 → GL-6 → GL-7. @@ -146,14 +147,28 @@ Recommended order: **GL-1 and GL-3 immediately** (operator-heavy, unblock everyt to 0.76.0.** Scenario-B red-proof run→fail→revert; bringup.go zero line changes (DR re-run trigger not fired). +- 2026-07-08 — **GL-6 DRILL COMPLETE** (supervised, Viktor gating; `DRILL-GL6-2026-07-08.md`, + felhom.eu `87ba30ae`). Two full destroy/rebuild cycles on felhom-pve (byo #2 + appliance #6) + + two clean uninstalls; abort-path (local+PBS backup, restore-test parity ok) verified before any + destroy. **Key-pin legs 1–4 all green** (box==repo==manager fingerprints; armed from first boot; + the pinned operational key authorized a real signed `agent_update` round-trip 0.76.0→0.76.1→0.76.0). + **C7 live-validated** (verify's pool assert fired on induced drift). **F4 (resume-correctness, + HIGH) found + FIXED mid-drill (host-install v1.11.3) + live-proven** (the appliance install then + ran single-shot). Gate-0 ruling shipped **v1.11.2** (empty customer git creds → anonymous artifact + fetch; all 12 fetch targets 200-anon-verified). New findings F1/F2/F3/F6/F7 filed for a small + installer follow-up (none block Peti). G2/G4/G5/G6/G7 → LIVE-VALIDATED; G1 → DONE; G9 → deferred + to GL-7; G10 (open dashboard until operator password) → GL-7 prerequisite. + ## 6. Open questions & operator actions **Operator actions (Viktor):** -- **Hub manifest bump** (Configs → Day-0 artifacts) — enter verbatim (UPDATED for GL-5b): - agent `0.76.0` / `9828c5f75e719fb2e1fc3523f9c322b595a963ec1dcdf37ca42a96bedab7f50b`; - golden `0.103.0` / `8481e8a14e2aa0abe3831cc40e5aea4c32f2017e7561c15dca8a674df6a6026e`. - Until then fresh installs still land 0.63.0/0.98.3. Do NOT vouch 0.74.0 (broken guest-loss DR); - 0.75.0 was superseded unpublished. +- **Hub manifest bump — DONE** (confirmed at GL-6 Phase 0: the manifest vouches agent `0.76.0` + + golden `0.103.0`). No action. +- **Small installer follow-up (GL-6 findings, none block Peti)** — a v1.11.4 pass: + F1 (purge `/etc/felhom-agent` + `agent.json.bak-*` on uninstall), F2 (stop+remove the agent's + per-drive `mnt-*.mount` units on uninstall), F3 (narrow the populated-host leaf guard to + `felhom_guests`, or document `--allow-new-leaf` in the byo command), F6 (stop+mask the leftover + broken dnsmasq on a byo host), F7 (optional periodic pool-membership reconcile). - **Gitea visibility ruling** (new, see G3): decide deliberately — (a) keep felhom repos public as an open-source posture, but audit + privatize `homelab-manifests` and the personal repos; or (b) lock the instance, which REQUIRES first proving the day-0 package fetch with a