admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(pilot): GL-6 wrap-up — gates LIVE-VALIDATED, REPORT, decision log

Browse Source 
G2/G4/G5/G6/G7 -> LIVE-VALIDATED; G1 -> DONE (armed-key round-trip proven,
manifest bump confirmed); G9 -> deferred to GL-7; G10 -> GL-7 prereq.
Findings F1/F2/F3/F6/F7 filed for a v1.11.4 installer follow-up (none block
Peti). REPORT.md = the GL-6 drill. Next: GL-7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
authentik Default Admin
2026-07-08 16:07:51 +02:00
parent 87ba30ae91
commit 1dc86523b5
2 changed files with 71 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files
documentation/pilot/GO-LIVE-PACKAGE.md
+29 -14
View File
@@ -9,6 +9,7 @@ catalog `2ebe082a` · felhom.eu `1a1e42ad`/hub v0.35.0 · host-install v1.9.1 ·
**Updated 2026-07-07 (GL-2 shipped):** felhom.eu `02d63ed0` · host-install **v1.10.0** + `scripts/hostinstall-mode-harness.sh`.
**Updated 2026-07-08 (GL-4 + GL-5 shipped):** felhom.eu `f7cc6a72` · host-install **v1.11.0** (uninstall gap-closure + key-pin plumbing) · felhom-agent **v0.75.0** (DR bind overrides, live on felhom-pve).
**Updated 2026-07-08 (GL-5b shipped):** felhom-agent **v0.76.0** live + PUBLISHED (`9828c5f7…f50b`) — restore-test full-fidelity + mount parity (new G12); the manifest bump target is now 0.76.0.
**Updated 2026-07-08 (GL-6 DRILL COMPLETE):** felhom.eu `87ba30ae` · host-install **v1.11.3** (F4 resume fix shipped mid-drill; v1.11.2 anon-fetch ruling). Two full destroy/rebuild cycles on felhom-pve; manifest confirmed vouching agent 0.76.0 + golden 0.103.0. G2/G4/G5/G6/G7 LIVE-VALIDATED; G1 DONE; key-pin legs 1–4 proven. Record: `DRILL-GL6-2026-07-08.md`.
---
@@ -58,15 +59,15 @@ Status legend: ✔ DONE · ◐ PARTIAL · ○ OPEN · ⚠ BLOCKER
| ID | Item | Status | Evidence / gap |
|----|------|--------|----------------|
| **G1** | **Publish + vouch chain current & self-update alive.** ARTIFACTS PUBLISHED (RUNBOOK GL-1, felhom.eu `a63cc715`): agent **0.74.0** sha `1ec3f588…76af05` (the exact live-proven felhom-pve bytes, hash independently re-verified anonymously) and golden **0.103.0** sha `8481e8a1…6026e` (clean-room bake on the virgin drill VM, controller 0.103.0 baked, both split volumes in the vzdump). **Key-pin day-0 PLUMBING SHIPPED in GL-4** (v1.11.0: `OPERATOR_KEY_*` constants empty-by-default + `--operator-pubkey-file` → `authz.signers`; preserved-signers never clobbered; verify reports armed/dormant). Remaining: (a) **operator manifest bump — target agent 0.76.0** (published 2026-07-08, sha `9828c5f7…f50b`, the exact live felhom-pve bytes; 0.74.0's published artifact has broken guest-loss DR, 0.75.0 was superseded UNPUBLISHED); until the bump fresh installs still land 0.63.0/0.98.3; (b) the **key CEREMONY** (real keypairs + felhom-pve pin) = operator. | ◐ | RUNBOOK-GL1-publish-2026-07-07.md; host-install v1.11.0 |
| **G2** | **BYO-host install profile.** SHIPPED as `--mode appliance\|byo` (mandatory, no default; the flag was repurposed from the retired `provision\|dr` — `dr` stub seam preserved). byo: break-glass gated OFF at its single call site (:2040–2044 — root@pam never touched, nothing vaulted), caps mandatory, `--enable-oob`/`--rotate-recovery` refused, host-mutation disclosure + typed-hostname ack, post-write config asserts (`lan_resolver`/`wg_tunnel`/`oob` must be off — asserts run BEFORE `_state_mark agent_config`, so `--resume` re-asserts). Red-proofs RP-1..3 run→fail→revert. Appliance flow unchanged (regression cases green). | ◐ | host-install v1.10.0 @ `02d63ed0`; verified at file:line 2026-07-07. **Remaining: GL-6 live drill** (C7 verify-assert firing, interactive ack) |
| **G1** | **Publish + vouch chain current & self-update alive.** ARTIFACTS PUBLISHED (RUNBOOK GL-1, felhom.eu `a63cc715`): agent **0.74.0** sha `1ec3f588…76af05` (the exact live-proven felhom-pve bytes, hash independently re-verified anonymously) and golden **0.103.0** sha `8481e8a1…6026e` (clean-room bake on the virgin drill VM, controller 0.103.0 baked, both split volumes in the vzdump). **Key-pin day-0 PLUMBING SHIPPED in GL-4** (v1.11.0: `OPERATOR_KEY_*` constants empty-by-default + `--operator-pubkey-file` → `authz.signers`; preserved-signers never clobbered; verify reports armed/dormant). **DONE (GL-6):** hub Day-0 manifest confirmed vouching agent **0.76.0** + golden **0.103.0** (fetched via the box API at Phase 0); the key CEREMONY is complete (real op/rec keys pinned in v1.11.1, fingerprints box==repo==manager, ARMED from first boot); and the **pinned operational key authorized a real signed `agent_update` round-trip** end-to-end (0.76.0→0.76.1→0.76.0). Published chain consumed live via anonymous fetch (v1.11.2 ruling). | ✔ | DRILL-GL6-2026-07-08.md §Phase 3; RUNBOOK-GL1 |
| **G2** | **BYO-host install profile.** SHIPPED as `--mode appliance\|byo` (mandatory, no default; the flag was repurposed from the retired `provision\|dr` — `dr` stub seam preserved). byo: break-glass gated OFF at its single call site (:2040–2044 — root@pam never touched, nothing vaulted), caps mandatory, `--enable-oob`/`--rotate-recovery` refused, host-mutation disclosure + typed-hostname ack, post-write config asserts (`lan_resolver`/`wg_tunnel`/`oob` must be off — asserts run BEFORE `_state_mark agent_config`, so `--resume` re-asserts). Red-proofs RP-1..3 run→fail→revert. Appliance flow unchanged. **LIVE-VALIDATED (GL-6):** byo install on real hardware — disclosure printed, Viktor typed the hostname ack, NO 4b/chpasswd/vault, config asserts held, signers armed. | ✔ | DRILL-GL6 §Phase 2 |
| **G3** | **`git.token` scope-down + rotate — UNBLOCKED, and now coupled to a visibility ruling.** Gate 3c (GL-1): both generic packages are **anonymously world-readable**, so rotating the customer token to read-only cannot break fresh-install fetches; the empty-creds die on demo-felhom is the installer's `resolve_git_creds` precondition (:326–332), not a Gitea requirement. **NEW FINDING (project-Claude probe, 2026-07-08): the ENTIRE Gitea instance is anonymously readable — all nine repos `private=False`, including `homelab-manifests` (k3s manifests — audit for raw Secret objects) and personal repos (`work`, `revfulop-calendar`). Only the geo-fence (HU/DE/US) limits reach.** Coupling warning: if the reaction is to lock the instance ("require sign-in"), the anonymous day-0 package fetch dies and read-scoped tokens become MANDATORY — retest the fetch path with a token before any visibility change. | ○ ⚠ | GL-1 record §Gate 3c; anonymous `repos/search` probe 2026-07-08 |
| **G4** | **Preflight guards (refuse-to-provision gate).** SHIPPED in v1.10.0: byo dies on non-9.x PVE (unless `--force`) and on missing `--acl-storages` entries (named); `:53` DNS info line + existing-guest count; `--preflight-only` runs the full mode-aware checks with a PASS/FAIL verdict and **zero state marks** (live-proven on felhom-pve, incl. the FAIL verdict on a bogus storage). Verify step now asserts pool membership + all scoped-ACL grants (user+token) in BOTH modes — R2's lesson made structural. | ◐ | host-install v1.10.0 @ `02d63ed0` (verify asserts read at source). **Remaining: GL-6** (asserts firing on real drift) |
| **G5** | **LXC caps mandatory on BYO hosts.** SHIPPED: byo refuses to run without explicit `--cores` AND `--memory` (:947, fires before the passphrase prompt); existing cap-sanity warns retained. Appliance unchanged (optional, golden default). Cap VALUES for Peti's guest still to be chosen (§6). | ◐ | host-install v1.10.0 :947. **Remaining: pick values + GL-6** |
| **G6** | **Uninstaller gap-closure (customer offboarding).** SHIPPED in host-install v1.11.0 (GL-4, `f7cc6a72`): 4b4 self-update-artifact removal (wrapper, .prev/.new.* slots, rollback unit, limits drop-in — derived from the guarded script itself), per-drive unmounts under `/mnt/felhom-drives` (plain umount ONLY, busy = warn+guidance, root-bind guarded), the **KEPT-vs-WIPED statement** in BOTH modes (drives/PBS/hub record/escrow/vaulted recovery credential named as living on), guest-only drive note from the bind store. Harness: disclosure↔uninstall parity check (GL4-D) + full-uninstall DRY transcript vs live 9201; red-proofs RP-1..3. | ◐ | host-install v1.11.0 @ `f7cc6a72`. **Remaining: GL-6** (real teardown incl. a busy drive) |
| **G7** | **Demo-box drill: provision → uninstall → re-provision** (runbook, supervised). Proves G2/G4/G5/G6 end-to-end before they touch Peti's hardware. | ○ | D4 |
| **G4** | **Preflight guards (refuse-to-provision gate).** SHIPPED in v1.10.0: byo dies on non-9.x PVE (unless `--force`) and on missing `--acl-storages` entries (named); `:53` DNS info line + existing-guest count; `--preflight-only` runs the full mode-aware checks with a PASS/FAIL verdict and **zero state marks** (live-proven on felhom-pve, incl. the FAIL verdict on a bogus storage). Verify asserts pool membership + all scoped-ACL grants in BOTH modes. **LIVE-VALIDATED (GL-6): C7 fired** — an induced pool-membership drop made `step_verify` `log_error` + WITH-WARNINGS, read-only; `--preflight-only` also proven live. | ✔ | DRILL-GL6 §Phase 4 (C7) |
| **G5** | **LXC caps mandatory on BYO hosts.** SHIPPED: byo refuses to run without explicit `--cores` AND `--memory` (:947, fires before the passphrase prompt); existing cap-sanity warns retained. Appliance unchanged (optional). **LIVE-VALIDATED (GL-6):** byo install ran with `--cores 4 --memory 8192` (Viktor's N100 values); the guest's `pct config` shows cores 4 / memory 8192. | ✔ | DRILL-GL6 §Phase 2 |
| **G6** | **Uninstaller gap-closure (customer offboarding).** SHIPPED in host-install v1.11.0 (GL-4, `f7cc6a72`): 4b4 self-update-artifact removal (wrapper, .prev/.new.* slots, rollback unit, limits drop-in — derived from the guarded script itself), per-drive unmounts under `/mnt/felhom-drives` (plain umount ONLY, busy = warn+guidance, root-bind guarded), the **KEPT-vs-WIPED statement** in BOTH modes (drives/PBS/hub record/escrow/vaulted recovery credential named as living on), guest-only drive note from the bind store. **LIVE-VALIDATED (GL-6): two real uninstalls** — busy-drive prop handled (warn+guidance, no `-l/-f`, statement=retry), KEPT/WIPED statement verbatim, host audit clean, **drive data intact ×3** (witness hash matched every time). Findings F1 (config `.bak` residue) + F2 (mount-unit residue) filed. | ✔ | DRILL-GL6 §Phase 1/5 |
| **G7** | **Demo-box drill** — **DONE 2026-07-08.** Two full destroy/rebuild cycles (byo + appliance) + two uninstalls on felhom-pve, Viktor gating every phase; F4 found+fixed+proven mid-drill (v1.11.3), C7 + armed-key chain live-validated, data intact ×3. | ✔ | DRILL-GL6-2026-07-08.md |
| **G8** | **DR bring-up bind-override fix** (agent). SHIPPED as **v0.75.0** (GL-5, `b3446213`, LIVE on felhom-pve) + **scratch-DR live-validated end-to-end** (9310 from a real 9201 archive: mp0 200G + mp1 50G restored with content, real mp8/mp9 binds, zero unusedN, clean auto-teardown; the same op failed outright on v0.74.0). **TWO live-discovered PVE rules beyond the spike:** explicit-params restore requires an explicit rootfs AND silently drops unlisted mountpoints — the full param set now derives from the archive's embedded config (`ExtractArchiveConfig`, 200 under the scoped token; bind LAYOUT stays the known constants). | ◐ | agent v0.75.0 @ `b3446213`; REPORT.md. **Remaining: the full customer-data DR drill (GL-6/S5). (The publish follow-up landed as 0.76.0 — GL-5b/G12.)** |
| **G9** | **Auth-on onboarding.** Auth + CSRF proven in campaign 2; remaining work is procedural: Peti's onboarding sets the dashboard password via the REAL settings pipeline (anti-F9) and verifies login. Runbook step in GL-7. | ◐ | controller v0.103.0 (F-C2-1 live-validated) |
| **G9** | **Auth-on onboarding.** Auth + CSRF proven in campaign 2; remaining work is procedural: GL-6 confirmed the dashboard password is **operator-set via the hub config** (anti-F9 pipeline), NOT a customer-dashboard field — until set the public dashboard is OPEN (**G10 exposure**). Rehearsed the location; **password-set + geo-restriction deferred to GL-7 with Peti** (a GL-7 prerequisite). | ◐ | DRILL-GL6 §Phase 2b |
| **G10** | **Local-API posture documented + spot-verified.** Token→vmid binding is structural (server.go:23, :53–55, :153 — verified this session); write it into the security notes, including the vmbr0-reachable surface and its TLS+token defense. One CC spot-check: a token minted for guest A is refused for a guest-B-scoped op (test exists? verify; add if hollow). | ◐ | localapi/server.go as cited |
| **G12** | **Restore-test full-fidelity verification** (GL-5b, agent **v0.76.0**). The restore-test had GL-5 finding #2's mirror image: its live-source-config override path tripped PVE's drop-unlisted-mountpoints rule, so it boot-verified scratch guests WITHOUT their storage mpN. Now: params derive from the ARCHIVE's embedded config (`drRestoreOverrides`, same as DR) + a **mount-parity assert** (restored mpN set vs the archive's; missing/mispathed/undersized/extra = FAIL naming the delta) so the rule can never regress into a green light. `MountParity`+`MountInventory` ride the hub wire record (additive). Honest cost, measured: **3m4s** on the local tier (the extraction adds ~2min over data-less; cheaper than the DR-derived ~7m estimate; PBS tier will run longer). Live-proven on felhom-pve: parity ok, inventory mp0 200G + mp1 50G + 2 throwaways; a rotated-out archive volid refuses up front. | ◐ | agent v0.76.0 (published `9828c5f7…f50b`); agent REPORT.md. Remaining: parity-on-real-drift (GL-6 family) + PBS-tier runtime |
| **G11** | **Pilot backup statement.** Assert local backups green on Peti's box at onboarding; the tester agreement states plainly: *no offsite copy yet — a dead/stolen box or dead drive without a second local target loses data*. Honesty is the sovereignty pitch. | ○ | D3; agent CONTEXT.md v0.66/0.67 (retarget reverted; Tier-1/2 split pending) |
@@ -87,8 +88,8 @@ BUNDLE cert/key/token migration item; hub-floor auto-update.
| **GL-3** git.token rotate + scope-down | Operator (small) | Gitea admin + config template | G3 | — |
| **GL-4** Uninstaller gap-closure + key-pin fold | ✔ **DONE 2026-07-08** (host-install v1.11.0, `f7cc6a72`) | felhom.eu/scripts | G6 + G1-plumbing | GL-2 |
| **GL-5** DR bind-override | ✔ **DONE 2026-07-08** (agent v0.75.0, `b3446213`, live-validated) | felhom-agent | G8 | — |
| **GL-6** Demo-box drill | RUNBOOK, supervised | — | G7 | GL-1, GL-2, GL-4 |
| **GL-7** Peti day-0 runbook + onboarding | RUNBOOK, supervised | felhom.eu/documentation/pilot | G9, G10, G11 | GL-6 green |
| **GL-6** Demo-box drill | ✔ **DONE 2026-07-08** (`DRILL-GL6-2026-07-08.md`; F4 fix v1.11.3 mid-drill) | felhom.eu/scripts (v1.11.3) | G7 | GL-1, GL-2, GL-4 |
| **GL-7** Peti day-0 runbook + onboarding | **NEXT** (GL-6 green) | felhom.eu/documentation/pilot | G9, G10, G11 + findings F1/F2/F3/F6/F7 | GL-6 green |
Recommended order: **GL-1 and GL-3 immediately** (operator-heavy, unblock everything), GL-2 next
(the biggest CC task), GL-5 in parallel (independent repo), then GL-4 → GL-6 → GL-7.
@@ -146,14 +147,28 @@ Recommended order: **GL-1 and GL-3 immediately** (operator-heavy, unblock everyt
to 0.76.0.** Scenario-B red-proof run→fail→revert; bringup.go zero line changes (DR re-run
trigger not fired).
- 2026-07-08 — **GL-6 DRILL COMPLETE** (supervised, Viktor gating; `DRILL-GL6-2026-07-08.md`,
felhom.eu `87ba30ae`). Two full destroy/rebuild cycles on felhom-pve (byo #2 + appliance #6) +
two clean uninstalls; abort-path (local+PBS backup, restore-test parity ok) verified before any
destroy. **Key-pin legs 1–4 all green** (box==repo==manager fingerprints; armed from first boot;
the pinned operational key authorized a real signed `agent_update` round-trip 0.76.0→0.76.1→0.76.0).
**C7 live-validated** (verify's pool assert fired on induced drift). **F4 (resume-correctness,
HIGH) found + FIXED mid-drill (host-install v1.11.3) + live-proven** (the appliance install then
ran single-shot). Gate-0 ruling shipped **v1.11.2** (empty customer git creds → anonymous artifact
fetch; all 12 fetch targets 200-anon-verified). New findings F1/F2/F3/F6/F7 filed for a small
installer follow-up (none block Peti). G2/G4/G5/G6/G7 → LIVE-VALIDATED; G1 → DONE; G9 → deferred
to GL-7; G10 (open dashboard until operator password) → GL-7 prerequisite.
## 6. Open questions & operator actions
**Operator actions (Viktor):**
- **Hub manifest bump** (Configs → Day-0 artifacts) — enter verbatim (UPDATED for GL-5b):
agent `0.76.0` / `9828c5f75e719fb2e1fc3523f9c322b595a963ec1dcdf37ca42a96bedab7f50b`;
golden `0.103.0` / `8481e8a14e2aa0abe3831cc40e5aea4c32f2017e7561c15dca8a674df6a6026e`.
Until then fresh installs still land 0.63.0/0.98.3. Do NOT vouch 0.74.0 (broken guest-loss DR);
0.75.0 was superseded unpublished.
- **Hub manifest bump — DONE** (confirmed at GL-6 Phase 0: the manifest vouches agent `0.76.0` +
golden `0.103.0`). No action.
- **Small installer follow-up (GL-6 findings, none block Peti)** — a v1.11.4 pass:
F1 (purge `/etc/felhom-agent` + `agent.json.bak-*` on uninstall), F2 (stop+remove the agent's
per-drive `mnt-*.mount` units on uninstall), F3 (narrow the populated-host leaf guard to
`felhom_guests`, or document `--allow-new-leaf` in the byo command), F6 (stop+mask the leftover
broken dnsmasq on a byo host), F7 (optional periodic pool-membership reconcile).
- **Gitea visibility ruling** (new, see G3): decide deliberately — (a) keep felhom repos public
as an open-source posture, but audit + privatize `homelab-manifests` and the personal repos;
or (b) lock the instance, which REQUIRES first proving the day-0 package fetch with a