docs(pilot): GL-6 wrap-up — gates LIVE-VALIDATED, REPORT, decision log
G2/G4/G5/G6/G7 -> LIVE-VALIDATED; G1 -> DONE (armed-key round-trip proven, manifest bump confirmed); G9 -> deferred to GL-7; G10 -> GL-7 prereq. Findings F1/F2/F3/F6/F7 filed for a v1.11.4 installer follow-up (none block Peti). REPORT.md = the GL-6 drill. Next: GL-7. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
@@ -2,79 +2,57 @@
|
||||
|
||||
> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md).
|
||||
|
||||
## TASK GL-4 — uninstaller gap-closure + operator-key day-0 fold — host-install v1.11.0 (2026-07-08)
|
||||
## RUNBOOK GL-6 — supervised go-live drill (demo box, two full cycles) — 2026-07-08
|
||||
|
||||
**Baseline confirmed:** felhom.eu @ `a63cc715`, `SCRIPT_VERSION="1.10.0"` → **`1.11.0`**;
|
||||
felhom-agent reference @ `4c408467` (read-only — authz schema from `internal/config/config.go`
|
||||
SignerKey, self-update paths from `configs/felhom-selfupdate-guarded`, drive root from
|
||||
`internal/storage/{claim,netmount}.go`, bind store `/var/lib/felhom-agent/guest-binds.json` from
|
||||
`cmd/felhom-agent/main.go:514`). §12 STOP honored: zero live install/uninstall runs anywhere.
|
||||
Full per-phase record: `documentation/pilot/DRILL-GL6-2026-07-08.md`. Supervised, Viktor gating
|
||||
every phase; destructive (guest 9201 destroyed + rebuilt twice on felhom-pve). Go-live item **G7**
|
||||
and the live-validation landing zone for GL-1…GL-5b. Ended with the demo healthy on normal life.
|
||||
|
||||
**Part 0 — GO-LIVE-PACKAGE.md was ABSENT AGAIN.** The spec said the operator attaches it; it is
|
||||
not in the repo, not anywhere under `E:\git`. Per the spec's own fallback: G6/G1 status recorded
|
||||
in CONTEXT.md, no Commit 1. Third task in a row without the doc — flagging it loudly.
|
||||
**Baselines:** felhom.eu `bd97bc34` → **`87ba30ae`** (host-install `v1.11.1` → **`v1.11.3`** — two
|
||||
fixes shipped mid-drill). felhom-agent `7f073936` (v0.76.0). Manifest confirmed vouching agent
|
||||
0.76.0 + golden 0.103.0.
|
||||
|
||||
### Files modified
|
||||
### Result: DRILL COMPLETE — all gates GO
|
||||
|
||||
- `scripts/felhom-host-install.sh` → v1.11.0 (Parts 1–3; feature detail in scripts/CHANGELOG.md)
|
||||
- `scripts/hostinstall-mode-harness.sh` — +13 static cases + PVE-tier GL4 H-U (extended, not forked)
|
||||
- `documentation/runbooks/day0-install.md` — §C.5b key-pin section; Part E teardown additions
|
||||
- `scripts/CHANGELOG.md`, `REUSE.md` (parity-pattern row), `CONTEXT.md`, this file
|
||||
- **Key-pin chain (GL-1's whole point) — legs 1–4 all green:** box==repo==password-manager
|
||||
fingerprints; signers ARMED from first boot (byo + appliance); the **pinned operational key
|
||||
authorized a real signed `agent_update` round-trip** 0.76.0→0.76.1→0.76.0 (verify vs pinned
|
||||
`felhom-op-1` + nonce, A/B swap, dwell, commit, crash-guard silent), throwaway 0.76.1 deleted.
|
||||
- **Two full destroy/rebuild cycles + two clean uninstalls.** Uninstall: busy-drive prop handled
|
||||
(warn+guidance, never `-l/-f`, statement=retry), KEPT/WIPED statement verbatim, host audit clean,
|
||||
**drive data intact on all three checks** (witness hash matched). byo (Phase 2) + appliance
|
||||
(Phase 6) installs both green; appliance's 4b vaulted the root@pam recovery credential to the hub.
|
||||
- **C7 live-validated:** an induced pool-membership drop made `step_verify` `log_error` +
|
||||
WITH-WARNINGS (read-only); repaired via `--adopt-pool` → green.
|
||||
- **Drive re-enrollment via the real UI** (Chrome bridge) — additive, pre-drill data reappeared;
|
||||
restore-test `mount_parity: ok` on both PBS (5m35s) and local tiers.
|
||||
|
||||
### Per-scenario results
|
||||
### Two fixes shipped mid-drill
|
||||
|
||||
| Scenario | Result | How |
|
||||
|---|---|---|
|
||||
| A — full uninstall dry covers everything | **PASS (live dry)** | GL4 H-U on felhom-pve vs the real guest 9201: selfupdate-artifact removals + kept-vs-wiped statement present, no `umount -l/-f`, no destructive op on any `/mnt/felhom-drives/` path (no drives currently mounted there → per-mount umount lines correctly conditional) |
|
||||
| B — guest-only | **static** | branch adds `_guest_drive_note` (bind-store best-effort, generic fallback) + guest-scoped statement; not transcript-runnable on felhom-pve (no second Felhom guest) — GL-6 |
|
||||
| C1 dormant WARN | grep-shape PASS (GL4-C1) + verify code path | |
|
||||
| C2 keys written | **runtime PASS** (GL4-C2: valid file passes resolution, dies later at preflight, never at key parse) + write-shape in GL4-C4 | |
|
||||
| C3 malformed file | **runtime PASS ×5** (unknown role / non-key line / missing key_id comment / empty file / missing file — each dies at argv naming the line) | |
|
||||
| C4 preserve rule | grep-shape PASS (`if signers:` guard + pin-rotation notice) + red-proof RP-2 | |
|
||||
| C5 file overrides constants | grep-shape PASS (notice + constant reset) — not runtime-testable while the shipped constants are empty (deliberate); the notice fires only when both sources are set | |
|
||||
| D parity | **PASS** (curated token list over the uninstall section; every disclosure artifact covered by a removal or an explicit KEPT line) | |
|
||||
1. **host-install v1.11.2 (Gate-0 ruling, G3):** empty customer git creds → **anonymous artifact
|
||||
fetch** (the Felhom artifacts are world-readable by operator ruling; the hub-vouched sha256 stays
|
||||
the integrity root). All 12 installer fetch targets 200-anon-verified first. Harness `+GL6-ANON`.
|
||||
2. **host-install v1.11.3 (finding F4, HIGH — Viktor HOLD):** `--resume` fully skipped completed
|
||||
producer steps (`should_skip X || step_X`), but token/enroll/grows produce in-memory outputs
|
||||
later steps consume → a resumed install wrote a config missing `hub.host_id`/`proxmox.token`
|
||||
(daemon crash-loop) and passed `-rootfs-grow ""` (flag parse error). Fix: those three re-run every
|
||||
pass (idempotent); golden's volid re-derived from local. **Live-proven** (crafted dry-resume +
|
||||
the subsequent appliance install ran single-shot). Harness `+GL6-F4`; 27/27; shellcheck clean.
|
||||
|
||||
**Red-proofs (run→fail→revert on scratch copies; repo file never mutated):**
|
||||
RP-1 dropped the 4b4 block → GL4-D FAILED (24/28). RP-2 made the signers write unconditional →
|
||||
GL4-C4 FAILED. RP-3 dropped the unknown-role die → GL4-C3a FAILED. All reverted (scratch deleted).
|
||||
### Findings filed (small installer follow-up, none block Peti)
|
||||
|
||||
**Gates:** `bash -n` clean; shellcheck 0.10.0 `--severity=warning` clean on both scripts (the two
|
||||
pre-existing SC2015 infos on untouched v1.9.1 lines remain triaged); harness **25/25 static
|
||||
locally, 28/28 on felhom-pve** (H-A/H-B still cred-gated by demo-felhom's empty git credentials —
|
||||
unchanged since GL-2). GL-2's Scenario-A contract re-verified: all v1.10.0 cases still pass.
|
||||
F1 config `.bak` residue on uninstall · F2 per-drive `mnt-*.mount` residue · F3 byo fresh install on
|
||||
a populated host needs `--allow-new-leaf` (guard counts any guest) · F6 leftover broken dnsmasq
|
||||
breaks LAN-client DNS on a byo host · F7 pool re-assert is bring-up-only (no continuous heal;
|
||||
`--adopt-pool` repairs). (F5 was a self-corrected misdiagnosis — the public DNS wildcard works.)
|
||||
|
||||
### Implementation notes / judgment calls
|
||||
### Deferred to GL-7 (with Peti)
|
||||
|
||||
1. **4b4 inventory** (from the guarded script, not spec memory): wrapper +
|
||||
`felhom-agent.prev` + `felhom-agent.new.*` + `felhom-agent-rollback.service` + the
|
||||
`felhom-agent-limits.conf` drop-in (+dir). `pending.json` + the staging dir live under
|
||||
`$AGENT_STATE_DIR` — already removed in step 4 (noted in the block comment).
|
||||
2. **Root-bind guard**: with a busy child mount, v1.10.0's `run umount /mnt/felhom-drives` would
|
||||
have DIED mid-teardown (set -e); the root umount is now skipped with a warn when children stayed
|
||||
busy — the statement lists them as "retry".
|
||||
3. The statement's conditional lines: PBS (any `pbs`-type storage present), recovery credential
|
||||
(`_state_has break_glass`, snapshotted BEFORE the state file is deleted); hub record + escrow
|
||||
always printed (escrow phrased "if one exists" — there is no cheap local detector).
|
||||
4. Key-file validation happens at argv time (before the passphrase prompt) so all C3 cases run on
|
||||
any machine; options-prefixed authorized_keys lines (e.g. `command="…"`) are rejected as
|
||||
"bad key type" — the pin format is deliberately plain `<type> <b64> <key_id>`.
|
||||
5. H-U initially FAILED on felhom-pve because the naive assertion flagged the legitimate
|
||||
`rm -f /usr/local/sbin/felhom-mkfs-guarded` line as "contains mkfs" — fixed to Scenario A's
|
||||
real invariant (destructive ops on `/mnt/felhom-drives/` paths only).
|
||||
Dashboard password (operator-set via hub — G9) + geo-restriction; **G10** (public dashboard is open
|
||||
until the operator sets a password) is a GL-7 prerequisite. Full demo app redeploy + LAN-DNS
|
||||
alignment (re-provision changed the guest DHCP IP) are Viktor's return-to-normal.
|
||||
|
||||
### NOT live-validated — awaiting supervised GL-6
|
||||
### Honest leftovers (unchanged)
|
||||
|
||||
- The real (non-dry) full uninstall: drive umounts incl. a genuinely busy mount, the statement on
|
||||
a real teardown, residue re-diff at v1.11.0.
|
||||
- Guest-only mode on a multi-guest host (Scenario B transcript).
|
||||
- An armed key-pin install end-to-end (needs the operator's real keys — the pin CEREMONY, incl.
|
||||
pinning felhom-pve, is the operator's; the constants ship empty).
|
||||
- C7-class verify-drift firing.
|
||||
|
||||
### Follow-ups
|
||||
|
||||
- **OPERATOR:** the key ceremony (offline keypairs → fill `OPERATOR_KEY_*` or keep a pubkey file);
|
||||
the GL-1 manifest bump is still pending too (agent 0.74.0 / golden 0.103.0).
|
||||
- **OPERATOR:** actually attach/commit GO-LIVE-PACKAGE.md (absent for the third task running).
|
||||
- Observation: `install-v191.sh` (v1.9.1) is what `~/drill` still carries; GL-6 should fetch the
|
||||
served v1.11.0 from felhom.eu (git-sync auto-deploys this push in ~1–2 min).
|
||||
S5 customer-data DR drill; G12 parity-on-real-drift + PBS-tier runtime; CGNAT retest; Impl-3;
|
||||
OOB spike. **Next: GL-7** (Peti's day-0 runbook + tester agreement).
|
||||
|
||||
Reference in New Issue
Block a user