From 02d63ed0702e4336ef25d6fc16b889ac810f2f72 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Tue, 7 Jul 2026 20:26:32 +0200 Subject: [PATCH] GL-2: --mode appliance|byo install profile (host-install v1.10.0) + harness Mandatory install profile (no default): byo = a host the operator does not own. Break-glass gated OFF at its call site in byo (root@pam never touched), mandatory --cores/--memory, argv-time refusals (--enable-oob/--rotate-recovery, non-9.x PVE, missing --acl-storages), host-mutation disclosure + typed-hostname ack, byo config asserts (lan_resolver/wg_tunnel/oob off; byo flips the lan_resolver write default to off), pool+ACL verify asserts in BOTH modes (R2), --preflight-only (no state, PASS/FAIL verdict), resume mode-mismatch refusal, FELHOM_INSTALL_STATE_DIR harness override. NEW scripts/hostinstall-mode-harness.sh: static refusal matrix C1-C4 + grep invariants + PVE tier (C5 + A/B dry transcripts). 16/16 PASS on felhom-pve (C5 live); red-proofs RP-1..RP-3 run->fail->revert. shellcheck clean at severity=warning. Docs: day0-install SC.5 byo section + trust model; REUSE row; CONTEXT + REPORT. Live drill = GL-6 (supervised); STOP honored (no non-dry run). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6 --- CONTEXT.md | 18 ++ REPORT.md | 122 +++++++---- REUSE.md | 1 + documentation/runbooks/day0-install.md | 69 ++++++- scripts/CHANGELOG.md | 34 ++++ scripts/felhom-host-install.sh | 267 +++++++++++++++++++++++-- scripts/hostinstall-mode-harness.sh | 236 ++++++++++++++++++++++ 7 files changed, 687 insertions(+), 60 deletions(-) create mode 100644 scripts/hostinstall-mode-harness.sh diff --git a/CONTEXT.md b/CONTEXT.md index 24bd107..3f48ea9 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -3,6 +3,24 @@ > Created with the REUSE.md rollout (2026-07-03). Authoritative history: `hub/CHANGELOG.md` (hub), > `website/CHANGELOG.md`, `scripts/CHANGELOG.md`; end-of-task detail in `REPORT.md`. +- **2026-07-07 — TASK GL-2 SHIPPED: `--mode appliance|byo` install profile (host-install v1.10.0 + + harness)** — go-live items G2/G4/G5 implemented, **awaiting the supervised GL-6 drill** for live + provision/uninstall + C7 verify-drift. DECISIONS: the flag is MANDATORY (no default; the old + `--mode provision` value retired with a specific die, `dr` stub kept); break-glass gated at its + CALL SITE (byo never touches root@pam); byo requires `--cores`+`--memory`, refuses + `--enable-oob`/`--rotate-recovery`/non-9.x-PVE/missing `--acl-storages`; disclosure + typed-hostname + ack; **the config write's `lan_resolver` default flips to OFF in byo** (the spec assumed + default-off — it was default-ON; appliance unchanged) + post-write asserts refuse + lan_resolver/wg_tunnel/oob=true; pool+ACL verify asserts in BOTH modes (R2 lesson); + `--preflight-only` (no state, PASS/FAIL verdict, proven live on felhom-pve). NEW + `scripts/hostinstall-mode-harness.sh` (static C1–C4 + invariants everywhere; PVE tier: C5 + + A/B dry transcripts) — 16/16 PASS on felhom-pve incl. C5 live; red-proofs RP-1..3 run→fail→revert. + FINDING: demo-felhom's controller.yaml has EMPTY git.username/git.token → any step-5 + (re)install for that customer dies until the operator sets real Gitea read creds (relates to the + standing scope-down+rotate follow-up). `documentation/pilot/GO-LIVE-PACKAGE.md` is NOT in the + repo and no operator copy was provided — the G2/G4/G5 status flip is recorded here instead; + add the doc when the operator supplies it. Next: **GL-6 supervised drill** (appliance + byo + end-to-end, uninstall/re-provision, C7, the ack prompt interactively). - **2026-07-05 — TASK H1 SHIPPED: OOB operator access (hub v0.35.0 + installer + endpoint; agent v0.72.0)** — merged E1+H1. Hub: operator OOB peer (`store/wg_operator.go`, `PUT/GET /admin/wg/operator-peer` global key) + `oob_peer_ip`/`oob_operator_ssh_key` in the desired-state diff --git a/REPORT.md b/REPORT.md index 6db6013..7cb4ba2 100644 --- a/REPORT.md +++ b/REPORT.md @@ -2,46 +2,92 @@ > **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md). -## TASK H1 — OOB operator access (hub + installer + endpoint half) — hub v0.35.0 (2026-07-05) +## TASK GL-2 — BYO-host install profile (`--mode appliance|byo`) — host-install v1.10.0 (2026-07-07) -**Baseline:** felhom.eu @ `a3ee93e` → pushed. Hub `0.34.1` → **`0.35.0`** (live via ArgoCD). Agent half -= felhom-agent v0.72.0. The merged E1+H1 operator-SSH-access feature. Provenance: both -`SPIKE-{felhom-sshd,oob-wg-operator-peer}-2026-07-05`. +**Baseline confirmed:** felhom.eu @ `1a1e42ad`, `SCRIPT_VERSION="1.9.1"` → **`1.10.0`**. +Implementation + static/dry validation only per spec §0 — **the live provision/uninstall runs are +GL-6 (operator-supervised)**; the §13 STOP was honored (zero non-dry installer invocations anywhere). -### Shipped -- **Operator OOB peer** (`store/wg_operator.go`): the fleet operator peer as an UNBOUND `wg_peers` row - (host_id '', note operator-oob) at an EXPLICIT /32; validated; last-write-wins rotation; rides - ListWGPeers → peersync pushes it to the endpoint. `PUT/GET /admin/wg/operator-peer` (global key); - PUT also takes `ssh_pubkey` (hub_settings) + bumps every host's generation. -- **Desired-state** (`api/wg.go` mergeWireguard): the served wireguard block gains `oob_peer_ip` - (rendered into the box's AllowedIPs — survives self-heal [OF-1]) + `oob_operator_ssh_key` (agent - writes felhom-sshd's authorized_keys). Absent operator peer → byte-identical pass-through. -- **OOB health** (`monitor/host_oob.go`): ingests the agent `oob` heartbeat stanza; transition-based - `oob_degraded`/`oob_recovered` warning (felhom-sshd down while the operator peer is configured, OR - config invalid). Unconfigured OOB never alerts. -- **host-install** (`scripts`): `--enable-oob` + `install_oob` install the static felhom-sshd + belt + - `felhom-op` user (RuntimeDirectory-guarded); uninstall removes them. -- **Doc 06 §4.5/§4.6 amended**: forwarding ON but per-pair allow-listed, box↔box drop now explicit; - the `oob` health stanza + `oob_degraded` alert. Endpoint runbook §9 added (forward chain + operator - peer registration). +### Files created / modified (single repo, `main`) -### Tests + red-proofs (all green: `go build/vet/test ./...`) -- store: operator-peer round-trip/rotate/unbound + validation (reserved/taken/out-of-subnet); merge - includes oob_peer_ip when configured (absent = byte-identical). -- api: operator-peer PUT self-scoped-global-only; ssh_pubkey validation. -- monitor: degraded/recovered transitions; config-invalid alerts; unconfigured-never-alerts; - no-stanza-ignored. Red-proof: neuter the emit → the alert test fails. +- `scripts/felhom-host-install.sh` — v1.10.0 (Parts 1–3; see scripts/CHANGELOG.md for the feature list) +- `scripts/hostinstall-mode-harness.sh` — NEW (Part 4) +- `documentation/runbooks/day0-install.md` — `--mode` in every command, new §C.5 (byo command + + trust-model paragraph + preflight-only), §C.6 (pre-v1.10.0 resume note), 4 new troubleshooting rows +- `scripts/CHANGELOG.md`, `REUSE.md` (install-profile-gate pattern row), `CONTEXT.md`, this file -### Live validation (felhom-pve + dev endpoint) -Endpoint set up first (ip_forward=1 + per-pair forward chain + operator peer registered → pushed to -wg0). Both spikes' key probes re-run as acceptance — all pass: operator→box SSH as felhom-op with -scoped sudo; the OF-1 self-heal /32 survival; coexistence (stock :22 PID 922 unchanged, distinct host -keys, felhom-op denied on :22); the belt (LAN→port dropped, :22 untouched); box↔box drop (counter) + -peersync survival; PBS unaffected throughout; the healthy `oob` stanza reaches the hub + -`oob_degraded`/`oob_recovered` fired around a real felhom-sshd downtime. Full detail + the 5 -live-found-and-fixed agent bugs: felhom-agent `REPORT.md`. +### Spec deviations / judgment calls (each verified against the real code) -### Notes -- CGNAT still unproven; IPv6/AAAA out of scope; operator-key auto-rotation is a manual re-PUT; the - "customer network fully down" case is explicitly OUT OF SCOPE / accepted risk. Operator (global) key - = hub `report_api_key`. felhom-pve's operator peer + felhom-sshd + endpoint forwarding stay live. +1. **`--mode` collision:** v1.9.1 already used `--mode provision|dr`. Resolved: `appliance|byo` are + the only install values, `provision` retired with a specific die, the `dr` stub dispatch kept + verbatim (the documented 10D seam). +2. **Spec §2 was wrong about the lan-resolver**: the config write did `setdefault('lan_resolver', + {"enable": True})` — default-ON, so the spec's byo assert would have died on every byo install. + Fixed by flipping the byo DEFAULT to off (appliance untouched); the assert still dies if a + `--preserve-from` config carries true, which is the intended refusal. +3. The byo skip line reads `break-glass (step 4b) SKIPPED (byo: …)` — NOT the spec's literal + `"4b/8 …"` text, because Scenario B / harness H-B forbid the string `4b/8` in the byo transcript + (an internal spec contradiction; the greppable invariant won). +4. `--preflight-only` skips the byo typed ack (prints the disclosure, logs the skip) — nothing will + be mutated, and a mandatory prompt would break its re-runnable/scripted purpose. +5. `documentation/pilot/GO-LIVE-PACKAGE.md` is not in the repo and no operator copy arrived with the + task — the G2/G4/G5 → "implemented, awaiting GL-6" flip is recorded in CONTEXT.md; add the doc + when supplied. + +### Harness results + +Static tier (Windows Git Bash AND felhom-pve, identical): **bash -n, C1/C1b/C1c, C2a/C2b, C3/C3b/C3c, +C4, C6-static, INV-1 (one appliance-gated break-glass call site), INV-2 (chpasswd unreachable outside +step_break_glass), INV-3 (--mode in usage) — all PASS.** + +PVE tier (felhom-pve, read-only, temp `FELHOM_INSTALL_STATE_DIR`): **C5 PASS live** (bogus +`--acl-storages` entry → die naming it, pre-hub, pre-passphrase). H-A/H-B dry transcripts ran with +the out-of-band demo passphrase (file→file, never printed) but **die at step 5 with rc=1 — a +customer-data blocker, not a code bug**: demo-felhom's controller.yaml serves EMPTY +`git.username`/`git.token`. The transcript span through the break-glass gate was validated manually +instead: appliance transcript contains `4b/8`; byo transcript contains **no** +`4b/8`/`chpasswd`/`recovery-credential` and **does** contain the disclosure block, the dry ack line, +`acl storages all present`, and the host-DNS `:53` line (it correctly showed the live dnsmasq). +`--preflight-only` proven live both ways: PASS verdict + rc 0 + **zero state entries**; bogus-storage +run → specific die + `PRE-FLIGHT FAIL (exit 1)` verdict. + +**Red-proofs (run → fail → revert, against mutated scratch copies — the repo file was never mutated):** +- RP-1 un-gated the break-glass call site → INV-1 FAIL (13/16). Reverted (scratch deleted). +- RP-2 dropped the byo caps requirement → C1/C1b/C1c FAIL (11/16). Reverted. +- RP-3 dropped the resume mode-mismatch check → C4 FAIL (13/16). Reverted. + +### §9 rule 9 — the `_state_mark` dry-run finding + +Verified in code: **both** `_state_mark` and `_state_put` begin with `$DRY_RUN && return 0` — dry +runs write nothing. Belt on top: the harness points `FELHOM_INSTALL_STATE_DIR` (new, harness-only +env override — none existed) at a throwaway temp dir for EVERY invocation, so even non-dry refusal +cases can never touch a live `state.json`. Confirmed live: felhom-pve's real state.json was +byte-untouched (it only ever saw the temp dir) and the preflight-only runs left 0 entries. + +### Shellcheck + +v0.10.0 (build server 180 + felhom-pve; not installed locally). **Clean at `--severity=warning`** +on both scripts. Full-severity: 2 pre-existing SC2015 *info* notes on untouched v1.9.1 lines +(A&&B||C in step_verify) — triaged, not refactored (spec rule 2); 1 new SC2154 false positive on +the EXIT trap (assignment inside the trap string) — targeted disable comment. + +### NOT yet live-validated — awaiting supervised GL-6 + +- The full appliance AND byo installs end-to-end (rc=0 transcripts blocked today by the + demo-felhom git-creds gap below). +- The uninstall → re-provision drill under both modes. +- C7: the verify pool/ACL asserts firing on real drift (needs a deliberately-broken live box). +- The byo typed-hostname ack answered interactively (only its dry/preflight-only branches ran). + +### Observations (documented, not acted on) + +- **demo-felhom has EMPTY `git.username`/`git.token`** in its served controller.yaml — any step-5 + (re)install for that customer dies until real Gitea read creds are set (operator; ties into the + standing "scope down + rotate the package-WRITE git token" security follow-up). +- felhom-pve's live install predates `_state_put` — its state.json has only `completed` (no + `customer_id`/`provisioned_vmid`/`mode`); a future `--uninstall` there needs an explicit `--vmid`, + and §C.6 of the runbook covers the missing-mode resume. +- A `--resume` that skips preflight never records `mode` into an old state file (it only writes + during a full preflight) — harmless (C4 only fires when a mode IS recorded), noted for GL-6. +- The live host-DNS line on felhom-pve shows the agent's own dnsmasq (lan_resolver is ON there — + correct for an appliance; it is exactly what byo now refuses). diff --git a/REUSE.md b/REUSE.md index ece574e..50ce5a6 100644 --- a/REUSE.md +++ b/REUSE.md @@ -98,6 +98,7 @@ | Website page | website/index.html | UTF-8 **with BOM**; shared `