GL-2: --mode appliance|byo install profile (host-install v1.10.0) + harness

Mandatory install profile (no default): byo = a host the operator does not
own. Break-glass gated OFF at its call site in byo (root@pam never touched),
mandatory --cores/--memory, argv-time refusals (--enable-oob/--rotate-recovery,
non-9.x PVE, missing --acl-storages), host-mutation disclosure + typed-hostname
ack, byo config asserts (lan_resolver/wg_tunnel/oob off; byo flips the
lan_resolver write default to off), pool+ACL verify asserts in BOTH modes (R2),
--preflight-only (no state, PASS/FAIL verdict), resume mode-mismatch refusal,
FELHOM_INSTALL_STATE_DIR harness override.

NEW scripts/hostinstall-mode-harness.sh: static refusal matrix C1-C4 + grep
invariants + PVE tier (C5 + A/B dry transcripts). 16/16 PASS on felhom-pve
(C5 live); red-proofs RP-1..RP-3 run->fail->revert. shellcheck clean at
severity=warning. Docs: day0-install SC.5 byo section + trust model; REUSE row;
CONTEXT + REPORT. Live drill = GL-6 (supervised); STOP honored (no non-dry run).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-07 20:26:32 +02:00
parent 1a1e42addb
commit 02d63ed070
7 changed files with 687 additions and 60 deletions
+18
View File
@@ -3,6 +3,24 @@
> Created with the REUSE.md rollout (2026-07-03). Authoritative history: `hub/CHANGELOG.md` (hub),
> `website/CHANGELOG.md`, `scripts/CHANGELOG.md`; end-of-task detail in `REPORT.md`.
- **2026-07-07 — TASK GL-2 SHIPPED: `--mode appliance|byo` install profile (host-install v1.10.0 +
harness)** — go-live items G2/G4/G5 implemented, **awaiting the supervised GL-6 drill** for live
provision/uninstall + C7 verify-drift. DECISIONS: the flag is MANDATORY (no default; the old
`--mode provision` value retired with a specific die, `dr` stub kept); break-glass gated at its
CALL SITE (byo never touches root@pam); byo requires `--cores`+`--memory`, refuses
`--enable-oob`/`--rotate-recovery`/non-9.x-PVE/missing `--acl-storages`; disclosure + typed-hostname
ack; **the config write's `lan_resolver` default flips to OFF in byo** (the spec assumed
default-off — it was default-ON; appliance unchanged) + post-write asserts refuse
lan_resolver/wg_tunnel/oob=true; pool+ACL verify asserts in BOTH modes (R2 lesson);
`--preflight-only` (no state, PASS/FAIL verdict, proven live on felhom-pve). NEW
`scripts/hostinstall-mode-harness.sh` (static C1C4 + invariants everywhere; PVE tier: C5 +
A/B dry transcripts) — 16/16 PASS on felhom-pve incl. C5 live; red-proofs RP-1..3 run→fail→revert.
FINDING: demo-felhom's controller.yaml has EMPTY git.username/git.token → any step-5
(re)install for that customer dies until the operator sets real Gitea read creds (relates to the
standing scope-down+rotate follow-up). `documentation/pilot/GO-LIVE-PACKAGE.md` is NOT in the
repo and no operator copy was provided — the G2/G4/G5 status flip is recorded here instead;
add the doc when the operator supplies it. Next: **GL-6 supervised drill** (appliance + byo
end-to-end, uninstall/re-provision, C7, the ack prompt interactively).
- **2026-07-05 — TASK H1 SHIPPED: OOB operator access (hub v0.35.0 + installer + endpoint; agent
v0.72.0)** — merged E1+H1. Hub: operator OOB peer (`store/wg_operator.go`, `PUT/GET
/admin/wg/operator-peer` global key) + `oob_peer_ip`/`oob_operator_ssh_key` in the desired-state
+84 -38
View File
@@ -2,46 +2,92 @@
> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md).
## TASK H1 — OOB operator access (hub + installer + endpoint half) — hub v0.35.0 (2026-07-05)
## TASK GL-2 — BYO-host install profile (`--mode appliance|byo`) — host-install v1.10.0 (2026-07-07)
**Baseline:** felhom.eu @ `a3ee93e` → pushed. Hub `0.34.1`**`0.35.0`** (live via ArgoCD). Agent half
= felhom-agent v0.72.0. The merged E1+H1 operator-SSH-access feature. Provenance: both
`SPIKE-{felhom-sshd,oob-wg-operator-peer}-2026-07-05`.
**Baseline confirmed:** felhom.eu @ `1a1e42ad`, `SCRIPT_VERSION="1.9.1"`**`1.10.0`**.
Implementation + static/dry validation only per spec §0 — **the live provision/uninstall runs are
GL-6 (operator-supervised)**; the §13 STOP was honored (zero non-dry installer invocations anywhere).
### Shipped
- **Operator OOB peer** (`store/wg_operator.go`): the fleet operator peer as an UNBOUND `wg_peers` row
(host_id '', note operator-oob) at an EXPLICIT /32; validated; last-write-wins rotation; rides
ListWGPeers → peersync pushes it to the endpoint. `PUT/GET /admin/wg/operator-peer` (global key);
PUT also takes `ssh_pubkey` (hub_settings) + bumps every host's generation.
- **Desired-state** (`api/wg.go` mergeWireguard): the served wireguard block gains `oob_peer_ip`
(rendered into the box's AllowedIPs — survives self-heal [OF-1]) + `oob_operator_ssh_key` (agent
writes felhom-sshd's authorized_keys). Absent operator peer → byte-identical pass-through.
- **OOB health** (`monitor/host_oob.go`): ingests the agent `oob` heartbeat stanza; transition-based
`oob_degraded`/`oob_recovered` warning (felhom-sshd down while the operator peer is configured, OR
config invalid). Unconfigured OOB never alerts.
- **host-install** (`scripts`): `--enable-oob` + `install_oob` install the static felhom-sshd + belt +
`felhom-op` user (RuntimeDirectory-guarded); uninstall removes them.
- **Doc 06 §4.5/§4.6 amended**: forwarding ON but per-pair allow-listed, box↔box drop now explicit;
the `oob` health stanza + `oob_degraded` alert. Endpoint runbook §9 added (forward chain + operator
peer registration).
### Files created / modified (single repo, `main`)
### Tests + red-proofs (all green: `go build/vet/test ./...`)
- store: operator-peer round-trip/rotate/unbound + validation (reserved/taken/out-of-subnet); merge
includes oob_peer_ip when configured (absent = byte-identical).
- api: operator-peer PUT self-scoped-global-only; ssh_pubkey validation.
- monitor: degraded/recovered transitions; config-invalid alerts; unconfigured-never-alerts;
no-stanza-ignored. Red-proof: neuter the emit → the alert test fails.
- `scripts/felhom-host-install.sh` — v1.10.0 (Parts 13; see scripts/CHANGELOG.md for the feature list)
- `scripts/hostinstall-mode-harness.sh` — NEW (Part 4)
- `documentation/runbooks/day0-install.md``--mode` in every command, new §C.5 (byo command +
trust-model paragraph + preflight-only), §C.6 (pre-v1.10.0 resume note), 4 new troubleshooting rows
- `scripts/CHANGELOG.md`, `REUSE.md` (install-profile-gate pattern row), `CONTEXT.md`, this file
### Live validation (felhom-pve + dev endpoint)
Endpoint set up first (ip_forward=1 + per-pair forward chain + operator peer registered → pushed to
wg0). Both spikes' key probes re-run as acceptance — all pass: operator→box SSH as felhom-op with
scoped sudo; the OF-1 self-heal /32 survival; coexistence (stock :22 PID 922 unchanged, distinct host
keys, felhom-op denied on :22); the belt (LAN→port dropped, :22 untouched); box↔box drop (counter) +
peersync survival; PBS unaffected throughout; the healthy `oob` stanza reaches the hub +
`oob_degraded`/`oob_recovered` fired around a real felhom-sshd downtime. Full detail + the 5
live-found-and-fixed agent bugs: felhom-agent `REPORT.md`.
### Spec deviations / judgment calls (each verified against the real code)
### Notes
- CGNAT still unproven; IPv6/AAAA out of scope; operator-key auto-rotation is a manual re-PUT; the
"customer network fully down" case is explicitly OUT OF SCOPE / accepted risk. Operator (global) key
= hub `report_api_key`. felhom-pve's operator peer + felhom-sshd + endpoint forwarding stay live.
1. **`--mode` collision:** v1.9.1 already used `--mode provision|dr`. Resolved: `appliance|byo` are
the only install values, `provision` retired with a specific die, the `dr` stub dispatch kept
verbatim (the documented 10D seam).
2. **Spec §2 was wrong about the lan-resolver**: the config write did `setdefault('lan_resolver',
{"enable": True})` — default-ON, so the spec's byo assert would have died on every byo install.
Fixed by flipping the byo DEFAULT to off (appliance untouched); the assert still dies if a
`--preserve-from` config carries true, which is the intended refusal.
3. The byo skip line reads `break-glass (step 4b) SKIPPED (byo: …)` — NOT the spec's literal
`"4b/8 …"` text, because Scenario B / harness H-B forbid the string `4b/8` in the byo transcript
(an internal spec contradiction; the greppable invariant won).
4. `--preflight-only` skips the byo typed ack (prints the disclosure, logs the skip) — nothing will
be mutated, and a mandatory prompt would break its re-runnable/scripted purpose.
5. `documentation/pilot/GO-LIVE-PACKAGE.md` is not in the repo and no operator copy arrived with the
task — the G2/G4/G5 → "implemented, awaiting GL-6" flip is recorded in CONTEXT.md; add the doc
when supplied.
### Harness results
Static tier (Windows Git Bash AND felhom-pve, identical): **bash -n, C1/C1b/C1c, C2a/C2b, C3/C3b/C3c,
C4, C6-static, INV-1 (one appliance-gated break-glass call site), INV-2 (chpasswd unreachable outside
step_break_glass), INV-3 (--mode in usage) — all PASS.**
PVE tier (felhom-pve, read-only, temp `FELHOM_INSTALL_STATE_DIR`): **C5 PASS live** (bogus
`--acl-storages` entry → die naming it, pre-hub, pre-passphrase). H-A/H-B dry transcripts ran with
the out-of-band demo passphrase (file→file, never printed) but **die at step 5 with rc=1 — a
customer-data blocker, not a code bug**: demo-felhom's controller.yaml serves EMPTY
`git.username`/`git.token`. The transcript span through the break-glass gate was validated manually
instead: appliance transcript contains `4b/8`; byo transcript contains **no**
`4b/8`/`chpasswd`/`recovery-credential` and **does** contain the disclosure block, the dry ack line,
`acl storages all present`, and the host-DNS `:53` line (it correctly showed the live dnsmasq).
`--preflight-only` proven live both ways: PASS verdict + rc 0 + **zero state entries**; bogus-storage
run → specific die + `PRE-FLIGHT FAIL (exit 1)` verdict.
**Red-proofs (run → fail → revert, against mutated scratch copies — the repo file was never mutated):**
- RP-1 un-gated the break-glass call site → INV-1 FAIL (13/16). Reverted (scratch deleted).
- RP-2 dropped the byo caps requirement → C1/C1b/C1c FAIL (11/16). Reverted.
- RP-3 dropped the resume mode-mismatch check → C4 FAIL (13/16). Reverted.
### §9 rule 9 — the `_state_mark` dry-run finding
Verified in code: **both** `_state_mark` and `_state_put` begin with `$DRY_RUN && return 0` — dry
runs write nothing. Belt on top: the harness points `FELHOM_INSTALL_STATE_DIR` (new, harness-only
env override — none existed) at a throwaway temp dir for EVERY invocation, so even non-dry refusal
cases can never touch a live `state.json`. Confirmed live: felhom-pve's real state.json was
byte-untouched (it only ever saw the temp dir) and the preflight-only runs left 0 entries.
### Shellcheck
v0.10.0 (build server 180 + felhom-pve; not installed locally). **Clean at `--severity=warning`**
on both scripts. Full-severity: 2 pre-existing SC2015 *info* notes on untouched v1.9.1 lines
(A&&B||C in step_verify) — triaged, not refactored (spec rule 2); 1 new SC2154 false positive on
the EXIT trap (assignment inside the trap string) — targeted disable comment.
### NOT yet live-validated — awaiting supervised GL-6
- The full appliance AND byo installs end-to-end (rc=0 transcripts blocked today by the
demo-felhom git-creds gap below).
- The uninstall → re-provision drill under both modes.
- C7: the verify pool/ACL asserts firing on real drift (needs a deliberately-broken live box).
- The byo typed-hostname ack answered interactively (only its dry/preflight-only branches ran).
### Observations (documented, not acted on)
- **demo-felhom has EMPTY `git.username`/`git.token`** in its served controller.yaml — any step-5
(re)install for that customer dies until real Gitea read creds are set (operator; ties into the
standing "scope down + rotate the package-WRITE git token" security follow-up).
- felhom-pve's live install predates `_state_put` — its state.json has only `completed` (no
`customer_id`/`provisioned_vmid`/`mode`); a future `--uninstall` there needs an explicit `--vmid`,
and §C.6 of the runbook covers the missing-mode resume.
- A `--resume` that skips preflight never records `mode` into an old state file (it only writes
during a full preflight) — harmless (C4 only fires when a mode IS recorded), noted for GL-6.
- The live host-DNS line on felhom-pve shows the agent's own dnsmasq (lan_resolver is ON there —
correct for an appliance; it is exactly what byo now refuses).
+1
View File
@@ -98,6 +98,7 @@
| Website page | website/index.html | UTF-8 **with BOM**; shared `<nav>`/`<footer>` byte-identical across pages (only `class="active"` differs); two-tone H1 = `<h1>…<span>accent</span></h1>`; all styling in website/assets/site.css tokens (`:root`) — zero embedded `<style>`; `?v=N` cache-bust on site.css/icons.svg; umami snippet; no CDN fonts; no emoji (sprite icons.svg instead). |
| Gate script | scripts/site_gates.py | Byte-level mechanical gates (BOM, emoji codepoint ranges, nav/footer diff, analytics, banned tokens, cache-bust); run `python scripts/site_gates.py` after ANY website change; non-zero exit on failure. |
| Fetch-validate-install (shell) | scripts/felhom-host-install.sh `step_agent_install` (~L1108) | `fetch_raw` to mktemp → syntax-check (`bash -n`) → `install -m0755 -o root -g root` → only then activate; guarded-mkfs wrapper installed BEFORE the sudoers that references it (ordering is the safety property). All mutations through `run()` (dry-run aware). |
| Install-profile gate (shell) | scripts/felhom-host-install.sh `--mode appliance\|byo` (GL-2, v1.10.0) | Mandatory-flag profile (no default), refusals at argv time BEFORE any prompt/step, risky step gated at its CALL SITE (one auditable place — never a branch inside the step), mode persisted to state.json + resume-mismatch refusal, `FELHOM_INSTALL_STATE_DIR` override for harness isolation. Harness: scripts/hostinstall-mode-harness.sh (static refusal matrix + grep-invariants + PVE dry-transcript tier; red-proofs run against a mutated scratch copy). |
| Website deploy (manifest) | manifests/webpage.yaml | git-sync sidecar (sparse-checkout `/website/` + `/scripts/`, `--link=current`) + init container waits for first sync; nginx serves `current/website`; push to main = deployed, no image build. |
| Secret handling (manifest) | manifests/hub.yaml (env, ~L142) | Secrets via `secretKeyRef` to OUT-OF-BAND secrets created per documentation/runbooks/secrets.md — never inline stringData (see §3). ERRATA (2026-07-03): only `resend-api` is truly out-of-band today; `gitea-creds` is COMMITTED in manifests/felhom.secret.yaml AND live-consumed by hub.yaml — rotation + de-git is a pending operator task (spike SPIKE-a1 appendix). |
| Hub deploy (GitOps) | manifests/hub.yaml `image:` (~L129) | Pinned explicit tag, bumped in git, deliberate ArgoCD sync (auto-sync OFF). Code push alone deploys nothing. |
+65 -4
View File
@@ -14,6 +14,12 @@
>
> Scope: a **shared** Proxmox box (the colleague's-box model) — a box that already runs, or will run,
> non-Felhom guests. A dedicated box is the same procedure minus the sizing caution.
>
> **v1.10.0 (GL-2): every install now REQUIRES `--mode appliance|byo`** — there is no default. Use
> `appliance` for a box Felhom owns/manages end-to-end (this guide's drilled path, unchanged apart
> from the flag); use `byo` for a host the operator does NOT own (the pilot's own PVE) — see §C.5.
> The byo profile is implemented + statically validated but **not yet drill-validated** (that is the
> supervised GL-6 run).
## Who does what
@@ -135,7 +141,7 @@ Notes:
```bash
curl -fsSO https://felhom.eu/scripts/felhom-host-install.sh
chmod +x felhom-host-install.sh
./felhom-host-install.sh -h | head -3 # sanity: must print v1.9.1 (or newer) — the version this guide was drilled against
./felhom-host-install.sh -h | head -3 # sanity: must print v1.10.0 (or newer) — older scripts don't know --mode
```
### C.2 Preview (recommended)
@@ -144,7 +150,7 @@ chmod +x felhom-host-install.sh
read-only and asks for the passphrase):
```bash
./felhom-host-install.sh --customer-id <CUSTOMER-ID> --vmid <VMID> \
./felhom-host-install.sh --customer-id <CUSTOMER-ID> --mode appliance --vmid <VMID> \
--cores 2 --memory 4096 \
--force-gitea-golden --acl-storages "local local-lvm" \
--dry-run
@@ -153,7 +159,7 @@ read-only and asks for the passphrase):
### C.3 The canonical shared-box install command
```bash
./felhom-host-install.sh --customer-id <CUSTOMER-ID> --vmid <VMID> \
./felhom-host-install.sh --customer-id <CUSTOMER-ID> --mode appliance --vmid <VMID> \
--cores 2 --memory 4096 \
--force-gitea-golden --acl-storages "local local-lvm"
```
@@ -163,6 +169,7 @@ Placeholders and flags — what and why:
| Flag | Value | Why |
|---|---|---|
| `--customer-id` | the Part A customer ID | keys every hub call |
| `--mode` | `appliance` or `byo` | REQUIRED, no default. `appliance` = a Felhom-owned/managed box (this section). `byo` = a host the operator does not own — use §C.5's command instead |
| `--vmid` | the free vmid from Part B | ALWAYS pass it explicitly — deterministic, and recorded in the install state for a later `--uninstall` |
| `--cores 2 --memory 4096` | the appliance cap | protects the other guests on a shared box; size to roughly half the host if the box is small (cap ≤ host resources) |
| `--force-gitea-golden` | — | a fresh box has no local golden archive; fetch it from Gitea and verify its sha256 against the hub manifest (this is the normal customer path) |
@@ -196,7 +203,57 @@ Expected duration: minutes; dominated by the two Gitea downloads (agent ~20 MB,
the customer's uplink.
If a step fails: read the error (they are specific), fix the cause, re-run the SAME command with
`--resume` appended.
`--resume` appended (with the SAME `--mode` — the script refuses a mode flip on a half-done install).
### C.5 BYO hosts (`--mode byo`) — installing on a Proxmox server the operator does NOT own
**Trust model.** On a BYO host the roles invert: the box owner is root and stays root; Felhom is the
guest. The installer therefore (a) never touches the owner's credentials — the break-glass step
(4b/8, root@pam reset + hub vault) is skipped entirely, nothing is vaulted; (b) treats the owner's
workloads as the thing to protect — the CPU/RAM caps are mandatory, they are the only
noisy-neighbor containment there; (c) refuses anything host-invasive that the appliance profile
allows (`--enable-oob`, `--rotate-recovery`, unvalidated PVE majors, ACL grants on storages the box
doesn't have); and (d) asserts the written agent config keeps `lan_resolver`/`wg_tunnel`/`oob` OFF
— the agent must not take over the owner's DNS or open tunnels. Recovery on a BYO box = the owner's
own console access, by design.
**Recommended first run — preflight only** (all step-1 checks incl. the byo gates, an explicit
PASS/FAIL verdict, exit 0/1, writes nothing — re-runnable any number of times):
```bash
./felhom-host-install.sh --customer-id <CUSTOMER-ID> --mode byo --vmid <VMID> \
--cores <N> --memory <MiB> \
--force-gitea-golden --acl-storages "local local-lvm" \
--preflight-only
```
**The install command** (same flags, minus `--preflight-only`):
```bash
./felhom-host-install.sh --customer-id <CUSTOMER-ID> --mode byo --vmid <VMID> \
--cores <N> --memory <MiB> \
--force-gitea-golden --acl-storages "local local-lvm"
```
What byo does differently (everything else matches C.4's eight steps):
- `--cores` AND `--memory` are **required** — size them WITH the owner (the guest must never starve
the owner's guests). `--acl-storages` must name storages that actually exist on the box, or the
preflight dies naming the missing ones.
- Preflight additionally prints what already listens on the host's `:53` (informational — Felhom
never configures DNS on a byo box) and the existing-guest count.
- At the end of preflight the script prints the **complete host-mutation list** (pveum
roles/user/token/ACL/pool, agent user/binary/unit/sudoers/config/state, the watchdog + guarded
wrappers, the guest + golden) and requires typing the host's **short hostname** to proceed —
show this list to the box owner; it is the consent artifact.
- Step 4b/8 (break-glass) is skipped; the verify step asserts pool membership + the scoped ACL
grants landed (both modes do this from v1.10.0).
### C.6 Post-hoc mode note for pre-v1.10.0 installs
Boxes installed by ≤ v1.9.1 have no recorded mode; their state file simply predates it. `--resume`
on such a box accepts whichever `--mode` you pass — pass the mode that matches how the box is
actually operated (existing Felhom-owned boxes: `appliance`).
---
@@ -306,6 +363,10 @@ roles/ACL/token/user, the pool (if empty), the install state file.
| Symptom | Cause | Fix |
|---|---|---|
| dies immediately: "--mode is required" | v1.10.0+ has no default profile | add `--mode appliance` (Felhom-owned box) or `--mode byo` (owner's box, §C.5) |
| dies immediately: "byo mode requires explicit --cores and --memory" | byo caps are mandatory | size the caps with the box owner and pass both |
| dies: "install started as X; resume with --mode X" | `--resume` with the other mode | resume with the recorded mode, or `--uninstall` and start over |
| byo dies: "acl storage(s) not found on this box" | `--acl-storages` names a storage the box lacks | pass the box's real storages (check `pvesm status`) |
| step 1 dies: "this is a N-node cluster" | multi-node cluster | re-run with `--node <name>` |
| step 5 dies: "hub artifact manifest has no agent version" | Day-0 manifest unset/incomplete | Part A.3 — set it in the operator UI |
| step 5 dies: "no git token in controller.yaml" | customer created without git credentials | Part A.2 — add `git.username`/`git.token`, regenerate config |
+34
View File
@@ -1,5 +1,39 @@
# Felhom scripts — Changelog
## felhom-host-install v1.10.0 — --mode appliance|byo install profile (TASK GL-2, go-live G2/G4/G5) (2026-07-07)
- **`--mode appliance|byo` is now REQUIRED** for a fresh install / `--resume` (no default — the
build-golden v2.0.0 precedent). `appliance` = the historical behavior, dry-transcript-preserved.
`byo` = a Proxmox host the operator does NOT own (the pilot): **`step_break_glass` is gated OFF at
its call site** (root@pam never reset, nothing vaulted — the one place the skip is auditable),
`--cores` + `--memory` both mandatory (the only noisy-neighbor protection there),
`--enable-oob`/`--rotate-recovery` refused at argv time, PVE-major gate (die on non-9.x unless
`--force`), every `--acl-storages` entry must exist on the box (die naming ALL missing; appliance
warns), informational host-DNS `:53` line, full host-mutation disclosure + typed-hostname
acknowledgement (`/dev/tty`, dry-run-aware), and post-write config asserts that refuse
`lan_resolver.enable`/`wg_tunnel.enabled`/`oob.enabled` = true (byo also flips the config write's
lan_resolver DEFAULT to off; appliance keeps default-on).
- The old `--mode provision` value is retired (specific die message); the `dr` stub seam is kept.
The mode is persisted in state.json; a `--resume` under the other mode refuses (C4).
- **Verify asserts (BOTH modes — campaign-2 R2 lesson):** the verify step now asserts pool
membership of the provisioned vmid + every scoped ACL grant (user AND token: Base@/,
Guest@/pool/felhom, Store@each storage); any miss → error + "WITH WARNINGS" verdict.
- **`--preflight-only`**: runs the full mode-aware step-1 checks, prints an explicit
`PRE-FLIGHT PASS/FAIL` verdict, exits 0/1, writes NO state marks (never lets a later `--resume`
skip the real preflight). Existing-guest count line added to preflight (both modes).
- `FELHOM_INSTALL_STATE_DIR` env override (test-harness only) so harness cases can never touch a
live `state.json`; `usage()` switched to a marker-based range (the numeric one had drifted).
- **NEW `scripts/hostinstall-mode-harness.sh`**: static tier (bash -n, shellcheck, C1C4 refusal
matrix asserting exact die messages, grep-invariants: one gated break-glass call site, chpasswd
unreachable outside it, --mode in usage) + PVE tier (C5 live-refusal + Scenario A/B dry
transcripts via `FELHOM_TEST_CUSTOMER`/`FELHOM_TEST_PASSFILE`). Red-proofs RP-1..RP-3 executed
(gate removed / caps requirement dropped / mode-mismatch check dropped → matching case FAILs).
- Live-validated read-only on felhom-pve: full static tier + C5 + byo dry transcript (no
4b/8/chpasswd/recovery-credential; disclosure+ack+DNS+storage lines present) + `--preflight-only`
PASS and FAIL verdicts with zero state entries. Full A/B rc=0 transcripts + C7 await the
supervised GL-6 drill (blocked today by demo-felhom's EMPTY git.username/git.token — any step-5
re-run for that customer dies until the operator sets real Gitea read creds).
## felhom-host-install — --enable-oob installs the dedicated felhom-sshd OOB instance + belt (TASK H1) (2026-07-05)
- `--enable-oob` + `install_oob` (in step 5): lays down the STATIC OOB parts — `/etc/felhom-sshd` tree
+249 -18
View File
@@ -1,6 +1,6 @@
#!/bin/bash
#===============================================================================
# felhom-host-install.sh v1.9.1
# felhom-host-install.sh v1.10.0
# Day-0 host-bootstrap for a Felhom Proxmox host (operator-deploy model).
#
# Run by the operator on a FRESHLY-PVE-INSTALLED box (after a manual PVE install
@@ -19,16 +19,39 @@
# new credential. The checksum trust root is the HUB, not Gitea. This removes the
# old prerequisite "install the agent binary + unit manually".
#
# v1.10.0 (GL-2, go-live G2/G4/G5): explicit --mode appliance|byo install profile — the flag is now
# REQUIRED for a fresh install (no default; build-golden v2.0.0 precedent: defaults rot). byo =
# BYO-host hardening for a Proxmox host the operator does NOT own: break-glass (step 4b) gated OFF
# at its call site (root@pam is never touched), mandatory --cores/--memory noisy-neighbor caps,
# strict preflight (PVE-major gate, --acl-storages existence, host-DNS :53 info line, host-mutation
# disclosure + typed-hostname acknowledgement), byo config asserts (lan_resolver/wg_tunnel/oob must
# stay off), pool+ACL verify asserts (BOTH modes — campaign-2 R2 lesson), and --preflight-only.
# Test harness: scripts/hostinstall-mode-harness.sh (static tier runs anywhere; PVE tier dry-only).
#
# Grounding: documentation/audits/SPIKE-day0-firstboot-handshake-2026-06-26.md
#
# Usage:
# sudo ./felhom-host-install.sh --customer-id ID [options]
# sudo ./felhom-host-install.sh --customer-id ID --mode appliance|byo [options]
#
# Required:
# --customer-id ID Customer (must already exist in the hub)
# --mode appliance|byo Install profile (REQUIRED for install/--resume — no default):
# appliance a Felhom-owned/managed box. Full Day-0 incl. the
# break-glass root@pam credential (step 4b).
# byo a host the operator does NOT own (BYO / pilot). NEVER
# touches root@pam (step 4b skipped), REQUIRES explicit
# --cores AND --memory, refuses --enable-oob and
# --rotate-recovery, dies on non-9.x PVE (unless --force)
# and on --acl-storages entries absent from this box,
# prints the full host-mutation list and requires typing
# the host's short hostname, and asserts the agent config
# keeps lan_resolver/wg_tunnel/oob OFF.
# (dr the 10D DR stub — documented seam, not implemented.)
#
# Options:
# --mode provision|dr provision (Day-0, default) | dr (10D stub — not impl.)
# --preflight-only run ONLY the (mode-aware) step-1 checks: prints PRE-FLIGHT PASS/FAIL,
# exits 0/1, writes NO state, executes NO later step. Re-runnable; valid
# with either mode (mode-specific checks need --mode).
# --hub-url URL default https://hub.felhom.eu
# --vmid N guest VMID to provision. Default 9201; if omitted and 9201 is already
# in use, the script auto-picks the next free id (pct+qm) and asks to
@@ -46,7 +69,8 @@
# --datavol-grow N grow Docker-data vol by N GiB (default: auto-compute)
# --sysdata-grow N grow user-data vol by N GiB (default: auto-compute)
#
# Appliance cap (optional — protect a SHARED host's other guests; needs agent >= v0.52.0):
# Guest cap (appliance: optional — protect a SHARED host's other guests; byo: BOTH REQUIRED —
# the only noisy-neighbor protection on a host you do not own; needs agent >= v0.52.0):
# --cores N cap the guest to N CPU cores (0/unset = golden default)
# --memory M cap the guest RAM to M MiB (0/unset = golden default)
#
@@ -105,7 +129,7 @@
set -euo pipefail
SCRIPT_VERSION="1.9.1" # keep in sync with the header line at the top of this file
SCRIPT_VERSION="1.10.0" # keep in sync with the header line at the top of this file
#-------------------------------------------------------------------------------
# Logging (mirrors felhom-controller/scripts/docker-setup.sh)
@@ -125,7 +149,8 @@ die() { log_error "$1"; exit 1; }
# Defaults
#-------------------------------------------------------------------------------
CUSTOMER_ID=""
MODE="provision"
MODE="" # --mode appliance|byo (GL-2: REQUIRED for install/--resume, no default; "dr" = the 10D stub)
PREFLIGHT_ONLY=false # --preflight-only: run the mode-aware step-1 checks only; no state writes, no later step
HUB_URL="https://hub.felhom.eu"
VMID="9201"
VMID_EXPLICIT=false # set true when --vmid is given; gates the auto-pick-a-free-vmid behavior
@@ -194,7 +219,9 @@ PVE_PRIVS_BASE="Sys.Audit SDN.Use Datastore.Audit"
# offsite felhom-pbs MUST be included or the agent's DR backup 403s (SPIKE residual #1). --acl-storages overrides.
PVE_STORAGES=(local local-lvm felhom-pbs)
STATE_DIR="/var/lib/felhom-install"
# FELHOM_INSTALL_STATE_DIR: test-harness-only override (hostinstall-mode-harness.sh) so dry/refusal
# cases can never touch a live install's state.json. Production runs never set it.
STATE_DIR="${FELHOM_INSTALL_STATE_DIR:-/var/lib/felhom-install}"
STATE_FILE="${STATE_DIR}/state.json"
AGENT_CONFIG="" # resolved in preflight
HARD_MIN_LVM_GIB=120 # a useful appliance won't fit below this on local-lvm
@@ -214,7 +241,9 @@ ART_GOLDEN_SHA=""
#-------------------------------------------------------------------------------
# Helpers
#-------------------------------------------------------------------------------
usage() { sed -n '2,95p' "$0" | sed 's/^# \{0,1\}//'; exit 0; }
# Print the header through the last option line (the "# State (" line is the end marker — keeps the
# range from drifting as the header grows; v1.9.1's numeric '2,95p' had already drifted).
usage() { sed -n '2,/^# State (/p' "$0" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0; }
run() { # simple (no pipes/redirects) mutating command
if $DRY_RUN; then log_dry "$*"; else "$@"; fi
@@ -825,6 +854,7 @@ while [[ $# -gt 0 ]]; do
--rotate-recovery) ROTATE_RECOVERY=true; shift ;;
--enable-oob) ENABLE_OOB=true; shift ;;
--acl-storages) read -ra PVE_STORAGES <<< "$2"; shift 2 ;;
--preflight-only) PREFLIGHT_ONLY=true; shift ;;
--dry-run) DRY_RUN=true; shift ;;
--resume) RESUME=true; shift ;;
-h|--help) usage ;;
@@ -893,7 +923,45 @@ if [[ "$MODE" == "dr" ]]; then
EOF
exit 2
fi
[[ "$MODE" == "provision" ]] || die "Unknown --mode: $MODE (provision|dr)"
#===============================================================================
# GL-2 install-profile validation — everything below runs a real install (or its --preflight-only /
# --resume variant), so the profile is decided HERE, before the passphrase prompt and any step. The
# refusals are deliberately argv-time (the harness relies on them firing on a non-PVE machine too).
#===============================================================================
case "$MODE" in
appliance|byo) ;;
"")
if $PREFLIGHT_ONLY; then
log_warn "--preflight-only without --mode: running the mode-agnostic base checks only"
else
die "--mode is required: pass --mode appliance (a Felhom-owned box) or --mode byo (a host you do not own). See -h."
fi
;;
provision) die "--mode provision was retired in v1.10.0 — pass --mode appliance (Felhom-owned box) or --mode byo (a host you do not own)." ;;
*) die "Unknown --mode: $MODE (appliance|byo)" ;;
esac
# BYO argument refusals (C1/C2) — before the passphrase prompt, before any step.
if [[ "$MODE" == "byo" ]]; then
if [[ -z "$CPU_CORES" || -z "$MEM_MIB" ]]; then
die "byo mode requires explicit --cores and --memory (noisy-neighbor caps on a host you do not own)"
fi
if $ENABLE_OOB; then
die "--enable-oob is not allowed in byo mode (OOB access on a host the operator does not own is the owner's call; provision it as a separate, owner-consented step)"
fi
if $ROTATE_RECOVERY; then
die "--rotate-recovery is not allowed in byo mode (byo never touches root@pam — there is no recovery credential to rotate)"
fi
fi
# Resume mode-mismatch (C4): an install keeps the mode it started with — mode-flipping a half-done
# install would skip already-completed steps under the WRONG profile's guarantees.
if $RESUME; then
_recorded_mode=$(_state_get mode || true)
if [[ -n "$_recorded_mode" && "$_recorded_mode" != "$MODE" ]]; then
die "install started as $_recorded_mode; resume with --mode $_recorded_mode or start over"
fi
fi
#===============================================================================
# PROVISION MODE
@@ -901,7 +969,7 @@ fi
[[ -n "$CUSTOMER_ID" ]] || die "--customer-id is required (use -h)"
echo ""
log_info "felhom-host-install v${SCRIPT_VERSION} — mode=provision customer=${CUSTOMER_ID} vmid=${VMID}"
log_info "felhom-host-install v${SCRIPT_VERSION} — mode=${MODE:-<preflight-only, no mode>} customer=${CUSTOMER_ID} vmid=${VMID}"
$DRY_RUN && log_warn "DRY-RUN: no mutations will be performed"
echo ""
@@ -922,6 +990,47 @@ read_passphrase() {
[[ -n "$PASSPHRASE" ]] || die "empty passphrase"
}
#-------------------------------------------------------------------------------
# BYO host-mutation disclosure + typed acknowledgement (GL-2). Printed at the END of preflight so
# every value shown (vmid, storages, config path) is final. The list follows the CODE (steps 2-8 +
# the agent's runtime installs), not memory — keep it in sync when a step grows a new host artifact.
# The ack reads /dev/tty (the script is often `curl | bash`) and mirrors the uninstall confirm's
# dry-run branch. NO credential is ever printed here — artifact PATHS only.
#-------------------------------------------------------------------------------
_byo_disclosure_ack() {
echo ""
log_step "BYO disclosure — installing Felhom on a host the operator does NOT own will create:"
cat <<EOF
pveum: roles ${PVE_ROLE_BASE}/${PVE_ROLE_GUEST}/${PVE_ROLE_STORE} + user ${PVE_USER} + token ${PVE_USER}!${PVE_TOKENID}
+ scoped ACL grants (Base@/, Guest@/pool/${PVE_POOL}, Store@[${PVE_STORAGES[*]}]) + the '${PVE_POOL}' pool
agent: service user ${AGENT_USER} + ${AGENT_BIN} (+ .bak-* backups; A/B slots via the guarded self-update)
+ ${AGENT_UNIT} (+ felhom-agent-rollback.service + start-limit drop-in)
+ ${AGENT_SUDOERS} + state dir ${AGENT_STATE_DIR} + config ${AGENT_CONFIG} (0600)
host: /usr/local/sbin/felhom-mkfs-guarded + /usr/local/sbin/felhom-selfupdate-guarded
+ felhom-mgmt-watchdog service+timer+script + /etc/tmpfiles.d/felhom-privsep.conf
+ guest-hook snippet under /var/lib/vz/snippets/ (agent-installed at runtime)
+ the 'sudo' package if absent + install state dir ${STATE_DIR}
EOF
if $SKIP_PROVISION; then
echo " guest: none (--skip-provision)"
else
echo " guest: the provisioned Felhom LXC (vmid ${VMID}, capped ${CPU_CORES} cores / ${MEM_MIB} MiB) + its volumes"
echo " + the golden vzdump imported onto storage '${ARCHIVE_STORAGE}'"
fi
echo " NOT touched in byo mode: root@pam (no break-glass), host DNS (:53), WireGuard tunnels, OOB sshd."
echo ""
if $DRY_RUN; then
log_dry "would prompt: Type this host's short hostname ($(hostname -s)) to acknowledge the byo install"
elif $PREFLIGHT_ONLY; then
log_info " (--preflight-only: acknowledgement prompt skipped — nothing will be mutated)"
else
local _hn _ans; _hn=$(hostname -s)
read -rp "Type this host's short hostname (${_hn}) to acknowledge the byo install: " _ans < /dev/tty
[[ "$_ans" == "$_hn" ]] || die "acknowledgement mismatch (got '${_ans}', expected '${_hn}') — aborting, nothing installed"
log_success " byo install acknowledged for host ${_hn}"
fi
}
#-------------------------------------------------------------------------------
# STEP 1 — pre-flight (fail fast before any mutation)
#-------------------------------------------------------------------------------
@@ -935,6 +1044,11 @@ step_preflight() {
command -v python3>/dev/null || die "python3 not found"
local pvever; pvever=$(pveversion | head -1)
# GL-2 PVE-major gate: byo installs are validated on PVE 9.x ONLY — on a host we don't own, an
# unvalidated major is a refusal, not a warning. Appliance keeps the historical warn-only.
if [[ "$pvever" != *"/9."* && "$MODE" == "byo" ]] && ! $FORCE; then
die "byo mode is validated on PVE 9.x only (got: $pvever). Pass --force to override deliberately."
fi
[[ "$pvever" == *"/9."* ]] || log_warn "expected PVE 9.x, got: $pvever"
log_info " $pvever"
@@ -1004,6 +1118,11 @@ step_preflight() {
fi
fi
# Existing-guest context (GL-2, both modes): cheap situational awareness — on a byo/shared box
# this is the population the caps + scoped ACL protect.
local _guest_count; _guest_count=$( { used_vmids || true; } | wc -l)
log_info " existing guests on this host: ${_guest_count} (pct+qm)"
# archive-storage-exists guard (provision only — the golden lives there + the restore reads it).
if ! $SKIP_PROVISION; then
if pvesm status --storage "$ARCHIVE_STORAGE" >/dev/null 2>&1; then
@@ -1013,6 +1132,40 @@ step_preflight() {
fi
fi
# --acl-storages existence (GL-2): the scoped ACL grants Datastore.* at each of these paths — a
# grant on a storage this box doesn't have silently protects nothing (campaign-2 R2 class). byo
# refuses; appliance warns (the default set includes felhom-pbs, absent on most customer boxes).
local _missing_storages=() _acls
for _acls in "${PVE_STORAGES[@]}"; do
pvesm status --storage "$_acls" >/dev/null 2>&1 || _missing_storages+=("$_acls")
done
if [[ ${#_missing_storages[@]} -gt 0 ]]; then
if [[ "$MODE" == "byo" ]]; then
die "acl storage(s) not found on this box: ${_missing_storages[*]} — pass --acl-storages with this box's real storages"
fi
for _acls in "${_missing_storages[@]}"; do
log_warn " acl storage '$_acls' not found (pvesm status) — its grant will point at nothing"
done
elif [[ "$MODE" == "byo" ]]; then
log_info " acl storages all present: ${PVE_STORAGES[*]}"
fi
# Host DNS info line (byo only, informational ONLY): the installer never configures DNS and the
# byo config asserts keep the agent's lan-resolver off — this line just tells the operator what
# already listens on :53 (the owner may run a resolver Felhom must not disturb).
if [[ "$MODE" == "byo" ]]; then
local _dns53
if _dns53=$(ss -H -ltnup 'sport = :53' 2>/dev/null); then
if [[ -n "$_dns53" ]]; then
log_info " host DNS (:53): $(echo "$_dns53" | tr -s '[:space:]' ' ' | cut -c1-160)"
else
log_info " host DNS (:53): nothing bound"
fi
else
log_warn " host DNS (:53): could not query (ss failed/absent) — informational check skipped"
fi
fi
# hub reachable
local hc; hc=$(http_code "$HUB_URL/api/v1/config/$CUSTOMER_ID" -H "X-Retrieval-Password: preflight-no-op" || echo 000)
[[ "$hc" != "000" ]] || die "hub unreachable at $HUB_URL"
@@ -1065,9 +1218,19 @@ step_preflight() {
log_success " using auto-selected vmid $VMID"
fi
fi
# Record the customer into the install state (foundation for a later automatic --uninstall).
_state_put customer_id "$CUSTOMER_ID"
_state_mark preflight
# BYO disclosure + typed ack — last, so every disclosed value (vmid incl. auto-pick) is final.
if [[ "$MODE" == "byo" ]]; then
_byo_disclosure_ack
fi
# Record the customer + install profile into the state (foundation for a later automatic
# --uninstall; the mode key backs the C4 resume-mismatch refusal). --preflight-only writes NO
# state — it must stay re-runnable and must never let a later --resume skip the real preflight.
if ! $PREFLIGHT_ONLY; then
_state_put customer_id "$CUSTOMER_ID"
_state_put mode "$MODE"
_state_mark preflight
fi
log_success "pre-flight passed"
}
@@ -1553,6 +1716,11 @@ step_agent_config() {
if $DRY_RUN; then
log_dry "write $AGENT_CONFIG (0600): proxmox{endpoint,node=$NODE,token=<secret>,tls.fingerprint=$fp} hub{url=$HUB_URL,host_id=$HOST_ID,api_key=<secret>} local_api{$BRIDGE_ADDR}"
if [[ "$MODE" == "byo" ]]; then
log_dry "assert (byo) written config: lan_resolver.enable is false/absent"
log_dry "assert (byo) written config: wg_tunnel.enabled is false/absent"
log_dry "assert (byo) written config: oob.enabled is false/absent"
fi
log_dry "systemctl restart felhom-agent && felhom-agent --config $AGENT_CONFIG --selftest"
_state_mark agent_config; return 0
fi
@@ -1562,7 +1730,7 @@ step_agent_config() {
# Secrets passed via env (NOT argv) to avoid ps exposure.
PVE_TOKEN="$PVE_TOKEN" HOST_API_KEY="$HOST_API_KEY" \
NODE="$NODE" FP="$fp" HUB_URL="$HUB_URL" HOST_ID="$HOST_ID" BRIDGE_ADDR="$BRIDGE_ADDR" \
PRESERVE_FROM="$PRESERVE_FROM" OUT="$AGENT_CONFIG" python3 <<'PY'
PRESERVE_FROM="$PRESERVE_FROM" INSTALL_MODE="$MODE" OUT="$AGENT_CONFIG" python3 <<'PY'
import json, os
out = os.environ['OUT']
base = {}
@@ -1589,7 +1757,13 @@ base['local_api']['listen_addr'] = os.environ['BRIDGE_ADDR']
base['local_api'].setdefault('cert_file','/var/lib/felhom-agent/local-api.crt')
base['local_api'].setdefault('key_file','/var/lib/felhom-agent/local-api.key')
base['local_api'].setdefault('token_store','/var/lib/felhom-agent/local-tokens.log')
base.setdefault('lan_resolver', {"enable": True})
# GL-2: byo defaults the lan-resolver OFF (the installer must never take over the owner's host DNS;
# the post-write byo asserts refuse a true value however it got in). Appliance keeps the historical
# default-on. setdefault either way — a --preserve-from section wins and the asserts judge it.
if os.environ.get('INSTALL_MODE') == 'byo':
base.setdefault('lan_resolver', {"enable": False})
else:
base.setdefault('lan_resolver', {"enable": True})
# Day-0 overrides (always authoritative)
base['proxmox'] = {
"endpoint":"https://127.0.0.1:8006",
@@ -1614,6 +1788,25 @@ PY
chmod 600 "$AGENT_CONFIG"
log_success " wrote $AGENT_CONFIG (0600 ${AGENT_USER})"
# GL-2 byo config asserts: these three features take over host DNS / open tunnels — forbidden on
# a customer/BYO box until their production endpoints exist. Parse the JUST-WRITTEN file (same
# interpreter pattern as the write above) and refuse to start the daemon on any true value —
# a --preserve-from carrying lan_resolver.enable=true is exactly what this catches.
if [[ "$MODE" == "byo" ]]; then
AGENT_CONFIG="$AGENT_CONFIG" python3 <<'PY' || die "byo config assert FAILED — see the keys above; a byo box must keep lan_resolver/wg_tunnel/oob off (fix the preserved config and re-run with --resume)"
import json, os, sys
d = json.load(open(os.environ['AGENT_CONFIG']))
bad = []
if d.get('lan_resolver', {}).get('enable'): bad.append('lan_resolver.enable (takes over host DNS on :53)')
if d.get('wg_tunnel', {}).get('enabled'): bad.append('wg_tunnel.enabled (opens an outbound WG tunnel)')
if d.get('oob', {}).get('enabled'): bad.append('oob.enabled (starts a second operator sshd)')
if bad:
print('byo-forbidden config keys are TRUE: ' + '; '.join(bad), file=sys.stderr)
sys.exit(1)
PY
log_success " byo config asserts passed (lan_resolver.enable / wg_tunnel.enabled / oob.enabled all off)"
fi
# health: read-only selftest (proxmox) must pass before provisioning
if ! felhom-agent --config "$AGENT_CONFIG" --selftest >/dev/null 2>&1; then
felhom-agent --config "$AGENT_CONFIG" --selftest 2>&1 | tail -20 >&2
@@ -1731,12 +1924,31 @@ step_provision() {
#-------------------------------------------------------------------------------
step_verify() {
log_step "verify"
if $DRY_RUN; then log_dry "pct status/config $VMID; docker ps in-guest; host-report includes $VMID"; return 0; fi
if $DRY_RUN; then log_dry "pct status/config $VMID; docker ps in-guest; host-report includes $VMID; assert guest $VMID in pool $PVE_POOL + scoped ACL grants present (user+token: Base@/, Guest@/pool/$PVE_POOL, Store@[${PVE_STORAGES[*]}])"; return 0; fi
local ok=true
local st; st=$(pct status "$VMID" 2>/dev/null | awk '{print $2}')
[[ "$st" == "running" ]] && log_success " pct status: running" || { log_error " pct status: $st"; ok=false; }
if pct config "$VMID" 2>/dev/null | grep -q '^onboot: 1'; then log_success " onboot: 1"; else log_error " onboot NOT 1"; ok=false; fi
pct config "$VMID" 2>/dev/null | grep -E '^(rootfs|mp0|mp1|mp8):' | sed 's/^/ /'
# GL-2 pool + ACL asserts (BOTH modes — campaign-2 R2 proved membership can silently drop, and a
# guest outside the pool is a guest the scoped token cannot manage).
local members; members=" $(pool_members || true) "
if [[ "$members" == *" $VMID "* ]]; then
log_success " pool: guest $VMID is a member of $PVE_POOL"
else
log_error " pool: guest $VMID NOT in pool $PVE_POOL (the scoped token cannot manage it — adopt with --adopt-pool)"; ok=false
fi
local _pairs=("/ $PVE_ROLE_BASE" "/pool/$PVE_POOL $PVE_ROLE_GUEST") _pair _apath _arole _acls2
for _acls2 in "${PVE_STORAGES[@]}"; do _pairs+=("/storage/$_acls2 $PVE_ROLE_STORE"); done
for _pair in "${_pairs[@]}"; do
_apath="${_pair% *}"; _arole="${_pair#* }"
if _acl_grant_present "$_apath" user "$PVE_USER" "$_arole" \
&& _acl_grant_present "$_apath" token "${PVE_USER}!${PVE_TOKENID}" "$_arole"; then
log_success " acl: $_arole@$_apath present (user+token)"
else
log_error " acl: $_arole@$_apath MISSING for the user and/or the token (re-apply with --rescope-acl)"; ok=false
fi
done
# controller container healthy in-guest — bounded wait (the post-provision reboot + docker start
# take a while, especially on modest hardware; drill R6 re-verify)
local cstat="" _waited=0
@@ -1789,7 +2001,19 @@ step_verify_agent() {
#-------------------------------------------------------------------------------
# Main
#-------------------------------------------------------------------------------
trap 'PASSPHRASE=""; PVE_TOKEN=""; HOST_API_KEY=""; GIT_TOKEN=""' EXIT
# Scrub secret carriers on any exit; under --preflight-only also print the explicit FAIL verdict
# (the PASS verdict is printed by the preflight-only branch below).
# shellcheck disable=SC2154 # _rc IS assigned first inside the trap string itself
trap '_rc=$?; PASSPHRASE=""; PVE_TOKEN=""; HOST_API_KEY=""; GIT_TOKEN=""; if [[ $_rc -ne 0 ]] && $PREFLIGHT_ONLY; then log_error "PRE-FLIGHT FAIL (exit $_rc) — fix the finding above and re-run"; fi' EXIT
# --preflight-only: the full (mode-aware) step-1 checks, an explicit verdict, exit — no state marks,
# no later step (GL-2).
if $PREFLIGHT_ONLY; then
step_preflight
echo ""
log_success "PRE-FLIGHT PASS (mode=${MODE:-unset}) — no state written, no install step executed"
exit 0
fi
if $RESUME && _state_has preflight; then
# still need the passphrase for enroll/provision even on resume
@@ -1810,7 +2034,14 @@ fi
should_skip token || step_token
should_skip grows || step_grows
should_skip enroll || step_enroll
should_skip break_glass || step_break_glass
# GL-2: break-glass is gated at the CALL SITE (not inside the step) so the byo skip is auditable in
# one place. byo = a host the operator does not own: root@pam is the OWNER's credential — never
# reset, never vaulted.
if [[ "$MODE" == "appliance" ]]; then
should_skip break_glass || step_break_glass
else
log_skip "break-glass (step 4b) SKIPPED (byo: the host owner manages break-glass; root@pam untouched, nothing vaulted)"
fi
should_skip agent_install || step_agent_install
should_skip agent_config || step_agent_config
should_skip golden || step_golden
+236
View File
@@ -0,0 +1,236 @@
#!/bin/bash
#===============================================================================
# hostinstall-mode-harness.sh — GL-2 test harness for felhom-host-install.sh's
# --mode appliance|byo install profile (spec: TASK GL-2; pattern precedent:
# felhom-agent/scripts/mkfs-guarded-harness.sh).
#
# Two tiers, PASS/FAIL per case, nonzero exit on any FAIL:
#
# STATIC tier (runs anywhere with bash + python3 — incl. a Windows Git Bash):
# bash -n; shellcheck (if present); the argument-refusal matrix C1-C4
# (exit nonzero + the exact die message, asserted BEFORE any host access);
# the C6 gate's presence; grep-invariants (exactly one GATED
# step_break_glass call site, chpasswd unreachable outside its body,
# --mode in usage).
#
# PVE tier (only on a PVE host as root; skips itself cleanly elsewhere):
# C5 (byo dies naming a bogus --acl-storages entry — read-only, dies in
# preflight before any hub contact) and the Scenario A/B dry-run transcript
# checks. A/B need a real customer + retrieval passphrase — provide them
# via FELHOM_TEST_CUSTOMER + FELHOM_TEST_PASSFILE (a 0600 passphrase file,
# the script's EXISTING non-interactive path); unset -> those cases SKIP.
#
# STATE SAFETY (spec §9 rule 9): every invocation of the script under test runs
# with FELHOM_INSTALL_STATE_DIR pointed at a throwaway temp dir, so NO case can
# ever touch a live install's /var/lib/felhom-install/state.json. (Dry-run
# state helpers are additionally no-ops — this is the belt on top.)
#
# Red-proofs (spec Part 4): run the harness against a MUTATED COPY of the
# script (gate removed / requirement dropped) and watch the matching case FAIL:
# RP-1 un-gate the step_break_glass call site -> "break-glass call site gated" FAILs
# RP-2 drop the byo --cores/--memory requirement -> C1 FAILs
# RP-3 drop the resume mode-mismatch check -> C4 FAILs
#
# Usage: hostinstall-mode-harness.sh [path-to-felhom-host-install.sh]
#===============================================================================
set -euo pipefail
SCRIPT="${1:-$(dirname "$0")/felhom-host-install.sh}"
[[ -r "$SCRIPT" ]] || { echo "no script under test at $SCRIPT" >&2; exit 2; }
command -v python3 >/dev/null || { echo "python3 required (the script's state helpers use it)" >&2; exit 2; }
pass=0; fail=0; skip=0
verdict() { # PASS|FAIL|SKIP <name> [detail]
local v="$1" name="$2" detail="${3:-}"
case "$v" in
PASS) pass=$((pass+1)) ;;
FAIL) fail=$((fail+1)) ;;
SKIP) skip=$((skip+1)) ;;
esac
printf '%-4s %s\n' "$v" "$name"
[[ -n "$detail" ]] && printf ' %s\n' "$detail"
return 0
}
# Throwaway state dir for EVERY run of the script under test (never the live one). On Git Bash the
# script's python3 may be a native Windows build — hand it a Windows-syntax path via cygpath -m.
WORK="$(mktemp -d "${TMPDIR:-/tmp}/hostinstall-harness.XXXXXX")"
trap 'rm -rf "$WORK"' EXIT
STATE_OVERRIDE="$WORK/state"
mkdir -p "$STATE_OVERRIDE"
if command -v cygpath >/dev/null 2>&1; then
STATE_OVERRIDE_ENV="$(cygpath -m "$STATE_OVERRIDE")"
else
STATE_OVERRIDE_ENV="$STATE_OVERRIDE"
fi
# run_script <args...> — run the script under test with the state override; captures stdout+stderr
# into $out and the exit code into $rc. Never lets a nonzero rc kill the harness.
out=""; rc=0
run_script() {
set +e
out=$(FELHOM_INSTALL_STATE_DIR="$STATE_OVERRIDE_ENV" bash "$SCRIPT" "$@" 2>&1)
rc=$?
set -e
}
# expect_die <name> <msg-substr> -- <args...> — the script must exit nonzero AND print msg-substr.
expect_die() {
local name="$1" msg="$2"; shift 2
[[ "${1:-}" == "--" ]] && shift
run_script "$@"
if [[ $rc -ne 0 && "$out" == *"$msg"* ]]; then
verdict PASS "$name"
else
verdict FAIL "$name" "rc=$rc; wanted substring: '$msg'; got: $(echo "$out" | tail -3 | tr '\n' ' ')"
fi
}
echo "=== hostinstall-mode-harness — script under test: $SCRIPT ==="
echo ""
echo "--- STATIC tier ---"
# S1: syntax
if bash -n "$SCRIPT" 2>"$WORK/bashn.err"; then
verdict PASS "bash -n"
else
verdict FAIL "bash -n" "$(cat "$WORK/bashn.err")"
fi
# S2: shellcheck (best-effort — required by the green gate, but the harness itself degrades)
if command -v shellcheck >/dev/null 2>&1; then
if shellcheck --shell=bash --severity=warning "$SCRIPT" >"$WORK/sc.out" 2>&1; then
verdict PASS "shellcheck (severity>=warning)"
else
verdict FAIL "shellcheck (severity>=warning)" "$(head -5 "$WORK/sc.out")"
fi
else
verdict SKIP "shellcheck" "not installed here — run it on the build server (green gate still requires it)"
fi
# C1: byo without --cores/--memory refuses (both missing, and each alone)
expect_die "C1 byo without caps refused" \
"byo mode requires explicit --cores and --memory" \
-- --customer-id t --mode byo
expect_die "C1b byo with only --cores refused" \
"byo mode requires explicit --cores and --memory" \
-- --customer-id t --mode byo --cores 4
expect_die "C1c byo with only --memory refused" \
"byo mode requires explicit --cores and --memory" \
-- --customer-id t --mode byo --memory 8192
# C2: byo refuses --enable-oob / --rotate-recovery, naming the flag
expect_die "C2a byo --enable-oob refused" \
"--enable-oob is not allowed in byo mode" \
-- --customer-id t --mode byo --cores 4 --memory 8192 --enable-oob
expect_die "C2b byo --rotate-recovery refused" \
"--rotate-recovery is not allowed in byo mode" \
-- --customer-id t --mode byo --cores 4 --memory 8192 --rotate-recovery
# C3: fresh install without --mode refuses, naming both modes
expect_die "C3 fresh install without --mode refused" \
"--mode is required: pass --mode appliance" \
-- --customer-id t
expect_die "C3b unknown --mode refused" \
"Unknown --mode: bogus (appliance|byo)" \
-- --customer-id t --mode bogus
expect_die "C3c retired --mode provision refused" \
"--mode provision was retired" \
-- --customer-id t --mode provision
# C4: --resume with a state.json recording the OTHER mode refuses
printf '{"completed":["preflight"],"customer_id":"t","mode":"appliance"}\n' > "$STATE_OVERRIDE/state.json"
expect_die "C4 resume mode-mismatch refused" \
"install started as appliance; resume with --mode appliance or start over" \
-- --customer-id t --mode byo --cores 4 --memory 8192 --resume
rm -f "$STATE_OVERRIDE/state.json"
# C6 (static shape): the byo PVE-major gate + its message exist (a non-9.x host isn't available)
if grep -q 'byo mode is validated on PVE 9.x only' "$SCRIPT" \
&& grep -B3 'byo mode is validated on PVE 9.x only' "$SCRIPT" | grep -q 'MODE" == "byo"'; then
verdict PASS "C6 byo PVE-major gate present (static grep)"
else
verdict FAIL "C6 byo PVE-major gate present (static grep)"
fi
# INV-1: exactly ONE step_break_glass invocation, and it is gated on MODE == appliance.
# (Comment lines and the function definition itself don't count.)
inv_calls=$(grep -n 'step_break_glass' "$SCRIPT" \
| grep -v -E '^[0-9]+:[[:space:]]*#' \
| grep -v 'step_break_glass()' || true)
inv_count=$(echo "$inv_calls" | grep -c 'step_break_glass' || true)
if [[ "$inv_count" == "1" ]] \
&& echo "$inv_calls" | grep -q 'should_skip break_glass' \
&& grep -B4 'should_skip break_glass' "$SCRIPT" | grep -q 'MODE" == "appliance"'; then
verdict PASS "INV-1 break-glass call site: exactly one, appliance-gated"
else
verdict FAIL "INV-1 break-glass call site: exactly one, appliance-gated" "count=$inv_count; calls: $(echo "$inv_calls" | tr '\n' ' ')"
fi
# INV-2: chpasswd is unreachable outside step_break_glass's body (non-comment occurrences only).
body=$(awk '/^step_break_glass\(\)/{s=NR} s && /^\}/{print s, NR; exit}' "$SCRIPT")
bstart=${body% *}; bend=${body#* }
inv2_ok=true
while IFS=: read -r ln _; do
[[ -z "$ln" ]] && continue
if [[ "$ln" -lt "$bstart" || "$ln" -gt "$bend" ]]; then inv2_ok=false; fi
done < <(grep -n 'chpasswd' "$SCRIPT" | grep -v -E '^[0-9]+:[[:space:]]*#' || true)
if [[ -n "$bstart" && -n "$bend" ]] && $inv2_ok; then
verdict PASS "INV-2 chpasswd unreachable outside step_break_glass ($bstart-$bend)"
else
verdict FAIL "INV-2 chpasswd unreachable outside step_break_glass" "body=$bstart-$bend"
fi
# INV-3: --mode is documented in usage (run the real -h path)
run_script -h
if [[ $rc -eq 0 && "$out" == *"--mode appliance|byo"* ]]; then
verdict PASS "INV-3 usage documents --mode appliance|byo"
else
verdict FAIL "INV-3 usage documents --mode appliance|byo" "rc=$rc"
fi
echo ""
echo "--- PVE tier ---"
if ! command -v pveum >/dev/null 2>&1 || [[ "$(id -u)" != 0 ]]; then
verdict SKIP "PVE tier (C5 + Scenario A/B dry transcripts)" "needs a PVE host as root — run there"
else
# C5: byo with a bogus --acl-storages entry dies in preflight NAMING it (read-only: dies before
# any hub contact/passphrase and before any mutation; state override active).
expect_die "C5 byo bogus --acl-storages refused, named" \
"acl storage(s) not found on this box: definitely-not-a-storage" \
-- --customer-id t --mode byo --cores 4 --memory 8192 \
--acl-storages "local definitely-not-a-storage" --dry-run
if [[ -n "${FELHOM_TEST_CUSTOMER:-}" && -n "${FELHOM_TEST_PASSFILE:-}" && -r "${FELHOM_TEST_PASSFILE:-}" ]]; then
common=(--customer-id "$FELHOM_TEST_CUSTOMER" --passphrase-file "$FELHOM_TEST_PASSFILE" \
--vmid 990100 --cores 2 --memory 4096 --dry-run)
# H-A: appliance dry transcript still walks 4b/8 (regression guard)
run_script --mode appliance "${common[@]}"
if [[ $rc -eq 0 && "$out" == *"4b/8"* ]]; then
verdict PASS "H-A appliance dry transcript contains 4b/8"
else
verdict FAIL "H-A appliance dry transcript contains 4b/8" "rc=$rc"
fi
# H-B: byo dry transcript — no 4b/8 / chpasswd / recovery-credential; ack + caps + storages present
run_script --mode byo "${common[@]}"
hb_ok=true; hb_why=""
[[ $rc -eq 0 ]] || { hb_ok=false; hb_why+="rc=$rc "; }
for bad in "4b/8" "chpasswd" "recovery-credential"; do
[[ "$out" != *"$bad"* ]] || { hb_ok=false; hb_why+="contains '$bad' "; }
done
for want in "acknowledge the byo install" "acl storages all present" "-cores 2 -memory 4096"; do
[[ "$out" == *"$want"* ]] || { hb_ok=false; hb_why+="missing '$want' "; }
done
if $hb_ok; then
verdict PASS "H-B byo dry transcript (no root@pam path; ack+caps+storages present)"
else
verdict FAIL "H-B byo dry transcript" "$hb_why"
fi
else
verdict SKIP "H-A/H-B dry transcripts" "set FELHOM_TEST_CUSTOMER + FELHOM_TEST_PASSFILE (0600 passphrase file) to run"
fi
fi
echo ""
echo "=== $pass passed, $fail failed, $skip skipped ==="
[[ $fail -eq 0 ]]