felhom.eu/REPORT.md

felhom.eu — task reports

Overwrite this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in hub/CHANGELOG.md; the scripts history lives in scripts/CHANGELOG.md.

R-21 slice A: bare-metal Felhom ISO (build pipeline + first-boot bootstrap) — 2026-07-16

Repo: felhom.eu · Baseline: main @ 914d51a → scripts v1.16.0 (ISO pipeline tag; felhom-host-install was already v1.16.0 from 2026-07-13 — see Versioning note). Validation: nested VM 310 on felhom-pve; ISO builds on DooPlex. felhom-host-install.sh not modified (read-only, invoked only). Hub code untouched.

Files created (scripts/iso/)

Dockerfile.assistant, build-felhom-iso.sh, answer.toml.tmpl, stub-first-boot.sh, felhom-bootstrap.sh, felhom-bootstrap.service, profiles/nested-vm.profile, profiles/nested-canary.profile, README.md. Plus scripts/CHANGELOG.md, documentation/backlog/ROADMAP.md (R-21 → in-progress), this REPORT.md, CONTEXT.md.

Green gates

bash -n clean on all three scripts; shellcheck -x exit 0, no findings (run in the assistant container). Retry-unit behavior de-risked on systemd 257 before wiring (transient unit: Type=oneshot

• Restart=on-failure does retry — NRestarts climbed).

§4.4 finding — does the distribution channel embed a secret? (decides slice C)

The channel: no. The unattended install: yes (the passphrase). The hub install-command generator (hub/internal/web/templates/customer_unified.html, Option 1) emits curl -fsSL https://felhom.eu/scripts/felhom-host-install.sh -o … && sudo bash … --customer-id <id> --mode <appliance|byo> — a public URL, no token/secret in the command. But felhom-host-install.sh requires the customer retrieval passphrase (read_passphrase → die "empty passphrase"; used for config fetch + POST /host-enroll). Unattended ⇒ it must be supplied via --passphrase-file, so the slice-A ISO is secret-bearing (embeds the passphrase). Slice-C implication: the distributed ISO must NOT bake the passphrase — the claim/pairing model must deliver it (or the config) to the box after the customer claims it. Slice-A ISOs are supervised/single-use; the bootstrap shreds bootstrap.env on success.

2.3 retry-vs-resume ruling (source-verified, encoded once)

Plain first attempt; add --resume on any later attempt that finds /var/lib/felhom-install/state.json. Source landmark: felhom-host-install.sh header v1.11.3 ("--resume correctness — the producer steps (token/enroll/grows) run every pass … a resumed install no longer writes a config missing hub.host_id/proxmox.token") — so --resume is safe, whereas a plain re-invoke over an existing install state would re-hit the populated-host leaf guard / existing-vmid refusal. --mode is required in both forms. Encoded in felhom-bootstrap.sh with the landmark cited inline; empirically confirmed (below).

Per-scenario results (evidence under 180:~/felhom-iso/evidence/)

• C — build gate blocks a broken answer (+ red-proof) — PASS. Answer with both root-password and root-password-hashed: the output-parse gate surfaced `global.root-password` and `global.root-password-hashed` cannot be set at the same time / Error: Found issues and aborted (BUILD-EXIT=1, no ISO — count unchanged), even though validate-answer exits 0. Red-proof: gutting the gate to trust $? made it wrongly report [OK] answer validated; the broken answer was then stopped only by prepare-iso's own error plus the build's [[ -f out.iso ]] guard — notably prepare-iso also exited 0 on the error, so the file-existence check was the real backstop. Reverted (mutation done on the DooPlex copy; committed tree never touched).
• D — disk-filter fail-safe through the pipeline — PASS. Pipeline-built canary ISO (filter.ID_SERIAL_SHORT="no-such-disk-zzz") → ERROR: Installation failed: filter did not match any device, abort exit 1, no interactive fallback. 8 GB canary disk byte-identical after (sha256 -c: OK, host-side; scenarioD-filter-abort.png).
• A — full unattended chain — MECHANISM PASS; terminal success operator-gated. Zero keypress; stub ran once (uid 0) and enabled+started felhom-bootstrap; the unit fetched host-install from the REAL channel https://felhom.eu/scripts/felhom-host-install.sh and invoked it --customer-id … --mode appliance --hub-url …; host-install ran unattended through preflight (hub reachable) — with the drill customer absent it correctly failed at config preflight (404) and the unit retried (bootstrap-journal.log; NRestarts ≥3; 12 host-install invocations). Not live-proven: host-install rc-0 → hub-enrolled + guest-provisioned → done-flag + unit-disable + no-re-run. Reason: that needs a properly created drill customer, and CC cannot use the password-gated create-UI while a direct write to the live, locked hub hub.db (verified open by the running pod) risks corruption. The done-flag/disable/env-scrub success path is code-reviewed + shellcheck-clean; empirical proof requires an operator-created drill customer (supervised).
• B — no network at first boot — PASS. During a host-side link_down window (13:37:56→13:39:37) the chain failed repeatedly — host-install FAILED rc=6 (host-install's own hub fetch, no net) and ERROR: host-install fetch failed (no network yet?) — and the unit kept retrying (never died); on link-up it resumed automatically (13:40:04 fetch+run). No "single failure → dead unit". The stub-runs-without-network property is spike-proven (S8a); bootstrap-journal.log.
• 2.3 resume decision — PASS. After creating /var/lib/felhom-install/state.json, the next attempt logged prior install state present (…) -> adding --resume; earlier attempts ran plain.
• Exactly-once stub — PASS. Host-side reboot → /var/log/felhom-first-boot.log hash unchanged (13cf3fe4…), pending-flag consumed; the retry unit correctly persisted (still activating, not done).

ISO artifacts (build ≈ 12 s each, source already local)

ISO	sha256	size
felhom-pve-9.2-1-v1.16.0-nested-vm.iso	8cbe5eaeaa0d…82bb	1,707,737,088 B
felhom-pve-9.2-1-v1.16.0-nested-canary.iso	6b4568eaeac0…0301	1,707,737,088 B
(validation build, key baked) …-nested-vm-val.iso	0042acfdb70a…98e7	—

Source ISO proxmox-ve_9.2-1.iso sha256 4e88fe4…f2c6c (verified vs official SHA256SUMS). Assistant proxmox-installer-common 9.2.7 (trixie no-sub). Artifacts kept under 180:~/felhom-iso/out/, never committed (secret-bearing).

VM 310 final state

Stopped, kept; SeaBIOS, scsi0 32 GB + scsi1 8 GB (canary intact), ISO detached. Boundaries honored: VM 300 (drill) running untouched, 9000 stopped, LXC 9201 (demo) running untouched, hub untouched. Secret-bearing pipeline ISOs removed from felhom-pve (their embedded passphrase was a placeholder, not a real credential).

NOT-live-validated

• Physical N100 run (RUNBOOK B): real DMI/firmware, USB-boot media, vendor Secure Boot key set, real-NIC naming, prior-LVM blkdiscard on real hardware. Not started (out of scope by design).
• host-install rc-0 terminal success (hub-enrolled + guest-provisioned + done-flag + disable) and the 2.3 --resume executed against a real partial install — both need an operator-created drill customer (password-gated create-UI; live-DB insert unsafe). The decisions/plumbing are proven; only the terminal success is deferred.

Versioning note

felhom-host-install reached v1.16.0 on 2026-07-13 (before this task's stated 1.15.x baseline). The new ISO pipeline is tagged v1.16.0 to stay coherent with that release (the ISO's baked invocation targets host-install's flag surface). The scripts/CHANGELOG.md entry is a new component under the same release tag, not a bump of host-install.

Observations (recorded, not acted on)

• prepare-iso (not just validate-answer) was observed to exit 0 on a bad answer — the build's [[ -f out.iso ]] output-existence guard is therefore load-bearing, not just belt-and-suspenders.
• host-install prints unexpected hub status 404404 (doubled) for an absent customer — cosmetic host-install quirk; left untouched (read-only this task).
• host-install exits rc=6 when it cannot reach the hub (no net) — the bootstrap correctly treats any non-zero rc as retry.