admin
5bdd8372f8
Findings only — no script, profile or build file changed; no ISO built, nothing published. documentation/audits/SPIKE-universal-iso-2-2026-07-31.md BOTH Tier 0 boxes went offline mid-session (remote site, 12:28 CEST; four routes tried, our tailscale pod healthy). Q1/Q2/Q3 each keep a part needing a nested VM: those are BLOCKED, not answered. DooPlex was NOT used as a fallback — Tier 2, and this task did not authorise it. Established without them: - STRUCTURAL: ordering='before-network' maps to proxmox-first-boot-network-pre.service (Before=network-pre.target, Type=oneshot) — it completes before ANY interface is configured, so a rotation there has a zero-length window BY CONSTRUCTION, not by being fast. - R-148: the stub does not need 'fully-up'. stub-first-boot.sh has no pvesh/pct/pveum/qm call (grep rc=1); that usage is in felhom-bootstrap.sh under its own After=network-online unit. answer.toml.tmpl:27 justifies the current ordering with a dependency that does not exist. - R-149: the ordering enum has THREE values (before-network, network-online, fully-up), not two. - MECHANISM (container, not PVE): locked root closes sulogin — 'the root account is locked' for both '*' and '!', with a working control. So 'discard' and 'lock' are the SAME outcome for recovery, making the escrow decision binary. - R-150: all four proxmox-first-boot-* units are Condition-guarded; a failed condition is a SKIP, so a hook that never ran looks identical to one that succeeded. - R-151: demo-felhom was installed from an UNCOMMITTED profile — a Tier 0 reference box is not reproducible from main. - Q4: four gates in iso-repack.sh enforce the single-entry menu; default/timeout already settable. The first mechanism probe was invalid (uid 0 bypassed pam_unix; sulogin had no tty) and a teardown error (shredding the control plaintext) are both recorded as failures, not massaged. demo-hp teardown is INCOMPLETE and named as such; the command is recorded, not claimed done.
6.3 KiB
REPORT — SPIKE 2: closing the last mechanisms, and the universal ISO's shape (2026-07-31)
Written as
REPORT-universal-iso-spike.md, notREPORT.md: the shared file belongs to today's hub v0.85.0 session and the standing rule is that the second session in a shared clone never touches it. This file supersedes its own Spike 1 contents.
Class: Spike. Findings only — no production file changed, no ISO built, nothing published.
Evidence: documentation/audits/SPIKE-universal-iso-2-2026-07-31.md. Continues
SPIKE-universal-iso-2026-07-31.md.
The headline: the venue went away, and three answers are blocked
At 12:28 CEST both Tier 0 boxes went offline simultaneously and did not return. Four routes were
tried (tailnet direct, ssh demo-hp, demo-hp-lan via ProxyJump, ssh felhom-pve); our tailscale
pod is healthy and tailscale status shows both peers offline with a rising last seen. Both boxes
are at the same remote site — that site's power or uplink, not fixable from here.
No fallback venue was used. DooPlex is Tier 2, the 2026-07-25 ruling moved drill VMs off it, and
this task did not authorise it — an absent fence is not permission. So Q1, Q2 and Q3 each keep a
part that is BLOCKED, not answered, and they are labelled that way throughout. The fixture is built
and staged with Spike 1's two measurement defects already corrected (a NIC present at install time; a
reboot-mode = "power-off" completion signal instead of a disk hash).
What was established anyway
The best result of the session, and it is structural rather than measured — which makes it
stronger. ordering = "before-network" maps to proxmox-first-boot-network-pre.service
(proxmox-first-boot 9.2.5, read out of the ISO's own package), which is Before=network-pre.target
with Type=oneshot. It therefore runs to completion before any network interface is configured.
A credential rotation placed there has a zero-length exposure window by construction, not by being
fast — and that holds on every machine, unlike the timing number Spike 1 failed twice to obtain.
And the reason it was never used is a comment that is wrong. answer.toml.tmpl:27 justifies
fully-up as needed "so pvesh/pct work" — but stub-first-boot.sh makes no pvesh/pct/pveum/qm
call at all (grep rc=1). That usage lives in felhom-bootstrap.sh, which runs under its own
After=network-online.target … pveproxy.service unit. A comment asserting a dependency that does
not exist, which has constrained the design ever since. R-148.
A locked root closes sulogin — measured in an isolated container with a working control (correct
password → SU_OK, wrong → Authentication failure; then * and ! both → L and
Cannot open access to console, the root account is locked.). This reshapes the escrow question: if
nobody holds the password, sulogin refuses either way, so "discard" and "lock" are the same
outcome for recovery, and the decision is binary. Caveat stated plainly: container, not PVE.
The ordering enum has three values, not two — before-network, network-online, fully-up.
Spike 1 recorded two; the middle one has never been evaluated (R-149).
A first-boot hook that never runs is indistinguishable from one that succeeded — all four
proxmox-first-boot-* units are Condition*-guarded and a failed condition is a skip, not a
failure. Any rotation-in-first-boot design must verify by a positive marker (R-150).
Q4 answered fully from source. The single-entry menu is enforced by four gates in
iso-repack.sh (:153-155, :156, :160-164, and a post-remaster re-extraction at :314-319), so
restoring an interactive entry is a gate change rather than a template edit — but the banned-token
gate would pass unchanged for a stock Graphical entry, so the safety property survives. default
and timeout are already settable in grub/grub.cfg.tmpl.
What failed, and is recorded as failed
The first mechanism probe produced zero valid observations: it ran everything as uid 0, so su
never invoked pam_unix (rc=0 even with a wrong password), sulogin died on tcgetattr with no tty,
and a sed mangled the control hash. Rerun with a non-root caller, a real pty, and chpasswd.
A teardown error is also recorded rather than quietly fixed: shredding the control password
destroyed the plaintext scenario C existed to use, so C.toml was deleted rather than left looking
usable. It must be regenerated before the PVE-web-interface measurement is attempted.
The two decisions, and why neither should be ruled yet
Escrow versus discard — now binary (see above). Blocked on one measurement: whether a locked root also closes the PVE web interface. If it does, discarding removes the last remote route as well as the local one and the balance shifts sharply toward escrow. Do not rule before that is measured.
The default boot entry for a public image — the disk evidence points at interactive-default; the cost is the unattended experience. Partially blocked on the same measurement. The operator-built-versus-public split is cheap and available regardless of the ruling.
R-rows
R-148 (fully-up justified by a non-existent dependency) · R-149 (three ordering values, not
two) · R-150 (Condition-guarded first-boot hook skips silently) · R-151 (demo-felhom was
installed from an uncommitted profile — a Tier 0 reference box is not reproducible from main).
Teardown
Layer 1 (demo-hp) is INCOMPLETE and named as such — the box went offline while
/mnt/nvme-1tb/spike2/ held the staged fixture. Nothing was running on it (last check: 0 QEMU
processes, no install started this session), nothing on local-lvm, no PVE storage registered,
drill-r50 and 9201 never touched. The outstanding command is recorded in §10 of the findings doc
and is not claimed as done — R-131 exists because recorded commands never get run.
Layer 2 (demo-felhom): nothing placed this session. Layer 3 (DooPlex): scratchpad 4.4 GB → 48 K,
control password shred -u'd and verified absent, mechanism container --rm, no global Docker
cleanup, felhom-iso/out/ untouched at 17 ISOs, no production file modified. Hub-side: nothing
created — verified by fetching the customer list and searching it, not asserted. Nothing published.