SPIKE 2: before-network gives a zero window by construction; locked root closes sulogin

Findings only — no script, profile or build file changed; no ISO built, nothing published.
documentation/audits/SPIKE-universal-iso-2-2026-07-31.md

BOTH Tier 0 boxes went offline mid-session (remote site, 12:28 CEST; four routes tried, our
tailscale pod healthy). Q1/Q2/Q3 each keep a part needing a nested VM: those are BLOCKED, not
answered. DooPlex was NOT used as a fallback — Tier 2, and this task did not authorise it.

Established without them:

- STRUCTURAL: ordering='before-network' maps to proxmox-first-boot-network-pre.service
  (Before=network-pre.target, Type=oneshot) — it completes before ANY interface is configured,
  so a rotation there has a zero-length window BY CONSTRUCTION, not by being fast.
- R-148: the stub does not need 'fully-up'. stub-first-boot.sh has no pvesh/pct/pveum/qm call
  (grep rc=1); that usage is in felhom-bootstrap.sh under its own After=network-online unit.
  answer.toml.tmpl:27 justifies the current ordering with a dependency that does not exist.
- R-149: the ordering enum has THREE values (before-network, network-online, fully-up), not two.
- MECHANISM (container, not PVE): locked root closes sulogin — 'the root account is locked' for
  both '*' and '!', with a working control. So 'discard' and 'lock' are the SAME outcome for
  recovery, making the escrow decision binary.
- R-150: all four proxmox-first-boot-* units are Condition-guarded; a failed condition is a SKIP,
  so a hook that never ran looks identical to one that succeeded.
- R-151: demo-felhom was installed from an UNCOMMITTED profile — a Tier 0 reference box is not
  reproducible from main.
- Q4: four gates in iso-repack.sh enforce the single-entry menu; default/timeout already settable.

The first mechanism probe was invalid (uid 0 bypassed pam_unix; sulogin had no tty) and a teardown
error (shredding the control plaintext) are both recorded as failures, not massaged.

demo-hp teardown is INCOMPLETE and named as such; the command is recorded, not claimed done.

REPORT-universal-iso-spike.md

@@ -1,93 +1,95 @@
-# REPORT — SPIKE: what does a genuinely universal ISO require? (2026-07-31)
+# REPORT — SPIKE 2: closing the last mechanisms, and the universal ISO's shape (2026-07-31)
 
-> Written as **`REPORT-universal-iso-spike.md`**, not `REPORT.md`: the shared file was overwritten at
-> 09:24 today by the hub v0.85.0 Network-card session, and the standing rule is that the second
-> session in a shared clone never touches it.
+> Written as `REPORT-universal-iso-spike.md`, not `REPORT.md`: the shared file belongs to today's hub
+> v0.85.0 session and the standing rule is that the second session in a shared clone never touches it.
+> This file supersedes its own Spike 1 contents.
 
-**Class: Spike.** Output is a findings document. **No production code was changed, no ISO was built,
-nothing was published.** Full evidence:
-`documentation/audits/SPIKE-universal-iso-2026-07-31.md`.
+**Class: Spike.** Findings only — no production file changed, no ISO built, nothing published.
+Evidence: `documentation/audits/SPIKE-universal-iso-2-2026-07-31.md`. Continues
+`SPIKE-universal-iso-2026-07-31.md`.
 
-## Why this ran
+## The headline: the venue went away, and three answers are blocked
 
-The ISO build/publish task stopped before its Part 2. Its Part 0 found that every ISO bakes a root
-password hash into `/answer.toml`, and that `generic.profile` picks a disk by the name `sda` rather
-than by identity — both of which make a public download a product decision. This spike establishes
-what a genuinely universal image would require.
+At 12:28 CEST both Tier 0 boxes went offline **simultaneously** and did not return. Four routes were
+tried (tailnet direct, `ssh demo-hp`, `demo-hp-lan` via ProxyJump, `ssh felhom-pve`); our tailscale
+pod is healthy and `tailscale status` shows both peers `offline` with a rising `last seen`. Both boxes
+are at the same remote site — that site's power or uplink, not fixable from here.
 
-## What was answered
+**No fallback venue was used.** DooPlex is Tier 2, the 2026-07-25 ruling moved drill VMs off it, and
+this task did not authorise it — an absent fence is not permission. So **Q1, Q2 and Q3 each keep a
+part that is BLOCKED, not answered**, and they are labelled that way throughout. The fixture is built
+and staged with Spike 1's two measurement defects already corrected (a NIC present at install time; a
+`reboot-mode = "power-off"` completion signal instead of a disk hash).
 
-**Q1 — disk selection. The central finding, and it is worse than the known hazard.**
+## What was established anyway
 
-A filter that matches more than one device **does not fail safe**. Observed in a nested VM with two
-identical disks and `filter.ID_SERIAL_SHORT = "SPIKE*"`: the installer silently selected one and wiped
-it (`disk0` hash `de001102…` → `382b55ca…`; `disk1` byte-identical throughout). `validate-answer`
-accepts the answer — it cannot know what the filter resolves to on the target. The
-`filter did not match any devices` guard the safety profiles rely on covers the **zero**-match case
-only.
+**The best result of the session, and it is structural rather than measured — which makes it
+stronger.** `ordering = "before-network"` maps to `proxmox-first-boot-network-pre.service`
+(`proxmox-first-boot` 9.2.5, read out of the ISO's own package), which is `Before=network-pre.target`
+with `Type=oneshot`. It therefore runs **to completion before any network interface is configured**.
+A credential rotation placed there has a **zero-length exposure window by construction, not by being
+fast** — and that holds on every machine, unlike the timing number Spike 1 failed twice to obtain.
 
-And a filter that reliably matches exactly one disk on unseen hardware cannot be written, because
-**no property distinguishes an internal system disk from external media.** Measured on `demo-felhom`
-with its 1 TB external drive attached — the exact hazard configuration:
+**And the reason it was never used is a comment that is wrong.** `answer.toml.tmpl:27` justifies
+`fully-up` as needed "so pvesh/pct work" — but `stub-first-boot.sh` makes no `pvesh`/`pct`/`pveum`/`qm`
+call at all (grep `rc=1`). That usage lives in `felhom-bootstrap.sh`, which runs under its own
+`After=network-online.target … pveproxy.service` unit. A comment asserting a **dependency** that does
+not exist, which has constrained the design ever since. **R-148.**
 
-- `ID_BUS` is `ata` for **both** (the external unit is a SATA drive in a USB caddy)
-- `lsblk RM` is `0` for **both**; `device-info` exposes no removability property at all
-- on `demo-hp`, the NVMe carries **no `ID_BUS` and no `ID_TYPE`** — a different property set entirely
-- nothing anywhere expresses *role*: on `demo-hp` the system disk is the small SATA one and the big
-  NVMe is customer data
+**A locked root closes `sulogin`** — measured in an isolated container with a working control (correct
+password → `SU_OK`, wrong → `Authentication failure`; then `*` and `!` both → `L` and
+`Cannot open access to console, the root account is locked.`). This reshapes the escrow question: if
+nobody holds the password, `sulogin` refuses either way, so **"discard" and "lock" are the same
+outcome for recovery**, and the decision is binary. Caveat stated plainly: container, not PVE.
 
-**Q2 — root password.** A credential field is mandatory (`One of global.root-password or
-global.root-password-hashed must be set`), but a *usable* one is not: `root-password-hashed = "*"`
-validates **and installs to completion** (guest powered itself off after 200 s using
-`reboot-mode = "power-off"` as the completion signal). `[first-boot].ordering` also accepts
-`before-network` — the only ordering that could close the exposure window structurally, since today's
-bootstrap unit is `After=network-online.target pveproxy.service` and the pairing poll waits
-indefinitely for the operator's bind.
+**The ordering enum has three values, not two** — `before-network`, `network-online`, `fully-up`.
+Spike 1 recorded two; the middle one has never been evaluated (**R-149**).
 
-**Q3 — two-entry boot menu: yes, trivially.** `prepare-iso` leaves `grub.cfg` **byte-identical** to
-stock; a prepared ISO has 15 entries including automated *and* interactive. Felhom's repack is what
-collapses it to one. No PVE-side obstacle exists.
+**A first-boot hook that never runs is indistinguishable from one that succeeded** — all four
+`proxmox-first-boot-*` units are `Condition*`-guarded and a failed condition is a *skip*, not a
+failure. Any rotation-in-first-boot design must verify by a positive marker (**R-150**).
 
-**Q4 — what else is baked.** By enumeration against the stock ISO, not pattern-guessing: the pipeline
-adds exactly six paths; both OS squashfs payloads are byte-identical to stock; all three first-boot
-payloads match repo HEAD or carry only the hub URL. `/answer.toml` is the only secret-bearing content.
-
-**R-129 resolved.** demo-hp's key is the operator's own, added post-install — not baked. But
-demo-felhom's *is* baked, by an uncommitted profile, so the "no baked key" claim was already false for
-one demo box and nothing recorded it.
+**Q4 answered fully from source.** The single-entry menu is enforced by **four** gates in
+`iso-repack.sh` (`:153-155`, `:156`, `:160-164`, and a post-remaster re-extraction at `:314-319`), so
+restoring an interactive entry is a gate change rather than a template edit — but the banned-token
+gate would pass unchanged for a stock *Graphical* entry, so the safety property survives. `default`
+and `timeout` are already settable in `grub/grub.cfg.tmpl`.
 
 ## What failed, and is recorded as failed
 
-**The reachable-before-rotation measurement was attempted twice and both attempts were invalid.**
-Attempt 1 reported a plausible **`0 s`** that was an artefact — QEMU's `hostfwd` accepts on the host
-before touching the guest, and the same run then failed to read an SSH banner at all. Attempt 2, with
-a protocol-level detector, reported `never` because the install VM had no NIC, so the installed
-system's interface config could not match the boot VM's. **No number is reported.** The structural
-finding (unit ordering + unbounded pairing poll) stands instead and is the stronger claim.
+The first mechanism probe produced **zero valid observations**: it ran everything as uid 0, so `su`
+never invoked `pam_unix` (rc=0 even with a wrong password), `sulogin` died on `tcgetattr` with no tty,
+and a `sed` mangled the control hash. Rerun with a non-root caller, a real pty, and `chpasswd`.
 
-Also not established: that the locked root account is actually locked (install completion is a
-different claim), and whether GRUB single-user recovery works — the latter gates the escrow-vs-discard
-ruling and should be measured before that ruling is made.
+A **teardown error** is also recorded rather than quietly fixed: shredding the control password
+destroyed the plaintext scenario C existed to use, so `C.toml` was deleted rather than left looking
+usable. It must be regenerated before the PVE-web-interface measurement is attempted.
 
-## Asked of the operator
+## The two decisions, and why neither should be ruled yet
 
-One decision, framed in §4.5 of the findings doc: **escrow or discard** a per-box root password. Not
-recommended either way — the D6 trust model argues against a hub-held fleet-wide vault, and an unbound
-box has no agent access, which is exactly when console access matters.
+**Escrow versus discard** — now binary (see above). **Blocked on one measurement:** whether a locked
+root also closes the PVE web interface. If it does, discarding removes the last remote route as well
+as the local one and the balance shifts sharply toward escrow. **Do not rule before that is measured.**
+
+**The default boot entry for a public image** — the disk evidence points at interactive-default; the
+cost is the unattended experience. Partially blocked on the same measurement. The
+operator-built-versus-public split is cheap and available regardless of the ruling.
 
 ## R-rows
 
-**R-139** (HIGH, multi-match wipes silently) · **R-140** (`device-match` exits 0 on no-match) ·
-**R-141** (HIGH, mandatory + fleet-shared credential, unbounded window) · **R-142** (`FELHOM_ROOT_SSH_KEY`
-can ship a shared key, no assertion) · **R-143** (retired `kisfenyo@windows` key still live on
-demo-felhom) · **R-144** (`nested-probe` ISO unreproducible — profile gone) · **R-145** (variant naming
-undocumented) · **R-146** (`root-password-hashed = ""` validates) · **R-147** (hash substituted into
-the answer template's own comments) · **R-128** restated (a comment, not an assertion).
+**R-148** (`fully-up` justified by a non-existent dependency) · **R-149** (three ordering values, not
+two) · **R-150** (Condition-guarded first-boot hook skips silently) · **R-151** (demo-felhom was
+installed from an **uncommitted** profile — a Tier 0 reference box is not reproducible from `main`).
 
 ## Teardown
 
-All three layers verified positively: demo-hp scratch removed and `drill-r50`/9201/`storage.cfg`
-untouched, `local-lvm` never written; demo-felhom probe binary removed and its external drive observed
-only; DooPlex scratchpad 3.2 GB → 28 K with the throwaway password `shred`ed. **No hub object was
-created** — confirmed by fetching the customer list and searching it, not by assertion. Nothing
-deleted from `felhom-iso/out/` (17 ISOs intact). Nothing published.
+**Layer 1 (demo-hp) is INCOMPLETE and named as such** — the box went offline while
+`/mnt/nvme-1tb/spike2/` held the staged fixture. Nothing was running on it (last check: 0 QEMU
+processes, no install started this session), nothing on `local-lvm`, no PVE storage registered,
+`drill-r50` and 9201 never touched. The outstanding command is recorded in §10 of the findings doc
+and **is not claimed as done** — R-131 exists because recorded commands never get run.
+
+Layer 2 (demo-felhom): nothing placed this session. Layer 3 (DooPlex): scratchpad 4.4 GB → 48 K,
+control password `shred -u`'d and verified absent, mechanism container `--rm`, no global Docker
+cleanup, `felhom-iso/out/` untouched at 17 ISOs, no production file modified. **Hub-side: nothing
+created** — verified by fetching the customer list and searching it, not asserted. Nothing published.