Files

T

admin 19c932a693 SPIKE 2 complete: locked root closes the PVE web UI; before-network gives a measured zero window

Findings only — no script, profile or build file changed; no ISO built, nothing published.
documentation/audits/SPIKE-universal-iso-2-2026-07-31.md

Both Tier 0 boxes went offline mid-session (provider cable fault; four routes tried, no Tier 2
fallback used) and returned. All three scenarios then ran to completion on real PVE, each signalled
by reboot-mode='power-off' rather than a disk hash.

- A LOCKED ROOT CLOSES THE PVE WEB INTERFACE. Measured at the exact endpoint the UI uses
(POST /api2/json/access/ticket, root@pam) WITH A WORKING CONTROL: known-password install returns
HTTP 200 + ticket; locked install returns 401 for every password and none can exist.
passwd -S root = L, shadow = literal-asterisk, PVE uses the stock PAM stack.
- GRUB recovery mode is also closed ('the root account is locked') — but init=/bin/bash still gives
an unauthenticated root@(none):/#. A locked box is recoverable, operator-only, at the console.
The installed GRUB has NO password, so locking root is not a physical-security measure. R-152.
- before-network MEASURED (A/B, same image): the hook RUNS (marker, uptime 6.58s) with entropy 256,
writable /etc, all binaries and openssl_rand_len=32, while ip_global is EMPTY and
listen_22_8006 = 0. fully-up is the converse: sshd+pveproxy active, 3 listening. Zero window.
- R-148: answer.toml.tmpl:27 justifies fully-up with a pvesh/pct dependency the stub does not have
(grep rc=1) — it blocked the ordering now measured as the fix.
- R-149 three ordering values; R-150 Condition-guarded hooks skip silently; R-151 demo-felhom built
from an uncommitted profile.

Three probes failed and are recorded as failed: a container probe that ran as uid 0, a GRUB probe
that missed the 1-second menu timeout, and a kernel-line edit one line off (caught by a pre-typing
verification screendump). The interim 'Layer 1 teardown INCOMPLETE' is corrected — the fixture had
never landed, because the staging mkdir was in the SSH call that timed out.

2026-07-31 13:57:21 +02:00

5.7 KiB

Raw Blame History

REPORT — SPIKE 2: closing the last mechanisms, and the universal ISO's shape (2026-07-31)

Written as REPORT-universal-iso-spike.md, not REPORT.md: the shared file belongs to today's hub v0.85.0 session and the second session in a shared clone never touches it.

Class: Spike. Findings only — no production file changed, no ISO built, nothing published. Evidence: documentation/audits/SPIKE-universal-iso-2-2026-07-31.md.

All four questions are answered, and every answer that matters is measured on a real PVE install. The session lost both Tier 0 boxes mid-run (provider cable fault; four routes tried, no Tier 2 fallback used) and resumed when connectivity returned.

The two results that decide the product questions

A locked root closes the PVE web interface. Measured against the exact endpoint the UI logs in through — POST /api2/json/access/ticket, realm root@pam — with a working control: the known-password install returns HTTP 200 and a ticket; the locked install returns 401 for every password offered, and no password exists that could work. The control is what makes this a finding rather than a broken probe. passwd -S root reports L, the shadow field is literal-asterisk, and PVE uses the stock PAM stack.

GRUB's recovery mode is closed too — Cannot open access to console, the root account is locked. — but init=/bin/bash still works, giving an unauthenticated root@(none):/#. So a locked-root box is not unrecoverable, as the structural reading had feared; it is recoverable operator-only, at the physical console. The installed GRUB has no password, which also means locking root is not a physical-security measure and must never be described as one (R-152).

The mechanism that fixes the credential window

before-network measured, A/B, same image, only ordering differing. The hook runs (positive marker, uptime 6.58 s) with entropy 256, a writable /etc, every binary it needs and a demonstrated openssl_rand_len=32 — while ip_global is empty and listen_22_8006 is 0. The fully-up column is the converse proof: sshd and pveproxy already active, 3 sockets listening. Zero credential window, measured, and the shipped unit (Before=network-pre.target, Type=oneshot) generalises it to every machine.

And the reason it was never used is a comment that is wrong. answer.toml.tmpl:27 justifies fully-up as needed "so pvesh/pct work", but stub-first-boot.sh makes no pvesh/pct/pveum/qm call at all (grep rc=1); that usage lives in felhom-bootstrap.sh, under its own properly-ordered unit. R-148 — a comment asserting a dependency that does not exist, which blocked the fix.

Probes that failed, recorded as failed

Three, none massaged: a container probe that ran as uid 0 so su never invoked pam_unix (rc=0 even with a wrong password); a GRUB probe that missed the 1-second menu timeout and typed into the login prompt; and a kernel-line edit that landed one line off — caught by a pre-typing verification screendump before it was booted. All were rerun.

Also corrected: the interim report's "Layer 1 teardown INCOMPLETE" was over-cautious. When demo-hp returned, /mnt/nvme-1tb/spike2/ did not exist and disk usage was unchanged — the staging mkdir had been inside the SSH call that timed out as the link degraded, so the fixture never landed.

The operator's decisions

Escrow versus discard — no longer blocked, and reshaped twice. "Discard" and "lock" are the same outcome (nobody holds the password, so both sulogin and the web UI refuse), and neither makes a box unrecoverable because init=/bin/bash always works. So escrow buys only this: the operator keeps the web-UI and sulogin routes without physical presence. Against that stands D6 — a hub holding console root for the whole fleet is what the hub alone must not be. I recommend discard/lock for the public image, escrow remaining defensible for operator-built boxes. Your call; it is no longer waiting on a measurement.

The default boot entry — framed, unruled. The disk evidence (Spike 1) and the recovery evidence (this session) now point the same way: a tester whose unattended install succeeds but whose pairing fails has no web UI and no recovery shell. The cost is the unattended experience. Two profiles from one build is cheap, so operator-built and public need not match.

Still open

One gap in Q3: Felhom's real stub at before-network is argued structurally, not observed — the scenarios used the spike's own diagnostic as the hook. That is the one thing to measure before a spec is written (~40 min). Also: the SSH-as-root leg used a wrong password in the control too, so it discriminates nothing and is recorded as a gap rather than a result.

R-rows

R-148 (fully-up justified by a non-existent dependency) · R-149 (three ordering values) · R-150 (Condition-guarded hooks skip silently) · R-151 (demo-felhom built from an uncommitted profile) · R-152 (a locked root is not a physical-security measure).

Teardown

All three layers verified positively. demo-hp is back to its exact pre-spike state: spike2/ and /tmp/paia removed, 0 QEMU processes, 0 loop devices, disk usage 6.6 G — identical to pre-spike, 4 storages unchanged, nothing on local-lvm, drill-r50 stopped and 9201 running, both untouched. demo-felhom: probe binary removed, external USB drive observed only, still mounted as found. DooPlex: scratchpad 1.7 GB → 60 K, control password shred -u'd and verified absent, felhom-iso/out/ untouched at 17 ISOs, no production file modified. Hub-side: nothing created — the VMs had no route to it. Nothing published.

5.7 KiB Raw Blame History