SPIKE 3: [first-boot] does NOT fire on an interactive install
Findings only — no script, profile or build file changed; no release ISO built, nothing published. documentation/audits/SPIKE-universal-iso-3-2026-07-31.md MEASURED with a control from the SAME image (one ISO, 15 GRUB entries): - Automated entry -> hook fires: ttyS0 marker, marker file, /var/lib/proxmox-first-boot/proxmox-first-boot (0700), activation symlink, unit active. - Terminal-UI entry, normal manual install -> ALL absent, and the proxmox-first-boot PACKAGE is not installed at all. A whole-filesystem grep for the marker returns nothing. Mechanism cited: Config.pm:118 defaults first_boot.enabled=0 and set_first_boot_opt is never called in the Perl tree; Install.pm:746 returns early without it; Install.pm:1360 skips the package. proxinstall (graphical) has ZERO occurrences of first-boot. [first-boot] is an automated-installer feature, unavailable on every interactive path by construction. R-154. A delivery mechanism DOES exist and is UNTESTED: Install.pm:1343-1372 unpacks every .deb in the ISO's /proxmox/packages/ into the target on every path (fixed skip-list), then dpkg --configure -a runs postinsts (:1378) — how PVE ships first-boot itself. Read from source, not measured. Q5: the public image should carry NO answer.toml at all — that removes the baked root hash, the disk profile and the whole Spike 1-2 problem space, and makes it a one-line release gate. But iso-repack.sh:100-106 refuses an ISO without auto-installer-mode.toml. R-155. Incidental R-153: hub hostInstallVersion=1.19.0 vs SCRIPT_VERSION=1.22.0; hostinstall_gates.py detects it and exits 1 — the gate works, nothing runs it. Q3 (real stub at before-network) was NOT reached and is recorded as not reached.
This commit is contained in:
@@ -1,89 +1,88 @@
|
||||
# REPORT — SPIKE 2: closing the last mechanisms, and the universal ISO's shape (2026-07-31)
|
||||
# REPORT — SPIKE 3: on an interactive install, does the ISO deliver anything at all? (2026-07-31)
|
||||
|
||||
> Written as `REPORT-universal-iso-spike.md`, not `REPORT.md`: the shared file belongs to today's hub
|
||||
> v0.85.0 session and the second session in a shared clone never touches it.
|
||||
> v0.85.0 session and the second session in a shared clone never touches it. Supersedes this file's
|
||||
> Spike 1 and Spike 2 contents.
|
||||
|
||||
**Class: Spike.** Findings only — no production file changed, no ISO built, nothing published.
|
||||
Evidence: `documentation/audits/SPIKE-universal-iso-2-2026-07-31.md`.
|
||||
**Class: Spike.** Findings only — no production file changed, no release ISO built, nothing published.
|
||||
Evidence: `documentation/audits/SPIKE-universal-iso-3-2026-07-31.md`.
|
||||
|
||||
**All four questions are answered**, and every answer that matters is measured on a real PVE install.
|
||||
The session lost both Tier 0 boxes mid-run (provider cable fault; four routes tried, no Tier 2
|
||||
fallback used) and resumed when connectivity returned.
|
||||
## The answer: no, and more completely than expected
|
||||
|
||||
## The two results that decide the product questions
|
||||
**`[first-boot]` does not fire on an interactive install.** Measured, with a control from the *same*
|
||||
image (one ISO, 15 GRUB entries):
|
||||
|
||||
**A locked root closes the PVE web interface.** Measured against the exact endpoint the UI logs in
|
||||
through — `POST /api2/json/access/ticket`, realm `root@pam` — **with a working control**: the
|
||||
known-password install returns **HTTP 200 and a ticket**; the locked install returns 401 for every
|
||||
password offered, and no password exists that could work. The control is what makes this a finding
|
||||
rather than a broken probe. `passwd -S root` reports `L`, the shadow field is `literal-asterisk`, and
|
||||
PVE uses the stock PAM stack.
|
||||
- **Automated entry** → hook fires: `ttyS0` marker, marker file,
|
||||
`/var/lib/proxmox-first-boot/proxmox-first-boot` (0700), activation symlink, unit `active`.
|
||||
- **Terminal UI entry, normal manual install** → every one of those **absent**, and the
|
||||
`proxmox-first-boot` **package is not installed at all**. A whole-filesystem grep for the marker
|
||||
returns nothing.
|
||||
|
||||
**GRUB's recovery mode is closed too** — `Cannot open access to console, the root account is locked.`
|
||||
— **but `init=/bin/bash` still works**, giving an unauthenticated `root@(none):/#`. So a locked-root
|
||||
box is *not* unrecoverable, as the structural reading had feared; it is recoverable **operator-only, at
|
||||
the physical console**. The installed GRUB has no password, which also means locking root is not a
|
||||
physical-security measure and must never be described as one (**R-152**).
|
||||
It is not that the hook fails to run — nothing that could run it is ever placed on the system. The
|
||||
mechanism, cited: `Config.pm:118` defaults `first_boot.enabled => 0` and `set_first_boot_opt` is never
|
||||
called anywhere in the Perl tree; `Install.pm:746` returns early without the flag; `Install.pm:1360`
|
||||
skips the package. And `proxinstall`, the graphical installer, contains **zero** occurrences of
|
||||
`first-boot`. **This is a property of the PVE installer, not a Felhom defect** — but nothing in our
|
||||
tree records it, which is R-154.
|
||||
|
||||
## The mechanism that fixes the credential window
|
||||
## The commercially relevant part
|
||||
|
||||
**`before-network` measured, A/B, same image, only `ordering` differing.** The hook **runs** (positive
|
||||
marker, uptime 6.58 s) with entropy 256, a writable `/etc`, every binary it needs and a demonstrated
|
||||
`openssl_rand_len=32` — while `ip_global` is **empty** and `listen_22_8006` is **0**. The `fully-up`
|
||||
column is the converse proof: sshd and pveproxy already active, 3 sockets listening. **Zero credential
|
||||
window, measured**, and the shipped unit (`Before=network-pre.target`, `Type=oneshot`) generalises it
|
||||
to every machine.
|
||||
**A delivery mechanism does exist.** `Install.pm:1343-1372` unpacks **every** `.deb` in the ISO's
|
||||
`/proxmox/packages/` into the target on *every* install path — a fixed skip-list of known package
|
||||
names, everything else installed — then `dpkg --configure -a` runs postinsts. That is exactly how PVE
|
||||
ships `proxmox-first-boot` itself.
|
||||
|
||||
**And the reason it was never used is a comment that is wrong.** `answer.toml.tmpl:27` justifies
|
||||
`fully-up` as needed "so pvesh/pct work", but `stub-first-boot.sh` makes no `pvesh`/`pct`/`pveum`/`qm`
|
||||
call at all (grep `rc=1`); that usage lives in `felhom-bootstrap.sh`, under its own properly-ordered
|
||||
unit. **R-148** — a comment asserting a *dependency* that does not exist, which blocked the fix.
|
||||
**It is read from source, not measured**, and I have not proven either that our repack preserves an
|
||||
added `.deb` or that a postinst running in the installer chroot can do the job (it runs *inside the
|
||||
installer*, not on a booted system). Proving it is the top follow-up, ~45 min. Until then the honest
|
||||
position is: the *insert-stick-install-type-claim-code* experience is **plausible but unvalidated**,
|
||||
and the documented one-command manual path is what works today.
|
||||
|
||||
## Probes that failed, recorded as failed
|
||||
Incidentally confirmed: the interactive installer shows the target disk and a summary listing
|
||||
`Bootdisk(s) /dev/sda` before erasing. Your ruling needs no Felhom work for that warning.
|
||||
|
||||
Three, none massaged: a container probe that ran as uid 0 so `su` never invoked `pam_unix` (rc=0 even
|
||||
with a wrong password); a GRUB probe that missed the **1-second** menu timeout and typed into the login
|
||||
prompt; and a kernel-line edit that landed one line off — caught by a pre-typing verification
|
||||
screendump before it was booted. All were rerun.
|
||||
## The manual path is already most of the way there
|
||||
|
||||
Also corrected: the interim report's "Layer 1 teardown INCOMPLETE" was over-cautious. When demo-hp
|
||||
returned, `/mnt/nvme-1tb/spike2/` did not exist and disk usage was unchanged — the staging `mkdir` had
|
||||
been inside the SSH call that timed out as the link degraded, so **the fixture never landed**.
|
||||
Two of the three things I was asked to look for already exist:
|
||||
|
||||
## The operator's decisions
|
||||
- **The hub already renders the command with the customer-id filled in**
|
||||
(`customer_unified.html:563-564`, plus a JS generator that adjusts the flags per role).
|
||||
- **The URL is already short, public and always current** —
|
||||
`felhom.eu/scripts/felhom-host-install.sh`, git-synced from `main` on a 30-second period.
|
||||
- **Console display does not exist.** Nothing writes `/etc/issue`; PVE's own banner is what appears
|
||||
after install. That file is the obvious carrier. Reported, not built.
|
||||
|
||||
**Escrow versus discard — no longer blocked, and reshaped twice.** "Discard" and "lock" are the same
|
||||
outcome (nobody holds the password, so both `sulogin` and the web UI refuse), and **neither makes a box
|
||||
unrecoverable** because `init=/bin/bash` always works. So escrow buys only this: the operator keeps the
|
||||
web-UI and `sulogin` routes without physical presence. Against that stands D6 — a hub holding console
|
||||
root for the whole fleet is what the hub alone must not be. **I recommend discard/lock for the public
|
||||
image**, escrow remaining defensible for operator-built boxes. Your call; it is no longer waiting on a
|
||||
measurement.
|
||||
## Two things I'd flag
|
||||
|
||||
**The default boot entry — framed, unruled.** The disk evidence (Spike 1) and the recovery evidence
|
||||
(this session) now point the same way: a tester whose unattended install succeeds but whose pairing
|
||||
fails has no web UI and no recovery shell. The cost is the unattended experience. Two profiles from one
|
||||
build is cheap, so operator-built and public need not match.
|
||||
**A working gate is red and nobody runs it.** `configs.go:28` has `hostInstallVersion = "1.19.0"`
|
||||
against `SCRIPT_VERSION = "1.22.0"`, and `scripts/hostinstall_gates.py` detects it and exits 1. The
|
||||
comment saying drift "is now gated" is true of the gate's existence and false of its enforcement, so
|
||||
the customer page's install-command generator targets a flag surface three minor versions stale.
|
||||
**R-153.**
|
||||
|
||||
## Still open
|
||||
**The cleanest public-image posture is currently blocked by our own tooling.** §6 of the findings
|
||||
argues the public image should carry **no `answer.toml` at all** — which removes the baked root hash,
|
||||
the disk profile, and every question Spikes 1–2 wrestled with, and makes "no `answer.toml`" a
|
||||
one-line release gate. But `iso-repack.sh:100-106` refuses any ISO lacking `auto-installer-mode.toml`.
|
||||
That guard is sound for its original case and wrong for this one. **R-155.**
|
||||
|
||||
One gap in Q3: **Felhom's real stub at `before-network`** is argued structurally, not observed — the
|
||||
scenarios used the spike's own diagnostic as the hook. That is the one thing to measure before a spec
|
||||
is written (~40 min). Also: the SSH-as-root leg used a wrong password in the control too, so it
|
||||
discriminates nothing and is recorded as a gap rather than a result.
|
||||
## Not reached
|
||||
|
||||
**Q3 — the real stub at `before-network`** was not run; driving the interactive install consumed the
|
||||
session. It is recorded as not reached, not inferred. Spike 2's structural case stands unrefuted, and
|
||||
§2 narrows its scope: with no first-boot hook on the interactive path, that question now governs
|
||||
**operator-built images only**. ~40 min when wanted.
|
||||
|
||||
## R-rows
|
||||
|
||||
**R-148** (`fully-up` justified by a non-existent dependency) · **R-149** (three ordering values) ·
|
||||
**R-150** (Condition-guarded hooks skip silently) · **R-151** (demo-felhom built from an uncommitted
|
||||
profile) · **R-152** (a locked root is not a physical-security measure).
|
||||
**R-153** (a working gate is red and unrun) · **R-154** (`[first-boot]` is automated-only and nothing
|
||||
in our tree says so) · **R-155** (`iso-repack.sh` blocks the no-`answer.toml` posture).
|
||||
|
||||
## Teardown
|
||||
|
||||
All three layers verified positively. demo-hp is back to its exact pre-spike state: `spike2/` and
|
||||
`/tmp/paia` removed, 0 QEMU processes, 0 loop devices, **disk usage 6.6 G — identical to pre-spike**,
|
||||
4 storages unchanged, nothing on `local-lvm`, `drill-r50` stopped and 9201 running, both untouched.
|
||||
demo-felhom: probe binary removed, external USB drive observed only, still mounted as found. DooPlex:
|
||||
scratchpad 1.7 GB → 60 K, control password `shred -u`'d and verified absent, `felhom-iso/out/`
|
||||
untouched at 17 ISOs, no production file modified. **Hub-side: nothing created** — the VMs had no route
|
||||
to it. Nothing published.
|
||||
All three layers verified positively. demo-hp back to its exact pre-spike state — `spike3/` removed,
|
||||
0 QEMU processes, 0 loop devices, **disk usage 6.6 G, identical to pre-spike**, 4 storages unchanged,
|
||||
nothing on `local-lvm`, `drill-r50` stopped and 9201 running. demo-felhom: nothing placed, external
|
||||
drive observed only. DooPlex: scratchpad 4.4 GB → 80 K, both throwaway passwords destroyed,
|
||||
`felhom-iso/out/` untouched at 17 ISOs, no production file modified. **Hub-side: nothing created** —
|
||||
verified by fetching and searching the customer list, not asserted. Nothing published.
|
||||
|
||||
@@ -0,0 +1,290 @@
|
||||
# SPIKE 3 — on an interactive install, does the ISO deliver anything at all? (2026-07-31)
|
||||
|
||||
> **Class: Spike.** Findings only. No production file changed, no ISO built for release, nothing
|
||||
> published.
|
||||
>
|
||||
> Continues `SPIKE-universal-iso-2026-07-31.md` and `SPIKE-universal-iso-2-2026-07-31.md`; their
|
||||
> results are premises and are not re-derived.
|
||||
|
||||
**Q1 is answered, measured, with a control, and the answer is NO — decisively so.** The first-boot
|
||||
hook does not merely fail to run on an interactive install: the mechanism is absent from the installed
|
||||
system entirely, down to the package. Everything else in this document is contingent on that, and §3
|
||||
is where the contingency lands: **a delivery mechanism does exist**, and it is the one PVE uses itself.
|
||||
|
||||
---
|
||||
|
||||
## 1. Baselines and fixture
|
||||
|
||||
| Baseline | Value |
|
||||
|---|---|
|
||||
| `ISO_VERSION` | `1.25.0` — `scripts/iso/build-felhom-iso.sh:44` |
|
||||
| `SCRIPT_VERSION` | `1.22.0` — `scripts/felhom-host-install.sh:187` |
|
||||
| `felhom-bootstrap.sh` @ HEAD | `21bf6a6bde0cb13e3809e2f5c136a49929dcc82eb8d40bbdf6f290a886ee8ab7` |
|
||||
| PVE base | `proxmox-ve_9.2-1.iso`, `4e88fe416df9b527624a175f24c9aa07c714d3332afb1ee3dbf3879573ef2c6c` |
|
||||
| `felhom.eu` HEAD | **`19c932a`**, clean, `== origin/main` |
|
||||
| `proxmox-first-boot` pkg | `9.2.5` (from the ISO's `/proxmox/packages/`) |
|
||||
|
||||
**Fixture provenance.** One probe ISO, built on DooPlex from the baseline PVE ISO with
|
||||
`prepare-iso --fetch-from partition --partition-label proxmox-ais --on-first-boot hook.sh`, sha256
|
||||
`62d8a4cc948b31850a8153c129b8763353ca7dd01c19c06c3816ebed89b7c028`, **verified byte-identical at both
|
||||
ends** after transfer. It carries **15 GRUB entries** — Automated, Graphical, Terminal UI and the rest
|
||||
— so the control and the test come from *the same build*, which is what makes the negative meaningful.
|
||||
|
||||
Nested VMs ran under plain QEMU/KVM on **demo-hp** (Tier 0, the designated drill host) under
|
||||
`/mnt/nvme-1tb/spike3/`. Never `local-lvm`; no PVE storage registered; `drill-r50` and guest 9201
|
||||
untouched. The automated install used `reboot-mode = "power-off"` as its completion signal; the
|
||||
interactive install's completion was observed directly on the console.
|
||||
|
||||
**The hook leaves three independent positive observables** — a `ttyS0` line, a persistent marker file,
|
||||
and its own log — because R-150 established that a first-boot hook which never runs is
|
||||
indistinguishable from one that succeeded if you only look for errors.
|
||||
|
||||
**Evidence classes:** **[MEASURED]** on a real install · **[STRUCTURAL]** read from shipped installer
|
||||
source · **[NOT REACHED]**.
|
||||
|
||||
---
|
||||
|
||||
## 2. Q1 — does `[first-boot]` fire on an interactive install? **[MEASURED] No.**
|
||||
|
||||
### 2.1 The control — same image, automated entry
|
||||
|
||||
| Observable | Result |
|
||||
|---|---|
|
||||
| `ttyS0` marker | **present** — `@@SPIKE3_HOOK_FIRED@@ … uptime=19.31` |
|
||||
| `/var/log/SPIKE3-HOOK-FIRED` | **present**, written at uptime 19.30 |
|
||||
| `/var/lib/proxmox-first-boot/proxmox-first-boot` | **present**, 572 bytes, mode `0700` |
|
||||
| `/etc/systemd/system/proxmox-first-boot.service` | **present** → `/lib/systemd/system/proxmox-first-boot-multi-user.service` |
|
||||
| unit state | `alias` / `active` |
|
||||
| `pending-first-boot-setup` flag | **absent** — removed by the unit's `ExecStartPost`, i.e. it ran to completion |
|
||||
|
||||
So the hook demonstrably fires from this build.
|
||||
|
||||
### 2.2 The test — same image, Terminal UI entry, a normal manual install
|
||||
|
||||
Driven screen by screen with a screendump before every action. The installer behaved exactly as the
|
||||
operator's ruling assumes: it **showed the target disk** (`/dev/sda (QEMU HARDDISK) (20.00 GiB)`) and
|
||||
presented a summary listing `Bootdisk(s) /dev/sda` before erasing anything. The install completed and
|
||||
auto-rebooted. Installed hostname `spikethree.felhom.eu`, which identifies the box below.
|
||||
|
||||
| Observable | Result |
|
||||
|---|---|
|
||||
| `ttyS0` marker | **absent** (`grep -c` = 0) |
|
||||
| `/var/log/SPIKE3-HOOK-FIRED` | **absent** — `No such file or directory` |
|
||||
| `/var/lib/proxmox-first-boot/` | **does not exist** |
|
||||
| `/etc/systemd/system/proxmox-first-boot.service` | **absent** |
|
||||
| any `proxmox-first-boot*` unit known to systemd | **none** — only the stock `first-boot-complete.target` |
|
||||
| **the `proxmox-first-boot` package** | **not installed** — `dpkg-query: no packages found matching proxmox-first-boot` |
|
||||
| `grep -rl SPIKE3_HOOK_FIRED /var /etc /root /usr/local` | **no hits** |
|
||||
|
||||
That is the search behind the negative, stated rather than asserted. **The hook did not fail to run —
|
||||
nothing that could have run it was ever placed on the system.**
|
||||
|
||||
### 2.3 The mechanism — **[STRUCTURAL]**, cited
|
||||
|
||||
Three lines in the shipped installer explain it exactly.
|
||||
|
||||
1. `Proxmox/Install/Config.pm:118-122` — the default is
|
||||
```perl
|
||||
first_boot => { enabled => 0, ordering_target => 'multi-user' },
|
||||
```
|
||||
and `set_first_boot_opt` (`:300`) is **never called anywhere in the Perl tree**. The flag can only
|
||||
become true through `Config::merge`, i.e. from the low-level config the auto-installer supplies.
|
||||
|
||||
2. `Proxmox/Install.pm:746` — `setup_proxmox_first_boot_service` opens with
|
||||
```perl
|
||||
return if !Proxmox::Install::Config::get_first_boot_opt('enabled');
|
||||
```
|
||||
so on any path that does not set the flag, nothing is planted.
|
||||
|
||||
3. `Proxmox/Install.pm:1360` — the package loop skips the package itself:
|
||||
```perl
|
||||
next if $deb =~ /^proxmox-first-boot/ && !needs_first_boot_package();
|
||||
```
|
||||
which is why §2.2 found the package missing, not merely unconfigured.
|
||||
|
||||
And the front-ends agree with the measurement: **`proxinstall` — the graphical interactive installer —
|
||||
contains zero occurrences of `first-boot`**; `proxmox-low-level-installer` likewise zero;
|
||||
`proxmox-tui-installer` has exactly one, the serde field name in the shared low-level config struct,
|
||||
with no way to set it. `proxmox-auto-installer` has eight.
|
||||
|
||||
**Conclusion: `[first-boot]` is an automated-installer feature. It is unavailable on every interactive
|
||||
path, by construction, and this is not a Felhom defect but a property of the PVE installer.**
|
||||
|
||||
---
|
||||
|
||||
## 3. Q2 — can the ISO plant anything into the installed system? **[STRUCTURAL] Yes — one mechanism exists.**
|
||||
|
||||
The search, stated: the installer's file-copy and package paths (`Proxmox/Install.pm`), the
|
||||
`[first-boot]` plumbing (§2.3), the answer schema's `post-installation-webhook` (an `answer.toml` key,
|
||||
so automated-only), and the `proxmox-post-hook` binary (part of the auto-install flow). Of these, one
|
||||
is mode-independent.
|
||||
|
||||
**`Install.pm:1343-1372` installs every `.deb` in the ISO's `/proxmox/packages/` into the target**, on
|
||||
every install path:
|
||||
|
||||
```perl
|
||||
while (<${proxmox_pkgdir}/*.deb>) {
|
||||
...
|
||||
next if $deb =~ /grub-pc_/ && (...); # boot type / arch
|
||||
next if $deb =~ /^proxmox-secure-boot-support_/ && !$run_env->{secure_boot};
|
||||
next if $deb =~ /^proxmox-first-boot/ && !needs_first_boot_package();
|
||||
next if $deb =~ /^(amd64|intel)-microcode_/ && (...); # cpu vendor
|
||||
syscmd("chroot $targetdir dpkg ... --unpack /tmp/pkg/$deb") == 0 || die ...;
|
||||
}
|
||||
```
|
||||
followed by `chroot $targetdir dpkg --force-confold --configure -a` (`:1378`) — **which runs postinst
|
||||
scripts.**
|
||||
|
||||
The skip-list is a fixed set of known package-name patterns. A package whose name matches none of them
|
||||
is unpacked and configured unconditionally. **So a `.deb` added to `/proxmox/packages/` is a working,
|
||||
existing, mode-independent way to plant a systemd unit and a script into the installed system — and it
|
||||
is precisely the mechanism PVE uses to deliver `proxmox-first-boot` itself.**
|
||||
|
||||
**Not tested. This is read from source, not measured**, and it is the single most important thing to
|
||||
prove before any spec is written on it (§7). Two things in particular are unverified: that
|
||||
`prepare-iso` preserves an added `/proxmox/packages/*.deb` through its repack, and that a postinst
|
||||
running under `dpkg --configure -a` in the installer chroot can do what a first-boot hook does (it runs
|
||||
inside the installer's chroot, **not** on a booted system, so it can install a unit but cannot expect
|
||||
network, `pveproxy`, or a running systemd).
|
||||
|
||||
---
|
||||
|
||||
## 4. Q3 — the real stub at `before-network` — **[NOT REACHED]**
|
||||
|
||||
Not run. The interactive-install driving in §2.2 consumed the session's remaining room, and Q1 was the
|
||||
question everything else was contingent on.
|
||||
|
||||
**What it needs** (unchanged from the Spike 2 estimate, ~40 min): build a probe ISO with
|
||||
`stub-first-boot.sh` as the `--on-first-boot` payload and an answer with
|
||||
`ordering = "before-network"`, install, boot, and confirm the stub's own log
|
||||
(`/var/log/felhom-first-boot.log`) exists, that `felhom-bootstrap.service` is enabled and started, and
|
||||
that pairing proceeds.
|
||||
|
||||
Spike 2's structural case is unchanged and unrefuted: the stub makes no `pvesh`/`pct`/`pveum`/`qm` call
|
||||
(grep `rc=1`), and the unit it starts carries its own `After=network-online.target … pveproxy.service`.
|
||||
**It remains argued, not observed.** Note §2 narrows where it matters: on the interactive path there is
|
||||
no first-boot hook at all, so this question now governs **operator-built images only**.
|
||||
|
||||
---
|
||||
|
||||
## 5. Q4 — how good can the manual path be? **[MEASURED/STRUCTURAL] Two of three already exist.**
|
||||
|
||||
1. **The hub already renders the command with the customer-id filled in.**
|
||||
`hub/internal/web/templates/customer_unified.html:563-564`:
|
||||
```
|
||||
curl -fsSL https://felhom.eu/scripts/felhom-host-install.sh -o felhom-host-install.sh \
|
||||
&& sudo bash felhom-host-install.sh --customer-id {{.CustomerID}} --mode <appliance|byo>
|
||||
```
|
||||
with a JS generator at `:1258-1267` that rebuilds it per selected role, including the right flags.
|
||||
**Nothing needs building here.**
|
||||
|
||||
2. **The URL is already short, public and always current.**
|
||||
`https://felhom.eu/scripts/felhom-host-install.sh` is the website's git-sync working tree tracking
|
||||
`main` on a 30-second period (`documentation/.../day0-install.md:150-152`) — no release tag, no
|
||||
staging copy, no version selector. Pushing the script publishes it.
|
||||
|
||||
3. **Console display does not exist.** `felhom-host-install.sh` does not write `/etc/issue`,
|
||||
`/etc/issue.net` or any MOTD (grep: no match). PVE's own `/etc/issue` banner is what the console
|
||||
shows after install — observed in this session's screendumps as
|
||||
*"Welcome to the Proxmox Virtual Environment … connect to: https://10.0.2.15:8006/"*. That file is
|
||||
the obvious carrier for a Felhom line, and today nothing writes one. **Reported, not built.**
|
||||
|
||||
**An incidental finding while reading (2): the hub's version-sync gate is currently red.**
|
||||
`hub/internal/web/configs.go:28` sets `hostInstallVersion = "1.19.0"` while `SCRIPT_VERSION` is
|
||||
`1.22.0`, and the comment at `:26` states *"Drift is now gated: `scripts/hostinstall_gates.py` asserts
|
||||
this const == SCRIPT_VERSION (drill F-1)."* Running that gate:
|
||||
|
||||
```
|
||||
FAIL: hub Setup-tab hostInstallVersion=1.19.0 != SCRIPT_VERSION=1.22.0 (F-1: bump both together)
|
||||
hostinstall gates: 1 FAILURE(S)
|
||||
```
|
||||
with a **true exit code of 1** (verified without a pipe — an initial `rc=0` reading was `tail`'s exit
|
||||
status, not the script's, and is corrected here). So the gate is correct, works, and detects the drift;
|
||||
**nothing runs it.** The comment's "is now gated" is true of the gate's existence and false of its
|
||||
enforcement. **R-153.**
|
||||
|
||||
---
|
||||
|
||||
## 6. Q5 — what the public image contains, as constraints
|
||||
|
||||
Given the operator's ruling (no fully automatic install; disk selection always interactive) and Q1's
|
||||
negative, stated as constraints on a build spec — **not an implementation, and nothing was built.**
|
||||
|
||||
| # | Constraint | Release gate (checkable against the built artifact) |
|
||||
|---|---|---|
|
||||
| 1 | **The public image carries NO `answer.toml`.** Q1 makes it pointless on the interactive path, and the ruling removes the automated path. Dropping it removes the baked root hash, the disk profile, and every question Spikes 1–2 were wrestling with, at once. | `osirrox -indev <iso> -find / -maxdepth 1` contains **no `/answer.toml`**. One line, trivially assertable. |
|
||||
| 2 | **…and therefore no `auto-installer-mode.toml` either.** Without it the stock `grub.cfg` does not even emit the *Automated* entry (Spike 1 §5: that entry is inside `if [ -f auto-installer-mode.toml ]`). The menu becomes interactive-only *by construction*, with no gate to maintain. | `/auto-installer-mode.toml` absent; `grub.cfg` contains no `proxmox-start-auto-installer`. |
|
||||
| 3 | **No baked credential of any kind.** Falls out of 1: no `root-password-hashed`, no `root-ssh-keys`, and **no `.rootpw.txt` companion emitted**. The installer prompts the human, who sets their own. | `grep -c root-password-hashed` and `grep -c root-ssh-keys` over the ISO both `0`; no `*.rootpw.txt` beside the output. |
|
||||
| 4 | **The boot menu is restored from the collapse**, with the interactive entry as `default` and a **human-length timeout**. Spike 2 lost a probe to the installed system's **1-second** menu; a person at an unfamiliar screen needs far longer. 10–30 s is the range the stock ISO itself uses (10). | `final.iso`'s `grub.cfg`: `set default=` points at an interactive entry, `set timeout` ≥ 10, and `timeout_style` uses the **underscore** spelling (the stock hyphenated `timeout-style` is not a GRUB variable at all — Spike 2 §5.2). |
|
||||
| 5 | **Restoring the menu means relaxing gates, not deleting them.** Four gates in `iso-repack.sh` enforce the single entry (`:153-155`, `:156`, `:160-164`, `:314-319`). Keep the banned-token gate exactly as it is — it is the one that prevents a route to a debug/rescue/manual-disk-picker path, and a stock *Graphical* entry passes it unchanged. | `grub.cfg` still has no live `proxtui`/`proxdebug`/`nomodeset`/`Rescue Boot`/`memtest`/`fwsetup`; entry count assertion becomes an allowlist rather than `== 1`. |
|
||||
| 6 | **No first-boot hook on this image.** Q1: it cannot fire. Shipping one would be a payload that provably never executes — the inert-seam failure this project has had four times. | `/proxmox-first-boot` absent from the ISO. |
|
||||
| 7 | **Day-0 is the documented manual command**, and the release notes say so plainly. §5 shows the hub already emits it per customer and the URL is already public and current. | Release notes contain the `felhom.eu/scripts/felhom-host-install.sh` command; the hub customer page renders it (already true). |
|
||||
| 8 | **The profile the image uses must be COMMITTED.** Spike 1 found demo-felhom's baked SSH key comes from an uncommitted profile, so a Tier 0 reference box is not reproducible from `main`. | `--profile` path resolves inside the repo tree **and** `git status --porcelain` is empty for it. |
|
||||
| 9 | **The artifact must be reproducible.** Spike 1 R-144: `nested-probe`'s profile no longer exists anywhere. | Manifest records the committed profile path + repo commit; both resolve. |
|
||||
|
||||
**Is "no `answer.toml`" achievable?** Yes, and it is cheaper than the alternative. `prepare-iso` is what
|
||||
creates `/answer.toml` and `/auto-installer-mode.toml`; **an image that simply is not passed through
|
||||
`prepare-iso` has neither.** The public image then needs only the Felhom branding half of
|
||||
`iso-repack.sh`. What it costs: the `--fetch-from`/`--on-first-boot` machinery becomes unusable on this
|
||||
image, so any future automated delivery would need Q2's `.deb` route (§3) — which is untested.
|
||||
|
||||
---
|
||||
|
||||
## 7. What is still unknown, and what it would take
|
||||
|
||||
| # | Unknown | What it takes |
|
||||
|---|---|---|
|
||||
| 1 | **Q2's `.deb` mechanism is unproven** (§3) — and it is now the *only* candidate delivery path for a public image. Two sub-questions: does `prepare-iso`/`iso-repack` preserve an added `/proxmox/packages/*.deb`, and can a postinst running in the installer chroot usefully install a unit? | Build a trivial `.deb` with a marker postinst, add it to the ISO's package dir, run an interactive install, check for the marker. ~45 min. **Do this before any spec relies on §3.** |
|
||||
| 2 | **Q3: the real stub at `before-network`** (§4) — still argued, not observed. Now scoped to operator-built images only. | ~40 min, as described in §4. |
|
||||
| 3 | **Whether the graphical installer behaves identically to the TUI.** §2.2 measured the **Terminal UI** path. `proxinstall` has *zero* first-boot references (strictly less capability than the TUI's one), so the negative is a fortiori — but it is inference, not measurement. | Repeat §2.2 via the Graphical entry. ~30 min. |
|
||||
| 4 | **Whether an image never passed through `prepare-iso` still boots and brands correctly** (§6, constraint 1). `iso-repack.sh:100-106` currently **refuses** an ISO without `auto-installer-mode.toml` (`exit 10`), so this is a real code change, not just an omission. | Build one and boot it. Note this is the one constraint in §6 that today's tooling actively blocks. |
|
||||
| 5 | Spike 1's two open items — multi-match determinism, and whether the installer excludes its own boot medium. | Unchanged. |
|
||||
|
||||
---
|
||||
|
||||
## 8. R-rows opened
|
||||
|
||||
Continuing from R-152 (Spike 2).
|
||||
|
||||
| R | Severity | Finding |
|
||||
|---|---|---|
|
||||
| **R-153** | MEDIUM | **A working gate is red and nobody runs it.** `hub/internal/web/configs.go:28` has `hostInstallVersion = "1.19.0"` against `SCRIPT_VERSION = "1.22.0"`, and `scripts/hostinstall_gates.py` correctly detects it and exits 1. The comment at `configs.go:26` says drift "is now gated" — true of the gate's existence, false of its enforcement, because no pipeline invokes it. The customer page's install-command generator therefore targets a flag surface three minor versions stale. |
|
||||
| **R-154** | MEDIUM | **`[first-boot]` is automated-install-only, and nothing in the Felhom tree says so.** Measured (§2) and cited (§2.3). `answer.toml.tmpl:26-28` documents the hook as the day-0 delivery mechanism without recording that it is unavailable the moment anyone boots the Graphical or Terminal UI entry — which the same ISO offers. Any image that ships both paths and relies on `[first-boot]` silently delivers nothing on one of them. |
|
||||
| **R-155** | LOW | **`iso-repack.sh:100-106` blocks the cleanest public-image posture.** It refuses (`exit 10`) any ISO lacking `auto-installer-mode.toml`, on the sound reasoning that a single "Felhom telepítés" entry must not boot a manual installer. But §6's constraint 1 — ship no `answer.toml` at all — produces exactly such an ISO deliberately. The guard needs to distinguish "prepared, single-entry" from "unprepared, interactive-by-design" rather than treating the second as the first's failure mode. |
|
||||
|
||||
---
|
||||
|
||||
## 9. Teardown
|
||||
|
||||
All three layers, verified positively.
|
||||
|
||||
**Layer 1 — demo-hp.**
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| QEMU processes | `0` |
|
||||
| `/mnt/nvme-1tb/spike3/` | removed |
|
||||
| `/tmp/paia`, `/mnt/s3a`, loop devices | absent / removed / `0` |
|
||||
| `/mnt/nvme-1tb/` contents | `dump e2d-images felhom-data images lost+found` — the pre-spike set |
|
||||
| disk usage | **6.6 G — identical to pre-spike** |
|
||||
| `drill-r50` (VM 300) | `stopped`, untouched |
|
||||
| guest 9201 | `running`, untouched |
|
||||
| `/etc/pve/storage.cfg` | **4 storages, unchanged**; none registered |
|
||||
| `local-lvm` | never written |
|
||||
|
||||
**Layer 2 — demo-felhom.** Nothing was placed there this session; `/tmp/paia` confirmed absent, guest
|
||||
9201 `running`, external USB drive `931.5G usb /mnt/hdd_1` still attached and mounted as found.
|
||||
|
||||
**Layer 3 — DooPlex.** Scratchpad **4.4 GB → 80 K**; the probe ISO, squashfs extract, all screendumps
|
||||
and both throwaway passwords removed (`shred -u` on the answer-file password, verified absent; the
|
||||
interactive install's console password was written only to a `0600` file on demo-hp for `sshpass` and
|
||||
deleted in the same command). `felhom-iso/out/` untouched at **17 ISOs**. No production file modified —
|
||||
no `build-felhom-iso.sh`, no `iso-repack.sh`, no profile, no `answer.toml.tmpl`, no
|
||||
`felhom-bootstrap.sh`.
|
||||
|
||||
**Hub-side disposition — nothing to dispose.** No customer, host or appliance was created: every VM ran
|
||||
under plain QEMU user-mode networking with no route to the hub. Verified by fetching the customer list
|
||||
and searching it — **0** spike-named objects, the same single pre-existing customer. **R-131 gains no
|
||||
row.**
|
||||
|
||||
**Published artifacts: none.**
|
||||
@@ -1,3 +1,36 @@
|
||||
## SPIKE 3 (no code change) — `[first-boot]` does NOT fire on an interactive install (2026-07-31)
|
||||
|
||||
**No script, profile or build file was modified.** Findings:
|
||||
`documentation/audits/SPIKE-universal-iso-3-2026-07-31.md`. Opens R-153..R-155.
|
||||
|
||||
**MEASURED, with a control from the same image.** One probe ISO (15 GRUB entries: Automated,
|
||||
Graphical, Terminal UI). Automated entry -> the first-boot hook fires: ttyS0 marker, marker file,
|
||||
`/var/lib/proxmox-first-boot/proxmox-first-boot` (0700) and the activation symlink all present.
|
||||
Terminal-UI entry, normal manual install -> **every one of those is absent, and the
|
||||
`proxmox-first-boot` PACKAGE is not installed at all** (`dpkg-query: no packages found`). A
|
||||
whole-filesystem grep for the marker returns nothing.
|
||||
|
||||
Mechanism, cited: `Config.pm:118` defaults `first_boot.enabled => 0` and `set_first_boot_opt` is never
|
||||
called in the Perl tree; `Install.pm:746` returns early unless the flag is set; `Install.pm:1360` skips
|
||||
the package itself. `proxinstall` (the graphical installer) contains **zero** occurrences of
|
||||
`first-boot`. **`[first-boot]` is an automated-installer feature — unavailable on every interactive
|
||||
path, by construction.** R-154.
|
||||
|
||||
**A delivery mechanism DOES exist, and it is untested.** `Install.pm:1343-1372` unpacks EVERY `.deb`
|
||||
in the ISO's `/proxmox/packages/` into the target on every install path (fixed skip-list of known
|
||||
names), then `dpkg --configure -a` runs postinsts (`:1378`). That is how PVE ships first-boot itself.
|
||||
Read from source, NOT measured — proving it is the top follow-up.
|
||||
|
||||
Also: the interactive installer shows the target disk and a summary listing `Bootdisk(s) /dev/sda`
|
||||
before erasing — the operator's ruling needs no Felhom work for that warning.
|
||||
|
||||
**Incidental: a working gate is red and nobody runs it.** `hub/internal/web/configs.go:28` has
|
||||
`hostInstallVersion = "1.19.0"` vs `SCRIPT_VERSION = "1.22.0"`; `scripts/hostinstall_gates.py` detects
|
||||
it and exits 1. The comment claiming drift "is now gated" is true of the gate and false of its
|
||||
enforcement. R-153.
|
||||
|
||||
Q3 (the real stub at `before-network`) was NOT reached and is recorded as not reached.
|
||||
|
||||
## SPIKE 2 (no code change) — locked root closes the PVE web UI; before-network gives a measured zero window (2026-07-31)
|
||||
|
||||
**No script, profile or build file was modified.** Findings:
|
||||
|
||||
Reference in New Issue
Block a user