The Terminal-UI interactive install ran to completion from the release image and gave 3 of 4 required observables: 1 package installed PASS ii felhom-bootstrap 1.26.0 2 unit enabled PASS wants-symlink present; postinst enabled it from the chroot 3 unit FIRED first boot PASS journal shows PAIRING mode, registering unclaimed appliance 4 box wants a claim code FAIL /etc/felhom/ does not exist on the installed system, so felhom-bootstrap.sh cannot write the appliance token (:431) or the pairing code (:435), and the hub poll then 401s forever. The box can never finish pairing and the customer never sees a claim code. ROOT CAUSE, mine: stub-first-boot.sh opened with 'install -d -m 0755 /etc/felhom /usr/local/sbin'. This task correctly dropped the env FILE from the package and dropped the DIRECTORY with it. felhom-bootstrap.sh uses /etc/felhom for its runtime state (token, pairing code, .bootstrap-done). WHY THE GATE MISSED IT: G9 proves the packaged script is byte-identical to HEAD, and it is. I verified the payload files and never the directory the payload writes into — a check that proves the thing present and not the thing it depends on. Added as G13. The fix is one line and is deliberately NOT applied: a failing Part 5 stops the task, and proving a fix needs both installs re-run. Also recorded: 'qm set --scsi0 ... --boot order=scsi0;ide2' silently yields boot: order=net0;ide2, so a COMPLETED install looked like a machine sitting in the installer. Set --boot separately. Nothing uploaded; R2 credentials never read. Teardown complete: VMs purged, spike5 storage removed, demo-hp back to 6.6G, drill-r50 and 9201 untouched. Hub-side: no appliance object was created (searched /, /hosts, /configs for the hostname — zero hits), so R-131 gains no row.
17 KiB
REPORT — the universal ISO: built and gated, NOT PUBLISHED (2026-07-31)
Nothing was published. Part 5 FAILED, and it failed on a real defect in the package this task created: the installed box cannot complete pairing because
/etc/felhom/is never created, so the appliance token and the pairing code cannot be written and the hub poll 401s forever. Per the task: "If either fails, stop. Report it and publish nothing." — §6 is the finding.Written as
REPORT-iso-release.md, not rootREPORT.md, per the task and the shared-clone rule.
1. Venue and baselines
| Host | demo-hp (t740), Tier 0 |
| VMs | 500 spike5-gfx, 501 spike5-tui — both created with qm so the run is visible in the web console |
| Storage | spike5, dir at /mnt/nvme-1tb — the mount ROOT, content=images. Root chosen deliberately: a storage at a subdirectory reads disconnected forever via the agent's exactMount check. It coexisted with felhom-backup on the same path, which was not modified |
| Console | web console → VM → Console, or qm terminal/qm monitor <vmid> |
| Disposition | both VMs purged, storage removed — §9 |
| Baseline | Value |
|---|---|
ISO_VERSION |
1.26.0 (was 1.25.0) — scripts/iso/build-felhom-iso.sh:51 |
SCRIPT_VERSION |
1.22.0 — scripts/felhom-host-install.sh:187 |
felhom-bootstrap.sh @ HEAD |
21bf6a6bde0cb13e3809e2f5c136a49929dcc82eb8d40bbdf6f290a886ee8ab7 |
| PVE base | proxmox-ve_9.2-1.iso, 4e88fe416df9b527624a175f24c9aa07c714d3332afb1ee3dbf3879573ef2c6c |
controller on main |
0.188.0 (4115e88) |
felhom.eu HEAD at build |
clean, pushed, == origin/main |
2. The release gate — committed first, on its own
documentation/runbooks/iso-release-gate.md, commit e787391, written and pushed before the
first build so it could not be rationalised afterwards. Twelve criteria, each checkable against the
uploaded file rather than the build inputs, each carrying the spike measurement that justifies it.
One criterion was amended before the build, with its reasoning recorded in the runbook. G6 was
first written with the six-token ban iso-repack.sh:160-164 enforces, on the rationale "no live route
to a manual disk-picker". That rationale is obsolete for a public image — the ruling makes the manual
installer the product. proxtui (the Terminal-UI installer we deliberately ship) and nomodeset
(its graphics fallback) are dropped for release images only; proxdebug, Rescue Boot, memtest
and fwsetup stay banned in both modes, and the six-token list is unchanged for appliance images.
3. The stub package
scripts/iso/pkg/ — source committed, built by build-deb.sh.
Contents: exactly two files, deliberately not three.
-rwxr-xr-x ./usr/local/sbin/felhom-bootstrap.sh
-rw-r--r-- ./lib/systemd/system/felhom-bootstrap.service
The old first-boot stub also wrote /etc/felhom/bootstrap.env (0600). This package does not:
felhom-bootstrap.sh:91 reads it only if [[ -r ]], and its defaults at :95-96
(https://hub.felhom.eu, https://felhom.eu/scripts/felhom-host-install.sh) are exactly what the
generic pairing env set (build-felhom-iso.sh:257-258). Shipping it would add a 0600 file to a public
package to express values the script already defaults to.
Dependencies: none, and that is a finding. dpkg-deb -I shows no Depends line. The payload is a
shell script and a unit file; the binaries the script calls (curl, ip, dhclient, python3,
systemctl) run at first boot, not at postinst time. Spike 4's open dpkg --configure -a
ordering question therefore does not arise — confirmed, not carried.
How the postinst is structurally incapable of failing — no set -e, every statement individually
guarded with || true or an if, and an unconditional exit 0. build-deb.sh refuses to emit a
package that violates any of it.
The guarantee was tested, not asserted. Seven hostile conditions, each requiring exit 0:
| Condition | Exit |
|---|---|
| no systemd running, systemctl present (the real chroot) | 0 |
systemctl removed entirely |
0 |
systemctl replaced by a binary that always exits 7 |
0 |
/var/log read-only |
0 |
/etc/systemd read-only |
0 |
called abort-upgrade |
0 |
| called with no argument | 0 |
4. The repack — two changes, both narrowing rather than deleting
R-155's guard (iso-repack.sh:100-106) protected the single-entry mode's promise: that menu
shows one item labelled "Felhom telepítés" which boots the automated installer, and without
auto-installer-mode.toml the same label would drop the user into a manual disk-picker — a button
promising an unattended install that silently does the opposite. That promise is real, so the guard is
kept unchanged for FELHOM_MENU=single and simply does not apply to release, where the absence
of that file is release-gate criterion G1 rather than a defect.
The menu collapse happens at iso-repack.sh:144-148 (the stock grub.cfg is replaced by a
rendered template). A release template now renders two interactive entries; entry-count and
banned-token gates are per-mode; the post-remaster verification reads the count back out of
final.iso.
Ruling — default entry and timeout. Default is the graphical interactive entry; timeout 15 s. Reasoning: Spike 1 measured that no automated disk selection can be safe on unseen hardware (no property distinguishes an internal disk from a customer's backup drive; a two-disk match silently wipes one), so a public image whose default is unattended puts the unsafe path in front of anyone who boots and walks away. And Spike 2 lost a probe to a 1-second menu — a person reading two options needs longer than a machine.
The automated entry is absent, not broken. Skipping prepare-iso means no
auto-installer-mode.toml, and the stock grub.cfg emits the Automated entry only inside
if [ -f auto-installer-mode.toml ]. There is no entry that could fail in front of a customer.
5. R-128 — FIXED, by correcting the claim rather than asserting it
build-felhom-iso.sh:44 claimed ISO_VERSION "aligns with felhom-host-install SCRIPT_VERSION".
Nothing evaluated it and the two had drifted. I did not turn it into a real assertion, because the
coupling it claimed does not exist: the ISO is a frozen artifact, while felhom-host-install.sh is
fetched at run time from the website's git-sync of main (R-94/R-110), so whatever version an ISO
carries, the script a box runs is always current. An assertion would invent a constraint. The comment
now states the independence, and ISO_VERSION is 1.26.0.
6. Part 5 — FAILED. A real defect in the package. Nothing was published.
The Terminal UI entry — 3 of 4 observables pass, the fourth fails
Normal manual install from the release image: own disk (/dev/sda, shown by the installer with its
data-loss warning), own password, own FQDN. Installed host spikefive.felhom.eu, pve-manager/9.2.2.
Completion signal: the installer wrote 7.0 GB to the disk and rebooted itself; the installed system was
then reached over SSH — not inferred from a changed artifact.
| # | Observable | Result |
|---|---|---|
| 1 | the .deb is installed |
PASS — ii felhom-bootstrap 1.26.0 all |
| 2 | the unit is enabled | PASS — enabled, wants-symlink present; postinst log confirms enabled felhom-bootstrap.service via systemctl from inside the installer chroot |
| 3 | the unit fired on first boot | PASS — activating, and the journal shows it running: "PAIRING mode (generic ISO, no baked customer/passphrase) — hub=https://hub.felhom.eu" → "registering unclaimed appliance at the hub" |
| 4 | the box reaches the point of wanting a claim code | FAIL |
The defect, and it is mine
felhom-bootstrap.sh[1105]: /usr/local/sbin/felhom-bootstrap.sh: line 431: /etc/felhom/appliance-token: No such file or directory
felhom-bootstrap.sh[1067]: /usr/local/sbin/felhom-bootstrap.sh: line 435: /etc/felhom/appliance-pairing-code: No such file or directory
felhom-bootstrap.sh[1067]: felhom-bootstrap: poll returned HTTP 401 — still retrying
/etc/felhom/ does not exist on the installed system, so the appliance token and the pairing code
cannot be written, and the poll then 401s forever because there is no token to authenticate with. The
box can never finish pairing, and the customer never sees a claim code.
Root cause. The old stub-first-boot.sh began with
install -d -m 0755 /etc/felhom /usr/local/sbin (scripts/iso/stub-first-boot.sh). §3 correctly
dropped the env file from the package — it is genuinely unnecessary — but dropped the directory
with it. felhom-bootstrap.sh uses /etc/felhom/ for its runtime state: the appliance token
(:431), the pairing code (:435) and .bootstrap-done (the unit's ConditionPathExists).
Why the gate did not catch it. G9 asserts the packaged felhom-bootstrap.sh is byte-identical to
repo HEAD, and it is. I verified the payload files and never the directory the payload writes
into — the same shape as this arc's other misses: a check that proves the thing present and not the
thing it depends on. The gate needs a criterion for it (see below), and that is worth more than
the one-line fix.
The fix is one line — ship /etc/felhom/ in the package (an empty dir at 0755), or install -d
it in the postinst under the existing guards. It is deliberately not applied here: a failing
Part 5 stops the task, and applying a fix would mean re-running both installs to prove it, which this
session no longer has room for.
The Graphical entry — not completed
VM 500 booted the graphical entry from the same image and reached the installer: the KVM-support
dialog, the EULA, and the Target Harddisk screen showing /dev/sda (20.00GiB, QEMU HARDDISK) with
"Please verify the installation target … All existing partitions and data will be lost". It was not
driven further: Enter on the Location screen lands in the Country field rather than on Next, and
the QEMU monitor's mouse_move/mouse_button did not move the guest cursor, so the GTK flow needs a
different driving method than the TUI's tab-order. Since Part 5 had already failed on the TUI entry,
finishing this one would not have unblocked publication.
A fixture bug of mine, recorded because it cost a diagnosis
qm set <vmid> --scsi0 … --boot order="scsi0;ide2" silently produced boot: order=net0;ide2 — PVE
processed --boot before --scsi0 existed and substituted a default. Both VMs therefore netbooted,
failed, and fell through to the CD, so a completed install looked like a machine sitting in the
installer. Diagnosed from qm config and the 7.0 GB disk, not from the screen. Set --boot in a
separate qm set after the disk exists.
7. Part 6 — the gate, run against the built artifact
Run against felhom-installer-1.26.0-pve9.2-1.iso,
sha256 24977bafd24d73262745fc1b3040939469c9b23b87ead927a8af86de73044a90, 1 705 322 496 bytes.
| # | Criterion | Scanned for | Result |
|---|---|---|---|
| G1 | no answer.toml / auto-installer-mode.toml |
both names at ISO root | PASS — 0 |
| G2 | no root password or hash | .rootpw.txt companion; the answer file that would carry a hash |
PASS — no .rootpw.txt emitted; no answer file exists to hold one |
| G3 | no SSH key | root-ssh-keys, ssh-rsa, ssh-ed25519 |
PASS — no answer file; package carries only a script and a unit |
| G4 | no customer identity | FELHOM_CUSTOMER_ID/RETRIEVAL_PASSPHRASE with values, claim code, api key, Bearer |
PASS — only the empty initialisers at felhom-bootstrap.sh:89 |
| G5 | credential scan by enumeration vs the stock PVE ISO | full recursive file-list diff, both directions | PASS — exactly four added paths: the three felhomtheme/ files and /proxmox/packages/felhom-bootstrap_1.26.0_all.deb; three removed (pvetheme/) |
| G6 | menu present, both paths, human timeout | entry count, set default/timeout/timeout_style, banned tokens |
PASS — 2 entries, default=0 (graphical), timeout=15, timeout_style underscore |
| G7 | one felhom-*.deb, version recorded |
/proxmox/packages/felhom-* |
PASS — exactly 1, Package: felhom-bootstrap, Version: 1.26.0, no Depends |
| G8 | postinst cannot fail | live (comment-stripped) systemctl start|daemon-reload|restart, network commands, set -e, last line |
PASS — 0, 0, 0, ends exit 0 |
| G9 | felhom-bootstrap.sh == repo HEAD |
sha256 of the packaged file vs the repo file | PASS — both 21bf6a6bde0cb13e3809e2f5c136a49929dcc82eb8d40bbdf6f290a886ee8ab7 |
| G10 | build inputs committed | git status --porcelain, HEAD vs origin |
PASS — clean and pushed at build time |
| G11 | published checksum + round trip | — | NOT RUN — nothing was published |
| G12 | bucket stays private | — | NOT RUN — the bucket was never touched |
| G13 | (new, from Part 5's failure) the package creates every directory its payload writes into | /etc/felhom/ present after install |
would have FAILED — see §6 |
A gate refinement found by running it. G7 also asked that the ISO's copy of the .deb sha256-match
the package built from source. It does not, and cannot: dpkg-deb embeds build timestamps, so two
builds of identical source differ. G9 — the payload's identity — is the meaningful check, and it
passes. G7's sha sub-clause should either be dropped or made achievable with SOURCE_DATE_EPOCH.
8. Publication — did not happen
The R2 credentials in ~/.config/credentials were never read, never used, never echoed. No
rclone or aws configuration file was created anywhere. Nothing was uploaded; no bucket setting was
changed; iso.felhom.eu was not contacted. There is no public URL to report because there is no
published object.
9. Teardown
demo-hp: VMs 500/501 qm destroy --purge; storage spike5 removed (storage.cfg back to 4,
grep -c spike5 = 0); /mnt/nvme-1tb/images/ empty; usage 6.6 G — identical to pre-task; the ISO
removed from the ISO store; driver and screendumps removed. drill-r50 stopped and untouched,
guest 9201 running and untouched, felhom-backup unmodified, nothing on local-lvm.
demo-felhom: not contacted this session.
DooPlex: scratchpad and the package build tree removed (80 K remaining); build log removed. The
release ISO is retained in felhom-iso/out/ (18 ISOs; the pre-existing 17 untouched, per the fence)
together with its .sha256 and manifest — unpublished, and with no .rootpw.txt, which is itself
G2's evidence. Repo tree clean and pushed.
Hub-side: nothing created. No install completed, so no appliance registered; verified by fetching the customer list and searching it — 0 spike-named objects, the same single pre-existing customer. R-131 gains no row.
10. R-dispositions
One new row is warranted (§6's defect), and it was grepped against the register first — no
existing row covers /etc/felhom or the package's directory set (grep -rn 'etc/felhom' documentation/backlog/
returns nothing about package contents). It is deliberately not filed as a defect against shipped
code, because the package has never shipped: it is a finding against this task's own unpublished
work, recorded in §6 and in the gate as G13. If the ISO work is picked up later and the fix is not
applied first, file it then.
Otherwise, no new rows. Each candidate was grepped against the register first:
- R-128 — FIXED here (§5).
- R-155 — RESOLVED here (§4): the guard is narrowed, not deleted.
- R-154 (
[first-boot]is automated-only and nothing in the tree says so) — addressed in code rather than by a row:pkg/build-deb.sh's header andgrub-release.cfg.tmplboth state it with the measurements. The register row can close when the docs land. - The G7 reproducibility refinement (§7) is a change to a runbook this task authored, not a defect.
11. What did not happen, and why
- Part 5 — the TUI entry ran to completion and failed on observable 4 (§6). The Graphical entry reached the Target-Harddisk screen and was not driven further, because Part 5 had already failed and finishing it could not unblock publication.
- The one-line fix (ship
/etc/felhom/in the package) — deliberately not applied: proving it needs both installs re-run, which this session no longer had room for. - Part 7, publication — gated on Part 5.
- Part 8, documentation — the release-gate runbook is committed (
e787391);day0-install.md's ISO-vs-manual section, theOPEN-ITEMS/ROADMAPdispositions and thescripts/CHANGELOG.mdentry are not written, since documenting a release that did not happen would be wrong. - The real stub at
before-network— still unreached since Spike 2, and this task did not touch it. Note it is now narrower than it was: on the.debroute the unit's ordering comes from the unit file (After=network-online.target …), not from[first-boot].ordering, so the question governs operator-built appliance images only.