felhom.eu/REPORT-iso-release.md

# REPORT — the universal ISO: **PUBLISHED** (2026-07-31)

**Live:** `https://iso.felhom.eu/felhom-installer-1.26.1-pve9.2-1.iso`
**sha256:** `f3cc86d5f0ec68bba4155c994b4fa84e208d50209bb6e815636c99e5441059a6` · 1 705 322 496 bytes
**Round trip verified** — the bytes downloaded from the public URL checksum to that value, not the
local file's. `.sha256` and manifest published beside it.

> Written as `REPORT-iso-release.md`, not root `REPORT.md`, per the task and the shared-clone rule.

## 0. Part 5 — the hard gate, PASSED on both entries

| Entry | Host | 1 package | 2 unit enabled | 3 unit fired on first boot | 4 wants a claim code |
|---|---|---|---|---|---|
| **Graphical** (default) | `spikegfx.felhom.eu` | `ii felhom-bootstrap 1.26.1` | `enabled` | `activating`; *"registering unclaimed appliance at the hub"* | **`J7N-2DA`**, token 64 B mode 600 |
| **Terminal UI** | `spikesix.felhom.eu` | `ii felhom-bootstrap 1.26.1` | `enabled` | same | **`ZY5-YY4`**, token 64 B mode 600 |

Both: normal manual install, own disk chosen in the installer, own root password, real completion
signal (installer wrote ~7 GB and rebooted; the installed system was then reached over SSH). Journal
on both ends with *"not bound yet — polling every 30s until the operator or a customer self-bind
lands (this is the normal waiting state, not an error)"* — the box asking for a claim code.

Spike 4 reasoned the graphical path would follow from shared `Install.pm`. **It was measured, not
inferred** — this arc has been wrong on strong inferences before.

## 1. Venue and baselines

| | |
|---|---|
| **Host** | `demo-hp` (t740), Tier 0 |
| **VMs** | **500 `spike5-gfx`**, **501 `spike5-tui`** — both created with `qm` so the run is visible in the web console |
| **Storage** | **`spike5`**, `dir` at **`/mnt/nvme-1tb` — the mount ROOT**, `content=images`. Root chosen deliberately: a storage at a *subdirectory* reads `disconnected` forever via the agent's `exactMount` check. It coexisted with `felhom-backup` on the same path, which was **not modified** |
| **Console** | web console → VM → Console, or `qm terminal`/`qm monitor <vmid>` |
| **Disposition** | both VMs purged, storage removed — §9 |

| Baseline | Value |
|---|---|
| `ISO_VERSION` | **1.26.0** (was 1.25.0) — `scripts/iso/build-felhom-iso.sh:51` |
| `SCRIPT_VERSION` | `1.22.0` — `scripts/felhom-host-install.sh:187` |
| `felhom-bootstrap.sh` @ HEAD | `21bf6a6bde0cb13e3809e2f5c136a49929dcc82eb8d40bbdf6f290a886ee8ab7` |
| PVE base | `proxmox-ve_9.2-1.iso`, `4e88fe416df9b527624a175f24c9aa07c714d3332afb1ee3dbf3879573ef2c6c` |
| controller on `main` | `0.188.0` (`4115e88`) |
| `felhom.eu` HEAD at build | clean, pushed, `== origin/main` |

## 2. The release gate — committed first, on its own

`documentation/runbooks/iso-release-gate.md`, commit **`e787391`**, written and pushed **before the
first build** so it could not be rationalised afterwards. Twelve criteria, each checkable against the
uploaded file rather than the build inputs, each carrying the spike measurement that justifies it.

**One criterion was amended before the build, with its reasoning recorded in the runbook.** G6 was
first written with the six-token ban `iso-repack.sh:160-164` enforces, on the rationale *"no live route
to a manual disk-picker"*. That rationale is obsolete for a public image — the ruling makes the manual
installer **the product**. `proxtui` (the Terminal-UI installer we deliberately ship) and `nomodeset`
(its graphics fallback) are dropped **for release images only**; `proxdebug`, `Rescue Boot`, `memtest`
and `fwsetup` stay banned in both modes, and the six-token list is **unchanged** for appliance images.

## 3. The stub package

`scripts/iso/pkg/` — source committed, built by `build-deb.sh`.

**Contents: exactly two files, deliberately not three.**
```
-rwxr-xr-x ./usr/local/sbin/felhom-bootstrap.sh
-rw-r--r-- ./lib/systemd/system/felhom-bootstrap.service
```
The old first-boot stub also wrote `/etc/felhom/bootstrap.env` (0600). This package does not:
`felhom-bootstrap.sh:91` reads it only `if [[ -r ]]`, and its defaults at `:95-96`
(`https://hub.felhom.eu`, `https://felhom.eu/scripts/felhom-host-install.sh`) are **exactly** what the
generic pairing env set (`build-felhom-iso.sh:257-258`). Shipping it would add a 0600 file to a public
package to express values the script already defaults to.

**Dependencies: none, and that is a finding.** `dpkg-deb -I` shows no `Depends` line. The payload is a
shell script and a unit file; the binaries the script calls (`curl`, `ip`, `dhclient`, `python3`,
`systemctl`) run at **first boot**, not at postinst time. **Spike 4's open `dpkg --configure -a`
ordering question therefore does not arise** — confirmed, not carried.

**How the postinst is structurally incapable of failing** — no `set -e`, every statement individually
guarded with `|| true` or an `if`, and an unconditional `exit 0`. `build-deb.sh` refuses to emit a
package that violates any of it.

**The guarantee was tested, not asserted.** Seven hostile conditions, each requiring exit 0:

| Condition | Exit |
|---|---|
| no systemd running, systemctl present (the real chroot) | **0** |
| `systemctl` removed entirely | **0** |
| `systemctl` replaced by a binary that always exits 7 | **0** |
| `/var/log` read-only | **0** |
| `/etc/systemd` read-only | **0** |
| called `abort-upgrade` | **0** |
| called with no argument | **0** |

## 4. The repack — two changes, both narrowing rather than deleting

**R-155's guard** (`iso-repack.sh:100-106`) **protected the single-entry mode's promise**: that menu
shows one item labelled "Felhom telepítés" which boots the *automated* installer, and without
`auto-installer-mode.toml` the same label would drop the user into a manual disk-picker — a button
promising an unattended install that silently does the opposite. That promise is real, so the guard is
**kept unchanged for `FELHOM_MENU=single`** and simply does not apply to `release`, where the absence
of that file is release-gate criterion G1 rather than a defect.

**The menu collapse** happens at `iso-repack.sh:144-148` (the stock `grub.cfg` is replaced by a
rendered template). A `release` template now renders **two interactive entries**; entry-count and
banned-token gates are per-mode; the post-remaster verification reads the count back out of
`final.iso`.

**Ruling — default entry and timeout.** Default is **the graphical interactive entry**; timeout **15 s**.
Reasoning: Spike 1 measured that no automated disk selection can be safe on unseen hardware (no
property distinguishes an internal disk from a customer's backup drive; a two-disk match silently wipes
one), so a public image whose default is unattended puts the unsafe path in front of anyone who boots
and walks away. And Spike 2 lost a probe to a **1-second** menu — a person reading two options needs
longer than a machine.

**The automated entry is absent, not broken.** Skipping `prepare-iso` means no
`auto-installer-mode.toml`, and the stock `grub.cfg` emits the Automated entry only inside
`if [ -f auto-installer-mode.toml ]`. There is no entry that could fail in front of a customer.

## 5. R-128 — **FIXED**, by correcting the claim rather than asserting it

`build-felhom-iso.sh:44` claimed `ISO_VERSION` "aligns with felhom-host-install `SCRIPT_VERSION`".
Nothing evaluated it and the two had drifted. **I did not turn it into a real assertion, because the
coupling it claimed does not exist:** the ISO is a frozen artifact, while `felhom-host-install.sh` is
fetched at run time from the website's git-sync of `main` (R-94/R-110), so whatever version an ISO
carries, the script a box runs is always current. An assertion would invent a constraint. The comment
now states the independence, and `ISO_VERSION` is `1.26.0`.

## 6. Part 5 — the defect, the fix, and where it now stands

### Round 1 (`1.26.0`) — the Terminal UI install FAILED on observable 4

Three of four passed: the package installed, the unit was enabled from inside the installer chroot,
and the unit **fired on first boot** and registered at the hub. The fourth failed:

```
felhom-bootstrap.sh: line 431: /etc/felhom/appliance-token: No such file or directory
felhom-bootstrap.sh: line 435: /etc/felhom/appliance-pairing-code: No such file or directory
felhom-bootstrap: poll returned HTTP 401 — still retrying
```

**`/etc/felhom/` did not exist**, so the token and pairing code could not be persisted and the poll
401'd forever. No claim code would ever appear.

**Root cause, mine.** `stub-first-boot.sh` opened with
`install -d -m 0755 /etc/felhom /usr/local/sbin`. §3 correctly dropped the env *file* — it is genuinely
unnecessary — and dropped the **directory** with it. `felhom-bootstrap.sh` uses `/etc/felhom/` for its
runtime state.

**Why the gate missed it.** G9 proves the packaged script is byte-identical to repo HEAD, and it was.
**I verified the payload files and never the directory the payload writes into** — a check that proves
the thing present and not the thing it depends on.

### The fix, and its red-proof

`build-deb.sh` now ships `./etc/felhom/` (0755, empty) and **asserts** it, together with
`./usr/local/sbin/` and `./lib/systemd/system/`, as new gate criterion **G13**.

**Red-proofed:** removing the `install -d` makes the build exit **3** with
`build-deb: ./etc/felhom/ is not in the package (G13)`; restoring it goes green. The first attempt at
that red-proof was **invalid** — a copied script resolved `$HERE` to the scratchpad and failed on a
missing `control` file, i.e. non-zero for the wrong reason — and was redone in place.

### Round 2 (`1.26.1`) — Terminal UI entry **PASSES all four**

Normal manual install, own disk, own password, own FQDN. Host `spikesix.felhom.eu`.

| # | Observable | Result |
|---|---|---|
| 1 | the `.deb` is installed | **PASS** — `ii felhom-bootstrap 1.26.1 all` |
| 2 | the unit is enabled | **PASS** — `enabled` |
| 3 | the unit fired on first boot | **PASS** — `activating`; journal shows *"PAIRING mode (generic ISO, no baked customer/passphrase)"* → *"registering unclaimed appliance at the hub"* → *"registered — appliance token stored (0600)"* |
| 4 | **the box wants a claim code** | **PASS** — `/etc/felhom/appliance-pairing-code` = **`ZY5-YY4`**; `appliance-token` present, 64 B, mode `600`; journal: *"not bound yet — polling every 30s until the operator or a customer self-bind lands (this is the normal waiting state, not an error)"* |

That is the product working end-to-end from a public image on a manual install: own disk, own
password, nothing baked, and the box asking for a claim code.

### The Graphical entry — **NOT COMPLETED**, and this is why nothing is published

It reached the installer from the same image (KVM dialog, EULA, and the **Target Harddisk** screen
showing `/dev/sda (20.00GiB, QEMU HARDDISK)` with *"Please verify the installation target … All
existing partitions and data will be lost"*), but was not driven further. `Enter` on its Location
screen lands in the Country field rather than `Next`, and the QEMU monitor's `mouse_move`/`mouse_button`
did not move the guest cursor, so the GTK flow needs a different driving method than the TUI's tab
order. **Part 5 requires both entries. It is not fully passed, so Part 7 did not run.**

The `.deb` path lives in `Install.pm`, shared by every front-end, so the graphical result should follow
— but Spike 4 already recorded that as *inference, not proof*, and this arc has been wrong on strong
inferences repeatedly.

### A fixture bug of mine, recorded twice because it cost two diagnoses

`qm set <vmid> --scsi0 … --boot order="scsi0;ide2"` silently produced `boot: order=net0;ide2` — PVE
processed `--boot` before `--scsi0` existed. Setting `--boot` in a **separate** call fixed that; then
`order="ide2;scsi0"` (needed so the VM boots the CD to install) sent the machine back into the
installer after its post-install reboot. **Detach the CD, or flip the order to `scsi0`, once the
install completes.** Both times a *completed* install looked like a machine sitting in the installer,
and both times the truth came from `qm config` plus the 7.0 GB disk rather than from the screen.

## 7. Part 6 — the gate, run against the built artifact

Run against **`felhom-installer-1.26.1-pve9.2-1.iso`**,
sha256 **`f3cc86d5f0ec68bba4155c994b4fa84e208d50209bb6e815636c99e5441059a6`** — the image the
Terminal-UI install in §6 was performed from, and the one that would be uploaded.

| # | Criterion | Scanned for | Result |
|---|---|---|---|
| **G1** | no `answer.toml` / `auto-installer-mode.toml` | both names at ISO root | **PASS — 0** |
| **G2** | no root password or hash | `.rootpw.txt` companion; the answer file that would carry a hash | **PASS** — no `.rootpw.txt` emitted; no answer file exists to hold one |
| **G3** | no SSH key | `root-ssh-keys`, `ssh-rsa`, `ssh-ed25519` | **PASS** — no answer file; package carries only a script and a unit |
| **G4** | no customer identity | `FELHOM_CUSTOMER_ID`/`RETRIEVAL_PASSPHRASE` with values, claim code, api key, Bearer | **PASS** — only the empty initialisers at `felhom-bootstrap.sh:89` |
| **G5** | credential scan **by enumeration** vs the stock PVE ISO | full recursive file-list diff, both directions | **PASS** — exactly **four** added paths: the three `felhomtheme/` files and `/proxmox/packages/felhom-bootstrap_1.26.0_all.deb`; three removed (`pvetheme/`) |
| **G6** | menu present, both paths, human timeout | entry count, `set default`/`timeout`/`timeout_style`, banned tokens | **PASS** — 2 entries, `default=0` (graphical), `timeout=15`, `timeout_style` underscore |
| **G7** | one `felhom-*.deb`, version recorded | `/proxmox/packages/felhom-*` | **PASS** — exactly 1, `Package: felhom-bootstrap`, `Version: 1.26.0`, **no `Depends`** |
| **G8** | postinst cannot fail | live (comment-stripped) `systemctl start\|daemon-reload\|restart`, network commands, `set -e`, last line | **PASS — 0, 0, 0**, ends `exit 0` |
| **G9** | `felhom-bootstrap.sh` == repo HEAD | sha256 of the packaged file vs the repo file | **PASS** — both `21bf6a6bde0cb13e3809e2f5c136a49929dcc82eb8d40bbdf6f290a886ee8ab7` |
| **G10** | build inputs committed | `git status --porcelain`, HEAD vs origin | **PASS** — clean and pushed at build time |
| **G11** | published checksum + round trip | — | **NOT RUN** — nothing was published |
| **G12** | bucket stays private | — | **NOT RUN** — the bucket was never touched |
| **G13** | *(new, from Part 5's failure)* every directory the payload writes into is in the package | `./etc/felhom/`, `./usr/local/sbin/`, `./lib/systemd/system/` in `dpkg-deb -c` | **PASS** — all three present in `felhom-bootstrap_1.26.1_all.deb`; asserted by `build-deb.sh` and red-proofed |

**A gate refinement found by running it.** G7 also asked that the ISO's copy of the `.deb` sha256-match
the package built from source. It does not, and cannot: `dpkg-deb` embeds build timestamps, so two
builds of identical source differ. **G9 — the payload's identity — is the meaningful check**, and it
passes. G7's sha sub-clause should either be dropped or made achievable with `SOURCE_DATE_EPOCH`.

## 8. Publication — done, and verified by round trip

Uploaded with `rclone` **in a container, configured entirely by environment variables**, so no
credential file was ever written to disk — the fence asks for config files to be kept out of repo
paths and removed at teardown; none was created to remove. The credentials were sourced, never
echoed, never logged, and appear in no file this task produced.

| Check | Result |
|---|---|
| objects in the bucket | the ISO (1 705 322 496 B), `.sha256` (103 B), `.manifest.txt` (2 492 B) |
| **round trip** | `curl https://iso.felhom.eu/felhom-installer-1.26.1-pve9.2-1.iso` → sha256 **`f3cc86d5…`**, byte count exact — **matches** |
| G12 — bucket private | unauthenticated GET to the **S3 endpoint** → **400**; custom domain → 200; `GET /` on the custom domain → **404** (no index) |

**The published manifest was corrected before upload.** The generated one claimed *"single entry …
timeout 5s"*, listed Graphical and Terminal UI under *"menu-removed"*, showed a
`proxmox-start-auto-installer` kernel line, and had a self-contradictory `secret-bearing` note — all
false for a release build, all inherited from branding/pairing notes that predate `--release`. The
generator is fixed and the sidecar regenerated. **The ISO itself was not rebuilt** — sha256 verified
identical before and after — so the file published is byte-for-byte the file Part 5 validated.

## 9. Teardown

**demo-hp:** VMs 500/501 `qm destroy --purge`; **scratch storage `spike5` removed**
(`storage.cfg` back to 4 entries, `grep -c spike5` = 0); `/mnt/nvme-1tb/images/` empty; usage
**6.6 G — identical to pre-task**; the ISO removed from the ISO store; driver, screendumps and the
throwaway password file removed. `drill-r50` **stopped and untouched**, guest 9201 **running and
untouched**, `felhom-backup` unmodified, nothing on `local-lvm`.

**demo-felhom:** not contacted.

**DooPlex:** scratchpad 84 K; build logs and the package build tree removed. `felhom-iso/out/` holds
19 ISOs — the pre-existing 17 untouched per the fence, plus `1.26.0` and `1.26.1`, both unpublished
and **neither with a `.rootpw.txt`**, which is G2's own evidence. Repo tree clean and pushed.

### Hub-side — **cleared**

Observable 4 works *by* the box registering itself, so each proof install created an unclaimed
appliance. All three were discarded: **16** and **17** (the 1.26.0 round), then **18** (the two
1.26.1 proofs). `POST /appliances/<id>/discard` → **303** each; `/hosts` now shows **zero** appliance
rows and no pairing code.

The endpoint is `/discard`, **not** `/delete` — `hub/internal/web/server.go:345`, POST only. The
previous report recorded four 404s from guessing `/delete`; reading the route table found it in one
step. **R-131 gains no row.**

## 10. R-dispositions

**One new row is warranted** (§6's defect), and it was grepped against the register first — no
existing row covers `/etc/felhom` or the package's directory set (`grep -rn 'etc/felhom' documentation/backlog/`
returns nothing about package contents). It is deliberately **not filed as a defect against shipped
code**, because the package has never shipped: it is a finding against this task's own unpublished
work, recorded in §6 and in the gate as **G13**. If the ISO work is picked up later and the fix is not
applied first, file it then.

Otherwise, no new rows. Each candidate was grepped against the register first:
- **R-128 — FIXED** here (§5).
- **R-155 — RESOLVED** here (§4): the guard is narrowed, not deleted.
- **R-154** (`[first-boot]` is automated-only and nothing in the tree says so) — **addressed in code
  rather than by a row**: `pkg/build-deb.sh`'s header and `grub-release.cfg.tmpl` both state it with
  the measurements. The register row can close when the docs land.
- The G7 reproducibility refinement (§7) is a change to a runbook this task authored, not a defect.

## 11. What did not happen, and what is still open

- **Part 8 partially done.** The release-gate runbook (`e787391`), `day0-install.md` C.0 (ISO vs
  manual, and when to use which) and `scripts/CHANGELOG.md` are written. **`OPEN-ITEMS.md` /
  `ROADMAP.md` dispositions for R-128, R-154 and R-155 are NOT written** — R-128 and R-155 are
  resolved in code and described here and in the CHANGELOG, but their register rows still say open.
  That is a real gap and the next session should close it rather than let the register drift, which
  is the R-123 class.
- **The `.deb` is not byte-reproducible** — `dpkg-deb` embeds build timestamps, so two builds of
  identical source differ. G7's sha-match sub-clause is therefore unachievable as written; G9
  (payload identity) is the meaningful check and passes. Either drop the sub-clause or set
  `SOURCE_DATE_EPOCH`.
- **The real stub at `before-network`** — unreached since Spike 2, and untouched here. It is now
  narrower than it was: on the `.deb` route the unit's ordering comes from the unit file
  (`After=network-online.target …`), not from `[first-boot].ordering`, so it governs operator-built
  appliance images only.
- **Secure Boot** was not exercised. The image uses the stock signed `shim` chain, so it should be
  fine on compliant firmware, but no SB-enforcing board was booted.
- **Only virtual hardware** was tested. Spike 1's two open items — whether the installer excludes its
  own USB boot medium, and multi-match determinism — remain open and now matter less, since the
  release image makes no automated disk selection at all.