felhom.eu/REPORT-rehearsal-2026-08-09.md

# REPORT — BYO reinstall rehearsal, 2026-08-09 (session report)

*Written as `REPORT-<topic>.md` rather than `REPORT.md` per the repo's parallel-session rule.*

## The answer to the runbook's question, first

**Not answerable today, and the reason is the answer.** *"If a person's machine is set up again from
scratch, does their data come back through the product, in one sitting, with no command line inside
the machine?"* — the walk never reached the data. **A machine wiped today cannot be set up again at
all**: the install fails at step 5 of 8, on every host, because the agent version the hub vouches was
never given a git tag (**R-273**). The step before that had already refused, because removing Felhom
leaves behind the very thing that blocks reinstalling it (**R-272**).

The data was never at risk and is not now: the planted dataset sits in off-site snapshot `41c830db`
with its fingerprints recorded, plus a PBS snapshot and four local vzdump archives.

## Venue

**demo-hp (t740)**, operator-approved at STOP 1. It won every fidelity criterion that distinguishes
the two demo boxes: three customer apps against one, a registered storage path against none, and a
Secure-Boot `shim` install — the customer shape — against demo-felhom's SB-off `mkimage` firmware
workaround. `drill-r50` (VM 300) was verified not at risk before proceeding: it is outside the
`felhom` pool, on `local-lvm`, and `--uninstall` removes no storage and no non-pool guest.

## Findings, ranked by what they cost the person in front of you

**1 — stops the visit**
- **R-273** · the vouched agent (0.128.0) has no git tag; every install and reinstall dies at 5/8.
- **R-272** · Felhom's uninstall restarts its own dnsmasq unconstrained; it grabs `:53`; the next
  install refuses and appears to blame the owner's network.

**2 — costs the visit**
- **R-274** · a local golden is adopted with no version and no checksum check. The copy on demo-hp is
  controller **0.192.0** against a vouched **0.210.0** — and below 0.200.0, where the recovery screen
  the customer needs actually shipped.
- **R-276** · an uninstalled box keeps a live WireGuard tunnel into the off-site endpoint; declared
  in neither the KEPT nor the WIPED list.

**3 — misleads**
- **R-269** · a rotated-out local-API token still authorises until an unrelated lookup forces a
  reload. The shipped test passes only because of its lookup order.
- **R-270** · R-268's own rotation recipe is a step short; the controller never re-reads the mount.
- **R-271** · the `agent_channel_unauthorized` alarm can never close — its own advice silences the
  all-clear.
- **R-277** · three hub surfaces present a healthy off-site tier as absent. **This one caught me.**
- **R-278** · demo-felhom has had no off-site backup for six days, waiting on a ceremony nobody ran.

**4 — cosmetic / hygiene**
- **R-275** · five orphaned 0600 credential backups survive, and uid reuse hands them to the new
  service account. Superseded keys here; live if the backups were recent.
- **R-279** · no operator-triggerable off-site backup exists.

## What I got wrong, and corrected

I reported to the operator that the off-site tier had not run on **either** box since 2026-08-03.
That was true of demo-felhom and **false of demo-hp**, which had 18 unbroken daily snapshots. I had
read three hub surfaces that agreed with each other and none of which said what I took them to say
(now **R-277**). I corrected it before it changed any decision, and the operator's "repair off-site
first" ruling turned out to be unnecessary for the chosen venue.

I also raised **R-275**'s sudoers half as a likely privilege-escalation on reinstall, then **tested it
and refuted my own hypothesis**: sudo skips filenames containing dots, so the leftover file is inert.
`visudo -c -f` parsing a file OK is not evidence that sudo loads it.

## Integrity verdict

**Not reached** — P6 never ran. The instrument is proven and waiting: the comparator was watched
**failing** three ways, including an NFC→NFD rename that renders identically to the eye and would
have passed a rendered-text comparison. Four files, two with Hungarian accented names
(`árvíztűrő-tükörfúrógép.txt`, `nested/őszibarack.md`), one 3 MB binary; manifest keyed on raw name
bytes, committed as `evidence-rehearsal-2026-08-09/GATE0-before-manifest.json`; all four confirmed
present in snapshot `41c830db`.

## Wall clocks

| phase | duration |
|---|---|
| Pre-phase (R-268 rotation, proved both ways) | ~25 min |
| Gate 0 (venue, dataset, positive control, off-site run, capture) | ~55 min |
| P1 uninstall | **60 s** (08:37:23 → 08:38:23 UTC) |
| P1 leave-behind measurement | ~12 min |
| P2 preflight (3 runs: 2 refusals, 1 pass) | ~6 min |
| **P3 install — FAILED** | **44 s** (08:51:33 → 08:52:17 UTC) |
| bare machine → visible files | **not reached** |

## Steps taken off-path, and what they cost

1. **R-268 rotation on demo-felhom** — required by the runbook's pre-phase; not the venue.
2. **demo-hp's dashboard password re-set to the credentials-file value**, on operator instruction.
   The customer-owned password was unknown to this session and no operator route to the off-site
   button exists (R-279). Prior hash preserved in-guest; destroyed with the guest at P1. Cost: none —
   P1 wiped it and P4 re-claims.
3. **The off-site run was started by a script pressing the dashboard's own endpoint** with a real
   session and CSRF token, not by a person clicking. Identical server path; only the click synthetic.
4. **`--passphrase-file` instead of the no-echo prompt** — a first-class documented option with a
   permission check, so the secret still never touched argv. A person would type it.
5. **`systemctl stop dnsmasq && systemctl disable dnsmasq`** — the action the refusal message tells
   the owner to take, used as the counterfactual that confirmed R-272.

Everything after the install command returned was read-only. No repair was attempted on the
half-installed box.

## Teardown — all four layers

1. **The machine** — nothing created beyond the half-install itself, which is **left in place
   deliberately** for inspection and resumption (`state.json` completed: preflight, token, grows,
   enroll). demo-hp is **not serving** right now: no guest, agent installed but no unit.
2. **The host** — `local-lvm` 20 904 790 → 12 355 143 KiB (**≈8.5 GiB returned**); `local` ≈64 MiB;
   NVMe unchanged, backups deliberately kept. Pre-existing leftovers found and **not** removed
   (not this run's, recorded instead): storage `c11-scratch`, the orphaned `vzdump-lxc-9100` archive,
   and `/root/.dpw`, `.h`, `.sec.html` in the old guest (now destroyed with it).
3. **The hub** — **no customer or appliance record was created**; the `demo-hp` customer is retained
   deliberately, as the runbook requires. Nothing to delete.
4. **The off-site side** — **one write, and it was the intended one**: the Gate 0 backup that created
   snapshots `41c830db`, `9e38b84c`, `78b93f04`. **No prune, no forget, no delete.** How I know: every
   restic call was `snapshots`, `ls`, or the product's own `POST /backup/offbox/run`; retention runs
   inside that product path and is ep0's server-side job (R-89, boxes keep `keep_last: 0`).

## Secrets handling

No secret reached stdout. Token values, the retrieval passphrase, the controller password and the hub
DB copy were handled file→file at 0600 and shredded; the hub DB copy (which carries every host's
break-glass credential) was shredded immediately after the one hash comparison it was taken for.
Credential comparisons were done by sha256 prefix, never by value.

## State for whoever resumes

- **Blocked on one command** — `git tag v0.128.0 28ba8593b8 && git push origin v0.128.0`.
- **Nothing in Gate 0 needs redoing.** Snapshot `41c830db` holds the dataset; the before-manifest is
  committed; the comparator is proven.
- **First thing to record at step 7/8** when the install resumes: which golden it takes (R-274 is the
  one finding still owed an end-to-end observation).
- **P4 will need you** — the claim code is emailed to `doodoo21@freemail.hu` and there is no
  operator-side route to it (R-119).

Full account: `documentation/audits/REHEARSAL-byo-reinstall-2026-08-09.md`.