# REPORT — BYO reinstall rehearsal, 2026-08-09 (session report) *Written as `REPORT-.md` rather than `REPORT.md` per the repo's parallel-session rule.* ## The answer to the runbook's question, first **Yes, the data comes back byte for byte. No, not in one sitting, and not without a shell.** All four planted files returned **BYTE-IDENTICAL** — including two Hungarian accented filenames verified as *raw name bytes*, not as rendered text. The unlock took **21 s**, the restore **13.2 s**. But the walk completed only because two hard stops were cleared by someone who could open a terminal and read source. **R-273**: the install died at step 5/8 on an agent version that was published as a package but never git-tagged — cleared by completing the release. **R-280**: the reinstalled machine could not re-attach its own data drive through any dashboard route, while the restore page said „**Ez két kattintás**" and pointed at an empty list — cleared by POSTing an internal path (`/mnt/sys_drive`) that no household could produce. Neither is a data-integrity problem. Both stop a household dead. **This is the same shape the R-201 walks kept finding: the data half passes, the journey half fails.** ## Venue **demo-hp (t740)**, operator-approved at STOP 1. It won every fidelity criterion that distinguishes the two demo boxes: three customer apps against one, a registered storage path against none, and a Secure-Boot `shim` install — the customer shape — against demo-felhom's SB-off `mkimage` firmware workaround. `drill-r50` (VM 300) was verified not at risk before proceeding: it is outside the `felhom` pool, on `local-lvm`, and `--uninstall` removes no storage and no non-pool guest. ## Findings, ranked by what they cost the person in front of you **1 — stops the visit** - **R-273** · the vouched agent (0.128.0) had no git tag; every install died at 5/8. **CLOSED** — tag pushed on your instruction after an independent sha check; install then succeeded in 3 m 49 s. The two guards that would prevent a recurrence are still owed. - **R-280** · a reinstalled machine cannot re-attach its data drive through any route, and the restore page promises „két kattintás" at an empty list. **The one to fix before the tester's visit.** - **R-272** · Felhom's uninstall restarts its own dnsmasq unconstrained; it grabs `:53`; the next install refuses and appears to blame the owner's network. **2 — costs the visit** - **R-274** · a local golden is adopted with no version and no checksum check. The copy on demo-hp is controller **0.192.0** against a vouched **0.210.0** — and below 0.200.0, where the recovery screen the customer needs actually shipped. - **R-276** · an uninstalled box keeps a live WireGuard tunnel into the off-site endpoint; declared in neither the KEPT nor the WIPED list. **3 — misleads** - **R-281** · the hub said nothing at all through the whole reinstall, and the tripwire for a sealed-backup unseal did not fire on a real one. - **R-282 / R-283** · one code, three names; the mail points at a page the box is not showing; the hub reads "Claimed 18d ago" while the box serves its setup page. - **R-269** · a rotated-out local-API token still authorises until an unrelated lookup forces a reload. The shipped test passes only because of its lookup order. - **R-270** · R-268's own rotation recipe is a step short; the controller never re-reads the mount. - **R-271** · the `agent_channel_unauthorized` alarm can never close — its own advice silences the all-clear. - **R-277** · three hub surfaces present a healthy off-site tier as absent. **This one caught me.** - **R-278** · demo-felhom has had no off-site backup for six days, waiting on a ceremony nobody ran. **4 — cosmetic / hygiene** - **R-275** · five orphaned 0600 credential backups survive, and uid reuse hands them to the new service account. Superseded keys here; live if the backups were recent. - **R-279** · no operator-triggerable off-site backup exists. ## What I got wrong, and corrected I reported to the operator that the off-site tier had not run on **either** box since 2026-08-03. That was true of demo-felhom and **false of demo-hp**, which had 18 unbroken daily snapshots. I had read three hub surfaces that agreed with each other and none of which said what I took them to say (now **R-277**). I corrected it before it changed any decision, and the operator's "repair off-site first" ruling turned out to be unnecessary for the chosen venue. I also raised **R-275**'s sudoers half as a likely privilege-escalation on reinstall, then **tested it and refuted my own hypothesis**: sudo skips filenames containing dots, so the leftover file is inert. `visudo -c -f` parsing a file OK is not evidence that sudo loads it. ## Integrity verdict — BYTE-IDENTICAL ``` expected 4 file(s); found 4 VERDICT: BYTE-IDENTICAL ``` Four expected, four restored, zero differences, compared against `evidence-rehearsal-2026-08-09/GATE0-before-manifest.json` — a manifest keyed on **raw name bytes**. `árvíztűrő-tükörfúrógép.txt` and `nested/őszibarack.md` came back with their name bytes intact (NFC preserved, `c3a1…`), which is the discriminator the Gate 0 positive control was built to enforce: the comparator had been watched **failing** on an NFC→NFD rename that renders identically to the eye. Restored out of snapshot `41c830db` into the verification folder the product names, with live data untouched. ## Wall clocks | phase | duration | |---|---| | Pre-phase (R-268 rotation, proved both ways) | ~25 min | | Gate 0 (venue, dataset, positive control, off-site run, capture) | ~55 min | | P1 uninstall | **60 s** (08:37:23 → 08:38:23 UTC) | | P1 leave-behind measurement | ~12 min | | P2 preflight (3 runs: 2 refusals, 1 pass) | ~6 min | | P3 install — first attempt, FAILED | 44 s (08:51:33 → 08:52:17 UTC) | | P3 install — resumed, SUCCESS | **3 m 49 s** | | P4 first contact (box live on its own URL) | within ~4 min of install | | STOP 3 unlock | **21 s** | | app redeploy (calibre-web) | **1 m 36 s** | | restore prepare + execute | **8 s + 13.2 s** | | **bare machine → verified files** | **1 h 49 m 22 s** (08:38:23 → 10:27:45 UTC) | | — of which the product's own work | **≈ 7 m 47 s** | **Neither figure is the customer number.** The 1 h 49 m is dominated by the R-273 diagnosis and release fix (~38 min) and two waits on a human. The 7 m 47 s is what the product costs when the operator already knows every answer. **The honest unaided figure is undefined, because an unaided household does not finish.** ## Steps taken off-path, and what they cost 1. **R-268 rotation on demo-felhom** — required by the runbook's pre-phase; not the venue. 2. **demo-hp's dashboard password re-set to the credentials-file value**, on operator instruction. The customer-owned password was unknown to this session and no operator route to the off-site button exists (R-279). Prior hash preserved in-guest; destroyed with the guest at P1. Cost: none — P1 wiped it and P4 re-claims. 3. **The off-site run was started by a script pressing the dashboard's own endpoint** with a real session and CSRF token, not by a person clicking. Identical server path; only the click synthetic. 4. **`--passphrase-file` instead of the no-echo prompt** — a first-class documented option with a permission check, so the secret still never touched argv. A person would type it. 5. **`systemctl stop dnsmasq && systemctl disable dnsmasq`** — the action the refusal message tells the owner to take, used as the counterfactual that confirmed R-272. 6. **Pushed the `v0.128.0` git tag** — outward-facing, done on your "proceed", and only after an independent download proved the published package's sha256 equalled the hub's vouched value. It completes a half-finished release rather than changing code; the release script's own recovery text is the same line. **Cost to the walk: the install that followed was a `--resume`, not a fresh run, which is why R-274 is only half-observed.** 7. **`POST /settings/storage/add` with `/mnt/sys_drive`** — the manual escape hatch, typed. This is the R-280 wall; a customer could not produce that path. **The biggest fidelity cost of the run.** 8. **SSH into the guest to fingerprint the restored tree.** This is my *instrument*, not a customer step — the customer's step (the restore) finished at the dashboard. Byte-comparison inherently needs file access; nothing about the product was driven this way. Everything else after the install returned was read-only, and no repair was attempted on the box. ## Teardown — all four layers 1. **The machine** — nothing created beyond the half-install itself, which is **left in place deliberately** for inspection and resumption (`state.json` completed: preflight, token, grows, enroll). demo-hp is **not serving** right now: no guest, agent installed but no unit. 2. **The host** — `local-lvm` 20 904 790 → 12 355 143 KiB (**≈8.5 GiB returned**); `local` ≈64 MiB; NVMe unchanged, backups deliberately kept. Pre-existing leftovers found and **not** removed (not this run's, recorded instead): storage `c11-scratch`, the orphaned `vzdump-lxc-9100` archive, and `/root/.dpw`, `.h`, `.sec.html` in the old guest (now destroyed with it). 3. **The hub** — **no customer or appliance record was created**; the `demo-hp` customer is retained deliberately, as the runbook requires. Nothing to delete. 4. **The off-site side** — **one write, and it was the intended one**: the Gate 0 backup that created snapshots `41c830db`, `9e38b84c`, `78b93f04`. **No prune, no forget, no delete.** How I know: every restic call was `snapshots`, `ls`, or the product's own `POST /backup/offbox/run`; retention runs inside that product path and is ep0's server-side job (R-89, boxes keep `keep_last: 0`). ## Secrets handling No secret reached stdout. Token values, the retrieval passphrase, the controller password and the hub DB copy were handled file→file at 0600 and shredded; the hub DB copy (which carries every host's break-glass credential) was shredded immediately after the one hash comparison it was taken for. Credential comparisons were done by sha256 prefix, never by value. ## State demo-hp was left in **Back in service and healthy** — agent 0.128.0, controller 0.210.0, guest 9201 running and onboot, claimed, storage path registered, `calibre-web` deployed, off-site repository unlocked and intact at 18 snapshots. `drill-r50` (VM 300) untouched throughout. **Deliberately left alone, and named rather than tidied:** the pre-existing `c11-scratch` storage and the three `vzdump-lxc-9100` golden archives on `local` (the teardown keeps goldens by design, and they now number three). The restored files sit in the product's verification folder, not back in place — that is R-213 and the product says so. ## Still owed - **R-280** — the drive wall. The one finding that would stop the tester's visit outright. - **R-273's two guards** — refuse a vouch whose tag does not resolve; check that a package and its tag ship together. The tag push fixed one box, not the class. - **R-274's missing observation** — a *fresh* (non-resume) install taking a stale local golden. Full account: `documentation/audits/REHEARSAL-byo-reinstall-2026-08-09.md`.