# THE FINAL WALK (R-201) — overnight, unattended — journal **Venue:** `demo-hp` VM **324 `finalwalk-appliance`** (`finalwalk.felhom.eu` @ 192.168.0.142), guest LXC **9201**, hub customer **`finalwalk`**, host id **`finalwalk-ed05d6`**. All three earlier venues were torn down on 2026-08-06; nothing was reused except a **freed** Storage Box sub-account number. **Written before the destruction, per §9.9.** --- ## THE HEADLINE FINDING — a fresh install does NOT get the fixes | | landed on | vouched | |---|---|---| | agent | **0.127.0** ✅ | 0.127.0 | | controller | **0.203.0** ❌ | golden 0.203.0 | | newest released controller | **0.205.0** | — | **No hand upgrade was needed and none was applied — that part works.** But the vouched golden still bakes controller **0.203.0**, so a box installed tonight is **two releases behind**: it has neither **R-237** (v0.204.0, the restore list keyed on the store) nor **R-234** (v0.205.0, the skipped-app verdict and the single-flight message). **This is not a regression — it is a delivery gap.** The fixes exist, are tested and are pushed; what is missing is a golden carrying them and a vouch. Three of tonight's five checks measure exactly those fixes, and they measure the OLD behaviour because that is what a customer receives. Timeline: bind **21:56:42Z** → agent 0.127.0 ONLINE **21:59:01Z** (2 m 19 s) → controller 0.203.0 reporting **22:01:03Z** → guest 9201 running. --- ## Phase A — the fixture (six records, §4) **1. Installed from the published ISO.** `felhom-installer-1.26.1-pve9.2-1.iso`, verified **byte-identical to the published copy** (`sha256 f3cc86d5f0ec…59a6` local == iso.felhom.eu). Served installer script tag confirmed `installer-v1.25.0` on both git-syncs. All three known TUI traps handled: GRUB's graphical default (down+ret inside one command, terminal installer first try); the Hungarian keymap switched to **U.S. English** — **positive control:** the administrator email rendered `finalwalk@felhom.eu`, and `@` is `shift-2` on US vs `AltGr+V` on HU, the only available evidence for the 24 masked password characters; and `--boot` set in its own `qm set` with the ISO detached, both verified from `qm config` **before first boot** (`boot: order=scsi0`, `ide2` lines = 0), with auto-reboot unchecked and confirmed `[ ]` with focus moved away. Day-0 fired unaided: pairing code **`T63-485`**, matching MAC `bc:24:11:c6:73:d0`. **2. Claimed** through the real `/claim` form. The reset-code hatch was used — **permitted in Phase A by §3**, and it is a guest command line, so it is counted as such and does not touch the Phase-B rule. **3. App + three sentinels.** `calibre-web` deployed with `HDD_PATH=/mnt/felhom-drives/adatok`, `state: running` **and `health_probe.healthy: true`**. | # | file | bytes | sha256 | |---|---|---|---| | A | `FINALWALK-SENTINEL-A.txt` | 57 | `863fa61c091c64488d8224b12f3915bfe264c46f9cad3d3d874b2ce725a1e5ee` | | B | `FINALWALK-őrszem-ékezetes-árvíztűrő.txt` | 80 | `b56668663035a332ae9b8a77b5847c8310c7ff48c5c2068114060561389ccf23` | | C | `FINALWALK-SENTINEL-C-12MB.bin` | 12 582 912 | `28630aa93119af0790b749671ef3896dbab88f8d239da8313ef5631fa068efda` | Sentinel B's filename **as hex**, identical at source and on the box: `46494e414c57414c4b2d c591 72737a656d2d c3a9 6b657a657465732d c3a1 7276 c3ad 7a74 c5b1 72 c591 2e747874` — `ő é á í ű ő`, **no `efbfbd`**. Written from explicit bytes via `pct push` + a Python placer; no shell chain ever saw the name. **4. Escrow ceremony.** Preflight **6 of 6 green** (`pbs_storage_id` · `dr_tier` · `age_binary` · `hub_upload` · `staged_secret` · `sudo_grant`), `agent_supported: true`. Result: `phase: done` · **`restic_pw_sealed: TRUE`** · `uploaded: true` · `entropy_bits: 129.24` · `key_fingerprint: 81:dc:91:ce:a1:d0:50:3a:…:68:d5`. **R** was claimed ONE-SHOT and streamed file→file to `~/.config/finalwalk/R_finalwalk.txt` (`0600`, **DooPlex only**); the guest and jump-host copies were `shred -u`'d and the raw response deleted. It was never rendered, never an argument, never a log line. Shape only: **10 words, 85 characters**. **5. Off-site backup, and the sentinels proved BY NAME.** ``` 1da4f80d 2026-08-06 22:24:33 finalwalk [felhom-offbox, calibre-web] /mnt/felhom-drives/adatok/userdata/media/books/FINALWALK-SENTINEL-A.txt 57 /mnt/felhom-drives/adatok/userdata/media/books/FINALWALK-SENTINEL-C-12MB.bin 12582912 /mnt/felhom-drives/adatok/userdata/media/books/FINALWALK-őrszem-ékezetes-árvíztűrő.txt 80 ``` > **HARNESS FAULT, and the §4 gate is what caught it.** The FIRST run reported **`ok` with a 26.6 KB > repository** — impossible for a 12 MB incompressible sentinel. Listing showed why: I had placed the > files under `…/adatok/**felhom-data**/userdata/…`, while this box's namespace root is > `/mnt/felhom-drives/adatok` **directly**, so they were never in the app's data at all. **The product > was correct throughout** — it captured the unit and the declared mandatory path, including > calibre-web's real `metadata.db`. Moved to the right path, re-run: 12.0 MB and all three listed. The > gate's rule — *prove by listing, never by a green status* — is exactly what stopped a destruction > that would have proven nothing. **6. Pre-destruction truth.** Controller `0.203.0` · agent `0.127.0` · PBS wrapper **matches vouched** · guests **1/1** · DR recipe **present** · key escrow **present** · snapshot `1da4f80d` · 1 snapshot · **12.0 MB** · repo `sftp:u629488-sub4@…:/home/felhom-repo` port 23. --- ## The five checks (§5) — on controller 0.203.0, which is what a customer gets | | check | observable | verdict | |---|---|---|---| | **T1** | selected app that cannot be captured (`opengist`, not deployed) → run | verdict **`ok`**; warning „Figyelmeztetés: 1 alkalmazásnak nincs elérhető mentése, ezek kimaradtak: opengist"; counters intact (1 snapshot, 12.0 MB, `last_success` advanced) | **old behaviour.** v0.205.0 also keeps `ok` for an *undeployed* app — deliberately — but says which, why and what to do. Here the message is a bare count with **no next step** | | **T2** | two off-site runs back to back | run #1 „A távoli mentés elindult…"; run #2 **the same message**, `flash_error` count **0** | **FAILS.** This is R-234's measured cause, live. v0.205.0 answers „Már fut egy távoli mentés — ez a kérés nem indított újat…" as an error | | **T3** | toggle future-backup **off** for an app that HAS a snapshot, then open the restore page | restore entries for calibre-web: **0**; wizard **302** away | **FAILS.** R-237 exactly: the customer's existing backup is hidden by a setting about the future | | **T4** | off-site run with nothing selected | verdict `ok`; warning **„Sikeres — nincs mentésre jelölt alkalmazás"** | unchanged, as required — **and the wording is still *"successful"* beside *"nothing is covered"*. Filed (§8) | | **T5** | full-restore wizard driven **as a browser does** | prepare → `&full_prep=calibre-web&full_size=12.8 MB` → following it revealed the confirm → commit → **`ok: true`**, „A(z) calibre-web teljes mentése visszaállítva ellenőrző mappába" | **PASSES.** R-238 confirmed a **harness artifact**, not a product defect — carrying the state the wizard hands back makes it complete | **Live sentinels re-verified after T5: all three MATCH** (the verification restore writes to a separate folder and left live data untouched). **T2 and T3 are the same finding as the headline**, seen from the customer's side: the fixes are written and pushed but not delivered, so tonight's box still exhibits both defects. --- ## Phase B.1 — the soak **Window 1: 22:44:05Z → 01:54:03Z (3 h 10 m), untouched.** Every periodic job fired at exactly its declared cadence, which is the positive control that the scheduler was running at all: | job | cadence | fired | expected in 190 min | |---|---|---|---| | `agent-channel-health` | 1 m | 185 | ~190 | | `stack-scan` | 2 m | 108 | ~95 | | `system-health` · `backup-cache` · **`offsite-credential-retry`** | 5 m | 44 each | ~38 | | `hub-report` | 15 m | 14 | ~13 | | `tier` · `fill-watch` · `db-dump` | — | 1 each | — | **`offsite-credential-retry` ran 44 times and did no work and said nothing.** That is R-218's asserted healthy-box behaviour — the job exists, ticks, completes in 0 s, and stays silent because the declaration it keys on is false. A box that had never seen the defect behaves exactly as designed. **No alert, notification or digest fired. Nothing on the must-not list fired.** The off-site state was unchanged throughout (`last_run` fixed at `22:38:54Z`, `snaps=1`, `12.0 MB`). > **AND THAT LAST LINE IS NOT A FINDING — IT IS MY PLANNING ERROR, STATED AS SUCH.** The daily jobs run > on the CONTROLLER's clock, and **the guest is UTC while the appliance is CEST** (`date +%Z`: > guest `UTC`, appliance `CEST`). The nightly local backup (~02:30) and off-site (~04:15) therefore > fall at **02:30Z and 04:15Z**, and I sized the window against CEST — so it closed at 01:54Z, before > either. Reporting "the nightly did not fire" as a defect would have been a false finding produced by > a badly-chosen window, which is precisely the shape §6.1 warns about in the other direction. > > **Window 2 (corrective): 01:56Z → ~02:35Z**, to cover the 02:30Z local backup. The 04:15Z off-site > nightly is deliberately **NOT** covered: finishing the walk and leaving the machine at the claim > screen before 07:00 is the primary deliverable (§11.1), and waiting for it would have put the > destruction at ~06:35 CEST with no margin. **Recorded as not run, with the reason** — the off-site > tier was exercised four times manually tonight instead, including a full listing by name. **Window 2: 01:56:28Z → 02:36:45Z — and the nightly DID fire, unprompted.** ``` 00:30:25Z db-dump 01:30:19Z tier + fill-watch ← the local tier legs 02:00:35Z metrics-prune 02:15:03Z offbox-backup ← the off-site nightly 02:16:05Z offbox-backup offbox: snaps 1 → 2, last_run 22:38:54Z → 02:15:24Z ``` So the soak covered a genuine scheduled cycle after all: the local legs in window 1 and the off-site nightly in window 2. **My 04:15 prediction was wrong in the other direction — it fires at ~02:15 controller time.** Both the prediction and the correction are recorded rather than quietly fixed. **BOTH DIRECTIONS, as §6.1 demands:** *What fired and should have:* every periodic job at its cadence; the DB-dump, tier and fill-watch legs; metrics-prune; the off-site nightly, which added the second snapshot with no prompting. *What should have fired and did not:* **nothing.** Six registered jobs were never seen in the log — `health-probes`, `status-refresh`, `ring-spill`, `deadapp-check`, `disk-health-check`, `selfupdate-check` — and **none of them is a finding.** The first four are **quiet by construction** (`scheduler.go:267` — `quiet := job.Interval <= 30*time.Second`, so a ≤30 s job never emits `Running job:`), and the last two run every 6 h, outside a ~4 h window. *An absent log line is not evidence* cuts this way too: I checked the source rather than filing four phantom defects. *What fired and should not have:* nothing. No alert, notification, email or digest. The only `WARN` lines in five hours were **three of mine** — a failed login attempt, a CSRF token I mangled, and T1's deliberate `opengist` skip, which logged correctly. **One observation worth keeping:** `offbox-backup` ticked **twice, 62 s apart**, and the snapshot count went 1 → 2, not 1 → 3. The second run was silently dropped by the single-flight — which for the NIGHTLY path is correct and deliberate (nobody asked; the next run retries). It is the same mechanism that, on the MANUAL path, produced R-234; v0.205.0 changes only the manual half and leaves this one silent, and this soak is live corroboration that the nightly half genuinely needs to stay quiet. --- ## Phase B.2 — the destruction and the rebuild All five §6.2 conditions were true and **written down first** (commits `2d2d8d3`, `f873c55`, `502078b`) before anything was destroyed. ``` 02:40:31Z pct destroy 9201 --purge (guarded on hostname = finalwalk; demo-hp also has a guest 9201) both logical volumes removed; pct list empty /mnt/adatok and /mnt/mentes wiped to 20 K, MOUNTS LEFT IN PLACE — deliberately: the surviving raw mount IS the R-220 condition 02:40:56Z felhom-host-install.sh v1.25.0, fetched live from felhom.eu/scripts/ 02:43:28Z Day-0 provision SUCCESS — 2 m 32 s ``` **What the rebuild landed on — the second measurement of R-239:** | | before | after | |---|---|---| | agent | 0.127.0 | **0.127.0** — no downgrade, no hand upgrade | | controller | 0.203.0 | **0.203.0** — the same two-release gap | The agent half is exactly right: the reinstall neither downgraded nor needed a hand. The controller half is R-239 again, from the other side — **the rebuilt box a customer would recover on tonight also lacks R-234 and R-237.** Golden used: `vzdump-lxc-9100-2026_08_06-23_58_32.tar.zst`. Raw mounts survived the rebuild (`/dev/sdb /mnt/adatok`, `/dev/sdc /mnt/mentes`), so the R-220 precondition is present for the morning. ## Phase B.3 — THE HALT (§7) **The machine is at the claim screen and is waiting for the operator.** Verified over HTTP **from the appliance**, with no guest shell: ``` GET / -> 200, A szerver beállítása — Felhom forms: POST /claim · POST /claim/request-new-code claimed: null · offbox config: absent (pristine rebuilt state) ``` **A claim code has already been requested and emailed**, through the customer-facing „Új kód kérése" path (`POST /claim/request-new-code` → 200, „Ha az e-mail cím regisztrálva van, elküldtük a kódot"), so the morning is *paste a code*, not *request one and then paste it*. **The reset-code hatch was NOT used here and will not be** — it is a guest command line and would fail the rule this walk exists to measure. It was used once, in Phase A, where §3 permits it. ### Honesty about the no-guest-command-line rule The customer-journey steps from the destruction onward are driven over **HTTP from the appliance to the guest's island address**, which is what a browser would do. Some **instrumentation reads** — `pct exec … python3` against `settings.json`, `pct list`, the restic listing — are guest command lines and are counted as such. They are not steps of the journey: none of them changed state, and none was needed to progress it. The distinction matters because conflating the two is how a walk claims a property it does not have.