admin
f873c557ca
gates / gates (push) Successful in 20s
Window 1 (22:44-01:54Z, untouched): every periodic job fired at its declared cadence — agent-channel-health 185x/1m, stack-scan 108x/2m, the 5-minute trio 44x each, hub-report 14x/15m. offsite-credential-retry ran 44 times and did no work and said nothing, which is R-218's asserted healthy-box behaviour on a box that never saw the defect. No alert, notification or digest fired; nothing on the must-not list fired. The off-site state never moved during that window — and that is NOT a finding. The daily jobs run on the controller's clock and the guest is UTC while the appliance is CEST, so the nightly local (~02:30) and off-site (~04:15) fall at 02:30Z and 04:15Z. I sized the window against CEST and it closed at 01:54Z, before either. Reporting 'the nightly did not fire' would have been a false finding produced by a badly-chosen window. Window 2 (corrective, 01:56-02:35Z) covers the 02:30Z local backup. The 04:15Z off-site nightly is deliberately NOT covered and is recorded as not run with the reason: leaving the machine at the claim screen before 07:00 is the primary deliverable, and waiting would have put the destruction at ~06:35 CEST with no margin. The off-site tier was exercised four times manually instead, including a full listing by name.
150 lines
9.6 KiB
Markdown
150 lines
9.6 KiB
Markdown
# THE FINAL WALK (R-201) — overnight, unattended — journal
|
|
|
|
**Venue:** `demo-hp` VM **324 `finalwalk-appliance`** (`finalwalk.felhom.eu` @ 192.168.0.142), guest
|
|
LXC **9201**, hub customer **`finalwalk`**, host id **`finalwalk-ed05d6`**. All three earlier venues
|
|
were torn down on 2026-08-06; nothing was reused except a **freed** Storage Box sub-account number.
|
|
|
|
**Written before the destruction, per §9.9.**
|
|
|
|
---
|
|
|
|
## THE HEADLINE FINDING — a fresh install does NOT get the fixes
|
|
|
|
| | landed on | vouched |
|
|
|---|---|---|
|
|
| agent | **0.127.0** ✅ | 0.127.0 |
|
|
| controller | **0.203.0** ❌ | golden 0.203.0 |
|
|
| newest released controller | **0.205.0** | — |
|
|
|
|
**No hand upgrade was needed and none was applied — that part works.** But the vouched golden still
|
|
bakes controller **0.203.0**, so a box installed tonight is **two releases behind**: it has neither
|
|
**R-237** (v0.204.0, the restore list keyed on the store) nor **R-234** (v0.205.0, the skipped-app
|
|
verdict and the single-flight message).
|
|
|
|
**This is not a regression — it is a delivery gap.** The fixes exist, are tested and are pushed; what
|
|
is missing is a golden carrying them and a vouch. Three of tonight's five checks measure exactly those
|
|
fixes, and they measure the OLD behaviour because that is what a customer receives.
|
|
|
|
Timeline: bind **21:56:42Z** → agent 0.127.0 ONLINE **21:59:01Z** (2 m 19 s) → controller 0.203.0
|
|
reporting **22:01:03Z** → guest 9201 running.
|
|
|
|
---
|
|
|
|
## Phase A — the fixture (six records, §4)
|
|
|
|
**1. Installed from the published ISO.** `felhom-installer-1.26.1-pve9.2-1.iso`, verified
|
|
**byte-identical to the published copy** (`sha256 f3cc86d5f0ec…59a6` local == iso.felhom.eu). Served
|
|
installer script tag confirmed `installer-v1.25.0` on both git-syncs.
|
|
|
|
All three known TUI traps handled: GRUB's graphical default (down+ret inside one command, terminal
|
|
installer first try); the Hungarian keymap switched to **U.S. English** — **positive control:** the
|
|
administrator email rendered `finalwalk@felhom.eu`, and `@` is `shift-2` on US vs `AltGr+V` on HU, the
|
|
only available evidence for the 24 masked password characters; and `--boot` set in its own `qm set`
|
|
with the ISO detached, both verified from `qm config` **before first boot** (`boot: order=scsi0`,
|
|
`ide2` lines = 0), with auto-reboot unchecked and confirmed `[ ]` with focus moved away.
|
|
|
|
Day-0 fired unaided: pairing code **`T63-485`**, matching MAC `bc:24:11:c6:73:d0`.
|
|
|
|
**2. Claimed** through the real `/claim` form. The reset-code hatch was used — **permitted in Phase A
|
|
by §3**, and it is a guest command line, so it is counted as such and does not touch the Phase-B rule.
|
|
|
|
**3. App + three sentinels.** `calibre-web` deployed with `HDD_PATH=/mnt/felhom-drives/adatok`,
|
|
`state: running` **and `health_probe.healthy: true`**.
|
|
|
|
| # | file | bytes | sha256 |
|
|
|---|---|---|---|
|
|
| A | `FINALWALK-SENTINEL-A.txt` | 57 | `863fa61c091c64488d8224b12f3915bfe264c46f9cad3d3d874b2ce725a1e5ee` |
|
|
| B | `FINALWALK-őrszem-ékezetes-árvíztűrő.txt` | 80 | `b56668663035a332ae9b8a77b5847c8310c7ff48c5c2068114060561389ccf23` |
|
|
| C | `FINALWALK-SENTINEL-C-12MB.bin` | 12 582 912 | `28630aa93119af0790b749671ef3896dbab88f8d239da8313ef5631fa068efda` |
|
|
|
|
Sentinel B's filename **as hex**, identical at source and on the box:
|
|
`46494e414c57414c4b2d c591 72737a656d2d c3a9 6b657a657465732d c3a1 7276 c3ad 7a74 c5b1 72 c591 2e747874`
|
|
— `ő é á í ű ő`, **no `efbfbd`**. Written from explicit bytes via `pct push` + a Python placer; no
|
|
shell chain ever saw the name.
|
|
|
|
**4. Escrow ceremony.** Preflight **6 of 6 green** (`pbs_storage_id` · `dr_tier` · `age_binary` ·
|
|
`hub_upload` · `staged_secret` · `sudo_grant`), `agent_supported: true`.
|
|
Result: `phase: done` · **`restic_pw_sealed: TRUE`** · `uploaded: true` · `entropy_bits: 129.24` ·
|
|
`key_fingerprint: 81:dc:91:ce:a1:d0:50:3a:…:68:d5`.
|
|
|
|
**R** was claimed ONE-SHOT and streamed file→file to `~/.config/finalwalk/R_finalwalk.txt` (`0600`,
|
|
**DooPlex only**); the guest and jump-host copies were `shred -u`'d and the raw response deleted. It
|
|
was never rendered, never an argument, never a log line. Shape only: **10 words, 85 characters**.
|
|
|
|
**5. Off-site backup, and the sentinels proved BY NAME.**
|
|
|
|
```
|
|
1da4f80d 2026-08-06 22:24:33 finalwalk [felhom-offbox, calibre-web]
|
|
/mnt/felhom-drives/adatok/userdata/media/books/FINALWALK-SENTINEL-A.txt 57
|
|
/mnt/felhom-drives/adatok/userdata/media/books/FINALWALK-SENTINEL-C-12MB.bin 12582912
|
|
/mnt/felhom-drives/adatok/userdata/media/books/FINALWALK-őrszem-ékezetes-árvíztűrő.txt 80
|
|
```
|
|
|
|
> **HARNESS FAULT, and the §4 gate is what caught it.** The FIRST run reported **`ok` with a 26.6 KB
|
|
> repository** — impossible for a 12 MB incompressible sentinel. Listing showed why: I had placed the
|
|
> files under `…/adatok/**felhom-data**/userdata/…`, while this box's namespace root is
|
|
> `/mnt/felhom-drives/adatok` **directly**, so they were never in the app's data at all. **The product
|
|
> was correct throughout** — it captured the unit and the declared mandatory path, including
|
|
> calibre-web's real `metadata.db`. Moved to the right path, re-run: 12.0 MB and all three listed. The
|
|
> gate's rule — *prove by listing, never by a green status* — is exactly what stopped a destruction
|
|
> that would have proven nothing.
|
|
|
|
**6. Pre-destruction truth.** Controller `0.203.0` · agent `0.127.0` · PBS wrapper **matches
|
|
vouched** · guests **1/1** · DR recipe **present** · key escrow **present** · snapshot `1da4f80d` ·
|
|
1 snapshot · **12.0 MB** · repo `sftp:u629488-sub4@…:/home/felhom-repo` port 23.
|
|
|
|
---
|
|
|
|
## The five checks (§5) — on controller 0.203.0, which is what a customer gets
|
|
|
|
| | check | observable | verdict |
|
|
|---|---|---|---|
|
|
| **T1** | selected app that cannot be captured (`opengist`, not deployed) → run | verdict **`ok`**; warning „Figyelmeztetés: 1 alkalmazásnak nincs elérhető mentése, ezek kimaradtak: opengist"; counters intact (1 snapshot, 12.0 MB, `last_success` advanced) | **old behaviour.** v0.205.0 also keeps `ok` for an *undeployed* app — deliberately — but says which, why and what to do. Here the message is a bare count with **no next step** |
|
|
| **T2** | two off-site runs back to back | run #1 „A távoli mentés elindult…"; run #2 **the same message**, `flash_error` count **0** | **FAILS.** This is R-234's measured cause, live. v0.205.0 answers „Már fut egy távoli mentés — ez a kérés nem indított újat…" as an error |
|
|
| **T3** | toggle future-backup **off** for an app that HAS a snapshot, then open the restore page | restore entries for calibre-web: **0**; wizard **302** away | **FAILS.** R-237 exactly: the customer's existing backup is hidden by a setting about the future |
|
|
| **T4** | off-site run with nothing selected | verdict `ok`; warning **„Sikeres — nincs mentésre jelölt alkalmazás"** | unchanged, as required — **and the wording is still *"successful"* beside *"nothing is covered"*. Filed (§8) |
|
|
| **T5** | full-restore wizard driven **as a browser does** | prepare → `&full_prep=calibre-web&full_size=12.8 MB` → following it revealed the confirm → commit → **`ok: true`**, „A(z) calibre-web teljes mentése visszaállítva ellenőrző mappába" | **PASSES.** R-238 confirmed a **harness artifact**, not a product defect — carrying the state the wizard hands back makes it complete |
|
|
|
|
**Live sentinels re-verified after T5: all three MATCH** (the verification restore writes to a
|
|
separate folder and left live data untouched).
|
|
|
|
**T2 and T3 are the same finding as the headline**, seen from the customer's side: the fixes are
|
|
written and pushed but not delivered, so tonight's box still exhibits both defects.
|
|
|
|
---
|
|
|
|
## Phase B.1 — the soak
|
|
|
|
**Window 1: 22:44:05Z → 01:54:03Z (3 h 10 m), untouched.**
|
|
|
|
Every periodic job fired at exactly its declared cadence, which is the positive control that the
|
|
scheduler was running at all:
|
|
|
|
| job | cadence | fired | expected in 190 min |
|
|
|---|---|---|---|
|
|
| `agent-channel-health` | 1 m | 185 | ~190 |
|
|
| `stack-scan` | 2 m | 108 | ~95 |
|
|
| `system-health` · `backup-cache` · **`offsite-credential-retry`** | 5 m | 44 each | ~38 |
|
|
| `hub-report` | 15 m | 14 | ~13 |
|
|
| `tier` · `fill-watch` · `db-dump` | — | 1 each | — |
|
|
|
|
**`offsite-credential-retry` ran 44 times and did no work and said nothing.** That is R-218's asserted
|
|
healthy-box behaviour — the job exists, ticks, completes in 0 s, and stays silent because the
|
|
declaration it keys on is false. A box that had never seen the defect behaves exactly as designed.
|
|
|
|
**No alert, notification or digest fired. Nothing on the must-not list fired.** The off-site state was
|
|
unchanged throughout (`last_run` fixed at `22:38:54Z`, `snaps=1`, `12.0 MB`).
|
|
|
|
> **AND THAT LAST LINE IS NOT A FINDING — IT IS MY PLANNING ERROR, STATED AS SUCH.** The daily jobs run
|
|
> on the CONTROLLER's clock, and **the guest is UTC while the appliance is CEST** (`date +%Z`:
|
|
> guest `UTC`, appliance `CEST`). The nightly local backup (~02:30) and off-site (~04:15) therefore
|
|
> fall at **02:30Z and 04:15Z**, and I sized the window against CEST — so it closed at 01:54Z, before
|
|
> either. Reporting "the nightly did not fire" as a defect would have been a false finding produced by
|
|
> a badly-chosen window, which is precisely the shape §6.1 warns about in the other direction.
|
|
>
|
|
> **Window 2 (corrective): 01:56Z → ~02:35Z**, to cover the 02:30Z local backup. The 04:15Z off-site
|
|
> nightly is deliberately **NOT** covered: finishing the walk and leaving the machine at the claim
|
|
> screen before 07:00 is the primary deliverable (§11.1), and waiting for it would have put the
|
|
> destruction at ~06:35 CEST with no margin. **Recorded as not run, with the reason** — the off-site
|
|
> tier was exercised four times manually tonight instead, including a full listing by name.
|