R-201 PASSED: a customer's file survived a machine rebuild and came back byte-identical

Link 6 of the recovery chain had no client. The hub has served the identity blob since
slice 10D from handleReEnroll / handleGetRestoreDirective, gated on operator-armed recovery
mode and the global key -- and nothing in the agent, the hub UI, any script or any runbook
ever called either. The only documented retrieval was sqlite3 writefile() by hand on a
kubectl cp-ed database.

GET /api/v1/hosts/{host_id}/escrow is the box-authenticated mirror of the PUT that put the
blob there. Self-scoped (a per-host key reads only its own; global may read any). A host with
no bundle gets 200 {present:false} -- a 404 is indistinguishable from an unknown host and a
bare empty 200 from a zero-length blob.

THE TRADE IS RECORDED IN THE HANDLER, not inferred: obtaining the blob used to require the
operator to arm recovery mode; now whoever controls a rebuilt box can obtain it with that
box's own credential. They still cannot open it -- the hub has never held R and a wrong code
fails closed at age's scrypt KDF. The mitigation is that every retrieval raises
escrow_blob_served (warning, operator-only), recorded before the bytes leave.

escrowSelfServiceRetrieval is the single decision point: flip it to false and the endpoint
additionally requires recovery mode, changing nothing else.

The operator-driven DR path is untouched, pinned by a test. Red-proofs observed: removing the
ownership check serves host B's blob to host A; removing the record makes it silent.

CLAUDE.md

@@ -213,6 +213,22 @@ local and skippable, and only CI is neither.
 - **Website** auto-deploys via git-sync; just push to `main` (live in 1–2 min). Website changes go
   through `repo_gates.py` above (it runs `site_gates.py`); new pages go into that gate's `PAGES`
   list. Emergency edits: https://files.felhom.eu. All `website/` HTML is **UTF-8 with BOM** — preserve it.
+- **THE INSTALLER DOES NOT (R-110, 2026-08-03).** `manifests/webpage.yaml` runs **two** git-syncs:
+  the website from `main` as above, and `/scripts/` from the tag **`installer-v<SCRIPT_VERSION>`**.
+  Pushing `scripts/felhom-host-install.sh` therefore changes nothing that any machine downloads —
+  which it used to, within thirty seconds, for the one artifact that runs as **root on a virgin box**.
+  - **To publish:** cut `installer-v<new SCRIPT_VERSION>`, bump the `--ref` in `webpage.yaml`
+    (both the sidecar and the init container), commit, and sync. `hostinstall_gates.py` gate 6
+    fails if the manifest stops naming an `installer-v…` tag or if the website stops tracking `main`.
+  - **To roll back:** move the tag back to the previous commit and wait ~30 s. **No ArgoCD sync and
+    no deploy** — git-sync picks up a moved tag on its next period, measured live on 2026-08-03 in
+    both directions. That is the emergency lever; fix forward with a new version afterwards.
+  - **Do NOT pin the website to the tag.** The sparse-checkout used to cover `/website/` and
+    `/scripts/` in one sync, and pinning that would turn every copy edit into a release.
+  - The **URL never carries a ref** (`https://felhom.eu/scripts/felhom-host-install.sh`), so
+    `felhom-bootstrap.sh` and the hub's day-0 command follow the tag with no edit — do not add one.
+  - The installer's own sixteen run-time fetches are pinned separately, to `raw/tag/v$ART_AGENT_VER`
+    in the **agent** repo (R-183) — they are the agent's configs, not this repo's.
 - **Manifests** are GitOps via the `felhom` app — commit to `main`, then deliberate sync.
 
 ## Key patterns

CONTEXT.md

@@ -17,6 +17,357 @@
 
 ## Standing rulings
 
+**S-31 — THE DRILL PASSED: a customer's file survives a machine rebuild and comes back byte-identical.
+The capability is proven; the customer JOURNEY is four undocumented manual steps (2026-08-04, R-201/R-204).**
+
+**Proven on hardware, end to end:** demo-hp's controller data volume was destroyed and the sentinel
+deleted from disk. The recovery code produced `8a9e33aa4da6…` (byte-identical to the pre-wipe on-disk
+key AND the hub's independent record); it installed on the bare box; the **existing repository opened**
+(`repo_state: null`, 3 snapshots, 42 026 B = the pre-wipe size — not a fresh history); and the customer
+restore flow returned the sentinel at `643166269103a25c…`, **byte-identical**. `identity_blob` was
+unchanged throughout and **no ceremony was run at any point** (superseded rows still 2).
+
+**But it took four manual steps that are in no design document (R-204), all measured:**
+1. a rebuilt controller cannot configure its off-site tier — the one-time password was consumed by its
+   predecessor (`no unconsumed offsite password`, R-193);
+2. the Re-issue that fixes that sets `stale_at` **while `restic_pw_sha256` is unchanged** (R-196);
+3. a stale escrow makes the hub withhold the hash from the ACK → `EscrowAutoConfirmer` can never flip
+   `pending → escrowed` → `OffboxRunnable` refuses every run;
+4. it was cleared with the **manual** confirm (`/backup/offbox/confirm-escrow`), NOT a ceremony — a
+   ceremony would have superseded the identity blob and destroyed the recovered key;
+5. and the restore's **default `mode=unit` returns the recovery unit, not the customer's files** —
+   `mode=full` is what returns documents. A customer following the obvious path gets the wrong thing
+   and is told nothing.
+
+**Before any of it, a rebuilt box is unclaimed** and the claim gate intercepts every controller
+endpoint. **The local escape hatch does not work unaided:** `--print-reset-code` writes the new hash to
+`settings.json` while the running controller keeps its old copy in memory, so the code never matches —
+**restart the controller between minting and claiming.**
+
+*Facts a future session needs:*
+- **A guest rebuild in this fleet is a controller-DATA-VOLUME loss, not a guest reprovision.** The
+  2026-08-03 incident R-193 is filed against ran with guest 9201 up throughout — no `pct destroy`, no
+  `pct restore`, no `--selftest=provision`. Reproduce it that way.
+- **A good snapshot is not durable against a later bad run on the same day.** `forget --keep-daily 7
+  --group-by host,tags` keeps one per tag per day; a later, worse snapshot evicts a good one.
+- **Never run a ceremony while a recovery is in flight** — it supersedes the identity blob. Under
+  v0.93.0 the old blob is retained, but nothing serves a superseded blob back (R-199).
+
+**S-30 — the R-201 drill was PREPARED and HALTED BEFORE THE WIPE (2026-08-04). Nothing was wiped.**
+
+It stopped at step 4 because the sentinel file was **not in the off-site snapshot** while the run
+reported `ok` (**R-203**): the deploy-time `${USERDATA_PATH}` root and the backup-time `NamespaceRoot`
+disagree for an app whose `HDD_PATH` equals `system_data_path` — and the same compose used both roots.
+Wiping would have destroyed the only copy of the sentinel and proven nothing.
+
+*Established live on the way, all firsts, and all worth carrying:*
+- **A rebuilt box's off-site run REFUSES** — `repo_state: orphaned`, the orphan card, and
+  `offbox_repo_orphaned` at the hub. It does **not** silently start a fresh history. Closes R-193's Q3.
+- **The orphan reset works**: move-aside to `/home/felhom-repo.orphaned-20260804`, never delete.
+- **demo-hp's pre-rebuild off-site history is permanently unrecoverable** — its key is in superseded
+  row id 3 with `identity_blob` NULL, superseded **four hours before** v0.93.0 fixed the retention.
+- **Neither pre-existing off-site-toggled app has a restorable file leg** — both are named-volume-only,
+  which the tier tars but the customer restore never unpacks. Any drill needing a recoverable file
+  needs an app with a **mandatory userdata path**; `calibre-web` is now deployed on demo-hp as that
+  fixture, with the sentinel in place.
+
+*Still not established, unchanged:* **no file has ever been restored from an off-site backup after a
+wipe**, and Part 0's install path (controller v0.196.0) has never run against a live recovery.
+
+**S-29 — a box may fetch its OWN sealed recovery blob with its OWN credential; the operator-driven DR
+path is a separate thing and stays gated (2026-08-04, R-199; hub v0.94.0 + agent v0.125.0 + controller
+v0.195.0).**
+
+**The decision, so it is never re-litigated from scratch.** Self-service recovery needs the box to
+obtain its own sealed package. The only retrieval that existed was operator-gated behind recovery mode
+and the global key (`dr.go`), which cannot serve a customer-present flow. `GET /hosts/{id}/escrow` now
+serves that ONE object to its authenticated owner.
+
+- **Why it is safe:** the blob is opaque; the hub has never held R and has no decrypt path; a wrong
+  code fails closed at age's scrypt KDF (measured live — `exit status 1`, no plaintext, nothing
+  written); the caller already authenticates as this host for its report, desired state, WG
+  registration and PBS token, so this adds no new identity, only a new object.
+- **What it changes, which is the whole of the trade:** obtaining the blob used to require the
+  OPERATOR to arm recovery mode. It now needs only that box's credential. **Fewer parties are
+  required.** The mitigation is that the capability is AUDITED, not silent: every retrieval raises
+  `escrow_blob_served` (warning, operator-only), written before the bytes leave.
+- **The override is one constant.** `api.escrowSelfServiceRetrieval` — false re-imposes the
+  recovery-mode requirement on this endpoint and changes nothing else. Do not scatter the condition.
+- **`handleReEnroll` / `handleGetRestoreDirective` are UNTOUCHED** and must stay so: they rotate the
+  host API key and serve the K-escrow and the directive as well. Different object, different gate.
+  Pinned by `TestEscrowGet_OperatorDRPathUnchanged`.
+
+**Facts a future session needs before touching this area:**
+- **The unseal lives in the AGENT and must stay there.** `age` is an agent runtime dependency
+  (`/usr/bin/age`, present on both demo hosts) and is deliberately absent from the controller image;
+  the blob is host-scoped and its only writer is the agent under the per-host key.
+- **The agent returns ONE field.** Not the tunnel token, not the PBS token, not the WG key — the
+  controller is a trust tier down. Narrowing cost nothing and is not recoverable later.
+- **R's rules:** in memory for one call, cleared on every path, never on disk, never in argv, never
+  logged, never echoed. The controller CLI reads it from STDIN for exactly that reason. Verified live:
+  0 log lines, 0 files, 0 leftover staging dirs, with a positive control proving the search worked.
+- **The chain today: links 1–8 walked, 9–11 not.** The KEY comes back. Nothing installs it, reopens a
+  repository with it, or restores a file — R-200's remaining half and R-201.
+
+**S-28 — the escrow retention now covers the OFF-SITE data key, and customer-present recovery is the
+accepted design, which makes that retention load-bearing (2026-08-04, R-198/R-197; hub v0.93.0).**
+
+**The contract changed, and this is the sentence to carry:** when an escrow blob supersedes another,
+`demoteCurrentEscrowTx` retains **both** sealed keys — the K-escrow `blob` (PBS datastore key) **and**
+`identity_blob` (the age-wrapped bundle carrying the restic **repository** password). Until v0.93.0 it
+retained only the first, so the retention preserved the whole-guest key and destroyed the off-site data
+key, and **the destroying act was the escrow ceremony a rebuilt box asks its customer to run.**
+
+*Why this is load-bearing rather than tidy:* the operator ruled on 2026-08-04 that **candidate (c) is
+refused — no repository password is retained on the Proxmox host.** With no host-retained copy, the
+**customer-present recovery path is the only way back from a rebuild**, and it runs entirely through
+the retained identity blob. A session that touches escrow custody is touching the single copy.
+
+*Facts a future session needs before touching this area:*
+- **The ordering is load-bearing and invisible from the routine.** `SaveHostDRBundle` writes
+  `identity_blob` **after** `SaveHostEscrow` returns (the escrow PUT, `api/handler.go`), so at demote
+  time `host_escrow` still holds the OLD identity blob. Invert that and the retained bytes become the
+  NEW blob filed under the OLD hash — recoverable-looking and wrong. Pinned by
+  `TestSaveHostEscrow_RetainsIdentityBlob`.
+- **`demoteCurrentEscrowTx` is shared by re-escrow and host-delete.** Both have a test; fixing one
+  caller instead of the routine is a demonstrated red-proof, not a hypothetical.
+- **Nothing was backfilled and nothing could be** — pre-v0.93.0 retained rows have no identity blob and
+  their source rows are already overwritten. Both demo boxes' pre-2026-08-04 repository passwords are
+  gone permanently, recovery code or not.
+- **A retained key is not a recovery.** The hub's blob-serving endpoints have no client (R-199), the
+  injection seam has no form (R-200), and the path has never been exercised (R-201). v0.93.0 makes the
+  key survive; it does not hand it back.
+- **`offsite_repo_key_changed`** (R-197, warning, operator-only, edge-triggered on a supersession where
+  both hashes are known and differ) is the evidential signal that a box's off-site data key moved. It
+  carries **no hash value**. `MarkEscrowStale` is **precautionary**, not evidential — see S-26(a).
+
+**S-27 — a customer with NO machine ever bound is UNKNOWN, silently; one that was bound and went quiet
+still alarms (2026-08-04, R-195; hub v0.92.0).** Operator ruling, implemented as
+`store.HasEverBoundHost` (live `hosts` row OR `host_deletions` tombstone) consulted once at the top of
+`CheckBackupDeadlines`' loop. **The discriminator is "was a host EVER bound", never "has a report
+arrived"** — a box that was installed and never phoned home is a real fault. Fail-**open** on a read
+error; the deferral is logged with its own counter; the R-81 anchored-verdict structure is untouched.
+
+*The mechanism is worth carrying, because the guard that should have covered this was keyed off the
+wrong thing:* the existing down-skip reads `StalenessChecker.GetState()`, whose map is seeded from
+`store.GetCustomers()` — **a query over the `reports` table**. A customer with zero reports is in no
+row, gets no state, and `GetState()` returns `""` not `"down"`, so **the skip misses exactly the
+customer it would most obviously cover.** `david` (created 2026-08-01, no machine) e-mailed an
+`expected_dbdump_missed` ERROR three mornings running; `peti-felhom` — active, host deleted
+2026-07-15 — does not, because its 482 old reports make it `down`. Generalise it: **a "skip the dead"
+guard built on evidence of life cannot see something that was never alive.**
+
+**S-26 — the one-shot secret is the recoverable one; the irreplaceable one is minted fresh on every
+guest rebuild (2026-08-04, R-193 spike — `audits/SPIKE-offsite-credential-recovery-2026-08-04.md`).
+No code shipped for it; the decision is the operator's.**
+
+Two sentences, because they are what every future session needs before touching this area:
+
+- **Q1 — what is one-shot.** Of the three secrets in the offsite path, the hub's **one-time
+  provider password** is one-shot but freely re-issuable, the box's **SFTP key** is regenerated on
+  every apply by design, and the **restic repository password** — the DATA key, which the agent's own
+  source calls *"irreplaceable"* (`felhom-agent/internal/escrow/identity.go:35-39`) — is the only one
+  nothing can restage.
+- **Q2 — what a rebuild costs.** A rebuilt controller **mints a brand-new repository password**
+  (`WriteOffboxSecrets`, `offbox.go:392`, generates whenever `<DataDir>/offbox/repo_password` is
+  absent) and **no automatic path ever consults the escrowed one** — `InjectOffboxPassword` has exactly
+  one caller in the whole repo, a web form a human pastes into — so **every guest rebuild orphans the
+  previous off-site history.**
+
+*Measured without touching a box, and the method is reusable:* `host_escrow.restic_pw_sha256` vs
+`host_escrow_superseded.restic_pw_sha256`. demo-hp `8e03eddf…`→`8a9e33aa…` (15 snapshots / 40.9 MB
+orphaned); demo-felhom `48741892…`→`c60c8bc7…` (**36 snapshots / 1.14 GB**). **demo-felhom is the half
+that matters:** R-193 recorded it as having survived the rebuild by luck, and it did — its *delivery*
+recovered in 76 s off a stale staged secret. **Its repository did not**, and nothing said so for 13 h.
+*Luck restored the plumbing, not the data.*
+
+*Three corrections that must not be re-inherited.* **(a)** `ReissueCredentials` does **not** rotate the
+restic password — R-39's record, `offsite.go:198-201` and `api/handler.go:1067-1069` all say it does,
+and all three are wrong (→ **R-196**, the eighth entry in `CLAUDE.md`'s table). **(b)** Candidate (b),
+*"recoverable from escrow at re-bootstrap"*, is **not implementable** — the escrow is R-wrapped and
+zero-knowledge (D6), so only a customer-present ceremony can open it, which is the manual form that
+already exists. **(c)** Candidate (a), *"the hub restages automatically"*, **already exists** as
+`reissueOnReenroll`'s F3 leg and is wired to the wrong event — it sits behind `handleHostEnroll`'s
+mint-once-reuse short-circuit, and a **guest** rebuild leaves the `hosts` row intact. **Shipping it
+first would have made both boxes look healthy on 2026-08-04 while their snapshots were orphaned** —
+strictly worse than the current loud failure.
+
+*The candidate nobody had named,* and the only one aimed at the actual harm: **the agent survives a
+guest rebuild**, already receives the repo password over the pinned local API
+(`POST /escrow/stage-secret`) and already writes it to a fixed 0600 path — it merely **wipes** it after
+the ceremony. Retaining and serving it back needs no new seam. Its price is one real trade: a copy of
+the data key at rest on the Proxmox host. **That trade is the operator's to make and the spike does not
+make it.**
+
+**S-24 — offsite retention is ep0's, and the box asks for none (2026-08-04, R-191; installer 1.25.0).**
+R-89 moved offsite pruning server-side and box tokens stay write-only. The 2026-07-26 "two weeks"
+ruling was not reversed — **where it is ENFORCED moved, and the installer's `keep_last: 2` did not
+follow**, so every weekly run uploaded fine and then failed the job on a refused prune. The offsite
+tier now writes `keep_last: 0`, which the agent's existing guard already reads as *never prune from
+the box*; the local tier is untouched. A gate asserts it. **Before switching it off, ep0 was read:**
+both namespaces have a prune job at 03:30 keep-last 2 that has run daily since 2026-07-27 (18 tasks,
+all OK). **If that ever stops, `keep_last: 0` is unbounded growth** — check ep0's prune jobs before
+assuming the offsite tier is retained.
+
+**S-25 — a lost storage grant repairs itself, and the repair is REPORTED (2026-08-04, R-190; agent
+v0.124.1).** On a missing grant the agent runs the existing root wrapper (`felhom-backup-target-apply
+grant <id>`, already sudoers-permitted for any id) and re-reads once — the pbsdr R-22 shape. Bounded
+at one attempt per tier per hour.
+
+*The record is the point, and getting it wrong is easy.* A repair that leaves only `ok` behind
+destroys the only evidence a permission vanished. v0.124.0 reported degraded for "one cycle" and
+**that failed in production while every unit test passed**: `probeAll` is called independently by the
+self-check log and by the collector, the repairing call was the log's, and the host-report three
+seconds later sent `ok`. The fix is a latch on **time** (20 min > the 900 s report interval), so at
+least one report must carry it. The explanation rides `Feature`, because that is the field the hub
+interpolates into the operator's e-mail — `Reason` does not travel. **Nothing new was built:** the
+hub's existing ok→degraded→ok edge is the channel.
+
+*Caveat measured live:* **PVE caches permissions** (~40 s and ~16 min observed), so detection lags the
+loss and a single permission read is a lagging indicator → R-194.
+
+**S-23 — the host (on-box) whole-guest tier is restore-PROVEN, unattended, on both demo boxes
+(2026-08-04). Scope: those two boxes, not the fleet.**
+
+Four SCHEDULED runs overnight, none triggered by hand: demo-felhom host **83.8 s** / offsite
+**540.4 s**; demo-hp host **109.3 s** / offsite **300.1 s**. Every one restored into a scratch guest,
+booted, verified and destroyed itself.
+
+*What this closes.* Until yesterday every live restore-proof this project held was on the OFFSITE
+tier. The on-box tier — the one an ordinary recovery uses — had never been proven on either box, and
+not because it failed: the agent could not read the storage it lives on (R-185), so it never saw an
+archive there to test.
+
+*What was observed for the first time.* Both boxes had BOTH tiers due simultaneously. Never-proven
+sorts first, so each took its host tier, deferred the offsite one, and picked that up on the next
+evaluation six hours later — R-86's ordering and the one-heavy-operation gate, working together,
+unsupervised. The host-tier proof then reached the hub through R-189's merge, which is that path
+carrying a host-tier entry for the first time.
+
+*The asymmetry worth remembering:* a host-tier restore is **83–109 s**; an offsite one is
+**300–540 s**. The tier that matters for an ordinary recovery is also the cheapest to prove.
+
+**S-21 — an empty listing cannot distinguish FORBIDDEN from NEWBORN, so the box asks the permission
+question directly (2026-08-03, R-185; agent v0.123.0 + installer 1.24.0).**
+
+*The defect.* On both demo boxes the agent's token had `FelhomAgentStore` on `local`, `local-lvm` and
+`felhom-pbs` and **not** on `felhom-backup` — the storage the same installer configured as
+`local_backup_target`. The content API answered `{"data":[]}` through the token while root listed
+three archives. `pickForThisRun` skipped the tier as *"no settled archive yet"*, which is exactly
+what a brand-new tier reports, so the host tier was never restore-testable and nothing said so.
+
+*The rule.* The permission question has a definite answer where the listing does not. `Permissions`
+reads `/access/permissions?path=/storage/<target>` **as the agent's own token** — asking as root
+answers a different question and always says yes — and one `capability.Status` per configured tier
+reports it. The probed set comes from `BackupTiers()`, never a fixed list: a hardcoded probe list is
+the defect reproduced inside the fix.
+
+*The measured trap, because the obvious reading is wrong.* An ungranted path answers **neither empty
+nor 403**: it carries the privileges inherited from the box-wide `/` grant
+(`Sys.Audit, SDN.Use, Datastore.Audit`). Testing for path-presence, or for `Datastore.Audit`, reports
+a blinded storage HEALTHY. The probe tests **`Datastore.AllocateSpace`**, and re-measuring is required
+before that constant is ever changed.
+
+*Criticality, weighed once.* Critical, because the hub alerts only on critical and a non-critical
+entry would ride the report and alert nobody. **Except** the `local` fallback target, which
+host-install's own comment calls the DEGRADED configuration: still probed, still reported, but it
+does not page — turning an ordinary documented setup into an alert is how a signal becomes something
+an operator archives unread. It never consults content, so it cannot alarm on a newborn tier by
+construction, and it never reports ok when it could not ask.
+
+**S-22 — the installer's Scenario-F arm must finish the job, not just leave the definition alone
+(2026-08-03, R-185).** `configure_backup_target` has two arms. Case A creates the storage and grants
+in the same breath. The reuse arm — *"the target already exists"* — returned **without granting**, and
+that, not `PVE_STORAGES`, is where the drift came from: a box whose target pre-dated the install
+(the vzdump-target-move runbook, or a reinstall) pointed `local_backup_target` at a storage its token
+could not read. The reuse arm now ensures the ACL through the same guarded wrapper. **Scenario F is
+unviolated** — the storage DEFINITION is untouched, and granting the role the agent is supposed to
+have on the target this script is about to write into `agent.json` is finishing the job, not
+retargeting the box. `$BACKUP_TARGET_ID` stays OUT of `PVE_STORAGES`: that list is granted a step
+before the target is resolved, and `--acl-storages` entries are preflight-checked for existence.
+A gate asserts every arm that resolves the target also grants on it.
+
+**S-19 — a restore-test PROOF is durable and reportable; a FAILURE is neither, and that asymmetry is
+the design (2026-08-03, R-189; agent v0.122.0).**
+
+*The rule.* Only successful restore-tests are written to `RestoreTestState`, and that state is what
+the host report carries after a restart. Failures live only in the in-memory `backup.Store`.
+
+*Why, in one line each.* A **success suppresses future work** — under R-86's per-archive due-check a
+proven archive is never re-tested, so a lost proof leaves the box quietly less tested than it
+believes, for a whole archive generation (a week on the offsite tier). A **failure causes future
+work** — a failing tier stays due and is retried at the next evaluation, so a lost failure heals
+itself within one interval, while a *persisted* failure would outlive the fault it describes.
+
+*What the report does with the two.* The collector merges them: **one entry per tier, newest by
+`TestedAt` wins**. A fresh failure therefore beats a stored success (the failure is the news and
+exists nowhere else), a stored success beats a stale in-memory entry after a restart, and a tier can
+never appear twice — the hub would read that as two tests.
+
+*It refuses to lie.* A persisted record missing the archive **or** the tier produces **no entry**: an
+unproven tier reading as proven would be worse than the defect this closes. Run mechanics (scratch
+VMID, duration) are not re-invented — an absent duration is not a claim, a fabricated one would be.
+**Migration consequence, seen live:** a pre-R-189 record has no tier, so upgrading does not
+retroactively make an old proof visible to the hub; the tier's next real proof fills it in.
+
+**S-20 — the release order is build → tag LOCALLY → publish → push tag, and every step protects
+something (2026-08-03, R-188 + R-186).**
+
+The tag is created before the publish, so the build and the tag describe the same commit. It is
+**pushed** after, because the push is what wakes CI (`on: [push]`) and a tag visible before its
+package made `check-published-versions.py` correctly fail a *correct* release — measured on roughly
+every second release, and R-168 mails those failures to the operator.
+
+The invariant the old order protected is **asserted directly instead**: the gate now also refuses a
+**published version with no tag**, as a bounded probe (frontier + patch gaps) that prints its own
+coverage, because the package listing api is 401 without a token and absence cannot be enumerated.
+A half-done release is loud: publish-then-failed-push dies naming the recovery command, and a failed
+publish deletes the local-only tag so a retry is clean.
+
+**A released binary is independently verifiable** — `-trimpath -buildvcs=false` means the same source
+yields the same bytes with or without the tag; the verification command lives in
+`felhom-agent/CLAUDE.md`. Both build paths (`release-agent.sh` and `publish-agent.sh`'s fallback) use
+identical flags: they differed by `CGO_ENABLED=0` and produced binaries 74 KB apart for one version.
+
+**S-17 — restore-testing is PER ARCHIVE GENERATION, and the hub's staleness window follows each
+tier's own rhythm (2026-08-03, R-86; agent v0.121.0 + hub v0.91.0).**
+
+*The rule.* Let **A** be the newest archive on a tier that has settled for at least the settle lag
+(24 h). The tier is **DUE** when A exists and **A has not already been proven**. The daemon-start
+ticker survives only as the **evaluation interval** (6 h). A daily tier is proved daily on yesterday's
+archive; a weekly tier weekly on its own; a tier with no archive is UNKNOWN, never a fault.
+
+*The trap, written down so it is not reintroduced.* The literal reading of R-86 — *"due when the
+newest archive is ≥24 h old"* — is **never true on a daily tier**, because a new archive resets the
+newest-archive age to zero long before it reaches the lag. It would have switched restore-testing off
+for the tier that matters most, silently. Red-proved at 0 runs over 5 simulated days
+(`felhom-agent/internal/backup/restoretest_due_test.go`).
+
+*What the state holds now.* `RestoreTestState` records **which archive** was proven, not just when a
+tier passed — a timestamp cannot answer *"have we proven THIS archive"*. A pre-R-86 file keeps its
+time (rotation ordering survives a deploy) and yields no proven archive, so each tier is due exactly
+once after the upgrade.
+
+*The old config key.* `backup.restore_test_cadence_seconds` is DEPRECATED. **Negative still disables**
+verbatim; a positive value now seeds the **settle lag** only, and the daemon WARNs once at start-up
+naming `restore_test_eval_interval_seconds` (default 6 h) and `restore_test_settle_seconds`
+(default 24 h). It is deliberately NOT carried into the evaluation interval.
+
+*The hub half is not optional.* `restoreProvenStaleAfter` was a flat 7 days **derived from the cadence
+R-86 removes**, and a healthy weekly tier's proof age reaches EXACTLY 168 h just before its next
+proof — it sat ON the line. `restoreProvenWindow(tier, observed, ok)` now takes the tier's own
+observed archive interval × 4 generations, floored at 7 days, capped at 12 days (strictly inside the
+2-week offsite retention), falling back to the tier's **declared** rhythm (`backupStaleAfter` 26 h /
+`offsiteBackupStaleAfter` 8 d — the backup-freshness checker's own thresholds) when history is too
+short to observe one. Shipping Part 1 alone would have produced a nightly false alarm.
+
+**S-18 — `ep0` is Tier 2, PROTECTED (operator ruling, 2026-08-03).** D-d named two protected machines
+and did not name ep0 either way; `runbooks/target-selection.md` carried the question in writing for
+two days. The ruling **extends D-d's protected list to three machines**: DooPlex, Peti's cluster,
+**ep0**. It is a classification, not a new set of prohibitions — destroying datastores, prune jobs,
+tunnel config or nftables rules was already forbidden by what it would destroy, and the ordinary
+off-site READ a restore-test performs remains permitted.
+
 **S-13 — the `mp1` merge landed, and the variant was chosen on measurement (2026-08-03, R-165 / D-a).**
 The appliance's two data volumes are one. **Variant V-c**: the volume mounts at the NEUTRAL path
 `/var/lib/felhom`, and both `/var/lib/docker` and `/mnt/sys_drive` are binds of subdirectories of it.
@@ -97,6 +448,59 @@ not provide, and the fourth of those found on live hardware rather than by revie
   no run scope, so a refused app re-alerts on every status refresh (measured: a second identical alert
   pair 13 s after the run's). Pre-existing in v0.192.0; R-181 changed neither caller.
 
+**S-15 — publishing is an act, not a side-effect of pushing (2026-08-03, R-110 + R-115 + R-183).**
+Two rulings, one shape: something became live because someone pushed, not because anyone decided.
+
+- **The installer.** `/scripts/` now git-syncs the tag `installer-v<SCRIPT_VERSION>`; the **website
+  keeps tracking `main`** in a second sync, because pinning both would make every copy edit a
+  release. Publish = cut the next tag + bump the manifest `--ref` + sync. **Roll back = move the tag
+  back**, which takes ~30 s and needs no ArgoCD sync at all — git-sync v4.4.0 follows a moved tag,
+  and that half was measured before the manifest was touched because the whole model rests on it.
+- **The sixteen run-time fetches were NOT what the spec described** — sixteen, not nine, and from
+  `felhom-agent`, not this repo — so no tag here could cover them. They are pinned to
+  `raw/tag/v$ART_AGENT_VER` instead, which is strictly better: the agent's configs now come from the
+  same ref as the agent binary being installed. That closed a real skew (**R-183**), not just a
+  channel.
+- **The URL needed no change**, and that is worth knowing rather than re-deriving: it never carried
+  a ref, so both producers follow the tag automatically — and no hub change means no hub bump.
+- **The agent.** `scripts/release-agent.sh` is THE release path: build → tag → publish → **verify by
+  an independent download**. It does not vouch. `check-published-versions.py` refuses a `v<semver>`
+  tag with no downloadable package, and **CI now runs the full gate set** rather than `--fast`,
+  without which that gate would have been registered and never run.
+- **The gate's invariant is not the one specified, and P-C is why:** the hub manifest and Gitea's
+  package listing are both **401** anonymously; the package download and the tags api are not. So CI
+  can ask *is this installable* but not *what is vouched*. The residue is **R-184**.
+- **Neither gate asserts "the newest version is published."** That would go red on the very push
+  that bumps a version, before publishing — and a gate that fails on the normal path is one people
+  learn to ignore.
+
+**S-16 — a backup run NOTIFIES ONCE and RECORDS ALWAYS, and those are different things
+(2026-08-03, R-182; controller v0.194.0 + hub v0.90.0/.1).** Measured: nine per-app capture failures
+reached the hub, two were mailed, seven were dropped by a cooldown whose key carries no app
+identifier — *before* `LogNotification`, so they left no row anywhere.
+
+- **The record:** `recovery_unit_capture_failed`, per app, unconditionally, now routed
+  **record-only** by the hub (`recordOnlyEvents`) — stored and logged every time, never competing
+  for an e-mail slot.
+- **The notification:** `backup_run_failures`, ONE per run, listing every failed app with its leg
+  and reason plus the counts and free space. **A clean run emits nothing.**
+- **A suppressed operator event now leaves a `suppressed` row** carrying the key that suppressed it —
+  for every operator type. *"We chose not to e-mail you"* and *"nothing happened"* must never look
+  identical; that is the whole finding, stated as a rule.
+- **The periodic sweep gets a digest too, with NO `run_id`**, so it stays under the ordinary hourly
+  cooldown. Without it the sweep's failures would be recorded and never notified — a new silence
+  created while closing one. A real run's digest carries a unique `run_id` precisely so the cooldown
+  can never collapse a manual run into the nightly one.
+- **Why the silence is safe:** the hub's deadline check raises `expected_backup_missed` from report
+  freshness, independently of any mail the box sends (`monitor/deadline.go:396,417`). **If that check
+  is ever weakened, this design loses its footing.**
+- **Not taken, and why:** putting `app` in the cooldown key. It fixes the swallowing by producing one
+  mail per failing app — a dozen on a full disk.
+
+**ep0 was rescaled by the operator to a CX33 (2026-08-03): 4 vCPU, 8 GB RAM, measured on the box, and
+the 4 GiB swapfile survived. The 40 GB local disk is UNCHANGED** — a CPU/RAM resize only, so no disk
+figure in any runbook needed correcting. That closed **R-90** and unblocked **R-86**.
+
 **S-11 — D-c's routing, and why R-158's own proposal was overruled (2026-08-02, R-167 SHIPPED).**
 Decision D-c splits two signals by AUDIENCE, and the split is the ruling: **a fill warning is the
 CUSTOMER's** (they can free space, delete files, add a drive) and **a per-app backup capture failure

REPORT-campaign10.md

@@ -1,113 +0,0 @@
-# REPORT — Campaign 10, two-storage adversarial soak (2026-07-31)
-
-Follows `REPORT-campaign7/8/9.md`. Root `REPORT.md` is another session's (hub v0.85.0) and was not
-clobbered — same shared-clone reasoning as `REPORT-iso-release.md`.
-
-**Full audit + evidence:** `documentation/audits/CAMPAIGN-10-two-storage-soak-2026-07-31.md`,
-`documentation/tests/campaign10-evidence-2026-07-31/`.
-
-## The sentence that matters
-
-**Phase A passed every gate. Phase B ran to 39 consecutive cycles with the full atom set — past the
-"drift at the thirty-eighth" depth the brief asked for. 66 restores, 66 correct discriminators. No
-resource leak over 13.5 h. Two findings: R-156 and R-157.**
-
-## What was established
-
-- **Venue** — VM 311 on demo-hp (Tier 0), 200 G system + 2 × 50 G data, scratch storage at the
-  `/mnt/nvme-1tb` **mount root** (a subdirectory would have emitted `storage_disconnected` for demo-hp
-  all night — the exact signal I1/I2 discriminate).
-- **Baselines, all read fresh.** controller `main` 0.188.0, **golden 0.188.0 (not behind)**, agent
-  0.119.0 published+vouched, hub 0.86.0, ISO **1.26.1** (`f3cc86d5…`, round-trip verified live).
-  The brief's ISO assumption (v1.25.0) was ~90 minutes stale; its "no baked SSH key" claim is R-129.
-- **Isolation gate — both denials captured, each with a positive control.** The PBS control **failed
-  first**: four clean-looking 403s were worthless because the token was denied on its own datastore
-  too (PBS token privilege separation). Fixed, re-run, denials stand.
-- **A1** fresh install from the **published** ISO. 1.26.1 is a public release image — verified against
-  its bytes that it has no auto-install path — so it was driven blind via screendump + `sendkey`
-  through the Terminal UI. Caught the Hungarian-keymap trap before typing the root password, which
-  would otherwise have been mangled and locked the box out.
-- **A2** claimed for real; discriminator flipped `dashboard not yet claimed` → `authentication required`.
-- **A3** both drives enrolled through the **real** endpoint; `mentes` accepted as backup target via
-  the offer flow, ending `degraded:false / target:felhom-backup` — the I5/I6 healthy baseline.
-  Four apps healthy spanning both sides of D5's split (4 × `type: secret`, 1 × `type: password`).
-- **A4** discriminators seed and read back across all four apps; rallly's over the path
-  `DATABASE_URL` actually names, not the trusted socket that produced D5's false pass.
-
-## Phase B — the soak
-
-Three passes: run 1 (27 cycles, 6 atom families, 0 violations), run 2a (10 cycles, **stopped
-deliberately** — two violations were harness defects), run 2b (**39 cycles, 12 atom families**).
-1 461 invariant checks in total.
-
-- **I7 is the headline: 66 restores across both passes, 66 correct discriminators.** Never stale,
-  never empty. Run 2b added an `I7-SKIP` verdict so a check with unmet preconditions is recorded as
-  skipped rather than silently green — it fired once.
-- **I2, I3, I4, I5, I6, I10, I11: zero violations in either pass**, including the abort-in-place
-  variants and 330 secret-class assertions covering both sides of D5's split.
-- **`I1-under-load` 5/5**: the backup target pulled **while a backup was running** still produced
-  `backup_target_absent` and a clean recovery.
-- **R-117's Q7 case holds** — a filesystem aborted *in place* (device still present) surfaces via
-  `bound_under_parent=false`, the gate stops the app on the dead namespace, and the storage page names
-  it. That is the case R-117's spike called "the worse half".
-- **RTO, both bands measured.** S: 66 MB → **42.0 s** / **41.4 s** across two passes (66 restores).
-  M: 21.1 GB → **608 s** mean over 2 reps, both returning the correct discriminator. 327× the data
-  cost 14.5× the time, giving **RTO ≈ 40 s + 26.9 s/GB** (backup ≈ 29 s + 17.4 s/GB). The fixed ~40 s
-  dominates below ~1.5 GB — that is the S band, and why its numbers clustered so tightly.
-- **Capacity ceiling, and the more consequential result:** a DB-backed app's recovery unit is **1.90×**
-  its data (volume tar + SQL dump). The default `/mnt/sys_drive` is **20 GB**, so on a default box the
-  largest locally-backupable app is **≈ 10 GB** — **the M band does not fit at all** without a
-  per-customer `SysDataGrowGB`. **RPO still not measured.**
-- **No resource leak.** 9 457 samples of 19 metrics over 13.5 h: controller and agent RSS flat, fds
-  flat, and **no orphaned volumes, images or containers** despite dozens of redeploys, kills, reboots
-  and hard resets. The only curve with real slope is the **agent journal, 194 → 463 MB (~20 MB/h)** —
-  bounded by journald, but a lot of logging.
-- Every atom and invariant was **proven by hand before automation**; the runner asserts nothing that
-  was not first observed live.
-- **A Phase A gap was caught before the run:** no app had `HDD_PATH`, so all data sat on the system
-  disk and I3 could never have fired. calibre-web was deployed onto `adatok` first — otherwise the
-  soak would have produced green cycles that tested nothing cross-drive.
-- **Two violations were my harness, not the product**, and run 2a was stopped for them: a seed that
-  never landed became a fake "stale restore", and a real one would have looked identical. Fixed and
-  red-proofed before restarting.
-
-## Findings
-
-- **R-156 (new)** — **papra's data is neither persisted nor backed up, and it reports healthy.** The
-  template mounts `papra_data:/app/data`; the app writes `/app/app-data/db/db.sqlite`. The volume is
-  empty and root-owned (the image is `-rootless`, so the app cannot even write there), the real DB
-  sits in the container's writable layer, and the healthcheck only probes the HTTP port. Its
-  Tier-1/Tier-2 backup is real, verifiable, and contains nothing. Not fixed.
-- **R-157 (new)** — **bootrecon's start-once sweep misses the boot orphan it exists to recover.** Two
-  mechanisms. **A:** the container is left `Exited`, the sweep runs ~5 s after controller start while
-  docker is still restoring, sees "no boot-orphaned apps", and never re-checks (3 occurrences,
-  intermittent ~50%). **B:** the interruption leaves the stack with **zero containers**, which is
-  exactly the signature bootrecon deliberately skips as a user's Stop — and in that state the deadapp
-  check reported **`0 currently down`** while a `deployed: true` app was not running, i.e. silent on
-  every channel. A settle-condition fix closes A and leaves B open. Not fixed.
-- **Tier 3 could not be isolated, so it was not run.** Offsite hard-requires the DR tier
-  (`configs.go:1300`), and the DR tier only provisions on ep0 (per-endpoint allocation deferred,
-  `hub/README.md:260`). Both are recorded deliberate positions, so **no R-n minted**. The campaign
-  therefore touched neither ep0 nor the Storage Box — stronger isolation than asked for, obtained by
-  not running the tier. Cost: all Tier-3 atoms, I8, and the Tier-3 RTO/RPO rows.
-
-## What did not run
-
-**12 of the brief's ~13 atom families ran** (run 1 covered 6; run 2b added abort-fs-in-place,
-kill-agent-mid-backup, hard-reset-mid-write, reboot-VM, both concurrency atoms and fill-drive).
-Previously reported as 6 of 12 — that was run 1 only.
-
-**Superseded detail:** Still not run: **Tier-3 backup/restore** (§3, structurally un-isolatable) and **I8**. **I9** was not
-automated — cited from the tester-gate run on this same controller 0.188.0, not re-claimed.
-`kill_controller` is still not literally "mid-backup"; the dedicated concurrent backup+detach atom
-covers that case properly. The run-1 flaw where `reboot` never interleaved with a detach was fixed.
-
-**Depth reached: 39 consecutive cycles**, past the brief's "thirty-eighth", with c34–c39 clean on
-every invariant. Beyond 39 is untested, not proven clean.
-
-## Teardown — OWED, nothing removed
-
-Still intact: the rig is reusable for the atoms that did not run. VM 311, `c10-scratch`, PBS datastore
-`felhom-c10` + user/token, restic subaccount `u629488-sub4`, and **hub customer `c10-soak` (disposition:
-DELETE)** are all outstanding, with commands in the audit §9. Named explicitly because R-131 is four
-orphaned scratch customers left by exactly this omission.

REPORT-campaign7.md

@@ -1,43 +0,0 @@
-# REPORT — CAMPAIGN 7 (felhom.eu side: docs only)
-
-> Written as `REPORT-campaign7.md`, **not** the shared `REPORT.md`, per the convention this run
-> added to `CLAUDE.md`: `REPORT.md` is overwritten, so a second concurrent session in this repo
-> would clobber it. This session's implementation work was in `app-catalog-felhom.eu`; here it only
-> touched documentation.
-
-**Run:** 2026-07-18 evening → 2026-07-19 morning. **Class:** campaign (record-and-rank + a defined
-allowed-fix set). **Implementation repo:** `app-catalog-felhom.eu` (see its `REPORT.md`).
-
-## What changed in this repo
-
-| file | change |
-|---|---|
-| `documentation/audits/CAMPAIGN-7-catalog-sweep-2026-07-19.md` | **new** — method, uninstall-semantics map, trio detail, full 53-app matrix, ranked findings, coverage |
-| `documentation/backlog/ROADMAP.md` | **+3 items** — R-40 (multi-hop major upgrade path), R-41 (no standing catalog deployability check), R-42 (sidecar-major ruling) |
-| `CLAUDE.md` | REPORT.md parallel-session rule: the second session writes `REPORT-<topic>.md` |
-
-No hub/agent/scripts/website code was touched (campaign scope: catalog + docs).
-
-## Headline for this repo's readers
-
-1. **Uninstall semantics map row PARTIAL → PROVEN** (campaign doc §2), with live evidence from all
-   three trio apps: remove requires stop first; named docker volumes are **always destroyed**
-   (including the app's database); HDD bind-mount data and `backups/primary/<app>` survive unless
-   explicitly ticked; images are kept; `app.yaml` goes, the template stays; the per-app **offsite
-   toggle survives** the uninstall while tier-2 config is cleared. The confirmation modal does warn
-   about the volumes, so there is **no consent gap**.
-2. **A lying healthcheck takes an app OFF-LINE, it does not merely mislead.** Traefik will not route
-   to an `unhealthy` container, so a probe that cannot execute → permanent unhealthy → **404 to the
-   customer while the app serves 200 on its own port**. 7 of 53 apps were in that state.
-3. **The pre-flight gate's own signal is missing:** the 0.145.0 → 0.146.0 floor-lift emitted no
-   `controller_updated` event, though the identical bootstrap path emitted one for 0.143.0 → 0.145.0
-   two hours earlier (§0, finding F1). The box did converge — golden, floor and runtime all agreed —
-   but the event trail under-reports version transitions.
-
-## Open items owned outside this repo
-
-- **plant-it / wanderer** — images do not resolve at all (neither the new tag nor the one the
-  catalog already ships). Upstream research needed; recorded as findings, not deletions.
-- **gokapi** — pinned back to v1.9.6; v2 needs the seeded `config.json` regenerated. Security-
-  relevant, should not sit on a superseded line indefinitely.
-- **glance** — never had a seeded `glance.yml`; proven pre-existing.

REPORT-campaign8.md

@@ -1,62 +0,0 @@
-# REPORT — CAMPAIGN 8: the backup & restore subsystem (2026-07-27/28)
-
-Adversarial, destructive, unattended run against `demo-felhom`, `demo-hp` and `ep0`.
-Full report: `documentation/audits/CAMPAIGN-8-backup-restore-2026-07-27.md`.
-Evidence: `DooPlex:~/campaign8/evidence/` (103 files, 35 MB, written continuously by 11 collectors).
-
-**No production code was changed.** Findings are recorded and ranked, never fixed inline, per the
-campaign's own rules.
-
-## Scope safety
-`peti-felhom`, its namespace and `u629488-sub2` were never touched. Phase 0 established with five
-documented probes that peti has **no data at all** in `felhom-offsite`, which is what made the
-operator-approved 100% datastore-fill safe. The 13 GB rollback copy `/srv/pbs-felhom` on ep0 is
-intact.
-
-## Findings
-
-| # | Finding | Severity | Class |
-|---|---|---|---|
-| F-CRIT-1 | An app that fails to restart after a quiesce **never alarms**, on any channel | **HIGH** | DEFECT |
-| F-CRIT-2 | A failed offsite backup leaves a phantom snapshot that **resets tier freshness** (up to 7 days silent on real cadences) | **HIGH** | DEFECT |
-| F-A1 | A restore-test in progress makes a healthy backup report as FAILED, arms the breaker, pages the operator | MEDIUM | DEFECT (behaviour) / ARTIFACT (frequency) |
-| F-HUB | The hub dropped an event under concurrent load (`SQLITE_BUSY`), no retry, cause unnamed | MEDIUM | DEFECT |
-| F-LEAK | A **failed** restore-test cannot destroy its own scratch guest (403 `VM.Allocate`); leaks are never reclaimed | MEDIUM | DEFECT (root-caused by fault 18) |
-| F-REBOOT | A guest rebooted during its backup **does not come back** — shutdown completes, start never happens, no self-heal | MEDIUM | DEFECT |
-| F-DIAG | Four distinct offsite failure causes collapse into two operator-visible strings | LOW–MED | DEFECT |
-| F-OBS | `deadapp-check` leaves no positive observable on a default (info-level) box | LOW | DEFECT |
-| F-OPS | A manual `pct restore` inherits the source guest's binds (live data drive + another guest's credentials) | LOW | Operational |
-
-Both HIGH findings are in the same place: **the system's ability to tell you a backup did not
-happen.** Both cite the code and the comment that asserts the property the code does not provide.
-
-## What is now proven that was not before
-R-88 breaker arming **and its full ladder** (15m/30m/1h/2h/4h/4h-cap) · per-tier isolation under a
-real one-tier-fails case · `whole_guest_backup_failed` end-to-end with correct tier attribution ·
-R-97c operator-only routing verified against the hub DB (zero customer rows with `status='sent'`) ·
-`whole_guest_backup_recovered` + the R-68 pairing gate firing live · **`age_state=absent`** ·
-R-97b's suppression half · the crash-recovery unquiesce by an actual SIGKILL (1 s) ·
-**R-87 — the first restic restore round-trip ever performed**, byte-verified (6/7 sha256 identical,
-the 7th explained) · R-82 one-quiesce-two-tiers · single-flight on two independent paths.
-
-## Restore round-trips
-restic (R-87) · local vzdump → fresh CT · PBS offsite → fresh CT · corrupted snapshot → fails
-cleanly. `mount_parity` exact on both whole-guest tiers, `unprivileged: 1` preserved.
-
-## Fleet state
-**Healthy. Nothing left broken.** All four compression knobs reverted and verified; every fault
-unwound; no leaked scratch guests, nft rules, ballast files or clock skew; ep0 datastore clean with
-zero `.bad` chunks. demo-felhom 15/15 containers healthy, demo-hp 8/8.
-
-## Not tested (with reasons)
-Fault 4 (restic transport — four injection approaches defeated by guest-bridged networking; **the
-most valuable follow-up**, because F-CRIT-2 raises the same question for restic), fault 12 (host
-reboot — reasoned skip), and the agent's own DR bring-up path. Faults 6 and 8 were inconclusive for
-documented reasons. Faults 11 and 18 WERE run in the campaign's tail and both produced findings.
-
-**Campaign-caused outage, stated plainly:** fault 11 took demo-hp guest 9201 down for ~9m47s
-(the guest did not restart after a mid-backup reboot) until manually started. Fleet healthy after.
-
-## Note on repo conventions
-This run touched no `hub/`, `scripts/` or `website/` code, so none of the per-area CHANGELOGs has an
-entry — there is nothing shipped to log. The deliverable is the audit document plus this report.

REPORT-campaign9.md

@@ -1,85 +0,0 @@
-# REPORT — CAMPAIGN 9: the restore paths, proven (2026-07-28)
-
-**Overwritten** per the standing rule. **No production code shipped** — this was a proof campaign,
-and findings are recorded, never fixed inline. Full write-up:
-`documentation/audits/CAMPAIGN-9-restore-proof-2026-07-28.md`.
-Evidence: `DooPlex:~/campaign9/evidence/` (69 files, 221 MB, 7 collectors, written continuously).
-
-Fleet unchanged and healthy at close: hub v0.80.0, agent v0.110.0, controller v0.182.0 on both boxes.
-**`peti-felhom` was never touched.** The ep0 rollback copy `/srv/pbs-felhom` (13 G) is intact.
-
-## The headline — two never-proven restore paths are now proven
-
-Driven through the **real endpoints the UI posts to**, over https through traefik with a real session
-and CSRF token, on live hardware.
-
-| proof | result |
-|---|---|
-| **A1** — Tier-2 restore of ordinary app data (`paperless-ngx`, demo-hp) | 6 deleted files back **byte-identical** (`sha256sum -c` all OK) |
-| A1 — „A meglévő fájlok NEM módosulnak és NEM törlődnek" | 2 created files survived; 1 locally-edited file **not overwritten** (edit marker intact) |
-| A1 — app stopped/restarted and healthy | stop→copy→start in 39 s, `paperless-webserver` healthy |
-| A1 — data **usable by the app**, not just on disk | paperless resolved all 3 docs, checksums matched its own DB, and **served the restored bytes over its own HTTP API** at the exact pre-deletion sha256 |
-| **A2** — Tier-1 recovery-unit restore is a **distinct** path | `POST /backup/restore` → `RestoreFromRecoveryUnit`; ran end-to-end in 18 s, 1 volume restored, app healthy |
-| **A3** — restore after **total loss** (whole appdata dir `rm -rf`) | loss proven by doc download going **200 → 404**; restore returned **43/43 files byte-identical**, `documents_ok 16 of 16`, downloads back to 200 |
-
-The honest boundary A1+A3 together establish: **existing files are untouched; destroyed files return
-at their last-backup state.**
-
-## Findings — 3 defects, ranked (none fixed)
-
-| # | finding | severity |
-|---|---|---|
-| **C9-F1** | The Tier-2 restore button is offered for apps it can **never** restore (BookStack, Docmost). It takes a real app outage, restores 0 files, and reports „Nincs hiányzó fájl — minden fájl megvan a helyén." — while 156 MB of that app's data sits unread in the same copy | **HIGH** |
-| **C9-F2** | An app in a **crash loop never alarms on any channel**. `StateRestarting` is in no down-set, so the dead-app heartbeat printed *"180 scans … 0 currently down"* while the app had been looping for 9 minutes | **HIGH** |
-| **C9-F3** | An **interrupted offsite run** leaves an exclusive restic lock the existing self-heal cannot reach; the tier is dead until a human unlocks, and the operator is told *"unknown reason"* | **MEDIUM** |
-
-Two things were deliberately **not** filed as defects: a recovery-unit poisoning that the catalog
-sync self-healed within ~3 minutes (proven live — reporting it would have been reporting an
-artifact), and a `snapshot_id` that looked ignored but is documented as logging-only and confirmed
-so live.
-
-## Mechanisms confirmed working, live
-
-R-82's one-quiesce rule under mixed outcomes (2 tiers due, apps stopped **once**, per-target
-breaker); R-88's breaker (edge-triggered, one WARN, one event, three silent DEBUG skips, **no app
-thrash**); F-A1's contention deferral (409 → no breaker, no event, prompt restart — both sides of
-the seam captured in the same second); **F-CRIT-2's size filter against a real 1-byte phantom** on
-demo-hp, confirmed independently on ep0's filesystem; R-100's success anchor twice; **F-DIAG's
-sanitiser on the exact bare-hostname case that defeated its first version** (nothing raw reaches the
-hub event or the report); F-OBS's positive observable — which is precisely what made C9-F2 provable;
-F-LEAK's fenced destroy (no leaked `990000` guests across ~10 restore-tests).
-
-## Where it stopped, and what remains
-
-Stopped at the **end of Phase B**, plus Phase D item 10, then full recovery. Phase C item 6 (host
-reboot mid-backup) was deliberately not started — a large new fault class against boxes that are
-remote until ~08-02, and starting it would have meant rushing it or leaving the fleet unknown.
-
-**Approved but impossible:** Phase 0 cleared compressing the hub's `staleAfter` for R-100's
-threshold test. It is **not a knob** — `cmd/hub/main.go:552` passes `0`, selecting the compile-time
-`defaultOffsiteStaleAfter = 48h`. Compressing it needed a hub code change, which the campaign
-forbids. Reported rather than worked around. The no-code-change alternative (age the controller's
-reported `last_success` past 48 h and let the hub judge at its real threshold) is the recommended
-method next time.
-
-**The honest residue — still not proven:** Tier-1 **content** recovery after real loss (A2 ran on an
-intact app; A3 used Tier-2) — now the most valuable open item; host reboot mid-backup; three-way
-concurrency with GC; Scenario C live; `offsite_stale` actually firing; F-HUB `SQLITE_BUSY`.
-
-## Recovery
-
-Every config reverted from `evidence/config-before/REVERT.md`, each verified with a **positive
-observable**: agent cadences back to `0 / 302400 / 604800` on both hosts (`is-active` = active),
-windows back to `02:30`, `pvesm` shows `felhom-pbs active` on both, 0 campaign iptables rules on
-either host or guest, 0 scratch guests in the `990000` band, all stacks healthy on both boxes, and
-the offsite tier not merely unblocked but **proven working again** (`ok`, 1m35s, 8 snapshots).
-
-One benign residue: the in-memory R-88 breaker still holds a `felhom-pbs` failure count on each box.
-Its `until` is long past so it blocks nothing; it clears on the next successful backup or any
-controller restart (by design, not persisted). Clearing it would have cost another app outage for no
-benefit.
-
-**One operational lesson worth a runbook line:** a hand-run `docker compose up -d` in
-`/opt/docker/stacks/<app>` starts a Felhom app **without its secrets** — they are injected by the
-controller's `stackEnv` at start time, not stored in a `.env`. It turned a healthy docmost into a
-crash loop during recovery. Manual recovery must go through `POST /api/stacks/<name>/restart`.

REPORT-diag-immich-restore.md

@@ -1,71 +0,0 @@
-# REPORT — DIAGNOSE immich offsite restore (felhom.eu side: docs only)
-
-> Written as `REPORT-diag-immich-restore.md`, **not** the shared `REPORT.md`, per the CLAUDE.md
-> convention — `REPORT.md` is overwritten and currently holds the 2026-07-18 website refresh.
-> No code shipped in this run; findings only.
-
-**Date:** 2026-07-19 · **Box:** demo-felhom (felhom-pve guest 9201), controller 0.146.0, immich v3.0.3
-
-> **CLOSED IN CODE 2026-07-19 — controller v0.148.0.** Findings 1 and 2 shipped as R-43 (offsite
-> reconstitution: safety dump → stop → overwrite files → start → replay the snapshot's dump) and
-> R-44 (every run dumps before it captures; manifests carry `offsite_run_id` + `dumps_at`). Deployed
-> to demo 9201, healthy. **The §9 live acceptance has NOT run**, so no capability-map flip: the
-> offsite row is PARTIAL, the customer-restore row stays MISSING, R-3 stays DRAFT. Implementation
-> detail lives in `felhom-controller/REPORT.md`.
-
-## What ran
-
-A diagnosis of "immich offsite restore succeeds but photos do not reappear". No product code was
-changed: no restore/backup logic, no labels, no flashes, no `restic prune`/`forget`, no snapshot or
-escrow changes.
-
-## Outcome
-
-The restore did not fail. **It was never invoked on missing data, and could not have worked
-if it had been.**
-
-1. Viktor deleted the 11 photos in the immich UI to test offsite restore. A UI delete means
-   **trash**, not deletion — no file left the disk. Both „csak a hiányzó fájlok" runs merged
-   **0 files**, correctly, and flashed success. The test proved nothing.
-2. A *valid* test would also have failed: **no offsite path loads a database.** All three buttons
-   are file-only. Files would return; the library would stay empty.
-3. The shipped dump is additionally stale by design — from the 02:30 local run, never refreshed
-   before a manual push. Probed: **`asset: 0`, `user: 0`, `album: 0`**.
-
-**Photos:** left in trash at Viktor's instruction (recovery not wanted). All 11 files verified
-present on disk and all 11 rows intact, so an ordinary un-trash recovers them until immich's
-30-day auto-purge.
-
-**Answer to "can a customer trust same-day offsite?"** For a DB-indexed app: **no — files come
-back, content does not.** The backup half is honest; the restore half cannot reconstitute the app.
-
-## Decisive evidence
-
-- `updatedAt` == `deletedAt` on all 11 asset rows ⇒ **no restore operation touched the DB.**
-- Unit dump `immich-postgres.sql`, 51 954 452 B, mtime **02:30 CEST** ⇒ `asset: 0 / user: 0 /
-  album: 0`. The 52 MB is entirely immich's shipped geodata reference tables. It predates the admin
-  user (07:56:25) and the photos (07:57).
-- **Latent hazard:** had a full restore loaded that dump, it would have written an empty DB over the
-  live one, destroying the trashed rows that were the only surviving recovery path.
-
-## Files written
-
-| File | Change |
-|---|---|
-| `documentation/audits/DIAG-immich-restore-2026-07-19.md` | new — timeline, evidence, source mechanism table, 7 findings |
-| `documentation/backlog/ROADMAP.md` | **R-43** (P2-HIGH, offsite restore cannot restore a DB) + **R-44** (P2-HIGH, manual push ships unrefreshed dump) |
-| `documentation/architecture/00-capability-map.md` | customer-restore row **stays MISSING**, gains the finding + a method note for R-3; offsite-restore row flagged *scope contested* |
-
-## Needs Viktor
-
-1. **Capability-map ruling (line 61).** The offsite-restore row claims PROVEN-LIVE citing
-   "immich end-to-end from offsite alone" (`CAMPAIGN-6D`). That phrase is contested by this
-   diagnosis. I flagged it rather than downgrading a proven row — did 6D's accept leg exercise the
-   DB half, or only the file half?
-2. **Optional real red-proof.** Emptying the trash would create genuine data loss and convert the
-   dump-probe inference into a live proof. Offered, **not run** — irreversible, and the probe
-   already settles it.
-3. **Unreconciled: the 704.6 MiB figure.** Measured 126 MB total on the library storage. If
-   704.6 MiB came off a controller Storage page, that gap is its own defect.
-4. **Orphaned pre-v3 tree** `dccc13fe…` (~55 MB across upload/thumbs/encoded-video) stranded by the
-   immich 2→3 redeploy — worth a sweep policy for major redeploys.

REPORT-diag-sharing-2026-07-20.md

@@ -1,53 +0,0 @@
-# REPORT — Megosztás diagnosis (2026-07-20)
-
-Topic-scoped report (parallel-session rule: shared `REPORT.md` untouched).
-
-**Run:** RUNBOOK "Megosztás diagnosis — SMB unreachable from Mac + sharing-page reload loop".
-Read-only diagnosis. **No code changes, no version bumps, no builds, no restarts.**
-
-**Deliverable:** `documentation/audits/DIAG-sharing-2026-07-20.md`.
-
-## Verdicts
-
-* **Reload loop — ROOT-CAUSED (HIGH).** `sharingStatusHandler` (`sharing_handlers.go:246`, added in
-  `b5d78d1`, controller v0.147.0, 2026-07-19) coerces `idle` → `running` whenever the samba
-  container is alive. `sharing.html` L320–326 treats `running` as a one-shot job-success edge and
-  calls `location.reload()` 1.2 s later. The first `tick()` fires synchronously on every page load,
-  so the page reloads forever. Unconditional for any customer with sharing enabled — the Megosztás
-  page is currently unusable. Proven live: 6 consecutive `/sharing/status` polls all returned
-  `{"phase":"running","running":true}`, and the controller log shows **no ensure job ran at all**,
-  so the phase is manufactured by that line rather than left over from a stuck job.
-* **`smb://192.168.0.162` — ROOT-CAUSED.** `.162` is the Proxmox host and never was an SMB endpoint.
-  smbd runs in guest 9201 and binds `192.168.0.104:445`. `nc` from the host: `.104:445` **open**,
-  `.162:445` **refused**. Wrong target, stale Finder favourite.
-* **`smb://FELHOM` — OPEN, narrowed.** NetBIOS resolution works on the wire
-  (`nmblookup -B 192.168.0.255 FELHOM` → `192.168.0.104 FELHOM<00>`), but the stack advertises **no
-  mDNS/Bonjour** (nothing on udp/5353; the R-6 spike selected `smbd + nmbd + wsdd` only) — the
-  mechanism macOS Finder prefers. Closing this needs one probe from the Mac (`smbutil lookup FELHOM`
-  / `dns-sd -B _smb._tcp`), listed in the audit's Mac test matrix.
-* **H2 (container down/crash-looping) — RULED OUT** at the first probe: `felhom-samba` Up 3 h, clean
-  logs, smbd/nmbd/wsdd all bound as `infra/samba.go` intends, live `smb.conf` matches the renderer
-  with no baked address literal, no PVE firewall in the path.
-
-## Findings (full table + evidence in the audit doc)
-
-S-1 HIGH reload loop (XS fix: latch a `sawInFlight` flag in the JS; red-proof required) ·
-S-2 MED the UI never shows the connect address, only `\\FELHOM` — customers guess IPs ·
-S-3 MED no mDNS advertisement (image slice, needs republish) ·
-S-4 LOW ensure-job phase never resets — fold into the async-job-feedback roadmap item ·
-S-5 INFO the guest's LAN IP is DHCP, so any displayed address must be read live, never cached.
-
-Recommended packaging: S-1 as an immediate patch task (it bricks a shipped page), S-2 alongside it
-if the guest IP is reachable from the sharing handler, S-3 as its own slice.
-
-## Also noticed
-
-The **remote site's LAN is `192.168.0.0/24` — the same prefix as the DooPlex home LAN** that the
-Tailscale subnet router advertises (`192.168.0.180` shows `FAILED` in felhom-pve's neighbour table).
-A successful `ping 192.168.0.162` therefore does not by itself prove the Mac is on the remote
-segment; the Mac matrix starts by confirming which network it is on.
-
-## Actions taken
-
-None. Every command was a read, except a `POST /login` to obtain a session for the status polls.
-No secrets are recorded in either document.

REPORT-e2d.md

@@ -1,85 +0,0 @@
-# REPORT — R-111 fixed, then E-2 proven on a fresh box (2026-07-29)
-
-Two phases in one session. Full evidence: `documentation/audits/E2D-fresh-vm-2026-07-29.md`.
-Root `REPORT.md` untouched.
-
-## Phase 1 — R-111: the Day-0 channel now serves the current software
-
-A Phase 0 gate earlier the same day stopped the E-2d run before any VM existed: a fresh box would
-have installed **agent 0.96.0 + controller 0.161.0**, ~17 and ~24 releases behind `main`.
-
-| | Before | Now |
-|---|---|---|
-| agent (Gitea generic) | 0.96.0 | **0.113.0**, sha `5f3247f7…`, round-trip verified |
-| golden (Gitea generic) | 0.161.0 | **0.185.1**, sha `dba00f3e…`, embeds controller 0.185.1 |
-| hub `min_agent` | 0.93.0 | **0.113.0** (what controller v0.185.0 declares) |
-
-Bake clean on every marker: `Result=success`, overlay2, **all three mounts in the archive**, 0
-FATAL/exclusions, HTTP 201, token-leak grep 0. GL-1 teardown: guest 9100 purged, secrets shredded,
-drill disk restored to `virgin`. Agent + golden moved in **one** manifest POST so it never vouched a
-new agent against an old golden. `min_agent` verified zero-impact first (all three enrolled hosts
-already at 0.113.0). Global floor deliberately **not** raised — the golden now bakes 0.185.1.
-
-Commit `3dff357`.
-
-## Phase 2 — the E-2d run, full ISO/PAIRING route
-
-Nested PVE VM on demo-hp, one disk, outside the `felhom` pool. Bind → running controller in
-**3 m 35 s**. The install fetched exactly the artifacts published an hour earlier and restored
-`vzdump-lxc-9100-2026_07_29-12_37_56` — the golden baked 20 minutes before. The publish train is
-proven end to end on a real install.
-
-| Claim | Verdict |
-|---|---|
-| **C1** host-install 1.22.0 completes a real install, rc=0 | ✅ **PROVEN** |
-| **C2** Case B fires naturally | ✅ **PROVEN** — both DEGRADED lines verbatim, `local_backup_target=local`, install did not abort |
-| **C3** degraded banner renders **to a customer** | ⚠️ **PARTIAL** — API byte-exact; **no UI consumer exists** → **R-112** |
-| **C4** offer appears and moves the target | ⚠️ **PARTIAL** — decline path, `restart_required:true`, no self-restart, E-2a wrapper, healthy-renders-nothing all PROVEN at API level; offer equally invisible → **R-112** |
-| **C5** `backup_target_absent` end to end | ❌ **FAILED** — zero events on any channel → **R-113** |
-
-## The three findings
-
-**R-112 (P1)** — E-2's banner and offer have **no UI consumer**. The endpoint returns byte-exact copy;
-`grep 'backup-target'` across every `*.html`/`*.js`/`*.css` → **0 hits**, and no page handler injects
-the state. Decisive contrast: templates fetch **18** distinct `/api/storage/*` endpoints;
-`backup-target` and `backup-target/assign` are the only two with zero references. v0.185.1 fixed the
-router mount and stopped one layer short of the render. Fifth instance of seam-built-but-never-wired.
-
-**R-113 (P1)** — the drive-absent gate **cannot fire on device loss**. `planDriveGates` reads presence
-from `BoundUnderParent` = "is this path in the guest's mountinfo". The raw mount is a device-bound
-systemd unit and dies; **the agent's own bind is not device-bound and outlives the device**, so the
-gate sees "present" forever. Live: agent said `enrolled drive absent by UUID` every 20 s for 4½
-minutes, controller logged **0** `[gate]` lines, hub got **zero** events — neither the specific nor the
-generic one. Sixth instance of the class, one layer deeper: E-2b wired the seam to a condition that
-cannot occur.
-
-**R-114** — on target-drive loss the message says the backup is *"on the same disk as the system"*
-(false) and offers **the drive that just vanished**. Invisible today only because of R-112 — so
-**R-114 must be fixed before R-112 is wired.**
-
-Also filed as a **second instance under R-110** (not a new ID): host-install fetches **nine** files
-from `raw/branch/main` and the hub vouches a sha for **one**; E-2a's wrapper is installed 0755 to
-`/usr/local/sbin`, root-fenced in sudoers, validated only by `bash -n`.
-
-## Record
-
-- `OPEN-ITEMS.md` — **R-112/R-113/R-114 opened**; E-2d re-stated with results and left open for the
-  residue; E-2's "NOT yet live-proven" list resolved into proven / known-broken; R-94 fully unblocked;
-  R-110 extended. The drill-cleanup row was opened and then **closed the same session** once the
-  teardown completed, so it is not carried in the register.
-- `ROADMAP.md` — R-112/R-113/R-114 under P1; R-111 marked SHIPPED.
-- **`architecture/00-capability-map.md` not touched** — for two reasons: the customer-facing legs are
-  broken rather than proven, and the map has **no E-2 / backup-target rows at all** (worth noting
-  against the ROADMAP's coupling rule).
-
-## Teardown
-
-VM destroyed, scratch storage removed, **`pvesm status` after == before** (`local-lvm` 38.77 %,
-byte-identical), guest 9201 and drill-r50 untouched. **Hub records removed — teardown complete.** The delete was correctly refused at four gates while the host still read ONLINE; once the destroyed host aged to DOWN (`delete-impact` → `deletable:true`) the documented cascade ran and completed: host deleted, PBS tenancy deprovisioned, claim reset, residue purged. Verified after: **0** `e2d` occurrences on the hosts page, fleet unchanged. The one purged `appliance_registrations=1` was this run's own appliance; the unrelated stale 2026-07-25 appliance (`206c8838…`) was not touched by the cascade — the operator removed it separately.
-
-## One human step, and a premise correction
-
-The runbook's §5.1a operator STOP (the bind) is **retired** — CC did it. But E-2d's premise that a
-fresh install yields a CC-drivable claimable customer is **wrong**: the claim code is bcrypt-hashed and
-email-only, and the gate covers everything except `/claim`, `/api/health`, `/static/`. One operator
-relay of the emailed code was required — which also proved the claim flow end to end.

REPORT-ep0-datastore-volume-2026-07-27.md

@@ -1,92 +0,0 @@
-# REPORT — ep0 PBS datastore relocated onto the 100 GB volume (2026-07-27)
-
-**Class:** supervised operational run (RUNBOOK execution). **No code changed. No version bump.**
-Written as `REPORT-<topic>.md` per the parallel-session rule — the shared `REPORT.md` was not touched.
-
-**Full record with all evidence:** `documentation/runbooks/RUNBOOK-ep0-datastore-volume-2026-07-27.md`
-
----
-
-## Outcome: DONE and verified
-
-`felhom-offsite` now lives on a dedicated 100 GB Hetzner Cloud Volume instead of ep0's 40 GB root disk.
-
-| | Before | After |
-|---|---|---|
-| Path | `/srv/pbs-felhom` (root disk) | **`/mnt/pbs-datastore`** (volume) |
-| Datastore total | 37.2 GB | **98 GB** (hub gauge: 97.9 GB) |
-| Used | 28.9 % | **13 %** (hub gauge: 12.6 GB, 13 %) |
-| Headroom to the 80 % warn | 19 GB | **≈65 GB** |
-| Additional customers before warn | ≈2 | **≈7–13** |
-
-Datastore **name unchanged** — the PBS-DR descriptors, per-box storage ids, ACLs and namespace
-layout that R-39/R-82 made self-healing are untouched.
-
-**Window:** 06:58 → 07:19 UTC (PBS down 07:00 → 07:17). **Nothing was deleted.**
-
-### Acceptance evidence
-
-| Gate | Result |
-|---|---|
-| Copy integrity | 13,242,207,822 = 13,242,207,822 B · **9,748 = 9,748 chunks** · 7 = 7 snapshots · `backup:backup` · itemised dry-run **0 lines** |
-| Snapshot counts per ns | `demo-felhom` 2=2, `demo-felhom-01` 3=3, `demo-hp` 2=2 |
-| atime semantics | `rw,relatime,discard` — **`relatime` present, `noatime` absent** (GC correctness) |
-| Verify job | `TASK OK`, 3/3 groups, forced re-verification of every snapshot, **0 errors** |
-| §6 mount guard | **refusal observed** — `Job … failed with result 'dependency'`; mountpoint stayed empty |
-| §8 restore round-trip | `source_tier: pbs`, `pass: true`, `mount_parity: ok`, clean teardown, 12m1s |
-
----
-
-## Three findings the operator should act on
-
-1. **`scratch` datastore is configured at a path that does not exist** (`/srv/pbs-scratch`).
-   Pre-existing, not caused here, but now logs `ENOENT` on every PBS start. This is the PRIME RISK
-   shape ("reports fine, is not there") already live in the config. **Decision needed:** remove the
-   stanza or create the directory.
-
-2. **The runbook's §6 acceptance test proves the wrong proposition.** `RequiresMountsFor` is a
-   mount-first ordering guarantee, not a refusal — systemd silently *re-mounts* an unmounted volume
-   and PBS then starts safely. The test only bites when the device is genuinely unavailable, which
-   is how it was re-run and passed. **Amendment recommended in the runbook record.**
-
-3. **§11 — storage box `u629193` is NOT simply unused.** No live backup path references it (no
-   datastore, no restic repo, no fstab, no `known_hosts` pin; R-17 already deleted `u629193-sub1`),
-   **but ep0 carries an enabled, currently-mounted sshfs unit** `mnt-pbs\x2dstoragebox.mount` →
-   `/mnt/pbs-storagebox`, holding spike leftovers. Disable and remove that unit before deleting the
-   box, or ep0 logs a failed mount every boot. **The deletion is the operator's console click.**
-
-## Deviations from the runbook as written
-
-- **The volume arrived already formatted and mounted** by Hetzner at `/mnt/HC_Volume_106469259`
-  (§2 assumed neither). Operator ruled: reformat + repath. The 5 % reserve was reclaimed (`-m 0`).
-- **§8 ran on demo-felhom, not demo-hp** — DooPlex holds no SSH key for demo-hp (the G1 gap). Same
-  tier, same relocated datastore, larger archive.
-- **The window was contended** by a stale 10-minute restore-test cadence on demo-felhom: the config
-  had already been reverted to 3.5 days on disk, but the cadence is read once at daemon start and
-  `NRestarts=0`. Restarting the agent applied it (`cadence=84h0m0s`). The in-flight test was allowed
-  to finish rather than aborted.
-
-## Process errors made during this run (recorded deliberately)
-
-- `rsync -aHAX` **OOM-killed** ep0 (3.7 GB RAM, no swap). Cause: a PBS `.chunks/` tree pre-creates
-  all 65536 shard dirs → 75,341 inodes, and `-H` retains the whole inode map. `-H` was dropped only
-  after **proving** no hardlinks exist (`-links +1` → 0; max link count → 1); PBS references chunks
-  by digest, never by hardlink.
-- `/usr/bin/time -v` is not installed on ep0 → exit 127, rsync never ran, and a `| grep … || true`
-  wrapper swallowed it while printing a success-looking line.
-- `rsync --version | head -1` reported a working rsync 3.4.1 as "missing" — **the §12 pipe-into-head
-  trap, fourth recorded instance in this project.** Both fixed by capturing the command's own `$?`.
-
-## Deferred
-
-1. **Old copy retained** at `/srv/pbs-felhom` (13 GB, 9,748 chunks) as the rollback. Rollback is a
-   two-line `datastore.cfg` revert. Reclaim only after a new weekly offsite backup lands on the
-   volume, with explicit go-ahead.
-2. **GC not run** — now unblocked by the round-trip, but left for a separate deliberate run. No GC
-   schedule is configured on this PBS at all.
-3. ~~Hub PBS-DR capacity gauge not re-read.~~ **CLOSED — verified correct.** The hub operator UI
-   (Offsite → PBS DR) reports `felhom-offsite (ep0)` at **97.9 GB capacity, 12.6 GB used, 13 % full**,
-   agreeing with the on-box `df`. The gauge follows the datastore's configured path, so the move
-   needed no hub-side change and the suspected "wrong filesystem" bug does not exist.
-4. **ep0 has no swap** (temporary 4 GB file removed; box left as found). Worth a small permanent
-   swapfile — outside this runbook's scope.

REPORT-fcrit1-fa1.md

@@ -1,63 +0,0 @@
-# REPORT — F-CRIT-1 + F-A1 fixed (controller v0.179.0, 2026-07-28)
-
-Docs here. Implementation, all six red-proofs and the full live replay live in
-`felhom-controller/REPORT.md`. The campaign that found both:
-`documentation/audits/CAMPAIGN-8-backup-restore-2026-07-27.md`.
-
-## What changed
-**F-CRIT-1** — an app that failed to restart after a quiesce never alarmed, for two independent
-reasons, either of which alone kept it dead: `restartAll` returned nothing (the failure was logged
-and dropped), and `classifyRunStates` whitelisted `StateStopped` on invariant I1 ("the user stopped
-it") — which the quiesce loop had made false by stopping stacks the same `docker compose down` way.
-A failed restart and a user stop are the *same* Docker state; the only difference is that the loop
-tried and could not, now surfaced by `Loop.FailedRestarts()`.
-
-**F-A1** — HTTP 409 is the agent's single-flight gate refusing while a restore-test holds it, not a
-failure. It is now contention: the tier stays DUE, is dropped before anything stops, and unending
-contention raises its own **BLOCKED** signal rather than going silent.
-
-## Bounds, justified against measured reality
-- `contentionRetryAfter` **15m** — longest restore-test observed on the fleet is 12m01s; the agent's
-  local restore-test wait is 10m. Caps app-stop churn at 4/hour instead of 12/hour.
-- `contentionAlarmAfter` **3h** — the agent's own PBS restore-test task is capped at 120 minutes, so
-  contention outliving that is a *stuck* gate, not a busy one. 3h adds margin and is 15× the longest
-  contention actually observed.
-
-## Verified live, with the hub DB as arbiter — not from logs
-Same box, same day, same event type; the only difference is 409 versus a genuine error:
-
-| injection | operator emails (demo-hp) |
-|---|---|
-| **409 contention** | 8 → **8** (none) |
-| **real transport failure** | 8 → **9** |
-
-And for F-CRIT-1: the failed restart alarmed **9 seconds** after grace expiry with the dashboard
-banner naming the `(stopped)` state, while a **deliberate** user stop on the same box stayed silent
-through **9** dead-app scans (the positive observable that the silence is suppression, not a dead
-detector).
-
-## The rule this arc earned
-Added to **both** copies of `CLAUDE.md` (live + `documentation/runbooks/workspace-CLAUDE.md`):
-**a comment asserting an invariant needs a test pinning it, or it is a wish.** Six instances in this
-project have shipped guarantees the code did not provide — `EffectiveProtected`, `newestArchiveOn`,
-the R-97a operator-only claim, `classifyRunStates`' I1, `inflight.go`'s defer claim, and
-`quiesce.go`'s spurious-failure claim. Two were found only on live hardware, and one of those had a
-green, red-proofed test suite over a production path broken two independent ways.
-
-Corollary recorded with it: prefer a test that asserts the **consequence** (does the alarm fire?)
-over one that asserts the **mechanism** (does suppression expire?). R-97b's Scenario F proved the
-mechanism; the consequence was still broken.
-
-## Docs touched
-- `documentation/backlog/OPEN-ITEMS.md` — F-CRIT-1 and F-A1 → SHIPPED + PROVEN-LIVE.
-- `documentation/audits/CAMPAIGN-8-backup-restore-2026-07-27.md` — both rows struck through, closing
-  section added. **All three of the campaign's alarm findings are now closed** (F-CRIT-1, F-CRIT-2,
-  F-A1).
-- `documentation/runbooks/workspace-CLAUDE.md` — the invariant rule.
-
-## Still open, highest first
-**Fault 4** (restic transport interruption) — four injection approaches were defeated by
-guest-bridged networking, and it is now the most valuable follow-up: F-CRIT-2 answered the phantom
-question for PBS and left the identical question open for restic. Then **R-99** (prune never removes
-phantoms) and **F-LEAK** (a failed restore-test cannot destroy its own scratch guest — observed
-again during this work).

REPORT-fcrit2.md

@@ -1,40 +0,0 @@
-# REPORT — F-CRIT-2 fixed: a failed backup no longer looks like a fresh one (2026-07-28)
-
-Scope: `felhom-agent` v0.105.0 → **v0.106.0**. Docs here. Implementation detail and the full live
-re-test live in `felhom-agent/REPORT.md`; the campaign that found it is
-`documentation/audits/CAMPAIGN-8-backup-restore-2026-07-27.md`.
-
-## What changed
-`NewestArchiveTime` counted an aborted PBS upload (1 byte, manifest-less, and NEWEST) as a
-successful backup, so the tier read fresh, went **not due**, and was never retried — seven days of
-silence on the real 168h cadence, invisible to both the R-88 breaker (defers only *due* tiers) and
-the hub deadline monitor (reads the same freshness). It now counts only *plausibly complete*
-entries via a measured 1 MiB floor; undecidable ⇒ not counted.
-
-**Size is the only tier-agnostic discriminator.** `verification` and `encrypted` are absent on every
-local (dir) archive AND on a good PBS snapshot until `verify-new` catches up — gating on either
-would have rejected 100% of local backups and produced fleet-wide backup thrash. That inverse risk
-is a first-class test, red-proofed by making the filter reject everything.
-
-## Verified live, not just in unit tests
-Campaign fault 2 was replayed against the fixed agent on demo-hp — phantom created, rejected and
-announced once; the tier correctly reported DUE and backed up (4,359,968,099 B landed); and the
-inverse showed **no thrash**, with 91 scheduler ticks as the positive observable that the loop was
-alive rather than dead.
-
-## Settled along the way — no retention bug
-Server-side prune does **not** count phantoms toward `keep-last`: a dry-run against three real
-snapshots plus a phantom retained two real ones plus the phantom. The feared "two phantoms ⇒ zero
-real backups" does not occur. Prune never removes them either, so they accumulate one per aborted
-upload — filed as **R-99** (LOW, hygiene), not as a retention bug.
-
-## Docs touched
-- `documentation/backlog/OPEN-ITEMS.md` — F-CRIT-2 → SHIPPED+PROVEN-LIVE; **R-99** filed;
-  **F-CRIT-1** filed as READY-HIGHEST (Campaign 8's other HIGH finding, untouched here).
-- `documentation/audits/CAMPAIGN-8-backup-restore-2026-07-27.md` — F-CRIT-2 row struck through and
-  a closing section added.
-
-## Still open, highest first
-**F-CRIT-1** — an app that fails to restart after a quiesce never alarms, for two independent
-reasons. Then fault 4 (restic transport interruption), which this fix makes more pointed: the
-phantom question is now answered for PBS and still open for restic.

REPORT-freboot-fleak-fobs.md

@@ -1,140 +0,0 @@
-# REPORT — F-REBOOT + F-LEAK + F-OBS, and two investigations (2026-07-28)
-
-Scope in this repo: **`scripts/felhom-host-install.sh` v1.20.0 → v1.21.0** (which is where F-LEAK's
-*actual* fix lives), plus the Campaign 8 audit doc and `OPEN-ITEMS.md`. Written as
-`REPORT-freboot-fleak-fobs.md` so the shared `REPORT.md` is not clobbered.
-
-Code companions: `felhom-agent` v0.106.0 → **v0.110.0**, `felhom-controller` v0.179.0 → **v0.180.0**.
-
-**Correction to this repo's part of the story:** v1.21.0's band-scoped ACL is *not* the final F-LEAK
-fix. It works, but only **once per slot** — PVE's destroy path calls
-`AccessControl::remove_vm_access($vmid)` (`API2/LXC.pm:906`), which deletes every ACL at `/vms/<vmid>`
-(`AccessControl.pm:1898`), so **the grant is consumed by the operation it authorises**. Found by counting
-ACL rows after the first successful teardown (`/vms/990000` → 0 grants), not by reasoning about it. The
-durable fix is agent **v0.110.0**'s band-scoped fenced destroy; v1.21.0 remains valuable because it makes
-the common case need no privileged call, and it is now the *first* of two layers rather than the only one.
-
-## Baselines (reconfirmed, not copied)
-`felhom.eu d0cec9d`, `felhom-agent af1c21a`, `felhom-controller fb91c8d`, all clean. Agent `0.106.0`
-and controller `0.179.0` live on both demo boxes.
-
----
-
-## host-install v1.21.0 — F-LEAK, and why the fix landed *here* rather than in the agent
-
-**The finding.** A restore-test whose restore **fails** leaves a scratch guest the agent cannot destroy
-(`403 missing privilege VM.Allocate`), so a half-restored guest holds its disks until a human removes
-it and the 10-slot scratch band shrinks silently.
-
-**The cause is structural, not a missing privilege in the role.** `FelhomAgentGuest` is granted at
-`/pool/felhom`, and **a guest joins that pool only when its restore completes**. A failed restore
-therefore produces a guest that exists, is in no pool, and is out of the token's reach entirely.
-
-**The first fix was wrong, and its own live replay is what proved it.** Agent v0.107.0 shipped a
-teardown fallback that adopted the stranded guest into the pool and retried — reasoning from
-`Pool.Allocate` on `/pool/felhom`. It fired exactly as designed and PVE refused it:
-
-```
-ERROR restore-test: pool adoption failed; left for Recover  vmid=990000
-      err="proxmox: PUT /pools/felhom -> HTTP 500: permission denied at /vms/990000 (missing privilege ...)"
-```
-
-`PUT /pools/{pool}` **also** requires `VM.Allocate` on the VM being added. **Pool membership cannot
-bootstrap its own authority.** Removed in agent v0.108.0 rather than left in place — a path that
-provably cannot work is worse than none, because it reads as a fix.
-
-**What shipped instead.** `apply_scoped_acl` now grants `FelhomAgentGuest` at each `/vms/<id>` in
-`PVE_SCRATCH_VMID_MIN..PVE_SCRATCH_VMID_MAX` (990000–990009 — the band the restore-test already picks
-from), to **both** the user and the token, because the privsep-intersection rule applies here as
-everywhere.
-
-Two supporting changes, both load-bearing rather than tidy-up:
-- **`remove_scoped_acl` deletes the band grants before the role delete.** PVE refuses to delete a role
-  still referenced by any ACL, so omitting this would have broken the uninstall — a failure that would
-  only surface on a decommission.
-- **`step_verify` asserts the band grants.** A missing one is otherwise invisible until a restore-test
-  *fails*, which is precisely the case that leaked a guest in the first place.
-
-### Why the grant is still not a widening — proven live, at the seam the defect lives in
-
-A real PBS restore to `990000` **without `--pool`** reproduced the exact stranded state
-(`990000 stopped`; `felhom pool members: [9201]`; `990000 in pool: False`). Then, with the agent's own
-token, same guest, minutes apart:
-
-| | `DELETE /nodes/<node>/lxc/990000` |
-|---|---|
-| **grant removed** (the original defect) | `403 Permission check failed (/vms/990000, VM.Allocate)` — guest still present |
-| **grant restored** (the fix) | `200 UPID:...:vzdestroy:990000:felhom-agent@pve!agent` — guest gone |
-
-And it still cannot reach anything else:
-
-| target | result |
-|---|---|
-| `/vms/990010` (one past the band) | **403** `Permission check failed (/vms/990010, VM.Allocate)` |
-| `/vms/100` | **403** same |
-
-`990010` does not exist and PVE **still** answered 403 rather than "does not exist" — so PVE evaluates
-**permission before existence**, which makes these genuine authorization refusals rather than artifacts
-of a missing guest. Granting at `/vms` was considered and rejected: it would authorise destroying every
-guest on the box, including a co-tenant's.
-
-Applied on **both** demo boxes (demo-hp and demo-felhom) so the live fleet matches the installer.
-
-**A careless step of mine, recorded rather than buried.** The probe loop also issued a live `DELETE`
-against running guest **9201**. It was refused with `500 container is running` — but the *permission
-check passed* (9201 is a pool member by design), so had the guest been stopped I would have destroyed
-the live demo guest. The scratch-band probes were the safe ones; 9201 had no business in that list.
-
----
-
-## Documentation changes
-
-- **`documentation/audits/CAMPAIGN-8-backup-restore-2026-07-27.md`** — F-REBOOT, F-LEAK and F-OBS
-  written up as FIXED with their live evidence, including F-LEAK's refuted first attempt (recorded
-  precisely *because* it looked right), plus a new **§6b** for the follow-up investigation.
-- **`documentation/backlog/OPEN-ITEMS.md`** — three findings closed, **R-100** filed.
-
----
-
-## R-100 — the investigation's finding, deliberately NOT fixed
-
-**A restic offsite tier that fails every night never goes stale on the hub.** This is **F-CRIT-2's
-defect class one layer up and on the other tier** — a *failed* run resetting the freshness clock — and
-it was found by asking whether F-CRIT-2's shape existed anywhere else.
-
-Both halves verified in the source, not inferred:
-- **Controller:** `o.LastRun = time.Now()` is set **unconditionally** at
-  `controller/internal/backup/offbox.go:716`, *outside* the `runErr` branch. The failure is recorded
-  faithfully, but into a different field — `o.LastStatus = "error"` at `:725`.
-- **Hub:** `isStale()` reads **only** `off.LastRun` (`hub/internal/monitor/offsite.go:120`, `:127`,
-  `:131`) and never consults `LastStatus`.
-
-So a nightly restic run that fails every night keeps `LastRun` fresh, `isStale` is permanently false,
-and the staleness alarm never fires — with no successful offsite backup having occurred at all.
-
-**Scope of the silence, stated precisely rather than dramatically.** `LastStatus` *does* reach the hub —
-it is parsed into the report struct and **only logged** (`offsite.go:270`); it drives no checker and no
-notification. The controller's own guest UI surfaces `LastStatus="error"`, so the failure is visible to
-someone who looks. What is missing is the **push**: the operator's fleet-wide alarm plane is silent,
-which is the plane that matters for an unattended appliance.
-
-Not fixed, per this task's investigation-only scope. Fix direction: gate staleness on the last
-*successful* run rather than the last attempt — exactly what F-CRIT-2's `NewestArchiveTime` fix did for
-the PBS tier.
-
----
-
-## Verification
-
-`bash -n scripts/felhom-host-install.sh` clean. The ACL behaviour was verified **live on real hardware**
-rather than by dry-run, since the whole finding is about what PVE's authorizer actually does — and the
-live run is what refuted my first design.
-
-## Fleet state
-Agent **0.110.0** (with the updated sudoers) and controller **0.180.0** on both demo boxes, all healthy.
-Scratch-band ACLs at 20 rows on both — re-applied after the attempt-2 destroy consumed one. No leftover scratch guests. demo-hp's `restore_test_cadence_seconds` reverted **600 → 302400**
-(a bounded change made for the replay).
-
-`felhom.eu`: this repo has a **foreign uncommitted WIP file** (`documentation/PROMPT-TEMPLATE.md`) from
-another session in the shared worktree. Left untouched; my commits staged explicit paths only, per the
-never-`git add -A` rule.

REPORT-iso-branding-website-grid-2026-07-19.md

@@ -1,119 +0,0 @@
-# REPORT — ISO boot branding + single-entry GRUB menu (R-38) · website grid restored (2026-07-19)
-
-> `REPORT-<topic>.md` per this repo's parallel-session rule: another session was writing in this
-> clone tonight (CAMPAIGN 7 / `DIAG-immich-restore-2026-07-19.md`), so the shared `REPORT.md` is left
-> untouched.
-
-Parts 1 and 2 of the polish train. Parts 3 and 4 landed in `felhom-agent` and `felhom-controller`;
-see their own `REPORT.md`.
-
----
-
-## Part 1 — scripts v1.22.0: GRUB branding + single-entry safety (R-38)
-
-### What shipped
-
-Every ISO is now **repacked** after `prepare-iso`. `mkimage-surgery.sh` → **`iso-repack.sh`**:
-branding and the slice-B loader swap need the same extract → modify → re-master cycle, so they share
-one pass instead of re-mastering twice. **The mkimage recipe is untouched.**
-
-**The safety half — the one that matters.** The stock PVE menu offers *Graphical*, *Terminal UI*, a
-serial variant, and an **Advanced Options** submenu holding two `nomodeset` entries, three debug
-entries, *Rescue Boot*, memtest and *UEFI Firmware Settings*. Every one of those reaches the
-**manual** installer, whose first question is which disk to wipe. They are **not emitted** — not
-hidden, not password-gated. What ships is one entry, „Felhom telepítés", default, 5 s.
-
-**Boot behavior is unchanged.** The `linux`/`initrd` lines are lifted **verbatim at repack time**
-from the ISO's own *Install Proxmox VE (Automated)* entry rather than frozen into a copy in this
-repo, so a PVE bump that moves the kernel path or edits the append line tracks automatically. The
-build **fails** if they cannot be found, if the append line has lost `proxmox-start-auto-installer`,
-or if `auto-installer-mode.toml` is absent — that last one because without it the single
-Felhom-labelled entry would boot a *manual* installer, i.e. exactly what this change prevents.
-
-**Gates, then a re-check against the shipped artifact.** The rendered menu is asserted to have
-exactly 1 `menuentry`, 0 `submenu`s and no *live* reference to
-`proxtui`/`proxdebug`/`nomodeset`/`Rescue Boot`/`memtest`/`fwsetup` (comments are stripped first —
-the template's header names the dropped entries deliberately). Then the menu and theme background
-are read back **out of `final.iso`**, not out of the extract tree.
-
-**The boot card.** `grub/generate-grub-background.sh` letterboxes `website/assets/og-image_2.png`
-onto a 1024×768 gfxterm canvas at repack time (ImageMagick added to the assistant image), so the boot
-screen has **one source** and not a second pre-rendered PNG to drift. The card's own subtle grid
-(measured: 4px lines of `#0D131A` on `#0D1117`, pitch 131px) is continued across the letterbox fill
-**phase-locked** to where the card's grid lands, so the fill is seamless instead of a 500px square of
-grid floating in flat navy. The generator refuses a source whose geometry no longer matches the
-measured constants — a swapped asset would misplace every line, and that only shows up on a boot
-screen nobody re-checks. Menu positioning needs a gfxmenu theme (plain `background_image` cannot move
-the menu off the wordmark), so `grub/felhom-theme.txt` puts it in the lower third the layout leaves
-empty, optically centered (measured off a canary screenshot; the comment records the measurement).
-
-### Live validation — nested canary, UEFI/OVMF, PVE 9.2-1
-
-Booted the built canary ISO under QEMU with OVMF and captured the framebuffer.
-
-| Check | Result |
-|---|---|
-| GRUB renders the Felhom card | **PASS** — background + grid visible at 1024×768 |
-| Exactly one entry, selected | **PASS** — „Felhom telepítés" only |
-| Hungarian accents under gfxterm | **PASS** — „telepítés", „Indítás … másodperc múlva" render correctly |
-| Countdown visible and counting | **PASS** — 5 → 0 |
-| Auto-fires at 0 | **PASS** — serial shows ``Booting `Felhom telepítés'`` |
-| Unattended install proceeds | **PASS** — „Fetching answers for automatic installation" → auto installer |
-| Same abort as v1.21.0 | **PASS** — `ERROR: Installation failed: filter did not match any device` → `Installation aborted`; no disk touched |
-
-A first build **correctly failed closed**: the banned-entry gate matched the template's own
-explanatory header. Fixed to strip comments before matching (a comment naming a removed entry is the
-point; a directive using one is the bug), which is a gate behaving as designed.
-
-### Artifacts (rebuilt on 180, `/mnt/5_hdd/felhom.eu/felhom-iso/out/`)
-
-| ISO | sha256 | bytes |
-|---|---|---|
-| `felhom-pve-9.2-1-v1.22.0-n100-generic-mkimage.iso` (safety) | `ff6f06ba1dbfe10f27d703afc29516001000349147426b43c9a424a0ea28bdbf` | 1 704 482 816 |
-| `felhom-pve-9.2-1-v1.22.0-n100-demo-generic-mkimage.iso` (real) | `494db0ddf859b6b152cad4d0e0d9e9cefd27255cde07e2b41aba3ac12a217888` | 1 704 482 816 |
-| `felhom-pve-9.2-1-v1.22.0-nested-canary-generic.iso` (validation) | `83c61c0413c84e27b26a37bb5dfaed2fcd44fd25e3e571c7310142bd305f2f9d` | 1 705 338 880 |
-
-Both shipping ISOs: `embedding 60 modules`, `El Torito boot images=2`, fs-uuid preserved, and the
-post-re-master verification confirming 1 entry + theme background inside the finished image.
-
-**Deliberately not done** (per the task): no squashfs/initrd rebranding — post-GRUB screens are still
-Proxmox-branded; no disk-setup or answer-generation change; the pairing banner is untouched.
-
----
-
-## Part 2 — website: the index grid background, restored
-
-**Archaeology.** Not a deliberate removal. The grid lived as a fixed `body::before` in
-`index.html`'s inline `<style>` block and was dropped in **`bed8675`** ("D3 Part 2: index + kapcsolat
-on design system v2"), the commit that migrated the page onto the shared `assets/site.css`.
-`dd54e4c`, which *created* `site.css`, has no `body::before` at all — it was a porting omission and
-nothing took its place. `ccbb13a` (the other five pages) never had it. No asset was lost: the
-mechanism was pure CSS (two stacked `linear-gradient`s), which is why nothing looked missing in the
-worktree.
-
-**Restoration, not redesign.** Same 50px cells, same 1px lines, same 3% opacity, same
-`position:fixed` / `z-index:-1`. One deliberate difference: the accent is the v2 `--blue` `#0083D8`
-instead of the retired legacy `#0088cc`, which `site_gates.py` bans. Scoped to `body.page-index`,
-because index is the only page that ever had it. `site.css` cache-bust bumped `?v=1` → `?v=2` across
-all seven pages (nginx caches 7 days); BOM preserved on every file.
-
-**Live verification** (felhom.eu, after git-sync deploy):
-
-- Desktop: grid renders behind the hero, at its original subtlety.
-- **376px viewport** (via a same-origin iframe — the browser window would not resize in this
-  environment, so the narrow case was exercised for real rather than asserted): grid renders, mobile
-  layout unchanged, `scrollWidth === clientWidth` so **no horizontal overflow**.
-- Computed style confirmed live: `linear-gradient(rgba(0,131,216,0.03) 1px, …)`, `50px 50px`,
-  `position: fixed`, `z-index: -1`, `pointer-events: none`.
-- `python scripts/site_gates.py` — **OK** (BOM, no legacy tokens, no `<style>` blocks, cache-busted).
-
----
-
-## Docs
-
-- `scripts/CHANGELOG.md` — v1.22.0 entry.
-- `website/CHANGELOG.md` — grid restoration entry.
-- `documentation/backlog/ROADMAP.md` — **R-38 flipped to SHIPPED**; **R-45** (unified async-job
-  feedback) and **R-46** (verification-copy browse + expiry) added; pre-invite checklist gained the
-  "golden ≥ 0.147.x carries all four infra images" line.
-- Capability map: **untouched** — no capability moved. These are UX and packaging.

REPORT-iso-release.md

@@ -1,307 +0,0 @@
-# REPORT — the universal ISO: **PUBLISHED** (2026-07-31)
-
-**Live:** `https://iso.felhom.eu/felhom-installer-1.26.1-pve9.2-1.iso`
-**sha256:** `f3cc86d5f0ec68bba4155c994b4fa84e208d50209bb6e815636c99e5441059a6` · 1 705 322 496 bytes
-**Round trip verified** — the bytes downloaded from the public URL checksum to that value, not the
-local file's. `.sha256` and manifest published beside it.
-
-> Written as `REPORT-iso-release.md`, not root `REPORT.md`, per the task and the shared-clone rule.
-
-## 0. Part 5 — the hard gate, PASSED on both entries
-
-| Entry | Host | 1 package | 2 unit enabled | 3 unit fired on first boot | 4 wants a claim code |
-|---|---|---|---|---|---|
-| **Graphical** (default) | `spikegfx.felhom.eu` | `ii felhom-bootstrap 1.26.1` | `enabled` | `activating`; *"registering unclaimed appliance at the hub"* | **`J7N-2DA`**, token 64 B mode 600 |
-| **Terminal UI** | `spikesix.felhom.eu` | `ii felhom-bootstrap 1.26.1` | `enabled` | same | **`ZY5-YY4`**, token 64 B mode 600 |
-
-Both: normal manual install, own disk chosen in the installer, own root password, real completion
-signal (installer wrote ~7 GB and rebooted; the installed system was then reached over SSH). Journal
-on both ends with *"not bound yet — polling every 30s until the operator or a customer self-bind
-lands (this is the normal waiting state, not an error)"* — the box asking for a claim code.
-
-Spike 4 reasoned the graphical path would follow from shared `Install.pm`. **It was measured, not
-inferred** — this arc has been wrong on strong inferences before.
-
-## 1. Venue and baselines
-
-| | |
-|---|---|
-| **Host** | `demo-hp` (t740), Tier 0 |
-| **VMs** | **500 `spike5-gfx`**, **501 `spike5-tui`** — both created with `qm` so the run is visible in the web console |
-| **Storage** | **`spike5`**, `dir` at **`/mnt/nvme-1tb` — the mount ROOT**, `content=images`. Root chosen deliberately: a storage at a *subdirectory* reads `disconnected` forever via the agent's `exactMount` check. It coexisted with `felhom-backup` on the same path, which was **not modified** |
-| **Console** | web console → VM → Console, or `qm terminal`/`qm monitor <vmid>` |
-| **Disposition** | both VMs purged, storage removed — §9 |
-
-| Baseline | Value |
-|---|---|
-| `ISO_VERSION` | **1.26.0** (was 1.25.0) — `scripts/iso/build-felhom-iso.sh:51` |
-| `SCRIPT_VERSION` | `1.22.0` — `scripts/felhom-host-install.sh:187` |
-| `felhom-bootstrap.sh` @ HEAD | `21bf6a6bde0cb13e3809e2f5c136a49929dcc82eb8d40bbdf6f290a886ee8ab7` |
-| PVE base | `proxmox-ve_9.2-1.iso`, `4e88fe416df9b527624a175f24c9aa07c714d3332afb1ee3dbf3879573ef2c6c` |
-| controller on `main` | `0.188.0` (`4115e88`) |
-| `felhom.eu` HEAD at build | clean, pushed, `== origin/main` |
-
-## 2. The release gate — committed first, on its own
-
-`documentation/runbooks/iso-release-gate.md`, commit **`e787391`**, written and pushed **before the
-first build** so it could not be rationalised afterwards. Twelve criteria, each checkable against the
-uploaded file rather than the build inputs, each carrying the spike measurement that justifies it.
-
-**One criterion was amended before the build, with its reasoning recorded in the runbook.** G6 was
-first written with the six-token ban `iso-repack.sh:160-164` enforces, on the rationale *"no live route
-to a manual disk-picker"*. That rationale is obsolete for a public image — the ruling makes the manual
-installer **the product**. `proxtui` (the Terminal-UI installer we deliberately ship) and `nomodeset`
-(its graphics fallback) are dropped **for release images only**; `proxdebug`, `Rescue Boot`, `memtest`
-and `fwsetup` stay banned in both modes, and the six-token list is **unchanged** for appliance images.
-
-## 3. The stub package
-
-`scripts/iso/pkg/` — source committed, built by `build-deb.sh`.
-
-**Contents: exactly two files, deliberately not three.**
-```
--rwxr-xr-x ./usr/local/sbin/felhom-bootstrap.sh
--rw-r--r-- ./lib/systemd/system/felhom-bootstrap.service
-```
-The old first-boot stub also wrote `/etc/felhom/bootstrap.env` (0600). This package does not:
-`felhom-bootstrap.sh:91` reads it only `if [[ -r ]]`, and its defaults at `:95-96`
-(`https://hub.felhom.eu`, `https://felhom.eu/scripts/felhom-host-install.sh`) are **exactly** what the
-generic pairing env set (`build-felhom-iso.sh:257-258`). Shipping it would add a 0600 file to a public
-package to express values the script already defaults to.
-
-**Dependencies: none, and that is a finding.** `dpkg-deb -I` shows no `Depends` line. The payload is a
-shell script and a unit file; the binaries the script calls (`curl`, `ip`, `dhclient`, `python3`,
-`systemctl`) run at **first boot**, not at postinst time. **Spike 4's open `dpkg --configure -a`
-ordering question therefore does not arise** — confirmed, not carried.
-
-**How the postinst is structurally incapable of failing** — no `set -e`, every statement individually
-guarded with `|| true` or an `if`, and an unconditional `exit 0`. `build-deb.sh` refuses to emit a
-package that violates any of it.
-
-**The guarantee was tested, not asserted.** Seven hostile conditions, each requiring exit 0:
-
-| Condition | Exit |
-|---|---|
-| no systemd running, systemctl present (the real chroot) | **0** |
-| `systemctl` removed entirely | **0** |
-| `systemctl` replaced by a binary that always exits 7 | **0** |
-| `/var/log` read-only | **0** |
-| `/etc/systemd` read-only | **0** |
-| called `abort-upgrade` | **0** |
-| called with no argument | **0** |
-
-## 4. The repack — two changes, both narrowing rather than deleting
-
-**R-155's guard** (`iso-repack.sh:100-106`) **protected the single-entry mode's promise**: that menu
-shows one item labelled "Felhom telepítés" which boots the *automated* installer, and without
-`auto-installer-mode.toml` the same label would drop the user into a manual disk-picker — a button
-promising an unattended install that silently does the opposite. That promise is real, so the guard is
-**kept unchanged for `FELHOM_MENU=single`** and simply does not apply to `release`, where the absence
-of that file is release-gate criterion G1 rather than a defect.
-
-**The menu collapse** happens at `iso-repack.sh:144-148` (the stock `grub.cfg` is replaced by a
-rendered template). A `release` template now renders **two interactive entries**; entry-count and
-banned-token gates are per-mode; the post-remaster verification reads the count back out of
-`final.iso`.
-
-**Ruling — default entry and timeout.** Default is **the graphical interactive entry**; timeout **15 s**.
-Reasoning: Spike 1 measured that no automated disk selection can be safe on unseen hardware (no
-property distinguishes an internal disk from a customer's backup drive; a two-disk match silently wipes
-one), so a public image whose default is unattended puts the unsafe path in front of anyone who boots
-and walks away. And Spike 2 lost a probe to a **1-second** menu — a person reading two options needs
-longer than a machine.
-
-**The automated entry is absent, not broken.** Skipping `prepare-iso` means no
-`auto-installer-mode.toml`, and the stock `grub.cfg` emits the Automated entry only inside
-`if [ -f auto-installer-mode.toml ]`. There is no entry that could fail in front of a customer.
-
-## 5. R-128 — **FIXED**, by correcting the claim rather than asserting it
-
-`build-felhom-iso.sh:44` claimed `ISO_VERSION` "aligns with felhom-host-install `SCRIPT_VERSION`".
-Nothing evaluated it and the two had drifted. **I did not turn it into a real assertion, because the
-coupling it claimed does not exist:** the ISO is a frozen artifact, while `felhom-host-install.sh` is
-fetched at run time from the website's git-sync of `main` (R-94/R-110), so whatever version an ISO
-carries, the script a box runs is always current. An assertion would invent a constraint. The comment
-now states the independence, and `ISO_VERSION` is `1.26.0`.
-
-## 6. Part 5 — the defect, the fix, and where it now stands
-
-### Round 1 (`1.26.0`) — the Terminal UI install FAILED on observable 4
-
-Three of four passed: the package installed, the unit was enabled from inside the installer chroot,
-and the unit **fired on first boot** and registered at the hub. The fourth failed:
-
-```
-felhom-bootstrap.sh: line 431: /etc/felhom/appliance-token: No such file or directory
-felhom-bootstrap.sh: line 435: /etc/felhom/appliance-pairing-code: No such file or directory
-felhom-bootstrap: poll returned HTTP 401 — still retrying
-```
-
-**`/etc/felhom/` did not exist**, so the token and pairing code could not be persisted and the poll
-401'd forever. No claim code would ever appear.
-
-**Root cause, mine.** `stub-first-boot.sh` opened with
-`install -d -m 0755 /etc/felhom /usr/local/sbin`. §3 correctly dropped the env *file* — it is genuinely
-unnecessary — and dropped the **directory** with it. `felhom-bootstrap.sh` uses `/etc/felhom/` for its
-runtime state.
-
-**Why the gate missed it.** G9 proves the packaged script is byte-identical to repo HEAD, and it was.
-**I verified the payload files and never the directory the payload writes into** — a check that proves
-the thing present and not the thing it depends on.
-
-### The fix, and its red-proof
-
-`build-deb.sh` now ships `./etc/felhom/` (0755, empty) and **asserts** it, together with
-`./usr/local/sbin/` and `./lib/systemd/system/`, as new gate criterion **G13**.
-
-**Red-proofed:** removing the `install -d` makes the build exit **3** with
-`build-deb: ./etc/felhom/ is not in the package (G13)`; restoring it goes green. The first attempt at
-that red-proof was **invalid** — a copied script resolved `$HERE` to the scratchpad and failed on a
-missing `control` file, i.e. non-zero for the wrong reason — and was redone in place.
-
-### Round 2 (`1.26.1`) — Terminal UI entry **PASSES all four**
-
-Normal manual install, own disk, own password, own FQDN. Host `spikesix.felhom.eu`.
-
-| # | Observable | Result |
-|---|---|---|
-| 1 | the `.deb` is installed | **PASS** — `ii felhom-bootstrap 1.26.1 all` |
-| 2 | the unit is enabled | **PASS** — `enabled` |
-| 3 | the unit fired on first boot | **PASS** — `activating`; journal shows *"PAIRING mode (generic ISO, no baked customer/passphrase)"* → *"registering unclaimed appliance at the hub"* → *"registered — appliance token stored (0600)"* |
-| 4 | **the box wants a claim code** | **PASS** — `/etc/felhom/appliance-pairing-code` = **`ZY5-YY4`**; `appliance-token` present, 64 B, mode `600`; journal: *"not bound yet — polling every 30s until the operator or a customer self-bind lands (this is the normal waiting state, not an error)"* |
-
-That is the product working end-to-end from a public image on a manual install: own disk, own
-password, nothing baked, and the box asking for a claim code.
-
-### The Graphical entry — **NOT COMPLETED**, and this is why nothing is published
-
-It reached the installer from the same image (KVM dialog, EULA, and the **Target Harddisk** screen
-showing `/dev/sda (20.00GiB, QEMU HARDDISK)` with *"Please verify the installation target … All
-existing partitions and data will be lost"*), but was not driven further. `Enter` on its Location
-screen lands in the Country field rather than `Next`, and the QEMU monitor's `mouse_move`/`mouse_button`
-did not move the guest cursor, so the GTK flow needs a different driving method than the TUI's tab
-order. **Part 5 requires both entries. It is not fully passed, so Part 7 did not run.**
-
-The `.deb` path lives in `Install.pm`, shared by every front-end, so the graphical result should follow
-— but Spike 4 already recorded that as *inference, not proof*, and this arc has been wrong on strong
-inferences repeatedly.
-
-### A fixture bug of mine, recorded twice because it cost two diagnoses
-
-`qm set <vmid> --scsi0 … --boot order="scsi0;ide2"` silently produced `boot: order=net0;ide2` — PVE
-processed `--boot` before `--scsi0` existed. Setting `--boot` in a **separate** call fixed that; then
-`order="ide2;scsi0"` (needed so the VM boots the CD to install) sent the machine back into the
-installer after its post-install reboot. **Detach the CD, or flip the order to `scsi0`, once the
-install completes.** Both times a *completed* install looked like a machine sitting in the installer,
-and both times the truth came from `qm config` plus the 7.0 GB disk rather than from the screen.
-
-## 7. Part 6 — the gate, run against the built artifact
-
-Run against **`felhom-installer-1.26.1-pve9.2-1.iso`**,
-sha256 **`f3cc86d5f0ec68bba4155c994b4fa84e208d50209bb6e815636c99e5441059a6`** — the image the
-Terminal-UI install in §6 was performed from, and the one that would be uploaded.
-
-| # | Criterion | Scanned for | Result |
-|---|---|---|---|
-| **G1** | no `answer.toml` / `auto-installer-mode.toml` | both names at ISO root | **PASS — 0** |
-| **G2** | no root password or hash | `.rootpw.txt` companion; the answer file that would carry a hash | **PASS** — no `.rootpw.txt` emitted; no answer file exists to hold one |
-| **G3** | no SSH key | `root-ssh-keys`, `ssh-rsa`, `ssh-ed25519` | **PASS** — no answer file; package carries only a script and a unit |
-| **G4** | no customer identity | `FELHOM_CUSTOMER_ID`/`RETRIEVAL_PASSPHRASE` with values, claim code, api key, Bearer | **PASS** — only the empty initialisers at `felhom-bootstrap.sh:89` |
-| **G5** | credential scan **by enumeration** vs the stock PVE ISO | full recursive file-list diff, both directions | **PASS** — exactly **four** added paths: the three `felhomtheme/` files and `/proxmox/packages/felhom-bootstrap_1.26.0_all.deb`; three removed (`pvetheme/`) |
-| **G6** | menu present, both paths, human timeout | entry count, `set default`/`timeout`/`timeout_style`, banned tokens | **PASS** — 2 entries, `default=0` (graphical), `timeout=15`, `timeout_style` underscore |
-| **G7** | one `felhom-*.deb`, version recorded | `/proxmox/packages/felhom-*` | **PASS** — exactly 1, `Package: felhom-bootstrap`, `Version: 1.26.0`, **no `Depends`** |
-| **G8** | postinst cannot fail | live (comment-stripped) `systemctl start\|daemon-reload\|restart`, network commands, `set -e`, last line | **PASS — 0, 0, 0**, ends `exit 0` |
-| **G9** | `felhom-bootstrap.sh` == repo HEAD | sha256 of the packaged file vs the repo file | **PASS** — both `21bf6a6bde0cb13e3809e2f5c136a49929dcc82eb8d40bbdf6f290a886ee8ab7` |
-| **G10** | build inputs committed | `git status --porcelain`, HEAD vs origin | **PASS** — clean and pushed at build time |
-| **G11** | published checksum + round trip | — | **NOT RUN** — nothing was published |
-| **G12** | bucket stays private | — | **NOT RUN** — the bucket was never touched |
-| **G13** | *(new, from Part 5's failure)* every directory the payload writes into is in the package | `./etc/felhom/`, `./usr/local/sbin/`, `./lib/systemd/system/` in `dpkg-deb -c` | **PASS** — all three present in `felhom-bootstrap_1.26.1_all.deb`; asserted by `build-deb.sh` and red-proofed |
-
-**A gate refinement found by running it.** G7 also asked that the ISO's copy of the `.deb` sha256-match
-the package built from source. It does not, and cannot: `dpkg-deb` embeds build timestamps, so two
-builds of identical source differ. **G9 — the payload's identity — is the meaningful check**, and it
-passes. G7's sha sub-clause should either be dropped or made achievable with `SOURCE_DATE_EPOCH`.
-
-## 8. Publication — done, and verified by round trip
-
-Uploaded with `rclone` **in a container, configured entirely by environment variables**, so no
-credential file was ever written to disk — the fence asks for config files to be kept out of repo
-paths and removed at teardown; none was created to remove. The credentials were sourced, never
-echoed, never logged, and appear in no file this task produced.
-
-| Check | Result |
-|---|---|
-| objects in the bucket | the ISO (1 705 322 496 B), `.sha256` (103 B), `.manifest.txt` (2 492 B) |
-| **round trip** | `curl https://iso.felhom.eu/felhom-installer-1.26.1-pve9.2-1.iso` → sha256 **`f3cc86d5…`**, byte count exact — **matches** |
-| G12 — bucket private | unauthenticated GET to the **S3 endpoint** → **400**; custom domain → 200; `GET /` on the custom domain → **404** (no index) |
-
-**The published manifest was corrected before upload.** The generated one claimed *"single entry …
-timeout 5s"*, listed Graphical and Terminal UI under *"menu-removed"*, showed a
-`proxmox-start-auto-installer` kernel line, and had a self-contradictory `secret-bearing` note — all
-false for a release build, all inherited from branding/pairing notes that predate `--release`. The
-generator is fixed and the sidecar regenerated. **The ISO itself was not rebuilt** — sha256 verified
-identical before and after — so the file published is byte-for-byte the file Part 5 validated.
-
-## 9. Teardown
-
-**demo-hp:** VMs 500/501 `qm destroy --purge`; **scratch storage `spike5` removed**
-(`storage.cfg` back to 4 entries, `grep -c spike5` = 0); `/mnt/nvme-1tb/images/` empty; usage
-**6.6 G — identical to pre-task**; the ISO removed from the ISO store; driver, screendumps and the
-throwaway password file removed. `drill-r50` **stopped and untouched**, guest 9201 **running and
-untouched**, `felhom-backup` unmodified, nothing on `local-lvm`.
-
-**demo-felhom:** not contacted.
-
-**DooPlex:** scratchpad 84 K; build logs and the package build tree removed. `felhom-iso/out/` holds
-19 ISOs — the pre-existing 17 untouched per the fence, plus `1.26.0` and `1.26.1`, both unpublished
-and **neither with a `.rootpw.txt`**, which is G2's own evidence. Repo tree clean and pushed.
-
-### Hub-side — **cleared**
-
-Observable 4 works *by* the box registering itself, so each proof install created an unclaimed
-appliance. All three were discarded: **16** and **17** (the 1.26.0 round), then **18** (the two
-1.26.1 proofs). `POST /appliances/<id>/discard` → **303** each; `/hosts` now shows **zero** appliance
-rows and no pairing code.
-
-The endpoint is `/discard`, **not** `/delete` — `hub/internal/web/server.go:345`, POST only. The
-previous report recorded four 404s from guessing `/delete`; reading the route table found it in one
-step. **R-131 gains no row.**
-
-## 10. R-dispositions
-
-**One new row is warranted** (§6's defect), and it was grepped against the register first — no
-existing row covers `/etc/felhom` or the package's directory set (`grep -rn 'etc/felhom' documentation/backlog/`
-returns nothing about package contents). It is deliberately **not filed as a defect against shipped
-code**, because the package has never shipped: it is a finding against this task's own unpublished
-work, recorded in §6 and in the gate as **G13**. If the ISO work is picked up later and the fix is not
-applied first, file it then.
-
-Otherwise, no new rows. Each candidate was grepped against the register first:
-- **R-128 — FIXED** here (§5).
-- **R-155 — RESOLVED** here (§4): the guard is narrowed, not deleted.
-- **R-154** (`[first-boot]` is automated-only and nothing in the tree says so) — **addressed in code
-  rather than by a row**: `pkg/build-deb.sh`'s header and `grub-release.cfg.tmpl` both state it with
-  the measurements. The register row can close when the docs land.
-- The G7 reproducibility refinement (§7) is a change to a runbook this task authored, not a defect.
-
-## 11. What did not happen, and what is still open
-
-- **Part 8 partially done.** The release-gate runbook (`e787391`), `day0-install.md` C.0 (ISO vs
-  manual, and when to use which) and `scripts/CHANGELOG.md` are written. **`OPEN-ITEMS.md` /
-  `ROADMAP.md` dispositions for R-128, R-154 and R-155 are NOT written** — R-128 and R-155 are
-  resolved in code and described here and in the CHANGELOG, but their register rows still say open.
-  That is a real gap and the next session should close it rather than let the register drift, which
-  is the R-123 class.
-- **The `.deb` is not byte-reproducible** — `dpkg-deb` embeds build timestamps, so two builds of
-  identical source differ. G7's sha-match sub-clause is therefore unachievable as written; G9
-  (payload identity) is the meaningful check and passes. Either drop the sub-clause or set
-  `SOURCE_DATE_EPOCH`.
-- **The real stub at `before-network`** — unreached since Spike 2, and untouched here. It is now
-  narrower than it was: on the `.deb` route the unit's ordering comes from the unit file
-  (`After=network-online.target …`), not from `[first-boot].ordering`, so it governs operator-built
-  appliance images only.
-- **Secure Boot** was not exercised. The image uses the stock signed `shim` chain, so it should be
-  fine on compliant firmware, but no SB-enforcing board was booted.
-- **Only virtual hardware** was tested. Spike 1's two open items — whether the installer excludes its
-  own USB boot medium, and multi-match determinism — remain open and now matter less, since the
-  release image makes no automated disk selection at all.

REPORT-pbs-prune-serverside-2026-07-27.md

@@ -1,71 +0,0 @@
-# REPORT — PBS prune moved server-side, write proof closed (2026-07-27)
-
-**Class:** supervised operational run. **No code, no version bump.** Topic-scoped per the
-parallel-session rule; shared `REPORT.md` untouched.
-
-**Full record:** `documentation/runbooks/RUNBOOK-pbs-prune-serverside-2026-07-27.md`
-
----
-
-## Outcome — all parts complete
-
-| Part | Result |
-|---|---|
-| 1 — prune gate | **Config-gated.** `keep_last: 0` on the PBS tier, both boxes → `prune_pbs_allowed=false`. No code, **no grant** |
-| 2 — prune jobs | 2 jobs, per live namespace, `keep-last 2`, daily **03:30 UTC / 05:30 CEST** |
-| 3 — dry run → real | Gate passed; both `TASK OK`; demo-hp 3→2, demo-felhom untouched |
-| 4 — write proof | **CLOSED — `TASK OK`, no job errors** |
-| 5 — GC | Scheduled `sun 04:30 UTC / 06:30 CEST`. **NOT run** |
-| 6 — `verify-new` | **Enabled** (operator ruling) |
-| — legacy ns | `demo-felhom-01` deleted with its ACLs + token (operator ruling, confirmed twice) |
-| 7 — roadmap | **R-89** + CONTEXT.md note |
-
-## The fix, in one line
-
-`allowPBSPrune := !t.Primary && t.KeepLast > 0` — so setting the PBS tier's `keep_last` to `0`
-disables both the `--prune-backups` value and the gate, in one config edit, **while the tier stays
-armed**. Verified: `backup tier armed target=felhom-pbs cadence=168h0m0s keep_last=0
-prune_pbs_allowed=false`, no `tier REJECTED` line.
-
-## The proof
-
-```
-07-27 08:25:47 UTC  vzdump (felhom-pbs) -> job errors     ← prune denied
-07-27 09:37:29 UTC  vzdump (felhom-pbs) -> OK             ← after the change
-```
-
-New snapshot `ns/demo-hp/ct/9201/2026-07-27T09:37:29Z`, chunks 9,787 → **9,813**, 97.0 % reused,
-45.80 s, **prune step absent entirely**. Driven via `POST /api/guest-backup/trigger` → `TriggerNow()`
-— the UI's „Mentés most" path, not `--selftest`, not raw `vzdump`.
-
-**Hub gauge evidence NOT satisfied** — a +32.8 MB delta is below its 0.1 GB display granularity, so it
-still reads 12.6 GB / 13 %. Stated plainly rather than dressed up.
-
-## The demo-felhom prediction — CLOSED
-
-The claim was that demo-felhom's next weekly backup would make 3 snapshots and reproduce the prune
-failure. Neutralised on both halves: the box no longer attempts prune, and `prune-demo-felhom` covers
-the namespace server-side (verified live, `TASK OK`). **It will not reproduce.**
-
-## Why it mattered more than the unpruned snapshots
-
-demo-hp's PBS tier had reported failure on **every** backup since the tier was created on 07-26, while
-the data landed correctly every time. A tier that cries wolf on every success makes a genuine failure
-invisible — which is precisely what happened at 07:13 UTC, when a real outage produced an
-indistinguishable result.
-
-## Security property preserved
-
-**No prune right was granted to any box.** Final ACLs are four entries, write-only
-(`DatastoreBackup`), live namespaces only. A compromised box still cannot delete its own offsite
-backups. `felhom-tenantsync.sh` was **not** edited — the ruling makes its current grant correct.
-
-## Open
-
-1. **R-89** — hub-owned retention policy (today's jobs are increment 1, not a stopgap).
-2. **Does the restic key on `storage-box-pool-1` have DELETE rights?** Unanswered, carried in R-89,
-   and the more urgent half — if so, the daily app-data tier has the identical exposure and
-   append-only mode is the equivalent answer. Rule once for both tiers.
-3. **GC has still never run.** First execution Sunday 04:30 UTC; worth watching, as nothing has ever
-   exercised it here.
-4. Old 13 GB datastore copy still at `/srv/pbs-felhom` — rollback intact.

REPORT-r100.md

@@ -1,194 +0,0 @@
-# REPORT — R-100: a failing offsite tier must go stale (2026-07-28)
-
-Hub **v0.79.0 → v0.80.0**; companion `felhom-controller` **v0.180.0 → v0.181.0** (the producer, shipped
-first). Written as `REPORT-r100.md` so the shared `REPORT.md` is not clobbered.
-
-## Baselines (reconfirmed, not copied)
-`felhom.eu 6369570`, `felhom-controller 4056fec`, `felhom-agent d5c7691` — all = origin/main. The only
-dirt in `felhom.eu` was a **foreign** `documentation/PROMPT-TEMPLATE.md` (shared worktree, untouched).
-Hub manifest **and** running pod both `0.79.0`; `staleAfter` = 48h; controller 0.180.0 and agent 0.110.0
-live on both boxes.
-
----
-
-## The premise was wrong, and it was mine
-
-R-100 was filed yesterday claiming *"the operator's fleet-wide alarm plane is silent"*. Phase 0 refuted
-that, twice:
-
-1. **A failing offsite run does alarm.** `main.go:655` wires `SetOffboxNotify` → `NotifyBackupFailed`;
-   the notify cooldown is 6h against a 24h cadence, so a nightly failure alarms nightly. Live hub DB:
-   `backup_failed | operator | sent | 5`, latest 2026-07-27 17:42. The `isStale` doc comment —
-   *"a recent-but-failing run is NOT stale (backup_failed owns that signal)"* — was **accurate**.
-2. **The orphaned-repo path I expected to be an indefinite hole is already covered.** The scheduled run
-   returns early at `offbox.go:606`, *before* the `LastRun` write at `:716`, so `LastRun` freezes and
-   `offsite_stale` fires normally.
-
-I could find no failure mode that both advances `LastRun` and produces no operator signal.
-
-**The real defect — defeated defence in depth.** `offsite_stale` is the hub-side, *pull-based* net that
-exists to be independent of controller-*pushed* events. Anchoring it on `LastRun` made it depend on the
-very thing it backs up: when the push is lost, the net cannot compensate, because the failing controller
-keeps refreshing the field the net reads. **F-HUB — this campaign's own finding, the hub dropping an
-event under `SQLITE_BUSY` with no retry** — is exactly that loss.
-
-**Honest severity: MEDIUM**, not the top-ranked item. The fix is unchanged; the justification is not.
-
----
-
-## Phase 0 answers
-
-**P0.1 — a last-success timestamp did not exist.** `OffboxTarget` carried `LastRun`/`LastStatus`/
-`LastError`/`LastDuration` only. Recording one is a new field, not a transmission of something known.
-
-**P0.2 — `LastStatus` on the wire**, from 4000 live reports (not from source alone):
-
-| value | count | paired with |
-|---|---|---|
-| `ok` | 2269 | `last_run` set |
-| absent/null | 541 | `last_run` **empty** — never-ran |
-| `error` | 27 | `last_run` set |
-| **`running`** | 7 | a report captured **mid-run** |
-
-Plus 1156 reports with no `offsite` object at all. **The legacy trap — status absent *with* a real
-`last_run` — occurs 0 times**, because `LastStatus="running"` is written the moment a run starts. It is
-still handled explicitly, but it is not a live shape. `running` being real is why the verdict ignores
-status entirely.
-
-**P0.3 — sweep**
-
-| tier | `LastRun` written on failure? | read as success by a verdict? |
-|---|---|---|
-| **Offsite restic** | YES (`offbox.go:716`) | **YES — hub `isStale`.** The defect |
-| **Tier 2 cross-drive** | YES (`recordTier2Failure`) | No hub verdict; UI only → **R-101, filed** |
-| Tier 1 recovery units | **NO** — derived from an actual artifact | structurally immune |
-| Shares offsite leg | YES | `sharing.html:180` shows the time only when status=="ok" — honest |
-| DB dump | n/a — **event-based** (`db_dump_completed`/`db_dump_failed`) | immune by design |
-
-`offsite.go` is the **only** hub verdict anchored on a `LastRun`-shaped field. The deadline checker
-already uses distinct success/failure *events* — the pattern this converges on.
-
-**P0.4 — the customer is NOT shown a failed offsite run as successful.** `backups_remote.html:34-36`
-leads with the status (`✓ Rendben` / `✗ Hiba` / `Fut…`). Two narrower Tier-2 instances → **R-101**.
-
----
-
-## The fix
-
-**Controller v0.181.0 (producer, shipped first).** `OffboxTarget.LastSuccess`, carried on the report as
-`last_success`. The rule is a pure function called unconditionally beside the `LastRun` write:
-
-```go
-func offboxAnchorAfterRun(prev, at string, runErr error) string {
-	if runErr != nil { return prev } // failures neither advance nor clear
-	return at
-}
-```
-
-Both directions are separate bugs: a failure must not **advance** it (the original defect) and must not
-**clear** it (one bad night making an established tier read as never-succeeded).
-
-**Two silent-wipe sites found and closed** — the "seam built but never wired" shape, where the field
-exists, the writer sets it, and an unrelated routine path zeroes it:
-- `offboxConfigHandler` rebuilds the target from the form and copies runtime status field by field, so
-  an ordinary settings save would have erased the anchor;
-- `ApplyOffsiteTarget` does the same on a hub re-apply.
-
-Neither would have surfaced until the verdict changed, days later. **The first was proven live** — see
-below.
-
-**Hub v0.80.0.** Three deliberate branches:
-- **never ran** — unchanged v0.73.0 anchored behaviour, still keyed on `last_run` on purpose: that field
-  answers "has anything ever happened here", and a box whose *first* run failed is a run, not a newborn.
-- **legacy** (`last_run` set, no `last_success`) — degrades **explicitly** to the old behaviour, logged
-  **once** per customer. Absence-as-failure would alarm the whole un-upgraded fleet; absence-as-success
-  keeps the bug. Same degrade direction as R-88 Part 2's `age_state`.
-- **anchored** — counts from `last_success`; `last_status` is deliberately not consulted, because
-  "error ⇒ stale" pages on every blip (the F-A1 noise path).
-
-**The alarm text had to move with the verdict.** `emitStale` still said `last run 8h ago` while firing on
-a six-day-old success — a true alarm that reads as false. `staleAge` now separates *"runs are happening
-and failing — check the error, not the schedule"* from *"the offsite leg is silently not running"*.
-
----
-
-## Red-proofs — all observed failing
-
-| # | red-proof | observed failure |
-|---|---|---|
-| A | restore the `LastRun` anchor | `a tier that has not succeeded in 6 days reads as FRESH — that is R-100` |
-| B | delete the never-ran branch | `a newborn box alarmed — this is the 2026-07-23 cry-wolf that v0.73.0 fixed` |
-| C | collapse to `LastStatus == "error"` | `a single transient failure alarmed — 20h ... well inside the 48h threshold` |
-| D | delete the legacy degrade | `a legacy controller alarmed — that is a fleet-wide alarm storm on an un-upgraded fleet` |
-| + | drop the `runErr` guard (controller) | `a FAILED run advanced LastSuccess ... that is the R-100 defect in mirror image` |
-| + | always return `prev` | `a successful run did not advance the anchor` |
-| + | drop the wire field | `OffboxReportStatus dropped LastSuccess — the hub would degrade forever` |
-| + | drop the handler preservation | `a settings save erased LastSuccess` |
-
-**A hollow test of my own, caught by red-proofing it.** The first version of the controller test
-re-implemented the rule in a local closure — mutating production code left it **green**. That is why
-`offboxAnchorAfterRun` was extracted: the test now calls the real rule.
-
-Fixtures are the **real** wire shapes from P0.2, not invented JSON.
-`go build`/`go vet`/`go test` green in both repos (hub 17 pkgs, controller 27 pkgs), run separately
-from every commit.
-
----
-
-## §6 — LIVE, on demo-hp (disposable; `peti-felhom` never touched)
-
-A genuine restic failure was induced by pointing the target at a **closed port** (23 → 2) — it creates
-nothing, touches no data, and is exactly reversible.
-
-```
-success run  →  last_status=ok     last_run=11:24:20Z   last_success=11:24:20Z
-INJECT port 23 → 2 ... and the settings save PRESERVED last_success = 11:24:20Z   ← the wipe-site fix, live
-failing run  →  last_status=error  last_run=11:25:48Z   last_success=11:24:20Z    ← ANCHOR HELD
-```
-
-**As the hub received it:**
-
-| box | status | `last_run` | `last_success` | anchor |
-|---|---|---|---|---|
-| **demo-hp** (induced failure) | `error` | 11:25:48Z | **11:24:20Z** | **HELD** |
-| **demo-felhom** (healthy) | `ok` | 11:29:22Z | **11:29:22Z** | **advanced** |
-
-Also observed live, unplanned: **Scenario E**. Both boxes were still on the old controller at hub
-startup, and the degrade logged **exactly once per customer** —
-`[WARN] [offsite] demo-hp: controller sends no last_success — staleness degraded to the last-ATTEMPT
-anchor`. Two lines, two customers, same second.
-
-**No spurious alarms:** 0 `offsite_stale` events since deploy (correct — both tiers succeeded minutes
-ago). `backup_failed` fired for demo-hp at 11:25:48 from the induced failure, confirming the
-pre-existing channel is intact and re-confirming the Phase 0 correction.
-
-**Config restored** and verified field by field: `host=u629488-sub3.your-storagebox.de port=23
-user=u629488-sub3 repo=/home/felhom-repo enabled=True escrow=escrowed`.
-
-### Proven live vs. proven by injected clock — stated plainly
-- **Live:** the anchor does not advance on failure; it does on success; it survives a settings save;
-  `last_success` reaches the hub; the legacy degrade fires once per customer; no spurious alarms.
-- **Unit, injected clock only:** the 48h **threshold** behaviour itself — Scenarios A/B/C/D turning on
-  elapsed time. A live threshold test would take days. **The threshold was NOT proven live.**
-
----
-
-## Part 2 — the rule
-**"Presence is not success"** added to `CLAUDE.md` and its versioned copy, with both instances
-(F-CRIT-2's phantom ctime, R-100's `LastRun`) and the corollary R-100's fix produced: when a verdict
-changes which field it counts from, the **alarm text must change with it**. `// R-100` notes sit at
-`isStale` and at the controller write site, each naming the test that pins it.
-
-## Filed, not fixed
-- **R-101** — Tier-2 `LastRun` is also written on failure, and three customer surfaces render it without
-  a status (two degraded branches plus the restore-confirm dialog). No hub verdict reads it.
-
-## NOT yet live-validated (carried forward)
-- **The 48h staleness threshold itself** (see above) — and with it Scenario A end-to-end: no
-  `offsite_stale` event has yet been *observed firing* from a genuinely stale success anchor, because
-  that needs 48h of failure.
-- **Fault 4** — restic transport interruption; four injection approaches defeated by guest-bridged
-  networking. (This task's closed-port injection sidesteps it rather than solving it.)
-- **R-99** — prune never removes phantom snapshots.
-- **R-101** — filed today, unvalidated.
-- `contentionAlarmAfter` (3h) — injected clock only.

REPORT-r101.md

@@ -1,181 +0,0 @@
-# REPORT — R-101 + F-DIAG + F-OPS (2026-07-28)
-
-Controller **v0.181.0 → v0.182.0**; `felhom.eu` gains the manual-restore runbook (F-OPS) and the
-OPEN-ITEMS rows. Written as `REPORT-r101.md` so the shared `REPORT.md` is not clobbered.
-
-## Baselines (reconfirmed, not copied)
-`felhom-controller 3db8bfb`, `felhom.eu 6b7d516`, `felhom-agent d5c7691` — all = origin/main; the only
-dirt in `felhom.eu` was a **foreign** `PROMPT-TEMPLATE.md`. Controller **0.181.0** live on both boxes,
-hub `felhom-hub:0.80.0` ready 1/1.
-
----
-
-## Phase 0
-
-**The render sites — three dishonest, two already honest.** The spec listed `backups_apps.html:216`
-as a defect site; it is in fact the one branch that *already* pairs its timestamp with a status badge.
-The real third site is the `Tier2DestInactive` branch.
-
-| site | rendered | honest? |
-|---|---|---|
-| `:231` **restore confirm dialog** | `Legutóbbi másolat: {{.Tier2LastRun}}` — raw RFC3339, no status | **NO** — the one that matters |
-| `:195` `Tier2DestDisconnected` | `Utolsó: …`, no status | **NO** |
-| `:206` `Tier2DestInactive` | `Utolsó: …`, no status | **NO** |
-| `:217` main configured branch | `Utolsó: …` **+ status badge** | already honest |
-| `sharing.html:181` | rendered **only** when status=="ok" | already honest |
-
-`Tier2LastStatus` was already set unconditionally at `handlers.go:1182`, so this was a wording/anchor
-problem, not a plumbing one. The restore button was gated on `{{if .Tier2LastRun}}`, so **Scenario C
-was live-reachable**: a tier that had attempted and never succeeded offered a restore and a timestamp.
-
-**`cd.LastRun` is written on failure** — `recordTier2Failure` (`tier2.go:573-574`) writes it alongside
-`LastStatus:"error"`. Identical shape to R-100.
-
-**Legacy state is universal, not an edge case.** All 7 Tier-2 rows across both boxes had `last_run` and
-no anchor. Scenario E was therefore the *initial state of every customer*, which is what made the
-legacy marker non-optional.
-
----
-
-## Part 1 — the strings shipped
-
-| case | string |
-|---|---|
-| dialog, normal | `… Legutóbbi sikeres másolat: 2026-07-28 16:43.` |
-| dialog, newest attempt failed | `… Legutóbbi sikeres másolat: 2026-07-28 16:40. Figyelem: a legutóbbi mentési kísérlet nem sikerült, ezért a visszaállított fájlok ennél régebbiek lehetnek.` |
-| card | `Utolsó sikeres: 2 perce` |
-| never succeeded | `Még nincs sikeres másolat` + `Még nincs sikeres másolat, amiből vissza lehetne állítani.` (restore removed) |
-| **legacy row** | `Utolsó: …` / `Legutóbbi másolat: …` — **today's wording, unchanged**, logged once per stack |
-
-**Timestamp made human-readable** (agreed): new `fmtTimeStr` renders Budapest-local `2026-07-28 16:40`
-instead of the raw UTC `2026-07-28T14:40:55Z` a customer was previously asked to reason about.
-
-**`SuccessTracked` is what makes the legacy case possible at all.** Without it, "row predates the
-anchor" and "row has an anchor and it is empty" are indistinguishable — both are `LastSuccess==""` —
-and every existing row would have rendered as never-succeeded on deploy. Legacy rows migrate on first
-touch: a row whose last known state was `ok` adopts that time (truthful — under the old code that run
-did succeed); a row whose last state was `error` seeds **nothing**, because the old data evidences no
-success.
-
-## Part 2 — the copy-site hazard, and it was in the path
-
-The three `record*` helpers each built a **whole `CrossDriveBackup` literal**, with a helper re-applying
-exactly two fields; everything else was zeroed on every status write. Adding `LastSuccess` to that shape
-would have had `recordTier2Failure` **clear** it — the mirror image of the defect, firing on the *first*
-failure rather than lying dormant.
-
-Replaced with **`tier2Update`**, which copies the existing row and overlays the outcome: **compile-safe
-by construction** — a new field carries over unless deliberately overwritten, so nothing is preserved by
-a list that can fall out of date. Callers now clear explicitly what a run invalidates, reproducing the
-old behaviour exactly.
-
-**Sweep of other rebuild sites:** `SetTier2Preference` mutates in place (safe); `SetCrossDriveConfig(name, nil)`
-in `api/router.go:774` is a deliberate delete. No others.
-
-## Part 3 — F-DIAG
-
-| class | signal it maps to | message head |
-|---|---|---|
-| `quota` | the pre-run soft-quota gate | `A távoli mentés nem fért el a tárhelykereten belül` |
-| `orphaned` | `ErrOffboxOrphaned` sentinel | `A távoli tárhely egy korábbi, már nem elérhető kulccsal készült` |
-| `no_repo` | restic "unable to open config file" | `A távoli tárhelyen nincs mentési adattár` |
-| `no_units` | "produced no snapshots" | `Nem volt mit menteni: egyetlen kijelölt alkalmazásnak sem található mentése` |
-| `transport` | refused/reset/timeout/authn/host-key | `A távoli tárhely nem érhető el (hálózat vagy bejelentkezés)` |
-| **`unknown`** | anything else | `A távoli mentés ismeretlen okból nem sikerült` |
-
-The `unknown` class is deliberate: a cause that cannot be told apart where the error is produced is
-reported as unknown rather than folded into a neighbour.
-
-**Secrets — and this caught a bug in my own first attempt.** The old message was
-`"…: " + err.Error()`, carrying the repo reference `sftp:<user>@<host>:<path>` off the box. My first
-sanitiser regex-matched `sftp:…` and `user@host` and *looked* complete; its own test caught it leaking
-on `ssh: connect to host <host> port 23: Connection refused` — a bare hostname in neither shape. It now
-redacts the target's **actual** host/user/repo-path literally, with the regex kept only as a backstop.
-Guessing at what a secret looks like fails exactly where it matters.
-
-## Part 4 — F-OPS
-
-`documentation/runbooks/RUNBOOK-manual-guest-restore.md`. Grounded in the real bind shape read off live
-guest 9201, not written from memory. Covers: which `mpN` are storage volumes (restored) versus **host
-binds** (taken as-is on the target); the `mp9` trap — it embeds the **source** VMID, so restoring to a
-different VMID can bind **another guest's bootstrap credentials**; strip-and-re-add before first boot;
-the hookscript check; and a positive pre-start verification that asserts every bind path exists rather
-than accepting "no error". Docs only, by design.
-
----
-
-## Red-proofs — all observed failing
-
-| # | red-proof | observed failure |
-|---|---|---|
-| A | dialog back on the attempt clock | `the dialog does not name the last SUCCESSFUL copy` |
-| C | gate the restore on `LastRun` again | `a tier that has NEVER succeeded still offers a restore — the dialog would promise a copy that does not exist` |
-| D | make the caution unconditional | `a HEALTHY tier shows the failed-attempt caution ("nem sikerült")` |
-| F | clear the anchor on failure | `a FAILED run wiped the success anchor (round 1) — one bad night would read as 'no copy has ever succeeded'` |
-| + | raw sanitiser | `the repo reference reached the message ("sftp:" leaked)` |
-
-**F exercises the real `recordTier2Success` → `recordTier2Failure` sequence**, not a modelled copy — the
-R-100 lesson. The Scenario A/C/D tests **render the production template tree** and assert on the string
-the customer reads; a test asserting a template variable would prove nothing about wording, which is
-the defect.
-
-`go build`, `go vet ./...`, `go test ./...` — 27 packages, `rc=0`; `template_id_gate.py` and
-`emoji_gate.py` both OK. Run separately from every commit.
-
----
-
-## LIVE on demo-hp — the rendered dialog, which is the deliverable
-
-**Legacy state** (before any run under v0.182.0) — today's wording, no fright:
-```
-Legutóbbi másolat: 2026-07-28 03:30
-```
-
-Failure induced genuinely: the Tier-2 destination directory was **moved aside** and replaced by a file,
-so `mkdir …/recovery-unit` fails. (`chmod` does not work — the controller runs as root, which bypasses
-permission bits; `chattr +i` is refused, the unprivileged container lacks `CAP_LINUX_IMMUTABLE`. Both
-were tried and reported rather than glossed.) The real data was only ever moved, never deleted.
-
-```
-status       = error
-last_run     = 2026-07-28T14:42:18Z     ← ADVANCED
-last_success = 2026-07-28T14:40:55Z     ← HELD
-last_error   = mkdir …/paperless-ngx/recovery-unit: …
-```
-
-**The rendered dialog, failed state:**
-```
-Visszaállítja a hiányzó fájlokat a másodlagos másolatból? A meglévő fájlok NEM módosulnak és NEM
-törlődnek. Az alkalmazás a művelet idejére leáll. Legutóbbi sikeres másolat: 2026-07-28 16:40.
-Figyelem: a legutóbbi mentési kísérlet nem sikerült, ezért a visszaállított fájlok ennél régebbiek
-lehetnek.
-```
-
-**The rendered dialog, healthy state** (after restoring the destination and a successful run) — no
-caution, no tonal change:
-```
-Visszaállítja a hiányzó fájlokat a másodlagos másolatból? A meglévő fájlok NEM módosulnak és NEM
-törlődnek. Az alkalmazás a művelet idejére leáll. Legutóbbi sikeres másolat: 2026-07-28 16:43.
-```
-
-Card lines: `Utolsó sikeres: 2 perce` → `Utolsó sikeres: most`.
-
-**Everything restored:** destination is a directory again, 86 MB intact, mode 755, `.r101-aside` gone,
-`status=ok`, `last_success=2026-07-28T14:43:23Z`.
-
-**demo-felhom is the untouched control:** all 5 rows still `tracked=None` after the deploy, rendering
-today's way, 15/15 containers up. Scenario E holding across a whole box nobody ran.
-
----
-
-## NOT yet live-validated (carried forward)
-- **F-DIAG's classes** — unit-proven only. No live offsite failure of each class was induced; the
-  `transport` class is the only one this arc exercised indirectly.
-- **Scenario C live** — the never-succeeded rendering is unit-proven; no fleet row is in that state
-  (every row either migrated or has a real success), and manufacturing one would mean breaking a
-  customer app's only Tier-2 history.
-- **The Tier-2 restore itself** was not executed — this arc changed what the dialog *says*, not what the
-  restore does.
-- **R-100's 48h staleness threshold** — injected clock only.
-- **Fault 4** (restic transport interruption), **R-99**, **F-HUB**, fault 12, the three-way concurrency
-  overlap — next campaign's material, untouched here.

REPORT-r106-r109-recipe-completeness.md

@@ -1,136 +0,0 @@
-# REPORT — R-106 + R-109 (+ R-122): closing the recipe-completeness set (2026-07-30)
-
-Non-overwritten sibling per `CLAUDE.md:82-87` — the shared `REPORT.md` holds R-117 and is not touched.
-
-Shipped: **agent v0.118.0 → v0.118.1** (`felhom-agent` `1c8a67e`, `6b5dade`) + **hub v0.83.0**
-(`felhom.eu` `acfc2b7`). Neither half is useful alone.
-
-**Read §3 first if you read nothing else:** v0.118.0's R-106 half shipped INERT and live validation is
-what caught it — the recipe still said `"root"`, now with `namespace_state: resolved` beside it. Full
-account in the audit §6, filed as **R-125**.
-
-## Part 0 — the answers, before the fix
-
-### 0.1 Which items are actually open, and R-105/R-106's registration
-
-`OPEN-ITEMS.md` calls itself "the single source of truth for open work" (`:1`), with `ROADMAP.md` keeping
-"the full history and reasoning" (`:3-4`).
-
-| item | `ROADMAP.md` | `OPEN-ITEMS.md` | verdict |
-|---|---|---|---|
-| R-105 | row, `READY — 2026-07-28` (`:108`) | **absent** | **open but UNREGISTERED** |
-| R-106 | row, `READY — 2026-07-28` (`:109`) | **absent** | **open but UNREGISTERED** |
-| R-108 | row (`:111`) | row (`:50`) | registered |
-| R-109 | row (`:112`) | row (`:61`) | registered |
-
-So R-109's own cell — "third recipe-completeness defect beside R-105/R-106" — was the **only** place in the
-register naming two open items. That is exactly the thread-loss the register exists to prevent, and it is
-itself a finding (filed **R-123**). Both now have rows.
-
-**The set this task closes is R-106 + R-109**, matching the arc's stated definition of done (`OPEN-ITEMS.md:14`).
-**R-105 is NOT in it** and was not worked: it is M-sized and is about three *hub-held DR records* being `{}`
-(`hosts.dr_record_json`, `host_escrow.directive_json`, and the `drives` third — already traced and populated
-by the 2026-07-28 target move). Different fields, different owner, different size.
-
-### 0.2 Where the recipe is generated — three producers, not two
-
-| half | repo | function |
-|---|---|---|
-| host (guests/pbs/drives/pve_storage) | `felhom-agent` | `BuildDRRecipeHostHalf`, `internal/hub/dr_recipe.go:86` |
-| app (customer/apps/offsite_restic) | `felhom-controller` | `controller/internal/report/dr_recipe.go` |
-| **assembly + delivery** | `felhom.eu/hub` | `AssembleDRRecipe`, `internal/store/dr_recipe.go:104`; served by `handleDRRecipeDownload`, `internal/web/dr_recipe.go:14`, route `internal/web/server.go:439` |
-
-R-109's "host-half" is therefore the **agent**, and the field must also pass the **hub's** allow-list — see §2.
-
-### 0.3 What the namespace field actually contained — verified, and the brief was RIGHT
-
-The eleven-session-old brief held up. Live, pre-fix, from the hub for **both** boxes:
-
-```json
-"pbs": { "repo_id": "felhom-pbs", "namespace": "root", "latest_snapshot_id": "9201" }
-```
-
-against `/etc/pve/storage.cfg` on the same boxes:
-
-```
-pbs: felhom-pbs
-        datastore felhom-offsite
-        namespace demo-felhom          # demo-hp reads: namespace demo-hp
-```
-
-Traced to source: `Snapshot.Namespace` decodes `ns` (`internal/pbs/client.go:97`), which PBS does not echo
-per item once the list is namespace-scoped via `?ns=` (`:118-120`) → always empty → `ToHub` normalises empty
-to `"root"` (`internal/pbs/report.go:22-25`) → `latestPBSCoord` writes it in.
-
-**The authority taken, and why:** storage.cfg's `namespace` on the pbs storage. It is the same field
-`vzdump --storage <pbs>` makes PVE read, and the agent's own verify client is built from it
-(`cmd/felhom-agent/main.go:1164`). Deriving the recipe from anything else is how it drifts again.
-
-## 1. R-109's ambiguity is real, in the boxes' own pre-fix recipe
-
-```json
-"pve_storage": [
-  { "name": "local-lvm",     "type": "lvmthin",   "content": "images,rootdir" },
-  { "name": "felhom-backup", "type": "local-dir", "content": "backup" },
-  { "name": "felhom-pbs",    "type": "pbs",       "content": "backup" },
-  { "name": "local",         "type": "local",     "content": "backup,import,vztmpl,iso" }
-]
-```
-
-No `backup_target` key anywhere. `felhom-backup` (live, `/mnt/hdd_1`) and `local` (`/var/lib/vz`, archives
-frozen 2026-07-28) are both `content=backup` dir storages; `local` is also the *historically* correct answer,
-which is what makes guessing it so easy.
-
-## 2. R-122 — a fourth defect, found here, and it had already shipped
-
-`AssembleDRRecipe`'s `hostHalfShape`/`appHalfShape` are **allow-lists** dressed as forward-compat. The
-controller has emitted `offsite_restic` since fork-4 (`controller/internal/report/dr_recipe.go:39-41`, "so DR
-knows WHERE to recover from"); `appHalfShape` never listed the key. Verified both ways:
-
-- **stored**: `dr_recipe.app_half_json` carries it for all three real customers —
-  `peti-felhom`, `demo-felhom` (`u629488-sub1.your-storagebox.de:23/home/felhom-repo`), `demo-hp`.
-- **delivered**: the downloaded recipe's top-level keys were
-  `recipe_version, customer, guests, pbs, drives, pve_storage, apps` — **no `offsite_restic`**.
-
-So a restorer reading the recipe had **no offsite location at all**, for the whole life of the feature, with
-a green suite throughout — because the test fixture `drAppHalf` is hand-written and omits the field.
-
-**Deviation from the task's §7.10 ("Findings — filed as R-n, none fixed"), stated rather than absorbed:**
-I fixed it. Reasons — (a) Part 0 authorises working the real set if it differs; (b) it is the same
-symptom the task is named for (the recipe is incomplete), and the worst instance, a whole section missing;
-(c) it is in the *same two structs* R-109 forced me to edit, and leaving one of three known keys off a
-drop-list I was already correcting would be indefensible. It is filed as R-122 with a SHIPPED disposition.
-
-## 3. The before/after recipe — both boxes, quoted
-
-```
-demo-felhom  BEFORE  "namespace":"root"           backup_target absent   offsite_restic absent
-             AFTER   "namespace":"demo-felhom"    backup_target {resolved, felhom-backup, /mnt/hdd_1}
-                                                  offsite_restic {u629488-sub1…}
-demo-hp      BEFORE  "namespace":"root"           backup_target absent   offsite_restic absent
-             AFTER   "namespace":"demo-hp"        backup_target {resolved, felhom-backup, /mnt/nvme-1tb}
-                                                  offsite_restic {u629488-sub3…}
-```
-
-The two boxes DISAGREEING is the point — nothing is hardcoded. And the ambiguity was not theoretical:
-on both boxes `felhom-backup` holds an archive from **07-30 04:36** while `local` stops at
-**07-28 17:5x**, frozen at the target-move date. The recipe now names the live one.
-
-Full evidence, all seven red-proofs and the publish observables:
-`documentation/audits/R106-R109-recipe-completeness-2026-07-30.md`.
-
-## 4. Findings filed (none of them fixed except R-122, see §2)
-
-| id | finding |
-|---|---|
-| **R-122** | `AssembleDRRecipe` allow-list dropped `offsite_restic` for the feature's whole life — **FIXED here**, hub v0.83.0 |
-| **R-123** | R-105 and R-106 were `READY` in `ROADMAP.md` with no `OPEN-ITEMS.md` row — referenced only inside R-109's prose. Registered here |
-| **R-125** | v0.118.0 shipped an inert R-106 because the "production path" test injected `fakeObserver` one layer below the break — **FIXED** in v0.118.1; filed for the doctrine point (name the seam you inject at) |
-| **R-124** | The recipe spells PBS's root namespace `"root"`, but the PBS API spells it `""` and there is no namespace literally named `root` — a restorer pasting it into `pct restore --ns root` would fail. Pre-existing wire convention, deliberately unchanged; documented at `PBSRootNamespace` |
-
-## 5. Not done, and why
-
-- **R-105, R-108, D5** — out of scope by the task's §6. R-108 blocks D5; starting either would leave both half-done.
-- **The backup machinery** — untouched. This corrects the record, not the doing.
-- **`sess-f` (0.116.0) and `drill-r50` (0.113.0) were not upgraded** — neither was named as a venue, and `drill-r50` is fenced by the task's §6.
-- **R-124 not fixed** — changing the wire's spelling of the root namespace mid-R-106 would shift the field's meaning during the fix meant to make it trustworthy.

REPORT-r116-diag.md

@@ -1,117 +0,0 @@
-# REPORT-r116-diag — the `/disks` payload captured, R-116's mechanism isolated (2026-07-30)
-
-Read-only diagnosis run by CC on DooPlex. **No code written, nothing built, nothing published.**
-Full evidence: `documentation/audits/DIAG-r116-disks-payload-2026-07-30.md`.
-
-A `REPORT-*.md` sibling, not the shared `REPORT.md` (`CLAUDE.md` parallel-session rule).
-
-## Outcome
-
-**Both goals met.** The `/disks` read path is solved and written down verbatim, proven by a
-present-drive control run *first*; and the absent-state payload was captured, which isolates the
-mechanism.
-
-**R-116 is theory #1 — "the registry-union row writes `false`" — the theory that was raised, declared
-wrong, and retracted. The retraction was the error.**
-
-In the absent state `/disks` returns **4 rows, not 3**. The drive appears twice and the two facts the
-controller needs sit on different rows:
-
-| row | source | `mount_path` | `guest_path` | `backup_target` |
-|---|---|---|---|---|
-| `felhom-backup` | Observe (`disks.go:196-284`) | `""` | `""` | **`true`** |
-| `694034cc-…` (the UUID) | registry union (`disks.go:297-339`) | `/mnt/cel` | `/mnt/felhom-drives/cel` | **field ABSENT ⇒ `false`** |
-
-So the row holding the flag contributes **no key** to `driveTargetByPath`, and the row that owns the key
-says `false` → `isTarget[a.Path]` is `false` → generic `storage_disconnected`. On return the rows
-re-merge into one carrying both facts → specific `backup_target_restored`. Applying
-`intermediary.go:602-618` to the captured payloads gives PRESENT `True` / ABSENT `False` /
-RETURNED `True` — **the live asymmetry reproduced from payload alone.**
-
-The union row's `MountPath` survives the device because the union source is the systemd **`.mount` unit
-file** (`registry_known.go:40-75` via `main.go:605`→`:764`), which never reads the mount table. The
-dedup at `:298` therefore does not fire, because `seen` is keyed on the one field the absent state
-empties (`:290-295`).
-
-**Theory #2 (the basis of the shipped v0.115.0) is false on both halves**; **#3 is false too**
-(`isTarget["/mnt/cel"]` is `false` as well). **v0.115.0 is provably inert** — its fallback calls
-`StablePathForRaw("")`, which returns `""` (`intermediary.go:69-75`), so it assigns nothing.
-
-## The read path (this cost two prior sessions — it should never cost again)
-
-The token plaintext exists in exactly one place: `bootstrap.json` **on the Proxmox host**. The agent's
-own store keeps SHA-256 hashes only (`tokenstore.go:26-32`), which is what defeated the earlier attempts.
-
-```bash
-ssh felhom-pve
-B=/var/lib/felhom-agent/guests/9201/bootstrap/bootstrap.json
-TOK=$(python3 -c "import json;print(json.load(open('$B'))['local_api']['token'])")
-EP=$(python3 -c "import json;print(json.load(open('$B'))['local_api']['endpoint'])")
-curl -sS -k -H "Authorization: Bearer $TOK" "https://$EP/disks" | python3 -m json.tool
-```
-
-Control run, live felhom-pve, drive present: **HTTP 200, 2483 bytes, 4 plausible rows** — so Part 5's
-0-rows-on-a-present-drive failure mode is excluded.
-
-## Where the absent state was staged
-
-No new box. The existing DooPlex **nested-PVE drill fixture** (`drill/drill.qcow2`, snapshot `virgin`) —
-my own host, zero production exposure, and it can hot-unplug a disk for a genuine device loss. Run with
-the **byte-identical live agent binary** (`sha256 f48544ad…`, `--version` 0.115.0) and every
-state-producing step through the real endpoints (`format` → `assign` → `guest-attach` →
-`backup/target`). Its present-state row matched felhom-pve's control run field-for-field before it was
-trusted. Non-production aspects (root/direct privileged mode, stubbed hub, a hand-written bearer-token
-record, no controller) are enumerated in the audit §4.
-
-## Two new findings, filed not chased
-
-- **R-117 (READY M) — outranks R-116.** After a detach/reattach the guest's bind is a **dead mount**:
-  host is healthy on the new device node, guest still names the old one, and `ls`/write through it
-  return **`EIO`** — while `/disks` reports `attached` + `bound_under_parent:true` + `backup_target:true`.
-  `planDriveGates` therefore takes the `Return` branch and **restarts the customer's apps onto a dead
-  namespace, reporting healthy, with no alarm on any channel.** R-113's conjunction cannot catch it:
-  one half is satisfied by the stale entry, the other by the new device, and neither compares them.
-  This is the "stale bind" seen and dismissed as cosmetic in three consecutive runs.
-- **R-118 (READY XS).** An absent drive's union row reports the **root filesystem's** capacity as its own
-  (46 GiB / 9.2 % for a 4 GB drive) — `statfsCapacity` at `disks.go:335-338` statfs's a bare directory on
-  root. `observe.go:176-183` guards the Observe path against exactly this; the union path does not.
-  `durable_id` is still correct, so re-attach identity is safe — it is a false capacity, not a DR mis-id.
-
-## Register
-
-`documentation/backlog/OPEN-ITEMS.md` — R-116 updated with the mechanism and the fix constraints;
-R-117 and R-118 added. The single register edit this session makes.
-
-## Record correction
-
-The brief's baseline `controller 0.185.1` is the version the **golden bakes**. **0.186.0** (R-114 +
-R-112, 2026-07-29) is what **demo-felhom** runs — **demo-hp is still on 0.185.1**, so the fleet is
-split, and R-114's `TargetAbsent` branch exists only on demo-felhom. Confirmed: agent **0.115.0**
-(felhom-pve) / **0.113.0** (demo-hp), hub **0.81.0** (manifest pin and live pod image agree),
-host-install **1.22.0**, `felhom.eu` HEAD `c3ce4c7`.
-
-> **Correction, 2026-07-30.** As first written this section said 0.186.0 was what *both* demo boxes run.
-> That was wrong — only felhom-pve's guest was sampled and the result generalised to the fleet. demo-hp
-> re-checked directly → `0.185.1`. Fixed here and in the audit's baseline table.
-
-## Teardown and fences
-
-Drill guest destroyed, scratch storage removed, mount unit deleted, secrets `shred -u`'d, VM powered
-off, **`drill.qcow2` restored to `virgin`** (the golden-bake fixture is exactly as found), scratch qcow2
-and console dumps deleted. DooPlex `/mnt/5_hdd` at 24 %, unchanged.
-
-Both demo boxes **read-only throughout** and re-verified after teardown: demo-hp `local-lvm` **38.83 %**
-(identical before/after and to Part 5), `drill-r50` still stopped, felhom-pve `felhom-backup` still
-active on `/dev/sdb`, both guests running, **v0.115.0 untouched**.
-
-`sess-d-0452c4` now reads **STALE**, and the delete gate refuses only on ONLINE
-(`hub/internal/web/customer_delete.go:220-228`) — so it **is** now deletable; the command is recorded in
-the audit rather than executed (customer delete runs external teardown plus a DB purge). **`sess-c` is
-also still present and was not recorded by the Session-C audit** — same terms.
-
-## Not done, deliberately
-
-No fresh controller gate-log / hub-event correlation: that observable was already captured live and
-identically twice, and only the payload was missing. Staging a controller meant a hub customer, a
-pairing, a golden fetch and a claim — the work that consumed the three prior sessions. The audit §8
-correlates the payload to those existing measurements and labels that step as inference.

REPORT-r117-spike.md

@@ -1,96 +0,0 @@
-# REPORT — SPIKE R-117: a dead bind that reports healthy (2026-07-30)
-
-Written as `REPORT-<topic>.md`, not `REPORT.md`, per this repo's parallel-session rule and the
-established local pattern (`REPORT-r116-diag.md`, `REPORT-session-c.md`, …). The shared `REPORT.md`
-was not touched.
-
-**Class: Spike.** Deliverable is a findings document. **No production code was written; no `.go` file
-in either repo was modified; nothing was built for deployment, published, or version-bumped.**
-
-## Deliverables
-
-| File | Action |
-|---|---|
-| `documentation/audits/SPIKE-r117-bind-liveness-2026-07-30.md` | **new** — the findings doc (Q1–Q7, evidence, recipe, probe comparison, recommendation) |
-| `documentation/backlog/OPEN-ITEMS.md` | **R-117 row rewritten** with the mechanism, the reproduction recipe and the fix constraint — the one register edit, per the brief |
-
-**No CHANGELOG entry.** This repo's changelogs are per-area (`hub/`, `scripts/`, `website/`); a
-documentation-only change belongs to none of them. Stated rather than silently skipped, per standing
-rule 4.
-
-## Baselines
-
-agent `main` **v0.116.0** @ `d4eb259` · controller `main` **v0.186.0** @ `b331f18` ·
-`felhom.eu` HEAD `29bcfeb` · hub **live 0.82.0** · golden **0.186.0** ·
-demo-hp PVE 9.2.2 / kernel 7.0.2-6-pve, **live agent 0.113.0** (= manifest `MinAgent`; never used as
-the source of a behavioural claim — every predicate result came from a probe built from `main`).
-
-## Results
-
-**All seven questions answered empirically.**
-
-- **Q1 — reproduced 3/3**, two device classes, on a purpose-built scratch LXC (9301) on demo-hp.
-  **The device-node change is a consequence of the defect, not a precondition** — control test: with
-  the stale bind held the drive returns as `sdc` (8:32); released, the letter is reused (`sdb`, 8:16).
-- **Q2 — two death states**: device removed ⇒ superopts gain `shutdown`, `EIO`(5) on read and write,
-  host and guest; device errors in place ⇒ `emergency_ro`, write `EROFS`(30), reads served from cache.
-  The raw host mount is genuinely healthy in both. **No cross-device mis-identification is possible**
-  on this path — the unit is fs-UUID-keyed.
-- **Q3 — confirmed at source and live.** Both halves of the R-113 conjunction compare **field 5** of a
-  mountinfo line and **never read field 3 (`major:minor`)**, so neither can see that the bind and the
-  raw mount name different devices. Measured `BoundUnderParent = TRUE` over an `EIO` namespace.
-- **Q4 — a pure-`/proc` check costs 0.16–0.45 ms**, cannot hang, spins up no disk, writes nothing.
-  **`statfs` and `getdents` both SUCCEED on a dead namespace** — probes built on either are hollow. The
-  hang case is below; it is the sharpest result in the run.
-- **Q5 — the agent**, and not on balance: the controller runs inside the guest and cannot see the host
-  mount tables the check needs.
-- **Q6 — recovery works in place, guest never restarted** (init PID identical). **The repair code
-  already exists and three call sites already invoke it**, including the controller's `Return` branch
-  *before* it restarts apps — all defeated by one early return.
-- **Q7 — a bind can die in steady state, no cycle at all.** The gate produces no action and **nothing
-  is emitted on any channel.** A `Return`-branch fix cannot reach this half.
-
-## Q4's hang case — measured, and it is the sharpest result
-
-Against a `dmsetup suspend`ed device (I/O queues instead of returning `EIO`):
-
-- **P1 and P2 completed in 364 µs / 206 µs.** They read `/proc`, so no block device is involved.
-- **`statfs` and `getdents` completed and reported HEALTHY** — on a wedged device they do not even hang.
-- **Every probe that touches the device blocked, including a buffered write with no `fsync`** — the
-  `O_CREAT` metadata path needs journal access (`wchan=do_get_write_access`). There is no cheap-and-safe
-  write probe.
-- **The blocked process survived `SIGTERM` and `SIGKILL`** (`stat=D`, still alive 3m50s after `kill -9`)
-  and died only when the device was resumed. So **`systemctl restart felhom-agent` would hang**, leaving
-  the agent unrecoverable until the device returns or the host reboots. The thread count does not reveal
-  the leak (5→5, 5→6).
-
-**A timeout protects the caller's control flow and nothing else.** This turns "prefer a cheap probe" into
-a fence: **the fix must issue no block I/O.**
-
-## Teardown — done, all three layers
-
-Guest 9301 destroyed; `r117scratch` removed; both dm devices and both loop devices gone; `scsi_debug`
-unloaded (`/dev/sd*` back to `sda1..3`); no `r117` mounts, `/mnt` and `/root` exactly as found; `local`
-**37.02 %** against a session-start **37.00 %**. Fences re-verified *after* teardown: 9201 `running`,
-`drill-r50` `stopped`, `local-lvm` **38.84 % byte-identical**, `felhom-backup` `content backup`
-unchanged, live `/mnt/felhom-drives` intact with both submounts, agent service `active`. **Layer 3 is
-genuinely empty** — 9301 had no network interface and ran no controller, so no hub-side record was ever
-created.
-
-**Ordering trap worth keeping:** a suspended dm device must be `dmsetup resume`d *before* any `umount`,
-or the teardown itself blocks on the same uninterruptible sleep.
-
-## Not measured
-
-No load or duration testing of the recommended check — P1/P2 were single calls, not a sustained
-reconcile loop on a many-drive box. Nothing suggests a problem (they are two `/proc` reads the code
-already performs), but it was not measured.
-
-## Findings filed, none fixed
-
-R-117 (mechanism + recipe), **R-117a** steady-state death with no event (HIGH, larger than R-117 as
-filed), **R-117b** `statfs`/`getdents` are hollow liveness probes, **R-117c** three untested comments
-promising "live + usable in the guest", **R-117d** the self-heal that already exists is short-circuited
-(HIGH), **R-117e** both demo boxes share one failure domain — no route survives the site losing internet,
-including the WireGuard OOB path, **R-117f** an I/O liveness probe turns a wedged drive into an
-unkillable agent (HIGH — disqualifies a whole probe class).

REPORT-r172-hub-wal.md

@@ -1,160 +0,0 @@
-# REPORT — hub v0.88.0: the WAL that never was (R-172), plus R-173 found
-
-**Session artefact naming:** written as `REPORT-r172-hub-wal.md`, not `REPORT.md`, per this repo's
-parallel-session rule — the shared `REPORT.md` belongs to the controller boot-recovery session that
-ran immediately before this one and must not be clobbered.
-
-**Repo:** `felhom.eu` (hub `v0.87.0` → **`v0.88.0`**) · **Trigger:** a `HOST STALE` banner the
-operator spotted on `hub.felhom.eu` after the previous session finished.
-
----
-
-## 1. What the alarm actually was
-
-**Not the agent, not the guest.** The agent was up **2 days**, never restarted, and actively
-reconciling; the controller was reporting normally (the header read "Last report just now", 0.190.0,
-10/10 containers). The failure was the hub **writing** the host report:
-
-```
-20:26:34 [ERROR] Failed to save host-report from demo-felhom-8363b5: database is locked (5) (SQLITE_BUSY)
-20:41:32 [ERROR] Failed to save host-report from demo-felhom-8363b5: database is locked (5) (SQLITE_BUSY)
-20:42:32 [INFO]  Host staleness: demo-felhom-8363b5 ok → stale (host_stale)
-20:42:33 [INFO]  Operator email sent for demo-felhom/host_stale
-```
-
-The chain, and the margin is exactly one retry wide: reports are every **15 min**, staleness fires at
-**30 min**, the hub returns **500** on `SQLITE_BUSY` without retrying, and the agent logs
-`keeping current interval` and waits a full interval without retrying either. **Two consecutive
-collisions = a false alarm.** It had already fired once that day (19:12:32, recovered 19:20:32).
-
-**Was it caused by the preceding session?** Partly amplified, not caused. 13 collisions in one pod
-lifetime; **the first at 15:56 CEST, ~3 h before that session's first deploy**. 7 of 13 fell inside
-its window of ~13 controller restarts, which raises write concurrency — so the burst made a
-pre-existing fault more likely, and the fault was not new.
-
-## 2. Root cause — the pragmas were never applied
-
-The DSN was `?_journal_mode=WAL&_busy_timeout=5000`. That is **mattn/go-sqlite3** syntax. The driver
-is **modernc.org/sqlite v1.45.0**, whose `applyQueryParams` (confirmed at source in the module cache)
-reads only `_pragma`, `_time_format`, `_time_integer_format`, `_txlock` and `_inttotime` — and
-**silently ignores everything else**. No error, no warning.
-
-So the hub ran in the default **rollback-journal** mode with **`busy_timeout=0`** for its entire life
-while its own source said WAL. In rollback-journal mode a reader excludes a writer, so rendering an
-operator page can block a host report — which is precisely the observed 500.
-
-**The observable that proved it before any code changed:** a 128 MB `/data/hub.db` with **no
-`-wal`/`-shm` file beside it while the database was open**. In WAL mode those must exist.
-
-This is the project's recurring class — a configuration asserting an invariant the code does not
-provide — and it is the second one this week.
-
-## 3. The fix
-
-```
-?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)&_txlock=immediate
-```
-
-| Parameter | Why it is not optional |
-|---|---|
-| `journal_mode(WAL)` | readers and one writer proceed concurrently, so a page render can no longer block a report; it is a property of the database FILE and persists once set |
-| `busy_timeout(5000)` | writers still serialise; without a timeout SQLite returns `SQLITE_BUSY` *immediately* rather than waiting |
-| `_txlock=immediate` | **the one that is easy to miss.** `database/sql`'s `Begin()` is DEFERRED, so a read-then-write transaction must upgrade its lock, and a failed upgrade is `SQLITE_BUSY_SNAPSHOT` — which **`busy_timeout` does not retry**. This store has **10+ `db.Begin()` sites and they are all write paths** (customer delete/reset, wg, appliance, pbsdr, telemetry, log bundles). WAL + busy_timeout alone would have shipped half a fix with a known un-retryable path left open |
-
-**Retry options (b) and (c) from R-172 were deliberately NOT taken.** With readers no longer blocking
-writers and the upgrade path covered, a `SQLITE_BUSY` reaching a handler should now be rare enough to
-be a real signal; a retry would hide it. Revisit only on evidence.
-
-## 4. Tests and the red-proof
-
-**Every assertion reads the value back from the DATABASE, never the DSN string** — a string assertion
-would have passed happily for the entire life of the bug. Six tests in `internal/store/pragma_test.go`:
-
-| Test | Asserts |
-|---|---|
-| `TestStorePragmasAreActuallyApplied` | runtime `journal_mode` = wal, `busy_timeout` ≥ 5000 |
-| `TestStoreWALFilesExistWhileOpen` | `-wal`/`-shm` exist beside an open DB — **the production signature, pinned** |
-| `TestStoreReaderDoesNotBlockWriter` | the CONSEQUENCE: a write during a held read succeeds |
-| `TestStoreConcurrentWritersDoNotReturnBusy` | 8 concurrent writers all wait rather than error |
-| `TestStoreTransactionUpgradeDoesNotReturnBusySnapshot` | 6 racing read-then-write transactions all commit |
-| `TestSQLiteDriverIgnoresMattnStyleParams` | guards the ROOT CAUSE: fails if the pragmas are "tidied" back to mattn form; skips itself with instructions if a future driver starts honouring them |
-
-**Red-proof — restore the DSN that shipped.** Observed FAIL, then reverted with a passing control:
-
-```
-journal_mode = "delete", want "wal"
-hub.db-wal is missing beside an OPEN database
-a write FAILED while a read was open: database is locked (5) (SQLITE_BUSY)   ← the live error, exactly
-```
-
-`go build ./... && go vet ./... && go test ./...` in `hub/` → **rc=0**. `scripts/repo_gates.py --fast`
-→ all 5 gates OK.
-
-## 5. Operational consequence — handled, not discovered later
-
-**A WAL database cannot be copied by taking `hub.db` alone.** A committed transaction may still live
-in `hub.db-wal`, so a bare `cat` yields a copy that **opens cleanly and silently omits the newest
-writes** — the worst possible shape for a credential lookup. The break-glass root-password retrieval
-in `documentation/operations/nodes.md` used exactly that command, and `_recovery-inventory-2026-07-28.md`
-records it as a past action that reads like a recipe. Both are now WAL-aware: copy the `-wal`
-alongside, `|| true` because an absent `-wal` is legitimate, and **shred both** (the WAL holds the
-same secrets).
-
-Not hypothetical: the live `-wal` measured **729,272 bytes** during verification, all of which a bare
-`cat` would have dropped.
-
-## 6. Live verification
-
-- Image built and pushed; **`manifests/hub.yaml` 0.87.0 → 0.88.0** (the only thing ArgoCD deploys
-  from), hard-refresh + deliberate sync (auto-sync is OFF). ArgoCD **Synced / Healthy**.
-- `/data/` now shows **`hub.db-wal` and `hub.db-shm`** beside the open DB — the exact observable whose
-  absence proved the bug.
-- **Zero `SQLITE_BUSY` since the rollout.**
-- Host report landed at 21:11:33; staleness checker reports `2 ok, 0 stale` — `demo-felhom-8363b5`
-  is back to `ok`.
-- `PRAGMA integrity_check` → **`ok`**, `journal_mode` → **`wal`**, all tables intact
-  (hosts 4, customer_configs 6, host_recovery 4, host_reports 2756, events 2293). Read via the new
-  WAL-aware copy recipe and shredded afterwards.
-
-## 7. A mistake I made, and it caused a real outage
-
-**I OOM-killed the hub pod three times** with a 60-concurrent page-render load test intended to prove
-the fix under contention. The pod's limit is **256 Mi**; 60 simultaneous renders of a heavy customer
-page exceeded it (`OOMKilled`, exit 137, readiness probe timeouts). The hub was unavailable for parts
-of ~6 minutes and recovered on its own.
-
-Three things worth stating plainly:
-
-1. **It was not the WAL change.** WAL's extra footprint is the 32 KB `-shm` mapping; the OOM was
-   template rendering under concurrency I created.
-2. **I should not have run it.** DooPlex is **Tier 2 — precious**, and the hub is part of the
-   recovery chain. A synthetic load test at that concurrency against a memory-limited pod on that box
-   was the wrong call; the unit tests already proved the property, and the live proof needed was the
-   `-wal` file plus a clean report — both of which I already had.
-3. **It did produce one piece of genuine evidence**, which does not excuse it: after three hard kills
-   the WAL replayed cleanly and `integrity_check` returned `ok`, which is a real (if unplanned)
-   demonstration of WAL crash-safety on this volume.
-
-No data was lost. The pod is `Ready`, restart count 3, serving normally.
-
-## 8. Backlog
-
-- **R-172 → CLOSED**, with the root cause recorded as *the pragmas were never applied*, not as tuning.
-- **R-173 → NEW.** While checking who copies `hub.db` for the WAL change, I found `pvc/hub-data`
-  carries `recurring-job-group.longhorn.io/default: disabled`, and `backup-daily` + `backup-weekly`
-  are the only recurring jobs and both target `default`. **The hub database has no volume-level
-  backup** — and it holds `host_recovery` (every box's break-glass root password), `host_escrow` +
-  `host_escrow_superseded`, `host_pbs_secrets`, `customer_configs`, `dr_recipe` and the wg peers.
-  Filed rather than fixed: whether the exclusion is deliberate is a question for the operator, and the
-  manual hot copy recorded in `_recovery-inventory` is not a backup. `grep` established the ID free.
-
-## 9. Observations — noticed, NOT acted on
-
-1. **The hub returns HTTP 500 for a transient lock**, which is what turned a retryable condition into
-   an alarm. Left as-is deliberately (§3) so a surviving `SQLITE_BUSY` stays visible.
-2. **The agent does not retry a failed report**, so one collision costs a full 15-minute interval —
-   half the staleness budget. Same reasoning; if collisions recur, this is the cheaper of the two
-   retry fixes.
-3. **`store.New` sets no `SetMaxOpenConns`**, so `database/sql` may open unbounded connections against
-   a single-writer database. Not changed here — WAL plus the immediate-lock covers the observed
-   failure, and bounding the pool changes latency characteristics that nothing currently measures.

REPORT-record-correction-2026-07-29.md

@@ -1,203 +0,0 @@
-# REPORT — installer-channel record correction + R-29 filing (2026-07-29)
-
-Two commits, documentation only. No code, no version bump, no CHANGELOG entry, no build, no deploy,
-no box touched. Written as `REPORT-<topic>.md` per `CLAUDE.md:82-87` so root `REPORT.md` (the E-2
-increment-1 report) is preserved.
-
-| # | Commit | Baseline | Scope |
-|---|--------|----------|-------|
-| 1 | `d4c07873ca0c3d3e547373a9fafc0e472a6535e8` | `36d635a4cdc1`, unmoved | Retract a false R-94/E-2d finding; open R-110 |
-| 2 | (this commit) | `d4c07873`, unmoved | File R-29 to the register; three record defects; this report |
-
----
-
-## Commit 1 — `d4c07873`
-
-### What was false
-
-`36d635a4` recorded that `felhom-bootstrap.sh` fetches the installer **from the hub**, that the hub
-therefore serves 1.19.0, and that a fresh ISO install runs the pre-E-2 installer. All three wrong.
-The claim had propagated into two `OPEN-ITEMS.md` rows, the ranking rationale, and `ROADMAP.md:149`.
-
-### Confirmation table — all PASS
-
-| # | Claim | Read at | Result |
-|---|-------|---------|--------|
-| F1 | bootstrap fetches from the **website** | `scripts/iso/felhom-bootstrap.sh:96` | PASS — `INSTALL_URL="${FELHOM_INSTALL_URL:-https://felhom.eu/scripts/felhom-host-install.sh}"` |
-| F2 | hub-rendered command points at the same URL | `customer_unified.html:563`, `:1262` | PASS — and **three** emission sites, not two: `:563` static, `:1262` JS error branch, **`:1267` JS success branch** |
-| F3 | website serves `/scripts/` from a git-sync tree tracking `main` | `manifests/webpage.yaml` — nginx `:74-77`, sparse-checkout CM `:211-218`, git-sync `:272-281`, init `:299-307` | PASS — `--branch=main --period=30s --link=current`; sparse-checkout `/website/` + `/scripts/`; `location /scripts/ { root …/current; }`. No image build, no ArgoCD step |
-| F4 | `hostInstallVersion` selects nothing | `configs.go:28`, `:487`; `render_test.go:219`; `customer_unified.html:494` | PASS — repo-wide grep returns exactly those 4 code sites; all other hits prose. Rendered as a text label |
-| F5 | every generated flag is parsed by 1.22.0 | generator `customer_unified.html:1206-1239` vs parser `felhom-host-install.sh:1175-1212` | PASS — `--mode --cores --memory --vmid --node --acl-storages --operator-pubkey-file --preserve-state-from --skip-provision --dry-run --preflight-only --allow-new-leaf` (+ `--customer-id`); every one a parser case. **No functional gap** |
-| F6 | installer is 1.22.0 | `scripts/felhom-host-install.sh:187` | PASS |
-| F7 | the drift test is hollow | `render_test.go:219-221` | PASS — `strings.Contains(html, hostInstallVersion)` compares the constant to itself; passes at any value |
-
-### Live command 1 — what the URL actually serves
-
-```
-$ curl -fsS https://felhom.eu/scripts/felhom-host-install.sh | grep -m1 '^SCRIPT_VERSION='
-SCRIPT_VERSION="1.22.0"   # the SINGLE version source (F-1): -h, the run banners, and the hub
-```
-
-### Live command 2 — the drift gate's real state
-
-```
-$ python3 scripts/hostinstall_gates.py; echo "exit=$?"
-  ok: SCRIPT_VERSION=1.22.0
-  ok: header has no version literal
-FAIL: hub Setup-tab hostInstallVersion=1.19.0 != SCRIPT_VERSION=1.22.0 (F-1: bump both together)
-  ok: age is in the installed package set
-  … (six further ok lines) …
-hostinstall gates: 1 FAILURE(S)
-exit=1
-```
-
-### Phase 0 source read — PAIRING reaches the same installer invocation
-
-Mode selection `felhom-bootstrap.sh:537-541`: a fresh VM with no baked customer-id calls
-`run_pairing`. On HTTP 200 from `/api/v1/appliance/poll` the loop writes the hub-delivered
-`FELHOM_CUSTOMER_ID` + `FELHOM_RETRIEVAL_PASSPHRASE` into the 0600 env, re-sources it, and calls
-`run_direct` **in the same invocation** (`:495-499`). `run_direct` is the single site that fetches
-`$INSTALL_URL` (`:322-330`), builds the args (`:334`) and invokes `bash "$SCRIPT_TMP" "${args[@]}"`
-(`:343`). The customer it yields is the one the operator bound — claimable. **So the ISO leg is the
-spine for E-2d**, not an obstacle to it.
-
-### Rows changed
-
-| ID | Before | After |
-|---|---|---|
-| R-94 (line 14) | `READY — deferred until E-2d`, blocked on E-2d, false ISO/hub framing | `READY (XS)`, blocked on nothing, retracted + re-scoped to three legs |
-| R-94 (line 16) | duplicate row, `READY #2`, 1.19.0 vs 1.20.0 | **deleted** — merged |
-| R-110 | did not exist | opened, `WAITING-ON-OPERATOR (S)` |
-| E-2d | ISO implied as obstacle | Next-action appended; ISO is the spine |
-| ranked list | 1 R-95 · 2 R-94 (high-consequence) · 3 R-86 · 4 R-87 | 1 R-95 · 2 R-94 **de-ranked** · 3 R-86 · 4 R-87 · 5 R-110 |
-
-Files: `documentation/backlog/OPEN-ITEMS.md`, `documentation/backlog/ROADMAP.md`,
-`documentation/runbooks/day0-install.md`.
-
----
-
-## Commit 2 — R-29 filing + record hygiene
-
-### The §1.1 ruling — is R-29 the right home for a non-design-v2 gate? **Yes. Proceeded.**
-
-R-29's title says *"the design-v2 green gates"*, and `scripts/hostinstall_gates.py` is not one — it
-comes from drill F-1 (2026-07-12) and postdates the item. Four things decide it anyway:
-
-1. **R-29's own audit list already spans well beyond design-v2 subject matter.** It names
-   `docker_run_volume_path_gate` (docker mount safety), `offbox_rename_gate`, `app_row_dedup_gate`
-   and `manifest_bearer_gate` (secrets — `runbooks/secrets.md:76`). The title is a misnomer relative
-   to the item's own body.
-2. **Part (b) — "the systemic half is the real item" in R-29's words — is stated with no
-   design-v2 restriction.** It is about the *enforcement mechanism*: "the gates run only when a human
-   remembers to run them… decide where they run (pre-push hook, `build.sh` step, or a CI job) and make
-   a red gate block the train." That is gate-agnostic and repo-wide.
-3. **`hub_confirm_gate.py` is already on R-29's list and lives in the same `scripts/` directory** as
-   `hostinstall_gates.py`. Wiring one and not the other would be arbitrary.
-4. **Identical failure shape, identical genre.** Both self-describe as mechanical grep-assertions
-   (`hostinstall_gates.py:2`, `hub_confirm_gate.py:1-8`); both exist, assert something true, and are
-   invoked by nothing.
-
-R-29 has already absorbed one independent re-raise without minting an ID (2026-07-18 rehearsal note)
-and says so explicitly. This is the third. No new ID minted.
-
-### Orphan-search evidence — `hostinstall_gates.py` and `hub_confirm_gate.py`
-
-Re-established at `d4c07873`. Pattern `hostinstall_gates\|hub_confirm_gate`.
-
-| # | Scope | Command | Result |
-|---|-------|---------|--------|
-| S1 | `felhom.eu`, all file types | `grep -rn "$PAT" . --exclude-dir=.git` | **19 hits, zero invocations.** All are docstrings (`scripts/hostinstall_gates.py:5`, `scripts/hub_confirm_gate.py:7`), code comments (`hub/internal/web/configs.go:27`, `scripts/felhom-host-install.sh:189`) or prose (`REUSE.md:62`, `CONTEXT.md:540,564`, `hub/CHANGELOG.md:371,1292,1351,1385`, `scripts/CHANGELOG.md:483,524`, 3 files under `documentation/audits/`, `ROADMAP.md:149,158`, `OPEN-ITEMS.md:14`) |
-| S2 | sibling repos | `grep -rln "$PAT" /mnt/5_hdd/felhom.eu/git --exclude-dir=.git` minus this repo | 3 files, all in `.claude-memory/` (`MEMORY.md`, `drtier-by-default-2026-07-12.md`, `polish-batch-2026-07-13.md`). Notes, not invokers |
-| S3 | `~/.claude` | `grep -rln "$PAT" /home/kisfenyo/.claude`, and targeted on `settings*.json`, `skills/`, `hooks/` | **Zero hits in settings, skills or hooks** — where an invoker would live. Remaining hits are `file-history/` (Claude Code's own backups of files edited in past sessions) and `paste-cache/` (pasted task specs). Neither is an invocation site |
-| S4 | git hooks | `ls -1 .git/hooks/ \| grep -v '\.sample$'` | **Empty — every hook is a `.sample` stub** |
-| S5 | build files | `find . -type f \( -iname Makefile -o -iname '*.mk' -o -iname justfile -o -iname 'Taskfile*' \)` | Only `hub/Makefile`; `grep -n gate hub/Makefile` → **zero occurrences** |
-| S6 | CI | `find . -type d \( -name .github -o -name .gitea -o -name .woodpecker* -o -name .drone* -o -name .circleci \)` | **Empty — `felhom.eu` has no CI configuration at all** |
-
-Of the four gates in `scripts/`, only `site_gates.py` is mandated (`CLAUDE.md:153`);
-`manifest_bearer_gate.py` is named in `runbooks/secrets.md:76`.
-
-### Rows changed
-
-| ID | Before | After |
-|---|---|---|
-| **R-29** | **absent from `OPEN-ITEMS.md`** (`grep -c` → 0) while present at `ROADMAP.md:158` since before the 2026-07-27 rebuild | **opened, `READY (S for (a) / M for (b))`**, owner CC |
-| R-94 | leg (b) stood alone | leg (b) carries `→ R-29` as its class. Row otherwise untouched |
-| E-2d | cited `:322-341` for an invocation at `:343` | cited `:322-343`, with the fetch / args / call each pinned separately |
-| R-95, R-86, R-87 | `READY #1`, `READY #3`, `READY #4` | `READY` — markers dropped (see below) |
-
-Not added to the ranked list under *"Why the READY rows rank this way"*: that list is a top-N
-rationale, not a complete ordering (R-99, R-102, R-103, R-104, R-108, R-109, R-89, R-92, R-93 and
-E-2d are all READY and absent from it). Ranking R-29 was not asked for and would be a priority claim
-this task has no basis to make.
-
-### §2.2 — markers dropped, not renumbered
-
-The `#1/#3/#4` markers duplicated ranked-list positions 1/3/4 for exactly those three rows; two
-orderings of one set is what produced the hole when the `#2` row was merged in `d4c07873`. Removing
-the duplicate leaves the ranked list as the single maintained ordering.
-
-### The other two record defects
-
-- `ROADMAP.md:147` — cited a non-existent **R-164**; it means controller **v0.164.0**'s
-  deliberate-stop filter. Corrected. (It had already cost one max-ID scan a false positive.)
-- `CONTEXT.md:564` — asserted in the present tense that the single VERSION source is *"gated by
-  `scripts/hostinstall_gates.py`"*. The gate exists, asserts exactly that, is red, and runs nowhere.
-  Corrected to say the cross-check exists but is not enforced, tracked as R-94 leg (b) / R-29.
-  `hub/internal/web/configs.go:27` carries the same false claim in a code comment and was left alone
-  — R-94 leg (b) territory, needs a hub build.
-
-Files: `documentation/backlog/OPEN-ITEMS.md`, `documentation/backlog/ROADMAP.md`,
-`documentation/backlog/README.md`, `CONTEXT.md`, this file.
-
----
-
-## Commit 3 — `de5a3e5` — the record-hygiene rider
-
-Baseline `7383400a`, unmoved. Four XS items from `7383400a` §9; the three deferred observations stay
-deferred.
-
-| File | Change |
-|---|---|
-| `CONTEXT.md:540` | *"`scripts/hub_confirm_gate.py` enforces"* → asserts, but is not enforced (R-29). Third instance of the class after `:564` and `configs.go:27` |
-| `REUSE.md:62` | Same claim, *"enforces zero"*. The **rule stays** — never native `confirm()`/`prompt()` is correct guidance and this is a reuse-reference row — only the enforcement claim changes |
-| `OPEN-ITEMS.md:4` | Root `REPORT.md` = overwritten per-session; `REPORT-<topic>.md` = non-clobbering sibling (`CLAUDE.md:82-87`), 14 of them. Prohibition unchanged |
-| `OPEN-ITEMS.md:55` | Heading scoped to *"the **TOP** READY rows"* + a half-sentence that it is deliberately not a full ordering. **No row added to the list** |
-
-`hub/internal/web/configs.go:27` — the fourth instance — left alone (R-94 leg (b), needs a hub build).
-
-### Part 3 NOT done — its stated evidence is false
-
-The spec proposed recording a convention: *"`ROADMAP`'s P3 `Status` column only ever takes `idea` or a
-`SHIPPED`/`PROVEN-LIVE` variant, and never takes `READY`, `BLOCKED` or `WAITING-ON-OPERATOR`"*, and
-concluded from that vocabulary split that `ROADMAP` `Status` is **disposition** while `OPEN-ITEMS`
-`State` is **live actionability**.
-
-Refuted inside P3 itself (`ROADMAP.md`, lines 113–172):
-
-| Line | ID | P3 `Status` cell |
-|---|---|---|
-| 151 | R-90 | `BLOCKED on Hetzner CX33 availability (operator, 2026-07-27)` |
-| 152 | R-91 | `WATCHING — gated on demo-felhom's first post-migration PBS backup` |
-| 150 | R-110 | `idea — found 2026-07-29, **WAITING-ON-OPERATOR (a ruling, not a defect)**` |
-| 137 | R-86 | `**NEXT — operator ruling 2026-07-27**` |
-
-And P2's recovery-gaps sub-table (lines 99–107) uses a bare `READY` for nine rows, including `E-2`.
-
-Both halves fail: the column **does** take the `OPEN-ITEMS` vocabulary, and `BLOCKED`/`WATCHING`/
-`NEXT` are live-actionability words, not dispositions. The real shape is a free-text status field
-that has absorbed both vocabularies over time — usually disposition, sometimes not. Writing the
-proposed convention down would install a **new false invariant** in the commit chain whose whole
-purpose was removing them, so per the spec's own instruction (*"If you think this ruling is wrong, say
-so and make no edit"*) `documentation/backlog/README.md` was not touched.
-
-The narrow question that prompted it still resolves benignly: R-29 and R-94 reading `idea` in
-`ROADMAP` and `READY` in `OPEN-ITEMS` is **not** a defect and needs no sync. What is unsupported is
-generalising that into a rule.
-
----
-
-## Standing note
-
-**No gate was wired, fixed, run as a hook, or deleted by any of the three commits.** Filing R-29 is
-not doing R-29; the enforcement decision (pre-push hook / `build.sh` step / CI) and the audit of the
-remaining eight gates are R-29 part (b), M-sized, and its own task.

REPORT-session-c.md

@@ -1,94 +0,0 @@
-# REPORT — Session C: R-113, R-114, R-112 proven on a real box; C5 fails on a new defect (2026-07-29)
-
-`RUNBOOK-session-c-2026-07-29.md`. Full evidence: `documentation/audits/SESSION-C-2026-07-29.md`.
-Root `REPORT.md` untouched.
-
-## Verdicts
-
-| Claim | Fix | Verdict |
-|---|---|---|
-| **C4** — offer appears and moves the target | R-112 | ✅ **PASS** |
-| **C3** — absent target tells the truth, offers nothing | R-114 | ✅ **PASS** |
-| **C5** — `backup_target_absent` + `backup_target_restored` | R-113 | ❌ **FAIL** — generic alarm, specific recovery → **R-116** |
-
-**All three shipped fixes work.** R-113's gate fires in **4 seconds** (E-2d measured zero over 4½
-minutes). R-114's message is correct. R-112's banner reaches the customer. C5 fails on a **fourth,
-separate defect that was unreachable until R-113 made the gate fire at all.**
-
-## What the drill box ran
-
-Agent **0.114.0 from the Day-0 manifest** — the shipped binary, so C5 tested the real artifact and
-closed R-115's observation 1 for R-113. Controller **0.186.0 hand-deployed** after install (§3.1
-ruling (a)); the vouched golden bakes 0.185.1, so **C3/C4 prove the code, not the shipped golden** —
-that lag is filed against R-115, not a new ID.
-
-## The three headline observables
-
-**R-113** — detach at 18:43:50, gate at **18:43:54**, on exactly the shape that defeated it before:
-```
-raw   /mnt/mentes               : NOT mounted
-bind  /mnt/felhom-drives/mentes : /dev/sdb[/felhom-data]   <- the stale bind SURVIVED
-```
-
-**R-114** — with the target absent: absent copy 1, **system-disk copy 0**, **offer block 0**. Both of
-E-2d's falsehoods gone.
-
-**R-112** — banner element 1 and the never-configured copy in the HTML; after the wizard, the offer
-with `data-path="/mnt/felhom-drives/mentes"`. Healthy renders nothing, **proven positively**: idle
-delta 0 `/backup/tiers` calls, page-load delta **+1**, single caller ⇒ the seam ran and chose silence.
-
-**Decline path proven** (registration confers no role), `restart_required:true`, agent did **not**
-self-restart, in-flight check recorded before I restarted it, wrapper created the storage at the
-drive's own mountpoint.
-
-## C5's failure
-
-```
-absent : Event pushed: storage_disconnected (error)      <- GENERIC
-return : Event pushed: backup_target_restored (info)     <- SPECIFIC
-```
-
-`backup_target_absent` count **0** across the whole run. The alarm and its recovery cannot be matched
-— precisely what `notifyDriveReturned`'s own comment forbids.
-
-**Root cause (R-116):** `driveTargetByPath` builds `out[GuestPath] = d.BackupTarget`, but the drive is
-**two `/disks` rows** and the flag and the guest path sit on different ones — the `felhom-backup`
-storage row carries `BackupTarget: true` and gets a guest path only while classified user-data; the
-registry union row carries the guest path and **never assigns `BackupTarget`**. Absent ⇒ they separate
-⇒ generic. Return ⇒ they rejoin ⇒ specific. v0.184.1 fixed the *keying*, not this.
-
-## Mirror + over-correction guard — PASS, with a caveat
-
-Non-target drive detached ⇒ `storage_disconnected`, `backup_target_absent` count 0. **Over-correction
-guard passes**: both drives present ⇒ 0 ABSENT lines, target stayed healthy — R-113's stricter presence
-did not make a healthy drive read absent. **Caveat: the mirror passes trivially**, because the target
-also produced the generic event; it confirms no over-correction but cannot confirm discrimination.
-
-## Record
-
-- `OPEN-ITEMS.md` — **R-113, R-114, R-112 → SHIPPED + PROVEN-LIVE**; **R-116 opened** (READY (S), P1);
-  **E-2 and E-2d CLOSED as partially proven** with R-116 as the one named open leg, per the runbook's
-  §9 decided-in-advance rule.
-- `ROADMAP.md` — R-116 under P1.
-- **Capability map NOT touched** — it still has **no E-2 / backup-target rows at all**, so no row could
-  be moved to PROVEN-LIVE. Creating them is a design act, not a validation act. Third session running
-  that this has been noted.
-
-## Teardown
-
-VM destroyed, storage removed, **`pvesm status` after == before** (`local-lvm` 38.78 %), guest 9201 and
-`drill-r50` untouched. **Customer ruling: DELETE**; attempted and correctly refused (`host … is
-ONLINE`) — deletable once the destroyed host ages to DOWN (>1 h), command recorded in `OPEN-ITEMS.md`.
-
-## What did not happen
-
-`backup_target_absent` never fired, so its severity, Hungarian copy and hub routing remain unexercised
-— R-116 blocks them. The offer was accepted via the endpoint the button POSTs, not a browser click (no
-browser automation on DooPlex); the rendered control and its non-auto-submission were verified in HTML.
-The stale bind still naming a dead device node after return was observed, not investigated.
-
-## The arc
-
-E-2 ends here. Its stated definition of done is **R-106 + R-109, R-108 and D5** — none of which this
-detour touched. The detour was worth taking: it found six real defects (R-111 through R-116), four of
-them customer-affecting, none of which any unit suite had caught.

REPORT-tester-gate-2026-07-31.md

@@ -1,98 +0,0 @@
-# REPORT — tester gate: golden re-baked to 0.188.0, fresh-install proof PASSED (2026-07-31)
-
-Written as `REPORT-<topic>.md` per `CLAUDE.md:82-87` so the shared `REPORT.md` (E-2 increment 1) is
-not clobbered. Full record with every observable: `documentation/audits/tester-gate-golden-0.188.0-2026-07-31.md`.
-
-## Outcome
-
-**§7.2 — YES: a fresh install is safe to hand to an external tester.** ISO boot → claimable,
-app-serving box in ~10 minutes unattended, and an app's data restored **from the drive with the
-guest's `app.yaml` gone**, proven readable by the application over its own TCP path.
-
-**Golden 0.186.0 → 0.188.0** baked, published, vouched. **No ISO rebuilt** — Part 0 proved none was
-needed. No existing box changed; floor still v0.156.0, MinAgent still 0.113.0.
-
-## Part 0 — the ISO does not need rebuilding
-
-Verified against the ISO **on disk**, not from source. It bakes exactly three Felhom payloads
-(`felhom-bootstrap.sh`, its unit, the secret-free pairing env) — full-base64 match, 1 hit each — and
-**0** hits for `SCRIPT_VERSION="1.2`, `felhom-controller`, `vzdump-lxc-9100`. The installer is fetched
-at run time (`felhom-bootstrap.sh:96`) and the live URL is byte-identical to repo HEAD
-(sha `ab8b283e…`, v1.22.0, committed six days *after* the ISO). The golden arrives via the hub-vouched
-artifact manifest (`felhom-host-install.sh:423-433`). The one genuinely baked, drift-capable thing is
-`felhom-bootstrap.sh` itself — currently at repo HEAD.
-
-**Proven live**, not just argued: the fresh box ran `felhom-host-install v1.22.0` and fetched golden
-**v0.188.0**, sha-verified.
-
-## Part 1 — bake / publish / vouch
-
-Baked **0.188.0**, not the brief's 0.187.0: 0.187.0 lacks D5, and Part 2 step 6 *is* the D5 claim, so
-that golden could not have passed the proof this task exists for. 0.188.0 satisfies R-120 anyway.
-Stated rather than absorbed, per standing rule 4.
-
-    GOLDEN_VERSION=0.188.0
-    GOLDEN_SHA256=7353d8beb63641f87a848e45f8aa12e465647e1190ad164a65b32ad01fc3d299
-
-Three observables: 404 pre-gate (with a 200 control on 0.186.0 so it is not vacuous), then an
-**anonymous** download returning `http=200 bytes=649310288` and a matching sha; the manifest read back
-showing `0.188.0` selected; and the consumer call `GET https://hub.felhom.eu/api/v1/artifacts/sess-g`
-returning the pair. Plus a fourth: a real fresh box fetched and sha-verified it.
-
-**R-120's gate evaluated and allowed.** Exercised both ways rather than inferred from silence —
-vouching 0.185.1 first produced `flash=golden_behind_fleet`, the logged `artifact vouch REFUSED`, and
-**no write** (the manifest still read 0.186.0); then 0.188.0 produced
-`Artifact manifest set: agent=0.118.1 golden=0.188.0`.
-
-## Part 2 — the clean-install proof, on demo-hp
-
-All seven steps PASS. Venue was demo-hp (Tier 0, the designated drill host) using the scratch dir
-storage at `/mnt/nvme-1tb` that `target-selection.md:38-40` names; `local-lvm`, `drill-r50` and both
-9201s untouched.
-
-Highlights: real day-0 pairing → bind → install; a **real** claim (the code is emailed-only, R-119 —
-the operator relayed it), with the gate flipping `dashboard not yet claimed` → `authentication
-required`; controller **0.188.0** confirmed *from the box*; **rallly** (postgres) + **homebox**
-deployed through the real endpoints.
-
-The D5 leg: recovery unit `portable-carried=2/2, withheld=0`; the carried `DB_PASSWORD` matched the
-live one **by fingerprint** (`14c8f515…`, never printed); guest `app.yaml` moved aside; restore
-returned `secrets recovered=2/2`. Step 7 read the data from **rallly's own network namespace** over
-TCP to `rallly-postgres` — not the localhost trust socket that produced D5's false pass — and the same
-path with a wrong password returned `FATAL: password authentication failed`, proving the credential
-does real work. **The discriminator held: PRE-BACKUP row = 1, POST-BACKUP row = 0.**
-
-## Part 3 — runbook integrity
-
-`RUNBOOK-manual-build.md` told the reader to use a "RECORDED" qemu line that is itself labelled
-*reconstructed*, and whose source says it *"was never saved"*. The real invocation is now captured
-from this bake and recorded as canonical in **§4.0**, alongside the bake/publish/teardown steps, the
-template-rot warning and where the R-120 gate actually lives. The old runbook's deviation entry is
-marked SUPERSEDED with a forward pointer.
-
-## Teardown — three layers
-
-1. VM 310 destroyed with `--purge --destroy-unreferenced-disks 1`; `/mnt/nvme-1tb/images/` empty.
-2. `cc-scratch` removed, `storage.cfg` back to its original four entries; `felhom-backup` available
-   **926 492 284 KiB before and after** — space returned exactly.
-3. Hub: **`sess-g` and its host record DELETED, full cascade** — `customer DELETE cascade COMPLETE
-   for sess-g (journal #8) — full teardown`, residue purged including `appliance_registrations=1`.
-   Verified positively: `/configs` and `/hosts` both loaded (10160 / 9880 bytes) with **0** hits for
-   `sess-g` and 0 for the appliance UUID. The gate refused twice first (409 host ONLINE, then 400
-   missing acknowledgements) — the record cannot be deleted until the destroyed box ages out of
-   ONLINE, ~30 min. **`sess-f` deliberately NOT deleted** (R-131); its command is in the audit §7.1.
-   Secrets shredded in the guest and on the box.
-
-## Findings — filed, none fixed
-
-`R-128` ISO_VERSION/SCRIPT_VERSION comment is false · `R-129` demo-hp's "no baked SSH key" is stale
-(key auth works) · `R-130` `HARD_MIN_LVM_GIB` warns and proceeds — a hard min that is not hard ·
-`R-131` `sess-f` is a fourth orphaned scratch customer · **`R-132` — `curl -w '%{redirect_url}'`
-printed the hub operator password into a session transcript; `HUB_PW` needs rotating.**
-
-## Not done, deliberately
-
-No ISO built; no defect fixed; no golden deployed to an existing box; no floor or MinAgent change;
-offsite/PBS-DR legs not exercised (the task forbids pointing anything at production PBS or the real
-restic offsite, so `sess-g` ran DR-tier off); Campaign 10, the demo-hp repartition and subdomain
-onboarding untouched.

REPORT-universal-iso-spike.md

@@ -1,97 +0,0 @@
-# REPORT — SPIKE 4: can a `.deb` in the ISO deliver the stub on an interactive install? (2026-07-31)
-
-> Written as `REPORT-universal-iso-spike.md`, not `REPORT.md`: the shared file belongs to today's hub
-> v0.85.0 session and the second session in a shared clone never touches it. Supersedes this file's
-> Spike 1–3 contents.
-
-**Class: Spike.** Findings only — no production file changed, no release ISO built, nothing published.
-Evidence: `documentation/audits/SPIKE-universal-iso-4-2026-07-31.md`.
-
-## The answer is yes, and it was measured with the negative control in the same box
-
-One ISO, 15 GRUB entries, a trivial probe `.deb` injected into `/proxmox/packages/`. Two VMs on
-demo-hp built with `qm` so the run was visible in the web console: **400 interactive**, **401
-automated control**.
-
-On the **interactive** install (`spikefour.felhom.eu`):
-
-- the package is installed — `ii felhom-spike4-probe 0.0.1`
-- its **postinst ran** — marker file present, content intact
-- it **enabled a systemd unit**, and **that unit fired on first boot** (uptime 7.98 s, `pid1: systemd`)
-- and on **that same machine**, `proxmox-first-boot` is not installed and `/var/lib/proxmox-first-boot`
-  does not exist — Spike 3's negative reproduced, not assumed
-
-So the two delivery mechanisms are independent, and the one that survives the path we are actually
-shipping is the `.deb`. **The product — insert the stick, install Proxmox normally choosing your own
-disk and password, box sets itself up and waits for a claim code — is now measured rather than hoped
-for.** With one honest caveat: what was measured is a trivial probe package, not Felhom's real stub.
-Packaging the real stub and confirming pairing end-to-end is the last step before a spec (~60 min).
-
-## What a postinst may and may not do
-
-Identical on both paths: `pid1 = unconfigured.sh`, **no running systemd**, `/proc` and `/sys` mounted,
-and **`systemctl enable` succeeds** (it wrote the symlink). Network and DNS *happened* to be up —
-inherited from the installer's own DHCP.
-
-Four constraints for the real postinst, so they get written against rather than discovered:
-
-1. Never `systemctl start` or `daemon-reload` — there is no systemd running. `enable` is the only verb.
-2. **Never require the network**, despite it being present here. A box installed with the cable out
-   gives a postinst no route, and a failing postinst breaks the customer's install.
-3. Never fail — guard everything, `exit 0`.
-4. Do the real work in the unit at first boot, where systemd, network and a booted kernel exist.
-
-## Two smaller results
-
-**The repack preserves the `.deb`, but not naively.** `xorriso … -boot_image any replay` fails with
-*"Overlapping MBR partition entries"* — and `iso-repack.sh:270-292` already documents that exact
-failure and its fix. Mirroring it produced a working image (19 El Torito entries; the `.deb` extracted
-back out is byte-identical). So this is an insertion into an extract→modify→re-master cycle our repack
-already performs, not a new build stage.
-
-**Q3:** `iso-repack.sh:100-106` refuses an ISO without `auto-installer-mode.toml`. It is a guard, not
-a structural requirement, and its reasoning is sound for the shape it was written for — already R-155,
-cited exactly here. With no mode file the stock grub.cfg does not emit the Automated entry at all; with
-a mode file but no answer, that entry aborts safely and loudly.
-
-## A correction I owe you from last session
-
-**R-153 is retracted.** The register grep this task mandated shows R-94 already carries it verbatim at
-`OPEN-ITEMS.md:15`, status `READY (XS)`, with leg (b) being precisely "the gate fails today and is
-invoked by nothing" — and R-29, the class, says in terms *"do not mint a new ID for a new instance."*
-Spike 3 filed a duplicate.
-
-**And the substantive half of that Spike 3 claim was wrong.** I wrote that the drift left the customer
-page's install-command generator "targeting a flag surface three minor versions stale." R-94 explicitly
-retracts exactly that reading: the constant selects no script — it renders as a text label, and the
-command beneath it fetches the script the website git-syncs from `main`, so **1.22.0 is what every
-install already gets**. It is a wrong number on your screen and nothing more. I overstated it.
-
-## Still unknown
-
-**The real stub has not been packaged** — that is the one thing between here and a build spec.
-Also unproven: `dpkg --configure -a` ordering for a package with dependencies; an ISO that never went
-through `prepare-iso` (blocked by R-155, which this spike was fenced from changing); and the Graphical
-installer, where the result should hold *a fortiori* since the `.deb` path is in `Install.pm`, shared
-by all front-ends — but that is inference, not measurement.
-
-**Spike 3's Q3** — the real stub at `before-network` — **this session did not touch it.** Note it is
-now partly superseded: on the `.deb` route the unit's ordering comes from the unit file, not from
-`[first-boot].ordering`.
-
-## R-rows
-
-**None opened.** Each candidate was grepped against the register first: the delivery result is a
-positive finding, the postinst constraints belong in the build spec, and the repack guard is already
-R-155. **R-153 retracted** into R-94 leg (b) / R-29.
-
-## Teardown
-
-All three layers plus the scratch storage, verified positively. demo-hp: VMs 400/401 purged, **storage
-`spike4` removed** (`storage.cfg` back to 4, `grep -c spike4` = 0), `/mnt/nvme-1tb/images/` empty,
-**disk usage 6.6 G — identical to pre-spike**, probe ISO and driver removed, 0 loop devices,
-`drill-r50` stopped and untouched, 9201 running, `felhom-backup` unmodified, nothing on `local-lvm`.
-DooPlex: workspace scratch **4.8 GB removed**, scratchpad **3.3 GB → 88 K**, both throwaway passwords
-destroyed, 17 ISOs in `out/` untouched, no production file modified. **Hub-side: nothing created** —
-the VMs took LAN DHCP leases but never ran `felhom-host-install.sh` or contacted the hub; verified by
-fetching and searching the customer list. Nothing published.

REPORT.md

@@ -1,216 +1,124 @@
-# REPORT — R-181 (the reserve guards the write that fills the disk) + R-156 (papra) + two operator rulings
+# REPORT — R-201: **THE DRILL PASSED.** A customer's file survived a machine rebuild
 
-**Date:** 2026-08-03 · **Repos:** `felhom-controller` (v0.192.0 → **v0.193.1**), `app-catalog-felhom.eu`, `felhom.eu` (docs only — **no hub change, no hub version bump**)
+**2026-08-04, 21:30–23:15** · `demo-hp` deliberately rebuilt · no code, no version bump ·
+`demo-felhom` untouched. Record: `documentation/audits/DRILL-r201-night-run-2026-08-04.md`.
 
-## 1. Baselines — re-read on arrival, all matched §1
+---
 
-| Repo | `main` @ arrival | Version | Shipped |
-|---|---|---|---|
-| `felhom-controller` | `4be6467b501b` | v0.192.0 | **v0.193.0 `fef07c3`** → **v0.193.1 `6c43bf6`** |
-| `app-catalog-felhom.eu` | `7cb58ecdf8e7` | n/a | `122bbee` |
-| `felhom.eu` | `6b5d64c1fa73` | hub v0.89.0 | docs only, **no bump** |
-
-All three clean (`git status --porcelain` empty, `HEAD == origin/main`) before every build.
-
-## 2. The fix
-
-**One admission verdict per app per run** (`controller/internal/backup/admission.go`), taken before
-that app's **first** write and consulted by all three legs — DB dump, volume dump, unit capture. The
-three write under one per-app root (`appbackup.RecoveryUnitPath`), which is what makes one verdict
-able to cover them honestly.
-
-- **Lazy, not run-wide.** App A's dump can put app B under the reserve; a run-start verdict reads a
-  disk that no longer exists. **Never re-decided between an app's own legs** — that is the split being
-  closed. **Reset per run.**
-- **Ahead of `DumpAppVolumesSafe`**, which stops the stack as its first act, so a refused app is never
-  bounced. **After** the volume-less check, which has no write to gate.
-- **Exactly one operator alert per refused app per run.** Leg order unchanged.
-- **Size term added:** *would this app's write cross the reserve?* — estimated from its previous
-  `.sql` + `.tar`. **No history → headroom-only**, or the first backup becomes the one that can never
-  happen; the alert says so when that applies.
-
-## 3. Files
-
-| File | |
-|---|---|
-| `controller/internal/backup/admission.go` | **new** — the gate, the memo, the estimator |
-| `controller/internal/backup/admission_test.go` | **new** — 11 tests |
-| `controller/internal/backup/backup.go` | run scope + gates in the DB and volume legs |
-| `controller/internal/backup/recovery_unit.go` | `floorVerdict` size-aware; capture leg via `admitApp` |
-| `controller/internal/backup/capture_floor_test.go` | 3 call sites updated for the new signature |
-| `controller/README.md`, `REUSE.md`, `CHANGELOG.md` | |
-| `app-catalog-felhom.eu/templates/papra/docker-compose.yml` | mount moved to `/app/app-data` |
-
-## 4. Tests — 28 packages `ok`, `rc=0` (read separately from any commit)
-
-All 11 new tests pass, plus the pre-existing floor suite. Refusal assertions are **sha256 tree
-fingerprints before and after**, never log lines — the defect being fixed *is* a log line the tree
-contradicted.
-
-The DB leg cannot run without Docker (`DiscoverDatabases` shells out), so its gate is pinned by an
-**AST walk** of `backup.go` asserting `admitApp` precedes `DumpOne`. `strings.Contains` is
-insufficient: a commented-out call still contains the string.
-
-### Red-proofs — each demonstrated failing, then restored
-
-| # | Mutation | Result |
-|---|---|---|
-| 1 | **Both** dump-leg `admitApp` gates removed (= exactly v0.192.0) | Scenario A **RED** — *"the VOLUME leg ran for a refused app"*; with the leg assertions temporarily made non-fatal, the **tree fingerprint changed** too. Also red: Scenario C, Scenario D, and the AST wiring test (which named the DB leg specifically) |
-| 2 | The entire size term removed from `floorVerdict` (both its thresholds) | Scenario D **RED** — 0 alerts where 1 was required |
-| 3a | The reserve removed entirely | Scenario F **PASSED — recorded honestly.** The specified mutation does not exercise the assertion: removing the reserve makes every app write, which overwrites and adds but **deletes nothing**, so a deletion-watching test correctly stays green |
-| 3b | A prune injected into the refusal path | Scenario F **RED** — this is the mutation that proves the test watches deletion |
-| 4 | Floor moved above the warning band (90% / 6 GiB) | `TestFloorSitsBelowTheCriticalWarningBand` **RED** |
-
-Every mutation removed **every** guard its test covers (#1 removed both dump-leg gates, not one).
-
-## 5. Live validation — demo-hp guest 9201 (Tier 0), the method that found the defect
-
-**Method:** endpoint-level — `POST /api/debug/backup/dbdump`, the exact endpoint the debug UI button
-calls, which runs the production `RunDBDumps`. No browser on DooPlex.
-
-**The instrument was re-proven before use.** demo-hp's thin pool is 53.93 GiB, so a real fill of a
-70 G volume would exhaust it and corrupt every guest. A 5 GiB `fallocate` step moved guest `df`
-1.2G → 6.2G while thin-pool `data_percent` held **36.83 → 36.83** — zero blocks allocated. Re-checked
-at every step of the fill.
-
-### Headroom term — 08:59:46, 906 MB free / 99% used
-
-| Observable | Result |
-|---|---|
-| Tree fingerprint before | `TREE_SHA=111d1760c18d3440f700634ab325f8b8` (10 files; opengist's tar **182,272 B** — R-181's own "before" figure) |
-| Tree fingerprint after | **`111d1760c18d3440f700634ab325f8b8` — identical** |
-| Volume dumps written | **0** (baseline run at 08:58 wrote 2) |
-| `Stopping <app> for safe volume dump` | **absent** — and this is evidence, not an absence, because that line **is** present in the 08:58 baseline |
-| Operator alerts | one `recovery_unit_capture_failed` per app, severity `error`, HTTP 200 |
-
-Free space restored → re-run at **09:01:33**: both apps captured normally.
-
-### Size term — 09:03:00, proven separately
-
-Reproducing the original sequence: a real 2 GiB file planted in opengist's volume, backed up so its
-**previous** tar became **2,147,666,432 B** (the exact live figure), then the filesystem set to
-**91% used / 2.9 GB free — both headroom terms deliberately clear**.
-
-- **opengist refused `(size)`** — *"this app's last backup was 2.0 GB and writing it again would cross the reserve"*
-- **privatebin ADMITTED and dumped normally** — the term is per-app, not a global halt
-- Tree unchanged; 1 volume dump instead of 2
-
-### One honest correction to the "app not stopped" claim
-
-`StartedAt` on both apps *did* move, 26 s **after** the refusal. It was the **quiesce loop** for the
-whole-guest PBS backup, which my fill had broken — not the app-data path. Its own backoff logic then
-behaved correctly (*"deferring its next quiesce by 15m so the apps are not stopped again for a backup
-that cannot succeed"*). The app-data claim rests on the **absence of the `Stopping … for safe volume
-dump` line**, which is the line that appears when that leg bounces an app.
-
-## 6. The `du` measurement (§Part 1.3) — measured, then rejected
-
-**66 timed runs** on demo-hp guest 9201, `docker run --rm -v <vol>:/v alpine du -sb /v`:
-**median ~355 ms per volume, range 341–404 ms** — on volumes holding **tens of KB**. The cost is
-container start-up, not the walk, so it does not shrink for small apps and only grows for real ones.
-
-**Rejected**, on two grounds beyond the number: `docker run` needs the writable layer, so the
-measurement mechanism can fail under exactly the disk pressure the reserve exists to handle; and the
-previous-dump estimate measures the **artifact that will be written** rather than the live volume,
-which is the truer predictor. The previous-dump estimate stands.
-
-## 7. The refusal message as shipped, and what it guarantees
+## 1. THE VERDICT — PASS
 
 ```
-[WARN] [backup] App backup REFUSED for opengist (headroom) — refused: backing up this app would
-leave the filesystem below the reserve (reserve: 97% used or 1.0 GiB free; the filesystem is already
-below it, before this app's estimated 178.0 KB write) — /mnt/sys_drive: 64.3/68.7 GB used (94%),
-0.9 GB free; NO database dump, NO volume dump and NO recovery-unit capture was written for it, the
-previous unit is untouched and NOTHING was deleted
+pre-wipe   sha256: 643166269103a25cf41d34a26b75fd6ebba0a837bbcd7e0d2b5aba7106bcbe7c
+restored   sha256: 643166269103a25cf41d34a26b75fd6ebba0a837bbcd7e0d2b5aba7106bcbe7c
 ```
 
-**It guarantees, for that app in that run:** no DB dump, no volume dump and no capture were written;
-every file under `backups/primary/<app>` is byte-identical; the app was not stopped; nothing anywhere
-was deleted; exactly one operator alert was sent. All five verified by fingerprint above.
+**Byte-identical.** The controller's data volume was destroyed and the file deleted from disk; the
+customer's recovery code, the customer's restore flow, and the file came back. **The Felhom backup
+story is proved end to end for the first time.**
 
-**The wording was not weakened to fit the behaviour** — the behaviour moved so the wording became
-true. What was *added* is the bound term (`headroom` / `size`) and the estimate.
+## 2. Snapshot count at step 9 — **3, not 1**
 
-**v0.193.1 — found by this very proof run.** The estimate was rendered fixed to two-decimal GiB, so
-opengist's real **178 KB** printed as `estimated 0.00 GiB write`, which reads as *no estimate was
-available* — the opposite of what happened. Shipped the same session because it is the same defect
-class the whole task is about. Re-verified live after redeploy: `estimated 178.0 KB write`.
+`repo_state: null` (the repository **opened**, not orphaned), `snapshot_count: 3`, `repo_size_bytes:
+42 026` — the pre-wipe size exactly. The pre-wipe snapshot `e6132ae5` was still present with the file
+inside it. **No fresh history was started over the old one** — the failure this drill was designed to
+catch did not occur.
 
-## 8. papra (R-156, last leg)
+## 3. §5's five conditions, recorded before the wipe
 
-**Precondition checked, not inherited** — both boxes were wiped and rebuilt today, so the 2 August
-evidence was re-measured: `docker ps -a` (**including stopped**) on **both** demo guests → no papra;
-hub `/hosts` → exactly two enrolled hosts (`demo-felhom-8363b5`, `demo-hp-bb76ea`), **zero** papra.
+1. sentinel **listed by name** in snapshot `e6132ae5` (19:36:26), 181 B.
+2. rollback archive `vzdump-lxc-9201-2026_08_04-21_44_24.tar.zst` (1 606 765 083 B) **verified by a
+   full zstd stream read** (4 867 573 760 B) with the sentinel confirmed inside.
+3. §3's option — §4 below.
+4. `demo-felhom` `health=ok`, `escrow_state=escrowed`, untouched throughout.
+5. nvme-1tb 883 GB free, root 20 GB, local-lvm 34.32 %.
 
-**Decided from the image, not the README:** `WORKDIR=/app`, `DATABASE_URL=file:./app-data/db/db.sqlite`,
-`DOCUMENT_STORAGE_FILESYSTEM_ROOT=./app-data/documents`, `PAPRA_CONFIG_DIR=./app-data` — and
-**`/app/data` does not exist in the image at all**.
+**One precondition had drifted and was repaired, not worked around:** the staged snapshot had lost the
+sentinel because the afternoon's experiment produced a *later* same-day snapshot and `forget
+--keep-daily 7 --group-by host,tags` pruned the good one. One backup re-established it, re-verified by
+listing. **A good snapshot is not durable against a later bad run on the same day.**
 
-**Departure from the task's stated preference order, stated because it was deliberate.** Option (1)
-(reconfigure the app to write to `/app/data`) *was* available — all three paths are env-settable. Not
-taken: it enumerates data paths, so a fourth added upstream would silently escape to the writable
-layer again — this defect re-armed and invisible. Mounting the app's own data **root** captures every
-current and future path by construction.
+## 4. §3 — the recovery code
 
-**Gate output — the arbiter, run in both directions:**
+**Option B as already in place, improved: no new copy was made, so nothing needed shredding.** The
+operator's own `~/.config/credentials` (`0600`) was the source; the code was **piped to stdin** for
+each invocation — never an argument, never exported, never a second file, never logged. Destroying the
+operator's own store would have destroyed their record; because no extra copy existed, there is nothing
+left to prove gone.
 
-- fixed → `papra CLEAN`, with the self-test passing on that run: *"prober flags the R-156 signature and clears a correct template — trustworthy"*
-- reverted to `/app/data` (red-proof on the **real template**, not just the canary) → `BROKEN`: *"mount /app/data is NOT writable by the app's own uid=999"*, *"DATA in the writable layer at /app/app-data/db (db_signature=True, e.g. ['db.sqlite'])"*, *"declared volume /app/data is EMPTY"*
-- `catalog_gates.py papra` (full, not `--fast`) → **rc=0**, all three gates OK
+**Verified anyway, with the planted-copy positive control:** 0 hits in the agent journal, 0 in the
+controller log, 0 files under `/tmp`, `/var/tmp`, `/var/lib/felhom-agent`, `/root`, 0 leftover
+`felhom-idesc-*` dirs — and the same sweep found a planted copy (**1**), then **0** after shredding it.
 
-**Two operational findings about the gate:** it needs **root** (it reads `/var/lib/docker/volumes`,
-mode `drwx--x---`; as a normal user its own canary fails UNDETERMINED and it correctly refuses a
-verdict — fail-closed working as designed), and it hardcodes scratch path `/srv/felhom-gate`, created
-on DooPlex. Unscoped it deploys all 53 templates; that run was aborted after 10 minutes and its
-`volgate-*` scratch projects were cleaned up.
+## 5. Every step's observable
 
-## 9. §3's correction — confirmed in passing, not chased
-
-`restore_points.go:57-59` takes the manifest's mtime and then `newestArtifact` over the `.sql` and
-`.tar` files, so **the newest of the three wins**. The restore point does **not** show a stale
-timestamp. Confirmed and dropped, as instructed.
-
-## 10. Register
-
-| ID | Change |
+| step | observable |
 |---|---|
-| **R-181** | **CLOSED — SHIPPED** (v0.193.0 + v0.193.1), with the live evidence above |
-| **R-156** | **CLOSED** — all three apps fixed |
-| **R-110** | WAITING-ON-OPERATOR → **READY**, ruling attached: **option (b), tag-tracked**, and it must cover **both** channels (the `/scripts/` git-sync *and* the nine files fetched from `raw/branch/main`) or it only half-works |
-| **R-115** | WAITING-ON-OPERATOR → **READY**, ruling attached: **mechanism (b)**, a build-side gate refusing to deploy or vouch an unpublished version; the third instance (agent v0.120.0) would have silently downgraded both demo boxes while reporting success |
-| **R-182** | **NEW.** ID established free: `grep -ro "R-182\b"` over `documentation/` and `*.md` → 2 hits, both prose in `REPORT.md` recording it as *"checked and left unused"*; `R-183` → 0 hits and remains free |
+| 6 | fresh data dir `20:00:2x`; new `encryption.key`; `claimed = None`; `offbox = null` |
+| **7** | **`identity_blob` 572 B, `restic_pw_sha256` `8a9e33aa4da6…`, `updated_at` still `11:11:37` — UNCHANGED.** Nothing re-escrowed itself |
+| **8** | **recovered `8a9e33aa4da6…`** — matches the pre-wipe on-disk key and the hub's record |
+| 9a | `[INSTALLED] … reads back identical` |
+| 9b | after Re-issue + apply, the on-disk key is **still** the recovered one |
+| 9c | **repository OPENED** — `repo_state: null`, 3 snapshots, 42 026 B |
+| **10** | **restored sha256 byte-identical** |
+| 11 | this record |
 
-**R-165** is collapsed to CLOSED/PROVEN-LIVE in `ROADMAP.md`; the capability map's local-backup row
-moves to **PROVEN-LIVE, both halves**, because the live fill proved the fixed behaviour for **both**
-reserve terms.
+Superseded escrow rows remained **2** throughout — **no ceremony was run at any point.**
 
-## 11. Observations — noticed, documented, NOT acted on
+## 6. Part 2 — not run
 
-1. **R-182 (filed).** The periodic status refresh (`GetFullStatus` → `captureAllRecoveryUnits`) runs
-   with no admission scope, so a refused app re-alerts on every poll — measured live: a second
-   identical alert pair 13 s after the run's. **Pre-existing in v0.192.0**; R-181 changed neither
-   caller. Its mitigation is a *comment* claiming the hub owns cooldown — which is exactly the
-   "invariant asserted in a comment with no test pinning it" shape, so verify at the hub before
-   scoping.
-2. **A reserve refusal does not make the run fail.** The DB and volume legs record `SKIP`, not `FAIL`,
-   so `lastDBDump.Success` stays true and the customer-facing status does not turn red. Deliberate and
-   consistent with v0.192.0 (the capture refusal never set it either), and the operator alert is the
-   signal — but it means "backup succeeded" and "every app was backed up" are not the same statement.
-3. **`UnitSpace.UsedPercent` and `df` disagree** — `df` reported 99% where the alert said 94%, because
-   `df`'s figure accounts for ext4 reserved blocks and the floor's does not. Harmless here (the
-   free-byte term bound), but a percent-term threshold is being compared against a number the operator
-   cannot reproduce with `df`.
-4. **The whole-guest PBS backup fails when the volume is near-full**, pushing
-   `whole_guest_backup_failed` (severity `error`). Expected under a deliberate fill, and its backoff
-   behaved correctly; noted because it is collateral any future fill test will also produce.
+Gate: *"the drill PASSED; its evidence is written down; and there is time."* The first two are met; the
+third is not — 23:15, after a full destructive cycle, and Part 2 is a second wipe. Deliberately left
+for a fresh session with drill 1's result already recorded, so a second wipe cannot overwrite it.
+**R-198's retention remains unit-proven only** — nothing has yet superseded a key in production, and it
+is now the last unproven link in this chain.
 
-## 12. Teardown
+## 7. Teardown — three layers
 
-Fill file removed; the planted 2 GiB file removed; a final backup regenerated a correct 178 KB tar;
-`pct fstrim 9201` returned 67.5 GiB and the thin pool settled at **29.43%**, *below* its 36.83%
-baseline. The backups tree is byte-identical to the pre-test fingerprint. Guest helper scripts and the
-credential file `shred`-ed. `volgate-*` scratch compose projects removed; the unrelated 9-day-old
-`jarr-*` containers on DooPlex were left untouched. papra is **not** left deployed.
+| layer | state |
+|---|---|
+| the guest | **healthy and re-armed**: controller v0.197.0 claimed, six app containers serving, all three apps re-toggled for off-site, the sentinel restored to its live location and verified. The verification-copy tree is **kept as evidence**. Scratch band empty |
+| the host | vzdump snapshot LVs released cleanly; one new 1.6 GB archive on nvme-1tb (883 GB free); nothing deleted |
+| the hub | **no new customer records** — `demo-hp` is the same row throughout (a controller-data rebuild, not a re-enrolment). One Re-issue staged a one-time secret (consumed 20:16) and set `stale_at`. **The two superseded escrow rows are unchanged** |
 
-No `--no-verify` was used on any push; the `felhom-controller` pre-push hook ran and reported
-`gates OK` on both pushes.
+**Nothing deleted on the storage endpoint**, including the ~1.2 GB of orphaned ciphertext — ruled,
+still owed, deliberately not ridden along with a drill.
+
+## 8. The capability-map row
+
+Now reads *"A customer's file survives a machine rebuild and comes back — the whole off-site story, end
+to end"*, **PROVEN-LIVE**. What it deliberately does **not** claim: the journey is manual and
+undocumented (R-204); the scope is `demo-hp` and a **controller-data** rebuild, **not** a total host
+loss and **not** a guest reprovision; and R-198's retention is still unit-proven.
+
+## 9. New findings — R-204, expanded into the gap list
+
+The four steps between a recovered key and a restored file, all measured while walking them:
+
+1. **R-193** — a rebuilt controller cannot configure its off-site tier (`no unconsumed offsite
+   password`; the ledger shows its predecessor consumed it at 07:12:06). Needs an operator Re-issue.
+2. **The claim gate** — a rebuilt box is unclaimed, so *every* controller endpoint is intercepted. And
+   **the local escape hatch does not work unaided**: `--print-reset-code` writes the new hash to
+   `settings.json` while the running controller keeps its old copy in memory, so `effectiveClaimCode()`
+   never sees it. **Restart the controller between minting and claiming** — two claim attempts failed
+   before this was diagnosed.
+3. **R-196** — the Re-issue sets `stale_at` (`20:15:49`) while `restic_pw_sha256` is unchanged, which
+   gates every off-site run. Cleared with the **manual** confirm, never a ceremony.
+4. **`mode=unit` is the restore default** and returns the recovery unit, **not** the userdata leg. A
+   customer told to "restore from off-site" gets their app definition and not their documents, and
+   nothing in the outcome says so. **The worst of the four**, because it fails silently at the last
+   step.
+
+## 10. CI
+
+Docs push only; run number and task id in the session summary. **`--no-verify` not used.**
+
+## 11. Observations — noticed, NOT acted on
+
+1. **A same-day re-run replaces the day's snapshot** (`forget --keep-daily 7 --group-by host,tags`).
+   Any drill depending on a specific snapshot surviving must account for it.
+2. **The hub's ClusterIP is not reachable from DooPlex's host network** — the operator UI needs a
+   `kubectl port-forward`; the memory's `curl -u :$HUB_PW` recipe omits that.
+3. **`--print-reset-code` mints a new generation on every invocation** — three wasted generations here
+   (4, 5, 6) while diagnosing the in-memory staleness.
+4. **The restore wrote into `backups/offsite-restore/<app>/` mirroring the full absolute path** — deep
+   but unambiguous; worth knowing before writing customer-facing copy about where files land.

REUSE.md

@@ -75,8 +75,11 @@
 | `(*Store).GetHostRecoveryMeta` + `(*Server).handleHostRevealRecoveryCredential` | hub/internal/store/host_recovery.go · hub/internal/web/hosts.go | `(hostID) (*HostRecoveryMeta, error)` · `POST /hosts/{id}/reveal-recovery-credential` | The break-glass console credential, split into a RENDER half and a RETRIEVE half (v0.84.0) | **Use `GetHostRecoveryMeta` on any page-render path** — its struct and its `SELECT` both omit the `secret` column, so it cannot leak one; `GetHostRecoveryCredential` (which does select it) belongs only to the two retrieval handlers. The reveal is POST so the ServeHTTP-level CSRF check applies and no secret is reachable by URL; it writes ONE `recovery_credential_revealed` event via `SaveEvent` and calls NO dispatcher (the `handleRequestLogTail` shape). `api/handler.go handleAdminGetRecoveryCredential` (global key) is the independent fallback for when the UI is down — never route the UI through it. Secret at rest is plaintext → R-133. |
 | `host_detail_body` sub-template | hub/internal/web/templates/host_detail_body.html | `{{template "host_detail_body" .}}` | Rendering a host's detail sections on ANY surface | One namespace across ParseFS (icons.html pattern). Renders per-host — id-suffix any new element ids with `{{.HostID}}` (the customer page renders N instances). |
 | `(*Store).ListHostsByCustomer` | hub/internal/store/store.go (~L1620) | `(customerID) ([]Host, error)` | A customer's hosts, host_id order | A LIST by design (HA-cluster roadmap) — don't collapse to GetHostByCustomer. |
+| `(*Store).HasEverBoundHost` (v0.92.0, R-195) | hub/internal/store/store.go | `(customerID) (bool, error)` | Any verdict that must not fire for a customer with **no machine ever bound** — "was anything ever expected of this customer" | `hosts` row **OR** `host_deletions` tombstone. **NOT a liveness check and never a substitute for one:** a box that was bound and went silent returns `true` and must keep alarming — that is the case any change here breaks first (pinned by `TestCheckBackupDeadlines_BoundButNeverReported_StillAlarms`). Callers **fail OPEN** on its error: an unreadable binding must never SUPPRESS an alarm. Do **not** re-derive this from report presence — `store.GetCustomers()` (and therefore the staleness checker's `down` state) is a query over `reports`, so a never-reported customer has no state at all, which is exactly how the daily false alarm reached `david`. |
 | `(*Server).configFormData` (v0.49.0) | hub/internal/web/configs.go (~L430) | `(r, isNew, cfg, overrides, errMsg) configFormView` | The ONE view-model builder for the customer config form (standalone chrome + the customer page Edit tab) | `overrides=nil` → parses the STORED cfg.ConfigJSON; pass the SUBMITTED map on the update validation-error re-render or typed values reset (red-proofed). |
 | `config_form_body` sub-template (v0.49.0) | hub/internal/web/templates/config_form_body.html | `{{template "config_form_body" <configFormView>}}` | Rendering the config form on ANY surface (config_form.html chrome + customer Edit tab) | The floor/geo/danger cards on the Edit tab are SIBLINGS after `</form>` — never nest a form inside it (breaks the offsite/PBS formaction sub-buttons). Includes the F5 in-flight `<script>`. |
+| `handleHostEscrowGet` + `escrowSelfServiceRetrieval` | hub/internal/api/handler.go | `GET /hosts/{id}/escrow` | The box-authenticated MIRROR of the escrow PUT — self-scoped by the per-host key. **`escrowSelfServiceRetrieval` is THE single decision point** for whether a box may read its own blob without operator-armed recovery mode (§8.2 vs §8.3, R-199): flip it, do not scatter the condition. **Never merge this with `dr.go`'s `handleReEnroll`/`handleGetRestoreDirective`** — those rotate the host API key and serve the K-escrow and directive too, and keep their recovery-mode gate (pinned by `TestEscrowGet_OperatorDRPathUnchanged`). Every successful retrieval MUST raise `escrow_blob_served` before the bytes leave; that audit row is the mitigation the trade rests on. |
+| `demoteCurrentEscrowTx` (+ `(*Store).SaveHostEscrow`) | hub/internal/store/store.go (~L2597/~L2612) | `(tx, hostID) (int64, error)` / `(hostID, blob, fp, posture, createdAt, resticPwSHA) (superseded bool, prevResticPwSHA string, err error)` | **THE ONE escrow row-copy routine** — used by the re-escrow retention AND by `DeleteHost`'s custody demotion; never write a second one | **It must copy BOTH sealed artifacts** — `blob` (K-escrow / PBS key) and `identity_blob` (the age bundle carrying the offsite restic repo password). Omitting the second is R-198: two months of retaining the wrong key, with the ceremony as the destroying act. **Ordering it depends on:** `SaveHostDRBundle` writes `identity_blob` AFTER `SaveHostEscrow` returns, so the demote sees the PREVIOUS generation — invert that and the retained bytes are the new blob under the old hash. Pinned by `TestSaveHostEscrow_RetainsIdentityBlob` + `TestDeleteHost_DemotesIdentityBlob` (both callers). `prevResticPwSHA` feeds R-197's changed-key signal; it is a hash and never leaves the store. |
 | `(*Store).CountHostArtifacts` / `DeleteHost` | hub/internal/store/store.go (~L1640/~L1690) | `(hostID) (HostArtifacts, error)` / `(hostID, deleteEscrow bool) error` | Host-delete impact preview + the ONE-transaction cascade | ONLINE gate lives in the handler, escrow gate in the store (`ErrHostEscrowPresent`, tx never starts). log_bundles die by `scope_id == host_id` ONLY (customer-scoped bundles survive). The wg_peers delete is INSIDE the tx — never split it out. |
 | `(*Server).commitCustomerReset` (v0.69.0) | hub/internal/web/customer_reset.go (~L165) | `(ctx, cfg, resetID int64, purgeEscrow bool) *resetLegError` | THE committed RESET sequence — external teardown FIRST (Hetzner, PBS), then claim → descriptor → DB purge, each leg stamped into the `customer_resets` journal | Owns NO gate, NO audit event, NO journal open/close, NO redirect — those are the caller's (the two callers differ there). `purgeEscrow` governs ONLY whether `PurgeCustomerResetDBState` destroys retained custody: standalone RESET passes the operator's `escrow_ack`; the DELETE cascade passes **false** so custody dies exactly once, in its leg 3. Returns a `resetLegError` carrying the leg name + the exact status/message the standalone handler has always returned — do not re-word them. |
 | `(*Store).CustomerResidue` / `PurgeCustomerResidue` (v0.70.0) | hub/internal/store/customer_delete.go | `(customerID) (*CustomerResidue, error)` / `(customerID) error` | Counting + purging the report-derived state and the credential-bearing bindings a deleted customer leaves behind | **`GetCustomers()` is REPORT-derived** — until the reports are gone the customer stays on the Customers list AND stays in the staleness/offsite checkers’ work list, so a deleted customer keeps emailing the operator. Both funcs walk ONE shared `residueQueries` list so a table can never be counted-but-not-purged. Includes `appliance_registrations` + `selfbind_tokens` (credential-bearing, not telemetry). NEVER touches `events`, `notification_log`, `host_deletions`, `customer_resets`. |
@@ -104,7 +107,7 @@
 | `offsite.DeliveryStateFor` (+ `DeliveryStatus`) | hub/internal/offsite/delivery.go | `(st, customerID) (DeliveryStatus, error)` | THE R-70 offsite last-mile detector — one implementation for every consumer (customer card `deliveryViewFor`, `monitor.OffsiteDeliveryChecker` event + R-71c heal) | Precedence: `applied` (latest report has offsite) wins over every secret-row shape; applied+unconsumed-staged = applied + `StaleStagedSince` flag (demo-felhom's live specimen). Never add a sibling derivation — consumers read THIS. |
 | `(*Store).GetOneTimeSecretInfo` / `LastEventAt` / `LatestReportOffsitePresence` / `CountReportsOffsiteSince` | hub/internal/store/store.go | `(customerID) (*OneTimeSecretInfo, error)` / `(customerID, eventType) (time.Time, error)` / … | Detector inputs + DURABLE event-cooldown source (events table survives restarts — prefer over in-memory maps for hub-emitted checker events) | `GetOneTimeSecretInfo` never selects the value column — keep it that way. `SetOneTimeSecretTimesForTest` is the back-dating seam (PBSDR pattern). |
 | `monitor.OffsiteDeliveryChecker` + `OffsiteReissuer` | hub/internal/monitor/offsite_delivery.go | `NewOffsiteDeliveryChecker(st, reissuer, onEvent, logger)` | R-70 stuck event + R-71c self-heal on the shared 60 s ticker | THE R-39(a) GUARD lives in `maybeHeal`: re-reads the secret row at act time and refuses over an UNCONSUMED row — `SaveOneTimeSecret` clobbers by design (Re-issue depends on supersede); never "fix" the store, never bypass the guard. reissuer nil = heal disabled (no provisioner) — required, else a heal-event fires for a silent no-op. |
-| `monitor.RestoreTestChecker` + `assessRestoreProven` | hub/internal/monitor/restoretest.go | `NewRestoreTestChecker(st, onEvent, logger)`; `.Check()` | R-85: turns a restore-test result into a SIGNAL — it was a `[WARN]` log line and nothing else, even for the tier already being tested | **TWO event types, never merged**: `restore_test_failed` (broken now, error) vs `restore_test_stale` (unverified — *not* known-broken, warning). Merging collapses the second into the first, and the second is what quietly becomes the first. **Anchored on R-81** (`assessRestoreProven` reuses `backupAssessment`/`verdict*`): a never-proven tier on a newborn box is UNKNOWN, not FAILED. Per-tier proof comes from the hub's RETAINED WINDOW — the agent reports only its latest run, so the latest report alone cannot answer "when was the OTHER tier last proven?". Operator-tier only: **no `customerMessages` entry** — do not add one without copy review. |
+| `monitor.RestoreTestChecker` + `assessRestoreProven` | hub/internal/monitor/restoretest.go | `NewRestoreTestChecker(st, onEvent, logger)`; `.Check()` | R-85: turns a restore-test result into a SIGNAL — it was a `[WARN]` log line and nothing else, even for the tier already being tested | **TWO event types, never merged**: `restore_test_failed` (broken now, error) vs `restore_test_stale` (unverified — *not* known-broken, warning). Merging collapses the second into the first, and the second is what quietly becomes the first. **Anchored on R-81** (`assessRestoreProven` reuses `backupAssessment`/`verdict*`): a never-proven tier on a newborn box is UNKNOWN, not FAILED. Per-tier proof comes from the hub's RETAINED WINDOW — the agent reports only its latest run, so the latest report alone cannot answer "when was the OTHER tier last proven?". Operator-tier only: **no `customerMessages` entry** — do not add one without copy review. **R-86 (2026-08-03): the window is PER TIER, not one constant.** `restoreProvenWindow(tier, observed, ok)` = `clamp(4 × max(observed, declared), floor 7d, cap 12d)`, where `declared` is that tier's own backup-freshness threshold (`backupStaleAfter` 26 h / `offsiteBackupStaleAfter` 8 d — reuse those, never a second opinion) and `observed` comes from `observedArchiveIntervals` over the retained window. **Observation may only WIDEN**: a gap shorter than the declared rhythm is routine (a retry, a heal, a catch-up) and a live box proved it — demo-felhom's two PBS snapshots sit 8 h 54 m apart, which would read a WEEKLY tier as nine-hourly and re-create the false alarm. The cap keeps the window strictly inside offsite retention. `assessRestoreProven` takes the window as an argument and **every reason string names it** (R-100's corollary). |
 | `(*Server).applyPBSDR` + `mergePBSDR`/`readPBSDR` | hub/internal/web/pbsdr.go | `(ctx, r, cfg) error` | The config form's DR-tier section → HOST desired_json `pbs_dr` descriptor + generation bump | Descriptor lives in the host desired_json, NOT ConfigJSON (buildConfigJSON drops foreign keys on re-save). v0.51.0: driven by `cfg.DRTier` (set from the form BEFORE applyOffsite/applyPBSDR); UNMET preconditions are honest waiting stages (save succeeds), REAL failures stay fail-closed; already-provisioned = success-no-op (red-proofed); disable keeps the ep0 tenancy. |
 | `(*Server).pbsdrProvisionAtom` + `PBSDRAutoProvision` | hub/internal/web/pbsdr.go | `(ctx, customerID, host, storageID) (blocked string, err error)` / `(ctx, customerID)` | The shared fresh-provision cascade atom; the WG-registration hook target (api `SetWGRegisteredHook`, wired in hub/cmd/hub/main.go when tenantsync is on) | `blocked != ""` = waiting stage (never an error); the hook runs in a detached goroutine and must never fail registration. Scenario-A e2e test: TestPBSDR_AutoProvisionOnWGRegistration. |
 | `cfg.DRTier` + offsite coupling | hub/internal/store/store.go (CustomerConfig), hub/internal/web/configs.go (applyOffsite guard) | bool | Per-customer DR-tier flag: new-customer default ON (handleConfigNewForm); offsite REFUSED without it (exact F-6 message) | One-time migration backfill initializes legacy rows from descriptor reality — never re-runs (opt-outs survive re-open; store test pins it). Form field `dr_tier` (formBool helper). |

STATUS.md

@@ -1,6 +1,6 @@
 # STATUS — what works, what's broken, what's next
 
-**Updated 2026-08-03.**
+**Updated 2026-08-04.**
 
 > **A view, not a source.** `documentation/backlog/OPEN-ITEMS.md` is the authority on open work; this
 > page restates part of it in plain words, and **nothing may exist only here**. **Not `CONTEXT.md`**,
@@ -14,98 +14,178 @@ A blank machine boots the Felhom disc, installs itself unattended, and is claime
 who sets their own password. They install apps from a catalogue of fifty-three, share files over the
 home network, and open apps from a launcher or a shared link. Backups run on their own to three
 places — the machine's drive, a second drive, and an encrypted off-site copy — and a customer can
-restore files and app data from the drive alone. Proven end to end on real hardware.
-
-**Apps come back after a power cut.** The machine tells an app the customer switched off from one
-that simply did not come back, and waits for the system to finish starting before deciding instead of
-glancing once, five seconds in. Hard-reset the demo box six times in a row: everything came back every
-time, and an app switched off deliberately stayed off every time.
+restore files and app data from the drive alone. Apps come back after a power cut: hard-reset the demo
+box six times, everything returned every time, and an app switched off deliberately stayed off.
+Proven end to end on real hardware.
 
 ## What's broken
 
-**The off-site copy can be erased by the machine that made it** — the credential that writes it can
-also delete it. A daily snapshot is armed as a stopgap, and we have never restored from that copy.
-*(R-95, R-87)*
-
-**A full disk emails you repeatedly instead of once.** When the reserve refuses an app's backup you
-are told once by the backup run — correctly — but the page showing backup status re-checks on a timer
-and sends the same message again each time. Not new: as old as the reserve itself, and seen only
-because we watched the alerts closely while proving the fix below. Harmless if the hub already
-collapses repeats — which a comment claims and nobody has checked. *(R-182)*
+- **Rebuilding a machine still throws away its off-site backup HISTORY.** The machine invents the key
+  that encrypts its own off-site backups, and a rebuilt machine invents a brand-new one, so the old
+  backups can no longer be opened by the machine that wrote them. Both demo machines did this on 3–4
+  August — demo-hp lost access to 15 backups (41 MB), demo-felhom to 36 (1.14 GB), and **the second one
+  happened in silence for thirteen hours**. From today that silence is fixed (below) and the old key
+  survives the ceremony (below), but **the rebuild itself still starts a fresh history**, and getting
+  the old one back is a path nobody has walked. *(R-193)*
+- *(fixed 4 Aug)* ~~The safety net we believed was under all of this is not there.~~ **The central hub
+  was keeping the wrong key, and the fix shipped the same day.** When a machine re-seals its backups
+  under a new recovery code, the hub keeps the old sealed package — but it was keeping the one for the
+  whole-machine backups and **not** the one for the off-site file backups, which is the whole point. And
+  because the swap happens in one motion, **the act that destroyed the old key was the recovery-code
+  ceremony itself** — the exact thing a rebuilt machine asks its customer to do, on a screen promising
+  the old backups stay recoverable. Both keys are now kept. **What this cannot undo:** anything
+  superseded before today is gone for good, which includes both demo machines' pre-4-August keys — so
+  those 51 orphaned backups were beyond reach even if the recovery codes had been kept. *(R-198)*
+- **IT WORKS. A customer's file survived a machine being destroyed and came back — proved last night on
+  real hardware, end to end.** We wiped the HP machine's controller data on purpose and deleted the
+  marked file from its disk. Using the recovery code you saved: the backup key came back **identical,
+  character for character**; the existing off-site store **opened** rather than starting over (the same
+  three backups, the same 42 026 bytes as before the wipe); and the file was restored **byte for byte
+  identical**. That is the whole backup promise, demonstrated for the first time. *(R-201)*
+- **But a customer could not have done it alone, and that is now the open work.** Getting from "the key
+  is recoverable" to "the file is back" took four steps that appear in no instructions: re-issuing the
+  storage credential; re-claiming the machine (whose local reset-code tool **does not work until the
+  controller is restarted** — two attempts failed before we spotted it); manually clearing a "stale"
+  mark that the re-issue sets even though the key never changed; and choosing the *full* restore,
+  because the default one returns the app's settings and **not** the customer's documents — with
+  nothing saying so. **The capability is real; the experience is not built yet.** *(R-204)*
+- **One screen still tells the customer something we cannot yet promise.** The „elárvult tároló" card
+  says the old backups may later be restorable with the matching recovery code. From today that is true
+  for machines that re-seal from now on and **false for anything already orphaned** — and the machine
+  cannot tell which case it is in without the recovery path above. We deliberately did **not** patch the
+  sentence: a conditional promise that can still be wrong is worse on that screen than a vague one.
+  *(R-202)*
+- *(fixed 4 Aug)* ~~The weekly off-site backup reports FAILED although it worked.~~ It uploaded fine
+  and then tripped on a tidy-up step it is deliberately not allowed to perform. The machine no longer
+  asks — tidying up is the endpoint's job, and **that was checked first**: the endpoint has been doing
+  it every day since 27 July. Both machines and all future ones. *(R-191)*
+- **The off-site copy can be erased by the machine that made it.** The credential that writes it can
+  also delete it. A daily snapshot is armed as a stopgap.
+  *(R-95, R-87)*
 
 ## What shipped recently
 
-**The backup partition is gone, and both demo machines run on the new shape.** Wiped and rebuilt on
-3 August and taken through the whole customer journey — set up, install an app, back it up, restore
-it. One storage area instead of two; the space a backup can use went from 19 GB to 65 GB on the small
-machine and 45 GB to 233 GB on the big one. Three reboots each, correct every time. The two were
-rebuilt deliberately differently — one from a local copy of the image, one by the ordinary customer
-route with the published fingerprint checked — so the disk shape and the delivery route are both
-proven, rather than one proven twice. Their previous demo apps and data are gone; that was the point
-of a wipe, and you approved it. *(R-165, R-178)*
+- **The daily error email about David is gone.** David's record was created when he said he would try
+  Felhom; no machine was ever attached to it, so nothing has ever been expected of it — yet every
+  morning at five the system reported that a database backup had not happened. **A customer with no
+  machine ever attached now produces no alarm.** A machine that *was* attached and went quiet still
+  does, because that is a real fault, and there is a test whose whole job is to make sure a future
+  change cannot take that away. David's record was not touched — the record was right; the alarm was
+  wrong. **The first quiet morning will be 5 August, so this is not confirmed yet.** *(R-195)*
 
-**What replaced the wall — and it now watches the right moment.** The wall was quietly doing a second
-job: keeping a runaway backup from eating the space the machine needs to keep running. That job is now
-explicit, and as first built it was checked too late — the big write happened first, unchecked, and
-only the small write after it was refused, while the message still promised your last good copy was
-untouched. **Fixed and proven on 3 August.** The machine now decides once, per app, **before it writes
-anything at all**, and that one answer covers all three steps: a refused app writes nothing, is not
-restarted, and the promise is now literally true — checked by fingerprinting every file before and
-after. It also stopped being blind to size, so an app is no longer waved through at 96% full and then
-allowed to write two gigabytes. Proven by deliberately filling a demo machine, once for each way it
-can refuse. Nothing is ever deleted to make room: every app has only one local copy, so "delete the
-oldest" would always mean destroying some other app's only copy. *(R-181)*
+- **A permission the machine needs can now repair itself — and it tells you it had to.** Yesterday we
+  found a permission that vanished with no reinstall and no recorded action; the machine now notices,
+  restores it through the same controlled helper that grants it in the first place, and **raises an
+  alert saying it was missing and was restored**. The alert is the point: a permission that silently
+  fixes itself is one that can be lost over and over and never investigated. Proven twice on a demo
+  machine by taking the permission away by hand. *(R-190 — the repair; why it vanished is still open)*
 
-**The last of the three apps that never saved their data is fixed.** Installed nowhere, so nothing was
-stranded — checked on both demo machines and in the fleet list rather than assumed. Proven by the check
-that caught it, run in both directions: it clears the fixed version and still convicts the old one.
-*(R-156)*
+- **The on-machine backup copy has now been proved to restore — by the machines themselves**, both of
+  them, overnight and unasked (84 and 109 seconds). Every restore proof before this was of the
+  off-site copy. *(R-86, R-185)*
+- **Three ways the alarm system was misreporting its own work — all fixed.** None of them ever risked
+  data. **(1)** When the machine proved a backup restores, that result could vanish if the agent was
+  restarted in the following quarter-hour — and yesterday's change made the gap a week rather than a
+  day, because the machine correctly refuses to re-prove an archive it has already proven. It is now
+  written to disk with the result and survives. This was caught happening, not predicted: a real
+  14.5 GB off-site restore passed and left no record at all. **(2)** Every release had about a
+  fifty-fifty chance of emailing you a failure for a release that worked; the version tag is now
+  published after the binary, and a new check catches the opposite mistake so nothing is traded away.
+  **(3)** A released binary can now be rebuilt by anyone and checked against the fingerprint you
+  approve — until today, rebuilding produced different bytes. *(R-189, R-188, R-186)*
 
-**A filling disk warns the customer before anything breaks, and a failed backup reaches you** — the
-customer while there is still room to act, naming the drive and the space left; you when one app's
-backup fails, with the disk figures. The customer is deliberately not told about the second: they can
-free space, but they can do nothing about a failed backup. Both proven by filling a real disk. There
-are two rules and not one because the serious warning fired on free space while the disk was only 91%
-full — a percentage alone would have missed it. *(R-167, R-158)*
-
-**The checks have two nets and the second emails you.** Every repository has one command that runs all
-its checks, before every push. That one can be skipped, so the build server runs them again and emails
-you on failure. It cannot *stop* a change — everything goes straight to the main copy with no review
-step — but it notices quickly and tells you. *(R-29, R-161, R-168, R-169)*
 
 ## What we're working on
 
-- **Now:** both of today's items are done — the reserve and the last unsaved app. Your two decisions
-  are written down and are ours to build.
-- **Next:** building those two — moving the installer onto a labelled version so publishing is one
-  step you can undo, and a check that refuses to install a version nobody can download *(R-110, R-115)*.
-- **After:** the off-site copy that the machine making it can still erase *(R-95, R-87)*.
+- **Now:** making that journey something a customer can actually follow — the four steps above, worst
+  first: the restore default that silently returns the wrong thing. The HP machine is sitting
+  mid-drill with the recovered key already on it; it needs re-claiming and one setting confirmed, then
+  the backup runs and the file is restored. **Do not let it make a new recovery code** — that would
+ The marked file now lands in
+  the off-site backup, so there is finally something to recover. Everything else is already in place on
+  the HP machine — the recovery code you saved, a working off-site store, the app and the file. Nothing
+  has been wiped; that step waits for your go-ahead at a marked stop point. Both honesty fixes shipped today: a changed backup key now
+  raises an alarm on the day, and the email that stated the opposite of what it measured now describes
+  what it actually saw.
+- **Also tomorrow:** confirming what the 04:15 off-site run does on both machines. We expect it to
+  refuse and ask, rather than start fresh — but that is a prediction from the code, not something
+  anyone has watched happen.
+- **Next:** proving the off-site *app-data* copy can actually be restored — the one tier nothing
+  tests unattended. Most of the machinery it needed arrived with the restore-test change below.
+  *(R-87)*
+- **After:** the off-site copy that the machine making it can still erase. *(R-95)*
 
 ## Waiting on you
 
+- *(decided 4 Aug — recorded here until the work it unblocks is done)* You chose **not** to keep a
+  copy of the backup key on the Proxmox host. That is a coherent choice and it has a consequence worth
+  restating: **the customer's own recovery code is now the only route back from a rebuild**, so the
+  sealed package the hub keeps is the single copy. Today's fix is what makes that true rather than
+  wishful — until this morning the ceremony destroyed it. *(R-193)*
+- **Your ruling about the recovery screen has been priced, and it can be built.** You said: a freshly
+  installed machine that finds a sealed package waiting should say so loudly, offer the customer a box
+  to type their recovery code into, and show what would come back before doing anything. **All of it is
+  buildable, and one part is already free** — the hub is *already* telling every machine, on every
+  check-in, that a sealed package exists and which key it covers, and the machine currently throws that
+  message away. Showing a preview is also cheap: listing what is in an off-site store reads it without
+  writing to it, so the customer can see how many backups, from when, and for which apps before
+  committing. The real work is one new connection: the unsealing has to happen in the part that runs on
+  the Proxmox host, because the customer-facing part cannot do it — the same crossing your recovery-code
+  ceremony already makes in the other direction. **One thing for you to weigh, which we deliberately did
+  not decide:** a screen that takes a recovery code and then shows what is in a backup store is reachable
+  by anyone with the household's dashboard password, and the preview reveals backup dates and app names.
+  *(R-193)*
+- **Should we run the proof?** Nothing about recovery has ever actually been done, so we designed the
+  exercise: on the spare demo machine, make a recovery code and **keep it**, put a marked file into an
+  app, back it up off-site, wipe the machine, reinstall, recover with the saved code, and check the
+  marked file comes back byte for byte. About half a day, of which under an hour needs you. The failure
+  we are watching for is precise: **if the recovered machine reports one backup instead of the ones we
+  put there, it started a fresh history and the proof failed** — "the store opened" is not good enough.
+  It costs that machine's contents if it goes wrong, which is acceptable there. **You said yes, after
+  R-198 — and R-198 shipped this morning, so this is ready to run on your word.** *(R-201)*
+- **The orphaned backups on the storage box — you said delete, and it is still owed.** About 1.2 GB
+  across the two demo machines, in set-aside stores nobody can open and nothing prunes. Deliberately not
+  done today: erasing data on the storage endpoint is not something that should ride along with a
+  database change, so it wants its own session. *(R-193)*
 - **A job, not a decision: the hub password needs changing.** A diagnostic command printed it into a
   session log; nothing suggests anyone else saw it. *(R-132)*
-- **Nothing else.** You settled both open questions on 3 August — the installer moves onto a labelled
-  version, and a check will refuse to install a version nobody can download. Both are written down and
-  are ours to build. *(R-110, R-115)*
+- **One small question, not urgent.** The automatic check cannot see which version you have told
+  machines to install, only which ones exist. Closing that needs either a password given to the build
+  server or a check inside the hub itself. *(R-184)*
+- **Nothing else.**
 
 ## Changed since last update
 
-- **2026-08-03** — The reserve now guards the step that fills the disk, and its promise is true; the
-  last app whose data was never saved is fixed. Both proven on a demo machine, not just in tests.
-  Earlier the same day: both demo machines wiped and rebuilt from the new base image and taken through
-  set-up → install an app → back it up → restore it, with the backup space ceiling gone and measured.
-
-- **2026-08-02** — The false "host offline" warning is fixed. The hub's database was supposed to be in
-  a mode where reading a page cannot block a machine's status update; a one-word difference meant that
-  setting had **never taken effect**, for the hub's whole life. Fixed and verified live. **Also found:
-  the hub's own database is in no automatic backup** — it holds every machine's emergency password.
-  Filed, not yet fixed.
-
-- **2026-08-02** — Boot recovery finished; six hard resets, everything back every time. Two instances
-  of the same hole — starting an app whose external drive was missing — were found by reading the code
-  and fixed the same day.
-
-- **2026-08-02** — Thirteen mechanical checks had built up and nothing ran most of them; two were
-  failing quietly. Fixed. Decided the same day: the 20 GB backup partition goes away; and only this
-  machine and the tester's box are protected, every other box may be broken or reinstalled freely.
+- **2026-08-04 (night)** — **The drill PASSED.** We destroyed a machine on purpose and the customer's
+  file came back byte-for-byte identical, using the recovery code you saved. First time the backup
+  story has been proved end to end. It needed four undocumented manual steps to get there, which are
+  now the next piece of work. The machine is up, healthy and re-armed. *(R-201, R-204)*
+- **2026-08-04 (evening)** — **Fixed the folder-left-out-of-the-backup problem, both halves.** The app
+  and its backup now look in the same directory, and a backup that misses a folder marked essential
+  reports *incomplete* instead of success. Proved by listing the backup's own contents and finding the
+  marked file. The wipe-and-restore proof is unblocked. *(R-203, R-201)*
+- **2026-08-04 (late)** — Set up the wipe-and-restore proof on the HP machine and **stopped before the
+  wipe**: a folder marked as protected was missing from the off-site backup while the backup reported
+  success. Three things were proved on the way, all firsts: a rebuilt machine's off-site backup now
+  **refuses and says so** instead of quietly starting over; the "start a new store" repair works and
+  keeps the old data aside; and the HP machine's pre-3-August history is gone for good — its key was
+  destroyed four hours before the fix that would have kept it. *(R-203, R-201, R-193)*
+- **2026-08-04 (evening)** — **Proved the backup key comes back.** A demo machine fetched its own
+  sealed package, opened it with the saved recovery code, and produced a key identical to the one it
+  uses. A wrong code was refused and wrote nothing. The recovery code left no trace anywhere on the
+  machine — searched, with a planted copy first to prove the search worked. *(R-199, R-200)*
+- **2026-08-04 (earlier)** — **Fixed the worst of it.** The hub now keeps the off-site backup key when a
+  machine re-seals, instead of only the whole-machine one — four lines of database change that stop the
+  recovery ceremony from destroying the thing it exists to protect. A changed backup key now raises an
+  alarm the same day. The daily email that described the opposite of what it measured now states what it
+  saw, and a refusal to auto-repair leaves a record instead of silence. *(R-198, R-197, R-192, R-196)*
+- **2026-08-04 (later)** — Established what a machine rebuild actually destroys: not the storage
+  password we had been re-issuing, but the key that encrypts the backups. Both demo machines lost
+  access to their off-site history, one of them silently. A decision is now waiting on you. The daily
+  false alarm about David is fixed. *(R-193, R-195)*
+- **2026-08-04** — Both demo machines proved their on-machine backup restores, on their own,
+  overnight — the copy an ordinary recovery uses, never tested until now. Found while checking: the
+  weekly off-site backup reports failure after a successful upload. *(R-185, R-191)*
+- **2026-08-03** — Fixed three ways the alarm system misreported itself: a proof of a working backup
+  that could vanish on a restart (seen happening), a release that emailed a failure for a release
+  that worked, and a released binary nobody could rebuild and check. *(R-189, R-188, R-186)*

documentation/architecture/03-host-agent.md

@@ -378,6 +378,60 @@ nicety.
   **loses Felhom's backup guarantees** — no PBS verify / monitoring on storage we can't reach. An
   honest signup-time tradeoff, not a hidden one.
 
+#### Custody across generations — what the hub retains when a blob is superseded (hub v0.60.0; **contract corrected hub v0.93.0, R-198**)
+
+A ceremony that seals a *different* passphrase does not destroy the blob it replaces: the hub copies
+the current `host_escrow` row into `host_escrow_superseded` **before** overwriting it, so a previous
+generation stays openable with the recovery code that sealed it. The same routine
+(`demoteCurrentEscrowTx`) is used when a host is deleted, which **demotes** custody rather than
+destroying it. The hub never decrypts either table.
+
+**The contract as originally shipped was incomplete, and the gap is worth recording because it was
+invisible for two months.** An escrow generation is **two** sealed artifacts — the K-escrow `blob`
+(the PBS datastore key `K`) and `identity_blob` (the age-wrapped `IdentityBundle`, which carries the
+offsite **restic repository password** as well as the tunnel/PBS tokens and the WG key). Retention
+covered only the first. So a supersession preserved the whole-guest key and destroyed the off-site
+data key — and because the copy happens as the new blob lands, **the destroying act was the ceremony
+itself**, i.e. the step a rebuilt box asks its customer to perform. Corrected in hub v0.93.0: the
+retained row carries both.
+
+**Two consequences that belong in this doc rather than a changelog:**
+
+- **Generational custody is now the recovery precondition for the offsite tier.** The operator ruled
+  on 2026-08-04 that no repository password is retained on the Proxmox host (R-193 candidate (c),
+  refused). With no host-side copy, a rebuilt box's only route back to its previous off-site history
+  is *the customer's older recovery code opening the retained identity blob* — which exists only if
+  the supersession that replaced it happened on hub v0.93.0 or later.
+- **Retention is custody, not recovery, and this doc must not be read as claiming otherwise.** The
+  hub's restore-mode serving of these blobs (§ slice 10D) has **no client** in the agent, the
+  controller's recovered-password injection seam has **no user interface**, and no part of the chain
+  has ever been exercised outside a unit test — the one live identity round-trip (2026-06-10)
+  predates the `ResticRepoPassword` field. Tracked as R-199/R-200/R-201; evidence
+  `documentation/audits/RECON-offsite-dr-chain-2026-08-04.md`.
+
+#### Serving a blob back to its own box (hub v0.94.0 + agent v0.125.0, R-199)
+
+Retention without retrieval is custody, not recovery. Since 2026-08-04 a host reads its own sealed
+identity blob from `GET /api/v1/hosts/{host_id}/escrow` — self-scoped by the per-host key, the mirror
+of the PUT that stored it — and opens it locally with the customer's recovery code via the agent's
+`POST /escrow/recover-offsite-password`, which returns **only** the offsite repository password.
+
+**This is a deliberate widening of who can obtain the ciphertext**, recorded in `CONTEXT.md` S-29 and
+in the handler itself: it previously took an operator arming recovery mode, and now takes only that
+box's own credential. The blob remains unopenable without R, and every retrieval raises an
+operator-visible `escrow_blob_served` event. The operator-driven re-enroll/restore-directive path is
+unchanged and keeps its recovery-mode gate.
+
+**Proven end to end on demo-felhom, 2026-08-04:** the recovered repository password's sha256 was
+byte-identical to the one on the box, and to the hash the hub had independently recorded. **The chain
+stops there** — nothing installs a recovered password, reopens a repository with it, or restores a
+file.
+
+**The honesty property above is unchanged and now has a second edge:** losing `R` and the box makes
+the offsite backups unrecoverable by anyone — *and* an older generation's data is recoverable only
+with **that generation's** recovery code, which the customer must still hold. A current recovery code
+does not open a previous generation's blob.
+
 ## 9. Provisioning & DR flows
 
 **Provisioning (reconcile-driven, by restore).** Fresh creation of a Docker-capable LXC needs

documentation/architecture/07-backup-architecture.md

@@ -132,6 +132,41 @@ executed** (`CAMPAIGN-8…:522`), the host-loss plan **executes nothing by const
 (`felhom-agent/internal/dr/plan.go:1-4`), and **no host has ever been rebuilt as its former self**
 (INV Part D1).
 
+### Lane 2's restore-test is scheduled PER ARCHIVE GENERATION (R-86, 2026-08-03)
+
+**[CONTRACT, changed 2026-08-03 — agent v0.121.0 + hub v0.91.0.]** The scheduled restore-test used to
+fire on an interval started at daemon start. It no longer does. The rule is:
+
+> Let **A** be the newest archive on a tier that has settled for at least the settle lag (24 h).
+> The tier is **DUE** when **A** exists and **A has not already been proven**.
+
+So a tier is proved **once per archive**, on its own archive, and the proof follows the backup rather
+than the process's uptime:
+
+| tier rhythm | what is proved, and when |
+|---|---|
+| daily (host tier) | yesterday's archive, once a day |
+| weekly (offsite tier) | last week's archive, once a week |
+| newborn (no archive yet) | nothing — **UNKNOWN, never a fault** |
+
+**The trap in the obvious formulation, recorded so it is not reintroduced:** *"due when the newest
+archive is ≥ 24 h old"* is never true on a **daily** tier — a new archive resets the newest-archive
+age to zero long before it reaches the lag — so the literal reading silently switches restore-testing
+off for the tier that matters most.
+
+What survives unchanged: the restore-test itself (restore → boot → verify → destroy the scratch), its
+journal and crash recovery, the scratch VMID band, the one-heavy-operation gate, proof credit only on
+success, and oldest-proven ordering, which is now the tie-break **between due tiers**. A ticker
+remains, but only as the **evaluation interval** (6 h by default, chosen from a measured cost: one
+due-check is 18 ms on a local dir storage and 392 ms on the PBS tier over the WAN).
+
+**The hub's half is not optional.** `restoreProvenStaleAfter` was a flat 7 days derived from the very
+cadence this replaced, and a weekly tier proved weekly reaches a proof age of **exactly** one interval
+just before its next proof — 168 h against a 168 h window. It sat ON the line, so any ordinary delay
+tipped a healthy tier into a nightly alarm. The window is now per tier, from that tier's observed
+archive interval, floored at the old 7 days, capped at 12 days (strictly inside the two-week offsite
+retention), and falling back to the tier's declared rhythm when history is too short to observe one.
+
 ### Why the split is right, stated once
 
 **[DESIGN]** A customer can reason about "my photos are gone". A customer cannot reason about
@@ -243,7 +278,10 @@ The tiers are **inputs to recovery**, not recovery routes. §7 and §8 say what
 | **Tier-2** cross-drive | `<target nsRoot>/backups/secondary/<app>/` | **always** a full mirror of the unit (`tier2.go:368-369`) **plus** `mandatory + optional` file legs, v2 layout `hdd/<rel>` + `userdata/<rel>` | nightly at W+60m | mirror (rsync) | no |
 | **Tier-3** restic offsite | Hetzner Storage Box over SFTP, one multi-path snapshot per app per run | the unit **plus** `mandatory` legs only; a separate `_shares` snapshot | nightly at W+105m | `--keep-daily 7 --keep-weekly 4 --keep-monthly 6` | **yes** (restic) |
 | **Plane-2** whole-guest, local | `local:` → `/var/lib/vz/dump` on the host | rootfs + `mp0 /var/lib/docker` + `mp1 /mnt/sys_drive` | **24 h** (`backup_cadence_seconds: 0`) | `local_backup_retention: 3` | **no** |
-| **Plane-2** whole-guest, offsite | `felhom-pbs:` → ep0 datastore `felhom-offsite`, per-customer namespace, over WireGuard | same contents | **7 days** (`604800`) | server-side prune on ep0, `keep-last 2` at `03:30` | **yes** (per-customer key) |
+| **Plane-2** whole-guest, offsite | `felhom-pbs:` → ep0 datastore `felhom-offsite`, per-customer namespace, over WireGuard | same contents | **7 days** (`604800`) | server-side prune on ep0, `keep-last 2` at `03:30` — **and the box asks for none** (R-191) | **yes** (per-customer key) |
+
+> **R-191 (2026-08-04) — this row was RIGHT and the configuration disagreed with it, weekly, for as long as R-89 has been in force.** The contract has not changed: offsite retention is ep0's, the box's token is write-only, and the box cannot delete its own history. What had not followed was the installer's `keep_last: 2` on the offsite tier, so every weekly run uploaded its snapshot successfully and then failed the whole JOB on a prune the token is refused — `whole_guest_backup_failed` in the operator's inbox about a backup that had already succeeded. Fixed in installer **1.25.0** (`keep_last: 0`) and on both live boxes; a gate now asserts it. **Verified before changing it:** ep0's two prune jobs have run every day since 2026-07-27, 18 tasks, all OK. A doc that states the contract does not enforce it — the gate does.
+
 
 **[FACT]** The three nightly legs derive from **one** customer-settable window start W at fixed
 offsets — db-dump at W, Tier-2 at W+60m, offsite at W+105m — so they can never be misordered
@@ -574,6 +612,30 @@ because that was a different filesystem. On a merged box it can. Decision **B2**
 It **refuses per app and never deletes**: nothing on this filesystem is generational, so pruning could
 only destroy a different app's only local copy.
 
+**WHAT IS RECORDED, WHAT IS E-MAILED, AND HOW OFTEN (controller v0.194.0 + hub v0.90.x, R-182).**
+The two are deliberately different mechanisms, because conflating them is how seven failures went
+missing on 2026-08-03 without leaving a trace.
+
+| | Record | Notification |
+|---|---|---|
+| what | `recovery_unit_capture_failed`, one per failed app | `backup_run_failures`, one per RUN |
+| when | every time, unconditionally | at the end of a run, **only if something failed** |
+| gated by | nothing — not cooldowns, preferences or delivery | the hub's operator cooldown |
+| where it lands | the events table **and** `notification_log` (status `recorded`) | the operator's inbox |
+
+- **A clean run e-mails nothing.** Silence means the run finished and found nothing wrong — and that
+  is only safe because the hub's daily deadline check raises `expected_backup_missed` from the box's
+  REPORT freshness, independent of any mail the box sends. That check is load-bearing for this
+  design; weakening it re-opens a silent-failure path.
+- **A suppressed operator notification leaves a `suppressed` row** naming the key that suppressed it.
+  Deciding not to tell someone is itself an event worth recording.
+- **Deliberate skips are not failures** and never appear in the digest — a disconnected or
+  decommissioned drive has its own alert, and a nightly digest about an unplugged drive is one the
+  operator stops reading.
+- **Cadence:** a nightly run gives at most one mail a day. A manual run always reports, even within
+  the hour, because someone pressing the button is actively trying to get a backup. The periodic
+  capture sweep is capped by the ordinary hourly cooldown.
+
 **THE CONTRACT, stated as what the code provides (controller v0.193.0, R-181).** The reserve is a
 **per-app, per-run ADMISSION decision, not a capture check.** It is taken once for an app, immediately
 before that app's FIRST write of the run, and it covers **all three write legs — the database dump, the
@@ -796,7 +858,7 @@ does **not** hold as written. → **R-108**
 | ~~**R-108**~~ | ~~Network storage can host an app's namespace~~ | **CLOSED 2026-07-30, controller v0.187.0 — D5 UNBLOCKED.** An app namespace may no longer be placed on network storage (5 surfaces guarded by one fail-closed predicate); the share-root bind is deliberately UNCHANGED because it is load-bearing and unscopable (§10.1). `audits/R108-network-app-namespace-2026-07-30.md` |
 | **R-126** | A `.fab` bundle — plaintext secrets, optional password — can be exported ONTO a NAS: `storageDriveList()` (`internal/web/handler_export.go`) does not filter network paths | split out of R-108, which closed without it. NOT a D5 precondition: an explicit customer-chosen export destination, not a browsing surface reaching a backup tree (§5, §7.3) |
 | R-95 (open) | The restic offsite credential **can delete** — the box can `forget --prune` its own repo | the tier holding the customer's documents and photos is the one whose credential can destroy it (matrix row 10) |
-| R-86 (open) | Restore-tests are interval-scheduled, not backup-aligned | a tier's proof cadence is unrelated to when its archives are written |
+| ~~R-86~~ | ~~Restore-tests are interval-scheduled, not backup-aligned~~ | **CLOSED 2026-08-03 — agent v0.121.0 + hub v0.91.0.** Restore-testing is now **per archive generation**: a tier is due when its newest archive that has settled ~24 h has not been proven, so a daily tier is proved daily on its own archive and a weekly tier weekly on its own. The ticker survives only as the evaluation interval (6 h, chosen from a measured cost). The hub's staleness window moved with it — per tier, from that tier's observed archive rhythm — because a weekly tier proved weekly sat EXACTLY on the old flat 7-day line (§3, Lane 2's per-archive rule) |
 | R-87 (open) | The restic tier is never restore-tested | matrix row 4's route has no unattended proof |
 
 ### 10.3 Divergences that are documented elsewhere and are not re-opened here
@@ -862,7 +924,7 @@ to now *implement* D5 remains an open scheduling decision, not a blocked one.
 | whole-guest restore, local and PBS, exact mount parity | **PROVEN-LIVE** | CAMPAIGN-2 T-P9; CAMPAIGN-8 Phase C |
 | corrupted PBS snapshot fails cleanly | **PROVEN-LIVE** | CAMPAIGN-8 fault 17 |
 | the box cannot delete its own **PBS** snapshots | **PROVEN-LIVE** | CAMPAIGN-8, R-89 |
-| unattended restore-test across tiers | **IMPLEMENTED** (rotation not observed across consecutive cadences) | `00-capability-map.md:41`; LIVE per-tier timestamps this session |
+| unattended restore-test across tiers | **IMPLEMENTED**; **per-archive due-ness PROVEN-LIVE 2026-08-03** (agent v0.121.0) | `00-capability-map.md:41`; the due verdict + a real offsite run on demo-felhom (§3, Lane 2's per-archive rule) |
 | guest-power watchdog | **PROVEN-LIVE** | agent v0.107.0, 120 s |
 | quiesce crash recovery | **PROVEN-LIVE** | CAMPAIGN-8 fault 10, 1 s, by SIGKILL |
 | break-glass | **PROVEN-LIVE** | `runbooks/break-glass.md` |

documentation/audits/DRILL-r201-night-run-2026-08-04.md

@@ -0,0 +1,265 @@
+# DRILL — R-201, the night run: **PASS. The customer's file came back, byte-identical.**
+
+**Date:** 2026-08-04, 21:30–23:15 · **Box:** `demo-hp` (Tier 0) · unattended, then **finished with the
+operator present**. The box is up, healthy, and fully re-armed.
+
+> **THE VERDICT: PASS.**
+>
+> ```
+> pre-wipe   sha256: 643166269103a25cf41d34a26b75fd6ebba0a837bbcd7e0d2b5aba7106bcbe7c
+> restored   sha256: 643166269103a25cf41d34a26b75fd6ebba0a837bbcd7e0d2b5aba7106bcbe7c
+> ```
+>
+> A machine was destroyed on purpose — controller data volume gone, the customer's file deleted from
+> disk — and **the file came back byte-identical**, through the customer's own recovery code and the
+> customer's own restore flow. **This is the first time the Felhom backup story has been proved end to
+> end.**
+>
+> The key recovered byte-identical (`8a9e33aa4da6…`), the existing repository **opened rather than
+> being re-initialised** (`repo_state: null`, 3 snapshots, 42 026 B — the pre-wipe size exactly), and
+> the pre-wipe snapshot `e6132ae5` was still there with the file inside it.
+>
+> **It took four undocumented steps to get there**, each measured, and they are the other half of the
+> result — see §3. A customer hitting this alone today would not get their data back.
+
+---
+
+## 1. The verdict — **PASS**
+
+| | |
+|---|---|
+| sentinel sha256, pre-wipe (step 3) | `643166269103a25cf41d34a26b75fd6ebba0a837bbcd7e0d2b5aba7106bcbe7c` |
+| sentinel sha256, restored (step 10) | **`643166269103a25cf41d34a26b75fd6ebba0a837bbcd7e0d2b5aba7106bcbe7c`** |
+| **match** | **YES — byte-identical** |
+| snapshot count at step 9 | **3** — the pre-wipe count. **NOT 1.** No fresh history was started |
+| repo size at step 9 | **42 026 B** — the pre-wipe size exactly |
+| `repo_state` | **`null`** — the repository OPENED. Not orphaned |
+
+The restore ran through the customer flow (`POST /backup/offbox/restore`, `mode=full`, two-step
+confirm), out of snapshot `e6132ae5` — the one taken **before** the wipe.
+
+**A detail worth keeping:** the first restore attempt used the default `mode=unit`, which recovers the
+recovery unit only and **not** the userdata leg. `mode=full` is what returns the customer's files. A
+customer told to "restore from off-site" would, on the default, get their app definition back and not
+their documents — and nothing in that outcome says so.
+
+## 2. What was proven, in order, on hardware
+
+### Step 6 — the wipe
+
+The controller's data volume was destroyed and the bootstrap re-ran. Fresh data directory, everything
+stamped `2026-08-04 20:00:2x`:
+
+```
+encryption.key   32 B   2026-08-04 20:00:21     ← brand new: every pre-wipe app secret is undecryptable
+claimed = None                                   ← the fresh-install signal
+offbox  = null                                   ← no off-site target
+```
+
+**Faithful to the incident, and deliberately so.** The 2026-08-03 rebuild that R-193 is filed against
+was **not** a guest reprovision — the journal shows guest 9201 running continuously through that
+window, with no `pct destroy`, no `pct restore` and no `--selftest=provision`. What changed was the
+controller and its data volume. Reproducing *that* is what the drill needs; a guest destroy plus an
+unrehearsed provisioning chain, improvised unattended, is what §8.10 exists to prevent.
+
+### Step 7 — the assertion that keeps recovery possible: **PASSED**
+
+```
+host_escrow (demo-hp-bb76ea), AFTER the wipe:
+  identity_blob    = 572 bytes          ← unchanged
+  restic_pw_sha256 = 8a9e33aa4da6…      ← unchanged
+  updated_at       = 2026-08-04 11:11:37 ← unchanged; nothing re-escrowed
+  stale_at         = NULL
+```
+
+**No ceremony was run and nothing re-escrowed itself.** The sealed key survived the rebuild untouched.
+
+### Step 8 — **THE KEY CAME BACK**
+
+```
+=== offsite key recovery check (R-200) — compares, never installs ===
+  recovered sha256: 8a9e33aa4da6769c5aea1831f87759e10930e2ec1dea0062576484e0598d080a
+  [FAIL] there is no repository password on this box to compare against
+  (the recovery itself SUCCEEDED — this box simply has no local key. That is the
+   rebuilt-box shape, where the next step is to INSTALL rather than compare.)
+```
+
+Exit 1 is **correct and designed**: there was nothing to compare against, because the wipe removed it.
+The recovered hash matches the pre-wipe on-disk key and the hub's own record — three independent
+sources agreeing, one of them recovered through the full chain (hub → agent → unseal → extract) on a
+box that had just lost everything it knew.
+
+### Step 9a — installed cleanly
+
+```
+=== offsite key recovery INSTALL (R-200) ===
+  on-disk   sha256: (none — this box has no repository password)
+  recovered sha256: 8a9e33aa4da6…
+  [INSTALLED] the recovered repository password is in place and reads back identical.
+```
+
+The "installed" branch — the rebuilt-box shape v0.196.0 was written for — took its first real run.
+
+### Step 9b — the apply kept it
+
+After the target was reconfigured, the on-disk key was still `8a9e33aa4da6…`. `WriteOffboxSecrets`
+found the file present and kept it, exactly as documented.
+
+---
+
+## 3. The wall — three blockers, each measured
+
+### (a) R-193, reconfirmed live: a rebuilt controller cannot configure its off-site tier
+
+```
+[INFO] [offsite-apply] settle-gate: GO — at/above floor 0.156.0 (we are 0.197.0)
+[WARN] [offsite-apply] reconcile: offsite-apply: consume one-time password:
+       no unconsumed offsite password (already consumed or none provisioned)
+       (retries on next config refresh/restart)
+```
+
+The ledger, measured: `demo-hp` one-time secret created `07:11:51`, consumed `07:12:06` — **by the
+previous controller**. The rebuilt one has nothing to consume and retries forever. Exactly the state
+demo-hp sat in for 25 hours on 2026-08-03.
+
+**Remedy:** an operator Re-issue. Performed here through the designed endpoint
+(`POST /configs/demo-hp/offsite-reissue`, HTTP 303), after which the target configured normally.
+
+### (b) R-196, measured live — and it lands squarely on the recovery path
+
+```
+host_escrow (demo-hp-bb76ea), after the Re-issue:
+  stale_at         = 2026-08-04 20:15:49   ← set
+  restic_pw_sha256 = 8a9e33aa4da6…         ← UNCHANGED. The key did not move.
+```
+
+**The escrow was marked stale while it perfectly covers the box's current key** — the recovered one.
+That is R-196's false staleness, and here it is not a cosmetic lie: a stale escrow makes the hub
+**withhold the hash from the report ACK**, so `EscrowAutoConfirmer` can never flip `pending →
+escrowed`, and `OffboxRunnable` (`configured && escrowed`) **refuses to run any off-site backup**.
+
+> **The remedy for (a) disables the recovery it was needed for.** The only documented way to clear a
+> stale escrow is a fresh ceremony — which supersedes the identity blob, i.e. **destroys the very key
+> being recovered**. Under hub v0.93.0 the old blob is now *retained*, but nothing can serve a
+> superseded blob back (R-199's inventory: the endpoint serves the CURRENT row only).
+
+**Superseded rows still number 2** — no ceremony was run tonight. The key is intact.
+
+### (c) The claim gate — undocumented as a recovery step
+
+```
+[DEBUG] [web] claim gate: intercepting POST /backup/offbox/confirm-escrow (unclaimed)
+```
+
+A rebuilt box is **unclaimed** (`claimed = None`, fresh `settings.json`), and the claim gate correctly
+intercepts every non-claim route. **So no controller endpoint can be driven at all** — not the manual
+escrow confirm, not the backup trigger — until the customer re-claims the box. Both POSTs returned 302
+to the claim page and neither reached its handler; `last_run` stayed null, which is why the earlier
+reading of "the run was refused" needed this second look to be accurate.
+
+This is correct behaviour and it is **not wrong** — but it is a step in the customer's recovery journey
+that appears in no design document, and it comes *before* anything else can happen.
+
+---
+
+## 4. How it was finished, and the fourth undocumented step
+
+The unattended run stopped at the wall in §3 rather than improvising further at 22:35 (§8.10). With the
+operator present the remaining path was walked, and it took **four steps a customer would have to
+discover**:
+
+1. **Re-issue** the offsite credential (hub, operator-only) — otherwise the box can never configure its
+   tier. §3(a).
+2. **Re-claim the box.** The local escape hatch (`--print-reset-code`) mints a code — **and does not
+   work on its own**: it writes the new hash to `settings.json` while the RUNNING controller keeps its
+   old copy in memory, so `effectiveClaimCode()` never sees it and every attempt returns
+   *"Hibás vagy lejárt kód"*. **The controller must be restarted between minting and claiming.** Two
+   claim attempts failed on this before it was diagnosed; the generation counter reached 6.
+3. **Confirm the escrow** manually (`/backup/offbox/confirm-escrow`) → `escrowed`, so `OffboxRunnable`
+   permits a run. **Not a ceremony** — the identity blob was never superseded (still 2 retained rows).
+4. **Restore with `mode=full`.** The default `mode=unit` returns the recovery unit and **not** the
+   customer's files.
+
+**None of these four is in any design document, and the drill needed all four.**
+
+## 5. §5's five conditions, as recorded before the wipe
+
+| # | Condition | Evidence |
+|---|---|---|
+| 1 | §4 confirmed, sentinel **listed by name** | snapshot `e6132ae5` (19:36:26) — `-rw-r--r-- 1000 1000 181 … /DRILL-SENTINEL.txt` |
+| 2 | deliberate rollback archive, **verified** | `vzdump-lxc-9201-2026_08_04-21_44_24.tar.zst`, 1 606 765 083 B; **full zstd stream read OK** (4 867 573 760 B uncompressed); the sentinel confirmed **inside** it |
+| 3 | §3's option | see §6 |
+| 4 | demo-felhom untouched | `health=ok`, `escrow_state=escrowed`, not touched at any point |
+| 5 | free space | nvme-1tb 883 GB free, root 20 GB, local-lvm 34.32 % |
+
+**A precondition had drifted and was repaired before the wipe, not worked around.** The staged
+snapshot no longer contained the sentinel: the afternoon's `[WARN] mandatory data path missing`
+experiment produced a *later* same-day calibre-web snapshot, and restic's `forget --keep-daily 7
+--group-by host,tags` had pruned the good one in its favour. The fixture on disk was correct, so one
+backup re-established it and it was re-verified by listing. **Lesson worth carrying: a good snapshot is
+not durable against a later bad run on the same day.**
+
+## 6. §3 — the recovery code
+
+**Option B as already in place, with a strict improvement: no new copy was created, so nothing needed
+shredding.**
+
+The operator placed `R_DEMO-HP` in their own `~/.config/credentials` on DooPlex (mode `0600`) two
+sessions ago, deliberately, for this purpose. This session read it from there and **piped it to stdin**
+for each of the two invocations that needed it. It was never an argument, never exported, never written
+to a second file, and never logged.
+
+**Nothing was shredded, and that is the point:** the operator's own permanent store is theirs, not a
+drill artefact, and destroying it would have destroyed their record. Because no additional copy was
+made, there is nothing left behind to prove gone — a stronger position than option B's
+create-then-shred.
+
+**Verified afterwards** with the planted-copy positive control, exactly as on 2026-08-04 afternoon —
+see §8.
+
+---
+
+## 7. The state the box is left in — healthy and re-armed
+
+```
+controller   felhom-controller:0.197.0, healthy, CLAIMED
+apps         privatebin opengist calibre-web filebrowser cloudflared traefik  — all serving
+escrow_state escrowed          (no ceremony was run; identity_blob untouched)
+repo key     8a9e33aa4da6…     THE RECOVERED KEY — and the repository opens with it
+off-site     3 snapshots, 42 026 B, repo_state null; all three apps re-toggled for off-site
+sentinel     restored to its live location, sha256 643166269103a25c… — verified
+```
+
+The verification copy under `backups/offsite-restore/calibre-web/` is **left in place as evidence**.
+
+## 8. R persisted nowhere — searched, with a positive control
+
+Swept the agent journal, the controller container log, and `/tmp`, `/var/tmp`, `/var/lib/felhom-agent`,
+`/root` on demo-hp for the recovery code: **0 hits**, and **0** leftover `felhom-idesc-*` unseal
+staging directories. A planted copy was found by the same sweep (**1**) and not found after shredding
+(**0**), so the instrument is shown sensitive rather than assumed to be.
+
+---
+
+## 9. Teardown — three layers
+
+| layer | state |
+|---|---|
+| the guest | **Nothing torn down.** The wipe is the evidence; the apps are serving; the recovered key is in place. The scratch band is empty — no restore-test guest was created tonight. |
+| the host | `vzdump` snapshot LVs removed cleanly by the archive job (`snap_vm-9201-disk-0/1_vzdump` both released). One new archive, 1.6 GB, on `nvme-1tb` (883 GB free). Nothing deleted. |
+| the hub | **No new customer records.** `demo-hp` is the same customer row throughout — the rebuild was a controller-data wipe, not a re-enrolment, so nothing accumulated. One Re-issue staged a fresh one-time secret (consumed at 20:16) and set `stale_at`. The two superseded escrow rows are **unchanged** — no ceremony ran. |
+
+**Nothing was deleted on the storage endpoint**, including the ~1.2 GB of previously-orphaned
+ciphertext. Ruled, still owed, and deliberately not ridden along with a drill.
+
+## 10. Part 2 — not run
+
+Its gate is *"the drill PASSED; its evidence is written down; and there is time."* The first two are
+now met. **The third was not** — it was 23:15 after a full destructive cycle, and Part 2 is a second
+wipe. It is deliberately left for a session that starts fresh, with drill 1's result already recorded
+so a second wipe cannot overwrite it.
+
+**R-198's retention therefore remains unit-proven only.** Nothing has yet superseded a key in
+production, so the retention's live behaviour is still unobserved — and it is now the last unproven
+link in the chain this drill exercised. The check is cheap: wipe again, do NOT recover, let a ceremony
+seal a different password, then read the superseded row's `identity_blob`.

documentation/audits/DRILL-r201-offsite-recovery-2026-08-04.md

@@ -0,0 +1,311 @@
+# DRILL RECORD — R-201, the wipe-and-recover proof: **PREPARED, HALTED BEFORE THE WIPE**
+
+**Date:** 2026-08-04 · **Box:** `demo-hp` (Tier 0, the designated drill host) · **Nothing was wiped.**
+**Outcome:** the drill did not reach its verdict. It was halted at step 4 by a defect that makes the
+verdict unobtainable — and that defect is worth more than the drill.
+
+> **THE HEADLINE.** A customer-declared **mandatory** data directory was **silently absent from the
+> off-site snapshot**, while the backup reported `ok` with three snapshots. The only trace is one
+> `[WARN]` line inside the controller container. The hub, the card and the counters all say the backup
+> succeeded. → **R-203**
+>
+> **Nothing irreversible happened.** The wipe never ran. `demo-hp` is left in a BETTER state than it
+> started: it now has a working off-site repository (it had an orphaned one), three snapshots, and a
+> sentinel file on disk.
+
+---
+
+## 1. The verdict — not reached, and why that is the correct outcome
+
+The drill's pass condition is a byte-identical sentinel sha256 after a wipe. **Step 4 established that
+the sentinel is not in the off-site snapshot at all.** Wiping the box would therefore have:
+
+- destroyed the sentinel, which exists only on that box;
+- proven nothing about recovery, because there would be nothing to recover;
+- and done so *after* the point of no return.
+
+The runbook's own rule applies: *"If this session finds itself writing code beyond Part 0, stop. That
+means a precondition was wrong, and the finding outranks the drill."* A precondition was wrong. It was
+one the runbook's P1–P6 table did not contain, because nobody knew to look for it.
+
+**Sentinel sha256 (step 3), recorded and still on the box:**
+`643166269103a25cf41d34a26b75fd6ebba0a837bbcd7e0d2b5aba7106bcbe7c`
+at `/mnt/sys_drive/userdata/media/books/DRILL-SENTINEL.txt` (181 B).
+
+---
+
+## 2. R-203 — the defect that halted the drill
+
+**Measured, twice, on the live box.**
+
+| what | path | exists? |
+|---|---|---|
+| the app's live bind, where the customer's files actually land | `/mnt/sys_drive/userdata/media/books` | **YES** (the sentinel is here) |
+| the path the off-site capture set treats as the mandatory directory | `/mnt/sys_drive/felhom-data/userdata/media/books` | **NO** |
+
+The controller's own log, verbatim:
+
+```
+[WARN] [offbox] calibre-web: mandatory data path missing on disk, skipped from offsite:
+       /mnt/sys_drive/felhom-data/userdata/media/books
+[INFO] [offbox] backed up calibre-web (…/backups/primary/calibre-web, 0 mandatory path(s))
+[INFO] [offbox] backup OK: 3 app(s) backed up, 3 snapshot(s), 37s
+```
+
+**The run reported `ok`.** `last_status: ok`, `last_success` stamped, `snapshot_count: 3`. Nothing
+customer-visible, nothing hub-visible and nothing in any counter says the mandatory directory was
+dropped. This is `CLAUDE.md`'s recurring shape — *a path the customer thinks is protected is not in the
+snapshot* — and the code even has the right words for it in a WARN nobody reads.
+
+**The mechanism, from source.** `NamespaceRoot(drivePath, inGuestDrive)`
+(`appbackup/paths.go:28-33`) appends `felhom-data` **when the drive IS the system data path**
+(`m.namespaceRoot` = `NamespaceRoot(drivePath, drivePath != m.systemDataPath)`,
+`backup/backup.go:331`). The deploy-time bind does not: `${USERDATA_PATH}` is documented and computed
+as **`<HDD_PATH>/userdata`** (`stacks/classify_binds.go:14`). With `system_data_path: /mnt/sys_drive`
+and an app deployed at `HDD_PATH=/mnt/sys_drive`, those two produce different directories.
+
+**And the same compose file used BOTH roots.** From `docker inspect calibre-web`:
+
+```
+bind /mnt/sys_drive/felhom-data/userdata/import/calibre -> /cwa-book-ingest     ← felhom-data root
+bind /mnt/sys_drive/userdata/media/books               -> /calibre-library      ← NO felhom-data root
+```
+
+`${IMPORT_PATH}` resolved *with* the segment; `${USERDATA_PATH}` resolved *without* it. One deploy, one
+template, two roots.
+
+### What is measured and what is not — stated because the scope changes the fix
+
+- **MEASURED:** the two paths disagree; the mandatory directory is absent from the snapshot; the run
+  reports `ok`; the only signal is a container-log WARN.
+- **NOT ESTABLISHED:** whether `HDD_PATH=/mnt/sys_drive` is a *supported* choice. It was chosen because
+  demo-hp's only registered drive, `Felhom-Share`, is a NAS and was **correctly refused** as an app
+  namespace (R-108's `RefuseAsAppNamespace`, working as designed). `/mnt/sys_drive` was **accepted**
+  (HTTP 202) rather than refused.
+
+**Either branch is a defect, which is why this is filed regardless:**
+
+- if the system drive **is** a supported app namespace → the userdata path resolution is wrong for
+  every app deployed on it, and their mandatory directories are silently unprotected;
+- if it is **not** supported → the deploy accepted a namespace it should have refused, exactly as it
+  refused the NAS one call earlier, and the refusal that exists is not reaching this case.
+
+**What must NOT be concluded from this drill:** that off-site backups are broken generally. The two
+pre-existing apps (`opengist`, `privatebin`) declare **no** mandatory userdata paths — everything they
+own is in named volumes — so they are unaffected, and their snapshots are real.
+
+---
+
+## 3. Preconditions, each measured
+
+| # | Precondition | Result |
+|---|---|---|
+| **P1** | operator holds the recovery code | **PASS** — `R_DEMO-HP`, recorded by the operator in the DooPlex credentials file (`~/.config/credentials`). *The code itself appears nowhere in this record.* |
+| **P2** | demo-hp on agent v0.125.0 + controller v0.196.0 | **FAILED ON ARRIVAL, FIXED** — the box was on agent **v0.124.1**. Deployed the published v0.125.0 (sha `f7d8339b53d9…`, verified against the release) and controller v0.196.0. `age` present at `/usr/bin/age`. |
+| **P3** | the current `identity_blob` seals the repository under test | **PASS with a caveat that reshaped the drill** — `identity_blob` = 572 B, `restic_pw_sha256` = `8a9e33aa4da6…`. But **no repository existed under that key** (see §4). |
+| **P4** | a deliberate, verified whole-guest archive as rollback | **NOT TAKEN** — deliberately. It is only needed for the wipe, and the wipe did not happen. |
+| **P5** | demo-felhom untouched and healthy | **PASS** — not touched at any point. |
+| **P6** | space | **PASS** — `felhom-backup` 927 GB free, `local-lvm` 30.9 % used, guest 64 GB free. |
+
+---
+
+## 4. Step-by-step, with every observable
+
+### Step 1 — starting state (hub, read-only)
+
+`demo-hp-bb76ea`: `identity_blob` 572 B, `restic_pw_sha256` `8a9e33aa4da6769c5aea1831f87759e10930e2ec1dea0062576484e0598d080a`, created `2026-08-04T11:11:37Z`, `escrow_state: escrowed`.
+Report: `snapshot_count: 0`, `repo_size_bytes: 0`, **no `last_run`/`last_status` at all** — the shape of
+a controller that has never run an off-site backup in this lifetime.
+
+### Step 1b — the repository was ORPHANED, and this is the first live proof of a prediction
+
+The 2026-08-04 spike predicted from source — and could not measure — that a rebuilt box's next
+off-site run would hit a **third** outcome: neither reattaching the old snapshots nor silently starting
+a fresh history, but **refusing**. Measured here, twice over.
+
+**Read-only probe first** (`restic cat config` with the current key, writes nothing):
+
+```
+Fatal: wrong password or no key found
+```
+
+— the exact string `classifyResticProbe` maps to `"orphaned"`.
+
+**Then the real run, through the customer UI endpoint** (`POST /backup/offbox/run`):
+
+```
+[WARN] [offbox] offsite repo ORPHANED — remote holds backups written under a previous,
+       no-longer-available key; runs will skip until reset
+[WARN] [offbox] run skipped — offsite repo orphaned (card shown; awaiting reset)
+[INFO] Event pushed: offbox_repo_orphaned (warning) — A távoli mentési tároló elárvult …
+```
+
+`repo_state: "orphaned"`, `orphaned_at: 2026-08-04T12:37:30Z`, `last_status: "error"`, the orphan card
+rendered on `/backups/remote`, and the event reached the hub (HTTP 200).
+
+> **The system stopped and said so. It did not quietly start a new history over the old one.**
+> That closes R-193's open Q3 — and it is the good half of this month's story.
+
+**Why the repository was orphaned:** the 15 snapshots / 40.9 MB at `/home/felhom-repo` were written
+under key `8e03eddf9ff7…` before the 2026-08-03 rebuild. That key survives only in superseded escrow
+row **id 3**, which carries **`identity_blob` = NULL** — it was superseded at `2026-08-04 07:15:36`,
+**four hours before** hub v0.93.0 fixed the retention. Unrecoverable, permanently, with or without a
+recovery code.
+
+### Step 1c — the reset (operator-authorised)
+
+The orphan card's own reset, confirmed by the operator during the session:
+
+```
+[WARN] [offbox] resetting orphaned repo (operator-confirmed (claimed)):
+       move-aside /home/felhom-repo -> /home/felhom-repo.orphaned-20260804, then re-init
+[INFO] [offbox] orphaned repo reset complete — old history set aside at
+       /home/felhom-repo.orphaned-20260804 (move-aside, not deleted); fresh repo initialized
+[INFO] Event pushed: offbox_repo_reset (info)
+```
+
+**Nothing was deleted.** The reset path had never run in anger before; it works.
+
+### Steps 2–3 — the recovery code and the sentinel
+
+The operator ran the ceremonies for **both** boxes earlier the same day and saved the codes. Measured
+hub-side: the new escrow rows are stamped `2026-08-04T11:11:37Z` (demo-hp) and `11:13:06Z`
+(demo-felhom), and `restic_pw_sha256` is **unchanged** on both — so `SaveHostEscrow` correctly treated
+them as same-password re-ceremonies: **no superseded row, no `offsite_repo_key_changed`**. That is
+v0.93.0's Scenario E, live.
+
+**A file-leg app had to be deployed**, and this is a precondition the runbook did not anticipate:
+neither off-site-toggled app on the box (`opengist`, `privatebin`) has a restorable file leg — both
+keep everything in **named volumes**, which the off-site tier backs up as tars but the customer restore
+flow **never unpacks** (`offbox_reconstitute.go`). A sentinel in either would have been unrecoverable
+by design.
+
+`calibre-web` was chosen: it declares `userdata: media/books class: mandatory`, and it is a
+single-container app. Deployed through the real API (`POST /api/stacks/calibre-web/deploy`, HTTP 202),
+toggled for off-site, and a Tier-1 recovery unit captured
+(`Recovery unit captured for calibre-web → …/backups/primary/calibre-web`).
+
+Sentinel written, `sync`ed, hashed: **`643166269103a25c…`**, 181 B.
+
+### Step 4 — the pre-wipe off-site backup: **`ok`, and wrong**
+
+```
+last_run 2026-08-04T12:54:50Z · last_status ok · last_success stamped
+snapshot_count 3 · repo_size_bytes 30 636 · repo_state null
+```
+
+Three snapshots, three apps, 37 s — and **the sentinel is in none of them**, per §2.
+
+### Steps 5–11 — NOT RUN
+
+Step 5 (verify the pre-wipe snapshot contents) would have confirmed the absence a second way; it is
+moot given §2. Steps 6–11 (archive, wipe, reinstall, recover, install, restore, compare) were **not
+started**. The §7 STOP was never reached, because the drill failed its own precondition first.
+
+---
+
+## 5. Part 0 — shipped, tested, deployed (R-200's plumbing half)
+
+`--recover-offsite-install` (controller **v0.196.0**, commit `1b1366b`): same fetch → unseal → extract
+path as `--recover-offsite-check`, same STDIN discipline for R, but it **places** the recovered
+password via `InjectOffboxPassword`.
+
+- **Confirmation is a second invocation.** Without `--confirm-install` it prints both hashes and writes
+  nothing. A single interactive prompt would have had to share stdin with R.
+- **Three outcomes, named distinctly:** *installed* (no local password — the rebuilt-box shape),
+  *unchanged* (identical key already present, nothing written), *refused* (a DIFFERENT key present;
+  installing would clobber the key the current repository is encrypted under — exit 2, no force offered).
+- It re-reads the file after writing rather than trusting the call's return.
+
+**Tests + red-proof.** `go build && go vet && go test ./...` rc=0; `controller_gates.py --fast` OK.
+Removing the confirmation gate makes the dry run write the password and fails
+`TestRecoverAndInstall_InstallsOnABareBox` with *"the DRY RUN wrote the password"*. The R-persistence
+test carries a **positive control** — a planted copy of the code is found by the sweep, then removed
+and not found — because an absence check is worth only what its sensitivity is.
+
+**It was deployed to demo-hp and never exercised against a live recovery**, because the drill halted
+before step 9. Its unit proof stands; its live proof does not exist.
+
+---
+
+## 6. What this drill did and did not establish
+
+**Established, live, for the first time:**
+
+1. A rebuilt box's off-site repository is **orphaned and the run refuses** — the spike's predicted
+   third outcome, measured. It does not silently start a fresh history.
+2. The **orphan reset works**: move-aside, never delete, fresh repo initialised, event pushed.
+3. **A customer-declared mandatory data directory can be silently absent from the off-site snapshot
+   while the run reports `ok`** (R-203).
+4. demo-hp's pre-rebuild off-site history is **permanently unrecoverable** — its key was destroyed four
+   hours before the fix that would have kept it.
+
+**NOT established — and unchanged from before this session:**
+
+- **No file has ever been restored from an off-site backup after a wipe.** R-201's question is still
+  open, and its pass condition is unchanged.
+- Part 0's install path has never run against a live recovery.
+- The v0.93.0 `identity_blob` retention is **still unit-proven only** — nothing in this session
+  superseded a key, so nothing exercised it.
+
+---
+
+## 7. State left behind, and teardown
+
+**Deliberately not torn down** — this is evidence, and the box is better off than it was:
+
+| layer | state |
+|---|---|
+| the guest | `calibre-web` deployed and running, off-site-toggled, sentinel file in place. **Kept** as the drill fixture for the resumed run — it is the only app on either demo box with a restorable file leg. |
+| the off-site repository | fresh, working, 3 snapshots, 30 636 B. The old 15-snapshot history is **set aside**, not deleted, at `/home/felhom-repo.orphaned-20260804`. |
+| the host | agent v0.125.0, controller v0.196.0, `pvesm` unchanged apart from normal usage. |
+| the hub | `offbox_repo_orphaned` + `offbox_repo_reset` events recorded for `demo-hp`. No scratch customer records were created — **nothing was reinstalled**. |
+
+**The ~1.2 GB of previously-orphaned ciphertext was NOT deleted** — that act is ruled and still owed,
+and §8.3 of the runbook forbids riding it along with a drill. The reset added
+`/home/felhom-repo.orphaned-20260804` (≈41 MB) to what is set aside.
+
+**No `R` persisted anywhere** — the recovery code was never used in this session. It was read only to
+confirm the key exists in the credentials file; no unseal was performed on demo-hp.
+
+---
+
+## 7b. UPDATE, same day — the blocker is FIXED and the drill is ready to resume
+
+Controller **v0.197.0** shipped both halves of R-203:
+
+- **the paths agree.** On demo-hp the live bind moved from `/mnt/sys_drive/userdata/media/books` to
+  `/mnt/sys_drive/felhom-data/userdata/media/books` — the directory the capture set looks in. The
+  capture log went from `0 mandatory path(s)` to **`1 mandatory path(s)`**.
+- **`ok` means it.** A run that cannot capture a MANDATORY directory now reports **`incomplete`**,
+  names the app and the folders, and raises the operator digest — instead of `ok` with a warning
+  beside it.
+
+**And the sentinel is in the snapshot, listed by name:**
+
+```
+$ restic ls -l latest --tag calibre-web
+-rw-r--r--  1000  1000  181  2026-08-04 12:53:06  /mnt/sys_drive/felhom-data/userdata/media/books/DRILL-SENTINEL.txt
+```
+
+sha256 **`643166269103a25cf41d34a26b75fd6ebba0a837bbcd7e0d2b5aba7106bcbe7c`** — byte-identical to §1,
+verified after the fix's migration moved the file to the corrected directory.
+
+**One live check could NOT be reproduced, and is recorded rather than claimed.** Hiding the books
+directory to watch the `incomplete` verdict fire on hardware did not work: the running container's
+bind mount **recreated** the directory, so `os.Stat` succeeded and there was no gap. That is itself
+worth knowing — a bind-mounted directory cannot easily be "missing" while its app runs, so the
+mandatory-gap condition arises in practice when the path resolves somewhere the app never binds (the
+R-203 case), not when a live app's own directory vanishes. The verdict is proven by the run-level
+test, which drives the real `RunOffboxBackup` and asserts both `incomplete` and the operator signal.
+The fixture was restored and the sentinel re-verified at the same hash.
+
+## 8. To resume the drill
+
+1. **Fix or scope R-203.** Until the mandatory userdata path lands in the snapshot, no sentinel can
+   survive a wipe and the drill cannot reach its verdict.
+2. Re-run steps 4–5 and confirm the sentinel IS in the snapshot — by listing it, not by a green status.
+3. Then P4 (the deliberate archive), the §7 STOP, and steps 6–11 as written.
+
+Everything else is already in place: the code, the versions, the recovery code, the working repository,
+the file-leg app and the sentinel.

documentation/audits/RECON-offsite-dr-chain-2026-08-04.md

@@ -0,0 +1,531 @@
+# RECON — can a customer actually get their backups back?
+
+**Date:** 2026-08-04 · **Items:** R-193, R-192 (+ new: R-198…R-201) · **Class:** recon — **no production code**
+**Session scope:** read-only everywhere. Nothing built, deployed, wiped, rotated, re-issued or deleted.
+
+> **Headline, before the detail.** The chain is **not** assembled, and the reason is worse than "two
+> links are manual". **The hub's superseded-escrow retention — the feature built for exactly this
+> incident — does not retain the offsite repository password.** `host_escrow_superseded` has no
+> `identity_blob` column, and `demoteCurrentEscrowTx` copies only the K-escrow blob. So the last copy
+> of a repo password is destroyed by the **next escrow ceremony** — the very act the system tells the
+> customer to perform when their escrow goes `pending` after a rebuild. The recovery window closes
+> because the remedy closes it.
+
+---
+
+## 1. Confirmed baselines (re-read on arrival)
+
+| Repo | `main` @ commit | tree | Version |
+|------|-----------------|------|---------|
+| `felhom-agent` | `856a127cd60b0be797867c162f0ae49c7b64993e` | clean | v0.124.1 |
+| `felhom-controller` | `0887fd676d608e7c103fa13b3e87536ad2481811` | clean | v0.194.0 |
+| `felhom.eu` | `d26f49ad680e2be2ce2888fd58dff7dee886e4ad` | clean | hub v0.92.0 |
+
+All three match the prompt's cited commits exactly. `git status --porcelain` empty in each.
+
+**Highest register ID: R-197** (`OPEN-ITEMS.md` R-196/R-197 were minted by yesterday's spike;
+R-194/R-195 by the R-190 arc). Grepped across `OPEN-ITEMS.md`, `ROADMAP.md`, `STATUS.md`,
+`CONTEXT.md`. New items in this session therefore start at **R-198**.
+
+---
+
+## 2. Instruments — what each can and cannot answer
+
+| # | Instrument | Exact read | Known limit |
+|---|---|---|---|
+| I1 | source | the three repos at the commits above | the authority; every claim below cites a file:line |
+| I2 | hub SQLite snapshot | `kubectl -n felhom-system exec pod/hub-6775889d95-gc8sk -- cat /data/{hub.db,hub.db-wal,hub.db-shm}` → local `sqlite3` | the `-wal` is load-bearing (copied; `PRAGMA integrity_check` → `ok`) |
+| I2-fresh | freshness proof | `MAX(host_reports.received_at)` = `2026-08-04 10:00:40`, `datetime('now')` = `2026-08-04 10:03:48` | **positive observable**: newest row 3 m 08 s old, not "the query did not error" |
+| I3 | live filesystem, demo-felhom guest 9201 | `ssh felhom-pve "pct exec 9201 -- ls -la --time-style=full-iso …"` | read-only `ls`; mtimes only, no file contents read |
+| I4 | caller census | `grep -rn <symbol> --include=*.go` across all four repos | proves absence of a caller in Go; templates checked separately |
+
+**Not used, deliberately:** no ceremony, no re-issue, no rotation, no `selftest=identity-consume`, no
+write of any kind, nothing touched on the storage endpoint.
+
+---
+
+## 3. Q1 — the chain, link by link
+
+From a dead node to an open repository. **Status vocabulary as specified.**
+
+| # | Link | Component / function | Who invokes it | Status |
+|---|---|---|---|---|
+| 1 | repo password is minted | controller `Manager.WriteOffboxSecrets` → mint at `internal/backup/offbox.go:392-401`; generator `:405` | `ApplyOffsiteTarget` (`offbox.go:474`) ← `ConfigureOffbox` ← the apply-bridge (`offsiteapply.go:221,256`) | **PROVEN-LIVE** — fires on every fresh data dir; measured twice this week (I2 hash change, I3 mtime) |
+| 2 | password staged to the agent | controller `PushOffboxPasswordForEscrow` (`offbox.go:530`) → agent `POST /escrow/stage-secret` (`localapi/server.go:518`) → `escrow.StagedResticPasswordPath()` = `/var/lib/felhom-agent/escrow-stage/restic_repo_password`, 0600 (`identity.go:44-47`) | the controller enable/ceremony path | **PROVEN-LIVE** |
+| 3 | sealed under R at the ceremony | `AttachResticPassword` (`identity.go:73`) → `WrapIdentityBundle` → `WrapIdentity` (`identity.go:114`, `age -p -a`, scrypt + ChaCha20-Poly1305) | agent `--selftest=escrow-create`, driven by the controller wizard (`/escrow/ceremony`, `localapi/server.go:525-527`) | **PROVEN-LIVE** — customer-facing wizard first live firing 2026-07-18 (`00-capability-map.md`, escrow row) |
+| 4 | stored by the hub | `PUT /api/v1/hosts/<id>/escrow` (`api/handler.go:239`, `handleHostEscrowPut :1104`) → `host_escrow.identity_blob` | agent `main.go:2937` | **PROVEN-LIVE** — measured: `identity_blob` present, 572 B, for **both** hosts (I2) |
+| 5 | **old blob retained on supersession** | `demoteCurrentEscrowTx` (`store/store.go:2547-2556`) → `host_escrow_superseded` | `SaveHostEscrow`, `DeleteHost` | **BROKEN FOR THIS PURPOSE.** The INSERT copies `host_id, blob, key_fingerprint, posture, created_at, restic_pw_sha256` — **not `identity_blob`**, and the table has **no such column** (`PRAGMA table_info`, I2). The K-escrow survives; **the restic repo password does not** |
+| 6 | served back to a re-enrolling box | hub `handleReEnroll` (`api/dr.go:101`) and `handleGetRestoreDirective` (`:155`), returning `identity_escrow_b64`; recovery-mode gated, armed with the **global operator key** (`:34`) | **NOBODY.** I4: zero callers in `felhom-agent` (no `ReEnroll` symbol, no `/re-enroll` or `/restore-directive` string anywhere in the agent's hub client, `internal/hub/client.go` reaches only `desired-state`, `wg`, `pbs/consume-token`, `jobs`); zero in the hub UI (no template mentions them); zero in `scripts/` or any runbook | **implemented-never-exercised, and CLIENTLESS.** The only documented retrieval is by hand: `sqlite3 "SELECT writefile('/root/idblob', identity_blob) FROM host_escrow …"` on a `kubectl cp`-ed `hub.db` (project memory, 2026-07-04 S5 prep) |
+| 7 | unsealed with R | `UnwrapIdentityBundle` (`identity.go:172`) → `UnwrapIdentity` (`:139`) | **`runSelftestIdentityConsume` only** (`cmd/felhom-agent/main.go:2845-2900`), reading R from `FELHOM_RECOVERY_CODE`. I4: no other production caller | **manual-only — the only caller is a `--selftest` mode** |
+| 8 | the repo password extracted | none. The selftest writes the **whole bundle JSON** to `-keydest` 0600 (`main.go:2872`) and prints *"identity recovered (tunnel_token + pbs_token)"* — it does not mention, extract or route `restic_repo_password` at all | a human reads the JSON | **missing** |
+| 9 | placed on the rebuilt controller | `InjectOffboxPassword` (`offbox.go:541`) | one caller: `offboxInjectPasswordHandler` (`web/offbox_handlers.go:189`) at `POST /backup/offbox/inject-password` (`web/server.go:510`). **I4: no template in the repo contains that path or any form posting to it** | **manual-only, and UI-unreachable** — the handler exists, the form does not. A person must POST it by hand with a valid session cookie + CSRF token |
+| 10 | the existing repo opens | `ensureOffboxRepo` (`offbox.go:653`): `restic cat config` → success clears `RepoState` | the scheduled/manual offbox run | **implemented-never-exercised** on a *recovered* password (exercised constantly on a live one) |
+| 11 | a backup or restore proves it | — | — | **never** |
+
+### The chain in one sentence
+
+Links 1–4 are real and proven. **Link 5 silently drops the payload.** Links 6–9 are each individually
+manual, and link 6 has no client at all. Link 11 has never happened.
+
+---
+
+## 4. Q2 — where the chain stops today
+
+**The first link that is not automatic is #5, and it is not merely manual — it is lossy.**
+
+Two distinct stopping points, and the order matters:
+
+**(a) If the box has NOT re-escrowed since the rebuild** — the current `host_escrow.identity_blob`
+still seals the *old* password. Recovery is possible in principle, and a person must:
+
+1. Arm recovery mode on the hub (`PUT /api/v1/admin/hosts/<id>/recovery-mode`, global operator key) —
+   or skip it entirely, because nothing consumes the endpoint it gates;
+2. `kubectl cp` / `exec cat` the hub DB and `sqlite3 … writefile('/root/idblob', identity_blob)`;
+3. copy the blob to a machine with the agent binary and `age`;
+4. `FELHOM_RECOVERY_CODE=… felhom-agent --selftest=identity-consume -blob /root/idblob -keydest /root/bundle.json`;
+5. open `bundle.json` by hand and read `restic_repo_password` out of it;
+6. log in to the customer's dashboard and `POST /backup/offbox/inject-password` **by hand** (curl with
+   session + CSRF) — there is no form;
+7. re-run the apply so `WriteOffboxSecrets` finds the file present and keeps it;
+8. trigger a run and hope `restic cat config` succeeds.
+
+Eight manual steps, five of them requiring operator-tier access to the hub, one requiring the customer
+to be present with R, and one (step 6) with no user interface at all.
+
+**(b) If the box HAS re-escrowed** — which is what the system pushes the customer to do, because a
+rebuilt box lands in `EscrowState: pending`, offsite runs are blocked (`OffboxRunnable`,
+`offbox.go:570`), and the card says *„Helyreállítási kód szükséges"* — then the old `identity_blob`
+has been **overwritten** by `SaveHostEscrow` and **not carried** into the superseded row. At that point
+**no procedure exists**, with or without R, at any level of effort. The password is gone.
+
+Both demo boxes are in state (b) as of 2026-08-04 07:15:36 (demo-hp) and 07:20:08 (demo-felhom) — I2,
+`host_escrow.updated_at`. This is an **independent, and much stronger, reason** the 51 orphaned
+snapshots are unrecoverable than "nobody kept the recovery codes". Keeping R would not have helped.
+
+---
+
+## 5. Q3 — can a rebuilt controller know it should recover?
+
+### RULING: **YES — the hub already tells it, on every single report, and the controller throws the answer away.**
+
+**What the box knows at the mint decision.** `WriteOffboxSecrets` is called from
+`ApplyOffsiteTarget`, driven by the apply-bridge with the hub's offsite **descriptor**, whose fields
+are `host, user, port, repo_path, host_fingerprint, quota_gb` (`offsiteapply.go:198-204`). **Nothing
+about escrow is in scope at that point.** Locally, the only signal is the one the mint branch already
+reads: `os.Stat(m.offboxPwPath())` returning `IsNotExist` — which *is* the fresh-data-dir signal, at
+exactly the right line, but cannot distinguish "never configured" from "rebuilt".
+
+**What the box is nonetheless already told.** Every report ACK carries an `escrow` object
+(`api/handler.go:504-510`), consumed as `report.EscrowStatus`
+(`internal/report/escrow_confirm.go:19-23`):
+
+```go
+type EscrowStatus struct {
+	IdentityBlobPresent bool   `json:"identity_blob_present"`
+	ResticPwSHA256      string `json:"restic_pw_sha256"`
+	CreatedAt           string `json:"created_at"`
+}
+```
+
+That is precisely *"the hub has a blob, and here is the hash of the password it covers"* — arriving
+every report cycle, on a channel that already exists, with no new API and no new secret.
+
+**And it is discarded.** `EscrowAutoConfirmer.Reconcile` (`escrow_confirm.go:75-84`) returns
+immediately unless `Pending()` or `Escrowed()` — both of which require `OffboxConfigured()` **and** a
+non-nil target (`cmd/controller/main.go:580-590`). On a freshly rebuilt box, before the offsite apply,
+there is no target, so the status is dropped on the floor. It is never persisted anywhere: I4 finds
+exactly **one** reference to the ACK field in the whole controller
+(`cmd/controller/main.go:641: escrowConfirmer.Reconcile(resp.Escrow)`).
+
+**Therefore a fail-closed refusal IS implementable**, with no new hub endpoint:
+
+- persist the last-seen ACK escrow status alongside settings (the precedent exists — `ClaimSync`
+  caches the hub's claim-code state into `settings.json` idempotently by generation,
+  `internal/report/claim_sync.go:39-53`, and it does so *set-only, never cleared on hub silence*,
+  which is the correct shape here too);
+- at the mint branch, refuse when the password file is absent **and** the cached status says
+  `identity_blob_present && restic_pw_sha256 != ""`.
+
+**One ordering caveat, stated rather than assumed.** On a truly fresh box the apply can race the first
+ACK. Measured on demo-hp: first post-rebuild report `2026-08-03 06:12:19`, offsite apply
+`2026-08-04 07:12` — the ACK preceded the apply by 25 hours, but that gap was an incident, not a
+design. A fail-closed rule handles the race correctly by construction: refuse to mint until the escrow
+status is known, and retry next cycle (the apply-bridge is already built to retry and to persist
+nothing on a partial failure, `offsiteapply.go:1-6`).
+
+**Corroborating measurement that the data dir really is fresh on a rebuild** (I3, demo-felhom guest
+9201, `/var/lib/docker/volumes/felhom-controller-data/_data/data/`):
+
+| file | mtime |
+|---|---|
+| the data directory's parent | `2026-08-03 07:17:25` |
+| `encryption.key` (32 B) | `2026-08-03 07:17:26` |
+| `offbox/` | `2026-08-03 07:18:02` |
+| `offbox/repo_password` (64 B) | `2026-08-03 07:18:02` |
+
+The volume, the app-secret encryption key and the repo password are all newborn, 36 seconds apart.
+Corroborated hub-side (I2): `json_extract(report_json,'$.claimed')` reads **1 → 0 → 1** across the
+rebuild on both boxes — demo-hp `0` in 5 reports `2026-08-03 06:12:19`–`06:21:03`; demo-felhom `0` in 4
+reports `07:17:54`–`07:19:46`. `Claimed` is set only by `SetClaimed()`, whose sole caller is the claim
+submit handler (`web/claim.go:349`), so the return to `1` was a **human re-claim**, not automation.
+
+> **Note, not acted on:** a fresh `encryption.key` means every pre-rebuild `app.yaml` `ENC:` secret is
+> undecryptable — the C-1 circular dependency in `_recovery-inventory-2026-07-28.md` fired on both
+> demo boxes this week. That is outside R-193's scope and is recorded as an observation (§13).
+
+---
+
+## 6. Q4 — what the stale-escrow detector actually covers
+
+### RULING: **the prompt's premise conflates two different mechanisms. `escrow_confirm.go` DOES fire on a mint. But it fires into a log file, and the remedy it prescribes destroys the last copy of the old key.**
+
+**The conflation, stated because it matters.** Yesterday's spike's sentence *"wired to the one path
+that doesn't change the key and absent from the one that does"* is about the **hub's `escrow_stale`
+event**, emitted by `ReissueCredentials` (`hub/internal/offsite/offsite.go:198-201`) — that is R-196,
+and it stands. It is **not** about `controller/internal/report/escrow_confirm.go`. Two different
+detectors; only one of them was ever claimed to be misplaced.
+
+**What `escrow_confirm.go` does on a mint, from source.** After a rebuild the apply creates a target
+with `EscrowState: "pending"` (the `cur == nil` fallthrough, `offbox.go:482-494`). So `Pending()` is
+true, the pending branch runs (`escrow_confirm.go:86-107`), the hub's hash (sealing the OLD password)
+is compared against `LocalHash()` (the NEW minted one), they differ, and it logs:
+
+```
+[WARN] [escrow-confirm] the hub's escrow blob does not cover the CURRENT repo password
+(hub hash %.12s… != local %.12s…) — run the escrow ceremony … staying pending
+```
+
+**So: confirmed, it fires.** The detector is not absent from the mint path.
+
+**Three limits, each of which matters more than the fact that it fires.**
+
+1. **It is a controller-local log line.** No event, no e-mail, no hub-side signal. `StaleBlob()` — the
+   flag the web card renders — is set **only** by the escrowed branch (`reconcileEscrowed`,
+   `:130-165`); the pending branch sets no display flag. The operator learns nothing.
+2. **Its prescription is the destructive act.** *"run the escrow ceremony"* → a new ceremony →
+   `SaveHostEscrow` overwrites `host_escrow.identity_blob` → `demoteCurrentEscrowTx` does not carry it
+   → **the old repo password ceases to exist**. The detector correctly identifies the situation and
+   then tells the customer to close the door.
+3. **It cannot distinguish the two situations that need different answers** — "you re-keyed
+   deliberately" and "your machine was rebuilt and your history is about to be orphaned" produce the
+   identical warning.
+
+So: **less needs building than the prompt feared for the *detection*, and more needs building for the
+*consequence*.** The comparison exists; what does not exist is a path from that comparison to either a
+signal or a recovery.
+
+---
+
+## 7. Q5 — is the superseded-escrow retention real, and what does it retain?
+
+### RULING: **the retention is real, it retains the wrong blob, and nothing can read it back.**
+
+**What it retains — source and live, agreeing.** `demoteCurrentEscrowTx` (`store/store.go:2547-2556`):
+
+```sql
+INSERT INTO host_escrow_superseded (host_id, blob, key_fingerprint, posture, created_at, restic_pw_sha256, superseded_at)
+SELECT host_id, blob, key_fingerprint, posture, created_at, COALESCE(restic_pw_sha256, ''), datetime('now')
+FROM host_escrow WHERE host_id = ?
+```
+
+`identity_blob` is not in the column list. Live schema (I2, `PRAGMA table_info`):
+
+| table | columns |
+|---|---|
+| `host_escrow` | `host_id, blob, key_fingerprint, posture, created_at, updated_at, **identity_blob**, directive_json, restic_pw_sha256, stale_at` |
+| `host_escrow_superseded` | `id, host_id, blob, key_fingerprint, posture, created_at, restic_pw_sha256, superseded_at` |
+
+**`blob` is the K-escrow — the PBS datastore encryption key**, wrapped by the PBS-native scrypt path.
+**`identity_blob` is the age-wrapped `IdentityBundle`** — the one carrying `ResticRepoPassword`
+(`felhom-agent/internal/escrow/identity.go:34-39`), plus `TunnelToken`, `PBSToken` and
+`WGPrivateKey`. Live sizes confirm they are different objects: both hosts' current rows hold
+`blob` = 383 B **and** `identity_blob` = 572 B; both superseded rows hold `blob` = 383 B and nothing
+else.
+
+**So, exactly what a customer with the matching OLD recovery code could recover from a retained blob:**
+
+- ✅ the PBS datastore encryption key **K** — i.e. the Plane-2 whole-guest offsite ciphertext;
+- ❌ **not** the restic repository password — the Tier-3 offsite app-data history;
+- ❌ not the tunnel token, not the PBS access token, not the WireGuard private key.
+
+**And there is no read path.** I4: `ListSupersededEscrow` (`store/store.go:2617`) has **zero
+production callers** — only two tests. The only production surface is `CountSupersededEscrow`, used
+for a number on the host page (`web/hosts.go:573`) and a log line (`api/handler.go:1151`). The schema
+comment says so itself: *"The ACK and restore-serving read `host_escrow` (the CURRENT row) — never
+this table."* Retrieval is `sqlite3` on the hub pod, by hand, by the operator.
+
+**The claim this contradicts.** `00-capability-map.md` (escrow-ceremony row) records hub v0.60.0 as:
+
+> *"a re-escrow with a DIFFERENT sealed passphrase no longer destroys the old blob — the hub RETAINS
+> it (`host_escrow_superseded`), so a previous passphrase stays recoverable with its recovery code
+> (turns the reinstall-orphan incident from 'history destroyed' into 'history recoverable')."*
+
+The same claim appears in the schema comment (`store.go:370-375`) and, in Hungarian, on the customer's
+own orphan card (`controller/internal/web/templates/backups_remote.html:66,69`):
+
+> *„A meglévő mentések nem sérültek — a hozzájuk tartozó helyreállítási kóddal később visszaállíthatók
+> lehetnek."* · *„A régi előzmény félretéve marad (nem törlődik), és a hozzá tartozó helyreállítási
+> kóddal később visszaállítható lehet."*
+
+**For the offsite restic repository — the incident it names — that is false.** The retained blob does
+not contain the key that opens it. This is the **ninth** entry in `CLAUDE.md`'s table of comments
+asserting an invariant the code does not provide, and the first that is also **customer-facing copy**.
+→ **R-198**, and the capability-map correction in §9.
+
+---
+
+## 8. Q6 — has any part of this ever been exercised?
+
+**Method:** read every audit, validation doc, findings file and REPORT that mentions escrow consume,
+identity-consume, re-enroll or inject-password; then check what the cited evidence actually covers.
+
+### Proven
+
+| what | evidence | scope of the proof |
+|---|---|---|
+| ceremony (create + seal + upload) | `00-capability-map.md` escrow row; `tests/VALIDATION-n100-rehearsal-2026-07-18.md` S6 — customer drove the wizard, R claimed one-shot 16:56:39, auto-confirmed 16:56:41 | the **create** side only |
+| identity-bundle round-trip with a real R on a secret-less box | `tests/slice10d-identity-restore-spike-findings.md` §1, 2026-06-10 — wrap→unwrap byte-identical (`sha256` match); wrong-R **fails closed**, no plaintext emitted | **PRE-FORK-4.** The bundle then was `{tunnel_token, pbs_token}`. `ResticRepoPassword` was added in agent **v0.77.0 on 2026-07-09** (`felhom-agent/CHANGELOG.md:2344`) — a month later. **The proof does not cover the field this whole question is about** |
+| `ResticRepoPassword` survives the wrap/unwrap | agent v0.77.0 CHANGELOG: *"`IdentityBundle` round-trip carries `ResticRepoPassword` byte-exact + not-in-blob + wrong-R fails closed"* | **unit test only** |
+| `InjectOffboxPassword` places and refuses correctly | `controller/internal/backup/offbox_test.go:952-990`, incl. the companion showing a fresh mint without inject | **unit test only** |
+
+### Never exercised — named in those words
+
+- **A blob has never been served to a box.** Link 6 has no client (§3).
+- **A fork-4 bundle has never been unsealed with a real R outside a unit test.** The only production
+  caller is a `--selftest` mode; the one live consume ever prepared (S5 Part 4-B, 2026-07-04) was
+  explicitly **deferred and never run** — its own resume note still reads *"DEFERRED 2026-07-04, not
+  run"* (project memory; `06-offsite-connectivity.md:327` marks the destructive drill operator-gated).
+- **A recovered repo password has never been injected into a controller.**
+- **An existing offsite repository has never been reopened with a recovered password.**
+- **No restore of any kind has ever been performed from a recovered secret.**
+
+`_recovery-inventory-2026-07-28.md` already said most of this and said it accurately — A.2.7 records
+*"the **consume** side is proven at spike level … the destructive in-place drill is explicitly
+operator-gated and unrun"*, and C.1 row 1 says *"escrow **creation** proven live; **consume never
+exercised destructively**"*. **That document is right and this session confirms it.** What it did not
+catch — because it was written from `host_escrow`, not from the supersession path — is Q5.
+
+**Capability-map correction owed:** not for a "recovery proven" claim (no row makes one), but for the
+**retention** claim quoted in §7. See §9.
+
+---
+
+## 9. Q7 — the operator's ruled shape, assessed row by row
+
+> **Operator ruling, 2026-08-04, verbatim:** *If a node is a fresh install AND the hub has a recovery
+> blob, then the controller should yell that recovery is available, and provide a form for the customer
+> to enter the recovery key. After unlocking the blob, the controller should show what will be
+> recovered before proceeding.*
+
+| The ruling requires | Exists? | What it needs, and what it costs |
+|---|---|---|
+| the controller knows it is a **fresh install** | **YES, at the right line** | `os.Stat(m.offboxPwPath())` → `IsNotExist` is already the mint branch (`offbox.go:392`). It is trustworthy as *"this controller has no repo password"*, which is the only fact the decision needs; it does **not** by itself distinguish first-ever install from rebuild — that is what the next row supplies. Corroborators exist but are weaker: `Claimed` is set-only and was measured re-set by a human within minutes (§5); a fresh `encryption.key` is a stronger signal but is not currently read by anything. **Cost: zero — the branch already exists.** |
+| the controller knows **the hub has a blob** | **YES on the wire, NO in the code** | `EscrowStatus{IdentityBlobPresent, ResticPwSHA256, CreatedAt}` arrives on every report ACK and is discarded when no offbox target exists (§5). **Needs:** persist it (the `ClaimSync` set-only-by-generation pattern is the precedent, `claim_sync.go:39-53`). **Cost: small — controller only, one settings field, one reconciler, no hub change.** |
+| it can **yell** rather than silently mint | **NO** | The refusal belongs **inside `WriteOffboxSecrets`, at the mint branch** — the one place that decides. Returning a new sentinel (`ErrOffboxRecoveryAvailable`, sibling of `ErrOffboxOrphaned`) propagates through `ApplyOffsiteTarget` → `ConfigureOffbox` → the apply-bridge, which already treats a failed configure as "persist nothing, retry next cycle" (`offsiteapply.go:5-6, 256`) — so a refusal is **fail-safe by construction**, not a new failure mode. Meanwhile the customer sees a card on `/backups/remote` (the orphan card at `backups_remote.html:60-70` is the exact precedent) and offsite runs stay blocked, which they already are (`OffboxRunnable`). **Cost: small-medium — controller only.** **Design hazard to name:** a first-ever install of a *re-used customer record* would also trip this; the refusal must be dismissible ("start fresh anyway"), and dismissal must be the act that mints. |
+| a **form takes R** | **NO — and R is not a password** | The controller UI **emits** R (`backups_escrow.html:235-236`, `/backup/escrow/…/claim` returns it once) and has **never accepted** one. The only thing resembling an entry point is `POST /backup/offbox/inject-password`, which takes a **64-hex repo password** (`offboxRepoPwPattern`, `offbox.go:543`) — a different secret at a different layer — and **has no form in any template** (§3 link 9). **Cost: small for the form; the hard part is the next row.** |
+| the blob is **unsealed** with R | **NO — and there is a real boundary** | Unsealing is `age -d`. `ageBinary = "/usr/bin/age"` (`identity.go:24`) and the code live in the **agent**. The **controller image does not ship `age`** (`controller/Dockerfile:56-70` — restic, openssh-client, sshpass, sqlite3, docker-cli; no age) and must not: the agent is the privileged tier by design. **The crossing already has a transport and a precedent:** the pinned local API carries `POST /escrow/ceremony` + `GET /escrow/ceremony/status` + `POST /escrow/ceremony/claim` (`localapi/server.go:524-527`) — i.e. the agent **already** handles R server-side for the create direction and hands the result back exactly once. **Needs: one new agent local-API endpoint** (`POST /escrow/unseal`: blob + R in, the single `restic_repo_password` field out, R never logged, R never persisted) — the exact mirror of the claim endpoint. **Plus** a way for the blob to reach the agent: the agent's hub client has no escrow **GET** (only the `PUT` at `main.go:2937`), and `dr.go`'s re-enroll is the wrong tool (it rotates the host API key and needs operator-armed recovery mode). **Needs: a second small hub endpoint** — `GET /hosts/<id>/escrow`, per-host-key authed, serving opaque bytes to the box that owns them. Zero-knowledge is untouched: the hub still never decrypts, and the bytes are useless without R. **Cost: medium — three repos, two new endpoints, both narrow.** |
+| **"show what will be recovered" before proceeding** | **the primitive EXISTS; the flow does not** | `restic snapshots --json` + `restic stats --mode raw-data --json` are already how the box counts snapshots and sizes the repo (`offboxRecordStats`, `offbox.go:1234-1265`). Both are **read-only against the repository** — nothing is written, no lock is taken, no `init`. So a preview can show: **snapshot count, per-snapshot time, host, tags (= app names), paths, and the deduplicated repo size** — all before committing. **Committing requires only that the recovered password be written to `offboxPwPath()`**, and even that is reversible in principle. **The clean shape:** unseal → hold the recovered password **in memory** → run the two read-only probes against the existing repo → render → and only on the customer's confirmation call `InjectOffboxPassword`. **Cost: small — one handler, one template, no new restic capability.** **Caveat, measured not assumed:** `offboxRecordStats` today also persists `RepoSizeBytes` into settings as a side effect; a preview path must not reuse it verbatim. |
+| **proceeding** actually recovers | **NO end-to-end** | `InjectOffboxPassword` → next `WriteOffboxSecrets` keeps the file → `ensureOffboxRepo` probes and clears `RepoState`. Every piece is implemented; **none has ever run against a recovered password** (§8). **Cost: zero new code, but it is precisely what the drill in §10 has to prove.** |
+
+### Things the ruling implies that do not exist, priced
+
+| missing piece | cost | note |
+|---|---|---|
+| persisted ACK escrow status + fail-closed mint refusal | **S** | controller only; the `ClaimSync` pattern applies directly |
+| the "recovery is available" card + dismissal | **S** | one template, one flag; the orphan card is the template precedent |
+| a customer-facing **recovery-code** form (distinct from the password form) | **S** | the password form's handler exists but has no UI; both need building |
+| agent local-API `POST /escrow/unseal` | **M** | new privileged surface; mirror of `/escrow/ceremony/claim` |
+| hub `GET /hosts/<id>/escrow` (opaque, per-host key) | **S** | new read surface on the hub; zero-knowledge preserved |
+| preview handler + template driven by read-only restic probes | **S** | primitive already exists |
+| **carrying `identity_blob` into `host_escrow_superseded`** | **S** | one column, one `SELECT` list — **and without it the whole feature protects only boxes that have not yet re-escrowed** (→ R-198). This is a prerequisite, not a nice-to-have |
+
+### The security question the ruling raises — stated, not answered
+
+**What gates the surface today.** The controller dashboard is behind `RequireAuth`
+(`internal/web/auth.go:52`): a bcrypt password (`effectivePasswordHash()` — customer-set in
+`settings.json`, else operator-provisioned in `controller.yaml`), a 7-day session cookie, 5 failed
+logins per minute then lockout, and `CsrfProtect` on every mutating route. The claim gate serves only
+the claim page on an unclaimed box. So a recovery form would sit **behind the customer's dashboard
+password**, on a controller reachable at whatever address that box's dashboard is reachable at.
+
+**What a preview would expose to someone who guesses or steals R.** Not file contents — but the
+snapshot list is not nothing: **backup dates and cadence, the app names (restic tags), the paths, and
+the repository size.** That is a map of the household's digital life, and it is precisely the metadata
+the zero-knowledge posture otherwise keeps out of everyone's hands including the operator's.
+
+**Three specifics the operator should weigh, without this session choosing:**
+
+1. **Guessing R is not the threat; holding the dashboard password is.** R is ten EFF words
+   (~129 bits, `00-capability-map.md` agent v0.93.0 note) and `age`'s scrypt is deliberately slow —
+   online guessing is not a route. But **anyone with the dashboard password can already restore live
+   offsite data** (`_recovery-inventory-2026-07-28.md` A.1.3–A.1.6). The marginal new exposure is
+   confined to the *orphaned/previous* repository, which today nobody can read at all.
+2. **The form is an oracle.** A correct/incorrect R answer distinguishable in the UI turns the
+   dashboard into a verification service for a stolen recovery code. `UnwrapIdentity` already
+   fails closed with a single generic error (`identity.go:157`) — the UI must not do better than that,
+   and should rate-limit like the login path does.
+3. **R will pass through the agent.** Whatever the form does, R crosses the local API to a
+   root-adjacent daemon. The ceremony already does this in the other direction, so the precedent is
+   set — but it means R is momentarily present in agent process memory on the Proxmox host, which is
+   the same host D6 says the operator has root on. **This is the same trade as spike option (c), in a
+   smaller and time-bounded form**, and it should be decided alongside it rather than separately.
+
+---
+
+## 10. Q8 — the drill design
+
+**Why a deliberate cycle.** No recovery code exists for either demo box, and — per §7 — even one would
+no longer open the old repositories. The proof therefore requires creating R, keeping it, and walking
+the full circle. **This section is the design; the drill is not run in this session.**
+
+### Target and preconditions
+
+| | |
+|---|---|
+| **Box** | **`demo-hp`** (HP t740, node `felhom-host`, host `demo-hp-bb76ea`). Tier 0 — disposable, and the designated drill host by the 2026-07-25 operator ruling. **Not demo-felhom** — it is the N100 that also carries the ISO/rehearsal lineage; leave one demo box in a known-good state throughout |
+| **Guest** | the customer guest on that host (rebuild target) |
+| **Data** | one deliberately identifiable app + a sentinel file whose content is a known string and whose sha256 is recorded before the wipe |
+| **Prereq** | the box is healthy, offsite target configured and `escrow_state: escrowed`, and its **current** `host_escrow.identity_blob` seals the password of the repo that will be tested |
+| **Duration** | ~3–4 h wall clock, of which ~45 min is operator-attended (steps 2, 5, 8, 9) |
+| **Blocking dependency** | **steps 8–9 cannot be executed as designed today** — links 6, 8 and 9 of the chain do not exist as automation. The drill as written uses the manual substitutes (§4a) and **that is the point**: it measures the *existing* chain honestly before anything is built |
+
+### The cycle
+
+| # | Step | Who | Observable that must be recorded |
+|---|---|---|---|
+| 1 | Record the starting state: `host_escrow.restic_pw_sha256`, `identity_blob` length, `$.offsite.snapshot_count`, `repo_size_bytes` | CC | a hash and two counters, from the hub DB |
+| 2 | **Run the escrow ceremony through the customer wizard and KEEP R** | **HUMAN** | R written to the operator's password manager under a named entry; **its storage location recorded in the drill report — the code itself never** |
+| 3 | Deploy an app, write the sentinel file, record its sha256 | CC | sha256 + path |
+| 4 | Run an off-site backup through the real UI endpoint; wait for completion | CC | `$.offsite.last_status = ok`, `snapshot_count` increased by ≥1, `last_success` stamped |
+| 5 | **Verify the snapshot from outside**: `restic snapshots --json` against the repo | CC | snapshot id + tag + timestamp — the pre-wipe truth to compare against |
+| 6 | **Wipe**: destroy the guest and reinstall it (the same procedure that produced the 2026-08-03 rebuild) | **HUMAN** decision, CC executes | new `controller_started`; `claimed` drops to 0; a new `encryption.key` mtime |
+| 7 | Let the box come back and reach the mint. **Do NOT run a new ceremony** | CC | the controller log carries the `escrow-confirm` mismatch WARN (§6); `escrow_state: pending`; `host_escrow.identity_blob` **unchanged** (this is the assertion that keeps recovery possible) |
+| 8 | **Recover**: export `identity_blob` from the hub, unseal with the kept R, extract `restic_repo_password` | **HUMAN** (R is theirs) | `identity-consume` exits 0; the bundle JSON contains a non-empty `restic_repo_password`; **the value is never printed or logged** |
+| 9 | **Inject** the recovered password (`POST /backup/offbox/inject-password`, by hand — there is no form), re-apply, run an off-site backup | **HUMAN**-assisted | `restic cat config` succeeds; **`snapshot_count` ≥ the step-5 count + 1**, not 1 |
+| 10 | **Restore the sentinel** through the customer restore flow and compare sha256 | CC | **byte-identical sha256 to step 3** |
+| 11 | Record everything, tear down nothing that would hide the evidence | CC | drill report |
+
+### The single observation that makes this a FAILURE rather than a pass
+
+> **Step 10's sha256 does not match step 3's — or step 9 reports a snapshot count of 1.**
+
+A snapshot count of 1 means a **new, empty history** was started over the old one: the repository did
+not reopen, and every earlier snapshot is orphaned. **Explicitly: "the repository opened" is not the
+pass condition.** `restic cat config` succeeding proves the password is right; it does not prove the
+customer's data came back. Only the sha256 comparison does.
+
+### Secondary failures worth recording separately (each is a finding, not a pass/fail)
+
+- Step 7 finds `identity_blob` **changed** — something re-escrowed automatically → the recovery window
+  closes without a human act (would be new and serious).
+- Step 8's bundle has an **empty** `restic_repo_password` — a pre-fork-4 or hash-less blob shape.
+- Step 9's `ensureOffboxRepo` classifies `orphaned` despite the correct password → the classifier or
+  the transport, not the key.
+
+### What could make it fail for uninteresting reasons
+
+The `age` binary absent on the machine running step 8; the hub DB export producing a truncated blob
+(use `writefile`, verify the byte length against `length(identity_blob)` = 572); the CSRF token in
+step 9 (`felhom_session` is dropped by curl's cookie jar — project memory
+`controller-api-headless-access`); and the R-71a settle gate deferring the apply in step 9 (wait for
+its GO line rather than retrying).
+
+### Rollback, and whether the loss is acceptable
+
+**If the recovery does not work, what has been lost is:** demo-hp's guest, its app data since the last
+whole-guest archive, and its off-site history. **On that box that is acceptable** — it is Tier 0,
+explicitly disposable, and it is the box the target-selection runbook names for drills. **The
+rollback** is a whole-guest restore from the local/PBS archive taken immediately before step 6, which
+is a proven path on this fleet (restore-test passed on demo-hp's host tier in 109.3 s on 2026-08-04).
+**Take that archive deliberately as part of step 6 and verify it exists before destroying anything** —
+do not rely on the nightly.
+
+**Not acceptable, and therefore not proposed:** running this on demo-felhom (leaves no healthy demo
+box), on DooPlex (Tier 2, it *is* the recovery chain), or on any real customer's box.
+
+---
+
+## 11. Every query used
+
+| # | Source | Query / read | Value read |
+|---|---|---|---|
+| 1 | git | `git -C <repo> rev-parse HEAD`; `git status --porcelain` | the three commits in §1; all empty |
+| 2 | hub DB | `PRAGMA integrity_check` | `ok` |
+| 3 | hub DB | `SELECT MAX(received_at) FROM host_reports; SELECT datetime('now')` | `2026-08-04 10:00:40` / `2026-08-04 10:03:48` |
+| 4 | hub DB | `SELECT host_id, length(blob), length(identity_blob), substr(restic_pw_sha256,1,12), created_at, updated_at FROM host_escrow` | `demo-felhom-8363b5, 383, 572, c60c8bc737a6, 2026-08-04T07:20:07Z, 2026-08-04 07:20:08` · `demo-hp-bb76ea, 383, 572, 8a9e33aa4da6, 2026-08-04T07:15:36Z, 2026-08-04 07:15:36` |
+| 5 | hub DB | `SELECT id, host_id, length(blob), substr(restic_pw_sha256,1,12), created_at, superseded_at FROM host_escrow_superseded` | `3, demo-hp-bb76ea, 383, 8e03eddf9ff7, 2026-07-23T10:01:17Z, 2026-08-04 07:15:36` · `4, demo-felhom-8363b5, 383, 48741892f0ef, 2026-07-21T08:38:33Z, 2026-08-04 07:20:08` |
+| 6 | hub DB | `PRAGMA table_info(host_escrow)` / `(host_escrow_superseded)` | the two column lists in §7 — `identity_blob` present in the first, **absent from the second** |
+| 7 | hub DB | latest report per customer: `json_extract(report_json,'$.claimed' / '$.offsite.escrow_state' / '.snapshot_count' / '.repo_state' / '.last_status' / '.last_run')` | demo-felhom `2026-08-04 09:49:42`: `1, escrowed, 0, NULL, NULL, NULL` · demo-hp `2026-08-04 09:56:57`: `1, escrowed, 0, NULL, NULL, NULL` |
+| 8 | hub DB | `claimed` grouped over reports since `2026-08-03 05:00` | demo-hp: `1` ×124, **`0` ×5 (06:12:19→06:21:03)** · demo-felhom: `1` ×120, **`0` ×4 (07:17:54→07:19:46)** |
+| 9 | hub DB | `SELECT customer_id, event_type, severity, created_at FROM events WHERE created_at > '2026-08-03 00:00'` | no `escrow_stale`, no `offsite_reissued`, no `offbox_repo_orphaned`, no claim event in the window; `controller_started` ×4 demo-hp / ×4 demo-felhom around the rebuilds |
+| 10 | guest 9201 (I3) | `ls -la --time-style=full-iso …/felhom-controller-data/_data/data{,/offbox}` | the mtime table in §5 |
+| 11 | source (I4) | `grep -rn "InjectOffboxPassword" --include=*.go .` | 1 definition, 1 handler caller, 5 test uses, 1 comment — **no production caller besides the handler** |
+| 12 | source (I4) | `grep -rn "inject" internal/web/templates/` | one unrelated hit (an XSS comment) — **no form** |
+| 13 | source (I4) | `grep -rni "reenroll\|re_enroll" --include=*.go .` in `felhom-agent` | **zero hits** |
+| 14 | source (I4) | `grep -rn "re-enroll\|recovery-mode\|restore-directive" --include=*.sh --include=*.md --include=*.py` workspace-wide | only design docs + project memory; no script, no runbook step |
+| 15 | source (I4) | `grep -rn "ListSupersededEscrow"` in `felhom.eu` | 2 test callers, 0 production |
+| 16 | source (I4) | `grep -rn "\.Escrow\b" --include=*.go .` in `felhom-controller` | one hit: `cmd/controller/main.go:641` |
+| 17 | source | `felhom-agent/CHANGELOG.md` | `v0.77.0 — fork-4: escrow the offsite restic repo password under R (2026-07-09)` |
+| 18 | source | `controller/Dockerfile:36-70` | runtime package list — **no `age`** |
+
+---
+
+## 12. What could not be established
+
+Named, not guessed.
+
+1. **The 2026-08-05 ~02:15 UTC scheduled run's outcome.** Still unmeasured — both boxes became
+   runnable only on 2026-08-04 morning and neither has run since (query 7: `last_run` absent on both).
+   Yesterday's spike predicts `ErrOffboxOrphaned`; that prediction is **unchanged and still unproven**.
+2. **Whether demo-hp's data dir was fresh.** Established for demo-felhom by direct mtime measurement
+   (I3). demo-hp was not measured — it has no baked SSH key and reaching it needs the break-glass
+   credential, which this read-only session did not use. The hub-side `claimed 1→0→1` signature is
+   identical on both boxes, which is suggestive, not decisive.
+3. **Whether the orphaned ciphertext still exists at the provider.** Unchanged from yesterday: reading
+   the endpoint needs the SFTP credential, which was not touched.
+4. **Whether any pre-2026-07-21 escrow generation ever existed for demo-felhom.** The superseded table
+   holds two rows in the entire database; earlier generations are not retained (and, per §7, would not
+   have carried the identity blob anyway).
+5. **Whether `age` is present on any machine that would run step 8 of the drill.** Not checked — the
+   agent declares `/usr/bin/age` as a runtime dependency but this session did not verify it on either
+   demo host.
+
+---
+
+## 13. Observations — noticed, not acted on
+
+1. **A fresh `encryption.key` on every rebuild** (I3: 32 B, mtime `2026-08-03 07:17:26`) means every
+   pre-rebuild `app.yaml` `ENC:` secret on that box is undecryptable. This is exactly circular
+   dependency **C-1** in `_recovery-inventory-2026-07-28.md`, and it fired live on both demo boxes this
+   week without anyone naming it. It is a strictly larger blast radius than the offsite repo and it is
+   **not** covered by R-193. Worth its own item if the operator agrees it is not already implied by C-1.
+2. **`Claimed` returning to `1` within minutes** of a rebuild, with no event of any kind recording it,
+   means "the customer claimed this box" has no audit trail. There is no `customer_claimed` event type.
+3. **`offboxRecordStats` persists `RepoSizeBytes` as a side effect of counting** — harmless today,
+   a trap for anyone reusing it as a read-only preview primitive (§9).
+4. **`handleReEnroll` returns the new API key in the response body** (`dr.go:148`) while also logging
+   the rotation. Not a defect (the box needs it), but it is a secret in an HTTP response on an endpoint
+   nothing currently calls — worth a second look before anything starts calling it.
+
+---
+
+## 14. The operator's decisions — stated, and NOT answered here
+
+1. **Run the drill (§10) or not?** It is the only thing that converts *"the pieces look right"* into
+   *"a customer got their data back"*. It costs demo-hp for an afternoon and, if it fails, that box's
+   history. **Recommendation: yes — but fix `identity_blob` retention (R-198) first**, because a drill
+   that walks the current chain will pass or fail on a chain that is missing a link everyone believed
+   was there.
+2. **The orphaned snapshots at the endpoint** (40.9 MB + 1.14 GB, presumed present, §12.3). They are
+   **unrecoverable by two independent mechanisms** now: no recovery code was kept, *and* the sealed
+   passwords were destroyed by the 2026-08-04 re-ceremonies. They consume the 50 GB soft quota
+   indefinitely; nothing prunes a moved-aside restic repo. **Delete, or leave?** This session did not
+   act on it and states it as the operator's.
+3. **The unchanged R-193 question** — agent-retained repo password (spike option c) versus accepting
+   that every rebuild starts a new repository. **This session adds one input to that decision:** the
+   "accept it, provided the system says so loudly" branch currently rests on a retention feature that
+   does not retain the relevant secret. Under option (d)-alone as it stands today, *"the old history
+   stays recoverable with your recovery code"* is not true. Either R-198 is fixed, or that branch has
+   to be re-described honestly to the customer.
+</content>
+</invoke>

documentation/audits/SPIKE-connectivity-wireguard-2026-07-03.md

@@ -10,7 +10,7 @@
 **Class:** SPIKE (empirical validation; no product code). **Repos:** felhom.eu (this doc only);
 felhom-agent read-only for grounding (`internal/pbs/{client,pin}.go`, `configs/build-golden.sh`,
 `internal/hub/cloudflared.go`, `internal/escrow/identity.go`).
-**Probe ends:** `felhom-hetzner` = Hetzner CX23, Debian 13.4, public IPv4 `167.233.158.164`,
+**Probe ends:** `felhom-hetzner` = Hetzner CX23 (**note added 2026-08-03: rescaled to a CX33, 8 GB RAM — this spike records the machine as probed and its body is deliberately unchanged**), Debian 13.4, public IPv4 `167.233.158.164`,
 global IPv6 `2a01:4f8:...::/64` (throwaway — NOT the live jarrs.eu box) ⟷ **demo-felhom** =
 the real PVE 9.2.2 host on the operator's home line (One Hungary fixed cable, Budapest), driven
 over the existing LAN SSH path; the tunnel itself always dialed **out**.

documentation/audits/SPIKE-offsite-credential-recovery-2026-08-04.md

@@ -0,0 +1,540 @@
+# SPIKE — what is actually one-shot, and what a guest rebuild really costs
+
+**Date:** 2026-08-04 · **Items:** R-193 (rebuild drops offsite), R-192 (the alert says the opposite of
+what it measured) · **Class:** spike — **no production code shipped for R-193 or R-192**
+**Baselines read:** `felhom.eu` @ `f456835bbcc4` (hub v0.91.1) · `felhom-controller` @ `0887fd67` ·
+`felhom-agent` @ `856a127`
+
+> **This document ends in ranked options and a STOP.** It establishes facts; it does not choose.
+> The operator's question is stated, unanswered, at the end.
+
+---
+
+> ### ANNOTATION 2026-08-04 (later the same day) — read this before acting on §Q8
+>
+> A follow-up recon traced the escrow→recovery chain link by link
+> (`audits/RECON-offsite-dr-chain-2026-08-04.md`). **Two corrections to this document, and they pull
+> in opposite directions.**
+>
+> **(1) Q8 candidate (b) — "not implementable as specified" — is TOO STRONG and is overturned in
+> part.** Zero-knowledge means the *hub* cannot open the blob; the *customer* can, with R, which is
+> what the ceremony exists for. The mechanism to hand a rebuilt box its old repo password back —
+> customer-present, not unattended — is a real design, and the operator has since ruled on its shape.
+> What (b) got right is narrower than what it said: **an UNATTENDED rebuild cannot recover from a
+> zero-knowledge escrow.** A customer-present one can. The recon prices that shape row by row.
+>
+> **(2) A LOAD-BEARING FACT THIS SPIKE DID NOT CHECK, and it makes (d)-alone weaker than ranked
+> here.** `host_escrow_superseded` **has no `identity_blob` column**, and `demoteCurrentEscrowTx`
+> (`hub/internal/store/store.go:2547-2556`) copies only the K-escrow `blob`. So the retained
+> "superseded" blob holds the **PBS datastore key**, not the **restic repository password**. The
+> retention feature does not retain the secret it was built to retain, and the **next escrow ceremony
+> — which the system tells the customer to run whenever a rebuild leaves the escrow `pending` — is
+> what destroys the last copy.** Both demo boxes crossed that line on 2026-08-04 at 07:15:36 and
+> 07:20:08. → **R-198**.
+>
+> **Consequence for this document's recommendation.** "Ship (d), then decide (c)" still holds, but
+> (d)'s implicit promise — *the old history stays recoverable with the recovery code* — is **false as
+> the code stands**, and is repeated verbatim to the customer in Hungarian on the orphan card. (d) is
+> not complete without R-198.
+>
+> **Unchanged and still correct:** Q1, Q2, Q4 (R-196), Q5, Q6, Q7 (R-192) and the Q8 (a)/(c)/(d)
+> analyses. **Q3 remains UNMEASURED** — neither box has run since; the 2026-08-05 ~02:15 UTC run is
+> still the decisive one.
+
+---
+
+## 0. Why this exists
+
+Two candidate fixes were named when R-193 was filed, and neither could be weighed, because two prior
+session reports described the same mechanism in contradictory terms:
+
+- R-39's record: Re-issue *"rotates the restic password and makes the escrow stale"*.
+- R-193's own resolution note: the one-time password is *"only the transport credential used once to
+  install the box's own SSH key"*.
+
+Those two worlds imply very different fates for the fifteen existing snapshots. This spike settles it
+**from source first, live state second**, so a measurement can contradict the code rather than be
+interpreted by it.
+
+**The headline, before the detail:** *both prior claims are wrong in the way that matters.* Re-issue
+does **not** touch the restic repository password — but a **guest rebuild does**, silently, on every
+box, with nothing marking the escrow stale and nothing telling anyone. Both demo boxes lost repository
+continuity on 2026-08-03/04. The one that "recovered by luck" lost it too.
+
+---
+
+## 1. The instruments, and what each one can and cannot answer
+
+Stated up front so a wrong instrument is visible here rather than inside a conclusion (§8 rule 2).
+
+| # | Instrument | Exact query / read | Known limit |
+|---|---|---|---|
+| I1 | hub SQLite snapshot | `kubectl exec deploy/hub -- cat /data/{hub.db,hub.db-wal,hub.db-shm}` → local `sqlite3` | **The `-wal` is load-bearing**: `hub.db` alone was 2 h 15 m stale at copy time (mtime 08:34 vs wal 10:45 CEST). Copying only the main file would have produced confident answers from a stale file — the exact R-3 shape. |
+| I1-check | freshness proof | `PRAGMA integrity_check` → `ok`; `MAX(host_reports.received_at)` = `2026-08-04 08:45:38`, `datetime('now')` = `08:49:46` | **Positive observable**: newest row 4 min old. Not "the query returned no error". |
+| I2 | `host_escrow.restic_pw_sha256` + `host_escrow_superseded.restic_pw_sha256` | see §2 | sha256 of a 256-bit random secret — non-reversible, safe to compare. Answers *"did the repo password change"* **without touching a box or a credential**. |
+| I3 | `reports.report_json → $.offsite` | grouped distinct-object query with first/last seen | The report echoes the box's own target state; absence of the key = no target configured. |
+| I4 | `one_time_secrets` | `SELECT customer_id, created_at, consumed_at` | **`customer_id` is the PRIMARY KEY** — one row per customer, last-write-wins. It **cannot** answer "how many credentials were delivered"; a Re-issue destroys the prior row. demo-hp's 2026-07-23 consume is no longer in this table. |
+| I5 | source | `hub/internal/offsite/*`, `hub/internal/api/*`, `controller/internal/{backup,offsiteapply}`, `agent/internal/escrow` | — |
+
+**Not used, deliberately:** no `ReissueCredentials`, no ceremony, no config change, no deletion, no
+write to any box. Nothing on either box or on the storage endpoint was altered by this session.
+
+---
+
+## Q2 — Does a rebuilt controller reuse the escrowed repository password, or mint a new one?
+
+### **RULING: it MINTS A NEW ONE. The escrowed password is never consulted on any automatic path.**
+
+**Method — source.** The apply-bridge's terminal step is
+`Enabler.ConfigureOffbox(...)` (`controller/internal/offsiteapply/offsiteapply.go:221,256`), which
+reaches `Manager.ApplyOffsiteTarget` → `WriteOffboxSecrets`
+(`controller/internal/backup/offbox.go:370`). Quoted verbatim, `offbox.go:392-401`:
+
+```go
+// Auto-generate the repo password once (0600), never log it.
+if _, err := os.Stat(m.offboxPwPath()); os.IsNotExist(err) {
+    pw, gerr := generateOffboxPassword()
+    ...
+    if werr := os.WriteFile(m.offboxPwPath(), []byte(pw), 0o600); werr != nil {
+```
+
+`offboxPwPath()` is `<DataDir>/offbox/repo_password` (`offbox.go:363`). A rebuilt guest has a fresh
+data dir, so the file is absent, so **a fresh 256-bit password is minted**.
+
+**The only path that recovers the old one is `InjectOffboxPassword`** (`offbox.go:541`), whose sole
+caller in the entire repo is `offboxInjectPasswordHandler`
+(`controller/internal/web/offbox_handlers.go:189`) — a **web form where a human pastes the recovered
+password**. Verified by census: `grep -rn "InjectOffboxPassword" --include=*.go .` → three hits, one
+definition, one caller, one comment. **The apply-bridge never calls it.**
+
+**Method — live, and it is decisive without touching a box (I2).** The hub already stores the sha256
+of the escrowed repo password, so the question is answerable as a hash comparison:
+
+| host | superseded (pre-rebuild) | current (post-rebuild) | verdict |
+|---|---|---|---|
+| `demo-hp-bb76ea` | `8e03eddf9ff7…` created `2026-07-23T10:01:17Z` | `8a9e33aa4da6…` created `2026-08-04T07:15:36Z` | **CHANGED** |
+| `demo-felhom-8363b5` | `48741892f0ef…` created `2026-07-21T08:38:33Z` | `c60c8bc737a6…` created `2026-08-04T07:20:07Z` | **CHANGED** |
+
+**Both boxes minted a new repository password.** demo-hp's pre-rebuild escrow (`8e03…`, sealed eight
+minutes after its 2026-07-23 09:53:41 apply) is the key to its 15 snapshots. That key now exists
+**only inside a superseded, R-wrapped escrow blob** that the hub cannot open.
+
+### **This is bigger than R-193 as filed, and it hits the box R-193 called lucky.**
+
+R-193 records demo-felhom as having *"survived the SAME rebuild by luck"* — 76 seconds of downtime.
+Measured (I3), demo-felhom's pre-rebuild offsite object was:
+
+```
+last_run 2026-08-03T02:17:44Z · last_status ok · snapshot_count 36 · repo_size_bytes 1 136 685 919
+```
+
+**36 snapshots, 1.14 GB.** Since 2026-08-03 07:19:10 UTC it has reported `snapshot_count: 0,
+repo_size_bytes: 0` in every one of 109 reports, and its repo password hash changed.
+
+> **The luck recovered DELIVERY. It did not recover the REPOSITORY.** The staged secret restored the
+> transport in 76 seconds and the box then minted a brand-new repo password anyway. The contrast
+> R-193 draws between the two boxes is real for the *credential*, and **false for the data**: both
+> boxes lost repository continuity, one loudly and one silently. The silent one is worse.
+
+---
+
+## Q3 — What did the 04:15 run report?
+
+### **RULING: UNMEASURED — the decisive run has not happened yet. And the binary the question offers is the wrong one; source says the answer will be NEITHER 15 nor 1.**
+
+**Why it has not happened.** The scheduled off-box run fires at ~02:15 UTC (04:15 CEST). Measured
+from I3 — `last_run` values `2026-08-02T02:15:52Z`, `2026-08-03T02:16:39Z` on demo-hp;
+`2026-08-01T02:17:54Z`, `2026-08-02T02:17:01Z`, `2026-08-03T02:17:44Z` on demo-felhom. **Daily, both
+boxes.** Then:
+
+| box | why 2026-08-04 02:15 UTC produced nothing |
+|---|---|
+| demo-hp | no offsite target existed at all — the `offsite` key was **ABSENT** from every report between `2026-08-03 06:12:19` and `2026-08-04 07:12:01` UTC (111 reports). The re-issue landed at 07:15:47, five hours after the window. |
+| demo-felhom | the target existed but `escrow_state: pending` from `2026-08-03 07:19:10` to `2026-08-04 07:20:13`. `OffboxRunnable` requires configured **AND** escrowed (`offbox.go:564-569`), so the fork-4 gate blocked the run. The ceremony landed at 07:20:28. |
+
+Both boxes became runnable only this morning. **The decisive run is 2026-08-05 ~02:15 UTC on both.**
+Reported as unmeasured rather than inferred.
+
+### What source says will happen — and it is a third outcome
+
+Same sub-account, same repo path, new password. Measured coordinates (I3, `$.dr_recipe`):
+`u629488-sub3.your-storagebox.de:/home/felhom-repo` — **unchanged** across the whole incident, and
+`repoPath` is a compile-time constant (`hub/internal/offsite/offsite.go:85`). So the next run opens an
+**existing repo with the wrong key**, which the codebase already has a name for
+(`controller/internal/backup/offbox.go:67-93`):
+
+```go
+// ErrOffboxOrphaned is the sentinel returned when the offsite repo exists but is keyed under a
+// passphrase this controller no longer has (the reinstall shape) ...
+case strings.Contains(s, "wrong password or no key found"):
+    return "orphaned"
+```
+
+and `ensureOffboxRepo` (`offbox.go:663-679`) branches on claim state:
+
+```go
+if !m.settings.GetClaimed() {  // UNCLAIMED → auto move-aside + re-init
+...
+m.markOrphaned()
+return ErrOffboxOrphaned      // CLAIMED → skip the run, show the orphan card
+```
+
+Both boxes report `claimed: 1` (I3, `$.claimed`, latest report each). **So the predicted outcome is:
+the run REFUSES with `ErrOffboxOrphaned`, produces no snapshot at all, and waits for the customer to
+confirm a reset.** Not 15 (reattached) and not 1 (silently fresh) — a **third** outcome the question's
+framing did not contain. That is the good news half: the system is not going to quietly start a new
+history over the old one. It will stop and say so.
+
+**Consequence if the customer confirms the reset:** `resetOrphanedRepo` (`offbox.go:270-314`) moves the
+old repo aside and re-inits — *"move-aside, not deleted"*. The 40.9 MB + 1.14 GB of old ciphertext
+survives, unreadable without the superseded escrow, **and keeps consuming the 50 GB soft quota
+indefinitely.** Nothing prunes a moved-aside restic repo (S-24 covers ep0's PBS namespaces, a
+different tier).
+
+**To measure it tomorrow:** re-read `$.offsite.last_status` / `last_error` on both boxes after
+02:20 UTC, and `$.offsite.repo_state` for the orphan flag. **Predicted `ErrOffboxOrphaned`, not a
+snapshot count.** Whoever picks this up should record which of the three actually occurred — a
+prediction from source is not a measurement.
+
+---
+
+## Q1 — What is one-shot, and what is not?
+
+Three secrets, and they have almost nothing in common.
+
+| | **① storage-provider password** | **② the box's SFTP key** | **③ restic repository password** |
+|---|---|---|---|
+| **What it is** | the Hetzner sub-account / box password | ed25519 keypair for `sftp` transport | the **data** key — restic's repo encryption |
+| **Generated by** | **hub** — `genPassword()` (`offsite.go:481`) | **controller** — `KeyGen.Generate()` (`offsiteapply.go:234`) | **controller** — `generateOffboxPassword()`, 32 random bytes hex (`offbox.go:406`) |
+| **Stored where** | `one_time_secrets` (hub, plaintext, **one row per customer**) | `<DataDir>/offbox/ssh_key` 0600 (box only) | `<DataDir>/offbox/repo_password` 0600 (box) + inside the R-wrapped escrow blob (hub, opaque) |
+| **Delivered how** | `GET …/offsite/consume-password`, **served exactly once** then marked consumed (`api/offsite.go:9-31`) | never delivered — installed onto the provider by ssh-copy-id using ① | never delivered anywhere; it never leaves the box except into the escrow ceremony |
+| **Re-issuable?** | **YES** — `ReissueCredentials` resets it at the provider, any time, operator-initiated | **YES** — regenerated on every full apply; `dr_recipe.go:45` states it plainly: *"the SFTP access key is regenerated at DR"* | **NO automatic path.** Recoverable only by unsealing the escrow with the customer's recovery code and pasting it into `offboxInjectPasswordHandler` |
+| **What a guest rebuild does to it** | nothing (it lives on the hub / at the provider) | destroys it; a fresh one is minted and installed — **harmless** | **destroys it; a fresh one is minted — CATASTROPHIC for the existing repo** |
+| **Escrowed?** | no | no | **yes** — `IdentityBundle.ResticRepoPassword` |
+
+The agent's own source names the asymmetry, and it is the sentence the whole spike turns on
+(`felhom-agent/internal/escrow/identity.go:35-39`):
+
+> `ResticRepoPassword` … **It is the DATA key for the offsite tier — irreplaceable (unlike the SFTP
+> access key, which is regenerable at DR).**
+
+**The one-shot thing (①) is the recoverable one. The irreplaceable thing (③) is the one nothing
+re-stages.** R-193 was filed against ①. The damage is in ③.
+
+**Instrument caveat, worth carrying (I4):** `one_time_secrets` has `customer_id` as PRIMARY KEY, and
+`SaveOneTimeSecret` is last-write-wins **by design** (the R-39(a) guard comment in
+`monitor/offsite_delivery.go:26-30` depends on it). So the delivery ledger holds only the newest
+credential per customer: demo-hp's 2026-07-23 consume, on which R-192's whole diagnosis rests, was
+**overwritten** by the 2026-08-04 Re-issue and is no longer readable from that table.
+
+---
+
+## Q4 — Which claim about Re-issue is true?
+
+### **RULING: `ReissueCredentials` touches ONLY the provider credential (①). It does NOT touch the restic repository password. R-39's record is wrong — and the hub's own comments repeat the same wrong claim in three places.**
+
+**Method — source, quoted rather than summarised.** `ReissueCredentials`
+(`hub/internal/offsite/offsite.go:150-228`) does exactly four things:
+
+1. `genPassword()`;
+2. `ResetSubaccountPassword` / `ResetBoxPassword` + `WaitAction` — **the provider account password**;
+3. `Store.SaveOneTimeSecret(customerID, pw)` — stage ① for delivery;
+4. bookkeeping: `MarkEscrowStale` + `offsite_reissued` + `escrow_stale` events.
+
+There is **no reference to a restic password anywhere in the function**, and none is possible — the
+repo password is generated on the box and never leaves it except into the escrow blob. The hub does
+not hold it in any openable form (`api/handler.go`: *"The hub stores the bytes and NEVER decrypts them
+(it has no recovery code)"*).
+
+**But step 4 is justified by a claim that is false**, `offsite.go:198-201`:
+
+> `// v0.57.0 (2.3, the escrow-honesty fix): the restic repo password just changed, so any existing`
+> `// key-escrow blob — which sealed the OLD password — is now STALE.`
+
+The escrow *is* marked stale; the stated reason for marking it is not something this function does.
+The same false premise appears twice more: `api/handler.go:1067-1069` (*"The re-issuer resets the
+restic repo password, which makes the OLD escrow blob stale"*) and `OPEN-ITEMS.md`'s R-193 row, which
+inherited it from R-39.
+
+> **This is the EIGHTH entry in `CLAUDE.md`'s table of comments asserting an invariant the code does
+> not provide** — and the first where the comment is not merely unenforced but factually describes a
+> different function. It survived because it reads as settled and because its *effect* (a stale escrow)
+> is real, so nobody checked the *cause*.
+
+**Consequence, and it is a live defect, not a documentation nit.** On the ordinary Re-issue shape —
+a consumed-but-failed install on a box that still has its `repo_password` file — the box re-applies,
+`WriteOffboxSecrets` finds the file present and **keeps it**, and the repo password is unchanged. The
+hub has nonetheless told the customer, in Hungarian, that their recovery escrow is stale and asked
+them to re-run the ceremony. **A false staleness alarm and an unnecessary ceremony.** → **R-196**.
+
+**Corroboration against what actually happened on demo-hp yesterday.** The escrow went `pending` after
+the Re-issue and the task asks: if the repo password was untouched, what explains that? **It is fully
+explained, and not by the Re-issue.** Two independent mechanisms, both hub-side-invisible:
+
+1. `ApplyOffsiteTarget` (`offbox.go:482-494`) carries `EscrowState` over from the *existing* target —
+   but the rebuilt guest had **no existing target**, so `cur` was nil, so the fallthrough
+   `if tgt.EscrowState != "escrowed" { tgt.EscrowState = "pending" }` fired.
+2. The box had minted a **new** repo password moments earlier (Q2), so `pending` was *correct* — the
+   new key genuinely was not escrowed.
+
+**The Re-issue's `escrow_stale` event fired at 07:11:51 for the wrong reason and happened to be true
+for a different one.** Coincidence, not correctness. The proof that these are independent is
+**demo-felhom**: it received **no Re-issue at all** (measured — its only `escrow_stale` /
+`offsite_reissued` events in the entire history are dated `2026-07-21 08:29:29`), its repo password
+changed anyway, and **nothing marked its escrow stale for thirteen hours.**
+
+> **The mechanism is exactly inverted.** `escrow_stale` is wired to the one path that does **not**
+> change the repo password, and absent from the path that **does**.
+
+---
+
+## Q5 — Why did one box recover itself and the other not?
+
+### **RULING: confirmed from the ledger — an unconsumed staged secret, thirteen days old. And the recovery was partial in a way the narrative missed.**
+
+**Method — I4, the `one_time_secrets` ledger, no values read:**
+
+| customer | `created_at` | `consumed_at` |
+|---|---|---|
+| `demo-felhom` | `2026-07-21 08:29:29` | `2026-08-03 07:17:58` |
+| `demo-hp` | `2026-08-04 07:11:51` | `2026-08-04 07:12:06` |
+
+demo-felhom's row was **created 2026-07-21 and sat UNCONSUMED for 13 days** — traceable to a Re-issue
+at that exact timestamp (its `offsite_reissued` + `escrow_stale` event pair, `2026-07-21 08:29:29`).
+When the rebuild dropped its target on 2026-08-03, the fresh controller consumed the spare at 07:17:58
+and was reporting `offsite` again by 07:19:10. **76 seconds, on a credential nobody had planned to
+leave there.** Confirmed from the ledger, not the narrative.
+
+**What would have happened without one:** exactly demo-hp — `consume` → **404** (`api/offsite.go:20-22`,
+`no unconsumed offsite password`), the apply-bridge returns
+`offsite-apply: consume one-time password: …` and retries forever, and the tier stays absent until an
+operator acts. demo-hp sat there **25 hours** (`2026-08-03 06:12:19` → `2026-08-04 07:12:01`).
+
+**The correction to the contrast (see Q2):** demo-felhom's spare restored *delivery* only. Its 36
+snapshots / 1.14 GB were orphaned by the same rebuild. **Neither box's data survived; one box's
+plumbing did.**
+
+*Not established:* demo-felhom's escrow generation before 2026-07-21. `host_escrow_superseded` holds
+only two rows in the whole database (ids 3 and 4, one per box, both superseded 2026-08-04), so whether
+the 2026-07-21 Re-issue's ceremony re-sealed an *unchanged* password — the predicted false-staleness
+shape — **cannot be shown from this data**. Stated as unknown rather than asserted.
+
+---
+
+## Q6 — Why is delivery one-shot at all?
+
+### **RULING: the design's reason is that ① is a LIVE PROVIDER PASSWORD, not a bootstrap token; and the honest answer to the security question is that an automatic restage weakens EXPOSURE WINDOW, not authentication.**
+
+**The design's own reasons, from source:**
+
+- `api/offsite.go:9-12` — *"serves the one-time transient offsite password to the controller **EXACTLY
+  ONCE** … The value is returned once then marked consumed — a second call 404s. NEVER logged."*
+- `offsite.go:145-149` — *"the **EXPLICIT** operator recovery for a consumed-password dead-end … It is
+  **NOT** implicit rotation: ProvisionOffsite never calls this."*
+- `offsiteapply.go:1-6` — the apply-bridge is *"idempotent (a descriptor hash marker prevents
+  **re-consuming a spent password**)"*, i.e. the design treats a wasted consume as a real hazard, which
+  R-71a's whole settle-gate exists to prevent.
+
+**The security half, stated rather than skipped.** The question is: *could a machine that is not the
+real one obtain a credential via an automatic restage?*
+
+**What stands there today is authentication, and a restage does not remove it.** The consume endpoint
+is gated by `checkAuthCustomer` (`api/offsite.go:14-18`) — **the customer's API key, the same
+credential as the config pull**. Anything that could consume a restaged password can already pull the
+customer's whole config. So an automatic restage **grants no new access to an unauthenticated party**.
+
+**What it genuinely weakens is the exposure window.** Today a live Hetzner sub-account password is
+fetchable only in the seconds after an operator deliberately staged one. Under an automatic restage it
+becomes fetchable **whenever the box reports no offsite target** — a condition an attacker holding the
+API key can arguably induce. The credential's blast radius at the provider is the customer's own
+sub-account (label-scoped; `ReissueCredentials` refuses unless the label lookup finds **exactly one**,
+`offsite.go:164,180`), so this is a widening of *when*, not of *what*.
+
+**And one further guard must survive any change** — R-39(a), `monitor/offsite_delivery.go:26-30`:
+`SaveOneTimeSecret` is last-write-wins, so restaging on top of an **unconsumed** secret clobbers a
+password a box may be about to consume. Any automatic restage must re-read the row immediately before
+acting and refuse unless it is still a **consumed** row. That guard is correct and is not what R-192 is
+about.
+
+---
+
+## Q7 — R-192's guard: exactly what it reads, and why it read wrong
+
+### **RULING: both halves confirmed, with numbers. The guard reads the OLDEST 500 reports since the consume, and for demo-hp all 500 predated the rebuild by six days.**
+
+**The query, quoted** (`hub/internal/store/store.go:987`):
+
+```sql
+SELECT report_json FROM reports WHERE customer_id = ? AND received_at > ? ORDER BY id LIMIT 500
+```
+
+`ORDER BY id` ascending with `LIMIT 500` = **the oldest 500**, not the newest.
+
+**Reproduced against the live data (I1)**, with demo-hp's real consume anchor `2026-07-23 09:53:41`
+(recovered from R-192's record — it is no longer in `one_time_secrets`, see I4):
+
+| measurement | value |
+|---|---|
+| `total` (what the guard sees) | **500** |
+| `withOffsite` (what the guard sees) | **500** |
+| oldest report in that window | `2026-07-23 09:53:47` |
+| newest report in that window | **`2026-07-28 11:17:40`** |
+| true `total` over the same window, no LIMIT | 1174 |
+| true `withOffsite` over the same window, no LIMIT | 1063 (⇒ 111 without — matching the 111 ABSENT reports in I3) |
+
+**The whole 500-report evidence set ends 2026-07-28 — six days before the 2026-08-03 rebuild.**
+
+**Defect (a) — the message is exactly inverted.** `maybeEmitStuck`
+(`monitor/offsite_delivery.go:~110`) interpolates `status.ReportsSinceConsume` into a string that
+hardcodes the phrase *"carry no offbox target"*, and never reads `OffsiteReportsSinceConsume`. The
+live e-mail, quoted in full from the events table:
+
+> *"Offsite delivery stuck: one-time password consumed 284h19m0s ago and **500 report(s) since carry
+> no offbox target** — the credential is likely burned (apply died between consume and persist).
+> Re-issue delivers a fresh one."*
+
+`OffsiteReportsSinceConsume` for that same call was **500**. Every one of them carried a target. The
+message states the precise negation of its own measurement, and prescribes a remedy for a failure mode
+that did not occur.
+
+**Defect (b) — the heal refuses silently, on stale evidence.** `maybeHeal` returns bare on
+`status.OffsiteReportsSinceConsume != 0` (it was 500), with **no log line**. `offsite_credential_restaged`
+has never fired for any customer — confirmed: zero rows of that event type in the database.
+
+**What the guard SHOULD discriminate on — the shape, not the code.** The guard's intent is right:
+*"there is offbox evidence, so this is a regressed apply, not a burned credential — the operator's
+call."* Its error is that **it asks the question of the wrong time period**. Offbox evidence from
+before a rebuild is not evidence that the credential still works. The discriminator must be
+**recency-bounded and rebuild-aware**: judge on evidence *after the newest `controller_started` /
+config-hash change*, or on the latest N reports — never on "everything since the consume". And a
+refusal must **name its reason in the log**, because "we chose not to heal" and "the heal never ran"
+must not look identical (the S-16 rule, one layer down).
+
+**Do not fix it in isolation.** Under Q2's finding, a successful auto-restage on demo-hp would have
+restored the transport and the box would *still* have minted a new repo password. **The heal cannot
+protect the data; it can only protect the plumbing.** Whatever shape it takes must say so.
+
+---
+
+## Q8 — What each candidate would cost
+
+**A fourth option exists and is the important one**, because (a) and (b) both address ① and Q2 shows
+the damage is in ③.
+
+### (a) The hub restages automatically when a re-enrolled box reports no offsite
+
+**A version of this ALREADY EXISTS and is wired to the wrong event.** `reissueOnReenroll`
+(`hub/internal/api/handler.go:1051-1084`), leg **F3**:
+
+> *"F3 — offsite continuity: re-stage the one-time offsite password to the fresh controller (the
+> one-time password only ever reached the OLD controller)."*
+
+It is called from `handleHostEnroll` — but **after** a mint-once-reuse short-circuit
+(`handler.go:1004-1016`): `if existing != nil { …return… }`. **A guest rebuild leaves the `hosts` row
+intact** (measured: `demo-hp-bb76ea` is the same host_id throughout the incident; only the agent leaf
+re-keyed, `host_leaf_changed` 2026-08-03 06:09:40), so re-enroll returns the existing credential and
+**F3 is never reached.**
+
+- **Changes:** hub only. Either lower the trigger from host-enrollment to a controller-level rebuild
+  signal (the `config_hash` change + `controller_started` pair the hub already receives and already
+  logs), or fix R-192's guard so the existing R-71c self-heal can act.
+- **Weakens:** the exposure window in Q6. Needs the R-39(a) unconsumed-secret guard kept intact.
+- **Does NOT solve:** ③. The box still mints a new repo password and still orphans the repo. **This
+  option restores the plumbing to a box whose data key is already gone.**
+- **Cost:** small. **Value against the actual harm: near zero.**
+
+### (b) The credential becomes recoverable from escrow at re-bootstrap
+
+> **ANNOTATED 2026-08-04 (recon) — this ruling is overturned in part. See the annotation at the top.**
+> The constraint is real but narrower than written: an **unattended** rebuild cannot recover from a
+> zero-knowledge escrow; a **customer-present** one can, and the operator has ruled on that shape.
+> The recon also found that the chain this option would use is clientless at three links and lossy at
+> a fourth (`RECON-offsite-dr-chain-2026-08-04.md` §3–§4).
+
+- **BLOCKED AS STATED, and this is a hard constraint, not an estimate.** The escrow blob is
+  **R-wrapped and zero-knowledge** — the hub holds opaque bytes and has no recovery code
+  (`api/handler.go`, and D6 in `CONTEXT.md` S-3). **A rebuilt box cannot unseal it without the
+  customer's recovery code.** Any design that says "recover it at re-bootstrap" is describing a
+  customer-present ceremony, i.e. what the existing manual `offboxInjectPasswordHandler` already is.
+- **Does NOT solve:** an unattended rebuild, which is the whole scenario.
+- **Cost: not implementable as specified.**
+
+### (c) — NOT PREVIOUSLY NAMED — the agent retains the repo password across a guest rebuild
+
+The agent lives **on the Proxmox host** and survives a guest rebuild. It already receives the repo
+password today: the controller pushes it over the pinned local API
+(`POST /escrow/stage-secret` → `agent/internal/localapi/escrow_stage.go`), and the agent writes it to
+the fixed 0600 path `/var/lib/felhom-agent/escrow-stage/restic_repo_password`
+(`agent/internal/escrow/identity.go:44-46`) **transiently — wiped by the ceremony**
+(`WipeStagedResticPassword`). The seam, the transport, the pinning and the file are all already built.
+
+- **Changes:** agent (retain rather than wipe; serve back over the same pinned local API) + controller
+  (on a fresh data dir, ask the agent before minting — the `InjectOffboxPassword` seam already exists
+  and is exactly the right shape).
+- **Weakens:** the escrow's *zero-knowledge* posture in one specific way — the irreplaceable data key
+  would now sit at rest on the Proxmox host, which the operator has root on. **Measured against D6,
+  that is not a new exposure in principle** (D6 already states plainly that *"the operator cannot read
+  customer data"* was never the security property, and the operator holds root on every box) — but it
+  IS a new copy, and copies are the thing R-133 is already open about for the hub DB. **This is a real
+  trade and it is the operator's to make.**
+- **Solves:** exactly the failure that occurred, unattended, on every rebuild, without any hub
+  involvement and without any credential leaving the premises.
+- **Cost:** medium. Two repos, one existing seam each.
+
+### (d) Do neither — keep the manual Re-issue, and fix what LIES
+
+- **Changes:** R-192's message + refusal log; correct the three false comments about what Re-issue
+  rotates; and — the part that actually matters — **make a repo-password change VISIBLE**. The hub
+  already receives `restic_pw_sha256` on every escrow upload and already stores it. Comparing the new
+  hash against the superseded one is a two-line verdict, and it is the signal that was missing on
+  demo-felhom for thirteen hours → **R-197**. (Note the shape: **this spike answered its own hardest
+  question with a comparison the hub could be making automatically and is not.** Both values were
+  already in the database; nothing read them.)
+- **Weakens:** nothing.
+- **Does NOT solve:** the data loss. Every rebuild still orphans the repo; the operator simply finds
+  out the same day instead of never.
+- **Cost:** small.
+
+### Ranked, with a recommendation
+
+| rank | option | why |
+|---|---|---|
+| **1** | **(d) now — the honesty pass** | It is cheap, it weakens nothing, and **it is the only option that helps the incident already in flight.** The alarm that lies is worse than no alarm; the repo-password-changed detector is the one signal whose absence let demo-felhom lose 1.14 GB of history silently. Ship this regardless of what is decided about the rest. |
+| **2** | **(c) — agent-retained repo password** | The only candidate that addresses ③, which is where the harm is. Every seam it needs exists. It costs one deliberate trade-off the operator must actually make. |
+| **3** | **(a) — auto-restage** | Worth doing **after** (c), never instead of it. On its own it restores the transport to a box whose data key is already gone, and would have made both boxes look healthy on 2026-08-04 while the snapshots were orphaned — a *strictly worse* outcome than the current loud failure. |
+| **4** | **(b) — recover from escrow at re-bootstrap** | Not implementable as stated; the escrow is zero-knowledge by design. |
+
+**Recommendation: ship (d), then decide (c).** And do not ship (a) first — it would have hidden this.
+
+---
+
+## What could not be established
+
+Named rather than guessed (§8 rule 4).
+
+1. **The 2026-08-05 02:15 UTC run's actual outcome** (Q3). Predicted `ErrOffboxOrphaned` from source +
+   claim state; **not measured**.
+2. **Whether the orphaned ciphertext still exists at the provider.** 40.9 MB (demo-hp) + 1.14 GB
+   (demo-felhom) are *presumed* present at `/home/felhom-repo` — reading the endpoint needs the SFTP
+   credential, which this session did not touch. The Hetzner API has no directory-listing surface.
+3. **Whether demo-felhom's 2026-07-21 Re-issue re-sealed an unchanged password** (the predicted
+   false-staleness shape, Q4/Q5). Only two `host_escrow_superseded` rows exist in the whole database;
+   the generation before 2026-07-21 is not retained.
+4. **What removed demo-hp's offsite target at the rebuild** is fully explained (fresh data dir), but
+   **why the guest was rebuilt on both boxes on consecutive mornings** is outside this spike.
+
+---
+
+## The operator's question — stated, and NOT answered here
+
+> **The irreplaceable secret in the offsite tier is the restic repository password. It is generated on
+> the box, it is destroyed by a guest rebuild, and the only copy that survives is sealed under the
+> customer's recovery code — which nothing but a human can open. Every machine is going to be
+> reinstalled.**
+>
+> **Do you want that key to survive a rebuild unattended — by keeping a copy on the Proxmox host, where
+> you already have root — or do you want it to stay sealed under the customer's recovery code and
+> accept that every rebuild starts a new repository and orphans the old one, provided the system says
+> so loudly on the day it happens?**
+
+Option (c) is the first. Option (d) alone is the second. **This spike does not choose.**

documentation/backlog/ROADMAP.md

@@ -20,13 +20,13 @@
 
 | ID | Item | Size | Status | Notes / map rows flipped |
 |----|------|------|--------|--------------------------|
-| R-115 | **Publishing is a remembered step — forgotten within eight hours of being documented as forgettable** | M | **READY** — operator ruling 2026-08-03: **mechanism (b), a build-side gate that REFUSES to deploy or vouch an unpublished version.** THIRD instance the same day (agent v0.120.0 deployed to both boxes while unpublished; a documented-path reinstall would have silently downgraded them and *succeeded*). CC's to build | A box installs the agent from a Gitea generic package the hub explicitly vouches, never from git. Nothing in the build, deploy or session-end path publishes or checks that a version was published, so **"deployed" and "installable" are independent states that drift silently**. **Instance 1 — R-111** (morning): 17 agent releases v0.97.0–v0.113.0 stranded; a new customer would have installed without the whole R-82 tiered-backup arc plus F-CRIT-2 and F-REBOOT. Found only because the E-2d Phase 0 gate happened to look. **Instance 2 — agent 0.114.0** (same afternoon): the R-113 fix, built and pushed at `b58d7bc`, deployed to felhom-pve, never published — which blocked Session C, since a fresh drill box would have installed 0.113.0 and proven the bug rather than the fix. **The finding is the RECURRENCE, not either instance** — both are fixed. R-111 named this leg in its own text (*"decide whether publishing joins the release train rather than staying a remembered step"*) and closed SHIPPED without it; it recurred the same day, which is the evidence that **a note is not a mechanism**. **Class: → R-29, one layer up** (a control that exists and is never walked) — deliberately NOT given a second ID. **Filed as its own item rather than reopening R-111** because R-111's finding (the channel *was* stale) is closed and verified end-to-end by the E-2d install, while the process defect that caused it is a distinct problem with a distinct fix and a distinct owner. **Operator's decision, mechanisms first:** (a) publish as a step in the build/release path so deployed and installable cannot diverge; (b) a gate that refuses to deploy an unpublished+unvouched version — strongest, fails closed; (c) a session-end checklist entry; (d) accept manual + a pre-Session-C verification. **(a)/(b) are mechanisms, (c)/(d) are reminders — and R-29's whole finding is that reminders do not hold.** No code written when filed, by design |
+| R-115 | ~~**Publishing is a remembered step — forgotten within eight hours of being documented as forgettable**~~ | M | **CLOSED — SHIPPED 2026-08-03** (`release-agent.sh` + `check-published-versions.py`, no version bump). Releasing now builds, tags, publishes and **verifies by an independent download** in one act; a `v<semver>` tag with no package fails the gate, and **CI runs the full gate set** so it actually runs. Red-proof measured on real CI: same commit, green before a tagged-unpublished version existed, red after. Residue → **R-184** | A box installs the agent from a Gitea generic package the hub explicitly vouches, never from git. Nothing in the build, deploy or session-end path publishes or checks that a version was published, so **"deployed" and "installable" are independent states that drift silently**. **Instance 1 — R-111** (morning): 17 agent releases v0.97.0–v0.113.0 stranded; a new customer would have installed without the whole R-82 tiered-backup arc plus F-CRIT-2 and F-REBOOT. Found only because the E-2d Phase 0 gate happened to look. **Instance 2 — agent 0.114.0** (same afternoon): the R-113 fix, built and pushed at `b58d7bc`, deployed to felhom-pve, never published — which blocked Session C, since a fresh drill box would have installed 0.113.0 and proven the bug rather than the fix. **The finding is the RECURRENCE, not either instance** — both are fixed. R-111 named this leg in its own text (*"decide whether publishing joins the release train rather than staying a remembered step"*) and closed SHIPPED without it; it recurred the same day, which is the evidence that **a note is not a mechanism**. **Class: → R-29, one layer up** (a control that exists and is never walked) — deliberately NOT given a second ID. **Filed as its own item rather than reopening R-111** because R-111's finding (the channel *was* stale) is closed and verified end-to-end by the E-2d install, while the process defect that caused it is a distinct problem with a distinct fix and a distinct owner. **Operator's decision, mechanisms first:** (a) publish as a step in the build/release path so deployed and installable cannot diverge; (b) a gate that refuses to deploy an unpublished+unvouched version — strongest, fails closed; (c) a session-end checklist entry; (d) accept manual + a pre-Session-C verification. **(a)/(b) are mechanisms, (c)/(d) are reminders — and R-29's whole finding is that reminders do not hold.** No code written when filed, by design |
 | R-116 | **The drive-absent alarm and its recovery are a mismatched pair — generic on the way out, specific on the way back** | S | idea — **PROVEN LIVE 2026-07-29** | Absent fires `storage_disconnected`; return fires `backup_target_restored`. `backup_target_absent` never fires at all (count 0 across a full Session-C run), so an operator gets an alarm they cannot match to its recovery — exactly what `notifyDriveReturned`'s own comment forbids. Root cause: `notifyDriveAbsent` (`intermediary.go:635-646`) branches on `isTarget[a.Path]` with `a.Path` the GUEST path, and `driveTargetByPath` (`:602-616`) builds it as `out[GuestPath] = d.BackupTarget` — but **the drive is TWO `/disks` rows and the flag and the guest path sit on different ones**: the `felhom-backup` storage row has `BackupTarget: true` (`felhom-agent/internal/localapi/disks.go:211`) and gets a guest path only while classified user-data, while the registry union row has the guest path and **never assigns `BackupTarget`** (`disks.go:265-267`). Absent ⇒ the flagged row loses its guest path ⇒ the union row writes `false` ⇒ generic. On return the rows rejoin ⇒ specific. v0.184.1 fixed the KEYING, not this. **Only reachable because R-113 made the gate fire at all.** Fix likely agent-side; decide the repo first. Blocks E-2's C5. Evidence: `audits/SESSION-C-2026-07-29.md` §5 |
 | R-113 | **The drive-absent gate cannot fire on device loss — E-2b's alarm is wired to an unreachable condition** | M | idea — **PROVEN LIVE 2026-07-29** | `planDriveGates` (`felhom-controller/internal/web/intermediary.go:216-262`) treats a path as present by OR-ing in `d.BoundUnderParent`, which the agent derives from `GuestSeesMount()` — *"is this path a mount target in the guest's `/proc/<pid>/mountinfo`"* (`internal/localapi/disks.go:210`). The raw drive mount is a **device-bound systemd unit** and dies with the device; **the agent's own bind under the shared parent is not device-bound and its mountinfo entry outlives the device**, so the gate reads it as present and `notifyDriveAbsent` is never called. Live on a fresh box: target drive hot-detached, agent said `enrolled drive absent by UUID` every 20 s for 4½ min, controller logged **0** `[gate]` lines, hub received **zero** events — neither `backup_target_absent` nor the generic `storage_disconnected`. Not a virtualisation artefact (device-bound-mount vs manual-bind is the same on metal); caveat: SCSI hot-detach, physical unplug not staged. **Sixth instance of seam-built-but-never-wired — E-2b wired the seam to a condition that cannot occur.** Evidence: `audits/E2D-fresh-vm-2026-07-29.md` §5.2 |
 | R-112 | **E-2's degraded banner and offer have no UI consumer — correct endpoint, invisible to the customer** | S | idea — **PROVEN LIVE 2026-07-29** | `GET /api/storage/backup-target` returns byte-exact Hungarian copy (verified on a live box), and nothing in the product asks for it: `grep 'backup-target'` across every `*.html`/`*.js`/`*.css` → **0 hits**; no template references `OfferPath`/`Degraded`/the copy; `resolveBackupTargetState` and `degradedMessageFor` are consumed **only** by the JSON handler, with **no page handler injecting the state**. Decisive contrast: the templates fetch **18 distinct `/api/storage/*` endpoints** — `backup-target` and `backup-target/assign` are the only two with zero references. The handler's own comment calls itself *"the dashboard's source for the degraded banner and the offer"*. v0.185.1 shipped as *"the offer endpoints were mounted where nothing routed to them"* and fixed the **mount**, stopping one layer short of the **render**; its test pins dispatch, not reachability. **Fifth instance of the class. Fix R-114 first** — wiring this alone starts showing customers a wrong message. Evidence: `audits/E2D-fresh-vm-2026-07-29.md` §5.1 |
 | R-114 | **On target-drive loss the customer is told the wrong story and offered the drive that vanished** | S | idea — **PROVEN LIVE 2026-07-29** | With the assigned target absent the endpoint returned `degraded:true, target:"felhom-backup"` **plus** the *"ugyanazon a lemezen van, mint a rendszer"* message — false, the target is a missing drive, not the system disk — **and** an `offer_path` pointing at the drive that just disappeared. `resolveBackupTargetState` falls through to the generic degraded branch whenever no disk satisfies `d.BackupTarget && d.MountPath != ""`, never distinguishing *never configured* from *configured and now missing*. Shares R-113's root cause (two disagreeing presence signals), different code path and fix. **Invisible today only because of R-112.** Evidence: `audits/E2D-fresh-vm-2026-07-29.md` §5.3 |
 | R-111 | ~~**The Day-0 artifact channel is 17 agent releases stale — a box installed today gets agent `0.96.0`, not `0.113.0`**~~ | M | **SHIPPED 2026-07-29** — agent 0.113.0 published (sha `5f3247f7…`, round-trip verified) + golden **0.185.1** baked and published (sha `dba00f3e…`, embeds controller 0.185.1); hub Day-0 manifest moved both together with `min_agent` 0.93.0→0.113.0 (zero fleet impact — all three enrolled hosts already at 0.113.0). **Proven end-to-end the same day:** the E-2d fresh install fetched exactly these and restored the golden baked 20 minutes earlier. Original finding: | **The fleet's live versions are not the fleet's INSTALLABLE versions, and only the first were ever checked.** `felhom-host-install.sh` does not use `main`: `resolve_artifacts()` (`:423-436`) reads the hub-vouched manifest (`GET /api/v1/artifacts/<customer>`, `hub/internal/api/handler.go:2120`) and fetches versioned **Gitea generic packages** — agent from `:1945`, golden from `:2573`. Gitea holds **`felhom-agent` newest `0.96.0`** and **`felhom-golden` newest `0.161.0`**; the hub's saved manifest selects exactly those. So a fresh box lands on **agent 0.96.0 + controller 0.161.0** (golden bake; the global floor is `v0.156.0` < 0.161.0, so it does not self-update) against `main`'s 0.113.0 / 0.185.1. Agent 0.113.0 reached both demo boxes by **direct deploy and is not in the channel at all** — demo-hp reports `felhom-agent 0.113.0` while Gitea's newest is 0.96.0. **17 unpublished releases (`felhom-agent/CHANGELOG.md` v0.97.0–v0.113.0)**, including the ENTIRE R-82 per-target backup-tier arc (v0.97.0–v0.104.0), **F-CRIT-2** (v0.106.0 — a failed backup looking fresh, 7 days silent), **F-REBOOT** (v0.107.0 — a guest rebooted mid-backup never returns), F-LEAK (v0.108.0/0.110.0), F-OBS (v0.109.0), E-2c (v0.111.0), E-2b (v0.112.0), E-2a (v0.113.0). **P1 because it gates the first remote tester:** their box would install an agent predating the tiered-backup model and both silent-failure fixes. **Mirror of R-110, not a duplicate:** R-110 is *the installer publishes instantly with no staging*; this is *the agent and golden have a deliberate publish+vouch gate and it was never walked* — opposite failure modes of one subject, different fixes. Contrast worth keeping: the **controller** is fine (registry has 0.185.1; it self-updates from the floor), so the gap is specific to the two Gitea-generic artifacts. **Decide as part of the fix:** whether publishing becomes part of the release train rather than a separate remembered step — this is R-29's shape (a gate that exists and is never walked) one layer up. Evidence: `audits/E2D-fresh-vm-2026-07-29.md` §2 |
-| R-1 | **Peti convergence** — *the appliance half is DONE; this item is now Peti-only.* **Rehearsal EXECUTED 2026-07-18** (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`): the full final-product flow ran on real metal in one pass (RESET → generic ISO → **customer self-bind** → day-0 → floor lift → escrow ceremony → offsite snapshots), which retires the "supervised rehearsal" dependency that R-13/R-21/R-23/R-24/R-27/R-28 were all parked behind. **Surviving half: Peti's clean-slate proxmox2 reinstall + the parked publish trains on a REAL REMOTE customer** — the one thing a demo box on the operator's own LAN can never prove. | L | **rehearsal DONE; Peti half open** | Flips: publish train PARTIAL→PROVEN-LIVE; appliance/BYO/day-0 "real customer" notes; escrow ceremony. The single biggest unproven surface — an alpha where fixes can't ship remotely is dead. **Reinstall arc SHIPPED hub v0.57.0 (2026-07-16):** the clean-slate reinstall-of-existing-customer path is now first-class — claim re-issue (F2), offsite re-issue (F3), escrow-honesty-on-re-issue (2.3) all auto-fire on re-enrollment. Peti's proxmox2 clean-slate now walks a supported path |
+| R-1 | **Peti convergence** — *the appliance half is DONE; this item is now Peti-only.* **Rehearsal EXECUTED 2026-07-18** (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`): the full final-product flow ran on real metal in one pass (RESET → generic ISO → **customer self-bind** → day-0 → floor lift → escrow ceremony → offsite snapshots), which retires the "supervised rehearsal" dependency that R-13/R-21/R-23/R-24/R-27/R-28 were all parked behind. **Surviving half: Peti's clean-slate proxmox2 reinstall + the parked publish trains on a REAL REMOTE customer** — the one thing a demo box on the operator's own LAN can never prove. | L | **rehearsal DONE; Peti half open** | Flips: publish train PARTIAL→PROVEN-LIVE; appliance/BYO/day-0 "real customer" notes; escrow ceremony. The single biggest unproven surface — an alpha where fixes can't ship remotely is dead. **Reinstall arc SHIPPED hub v0.57.0 (2026-07-16):** the clean-slate reinstall-of-existing-customer path is now first-class — claim re-issue (F2), offsite re-issue (F3), escrow-honesty-on-re-issue (2.3) all auto-fire on re-enrollment. Peti's proxmox2 clean-slate now walks a supported path. **SCOPE-CORRECTED 2026-08-04 (R-193 spike, `audits/SPIKE-offsite-credential-recovery-2026-08-04.md`): "auto-fire on re-enrollment" means HOST re-enrollment, and only when the host record was DELETED first.** `reissueOnReenroll` sits behind `handleHostEnroll`'s mint-once-reuse short-circuit (`if existing != nil { return }`, `api/handler.go:1004-1016`), so a **guest** rebuild — which leaves the `hosts` row intact — never reaches F2 or F3. Measured live: demo-hp's guest was rebuilt 2026-08-03, `demo-hp-bb76ea` was unchanged throughout, F3 did not fire, and the box had **no off-site copy for 25 hours** until an operator ran Re-issue by hand. Peti's clean-slate (a real host delete + re-enroll) still walks the supported path; **the arc does not cover a guest rebuild, and that is the gap R-193 owns.** Note also that F3 alone would NOT have saved the data — the rebuilt controller mints a fresh restic repo password regardless (spike Q2) |
 | R-2 | ~~Resolve ~215 lines of foreign WIP in felhom.eu clone (`hub/internal/notify/`, `store.go`, `hub/internal/claim/`)~~ | S | **killed** (2026-07-16) | Not a real issue: the "foreign WIP" was in-flight code from a concurrent CC session on the customer-claim arc, snapshotted before it committed. All of it landed cleanly — `notify/`+`claim/engine.go` in `6b40eb8` (v0.50.0), `store.go` in `a1d0450` (v0.54.0), plus follow-up `e205a2d`; v0.55.0 shipped. Working tree is clean, no stashes. Lesson already codified: never run two writing sessions on one felhom.eu clone (CLAUDE.md §git add -A) |
 | R-3 | Friend-alpha onboarding runbook (generalized from `pilot/RUNBOOK-peti-return-2026-07-13`): hardware prep → golden → install → claim → ceremony → "first restore by the customer" scripted step | M | idea | Flips: "customer performs a restore" MISSING row; produces the tester-agreement sibling of `PETI-tester-agreement.md`. **Next from-scratch rehearsal to include customer DELETE + re-create** — the ESCROW cascade is now DEFINED (hub v0.60.1): host delete DEMOTES escrow to retained custody (never destroys), customer Danger-zone delete PURGES it (the one true purge point). **S6b (manual stale-host delete before re-enroll) is OBSOLETE** — re-enrollment upserts the existing host row cleanly (`store.UpsertHost` ON CONFLICT DO UPDATE; `handleAdminCreateHost` no duplicate refusal) + the v0.57.0 arc auto-fires the re-issues; the rehearsal live-confirms it. **NON-escrow offboarding NOW ANSWERED by the middle-tier Customer RESET (hub v0.61.0, LIVE):** one operator action deprovisions the Hetzner sub-account/box (repo data destroyed), destroys the PBS namespace + backup groups + token, clears the DR recipe / one-time secret / claim state / retained escrow custody (separate ack) — identity + basic config survive. WG peer release rides host delete (peers are host-scoped, gone before RESET runs — RESET refuses while any host row exists). **Remaining consistency gap:** the customer Danger-zone DELETE still leaves host rows and does NOT run the offsite/PBS teardown (RESET is the teardown path; DELETE is escrow-purge + config-drop). Decide whether DELETE should require a prior RESET (or subsume it) — new item R-25b |
 | R-4 | Claim-code deliverability: test-send to gmail.com / freemail.hu; tighten DMARC `p=none` → `p=quarantine` | S | **DONE 2026-07-21 (all three halves)** | A claim code in spam bricks onboarding at step 1. **DMARC now `p=quarantine`** (`RUNBOOK-publish-0.90-0.143-2026-07-18` Phase 4): `_dmarc.felhom.eu` TXT `v=DMARC1; p=quarantine; rua=mailto:admin@felhom.eu` set by Viktor (CF, Path B); `dig` gate green (1.1.1.1 + 8.8.8.8). Alignment safe (SES/Resend DKIM `d=felhom.eu` + relaxed `send.` SPF). **gmail half CLOSED by the rehearsal (2026-07-18):** the run's claim email was the first sent under `p=quarantine` and **landed in the gmail Inbox, not spam** (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`). **freemail.hu half CLOSED — operator-verified 2026-07-21**: a `[Felhom] Teszt értesítés` was received at a freemail.hu address. With the gmail half closed by the 2026-07-18 rehearsal and the DNS half by `p=quarantine`, **R-4 is COMPLETE** |
@@ -57,6 +57,17 @@
 | R-16 | Operator hygiene: campaign6 autofs orphan (clears on host reboot) + tied-CreatedAt flash duplicates (audiobookshelf/komga/romm) | XS | open (doc-drift bit CLOSED) | Viktor's own action items from 6D/6E. **Doc-drift leftover CLOSED (host-install v1.17.0, 2026-07-17):** the R-20-noted stale "EMPTY by default" operator-key comment corrected (keys are PINNED). Remaining = the two operator items above |
 | R-22 | **PBS-DR pre-check self-grant (F4).** On a non-default storage id the token-auth `GET /storage/<id>` pre-check 403s (no ACL yet) and used to abort before the root-run `grant` that creates it. | S | **SHIPPED + PROVEN-LIVE agent v0.89.0** (2026-07-17) | On a 403 the reconcile self-grants via the root wrapper + re-reads, then converges. Red-proof `TestSelfGrant_PreCheck403DoesNotAbortBeforeGrant`; live-reproduced on the demo (marker aside + ACLs revoked → self-grant → `converged state=adopted` in ~3 s, ACLs restored, offsite active). Origin `tests/VALIDATION-n100-baremetal-2026-07-16.md` F4. |
 | R-17 | Old-box archive (u629193-sub1) retirement — 9/9 byte-identical restores verified | XS | **CLOSED 2026-07-22 — archive deleted (operator console)** | **operator ruling 2026-07-21**: delete the archive now; the 9/9 byte-identical restores are the evidence it is no longer needed. **CC spike-lite (2026-07-21) — it CANNOT be deleted through existing tooling, and CC did not improvise credentialed API calls:** (a) `felhom-tenantsync.sh` does not touch Hetzner at all — it manages PBS namespaces/tokens only; (b) the hub CAN delete sub-accounts (`hetznerapi.DeleteSubaccount`), but its ONLY caller is `offsite.Deprovision`, which selects by the `felhom-customer=<id>` label and runs as part of a customer RESET — a legacy archive sub-account from the 2026-07-09/10 spikes carries no such label, so no tooling verb targets it; (c) the only in-repo references are TEST FIXTURES (`hub/internal/hetznerapi/fake.go` generates the `u629193-sub%d` pattern; `felhom-controller/.../dr_recipe_test.go` uses it as a literal) — nothing live points at it. **Operator console steps:** Hetzner Storage Box console → box `u629193` → Sub-accounts → `u629193-sub1` → confirm the directory holds only the old-box archive → Delete. Deleting the sub-account destroys its data; the 9/9 verification is the sign-off. **CLOSED 2026-07-22: the operator executed the console action — `u629193-sub1` deleted, confirmed no sub-accounts remain on PBS-storage-1** |
+| R-198 | **The superseded-escrow retention kept the K-escrow and dropped the identity blob** — so it preserved the PBS key and destroyed the offsite restic repository password, with the escrow CEREMONY as the destroying act | S | **SHIPPED (hub v0.93.0, 2026-08-04)** | Corrects the escrow-ceremony capability-map row's v0.60.0 retention claim (was struck through 2026-08-04, now restated as what ships). One column + one SELECT list, in the shared `demoteCurrentEscrowTx`, so both callers are fixed at once. Nothing backfillable. **Load-bearing, not tidy:** with R-193(c) refused, customer-present recovery is the only route back from a rebuild and it runs entirely through the retained identity blob |
+| R-197 | **The hub stored both halves of "did this box's offsite data key change" and compared them nowhere** | S | **SHIPPED (hub v0.93.0, 2026-08-04)** | New `offsite_repo_key_changed` (warning, operator-only, edge-triggered, no hash values). Flips nothing in the capability map — it adds a signal, not a capability |
+| R-196 | **Five comments claimed `ReissueCredentials` rotates the restic repo password** — it resets the PROVIDER password and cannot touch the repo password | S | **comments corrected (hub v0.93.0); the BEHAVIOUR stays open** | The false-staleness alarm is a behaviour change and must not ride a comment-correction release — and it matters MORE now, because under R-198 an unnecessary ceremony supersedes a blob rather than being harmless bookkeeping |
+| R-202 | **The orphan card promises recoverability unconditionally**, which after R-198 is true going forward and false for anything already orphaned | S | **gate hit 2026-08-04 — card untouched, sentence still live** | Blocked on knowing which escrow generation an orphaned repo belongs to (R-199/R-201). A single ACK boolean can say a retained recoverable blob EXISTS but not that one COVERS this repo; a conditional promise that can still be false is worse on that surface than a hedged one |
+| R-199 | **The hub served recovery blobs on endpoints with no client anywhere** — the DR capstone's retrieval had never been called by the agent, the hub UI, a script or a runbook | M | **SHIPPED + PROVEN-LIVE (hub v0.94.0 + agent v0.125.0, 2026-08-04)** | Adds the capability-map row *"the offsite repository password can be RECOVERED from the sealed escrow"* as PROVEN-LIVE — for the KEY only, explicitly not for a restore. Chain links 6–8 walked on demo-felhom: recovered sha256 == on-disk sha256 == the hub's stored hash. Carries a deliberate security trade (a box's own credential now suffices where an operator-armed recovery mode was required), recorded in the handler and overridable by one constant |
+| R-200 | **The password-injection seam had a handler and no form** | M | **plumbing SHIPPED (controller v0.195.0); the form is NOT built** | `--recover-offsite-check` compares by hash and refuses to install. The customer-facing shape (yell → recovery-code form → preview → proceed) is priced against a chain that now exists rather than one that was assumed |
+| R-203 | **A mandatory customer data directory was silently absent from the off-site snapshot while the run reported `ok`** — the deploy-time `${USERDATA_PATH}` root and the backup-time namespace root disagree for an app on the system drive | M | **OPEN — halted the R-201 drill 2026-08-04** | Flips nothing yet. Blocks the off-site app-data row from ever earning a customer-file-restored badge. Fix shape: one root function, and a MANDATORY skip must be customer/hub-visible rather than a container-log WARN |
+| R-201 | **The wipe-and-recover drill** | L | **PREPARED, HALTED BEFORE THE WIPE (2026-08-04)** | Nothing irreversible done. Established live for the first time: a rebuilt box's off-site run REFUSES (the orphan card, not a silent fresh history — closes R-193 Q3), the orphan reset works move-aside-never-delete, and demo-hp's pre-rebuild key is permanently gone (superseded four hours before v0.93.0). Resume needs R-203 |
+| R-203 | **The app and its backup looked in different directories, and a run that skipped a mandatory folder still said `ok`** | M | **SHIPPED + PROVEN-LIVE (controller v0.197.0, 2026-08-04)** | Adds the capability-map row *"off-site app-data capture covers MANDATORY paths on both drive layouts"* as PROVEN-LIVE, and corrects that row's predecessor, which was optimistic. Unblocks R-201 |
+| R-204 | **The recovered key cannot be used: the Re-issue that reconfigures a rebuilt box's off-site tier marks the escrow stale, which gates every run, and the only way to clear it destroys the key** | M | **OPEN — the wall the 2026-08-04 night drill hit** | Blocks R-201's verdict. Fix is most likely R-196's (mark stale on evidence, not on a Re-issue) or letting a verified recovered key confirm the escrow without a ceremony |
+| R-201 | **The wipe-and-recover drill** | L | **PASSED + PROVEN-LIVE (2026-08-04 night)** | Flips the off-site app-data row to *"a customer's file survives a machine rebuild and comes back"* — PROVEN-LIVE, scoped to demo-hp and to a controller-data rebuild. The journey took four undocumented manual steps (R-204); the capability is real, the customer experience is not yet |
 | R-19 | Internet-outage customer-experience drill: pull WAN on demo, verify lan_resolver path, document what the customer actually sees/does | S | idea | Flips map row E "LAN access" IMPLEMENTED→PROVEN-LIVE |
 | R-20 | ~~Verify operator-key pinning is fully in the day-0 install flow~~ | XS | **closed** (2026-07-16) | Confirmed against `scripts/felhom-host-install.sh` source (not changelog): keys resolve at L1181–1219 (script constants `OPERATOR_KEY_*`, populated, `--operator-pubkey-file` override), pinned automatically by `step_agent_config()` "STEP 6/8" (L2044; python builds `authz.signers` L2146–2156, reinstall preserves existing), verified at L2332–2337 ("authz signers: N … operator-signed self-update armed"). No interactive prompt or post-install hand-edit — fully automatic. Doc-drift note: the L193–197 "EMPTY by default" comment is stale vs the now-populated constants (→ R-16 hygiene) |
 | R-23 | **Immediate-sync Direction-2 follow-ups** (hub v0.58 / controller v0.140, 2026-07-16): **(a) — BANKED 2026-07-21 (both legs).** The operator-UI save->apply round trip is PROVEN: the STOP-2 global-floor save (hub `18:56:27 CEST`) released the controller's held wait in the **same second** (`16:56:27Z wait woke: generation=1 - firing out-of-cycle report`), with the report built 2 s later; the ring also shows `wait baseline generation=0` at startup (baseline recorded WITHOUT firing, as designed) then `generation=1`, so the generation advanced past 0. **RESTART LEG BANKED 2026-07-21 — the floor was moved to a version the box did NOT run, and the swap fired EXACTLY ONCE.** Operator saved global floor 0.153.0 → **v0.154.0** (a real version boundary, unlike the 2026-07-20 attempt which targeted an already-running version and therefore proved nothing). Timeline (guest UTC): `06:57:13` UpdateState `pending` written `initiated_by=auto-floor` → `06:57:17` agent `controller swap requested 0.153.0 -> 0.154.0` → `06:57:19` `image file written, restarting bootstrap` → `06:57:21` container StartedAt + UpdateState `completed_at` → `06:57:29` agent `new controller healthy`. **16 s end to end.** Assertions over the whole window (06:50 → 07:29, 39 min): `controller swap requested` = **1**, agent-driven bootstrap restarts = **1**, `new controller healthy` = **1**, rollback/swap-failed/unhealthy = **0**, container `RestartCount` = **0**. `VerifyStartup` banked it on the next boot (`Post-update startup: update successful (0.153.0 → 0.154.0)`) and the `06:57:52` periodic check logged `Current version 0.154.0 is up to date` — the at/above-floor branch correctly doing nothing. **No storm, no rollback, no second attempt.** *Caveat, disclosed: a hand-deploy of v0.155.0 at `07:17:10` sits inside the observation window and is what StartedAt shows after that point; it never touches `SwapController`, so the swap-count assertions above are uncontaminated across the full window. A second, unplanned confirmation of the at/above-floor branch came with it — after the hand-deploy the box ran 0.155.0 against a 0.154.0 floor and the updater logged `Current version 0.155.0 is up to date` and did nothing.* Evidence: `felhom-controller/REPORT.md` §6 (2026-07-21). *(Superseded note:)* **the self-restart single-fire leg was** - the floor was set to a version the box ALREADY ran, so there was no work and no restart. Finish by bumping the floor to a version the box does NOT run, debug ring open, asserting EXACTLY ONE restart. **Trap found while banking this: the wake is `logx.Debugf`, so it is INVISIBLE in `docker logs` at INFO** and lives only in the debug ring (`GET /api/debug/logs?level=DEBUG`) - a hunter looking at stdout wrongly concludes the box never woke; arguably (b) generalised. (b) cosmetic: the Waiter's "recovered" INFO logs on the next hold completion (`pollOnce` blocks ~240 s), not at reconnect | S | **(a) BANKED in full; only (b) cosmetic remains** | Map row "config/state change round-trips in seconds" flipped PARTIAL->PROVEN-LIVE 2026-07-21 on this evidence. Evidence: `felhom-controller/REPORT.md` 4f |
@@ -144,7 +155,7 @@
 | R-83 | **Ratify or retire `07-backup-architecture.md`** | S | **DISCHARGED (2026-07-26) — brought current, NOT ratified** | Per CONTEXT ruling S-2 the doc now opens with an honest status header: what each part was verified against, and an explicit ⚠️ **STALE outside the PBS tier** verdict (the controller-side tiers were last verified 41 controller versions ago). The §2 PBS row was corrected — it claimed "PBS on DooPlex" (the retired spike store) with no cadence; it now names `felhom-pbs` → `felhom-offsite` on ep0 over `wg-felhom`, weekly, keep_last=2 — and §9.1 records the measured capacity. **Deliberately NOT marked ratified: ratification is Viktor's review of the §10 list, not an editor's.** R-82 re-verified only the rows it touched and says so, rather than implying a freshness it did not earn. |
 | R-84 | **The agent's cold backup `Store` no longer causes a redundant backup** | S | **SHIPPED (agent v0.103.0, 2026-07-26)** | Observed live: three redundant local backups on demo-felhom in one afternoon of deploys. The `Store` is in-memory ("lost on restart; the cadence re-populates"), so after every restart `/backup/due` said *no successful backup recorded yet* and the controller took another — on the OFFSITE tier a wasted multi-hour WAN upload after every agent deploy. **Resolved by asking the STORAGE rather than persisting the store** (`BackupRunner.NewestArchiveTime` + the optional `localapi.BackupArchiveLister`): ground truth, so a pruned archive correctly stops counting where a persisted record would keep claiming a backup that no longer exists; no new on-disk state, no migration. It answers only *when did a backup last land* — the richer fields stay with real records, so the host-report never carries invented numbers. Fail-safes: read error → fall back to memory (never fake freshness, never suppress); genuinely empty → due; old archive → still due. **Proven live on both boxes** immediately after a restart with the store cold: demo-felhom `local age=5298s / felhom-pbs age=14373s`, demo-hp `local age=498s / felhom-pbs age=2346s`, all `due:false`. |
 | R-85 | **The DR tier must be restore-tested UNATTENDED, and its failure must be HEARD** | M | **Code SHIPPED (agent v0.104.0 + hub v0.77.0, 2026-07-27); rotation NOT YET OBSERVED LIVE** | R-82 proved a PBS restore round-trip **by hand**. Three defects kept the unattended path from covering it, each verified at source: **(a)** `buildRestoreTestScheduler` took `cfg.Backup.BackupTarget()` and built ONE runner on it, so the offsite tier's archives were never candidates; **(b)** the `Spec` was an immediately-invoked function evaluated ONCE at daemon start, so nothing tier-varying was expressible and any storage-type/config change stayed stale until restart; **(c)** a failed restore-test was a `[WARN]` log line with **no event, no notification, no gauge** — true for the LOCAL tier that was already being tested, so the loudest DR signal this system produces was inaudible. (c) is the one that mattered: rotating tiers without it would only mean two tiers can fail silently instead of one. **Selection ruling (operator, 2026-07-26): oldest-first (Option 1)** — never-proven sorts first, which is where the offsite tier starts; no new cadence knob. The per-tier last-proven timestamp is now reported **so the achieved interval can be MEASURED before anyone decides whether a knob is needed** — measure, then configure, not the reverse. **Shipped:** per-run `SpecBuilder`; `PickRestoreCandidateOn` (a tier with no archive NO-OPS, never fails); persisted `RestoreTestState` (rotation survives a restart — credit only on SUCCESS, or a permanently failing tier looks freshly proven); `backup.InFlight`, the host-wide one-heavy-op gate shared with the backup path (a LINK concern, not a lock one — an offsite restore pulls multi-GB over the tunnel a backup pushes one); and hub `restore_test_failed` / `restore_test_stale`, kept **distinct** and anchored on R-81's verdict structure, operator-tier only. **NOT yet proven:** rotation has not been observed selecting both tiers across consecutive unattended cadences — a multi-day window at a 24h cadence. Manual restore-proof IS proven (R-82). |
-| R-86 | **Backup-ALIGNED restore-test scheduling — test a tier ~1 day after ITS OWN backup** | S | **NEXT — operator ruling 2026-07-27** | R-85 schedules restore-tests on a free-running INTERVAL and rotates oldest-proven-first. That cannot express *"the day after the PBS backup"*: any fixed offset drifts against the backup schedule, so alignment would be luck rather than design. **The ruling:** weekly per tier (daily is overkill), timed **~1 day after that tier produced a new backup** — so the gap between an archive being written and being proven stays small, and a tier is never proven against an archive about to be pruned (offsite retention is 2 weeks). **Shape:** trigger a tier from **its own last successful backup** rather than a clock — test tier X once its newest archive is ~24 h old and not yet proven. Self-aligning when a backup runs late, and it naturally stops testing a tier that has stopped producing backups (today an interval keeps re-testing the same stale archive). Needs the scheduler to consult per-tier backup times (already available: `BackupRunner.NewestArchiveTime`, R-84) alongside `RestoreTestState` (R-85). **Interim in force:** `restore_test_cadence_seconds = 302400` (3.5 d) on both boxes — oldest-first over two tiers lands each tier ~weekly, which matches the cadence half of the ruling but NOT the alignment half. |
+| R-86 | ~~**Backup-ALIGNED restore-test scheduling — test a tier ~1 day after ITS OWN backup**~~ | S | **CLOSED — SHIPPED + PROVEN-LIVE 2026-08-03 (agent v0.121.0 + hub v0.91.0)** | **The reasoning that produced the row still holds and is worth keeping:** R-85 scheduled on a free-running INTERVAL with oldest-proven-first rotation, which cannot express *"the day after the PBS backup"* — any fixed offset drifts against the backup schedule, so alignment would be luck rather than design. The operator's ruling was weekly per tier, ~1 day after that tier produced a new backup, so the gap between an archive being written and being proven stays small and a tier is never proven against an archive about to be pruned (offsite retention is 2 weeks). **What shipped is that ruling, with one correction the row's own wording would have got wrong:** the shape written here — *"test tier X once its newest archive is ~24 h old and not yet proven"* — is NEVER true on a DAILY tier, because a new archive resets the newest-archive age to zero before it reaches 24 h. The rule that works is *the newest archive that HAS settled 24 h, and has not been proven*: `A = newest archive older than the settle lag`, due iff A exists and A ≠ the proven archive. **The interim is retired:** `restore_test_cadence_seconds = 302400` (3.5 d) is no longer the mechanism; that key now only disables (negative) or seeds the settle lag. Alignment is no longer approximated by a cadence — it is the schedule. Full detail: `OPEN-ITEMS.md` R-86, `07-backup-architecture.md` §3 |
 | R-87 | **The restic (app-data offsite) tier is NEVER restore-tested** | M | idea — surfaced 2026-07-27 while closing R-85 | **R-85 covers whole-guest vzdump tiers only** (`local`, `felhom-pbs`); the agent has no restic surface at all. restic is the CONTROLLER's app-data offsite backup to the Hetzner Storage Box, a separate mechanism — so the tier that is arguably most important to a customer is the one nothing verifies. It is the only tier that survives losing the box **and** carries their actual app data: the whole-guest snapshot deliberately excludes the bind-mounted data drives (`/mnt/felhom-drives`). Restore code exists and has been exercised BY HAND (the immich destroy-and-recover drill, PROVEN-LIVE), but nothing tests it unattended — **exactly the state PBS was in before R-85: it works when someone tries it, and nobody would know if it stopped.** Needs its own design: a restic restore-test is controller-side, has no scratch-guest analogue, and would verify into a scratch dir rather than a booted guest, so R-85's machinery does not transfer. |
 | R-88 | **An UNREACHABLE backup target reads as "no backup exists" → the agent fires a doomed backup at it** | S | **Part 1 SHIPPED (controller v0.176.0, 2026-07-27); Part 2 OPEN (agent wire change)** — **Part 1** added the failure breaker: consecutive failures tracked per TARGET, backoff `15m→30m→1h→2h→4h` capped, reset on success, gating the QUIESCE rather than the backup so no stack is stopped for a doomed attempt. Never permanent (the cap bounds the retry interval — a latched breaker is a silent backup outage), never global (a broken offsite tier cannot suppress a healthy local one), never applied to `TriggerNow`, and `stillRunning` is not a failure. In-memory on purpose: forgetting a backoff costs one attempt, persisting one could outlive the fix. **Part 2 is agent-side and needs a wire change** — see the P0 finding below. Live on both boxes; the loop reproduction is UNIT-TEST ONLY (reproducing it live would need PBS taken down on ep0, which has no swap). | **The R-81 class (absence-is-not-failure), reappearing one layer down in the AGENT's due-check.** R-84 replaced the in-memory `Store` with storage ground truth (`NewestArchiveTime`), and fails safe on a read error by *falling back to the in-memory record* — "never fake freshness, never suppress". That fallback is correct in isolation but wrong in the one composition where it actually runs: **a cold store plus an unreachable target.** After a restart the in-memory record is empty, so the fallback answers *no backup has ever been recorded* → tier due → the agent fires a backup at the very storage it just failed to read. **Observed:** `felhom-agent` restarted on demo-felhom at 07:02:57 UTC; `local-api: could not read the backup storage for the due-check — falling back to the in-memory record` (vmid=9201 target=felhom-pbs); 11 s later `vzdump 9201 → felhom-pbs` failed with `could not activate storage 'felhom-pbs' … Can't connect to 10.77.0.1:8007 (Connection refused)`. The offsite PBS service was down at the time — so the trigger was real, not synthetic. **NOT bounded, and it also DISABLES the business-hours protection — an availability fault, not a logging nuisance.** The first reading ("fails in ~0 s, so the cost is one spurious event per restart") was wrong on both halves, and a second reading ("it stopped after three tries, so something limits it") was wrong too — the live run stopped after three only because the storage came back. **Verified in code: there is no limiter.** `internal/quiesce` holds no failure counter, no backoff, no breaker, no attempt budget; `runOnce` keeps no state between ticks, and the driver is a plain 5-minute `time.Ticker` (`quiesce.go:149`, default `quiesce.go:107-109`). Had the target stayed down, the loop would have continued indefinitely.
 
@@ -157,7 +168,7 @@ Self-resolves the moment the target answers (the storage read succeeds, sees the
 | R-97 | **The whole-guest backup tier has NO failure signal to the hub — `internal/quiesce` never notifies** | S | **SHIPPED (controller v0.177.0 + hub v0.78.0, 2026-07-27)** — **R-97a:** `quiesce.TierNotifier`, a seam (not an import) wired by an init-only setter, edge-triggered on the R-88 breaker ARMING so a failing tier is reported once per run rather than once per retry; recovery rides `recordSuccess`'s existing bool. **NEW operator-only event types** `whole_guest_backup_failed`/`_recovered` — deliberately NOT `backup_failed`, which carries a customer Hungarian template AND sits in demo-felhom's live `enabled_events`, so reusing it would have emailed the CUSTOMER about a backup they cannot act on while it was still retrying. The recovery joins `recoveredPairedDownTypes` because its `info` severity would otherwise be dropped by `severityNotifies` — the operator would hear it break and never hear it heal. **The hub's operator cooldown was keyed `customerID:eventType` alone**, so one tier would have masked the other for an hour; now narrowly extended with a `tier` suffix taken from the event details, leaving every other event type unchanged. **R-97b:** a suppression window keyed to the quiesce CYCLE (not a state test — v0.164.0's `!= StateStopped` filter cannot see an app caught MID-RESTART, which is exactly how BookStack alarmed), consumed at the same single derivation point `classifyRunStates`. Grace = **180 s**, derived from the deploy flow's 120 s health timeout and Mealie's 60 s `start_period`; it **expires**, so an app that genuinely fails to come back still alarms. **PROVEN LIVE end-to-end with a control:** the new type POSTs 200 from inside guest 9201 while a bogus type 400s, and `notification_log` shows **1 operator row, 0 customer rows**. The quiesce→notify link itself is unit-proven only. | On 2026-07-27 three whole-guest backups failed and three quiesce cycles stopped and restarted every customer app stack, and **not one `backup_failed` event reached the hub.** It is not the allowlist — the hub already carries `backup_failed` and `backup_completed` (they are emitted by the controller's *app-data* backup path). The cause is that **`internal/quiesce` does not import `internal/notify` at all**: the tier R-82 built has no route to the hub, so a whole-guest backup can fail indefinitely in silence. The loop's only trace was `app_start_failed` — **info** severity, **Hungarian**, on the **customer** channel — telling the customer BookStack was down (it had been caught mid-restart by the third cycle) without saying why, during an outage the system itself caused. So the one signal that did fire was both the wrong tier and the wrong story. **Shape:** emit `backup_failed`/`backup_completed` from `quiesceAndPollTiers` naming the TIER, operator-tier; and decide whether a quiesce-induced restart should suppress `app_start_failed` the way controller **v0.164.0**'s deliberate-stop filter does — an app the backup stopped on purpose is not a fault. R-88's breaker bounds the repetition but changes nothing about the silence |
 | R-95 | **The restic offsite tier's credential CAN DELETE — R-89's "parallel question", now ANSWERED** | M | idea — established read-only 2026-07-27 | **The exposure closed on the weekly PBS tier is fully open on the daily restic tier**, which holds the customer's actual documents and photos and is the only tier that survives losing the box. Established without mutating anything: **(1) Identity** — a per-customer *subaccount* on `storage-box-pool-1` (box 611714, bx11, `u629488`): `u629488-sub1` home `felhom-demo-felhom`, `sub2` peti-felhom, `sub3` demo-hp, each labelled `felhom-customer`. Auth is an **SSH key stored ON THE BOX** (`…/felhom-controller-data/_data/data/offbox/ssh_key`, 0600, beside `repo_password` + a pinned `known_hosts`) — customer-side, not hub-side, so a compromised guest holds it. **(2) Read-write: YES** — the API reports **`readonly=False` on all three subaccounts**, and it is not merely latent: the controller runs `restic forget --group-by host,tags --keep-daily 7 --keep-weekly … --prune` **from the box** (`backup/offbox.go:984`, also `:1070`). Delete rights are exercised on every run. **(3) Append-only: NO, and not expressible** — the repo is built as `sftp:` (`offbox.go:482`); restic's append-only mode requires the **REST server** backend, which plain SFTP cannot provide. **(4) A zero-code mitigation exists and is unused:** the box type carries `snapshot_limit=10` and the API reports `snapshot_plan=null` with **0 snapshots** and `size_snapshots=0`. Hetzner Storage Box snapshots are taken **server-side, outside the SFTP namespace** — an SFTP subaccount cannot delete them — so they are a genuine immutability layer at no extra cost and with no code change. **Rule once for both tiers, per R-89.** Options, cheapest first: enable a snapshot plan (operator click, immediate); split backup-write from prune so pruning runs somewhere the box cannot reach; or move the repo to restic's REST server with `--append-only`. Flips the capability-map row for offsite immutability |
 | R-94 | ~~A hand-synced version constant drifts, and the gate that would catch it is never run~~ | XS | **CLOSED — SHIPPED hub v0.87.0, 2026-08-02** | Closed by **deleting** the label rather than deriving it: the Setup command fetches the installer at run time from a 30 s-git-synced website (R-110), so no build-time value in the hub can be true. `hostinstall_gates.py` gate 1 inverted to pin the ABSENCE of a version literal; the tautological `render_test.go` assertion deleted (demonstrated passing at `9.9.9`). Detail: `OPEN-ITEMS.md` R-94 |
-| R-110 | **`main` is the installer's publish channel — there is no staging** | S | **READY** — operator ruling 2026-08-03: **option (b), the channel moves to a TAG**, so publishing is moving the tag and rollback is moving it back. **Must cover BOTH channels** — the nginx `/scripts/` git-sync AND the nine files the installer fetches from `raw/branch/main` — or it only half-works. CC's to build | `manifests/webpage.yaml` git-syncs `/scripts/` from `--branch=main` on a `--period=30s`, and nginx serves that working tree directly (`location /scripts/`, `root /usr/share/nginx/html/current`). So pushing `scripts/felhom-host-install.sh` **is** publishing it: within thirty seconds it is what every subsequent `felhom-bootstrap.sh` fetch (`scripts/iso/felhom-bootstrap.sh:96`) and every operator-run day-0 command (`customer_unified.html:563`, `:1262`; `runbooks/day0-install.md` C.1) receives. **There is no tag, no pinned-version path, no staging copy and no rollback other than another push** — for the one artifact that runs as **root on a virgin box**, the most privileged thing Felhom ships. **Two consequences worth stating plainly:** E-2d is not a gate *before* exposure — 1.22.0 has been the live installer since it hit `main` on 2026-07-29, so the proof run confirms what customers already receive rather than clearing it for release; and the precaution the old R-94 row recorded ("do not point every new box at an installer that has never run") **was never available to take**, because nothing points boxes at a version. **Open question for the operator, not a defect to fix blind:** should `/scripts/` serve a pinned release — a tag-tracked git-sync ref, or a versioned directory (`/scripts/1.22.0/…`) with the hub's generated command naming a version — or is `main`-tracking the accepted shape for a one-operator product where the alternative is a release ritual nobody performs? **Exposure today is zero** (no boxes are installing), which is exactly why it is cheap to decide now. Whichever way it goes, it decides whether R-94 leg (a) makes the label a *fact* (derived from the served script) or keeps it a *claim*. Flips no capability-map row — the map states what the platform does, and this changes nothing about that |
+| R-110 | ~~**`main` is the installer's publish channel — there is no staging**~~ | S | **CLOSED — SHIPPED 2026-08-03** (installer v1.23.0). `/scripts/` syncs `installer-v1.23.0`; the website still tracks `main`. Proven by HTTP: a push to `main` left the served bytes byte-identical, moving the tag published in ~40 s, moving it back restored the exact prior sha. The run-time fetches turned out to be **sixteen from the agent repo**, not nine from here — pinned to the agent version instead → **R-183** | `manifests/webpage.yaml` git-syncs `/scripts/` from `--branch=main` on a `--period=30s`, and nginx serves that working tree directly (`location /scripts/`, `root /usr/share/nginx/html/current`). So pushing `scripts/felhom-host-install.sh` **is** publishing it: within thirty seconds it is what every subsequent `felhom-bootstrap.sh` fetch (`scripts/iso/felhom-bootstrap.sh:96`) and every operator-run day-0 command (`customer_unified.html:563`, `:1262`; `runbooks/day0-install.md` C.1) receives. **There is no tag, no pinned-version path, no staging copy and no rollback other than another push** — for the one artifact that runs as **root on a virgin box**, the most privileged thing Felhom ships. **Two consequences worth stating plainly:** E-2d is not a gate *before* exposure — 1.22.0 has been the live installer since it hit `main` on 2026-07-29, so the proof run confirms what customers already receive rather than clearing it for release; and the precaution the old R-94 row recorded ("do not point every new box at an installer that has never run") **was never available to take**, because nothing points boxes at a version. **Open question for the operator, not a defect to fix blind:** should `/scripts/` serve a pinned release — a tag-tracked git-sync ref, or a versioned directory (`/scripts/1.22.0/…`) with the hub's generated command naming a version — or is `main`-tracking the accepted shape for a one-operator product where the alternative is a release ritual nobody performs? **Exposure today is zero** (no boxes are installing), which is exactly why it is cheap to decide now. Whichever way it goes, it decides whether R-94 leg (a) makes the label a *fact* (derived from the served script) or keeps it a *claim*. Flips no capability-map row — the map states what the platform does, and this changes nothing about that |
 | R-128 | **`ISO_VERSION` "aligns with SCRIPT_VERSION" was a comment nothing evaluated** | XS | **CLOSED — iso v1.26.0, 2026-07-31** | Closed by **correcting the claim, not asserting it**: the ISO is frozen while `felhom-host-install.sh` is fetched at run time from `main` (R-94/R-110), so an assertion would invent a constraint. `build-felhom-iso.sh:45-52`. Full reasoning in `OPEN-ITEMS.md` |
 | R-154 | **`[first-boot]` is automated-install-only, and nothing in the tree said so** | XS | **CLOSED — iso v1.26.0, 2026-07-31** | A PVE property, measured with a same-image control (`audits/SPIKE-universal-iso-3-2026-07-31.md` §2); recorded at `scripts/iso/pkg/build-deb.sh:6-11`. Superseded in practice by the `.deb` delivery route |
 | R-155 | **`iso-repack.sh` refused any ISO without `auto-installer-mode.toml`** | XS | **CLOSED — iso v1.26.0, 2026-07-31** | **Narrowed, not deleted** — unchanged for `FELHOM_MENU=single` (`iso-repack.sh:121-128`), does not apply to `release` where the file's absence *is* gate G1. Do not remove it wholesale |
@@ -175,7 +186,7 @@ Self-resolves the moment the target answers (the storage read succeeds, sees the
 | R-162 | **`docker diff` is the gate's only witness and its failure mode is quiet** | XS | WATCHING — 2026-08-02 | A limitation, not a defect. The gate's power is `docker diff` excluding mounted paths; on a driver where it is unsupported or lies, the gate degrades to mount-occupancy + writability **and would not say so**. It fails closed (the canary self-test stops reporting BROKEN and the gate then refuses to report), but the message blames the prober rather than the driver. Revisit only if a non-overlay driver ships |
 | R-163 | **`mp1` is RETENTION, not staging — and it is sized as if it were neither** | M | **WAITING-ON-OPERATOR** — the ratio is a tier-sizing ruling | Settled by reading: a unit is the KEPT copy on the app's **own** drive (`backup.go:245-255`), falling back to the system SSD for driveless apps — *"the SSD-only system-data fallback"* (`appbackup/paths.go:26-27`); **no post-copy deletion**, the only prune is F5 residue-on-old-drives (`backup.go:1053-1112`). So `mp1` **20 G** retains every driveless app's unit while `mp0` permits **50 G** of volumes, and a DB app's unit is up to **~2×** its data (measured 21.1 GB → 40.2 GB). `--sysdata-grow` defaults to 0 (`felhom-agent/.../main.go:178`), not derived from the drive; demo-hp's real guest ships 50G/20G. **`mp1` gates the whole chain** — Tier-2 mirrors the unit *"(always)"* (`tier2.go:302,368`) and Tier-3 carries it. Bounded: a unit is volume tars + DB dumps only, **never `mp8` userdata** (`recovery_unit.go:20-25`). **No number proposed.** → `07-backup-architecture.md` §7.5 |
 | R-164 | **C2's chain — the DB volume tar cannot be dropped until a sound dump predicate exists** | S | BLOCKED — on the predicate (2026-08-02) | The unit holds a volume tar **and** a SQL dump and the restore uses both: the dump is authoritative and replayed after the tar so it WINS (F17), with only the DB service up (R-47) — `restore_unit.go:262-266`. Dropping the DB tar would halve DB-app units and close R-127(b)'s initdb-skip trap. **The obvious gate is dead, measured:** `ValidateDump`'s empty-`accounts` warning was **correct** (the DB truly had 0 rows; seeding one stopped the warning and put the row in the dump) — but **a fresh appliance legitimately has zero accounts**, so gating on it blocks every new customer's first backup. Order: sound predicate (dump vs **live** per-table counts) → warn→gate → tar-drop. Pairs with **R-127** |
-| R-90 | **ep0 has 3.8 GB and NO swap — it OOMs under a restore-test, and that gates R-86** | S | BLOCKED on Hetzner CX33 availability (operator, 2026-07-27) | Verified read-only: `Swap: 0B`, nothing in `/etc/fstab`, no swap unit, no `/swapfile` — **swap was never added.** On 2026-07-27 06:58:12 a global OOM fired while `proxmox-backup-proxy` held a 3.2 GB peak serving a 14.46 GB restore-test read and the datastore-migration `rsync` held 1.9 GB; the kernel killed the rsync and PBS stopped serving for ~15 minutes. That outage is what exposed **R-88** downstream. Still exposed today: 3.8 GB, no swap, one kill on record. **This gates R-86** — backup-aligned restore-testing puts a multi-GB read on this box *on a schedule*, so the headroom question must be settled before that lands. Interim lever if CX33 stays unavailable: a swapfile, which needs no console |
+| R-90 | ~~**ep0 has 3.8 GB and NO swap — it OOMs under a restore-test, and that gates R-86**~~ | S | **CLOSED 2026-08-03 — the operator rescaled ep0 to a CX33.** MEASURED on the box, not read from an invoice: `Mem: 7757` MB (**8 GB**, was 3.8), `nproc` **4**, and the 4 GiB swapfile added 2026-07-27 **survived the resize** and is active (checked because a resize is a stop/start, so "still there" was an assumption until measured). **The 40 GB local disk is UNCHANGED** — a CPU/RAM resize only — so no disk figure in any runbook needed correcting. Unblocks R-86 | Runbooks corrected; SPIKE/evidence documents ANNOTATED not revised. `target-selection.md`'s "D-d did not name ep0 either way" stays open — it is the operator's |
 | R-91 | **The old 13 GB datastore copy is still on ep0's root disk** | XS | WATCHING — gated on demo-felhom's first post-migration PBS backup | The datastore moved to a Hetzner Cloud Volume on 2026-07-27 (`/dev/sdb`, 100 GiB, attached 06:29:40 UTC, now `/mnt/pbs-datastore`, 13 G used of 98 G). The pre-migration copy survives at **`/srv/pbs-felhom`, 13 G**, on `/` (38 G total, 16 G used, 21 G free). **Do not delete yet:** demo-hp has landed two post-migration snapshots (07-27 08:25:47Z, 09:37:29Z) but **demo-felhom's newest is 2026-07-26T12:21:48Z — before the migration**, so the new volume has not yet proven a write for that namespace. Delete once it has. **Doc drift to fix in the same commit:** `CONTEXT.md:1018` still records the datastore at `/srv/pbs-felhom` |
 | R-92 | **The hub's PBS-DR gauge is 0.1 GB-granular, so small deltas are unverifiable** | XS | idea — 2026-07-27 | The PBS-DR box card rounds to 0.1 GB, which is coarser than the changes an operator wants to confirm after a prune or a GC — a successful prune of a small namespace moves the number by less than one displayed digit, so the UI cannot distinguish "it worked" from "nothing happened". Cosmetic today; it becomes load-bearing the moment retention (R-89) is customer-visible and someone needs to see that a policy change took effect |
 | R-93 | **`drill-r50` is both a blocked customer and the only drift fixture** | XS | idea — 2026-07-27 | The drill customer is blocked in the hub (so it stops alarming) yet it is also the only record exercising the endpoint-drift path R-77 added. Blocking hides it from `GetActiveCustomerIDs`, so the fixture it provides is silently inert — a monitor with no live subject reads exactly like a monitor that passes. Decide: retire it and build a synthetic fixture, or unblock it and silence per-customer instead (the operator has a per-alert silencing feature planned). Related to the R-50 drill VM, now shut down |

documentation/runbooks/RUNBOOK-ep0-datastore-volume-2026-07-27.md

@@ -3,7 +3,8 @@
 **Class:** supervised operational run. **No repo version bump** — the only commits are this record
 and the capacity note. **Nothing was deleted.**
 
-**Host:** `ep0` / `felhom-hetzner`, `167.233.158.164`, Hetzner CX23, Nuremberg.
+**Host:** `ep0` / `felhom-hetzner`, `167.233.158.164`, Hetzner **CX33 (4 vCPU / 8 GB RAM)**, Nuremberg.
+> **Rescaled 2026-08-03** from the CX23 (2 vCPU / 3.8 GB) this runbook was written against. **The 40 GB local disk did NOT change** — this was a CPU/RAM resize — so every disk figure below still stands. The 4 GiB swapfile added on 2026-07-27 survived the resize.
 **Datastore moved:** `felhom-offsite`, `/srv/pbs-felhom` → **`/mnt/pbs-datastore`** (name unchanged).
 **Window:** 06:58 → 07:19 UTC. PBS down 07:00 → 07:17 UTC.
 

documentation/runbooks/RUNBOOK-pbs-prune-serverside-2026-07-27.md

@@ -229,7 +229,9 @@ as the hub 400ing an unknown event type. `verify-new` verifies each snapshot as
 `keep-last 2` that covers essentially the whole datastore and turns a dead check live, for a few
 minutes of ep0 CPU per weekly backup.
 
-> Watch item: ep0 is a 3.7 GB CX23 with **no swap**, and inline verification runs within the backup
+> Watch item (**superseded 2026-08-03**: ep0 is now a **CX33, 8 GB RAM**, and it HAS a 4 GiB swapfile
+> which survived the resize — so the pressure below is much reduced, though the shape of the concern
+> stands). As written: ep0 is a 3.7 GB CX23 with **no swap**, and inline verification runs within the backup
 > window. Today's full forced verify completed fine (~250 MiB/s, 0 errors), but see
 > `RUNBOOK-ep0-datastore-volume-2026-07-27.md` for the rsync OOM on this same box.
 
@@ -307,7 +309,7 @@ Untouched. Rollback remains a two-line `datastore.cfg` revert. Volume: 98 G, 13
    watching: it is the only thing that reclaims chunks, and nothing has ever exercised it here.
 4. **Hub PBS-DR gauge granularity** — 0.1 GB steps mean routine incremental backups are invisible to
    it. Not a fault, but it cannot be used as write-proof evidence for small deltas.
-5. **ep0 has no swap** (3.7 GB CX23) — see the volume runbook's OOM.
+5. ~~**ep0 has no swap** (3.7 GB CX23)~~ — **corrected 2026-08-03: ep0 is a CX33 with 8 GB RAM and an active 4 GiB swapfile.** See the volume runbook's OOM for the original incident.
 
 ## 11. Observations
 

documentation/runbooks/RUNBOOK-vzdump-target-move-2026-07-29.md

@@ -431,7 +431,7 @@ label. Filed under E-2.
 | 2 | **Assignment in the storage wizard** — suggestion by attribute, refusal of the absurd (a 32 GB FAT thumb drive), never a decision by transport or `removable` (§1.2 shows both fail on the reference hardware). |
 | 3 | **Unassigned drives do nothing automatically** — §2's rule, enforced in code. A drive must never acquire a role by appearing. |
 | 4 | **Stickiness** — an assigned target must not move because a new drive appeared, and must never silently retarget when absent. |
-| 5 | **New installs**: `felhom-host-install.sh` must create the target storage with `--is_mountpoint 1` **and issue the `FelhomAgentStore` grant** (§4), or a new box's first backup 403s. |
+| 5 | **New installs**: `felhom-host-install.sh` must create the target storage with `--is_mountpoint 1` **and issue the `FelhomAgentStore` grant** (§4), or a new box's first backup 403s. **ANNOTATION 2026-08-03 (R-185) — this happened, in the half nobody looked at.** The installer's CREATE arm did issue the grant, exactly as this item asked. Its **reuse** arm — *"the target already exists, leave it as it is"* — returned without granting, so a box whose target pre-dated the install (i.e. one moved by THIS runbook) ended up pointing `local_backup_target` at a storage its own token could not read. **CORRECTION 2026-08-03, same day, measured on the box: it DID surface as a 403, exactly as this item predicted — the earlier annotation here said otherwise and was wrong.** demo-felhom's local-api backup jobs 403'd **six times** between 09:24 and 17:34 CEST: `POST /nodes/demo-felhom/vzdump -> HTTP 403: permission denied at /storage/felhom-backup (missing privilege Datastore.Allocate)`. The hub raised `whole_guest_backup_failed` at the first one (*"retrying with backoff"*) and edge-triggering correctly suppressed the rest, so the operator was told once. It ALSO surfaced as the agent's **read** returning `{"data":[]}` while root saw three archives — so the tier was silently never restore-tested. Both demo boxes carried it. Closed by installer **1.24.0** (the reuse arm grants too, with a gate asserting every arm that resolves the target also grants on it) and agent **v0.123.0** (the box now asks whether it may read each tier, because an empty listing cannot distinguish forbidden from newborn). |
 | 6 | **Absent-target policy** per §6: decide fallback-vs-fail, and if fallback, alarm that protection is degraded rather than reporting a healthy tier. |
 | 7 | **Retention and space accounting** on a drive the customer also uses — today `keep-last=3` competes with customer data with no reservation and no ceiling. |
 | 8 | The honest **single-drive label**. |

documentation/runbooks/offsite-endpoint.md

@@ -5,7 +5,8 @@
 > firewall, and the hub-driven `felhom-peersync` reconcile surface. Re-running it on a fresh VM
 > re-creates the endpoint from nothing (that is the DR story, step 8).
 >
-> **Validated:** 2026-07-03 on the dev/test endpoint `felhom-hetzner` (Hetzner CX23, Debian 13,
+> **Validated:** 2026-07-03 on the dev/test endpoint `felhom-hetzner` (Hetzner CX23 **at the time — rescaled
+> to a CX33, 4 vCPU / 8 GB RAM, on 2026-08-03; the 40 GB local disk is unchanged**, Debian 13,
 > `167.233.158.164` / `2a01:4f8:1c16:7aa1::1`) with hub v0.32.0. The production endpoint is a
 > later re-run of this runbook on a production VM.
 >
@@ -31,7 +32,7 @@ Parameters used throughout (adjust for a new endpoint):
       points at nothing (live-run finding). Home-resolver propagation can lag public DNS by
       minutes — a client-side `wg-quick up` that fails to resolve right after record creation
       just needs a retry.
-- [ ] Sanity: `ssh root@167.233.158.164 hostname` → `felhom-hetzner` (the throwaway CX23), not
+- [ ] Sanity: `ssh root@167.233.158.164 hostname` → `felhom-hetzner` (the throwaway box, **CX33 since 2026-08-03**), not
       any production box.
 
 ## 1. Base (on the box, as root)

documentation/runbooks/target-selection.md

@@ -29,20 +29,23 @@ prohibition as covering the act it names and nothing more.
 |---|---|---|
 | **0 — disposable. Reach here first.** | Exists to be broken; reinstalling is a routine afternoon, not an incident. **A drill that needs a victim uses one of these.** | `demo-hp` (t740), `demo-felhom` (N100) |
 | **1 — create and destroy freely** | Throwaway VMs, guests, scratch customers — **hosted on a Tier 0 machine** | drill VMs, scratch guests |
-| **2 — protected. Never a drill target.** | Losing it costs the recovery chain or a real relationship | **DooPlex**, **Peti's cluster** — and, by D-d, **nothing else** |
+| **2 — protected. Never a drill target.** | Losing it costs the recovery chain or a real relationship | **DooPlex**, **Peti's cluster**, **`ep0`** (operator ruling 2026-08-03) — and nothing else |
 
 **DooPlex is Tier 2 because it *is* the recovery chain** — hub, Gitea, registry, PBS, k3s + Longhorn.
 Everything else rebuilds from it; it rebuilds from nothing. A bad moment in a DR drill there costs the
 thing under test, the source of truth for it, and the backups, at once.
 
-**`ep0` + the Hetzner Storage Boxes were Tier 2 until 2026-08-02 and are no longer** — D-d's protected
-list names two machines and ep0 is not one of them. **That does not make them scratch, and the
-difference is an act, not a tier** (see the rule above the table): ep0 holds the **PBS-DR datastore and
-the restic copy of a real customer's data**, which is the only off-premises copy that exists, so
-*deleting datastores, prune jobs, tunnel config or nftables rules* remains forbidden by what it would
-destroy rather than by what tier it sits in. Reads are fine; it is still never a drill target.
-**Flagged for the operator: D-d did not name ep0 either way.** Confirm it explicitly — this page has
-read it the narrow way (not protected, but not wipeable) rather than assume the broad one.
+**`ep0` is Tier 2 — PROTECTED. Operator ruling, 2026-08-03.** D-d named two protected machines and did
+not name ep0 either way, so this page carried the question in writing for two days and read it the
+narrow way meanwhile (not protected, but not wipeable). The ruling settles it and **extends D-d's
+protected list to three machines**: DooPlex, Peti's cluster, ep0.
+
+The reason it was never really in doubt: ep0 holds the **PBS-DR datastore and the restic copy of a
+real customer's data**, which is the only off-premises copy that exists. So *deleting datastores,
+prune jobs, tunnel config or nftables rules* was already forbidden by what it would destroy; the
+ruling makes the classification say so plainly instead of leaving each session to re-derive it.
+**Reads are fine** — including the ordinary off-site read a restore-test performs (R-86) — and it is
+never a drill target. The Hetzner Storage Boxes ride the same reasoning.
 
 **Standing ruling, 2026-07-25 (`operations/nodes.md`):** drill and build VMs live on the **t740** — not
 felhom-pve, and **moved off DooPlex**. This page exists because that ruling sat where no session reads.
@@ -95,11 +98,15 @@ still shares a device with its guest, so a drive failure is **offsite-only recov
 migrated, parked until the tester reinstalls (`PETI` in `backlog/OPEN-ITEMS.md`). Currently DOWN, no
 enrolled host. No access route from DooPlex, and nothing here needs one.
 
-### `ep0` (`felhom-hetzner`, `ep0.felhom.eu`) + the Hetzner Storage Boxes — **not protected by D-d; not scratch either**
+### `ep0` (`felhom-hetzner`, `ep0.felhom.eu`) + the Hetzner Storage Boxes — **Tier 2, PROTECTED** (operator ruling 2026-08-03)
 
 Reads are fine. It is the **offsite of last resort** (PBS-DR datastore, WireGuard hub, operator OOB
-path) and RAM-constrained (3.8 GB, R-90) so a large restore can OOM it. Do not delete datastores, prune
-jobs, tunnel config or nftables rules; never a drill target. The Storage Boxes hold the restic copy —
+path) and — until 2026-08-03 — RAM-constrained (3.8 GB, R-90); it is now a **CX33 with 8 GB RAM and a
+4 GiB swapfile**, which is what closed R-90. A very large restore is still worth watching — the 8 GB
+is comfortable, not unbounded, and the OOM that started R-90 was a 14.46 GB restore read against
+3.8 GB. Do not delete datastores, prune jobs, tunnel config or nftables rules; never a drill target.
+**The ordinary off-site READ a restore-test performs is permitted and unchanged by the ruling**
+(R-86): the classification forbids destruction, not use. The Storage Boxes hold the restic copy —
 customer documents and photos, on a credential that can still delete (R-95).
 **Access: `ssh root@167.233.158.164` from DooPlex** — *not* `felhom-pve → 10.77.0.1`, the route that
 produced a false "unreachable" verdict (standing rule 2).

documentation/tests/campaign10-evidence-2026-07-31/state/phaseA-journal.md

@@ -139,7 +139,7 @@ is not reachable, for two reasons that are each **already-recorded deliberate po
    `endpoint_id` only; per-endpoint allocation is an explicitly deferred arc (`hub/README.md:260`).
 
 So the only two configurations are: *DR tier on* → the campaign's PBS traffic lands on **ep0**, which is
-Tier 2, the offsite of last resort, RAM-constrained (3.8 GB, R-90) and fenced by §3 — or *DR tier off* →
+Tier 2, the offsite of last resort, RAM-constrained (3.8 GB, R-90 — **note added 2026-08-03: ep0 has since been rescaled to a CX33 with 8 GB RAM; this journal records what was true when it was written and is deliberately not revised**) and fenced by §3 — or *DR tier off* →
 no Tier 3 at all. **Chosen: DR tier OFF, offsite OFF**, which is the only option §3 permits.
 
 Consequence, stated plainly rather than discovered later: the campaign touches **neither ep0 nor the

hub/CHANGELOG.md

@@ -1,3 +1,314 @@
+## v0.94.0 — a box can fetch its own sealed recovery package (2026-08-04, R-199 link 6)
+
+**Chain link 6 had no client.** The hub has served the identity blob since slice 10D, from
+`handleReEnroll` / `handleGetRestoreDirective` — gated on operator-armed recovery mode and the global
+key. Nothing in the agent, the hub UI, any script or any runbook ever called either. The only
+documented retrieval was `sqlite3 … writefile(…)` by hand on a `kubectl cp`-ed database.
+
+**`GET /api/v1/hosts/{host_id}/escrow`** — the box-authenticated MIRROR of the PUT that put the blob
+there. Self-scoped: a per-host key reads only its own; the global key may read any, the same asymmetry
+the PUT has. A host with no sealed bundle gets `200 {present:false}` — a clean answer, because a 404
+is indistinguishable from an unknown host and a bare empty 200 from a zero-length blob, and neither
+of those is what "no ceremony has run yet" means.
+
+**THE TRADE, RECORDED IN THE HANDLER RATHER THAN INFERRED.** Before this, obtaining the blob required
+the OPERATOR to arm recovery mode. Now whoever controls a rebuilt box can obtain it with that box's
+own credential. That is a real reduction in the number of parties required, and it is the whole of the
+trade — they still cannot open it, because the hub has never held R and a wrong code fails closed at
+age's scrypt KDF. **The mitigation is that the capability is audited rather than silent:** every
+successful retrieval raises `escrow_blob_served` (warning, operator-only), recorded before the bytes
+leave. A silent capability on this object is the shape the last two weeks were spent removing.
+
+`escrowSelfServiceRetrieval` is a single named constant — the §8.2/§8.3 decision point. Flipping it to
+false re-imposes the recovery-mode requirement on this endpoint and changes nothing else, so the
+operator can overrule the trade at the cost of a boolean rather than a redesign.
+
+**The operator-driven DR path is untouched** — same gate, same behaviour, pinned by a test that
+exercises re-enroll and restore-directive with recovery mode off and on. Red-proofs observed: removing
+the ownership check makes a cross-host read succeed (host A served host B's blob); removing the audit
+record makes the retrieval silent.
+
+**Not in this release:** the customer-facing flow. No card, no form, no preview — those are designed on
+ground that has been walked, and R-200/R-201 are that walk.
+
+## v0.93.0 — the retention keeps the key it was built to keep, and three things stop lying (2026-08-04, R-198/R-197/R-196/R-192)
+
+### R-198 — the superseded-escrow retention was preserving the wrong key, and the ceremony was destroying the right one
+
+`host_escrow_superseded` shipped in v0.60.0 with a `blob` column — the **K-escrow**, i.e. the PBS
+datastore encryption key. `identity_blob` was added to `host_escrow` **later**, by the slice-10D
+`ALTER TABLE`, and **never to the retained table**. The offsite **restic repository password** — the
+data key for every off-site app-data snapshot — lives inside `identity_blob`. So
+`demoteCurrentEscrowTx`, whose own comment calls it *"THE ONE escrow row-copy routine"*, retained the
+whole-guest key and silently dropped the off-site data key: **precisely the secret the reinstall-orphan
+incident the feature was built for was about.**
+
+**What makes it the most serious defect this project has recorded is not the missing column — it is
+which act destroys the key.** The copy happens as the new blob overwrites the old, so the destroying
+act is the **escrow ceremony**: the exact thing a rebuilt box tells its customer to run, on a card
+that promises in Hungarian that the old backups stay recoverable with their recovery code. Both demo
+boxes crossed that line on 2026-08-04 (07:15:36 and 07:20:08), and their previous repository passwords
+are now unrecoverable whether or not anyone kept the recovery codes.
+
+- `host_escrow_superseded` gains `identity_blob BLOB` (in the CREATE for fresh databases **and** as an
+  additive `ALTER TABLE`, following the existing idiom; tolerated on re-run, changes no existing row).
+- `demoteCurrentEscrowTx` carries it — so **both** callers are fixed by one change: the re-escrow path
+  (`SaveHostEscrow`) and the host-delete demotion (`DeleteHost`).
+- `ListSupersededEscrow` reads it back and `store.HostEscrow` gains `IdentityBlob`, so a retained blob
+  is reachable from Go at all. `CountCurrentEscrowWithIdentity` is the census of who the fix protects.
+- The comment above the table now records that the ruling stated there was not met, and what it cost.
+- **Nothing is backfillable and the code says so:** rows superseded before this were written without
+  the identity blob and their source rows are already overwritten.
+
+**Tests assert the CONSEQUENCE, not the mechanism** — the pre-existing
+`TestSaveHostEscrow_RetainsSuperseded` was green throughout, because it asserted that a retained row
+exists with the old K-blob. `TestSaveHostEscrow_RetainsIdentityBlob` asserts the retained row can still
+yield a repository password, and pins the load-bearing ordering (the identity blob is written **after**
+`SaveHostEscrow`, so the demote sees the previous generation). `TestDeleteHost_DemotesIdentityBlob`
+proves the shared routine through its **other** caller — a shared routine tested through one caller is
+how a fix gets believed on a path nobody exercised. Red-proofs: dropping the column from the copy fails
+both; fixing only the re-escrow caller fails the delete one while the re-escrow one passes.
+
+### R-197 — the hub held both halves of the evidence and compared them nowhere
+
+`restic_pw_sha256` is stored on the current escrow and carried to the retained row on every
+supersession. Comparing the two is what answered the 2026-08-04 recon's hardest question in one query —
+and nothing in the hub did it. demo-felhom's repository password changed on 2026-08-03, orphaning 36
+snapshots / 1.14 GB, and **no event, e-mail, card or log line said so for thirteen hours.**
+
+`SaveHostEscrow` now returns the hash it replaced, and `handleHostEscrowPut` raises
+**`offsite_repo_key_changed`** (warning, operator-only, edge-triggered once per supersession) when both
+hashes are known and differ. The in-between shapes are logged rather than dropped, so *"we chose not to
+alarm"* and *"the check did not run"* never look identical. **No hash value travels** — not in the
+message, not in the details; the event names the host, the customer and the retained count.
+
+**Severity chosen for the world this release creates, not the one it inherits:** before R-198 a changed
+key meant the previous history was unopenable by anyone, ever, which would have argued for `error`.
+From v0.93.0 the superseding ceremony retains the old identity blob, so the situation is *"this
+customer's off-site history now depends on an older recovery code"* — operator-actionable, not a loss.
+`warning` also routes: the dispatcher treats `info` as an intentional non-notify.
+
+### R-192 (partially) — the alert describes what was measured, and a refusal leaves a trace
+
+demo-hp e-mailed this daily from 2026-08-03: *"one-time password consumed 284h19m0s ago and 500
+report(s) since carry no offbox target — the credential is likely burned … Re-issue delivers a fresh
+one."* `OffsiteReportsSinceConsume` for that same call was **500**: every one of those reports carried a
+target. The message stated the precise negation of its own measurement and prescribed a remedy for a
+failure mode that had not occurred — while the self-heal correctly declined, **in complete silence**.
+
+- The one stuck state is now reported as the two situations it actually covers, **burned** and
+  **regressed**, each stating its own measurement and carrying its own recommendation (the regressed
+  text explicitly withdraws Re-issue and points at what removes an offbox target — a guest rebuild,
+  R-193). `offsite_reports_since_consume` is in the details for the first time.
+- **The counting window is named in the text.** `CountReportsOffsiteSince` reads `ORDER BY id LIMIT
+  500` — the **oldest** 500 reports after the consume — so on a long-lived box the counts describe the
+  start of the window, not the present. **That guard defect stays OPEN (R-192)** because its correct
+  shape depends on the recovery chain that is not yet assembled (R-199/R-200/R-201); naming the window
+  is how it stays visible instead of being laundered into a confident sentence.
+- **Every self-heal refusal now leaves a record** — a `notification_log` row on the operator channel
+  with status `refused` and its reason, the R-182 suppressed-e-mail precedent. It rides the stuck
+  event's 24 h cadence, so it sits beside the e-mail it explains rather than accumulating per tick.
+  **The guard's logic is unchanged**: the set of situations in which the heal fires is what it was.
+- `offsite_delivery_stuck` and `offsite_credential_restaged` are added to `operatorOnlyEvents`. Neither
+  was ever registered, and neither has a `customerMessages` entry — which is **not** a block, so a
+  customer with a configured recipient was in line for an English e-mail about one-time passwords being
+  *"likely burned"*. Narrowing only; the operator channel is untouched.
+
+### R-196 — five comments corrected (the spec expected three)
+
+`ReissueCredentials` resets the **provider account** password. It does not and cannot touch the restic
+repository password, which is generated on the box and never leaves it except sealed under R. Five
+places in the hub said otherwise: `offsite/offsite.go` (the `MarkEscrowStale` justification),
+`api/handler.go` (the F3 re-enroll comment), and three in `store/store.go` (the `stale_at` ALTER, the
+`MarkEscrowStale` doc comment, and `EscrowStatus.Stale`). **That false belief sent two separate
+investigations the wrong way in one day.** All five now say what the code does, name the correction and
+cite the recon; the staleness mark is documented as **precautionary**, not evidential. **The behaviour
+is unchanged and R-196 stays OPEN** — the false-staleness alarm it causes is a behaviour change and
+must not ride a comment-correction release.
+
+### Not in this release
+
+**R-199, R-200 and R-201 remain open** — the hub's blob-serving endpoints still have no client, the
+password-injection seam still has no form, and nothing in the recovery path has ever been exercised.
+**Part 5 hit its gate and the orphan card is untouched** — see R-202: a single ACK boolean can say
+whether a retained recoverable blob exists, but not whether one covers *the key this orphaned
+repository was written under*, and a conditional promise that is still capable of being false is worse
+on that surface than a hedged sentence.
+
+## v0.92.0 — a customer with no machine ever bound does not alarm (2026-08-04, R-195)
+
+`david` is a real prospective customer whose record was created 2026-08-01 with **no host ever bound**
+— `hosts=0`, `host_deletions=0`, `host_reports=0`, `reports=0` — and it e-mailed an
+`expected_dbdump_missed` **error** at 03:00 UTC on 2026-08-02, 08-03 and 08-04. Nothing has ever been
+expected of that customer, so the honest verdict is UNKNOWN, not missed.
+
+**The mechanism is the interesting half, because the guard that should have covered this is keyed off
+the wrong thing.** `CheckBackupDeadlines` already skips nodes that are `down`, and that skip reads
+`StalenessChecker.GetState()`, whose map is seeded from `store.GetCustomers()` — **a query over the
+`reports` table**. A customer with zero reports is in no row, gets no state at all, and `GetState()`
+returns `""` rather than `"down"`. **The skip misses exactly the customer it would most obviously
+cover.** Corroborated on live data: `peti-felhom` is active with a host deleted 2026-07-15 and does
+*not* alarm, because it has 482 old reports and is therefore `down`. The backup half was already safe
+(`reportJSON == ""` → skip); the DB-dump half had no guard at all.
+
+- **`store.HasEverBoundHost(customerID)`** — a live `hosts` row **OR** a `host_deletions` tombstone.
+  Consulted once per customer at the top of the deadline loop; a false verdict skips the customer with
+  an INFO line and its own counter in the summary.
+- **The discriminator is "was a host EVER bound", NOT "has a report arrived", and that is the whole
+  design.** A box that was installed, bound and then went silent is a real fault and must keep
+  alarming — it has a `hosts` row, so it is judged. Only a customer that never had a machine is
+  UNKNOWN. The tombstone is included for the same reason: a customer whose host was deleted *had* one,
+  and that judgement belongs to the staleness checker, not to this predicate.
+- **Fail-OPEN on a read error.** An unreadable binding must never SUPPRESS a real alarm; it logs and
+  judges anyway.
+- **The anchored-verdict structure is untouched** (R-81/v0.73.0). This is the same invariant — absence
+  is UNKNOWN until an anchor elapses — applied one level up, at the question of whether there is a
+  subject at all.
+
+**Tests, including the one that matters.** `TestCheckBackupDeadlines_BoundButNeverReported_StillAlarms`
+is a real shape — a machine installed and never phoned home — and it is indistinguishable from `david`
+on every signal except the one the guard reads. If the guard is ever "simplified" to key off report
+presence, customer age or a name pattern, it goes red. Plus a bound-then-went-quiet case, a
+deleted-host case, and the predicate's own three inputs. **Companion red-proof observed:** removing the
+guard fails `TestCheckBackupDeadlines_NeverBoundHost_Silent` with `got [expected_dbdump_missed]` —
+verbatim the event `david` sent three mornings running. Restored after.
+
+**`david`'s record was not modified.** The record is correct; the alarm was what was wrong.
+
+**Shipped alongside a spike that deliberately shipped NO code** —
+`documentation/audits/SPIKE-offsite-credential-recovery-2026-08-04.md` (R-193/R-192): what is one-shot
+in the offsite path, and what a guest rebuild really costs. It ends in ranked options and a question
+for the operator; R-196 and R-197 were minted from it and are not implemented here.
+
+## v0.91.1 — observation may only WIDEN a tier's window, never tighten it (2026-08-03, R-86 Part 2)
+
+**Found by checking v0.91.0 against the live box before trusting it, not by review.** demo-felhom's
+offsite tier holds two retained snapshots — `2026-07-27T19:55:41Z` and `2026-07-28T04:49:43Z` —
+**8 h 54 m apart**, because one is a healing artefact and the other a real weekly run. The mean-gap
+estimator therefore reads a **weekly** tier as nine-hourly: ×4 gives 36 h, the 7-day floor lifts it to
+168 h, and a weekly tier proved weekly reaches ~8.25 days of proof age. **The false alarm this whole
+task exists to prevent would have returned within a week, on the box it had just shipped to.**
+
+`restoreProvenWindow` now takes `max(observed, declared)`. Observation refines a tier's rhythm
+**upward** and is ignored downward, which is right on its own terms and not merely cautious: a gap
+SHORTER than the declared rhythm is routine and means nothing — a retry, a manual run, a heal, a
+catch-up after an outage — while a gap LONGER than it is real information, saying this tier genuinely
+receives archives less often than the model assumes and its window must widen or it alarms.
+
+**The cost, stated rather than hidden:** a tier that truly runs faster than its declared rhythm gets a
+wider window than it strictly needs, i.e. a slower `restore_test_stale` signal. That is the right
+direction for a signal whose message is *"unverified"*. *"Broken now"* is `restore_test_failed`, which
+is immediate and untouched.
+
+Three live-derived cases added to `TestRestoreProvenWindow_Contract`, including the exact 8 h 54 m
+gap measured on the box; red-proved by restoring the tighten-too branch
+(`window(pbs, observed=8h54m) = 168h, want 288h`).
+
+## v0.91.0 — a tier's staleness window learns the tier's own rhythm (2026-08-03, R-86 Part 2)
+
+**This ships WITH the agent's v0.121.0, not after it.** The agent now proves a tier once per ARCHIVE
+GENERATION rather than on a 24h timer, so a tier backed up weekly is proved weekly — correctly, and
+in perfect health. `restoreProvenStaleAfter` was a flat 7 days, and its own comment derived that
+number from the cadence R-86 removes:
+
+> *"the restore-test cadence is 24h and rotation is oldest-first across two tiers, so each tier is
+> proven roughly every 2 days. 7 days therefore tolerates ~3 consecutive missed opportunities."*
+
+A weekly tier proved weekly reaches a proof age of **exactly one interval** just before its next
+proof — 168h against a 168h window, false by a hair — so it did not merely sit near that line, it sat
+ON it, and any ordinary delay (a late backup, one deferral behind a running backup) tipped it into a
+nightly alarm about a system that is working. Shipping the agent alone would have converted the
+improvement into a false alarm.
+
+**The window is now per tier**, `restoreProvenWindow(tier, observed, ok)`:
+
+- **the tier's own archive interval**, observed from the host-reports the hub already holds —
+  `pbs_snapshots[]` plus successful `backups[]` attributed by TARGET TYPE (the slice-A.4 rule: a
+  PBS-targeted vzdump appears in both arrays, and classifying by array membership would attribute an
+  offsite archive to the host tier);
+- **× 4 generations** — the settle generation plus ~3 missed opportunities, deliberately the same
+  tolerance the flat constant expressed. The change is to the RHYTHM, not to the patience;
+- **floored at 7 days** — the old constant kept as a floor, so no tier is judged more tightly than
+  before;
+- **capped at 12 days** — strictly inside the 2-week offsite retention with two days to spare, so a
+  tier is never called stale against an archive PBS has already pruned;
+- **falling back to the DECLARED rhythm** when a box's history is too short to observe one:
+  `backupStaleAfter` (26h) for the host tier and `offsiteBackupStaleAfter` (8d) for the offsite tier
+  — the very thresholds the backup-freshness checker already judges those tiers against. A fresh box
+  with one snapshot has no observable interval, and falling back to the FLOOR there would recreate
+  the false alarm on exactly the tier this task is about.
+
+**Kept, because both were earned:** absence is UNKNOWN until an anchored window has passed (R-81's
+structure, untouched), and the stale signal stays edge-triggered. `restore_test_failed` and
+`restore_test_stale` remain DISTINCT — one says your recovery is broken, the other says it is
+unverified, and the second is the one that quietly becomes the first.
+
+**Every reason string now states the window it was judged against** — R-100's corollary: when a
+verdict changes what it counts from, the alarm text has to change with it, or an operator reads
+"limit 168h" under a tier actually judged at 288h and dismisses a true alarm.
+
+**The window READ is unchanged in cost** (14 days), which is both enough to find proof inside the
+widest window and enough to see two generations of a weekly tier.
+
+## v0.90.1 — the digest's per-app lines stop repeating the filesystem figures (2026-08-03, R-182)
+
+**Backfilled 2026-08-03 (R-86 session).** This version was built, deployed and recorded in `REPORT.md`
+and the R-182 row, but never given a CHANGELOG entry — and `REPORT.md` is overwritten every session,
+so the per-repo history under-reported what was actually running. The deployed image has been
+`felhom-hub:0.90.1` since `f21e7ca`.
+
+Found by reading the first REAL digest, not by design: every app row ended with the same usage clause
+the mail already prints once on its own Filesystem line. On a two-app box that is untidy; down a list
+of a dozen it is the same forty characters twelve times, pushing the part that DIFFERS off a phone
+screen at 07:00 — the only moment that mail has to work.
+
+The reserve's refusal message is authored for a single-app alert where naming the filesystem is
+right, so the message is unchanged and the DIGEST trims the duplicate when rendering.
+`trimRepeatedUsage` removes ONLY an exact "— <target path>:" suffix, so an unrelated reason is
+untouched and a reason that is nothing but the usage clause is left alone rather than emptied.
+
+Also inverted the operator half of `TestRecoveryUnitCaptureFailed_NeverReachesTheCustomer`: it
+required the operator to be e-mailed a per-app capture failure, which was correct when that event was
+the only signal and is wrong now that it is the RECORD and the digest is the notification. The
+customer-safety claim is unchanged — R-158's guarantee MOVED, it did not weaken.
+
+## v0.90.0 — a dropped notification leaves a trace, and the backup digest arrives (2026-08-03, R-182)
+
+**The smallest change on the board with the largest effect on trust: `processOperator`'s cooldown no
+longer returns bare.** It used to drop the event *before* `LogNotification`, so a suppressed operator
+alert and an event that never happened were indistinguishable — from the operator's side **and from
+the hub's own records**. Measured 2026-08-03: nine `recovery_unit_capture_failed` events arrived, two
+were mailed, and **seven left no row anywhere**. That is why the defect took a day to get the right
+way round: there was nothing to read.
+
+A suppressed operator event now writes a `suppressed` row carrying the message and **the key that
+suppressed it**, so the collision is readable without reading code. This applies to **every** operator
+event, not only the one that exposed it. It deliberately does **not** change the cooldown's duration
+or semantics — it makes the drop visible, not absent.
+
+**`backup_run_failures` — the per-run digest.** One operator mail at the end of a backup run listing
+every app that failed, its leg and its reason, with the counts and the target filesystem's free
+space. Added to `allowedEventTypes` **and** to `operatorOnlyEvents` — allowlisting alone does not make
+an event operator-only, and `FormatCustomerEmail` falls back to the raw English message rather than
+blocking. A test demonstrates a customer with the type in their enabled list receiving nothing.
+
+**`recordOnlyEvents` — a third routing class.** Types that are STORED and RECORDED but never mailed.
+`recovery_unit_capture_failed` moves here: it is the durable per-failure record, and the digest is the
+notification. Deliberately a register rather than downgrading the severity to `info`, which would have
+had the same routing effect while relabelling a genuine failure as informational in the events table,
+the operator UI and every historical query.
+
+**`cooldownRunSuffix` — the run discriminator.** A sibling of `cooldownTierSuffix` rather than a
+branch inside it, so `tier` keeps byte-identical semantics and R-97a's tests are untouched. It makes
+the cooldown effectively inert for the digest, **which is the intent**: a digest is already
+rate-limited by construction, one per run and only when something failed, so there is nothing for a
+timer to collapse — while the periodic refresh sweep sends **no** `run_id` and therefore stays under
+the ordinary hourly cooldown.
+
+**The e-mail is rendered as a list, not a JSON blob** — the one operator mail with a variable-length
+payload, and a dozen apps on one line is unreadable on a phone at 07:00. An absent space reading
+renders as *unavailable*, never as zeros.
+
 ## v0.89.0 — the two halves of decision D-c (2026-08-02, R-167 · R-158)
 
 **Decision D-c routes two new signals to two different audiences, and the hub is where that routing

hub/internal/api/backup_run_digest_event_test.go

@@ -0,0 +1,40 @@
+package api
+
+import "testing"
+
+// R-182 — the backup run digest needs the allowlist half, and NOT the customerMessages half.
+//
+// A new event type is a pair of register entries, and each half fails differently. For this type the
+// pair is unusual and that is the point:
+//
+//   - missing from `allowedEventTypes`            → POST /event returns 400 and the digest VANISHES,
+//     which would rebuild the exact silence R-182 exists to end;
+//   - PRESENT in `customerMessages`               → the customer would be e-mailed, in Hungarian, a
+//     list of which apps' backups failed and why — operator detail they can take no action on. So
+//     this type must be allowlisted and must NOT have a customer message.
+//
+// The customer-facing half of decision D-c is the FILL WARNING, which fires before this and is
+// actionable (free space, delete files, add a drive). This is the operator's half.
+//
+// Operator-only routing itself is enforced by `notify.operatorOnlyEvents`, NOT by the absence of a
+// customerMessages entry — that assumption shipped in v0.78.0 and was wrong, because
+// FormatCustomerEmail falls back to the raw message. It is pinned in
+// `internal/notify/backup_run_digest_test.go`, which demonstrates a customer with the type in their
+// enabled list receiving nothing.
+func TestBackupRunDigestIsAllowlisted(t *testing.T) {
+	if !allowedEventTypes["backup_run_failures"] {
+		t.Fatal("backup_run_failures must be in allowedEventTypes, or POST /event 400s and the " +
+			"whole run digest is dropped at the door — the silence R-182 was filed against")
+	}
+}
+
+// The per-app event is the RECORD and must not be removed while the digest is the notification.
+// Deleting it would make the digest the only trace, and a digest that fails to send would then take
+// the record with it — the coupling R-182's fix exists to break.
+func TestPerAppCaptureEventStaysAllowlisted(t *testing.T) {
+	if !allowedEventTypes["recovery_unit_capture_failed"] {
+		t.Fatal("recovery_unit_capture_failed was removed from allowedEventTypes — it is the " +
+			"durable per-failure RECORD, and the digest is only the notification; the operator " +
+			"register and every historical query depend on it")
+	}
+}

hub/internal/api/escrow_get_test.go

@@ -0,0 +1,203 @@
+package api
+
+import (
+	"encoding/base64"
+	"encoding/json"
+	"net/http"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-hub/internal/notify"
+	"gitea.dooplex.hu/admin/felhom-hub/internal/store"
+)
+
+// R-199 (hub v0.94.0) — the box-authenticated retrieval of a host's OWN sealed identity blob.
+// This is a new surface on the most sensitive object in the system; these tests exist to pin the
+// three properties that make it defensible: it is self-scoped, it is honest when there is nothing to
+// serve, and it is never silent.
+
+func seedEscrowedHost(t *testing.T, st *store.Store, hostID, customerID, apiKey string, identity []byte) {
+	t.Helper()
+	if err := st.UpsertHost(&store.Host{HostID: hostID, CustomerID: customerID, APIKey: apiKey}); err != nil {
+		t.Fatal(err)
+	}
+	if _, _, err := st.SaveHostEscrow(hostID, []byte("k-escrow"), "fp", "zero_knowledge", "2026-08-04T11:00:00Z", "SHA"); err != nil {
+		t.Fatal(err)
+	}
+	if len(identity) > 0 {
+		if err := st.SaveHostDRBundle(hostID, identity, `{}`); err != nil {
+			t.Fatal(err)
+		}
+	}
+}
+
+// The happy path: a host reads back exactly the bytes it uploaded, verbatim.
+func TestEscrowGet_ServesOwnBlobVerbatim(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	identity := []byte("\x00\x01age-wrapped-identity-bundle\xff")
+	seedEscrowedHost(t, st, "h1", "c1", "HKEY", identity)
+
+	rr := do(h, http.MethodGet, "/hosts/h1/escrow", "HKEY", "")
+	if rr.Code != http.StatusOK {
+		t.Fatalf("GET escrow = %d, want 200 (%s)", rr.Code, rr.Body.String())
+	}
+	var out struct {
+		HostID  string `json:"host_id"`
+		Present bool   `json:"present"`
+		B64     string `json:"identity_escrow_b64"`
+	}
+	if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil {
+		t.Fatal(err)
+	}
+	if !out.Present || out.HostID != "h1" {
+		t.Fatalf("unexpected envelope: %+v", out)
+	}
+	got, err := base64.StdEncoding.DecodeString(out.B64)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if string(got) != string(identity) {
+		t.Fatal("the served blob is not the stored blob — the hub must return ciphertext verbatim")
+	}
+}
+
+// Scenario C — a box can fetch ONLY its own blob.
+// RED-PROOF: delete the `!isGlobal && authHostID != pathHostID` check in handleHostEscrowGet →
+// cross-host retrieval succeeds → this FAILS. Without that line, any host key is a fleet-wide reader
+// of every customer's sealed bundle.
+func TestEscrowGet_CrossHostRefused(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	seedEscrowedHost(t, st, "hostA", "custA", "KEY-A", []byte("A-identity"))
+	seedEscrowedHost(t, st, "hostB", "custB", "KEY-B", []byte("B-identity-SECRET"))
+
+	rr := do(h, http.MethodGet, "/hosts/hostB/escrow", "KEY-A", "")
+	if rr.Code != http.StatusForbidden {
+		t.Fatalf("host A reading host B's blob = %d, want 403 — a cross-host read of a sealed bundle "+
+			"must be impossible on every code path (body: %s)", rr.Code, rr.Body.String())
+	}
+	if b := rr.Body.String(); len(b) > 0 && (contains(b, "B-identity") || contains(b, base64.StdEncoding.EncodeToString([]byte("B-identity-SECRET")))) {
+		t.Fatal("the refusal body leaked the other host's blob")
+	}
+	// Unauthenticated is refused too, and does not leak which hosts exist.
+	if rr := do(h, http.MethodGet, "/hosts/hostB/escrow", "", ""); rr.Code != http.StatusUnauthorized {
+		t.Fatalf("unauthenticated GET = %d, want 401", rr.Code)
+	}
+	// The global operator key MAY read any — the same asymmetry the PUT has.
+	if rr := do(h, http.MethodGet, "/hosts/hostB/escrow", globalKey, ""); rr.Code != http.StatusOK {
+		t.Fatalf("global key GET = %d, want 200", rr.Code)
+	}
+}
+
+func contains(hay, needle string) bool {
+	return len(needle) > 0 && len(hay) >= len(needle) && (func() bool {
+		for i := 0; i+len(needle) <= len(hay); i++ {
+			if hay[i:i+len(needle)] == needle {
+				return true
+			}
+		}
+		return false
+	})()
+}
+
+// Scenario D — a host with no sealed bundle gets a CLEAN answer: 200 with present=false. Not a 404
+// (indistinguishable from an unknown host), not an empty 200 without the flag (indistinguishable
+// from a zero-length blob). Both of those read as a fault to a caller whose situation is simply
+// "no ceremony has run yet".
+func TestEscrowGet_NoBlobIsCleanNone(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	seedEscrowedHost(t, st, "h1", "c1", "HKEY", nil) // K-escrow only, no identity blob
+
+	rr := do(h, http.MethodGet, "/hosts/h1/escrow", "HKEY", "")
+	if rr.Code != http.StatusOK {
+		t.Fatalf("no-blob GET = %d, want 200 with present=false", rr.Code)
+	}
+	var out struct {
+		Present bool   `json:"present"`
+		B64     string `json:"identity_escrow_b64"`
+	}
+	json.Unmarshal(rr.Body.Bytes(), &out)
+	if out.Present || out.B64 != "" {
+		t.Fatalf("a host with no bundle must report present=false and no bytes, got %+v", out)
+	}
+	// An unknown host is a DIFFERENT answer — the two must not collapse into one.
+	if rr := do(h, http.MethodGet, "/hosts/nope/escrow", globalKey, ""); rr.Code != http.StatusNotFound {
+		t.Fatalf("unknown host = %d, want 404 (distinct from a known host with no bundle)", rr.Code)
+	}
+}
+
+// Scenario E — every successful retrieval leaves a durable record naming the host.
+// RED-PROOF: remove the SaveEvent call in handleHostEscrowGet → no event → this FAILS. A silent
+// capability on this object is the whole reason §8.2's trade is acceptable at all.
+func TestEscrowGet_LeavesARecord(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	seedEscrowedHost(t, st, "h1", "c1", "HKEY", []byte("age-wrapped"))
+
+	if rr := do(h, http.MethodGet, "/hosts/h1/escrow", "HKEY", ""); rr.Code != http.StatusOK {
+		t.Fatalf("GET = %d", rr.Code)
+	}
+	ev, err := st.GetLatestEventByType("c1", eventEscrowBlobServed)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if ev == nil {
+		t.Fatal("R-199: a sealed recovery blob was served and NOTHING recorded it — the audit row is " +
+			"the mitigation that makes this endpoint's trade acceptable")
+	}
+	if ev.Severity != "warning" {
+		t.Errorf("severity = %q, want warning: info is an intentional non-notify, so the operator would "+
+			"never hear that the recovery path was used", ev.Severity)
+	}
+	if !contains(ev.Message, "h1") {
+		t.Errorf("the record must name the host, got %q", ev.Message)
+	}
+	// The record must not carry the blob itself — it is an audit row, not a second copy.
+	if contains(ev.Message, "age-wrapped") || contains(ev.DetailsJSON, "age-wrapped") ||
+		contains(ev.DetailsJSON, base64.StdEncoding.EncodeToString([]byte("age-wrapped"))) {
+		t.Errorf("the audit record embedded the blob: msg=%q details=%q", ev.Message, ev.DetailsJSON)
+	}
+	// A "no blob" answer is NOT a retrieval and must not raise one.
+	seedEscrowedHost(t, st, "h2", "c2", "KEY2", nil)
+	do(h, http.MethodGet, "/hosts/h2/escrow", "KEY2", "")
+	if ev, _ := st.GetLatestEventByType("c2", eventEscrowBlobServed); ev != nil {
+		t.Fatal("a present=false answer served no blob and must raise no retrieval record")
+	}
+}
+
+// Operator-tier by construction, registered in the same commit that mints the type.
+func TestEscrowBlobServed_IsOperatorOnly(t *testing.T) {
+	if !notify.IsOperatorOnly(eventEscrowBlobServed) {
+		t.Fatalf("%s is not registered operator-only — a customer would receive operator-grade English "+
+			"about their sealed recovery bundle being handed out", eventEscrowBlobServed)
+	}
+}
+
+// Scenario G — the operator-driven DR path is UNTOUCHED: same gate, same behaviour. The new sibling
+// must not loosen it, and must not share it.
+func TestEscrowGet_OperatorDRPathUnchanged(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	seedEscrowedHost(t, st, "h1", "c1", "HKEY", []byte("age-wrapped"))
+
+	// re-enroll without recovery mode: still refused.
+	if rr := do(h, http.MethodPost, "/hosts/h1/re-enroll", "HKEY", `{"new_api_key":"x"}`); rr.Code != http.StatusForbidden {
+		t.Fatalf("re-enroll without recovery mode = %d, want 403 — the new endpoint must not loosen it", rr.Code)
+	}
+	// restore-directive without recovery mode: still refused.
+	if rr := do(h, http.MethodGet, "/hosts/h1/restore-directive", "HKEY", ""); rr.Code != http.StatusForbidden {
+		t.Fatalf("restore-directive without recovery mode = %d, want 403", rr.Code)
+	}
+	// …and arming it still works, i.e. the old path is functional, not merely closed.
+	if err := st.SetRecoveryMode("h1", time.Now().UTC().Add(10*time.Minute)); err != nil {
+		t.Fatal(err)
+	}
+	if rr := do(h, http.MethodGet, "/hosts/h1/restore-directive", "HKEY", ""); rr.Code != http.StatusOK {
+		t.Fatalf("restore-directive WITH recovery mode = %d, want 200", rr.Code)
+	}
+	// The new endpoint does NOT depend on recovery mode (that is the §8.2 trade, made explicit here so
+	// a future change to escrowSelfServiceRetrieval is visible as a test change).
+	if err := st.ClearRecoveryMode("h1"); err != nil {
+		t.Fatal(err)
+	}
+	if rr := do(h, http.MethodGet, "/hosts/h1/escrow", "HKEY", ""); rr.Code != http.StatusOK {
+		t.Fatalf("self-service escrow GET with recovery mode OFF = %d, want 200 (escrowSelfServiceRetrieval=true)", rr.Code)
+	}
+}

hub/internal/api/escrow_repo_key_changed_test.go

@@ -0,0 +1,146 @@
+package api
+
+import (
+	"encoding/base64"
+	"encoding/json"
+	"net/http"
+	"strings"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-hub/internal/notify"
+	"gitea.dooplex.hu/admin/felhom-hub/internal/store"
+)
+
+// R-197 (v0.93.0) — a changed offsite repository key must be REPORTED. Both halves of the comparison
+// (host_escrow.restic_pw_sha256 and the row it replaces) have been in the database since SLICE 3 and
+// nothing read them: demo-felhom's repository password changed on 2026-08-03, orphaning 36 snapshots /
+// 1.14 GB, and no event, e-mail, card or log line said so for thirteen hours.
+//
+// Driven through the real endpoint (PUT /hosts/<id>/escrow), not by calling the emitter, so the wiring
+// is part of what is proven.
+
+// escrowBodyWithHash is escrowBody plus the sealed-password hash and an identity blob — the fork-4
+// upload shape. Hash VALUES here are test fixtures, never real secrets.
+func escrowBodyWithHash(blob []byte, resticPwSHA, identity string) string {
+	m := map[string]string{
+		"blob_b64":         base64.StdEncoding.EncodeToString(blob),
+		"key_fingerprint":  "ab:cd:ef",
+		"posture":          "zero_knowledge",
+		"created_at":       "2026-08-04T05:00:00Z",
+		"restic_pw_sha256": resticPwSHA,
+	}
+	if identity != "" {
+		m["identity_blob_b64"] = base64.StdEncoding.EncodeToString([]byte(identity))
+	}
+	b, _ := json.Marshal(m)
+	return string(b)
+}
+
+// Scenario D — a new escrow sealing a DIFFERENT repo password raises exactly one operator signal,
+// naming the host, carrying NO hash value.
+// RED-PROOF: remove the maybeEmitRepoKeyChanged call from handleHostEscrowPut (or make its predicate
+// always fall through) → no event → this FAILS.
+func TestEscrowPut_ChangedRepoKey_RaisesSignal(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
+
+	if rr := do(h, http.MethodPut, "/hosts/h1/escrow", "HKEY", escrowBodyWithHash([]byte("k1"), "SHA_GEN1", "age-gen1")); rr.Code != http.StatusOK {
+		t.Fatalf("first PUT = %d (%s)", rr.Code, rr.Body.String())
+	}
+	// Generation 1 is onboarding, not a change — nothing may fire yet.
+	if ev, _ := st.GetLatestEventByType("c1", eventRepoKeyChanged); ev != nil {
+		t.Fatalf("a FIRST escrow must not raise %s: %+v", eventRepoKeyChanged, ev)
+	}
+
+	if rr := do(h, http.MethodPut, "/hosts/h1/escrow", "HKEY", escrowBodyWithHash([]byte("k2"), "SHA_GEN2", "age-gen2")); rr.Code != http.StatusOK {
+		t.Fatalf("second PUT = %d (%s)", rr.Code, rr.Body.String())
+	}
+
+	ev, err := st.GetLatestEventByType("c1", eventRepoKeyChanged)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if ev == nil {
+		t.Fatal("R-197: the repository key demonstrably changed and NO signal was raised — this is the " +
+			"thirteen-hour silence that let demo-felhom lose 1.14 GB unremarked")
+	}
+	if ev.Severity != "warning" {
+		t.Errorf("severity = %q, want warning (info is an intentional non-notify — the operator would not be told)", ev.Severity)
+	}
+	if !strings.Contains(ev.Message, "h1") {
+		t.Errorf("message must name the host, got %q", ev.Message)
+	}
+	// NO HASH VALUE may travel — not in the message, not in the details.
+	for _, forbidden := range []string{"SHA_GEN1", "SHA_GEN2"} {
+		if strings.Contains(ev.Message, forbidden) || strings.Contains(ev.DetailsJSON, forbidden) {
+			t.Errorf("a repo-password hash value leaked into the event (%s): msg=%q details=%q", forbidden, ev.Message, ev.DetailsJSON)
+		}
+	}
+	// Exactly one, not one per anything.
+	evs, _ := st.GetRecentEvents("c1", 50)
+	n := 0
+	for _, e := range evs {
+		if e.EventType == eventRepoKeyChanged {
+			n++
+		}
+	}
+	if n != 1 {
+		t.Fatalf("%s fired %d times for one supersession, want 1 (edge-triggered)", eventRepoKeyChanged, n)
+	}
+}
+
+// Scenario E — an UNCHANGED key says nothing. A customer re-running a ceremony for good reasons must
+// not be punished for it with an alarm.
+func TestEscrowPut_UnchangedRepoKey_Silent(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
+
+	do(h, http.MethodPut, "/hosts/h1/escrow", "HKEY", escrowBodyWithHash([]byte("k1"), "SHA_SAME", "age-1"))
+	do(h, http.MethodPut, "/hosts/h1/escrow", "HKEY", escrowBodyWithHash([]byte("k2"), "SHA_SAME", "age-2"))
+
+	if ev, _ := st.GetLatestEventByType("c1", eventRepoKeyChanged); ev != nil {
+		t.Fatalf("a same-hash re-ceremony must be SILENT, got: %+v", ev)
+	}
+	// And it must not have created a superseded row either (pre-existing contract, re-asserted here
+	// because Scenario E's silence would also be produced by the supersede branch never running).
+	if n, _ := st.CountSupersededEscrow("h1"); n != 0 {
+		t.Fatalf("same-hash re-upload created %d superseded row(s), want 0", n)
+	}
+}
+
+// The in-between shapes must not be silent-and-indistinguishable either: a hash-less NEW blob cannot
+// show whether the key changed, and saying nothing at all would look identical to "unchanged".
+// Asserted at the level the code offers — no event (correct: nothing was measured) and no crash.
+func TestEscrowPut_HashlessSupersession_NoSignal(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
+
+	do(h, http.MethodPut, "/hosts/h1/escrow", "HKEY", escrowBodyWithHash([]byte("k1"), "SHA_GEN1", "age-1"))
+	if rr := do(h, http.MethodPut, "/hosts/h1/escrow", "HKEY", escrowBodyWithHash([]byte("k2"), "", "age-2")); rr.Code != http.StatusOK {
+		t.Fatalf("hash-less supersession PUT = %d", rr.Code)
+	}
+	if ev, _ := st.GetLatestEventByType("c1", eventRepoKeyChanged); ev != nil {
+		t.Fatalf("a hash-less new blob cannot EVIDENCE a change; it must not claim one: %+v", ev)
+	}
+	// The retention still ran (the hashes differ, so this IS a supersession) — the key is kept even
+	// though the change cannot be evidenced.
+	if n, _ := st.CountSupersededEscrow("h1"); n != 1 {
+		t.Fatalf("superseded rows = %d, want 1", n)
+	}
+}
+
+// R-97a's lesson, pinned in the SAME commit that mints the type: an operator-tier event that is not in
+// operatorOnlyEvents reaches customers as raw English, because a missing customerMessages entry is NOT
+// a block. Checked here rather than in notify so the type's two registers are asserted together.
+func TestRepoKeyChanged_IsOperatorOnly(t *testing.T) {
+	if !notify.IsOperatorOnly(eventRepoKeyChanged) {
+		t.Fatalf("%s is not registered operator-only — a customer would receive operator-grade English "+
+			"about escrow custody", eventRepoKeyChanged)
+	}
+	// The two R-192 types have the same property and the same reason (see the register's comment).
+	for _, et := range []string{"offsite_delivery_stuck", "offsite_credential_restaged"} {
+		if !notify.IsOperatorOnly(et) {
+			t.Errorf("%s is not registered operator-only", et)
+		}
+	}
+}

hub/internal/api/escrow_test.go

@@ -155,7 +155,7 @@ func TestReportACK_EscrowStatus(t *testing.T) {
 	}
 
 	// escrow row with identity blob + hash → the ACK carries all three fields
-	if _, err := st.SaveHostEscrow("hv1", []byte("k-blob"), "fp", "zero_knowledge", "2026-07-09T20:00:00Z", "abc123"); err != nil {
+	if _, _, err := st.SaveHostEscrow("hv1", []byte("k-blob"), "fp", "zero_knowledge", "2026-07-09T20:00:00Z", "abc123"); err != nil {
 		t.Fatal(err)
 	}
 	if err := st.SaveHostDRBundle("hv1", []byte("identity-blob"), "{}"); err != nil {

hub/internal/api/handler.go

@@ -239,6 +239,12 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	case r.Method == http.MethodPut && strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/escrow"):
 		hostID := strings.TrimSuffix(strings.TrimPrefix(path, "/hosts/"), "/escrow")
 		h.handleHostEscrowPut(w, r, hostID)
+	// R-199 (v0.94.0): the box-authenticated MIRROR of the PUT above — a host reads back its own
+	// opaque identity blob so it can be unsealed with the customer's recovery code. Distinct from the
+	// operator-driven DR path in dr.go, which stays exactly as it is (see handleHostEscrowGet).
+	case r.Method == http.MethodGet && strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/escrow"):
+		hostID := strings.TrimSuffix(strings.TrimPrefix(path, "/hosts/"), "/escrow")
+		h.handleHostEscrowGet(w, r, hostID)
 	// G1 break-glass: day-0 vaults the root@pam console credential (self-scoped host key); the
 	// operator retrieves it via the /admin/ path (global key only).
 	case r.Method == http.MethodPut && strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/recovery-credential"):
@@ -1065,9 +1071,13 @@ func (h *Handler) reissueOnReenroll(cc *store.CustomerConfig) {
 		}
 	}
 	// F3 — offsite continuity: re-stage the one-time offsite password to the fresh controller (the
-	// one-time password only ever reached the OLD controller). The re-issuer resets the restic repo
-	// password, which makes the OLD escrow blob stale — the offsite provisioner invalidates the
-	// escrow (2.3) and emits both events. Skips silently when offsite isn't provisioned/enabled.
+	// one-time password only ever reached the OLD controller). ⚠ CORRECTED 2026-08-04 (R-196): this
+	// used to claim "the re-issuer resets the restic repo password, which makes the OLD escrow blob
+	// stale". It does not and cannot — the re-issuer resets the PROVIDER account password; the
+	// repository password is generated on the box and never leaves it except sealed under R. The
+	// provisioner does mark the escrow stale, but PRECAUTIONARILY (see the reasoning at
+	// offsite.ReissueCredentials), not because this call rotated anything.
+	// Skips silently when offsite isn't provisioned/enabled.
 	if h.offsiteReissuer != nil {
 		if err := h.offsiteReissuer(context.Background(), cc.CustomerID); err != nil {
 			h.logger.Printf("[WARN] offsite re-issue on re-enroll for %s failed: %v", cc.CustomerID, err)
@@ -1141,7 +1151,7 @@ func (h *Handler) handleHostEscrowPut(w http.ResponseWriter, r *http.Request, pa
 	// Store the OPAQUE bytes. No decrypt path exists — the hub cannot open this. Part B (v0.60.0):
 	// when this upload supersedes a DIFFERENT-passphrase old blob, the old one is RETAINED (not
 	// overwritten) so its recovery-code-recoverable history survives (Viktor's data-first ruling).
-	superseded, serr := h.store.SaveHostEscrow(pathHostID, blob, req.KeyFingerprint, req.Posture, createdAt, req.ResticPwSHA256)
+	superseded, prevPwSHA, serr := h.store.SaveHostEscrow(pathHostID, blob, req.KeyFingerprint, req.Posture, createdAt, req.ResticPwSHA256)
 	if serr != nil {
 		h.logger.Printf("[ERROR] Failed to store escrow for host %s: %v", pathHostID, serr)
 		http.Error(w, "Internal error", http.StatusInternalServerError)
@@ -1157,6 +1167,10 @@ func (h *Handler) handleHostEscrowPut(w http.ResponseWriter, r *http.Request, pa
 				"A korábbi helyreállítási csomag megőrizve (új kulcs érkezett).", string(details), "hub"); eerr != nil {
 				h.logger.Printf("[WARN] escrow_superseded event save failed for %s: %v", pathHostID, eerr)
 			}
+			// R-197: the box's offsite DATA key demonstrably changed. Both halves of that comparison
+			// have been stored since SLICE 3 and nothing read them — demo-felhom's key changed on
+			// 2026-08-03 and nothing said so for thirteen hours.
+			h.maybeEmitRepoKeyChanged(host.CustomerID, pathHostID, prevPwSHA, req.ResticPwSHA256, n)
 		}
 	}
 	// Slice 10D.1: optionally store the IDENTITY escrow blob + the non-secret DR directive alongside
@@ -1185,6 +1199,188 @@ func (h *Handler) handleHostEscrowPut(w http.ResponseWriter, r *http.Request, pa
 	w.Write([]byte(`{"status":"ok"}`))
 }
 
+// eventRepoKeyChanged (R-197) — the box's offsite restic REPOSITORY password changed, proven by the
+// hub's own stored hashes. Hub-internal (not in allowedEventTypes, like escrow_superseded) and
+// registered operator-only in notify.operatorOnlyEvents.
+const eventRepoKeyChanged = "offsite_repo_key_changed"
+
+// eventEscrowBlobServed (R-199) — a host retrieved its own sealed identity blob. Hub-internal,
+// operator-only. See handleHostEscrowGet for why every retrieval is loud.
+const eventEscrowBlobServed = "escrow_blob_served"
+
+// escrowSelfServiceRetrieval is THE SINGLE DECISION POINT for the §8.2/§8.3 trade (R-199).
+//
+//	true  (§8.2, shipped v0.94.0) — a host may read its own blob whenever it authenticates as itself.
+//	false (§8.3, the fallback)    — the same read additionally requires operator-armed recovery mode.
+//
+// It is one condition on purpose: the operator may overrule the trade below, and switching must cost a
+// boolean rather than a redesign. Everything else in the recovery chain is identical either way.
+const escrowSelfServiceRetrieval = true
+
+// handleHostEscrowGet serves a host its OWN opaque identity-escrow blob (R-199, v0.94.0).
+//
+// WHAT THIS GIVES OUT, WHY IT IS SAFE, AND WHAT IT CHANGES ABOUT WHO IS REQUIRED — recorded here so the
+// next reader finds the trade rather than inferring it (the dr.go header convention).
+//
+// WHAT: the age-wrapped `IdentityBundle` — opaque ciphertext. It carries the offsite restic repository
+// password, the tunnel token, the PBS token and the WG key. The hub stores these bytes and has no
+// decrypt path; the recovery code R that opens them exists only in the customer's hands.
+//
+// WHY IT IS SAFE TO GIVE OUT: the blob is useless without R (age scrypt + ChaCha20-Poly1305; a wrong R
+// fails closed at the KDF, never to a plausible-but-wrong bundle), and a 10-word EFF code carries ~129
+// bits. The caller already authenticates as this host for its report, its desired state, its WG
+// registration and its PBS token — this adds no new identity, only a new object, and it is the exact
+// MIRROR of the PUT above, which is how the blob got here in the first place.
+//
+// WHAT IT CHANGES, STATED PLAINLY BECAUSE IT IS THE WHOLE OF THE TRADE: before this, obtaining the blob
+// required the OPERATOR to arm recovery mode with the global key (dr.go). Now whoever controls a
+// rebuilt box can obtain it with that box's own credential. That is a real reduction in the number of
+// parties required. They still cannot open it. The mitigation is that the capability is AUDITED rather
+// than silent: every successful retrieval raises an operator event (below), because a silent capability
+// is the shape this project has spent two weeks removing.
+//
+// THE OPERATOR-DRIVEN DR PATH IS UNTOUCHED. `handleReEnroll` / `handleGetRestoreDirective` keep their
+// recovery-mode gate and their global-key arming, and they serve the K-escrow and the directive as
+// well. This endpoint serves ONE object to ONE authenticated owner. Do not merge them.
+func (h *Handler) handleHostEscrowGet(w http.ResponseWriter, r *http.Request, pathHostID string) {
+	authHostID, _, isGlobal, ok := h.checkAuthHost(r)
+	if !ok {
+		http.Error(w, "Unauthorized", http.StatusUnauthorized)
+		return
+	}
+	if pathHostID == "" {
+		http.Error(w, "Missing host_id", http.StatusBadRequest)
+		return
+	}
+	// SELF-SCOPED: a per-host key reads only its OWN escrow. The global operator key may read any —
+	// the same asymmetry the PUT has. Without this line any host key is a fleet-wide blob reader.
+	if !isGlobal && authHostID != pathHostID {
+		h.logger.Printf("[WARN] escrow GET REFUSED: host %s asked for %s's blob (self-scope)", authHostID, pathHostID)
+		http.Error(w, "Forbidden: host_id mismatch", http.StatusForbidden)
+		return
+	}
+	host, err := h.store.GetHost(pathHostID)
+	if err != nil {
+		http.Error(w, "Internal error", http.StatusInternalServerError)
+		return
+	}
+	if host == nil {
+		http.Error(w, "Unknown host_id", http.StatusNotFound)
+		return
+	}
+	// §8.3 fallback lives here and nowhere else.
+	if !escrowSelfServiceRetrieval && !host.InRecoveryMode(time.Now().UTC()) {
+		h.logger.Printf("[WARN] escrow GET REFUSED for %s — self-service retrieval is disabled and recovery mode is not armed", pathHostID)
+		http.Error(w, "Forbidden: host not in recovery mode (operator must arm it)", http.StatusForbidden)
+		return
+	}
+
+	bundle, berr := h.store.GetHostDRBundle(pathHostID)
+	if berr != nil {
+		h.logger.Printf("[ERROR] escrow GET for %s: %v", pathHostID, berr)
+		http.Error(w, "Internal error", http.StatusInternalServerError)
+		return
+	}
+	// A host with no sealed bundle gets a CLEAN ANSWER, not a fault: 200 with present=false. A 404
+	// here would be indistinguishable from an unknown host, and an empty 200 without the flag would be
+	// indistinguishable from a zero-length blob — both read as "something is broken" to a caller whose
+	// situation is simply "no ceremony has run yet".
+	if bundle == nil || len(bundle.IdentityBlob) == 0 {
+		h.logger.Printf("[INFO] escrow GET for %s: no identity blob stored (no ceremony has run)", pathHostID)
+		writeJSON(w, http.StatusOK, map[string]any{"host_id": pathHostID, "present": false, "identity_escrow_b64": ""})
+		return
+	}
+
+	// THE MITIGATION (§8.2). Recorded BEFORE the bytes leave, so a retrieval cannot be served without
+	// its audit row; a save failure is logged and does NOT block the response (the blob is opaque and
+	// refusing it would break a recovery over an audit hiccup — but the log line always exists).
+	//
+	// SEVERITY = warning, i.e. it reaches the operator by e-mail. Retrieval is not routine today: it
+	// happens during a recovery and nowhere else. IF a customer-facing self-service flow ever makes it
+	// routine, revisit this — but revisit it deliberately, do not let it decay to info because the
+	// mail became annoying.
+	if host.CustomerID != "" {
+		msg := fmt.Sprintf("Recovery blob served: host %s retrieved its own sealed identity escrow (%d opaque bytes). "+
+			"This is the recovery path in use — the blob cannot be opened without the customer's recovery code, which the hub never holds. "+
+			"If no recovery is in progress on that box, investigate.", pathHostID, len(bundle.IdentityBlob))
+		details, _ := json.Marshal(map[string]any{
+			"host_id":    pathHostID,
+			"blob_bytes": len(bundle.IdentityBlob),
+			"self_scope": !isGlobal,
+		})
+		if _, eerr := h.store.SaveEvent(host.CustomerID, eventEscrowBlobServed, "warning", msg, string(details), "hub"); eerr != nil {
+			h.logger.Printf("[WARN] %s event save FAILED for %s (serving anyway): %v", eventEscrowBlobServed, pathHostID, eerr)
+		} else if h.dispatcher != nil {
+			go h.dispatcher.ProcessEvent(host.CustomerID, eventEscrowBlobServed, "warning", msg, string(details), "hub")
+		}
+	}
+	h.logger.Printf("[WARN] escrow blob SERVED to host %s (%d opaque bytes, self_scope=%v) — recovery path in use",
+		pathHostID, len(bundle.IdentityBlob), !isGlobal)
+	writeJSON(w, http.StatusOK, map[string]any{
+		"host_id":             pathHostID,
+		"present":             true,
+		"identity_escrow_b64": base64.StdEncoding.EncodeToString(bundle.IdentityBlob),
+	})
+}
+
+// maybeEmitRepoKeyChanged raises ONE operator signal per supersession when the sealed offsite repo
+// password demonstrably changed. Both hashes have been stored since SLICE 3 (host_escrow and, since
+// v0.60.0, host_escrow_superseded) and NOTHING compared them: demo-felhom's repository password
+// changed on 2026-08-03, orphaning 36 snapshots / 1.14 GB, and no event, e-mail, card or log line
+// said so for thirteen hours — the comparison that eventually found it is this one
+// (audits/RECON-offsite-dr-chain-2026-08-04.md, R-197).
+//
+// THE PREDICATE IS DELIBERATELY NARROW: both hashes known AND different. A first-ever hash (prev "")
+// is onboarding, not a change; a hash-less supersession (now "") cannot show a change happened; an
+// identical hash is a re-ceremony of the SAME password, which is a normal healthy act and must stay
+// silent or a customer is punished for re-running a ceremony. The in-between shapes are LOGGED rather
+// than dropped, so "we chose not to alarm" and "the check did not run" never look identical.
+//
+// SEVERITY = warning, chosen for the world v0.93.0 creates rather than the one it inherits. Before
+// R-198 a changed key meant the previous history was unopenable by anyone, ever — that would have
+// argued for error. From v0.93.0 the superseding ceremony RETAINS the old identity blob, so the
+// previous history stays recoverable with the recovery code that sealed it: the situation is "this
+// customer's off-site history now depends on an older recovery code", which is operator-actionable
+// (check the orphan card, expect a fresh repository) and is not a loss. warning also routes: the
+// dispatcher notifies on warning/error/critical and treats info as an intentional non-notify, and
+// the whole point of this row is that the operator learns on the day.
+//
+// EDGE-TRIGGERED: called only from the superseded branch of the escrow PUT, i.e. once per
+// supersession, never per report. No timer lives here — the dispatcher owns cooldown.
+//
+// NO HASH VALUE TRAVELS. The message and the details name the host, the customer and the retained
+// count only. The hashes are non-reversible, but a hash is still a fingerprint of a live secret and
+// this project's rule is that values do not leave the store.
+func (h *Handler) maybeEmitRepoKeyChanged(customerID, hostID, prevSHA, newSHA string, retained int) {
+	switch {
+	case prevSHA == "":
+		h.logger.Printf("[INFO] escrow for host %s: no previous repo-password hash recorded (first hash, or a legacy hash-less blob) — repo-key-change check not applicable", hostID)
+		return
+	case newSHA == "":
+		h.logger.Printf("[WARN] escrow for host %s: the NEW blob carries no repo-password hash (hash-less supersession) — whether the repository key changed CANNOT be determined from the hub's data", hostID)
+		return
+	case prevSHA == newSHA:
+		return // same password re-sealed: a healthy re-ceremony (Scenario E — silence is correct)
+	}
+	msg := fmt.Sprintf("Offsite repository key CHANGED for host %s: the new escrow seals a different repository password than the one it replaced. "+
+		"The previous off-site history is no longer opened by this box's current key. The superseding blob was retained (%d held), so that history stays recoverable with the recovery code that sealed it — verify the box's off-site tier reports a repository rather than an orphan card, and expect the next backup to start a fresh history.",
+		hostID, retained)
+	details, _ := json.Marshal(map[string]any{
+		"host_id":        hostID,
+		"retained_count": retained,
+		"repo_key":       "changed", // never the hash values
+	})
+	if _, err := h.store.SaveEvent(customerID, eventRepoKeyChanged, "warning", msg, string(details), "hub"); err != nil {
+		h.logger.Printf("[WARN] %s event save failed for %s: %v", eventRepoKeyChanged, hostID, err)
+		return // audit row first: an e-mail without its event row lies (the OffsiteChecker convention)
+	}
+	h.logger.Printf("[WARN] offsite repository key CHANGED for host %s (customer %s) — previous history now depends on the superseded recovery code; %d retained blob(s)",
+		hostID, customerID, retained)
+	if h.dispatcher != nil {
+		go h.dispatcher.ProcessEvent(customerID, eventRepoKeyChanged, "warning", msg, string(details), "hub")
+	}
+}
+
 // handleHostRecoveryCredentialPut vaults a host's break-glass root@pam console credential (TASK G1).
 // SELF-SCOPED (a host key writes only its own; global may write any) — day-0 posts it with the
 // host api_key. The secret is stored at rest and NEVER logged (only the username + a length are
@@ -1586,6 +1782,11 @@ var allowedEventTypes = map[string]bool{
 	// OPERATOR-ONLY IS ENFORCED BY `notify.operatorOnlyEvents` — see the paragraph above. This entry
 	// alone does NOT make it operator-only.
 	"recovery_unit_capture_failed": true,
+	// R-182. The per-run backup digest: one event at the end of a run, listing every app whose
+	// backup failed or was refused. Allowlisting it is NOT what keeps it away from customers —
+	// `notify.operatorOnlyEvents` is (see the comment there); both entries ship together and
+	// `backup_run_digest_event_test.go` pins the pair.
+	"backup_run_failures": true,
 
 	// Controller-pushed events
 	"controller_started":      true,

hub/internal/monitor/deadline.go

@@ -336,7 +336,7 @@ func CheckBackupDeadlines(s *store.Store, staleness *StalenessChecker, onEvent E
 	midnightBudapest := time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, budapest)
 	sinceUTC := midnightBudapest.UTC()
 
-	var backupMissed, dbdumpMissed, skipped, deferred int
+	var backupMissed, dbdumpMissed, skipped, deferred, unbound int
 
 	for _, id := range customerIDs {
 		// Skip nodes that are down — they already have staleness events
@@ -350,6 +350,37 @@ func CheckBackupDeadlines(s *store.Store, staleness *StalenessChecker, onEvent E
 			continue
 		}
 
+		// ── R-195: a customer with NO machine EVER bound is UNKNOWN, not missed ────────────────
+		//
+		// Both verdicts below ask "did the thing we expect every day happen?". For a customer
+		// that has never had a machine bound, nothing has ever been expected, so the honest
+		// answer is UNKNOWN — the same invariant assessBackupFreshness states above, applied one
+		// level up, at the question of whether there is a subject at all.
+		//
+		// The discriminator is "was a host EVER bound", NOT "has a report arrived". That is the
+		// case this check must not break: a box that was installed, bound, and then went silent
+		// has a real fault and must keep alarming. It is bound, so it is judged.
+		//
+		// WHY THIS WAS REACHABLE AT ALL, measured 2026-08-04: the down-skip above is what
+		// protects every other silent customer, and it reads the staleness checker's state — which
+		// is seeded from the `reports` table (store.GetCustomers). A customer that has NEVER
+		// reported appears in no report row, so it gets no staleness state at all and GetState()
+		// returns "" rather than "down". The skip misses exactly the customer it would most
+		// obviously cover, and the DB-dump half below then fires every night: `david`, a
+		// prospective customer whose record was created 2026-08-01 with no machine ever bound,
+		// e-mailed an expected_dbdump_missed ERROR at 03:00 UTC on three consecutive days.
+		//
+		// Fail-open on a read error: an unreadable binding must never SUPPRESS a real alarm.
+		if bound, berr := s.HasEverBoundHost(id); berr != nil {
+			logger.Printf("[WARN] Deadline check: failed to read host binding for %s (judging anyway): %v", id, berr)
+		} else if !bound {
+			// Visible, per the v0.73.0 Part-7 precedent below: a quiet check must never be
+			// indistinguishable from a check that did not run. Once daily, one line per customer.
+			logger.Printf("[INFO] Deadline check: %s has no host EVER bound — all deadline verdicts UNKNOWN (no alarm)", id)
+			unbound++
+			continue
+		}
+
 		// Backup freshness from the agent's host-report (PBS snapshots + vzdump),
 		// the authoritative offsite-backup signal post-slice-8C.
 		reportJSON, rerr := s.GetLatestHostReportJSON(id)
@@ -423,6 +454,6 @@ func CheckBackupDeadlines(s *store.Store, staleness *StalenessChecker, onEvent E
 		}
 	}
 
-	logger.Printf("[INFO] Deadline check: %d customers, %d backup missed, %d backup unknown (deferred), %d dbdump missed, %d skipped (down)",
-		len(customerIDs), backupMissed, deferred, dbdumpMissed, skipped)
+	logger.Printf("[INFO] Deadline check: %d customers, %d backup missed, %d backup unknown (deferred), %d dbdump missed, %d skipped (down), %d unknown (no host ever bound)",
+		len(customerIDs), backupMissed, deferred, dbdumpMissed, skipped, unbound)
 }

hub/internal/monitor/deadline_unbound_test.go

@@ -0,0 +1,148 @@
+package monitor
+
+import (
+	"io"
+	"log"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-hub/internal/store"
+)
+
+// R-195 — a customer with NO machine EVER bound must not alarm; a customer WITH one must.
+//
+// Origin, measured on the live hub 2026-08-04: `david` is a prospective customer whose record was
+// created 2026-08-01 16:51:49 with no host ever bound (hosts=0, host_deletions=0, host_reports=0,
+// reports=0). It e-mailed an `expected_dbdump_missed` ERROR at 03:00 UTC on 08-02, 08-03 and 08-04.
+//
+// The mechanism, established at source: the down-skip in CheckBackupDeadlines reads
+// StalenessChecker.GetState(), whose map is seeded from store.GetCustomers() — a query over the
+// `reports` table. A customer with zero reports is in no row, so it has no state, and GetState()
+// returns "" rather than "down". The skip that protects every other silent customer misses the one
+// that never reported at all.
+//
+// These tests pin BOTH halves. A suite that only proved the silence would pass against an
+// implementation that never alarms, which is strictly worse than the defect it replaces.
+
+// newUnboundStore creates a store holding ONE active customer and NO host row at all.
+func newUnboundStore(t *testing.T) *store.Store {
+	t.Helper()
+	st, err := store.New(filepath.Join(t.TempDir(), "test.db"), log.New(io.Discard, "", 0))
+	if err != nil {
+		t.Fatalf("store.New: %v", err)
+	}
+	t.Cleanup(func() { st.Close() })
+	if err := st.SaveCustomerConfig(&store.CustomerConfig{CustomerID: "c1", APIKey: "ck", RetrievalPassword: "p"}); err != nil {
+		t.Fatalf("SaveCustomerConfig: %v", err)
+	}
+	return st
+}
+
+// ── Half 1: the silence ────────────────────────────────────────────────────────────────────────
+
+// TestCheckBackupDeadlines_NeverBoundHost_Silent is the david case.
+//
+// COMPANION RED-PROOF (observed): deleting the HasEverBoundHost guard from CheckBackupDeadlines
+// makes this test fail with
+//
+//	deadline_unbound_test.go: a customer with NO host ever bound must raise NOTHING;
+//	got [expected_dbdump_missed]
+//
+// which is verbatim the event `david` e-mailed three mornings running. Restored after.
+func TestCheckBackupDeadlines_NeverBoundHost_Silent(t *testing.T) {
+	st := newUnboundStore(t)
+	// No UpsertHost, no host-report, no db_dump_completed event — nothing has ever been expected.
+	got := runDeadline(t, st)
+	if len(got) != 0 {
+		t.Fatalf("a customer with NO host ever bound must raise NOTHING; got %v", got)
+	}
+}
+
+// TestHasEverBoundHost pins the predicate itself across its three inputs, because the whole
+// behaviour above turns on it and a predicate that answered `false` for everything would make the
+// test above pass while silencing the entire fleet.
+func TestHasEverBoundHost(t *testing.T) {
+	st := newUnboundStore(t)
+
+	if bound, err := st.HasEverBoundHost("c1"); err != nil || bound {
+		t.Fatalf("no host rows → want (false,nil); got (%v,%v)", bound, err)
+	}
+
+	if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+		t.Fatalf("UpsertHost: %v", err)
+	}
+	if bound, err := st.HasEverBoundHost("c1"); err != nil || !bound {
+		t.Fatalf("live host row → want (true,nil); got (%v,%v)", bound, err)
+	}
+
+	// An unknown customer is never bound — the predicate must not answer from another customer's rows.
+	if bound, err := st.HasEverBoundHost("nobody"); err != nil || bound {
+		t.Fatalf("unknown customer → want (false,nil); got (%v,%v)", bound, err)
+	}
+}
+
+// ── Half 2: THE RED-PROOF THAT MATTERS — a bound machine must still alarm ───────────────────────
+
+// TestCheckBackupDeadlines_BoundButNeverReported_StillAlarms is the case the change could break,
+// and it is a real shape: a machine that was installed and bound and never phoned home. It has a
+// `hosts` row and zero reports — indistinguishable from `david` on every signal EXCEPT the one the
+// guard discriminates on. If the guard is ever "simplified" to key off report presence, customer
+// age or a name pattern, this test goes red.
+func TestCheckBackupDeadlines_BoundButNeverReported_StillAlarms(t *testing.T) {
+	st := newUnboundStore(t)
+	if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+		t.Fatalf("UpsertHost: %v", err)
+	}
+	// No host-report and no db_dump_completed — the box was bound and never said anything.
+	got := runDeadline(t, st)
+	if !has(got, "expected_dbdump_missed") {
+		t.Fatalf("a BOUND machine that never reported is a real fault and must still alarm; got %v", got)
+	}
+}
+
+// TestCheckBackupDeadlines_BoundThenWentQuiet_StillAlarms: the machine reported once, days ago,
+// and stopped. Nothing about the guard may suppress that.
+//
+// The staleness checker's down-skip is nil here (runDeadline passes nil), which is deliberate: it
+// isolates THIS guard. In production a genuinely down node is skipped by staleness and gets its own
+// node_down event — that path is unchanged and is not what this test is about.
+func TestCheckBackupDeadlines_BoundThenWentQuiet_StillAlarms(t *testing.T) {
+	st := newUnboundStore(t)
+	if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+		t.Fatalf("UpsertHost: %v", err)
+	}
+	// One host-report whose newest backup evidence is 9 days old, and a db dump that last
+	// completed 5 days ago (i.e. not since midnight).
+	report := hostReportJSON(t, [][2]string{{rfc(-9 * 24 * time.Hour), "ok"}}, nil)
+	if err := st.SaveHostReport("h1", "c1", []byte(report), store.HostReportDenorm{}); err != nil {
+		t.Fatalf("SaveHostReport: %v", err)
+	}
+	got := runDeadline(t, st)
+	if !has(got, "expected_backup_missed") {
+		t.Fatalf("a bound machine that went quiet with stale backups must still raise expected_backup_missed; got %v", got)
+	}
+	if !has(got, "expected_dbdump_missed") {
+		t.Fatalf("a bound machine that went quiet must still raise expected_dbdump_missed; got %v", got)
+	}
+}
+
+// TestCheckBackupDeadlines_DeletedHost_StillJudged: the host row is gone but a tombstone remains
+// (peti-felhom's live shape). The customer HAD a machine, so this check must not take over the
+// judgement — it hands off to the staleness down-skip exactly as before the change.
+func TestCheckBackupDeadlines_DeletedHost_StillJudged(t *testing.T) {
+	st := newUnboundStore(t)
+	if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+		t.Fatalf("UpsertHost: %v", err)
+	}
+	if err := st.DeleteHost("h1", false); err != nil {
+		t.Fatalf("DeleteHost: %v", err)
+	}
+	if bound, err := st.HasEverBoundHost("c1"); err != nil || !bound {
+		t.Fatalf("a DELETED host is still a machine that was once bound → want (true,nil); got (%v,%v)", bound, err)
+	}
+	got := runDeadline(t, st)
+	if !has(got, "expected_dbdump_missed") {
+		t.Fatalf("a customer whose host was deleted was still bound and stays judged here; got %v", got)
+	}
+}

hub/internal/monitor/offsite_delivery.go

@@ -94,40 +94,112 @@ func (c *OffsiteDeliveryChecker) Check() {
 		if age < stuckAfter {
 			continue // normal convergence window
 		}
-		c.maybeEmitStuck(cfg.CustomerID, status, age)
-		c.maybeHeal(cfg.CustomerID, status)
+		emitted := c.maybeEmitStuck(cfg.CustomerID, status, age)
+		c.maybeHeal(cfg.CustomerID, status, emitted)
 	}
 }
 
-// maybeEmitStuck emits offsite_delivery_stuck (warning) once per stuckCooldown per customer.
-func (c *OffsiteDeliveryChecker) maybeEmitStuck(customerID string, status offsite.DeliveryStatus, age time.Duration) {
+// deliveryShape names the two situations the ONE stuck state actually covers. They need different
+// text and different advice, and conflating them is R-192's defect (a).
+type deliveryShape string
+
+const (
+	// shapeBurned — NO report since the consume carried an offbox target: the apply never persisted.
+	// Re-issue is the indicated action.
+	shapeBurned deliveryShape = "burned"
+	// shapeRegressed — reports since the consume DID carry an offbox target and the latest does not:
+	// the credential worked and the target was later lost (a guest rebuild does exactly this, R-193).
+	// Re-issue is NOT indicated; it treats a symptom whose cause is elsewhere.
+	shapeRegressed deliveryShape = "regressed"
+)
+
+func shapeOf(status offsite.DeliveryStatus) deliveryShape {
+	if status.OffsiteReportsSinceConsume == 0 {
+		return shapeBurned
+	}
+	return shapeRegressed
+}
+
+// maybeEmitStuck emits offsite_delivery_stuck (warning) once per stuckCooldown per customer. Returns
+// whether it emitted, so the heal's refusal record rides the same cadence rather than inventing one.
+//
+// R-192 defect (a), fixed here: the message used to interpolate ReportsSinceConsume (the TOTAL) into
+// a hardcoded phrase "report(s) since carry no offbox target", and never consulted
+// OffsiteReportsSinceConsume — the field that says the opposite. On demo-hp it stated, daily, that
+// 500 reports carried no offbox target when all 500 of them did, and prescribed Re-issue for a
+// failure mode that had not occurred. The message now STATES WHAT WAS MEASURED and lets the operator
+// read it; the recommendation follows the shape rather than being hardcoded.
+//
+// THE WINDOW IS NAMED ON PURPOSE. CountReportsOffsiteSince reads `ORDER BY id LIMIT 500` — the OLDEST
+// 500 reports after the consume, not the newest — so on a long-lived customer these counts describe
+// the beginning of the window and not the present. That is a real scoping defect (R-192's other half)
+// and it stays OPEN because its correct shape depends on the recovery chain that is not yet
+// assembled (R-199/R-200/R-201). Naming the window in the text is how it stays visible instead of
+// being laundered into a confident sentence — an instrument that can silently mis-scope its results
+// must say so where it reports them.
+func (c *OffsiteDeliveryChecker) maybeEmitStuck(customerID string, status offsite.DeliveryStatus, age time.Duration) bool {
 	last, err := c.store.LastEventAt(customerID, eventDeliveryStuck)
 	if err != nil {
 		c.logger.Printf("[WARN] offsite-delivery: %s: cooldown read: %v", customerID, err)
-		return
+		return false
 	}
 	if !last.IsZero() && c.now().Sub(last) < stuckCooldown {
-		return
+		return false
+	}
+	shape := shapeOf(status)
+	var msg string
+	switch shape {
+	case shapeBurned:
+		msg = fmt.Sprintf("Offsite delivery stuck (BURNED-credential shape): the one-time password was consumed %s ago; of the first %d report(s) after that consume, NONE carried an offbox target, and the latest report carries none either. The credential never reached a persisted apply. Re-issue delivers a fresh one.",
+			age.Round(time.Minute), status.ReportsSinceConsume)
+	default:
+		msg = fmt.Sprintf("Offsite delivery stuck (REGRESSED-apply shape): the one-time password was consumed %s ago; of the first %d report(s) after that consume, %d DID carry an offbox target — and the latest report carries none. The credential was applied and worked; the target was lost afterwards. Re-issue is NOT the indicated action: find what removed the offbox target (a guest rebuild does, R-193). Automatic restage is deliberately withheld for this shape. NOTE: the counts cover at most the first 500 reports after the consume, so on a long-lived box they describe the start of the window, not now (R-192, open).",
+			age.Round(time.Minute), status.ReportsSinceConsume, status.OffsiteReportsSinceConsume)
 	}
-	msg := fmt.Sprintf("Offsite delivery stuck: one-time password consumed %s ago and %d report(s) since carry no offbox target — the credential is likely burned (apply died between consume and persist). Re-issue delivers a fresh one.",
-		age.Round(time.Minute), status.ReportsSinceConsume)
 	details, _ := json.Marshal(map[string]any{
-		"state":                 string(status.State),
-		"consumed_at":           status.Since.UTC().Format(time.RFC3339),
-		"reports_since_consume": status.ReportsSinceConsume,
+		"state":                         string(status.State),
+		"shape":                         string(shape),
+		"consumed_at":                   status.Since.UTC().Format(time.RFC3339),
+		"reports_since_consume":         status.ReportsSinceConsume,
+		"offsite_reports_since_consume": status.OffsiteReportsSinceConsume,
+		"count_window":                  "oldest 500 reports after consumed_at (R-192, open)",
 	})
 	c.emit(customerID, eventDeliveryStuck, "warning", msg, string(details))
+	return true
 }
 
 // maybeHeal fires the R-71c self-heal when the burned-credential shape is unambiguous:
 // consumed ≥ stuckAfter ago, ≥ healMinReports consecutive reports since with ZERO offbox evidence,
 // one heal per healCooldown — and the R-39(a) guard holds at act time.
-func (c *OffsiteDeliveryChecker) maybeHeal(customerID string, status offsite.DeliveryStatus) {
+//
+// R-192 defect (b), fixed here: every refusal above the "not configured" line now leaves a RECORD.
+// The regressed-shape branch used to be a bare `return`, so the operator received a daily e-mail with
+// the wrong story, no heal, and nothing anywhere saying why the heal declined — "we chose not to act"
+// and "the heal never ran" looked identical. `offsite_credential_restaged` has never fired for any
+// customer, and until now that fact was indistinguishable from the checker being dead.
+//
+// The record is a notification_log row (the dispatcher's suppressed-operator-e-mail precedent, R-182:
+// a decision not to act is written down on the channel it would have used). It rides `recordRefusal`
+// — true only when the stuck event was emitted this pass — so it appears once per stuckCooldown
+// beside the e-mail it explains, rather than once per monitor tick. The GUARD ITSELF IS UNCHANGED:
+// the set of situations in which the heal fires is byte-for-byte what it was; only the silence is
+// gone. The two conditions are split into separate branches solely so each refusal can name its own
+// reason.
+func (c *OffsiteDeliveryChecker) maybeHeal(customerID string, status offsite.DeliveryStatus, recordRefusal bool) {
 	if c.reissuer == nil {
+		return // no provisioner configured: the heal does not exist on this hub, so there is nothing to explain
+	}
+	if status.OffsiteReportsSinceConsume != 0 {
+		c.recordHealRefusal(customerID, recordRefusal, fmt.Sprintf(
+			"regressed-apply shape: %d of the first %d report(s) after the consume DID carry an offbox target, so a burned credential is ruled out — a restage would treat a symptom whose cause is elsewhere. Operator's call (R-193).",
+			status.OffsiteReportsSinceConsume, status.ReportsSinceConsume))
 		return
 	}
-	if status.ReportsSinceConsume < healMinReports || status.OffsiteReportsSinceConsume != 0 {
-		return // box not reporting enough, or offbox evidence exists (regressed-apply shape) → operator's call
+	if status.ReportsSinceConsume < healMinReports {
+		c.recordHealRefusal(customerID, recordRefusal, fmt.Sprintf(
+			"only %d report(s) since the consume (need %d): the box has not reported enough for the burned shape to be unambiguous.",
+			status.ReportsSinceConsume, healMinReports))
+		return
 	}
 	last, err := c.store.LastEventAt(customerID, eventCredentialRestaged)
 	if err != nil {
@@ -146,8 +218,8 @@ func (c *OffsiteDeliveryChecker) maybeHeal(customerID string, status offsite.Del
 		return
 	}
 	if info == nil || info.ConsumedAt.IsZero() {
-		c.logger.Printf("[INFO] offsite-delivery: %s: heal refused — secret row is now %s (R-39(a) guard: never restage over an unconsumed secret)",
-			customerID, secretShape(info))
+		c.recordHealRefusal(customerID, recordRefusal, fmt.Sprintf(
+			"R-39(a) guard: the secret row is now %s — restaging over an unconsumed secret would clobber a password the box may be about to consume.", secretShape(info)))
 		return
 	}
 	ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
@@ -167,6 +239,22 @@ func (c *OffsiteDeliveryChecker) maybeHeal(customerID string, status offsite.Del
 		string(details))
 }
 
+// recordHealRefusal makes a decision NOT to self-heal visible. Always logs; additionally writes a
+// notification_log row on the "operator" channel with status "refused" when `record` is set (the
+// stuck event was emitted this pass), so the refusal sits next to the e-mail that prompted the
+// question. A LogNotification failure is logged, never swallowed, and never blocks the refusal — the
+// refusal is the primary effect.
+func (c *OffsiteDeliveryChecker) recordHealRefusal(customerID string, record bool, reason string) {
+	c.logger.Printf("[INFO] offsite-delivery: %s: self-heal REFUSED — %s", customerID, reason)
+	if !record {
+		return
+	}
+	if err := c.store.LogNotification(customerID, eventCredentialRestaged, "warning",
+		"Automatic offsite credential restage was NOT performed.", "refused", reason, "operator"); err != nil {
+		c.logger.Printf("[WARN] offsite-delivery: %s: could not record the heal refusal: %v", customerID, err)
+	}
+}
+
 func secretShape(info *store.OneTimeSecretInfo) string {
 	if info == nil {
 		return "absent"

hub/internal/monitor/offsite_delivery_honesty_test.go

@@ -0,0 +1,223 @@
+package monitor
+
+import (
+	"encoding/json"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-hub/internal/store"
+)
+
+// R-192 (v0.93.0) — the alert must describe what was MEASURED, and a refusal to self-heal must leave a
+// trace.
+//
+// What was wrong, quoted from the live e-mail demo-hp sent daily from 2026-08-03: "one-time password
+// consumed 284h19m0s ago and 500 report(s) since carry no offbox target — the credential is likely
+// burned … Re-issue delivers a fresh one." OffsiteReportsSinceConsume for that same call was 500:
+// every one of those reports carried a target. The message stated the precise negation of its own
+// measurement and prescribed a remedy for a failure mode that had not occurred — while the self-heal
+// correctly declined, in complete silence.
+
+// regressedFixture puts c1 into demo-hp's real shape: consumed >1h ago, reports WITH an offbox target
+// after the consume, then reports WITHOUT — the latest carrying none.
+//
+// The report timestamps get jitter deliberately: a perfectly regular series lands exactly on
+// boundaries and survives mutations that a realistic one catches.
+func (h dtHarness) regressedFixture(t *testing.T, withOffsite, without int) {
+	t.Helper()
+	if err := h.st.SaveOneTimeSecret("c1", "x"); err != nil {
+		t.Fatal(err)
+	}
+	consumed := time.Now().UTC().Add(-30 * time.Hour).Format("2006-01-02 15:04:05")
+	staged := time.Now().UTC().Add(-31 * time.Hour).Format("2006-01-02 15:04:05")
+	if err := h.st.SetOneTimeSecretTimesForTest("c1", staged, consumed); err != nil {
+		t.Fatal(err)
+	}
+	jitter := []int{0, 3, 1, 7, 2, 5, 11, 4}
+	for i := 0; i < withOffsite; i++ {
+		if err := h.st.SaveReport("c1", []byte(dtReportWithOffsite)); err != nil {
+			t.Fatal(err)
+		}
+		time.Sleep(time.Duration(jitter[i%len(jitter)]) * time.Millisecond)
+	}
+	for i := 0; i < without; i++ {
+		if err := h.st.SaveReport("c1", []byte(dtReportNoOffsite)); err != nil {
+			t.Fatal(err)
+		}
+		time.Sleep(time.Duration(jitter[(i+3)%len(jitter)]) * time.Millisecond)
+	}
+}
+
+func (h dtHarness) refusalRows(t *testing.T) []store.NotificationLogEntry {
+	t.Helper()
+	all, err := h.st.GetRecentNotifications("c1", 50)
+	if err != nil {
+		t.Fatal(err)
+	}
+	var out []store.NotificationLogEntry
+	for _, n := range all {
+		if n.Status == "refused" {
+			out = append(out, n)
+		}
+	}
+	return out
+}
+
+// Scenario F — the message states the measured state, not a denied cause.
+// RED-PROOF: restore the old single hardcoded sentence (interpolate ReportsSinceConsume into
+// "report(s) since carry no offbox target") → the regressed assertions FAIL.
+func TestDeliveryChecker_RegressedShape_MessageMatchesMeasurement(t *testing.T) {
+	h := newDTHarness(t, true)
+	h.regressedFixture(t, 6, 2)
+
+	h.checker.Check()
+
+	saved := h.savedEvents(t, "offsite_delivery_stuck")
+	if len(saved) != 1 {
+		t.Fatalf("stuck events = %d, want 1", len(saved))
+	}
+	msg := saved[0].Message
+
+	// It must NOT assert the thing the data contradicts.
+	if strings.Contains(msg, "carry no offbox target") || strings.Contains(msg, "likely burned") {
+		t.Fatalf("R-192(a): the message still asserts a burned credential on a REGRESSED shape — this is "+
+			"the sentence that stated the negation of its own measurement:\n%s", msg)
+	}
+	// It must name the shape it detected and the measurement behind it.
+	if !strings.Contains(msg, "REGRESSED") {
+		t.Errorf("the message must name the detected shape, got:\n%s", msg)
+	}
+	if !strings.Contains(msg, "6 DID carry an offbox target") {
+		t.Errorf("the message must state the measured offbox-bearing count (6), got:\n%s", msg)
+	}
+	// It must not prescribe the action for a failure mode that did not occur.
+	if !strings.Contains(msg, "Re-issue is NOT the indicated action") {
+		t.Errorf("the message must withdraw the Re-issue recommendation for this shape, got:\n%s", msg)
+	}
+	// The counting window is named, because CountReportsOffsiteSince reads the OLDEST 500 — an
+	// instrument that can silently mis-scope must say so where it reports (R-192's other half, open).
+	if !strings.Contains(msg, "500") {
+		t.Errorf("the message must name its counting window, got:\n%s", msg)
+	}
+
+	var details map[string]any
+	if err := json.Unmarshal([]byte(saved[0].DetailsJSON), &details); err != nil {
+		t.Fatal(err)
+	}
+	if details["shape"] != "regressed" {
+		t.Errorf("details.shape = %v, want regressed", details["shape"])
+	}
+	// The field whose absence was defect (a) is now carried.
+	if got, ok := details["offsite_reports_since_consume"].(float64); !ok || int(got) != 6 {
+		t.Errorf("details must carry offsite_reports_since_consume=6, got %v", details["offsite_reports_since_consume"])
+	}
+}
+
+// The burned shape keeps its own (correct) text and its recommendation — the fix must not blunt the
+// case that was always right.
+func TestDeliveryChecker_BurnedShape_KeepsItsRecommendation(t *testing.T) {
+	h := newDTHarness(t, false)
+	h.burnedFixture(t, 5)
+
+	h.checker.Check()
+
+	saved := h.savedEvents(t, "offsite_delivery_stuck")
+	if len(saved) != 1 {
+		t.Fatalf("stuck events = %d, want 1", len(saved))
+	}
+	msg := saved[0].Message
+	if !strings.Contains(msg, "BURNED") || !strings.Contains(msg, "NONE carried an offbox target") {
+		t.Errorf("the burned shape must state its own measurement, got:\n%s", msg)
+	}
+	if !strings.Contains(msg, "Re-issue delivers a fresh one") {
+		t.Errorf("the burned shape must keep its recommendation, got:\n%s", msg)
+	}
+	var details map[string]any
+	_ = json.Unmarshal([]byte(saved[0].DetailsJSON), &details)
+	if details["shape"] != "burned" {
+		t.Errorf("details.shape = %v, want burned", details["shape"])
+	}
+}
+
+// Scenario G — a refusal to self-heal leaves a durable record naming its reason.
+// RED-PROOF: replace the recordHealRefusal call in the regressed branch with a bare `return`
+// (production behaviour up to v0.92.0) → no row → this FAILS.
+func TestDeliveryChecker_HealRefusal_LeavesARecord(t *testing.T) {
+	h := newDTHarness(t, true)
+	h.regressedFixture(t, 6, 2)
+
+	h.checker.Check()
+
+	if h.reissuer.count() != 0 {
+		t.Fatal("the heal must NOT fire on a regressed shape — the guard is unchanged by this fix")
+	}
+	rows := h.refusalRows(t)
+	if len(rows) != 1 {
+		t.Fatalf("R-192(b): the heal declined and left %d record(s), want 1 — 'we chose not to act' and "+
+			"'the heal never ran' must not look identical", len(rows))
+	}
+	r := rows[0]
+	if r.Channel != "operator" {
+		t.Errorf("refusal channel = %q, want operator (the channel the heal's own event would use)", r.Channel)
+	}
+	if !strings.Contains(r.ErrorMessage, "regressed-apply shape") {
+		t.Errorf("the record must name WHY, got %q", r.ErrorMessage)
+	}
+	if !strings.Contains(r.ErrorMessage, "6 of the first 8") {
+		t.Errorf("the record must carry the measurement it refused on, got %q", r.ErrorMessage)
+	}
+}
+
+// The R-39(a) guard's refusal is recorded too — it was the one refusal that already logged, and it
+// still must not be the only one that does.
+//
+// Reaching it needs the real TOCTOU, not a pre-staged row: staging BEFORE Check() makes the derive
+// itself return staged_awaiting_consume and the pass never reaches the heal at all. Same technique as
+// TestDeliveryChecker_R39aGuard_NeverRestagesOverUnconsumed — the operator stages mid-tick, on the
+// stuck event's dispatch, which runs after the derive and before maybeHeal.
+func TestDeliveryChecker_R39aRefusal_LeavesARecord(t *testing.T) {
+	h := newDTHarness(t, true)
+	h.burnedFixture(t, 5)
+
+	base := h.checker.onEvent
+	h.checker.onEvent = func(cid, et, sev, msg, det, src string) {
+		if et == "offsite_delivery_stuck" {
+			if err := h.st.SaveOneTimeSecret("c1", "operator-fresh"); err != nil {
+				t.Errorf("mid-tick stage: %v", err)
+			}
+			if err := h.st.SetOneTimeSecretTimesForTest("c1", "2026-07-23 12:00:00", ""); err != nil {
+				t.Errorf("mid-tick stamp: %v", err)
+			}
+		}
+		base(cid, et, sev, msg, det, src)
+	}
+
+	h.checker.Check()
+
+	if h.reissuer.count() != 0 {
+		t.Fatal("R-39(a): the heal must never restage over an unconsumed secret")
+	}
+	rows := h.refusalRows(t)
+	if len(rows) != 1 {
+		t.Fatalf("the R-39(a) refusal left %d record(s), want 1", len(rows))
+	}
+	if !strings.Contains(rows[0].ErrorMessage, "R-39(a)") {
+		t.Errorf("the record must name the guard, got %q", rows[0].ErrorMessage)
+	}
+}
+
+// The refusal record rides the stuck event's 24h cadence: it explains the e-mail the operator just
+// received, and must not accumulate one row per monitor tick.
+func TestDeliveryChecker_HealRefusal_RidesTheStuckCadence(t *testing.T) {
+	h := newDTHarness(t, true)
+	h.regressedFixture(t, 6, 2)
+
+	h.checker.Check()
+	h.checker.Check() // the stuck event is swallowed by its cooldown; so is the refusal record
+	h.checker.Check()
+
+	if n := len(h.refusalRows(t)); n != 1 {
+		t.Fatalf("refusal records = %d after three passes, want 1 (one per stuck e-mail, not per tick)", n)
+	}
+}

hub/internal/monitor/offsite_neverran_test.go

@@ -102,7 +102,7 @@ func TestOffsiteStale_NeverRanEscrowAnchorWins(t *testing.T) {
 		t.Fatal(err)
 	}
 	// escrow blob stored now (SaveHostEscrow stamps updated_at with datetime('now'))
-	if _, err := st.SaveHostEscrow("h1", []byte("blob"), "fp", "zero_knowledge", time.Now().UTC().Format(time.RFC3339), "sha"); err != nil {
+	if _, _, err := st.SaveHostEscrow("h1", []byte("blob"), "fp", "zero_knowledge", time.Now().UTC().Format(time.RFC3339), "sha"); err != nil {
 		t.Fatal(err)
 	}
 	saveOffsiteReport(t, st, "c1", offsiteJSON(true, "escrowed", "", "", 0, 50))

hub/internal/monitor/restoretest.go

@@ -32,15 +32,104 @@ import (
 // is a NEW monitor written straight after the third, so it copies R-81's verdict structure rather
 // than re-deriving it. A tier never proven on a newborn box is UNKNOWN, never FAILED.
 
-// restoreProvenStaleAfter is how long a tier may go unproven before it is called stale.
+// ── HOW LONG MAY A TIER GO UNPROVEN? (R-86 Part 2) ───────────────────────────────────────────
 //
-// Derivation, not a guess: the restore-test cadence is 24h and rotation is oldest-first across two
-// tiers, so each tier is proven roughly every 2 days. 7 days therefore tolerates ~3 consecutive
-// missed opportunities before alarming — loud enough to matter, quiet enough not to fire on one
-// skipped cycle (a deferral behind a long backup is normal, not a fault). It is also comfortably
-// inside the 2-week offsite retention (operator ruling 2026-07-26), so a tier is never reported
-// stale against an archive that is about to be pruned anyway.
-const restoreProvenStaleAfter = 7 * 24 * time.Hour
+// This was one flat constant, 7 days, and its comment derived that number like this:
+//
+//	"the restore-test cadence is 24h and rotation is oldest-first across two tiers, so each tier is
+//	 proven roughly every 2 days. 7 days therefore tolerates ~3 consecutive missed opportunities."
+//
+// **That premise is exactly what R-86 removed.** The agent no longer tests on an interval at all: a
+// tier is tested once per ARCHIVE GENERATION — when it holds a settled archive that has not been
+// proven. A tier backed up weekly is therefore proved weekly, by design and in perfect health, and
+// against a flat 7-day window it would sit on the line and alarm every night about a system that is
+// working. Shipping the agent's half alone would have converted the improvement into a false alarm,
+// which is why the two ship together.
+//
+// The window is now derived from **the tier's own backup rhythm**, which the hub can observe from
+// the reports it already receives, and it keeps everything the constant had earned:
+//
+//   - absence is UNKNOWN until an anchored window has passed (R-81's structure, untouched);
+//   - the signal stays edge-triggered;
+//   - it never exceeds the offsite retention, so a tier is never called stale against an archive
+//     that is about to be pruned;
+//   - and it is never TIGHTER than the 7 days that were already tolerated.
+const (
+	// restoreProvenGenerations is how many archive generations may pass unproven before alarming.
+	// 4 = the settle lag's own generation plus ~3 missed opportunities — deliberately the same
+	// tolerance the flat constant expressed, so the change is to the RHYTHM, not to the patience.
+	restoreProvenGenerations = 4
+
+	// restoreProvenWindowFloor is the shortest window that may be applied to any tier. It is the
+	// old constant, kept as a FLOOR rather than deleted: a daily tier computes 4 days from its own
+	// rhythm, and tightening a live threshold is not what this task is for. A deferral behind a
+	// long backup is normal, not a fault.
+	restoreProvenWindowFloor = 7 * 24 * time.Hour
+
+	// restoreProvenWindowCap keeps the window strictly inside the 2-week offsite retention
+	// (operator ruling 2026-07-26) with two days to spare. Beyond it the hub would be judging a
+	// tier against an archive PBS has already pruned — an alarm nobody can act on, and the bound
+	// the old constant respected in its own way.
+	restoreProvenWindowCap = 12 * 24 * time.Hour
+
+	// restoreWindowRead is how far back the hub reads host-reports for this check: far enough to
+	// find proof anywhere inside the widest window, and to see at least two archive generations of
+	// a WEEKLY tier so its rhythm is observable at all.
+	restoreWindowRead = 2 * restoreProvenWindowFloor
+)
+
+// declaredArchiveInterval is the rhythm the hub ALREADY attributes to a tier — the same thresholds
+// the backup-freshness checker judges it against (deadline.go / deadline_tiers.go). It is the
+// fallback when a box's history is too short to observe a rhythm, and it is the right fallback
+// precisely because it is not a second opinion: if these two checkers disagreed about how often a
+// tier is expected to receive an archive, one of them would be alarming on the other's model.
+//
+// It is stated per RESTORE tier name ("local"/"pbs" — what the agent reports as source_tier), which
+// is the same split the backup tiers use under different names ("host"/"offsite").
+func declaredArchiveInterval(tier string) time.Duration {
+	if tier == "pbs" {
+		return offsiteBackupStaleAfter // 8 days: the weekly cadence plus a day of headroom
+	}
+	return backupStaleAfter // 26 hours: the daily cadence plus headroom
+}
+
+// restoreProvenWindow is how long THIS tier may go unproven, given its observed archive interval.
+//
+// observedOK=false means the box's retained history did not contain two archive generations for
+// this tier, so the declared rhythm is used. That fallback matters most for exactly the tier this
+// task is about: a fresh box with a weekly offsite tier has one snapshot and no observable
+// interval, and falling back to the FLOOR there would recreate the false alarm.
+//
+// OBSERVATION MAY ONLY WIDEN, NEVER TIGHTEN — and this is not caution, it is a live measurement.
+// On demo-felhom (2026-08-03) the offsite tier's two retained snapshots are `2026-07-27T19:55:41Z`
+// and `2026-07-28T04:49:43Z`: **8 h 54 m apart**, because one is a healing artefact and the other a
+// real weekly run. A mean-gap estimate therefore reads a WEEKLY tier as nine-hourly, ×4 gives 36 h,
+// the floor lifts it to 7 days — and a weekly tier proved weekly reaches ~8.25 days of proof age, so
+// the false alarm this whole task exists to prevent would have returned within a week, on the very
+// box it shipped to.
+//
+// The asymmetry is right on its own terms too. A gap SHORTER than the declared rhythm is routine and
+// means nothing — a retry, a manual run, a heal, a catch-up after an outage. A gap LONGER than the
+// declared rhythm is real information: this tier genuinely receives archives less often than the
+// model says, and its window must widen or it alarms. So observation refines the rhythm upward and
+// is ignored downward. The cost is stated plainly: a tier that truly runs FASTER than its declared
+// rhythm gets a wider window than it strictly needs, i.e. a slower stale signal. That is the right
+// direction for a signal whose message is "unverified" — "broken NOW" is `restore_test_failed`, and
+// that one is immediate and unaffected.
+func restoreProvenWindow(tier string, observed time.Duration, observedOK bool) time.Duration {
+	interval := declaredArchiveInterval(tier)
+	if observedOK && observed > interval {
+		interval = observed
+	}
+	w := time.Duration(restoreProvenGenerations) * interval
+	if w < restoreProvenWindowFloor {
+		w = restoreProvenWindowFloor
+	}
+	if w > restoreProvenWindowCap {
+		w = restoreProvenWindowCap
+	}
+	return w
+}
 
 // Event types. Operator-tier only — see the dispatcher note in RestoreTestChecker.
 const (
@@ -156,12 +245,13 @@ func (c *RestoreTestChecker) checkStaleness(customerID, latestJSON string, now t
 		return
 	}
 
-	rows, err := c.store.GetHostReportsSince(customerID, now.Add(-2*restoreProvenStaleAfter))
+	rows, err := c.store.GetHostReportsSince(customerID, now.Add(-restoreWindowRead))
 	if err != nil {
 		c.logger.Printf("[WARN] restore-test check: window read failed for %s: %v", customerID, err)
 		return
 	}
 	proven := lastProvenPerTier(rows)
+	intervals := observedArchiveIntervals(rows)
 
 	first, ferr := c.store.GetFirstHostReportAt(customerID)
 	if ferr != nil {
@@ -170,7 +260,9 @@ func (c *RestoreTestChecker) checkStaleness(customerID, latestJSON string, now t
 	}
 
 	for _, tier := range tiers {
-		v := assessRestoreProven(tier, proven[tier], first, now)
+		observed, observedOK := intervals[tier]
+		window := restoreProvenWindow(tier, observed, observedOK)
+		v := assessRestoreProven(tier, proven[tier], first, now, window)
 		key := customerID + "|" + tier
 		c.mu.Lock()
 		prev := c.staleStates[key]
@@ -192,37 +284,120 @@ func (c *RestoreTestChecker) checkStaleness(customerID, latestJSON string, now t
 	}
 }
 
-// assessRestoreProven is the per-tier verdict. PURE (now injected) so the policy is unit-tested —
-// the property that made R-81 provable, kept deliberately.
+// assessRestoreProven is the per-tier verdict. PURE (now and the window injected) so the policy is
+// unit-tested — the property that made R-81 provable, kept deliberately.
 //
 //	no proof, anchor NOT elapsed → UNKNOWN (newborn box; never an alarm)
 //	no proof, anchor elapsed     → MISSED
 //	proof older than the window  → MISSED
 //	otherwise                    → OK
-func assessRestoreProven(tier string, provenAt, firstReportAt, now time.Time) backupAssessment {
+//
+// `window` is now the TIER'S OWN (R-86 Part 2) rather than one constant for every tier, and every
+// reason string states the window it was judged against. That is R-100's corollary applied here:
+// when a verdict changes what it counts from, the alarm text has to change with it, or an operator
+// reads "limit 168h" under a tier that was actually judged at 288h and dismisses a true alarm.
+func assessRestoreProven(tier string, provenAt, firstReportAt, now time.Time, window time.Duration) backupAssessment {
 	if provenAt.IsZero() {
 		if firstReportAt.IsZero() {
 			return backupAssessment{verdict: verdictMissed,
 				reason: fmt.Sprintf("%s tier: never restore-proven, and no first-contact anchor to defer against", tier)}
 		}
 		watched := now.Sub(firstReportAt)
-		if watched <= restoreProvenStaleAfter {
+		if watched <= window {
 			return backupAssessment{verdict: verdictUnknown,
 				reason: fmt.Sprintf("%s tier: not restore-proven yet, but only watching for %s (grace %s since first contact %s) — newborn, not a fault",
-					tier, watched.Round(time.Hour), restoreProvenStaleAfter, firstReportAt.Format(time.RFC3339))}
+					tier, watched.Round(time.Hour), window, firstReportAt.Format(time.RFC3339))}
 		}
 		return backupAssessment{verdict: verdictMissed,
-			reason: fmt.Sprintf("%s tier: NEVER successfully restore-proven in %s of watching (limit %s) — the tier is unverified, not known-broken",
-				tier, watched.Round(time.Hour), restoreProvenStaleAfter)}
+			reason: fmt.Sprintf("%s tier: NEVER successfully restore-proven in %s of watching (limit %s, this tier's own backup rhythm) — the tier is unverified, not known-broken",
+				tier, watched.Round(time.Hour), window)}
 	}
-	if age := now.Sub(provenAt); age > restoreProvenStaleAfter {
+	if age := now.Sub(provenAt); age > window {
 		return backupAssessment{verdict: verdictMissed,
-			reason: fmt.Sprintf("%s tier: last successful restore-test was %s ago (limit %s) — the tier is unverified, not known-broken",
-				tier, age.Round(time.Hour), restoreProvenStaleAfter)}
+			reason: fmt.Sprintf("%s tier: last successful restore-test was %s ago (limit %s, this tier's own backup rhythm) — the tier is unverified, not known-broken",
+				tier, age.Round(time.Hour), window)}
 	}
 	return backupAssessment{verdict: verdictOK}
 }
 
+// observedArchiveIntervals estimates how often each RESTORE tier actually receives an archive, from
+// the host-reports the hub already holds. Keyed by restore-tier name ("local"/"pbs").
+//
+// Evidence is every distinct archive timestamp in the window: `pbs_snapshots[]` for the offsite
+// tier (PBS enumerates its whole retention in each report, so one report usually settles the
+// question) and successful `backups[]` records attributed by TARGET TYPE for both tiers — the
+// slice-A.4 rule, because a PBS-targeted vzdump appears in BOTH arrays and classifying by array
+// membership would attribute an offsite archive to the host tier.
+//
+// The estimate is the MEAN gap: (newest − oldest) / (generations − 1). It needs two generations;
+// with fewer, ok=false and the caller falls back to the declared rhythm. It is deliberately crude,
+// and can afford to be: restoreProvenWindow clamps the result between a 7-day floor and a 12-day
+// cap, so the only discrimination this has to get right is "roughly daily" versus "several days or
+// slower" — which is exactly the distinction that turns a healthy weekly tier into a false alarm.
+func observedArchiveIntervals(rows []store.HostReportRow) map[string]time.Duration {
+	seen := map[string]map[int64]struct{}{ // tier → set of archive unix times
+		"local": {},
+		"pbs":   {},
+	}
+	add := func(tier string, t time.Time) {
+		if t.IsZero() {
+			return
+		}
+		seen[tier][t.UTC().Unix()] = struct{}{}
+	}
+
+	for _, r := range rows {
+		var hr hostReportBackups
+		if json.Unmarshal([]byte(r.ReportJSON), &hr) != nil {
+			continue // one malformed retained report must not blind the scan
+		}
+		pbs := pbsTargetSet(hr)
+		for _, ps := range hr.PBSSnapshots {
+			if t, ok := parseBackupTime(ps.BackupTime); ok {
+				add("pbs", t)
+			}
+		}
+		for _, b := range hr.Backups {
+			if !b.Success {
+				continue
+			}
+			t, ok := parseBackupTime(b.StartedAt)
+			if !ok {
+				continue
+			}
+			if pbs[b.TargetID] {
+				add("pbs", t)
+			} else {
+				add("local", t)
+			}
+		}
+	}
+
+	out := map[string]time.Duration{}
+	for tier, set := range seen {
+		if len(set) < 2 {
+			continue // not observable — the caller uses the declared rhythm
+		}
+		var oldest, newest int64
+		first := true
+		for ts := range set {
+			if first || ts < oldest {
+				oldest = ts
+			}
+			if first || ts > newest {
+				newest = ts
+			}
+			first = false
+		}
+		span := time.Duration(newest-oldest) * time.Second
+		if span <= 0 {
+			continue
+		}
+		out[tier] = span / time.Duration(len(set)-1)
+	}
+	return out
+}
+
 // expectedRestoreTiers names the tiers this box actually HAS, so a box without an offsite tier is
 // never reported stale for one. Same gate as Slice C's `expected`, and for the same reason: without
 // it every box lacking a tier would alarm once the anchor elapsed — absence-is-not-failure,

hub/internal/monitor/restoretest_test.go

@@ -182,26 +182,33 @@ func TestRestoreTest_NewbornDoesNotAlarm(t *testing.T) {
 	}
 }
 
-// The boundary, pinned by name so a refactor has to delete an obviously-named contract.
+// The boundary, pinned by name so a refactor has to delete an obviously-named contract. R-86 made
+// the limit per-tier, so the anchor is now measured against THE TIER'S OWN window — here the local
+// tier's, which clamps to the 7-day floor and so keeps this contract numerically identical to the
+// one the flat constant expressed.
 func TestRestoreTest_Contract_UnprovenIsUnknownUntilTheAnchorElapses(t *testing.T) {
 	now := time.Date(2026, 7, 26, 12, 0, 0, 0, time.UTC)
+	window := restoreProvenWindow("local", 24*time.Hour, true)
+	if window != restoreProvenWindowFloor {
+		t.Fatalf("precondition: a daily local tier must clamp to the floor; got %s", window)
+	}
 	cases := []struct {
 		name       string
 		watched    time.Duration
 		wantMissed bool
 	}{
 		{"newborn, 1h", time.Hour, false},
-		{"just inside", restoreProvenStaleAfter - time.Minute, false},
-		{"exactly at the limit", restoreProvenStaleAfter, false},
-		{"just outside", restoreProvenStaleAfter + time.Minute, true},
+		{"just inside", window - time.Minute, false},
+		{"exactly at the limit", window, false},
+		{"just outside", window + time.Minute, true},
 		{"long past", 30 * 24 * time.Hour, true},
 	}
 	for _, c := range cases {
 		t.Run(c.name, func(t *testing.T) {
-			got := assessRestoreProven("pbs", time.Time{}, now.Add(-c.watched), now)
+			got := assessRestoreProven("local", time.Time{}, now.Add(-c.watched), now, window)
 			if got.missed() != c.wantMissed {
 				t.Fatalf("CONTRACT VIOLATED: unproven for %s (limit %s) → missed=%v, want %v (reason %q)",
-					c.watched, restoreProvenStaleAfter, got.missed(), c.wantMissed, got.reason)
+					c.watched, window, got.missed(), c.wantMissed, got.reason)
 			}
 			if !c.wantMissed && got.verdict != verdictUnknown {
 				t.Fatalf("a deferred tier must be UNKNOWN (visible), not OK; got verdict=%d", got.verdict)
@@ -215,15 +222,17 @@ func TestRestoreTest_Contract_UnprovenIsUnknownUntilTheAnchorElapses(t *testing.
 func TestRestoreTest_StaleIsSeparateFromFailure(t *testing.T) {
 	now := time.Date(2026, 7, 26, 12, 0, 0, 0, time.UTC)
 
-	stale := assessRestoreProven("pbs", now.Add(-9*24*time.Hour), now.Add(-60*24*time.Hour), now)
+	// A DAILY tier judged on its own rhythm: the window clamps to the 7-day floor.
+	daily := restoreProvenWindow("local", 24*time.Hour, true)
+	stale := assessRestoreProven("local", now.Add(-9*24*time.Hour), now.Add(-60*24*time.Hour), now, daily)
 	if !stale.missed() {
-		t.Fatalf("a tier last proven 9 days ago (limit %s) must be stale; got %q", restoreProvenStaleAfter, stale.reason)
+		t.Fatalf("a daily tier last proven 9 days ago (limit %s) must be stale; got %q", daily, stale.reason)
 	}
 	// The wording must not read as "broken" — that is the other signal.
 	if !strings.Contains(stale.reason, "unverified, not known-broken") {
 		t.Fatalf("staleness must say UNVERIFIED, not broken — merging the two is the thing this avoids; got %q", stale.reason)
 	}
-	fresh := assessRestoreProven("pbs", now.Add(-2*24*time.Hour), now.Add(-60*24*time.Hour), now)
+	fresh := assessRestoreProven("local", now.Add(-2*24*time.Hour), now.Add(-60*24*time.Hour), now, daily)
 	if fresh.verdict != verdictOK {
 		t.Fatalf("a tier proven 2 days ago is fine; got verdict=%d reason=%q", fresh.verdict, fresh.reason)
 	}
@@ -283,3 +292,208 @@ func boolStr(b bool) string {
 	}
 	return "false"
 }
+
+// ── SCENARIO G — a healthy WEEKLY tier is never reported stale (R-86 Part 2) ─────────────────
+//
+// This is the test that pins the false alarm this change would otherwise have CREATED. The agent
+// now proves a tier once per archive generation, so a weekly offsite tier is proved weekly — in
+// perfect health. Against the old flat 7-day window it would sit on the line and alarm every night.
+//
+// COMPANION RED-PROOF (observed 2026-08-03): pin the window flat, as it was —
+//
+//   - window := restoreProvenWindow(tier, observed, observedOK)
+//   - window := restoreProvenWindowFloor   // the pre-R-86 flat 7 days
+//
+// →	--- FAIL: TestRestoreTest_HealthyWeeklyTierIsNeverStale
+//
+//	week 0: a weekly tier proved on its own archive must never be stale (proof age 172h0m0s,
+//	window 168h0m0s); verdict=2 reason="pbs tier: last successful restore-test was 172h0m0s ago
+//	(limit 168h0m0s, this tier's own backup rhythm) — the tier is unverified, not known-broken"
+//
+// Restored. The mutation is one line because the whole of Part 2 is one decision: whose rhythm.
+//
+// NOTE, because it is the finding this test nearly hid: the FIRST version of this fixture had NO
+// jitter, and it PASSED under the mutation. A perfectly regular weekly tier's proof age reaches
+// EXACTLY 168h just before the next proof, and `age > window` is false by a hair — a hollow test
+// that would have shipped Part 1 and its false alarm together. The jitter below is what makes this
+// a test, and it is also the truth about the old constant: a healthy weekly tier did not merely sit
+// near the line, it sat ON it, so any ordinary delay tipped it over.
+func TestRestoreTest_HealthyWeeklyTierIsNeverStale(t *testing.T) {
+	start := time.Date(2026, 6, 1, 3, 0, 0, 0, time.UTC)
+	firstContact := start.Add(-24 * time.Hour)
+
+	// The observable rhythm of a weekly tier, as the hub would compute it from the reports. No
+	// assertion about the window ITSELF here on purpose: that is the mechanism, and it is pinned in
+	// TestRestoreProvenWindow_Contract. What this test asserts is the CONSEQUENCE — does the alarm
+	// fire? — because R-97b proved a mechanism and shipped a broken consequence anyway.
+	weekly := restoreProvenWindow("pbs", 7*24*time.Hour, true)
+
+	// Walk several weeks of a HEALTHY tier, with the jitter a real one has: the backup does not land
+	// to the second, and a restore-test can be deferred one evaluation behind a running backup.
+	//
+	// The jitter is the point. A perfectly regular weekly tier's proof reaches an age of EXACTLY one
+	// interval (168h) just before the next proof, and against a flat 168h window `age > window` is
+	// false by a hair — so a regular fixture would pass against the very constant this task must
+	// change, and prove nothing. That is the brief's "sits exactly on that line": every real-world
+	// delay pushes it over, and the alarm is about a system that is working.
+	settle, evalLatency := 24*time.Hour, 6*time.Hour
+	archiveLate := []time.Duration{0, 4 * time.Hour, 2 * time.Hour, 6 * time.Hour, 0, 3 * time.Hour}
+	deferred := []time.Duration{0, 0, 6 * time.Hour, 0, 0, 6 * time.Hour} // one evaluation behind a backup
+
+	archiveAt := func(week int) time.Time {
+		return start.AddDate(0, 0, 7*week).Add(archiveLate[week])
+	}
+	provenAt := func(week int) time.Time {
+		return archiveAt(week).Add(settle + evalLatency + deferred[week])
+	}
+
+	var worst time.Duration
+	for week := 0; week+1 < len(archiveLate); week++ {
+		// The widest the proof's age ever gets: the instant before the NEXT week's proof lands.
+		now := provenAt(week + 1).Add(-time.Second)
+		age := now.Sub(provenAt(week))
+		if age > worst {
+			worst = age
+		}
+		v := assessRestoreProven("pbs", provenAt(week), firstContact, now, weekly)
+		if v.verdict != verdictOK {
+			t.Fatalf("week %d: a weekly tier proved on its own archive must never be stale (proof age %s, window %s); verdict=%d reason=%q",
+				week, age.Round(time.Hour), weekly, v.verdict, v.reason)
+		}
+	}
+	// The fixture must actually EXERCISE the boundary — a jitter-free walk would sit at exactly one
+	// interval and pass against a flat 7-day window, which is the hollow version of this test.
+	if worst <= restoreProvenWindowFloor {
+		t.Fatalf("this fixture never exceeds the old flat window (worst proof age %s) — it cannot detect the defect it exists for", worst)
+	}
+}
+
+// ...and a weekly tier that genuinely STOPS being proved must still alarm. A window that never
+// fires is not a fix, it is a deletion.
+func TestRestoreTest_WeeklyTierThatStopsBeingProvedStillAlarms(t *testing.T) {
+	now := time.Date(2026, 7, 26, 12, 0, 0, 0, time.UTC)
+	weekly := restoreProvenWindow("pbs", 7*24*time.Hour, true)
+
+	v := assessRestoreProven("pbs", now.Add(-weekly-time.Hour), now.Add(-90*24*time.Hour), now, weekly)
+	if !v.missed() {
+		t.Fatalf("a weekly tier unproven for longer than its own window MUST alarm; got verdict=%d reason=%q", v.verdict, v.reason)
+	}
+	if !strings.Contains(v.reason, weekly.String()) {
+		t.Fatalf("the alarm must state the window it was judged against (R-100's corollary); got %q", v.reason)
+	}
+}
+
+// The window's own contract: derived from the tier's rhythm, floored, capped, and never dependent
+// on an unobservable history for the tier that would suffer most from a wrong answer.
+func TestRestoreProvenWindow_Contract(t *testing.T) {
+	cases := []struct {
+		name       string
+		tier       string
+		observed   time.Duration
+		observedOK bool
+		want       time.Duration
+	}{
+		{"daily local clamps to the floor", "local", 24 * time.Hour, true, restoreProvenWindowFloor},
+		{"weekly pbs widens", "pbs", 7 * 24 * time.Hour, true, restoreProvenWindowCap},
+		{"3-day tier sits between", "pbs", 72 * time.Hour, true, 12 * 24 * time.Hour},
+		{"unobservable local falls back to its declared rhythm", "local", 0, false, restoreProvenWindowFloor},
+		{"unobservable pbs falls back WIDE, not to the floor", "pbs", 0, false, restoreProvenWindowCap},
+		{"a nonsense zero interval is ignored", "pbs", 0, true, restoreProvenWindowCap},
+		// MEASURED ON THE LIVE BOX, and the reason observation may only WIDEN. demo-felhom's two
+		// retained PBS snapshots sit 8h54m apart (one is a healing artefact), so a mean-gap estimate
+		// reads a WEEKLY tier as nine-hourly. Taking that at face value gives 4x9h = 36h → the 7-day
+		// floor → and a weekly tier proved weekly (~8.25d of proof age) alarms within a week of this
+		// shipping, on the box it shipped to.
+		{"a short observed gap must NOT tighten a weekly tier", "pbs", 8*time.Hour + 54*time.Minute, true, restoreProvenWindowCap},
+		{"a short observed gap must not tighten the host tier either", "local", 30 * time.Minute, true, restoreProvenWindowFloor},
+		// ...but a tier that genuinely runs SLOWER than its declared rhythm still widens.
+		{"a genuinely slower tier widens", "local", 4 * 24 * time.Hour, true, restoreProvenWindowCap},
+	}
+	// The relationship Part 1 depends on: a weekly tier's window must be WIDER than a daily tier's,
+	// or proving weekly (which is now correct behaviour) alarms on itself.
+	if restoreProvenWindow("pbs", 7*24*time.Hour, true) <= restoreProvenWindow("local", 24*time.Hour, true) {
+		t.Fatal("a weekly tier must earn a wider window than a daily one — otherwise R-86's agent half alarms about itself")
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			got := restoreProvenWindow(c.tier, c.observed, c.observedOK)
+			if got != c.want {
+				t.Fatalf("window(%s, observed=%s ok=%v) = %s, want %s", c.tier, c.observed, c.observedOK, got, c.want)
+			}
+			if got < restoreProvenWindowFloor || got > restoreProvenWindowCap {
+				t.Fatalf("every window must stay inside [%s, %s]; got %s", restoreProvenWindowFloor, restoreProvenWindowCap, got)
+			}
+		})
+	}
+}
+
+// The rhythm must be OBSERVED from the reports, not assumed — including the slice-A.4 rule that a
+// PBS-targeted vzdump appears in both arrays and must be attributed by TARGET TYPE.
+func TestObservedArchiveIntervals_FromReports(t *testing.T) {
+	base := time.Date(2026, 7, 1, 2, 0, 0, 0, time.UTC)
+	mk := func(localAt []time.Time, pbsAt []time.Time) string {
+		type stg struct{ Name, Type, Content string }
+		type bk struct {
+			TargetID  string `json:"target_id"`
+			Success   bool   `json:"success"`
+			StartedAt string `json:"started_at"`
+		}
+		type snap struct {
+			BackupTime string `json:"backup_time"`
+		}
+		payload := struct {
+			StorageTargets []struct {
+				Name    string `json:"name"`
+				Type    string `json:"type"`
+				Content string `json:"content"`
+			} `json:"storage_targets"`
+			Backups      []bk   `json:"backups"`
+			PBSSnapshots []snap `json:"pbs_snapshots"`
+		}{}
+		payload.StorageTargets = append(payload.StorageTargets, struct {
+			Name    string `json:"name"`
+			Type    string `json:"type"`
+			Content string `json:"content"`
+		}{"felhom-backup", "dir", "backup"}, struct {
+			Name    string `json:"name"`
+			Type    string `json:"type"`
+			Content string `json:"content"`
+		}{"felhom-pbs", "pbs", "backup"})
+		for _, at := range localAt {
+			payload.Backups = append(payload.Backups, bk{"felhom-backup", true, at.Format(time.RFC3339)})
+		}
+		for _, at := range pbsAt {
+			// The SAME archive appears as a vzdump record AND as a snapshot — slice A.4.
+			payload.Backups = append(payload.Backups, bk{"felhom-pbs", true, at.Format(time.RFC3339)})
+			payload.PBSSnapshots = append(payload.PBSSnapshots, snap{at.Format(time.RFC3339)})
+		}
+		b, err := json.Marshal(payload)
+		if err != nil {
+			t.Fatal(err)
+		}
+		return string(b)
+	}
+
+	rows := []store.HostReportRow{
+		{ReportJSON: mk(
+			[]time.Time{base, base.AddDate(0, 0, 1), base.AddDate(0, 0, 2)},
+			[]time.Time{base, base.AddDate(0, 0, 7)},
+		)},
+		{ReportJSON: `{{{malformed`}, // must not blind the scan
+	}
+
+	got := observedArchiveIntervals(rows)
+	if d, ok := got["local"]; !ok || d != 24*time.Hour {
+		t.Fatalf("a daily host tier must be observed as ~24h; got %s ok=%v", d, ok)
+	}
+	if d, ok := got["pbs"]; !ok || d != 7*24*time.Hour {
+		t.Fatalf("a weekly offsite tier must be observed as ~7d — and its vzdump record must NOT be "+
+			"counted into the host tier (slice A.4); got %s ok=%v", d, ok)
+	}
+
+	// One generation is not a rhythm: unobservable, so the caller falls back to the declared one.
+	single := []store.HostReportRow{{ReportJSON: mk(nil, []time.Time{base})}}
+	if d, ok := observedArchiveIntervals(single)["pbs"]; ok {
+		t.Fatalf("one archive cannot yield an interval; got %s", d)
+	}
+}

hub/internal/notify/backup_run_digest_test.go

@@ -0,0 +1,301 @@
+package notify
+
+import (
+	"io"
+	"log"
+	"strings"
+	"testing"
+)
+
+// R-182 — one e-mail per backup run, and nothing dropped without a trace.
+//
+// MEASURED, NOT SUPPOSED. On 2026-08-03 nine `recovery_unit_capture_failed` events reached the hub
+// and TWO operator e-mails went out. The operator cooldown key is
+// `customerID + ":" + eventType + cooldownTierSuffix(details)`, that event carries `app` but no
+// `tier`, so the key held no app identifier: the first refused app took the hour's slot and every
+// other app's failure was discarded — **before `LogNotification`**, so it left no row on any channel
+// and could not be found afterwards.
+//
+// The operator ruled against the obvious fix (putting `app` in the key), because on a full disk that
+// is one e-mail per app. These pin the shape that replaced it.
+
+// ── Scenario D — a suppressed operator event leaves a trace ───────────────────────────────────────
+
+// The bare `return` at the cooldown is the whole reason this defect took a day to get the right way
+// round: there was nothing to read. A drop must be as visible in the record as a send.
+//
+// DELIBERATELY EXERCISED ON A DIFFERENT EVENT TYPE than the one that exposed the defect.
+// `recovery_unit_capture_failed` is now record-only and never reaches the cooldown at all, so using
+// it here would prove nothing. `whole_guest_backup_failed` is an ordinary operator event, and using
+// it pins §2.1's actual claim: the suppression row applies to EVERY operator event, not only the one
+// that happened to be measured.
+func TestSuppressedOperatorEvent_LeavesARow(t *testing.T) {
+	st := newDispStore(t)
+	d := NewDispatcher(st, "test-key", "from@felhom.eu", "op@felhom.eu", true, log.New(io.Discard, "", 0))
+	sent := captureSeam(d)
+
+	// Two events of the SAME type with no discriminator — the second must be suppressed.
+	d.ProcessEvent("c1", "whole_guest_backup_failed", "error", "opengist failed", `{"app":"opengist"}`, "controller")
+	d.ProcessEvent("c1", "whole_guest_backup_failed", "error", "privatebin failed", `{"app":"privatebin"}`, "controller")
+
+	if got := len(mailsFor(*sent, "op@felhom.eu")); got != 1 {
+		t.Fatalf("operator mails = %d, want 1 — the premise of this test is that the second IS suppressed", got)
+	}
+
+	rows, err := st.GetRecentNotifications("c1", 50)
+	if err != nil {
+		t.Fatal(err)
+	}
+	var sentRows, suppressed []store2Row
+	for _, r := range rows {
+		if r.Channel != "operator" || r.EventType != "whole_guest_backup_failed" {
+			continue
+		}
+		switch r.Status {
+		case "sent":
+			sentRows = append(sentRows, store2Row{r.Status, r.Message, r.ErrorMessage})
+		case "suppressed":
+			suppressed = append(suppressed, store2Row{r.Status, r.Message, r.ErrorMessage})
+		}
+	}
+	if len(sentRows) != 1 {
+		t.Fatalf("want 1 'sent' operator row, got %d", len(sentRows))
+	}
+	if len(suppressed) != 1 {
+		t.Fatalf("want 1 'suppressed' operator row, got %d — a cooldown drop that writes NOTHING is "+
+			"indistinguishable from an event that never happened, which is exactly how seven "+
+			"failures went missing on 2026-08-03", len(suppressed))
+	}
+	// The row must name the app that was dropped, or it records that something was suppressed
+	// without recording WHAT — half a fix.
+	if !strings.Contains(suppressed[0].message, "privatebin") {
+		t.Fatalf("the suppressed row does not name the dropped event: %q", suppressed[0].message)
+	}
+	// And it must carry the key, so the reason it collided is readable without reading code.
+	if !strings.Contains(suppressed[0].errMsg, "key=") {
+		t.Fatalf("the suppressed row does not carry the cooldown key: %q", suppressed[0].errMsg)
+	}
+}
+
+type store2Row struct{ status, message, errMsg string }
+
+// ── Scenario E — two runs in a day each report ───────────────────────────────────────────────────
+
+// The operator ruled explicitly on this: someone pressing the backup button is actively trying to
+// get a backup, and finding out tomorrow would be worse than an extra mail in a rare case.
+func TestTwoRunsInAnHour_BothReport(t *testing.T) {
+	st := newDispStore(t)
+	d := NewDispatcher(st, "test-key", "from@felhom.eu", "op@felhom.eu", true, log.New(io.Discard, "", 0))
+	sent := captureSeam(d)
+
+	nightly := `{"run_id":"run-a","run_kind":"nightly","failed":2,"attempted":5,"apps":[{"app":"opengist","leg":"volume dump","reason":"reserve"},{"app":"privatebin","leg":"volume dump","reason":"reserve"}]}`
+	manual := `{"run_id":"run-b","run_kind":"manual","failed":2,"attempted":5,"apps":[{"app":"opengist","leg":"volume dump","reason":"reserve"},{"app":"privatebin","leg":"volume dump","reason":"reserve"}]}`
+
+	d.ProcessEvent("c1", "backup_run_failures", "error", "2 of 5 apps failed", nightly, "controller")
+	d.ProcessEvent("c1", "backup_run_failures", "error", "2 of 5 apps failed", manual, "controller")
+
+	if got := len(mailsFor(*sent, "op@felhom.eu")); got != 2 {
+		t.Fatalf("operator mails = %d, want 2 — the 1-hour cooldown swallowed the manual run's "+
+			"digest, which is the fix reappearing one level up: press the button, the run fails, "+
+			"and you are told nothing because the machine already wrote this hour", got)
+	}
+}
+
+// The run discriminator must be NARROW, exactly like its `tier` sibling — empty unless the producer
+// opts in — or every existing event type's cooldown silently stops collapsing anything.
+func TestCooldownRunSuffix_EmptyForEverythingElse(t *testing.T) {
+	cases := []struct{ name, details string }{
+		{"no details", ""},
+		{"details without run_id", `{"app":"opengist","error":"boom"}`},
+		{"empty run_id value", `{"run_id":""}`},
+		{"malformed json", `{{{nope`},
+		{"run_id mentioned in a STRING, not as a key", `{"error":"the run_id: abc failed"}`},
+		{"null details", `null`},
+	}
+	for _, c := range cases {
+		if got := cooldownRunSuffix(c.details); got != "" {
+			t.Errorf("%s: suffix must be EMPTY so every other type's cooldown is unchanged, got %q", c.name, got)
+		}
+	}
+	if got := cooldownRunSuffix(`{"run_id":"run-a"}`); got != ":run-a" {
+		t.Fatalf("suffix should be the run id, got %q", got)
+	}
+	// The two suffixes must not interfere: a tier event still keys on its tier and nothing else.
+	if got := cooldownTierSuffix(`{"tier":"felhom-pbs"}`) + cooldownRunSuffix(`{"tier":"felhom-pbs"}`); got != ":felhom-pbs" {
+		t.Fatalf("a tier-only event's key changed to %q — R-97a's behaviour must be byte-identical", got)
+	}
+}
+
+// ── Scenario G — the customer never receives the digest ──────────────────────────────────────────
+
+// v0.78.0 asserted in a COMMENT that a type with no `customerMessages` entry structurally cannot
+// reach a customer. It can: FormatCustomerEmail falls back to the raw English message and the only
+// customer gate is configuration. So this is demonstrated, not argued.
+func TestDigest_IsOperatorOnly_EvenWithAWideEnabledList(t *testing.T) {
+	st := newDispStore(t)
+	// A customer who has enabled EVERYTHING, including this type by name.
+	if err := st.SaveNotificationPrefs("c1", "cust@example.com",
+		[]string{"backup_run_failures", "node_down", "disk_warning"}, 6); err != nil {
+		t.Fatal(err)
+	}
+	d := NewDispatcher(st, "test-key", "from@felhom.eu", "op@felhom.eu", true, log.New(io.Discard, "", 0))
+	sent := captureSeam(d)
+
+	details := `{"run_id":"run-a","run_kind":"nightly","failed":1,"attempted":3,"apps":[{"app":"opengist","leg":"volume dump","reason":"reserve"}]}`
+	d.ProcessEvent("c1", "backup_run_failures", "error", "1 of 3 apps failed", details, "controller")
+
+	if got := mailsFor(*sent, "cust@example.com"); len(got) != 0 {
+		t.Fatalf("the CUSTOMER received an operator digest (%d mails) — a list of which apps' "+
+			"backups failed is not something they can act on, and the raw body is English", len(got))
+	}
+	if got := len(mailsFor(*sent, "op@felhom.eu")); got != 1 {
+		t.Fatalf("operator mails = %d, want 1", got)
+	}
+	if !operatorOnlyEvents["backup_run_failures"] {
+		t.Fatal("backup_run_failures is not in operatorOnlyEvents — allowlisting alone does NOT " +
+			"keep it from a customer; that assumption shipped once and was wrong (v0.78.0)")
+	}
+}
+
+// ── Part 3 — the e-mail a person actually reads ──────────────────────────────────────────────────
+
+func TestDigestEmail_ListsAppsLegsAndReasons(t *testing.T) {
+	details := `{"run_id":"run-a","run_kind":"nightly","failed":3,"attempted":40,` +
+		`"target_path":"/mnt/sys_drive","used_gb":64.3,"avail_gb":0.9,"total_gb":68.7,` +
+		`"used_percent":94,"space_known":true,"apps":[` +
+		`{"app":"opengist","leg":"volume dump","reason":"refused: below the reserve (headroom)"},` +
+		`{"app":"privatebin","leg":"volume dump","reason":"refused: below the reserve (headroom)"},` +
+		`{"app":"immich","leg":"database dump","reason":"pg_dump: connection refused"}]}`
+
+	subject, body := FormatOperatorEmail("demo-hp", "backup_run_failures", "error",
+		"3 of 40 apps failed to back up", details)
+
+	// The subject must carry the counts: the operator's first decision is made from it alone.
+	for _, want := range []string{"demo-hp", "3 of 40", "nightly"} {
+		if !strings.Contains(subject, want) {
+			t.Errorf("subject %q missing %q", subject, want)
+		}
+	}
+	// Every app, its leg and its reason.
+	for _, want := range []string{
+		"opengist", "privatebin", "immich",
+		"volume dump", "database dump",
+		"below the reserve", "pg_dump: connection refused",
+	} {
+		if !strings.Contains(body, want) {
+			t.Errorf("body missing %q:\n%s", want, body)
+		}
+	}
+	// The counts and the free space, so "one broken app" and "a full disk" read differently.
+	if !strings.Contains(body, "3 of 40") {
+		t.Errorf("body does not carry the failed-of-attempted count:\n%s", body)
+	}
+	if !strings.Contains(body, "0.9 GB free") {
+		t.Errorf("body does not carry the free space:\n%s", body)
+	}
+	// It must NOT be a JSON blob.
+	if strings.Contains(body, `"apps":[`) {
+		t.Errorf("the digest rendered as raw JSON — unreadable on a phone at 07:00:\n%s", body)
+	}
+}
+
+// An absent space reading must render as unavailable, never as zeros: "0 GB free" and "we could not
+// look" are opposite diagnoses, and the operator acts differently on each.
+func TestDigestEmail_UnknownSpaceIsNotZero(t *testing.T) {
+	details := `{"run_id":"r","run_kind":"nightly","failed":1,"attempted":2,"target_path":"/mnt/x",` +
+		`"space_known":false,"apps":[{"app":"a","leg":"capture","reason":"boom"}]}`
+	_, body := FormatOperatorEmail("demo-hp", "backup_run_failures", "error", "1 of 2 failed", details)
+	if strings.Contains(body, "0.0 GB free") {
+		t.Fatalf("an unreadable filesystem rendered as zeros:\n%s", body)
+	}
+	if !strings.Contains(body, "unavailable") {
+		t.Fatalf("an unreadable filesystem must say so:\n%s", body)
+	}
+}
+
+// A payload that cannot be parsed must still produce a mail — degraded, never swallowed.
+func TestDigestEmail_UnparseableDetailsStillMails(t *testing.T) {
+	_, body := FormatOperatorEmail("demo-hp", "backup_run_failures", "error", "something failed", `{{{`)
+	if body == "" || !strings.Contains(body, "something failed") {
+		t.Fatalf("an unparseable digest lost the mail entirely:\n%s", body)
+	}
+}
+
+// ── Scenario C — every failure is RECORDED, e-mailed or not ──────────────────────────────────────
+
+// The per-app event is the record; the digest is the notification. The record must not inherit the
+// notification's conditions — no cooldown, no preferences, no dependence on a mail going out.
+func TestPerAppFailure_IsRecordedButNotMailed(t *testing.T) {
+	st := newDispStore(t)
+	d := NewDispatcher(st, "test-key", "from@felhom.eu", "op@felhom.eu", true, log.New(io.Discard, "", 0))
+	sent := captureSeam(d)
+
+	apps := []string{"opengist", "privatebin", "immich", "homebox", "nextcloud"}
+	for _, a := range apps {
+		d.ProcessEvent("c1", "recovery_unit_capture_failed", "error",
+			"Recovery unit capture FAILED for \""+a+"\"", `{"app":"`+a+`"}`, "controller")
+	}
+
+	// NOT mailed — the digest is the notification.
+	if got := len(*sent); got != 0 {
+		t.Fatalf("%d mail(s) sent for per-app failures — they are the RECORD; one mail per app on a "+
+			"full disk is the volume problem wearing the correctness problem's clothes, which is "+
+			"exactly what the operator ruled against", got)
+	}
+
+	// But ALL FIVE recorded — this is the assertion yesterday's defect would have failed: nine
+	// arrived, two were mailed, seven left no row anywhere.
+	rows, err := st.GetRecentNotifications("c1", 50)
+	if err != nil {
+		t.Fatal(err)
+	}
+	seen := map[string]bool{}
+	for _, r := range rows {
+		if r.EventType == "recovery_unit_capture_failed" && r.Status == "recorded" {
+			for _, a := range apps {
+				if strings.Contains(r.Message, a) {
+					seen[a] = true
+				}
+			}
+		}
+	}
+	if len(seen) != len(apps) {
+		t.Fatalf("only %d of %d per-app failures were recorded (%v) — a failure that produced no row "+
+			"anywhere is the measured defect of 2026-08-03", len(seen), len(apps), seen)
+	}
+}
+
+// The per-app reason must not repeat the filesystem figures the digest already prints once. Reviewed
+// as copy against the first real digest, not designed in the abstract.
+func TestDigestEmail_ReasonDoesNotRepeatTheUsageLine(t *testing.T) {
+	reason := "refused: below the reserve (reserve: 97% used or 1.0 GiB free) — /mnt/sys_drive: 65.0/68.7 GB used (95%), 0.2 GB free"
+	details := `{"run_id":"r","run_kind":"nightly","failed":1,"attempted":2,"target_path":"/mnt/sys_drive",` +
+		`"used_gb":65,"avail_gb":0.2,"total_gb":68.7,"used_percent":95,"space_known":true,` +
+		`"apps":[{"app":"opengist","leg":"whole app","reason":"` + reason + `"}]}`
+	_, body := FormatOperatorEmail("demo-hp", "backup_run_failures", "error", "1 of 2 failed", details)
+
+	// The figures appear ONCE, on the Filesystem line — not again on every app row.
+	if strings.Count(body, "65.0/68.7 GB used") != 1 {
+		t.Fatalf("the usage clause appears %d times; it must appear once, on its own line — repeated "+
+			"down a list of a dozen apps it pushes the part that DIFFERS off a phone screen:\n%s",
+			strings.Count(body, "65.0/68.7 GB used"), body)
+	}
+	// But the reason itself survives — trimming must not eat the diagnosis.
+	if !strings.Contains(body, "below the reserve") {
+		t.Fatalf("the reason was trimmed away entirely:\n%s", body)
+	}
+}
+
+// A reason naming a DIFFERENT path, or none, must be left completely alone.
+func TestTrimRepeatedUsage_LeavesUnrelatedReasonsAlone(t *testing.T) {
+	for _, c := range []struct{ reason, target string }{
+		{"pg_dump: connection refused", "/mnt/sys_drive"},
+		{"tar failed — /mnt/other: 1/2 GB used (50%), 1 GB free", "/mnt/sys_drive"},
+		{"boom", ""},
+		{"", "/mnt/sys_drive"},
+	} {
+		if got := trimRepeatedUsage(c.reason, c.target); got != c.reason {
+			t.Errorf("reason %q (target %q) was altered to %q", c.reason, c.target, got)
+		}
+	}
+}

hub/internal/notify/dispatcher.go

@@ -116,6 +116,18 @@ func (d *Dispatcher) ProcessEvent(customerID, eventType, severity, message, deta
 		return
 	}
 
+	// R-182: record-only types are written down and never mailed. Placed BEFORE the severity gate
+	// so the row is written whatever the severity — the record must not inherit the notification's
+	// conditions, which is the coupling this whole finding is about.
+	if recordOnlyEvents[eventType] {
+		if err := d.store.LogNotification(customerID, eventType, severity, message, "recorded",
+			"record-only: the per-run digest (backup_run_failures) carries the notification", "operator"); err != nil {
+			d.logger.Printf("[WARN] Failed to record %s for %s: %v", eventType, customerID, err)
+		}
+		d.logger.Printf("[INFO] Recorded (not mailed) %s for %s — the run digest is the notification", eventType, customerID)
+		return
+	}
+
 	// warning / error / critical trigger notifications. "info" is an intentional non-notify (status/
 	// recovery events). Anything else is UNRECOGNIZED — log it (don't silently drop), so a bad severity
 	// surfaces instead of vanishing (the felhom-pve-class lesson: a critical event must never be lost).
@@ -260,15 +272,67 @@ func cooldownTierSuffix(detailsJSON string) string {
 	return ":" + d.Tier
 }
 
+// cooldownRunSuffix returns ":"+run_id when the event's details carry a non-empty `run_id`, else "".
+//
+// R-182. `cooldownTierSuffix`'s sibling, and deliberately a SEPARATE function rather than an extra
+// branch inside it: `tier` keeps byte-identical semantics for every type that uses it, so R-97a's
+// behaviour and its tests are untouched by this.
+//
+// WHY A BACKUP RUN NEEDS ONE. The run digest describes ONE RUN, and a box can have two in a day —
+// the nightly one and a manual one the operator triggered *because* something looked wrong. With no
+// run-scoped discriminator the 1-hour cooldown would swallow the second, which is the failure this
+// row exists to fix, reappearing one level up: the operator presses the button, the run fails, and
+// they are told nothing because the machine already wrote that hour.
+//
+// IT MAKES THE COOLDOWN EFFECTIVELY INERT FOR THIS TYPE, AND THAT IS THE INTENT, NOT AN OVERSIGHT.
+// A digest is already rate-limited by construction — one per run, emitted only when something
+// failed — so there is nothing for a timer to collapse. The cooldown protects against a repeating
+// identical alert; a digest cannot repeat, because each run is a different run.
+//
+// NARROW, LIKE ITS SIBLING: empty unless the producer opts in by sending a `run_id`, so no existing
+// event type's cooldown behaviour changes.
+func cooldownRunSuffix(detailsJSON string) string {
+	if detailsJSON == "" || !strings.Contains(detailsJSON, "\"run_id\"") {
+		return ""
+	}
+	var d struct {
+		RunID string `json:"run_id"`
+	}
+	if err := json.Unmarshal([]byte(detailsJSON), &d); err != nil || d.RunID == "" {
+		return ""
+	}
+	return ":" + d.RunID
+}
+
 func (d *Dispatcher) processOperator(customerID, eventType, severity, message, detailsJSON, source string) {
 	if !d.operatorOn || d.operatorEmail == "" {
 		return
 	}
 
-	cooldownKey := customerID + ":" + eventType + cooldownTierSuffix(detailsJSON)
+	cooldownKey := customerID + ":" + eventType + cooldownTierSuffix(detailsJSON) + cooldownRunSuffix(detailsJSON)
 	d.mu.Lock()
 	if last, ok := d.opCooldowns[cooldownKey]; ok && time.Since(last) < 1*time.Hour {
 		d.mu.Unlock()
+		// R-182: RECORD THE SUPPRESSION. This used to be a bare `return` — the event was dropped
+		// before any LogNotification, so a cooldown drop and an event that never happened were
+		// indistinguishable from the operator's side AND from the hub's own records.
+		//
+		// Measured 2026-08-03: nine `recovery_unit_capture_failed` events arrived, two emails were
+		// sent, and the other seven left NO ROW ON ANY CHANNEL. The defect that hid was serious —
+		// the cooldown key carries no app identifier, so the first refused app took the slot and
+		// every other app's failure that hour was discarded — but the reason it took a day to find
+		// the right way round is this line: there was nothing to read.
+		//
+		// "We chose not to e-mail you" and "nothing happened" must never look identical. This
+		// applies to EVERY operator event, not only the one that exposed it. It makes the drop
+		// visible; it deliberately does NOT change the cooldown's duration or semantics.
+		if err := d.store.LogNotification(customerID, eventType, severity, message,
+			"suppressed", "operator cooldown 1h, key="+cooldownKey, "operator"); err != nil {
+			d.logger.Printf("[WARN] Failed to record suppressed operator notification for %s/%s: %v",
+				customerID, eventType, err)
+		}
+		d.logger.Printf("[INFO] Operator email suppressed for %s/%s — cooldown (key=%s)",
+			customerID, eventType, cooldownKey)
 		return
 	}
 	d.opCooldowns[cooldownKey] = time.Now()
@@ -285,6 +349,33 @@ func (d *Dispatcher) processOperator(customerID, eventType, severity, message, d
 	d.store.LogNotification(customerID, eventType, severity, message, "sent", "", "operator")
 }
 
+// recordOnlyEvents are STORED and RECORDED but never e-mailed, on either channel.
+//
+// R-182. The distinction this register exists to make is the whole of that finding: **the record and
+// the notification are different things.** A per-app backup failure must always be written down —
+// every time, unconditionally, regardless of cooldowns, preferences or whether any mail went out —
+// and it must NOT compete for an e-mail slot, because the per-run digest
+// (`backup_run_failures`) is what a person is meant to read.
+//
+// Before this, `recovery_unit_capture_failed` was both at once, and it did neither well: on
+// 2026-08-03 nine of them arrived, two were e-mailed, and the other seven were dropped by the
+// 1-hour cooldown BEFORE anything was written down. So the operator was told about one app, the
+// other apps' failures were discarded, and nothing anywhere recorded that a choice had been made.
+//
+// WHY A REGISTER AND NOT severity "info". Downgrading the severity would have the same routing
+// effect — `severityNotifies` drops info — but it would also relabel a genuine failure as
+// informational in the events table, the operator UI and every historical query, and it would
+// silently drop the X-Priority handling if the type were ever promoted back. This says what it
+// means: not silent, not urgent, RECORDED.
+//
+// IT IS NOT A WAY TO MUTE THINGS. A type belongs here only when something else carries its
+// notification. Adding one with no digest behind it rebuilds the silence R-182 was filed against.
+var recordOnlyEvents = map[string]bool{
+	// The per-app Tier-1 capture/refusal failure. Its notification is the run digest, which lists
+	// every failed app in one mail; this row is the durable per-failure record behind it.
+	"recovery_unit_capture_failed": true,
+}
+
 // operatorOnlyEvents are event types that must NEVER reach a customer, whatever their preferences say.
 //
 // R-97c. This register exists because the guarantee it provides was previously ASSERTED IN A COMMENT
@@ -321,6 +412,38 @@ var operatorOnlyEvents = map[string]bool{
 	// figures, the raw error). The customer's half of D-c is the FILL WARNING, which fires BEFORE
 	// this and is actionable: free space, delete files, add a drive.
 	"recovery_unit_capture_failed": true,
+	// R-182. The per-run backup digest. It is the same class as the line above and for the same
+	// reason — a customer can act on a full disk (that is the fill warning, which fires first and
+	// IS customer-facing) but not on a list of which apps' backups failed and why. It also carries
+	// operator-grade detail: per-app leg names, raw refusal reasons and byte figures.
+	//
+	// Listed here rather than relying on the absence of a `customerMessages` entry, which is NOT a
+	// block — `FormatCustomerEmail` falls back to the raw English message. That mistake shipped
+	// once (v0.78.0) and the comment above records it.
+	"backup_run_failures": true,
+	// R-197 (v0.93.0). "The sealed offsite repository key changed" is a custody fact about escrow
+	// blobs. A customer can take no action on it — the remedy is the operator's inspection of the
+	// off-site tier — and the text is operator-grade English naming host ids and retained-blob
+	// counts. Listed here in the SAME commit that mints the type: an operator-tier type that is not
+	// registered here reaches customers as raw English, because a missing customerMessages entry is
+	// NOT a block (the v0.78.0 defect recorded above).
+	"offsite_repo_key_changed": true,
+	// R-192 (v0.93.0). These two predate the register and were never added to it — a real gap, not a
+	// tidy-up. `offsite_delivery_stuck` is severity warning, has no customerMessages entry, and
+	// therefore fell through to FormatCustomerEmail's raw-English fallback: a customer whose box hit
+	// the stuck shape was in line for an English e-mail about one-time passwords being "likely
+	// burned". Measured on the live hub: notification_log holds operator rows for demo-hp and no
+	// customer rows — which is NOT evidence the leg is blocked (it is equally consistent with no
+	// configured recipient), so the register makes it structural instead of incidental. Narrowing
+	// only: the operator channel is untouched.
+	"offsite_delivery_stuck":      true,
+	"offsite_credential_restaged": true,
+	// R-199 (v0.94.0). A host retrieved its own sealed recovery blob. Operator-tier by construction:
+	// it names host ids and opaque byte counts, the customer can take no action on it, and its whole
+	// purpose is that the operator sees a capability being used. Registered in the same commit that
+	// mints the type — an operator-tier type absent from this register reaches customers as raw
+	// English (the v0.78.0 defect recorded above).
+	"escrow_blob_served": true,
 }
 
 // IsOperatorOnly reports whether an event type is barred from customer dispatch. Exported so the

hub/internal/notify/recovery_unit_dispatch_test.go

@@ -39,33 +39,45 @@ func TestRecoveryUnitCaptureFailed_NeverReachesTheCustomer(t *testing.T) {
 		}
 	}
 
-	// The operator must still get it: the register mutes the customer channel, not the signal.
-	gotOperator := false
+	// R-182 CHANGED WHAT THIS ASSERTS, DELIBERATELY, AND THE OLD ASSERTION IS WORTH KEEPING IN VIEW.
+	//
+	// Until 2026-08-03 this test required the OPERATOR to be e-mailed here, on the grounds that "the
+	// alert is the whole point of R-158". That was right when this event was the only signal, and it
+	// is wrong now: measured, nine of these arrived at the hub and two were mailed, because the
+	// operator cooldown key carries no app identifier — so as an alarm it told the operator about one
+	// app and threw the rest away.
+	//
+	// The type is now RECORD-ONLY: written down every time, never mailed. R-158's guarantee — the
+	// operator learns WHICH app failed and WHY — is not weakened, it MOVED: the per-run digest
+	// `backup_run_failures` carries every failed app in one mail, and is pinned by
+	// backup_run_digest_test.go. The customer safety claim below is untouched and is the reason this
+	// test still exists.
 	for _, to := range rec.to {
 		if to == "operator@felhom.eu" {
-			gotOperator = true
+			t.Fatal("the operator was e-mailed a PER-APP capture failure — this type is the record " +
+				"now, not the alarm. One mail per failing app on a full disk is a dozen mails, which " +
+				"is the volume problem the operator ruled against; the digest is the notification")
 		}
 	}
-	if !gotOperator {
-		t.Fatal("the operator was not notified of a recovery-unit capture failure — the alert is the " +
-			"whole point of R-158 and it went nowhere")
-	}
 
-	// The skip must be VISIBLE. An absent log row is equally consistent with "correctly skipped" and
-	// "the dispatcher never ran" — the positive observable is the row itself (standing rule 3).
+	// The RECORD must exist, always. It is what makes the digest trustworthy: if the digest is ever
+	// lost, delayed or suppressed, the failures are still individually written down. An absent row is
+	// equally consistent with "correctly not mailed" and "the dispatcher never ran" — the positive
+	// observable is the row itself (standing rule 3).
 	logs, err := st.GetRecentNotifications("c1", 20)
 	if err != nil {
 		t.Fatalf("GetRecentNotifications: %v", err)
 	}
 	found := false
 	for _, l := range logs {
-		if l.Channel == "customer" && l.Status == "skipped" && strings.Contains(l.ErrorMessage, "operator_only") {
+		if l.EventType == "recovery_unit_capture_failed" && l.Status == "recorded" &&
+			strings.Contains(l.ErrorMessage, "record-only") {
 			found = true
 		}
 	}
 	if !found {
-		t.Fatalf("the customer skip is not logged as skipped/operator_only — it is indistinguishable "+
-			"from a delivery that never happened; got %d row(s)", len(logs))
+		t.Fatalf("the per-app failure left no 'recorded' row — a failure that is neither mailed nor "+
+			"written down is exactly the 2026-08-03 defect, rebuilt; got %d row(s)", len(logs))
 	}
 }
 

hub/internal/notify/templates.go

@@ -1,6 +1,7 @@
 package notify
 
 import (
+	"encoding/json"
 	"fmt"
 	"strings"
 	"time"
@@ -41,6 +42,17 @@ Severity: %s
 Time:     %s
 Message:  %s`, customerID, eventType, severity, now, message)
 
+	// R-182: the backup run digest gets a rendered list instead of a raw JSON blob. It is the one
+	// operator mail that carries a VARIABLE-LENGTH payload, and a dozen apps as one line of JSON is
+	// unreadable on a phone at 07:00, which is the only time it matters.
+	if eventType == "backup_run_failures" {
+		if rendered, sub, ok := renderBackupRunFailures(customerID, detailsJSON); ok {
+			return sub, body + rendered + fmt.Sprintf("\n\nDashboard: https://hub.felhom.eu/customers/%s", customerID)
+		}
+		// Unparseable details fall through to the raw form below rather than losing the mail. A
+		// digest that renders badly still tells the operator something; a swallowed one does not.
+	}
+
 	if detailsJSON != "" && detailsJSON != "{}" {
 		body += fmt.Sprintf("\nDetails:  %s", detailsJSON)
 	}
@@ -282,3 +294,115 @@ Ha nem te kérted ezt, hagyd figyelmen kívül ezt az e-mailt.
 Felhom.eu`, link)
 	return subject, body
 }
+
+// ──────────────────────────────────────────────────────────────────────
+// R-182 — the backup run digest
+// ──────────────────────────────────────────────────────────────────────
+
+// backupRunFailure is one app's failed leg within a run.
+type backupRunFailure struct {
+	App    string `json:"app"`
+	Leg    string `json:"leg"`
+	Reason string `json:"reason"`
+}
+
+// backupRunDetails is the digest payload the controller sends.
+type backupRunDetails struct {
+	RunID       string             `json:"run_id"`
+	RunKind     string             `json:"run_kind"`
+	Failed      int                `json:"failed"`
+	Attempted   int                `json:"attempted"`
+	TargetPath  string             `json:"target_path"`
+	UsedGB      float64            `json:"used_gb"`
+	AvailGB     float64            `json:"avail_gb"`
+	TotalGB     float64            `json:"total_gb"`
+	UsedPercent float64            `json:"used_percent"`
+	SpaceKnown  bool               `json:"space_known"`
+	Apps        []backupRunFailure `json:"apps"`
+}
+
+// renderBackupRunFailures turns the digest details into an operator-readable block and a subject
+// that says the count without being opened. Returns ok=false when the payload cannot be parsed or
+// names no apps, so the caller can fall back to the raw rendering rather than mail an empty list.
+//
+// THE SUCCESS COUNT IS NOT DECORATION. "3 of 4 apps failed" is a catastrophe and "3 of 40" is a bad
+// night; the list alone cannot tell them apart, and the operator's first decision — get up now, or
+// look after coffee — depends entirely on which it is.
+func renderBackupRunFailures(customerID, detailsJSON string) (string, string, bool) {
+	if detailsJSON == "" {
+		return "", "", false
+	}
+	var d backupRunDetails
+	if err := json.Unmarshal([]byte(detailsJSON), &d); err != nil || len(d.Apps) == 0 {
+		return "", "", false
+	}
+
+	kind := d.RunKind
+	if kind == "" {
+		kind = "backup"
+	}
+	subject := fmt.Sprintf("[Felhom] 🔴 %s: %d of %d apps failed to back up (%s run)",
+		customerID, d.Failed, d.Attempted, kind)
+
+	// Column-align the app names so the leg and reason line up and the block scans vertically.
+	width := 0
+	for _, a := range d.Apps {
+		if len(a.App) > width {
+			width = len(a.App)
+		}
+	}
+	legWidth := 0
+	for _, a := range d.Apps {
+		if len(a.Leg) > legWidth {
+			legWidth = len(a.Leg)
+		}
+	}
+
+	var b strings.Builder
+	fmt.Fprintf(&b, "\n\nFAILED: %d of %d apps attempted in this %s run.\n\n", d.Failed, d.Attempted, kind)
+	for _, a := range d.Apps {
+		reason := trimRepeatedUsage(a.Reason, d.TargetPath)
+		if reason == "" {
+			reason = "(no reason recorded)"
+		}
+		fmt.Fprintf(&b, "  %-*s  %-*s  %s\n", width, a.App, legWidth, a.Leg, reason)
+	}
+
+	// The space figures answer "is this one broken app or a full disk" before the reasons are read.
+	// An absent reading renders as unavailable, never as zeros — "0 GB free" and "we could not look"
+	// are opposite diagnoses (the UnitSpace rule, same reasoning, other side of the wire).
+	if d.SpaceKnown {
+		fmt.Fprintf(&b, "\nFilesystem: %s — %.1f/%.1f GB used (%.0f%%), %.1f GB free\n",
+			d.TargetPath, d.UsedGB, d.TotalGB, d.UsedPercent, d.AvailGB)
+	} else {
+		fmt.Fprintf(&b, "\nFilesystem: %s — usage unavailable (the filesystem could not be read)\n", d.TargetPath)
+	}
+
+	b.WriteString("\nEvery failure above is also recorded individually in the notification log,\n")
+	b.WriteString("whether or not this mail was sent.")
+	return b.String(), subject, true
+}
+
+// trimRepeatedUsage strips the trailing "— /path: X/Y GB used (Z%), W GB free" clause from a per-app
+// reason, because the digest prints those figures ONCE for the whole run on its own line.
+//
+// This is a copy fix, and it was made after reading the first real digest rather than from the
+// design. The reserve's refusal message is authored for a single-app alert, where naming the
+// filesystem is exactly right; repeated down a list of a dozen apps it is the same forty characters
+// twelve times, and it pushes the part that differs off the right-hand edge of a phone screen at
+// 07:00 — which is the only moment this mail has to work.
+//
+// It trims ONLY an exact "— <target path>:" suffix, so a reason that mentions a different path, or
+// none, is left completely alone. A reason that is nothing but the usage clause is left alone too:
+// removing everything would turn a bad line into an empty one.
+func trimRepeatedUsage(reason, targetPath string) string {
+	if reason == "" || targetPath == "" {
+		return reason
+	}
+	marker := " — " + targetPath + ":"
+	i := strings.LastIndex(reason, marker)
+	if i <= 0 {
+		return reason
+	}
+	return strings.TrimSpace(reason[:i])
+}

hub/internal/offsite/offsite.go

@@ -195,13 +195,30 @@ func (p *Provisioner) ReissueCredentials(ctx context.Context, customerID, typ st
 		return fmt.Errorf("offsite: store re-issued one-time password: %w", err)
 	}
 
-	// v0.57.0 (2.3, the escrow-honesty fix): the restic repo password just changed, so any existing
-	// key-escrow blob — which sealed the OLD password — is now STALE. A recovery code minted against
-	// it would decrypt a password that no longer opens the repo. Mark the escrow stale so the hub
-	// stops advertising "ceremony done" and the customer's escrow wizard is offered again; a fresh
-	// ceremony seals the new password and clears the flag. Every credential change also emits a
-	// visible customer event (offsite_reissued always; escrow_stale only when a blob was invalidated).
-	// Best-effort: the password reset already succeeded — a bookkeeping failure here must not fail it.
+	// v0.57.0 (2.3, the escrow-honesty fix). ⚠ ITS STATED REASON WAS FALSE AND IS CORRECTED HERE
+	// (R-196, 2026-08-04). The comment used to read "the restic repo password just changed" — this
+	// function does NOT change it and CANNOT: the repository password is generated on the box
+	// (controller `WriteOffboxSecrets`) and never leaves it except sealed inside the R-wrapped escrow.
+	// What this function changes is the PROVIDER account password (①), which is a different secret at
+	// a different layer. The false premise survived because the EFFECT (a stale escrow) is real, so
+	// nobody checked the CAUSE — and it sent two separate investigations the wrong way in one day
+	// (audits/SPIKE-offsite-credential-recovery-2026-08-04.md Q4).
+	//
+	// WHY THE MARK IS STILL MADE, on honest grounds: a re-issue is the operator's response to a
+	// broken delivery, and the box's re-apply may mint a fresh repository password (it does exactly
+	// that whenever `<DataDir>/offbox/repo_password` is absent — the guest-rebuild shape). The mark is
+	// therefore PRECAUTIONARY, not evidential. The evidential signal is R-197's
+	// `offsite_repo_key_changed`, which fires on a measured hash difference at the next ceremony.
+	// KNOWN CONSEQUENCE, left in place deliberately: on the ordinary re-issue shape — a box that still
+	// holds its repo_password file — the password does NOT change, so this marks a healthy escrow
+	// stale and asks the customer for an unnecessary ceremony. R-196 stays OPEN for that; it is a
+	// behaviour change, not a comment fix, and it must not ride a comment-correction commit.
+	//
+	// Mark the escrow stale so the hub stops advertising "ceremony done" and the customer's escrow
+	// wizard is offered again; a fresh ceremony seals the current password and clears the flag. Every
+	// credential change also emits a visible customer event (offsite_reissued always; escrow_stale
+	// only when a blob was invalidated). Best-effort: the password reset already succeeded — a
+	// bookkeeping failure here must not fail it.
 	escrowStaled := false
 	if host, herr := p.Store.GetHostByCustomer(customerID); herr == nil && host != nil {
 		if esc, eerr := p.Store.GetHostEscrow(host.HostID); eerr == nil && esc != nil {

hub/internal/offsite/offsite_test.go

@@ -226,7 +226,7 @@ func TestReissue_InvalidatesEscrow(t *testing.T) {
 	if err := st.UpsertHost(&store.Host{HostID: cust + "-01", CustomerID: cust, APIKey: "k"}); err != nil {
 		t.Fatal(err)
 	}
-	if _, err := st.SaveHostEscrow(cust+"-01", []byte("opaque-blob"), "SHA256:fp", "zero_knowledge", "2026-07-16T00:00:00Z", "OLDHASH"); err != nil {
+	if _, _, err := st.SaveHostEscrow(cust+"-01", []byte("opaque-blob"), "SHA256:fp", "zero_knowledge", "2026-07-16T00:00:00Z", "OLDHASH"); err != nil {
 		t.Fatal(err)
 	}
 	// Before re-issue: current escrow — the hub serves the sealed hash and is NOT stale.
@@ -256,7 +256,7 @@ func TestReissue_InvalidatesEscrow(t *testing.T) {
 	}
 
 	// A fresh ceremony (new blob sealing the new password) clears stale + serves the new hash.
-	if _, err := st.SaveHostEscrow(cust+"-01", []byte("opaque-blob-2"), "SHA256:fp", "zero_knowledge", "2026-07-16T01:00:00Z", "NEWHASH"); err != nil {
+	if _, _, err := st.SaveHostEscrow(cust+"-01", []byte("opaque-blob-2"), "SHA256:fp", "zero_knowledge", "2026-07-16T01:00:00Z", "NEWHASH"); err != nil {
 		t.Fatal(err)
 	}
 	es, _ = st.GetEscrowStatusForCustomer(cust)

hub/internal/store/customer_reset_test.go

@@ -25,10 +25,10 @@ func seedRetainedBlob(t *testing.T, st *Store, customerID string) {
 	if err := st.UpsertHost(&Host{HostID: hostID, CustomerID: customerID, APIKey: "h"}); err != nil {
 		t.Fatalf("upsert host: %v", err)
 	}
-	if _, err := st.SaveHostEscrow(hostID, []byte("A"), "fpA", "p", "2026-01-01T00:00:00Z", "shaA"); err != nil {
+	if _, _, err := st.SaveHostEscrow(hostID, []byte("A"), "fpA", "p", "2026-01-01T00:00:00Z", "shaA"); err != nil {
 		t.Fatalf("escrow A: %v", err)
 	}
-	if _, err := st.SaveHostEscrow(hostID, []byte("B"), "fpB", "p", "2026-01-02T00:00:00Z", "shaB"); err != nil {
+	if _, _, err := st.SaveHostEscrow(hostID, []byte("B"), "fpB", "p", "2026-01-02T00:00:00Z", "shaB"); err != nil {
 		t.Fatalf("escrow B: %v", err)
 	}
 	if err := st.DeleteHost(hostID, true); err != nil {

hub/internal/store/escrow_identity_retention_test.go

@@ -0,0 +1,168 @@
+package store
+
+import (
+	"bytes"
+	"testing"
+)
+
+// R-198 (v0.93.0) — the retention must keep the OFF-SITE data key, not only the PBS key.
+//
+// WHAT WAS BROKEN, and why these tests are the ones that would have caught it: host_escrow_superseded
+// shipped with `blob` (the K-escrow / PBS datastore key) and `identity_blob` was added to host_escrow
+// by a later ALTER and never to the retained table. The restic REPOSITORY password lives inside
+// identity_blob. So every supersession retained the whole-guest key and destroyed the off-site data
+// key — the exact secret the retention exists to preserve — and the destroying act is the escrow
+// ceremony a rebuilt box asks its customer to run.
+//
+// The pre-existing TestSaveHostEscrow_RetainsSuperseded was GREEN throughout, because it asserts the
+// MECHANISM (a retained row exists, with the old K-blob) and not the CONSEQUENCE (the retained row can
+// still yield a repository password). These assert the consequence.
+
+// Scenario A — a re-escrow retains BOTH sealed keys.
+// RED-PROOF: drop `identity_blob` from demoteCurrentEscrowTx's INSERT/SELECT (production behaviour up
+// to v0.92.0) → the retained row's identity blob is nil → this FAILS.
+func TestSaveHostEscrow_RetainsIdentityBlob(t *testing.T) {
+	st := newTestStore(t)
+	const h = "hid-1"
+	oldIdentity := []byte("age-blob-sealing-REPO-PASSWORD-OLD")
+	newIdentity := []byte("age-blob-sealing-REPO-PASSWORD-NEW")
+
+	// Generation 1: the K-escrow, then the identity blob — the real order the escrow PUT uses
+	// (SaveHostEscrow, then SaveHostDRBundle).
+	if _, _, err := st.SaveHostEscrow(h, []byte("k-blob-old"), "fp-old", "zk", "2026-07-09T00:00:00Z", "SHA_OLD"); err != nil {
+		t.Fatal(err)
+	}
+	if err := st.SaveHostDRBundle(h, oldIdentity, `{"gen":1}`); err != nil {
+		t.Fatal(err)
+	}
+
+	// Generation 2 with a DIFFERENT sealed repo password → supersede.
+	sup, prev, err := st.SaveHostEscrow(h, []byte("k-blob-new"), "fp-new", "zk", "2026-07-16T00:00:00Z", "SHA_NEW")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !sup {
+		t.Fatal("a different-passphrase upload must supersede")
+	}
+	if prev != "SHA_OLD" {
+		t.Fatalf("prevResticPwSHA256 = %q, want SHA_OLD (R-197 needs the replaced hash)", prev)
+	}
+	if err := st.SaveHostDRBundle(h, newIdentity, `{"gen":2}`); err != nil {
+		t.Fatal(err)
+	}
+
+	retained, err := st.ListSupersededEscrow(h)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(retained) != 1 {
+		t.Fatalf("retained rows = %d, want 1", len(retained))
+	}
+	// THE ASSERTION THIS WHOLE ITEM IS ABOUT: the off-site data key survived the supersession.
+	if retained[0].IdentityBlob == nil {
+		t.Fatal("R-198: the retained row carries NO identity blob — the off-site repository password " +
+			"was destroyed by the ceremony that was supposed to preserve it")
+	}
+	// And it is the PREVIOUS generation's blob, not the one that replaced it. This pins the ordering
+	// dependency named on demoteCurrentEscrowTx: the identity blob is written AFTER SaveHostEscrow, so
+	// the demote sees the old one. If that order ever inverts, the retained bytes would silently be
+	// the new blob filed under the old hash — recoverable-looking and wrong.
+	if !bytes.Equal(retained[0].IdentityBlob, oldIdentity) {
+		t.Fatalf("retained identity blob is not the PREVIOUS generation (got %q) — the demote ran after the overwrite",
+			retained[0].IdentityBlob)
+	}
+	if retained[0].ResticPwSHA256 != "SHA_OLD" || string(retained[0].Blob) != "k-blob-old" {
+		t.Fatalf("retained row is not the old generation: %+v", retained[0])
+	}
+	// Current row unchanged in behaviour: the NEW generation, both blobs.
+	if bundle, berr := st.GetHostDRBundle(h); berr != nil || bundle == nil || !bytes.Equal(bundle.IdentityBlob, newIdentity) {
+		t.Fatalf("current identity blob is not the new one: %+v (%v)", bundle, berr)
+	}
+}
+
+// Scenario B — deleting a host demotes BOTH blobs too. demoteCurrentEscrowTx is shared by the
+// re-escrow path and the host-delete path; a shared routine proven through one caller is how a fix
+// gets believed on a path nobody exercised.
+// RED-PROOF: fix only the re-escrow caller (e.g. carry the column in SaveHostEscrow's own SQL instead
+// of in the shared routine) → this FAILS while Scenario A passes.
+func TestDeleteHost_DemotesIdentityBlob(t *testing.T) {
+	s := newTestStore(t)
+	const hostID, cust = "hid-del", "cust-del"
+	identity := []byte("age-blob-sealing-REPO-PASSWORD")
+	if err := s.UpsertHost(&Host{HostID: hostID, CustomerID: cust, APIKey: "k"}); err != nil {
+		t.Fatal(err)
+	}
+	if _, _, err := s.SaveHostEscrow(hostID, []byte("k-blob"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA_A"); err != nil {
+		t.Fatal(err)
+	}
+	if err := s.SaveHostDRBundle(hostID, identity, `{}`); err != nil {
+		t.Fatal(err)
+	}
+
+	if err := s.DeleteHost(hostID, true); err != nil {
+		t.Fatalf("DeleteHost: %v", err)
+	}
+	retained, err := s.ListSupersededEscrow(hostID)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(retained) != 1 {
+		t.Fatalf("demoted rows = %d, want 1", len(retained))
+	}
+	if !bytes.Equal(retained[0].IdentityBlob, identity) {
+		t.Fatalf("R-198: host delete demoted custody WITHOUT the identity blob (got %q) — the off-site "+
+			"repository password was destroyed by a host delete", retained[0].IdentityBlob)
+	}
+}
+
+// Scenario C — a legacy host whose current escrow has NO identity blob supersedes cleanly. The column
+// is nullable on purpose: a NOT NULL constraint here would make the fix block a ceremony, which is a
+// worse failure than the one it repairs.
+func TestSaveHostEscrow_SupersedesWithoutIdentityBlob(t *testing.T) {
+	st := newTestStore(t)
+	const h = "hid-legacy"
+	if _, _, err := st.SaveHostEscrow(h, []byte("k-old"), "fp", "zk", "2026-07-09T00:00:00Z", "SHA_OLD"); err != nil {
+		t.Fatal(err)
+	}
+	// No SaveHostDRBundle — a slice-7-era upload.
+	sup, prev, err := st.SaveHostEscrow(h, []byte("k-new"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA_NEW")
+	if err != nil {
+		t.Fatalf("a supersession of an identity-less escrow must not fail: %v", err)
+	}
+	if !sup || prev != "SHA_OLD" {
+		t.Fatalf("superseded=%v prev=%q, want true/SHA_OLD", sup, prev)
+	}
+	retained, err := st.ListSupersededEscrow(h)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(retained) != 1 {
+		t.Fatalf("retained rows = %d, want 1", len(retained))
+	}
+	if retained[0].IdentityBlob != nil {
+		t.Fatalf("a legacy row must retain a NULL identity blob, got %q", retained[0].IdentityBlob)
+	}
+}
+
+// CountCurrentEscrowWithIdentity is the census §8.1 asks for: which hosts hold an identity blob today
+// and are therefore protected from the next ceremony by this fix. Asserted rather than eyeballed,
+// because the report quotes its numbers.
+func TestCountCurrentEscrowWithIdentity(t *testing.T) {
+	st := newTestStore(t)
+	if _, _, err := st.SaveHostEscrow("with-id", []byte("k"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA1"); err != nil {
+		t.Fatal(err)
+	}
+	if err := st.SaveHostDRBundle("with-id", []byte("age-blob"), `{}`); err != nil {
+		t.Fatal(err)
+	}
+	if _, _, err := st.SaveHostEscrow("without-id", []byte("k"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA2"); err != nil {
+		t.Fatal(err)
+	}
+	total, withIdentity, err := st.CountCurrentEscrowWithIdentity()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if total != 2 || withIdentity != 1 {
+		t.Fatalf("census = %d/%d, want 2 total / 1 with identity", withIdentity, total)
+	}
+}

hub/internal/store/escrow_superseded_test.go

@@ -11,7 +11,7 @@ func TestSaveHostEscrow_RetainsSuperseded(t *testing.T) {
 	const h = "h1"
 
 	// 1st upload (P_old) — nothing to supersede.
-	sup, err := st.SaveHostEscrow(h, []byte("blob-old"), "fp-old", "zk", "2026-07-09T00:00:00Z", "SHA_OLD")
+	sup, _, err := st.SaveHostEscrow(h, []byte("blob-old"), "fp-old", "zk", "2026-07-09T00:00:00Z", "SHA_OLD")
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -20,7 +20,7 @@ func TestSaveHostEscrow_RetainsSuperseded(t *testing.T) {
 	}
 
 	// 2nd upload (P_new, DIFFERENT sha) — must supersede + retain the old.
-	sup, err = st.SaveHostEscrow(h, []byte("blob-new"), "fp-new", "zk", "2026-07-16T00:00:00Z", "SHA_NEW")
+	sup, _, err = st.SaveHostEscrow(h, []byte("blob-new"), "fp-new", "zk", "2026-07-16T00:00:00Z", "SHA_NEW")
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -45,7 +45,7 @@ func TestSaveHostEscrow_RetainsSuperseded(t *testing.T) {
 	}
 
 	// 3rd upload, SAME sha as current — idempotent (re-ceremony of the same password): NO supersede row.
-	sup, err = st.SaveHostEscrow(h, []byte("blob-new-2"), "fp-new", "zk", "2026-07-16T01:00:00Z", "SHA_NEW")
+	sup, _, err = st.SaveHostEscrow(h, []byte("blob-new-2"), "fp-new", "zk", "2026-07-16T01:00:00Z", "SHA_NEW")
 	if err != nil {
 		t.Fatal(err)
 	}

hub/internal/store/host_delete_demote_test.go

@@ -15,10 +15,10 @@ func TestDeleteHost_DemotesEscrowNeverDestroys(t *testing.T) {
 		t.Fatal(err)
 	}
 	// current escrow = SHA_A, one superseded = SHA_OLD (two uploads with different passphrases).
-	if _, err := s.SaveHostEscrow(hostID, []byte("blob-old"), "fp", "zk", "2026-07-09T00:00:00Z", "SHA_OLD"); err != nil {
+	if _, _, err := s.SaveHostEscrow(hostID, []byte("blob-old"), "fp", "zk", "2026-07-09T00:00:00Z", "SHA_OLD"); err != nil {
 		t.Fatal(err)
 	}
-	if _, err := s.SaveHostEscrow(hostID, []byte("blob-A"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA_A"); err != nil {
+	if _, _, err := s.SaveHostEscrow(hostID, []byte("blob-A"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA_A"); err != nil {
 		t.Fatal(err)
 	}
 	if n, _ := s.CountSupersededEscrow(hostID); n != 1 {

hub/internal/store/host_delete_test.go

@@ -57,7 +57,7 @@ func seedHostWithArtifacts(t *testing.T, s *Store, hostID, customerID string) {
 		"PK-"+hostID, "ip-"+hostID, hostID); err != nil {
 		t.Fatal(err)
 	}
-	if _, err := s.SaveHostEscrow(hostID, []byte("opaque-escrow"), "fp", "posture", "2026-07-01T00:00:00Z", ""); err != nil {
+	if _, _, err := s.SaveHostEscrow(hostID, []byte("opaque-escrow"), "fp", "posture", "2026-07-01T00:00:00Z", ""); err != nil {
 		t.Fatal(err)
 	}
 }

hub/internal/store/store.go

@@ -374,6 +374,22 @@ func (s *Store) migrate() error {
 		-- "history destroyed" into "history recoverable with the recovery code"). Append-only; the hub
 		-- never decrypts; NO pruning (the blobs are tiny + R-encrypted; custody unchanged). The ACK and
 		-- restore-serving read host_escrow (the CURRENT row) — never this table.
+		--
+		-- THE RULING ABOVE WAS NOT MET FOR TWO MONTHS, AND THIS IS THE RECORD OF IT (R-198, fixed
+		-- v0.93.0). This table shipped with the blob column — the K-escrow, i.e. the PBS datastore key
+		-- — and identity_blob was added to host_escrow LATER (the slice-10D ALTER below), never here.
+		-- The offsite restic REPOSITORY password lives in identity_blob, not in blob. So the retention
+		-- preserved the whole-guest key and silently dropped the off-site data key: precisely the
+		-- secret the reinstall-orphan incident was about. Worse, the copy happens as the new blob
+		-- overwrites the old, so the destroying act was the ESCROW CEREMONY — the exact thing a
+		-- rebuilt box asks its customer to run, on a card promising the old backups stay recoverable.
+		-- Both demo boxes crossed that line on 2026-08-04 (07:15:36 and 07:20:08) and their previous
+		-- repository passwords are unrecoverable, recovery code or not.
+		-- identity_blob is now carried (see demoteCurrentEscrowTx, which is still THE ONE row-copy
+		-- routine). Pinned by TestSaveHostEscrow_RetainsIdentityBlob and
+		-- TestDeleteHost_DemotesIdentityBlob — the routine is proven through BOTH of its callers,
+		-- because a shared routine tested through one caller is how a fix gets believed on a path
+		-- nobody exercised. Evidence: audits/RECON-offsite-dr-chain-2026-08-04.md §7.
 		CREATE TABLE IF NOT EXISTS host_escrow_superseded (
 			id               INTEGER PRIMARY KEY AUTOINCREMENT,
 			host_id          TEXT NOT NULL,
@@ -382,7 +398,8 @@ func (s *Store) migrate() error {
 			posture          TEXT NOT NULL DEFAULT '',
 			created_at       DATETIME NOT NULL,
 			restic_pw_sha256 TEXT NOT NULL DEFAULT '',
-			superseded_at    DATETIME NOT NULL DEFAULT (datetime('now'))
+			superseded_at    DATETIME NOT NULL DEFAULT (datetime('now')),
+			identity_blob    BLOB
 		);
 		CREATE INDEX IF NOT EXISTS idx_host_escrow_superseded_host ON host_escrow_superseded(host_id);
 
@@ -414,16 +431,28 @@ func (s *Store) migrate() error {
 	s.db.Exec(`ALTER TABLE host_escrow ADD COLUMN identity_blob BLOB`)
 	s.db.Exec(`ALTER TABLE host_escrow ADD COLUMN directive_json TEXT NOT NULL DEFAULT '{}'`)
 
+	// R-198 (v0.93.0) — the SAME column on the RETAINED table. It was added above and not here, and
+	// that omission is what made the retention keep the wrong key for two months (see the comment on
+	// host_escrow_superseded). Additive and tolerated on re-run, exactly like the lines above; it
+	// changes no existing row. Rows superseded BEFORE this ships were written without the identity
+	// blob and their source rows are already overwritten — there is nothing to backfill, and the
+	// v0.93.0 report records that as a looked-at fact rather than a deduction.
+	s.db.Exec(`ALTER TABLE host_escrow_superseded ADD COLUMN identity_blob BLOB`)
+
 	// SLICE 3 (escrow auto-confirm) — sha256 hex of the offsite restic repo password sealed in the
 	// identity blob. The hash of a 256-bit random secret is non-reversible/non-brute-forceable — safe to
 	// store and serve; it lets the controller VERIFY "the escrow covers the CURRENT repo password"
 	// instead of trusting blob-presence. NULL/'' = a legacy or password-less blob (never auto-confirms).
 	s.db.Exec(`ALTER TABLE host_escrow ADD COLUMN restic_pw_sha256 TEXT`)
 
-	// v0.57.0 (2.3, escrow honesty on offsite re-issue) — stale_at is set when the offsite repo
-	// password is re-issued: the blob then seals a password that no longer opens the repo, so the
-	// hub must stop advertising "ceremony done" and withhold the (now non-matching) restic_pw_sha256
-	// from the auto-confirm ACK. NULL = current; a fresh ceremony (SaveHostEscrow) clears it.
+	// v0.57.0 (2.3, escrow honesty on offsite re-issue) — stale_at is set when the offsite PROVIDER
+	// credentials are re-issued. ⚠ CORRECTED 2026-08-04 (R-196): it used to say "when the offsite repo
+	// password is re-issued", which nothing in the hub does — the repository password is generated on
+	// the box and never leaves it except sealed under R. The flag is PRECAUTIONARY (the box's re-apply
+	// MAY mint a fresh repository password), not evidence that it changed; the evidential signal is
+	// R-197's offsite_repo_key_changed. While set, the hub stops advertising "ceremony done" and
+	// withholds the possibly-non-matching restic_pw_sha256 from the auto-confirm ACK. NULL = current;
+	// a fresh ceremony (SaveHostEscrow) clears it.
 	s.db.Exec(`ALTER TABLE host_escrow ADD COLUMN stale_at DATETIME`)
 
 	// dr_recipe (SPIKE-dr-recipe-2026-06-16): the secret-free DR reconstruction recipe, stored
@@ -815,7 +844,7 @@ type NotificationLogEntry struct {
 	EventType    string
 	Severity     string
 	Message      string
-	Status       string // "sent", "skipped", "failed"
+	Status       string // "sent", "skipped", "failed", "suppressed" (R-182: a cooldown drop, recorded rather than silent)
 	ErrorMessage string
 	Channel      string // "operator" or "customer"
 	CreatedAt    time.Time
@@ -2262,6 +2291,28 @@ func (s *Store) GetHostByCustomer(customerID string) (*Host, error) {
 	return h, err
 }
 
+// HasEverBoundHost reports whether a machine was EVER bound to this customer — a live row in
+// `hosts` OR a tombstone in `host_deletions`. It answers "was anything ever expected of this
+// customer", which is the question the deadline verdicts actually need (R-195).
+//
+// It is deliberately NOT "has a report arrived", and the distinction is the whole point: a box
+// that was installed, bound, and then went silent IS bound, and its silence is a real fault that
+// must keep alarming. Only a customer that never had a machine at all is UNKNOWN.
+//
+// `host_deletions` is included because a customer whose host was removed HAD one — the deadline
+// caller reaches its down-skip for that shape, and this predicate must not quietly take over a
+// judgement the staleness checker owns.
+func (s *Store) HasEverBoundHost(customerID string) (bool, error) {
+	var n int
+	if err := s.db.QueryRow(
+		`SELECT EXISTS(SELECT 1 FROM hosts WHERE customer_id = ?)
+		     OR EXISTS(SELECT 1 FROM host_deletions WHERE customer_id = ?)`,
+		customerID, customerID).Scan(&n); err != nil {
+		return false, err
+	}
+	return n != 0, nil
+}
+
 // ListHostsByCustomer returns the customer's hosts ordered by host_id (v0.47.0 — the
 // customer page's Host tab is a LIST by design: 1 host today, N for a later HA cluster).
 // Uses the idx_hosts_customer index.
@@ -2507,6 +2558,11 @@ type HostEscrow struct {
 	// StaleAt (v0.57.0, 2.3) — non-empty when the offsite password was re-issued after this blob was
 	// sealed: the blob is stale (seals a password that no longer opens the repo). Cleared by a fresh ceremony.
 	StaleAt string
+	// IdentityBlob (R-198, v0.93.0) — the age-wrapped identity bundle, which is where the offsite
+	// restic REPOSITORY password lives. Populated by ListSupersededEscrow so a retained blob is
+	// reachable from Go at all; nil for pre-v0.93.0 retained rows and for hosts that never uploaded
+	// one. Opaque: useless without the customer's recovery code, which the hub never holds.
+	IdentityBlob []byte
 }
 
 // SaveHostEscrow stores (last-write-wins) the OPAQUE escrow blob for a host. The hub keeps the
@@ -2516,16 +2572,35 @@ type HostEscrow struct {
 // host_escrow_superseded before overwriting the current row (Part B, v0.60.0). A same-sha re-upload
 // (idempotent re-ceremony of the same password) refreshes the current row and does NOT create a
 // superseded row.
+//
+// R-197 (v0.93.0): it also returns prevResticPwSHA256 — the hash the row being replaced sealed ("" when
+// no row existed, or when it was a legacy hash-less blob). Both halves of "did the box's offsite DATA
+// key change?" have been in this database since SLICE 3 and NOTHING compared them; demo-felhom's key
+// changed on 2026-08-03 and no signal of any kind fired for thirteen hours. The comparison is the
+// caller's (api.handleHostEscrowPut) because the event needs the customer id; returning the value is
+// this function's part. The VALUE is a non-reversible hash of a 256-bit random secret and is never
+// logged, mailed or written to a report — see the caller.
 // demoteCurrentEscrowTx copies the host's CURRENT host_escrow row (if any) into
 // host_escrow_superseded as a retained blob, inside the given tx. This is THE ONE escrow row-copy
 // routine (v0.60.0): SaveHostEscrow uses it to retain a superseded different-passphrase blob before
 // overwriting, and DeleteHost (v0.60.1) uses it to DEMOTE the current blob to retained custody
 // instead of destroying it. Returns the number of rows copied (0 when the host has no current row).
 // The hub never decrypts; custody is unchanged.
+//
+// R-198 (v0.93.0): `identity_blob` is copied too. It was omitted from this SELECT for two months, so
+// every supersession retained the PBS datastore key and destroyed the offsite restic repository
+// password — the one secret the retention exists to preserve. Copying more opaque bytes gains the hub
+// NO knowledge: it still has no recovery code and no decrypt path.
+//
+// ORDERING THIS DEPENDS ON, stated because it is load-bearing and invisible from here: the identity
+// blob is written by SaveHostDRBundle AFTER SaveHostEscrow returns (api/handler.go, the escrow PUT),
+// so at demote time host_escrow still holds the OLD identity blob. If that order ever changes, this
+// routine silently retains the NEW blob under the OLD blob's hash — pinned by
+// TestSaveHostEscrow_RetainsIdentityBlob, which asserts the retained bytes are the previous ones.
 func demoteCurrentEscrowTx(tx *sql.Tx, hostID string) (int64, error) {
 	res, err := tx.Exec(`
-		INSERT INTO host_escrow_superseded (host_id, blob, key_fingerprint, posture, created_at, restic_pw_sha256, superseded_at)
-		SELECT host_id, blob, key_fingerprint, posture, created_at, COALESCE(restic_pw_sha256, ''), datetime('now')
+		INSERT INTO host_escrow_superseded (host_id, blob, key_fingerprint, posture, created_at, restic_pw_sha256, superseded_at, identity_blob)
+		SELECT host_id, blob, key_fingerprint, posture, created_at, COALESCE(restic_pw_sha256, ''), datetime('now'), identity_blob
 		FROM host_escrow WHERE host_id = ?`, hostID)
 	if err != nil {
 		return 0, err
@@ -2533,10 +2608,10 @@ func demoteCurrentEscrowTx(tx *sql.Tx, hostID string) (int64, error) {
 	return res.RowsAffected()
 }
 
-func (s *Store) SaveHostEscrow(hostID string, blob []byte, keyFingerprint, posture, createdAt, resticPwSHA256 string) (superseded bool, err error) {
+func (s *Store) SaveHostEscrow(hostID string, blob []byte, keyFingerprint, posture, createdAt, resticPwSHA256 string) (superseded bool, prevResticPwSHA256 string, err error) {
 	tx, err := s.db.Begin()
 	if err != nil {
-		return false, err
+		return false, "", err
 	}
 	defer func() {
 		if err != nil {
@@ -2555,11 +2630,14 @@ func (s *Store) SaveHostEscrow(hostID string, blob []byte, keyFingerprint, postu
 		exists = false
 	default:
 		err = scanErr
-		return false, err
+		return false, "", err
+	}
+	if exists {
+		prevResticPwSHA256 = curSHA // R-197: the caller compares; "" = no row or a legacy hash-less blob
 	}
 	if exists && curSHA != resticPwSHA256 {
 		if _, err = demoteCurrentEscrowTx(tx, hostID); err != nil {
-			return false, err
+			return false, prevResticPwSHA256, err
 		}
 		superseded = true
 	}
@@ -2576,10 +2654,10 @@ func (s *Store) SaveHostEscrow(hostID string, blob []byte, keyFingerprint, postu
 			stale_at = NULL,
 			updated_at = datetime('now')`,
 		hostID, blob, keyFingerprint, posture, createdAt, resticPwSHA256); err != nil {
-		return false, err
+		return false, prevResticPwSHA256, err
 	}
 	err = tx.Commit()
-	return superseded, err
+	return superseded, prevResticPwSHA256, err
 }
 
 // CountSupersededEscrow returns how many retained (superseded) escrow blobs the hub holds for a host
@@ -2594,7 +2672,7 @@ func (s *Store) CountSupersededEscrow(hostID string) (int, error) {
 // first. Opaque bytes — the hub never decrypts. Seeds the future guided-recovery flow (R-26).
 func (s *Store) ListSupersededEscrow(hostID string) ([]HostEscrow, error) {
 	rows, err := s.db.Query(`
-		SELECT host_id, blob, key_fingerprint, posture, created_at, restic_pw_sha256, superseded_at
+		SELECT host_id, blob, key_fingerprint, posture, created_at, restic_pw_sha256, superseded_at, identity_blob
 		FROM host_escrow_superseded WHERE host_id = ? ORDER BY id DESC`, hostID)
 	if err != nil {
 		return nil, err
@@ -2603,7 +2681,7 @@ func (s *Store) ListSupersededEscrow(hostID string) ([]HostEscrow, error) {
 	var out []HostEscrow
 	for rows.Next() {
 		var e HostEscrow
-		if err := rows.Scan(&e.HostID, &e.Blob, &e.KeyFingerprint, &e.Posture, &e.CreatedAt, &e.ResticPwSHA256, &e.UpdatedAt); err != nil {
+		if err := rows.Scan(&e.HostID, &e.Blob, &e.KeyFingerprint, &e.Posture, &e.CreatedAt, &e.ResticPwSHA256, &e.UpdatedAt, &e.IdentityBlob); err != nil {
 			return nil, err
 		}
 		out = append(out, e)
@@ -2611,10 +2689,22 @@ func (s *Store) ListSupersededEscrow(hostID string) ([]HostEscrow, error) {
 	return out, rows.Err()
 }
 
-// MarkEscrowStale flags a host's escrow blob as stale (v0.57.0, 2.3) — called when the offsite repo
-// password is re-issued, because the blob then seals a password that no longer opens the repo. No-op
-// when no escrow row exists; idempotent (only stamps the first re-issue since the last ceremony; a
-// fresh ceremony clears stale_at via SaveHostEscrow's ON CONFLICT).
+// CountCurrentEscrowWithIdentity returns how many hosts hold a CURRENT escrow row, and how many of
+// those carry an identity blob — i.e. the population whose offsite repository password R-198's fix
+// now protects from the next ceremony, and the remainder for whom there is nothing to protect
+// because no identity blob was ever uploaded. Read-only; no blob or hash leaves this call.
+func (s *Store) CountCurrentEscrowWithIdentity() (total, withIdentity int, err error) {
+	err = s.db.QueryRow(`
+		SELECT COUNT(*), COALESCE(SUM(identity_blob IS NOT NULL), 0) FROM host_escrow`).Scan(&total, &withIdentity)
+	return total, withIdentity, err
+}
+
+// MarkEscrowStale flags a host's escrow blob as stale (v0.57.0, 2.3) — called when the offsite
+// PROVIDER credentials are re-issued. ⚠ CORRECTED 2026-08-04 (R-196): it used to say "when the offsite
+// repo password is re-issued", which no hub path does; see the reasoning at offsite.ReissueCredentials.
+// The flag is precautionary — the box's re-apply may mint a fresh repository password — not a
+// measurement that one did. No-op when no escrow row exists; idempotent (only stamps the first
+// re-issue since the last ceremony; a fresh ceremony clears stale_at via SaveHostEscrow's ON CONFLICT).
 func (s *Store) MarkEscrowStale(hostID string) error {
 	_, err := s.db.Exec(`UPDATE host_escrow SET stale_at = datetime('now') WHERE host_id = ? AND stale_at IS NULL`, hostID)
 	return err
@@ -2643,9 +2733,11 @@ type EscrowStatus struct {
 	IdentityBlobPresent bool   `json:"identity_blob_present"`
 	ResticPwSHA256      string `json:"restic_pw_sha256,omitempty"`
 	CreatedAt           string `json:"created_at,omitempty"`
-	// Stale (v0.57.0, 2.3) — true when the offsite password was re-issued after the blob was sealed.
+	// Stale (v0.57.0, 2.3) — true when the offsite PROVIDER credentials were re-issued after the blob
+	// was sealed. ⚠ CORRECTED 2026-08-04 (R-196): it used to say "the offsite password was re-issued",
+	// which reads as the repository password and is not what happens; see MarkEscrowStale.
 	// When stale the ResticPwSHA256 is WITHHELD (emptied) so the controller cannot auto-confirm against
-	// a hash that no longer matches the live repo password — the ceremony must run again.
+	// a hash that may no longer match the live repo password — the ceremony must run again.
 	Stale bool `json:"escrow_stale,omitempty"`
 }
 

hub/internal/web/customer_delete_test.go

@@ -74,11 +74,11 @@ func seedDeletable(t *testing.T, st *store.Store, customerID string) string {
 	if err := st.UpsertHost(&store.Host{HostID: hostID, CustomerID: customerID, APIKey: "hapi-" + hostID, LastReportAt: &long}); err != nil {
 		t.Fatalf("seed host: %v", err)
 	}
-	if _, err := st.SaveHostEscrow(hostID, []byte("blobA"), "fpA", "posture", "2026-01-01T00:00:00Z", "shaA"); err != nil {
+	if _, _, err := st.SaveHostEscrow(hostID, []byte("blobA"), "fpA", "posture", "2026-01-01T00:00:00Z", "shaA"); err != nil {
 		t.Fatalf("seed escrow A: %v", err)
 	}
 	// A second save supersedes A → one RETAINED blob + one CURRENT blob before the cascade runs.
-	if _, err := st.SaveHostEscrow(hostID, []byte("blobB"), "fpB", "posture", "2026-01-02T00:00:00Z", "shaB"); err != nil {
+	if _, _, err := st.SaveHostEscrow(hostID, []byte("blobB"), "fpB", "posture", "2026-01-02T00:00:00Z", "shaB"); err != nil {
 		t.Fatalf("seed escrow B: %v", err)
 	}
 	if err := st.SaveOneTimeSecret(customerID, "one-time-pw"); err != nil {

hub/internal/web/customer_reset_test.go

@@ -37,10 +37,10 @@ func seedResettable(t *testing.T, st *store.Store, customerID string) {
 	if err := st.UpsertHost(&store.Host{HostID: hostID, CustomerID: customerID, APIKey: "hapi"}); err != nil {
 		t.Fatalf("seed host: %v", err)
 	}
-	if _, err := st.SaveHostEscrow(hostID, []byte("blobA"), "fpA", "posture", "2026-01-01T00:00:00Z", "shaA"); err != nil {
+	if _, _, err := st.SaveHostEscrow(hostID, []byte("blobA"), "fpA", "posture", "2026-01-01T00:00:00Z", "shaA"); err != nil {
 		t.Fatalf("seed escrow A: %v", err)
 	}
-	if _, err := st.SaveHostEscrow(hostID, []byte("blobB"), "fpB", "posture", "2026-01-02T00:00:00Z", "shaB"); err != nil {
+	if _, _, err := st.SaveHostEscrow(hostID, []byte("blobB"), "fpB", "posture", "2026-01-02T00:00:00Z", "shaB"); err != nil {
 		t.Fatalf("seed escrow B: %v", err)
 	}
 	if err := st.DeleteHost(hostID, true); err != nil { // demotes current → retained; records host_deletions

hub/internal/web/hosts_delete_test.go

@@ -64,7 +64,7 @@ func TestHostDelete_EscrowAckRequired(t *testing.T) {
 	if err := st.UpsertHost(&store.Host{HostID: "esc-host", CustomerID: "c2", APIKey: "k"}); err != nil {
 		t.Fatal(err)
 	}
-	if _, err := st.SaveHostEscrow("esc-host", []byte("blob"), "fp", "p", "2026-07-01T00:00:00Z", ""); err != nil {
+	if _, _, err := st.SaveHostEscrow("esc-host", []byte("blob"), "fp", "p", "2026-07-01T00:00:00Z", ""); err != nil {
 		t.Fatal(err)
 	}
 
@@ -110,7 +110,7 @@ func TestHostDelete_ImpactJSON(t *testing.T) {
 	if err := st.UpsertHost(&store.Host{HostID: "imp-host", CustomerID: "c4", APIKey: "SECRET-KEY"}); err != nil {
 		t.Fatal(err)
 	}
-	if _, err := st.SaveHostEscrow("imp-host", []byte("blob"), "fp", "p", "2026-07-01T00:00:00Z", ""); err != nil {
+	if _, _, err := st.SaveHostEscrow("imp-host", []byte("blob"), "fp", "p", "2026-07-01T00:00:00Z", ""); err != nil {
 		t.Fatal(err)
 	}
 	if err := st.UpsertGuestFromReport(&store.Guest{GuestID: store.GuestID("imp-host", 100),
@@ -159,7 +159,7 @@ func TestHostDelete_HappyPath(t *testing.T) {
 	if err := st.UpsertHost(&store.Host{HostID: "dr-drill", CustomerID: "c5", APIKey: "k"}); err != nil {
 		t.Fatal(err)
 	}
-	if _, err := st.SaveHostEscrow("dr-drill", []byte("blob"), "fp", "p", "2026-07-01T00:00:00Z", ""); err != nil {
+	if _, _, err := st.SaveHostEscrow("dr-drill", []byte("blob"), "fp", "p", "2026-07-01T00:00:00Z", ""); err != nil {
 		t.Fatal(err)
 	}
 	rr := postHostDelete(t, s, "dr-drill", url.Values{

hub/internal/web/hosts_test.go

@@ -130,7 +130,7 @@ func TestHandleHostDetail(t *testing.T) {
 		t.Fatal(err)
 	}
 	// DR + escrow present (escrow row must exist before the DR bundle UPDATE).
-	if _, err := st.SaveHostEscrow("demo-felhom-01", []byte("opaque-escrow"), "fp", "posture", "2026-06-01T00:00:00Z", ""); err != nil {
+	if _, _, err := st.SaveHostEscrow("demo-felhom-01", []byte("opaque-escrow"), "fp", "posture", "2026-06-01T00:00:00Z", ""); err != nil {
 		t.Fatal(err)
 	}
 	if err := st.SaveHostDRBundle("demo-felhom-01", []byte("opaque-identity"), `{"v":1}`); err != nil {

hub/internal/web/pbsdr_test.go

@@ -404,7 +404,7 @@ func TestPBSDR_F14AutoReissueOnAckedDeletion(t *testing.T) {
 	if err := st.UpsertHost(&store.Host{HostID: "peti-00-dead", CustomerID: "peti", APIKey: "oldkey"}); err != nil {
 		t.Fatal(err)
 	}
-	if _, err := st.SaveHostEscrow("peti-00-dead", []byte("opaque"), "fp", "posture", "2026-07-01T00:00:00Z", ""); err != nil {
+	if _, _, err := st.SaveHostEscrow("peti-00-dead", []byte("opaque"), "fp", "posture", "2026-07-01T00:00:00Z", ""); err != nil {
 		t.Fatal(err)
 	}
 	if err := st.DeleteHost("peti-00-dead", true); err != nil {

manifests/hub.yaml

@@ -125,7 +125,7 @@ spec:
     spec:
       containers:
         - name: hub
-          image: gitea.dooplex.hu/admin/felhom-hub:0.89.0
+          image: gitea.dooplex.hu/admin/felhom-hub:0.94.0
           ports:
             - containerPort: 8080
               name: http

manifests/webpage.yaml

@@ -324,7 +324,7 @@ spec:
           image: registry.k8s.io/git-sync/git-sync:v4.4.0
           args:
             - --repo=https://gitea.dooplex.hu/admin/felhom.eu.git
-            - --ref=installer-v1.23.0
+            - --ref=installer-v1.25.0
             - --root=/git-scripts
             - --link=current
             - --period=30s
@@ -369,7 +369,7 @@ spec:
           image: registry.k8s.io/git-sync/git-sync:v4.4.0
           args:
             - --repo=https://gitea.dooplex.hu/admin/felhom.eu.git
-            - --ref=installer-v1.23.0
+            - --ref=installer-v1.25.0
             - --root=/git-scripts
             - --link=current
             - --one-time

scripts/CHANGELOG.md

@@ -1,3 +1,55 @@
+## 1.25.0 — the off-site tier stops asking to prune (2026-08-04, R-191)
+
+**A backup that worked must not report failure.** The off-site tier was written with `keep_last: 2`,
+so every weekly run uploaded its snapshot successfully and then failed the whole job on a prune the
+box's token is deliberately refused: `prune 'ct/9201': permission check failed - missing
+Datastore.Modify|Datastore.Prune` → `TASK ERROR: job errors` → `whole_guest_backup_failed` in the
+operator's inbox. Every week, on both boxes, about a backup that had already succeeded.
+
+**R-89 moved off-site pruning SERVER-SIDE** — ep0 runs a per-namespace prune job and box tokens stay
+write-only, so a box can never delete its own off-site history. The 2026-07-26 "two weeks" ruling was
+not reversed; where it is ENFORCED moved, and this value did not follow. The tier now writes
+`keep_last: 0`, which the agent's existing guard (`allowPBSPrune = !primary && keep_last > 0`) already
+reads as "never prune from the box" — no agent change needed.
+
+**VERIFIED BEFORE CHANGING IT** (read-only on ep0, 2026-08-04): prune jobs `prune-demo-felhom` and
+`prune-demo-hp` exist on datastore `felhom-offsite`, one per namespace, schedule 03:30, keep-last 2,
+and have run **every day since 2026-07-27 — 18 tasks, all `status=OK`**, the newest showing
+`retention options: --ns demo-felhom --max-depth 0 --keep-last 2` and keeping exactly two. Disabling
+the client-side prune without that check would have traded a weekly false alarm for unbounded growth.
+
+A gate now asserts the off-site tier carries no client-side prune, so the value cannot drift back
+quietly. The local tier's retention is untouched.
+
+## 1.24.0 — a pre-existing backup target is granted too (2026-08-03, R-185)
+
+**`configure_backup_target` has two arms and only one of them granted.** The Case A arm creates the
+storage and calls `felhom-backup-target-apply grant` in the same breath — a box that builds its own
+target has always been fine. The **Scenario-F arm** — *"the target already exists, leave it exactly as
+it is"* — returned without ever granting.
+
+So a box whose `felhom-backup` pre-dated the install (created by the vzdump-target-move runbook, or
+surviving a reinstall) ended up with `local_backup_target: felhom-backup` while its token held
+`FelhomAgentStore` on only `local`, `local-lvm` and `felhom-pbs`. Measured on **both** demo boxes
+2026-08-03: the content API answers `{"data":[]}` through the agent's token while root lists three
+archives. That tier was invisible to the agent and never restore-tested — and nothing said so,
+because an empty listing is also what a brand-new tier returns.
+
+The reuse arm now ensures the ACL through the same guarded wrapper, so both arms leave the box in the
+same state. **Scenario F is unviolated:** the storage DEFINITION is still untouched — granting the
+role the agent is supposed to have on the target this script is about to write into `agent.json` is
+finishing the job, not retargeting the box. `pveum acl modify` is idempotent, so a box that already
+has the grant is unchanged and a box whose token was rotated gets it back.
+
+`$BACKUP_TARGET_ID` is deliberately **not** added to `PVE_STORAGES`, and the comment now says why: that
+list is granted in step 4/5, before the target has been resolved in step 6, and `--acl-storages`
+entries are preflight-checked for existence. The grant belongs with the resolution, which is where it
+already was for a newly created target.
+
+**A gate now asserts it** (`hostinstall_gates.py`): every arm of `configure_backup_target` that
+resolves the target must also grant on it. Red-proved by reverting the reuse arm — `resolves the
+backup target in 2 place(s) but grants in only 1`.
+
 ## v1.23.0 — the installer is published, not pushed (2026-08-03, R-110 + R-183)
 
 **Two channels moved off `main` in the same change, because either one left behind makes the other

scripts/felhom-host-install.sh

@@ -184,7 +184,7 @@
 
 set -euo pipefail
 
-SCRIPT_VERSION="1.23.0"   # the SINGLE version source (F-1): -h and the run banners follow it.
+SCRIPT_VERSION="1.25.0"   # the SINGLE version source (F-1): -h and the run banners follow it.
                           # The hub used to carry a copy for its Setup tab; R-94 DELETED it
                           # (2026-08-02) because the hub cannot know which version a box runs —
                           # the Setup command fetches this script at run time. scripts/
@@ -307,6 +307,14 @@ PVE_PRIVS_STORE="Datastore.Allocate Datastore.AllocateSpace"
 PVE_PRIVS_BASE="Sys.Audit SDN.Use Datastore.Audit"
 # Storages the agent reads/writes (archive+dump=local, restore=local-lvm, offsite DR=felhom-pbs). The
 # offsite felhom-pbs MUST be included or the agent's DR backup 403s (SPIKE residual #1). --acl-storages overrides.
+#
+# `$BACKUP_TARGET_ID` is deliberately NOT in this list, and the reason matters because the obvious
+# tidy-up is to add it (R-185, 2026-08-03). This list is granted in STEP 4/5, before
+# configure_backup_target has run in STEP 6 — so at this point the target may not exist yet, and
+# --acl-storages entries are preflight-checked for existence. The target's grant therefore belongs
+# with the target's RESOLUTION, where it already is for a freshly created one, and now also for a
+# pre-existing one. Adding it here would grant on a storage that may not exist and would still leave
+# the resolution path as the single owner of that decision, split across two places.
 PVE_STORAGES=(local local-lvm felhom-pbs)
 # E-2: the whole-guest backup target storage id, and what configure_backup_target resolved to.
 # BACKUP_TARGET_RESOLVED feeds agent.json's local_backup_target; "local" means DEGRADED (Case B).
@@ -652,8 +660,31 @@ configure_backup_target() {
     # existing id is an error, and repointing a live target is exactly the silent retarget this
     # whole arc closes.
     if pvesm status --storage "$BACKUP_TARGET_ID" >/dev/null 2>&1; then
-        log_skip "  backup target '$BACKUP_TARGET_ID' already exists — leaving it exactly as it is (Scenario F)"
+        log_skip "  backup target '$BACKUP_TARGET_ID' already exists — leaving its DEFINITION exactly as it is (Scenario F)"
         BACKUP_TARGET_RESOLVED="$BACKUP_TARGET_ID"
+        # R-185: …but STILL ensure the ACL. "The storage already exists" says nothing about whether
+        # the agent may READ it, and this early return is where the two came apart.
+        #
+        # THE DEFECT THIS CLOSES, measured on both demo boxes 2026-08-03. The CASE A path below
+        # creates the storage and grants in the same breath, so a box that built its own target is
+        # fine. A box whose target ALREADY existed — created by the vzdump-target-move runbook, or
+        # surviving a reinstall — returned here and never granted. The result: `local_backup_target`
+        # pointed at `felhom-backup` while the token held FelhomAgentStore only on local, local-lvm
+        # and felhom-pbs, so the API answered `{"data":[]}` for that storage while root saw three
+        # archives. The tier was invisible to the agent and never restore-tested, and nothing said so
+        # — because an empty listing is also what a brand-new tier returns.
+        #
+        # Scenario F is UNVIOLATED: the storage definition is still untouched. Granting a role the
+        # agent is supposed to have on the target this same script is about to write into
+        # agent.json is not "touching the box's target", it is finishing the job. `pveum acl modify`
+        # is idempotent, so a box that already has the grant is unchanged, and a box that had its
+        # token rotated gets it back.
+        if $DRY_RUN; then
+            log_dry "felhom-backup-target-apply grant $BACKUP_TARGET_ID   # R-185: ACL on a pre-existing target"
+        else
+            /usr/local/sbin/felhom-backup-target-apply grant "$BACKUP_TARGET_ID" \
+                || die "backup target grant failed on the pre-existing target — the agent could not read its own backup tier (R-185)"
+        fi
         return 0
     fi
     local mp
@@ -2448,8 +2479,30 @@ for _k,_v in {"unit_dir":"/etc/systemd/system","stage_dir":"/var/lib/felhom-agen
 base.setdefault('storage', {"watchdog_interval_seconds":5,"watchdog_debounce_seconds":15,"known_refresh_seconds":20})
 # R-82: local DAILY + offsite WEEKLY. The two tiers carry SEPARATE cadences and retentions —
 # keep_last=3 is three DAYS on the daily tier and three WEEKS on a weekly one, so one shared knob
-# would guarantee that one of them is wrong. keep_last=2 on the offsite tier = two weeks (operator
-# ruling 2026-07-26).
+# would guarantee that one of them is wrong.
+#
+# THE OFFSITE TIER CARRIES NO CLIENT-SIDE RETENTION, AND THAT IS THE POINT (R-191, 2026-08-04).
+# It used to be written `keep_last: 2` ("two weeks", operator ruling 2026-07-26). **R-89 then moved
+# offsite pruning SERVER-SIDE** — ep0 runs a per-namespace prune job and box tokens stay write-only,
+# deliberately, so that a box can never delete its own offsite history. The 2026-07-26 ruling was not
+# reversed; where it is ENFORCED moved, and this value did not follow.
+#
+# The consequence was weekly and wrong in the worst direction: vzdump UPLOADED the snapshot fine and
+# then failed the whole job on the prune the token is refused —
+#   `prune 'ct/9201': permission check failed - missing Datastore.Modify|Datastore.Prune`
+# → `TASK ERROR: job errors` → `whole_guest_backup_failed` in the operator's inbox. Every week, on
+# both boxes, about a backup that had already succeeded. A tier that cries wolf weekly is a tier
+# whose real failure nobody will see.
+#
+# VERIFIED BEFORE CHANGING IT (2026-08-04, read-only on ep0): prune jobs `prune-demo-felhom` and
+# `prune-demo-hp` exist on datastore `felhom-offsite`, one per namespace, schedule 03:30, keep-last 2,
+# and have run EVERY DAY since 2026-07-27 — 18 tasks, all `status=OK`, the newest showing
+# `retention options: --ns demo-felhom --max-depth 0 --keep-last 2` and keeping exactly two. Retention
+# happens; it happens THERE. **If that ever stops being true, this zero is unbounded growth** — check
+# ep0's prune jobs before assuming the offsite tier is retained.
+#
+# `keep_last: 0` means "never prune from the box" and is the value the agent's own guard reads
+# (allowPBSPrune = !primary && keep_last > 0), so no agent change is needed to honour it.
 #
 # The offsite tier is written even though `felhom-pbs` does not exist yet: that storage appears only
 # when the hub provisions the DR tier. The agent DEFERS a tier whose target storage is absent
@@ -2460,7 +2513,7 @@ base.setdefault('storage', {"watchdog_interval_seconds":5,"watchdog_debounce_sec
 # setdefault: an EXISTING box's backup block is preserved WHOLE. Upgrades never gain the tier here —
 # they are migrated explicitly (R-82 Slice D.2), so an in-place upgrade can never silently start
 # writing to an offsite datastore.
-base.setdefault('backup', {"local_backup_target":os.environ.get('BACKUP_TARGET_RESOLVED','local'),"local_backup_retention":3,"restore_storage":"local-lvm","restore_test_cadence_seconds":0,"scratch_vmid_min":990000,"scratch_vmid_max":990009,"pbs_secret_dir":"/etc/pve/priv/storage","backup_cadence_seconds":0,"backup_targets":[{"target_id":"felhom-pbs","cadence_seconds":604800,"keep_last":2}]})
+base.setdefault('backup', {"local_backup_target":os.environ.get('BACKUP_TARGET_RESOLVED','local'),"local_backup_retention":3,"restore_storage":"local-lvm","restore_test_cadence_seconds":0,"scratch_vmid_min":990000,"scratch_vmid_max":990009,"pbs_secret_dir":"/etc/pve/priv/storage","backup_cadence_seconds":0,"backup_targets":[{"target_id":"felhom-pbs","cadence_seconds":604800,"keep_last":0}]})
 base.setdefault('local_api', {})
 base['local_api'].setdefault('enable', True)
 base['local_api']['listen_addr'] = os.environ['BRIDGE_ADDR']

scripts/hostinstall_gates.py

@@ -202,6 +202,63 @@ if wp is not None:
         fail("manifests/webpage.yaml no longer tracks main for the website — pinning the SITE to "
              "the installer tag turns every copy edit into a release")
 
+# ── R-185: every path that RESOLVES the backup target must also grant on it ──────────────────
+#
+# THE DEFECT THIS WOULD HAVE CAUGHT, measured on both demo boxes 2026-08-03. `configure_backup_target`
+# has two arms. The CASE A arm creates the storage and grants in the same breath. The Scenario-F arm —
+# "the target already exists, leave it alone" — returned WITHOUT granting, so a box whose target
+# pre-dated the install pointed `local_backup_target` at a storage its own token could not read. The
+# API answered `{"data":[]}` while root saw three archives, and nothing said so, because an empty
+# listing is also what a brand-new tier returns.
+#
+# The assertion is deliberately about the FUNCTION, not about PVE_STORAGES: the target's grant belongs
+# with the target's resolution (PVE_STORAGES is granted a step earlier, before the target exists), so
+# what must hold is that no arm of that function can resolve a target and skip the grant.
+fn = re.search(r'^configure_backup_target\(\)\s*\{(.*?)^\}', src, re.S | re.M)
+if not fn:
+    fail("cannot find configure_backup_target() — the backup-target ACL assertion cannot run, and a "
+         "check that cannot run must never report OK (R-185)")
+else:
+    body = fn.group(1)
+    resolutions = len(re.findall(r'BACKUP_TARGET_RESOLVED="\$BACKUP_TARGET_ID"', body))
+    grants = len(re.findall(r'felhom-backup-target-apply grant', body))
+    if resolutions == 0:
+        fail("configure_backup_target no longer resolves BACKUP_TARGET_ID anywhere — re-read it")
+    elif grants >= resolutions:
+        ok("every arm that resolves the backup target also grants on it (%d resolution(s), %d grant(s))"
+           % (resolutions, grants))
+    else:
+        fail("configure_backup_target resolves the backup target in %d place(s) but grants in only %d "
+             "— an arm resolves a target the agent may not READ. That is R-185: the tier's archives are "
+             "invisible to the agent, it is never restore-tested, and an empty listing looks exactly "
+             "like a brand-new tier." % (resolutions, grants))
+
+# ── R-191: the OFFSITE tier must not arm a client-side prune ─────────────────────────────────
+#
+# R-89 moved offsite pruning SERVER-SIDE — ep0 runs a per-namespace prune job and box tokens stay
+# write-only, so the box is REFUSED if it asks. When this default was `keep_last: 2` the effect was a
+# weekly lie: vzdump uploaded the snapshot, then failed the whole job on the prune, and the operator
+# was told the offsite backup had failed when it had succeeded.
+#
+# The assertion is on the OFFSITE entry only. The local tier's `local_backup_retention` is untouched
+# and must stay untouched — it prunes correctly and is allowed to.
+m = re.search(r'"backup_targets":\s*\[(.*?)\]', src, re.S)
+if not m:
+    fail("cannot find backup_targets in the rendered agent.json defaults — the offsite-retention "
+         "assertion cannot run, and a check that cannot run must never report OK (R-191)")
+else:
+    targets = m.group(1)
+    kl = re.search(r'"keep_last"\s*:\s*(\d+)', targets)
+    if not kl:
+        fail("the offsite backup_target carries no keep_last at all — expected an explicit 0 "
+             "(R-191: 0 means 'never prune from the box'; absent is not the same statement)")
+    elif kl.group(1) != "0":
+        fail("the offsite backup_target arms a CLIENT-SIDE prune (keep_last=%s). R-89 moved offsite "
+             "pruning server-side to ep0 and box tokens are write-only, so every weekly run will "
+             "upload successfully and then FAIL the job on a refused prune (R-191)." % kl.group(1))
+    else:
+        ok("the offsite tier arms no client-side prune (keep_last=0; retention is ep0's prune jobs)")
+
 print()
 if fails:
     print("hostinstall gates: %d FAILURE(S)" % len(fails))