From 7fff45d688fa9bb36dc9cce4a5057769dece7d28 Mon Sep 17 00:00:00 2001
From: kisfenyo <nagyfenyvesi.viktor@gmail.com>
Date: Tue, 4 Aug 2026 11:04:39 +0200
Subject: [PATCH] R-195: a customer with no machine ever bound does not alarm
 (hub v0.92.0) + R-193/R-192 spike
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Part 4 (ships): `david` — a prospective customer with hosts=0, host_deletions=0,
reports=0 — e-mailed an expected_dbdump_missed ERROR at 03:00 UTC three mornings
running. The existing down-skip could never cover it: it reads the staleness
checker's state, which is seeded from a query over the `reports` table, so a
customer that never reported has no state at all and GetState() returns "" rather
than "down". store.HasEverBoundHost (hosts row OR host_deletions tombstone) is
consulted once per customer at the top of the deadline loop. The discriminator is
"was a host EVER bound", never "has a report arrived" — a box installed and never
heard from is a real fault and keeps alarming. Fail-OPEN on a read error. Red-proof
observed: removing the guard fails with `got [expected_dbdump_missed]`, verbatim the
event david sent.

Parts 0-3 (spike, NO production code for R-193/R-192):
audits/SPIKE-offsite-credential-recovery-2026-08-04.md establishes that the one-shot
provider password is the RECOVERABLE secret and the restic repository password is the
irreplaceable one — and that a guest rebuild mints a fresh one, orphaning the previous
off-site history. Measured without touching a box, by comparing
host_escrow.restic_pw_sha256 against host_escrow_superseded: BOTH demo boxes changed
(demo-hp 15 snapshots / 40.9 MB, demo-felhom 36 snapshots / 1.14 GB). demo-felhom's
"lucky" 76-second recovery restored delivery and not the repository, silently, for 13h.
ReissueCredentials does NOT rotate the restic password (R-39's record and two hub
comments are wrong -> R-196); candidate (b) is not implementable against a
zero-knowledge escrow; candidate (a) already exists as F3 and is wired to the wrong
event. Ends in ranked options and an unanswered question for the operator.

R-195 SHIPPED; R-196 + R-197 filed; R-192 + R-193 updated, neither closed.
---
 CONTEXT.md                                    |  58 ++
 REUSE.md                                      |   1 +
 .../architecture/00-capability-map.md         |   2 +-
 ...-offsite-credential-recovery-2026-08-04.md | 501 ++++++++++++++++++
 documentation/backlog/OPEN-ITEMS.md           |   7 +-
 hub/CHANGELOG.md                              |  45 ++
 hub/internal/monitor/deadline.go              |  37 +-
 hub/internal/monitor/deadline_unbound_test.go | 148 ++++++
 hub/internal/store/store.go                   |  22 +
 9 files changed, 815 insertions(+), 6 deletions(-)
 create mode 100644 documentation/audits/SPIKE-offsite-credential-recovery-2026-08-04.md
 create mode 100644 hub/internal/monitor/deadline_unbound_test.go

diff --git a/CONTEXT.md b/CONTEXT.md
index 07a4fd1..ed6653f 100644
--- a/CONTEXT.md
+++ b/CONTEXT.md
@@ -17,6 +17,64 @@
 
 ## Standing rulings
 
+**S-27 — a customer with NO machine ever bound is UNKNOWN, silently; one that was bound and went quiet
+still alarms (2026-08-04, R-195; hub v0.92.0).** Operator ruling, implemented as
+`store.HasEverBoundHost` (live `hosts` row OR `host_deletions` tombstone) consulted once at the top of
+`CheckBackupDeadlines`' loop. **The discriminator is "was a host EVER bound", never "has a report
+arrived"** — a box that was installed and never phoned home is a real fault. Fail-**open** on a read
+error; the deferral is logged with its own counter; the R-81 anchored-verdict structure is untouched.
+
+*The mechanism is worth carrying, because the guard that should have covered this was keyed off the
+wrong thing:* the existing down-skip reads `StalenessChecker.GetState()`, whose map is seeded from
+`store.GetCustomers()` — **a query over the `reports` table**. A customer with zero reports is in no
+row, gets no state, and `GetState()` returns `""` not `"down"`, so **the skip misses exactly the
+customer it would most obviously cover.** `david` (created 2026-08-01, no machine) e-mailed an
+`expected_dbdump_missed` ERROR three mornings running; `peti-felhom` — active, host deleted
+2026-07-15 — does not, because its 482 old reports make it `down`. Generalise it: **a "skip the dead"
+guard built on evidence of life cannot see something that was never alive.**
+
+**S-26 — the one-shot secret is the recoverable one; the irreplaceable one is minted fresh on every
+guest rebuild (2026-08-04, R-193 spike — `audits/SPIKE-offsite-credential-recovery-2026-08-04.md`).
+No code shipped for it; the decision is the operator's.**
+
+Two sentences, because they are what every future session needs before touching this area:
+
+- **Q1 — what is one-shot.** Of the three secrets in the offsite path, the hub's **one-time
+  provider password** is one-shot but freely re-issuable, the box's **SFTP key** is regenerated on
+  every apply by design, and the **restic repository password** — the DATA key, which the agent's own
+  source calls *"irreplaceable"* (`felhom-agent/internal/escrow/identity.go:35-39`) — is the only one
+  nothing can restage.
+- **Q2 — what a rebuild costs.** A rebuilt controller **mints a brand-new repository password**
+  (`WriteOffboxSecrets`, `offbox.go:392`, generates whenever `<DataDir>/offbox/repo_password` is
+  absent) and **no automatic path ever consults the escrowed one** — `InjectOffboxPassword` has exactly
+  one caller in the whole repo, a web form a human pastes into — so **every guest rebuild orphans the
+  previous off-site history.**
+
+*Measured without touching a box, and the method is reusable:* `host_escrow.restic_pw_sha256` vs
+`host_escrow_superseded.restic_pw_sha256`. demo-hp `8e03eddf…`→`8a9e33aa…` (15 snapshots / 40.9 MB
+orphaned); demo-felhom `48741892…`→`c60c8bc7…` (**36 snapshots / 1.14 GB**). **demo-felhom is the half
+that matters:** R-193 recorded it as having survived the rebuild by luck, and it did — its *delivery*
+recovered in 76 s off a stale staged secret. **Its repository did not**, and nothing said so for 13 h.
+*Luck restored the plumbing, not the data.*
+
+*Three corrections that must not be re-inherited.* **(a)** `ReissueCredentials` does **not** rotate the
+restic password — R-39's record, `offsite.go:198-201` and `api/handler.go:1067-1069` all say it does,
+and all three are wrong (→ **R-196**, the eighth entry in `CLAUDE.md`'s table). **(b)** Candidate (b),
+*"recoverable from escrow at re-bootstrap"*, is **not implementable** — the escrow is R-wrapped and
+zero-knowledge (D6), so only a customer-present ceremony can open it, which is the manual form that
+already exists. **(c)** Candidate (a), *"the hub restages automatically"*, **already exists** as
+`reissueOnReenroll`'s F3 leg and is wired to the wrong event — it sits behind `handleHostEnroll`'s
+mint-once-reuse short-circuit, and a **guest** rebuild leaves the `hosts` row intact. **Shipping it
+first would have made both boxes look healthy on 2026-08-04 while their snapshots were orphaned** —
+strictly worse than the current loud failure.
+
+*The candidate nobody had named,* and the only one aimed at the actual harm: **the agent survives a
+guest rebuild**, already receives the repo password over the pinned local API
+(`POST /escrow/stage-secret`) and already writes it to a fixed 0600 path — it merely **wipes** it after
+the ceremony. Retaining and serving it back needs no new seam. Its price is one real trade: a copy of
+the data key at rest on the Proxmox host. **That trade is the operator's to make and the spike does not
+make it.**
+
 **S-24 — offsite retention is ep0's, and the box asks for none (2026-08-04, R-191; installer 1.25.0).**
 R-89 moved offsite pruning server-side and box tokens stay write-only. The 2026-07-26 "two weeks"
 ruling was not reversed — **where it is ENFORCED moved, and the installer's `keep_last: 2` did not
diff --git a/REUSE.md b/REUSE.md
index a7c53bb..d37ae96 100644
--- a/REUSE.md
+++ b/REUSE.md
@@ -75,6 +75,7 @@
 | `(*Store).GetHostRecoveryMeta` + `(*Server).handleHostRevealRecoveryCredential` | hub/internal/store/host_recovery.go · hub/internal/web/hosts.go | `(hostID) (*HostRecoveryMeta, error)` · `POST /hosts/{id}/reveal-recovery-credential` | The break-glass console credential, split into a RENDER half and a RETRIEVE half (v0.84.0) | **Use `GetHostRecoveryMeta` on any page-render path** — its struct and its `SELECT` both omit the `secret` column, so it cannot leak one; `GetHostRecoveryCredential` (which does select it) belongs only to the two retrieval handlers. The reveal is POST so the ServeHTTP-level CSRF check applies and no secret is reachable by URL; it writes ONE `recovery_credential_revealed` event via `SaveEvent` and calls NO dispatcher (the `handleRequestLogTail` shape). `api/handler.go handleAdminGetRecoveryCredential` (global key) is the independent fallback for when the UI is down — never route the UI through it. Secret at rest is plaintext → R-133. |
 | `host_detail_body` sub-template | hub/internal/web/templates/host_detail_body.html | `{{template "host_detail_body" .}}` | Rendering a host's detail sections on ANY surface | One namespace across ParseFS (icons.html pattern). Renders per-host — id-suffix any new element ids with `{{.HostID}}` (the customer page renders N instances). |
 | `(*Store).ListHostsByCustomer` | hub/internal/store/store.go (~L1620) | `(customerID) ([]Host, error)` | A customer's hosts, host_id order | A LIST by design (HA-cluster roadmap) — don't collapse to GetHostByCustomer. |
+| `(*Store).HasEverBoundHost` (v0.92.0, R-195) | hub/internal/store/store.go | `(customerID) (bool, error)` | Any verdict that must not fire for a customer with **no machine ever bound** — "was anything ever expected of this customer" | `hosts` row **OR** `host_deletions` tombstone. **NOT a liveness check and never a substitute for one:** a box that was bound and went silent returns `true` and must keep alarming — that is the case any change here breaks first (pinned by `TestCheckBackupDeadlines_BoundButNeverReported_StillAlarms`). Callers **fail OPEN** on its error: an unreadable binding must never SUPPRESS an alarm. Do **not** re-derive this from report presence — `store.GetCustomers()` (and therefore the staleness checker's `down` state) is a query over `reports`, so a never-reported customer has no state at all, which is exactly how the daily false alarm reached `david`. |
 | `(*Server).configFormData` (v0.49.0) | hub/internal/web/configs.go (~L430) | `(r, isNew, cfg, overrides, errMsg) configFormView` | The ONE view-model builder for the customer config form (standalone chrome + the customer page Edit tab) | `overrides=nil` → parses the STORED cfg.ConfigJSON; pass the SUBMITTED map on the update validation-error re-render or typed values reset (red-proofed). |
 | `config_form_body` sub-template (v0.49.0) | hub/internal/web/templates/config_form_body.html | `{{template "config_form_body" <configFormView>}}` | Rendering the config form on ANY surface (config_form.html chrome + customer Edit tab) | The floor/geo/danger cards on the Edit tab are SIBLINGS after `</form>` — never nest a form inside it (breaks the offsite/PBS formaction sub-buttons). Includes the F5 in-flight `<script>`. |
 | `(*Store).CountHostArtifacts` / `DeleteHost` | hub/internal/store/store.go (~L1640/~L1690) | `(hostID) (HostArtifacts, error)` / `(hostID, deleteEscrow bool) error` | Host-delete impact preview + the ONE-transaction cascade | ONLINE gate lives in the handler, escrow gate in the store (`ErrHostEscrowPresent`, tx never starts). log_bundles die by `scope_id == host_id` ONLY (customer-scoped bundles survive). The wg_peers delete is INSIDE the tx — never split it out. |
diff --git a/documentation/architecture/00-capability-map.md b/documentation/architecture/00-capability-map.md
index eed52b3..109d297 100644
--- a/documentation/architecture/00-capability-map.md
+++ b/documentation/architecture/00-capability-map.md
@@ -76,7 +76,7 @@
 | Whole-guest backup lands OFF the guest's own physical device; a single-drive box is recorded **DEGRADED** rather than silently normal (installer Case A/B) | agent v0.113, host-install v1.22.0 | **PROVEN-LIVE** | `E2D-fresh-vm-2026-07-29` C1 (real 1.22.0 install, rc=0, `Day-0 provision SUCCESS`) + C2 (both DEGRADED lines verbatim, `local_backup_target=local`, install did not abort) | Case A (a second drive already present at install) has never fired naturally — only Case B has |
 | Nightly DB dumps (postgres/mariadb autodiscovery), atomic writes | controller v0.118 | **PROVEN-LIVE** | `CAMPAIGN-2` T-BAK-FULL (pg+mariadb autodiscovered); atomicity `CAMPAIGN-6B` P4 + `CAMPAIGN-6E` B1/B2 (SIGKILL mid-write → only `.tar.tmp` touched, last-good byte-unchanged); DB restore `CAMPAIGN-6D` P-FAB | (Cited `CAMPAIGN-3` F7 is the *finding* of non-atomic writes, and T-RST-DB was auth-hollow — corrected to the 6B/6E fix-proofs.) T-6E-1 dir-fsync asymmetry (LOW) → R-10 **DB replay route → `07-backup-architecture.md` §8 row 3** |
 | Tier-2 secondary-drive copy: class-driven legs, v2 relpath layout, NAS-target exclusion, safe-remove boundary | controller v0.135 | **PROVEN-LIVE** | `CAMPAIGN-6E-2026-07-15` (P-TIER2 deep-4 PASS), `CAMPAIGN-6C` | **Route + RTO → `07-backup-architecture.md` §8 rows 1, 2, 4, 5.** The matrix records that the copy's `recovery-unit/` mirror is read by no path (→ R-102) |
-| Offsite (restic → Hetzner Storage Box): mandatory class only, raw-data quota, enlargement gate, retention regrouping | controller v0.134, agent, hub | **PROVEN-LIVE** | `CAMPAIGN-6D-2026-07-15` (mandatory-only P-IMMICH; enlargement gate fired at real 50GiB quota P3-DELIVERY); `VALIDATION-offbox-storagebox-2026-07-09` (byte-perfect round-trip) | Raw-data quota (SP-1) + retention regrouping (SP-2) are `SPIKE-restic-snapshot-shape` **dry-run** verdicts — mechanism validated, not fired in a live product run; only the enlargement gate is live-fired. **Reinstall-continuity (controller v0.142.0, 2026-07-17):** a recreated data volume that orphaned the repo (new passphrase can't open the old keys) is now CLASSIFIED (`wrong password or no key found`) → explicit ORPHANED card + event (not nightly-spam) + a move-aside (never-delete) reset (unclaimed auto / claimed confirm), instead of a raw nightly restic error. Fake-based scenarios + red-proofs. **The live leg FIRED on its own during the 2026-07-18 rehearsal** (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`, S7): after a RESET + re-enable, the first offsite run hit the previous lifecycle's ciphertext and the guard **classified it, pushed `offbox_repo_orphaned`, skipped the run and showed the card (16:58:14)** rather than nightly-spamming a raw restic error; the operator-confirmed reset then **moved the repo aside (never deleted) to `.orphaned-20260718` and re-initialised (16:59:26→16:59:32)**, and the next run produced 2 snapshots / 48.717 MiB. The guard behaved exactly as designed — **the finding is that it had to fire at all** (R-32: RESET destroys custody, so the ciphertext it leaves behind is dead by design and should be purged, while the move-aside guard stays correct for reinstall-WITHOUT-RESET). `DIAGNOSE-offbox-repo-orphaned-2026-07-17` **Route + RTO → `07-backup-architecture.md` §8 rows 4, 10, 12, 15** (incl. the R-95 delete exposure and the R-104 stale-lock defect) |
+| Offsite (restic → Hetzner Storage Box): mandatory class only, raw-data quota, enlargement gate, retention regrouping | controller v0.134, agent, hub | **PROVEN-LIVE** | `CAMPAIGN-6D-2026-07-15` (mandatory-only P-IMMICH; enlargement gate fired at real 50GiB quota P3-DELIVERY); `VALIDATION-offbox-storagebox-2026-07-09` (byte-perfect round-trip) | Raw-data quota (SP-1) + retention regrouping (SP-2) are `SPIKE-restic-snapshot-shape` **dry-run** verdicts — mechanism validated, not fired in a live product run; only the enlargement gate is live-fired. **Reinstall-continuity (controller v0.142.0, 2026-07-17):** a recreated data volume that orphaned the repo (new passphrase can't open the old keys) is now CLASSIFIED (`wrong password or no key found`) → explicit ORPHANED card + event (not nightly-spam) + a move-aside (never-delete) reset (unclaimed auto / claimed confirm), instead of a raw nightly restic error. Fake-based scenarios + red-proofs. **The live leg FIRED on its own during the 2026-07-18 rehearsal** (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`, S7): after a RESET + re-enable, the first offsite run hit the previous lifecycle's ciphertext and the guard **classified it, pushed `offbox_repo_orphaned`, skipped the run and showed the card (16:58:14)** rather than nightly-spamming a raw restic error; the operator-confirmed reset then **moved the repo aside (never deleted) to `.orphaned-20260718` and re-initialised (16:59:26→16:59:32)**, and the next run produced 2 snapshots / 48.717 MiB. The guard behaved exactly as designed — **the finding is that it had to fire at all** (R-32: RESET destroys custody, so the ciphertext it leaves behind is dead by design and should be purged, while the move-aside guard stays correct for reinstall-WITHOUT-RESET). `DIAGNOSE-offbox-repo-orphaned-2026-07-17` **Route + RTO → `07-backup-architecture.md` §8 rows 4, 10, 12, 15** (incl. the R-95 delete exposure and the R-104 stale-lock defect). **2026-08-04 (R-193/R-197, `audits/SPIKE-offsite-credential-recovery-2026-08-04.md`) — the row is NOT overclaiming and the guard is not the gap; the CADENCE is.** This row already recorded that a recreated data volume orphans the repo, and the spike confirms the mechanism at source: `WriteOffboxSecrets` (`offbox.go:392`) mints a fresh 256-bit repo password whenever `<DataDir>/offbox/repo_password` is absent, and **no automatic path ever consults the escrowed one** — `InjectOffboxPassword` has exactly one caller, a web form a human pastes into. What was NOT recorded is that this now fires on an **ordinary, planned, unattended guest rebuild**, on every box: measured on BOTH demo boxes 2026-08-03/04 by comparing `host_escrow.restic_pw_sha256` against `host_escrow_superseded.restic_pw_sha256` (demo-hp `8e03eddf…`→`8a9e33aa…`, demo-felhom `48741892…`→`c60c8bc7…`), orphaning **15 snapshots / 40.9 MB** and **36 snapshots / 1.14 GB** respectively. **demo-felhom is the important half:** it kept its DELIVERY (a stale staged secret restored the target in 76 s) and lost its REPOSITORY anyway, with **nothing marking the escrow stale for 13 h** — `escrow_stale` is wired to `ReissueCredentials`, the one path that does NOT change the repo password (**R-196**), and absent from the rebuild path that does. Status unchanged: the classify-and-move-aside guard remains PROVEN-LIVE and correct, and is predicted (not yet measured) to refuse the 2026-08-05 run on both boxes rather than start a silent fresh history |
 | Offsite restore: local-preferred scratch, unit-only default, full two-step, missing-only place-to-live | controller v0.134/134.1/135 | **PROVEN-LIVE** (2026-07-20) | `CAMPAIGN-6D` accept legs (immich end-to-end from offsite alone) | **2026-07-19:** `audits/DIAG-immich-restore-2026-07-19.md` finds **no offsite path loads a DB dump** — all three buttons are file-only (R-43). The mechanics in this row's title are each proven; the phrase "**immich end-to-end from offsite alone**" is what is contested, since a DB-indexed app cannot be reconstituted by any offsite action. **RULED 2026-07-19 (Viktor): 6D's destruction hit the FILE TREE ONLY — the database survived in its named volume** (`immich_postgres_data` is a named volume in both the v2 and v3 template eras), so "immich end-to-end from offsite alone" **overclaimed scope**: the file half was proven, the DB half was never destroyed and therefore never restored. Row downgraded PROVEN-LIVE → **PARTIAL**, scope-corrected. Evidence: `audits/DIAG-immich-restore-2026-07-19.md` (no offsite path could replay a DB at all) + the P-FAB destructive re-import (the proven-replay evidence, on the LOCAL path). **2026-07-19, controller v0.148.0:** the DB half now exists in code (R-43 + R-44) and its replay reached a live box — but **round 2 found it aborts against a running app** (`audits/DIAG-immich-restore-round2-2026-07-19.md`, H4: the replay races immich's own schema repair; `clip_index` recreated by the app 2 s before the dump's CREATE INDEX). **2026-07-20, controller v0.153.0: H4 IS CLOSED (R-47)** — both restore paths now replay into a DB-ONLY window (`StartStackServices` brings up the database service alone; the app starts only after the replay exits 0), with a fail-closed refusal when a dump has no identifiable DB service. *(The earlier note here said "closes in v0.149" — that was wrong: v0.149.0 was the F3 dashboard `BackupStatus` fix. R-47 shipped in v0.153.0.)* **2026-07-20: the clean run HAPPENED** — endpoint-level supervised reconstitute of immich from snapshot `49e7cb46` (the very snapshot that aborted in round 2): stop → DB-service-only start → replay rc-0 → full start, no `already exists`, operation reported SUCCESS, immich's own DatabaseService logged `No schema drift detected` twice, 11 assets `active`, 4/4 containers healthy, 231 `public` indexes. **Operator confirmed the immich timeline renders correctly after the reconstitute** (screenshot held, 2026-07-20). Evidence: `felhom-controller/REPORT.md` §4b. **2026-07-20, LATER THE SAME DAY — the destructive drill RAN and the row now earns PROVEN-LIVE.** The operator deleted the photos in immich own UI **and emptied the trash** (the step whose absence makes a drill prove nothing — the round-1 lesson), then restored through the customer-facing UI. **`40 file(s) placed`** against the 6 of the earlier non-destructive run — the files were really gone and really came back — plus 1 DB dump replayed rc-0, 11 assets `active`, `No schema drift detected`, timeline confirmed by the operator. This is the destroy-then-recover proof the 6D downgrade asked for, and it was taken through the customer own buttons, not endpoint shortcuts. Evidence: `felhom-controller/REPORT.md` 4e **Route + RTO → `07-backup-architecture.md` §8 rows 3, 4** — the matrix also records that no offsite action unpacks the named-volume tars it captures (→ R-107) |
 | Manual `.fab` export/import: class-scoped capture, browser up/download, tunnel-proof chunking | controller v0.125/128/130/136 | **PROVEN-LIVE** | `CAMPAIGN-6D` P-FAB / Accept #1 (1.7 GB full circle, byte-identical, app boots); chunking `CAMPAIGN-6B` P2 (100 MiB via real CF edge, 120 MiB→413) | Chunking proven at the real CF edge via `curl --resolve`; the **rendered browser file-picker** upload leg is still Viktor's open full-circle test (6C ran it NOT-RUN). C6B-F1 was the 6B *finding*; fix verified in 6D |
 | Guest-loss DR: PBS restore with full-fidelity layout from archive, restore-test verification | agent v0.75/0.76, PBS | **PROVEN-LIVE** | `CAMPAIGN-2` T-P9-DESTROY-RESTORE (whole-guest `pct restore` of 9201 → running+healthy) + T-PBS-VERIFY (`verify_state: ok`, 13 snapshots); `DRILL-GL6-2026-07-08` Phase 0d (restore-test `mount_parity: ok`) | (Cited `VALIDATION-newbox-restore` is offbox **restic** file-restore, wrong tier — corrected.) Real **offsite** guest-loss round-trip still R1-blocked → S5 DR drill **Route + RTO → `07-backup-architecture.md` §8 rows 6, 8, 9** — measured 84–112 s local / 1101 s PBS into a scratch guest; a restore to a DIFFERENT host is unmeasured |
diff --git a/documentation/audits/SPIKE-offsite-credential-recovery-2026-08-04.md b/documentation/audits/SPIKE-offsite-credential-recovery-2026-08-04.md
new file mode 100644
index 0000000..8a81fc4
--- /dev/null
+++ b/documentation/audits/SPIKE-offsite-credential-recovery-2026-08-04.md
@@ -0,0 +1,501 @@
+# SPIKE — what is actually one-shot, and what a guest rebuild really costs
+
+**Date:** 2026-08-04 · **Items:** R-193 (rebuild drops offsite), R-192 (the alert says the opposite of
+what it measured) · **Class:** spike — **no production code shipped for R-193 or R-192**
+**Baselines read:** `felhom.eu` @ `f456835bbcc4` (hub v0.91.1) · `felhom-controller` @ `0887fd67` ·
+`felhom-agent` @ `856a127`
+
+> **This document ends in ranked options and a STOP.** It establishes facts; it does not choose.
+> The operator's question is stated, unanswered, at the end.
+
+---
+
+## 0. Why this exists
+
+Two candidate fixes were named when R-193 was filed, and neither could be weighed, because two prior
+session reports described the same mechanism in contradictory terms:
+
+- R-39's record: Re-issue *"rotates the restic password and makes the escrow stale"*.
+- R-193's own resolution note: the one-time password is *"only the transport credential used once to
+  install the box's own SSH key"*.
+
+Those two worlds imply very different fates for the fifteen existing snapshots. This spike settles it
+**from source first, live state second**, so a measurement can contradict the code rather than be
+interpreted by it.
+
+**The headline, before the detail:** *both prior claims are wrong in the way that matters.* Re-issue
+does **not** touch the restic repository password — but a **guest rebuild does**, silently, on every
+box, with nothing marking the escrow stale and nothing telling anyone. Both demo boxes lost repository
+continuity on 2026-08-03/04. The one that "recovered by luck" lost it too.
+
+---
+
+## 1. The instruments, and what each one can and cannot answer
+
+Stated up front so a wrong instrument is visible here rather than inside a conclusion (§8 rule 2).
+
+| # | Instrument | Exact query / read | Known limit |
+|---|---|---|---|
+| I1 | hub SQLite snapshot | `kubectl exec deploy/hub -- cat /data/{hub.db,hub.db-wal,hub.db-shm}` → local `sqlite3` | **The `-wal` is load-bearing**: `hub.db` alone was 2 h 15 m stale at copy time (mtime 08:34 vs wal 10:45 CEST). Copying only the main file would have produced confident answers from a stale file — the exact R-3 shape. |
+| I1-check | freshness proof | `PRAGMA integrity_check` → `ok`; `MAX(host_reports.received_at)` = `2026-08-04 08:45:38`, `datetime('now')` = `08:49:46` | **Positive observable**: newest row 4 min old. Not "the query returned no error". |
+| I2 | `host_escrow.restic_pw_sha256` + `host_escrow_superseded.restic_pw_sha256` | see §2 | sha256 of a 256-bit random secret — non-reversible, safe to compare. Answers *"did the repo password change"* **without touching a box or a credential**. |
+| I3 | `reports.report_json → $.offsite` | grouped distinct-object query with first/last seen | The report echoes the box's own target state; absence of the key = no target configured. |
+| I4 | `one_time_secrets` | `SELECT customer_id, created_at, consumed_at` | **`customer_id` is the PRIMARY KEY** — one row per customer, last-write-wins. It **cannot** answer "how many credentials were delivered"; a Re-issue destroys the prior row. demo-hp's 2026-07-23 consume is no longer in this table. |
+| I5 | source | `hub/internal/offsite/*`, `hub/internal/api/*`, `controller/internal/{backup,offsiteapply}`, `agent/internal/escrow` | — |
+
+**Not used, deliberately:** no `ReissueCredentials`, no ceremony, no config change, no deletion, no
+write to any box. Nothing on either box or on the storage endpoint was altered by this session.
+
+---
+
+## Q2 — Does a rebuilt controller reuse the escrowed repository password, or mint a new one?
+
+### **RULING: it MINTS A NEW ONE. The escrowed password is never consulted on any automatic path.**
+
+**Method — source.** The apply-bridge's terminal step is
+`Enabler.ConfigureOffbox(...)` (`controller/internal/offsiteapply/offsiteapply.go:221,256`), which
+reaches `Manager.ApplyOffsiteTarget` → `WriteOffboxSecrets`
+(`controller/internal/backup/offbox.go:370`). Quoted verbatim, `offbox.go:392-401`:
+
+```go
+// Auto-generate the repo password once (0600), never log it.
+if _, err := os.Stat(m.offboxPwPath()); os.IsNotExist(err) {
+    pw, gerr := generateOffboxPassword()
+    ...
+    if werr := os.WriteFile(m.offboxPwPath(), []byte(pw), 0o600); werr != nil {
+```
+
+`offboxPwPath()` is `<DataDir>/offbox/repo_password` (`offbox.go:363`). A rebuilt guest has a fresh
+data dir, so the file is absent, so **a fresh 256-bit password is minted**.
+
+**The only path that recovers the old one is `InjectOffboxPassword`** (`offbox.go:541`), whose sole
+caller in the entire repo is `offboxInjectPasswordHandler`
+(`controller/internal/web/offbox_handlers.go:189`) — a **web form where a human pastes the recovered
+password**. Verified by census: `grep -rn "InjectOffboxPassword" --include=*.go .` → three hits, one
+definition, one caller, one comment. **The apply-bridge never calls it.**
+
+**Method — live, and it is decisive without touching a box (I2).** The hub already stores the sha256
+of the escrowed repo password, so the question is answerable as a hash comparison:
+
+| host | superseded (pre-rebuild) | current (post-rebuild) | verdict |
+|---|---|---|---|
+| `demo-hp-bb76ea` | `8e03eddf9ff7…` created `2026-07-23T10:01:17Z` | `8a9e33aa4da6…` created `2026-08-04T07:15:36Z` | **CHANGED** |
+| `demo-felhom-8363b5` | `48741892f0ef…` created `2026-07-21T08:38:33Z` | `c60c8bc737a6…` created `2026-08-04T07:20:07Z` | **CHANGED** |
+
+**Both boxes minted a new repository password.** demo-hp's pre-rebuild escrow (`8e03…`, sealed eight
+minutes after its 2026-07-23 09:53:41 apply) is the key to its 15 snapshots. That key now exists
+**only inside a superseded, R-wrapped escrow blob** that the hub cannot open.
+
+### **This is bigger than R-193 as filed, and it hits the box R-193 called lucky.**
+
+R-193 records demo-felhom as having *"survived the SAME rebuild by luck"* — 76 seconds of downtime.
+Measured (I3), demo-felhom's pre-rebuild offsite object was:
+
+```
+last_run 2026-08-03T02:17:44Z · last_status ok · snapshot_count 36 · repo_size_bytes 1 136 685 919
+```
+
+**36 snapshots, 1.14 GB.** Since 2026-08-03 07:19:10 UTC it has reported `snapshot_count: 0,
+repo_size_bytes: 0` in every one of 109 reports, and its repo password hash changed.
+
+> **The luck recovered DELIVERY. It did not recover the REPOSITORY.** The staged secret restored the
+> transport in 76 seconds and the box then minted a brand-new repo password anyway. The contrast
+> R-193 draws between the two boxes is real for the *credential*, and **false for the data**: both
+> boxes lost repository continuity, one loudly and one silently. The silent one is worse.
+
+---
+
+## Q3 — What did the 04:15 run report?
+
+### **RULING: UNMEASURED — the decisive run has not happened yet. And the binary the question offers is the wrong one; source says the answer will be NEITHER 15 nor 1.**
+
+**Why it has not happened.** The scheduled off-box run fires at ~02:15 UTC (04:15 CEST). Measured
+from I3 — `last_run` values `2026-08-02T02:15:52Z`, `2026-08-03T02:16:39Z` on demo-hp;
+`2026-08-01T02:17:54Z`, `2026-08-02T02:17:01Z`, `2026-08-03T02:17:44Z` on demo-felhom. **Daily, both
+boxes.** Then:
+
+| box | why 2026-08-04 02:15 UTC produced nothing |
+|---|---|
+| demo-hp | no offsite target existed at all — the `offsite` key was **ABSENT** from every report between `2026-08-03 06:12:19` and `2026-08-04 07:12:01` UTC (111 reports). The re-issue landed at 07:15:47, five hours after the window. |
+| demo-felhom | the target existed but `escrow_state: pending` from `2026-08-03 07:19:10` to `2026-08-04 07:20:13`. `OffboxRunnable` requires configured **AND** escrowed (`offbox.go:564-569`), so the fork-4 gate blocked the run. The ceremony landed at 07:20:28. |
+
+Both boxes became runnable only this morning. **The decisive run is 2026-08-05 ~02:15 UTC on both.**
+Reported as unmeasured rather than inferred.
+
+### What source says will happen — and it is a third outcome
+
+Same sub-account, same repo path, new password. Measured coordinates (I3, `$.dr_recipe`):
+`u629488-sub3.your-storagebox.de:/home/felhom-repo` — **unchanged** across the whole incident, and
+`repoPath` is a compile-time constant (`hub/internal/offsite/offsite.go:85`). So the next run opens an
+**existing repo with the wrong key**, which the codebase already has a name for
+(`controller/internal/backup/offbox.go:67-93`):
+
+```go
+// ErrOffboxOrphaned is the sentinel returned when the offsite repo exists but is keyed under a
+// passphrase this controller no longer has (the reinstall shape) ...
+case strings.Contains(s, "wrong password or no key found"):
+    return "orphaned"
+```
+
+and `ensureOffboxRepo` (`offbox.go:663-679`) branches on claim state:
+
+```go
+if !m.settings.GetClaimed() {  // UNCLAIMED → auto move-aside + re-init
+...
+m.markOrphaned()
+return ErrOffboxOrphaned      // CLAIMED → skip the run, show the orphan card
+```
+
+Both boxes report `claimed: 1` (I3, `$.claimed`, latest report each). **So the predicted outcome is:
+the run REFUSES with `ErrOffboxOrphaned`, produces no snapshot at all, and waits for the customer to
+confirm a reset.** Not 15 (reattached) and not 1 (silently fresh) — a **third** outcome the question's
+framing did not contain. That is the good news half: the system is not going to quietly start a new
+history over the old one. It will stop and say so.
+
+**Consequence if the customer confirms the reset:** `resetOrphanedRepo` (`offbox.go:270-314`) moves the
+old repo aside and re-inits — *"move-aside, not deleted"*. The 40.9 MB + 1.14 GB of old ciphertext
+survives, unreadable without the superseded escrow, **and keeps consuming the 50 GB soft quota
+indefinitely.** Nothing prunes a moved-aside restic repo (S-24 covers ep0's PBS namespaces, a
+different tier).
+
+**To measure it tomorrow:** re-read `$.offsite.last_status` / `last_error` on both boxes after
+02:20 UTC, and `$.offsite.repo_state` for the orphan flag. **Predicted `ErrOffboxOrphaned`, not a
+snapshot count.** Whoever picks this up should record which of the three actually occurred — a
+prediction from source is not a measurement.
+
+---
+
+## Q1 — What is one-shot, and what is not?
+
+Three secrets, and they have almost nothing in common.
+
+| | **① storage-provider password** | **② the box's SFTP key** | **③ restic repository password** |
+|---|---|---|---|
+| **What it is** | the Hetzner sub-account / box password | ed25519 keypair for `sftp` transport | the **data** key — restic's repo encryption |
+| **Generated by** | **hub** — `genPassword()` (`offsite.go:481`) | **controller** — `KeyGen.Generate()` (`offsiteapply.go:234`) | **controller** — `generateOffboxPassword()`, 32 random bytes hex (`offbox.go:406`) |
+| **Stored where** | `one_time_secrets` (hub, plaintext, **one row per customer**) | `<DataDir>/offbox/ssh_key` 0600 (box only) | `<DataDir>/offbox/repo_password` 0600 (box) + inside the R-wrapped escrow blob (hub, opaque) |
+| **Delivered how** | `GET …/offsite/consume-password`, **served exactly once** then marked consumed (`api/offsite.go:9-31`) | never delivered — installed onto the provider by ssh-copy-id using ① | never delivered anywhere; it never leaves the box except into the escrow ceremony |
+| **Re-issuable?** | **YES** — `ReissueCredentials` resets it at the provider, any time, operator-initiated | **YES** — regenerated on every full apply; `dr_recipe.go:45` states it plainly: *"the SFTP access key is regenerated at DR"* | **NO automatic path.** Recoverable only by unsealing the escrow with the customer's recovery code and pasting it into `offboxInjectPasswordHandler` |
+| **What a guest rebuild does to it** | nothing (it lives on the hub / at the provider) | destroys it; a fresh one is minted and installed — **harmless** | **destroys it; a fresh one is minted — CATASTROPHIC for the existing repo** |
+| **Escrowed?** | no | no | **yes** — `IdentityBundle.ResticRepoPassword` |
+
+The agent's own source names the asymmetry, and it is the sentence the whole spike turns on
+(`felhom-agent/internal/escrow/identity.go:35-39`):
+
+> `ResticRepoPassword` … **It is the DATA key for the offsite tier — irreplaceable (unlike the SFTP
+> access key, which is regenerable at DR).**
+
+**The one-shot thing (①) is the recoverable one. The irreplaceable thing (③) is the one nothing
+re-stages.** R-193 was filed against ①. The damage is in ③.
+
+**Instrument caveat, worth carrying (I4):** `one_time_secrets` has `customer_id` as PRIMARY KEY, and
+`SaveOneTimeSecret` is last-write-wins **by design** (the R-39(a) guard comment in
+`monitor/offsite_delivery.go:26-30` depends on it). So the delivery ledger holds only the newest
+credential per customer: demo-hp's 2026-07-23 consume, on which R-192's whole diagnosis rests, was
+**overwritten** by the 2026-08-04 Re-issue and is no longer readable from that table.
+
+---
+
+## Q4 — Which claim about Re-issue is true?
+
+### **RULING: `ReissueCredentials` touches ONLY the provider credential (①). It does NOT touch the restic repository password. R-39's record is wrong — and the hub's own comments repeat the same wrong claim in three places.**
+
+**Method — source, quoted rather than summarised.** `ReissueCredentials`
+(`hub/internal/offsite/offsite.go:150-228`) does exactly four things:
+
+1. `genPassword()`;
+2. `ResetSubaccountPassword` / `ResetBoxPassword` + `WaitAction` — **the provider account password**;
+3. `Store.SaveOneTimeSecret(customerID, pw)` — stage ① for delivery;
+4. bookkeeping: `MarkEscrowStale` + `offsite_reissued` + `escrow_stale` events.
+
+There is **no reference to a restic password anywhere in the function**, and none is possible — the
+repo password is generated on the box and never leaves it except into the escrow blob. The hub does
+not hold it in any openable form (`api/handler.go`: *"The hub stores the bytes and NEVER decrypts them
+(it has no recovery code)"*).
+
+**But step 4 is justified by a claim that is false**, `offsite.go:198-201`:
+
+> `// v0.57.0 (2.3, the escrow-honesty fix): the restic repo password just changed, so any existing`
+> `// key-escrow blob — which sealed the OLD password — is now STALE.`
+
+The escrow *is* marked stale; the stated reason for marking it is not something this function does.
+The same false premise appears twice more: `api/handler.go:1067-1069` (*"The re-issuer resets the
+restic repo password, which makes the OLD escrow blob stale"*) and `OPEN-ITEMS.md`'s R-193 row, which
+inherited it from R-39.
+
+> **This is the EIGHTH entry in `CLAUDE.md`'s table of comments asserting an invariant the code does
+> not provide** — and the first where the comment is not merely unenforced but factually describes a
+> different function. It survived because it reads as settled and because its *effect* (a stale escrow)
+> is real, so nobody checked the *cause*.
+
+**Consequence, and it is a live defect, not a documentation nit.** On the ordinary Re-issue shape —
+a consumed-but-failed install on a box that still has its `repo_password` file — the box re-applies,
+`WriteOffboxSecrets` finds the file present and **keeps it**, and the repo password is unchanged. The
+hub has nonetheless told the customer, in Hungarian, that their recovery escrow is stale and asked
+them to re-run the ceremony. **A false staleness alarm and an unnecessary ceremony.** → **R-196**.
+
+**Corroboration against what actually happened on demo-hp yesterday.** The escrow went `pending` after
+the Re-issue and the task asks: if the repo password was untouched, what explains that? **It is fully
+explained, and not by the Re-issue.** Two independent mechanisms, both hub-side-invisible:
+
+1. `ApplyOffsiteTarget` (`offbox.go:482-494`) carries `EscrowState` over from the *existing* target —
+   but the rebuilt guest had **no existing target**, so `cur` was nil, so the fallthrough
+   `if tgt.EscrowState != "escrowed" { tgt.EscrowState = "pending" }` fired.
+2. The box had minted a **new** repo password moments earlier (Q2), so `pending` was *correct* — the
+   new key genuinely was not escrowed.
+
+**The Re-issue's `escrow_stale` event fired at 07:11:51 for the wrong reason and happened to be true
+for a different one.** Coincidence, not correctness. The proof that these are independent is
+**demo-felhom**: it received **no Re-issue at all** (measured — its only `escrow_stale` /
+`offsite_reissued` events in the entire history are dated `2026-07-21 08:29:29`), its repo password
+changed anyway, and **nothing marked its escrow stale for thirteen hours.**
+
+> **The mechanism is exactly inverted.** `escrow_stale` is wired to the one path that does **not**
+> change the repo password, and absent from the path that **does**.
+
+---
+
+## Q5 — Why did one box recover itself and the other not?
+
+### **RULING: confirmed from the ledger — an unconsumed staged secret, thirteen days old. And the recovery was partial in a way the narrative missed.**
+
+**Method — I4, the `one_time_secrets` ledger, no values read:**
+
+| customer | `created_at` | `consumed_at` |
+|---|---|---|
+| `demo-felhom` | `2026-07-21 08:29:29` | `2026-08-03 07:17:58` |
+| `demo-hp` | `2026-08-04 07:11:51` | `2026-08-04 07:12:06` |
+
+demo-felhom's row was **created 2026-07-21 and sat UNCONSUMED for 13 days** — traceable to a Re-issue
+at that exact timestamp (its `offsite_reissued` + `escrow_stale` event pair, `2026-07-21 08:29:29`).
+When the rebuild dropped its target on 2026-08-03, the fresh controller consumed the spare at 07:17:58
+and was reporting `offsite` again by 07:19:10. **76 seconds, on a credential nobody had planned to
+leave there.** Confirmed from the ledger, not the narrative.
+
+**What would have happened without one:** exactly demo-hp — `consume` → **404** (`api/offsite.go:20-22`,
+`no unconsumed offsite password`), the apply-bridge returns
+`offsite-apply: consume one-time password: …` and retries forever, and the tier stays absent until an
+operator acts. demo-hp sat there **25 hours** (`2026-08-03 06:12:19` → `2026-08-04 07:12:01`).
+
+**The correction to the contrast (see Q2):** demo-felhom's spare restored *delivery* only. Its 36
+snapshots / 1.14 GB were orphaned by the same rebuild. **Neither box's data survived; one box's
+plumbing did.**
+
+*Not established:* demo-felhom's escrow generation before 2026-07-21. `host_escrow_superseded` holds
+only two rows in the whole database (ids 3 and 4, one per box, both superseded 2026-08-04), so whether
+the 2026-07-21 Re-issue's ceremony re-sealed an *unchanged* password — the predicted false-staleness
+shape — **cannot be shown from this data**. Stated as unknown rather than asserted.
+
+---
+
+## Q6 — Why is delivery one-shot at all?
+
+### **RULING: the design's reason is that ① is a LIVE PROVIDER PASSWORD, not a bootstrap token; and the honest answer to the security question is that an automatic restage weakens EXPOSURE WINDOW, not authentication.**
+
+**The design's own reasons, from source:**
+
+- `api/offsite.go:9-12` — *"serves the one-time transient offsite password to the controller **EXACTLY
+  ONCE** … The value is returned once then marked consumed — a second call 404s. NEVER logged."*
+- `offsite.go:145-149` — *"the **EXPLICIT** operator recovery for a consumed-password dead-end … It is
+  **NOT** implicit rotation: ProvisionOffsite never calls this."*
+- `offsiteapply.go:1-6` — the apply-bridge is *"idempotent (a descriptor hash marker prevents
+  **re-consuming a spent password**)"*, i.e. the design treats a wasted consume as a real hazard, which
+  R-71a's whole settle-gate exists to prevent.
+
+**The security half, stated rather than skipped.** The question is: *could a machine that is not the
+real one obtain a credential via an automatic restage?*
+
+**What stands there today is authentication, and a restage does not remove it.** The consume endpoint
+is gated by `checkAuthCustomer` (`api/offsite.go:14-18`) — **the customer's API key, the same
+credential as the config pull**. Anything that could consume a restaged password can already pull the
+customer's whole config. So an automatic restage **grants no new access to an unauthenticated party**.
+
+**What it genuinely weakens is the exposure window.** Today a live Hetzner sub-account password is
+fetchable only in the seconds after an operator deliberately staged one. Under an automatic restage it
+becomes fetchable **whenever the box reports no offsite target** — a condition an attacker holding the
+API key can arguably induce. The credential's blast radius at the provider is the customer's own
+sub-account (label-scoped; `ReissueCredentials` refuses unless the label lookup finds **exactly one**,
+`offsite.go:164,180`), so this is a widening of *when*, not of *what*.
+
+**And one further guard must survive any change** — R-39(a), `monitor/offsite_delivery.go:26-30`:
+`SaveOneTimeSecret` is last-write-wins, so restaging on top of an **unconsumed** secret clobbers a
+password a box may be about to consume. Any automatic restage must re-read the row immediately before
+acting and refuse unless it is still a **consumed** row. That guard is correct and is not what R-192 is
+about.
+
+---
+
+## Q7 — R-192's guard: exactly what it reads, and why it read wrong
+
+### **RULING: both halves confirmed, with numbers. The guard reads the OLDEST 500 reports since the consume, and for demo-hp all 500 predated the rebuild by six days.**
+
+**The query, quoted** (`hub/internal/store/store.go:987`):
+
+```sql
+SELECT report_json FROM reports WHERE customer_id = ? AND received_at > ? ORDER BY id LIMIT 500
+```
+
+`ORDER BY id` ascending with `LIMIT 500` = **the oldest 500**, not the newest.
+
+**Reproduced against the live data (I1)**, with demo-hp's real consume anchor `2026-07-23 09:53:41`
+(recovered from R-192's record — it is no longer in `one_time_secrets`, see I4):
+
+| measurement | value |
+|---|---|
+| `total` (what the guard sees) | **500** |
+| `withOffsite` (what the guard sees) | **500** |
+| oldest report in that window | `2026-07-23 09:53:47` |
+| newest report in that window | **`2026-07-28 11:17:40`** |
+| true `total` over the same window, no LIMIT | 1174 |
+| true `withOffsite` over the same window, no LIMIT | 1063 (⇒ 111 without — matching the 111 ABSENT reports in I3) |
+
+**The whole 500-report evidence set ends 2026-07-28 — six days before the 2026-08-03 rebuild.**
+
+**Defect (a) — the message is exactly inverted.** `maybeEmitStuck`
+(`monitor/offsite_delivery.go:~110`) interpolates `status.ReportsSinceConsume` into a string that
+hardcodes the phrase *"carry no offbox target"*, and never reads `OffsiteReportsSinceConsume`. The
+live e-mail, quoted in full from the events table:
+
+> *"Offsite delivery stuck: one-time password consumed 284h19m0s ago and **500 report(s) since carry
+> no offbox target** — the credential is likely burned (apply died between consume and persist).
+> Re-issue delivers a fresh one."*
+
+`OffsiteReportsSinceConsume` for that same call was **500**. Every one of them carried a target. The
+message states the precise negation of its own measurement, and prescribes a remedy for a failure mode
+that did not occur.
+
+**Defect (b) — the heal refuses silently, on stale evidence.** `maybeHeal` returns bare on
+`status.OffsiteReportsSinceConsume != 0` (it was 500), with **no log line**. `offsite_credential_restaged`
+has never fired for any customer — confirmed: zero rows of that event type in the database.
+
+**What the guard SHOULD discriminate on — the shape, not the code.** The guard's intent is right:
+*"there is offbox evidence, so this is a regressed apply, not a burned credential — the operator's
+call."* Its error is that **it asks the question of the wrong time period**. Offbox evidence from
+before a rebuild is not evidence that the credential still works. The discriminator must be
+**recency-bounded and rebuild-aware**: judge on evidence *after the newest `controller_started` /
+config-hash change*, or on the latest N reports — never on "everything since the consume". And a
+refusal must **name its reason in the log**, because "we chose not to heal" and "the heal never ran"
+must not look identical (the S-16 rule, one layer down).
+
+**Do not fix it in isolation.** Under Q2's finding, a successful auto-restage on demo-hp would have
+restored the transport and the box would *still* have minted a new repo password. **The heal cannot
+protect the data; it can only protect the plumbing.** Whatever shape it takes must say so.
+
+---
+
+## Q8 — What each candidate would cost
+
+**A fourth option exists and is the important one**, because (a) and (b) both address ① and Q2 shows
+the damage is in ③.
+
+### (a) The hub restages automatically when a re-enrolled box reports no offsite
+
+**A version of this ALREADY EXISTS and is wired to the wrong event.** `reissueOnReenroll`
+(`hub/internal/api/handler.go:1051-1084`), leg **F3**:
+
+> *"F3 — offsite continuity: re-stage the one-time offsite password to the fresh controller (the
+> one-time password only ever reached the OLD controller)."*
+
+It is called from `handleHostEnroll` — but **after** a mint-once-reuse short-circuit
+(`handler.go:1004-1016`): `if existing != nil { …return… }`. **A guest rebuild leaves the `hosts` row
+intact** (measured: `demo-hp-bb76ea` is the same host_id throughout the incident; only the agent leaf
+re-keyed, `host_leaf_changed` 2026-08-03 06:09:40), so re-enroll returns the existing credential and
+**F3 is never reached.**
+
+- **Changes:** hub only. Either lower the trigger from host-enrollment to a controller-level rebuild
+  signal (the `config_hash` change + `controller_started` pair the hub already receives and already
+  logs), or fix R-192's guard so the existing R-71c self-heal can act.
+- **Weakens:** the exposure window in Q6. Needs the R-39(a) unconsumed-secret guard kept intact.
+- **Does NOT solve:** ③. The box still mints a new repo password and still orphans the repo. **This
+  option restores the plumbing to a box whose data key is already gone.**
+- **Cost:** small. **Value against the actual harm: near zero.**
+
+### (b) The credential becomes recoverable from escrow at re-bootstrap
+
+- **BLOCKED AS STATED, and this is a hard constraint, not an estimate.** The escrow blob is
+  **R-wrapped and zero-knowledge** — the hub holds opaque bytes and has no recovery code
+  (`api/handler.go`, and D6 in `CONTEXT.md` S-3). **A rebuilt box cannot unseal it without the
+  customer's recovery code.** Any design that says "recover it at re-bootstrap" is describing a
+  customer-present ceremony, i.e. what the existing manual `offboxInjectPasswordHandler` already is.
+- **Does NOT solve:** an unattended rebuild, which is the whole scenario.
+- **Cost: not implementable as specified.**
+
+### (c) — NOT PREVIOUSLY NAMED — the agent retains the repo password across a guest rebuild
+
+The agent lives **on the Proxmox host** and survives a guest rebuild. It already receives the repo
+password today: the controller pushes it over the pinned local API
+(`POST /escrow/stage-secret` → `agent/internal/localapi/escrow_stage.go`), and the agent writes it to
+the fixed 0600 path `/var/lib/felhom-agent/escrow-stage/restic_repo_password`
+(`agent/internal/escrow/identity.go:44-46`) **transiently — wiped by the ceremony**
+(`WipeStagedResticPassword`). The seam, the transport, the pinning and the file are all already built.
+
+- **Changes:** agent (retain rather than wipe; serve back over the same pinned local API) + controller
+  (on a fresh data dir, ask the agent before minting — the `InjectOffboxPassword` seam already exists
+  and is exactly the right shape).
+- **Weakens:** the escrow's *zero-knowledge* posture in one specific way — the irreplaceable data key
+  would now sit at rest on the Proxmox host, which the operator has root on. **Measured against D6,
+  that is not a new exposure in principle** (D6 already states plainly that *"the operator cannot read
+  customer data"* was never the security property, and the operator holds root on every box) — but it
+  IS a new copy, and copies are the thing R-133 is already open about for the hub DB. **This is a real
+  trade and it is the operator's to make.**
+- **Solves:** exactly the failure that occurred, unattended, on every rebuild, without any hub
+  involvement and without any credential leaving the premises.
+- **Cost:** medium. Two repos, one existing seam each.
+
+### (d) Do neither — keep the manual Re-issue, and fix what LIES
+
+- **Changes:** R-192's message + refusal log; correct the three false comments about what Re-issue
+  rotates; and — the part that actually matters — **make a repo-password change VISIBLE**. The hub
+  already receives `restic_pw_sha256` on every escrow upload and already stores it. Comparing the new
+  hash against the superseded one is a two-line verdict, and it is the signal that was missing on
+  demo-felhom for thirteen hours → **R-197**. (Note the shape: **this spike answered its own hardest
+  question with a comparison the hub could be making automatically and is not.** Both values were
+  already in the database; nothing read them.)
+- **Weakens:** nothing.
+- **Does NOT solve:** the data loss. Every rebuild still orphans the repo; the operator simply finds
+  out the same day instead of never.
+- **Cost:** small.
+
+### Ranked, with a recommendation
+
+| rank | option | why |
+|---|---|---|
+| **1** | **(d) now — the honesty pass** | It is cheap, it weakens nothing, and **it is the only option that helps the incident already in flight.** The alarm that lies is worse than no alarm; the repo-password-changed detector is the one signal whose absence let demo-felhom lose 1.14 GB of history silently. Ship this regardless of what is decided about the rest. |
+| **2** | **(c) — agent-retained repo password** | The only candidate that addresses ③, which is where the harm is. Every seam it needs exists. It costs one deliberate trade-off the operator must actually make. |
+| **3** | **(a) — auto-restage** | Worth doing **after** (c), never instead of it. On its own it restores the transport to a box whose data key is already gone, and would have made both boxes look healthy on 2026-08-04 while the snapshots were orphaned — a *strictly worse* outcome than the current loud failure. |
+| **4** | **(b) — recover from escrow at re-bootstrap** | Not implementable as stated; the escrow is zero-knowledge by design. |
+
+**Recommendation: ship (d), then decide (c).** And do not ship (a) first — it would have hidden this.
+
+---
+
+## What could not be established
+
+Named rather than guessed (§8 rule 4).
+
+1. **The 2026-08-05 02:15 UTC run's actual outcome** (Q3). Predicted `ErrOffboxOrphaned` from source +
+   claim state; **not measured**.
+2. **Whether the orphaned ciphertext still exists at the provider.** 40.9 MB (demo-hp) + 1.14 GB
+   (demo-felhom) are *presumed* present at `/home/felhom-repo` — reading the endpoint needs the SFTP
+   credential, which this session did not touch. The Hetzner API has no directory-listing surface.
+3. **Whether demo-felhom's 2026-07-21 Re-issue re-sealed an unchanged password** (the predicted
+   false-staleness shape, Q4/Q5). Only two `host_escrow_superseded` rows exist in the whole database;
+   the generation before 2026-07-21 is not retained.
+4. **What removed demo-hp's offsite target at the rebuild** is fully explained (fresh data dir), but
+   **why the guest was rebuilt on both boxes on consecutive mornings** is outside this spike.
+
+---
+
+## The operator's question — stated, and NOT answered here
+
+> **The irreplaceable secret in the offsite tier is the restic repository password. It is generated on
+> the box, it is destroyed by a guest rebuild, and the only copy that survives is sealed under the
+> customer's recovery code — which nothing but a human can open. Every machine is going to be
+> reinstalled.**
+>
+> **Do you want that key to survive a rebuild unattended — by keeping a copy on the Proxmox host, where
+> you already have root — or do you want it to stay sealed under the customer's recovery code and
+> accept that every rebuild starts a new repository and orphans the old one, provided the system says
+> so loudly on the day it happens?**
+
+Option (c) is the first. Option (d) alone is the second. **This spike does not choose.**
diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md
index 34a1e1c..572077e 100644
--- a/documentation/backlog/OPEN-ITEMS.md
+++ b/documentation/backlog/OPEN-ITEMS.md
@@ -36,8 +36,11 @@ State: `BLOCKED` · `READY` · `WAITING-ON-OPERATOR` · `WATCHING`. Every row ha
 | **R-190** | **A storage ACL that demonstrably WORKED in the morning was gone by mid-morning, and nothing recorded its removal.** On demo-felhom, a `vzdump` by `felhom-agent@pve!agent` with `--storage felhom-backup` completed **OK at 04:44:50 CEST 2026-08-03** (task log read in full). From **09:24:56** the same path returned `HTTP 403 … missing privilege Datastore.Allocate at /storage/felhom-backup`, six times through the day, until the grant was re-applied by hand at 18:54. By ~14:50 `pveum acl list` showed **no row at all** for that path | **MITIGATION SHIPPED 2026-08-04** (agent **v0.124.0 → v0.124.1**) — **MECHANISM STILL OPEN** | — | **Why this is not just R-185 restated:** R-185's mechanism (the installer's Scenario-F arm resolves a pre-existing target without granting) explains a box that NEVER had the grant. This box HAD it and lost it, inside five hours, with the machine up throughout. **Ruled out, each by measurement:** a host reinstall (`uptime` = 12 days); any `pveum`/ACL/`user.cfg` activity in syslog between 04:00 and 10:00 (none); any ACL entry in `/cluster/log` (none). **Correlated, not established:** `host_leaf_changed` at 09:15 and `controller_started` at 09:19 — guest 9201 was reprovisioned nine minutes before the first 403. PVE removes ACLs at `/vms/<vmid>` when a guest is destroyed (`AccessControl::remove_vm_access`, the F-LEAK mechanism); whether any path can take a `/storage/<id>` row with it has NOT been established and is the first thing to check. **Why it matters more than the grant did:** a permission that can vanish silently makes every ACL-based guarantee on these hosts provisional, and the agent's new store-grant probe (v0.123.0) now detects the STATE but says nothing about the TRANSITION. **Worth pairing with:** whether the probe should report a grant it once had and no longer has as a distinct, louder signal than one it never had **THE ROW NOW REFLECTS THE MITIGATION, NOT THE CAUSE — stated plainly because the two are different things.** The box is resilient; the loss is still unexplained. **Mitigation:** when the store-grant probe finds the grant absent, the agent runs the EXISTING root wrapper (`felhom-backup-target-apply grant <id>`) and **re-reads once** to confirm — the pbsdr R-22 self-grant shape, including its restraint. **No new privileged surface:** the sudoers vector `grant *` already covers any storage id (confirmed in `configs/felhom-agent.sudoers`, not assumed), and the verb already grants BOTH user and token. The verb existed, was permitted, and had only ever been called at storage CREATION — the *built but never wired* shape in a verb rather than a seam, this project's seventh instance. Bounded at one attempt per tier per hour (a storage can be unreadable for reasons an ACL cannot fix; re-granting every cycle is a repair loop wearing a fix's clothes). **THE RECORD IS THE HALF THIS ROW IS ABOUT, and v0.124.0 got it wrong in production while every unit test passed.** It reported degraded for *one cycle* — meaning the probe call that repaired. But `probeAll` is invoked INDEPENDENTLY by the self-check log and by the collector building a host-report: on the box the repairing call was the log's (`09:39:34`, journal shows the repair and `degraded=1`) and the report three seconds later found the grant present and sent **`ok`**. The agent's journal had the record, the hub had nothing, and the operator would have learned nothing — the exact silence this row exists for, re-created inside its own mitigation. **v0.124.1** replaces it with a latch on TIME (20 min > the 900 s report interval), so at least one report must carry it. **PROVEN LIVE, twice, on demo-felhom** (grant deleted by hand, both rows): agent logs `store-grant: GRANT WAS MISSING AND HAS BEEN SELF-REPAIRED — investigate the loss (R-190) target=felhom-backup privilege=Datastore.AllocateSpace action="felhom-backup-target-apply grant felhom-backup" confirmed_by=re-read`; the ACL rows return; and on v0.124.1 the **host-report at 08:00:30Z carried `status=degraded`** with the explanation and the hub raised `agent_capability_degraded` and **e-mailed the operator** at 08:00:40. Nothing new was built to carry it — the hub's existing ok→degraded→ok edge is the channel, and the text rides `Feature` because that is the field the hub interpolates into the e-mail (`Reason` does not travel). **PART 3 — the single bounded pass at the mechanism, with the negatives named.** The lead §8.6 nominated is real as a CLASS and is documented in our own installer: *"`pveum user token remove` purges the token's ACL, so re-applying post-rotate is mandatory"*. **It does NOT fit this box.** A rotation purges ALL of the token's ACLs and mints a NEW secret; demo-felhom's token still authenticates with the same secret (`--selftest` OK), it retained its other three storage grants throughout, and only `felhom-backup` was refused. No installer run is evidenced (no 2026-08-03 install log; host uptime 12 days at the time). Previously ruled out and unchanged: a host reinstall, any `pveum`/ACL/`user.cfg` activity in syslog 04:00–10:00, any cluster-log ACL entry. **Ruled out on THIS box; NOT ruled out fleet-wide** — any installer run still purges and re-grants only the hardcoded `PVE_STORAGES` set, though installer 1.24.0's reuse-arm fix now re-grants the backup target on that path. **A NEW OBSERVATION FROM THE LIVE RUNS, relevant to the timeline:** PVE **caches permissions** — after deleting both ACL rows the probe still read the privilege as present for ~40 s in one run and ~16 min in another. Detection is only as prompt as that cache, and a cache expiry could equally explain why a box kept working for hours after a grant was removed. → **R-194**. **The alert pair CLOSED on its own at 10:30:40** (`degraded → ok`, `agent_capability_recovered`) once the 20-minute latch expired — one lost grant, one e-mail, one recovery, nothing further. | CC |
 | **R-191** | **Every weekly offsite backup UPLOADS successfully and then FAILS the job on a prune the box is deliberately not allowed to do — on both demo boxes.** Measured on demo-felhom 2026-08-04 06:49–06:53: the upload completed (223 s, 629 MiB of 1.874 GiB, **67.2 % reused incrementally**), then `ERROR: prune 'ct/9201': proxmox-backup-client failed: Error: permission check failed - missing Datastore.Modify\|Datastore.Prune on /datastore/felhom-offsite/demo-felhom` → `ERROR: Backup of VM 9201 failed - error pruning backups` → `TASK ERROR: job errors`. The hub raised `whole_guest_backup_failed` | **CLOSED — SHIPPED 2026-08-04** (installer **1.25.0**; both live boxes corrected) | — | **This is R-89's rule not reaching the config.** R-89 moved PBS pruning SERVER-SIDE — *"boxes set `keep_last: 0`, ep0 runs prune jobs; box tokens stay write-only, never widen the grant"*. The token behaves exactly as designed: it refuses. But **both** demo boxes still arm the offsite tier with `keep_last=2 prune_pbs_allowed=true` (`backup_targets: [{target_id: felhom-pbs, cadence_seconds: 604800, keep_last: 2}]`), so every run asks for a prune that must fail. **The data is SAFE and that is why this is not a P1:** the snapshot lands before the prune is attempted; what is wrong is the job's VERDICT and the weekly operator e-mail it produces. **But it is corrosive in the specific way this project keeps finding:** a backup that reports FAILED while succeeding trains the operator to discount `whole_guest_backup_failed`, which is the same alert that would carry a real one — and it is exactly the failure the R-100 corollary warns about, an alarm whose text is true and whose trigger is not the thing you would act on. **Fix is one config line per box** (`keep_last: 0` on the PBS tier) plus whatever writes it on a fresh install; **deliberately NOT applied in this session** — the session was a runbook with an explicit "change nothing, and if a change appears necessary, stop and report" rule, and a retention field on a live backup tier is not a change to slip into an observation run. **Check before fixing:** whether ep0's prune jobs actually cover these two namespaces, or the snapshots simply accumulate once the box stops asking **THE GATE WAS RUN FIRST, AND IT MATTERED.** Before disabling anything, ep0 was read (read-only, Tier 2): prune jobs `prune-demo-felhom` and `prune-demo-hp` exist on datastore `felhom-offsite`, one per namespace, `schedule 03:30`, `keep-last 2`, comment *"R-82 retention keep-last=2, server-side (box tokens are write-only)"* — and they have run **every day since 2026-07-27: 18 tasks, all `status=OK`**. The newest task log reads `retention options: --ns demo-felhom --max-depth 0 --keep-last 2` / `keep ct/9201/2026-07-27…` / `keep ct/9201/2026-07-28…` / `TASK OK`. Retention happens, and it happens there. **A METHODOLOGICAL WARNING WORTH MORE THAN THE FIX.** Three separate queries said the OPPOSITE — *no prune jobs have ever run* — and **all three were broken instruments**: `worker-type` where the field is `worker_type`; the value `prune` where the worker type is `prunejob`; and `journalctl -u proxmox-backup` where the unit is `proxmox-backup-proxy`. A fourth reading (3 snapshots under keep-last 2) was mis-framed by CC and self-corrected — the third snapshot had landed AFTER that day's 03:30 window. Acting on any of them would have disabled the only pruning ATTEMPT while reporting that nothing prunes: a weekly false alarm traded for unbounded growth on the protected endpoint, invisible for months. **The gate is what caught it, and only because it demanded evidence rather than a verdict.** **Shipped:** installer **1.25.0** writes `keep_last: 0` on the offsite tier (the agent's existing guard `allowPBSPrune = !primary && keep_last > 0` already reads that as *never prune from the box* — no agent change), the justifying paragraph is rewritten to say where retention lives and cite R-89, and `hostinstall_gates.py` asserts it (red-proved: pinning `keep_last: 2` back fails the gate). **Both live boxes corrected in their own config** — `backup tier armed target=felhom-pbs … keep_last=0 … prune_pbs_allowed=false` on demo-felhom and demo-hp, with the local tier untouched at `keep_last=3`. Served over HTTPS at `1.25.0` with `"keep_last":0` in the served bytes. **STILL TO OBSERVE:** the next weekly offsite run completing OK end-to-end. The change removes the failing step; the *schedule* proving it is next week's event, and this row should carry that line when it happens. | CC |
 | **R-194** | **PVE's permission cache delays every grant-state verdict by an unknown amount, so "the agent can read it" and "the ACL exists" are not the same measurement.** Observed twice while validating R-190's self-repair on demo-felhom 2026-08-04: both ACL rows for `/storage/felhom-backup` were deleted, and `GET /access/permissions` continued to report `Datastore.AllocateSpace` present — for **~40 s** in one run and **~16 minutes** in another. During that window the capability probe reads healthy and the self-repair does not fire | **OPEN** | — | **Why it matters beyond the delay:** it puts a floor under how fast a lost grant can be noticed, it makes any single permission read a lagging indicator, and — the interesting part — **it is a candidate contributor to R-190's own timeline**: a grant removed at an unknown moment could keep working until a cache expiry, which is exactly the shape of *worked at 04:44, refused at 09:24*. That does not explain what removed it, but it may explain when the refusal SURFACED, and the two have been treated as the same instant. **Not a defect in our code** — it is PVE behaviour, and the mitigation already tolerates it (the repair fires on the next probe after the cache clears). **What is worth deciding:** whether the store-grant probe should ALSO consult the storage content listing as a second signal, since that appeared to reflect the loss immediately (`{"data":[]}` while the permission read still said present) — two signals disagreeing is itself information, and today only one of them is read | CC |
-| **R-192** | **`offsite_delivery_stuck` tells the operator the opposite of what the detector measured, and the self-heal silently refuses for exactly the reason the message denies.** demo-hp has been e-mailing this daily since 2026-08-03 06:12 UTC: *"one-time password consumed 284h ago and **500 report(s) since carry no offbox target** — the credential is likely burned (apply died between consume and persist). Re-issue delivers a fresh one."* **Measured against the hub's own data: all 500 of those reports DO carry an offbox target** | **OPEN** | — | **What actually happened on that box:** the credential was consumed 2026-07-23 09:53:41 and **applied successfully** — the controller reported an `offsite` object continuously until **2026-08-03 05:59:21 UTC**, then it **vanished at 06:12:19** and has been absent for **108 consecutive reports** since. So this is a **regressed apply**, not a burned credential. **Two distinct defects, and the second explains the first's invisibility.** **(a)** `maybeEmitStuck` builds its message from `status.ReportsSinceConsume` (the TOTAL) while hardcoding the phrase *"carry no offbox target"*, and never consults `status.OffsiteReportsSinceConsume` — which is the field that says the opposite. The recommended action (*Re-issue*) is aimed at a failure mode that did not occur. This is R-100's corollary again: an alarm whose text stopped matching what its verdict counts. **(b)** `maybeHeal` refuses **silently** (`OffsiteReportsSinceConsume != 0` → *"regressed-apply shape → operator's call"*, a bare `return` with no log line), so the operator gets a daily e-mail with the wrong story, no heal, and nothing anywhere saying why the heal declined. `offsite_credential_restaged` has **never** fired, on any customer. **The underlying condition is REAL and is the part that matters:** demo-hp currently reports no offsite target at all, i.e. that box's customer app-data has **no off-site copy right now** — and it has been that way since 08:12 CEST on 2026-08-03. A spot check inside the controller container found no restic environment, consistent with the report. **What removed it is not established** and is the first thing to find out. **Fix shape:** the message must state which shape was detected (burned vs regressed) and say what to do for each; the heal's refusal must log its reason; and the regressed shape probably deserves its own event type rather than borrowing the burned one. **Do NOT 'fix' it by widening the heal to restage over a regression** — the guard is right, only mute **CAUSE ESTABLISHED 2026-08-04 (operator confirms no hub-side offsite config change).** The regression is a **guest REBUILD**: at 06:09:40 `host_leaf_changed` (agent re-keyed), at 06:12:18 `controller_started (0.192.0)` — the controller went **0.187.0 → 0.192.0** with a **new config hash** (`1f725a2e843c` → `744e83d72c80`) — and the report at 06:12:19 is the first without `offsite`. The pre-rebuild object was fully healthy: `escrow_state: escrowed, last_status: ok, last_success 2026-08-03T02:16:39Z, snapshot_count 15, repo_size 40.9 MB`. → **R-193** owns the rebuild half. **AND THE HEAL'S GUARD IS WRONG FOR EXACTLY THIS CASE, which is why the automation that exists to fix it declined.** `maybeHeal` refuses when `OffsiteReportsSinceConsume != 0`, reading that as *"the apply regressed, so it is the operator's call"*. But `CountReportsOffsiteSince` counts the **OLDEST 500 reports since the consume** (`ORDER BY id LIMIT 500`) — for demo-hp all 500 predate the rebuild. **Offbox evidence from before a rebuild is not evidence that the credential still works**, so the guard reads healthy history as a reason not to heal a box that demonstrably cannot apply. The fix is to judge on RECENT evidence (e.g. the latest N reports, or evidence after the newest `controller_started`), not on everything since the consume. | CC |
-| **R-193** | **A guest rebuild silently drops the off-site app-data tier, and nothing restages the credential.** demo-hp was rebuilt on 2026-08-03 (controller 0.187.0 → 0.192.0, new config hash, agent leaf re-keyed at 06:09:40). Before it, the offsite tier was healthy and working — `escrow_state: escrowed`, `last_status: ok`, last success **02:16:39Z that morning**, **15 snapshots, 40.9 MB**. After it: no `offsite` object in any of **108** reports, and **no off-site copy of that customer's app data since 08:12 CEST on 2026-08-03** | **OPEN** | — | **Mechanism, fully evidenced.** The restic credential reaches a box exactly once, as a one-time secret. demo-hp's was consumed **2026-07-23 09:53:41**; the rebuilt controller came up with a fresh data volume, no copy of it, and **no way to ask for another** — the hub is the only side that can stage one, and it will not re-stage a consumed secret on its own (the R-71c self-heal would, but it refuses — see R-192). **demo-felhom survived the SAME rebuild by luck, and the contrast is the proof:** its secret was created 2026-07-21 and still **UNCONSUMED**, so when its config hash changed at 07:17:54 and `offsite` dropped for exactly one report, it consumed the staged secret at **07:17:58** and was reporting `offsite` again by 07:19:10. One box had a spare credential staged and recovered in 76 seconds; the other did not and has been unprotected for a day. **That difference was not a design decision — it was an accident of which box happened to have an unconsumed secret lying around.** **Why this is not just "re-issue it":** the remedy (Re-issue) resets the sub-account password via the Hetzner API and, per R-39's record, **rotates the restic password and makes the escrow STALE** — so it needs the recovery-code ceremony re-run, and the continuity of the 15 existing snapshots under the new credential must be VERIFIED, not assumed (`hub v0.60.0` retains superseded escrow, and the orphan guard is move-aside-never-delete). That is an operator act with a customer-facing consequence, so it is not something to fire automatically without deciding the escrow question first. **What to design:** a rebuild is a normal, expected event on these boxes — the offsite tier must survive one, either by the hub restaging automatically when a re-enrolled box reports no offsite (the R-192 guard fix makes this safe), or by the credential being recoverable from escrow at re-bootstrap rather than delivered once and unrecoverable **RESOLVED ON THE BOX 2026-08-04 (operator-authorised).** Re-issue fired through the designed endpoint (`POST /configs/demo-hp/offsite-reissue`, HTTP 303): hub staged a fresh one-time password at **07:11:51**, the box's config hash moved `744e83d7` → `5eee0e42`, R-71a's settle-gate reported **GO** (*at/above floor 0.156.0, we are 0.194.0*), the password was **consumed 15 s later at 07:12:06**, and the controller logged *offsite configured for u629488-sub3@…:/home/felhom-repo* at 07:12:09 — the **same sub-account (275124) and the same repo path**, since Re-issue resets the sub-account password and the one-time password is only the transport credential used once to install the box's own SSH key. **THE ESCROW DID NOT RECOVER BY ITSELF — a correction to this session's own first reading.** `escrow_state` went `pending` → `escrowed` 15 s after the apply and CC inferred an automatic re-escrow; **the operator had run the ceremony**. It needed a human, on BOTH boxes: demo-hp escrowed 07:16:02, demo-felhom (whose offsite re-applied on its own the previous day but whose escrow had been `pending` ever since) escrowed 07:20:28. **A 15-second state change is not evidence of automation** — that is the same class as reading an absent log line as success. **Snapshot continuity is NOT yet established and must not be assumed from the counters:** both boxes report `snapshot_count: 0, repo_size_bytes: 0`, but the run-history keys (`last_run`, `last_status`, `last_success`) are **absent entirely** rather than zeroed — the shape of a controller that has never run an offbox backup in this lifetime, not of an empty repo. demo-hp's pre-rebuild object carried all three plus 15 snapshots / 40.9 MB. **The next scheduled `offbox-backup` (04:15) decides it:** 15+ snapshots ⇒ the repo reattached; 1 ⇒ it started fresh and the old snapshots are orphaned-but-retained. Record the answer here. | CC |
+| **R-195** | ~~**A customer with no machine ever bound e-mailed an `expected_dbdump_missed` ERROR every morning.** `david` — a real prospective customer whose record was created 2026-08-01 16:51:49 with **hosts=0, host_deletions=0, host_reports=0, reports=0** — raised the alarm at 03:00 UTC on 08-02, 08-03 and 08-04~~ | **SHIPPED** (hub **v0.92.0**, 2026-08-04) | — | **The mechanism, and it is the interesting half: the skip that protects every other silent customer is keyed off having reported at least once.** `CheckBackupDeadlines`' down-skip reads `StalenessChecker.GetState()`, whose map is seeded from `store.GetCustomers()` — a query **over the `reports` table**. A customer with zero reports appears in no row, gets no state, and `GetState()` returns `""` rather than `"down"`. The skip therefore misses exactly the customer it would most obviously cover. (Corroborated live: `peti-felhom` is active with a host deleted 2026-07-15 and **does not** alarm — it has 482 old reports, so it is `down` and skipped.) The backup half was already safe (`reportJSON == ""` → skip); the DB-dump half had no guard at all. **Fix:** `store.HasEverBoundHost(customerID)` = a live `hosts` row **OR** a `host_deletions` tombstone, consulted once per customer at the top of the deadline loop. **The discriminator is deliberately "was a host EVER bound", NOT "has a report arrived"** — a box that was installed and went silent is a real fault and must keep alarming; that is the case the change could break and it has its own test. Fail-**open** on a read error (an unreadable binding must never SUPPRESS a real alarm), and the deferral is LOGGED with its own counter (the v0.73.0 Part-7 precedent: a quiet check must not look like a check that did not run). The anchored-verdict structure is untouched. **Red-proof observed:** deleting the guard fails `TestCheckBackupDeadlines_NeverBoundHost_Silent` with `got [expected_dbdump_missed]` — verbatim the event `david` sent three mornings running. `david`'s record was NOT modified; the record was correct and the alarm was what was wrong | — |
+| **R-196** | **`escrow_stale` is wired to the ONE path that does not change the repo password, and absent from the path that does.** `ReissueCredentials` (`hub/internal/offsite/offsite.go:150-228`) resets **only** the Hetzner sub-account/box password and stages a fresh one-time secret — it contains no reference to a restic password and *cannot*, since that password is generated on the box and never leaves it except into the R-wrapped escrow. Yet it calls `MarkEscrowStale` on the stated grounds that *"the restic repo password just changed"* (`offsite.go:198-201`), and the same false premise is repeated at `api/handler.go:1067-1069` and in R-39's record | **OPEN** | — | **This is the EIGHTH entry in `CLAUDE.md`'s table of comments asserting a guarantee the code does not provide** — and the first where the comment factually describes a *different function*. It survived because its EFFECT (a stale escrow) is real, so nobody checked its CAUSE. **The live defect, not a documentation nit:** on the ordinary Re-issue shape (a consumed-but-failed install on a box that still holds its `repo_password` file) the box re-applies, `WriteOffboxSecrets` finds the file present and **keeps it**, the repo password is unchanged — and the hub has told the customer in Hungarian that their recovery escrow is stale and asked them to re-run the ceremony. **A false staleness alarm and an unnecessary ceremony.** **The inverse is the worse half and is R-193's:** demo-felhom's repo password *did* change on 2026-08-03 with **no Re-issue anywhere in its history** (its only `escrow_stale`/`offsite_reissued` pair is dated 2026-07-21 08:29:29) and therefore **nothing marked its escrow stale for 13 h**. **Fix shape:** mark the escrow stale on the evidence that it IS stale — a changed `restic_pw_sha256` (→ R-197) — not on a Re-issue; and correct all three comments in the same commit. **Not established, so not asserted:** whether the 2026-07-21 Re-issue on demo-felhom re-sealed an unchanged password (the predicted false-staleness shape) — `host_escrow_superseded` holds only two rows in the whole DB, both from 2026-08-04, so the prior generation is not retained. Source: `audits/SPIKE-offsite-credential-recovery-2026-08-04.md` Q4 | CC |
+| **R-197** | **The hub holds both halves of the evidence that a box's offsite DATA key changed, and reads neither.** `restic_pw_sha256` is stored on `host_escrow` and carried to `host_escrow_superseded` on every re-escrow. Comparing the two is what let this spike answer its hardest question in one query — and **nothing in the hub does it** | **OPEN** | — | **Why this is the cheapest real fix on the table.** A changed repo password means the previous offsite repository is now unopenable by the box, i.e. the customer's off-site history is orphaned. That is the single most consequential state change in the backup system, it is **already fully observable from data the hub owns**, and today it produces **no event, no e-mail, no card and no log line** — demo-felhom's went unremarked for 13 hours and would have gone unremarked indefinitely had this spike not run. **Two-line verdict:** on `SaveHostEscrow`, if the incoming `restic_pw_sha256` differs from the row being superseded, emit a distinct operator event naming the orphaned generation. **Deliberately an EVENT, not a heal** — nothing should act on this automatically until R-193's (c)-vs-accept decision is taken; the point is that the operator learns on the day. **Pair with R-196**, which is the same signal aimed at the right trigger. **Generalises past this row:** *a comparison the system could be making from data it already stores, and is not, is a silence with no cost of entry* — cf. R-190's store-grant probe, where the state was read and the TRANSITION was not. Source: `audits/SPIKE-offsite-credential-recovery-2026-08-04.md` Q8 option (d) | CC |
+| **R-192** | **`offsite_delivery_stuck` tells the operator the opposite of what the detector measured, and the self-heal silently refuses for exactly the reason the message denies.** demo-hp has been e-mailing this daily since 2026-08-03 06:12 UTC: *"one-time password consumed 284h ago and **500 report(s) since carry no offbox target** — the credential is likely burned (apply died between consume and persist). Re-issue delivers a fresh one."* **Measured against the hub's own data: all 500 of those reports DO carry an offbox target** | **OPEN** | — | **What actually happened on that box:** the credential was consumed 2026-07-23 09:53:41 and **applied successfully** — the controller reported an `offsite` object continuously until **2026-08-03 05:59:21 UTC**, then it **vanished at 06:12:19** and has been absent for **108 consecutive reports** since. So this is a **regressed apply**, not a burned credential. **Two distinct defects, and the second explains the first's invisibility.** **(a)** `maybeEmitStuck` builds its message from `status.ReportsSinceConsume` (the TOTAL) while hardcoding the phrase *"carry no offbox target"*, and never consults `status.OffsiteReportsSinceConsume` — which is the field that says the opposite. The recommended action (*Re-issue*) is aimed at a failure mode that did not occur. This is R-100's corollary again: an alarm whose text stopped matching what its verdict counts. **(b)** `maybeHeal` refuses **silently** (`OffsiteReportsSinceConsume != 0` → *"regressed-apply shape → operator's call"*, a bare `return` with no log line), so the operator gets a daily e-mail with the wrong story, no heal, and nothing anywhere saying why the heal declined. `offsite_credential_restaged` has **never** fired, on any customer. **The underlying condition is REAL and is the part that matters:** demo-hp currently reports no offsite target at all, i.e. that box's customer app-data has **no off-site copy right now** — and it has been that way since 08:12 CEST on 2026-08-03. A spot check inside the controller container found no restic environment, consistent with the report. **What removed it is not established** and is the first thing to find out. **Fix shape:** the message must state which shape was detected (burned vs regressed) and say what to do for each; the heal's refusal must log its reason; and the regressed shape probably deserves its own event type rather than borrowing the burned one. **Do NOT 'fix' it by widening the heal to restage over a regression** — the guard is right, only mute **CAUSE ESTABLISHED 2026-08-04 (operator confirms no hub-side offsite config change).** The regression is a **guest REBUILD**: at 06:09:40 `host_leaf_changed` (agent re-keyed), at 06:12:18 `controller_started (0.192.0)` — the controller went **0.187.0 → 0.192.0** with a **new config hash** (`1f725a2e843c` → `744e83d72c80`) — and the report at 06:12:19 is the first without `offsite`. The pre-rebuild object was fully healthy: `escrow_state: escrowed, last_status: ok, last_success 2026-08-03T02:16:39Z, snapshot_count 15, repo_size 40.9 MB`. → **R-193** owns the rebuild half. **AND THE HEAL'S GUARD IS WRONG FOR EXACTLY THIS CASE, which is why the automation that exists to fix it declined.** `maybeHeal` refuses when `OffsiteReportsSinceConsume != 0`, reading that as *"the apply regressed, so it is the operator's call"*. But `CountReportsOffsiteSince` counts the **OLDEST 500 reports since the consume** (`ORDER BY id LIMIT 500`) — for demo-hp all 500 predate the rebuild. **Offbox evidence from before a rebuild is not evidence that the credential still works**, so the guard reads healthy history as a reason not to heal a box that demonstrably cannot apply. The fix is to judge on RECENT evidence (e.g. the latest N reports, or evidence after the newest `controller_started`), not on everything since the consume. **SPIKE 2026-08-04 — BOTH HALVES CONFIRMED WITH NUMBERS, still OPEN, still not fixed here** (`audits/SPIKE-offsite-credential-recovery-2026-08-04.md` Q7). The query is quoted at source (`store.go:987`): `... ORDER BY id LIMIT 500` = **the oldest 500**. Reproduced against the live hub DB with demo-hp's real consume anchor `2026-07-23 09:53:41`: the guard sees `total=500, withOffsite=500`, spanning `2026-07-23 09:53:47` → **`2026-07-28 11:17:40`** — the entire evidence set ends **six days before** the 2026-08-03 rebuild. The true window totals are `1174 / 1063` (⇒ 111 without, matching the 111 offsite-less reports). So the e-mail's *"500 report(s) since carry no offbox target"* interpolates `ReportsSinceConsume` while `OffsiteReportsSinceConsume` was **500** — the message states the precise negation of its own measurement. `offsite_credential_restaged` has **never fired for any customer** (zero rows of that type in the DB — checked, not assumed). **A NEW REASON NOT TO FIX THIS IN ISOLATION, from the same spike:** under R-193's Q2 finding a successful auto-restage would have restored demo-hp's TRANSPORT while the box minted a new repo password anyway — the heal can protect the plumbing and **cannot** protect the data, and had it fired on 2026-08-04 both boxes would have looked healthy while their snapshots were orphaned. **That is strictly worse than the current loud failure.** Whatever shape the fix takes must say so in its message. | CC |
+| **R-193** | **A guest rebuild silently drops the off-site app-data tier, and nothing restages the credential.** demo-hp was rebuilt on 2026-08-03 (controller 0.187.0 → 0.192.0, new config hash, agent leaf re-keyed at 06:09:40). Before it, the offsite tier was healthy and working — `escrow_state: escrowed`, `last_status: ok`, last success **02:16:39Z that morning**, **15 snapshots, 40.9 MB**. After it: no `offsite` object in any of **108** reports, and **no off-site copy of that customer's app data since 08:12 CEST on 2026-08-03** | **OPEN** | — | **Mechanism, fully evidenced.** The restic credential reaches a box exactly once, as a one-time secret. demo-hp's was consumed **2026-07-23 09:53:41**; the rebuilt controller came up with a fresh data volume, no copy of it, and **no way to ask for another** — the hub is the only side that can stage one, and it will not re-stage a consumed secret on its own (the R-71c self-heal would, but it refuses — see R-192). **demo-felhom survived the SAME rebuild by luck, and the contrast is the proof:** its secret was created 2026-07-21 and still **UNCONSUMED**, so when its config hash changed at 07:17:54 and `offsite` dropped for exactly one report, it consumed the staged secret at **07:17:58** and was reporting `offsite` again by 07:19:10. One box had a spare credential staged and recovered in 76 seconds; the other did not and has been unprotected for a day. **That difference was not a design decision — it was an accident of which box happened to have an unconsumed secret lying around.** **Why this is not just "re-issue it":** the remedy (Re-issue) resets the sub-account password via the Hetzner API and, per R-39's record, **rotates the restic password and makes the escrow STALE** — so it needs the recovery-code ceremony re-run, and the continuity of the 15 existing snapshots under the new credential must be VERIFIED, not assumed (`hub v0.60.0` retains superseded escrow, and the orphan guard is move-aside-never-delete). That is an operator act with a customer-facing consequence, so it is not something to fire automatically without deciding the escrow question first. **What to design:** a rebuild is a normal, expected event on these boxes — the offsite tier must survive one, either by the hub restaging automatically when a re-enrolled box reports no offsite (the R-192 guard fix makes this safe), or by the credential being recoverable from escrow at re-bootstrap rather than delivered once and unrecoverable **RESOLVED ON THE BOX 2026-08-04 (operator-authorised).** Re-issue fired through the designed endpoint (`POST /configs/demo-hp/offsite-reissue`, HTTP 303): hub staged a fresh one-time password at **07:11:51**, the box's config hash moved `744e83d7` → `5eee0e42`, R-71a's settle-gate reported **GO** (*at/above floor 0.156.0, we are 0.194.0*), the password was **consumed 15 s later at 07:12:06**, and the controller logged *offsite configured for u629488-sub3@…:/home/felhom-repo* at 07:12:09 — the **same sub-account (275124) and the same repo path**, since Re-issue resets the sub-account password and the one-time password is only the transport credential used once to install the box's own SSH key. **THE ESCROW DID NOT RECOVER BY ITSELF — a correction to this session's own first reading.** `escrow_state` went `pending` → `escrowed` 15 s after the apply and CC inferred an automatic re-escrow; **the operator had run the ceremony**. It needed a human, on BOTH boxes: demo-hp escrowed 07:16:02, demo-felhom (whose offsite re-applied on its own the previous day but whose escrow had been `pending` ever since) escrowed 07:20:28. **A 15-second state change is not evidence of automation** — that is the same class as reading an absent log line as success. **Snapshot continuity is NOT yet established and must not be assumed from the counters:** both boxes report `snapshot_count: 0, repo_size_bytes: 0`, but the run-history keys (`last_run`, `last_status`, `last_success`) are **absent entirely** rather than zeroed — the shape of a controller that has never run an offbox backup in this lifetime, not of an empty repo. demo-hp's pre-rebuild object carried all three plus 15 snapshots / 40.9 MB. ~~**The next scheduled `offbox-backup` (04:15) decides it:** 15+ snapshots ⇒ the repo reattached; 1 ⇒ it started fresh and the old snapshots are orphaned-but-retained.~~ **SPIKE 2026-08-04 SETTLED IT WITHOUT WAITING, AND THE ITEM IS BIGGER THAN FILED** (`audits/SPIKE-offsite-credential-recovery-2026-08-04.md`). **(1) THE ONE-SHOT SECRET IS NOT WHERE THE HARM IS.** Three secrets exist; the one-time password is the *recoverable* one (the operator can reset it at the provider any time) and the box's SFTP key is regenerable by design. The **restic repository password** — the DATA key, which the agent's own source calls *"irreplaceable (unlike the SFTP access key…)"* (`felhom-agent/internal/escrow/identity.go:35-39`) — is the one a rebuild destroys, and **nothing automatic ever restages it**: `WriteOffboxSecrets` mints a fresh 256-bit password whenever `<DataDir>/offbox/repo_password` is absent (`offbox.go:392`), and the only recovery path, `InjectOffboxPassword`, has **exactly one caller in the whole repo** — a web form a human pastes into (`web/offbox_handlers.go:189`). **(2) MEASURED, WITHOUT TOUCHING A BOX:** the hub already stores `restic_pw_sha256` on both the live and the superseded escrow, so the question is a hash comparison. demo-hp `8e03eddf…` → `8a9e33aa…`; demo-felhom `48741892…` → `c60c8bc7…`. **Both boxes minted a new repository password.** **(3) THE CONTRAST IN THIS ROW IS FALSE FOR THE DATA.** demo-felhom's 76-second "lucky" recovery restored **delivery only** — its pre-rebuild object carried **36 snapshots / 1.14 GB** (`repo_size_bytes 1136685919`) and it has reported `snapshot_count: 0` in all 109 reports since, with a changed repo password and **nothing marking its escrow stale for 13 h**. Both boxes lost repository continuity; one loudly, one silently, and **the silent one is worse**. **(4) Q3 IS UNMEASURED AND THE BINARY WAS WRONG.** Neither box could run on 2026-08-04 02:15 UTC (demo-hp had no target at all until 07:15; demo-felhom's target was `escrow_state: pending`, which `OffboxRunnable` blocks) — **the decisive run is 2026-08-05 ~02:15 UTC**. Source predicts a **third** outcome, neither 15 nor 1: same account + same repo path (`u629488-sub3:/home/felhom-repo`, and `repoPath` is a compile-time constant) + new password + `claimed: 1` ⇒ `ensureOffboxRepo` classifies **orphaned** and returns `ErrOffboxOrphaned` — **the run refuses and shows the orphan card**, it does not start a silent fresh history. **Record which of the three actually occurs; a prediction from source is not a measurement.** **(5) CANDIDATE (b) IS NOT IMPLEMENTABLE AS STATED** — the escrow is R-wrapped/zero-knowledge and the hub has no recovery code, so "recoverable from escrow at re-bootstrap" describes a customer-present ceremony, i.e. the manual form that already exists. **(6) CANDIDATE (a) ALREADY EXISTS AND IS WIRED TO THE WRONG EVENT:** `reissueOnReenroll`'s **F3 leg** does exactly this (`api/handler.go:1051-1084`) but sits behind `handleHostEnroll`'s mint-once-reuse short-circuit (`if existing != nil { return }`), and a **guest** rebuild leaves the `hosts` row intact — so F3 is never reached. **(7) A NEW CANDIDATE (c), not previously named and recommended second:** the **agent survives a guest rebuild**, already receives the repo password over the pinned local API (`POST /escrow/stage-secret`) and already writes it to a fixed 0600 path — it merely **wipes** it after the ceremony. Retaining it and serving it back is the only candidate that addresses the irreplaceable secret, and every seam it needs exists. Its cost is one deliberate trade the operator must make (a copy of the data key at rest on the Proxmox host — see D6). **SPIKE RECOMMENDATION: ship the honesty pass (R-196 + R-197) now; then decide (c). Do NOT ship (a) first — it would have hidden this.** **NOT CLOSED — WAITING-ON-OPERATOR for the (c)-vs-accept-it decision, stated at the end of the spike.** | CC + operator |
 | — | Storage Box **snapshots** on `storage-box-pool-1` — plan SET (daily 00:00, keep 7) but **0 taken yet** | WATCHING | first run tonight 00:00 | Confirm `size_snapshots > 0` tomorrow; until then the mitigation is armed, not proven | CC |
 | — | `PBS-storage-1` (u629193, box 611421) still `status=active`, 19.9 MB | WAITING-ON-OPERATOR | operator console | Delete the box | operator |
 | **R-90** | ~~ep0 RAM headroom — 4 GiB swap survived its first reboot 2026-07-27; 3.8 GB RAM unchanged~~ | **CLOSED — the operator rescaled ep0 to a CX33 on 2026-08-03** | — | **MEASURED ON THE BOX, not read from an invoice:** `felhom-hetzner` reports `Mem: 7757` MB total (**8 GB**, was 3.8) and `nproc` **4**. **The interim lever survived and was checked rather than assumed** — a resize is a stop/start, so "the swapfile is still there" was an assumption until measured: `/swapfile`, 4 GiB, dated `Jul 27 14:40`, **active** (`swapon --show` → `/swapfile file 4G 0B -2`), 0 B in use on an idle box. **THE 40 GB LOCAL DISK DID NOT CHANGE** and must not be "corrected" alongside the RAM: `/` is 38 G, 58% used. This was a CPU/RAM resize only, so every disk figure in the runbooks still stands — the separate 98 G volume at `/mnt/pbs-datastore` (R-82 P0.3) is unaffected. **Why this was BLOCKED and no longer is:** the row recorded CX33 as *"confirmed unavailable even powered OFF"* — the Cost-Optimized line's limited availability, not a power-state problem. It became available and the operator took it. **Documentation corrected** (`RUNBOOK-ep0-datastore-volume`, `RUNBOOK-pbs-prune-serverside` ×2, `runbooks/offsite-endpoint.md` ×2, `runbooks/target-selection.md`) and **audit/evidence documents ANNOTATED, not revised** (`SPIKE-connectivity-wireguard-2026-07-03`, campaign-10 `phaseA-journal`) — they record what was true when written and that is their value. **Still open and still the operator's, deliberately untouched:** `target-selection.md`'s *"D-d did not name ep0 either way. Confirm it explicitly."* | — |
diff --git a/hub/CHANGELOG.md b/hub/CHANGELOG.md
index 5b41e1f..5136feb 100644
--- a/hub/CHANGELOG.md
+++ b/hub/CHANGELOG.md
@@ -1,3 +1,48 @@
+## v0.92.0 — a customer with no machine ever bound does not alarm (2026-08-04, R-195)
+
+`david` is a real prospective customer whose record was created 2026-08-01 with **no host ever bound**
+— `hosts=0`, `host_deletions=0`, `host_reports=0`, `reports=0` — and it e-mailed an
+`expected_dbdump_missed` **error** at 03:00 UTC on 2026-08-02, 08-03 and 08-04. Nothing has ever been
+expected of that customer, so the honest verdict is UNKNOWN, not missed.
+
+**The mechanism is the interesting half, because the guard that should have covered this is keyed off
+the wrong thing.** `CheckBackupDeadlines` already skips nodes that are `down`, and that skip reads
+`StalenessChecker.GetState()`, whose map is seeded from `store.GetCustomers()` — **a query over the
+`reports` table**. A customer with zero reports is in no row, gets no state at all, and `GetState()`
+returns `""` rather than `"down"`. **The skip misses exactly the customer it would most obviously
+cover.** Corroborated on live data: `peti-felhom` is active with a host deleted 2026-07-15 and does
+*not* alarm, because it has 482 old reports and is therefore `down`. The backup half was already safe
+(`reportJSON == ""` → skip); the DB-dump half had no guard at all.
+
+- **`store.HasEverBoundHost(customerID)`** — a live `hosts` row **OR** a `host_deletions` tombstone.
+  Consulted once per customer at the top of the deadline loop; a false verdict skips the customer with
+  an INFO line and its own counter in the summary.
+- **The discriminator is "was a host EVER bound", NOT "has a report arrived", and that is the whole
+  design.** A box that was installed, bound and then went silent is a real fault and must keep
+  alarming — it has a `hosts` row, so it is judged. Only a customer that never had a machine is
+  UNKNOWN. The tombstone is included for the same reason: a customer whose host was deleted *had* one,
+  and that judgement belongs to the staleness checker, not to this predicate.
+- **Fail-OPEN on a read error.** An unreadable binding must never SUPPRESS a real alarm; it logs and
+  judges anyway.
+- **The anchored-verdict structure is untouched** (R-81/v0.73.0). This is the same invariant — absence
+  is UNKNOWN until an anchor elapses — applied one level up, at the question of whether there is a
+  subject at all.
+
+**Tests, including the one that matters.** `TestCheckBackupDeadlines_BoundButNeverReported_StillAlarms`
+is a real shape — a machine installed and never phoned home — and it is indistinguishable from `david`
+on every signal except the one the guard reads. If the guard is ever "simplified" to key off report
+presence, customer age or a name pattern, it goes red. Plus a bound-then-went-quiet case, a
+deleted-host case, and the predicate's own three inputs. **Companion red-proof observed:** removing the
+guard fails `TestCheckBackupDeadlines_NeverBoundHost_Silent` with `got [expected_dbdump_missed]` —
+verbatim the event `david` sent three mornings running. Restored after.
+
+**`david`'s record was not modified.** The record is correct; the alarm was what was wrong.
+
+**Shipped alongside a spike that deliberately shipped NO code** —
+`documentation/audits/SPIKE-offsite-credential-recovery-2026-08-04.md` (R-193/R-192): what is one-shot
+in the offsite path, and what a guest rebuild really costs. It ends in ranked options and a question
+for the operator; R-196 and R-197 were minted from it and are not implemented here.
+
 ## v0.91.1 — observation may only WIDEN a tier's window, never tighten it (2026-08-03, R-86 Part 2)
 
 **Found by checking v0.91.0 against the live box before trusting it, not by review.** demo-felhom's
diff --git a/hub/internal/monitor/deadline.go b/hub/internal/monitor/deadline.go
index a1ae70d..73ec98b 100644
--- a/hub/internal/monitor/deadline.go
+++ b/hub/internal/monitor/deadline.go
@@ -336,7 +336,7 @@ func CheckBackupDeadlines(s *store.Store, staleness *StalenessChecker, onEvent E
 	midnightBudapest := time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, budapest)
 	sinceUTC := midnightBudapest.UTC()
 
-	var backupMissed, dbdumpMissed, skipped, deferred int
+	var backupMissed, dbdumpMissed, skipped, deferred, unbound int
 
 	for _, id := range customerIDs {
 		// Skip nodes that are down — they already have staleness events
@@ -350,6 +350,37 @@ func CheckBackupDeadlines(s *store.Store, staleness *StalenessChecker, onEvent E
 			continue
 		}
 
+		// ── R-195: a customer with NO machine EVER bound is UNKNOWN, not missed ────────────────
+		//
+		// Both verdicts below ask "did the thing we expect every day happen?". For a customer
+		// that has never had a machine bound, nothing has ever been expected, so the honest
+		// answer is UNKNOWN — the same invariant assessBackupFreshness states above, applied one
+		// level up, at the question of whether there is a subject at all.
+		//
+		// The discriminator is "was a host EVER bound", NOT "has a report arrived". That is the
+		// case this check must not break: a box that was installed, bound, and then went silent
+		// has a real fault and must keep alarming. It is bound, so it is judged.
+		//
+		// WHY THIS WAS REACHABLE AT ALL, measured 2026-08-04: the down-skip above is what
+		// protects every other silent customer, and it reads the staleness checker's state — which
+		// is seeded from the `reports` table (store.GetCustomers). A customer that has NEVER
+		// reported appears in no report row, so it gets no staleness state at all and GetState()
+		// returns "" rather than "down". The skip misses exactly the customer it would most
+		// obviously cover, and the DB-dump half below then fires every night: `david`, a
+		// prospective customer whose record was created 2026-08-01 with no machine ever bound,
+		// e-mailed an expected_dbdump_missed ERROR at 03:00 UTC on three consecutive days.
+		//
+		// Fail-open on a read error: an unreadable binding must never SUPPRESS a real alarm.
+		if bound, berr := s.HasEverBoundHost(id); berr != nil {
+			logger.Printf("[WARN] Deadline check: failed to read host binding for %s (judging anyway): %v", id, berr)
+		} else if !bound {
+			// Visible, per the v0.73.0 Part-7 precedent below: a quiet check must never be
+			// indistinguishable from a check that did not run. Once daily, one line per customer.
+			logger.Printf("[INFO] Deadline check: %s has no host EVER bound — all deadline verdicts UNKNOWN (no alarm)", id)
+			unbound++
+			continue
+		}
+
 		// Backup freshness from the agent's host-report (PBS snapshots + vzdump),
 		// the authoritative offsite-backup signal post-slice-8C.
 		reportJSON, rerr := s.GetLatestHostReportJSON(id)
@@ -423,6 +454,6 @@ func CheckBackupDeadlines(s *store.Store, staleness *StalenessChecker, onEvent E
 		}
 	}
 
-	logger.Printf("[INFO] Deadline check: %d customers, %d backup missed, %d backup unknown (deferred), %d dbdump missed, %d skipped (down)",
-		len(customerIDs), backupMissed, deferred, dbdumpMissed, skipped)
+	logger.Printf("[INFO] Deadline check: %d customers, %d backup missed, %d backup unknown (deferred), %d dbdump missed, %d skipped (down), %d unknown (no host ever bound)",
+		len(customerIDs), backupMissed, deferred, dbdumpMissed, skipped, unbound)
 }
diff --git a/hub/internal/monitor/deadline_unbound_test.go b/hub/internal/monitor/deadline_unbound_test.go
new file mode 100644
index 0000000..f52510d
--- /dev/null
+++ b/hub/internal/monitor/deadline_unbound_test.go
@@ -0,0 +1,148 @@
+package monitor
+
+import (
+	"io"
+	"log"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-hub/internal/store"
+)
+
+// R-195 — a customer with NO machine EVER bound must not alarm; a customer WITH one must.
+//
+// Origin, measured on the live hub 2026-08-04: `david` is a prospective customer whose record was
+// created 2026-08-01 16:51:49 with no host ever bound (hosts=0, host_deletions=0, host_reports=0,
+// reports=0). It e-mailed an `expected_dbdump_missed` ERROR at 03:00 UTC on 08-02, 08-03 and 08-04.
+//
+// The mechanism, established at source: the down-skip in CheckBackupDeadlines reads
+// StalenessChecker.GetState(), whose map is seeded from store.GetCustomers() — a query over the
+// `reports` table. A customer with zero reports is in no row, so it has no state, and GetState()
+// returns "" rather than "down". The skip that protects every other silent customer misses the one
+// that never reported at all.
+//
+// These tests pin BOTH halves. A suite that only proved the silence would pass against an
+// implementation that never alarms, which is strictly worse than the defect it replaces.
+
+// newUnboundStore creates a store holding ONE active customer and NO host row at all.
+func newUnboundStore(t *testing.T) *store.Store {
+	t.Helper()
+	st, err := store.New(filepath.Join(t.TempDir(), "test.db"), log.New(io.Discard, "", 0))
+	if err != nil {
+		t.Fatalf("store.New: %v", err)
+	}
+	t.Cleanup(func() { st.Close() })
+	if err := st.SaveCustomerConfig(&store.CustomerConfig{CustomerID: "c1", APIKey: "ck", RetrievalPassword: "p"}); err != nil {
+		t.Fatalf("SaveCustomerConfig: %v", err)
+	}
+	return st
+}
+
+// ── Half 1: the silence ────────────────────────────────────────────────────────────────────────
+
+// TestCheckBackupDeadlines_NeverBoundHost_Silent is the david case.
+//
+// COMPANION RED-PROOF (observed): deleting the HasEverBoundHost guard from CheckBackupDeadlines
+// makes this test fail with
+//
+//	deadline_unbound_test.go: a customer with NO host ever bound must raise NOTHING;
+//	got [expected_dbdump_missed]
+//
+// which is verbatim the event `david` e-mailed three mornings running. Restored after.
+func TestCheckBackupDeadlines_NeverBoundHost_Silent(t *testing.T) {
+	st := newUnboundStore(t)
+	// No UpsertHost, no host-report, no db_dump_completed event — nothing has ever been expected.
+	got := runDeadline(t, st)
+	if len(got) != 0 {
+		t.Fatalf("a customer with NO host ever bound must raise NOTHING; got %v", got)
+	}
+}
+
+// TestHasEverBoundHost pins the predicate itself across its three inputs, because the whole
+// behaviour above turns on it and a predicate that answered `false` for everything would make the
+// test above pass while silencing the entire fleet.
+func TestHasEverBoundHost(t *testing.T) {
+	st := newUnboundStore(t)
+
+	if bound, err := st.HasEverBoundHost("c1"); err != nil || bound {
+		t.Fatalf("no host rows → want (false,nil); got (%v,%v)", bound, err)
+	}
+
+	if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+		t.Fatalf("UpsertHost: %v", err)
+	}
+	if bound, err := st.HasEverBoundHost("c1"); err != nil || !bound {
+		t.Fatalf("live host row → want (true,nil); got (%v,%v)", bound, err)
+	}
+
+	// An unknown customer is never bound — the predicate must not answer from another customer's rows.
+	if bound, err := st.HasEverBoundHost("nobody"); err != nil || bound {
+		t.Fatalf("unknown customer → want (false,nil); got (%v,%v)", bound, err)
+	}
+}
+
+// ── Half 2: THE RED-PROOF THAT MATTERS — a bound machine must still alarm ───────────────────────
+
+// TestCheckBackupDeadlines_BoundButNeverReported_StillAlarms is the case the change could break,
+// and it is a real shape: a machine that was installed and bound and never phoned home. It has a
+// `hosts` row and zero reports — indistinguishable from `david` on every signal EXCEPT the one the
+// guard discriminates on. If the guard is ever "simplified" to key off report presence, customer
+// age or a name pattern, this test goes red.
+func TestCheckBackupDeadlines_BoundButNeverReported_StillAlarms(t *testing.T) {
+	st := newUnboundStore(t)
+	if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+		t.Fatalf("UpsertHost: %v", err)
+	}
+	// No host-report and no db_dump_completed — the box was bound and never said anything.
+	got := runDeadline(t, st)
+	if !has(got, "expected_dbdump_missed") {
+		t.Fatalf("a BOUND machine that never reported is a real fault and must still alarm; got %v", got)
+	}
+}
+
+// TestCheckBackupDeadlines_BoundThenWentQuiet_StillAlarms: the machine reported once, days ago,
+// and stopped. Nothing about the guard may suppress that.
+//
+// The staleness checker's down-skip is nil here (runDeadline passes nil), which is deliberate: it
+// isolates THIS guard. In production a genuinely down node is skipped by staleness and gets its own
+// node_down event — that path is unchanged and is not what this test is about.
+func TestCheckBackupDeadlines_BoundThenWentQuiet_StillAlarms(t *testing.T) {
+	st := newUnboundStore(t)
+	if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+		t.Fatalf("UpsertHost: %v", err)
+	}
+	// One host-report whose newest backup evidence is 9 days old, and a db dump that last
+	// completed 5 days ago (i.e. not since midnight).
+	report := hostReportJSON(t, [][2]string{{rfc(-9 * 24 * time.Hour), "ok"}}, nil)
+	if err := st.SaveHostReport("h1", "c1", []byte(report), store.HostReportDenorm{}); err != nil {
+		t.Fatalf("SaveHostReport: %v", err)
+	}
+	got := runDeadline(t, st)
+	if !has(got, "expected_backup_missed") {
+		t.Fatalf("a bound machine that went quiet with stale backups must still raise expected_backup_missed; got %v", got)
+	}
+	if !has(got, "expected_dbdump_missed") {
+		t.Fatalf("a bound machine that went quiet must still raise expected_dbdump_missed; got %v", got)
+	}
+}
+
+// TestCheckBackupDeadlines_DeletedHost_StillJudged: the host row is gone but a tombstone remains
+// (peti-felhom's live shape). The customer HAD a machine, so this check must not take over the
+// judgement — it hands off to the staleness down-skip exactly as before the change.
+func TestCheckBackupDeadlines_DeletedHost_StillJudged(t *testing.T) {
+	st := newUnboundStore(t)
+	if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+		t.Fatalf("UpsertHost: %v", err)
+	}
+	if err := st.DeleteHost("h1", false); err != nil {
+		t.Fatalf("DeleteHost: %v", err)
+	}
+	if bound, err := st.HasEverBoundHost("c1"); err != nil || !bound {
+		t.Fatalf("a DELETED host is still a machine that was once bound → want (true,nil); got (%v,%v)", bound, err)
+	}
+	got := runDeadline(t, st)
+	if !has(got, "expected_dbdump_missed") {
+		t.Fatalf("a customer whose host was deleted was still bound and stays judged here; got %v", got)
+	}
+}
diff --git a/hub/internal/store/store.go b/hub/internal/store/store.go
index 24bea1a..91c4d63 100644
--- a/hub/internal/store/store.go
+++ b/hub/internal/store/store.go
@@ -2262,6 +2262,28 @@ func (s *Store) GetHostByCustomer(customerID string) (*Host, error) {
 	return h, err
 }
 
+// HasEverBoundHost reports whether a machine was EVER bound to this customer — a live row in
+// `hosts` OR a tombstone in `host_deletions`. It answers "was anything ever expected of this
+// customer", which is the question the deadline verdicts actually need (R-195).
+//
+// It is deliberately NOT "has a report arrived", and the distinction is the whole point: a box
+// that was installed, bound, and then went silent IS bound, and its silence is a real fault that
+// must keep alarming. Only a customer that never had a machine at all is UNKNOWN.
+//
+// `host_deletions` is included because a customer whose host was removed HAD one — the deadline
+// caller reaches its down-skip for that shape, and this predicate must not quietly take over a
+// judgement the staleness checker owns.
+func (s *Store) HasEverBoundHost(customerID string) (bool, error) {
+	var n int
+	if err := s.db.QueryRow(
+		`SELECT EXISTS(SELECT 1 FROM hosts WHERE customer_id = ?)
+		     OR EXISTS(SELECT 1 FROM host_deletions WHERE customer_id = ?)`,
+		customerID, customerID).Scan(&n); err != nil {
+		return false, err
+	}
+	return n != 0, nil
+}
+
 // ListHostsByCustomer returns the customer's hosts ordered by host_id (v0.47.0 — the
 // customer page's Host tab is a LIST by design: 1 host today, N for a later HA cluster).
 // Uses the idx_hosts_customer index.