docs: R-86 closed and proven live; ep0 recorded as protected; R-185/186/187 filed
gates / gates (push) Successful in 8s

- OPEN-ITEMS: R-86 CLOSED with the trap in its own wording recorded (the literal
  reading is never true on a daily tier); R-87 re-ranked UP because R-86 built
  most of what it waited for; R-185 (the agent cannot list demo-felhom's host
  backup tier — a missing storage ACL, pre-existing), R-186 (a released binary's
  sha is not reproducible from its tag), R-187 (R-115's publish leg had never
  actually run) filed. R-184 was the highest ID in use.
- ROADMAP: R-86 collapsed, keeping the reasoning and correcting the shape the row
  itself proposed — which would have been the never-fires version.
- 07-backup-architecture: new contract section — restore-testing is per ARCHIVE
  GENERATION, with the trap and what did not change (S-1).
- 00-capability-map: the unattended restore-proof row upgraded to PROVEN-LIVE on
  the 635 s due-triggered offsite run, with the restart and teardown evidence.
- CONTEXT: S-17 (the rule, the trap, the config key, the hub's derivation) and
  S-18 (ep0 is Tier 2 — extends D-d's protected list to three machines).
  Numbered 17/18 because S-14 and S-15 were already duplicated in the file.
- STATUS: rewritten for the operator, trimmed back to one screen.
This commit is contained in:
2026-08-03 15:28:32 +02:00
parent ff2655cf19
commit c102832892
7 changed files with 214 additions and 187 deletions
@@ -132,6 +132,41 @@ executed** (`CAMPAIGN-8…:522`), the host-loss plan **executes nothing by const
(`felhom-agent/internal/dr/plan.go:1-4`), and **no host has ever been rebuilt as its former self**
(INV Part D1).
### Lane 2's restore-test is scheduled PER ARCHIVE GENERATION (R-86, 2026-08-03)
**[CONTRACT, changed 2026-08-03 — agent v0.121.0 + hub v0.91.0.]** The scheduled restore-test used to
fire on an interval started at daemon start. It no longer does. The rule is:
> Let **A** be the newest archive on a tier that has settled for at least the settle lag (24 h).
> The tier is **DUE** when **A** exists and **A has not already been proven**.
So a tier is proved **once per archive**, on its own archive, and the proof follows the backup rather
than the process's uptime:
| tier rhythm | what is proved, and when |
|---|---|
| daily (host tier) | yesterday's archive, once a day |
| weekly (offsite tier) | last week's archive, once a week |
| newborn (no archive yet) | nothing — **UNKNOWN, never a fault** |
**The trap in the obvious formulation, recorded so it is not reintroduced:** *"due when the newest
archive is ≥ 24 h old"* is never true on a **daily** tier — a new archive resets the newest-archive
age to zero long before it reaches the lag — so the literal reading silently switches restore-testing
off for the tier that matters most.
What survives unchanged: the restore-test itself (restore → boot → verify → destroy the scratch), its
journal and crash recovery, the scratch VMID band, the one-heavy-operation gate, proof credit only on
success, and oldest-proven ordering, which is now the tie-break **between due tiers**. A ticker
remains, but only as the **evaluation interval** (6 h by default, chosen from a measured cost: one
due-check is 18 ms on a local dir storage and 392 ms on the PBS tier over the WAN).
**The hub's half is not optional.** `restoreProvenStaleAfter` was a flat 7 days derived from the very
cadence this replaced, and a weekly tier proved weekly reaches a proof age of **exactly** one interval
just before its next proof — 168 h against a 168 h window. It sat ON the line, so any ordinary delay
tipped a healthy tier into a nightly alarm. The window is now per tier, from that tier's observed
archive interval, floored at the old 7 days, capped at 12 days (strictly inside the two-week offsite
retention), and falling back to the tier's declared rhythm when history is too short to observe one.
### Why the split is right, stated once
**[DESIGN]** A customer can reason about "my photos are gone". A customer cannot reason about
@@ -820,7 +855,7 @@ does **not** hold as written. → **R-108**
| ~~**R-108**~~ | ~~Network storage can host an app's namespace~~ | **CLOSED 2026-07-30, controller v0.187.0 — D5 UNBLOCKED.** An app namespace may no longer be placed on network storage (5 surfaces guarded by one fail-closed predicate); the share-root bind is deliberately UNCHANGED because it is load-bearing and unscopable (§10.1). `audits/R108-network-app-namespace-2026-07-30.md` |
| **R-126** | A `.fab` bundle — plaintext secrets, optional password — can be exported ONTO a NAS: `storageDriveList()` (`internal/web/handler_export.go`) does not filter network paths | split out of R-108, which closed without it. NOT a D5 precondition: an explicit customer-chosen export destination, not a browsing surface reaching a backup tree (§5, §7.3) |
| R-95 (open) | The restic offsite credential **can delete** — the box can `forget --prune` its own repo | the tier holding the customer's documents and photos is the one whose credential can destroy it (matrix row 10) |
| R-86 (open) | Restore-tests are interval-scheduled, not backup-aligned | a tier's proof cadence is unrelated to when its archives are written |
| ~~R-86~~ | ~~Restore-tests are interval-scheduled, not backup-aligned~~ | **CLOSED 2026-08-03 — agent v0.121.0 + hub v0.91.0.** Restore-testing is now **per archive generation**: a tier is due when its newest archive that has settled ~24 h has not been proven, so a daily tier is proved daily on its own archive and a weekly tier weekly on its own. The ticker survives only as the evaluation interval (6 h, chosen from a measured cost). The hub's staleness window moved with it — per tier, from that tier's observed archive rhythm — because a weekly tier proved weekly sat EXACTLY on the old flat 7-day line (§3, Lane 2's per-archive rule) |
| R-87 (open) | The restic tier is never restore-tested | matrix row 4's route has no unattended proof |
### 10.3 Divergences that are documented elsewhere and are not re-opened here
@@ -886,7 +921,7 @@ to now *implement* D5 remains an open scheduling decision, not a blocked one.
| whole-guest restore, local and PBS, exact mount parity | **PROVEN-LIVE** | CAMPAIGN-2 T-P9; CAMPAIGN-8 Phase C |
| corrupted PBS snapshot fails cleanly | **PROVEN-LIVE** | CAMPAIGN-8 fault 17 |
| the box cannot delete its own **PBS** snapshots | **PROVEN-LIVE** | CAMPAIGN-8, R-89 |
| unattended restore-test across tiers | **IMPLEMENTED** (rotation not observed across consecutive cadences) | `00-capability-map.md:41`; LIVE per-tier timestamps this session |
| unattended restore-test across tiers | **IMPLEMENTED**; **per-archive due-ness PROVEN-LIVE 2026-08-03** (agent v0.121.0) | `00-capability-map.md:41`; the due verdict + a real offsite run on demo-felhom (§3, Lane 2's per-archive rule) |
| guest-power watchdog | **PROVEN-LIVE** | agent v0.107.0, 120 s |
| quiesce crash recovery | **PROVEN-LIVE** | CAMPAIGN-8 fault 10, 1 s, by SIGKILL |
| break-glass | **PROVEN-LIVE** | `runbooks/break-glass.md` |