hub v0.91.0 — the staleness window learns each tier's own rhythm (R-86 Part 2)
gates / gates (push) Successful in 7s
gates / gates (push) Successful in 7s
Ships WITH agent v0.121.0, not after it. The agent now proves a tier once per
ARCHIVE GENERATION, so a weekly tier is proved weekly — in perfect health. The
flat 7-day restoreProvenStaleAfter derived its number from the 24h cadence R-86
removes, and a healthy weekly tier's proof age reaches EXACTLY 168h just before
its next proof: it sat ON the line, so any ordinary delay tipped it into a
nightly alarm about a working system.
restoreProvenWindow(tier, observed, ok):
- the tier's own archive interval, OBSERVED from reports the hub already holds
(pbs_snapshots + successful backups attributed by TARGET TYPE, slice A.4)
- x4 generations = the same tolerance the flat constant expressed
- floored at 7d (never tighter than before), capped at 12d (strictly inside the
2-week offsite retention)
- falls back to the DECLARED rhythm (26h host / 8d offsite — the thresholds the
backup-freshness checker already uses) when history is too short to observe
one; falling back to the FLOOR would recreate the false alarm on a fresh box
Kept: absence is UNKNOWN until the anchored window passes; the signal stays
edge-triggered; failed and stale remain distinct events. Every reason string now
states the window it was judged against (R-100's corollary).
Also backfills the missing v0.90.1 CHANGELOG entry (deployed since f21e7ca), and
records the operator's 2026-08-03 ruling that ep0 is Tier 2 / protected.
This commit is contained in:
@@ -29,20 +29,23 @@ prohibition as covering the act it names and nothing more.
|
||||
|---|---|---|
|
||||
| **0 — disposable. Reach here first.** | Exists to be broken; reinstalling is a routine afternoon, not an incident. **A drill that needs a victim uses one of these.** | `demo-hp` (t740), `demo-felhom` (N100) |
|
||||
| **1 — create and destroy freely** | Throwaway VMs, guests, scratch customers — **hosted on a Tier 0 machine** | drill VMs, scratch guests |
|
||||
| **2 — protected. Never a drill target.** | Losing it costs the recovery chain or a real relationship | **DooPlex**, **Peti's cluster** — and, by D-d, **nothing else** |
|
||||
| **2 — protected. Never a drill target.** | Losing it costs the recovery chain or a real relationship | **DooPlex**, **Peti's cluster**, **`ep0`** (operator ruling 2026-08-03) — and nothing else |
|
||||
|
||||
**DooPlex is Tier 2 because it *is* the recovery chain** — hub, Gitea, registry, PBS, k3s + Longhorn.
|
||||
Everything else rebuilds from it; it rebuilds from nothing. A bad moment in a DR drill there costs the
|
||||
thing under test, the source of truth for it, and the backups, at once.
|
||||
|
||||
**`ep0` + the Hetzner Storage Boxes were Tier 2 until 2026-08-02 and are no longer** — D-d's protected
|
||||
list names two machines and ep0 is not one of them. **That does not make them scratch, and the
|
||||
difference is an act, not a tier** (see the rule above the table): ep0 holds the **PBS-DR datastore and
|
||||
the restic copy of a real customer's data**, which is the only off-premises copy that exists, so
|
||||
*deleting datastores, prune jobs, tunnel config or nftables rules* remains forbidden by what it would
|
||||
destroy rather than by what tier it sits in. Reads are fine; it is still never a drill target.
|
||||
**Flagged for the operator: D-d did not name ep0 either way.** Confirm it explicitly — this page has
|
||||
read it the narrow way (not protected, but not wipeable) rather than assume the broad one.
|
||||
**`ep0` is Tier 2 — PROTECTED. Operator ruling, 2026-08-03.** D-d named two protected machines and did
|
||||
not name ep0 either way, so this page carried the question in writing for two days and read it the
|
||||
narrow way meanwhile (not protected, but not wipeable). The ruling settles it and **extends D-d's
|
||||
protected list to three machines**: DooPlex, Peti's cluster, ep0.
|
||||
|
||||
The reason it was never really in doubt: ep0 holds the **PBS-DR datastore and the restic copy of a
|
||||
real customer's data**, which is the only off-premises copy that exists. So *deleting datastores,
|
||||
prune jobs, tunnel config or nftables rules* was already forbidden by what it would destroy; the
|
||||
ruling makes the classification say so plainly instead of leaving each session to re-derive it.
|
||||
**Reads are fine** — including the ordinary off-site read a restore-test performs (R-86) — and it is
|
||||
never a drill target. The Hetzner Storage Boxes ride the same reasoning.
|
||||
|
||||
**Standing ruling, 2026-07-25 (`operations/nodes.md`):** drill and build VMs live on the **t740** — not
|
||||
felhom-pve, and **moved off DooPlex**. This page exists because that ruling sat where no session reads.
|
||||
@@ -95,12 +98,15 @@ still shares a device with its guest, so a drive failure is **offsite-only recov
|
||||
migrated, parked until the tester reinstalls (`PETI` in `backlog/OPEN-ITEMS.md`). Currently DOWN, no
|
||||
enrolled host. No access route from DooPlex, and nothing here needs one.
|
||||
|
||||
### `ep0` (`felhom-hetzner`, `ep0.felhom.eu`) + the Hetzner Storage Boxes — **not protected by D-d; not scratch either**
|
||||
### `ep0` (`felhom-hetzner`, `ep0.felhom.eu`) + the Hetzner Storage Boxes — **Tier 2, PROTECTED** (operator ruling 2026-08-03)
|
||||
|
||||
Reads are fine. It is the **offsite of last resort** (PBS-DR datastore, WireGuard hub, operator OOB
|
||||
path) and — until 2026-08-03 — RAM-constrained (3.8 GB, R-90); it is now a **CX33 with 8 GB RAM and a
|
||||
4 GiB swapfile**, which is what closed R-90. A very large restore is still worth watching. Do not delete datastores, prune
|
||||
jobs, tunnel config or nftables rules; never a drill target. The Storage Boxes hold the restic copy —
|
||||
4 GiB swapfile**, which is what closed R-90. A very large restore is still worth watching — the 8 GB
|
||||
is comfortable, not unbounded, and the OOM that started R-90 was a 14.46 GB restore read against
|
||||
3.8 GB. Do not delete datastores, prune jobs, tunnel config or nftables rules; never a drill target.
|
||||
**The ordinary off-site READ a restore-test performs is permitted and unchanged by the ruling**
|
||||
(R-86): the classification forbids destruction, not use. The Storage Boxes hold the restic copy —
|
||||
customer documents and photos, on a credential that can still delete (R-95).
|
||||
**Access: `ssh root@167.233.158.164` from DooPlex** — *not* `felhom-pve → 10.77.0.1`, the route that
|
||||
produced a false "unreachable" verdict (standing rule 2).
|
||||
|
||||
Reference in New Issue
Block a user