R-195: a customer with no machine ever bound does not alarm (hub v0.92.0) + R-193/R-192 spike
gates / gates (push) Successful in 7s
gates / gates (push) Successful in 7s
Part 4 (ships): `david` — a prospective customer with hosts=0, host_deletions=0, reports=0 — e-mailed an expected_dbdump_missed ERROR at 03:00 UTC three mornings running. The existing down-skip could never cover it: it reads the staleness checker's state, which is seeded from a query over the `reports` table, so a customer that never reported has no state at all and GetState() returns "" rather than "down". store.HasEverBoundHost (hosts row OR host_deletions tombstone) is consulted once per customer at the top of the deadline loop. The discriminator is "was a host EVER bound", never "has a report arrived" — a box installed and never heard from is a real fault and keeps alarming. Fail-OPEN on a read error. Red-proof observed: removing the guard fails with `got [expected_dbdump_missed]`, verbatim the event david sent. Parts 0-3 (spike, NO production code for R-193/R-192): audits/SPIKE-offsite-credential-recovery-2026-08-04.md establishes that the one-shot provider password is the RECOVERABLE secret and the restic repository password is the irreplaceable one — and that a guest rebuild mints a fresh one, orphaning the previous off-site history. Measured without touching a box, by comparing host_escrow.restic_pw_sha256 against host_escrow_superseded: BOTH demo boxes changed (demo-hp 15 snapshots / 40.9 MB, demo-felhom 36 snapshots / 1.14 GB). demo-felhom's "lucky" 76-second recovery restored delivery and not the repository, silently, for 13h. ReissueCredentials does NOT rotate the restic password (R-39's record and two hub comments are wrong -> R-196); candidate (b) is not implementable against a zero-knowledge escrow; candidate (a) already exists as F3 and is wired to the wrong event. Ends in ranked options and an unanswered question for the operator. R-195 SHIPPED; R-196 + R-197 filed; R-192 + R-193 updated, neither closed.
This commit is contained in:
+58
@@ -17,6 +17,64 @@
|
||||
|
||||
## Standing rulings
|
||||
|
||||
**S-27 — a customer with NO machine ever bound is UNKNOWN, silently; one that was bound and went quiet
|
||||
still alarms (2026-08-04, R-195; hub v0.92.0).** Operator ruling, implemented as
|
||||
`store.HasEverBoundHost` (live `hosts` row OR `host_deletions` tombstone) consulted once at the top of
|
||||
`CheckBackupDeadlines`' loop. **The discriminator is "was a host EVER bound", never "has a report
|
||||
arrived"** — a box that was installed and never phoned home is a real fault. Fail-**open** on a read
|
||||
error; the deferral is logged with its own counter; the R-81 anchored-verdict structure is untouched.
|
||||
|
||||
*The mechanism is worth carrying, because the guard that should have covered this was keyed off the
|
||||
wrong thing:* the existing down-skip reads `StalenessChecker.GetState()`, whose map is seeded from
|
||||
`store.GetCustomers()` — **a query over the `reports` table**. A customer with zero reports is in no
|
||||
row, gets no state, and `GetState()` returns `""` not `"down"`, so **the skip misses exactly the
|
||||
customer it would most obviously cover.** `david` (created 2026-08-01, no machine) e-mailed an
|
||||
`expected_dbdump_missed` ERROR three mornings running; `peti-felhom` — active, host deleted
|
||||
2026-07-15 — does not, because its 482 old reports make it `down`. Generalise it: **a "skip the dead"
|
||||
guard built on evidence of life cannot see something that was never alive.**
|
||||
|
||||
**S-26 — the one-shot secret is the recoverable one; the irreplaceable one is minted fresh on every
|
||||
guest rebuild (2026-08-04, R-193 spike — `audits/SPIKE-offsite-credential-recovery-2026-08-04.md`).
|
||||
No code shipped for it; the decision is the operator's.**
|
||||
|
||||
Two sentences, because they are what every future session needs before touching this area:
|
||||
|
||||
- **Q1 — what is one-shot.** Of the three secrets in the offsite path, the hub's **one-time
|
||||
provider password** is one-shot but freely re-issuable, the box's **SFTP key** is regenerated on
|
||||
every apply by design, and the **restic repository password** — the DATA key, which the agent's own
|
||||
source calls *"irreplaceable"* (`felhom-agent/internal/escrow/identity.go:35-39`) — is the only one
|
||||
nothing can restage.
|
||||
- **Q2 — what a rebuild costs.** A rebuilt controller **mints a brand-new repository password**
|
||||
(`WriteOffboxSecrets`, `offbox.go:392`, generates whenever `<DataDir>/offbox/repo_password` is
|
||||
absent) and **no automatic path ever consults the escrowed one** — `InjectOffboxPassword` has exactly
|
||||
one caller in the whole repo, a web form a human pastes into — so **every guest rebuild orphans the
|
||||
previous off-site history.**
|
||||
|
||||
*Measured without touching a box, and the method is reusable:* `host_escrow.restic_pw_sha256` vs
|
||||
`host_escrow_superseded.restic_pw_sha256`. demo-hp `8e03eddf…`→`8a9e33aa…` (15 snapshots / 40.9 MB
|
||||
orphaned); demo-felhom `48741892…`→`c60c8bc7…` (**36 snapshots / 1.14 GB**). **demo-felhom is the half
|
||||
that matters:** R-193 recorded it as having survived the rebuild by luck, and it did — its *delivery*
|
||||
recovered in 76 s off a stale staged secret. **Its repository did not**, and nothing said so for 13 h.
|
||||
*Luck restored the plumbing, not the data.*
|
||||
|
||||
*Three corrections that must not be re-inherited.* **(a)** `ReissueCredentials` does **not** rotate the
|
||||
restic password — R-39's record, `offsite.go:198-201` and `api/handler.go:1067-1069` all say it does,
|
||||
and all three are wrong (→ **R-196**, the eighth entry in `CLAUDE.md`'s table). **(b)** Candidate (b),
|
||||
*"recoverable from escrow at re-bootstrap"*, is **not implementable** — the escrow is R-wrapped and
|
||||
zero-knowledge (D6), so only a customer-present ceremony can open it, which is the manual form that
|
||||
already exists. **(c)** Candidate (a), *"the hub restages automatically"*, **already exists** as
|
||||
`reissueOnReenroll`'s F3 leg and is wired to the wrong event — it sits behind `handleHostEnroll`'s
|
||||
mint-once-reuse short-circuit, and a **guest** rebuild leaves the `hosts` row intact. **Shipping it
|
||||
first would have made both boxes look healthy on 2026-08-04 while their snapshots were orphaned** —
|
||||
strictly worse than the current loud failure.
|
||||
|
||||
*The candidate nobody had named,* and the only one aimed at the actual harm: **the agent survives a
|
||||
guest rebuild**, already receives the repo password over the pinned local API
|
||||
(`POST /escrow/stage-secret`) and already writes it to a fixed 0600 path — it merely **wipes** it after
|
||||
the ceremony. Retaining and serving it back needs no new seam. Its price is one real trade: a copy of
|
||||
the data key at rest on the Proxmox host. **That trade is the operator's to make and the spike does not
|
||||
make it.**
|
||||
|
||||
**S-24 — offsite retention is ep0's, and the box asks for none (2026-08-04, R-191; installer 1.25.0).**
|
||||
R-89 moved offsite pruning server-side and box tokens stay write-only. The 2026-07-26 "two weeks"
|
||||
ruling was not reversed — **where it is ENFORCED moved, and the installer's `keep_last: 2` did not
|
||||
|
||||
Reference in New Issue
Block a user