hub v0.56.0: PBS-DR self-heal reconciler (re-stage a consumable secret)
Implements SPIKE-pbsdr-selfheal-2026-07-15 (e8f8c44). A box re-installed/rolled
back onto its stable host_id loses its agent-side converged marker; the hub
keeps the enabled descriptor + a CONSUMED one-time secret, the WG peer persists
(changed==false, cascade can't re-fire), so the agent sits in waiting_secret
forever. The missing piece is a consumable secret, not the descriptor.
New internal/pbsdrheal reconciler (5m, wgsync shape): for enabled+provisioned
hosts whose latest report pbs_dr.state is a stuck state past a >=2-distinct-report
debounce, re-stage the stored secret (store.RestageHostPBSSecret: clear
consumed_at, no ep0 call, NO generation bump); escalate to Re-issue (web
ReissuePBSDR) only when no secret is stored or the agent reports consumed_failed.
Converged/disabled/verify_failed/DR-OFF = no-op. PBSDRHEAL_ONLY_HOST scopes a
supervised rollout. Scenarios A-F + all six red-proofs verified. No agent change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HEPuEwyyGDJdcsXLFsTWJn
This commit is contained in:
@@ -1,5 +1,42 @@
|
||||
# Felhom Hub — Changelog
|
||||
|
||||
## v0.56.0 — PBS-DR self-heal reconciler (re-stage a consumable secret) (2026-07-15)
|
||||
|
||||
Implements `SPIKE-pbsdr-selfheal-2026-07-15` (`e8f8c44`). **⚠️ ARCHITECTURE IMPACT:** before this,
|
||||
there was **no automatic recovery** for the commonest real event — a customer box re-installed /
|
||||
restored / rolled back onto its **stable `host_id`**. The hub kept the durable `pbs_dr` descriptor
|
||||
(enabled) + a durable **consumed** one-time secret; the WG peer still existed (same pubkey →
|
||||
`changed==false`, so the provision cascade could not re-fire) and `applyPBSDR`'s "already
|
||||
provisioned → no-op" meant even a config re-save minted nothing. The agent sat in
|
||||
`pbs_dr.state="waiting_secret"` forever — PBS-DR never converged, so escrow could not run and offsite
|
||||
never armed. The spike proved (SQ-2b′) the **missing piece is a consumable secret, not the
|
||||
descriptor**: re-staging the stored secret converged a stuck box in one ~30 s agent tick, using the
|
||||
existing ep0 token, **zero churn**. This reconciler closes that gap.
|
||||
|
||||
- **`internal/pbsdrheal/reconciler.go` (new):** a hub periodic reconciler (5 min, `wgsync` shape).
|
||||
For each host whose descriptor is **enabled + provisioned** and whose **latest report**
|
||||
`pbs_dr.state` is a stuck state held across a **debounce** (≥2 distinct reports — so a box briefly
|
||||
`waiting_secret` between provision and its first consume is not touched): `waiting_secret` →
|
||||
**re-stage the stored secret** (no ep0 call, **no generation bump**); no stored secret →
|
||||
**escalate to Re-issue**; `consumed_failed` → **escalate to Re-issue only** (a re-stage would
|
||||
re-feed a burned secret). A converged/`disabled`/`verify_failed`/unprovisioned/DR-OFF host is a
|
||||
**pure no-op**. Each heal emits a distinct audit event (`pbsdr_selfheal_restaged` /
|
||||
`_reissued` / `_consumed_failed`). Reads the hub DB only — never the box.
|
||||
- **`internal/store/pbsdr.go`:** `RestageHostPBSSecret(hostID) (bool, error)` — clears `consumed_at`
|
||||
IFF a row exists (no INSERT, no value change, **no generation bump**); `restaged=false` → the
|
||||
caller escalates. `PBSDRHealStates()` — one query joining each host's descriptor enable/provision
|
||||
flags to its latest report's `pbs_dr.state` + id (mirrors `GetHostOOBStates`).
|
||||
- **`internal/web/pbsdr.go`:** `ReissuePBSDR(ctx, customerID)` — the non-HTTP core of the operator
|
||||
Re-issue button (tenantsync reissue → fresh consume-once secret → descriptor refresh + gen bump),
|
||||
now the reconciler's escalation seam. The operator handler is unchanged.
|
||||
- **`cmd/hub/main.go`:** the reconciler is started unconditionally (the primary re-stage heal needs
|
||||
no endpoint). **`PBSDRHEAL_ONLY_HOST`** env scopes a supervised first rollout to one host
|
||||
(empty = whole fleet — the steady state).
|
||||
- **Tests:** `internal/pbsdrheal/reconciler_test.go` (Scenarios A–F + scope + no-re-heal, real store
|
||||
+ fake action seam) and `internal/store/pbsdr_test.go` (re-stage semantics + the **no-generation-
|
||||
bump** guard + `PBSDRHealStates` parsing). All six §10 red-proofs verified (mutation → FAIL →
|
||||
revert). No agent changes (the agent already self-heals once a secret is consumable).
|
||||
|
||||
## v0.55.0 — accept the offbox_enlarge_blocked event (Task 3a-fix delivery chain) (2026-07-15)
|
||||
|
||||
The controller (v0.134.1) sends an `offbox_enlarge_blocked` warning when an app's enlarged offsite
|
||||
|
||||
Reference in New Issue
Block a user