hub v0.56.0: PBS-DR self-heal reconciler (re-stage a consumable secret)
Implements SPIKE-pbsdr-selfheal-2026-07-15 (e8f8c44). A box re-installed/rolled
back onto its stable host_id loses its agent-side converged marker; the hub
keeps the enabled descriptor + a CONSUMED one-time secret, the WG peer persists
(changed==false, cascade can't re-fire), so the agent sits in waiting_secret
forever. The missing piece is a consumable secret, not the descriptor.
New internal/pbsdrheal reconciler (5m, wgsync shape): for enabled+provisioned
hosts whose latest report pbs_dr.state is a stuck state past a >=2-distinct-report
debounce, re-stage the stored secret (store.RestageHostPBSSecret: clear
consumed_at, no ep0 call, NO generation bump); escalate to Re-issue (web
ReissuePBSDR) only when no secret is stored or the agent reports consumed_failed.
Converged/disabled/verify_failed/DR-OFF = no-op. PBSDRHEAL_ONLY_HOST scopes a
supervised rollout. Scenarios A-F + all six red-proofs verified. No agent change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HEPuEwyyGDJdcsXLFsTWJn
This commit is contained in:
@@ -94,6 +94,9 @@
|
||||
| `(*Server).applyPBSDR` + `mergePBSDR`/`readPBSDR` | hub/internal/web/pbsdr.go | `(ctx, r, cfg) error` | The config form's DR-tier section → HOST desired_json `pbs_dr` descriptor + generation bump | Descriptor lives in the host desired_json, NOT ConfigJSON (buildConfigJSON drops foreign keys on re-save). v0.51.0: driven by `cfg.DRTier` (set from the form BEFORE applyOffsite/applyPBSDR); UNMET preconditions are honest waiting stages (save succeeds), REAL failures stay fail-closed; already-provisioned = success-no-op (red-proofed); disable keeps the ep0 tenancy. |
|
||||
| `(*Server).pbsdrProvisionAtom` + `PBSDRAutoProvision` | hub/internal/web/pbsdr.go | `(ctx, customerID, host, storageID) (blocked string, err error)` / `(ctx, customerID)` | The shared fresh-provision cascade atom; the WG-registration hook target (api `SetWGRegisteredHook`, wired in hub/cmd/hub/main.go when tenantsync is on) | `blocked != ""` = waiting stage (never an error); the hook runs in a detached goroutine and must never fail registration. Scenario-A e2e test: TestPBSDR_AutoProvisionOnWGRegistration. |
|
||||
| `cfg.DRTier` + offsite coupling | hub/internal/store/store.go (CustomerConfig), hub/internal/web/configs.go (applyOffsite guard) | bool | Per-customer DR-tier flag: new-customer default ON (handleConfigNewForm); offsite REFUSED without it (exact F-6 message) | One-time migration backfill initializes legacy rows from descriptor reality — never re-runs (opt-outs survive re-open; store test pins it). Form field `dr_tier` (formBool helper). |
|
||||
| `pbsdrheal.Reconciler` + `NewActions` | hub/internal/pbsdrheal/reconciler.go | `NewReconciler(st, act, logger)` · `RestrictToHost(hostID)` · `Run(ctx)`/`Trigger()` | PBS-DR SELF-HEAL: re-arms a consumable secret for a box stuck in `waiting_secret`/`consumed_failed` after losing its converged marker (re-install/rollback). From `SPIKE-pbsdr-selfheal-2026-07-15`. | Primary heal = **re-stage** the stored secret (no ep0 call, **NO generation bump** — a bump = agent refetch loop). Escalate to Re-issue only when no stored secret / `consumed_failed`. NEVER re-run `pbsdrProvisionAtom` (refuses `ErrTokenExists`) or blind-timer Re-issue (hash/gen thrash). Converged/`disabled`/`verify_failed`/DR-OFF = no-op. Debounce ≥2 distinct reports. `PBSDRHEAL_ONLY_HOST` scopes a supervised rollout. Fake seam: `fakeActions` in reconciler_test.go. |
|
||||
| `(*Store).RestageHostPBSSecret` + `PBSDRHealStates` | hub/internal/store/pbsdr.go | `(hostID) (restaged bool, err)` / `() ([]PBSDRHealRow, error)` | The self-heal store primitives: clear `consumed_at` IFF a row exists (re-arm the SAME value); the fleet work-set query (descriptor enable/provision + latest report `pbs_dr.state` + id) | `RestageHostPBSSecret` does NO insert, NO value change, NO generation bump (`restaged=false` = no row → caller escalates). `PBSDRHealStates` mirrors `GetHostOOBStates`' latest-report-per-host join; malformed JSON → zero values, never an error. |
|
||||
| `(*Server).ReissuePBSDR` | hub/internal/web/pbsdr.go | `(ctx, customerID) error` | The non-HTTP core of the operator Re-issue button — the self-heal reconciler's escalation seam (satisfies `pbsdrheal.Reissuer`) | Reuses `tenantsync.Reissue` + `SaveHostPBSSecret` + descriptor bump — NOT a re-run of `pbsdrProvisionAtom`. Keep in lockstep with the tail of `handlePBSDRReissue` (which is unchanged; the operator button's 303/400 codes are pinned by tests). |
|
||||
| `parseHostCapabilities` + `capabilityView` | hub/internal/web/hosts.go | `(reportJSON) []capabilityView` | Host-page capability chips (ok/degraded/inactive) | `inactive` (agent v0.86.0) = badge-neutral, NEVER warn/error — disabled ≠ degraded; unknown future statuses fall to the degraded styling (surface, don't hide). `capabilitiesNeedDRMigration` keys the pre-v1.15.0 migration hint on pbsdr-* + "binary not found". |
|
||||
|
||||
### Config generation & secrets hygiene (hub/internal/configgen/)
|
||||
|
||||
Reference in New Issue
Block a user