hub v0.56.0: PBS-DR self-heal reconciler (re-stage a consumable secret)

Implements SPIKE-pbsdr-selfheal-2026-07-15 (e8f8c44). A box re-installed/rolled
back onto its stable host_id loses its agent-side converged marker; the hub
keeps the enabled descriptor + a CONSUMED one-time secret, the WG peer persists
(changed==false, cascade can't re-fire), so the agent sits in waiting_secret
forever. The missing piece is a consumable secret, not the descriptor.

New internal/pbsdrheal reconciler (5m, wgsync shape): for enabled+provisioned
hosts whose latest report pbs_dr.state is a stuck state past a >=2-distinct-report
debounce, re-stage the stored secret (store.RestageHostPBSSecret: clear
consumed_at, no ep0 call, NO generation bump); escalate to Re-issue (web
ReissuePBSDR) only when no secret is stored or the agent reports consumed_failed.
Converged/disabled/verify_failed/DR-OFF = no-op. PBSDRHEAL_ONLY_HOST scopes a
supervised rollout. Scenarios A-F + all six red-proofs verified. No agent change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HEPuEwyyGDJdcsXLFsTWJn

hub/internal/web/pbsdr.go

@@ -299,6 +299,54 @@ func (s *Server) PBSDRAutoProvision(ctx context.Context, customerID string) {
 	s.logger.Printf("[INFO] pbsdr auto-provisioned for %s on WG registration (hands-free cascade)", customerID)
 }
 
+// ReissuePBSDR re-keys the customer's ep0 PBS token and re-arms the agent — the non-HTTP core shared
+// by the operator button (handlePBSDRReissue) and the pbsdrheal self-heal reconciler (the escalation
+// path when no stored secret is re-stageable, or the agent burned one and reports consumed_failed).
+// tenantsync reissue → fresh consume-once secret (SaveHostPBSSecret) → descriptor refresh with the
+// NEW token_id/fingerprint + generation bump (the agent's re-consume signal). This reuses the same
+// reissue op the handler does — it is NOT a re-run of pbsdrProvisionAtom (which refuses ErrTokenExists
+// and would not re-key). The secret value is never logged. Keep this in lockstep with the tail of
+// handlePBSDRReissue.
+func (s *Server) ReissuePBSDR(ctx context.Context, customerID string) error {
+	if s.tenantsync == nil {
+		return fmt.Errorf("pbsdr: provisioning not configured on this hub")
+	}
+	host, err := s.store.GetHostByCustomer(customerID)
+	if err != nil {
+		return fmt.Errorf("pbsdr reissue: host lookup: %w", err)
+	}
+	if host == nil {
+		return fmt.Errorf("pbsdr reissue: no host enrolled for %s", customerID)
+	}
+	cur := readPBSDR(host.DesiredJSON)
+	if cur == nil || cur.Namespace == "" {
+		return fmt.Errorf("pbsdr reissue: no provisioned PBS DR tier for %s", customerID)
+	}
+	// Same detached-ctx discipline as applyPBSDR: reissue→store→bump must complete atomically.
+	rctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 2*time.Minute)
+	defer cancel()
+	res, err := s.tenantsync.Reissue(rctx, customerID)
+	if err != nil {
+		return fmt.Errorf("pbsdr reissue for %s: %w", customerID, err)
+	}
+	if err := s.store.SaveHostPBSSecret(host.HostID, res.TokenSecret); err != nil {
+		return fmt.Errorf("pbsdr reissue for %s: store secret: %w", customerID, err)
+	}
+	cur.TokenID = res.TokenID
+	cur.Fingerprint = res.Fingerprint
+	cur.Datastore = res.Datastore
+	cur.Namespace = res.Namespace
+	merged, err := mergePBSDR(host.DesiredJSON, cur)
+	if err != nil {
+		return fmt.Errorf("pbsdr reissue for %s: merge descriptor: %w", customerID, err)
+	}
+	if _, err := s.store.SetHostDesired(host.HostID, []byte(merged)); err != nil {
+		return fmt.Errorf("pbsdr reissue for %s: descriptor bump: %w", customerID, err)
+	}
+	s.logger.Printf("[INFO] pbsdr credentials re-issued for %s (host %s; fresh consume-once secret stored, withheld from logs)", customerID, host.HostID)
+	return nil
+}
+
 // handlePBSDRReissue explicitly re-keys the customer's ep0 PBS token (the offsite F4 precedent):
 // tenantsync reissue → fresh consume-once secret → descriptor refresh + generation bump so the
 // agent re-runs its bridge and consumes the fresh secret. The secret value is never logged.