admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

hub v0.56.0: PBS-DR self-heal reconciler (re-stage a consumable secret)

Browse Source 
Implements SPIKE-pbsdr-selfheal-2026-07-15 (e8f8c44). A box re-installed/rolled
back onto its stable host_id loses its agent-side converged marker; the hub
keeps the enabled descriptor + a CONSUMED one-time secret, the WG peer persists
(changed==false, cascade can't re-fire), so the agent sits in waiting_secret
forever. The missing piece is a consumable secret, not the descriptor.

New internal/pbsdrheal reconciler (5m, wgsync shape): for enabled+provisioned
hosts whose latest report pbs_dr.state is a stuck state past a >=2-distinct-report
debounce, re-stage the stored secret (store.RestageHostPBSSecret: clear
consumed_at, no ep0 call, NO generation bump); escalate to Re-issue (web
ReissuePBSDR) only when no secret is stored or the agent reports consumed_failed.
Converged/disabled/verify_failed/DR-OFF = no-op. PBSDRHEAL_ONLY_HOST scopes a
supervised rollout. Scenarios A-F + all six red-proofs verified. No agent change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HEPuEwyyGDJdcsXLFsTWJn
This commit is contained in:
authentik Default Admin
2026-07-15 18:27:39 +02:00
parent e8f8c441fa
commit 6218e7919d
9 changed files with 878 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hub/cmd/hub/main.go
+18
View File
@@ -20,6 +20,7 @@ import (
"gitea.dooplex.hu/admin/felhom-hub/internal/hetznerapi"
"gitea.dooplex.hu/admin/felhom-hub/internal/mailrelay"
"gitea.dooplex.hu/admin/felhom-hub/internal/offsite"
"gitea.dooplex.hu/admin/felhom-hub/internal/pbsdrheal"
"gitea.dooplex.hu/admin/felhom-hub/internal/monitor"
"gitea.dooplex.hu/admin/felhom-hub/internal/notify"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
@@ -410,6 +411,23 @@ func main() {
}
}
// PBS-DR self-heal reconciler (internal/pbsdrheal, from SPIKE-pbsdr-selfheal-2026-07-15). Re-arms
// a consumable secret for boxes stuck in waiting_secret/consumed_failed after losing their
// converged marker (re-install / restore / snapshot rollback onto a stable host_id — the case no
// existing path recovers). The primary heal (re-stage the stored secret) needs no endpoint; the
// escalation reuses webServer.ReissuePBSDR (a no-op-error if tenantsync is unconfigured). Reads
// the hub DB only; a converged/disabled host is a pure no-op. PBSDRHEAL_ONLY_HOST scopes a
// supervised first rollout to one host (empty = whole fleet — the steady state).
{
pbsdrReconciler := pbsdrheal.NewReconciler(dataStore, pbsdrheal.NewActions(dataStore, webServer), logger)
if only := os.Getenv("PBSDRHEAL_ONLY_HOST"); only != "" {
pbsdrReconciler.RestrictToHost(only)
logger.Printf("[INFO] PBS-DR self-heal reconciler RESTRICTED to host %s (supervised rollout scope)", only)
}
go pbsdrReconciler.Run(ctx)
logger.Printf("[INFO] PBS-DR self-heal reconciler started (interval 5m)")
}
// Session cleanup — removes expired sessions every hour
go webServer.CleanupSessions(ctx)