controller v0.71.0: fix guest-reboot recovery (boot-race + agent-path blocker)
Live diagnosis of drive-backed apps stuck Exited after a pct reboot pinned THREE sub-causes, fixed together (hardening the existing processGuestBootChange, not a parallel mechanism): 1. Agent-path blocker (live root cause): agentClient() returned "agent not configured" (cfg.LocalAPI.Endpoint empty), so processGuestBootChange AND the whole drive gate bailed at the first guard. bootstrap.json had a complete local_api block, but MaybeIngest returned immediately on "already configured" so a controller.yaml seeded before local_api existed never got the agent path. Fix: MaybeIngest now calls ensureLocalAPI on the already-configured path, merging local_api from bootstrap.json into the existing controller.yaml when missing (no hub re-pull, config preserved; idempotent + fail-safe). 2. Boot-race readiness gate: processGuestBootChange sampled BoundUnderParent once during fast startup, racing the ~18s rebind, recreated nothing, burned its boot-id one-shot. Fix: gate on the REAL live in-guest bind -- driveBindLive checks /mnt/felhom-drives/<drive> is a mountpoint in the controller's own /mnt rslave /proc/self/mountinfo; pollLiveBinds waits for it (bounded ~120s) before recreating via the normal pipeline. shouldRecreateOnBoot stays state-independent so stuck-Exited create-time-failure apps are included. 3. Single-shot fragility: processGuestBootChange ran only once at startup; a briefly-unreachable agent right after a guest reboot stranded recovery. Fix: driveGateLoop runs it every periodic tick too (idempotent, boot-id gated). Tests (non-hollow, pre-fix companions, red-proofed): pollLiveBinds waits then reports live / never-live stays absent / single early sample misses; ensureLocalAPI merges local_api into a configured controller.yaml that lacks it / no-ops when present. Live-accepted with repeated pct reboot 9201. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -296,6 +296,12 @@ func (s *Server) driveGateLoop() {
|
||||
t := time.NewTicker(30 * time.Second)
|
||||
defer t.Stop()
|
||||
for range t.C {
|
||||
// Re-run the boot-change converger every tick (not just once at startup): right after a GUEST
|
||||
// reboot the agent's local API may briefly be unreachable / its per-guest token stale, so the
|
||||
// single startup attempt can bail before reading the boot-id. It is idempotent (boot-id gated:
|
||||
// a no-op once the current boot has been converged), so retrying until the agent is reachable
|
||||
// is safe and is what makes guest-reboot recovery robust.
|
||||
s.processGuestBootChange()
|
||||
s.ReconcileDriveGates()
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user