controller v0.71.0: fix guest-reboot boot-race stranding drive-backed apps
Sub-cause: on guest pct reboot, in-guest dockerd auto-starts unless-stopped drive-backed apps ~18s BEFORE the agent re-binds the drive; the create-time volume bind fails (mkdir /mnt/felhom-drives/<drive>/userdata: permission denied) and RestartCount=0 means it's never retried -> stuck Exited. The existing recovery (processGuestBootChange) RAN but raced the rebind: it sampled the agent's BoundUnderParent once during fast startup (not live yet), recreated nothing, and persisted the new boot-id -> burned its one-shot. The periodic gate never recovered them either (first observation after the rebind -> no transition). Fix (harden the existing mechanism, no parallel one): processGuestBootChange now gates on the REAL live in-guest bind. driveBindLive checks whether /mnt/felhom-drives/<drive> is an actual mountpoint in the controller's own /mnt (rslave) /proc/self/mountinfo -- true only once the agent's bind propagated, exactly when docker can recreate the app. pollLiveBinds waits for that (bounded ~120s, poll 2s; rebind lands ~18s) and only then recreates via the normal pipeline, including stuck-Exited create-time-failure apps (shouldRecreateOnBoot is state-independent). Single-flight; absent-after-window drives left to the gate; host-reboot path unaffected; guest-only reboot path now covered. Tests: pollLiveBinds waits through the rebind then reports live (recreate fires); never-live drive stays absent; pre-fix companion (single early sample misses the not-yet-live bind). Red-proofed against a no-wait single-sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,5 +1,34 @@
|
||||
## Changelog
|
||||
|
||||
### v0.71.0 — fix the guest-reboot boot-race that strands drive-backed apps (2026-06-16)
|
||||
|
||||
A `pct reboot` of the guest left drive-backed apps (audiobookshelf, calibre-web, immich-server,
|
||||
jellyfin, komga, radarr, romm, paperless-webserver) stuck `Exited` forever. **Sub-cause (diagnosed,
|
||||
not guessed):** on guest boot the in-guest dockerd auto-starts the `unless-stopped` apps **~18s before**
|
||||
the agent re-binds the drive under the stable parent; the create-time volume bind fails
|
||||
(`mkdir /mnt/felhom-drives/<drive>/userdata: permission denied` on the empty fail-closed placeholder)
|
||||
and, being a create-time failure (`RestartCount=0`), is **never retried**. The existing recovery,
|
||||
`processGuestBootChange`, *ran* but **raced the rebind**: it sampled the agent's `BoundUnderParent`
|
||||
**once** during fast controller startup (bind not live yet) → recreated nothing → **persisted the new
|
||||
boot-id**, burning its one-shot. The periodic drive-gate never recovered them either (its first
|
||||
observation was *after* the rebind: present + not-disconnected → no transition).
|
||||
|
||||
**Fix (harden the existing mechanism — no parallel one):** `processGuestBootChange` now gates on the
|
||||
**REAL live in-guest bind** instead of the once-sampled agent view. New `driveBindLive` checks whether
|
||||
`/mnt/felhom-drives/<drive>` is an actual mountpoint in the controller's own `/mnt` (rslave)
|
||||
`/proc/self/mountinfo` — true only once the agent's bind has propagated, exactly when docker can
|
||||
recreate the app. New `pollLiveBinds` waits for that (bounded ~120s, polling 2s; the rebind lands ~18s)
|
||||
and only then recreates the deployed drive-backed apps via the normal pipeline (`compose down`→`up -d`).
|
||||
`shouldRecreateOnBoot` is unchanged and state-independent, so a stuck-`Exited` create-time-failure app
|
||||
is included. Single-flight (runs once before the periodic gate); apps on a drive that never goes live
|
||||
in the window are left to the gate (drive-absent → stop→return→restart). The host-reboot path the
|
||||
earlier sweep validated is unaffected (same code path, now strictly more robust — it waits for the
|
||||
bind). The **guest-only reboot path** (which the host-reboot sweep never exercised) is now covered.
|
||||
|
||||
Tests: `pollLiveBinds` waits through the rebind window then reports live (recreate fires);
|
||||
never-live drive stays absent (no spurious recreate); plus an explicit pre-fix companion that a single
|
||||
early sample misses the not-yet-live bind. Live-accepted with repeated `pct reboot 9201`.
|
||||
|
||||
### v0.70.0 — config-apply self-restart + geo-restriction UX fixes (2026-06-16)
|
||||
|
||||
Fixes found during live geo testing (rotating the Cloudflare API token).
|
||||
|
||||
Reference in New Issue
Block a user