admin/felhom-controller
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

controller v0.71.0: fix guest-reboot boot-race stranding drive-backed apps

Browse Source 
Sub-cause: on guest pct reboot, in-guest dockerd auto-starts unless-stopped
drive-backed apps ~18s BEFORE the agent re-binds the drive; the create-time
volume bind fails (mkdir /mnt/felhom-drives/<drive>/userdata: permission denied)
and RestartCount=0 means it's never retried -> stuck Exited. The existing
recovery (processGuestBootChange) RAN but raced the rebind: it sampled the
agent's BoundUnderParent once during fast startup (not live yet), recreated
nothing, and persisted the new boot-id -> burned its one-shot. The periodic gate
never recovered them either (first observation after the rebind -> no transition).

Fix (harden the existing mechanism, no parallel one): processGuestBootChange now
gates on the REAL live in-guest bind. driveBindLive checks whether
/mnt/felhom-drives/<drive> is an actual mountpoint in the controller's own /mnt
(rslave) /proc/self/mountinfo -- true only once the agent's bind propagated,
exactly when docker can recreate the app. pollLiveBinds waits for that (bounded
~120s, poll 2s; rebind lands ~18s) and only then recreates via the normal
pipeline, including stuck-Exited create-time-failure apps (shouldRecreateOnBoot
is state-independent). Single-flight; absent-after-window drives left to the gate;
host-reboot path unaffected; guest-only reboot path now covered.

Tests: pollLiveBinds waits through the rebind then reports live (recreate fires);
never-live drive stays absent; pre-fix companion (single early sample misses the
not-yet-live bind). Red-proofed against a no-wait single-sample.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
authentik Default Admin
2026-06-16 15:46:51 +02:00
parent 38c82db522
commit 25e5cb5850
4 changed files with 213 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
controller/README.md
+10
View File
@@ -646,6 +646,16 @@ not just those with HDD data. Non-HDD apps can configure destination, method, an
> - **Guest-reboot convergence is DETERMINISTIC** via the agent's `guest_boot_id`: the controller persists
> `LastGuestBootID` and, when it changes, recreates EVERY deployed drive-backed app onto the
> re-propagated drive (`processGuestBootChange` — no fragile container-uptime sampling).
> **v0.71.0 — boot-race fix:** on a guest `pct reboot`, in-guest dockerd auto-starts the apps ~18s
> BEFORE the agent re-binds the drive, so their volume bind fails at create-time
> (`mkdir …/userdata: permission denied`, `RestartCount=0` → never retried → stuck `Exited`). The old
> recovery sampled the agent's `BoundUnderParent` ONCE, raced that rebind, recreated nothing, and burned
> its boot-id one-shot. `processGuestBootChange` now **gates on the REAL live in-guest bind**
> (`driveBindLive`: is `/mnt/felhom-drives/<drive>` an actual mountpoint in the controller's own `/mnt`
> rslave `/proc/self/mountinfo`?) and **waits** for it (`pollLiveBinds`, bounded ~120s) before recreating
> — including apps stuck `Exited` with a create-time mount failure (`shouldRecreateOnBoot` is
> state-independent). The **guest-only reboot path** (which the host-reboot sweep never exercised) is now
> covered; drives that never go live in the window are left to the drive-absent gate.
>
> **⚠️ Rebuilt on the agent-delegated disk model (v0.43.0), made ROLE-AWARE in v0.44.0, UX-polished in
> v0.45.0.** After the 8C