agent v0.49.0: reboot-during-backup stale-lock recovery (F2-b) + shared-parent script redeploy fix (F2-a)

F2-b: at startup, recover a guest left with a stale vzdump lock by a
reboot-during-backup — pct unlock -> delete dangling vzdump snapshot ->
start iff onboot, guarded by a no-vzdump-running invariant (fail-safe).
New internal/localapi/stalelock.go; proxmox GuestConfig.Lock()/OnBoot(),
ListSnapshots, ListRunningTasks, Snapshot type. New narrow sudoers grant
FELHOM_STALELOCK (pct unlock) + Critical capability stalelock-unlock.

F2-a: EnsureSharedParent only redeployed the boot script when the UNIT
differed, so the v0.36.6 make-private fix never reached hosts whose unit
was current -> /mnt/felhom-drives stayed in root's shared:1 and doubled
every drive bind. New sharedParentInstallStale compares BOTH script and
unit. Boot-time-only; never churns the live mount.

Both root causes confirmed live on felhom-pve before fixing. Green gate
(build/vet/test) all pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0162BnMpUXscPsUB1cU8Tr6K
This commit is contained in:
2026-06-30 09:00:32 +02:00
parent 81954c3d2a
commit 6e38e2f921
12 changed files with 743 additions and 32 deletions
+36
View File
@@ -1,3 +1,39 @@
## v0.49.0 — reboot-during-backup stale-lock recovery (F2-b) + shared-parent script redeploy fix (F2-a) (2026-06-30)
Closes the two host-reboot findings from `TESTRUN-fullstack-2026-06-29.md`.
- **F2-b — startup stale-lock recovery (`internal/localapi/stalelock.go`, NEW).** A host reboot DURING a
vzdump backup leaves the guest with a `snapshot-delete`/`backup` lock + a dangling `vzdump` snapshot;
`onboot:1` then can't start the locked CT → the customer box stays DOWN until a human runs `pct unlock`.
The agent now self-heals at startup (`Server.RecoverStaleLockedGuests`, called alongside
`ReassertGuestBinds`/`RecoverFormatJob`): for each guest carrying a backup lock, **only when no vzdump is
genuinely in-flight** (the load-bearing invariant — at startup the agent's own backup loop hasn't run, so
the lock is stale; the guard fails SAFE if it can't confirm), it `pct unlock`s → deletes the dangling
`vzdump` snapshot (API + WaitTask) → starts the CT **iff** `onboot` and not already running. Scope is
strictly the two vzdump locks; `migrate`/`disk`/`create`/… are left untouched. Idempotent.
- **`internal/proxmox`:** new reads `GuestConfig.Lock()`/`OnBoot()`, `Client.ListSnapshots`,
`Client.ListRunningTasks`, and the `Snapshot` type. Reads + snapshot-delete + start go through the API
token; only `pct unlock` shells out (no API equivalent).
- **sudoers + capability manifest:** new narrow grant `FELHOM_STALELOCK = /usr/sbin/pct unlock [0-9]*`
and Critical capability `stalelock-unlock` (a stuck-locked guest = customer box down). `visudo -cf` clean;
covered by the manifest↔sudoers build gate.
- Tests: recovery sequence + companions — no-lock touches nothing; non-backup lock left alone; onboot=0
unlocked-but-not-started; delsnapshot only when a snapshot exists; **invariant guard** (live backup →
not cleared; unconfirmable → fail-safe); already-running → not restarted.
- **F2-a — shared-parent boot script never redeployed (`internal/localapi/intermediary.go`).** The host's
`/mnt/felhom-drives` was still in root's `shared:1` peer group (so every drive bind DOUBLED) because the
live boot script predated the v0.36.6 `make-private` fix. Root cause: `EnsureSharedParent` gated the
(re)install on the **unit** file only, so a script-only change never deployed. Fixed: the new
`sharedParentInstallStale` helper compares **both** the script and the unit (missing or differing →
reinstall). Boot-time-only — it rewrites the on-disk script; it does NOT churn the live mount (the live
bind/make-private/make-shared stays guarded on `!isHostMountpoint`). Verified empirically on the host:
the correct `bind → make-private → make-shared` sequence gives the parent its own group + no doubling.
- Tests: a stale-script/current-unit case triggers reinstall (the F2-a regression); both-current is a
no-op; missing files are stale; a content guard asserts the shipped script keeps `make-private`.
- Version `0.48.0 → 0.49.0`.
## v0.48.0 — report the served local-API leaf fingerprint (hub-side re-key detection, Part A) (2026-06-29)
The agent now rides its **served leaf fingerprint** on every host report so the hub can detect an