admin/felhom-controller
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.110.0: offbox stale-lock self-heal (campaign C2) + crash-truthful status (C1)

Browse Source 
resticStep escalates a restic lock error to `unlock --remove-all` + one
retry (safe: single-writer repo — sub-account isolation + single-flight
mutex); plain `unlock` is stale-only and can't clear a crash lock across a
container-hostname change. Pre-run stale unlock hygiene on run+restore.
C1: NewManager flips a persisted LastStatus=running to a truthful error.
Both red-proofed (A reproduces the exact campaign backup failure).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
authentik Default Admin
2026-07-10 07:02:23 +02:00
parent 0bd4cd02be
commit cb9e992c7a
5 changed files with 224 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+22
View File
@@ -1,5 +1,27 @@
## Changelog
### v0.110.0 — offbox stale-lock self-heal (campaign C2) + crash-truthful status (C1) (2026-07-10)
Fixes the overnight campaign's HIGH finding: a crash mid-prune left a restic EXCLUSIVE lock the controller
couldn't clear, failing every subsequent offsite run until manual `restic unlock`. Root nuance from the
evidence: plain `restic unlock` (stale-only) does NOT clear it — the recreated container has a new hostname,
so restic can't verify the dead PID and won't treat the lock as stale for ~30 min.
- **C2 — `internal/backup`:** `resticStep` wraps the backup/prune/restore restic calls: on a lock error
(`repository is already locked`) it escalates to `unlock --remove-all` and **retries the step ONCE**,
justified by the ARCHITECTURAL single-writer guarantee (one controller per repo via per-customer
sub-account isolation + the in-process single-flight mutex every caller holds → no live sibling). A second
lock failure surfaces the error (never loops). Plus cheap pre-run `unlock` (stale-only) hygiene before
every run + restore. **Boundary (documented):** a DR-cloned second controller writing the same repo would
defeat the single-writer premise — operator-supervised territory.
- **C1 — `NewManager.reconcileCrashedRun`:** on startup, a persisted `LastStatus="running"` (a controller
that died mid-run) flips to `error` + the Hungarian "megszakadt futás (a vezérlő újraindult futás közben)"
— truthful after a crash; the next successful run clears it.
- Tests + red-proofs: self-heal-and-retry (A, **red-proof:** neuter the escalation → the exact campaign
failure `offbox backup rallly: exit status 1` → FAIL); persistent-lock → one `--remove-all` + one retry,
error surfaced, no loop (B); pre-run stale unlock issued every run (C); no lock → `--remove-all` never
fires (E); crash-status flip (D, **red-proof:** drop the flip → status lies "running" → FAIL).
### v0.109.1 — re-apply must preserve escrow custody + runtime status (live finding) (2026-07-10)
Found deploying v0.109.0: including `QuotaGB` in the bridge's descriptor hash triggered a one-time