# REPORT — controller v0.110.0: offbox stale-lock self-heal (C2) + crash-truthful status (C1) — DEPLOYED **Date:** 2026-07-10 · **Class:** implementation (controller only). Baseline v0.109.1 → **v0.110.0** (live on 9201, healthy). Fixes the overnight campaign's HIGH finding (C2) + the cosmetic C1. No hub/agent change. ## What shipped - **C2 self-heal (`internal/backup`):** `resticStep` wraps the backup/prune/restore restic calls — on a lock error (`repository is already locked`) it escalates to `unlock --remove-all` and retries the step ONCE, safe because the repo has a SINGLE legitimate writer (per-customer sub-account isolation + the in-process single-flight mutex every caller holds). A second lock failure surfaces the error (never loops). Plus a cheap pre-run `unlock` (stale-only) hygiene before every run + restore. **Boundary (documented in code + README):** a DR-cloned second controller writing the same repo would defeat the single-writer premise — operator-supervised, out of scope. - **C1 (`NewManager.reconcileCrashedRun`):** a persisted `LastStatus="running"` at startup (a controller that died mid-run) flips to `error` + "megszakadt futás (a vezérlő újraindult futás közben)"; the next successful run clears it. Nil-settings guarded. ## Tests + companion red-proofs (ALL-GREEN, both red-proofs fired) - A — lock-on-backup → `unlock --remove-all` → retry → ok. **Red-proof:** neuter the escalation → the exact campaign failure `offbox backup rallly: exit status 1` → FAIL. Reverted. - B — lock persists twice → exactly one `--remove-all` + one retry → error surfaced, no unlock loop. - C — pre-run stale `unlock` issued on every run (asserted via the fake's call count). - E — no lock → `--remove-all` NEVER fires (the escalation can't trigger spuriously). - D (C1) — Manager built over `LastStatus="running"` flips to the Hungarian interrupted error. **Red-proof:** drop the flip → status lies "running" forever → FAIL. Reverted. ## Live validation (bounded, demo-safe — mirrors the campaign) - Deployed 0.110.0; demo untouched (escrowed, quota 50, ok). - **C1 live:** crashed the controller mid-run (`docker restart`) → on startup the log showed "previous run was interrupted by a controller restart…" and settings flipped to `last_status="error"`, `last_error="megszakadt futás…"`. ✓ - **C2 live end-to-end:** crashed the controller DURING a backup (restic holding the exclusive lock) → the NEXT run self-healed and completed "backup OK: 2 app(s) backed up, 43s" with **zero manual restic**. The demo returned to `escrowed`/`ok`; the 04:15 schedule is safe. ✓ - **Refinement vs the task premise:** the live evidence (and, on re-reading, the campaign's own teardown log where plain `restic unlock` already reported "successfully removed locks") shows the pre-run stale `unlock` hygiene ALONE clears these crash locks — the container's hostname is stable (`demo-felhom`), so restic detects the dead PID as stale. So in practice the lock is cleared before the backup step ever sees it; the `--remove-all` escalation is the genuine can't-detect-staleness fallback (unit-tested + red-proofed, not needed in this live run). Both layers ship — hygiene first, escalation as the safety net. ## NEXT Peti onboarding runbook (the session-gated tracks: auto-confirm happy chain, fresh-enable full cycle, live quota-crossing, hub-orchestrated provisioning + F4/freeze end-to-end); old-box archive retirement (Track E hashes in hand — the new box restores byte-perfect).