# REPORT — F-CRIT-1 + F-A1 fixed (controller v0.179.0, 2026-07-28)

Docs here. Implementation, all six red-proofs and the full live replay live in
`felhom-controller/REPORT.md`. The campaign that found both:
`documentation/audits/CAMPAIGN-8-backup-restore-2026-07-27.md`.

## What changed
**F-CRIT-1** — an app that failed to restart after a quiesce never alarmed, for two independent
reasons, either of which alone kept it dead: `restartAll` returned nothing (the failure was logged
and dropped), and `classifyRunStates` whitelisted `StateStopped` on invariant I1 ("the user stopped
it") — which the quiesce loop had made false by stopping stacks the same `docker compose down` way.
A failed restart and a user stop are the *same* Docker state; the only difference is that the loop
tried and could not, now surfaced by `Loop.FailedRestarts()`.

**F-A1** — HTTP 409 is the agent's single-flight gate refusing while a restore-test holds it, not a
failure. It is now contention: the tier stays DUE, is dropped before anything stops, and unending
contention raises its own **BLOCKED** signal rather than going silent.

## Bounds, justified against measured reality
- `contentionRetryAfter` **15m** — longest restore-test observed on the fleet is 12m01s; the agent's
  local restore-test wait is 10m. Caps app-stop churn at 4/hour instead of 12/hour.
- `contentionAlarmAfter` **3h** — the agent's own PBS restore-test task is capped at 120 minutes, so
  contention outliving that is a *stuck* gate, not a busy one. 3h adds margin and is 15× the longest
  contention actually observed.

## Verified live, with the hub DB as arbiter — not from logs
Same box, same day, same event type; the only difference is 409 versus a genuine error:

| injection | operator emails (demo-hp) |
|---|---|
| **409 contention** | 8 → **8** (none) |
| **real transport failure** | 8 → **9** |

And for F-CRIT-1: the failed restart alarmed **9 seconds** after grace expiry with the dashboard
banner naming the `(stopped)` state, while a **deliberate** user stop on the same box stayed silent
through **9** dead-app scans (the positive observable that the silence is suppression, not a dead
detector).

## The rule this arc earned
Added to **both** copies of `CLAUDE.md` (live + `documentation/runbooks/workspace-CLAUDE.md`):
**a comment asserting an invariant needs a test pinning it, or it is a wish.** Six instances in this
project have shipped guarantees the code did not provide — `EffectiveProtected`, `newestArchiveOn`,
the R-97a operator-only claim, `classifyRunStates`' I1, `inflight.go`'s defer claim, and
`quiesce.go`'s spurious-failure claim. Two were found only on live hardware, and one of those had a
green, red-proofed test suite over a production path broken two independent ways.

Corollary recorded with it: prefer a test that asserts the **consequence** (does the alarm fire?)
over one that asserts the **mechanism** (does suppression expire?). R-97b's Scenario F proved the
mechanism; the consequence was still broken.

## Docs touched
- `documentation/backlog/OPEN-ITEMS.md` — F-CRIT-1 and F-A1 → SHIPPED + PROVEN-LIVE.
- `documentation/audits/CAMPAIGN-8-backup-restore-2026-07-27.md` — both rows struck through, closing
  section added. **All three of the campaign's alarm findings are now closed** (F-CRIT-1, F-CRIT-2,
  F-A1).
- `documentation/runbooks/workspace-CLAUDE.md` — the invariant rule.

## Still open, highest first
**Fault 4** (restic transport interruption) — four injection approaches were defeated by
guest-bridged networking, and it is now the most valuable follow-up: F-CRIT-2 answered the phantom
question for PBS and left the identical question open for restic. Then **R-99** (prune never removes
phantoms) and **F-LEAK** (a failed restore-test cannot destroy its own scratch guest — observed
again during this work).