Campaign 10: R-157 second mechanism — the zero-container case, which is SILENT

The 4th hard-reset failure had a different signature, verified not assumed: all of
rallly healthy, papra missing entirely with state=stopped deployed=True
containers=0. Zero containers is exactly what bootrecon deliberately never touches,
because the UI's Stop is compose down which removes containers - but a hard reset
landing during a compose operation produces the identical state. The signature the
safety rule depends on cannot distinguish the two.

Worse: in that state the deadapp check reported 0 currently down while a deployed
app was not running. No app_start_failed, no banner. That is the workspace's own
false-invariant #4 (F-CRIT-1, StateStopped assumed deliberate) recurring through a
hard reset rather than quiesce. NOT filed as new - CLAUDE.md already records it -
but confirmed live on 0.188.0 via a new path.

papra returned after ~15 min, later than the harness's 10-min window, so this
instance was slow rather than permanent and the doc says so. What restarted it is
not established.

Mechanism A (Exited, missed by the unsettled snapshot) alarms; mechanism B (zero
containers) is invisible on every channel. A settle-condition fix closes A only.
This commit is contained in:
2026-08-02 05:35:57 +02:00
parent 2d64ee7241
commit 1931dfcb0c
@@ -107,3 +107,51 @@ mechanism, per the workspace rule about invariants needing a test that fails whe
`R-52` is the feature; grep found **no existing item covering its timing**. `R-156` is this campaign's
papra finding. **R-157 is free and is claimed here.**
---
## Second mechanism, observed 2026-08-02 ~05:25 (run 2b cycle 34) — the zero-container case
The 4th hard-reset failure had a **different signature**, and it matters because it is not a race at
all. Verified rather than assumed (the two preceding non-R-157 violations in this campaign turned out
to be harness artifacts, so each occurrence is now checked):
all of rallly healthy this time; papra MISSING ENTIRELY
papra: state=stopped deployed=True containers=0
bootrecon 03:14:34: "no boot-orphaned apps" (refresh 7 → 6 → 7 after it, same churn)
`containers=0` is precisely the state bootrecon **deliberately never touches** — its documented safety
rule is *"a stack with ZERO containers is deliberately never touched"*, because the UI's Stop is
`docker compose down`, which removes containers. But a hard reset landing **during** a compose
operation produces the identical zero-container state. **The signature the safety rule relies on
cannot distinguish "the customer stopped this" from "an interruption caught it between containers".**
### And in this state the app is SILENT, not merely unrecovered
[deadapp] check alive: 20 scans since boot, 5 deployed app(s) evaluated, 0 currently down
`papra` was `deployed=True` and not running, and the deadapp check counted **0 down**. No
`app_start_failed`, no banner. This is the workspace's own false-invariant #4 recurring —
*"`StateStopped` means deliberately stopped by the user"* (F-CRIT-1), reached here through a hard
reset rather than through quiesce. **Not filed as new**: `CLAUDE.md` already records it. Confirmed
live, on controller 0.188.0, via a new path.
That makes the zero-container case **worse than the Exited case** this document opens with: the Exited
one at least alarmed (`app_start_failed`, `1 currently down`); this one is invisible on every channel.
### Recovery in this instance
`papra` did come back roughly 15 minutes after the reset — later than the harness's 10-minute window,
which is why the violation fired. **So this instance was slow, not permanent**, and the document does
not claim otherwise. What is not established is *what* restarted it: the campaign's own later atoms
are a candidate, and nothing in the logs attributes the recovery to a product mechanism.
## Summary — two mechanisms, one root
| | signature | bootrecon's view | alarmed? | observed |
|---|---|---|---|---|
| **A** | container `Exited (255)`, `state=degraded` | missed — snapshot taken before settle | yes (`app_start_failed`, `1 down`) | 3× |
| **B** | **zero containers**, `state=stopped` | deliberately skipped — matches a user Stop | **no — `0 currently down`** | 1× |
Both leave a `deployed: true` app not running after an interrupted boot, which is the exact condition
R-52 exists to eliminate. A fix that only adds a settle condition closes **A** and leaves **B** open.