F-OBS: the dead-app check gets a positive observable (v0.180.0)

deadapp-check had no observable at default info level: its per-cycle line goes
through Scheduler.dbg(), gated on logging.level==debug, so on a default box it is
never PRODUCED (not merely filtered) and cannot reach the always-DEBUG ring. A
30s interval also puts it on the scheduler's quiet path. 'No alarms' was
therefore indistinguishable from 'the detector never ran' — which undermines
confidence in the F-CRIT-1 fix in the field.

A periodic summary, not a line per run: at 30s a per-run line is 2880 lines/day,
which is why the original author chose silence. Every 20th scan (~10 min) emits
one INFO with the scan count, apps evaluated and apps down. A test pins the
cadence so it cannot be widened into uselessness.

Also corrects the 'unquiesce guaranteed by defer' comment — fault 10 established
the guarantee is the crash marker plus Recover().
This commit is contained in:
2026-07-28 10:27:28 +02:00
parent a63409c843
commit fb91c8d766
4 changed files with 171 additions and 1 deletions
+5 -1
View File
@@ -449,7 +449,11 @@ func (l *Loop) allTiersForManualRun(ctx context.Context) []dueTier {
// matters — see resolveDueTiers.
//
// Crash-safety is unchanged and non-negotiable: the marker is written BEFORE anything stops,
// unquiesce is guaranteed by defer, and it fires exactly once no matter which tier fails. A crash
// unquiesce fires exactly once no matter which tier fails, and the GUARANTEE is the crash MARKER plus
// Recover() — not the defer. Campaign 8 fault 10 established that: a SIGKILL mid-quiesce runs no
// deferred function, and what brought the stacks back was Recover() reading the marker 1 s after
// restart. The defer covers the graceful exits (backup error, poll error, max-quiesce bound, context
// cancellation); the marker covers the hard crash. Do not describe this as "guaranteed by defer". A crash
// between two backups leaves the marker on disk and Recover() restarts the stacks at startup.
func (l *Loop) quiesceAndPollTiers(ctx context.Context, tiers []dueTier) error {
if len(tiers) == 0 {