F-OBS: the dead-app check gets a positive observable (v0.180.0)
deadapp-check had no observable at default info level: its per-cycle line goes through Scheduler.dbg(), gated on logging.level==debug, so on a default box it is never PRODUCED (not merely filtered) and cannot reach the always-DEBUG ring. A 30s interval also puts it on the scheduler's quiet path. 'No alarms' was therefore indistinguishable from 'the detector never ran' — which undermines confidence in the F-CRIT-1 fix in the field. A periodic summary, not a line per run: at 30s a per-run line is 2880 lines/day, which is why the original author chose silence. Every 20th scan (~10 min) emits one INFO with the scan count, apps evaluated and apps down. A test pins the cadence so it cannot be widened into uselessness. Also corrects the 'unquiesce guaranteed by defer' comment — fault 10 established the guarantee is the crash marker plus Recover().
This commit is contained in:
@@ -449,7 +449,11 @@ func (l *Loop) allTiersForManualRun(ctx context.Context) []dueTier {
|
||||
// matters — see resolveDueTiers.
|
||||
//
|
||||
// Crash-safety is unchanged and non-negotiable: the marker is written BEFORE anything stops,
|
||||
// unquiesce is guaranteed by defer, and it fires exactly once no matter which tier fails. A crash
|
||||
// unquiesce fires exactly once no matter which tier fails, and the GUARANTEE is the crash MARKER plus
|
||||
// Recover() — not the defer. Campaign 8 fault 10 established that: a SIGKILL mid-quiesce runs no
|
||||
// deferred function, and what brought the stacks back was Recover() reading the marker 1 s after
|
||||
// restart. The defer covers the graceful exits (backup error, poll error, max-quiesce bound, context
|
||||
// cancellation); the marker covers the hard crash. Do not describe this as "guaranteed by defer". A crash
|
||||
// between two backups leaves the marker on disk and Recover() restarts the stacks at startup.
|
||||
func (l *Loop) quiesceAndPollTiers(ctx context.Context, tiers []dueTier) error {
|
||||
if len(tiers) == 0 {
|
||||
|
||||
Reference in New Issue
Block a user