admin
5da11c4480
gates / gates (push) Successful in 11s
R-384. aggregateState returned StateUnhealthy the moment unhealthy > 0, and the R-51 mixed-case block that asks "is a supervised member dead?" sat below it. A two-container app whose database exits goes unhealthy BECAUSE it cannot reach that database - so the symptom the dead database causes was what suppressed the alarm for it. unhealthy is not a down state, so classifyRunStates never marked the app down and app_start_failed never fired. Measured live on demo-hp 2026-08-22: bookstack-db stopped at 21:27:01 and the F-OBS heartbeat printed "0 currently down" throughout. R-51's 18-hour immich failure, back through a different door. Two things moved, and either alone leaves the defect standing: the supervised test is hoisted above the unhealthy/starting/restarting returns, and "some members are up" now counts ANY member not in the down bucket. The old guard was running > 0, which made the R-51 block unreachable in exactly the case it was written for. IsDownState is byte-identical - unhealthy stays excluded, because an unhealthy container is running and folding it in reintroduces the flapping that exclusion exists to stop. No new state was minted. Only the ORDER changed. The priority comment was rewritten because it asserted an ordering the code no longer has. Three subtests in TestAggregateState_UnchangedBranches were AMENDED: they asserted an unhealthy/starting/restarting member beat an exited peer on unless-stopped, which pinned the defect as settled behaviour. They keep their intent with the down member given a benign policy. R-383. The double-failure message said the previous state's backup EXISTS, built from the returned path without asking the filesystem - and a missing file is one of the two ways that rollback fails. undoCopyPhrase now describes the copy from disk: present, partial, missing (still naming where it should be), or never written. Zero-length counts as missing. Test count 1494 -> 1504. Four red-proofs planted, four seen failing; the two halves of R-384 convict independently.
62 lines
2.8 KiB
Go
62 lines
2.8 KiB
Go
package main
|
|
|
|
import (
|
|
"testing"
|
|
"time"
|
|
|
|
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
|
)
|
|
|
|
// THE DETECTOR'S OWN POSITIVE CONTROL, at the layer it lives in.
|
|
//
|
|
// Part 3 of the 2026-08-22 R-361 session measured that a HELD app raises no dead-app alarm. That is
|
|
// an ABSENCE claim, and an absence claim is worthless unless the detector is shown working. On the
|
|
// live box it was hard to hold a stack in a down state long enough to see one: `aggregateState`
|
|
// checked `unhealthy > 0` BEFORE the mixed-case degraded branch (internal/stacks/manager.go), so an
|
|
// app whose database died went `degraded` for a moment and then `unhealthy` as its own healthcheck
|
|
// failed — and `unhealthy` is deliberately not in `IsDownState`.
|
|
//
|
|
// **That sentence described R-384, and nobody filed it.** It was written here as an inconvenience
|
|
// while building this control; it was in fact the mechanism by which a dead database raised no alarm
|
|
// at all. Fixed in v0.222.0 by asking the supervised-down question FIRST — see
|
|
// `internal/stacks/manager.go` aggregateState and TestR384_DeadSupervisedMemberIsAskedAboutFirst.
|
|
// The past tense above is deliberate: the ordering it describes is no longer the code's.
|
|
//
|
|
// `classifyRunStates` is pure, so the control is exact here rather than a race against health probes.
|
|
func TestClassifyRunStates_PositiveControl_ADownStackDoesAlarm(t *testing.T) {
|
|
now := time.Now()
|
|
for _, st := range []stacks.ContainerState{stacks.StateDegraded, stacks.StateExited} {
|
|
sts := []stacks.Stack{{Name: "victim", Deployed: true, State: st}}
|
|
dead, states := classifyRunStates(sts, nil, nil, now)
|
|
if len(dead) != 1 {
|
|
t.Errorf("%s: the dead-app banner must fire, got %d entries", st, len(dead))
|
|
}
|
|
if len(states) != 1 || !states[0].Down {
|
|
t.Errorf("%s: the run state must be Down, got %+v", st, states)
|
|
}
|
|
}
|
|
}
|
|
|
|
// ...and the states measured on the live box do NOT alarm, which is why the held app was silent.
|
|
// This is the other half of the same control: it pins WHY, so the next reader does not re-derive it.
|
|
func TestClassifyRunStates_TheStatesMeasuredLiveDoNotAlarm(t *testing.T) {
|
|
now := time.Now()
|
|
cases := []struct {
|
|
state stacks.ContainerState
|
|
why string
|
|
}{
|
|
{stacks.StateUnhealthy, "a held app, and any app whose database died, aggregates here — unhealthy is not in IsDownState"},
|
|
{stacks.StateStopped, "a fully stopped app is whitelisted as a deliberate user stop unless quiesce failed to restart it"},
|
|
}
|
|
for _, c := range cases {
|
|
sts := []stacks.Stack{{Name: "victim", Deployed: true, State: c.state}}
|
|
dead, states := classifyRunStates(sts, nil, nil, now)
|
|
if len(dead) != 0 {
|
|
t.Errorf("%s: expected no banner (%s), got %d", c.state, c.why, len(dead))
|
|
}
|
|
if len(states) != 1 || states[0].Down {
|
|
t.Errorf("%s: expected not-Down (%s), got %+v", c.state, c.why, states)
|
|
}
|
|
}
|
|
}
|