boot window: sample REFRESHES first — a cached fleet made 'settled' meaningless
gates / gates (push) Successful in 9s

Found by live validation on 9201, not by review. GetStacks() is the Manager's
in-memory map refreshed by the scheduler every 10s; sampling it every 5s without
refreshing means two identical samples can mean the cache did not update rather
than that the fleet settled. A container removed ~5s before the window closed was
still in the sampled fleet and the sweep logged 'no boot-orphaned apps' for an app
that had none. sampleBootFleet now refreshes first; a refresh error degrades
rather than aborting the window.
This commit is contained in:
2026-08-02 20:17:12 +02:00
parent 582135f861
commit dcc3363d2f
2 changed files with 78 additions and 6 deletions
+18 -1
View File
@@ -1406,8 +1406,25 @@ type bootFleetSample struct {
containers int
}
// sampleBootFleet returns the fleet snapshot, sorted, so two samples compare by equality.
// sampleBootFleet REFRESHES, then returns the fleet snapshot, sorted, so two samples compare by
// equality.
//
// THE REFRESH IS LOAD-BEARING, and it was found by live validation, not by review. `GetStacks()`
// returns the Manager's IN-MEMORY map, which the scheduler refreshes on its own 10 s cadence
// (`status-refresh`, main.go). Sampling it every 5 s without refreshing means two consecutive samples
// can straddle one refresh and be identical because THE CACHE DID NOT UPDATE — not because the fleet
// stopped moving. The window would then declare "settled" on stale data and sweep on a picture of the
// box from up to 10 s ago, which is a quieter version of the exact defect R-157 mechanism A is.
//
// Observed on guest 9201 on 2026-08-02: a container removed ~5 s before the window closed was still
// present in the sampled fleet, so the sweep logged "no boot-orphaned apps" for an app that had none.
//
// `RefreshStatus` is a cheap `docker ps`-based refresh of the in-memory map (the same one the
// scheduler runs every 10 s), so at most ~10 extra calls per boot. A refresh error is logged and the
// sample proceeds on whatever the map holds — degraded, but never silently: a boot window that cannot
// see docker is exactly when a stale verdict is most dangerous.
func sampleBootFleet(mgr bootrecon.StackProvider) []bootFleetSample {
_ = mgr.RefreshStatus()
stacksNow := mgr.GetStacks()
out := make([]bootFleetSample, 0, len(stacksNow))
for _, s := range stacksNow {