feat(v0.156.0): dead-primary alerting (R-51) + boot desired-state reconciliation (R-52)
R-51: aggregateState's mixed branch returned StateRunning ("partial"), so a stack whose
MAIN container was dead behind live helpers alerted on nothing — immich-server sat Exited
for 18 h, 100 % unreachable, no banner and no app_start_failed (audit F4). New
StateDegraded: a DOWN member whose docker restart policy is always/unless-stopped is a
fault (degraded, a down state); no/on-failure is a finished one-shot and stays benign; an
unreadable policy fails CLOSED. The unhealthy/restarting/paused/unknown exclusions are
byte-identical — folding unhealthy into down is the flapping fix-3 avoided.
R-52: new internal/bootrecon — one bounded start-once sweep at startup (2 attempts, 30 s
apart) for apps an interrupted boot left behind, inside the 90 s boot grace so a success
is silent and a failure still alerts. A zero-container stack is NEVER touched: the UI's
Stop is compose down, so a deliberate stop survives a reboot.
Both features carry a production-path wiring test (the v0.154.0 / v0.91.0 inert-seam
class). The main() assertion is an AST walk, not strings.Contains — the substring version
passed its own red-proof, because a commented-out call still contains the string.
Red-proofs run and restored: mix branch reverted -> "running" on the immich fixture;
boot hook commented out -> wiring test fails; zero-container gate dropped -> the
user-stopped app gets started.
NOTE: controller/cmd/controller/ is matched by .gitignore's `controller` entry, so new
files there need `git add -f` (and ripgrep silently skips main.go without --no-ignore).
This commit is contained in:
@@ -37,7 +37,9 @@ func getTimezone() *time.Location {
|
||||
// so an unhealthy app with a dead URL isn't mistaken for a merely-degraded-but-reachable one.
|
||||
func routeUnpublished(state stacks.ContainerState) bool {
|
||||
switch state {
|
||||
case stacks.StateUnhealthy, stacks.StateRestarting:
|
||||
// StateDegraded (R-51): the dead member is typically the one Traefik routes to, so the public
|
||||
// URL 404s exactly as it does for an unhealthy container.
|
||||
case stacks.StateUnhealthy, stacks.StateRestarting, stacks.StateDegraded:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
@@ -64,6 +66,9 @@ func (s *Server) templateFuncMap() template.FuncMap {
|
||||
case stacks.StateRestarting:
|
||||
// a restart loop is a problem, not progress
|
||||
return "warn"
|
||||
case stacks.StateDegraded:
|
||||
// R-51: a supervised member is dead — a genuine failure, not a user action
|
||||
return "warn"
|
||||
case stacks.StateStopped, stacks.StateExited:
|
||||
return "neutral"
|
||||
case stacks.StatePaused:
|
||||
@@ -84,6 +89,8 @@ func (s *Server) templateFuncMap() template.FuncMap {
|
||||
return "Telepítés..."
|
||||
case stacks.StateUnhealthy:
|
||||
return "Nem egészséges"
|
||||
case stacks.StateDegraded:
|
||||
return "Részlegesen leállt"
|
||||
case stacks.StateStopped, stacks.StateExited:
|
||||
return "Leállítva"
|
||||
case stacks.StateRestarting:
|
||||
@@ -102,7 +109,7 @@ func (s *Server) templateFuncMap() template.FuncMap {
|
||||
return "●"
|
||||
case stacks.StateStarting, stacks.StateDeploying:
|
||||
return "◐"
|
||||
case stacks.StateUnhealthy:
|
||||
case stacks.StateUnhealthy, stacks.StateDegraded:
|
||||
return "◑"
|
||||
case stacks.StateStopped, stacks.StateExited:
|
||||
return "○"
|
||||
@@ -119,7 +126,7 @@ func (s *Server) templateFuncMap() template.FuncMap {
|
||||
// and is not stopped/exited — used by templates for showing action buttons
|
||||
"isOperational": func(state stacks.ContainerState) bool {
|
||||
switch state {
|
||||
case stacks.StateRunning, stacks.StateStarting, stacks.StateUnhealthy, stacks.StateRestarting:
|
||||
case stacks.StateRunning, stacks.StateStarting, stacks.StateUnhealthy, stacks.StateRestarting, stacks.StateDegraded:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
@@ -201,7 +208,9 @@ func (s *Server) templateFuncMap() template.FuncMap {
|
||||
switch state {
|
||||
case stacks.StateRunning, stacks.StateStarting, stacks.StateUnhealthy, stacks.StateRestarting:
|
||||
return "running"
|
||||
case stacks.StateStopped, stacks.StateExited, stacks.StatePaused:
|
||||
case stacks.StateStopped, stacks.StateExited, stacks.StatePaused, stacks.StateDegraded:
|
||||
// R-51: degraded filters with the stopped set — the customer's question is
|
||||
// "is it working", and a stack with a dead supervised member is not.
|
||||
return "stopped"
|
||||
default:
|
||||
if deployed {
|
||||
|
||||
@@ -148,7 +148,9 @@ func (s *Server) dashboardHandler(w http.ResponseWriter, r *http.Request) {
|
||||
switch st.State {
|
||||
case stacks.StateRunning, stacks.StateStarting, stacks.StateUnhealthy, stacks.StateRestarting:
|
||||
running++
|
||||
case stacks.StateStopped, stacks.StateExited:
|
||||
// R-51: degraded counts with stopped — the dashboard counter answers "how many of my apps
|
||||
// work", and a stack with a dead supervised member does not.
|
||||
case stacks.StateStopped, stacks.StateExited, stacks.StateDegraded:
|
||||
stopped++
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user