v0.223.0: the app-down alarm reached nobody (R-329), and the stop nobody heard (R-386)

R-329. NotifyAppStartFailures emitted severity "warn". The hub accepts exactly
{info, warning, error, critical} and silently coerces anything else to "info",
which severityNotifies then drops BEFORE both legs. Banner shown, event stored,
POST 200, no mail sent. One word.

This is the second time: DiskAlertKind.Severity emitted "warn" until v0.215.0
and its own comment records that every warning-level disk alert went to nobody.
A comment recorded the lesson and nothing enforced it. The guard is now an AST
walk over the whole controller - grep cannot work here, since "warn" appears
legitimately nine times as a healthcheck status vocabulary.

The sweep found exactly one bad severity. Its limits are stated: the walk cannot
follow a variable, so all six dynamic call sites are registered by name with the
values each can take, and a new one fails the test. Two of the six were found by
the guard, not by the hand sweep before it.

Also pinned: fillwatch.Band.Severity() returns "" for BandOK, which would vanish
the same way. It is unreachable because Check() notifies only on escalation -
but that safety lives in a different function from the one that looks unsafe, so
the test asserts the consequence rather than the mapping.

app_start_failed gains a customer toggle, DEFAULT OFF, per operator ruling. The
operator is mailed either way: processOperator never consults customer prefs.
It is deliberately NOT in operatorOnlyEvents, which would make the toggle a lie.

R-386. classifyRunStates decided "the customer stopped this" from the STATE, so
every stopped stack was assumed deliberate. Measured on demo-hp: privatebin
stopped out of band, nine scans, zero events, zero banner - while the comment
beside it claimed an out-of-band stop still alerts.

DesiredState already records the answer and has exactly one writer. Stopped ->
no alarm; Running -> alarm; absent -> UNKNOWN, keep today's behaviour AND say
so. Absent stays silent deliberately: reading it as "nobody asked" would email
about every app anyone ever stopped, fleet-wide, on the first cycle after
upgrade. The gap is bounded not silent - IntentUnknown is set and the names are
logged at INFO on the heartbeat cadence. failedRestart still lifts a Stopped
intent, or F-CRIT-1 re-opens. No new DesiredState writer.

Two settings toggles each governed two alarms. "Lemez figyelmeztetes (90%+)"
also wrote disk_critical, the drive-is-FAILING alarm. Now four honest toggles;
12 became 15. A no-op save stores the existing slice verbatim, so byte identity
is by construction - without that guard the defaults case reorders, which the
red-proof caught.

Test count 1504 -> 1522. Five red-proofs, five seen failing; one passed first
time and is reported - that mutation was inert, not the test weak.

controller/internal/notify/notifier.go

@@ -508,6 +508,14 @@ type AppRunState struct {
 	Name        string
 	DisplayName string
 	Down        bool
+	// IntentUnknown is set when this app is STOPPED and no customer intent was ever recorded, so the
+	// alarm was suppressed by the §4 fallback rather than by a decision anyone made (R-386).
+	//
+	// It rides here rather than being a third return value or a logger parameter so that the log line
+	// and the suppression come from the SAME computation — a separately-derived log is a second
+	// source of truth, and the two drift. `classifyRunStates` stays pure and its signature does not
+	// move, which is what lets its existing tests keep testing what they were written to test.
+	IntentUnknown bool
 }
 
 // NotifyAppStartFailures fires an `app_start_failed` hub event ONCE per running→down transition
@@ -543,7 +551,14 @@ func (n *Notifier) NotifyAppStartFailures(apps []AppRunState) {
 		if name == "" {
 			name = a.Name
 		}
-		n.emit("app_start_failed", "warn",
+		// R-329: "warning", NOT "warn". The hub's vocabulary is exactly
+		// {info, warning, error, critical} and it COERCES anything else to "info" at ingest, silently
+		// — after which severityNotifies drops it and NEITHER leg runs. See DiskAlertKind.Severity's
+		// doc comment, which records the same mistake shipping once before (v0.215.0). This one was
+		// worse: it was invisible for months because R-384's ordering defect meant the event could
+		// not fire at all, so a broken severity had nothing to break.
+		// Pinned by TestR329_EveryEmittedSeverityIsInTheHubVocabulary (AST walk over this package).
+		n.emit("app_start_failed", "warning",
 			fmt.Sprintf("Telepített alkalmazás nem fut: %s", name),
 			AppDetails{StackName: a.Name, DisplayName: a.DisplayName})
 	}

controller/internal/notify/r329_severity_contract_test.go

@@ -0,0 +1,212 @@
+package notify
+
+import (
+	"fmt"
+	"go/ast"
+	"go/parser"
+	"go/token"
+	"io/fs"
+	"path/filepath"
+	"sort"
+	"strconv"
+	"strings"
+	"testing"
+)
+
+// R-329 — the hub's severity vocabulary, pinned by an AST walk over the WHOLE controller.
+//
+// THE DEFECT THIS EXISTS TO KILL. `NotifyAppStartFailures` emitted severity `"warn"`. The hub accepts
+// only {info, warning, error, critical} and **silently coerces** anything else to `info` at ingest,
+// after which `severityNotifies` drops it and NEITHER the operator nor the customer leg runs. So the
+// dashboard showed the app down, the hub stored the event, and no mail left the building.
+//
+// **This is the SECOND time.** `DiskAlertKind.Severity`'s own doc comment records that it emitted
+// `"warn"` until v0.215.0, and that every Figyelmeztetés-level disk alert the product ever produced
+// was delivered to nobody. A comment recorded the lesson; nothing enforced it; it happened again.
+// **A comment is not a guard** — this is the guard.
+//
+// THE LAYER. This sits at the EMITTER, which is where the defect lives: the hub's coercion is
+// deliberate and correct (a lost alarm is worse than a mis-routed one), so nothing downstream can
+// detect a bad word — by design, the bad word ceases to exist at the hub's front door. The only
+// place the mistake is still visible is where it is written.
+//
+// WHY AN AST WALK AND NOT grep. `strings.Contains` over source cannot tell an emitted severity from
+// the word "warn" in a comment, a healthcheck status vocabulary (`internal/monitor`,
+// `internal/selftest` both legitimately use "warn"), or a `case "warn":` in an unrelated switch. The
+// 2026-08-23 sweep found nine such hits and exactly one real defect.
+//
+// RED-PROOF (observed, see REPORT.md): put `"warn"` back at notifier.go's app_start_failed emit and
+// this fails naming the file, line, call and value.
+
+// hubSeverityVocabulary is the hub's set, verbatim. Sources, both named so a reader can check rather
+// than trust: felhom.eu/hub/internal/api/handler.go (the ingest switch) and
+// felhom.eu/hub/internal/notify/dispatcher.go (severityNotifies).
+var hubSeverityVocabulary = map[string]bool{
+	"info": true, "warning": true, "error": true, "critical": true,
+}
+
+// severityArg names the functions that take a hub severity, and which argument it is.
+var severityArg = map[string]int{
+	"emit":      1, // (eventType, severity, message, details)
+	"PushEvent": 1, // (eventType, severity, message, details)
+}
+
+// knownDynamicSeveritySites are the call sites that pass a VARIABLE rather than a literal, so this
+// walk cannot check their value. Each was traced by hand on 2026-08-23 and every reachable value was
+// in the vocabulary. **They are listed so that a NEW dynamic site cannot appear unnoticed** — the
+// honest limit of an AST check is that it cannot follow a variable, and an unlisted limit is not a
+// limit, it is a hole.
+var knownDynamicSeveritySites = map[string]string{
+	"internal/notify/notifier.go:emit":                     `pass-through of its own severity parameter to PushEvent — the value is checked at emit's CALLERS, above`,
+	"internal/notify/notifier.go:NotifyControllerUpdated":  `local var: "info", or "error" when the update failed`,
+	"internal/notify/notifier.go:NotifyDRCompleted":        `local var: "info", or "warning" when failCount > 0`,
+	"internal/notify/notifier.go:NotifyAgentChannelDown":   `internal/channelhealth's classifier — every severity literal in checker.go is "warning" or "error"`,
+	"internal/notify/notifier.go:NotifyDiskHealthDegraded": `DiskAlertKind.Severity() — pinned behaviourally by TestR329_DiskAlertSeveritiesStillSatisfyTheContract`,
+	"cmd/controller/main.go:main":                          `fillwatch.Band.Severity() in the SetNotify closure — pinned by TestR329_FillwatchNeverEmitsTheEmptySeverity`,
+}
+
+func walkControllerFiles(t *testing.T, fn func(path string, fset *token.FileSet, f *ast.File)) {
+	t.Helper()
+	root, err := filepath.Abs("../..") // the controller module root
+	if err != nil {
+		t.Fatal(err)
+	}
+	fset := token.NewFileSet()
+	err = filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if d.IsDir() {
+			if d.Name() == "vendor" || d.Name() == ".git" || d.Name() == "node_modules" {
+				return filepath.SkipDir
+			}
+			return nil
+		}
+		if !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") {
+			return nil
+		}
+		f, perr := parser.ParseFile(fset, path, nil, parser.ParseComments)
+		if perr != nil {
+			return fmt.Errorf("parse %s: %w", path, perr)
+		}
+		rel, _ := filepath.Rel(root, path)
+		fn(rel, fset, f)
+		return nil
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+}
+
+// calleeName returns the bare function name for `foo(...)` and `x.foo(...)`.
+func calleeName(call *ast.CallExpr) string {
+	switch fn := call.Fun.(type) {
+	case *ast.Ident:
+		return fn.Name
+	case *ast.SelectorExpr:
+		return fn.Sel.Name
+	}
+	return ""
+}
+
+func TestR329_EveryEmittedSeverityIsInTheHubVocabulary(t *testing.T) {
+	var checked int
+	var dynamic []string
+
+	walkControllerFiles(t, func(rel string, fset *token.FileSet, f *ast.File) {
+		// Attribute each call to the FuncDecl that encloses it, by walking declarations rather than
+		// trusting Inspect's visit order — a closure inside main() must read as "main", and a call in
+		// a var block must not inherit the previous function's name.
+		for _, decl := range f.Decls {
+			fd, isFunc := decl.(*ast.FuncDecl)
+			enclosing := "(file scope)"
+			var scope ast.Node = decl
+			if isFunc {
+				enclosing = fd.Name.Name
+				scope = fd
+			}
+			inspectForSeverity(t, rel, enclosing, scope, fset, &checked, &dynamic)
+		}
+	})
+
+	// The walk must actually have found call sites — a guard that silently examines nothing is the
+	// "instrument that can drop results" failure, and it would pass forever.
+	if checked < 15 {
+		t.Fatalf("only %d severity literals examined — the AST walk is not reaching the call sites; "+
+			"fix this test before trusting a green from it", checked)
+	}
+	t.Logf("checked %d severity literals across the controller", checked)
+
+	// The dynamic sites are a stated limit, and the list is the mechanism that keeps it stated.
+	sort.Strings(dynamic)
+	seen := map[string]bool{}
+	for _, d := range dynamic {
+		if seen[d] {
+			continue
+		}
+		seen[d] = true
+		if _, known := knownDynamicSeveritySites[d]; !known {
+			t.Errorf("NEW dynamic severity call site %q — this walk cannot check a variable's value. "+
+				"Trace every value it can take; if all are in the hub vocabulary, add it to "+
+				"knownDynamicSeveritySites with the reason. Do not delete this check.", d)
+		}
+	}
+	for known := range knownDynamicSeveritySites {
+		if !seen[known] {
+			t.Errorf("registered dynamic site %q no longer exists — remove it from "+
+				"knownDynamicSeveritySites so the register stays honest", known)
+		}
+	}
+}
+
+// The vocabulary itself must match the hub's, and DiskAlertKind.Severity — the function whose doc
+// comment records the first occurrence — must still satisfy it.
+func TestR329_DiskAlertSeveritiesStillSatisfyTheContract(t *testing.T) {
+	for _, k := range []DiskAlertKind{
+		DiskAlertWarn, DiskAlertFailSelfReported, DiskAlertFailSectors,
+		DiskAlertFailTemperature, DiskAlertFailWorsened,
+	} {
+		if got := k.Severity(); !hubSeverityVocabulary[got] {
+			t.Errorf("DiskAlertKind(%d).Severity() = %q, not in the hub vocabulary", k, got)
+		}
+	}
+	// And the one that regressed: a warning must be a *warning*, not "warn" and not "info".
+	if got := DiskAlertWarn.Severity(); got != "warning" {
+		t.Errorf("DiskAlertWarn.Severity() = %q, want \"warning\"", got)
+	}
+}
+
+// inspectForSeverity checks every severity-taking call inside one declaration.
+func inspectForSeverity(t *testing.T, rel, enclosing string, scope ast.Node, fset *token.FileSet, checked *int, dynamic *[]string) {
+	t.Helper()
+	ast.Inspect(scope, func(n ast.Node) bool {
+		call, ok := n.(*ast.CallExpr)
+		if !ok {
+			return true
+		}
+		idx, wanted := severityArg[calleeName(call)]
+		if !wanted || len(call.Args) <= idx {
+			return true
+		}
+		lit, isLit := call.Args[idx].(*ast.BasicLit)
+		if !isLit || lit.Kind != token.STRING {
+			*dynamic = append(*dynamic, rel+":"+enclosing)
+			return true
+		}
+		val, err := strconv.Unquote(lit.Value)
+		if err != nil {
+			t.Errorf("%s: unparseable severity literal %s", fset.Position(lit.Pos()), lit.Value)
+			return true
+		}
+		*checked++
+		if !hubSeverityVocabulary[val] {
+			t.Errorf("%s: %s(...) emits severity %q, which is NOT in the hub's vocabulary "+
+				"{info, warning, error, critical}.\n"+
+				"    The hub COERCES it to \"info\" at ingest, SILENTLY, and severityNotifies then "+
+				"drops \"info\" — so this event is stored and emailed to NOBODY, on either leg.\n"+
+				"    This exact mistake shipped once before (DiskAlertKind.Severity, fixed v0.215.0).",
+				fset.Position(lit.Pos()), calleeName(call), val)
+		}
+		return true
+	})
+}