v0.191.0 — warn before the wall comes down (R-167, R-158, R-174)

R-167: new internal/fillwatch warns the CUSTOMER before a filesystem fills. It emits the PRE-EXISTING disk_warning/disk_critical pair, which was allowlisted, copy'd, default-enabled and checkbox'd with no producer in any repo — the sixth "built but never wired" instance here. Two threshold terms (85% or 5 GiB free; critical 95%/2 GiB) because a percentage alone lies at both ends of this fleet's size range. Edge-triggered on escalation only, state persisted, hysteresis dead zone at 75%/7 GiB pinned by a test. A nil usage read is never a warning and never clears one. Per filesystem, never per app. Daily 03:30, before the nightly app-data legs. R-158: new unitNotify seam fires per app when a Tier-1 recovery-unit capture fails, loop continuing, carrying the target filesystem's used/free bytes. Operator-tier (recovery_unit_capture_failed) — deliberately NOT backup_failed, which is customer-enabled and would email the customer about a failure they cannot act on. D-c overrides R-158's own proposal here. R-174: the app-stop guard no longer starts apps onto MISSING drives — a regression in v0.189.0 code, found by review and closed the same session. SetStarter got the raw stack manager, whose StartStack has no drive gate, and Recover runs at startup. R-171 one path over. bootDriveGate could not be reused whole (its holder #2 is the guard's own marker, and holders #1/#2 read vars assigned after Recover runs), so holder #3 is extracted into a shared driveStartGate with a test pinning the delegation. ErrStartRefused splits a refusal from a failure: both keep the marker, only Failed alarms, because routing a deliberate hold into NotifyBackupFailed is the same false alarm. Tests 1157 -> 1184. All red-proofs demonstrated failing and restored.
2026-08-02 23:18:51 +02:00
parent 95eb5c2c1a
commit cf48214f6c
12 changed files with 1785 additions and 22 deletions
@@ -0,0 +1,373 @@
+// Package fillwatch warns the CUSTOMER that a filesystem is filling, BEFORE anything fails.
+//
+// R-167, operator decision D-c (2026-08-02, felhom.eu CONTEXT.md S-5). Nothing warned before this:
+// the first sign of a full filesystem was a backup that did not happen. The healthcheck folded disk
+// pressure into a generic `health_degraded` at 90%, for REGISTERED STORAGE PATHS ONLY — it never
+// looked at the docker area or the system-data area, never reported free bytes, and never named the
+// drive.
+//
+// WHY IT SHIPS BEFORE THE mp1→mp0 MERGE (D-a / R-165), not with it. Today an app that outgrows the
+// 20 G backup area is refused per app with its last good recovery unit preserved byte-identical —
+// a wall, but one that fails safely, one app at a time. The merge removes that wall so backups share
+// the large area. D-a's own condition (2) says the monitoring lands in the same step and never after;
+// landing it FIRST is strictly better and costs nothing, so the warnings go in and get proven on real
+// hardware while the wall is still standing.
+//
+// WHY THE FILESYSTEM IS THE UNIT AND NOT THE APP. One full disk holding ten apps would produce ten
+// identical warnings, nine of them noise. The customer's action — free space, delete files, add a
+// drive — is per filesystem too.
+package fillwatch
+
+import (
+	"encoding/json"
+	"fmt"
+	"log"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+	"sync"
+)
+
+// ── Thresholds ───────────────────────────────────────────────────────────────────────────────────
+//
+// TWO TERMS, AND WHICHEVER TRIPS FIRST WINS, because a percentage alone lies at both ends of the
+// size range this fleet actually has: 85% of a 20 G backup area leaves 3 G — not enough for one
+// DB-backed app's recovery unit, which is up to ~2× its data (measured: 21.1 GB → 40.2 GB,
+// architecture/07-backup-architecture.md §7.5) — while 85% of a 4 TB media drive leaves 600 G and is
+// nothing to write home about. A free-bytes floor catches the first; a percentage catches the second.
+const (
+	// WarnUsedPercent / WarnFreeGiB — enter the warning band.
+	WarnUsedPercent = 85.0
+	WarnFreeGiB     = 5.0
+
+	// CritUsedPercent / CritFreeGiB — enter the critical band. Below 2 GiB a volume tar of almost any
+	// real app fails, so this is "the next backup will not complete", not "it is getting tight".
+	CritUsedPercent = 95.0
+	CritFreeGiB     = 2.0
+
+	// ClearUsedPercent / ClearFreeGiB — the RETURN threshold, deliberately well below the warn pair.
+	// Both must hold. Between clear and warn is a dead zone in which the previous band is HELD, so a
+	// filesystem hovering on the line does not flap between warned and cleared. The gap is pinned by
+	// TestThresholdsKeepTheirHysteresisGap — a warn and clear threshold that can be edited into
+	// equality is a flapping bug waiting to be introduced.
+	ClearUsedPercent = 75.0
+	ClearFreeGiB     = 7.0
+)
+
+// Band is a filesystem's fill state. Ordered: escalation is an increase.
+type Band int
+
+const (
+	BandOK Band = iota
+	BandWarning
+	BandCritical
+)
+
+func (b Band) String() string {
+	switch b {
+	case BandWarning:
+		return "warning"
+	case BandCritical:
+		return "critical"
+	default:
+		return "ok"
+	}
+}
+
+// EventType maps a band to the hub event type it emits.
+//
+// These two types already existed in the hub's allowedEventTypes, already carried Hungarian copy,
+// already sat in the controller's DefaultEnabledEvents and already had a UI checkbox — and NOTHING
+// IN ANY REPO EMITTED THEM. A complete customer pipeline with no producer, the sixth "built but
+// never wired" instance in this project. This package is that producer; a new near-duplicate event
+// type would have left the pair inert forever.
+func (b Band) EventType() string {
+	switch b {
+	case BandCritical:
+		return "disk_critical"
+	case BandWarning:
+		return "disk_warning"
+	default:
+		return ""
+	}
+}
+
+// Severity is the hub severity for a band. `disk_warning` must be "warning" and `disk_critical`
+// "critical" — the dispatcher drops "info" entirely (severityNotifies), so getting this wrong stores
+// the event and mails nobody.
+func (b Band) Severity() string {
+	switch b {
+	case BandCritical:
+		return "critical"
+	case BandWarning:
+		return "warning"
+	default:
+		return ""
+	}
+}
+
+// Usage is one filesystem's occupancy. Deliberately not `system.DiskUsageInfo` — this package must be
+// testable without a real filesystem, and the seam is the reason it is.
+type Usage struct {
+	UsedPercent float64
+	AvailGB     float64
+	UsedGB      float64
+	TotalGB     float64
+}
+
+// Target is a watched filesystem: the path to stat, and the name a CUSTOMER will recognise.
+type Target struct {
+	Path  string
+	Label string
+}
+
+// Event is one crossing, handed to the notify seam.
+type Event struct {
+	Target  Target
+	Band    Band
+	Usage   Usage
+	Message string // the Hungarian customer text, already rendered
+}
+
+// Watcher holds the persisted per-filesystem band and emits on escalation only.
+type Watcher struct {
+	statePath string
+	logger    *log.Logger
+
+	// targets returns the filesystems to watch, resolved at CHECK time — a drive added or removed
+	// between checks must be picked up without a restart.
+	targets func() []Target
+	// usage reads one filesystem. NIL RESULT MEANS UNREADABLE, NEVER FULL (§8.4).
+	usage func(path string) *Usage
+	// notify pushes the customer event. Nil-safe.
+	notify func(Event)
+
+	mu    sync.Mutex
+	bands map[string]Band
+}
+
+// New builds a Watcher over a state file. Nothing is read until Check runs.
+func New(statePath string, logger *log.Logger, targets func() []Target, usage func(string) *Usage) *Watcher {
+	if logger == nil {
+		logger = log.Default()
+	}
+	return &Watcher{
+		statePath: statePath,
+		logger:    logger,
+		targets:   targets,
+		usage:     usage,
+		bands:     make(map[string]Band),
+	}
+}
+
+// SetNotify wires the customer event push. INIT-ONLY — call once at startup.
+func (w *Watcher) SetNotify(fn func(Event)) { w.notify = fn }
+
+// classify decides the band from a reading AND the previous band, which is what makes the hysteresis
+// work: between the clear and warn thresholds the previous band is HELD rather than recomputed.
+//
+// Pure, so it is unit-testable without a filesystem, a clock or a notifier.
+func classify(u Usage, prev Band) Band {
+	switch {
+	case u.UsedPercent >= CritUsedPercent || u.AvailGB < CritFreeGiB:
+		return BandCritical
+	case u.UsedPercent >= WarnUsedPercent || u.AvailGB < WarnFreeGiB:
+		return BandWarning
+	case u.UsedPercent <= ClearUsedPercent && u.AvailGB >= ClearFreeGiB:
+		return BandOK
+	default:
+		// The dead zone. Holding `prev` is the whole hysteresis: a filesystem sitting at 80% neither
+		// warns (it is below the warn line) nor clears (it is above the clear line).
+		return prev
+	}
+}
+
+// Check reads every target once and emits for each ESCALATION. Safe to call from the scheduler.
+func (w *Watcher) Check() error {
+	if w == nil {
+		return nil
+	}
+	w.mu.Lock()
+	defer w.mu.Unlock()
+	w.loadLocked()
+
+	targets := w.targets()
+	seen := make(map[string]bool, len(targets))
+	changed := false
+
+	for _, t := range targets {
+		if t.Path == "" {
+			continue
+		}
+		seen[t.Path] = true
+		u := w.usage(t.Path)
+		if u == nil {
+			// §8.4 — NOT-YET-MOUNTED IS NOT FULL. An absent, unmounted or unreadable filesystem is
+			// the drive gate's business and already has its own alert (storage_disconnected /
+			// backup_target_absent). Reporting it as "full" would be a false alarm with a misleading
+			// cause, and would tell the customer to delete files that are not the problem.
+			w.logger.Printf("[DEBUG] [fillwatch] %s: usage unreadable — skipped (an unreadable filesystem is not a full one)", t.Path)
+			continue
+		}
+		prev := w.bands[t.Path]
+		next := classify(*u, prev)
+		if next == prev {
+			continue
+		}
+		w.bands[t.Path] = next
+		changed = true
+
+		if next <= prev {
+			// De-escalation, including the return to OK, is SILENT. The customer already acted, or
+			// the app deleted its own temp files; telling them a resolved problem is resolved is
+			// noise. The state change re-arms the warning, which is what Scenario F asks for.
+			w.logger.Printf("[INFO] [fillwatch] %s: %s → %s (%.0f%% used, %.1f GB free) — cleared silently, re-armed",
+				t.Path, prev, next, u.UsedPercent, u.AvailGB)
+			continue
+		}
+
+		msg := Message(t, *u, next)
+		w.logger.Printf("[WARN] [fillwatch] %s (%q): %s → %s — %.0f%% used, %.1f GB free of %.1f GB; notifying the customer",
+			t.Path, t.Label, prev, next, u.UsedPercent, u.AvailGB, u.TotalGB)
+		if w.notify != nil {
+			w.notify(Event{Target: t, Band: next, Usage: *u, Message: msg})
+		}
+	}
+
+	// Forget filesystems that no longer exist (a decommissioned drive), so the state file cannot grow
+	// without bound and a re-added drive starts from OK rather than from a stale band.
+	for path := range w.bands {
+		if !seen[path] {
+			delete(w.bands, path)
+			changed = true
+		}
+	}
+
+	if !changed {
+		return nil
+	}
+	return w.saveLocked()
+}
+
+// Bands returns a copy of the current per-path state (diagnostics + tests).
+func (w *Watcher) Bands() map[string]Band {
+	w.mu.Lock()
+	defer w.mu.Unlock()
+	out := make(map[string]Band, len(w.bands))
+	for k, v := range w.bands {
+		out[k] = v
+	}
+	return out
+}
+
+// ── State persistence ────────────────────────────────────────────────────────────────────────────
+//
+// Persisted so a controller restart does not re-warn about a filesystem the customer has already
+// been told about — the offboxEnlargeBlockedNotify discipline. LOSING IT IS THE ACCEPTABLE
+// DIRECTION: at most one extra warning, never a missed one. THE HUB OWNS COOLDOWN; this package must
+// never add a timer of its own.
+
+type persisted struct {
+	Bands map[string]string `json:"bands"`
+}
+
+func (w *Watcher) loadLocked() {
+	if w.statePath == "" {
+		return
+	}
+	data, err := os.ReadFile(w.statePath)
+	if err != nil {
+		if !os.IsNotExist(err) {
+			w.logger.Printf("[WARN] [fillwatch] could not read %s: %v — starting from OK (at most one extra warning)", w.statePath, err)
+		}
+		return
+	}
+	var p persisted
+	if err := json.Unmarshal(data, &p); err != nil {
+		w.logger.Printf("[WARN] [fillwatch] %s is corrupt: %v — starting from OK (at most one extra warning)", w.statePath, err)
+		return
+	}
+	for path, band := range p.Bands {
+		switch band {
+		case "critical":
+			w.bands[path] = BandCritical
+		case "warning":
+			w.bands[path] = BandWarning
+		}
+	}
+}
+
+func (w *Watcher) saveLocked() error {
+	if w.statePath == "" {
+		return nil
+	}
+	p := persisted{Bands: make(map[string]string, len(w.bands))}
+	for path, band := range w.bands {
+		if band == BandOK {
+			continue // OK is the default — do not persist it
+		}
+		p.Bands[path] = band.String()
+	}
+	data, err := json.MarshalIndent(p, "", "  ")
+	if err != nil {
+		return err
+	}
+	if err := os.MkdirAll(filepath.Dir(w.statePath), 0o755); err != nil {
+		return err
+	}
+	tmp := w.statePath + ".tmp"
+	if err := os.WriteFile(tmp, data, 0o600); err != nil {
+		return err
+	}
+	return os.Rename(tmp, w.statePath)
+}
+
+// ── Customer copy (Hungarian) ────────────────────────────────────────────────────────────────────
+
+// Message renders the customer-facing Hungarian warning.
+//
+// IT MUST SAY WHAT TO DO, not merely that something is happening — a warning a customer cannot act
+// on is an operator alert wearing the wrong clothes. It names the storage by its LABEL (the customer
+// named it; a path means nothing to them) and gives the free space in GB.
+//
+// It is sent as the event's MESSAGE, and the hub deliberately has no `customerMessages` entry for
+// these two types: `FormatCustomerEmail` PREFERS the entry over the message, so a static template
+// would discard the label and the figures — exactly why `offbox_enlarge_blocked` and
+// `disk_health_degraded` have none either.
+func Message(t Target, u Usage, band Band) string {
+	name := t.Label
+	if name == "" {
+		name = t.Path
+	}
+	if band == BandCritical {
+		return fmt.Sprintf(
+			"A(z) „%s” tároló kritikusan megtelt: %s szabad hely maradt (%s foglalt). "+
+				"A biztonsági mentések és az alkalmazások írásai bármikor meghiúsulhatnak. "+
+				"Kérjük, mielőbb szabadíts fel helyet: törölj felesleges fájlokat, vagy csatlakoztass új meghajtót.",
+			name, hunGB(u.AvailGB), hunPercent(u.UsedPercent))
+	}
+	return fmt.Sprintf(
+		"A(z) „%s” tároló %s foglalt — %s szabad hely maradt. "+
+			"Kérjük, szabadíts fel helyet, mielőtt megtelik: törölj felesleges fájlokat, vagy csatlakoztass új meghajtót. "+
+			"Ha megtelik, a biztonsági mentések meghiúsulnak.",
+		name, hunPercent(u.UsedPercent), hunGB(u.AvailGB))
+}
+
+// hunGB formats a GB figure the Hungarian way — decimal COMMA, one decimal place. A "4.2 GB" in a
+// Hungarian sentence reads as a typo to the customer.
+func hunGB(gb float64) string {
+	if gb >= 1024 {
+		return strings.Replace(fmt.Sprintf("%.1f TB", gb/1024), ".", ",", 1)
+	}
+	return strings.Replace(fmt.Sprintf("%.1f GB", gb), ".", ",", 1)
+}
+
+func hunPercent(p float64) string { return fmt.Sprintf("%.0f%%", p) }
+
+// SortTargets gives Check a deterministic order, so a multi-filesystem crossing produces its events
+// in a stable sequence (tests, and a readable log).
+func SortTargets(ts []Target) []Target {
+	sort.Slice(ts, func(i, j int) bool { return ts[i].Path < ts[j].Path })
+	return ts
+}
@@ -0,0 +1,320 @@
+package fillwatch
+
+import (
+	"io"
+	"log"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// R-167 / decision D-c, customer half — the customer is warned BEFORE a fill, once.
+
+type harness struct {
+	w      *Watcher
+	events []Event
+	usage  map[string]*Usage
+	target []Target
+}
+
+func newHarness(t *testing.T, targets ...Target) *harness {
+	t.Helper()
+	h := &harness{usage: map[string]*Usage{}, target: targets}
+	h.w = New(filepath.Join(t.TempDir(), "fillwatch.json"), log.New(io.Discard, "", 0),
+		func() []Target { return h.target },
+		func(p string) *Usage { return h.usage[p] })
+	h.w.SetNotify(func(e Event) { h.events = append(h.events, e) })
+	return h
+}
+
+func (h *harness) set(path string, usedPct, availGB float64) {
+	h.usage[path] = &Usage{UsedPercent: usedPct, AvailGB: availGB, TotalGB: 100, UsedGB: usedPct}
+}
+
+func (h *harness) check(t *testing.T) {
+	t.Helper()
+	if err := h.w.Check(); err != nil {
+		t.Fatalf("Check: %v", err)
+	}
+}
+
+var photos = Target{Path: "/mnt/felhom-drives/hdd_1", Label: "Fotók"}
+
+// --- Scenario E — warned once, and the second pass is silent -------------------------------------
+
+func TestWarnsOnceThenIsSilent(t *testing.T) {
+	h := newHarness(t, photos)
+	h.set(photos.Path, 87, 4.2)
+
+	h.check(t)
+	if len(h.events) != 1 {
+		t.Fatalf("got %d events on the first crossing, want 1 — the customer was not warned before "+
+			"the fill, which is the whole customer half of D-c", len(h.events))
+	}
+	e := h.events[0]
+	if e.Band != BandWarning {
+		t.Fatalf("band = %v, want warning", e.Band)
+	}
+	if e.Band.EventType() != "disk_warning" {
+		t.Fatalf("event type = %q, want disk_warning", e.Band.EventType())
+	}
+	if e.Band.Severity() != "warning" {
+		t.Fatalf("severity = %q, want warning — the hub's severityNotifies DROPS \"info\", so a "+
+			"wrong severity stores the event and mails nobody", e.Band.Severity())
+	}
+
+	// SECOND PASS, nothing changed. Edge-triggered means exactly nothing fires.
+	h.check(t)
+	if len(h.events) != 1 {
+		t.Fatalf("got %d events after an unchanged second pass, want still 1 — the warning is not "+
+			"edge-triggered, so a daily schedule would re-warn the customer every single night",
+			len(h.events))
+	}
+}
+
+// The state must survive a restart — a fresh Watcher over the SAME file must not re-warn.
+func TestEdgeStateSurvivesARestart(t *testing.T) {
+	dir := t.TempDir()
+	state := filepath.Join(dir, "fillwatch.json")
+	usage := map[string]*Usage{photos.Path: {UsedPercent: 87, AvailGB: 4.2, TotalGB: 100}}
+	targets := func() []Target { return []Target{photos} }
+	read := func(p string) *Usage { return usage[p] }
+
+	var first []Event
+	w1 := New(state, log.New(io.Discard, "", 0), targets, read)
+	w1.SetNotify(func(e Event) { first = append(first, e) })
+	if err := w1.Check(); err != nil {
+		t.Fatal(err)
+	}
+	if len(first) != 1 {
+		t.Fatalf("first controller: %d events, want 1", len(first))
+	}
+
+	// <restart> — a brand-new Watcher, same file.
+	var second []Event
+	w2 := New(state, log.New(io.Discard, "", 0), targets, read)
+	w2.SetNotify(func(e Event) { second = append(second, e) })
+	if err := w2.Check(); err != nil {
+		t.Fatal(err)
+	}
+	if len(second) != 0 {
+		t.Fatalf("a restarted controller re-warned about an already-warned filesystem (%d events) — "+
+			"the state is not persisted, so every restart nags the customer", len(second))
+	}
+}
+
+// --- Scenario F — it clears, and it can fire again ------------------------------------------------
+
+func TestClearsSilentlyThenReArms(t *testing.T) {
+	h := newHarness(t, photos)
+
+	h.set(photos.Path, 87, 4.2)
+	h.check(t)
+	if len(h.events) != 1 {
+		t.Fatalf("warn: got %d events, want 1", len(h.events))
+	}
+
+	// Drop below the CLEAR thresholds (both must hold): 70% used, 12 GB free.
+	h.set(photos.Path, 70, 12)
+	h.check(t)
+	if len(h.events) != 1 {
+		t.Fatalf("clearing fired an event (%d total) — a resolved problem must clear SILENTLY", len(h.events))
+	}
+	if b := h.w.Bands()[photos.Path]; b != BandOK {
+		t.Fatalf("band after clearing = %v, want ok — it is latched, and the customer can never be "+
+			"warned about this filesystem again", b)
+	}
+
+	// Cross again — a NEW crossing must warn again.
+	h.set(photos.Path, 88, 3.9)
+	h.check(t)
+	if len(h.events) != 2 {
+		t.Fatalf("a NEW crossing after a clear produced %d events total, want 2 — the warning is "+
+			"latched forever after one firing", len(h.events))
+	}
+}
+
+// The dead zone is the hysteresis. A filesystem that falls back to 80% — below warn, above clear —
+// must NOT clear, or it flaps warned/cleared/warned as it wobbles across one line.
+func TestDeadZoneHoldsThePreviousBand(t *testing.T) {
+	h := newHarness(t, photos)
+	h.set(photos.Path, 87, 4.2)
+	h.check(t)
+
+	h.set(photos.Path, 80, 6) // between clear (75 / 7 GB) and warn (85 / 5 GB)
+	h.check(t)
+	if b := h.w.Bands()[photos.Path]; b != BandWarning {
+		t.Fatalf("band in the dead zone = %v, want warning held — without hysteresis a filesystem "+
+			"hovering on the line flaps between warned and cleared", b)
+	}
+	if len(h.events) != 1 {
+		t.Fatalf("the dead zone produced an extra event (%d total)", len(h.events))
+	}
+}
+
+// Escalation warning → critical MUST fire: it is a different message and a different urgency.
+func TestEscalationToCriticalFires(t *testing.T) {
+	h := newHarness(t, photos)
+	h.set(photos.Path, 87, 4.2)
+	h.check(t)
+
+	h.set(photos.Path, 96, 1.4)
+	h.check(t)
+	if len(h.events) != 2 {
+		t.Fatalf("got %d events, want 2 — an escalation from warning to critical went unreported", len(h.events))
+	}
+	e := h.events[1]
+	if e.Band != BandCritical || e.Band.EventType() != "disk_critical" {
+		t.Fatalf("second event = %v/%s, want critical/disk_critical", e.Band, e.Band.EventType())
+	}
+	// De-escalating critical → warning must be silent (it is still bad; do not celebrate).
+	h.set(photos.Path, 87, 4.2)
+	h.check(t)
+	if len(h.events) != 2 {
+		t.Fatalf("a critical→warning de-escalation fired (%d total) — only escalation notifies", len(h.events))
+	}
+}
+
+// --- Scenario I / §8.4 — a nil usage read is never a warning -------------------------------------
+
+func TestUnreadableFilesystemNeverWarns(t *testing.T) {
+	h := newHarness(t, photos)
+	// No entry in h.usage → the seam returns nil, which is what system.GetDiskUsage does on error.
+	h.check(t)
+
+	if len(h.events) != 0 {
+		t.Fatalf("an UNREADABLE filesystem produced %d warning(s) — an absent, unmounted or "+
+			"unreadable drive is the drive gate's business and has its own alert; reporting it as "+
+			"\"full\" is a false alarm with a misleading cause, and tells the customer to delete "+
+			"files that are not the problem (§8.4)", len(h.events))
+	}
+	if b, ok := h.w.Bands()[photos.Path]; ok && b != BandOK {
+		t.Fatalf("an unreadable filesystem was recorded as %v", b)
+	}
+}
+
+// An unreadable filesystem must not CLEAR an existing warning either — that would silently retract a
+// true alarm the moment a drive blipped.
+func TestUnreadableDoesNotClearAnExistingWarning(t *testing.T) {
+	h := newHarness(t, photos)
+	h.set(photos.Path, 87, 4.2)
+	h.check(t)
+
+	delete(h.usage, photos.Path) // now unreadable
+	h.check(t)
+	if b := h.w.Bands()[photos.Path]; b != BandWarning {
+		t.Fatalf("band after an unreadable read = %v, want the warning HELD — a blipping drive "+
+			"would otherwise silently retract a true alarm", b)
+	}
+}
+
+// --- Group H — the thresholds keep their gap ------------------------------------------------------
+
+// A warn and a clear threshold that can be edited into equality is a flapping bug waiting to be
+// introduced. This pins the ORDERING and a real margin, not the literal numbers.
+func TestThresholdsKeepTheirHysteresisGap(t *testing.T) {
+	if ClearUsedPercent >= WarnUsedPercent {
+		t.Fatalf("ClearUsedPercent (%.1f) must be strictly BELOW WarnUsedPercent (%.1f) — equal "+
+			"thresholds make a filesystem sitting on the line warn, clear, warn, clear every check",
+			ClearUsedPercent, WarnUsedPercent)
+	}
+	if ClearFreeGiB <= WarnFreeGiB {
+		t.Fatalf("ClearFreeGiB (%.1f) must be strictly ABOVE WarnFreeGiB (%.1f) — the free-byte term "+
+			"needs the same hysteresis as the percentage term, or it flaps on its own",
+			ClearFreeGiB, WarnFreeGiB)
+	}
+	// A margin, not merely an inequality: a 0.1-point gap is arithmetically a gap and practically none.
+	if WarnUsedPercent-ClearUsedPercent < 5 {
+		t.Fatalf("the used-percent hysteresis gap is %.1f points — too narrow to damp real wobble",
+			WarnUsedPercent-ClearUsedPercent)
+	}
+	if ClearFreeGiB-WarnFreeGiB < 1 {
+		t.Fatalf("the free-space hysteresis gap is %.1f GiB — too narrow", ClearFreeGiB-WarnFreeGiB)
+	}
+	if CritUsedPercent <= WarnUsedPercent || CritFreeGiB >= WarnFreeGiB {
+		t.Fatal("the critical band must be strictly tighter than the warning band on BOTH terms, " +
+			"or a filesystem can be critical without ever having been warned")
+	}
+}
+
+// Both terms must be able to trip INDEPENDENTLY — that is the entire reason there are two.
+func TestEitherTermCanTripTheWarning(t *testing.T) {
+	// A big drive: only 60% used, but under the free-space floor. 85% of a 4 TB drive leaves 600 GB,
+	// so the percentage alone would never fire here.
+	if got := classify(Usage{UsedPercent: 60, AvailGB: 3}, BandOK); got != BandWarning {
+		t.Fatalf("60%% used with 3 GB free classified as %v — the free-byte term did not trip, so a "+
+			"large drive can run out of space without ever warning", got)
+	}
+	// A small volume: plenty of GB free in absolute terms is impossible here, so the percentage is
+	// what must fire. 88% of a 100 GB volume leaves 12 GB — above the 5 GiB floor.
+	if got := classify(Usage{UsedPercent: 88, AvailGB: 12}, BandOK); got != BandWarning {
+		t.Fatalf("88%% used with 12 GB free classified as %v — the percentage term did not trip", got)
+	}
+	if got := classify(Usage{UsedPercent: 50, AvailGB: 50}, BandOK); got != BandOK {
+		t.Fatalf("a healthy filesystem classified as %v", got)
+	}
+}
+
+// --- The copy ------------------------------------------------------------------------------------
+
+// The customer message must be ACTIONABLE and specific. A warning that says only "something is
+// filling" is an operator alert wearing the wrong clothes.
+func TestCustomerMessageNamesTheDriveTheSpaceAndTheAction(t *testing.T) {
+	msg := Message(photos, Usage{UsedPercent: 87, AvailGB: 4.2, TotalGB: 100}, BandWarning)
+
+	if !strings.Contains(msg, "Fotók") {
+		t.Fatalf("the message does not name the storage by its LABEL — a path means nothing to a "+
+			"customer. Got: %s", msg)
+	}
+	if !strings.Contains(msg, "4,2 GB") {
+		t.Fatalf("the message does not give the free space in Hungarian number format (decimal "+
+			"COMMA). Got: %s", msg)
+	}
+	if !strings.Contains(msg, "87%") {
+		t.Fatalf("the message does not give the fill percentage. Got: %s", msg)
+	}
+	if !strings.Contains(msg, "szabadíts fel helyet") && !strings.Contains(msg, "szabadíts fel helyet:") {
+		t.Fatalf("the message does not tell the customer WHAT TO DO. Got: %s", msg)
+	}
+	// Design-system rule: no emoji in customer copy.
+	for _, r := range msg {
+		if r > 0x2100 {
+			t.Fatalf("the message contains an emoji/symbol %q — the design system forbids it in "+
+				"customer copy. Got: %s", string(r), msg)
+		}
+	}
+
+	crit := Message(photos, Usage{UsedPercent: 96, AvailGB: 1.4, TotalGB: 100}, BandCritical)
+	if crit == msg {
+		t.Fatal("the critical message is identical to the warning — the urgency must differ")
+	}
+	if !strings.Contains(crit, "1,4 GB") || !strings.Contains(crit, "Fotók") {
+		t.Fatalf("the critical message lost its specifics. Got: %s", crit)
+	}
+}
+
+// A label-less target must still produce a usable message rather than an empty quote.
+func TestMessageFallsBackToThePathWhenUnlabelled(t *testing.T) {
+	msg := Message(Target{Path: "/mnt/sys_drive"}, Usage{UsedPercent: 90, AvailGB: 1.5}, BandWarning)
+	if !strings.Contains(msg, "/mnt/sys_drive") {
+		t.Fatalf("an unlabelled target rendered without any identifier: %s", msg)
+	}
+}
+
+// A decommissioned drive must fall out of the state file, so it cannot grow without bound and a
+// re-added drive starts fresh.
+func TestVanishedTargetIsForgotten(t *testing.T) {
+	h := newHarness(t, photos)
+	h.set(photos.Path, 87, 4.2)
+	h.check(t)
+	if _, ok := h.w.Bands()[photos.Path]; !ok {
+		t.Fatal("the warned filesystem was not recorded")
+	}
+
+	h.target = nil // the drive is decommissioned
+	h.check(t)
+	if _, ok := h.w.Bands()[photos.Path]; ok {
+		t.Fatal("a removed filesystem kept its band — the state file grows without bound and a " +
+			"re-added drive would resume from a stale band instead of warning afresh")
+	}
+}