fix(disk-health): the alert that never sent — severity, a real Hiba level, and a memory that survives a restart

Three defects made the disk-health feature silent in exactly the case it
exists for. Evidence: felhom.eu documentation/audits/DIAG-smart-passed-trap-2026-08-14.md

1. SEVERITY (the one that changes whether anything arrives at all).
   NotifyDiskHealthDegraded emitted severity "warn", which is NOT in the
   hub's accepted set {info,warning,error,critical}. The hub coerced it to
   "info" (hub/internal/api/handler.go) and severityNotifies dropped it
   (hub/internal/notify/dispatcher.go), so every Figyelmeztetes-level disk
   alert was filed as an informational notice and emailed to NOBODY, on the
   customer and the operator leg alike. Now "warning". DiskAlertKind.Severity()
   is exported so the contract is checkable from any package.

2. NO LEVEL ABOVE "worth an eye". smart_status.passed CANNOT fail on
   unreadable sectors (attrs 187/197/198 all carry thresh 0 and a normalized
   value floors at 1), so Hiba was unreachable for this whole fault class.
   DiskVerdictFor now takes a DiskPrior and implements a 14-row top-down
   ladder: sustained unreadable sectors, a count too large to be a blip (64),
   unreadable+remapping together, overheating, NVMe critical flag or spent
   endurance all reach Hiba. No fourth label — predicted failure is "Hiba".

3. IT SPOKE ONCE, AND FORGOT ON RESTART. The baseline was in-memory, so a box
   that rebooted while a disk was failing never alerted again; and between 8
   and 352 sectors nothing was emitted at all. State is now persisted
   (disk-health-state.json, atomic tmp+rename), the decision compares against
   the last ALERTED verdict (collapsing flaps to one alert while letting a
   genuine escalation fire immediately), and a disk already at Hiba re-alerts
   once it has BOTH doubled its count and waited out a 24h cooldown.

The card replays the same prior the check used (diskRecord.PriorSawUncorrectable)
so the chip and the email cannot disagree — the property the shared verdict
function exists to guarantee, now pinned rather than asserted.

Tests: 12 scenario groups A-L. Group L builds the Server through web.NewServer,
the same call main.go makes, over a real file.
This commit is contained in:
2026-08-14 08:10:59 +02:00
parent 3e3ee94b7b
commit bb50e1293c
9 changed files with 1534 additions and 228 deletions
+71 -17
View File
@@ -556,24 +556,78 @@ type DiskHealthDetails struct {
Critical bool `json:"critical"`
}
// NotifyDiskHealthDegraded fires a disk_health_degraded hub event on a disk-health DEGRADATION only
// (the controller's 6h check owns the transition logic — never first-run/recovery/UNKNOWN). severity
// is warn for a Figyelmeztetés, critical for a Hiba (FAILING). The hub applies its own per-event-type
// cooldown. NOTE: the event type "disk_health_degraded" MUST be in the hub's allowedEventTypes (else
// the hub 400s the POST) — added in the hub's v0.x bump alongside this.
func (n *Notifier) NotifyDiskHealthDegraded(label string, attrs []string, critical bool) {
severity := "warn"
var msg string
switch {
case critical:
severity = "critical"
msg = fmt.Sprintf("Lemez állapot romlás: %s — a lemez SMART önellenőrzése hibát jelez. Kérjük, mentse az adatait, és vegye fel velünk a kapcsolatot.", label)
case len(attrs) > 0:
msg = fmt.Sprintf("Lemez állapot romlás: %s — romló érték: %s. Javasolt figyelemmel kísérni.", label, strings.Join(attrs, ", "))
default:
msg = fmt.Sprintf("Lemez állapot romlás: %s — a lemez állapota romlott. Javasolt figyelemmel kísérni.", label)
// DiskAlertKind selects the customer-facing message shape. A customer's ACTION differs by kind —
// "back up and call us for a replacement" is not "check the ventilation" — so the kind travels with
// the alert rather than being flattened into a single sentence.
type DiskAlertKind int
const (
DiskAlertWarn DiskAlertKind = iota // Figyelmeztetés — worth keeping an eye on
DiskAlertFailSelfReported // Hiba — the drive's own SMART verdict says FAILING
DiskAlertFailSectors // Hiba — reached from unreadable-sector counters
DiskAlertFailTemperature // Hiba — reached from heat
DiskAlertFailWorsened // Hiba — already reported, and still getting worse
)
// DiskAlert is the payload for one disk-health alert. It carries enough for the notifier to pick a
// message shape and fill in the counts; message CONSTRUCTION stays here because the notifier owns
// customer copy, and moving it to the caller would scatter Hungarian across packages.
type DiskAlert struct {
Label string // customer-facing disk label (device model where known)
Attributes []string // Hungarian attribute names behind the verdict (nil for a self-reported FAILING)
Kind DiskAlertKind
Sectors int // max(pending, offline_uncorrectable) — quoted in the sector/worsened shapes
TemperatureC int // °C — quoted in the temperature shape
}
// Severity is the hub-accepted severity string for this alert.
//
// THE VOCABULARY IS EXACT AND IT IS THE HUB'S, NOT OURS. The hub accepts only
// {"info","warning","error","critical"} and silently COERCES anything else to "info"
// (felhom.eu/hub/internal/api/handler.go, the severity switch in the event-ingest handler); "info" is
// then dropped by severityNotifies (felhom.eu/hub/internal/notify/dispatcher.go), which routes only
// warning/error/critical. So a severity outside that set is stored and emailed to NOBODY — neither
// the customer nor the operator leg.
//
// Exported so any caller — and any test in any package — can check the contract against the two
// named hub locations instead of duplicating the literal.
//
// Until v0.215.0 this function emitted "warn", which is not in the set. Every Figyelmeztetés-level
// disk alert the product ever produced was filed as an informational notice and delivered to no one.
func (k DiskAlertKind) Severity() string {
if k == DiskAlertWarn {
return "warning"
}
n.emit("disk_health_degraded", severity, msg, DiskHealthDetails{Disk: label, Attributes: attrs, Critical: critical})
return "critical"
}
// NotifyDiskHealthDegraded fires a disk_health_degraded hub event. The controller's periodic check
// owns the decision to call this at all (transitions, flap damping, the re-alert cooldown) — never
// first-run, never recovery, never UNKNOWN.
//
// The hub applies its own per-event-type cooldown ON TOP of ours. NOTE: the event type
// "disk_health_degraded" MUST be in the hub's allowedEventTypes (else the hub 400s the POST) — it is.
func (n *Notifier) NotifyDiskHealthDegraded(a DiskAlert) {
critical := a.Kind != DiskAlertWarn
var msg string
switch a.Kind {
case DiskAlertFailSelfReported:
msg = fmt.Sprintf("Lemez állapot romlás: %s — a lemez SMART önellenőrzése hibát jelez. Kérjük, mentse az adatait, és vegye fel velünk a kapcsolatot.", a.Label)
case DiskAlertFailSectors:
msg = fmt.Sprintf("Lemez hiba: %s — a meghajtón %d olvashatatlan szektor van. Mentse az adatait, és keressen meg minket a meghajtó cseréjéhez.", a.Label, a.Sectors)
case DiskAlertFailTemperature:
msg = fmt.Sprintf("Lemez hiba: %s — a meghajtó túlmelegedett (%d °C). Ellenőrizze a gép szellőzését, és keressen meg minket.", a.Label, a.TemperatureC)
case DiskAlertFailWorsened:
msg = fmt.Sprintf("Lemez hiba: %s — a meghajtó állapota tovább romlott, már %d olvashatatlan szektor van. Ha még nem tette meg, mentse az adatait.", a.Label, a.Sectors)
default: // DiskAlertWarn
if len(a.Attributes) > 0 {
msg = fmt.Sprintf("Lemez állapot romlás: %s — romló érték: %s. Javasolt figyelemmel kísérni.", a.Label, strings.Join(a.Attributes, ", "))
} else {
msg = fmt.Sprintf("Lemez állapot romlás: %s — a lemez állapota romlott. Javasolt figyelemmel kísérni.", a.Label)
}
}
n.emit("disk_health_degraded", a.Kind.Severity(), msg,
DiskHealthDetails{Disk: a.Label, Attributes: a.Attributes, Critical: critical})
}
// emit sends an event through the test seam if set, else the real async PushEvent.