diff --git a/CHANGELOG.md b/CHANGELOG.md index 0b8adf9..436dabd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,35 @@ ## Changelog +### v0.169.0 — Disk-health card + degradation notification ("Lemezek állapota") (2026-07-24) + +Consumes the agent's new `smart` payload field (agent **v0.94.0**); **MinAgent floor unchanged** — the +feature detects by payload presence (nil → "Nincs adat", never alarms). Pairs with the hub allowlist +bump (adds `disk_health_degraded`). No new smartctl load anywhere — the agent serializes +already-computed SMART; the controller only reads it. + +- **`agentapi`:** `SmartSummary` extended to the full counter set (SATA reallocated/pending/offline + + NVMe critical/media/percentage_used + power-on-hours); `DiskInfo` gains `Smart *SmartSummary`; new + pure `DiskVerdictFor(*SmartSummary) DiskVerdict` (the SINGLE source of truth for card + check) with + `Label()` (Rendben / Figyelmeztetés / Hiba / Nincs adat) + `DegradedAttributes`. Mapping: FAILING → + Hiba; PASSED with any of reallocated>0 / pending>0 / offline_uncorrectable>0 / critical_warning>0 / + media_errors>0 / percentage_used ≥ 90 → Figyelmeztetés; PASSED clean → Rendben; nil/UNKNOWN/empty → + Nincs adat (never alarms). +- **Dashboard "Lemezek állapota" card:** one row per PHYSICAL disk (label + colored verdict chip + + temperature). Fed by a **60 s in-process TTL cache** around `/disks` so dashboard refresh-spam cannot + smartctl-storm the host. An unreachable agent renders "Nincs adat" — the page never blocks. +- **6-hourly `disk-health-check`:** compares each physical disk's verdict against an in-memory baseline + and emits `disk_health_degraded` **only on a degradation** (verdict worsened). First run baselines + silently; recovery/improvement notifies nothing; **UNKNOWN is excluded both directions** (a transient + UNKNOWN blip never fires and never erases history); multiple attributes on one disk → ONE event. + Severity: warn (Figyelmeztetés) / critical (Hiba). The hub applies its own per-event-type cooldown. +- **Deliberately no global alert banner** (CONTEXT ruling) — the card + email carry it; banner fatigue + is a real cost. Not wired into the dead-app/alert-banner machinery. Controller restart re-baselines + silently (accepted, consistent with the health-change pattern). + +Tests: verdict table (+ ≥90 boundary red-proof); notifier emit (type/severity/subject); check +first-run-silent (red-proof: disable the guard → first run notifies), degradation-once, recovery-silent, +UNKNOWN-excluded, FAILING→critical, nil-smart card graceful, TTL cache. + ### v0.168.0 — Customer-configurable backup window ("Mentési időablak") (2026-07-24) No agent coupling; MinAgent unchanged (the disk-tier gate is controller-side; the agent's cadence-based diff --git a/CONTEXT.md b/CONTEXT.md index 6fd6dd3..6d07825 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -7,6 +7,23 @@ > > Ask Claude Code: "Please update CONTEXT.md with what we did today" +Last updated: 2026-07-24 (v0.169.0 — disk-health card + degradation notification "Lemezek állapota") + +> **2026-07-24 — v0.169.0 (disk-health card + degradation alert).** Consumes the agent's new `smart` +> field (agent v0.94.0; MinAgent floor unchanged — feature-detect by presence). **Rulings:** (1) ONE +> pure verdict fn `agentapi.DiskVerdictFor` is the shared truth for the card chip AND the 6h check — they +> can never disagree. Thresholds: FAILING→Hiba; PASSED + any(reallocated>0/pending>0/offline_unc>0/ +> critical_warning>0/media_errors>0/percentage_used **≥90**)→Figyelmeztetés; PASSED clean→Rendben; +> nil/UNKNOWN→Nincs adat (never alarms). (2) **No global alert banner** — the card + email carry disk +> health; banner fatigue is a real cost, so this is deliberately NOT wired into the dead-app/alert-banner +> machinery. (3) Degradation-only notification with an in-memory baseline: first run baselines silently, +> recovery never notifies, **UNKNOWN excluded both directions** (a transient blip neither fires nor erases +> history). (4) **Controller restart re-baselines silently** (in-memory baseline lost on restart) — an +> accepted trade consistent with the health-change pattern (a real post-restart degradation still fires on +> the following 6h check once a baseline exists). (5) A **60s TTL cache** wraps the card's /disks call so +> dashboard refresh-spam can't smartctl-storm the host; the 6h check fetches FRESH (cache-independent). +> Pairs with hub +1 (allowlist `disk_health_degraded`). No new smartctl load — serialization only. + Last updated: 2026-07-24 (v0.168.0 — customer-configurable backup window "Mentési időablak") > **2026-07-24 — v0.168.0 (customer-configurable backup window).** ONE customer setting — the window diff --git a/REUSE.md b/REUSE.md index b4977ac..4df4d69 100644 --- a/REUSE.md +++ b/REUSE.md @@ -136,6 +136,8 @@ | `Client.AddNetStorage/ListNetStorage/RemoveNetStorage` | controller/internal/agentapi/client.go | NAS mounts (A1) | Network storage | Password passes through to agent's 0600 cred file; controller NEVER persists it | | `agentapi.StatusError` | controller/internal/agentapi/client.go | `{Path, Code}` typed non-2xx GET error | Distinguishing HTTP statuses from transport errors (`errors.As`) | NEVER string-match agent error text — the capability probe keys on `Code==404` | | `SupportCache.Supports` / `Client.Supports` | controller/internal/agentapi/features.go | `(ctx, prober, Feature) SupportState` | Agent-capability gate for COUPLED features (route probe, TTL 5m) | 404 ⇒ No; transport/5xx ⇒ Unknown (NEVER refuse on Unknown). New coupled feature = new `featureProbes` row + gate call at the entry point + `MinAgent:` in the CHANGELOG header (publish-train-rules.md). Web layer: `Server.netFeatures` through the `netAgent` seam | +| `agentapi.DiskVerdictFor` / `DiskVerdict.Label` / `DegradedAttributes` | controller/internal/agentapi/diskverdict.go | `(*SmartSummary) DiskVerdict` | THE shared disk-health verdict (card chip + 6h check) — v0.169.0 | Pure; nil/UNKNOWN → `DiskVerdictUnknown` (Nincs adat, NEVER alarms); percentage_used threshold is **≥90**. Feature-detects the agent's `DiskInfo.Smart` (nil = old agent). Do NOT recompute the verdict inline anywhere | +| `Server.cachedDisks` / `RunDiskHealthCheck` | controller/internal/web/disk_health.go | `(ctx)` | Card fetch (60s TTL) / the 6h degradation check | Card uses the 60s TTL cache (anti-smartctl-storm); the CHECK fetches FRESH (`fetchDisks`). Test seams: `Server.disksFn` (source) + `Server.diskNotifyFn` (sink). Baseline is in-memory (restart re-baselines) | ### Notifications / hub sync diff --git a/controller/README.md b/controller/README.md index ecda5f0..ec81348 100644 --- a/controller/README.md +++ b/controller/README.md @@ -201,6 +201,22 @@ backups, monitoring and notifications. All Proxmox/disk operations are delegated ## Features +### Disk health — "Lemezek állapota" card + degradation alert (v0.169.0) + +The dashboard shows a per-physical-disk health card driven by the agent's SMART summary (serialized +into `/disks` from agent v0.94.0 — no new smartctl load; the controller only reads it). One pure +verdict function (`agentapi.DiskVerdictFor`) is the shared truth for the card chip and the check: + +- **Rendben** (PASSED, clean) · **Figyelmeztetés** (PASSED but reallocated/pending/offline-uncorrectable + or NVMe critical/media/percentage-used ≥ 90) · **Hiba** (FAILING) · **Nincs adat** (nil/UNKNOWN/old + agent — never alarms). +- The card fetches `/disks` through a **60 s TTL cache** (dashboard refresh-spam can't smartctl-storm + the host); an unreachable agent renders "Nincs adat" and the page still loads. +- A **6-hourly `disk-health-check`** emits `disk_health_degraded` (warn/critical) **only on a + degradation** vs an in-memory baseline — first run baselines silently, recovery/UNKNOWN never notify, + and a controller restart re-baselines silently. **No global banner** (deliberate): the card + email + carry it. The hub allowlist must include `disk_health_degraded`. + ### 1. App Management The controller manages Docker Compose stacks through a complete lifecycle: catalog sync, first-time deployment, runtime operations, and deletion. diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index 826bdf8..7f64eac 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -959,6 +959,13 @@ func main() { stackMgr.RecoverMigration(ctx) webServer.SetEncryptionKey(encKey) webServer.SetAppExporter(appExporter) + // Disk-health degradation check (v0.169.0): every 6h, compare each physical disk's SMART verdict + // against the in-memory baseline and emit disk_health_degraded on a degradation only (first run + // baselines silently; recovery/UNKNOWN never notify). Only on a provisioned guest (an agent to + // read /disks from); the check no-ops gracefully if the agent is unreachable. + if cfg.LocalAPI.Endpoint != "" { + sched.Every("disk-health-check", 6*time.Hour, webServer.RunDiskHealthCheck) + } // Browser .fab upload (v0.128.0): upload state is in-memory, so a restart strands the .part — // GC stray part files in every registered drive's exports dir at startup. webServer.CleanupStaleUploadParts() diff --git a/controller/internal/agentapi/client.go b/controller/internal/agentapi/client.go index 8105ffe..c611f08 100644 --- a/controller/internal/agentapi/client.go +++ b/controller/internal/agentapi/client.go @@ -322,6 +322,10 @@ type DiskInfo struct { // BoundUnderParent reports whether the drive's felhom-data is currently bound under the shared parent // (live + usable in the guest). The controller's drive-absent gate keys on this + State. BoundUnderParent bool `json:"bound_under_parent"` + // Smart is the per-disk SMART health (agent v0.94.0+), nil when the device exposes no SMART or the + // agent predates the field — the disk-health card + 6h degradation check feature-detect on this and + // render "Nincs adat" (never alarm) when nil. See DiskVerdictFor. + Smart *SmartSummary `json:"smart,omitempty"` } // FSUUID returns the raw filesystem UUID from a "uuid:<…>" DurableID, or "" if this disk's identity @@ -1003,14 +1007,31 @@ type ThinPoolFill struct { MetadataUsedFraction *float64 `json:"metadata_used_fraction"` } -// SmartSummary mirrors the agent's per-disk SMART health (only the fields the UI renders). Pointers -// are null when the device type does not expose that attribute. +// SmartSummary mirrors the agent's per-disk SMART health. Pointers are null when the device type +// does not expose that attribute (a null is "unknown / not-applicable", distinct from a real zero). +// The SATA set (reallocated/pending/offline-uncorrectable) and the NVMe set +// (critical_warning/media_errors/percentage_used) are both carried; a device populates only its own. type SmartSummary struct { - Health string `json:"health"` // PASSED | FAILING | UNKNOWN - TemperatureC *int `json:"temperature_c"` - PercentageUsed *int `json:"percentage_used"` // NVMe wear (%); null for SATA/USB + Health string `json:"health"` // PASSED | FAILING | UNKNOWN + TemperatureC *int `json:"temperature_c"` + PowerOnHours *int `json:"power_on_hours"` + // SATA attributes. + ReallocatedSectors *int `json:"reallocated_sectors"` + PendingSectors *int `json:"pending_sectors"` + OfflineUncorrectable *int `json:"offline_uncorrectable"` + // NVMe attributes. + CriticalWarning *int `json:"critical_warning"` + MediaErrors *int `json:"media_errors"` + PercentageUsed *int `json:"percentage_used"` // NVMe wear (%); null for SATA/USB } +// SMART health vocabulary (mirrors the agent's). +const ( + SmartPassed = "PASSED" + SmartFailing = "FAILING" + SmartUnknown = "UNKNOWN" +) + // StorageTarget mirrors the agent's GET /host/metrics storage_targets entry (the per-storage // capacity + health the monitoring view renders). It is a SUBSET of the agent's wire shape — only // the fields the UI reads; unknown JSON keys are ignored. diff --git a/controller/internal/agentapi/diskverdict.go b/controller/internal/agentapi/diskverdict.go new file mode 100644 index 0000000..185824a --- /dev/null +++ b/controller/internal/agentapi/diskverdict.go @@ -0,0 +1,87 @@ +package agentapi + +// DiskVerdict is the customer-facing disk-health verdict derived from a SmartSummary (v0.169.0). +// It is the SHARED source of truth for both the "Lemezek állapota" dashboard card and the 6-hourly +// degradation check — one pure function so the chip and the alert can never disagree. +type DiskVerdict int + +const ( + // DiskVerdictUnknown — no SMART data (nil / UNKNOWN / old agent). Renders "Nincs adat"; NEVER + // alarms and NEVER participates in degradation transitions (excluded both directions). + DiskVerdictUnknown DiskVerdict = iota + DiskVerdictOK // "Rendben" — PASSED, all counters clean + DiskVerdictWarn // "Figyelmeztetés" — PASSED but a wear/relocation counter is non-zero (or NVMe ≥90%) + DiskVerdictFail // "Hiba" — FAILING +) + +// percentageUsedWarn is the NVMe wear threshold (inclusive) at which a still-PASSED disk warns. +const percentageUsedWarn = 90 + +// DiskVerdictFor maps a SmartSummary to a verdict per the v0.169.0 rules: +// +// FAILING → Hiba +// PASSED + any(reallocated>0, pending>0, offline_unc>0, +// critical_warning>0, media_errors>0, +// percentage_used >= 90) → Figyelmeztetés +// PASSED otherwise → Rendben +// nil / UNKNOWN / empty → Nincs adat +func DiskVerdictFor(s *SmartSummary) DiskVerdict { + if s == nil || s.Health == "" || s.Health == SmartUnknown { + return DiskVerdictUnknown + } + if s.Health == SmartFailing { + return DiskVerdictFail + } + // Health == PASSED (or any non-empty non-FAILING value we treat as passing): inspect the counters. + if positive(s.ReallocatedSectors) || positive(s.PendingSectors) || positive(s.OfflineUncorrectable) || + positive(s.CriticalWarning) || positive(s.MediaErrors) || atLeast(s.PercentageUsed, percentageUsedWarn) { + return DiskVerdictWarn + } + return DiskVerdictOK +} + +// Label is the exact Hungarian customer copy for the verdict (shared by the card chip and the email). +func (v DiskVerdict) Label() string { + switch v { + case DiskVerdictOK: + return "Rendben" + case DiskVerdictWarn: + return "Figyelmeztetés" + case DiskVerdictFail: + return "Hiba" + default: + return "Nincs adat" + } +} + +// DegradedAttributes returns the human-readable Hungarian names of the attribute(s) that pushed a +// PASSED disk to Figyelmeztetés (empty for OK/Fail/Unknown) — for the alert body. FAILING is a +// whole-disk verdict with no single triggering counter, so it returns nil there. +func DegradedAttributes(s *SmartSummary) []string { + if s == nil { + return nil + } + var out []string + if positive(s.ReallocatedSectors) { + out = append(out, "áthelyezett szektorok") + } + if positive(s.PendingSectors) { + out = append(out, "függőben lévő szektorok") + } + if positive(s.OfflineUncorrectable) { + out = append(out, "javíthatatlan szektorok") + } + if positive(s.CriticalWarning) { + out = append(out, "kritikus figyelmeztetés") + } + if positive(s.MediaErrors) { + out = append(out, "adathordozó-hibák") + } + if atLeast(s.PercentageUsed, percentageUsedWarn) { + out = append(out, "elhasználódás") + } + return out +} + +func positive(p *int) bool { return p != nil && *p > 0 } +func atLeast(p *int, n int) bool { return p != nil && *p >= n } diff --git a/controller/internal/agentapi/diskverdict_test.go b/controller/internal/agentapi/diskverdict_test.go new file mode 100644 index 0000000..a78f1b3 --- /dev/null +++ b/controller/internal/agentapi/diskverdict_test.go @@ -0,0 +1,63 @@ +package agentapi + +import "testing" + +func ip(v int) *int { return &v } + +// Verdict table (Part 2). Red-proof: change the PercentageUsed boundary from `>= 90` to `> 90` in +// DiskVerdictFor → the "NVMe percentage_used exactly 90 → Figyelmeztetés" case fails. +func TestDiskVerdictFor(t *testing.T) { + cases := []struct { + name string + in *SmartSummary + want DiskVerdict + }{ + {"nil → unknown", nil, DiskVerdictUnknown}, + {"empty health → unknown", &SmartSummary{Health: ""}, DiskVerdictUnknown}, + {"UNKNOWN → unknown", &SmartSummary{Health: SmartUnknown}, DiskVerdictUnknown}, + {"FAILING → fail", &SmartSummary{Health: SmartFailing}, DiskVerdictFail}, + {"FAILING beats counters", &SmartSummary{Health: SmartFailing, ReallocatedSectors: ip(0)}, DiskVerdictFail}, + {"PASSED clean → ok", &SmartSummary{Health: SmartPassed, ReallocatedSectors: ip(0), PendingSectors: ip(0), TemperatureC: ip(30)}, DiskVerdictOK}, + {"PASSED nil counters → ok", &SmartSummary{Health: SmartPassed}, DiskVerdictOK}, + {"reallocated>0 → warn", &SmartSummary{Health: SmartPassed, ReallocatedSectors: ip(1)}, DiskVerdictWarn}, + {"pending>0 → warn", &SmartSummary{Health: SmartPassed, PendingSectors: ip(5)}, DiskVerdictWarn}, + {"offline_unc>0 → warn", &SmartSummary{Health: SmartPassed, OfflineUncorrectable: ip(2)}, DiskVerdictWarn}, + {"critical_warning>0 → warn", &SmartSummary{Health: SmartPassed, CriticalWarning: ip(1)}, DiskVerdictWarn}, + {"media_errors>0 → warn", &SmartSummary{Health: SmartPassed, MediaErrors: ip(3)}, DiskVerdictWarn}, + {"percentage_used 89 → ok", &SmartSummary{Health: SmartPassed, PercentageUsed: ip(89)}, DiskVerdictOK}, + {"percentage_used exactly 90 → warn", &SmartSummary{Health: SmartPassed, PercentageUsed: ip(90)}, DiskVerdictWarn}, + {"percentage_used 95 → warn", &SmartSummary{Health: SmartPassed, PercentageUsed: ip(95)}, DiskVerdictWarn}, + } + for _, c := range cases { + if got := DiskVerdictFor(c.in); got != c.want { + t.Errorf("%s: DiskVerdictFor = %d, want %d", c.name, got, c.want) + } + } +} + +func TestDiskVerdict_Label(t *testing.T) { + want := map[DiskVerdict]string{ + DiskVerdictUnknown: "Nincs adat", + DiskVerdictOK: "Rendben", + DiskVerdictWarn: "Figyelmeztetés", + DiskVerdictFail: "Hiba", + } + for v, w := range want { + if got := v.Label(); got != w { + t.Errorf("verdict %d Label = %q, want %q", v, got, w) + } + } +} + +// A warn lists every triggering attribute at once (Scenario "multiple attributes degrade" → ONE event). +func TestDegradedAttributes_ListsAll(t *testing.T) { + s := &SmartSummary{Health: SmartPassed, PendingSectors: ip(5), ReallocatedSectors: ip(2), PercentageUsed: ip(91)} + got := DegradedAttributes(s) + if len(got) != 3 { + t.Fatalf("want 3 attributes, got %d: %v", len(got), got) + } + // clean disk → none + if a := DegradedAttributes(&SmartSummary{Health: SmartPassed}); len(a) != 0 { + t.Errorf("clean disk should list no attributes, got %v", a) + } +} diff --git a/controller/internal/notify/disk_health_test.go b/controller/internal/notify/disk_health_test.go new file mode 100644 index 0000000..2b873f1 --- /dev/null +++ b/controller/internal/notify/disk_health_test.go @@ -0,0 +1,44 @@ +package notify + +import ( + "strings" + "testing" +) + +// NotifyDiskHealthDegraded emits the right event type + severity + a message carrying the required +// "Lemez állapot romlás: