hub v0.89.0 — the two halves of decision D-c (R-167, R-158)
gates / gates (push) Successful in 7s

New OPERATOR-ONLY event type recovery_unit_capture_failed (controller
v0.191.0, R-158): in allowedEventTypes AND notify.operatorOnlyEvents.
Deliberately not a reuse of backup_failed, which carries customer copy and
sits in the controller's DefaultEnabledEvents — reusing it would email the
customer in Hungarian about a failure they cannot act on. R-158's own
proposal said backup_failed; D-c overrides it.

disk_warning/disk_critical lose their generic customerMessages entries.
Both were allowlisted, copy'd, default-enabled and checkbox'd with NO
producer anywhere; controller v0.191.0 becomes that producer and sends a
DYNAMIC Hungarian message naming the drive and its free space.
FormatCustomerEmail prefers the entry over the message, so keeping a static
entry would discard the label and the byte figures — the same reason
offbox_enlarge_blocked and disk_health_degraded have none. The deletion is
pinned by a test.

New notify.IsOperatorOnly so the api package can pin BOTH registers of a new
event type in ONE test; allowlisted-but-not-operator-only is invisible when
they are checked separately, and it is the defect v0.78.0 shipped. The
register itself stays unexported.

REUSE.md's "new event type" extension point rewritten: it told readers to
always add a customerMessages entry, which is wrong for operator-only types
and harmful for dynamic-message ones.

Tests 574 -> 579. Red-proof: removing the operatorOnlyEvents entry shows the
customer being emailed; the skipped/operator_only row is asserted as a
positive observable.
This commit is contained in:
2026-08-02 23:19:13 +02:00
parent 8ef92a3fa7
commit 179dd79882
7 changed files with 259 additions and 4 deletions
+6 -1
View File
@@ -189,7 +189,12 @@
## 5. Extension points (where new features plug in)
- **New event type**: add to `allowedEventTypes` (hub/internal/api/handler.go ~L1063) **and** `customerMessages` (hub/internal/notify/templates.go) **and** the customer-prefs default list if customer-notifiable. Missing the first = controller POST 400s (the known gotcha).
- **New event type — THREE registers, and which ones depend on the AUDIENCE.** Always: `allowedEventTypes` (hub/internal/api/handler.go) — missing it means the controller's POST 400s and the event vanishes (the known gotcha). Then decide the audience and stop guessing from the other registers:
- **Operator-only** → add to `notify.operatorOnlyEvents` (hub/internal/notify/dispatcher.go) and give it **no** `customerMessages` entry. **Allowlisting alone does NOT make a type operator-only**`FormatCustomerEmail` treats a missing `customerMessages` entry as a *fallback to the raw message*, not a block, and the only customer gate is configuration. v0.78.0 asserted the opposite in a comment and shipped the defect (R-97c). Examples: `whole_guest_backup_failed`, `recovery_unit_capture_failed`.
- **Customer-facing with a STATIC message** → add a `customerMessages` entry (hub/internal/notify/templates.go) and the controller's `settings.DefaultEnabledEvents` if it should be on by default.
- **Customer-facing with a DYNAMIC message** (the producer builds Hungarian text carrying names/numbers) → deliberately **no** `customerMessages` entry: `FormatCustomerEmail` PREFERS the entry over the message, so adding one silently discards the specifics. Examples: `offbox_enlarge_blocked`, `disk_health_degraded`, and since v0.89.0 `disk_warning`/`disk_critical`.
- Pin BOTH registers in ONE test (hub/internal/api/recovery_unit_event_test.go is the model) — fixing one and not the other is the realistic mistake, and `notify.IsOperatorOnly` exists so the api package can assert it.
- **A type in these registers with no PRODUCER is inert.** `disk_warning`/`disk_critical` were allowlisted, copy'd, default-enabled and checkbox'd from early on, and nothing in any repo emitted them until controller v0.191.0 — grep for an emitter before assuming a type works.
- **New monitor checker**: copy hub/internal/monitor/staleness.go (§2 pattern); wire in hub/cmd/hub/main.go with an `EventNotifyFunc`; severity must be warning/error/critical to notify.
- **New API route**: switch in `api.ServeHTTP` (handler.go ~L139); auth helper first line.
- **New web page/action**: switch in `web.ServeHTTP` (server.go ~L182) — non-GET gets CSRF automatically; template into hub/internal/web/templates/ (embedded FS, parsed in `web.New`); new helpers into the funcMap (server.go ~L67).
+33
View File
@@ -1,3 +1,36 @@
## v0.89.0 — the two halves of decision D-c (2026-08-02, R-167 · R-158)
**Decision D-c routes two new signals to two different audiences, and the hub is where that routing
is enforced.** A customer can free space, delete files or add a drive, so a FILL WARNING is theirs. A
customer can do nothing about a per-app backup capture failure, so it is not.
**New operator-only event type `recovery_unit_capture_failed`** (controller v0.191.0, R-158). Added to
`allowedEventTypes` **and** to `notify.operatorOnlyEvents`. Deliberately NOT a reuse of
`backup_failed`, which carries a `customerMessages` entry and sits in the controller's
`DefaultEnabledEvents` — reusing it would email the customer, in Hungarian, about a failure they
cannot act on. R-158's original proposal named `backup_failed`; D-c overrides it.
**`disk_warning` / `disk_critical` lose their generic `customerMessages` entries.** These two types
were allowlisted here, carried Hungarian copy, sat in the controller's default enabled events and had
a UI checkbox — and **nothing in any repo emitted them**, a complete customer pipeline with no
producer. Controller v0.191.0 becomes that producer, and it sends a **dynamic** Hungarian message
naming the drive and its free space. A static entry would be actively harmful: `FormatCustomerEmail`
PREFERS the entry over the message, so keeping one would discard the label and the byte figures and
leave the customer with *"A lemezterület 90% felett van"* — a warning with nothing to act on. Same
reason `offbox_enlarge_blocked` and `disk_health_degraded` have no entry. The deletion itself is
pinned by a test.
**New `notify.IsOperatorOnly`** — a read-only accessor so the `api` package can pin BOTH registers of
a new event type in ONE test. Allowlisted-but-not-operator-only is invisible when the two are checked
separately, and it is the defect v0.78.0 actually shipped. The register stays unexported so nothing
can widen it at runtime.
**Tests:** 574 → **579**. The operator half is proven through the real dispatch path under the
*breaking* configuration — the customer has the event enabled and has an email address — because that
is the only configuration in which a missing `operatorOnlyEvents` entry is visible. Red-proof:
removing the entry shows the customer being emailed, and the `skipped/operator_only` log row is
asserted as a positive observable rather than inferred from an absent delivery.
## v0.88.0 — the WAL that never was (2026-08-02, R-172)
**The hub has never actually been in WAL mode.** `store.New` opened the database with
+14
View File
@@ -1573,6 +1573,20 @@ var allowedEventTypes = map[string]bool{
"whole_guest_backup_failed": true,
"whole_guest_backup_recovered": true,
// R-158 / R-167 (controller v0.191.0, decision D-c): a per-app Tier-1 recovery-unit capture
// failed. Until then a `[WARN]` line in the controller reached no hub channel at all — the fifth
// instance in this project of a mechanism built and left disconnected.
//
// DELIBERATELY NOT `backup_failed`, for exactly the reason recorded above for the whole-guest
// pair: that type carries a customerMessages entry AND sits in the controller's
// DefaultEnabledEvents, so reusing it emails the CUSTOMER, in Hungarian, about a failure they
// cannot act on. R-158's original proposal named `backup_failed`; D-c routes this to the
// operator, and where the two disagree D-c wins.
//
// OPERATOR-ONLY IS ENFORCED BY `notify.operatorOnlyEvents` — see the paragraph above. This entry
// alone does NOT make it operator-only.
"recovery_unit_capture_failed": true,
// Controller-pushed events
"controller_started": true,
"claim_lockout": true, // v0.50.0 — claim/reset code brute-force lockout tripped
@@ -0,0 +1,50 @@
package api
import (
"testing"
"gitea.dooplex.hu/admin/felhom-hub/internal/notify"
)
// R-158 / R-167 (D-c) — the per-app Tier-1 recovery-unit capture failure alert.
//
// A new event type is a PAIR of registers, and each half fails differently:
// - missing from allowedEventTypes → POST /event returns 400 and the event VANISHES (R-97a);
// - missing from operatorOnlyEvents → it is delivered TO THE CUSTOMER, in raw English, about a
// failure they can take no action on.
//
// The second is the quiet one — delivery "works", so nothing looks broken. It is also the defect
// v0.78.0 actually shipped: adding a type to allowedEventTypes and ASSUMING that made it
// operator-only. `FormatCustomerEmail` treats a missing customerMessages entry as a fallback to the
// raw message, not a block, and the only customer gate is configuration. Both halves are pinned
// here, in the same test, because fixing one and not the other is the realistic mistake.
func TestRecoveryUnitCaptureFailedIsAllowlistedAndOperatorOnly(t *testing.T) {
const et = "recovery_unit_capture_failed"
if !allowedEventTypes[et] {
t.Fatalf("%s must be in allowedEventTypes, or POST /event 400s and a per-app Tier-1 backup "+
"failure reaches no hub channel at all — which is the R-158 gap, un-fixed", et)
}
if !notify.IsOperatorOnly(et) {
t.Fatalf("%s is allowlisted but NOT in notify.operatorOnlyEvents — the customer would be "+
"emailed about a recovery-unit capture failure they can take no action on. Adding a type "+
"to allowedEventTypes does NOT make it operator-only; that was the v0.78.0 defect "+
"(corrected in v0.79.0/R-97c) and this is the same mistake one event later", et)
}
}
// The customer's half of D-c must NOT be operator-only — a fill warning is precisely the alert a
// customer CAN act on (free space, delete files, add a drive). Pinned in the same file as its
// operator sibling so the routing decision is read as one thing, which is what D-c is.
func TestFillWarningReachesTheCustomer(t *testing.T) {
for _, et := range []string{"disk_warning", "disk_critical"} {
if !allowedEventTypes[et] {
t.Fatalf("%s must stay in allowedEventTypes — it is the customer fill warning the "+
"controller emits from v0.191.0 (R-167)", et)
}
if notify.IsOperatorOnly(et) {
t.Fatalf("%s is in operatorOnlyEvents — the customer would never be warned that their "+
"disk is filling, which is the whole customer half of decision D-c", et)
}
}
}
+12
View File
@@ -315,8 +315,20 @@ var operatorOnlyEvents = map[string]bool{
// a consequence of another type's routing — true today, and silently untrue the moment the failed
// event becomes customer-visible. Belt, not inference.
"whole_guest_backup_recovered": true,
// R-158 / R-167 (D-c). A per-app Tier-1 recovery-unit capture failure. A customer can take no
// action on it — the causes are a full filesystem, a permission fault or a broken dump, all of
// which the operator resolves — and the alert carries operator-grade detail (target path, byte
// figures, the raw error). The customer's half of D-c is the FILL WARNING, which fires BEFORE
// this and is actionable: free space, delete files, add a drive.
"recovery_unit_capture_failed": true,
}
// IsOperatorOnly reports whether an event type is barred from customer dispatch. Exported so the
// api package can pin BOTH registers of a new event type in one test — allowlisted-but-not-
// operator-only is the v0.78.0 defect, and it is only visible when the two are checked together.
// Read-only: the register itself stays unexported so nothing can widen it at runtime.
func IsOperatorOnly(eventType string) bool { return operatorOnlyEvents[eventType] }
func (d *Dispatcher) processCustomer(customerID, eventType, severity, message, detailsJSON, source string) {
// R-97c: operator-tier events stop here, BEFORE prefs are consulted — the point is that no
// customer configuration can opt in. Logged rather than dropped, so the skip is visible in
@@ -0,0 +1,130 @@
package notify
import (
"io"
"log"
"strings"
"testing"
)
func discardLogger() *log.Logger { return log.New(io.Discard, "", 0) }
// R-158 / R-167 Scenario G — the two new signals of decision D-c sit on OPPOSITE sides of the
// operator/customer line, and each is tested through the real dispatch path rather than by reading
// the register.
//
// D-c's rule, restated: a customer can free space, delete files or add a drive, so a FILL WARNING is
// theirs. A customer can do nothing about a recovery-unit capture failure, so it is not.
// The operator half. Run under the BREAKING configuration — the customer has the event explicitly
// enabled and has an email address — because that is the only configuration in which a missing
// operatorOnlyEvents entry is visible. This is the v0.78.0 defect, demonstrated rather than argued.
func TestRecoveryUnitCaptureFailed_NeverReachesTheCustomer(t *testing.T) {
st := opOnlyStore(t)
if err := st.SaveNotificationPrefs("c1", "customer@example.com",
[]string{"recovery_unit_capture_failed", "backup_failed"}, 6); err != nil {
t.Fatalf("SaveNotificationPrefs: %v", err)
}
rec := &sentTo{}
d := opOnlyDispatcher(t, st, rec)
d.ProcessEvent("c1", "recovery_unit_capture_failed", "error",
`Recovery unit capture FAILED for "immich" — the app has no fresh local (Tier-1) backup`,
`{"app":"immich","used_percent":100,"space_known":true}`, "controller")
for _, to := range rec.to {
if to == "customer@example.com" {
t.Fatalf("a customer was emailed the OPERATOR-ONLY recovery_unit_capture_failed (%s) — "+
"enabled_events must not be able to opt in to a failure they cannot act on", to)
}
}
// The operator must still get it: the register mutes the customer channel, not the signal.
gotOperator := false
for _, to := range rec.to {
if to == "operator@felhom.eu" {
gotOperator = true
}
}
if !gotOperator {
t.Fatal("the operator was not notified of a recovery-unit capture failure — the alert is the " +
"whole point of R-158 and it went nowhere")
}
// The skip must be VISIBLE. An absent log row is equally consistent with "correctly skipped" and
// "the dispatcher never ran" — the positive observable is the row itself (standing rule 3).
logs, err := st.GetRecentNotifications("c1", 20)
if err != nil {
t.Fatalf("GetRecentNotifications: %v", err)
}
found := false
for _, l := range logs {
if l.Channel == "customer" && l.Status == "skipped" && strings.Contains(l.ErrorMessage, "operator_only") {
found = true
}
}
if !found {
t.Fatalf("the customer skip is not logged as skipped/operator_only — it is indistinguishable "+
"from a delivery that never happened; got %d row(s)", len(logs))
}
}
// The customer half. The fill warning MUST be delivered, and it must render the controller's own
// Hungarian text — which names the drive and the free space — rather than a generic template or the
// raw English.
func TestDiskFillWarning_ReachesTheCustomerInHungarianWithItsNumbers(t *testing.T) {
st := opOnlyStore(t)
if err := st.SaveNotificationPrefs("c1", "customer@example.com", []string{"disk_warning"}, 6); err != nil {
t.Fatal(err)
}
// The exact shape the controller sends from v0.191.0.
const hun = `A(z) „Fotók" tároló 87%-osan megtelt — 4,2 GB szabad hely maradt. ` +
`Szabadíts fel helyet (törölj felesleges fájlokat, vagy csatlakoztass új meghajtót), ` +
`különben a biztonsági mentések hamarosan meghiúsulnak.`
var bodies []string
d := NewDispatcher(st, "test-key", "from@felhom.eu", "operator@felhom.eu", true, discardLogger())
d.sendEmailFn = func(to, subject, body string, headers map[string]string) error {
if to == "customer@example.com" {
bodies = append(bodies, subject+"\n"+body)
}
return nil
}
d.ProcessEvent("c1", "disk_warning", "warning", hun,
`{"label":"Fotók","used_percent":87,"avail_gb":4.2}`, "controller")
if len(bodies) == 0 {
t.Fatal("the customer was NOT warned that a disk is filling — this is the customer half of " +
"decision D-c, and nothing reached them")
}
got := strings.Join(bodies, "\n")
// The controller's dynamic text must survive. A customerMessages entry for disk_warning would
// OVERRIDE it (templates.go prefers the entry) and discard the drive name and the free space,
// leaving the customer with "A lemezterület 90% felett van" and nothing to act on — which is
// exactly why v0.191.0 removes those two entries.
if !strings.Contains(got, "Fotók") {
t.Fatalf("the customer email does not name the drive — a generic customerMessages entry has "+
"discarded the controller's dynamic text (templates.go priority). Got:\n%s", got)
}
if !strings.Contains(got, "4,2 GB") {
t.Fatalf("the customer email does not carry the free-space figure. Got:\n%s", got)
}
if !strings.Contains(got, "Szabadíts fel helyet") {
t.Fatalf("the customer email does not tell the customer what to DO. Got:\n%s", got)
}
}
// The generic entries must STAY REMOVED. A well-meaning re-add would silently re-break the test
// above's guarantee for every future reader — this pins the deletion itself.
func TestDiskFillTypesHaveNoGenericCustomerMessage(t *testing.T) {
for _, et := range []string{"disk_warning", "disk_critical"} {
if msg, ok := customerMessages[et]; ok {
t.Fatalf("customerMessages[%q] = %q — a static entry OVERRIDES the controller's dynamic "+
"Hungarian text (FormatCustomerEmail prefers the entry), discarding the drive name "+
"and the free-space figure the customer needs. Same reason offbox_enlarge_blocked "+
"and disk_health_degraded deliberately have none", et, msg)
}
}
}
+14 -3
View File
@@ -71,9 +71,20 @@ var customerMessages = map[string]string{
"offbox_repo_orphaned": "A távoli mentési tároló elárvult: a benne lévő mentések egy korábbi, már nem elérhető kulccsal készültek (jellemzően újratelepítés után). Új mentés a tároló visszaállításáig nem készül — nyisd meg a Távoli mentés oldalt.",
"offbox_repo_reset": "A távoli mentési tároló visszaállítva: a régi előzmény félretéve (nem törölve), és egy üres, új tároló jött létre a mostani kulccsal.",
// Disk events (GUEST — the controller's own cgroup view)
"disk_warning": "A lemezterület 90% felett van — kérjük, szabadíts fel helyet.",
"disk_critical": "A lemezterület kritikusan magas (95%+) — azonnali beavatkozás szükséges!",
// Disk events (GUEST — the controller's own view) — `disk_warning` / `disk_critical`.
//
// DELIBERATELY NO ENTRY, from hub v0.89.0 / controller v0.191.0 (R-167, decision D-c). These two
// types were allowlisted here, carried generic Hungarian copy, sat in the controller's
// DefaultEnabledEvents and had a UI checkbox — and NOTHING IN ANY REPO EMITTED THEM. A complete
// customer pipeline with no producer; the sixth "built but never wired" instance in this project.
// The controller became their producer in v0.191.0.
//
// The producer sends a DYNAMIC Hungarian message naming the filesystem and its free space, so a
// static entry here would be actively harmful: FormatCustomerEmail PREFERS the entry over the
// message, so re-adding one would discard the drive name and the byte figures and leave the
// customer with "A lemezterület 90% felett van" — a warning with nothing to act on. Same reason
// `offbox_enlarge_blocked` and `disk_health_degraded` have no entry. Pinned by
// TestDiskFillTypesHaveNoGenericCustomerMessage.
// Host disk events (the Proxmox HOST root filesystem — distinct from the guest disk above)
"host_disk_warning": "A házszerver alaprendszerének (Proxmox-gazda) gyökérlemeze 90% felett van — kérjük, szabadíts fel helyet (pl. régi biztonsági mentések).",