Image built and pushed to the registry BEFORE this manifest bump lands, so a sync can never point at a missing tag. Auto-sync is off; the sync that follows is deliberate. Never kubectl set image.
This commit is contained in:
@@ -1,3 +1,56 @@
|
||||
## v0.108.0 — only the first broken app per hour reached the operator (2026-08-23, R-389)
|
||||
|
||||
**The hour is unchanged. The GRAIN is what was wrong.**
|
||||
|
||||
`processOperator` keys its 1-hour cooldown on `customerID:eventType`, plus the tier and run suffixes.
|
||||
**None of them names an app.** So every app that went down inside the same hour collapsed onto one
|
||||
key and only the first was mailed. Measured live on `demo-hp` 2026-08-23: `bookstack` alarmed at
|
||||
09:27:51 and was `sent`; `privatebin` alarmed four minutes later and was recorded
|
||||
`suppressed — operator cooldown 1h, key=demo-hp:app_start_failed`. **Three apps dying together
|
||||
produced one mail.**
|
||||
|
||||
The app's identity was already on the wire — `AppDetails{StackName, DisplayName}` serialises as
|
||||
`stack_name` — and the hub already made exactly this distinction twice, for backup tiers and for
|
||||
backup runs.
|
||||
|
||||
### The fix: a third sibling, allow-listed to ONE event type
|
||||
|
||||
`cooldownStackSuffix` follows `cooldownTierSuffix` and `cooldownRunSuffix` exactly: guard on the
|
||||
substring, unmarshal a one-field struct, return `""` on any doubt. A **separate function**, for the
|
||||
reason `cooldownRunSuffix`'s docstring already gives — the existing two keep byte-identical semantics
|
||||
for every type that uses them, so R-97a's and R-182's behaviour and their tests are untouched.
|
||||
|
||||
**It takes the event type as well as the details, unlike its siblings, and that asymmetry is the whole
|
||||
safety property.** `tier` and `run_id` appear only on types that want that grain; `stack_name` does
|
||||
not have that property, so a payload-shape rule would have been wrong.
|
||||
|
||||
**`perAppCooldownEvents` is a named register with `app_start_failed` and nothing else.** The backup
|
||||
family's cooldown is coarse **on purpose** — R-97a and R-182 exist so one full disk sends one digest
|
||||
rather than one mail per app. **This is not hypothetical: `crossdrive_failed` is severity `error`,
|
||||
reaches the operator leg, and carries `stack_name`** through a different struct (`CrossDriveDetails`,
|
||||
not `AppDetails`). A global suffix would have split it into one mail per app, silently.
|
||||
|
||||
**`app_start_failed` gets per-app because it has no digest.** There is no `apps_down_run` summarising
|
||||
a scan the way `backup_run_failures` summarises a run, so per-app is the only grain available that
|
||||
does not lose alarms.
|
||||
|
||||
**Fail-soft:** absent, empty or malformed details return `""`, the key degrades to v0.107.0's, and the
|
||||
mail still goes. Losing an alarm is worse than mis-routing one.
|
||||
|
||||
### Tests
|
||||
|
||||
`internal/notify/r389_cooldown_grain_test.go`. Test count **709 → 716**.
|
||||
|
||||
Scenario A asserts **two `sent` operator rows in the stored notification log**, not a return value,
|
||||
and runs a positive control first — the operator leg must be shown delivering before any count means
|
||||
anything. Scenario C's "no other key changed" is an absence claim and carries its own control: the
|
||||
test proves it can *see* a key change before reporting that none occurred.
|
||||
|
||||
**Red-proofs, both seen failing.** Dropping the suffix from the key reproduces the live symptom
|
||||
verbatim (`1 operator mail(s), want 2`, and a suppression row reading `key=c1:app_start_failed`).
|
||||
Removing the allow-list splits `crossdrive_failed`, `app_deployed`, `app_removed` and `backup_failed`
|
||||
per app — the fence convicting exactly the types it was written for.
|
||||
|
||||
## v0.107.0 — the hub rewrote a severity and said nothing, and the guard for that sat downstream of the rewrite (2026-08-23, R-387)
|
||||
|
||||
**One handler, two fields, opposite discipline.** An unknown **event type** is rejected with a loud
|
||||
|
||||
+1
-1
@@ -125,7 +125,7 @@ spec:
|
||||
spec:
|
||||
containers:
|
||||
- name: hub
|
||||
image: gitea.dooplex.hu/admin/felhom-hub:0.107.0
|
||||
image: gitea.dooplex.hu/admin/felhom-hub:0.108.0
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
name: http
|
||||
|
||||
Reference in New Issue
Block a user