admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(R-329/R-386/R-387): the severity contract, the intent ruling, and Part 5 recorded
gates / gates (push) Successful in 17s
Details

Browse Source 
The alarm ladder gains the severity contract (the hub's vocabulary is exact, it
coerces silently, and three things now hold it) and the intent test with its
three-way ruling on unknown. Both marked [DESIGN] with the live measurements.

Part 5 is RECORDED AND NOT IMPLEMENTED: the operator's notification philosophy,
verbatim, marked plainly as direction rather than current behaviour, with the
12 -> 15 toggle growth as the argument. Filed as R-388, a product decision.

R-329 and R-386 compressed into CLOSED-ITEMS with their rules kept and the
full-text commit named. R-387 filed closed - including WHY the dispatcher branch
was kept rather than deleted, which is evidence (three monitor checkers call
ProcessEvent directly) and not caution.

The drill record names three things that had to be re-run: an inert red-proof
mutation, Scenario G refused twice behind an HTTP 200, and the live Scenario A
NOT proving the customer gate because demo-hp has no prefs row at all.

Register: OPEN 328325 -> 328132 B, CLOSED 71441 -> 74642 B.
This commit is contained in:
authentik Default Admin
2026-08-23 12:03:49 +02:00
parent 68a9f5475c
commit 2f7c9a6ce5
33 changed files with 5774 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files
REPORT.md
+91 -68
View File
@@ -1,97 +1,120 @@
# REPORT — felhom.eu: the golden-currency gate could not see an unrecorded golden (R-385)
# REPORT — felhom.eu: hub v0.107.0 (R-387), the alarm ladder, and golden 0.223.0
**Session 2026-08-23.** Companion to `felhom-controller` v0.222.0 (R-384, R-383) — see that repo's
**Session 2026-08-23.** Companion to `felhom-controller` v0.223.0 (R-329, R-386) — see that repo's
`REPORT.md` for the controller work and the full live walk.
## What was wrong here
## 1. Baselines, and the hub's four numbers as read
`scripts/golden_currency_gate.py` asked ONE question — *is the golden BEHIND the record?* — and
therefore could only ever catch a forgotten bake. **It said nothing when the golden was AHEAD of the
record**, and that direction is not harmless: a golden ahead of every CHANGELOG heading was built
from something never written down.
| Repo | at start | at end |
|---|---|---|
| felhom.eu | `55274d5e` | hub **v0.107.0** deployed |
| felhom-controller | `14137efa` (v0.222.0) | **v0.223.0** deployed |
| felhom-agent | `40d857b5` | untouched |
That is not hypothetical. Controller **0.221.1** was built, baked **and vouched** on 2026-08-23 while
the newest heading in the controller CHANGELOG still read `v0.221.0`. Measured on the real history,
with the old gate:
**Hub's four numbers, read live from `GET /configuration` before starting:**
`golden_version` **0.222.0** · `agent_version` **0.130.0** · `min_agent` **0.129.0** ·
controller floor **0.222.0**. All four as the task predicted; the operator's 0.222.0 vouch had landed.
```
newest released controller : 0.221.0 (## v0.221.0 — taking the undo copy destroyed …)
newest golden baked : 0.221.1 (documentation/tests/golden-0.221.1-2026-08-23)
golden currency gate OK …
EXIT=0
```
## 2. The hub's deployment path — §6's premise was wrong, and here it is
Every gate was green while the fleet ran a version the record did not name.
**`felhom.eu/manifests/hub.yaml`, line 128.** ArgoCD `Application/felhom` tracks
`https://gitea.dooplex.hu/admin/felhom.eu.git`, path `manifests`, with `syncPolicy.automated.enabled
= false`. Bumped `0.106.0 → 0.107.0` in commit **`68a9f54`**. **There is no out-of-git deployment
path** — the finding §6 braced for does not exist. The image was built and pushed to the registry
**before** the manifest landed, so a sync could never have pointed at a missing tag, and the sync was
then requested deliberately (`refresh=hard`, then a patched `operation`). Never `kubectl set image`.
## The fix, and why it is membership and not a comparison
## 3. R-387 — what was wrong
The gate now asks **"is the version we are shipping WRITTEN DOWN?"** — the baked version must have its
own `## vX.Y.Z` heading **anywhere** in the controller CHANGELOG, not merely at the top (an entry may
legitimately be overtaken by later ones; what may never happen is that it is absent).
One handler, two fields, opposite discipline: an unknown `event_type` is rejected with a loud `400`;
an unknown `severity` was rewritten to `info` **without a word**, and `severityNotifies` drops `info`
before *both* legs. **The guard built to catch exactly this sat downstream of the rewrite** — the
dispatcher's `unrecognized severity` line can never execute for an API event, because the coercion one
line earlier guarantees the value it looks for cannot arrive.
**Membership, not `baked > released`, deliberately:** a comparison against the newest heading alone
goes green the moment ANY later entry is written — which would have left 0.221.1 permanently
unrecorded and the gate permanently silent about it.
**Measured on the live hub DB:** `91` `app_start_failed` events stored all-time, **`0`
`notification_log` rows before this session** — not one, on any channel, while every POST returned 200.
Preserved unchanged: **INCONCLUSIVE (exit 2)** for an absent clone or an unparseable CHANGELOG — *not
knowing is never a pass, and never a conviction*. Every refusal names a reason **and** a route,
including what to do if a bake was a throwaway that must never be delivered.
**The coercion stays.** A rejected event is a *lost* event, and losing an alarm is worse than
mis-routing one. Only the silence is fixed.
## Red-proofs — both directions, against the real history
## 4. The dead-branch decision, and the reason
| Run | Gate | CHANGELOG | Golden baked | Exit | |
|---|---|---|---|---|---|
| `gate-01` | **old** | v0.221.0 | 0.221.1 | **0** | the blindness, reproduced |
| `gate-02` | **new** | v0.221.0 | 0.221.1 | **1** | convicted |
| `gate-03` | new | v0.221.1 | 0.221.1 | **0** | Part 0's heading makes it pass |
| `gate-04` | new | absent clone | — | **2** | INCONCLUSIVE preserved |
| `gate-05` | new | v0.222.0 | 0.222.0 | **0** | post-bake |
**KEPT.** Not caution — evidence. `cmd/hub/main.go` wires `dispatcher.ProcessEvent` **directly** as
the `monitor.EventNotifyFunc` for the staleness, host-staleness and offsite-box checkers, and those
hub-generated events never pass through the ingest handler at all. For every one of them that line is
the **only** severity guard there is. Deleting it as "dead" would have removed the live half while the
dead half supplied the justification.
Transcripts: `documentation/audits/DRILL-r384-dead-db-alarm-2026-08-23/evidence/gate-0*.txt`.
Verified while deciding: **all 90 severity literals in `internal/monitor` are already valid**, so the
guard is silent because the producers are correct. (`"warn"` in `internal/web` is UI badge vocabulary,
not a severity.)
## Files changed
## 5. Files changed, commits, CI
| File | Change |
| Commit | Contents |
|---|---|
| `scripts/golden_currency_gate.py` | the unrecorded-golden conviction; `newest_released` → `released_versions` returning the whole set; docstring records the second blindness |
| `documentation/architecture/08-alarm-ladder.md` | **NEW.** The alarm ladder as a dated [DESIGN] |
| `documentation/architecture/00-capability-map.md` | R-384 marked closed with its live evidence; points at the new doc |
| `documentation/backlog/OPEN-ITEMS.md` | R-383/R-384 removed (closed); **R-385** (closed) and **R-386** (open) filed |
| `documentation/backlog/CLOSED-ITEMS.md` | R-383 + R-384 compressed, each keeping its rules and naming `git show 1eb64bec5183:…` for the full text |
| `documentation/tests/golden-0.222.0-2026-08-23/` | **NEW.** Bake evidence + log + the vouching instructions |
| `STATUS.md` | the 0.222.0 vouch replaces the (now completed) 0.221.1 one; R-386 added in plain words |
| `documentation/audits/DRILL-r384-dead-db-alarm-2026-08-23/` | **NEW.** The drill record and 29 evidence files |
| **`68a9f54`** | hub v0.107.0 (ingest WARN + kept-branch note + tests), manifest bump, golden 0.223.0 evidence |
| **`<docs>`** | alarm ladder §6.1/§7/§8, register, `STATUS.md`, `REPORT.md`, drill record |
## The alarm ladder had no owning document — that absence is a finding
Files: `hub/internal/api/handler.go`, `hub/internal/notify/dispatcher.go`, `hub/CHANGELOG.md`,
`manifests/hub.yaml`, `documentation/architecture/08-alarm-ladder.md`,
`documentation/backlog/{OPEN,CLOSED}-ITEMS.md`, `documentation/tests/golden-0.223.0-2026-08-23/`,
`documentation/audits/DRILL-r329-r386-2026-08-23/`, plus two new test files.
Nothing in `documentation/architecture/` owned the question *"when does a customer's app being broken
raise an alarm?"* The rules lived as comments across four packages, each locally correct, with the
ordering between them legible only by reading `aggregateState` top to bottom. **That is precisely how
R-384 survived review**, and three separate defects in this ladder (R-51, C9-F2, R-384) were each
found on live hardware rather than by reading. `08-alarm-ladder.md` now owns it.
**CI runs confirmed BY ID** (`id` and `run_number` diverge — both printed): see §5 of the controller
REPORT for its runs; felhom.eu's are listed at the end of this file.
## Golden
## 6. Tests and red-proofs
**Baked and PUBLISHED: 0.222.0.** `GOLDEN_SHA256 = 19f5904f5379…`, `upload OK (HTTP 201)`, round-trip
`HTTP 206` from the package URL, all acceptance markers counted.
**VOUCHING IS THE OPERATOR'S ACT AND WAS NOT DONE HERE.**
`internal/api/r387_severity_visibility_test.go` — the event is **not lost**, the stored severity is
still `info`, and the WARN names customer + type + value; plus a guard that a **valid** severity stays
silent, because an alarm on the normal path is one people learn to ignore.
`internal/notify/r329_app_start_failed_test.go` — the routing consequence: operator emailed, customer
not, unless opted in, in which case both legs deliver and the customer's copy carries the Hungarian
template. Scenario B is also the **positive control** for Scenario A's absence claim.
**Deviation recorded:** the bake runbook's §4.1 is missing a `pveam update`. On the `virgin` snapshot
the template index is stale, so the listed template cannot be downloaded and the failure presents as
`400 Parameter verification failed. template: no such template` rather than as a stale index.
Test count **702 → 709**.
## Register size
**Red-proof (seen failing):** delete the ingest `WARN` → `the hub rewrote a severity and said
nothing`, with the log showing only the ordinary `[INFO] Event from c1: backup_failed (info)`. The
guard sits at **ingest**, because that is the last point at which the offending value still exists.
## 7. Golden
**Baked and PUBLISHED: 0.223.0.** `GOLDEN_SHA256 =
9eaf39ac39219b42ec9e6cbf890275febcdcc6f53325fe0c0f591d3431044f17`; `upload OK (HTTP 201)`; round-trip
**HTTP 206**; all five acceptance markers counted (`docker OK (overlay2` 1, `including mount point` 2,
`upload OK` 1, `excluding` 0, `FATAL` 0). **VOUCHING IS THE OPERATOR'S ACT AND WAS NOT DONE HERE.**
**Runbook deviation, second session running:** §4.1 omits `pveam update`, so the `virgin` snapshot's
stale template index fails as `400 … no such template`.
## 8. Scenario H, live against v0.107.0
```
[WARN] [api] Event from demo-hp: severity "warn" is not in {info,warning,error,critical} — coercing
to "info", which severityNotifies DROPS, so this backup_failed alert will reach NOBODY. Fix the
emitting controller; this event is stored but not routed.
```
Both the bad-severity POST and the `error` control returned **200** (nothing lost), and the control
produced **no** warning — the guard does not fire on the normal path.
## 9. Register size
| File | Before | After |
|---|---|---|
| `OPEN-ITEMS.md` | 327,266 B | **328,325 B** |
| `CLOSED-ITEMS.md` | 68,464 B | **71,441 B** |
| `OPEN-ITEMS.md` | 328,325 B | **328,132 B** |
| `CLOSED-ITEMS.md` | 71,441 B | **74,642 B** |
OPEN grew ~1 KB despite two closures, because R-386 is a substantial new finding. Recorded rather
than smoothed over.
R-329 and R-386 compressed into CLOSED with their rules kept; **R-387** (closed) and **R-388** (the
notification-model product decision, open, operator's call) filed.
## Hub numbers as read at session start (live, `GET /configuration`)
## 10. Observations — recorded, not acted on
`golden_version` **0.221.1** · `agent_version` **0.130.0** · `min_agent` **0.129.0** ·
controller floor **0.221.1**. The task expected 0.220.2/0.220.2; the operator had already vouched.
**The hub was READ ONLY this session** — nothing was written to it.
1. **The operator cooldown key carries no app identifier.** PrivateBin's alarm four minutes after
BookStack's was logged `suppressed — operator cooldown 1h, key=demo-hp:app_start_failed`, so **only
the first app-down per hour reaches the operator by e-mail**. R-182's known shape; harmless while
the event was undeliverable, and no longer. Not fixed here.
2. Two probe events remain as rows for `demo-hp` from Scenario H — inert, and named rather than left.