# REPORT — hub v0.108.0 (R-389), gate 11, and the instruction that invited the gap **Session 2026-08-23.** `felhom.eu` is the subject; the controller and agent were touched only to register the shared gate. **No controller release — no golden bake, no vouch, no floor.** **No halt condition fired.** Nothing was dropped. ## 1. Baselines, and the hub's four numbers as read | Repo | at start | at end | |---|---|---| | felhom.eu | `2f7c9a6` (hub v0.107.0) | **hub v0.108.0** deployed | | felhom-controller | `1da2c9c` (v0.223.0) | **unchanged** — runner registration only | | felhom-agent | `40d857b` (v0.130.0) | **unchanged** — runner registration only | **Hub's four numbers, live from `GET /configuration` before starting:** | Field | Value | |---|---| | `golden_version` | **0.223.0** | | `agent_version` | **0.130.0** | | `min_agent` | **0.129.0** | | controller floor (`min_controller_version`) | **0.222.0** | **The task predicted the floor at 0.223.0 and it reads 0.222.0.** The operator vouched the golden but has not yet raised the floor — the last step of the previous release, which `STATUS.md` says to do "last, in its own save". Carried forward as item 1 there. Not a halt; the correct reading is simply different from the prediction. **The golden-currency gate stayed green throughout** and no bake was needed, exactly as §1 said it should be: the controller CHANGELOG's newest entry and the newest baked golden are both 0.223.0 and this session moved neither. ## 2. Documents read `hub/internal/notify/dispatcher.go` (`cooldownTierSuffix` + `cooldownRunSuffix` docstrings in full, `processOperator` and its R-182 suppression-logging block, `operatorOnlyEvents`), `scripts/repo_gates.py` (whole docstring, including "WHY 10 IS HERE" and "WHY 7 IS HERE"), `scripts/due_checks_gate.py`, `documentation/PROMPT-TEMPLATE.md` §15, and the alarm ladder at **`documentation/architecture/08-alarm-ladder.md`** — extended here with §6.2, the delivery grain. ## 3. The `AppDetails` emitter count, measured **Three, exactly as §3 said.** `grep -rn "AppDetails{" --include=*.go` over the controller, excluding tests: | Emitter | Event | Severity | Reaches the operator leg? | |---|---|---|---| | `notifier.go:502` | `app_deployed` | `info` | no — `info` is dropped by `severityNotifies` | | `notifier.go:563` | `app_start_failed` | `warning` | **yes** | | `notifier.go:661` | `app_removed` | `info` | no | **No fourth emitter. No halt.** **But the sweep found something the `AppDetails` question could not:** `stack_name` is also carried by a **different struct**, `CrossDriveDetails` (`notifier.go:151-157`), used by `crossdrive_failed` (severity **`error`**, so it *does* reach the operator leg) and `crossdrive_completed`. This is what makes the allow-list load-bearing in fact rather than in principle — a payload-shape rule would have split a backup-family event per app and silently undone R-182. It is Scenario C's live subject. ## 4. Files, commits, CI | Commit | Repo | Contents | |---|---|---| | **`f751aea`** | felhom.eu | R-389 filed — **alone, before any code** (Phase 1) | | **`2fc4a15`** | felhom.eu | the suffix + allow-list + tests, gate 11, template fix, R-390/R-391 | | **`45659bd`** | felhom.eu | hub v0.108.0 CHANGELOG + manifest bump | | **`f8c9390`** | felhom-controller | gate 11 registered; its REPORT's observations marked up | | **`058b945`** | felhom-agent | gate 11 registered | **CI runs confirmed BY ID** — and the listing was checked for truncation rather than trusted, since a silently truncated listing has already cost this arc a false claim: | Commit | Repo | CI `id` | `run_number` | Result | |---|---|---|---|---| | `f751aea` | felhom.eu | **412** | 263 | success | | `2fc4a15` | felhom.eu | **413** | 264 | success | | `45659bd` | felhom.eu | **416** | 265 | success | | `f8c9390` | felhom-controller | **414** | 90 | success | | `058b945` | felhom-agent | **415** | 55 | success | The API reported `total_count` 265 / 91 / 55 against 3 rows shown in each case — i.e. the pages were known-partial and the newest rows are the ones quoted, not the whole set. ## 5. Red-proofs — two planted, both seen failing | # | Mutation | Layer, and why that layer | Observed | |---|---|---|---| | 1 | `cooldownStackSuffix` dropped from the key expression | **`processOperator`'s KEY** — where the collapse physically happens | `2 apps down inside the hour produced 1 operator mail(s), want 2 (suppressed=1)`, and the suppression row reads `key=c1:app_start_failed` — the live shape reproduced in a unit test | | 2 | the allow-list check removed from the suffix | **the REGISTER** — the fence that keeps the backup family coarse | `crossdrive_failed` … `= ":bookstack", want ""`, plus `app_deployed`, `app_removed`, `backup_failed` — the fence convicting exactly the types it was written for | Both mutations asserted their pre-fix text was present before rewriting and printed `MUTATION APPLIED`. **Neither passed first time**, and the check for that was explicit after yesterday's inert mutation. Gate 11 carries its own controls rather than a mutation, because the gate *is* the guard: ten cases in §Part 3 of the drill record, including the historical red-proof against yesterday's real file. ## 6. Test counts | Repo | Before | After | |---|---|---| | felhom.eu hub | 709 | **716** | `go build ./... && go vet ./... && go test ./...` in the hub → **exit 0, zero failures**. `python3 scripts/repo_gates.py --fast` → **12/12 OK** in felhom.eu; controller and agent runners both OK with gate 11 registered. ## 7. Deployed hub version, and the manifest commit **`gitea.dooplex.hu/admin/felhom-hub:0.108.0`**, ArgoCD `Synced` / `Healthy`, rolled out. Deployed by **`45659bd`**, which bumped `manifests/hub.yaml:128`. The image was pushed to the registry **before** that commit landed, so a sync could never have pointed at a missing tag. Never `kubectl set image`. **One thing worth stating because it looked like success and was not:** the first `refresh=hard` + sync reported `successfully rolled out` while the deployment still read **0.107.0** and the app read `OutOfSync` — ArgoCD had synced a pre-push revision. A second hard refresh took it to `Synced rev=45659bd` and the image then read 0.108.0. **The rollout message alone would have been a false confirmation**; the image tag is the observable that settles it. ## 8. The live walk All counts filtered `created_at >= T0` (`2026-08-23 11:56:06Z`) so yesterday's two inert Scenario H probe rows cannot contaminate them. ### Step 1 — Scenario A: two different apps, four minutes apart ✅ ``` sent operator Telepített alkalmazás nem fut: OpenGist 2026-08-23 11:56:57 sent operator Telepített alkalmazás nem fut: Calibre-Web A 2026-08-23 12:00:57 sent: 2 suppressed: 0 ``` Yesterday, the identical shape: `bookstack` **sent** 09:27:51, `privatebin` **suppressed** 09:31:51 under `key=demo-hp:app_start_failed`. ### Step 2 — Scenario B: each app again inside the hour ✅ ``` suppressed OpenGist operator cooldown 1h, key=demo-hp:app_start_failed:opengist 12:02:48 suppressed Calibre-Web operator cooldown 1h, key=demo-hp:app_start_failed:calibre-web 12:02:48 ``` One `sent` and one `suppressed` per app — **the hour is unchanged** — and the two keys **differ by the app**, against v0.107.0's single shared key. The hub records the key only on a suppression, which is why this step is what exposes it. *Method note:* the repeats were posted through `/api/v1/event`, the exact endpoint the controller invokes, with the controller's own `AppDetails` payload. The controller's own event is edge-triggered per app, so a down→down cycle is silent **by design** and cannot re-fire from the box. ### Step 3 — Scenario C: a backup-family event carrying `stack_name` ✅ ``` sent opengist suppressed calibre-web operator cooldown 1h, key=demo-hp:crossdrive_failed 12:03:44 ``` **Byte-identical to v0.107.0's key, with no app suffix.** The "before" value was obtained two independent ways, both stated: derivation from the v0.107.0 expression (which has no app term), and `TestR389_NoOtherEventTypeKeyChanges`, which models that expression inline for 12 event types **and carries a positive control proving it can see a key change before reporting that none occurred**. ### Step 4 — Part 2's burst ✅ (see §9) ### Step 5 — Part 3's gate ✅ all controls plus the historical red-proof (see §10) ## 9. Part 2's three counts, and the judgement Three apps stopped in one scan, after checking none carried a live cooldown — a stale one would have halved the count and made the answer look better than it is: | | | |---|---| | attempted | **3** | | sent | **3** | | suppressed | **0** | **Plain judgement: per-app is the right grain and this volume is acceptable.** The reference box has 8 deployed apps, so a total outage is 8 mails; the boot grace (90 s), the quiesce grace (180 s) and the per-app edge trigger absorb reboots, backup cycles and persistently-dead apps. **No burst-digest row was filed**, and the condition that would reopen it is recorded rather than left implicit: the volume scales linearly with app count and has no ceiling, so a box large enough that a total outage is unreadable is the point at which the answer becomes a digest with a customer message — not a wider cooldown. ## 10. Which repos gate 11 is registered in | Repo | Registered | Note | |---|---|---| | `felhom.eu` | **yes** — gate 11 | its own `REPORT.md` is the gate's first real subject | | `felhom-controller` | **yes** | already had `SHARED_*` constants; one constant + one `GATES` line | | `felhom-agent` | **yes** | same; no observations section today, so it passes quietly | | `app-catalog-felhom.eu` | **NO** | filed as **R-391** | **Why not the catalog.** `catalog_gates.py` has no shared-gate mechanism at all: `run_gate` joins every entry against its **own** `scripts/` directory, so it cannot invoke a sibling repo's script; and the loop appends `--all` to every gate unconditionally, which the observations gate would read as a path. Registering there needs `run_gate`'s contract widened **and** its argument handling changed — a refactor of a runner whose shape is deliberately different, in a repo this task marked out of scope. Exposure today is nil (that repo's `REPORT.md` has no observations section, and the gate passes quietly on that), but a future session could write one. **Filed rather than left as a sentence in a report, which is the exact failure this session exists to fix.** ## 11. Evidence `documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/` — 17 files: 2 red-proof transcripts, 3 gate-11 control files covering 10 cases, 11 live-walk files, and a 1803-line controller-log window **pulled off before the apps were restored**. ## 12. Teardown, three layers 1. **Guest 9201 / apps** — nothing provisioned. Five apps stopped across the walk (`opengist`, `calibre-web`, `kimai`, `romm`, `paperless-ngx`); **all restarted and confirmed healthy**, 17 containers up. The three retained subjects (`docmost`, `bookstack`, `privatebin`) were not touched. No app rebuilt, redeployed or restored. 2. **No VM, no bake** — this session built no golden and started no drill VM. 3. **Hub-side, stated explicitly.** The hub *was* written: deployed to v0.108.0 via the manifest, and **six probe events POSTed for Scenarios B and C** (two `app_start_failed`, two `crossdrive_failed`, plus yesterday's two, left in place deliberately). They are inert event rows for `demo-hp`, named here rather than left to be found, and **every count in this report is `created_at`-filtered so they cannot contaminate it**. Nothing else: no appliance registered, no customer created, no artifact manifest changed, floor untouched. ## 13. Register size | File | Before | After | |---|---|---| | `OPEN-ITEMS.md` | 328,132 B | **331,024 B** | | `CLOSED-ITEMS.md` | 74,642 B | **76,855 B** | R-389 filed, then closed and compressed into `CLOSED-ITEMS.md` in the same session. **R-390** (the golden-bake runbook's missing `pveam update`) and **R-391** (the catalog runner) filed open. ## 14. Observations > **Gate 11's first real subject is this section.** Each item carries `FILED: R-NNN` naming a row > opened this session, or `NOT-A-FINDING:` with its reason. 1. **The gate's specification would have passed the item the gate exists to catch.** It said an observation may "cite an `R-NNN` that resolves" — but yesterday's lost item cites `R-182`, which resolves, as an **analogy** rather than as its own row. No parser can tell citation-as-precedent from citation-as-filing by reading prose, so the marker is explicit instead. The discrepancy is recorded in the gate's docstring and proven by `EDGE 7`. NOT-A-FINDING: this is a design decision taken and documented inside the deliverable itself, not a defect left behind — the gate ships with the stricter rule and its reasoning, so there is nothing outstanding for a row to track. 2. **The burst has no ceiling.** Three apps in one scan produce three mails; the reference box's worst case is 8, and it scales linearly with app count. NOT-A-FINDING: measured and judged acceptable at today's scale in §9, with the reopening condition stated there; filing a row for a digest would queue work the operator has not asked for and that needs a customer message and their call on volume. 3. **ArgoCD reported "successfully rolled out" while still running the old image.** The first sync ran against a pre-push revision; only a second hard refresh moved it. The rollout message alone was a false confirmation and the image tag was the observable that settled it. NOT-A-FINDING: the existing runbook already says to verify the image tag after a sync, and this run followed it and caught the discrepancy — the procedure worked; recording the near-miss here is the appropriate weight. 4. **The golden-bake runbook still omits `pveam update`**, and its failure names the wrong cause. FILED: R-390 5. **Gate 11 is registered in three runners, not four** — `catalog_gates.py` cannot invoke a sibling script and appends `--all` to every gate. FILED: R-391 6. **Deliberately left open, untouched:** R-102, R-359, R-385, R-387, R-388's redesign. NOT-A-FINDING: a pointer to rows that already exist, carried so their absence from this session reads as deliberate rather than forgotten.