felhom.eu/REPORT.md

REPORT — hub v0.108.0 (R-389), gate 11, and the instruction that invited the gap

Session 2026-08-23. felhom.eu is the subject; the controller and agent were touched only to register the shared gate. No controller release — no golden bake, no vouch, no floor. No halt condition fired. Nothing was dropped.

1. Baselines, and the hub's four numbers as read

Repo	at start	at end
felhom.eu	2f7c9a6 (hub v0.107.0)	hub v0.108.0 deployed
felhom-controller	1da2c9c (v0.223.0)	unchanged — runner registration only
felhom-agent	40d857b (v0.130.0)	unchanged — runner registration only

Hub's four numbers, live from GET /configuration before starting:

Field	Value
golden_version	0.223.0
agent_version	0.130.0
min_agent	0.129.0
controller floor (min_controller_version)	0.222.0

The task predicted the floor at 0.223.0 and it reads 0.222.0. The operator vouched the golden but has not yet raised the floor — the last step of the previous release, which STATUS.md says to do "last, in its own save". Carried forward as item 1 there. Not a halt; the correct reading is simply different from the prediction.

The golden-currency gate stayed green throughout and no bake was needed, exactly as §1 said it should be: the controller CHANGELOG's newest entry and the newest baked golden are both 0.223.0 and this session moved neither.

2. Documents read

hub/internal/notify/dispatcher.go (cooldownTierSuffix + cooldownRunSuffix docstrings in full, processOperator and its R-182 suppression-logging block, operatorOnlyEvents), scripts/repo_gates.py (whole docstring, including "WHY 10 IS HERE" and "WHY 7 IS HERE"), scripts/due_checks_gate.py, documentation/PROMPT-TEMPLATE.md §15, and the alarm ladder at documentation/architecture/08-alarm-ladder.md — extended here with §6.2, the delivery grain.

3. The AppDetails emitter count, measured

Three, exactly as §3 said. grep -rn "AppDetails{" --include=*.go over the controller, excluding tests:

Emitter	Event	Severity	Reaches the operator leg?
notifier.go:502	app_deployed	info	no — info is dropped by severityNotifies
notifier.go:563	app_start_failed	warning	yes
notifier.go:661	app_removed	info	no

No fourth emitter. No halt.

But the sweep found something the AppDetails question could not: stack_name is also carried by a different struct, CrossDriveDetails (notifier.go:151-157), used by crossdrive_failed (severity error, so it does reach the operator leg) and crossdrive_completed. This is what makes the allow-list load-bearing in fact rather than in principle — a payload-shape rule would have split a backup-family event per app and silently undone R-182. It is Scenario C's live subject.

4. Files, commits, CI

Commit	Repo	Contents
f751aea	felhom.eu	R-389 filed — alone, before any code (Phase 1)
2fc4a15	felhom.eu	the suffix + allow-list + tests, gate 11, template fix, R-390/R-391
45659bd	felhom.eu	hub v0.108.0 CHANGELOG + manifest bump
f8c9390	felhom-controller	gate 11 registered; its REPORT's observations marked up
058b945	felhom-agent	gate 11 registered

CI runs confirmed BY ID — and the listing was checked for truncation rather than trusted, since a silently truncated listing has already cost this arc a false claim:

Commit	Repo	CI id	run_number	Result
f751aea	felhom.eu	412	263	success
2fc4a15	felhom.eu	413	264	success
45659bd	felhom.eu	416	265	success
f8c9390	felhom-controller	414	90	success
058b945	felhom-agent	415	55	success

The API reported total_count 265 / 91 / 55 against 3 rows shown in each case — i.e. the pages were known-partial and the newest rows are the ones quoted, not the whole set.

5. Red-proofs — two planted, both seen failing

#	Mutation	Layer, and why that layer	Observed
1	cooldownStackSuffix dropped from the key expression	processOperator's KEY — where the collapse physically happens	2 apps down inside the hour produced 1 operator mail(s), want 2 (suppressed=1), and the suppression row reads key=c1:app_start_failed — the live shape reproduced in a unit test
2	the allow-list check removed from the suffix	the REGISTER — the fence that keeps the backup family coarse	crossdrive_failed … = ":bookstack", want "", plus app_deployed, app_removed, backup_failed — the fence convicting exactly the types it was written for

Both mutations asserted their pre-fix text was present before rewriting and printed MUTATION APPLIED. Neither passed first time, and the check for that was explicit after yesterday's inert mutation.

Gate 11 carries its own controls rather than a mutation, because the gate is the guard: ten cases in §Part 3 of the drill record, including the historical red-proof against yesterday's real file.

6. Test counts

Repo	Before	After
felhom.eu hub	709	716

go build ./... && go vet ./... && go test ./... in the hub → exit 0, zero failures. python3 scripts/repo_gates.py --fast → 12/12 OK in felhom.eu; controller and agent runners both OK with gate 11 registered.

7. Deployed hub version, and the manifest commit

gitea.dooplex.hu/admin/felhom-hub:0.108.0, ArgoCD Synced / Healthy, rolled out. Deployed by 45659bd, which bumped manifests/hub.yaml:128. The image was pushed to the registry before that commit landed, so a sync could never have pointed at a missing tag. Never kubectl set image.

One thing worth stating because it looked like success and was not: the first refresh=hard + sync reported successfully rolled out while the deployment still read 0.107.0 and the app read OutOfSync — ArgoCD had synced a pre-push revision. A second hard refresh took it to Synced rev=45659bd and the image then read 0.108.0. The rollout message alone would have been a false confirmation; the image tag is the observable that settles it.

8. The live walk

All counts filtered created_at >= T0 (2026-08-23 11:56:06Z) so yesterday's two inert Scenario H probe rows cannot contaminate them.

Step 1 — Scenario A: two different apps, four minutes apart ✅

sent    operator  Telepített alkalmazás nem fut: OpenGist        2026-08-23 11:56:57
sent    operator  Telepített alkalmazás nem fut: Calibre-Web A   2026-08-23 12:00:57
                                                    sent: 2   suppressed: 0

Yesterday, the identical shape: bookstack sent 09:27:51, privatebin suppressed 09:31:51 under key=demo-hp:app_start_failed.

Step 2 — Scenario B: each app again inside the hour ✅

suppressed  OpenGist      operator cooldown 1h, key=demo-hp:app_start_failed:opengist     12:02:48
suppressed  Calibre-Web   operator cooldown 1h, key=demo-hp:app_start_failed:calibre-web  12:02:48

One sent and one suppressed per app — the hour is unchanged — and the two keys differ by the app, against v0.107.0's single shared key. The hub records the key only on a suppression, which is why this step is what exposes it.

Method note: the repeats were posted through /api/v1/event, the exact endpoint the controller invokes, with the controller's own AppDetails payload. The controller's own event is edge-triggered per app, so a down→down cycle is silent by design and cannot re-fire from the box.

Step 3 — Scenario C: a backup-family event carrying stack_name ✅

sent        opengist
suppressed  calibre-web   operator cooldown 1h, key=demo-hp:crossdrive_failed   12:03:44

Byte-identical to v0.107.0's key, with no app suffix. The "before" value was obtained two independent ways, both stated: derivation from the v0.107.0 expression (which has no app term), and TestR389_NoOtherEventTypeKeyChanges, which models that expression inline for 12 event types and carries a positive control proving it can see a key change before reporting that none occurred.

Step 4 — Part 2's burst ✅ (see §9)

Step 5 — Part 3's gate ✅ all controls plus the historical red-proof (see §10)

9. Part 2's three counts, and the judgement

Three apps stopped in one scan, after checking none carried a live cooldown — a stale one would have halved the count and made the answer look better than it is:

attempted	3
sent	3
suppressed	0

Plain judgement: per-app is the right grain and this volume is acceptable. The reference box has 8 deployed apps, so a total outage is 8 mails; the boot grace (90 s), the quiesce grace (180 s) and the per-app edge trigger absorb reboots, backup cycles and persistently-dead apps. No burst-digest row was filed, and the condition that would reopen it is recorded rather than left implicit: the volume scales linearly with app count and has no ceiling, so a box large enough that a total outage is unreadable is the point at which the answer becomes a digest with a customer message — not a wider cooldown.

10. Which repos gate 11 is registered in

Repo	Registered	Note
felhom.eu	yes — gate 11	its own REPORT.md is the gate's first real subject
felhom-controller	yes	already had SHARED_* constants; one constant + one GATES line
felhom-agent	yes	same; no observations section today, so it passes quietly
app-catalog-felhom.eu	NO	filed as R-391

Why not the catalog. catalog_gates.py has no shared-gate mechanism at all: run_gate joins every entry against its own scripts/ directory, so it cannot invoke a sibling repo's script; and the loop appends --all to every gate unconditionally, which the observations gate would read as a path. Registering there needs run_gate's contract widened and its argument handling changed — a refactor of a runner whose shape is deliberately different, in a repo this task marked out of scope. Exposure today is nil (that repo's REPORT.md has no observations section, and the gate passes quietly on that), but a future session could write one. Filed rather than left as a sentence in a report, which is the exact failure this session exists to fix.

11. Evidence

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/ — 17 files: 2 red-proof transcripts, 3 gate-11 control files covering 10 cases, 11 live-walk files, and a 1803-line controller-log window pulled off before the apps were restored.

12. Teardown, three layers

1. Guest 9201 / apps — nothing provisioned. Five apps stopped across the walk (opengist, calibre-web, kimai, romm, paperless-ngx); all restarted and confirmed healthy, 17 containers up. The three retained subjects (docmost, bookstack, privatebin) were not touched. No app rebuilt, redeployed or restored.
2. No VM, no bake — this session built no golden and started no drill VM.
3. Hub-side, stated explicitly. The hub was written: deployed to v0.108.0 via the manifest, and six probe events POSTed for Scenarios B and C (two app_start_failed, two crossdrive_failed, plus yesterday's two, left in place deliberately). They are inert event rows for demo-hp, named here rather than left to be found, and every count in this report is created_at-filtered so they cannot contaminate it. Nothing else: no appliance registered, no customer created, no artifact manifest changed, floor untouched.

13. Register size

File	Before	After
OPEN-ITEMS.md	328,132 B	331,024 B
CLOSED-ITEMS.md	74,642 B	76,855 B

R-389 filed, then closed and compressed into CLOSED-ITEMS.md in the same session. R-390 (the golden-bake runbook's missing pveam update) and R-391 (the catalog runner) filed open.

14. Observations

Gate 11's first real subject is this section. Each item carries FILED: R-NNN naming a row opened this session, or NOT-A-FINDING: with its reason.

1. The gate's specification would have passed the item the gate exists to catch. It said an observation may "cite an R-NNN that resolves" — but yesterday's lost item cites R-182, which resolves, as an analogy rather than as its own row. No parser can tell citation-as-precedent from citation-as-filing by reading prose, so the marker is explicit instead. The discrepancy is recorded in the gate's docstring and proven by EDGE 7. NOT-A-FINDING: this is a design decision taken and documented inside the deliverable itself, not a defect left behind — the gate ships with the stricter rule and its reasoning, so there is nothing outstanding for a row to track.
2. The burst has no ceiling. Three apps in one scan produce three mails; the reference box's worst case is 8, and it scales linearly with app count. NOT-A-FINDING: measured and judged acceptable at today's scale in §9, with the reopening condition stated there; filing a row for a digest would queue work the operator has not asked for and that needs a customer message and their call on volume.
3. ArgoCD reported "successfully rolled out" while still running the old image. The first sync ran against a pre-push revision; only a second hard refresh moved it. The rollout message alone was a false confirmation and the image tag was the observable that settled it. NOT-A-FINDING: the existing runbook already says to verify the image tag after a sync, and this run followed it and caught the discrepancy — the procedure worked; recording the near-miss here is the appropriate weight.
4. The golden-bake runbook still omits pveam update, and its failure names the wrong cause. FILED: R-390
5. Gate 11 is registered in three runners, not four — catalog_gates.py cannot invoke a sibling script and appends --all to every gate. FILED: R-391
6. Deliberately left open, untouched: R-102, R-359, R-385, R-387, R-388's redesign. NOT-A-FINDING: a pointer to rows that already exist, carried so their absence from this session reads as deliberate rather than forgotten.