felhom.eu/REPORT.md

REPORT — MariaDB finishes its own conversion, golden 0.236.0, and goldens move to a cadence (2026-09-13)

Overwritten each session. Nothing durable lives only here — every finding below has a register row.

Everything the task asked for shipped, and every scenario A–H passed with the evidence quoted below. Four templates carry MARIADB_AUTO_UPGRADE=1; the harness saw the conversion RUN; the change landed on demo-hp through the real 15-minute cycle and recreated nothing; an engine-major gate holds every database engine inside its major until Slice 4; golden 0.236.0 is baked, round-tripped, vouched and the floor raised; and golden_currency_gate.py reads a dated waiver. Three claims in the prompt turned out wrong or imprecise — named first, in §1.

1. Claims in the prompt that turned out wrong, named first

1. "a throwaway guest on demo-hp, disk on /mnt/nvme-1tb at its root." /mnt/nvme-1tb does not exist on demo-hp; the 1 TB NVMe is mounted at /mnt/hdd_1 (already recorded by the two September spikes and R-461). The guest's disk went on a dir storage at /mnt/hdd_1's root, as those spikes did. target-selection.md still names the wrong path — that is R-461, not re-filed.
2. "the waiver is R-242's remaining half." The gate's docstring names TWO things: the honest fix for a release nobody wants a golden for is "a recorded waiver, never a bypass" (built today), and R-242's remaining half is that nothing gates the VOUCH (unbuilt, unchanged, still open on R-242). The task text conflated them. The docstring, the row and this report keep them apart.
3. "Add the bake to the nightly-session checklist." No document by that name exists in any repo (grep -rln nightly finds none under documentation/runbooks/). The step went into the two routines that do exist: RUNBOOK-manual-build.md §4.2 (the cadence) and this repo's CLAUDE.md end-of-session checklist. If a nightly checklist is created later, it points at §4.2.

Also imprecise, not wrong: the runbook's vouch step says to read MinAgent from the golden's controller CHANGELOG header, and the last four headers carry no such line — filed as R-470.

2. Confirmed baselines

repo	before	after (pushed to main)
app-catalog-felhom.eu	b7ef0c4a09d6	eec1228 templates → bd32830 gate → 3525e35 CHANGELOG/REPORT
felhom.eu	4b2e5608c227	ae59c31 (documents + gate + evidence) → the compression commit on top
felhom-controller	155271672265 (v0.236.0)	unchanged — no code. Golden bake only.

Highest R-id before: 467. Minted: R-468 (waiver), R-469 (engine-major rule expiry), R-470 (MinAgent header line), R-471 (observations decoy hole, pre-existing).

3. Scenario results, with the evidence quoted

Evidence directory: documentation/audits/r459-close-2026-09-13/ (36 files); golden: documentation/tests/golden-0.236.0-2026-09-13/.

A — the harness sees the conversion happen ✅

Throwaway LXC 9403 on demo-hp (Debian 13, Docker 29.8.0, Compose v5.5.1), harness copied from the local commit WITH the setting. engine_state_after.bookstack-db.answer, verbatim, on both edges:

12.3.3-MariaDB | This installation of MariaDB is already upgraded to 12.3.3-MariaDB. There is no need to run mariadb-upgrade again. [exit=1]

edge	verdict	seed before / after	abort
E3 (app + engine 11.6→12.3)	proven	true / true	starts-and-serves
E3b (engine half alone)	proven	true / true	starts-and-serves

The entrypoint, verbatim (E3, harness/E3/to-full.log):

[Note] [Entrypoint]: Backing up system database to system_mysql_backup_11.6.2-MariaDB.sql.zst
[Note] [Entrypoint]: Backing up complete
[Note] [Entrypoint]: Starting mariadb-upgrade
Major version upgrade detected from 11.6.2-MariaDB to 12.3.3-MariaDB. Check required!
[Note] [Entrypoint]: Finished mariadb-upgrade

skipped due to $MARIADB_AUTO_UPGRADE: 0 lines in both TO logs (it appeared on every start before this change — spike §4). Conversion 09:53:20 → 09:53:26 = 6 s. The abort log prints MariaDB upgrade not required — the R-464 trap, quoted as an observation and not as soundness.

B — the negative control still works ✅

C3 (privatebin/pdo:2.0.5 → alpine:3.20): failed, healthy_after=false, container restarting. Ran first, as the CLAUDE.md rule says.

C — the change lands harmlessly on a live box ✅

Pushed 3525e35 at 07:58:02 UTC; the box's previous sync was 07:54:17, so the change waited for the real tick. live-9201/00-baseline-before-push.txt → 01-after-sync.txt → 02-restart.txt.

The sync's own lines, verbatim:

2026/09/13 08:09:17 sync.go:396: [INFO] [sync] Updated bookstack/docker-compose.yml
2026/09/13 08:09:17 sync.go:409: [DEBUG] [sync] bookstack: stored definition refreshed with the delivered fix
2026/09/13 08:09:17 sync.go:135: [INFO] [sync] Periodic sync: Sablonok frissítve — frissítve: bookstack, kimai, nextcloud, romm

Live docker-compose.yml fragment from 9201 after the sync:

  bookstack-db:
    image: mariadb:12.3
    ...
      - TZ=Europe/Budapest
      # MARIADB_AUTO_UPGRADE: on a MAJOR engine move the engine converts its own datadir (~7 s on a
      ...
      - MARIADB_AUTO_UPGRADE=1

Not recreated by the sync: both container IDs and StartedAt identical to the pre-push baseline (bookstack 26b555d7… started 04:21:07Z; bookstack-db b02f7a09… started 04:21:01Z), docker ps Up 4 hours (healthy). applied-compose.yml carries the setting too (fixes flow INTO the pin).

The one deliberate POST /api/stacks/bookstack/restart at 08:10:32 UTC → {"ok":true,"message":"Stack bookstack restart completed"}, healthy after 8 s. The engine's log after the restart, verbatim:

2026-09-13 10:10:34+02:00 [Note] [Entrypoint]: MariaDB upgrade not required
2026-09-13 10:10:35 0 [Note] mariadbd: ready for connections.

and the engine asked directly (R-464): 12.3.2-MariaDB / This installation of MariaDB is already upgraded to 12.3.2-MariaDB. [exit=1]; MARIADB_AUTO_UPGRADE=1 present in the running container; GET /login → 200. Nothing else on 9201 was touched; bentopdf stays.

D — the rule exists and a gate knows it ✅

app-catalog-felhom.eu/scripts/check-engine-major.py, fourth row of catalog_gates.py, run by the pre-push hook with --range=<remote sha>..<local sha>. Refusal, verbatim:

ENGINE-MAJOR GATE FAILED: templates/kimai/docker-compose.yml service kimai-db moves mariadb 11 -> 12 (mariadb:11.6 -> mariadb:12.3).
RULE (app-catalog CLAUDE.md, operator ruling 2026-09-13): until the Update button takes a VERIFIED BACKUP as its precondition (Slice 4, felhom.eu OPEN-ITEMS.md R-448), no template may move a database-engine image across a MAJOR version.
WHY: MariaDB sidecars now carry MARIADB_AUTO_UPGRADE=1 and WILL convert the customer's datadir on the next Update; PostgreSQL's image refuses to start on an older major's datadir (R-463). Either way this is a customer-data event with no backup in front of it.
EXPIRY: this rule is removed DELIBERATELY when R-448 ships — the removal is its own register row, not a silent edit. Until then, keep the engine within its major.

Pass, verbatim (this push's own range): engine-major gate OK — no database engine crosses a major version (rule: CLAUDE.md, until Slice 4 / R-448 ships). Red-proof (engine-major-gate/redproof-7-cases.txt): 11.6→12.3 rc=1, postgres:16→17 rc=1, mariadb:lts rc=2, 11.6→11.8 rc=0, and three decoys (comment + serverVersion= env, the app's own image, README) rc=0. CI cannot run it — the runner fetches at --depth 1; the runner announces the skip on a shallow clone (pinned by test_catalog_gates.py). Same gap as R-452, not re-filed.

E/F/G/H — the golden gate in all four waiver states ✅ (golden-waiver-states/)

state	exit	the line that proves it
E valid waiver, golden BEHIND	0	GOLDEN CURRENCY GATE ADVISORY — WAIVED, NOT CLEAN: controller v9.9.9 is released and NO golden carries it (newest bake is 9.9.7; 2 release(s) behind: 9.9.8, 9.9.9). … Waived by R-468 until 2026-09-26 (13 day(s) left)
F expired waiver, behind	1	The waiver at documentation/tests/golden-waiver.yml EXPIRED on 2026-09-12 (R-468). It ran out, as a dated waiver is meant to
G valid waiver, golden UNRECORDED	1	A waiver exists (R-468, until 2026-09-26) and DOES NOT COVER THIS: it covers a golden that is behind the record, never one that is unrecorded (R-385).
H 15-day waiver	2	the waiver … is MALFORMED — expires:is 15 days afterissued: — the hard limit is 14.
H decoy: a file saying only expires	2	MALFORMED — issued: is absent or not a YYYY-MM-DD date

Red-proof on the REAL tree, while it was still behind (before the bake landed; R-468 row present, a valid waiver planted): the OLD gate (4b2e560) → GOLDEN CURRENCY GATE FAILED … exit=1 (it cannot read a waiver); the NEW gate → ADVISORY — WAIVED … 4 release(s) behind: 0.233.0, 0.234.0, 0.235.0, 0.236.0 … exit=0. Tests: scripts/test_golden_currency_gate.py cases 5–15, all green (19 cases in the file now, 4 before).

The first waiver, as committed (documentation/tests/golden-waiver.yml, issued AFTER the bake):

issued: 2026-09-13
expires: 2026-09-27
reason: pre-customer development; goldens on a weekly cadence (operator ruling 2026-09-13)
register_row: R-468

Gate on the real tree now: newest released controller : 0.236.0 / newest golden baked : 0.236.0 / waiver : VALID until 2026-09-27 (14 day(s) left) → OK, exit 0. Before the bake it read FAILED … 4 release(s) behind, exit 1 (golden-waiver-states/00-real-tree-before-bake-no-waiver.txt).

4. The golden (R-467) — version and vouch evidence

GOLDEN_VERSION=0.236.0, GOLDEN_SHA256=58a3cc24c61dc2271f2cb508ef5a269af97005f386b671f18011258df5f958bf, 654 115 664 B. Three readers agreed (bake log, round trip hashed from the DOWNLOADED bytes, the hub's dropdown); ./etc/felhom-controller-image out of the archive says felhom-controller:0.236.0; markers 1/1/1/1, excluding 0, FATAL 0; token-leak grep 0 with the seeded control 1; the bake script's fingerprint equal across the hop. Vouch: golden_version 0.232.0 → 0.236.0, agent_version 0.130.0 and min_agent 0.129.0 unchanged, re-read from the page, R-120 banner absent; floor 0.232.0 → 0.236.0 re-read. No box moved — both already ran 0.236.0 by hand; the chain was last exercised 2026-09-01 and nothing about it changed. This bake carried four unbaked releases and is the last per-release one. No --no-verify anywhere in this session.

5. Files created / modified

app-catalog-felhom.eu (pushed 3525e35): templates/{bookstack,kimai,nextcloud,romm}/docker-compose.yml, scripts/check-engine-major.py (new), scripts/test_gate_decoys.py (new), scripts/catalog_gates.py, scripts/test_catalog_gates.py, .githooks/pre-push, CLAUDE.md, REUSE.md, CHANGELOG.md, REPORT.md.

felhom.eu: scripts/golden_currency_gate.py, scripts/test_golden_currency_gate.py, scripts/CHANGELOG.md, documentation/tests/golden-waiver.yml (new), documentation/tests/golden-0.236.0-2026-09-13/ (new, 12 files), documentation/audits/r459-close-2026-09-13/ (new, 36 files), documentation/runbooks/RUNBOOK-manual-build.md (§4.2), CLAUDE.md (checklist step), documentation/architecture/09-update-architecture.md (§3 decisions 5 and 6, §8.7, engines section), documentation/backlog/OPEN-ITEMS.md, documentation/backlog/CLOSED-ITEMS.md, CONTEXT.md, STATUS.md, this file.

6. Tests

suite	before	after
app-catalog/scripts/test_catalog_gates.py	5	7, green
app-catalog/scripts/test_gate_decoys.py	—	7 cases, green
felhom.eu/scripts/test_golden_currency_gate.py	4	19 cases, green
felhom.eu/scripts/repo_gates.py --fast	14 gates	14 gates, all OK
felhom.eu/scripts/decoy_coverage_gate.py on the catalog	3 exempt	1 covered + 3 exempt, 0 unaccounted

No Go code changed in any repo; go build/vet/test not applicable.

python3 scripts/unproven.py --summary: 55 claims, walked 20 / partial 17 / built 14 / missing 4, NOT WALKED 35 of 55 — unchanged (nothing in this session walked a capability claim).

CI, by head_sha: app-catalog job 536 for 3525e35 → completed / success. The felhom.eu run for this push is quoted in the follow-up commit that records it.

7. Register — rows opened / closed, size

Closed: R-459 (harness + live evidence), R-467 (the bake). Narrowed: R-242 (waiver half built; vouch half open). Opened: R-468 (WATCHING — the waiver; renew ≤ 14 days or bake; retire at the first external install), R-469 (BLOCKED on R-448 — remove the engine-major rule), R-470 (READY — MinAgent header line), R-471 (READY — observations decoy hole, pre-existing). Register size: 210 open rows / 169 closed before → 212 open / 171 closed after (R-459 and R-467 compressed to CLOSED-ITEMS.md in the second commit, citing ae59c31 for the full text; nothing deleted). OPEN-ITEMS.md: 433 845 → 434988 bytes with the four new rows and two closures written out, → 429692 bytes after compression.

8. Evidence off the machine before every teardown

• Harness (9403): evidence/ tarred and pct pulled, then scp'd to DooPlex, before pct fstrim / pct destroy — harness/teardown-9403.txt.
• Bake (drill VM): bake.log scp'd out and markers counted before pct destroy 9100, the shred -u and the poweroff — 05-teardown.txt, leftovers in the VM: 0.
• Live (9201): baseline captured before the push; after-sync and after-restart captured at the moment; nothing on 9201 was reverted.

9. Teardown — three layers

layer	before	after
1 — machines	LXC 9403 on demo-hp (scratch dir storage scratch-r459c at /mnt/hdd_1); LXC 9100 inside the drill VM	9403 destroyed, storage removed, template deleted, pct list shows only 9201; 9100 destroyed, drill VM powered off, qemu confirmed exited (ps -eo comm), disk reverted to the single virgin snapshot
2 — hosts	demo-hp local-lvm 35.32 %, local 20 899 060 KiB, /mnt/hdd_1 5 774 620 KiB	local-lvm 35.32 % — never touched; local 20 908 828 KiB (+9.5 MB); /mnt/hdd_1 5 774 620 KiB — identical; pct fstrim 9403 returned 55.5 GiB; guest peak 3.3 GB (2.21 GB images). DooPlex /mnt/5_hdd 37 % before and after
3 — hub	2 enrolled hosts	Checked, not asserted (harness/teardown-layer3-hub.txt): /hosts lists exactly demo-felhom-8363b5 and demo-hp-bb76ea; this run created no customer, no appliance and no host record — 9403 never enrolled, 9100 is the bake fixture. The hub's ONLY change is the vouch + floor, which is the deliverable.

Helper files on demo-hp (ctrl_pw, dh_user, dh_pat, upg.tgz, guest-setup.sh, logs) were shred -u'd / removed; the Docker Hub login inside 9403 was logged out before the guest was destroyed.

10. NOT live-validated — stated plainly

• The waiver in the "behind" state on the real tree, after this push: it is dormant (the golden is current). Its real-tree behaviour was proven before the bake (§3 E, red-proof); after today it is exercised the next time a release ships without a bake — which is the intended cadence.
• The self-update chain for 0.236.0: not re-exercised (both boxes were already there). Last proof 2026-09-01.
• Kimai, Nextcloud, RomM under an actual engine major move: the setting is inert until their pins move, and the engine-major rule forbids that until Slice 4. Only bookstack has a measurable edge.
• The engine-major gate in CI: cannot run there (--depth 1); the hook is the enforcement.

11. Observations — noticed, documented, NOT acted on

1. felhom.eu/scripts/test_gate_decoys.py reports a LIVE HOLE at HEAD, before this session: observations/R-419: decoy PASSED (rc=0) — the gate scans the report's first Observations section and not an appended second one. Reproduced on a clean worktree of 4b2e560. FILED: R-471.
2. Four consecutive controller CHANGELOG headers carry no MinAgent: line while the vouch runbook says to read it from the header. FILED: R-470.
3. The stack restart recreated only the changed service — bookstack-db got a new ID, the bookstack app container kept its 4-hour uptime — because the restart is compose up -d, which the update architecture §1.3 records as a chosen behaviour. NOT-A-FINDING: documented design, and "restart completed" is accurate for a compose-level restart.
4. mariadb:12.3 resolved to 12.3.3 in the harness and 12.3.2 on 9201 — the floating-pin class. NOT-A-FINDING: already R-446.
5. target-selection.md names /mnt/nvme-1tb, which does not exist on demo-hp. NOT-A-FINDING: already R-461, and it says exactly this.
6. The session ran with permission prompts disabled, which the workspace CLAUDE.md says not to do on this host. Nothing outside the workspace, ~/build, the drill directory and the two Tier-0 boxes was touched; every host-side act is listed in §9. NOT-A-FINDING: an operator setting, not a product defect; recorded so it is not hidden.