docs: slice 3 Part A live evidence, and two findings (R-583, R-584)

The live proof: the box's own "send test notification" button pressed twice, 74 seconds apart, on demo-hp. Reporting `en` it produced "[Felhom] Test notification / Dear Customer, ..."; switched to `hu` it produced "[Felhom] Teszt értesítés / Kedves Ügyfél! ...", byte-for-byte the v0.117.0 literal. The operator's copy is identical in both, which is the half worth stating. R-583 (closed, hub v0.118.1): the test mail was the one customer mail that did not follow the language, and it is the mail an operator would use to CHECK that the language works. The surface you would use to check a feature is the one most worth checking first. R-584 (open, P2): five probe scripts from slice 2's releases B/C/D were still in the guest's /tmp carrying the controller password INLINE. The rule to delete them exists, was loaded, and was not followed three times running - so the rule is not the mechanism. All shredded; whether to rotate the shared demo password is the operator's call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-09-18 16:33:49 +02:00
parent a2c52ebf2a
commit 1637fa655d
3 changed files with 119 additions and 0 deletions
@@ -0,0 +1,92 @@
+# Slice 3 Part A — hub v0.118.0 / v0.118.1, live evidence (2026-09-18)
+
+**Method: endpoint-level**, plus one real e-mail read in the operator's own inbox. No browser on
+DooPlex. Nothing was installed, nothing removed; the deploy endpoint was never touched.
+
+## 1. The wire already worked — measured BEFORE writing any code
+
+Read-only copy of the live `hub.db` (+ `-wal`, or it is hours stale):
+
+```
+demo-felhom  14:12 UTC  language='hu'   controller 0.255.0
+demo-hp      14:09 UTC  language='en'   controller 0.255.0
+drill-r50 / peti-felhom / tester-1      language=<ABSENT>  (0.213.0 / 0.115.0 / 0.245.0)
+```
+
+The positive and the negative control in one read: every box on ≥ 0.247.0 publishes the field, every
+older one sends nothing at all — which is the case the empty default exists for.
+
+## 2. The migration applied
+
+```
+reports.language          present: True
+customer_configs.language present: True
+customer_configs.language = 'hu' for all five customers (the default backfill)
+```
+
+The first reports written by the NEW hub, minutes after the sync, carry the denormalised value:
+
+```
+demo-felhom  id=25717  14:22:59  language='hu'
+demo-hp      id=25718  14:23:00  language='en'
+```
+
+**Rollback, written before the sync:** revert the manifest tag to 0.117.0 and sync. The two columns
+stay and need no undo — v0.117.0 neither reads nor writes them and both carry a DEFAULT, so every
+INSERT the old binary performs still succeeds. Rolling back the image is the whole rollback.
+
+## 3. One real e-mail, both directions, 74 seconds apart
+
+The path is the product's own: sign in to the box, press **Send test notification** on the
+notifications page (`POST /settings/notifications/test`) — which is what a customer's own click does.
+The box POSTs `event_type=test` to the hub, and the hub composes the mail.
+
+**Box reporting `en`** — 14:30:43 UTC, to the household address `drill@felhom.eu`:
+
+```
+Subject: [Felhom] Test notification
+         Dear Customer,
+         This is a test notification from the Felhom monitoring system.
+         Notifications are working correctly.
+         Best regards,
+         Felhom.eu monitoring
+```
+
+**The same button, box switched to `hu`** — 14:31:57 UTC, same address:
+
+```
+Subject: [Felhom] Teszt értesítés
+         Kedves Ügyfél! Ez egy teszt értesítés a Felhom monitoring rendszerből.
+         Az értesítések megfelelően működnek. Üdvözlettel, Felhom.eu monitoring
+```
+
+The Hungarian is byte-for-byte the literal that shipped in v0.117.0 — it was extracted into the
+bundle by script, never retyped.
+
+**The operator's copy is identical in both**, to `admin@felhom.eu`:
+`[Felhom] ✅ demo-hp: teszt / operator channel OK`. That is the half worth stating: the household's
+language changed twice and nothing the operator reads moved.
+
+## 4. What this did NOT prove
+
+- **`message_customer` has no live proof yet.** No box sends it until controller v0.256.0 (Part B).
+  The hub half is covered by tests only.
+- The `test` mail bypasses `FormatCustomerEmail`, so this proof exercises the bundle and the language
+  resolution, **not** the event-mail wrapper. The wrapper's live proof waits for Part B.
+- Nobody looked at these mails in a mail CLIENT. They are plain text; the bytes above are what was
+  sent.
+
+## 5. A lapse, recorded
+
+Earlier in this same session (slice 2 releases B/C/D) I left five probe scripts in the guest's
+`/tmp` — `probeB.sh`, `probeB2.sh`, `probeB3.sh`, `probeC.sh`, `probeD.sh` — and they carried the
+controller password inline. The standing rule says a credential-bearing helper is deleted from
+`/tmp` on both host and guest when done; they sat there for hours instead. Found while cleaning up
+after this proof, by grepping my own litter for secret-shaped strings before deleting it. All are
+now shredded, along with this run's password file on the host, the guest and DooPlex. **This run
+used a file→file password (never an inline literal), which is why its own scripts were clean.**
+
+## 6. State at the end
+
+Hub **0.118.1**, Synced/Healthy, clean startup log. demo-hp back on **Hungarian**. Nothing
+installed, nothing removed, no floor change, no golden. Guest `/tmp` clean.
@@ -0,0 +1,25 @@
+=== BEFORE (rollback point):
+gitea.dooplex.hu/admin/felhom-hub:0.117.0
+sync=Synced health=Healthy rev=20aafc3dec2008f8f4697129bcf95bd354edef20
+ROLLBACK LINE (written BEFORE the sync):
+  git revert the manifest tag to 0.117.0, commit, push, sync.
+  The two new COLUMNS stay and need no undo: v0.117.0 neither reads nor writes
+  them, both carry a DEFAULT, so every INSERT the old binary performs still
+  succeeds. SQLite cannot drop a column before 3.35 and the data is worth
+  keeping. Rolling back the IMAGE is the whole rollback.
+=== AFTER:
+sync=Synced health=Healthy rev=9167cf53afbc9cf62fdc92ca248d37a493e915e7
+image=gitea.dooplex.hu/admin/felhom-hub:0.118.0
+=== startup log:
+2026/09/18 16:22:18 [INFO] Staleness checker initialized: 2 ok, 0 stale, 2 down (node_stale after 45m0s, node_down after 1h30m0s)
+2026/09/18 16:22:18 [INFO] Host staleness checker initialized: 2 ok, 0 stale, 0 down (stale/down left unseeded → first Check emits; host_stale after 45m0s, host_down after 1h30m0s)
+2026/09/18 16:22:20 [INFO] Host capability checker initialized: 2 ok, 0 degraded (degraded left unseeded → first Check emits)
+2026/09/18 16:22:20 [INFO] Host leaf checker initialized: 2 host fingerprint(s) seeded
+2026/09/18 16:22:20 [INFO] Host disk checker initialized: warn=90% crit=95%, 2 ok seeded, 0 already-breached left unseeded (first Check emits)
+2026/09/18 16:22:20 [INFO] Storage fill checker initialized: warn=90% crit=95%, 6 ok seeded, 0 already-breached left unseeded, 2 root-backed excluded
+2026/09/18 16:22:20 [INFO] Host mgmt-plane checker initialized: 0 host heal-state(s) seeded
+2026/09/18 16:22:20 [INFO] Host OOB checker initialized: 0 host(s) seeded degraded
+2026/09/18 16:22:20 [WARN] [offsite] tester-1: controller sends no last_success — staleness degraded to the last-ATTEMPT anchor (pre-v0.181.0 controller; a persistently failing tier will not go stale here until it upgrades)
+2026/09/18 16:22:20 [INFO] Offsite checker initialized: fill warn=90% crit=95%, stale after 48h0m0s, 4 ok-seeded
+2026/09/18 16:22:20 [INFO] Listening on :8080
+2026/09/18 16:22:20 [INFO] deadline-check: next run at 2026-09-19 05:00 CEST (in 12h37m40s)