Night power drill complete: 8 real cuts on demo-hp (all came back), the estimator checked on a second machine and on 8 GB; R-1003..R-1009 filed; STATUS to one screen (history moved byte for byte); G18 and F5 corrected
gates / gates (push) Successful in 6m39s

This commit is contained in:
2026-10-14 09:40:16 +02:00
parent 8ac7e14992
commit 048a43c521
111 changed files with 87375 additions and 1365 deletions
@@ -212,7 +212,7 @@
| <a id="f2"></a>[F2](capability-map-detail/F.md#f2) | Health-degradation email (edge-triggered, cooldowns, Hungarian) via hub → Resend | controller, hub | **IMPLEMENTED** | the health trigger never fired a mail live | `audits/CAMPAIGN-6D-2026-07-15.md` (2026-07-15); R-4 |
| <a id="f3"></a>[F3](capability-map-detail/F.md#f3) | Event catalog: app_start_failed, dead-app, offbox_enlarge_blocked, claim/reset codes, critical severity | controller, hub v0.31/48/50/55 | **PROVEN-LIVE** | — | `audits/CAMPAIGN-6D-2026-07-15.md` (2026-07-15); `audits/DRILL-day0-vm-2026-07-12.md` (2026-07-12); `audits/DRILL-day0-take2-2026-07-12.md` (2026-07-12) |
| <a id="f4"></a>[F4](capability-map-detail/F.md#f4) | Prefs safety: empty-email wipe guard | controller v0.137 + hub v0.71.0 | **IMPLEMENTED** | red-proofed 2026-07-15 / 2026-07-22 | unit red-proofs in the controller and hub repos (outside `documentation/`) |
| <a id="f5"></a>[F5](capability-map-detail/F.md#f5) | Paired recovery notifications + prefs seeding at claim + priority headers (power-outage audit F11/F12/F14-light) | hub v0.71.0 | **IMPLEMENTED** | recovery leg partial until a live staleness cycle | R-69 |
| <a id="f5"></a>[F5](capability-map-detail/F.md#f5) | Paired recovery notifications + prefs seeding at claim + priority headers (power-outage audit F11/F12/F14-light) | hub v0.71.0 | **IMPLEMENTED** | operator recovery leg seen live 2026-10-14 (real 50-min power cut: stale → recovered pairs for two boxes); the household leg not exercised | R-69; `audits/night-power-2026-10-13/README.md` R6 |
| <a id="f6"></a>[F6](capability-map-detail/F.md#f6) | System + container metrics (SQLite, Chart.js, 30-day downsampling) | controller | **IMPLEMENTED** | validated in no campaign | `tests/CAMPAIGN-2-2026-07-07.md` (2026-07-07) |
| <a id="f7"></a>[F7](capability-map-detail/F.md#f7) | Always-on debug rings + on-demand log-bundle pulls with TTL/custody | controller v0.116, agent v0.83, hub v0.46 | **PROVEN-LIVE** | — | `audits/CAMPAIGN-3-2026-07-11.md` (2026-07-11) |
| <a id="f8"></a>[F8](capability-map-detail/F.md#f8) | Operator alerting (Healthchecks → monitoring@felhom.eu) | k3s, Resend | **IMPLEMENTED** | no corpus validation doc | operator infra; no evidence document |
@@ -241,7 +241,7 @@
| <a id="g15"></a>[G15](capability-map-detail/G.md#g15) | The hub database survives the loss of DooPlex: a nightly consistent copy, encrypted, on ep0; restore-tested weekly; an alarm when either stops | hub v0.136.0 (R-173), `scripts/hub-db-backup/`, homelab-manifests rules | **PROVEN-LIVE** | — | `audits/hub-db-offsite-2026-10-05/` (2026-10-05); `audits/dooplex-survival-2026-10-09/` (2026-10-09) |
| <a id="g16"></a>[G16](capability-map-detail/G.md#g16) | Gitea (all code), the operator's password manager (Vaultwarden, since 2026-10-09 afternoon, R-923) and DooPlex's secrets survive the loss of DooPlex: a nightly encrypted copy on ep0; restore-tested weekly; an alarm when either stops; a failure mail | `scripts/dooplex-offsite/` (R-232), homelab-manifests rules | **PROVEN-LIVE** | — | `audits/dooplex-survival-2026-10-09/` (2026-10-09) |
| <a id="g17"></a>[G17](capability-map-detail/G.md#g17) | The operator surface refuses a cross-site form post on BOTH login paths (session token; Basic auth + `X-Felhom-Operator`) | hub v0.135.0 (R-135) | **PROVEN-LIVE** | — | `audits/hub-safety-2026-10-05/partA/live.txt` (2026-10-05) |
| <a id="g18"></a>[G18](capability-map-detail/G.md#g18) | Box operating-system security updates (Proxmox host, guest Debian, Docker engine) | agent v0.143.0, hub v0.133.0 | **PARTIAL** | kernel lane missing | `audits/os-guest-lane-2026-10-04/` (2026-10-04); `audits/os-host-lane-2026-10-04/` (2026-10-04); `audits/os-docker-crash-2026-10-04/` (2026-10-04) |
| <a id="g18"></a>[G18](capability-map-detail/G.md#g18) | Box operating-system security updates (Proxmox host, guest Debian, Docker engine) | agent v0.143.0, hub v0.133.0; kernel lane agent v0.152.0+ | **PROVEN-LIVE** | all four lanes; the kernel lane on both demo boxes (2026-10-08, 2026-10-10); no automatic undo | `audits/kernel-night-2026-10-09/readback/RESULT.md` (2026-10-10); `audits/kernel-night-2026-10-08/readback/` (2026-10-08); `audits/os-guest-lane-2026-10-04/` (2026-10-04); `audits/os-host-lane-2026-10-04/` (2026-10-04); `audits/os-docker-crash-2026-10-04/` (2026-10-04) |
| <a id="g19"></a>[G19](capability-map-detail/G.md#g19) | An ENGLISH-SPEAKING household's first hour: download, install, pair, bind, claim, two apps | controller v0.259.0 + hub v0.119.0 + ISO 1.29.0 + the whole catalog | **PROVEN-LIVE** | walked on 0.258.0 with one blocker; fixed, the walk not repeated | `audits/DRILL-first-hour-en-0258-2026-09-20.md` (2026-09-20); `audits/i18n-closing-2026-09-21/live/` (2026-09-21) |
| <a id="g20"></a>[G20](capability-map-detail/G.md#g20) | A deletion of a customer's off-site history is NOTICED within a day | hub v0.111.0 (R-431) | **IMPLEMENTED** | not yet proven live | R-431 |
@@ -91,6 +91,8 @@
**Evidence (as written):** `hub/CHANGELOG.md` v0.71.0; 17 tests + 4 red-proofs (`REPORT.md` 2026-07-22); Resend `headers` mechanism probed live (HTTP 200) pre-implementation; operator+customer `test` rows live-fired via the controller's own test endpoint
**Seen live (2026-10-14):** a real 50-minute power cut of demo-hp and Tester 1 (`audits/night-power-2026-10-13/README.md`, R6): `node_stale` 23:32, `host_stale` 23:45, `host_recovered` 23:53, `node_recovered` 23:55 — paired, true, in the operator mailbox; the household was told nothing („no unanswered customer down mail”), which is consistent. The household leg (after a `host_down` at 90 min) is still not seen live. The recovery text reports the stale time, not the outage (R-1005).
**Gap / roadmap (as written):** Recovery = explicit eventType branch, severity semantics frozen; customer gate = PAIRING (`notification_log` evidence), not `enabled_events`. Live legs pending: a natural `*_recovered` mail (next real staleness cycle or the reboot-drill arc — never fabricated by blocking reports) and seed-at-claim on a real claim (Peti Friday reinstall). F14-full (operator push channel, ntfy/Telegram) stays open → R-69
---
@@ -308,6 +308,8 @@
**Status (as written):** **PARTIAL — the GUEST and HOST Debian fast lanes and the DOCKER engine slow lane are PROVEN-LIVE (2026-10-04), with the System page, the fleet view and the alarms; the KERNEL lane is MISSING**
**Status (corrected 2026-10-14):** **PROVEN-LIVE — the KERNEL lane too.** Built 2026-10-07 (`audits/kernel-lane-2026-10-07/`, agent v0.152.0+), read back on real nights: demo-felhom restarted into its new kernel 2026-10-08 (`audits/kernel-night-2026-10-08/readback/`); demo-hp and demo-felhom both on 2026-10-10 (`audits/kernel-night-2026-10-09/readback/RESULT.md`: healthy 1 m 59 s and 39 s after the agent started, no false alarm). The row said „kernel lane missing” after that work: stale, not a gap. The remaining gap is below (no automatic undo).
**Evidence (as written):** Guest: `audits/os-guest-lane-2026-10-04/`. Host + fleet + alarms: `audits/os-host-lane-2026-10-04/`. Docker + System page + crash guard: `audits/os-docker-crash-2026-10-04/` — live-restore on with the same container ids on every box; Docker 29.8.2 on both demo boxes; operator-approved Docker release; a signed undo and a signed ring-1 step; a replay refused; the crash guard restarted demo-hp twice and kept it off the third time. Design `architecture/11-os-updates.md` §5.8, §5.9, §8
**Gap / roadmap (as written):** **No automatic undo** (guest: last night's backup; host: by-hand runbook; Docker: a signed undo job); existing boxes get root-owned files by the signed config bundle since 2026-10-04 (R-840 CLOSED; a box from before agent 0.143.0 needs one by-hand bootstrap — Tester 2: R-862; `audits/r840-config-bundle-2026-10-04/`); test approvals now end with the test (R-859); the agent's sudoers is root-equivalent (R-861); the kernel lane (R-836); facts reach the hub late after a boot (R-853). **2026-10-05 (agent v0.144.1):** R8 measures the real download (R-865); a killed pass still reports (R-868, live); the debug pass runs with the hub away (R-866, live); **a power cut mid-update was proven by day on demo-hp — the box came back by itself in 37 s, but the next pass fails until `dpkg --configure -a` is run by hand (R-876, P2, open)** — `audits/night-fixes-2026-10-05/`. **2026-10-05 afternoon (agent v0.145.0): R-876 FIXED and proven live — after a second crash mid-unpack the next pass repaired dpkg by itself (`REPAIR … journal=1`) and finished** — `audits/catchup-2026-10-05/partD/`