manifests: hub 0.133.0

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
hub v0.133.0 (R-859 test approvals end with the test; R-840 bundle on the System page, manifest, alarm); installer 1.31.0 (root files from the config bundle); bundle bootstrap script; golden 0.293.0 evidence; rulings 96–99; drill-r50 removed (evidence)
2026-10-04 20:19:40 +02:00 · 2026-10-04 20:18:37 +02:00 · 2026-10-04 18:35:40 +02:00 · 2026-10-04 17:57:54 +02:00 · 2026-10-04 17:51:09 +02:00 · 2026-10-04 17:49:30 +02:00
202 changed files with 12149 additions and 207 deletions
@@ -16,6 +16,56 @@
 > and holds nothing of its own; this file does hold its own content, namely the standing rulings below.


+> **2026-10-04 (~18:49) — rulings 96–99 (the R-840 / Tester 2 brief), recorded before the work.** `09` §3: **96** build
+> R-840 now (option A, a signed config bundle; replaces 82); **97** Tester 2 may receive only the signed agent update,
+> the bundle by the route and the one-time live-restore reload (~19:05: the operator will install the bootstrap file by
+> hand through his tunnel — the route cannot reach a root side that predates it); **98** drill-r50 removed from the hub,
+> ep0 touch allowed (~19:05); tester-1 stays; **99** the operator's cause with the reviewer's correction.
+
+> **2026-10-04 (~18:00) — R-858 incident + ruling 95.** The operator saw demo-felhom DOWN: the 14:13 UTC Docker step
+> left `felhom-controller` and `traefik` on the OLD socket (live-restore kept them running; their bind-mounted socket
+> file was recreated). Repaired by restarting the two (15:57 UTC). `09` §3 decision **95**: the wrapper restarts the
+> socket users after a step + the health rule checks the controller reaches Docker (agent v0.142.1). Demo ring-0 Docker
+> marks OFF until released.
+
+> **2026-10-04 (evening) — the System page, the Docker engine slow lane and the crash guard BUILT.** Agent v0.142.0,
+> hub v0.132.0, installer 1.30.0, `build-golden.sh` 3.1.0. **Decided by CC unattended — operator may reverse:** `09` §3
+> decisions **90** (crash signal = a clean-shutdown marker; pstore saved nothing), **91** (`panic_on_oops` stays 0; an
+> oops is reported), **92** (the 3rd unclean stop in 60 min leaves the box off; 10 s / 24 h), **93** (the wrapper
+> verifies Docker authority against root-owned files), **94** (page colours = alarm thresholds). Live: live-restore on by
+> reload with the same ids (9202 6/6, demo-hp 24/24, demo-felhom 5/5); Docker 29.7 → 29.8.2 on both demo boxes, ids kept;
+> the operator's button approved `os-docker-20261004-142842` (TEST 0-night wait, reverted); a signed undo to 29.7.2 on
+> demo-hp and back; demo-felhom as ring 1 by a signed job; a replay refused; crash 1–2 restarted in 54/53 s, the guard
+> tripped, crash 3 stayed off, the hub mailed the trip, re-armed. `REPORT-os-docker-crash-2026-10-04.md`.
+
+> **Rulings 2026-10-04 (~15:17) — recorded before the work (System page + Docker lane + crash restart brief).** `09` §3
+> decisions **87** (Docker `live-restore` ON fleet-wide, never off by a plain restart), **88** (a crashed host restarts by
+> itself, with a limit — R-851) and **89** (the operator sees the boxes' OS / Proxmox versions in the hub — R-852).
+
+> **2026-10-04 (late afternoon) — OS updates: host fast lane + fleet view + alarms BUILT (`11` §8.2–§8.3); the tunnel
+> status is true (R-841).** Agent v0.141.0 → v0.141.1, hub v0.131.0 → v0.131.1, controller v0.292.0 (cloudflared
+> readiness health check). **Decided by CC unattended — operator may reverse:** `09` §3 decisions **84** (appliance proof
+> = the root-owned install record), **85** (alarm numbers 7/14/7/14 days, configuration), **86** (a same-session patch
+> release of agent and hub for the host "reboot needed" defect, R-846). Live: `tunnel_down`/`tunnel_recovered` on
+> demo-hp; ring 0 host pass on demo-felhom (108 Debian packages); a 605-package host release; ring 1 exact install;
+> by-hand host undo proved; leg 23–32 s with nothing to install. Kernel spike (R-836, narrowed): GRUB's one-shot is not
+> a one-shot on LVM `/boot`; Secure Boot fine; `kernel.panic = 0` (R-851); `sp5100_tco` answers. demo-hp now runs and
+> defaults to kernel 7.0.14-20. Docker slow lane designed (`11` §5.8). Golden 0.292.0 baked + vouched with agent 0.141.1 (min_agent
+> 0.131.0); the golden waiver deleted (not needed). `REPORT-os-host-lane-2026-10-04.md`.
+
+> **Rulings 2026-10-04 (~12:20) — recorded before the work (host fast lane brief).** `09` §3 decisions **81** (R-842 A:
+> the undo is the whole-guest backup by hand; R-842 closed), **82** (R-840 not built now — "There are no older boxes";
+> row kept open with the reviewer's note) and **83** (next: R-841, the host fast lane, OS-update improvements).
+
+> **2026-10-04 (afternoon) — OS updates, guest fast lane BUILT (`11` §8.1).** Agent v0.140.0: `configs/felhom-os-apply`
+> (R1–R13, repair first, snapshot.debian.org fallback; FELHOM_OSAPPLY sudoers), `internal/osupdate` (the leg after a
+> successful primary backup, under the heavy-op gate; health baseline = start of the leg), `--selftest=os-update`.
+> Hub v0.130.0: `internal/osupdates` (rings, switch default ON, candidate = what ALL ring-0 boxes run, approval 24 h +
+> 1 night via `OS_APPROVE_AFTER` / `OS_APPROVE_NIGHTS`, `os_update` desired block, `POST /hosts/{id}/os-report`,
+> operator `/os/*`). Controller v0.291.0: decision 78 (R-726), infra pins raised (R-838) + `scripts/check-infra-pins.py`.
+> Installer 1.29.0 installs the wrapper. **No automatic undo** (R-837 measured → R-842). **Existing boxes need the
+> wrapper + sudoers by hand** (R-840 — the demo boxes got them by hand). `REPORT-os-guest-lane-2026-10-04.md`.
+
 > **Rulings 2026-10-04 (~10:17) — recorded before the work (guest fast lane build).** `09` §3 decisions **78** (R-726
 > option A: a returning household's box sets the old off-site copy aside on night one), **79** (`snapshot.debian.org`
 > fallback for a replaced approved version — option A; the reviewer had picked B) and **80** (next build: the guest
@@ -0,0 +1,99 @@
+# REPORT — System page, Docker slow lane, crash restart (2026-10-04, late afternoon–evening)
+
+Brief: "OS updates — a System page …; the Docker engine slow lane built (live-restore ON, decision); a crashed host
+restarts by itself, with a limit (decision)". Architecture read first: `documentation/architecture/11-os-updates.md`
+(owner; §5.6, §5.8, §8.1–8.3), `05-hub-architecture.md`, `03-host-agent.md`, `08-alarm-ladder.md`, `07` §6.1. Rulings
+recorded before the work: `09` §3 decisions 87–89. Evidence for every claim: `documentation/audits/os-docker-crash-2026-10-04/`.
+
+## Part table
+
+| Part | What | Result | Evidence |
+|---|---|---|---|
+| A | System page + version report (R-852) | **DONE.** The box reports Proxmox + kernel (API) and the wrapper's read-only facts (host Debian, next-boot kernel, held packages, taint, `kernel.panic`, crash guard; guest Debian, Docker, containerd, live-restore); unreadable = `unknown`. Hub: **System** tab on every page — per box ring + switch with buttons, tunnel, host, guest, Docker, last leg; releases per layer; what ring 0 runs; "Approve now" (confirm) and "Approve Docker set" (only when allowed). Hosts gets a Proxmox / kernel column. R-849: guest scanned every pass. Rendered with real data from both demo boxes. | `partA/` (`live/system-page.html`, `hosts-page.html`, facts) |
+| B | Docker engine slow lane (`11` §5.8) | **DONE.** live-restore on by RELOAD: same ids on 9202 (6/6, by hand — R10 refuses a scratch guest by design), demo-hp (24/24, 7.5 s), demo-felhom (5/5). Ring 0 on both: 29.7.x → **29.8.2**, every id kept (122 s / 99 s whole pass). Approval with the page button under a TEST 0-night wait (logged, reverted): `os-docker-20261004-142842`. **Signed undo** on demo-hp → 29.7.2 (wrapper `authority=signed UNDO`, 24/24 ids, 41 s) and back to 29.8.2 by its ring-0 pass. **demo-felhom as ring 1**: its night pass skips Docker; a signed job with the approved set verified by the wrapper (already current → nothing); the **replayed** job refused ("nonce already seen"); back to ring 0. | `partB/` |
+| C | Crash restart with a limit (R-851) | **DONE.** Spike (your word before each crash): `kernel.panic=10` + `echo c` → back by itself in 54 s, same kernel; pstore saved nothing; no oops history. Guard built (decision 88, 90–92). Live: crash 1 → 54 s, crash 2 → 53 s and the guard **tripped**, crash 3 → **stayed off** (184 s watched) until you switched it on; the hub mailed `host_crash_guard_tripped` (+3 restart events, 3 household lines); System page red; re-armed by `felhom-crash-guard rearm`. | `partC/` |
+| D | Releases, golden, records | Agent **v0.142.0** (signed to both demo boxes), hub **v0.132.0**, installer **1.30.0** (tagged, public, verified), `build-golden.sh` 3.1.0. Golden: see below. Records: `11` §5.7/§5.8/§5.9, `00`, `03`, `07`, `08`, decisions 90–94, two runbooks. | `partD/`, git |
+
+## Claims in the brief that turned out wrong (or half right)
+
+- **"No box reports its versions"** — half right: every box already sent `pveversion` and `kversion` inside the Proxmox
+  API answer the agent reads each report (`NodeStatus`); nothing stored or showed them. Debian, the next-boot kernel and
+  everything Docker were truly not reported.
+- **"A reload turns live-restore on with no container restart, on the demo boxes too"** — TRUE, measured: 24/24 and 5/5
+  ids kept. But "prove on 9202 first" could not use the product path: 9202 binds a scratch folder, not the drives, so the
+  wrapper refuses it (R10, by design); proved there by hand with the same two steps.
+- **"A crash boot can be told apart from a clean one"** — only from a CLEAN one. Not from a power cut or a hard reset:
+  `efi_pstore` is on, yet a real panic saved nothing on demo-hp. The guard counts every unclean stop (decision 90).
+- **"`echo c` crashes the host and `kernel.panic` brings it back"** — TRUE (54 s, 53 s). But `sysctl -w` does not survive
+  the restart (back to 0), so it must be set at every boot — the guard does that.
+- **The guard's count** — the brief said both "at 3 … the next crash leaves the box off" (the 4th) and "crashes 3 times
+  within one hour, it stays off" (the 3rd). Built per your words: the 3rd (decision 92).
+
+## Decisions taken by CC unattended (operator may reverse) — `09` §3
+
+90 crash signal = clean-stop marker · 91 `panic_on_oops` stays 0, an oops is mailed · 92 the 3rd unclean stop in 60 min
+stays off; 10 s; 24 h · 93 the wrapper verifies Docker authority against root-owned files · 94 page colours = alarm
+thresholds.
+
+## Releases and what was copied by hand
+
+- **Agent v0.142.0** (`b1746c2`, sha256 `7beb3222…de6`): signed `agent_update` to both demo boxes, both COMPLETED.
+- **Hub v0.132.0** (`175ecfc`): image tag verified on the pod; ArgoCD Synced/Healthy.
+- **Installer 1.30.0**: tag `installer-v1.30.0`, both git-sync refs; `https://felhom.eu/scripts/felhom-host-install.sh`
+  serves `SCRIPT_VERSION="1.30.0"`.
+- **By hand on both demo hosts** (R-840; `partD/copied-by-hand-*.txt`, hashes equal to tag v0.142.0):
+  `/usr/local/sbin/felhom-os-apply`, `/usr/local/sbin/felhom-crash-guard`, `/etc/systemd/system/felhom-crash-guard.service`,
+  `…/felhom-crash-guard-check.service`, `…/felhom-crash-guard-check.timer`, `/etc/felhom/crash-guard.conf`,
+  `/etc/felhom/operator-signers`, `/etc/felhom/os-trust.json` (with `ring0_slow_lane: true` — the demo boxes only);
+  units enabled. Previous wrapper kept as `/root/felhom-os-apply.bak-0.141.1`. A test binary (`felhom-agent-0.142.0-rc1`)
+  ran the debug actions before the release and was removed after.
+
+## Golden
+
+- **Re-baked golden 0.292.0** with `build-golden.sh` 3.1.0 (by a helper agent, RUNBOOK-manual-build §4.0/§4.1; the
+  documented publish replaces the same version: pre-delete 204, upload 201). The log shows "Docker engine set PINNED",
+  the six approved versions (docker-ce 5:29.8.2, containerd.io 2.3.6, buildx 0.37.1, compose 5.6.0, …) and
+  "live-restore: on". New sha256 `79a1dce3…d43a`, re-hashed by download in the main session: match. Drill VM destroyed,
+  drill disk back to `virgin`. Evidence `documentation/tests/golden-0.292.0-2026-10-04-rebake/` (commit `208d21d`).
+- **Re-vouched:** agent 0.142.0 + golden 0.292.0 (new sha), `min_agent` 0.131.0 → `artifacts_set` (17:48). Between the
+  re-bake upload and the re-vouch the hub vouched the old sha — a fresh install would have failed closed; none ran (R-857).
+- The crash guard is NOT in the golden (it is a host program): the installer 1.30.0 installs it.
+- The golden waiver stays deleted: the newest controller (0.292.0) has its golden.
+
+## Register
+
+Open rows **334 → 333** (333 at the start + R-852 filed first). Closed: **R-852, R-835, R-848, R-849, R-851**; filed and
+closed: **R-854**. Opened: **R-853** (facts reach the hub ~15 min late after a boot), **R-855** (cosmetic TEST log line),
+**R-856** (after a crash the household also gets app mails — your choice later), **R-857** (a same-version golden
+re-bake: the gate shows the first sha; a window until the re-vouch). Narrowed: **R-812** (Docker lane built;
+kernel left), **R-840** (by hand again). `unproven.py`: unchanged (35 of 55 not walked).
+
+## Teardown — three layers
+
+- **Machine:** demo-hp and demo-felhom customer guests running, live-restore on, Docker 29.8.2, all apps up. 9202
+  running, live-restore on (its old daemon.json kept as `/root/daemon.json.bak-2026-10-04` in the guest).
+- **Host:** both hosts run agent 0.142.0, the crash guard ARMED (`kernel.panic = 10`), ring 0, switch ON; the test
+  binary and the id lists removed. demo-hp booted 3 extra times today (the crash test); kernel unchanged (7.0.14-20).
+- **Hub:** the TEST Docker wait reverted (log: 2 nights); the approval `os-docker-20261004-142842` stays (a real
+  approval of what ring 0 runs). The crash and trip mails for demo-hp were sent on purpose. The hub password copy in
+  the scratchpad shredded at the end. The hub announced the re-arm (`host_crash_guard_rearmed`, 17:47).
+
+## CI
+
+Checked by `head_sha` over every page of the Gitea `jobs` endpoint: felhom.eu — all 10 commits from `bee277f` (rulings)
+to `21986d0` (re-vouch records) **success** (incl. hub `175ecfc` 1265→, installer, manifests, docs); felhom-agent —
+`b1746c2` (1273, 1274), `f24dce5` (1275), `42af3ab` (1281) **success**. This report's own commit: checked after the push
+(see the session's final message).
+
+## Addendum (~18:30) — R-858, found by the operator
+
+The N100 showed DOWN from 14:18 UTC. Cause: v0.142.0's Docker step restarted dockerd (14:13); live-restore kept the
+containers running, but `felhom-controller` and `traefik` bind-mount the socket FILE and kept the deleted inode, so the
+controller could not reach Docker. My Docker health rule passed it (the controller's own check said healthy) — the rule
+checked the mechanism, not the consequence. Repaired by restarting the two containers (15:57 UTC). Ruling 95: agent
+**v0.142.1** (`4950030`, sha256 `003f882a…62bd`) restarts only the socket users after a step and fails health when the
+controller cannot reach Docker. Proven live on demo-hp before the release (signed undo, then forward): `applied, healthy`,
+guest / controller / traefik on the same socket inode both times. Ring-0 marks were OFF during the fix, back ON after.
+Both boxes on 0.142.1; vouched for new installs (golden unchanged). Red-proofs 4/4. Register: R-858 opened and closed
+(still **333** open). Evidence `partE-incident/`. Not touched: Tester 1 shows DOWN for 4 days on the dashboard — a
+fenced tester box, outside this brief.
@@ -0,0 +1,60 @@
+# REPORT — OS updates build step 1: the guest's Debian fast lane; decision 78; the infrastructure images — 2026-10-04
+
+Architecture read: `11-os-updates.md` (with C1–C12, §5.4.1, §7.1 — the design; it won wherever it differed from the
+brief, see below), `03-host-agent.md`, `07` §6.1, `09` §3 decisions 11/12/15/18, `08`. Baselines (re-verified):
+felhom.eu `1b74ddc0c9` (hub 0.129.0), agent `596238cc2e` (0.139.0), controller `99a1497560` (0.290.0), catalog
+`917a779cca`. Register 331, highest R-839. Rulings recorded first: `09` §3 decisions 78–80 (`6ed79cd`). Evidence: `documentation/audits/os-guest-lane-2026-10-04/` (parts A–G).
+
+## The Part table
+
+| Part | Result | Notes |
+|---|---|---|
+| A — the snapshot undo first (R-837) | **done — and it FAILED: no snapshot is possible** | PVE refuses any snapshot not named `vzdump` of a guest with host-path binds (mp8/mp9), as the agent's token (which has `VM.Snapshot` + `VM.Snapshot.Rollback`) and as root. By the brief's rule: **no automatic undo built**; the decision is in STATUS (R-842). Steps 2–5 (apply, roll back, re-apply) had nothing to roll back to; 9201 was brought current by the product's own leg in Part G. Thin pool unchanged. |
+| B — the wrapper | **done** | `felhom-os-apply` (Python 3 stdlib), R1–R13, repair first, snapshot.debian.org fallback, log lines; host layer and slow lane refused. 35 tests; **every refusal red-proved** (13/13). `visudo -cf` OK. **Changed:** Python not shell (a JSON plan cannot be parsed safely in sh — so "shellcheck clean" became `ast`/compile-checked + the suite); one sudoers entry with a plan `mode` instead of a separate `--repair-only`. **The route for existing boxes: none exists** (R-840, with a proposal). |
+| C — the agent's leg | **done** | After a successful primary whole-guest backup, under the heavy-op gate (red-proved: the gate is held), once per 20 h, 90 s settle. Health rule written and pinned (`HealthVerdict`). Report: full installed set with origins, pending, not covered, restart-needed. Debug action `--selftest=os-update`. 7 leg red-proofs + 2 hook red-proofs. |
+| D — the hub | **done** — hub v0.130.0 | Rings, per-box switch (default ON), the candidate/approval rule (24 h + 1 night, config), approve-now, events, fleet JSON. 5 approval red-proofs; the `os_update` wire golden byte-identical in both repos. |
+| E — household line + decision 78 | **done** | Line = hub customer event `os_update_applied` (info: on the household's timeline, not mailed; hu/en in the bundle). **Changed:** there is no box-side event surface, so the hub event is it (R-844). Decision 78 built in controller v0.291.0, red-proved both ways. |
+| F — infrastructure images (R-838) | **done** | traefik v3.7.13, cloudflared 2026.9.3, filebrowser 1.5.6-stable; breaking changes named (none we use). A release moves all three (9202: ≤ 1.9 s / ≤ 1.5 s; demo boxes: public gap ≤ 19.6 s / ≤ 14.7 s incl. the controller restart). `scripts/check-infra-pins.py` + runbook section. **Changed:** the standing brief `claude/MONTHLY-security-retest.md` lives in the claude.ai project, not the repo — the repo half is the runbook; the project file is the operator's to update. `03` corrected (3 lines). |
+| G — live proof | **done, one part changed** | Ring 0 on both boxes (53 packages each, healthy); approval with a 2-minute TEST wait (272 packages, auto), then the ruled values back; ring 1 on demo-felhom (exactly the 3 approved versions, nothing newer); a failed health check → `health_failed`, operator mail, household line. **Changed:** "show the rollback" — there is none (Part A). Teardown: no snapshot, no plan files, test config gone, demo-felhom back to ring 0. |
+| H — release, golden, records | **done** (see Teardown for the golden) | Agent 0.140.0 (signed per box, both demo boxes on it), hub 0.130.0, controller 0.291.0 (floor 0.291.0, MinAgent 0.131.0 declared), installer 1.29.0. `11` §8.1, `00`, `07` §6.1, `03` updated. |
+
+## Claims in the brief that turned out wrong (named)
+
+1. **"The agent's token can snapshot and roll back"** — it HAS the rights, but no snapshot of a customer guest is
+   possible at all (bind mounts). Neither the token nor root can.
+2. **"A snapshot rollback leaves the thin pool clean"** — unmeasurable: there was no snapshot.
+3. **"A new sudoers line can reach an installed box through the product"** — false. Only the installer writes it;
+   the signed agent update replaces the binary only (R-840). The demo boxes got the wrapper + sudoers BY HAND.
+4. **"A controller release moves the infrastructure containers"** — TRUE for all three. (I first wrote the opposite
+   for the file browser and corrected it the same hour: its start-up mount sync renders the new image.)
+5. **"An agent event can reach the household's timeline"** — only through the hub (a hub customer event); the box has
+   no timeline of its own (R-844).
+6. **"Before each guest update, the box takes a snapshot"** (the one-page summary) — impossible (Part A).
+7. `11` vs the brief: `11` §5.4.1's `--repair-only` flag was folded into the plan; `11`'s "a missed night waits" holds.
+
+## Found and fixed live (before the release)
+
+- `--selftest=os-update` was refused by the flag's allow-list — and so was `--selftest=wgtunnel`, since S3 (R-843,
+  opened and closed; a new test pins every dispatched mode).
+- The wrapper logged an UPDATED conffile as "kept" (dpkg's two message shapes; fixed + tested).
+- An app stopped between the inventory and the apply escaped the health check; the baseline is now the start of the
+  leg (fixed + red-proved).
+- My stopped-app test also made the box mail one `app_start_failed` (a second one was held by the cooldown).
+
+## Rows
+
+Closed: **R-837** (measured), **R-838**, **R-726**, **R-843** (opened and closed). Opened: **R-840** (no product route
+to installed boxes, P2), **R-841** (the agent's cloudflared probe reads a host unit that does not exist, P3), **R-842**
+(the undo decision, waiting on the operator), **R-844** (household line only on the hub, P4), **R-845** (a pass takes
+3–4 min, P4). Narrowed: **R-812**. Register **331 → 333**.
+
+## Teardown, three layers
+
+- **Machines:** no snapshot on either 9201; no plan files; privatebin restarted and healthy; demo-felhom back to ring 0;
+  both 9201s fully Debian-current (openssl at the approved u3). 9202 runs controller 0.291.0 (from Part F).
+  **Kept on purpose:** the wrapper + sudoers on both demo hosts (installed by hand; the old sudoers saved as
+  `/root/felhom-agent.sudoers.bak-pre-osapply`); agent 0.140.0 (signed update).
+- **Host (DooPlex):** helper scripts in the scratchpad only; the hub password copy shredded at the end.
+- **Hub:** v0.130.0 at the ruled 24 h + 1 night (the TEST override reverted and the start log shows no override);
+  both demo boxes ring 0, ON; release `os-20261004-091417` approved (it was approved under the TEST wait — ring 1 boxes
+  will install it; every version in it already runs on both demo boxes). Floor 0.291.0.
@@ -0,0 +1,90 @@
+# REPORT — OS updates, build steps 3 + 4 (2026-10-04, afternoon–evening)
+
+Brief: "OS updates, build step 3 + 4" (Parts A–G). Architecture read first: `documentation/architecture/11-os-updates.md`
+(owner), `08-alarm-ladder.md`, `03-host-agent.md`, `07-backup-architecture.md` §6.1. Evidence for every claim:
+`documentation/audits/os-host-lane-2026-10-04/` (partA … partG).
+
+## Part table
+
+| Part | What | Result | Evidence |
+|---|---|---|---|
+| A | The tunnel status is true (R-841) | **DONE.** Three states from the guest container + controller v0.292.0's readiness check; `unknown` never alarms; `tunnel_down` after two `not_running` reports. Live on demo-hp: port 7844 blocked → `tunnel_down` mailed 11:52 UTC (2nd report); unblocked → `tunnel_recovered` 12:07 UTC. No new sudoers line. | `partA/` (red-proofs agent/hub/controller, `live/`) |
+| B | Host fast lane (`11` §8 step 3) | **DONE.** R12 lifted for lane fast / layer host on an appliance only (root-owned install record); R14 refuses kernel/boot/firmware; host step after a healthy guest step; host health rule in `11` §8.2; reboot-needed with first date, never reboots; separate per-layer approved sets. Live: demo-felhom ring 0 → 108 Debian host packages (all Debian origin, checked against apt), healthy; demo-hp ring 0 (nothing pending); 605-package host release approved under a TEST wait (2 m / 0 nights, logged, reverted); demo-felhom as ring 1 installed exactly the 1 version it lacked; back to ring 0 and the ruled 24 h + 1 night. Host undo runbook proved on demo-hp (`tzdata`). | `partB/` (`live/`, `ring1/`, `undo/`, red-proofs) |
+| C | Fleet view + four alarms (`11` §8 step 4) | **DONE.** `GET /os/fleet`: one line per box, ring, switch, tunnel, per layer release/pending/not-covered/last good leg/reboot-since. Alarms `os_update_stale`, `os_reboot_needed`, `os_ring0_stalled`, `os_not_covered`, hourly, operator-only, each red-proved (9 mutations caught). Numbers in `11` §8.3 / `09` decision 85 as CC-decided. | `partC/` |
+| D | The leg is fast (R-845) | **DONE.** Nothing to install, both layers: **23.3 s** (demo-felhom), **31.5 s** (demo-hp). Before (agent 0.140.0, guest only, nothing to install): 14.0 s. A 108-package host pass: 70 s. | `partD/`, `partB/live/` |
+| E | Kernel one-shot spike on demo-hp (R-836), operator's word before each reboot | **DONE (measured).** Secure Boot ON boots the new kernel fine; **`grub-reboot` is NOT a one-shot here** — `/boot` on LVM, GRUB cannot clear `next_entry`, reboot 2 (no command) came back on the NEW kernel. `kernel.panic = 0`. `sp5100_tco` loads and answers (sysfs only, never armed, unloaded). No kernel installed (7.0.14-20 was already there). Left: runs 7.0.14-20, saved default 7.0.14-20, both kernels installed, `GRUB_DEFAULT=saved`. | `partE/` |
+| F | Docker slow lane design | **DONE (design only).** `11` §5.8. One STATUS decision: `live-restore` on, fleet-wide. | `11` §5.8, STATUS |
+| G | Releases, golden, records | See below. Agent **v0.141.0 + v0.141.1**, hub **v0.131.0 + v0.131.1** (decision 86 — a second release in each, for a live-found defect), controller **v0.292.0**. Installer: **no new release** (see below). | `partG/` |
+
+## Claims in the brief that turned out wrong (or only half right)
+
+- **cloudflared readiness** — it EXISTS (`/ready`, 200 only with a connection), but nothing exposed it: the metrics
+  port was random and there was no health check. A container state alone lies (wrong token: `running`, `/ready` 503).
+  Controller v0.292.0 adds the fixed port + Docker health check; the agent reads it with its existing sudoers line.
+- **"Find where the install mode is known"** — known in TWO places, and only one is trustworthy: `agent.json`
+  `deployment_mode` (the agent can write it) and the installer's root-owned `state.json` `mode`. The wrapper trusts
+  only the second (decision 84).
+- **`grub-reboot` with Secure Boot** — Secure Boot was not the problem (it booted fine). The one-shot itself fails on
+  these boxes because GRUB cannot write its state on LVM `/boot`.
+- **Panic auto-restart** — there is none: `kernel.panic = 0`; a panicked host stays down (R-851).
+- **Under 60 s** — met (23–32 s). But the leg was ALREADY under 60 s with nothing to install (14 s, guest only); the
+  3–4-minute passes of R-845 were passes that installed something.
+
+## Releases
+
+- **Agent v0.141.0** (`cfba0d0`, sha256 `6eaad980…`) and **v0.141.1** (`a6bc3f1`, sha256 `b712f577…`) — signed
+  `agent_update` jobs to both demo boxes (both COMPLETED). v0.141.1 fixes R-846 (host "reboot needed" hid `lxc-start`,
+  and a reboot never cleared it).
+- **Hub v0.131.0** (`88b0a2e`) and **v0.131.1** (`fd1f985`) — deployed via ArgoCD; image tag verified on the pod.
+- **Controller v0.292.0** (`09e634d`) — floor raised (`min_controller_version` 0.292.0, `min_agent` 0.131.0); both demo
+  boxes run it; cloudflared recreated once, `healthy`.
+- **Installer — no new release, deliberately** (recommendation not followed, one line why): the installer fetches
+  `configs/felhom-os-apply` from the VOUCHED agent's tag, the file name did not change and the sudoers line did not
+  change, so vouching agent 0.141.1 delivers the new wrapper to every fresh install with no installer change.
+- **Copied by hand to both demo hosts** (`partG/wrapper-copied-by-hand.txt`): `/usr/local/sbin/felhom-os-apply` only —
+  first from v0.141.0 (sha `4729769c…`), then from v0.141.1 (sha `51e100ad…`), installed `0755 root:root`, the old
+  file kept as `/root/felhom-os-apply.bak-0.140.0`. `/etc/sudoers.d/felhom-agent` was NOT copied: it already equals the
+  repo's (`02df92d7…` on both).
+
+## Golden
+
+- **Golden 0.292.0 baked** (by a helper agent, RUNBOOK-manual-build §4.0/§4.1, same `build-golden.sh` bytes as
+  0.291.0; drill VM 9100 destroyed, drill disk back to `virgin`): sha256 `d6cf8b33…16671`, re-hashed by download in the
+  main session — match. Evidence `documentation/tests/golden-0.292.0-2026-10-04/` (commit `f93738d`).
+- **Vouched:** agent 0.141.1 + golden 0.292.0, `min_agent` 0.131.0 → `artifacts_set` (`05-vouch.txt`). Fresh installs
+  now get agent 0.141.1, its wrapper, and controller 0.292.0.
+- **Waiver REMOVED** (`documentation/tests/golden-waiver.yml` deleted): the newest controller (0.292.0) now has its
+  golden, so `golden_currency_gate.py` passes without it. Golden 0.291.0 alone could NOT have retired it — this session
+  released controller 0.292.0, which put 0.291.0 behind.
+
+## Decisions taken by CC unattended (operator may reverse) — `09` §3
+
+- **84** — the appliance proof is the root-owned install record.
+- **85** — alarm numbers 7 / 14 / 7 / 14 days, all configuration.
+- **86** — a second same-session release of agent and hub (R-846).
+
+## Register
+
+Open rows **332 → 333**. Closed: **R-841**, **R-845**; filed and closed the same day: **R-846**, **R-850**. Opened:
+**R-848** (a held host package is invisible to the hub), **R-849** (the guest "reboot needed since" never clears),
+**R-851** (a panicked host stays down — operator). Narrowed: **R-836** (GRUB one-shot measured: not a one-shot),
+**R-812** (host lane built). `unproven.py`: unchanged (35 of 55 not walked).
+
+## Teardown — three layers
+
+- **Machine:** demo-hp guest 9201 — the port-7844 block removed (`DOCKER-USER` empty, verified); cloudflared
+  `healthy`. demo-hp host — `tzdata` back on 2026c, no holds, no snapshot source left; `sp5100_tco` unloaded; GRUB:
+  `GRUB_DEFAULT=saved`, saved default 7.0.14-20, `next_entry` cleared, `/etc/default/grub` backup at
+  `/root/grub.default.bak-2026-10-04`. demo-felhom host — `tzdata` 2026c (re-installed by its ring-1 run), no snapshot
+  source left.
+- **Host:** both demo hosts run agent 0.141.1 + wrapper `51e100ad…`; both ring 0, switch ON.
+- **Hub:** the TEST approval wait reverted (log: 24 h and 1 night); the TEST releases `os-guest-20261004-123933` and
+  `os-host-20261004-124034` stay (they are real approvals of what ring 0 runs). The operator mails `tunnel_down` /
+  `tunnel_recovered` for demo-hp were sent on purpose (Part A). The hub password copy in the scratchpad was shredded.
+
+## CI
+
+Checked by `head_sha` over every page of the Gitea `jobs` endpoint (felhom.eu `CLAUDE.md` recipe):
+agent `cfba0d0` (jobs 1259, 1260), `3bf77c3` (1261), `a6bc3f1` (1262, 1263), `2e2e8f5` (1264) — all **success**;
+controller `09e634d` (1258) **success**; felhom.eu `88b0a2e` (1256), `fd1f985` (1265) **success**. The last docs pushes
+(`31bdb4b` and this report's commit): see the session's final message — checked after the push.
@@ -2,8 +2,122 @@

 **Ready for the first real tester (Tester-2): yes. You confirmed the tunnel route and the connect mails (2026-09-30).**

-**Updated 2026-10-04 (day): the off-site topic is closed; the OS-update test is done. Both demo boxes run controller
-0.290.0 and host agent 0.139.0. Hub 0.129.0. New installs get golden 0.290.0; every box's floor is 0.290.0.**
+**Updated 2026-10-04 (late evening): the hub has a System page with every box's versions and the update buttons; Docker
+updates are built; a crashed box restarts by itself, at most twice an hour. Both demo boxes run host agent 0.142.1 and
+controller 0.292.0. Hub 0.132.0. Installer 1.30.0. New installs get golden 0.292.0 (re-made: live-restore on, Docker 29.8.2) with agent 0.142.1.**
+
+## Today (2026-10-04, ~18:30): you found a bug — the Docker update blinded the box's controller
+
+- **What happened:** the Docker update on the N100 restarted Docker itself. The apps kept running (as designed), but
+  the controller and the web router kept a connection to the OLD Docker, so the controller could not see anything:
+  the hub showed the N100 DOWN from 14:18 to 15:57. demo-hp had the same fault; the crash test happened to heal it.
+- **Fixed (your choice):** after a Docker update the box now restarts just those two (about 10 seconds, apps untouched),
+  and the update's health check now asks "can the controller really reach Docker?". Released as host agent 0.142.1,
+  proven twice on demo-hp, on both demo boxes now, and approved for new installs.
+
+## Today (2026-10-04, late evening): the System page, Docker updates, the crash restart
+
+**Decisions I took myself (you may reverse each):**
+- The box knows a crash only as "it did not shut down cleanly" (the crash memory chip saved nothing). So a power cut
+  also counts as a crash.
+- An "oops" (a kernel error the box survives) does not restart the box; you get a mail instead.
+- Your words win where the brief disagreed: the **3rd** crash within one hour leaves the box off. Restart after 10 s,
+  re-arm after 24 h. All settings.
+- Only the box itself decides whether a Docker update is allowed: it checks your signature with a key file only root
+  can change (not the agent's own settings, which the agent could change).
+- The System page uses the alarm limits for its colours (red = an alarm would fire).
+
+**No decision needed from you today.**
+
+**What I did:**
+- **The System tab** in the hub: per box the Proxmox, kernel (now and next boot), Debian and Docker versions, what is
+  waiting, held packages, "restart needed", the crash guard and the last update run — with buttons for ring, on/off,
+  "Approve now" and "Approve Docker set". The Hosts page shows Proxmox and kernel too.
+- **Docker updates:** "live-restore" is on in every box (no app restarted: 24 of 24 and 5 of 5 containers kept running).
+  Both demo boxes moved to Docker 29.8.2, every app kept running. I approved that set with the new button (a 0-night
+  test wait, then back to 2 nights). An undo signed by your key put demo-hp back one version and forward again, apps
+  running throughout. A copied (replayed) signed job was refused.
+- **Crash restart (your 3 crashes on demo-hp):** crash 1 and 2 — back by itself in under a minute; crash 3 — it stayed
+  off until you switched it on. You got the "guard tripped" mail. I re-armed it.
+- **Found:** the hub learns about a crash up to 15 minutes late (nothing lost). After a crash, the household can also get
+  an "app stopped" mail besides "restarted after a crash" — whether to calm that is a later choice for you.
+- **New-install image re-made** with live-restore on and the approved Docker version, and approved in the hub with host
+  agent 0.142.0. New boxes also get the crash guard (installer 1.30.0).
+- **Rows:** 6 closed, 1 opened-and-closed the same day, 5 opened. The list went from 334 to 333.
+
+**Needs you later (nothing breaks if you wait):**
+- Installed boxes still get new root files only by hand (the long-standing gap; there are no other boxes today).
+- Kernel updates are still not built (a hung new kernel would stay — needs a fix first).
+
+## Today (2026-10-04, evening): host fixes, the fleet view, a true tunnel status
+
+**Decisions I took myself (you may reverse each):**
+- Only a box the installer itself recorded as "appliance" gets host updates (a record the agent cannot change).
+- The four alarm times: no update run for 7 days; "reboot needed" for 14 days; the demo boxes approve nothing for 7
+  days; a box has fixes nobody approved for 14 days. All four are settings.
+- I released the host agent and the hub a second time today. The first release said "reboot needed" wrongly; the new
+  14-day alarm would then have mailed you about boxes you had already rebooted.
+
+**One decision for you (a safe default if you say nothing) — the Docker engine updates (designed, not built):**
+1. **Turn on Docker's "live-restore" on every box.** With it, a Docker engine update restarts no app (measured: 0
+   restarts). Without it, every app stops for about 30 seconds per engine update.
+   - **A (my pick):** turn it on — in the new-install image and once on existing boxes. It goes on without restarting
+     anything. Cost: it must never be turned off by a plain restart again (that stops every app and starts none).
+   - **B:** leave it off. Every Docker update then means ~30 seconds of every app being down, at night.
+   - **If you say nothing:** nothing changes; Docker updates stay unbuilt.
+
+**What I did:**
+- **The host's Debian fixes now install themselves**, after the guest's, on the same night run, only on appliances,
+  never a kernel or boot package, never a reboot. Proven: demo-felhom installed 108 host fixes and stayed healthy; all
+  108 came from Debian. A box made "ring 1" for a test installed exactly the one approved host version it lacked.
+- **A way to put one host package back by hand** is written and proven on demo-hp (and the test taught it two fixes).
+- **The fleet view** in the hub: one line per box with its updates, "reboot needed since", and the tunnel.
+- **The tunnel status is now true:** running, not running, or unknown. I blocked demo-hp's tunnel: after two reports
+  (about 30 minutes) you got a "tunnel down" mail; when I unblocked it, "recovered". A simply stopped tunnel heals
+  itself within 5 minutes, before the hub can even see it.
+- **The night run is fast:** 23–32 seconds when there is nothing to install (target was under 60).
+- **The kernel test on demo-hp (your two reboots):** Secure Boot works with it, but GRUB's "boot once" does not work
+  on our boxes: the second plain reboot came up on the NEW kernel again. So a new kernel that hangs would stay. That
+  must be solved before kernel updates. demo-hp now runs the newer kernel, healthy. A watchdog chip exists on demo-hp.
+- **Found and fixed during the live test:** "reboot needed" was wrong in two ways (fixed in the second release).
+- **New-install image baked and vouched** (0.292.0 with host agent 0.141.1). The golden waiver is gone: not needed.
+- **Rows:** 2 closed, 2 opened-and-closed the same day, 3 opened. The list went from 332 to 333.
+
+**Needs you later (nothing breaks if you wait):**
+- **A host that crashes does not restart by itself** (Linux's "panic" setting is off). Changing it changes how every
+  box behaves, so it is your call, together with kernel updates.
+
+## Today (2026-10-04, afternoon): the guest's security fixes install themselves
+
+**One decision for you (a safe default if you say nothing):**
+
+1. **Undo for a guest update that goes wrong.** I tested it first, as you asked: Proxmox cannot take a snapshot of a
+   customer box at all (the box is linked to the household's drives, and Proxmox refuses). So there is no automatic undo.
+   Today a failed check stops, mails you, and last night's whole-box backup (minutes old) is the undo, by hand.
+   - **A (my pick):** keep it so. Nothing new to build. A restore takes about 1–3 minutes plus losing what apps wrote
+     since the backup.
+   - **B:** build our own disk snapshot under Proxmox. Automatic, but a new mechanism nobody has tested, with a risk to
+     the disk pool.
+   - **If you say nothing:** A stays.
+
+**What I did:**
+- **Your two choices are built.** A returning household's new box sets the old off-site copy aside on night one and
+  starts a new one (nothing deleted). An approved version that Debian already replaced comes from Debian's dated archive.
+- **Guest security fixes now install themselves.** Each night, after the whole-box backup, the demo boxes install
+  Debian's fixes. When both demo boxes run the same versions healthy for 24 hours and one night, the hub approves that
+  set; every other box then installs exactly those versions. Docker, the host and the kernel are not touched.
+  - Proven live: each demo box installed 53 fixes and stayed healthy. With a 2-minute test wait the hub approved the
+    set; then I put back 24 hours. demo-felhom, made "ring 1" for the test, installed exactly the 3 approved versions it
+    lacked and nothing newer. A run where I stopped an app failed its check and mailed you (you got that mail).
+  - You can switch OS updates off per box in the hub. They are ON by default. The household sees one line in its
+    timeline: "System security fixes installed".
+- **The tunnel and two other built-in programs are current** (cloudflared was 4 months old). A new monthly check
+  catches them falling behind. The tunnel came back by itself on both demo boxes within 20 seconds.
+- **Three bugs found and fixed during the live test**, before release.
+- **Two gaps found, now rows:** existing boxes cannot receive the new update tool through the product (only new
+  installs, or by hand — the demo boxes got it by hand); and the hub's "tunnel status" for every box always reads
+  "inactive" because it checks the wrong place.
+- **Rows:** 4 closed (one opened and closed the same day), 5 opened. The list went from 331 to 333.

 ## Today (2026-10-04, day): off-site closed, operating-system updates measured

@@ -148,8 +262,8 @@ Your licence decisions are recorded: Emby, Plex and n8n stay. recipe-importer ne

 ## What needs you

-0. **The two decisions at the top of today's section** (a returning household's first night; approved OS updates
-   when Debian has moved on). Each has a safe default if you say nothing. The Hetzner key change stays your call (3 steps, in the list).
+0. **The undo choice at the top of today's section** (A: keep the backup as the undo; B: build our own snapshot).
+   If you say nothing, A stays. Earlier today's two decisions are built. The Hetzner key change stays your call.
 1. **plant-it:** keep the hidden template as it is, or remove it entirely (its image no longer exists). **If you say
   nothing:** it stays hidden; nothing runs it.
 2. **Send the SparkyFitness request, and ask the Tandoor authors** (the "Before the first paying customer" list).
@@ -232,7 +232,7 @@ likewise silent. Evidence: `audits/DRILL-r361-2026-08-22/evidence/06-part3-decis
 | **The hub reports LOSS OF VISIBILITY into either off-site store (not just how full it is)** | hub **v0.106.0** (R-339) | **IMPLEMENTED — deliberately NOT proven-live** | Both box checkers count consecutive failed fetch windows and emit `pbsdr_box_unreachable` / `offsite_box_unreachable` (severity `warning`) past a default 3 windows (≈30–45 min), each with a paired `*_recovered` all-clear routed via `recoveredPairedDownTypes` — required because the recoveries are severity `info`, which `severityNotifies` drops. Scopes stay customer-less (`pbsdr-box` / `pool-box`) → operator channel only. Fill logic untouched: a degraded read still drives no band transition. Evidence: `internal/monitor/box_reachability_test.go` + the cross-package wiring test in `internal/notify/`, which asserts an actual operator mail rather than a map entry. **Filed BECAUSE of a measured gap**, not a hypothesis: the 2026-08-18 ep0 outage ran 9 h 37 m with the hub silent | **The gap that remains is R-340**, and it is not small: the ep0 read is the `usage` op, which rides the LOCAL API daemon — the daemon that incident explicitly cleared — so this check would have shown GREEN for that entire outage. It closes "ep0 is unreachable as a host"; it does not close what actually happened. **No live or constructed outage has exercised the emit path**, and one cannot be manufactured against ep0 (Tier 2, protected) |
 | Secrets hygiene: bearer in k8s Secret, no secrets in git, single-quote credential store | hub v0.53, conventions | **IMPLEMENTED** | 07-13 closing bundle | |
 | Operator login password changeable from UI | hub v0.54 | **IMPLEMENTED** | 07-13 | |
-| Box operating-system security updates (Proxmox host, guest Debian, Docker engine) | — | **MISSING** | `felhom-host-install.sh:2133-2136` ("No upgrades are run") | Nothing runs them after install → finding R-812, intention R-808 (added 2026-10-03). Design: `architecture/11-os-updates.md` (NOT RATIFIED, 2026-10-04). **Spike done 2026-10-04** (`audits/os-updates-spike-2026-10-04/`): measured, nothing built — still MISSING |
+| Box operating-system security updates (Proxmox host, guest Debian, Docker engine) | agent v0.142.0, hub v0.132.0 | **PARTIAL — the GUEST and HOST Debian fast lanes and the DOCKER engine slow lane are PROVEN-LIVE (2026-10-04), with the System page, the fleet view and the alarms; the KERNEL lane is MISSING** | Guest: `audits/os-guest-lane-2026-10-04/`. Host + fleet + alarms: `audits/os-host-lane-2026-10-04/`. Docker + System page + crash guard: `audits/os-docker-crash-2026-10-04/` — live-restore on with the same container ids on every box; Docker 29.8.2 on both demo boxes; operator-approved Docker release; a signed undo and a signed ring-1 step; a replay refused; the crash guard restarted demo-hp twice and kept it off the third time. Design `architecture/11-os-updates.md` §5.8, §5.9, §8 | **No automatic undo** (guest: last night's backup; host: by-hand runbook; Docker: a signed undo job); existing boxes need the wrapper, the trust files and the guard by hand (R-840); the kernel lane (R-836); facts reach the hub late after a boot (R-853) |
 | **An ENGLISH-SPEAKING household's first hour: download, install, pair, bind, claim, two apps** | controller **v0.259.0** + hub **v0.119.0** + ISO 1.29.0 + the whole catalog | **PROVEN-LIVE on 0.258.0 with one blocker; THE BLOCKER IS FIXED AND PROVEN, THE WALK IS NOT REPEATED** | `audits/DRILL-first-hour-en-0258-2026-09-20.md` — a fresh install 2026-09-20, one intervention (R-494), stop rule not reached. Then `audits/i18n-closing-2026-09-21/live/` — the three blockers fixed and each proven on a live box or in the operator's inbox: the claim page answers English through the real cookie path; the Backup page's tier names follow the language; and the setup mail carries **four plain-ASCII English words** where the drill's carried `képző-szkítia-ásatás`, one day apart in the same inbox. | **R-596, R-597 and R-598 are CLOSED.** What this row still does NOT claim: **the fixed journey has not been walked end to end by a stranger on a fresh install.** Three fixes proven at the endpoint are not an hour proven by a person, and this project's own rule is that fixes are not a journey (see the recovery-journey row). **Also not walked:** the recovery code (needs ep0), backup/restore/remove/power-cut (proven 2026-09-14), and the two Backup-page *warnings* themselves — guest 9201 is healthy and a healthy box renders none, so they are covered by handler render tests, not live. **Verdict: nothing known now stands between an English-speaking tester and their box — and that is a different sentence from "the walk passed".** |
 | **A deletion of a customer's off-site history is NOTICED within a day** | hub **v0.111.0** (R-431) | **IMPLEMENTED — not yet PROVEN-LIVE** | 09-01 | `hub/internal/monitor/offsite.go` — third signal beside FILL and STALENESS. **On the hub deliberately:** a detector on the box is one the deletion can silence. Alarms when the reported count falls by more than HALF and by at least 5, guarded by `StatsKnown` (R-331), the declared `State` (R-204) and run success (R-100). **Threshold reasoned, not invented:** over 12 898 reports every decrease lands on ZERO and predates `stats_known`; in the 380-report `stats_known` window there are none. **ACCEPTANCE: 9 009 real points replayed → ZERO alarms** (`offsite_r431_test.go`, fixture committed). **What PROVEN-LIVE would need and this does NOT have:** a real drop observed on a live box producing a real mail — the live firing done at ship time was driven through the hub's own path with synthetic counts, which is an end-to-end delivery proof, not a proof that a genuine deletion is caught. |

@@ -47,7 +47,13 @@ Owns:
 1. **Proxmox lifecycle** — create/start/stop/destroy guests, snapshots, storage allocation. Via a scoped Proxmox API token (the **`FelhomAgent` operator role** — `proxmox-platform.md` §3.6, validated Phase 3 B3) for everything the API covers; raw host ops only where unavoidable.
 2. **Storage management** — attach/classify targets, reconcile the storage manifest, mount USB-by-UUID, present mounts into guests.
 3. **Backup/restore orchestration** — vzdump to the tiers, PBS, snapshot management, and the **self-restore-test**.
-4. **Host & tunnel monitoring** — host metrics, guest up/down, storage-target status, and `cloudflared` health; reports the host domain to the hub.
+4. **Host & tunnel monitoring** — host metrics, guest up/down, storage-target status, and `cloudflared` health; reports the host domain to the hub. **[FACT, 2026-10-04] The "cloudflared health" leg reads a host unit that does not exist:** `internal/hub/cloudflared.go` runs `systemctl is-active cloudflared` on the HOST, but cloudflared is a container in the guest (below), so every box reports `inactive` (`Unit cloudflared.service could not be found`, demo-hp). R-841. **[FACT, FIXED agent v0.141.0 / controller v0.292.0, R-841 CLOSED]** The agent now reads the guest's `cloudflared` container through the existing `pct exec [0-9]* -- docker inspect -f *` sudoers line: state, exit code and the Docker health status of a check the controller adds (`cloudflared tunnel --metrics localhost:20241 ready` → cloudflared's own `/ready`, 200 only with a connection). Three states: `running` (healthy), `not_running` (stopped, absent, or running but NOT connected), `unknown` (could not ask, or the check is still starting) — `unknown` never alarms. No new sudoers line. **[FACT, agent v0.142.0, R-852]** The host report gains `system`: the Proxmox version and kernel from the
+Proxmox API, plus the OS wrapper's read-only `facts` (host Debian, next-boot kernel, held packages, taint, the crash
+guard; guest Debian, Docker engine, containerd, live-restore), at most every 10 min. **Still no new sudoers line** — the
+facts, `live-restore-on` and the Docker layer all ride `FELHOM_OSAPPLY`. New ROOT-owned files instead (installer 1.30.0;
+by hand on installed boxes, R-840): `/etc/felhom/os-trust.json`, `/etc/felhom/operator-signers` (the wrapper verifies
+Docker authority against them, never against the agent-writable config — `09` decision 93), and the crash guard
+(`/usr/local/sbin/felhom-crash-guard`, its units, `/etc/felhom/crash-guard.conf`).
 5. **Provisioning** — provision a guest **by restoring the golden base image** (§9), deploy the controller into it, hand it its bootstrap config; also **build and refresh the golden base image** itself.
 6. **Hub control loop** — poll for desired state + signed jobs, reconcile, execute, report, heartbeat.
 7. **Local API** — the per-guest authorization gate the controller calls.
@@ -64,7 +70,7 @@ Explicitly does **not**:

 - **Native Go binary, systemd service** on the host: boot-start, `Restart=always`, systemd watchdog (kill+restart on hang), journald logging, resource limits.
 - **Root-minimized (boundary settled — Phase 3 B3).** The agent runs as a **non-root** service user with the scoped `FelhomAgent` token for all API-covered work + a **narrow `sudoers` allowlist** for true host ops. Per Phase 3 (B3) the boundary is settled: the entire per-customer guest lifecycle — provision (by restore, §9), config, start/stop, snapshot, backup, **restore**, destroy — is token-covered. Genuine OS-root is confined to: (1) building/refreshing the **golden base image** (`keyctl` create is `root@pam`-only — one-time at enrollment + a maintenance cadence, §9); (2) **host mounts** (USB mount-by-UUID, systemd mount units / fstab); (3) **SMART / hardware sensors**. Root therefore never sits on the per-customer path. See `proxmox-platform.md` §3.6 for the role + boundary table.
- **`cloudflared` is a separate systemd service**, not embedded in the agent. This is what makes the data path survive control-plane death by construction. The agent **manages and health-watches** it (see §5) but the tunnel does not live or die with the agent process.
+- ~~**`cloudflared` is a separate systemd service**, not embedded in the agent. … The agent **manages and health-watches** it (see §5) but the tunnel does not live or die with the agent process.~~ **[FACT, corrected 2026-10-04 — `11-os-updates.md` C8, R-838]** `cloudflared` is a **container in the customer guest** (`cloudflare/cloudflared:<pin>`), rendered and kept up by the in-guest controller (`felhom-controller/controller/internal/infra/infra.go`, `internal/stacks/infra.go`) and baked into the golden; there is no host systemd unit. It is still NOT embedded in the agent, so the data path survives the agent's death — and the agent does not manage it; it READS its health (R-841, agent v0.141.0). Its version moves only by a controller release (the pin), on the monthly re-test (`runbooks/monthly-floating-retest.md` "Infrastructure pins").

 ## 4. Control model — reconcile + signed destructive ops

@@ -143,7 +149,7 @@ notification) is the control.

 **Box-initiated poll.** The hub never connects inbound. Each poll cycle exchanges:

- **Up:** heartbeat + a host-domain state report — host CPU/RAM/disk, per-guest up/down + spec, storage-target status (USB connected? NFS/CIFS reachable? PBS reachable?), last backup per target, last restore-test result, `cloudflared` health, agent + controller versions, audit-log tail.
+- **Up:** heartbeat + a host-domain state report — host CPU/RAM/disk, per-guest up/down + spec, storage-target status (USB connected? NFS/CIFS reachable? PBS reachable?), last backup per target, last restore-test result, `cloudflared` health (`running` / `not_running` / `unknown` from the guest container's readiness check — R-841), agent + controller versions, audit-log tail.
 - **Down:** the current desired state, any pending signed one-shot jobs, and config (poll interval, update window, policy changes).

 **Dead-man's-switch (essential, not optional).** In a box-initiated model the heartbeat
@@ -633,6 +639,13 @@ buildable until then; recorded here so the front-half built in slice 7 lands rea
  never — the binary is what flips. Report field `selfupdate_pending` surfaces a runs-but-never-commits
  binary. v1 scope-outs: no hub-floor auto-update, no failed-update auto-retry (the operator re-signs),
  no pending-timeout auto-rollback.
+- **OS updates, guest fast lane (agent v0.140.0, `11-os-updates.md` §8.1).** A second root-owned wrapper,
+  `/usr/local/sbin/felhom-os-apply`, behind ONE sudoers entry (`FELHOM_OSAPPLY`: `felhom-os-apply --plan
+  /var/lib/felhom-agent/os/plan-*.json`). The agent has no `apt` grant of its own for this; every rule (no removal,
+  no downgrade, no new or unlisted package, Debian origin only, the box's own customer guest only) is in the wrapper,
+  red-proved per rule. The leg runs after a successful primary whole-guest backup, under the heavy-op gate.
+  **[FACT] The signed agent update does NOT carry the wrapper or the sudoers line** — only the installer installs
+  them (R-840).
 - **Controller (the easy case — it's a guest).** The agent owns the controller's lifecycle,
  so the **agent updates the controller**: snapshot-before-update (free rollback, because the
  controller *is* a snapshottable guest) → pull new image → redeploy → health-check → rollback
@@ -362,6 +362,18 @@ never moved by an update or an undo, and the unit restore accepts it.
 > **R-191 (2026-08-04) — this row was RIGHT and the configuration disagreed with it, weekly, for as long as R-89 has been in force.** The contract has not changed: offsite retention is ep0's, the box's token is write-only, and the box cannot delete its own history. What had not followed was the installer's `keep_last: 2` on the offsite tier, so every weekly run uploaded its snapshot successfully and then failed the whole JOB on a prune the token is refused — `whole_guest_backup_failed` in the operator's inbox about a backup that had already succeeded. Fixed in installer **1.25.0** (`keep_last: 0`) and on both live boxes; a gate now asserts it. **Verified before changing it:** ep0's two prune jobs have run every day since 2026-07-27, 18 tasks, all OK. A doc that states the contract does not enforce it — the gate does.


+**[FACT, 2026-10-04 — agent v0.140.0, `11-os-updates.md` §8.1] The OS leg closes the night.** After the
+whole-guest backup (the controller drives it, inside [W+2h, W+6h)) ends SUCCESSFULLY on the primary tier, the agent
+waits 90 s and runs the guest's Debian fast lane — still holding the host-wide heavy-op gate, so it never overlaps a
+backup or a restore-test; at most once per 20 h. The backup minutes old is the guest's undo (no snapshot is possible,
+R-837). A failed or missed backup → no OS leg that night. **[FACT, 2026-10-04 — agent v0.141.1, `11` §8.2]** On an
+appliance, the HOST step follows under the same gate: after a healthy guest step only (a failed or unhealthy guest
+step skips it), Debian-origin fixes only, never a kernel, boot or firmware package, never a reboot. Measured: both
+steps with nothing to install, 23–32 s; a 108-package host pass, 70 s. **[FACT, 2026-10-04 — agent v0.142.0, `11` §5.8]** On a RING-0 box a third step
+follows, still under the gate: the guest's Docker engine set (live-restore turned on first, once, by reload; every
+container must keep its id). Ring 1 never takes it in the night leg. Measured: the three steps with nothing to install,
+38–51 s; a six-package Docker step, ~50 s.
+
 **[FACT]** The three nightly legs derive from **one** customer-settable window start W at fixed
 offsets — db-dump at W, Tier-2 at W+60m, offsite at W+105m — so they can never be misordered
 (`cmd/controller/main.go:604-607`). Both boxes run W = `02:30`.
@@ -310,6 +310,36 @@ message, not a wider cooldown.

 ---

+## 6.3 Box alarms outside the app ladder: the tunnel and OS updates [DESIGN, hub v0.131.0, 2026-10-04]
+
+These are **operator-only** (the household can act on none of them — except `host_restarted_after_crash`, the household's
+one info line beside `host_crash_restart`; `operatorOnlyEvents`, pinned by
+`TestOSUpdateEvents_OperatorOnlyExceptApplied`). They go through the same dispatcher and severity contract (§6.1):
+`info` is recorded and never mailed; `warning` and `error` are mailed.
+
+| Event | Severity | Raised when | Cleared | Pinned by |
+|---|---|---|---|---|
+| `tunnel_down` | error | the box's two newest host reports say the tunnel is `not_running` (more than one report cycle, 15 min) and the one before did not | the first `running` after it → `tunnel_recovered` (info) | `api/tunnel_test.go` |
+| `os_update_stale` | warning | no successful OS leg for **7 days** while the switch is ON (agents that can run the leg only); the mail names the likely reason (box not reporting / the last leg's failure / no good night backup) | a successful leg | `TestAlarm_StaleLeg`, `TestAlarm_StaleNamesTheReason` |
+| `os_reboot_needed` | warning | the host has needed a reboot for **14 days** (from the FIRST scanned report that said so) | a scanned pass that finds nothing (agent ≥ 0.141.1 scans the host every pass) | `TestAlarm_RebootNeeded`, `TestRebootNeeded_ClearedByAScannedPass` |
+| `os_ring0_stalled` | error | ring 0 approved nothing for **7 days** in a layer while it has pending FAST-lane updates (a pending kernel does not count) | a new release | `TestAlarm_Ring0Stalled` |
+| `os_not_covered` | warning | a ring-1 box has had fast-lane packages no approved release names for **14 days** | the packages are covered or gone | `TestAlarm_NotCovered` |
+| `host_crash_restart` | warning | the box's crash guard reports a NEW unclean boot (a crash, a power cut or a hard reset; hub v0.132.0) | — (one per boot) | `api/crash_test.go` |
+| `host_crash_guard_tripped` | error | the guard tripped: the next crash leaves the box OFF | the re-arm → `host_crash_guard_rearmed` (info) | `api/crash_test.go` |
+| `host_kernel_oops` | warning | a kernel oops this boot (taint D) — the box keeps running | — (once per boot) | `api/crash_test.go` |
+
+- **`unknown` never alarms** (R-96 rule 3): a probe that could not ask is neither up nor down. An `unknown` report
+  breaks a `not_running` run.
+- **A stopped cloudflared heals itself before the hub can see it** (measured 2026-10-04): the controller's
+  protected-container check recreates it within 5 minutes, and the host reports every 15. So `tunnel_down` catches
+  what the box cannot heal — a running container with no connection (wrong token, blocked network).
+- The OS alarms are checked **hourly**, re-sent at most **once a week** while true, and forgotten when false, so the
+  next occurrence is announced again. The four numbers are configuration (`OS_ALARM_STALE_AFTER`,
+  `OS_ALARM_REBOOT_AFTER`, `OS_ALARM_RING0_STALL_AFTER`, `OS_ALARM_NOT_COVERED_AFTER`) — *decided by CC unattended,
+  operator may reverse* (`11` §8.3).
+
+---
+
 ## 7. The intent test [DESIGN, R-386 — CLOSED controller v0.223.0]

 **"The customer stopped this" is asked of the FIELD THAT RECORDS IT, never inferred from the state.**
@@ -765,6 +765,98 @@ its length, and both fixes cost something the household would notice — operato
 80. **Next build: the guest fast lane** (`11` §8 step 2), with R-837 (the snapshot undo) measured first and R-838 (the
    infrastructure images) in the same session. *Operator ruling 2026-10-04 ~10:17.*

+### 2026-10-04 (~12:20) — three operator rulings (recorded before the work)
+
+81. **The undo for a failed OS update is the whole-guest backup, restored by hand (R-842, option A).** Nothing new is
+    built. **Rejected:** a home-made LVM-thin snapshot behind Proxmox's back (unmeasured; thin-pool risk).
+    *Operator ruling 2026-10-04 ~12:20.*
+82. **R-840 (a route for wrappers/sudoers to installed boxes) is not built now** — *"There are no older boxes."* No
+    installed box exists outside the two demo boxes, and they take wrapper changes by hand. The row stays open, not
+    re-ranked. *Reviewer's note, recorded as written:* the ruling is right for today, but every box installed from now on
+    becomes an "older box" the first time the wrapper or the sudoers line changes again; R-840 must be solved before the
+    first box that cannot be reached by hand. *Operator ruling 2026-10-04 ~12:20.*
+83. **Next: the tunnel status (R-841), the host fast lane (`11` §8 step 3), and other OS-update improvements.**
+    *Operator ruling 2026-10-04 ~12:20.*
+
+### 2026-10-04 (~18:00) — operator ruling after the R-858 incident
+
+95. **A Docker engine step restarts the containers that mount the Docker socket** (R-858): after every step that
+    installed something, the wrapper restarts ONLY those containers (today `felhom-controller` and `traefik`, ~10 s;
+    apps untouched, the engine untouched), and the health rule checks that the controller reaches Docker from inside its
+    container. **Rejected:** mounting a socket folder instead of the file (cleaner, but it changes Docker's config, the
+    controller templates and the golden on every box). Until the fix is released, the demo boxes' root-owned ring-0
+    Docker mark is OFF (no unsigned step). *Operator ruling 2026-10-04 ~18:00.*
+
+### 2026-10-04 (~18:49) — four operator rulings (recorded before the work; the R-840 / Tester 2 brief)
+
+96. **R-840: build the route now, option A** (a signed `agent_config_update` op pinning the agent tag and the sha256 of
+    a config bundle; the root side installs it after its own checks), and use it on Tester 2. Every later box needs it.
+    **This replaces decision 82.** *Operator ruling 2026-10-04 ~18:49.*
+97. **Tester 2 may receive, in this session only:** the signed agent update to the vouched agent; the config bundle
+    through the new route; the one-time `live-restore` reload through the wrapper (a reload, never a restart). Nothing
+    else: no Docker engine step, no reboot, no crash test, no app change. *Operator ruling 2026-10-04 ~18:49.*
+    **Added ~19:05, the operator's answer:** the first bundle cannot reach Tester 2 by the route (its root side predates
+    the route — see `11` §5.4.2); the operator will try to reach Tester 2 through his own WireGuard tunnel and install the
+    one bootstrap file by hand, with CC's written steps.
+98. **drill-r50 is removed from the hub. tester-1 stays** (CC's disposable test box, a VM on the HP box, down when not
+    used). **Added ~19:05, the operator's answer:** the product delete may touch ep0 — it destroys drill-r50's PBS
+    namespace and token there and removes its WireGuard peer. *Operator ruling 2026-10-04 ~18:49 / ~19:05.*
+99. **The operator's cause, with the reviewer's correction:** *"we didn't bake a new golden before he joined."* Partly:
+    the golden carries `live-restore` and the Docker version; the crash guard and the operator-signers file come from the
+    INSTALLER; the wrapper comes from the agent tag the installer pins. A newer golden alone would not have fixed it, and
+    with perfect timing the next wrapper change would still leave Tester 2 behind — that is R-840. **CC's measurement
+    (same evening) corrects the premise further:** Tester 2 was bound at 16:06 **UTC** (18:06 local), after installer
+    1.30.0, agent 0.142.0 and the re-made golden; it lacks only agent 0.142.1's wrapper fix (R-858). *Operator ruling
+    2026-10-04 ~18:49.*
+
+### 2026-10-04 (evening) — decided by CC unattended — operator may reverse (System page / Docker / crash-restart brief)
+
+90. **How does a box tell a crash boot from a clean one?** Options: (a) `pstore` — measured on demo-hp: `efi_pstore` is on,
+    yet a real panic saved NOTHING; (b) the previous boot's journal ends without shutdown lines — readable only after
+    the journal is up, and slow; (c) a marker written by an ExecStop at every orderly shutdown. **Chosen (c):** cheap,
+    early, measured to work; cost: a power cut and a hard reset count as a crash too (they cannot be told apart on these
+    boxes). `11` §5.9.
+91. **`kernel.panic_on_oops`?** Options: set it (an oops becomes a restart, which may loop on a bad driver) or leave it
+    0 and REPORT an oops (taint bit D) to the operator. **Chosen: leave 0, report** — a box with an oops usually keeps
+    serving the household; the hub mails `host_kernel_oops`. `11` §5.9.
+92. **The guard numbers.** The brief said both "at 3 it sets panic=0 so the next crash leaves the box off" (= the 4th)
+    and "if it crashes 3 times within one hour, it stays off" (= the 3rd, the operator's own page). **Chosen: the
+    operator's words** — the 3rd unclean stop within 60 minutes leaves the box off (the guard trips at the 2nd crash
+    boot); `kernel.panic` 10 s; re-arm after 24 h of normal running. All four are `/etc/felhom/crash-guard.conf`.
+93. **Who may authorize a Docker step on the box itself?** Options: the agent's word (its config is agent-writable);
+    the wrapper verifying the signed job against ROOT-owned files. **Chosen: the wrapper verifies** — `ssh-keygen -Y
+    verify` against `/etc/felhom/operator-signers`, bound to `/etc/felhom/os-trust.json` `host_id`, a root-owned nonce
+    record; an unsigned ring-0 step only with that file's `ring0_slow_lane: true`, set by hand on the demo boxes only.
+    Cost: two more root files per box (the installer writes them; R-840 for installed boxes). `11` §5.8.
+94. **The System page colours** use the alarm thresholds themselves (red = an `08` alarm would fire; amber = worth a
+    look; `unknown` = amber with its reason). No second set of numbers. `11` §5.7.
+
+### 2026-10-04 (~15:17) — three operator rulings (recorded before the work)
+
+87. **Docker `live-restore` is ON for every box** (`11` §5.8, option A). It is turned on once — by the golden and by a
+    one-time step on each installed box — and never turned off by a plain restart (R-835). *Operator ruling 2026-10-04
+    ~15:17.*
+88. **A crashed host restarts by itself, with a limit** (R-851): *"Yes, but maybe not indefinitely."* After a limit of
+    crashes in a short time the box stays off and the operator is told. *Operator ruling 2026-10-04 ~15:17.*
+89. **The operator must see the boxes' OS and Proxmox versions in the hub** (*"Where should I be able to see the
+    OS/Proxmox versions of the boxes?"* — today: nowhere; R-852). *Operator ruling 2026-10-04 ~15:17.*
+
+### 2026-10-04 (afternoon) — decided by CC unattended — operator may reverse (host fast lane brief)
+
+84. **Which record proves a box is an appliance, for the host fast lane?** Options: (a) `agent.json`
+    `deployment_mode` — the agent can write it, so a compromised agent could claim "appliance" and unlock host
+    updates; (b) the installer's ROOT-owned `/var/lib/felhom-install/state.json` `mode` — written once, as root, at
+    install. **Chosen (b):** the wrapper is the root fence and must not trust a file the agent can change. Cost: a box
+    whose install record is missing gets no host step (fails closed). `11` §8.2.
+85. **The four OS alarm thresholds.** Options: shorter (3/7 days — noisy: a weekend away alarms) or longer (14/30 —
+    a stopped box goes unseen for weeks). **Chosen:** no OS leg 7 days, reboot needed 14, ring 0 stalled 7, not covered
+    14; all four are configuration. Cost: a real stop is seen after a week, not a night. `11` §8.3, `08` §6.3.
+86. **A second release of the agent (v0.141.1) and the hub (v0.131.1) in the same session**, against "one release per
+    repo". Options: (a) keep v0.141.0 / v0.131.0 and file the defect — the host "reboot needed" stays wrong (it hid
+    `lxc-start`, and a reboot never cleared it), so the new 14-day alarm would fire on rebooted hosts; (b) patch now.
+    **Chosen (b):** a known-false operator alarm is worse than an extra release; both patches are small and red-proved.
+    R-846.
+
 ### 2026-09-30 (day) — operator notes, recorded before the work

 - **The day brief runs by day.** Every backup and automatic-update test is started by hand — the night chain's debug
@@ -2,7 +2,7 @@

 > | | |
 > |---|---|
-> | **Status** | **NOT RATIFIED — a PROPOSAL with one operator ruling, corrected by the 2026-10-04 spike (§7.1, corrections C1–C12 below).** Ratification is Viktor's review, not an editor's. |
+> | **Status** | **NOT RATIFIED — a PROPOSAL with operator rulings, corrected by the 2026-10-04 spike (§7.1, C1–C12); §8 step 2 BUILT 2026-10-04 (§8.1).** Ratification is Viktor's review, not an editor's. |
 > | **Written** | 2026-10-04, by the reviewer (project Claude), before any spike. |
 > | **Verified against** | felhom.eu `d07a1a9` · felhom-controller `99a1497` (v0.290.0) · felhom-agent `d766666` (v0.138.0) · hub v0.128.0 |
 > | **Freshness** | **CURRENT** as of 2026-10-04. The spike `TASK-backup-close-and-os-updates-spike-2026-10-04` adds measurements here as `[FACT]` and corrects every claim it disproves. Mark this file STALE when it falls behind what the product does. |
@@ -324,18 +324,81 @@ must never overlap a backup, a restore-test or a self-update.~~
 |---|---|---|
 | Guest packages | Restore the guest snapshot taken just before the update | It also undoes app data written after the snapshot. Use it only inside the health window, before apps have written much. After that, install the previous version (needs OPEN Q1). **[FACT] (C6)** Customer guests are on LVM-thin and can snapshot; scratch 9202 (`dir` storage) cannot (`snapshot feature is not available`), so the measured undo was a whole-guest backup + restore: **73 s down**, all apps healthy, libc back. The snapshot rollback itself is unmeasured (R-837). |
 | Docker engine | Install the previous version (Docker's repository keeps old versions — OPEN Q2) | Every container restarts again. **[FACT] (C5)** Docker keeps 46 `docker-ce` versions. A step (either way) without `live-restore`: 6 of 6 containers restart, the app silent **26.5–30 s**, healthy at +42–45 s. With `live-restore` on: **0 restarts, no gap**, also across a containerd step. But a restart that turns `live-restore` OFF stops every container and starts NONE (`unless-stopped` ignored) — on 9202 they stayed down until restarted by hand; `systemctl reload` turns it on but not off. |
-| Host packages | Install the previous version | Only if the source still has it (OPEN Q1/Q2). **[FACT]** `rsync` back to its pre-update version: `not found`; `libpng16-16t64` back to the point-release version: worked. A Debian undo needs the snapshot archive (C2). |
-| Host kernel | ~~Boot the previous kernel. Proxmox can boot a new kernel **once** (`proxmox-boot-tool kernel pin <ver> --next-boot`). If that boot fails, the next boot uses the old kernel again. Make the new kernel permanent only after a healthy boot.~~ **[FACT] (C4)** Both demo hosts boot UEFI + GRUB (no proxmox-boot-tool ESPs). **Installing a kernel makes it the GRUB default at once.** `--next-boot` on GRUB writes an ordinary `GRUB_DEFAULT` + `update-grub`; `proxmox-boot-cleanup.service` clears it only once a boot reaches userspace. Measured on demo-hp (old kernel pinned permanently FIRST, new pinned for next boot): boot 1 → `7.0.14-20-pve`, healthy, 60 s; boot 2 → back on `7.0.2-6-pve`, healthy, 76 s. **So the fallback works after a boot that succeeds; read from the code, a kernel that hangs before userspace stays the default on every power cycle.** `[PROPOSAL]` use GRUB's own one-shot (`GRUB_DEFAULT=saved` + `grub-reboot`) with the old kernel as the saved default — unmeasured (R-836). | ~~If the new kernel hangs, someone must switch the box off and on. The spike checks whether a hardware watchdog can do that (OPEN Q4).~~ **[FACT]** Only `softdog` runs (loaded by `watchdog-mux`); it cannot rescue a kernel that never boots. demo-hp has an AMD FCH whose `sp5100_tco` driver ships but is not loaded; untested. A hang still needs a person. |
+| Host packages | Install the previous version | Only if the source still has it (OPEN Q1/Q2). **[FACT]** `rsync` back to its pre-update version: `not found`; `libpng16-16t64` back to the point-release version: worked. A Debian undo needs the snapshot archive (C2). **[FACT, 2026-10-04]** Proved by hand: `runbooks/os-updates-host-undo.md` (demo-hp, `tzdata` back one version from `snapshot.debian.org`, held, released). Use the NEW version's `first_seen` as the timestamp when there is no previous host release; a package that pins its siblings (`eject` → `libmount1 =`) goes back only with them. |
+| Host kernel | ~~Boot the previous kernel. Proxmox can boot a new kernel **once** (`proxmox-boot-tool kernel pin <ver> --next-boot`). If that boot fails, the next boot uses the old kernel again. Make the new kernel permanent only after a healthy boot.~~ **[FACT] (C4)** Both demo hosts boot UEFI + GRUB (no proxmox-boot-tool ESPs). **Installing a kernel makes it the GRUB default at once.** `--next-boot` on GRUB writes an ordinary `GRUB_DEFAULT` + `update-grub`; `proxmox-boot-cleanup.service` clears it only once a boot reaches userspace. Measured on demo-hp (old kernel pinned permanently FIRST, new pinned for next boot): boot 1 → `7.0.14-20-pve`, healthy, 60 s; boot 2 → back on `7.0.2-6-pve`, healthy, 76 s. **So the fallback works after a boot that succeeds; read from the code, a kernel that hangs before userspace stays the default on every power cycle.** ~~`[PROPOSAL]` use GRUB's own one-shot (`GRUB_DEFAULT=saved` + `grub-reboot`) with the old kernel as the saved default — unmeasured (R-836).~~ **[FACT, 2026-10-04, demo-hp, operator's word before each reboot] GRUB's one-shot is NOT a one-shot here either.** With `GRUB_DEFAULT=saved` (old 7.0.2-6 saved) and `grub-reboot` 7.0.14-20: boot 1 → 7.0.14-20, **Secure Boot ON, booted fine** (signed kernel, shim → GRUB); but `/boot` is ext4 on LVM, GRUB cannot write its environment block there (`grub-reboot` warns so itself), `next_entry` was never cleared, and boot 2 with no command → **7.0.14-20 again**. A kernel lane needs a writable env block (the ESP) or a userspace "boot good" step — R-836. demo-hp left on 7.0.14-20, saved default 7.0.14-20, both kernels installed (`audits/os-host-lane-2026-10-04/partE/`). | ~~If the new kernel hangs, someone must switch the box off and on. The spike checks whether a hardware watchdog can do that (OPEN Q4).~~ **[FACT]** Only `softdog` runs (loaded by `watchdog-mux`); it cannot rescue a kernel that never boots. demo-hp has an AMD FCH whose `sp5100_tco` driver ships but is not loaded; untested. A hang still needs a person. **[FACT, 2026-10-04]** `sp5100_tco` is blacklisted by the Proxmox kernel package; loaded by hand it answers (`SP5100 TCO timer`, 60 s, inactive, nowayout 0 — read from sysfs, never opened, so never armed; unloaded). `kernel.panic = 0`: a panic leaves the host stopped (R-851). |

 ### 5.7 Telling people

 - **Operator:** a hub event for each update and each failure; a fleet view showing each box's OS release,
  how far behind it is, and whether it needs a reboot. A box that is more than N days behind the newest
  approved release raises an alarm (`08`).
+- **[FACT, hub v0.132.0] The System page** (`/system`, R-852, decision 89): one row per box — ring and switch with
+  buttons, the tunnel, host Proxmox / running and next-boot kernel / Debian / release / pending / not covered / held /
+  reboot needed / `kernel.panic` / oops / crash restarts / the guard, guest Debian / release / pending / restart needed,
+  Docker engine / containerd / live-restore / release, the last leg — and above it the releases, what ring 0 runs, "Approve
+  now" and "Approve Docker set". Colours are the alarm thresholds (decision 94). Hosts shows Proxmox / kernel too.
 - **Household:** one line on the timeline in both languages, informal voice: what was updated and
  whether the box restarted. Telling households in advance that the box may restart at night is a
  **promise to users**. That is the operator's decision when the slow lane is built.

+### 5.8 The Docker engine slow lane — BUILT 2026-10-04 (agent v0.142.0, hub v0.132.0)  `[FACT]`
+
+**As built** (evidence `audits/os-docker-crash-2026-10-04/partB/`; decisions 87, 93):
+- **live-restore ON**: the golden bakes it (`build-golden.sh` 3.1.0, fail-closed assertion); an installed box gets it once
+  by the wrapper's `live-restore-on` (merge into daemon.json + `systemctl reload docker`). Measured: the same container ids
+  after (9202 6/6 by hand — R10 refuses a scratch guest by design —, demo-hp 24/24, demo-felhom 5/5).
+- **The step** runs in the night leg after a healthy guest and host step, **ring 0 only**, `select pending-docker`,
+  allowed by the wrapper only with the root-owned `ring0_slow_lane` mark. **Ring 1 and every undo** only through a signed
+  `os_docker_step` the wrapper re-verifies itself (decision 93). Measured: 29.7.x → 29.8.2 on both demo boxes, every id
+  kept; a signed undo to 29.7.2 on demo-hp (41 s, ids kept) and back; demo-felhom as ring 1 by a signed job; a replayed
+  job refused by the agent (nonce).
+- **Approval**: the hub never approves a Docker set automatically; the System page's button works after every ring-0 box
+  ran the set in 2 healthy night Docker steps (`OS_DOCKER_APPROVE_NIGHTS` TEST override, logged). An approval nudges no
+  box. Undo: `runbooks/os-updates-docker-undo.md`.
+
+**The design as written before the build:**
+
+Built from C5 (measured) and R-835. **The one decision it needs is in STATUS: `live-restore` on, fleet-wide.**
+
+- **What moves:** `docker-ce`, `docker-ce-cli`, `containerd.io`, `docker-buildx-plugin`, `docker-compose-plugin`,
+  `docker-ce-rootless-extras` in the customer guest — today the six "not covered" packages on every box. One
+  approved **engine set** at a time, like a controller floor: the operator approves it (slow lane, §5.2), after ring 0
+  has run it for at least 2 nights healthy. Never two steps in one night.
+- **Precondition: `live-restore` ON.** Without it an engine step restarts every container — 26.5–30 s of silence,
+  healthy at +42–45 s (C5). With it: 0 restarts, no gap, also across a containerd step. Turning it ON is safe
+  (`systemctl reload docker` applies it without a restart, C5); turning it OFF later by a plain restart stops every
+  container and starts none (R-835) — so it is turned on once, by the golden and by a one-time fleet step, and never
+  turned off by the lane.
+- **Who and when:** the agent, through the same wrapper (`lane: slow`, refusal R3: only inside a verified signed
+  operator job, R-530's mechanism), in the guest, after the guest and host fast-lane steps, under the same heavy-op
+  gate, on a night the operator scheduled. Debian origin rule replaced by "origin `Docker CE`, exactly these names".
+- **Health:** the guest rule (§8.1) plus `docker version` reports the approved engine, and every container running
+  at the start is running with the SAME container id (proof that `live-restore` held). A changed id is
+  `health_failed` even if the app is healthy — it means the households' apps restarted when they should not have.
+- **Undo:** install the previous engine set (Docker's repository keeps 46 versions, C2) — by an operator job, with
+  `live-restore` still on, so the undo is also restart-free.
+- **Not covered here:** the golden's own engine (baked weekly; a new golden carries the approved set), and BYO hosts
+  (the guest is ours on both, so the lane applies there too).
+
+### 5.9 A crashed host restarts, with a limit — BUILT 2026-10-04 (agent v0.142.0, installer 1.30.0)  `[FACT]`
+
+Decision 88 (R-851): *"Yes, but maybe not indefinitely."* Evidence `audits/os-docker-crash-2026-10-04/partC/`.
+- **Measured first (demo-hp, the operator's word before each crash):** `kernel.panic = 10` + `echo c >
+  /proc/sysrq-trigger` → the box restarted by itself in 54 s, on the same kernel (the saved default), the agent up 18 s
+  after the boot. `kernel.panic` set by `sysctl -w` is gone after the restart (back to 0) — it must be set at every boot.
+- **The crash signal** (decision 90): `efi_pstore` is on, yet a real panic saved NOTHING; the journal and `last` show
+  only "no shutdown". The guard uses a **clean-stop marker** (the unit's ExecStop at every orderly shutdown); a boot
+  without it followed a crash, a power cut or a hard reset — counted alike.
+- **The guard** (`felhom-crash-guard`, early boot unit + hourly re-arm timer): armed → `kernel.panic = 10`; the 2nd
+  unclean boot within 60 minutes TRIPS it (`kernel.panic = 0`), so **the 3rd crash within the hour leaves the box off**
+  (decision 92 — the operator's words); re-arms after 24 h of normal running or `felhom-crash-guard rearm`. A crash before
+  the unit runs (very early boot) leaves the box off: the safe side. `panic_on_oops` stays 0; an oops is reported
+  (decision 91).
+- **Telling people:** each unclean boot = an operator mail (`host_crash_restart`) + the household's line; the trip = an
+  alarm (`host_crash_guard_tripped`); re-arm and oops announced once (`08` §6.3). The System page shows the guard.
+- **Measured live:** crash 1 → back in 54 s; crash 2 → back in 53 s, guard tripped; crash 3 → **stayed off** until the
+  operator switched it on; the hub mailed the trip (15 min after the boot — R-853); re-armed by hand.
+
 ---

 ## 6. Risks and edge cases
@@ -403,15 +466,99 @@ not covered: 79 Proxmox + 1 Tailscale on the host (slow lane / not ours), 6 Dock
 Each step returns to the operator for go or no-go.

 1. **Spike** (measure Q1–Q10; no product code).
-2. **Guest Debian, fast lane.** Lowest risk: a snapshot undo exists.
-3. **Host Debian, fast lane** (no kernel, no Proxmox packages).
-4. **Fleet view and alarms** (§5.7).
-5. **Slow lane: Docker engine.**
+2. **Guest Debian, fast lane.** ~~Lowest risk: a snapshot undo exists.~~ **BUILT 2026-10-04** — agent v0.140.0, hub
+   v0.130.0, installer 1.29.0; §8.1. **There is no snapshot undo** (R-837, measured).
+3. **Host Debian, fast lane** (no kernel, no Proxmox packages). **BUILT 2026-10-04** — agent v0.141.1, hub v0.131.1;
+   §8.2.
+4. **Fleet view and alarms** (§5.7). **BUILT 2026-10-04** — hub v0.131.0/v0.131.1; §8.3.
+5. **Slow lane: Docker engine.** **BUILT 2026-10-04** — agent v0.142.0, hub v0.132.0; §5.8.
 6. **Slow lane: host kernel and Proxmox packages, with the reboot.**
 7. **Later:** the Proxmox major upgrade (PVE 9 → 10), drilled on ring 0 first.

 ---

+### 8.1 Step 2 as BUILT (2026-10-04)  `[FACT]`
+
+Evidence: `audits/os-guest-lane-2026-10-04/` (parts A–G). Brief: guest fast lane, decisions 78–80 (`09` §3).
+
+- **The undo (R-837, measured first): none automatic.** PVE refuses ANY snapshot of a customer guest —
+  `PVE/AbstractConfig.pm:755-757` skips non-snapshot mounts only for a snapshot named `vzdump`, and every customer
+  guest carries the host-path binds mp8/mp9. As the agent's token and as root: `snapshot feature is not available`.
+  The token's role HAS `VM.Snapshot` / `VM.Snapshot.Rollback`. So §5.6's first row does not exist: a failed health
+  check stops, reports `health_failed`, and the hub mails the operator; the whole-guest backup taken minutes earlier
+  is the undo, by hand. The choice of a real undo is in STATUS (R-842).
+- **The wrapper** `felhom-os-apply` (agent repo `configs/`), Python 3 stdlib, per §5.4.1 with refusals R1–R13 (R12:
+  the host layer; R13: dpkg still broken after the repair). **Changed from the draft** *(decided by CC unattended —
+  operator may reverse)*: one sudoers entry (`--plan <file>`); the plan's `mode` field (`inventory` / `apply` /
+  `health`) replaces a separate `--repair-only` (the repair runs first on every apply); Python, because a JSON plan
+  cannot be parsed safely in sh. The box's own customer guest is "the guest that binds `/mnt/felhom-drives`" (R10).
+- **The leg** (agent `internal/osupdate`): after a SUCCESSFUL primary whole-guest backup, inside the backup's
+  goroutine before the host-wide heavy-op gate is released (so never beside a backup or a restore-test, C10), 90 s
+  after the backup, at most once per 20 h. *Decided by CC unattended — operator may reverse:* the 90 s settle and the
+  20 h gap; the controller's own self-update (04:30) is not detected — the 5-minute health wait absorbs a restart.
+- **The health rule** (`HealthVerdict`, pinned): docker answers, the guest resolves `deb.debian.org`, the controller's
+  health check is `healthy`, and every container running at the START of the leg runs again (healthy if it was). The
+  baseline merges the inventory's reading with the apply's own — found live: an app stopped between them escaped the
+  first rule. Wait 5 min, poll 15 s.
+- **Rings and the switch** (hub, per box): ring 0 = demo-hp + demo-felhom, everything else ring 1; switch ON by
+  default; OFF → the box reports, installs nothing. A box with no `os_update` block (older hub) = ring 1, ON, no
+  release.
+- **The approval rule (the ruled "1–2 day wait")**: every Debian / Debian-Security package=version that ALL ring-0
+  boxes having it agree on; approved when, since that set was first seen, **24 h** passed with every ring-0 report
+  healthy and every ring-0 box completed **1** post-backup night run (`OS_APPROVE_AFTER`, `OS_APPROVE_NIGHTS`; an
+  override is logged as a TEST configuration). The approval time is the snapshot.debian.org timestamp (decision 79).
+  "Approve now" is an operator event. *The 24 h / 1 night numbers: decided by CC unattended within the ruled 1–2 days.*
+- **The household's line**: hub event `os_update_applied` (info: on the household's hub timeline, never mailed;
+  hu/en in the bundle). There is no surface on the box itself (R-844).
+- **Measured live**: ring 0 — 53 Debian packages on each demo box (18.7–31.7 s inside the wrapper; 174–226 s for the
+  whole leg incl. the inventory), healthy, 6 Docker updates "not covered"; approval — with a 2-minute TEST wait, a
+  272-package release approved automatically, then the ruled values restored; ring 1 — demo-felhom installed exactly
+  the 3 approved versions it lacked (269 already current) and left the newer Docker packages alone; a deliberately
+  stopped app → `health_failed` after 5 min, the operator mailed, the household line recorded.
+- **Not delivered to existing boxes by the product**: the wrapper and the sudoers line reach a box only through the
+  installer; the signed agent update replaces the binary only (R-840). The demo boxes got them BY HAND.
+
+### 8.2 Step 3 as BUILT (2026-10-04)  `[FACT]`
+
+Evidence: `audits/os-host-lane-2026-10-04/` (parts A–G).
+
+- **Where it runs:** appliances only. The proof is the ROOT-owned install record `/var/lib/felhom-install/state.json`
+  `mode: appliance` (written by the installer as root); the agent-writable `agent.json` `deployment_mode` is not
+  trusted for this. A BYO host gets no host step (wrapper refusal **R12**, now lifted only for lane fast / layer host
+  on an appliance). *Decided by CC unattended — operator may reverse.*
+- **What:** origin `Debian` / `Debian-Security` only, and never a kernel, boot or firmware package (name pattern
+  `HOST_SLOW_RE` — `linux-*`, `proxmox-kernel*`, `pve-kernel*`, `pve-firmware`, `firmware-*`, `grub*`, `shim*`,
+  `systemd-boot*`, `*-microcode`, `efibootmgr`; refusal **R14**). The hub leaves the same names out of the host
+  candidate.
+- **When:** in the same leg, after the guest step, under the same heavy-op gate. A failed or unhealthy guest step
+  skips the host step.
+- **The host health rule** (`HostHealthVerdict`, pinned in `internal/osupdate`): `felhom-agent`, `pveproxy`,
+  `pvedaemon`, `pvestatd` and `pve-cluster` are `active`; the customer guest runs; the guest health rule (§8.1)
+  passes; the tunnel is `running` — an `unknown` tunnel does not fail it, `not_running` does. Same 5-minute wait.
+- **Separate approved sets:** host and guest releases are separate (`os-host-…`, `os-guest-…`), each by the same rule
+  (24 h, 1 night of THAT layer, every ring-0 box). Ring 1 receives `host_release` beside `release`.
+- **Reboot needed:** reported when PID 1 or `lxc-start` maps a replaced file, with the date of the first scanned
+  report that said so. **Never reboots.** The host is scanned on every pass, so a reboot clears it (v0.141.1 — v0.141.0
+  hid `lxc-start` and never cleared, R-846).
+- **Undo:** by hand, `runbooks/os-updates-host-undo.md` (proved). No automatic undo.
+- **Speed (R-845):** one call per layer; both steps with nothing to install 23–32 s; a 108-package host pass 70 s.
+- **Measured live:** demo-felhom ring 0 installed 108 Debian host packages, all Debian origin (checked against apt),
+  healthy; a 605-package host release approved (TEST wait 2 min / 0 nights, logged, reverted to 24 h + 1 night);
+  demo-felhom as ring 1 installed exactly the one version it lacked (604 already current).
+
+### 8.3 Step 4 as BUILT (2026-10-04)  `[FACT]`
+
+- **The fleet view** (`GET /os/fleet`, operator): one line per box — ring, switch, the tunnel, and per layer: the
+  release, the last outcome, the last successful leg, pending, not covered, restart needed, reboot needed since, the
+  wrapper's own seconds.
+- **The four alarms** (`08` §6.3), operator-only, hourly, at most weekly while true: no successful OS leg for
+  **7 days** while the switch is ON (naming the likely reason); reboot needed for **14 days**; ring 0 approved nothing
+  for **7 days** while it has pending fast-lane updates; not-covered fast-lane packages for **14 days**. The four
+  numbers are configuration (`OS_ALARM_*`). *Decided by CC unattended — operator may reverse:* 7 days = a week of
+  missed nights is past any normal hiccup (a box off for a weekend does not alarm); 14 days for reboot and coverage =
+  two weekly golden cycles, both need a person anyway.
+- **The tunnel** (R-841): `running` / `not_running` / `unknown`; `tunnel_down` after two `not_running` reports.
+
 ## 9. Where the rest lives

 - The finding: **R-812** (`backlog/OPEN-ITEMS.md`). The intention: **R-808** (`backlog/ROADMAP.md`).
@@ -0,0 +1,31 @@
+# Hub v0.132.0 red-proofs, 2026-10-04T14:04:22Z
+# First run: 2 of 10 NOT CAUGHT (the parser test never fed empty facts fields; the page test counted the word 'unknown' but not its amber class). Both tests strengthened; the rerun below catches all 10.
+[CAUGHT] unknown, never empty (parser): internal/sysfacts/sysfacts.go mutated; TestParse_AbsentAndPartialAreUnknown ran=True failed=True
+    sysfacts_test.go:38: empty facts fields: {Present:true PVEVersion:x KernelVersion:unknown VMID:0 FactsError: ReadAt: Host:{Debian:unknown KernelRunning:unknown KernelNextBoot:unknown KernelNextBootSou
+[CAUGHT] Docker never auto-approved: internal/osupdates/service.go mutated; TestDocker_NeverAutoApproved ran=True failed=True
+    docker_test.go:33: a Docker set was auto-approved: &{ID:os-docker-20261006-140000 Layer:docker Fingerprint:a4ec54235cdd17a5 ApprovedAt:2026-10-06 14:00:00 +0000 UTC ApprovedBy:auto PackagesJSON:[{"nam
+[CAUGHT] Docker button needs 2 nights: internal/osupdates/service.go mutated; TestDocker_ApproveNeedsTwoHealthyNightsOnEveryRing0Box ran=True failed=True
+    docker_test.go:45: approved after one night: <nil>
+[CAUGHT] a Docker approval nudges no box: internal/osupdates/service.go mutated; TestDocker_ApproveNeedsTwoHealthyNightsOnEveryRing0Box ran=True failed=True
+    docker_test.go:62: a Docker approval must nudge no box (ring 1 takes it by a signed job): [cust1]
+[CAUGHT] crash event once per boot: internal/api/crash.go mutated; TestCheckCrash_EventsOncePerFact ran=True failed=True
+    crash_test.go:45: restart events: op=4 household=4
+[CAUGHT] first sighting is quiet: internal/api/crash.go mutated; TestCheckCrash_EventsOncePerFact ran=True failed=True
+    crash_test.go:39: first sighting flooded 1 events
+[CAUGHT] page: unknown cell: internal/web/system.go mutated; TestSystemPage_FullPartialUnknown ran=True failed=True
+    system_test.go:65: an unknown value must be shown amber with its reason, got 0
+[CAUGHT] page: Docker button gate: internal/web/templates/system.html mutated; TestSystemPage_DockerButtonOnlyWhenReady ran=True failed=True
+    system_test.go:87: button shown after ONE night
+[CAUGHT] buttons need the operator login: internal/web/server.go mutated; TestSystemButtons_NeedTheOperatorLogin ran=True failed=True
+    system_test.go:117: POST /os/ring/full-1 with auth "" = 303 — must be refused
+    system_test.go:117: POST /os/ring/full-1 with auth "Bearer k-full-1" = 303 — must be refused
+    system_test.go:117: POST /os/enabled/full-1 with auth "" = 303 — must be refused
+    system_test.go:117: POST /os/enabled/full-1 with auth "Bearer k-full-1" = 303 — must be refused
+    system_test.go:117: POST /os/approve-now with auth "" = 303 — must be refused
+    system_test.go:117: POST /os/approve-now with auth "Bearer k-full-1" = 303 — must be refused
+    system_test.go:117: POST /os/approve-docker with auth "" = 303 — must be refused
+    system_test.go:117: POST /os/approve-docker with auth "Bearer k-full-1" = 303 — must be refused
+    system_test.go:122: an unauthenticated POST changed a box
+[CAUGHT] Hosts column: internal/web/templates/hosts.html mutated; TestHostsPage_ProxmoxKernelColumn ran=True failed=True
+    system_test.go:153: hosts column missing:
+after restore: ok  	gitea.dooplex.hu/admin/felhom-hub/internal/sysfacts	0.003s | ok  	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.682s | ok  	gitea.dooplex.hu/admin/felhom-hub/internal/api	4.765s | ok  	gitea.dooplex.hu/admin/felhom-hub/internal/web	15.831s rc= 0
@@ -0,0 +1,54 @@
+import subprocess, shutil, sys
+M = [
+ ("unknown, never empty (parser)", "internal/sysfacts/sysfacts.go",
+  "out.Guest.Containerd, out.Guest.LiveRestore = orUnknown(out.Guest.Containerd), orUnknown(out.Guest.LiveRestore)",
+  "out.Guest.Containerd, out.Guest.LiveRestore = out.Guest.Containerd, out.Guest.LiveRestore",
+  "./internal/sysfacts/", "TestParse_AbsentAndPartialAreUnknown"),
+ ("Docker never auto-approved", "internal/osupdates/service.go",
+  "var Layers = []string{LayerGuest, LayerHost}", "var Layers = []string{LayerGuest, LayerHost, LayerDocker}",
+  "./internal/osupdates/", "TestDocker_NeverAutoApproved"),
+ ("Docker button needs 2 nights", "internal/osupdates/service.go",
+  "\tcase need == 0:\n\t\tneed = 2", "\tcase need == 0:\n\t\tneed = 1",
+  "./internal/osupdates/", "TestDocker_ApproveNeedsTwoHealthyNightsOnEveryRing0Box"),
+ ("a Docker approval nudges no box", "internal/osupdates/service.go",
+  "if s.Bump != nil && layer != LayerDocker {", "if s.Bump != nil {",
+  "./internal/osupdates/", "TestDocker_ApproveNeedsTwoHealthyNightsOnEveryRing0Box"),
+ ("crash event once per boot", "internal/api/crash.go",
+  "\t\tif known[b] {\n\t\t\tcontinue\n\t\t}", "\t\tif false {\n\t\t\tcontinue\n\t\t}",
+  "./internal/api/", "TestCheckCrash_EventsOncePerFact"),
+ ("first sighting is quiet", "internal/api/crash.go",
+  "\t\tif first {\n\t\t\treturn\n\t\t}\n\t\td := map", "\t\td := map",
+  "./internal/api/", "TestCheckCrash_EventsOncePerFact"),
+ ("page: unknown cell", "internal/web/system.go",
+  'if s == "" || s == sysfacts.Unknown {', 'if false {',
+  "./internal/web/", "TestSystemPage_FullPartialUnknown"),
+ ("page: Docker button gate", "internal/web/templates/system.html",
+  '(not .Approved) (eq .Waiting "")}}', '(not .Approved)}}',
+  "./internal/web/", "TestSystemPage_DockerButtonOnlyWhenReady"),
+ ("buttons need the operator login", "internal/web/server.go",
+  'if r.URL.Path == "/login" || isPublicBindPath(r.URL.Path) {', 'if r.URL.Path == "/login" || isPublicBindPath(r.URL.Path) || strings.HasPrefix(r.URL.Path, "/os/") {',
+  "./internal/web/", "TestSystemButtons_NeedTheOperatorLogin"),
+ ("Hosts column", "internal/web/templates/hosts.html",
+  "<th>Proxmox / kernel</th>", "<th>Proxmox</th>",
+  "./internal/web/", "TestHostsPage_ProxmoxKernelColumn"),
+]
+ok = True
+for name, f, old, new, pkg, test in M:
+    src = open(f).read()
+    assert src.count(old) == 1, (name, src.count(old))
+    shutil.copy(f, f + ".bak")
+    open(f, "w").write(src.replace(old, new))
+    r = subprocess.run(["go", "test", pkg, "-run", "^" + test + "$", "-v", "-count=1"], capture_output=True, text=True)
+    shutil.move(f + ".bak", f)
+    out = r.stdout + r.stderr
+    ran = ("=== RUN   " + test) in out
+    failed = ("--- FAIL: " + test) in out
+    v = "CAUGHT" if ran and failed else "NOT CAUGHT"
+    ok &= v == "CAUGHT"
+    print(f"[{v}] {name}: {f} mutated; {test} ran={ran} failed={failed}")
+    for l in out.splitlines():
+        if "_test.go:" in l or "build failed" in l or l.startswith("#"):
+            print("    " + l.strip()[:200])
+r = subprocess.run(["go", "test", "./internal/sysfacts/", "./internal/osupdates/", "./internal/api/", "./internal/web/", "-count=1"], capture_output=True, text=True)
+print("after restore:", r.stdout.strip().replace("\n", " | "), "rc=", r.returncode)
+sys.exit(0 if ok and r.returncode == 0 else 1)
@@ -0,0 +1,39 @@
+  --- facts (vmid 9201, 2.1s) ---
+  {
+    "guest": {
+      "containerd": "2.3.3-1~debian.13~trixie",
+      "debian": "13.7",
+      "docker_engine": "29.7.2",
+      "live_restore": "on"
+    },
+    "host": {
+      "crash_guard": {
+        "armed": true,
+        "boot_id": "0defc390-c295-4e9f-aa6e-13d69a1f24c3",
+        "config": {
+          "LIMIT": 3,
+          "PANIC_SECONDS": 10,
+          "REARM_HOURS": 24,
+          "WINDOW_MINUTES": 60
+        },
+        "kernel_panic": 10,
+        "last_boot_at": "2026-10-04T13:39:06Z",
+        "last_boot_unclean": false,
+        "tripped": false,
+        "unclean_boots": [],
+        "unclean_boots_24h": 0,
+        "unclean_boots_in_window": 0,
+        "updated_at": "2026-10-04T14:08:23Z",
+        "version": 1
+      },
+      "debian": "13.7",
+      "held": [],
+      "kernel_next_boot": "7.0.14-20-pve",
+      "kernel_next_boot_source": "saved default",
+      "kernel_panic": 10,
+      "kernel_running": "7.0.14-20-pve",
+      "oops_this_boot": false,
+      "tainted": 4097,
+      "warn_this_boot": false
+    }
+  }
@@ -0,0 +1,143 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Hosts — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.132.0">
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    
+<script>
+
+
+function felhomConfirm(el,question,onYes){
+    if(!el||el.dataset.fcOpen)return;
+    el.dataset.fcOpen='1';
+    var wrap=document.createElement('span');wrap.className='inline-confirm';
+    var q=document.createElement('span');q.className='inline-confirm-q';q.textContent=question;
+    var yes=document.createElement('button');yes.type='button';yes.className='btn btn-sm btn-danger';yes.textContent='Igen';
+    var no=document.createElement('button');no.type='button';no.className='btn btn-sm btn-outline';no.textContent='Mégse';
+    wrap.appendChild(q);wrap.appendChild(yes);wrap.appendChild(no);
+    el.style.display='none';el.parentNode.insertBefore(wrap,el.nextSibling);
+    function close(){wrap.remove();el.style.display='';delete el.dataset.fcOpen;}
+    no.addEventListener('click',close);
+    yes.addEventListener('click',function(){close();onYes();});
+}
+document.addEventListener('click',function(e){
+    var btn=e.target.closest?e.target.closest('[data-confirm]'):null;
+    if(!btn)return;
+    e.preventDefault();
+    felhomConfirm(btn,btn.getAttribute('data-confirm'),function(){
+        var form=btn.closest('form');
+        if(form){if(form.requestSubmit)form.requestSubmit(btn);else form.submit();}
+    });
+});
+</script>
+
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link active">Hosts</a>
+                <a href="/system" class="nav-link">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+
+        <h2 style="margin-bottom: 1rem;">Hosts</h2>
+
+        
+
+        
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <table class="data-table">
+                <thead>
+                    <tr>
+                        <th>Host</th>
+                        <th>Customer</th>
+                        <th>Agent</th>
+                        <th>Proxmox / kernel</th>
+                        <th>Status</th>
+                        <th>Guests</th>
+                        <th>CPU</th>
+                        <th>Mem</th>
+                        <th>Disk</th>
+                        <th>Cloudflared</th>
+                        <th>Worst Storage</th>
+                    </tr>
+                </thead>
+                <tbody>
+                    
+                    <tr onclick="window.location='/hosts/demo-felhom-8363b5'" style="cursor: pointer;">
+                        <td><a href="/hosts/demo-felhom-8363b5">demo-felhom-8363b5</a></td>
+                        <td>Demo Ügyfél</td>
+                        <td><code>0.142.0</code></td>
+                        <td style="font-size: 0.85em;">9.2.2<br><span class="text-muted">7.0.2-6-pve</span></td>
+                        <td><span class="status-badge status-badge-ok">ONLINE</span></td>
+                        <td>1/2</td>
+                        <td>0%</td>
+                        <td>17%</td>
+                        <td>29%</td>
+                        <td>running (connected)</td>
+                        <td>29% <span class="text-muted">local</span></td>
+                    </tr>
+                    
+                    <tr onclick="window.location='/hosts/demo-hp-bb76ea'" style="cursor: pointer;">
+                        <td><a href="/hosts/demo-hp-bb76ea">demo-hp-bb76ea</a></td>
+                        <td>Demo HP</td>
+                        <td><code>0.142.0</code></td>
+                        <td style="font-size: 0.85em;">9.2.2<br><span class="text-muted">7.0.14-20-pve</span></td>
+                        <td><span class="status-badge status-badge-ok">ONLINE</span></td>
+                        <td>1/2</td>
+                        <td>16%</td>
+                        <td>19%</td>
+                        <td>51%</td>
+                        <td>running (connected)</td>
+                        <td>63% <span class="text-muted">local-lvm</span></td>
+                    </tr>
+                    
+                    <tr onclick="window.location='/hosts/drill-r50-0a4f9a'" style="cursor: pointer;">
+                        <td><a href="/hosts/drill-r50-0a4f9a">drill-r50-0a4f9a</a></td>
+                        <td>drill-r50</td>
+                        <td><code>0.129.0</code> <span class="status-badge status-warn" title="held: agent 0.129.0 &lt; MinAgent 0.131.0">floor held</span></td>
+                        <td style="font-size: 0.85em;">unknown<br><span class="text-muted">unknown</span></td>
+                        <td><span class="status-badge status-badge-down">DOWN</span></td>
+                        <td>1/1</td>
+                        <td>20%</td>
+                        <td>20%</td>
+                        <td>10%</td>
+                        <td>inactive</td>
+                        <td>10% <span class="text-muted">local</span></td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+        </section>
+        
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.132.0</span>
+        </footer>
+    </div>
+</body>
+</html>
@@ -0,0 +1,235 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>System — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.132.0">
+    <style>
+        .sys td, .sys th { white-space: nowrap; font-size: 0.82em; vertical-align: top; }
+        .sys .grp { border-left: 2px solid var(--border, #444); }
+        .c-warn { color: var(--warn); font-weight: 600; }
+        .c-bad { color: var(--danger, #e5534b); font-weight: 700; }
+        .sys form { display: inline; }
+        .rel-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(16rem, 1fr)); gap: 0.75rem; }
+    </style>
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    
+<script>
+
+
+function felhomConfirm(el,question,onYes){
+    if(!el||el.dataset.fcOpen)return;
+    el.dataset.fcOpen='1';
+    var wrap=document.createElement('span');wrap.className='inline-confirm';
+    var q=document.createElement('span');q.className='inline-confirm-q';q.textContent=question;
+    var yes=document.createElement('button');yes.type='button';yes.className='btn btn-sm btn-danger';yes.textContent='Igen';
+    var no=document.createElement('button');no.type='button';no.className='btn btn-sm btn-outline';no.textContent='Mégse';
+    wrap.appendChild(q);wrap.appendChild(yes);wrap.appendChild(no);
+    el.style.display='none';el.parentNode.insertBefore(wrap,el.nextSibling);
+    function close(){wrap.remove();el.style.display='';delete el.dataset.fcOpen;}
+    no.addEventListener('click',close);
+    yes.addEventListener('click',function(){close();onYes();});
+}
+document.addEventListener('click',function(e){
+    var btn=e.target.closest?e.target.closest('[data-confirm]'):null;
+    if(!btn)return;
+    e.preventDefault();
+    felhomConfirm(btn,btn.getAttribute('data-confirm'),function(){
+        var form=btn.closest('form');
+        if(form){if(form.requestSubmit)form.requestSubmit(btn);else form.submit();}
+    });
+});
+</script>
+
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link">Hosts</a>
+                <a href="/system" class="nav-link active">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+
+        <h2 style="margin-bottom: 1rem;">System — versions and OS updates</h2>
+
+        
+        
+
+        <section class="card" style="margin-bottom: 1.5rem;">
+            <h3 style="margin-top: 0;">Approved releases</h3>
+            <div class="rel-grid">
+                
+                <div><strong>guest</strong>: <code>os-guest-20261004-123933</code><br>
+                    <span class="text-muted">272 packages · 2026-10-04 12:39 UTC · by auto</span></div>
+                
+                <div><strong>host</strong>: <code>os-host-20261004-124133</code><br>
+                    <span class="text-muted">607 packages · 2026-10-04 12:41 UTC · by auto</span></div>
+                
+            </div>
+            <h3>What ring 0 runs now</h3>
+            <div class="rel-grid">
+                
+                <div><strong>guest</strong>:
+                    0 packages, first seen 2026-10-04 14:14 UTC<br>
+                    <span class="text-muted">healthy for 10m0s of 24h0m0s</span>
+                    
+                </div>
+                
+                <div><strong>host</strong>:
+                    607 packages, first seen 2026-10-04 12:24 UTC<br>
+                    <span class="text-muted">approved as os-host-20261004-124133</span>
+                    
+                    
+                </div>
+                
+                <div><strong>docker</strong>:
+                    <span class="text-muted">—</span><br>
+                    <span class="text-muted">a ring-0 box has not reported a Docker step</span>
+                    
+                </div>
+                
+            </div>
+            <form method="POST" action="/os/approve-now" style="margin-top: 0.8rem;">
+                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                <button type="submit" class="btn btn-sm btn-danger" data-confirm="Approve the guest and host sets ring 0 runs NOW, without the 24 h + 1 night wait? Every ring-1 box installs them at its next night run.">Approve now (guest + host)</button>
+                <span class="text-muted" style="font-size: 0.85em;">An urgent fix only — normally the hub approves after 24 h and one night.</span>
+            </form>
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow-x: auto;">
+            <table class="data-table sys">
+                <thead>
+                    <tr>
+                        <th>Box</th><th>Ring / updates</th><th>Tunnel</th>
+                        <th class="grp">Proxmox</th><th>Kernel (running)</th><th>Kernel (next boot)</th><th>Debian</th><th>Felhom release</th><th>Pending</th><th>Not covered</th><th>Held</th><th>Reboot needed</th><th>kernel.panic</th><th>Oops</th><th>Crash restarts 24 h</th><th>Crash guard</th>
+                        <th class="grp">Guest Debian</th><th>Felhom release</th><th>Pending</th><th>Restart needed</th>
+                        <th class="grp">Docker</th><th>containerd</th><th>live-restore</th><th>Docker release</th>
+                        <th class="grp">Last OS leg</th>
+                    </tr>
+                    <tr class="text-muted"><th></th><th></th><th></th><th class="grp" colspan="13">host</th><th class="grp" colspan="4">guest</th><th class="grp" colspan="4">Docker engine</th><th class="grp"></th></tr>
+                </thead>
+                <tbody>
+                    
+                    <tr>
+                        <td><a href="/hosts/demo-felhom-8363b5">demo-felhom-8363b5</a><br><span class="text-muted">Demo Ügyfél</span>
+                            </td>
+                        <td>
+                            ring 0
+                            <form method="POST" action="/os/ring/demo-felhom-8363b5">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="1"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make demo-felhom-8363b5 a normal (ring 1) box? It then installs only approved releases.">→ normal</button>
+                                
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/demo-felhom-8363b5">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for demo-felhom-8363b5? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td>running</td>
+                        <td class="grp ">9.2.2</td>
+                        <td>7.0.2-6-pve</td><td>7.0.2-6-pve</td><td>13.7</td>
+                        <td>ring0-20261004T141237Z</td><td>80</td><td>0</td>
+                        <td>none</td><td class="c-warn">since 2026-10-04</td><td>10 s</td><td>no</td>
+                        <td>0</td><td>armed</td>
+                        <td class="grp ">13.7</td>
+                        <td>ring0-20261004T141237Z</td><td>0</td><td>10</td>
+                        <td class="grp ">29.8.2</td>
+                        <td>2.3.6-1~debian.13~trixie</td><td>on</td><td>—</td>
+                        <td class="grp " title="last successful leg: 10 min ago">10 min ago · applied · 34 s</td>
+                    </tr>
+                    
+                    <tr>
+                        <td><a href="/hosts/demo-hp-bb76ea">demo-hp-bb76ea</a><br><span class="text-muted">Demo HP</span>
+                            </td>
+                        <td>
+                            ring 0
+                            <form method="POST" action="/os/ring/demo-hp-bb76ea">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="1"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make demo-hp-bb76ea a normal (ring 1) box? It then installs only approved releases.">→ normal</button>
+                                
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/demo-hp-bb76ea">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for demo-hp-bb76ea? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td>running</td>
+                        <td class="grp ">9.2.2</td>
+                        <td>7.0.14-20-pve</td><td>7.0.14-20-pve</td><td>13.7</td>
+                        <td>ring0-20261004T140947Z</td><td>78</td><td>0</td>
+                        <td>none</td><td>no</td><td>10 s</td><td>no</td>
+                        <td>0</td><td>armed</td>
+                        <td class="grp ">13.7</td>
+                        <td>ring0-20261004T140947Z</td><td>0</td><td>1</td>
+                        <td class="grp ">29.8.2</td>
+                        <td>2.3.6-1~debian.13~trixie</td><td>on</td><td>—</td>
+                        <td class="grp " title="last successful leg: 13 min ago">13 min ago · applied · 52 s</td>
+                    </tr>
+                    
+                    <tr>
+                        <td><a href="/hosts/drill-r50-0a4f9a">drill-r50-0a4f9a</a><br><span class="text-muted">drill-r50</span>
+                            <br><span class="c-warn" style="font-weight: normal;">no versions reported (agent older than v0.142.0)</span></td>
+                        <td>
+                            ring 1
+                            <form method="POST" action="/os/ring/drill-r50-0a4f9a">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make drill-r50-0a4f9a a DEMO (ring 0) box? It then installs every new fix first and takes unsigned Docker steps if its root-owned ring-0 mark allows.">→ demo</button>
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/drill-r50-0a4f9a">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for drill-r50-0a4f9a? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td class="c-bad">inactive</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td>
+                        <td>—</td><td>0</td><td>0</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>no</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>no</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn">not installed</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td>—</td><td>0</td><td>0</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>—</td>
+                        <td class="grp " title="last successful leg: never">never · — · 0 s</td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+        </section>
+        <p class="text-muted" style="font-size: 0.85em;">Amber: worth a look. Red: an operator alarm fires (`08` §6.3). "unknown": the box could not read the value — never a guess.</p>
+        
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.132.0</span>
+        </footer>
+    </div>
+</body>
+</html>
+
@@ -0,0 +1,106 @@
+# Agent v0.142.0 red-proofs, 2026-10-04T13:54:18Z
+[CAUGHT] docker pkg in a fast plan refused (R2): configs/felhom-os-apply mutated; DockerLane.test_docker_package_in_a_fast_plan_is_refused ran=True rc=1
+    test_docker_package_in_a_fast_plan_is_refused (__main__.DockerLane.test_docker_package_in_a_fast_plan_is_refused) ... FAIL
+    FAIL: test_docker_package_in_a_fast_plan_is_refused (__main__.DockerLane.test_docker_package_in_a_fast_plan_is_refused)
+    self.assertEqual(rc, 2, rep)
+    AssertionError: 0 != 2 : {'docker_restart_needed': False, 'health_after': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'running'}, 'felhom-controller': {'health': 'healthy', '
+    FAILED (failures=1)
+[CAUGHT] slow lane without authority refused (R3): configs/felhom-os-apply mutated; DockerLane.test_no_authority_is_refused ran=True rc=1
+    test_no_authority_is_refused (__main__.DockerLane.test_no_authority_is_refused) ... FAIL
+    FAIL: test_no_authority_is_refused (__main__.DockerLane.test_no_authority_is_refused)
+    self.assertEqual(rc, 2, rep)
+    AssertionError: 0 != 2 : {'authority': 'ring0', 'docker_engine': '29.7.2', 'docker_restart_needed': False, 'health_after': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'runnin
+    FAILED (failures=1)
+[CAUGHT] bad signature refused (R3): configs/felhom-os-apply mutated; DockerLane.test_bad_signature_is_refused ran=True rc=1
+    test_bad_signature_is_refused (__main__.DockerLane.test_bad_signature_is_refused) ... FAIL
+    FAIL: test_bad_signature_is_refused (__main__.DockerLane.test_bad_signature_is_refused)
+    self.assertEqual(rc, 2, rep)
+    AssertionError: 0 != 2 : {'authority': 'signed', 'docker_engine': '29.7.2', 'docker_restart_needed': False, 'health_after': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'runni
+    FAILED (failures=1)
+[CAUGHT] replay refused (R3): configs/felhom-os-apply mutated; DockerLane.test_replayed_signed_job_is_refused ran=True rc=1
+    test_replayed_signed_job_is_refused (__main__.DockerLane.test_replayed_signed_job_is_refused) ... FAIL
+    FAIL: test_replayed_signed_job_is_refused (__main__.DockerLane.test_replayed_signed_job_is_refused)
+    self.assertEqual(rc, 2, rep)
+    AssertionError: 0 != 2 : {'authority': 'signed', 'docker_engine': '29.7.2', 'docker_restart_needed': False, 'health_after': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'runni
+    FAILED (failures=1)
+[CAUGHT] agent-writable trust file refused (R3): configs/felhom-os-apply mutated; DockerLane.test_agent_writable_trust_file_is_refused ran=True rc=1
+    test_agent_writable_trust_file_is_refused (__main__.DockerLane.test_agent_writable_trust_file_is_refused) ... FAIL
+    FAIL: test_agent_writable_trust_file_is_refused (__main__.DockerLane.test_agent_writable_trust_file_is_refused)
+    self.assertEqual(rc, 2, rep)
+    AssertionError: 0 != 2 : {'authority': 'ring0', 'docker_engine': '29.7.2', 'docker_restart_needed': False, 'health_after': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'runnin
+    FAILED (failures=1)
+[CAUGHT] live-restore off refused (R15): configs/felhom-os-apply mutated; DockerLane.test_live_restore_off_is_refused ran=True rc=1
+    test_live_restore_off_is_refused (__main__.DockerLane.test_live_restore_off_is_refused) ... FAIL
+    FAIL: test_live_restore_off_is_refused (__main__.DockerLane.test_live_restore_off_is_refused)
+    self.assertEqual(rc, 2, rep)
+    AssertionError: 0 != 2 : {'authority': 'signed', 'docker_engine': '29.7.2', 'docker_restart_needed': False, 'health_after': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'runni
+    FAILED (failures=1)
+[CAUGHT] live-restore uses reload, never restart: configs/felhom-os-apply mutated; LiveRestore.test_turns_it_on_with_a_reload_never_a_restart ran=True rc=1
+    test_turns_it_on_with_a_reload_never_a_restart (__main__.LiveRestore.test_turns_it_on_with_a_reload_never_a_restart) ... FAIL
+    FAIL: test_turns_it_on_with_a_reload_never_a_restart (__main__.LiveRestore.test_turns_it_on_with_a_reload_never_a_restart)
+    self.assertEqual(rc, 0, rep)
+    AssertionError: 3 != 0 : {'failed': {'rc': 3, 'reason': '', 'step': 'live-restore'}, 'layer': 'guest', 'live_restore': {'containers_before': 2, 'reload_rc': 0, 'result': 'failed', 'same_ids': True}, '
+    FAILED (failures=1)
+[CAUGHT] guest scanned every pass (R-849): configs/felhom-os-apply mutated; HostLayer.test_every_layer_scans_every_pass ran=True rc=1
+    test_every_layer_scans_every_pass (__main__.HostLayer.test_every_layer_scans_every_pass) ... FAIL
+    FAIL: test_every_layer_scans_every_pass (__main__.HostLayer.test_every_layer_scans_every_pass)
+    self.assertTrue(rep["reboot_scanned"] and rep["reboot_needed"], (layer, rep))
+    AssertionError: False is not true : ('guest', {'health_after': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'running'}, 'felhom-controller': {'health': 'healthy', 'id': 'aaa11
+    FAILED (failures=1)
+[CAUGHT] facts: next-boot by dpkg order: configs/felhom-os-apply mutated; Facts.test_next_entry_wins_and_default_zero_is_the_newest ran=True rc=1
+    test_next_entry_wins_and_default_zero_is_the_newest (__main__.Facts.test_next_entry_wins_and_default_zero_is_the_newest) ... FAIL
+    FAIL: test_next_entry_wins_and_default_zero_is_the_newest (__main__.Facts.test_next_entry_wins_and_default_zero_is_the_newest)
+    self.assertEqual(fa["host"]["kernel_next_boot"], "7.0.14-20-pve", "dpkg order, not string order (7.0.2 < 7.0.14)")
+    AssertionError: '7.0.2-6-pve' != '7.0.14-20-pve'
+    FAILED (failures=1)
+[CAUGHT] crash guard: 3rd crash leaves the box off: configs/felhom-crash-guard mutated; Guard.test_third_crash_in_an_hour_leaves_the_box_off ran=True rc=1
+    FAIL
+    FAIL: test_third_crash_in_an_hour_leaves_the_box_off (__main__.Guard.test_third_crash_in_an_hour_leaves_the_box_off)
+    self.assertTrue(s["tripped"], s)
+    AssertionError: False is not true : {'armed': True, 'boot_id': 'unknown', 'config': {'LIMIT': 3, 'PANIC_SECONDS': 10, 'REARM_HOURS': 24, 'WINDOW_MINUTES': 60}, 'kernel_panic': 10, 'last_boot_at': '202
+    FAILED (failures=1)
+[CAUGHT] crash guard: clean reboots never count: configs/felhom-crash-guard mutated; Guard.test_clean_reboots_never_count ran=True rc=1
+    FAIL
+    FAIL: test_clean_reboots_never_count (__main__.Guard.test_clean_reboots_never_count)
+    self.assertEqual(s["unclean_boots_in_window"], 0)
+    AssertionError: 5 != 0
+    FAILED (failures=1)
+[CAUGHT] crash guard: 24 h re-arm not earlier: configs/felhom-crash-guard mutated; Guard.test_rearms_after_24h_of_normal_running ran=True rc=1
+    FAIL
+    FAIL: test_rearms_after_24h_of_normal_running (__main__.Guard.test_rearms_after_24h_of_normal_running)
+    self.assertTrue(self.e.state()["tripped"], "not before 24 h")
+    AssertionError: False is not true : not before 24 h
+    FAILED (failures=1)
+[CAUGHT] docker health: changed id fails: internal/osupdate/leg.go mutated; TestDockerHealthVerdict ran=True failed=True
+    leg_test.go:442: a changed container id passed — live-restore failed and the apps restarted
+    --- FAIL: TestDockerHealthVerdict (0.00s)
+    FAIL
+    FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.008s
+    FAIL
+[CAUGHT] docker health: changed id -> health_failed: internal/osupdate/leg.go mutated; TestDocker_ChangedIDIsHealthFailed ran=True failed=True
+    leg_test.go:465: docker = {RunID:20261004T040000Z Layer:docker Trigger:night Mode:apply Ring:0 ReleaseID:ring0-20261004T040000Z Outcome:applied Healthy:true HealthReason: VMID:9201 Upgraded:[{Name:doc
+    --- FAIL: TestDocker_ChangedIDIsHealthFailed (0.00s)
+    FAIL
+    FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.011s
+    FAIL
+[CAUGHT] docker health: engine must move: internal/osupdate/leg.go mutated; TestDockerHealthVerdict ran=True failed=True
+    leg_test.go:445: the engine did not move and the step passed
+    --- FAIL: TestDockerHealthVerdict (0.00s)
+    FAIL
+    FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.007s
+    FAIL
+[CAUGHT] ring 1 never steps in the night leg: internal/osupdate/leg.go mutated; TestDocker_Ring1NightLegNeverSteps ran=True failed=True
+    leg_test.go:390: ring 1 took a docker step: guest:inventory,host:inventory,guest:live-restore-on,docker:apply
+    --- FAIL: TestDocker_Ring1NightLegNeverSteps (0.00s)
+    FAIL
+    FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.009s
+    FAIL
+after restore: ok  	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.628s rc=0
+
+[executor] signed envelope reaches the plan — mutation: drop plan["signed"]
+=== RUN   TestDockerStepExecutor_PassesTheSignedEnvelope
+    dockerjob_test.go:31: docker plan = map[lane:slow layer:docker mode:apply packages:[map[name:docker-ce origin:Docker CE version:5:29.8.2-1~debian.13~trixie]] release_id:os-docker-1 select:listed s
+--- FAIL: TestDockerStepExecutor_PassesTheSignedEnvelope (0.00s)
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.009s
+FAIL
@@ -0,0 +1,2 @@
+HTTP/1.1 303 See Other
+Location: /system?flash=approved+os-docker-20261004-142842
@@ -0,0 +1,240 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>System — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.132.0">
+    <style>
+        .sys td, .sys th { white-space: nowrap; font-size: 0.82em; vertical-align: top; }
+        .sys .grp { border-left: 2px solid var(--border, #444); }
+        .c-warn { color: var(--warn); font-weight: 600; }
+        .c-bad { color: var(--danger, #e5534b); font-weight: 700; }
+        .sys form { display: inline; }
+        .rel-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(16rem, 1fr)); gap: 0.75rem; }
+    </style>
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    
+<script>
+
+
+function felhomConfirm(el,question,onYes){
+    if(!el||el.dataset.fcOpen)return;
+    el.dataset.fcOpen='1';
+    var wrap=document.createElement('span');wrap.className='inline-confirm';
+    var q=document.createElement('span');q.className='inline-confirm-q';q.textContent=question;
+    var yes=document.createElement('button');yes.type='button';yes.className='btn btn-sm btn-danger';yes.textContent='Igen';
+    var no=document.createElement('button');no.type='button';no.className='btn btn-sm btn-outline';no.textContent='Mégse';
+    wrap.appendChild(q);wrap.appendChild(yes);wrap.appendChild(no);
+    el.style.display='none';el.parentNode.insertBefore(wrap,el.nextSibling);
+    function close(){wrap.remove();el.style.display='';delete el.dataset.fcOpen;}
+    no.addEventListener('click',close);
+    yes.addEventListener('click',function(){close();onYes();});
+}
+document.addEventListener('click',function(e){
+    var btn=e.target.closest?e.target.closest('[data-confirm]'):null;
+    if(!btn)return;
+    e.preventDefault();
+    felhomConfirm(btn,btn.getAttribute('data-confirm'),function(){
+        var form=btn.closest('form');
+        if(form){if(form.requestSubmit)form.requestSubmit(btn);else form.submit();}
+    });
+});
+</script>
+
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link">Hosts</a>
+                <a href="/system" class="nav-link active">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+
+        <h2 style="margin-bottom: 1rem;">System — versions and OS updates</h2>
+
+        
+        
+
+        <section class="card" style="margin-bottom: 1.5rem;">
+            <h3 style="margin-top: 0;">Approved releases</h3>
+            <div class="rel-grid">
+                
+                <div><strong>guest</strong>: <code>os-guest-20261004-123933</code><br>
+                    <span class="text-muted">272 packages · 2026-10-04 12:39 UTC · by auto</span></div>
+                
+                <div><strong>host</strong>: <code>os-host-20261004-124133</code><br>
+                    <span class="text-muted">607 packages · 2026-10-04 12:41 UTC · by auto</span></div>
+                
+            </div>
+            <h3>What ring 0 runs now</h3>
+            <div class="rel-grid">
+                
+                <div><strong>guest</strong>:
+                    272 packages, first seen 2026-10-04 11:07 UTC<br>
+                    <span class="text-muted">approved as os-guest-20261004-123933</span>
+                    
+                    
+                </div>
+                
+                <div><strong>host</strong>:
+                    607 packages, first seen 2026-10-04 12:24 UTC<br>
+                    <span class="text-muted">approved as os-host-20261004-124133</span>
+                    
+                    
+                </div>
+                
+                <div><strong>docker</strong>:
+                    6 packages, first seen 2026-10-04 14:28 UTC<br>
+                    
+                    
+                    <form method="POST" action="/os/approve-docker" style="margin-top: 0.3rem;">
+                        <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                        <button type="submit" class="btn btn-sm" data-confirm="Approve this Docker engine set? Ring-1 boxes take it only through a signed operator job.">Approve Docker set</button>
+                    </form>
+                </div>
+                
+            </div>
+            <form method="POST" action="/os/approve-now" style="margin-top: 0.8rem;">
+                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                <button type="submit" class="btn btn-sm btn-danger" data-confirm="Approve the guest and host sets ring 0 runs NOW, without the 24 h + 1 night wait? Every ring-1 box installs them at its next night run.">Approve now (guest + host)</button>
+                <span class="text-muted" style="font-size: 0.85em;">An urgent fix only — normally the hub approves after 24 h and one night.</span>
+            </form>
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow-x: auto;">
+            <table class="data-table sys">
+                <thead>
+                    <tr>
+                        <th>Box</th><th>Ring / updates</th><th>Tunnel</th>
+                        <th class="grp">Proxmox</th><th>Kernel (running)</th><th>Kernel (next boot)</th><th>Debian</th><th>Felhom release</th><th>Pending</th><th>Not covered</th><th>Held</th><th>Reboot needed</th><th>kernel.panic</th><th>Oops</th><th>Crash restarts 24 h</th><th>Crash guard</th>
+                        <th class="grp">Guest Debian</th><th>Felhom release</th><th>Pending</th><th>Restart needed</th>
+                        <th class="grp">Docker</th><th>containerd</th><th>live-restore</th><th>Docker release</th>
+                        <th class="grp">Last OS leg</th>
+                    </tr>
+                    <tr class="text-muted"><th></th><th></th><th></th><th class="grp" colspan="13">host</th><th class="grp" colspan="4">guest</th><th class="grp" colspan="4">Docker engine</th><th class="grp"></th></tr>
+                </thead>
+                <tbody>
+                    
+                    <tr>
+                        <td><a href="/hosts/demo-felhom-8363b5">demo-felhom-8363b5</a><br><span class="text-muted">Demo Ügyfél</span>
+                            </td>
+                        <td>
+                            ring 0
+                            <form method="POST" action="/os/ring/demo-felhom-8363b5">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="1"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make demo-felhom-8363b5 a normal (ring 1) box? It then installs only approved releases.">→ normal</button>
+                                
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/demo-felhom-8363b5">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for demo-felhom-8363b5? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td>running</td>
+                        <td class="grp ">9.2.2</td>
+                        <td>7.0.2-6-pve</td><td>7.0.2-6-pve</td><td>13.7</td>
+                        <td>ring0-20261004T142539Z</td><td>80</td><td>0</td>
+                        <td>none</td><td class="c-warn">since 2026-10-04</td><td>10 s</td><td>no</td>
+                        <td>0</td><td>armed</td>
+                        <td class="grp ">13.7</td>
+                        <td>ring0-20261004T142539Z</td><td>0</td><td>10</td>
+                        <td class="grp ">29.8.2</td>
+                        <td>2.3.6-1~debian.13~trixie</td><td>on</td><td>ring0-20261004T142539Z</td>
+                        <td class="grp " title="last successful leg: 2 min ago">2 min ago · nothing · 13 s</td>
+                    </tr>
+                    
+                    <tr>
+                        <td><a href="/hosts/demo-hp-bb76ea">demo-hp-bb76ea</a><br><span class="text-muted">Demo HP</span>
+                            </td>
+                        <td>
+                            ring 0
+                            <form method="POST" action="/os/ring/demo-hp-bb76ea">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="1"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make demo-hp-bb76ea a normal (ring 1) box? It then installs only approved releases.">→ normal</button>
+                                
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/demo-hp-bb76ea">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for demo-hp-bb76ea? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td>running</td>
+                        <td class="grp ">9.2.2</td>
+                        <td>7.0.14-20-pve</td><td>7.0.14-20-pve</td><td>13.7</td>
+                        <td>ring0-20261004T142618Z</td><td>78</td><td>0</td>
+                        <td>none</td><td>no</td><td>10 s</td><td>no</td>
+                        <td>0</td><td>armed</td>
+                        <td class="grp ">13.7</td>
+                        <td>ring0-20261004T142618Z</td><td>0</td><td>1</td>
+                        <td class="grp ">29.8.2</td>
+                        <td>2.3.6-1~debian.13~trixie</td><td>on</td><td>ring0-20261004T142618Z</td>
+                        <td class="grp " title="last successful leg: 2 min ago">1 min ago · nothing · 17 s</td>
+                    </tr>
+                    
+                    <tr>
+                        <td><a href="/hosts/drill-r50-0a4f9a">drill-r50-0a4f9a</a><br><span class="text-muted">drill-r50</span>
+                            <br><span class="c-warn" style="font-weight: normal;">no versions reported (agent older than v0.142.0)</span></td>
+                        <td>
+                            ring 1
+                            <form method="POST" action="/os/ring/drill-r50-0a4f9a">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make drill-r50-0a4f9a a DEMO (ring 0) box? It then installs every new fix first and takes unsigned Docker steps if its root-owned ring-0 mark allows.">→ demo</button>
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/drill-r50-0a4f9a">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for drill-r50-0a4f9a? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td class="c-bad">inactive</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td>
+                        <td>—</td><td>0</td><td>0</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>no</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>no</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn">not installed</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td>—</td><td>0</td><td>0</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>—</td>
+                        <td class="grp " title="last successful leg: never">never · — · 0 s</td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+        </section>
+        <p class="text-muted" style="font-size: 0.85em;">Amber: worth a look. Red: an operator alarm fires (`08` §6.3). "unknown": the box could not read the value — never a guess.</p>
+        
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.132.0</span>
+        </footer>
+    </div>
+</body>
+</html>
+
@@ -0,0 +1,80 @@
+ pct exec 9202 -- docker ps --no-trunc --format '{{.ID}} {{.Names}} {{.Status}}'
+ sort
+ cat /root/9202-ids-before.txt
+ cut -c1-12,65-
+1b8dc9032f36 paperless-webserver Up 27 minutes (healthy)
+2d6019eb0042 paperless-redis Up 27 minutes (healthy)
+3f8f9651ee32 felhom-controller Up 27 minutes (healthy)
+468aa784345b paperless-postgres Up 27 minutes (healthy)
+7504fa621b39 filebrowser Up 27 minutes (healthy)
+8d8778d4dcf8 traefik Up 27 minutes
+ pct exec 9202 -- docker info --format 'live={{.LiveRestoreEnabled}}'
+live=false
+ pct exec 9202 -- cat /etc/docker/daemon.json
+{
+  "features": { "containerd-snapshotter": false },
+  "log-driver": "json-file",
+  "log-opts": { "max-size": "10m", "max-file": "3" }
+}
++ date +%s.%N
+ S=1791122916.729550605
+ sudo -u felhom-agent /usr/local/bin/felhom-agent-0.142.0-rc1 --config /etc/felhom-agent/agent.json --selftest=live-restore -vmid 9202
+ grep -E 'LIVE-RESTORE|live-restore|error|refused'
+selftest=live-restore: refused: {"code": "R10", "reason": "vmid 9202 does not bind /mnt/felhom-drives \u2014 it is not this box's customer guest"}
++ bc
+++ date +%s.%N
++ echo 1791122916.827586880-1791122916.729550605
+ echo WALL=.098036275
+WALL=.098036275
+ pct exec 9202 -- docker ps --no-trunc --format '{{.ID}} {{.Names}} {{.Status}}'
+ sort
+ cut '-d ' -f1,2 /root/9202-ids-before.txt
+ cut '-d ' -f1,2 /root/9202-ids-after.txt
+ diff /tmp/a /tmp/b
++ wc -l
+IDS IDENTICAL (6 containers)
+ echo 'IDS IDENTICAL (6 containers)'
+ pct exec 9202 -- docker info --format 'live={{.LiveRestoreEnabled}}'
+live=false
+ pct exec 9202 -- cat /etc/docker/daemon.json
+{
+  "features": { "containerd-snapshotter": false },
+  "log-driver": "json-file",
+  "log-opts": { "max-size": "10m", "max-file": "3" }
+}
+ pct exec 9202 -- systemctl show docker -p ActiveEnterTimestamp -p NRestarts
+NRestarts=0
+ActiveEnterTimestamp=Sun 2026-10-04 13:40:57 UTC
+ pct exec 9202 -- cp -p /etc/docker/daemon.json /root/daemon.json.bak-2026-10-04
+ pct exec 9202 -- python3 -c '
+import json; p="/etc/docker/daemon.json"; d=json.load(open(p)); d["live-restore"]=True; open(p,"w").write(json.dumps(d,indent=2,sort_keys=True)+"\n")'
++ date +%s.%N
+ S=1791122938.796302354
+ pct exec 9202 -- systemctl reload docker
++ bc
+++ date +%s.%N
++ echo 1791122939.730342275-1791122938.796302354
+reload_rc=0 took=.934039921
+ echo 'reload_rc=0 took=.934039921'
+ sleep 3
+ pct exec 9202 -- docker info --format 'live={{.LiveRestoreEnabled}}'
+live=true
+ pct exec 9202 -- docker ps --no-trunc --format '{{.ID}} {{.Names}} {{.Status}}'
+ sort
+ cut '-d ' -f1,2 /root/9202-ids-before.txt
+ cut '-d ' -f1,2 /root/9202-ids-after.txt
+ diff /tmp/a /tmp/b
++ wc -l
+IDS IDENTICAL (6 containers)
+ echo 'IDS IDENTICAL (6 containers)'
+ cut -c1-12,65- /root/9202-ids-after.txt
+1b8dc9032f36 paperless-webserver Up 28 minutes (healthy)
+2d6019eb0042 paperless-redis Up 28 minutes (healthy)
+3f8f9651ee32 felhom-controller Up 28 minutes (healthy)
+468aa784345b paperless-postgres Up 28 minutes (healthy)
+7504fa621b39 filebrowser Up 28 minutes (healthy)
+8d8778d4dcf8 traefik Up 28 minutes
+ pct exec 9202 -- systemctl show docker -p ActiveEnterTimestamp -p NRestarts -p ExecMainPID
+NRestarts=0
+ExecMainPID=227
+ActiveEnterTimestamp=Sun 2026-10-04 13:40:57 UTC
@@ -0,0 +1,7 @@
+before: 24 containers, live=false
+os-apply: LIVE-RESTORE reload_rc=0 state=true containers=24 same-ids=yes
+live-restore: on (see the wrapper's LIVE-RESTORE line above for the container ids)
+WALL=7.532254689
+IDS IDENTICAL (24 containers)
+after: live=true
+NRestarts=0 ActiveEnterTimestamp=Sun 2026-10-04 13:41:02 UTC 
@@ -0,0 +1,202 @@
+=== felhom-agent 0.142.0-rc1 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T16:09:47.878+02:00 level=INFO msg="osupdate: START" run=20261004T140947Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T140947Z
+time=2026-10-04T16:10:04.518+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T140947Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T16:10:04.518+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:10:04.518+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:10:04.518+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:10:04.519+02:00 level=INFO msg="osupdate: DONE" run=20261004T140947Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=0 reboot_needed=false wrapper_seconds=16.6
+time=2026-10-04T16:10:04.530+02:00 level=INFO msg="osupdate: START" run=20261004T140947Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T140947Z
+time=2026-10-04T16:10:19.645+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T140947Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T16:10:19.645+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:10:19.645+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:10:19.645+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:10:20.691+02:00 level=INFO msg="osupdate: DONE" run=20261004T140947Z layer=host vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=78 not_covered=78 restart_needed=0 reboot_needed=false wrapper_seconds=15
+time=2026-10-04T16:10:22.649+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: LIVE-RESTORE already on"
+time=2026-10-04T16:10:22.649+02:00 level=INFO msg="osupdate: live-restore" vmid=9201 result="{\"result\": \"already on\"}"
+time=2026-10-04T16:10:22.649+02:00 level=INFO msg="osupdate: START" run=20261004T140947Z layer=docker vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T140947Z
+time=2026-10-04T16:11:14.270+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T140947Z layer=docker:9201 lane=slow mode=apply select=pending-docker packages=0 authority=ring0"
+time=2026-10-04T16:11:14.270+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:11:14.270+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=6 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:11:14.271+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=24.4 upgraded=6 restart-needed=containerd-shim reboot-needed=no"
+time=2026-10-04T16:11:50.070+02:00 level=INFO msg="osupdate: DONE" run=20261004T140947Z layer=docker vmid=9201 ring=0 trigger=debug outcome=applied healthy=true reason="" upgraded=6 pending=0 not_covered=0 restart_needed=1 reboot_needed=false wrapper_seconds=51.5
+  --- os-update report (guest) ---
+  {
+    "authority": "",
+    "docker_engine": "",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T140947Z",
+    "restart_needed": [],
+    "ring": 0,
+    "run_id": "20261004T140947Z",
+    "upgraded": [],
+    "wrapper_seconds": 16.6
+  }
+  --- os-update report (host) ---
+  {
+    "authority": "",
+    "docker_engine": "",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "proxmox-secure-boot-support",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "amd64-microcode",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "pve-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed"
+    ],
+    "outcome": "nothing",
+    "pending": 78,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T140947Z",
+    "restart_needed": [],
+    "ring": 0,
+    "run_id": "20261004T140947Z",
+    "upgraded": [],
+    "wrapper_seconds": 15
+  }
+  --- os-update report (docker) ---
+  {
+    "authority": "ring0",
+    "docker_engine": "29.8.2",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": null,
+    "outcome": "applied",
+    "pending": 0,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T140947Z",
+    "restart_needed": [
+      "containerd-shim"
+    ],
+    "ring": 0,
+    "run_id": "20261004T140947Z",
+    "upgraded": [
+      {
+        "name": "docker-ce-cli",
+        "version": "5:29.8.2-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "containerd.io",
+        "version": "2.3.6-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "docker-ce",
+        "version": "5:29.8.2-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "docker-buildx-plugin",
+        "version": "0.37.1-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "docker-ce-rootless-extras",
+        "version": "5:29.8.2-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "docker-compose-plugin",
+        "version": "5.6.0-1~debian.13~trixie",
+        "origin": ""
+      }
+    ],
+    "wrapper_seconds": 51.5
+  }
+  pass took 2m2.2s
+WALL=122.327970106
+IDS IDENTICAL (24 containers)
+engine=29.8.2
+live=true
@@ -0,0 +1,259 @@
+before: 5 containers, engine=29.7.1, live=false
+=== felhom-agent 0.142.0-rc1 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T16:12:37.832+02:00 level=INFO msg="osupdate: START" run=20261004T141237Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T141237Z
+time=2026-10-04T16:12:50.352+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T141237Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T16:12:50.352+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:12:50.352+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:12:50.352+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:12:50.352+02:00 level=INFO msg="osupdate: DONE" run=20261004T141237Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=10 reboot_needed=true wrapper_seconds=12.5
+time=2026-10-04T16:12:50.364+02:00 level=INFO msg="osupdate: START" run=20261004T141237Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T141237Z
+time=2026-10-04T16:13:00.773+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T141237Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T16:13:00.773+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:13:00.773+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:13:00.773+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:13:01.549+02:00 level=INFO msg="osupdate: DONE" run=20261004T141237Z layer=host vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=80 not_covered=80 restart_needed=34 reboot_needed=true wrapper_seconds=10.4
+time=2026-10-04T16:13:08.002+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: LIVE-RESTORE reload_rc=0 state=true containers=5 same-ids=yes"
+time=2026-10-04T16:13:08.002+02:00 level=INFO msg="osupdate: live-restore" vmid=9201 result="{\"containers_before\": 5, \"reload_rc\": 0, \"result\": \"on\", \"same_ids\": true}"
+time=2026-10-04T16:13:08.002+02:00 level=INFO msg="osupdate: START" run=20261004T141237Z layer=docker vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T141237Z
+time=2026-10-04T16:13:42.395+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T141237Z layer=docker:9201 lane=slow mode=apply select=pending-docker packages=0 authority=ring0"
+time=2026-10-04T16:13:42.395+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:13:42.395+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=6 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:13:42.395+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=13.7 upgraded=6 restart-needed=agetty,containerd-shim,cron,dbus-daemon,dhclient,sshd,systemd,systemd-journal,systemd-logind,systemd-network reboot-needed=yes"
+time=2026-10-04T16:14:16.982+02:00 level=INFO msg="osupdate: DONE" run=20261004T141237Z layer=docker vmid=9201 ring=0 trigger=debug outcome=applied healthy=true reason="" upgraded=6 pending=0 not_covered=0 restart_needed=10 reboot_needed=true wrapper_seconds=34.3
+  --- os-update report (guest) ---
+  {
+    "authority": "",
+    "docker_engine": "",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "reboot_needed": true,
+    "refused": null,
+    "release_id": "ring0-20261004T141237Z",
+    "restart_needed": [
+      "agetty",
+      "containerd",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd",
+      "systemd-journal",
+      "systemd-logind",
+      "systemd-network"
+    ],
+    "ring": 0,
+    "run_id": "20261004T141237Z",
+    "upgraded": [],
+    "wrapper_seconds": 12.5
+  }
+  --- os-update report (host) ---
+  {
+    "authority": "",
+    "docker_engine": "",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-first-boot",
+      "pve-firmware",
+      "proxmox-kernel-7.0.14-20-pve-signed",
+      "proxmox-kernel-7.0",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed",
+      "tailscale"
+    ],
+    "outcome": "nothing",
+    "pending": 80,
+    "reboot_needed": true,
+    "refused": null,
+    "release_id": "ring0-20261004T141237Z",
+    "restart_needed": [
+      "agetty",
+      "blkmapd",
+      "chronyd",
+      "cron",
+      "dbus-daemon",
+      "dmeventd",
+      "ksmtuned",
+      "lxc-monitord",
+      "lxc-start",
+      "lxcfs",
+      "pmxcfs",
+      "proxmox-firewal",
+      "pve-firewall",
+      "pve-ha-crm",
+      "pve-ha-lrm",
+      "pve-lxc-syscall",
+      "pvedaemon",
+      "pvedaemon worke",
+      "pvefw-logger",
+      "pveproxy",
+      "pveproxy worker",
+      "pvescheduler",
+      "pvestatd",
+      "qmeventd",
+      "rpcbind",
+      "rrdcached",
+      "smartd",
+      "spiceproxy",
+      "spiceproxy work",
+      "sshd",
+      "systemd-logind",
+      "systemd-udevd",
+      "watchdog-mux",
+      "zed"
+    ],
+    "ring": 0,
+    "run_id": "20261004T141237Z",
+    "upgraded": [],
+    "wrapper_seconds": 10.4
+  }
+  --- os-update report (docker) ---
+  {
+    "authority": "ring0",
+    "docker_engine": "29.8.2",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": null,
+    "outcome": "applied",
+    "pending": 0,
+    "reboot_needed": true,
+    "refused": null,
+    "release_id": "ring0-20261004T141237Z",
+    "restart_needed": [
+      "agetty",
+      "containerd-shim",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd",
+      "systemd-journal",
+      "systemd-logind",
+      "systemd-network"
+    ],
+    "ring": 0,
+    "run_id": "20261004T141237Z",
+    "upgraded": [
+      {
+        "name": "docker-ce-cli",
+        "version": "5:29.8.2-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "containerd.io",
+        "version": "2.3.6-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "docker-ce",
+        "version": "5:29.8.2-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "docker-buildx-plugin",
+        "version": "0.37.1-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "docker-ce-rootless-extras",
+        "version": "5:29.8.2-1~debian.13~trixie",
+        "origin": ""
+      },
+      {
+        "name": "docker-compose-plugin",
+        "version": "5.6.0-1~debian.13~trixie",
+        "origin": ""
+      }
+    ],
+    "wrapper_seconds": 34.3
+  }
+  pass took 1m39.2s
+WALL=99.343682135
+IDS IDENTICAL (5 containers)
+after: engine=29.8.2 live=true
@@ -0,0 +1,163 @@
+=== felhom-agent 0.142.0 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T16:26:18.795+02:00 level=INFO msg="osupdate: START" run=20261004T142618Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T142618Z
+time=2026-10-04T16:26:34.572+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T142618Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T16:26:34.572+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:26:34.572+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:26:34.572+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:26:34.574+02:00 level=INFO msg="osupdate: DONE" run=20261004T142618Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=0 not_covered=0 restart_needed=1 reboot_needed=false wrapper_seconds=15.7
+time=2026-10-04T16:26:34.583+02:00 level=INFO msg="osupdate: START" run=20261004T142618Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T142618Z
+time=2026-10-04T16:26:49.369+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T142618Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T16:26:49.369+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:26:49.369+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:26:49.369+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:26:50.399+02:00 level=INFO msg="osupdate: DONE" run=20261004T142618Z layer=host vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=78 not_covered=78 restart_needed=0 reboot_needed=false wrapper_seconds=14.7
+time=2026-10-04T16:26:52.343+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: LIVE-RESTORE already on"
+time=2026-10-04T16:26:52.343+02:00 level=INFO msg="osupdate: live-restore" vmid=9201 result="{\"result\": \"already on\"}"
+time=2026-10-04T16:26:52.343+02:00 level=INFO msg="osupdate: START" run=20261004T142618Z layer=docker vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T142618Z
+time=2026-10-04T16:27:09.714+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T142618Z layer=docker:9201 lane=slow mode=apply select=pending-docker packages=0 authority=ring0"
+time=2026-10-04T16:27:09.715+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:27:09.715+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:27:09.715+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:27:09.716+02:00 level=INFO msg="osupdate: DONE" run=20261004T142618Z layer=docker vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=0 not_covered=0 restart_needed=1 reboot_needed=false wrapper_seconds=17.3
+  --- os-update report (guest) ---
+  {
+    "authority": "",
+    "docker_engine": "",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": null,
+    "outcome": "nothing",
+    "pending": 0,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T142618Z",
+    "restart_needed": [
+      "containerd-shim"
+    ],
+    "ring": 0,
+    "run_id": "20261004T142618Z",
+    "upgraded": [],
+    "wrapper_seconds": 15.7
+  }
+  --- os-update report (host) ---
+  {
+    "authority": "",
+    "docker_engine": "",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "proxmox-secure-boot-support",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "amd64-microcode",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "pve-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed"
+    ],
+    "outcome": "nothing",
+    "pending": 78,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T142618Z",
+    "restart_needed": [],
+    "ring": 0,
+    "run_id": "20261004T142618Z",
+    "upgraded": [],
+    "wrapper_seconds": 14.7
+  }
+  --- os-update report (docker) ---
+  {
+    "authority": "ring0",
+    "docker_engine": "29.8.2",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": null,
+    "outcome": "nothing",
+    "pending": 0,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T142618Z",
+    "restart_needed": [
+      "containerd-shim"
+    ],
+    "ring": 0,
+    "run_id": "20261004T142618Z",
+    "upgraded": [],
+    "wrapper_seconds": 17.3
+  }
+  pass took 50.9s
+WALL=51.005070963
@@ -0,0 +1,218 @@
+=== felhom-agent 0.142.0 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T16:25:39.774+02:00 level=INFO msg="osupdate: START" run=20261004T142539Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T142539Z
+time=2026-10-04T16:25:52.120+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T142539Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T16:25:52.120+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:25:52.120+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:25:52.120+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:25:52.120+02:00 level=INFO msg="osupdate: DONE" run=20261004T142539Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=0 not_covered=0 restart_needed=10 reboot_needed=true wrapper_seconds=12.3
+time=2026-10-04T16:25:52.131+02:00 level=INFO msg="osupdate: START" run=20261004T142539Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T142539Z
+time=2026-10-04T16:26:02.540+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T142539Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T16:26:02.540+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:26:02.540+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:26:02.540+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:26:03.376+02:00 level=INFO msg="osupdate: DONE" run=20261004T142539Z layer=host vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=80 not_covered=80 restart_needed=34 reboot_needed=true wrapper_seconds=10.4
+time=2026-10-04T16:26:04.999+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: LIVE-RESTORE already on"
+time=2026-10-04T16:26:04.999+02:00 level=INFO msg="osupdate: live-restore" vmid=9201 result="{\"result\": \"already on\"}"
+time=2026-10-04T16:26:04.999+02:00 level=INFO msg="osupdate: START" run=20261004T142539Z layer=docker vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T142539Z
+time=2026-10-04T16:26:18.174+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T142539Z layer=docker:9201 lane=slow mode=apply select=pending-docker packages=0 authority=ring0"
+time=2026-10-04T16:26:18.174+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T16:26:18.174+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:26:18.174+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:26:18.174+02:00 level=INFO msg="osupdate: DONE" run=20261004T142539Z layer=docker vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=0 not_covered=0 restart_needed=10 reboot_needed=true wrapper_seconds=13.1
+  --- os-update report (guest) ---
+  {
+    "authority": "",
+    "docker_engine": "",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": null,
+    "outcome": "nothing",
+    "pending": 0,
+    "reboot_needed": true,
+    "refused": null,
+    "release_id": "ring0-20261004T142539Z",
+    "restart_needed": [
+      "agetty",
+      "containerd-shim",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd",
+      "systemd-journal",
+      "systemd-logind",
+      "systemd-network"
+    ],
+    "ring": 0,
+    "run_id": "20261004T142539Z",
+    "upgraded": [],
+    "wrapper_seconds": 12.3
+  }
+  --- os-update report (host) ---
+  {
+    "authority": "",
+    "docker_engine": "",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-first-boot",
+      "pve-firmware",
+      "proxmox-kernel-7.0.14-20-pve-signed",
+      "proxmox-kernel-7.0",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed",
+      "tailscale"
+    ],
+    "outcome": "nothing",
+    "pending": 80,
+    "reboot_needed": true,
+    "refused": null,
+    "release_id": "ring0-20261004T142539Z",
+    "restart_needed": [
+      "agetty",
+      "blkmapd",
+      "chronyd",
+      "cron",
+      "dbus-daemon",
+      "dmeventd",
+      "ksmtuned",
+      "lxc-monitord",
+      "lxc-start",
+      "lxcfs",
+      "pmxcfs",
+      "proxmox-firewal",
+      "pve-firewall",
+      "pve-ha-crm",
+      "pve-ha-lrm",
+      "pve-lxc-syscall",
+      "pvedaemon",
+      "pvedaemon worke",
+      "pvefw-logger",
+      "pveproxy",
+      "pveproxy worker",
+      "pvescheduler",
+      "pvestatd",
+      "qmeventd",
+      "rpcbind",
+      "rrdcached",
+      "smartd",
+      "spiceproxy",
+      "spiceproxy work",
+      "sshd",
+      "systemd-logind",
+      "systemd-udevd",
+      "watchdog-mux",
+      "zed"
+    ],
+    "ring": 0,
+    "run_id": "20261004T142539Z",
+    "upgraded": [],
+    "wrapper_seconds": 10.4
+  }
+  --- os-update report (docker) ---
+  {
+    "authority": "ring0",
+    "docker_engine": "29.8.2",
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": null,
+    "outcome": "nothing",
+    "pending": 0,
+    "reboot_needed": true,
+    "refused": null,
+    "release_id": "ring0-20261004T142539Z",
+    "restart_needed": [
+      "agetty",
+      "containerd-shim",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd",
+      "systemd-journal",
+      "systemd-logind",
+      "systemd-network"
+    ],
+    "ring": 0,
+    "run_id": "20261004T142539Z",
+    "upgraded": [],
+    "wrapper_seconds": 13.1
+  }
+  pass took 38.4s
+WALL=38.491768907
@@ -0,0 +1 @@
+nonce=cb4a902da3007b8f4c3a440d3f2d9a60
@@ -0,0 +1 @@
+{"release_id":"os-docker-20261004-142842","undo":false,"packages":[{"name":"containerd.io","version":"2.3.6-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-buildx-plugin","version":"0.37.1-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce","version":"5:29.8.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce-cli","version":"5:29.8.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce-rootless-extras","version":"5:29.8.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-compose-plugin","version":"5.6.0-1~debian.13~trixie","origin":"Docker CE"}]}
@@ -0,0 +1,2 @@
+HTTP/1.1 303 See Other
+Location: /system?flash=done
@@ -0,0 +1,11 @@
+=== felhom-agent 0.142.0 selftest=os-update vmid=9201 ring=1 enabled=true guest-release=true host-release=true appliance=true ===
+time=2026-10-04T16:42:05.041+02:00 level=INFO msg="osupdate: DONE" run=20261004T144154Z layer=guest vmid=9201 ring=1 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=0 not_covered=0 restart_needed=10 reboot_needed=true wrapper_seconds=10.6
+time=2026-10-04T16:42:15.090+02:00 level=INFO msg="osupdate: DONE" run=20261004T144154Z layer=host vmid=9201 ring=1 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=80 not_covered=80 restart_needed=34 reboot_needed=true wrapper_seconds=9.2
+time=2026-10-04T16:42:15.108+02:00 level=INFO msg="osupdate: docker step skipped — ring 1 takes an engine set only inside a signed operator job (`11` §5.8)" run=20261004T144154Z vmid=9201 trigger=debug ring=1 enabled=true
+  docker step: skipped (see the log line above)
+containerd.io	2.3.6-1~debian.13~trixie
+docker-buildx-plugin	0.37.1-1~debian.13~trixie
+docker-ce	5:29.8.2-1~debian.13~trixie
+docker-ce-cli	5:29.8.2-1~debian.13~trixie
+docker-ce-rootless-extras	5:29.8.2-1~debian.13~trixie
+docker-compose-plugin	5.6.0-1~debian.13~trixie
@@ -0,0 +1,2 @@
+signed: op=os_docker_step host=demo-felhom-8363b5 guest="" key_id=felhom-op-1 nonce=cb4a902da3007b8f4c3a440d3f2d9a60 expires=2026-10-04T15:27:25Z
+uploaded signed op to the hub jobs queue
@@ -0,0 +1,7 @@
+time=2026-10-04T16:52:54.082+02:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=b7956e7339bfda31 op=os_docker_step key_id=felhom-op-1 nonce=cb4a902da3007b8f4c3a440d3f2d9a60
+time=2026-10-04T16:52:55.697+02:00 level=INFO msg="osupdate: START" run=20261004T145254Z layer=docker vmid=9201 ring=1 trigger=signed enabled=true release=os-docker-20261004-142842
+time=2026-10-04T16:53:07.878+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=os-docker-20261004-142842 layer=docker:9201 lane=slow mode=apply select=listed packages=6 authority=signed"
+time=2026-10-04T16:53:07.878+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=6 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:53:07.878+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T16:53:07.879+02:00 level=INFO msg="osupdate: DONE" run=20261004T145254Z layer=docker vmid=9201 ring=1 trigger=signed outcome=nothing healthy=true reason="" upgraded=0 pending=0 not_covered=0 restart_needed=10 reboot_needed=true wrapper_seconds=
+time=2026-10-04T16:53:07.884+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=b7956e7339bfda31 op=os_docker_step
@@ -0,0 +1,2 @@
+signed: op=os_docker_step host=demo-felhom-8363b5 guest="" key_id=felhom-op-1 nonce=cb4a902da3007b8f4c3a440d3f2d9a60 expires=2026-10-04T15:38:36Z
+uploaded signed op to the hub jobs queue
@@ -0,0 +1,3 @@
+time=2026-10-04T16:52:54.082+02:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=b7956e7339bfda31 op=os_docker_step key_id=felhom-op-1 nonce=cb4a902da3007b8f4c3a440d3f2d9a60
+time=2026-10-04T16:53:07.884+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=b7956e7339bfda31 op=os_docker_step
+time=2026-10-04T17:07:54.007+02:00 level=WARN msg="signedjobs: REJECTED signed op — executor not called" job=dec707a162451c5b op=os_docker_step reason=rejected err="authz: replay (nonce already seen): nonce cb4a902da3007b8f4c3a440d3f2d9a60"
@@ -0,0 +1,2 @@
+HTTP/1.1 303 See Other
+Location: /system?flash=done
@@ -0,0 +1,16 @@
+time=2026-10-04T16:39:43.360+02:00 level=INFO msg="audit: gate decision" class=os_docker_step host=demo-hp-bb76ea guest="" source=one_shot_job disposition=destructive allowed=true reason=signed key_id=felhom-op-1 nonce=f21cbf93… durable_id=""
+time=2026-10-04T16:39:43.360+02:00 level=INFO msg="gate decision" class=os_docker_step guest="" source=one_shot_job disposition=destructive allowed=true reason=signed
+time=2026-10-04T16:39:43.360+02:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=c9f9efc098335bf9 op=os_docker_step key_id=felhom-op-1 nonce=f21cbf9302fb8b655b3ccf3e36f1cbea
+os-apply: LIVE-RESTORE already on
+time=2026-10-04T16:39:45.427+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: LIVE-RESTORE already on"
+time=2026-10-04T16:39:45.427+02:00 level=INFO msg="osupdate: START" run=20261004T143943Z layer=docker vmid=9201 ring=0 trigger=signed enabled=true release=undo-to-29.7.2
+os-apply: START release=undo-to-29.7.2 layer=docker:9201 lane=slow mode=apply select=listed packages=6 authority=signed UNDO
+os-apply: PLAN upgrade=6 already=0 not-installed=0 from-snapshot=0
+os-apply: DONE rc=0 seconds=16.7 upgraded=6 restart-needed=containerd-shim reboot-needed=no
+time=2026-10-04T16:40:26.840+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=undo-to-29.7.2 layer=docker:9201 lane=slow mode=apply select=listed packages=6 authority=signed UNDO"
+time=2026-10-04T16:40:26.840+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=6 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T16:40:26.840+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=16.7 upgraded=6 restart-needed=containerd-shim reboot-needed=no"
+time=2026-10-04T16:41:02.662+02:00 level=INFO msg="osupdate: DONE" run=20261004T143943Z layer=docker vmid=9201 ring=0 trigger=signed outcome=applied healthy=true reason="" upgraded=6 pending=6 not_covered=0 restart_needed=1 reboot_needed=false wrapper_seconds=41.3
+time=2026-10-04T16:41:02.716+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=c9f9efc098335bf9 op=os_docker_step
+IDS IDENTICAL (24 containers)
+after: engine=29.7.2 live=true
@@ -0,0 +1,8 @@
+time=2026-10-04T16:42:55.380+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+guest nothing healthy=true
+time=2026-10-04T16:43:10.212+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+host nothing healthy=true
+time=2026-10-04T16:43:58.115+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=6 already=0 not-installed=0 from-snapshot=0"
+docker applied healthy=true
+IDS IDENTICAL (24 containers)
+engine=29.8.2
@@ -0,0 +1 @@
+before: 24 containers engine=29.8.2
@@ -0,0 +1 @@
+{"release_id":"undo-to-29.7.2","undo":true,"packages":[{"name":"containerd.io","version":"2.3.3-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-buildx-plugin","version":"0.36.1-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce","version":"5:29.7.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce-cli","version":"5:29.7.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce-rootless-extras","version":"5:29.7.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-compose-plugin","version":"5.5.0-1~debian.13~trixie","origin":"Docker CE"}]}
@@ -0,0 +1,2 @@
+signed: op=os_docker_step host=demo-hp-bb76ea guest="" key_id=felhom-op-1 nonce=f21cbf9302fb8b655b3ccf3e36f1cbea expires=2026-10-04T15:15:26Z
+uploaded signed op to the hub jobs queue
@@ -0,0 +1,53 @@
+ sysctl kernel.panic kernel.panic_on_oops kernel.sysrq kernel.panic_on_warn kernel.softlockup_panic kernel.hung_task_panic
+kernel.panic = 0
+kernel.panic_on_oops = 0
+kernel.sysrq = 438
+kernel.panic_on_warn = 0
+kernel.softlockup_panic = 0
+kernel.hung_task_panic = 0
+ ls -la /sys/fs/pstore/ /var/lib/systemd/pstore/
+/sys/fs/pstore/:
+total 0
+drwxr-x---  2 root root 0 Oct  4 14:36 .
+drwxr-xr-x 10 root root 0 Oct  4 14:36 ..
+
+/var/lib/systemd/pstore/:
+total 8
+drwxr-xr-x  2 root root 4096 May 21 13:32 .
+drwxr-xr-x 12 root root 4096 Aug 21 17:44 ..
+ mount
+ grep pstore
+none on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
+ cat /sys/module/pstore/parameters/backend
+efi_pstore
+ dmesg
+ grep -iE 'pstore|efi_pstore|ramoops|erst'
+ head
+[    4.129612] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
+[    4.153714] pstore: Using crash dump compression: deflate
+[    4.206148] pstore: Registered efi_pstore as persistent store backend
+[    4.207174] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
+[    4.207514] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
+[    4.210265] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore
+ grep -E '^#?Storage' /etc/systemd/journald.conf
+#Storage=auto
+ ls /var/log/journal
+ head -2
+9756a908fef046fda6a56428d3fefe3c
+ journalctl --list-boots --no-pager
+ tail -4
+ -3 9cfa86adcf9f4dec99e324f5e2645032 Sun 2026-10-04 09:44:31 CEST Sun 2026-10-04 09:45:46 CEST
+ -2 1407ed1ad3ba4a37ac84ddae7cb05b3b Sun 2026-10-04 09:46:18 CEST Sun 2026-10-04 14:26:34 CEST
+ -1 ffd2f9aa02a344399338755a7958219f Sun 2026-10-04 14:27:08 CEST Sun 2026-10-04 14:35:43 CEST
+  0 7e320e5daf1f4ee79760c2de1aaf39ff Sun 2026-10-04 14:36:16 CEST Sun 2026-10-04 15:26:56 CEST
+ systemctl is-enabled systemd-pstore.service
+enabled
+ lsmod
+ grep -iE 'efi_pstore|ramoops'
+efi_pstore             12288  0
+ ls /sys/firmware/efi/efivars
+ grep -ci dump
+0
+ which kdump-config
+ cat /proc/cmdline
+BOOT_IMAGE=/boot/vmlinuz-7.0.14-20-pve root=/dev/mapper/pve-root ro quiet
@@ -0,0 +1,85 @@
+== before crash 1, 2026-10-04T13:38:26Z
+kernel.panic = 10
+7e320e5d-af1f-4ee7-9760-c2de1aaf39ff
+7.0.14-20-pve
+2026-10-04 14:36:11
+crash issued 2026-10-04T13:38:27Z
+== after crash 1, 2026-10-04T13:39:27Z
+ uname -r
+7.0.14-20-pve
+ uptime -s
+2026-10-04 15:39:06
+ cat /proc/sys/kernel/random/boot_id
+0defc390-c295-4e9f-aa6e-13d69a1f24c3
+ sysctl kernel.panic
+kernel.panic = 0
+ ls -la /sys/fs/pstore/ /var/lib/systemd/pstore/
+/sys/fs/pstore/:
+total 0
+drwxr-x---  2 root root 0 Oct  4 15:39 .
+drwxr-xr-x 10 root root 0 Oct  4 15:39 ..
+
+/var/lib/systemd/pstore/:
+total 8
+drwxr-xr-x  2 root root 4096 May 21 13:32 .
+drwxr-xr-x 12 root root 4096 Aug 21 17:44 ..
+ find /var/lib/systemd/pstore -type f
+ head
++ find /var/lib/systemd/pstore -type f
++ head -2
+ journalctl -b -1 --no-pager -n 8 -o short-iso
+2026-10-04T15:38:27+02:00 demo-hp systemd[205455]: Listening on gpg-agent.socket - GnuPG cryptographic agent and passphrase cache.
+2026-10-04T15:38:27+02:00 demo-hp systemd[205455]: Listening on ssh-agent.socket - OpenSSH Agent socket.
+2026-10-04T15:38:27+02:00 demo-hp systemd[205455]: Reached target sockets.target - Sockets.
+2026-10-04T15:38:27+02:00 demo-hp systemd[205455]: Reached target basic.target - Basic System.
+2026-10-04T15:38:27+02:00 demo-hp systemd[205455]: Reached target default.target - Main User Target.
+2026-10-04T15:38:27+02:00 demo-hp systemd[205455]: Startup finished in 310ms.
+2026-10-04T15:38:27+02:00 demo-hp systemd[1]: Started user@0.service - User Manager for UID 0.
+2026-10-04T15:38:27+02:00 demo-hp systemd[1]: Started session-15.scope - Session 15 of User root.
+ journalctl -b -2 --no-pager -n 4 -o short-iso
+2026-10-04T14:35:43+02:00 demo-hp systemd-shutdown[1]: Syncing filesystems and block devices.
+2026-10-04T14:35:43+02:00 demo-hp systemd-shutdown[1]: Sending SIGTERM to remaining processes...
+2026-10-04T14:35:43+02:00 demo-hp systemd-journald[349]: Received SIGTERM from PID 1 (systemd-shutdow).
+2026-10-04T14:35:43+02:00 demo-hp systemd-journald[349]: Journal stopped
+ journalctl -b 0 -u systemd-pstore --no-pager -o cat
+ head
+ last -x
+ head -5
+reboot   system boot  7.0.14-20-pve    Sun Oct  4 15:39 - still running
+reboot   system boot  7.0.14-20-pve    Sun Oct  4 14:36 - crash 
+reboot   system boot  7.0.14-20-pve    Sun Oct  4 14:27 - 14:35  (00:08)
+shutdown system down  7.0.14-20-pve    Sun Oct  4 14:35 - 14:36  (00:00)
+reboot   system boot  7.0.2-6-pve      Sun Oct  4 09:46 - 14:26  (04:39)
+ cat /sys/module/efi_pstore/parameters/pstore_disable
+N
+ cat /sys/module/pstore/parameters/max_reason
+cat: /sys/module/pstore/parameters/max_reason: No such file or directory
+ cat /sys/module/pstore/parameters/kmsg_bytes
+10240
++ uname -r
+ grep -E 'EFI_VARS_PSTORE|PSTORE_DEFAULT|CONFIG_PSTORE=' /boot/config-7.0.14-20-pve
+CONFIG_EFI_VARS_PSTORE=m
+# CONFIG_EFI_VARS_PSTORE_DEFAULT_DISABLE is not set
+CONFIG_PSTORE=y
+CONFIG_PSTORE_DEFAULT_KMSG_BYTES=10240
+ cat /sys/module/printk/parameters/always_kmsg_dump
+N
+ journalctl -b 0 -u felhom-agent --no-pager -o short-iso
+ grep -iE 'started|report|hub:'
+ head -5
+ cut -c1-200
+2026-10-04T15:39:22+02:00 demo-hp systemd[1]: Started felhom-agent.service - Felhom host agent (Proxmox host tier; hub control loop + PBS verify + storage watchdog).
+2026-10-04T15:39:23+02:00 demo-hp sudo[1492]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
+2026-10-04T15:39:24+02:00 demo-hp felhom-agent[1304]: time=2026-10-04T15:39:24.149+02:00 level=INFO msg="guest-power: watchdog started" interval=1m0s max_attempts=3
+2026-10-04T15:39:24+02:00 demo-hp felhom-agent[1304]: time=2026-10-04T15:39:24.149+02:00 level=INFO msg="controller-supervisor: started" interval=30s confirm_sweeps=2 crashloop_max=3 crashloop_window=
+2026-10-04T15:39:24+02:00 demo-hp sudo[1585]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
+ pct status 9201
+status: running
+ journalctl -k -b -2 --no-pager
+ grep -ciE 'oops|BUG:'
+0
+ journalctl -k --no-pager
+ grep -iE 'Oops|BUG:|Call Trace'
+ head -3
+ cat /proc/sys/kernel/tainted
+4097
@@ -0,0 +1,29 @@
+== crash 1 issued 2026-10-04T14:51:28Z
+kernel.panic = 10
+0defc390-c295-4e9f-aa6e-13d69a1f24c3
+back: ping+ssh after 54s
+7.0.14-20-pve
+kernel.panic = 10
+{
+  "armed": true,
+  "boot_id": "cc0aa40b-ac53-4524-a46f-6c5983ce5ed9",
+  "config": {
+    "LIMIT": 3,
+    "PANIC_SECONDS": 10,
+    "REARM_HOURS": 24,
+    "WINDOW_MINUTES": 60
+  },
+  "kernel_panic": 10,
+  "last_boot_at": "2026-10-04T14:52:08Z",
+  "last_boot_unclean": true,
+  "tripped": false,
+  "unclean_boots": [
+    "2026-10-04T14:52:08Z"
+  ],
+  "unclean_boots_24h": 1,
+  "unclean_boots_in_window": 1,
+  "updated_at": "2026-10-04T14:52:18Z",
+  "version": 1
+}
+hub log since the boot:
+2026/10/04 16:52:57 [INFO] host-report from demo-hp-bb76ea (1 guests, 4 storage targets, 0 backups, 2 restore-tests, 2 pbs-snapshots, 13538 bytes)
@@ -0,0 +1,33 @@
+== crash 2 issued 2026-10-04T14:53:36Z
+kernel.panic = 10
+cc0aa40b-ac53-4524-a46f-6c5983ce5ed9
+back: ping+ssh after 53s
+7.0.14-20-pve
+kernel.panic = 0
+{
+  "armed": false,
+  "boot_id": "53f69dfb-93ef-44c3-a9ad-803d86dfb0b3",
+  "config": {
+    "LIMIT": 3,
+    "PANIC_SECONDS": 10,
+    "REARM_HOURS": 24,
+    "WINDOW_MINUTES": 60
+  },
+  "kernel_panic": 0,
+  "last_boot_at": "2026-10-04T14:54:33Z",
+  "last_boot_unclean": true,
+  "tripped": true,
+  "tripped_at": "2026-10-04T14:54:43Z",
+  "tripped_reason": "2 unclean boots within 60 minutes \u2014 the next crash leaves the box off (limit 3)",
+  "unclean_boots": [
+    "2026-10-04T14:52:08Z",
+    "2026-10-04T14:54:33Z"
+  ],
+  "unclean_boots_24h": 2,
+  "unclean_boots_in_window": 2,
+  "updated_at": "2026-10-04T14:54:43Z",
+  "version": 1
+}
+hub log since the boot:
+2026/10/04 16:52:57 [INFO] host-report from demo-hp-bb76ea (1 guests, 4 storage targets, 0 backups, 2 restore-tests, 2 pbs-snapshots, 13538 bytes)
+2026/10/04 16:54:53 [INFO] host-report from demo-hp-bb76ea (1 guests, 4 storage targets, 0 backups, 2 restore-tests, 2 pbs-snapshots, 13395 bytes)
@@ -0,0 +1,4 @@
+== crash 3 issued 2026-10-04T14:55:13Z
+kernel.panic = 0
+53f69dfb-93ef-44c3-a9ad-803d86dfb0b3
+STAYED OFF for 184s (no ping)
@@ -0,0 +1,29 @@
+== after the operator's power-on, ssh back 2026-10-04T15:17:41Z (27s after his word)
+7.0.14-20-pve
+kernel.panic = 0
+{
+  "armed": false,
+  "boot_id": "182af911-22a9-41ec-97ac-7ee247a4d470",
+  "config": {
+    "LIMIT": 3,
+    "PANIC_SECONDS": 10,
+    "REARM_HOURS": 24,
+    "WINDOW_MINUTES": 60
+  },
+  "kernel_panic": 0,
+  "last_boot_at": "2026-10-04T15:17:26Z",
+  "last_boot_unclean": true,
+  "tripped": true,
+  "tripped_at": "2026-10-04T14:54:43Z",
+  "tripped_reason": "2 unclean boots within 60 minutes \u2014 the next crash leaves the box off (limit 3)",
+  "unclean_boots": [
+    "2026-10-04T14:52:08Z",
+    "2026-10-04T14:54:33Z",
+    "2026-10-04T15:17:26Z"
+  ],
+  "unclean_boots_24h": 3,
+  "unclean_boots_in_window": 3,
+  "updated_at": "2026-10-04T15:17:36Z",
+  "version": 1
+}
+status: stopped
@@ -0,0 +1,12 @@
+2026/10/04 17:21:00 [INFO] Operator email sent for demo-hp/app_start_failed
+2026/10/04 17:26:30 [INFO] Operator email sent for demo-hp/app_stopped_unhealthy
+2026/10/04 17:26:30 [INFO] Customer email sent to drill@felhom.eu for demo-hp/app_stopped_unhealthy
+2026/10/04 17:32:52 [WARN] host demo-hp-bb76ea crash guard: host_crash_restart
+2026/10/04 17:32:52 [WARN] host demo-hp-bb76ea crash guard: host_restarted_after_crash
+2026/10/04 17:32:52 [WARN] host demo-hp-bb76ea crash guard: host_crash_restart
+2026/10/04 17:32:52 [WARN] host demo-hp-bb76ea crash guard: host_restarted_after_crash
+2026/10/04 17:32:52 [WARN] host demo-hp-bb76ea crash guard: host_crash_restart
+2026/10/04 17:32:52 [WARN] host demo-hp-bb76ea crash guard: host_restarted_after_crash
+2026/10/04 17:32:52 [WARN] host demo-hp-bb76ea crash guard: host_crash_guard_tripped
+2026/10/04 17:32:52 [INFO] Operator email sent for demo-hp/host_crash_restart
+2026/10/04 17:32:52 [INFO] Operator email sent for demo-hp/host_crash_guard_tripped
@@ -0,0 +1,240 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>System — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.132.0">
+    <style>
+        .sys td, .sys th { white-space: nowrap; font-size: 0.82em; vertical-align: top; }
+        .sys .grp { border-left: 2px solid var(--border, #444); }
+        .c-warn { color: var(--warn); font-weight: 600; }
+        .c-bad { color: var(--danger, #e5534b); font-weight: 700; }
+        .sys form { display: inline; }
+        .rel-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(16rem, 1fr)); gap: 0.75rem; }
+    </style>
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    
+<script>
+
+
+function felhomConfirm(el,question,onYes){
+    if(!el||el.dataset.fcOpen)return;
+    el.dataset.fcOpen='1';
+    var wrap=document.createElement('span');wrap.className='inline-confirm';
+    var q=document.createElement('span');q.className='inline-confirm-q';q.textContent=question;
+    var yes=document.createElement('button');yes.type='button';yes.className='btn btn-sm btn-danger';yes.textContent='Igen';
+    var no=document.createElement('button');no.type='button';no.className='btn btn-sm btn-outline';no.textContent='Mégse';
+    wrap.appendChild(q);wrap.appendChild(yes);wrap.appendChild(no);
+    el.style.display='none';el.parentNode.insertBefore(wrap,el.nextSibling);
+    function close(){wrap.remove();el.style.display='';delete el.dataset.fcOpen;}
+    no.addEventListener('click',close);
+    yes.addEventListener('click',function(){close();onYes();});
+}
+document.addEventListener('click',function(e){
+    var btn=e.target.closest?e.target.closest('[data-confirm]'):null;
+    if(!btn)return;
+    e.preventDefault();
+    felhomConfirm(btn,btn.getAttribute('data-confirm'),function(){
+        var form=btn.closest('form');
+        if(form){if(form.requestSubmit)form.requestSubmit(btn);else form.submit();}
+    });
+});
+</script>
+
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link">Hosts</a>
+                <a href="/system" class="nav-link active">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+
+        <h2 style="margin-bottom: 1rem;">System — versions and OS updates</h2>
+
+        
+        
+
+        <section class="card" style="margin-bottom: 1.5rem;">
+            <h3 style="margin-top: 0;">Approved releases</h3>
+            <div class="rel-grid">
+                
+                <div><strong>guest</strong>: <code>os-guest-20261004-123933</code><br>
+                    <span class="text-muted">272 packages · 2026-10-04 12:39 UTC · by auto</span></div>
+                
+                <div><strong>host</strong>: <code>os-host-20261004-124133</code><br>
+                    <span class="text-muted">607 packages · 2026-10-04 12:41 UTC · by auto</span></div>
+                
+                <div><strong>docker</strong>: <code>os-docker-20261004-142842</code><br>
+                    <span class="text-muted">6 packages · 2026-10-04 14:28 UTC · by operator</span></div>
+                
+            </div>
+            <h3>What ring 0 runs now</h3>
+            <div class="rel-grid">
+                
+                <div><strong>guest</strong>:
+                    272 packages, first seen 2026-10-04 11:07 UTC<br>
+                    <span class="text-muted">approved as os-guest-20261004-123933</span>
+                    
+                    
+                </div>
+                
+                <div><strong>host</strong>:
+                    607 packages, first seen 2026-10-04 12:24 UTC<br>
+                    <span class="text-muted">approved as os-host-20261004-124133</span>
+                    
+                    
+                </div>
+                
+                <div><strong>docker</strong>:
+                    6 packages, first seen 2026-10-04 14:28 UTC<br>
+                    <span class="text-muted">approved as os-docker-20261004-142842</span>
+                    
+                    
+                </div>
+                
+            </div>
+            <form method="POST" action="/os/approve-now" style="margin-top: 0.8rem;">
+                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                <button type="submit" class="btn btn-sm btn-danger" data-confirm="Approve the guest and host sets ring 0 runs NOW, without the 24 h + 1 night wait? Every ring-1 box installs them at its next night run.">Approve now (guest + host)</button>
+                <span class="text-muted" style="font-size: 0.85em;">An urgent fix only — normally the hub approves after 24 h and one night.</span>
+            </form>
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow-x: auto;">
+            <table class="data-table sys">
+                <thead>
+                    <tr>
+                        <th>Box</th><th>Ring / updates</th><th>Tunnel</th>
+                        <th class="grp">Proxmox</th><th>Kernel (running)</th><th>Kernel (next boot)</th><th>Debian</th><th>Felhom release</th><th>Pending</th><th>Not covered</th><th>Held</th><th>Reboot needed</th><th>kernel.panic</th><th>Oops</th><th>Crash restarts 24 h</th><th>Crash guard</th>
+                        <th class="grp">Guest Debian</th><th>Felhom release</th><th>Pending</th><th>Restart needed</th>
+                        <th class="grp">Docker</th><th>containerd</th><th>live-restore</th><th>Docker release</th>
+                        <th class="grp">Last OS leg</th>
+                    </tr>
+                    <tr class="text-muted"><th></th><th></th><th></th><th class="grp" colspan="13">host</th><th class="grp" colspan="4">guest</th><th class="grp" colspan="4">Docker engine</th><th class="grp"></th></tr>
+                </thead>
+                <tbody>
+                    
+                    <tr>
+                        <td><a href="/hosts/demo-felhom-8363b5">demo-felhom-8363b5</a><br><span class="text-muted">Demo Ügyfél</span>
+                            </td>
+                        <td>
+                            ring 0
+                            <form method="POST" action="/os/ring/demo-felhom-8363b5">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="1"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make demo-felhom-8363b5 a normal (ring 1) box? It then installs only approved releases.">→ normal</button>
+                                
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/demo-felhom-8363b5">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for demo-felhom-8363b5? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td>running</td>
+                        <td class="grp ">9.2.2</td>
+                        <td>7.0.2-6-pve</td><td>7.0.2-6-pve</td><td>13.7</td>
+                        <td>os-host-20261004-124133</td><td>80</td><td>0</td>
+                        <td>none</td><td class="c-warn">since 2026-10-04</td><td>10 s</td><td>no</td>
+                        <td>0</td><td>armed</td>
+                        <td class="grp ">13.7</td>
+                        <td>os-guest-20261004-123933</td><td>0</td><td>10</td>
+                        <td class="grp ">29.8.2</td>
+                        <td>2.3.6-1~debian.13~trixie</td><td>on</td><td>os-docker-20261004-142842</td>
+                        <td class="grp " title="last successful leg: 51 min ago">40 min ago · nothing · 12 s</td>
+                    </tr>
+                    
+                    <tr>
+                        <td><a href="/hosts/demo-hp-bb76ea">demo-hp-bb76ea</a><br><span class="text-muted">Demo HP</span>
+                            </td>
+                        <td>
+                            ring 0
+                            <form method="POST" action="/os/ring/demo-hp-bb76ea">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="1"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make demo-hp-bb76ea a normal (ring 1) box? It then installs only approved releases.">→ normal</button>
+                                
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/demo-hp-bb76ea">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for demo-hp-bb76ea? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td>running</td>
+                        <td class="grp ">9.2.2</td>
+                        <td>7.0.14-20-pve</td><td>7.0.14-20-pve</td><td>13.7</td>
+                        <td>ring0-20261004T144238Z</td><td>78</td><td>0</td>
+                        <td>none</td><td>no</td><td class="c-warn">0 (stays off)</td><td>no</td>
+                        <td class="c-warn">3</td><td class="c-bad" title="2 unclean boots within 60 minutes — the next crash leaves the box off (limit 3)">TRIPPED 2026-10-04T14:54:43Z</td>
+                        <td class="grp ">13.7</td>
+                        <td>ring0-20261004T144238Z</td><td>6</td><td>1</td>
+                        <td class="grp ">29.8.2</td>
+                        <td>2.3.6-1~debian.13~trixie</td><td>on</td><td>ring0-20261004T144238Z</td>
+                        <td class="grp " title="last successful leg: 50 min ago">48 min ago · applied · 45 s</td>
+                    </tr>
+                    
+                    <tr>
+                        <td><a href="/hosts/drill-r50-0a4f9a">drill-r50-0a4f9a</a><br><span class="text-muted">drill-r50</span>
+                            <br><span class="c-warn" style="font-weight: normal;">no versions reported (agent older than v0.142.0)</span></td>
+                        <td>
+                            ring 1
+                            <form method="POST" action="/os/ring/drill-r50-0a4f9a">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="ring" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Make drill-r50-0a4f9a a DEMO (ring 0) box? It then installs every new fix first and takes unsigned Docker steps if its root-owned ring-0 mark allows.">→ demo</button>
+                            </form><br>
+                            updates <strong>ON</strong>
+                            <form method="POST" action="/os/enabled/drill-r50-0a4f9a">
+                                <input type="hidden" name="_csrf" value=""><input type="hidden" name="return" value="/system">
+                                <input type="hidden" name="on" value="0"><button type="submit" class="btn btn-sm btn-outline" data-confirm="Switch OS updates OFF for drill-r50-0a4f9a? It keeps reporting and installs nothing.">switch off</button>
+                                
+                            </form>
+                        </td>
+                        <td class="c-bad">inactive</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td>
+                        <td>—</td><td>0</td><td>0</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>no</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>no</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn">not installed</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td>—</td><td>0</td><td>0</td>
+                        <td class="grp c-warn">unknown</td>
+                        <td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td class="c-warn" title="the box could not read it (agent older than v0.142.0, or the guest is down)">unknown</td><td>—</td>
+                        <td class="grp " title="last successful leg: never">never · — · 0 s</td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+        </section>
+        <p class="text-muted" style="font-size: 0.85em;">Amber: worth a look. Red: an operator alarm fires (`08` §6.3). "unknown": the box could not read the value — never a guess.</p>
+        
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.132.0</span>
+        </footer>
+    </div>
+</body>
+</html>
+
@@ -0,0 +1,3 @@
+crash-guard: RE-ARMED by operator (was tripped: True); kernel.panic=10
+kernel.panic = 10
+{'armed': True, 'tripped': False, 'rearmed_at': '2026-10-04T15:33:25Z', 'rearmed_by': 'operator', 'unclean_boots_in_window': 0, 'unclean_boots_24h': 3, 'last_trip': {'at': '2026-10-04T14:54:43Z', 'reason': '2 unclean boots within 60 minutes — the next crash leaves the box off (limit 3)'}}
@@ -0,0 +1 @@
+2026/10/04 17:47:51 [WARN] host demo-hp-bb76ea crash guard: host_crash_guard_rearmed
@@ -0,0 +1,10 @@
+755 root 41c759e4c195d349 /usr/local/sbin/felhom-os-apply
+755 root f45b99817fe8ff50 /usr/local/sbin/felhom-crash-guard
+644 root 5580ad6cdd51990b /etc/systemd/system/felhom-crash-guard.service
+644 root 93d2c4749fe63ab1 /etc/systemd/system/felhom-crash-guard-check.service
+644 root b214e79e483c339f /etc/systemd/system/felhom-crash-guard-check.timer
+644 root 9b9d305b421f81d2 /etc/felhom/crash-guard.conf
+644 root d17cb8e4456575b1 /etc/felhom/operator-signers
+644 root 338a564cc8392bad /etc/felhom/os-trust.json
+755 root 18acdf69f9c915be /usr/local/bin/felhom-agent-0.142.0-rc1
+kernel.panic = 10
@@ -0,0 +1,33 @@
+Created symlink '/etc/systemd/system/sysinit.target.wants/felhom-crash-guard.service' -> '/etc/systemd/system/felhom-crash-guard.service'.
+Created symlink '/etc/systemd/system/timers.target.wants/felhom-crash-guard-check.timer' -> '/etc/systemd/system/felhom-crash-guard-check.timer'.
+755 root 41c759e4c195d349 /usr/local/sbin/felhom-os-apply
+755 root f45b99817fe8ff50 /usr/local/sbin/felhom-crash-guard
+644 root 5580ad6cdd51990b /etc/systemd/system/felhom-crash-guard.service
+644 root 93d2c4749fe63ab1 /etc/systemd/system/felhom-crash-guard-check.service
+644 root b214e79e483c339f /etc/systemd/system/felhom-crash-guard-check.timer
+644 root 9b9d305b421f81d2 /etc/felhom/crash-guard.conf
+644 root d17cb8e4456575b1 /etc/felhom/operator-signers
+644 root af0890ef7a5778d0 /etc/felhom/os-trust.json
+755 root 18acdf69f9c915be /usr/local/bin/felhom-agent-0.142.0-rc1
+active
+active
+kernel.panic = 10
+{
+  "armed": true,
+  "boot_id": "0defc390-c295-4e9f-aa6e-13d69a1f24c3",
+  "config": {
+    "LIMIT": 3,
+    "PANIC_SECONDS": 10,
+    "REARM_HOURS": 24,
+    "WINDOW_MINUTES": 60
+  },
+  "kernel_panic": 10,
+  "last_boot_at": "2026-10-04T13:39:06Z",
+  "last_boot_unclean": false,
+  "tripped": false,
+  "unclean_boots": [],
+  "unclean_boots_24h": 0,
+  "unclean_boots_in_window": 0,
+  "updated_at": "2026-10-04T14:08:23Z",
+  "version": 1
+}
@@ -0,0 +1,20 @@
+== 2026-10-04T15:56:54Z demo-felhom guest 9201
+1506210 srw-rw---- 1 root docker 0 Oct  4 14:13 /var/run/docker.sock
+143 srw-rw---- 0 root 991 0 Aug 10 07:25 /var/run/docker.sock
+cloudflared mounts-socket=no
+felhom-controller mounts-socket=socket
+filebrowser mounts-socket=no
+traefik mounts-socket=socket
+opengist mounts-socket=no
+2026-10-04T14:13:33+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:33.518477721Z" level=info msg="Restoring containers: start."
+2026-10-04T14:13:33+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:33.566951360Z" level=info msg="Deleting nftables IPv4 rules" error="running nft: /dev/stdin:1:17-30: Error: Could not process rule: No such file or directory\ndelete table ip docker-bridges\n                ^^^^^^^^^^^^^^\n exit status 1"
+2026-10-04T14:13:33+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:33.575940033Z" level=info msg="Deleting nftables IPv6 rules" error="running nft: /dev/stdin:1:18-31: Error: Could not process rule: No such file or directory\ndelete table ip6 docker-bridges\n                 ^^^^^^^^^^^^^^\n exit status 1"
+2026-10-04T14:13:34+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:34.060369627Z" level=info msg="there are running containers, updated network configuration will not take affect"
+2026-10-04T14:13:34+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:34.060647190Z" level=info msg="Loading containers: done."
+2026-10-04T14:13:34+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:34.074298789Z" level=info msg="Docker daemon" commit=8af9fe3 containerd-snapshotter=false storage-driver=overlay2 version=29.8.2
+2026-10-04T14:13:34+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:34.074346504Z" level=info msg="Initializing buildkit"
+2026-10-04T14:13:34+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:34.074487454Z" level=warning msg="failed check for fsverity support" error="enable fsverity failed: operation not supported" path=/var/lib/docker/buildkit/content
+2026-10-04T14:13:34+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:34.219975776Z" level=info msg="Completed buildkit initialization"
+2026-10-04T14:13:34+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:34.223588774Z" level=info msg="Daemon has completed initialization"
+2026-10-04T14:13:34+00:00 demo-felhom dockerd[3505106]: time="2026-10-04T14:13:34.223626300Z" level=info msg="API listen on /run/docker.sock"
+2026-10-04T14:13:34+00:00 demo-felhom systemd[1]: Started docker.service - Docker Application Container Engine.
@@ -0,0 +1,40 @@
+2026/10/04 11:57:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 11:57:03 [INFO] [scheduler] Job agent-channel-health completed (took 837ms)
+2026/10/04 11:58:02 [INFO] [scheduler] Running job: system-health
+2026/10/04 11:58:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 11:58:02 [INFO] [monitor] Health check: status=ok
+2026/10/04 11:58:02 [INFO] [scheduler] Job system-health completed (took 194ms)
+2026/10/04 11:58:03 [INFO] [scheduler] Job agent-channel-health completed (took 958ms)
+2026/10/04 11:58:42 [INFO] Health probes: 1 ok (of 1 probed)
+2026/10/04 11:59:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 11:59:03 [INFO] [scheduler] Job agent-channel-health completed (took 838ms)
+2026/10/04 12:00:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:00:03 [INFO] [scheduler] Job agent-channel-health completed (took 833ms)
+2026/10/04 12:01:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:01:03 [INFO] [scheduler] Job agent-channel-health completed (took 825ms)
+2026/10/04 12:02:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:02:03 [INFO] [scheduler] Job agent-channel-health completed (took 828ms)
+2026/10/04 12:03:02 [INFO] [scheduler] Running job: system-health
+2026/10/04 12:03:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:03:02 [INFO] [monitor] Health check: status=ok
+2026/10/04 12:03:02 [INFO] [scheduler] Job system-health completed (took 239ms)
+2026/10/04 12:03:02 [INFO] [monitor] Health check: status=ok
+2026/10/04 12:03:03 [INFO] [scheduler] Job agent-channel-health completed (took 1.085s)
+2026/10/04 12:03:42 [INFO] Health probes: 1 ok (of 1 probed)
+2026/10/04 12:04:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:04:03 [INFO] [scheduler] Job agent-channel-health completed (took 831ms)
+2026/10/04 12:05:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:05:03 [INFO] [scheduler] Job agent-channel-health completed (took 838ms)
+2026/10/04 12:06:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:06:03 [INFO] [scheduler] Job agent-channel-health completed (took 860ms)
+2026/10/04 12:07:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:07:03 [INFO] [scheduler] Job agent-channel-health completed (took 846ms)
+2026/10/04 12:08:02 [INFO] [scheduler] Running job: system-health
+2026/10/04 12:08:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:08:02 [INFO] [monitor] Health check: status=ok
+2026/10/04 12:08:02 [INFO] [scheduler] Job system-health completed (took 188ms)
+2026/10/04 12:08:03 [INFO] [scheduler] Job agent-channel-health completed (took 968ms)
+2026/10/04 12:08:52 [INFO] Health probes: 1 ok (of 1 probed)
+2026/10/04 12:09:02 [INFO] [scheduler] Running job: agent-channel-health
+2026/10/04 12:09:03 [INFO] [scheduler] Job agent-channel-health completed (took 836ms)
+2026/10/04 12:10:02 [INFO] [scheduler] Running job: agent-channel-health
@@ -0,0 +1,14 @@
+repair 2026-10-04T15:57:13Z: docker restart traefik felhom-controller (containers only; the engine untouched)
+traefik
+felhom-controller
+controller health=healthy
+1506210 srw-rw---- 1 root 991 0 Oct  4 14:13 /var/run/docker.sock
+1506210 srw-rw----    1 root     991              0 Oct  4 14:13 /var/run/docker.sock
+cloudflared | Up 5 hours (healthy)
+felhom-controller | Up 9 seconds (healthy)
+filebrowser | Up 7 hours (healthy)
+traefik | Up 9 seconds
+opengist | Up 10 hours (healthy)
+ActiveEnterTimestamp=Sun 2026-10-04 14:13:34 UTC
+2026/10/04 15:57:16 [INFO] [monitor] Health check: status=ok
+2026/10/04 15:57:21 [INFO] [monitor] Health check: status=ok
@@ -0,0 +1,4 @@
+felhom-pve: {"host_id":"demo-felhom-8363b5","ring0_slow_lane":false}
+felhom-pve: 644 root
+demo-hp: {"host_id":"demo-hp-bb76ea","ring0_slow_lane":false}
+demo-hp: 644 root
@@ -0,0 +1,2 @@
+demo-hp wrapper (R-858 fix, pre-release) bdf60f5c79a84db7
+demo-felhom wrapper v0.142.1 bdf60f5c79a84db7
@@ -0,0 +1,8 @@
+time=2026-10-04T18:02:51.747+02:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=2f5f79f8a5c22923 op=os_docker_step key_id=felhom-op-1 nonce=f0715c2a4d18668ca2de50258f4ea07c
+os-apply: SOCKET-USERS restarted=felhom-controller,traefik rc=0 (R-858: they held the old docker socket)
+time=2026-10-04T18:04:05.435+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: SOCKET-USERS restarted=felhom-controller,traefik rc=0 (R-858: they held the old docker socket)"
+time=2026-10-04T18:04:05.466+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=2f5f79f8a5c22923 op=os_docker_step
+controller sees engine 29.7.2
+guest socket: 12159  controller: 12159  traefik: 12159
+felhom-controller Up 17 minutes (healthy)
+traefik Up 17 minutes
@@ -0,0 +1,5 @@
+[CAUGHT] socket users restarted after a step: DockerLane.test_step_restarts_only_the_socket_users rc=1
+[CAUGHT] only socket users, not every container: DockerLane.test_step_restarts_only_the_socket_users rc=1
+[CAUGHT] controller reach check in health: DockerLane.test_health_says_when_the_controller_cannot_reach_docker rc=1
+[CAUGHT] health rule fails a blind controller (Go)
+after restore: ok  	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.730s
@@ -0,0 +1,8 @@
+time=2026-10-04T18:04:05.435+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: SOCKET-USERS restarted=felhom-controller,traefik rc=0 (R-858: they held the old docker socket)"
+time=2026-10-04T18:32:54.303+02:00 level=INFO msg="osupdate: START" run=20261004T163252Z layer=docker vmid=9201 ring=0 trigger=signed enabled=true release=r858-proof-forward
+time=2026-10-04T18:34:03.314+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=r858-proof-forward layer=docker:9201 lane=slow mode=apply select=listed packages=6 authority=signed"
+time=2026-10-04T18:34:03.314+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: SOCKET-USERS restarted=felhom-controller,traefik rc=0 (R-858: they held the old docker socket)"
+controller sees engine 29.8.2
+socket guest=18453 controller=18453 traefik=18453
+2026/10/04 18:04:05 [INFO] osupdates: demo-hp-bb76ea reported docker run 20261004T160251Z: ring=0 mode=apply outcome=applied healthy=true upgraded=6 pending=6 not-covered=0 restart-needed=1 reboot-needed=false wrapper=71.6s
+2026/10/04 18:34:03 [INFO] osupdates: demo-hp-bb76ea reported docker run 20261004T163252Z: ring=0 mode=apply outcome=applied healthy=true upgraded=6 pending=0 not-covered=0 restart-needed=1 reboot-needed=false wrapper=68.9s
@@ -0,0 +1,2 @@
+felhom-pve: {"host_id":"demo-felhom-8363b5","ring0_slow_lane":true}
+demo-hp: {"host_id":"demo-hp-bb76ea","ring0_slow_lane":true}
@@ -12,3 +12,7 @@ R10: 4 raise(s) removed -> suite rc=1; failing tests: test_R10_bind_only_in_a_sn
 R11: 9 raise(s) removed -> suite rc=1; failing tests: test_R11_bad_name, test_R11_bad_version_string, test_R11_duplicate; own test(s) failed: YES
 R12: 1 raise(s) removed -> suite rc=1; failing tests: test_R12_host_layer; own test(s) failed: YES
 R13: 1 raise(s) removed -> suite rc=1; failing tests: test_R13_repair_does_not_fix_it; own test(s) failed: YES
+== RP (wrapper conffile): the old wording (every conffile line reported as kept)
+FAIL: test_updated_vs_kept (__main__.Conffiles.test_updated_vs_kept)
+AssertionError: 'os-apply: CONFFILE updated /etc/debian_version (it was not changed locally)' not found in ['os-apply: START release=os-t1 layer=guest:9201 lane=fast mode=apply packages=2', 'os-apply: REPAIR configured=0 fixed=0', 'os-apply: PLAN upgrade=2 already=0 not-installed=0 from-snapshot=0', 'os-apply: CONFFILE kept /etc/debian_version', 'os-apply: CONFFILE kept /etc/ssh/sshd_config', 'os-apply: DONE rc=0 seconds=0.0 upgraded=2']
+FAILED (failures=1)
@@ -33,3 +33,9 @@ FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/localapi	0.563s
 == RP (local API hook): run the leg after a FAILED backup too
        afterbackup_test.go:60: the leg ran after a FAILED backup: [8200]
 FAIL (recorded from the run above)
+== RP (selftest flag): drop the os-update case from selftestFlag.Set
+    selftest_flag_test.go:37: dispatched but refused by --selftest: [os-update]
+FAIL	gitea.dooplex.hu/admin/felhom-agent/cmd/felhom-agent	0.010s
+== RP (leg): baseline = the apply's own before-reading only (the pre-fix rule)
+    leg_test.go:279: an app that stopped during the run passed: {RunID:20261004T040000Z Trigger:night Mode:apply Ring:0 ReleaseID:ring0-20261004T040000Z Outcome:applied Healthy:true HealthReason: VMID:9201 Upgraded:[{Name:libc6 Version: Origin:}] Installed:[] Pending:[] NotCovered:[] RestartNeeded:[] DockerRestartNeeded:false RebootNeeded:false Refused:[]}
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.195s
@@ -0,0 +1,90 @@
+1791104489.428006469 302
+1791104490.114048259 302
+1791104490.867431883 302
+1791104491.567710242 302
+1791104492.259675188 302
+1791104492.950311346 302
+1791104493.658119300 302
+1791104494.345122536 302
+1791104495.027891977 302
+1791104495.724212868 302
+1791104496.422407509 302
+1791104497.128727616 302
+1791104497.800224130 302
+1791104498.519248669 302
+1791104499.203467396 302
+1791104499.898045533 502
+1791104500.634582730 502
+1791104501.341338498 502
+1791104502.054985899 502
+1791104502.792687510 502
+1791104503.493970117 502
+1791104504.236100842 502
+1791104504.980276912 502
+1791104505.689072396 502
+1791104506.471770326 502
+1791104507.185250940 502
+1791104508.316227102 502
+1791104509.010368824 404
+1791104509.690610573 302
+1791104510.381269521 302
+1791104511.048606319 302
+1791104511.718435404 302
+1791104512.462336250 502
+1791104513.187968905 502
+1791104513.871106880 302
+1791104514.590549839 302
+1791104515.281227748 302
+1791104515.964558604 302
+1791104516.720756559 302
+1791104517.426584135 302
+1791104518.115760697 302
+1791104518.810201702 302
+1791104519.507291973 302
+1791104520.172275341 302
+1791104520.866469463 302
+1791104521.541290853 302
+1791104522.219399888 302
+1791104522.891993819 302
+1791104523.560829339 302
+1791104524.240801665 302
+1791104524.946212414 302
+1791104525.627388820 302
+1791104526.329685610 302
+1791104527.011843100 302
+1791104527.684678463 302
+1791104528.380214494 302
+1791104529.058330821 302
+1791104529.740540446 302
+1791104530.436470651 302
+1791104531.121601771 302
+1791104531.809593181 302
+1791104532.503149505 302
+1791104533.173713227 302
+1791104533.856540265 302
+1791104534.544973430 302
+1791104535.253316994 302
+1791104535.932080467 302
+1791104536.599649531 302
+1791104537.294371521 302
+1791104537.968883762 302
+1791104538.662330307 302
+1791104539.355172110 302
+1791104540.014510174 302
+1791104540.732988090 302
+1791104541.409134391 302
+1791104542.085884593 302
+1791104542.786152655 302
+1791104543.468040923 302
+1791104544.129791932 302
+1791104545.053502785 302
+1791104545.747716347 302
+1791104546.430378570 302
+1791104547.089481515 302
+1791104547.826344317 302
+1791104548.491677889 302
+1791104549.200576500 302
+1791104549.873171847 302
+1791104550.549674984 302
+1791104551.223931502 302
+1791104551.897581518 302
@@ -0,0 +1,93 @@
+1791104489.428473157 302
+1791104490.154456409 302
+1791104490.836711390 302
+1791104491.512992898 302
+1791104492.188921801 302
+1791104492.892582650 302
+1791104493.561025000 302
+1791104494.243801578 302
+1791104494.946931718 302
+1791104495.612770541 302
+1791104496.296371937 302
+1791104496.959464984 302
+1791104497.624069604 302
+1791104498.287369613 302
+1791104498.954218253 302
+1791104499.625720367 302
+1791104500.285823257 302
+1791104500.962562788 302
+1791104501.623569883 302
+1791104502.280153375 502
+1791104502.973783514 502
+1791104503.699803496 502
+1791104504.477656042 502
+1791104505.219746840 502
+1791104505.935741922 502
+1791104506.696658409 502
+1791104507.383980538 502
+1791104508.099312305 502
+1791104508.778478656 502
+1791104509.485583013 502
+1791104510.206577081 502
+1791104510.894142842 502
+1791104511.583745893 502
+1791104512.301754435 502
+1791104512.965882550 502
+1791104513.674358904 502
+1791104514.365515398 302
+1791104515.084994873 302
+1791104515.780146089 302
+1791104516.467673569 302
+1791104517.127579680 302
+1791104517.806574655 302
+1791104518.464909459 302
+1791104519.137104416 302
+1791104519.819783794 502
+1791104520.535296664 530
+1791104521.190614199 302
+1791104521.864351794 302
+1791104522.541227619 302
+1791104523.210468864 302
+1791104523.862415986 302
+1791104524.545740539 302
+1791104525.247498962 302
+1791104525.907280086 302
+1791104526.568936189 302
+1791104527.226485539 302
+1791104527.883240645 302
+1791104528.572125595 302
+1791104529.261611733 302
+1791104529.990339048 302
+1791104530.672950131 302
+1791104531.335651483 302
+1791104532.020457877 302
+1791104532.681731917 302
+1791104533.347279268 302
+1791104533.999334640 302
+1791104534.662664742 302
+1791104535.326620624 302
+1791104535.998590184 302
+1791104536.651396219 302
+1791104537.299040037 302
+1791104537.937915214 302
+1791104538.640295166 302
+1791104539.329815017 302
+1791104539.985113137 302
+1791104540.672365246 302
+1791104541.358812693 302
+1791104542.035139317 302
+1791104542.698601778 302
+1791104543.351804369 302
+1791104544.018269004 302
+1791104544.686076951 302
+1791104545.360390747 302
+1791104546.033087077 302
+1791104546.697562917 302
+1791104547.355845574 302
+1791104548.046312941 302
+1791104548.728344788 302
+1791104549.396103966 302
+1791104550.084478791 302
+1791104550.772128650 302
+1791104551.425494194 302
+1791104552.081996680 302
@@ -0,0 +1,2 @@
+demo-hp (public URL through Cloudflare, 09:01:30-09:06:00Z): samples 92 codes {'302': 73, '502': 18, '530': 1}; first non-302 09:01:42Z, last 09:02:00Z; gap at most 19.6 s
+demo-felhom (public URL through Cloudflare, 09:01:30-09:06:00Z): samples 89 codes {'302': 74, '502': 14, '404': 1}; first non-302 09:01:39Z, last 09:01:53Z; gap at most 14.7 s
@@ -0,0 +1,306 @@
+=== felhom-agent 0.140.0-rc2 selftest=os-update vmid=9201 ring=0 enabled=true release=false ===
+time=2026-10-04T11:08:52.934+02:00 level=INFO msg="osupdate: START" run=20261004T090852Z vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T090852Z
+time=2026-10-04T11:09:08.241+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T090852Z layer=guest:9201 lane=fast mode=inventory packages=0"
+time=2026-10-04T11:11:46.297+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T090852Z layer=guest:9201 lane=fast mode=apply packages=53"
+time=2026-10-04T11:11:46.297+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T11:11:46.297+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=53 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T11:11:46.297+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: CONFFILE updated /etc/debian_version (it was not changed locally)"
+time=2026-10-04T11:11:46.297+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=18.7 upgraded=53"
+time=2026-10-04T11:11:46.297+02:00 level=INFO msg="osupdate: DONE" run=20261004T090852Z vmid=9201 ring=0 trigger=debug outcome=applied healthy=true reason="" upgraded=53 pending=6 not_covered=6 restart_needed=7
+  --- os-update report ---
+  {
+    "docker_restart_needed": true,
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "applied",
+    "pending": 6,
+    "refused": null,
+    "release_id": "ring0-20261004T090852Z",
+    "restart_needed": [
+      "agetty",
+      "containerd",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd-logind"
+    ],
+    "ring": 0,
+    "run_id": "20261004T090852Z",
+    "upgraded": [
+      {
+        "name": "libc6",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "base-files",
+        "version": "13.8+deb13u7",
+        "origin": ""
+      },
+      {
+        "name": "bash",
+        "version": "5.2.37-2+b10",
+        "origin": ""
+      },
+      {
+        "name": "bsdutils",
+        "version": "1:2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "gzip",
+        "version": "1.13-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libperl5.40",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl-base",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl-modules-5.40",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "liblastlog2-2",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bsdextrautils",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "util-linux-extra",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libblkid1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libmount1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libsmartcols1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "mount",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "fdisk",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libuuid1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "util-linux",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libfdisk1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libaudit-common",
+        "version": "1:4.0.2-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libaudit1",
+        "version": "1:4.0.2-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libsqlite3-0",
+        "version": "3.46.1-7+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "libc-bin",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "login",
+        "version": "1:4.16.0-2+really2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "logsave",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libext2fs2t64",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "e2fsprogs",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libexpat1",
+        "version": "2.8.3-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "openssl-provider-legacy",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "libssl3t64",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "postfix",
+        "version": "3.10.13-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "python3.13",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libpython3.13-stdlib",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "python3.13-minimal",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libpython3.13-minimal",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "tzdata",
+        "version": "2026c-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libcap2",
+        "version": "1:2.75-10+deb13u1+b3",
+        "origin": ""
+      },
+      {
+        "name": "libpcre2-8-0",
+        "version": "10.46-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "dhcpcd-base",
+        "version": "1:10.1.0-11+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "bind9-dnsutils",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bind9-host",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bind9-libs",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libc-l10n",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "locales",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "libssh2-1t64",
+        "version": "1.11.1-1+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "curl",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libcurl4t64",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libcap2-bin",
+        "version": "1:2.75-10+deb13u1+b3",
+        "origin": ""
+      },
+      {
+        "name": "libcom-err2",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libcurl3t64-gnutls",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libss2",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "openssl",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      }
+    ]
+  }
@@ -0,0 +1,306 @@
+=== felhom-agent 0.140.0-rc2 selftest=os-update vmid=9201 ring=0 enabled=true release=false ===
+time=2026-10-04T11:04:21.448+02:00 level=INFO msg="osupdate: START" run=20261004T090421Z vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T090421Z
+time=2026-10-04T11:04:42.234+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T090421Z layer=guest:9201 lane=fast mode=inventory packages=0"
+time=2026-10-04T11:08:07.757+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T090421Z layer=guest:9201 lane=fast mode=apply packages=53"
+time=2026-10-04T11:08:07.757+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T11:08:07.757+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=53 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T11:08:07.757+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: CONFFILE kept /etc/debian_version"
+time=2026-10-04T11:08:07.757+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=31.7 upgraded=53"
+time=2026-10-04T11:08:07.758+02:00 level=INFO msg="osupdate: DONE" run=20261004T090421Z vmid=9201 ring=0 trigger=debug outcome=applied healthy=true reason="" upgraded=53 pending=6 not_covered=6 restart_needed=7
+  --- os-update report ---
+  {
+    "docker_restart_needed": true,
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "applied",
+    "pending": 6,
+    "refused": null,
+    "release_id": "ring0-20261004T090421Z",
+    "restart_needed": [
+      "agetty",
+      "containerd",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd-logind"
+    ],
+    "ring": 0,
+    "run_id": "20261004T090421Z",
+    "upgraded": [
+      {
+        "name": "libc6",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "base-files",
+        "version": "13.8+deb13u7",
+        "origin": ""
+      },
+      {
+        "name": "bash",
+        "version": "5.2.37-2+b10",
+        "origin": ""
+      },
+      {
+        "name": "bsdutils",
+        "version": "1:2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "gzip",
+        "version": "1.13-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libperl5.40",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl-base",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl-modules-5.40",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "liblastlog2-2",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bsdextrautils",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "util-linux-extra",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libblkid1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libmount1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libsmartcols1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "mount",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "fdisk",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libuuid1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "util-linux",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libfdisk1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libaudit-common",
+        "version": "1:4.0.2-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libaudit1",
+        "version": "1:4.0.2-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libsqlite3-0",
+        "version": "3.46.1-7+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "libc-bin",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "login",
+        "version": "1:4.16.0-2+really2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "logsave",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libext2fs2t64",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "e2fsprogs",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libexpat1",
+        "version": "2.8.3-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "openssl-provider-legacy",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "libssl3t64",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "postfix",
+        "version": "3.10.13-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "python3.13",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libpython3.13-stdlib",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "python3.13-minimal",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libpython3.13-minimal",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "tzdata",
+        "version": "2026c-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libcap2",
+        "version": "1:2.75-10+deb13u1+b3",
+        "origin": ""
+      },
+      {
+        "name": "libpcre2-8-0",
+        "version": "10.46-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "dhcpcd-base",
+        "version": "1:10.1.0-11+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "bind9-dnsutils",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bind9-host",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bind9-libs",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libc-l10n",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "locales",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "libssh2-1t64",
+        "version": "1.11.1-1+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "curl",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libcurl4t64",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libcap2-bin",
+        "version": "1:2.75-10+deb13u1+b3",
+        "origin": ""
+      },
+      {
+        "name": "libcom-err2",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libcurl3t64-gnutls",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libss2",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "openssl",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      }
+    ]
+  }
@@ -0,0 +1,58 @@
+=== felhom-agent 0.140.0-rc2 selftest=os-update vmid=9201 ring=1 enabled=true release=true ===
+time=2026-10-04T11:15:59.945+02:00 level=INFO msg="osupdate: START" run=20261004T091559Z vmid=9201 ring=1 trigger=debug enabled=true release=os-20261004-091417
+time=2026-10-04T11:16:10.634+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=os-20261004-091417 layer=guest:9201 lane=fast mode=inventory packages=0"
+time=2026-10-04T11:20:04.641+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=os-20261004-091417 layer=guest:9201 lane=fast mode=apply packages=272"
+time=2026-10-04T11:20:04.641+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T11:20:04.641+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=3 already=269 not-installed=0 from-snapshot=0"
+time=2026-10-04T11:20:04.641+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=9.3 upgraded=3"
+time=2026-10-04T11:20:04.642+02:00 level=INFO msg="osupdate: DONE" run=20261004T091559Z vmid=9201 ring=1 trigger=debug outcome=applied healthy=true reason="" upgraded=3 pending=6 not_covered=6 restart_needed=10
+  --- os-update report ---
+  {
+    "docker_restart_needed": true,
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "applied",
+    "pending": 6,
+    "refused": null,
+    "release_id": "os-20261004-091417",
+    "restart_needed": [
+      "agetty",
+      "containerd",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd",
+      "systemd-journal",
+      "systemd-logind",
+      "systemd-network"
+    ],
+    "ring": 1,
+    "run_id": "20261004T091559Z",
+    "upgraded": [
+      {
+        "name": "libssl3t64",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "openssl",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "openssl-provider-legacy",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      }
+    ]
+  }
@@ -0,0 +1,58 @@
+=== felhom-agent 0.140.0-rc3 selftest=os-update vmid=9201 ring=0 enabled=true release=false ===
+time=2026-10-04T11:22:44.828+02:00 level=INFO msg="osupdate: START" run=20261004T092244Z vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T092244Z
+time=2026-10-04T11:22:59.130+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T092244Z layer=guest:9201 lane=fast mode=inventory packages=0"
+time=2026-10-04T11:23:37.731+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T092244Z layer=guest:9201 lane=fast mode=apply packages=3"
+time=2026-10-04T11:23:37.731+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T11:23:37.731+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=3 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T11:23:37.731+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=3.8 upgraded=3"
+time=2026-10-04T11:28:41.271+02:00 level=INFO msg="osupdate: DONE" run=20261004T092244Z vmid=9201 ring=0 trigger=debug outcome=health_failed healthy=false reason="privatebin was running and is not" upgraded=3 pending=6 not_covered=6 restart_needed=10
+  --- os-update report ---
+  {
+    "docker_restart_needed": true,
+    "health_reason": "privatebin was running and is not",
+    "healthy": false,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "health_failed",
+    "pending": 6,
+    "refused": null,
+    "release_id": "ring0-20261004T092244Z",
+    "restart_needed": [
+      "agetty",
+      "containerd",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd",
+      "systemd-journal",
+      "systemd-logind",
+      "systemd-network"
+    ],
+    "ring": 0,
+    "run_id": "20261004T092244Z",
+    "upgraded": [
+      {
+        "name": "openssl-provider-legacy",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "libssl3t64",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "openssl",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      }
+    ]
+  }
@@ -0,0 +1,38 @@
+=== felhom-agent 0.140.0 selftest=os-update vmid=9201 ring=0 enabled=true release=false ===
+time=2026-10-04T11:38:36.515+02:00 level=INFO msg="osupdate: START" run=20261004T093836Z vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T093836Z
+time=2026-10-04T11:38:50.533+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T093836Z layer=guest:9201 lane=fast mode=inventory packages=0"
+time=2026-10-04T11:38:50.534+02:00 level=INFO msg="osupdate: DONE" run=20261004T093836Z vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=10
+  --- os-update report ---
+  {
+    "docker_restart_needed": true,
+    "health_reason": "",
+    "healthy": true,
+    "mode": "inventory",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "refused": null,
+    "release_id": "ring0-20261004T093836Z",
+    "restart_needed": [
+      "agetty",
+      "containerd",
+      "cron",
+      "dbus-daemon",
+      "dhclient",
+      "sshd",
+      "systemd",
+      "systemd-journal",
+      "systemd-logind",
+      "systemd-network"
+    ],
+    "ring": 0,
+    "run_id": "20261004T093836Z",
+    "upgraded": null
+  }
@@ -0,0 +1,6 @@
+os_update_applied|System security fixes installed (3 package(s)).|hub|Oct 04 09:28
+os_update_health_failed|OS update on demo-hp-bb76ea: the guest was NOT healthy after 3 package(s) were installed (privatebin was running and is not). Nothing was undone automatically (no guest snapshot is possible, R-837); last night's whole-guest backup is th
+os_update_applied|System security fixes installed (3 package(s)).|hub|Oct 04 09:21
+os_update_applied|System security fixes installed (53 package(s)).|hub|Oct 04 09:03
+sent|OS update on demo-hp-bb76ea: the guest was NOT healthy after 3 package(s) were installed (privatebin was running and is not). Nothing was undone automatically (no guest snapshot is possible, R-837); last night's whole-guest backup is the undo.|Oct 04 09:2
+skipped|OS update on demo-hp-bb76ea: the guest was NOT healthy after 3 package(s) were installed (privatebin was running and is not). Nothing was undone automatically (no guest snapshot is possible, R-837); last night's whole-guest backup is the undo.|Oct 04 0
@@ -0,0 +1,9 @@
+# agent v0.141.0 tunnel-probe red-proofs (each mutation compiles; result line printed)
+== mutation: container-state-only
+    cloudflared_test.go:32: running but not connected: got "running" (container running but the tunnel is NOT connected (cloudflared /ready fails)), want "not_running" (…NOT connected…)
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/hub	0.009s
+== mutation: unknown-as-down
+    cloudflared_test.go:32: guest not running: got "not_running" (could not ask the guest: CT 9201 not running), want "unknown" (…could not ask…)
+    cloudflared_test.go:32: sudo refused: got "not_running" (could not ask the guest: sudo: a password is required), want "unknown" (…could not ask…)
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/hub	0.009s
+ok  	gitea.dooplex.hu/admin/felhom-agent/internal/hub	(cached)
@@ -0,0 +1,3 @@
+== RP (controller): the healthcheck block removed from the cloudflared template
+    infra_tunnel_test.go:300: tunnel=false: compose lacks "test: [\"CMD\", \"cloudflared\", \"tunnel\", \"--metrics\", \"localhost:20241\", \"ready\"]":
+FAIL	gitea.dooplex.hu/admin/felhom-controller/internal/stacks	0.007s
@@ -0,0 +1,17 @@
+# hub v0.131.0 tunnel-alarm red-proofs, 2026-10-04 (each mutation compiles; the package result line is printed)
+== mutation: one-report
+    --- FAIL: TestTunnelAlarm_Sequences/a_single_bad_report_does_not_alarm (0.02s)
+        tunnel_test.go:47: tunnel_down=1 tunnel_recovered=0, want 0 / 0
+    --- FAIL: TestTunnelAlarm_Sequences/unknown_never_alarms_and_breaks_the_run (0.03s)
+        tunnel_test.go:47: tunnel_down=1 tunnel_recovered=0, want 0 / 0
+== mutation: unknown-counts-as-down
+    --- FAIL: TestTunnelAlarm_Sequences/unknown_never_alarms_and_breaks_the_run (0.03s)
+        tunnel_test.go:47: tunnel_down=1 tunnel_recovered=0, want 0 / 0
+FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/api	0.164s
+== mutation: no-alarm
+    --- FAIL: TestTunnelAlarm_Sequences/two_in_a_row_alarm_once (0.03s)
+        tunnel_test.go:47: tunnel_down=0 tunnel_recovered=0, want 1 / 0
+    --- FAIL: TestTunnelAlarm_Sequences/recovery_after_an_alarm (0.04s)
+        tunnel_test.go:47: tunnel_down=0 tunnel_recovered=1, want 1 / 1
+== reverted
+ok  	gitea.dooplex.hu/admin/felhom-hub/internal/api	0.168s
@@ -0,0 +1,19 @@
+11:34:43 container=running/healthy hub=running alarm=none
+11:35:45 container=running/healthy hub=running alarm=none
+11:36:47 container=running/unhealthy hub=running alarm=none
+11:37:49 container=running/unhealthy hub=not_running alarm=none
+11:38:51 container=running/unhealthy hub=not_running alarm=none
+11:39:53 container=running/unhealthy hub=not_running alarm=none
+11:40:55 container=running/unhealthy hub=not_running alarm=none
+11:41:57 container=running/unhealthy hub=not_running alarm=none
+11:42:59 container=running/unhealthy hub=not_running alarm=none
+11:44:01 container=running/unhealthy hub=not_running alarm=none
+11:45:03 container=running/unhealthy hub=not_running alarm=none
+11:46:05 container=running/unhealthy hub=not_running alarm=none
+11:47:07 container=running/unhealthy hub=not_running alarm=none
+11:48:09 container=running/unhealthy hub=not_running alarm=none
+11:49:10 container=running/unhealthy hub=not_running alarm=none
+11:50:12 container=running/unhealthy hub=not_running alarm=none
+11:51:14 container=running/unhealthy hub=not_running alarm=none
+11:52:16 container=running/unhealthy hub=not_running alarm=none
+11:53:18 container=running/unhealthy hub=not_running alarm=2026/10/04 13:52:29 [WARN] host demo-hp-bb76ea tunnel: tunnel_down (container running but the tunnel is NOT connected (cloudflared /ready fails))
@@ -0,0 +1,8 @@
+block at 2026-10-04T11:34:33Z: DOCKER-USER DROP tcp+udp dport 7844 in guest 9201 on demo-hp
+Chain DOCKER-USER (1 references)
+target     prot opt source               destination         
+DROP       tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:7844
+DROP       udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:7844
+unblock at 2026-10-04T11:53:29Z
+Chain DOCKER-USER (1 references)
+target     prot opt source               destination         
@@ -0,0 +1,14 @@
+11:53:39 container=unhealthy hub=not_running
+11:54:41 container=healthy hub=not_running
+11:55:43 container=healthy hub=not_running
+11:56:45 container=healthy hub=not_running
+11:57:47 container=healthy hub=not_running
+11:58:48 container=healthy hub=not_running
+11:59:50 container=healthy hub=not_running
+12:00:52 container=healthy hub=not_running
+12:01:54 container=healthy hub=not_running
+12:02:56 container=healthy hub=not_running
+12:03:58 container=healthy hub=not_running
+12:05:00 container=healthy hub=not_running
+12:06:02 container=healthy hub=not_running
+12:07:04 container=healthy hub=not_running
@@ -0,0 +1,22 @@
+11:23:25 running
+11:23:56 running
+11:24:26 running
+11:24:57 running
+11:25:27 running
+11:25:58 running
+11:26:28 running
+11:26:59 running
+11:27:29 running
+11:28:00 running
+11:28:31 running
+11:29:01 running
+11:29:32 running
+11:30:02 running
+11:30:33 running
+11:31:03 running
+11:31:34 running
+11:32:04 running
+11:32:35 running
+11:33:05 running
+11:33:36 running
+11:34:06 running
@@ -0,0 +1,3 @@
+stop at 2026-10-04T11:23:17Z
+cloudflared
+exited 0
@@ -0,0 +1,5 @@
+2026/10/04 13:52:29 [WARN] host demo-hp-bb76ea tunnel: tunnel_down (container running but the tunnel is NOT connected (cloudflared /ready fails))
+2026/10/04 13:52:30 [INFO] Operator email sent for demo-hp/tunnel_down
+2026/10/04 13:52:29 [WARN] host demo-hp-bb76ea tunnel: tunnel_down (container running but the tunnel is NOT connected (cloudflared /ready fails))
+2026/10/04 13:52:30 [INFO] Operator email sent for demo-hp/tunnel_down
+2026/10/04 14:07:29 [WARN] host demo-hp-bb76ea tunnel: tunnel_recovered (connected)
@@ -0,0 +1,22 @@
+# agent v0.141.0 leg red-proofs (each mutation compiles; result line printed)
+== mutation: byo-gets-host
+--- FAIL: TestBYO_NoHostPlan (0.00s)
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.440s
+== mutation: host-after-failed-guest
+--- FAIL: TestGuestFailure_SkipsTheHost (0.00s)
+--- FAIL: TestHealth_FailsAfterTheWait (0.00s)
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.420s
+== mutation: host-ignores-services
+--- FAIL: TestHostHealthVerdict (0.00s)
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.441s
+== mutation: host-ignores-guest
+--- FAIL: TestHostHealthVerdict (0.00s)
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.410s
+== mutation: host-ignores-tunnel
+--- FAIL: TestHost_TunnelDownFailsTheHostStep (0.00s)
+--- FAIL: TestHostHealthVerdict (0.00s)
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.427s
+== mutation: host-uses-guest-release
+--- FAIL: TestRing1_EachLayerItsOwnRelease (0.00s)
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.461s
+ok  	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	(cached)
@@ -0,0 +1,36 @@
+# Live-found defects in agent 0.141.0 / hub 0.131.0 and their red-proofs, 2026-10-04T11:39:07Z
+
+## 1. the host restart scan hid lxc-start (cgroup 0::/lxc.monitor/9201 matched the skip pattern "lxc")
+live: demo-felhom after the 108-package host pass: lxc-start had 20 deleted maps, report said reboot_needed=false
+mutation: RESTART_SKIP_CGROUP host ':/lxc/' -> 'lxc'
+    self.assertFalse(self.grep(host, "0::/lxc.monitor/9201"), "lxc-start must NOT be skipped (it runs the guest)")
+    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+AssertionError: True is not false : lxc-start must NOT be skipped (it runs the guest)
+
+----------------------------------------------------------------------
+Ran 1 test in 0.005s
+
+FAILED (failures=1)
+
+## 2. the host scanned only after an install, so a reboot never cleared 'reboot needed'
+mutation: wrapper 'if self.layer == "host" and ...' -> 'if False and ...'
+----------------------------------------------------------------------
+Ran 1 test in 0.002s
+
+FAILED (failures=1)
+
+mutation: agent leg drops rep.RebootScanned copy
+=== RUN   TestHostReport_CarriesRebootScanned
+    leg_test.go:367: hub got [{RunID:20261004T040000Z Layer:guest Trigger:night Mode:apply Ring:0 ReleaseID:ring0-20261004T040000Z Outcome:nothing Healthy:true HealthReason: VMID:9201 Upgraded:[] Inst
+--- FAIL: TestHostReport_CarriesRebootScanned (0.00s)
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/osupdate	0.008s
+FAIL
+
+mutation: hub layerLine predicate ignores RebootScanned
+=== RUN   TestRebootNeeded_ClearedByAScannedPass
+    alarms_test.go:323: a scanned pass after the reboot did not clear it: 2026-10-04 12:00:00 +0000 UTC
+--- FAIL: TestRebootNeeded_ClearedByAScannedPass (0.03s)
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.037s
+FAIL
@@ -0,0 +1,716 @@
+=== felhom-agent 0.141.0 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T13:34:48.263+02:00 level=INFO msg="osupdate: START" run=20261004T113448Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T113448Z
+time=2026-10-04T13:34:59.917+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T113448Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T13:34:59.917+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T13:34:59.917+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T13:34:59.917+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T13:34:59.917+02:00 level=INFO msg="osupdate: DONE" run=20261004T113448Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=0 reboot_needed=false wrapper_seconds=11.6
+time=2026-10-04T13:34:59.927+02:00 level=INFO msg="osupdate: START" run=20261004T113448Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T113448Z
+time=2026-10-04T13:35:57.810+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T113448Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T13:35:57.810+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T13:35:57.810+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=108 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T13:35:57.810+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: CONFFILE updated /etc/debian_version (it was not changed locally)"
+time=2026-10-04T13:35:57.810+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=41.1 upgraded=108 restart-needed=(sd-pam),agetty,bash,blkmapd,chronyd,cron,dbus-daemon,dmeventd,felhom-agent,felhom-os-apply,ksmtuned,pmxcfs,proxmox-firewal,pve-firewall,pve-ha-crm,pve-ha-lrm,pvedaemon,pvedaemon worke,pvefw-logger,pveproxy,pveproxy worker,pvescheduler,pvestatd,qmeventd,rpcbind,rrdcached,smartd,spiceproxy,spiceproxy work,sshd,sshd-session,sudo,systemd-logind,systemd-udevd,watchdog-mux,zed reboot-needed=no"
+time=2026-10-04T13:35:58.613+02:00 level=INFO msg="osupdate: DONE" run=20261004T113448Z layer=host vmid=9201 ring=0 trigger=debug outcome=applied healthy=true reason="" upgraded=108 pending=80 not_covered=80 restart_needed=36 reboot_needed=false wrapper_seconds=57.8
+  --- os-update report (guest) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T113448Z",
+    "restart_needed": null,
+    "ring": 0,
+    "run_id": "20261004T113448Z",
+    "upgraded": [],
+    "wrapper_seconds": 11.6
+  }
+  --- os-update report (host) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-first-boot",
+      "pve-firmware",
+      "proxmox-kernel-7.0.14-20-pve-signed",
+      "proxmox-kernel-7.0",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed",
+      "tailscale"
+    ],
+    "outcome": "applied",
+    "pending": 80,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T113448Z",
+    "restart_needed": [
+      "(sd-pam)",
+      "agetty",
+      "bash",
+      "blkmapd",
+      "chronyd",
+      "cron",
+      "dbus-daemon",
+      "dmeventd",
+      "felhom-agent",
+      "felhom-os-apply",
+      "ksmtuned",
+      "pmxcfs",
+      "proxmox-firewal",
+      "pve-firewall",
+      "pve-ha-crm",
+      "pve-ha-lrm",
+      "pvedaemon",
+      "pvedaemon worke",
+      "pvefw-logger",
+      "pveproxy",
+      "pveproxy worker",
+      "pvescheduler",
+      "pvestatd",
+      "qmeventd",
+      "rpcbind",
+      "rrdcached",
+      "smartd",
+      "spiceproxy",
+      "spiceproxy work",
+      "sshd",
+      "sshd-session",
+      "sudo",
+      "systemd-logind",
+      "systemd-udevd",
+      "watchdog-mux",
+      "zed"
+    ],
+    "ring": 0,
+    "run_id": "20261004T113448Z",
+    "upgraded": [
+      {
+        "name": "libc6",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "base-files",
+        "version": "13.8+deb13u7",
+        "origin": ""
+      },
+      {
+        "name": "bash",
+        "version": "5.2.37-2+b10",
+        "origin": ""
+      },
+      {
+        "name": "bsdutils",
+        "version": "1:2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "gzip",
+        "version": "1.13-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libperl5.40",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl-base",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "perl-modules-5.40",
+        "version": "5.40.1-6+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "liblastlog2-2",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "eject",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bsdextrautils",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "util-linux-extra",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "fdisk",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libsmartcols1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libblkid1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libmount1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "mount",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libuuid1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "util-linux",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libfdisk1",
+        "version": "2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libaudit-common",
+        "version": "1:4.0.2-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libaudit1",
+        "version": "1:4.0.2-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "sqlite3",
+        "version": "3.46.1-7+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "libsqlite3-0",
+        "version": "3.46.1-7+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "libc-bin",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "login",
+        "version": "1:4.16.0-2+really2.41.5-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "logsave",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libext2fs2t64",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "e2fsprogs",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "dnsmasq-base",
+        "version": "2.91-1+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "dnsmasq",
+        "version": "2.91-1+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "libcap2",
+        "version": "1:2.75-10+deb13u1+b3",
+        "origin": ""
+      },
+      {
+        "name": "libpcre2-8-0",
+        "version": "10.46-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "libpcre2-posix3",
+        "version": "10.46-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "libexpat1",
+        "version": "2.8.3-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "openssl-provider-legacy",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "libssl3t64",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "postfix",
+        "version": "3.10.13-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "python3.13",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libpython3.13-stdlib",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "python3.13-minimal",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libpython3.13-minimal",
+        "version": "3.13.5-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "tzdata",
+        "version": "2026c-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "liblzma5",
+        "version": "5.8.1-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "rsync",
+        "version": "3.5.0+ds1-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "openssl",
+        "version": "3.5.7-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "dhcpcd-base",
+        "version": "1:10.1.0-11+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "libgssapi-krb5-2",
+        "version": "1.21.3-5+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libkrb5-3",
+        "version": "1.21.3-5+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libkrb5support0",
+        "version": "1.21.3-5+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libk5crypto3",
+        "version": "1.21.3-5+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libcom-err2",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libxml2",
+        "version": "2.12.7+dfsg+really2.9.14-2.1+deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "bind9-dnsutils",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bind9-host",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "bind9-libs",
+        "version": "1:9.20.29-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "krb5-locales",
+        "version": "1.21.3-5+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libc-l10n",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "locales",
+        "version": "2.41-12+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "xz-utils",
+        "version": "5.8.1-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "busybox",
+        "version": "1:1.37.0-6+b9",
+        "origin": ""
+      },
+      {
+        "name": "curl",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libssh2-1t64",
+        "version": "1.11.1-1+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "libcurl4t64",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "librabbitmq4",
+        "version": "0.15.0-1+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "libnss3",
+        "version": "2:3.110-1+deb13u4",
+        "origin": ""
+      },
+      {
+        "name": "libgcrypt20",
+        "version": "1.11.0-7+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "gpgsm",
+        "version": "2.4.7-21+deb13u1+b5",
+        "origin": ""
+      },
+      {
+        "name": "dirmngr",
+        "version": "2.4.7-21+deb13u1+b5",
+        "origin": ""
+      },
+      {
+        "name": "gpg",
+        "version": "2.4.7-21+deb13u1+b5",
+        "origin": ""
+      },
+      {
+        "name": "gpgconf",
+        "version": "2.4.7-21+deb13u1+b5",
+        "origin": ""
+      },
+      {
+        "name": "gpg-agent",
+        "version": "2.4.7-21+deb13u1+b5",
+        "origin": ""
+      },
+      {
+        "name": "libasound2t64",
+        "version": "1.2.14-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libasound2-data",
+        "version": "1.2.14-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libbytes-random-secure-perl",
+        "version": "0.29-4~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libcap2-bin",
+        "version": "1:2.75-10+deb13u1+b3",
+        "origin": ""
+      },
+      {
+        "name": "libcurl3t64-gnutls",
+        "version": "8.14.1-2+deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libevent-2.1-7t64",
+        "version": "2.1.13-stable-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libevent-core-2.1-7t64",
+        "version": "2.1.13-stable-1~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libgbm1",
+        "version": "25.0.7-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "mesa-libgallium",
+        "version": "25.0.7-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libglib2.0-0t64",
+        "version": "2.84.4-3~deb13u5",
+        "origin": ""
+      },
+      {
+        "name": "libgraphite2-3",
+        "version": "1.3.14-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libgstreamer-plugins-base1.0-0",
+        "version": "1.26.2-1+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "libhtml-parser-perl",
+        "version": "3.83-2~deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libhttp-daemon-perl",
+        "version": "6.16-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libtalloc2",
+        "version": "2:2.4.3+samba4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libtevent0t64",
+        "version": "2:0.16.2+samba4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libsmbclient0",
+        "version": "2:4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "samba-common",
+        "version": "2:4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "smbclient",
+        "version": "2:4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libtdb1",
+        "version": "2:1.4.13+samba4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libldb2",
+        "version": "2:2.11.0+samba4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "samba-libs",
+        "version": "2:4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libwbclient0",
+        "version": "2:4.22.11+dfsg-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libnet-dns-perl",
+        "version": "1.56-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libpcre2-16-0",
+        "version": "10.46-1~deb13u3",
+        "origin": ""
+      },
+      {
+        "name": "libpng16-16t64",
+        "version": "1.6.48-1+deb13u6",
+        "origin": ""
+      },
+      {
+        "name": "libslirp0",
+        "version": "4.8.0-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libxml-libxml-perl",
+        "version": "2.0207+dfsg+really+2.0134-5+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "socat",
+        "version": "1.8.0.3-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libss2",
+        "version": "1.47.2-3+b12",
+        "origin": ""
+      },
+      {
+        "name": "libtasn1-6",
+        "version": "4.20.0-2+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "libunbound8",
+        "version": "1.26.1-0+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "python3-idna",
+        "version": "3.10-1+deb13u1",
+        "origin": ""
+      },
+      {
+        "name": "python3-urllib3",
+        "version": "2.3.0-3+deb13u2",
+        "origin": ""
+      },
+      {
+        "name": "xfsprogs",
+        "version": "6.13.0-2+deb13u1",
+        "origin": ""
+      }
+    ],
+    "wrapper_seconds": 57.8
+  }
+  pass took 1m10.4s
+WALL_SECONDS=70.462688397
@@ -0,0 +1,135 @@
+=== felhom-agent 0.141.1 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T14:22:52.476+02:00 level=INFO msg="osupdate: START" run=20261004T122252Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T122252Z
+time=2026-10-04T14:23:08.079+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T122252Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T14:23:08.079+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T14:23:08.079+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T14:23:08.079+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T14:23:08.080+02:00 level=INFO msg="osupdate: DONE" run=20261004T122252Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=0 reboot_needed=false wrapper_seconds=15.5
+time=2026-10-04T14:23:08.089+02:00 level=INFO msg="osupdate: START" run=20261004T122252Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T122252Z
+time=2026-10-04T14:23:22.788+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T122252Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T14:23:22.788+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T14:23:22.788+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T14:23:22.788+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T14:23:23.852+02:00 level=INFO msg="osupdate: DONE" run=20261004T122252Z layer=host vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=78 not_covered=78 restart_needed=0 reboot_needed=false wrapper_seconds=14.6
+  --- os-update report (guest) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T122252Z",
+    "restart_needed": null,
+    "ring": 0,
+    "run_id": "20261004T122252Z",
+    "upgraded": [],
+    "wrapper_seconds": 15.5
+  }
+  --- os-update report (host) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "proxmox-secure-boot-support",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "amd64-microcode",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "pve-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed"
+    ],
+    "outcome": "nothing",
+    "pending": 78,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T122252Z",
+    "restart_needed": [],
+    "ring": 0,
+    "run_id": "20261004T122252Z",
+    "upgraded": [],
+    "wrapper_seconds": 14.6
+  }
+  pass took 31.4s
+WALL_SECONDS=31.520457543
@@ -0,0 +1,2 @@
+{"ok":true}
+ 200
@@ -0,0 +1,43 @@
+ PKG=tzdata
++ grep -oE 'tzdata:amd64 \([^,]+' /var/log/apt/history.log
++ tail -1
++ sed 's/.*(//'
+ OLD=2026b-0+deb13u1
++ dpkg-query -W '-f=${Version}' tzdata
+ NEW=2026c-0+deb13u1
+ echo OLD=2026b-0+deb13u1 NEW=2026c-0+deb13u1
+OLD=2026b-0+deb13u1 NEW=2026c-0+deb13u1
++ curl -s 'https://snapshot.debian.org/mr/binary/tzdata/2026c-0+deb13u1/binfiles?fileinfo=1'
++ python3 -c '
+import json,sys; d=json.load(sys.stdin)
+print(sorted(f["first_seen"] for v in d["fileinfo"].values() for f in v)[0])'
+ TS=20260831T204404Z
+ . /etc/os-release
++ PRETTY_NAME='Debian GNU/Linux 13 (trixie)'
++ NAME='Debian GNU/Linux'
++ VERSION_ID=13
++ VERSION='13 (trixie)'
++ VERSION_CODENAME=trixie
++ DEBIAN_VERSION_FULL=13.7
++ ID=debian
++ HOME_URL=https://www.debian.org/
++ SUPPORT_URL=https://www.debian.org/support
++ BUG_REPORT_URL=https://bugs.debian.org/
+ printf 'deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/%s %s main\ndeb [check-valid-until=no] http://snapshot.debian.org/archive/debian-security/%s %s-security main\n' 20260831T204404Z trixie 20260831T204404Z trixie
+ apt-get -q update
+ apt-get -s install --allow-downgrades tzdata=2026b-0+deb13u1
+ grep -E '^(Inst|Remv)|downgraded'
+0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 78 not upgraded.
+Inst tzdata [2026c-0+deb13u1] (2026b-0+deb13u1 Debian:13.6/stable [all])
+ DEBIAN_FRONTEND=noninteractive
+ apt-get -y -q install --allow-downgrades tzdata=2026b-0+deb13u1
+ rm -f /etc/apt/sources.list.d/felhom-undo-snapshot.list
+ apt-get -q update
+ dpkg-query -W tzdata
+tzdata	2026b-0+deb13u1
+ ls /etc/apt/sources.list.d/
+ceph.sources
+debian.sources
+pve-enterprise.sources
+pve-no-subscription.sources
+tailscale.list
@@ -0,0 +1,178 @@
+=== felhom-agent 0.141.1 selftest=os-update vmid=9201 ring=1 enabled=true guest-release=true host-release=true appliance=true ===
+time=2026-10-04T14:40:46.795+02:00 level=INFO msg="osupdate: START" run=20261004T124046Z layer=guest vmid=9201 ring=1 trigger=debug enabled=true release=os-guest-20261004-123933
+time=2026-10-04T14:40:56.895+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=os-guest-20261004-123933 layer=guest:9201 lane=fast mode=apply select=listed packages=272"
+time=2026-10-04T14:40:56.895+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T14:40:56.895+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=272 not-installed=0 from-snapshot=0"
+time=2026-10-04T14:40:56.895+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T14:40:56.896+02:00 level=INFO msg="osupdate: DONE" run=20261004T124046Z layer=guest vmid=9201 ring=1 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=0 reboot_needed=false wrapper_seconds=10.1
+time=2026-10-04T14:40:56.907+02:00 level=INFO msg="osupdate: START" run=20261004T124046Z layer=host vmid=9201 ring=1 trigger=debug enabled=true release=os-host-20261004-124034
+time=2026-10-04T14:41:10.034+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=os-host-20261004-124034 layer=host lane=fast mode=apply select=listed packages=605"
+time=2026-10-04T14:41:10.034+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T14:41:10.034+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=1 already=604 not-installed=0 from-snapshot=0"
+time=2026-10-04T14:41:10.034+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=1.4 upgraded=1 restart-needed=agetty,blkmapd,chronyd,cron,dbus-daemon,dmeventd,ksmtuned,lxc-monitord,lxc-start,lxcfs,pmxcfs,proxmox-firewal,pve-firewall,pve-ha-crm,pve-ha-lrm,pve-lxc-syscall,pvedaemon,pvedaemon worke,pvefw-logger,pveproxy,pveproxy worker,pvescheduler,pvestatd,qmeventd,rpcbind,rrdcached,smartd,spiceproxy,spiceproxy work,sshd,systemd-logind,systemd-udevd,watchdog-mux,zed reboot-needed=yes"
+time=2026-10-04T14:41:10.815+02:00 level=INFO msg="osupdate: DONE" run=20261004T124046Z layer=host vmid=9201 ring=1 trigger=debug outcome=applied healthy=true reason="" upgraded=1 pending=80 not_covered=80 restart_needed=34 reboot_needed=true wrapper_seconds=13.1
+  --- os-update report (guest) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "os-guest-20261004-123933",
+    "restart_needed": null,
+    "ring": 1,
+    "run_id": "20261004T124046Z",
+    "upgraded": [],
+    "wrapper_seconds": 10.1
+  }
+  --- os-update report (host) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-first-boot",
+      "pve-firmware",
+      "proxmox-kernel-7.0.14-20-pve-signed",
+      "proxmox-kernel-7.0",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed",
+      "tailscale"
+    ],
+    "outcome": "applied",
+    "pending": 80,
+    "reboot_needed": true,
+    "refused": null,
+    "release_id": "os-host-20261004-124034",
+    "restart_needed": [
+      "agetty",
+      "blkmapd",
+      "chronyd",
+      "cron",
+      "dbus-daemon",
+      "dmeventd",
+      "ksmtuned",
+      "lxc-monitord",
+      "lxc-start",
+      "lxcfs",
+      "pmxcfs",
+      "proxmox-firewal",
+      "pve-firewall",
+      "pve-ha-crm",
+      "pve-ha-lrm",
+      "pve-lxc-syscall",
+      "pvedaemon",
+      "pvedaemon worke",
+      "pvefw-logger",
+      "pveproxy",
+      "pveproxy worker",
+      "pvescheduler",
+      "pvestatd",
+      "qmeventd",
+      "rpcbind",
+      "rrdcached",
+      "smartd",
+      "spiceproxy",
+      "spiceproxy work",
+      "sshd",
+      "systemd-logind",
+      "systemd-udevd",
+      "watchdog-mux",
+      "zed"
+    ],
+    "ring": 1,
+    "run_id": "20261004T124046Z",
+    "upgraded": [
+      {
+        "name": "tzdata",
+        "version": "2026c-0+deb13u1",
+        "origin": ""
+      }
+    ],
+    "wrapper_seconds": 13.1
+  }
+  pass took 24.1s
+tzdata	2026c-0+deb13u1
@@ -0,0 +1,11 @@
+ PKG=eject
+ OLD=2.41-5
+ dpkg -s eject
+ grep -E '^(Status|Version)'
+Status: install ok installed
+Version: 2.41.5-0+deb13u1
+ curl -s 'https://snapshot.debian.org/mr/binary/eject/2.41-5/binfiles?fileinfo=1'
+ python3 -c '
+import json,sys; d=json.load(sys.stdin)
+print(sorted(f["first_seen"] for v in d["fileinfo"].values() for f in v)[0])'
+20250510T015204Z
@@ -0,0 +1,28 @@
+ PKG=eject
+ OLD=2.41-5
+ TS=20250510T015204Z
+ . /etc/os-release
++ PRETTY_NAME='Debian GNU/Linux 13 (trixie)'
++ NAME='Debian GNU/Linux'
++ VERSION_ID=13
++ VERSION='13 (trixie)'
++ VERSION_CODENAME=trixie
++ DEBIAN_VERSION_FULL=13.7
++ ID=debian
++ HOME_URL=https://www.debian.org/
++ SUPPORT_URL=https://www.debian.org/support
++ BUG_REPORT_URL=https://bugs.debian.org/
+ cat
++ date +%s
+ S=1791116624
+ apt-get -q update
+ tail -3
+Get:7 http://snapshot.debian.org/archive/debian/20250510T015204Z trixie/main amd64 Packages [9681 kB]
+Fetched 9904 kB in 5s (2165 kB/s)
+Reading package lists...
++ date +%s
+update_seconds=6
+ echo update_seconds=6
+ apt-get -s install --allow-downgrades eject=2.41-5
+ grep -E '^(Inst|Remv|Conf)|downgraded|newly|Err|E:'
+E: Version '2.41-5' for 'eject' was not found
@@ -0,0 +1,35 @@
+ PKG=eject
+ OLD=2.41-5
++ dpkg-query -W '-f=${Version}' eject
+ NEW=2.41.5-0+deb13u1
++ curl -s 'https://snapshot.debian.org/mr/binary/eject/2.41.5-0+deb13u1/binfiles?fileinfo=1'
++ python3 -c '
+import json,sys; d=json.load(sys.stdin)
+print(sorted(f["first_seen"] for v in d["fileinfo"].values() for f in v)[0])'
+TS=20260814T165831Z
+ TS=20260814T165831Z
+ echo TS=20260814T165831Z
+ . /etc/os-release
++ PRETTY_NAME='Debian GNU/Linux 13 (trixie)'
++ NAME='Debian GNU/Linux'
++ VERSION_ID=13
++ VERSION='13 (trixie)'
++ VERSION_CODENAME=trixie
++ DEBIAN_VERSION_FULL=13.7
++ ID=debian
++ HOME_URL=https://www.debian.org/
++ SUPPORT_URL=https://www.debian.org/support
++ BUG_REPORT_URL=https://bugs.debian.org/
+ cat
+ apt-get -q update
+ tail -1
+Reading package lists...
+ apt-cache madison eject
+     eject | 2.41.5-0+deb13u1 | http://deb.debian.org/debian trixie/main amd64 Packages
+     eject | 2.41.5-0+deb13u1 | http://security.debian.org/debian-security trixie-security/main amd64 Packages
+     eject | 2.41.5-0+deb13u1 | http://snapshot.debian.org/archive/debian-security/20260814T165831Z trixie-security/main amd64 Packages
+     eject |     2.41-5 | http://snapshot.debian.org/archive/debian/20260814T165831Z trixie/main amd64 Packages
+ apt-get -s install --allow-downgrades eject=2.41-5
+ grep -E '^(Inst|Remv|Conf)|downgraded|newly|Err|E:'
+E: Unable to correct problems, you have held broken packages.
+E: The following information from --solver 3.0 may provide additional context:
@@ -0,0 +1,37 @@
+ rm -f /etc/apt/sources.list.d/felhom-undo-snapshot.list
+ PKG=tzdata
+ OLD=2026b-0+deb13u1
++ dpkg-query -W '-f=${Version}' tzdata
+ NEW=2026c-0+deb13u1
+ echo NEW=2026c-0+deb13u1
+NEW=2026c-0+deb13u1
++ curl -s 'https://snapshot.debian.org/mr/binary/tzdata/2026c-0+deb13u1/binfiles?fileinfo=1'
++ python3 -c '
+import json,sys; d=json.load(sys.stdin)
+print(sorted(f["first_seen"] for v in d["fileinfo"].values() for f in v)[0])'
+TS=20260831T204404Z
+ TS=20260831T204404Z
+ echo TS=20260831T204404Z
+ . /etc/os-release
++ PRETTY_NAME='Debian GNU/Linux 13 (trixie)'
++ NAME='Debian GNU/Linux'
++ VERSION_ID=13
++ VERSION='13 (trixie)'
++ VERSION_CODENAME=trixie
++ DEBIAN_VERSION_FULL=13.7
++ ID=debian
++ HOME_URL=https://www.debian.org/
++ SUPPORT_URL=https://www.debian.org/support
++ BUG_REPORT_URL=https://bugs.debian.org/
+ cat
+ apt-get -q update
+ tail -1
+Reading package lists...
+ apt-cache madison tzdata
+    tzdata | 2026c-0+deb13u1 | http://deb.debian.org/debian trixie/main amd64 Packages
+    tzdata | 2026b-0+deb13u1 | http://snapshot.debian.org/archive/debian/20260831T204404Z trixie/main amd64 Packages
+ apt-get -s install --allow-downgrades tzdata=2026b-0+deb13u1
+ grep -E '^(Inst|Remv|Conf)|downgraded|newly|Err|E:'
+0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 77 not upgraded.
+Inst tzdata [2026c-0+deb13u1] (2026b-0+deb13u1 Debian:13.6/stable [all])
+Conf tzdata (2026b-0+deb13u1 Debian:13.6/stable [all])
@@ -0,0 +1,38 @@
+ PKG=tzdata
+ OLD=2026b-0+deb13u1
++ date +%s
+ S=1791116670
+ DEBIAN_FRONTEND=noninteractive
+ apt-get -y -q install --allow-downgrades tzdata=2026b-0+deb13u1
+ grep -E '^(Unpacking|Setting up)|downgraded'
+0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 77 not upgraded.
+Unpacking tzdata (2026b-0+deb13u1) over (2026c-0+deb13u1) ...
+Setting up tzdata (2026b-0+deb13u1) ...
+ apt-mark hold tzdata
+tzdata set on hold.
+ rm -f /etc/apt/sources.list.d/felhom-undo-snapshot.list
+ apt-get -q update
+ tail -1
+Reading package lists...
++ date +%s
+undo_seconds=4
+ echo undo_seconds=4
+ dpkg -s tzdata
+ grep -E '^(Status|Version)'
+Status: hold ok installed
+Version: 2026b-0+deb13u1
+ apt-mark showhold
+tzdata
+ systemctl is-active pveproxy pvedaemon pvestatd pve-cluster felhom-agent
+active
+active
+active
+active
+active
+ pct status 9201
+status: running
+ ls /etc/apt/sources.list.d/
+ceph.sources
+debian.sources
+pve-enterprise.sources
+pve-no-subscription.sources
@@ -0,0 +1,135 @@
+=== felhom-agent 0.141.1 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T14:24:41.910+02:00 level=INFO msg="osupdate: START" run=20261004T122441Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T122441Z
+time=2026-10-04T14:24:57.242+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T122441Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T14:24:57.242+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T14:24:57.242+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T14:24:57.242+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T14:24:57.243+02:00 level=INFO msg="osupdate: DONE" run=20261004T122441Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=0 reboot_needed=false wrapper_seconds=15.3
+time=2026-10-04T14:24:57.251+02:00 level=INFO msg="osupdate: START" run=20261004T122441Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T122441Z
+time=2026-10-04T14:25:12.127+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T122441Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T14:25:12.127+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T14:25:12.127+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T14:25:12.127+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T14:25:13.136+02:00 level=INFO msg="osupdate: DONE" run=20261004T122441Z layer=host vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=78 not_covered=78 restart_needed=0 reboot_needed=false wrapper_seconds=14.8
+  --- os-update report (guest) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T122441Z",
+    "restart_needed": null,
+    "ring": 0,
+    "run_id": "20261004T122441Z",
+    "upgraded": [],
+    "wrapper_seconds": 15.3
+  }
+  --- os-update report (host) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "proxmox-secure-boot-support",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "amd64-microcode",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "pve-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed"
+    ],
+    "outcome": "nothing",
+    "pending": 78,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T122441Z",
+    "restart_needed": [],
+    "ring": 0,
+    "run_id": "20261004T122441Z",
+    "upgraded": [],
+    "wrapper_seconds": 14.8
+  }
+  pass took 31.2s
+tzdata	2026b-0+deb13u1
@@ -0,0 +1,143 @@
+Canceled hold on tzdata.
+=== felhom-agent 0.141.1 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T14:25:22.024+02:00 level=INFO msg="osupdate: START" run=20261004T122522Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T122522Z
+time=2026-10-04T14:25:37.515+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T122522Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T14:25:37.515+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T14:25:37.515+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T14:25:37.515+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T14:25:37.516+02:00 level=INFO msg="osupdate: DONE" run=20261004T122522Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=0 reboot_needed=false wrapper_seconds=15.4
+time=2026-10-04T14:25:37.526+02:00 level=INFO msg="osupdate: START" run=20261004T122522Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T122522Z
+time=2026-10-04T14:25:57.778+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T122522Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T14:25:57.778+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T14:25:57.778+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=1 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T14:25:57.778+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=2.0 upgraded=1 restart-needed=- reboot-needed=no"
+time=2026-10-04T14:25:58.809+02:00 level=INFO msg="osupdate: DONE" run=20261004T122522Z layer=host vmid=9201 ring=0 trigger=debug outcome=applied healthy=true reason="" upgraded=1 pending=78 not_covered=78 restart_needed=0 reboot_needed=false wrapper_seconds=20.2
+  --- os-update report (guest) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T122522Z",
+    "restart_needed": null,
+    "ring": 0,
+    "run_id": "20261004T122522Z",
+    "upgraded": [],
+    "wrapper_seconds": 15.4
+  }
+  --- os-update report (host) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "proxmox-secure-boot-support",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "amd64-microcode",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "pve-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed"
+    ],
+    "outcome": "applied",
+    "pending": 78,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T122522Z",
+    "restart_needed": [],
+    "ring": 0,
+    "run_id": "20261004T122522Z",
+    "upgraded": [
+      {
+        "name": "tzdata",
+        "version": "2026c-0+deb13u1",
+        "origin": ""
+      }
+    ],
+    "wrapper_seconds": 20.2
+  }
+  pass took 36.9s
+tzdata	2026c-0+deb13u1
+0
@@ -0,0 +1,20 @@
+# felhom-os-apply (agent v0.141.0) red-proof, 2026-10-04: each mutation is applied to a COPY, syntax-checked, and the suite run against it.
+R1 (7 raise(s) removed): compiles=True suite rc=1; failing: test_R1_not_owned_by_the_agent, test_R1_path_outside_the_plan_dir, test_R1_symlink; expected test failed: YES
+R2 (2 raise(s) removed): compiles=True suite rc=1; failing: test_R2_non_debian_origin_in_the_plan, test_R2_non_debian_origin_in_the_simulation; expected test failed: YES
+R3 (1 raise(s) removed): compiles=True suite rc=1; failing: test_R3_slow_lane; expected test failed: YES
+R4 (1 raise(s) removed): compiles=True suite rc=1; failing: test_R4_removal; expected test failed: YES
+R5 (1 raise(s) removed): compiles=True suite rc=1; failing: test_R5_downgrade_exact; expected test failed: YES
+R6 (4 raise(s) removed): compiles=True suite rc=1; failing: test_R6_allow_new_is_slow_lane, test_R6_new_package, test_R6_unlisted_package; expected test failed: YES
+R7 (7 raise(s) removed): compiles=True suite rc=1; failing: test_R7_not_downloadable_and_no_snapshot, test_snapshot_does_not_have_it_either; expected test failed: YES
+R8 (1 raise(s) removed): compiles=True suite rc=1; failing: test_R8_free_space; expected test failed: YES
+R9 (2 raise(s) removed): compiles=True suite rc=1; failing: test_R9_apt_lock_held, test_R9_guest_locked_by_a_backup; expected test failed: YES
+R10 (4 raise(s) removed): compiles=True suite rc=1; failing: test_R10_bind_only_in_a_snapshot_section, test_R10_not_running, test_R10_not_the_boxs_own_guest, test_R10_reserved_vmid; expected test failed: YES
+R11 (12 raise(s) removed): compiles=True suite rc=1; failing: test_R11_bad_name, test_R11_bad_version_string, test_R11_duplicate; expected test failed: YES
+R12 (5 raise(s) removed): compiles=True suite rc=1; failing: test_R12_host_on_a_byo_box, test_R12_host_record_not_root_owned, test_R12_host_without_an_install_record, test_R12_unknown_layer; expected test failed: YES
+R13 (1 raise(s) removed): compiles=True suite rc=1; failing: test_R13_repair_does_not_fix_it; expected test failed: YES
+R14 (2 raise(s) removed): compiles=True suite rc=1; failing: test_R14_kernel_package_in_a_host_plan, test_R14_kernel_package_pulled_by_the_simulation; expected test failed: YES
+host layer runs in the guest instead: compiles=True suite rc=1; failing: test_host_runs_on_the_host_not_in_the_guest; expected test failed: YES
+pending-fast takes every pending upgrade: compiles=True suite rc=1; failing: test_pending_fast_skips_proxmox_docker_and_kernel; expected test failed: YES
+version checks in the target (pct exec per package): compiles=True suite rc=1; failing: test_no_per_package_guest_calls; expected test failed: YES
+lxc-start does not mean reboot: compiles=True suite rc=0; failing: NONE; expected test failed: NO
+lxc-start does not mean reboot (after adding test_reboot_needed_for_lxc_start_alone): suite rc=1; failing: test_reboot_needed_for_lxc_start_alone
@@ -0,0 +1,8 @@
+# Agent red-proof: the host_release wire tag (contract with the hub golden), 2026-10-04T11:04:25Z
+# mutation: report.go json:"host_release" -> json:"hostrelease"; restored after
+=== RUN   TestOSUpdateGolden_Decodes
+    osupdate_contract_test.go:34: host_release = <nil>
+--- FAIL: TestOSUpdateGolden_Decodes (0.00s)
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-agent/internal/hub	0.007s
+FAIL
@@ -0,0 +1,30 @@
+# Hub red-proofs, Parts B+C (hub v0.131.0), 2026-10-04T11:03:52Z
+# script: each mutation applied, the named test run with -v, file restored.
+[CAUGHT] host candidate leaves out kernel/boot/firmware: mutated internal/osupdates/service.go; TestCandidate_HostLeavesOutKernelBootFirmware ran=True failed=True rc=1
+    alarms_test.go:27: host candidate = map[firmware-realtek:{Name:firmware-realtek Version:20250410-2 Origin:Debian} grub-efi-amd64:{Name:grub-efi-amd64 Version:2.12-9 Origin:Debian} intel-microcode:{Name:intel-microcode Ve
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.034s
+[CAUGHT] host nights counted per layer: mutated internal/osupdates/service.go; TestLayers_SeparateSets ran=True failed=True rc=1
+    alarms_test.go:60: host approved on a GUEST night: &{ID:os-host-20261005-130000 Layer:host Fingerprint:535006df2f78491d ApprovedAt:2026-10-05 13:00:00 +0000 UTC ApprovedBy:auto PackagesJSON:[{"name":"libssl3t64","version
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.042s
+[CAUGHT] alarm 1 stale threshold: mutated internal/osupdates/service.go; TestAlarm_StaleLeg ran=True failed=True rc=1
+    alarms_test.go:151: no alarm at 8 days: []
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.041s
+[CAUGHT] alarm 1 switch OFF not watched: mutated internal/osupdates/service.go; TestAlarm_StaleLeg ran=True failed=True rc=1
+    alarms_test.go:168: alarmed with the switch OFF
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.054s
+[CAUGHT] alarm 2 reboot-needed counts from the FIRST report: mutated internal/osupdates/service.go; TestAlarm_RebootNeeded ran=True failed=True rc=1
+    alarms_test.go:211: reboot-needed since = 2026-10-17 12:00:00 +0000 UTC
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.045s
+[CAUGHT] alarm 3 counts only fast-lane pending: mutated internal/osupdates/service.go; TestAlarm_Ring0Stalled ran=True failed=True rc=1
+    alarms_test.go:235: a pending KERNEL counted as a stalled fast lane
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.040s
+[CAUGHT] alarm 4 ring 0 never alarms: mutated internal/osupdates/service.go; TestAlarm_NotCovered ran=True failed=True rc=1
+    alarms_test.go:278: not-covered alarms = ["OS updates (guest): cust1 has had 1 fast-lane package(s) no approved release covers since 2026-10-04: libsomething-hw. Ring 0 does not run them (other hardware?) — add matching 
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.060s
+[CAUGHT] alarm events are operator-only: mutated internal/notify/dispatcher.go; TestOSUpdateEvents_OperatorOnlyExceptApplied ran=True failed=True rc=1
+    os_alarms_operator_test.go:17: os_ring0_stalled is not operator-only
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/notify	0.009s
+[CAUGHT] host_release served to ring 1: mutated internal/osupdates/service.go; TestOSUpdate_DesiredBlockMatchesTheGolden ran=True failed=True rc=1
+    os_updates_test.go:59: served os_update diverged from the golden:
+    FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/api	0.055s
+after restore: ok  	gitea.dooplex.hu/admin/felhom-hub/internal/osupdates	0.544s | ok  	gitea.dooplex.hu/admin/felhom-hub/internal/notify	1.169s | ok  	gitea.dooplex.hu/admin/felhom-hub/internal/api	4.690s rc= 0
@@ -0,0 +1,44 @@
+import subprocess, sys, shutil
+M = [
+ ("host candidate leaves out kernel/boot/firmware", "internal/osupdates/service.go",
+  "if layer == LayerHost && hostSlowRE.MatchString(p.Name) {\n\t\t\t\tcontinue", "if false && hostSlowRE.MatchString(p.Name) {\n\t\t\t\tcontinue",
+  "./internal/osupdates/", "TestCandidate_HostLeavesOutKernelBootFirmware"),
+ ("host nights counted per layer", "internal/osupdates/service.go",
+  "reps, err := s.Store.OSReportsSince(h, layer, first)", "reps, err := s.Store.OSReportsSince(h, LayerGuest, first)",
+  "./internal/osupdates/", "TestLayers_SeparateSets"),
+ ("alarm 1 stale threshold", "internal/osupdates/service.go",
+  "now.Sub(ref) >= stale", "now.Sub(ref) >= 10*stale", "./internal/osupdates/", "TestAlarm_StaleLeg"),
+ ("alarm 1 switch OFF not watched", "internal/osupdates/service.go",
+  "holds := st.Enabled && canRun", "holds := canRun", "./internal/osupdates/", "TestAlarm_StaleLeg"),
+ ("alarm 2 reboot-needed counts from the FIRST report", "internal/osupdates/service.go",
+  "\t\tsince = rep.ReceivedAt\n", "\t\tif since.IsZero() {\n\t\t\tsince = rep.ReceivedAt\n\t\t}\n", "./internal/osupdates/", "TestAlarm_RebootNeeded"),
+ ("alarm 3 counts only fast-lane pending", "internal/osupdates/service.go",
+  "pendingAny += len(fastPending(layer, r.Pending))", "pendingAny += len(r.Pending)", "./internal/osupdates/", "TestAlarm_Ring0Stalled"),
+ ("alarm 4 ring 0 never alarms", "internal/osupdates/service.go",
+  "\tif ring == 0 {\n\t\treturn nil\n\t}\n", "", "./internal/osupdates/", "TestAlarm_NotCovered"),
+ ("alarm events are operator-only", "internal/notify/dispatcher.go",
+  '\t"os_ring0_stalled": true,\n', "", "./internal/notify/", "TestOSUpdateEvents_OperatorOnlyExceptApplied"),
+ ("host_release served to ring 1", "internal/osupdates/service.go",
+  "b.Release, b.HostRelease = s.releaseBlock(LayerGuest), s.releaseBlock(LayerHost)", "b.Release = s.releaseBlock(LayerGuest)",
+  "./internal/api/", "TestOSUpdate_DesiredBlockMatchesTheGolden"),
+]
+ok = True
+for name, f, old, new, pkg, test in M:
+    src = open(f).read()
+    assert src.count(old) == 1, (name, src.count(old))
+    shutil.copy(f, f + ".bak")
+    open(f, "w").write(src.replace(old, new))
+    r = subprocess.run(["go", "test", pkg, "-run", "^" + test + "$", "-v", "-count=1"], capture_output=True, text=True)
+    shutil.move(f + ".bak", f)
+    out = r.stdout + r.stderr
+    ran = ("=== RUN   " + test) in out
+    failed = ("--- FAIL: " + test) in out
+    verdict = "CAUGHT" if (ran and failed and r.returncode != 0) else "NOT CAUGHT"
+    if verdict != "CAUGHT": ok = False
+    print(f"[{verdict}] {name}: mutated {f}; {test} ran={ran} failed={failed} rc={r.returncode}")
+    for l in out.splitlines():
+        if "_test.go:" in l or l.startswith("FAIL\t") or "build failed" in l or l.startswith("#"):
+            print("    " + l.strip()[:220])
+r = subprocess.run(["go", "test", "./internal/osupdates/", "./internal/notify/", "./internal/api/", "-count=1"], capture_output=True, text=True)
+print("after restore:", r.stdout.strip().replace("\n", " | "), "rc=", r.returncode)
+sys.exit(0 if ok and r.returncode == 0 else 1)
@@ -0,0 +1,104 @@
+{
+    "boxes": [
+        {
+            "HostID": "demo-felhom-8363b5",
+            "Ring": 1,
+            "Enabled": true,
+            "Tunnel": "running",
+            "Guest": {
+                "ReleaseID": "os-guest-20261004-123933",
+                "LastOutcome": "nothing",
+                "LastAt": "2026-10-04T12:40:56Z",
+                "LastSuccessfulLeg": "2026-10-04T12:40:56Z",
+                "Pending": 6,
+                "NotCovered": 6,
+                "NotCoveredFast": 0,
+                "RestartNeeded": 0,
+                "RebootNeededSince": "2026-10-04T09:20:04Z",
+                "WrapperPassSeconds": 10.1
+            },
+            "Host": {
+                "ReleaseID": "os-host-20261004-124034",
+                "LastOutcome": "applied",
+                "LastAt": "2026-10-04T12:41:10Z",
+                "LastSuccessfulLeg": "2026-10-04T12:41:10Z",
+                "Pending": 80,
+                "NotCovered": 80,
+                "NotCoveredFast": 0,
+                "RestartNeeded": 34,
+                "RebootNeededSince": "2026-10-04T11:53:20Z",
+                "WrapperPassSeconds": 13.1
+            }
+        },
+        {
+            "HostID": "demo-hp-bb76ea",
+            "Ring": 0,
+            "Enabled": true,
+            "Tunnel": "unknown",
+            "Guest": {
+                "ReleaseID": "ring0-20261004T122522Z",
+                "LastOutcome": "nothing",
+                "LastAt": "2026-10-04T12:25:37Z",
+                "LastSuccessfulLeg": "2026-10-04T12:25:37Z",
+                "Pending": 6,
+                "NotCovered": 6,
+                "NotCoveredFast": 0,
+                "RestartNeeded": 0,
+                "RebootNeededSince": "2026-10-04T09:21:28Z",
+                "WrapperPassSeconds": 15.4
+            },
+            "Host": {
+                "ReleaseID": "ring0-20261004T122522Z",
+                "LastOutcome": "applied",
+                "LastAt": "2026-10-04T12:25:58Z",
+                "LastSuccessfulLeg": "2026-10-04T12:25:58Z",
+                "Pending": 78,
+                "NotCovered": 78,
+                "NotCoveredFast": 0,
+                "RestartNeeded": 0,
+                "RebootNeededSince": "0001-01-01T00:00:00Z",
+                "WrapperPassSeconds": 20.2
+            }
+        },
+        {
+            "HostID": "drill-r50-0a4f9a",
+            "Ring": 1,
+            "Enabled": true,
+            "Tunnel": "inactive",
+            "Guest": {
+                "ReleaseID": "",
+                "LastOutcome": "",
+                "LastAt": "0001-01-01T00:00:00Z",
+                "LastSuccessfulLeg": "0001-01-01T00:00:00Z",
+                "Pending": 0,
+                "NotCovered": 0,
+                "NotCoveredFast": 0,
+                "RestartNeeded": 0,
+                "RebootNeededSince": "0001-01-01T00:00:00Z",
+                "WrapperPassSeconds": 0
+            },
+            "Host": {
+                "ReleaseID": "",
+                "LastOutcome": "",
+                "LastAt": "0001-01-01T00:00:00Z",
+                "LastSuccessfulLeg": "0001-01-01T00:00:00Z",
+                "Pending": 0,
+                "NotCovered": 0,
+                "NotCoveredFast": 0,
+                "RestartNeeded": 0,
+                "RebootNeededSince": "0001-01-01T00:00:00Z",
+                "WrapperPassSeconds": 0
+            }
+        }
+    ],
+    "latest_guest_release": {
+        "approved_at": "2026-10-04T12:39:33Z",
+        "approved_by": "auto",
+        "id": "os-guest-20261004-123933"
+    },
+    "latest_host_release": {
+        "approved_at": "2026-10-04T12:40:34Z",
+        "approved_by": "auto",
+        "id": "os-host-20261004-124034"
+    }
+}
@@ -0,0 +1,172 @@
+=== felhom-agent 0.141.1 selftest=os-update vmid=9201 ring=0 enabled=true guest-release=false host-release=false appliance=true ===
+time=2026-10-04T13:52:57.534+02:00 level=INFO msg="osupdate: START" run=20261004T115257Z layer=guest vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T115257Z
+time=2026-10-04T13:53:09.479+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T115257Z layer=guest:9201 lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T13:53:09.479+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T13:53:09.479+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T13:53:09.479+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T13:53:09.480+02:00 level=INFO msg="osupdate: DONE" run=20261004T115257Z layer=guest vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=6 not_covered=6 restart_needed=0 reboot_needed=false wrapper_seconds=11.9
+time=2026-10-04T13:53:09.491+02:00 level=INFO msg="osupdate: START" run=20261004T115257Z layer=host vmid=9201 ring=0 trigger=debug enabled=true release=ring0-20261004T115257Z
+time=2026-10-04T13:53:19.927+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: START release=ring0-20261004T115257Z layer=host lane=fast mode=apply select=pending-fast packages=0"
+time=2026-10-04T13:53:19.927+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: REPAIR configured=0 fixed=0"
+time=2026-10-04T13:53:19.927+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: PLAN upgrade=0 already=0 not-installed=0 from-snapshot=0"
+time=2026-10-04T13:53:19.927+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
+time=2026-10-04T13:53:20.705+02:00 level=INFO msg="osupdate: DONE" run=20261004T115257Z layer=host vmid=9201 ring=0 trigger=debug outcome=nothing healthy=true reason="" upgraded=0 pending=80 not_covered=80 restart_needed=34 reboot_needed=true wrapper_seconds=10.4
+  --- os-update report (guest) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "docker-ce-cli",
+      "containerd.io",
+      "docker-ce",
+      "docker-buildx-plugin",
+      "docker-ce-rootless-extras",
+      "docker-compose-plugin"
+    ],
+    "outcome": "nothing",
+    "pending": 6,
+    "reboot_needed": false,
+    "refused": null,
+    "release_id": "ring0-20261004T115257Z",
+    "restart_needed": null,
+    "ring": 0,
+    "run_id": "20261004T115257Z",
+    "upgraded": [],
+    "wrapper_seconds": 11.9
+  }
+  --- os-update report (host) ---
+  {
+    "health_reason": "",
+    "healthy": true,
+    "mode": "apply",
+    "not_covered": [
+      "frr",
+      "shim-signed-common",
+      "shim-unsigned",
+      "shim-helpers-amd64-signed",
+      "shim-signed",
+      "libradosstriper1",
+      "librgw2",
+      "ceph-common",
+      "librbd1",
+      "librados2",
+      "python3-cephfs",
+      "libcephfs2",
+      "python3-rgw",
+      "python3-rados",
+      "python3-ceph-argparse",
+      "python3-ceph-common",
+      "python3-rbd",
+      "ceph-fuse",
+      "chrony",
+      "libcorosync-common4",
+      "libcfg7",
+      "libcmap4",
+      "libcpg4",
+      "libknet1t64",
+      "libnozzle1t64",
+      "libquorum5",
+      "libvotequorum8",
+      "corosync",
+      "frr-pythontools",
+      "libjs-extjs",
+      "libnvpair3linux",
+      "libproxmox-acme-plugins",
+      "libproxmox-backup-qemu0",
+      "pve-qemu-kvm",
+      "libpve-notify-perl",
+      "libpve-cluster-api-perl",
+      "libpve-cluster-perl",
+      "pve-cluster",
+      "libpve-access-control",
+      "libpve-apiclient-perl",
+      "librados2-perl",
+      "proxmox-backup-client",
+      "proxmox-backup-file-restore",
+      "pve-manager",
+      "libproxmox-acme-perl",
+      "libpve-common-perl",
+      "libpve-guest-common-perl",
+      "qemu-server",
+      "libpve-storage-perl",
+      "pve-edk2-firmware-legacy",
+      "pve-edk2-firmware-ovmf",
+      "libpve-network-api-perl",
+      "libpve-network-perl",
+      "proxmox-firewall-data",
+      "pve-firewall",
+      "pve-container",
+      "pve-ha-manager",
+      "novnc-pve",
+      "proxmox-enterprise-support-keyring",
+      "proxmox-mini-journalreader",
+      "proxmox-widget-toolkit",
+      "pve-docs",
+      "pve-i18n",
+      "pve-xtermjs",
+      "pve-yew-mobile-i18n",
+      "pve-yew-mobile-gui",
+      "libuutil3linux",
+      "libzfs7linux",
+      "libzpool7linux",
+      "proxmox-first-boot",
+      "pve-firmware",
+      "proxmox-kernel-7.0.14-20-pve-signed",
+      "proxmox-kernel-7.0",
+      "proxmox-kernel-helper",
+      "pve-edk2-firmware-aarch64",
+      "pve-edk2-firmware",
+      "zfs-initramfs",
+      "zfsutils-linux",
+      "zfs-zed",
+      "tailscale"
+    ],
+    "outcome": "nothing",
+    "pending": 80,
+    "reboot_needed": true,
+    "refused": null,
+    "release_id": "ring0-20261004T115257Z",
+    "restart_needed": [
+      "agetty",
+      "blkmapd",
+      "chronyd",
+      "cron",
+      "dbus-daemon",
+      "dmeventd",
+      "ksmtuned",
+      "lxc-monitord",
+      "lxc-start",
+      "lxcfs",
+      "pmxcfs",
+      "proxmox-firewal",
+      "pve-firewall",
+      "pve-ha-crm",
+      "pve-ha-lrm",
+      "pve-lxc-syscall",
+      "pvedaemon",
+      "pvedaemon worke",
+      "pvefw-logger",
+      "pveproxy",
+      "pveproxy worker",
+      "pvescheduler",
+      "pvestatd",
+      "qmeventd",
+      "rpcbind",
+      "rrdcached",
+      "smartd",
+      "spiceproxy",
+      "spiceproxy work",
+      "sshd",
+      "systemd-logind",
+      "systemd-udevd",
+      "watchdog-mux",
+      "zed"
+    ],
+    "ring": 0,
+    "run_id": "20261004T115257Z",
+    "upgraded": [],
+    "wrapper_seconds": 10.4
+  }
+  pass took 23.2s
+WALL_SECONDS=23.268350584
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
admin	877e89ee2a	manifests: hub 0.133.0 gates / gates (push) Successful in 36s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 20:19:40 +02:00
admin	ff1db11db4	hub v0.133.0 (R-859 test approvals end with the test; R-840 bundle on the System page, manifest, alarm); installer 1.31.0 (root files from the config bundle); bundle bootstrap script; golden 0.293.0 evidence; rulings 96–99; drill-r50 removed (evidence) gates / gates (push) Successful in 29s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 20:18:37 +02:00
admin	6b820143f8	R-858 closed (agent v0.142.1, ruling 95): the Docker step restarts the socket users; incident evidence; STATUS; report addendum gates / gates (push) Successful in 34s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 18:35:40 +02:00
admin	242f0a15d5	R-858 filed (P2): a Docker engine step leaves the controller and traefik on the old docker socket; demo-felhom repaired by restarting the two containers; incident evidence gates / gates (push) Successful in 28s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 17:57:54 +02:00
admin	9291ae255a	REPORT-os-docker-crash-2026-10-04: CI line gates / gates (push) Successful in 32s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 17:51:09 +02:00
admin	21986d03b5	golden 0.292.0 re-vouched (re-bake: live-restore + approved Docker set) with agent 0.142.0; R-857 filed; STATUS; REPORT-os-docker-crash-2026-10-04 gates / gates (push) Successful in 32s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 17:49:30 +02:00
admin	208d21d18f	golden 0.292.0 re-bake evidence: build-golden.sh 3.1.0, pinned Docker set, live-restore on gates / gates (push) Successful in 37s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 17:39:28 +02:00
admin	c9052264f7	manifests: publish installer 1.30.0 (both git-sync refs) gates / gates (push) Successful in 34s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 17:36:21 +02:00
admin	0c55336fba	docs: 11 §5.7 System page, §5.8 Docker slow lane BUILT, §5.9 crash restart; 00/03/07/08; decisions 90-94 (CC unattended); runbooks docker-undo + crash-guard; register R-852 R-835 R-848 R-849 R-851 R-854 closed, R-853 R-855 R-856 opened, R-812 R-840 narrowed (334 -> 332); live evidence gates / gates (push) Successful in 33s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 17:36:10 +02:00
admin	245886ee92	manifests: revert the TEST-ONLY Docker approval wait (ruled 2 nights) gates / gates (push) Successful in 30s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 16:28:52 +02:00
admin	30cfdc1dc5	manifests: TEST-ONLY Docker approval wait 0 nights (os-docker-crash Part B; reverted in the same session) gates / gates (push) Successful in 31s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 16:27:19 +02:00
admin	3de64b138d	manifests: hub 0.132.0 gates / gates (push) Successful in 30s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 16:17:23 +02:00
admin	b1b5c7e7e1	installer 1.30.0 (not yet tagged): the crash guard units + config, the root-owned slow-lane trust files (os-trust.json, operator-signers), their removal on uninstall gates / gates (push) Successful in 28s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 16:16:24 +02:00
admin	175ecfcdd2	hub v0.132.0: the System page (versions + OS updates with the ring/switch/approve buttons, R-852), the Hosts Proxmox/kernel column, the operator-approved Docker engine release (2 healthy ring-0 nights), the crash-guard events (R-851); evidence audits/os-docker-crash-2026-10-04 gates / gates (push) Successful in 30s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 16:16:02 +02:00
admin	bee277ffad	rulings 87-89 recorded before the work (live-restore ON; crash restart with a limit; versions visible in the hub); R-852 filed (versions shown nowhere) gates / gates (push) Successful in 30s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 15:26:41 +02:00
admin	5efe6daec3	golden 0.292.0 vouched with agent 0.141.1; golden waiver deleted (newest controller has its golden); REPORT-os-host-lane-2026-10-04 with the Part table; STATUS/CONTEXT gates / gates (push) Successful in 33s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 14:53:50 +02:00
admin	f93738dc49	golden 0.292.0: bake + publish evidence (RUNBOOK-manual-build §4.0/§4.1 steps 1-4; not vouched, no floor change) gates / gates (push) Successful in 36s Details GOLDEN_SHA256=d6cf8b33ad58e5cfbf9566558044b2e1df692ef29d5353a40e39117794d16671, round trip MATCH; drill VM 9100 destroyed, VM reverted to virgin. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 14:51:35 +02:00
admin	31bdb4b549	docs: OS updates steps 3+4 BUILT (11 §8.2/§8.3, §5.6 kernel facts, §5.8 Docker slow lane design), 00/03/07/08 updated, decisions 84-86 (CC unattended), host undo runbook (proved), register: R-841 R-845 R-846 R-850 closed, R-848 R-849 R-851 opened, R-836 R-812 narrowed (332 -> 333); STATUS; live evidence gates / gates (push) Successful in 31s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 14:47:21 +02:00
admin	0e970ba384	manifests: revert the TEST-ONLY OS approval wait (ruled 24h + 1 night) gates / gates (push) Successful in 31s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 14:41:35 +02:00
admin	a73f728a55	manifests: TEST-ONLY OS approval wait 2m / 0 nights (os-host-lane Part B ring-1 proof; reverted in the same session) gates / gates (push) Successful in 28s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 14:38:17 +02:00
admin	771e12445c	manifests: hub 0.131.1 gates / gates (push) Successful in 30s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 13:40:46 +02:00
admin	fd1f98547e	hub v0.131.1: a scanned host pass (reboot_scanned, agent 0.141.1) clears 'reboot needed', so the 14-day alarm does not fire after a reboot; live evidence partB/partD/partG gates / gates (push) Successful in 30s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 13:39:48 +02:00
admin	693f3b45aa	manifests: hub 0.131.0 gates / gates (push) Successful in 31s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 13:05:59 +02:00
admin	88b0a2e761	hub v0.131.0: the tunnel status is true (R-841: three states, tunnel_down after two not_running reports, unknown never alarms); OS updates per layer (guest/host separate approved sets, host candidate leaves out kernel/boot/firmware, host_release in the box block), the fleet view and four hourly operator alarms (11 §8 steps 3+4); red-proofs in audits/os-host-lane-2026-10-04 gates / gates (push) Successful in 30s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 13:05:00 +02:00
admin	0ed2e8de4c	docs: rulings 2026-10-04 ~12:20 (decisions 81-83); R-842 closed (A); R-840 ruling recorded gates / gates (push) Successful in 28s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 12:24:07 +02:00
admin	e78726314e	OS updates guest fast lane: records — 11 §8.1 BUILT, 03 cloudflared corrected, 07 §6.1 OS leg, 00 PARTIAL, monthly runbook infra pins, golden 0.291.0 record + vouch, register 331 -> 333 (R-837/838/726/843 closed; R-840/841/842/844/845 opened), STATUS, report gates / gates (push) Successful in 31s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 11:41:50 +02:00
admin	e55b2ceb63	manifests: serve the installer from installer-v1.29.0 gates / gates (push) Successful in 29s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 11:40:40 +02:00
admin	eaae94a4b8	felhom-host-install.sh 1.29.0: installs the OS-update wrapper (felhom-os-apply) from the pinned agent tag; skipped with a warning on an agent older than 0.140.0; uninstall removes it gates / gates (push) Successful in 38s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 11:40:37 +02:00
admin	0e22a0ecc7	manifests: revert the TEST-ONLY OS approval wait (ruled 24h + 1 night) gates / gates (push) Successful in 29s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 11:14:42 +02:00
admin	44992360a4	manifests: TEST-ONLY OS approval wait 2m / 0 nights (Part G 2; reverted in the same session) gates / gates (push) Successful in 29s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 11:12:04 +02:00
admin	66b7540b7a	manifests: hub 0.130.0 gates / gates (push) Successful in 30s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-10-04 10:58:53 +02:00
				`@@ -0,0 +1 @@`
				{"release_id":"os-docker-20261004-142842","undo":false,"packages":[{"name":"containerd.io","version":"2.3.6-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-buildx-plugin","version":"0.37.1-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce","version":"5:29.8.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce-cli","version":"5:29.8.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce-rootless-extras","version":"5:29.8.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-compose-plugin","version":"5.6.0-1~debian.13~trixie","origin":"Docker CE"}]}
				`@@ -0,0 +1 @@`
				{"release_id":"undo-to-29.7.2","undo":true,"packages":[{"name":"containerd.io","version":"2.3.3-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-buildx-plugin","version":"0.36.1-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce","version":"5:29.7.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce-cli","version":"5:29.7.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-ce-rootless-extras","version":"5:29.7.2-1~debian.13~trixie","origin":"Docker CE"},{"name":"docker-compose-plugin","version":"5.5.0-1~debian.13~trixie","origin":"Docker CE"}]}
				`@@ -0,0 +1 @@`
				`2026/10/04 17:47:51 [WARN] host demo-hp-bb76ea crash guard: host_crash_guard_rearmed`