docs: OS updates steps 3+4 BUILT (11 §8.2/§8.3, §5.6 kernel facts, §5.8 Docker slow lane design), 00/03/07/08 updated, decisions 84-86 (CC unattended), host undo runbook (proved), register: R-841 R-845 R-846 R-850 closed, R-848 R-849 R-851 opened, R-836 R-812 narrowed (332 -> 333); STATUS; live evidence

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS

documentation/runbooks/os-updates-host-undo.md

@@ -0,0 +1,81 @@
+# Put one host package back by hand (OS updates, host fast lane)
+
+> **When:** the hub mailed `os_update_health_failed` for the **host** layer (the box's base system), or the operator
+> sees a host problem that started with a host OS pass, and ONE package is the suspect.
+> **Who:** the operator, or CC with the operator's word, as root on the box's Proxmox host (`ssh <host>`).
+> **Owner design:** `architecture/11-os-updates.md` §5.6 (host row), §8 step 3. **Proved** on demo-hp 2026-10-04 with
+> `tzdata` (2026c → 2026b, held; then released and re-installed by the next ring-0 pass) and again on demo-felhom for
+> the ring-1 test — evidence `audits/os-host-lane-2026-10-04/partB/undo/` and `partB/ring1/`.
+
+There is **no automatic undo** for the host. A host cannot be snapshotted the way the old design assumed, and the
+fast lane only ever installs Debian / Debian-Security packages, never a kernel, boot or firmware package (wrapper
+refusals R14, R12). So the undo is small: install the previous version of the suspect package from
+`snapshot.debian.org`, which keeps every version Debian ever published.
+
+## 1. Find the suspect and its previous version
+
+```bash
+# the host pass's own apt run (Requested-By: felhom-agent); each line "name:arch (old, new)"
+grep -B2 -A6 "Requested-By: felhom-agent" /var/log/apt/history.log | tail -20
+PKG=<name>; OLD=<old version from that line>
+```
+
+## 2. Pick the snapshot timestamp
+
+- **Normal case:** the timestamp of the **previous host release** — the hub fleet view (`GET /os/fleet`, the box's
+  host line names its release; the release's approval time IS its snapshot time, `YYYYMMDDTHHMMSSZ`). At that time
+  the old version was the current one.
+- **No previous host release** (a ring-0 box, or the first host pass): ask snapshot.debian.org when the **NEW**
+  (installed) version first appeared, and use that time. At that moment the suite still carried the old version.
+  **Do not use the OLD version's `first_seen`:** that is when it reached Debian *unstable*, and `trixie` may not
+  have had it yet — measured on demo-hp 2026-10-04: `eject 2.41-5` at its own `first_seen` → `Version not found`.
+
+```bash
+NEW=$(dpkg-query -W -f='${Version}' "$PKG")
+curl -s "https://snapshot.debian.org/mr/binary/$PKG/$NEW/binfiles?fileinfo=1" | python3 -c '
+import json,sys; d=json.load(sys.stdin)
+print(sorted(f["first_seen"] for v in d["fileinfo"].values() for f in v)[0])'
+# → e.g. 20260831T204404Z
+TS=<that timestamp>
+```
+
+## 3. Install the old version from the snapshot, then remove the snapshot source
+
+```bash
+. /etc/os-release
+cat > /etc/apt/sources.list.d/felhom-undo-snapshot.list <<EOF
+deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/$TS $VERSION_CODENAME main
+deb [check-valid-until=no] http://snapshot.debian.org/archive/debian-security/$TS $VERSION_CODENAME-security main
+EOF
+apt-get -q update
+apt-get -s install --allow-downgrades "$PKG=$OLD"     # READ the simulation: only $PKG may change. If apt
+                                                       # refuses, the package pins its siblings to the SAME
+                                                       # version (measured: eject needs libmount1 = 2.41-5) —
+                                                       # list them all in one command, or stop.
+apt-get -y install --allow-downgrades "$PKG=$OLD"
+apt-mark hold "$PKG"                                   # or the next ring-0 night installs the new one again
+rm -f /etc/apt/sources.list.d/felhom-undo-snapshot.list
+apt-get -q update
+dpkg -s "$PKG" | grep -E "^(Status|Version)"
+```
+
+**The hold is the important line.** Without it the next night pass (ring 0) or the next approved release (ring 1)
+installs the new version again. Write the hold into the register row that tracks the incident; take it off
+(`apt-mark unhold "$PKG"`) when a fixed version is out.
+
+## 4. Check the box
+
+- `systemctl is-active pveproxy pvedaemon pvestatd pve-cluster felhom-agent` → all `active`.
+- `pct status <customer vmid>` → `running`.
+- The host health rule (`11` §8.2) is what the leg checks; run the debug action to see it pass:
+  `sudo -u felhom-agent /usr/local/bin/felhom-agent --config /etc/felhom-agent/agent.json --selftest=os-update -vmid <vmid>`
+  — **note: on ring 0 this also installs every pending fast-lane fix.** A held package is NOT reported as pending
+  at all (measured on demo-hp: `pending` stayed 78 with `tzdata` held) — **the hub cannot see a hold**, so the hold
+  lives only in the register row (R-848).
+
+## What this runbook does NOT cover
+
+- A kernel: the fast lane never installs one (R14). The kernel lane is `11` §5.6 (not built).
+- A Proxmox-origin package (pve-*, qemu-server, …): never installed by the fast lane (R12/origin rule). Proxmox's
+  own repository keeps old versions; that undo is not written yet.
+- The customer guest: its undo is last night's whole-guest backup (decision 81).