R-840/R-859/R-860 records: 11 §5.3.1 + §5.4.2, 03, 04, 00; runbooks config-bundle + os-updates-test-waits; register 333→334 (R-840/859/860 closed, R-861/862 opened); STATUS; report; evidence

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS

documentation/runbooks/config-bundle.md

@@ -0,0 +1,69 @@
+# RUNBOOK — the config bundle: a box's root-owned files by a signed job (R-840)
+
+Design: `architecture/11-os-updates.md` §5.4.2. Decision 96 (`09` §3). Built 2026-10-04: agent v0.143.0, hub v0.133.0,
+installer 1.31.0. Evidence: `audits/r840-config-bundle-2026-10-04/`.
+
+## What it is
+
+Every root-owned file the installer writes for the agent (both sudoers files, the five wrappers, the crash guard and
+its units, the agent and rollback units, the start-limit drop-in, the mgmt watchdog, the OOB belt's files) travels as
+ONE file, `felhom-config-bundle.json`, published beside the agent binary (`felhom-agent/<version>/`). A new box gets it
+from the installer; an installed box gets it by a signed `agent_config_update`. The box's own root-owned
+`felhom-os-apply` checks the signature, the sha, every path and every file before it writes anything, keeps the
+previous copies, self-checks, and puts everything back on a failure.
+
+**The trust root is never in a bundle.** `/etc/felhom/operator-signers` and `/etc/felhom/os-trust.json` decide who may
+sign; no bundle may add, remove or change them. A box that has no signers file gets exactly the installer's pinned key
+(`felhom-op-1`), and only through a job that key signed. Signer rotation is a separate act (`04` §3).
+
+## Send a bundle to a box (CC or the operator, on DooPlex)
+
+1. The bundle's sha: the release output (`release-agent.sh` prints `bundle :`), or the hub's Configuration page after
+   vouching (the hub resolves it from the registry by exact name).
+2. Sign and queue (key `felhom-op-1`; the hub key is Secret `felhom-system/report-api`, read into a file, never printed):
+   ```bash
+   felhom-opsign -op agent_config_update -host <host_id> -key-id felhom-op-1 -key <operational key> \
+     -agent-version <X.Y.Z> -bundle-sha256 <sha> -ttl 45m -upload http://<hub ClusterIP>:8080 -hub-key "$(cat <file>)"
+   ```
+3. The agent takes it on its next job poll (measured 4–15 min). Positive observables in the box's journal:
+   `os-apply: BUNDLE DONE agent=<X> written=<n> … self-check=ok`, then `capability probe after the config bundle ok=71
+   total=71`. The hub's System page "Root files" column shows the version.
+4. **Undo** = send the previous release's bundle the same way. The previous copies also stay on the box in
+   `/var/lib/felhom-os-apply/bundle-prev/<time>-before-<version>/` (the last 3).
+
+## A box from before agent v0.143.0 — the ONE by-hand step (bootstrap)
+
+Such a box's `felhom-os-apply` has no bundle mode, and no signed job can write a root file there (that gap IS
+R-840). So once per box, as root, install the bundle-aware `felhom-os-apply` — and nothing else:
+
+```bash
+curl -fsSL -o /root/felhom-bundle-bootstrap.sh https://felhom.eu/scripts/felhom-bundle-bootstrap.sh
+sha256sum /root/felhom-bundle-bootstrap.sh     # f4a455fad9b1acac15c0a73a349d6f5d4353f734a3c426d8377d2e632762c548 (installer 1.31.0)
+bash /root/felhom-bundle-bootstrap.sh 0.143.0 8d7273cf5313ef62b867cb6f831c631923a436452d6f90b8ff7f0771170396ba
+```
+
+It must end with `DONE. This box can now take signed config bundles. Nothing else was changed.` It stops with
+`STOP:` and changes nothing on a wrong sha. Then send the bundle by the signed job (above). Done on demo-hp and
+demo-felhom on 2026-10-04 (`audits/r840-config-bundle-2026-10-04/partB/b2`, `b3`).
+
+### Tester 2 (`Tester-2-be8404`) — the operator's steps (decision 97)
+
+CC has no route to Tester 2: its door (`felhom-sshd` on 8822) admits only the operator's WireGuard peer, and the
+`felhom-op` user cannot become root by itself.
+
+1. Connect your WireGuard operator tunnel (`operations/nodes.md`, "OOB belt").
+2. `ssh -p 8822 felhom-op@10.77.0.5`
+3. Hub → Hosts → `Tester-2-be8404` → Console access → **Reveal** the root password (the hub writes one line on
+   Tester 2's own timeline: "the operator requested your console password" — that is by design).
+4. `su -` with that password, then the three commands above.
+5. Tell CC "bootstrap done". CC sends the bundle by the signed job and reads it back on the System page.
+
+## Release side
+
+`scripts/release-agent.sh` builds and publishes the bundle with every release (reproducible: same source → same sha)
+and verifies it by download. Vouching an agent version stores its bundle sha; the install manifest serves it to the
+installer. A box behind the vouched bundle for 7 days raises `os_config_bundle_behind` (operator mail;
+`OS_ALARM_BUNDLE_BEHIND_AFTER`).
+
+**Adding a root-owned file:** add it to `BUNDLE_FILES` in `felhom-agent/configs/felhom-os-apply` — never a new
+installer fetch. `test_every_root_file_the_installer_writes_is_in_the_bundle` fails on a path the bundle lacks.