audits: SPIKE controller-driven escrow ceremony — daemon-context mechanics ALL GO (drill VM, agent 0.87.0): PTY re-key no-TTY 3/3, sudoers refusal matrix 5/5, R pipe round-trip proven, env_reset clean, --upload hub-verified ~2.4s; premise correction: capability probes are list-mode

This commit is contained in:
2026-07-13 17:15:52 +02:00
parent fa0ecb2cd7
commit ac305fe57f
2 changed files with 253 additions and 37 deletions
+27 -37
View File
@@ -2,42 +2,32 @@
> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md).
## RUNBOOK-peti-return — Phase 0a/1 executed, **STOPPED at Gate P1 for a ruling** — 2026-07-13
## SPIKE — controller-driven escrow ceremony (daemon-context invocation mechanics) — 2026-07-13
Operational run, read-only throughout (hub API with the bearer + a `hub.db` snapshot queried on
180 and deleted after). Nothing signed, pushed, or changed on any box or in the hub. Full
evidence: `documentation/pilot/RUNBOOK-peti-return-2026-07-13.md` §EXECUTION RECORD.
Docs-only commit; findings at `documentation/audits/SPIKE-controller-escrow-2026-07-13.md`.
No production code written; agent/controller repos untouched. **All five mechanisms under test: GO**
— the production spec (local-API ceremony endpoint + `--output=json` + job/one-shot-R-claim) can be
written on observed behavior.
### Gate P1 FAILS — controller 0.115.0 is NOT converging to the 0.122.0 floor
- The hub **is serving** the floor (DB floor 0.122.0; agent 0.81.0 ≥ MinAgent 0.81.0; no HELD
lines in live hub logs). The stall is box-side.
- Self-update worked until 07-11 13:38 (0.110→0.112→0.113→0.115), then the **controller→agent
:8443 channel died at 07-11 14:01** ("no route to host 192.168.1.170:8443") and is **still
refusing today** (fresh `agent_channel_unreachable` 16:15 local, hours after host-reports
resumed). The floor auto-update delegates the container swap to the agent over that channel;
a failed attempt is persisted **once-per-floor** (anti-flapping) and never retried.
- **Root cause below it: the cluster split the runbook itself declares a STOP.** The agent's node
("proxmox", 128 GB RAM) rebooted ~3.5 h ago and reports **0 guests**; the guest's controller
reports 4.2 days uptime on an i5-2500/8 GB — **the felhom guest is not on the node the agent is
enrolled on** (proxmox1/proxmox2 shape; single host row verified).
### Phase 0a record (the notification-pipeline first-real-customer answer)
- Claim code: issued gen 1 + **emailed 2026-07-12 16:49:09Z**, NOT yet claimed.
- **FINDING P1-F1: the motioneye 100 % warning NEVER emailed Peti** — all 16 `storage_fill_critical`
notifications went operator-channel only; no `customer_notifications` row exists pre-claim.
Follow-up task: define intended pre-claim customer-notification behavior.
- Collateral: P1-F2 agent local vzdump of 9201 fails every cycle (`not a UPID: "OK"` — cross-node
vmid); P1-F3 `wg-handshake-read` DEGRADED (wg tools missing on his node); P1-F4 the MinAgent
conditional floor is served from last-known agent_version with no freshness check (floor was
served while his agent was 40 h dead, guaranteeing the failed swap). motioneye VG: 1.2 TB at
99.66 % (4.09 GB free).
### Ruling needed (all queued in the runbook doc)
1. Cluster split: migrate guest back / move the agent / accelerate agent-follows-guest.
2. Post-heal convergence path: floor bump (0.125.0 earmarked), manual trigger, or state clear.
3. Optional: hub controller-log pull to confirm the stall variant (CC did not trigger — write op).
4. Phase 0b/0c (message Peti, call slot) — Viktor, unaffected. Phase 4 rotations independent —
can proceed on GO.
- **Target:** the drill VM (qm 300, `192.168.0.152`, take-two end state, agent **0.87.0**); full
probe plan P0P8 ran, nothing skipped; demo host + live demo escrow untouched (hub row verified).
- **SQ1 (load-bearing): the PTY-driven PBS re-key works with NO controlling terminal** — 3/3
ceremonies exit 0 under `systemd-run --uid=felhom-agent` + `sudo -n` (the daemon shape), ~2.3 s
each, self-verify included.
- **Refusal matrix 5/5:** the fixed-argv sudoers line refused every altered argv (value change,
extra flag, reorder, `--config` omitted, alternate config path) — direct self-invocation is
sound; no guarded wrapper needed. Caller must pin argv order + `--` spelling byte-exact.
- **R pipe-capture round-trip PROVEN:** R parsed from the captured stdout opened the blob via
`escrow-consume` (fingerprint-gated, recovered key = live key size). Banner parsing is brittle →
`--output=json` is mandatory for production.
- **env_reset clean** (pinned `--config`; WG key auto-captured; hub upload authenticated);
**`--upload` hub-verified** (`host_escrow` row timestamped to the second, 383 B blob, empty
`restic_pw_sha256` — no staged secret existed, as gated in P0).
- **Timings:** 2.282.37 s incl. upload → job pattern with 2 s poll / 60 s timeout recommended;
ceremony process peaks ~264 MiB.
- **Correction to the task premise:** capability probes are LIST-mode (`sudo -n -l`), never
executing (manifest.go, verified live) — the escrow grant can get a normal probe entry.
- **Hygiene:** zero R fragments in journal/auth logs/state dir; all probe artifacts removed
(sudoers drop-in gone + post-removal refusal re-proven; scratch shredded; VM left running).
- **Recorded side effect (deliberate, drill scratch):** the drill box's take-two escrow blob is
superseded by a hash-less one; the take-two paper R no longer opens the CURRENT blob.