spike(pin-mismatch): §9 Outcome — R1 executed, controller↔agent service restored

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
This commit is contained in:
2026-06-29 15:08:58 +02:00
parent 8254b8823e
commit 1777c89fb9
@@ -7,6 +7,8 @@
agent migration on **2026-06-28 ~08:56 CEST**. The controller's pin (and `bootstrap.json` seed)
still reference the **pre-migration** leaf. A secondary blocker (empty token store) is latent
behind the pin and will surface the moment the pin is fixed.
- **RESOLVED 2026-06-29 via R1** (RUNBOOK, supervised) — service restored; see the closing
**§9 Outcome** at the bottom.
---
@@ -233,3 +235,40 @@ incident.
*No secrets recorded: the per-guest bearer token is referenced as "stored out-of-band"; the leaf
fingerprint is not a secret.*
---
## 9. Outcome — R1 executed 2026-06-29 (supervised), service RESTORED
Ran the R1 RUNBOOK (restore the pre-migration leaf **+** token store from
`/root/agent-backup-bundle-test/aside-var-lib/``/var/lib/felhom-agent/`, chown
`felhom-agent`, modes 644/600/600, agent restart). Current post-migration state was first backed
up to `/root/agent-rollback-20260629-150137/` (rollback source, retained).
**Gates — all passed:**
- **Gate 1 (loaded, not regenerated):** boot log `local-api leaf ready fingerprint_sha256=60b5974d…`
at 15:02:10 — the agent **loaded** the restored leaf (ownership/mode correct; no third cert minted).
- **Gate 2 (wire):** single `felhom-agent` listener on `192.168.0.162:8443`; on-wire leaf fp
`60b5974d…` == controller `PIN_FP`.
- **Gate 3 (real pinned+auth path):** authenticated `GET /disks` with the controller's
`local_api.token`**HTTP 200, ok=true, 5 disks**; a bogus token → **401** (auth genuinely
enforced). Token/hashes never left the host.
- **Controller corroboration:** the first quiesce cycle after the restart (15:05:33 CEST)
succeeded end-to-end — *"backup due — quiescing 5 stack(s) … backup job … started … snapshotted
— resuming app early … unquiescing: restarting 5 stack(s)"*. **No** `TLS pin mismatch` / 401
lines after 15:02. The disk UI path and the quiesce app-consistent-backup loop are both restored.
**Restored values:** agent now serves leaf `60b5974d…` (matches the in-guest pin and
`bootstrap.json` seed) and re-indexed the pre-migration token store (effective 9201 token = the
controller's current `local_api.token`). **Zero in-guest change** was needed.
**Retained (do not delete yet):** `/root/agent-backup-bundle-test/aside-var-lib/` (only copy of the
pinning material until the future-proofing fix lands) and `/root/agent-rollback-20260629-150137/`
(R1 rollback point).
**Still open (carried forward, NOT addressed by R1):**
- The root→non-root agent migration **regenerates the local-API leaf** (and truncates the token
store) because it does not carry `local-api.{crt,key}` + `local-tokens.log` into the new
`felhom-agent`-owned data dir. Until the migration/Day-0 path preserves these (or treats their
loss as a re-bootstrap trigger), any re-migration/host-reprovision will reintroduce this exact
mismatch. This restore reinstated the **old** leaf; it did not change the migration behavior.