admin
72d7f05bf1
Parts 1-2 shipped + deployed (v0.69.0); Part 3 not needed (sqlite3 export); Part 4-A re-attach safety already unit-proven; Part 4-B destructive 9201 drill prepared + operator-gated (pre-flight green), not executed by CC. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
74 lines
4.9 KiB
Markdown
74 lines
4.9 KiB
Markdown
# REPORT — S5: host-loss DR (recovered WG-key install + directive→PLAN; safe halves shipped)
|
|
|
|
**Date:** 2026-07-04 · **Class:** implementation (safe halves) + diagnosis + a prepared,
|
|
operator-present destructive drill (NOT executed by CC). **Design of record:** doc-06 §3.5 + S5 row;
|
|
doc-01 §9.
|
|
|
|
## 1. Baselines & commits (felhom-agent → `main`)
|
|
|
|
Start: `567cf9f` (v0.68.0). **`bd4bced`** v0.69.0 — the two safe DR halves (Parts 1-2). This commit
|
|
— REPORT + CONTEXT + doc-06 S5 row.
|
|
|
|
## 2. Part 1 (SAFE, shipped) — install the recovered WG key
|
|
|
|
`wgtunnel.InstallRecoveredKey(stateDir, privB64)` writes an escrow-recovered WG private key (32-byte
|
|
base64, re-encoded canonical) to the key file so the tunnel re-establishes with the **same
|
|
identity/pubkey → same hub /32**, no fresh keygen. **CREATE-ONLY** — refuses if a key file exists (a
|
|
present key may be a live identity); the value is never logged. Wired into
|
|
`--selftest=identity-consume -install-wg-key` (opt-in; installs `bundle.WGPrivateKey` after
|
|
`UnwrapIdentityBundle`; a pre-S3 blob with no WG key → logged fallback to fresh keygen + re-register,
|
|
which keeps the /32 via hub re-key-in-place).
|
|
**Test + red-proof:** install→`EnsureKey` LOADS it (same pubkey, no keygen); a second install REFUSES
|
|
(create-only — red-proofed against dropping the exists-check); invalid key → error, nothing written.
|
|
|
|
## 3. Part 2 (SAFE, shipped) — consume `restore_directive` → PLAN
|
|
|
|
New `internal/dr`: the host_loss `restore_directive` (was logged-and-ignored in `syncer.go`) is
|
|
consumed via the `desired.Syncer.AddConsumer` raw seam into an inspectable **RestorePlan** — per
|
|
guest `{vmid, archive, target storage, sizing}`, per drive `{durable_id → expected mount}`, + the
|
|
offsite PBS coord. **Derive-and-surface only:** the `Consumer` holds NO restore/destroy dependency, so
|
|
"execute nothing" is structural. `guest_loss`/absent → no plan; recipe fetched on-demand (rare
|
|
directive) via a fresh `Collect`.
|
|
**Test + red-proof:** host_loss builds the expected plan; guest_loss/absent/nil-recipe → none
|
|
(red-proofed against a relaxed mode gate); the consumer surfaces on host_loss + clears otherwise +
|
|
consults the recipe only then. Full `go test ./...` green; deployed 0.69.0; `-install-wg-key` flag
|
|
live; daemon starts clean (dr consumer wired).
|
|
|
|
## 4. Part 3 — hub escrow export: NOT needed
|
|
|
|
The operator has a clean blob-export path already: `sqlite3 … "SELECT writefile('/tmp/blob', blob),
|
|
writefile('/tmp/idblob', identity_blob) FROM host_escrow WHERE host_id='demo-felhom-01'"` on a
|
|
`kubectl cp`'d `hub.db`. No hub change (respects the read-only-unless-no-clean-way gate).
|
|
|
|
## 5. Part 4-A — re-attach wrong-disk safety: ALREADY PROVEN (no new probe built)
|
|
|
|
The re-attach matcher is `storage.ResolveStorageDevice` (the `uuid:<fs-uuid>` scheme DR drives use).
|
|
Its wrong-disk guard is unit-established: `TestResolveStorageDevice_ToleratesDeviceLetterMove` (a
|
|
MATCHING UUID resolves to the current /dev path) + `TestResolveStorageDevice_AbsentAndScheme` (an
|
|
absent/mismatched UUID **ERRORS** — "skips a gone drive instead of fail-mounting"; a bare node /
|
|
non-`uuid:` scheme is refused). It resolves ONLY the exact UUID's symlink — never a "nearest" disk.
|
|
Combined with S4.1's restore-to-scratch proof, the composition's safety is established; a new live
|
|
scratch+marker-drive probe would add confidence but not new safety, so it was not built (recorded).
|
|
|
|
## 6. Part 4-B — the destructive in-place 9201 drill: PREPARED + OPERATOR-GATED (not executed)
|
|
|
|
Pre-flight **GREEN** (non-destructive): offsite `ct/9201` snapshots present (the restore source);
|
|
9201 running; the escrow blob is on the hub (383 B + 499 B identity, offsite key `b0:fe:2a…`),
|
|
exportable via `sqlite3 writefile`; **S4.1 already proved 9201 restores from offsite (pass:true)**.
|
|
The destructive drill (destroy 9201 → restore in place from offsite → re-attach the data drive by
|
|
durable_id) is **operator-present + STOP-gated**: the operator runs the R-consume steps
|
|
(`escrow-consume` + `identity-consume -install-wg-key`, R via env — **CC never runs these**, §9-4a)
|
|
and confirms the destroy at the STOP checkpoint. Prepared procedure + the go/no-go is with the
|
|
operator (destroying the live demo controller is their decision, even with the backup confirmed
|
|
recoverable).
|
|
|
|
## 7. NOT yet live-validated / observations
|
|
|
|
- The live consume→install→tunnel→restore pipeline (the Part 4-B drill) — operator-present, pending
|
|
the operator's go-ahead + R-run. Parts 1-2 are unit-proven + deployed; their live consume path is
|
|
exercised only in that drill (R-gated).
|
|
- Deferred (per slice scope): guest_loss DR; hub-driven full-auto DR (the hub issuing recovery-mode +
|
|
directive itself); multi-guest host-loss. true-CGNAT-SIM retest still parked.
|
|
- Observation: the DR plan is surfaced via structured log + `Consumer.LastPlan()`; a report field
|
|
(so the hub/operator sees the plan without the box's journal) is a small future refinement.
|