REPORT: v0.19.0 bootstrap v2 — live-validated on demo (guest 9201 onboards ONLINE v0.40.0)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-11 13:37:12 +02:00
parent e5a18194f4
commit de34d170d2
+65 -38
View File
@@ -1,48 +1,75 @@
# REPORT — slice 10D (agent half): DR capstone — identity escrow + restore-mode consumption (v0.18.0) (2026-06-10)
# REPORT — agent v0.19.0: bootstrap contract v2 (relay hub passphrase, no host key in guest) (2026-06-11)
> Overwrite-latest report. Cumulative history: [CHANGELOG.md](CHANGELOG.md).
## What was implemented
Lockstep two-repo change with `felhom-controller` v0.40.0. Fixes the onboarding **401** found last
session: the agent used to bake its **host** hub key into the guest's `bootstrap.json`, but the hub's
`/api/v1/report` authenticates a **customer-scoped** key, so the controller could never report ONLINE.
The agent now bakes a **v2 bootstrap** carrying only what the controller needs to **pull** its own
config from the hub — the agent never touches the customer-scoped key or the CF tokens. Validated live
on the demo (guest 9201).
The agent half of the slice-10 DR capstone (closes slice 10), grounded by both 10-series spikes
(escrow-consumption + identity-restore). The hub half (recovery-mode, re-enroll, directive serving)
is hub v0.11.0. **Rotation model (locked):** the hub holds no Cloudflare write-power; the destructive
tunnel/PBS rotation is the operator's step from a trusted environment (same spirit as 10B).
## What changed
- **`internal/provision/doc.go`** — `SchemaV1 → SchemaV2 = "felhom.bootstrap/v2"`. `DocCustomer` drops
`name`/`domain`/`email` (keeps `id`); `DocHub` drops `api_key`/`host_id`, adds **`retrieval_password`**
(SECRET); `DocLocalAPI` unchanged. Byte-compatible with the controller's `internal/bootstrap.Bootstrap`.
- **`internal/provision/backhalf.go`** — renders the v2 Doc; validation now requires `customer.id` +
`hub.url` + `hub.retrieval_password` (was `customer.id` + `customer.domain`). Mint / 0600 write /
`chown 100000:100000` / `pct set -mp9 …,ro=1` unchanged.
- **`cmd/felhom-agent/main.go` `--selftest=provision`** — **new required `-hub-password`** flag (the
customer's hub retrieval passphrase; the customer must already exist in the hub). Stops baking
`cfg.Hub.APIKey` / `cfg.Hub.HostID`. `-customer-domain/-name/-email` still accepted (bring-up may use
them) but **not** baked into the v2 Doc. Success text updated (reboot → controller pulls + merges).
- **`configs/build-golden.sh`** — default `CONTROLLER_IMAGE` bumped off the stale `:v0.35.0`
`:0.40.0` (matches the registry's no-`v` tag convention; latent footgun fixed).
### Identity escrow (`internal/escrow/identity.go`)
- `WrapIdentity`/`UnwrapIdentity` (+ `…Bundle`) wrap the `{tunnel_token, pbs_token}` bundle under the
SAME recovery code `R` via **`age`** (scrypt + ChaCha20-Poly1305 — vetted passphrase-AEAD, not
hand-rolled), **reusing the K-escrow pty mechanism** (passphrase via the tty, data via files;
`R`/tokens never logged). Same two-factor, zero-knowledge shape as the K-escrow; a **wrong R fails
closed** (no bundle). `age` is a runtime dep for the identity path (like proxmox-backup-client for K).
- `escrow.Create` gains an optional `IdentityBundle` → emits `IdentityBlob` under the same R
(additive; the K-escrow + 10C `Consume` are byte-unchanged; self-verifies the identity round-trip).
- `--selftest=escrow-create -identity-bundle <f> -directive <f>` (wrap + upload the identity blob +
non-secret directive) and `--selftest=identity-consume -blob <f> -keydest <f>` (R via
`FELHOM_RECOVERY_CODE`; tokens written 0600, never logged) — the real code, drillable.
## Cross-repo contract checksum-diff (rendered bootstrap.json field set)
Agent renderer output ingested by the controller's `json.Unmarshal`**every field populated**, exact:
### Restore-mode orchestration (10D.3)
Composed from proven pieces: re-enroll (hub) → fetch directive (10A) → `Consume` K (10C, fingerprint
-gated) + `identity-consume` (R by hand) → restore guests from PBS (restore-overwrite gated by 10B) →
re-establish the tunnel (recovered token + reconstituted origin — proven in the 10D spike). The
destructive cred rotation is the operator's step.
| level | fields (agent emits == controller ingests) |
|---|---|
| top | `schema, customer, hub, local_api` |
| customer | `id` |
| hub | `url, retrieval_password` |
| local_api | `endpoint, fingerprint, token` |
## Tests
- identity round-trips (wrap→unwrap byte-identical; blob is opaque ciphertext); wrong R fails closed +
blob stays retryable; input validation. K-escrow/10C unchanged. `go test ./...` green (age
integration tests gated to a host with the `age` CLI).
## Tests — `internal/provision`, all green
- `doc_test.go`: pins the v2 key set — `customer:[id]`, `hub:[url, retrieval_password]` (asserts **no**
`api_key`/`host_id`), `local_api:[endpoint, fingerprint, token]`; schema == `felhom.bootstrap/v2`.
- `backhalf_test.go`: asserts the rendered bootstrap has `Schema==SchemaV2`, `hub.url` +
`hub.retrieval_password` (no host key), the per-guest token in `local_api`, 0600, chown to
`100000:100000`, `pct set -mp9 …,ro=1`, and the token never leaks into the `Result`.
## Versioning / docs
- Version `0.17.0 → 0.18.0`; `CHANGELOG.md`. Doc 03 §9 (10D done → **slice 10 CLOSED**) + the
host-loss DR flow with the operator-side rotation model.
`go build ./... && go test ./...` green.
## Deferred (non-blocking, per the locked model)
- The hub Config DR/Recovery **web UI** (functional via the recovery-mode admin API today) + a small
operator rotation CLI (the tunnel/PBS rotation + stale-connector delete is a documented operator
procedure from the trusted environment — the hub never holds Cloudflare write-power).
## Live validation (demo Proxmox `felhom-pve`)
- Agent **v0.19.0** built (`-X main.version=0.19.0`), streamed to the demo host (sha256-verified),
installed at `/usr/local/bin/felhom-agent`, daemon restarted (`active`, `felhom-agent 0.19.0`).
- Golden re-baked with controller `:0.40.0`
`local:backup/vzdump-lxc-9100-2026_06_11-13_26_45.tar.zst` (baked image confirmed `:0.40.0`).
- `--selftest=provision -customer-id demo-felhom -hub-password <passphrase>` → guest 9201 brought up +
bootstrap mounted. The written `bootstrap.json` (0600) has `customer:[id]`, `hub:[url,
retrieval_password]` (**no host key**), `local_api:{endpoint,fingerprint,token}`. ✓
- After `pct reboot` the golden's baked unit deployed controller `:0.40.0`, which **pulled** its
config from the hub (customer-scoped key `4b11c0c3…`) and merged in the per-guest `local_api`; the
guest reported **ONLINE at v0.40.0** (no 401). The agent baked **no** customer secret beyond the
per-guest local-API token (passphrase aside — see below). ✓
- 8C invariant unaffected: agent-direct `POST /disks/format` on data-bearing `/dev/sdb1` → **HTTP 403
pending_signature**, disk untouched. ✓
## Pending
- The operator-in-the-loop **DR drill** (live, throwaway identity): host-loss → recovery mode →
re-enroll (old key revoked) → R → consume → restore → re-establish → operator rotates/retires the
old connector → only the new box serves. CC drives the `[CC]` steps; the operator does the trusted
steps.
## What broke / what's missing
- **Finding #1 still open (separate spec):** provisioning while the daemon runs leaves the new guest's
local-API channel 401 until `systemctl restart felhom-agent` (the running daemon doesn't reload the
freshly-minted token). Workaround applied; reproduced.
- **Passphrase-at-rest:** the retrieval passphrase rides in the guest's 0600 `bootstrap.json` (decision
(d) — the agent relays it, never the customer key/CF tokens). Acceptable for now (single-customer
scoped; matches the old `docker-setup.sh --hub-password` model); post-onboard hardening (detach mount
/ rotate) is a separate future item.
- Operational note: the UTF-8 (Hungarian) passphrase must be transported byte-exact (base64) to the
provision call; a `kubectl cp` stderr line polluted the first capture and produced a garbage
passphrase (re-extracted + re-provisioned cleanly).
## Versions / artifacts
- Agent **v0.19.0** (CHANGELOG updated). Pushed to `main`: commit `e5a1819` (code) — this REPORT in
the follow-up commit. Lockstep controller **v0.40.0** (commit `6a594f9`). New golden volid above.
- No secrets committed (passphrase, customer key, CF tokens, local-api token — out-of-band/redacted).