Files

T

admin 57405c1a99 slice 7 Phase 1: unified bring-up reconcile job (provision + guest-loss DR) (v0.8.0)

The shared front half of provision and guest-loss DR as a journaled reconcile job
(internal/reconcile/bringup.go), mirroring the restore-test's crash-safety but keeping
the guest on success and applying a scenario-specific identity policy. Agent-only; no
hub/wire change. Grounded by the slice-7 bring-up spike (commit 3342993): F1/F3/F4.

- RunBringUp: restore -> reset identity -> size -> attach mounts -> start link-up;
  verdict is liveness (waitRunning), success KEEPS the guest.
- identity policy: provision = fresh MAC (net0 sans hwaddr -> PVE regen) + hostname,
  host-side; machine-id/host-keys regenerate guest-side (systemd + baked golden unit).
  dr_guest_loss = preserve continuity (keep hostname; keep MAC unless KeepMAC=false).
- compensating rollback: mid-flight failure destroys the just-created guest
  (SameTxnCreated provenance, gated); new Rollback journal flag + Recover.recoverBringUp
  reap a half-built guest from a crash.
- F4: coalesced config PUT + bounded retry on the transient PVE config-lock 500 only.
- --selftest=bring-up (mode/archive/vmid/hostname/keep).
- configs/build-golden.sh: validated golden recipe incl. the F3 first-boot host-key unit.
- doc-03 §9 + identity-reset settled/implemented.

Deferred (stated): provisioning back half -> slice 8; host-loss DR + escrow consumption
and the BringUpSpec source (hub desired-state) -> slice 10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-09 21:27:49 +02:00

4.4 KiB

Raw Blame History

REPORT — Slice 7 Phase 1: unified bring-up reconcile job (v0.8.0) (2026-06-09)

Overwrite-latest report (most recent significant work only). Cumulative history lives in CHANGELOG.md. Implements TASK — Slice 7 Phase 1: unified bring-up reconcile job. Agent-only — no hub/wire change (the new guest auto-appears in the host-report via ListLXC).

Outcome

The shared front half of provision and guest-loss DR shipped as a journaled reconcile job (internal/reconcile/bringup.go), mirroring the slice-6 restore-test's crash-safety but KEEPING the guest on success and applying a scenario-specific identity policy. Built from the slice-7 spike findings (commit 3342993): F1 (MAC reset unconditional on provision), F3 (host keys via a baked golden first-boot unit, not an agent guest-internal op), F4 (transient config-lock retry).

What landed

RunBringUp(BringUpSpec) BringUpResult — restore → identity reset → size → attach mounts → start LINK-UP, each mutation preceded by journaling the owning entry. Verdict is liveness (waitRunning), never the start exitstatus (reuses the v0.7.0 WARNINGS surface). Success keeps the guest (the key difference from the restore-test).
Identity policy (doc 03 §9): provision resets MAC unconditionally (PUT net0, hwaddr omitted → PVE regenerates; F1) + hostname, host-side via the token; machine-id + SSH host keys regenerate guest-side on first boot (systemd + the baked unit) — the agent never touches guest internals. dr_guest_loss preserves continuity (keep hostname; keep MAC unless KeepMAC=false); never resets restic/tunnel/hub identity.
Compensating rollback: any mid-flight failure destroys the just-created guest (ClassGuestDestroy benign via Provenance{SameTxnCreated:true}, gated). New journal flag Rollback + Recover.recoverBringUp reap a half-built guest from a mid-job crash (idempotent, via ListLXC) — distinct from the scratch path's audit label, same destroy machinery.
F4: identity+sizing+mounts coalesced into ONE PUT config; rootfs grow kept separate; setConfigWithLockRetry retries ONLY the transient PVE config-lock 500 (pveConfigLock), never a real error.
--selftest=bring-up (-mode provision|dr -archive -vmid -hostname [-keep]) — runs the real job after a Recover, then tears the guest down unless -keep.
configs/build-golden.sh — the validated golden recipe incl. the F3 first-boot felhom-regen-hostkeys.service (Condition-gated: fires on provision, no-ops on DR). The spike's golden archive (no unit) is superseded.

Tests (assert the effect)

go test ./... green; -race green on the build server. Provision happy path (fresh MAC = net0 without hwaddr, hostname, coalesced sizing+mount, rootfs-grow separate, started, guest NOT destroyed); compensating rollback injected at each step (restore / config / start-task / waitRunning → asserts the guest was destroyed); DR continuity (MAC kept, hostname not reset) + DR KeepMAC=false resets MAC; liveness verdict (warnings+running pass / not-running fail); F4 (lock-500 → retry → proceed; non-lock 500 → fail without retry); owning entry journaled before restore; reserved/existing VMID refused; Recover rolls back / clean.

Live validation (demo-felhom)

Built the real golden via configs/build-golden.sh (with the host-key unit) and exercised the job live:

provision (--selftest=bring-up -mode provision): restore → fresh MAC → hostname set → start link-up → Docker runs; SSH host keys regenerated by the baked unit (ssh.service active; the agent issued no ssh-keygen); machine-id unique; hostname propagated; fresh MAC + clean DHCP lease, no collision → torn down.
dr (-mode dr): continuity-identity branch — hostname + host keys preserved (unit no-op), MAC kept.
Recover: a deliberate mid-restore crash left a half-built guest → restart → Recover reaped the orphan (idempotent).

(Concrete volids/MACs/leases captured in the run; see CHANGELOG + the slice-7 findings doc.)

Deferred (stated, not built)

Provisioning BACK HALF (controller deploy, bootstrap, per-guest token mint) → slice 8; host-loss DR + escrow consumption → slice 10; the SOURCE of a BringUpSpec (hub desired-state) → slice 10 (GuestMount defined minimally, no hub coupling). No secrets committed.

4.4 KiB Raw Blame History