slice 7: golden host-key unit path fix + live-validation results (v0.8.0)

Live validation on demo-felhom caught the felhom-regen-hostkeys unit failing with
203/EXEC: ExecStart was /usr/sbin/ssh-keygen but on Debian 13 ssh-keygen is at
/usr/bin/ssh-keygen. Fixed build-golden.sh, rebuilt the golden, re-validated — host
keys now regenerate on first boot by the baked unit (agent issues no ssh-keygen).

All three live scenarios green: provision (fresh MAC, host keys via unit, machine-id,
Docker, DHCP), dr (continuity: hostname + host keys preserved), Recover (killed
mid-restore -> orphan rolled back idempotently). REPORT + CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

CHANGELOG.md

@@ -53,6 +53,15 @@ transient PVE config-lock 500 → bounded retry).
   not-running fail); F4 (lock-500→retry→proceed; non-lock-500→fail without retry); owning entry
   journaled BEFORE restore; reserved/existing VMID refused; `Recover` rolls back / clean.
 
+### Live-validated (demo-felhom)
+- provision: fresh MAC + hostname; **SSH host keys regenerated by the baked golden unit** (agent
+  issued no `ssh-keygen`), machine-id unique, Docker runs, clean DHCP lease → torn down.
+- dr: continuity preserved (hostname + host keys kept). Recover: a killed mid-restore left an
+  orphan; the re-run's `Recover` rolled it back (idempotent).
+- **Live caught a bug, then fixed:** the host-key unit's `ExecStart` was `/usr/sbin/ssh-keygen`
+  (203/EXEC); on Debian 13 it is `/usr/bin/ssh-keygen` — corrected in `build-golden.sh`, golden
+  rebuilt, re-validated. (Mocked unit tests couldn't surface this; the live run did.)
+
 ## v0.7.0 — restore-test: verdict is liveness, not start-task exitstatus (2026-06-09)
 
 Fixes a correctness bug found by the live hub-enrollment runbook: the self-restore-test reported

REPORT.md

@@ -46,20 +46,30 @@ DR `KeepMAC=false` resets MAC; liveness verdict (warnings+running pass / not-run
 (lock-500 → retry → proceed; non-lock 500 → fail without retry); owning entry journaled **before**
 restore; reserved/existing VMID refused; `Recover` rolls back / clean.
 
-## Live validation (demo-felhom)
+## Live validation (demo-felhom) — all green; caught + fixed one real bug
 
-Built the real golden via `configs/build-golden.sh` (with the host-key unit) and exercised the
+Golden built via `configs/build-golden.sh` → `local:backup/vzdump-lxc-9100-2026_06_09-21_32_58.tar.zst`
-job live:
+(Docker `overlayfs`, host-key unit baked + enabled). All runs via `--selftest=bring-up`.
-- **provision** (`--selftest=bring-up -mode provision`): restore → fresh MAC → hostname set →
-  start link-up → Docker runs; **SSH host keys regenerated by the baked unit** (`ssh.service`
-  active; the agent issued no `ssh-keygen`); machine-id unique; hostname propagated; fresh MAC +
-  clean DHCP lease, no collision → torn down.
-- **dr** (`-mode dr`): continuity-identity branch — hostname + host keys preserved (unit no-op),
-  MAC kept.
-- **Recover**: a deliberate mid-restore crash left a half-built guest → restart → `Recover` reaped
-  the orphan (idempotent).
 
-*(Concrete volids/MACs/leases captured in the run; see CHANGELOG + the slice-7 findings doc.)*
+- **Bug caught live (then fixed + re-validated):** the first golden's `felhom-regen-hostkeys`
+  unit used `ExecStart=/usr/sbin/ssh-keygen` (as drafted in the task) → `status=203/EXEC` ("No such
+  file"); on Debian 13 `ssh-keygen` is at `/usr/bin/ssh-keygen`. Fixed in `build-golden.sh`,
+  rebuilt the golden, re-validated. (The unit tests mock the guest, so only the live run could
+  surface this — exactly why the slice runs live.)
+- **provision** (vmid 8100): restore → fresh MAC `BC:24:11:4A:CF:3C` (≠ golden's) → hostname
+  `felhom-prov-8100` → link-up, 8s. Verified: `felhom-regen-hostkeys` **active** and SSH host keys
+  present (`root@felhom-prov-8100`) — **regenerated by the baked unit, the agent issued no
+  ssh-keygen**; `ssh.service` active; machine-id unique (`103a0325…`); hostname propagated; clean
+  DHCP lease `192.168.0.114`; `docker run hello-world` OK → torn down.
+- **dr** (vmid 8101, from a 9999 customer backup): continuity preserved — hostname stayed
+  `felhom-selftest-scratch` (NOT reset) and SSH host keys **preserved** (not regenerated). The
+  source had no net0, so MAC-keep was a no-op live (covered by the unit test instead).
+- **Recover** (vmid 8102): killed a provision mid-restore (`timeout -s KILL`); the server-side
+  restore left an orphan guest; the re-run's `Recover` logged `rolled back half-built bring-up
+  guest vmid=8102` (`BringUpRolledBack:1`), then re-provisioned cleanly → torn down. Idempotent.
+
+Teardown: no spike guests linger; the validated golden is retained as the artifact; the service
+runs v0.8.0.
 
 ## Deferred (stated, not built)
 

configs/build-golden.sh

@@ -64,7 +64,7 @@ Before=ssh.service sshd.service
 [Service]
 Type=oneshot
 RemainAfterExit=yes
-ExecStart=/usr/sbin/ssh-keygen -A
+ExecStart=/usr/bin/ssh-keygen -A
 
 [Install]
 WantedBy=multi-user.target