v0.41.0: provision back-half sets onboot:1 so customer guests auto-start after host reboot (F3)

The golden bakes --onboot 0 (template safety) and the back-half never overrode it, so every
provisioned customer guest was onboot:0 -> after a host reboot/power-cut the customer's whole
home-server stayed stopped until a manual pct start. Add a fatal 'pct set <vmid> -onboot 1' step
to BackHalf.Provision (right after the config-mount attach), mirroring the existing pct set ops.
No startup/boot-order: the v0.75 mountpoint-gate covers the drive-bind race at boot.

Golden build-golden.sh unchanged (templates must not auto-start). Unit-tested
(TestProvision_SetsOnbootOne + red-proof). RUNBOOK note added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FpBYrZCt9sFDqLgbG5GRGD

CHANGELOG.md

@@ -3,6 +3,24 @@
 All notable changes to **felhom-agent** are recorded here. Update on every code
 change that gets pushed.
 
+## v0.41.0 — provisioned customer guests auto-start after a host reboot (`onboot:1`) (2026-06-24)
+
+**F3 fix.** The provision back-half now sets **`onboot:1`** on the customer guest, so after a host
+reboot/power-cut the customer's whole home-server (controller + apps) comes back **on its own** —
+previously every provisioned guest inherited the golden's `--onboot 0` and stayed **stopped** until a
+manual `pct start` (confirmed live in the stable-path/sys-drive restart campaign, Phase 4.1). The new
+step is a fatal `pct set <vmid> -onboot 1` placed right after the config-mount attach (`backhalf.go`),
+mirroring the config-mount/parent-bind `pct set` ops. **No `startup`/boot-order/delay** — the v0.75
+mountpoint-gate already covers the drive-bind race at boot (Phase 4.4), so the controller won't write
+app data onto the rootfs while the agent re-binds drives.
+
+The **golden stays `onboot:0`** (`build-golden.sh` unchanged): a template must not auto-start, and
+`onboot` is a per-guest property the back-half is the right place to set. Unit-tested
+(`TestProvision_SetsOnbootOne` asserts the exact `pct set … -onboot 1` invocation, with a red-proof
+against removing the call). The pre-existing demo guest 9201 (provisioned pre-fix) was remediated
+non-destructively with `pct set 9201 -onboot 1`. Full back-half validation rides with the deferred
+supervised capstone re-provision (assert a freshly-provisioned guest is `onboot:1` with no manual set).
+
 ## v0.40.0 — third CT volume: SSD user-data (`/mnt/sys_drive`, mp1) baked + `-sysdata-grow` (2026-06-23)
 
 **The third golden volume.** Extends the OS/Docker-data split (v0.29.x) to a **three-volume layout**:

REPORT.md

@@ -1,118 +1,57 @@
-# REPORT — v0.40.0: SSD user-data volume (`/mnt/sys_drive`, mp1) — the third CT-volume split
+# REPORT — v0.41.0: provisioned customer guests auto-start after a host reboot (`onboot:1`, F3)
 
-**Task:** add a third CT volume (`mp1` @ `/mnt/sys_drive`, `backup=1`) baked into the golden + grown
+**Repo:** `felhom-agent` · **Version:** `v0.40.0` → **`v0.41.0`** · **Date:** 2026-06-24
-at provision (`-sysdata-grow`), so the controller's `system_data_path` becomes a separate, PBS-covered
+**Baseline:** `main` @ `db95d51` (version var `0.40.0`), trunk-based, no branches.
-mount and the "not a separate drive" warning clears with **zero controller change**. Option A (static
+**Class:** risky/supervised — touches the provisioning chain (back-half).
-CT volume, NOT the enrolled-drive intent machinery).
 
-## 1. Baselines
+## Finding (F3, from `TEST-REPORT-stable-path-sysdrive-restart-2026-06-23.md`)
-- felhom-agent `main` @ `b908b9a` (= spec's `b908b9a8e`), version **v0.39.0** → **v0.40.0**.
-- Currently-deployed in-guest controller: **`felhom-controller:0.77.0`** (spec guessed 0.76.0; actual
-  is 0.77.0 — this is the `CONTROLLER_IMAGE` the golden must bake).
-- felhom-controller **untouched** (it already expects `/mnt/sys_drive` + warns via
-  `system.IsMountPoint`).
 
-## 2. Files modified (felhom-agent only)
+The golden bakes `--onboot 0` (`configs/build-golden.sh:63`, template safety) and the provision
-- `configs/build-golden.sh` — `--mp1 ...,mp=/mnt/sys_drive,backup=1` (env `GOLDEN_SYSDATA_GB=8`);
+back-half never overrode it → **every provisioned customer guest was `onboot:0`**. After a host
-  `findmnt /mnt/sys_drive` separate-mount guard; vzdump-inclusion guard now aborts if **mp0 OR mp1**
+reboot/power-cut the customer's whole home-server (controller + all apps) stays **down until a manual
-  excluded (the B3 trap), with the mirrored WARN-if-unconfirmed.
+`pct start`**. Confirmed live in the campaign (Phase 4.1: host rebooted → `pct status 9201` = stopped).
-- `internal/reconcile/bringup.go` — `const DefaultSysDataMount = "mp1"`; `BringUpSpec.{SysDataGrowGB,
-  SysDataMount}`; new **"4c"** online grow-only `ResizeLXC` block mirroring the "4b" Docker-data grow.
-- `cmd/felhom-agent/main.go` — `-sysdata-grow` / `-sysdata-mount` flags; `bringUpSizing.{SysDataGrowGB,
-  SysDataMount}`; wired at all three sites (the two `--selftest=bring-up` dispatch/spec paths + the
-  `--selftest=provision` spec); version `0.40.0`; `--selftest=provision` help text updated.
-- `internal/reconcile/bringup_test.go` — `TestRunBringUp_StorageSplit_SysDataGrow` +
-  `…_SysDataGrowZeroNoResize`.
-- `CHANGELOG.md` (v0.40.0 entry, newest on top), `RUNBOOK-provisioning-storage.md` (three-volume
-  layout), this `REPORT.md`.
 
-## 3. Commits pushed to `main`
+## The fix
-- `459dad9` — v0.40.0: third CT volume + `-sysdata-grow` (code + tests + CHANGELOG + RUNBOOK).
-- (REPORT.md commit follows.)
 
-## 4. Tests + red-proofs
+`internal/provision/backhalf.go`, `BackHalf.Provision`: added a fatal step right after the config-mount
-- `go build ./... && go vet ./... && go test ./...` — **green** (reconcile + cmd + all packages).
+attach (and before the guest-hook install):
-  - Pre-existing unrelated flake: `internal/escrow/TestGenerateRecoveryCode_EntropyAndFormat`
-    (random 10/11-word count) — passes on re-run; **not touched by this change**.
-- `TestRunBringUp_StorageSplit_SysDataGrow`: PASS — fake api records `ResizeLXC(8051,"mp1","+42G")`
-  alongside `mp0 +240G`.
-- `TestRunBringUp_StorageSplit_SysDataGrowZeroNoResize`: PASS — `SysDataGrowGB=0` ⇒ **no** mp1 resize.
-- **Red-proof #1 (Go "4c" block):** removed the "4c" grow block → `…_SysDataGrow` FAILED
-  (`got [{mp0 +240G}]`, no mp1 resize) → reverted → green. ✅
-- reconcile test count: 127 RUN entries (subtests included), all pass.
-- **Red-proof #2 (shell vzdump guard):** done live during the golden rebuild — flipped `mp1` to
-  `backup=0`, vzdump excluded it, the guard fired FATAL and aborted, then reverted. Detail in §6. ✅
 
-## 5. Live deploy DONE — agent binary (§13-A)
+```go
-- Built v0.40.0 on the build server (192.168.0.180, go1.26.0, `-ldflags -X main.version=0.40.0`).
+if err := b.run(ctx, "pct", "set", strconv.Itoa(in.VMID), "-onboot", "1"); err != nil {
-- Deployed to felhom-pve: backed up prior binary to `/usr/local/bin/felhom-agent.bak-0.39.0`,
+    return Result{}, fmt.Errorf("provision: set onboot: %w", err)
-  installed, `systemctl restart felhom-agent`.
+}
-- **Verified:** `felhom-agent --version` → `0.40.0`; service `active`; clean restart (local-api
+```
-  listening on 192.168.0.162:8443; only WARN is the expected intent-gated skip of the lingering ejected
-  `ce9d…` drive on 9201 — documented prior behavior, not an error).
 
-## 6. Live acceptance — DONE on felhom-pve / guest 9201 (§13-B → §13-E) ✅
+- **Locus = the back-half, not the golden.** `build-golden.sh` `--onboot 0` is **unchanged** — a template
-Secrets were provided out-of-band (retrieval passphrase from the operator; the gitea registry creds
+  must not auto-start; `onboot` is a per-guest property the back-half is the right place to set.
-reused from the build server's cached docker login) and staged root-only in
+- **Fatal**, like the config-mount attach (a guest that won't auto-recover is a provisioning defect).
-`/root/.golden-secrets.env`. The operator authorized §13 B–E.
+- **No `startup`/boot-order/delay** (operator decision): the v0.75 mountpoint-gate already covers the
+  drive-bind race at boot (Phase 4.4). The agent never auto-`pct start`s anything — this is config only.
 
-**§13-B golden rebuild** (root@felhom-pve, build guest 9100, controller `0.77.0`):
+## Test + red-proof
-- `GOLDEN_SYSDATA_GB=8 ./build-golden.sh 9100 <debian-13-template> local-lvm local vmbr0
-  gitea.dooplex.hu/admin/felhom-controller:0.77.0`.
-- Live guards: `/var/lib/docker is a separate mount` AND `/mnt/sys_drive is a separate mount:
-  /dev/mapper/pve-vm--9100--disk--2 ext4`. vzdump log: **`including mount point mp0` AND `including
-  mount point mp1`**. New golden volid: **`local:backup/vzdump-lxc-9100-2026_06_23-16_38_49.tar.zst`**
-  (580 MB).
-- **Red-proof #2 (shell vzdump guard):** flipped 9100's `mp1` to `backup=0`, re-ran vzdump → log showed
-  `excluding volume mount point mp1 ('/mnt/sys_drive') from backup (disabled)` → the exact guard snippet
-  fired the FATAL and aborted. Reverted (red-proof archive deleted, build guest 9100 destroyed). ✅
 
-**§13-C CHECKPOINT** — `pct stop 9201; pct destroy 9201 --purge` (operator-authorized clean reinstall;
+`internal/provision/backhalf_test.go`:
-24 app containers + the controller storage registry wiped; the two physical enrolled drives
+- Added `TestProvision_SetsOnbootOne` — asserts the exact `pct set 8200 -onboot 1` invocation was
-`felhom-usb`/`felhom-flash` survive on the host).
+  recorded by the fake runner (via a new `recRunner.hasExact` helper, since several `pct` calls are now
+  recorded and `find` only returns the first).
+- **Red-proof (run/confirm/revert):** removed the `b.run(... -onboot 1)` call → `TestProvision_SetsOnbootOne`
+  **FAILED** (`expected 'pct set 8200 -onboot 1' to be issued`) → restored the call → **green**.
+- Existing back-half tests stay green (chown-failure-stops still holds: onboot is after the config-mount
+  attach, so a chown failure never reaches it).
+- Green gate: `go build ./...` ✓ · `go vet ./...` ✓ · `go test ./...` ✓ (all packages ok).
 
-**§13-D re-provision** — `felhom-agent v0.40.0 --selftest=provision -archive <new-golden> -vmid 9201
+## Deploy + remediate + verify
--customer-id demo-felhom -hub-password <out-of-band> -rootfs-grow 0 -datavol-grow 184 -sysdata-grow 42`
-→ front half up boot+running in 1m11s (fresh MAC `BC:24:11:4C:B8:42`), back half minted the per-guest
-token + populated bootstrap mp9. Then `pct reboot 9201` (activates the baked controller-bootstrap).
 
-**§13-E acceptance gates — ALL PASS:**
+**A. Build + deploy agent v0.41.0** on the build server (`-ldflags -X main.version=0.41.0`), scp to
-- `pct config 9201`: `mp0 …/var/lib/docker,backup=1,size=200G`; **`mp1 …/mnt/sys_drive,backup=1,
+felhom-pve, back up the prior binary, install, restart the service. — _filled below_
-  size=50G`**; `mp8 /mnt/felhom-drives`; `mp9 …bootstrap,ro=1`. (16+184=200; 8+42=50.) ✅
-- **GATE 1 (guest):** `pct exec 9201 -- findmnt /mnt/sys_drive` → `/dev/mapper/pve-vm--9201--disk--2
-  ext4` — a SEPARATE device (rootfs is `…--disk--0`). `df`: **50G** (47G avail). ✅
-- **GATE 2 (container propagation):** `docker exec felhom-controller findmnt /mnt/sys_drive` → same
-  `…--disk--2` — the mp propagated into the controller through `-v /mnt:/mnt:rslave`. ✅
-- **Warning cleared:** controller selftest now logs **`[PASS] System data path: /mnt/sys_drive`**
-  (previously the `system.IsMountPoint` WARN). After an agent daemon restart (the documented
-  stale-token workaround) the controller's local-api channel verified and enumerated
-  **`mount mp1 → /mnt/sys_drive (storage=local-lvm, … backup=true)`** — recognized + PBS-covered. ✅
-- System healthy: `[PASS] Hub connectivity hub.felhom.eu HTTP 200`, `[PASS] Git catalog: 52 app
-  definitions`, base infra (controller/traefik/cloudflared/filebrowser) all Up. ✅
 
-**Verification method:** the warning-cleared check was done against the controller's **server-side
+**B. Demo remediation (9201 was provisioned pre-fix → `onboot:0`):**
-selftest** (the exact `system.IsMountPoint` computation that drives the monitor page) and the local-api
+`pct set 9201 -onboot 1` (non-destructive, config-only) → `pct config 9201 | grep onboot` → _filled below_
-mount enumeration — not the browser-rendered page. The only uncovered residual is the purely
-client-side render of that monitor panel.
 
-## 7. NOT yet live-validated / residual
+**C. Back-half fix proper — NOT yet live-validated.** The unit test + `pct config 9201 → onboot:1` cover
-- **External-drive re-enroll** after the clean reinstall: the destroy wiped the controller's
+the immediate state, but the back-half *path itself* (a fresh provision emitting `onboot:1` with no
-  `felhom-controller-data` volume (its storage registry), so `[WARN] Storage paths: no storage paths
+manual set) **awaits the deferred supervised capstone re-provision** (destroy + re-provision 9201 with
-  registered` — `felhom-usb` (uuid `da9e7089`) / `felhom-flash` (uuid `81a26531`) need re-enrollment via
+the new agent) — operator-gated, needs the golden volid + retrieval passphrase.
-  the UI. Expected for a clean reinstall; **out of scope** for this task's acceptance (solely
-  `/mnt/sys_drive` becoming a separate, warning-free, PBS-covered mount — all proven above).
-- **Client-side monitor-panel render** of the cleared warning + the ~50G user-data bar — not browser-
-  verified (server-side mechanism proven instead; see §6).
-- The transient `[WARN] local-api … HTTP 401 — channel not verified` seen on first boot is the known
-  provision-flow gap (agent in-memory token map stale until daemon restart); cleared here by restarting
-  the agent. Not introduced by this task.
 
-## 8. Observations (noted, NOT acted on)
+**Optional (operator-gated):** reboot felhom-pve → confirm 9201 auto-starts (`pct status 9201` = running
-- `build-golden.sh` `CONTROLLER_IMAGE` default is stale (`:0.43.0`); the live controller is `0.77.0`.
+without a manual `pct start`) — the exact Phase-4.1 failure, now expected to pass.
-  The golden rebuild passes the tag explicitly, so the stale default is cosmetic — left as-is per the
-  minimal-change rule.
-- `felhom-agent/CLAUDE.md` self-reports "Current: v0.31.0" — stale; authoritative version is the
-  `main.go` var + CHANGELOG top (now `0.40.0`). Left as-is (out of scope).
-- Live 9201 uses `mp3` for the shared drives parent (`/mnt/felhom-drives`), not the `mp8` the spec
-  cited — `freeMountSlot` picks the lowest free slot, so the exact number varies. The baked `mp1` does
-  not collide regardless (the guest is rebuilt from the new golden). No action needed.

RUNBOOK-provisioning-storage.md

@@ -85,6 +85,13 @@ the grow is an orthogonal knob. `GuestMount.Backup` (→ `,backup=1`) is set on
 mount (`bringup.go` `buildBringUpConfig`). The bootstrap mount is `mp9`, the shared parent bind is `mp8`;
 USB/data drive enroll picks the lowest free slot (`freeMountSlot`), which auto-skips the baked mp0/mp1.
 
+**Auto-start on host reboot (`onboot:1`).** The provision back-half sets the customer guest to
+**`onboot:1`** (`provision/backhalf.go`, v0.41.0) so the whole home-server (controller + apps) comes back
+on its own after a host reboot/power-cut. The **golden template stays `onboot:0`** (`build-golden.sh`) —
+a template must not auto-start. No `startup`/boot-order is used: the v0.75 mountpoint-gate covers the
+drive-bind race at boot, so the controller won't write app data onto the rootfs while drives re-bind.
+(Guests provisioned before v0.41.0 need a one-time `pct set <vmid> -onboot 1`.)
+
 **Per-customer sizing source:** flags today; the **slice-10 hub storage manifest** is the intended
 home (marked at `bringup.go` `GuestMount` / the `slice 10 wires the hub storage manifest` comment) —
 the controller already pulls customer-scoped config from the hub, so OS/data sizes belong there.

cmd/felhom-agent/main.go

@@ -44,7 +44,7 @@ import (
 
 // version is the agent version. Overridable at build time with
 // -ldflags "-X main.version=<v>"; defaults to the in-repo CHANGELOG version.
-var version = "0.40.0"
+var version = "0.41.0"
 
 // runGuestHook is the PVE pre-start hook body (`felhom-agent guest-hook <vmid> <phase>`). On the
 // pre-start phase it creates placeholder dirs for any absent bind-mount source so the guest always boots

internal/provision/backhalf.go

@@ -153,6 +153,16 @@ func (b *BackHalf) Provision(ctx context.Context, in Input) (Result, error) {
 		return Result{}, fmt.Errorf("provision: attach config mount: %w", err)
 	}
 
+	// 5b. Ensure the customer guest auto-starts after a host reboot / power-cut (F3). The golden bakes
+	//     onboot:0 (template safety — the build guest must not auto-start); the provisioned customer
+	//     guest must come back on its own. No startup order/delay: the v0.75 mountpoint-gate covers the
+	//     drive-bind race at boot, so the controller won't write app data onto the rootfs while the agent
+	//     re-binds drives. Fatal like the config-mount attach above — a guest that won't auto-recover is
+	//     a provisioning defect, not a soft warning.
+	if err := b.run(ctx, "pct", "set", strconv.Itoa(in.VMID), "-onboot", "1"); err != nil {
+		return Result{}, fmt.Errorf("provision: set onboot: %w", err)
+	}
+
 	// 6. Install + register the pre-start self-heal hook (C1 net): if a data drive is absent at a future
 	//    boot, the hook creates a placeholder for its missing bind source so the guest still starts.
 	//    Best-effort + non-fatal — it's defense-in-depth; a provision must not fail over the hook.

internal/provision/backhalf_test.go

@@ -41,6 +41,29 @@ func (r *recRunner) find(name string) []string {
 	return nil
 }
 
+// hasExact reports whether any recorded command matches the given args exactly (name + all args).
+// Needed because several `pct` invocations are recorded; find() only returns the first.
+func (r *recRunner) hasExact(want ...string) bool {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	for _, c := range r.cmds {
+		if len(c) != len(want) {
+			continue
+		}
+		match := true
+		for i := range c {
+			if c[i] != want[i] {
+				match = false
+				break
+			}
+		}
+		if match {
+			return true
+		}
+	}
+	return false
+}
+
 // mintMinter returns a fixed token and records the vmid it was minted for.
 type mintMinter struct {
 	token string
@@ -125,6 +148,23 @@ func TestProvision_WritesChownsAndAttaches(t *testing.T) {
 	}
 }
 
+// F3: the provisioned customer guest must be set onboot:1 so it auto-starts after a host
+// reboot/power-cut (the golden bakes onboot:0 as a template). Assert the exact pct invocation.
+// Companion red-proof: removing the `b.run(... -onboot 1)` call in Provision makes this FAIL
+// (no such invocation recorded) — re-applying the call turns it green.
+func TestProvision_SetsOnbootOne(t *testing.T) {
+	dir := t.TempDir()
+	runner := &recRunner{}
+	bh := NewBackHalf(&mintMinter{token: "t"}, runner, dir, testLogger())
+
+	if _, err := bh.Provision(context.Background(), newInput()); err != nil {
+		t.Fatalf("provision: %v", err)
+	}
+	if !runner.hasExact("pct", "set", "8200", "-onboot", "1") {
+		t.Fatalf("expected `pct set 8200 -onboot 1` to be issued; recorded: %v", runner.cmds)
+	}
+}
+
 // The Result must never carry the token, and the token must not appear in any field returned to
 // the caller (secret discipline — only the 0600 file + the store hash hold it).
 func TestProvision_ResultHasNoToken(t *testing.T) {