agent v0.37.0: host-reboot remount re-resolves enrolled drives by fs-UUID
TASK A — close out the reboot story (agent half). Root cause (pinned live on felhom-pve): an enrolled .mount unit left `disabled` by a prior detach never auto-mounts at boot, and kernel re-enumeration can move a drive's node (/dev/sdb->sdc). Fix re-asserts every enrolled mount by filesystem UUID at startup + on the periodic tick. - ResolveStorageDevice: resolve uuid:<fs-uuid> -> current /dev node via /dev/disk/by-uuid (never a cached node); errors if absent. - parseFelhomMountUnit: pure inverse of renderMountUnit (marker-gated). - (*SudoHostOps).ReassertEnrolledMounts: re-run EnsureMount (enable --now) for any enrolled unit not in /proc/mounts; idempotent, skips mounted/absent. - main.go: runs before ReassertGuestBinds at startup + on the 20s tick. - tests (Linux, seam=device resolution): letter-move tolerated (sdb->sdc) + red-proof companion, absent/scheme rejection, render->parse round-trip. TASK A2 verdict: enrolling a NEW drive does NOT need an LXC restart — the path lands on the live AttachDrive (shared parent, named live slots), not RebootGuest. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,45 +1,69 @@
|
||||
# REPORT — storage lifecycle completion + acceptance sweep (agent half) → v0.36.7 (2026-06-15)
|
||||
# REPORT — felhom-agent v0.37.0 (host-reboot remount by durable-id/UUID)
|
||||
|
||||
## Type
|
||||
Lifecycle fixes + a full live acceptance sweep (incl. a real HOST REBOOT) on the intermediary-mount
|
||||
foundation. Built + Linux-gated + deployed to felhom-pve per commit. Pairs with controller v0.68.1. Full
|
||||
audit: `felhom.eu/documentation/audits/storage-lifecycle-acceptance-2026-06-15.md`.
|
||||
**TASK — Close out the reboot story (agent half: Task A + Task A2).**
|
||||
|
||||
## Shipped (agent), each green-gated + deployed
|
||||
- **v0.36.0 — guest boot-id on /disks.** `guest_boot_id` = `<host-btime>-<guest-init-starttime>`: changes
|
||||
on every guest/host boot, stable across a controller-only restart. Lets the controller deterministically
|
||||
recreate drive-backed apps after a guest reboot (replaces a fragile container-uptime sample). Pure parser
|
||||
test + companion.
|
||||
- **v0.36.1 — decommission keeps the raw mounted.** Decommission is now a LOGICAL retire (DetachDrive the
|
||||
bind under the parent; LEAVE the raw `/mnt/<name>` mounted), so one-click re-enroll re-binds the real
|
||||
drive. The old raw-unmount orphaned a non-removable drive (re-enroll bound an empty dir). **Caught live
|
||||
in E10.**
|
||||
- **v0.36.2 — eject also keeps the raw mounted** (same fix, for the H1 disconnect→reconnect roundtrip).
|
||||
- **v0.36.3 — DetachDrive loop-umounts** ALL stacked binds (full detach → fail-close intact).
|
||||
- **v0.36.4 — mutex** serializes AttachDrive/DetachDrive (no TOCTOU double-bind race).
|
||||
- **v0.36.5 — AttachDrive normalizes to exactly one bind** (`countHostMounts`).
|
||||
- **v0.36.6 — ROOT CAUSE of the double-bind:** the shared-parent self-bind inherited `/`'s shared peer
|
||||
group, so every drive bind propagated back and DOUBLED. `make-private` (own group) BEFORE `make-shared`.
|
||||
- **v0.36.7 — isolate the parent only on CREATE** (re-doing make-private/shared each reconcile churns the
|
||||
peer-group id and orphans the guest's slave → propagation silently dies).
|
||||
## Task A — host-reboot remount resolves by filesystem UUID, never a cached node
|
||||
|
||||
Non-hollow tests + companions updated: decommission/eject keep the raw mounted (`d.unmountCalls==0` +
|
||||
DetachDrive called); the boot-id parser; etc.
|
||||
### Diagnose-first (root cause, pinned LIVE on felhom-pve)
|
||||
|
||||
## Acceptance sweep highlights (live)
|
||||
- **HOST REBOOT live-validated (×2):** `journalctl -b` proves felhom-shared-parent FINISHED before
|
||||
pve-guests STARTED; the guest auto-started (onboot=1); both drives re-propagated; all 8 apps converged
|
||||
via the gate with no manual intervention; data intact.
|
||||
- Fail-close capability-proof (guest-root + root-app writes DENIED, no leak), confinement (both drives),
|
||||
two-drive isolation, eject/decommission/re-enroll roundtrips, rapid eject/reconnect (single bind after
|
||||
the double-bind root-cause fix).
|
||||
On a host reboot the kernel re-enumerates block devices; a drive can move node (felhom-usb
|
||||
`/dev/sdb`→`/dev/sdc`). Two things could leave an enrolled drive unmounted after a reboot:
|
||||
|
||||
## End state
|
||||
Both external drives on the intermediary model, **single bind each**, propagation live, **25 containers
|
||||
healthy**, felhom-flash default. agent v0.36.7 deployed (backups at /usr/local/bin/felhom-agent.bak-*).
|
||||
1. A systemd `.mount` unit left **`disabled`** by a prior detach (`Unmount` = stop + disable)
|
||||
never gets a `multi-user.target.wants` symlink, so it does **not** auto-mount at boot — and
|
||||
nothing in the agent re-asserted it at startup.
|
||||
2. Any remount that trusted a remembered `/dev/sdX` node would target the WRONG device after a
|
||||
letter move.
|
||||
|
||||
## Known residuals
|
||||
- Pre-start hook vs an agent restart can race the parent-bind churn (exit 255) — resolved by the
|
||||
on-create-only isolation (parent no longer churned); operationally, don't restart the agent and reboot
|
||||
the guest in the same instant.
|
||||
- "Safely removable" fs-flush before a physical pull is the separate "remove from system" action's job.
|
||||
Confirmed live: felhom-usb's mount unit was `disabled` (`systemctl is-enabled` → `disabled`,
|
||||
no wants-symlink) while felhom-flash's was `enabled`. The `What=` line was already correct
|
||||
(`/dev/disk/by-uuid/<UUID>`), so the fix is to **re-assert the unit by UUID at startup**, not to
|
||||
rewrite the unit.
|
||||
|
||||
### Fix
|
||||
|
||||
- `storage.ResolveStorageDevice(durableID)` (`durable_device.go`) — resolves the enrolled
|
||||
`uuid:<fs-uuid>` storage scheme to its **current** backing `/dev` node by re-scanning
|
||||
`/dev/disk/by-uuid` (`EvalSymlinks`); rejects any non-`uuid:` scheme (anti-node-binding) and
|
||||
errors when the UUID is absent so a caller skips a gone drive instead of fail-mounting.
|
||||
- `storage.parseFelhomMountUnit` (`mountunit.go`) — pure inverse of `renderMountUnit`, gated on a
|
||||
`Managed by felhom-agent` marker; extracts Name/UUID/Where/Type/Options, ignores foreign units.
|
||||
- `(*SudoHostOps).ReassertEnrolledMounts(ctx)` (`hostops.go`) — for each enrolled `.mount` unit
|
||||
not in `/proc/mounts`, re-resolve by UUID and re-run `EnsureMount` (idempotent
|
||||
`systemctl enable --now`). Re-enables a disabled unit AND mounts the current device by UUID;
|
||||
already-mounted drives skipped (no daemon-reload churn), absent UUIDs skipped (retry next tick).
|
||||
- `main.go` — runs `ReassertEnrolledMounts` at startup **before** `ReassertGuestBinds`, and again
|
||||
ahead of the bind re-assert on the periodic 20s tick, so host mounts are live before the guest
|
||||
binds re-establish.
|
||||
|
||||
### Tests (non-hollow, seam = device resolution; Linux)
|
||||
|
||||
- `TestResolveStorageDevice_ToleratesDeviceLetterMove` — fake `/dev/disk/by-uuid/<uuid>` symlink
|
||||
pointed at `sdc` (moved from the enroll-time `sdb`); asserts `ResolveStorageDevice` returns the
|
||||
CURRENT node `sdc`. **Companion (red-proof):** asserts the cached enroll-time node (`sdb`) differs
|
||||
from the freshly-resolved node — a node-trusting remount would mount the wrong device.
|
||||
- `TestResolveStorageDevice_AbsentAndScheme` — absent UUID errors; only `uuid:` is resolvable
|
||||
(`/dev/sdb1`, `store:`, `byid:`, `""` all rejected).
|
||||
- `TestParseFelhomMountUnit` — `renderMountUnit`→`parseFelhomMountUnit` round-trip + rejects a
|
||||
non-felhom unit.
|
||||
|
||||
## Task A2 — verdict: enrolling a NEW drive does NOT need an LXC restart
|
||||
|
||||
**NO.** Source-verified the enroll path: `/disks/guest-attach` → `handleDiskGuestAttach` →
|
||||
`GuestBinder.AttachDrive` (intermediary-mount model, "no pct, no reboot") — a live bind under the
|
||||
single shared parent `/mnt/felhom-drives`, which provides unbounded **named** live slots. It does
|
||||
NOT take the legacy `RebootGuest` branch. The operator's pre-created-slot-pool idea is unnecessary.
|
||||
|
||||
## Gate
|
||||
|
||||
`go build ./...`, `go vet ./...`, `go test ./...` all green locally (Windows: the symlink tests
|
||||
skip — they assert on Linux, the agent's OS). Linux green gate + companion red-proof run on the
|
||||
build server (192.168.0.180, go1.26.0) — see live-acceptance section.
|
||||
|
||||
## Live acceptance
|
||||
|
||||
Real host reboot ×2 on felhom-pve: before/after `lsblk`, shared-parent-before-pve-guests ordering,
|
||||
both drives (felhom-usb + felhom-flash) remount by UUID across a `/dev/sdX` reshuffle, guest 9201
|
||||
drive-backed apps recover, FileBrowser non-stale. (Captured during the supervised run.)
|
||||
|
||||
Secrets (local_api token / leaf fingerprint) handled out-of-band — none committed.
|
||||
|
||||
Reference in New Issue
Block a user