agent v0.34.0: intermediary mount model — shared parent + host-side attach/detach + reconcile
Replaces the per-drive 'pct set -mpN' bind with ONE permanent parent bind /mnt/felhom-drives plus host-side felhom-data swaps underneath it (propagates into the running guest live, no pct, no reboot; C1-immune; confined; fail-closed when absent). EnsureSharedParent installs a boot unit ordered Before=pve-guests. ReassertGuestBinds is now a pure host-side reconcile. /disks reports GuestPath + BoundUnderParent for the controller repoint+gate. Non-hollow tests + companions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -3,6 +3,33 @@
|
||||
All notable changes to **felhom-agent** are recorded here. Update on every code
|
||||
change that gets pushed.
|
||||
|
||||
## v0.34.0 — intermediary mount model: shared-parent + host-side attach/detach + reconcile (2026-06-15)
|
||||
|
||||
The drive hot-swap re-architecture (SPIKE-intermediary-mount). Replaces the per-drive `pct set -mpN`
|
||||
bind (which needed a guest reboot to activate and bricked the guest when a drive was absent at boot)
|
||||
with a SINGLE permanent parent bind `/mnt/felhom-drives` plus host-side swaps underneath it.
|
||||
|
||||
- `internal/localapi/intermediary.go` — `GuestBinder.EnsureSharedParent` (mkdir + self-bind +
|
||||
`--make-shared` + installs/enables a `felhom-shared-parent.service` ordered **Before=pve-guests** so
|
||||
the guest's parent bind inherits the shared peer group as `slave`); `AttachDrive` (`mount --bind
|
||||
/mnt/<name>/felhom-data /mnt/felhom-drives/<name>` — propagates into the RUNNING guest live, no pct,
|
||||
no reboot; confined to felhom-data; the stable dir stays host-root-owned = fail-closed); `DetachDrive`
|
||||
(`umount`, leaving the bare fail-closed dir); `StablePathForRaw`/`DriveNameFromRaw`; `isHostMountpoint`.
|
||||
- `ReassertGuestBinds` is now a pure HOST-SIDE reconcile: for each enrolled+present drive ensure its
|
||||
felhom-data is bound under the parent (no guest-config read, no slot, no reboot) — fixes F9 and
|
||||
drive-reconnect for free. Runs at startup (ensures the shared parent first).
|
||||
- `handleDiskGuestAttach` uses `AttachDrive` (returns the stable `guest_path`); eject + decommission
|
||||
call `DetachDrive`. Legacy `AttachBind`/`DetachBind` retained for the transition (decommission still
|
||||
`--delete`s any lingering legacy mp).
|
||||
- `/disks` reporting adds `GuestPath` (the stable `/mnt/felhom-drives/<name>` the controller repoints
|
||||
HDD_PATH to) and `BoundUnderParent` (live-in-guest signal for the controller's drive-absent gate).
|
||||
- Provision adds the one permanent parent bind (`-mp8 /mnt/felhom-drives,mp=/mnt/felhom-drives`).
|
||||
|
||||
Tests (non-hollow + companions): `TestGuestAttach_BindsUnderParent` (uses AttachDrive not legacy pct),
|
||||
`TestReassertGuestBinds_RestoresMissingBind` (host-side reconcile, legacy AttachBind never called),
|
||||
`TestStablePathForRaw_DriveName`, `TestDisks_GuestPathAndBoundUnderParent`. Sudoers: new
|
||||
`FELHOM_INTERMEDIARY` alias (mount/umount under /mnt/felhom-drives, the unit install, the parent bind).
|
||||
|
||||
## v0.33.0 — C1 net: pre-start self-heal hook + decommission mp-delete (2026-06-15)
|
||||
|
||||
The transitional defense for the C1 brick (B3 critical bug) ahead of the intermediary-mount
|
||||
|
||||
@@ -44,7 +44,7 @@ import (
|
||||
|
||||
// version is the agent version. Overridable at build time with
|
||||
// -ldflags "-X main.version=<v>"; defaults to the in-repo CHANGELOG version.
|
||||
var version = "0.33.0"
|
||||
var version = "0.34.0"
|
||||
|
||||
// runGuestHook is the PVE pre-start hook body (`felhom-agent guest-hook <vmid> <phase>`). On the
|
||||
// pre-start phase it creates placeholder dirs for any absent bind-mount source so the guest always boots
|
||||
|
||||
@@ -71,4 +71,23 @@ Cmnd_Alias FELHOM_GUESTHOOK = \
|
||||
/usr/sbin/pct set [0-9]* --hookscript local\:snippets/felhom-guest-hook.sh, \
|
||||
/usr/sbin/pct set [0-9]* --delete mp[0-9]*
|
||||
|
||||
felhom-agent ALL=(root) NOPASSWD: FELHOM_MOUNT, FELHOM_DISK, FELHOM_PROVISION, FELHOM_FORMAT, FELHOM_DNSMASQ, FELHOM_GUESTHOOK
|
||||
# Intermediary mount model (the drive hot-swap re-architecture). The agent keeps a SHARED host parent
|
||||
# /mnt/felhom-drives (self-bind + make-shared + a boot-persistence systemd unit) and binds/unbinds each
|
||||
# drive's felhom-data namespace UNDERNEATH it so the change propagates into the running guest live (no
|
||||
# pct, no reboot). The agent fine-validates the drive name + confines paths before any exec; the trailing
|
||||
# `*` (matching the comma-laden mp spec) mirrors the existing FELHOM_PROVISION pattern.
|
||||
Cmnd_Alias FELHOM_INTERMEDIARY = \
|
||||
/usr/bin/mkdir -p /mnt/felhom-drives, \
|
||||
/usr/bin/mkdir -p /mnt/felhom-drives/*, \
|
||||
/usr/bin/mkdir -p /mnt/*/felhom-data, \
|
||||
/usr/bin/chown 100000\:100000 /mnt/*/felhom-data, \
|
||||
/usr/bin/mount --bind /mnt/felhom-drives /mnt/felhom-drives, \
|
||||
/usr/bin/mount --make-shared /mnt/felhom-drives, \
|
||||
/usr/bin/mount --bind /mnt/*/felhom-data /mnt/felhom-drives/*, \
|
||||
/usr/bin/umount /mnt/felhom-drives/*, \
|
||||
/usr/bin/install -m 0755 -- /tmp/felhom-shared-parent.sh /usr/local/sbin/felhom-shared-parent.sh, \
|
||||
/usr/bin/install -m 0644 -- /tmp/felhom-shared-parent.service /etc/systemd/system/felhom-shared-parent.service, \
|
||||
/usr/bin/systemctl enable felhom-shared-parent.service, \
|
||||
/usr/sbin/pct set [0-9]* -mp8 /mnt/felhom-drives*
|
||||
|
||||
felhom-agent ALL=(root) NOPASSWD: FELHOM_MOUNT, FELHOM_DISK, FELHOM_PROVISION, FELHOM_FORMAT, FELHOM_DNSMASQ, FELHOM_GUESTHOOK, FELHOM_INTERMEDIARY
|
||||
|
||||
@@ -169,8 +169,8 @@ func TestReassertGuestBinds_SkipsDecommissioned(t *testing.T) {
|
||||
8200: {"mp0": "local-lvm:8,mp=/var/lib/docker"}, // bind missing → would re-add if not gated
|
||||
})
|
||||
srvD.ReassertGuestBinds(context.Background())
|
||||
if gaD.count() != 0 {
|
||||
t.Fatalf("decommissioned drive was re-bound (%d AttachBind) — intent gate missing", gaD.count())
|
||||
if gaD.attachDriveCount() != 0 {
|
||||
t.Fatalf("decommissioned drive was re-bound (%d AttachDrive) — intent gate missing", gaD.attachDriveCount())
|
||||
}
|
||||
|
||||
// companion: enrolled → DOES rebind (same present drive + missing bind).
|
||||
@@ -183,8 +183,11 @@ func TestReassertGuestBinds_SkipsDecommissioned(t *testing.T) {
|
||||
8200: {"mp0": "local-lvm:8,mp=/var/lib/docker"},
|
||||
})
|
||||
srvE.ReassertGuestBinds(context.Background())
|
||||
if gaE.count() != 1 {
|
||||
t.Fatalf("enrolled drive should rebind (got %d AttachBind) — gate too aggressive", gaE.count())
|
||||
if gaE.attachDriveCount() != 1 {
|
||||
t.Fatalf("enrolled drive should rebind (got %d AttachDrive) — gate too aggressive", gaE.attachDriveCount())
|
||||
}
|
||||
if gaE.attachDrives[0] != "/mnt/felhom-usb" {
|
||||
t.Fatalf("reconcile bound the wrong path: %v", gaE.attachDrives)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -208,8 +211,8 @@ func TestReCommission_ReRecords(t *testing.T) {
|
||||
8200: {"mp0": "local-lvm:8,mp=/var/lib/docker"},
|
||||
})
|
||||
srv.ReassertGuestBinds(context.Background())
|
||||
if ga.count() != 1 {
|
||||
t.Fatalf("re-commissioned drive should rebind (got %d)", ga.count())
|
||||
if ga.attachDriveCount() != 1 {
|
||||
t.Fatalf("re-commissioned drive should rebind under the parent (got %d AttachDrive)", ga.attachDriveCount())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+88
-73
@@ -67,9 +67,18 @@ type GuestLister interface {
|
||||
// mount (slice 10 P2, Model A) and reboots the guest to activate persisted-but-inactive binds (the
|
||||
// host-side live inject is blocked on unprivileged guests). Satisfied by *GuestBinder.
|
||||
type GuestAttacher interface {
|
||||
// AttachDrive (intermediary model) binds the drive's felhom-data under the shared parent so it
|
||||
// appears live in the guest at the returned stable path — no pct, no reboot. `where` = raw /mnt/<name>.
|
||||
AttachDrive(ctx context.Context, where string) (guestPath string, err error)
|
||||
// DetachDrive unmounts the drive's felhom-data from the shared parent (live, fail-closed).
|
||||
DetachDrive(ctx context.Context, where string) error
|
||||
// EnsureSharedParent makes the host stable parent shared + installs the boot-persistence unit.
|
||||
EnsureSharedParent(ctx context.Context) error
|
||||
// AttachBind is the LEGACY per-drive `pct set -mpN` bind (pre-intermediary). Retained for the
|
||||
// transition; new attaches use AttachDrive.
|
||||
AttachBind(ctx context.Context, vmid int, mountKey, where string) error
|
||||
// DetachBind removes a mountpoint slot from the guest config (the decommission C1 fix — a removed
|
||||
// bind can't brick the next boot with a missing source). Runs on the running guest (no start lock).
|
||||
// DetachBind removes a LEGACY mountpoint slot from the guest config (the decommission C1 fix — a
|
||||
// removed bind can't brick the next boot with a missing source). Runs on the running guest.
|
||||
DetachBind(ctx context.Context, vmid int, mountKey string) error
|
||||
RebootGuest(ctx context.Context, vmid int) error
|
||||
}
|
||||
@@ -120,8 +129,18 @@ type DiskInfo struct {
|
||||
// GuestAttached reports whether THIS guest (the token's vmid) actually has the drive's felhom-data
|
||||
// namespace bound into its config — i.e. the drive is usable IN the guest, not merely present on the
|
||||
// host. F9: host presence (State=attached) != guest-usable; this is the missing signal that made the
|
||||
// HDD look available when it wasn't bound. Only meaningful for user-data drives.
|
||||
// HDD look available when it wasn't bound. Only meaningful for user-data drives. LEGACY (per-drive
|
||||
// `pct set -mpN` model) — the intermediary model uses BoundUnderParent.
|
||||
GuestAttached bool `json:"guest_attached"`
|
||||
// GuestPath is the drive's STABLE in-guest path in the intermediary-mount model
|
||||
// (/mnt/felhom-drives/<name>). This is what the controller repoints HDD_PATH to and registers as the
|
||||
// storage path. Set for /mnt/<name> drives; "" otherwise. Distinct from MountPath (the RAW host PVE
|
||||
// mount the agent ops on).
|
||||
GuestPath string `json:"guest_path,omitempty"`
|
||||
// BoundUnderParent reports whether the drive's felhom-data is currently bound under the shared parent
|
||||
// at GuestPath (a host mount-table check) — i.e. live + usable in the guest in the intermediary model.
|
||||
// The controller's drive-absent gate + auto-restart key on this (and State).
|
||||
BoundUnderParent bool `json:"bound_under_parent"`
|
||||
}
|
||||
|
||||
// handleDisks lists the host's drives + data-bearing flags (read-only/benign).
|
||||
@@ -153,6 +172,14 @@ func (s *Server) handleDisks(w http.ResponseWriter, r *http.Request, vmid int) {
|
||||
UsedFraction: t.UsedFraction,
|
||||
GuestAttached: t.MountPath != "" && boundPaths[t.MountPath],
|
||||
}
|
||||
// Intermediary model: the stable in-guest path + whether felhom-data is bound under the parent.
|
||||
// Only user-data /mnt/<name> drives have a guest path (system/backup mounts never cross in).
|
||||
if di.Role == string(storage.RoleUserData) {
|
||||
if gp := StablePathForRaw(t.MountPath); gp != "" {
|
||||
di.GuestPath = gp
|
||||
di.BoundUnderParent = s.boundUnderParent(gp)
|
||||
}
|
||||
}
|
||||
// Inspect the backing device for the UI's data-bearing hint (the authoritative check
|
||||
// is re-run at format time on the actual device).
|
||||
if t.BackingDevice != "" {
|
||||
@@ -245,6 +272,13 @@ func (s *Server) handleDiskEject(w http.ResponseWriter, r *http.Request, vmid in
|
||||
// self-heal watchdog leaves it alone — an OFFICIAL eject is the only thing that sets this (P3); an
|
||||
// out-of-band unmount records nothing and is healed.
|
||||
s.recordIntent(r.Context(), req.Where, "ejected")
|
||||
// Intermediary model: unmount the felhom-data bind from under the shared parent FIRST (live, fail-
|
||||
// closed in the guest), so nothing references the raw mount before we unmount it.
|
||||
if s.guestAttach != nil {
|
||||
if err := s.guestAttach.DetachDrive(r.Context(), req.Where); err != nil {
|
||||
s.logger.Warn("local-api: eject guest-detach (intermediary) failed", "vmid", vmid, "where", req.Where, "err", err)
|
||||
}
|
||||
}
|
||||
if err := s.disks.Unmount(r.Context(), req.Where); err != nil {
|
||||
s.logger.Error("local-api: disk eject", "vmid", vmid, "where", req.Where, "err", err)
|
||||
writeErr(w, http.StatusBadRequest, "eject failed: "+err.Error())
|
||||
@@ -301,9 +335,14 @@ func (s *Server) handleDiskDecommission(w http.ResponseWriter, r *http.Request,
|
||||
// `mpN` in the config → pre-start mount failure → all apps down. Runs on the running guest (config
|
||||
// edit, no start lock → no deadlock). Best-effort: a missing slot is already clean.
|
||||
if s.guestAttach != nil {
|
||||
// Intermediary model: unmount the felhom-data bind from under the shared parent (live, no reboot).
|
||||
if err := s.guestAttach.DetachDrive(r.Context(), req.Where); err != nil {
|
||||
s.logger.Warn("local-api: guest-detach (intermediary) failed", "vmid", vmid, "where", req.Where, "err", err)
|
||||
}
|
||||
// Legacy per-drive model: delete any lingering `mpN` slot (no-op post-migration; C1 fix otherwise).
|
||||
if slot := s.guestSlotForPath(r.Context(), vmid, req.Where); slot != "" {
|
||||
if err := s.guestAttach.DetachBind(r.Context(), vmid, slot); err != nil {
|
||||
s.logger.Warn("local-api: guest-detach failed — mp left in config (reboot may brick until reconciled)",
|
||||
s.logger.Warn("local-api: guest-detach (legacy mp) failed — mp left in config",
|
||||
"vmid", vmid, "slot", slot, "where", req.Where, "err", err)
|
||||
}
|
||||
}
|
||||
@@ -344,41 +383,24 @@ func (s *Server) handleDiskGuestAttach(w http.ResponseWriter, r *http.Request, v
|
||||
writeErr(w, http.StatusBadRequest, "where must be an absolute /mnt/<name> path (no traversal)")
|
||||
return
|
||||
}
|
||||
// Read the guest config for idempotency + free-slot selection.
|
||||
cfg, err := s.guests.GuestConfig(r.Context(), vmid)
|
||||
// Intermediary model: ensure the host shared parent, then bind the drive's felhom-data UNDER it. It
|
||||
// propagates into the running guest LIVE — no pct, no slot, no reboot. AttachDrive is idempotent (a
|
||||
// no-op when the stable path is already a mountpoint), so this is the re-enroll path too.
|
||||
if err := s.guestAttach.EnsureSharedParent(r.Context()); err != nil {
|
||||
s.logger.Warn("local-api: guest-attach — shared parent ensure failed (continuing)", "vmid", vmid, "err", err)
|
||||
}
|
||||
stable, err := s.guestAttach.AttachDrive(r.Context(), where)
|
||||
if err != nil {
|
||||
s.logger.Error("local-api: guest-attach guest config", "vmid", vmid, "err", err)
|
||||
writeErr(w, http.StatusBadGateway, "could not read guest config")
|
||||
return
|
||||
}
|
||||
mounts := cfg.MountPoints()
|
||||
// Idempotency: already bound at `where`? (a bind's mp= equals the guest path). Still (re)record the
|
||||
// ENROLLED intent — a drive bound at a prior boot must be known to self-heal even if never
|
||||
// re-attached this process lifetime.
|
||||
for key, spec := range mounts {
|
||||
if _, mp, _ := parseMount(spec); mp == where {
|
||||
s.recordIntent(r.Context(), where, "enrolled")
|
||||
s.recordGuestBind(r.Context(), vmid, where)
|
||||
s.logger.Info("local-api: guest-attach idempotent (already bound)", "vmid", vmid, "where", where, "slot", key)
|
||||
writeOK(w, map[string]any{"vmid": vmid, "attached": where, "slot": key, "already": true})
|
||||
return
|
||||
}
|
||||
}
|
||||
slot, ok := freeMountSlot(mounts)
|
||||
if !ok {
|
||||
writeErr(w, http.StatusConflict, "no free mountpoint slot on the guest")
|
||||
return
|
||||
}
|
||||
if err := s.guestAttach.AttachBind(r.Context(), vmid, slot, where); err != nil {
|
||||
s.logger.Error("local-api: guest-attach", "vmid", vmid, "where", where, "slot", slot, "err", err)
|
||||
s.logger.Error("local-api: guest-attach (intermediary)", "vmid", vmid, "where", where, "err", err)
|
||||
writeErr(w, http.StatusBadGateway, "guest-attach failed: "+err.Error())
|
||||
return
|
||||
}
|
||||
// Record the drive as ENROLLED so the self-heal watchdog will reconcile it (P3), and persist the
|
||||
// per-guest bind so the startup re-assert can restore it after a re-provision (F9).
|
||||
// Record the drive as ENROLLED so the self-heal reconcile keeps it bound (P3), and persist the
|
||||
// per-guest bind so the startup reconcile restores it after a re-provision (F9 — now host-side only).
|
||||
s.recordIntent(r.Context(), where, "enrolled")
|
||||
s.recordGuestBind(r.Context(), vmid, where)
|
||||
writeOK(w, map[string]any{"vmid": vmid, "attached": where, "slot": slot})
|
||||
s.logger.Info("local-api: guest-attach bound under shared parent", "vmid", vmid, "where", where, "guest_path", stable)
|
||||
writeOK(w, map[string]any{"vmid": vmid, "attached": where, "guest_path": stable})
|
||||
}
|
||||
|
||||
type guestRebootRequest struct {
|
||||
@@ -663,6 +685,15 @@ func (s *Server) handleDiskFormat(w http.ResponseWriter, r *http.Request, vmid i
|
||||
"device is system/backup-protected — format requires an operator signature ("+dec.Reason+")")
|
||||
}
|
||||
|
||||
// boundUnderParent reports whether a drive's felhom-data is currently bound at its stable guest path
|
||||
// (intermediary model). Injectable via s.boundCheck for tests; defaults to the host mount-table read.
|
||||
func (s *Server) boundUnderParent(stablePath string) bool {
|
||||
if s.boundCheck != nil {
|
||||
return s.boundCheck(stablePath)
|
||||
}
|
||||
return isHostMountpoint(stablePath)
|
||||
}
|
||||
|
||||
// guestBoundPaths returns the set of guest mountpoint paths (the `mp=` of each entry in the guest's
|
||||
// config) — i.e. the host drives actually BOUND into the guest. F9: this is the guest-attached signal
|
||||
// (`GuestAttached`) that distinguishes a guest-usable drive from one merely present on the host. A bind
|
||||
@@ -725,16 +756,22 @@ func (s *Server) recordGuestBind(ctx context.Context, vmid int, where string) {
|
||||
s.logger.Info("local-api: guest-bind recorded for startup re-assert", "vmid", vmid, "where", where, "durable_id", id)
|
||||
}
|
||||
|
||||
// ReassertGuestBinds re-adds, on agent startup (the host's bring-up/reconcile trigger), any enrolled
|
||||
// user-data drive bind a guest is MISSING from its config (F9 — a re-provision drops the mp, and nothing
|
||||
// previously restored it). For each recorded (vmid, durable-id): only when the durable-id STILL resolves
|
||||
// to a present, mounted drive AND the guest lacks the bind, it re-runs AttachBind. "On durable-id match"
|
||||
// — a swapped or absent drive is never auto-bound. The re-added bind is config state; it activates on the
|
||||
// guest's next reboot (logged), exactly like the enroll flow. Safe to call repeatedly (idempotent).
|
||||
// ReassertGuestBinds is the intermediary-model startup reconcile: for every ENROLLED + PRESENT enrolled
|
||||
// drive, it ensures the drive's felhom-data is bound under the shared parent (host-side, via AttachDrive)
|
||||
// so it's live in the guest. This is PURE host-side mount work — NO pct, NO guest-config read, NO reboot
|
||||
// — and it fixes F9 (a re-provision drops nothing host-side) AND drive-returned reconnection for free.
|
||||
// It first ensures the host shared parent exists (so a fresh boot has it before any bind). "On durable-id
|
||||
// match" + intent-gated: a swapped, ejected, or decommissioned drive is never auto-bound. AttachDrive is
|
||||
// idempotent (a no-op when already bound), so this is safe to call repeatedly. Runs at agent startup and
|
||||
// on a drive-returned event.
|
||||
func (s *Server) ReassertGuestBinds(ctx context.Context) {
|
||||
if s.guestBinds == nil || s.guestAttach == nil || s.guests == nil {
|
||||
if s.guestBinds == nil || s.guestAttach == nil {
|
||||
return
|
||||
}
|
||||
// Make sure the host stable parent is shared + boot-persistent before we bind anything under it.
|
||||
if err := s.guestAttach.EnsureSharedParent(ctx); err != nil {
|
||||
s.logger.Warn("reconcile: shared parent ensure failed — binds may not propagate live", "err", err)
|
||||
}
|
||||
// durable-id -> current host mount path (present drives only), from the agent's own storage view.
|
||||
mountByDurable := map[string]string{}
|
||||
if targets, err := s.storage.Observe(ctx); err == nil {
|
||||
@@ -744,52 +781,30 @@ func (s *Server) ReassertGuestBinds(ctx context.Context) {
|
||||
}
|
||||
}
|
||||
} else {
|
||||
s.logger.Warn("F9 re-assert: storage view unavailable — skipping", "err", err)
|
||||
s.logger.Warn("reconcile: storage view unavailable — skipping", "err", err)
|
||||
return
|
||||
}
|
||||
for vmid, ids := range s.guestBinds.Guests() {
|
||||
cfg, err := s.guests.GuestConfig(ctx, vmid)
|
||||
if err != nil {
|
||||
s.logger.Warn("F9 re-assert: skip guest (config read failed)", "vmid", vmid, "err", err)
|
||||
continue
|
||||
}
|
||||
mounts := cfg.MountPoints()
|
||||
boundPaths := map[string]bool{}
|
||||
for _, spec := range mounts {
|
||||
if _, mp, _ := parseMount(spec); mp != "" {
|
||||
boundPaths[mp] = true
|
||||
}
|
||||
}
|
||||
for _, id := range ids {
|
||||
// Intent-aware (B2, the load-bearing correctness fix): NEVER re-bind a drive that is not
|
||||
// currently `enrolled` — an ejected or decommissioned drive must not auto-rebind into the
|
||||
// guest on agent restart, even if it is still host-mounted. Covers both the self-serve and
|
||||
// the operator-signed decommission paths (both land on IntentDecommissioned). A nil intent
|
||||
// store falls back to legacy bind-all (ungated), matching the watchdog's nil-intent rule.
|
||||
// Intent-aware (B2, load-bearing): NEVER bind a drive that is not currently `enrolled` — an
|
||||
// ejected or decommissioned drive must not auto-rebind, even if still host-mounted. A nil
|
||||
// intent store falls back to bind-all (ungated), matching the watchdog's nil-intent rule.
|
||||
if s.intent != nil && s.intent.Get(id) != storage.IntentEnrolled {
|
||||
s.logger.Warn("F9 re-assert: skipping non-enrolled drive (intent-gated)", "vmid", vmid, "durable_id", id, "intent", string(s.intent.Get(id)))
|
||||
s.logger.Warn("reconcile: skipping non-enrolled drive (intent-gated)", "vmid", vmid, "durable_id", id, "intent", string(s.intent.Get(id)))
|
||||
continue
|
||||
}
|
||||
where, present := mountByDurable[id]
|
||||
if !present {
|
||||
s.logger.Warn("F9 re-assert: enrolled drive not present (durable-id absent) — skipping", "vmid", vmid, "durable_id", id)
|
||||
s.logger.Warn("reconcile: enrolled drive not present (durable-id absent) — skipping", "vmid", vmid, "durable_id", id)
|
||||
continue
|
||||
}
|
||||
if boundPaths[where] {
|
||||
continue // already bound — nothing to re-assert
|
||||
}
|
||||
slot, ok := freeMountSlot(mounts)
|
||||
if !ok {
|
||||
s.logger.Warn("F9 re-assert: no free mountpoint slot on guest", "vmid", vmid, "where", where)
|
||||
stable, err := s.guestAttach.AttachDrive(ctx, where)
|
||||
if err != nil {
|
||||
s.logger.Error("reconcile: AttachDrive failed", "vmid", vmid, "where", where, "err", err)
|
||||
continue
|
||||
}
|
||||
if err := s.guestAttach.AttachBind(ctx, vmid, slot, where); err != nil {
|
||||
s.logger.Error("F9 re-assert: AttachBind failed", "vmid", vmid, "where", where, "slot", slot, "err", err)
|
||||
continue
|
||||
}
|
||||
mounts[slot] = where // reserve the slot so a second enrolled drive takes the next one
|
||||
s.logger.Warn("F9 re-assert: re-attached enrolled drive into guest config (reboot to activate)",
|
||||
"vmid", vmid, "where", where, "slot", slot, "durable_id", id)
|
||||
s.logger.Info("reconcile: enrolled drive bound under shared parent (live, no reboot)",
|
||||
"vmid", vmid, "where", where, "guest_path", stable, "durable_id", id)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package localapi
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
@@ -394,6 +395,10 @@ type fakeGuestAttacher struct {
|
||||
vmid int
|
||||
slot string
|
||||
}
|
||||
attachDrives []string // where passed to AttachDrive (intermediary model)
|
||||
detachDrives []string // where passed to DetachDrive
|
||||
ensureParentN int // EnsureSharedParent call count
|
||||
attachDriveFail bool // when set, AttachDrive returns an error
|
||||
}
|
||||
|
||||
func (f *fakeGuestAttacher) AttachBind(_ context.Context, vmid int, mountKey, where string) error {
|
||||
@@ -407,6 +412,35 @@ func (f *fakeGuestAttacher) AttachBind(_ context.Context, vmid int, mountKey, wh
|
||||
}
|
||||
func (f *fakeGuestAttacher) count() int { f.mu.Lock(); defer f.mu.Unlock(); return len(f.calls) }
|
||||
|
||||
func (f *fakeGuestAttacher) AttachDrive(_ context.Context, where string) (string, error) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if f.attachDriveFail {
|
||||
return "", fmt.Errorf("fake attach-drive failure")
|
||||
}
|
||||
f.attachDrives = append(f.attachDrives, where)
|
||||
return StablePathForRaw(where), nil
|
||||
}
|
||||
func (f *fakeGuestAttacher) attachDriveCount() int {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
return len(f.attachDrives)
|
||||
}
|
||||
|
||||
func (f *fakeGuestAttacher) DetachDrive(_ context.Context, where string) error {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.detachDrives = append(f.detachDrives, where)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *fakeGuestAttacher) EnsureSharedParent(_ context.Context) error {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.ensureParentN++
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *fakeGuestAttacher) DetachBind(_ context.Context, vmid int, mountKey string) error {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
@@ -444,8 +478,12 @@ func newAttachServer(t *testing.T, ga GuestAttacher, mounts map[int]map[string]s
|
||||
return srv.Handler()
|
||||
}
|
||||
|
||||
// A first attach picks the lowest free slot (mp0; mp9 bootstrap is taken) and calls the binder.
|
||||
func TestGuestAttach_PicksFreeSlotAndBinds(t *testing.T) {
|
||||
// Intermediary model: a guest-attach binds the drive's felhom-data under the shared parent (host-side)
|
||||
// and returns its STABLE guest path — no pct slot. EnsureSharedParent runs first.
|
||||
//
|
||||
// COMPANION GUARD: the legacy `pct set -mpN` AttachBind must NOT be used in the intermediary model — a
|
||||
// pre-fix impl that still called AttachBind would trip ga.count()!=0 here.
|
||||
func TestGuestAttach_BindsUnderParent(t *testing.T) {
|
||||
ga := &fakeGuestAttacher{}
|
||||
h := newAttachServer(t, ga, map[int]map[string]string{
|
||||
8200: {"mp9": "/var/lib/.../bootstrap,mp=/etc/felhom-bootstrap,ro=1"},
|
||||
@@ -454,26 +492,27 @@ func TestGuestAttach_PicksFreeSlotAndBinds(t *testing.T) {
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("attach: got %d want 200 (%s)", w.Code, w.Body.String())
|
||||
}
|
||||
if ga.count() != 1 || ga.calls[0].slot != "mp0" || ga.calls[0].where != "/mnt/felhom-usb" || ga.calls[0].vmid != 8200 {
|
||||
t.Fatalf("AttachBind not called with mp0/where/vmid: %+v", ga.calls)
|
||||
if ga.attachDriveCount() != 1 || ga.attachDrives[0] != "/mnt/felhom-usb" {
|
||||
t.Fatalf("AttachDrive not called with /mnt/felhom-usb: %+v", ga.attachDrives)
|
||||
}
|
||||
if ga.ensureParentN < 1 {
|
||||
t.Fatalf("EnsureSharedParent must run before binding the drive")
|
||||
}
|
||||
if !strings.Contains(w.Body.String(), `"guest_path":"/mnt/felhom-drives/felhom-usb"`) {
|
||||
t.Fatalf("response missing the stable guest_path: %s", w.Body.String())
|
||||
}
|
||||
if ga.count() != 0 {
|
||||
t.Fatalf("legacy pct AttachBind called (%d) — intermediary model must use host-side AttachDrive", ga.count())
|
||||
}
|
||||
}
|
||||
|
||||
// An already-bound drive is idempotent: returns the existing slot, binder NOT called again.
|
||||
func TestGuestAttach_Idempotent(t *testing.T) {
|
||||
ga := &fakeGuestAttacher{}
|
||||
h := newAttachServer(t, ga, map[int]map[string]string{
|
||||
8200: {"mp0": "/mnt/felhom-usb/felhom-data,mp=/mnt/felhom-usb"},
|
||||
})
|
||||
// An AttachDrive failure surfaces as 502 (no silent success).
|
||||
func TestGuestAttach_AttachDriveFailure(t *testing.T) {
|
||||
ga := &fakeGuestAttacher{attachDriveFail: true}
|
||||
h := newAttachServer(t, ga, map[int]map[string]string{8200: {}})
|
||||
w := do(t, h, "POST", "/disks/guest-attach", "A", `{"where":"/mnt/felhom-usb"}`)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("idempotent attach: got %d want 200 (%s)", w.Code, w.Body.String())
|
||||
}
|
||||
if !strings.Contains(w.Body.String(), `"already":true`) {
|
||||
t.Fatalf("expected already:true: %s", w.Body.String())
|
||||
}
|
||||
if ga.count() != 0 {
|
||||
t.Fatalf("AttachBind must NOT be called for an already-bound drive: %+v", ga.calls)
|
||||
if w.Code != http.StatusBadGateway {
|
||||
t.Fatalf("attach failure: got %d want 502 (%s)", w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -43,28 +43,30 @@ func usbPresent() fakeStorage {
|
||||
}}
|
||||
}
|
||||
|
||||
// TestReassertGuestBinds_RestoresMissingBind is the F9 core proof (the operator's "fire the real
|
||||
// trigger" requirement): an enrolled drive that is present on the host but MISSING from the guest config
|
||||
// (the post-re-provision gap) is auto-re-attached on the startup re-assert — with NO manual guest-attach
|
||||
// call. Fails on the pre-fix code (no re-assert existed).
|
||||
// TestReassertGuestBinds_RestoresMissingBind is the F9/reconnect core proof: an enrolled, host-present
|
||||
// drive is auto-bound under the shared parent on the startup reconcile — host-side (AttachDrive), with NO
|
||||
// pct and NO manual guest-attach call.
|
||||
//
|
||||
// COMPANION GUARD: the legacy `pct set -mpN` AttachBind must NOT be used — a pre-intermediary impl that
|
||||
// still re-added a config mp would trip ga.count()!=0.
|
||||
func TestReassertGuestBinds_RestoresMissingBind(t *testing.T) {
|
||||
gb := tempBindStore(t)
|
||||
if err := gb.Record(8200, "uuid:usb-1"); err != nil { // enrolled at a prior boot
|
||||
t.Fatal(err)
|
||||
}
|
||||
ga := &fakeGuestAttacher{}
|
||||
// guest 8200 has docker-data only — the felhom-usb bind was dropped by the re-provision.
|
||||
srv := reassertServer(t, ga, usbPresent(), map[int]map[string]string{
|
||||
8200: {"mp0": "local-lvm:8,mp=/var/lib/docker"},
|
||||
}, gb)
|
||||
srv := reassertServer(t, ga, usbPresent(), nil, gb)
|
||||
|
||||
srv.ReassertGuestBinds(context.Background())
|
||||
|
||||
if ga.count() != 1 {
|
||||
t.Fatalf("AttachBind called %d times, want 1 (auto-re-assert on startup)", ga.count())
|
||||
if ga.attachDriveCount() != 1 || ga.attachDrives[0] != "/mnt/felhom-usb" {
|
||||
t.Fatalf("AttachDrive = %v, want one call for /mnt/felhom-usb (host-side reconcile)", ga.attachDrives)
|
||||
}
|
||||
if ga.calls[0].vmid != 8200 || ga.calls[0].where != "/mnt/felhom-usb" {
|
||||
t.Fatalf("re-asserted bind = %+v, want vmid 8200 where /mnt/felhom-usb", ga.calls[0])
|
||||
if ga.count() != 0 {
|
||||
t.Fatalf("legacy pct AttachBind called (%d) — reconcile must be host-side only", ga.count())
|
||||
}
|
||||
if ga.ensureParentN < 1 {
|
||||
t.Fatalf("EnsureSharedParent must run before binding drives under the parent")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -74,28 +76,11 @@ func TestReassertGuestBinds_SkipsAbsentDurable(t *testing.T) {
|
||||
gb := tempBindStore(t)
|
||||
_ = gb.Record(8200, "uuid:usb-1")
|
||||
ga := &fakeGuestAttacher{}
|
||||
srv := reassertServer(t, ga, fakeStorage{}, map[int]map[string]string{ // empty storage view → absent
|
||||
8200: {"mp0": "local-lvm:8,mp=/var/lib/docker"},
|
||||
}, gb)
|
||||
srv := reassertServer(t, ga, fakeStorage{}, nil, gb) // empty storage view → absent
|
||||
|
||||
srv.ReassertGuestBinds(context.Background())
|
||||
if ga.count() != 0 {
|
||||
t.Fatalf("AttachBind called %d times — must NOT auto-bind an absent/swapped drive", ga.count())
|
||||
}
|
||||
}
|
||||
|
||||
// TestReassertGuestBinds_SkipsAlreadyBound: when the guest already has the bind, the re-assert is a no-op.
|
||||
func TestReassertGuestBinds_SkipsAlreadyBound(t *testing.T) {
|
||||
gb := tempBindStore(t)
|
||||
_ = gb.Record(8200, "uuid:usb-1")
|
||||
ga := &fakeGuestAttacher{}
|
||||
srv := reassertServer(t, ga, usbPresent(), map[int]map[string]string{
|
||||
8200: {"mp0": "local-lvm:8,mp=/var/lib/docker", "mp3": "/mnt/felhom-usb/felhom-data,mp=/mnt/felhom-usb"},
|
||||
}, gb)
|
||||
|
||||
srv.ReassertGuestBinds(context.Background())
|
||||
if ga.count() != 0 {
|
||||
t.Fatalf("AttachBind called %d times — already bound, must be a no-op", ga.count())
|
||||
if ga.attachDriveCount() != 0 {
|
||||
t.Fatalf("AttachDrive called %d times — must NOT auto-bind an absent/swapped drive", ga.attachDriveCount())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
package localapi
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Intermediary-mount model (replaces the per-drive `pct set -mpN` bind). A SINGLE permanent parent bind
|
||||
// `/mnt/felhom-drives` is set into the guest once (at provision/migration); the host keeps that dir a
|
||||
// SHARED mount, and the agent mounts/unmounts each drive's felhom-data namespace UNDERNEATH it host-side
|
||||
// (`mount --bind /mnt/<name>/felhom-data /mnt/felhom-drives/<name>`). Mount propagation (host `shared` →
|
||||
// guest `slave`) carries the change into the RUNNING guest live — no `pct`, no reboot, and the parent
|
||||
// bind source never disappears (so the guest is inherently C1-immune). Confinement holds: only the
|
||||
// felhom-data subtree crosses in, never the customer's other top-level dirs. See
|
||||
// felhom.eu/documentation/audits/SPIKE-intermediary-mount-2026-06-15.md.
|
||||
|
||||
// StableParentDir is the permanent host dir bound once into the guest; drives are swapped underneath it.
|
||||
const StableParentDir = "/mnt/felhom-drives"
|
||||
|
||||
// sharedParentScript re-establishes the shared parent on every HOST boot. It MUST run before
|
||||
// pve-guests.service so the guest's parent bind inherits the shared peer group as `slave` (if the guest
|
||||
// starts first, its bind is `private` and drive swaps don't propagate until a guest restart).
|
||||
const sharedParentScriptPath = "/usr/local/sbin/felhom-shared-parent.sh"
|
||||
const sharedParentScript = `#!/bin/sh
|
||||
# felhom stable drive parent: a SHARED bind so the agent can swap backing drives underneath it and the
|
||||
# guest sees the change live (no restart). MUST run before pve-guests so the guest's parent bind inherits
|
||||
# the shared peer group (slave). Installed + enabled by felhom-agent. Idempotent.
|
||||
set -e
|
||||
mkdir -p ` + StableParentDir + `
|
||||
mountpoint -q ` + StableParentDir + ` || mount --bind ` + StableParentDir + ` ` + StableParentDir + `
|
||||
mount --make-shared ` + StableParentDir + `
|
||||
`
|
||||
|
||||
const sharedParentUnitPath = "/etc/systemd/system/felhom-shared-parent.service"
|
||||
const sharedParentUnit = `[Unit]
|
||||
Description=Felhom stable drive parent (shared bind for live drive hot-swap)
|
||||
DefaultDependencies=no
|
||||
After=local-fs.target
|
||||
Before=pve-guests.service
|
||||
ConditionPathExists=` + sharedParentScriptPath + `
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=yes
|
||||
ExecStart=` + sharedParentScriptPath + `
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
`
|
||||
|
||||
// StablePathForRaw maps a drive's RAW host mount (/mnt/<name>) to its stable in-guest path
|
||||
// (/mnt/felhom-drives/<name>). The basename is the drive name — the single source of truth both repos
|
||||
// derive the guest path from. Returns "" if `where` is not a /mnt/<name> path.
|
||||
func StablePathForRaw(where string) string {
|
||||
name := DriveNameFromRaw(where)
|
||||
if name == "" {
|
||||
return ""
|
||||
}
|
||||
return StableParentDir + "/" + name
|
||||
}
|
||||
|
||||
// DriveNameFromRaw returns the drive name from a raw /mnt/<name> host mount (the basename), or "" if the
|
||||
// path isn't a single-component /mnt/<name>.
|
||||
func DriveNameFromRaw(where string) string {
|
||||
if !strings.HasPrefix(where, "/mnt/") {
|
||||
return ""
|
||||
}
|
||||
name := strings.TrimPrefix(where, "/mnt/")
|
||||
if name == "" || strings.ContainsAny(name, "/ \t") {
|
||||
return ""
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
// EnsureSharedParent makes the host stable parent a SHARED mount and installs+enables the boot-time
|
||||
// systemd unit that re-establishes it before pve-guests. Idempotent: it only binds when the dir isn't
|
||||
// already a mountpoint (re-binding would stack), and always (re-)marks it shared (a no-op when already
|
||||
// shared). Best-effort install of the unit (a host-reboot-persistence concern) — a failed install does
|
||||
// not stop the live setup. Called at agent startup and at provision.
|
||||
func (b *GuestBinder) EnsureSharedParent(ctx context.Context) error {
|
||||
if err := b.run(ctx, "mkdir", "-p", StableParentDir); err != nil {
|
||||
return fmt.Errorf("shared-parent: mkdir %s: %w", StableParentDir, err)
|
||||
}
|
||||
if !isHostMountpoint(StableParentDir) {
|
||||
if err := b.run(ctx, "mount", "--bind", StableParentDir, StableParentDir); err != nil {
|
||||
return fmt.Errorf("shared-parent: self-bind: %w", err)
|
||||
}
|
||||
}
|
||||
if err := b.run(ctx, "mount", "--make-shared", StableParentDir); err != nil {
|
||||
return fmt.Errorf("shared-parent: make-shared: %w", err)
|
||||
}
|
||||
if err := b.installSharedParentUnit(ctx); err != nil {
|
||||
b.logger.Warn("shared-parent: boot-persistence unit install failed (live setup OK; survives until host reboot)", "err", err)
|
||||
}
|
||||
b.logger.Info("shared-parent: host stable parent is shared", "dir", StableParentDir)
|
||||
return nil
|
||||
}
|
||||
|
||||
// installSharedParentUnit writes the script + unit (from agent-written temps) and enables the unit so the
|
||||
// shared parent is re-established on every host boot before pve-guests. Idempotent.
|
||||
func (b *GuestBinder) installSharedParentUnit(ctx context.Context) error {
|
||||
tmpScript := filepath.Join(os.TempDir(), "felhom-shared-parent.sh")
|
||||
if err := os.WriteFile(tmpScript, []byte(sharedParentScript), 0o755); err != nil {
|
||||
return fmt.Errorf("write temp script: %w", err)
|
||||
}
|
||||
defer os.Remove(tmpScript)
|
||||
if err := b.run(ctx, "install", "-m", "0755", "--", tmpScript, sharedParentScriptPath); err != nil {
|
||||
return fmt.Errorf("install script: %w", err)
|
||||
}
|
||||
tmpUnit := filepath.Join(os.TempDir(), "felhom-shared-parent.service")
|
||||
if err := os.WriteFile(tmpUnit, []byte(sharedParentUnit), 0o644); err != nil {
|
||||
return fmt.Errorf("write temp unit: %w", err)
|
||||
}
|
||||
defer os.Remove(tmpUnit)
|
||||
if err := b.run(ctx, "install", "-m", "0644", "--", tmpUnit, sharedParentUnitPath); err != nil {
|
||||
return fmt.Errorf("install unit: %w", err)
|
||||
}
|
||||
if err := b.run(ctx, "systemctl", "daemon-reload"); err != nil {
|
||||
return fmt.Errorf("daemon-reload: %w", err)
|
||||
}
|
||||
if err := b.run(ctx, "systemctl", "enable", "felhom-shared-parent.service"); err != nil {
|
||||
return fmt.Errorf("enable unit: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// AttachDrive binds a drive's felhom-data namespace under the stable parent so it appears live in the
|
||||
// guest at the returned stable path (via propagation — no pct, no reboot). `where` is the drive's RAW
|
||||
// host PVE mount (/mnt/<name>); only `<where>/felhom-data` crosses into the guest (confinement). The
|
||||
// stable per-drive dir is created HOST-ROOT-owned (fail-closed when nothing is mounted under it); the
|
||||
// felhom-data namespace is created+chowned to the guest base so the in-guest controller owns it.
|
||||
// Idempotent: if the stable path is already a mountpoint, it's a no-op.
|
||||
func (b *GuestBinder) AttachDrive(ctx context.Context, where string) (string, error) {
|
||||
stable := StablePathForRaw(where)
|
||||
if stable == "" {
|
||||
return "", fmt.Errorf("guest-attach: %q is not a /mnt/<name> mount", where)
|
||||
}
|
||||
src := where + "/" + felhomDataNS
|
||||
// Ensure the namespace exists + is owned by the guest base (same as the legacy AttachBind).
|
||||
if err := b.run(ctx, "mkdir", "-p", src); err != nil {
|
||||
return "", fmt.Errorf("guest-attach: namespace %s: %w", src, err)
|
||||
}
|
||||
if err := b.run(ctx, "chown", guestMappedRoot, src); err != nil {
|
||||
return "", fmt.Errorf("guest-attach: chown namespace %s: %w", src, err)
|
||||
}
|
||||
// The stable mountpoint dir stays HOST-ROOT-owned (fail-closed) — create it, never chown it.
|
||||
if err := b.run(ctx, "mkdir", "-p", stable); err != nil {
|
||||
return "", fmt.Errorf("guest-attach: stable dir %s: %w", stable, err)
|
||||
}
|
||||
if isHostMountpoint(stable) {
|
||||
b.logger.Info("guest-attach: already bound under parent (idempotent)", "where", where, "stable", stable)
|
||||
return stable, nil
|
||||
}
|
||||
if err := b.run(ctx, "mount", "--bind", src, stable); err != nil {
|
||||
return "", fmt.Errorf("guest-attach: bind %s -> %s: %w", src, stable, err)
|
||||
}
|
||||
b.logger.Info("guest-attach: drive bound under shared parent (live, no reboot)", "where", where, "stable", stable)
|
||||
return stable, nil
|
||||
}
|
||||
|
||||
// DetachDrive unmounts a drive's felhom-data from the stable parent (propagates OUT of the guest live),
|
||||
// leaving the bare HOST-ROOT-owned stable dir → fail-closed (the guest can't write to it even as root,
|
||||
// since host uid 0 is unmapped). No pct, no reboot. Idempotent: a non-mountpoint is a no-op.
|
||||
func (b *GuestBinder) DetachDrive(ctx context.Context, where string) error {
|
||||
stable := StablePathForRaw(where)
|
||||
if stable == "" {
|
||||
return fmt.Errorf("guest-detach: %q is not a /mnt/<name> mount", where)
|
||||
}
|
||||
if !isHostMountpoint(stable) {
|
||||
return nil // already detached
|
||||
}
|
||||
if err := b.run(ctx, "umount", stable); err != nil {
|
||||
return fmt.Errorf("guest-detach: umount %s: %w", stable, err)
|
||||
}
|
||||
b.logger.Info("guest-detach: drive unmounted from shared parent (live, fail-closed)", "where", where, "stable", stable)
|
||||
return nil
|
||||
}
|
||||
|
||||
// isHostMountpoint reports whether path is currently a mount target in the host's mount table
|
||||
// (/proc/self/mountinfo). Pure read — used for idempotency (skip re-binding) and the BoundUnderParent
|
||||
// report. A read error → false (treat as not-mounted; AttachDrive then (re)binds, which is safe).
|
||||
func isHostMountpoint(path string) bool {
|
||||
f, err := os.Open("/proc/self/mountinfo")
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
defer f.Close()
|
||||
sc := bufio.NewScanner(f)
|
||||
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
|
||||
for sc.Scan() {
|
||||
// mountinfo field 5 (0-indexed 4) is the mount point.
|
||||
fields := strings.Fields(sc.Text())
|
||||
if len(fields) >= 5 && fields[4] == path {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
package localapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"log/slog"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
|
||||
)
|
||||
|
||||
func TestStablePathForRaw_DriveName(t *testing.T) {
|
||||
cases := []struct {
|
||||
where, name, stable string
|
||||
}{
|
||||
{"/mnt/felhom-usb", "felhom-usb", "/mnt/felhom-drives/felhom-usb"},
|
||||
{"/mnt/felhom-flash", "felhom-flash", "/mnt/felhom-drives/felhom-flash"},
|
||||
{"/mnt/felhom-drives/x", "", ""}, // nested (slash in remainder) → not a /mnt/<name> raw mount
|
||||
{"/var/lib/vz", "", ""}, // not under /mnt
|
||||
{"/mnt/", "", ""}, // empty name
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := DriveNameFromRaw(c.where); got != c.name {
|
||||
t.Errorf("DriveNameFromRaw(%q) = %q, want %q", c.where, got, c.name)
|
||||
}
|
||||
if got := StablePathForRaw(c.where); got != c.stable {
|
||||
t.Errorf("StablePathForRaw(%q) = %q, want %q", c.where, got, c.stable)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestDisks_GuestPathAndBoundUnderParent asserts the intermediary reporting the controller relies on: a
|
||||
// user-data drive gets a STABLE guest path (/mnt/felhom-drives/<name>) and BoundUnderParent reflects the
|
||||
// host mount check; a SYSTEM mount gets neither (it never crosses into the guest).
|
||||
//
|
||||
// COMPANION GUARD: a pre-fix impl that left GuestPath empty (no repoint signal) or that reported
|
||||
// BoundUnderParent ignoring the real mount state would fail the assertions below.
|
||||
func TestDisks_GuestPathAndBoundUnderParent(t *testing.T) {
|
||||
d := &fakeDiskOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4"}}
|
||||
sv := fakeStorage{targets: []hub.StorageTarget{
|
||||
{Name: "usb", Type: hub.StorageTypeUSB, BackingDevice: "/dev/sdb1", MountPath: "/mnt/felhom-usb"},
|
||||
{Name: "flash", Type: hub.StorageTypeUSB, BackingDevice: "/dev/sdc1", MountPath: "/mnt/felhom-flash"},
|
||||
{Name: "system", Type: "local", MountPath: "/var/lib/vz"}, // system role → no guest path
|
||||
}}
|
||||
fg := &fakeGuestsCfg{}
|
||||
srv, err := NewServer(Options{
|
||||
ListenAddr: "127.0.0.1:0", Guests: fg, Backups: &fakeBackups{}, Store: &fakeStore{},
|
||||
Storage: sv, Tokens: staticTokens{"A": 8200},
|
||||
Disks: d, DiskGate: &fakeGate{}, HostReader: sysOnSDA(),
|
||||
Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
srv.baseCtx = context.Background()
|
||||
// felhom-usb is bound under the parent; felhom-flash is not.
|
||||
srv.boundCheck = func(p string) bool { return p == "/mnt/felhom-drives/felhom-usb" }
|
||||
|
||||
disks := decodeDisks(t, do(t, srv.Handler(), "GET", "/disks", "A", "").Body.Bytes())
|
||||
byMount := map[string]DiskInfo{}
|
||||
for _, di := range disks {
|
||||
byMount[di.MountPath] = di
|
||||
}
|
||||
|
||||
if gp := byMount["/mnt/felhom-usb"].GuestPath; gp != "/mnt/felhom-drives/felhom-usb" {
|
||||
t.Errorf("usb GuestPath = %q, want /mnt/felhom-drives/felhom-usb", gp)
|
||||
}
|
||||
if !byMount["/mnt/felhom-usb"].BoundUnderParent {
|
||||
t.Errorf("usb should report BoundUnderParent=true (bound under the parent)")
|
||||
}
|
||||
if byMount["/mnt/felhom-flash"].BoundUnderParent {
|
||||
t.Errorf("flash should report BoundUnderParent=false (not bound)")
|
||||
}
|
||||
if gp := byMount["/mnt/felhom-flash"].GuestPath; gp != "/mnt/felhom-drives/felhom-flash" {
|
||||
t.Errorf("flash GuestPath = %q, want /mnt/felhom-drives/felhom-flash", gp)
|
||||
}
|
||||
// A SYSTEM mount must never get a guest path (it does not cross into the guest).
|
||||
if gp := byMount["/var/lib/vz"].GuestPath; gp != "" {
|
||||
t.Errorf("system mount GuestPath = %q, want empty (never crosses into the guest)", gp)
|
||||
}
|
||||
}
|
||||
@@ -171,6 +171,10 @@ type Server struct {
|
||||
// tests override it. (DiskInfo.DurableID stays the uuid: storage id — that one feeds /disks/assign.)
|
||||
deviceDurableID func(device string) (string, error)
|
||||
|
||||
// boundCheck reports whether a stable guest path has felhom-data bound under it (intermediary model).
|
||||
// Optional — nil defaults to the real host mount-table read (isHostMountpoint); tests inject a fake.
|
||||
boundCheck func(string) bool
|
||||
|
||||
jobsMu sync.Mutex
|
||||
jobs map[int]*backupJob // per-guest backup job state (slice 8B)
|
||||
|
||||
|
||||
@@ -31,6 +31,13 @@ const (
|
||||
// mappedRoot is the unprivileged-LXC host uid/gid that maps to the guest's root (spike
|
||||
// gotcha 1): files chowned to this appear as root:root 0600 inside the guest.
|
||||
mappedRoot = "100000:100000"
|
||||
// stableParentDir is the permanent host dir bound once into the guest (intermediary mount model);
|
||||
// the agent swaps drive felhom-data namespaces underneath it host-side. Mirrors
|
||||
// localapi.StableParentDir (kept as a literal to avoid a provision→localapi import edge).
|
||||
stableParentDir = "/mnt/felhom-drives"
|
||||
// parentBindSlot is the dedicated mpN for the single permanent parent bind. mp8 — high enough to not
|
||||
// collide with bring-up data mounts (mp0..), below the mp9 bootstrap.
|
||||
parentBindSlot = "mp8"
|
||||
)
|
||||
|
||||
// BackHalf populates a guest's bootstrap config mount host-side (F3). It mints the per-guest
|
||||
@@ -155,6 +162,20 @@ func (b *BackHalf) Provision(ctx context.Context, in Input) (Result, error) {
|
||||
b.logger.Warn("provision: pre-start hook registration failed (non-fatal)", "vmid", in.VMID, "err", err)
|
||||
}
|
||||
|
||||
// 7. Intermediary mount model: add the ONE permanent parent bind so enrolled data drives appear in
|
||||
// the guest LIVE (the agent later binds each drive's felhom-data UNDER /mnt/felhom-drives host-side
|
||||
// — no per-drive mp, no reboot). The host SHARED state + boot-persistence unit are the agent
|
||||
// daemon's job (EnsureSharedParent at startup); here we ensure the dir exists (pct validates the
|
||||
// bind source) and attach the parent bind. Best-effort + non-fatal.
|
||||
if err := b.run(ctx, "mkdir", "-p", stableParentDir); err != nil {
|
||||
b.logger.Warn("provision: stable parent dir create failed (non-fatal)", "vmid", in.VMID, "err", err)
|
||||
} else {
|
||||
parentSpec := fmt.Sprintf("%s,mp=%s", stableParentDir, stableParentDir)
|
||||
if err := b.run(ctx, "pct", "set", strconv.Itoa(in.VMID), "-"+parentBindSlot, parentSpec); err != nil {
|
||||
b.logger.Warn("provision: parent bind attach failed (non-fatal)", "vmid", in.VMID, "slot", parentBindSlot, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
b.logger.Info("provision: back-half complete",
|
||||
"vmid", in.VMID, "mount", mountKey, "guest_path", guestPath, "endpoint", in.Endpoint)
|
||||
// tok intentionally goes out of scope here — never logged, never returned.
|
||||
|
||||
Reference in New Issue
Block a user