Files
felhom-agent/REUSE.md
T
Claude Code 5bca7bfc9a R-85 Phase 4: finalise v0.104.0 + register the new helpers in REUSE
Version finalised from v0.104.0-dev — an ldflags version disagreeing with the
CHANGELOG is the reconciliation problem hub 0.73.2 already caused.

REUSE gains backup.InFlight, backup.RestoreTestState and the SpecBuilder/
TierPicker/PickRestoreCandidateOn trio, each with the decision someone could
unknowingly undo: rotation credit only on success; InFlight callers defer and
never cancel; a tier with no archive no-ops rather than failing; SourceTier
comes from the archive, never the configured target.
2026-07-27 07:32:55 +02:00

37 KiB
Raw Blame History

REUSE.md — felhom-agent

Before writing new code, check here. Canonical helpers, patterns to copy, traps to avoid. Maintenance: update in the SAME commit that adds/changes/deprecates a shared helper. Entries cite file + symbol. Line numbers are landmarks only — reconfirm before editing.

1. Canonical helpers (MUST reuse — do not reinvent)

Allowlisted exec / privileged surface (sudoers)

Symbol File Short signature Use for Gotchas
Runner / ExecRunner.Run, RunStdin internal/proxmox/privileged.go Run(ctx, name, args...) (stdout, stderr []byte, err) ALL host command exec (direct or sudo -n prefix) Arg vectors only, never a shell string; capBuf caps output at 1 MiB
Privileged (CreateGoldenLXC/MountUSBByUUID/SMART/Sensors) internal/proxmox/privileged.go methods on *Privileged the 3 fenced root-CLI exceptions ONLY Do NOT add methods — fence is structural (routing_test.go asserts it)
SudoHostOps.run internal/storage/hostops.go run(ctx, name, args...) error allowlisted exec with stderr-wrapped error Every arg pre-validated via validate.go before this is called
Prober.Probe internal/capability/probe.go Probe(ctx) []Status live sudo-policy capability check (sudo -n -l --) Needs a DIRECT runner (never the sudo-prefixing one — double-sudo); never executes probed cmds. v0.86.0: config-gated caps (Capability.GatedBy + Prober.GateActive) report inactive/"disabled by configuration" ONLY when healthy — broken plumbing stays degraded; the pbsdr-* gate answers from pbsdr.Manager.DRConfigured (marker-backed across restarts)
stageTemp internal/localapi/intermediary.go stageTemp(pattern, content) (path, err) random-named temp before a root install (audit B1) Fixed /tmp names are a TOCTOU — sudoers globs expect /tmp/felhom-*-*.ext
guesthook.InstallSnippet / Register internal/guesthook/install.go InstallSnippet(ctx, runner) error pre-start self-heal hook install (C1 net) Same random-temp+install pattern; snippet delegates to the agent binary (no shell logic). Issues mkdir -p /var/lib/vz/snippets FIRST (v0.63.0, B2 — fresh boxes lack the dir; sudoers grants exactly that argv)

Disk / format safety (role gates, durable IDs, format guards)

Symbol File Short signature Use for Gotchas
SudoHostOps.Format internal/storage/hostops.go Format(ctx, device, fstype) error THE only mkfs path Guards, in order: ValidateBlockDevice + ValidateFSType → mandatory deviceUnclaimed (claim.go) → exec felhom-mkfs-guarded (sudoers allowlists ONLY the wrapper, not raw mkfs)
SudoHostOps.InspectDevice + DeviceProbe.DataBearing internal/storage/hostops.go InspectDevice(ctx, device) (DeviceProbe, error) data-bearing verdict from the AGENT's own read Fail-safe: Probed=false ⇒ DataBearing=true; blkid output is evidence, lsblk is read-success authority
classifyClaim / SudoHostOps.deviceUnclaimed internal/storage/claim.go classifyClaim(claimFacts) (unclaimed bool, reason string) "is this disk provably free to format" Pure function of gatherClaimFacts; ANY read error/ambiguity/empty-lsblk ⇒ CLAIMED (audit D2); Felhom's own /mnt/felhom-drives mounts are not a foreign claim
SudoHostOps.ListCandidateDisks internal/storage/candidates.go ListCandidateDisks(ctx) ([]CandidateDisk, error) enroll-candidate discovery Fail-safe: omits anything not provably unclaimed
antiRetargetResolveExpect (+ antiRetargetResolve, antiRetargetResolveBlank) internal/localapi/wipe_reresolve.go (durableID, expectDataBearing, resolve, derive, inspect) (device, err) pre-mkfs anti-retarget: resolve durable id → re-derive+match → re-inspect AGENT-001 + audit D3; refuses path-only bindings; wired via Server.reresolveWipe/reresolveBlank (test-injectable)
signedjobs.WipeExecutor.Execute internal/signedjobs/wipe.go Execute(ctx, op, params) error operator-signed data-bearing wipe Durable-id bound; nonce burned by gate BEFORE execute; refuses no-longer-data-bearing targets
selfupdate.Executor / selfupdate.Manager internal/selfupdate/{executor,commit}.go NewExecutor(Config) / NewManager(ManagerConfig) operator-signed agent self-update (D1): download+verify-vs-signed-sha → wrapper apply; startup dwell → commit sha is the ONLY integrity root; wrapper (felhom-selfupdate-guarded) re-verifies as root + does the A/B flip; NEVER rolls back (systemd + wrapper do). WrapperRunner seam. Report seam SelfUpdatePending()
Gate.AuthorizeStorageWipe internal/reconcile/gate.go AuthorizeStorageWipe(StorageWipeAuthz, *SignedOp) Decision tiered wipe authz user-data ⇒ customer confirm bound to agent's DeviceDurableID; system/backup ⇒ operator signature only, Confirmed IGNORED by role
Gate.Authorize internal/reconcile/gate.go Authorize(Intent, *SignedOp) Decision every destructive intent role-scoping (roleAuthorizes) + op-to-action binding; benign passes unsigned; audits every decision
storage.DeviceDurableID / ResolveDurableDevice internal/storage/durable_device.go DeviceDurableID(device) (string, error) WIPE-binding ids (byid:/byuuid:) Single seam for /disks list AND gate (F20-BUG2); ResolveDurableDevice refuses bare paths
storage.ResolveStorageDevice internal/storage/durable_device.go ResolveStorageDevice("uuid:<fs-uuid>") (dev, err) re-resolve enrolled STORAGE drives (remount) uuid: scheme ONLY — distinct from the wipe schemes; never trust a remembered /dev node
deriveDurableID internal/storage/durableid.go deriveDurableID(typ, s, backingDevice, uuid) string storage-target durable id (DR re-attach key) Deterministic per type; uuid: for usb/local-dir; PBS id includes #<fingerprint>
SystemDisks / isSystemBacked / RoleForStorage / RoleForRawDevice internal/storage/role.go RoleForRawDevice(device, sysDisks, sysKnown) DeviceRole protection-tier classification Fails safe to system (most protected) on any ambiguity; role is AGENT-derived, never caller-supplied
ValidateUUID/MountPath/BlockDevice/FSType/SMARTDevice/LVMName, UnitNameForMount internal/storage/validate.go Validate*(v) error EVERY arg that reaches a root shell-out The security boundary; strict whitelists (no by-* symlinks, no dm, no traversal); systemdEscapePath computed in-process
ValidateNetworkMountSpec internal/storage/netmount.go ValidateNetworkMountSpec(spec) error NAS mount input boundary Same discipline as validate.go; SMB requires a creds ref; mountpoint confined under NetworkMountRoot

Mount lifecycle (host + guest binds)

Symbol File Short signature Use for Gotchas
SudoHostOps.EnsureMount internal/storage/hostops.go EnsureMount(ctx, MountSpec) error persistent by-UUID systemd .mount Validate→render→stage→installenable --now; idempotent
SudoHostOps.Unmount internal/storage/hostops.go Unmount(ctx, where) error detach a mount unit DESTRUCTIVE — caller MUST have gated it; does not self-authorize
SudoHostOps.ReassertEnrolledMounts internal/storage/hostops.go ReassertEnrolledMounts(ctx) reboot remount (re-resolve by UUID) Re-asserts unless mounted AND enabled (shouldReassertMount); skips absent UUIDs
GuestBinder.AttachDrive / DetachDrive internal/localapi/intermediary.go AttachDrive(ctx, vmid, where) (guestPath, err) live drive hot-swap under /mnt/felhom-drives Normalizes to EXACTLY ONE bind via countHostMounts (converges double-binds); force re-bind when guest can't see it
GuestBinder.EnsureSharedParent internal/localapi/intermediary.go EnsureSharedParent(ctx) error shared-parent bind + boot unit make-private+make-shared ONLY on first bind — re-running orphans the guest's slave; F2-a: compares script AND unit for staleness
StablePathForRaw / DriveNameFromRaw internal/localapi/intermediary.go StablePathForRaw("/mnt/<n>") string raw host mount → stable guest path Single source of truth both repos derive from
GuestBinder.GuestSeesMount / GuestBootID internal/localapi/intermediary.go GuestSeesMount(ctx, vmid, path) bool guest-visible (usable) signal; reboot detection Host bind present ≠ guest sees it (non-recursive parent bind)
SudoHostOps.EnsureNetworkMount / RemoveNetworkMount / ListNetworkMounts internal/storage/netmount.go EnsureNetworkMount(ctx, spec) error NAS automount pair rm glob confined to mnt-felhom* units; NAS ≠ drive (no durable-id/SMART/wipe); RemoveNetworkMount doubles as the verify-fail rollback (idempotent)
NetworkMountedAt / NetworkEndpointReachable internal/storage/netmount.go NetworkMountedAt(where) bool verify mount-truth + the 2 s add pre-probe /proc/mounts is the ONLY mount-success judge (autofs trigger ≠ mounted; readability ≠ mounted — SPIKE-nas-verify §8)
SudoHostOps.ReassertNetworkAutomounts + Server.ReassertNetworkMounts + guesthook.PostStartNetworkReassert internal/storage/netreassert.go, internal/localapi/netreassert.go, internal/guesthook/netreassert.go ReassertNetworkAutomounts(ctx) []NetReassertResult NAS guest-reboot heal (RCA fix 1): re-arm idle automount triggers (stop + enable --now) so the fresh mount event propagates into running guests NEVER call from periodic health paths (an idle trigger is HEALTHY); active real mounts are never touched; hook leg runs as root (direct systemctl), daemon leg via sudo
ClassifyNetVerifyFailure internal/storage/netverify.go ClassifyNetVerifyFailure(journalTail, tcpReachable) (code, hint) NAS verify failure categories String-based BY DESIGN (every mount failure is rc=32); substrings verbatim from SPIKE-nas-verify Q4; nfs_export merges not-found/not-permitted (NFSv4 identical)

Durable stores (atomic state)

Symbol File Short signature Use for Gotchas
IntentStore (Get/SetEnrolled/SetEjected/SetDecommissioned/OnAbsent) internal/storage/intent.go OpenIntentStore(path) drive intent (4-state self-heal) Keyed by durable-id only; OnAbsent is the ONLY ejected→enrolled path; refuses empty ids
GuestBindStore (Record/Remove/Guests) internal/localapi/guestbindstore.go OpenGuestBindStore(path) per-guest enrolled binds (F9 re-assert) Same tmp+rename 0600 pattern as IntentStore
FormatJobStore + startFormatDetached + RecoverFormatJob internal/localapi/formatjob.go startFormatDetached(device, durableID, fstype, blank) <-chan error detached, restart-surviving mkfs (F20-BUG3) Runs off s.baseCtx (60-min bound) so a request deadline can't SIGKILL mkfs; recovery re-resolves by durable id; blank jobs re-check STILL-blank
TokenStore.Mint / Lookup internal/localapi/tokenstore.go Mint(vmid) (plaintext, error) per-guest local-API tokens Only the SHA-256 hash persists (fsync'd append log); constant-time compare on lookup; plaintext returned exactly once. Lookup RELOADS the file once on a miss (v0.63.0, B3): the one-shot provisioner mints into the same file the daemon indexes — cross-process coherence without a restart; append-only size check bounds the re-read
FileNonceStore.SeenOrRecord internal/authz/noncestore.go SeenOrRecord(nonce, exp) bool durable anti-replay fsync'd before returning false; prune only after exp
Journal (Append/Latest/InFlight/AlreadyApplied) internal/reconcile/journal.go OpenJournal(path) op journal + idempotency + crash recovery Recover consumes InFlight(); scratch entries special-cased

Local-API plumbing

Symbol File Short signature Use for Gotchas
Server.withGuest internal/localapi/server.go withGuest(fn(w, r, vmid)) http.HandlerFunc bearer auth + self-scoping for EVERY route Token→vmid is the authority; explicit ?vmid= only a consistency check (403 on mismatch)
Server.scopedFromBody internal/localapi/server.go scopedFromBody(w, bodyVMID, tokenVMID, path) bool POST-body vmid self-scope check Call right after decode; false = already 403'd
decodeBody / writeOK / writeErr / writeStatus internal/localapi/server.go writeStatus(w, code, ok, data, errMsg) ALL local-API JSON I/O Do not hand-roll response envelopes
EnsureLeaf internal/localapi/cert.go EnsureLeaf(certPath, keyPath, host) (cert, fingerprint, generated, err) pinned self-signed leaf generated=true invalidates every issued bootstrap pin — log LOUD (B.1)
Server.RecoverStaleLockedGuests internal/localapi/stalelock.go RecoverStaleLockedGuests(ctx) startup stale vzdump-lock heal (F2-b) Clears ONLY backup/snapshot-delete, only when no vzdump in-flight; A1 RESOLVED (v0.62.0): scan is pool-intersected (ListLXCClient.Pool), fail-safe skip on pool-read failure
ControllerSwapper.Swap + ValidControllerImage internal/localapi/controllerswap.go Swap(ctx, vmid, target) *ControllerSwapState agent-owned controller image swap + rollback Strict image regex (repo + 3-part semver); state file written BEFORE swap; no-healthcheck images need verifyDwell
MemoryOps + Server.readMemoryBounds internal/localapi/guestmemory.go readMemoryBounds(ctx, vmid) (memoryBounds, err) guest RAM resize (v0.90.0, R-24): GET/POST /guest/memory NEW narrow seam (never extend GuestAPI — it breaks every fake); the AGENT is the boundary — bounds recomputed FRESH per request (min 2048 / max host_total2048 / shrink floor max(2048, usage+512)); §8 UNITS TRAP (config memory=MB, status/node=bytes); verify maxmem==target after SetConfig before claiming success; SetConfig NEVER called on a refusal path

Proxmox client / hub / PBS / provisioning

Symbol File Short signature Use for Gotchas
Client.WaitTask internal/proxmox/task.go WaitTask(ctx, upid, opts) (TaskStatus, error) asserting EVERY mutating op POST 200 ≠ success; authz can fail at task exec; AllowWarnings opt-in
Client.Pool internal/proxmox/query.go Pool(ctx, name) (PoolInfo, error) felhom-pool membership (the ownership registry, A1) Needs Pool.Audit at /pool/<name> (host-install v1.9.0+); Pool.Allocate does NOT satisfy the read; members can be storages (type storage, vmid 0) — filter them
Client mutate wrappers (RestoreLXC/Vzdump/DestroyLXC/Snapshot/Rollback/SetConfig/ResizeLXC/Start/Stop) internal/proxmox/mutate.go return (upid, error) all API mutations Async → always pair with WaitTask; route via gate/queue, not ad-hoc
Client.PoolAddVMID internal/proxmox/mutate.go PoolAddVMID(ctx, pool, vmid) error re-assert pool membership after a restore-over-existing (campaign-2 R2) SYNC (no UPID, don't WaitTask); PVE PUT /pools is additive (merge, not replace) — delete=1 removes; idempotent (already-member swallowed); needs Pool.Allocate at /pool/<pool>. pct restore --pool sets membership only at CREATE — a restore over an existing vmid drops it, so bring-up re-asserts post-restore
TLSConfig.build / normalizeFingerprint internal/proxmox/tls.go build() (*tls.Config, error) PVE leaf-cert SHA-256 pinning No insecure default
pinnedTLS internal/pbs/pin.go pinnedTLS(fingerprint) (*tls.Config, error) PBS leaf pinning Same model as PVE; 64-hex fingerprint normalized
hub.Client.Report internal/hub/client.go Report(ctx, *HostReport) (*ControlEnvelope, error) the heartbeat Typed TransportError/HTTPError, never contain the bearer token
hub.Loop + MultiObserver internal/hub/loop.go NewLoop(...); MultiObserver(obs...) resilient report loop + envelope fan-out Errors logged, loop continues; interval clamped 603600 s
provision.BackHalf.Provision internal/provision/backhalf.go Provision(ctx, Input) (Result, error) guest bootstrap back-half mint→render→0600 write→chown 100000:100000→pct set ro bind→onboot; token NEVER logged/returned. Bootstrap local_api.endpoint = the caller's cfg.LocalAPI.ListenAddr (main.go) — moving the agent bind to the island moves the guest dial for free (R-50, no template)
buildBringUpConfig island NIC internal/reconcile/bringup.go (pure) BringUpSpec{IslandBridge,IslandGuestAddr}params["net1"] R-50 island control plane When BOTH island fields are set (from cfg.LocalAPI), attaches a static net1=name=eth1,bridge=<vmbr9>,ip=<.2/30> (no hwaddr → fresh MAC), so the controller reaches the agent over a fixed private address immune to LAN/DHCP/site moves. Empty = pre-R-50, no net1. All-or-nothing + CIDR enforced in LocalAPIConfig.Validate. The guestnet healer is eth0-only (parseMode is dev-scoped) so it never touches the static island NIC
reconcile.Queue.Submit internal/reconcile/queue.go Submit(vmid, fn) <-chan error per-guest serialization of ALL mutations Same vmid strictly FIFO; lanes parallel across guests
Engine.RunSignedJob internal/reconcile/job.go RunSignedJob(ctx, intent, signed, exec) JobResult executing a gated destructive job Idempotency by nonce; journaled
escrow.Create internal/escrow/escrow.go Create(ctx, CreateOptions) (CreateResult, R, error) PBS-key escrow (zero-knowledge) Recovery code returned SEPARATELY from the result (anti-log); self-verifies recoverability
escrow.GenerateRecoveryCode / joinSafe / RecoveryCodeSep internal/escrow/wordlist.go GenerateRecoveryCode() (string, error) minting the customer recovery code R Draws from the EFF large list filtered of every word containing RecoveryCodeSep (4 entries: drop-down, felt-tip, t-shirt, yo-yo) so a code always segments back into exactly 10 words — a hyphenated word made codes ambiguous to transcribe AND flaked the test ~1/5 (v0.93.0). Generation-only: already-issued codes stay valid, R is verified as a whole passphrase and never re-split. Never count words by splitting the joined string — count what the generator drew
escrow.CeremonyBinary / CeremonyArgs() / CeremonyOutput internal/escrow/ceremony.go the ONE fixed sudo self-invocation argv + the --output=json wire object (v1) controller-driven ceremony (v0.88.0) SINGLE SOURCE shared by the localapi exec, the capability manifest entry, and (byte-identically) the FELHOM_ESCROW sudoers line — TestEscrowCeremonyArgvPinned + TestManifestCoveredBySudoers lock all three. Never flag-helpers, never --- (spike §2.2)
localapi escrow ceremony job internal/localapi/escrow_ceremony.go POST /escrow/ceremony + status + ONE-SHOT claim + preflight the wizard's agent half R lives ONLY in Server.escrowR (NEVER the job struct — snapshots must be structurally R-free); zeroed on claim/supersede/10-min TTL (unclaimed_void); in-memory BY DESIGN (restart loses R safely; re-run supersedes); subprocess stdout is SECRET-BEARING → parsed then zeroed, never logged
poke.Listener + poke.Port internal/poke/poke.go NewListener(resolve, trigger, port, logger); poke.Port = 51822 agent-plane immediate-sync (Direction-2a, v0.89.0) Binds a contentless UDP socket EXCLUSIVELY to the box's WG /32 (wgtunnel.LoadAssignedAddr), fires the hub-loop out-of-band trigger. Port 51822 is a SHARED cross-repo contract — the hub poke sender + the ep0 felhom-poke forced-command target the SAME number; change one → change all three. Contentless (payload ignored), leading-edge debounced (DebounceWindow), WG-confined (kernel EKEYREJECTED refuses non-peer /32s). Wired only when wg_tunnel.enabled
wgtunnel.LoadAssignedAddr internal/wgtunnel/manager.go LoadAssignedAddr(stateDir) (netip.Addr, bool) the box's own WG /32 without a Manager Reads registered.json; ok=false until registered; strips the /32 → bare addr (the poke bind target)
fasttick.Loop + fasttick.SourceFunc internal/fasttick/fasttick.go New(out chan<- struct{}, interval, logger, sources...); Source.Unconverged() (bool, reason) agent-plane immediacy SECONDARY (v0.90.0, R-28): pulse the SAME out-of-band trigger every 30 s while ANY source is unconverged, self-disarm on convergence STATE-BASED (no timer, nothing to journal). Every source MUST be a CACHED read (no exec/network per tick) — desiredProvider.Generation(), reconcile.Engine.LastResult() (PlannedPending>0), pbsdrLoop.PBSDRStatus().State=="waiting_secret" ONLY, wgtunnel.Manager.TunnelConvergence(). The LOUD pbsdr states + destructive pending_signature are DELIBERATELY EXCLUDED (a stuck-loud box must not hammer). Pulses the cap-1 channel non-blocking (coalesces with poke/watchdog); NEVER touch MinPollSeconds/clampInterval/the ticker
wgtunnel.Manager.TunnelConvergence / reconcile.Engine.LastResult internal/wgtunnel/manager.go, internal/reconcile/engine.go TunnelConvergence() (desired, operational bool); LastResult() (Result, bool) cached convergence snapshots for the fast-tick TunnelConvergence is refreshed at the END of every Apply (its own cadence) so the fast-tick never execs wg/systemctl; LastResult is mutex-recorded per reconcileOnce, ok=false until the first pass

2. Canonical patterns (copy structure from THE named file)

Pattern Canonical file Key traits
Validate-then-exec privileged op internal/storage/hostops.go (EnsureMount) validate EVERY arg (validate.go) → render → stage in agent dir → root installsystemctl; refuse before any command exists
Fail-safe pure classifier over gathered facts internal/storage/claim.go (classifyClaim over claimFacts) pure function ⇒ fixture-testable; any error/ambiguity refuses; gather separated from verdict
Anti-retarget durable-id binding internal/localapi/wipe_reresolve.go resolve id → re-derive + exact match → re-inspect expected state → act on RE-RESOLVED device only
Atomic single-file JSON store internal/storage/intent.go Open* loads (missing=empty, corrupt=fail-loud), mutex, tmp+rename 0600, idempotent set
Durable append-only log + index internal/authz/noncestore.go (FileNonceStore) fsync before returning "new"; replay into index on open; expiry-only compaction
Injectable seam funcs on Server internal/localapi/server.go (reresolveWipe, deviceDurableID, boundCheck, net-verify: netTrigger/netMounted/netJournal/netReachable) prod default wired in NewServer; tests override — no real /dev, /proc/mounts, journalctl or TCP in tests
Detached IN-MEMORY verify job (single slot, deliberately unpersisted) internal/localapi/netverifyjob.go claim slot sync (single-flight 409) → detached pipeline off baseCtx → auto-rollback on fail; restart ⇒ slot empty ⇒ the CALLER rolls back (Scenario F) — contrast formatjob (persisted+recovered)
Optional dependency degradation internal/localapi/server.go (Options) nil dep ⇒ endpoint answers "not configured" (503), never a crash
Version channel (v0.82.0) internal/localapi/server.go (Options.AgentVersion; Handler() mux wrap) sets X-Felhom-Agent-Version on EVERY response (all routes/statuses, incl. auth-fail/404) — the controller's capability-comparison source; empty version ⇒ header omitted
Root-file install via random temp internal/localapi/intermediary.go (installSharedParentUnit) stageTemp (os.CreateTemp) → sudoers-globbed install -m → pinned destination
Detached destructive job + restart recovery internal/localapi/formatjob.go persist running → run off baseCtx → record outcome; recovery re-resolves durable id, never a path
Signed-op verify pipeline internal/authz/verifier.go (Verify) armor→namespace→key-material allowlist→crypto over RAW bytes→blob→target→window→nonce LAST
Resilient daemon loop internal/hub/loop.go (Loop.Run) ctx-cancel returns nil; errors logged and loop continues
Guarded-below-the-agent wrapper configs/felhom-mkfs-guarded.sh root re-checks catastrophic cases (system disk/LVM PV/foreign mount/RO/member FS) even against an agent bug

3. Dangerous lookalikes — do NOT reuse

Trap Why it bites Use instead
Acting on the caller's req.Device (or any remembered /dev path) after inspection classify→mkfs TOCTOU (AGENT-001): /dev re-enumeration retargets the node to a different physical disk Server.reresolveWipe/reresolveBlank → format the RE-RESOLVED device
Exec'ing raw mkfs.* (incl. Binaries.MkfsExt4/MkfsXfs) sudoers no longer allowlists raw mkfs; bypasses the claim filter + wrapper re-checks SudoHostOps.Format (→ felhom-mkfs-guarded)
DiskInfo.DurableID (uuid:) as a wipe-confirmation id wipe gate resolves byid:/byuuid: — a uuid: id is a binding_mismatch (F20-BUG2) DiskInfo.WipeDurableID / storage.DeviceDurableID
ResolveDurableDevice for enrolled-storage remounts (or vice versa) schemes differ: wipe = byid:/byuuid:, storage = uuid: — each refuses the other ResolveStorageDevice for mounts; ResolveDurableDevice for wipes
authz.MemoryNonceStore on a real host replay protection dies on restart authz.FileNonceStore
Adding methods to proxmox.Privileged breaks the 3-exception root-CLI fence (routing_test.go) proxmox.Runner + a new sudoers Cmnd_Alias + validate.go-style checks
Calling Client.DestroyLXC/Vzdump/SetConfig outside the gate/queue/journal skips classification, signature, per-guest serialization, crash recovery reconcile.Engine paths / RunSignedJob; queue via Queue.Submit
GuestBinder.AttachBind/DetachBind (per-drive pct set -mpN) legacy model; a missing bind source can brick guest boot (C1) AttachDrive/DetachDrive (intermediary model)
isHostMountpoint to reconcile bind state boolean can't converge stacked double-binds (the /mnt doubling bug) countHostMounts normalization inside AttachDrive
Acting on a raw ListLXC list as if it were "guests the agent owns" audit A1 (pre-v0.62.0 the stale-lock reaper did exactly this — contained only by the pool-scoped token) ownership must be PROVEN: intersect with Client.Pool membership like staleLockController.Guests() (v0.62.0), fail-safe on read failure

4. Seams & interfaces (testing + cross-repo)

Interface Defined in Implemented by Fakes/tests at
proxmox.Runner internal/proxmox/privileged.go *ExecRunner (direct/sudo) mockRunner internal/proxmox/mock_test.go; runner fakes in storage tests
storage.HostOps internal/storage/hostops.go *SudoHostOps (prod), NoopHostOps (degraded) fakes in internal/storage/observe_test.go, watchdog_test.go
storage.HostReader internal/storage/hostread.go *ProcHostReader fakeHostReader internal/localapi/disks_test.go; internal/storage/role_test.go. v0.87.0: BlockSlaves(name) lists /sys/block/<name>/slaves (root-free) — backs the SystemDisks dm/md walk (physicalDisksOf/walkSlaves, role.go); per-branch conservatism: an unresolvable slave fails the WHOLE walk → all-system fail-safe. NEVER weaken the signature test TestSystemDisks_WalkTopologies (root-backing disk always in the system set).
localapi.DiskOps / StorageGate / GuestAttacher / GuestLister internal/localapi/disks.go *storage.SudoHostOps; storageGateAdapter (cmd/felhom-agent/main.go); *GuestBinder; *proxmox.Client fakeDiskOps/fakeGate/fakeGuestAttacher/fakeGuestList internal/localapi/disks_test.go
localapi.GuestAPI / BackupService / BackupStore / TokenAuthority internal/localapi/server.go *proxmox.Client, *backup.BackupRunner, *backup.Store, *TokenStore fakeGuests/fakeBackups/fakeStore internal/localapi/server_test.go
backup.InFlight internal/backup/inflight.go TryAcquire(what) (release, busy, ok) / Busy() THE host-wide "one heavy guest operation at a time" gate — shared by the local-API backup path and the restore-test scheduler (R-85)
backup.RestoreTestState internal/backup/restoretest_state.go RecordSuccess(target,t) / LastSuccess(target) / OldestFirst(targets) Per-tier restore-test rotation state, persisted (atomic tmp+rename)
backup.SpecBuilder / backup.TierPicker / (*BackupRunner).PickRestoreCandidateOn internal/backup/schedule.go, runner.go func(ctx,archive) RestoreTestSpec; func(ctx,target) (string,error) The per-run restore-test spec + per-tier candidate lookup (R-85)
localapi.BackupTier + normalizeBackupTiers / config.BackupConfig.BackupTiers internal/localapi/backup_tiers.go, internal/config/config.go normalizeBackupTiers(tiers, legacy, cadence) []BackupTier; BackupTiers() ([]BackupTier, []string) THE R-82 multi-tier resolution — one runner per tier, primary first
localapi.StaleLockController internal/localapi/stalelock.go *staleLockController (Client + Runner + pool) fakeStaleLock (Server-level) stalelock_test.go; fakeStaleLockAPI (controller-level, tests the A1 pool intersect) stalelock_pool_test.go
localapi.GuestExecutor internal/localapi/controllerswap.go *GuestBinder (pct exec) fakeGuestExec internal/localapi/controllerswap_test.go
guestnet.Runner / guestnet.GuestSource (R-54, v0.92.0) internal/guestnet/{probe,watchdog}.go *proxmox.ExecRunner; the POOL-VERIFIED localapi.StaleLockController.Guests (ListLXC ∩ felhom pool, audit A1) scriptedRunner + fakeGuests internal/guestnet/watchdog_test.go. Never wire a bare ListLXC here — under a broad token that would run dhclient inside a co-tenant's container. Every assertion is an exec COUNT, and the load-bearing ones are the negatives: a static guest, an unprobeable guest, a boot-race guest and an unproven guest list must record zero heal calls
guestnet.Watchdog.SetDampers / now (clock seam) internal/guestnet/watchdog.go config guest_net.*; now defaults to time.Now tests advance a manual clock (the storage-watchdog pattern) and assert the heal ceilings EXACTLY — ≥10 min apart, ≤3/hour, and ≤30 over a scripted 10 hours of permanent failure. A damper with no test is a comment
hub.GuestNetReporter (R-54) internal/hub/collect.go *guestnet.Watchdog (GuestNetStatus) internal/hub/collect_guestnet_test.go asserts the stanza through the PRODUCTION Collect path AND that the guest_net key is ABSENT from the wire when no reporter is wired — an always-present empty stanza would make "not wired" and "found nothing" the same signal, which is the shape v0.91.0 hid behind
reconcile.OpVerifier internal/reconcile/gate.go *authz.Verifier fake verifier in internal/reconcile gate tests
signedjobs.WipeOps / Executor (ExecutorChain) internal/signedjobs/wipe.go + runner.go *storage.SudoHostOps; WipeExecutor+DecommissionExecutor internal/signedjobs wipe/runner/decommission tests
hub.reporter / collectorIface / EnvelopeObserver internal/hub/loop.go *hub.Client, *hub.Collector; desired.Syncer + signedjobs.Runner fakeReporter/fakeCollector internal/hub/loop_test.go; recordingReporter loop_logtail_test.go
applog.Ring (always-DEBUG capture ring) + fan-out applog.New → (logger, ring) internal/log/log.go wired in cmd/felhom-agent/main.go → localapi.Options.LogRing + Loop.SetLogTailSource(ring.Lines) internal/log/log_test.go; localapi/debuglogs_test.go — v0.83.0; the byte-capped Lines is the heartbeat tail source
pbsdr.StorageReader / SecretConsumer / Manager.probeFP (func seam) internal/pbsdr/manager.go *proxmox.Client; *hub.Client; pbs.ProbeFingerprint fakeStorage/fakeConsumer/fakeRunner internal/pbsdr/manager_test.go (argv+stdin recorder)
capability.Runner internal/capability/probe.go *proxmox.ExecRunner (RunnerDirect) fakeRunner internal/capability/probe_test.go
Cross-repo: local API ↔ controller internal/localapi/server.go routes; contract seeded by internal/provision/doc.go (bootstrap.json: endpoint + leaf fingerprint + token) felhom-controller's agentapi client pin = served leaf cert (memory gotcha)
Cross-repo: agent ↔ hub internal/hub/report.go (HostReport), ControlEnvelope; POST /api/v1/host-report hub mirrors structs field-for-field new event/report fields need hub-side ingest changes
Cross-repo: shipped host artifacts configs/felhom-agent.sudoers, configs/felhom-mkfs-guarded.sh, configs/felhom-pbs-apply, shared-parent script/unit (inline in internal/localapi/intermediary.go) deployed WITH the binary sudoers globs must match stageTemp patterns + staging dirs exactly
Operator signing internal/authz (OpBlob, SSHSIG) cmd/felhom-opsign (offline CLI) blob/verify tests in internal/authz

5. Extension points (where new features plug in)

  • Local-API route: add to Server.Handler (internal/localapi/server.go) wrapped in s.withGuest; new deps go into Options as OPTIONAL fields degrading to "not configured".
  • New signed-job verb: implement signedjobs.Executor (return ErrNoExecutor for foreign ops) and append to the signedjobs.ExecutorChain in cmd/felhom-agent/main.go; add the op class + role scoping in internal/reconcile (classify.go, gate.go roleAuthorizes).
  • New privileged host op: validate args (internal/storage/validate.go style) → exec via Runner → add a Cmnd_Alias to configs/felhom-agent.sudoers → add a probe vector to internal/capability/manifest.go (so degradation is visible) → ship sudoers with the binary.
  • New reconcile action: ActionKind + classOfAction (internal/reconcile/classify.go), plan emission in internal/reconcile/plan.go; destructive ⇒ gate handles it automatically.
  • Hub-report field: extend hub.HostReport (internal/hub/report.go) + Collector — hub side must mirror + allowlist it (cross-repo).
  • Envelope-driven behavior: implement hub.EnvelopeObserver, add to the MultiObserver in cmd/felhom-agent/main.go.
  • Selftest mode: selftestFlag + runSelftest* in cmd/felhom-agent/main.go.
  • Config: internal/config/config.go (Load + applyEnv FELHOM_AGENT_* overlay; keep secrets out of Redacted() output).

6. Known duplication (observed — NOT fixed)

  • Two lsblk -J parsers with near-identical structs: parseLsblkDevice/lsblkDevice (internal/storage/hostops.go) vs parseLsblkNodes/lsblkDev (internal/storage/claim.go).
  • Two smartctl -a -j paths: SudoHostOps.SMART (internal/storage/hostops.go, parsed hub.SmartSummary) vs Privileged.SMART (internal/proxmox/privileged.go, raw map).
  • SMART device resolution (v0.95.0): smartDeviceFor (internal/storage/observe.go) resolves partition→disk AND dm/LVM→disk (dmWholeDisk in internal/storage/smartdev.go, via /sys/block/<dm>/slaves, sysBlockRoot test seam). storage.SmartReader.SMARTForBacking is the shared read the localapi /disks union path uses (Fix B) — do NOT re-implement smartctl parsing. The builtin-local SMART device comes from containingMountDevice (SMART-only; never feeds backing/durable_id).
  • Atomic tmp+rename JSON store implemented 3×: IntentStore.saveLocked (internal/storage/intent.go), FormatJobStore.save (internal/localapi/formatjob.go), GuestBindStore.saveLocked (internal/localapi/guestbindstore.go) — comments say "mirrors", no shared helper.
  • run(ctx, name, args...) error stderr-wrapping helper duplicated 4×: SudoHostOps.run, Privileged.run, BackHalf.run (internal/provision/backhalf.go), GuestBinder.run (internal/localapi/guestbind.go).
  • Several independent /proc mount-table readers: SudoHostOps.mountedSet (internal/storage/hostops.go), ProcHostReader.Mounts (internal/storage/hostread.go), isHostMountpoint + countHostMounts (internal/localapi/intermediary.go).
  • Deliberate mirror: antiRetargetResolveExpect (internal/localapi/wipe_reresolve.go) duplicates WipeExecutor.Execute steps 13 (internal/signedjobs/wipe.go) across packages.
  • stableParentDir literal duplicated in internal/provision/backhalf.go to avoid a provision→localapi import edge (commented as intentional); trim (internal/storage/hostops.go) vs trimBody (internal/proxmox/errors.go) output-truncation twins.