felhom-agent/CLAUDE.md

CLAUDE.md — felhom-agent

Place at the repo root (felhom-agent/CLAUDE.md). Loads when Claude Code touches this repo. Keep under ~200 lines. The cross-repo orientation lives in the workspace-root e:\git\CLAUDE.md; this file is felhom-agent-specific.

What this repo is

felhom-agent is the operator-tier host agent that runs on each Proxmox host and owns all Proxmox interaction: provision/restore guests, host storage, backup/restore orchestration, the hub control loop, and a narrow per-guest local API. It is the most privilege-sensitive component.

• It is the renamed former proxmox-controller repo.
• Distinct from felhom-controller — that is the in-guest controller (Docker-only, no Proxmox creds). Do not confuse them.
• Control plane, not data plane: if the agent dies, apps keep serving; only management degrades.

Build / run

• Module gitea.dooplex.hu/admin/felhom-agent; binary felhom-agent (cmd/felhom-agent/).
• Pure Go stdlib + golang.org/x/crypto only — no web frameworks.
• go.mod directive go 1.25.0; dep golang.org/x/crypto v0.52.0 (declares go 1.25, will NOT build on Go 1.24). The build server (192.168.0.180) runs go1.26.0 (upstream Go on PATH, backward-compatible). Build/run the agent there for live tests (same LAN as the demo host).
• Version: version var in cmd/felhom-agent/main.go, overridable via -ldflags "-X main.version=<v>"; --version flag. Bump on meaningful changes + add a CHANGELOG entry. See CHANGELOG.md (top only) for the authoritative current state.
• Current: v0.31.0 (2026-06-14; deployed on demo host felhom-pve). Recent state (summarized from the CHANGELOG top — verify there, not here):
  • v0.31.0 — live-drive disk fixes: F9 guest data-drive bind survives a re-provision (GuestBindStore + ReassertGuestBinds on startup, durable-id-matched; DiskInfo.GuestAttached reporting); F20-BUG2 one wipe durable-id scheme (DiskInfo.WipeDurableID via a shared deviceDurableID seam used by list + gate); F20-BUG3 mkfs runs detached off baseCtx with a persisted formatJob + GET /disks/format/status + RecoverFormatJob startup recovery (survives a request deadline + an agent restart). Live-validated on 9201 + the 916 GB felhom-usb.
  • v0.30.0 — AGENT-001 security fix: the inline customer-confirmed wipe (localapi handleDiskFormat) re-resolves the confirmed durable id → current device, re-derives+matches, re-inspects, and formats the re-resolved device (never the mutable req.Device), closing a classify→mkfs TOCTOU. At parity with signedjobs.WipeExecutor (internal/localapi/wipe_reresolve.go).
  • v0.29.x — OS / Docker-data storage split (golden bakes split rootfs + Docker-data volume; provision) + lanresolver split-horizon DNS fix (RESTART, not reload, dnsmasq on a guest IP change).
  • v0.28.0 — backup re-target to offsite felhom-pbs (DR) + operator-signed decommission.
  • v0.23–0.27 — device-ROLE classification + tiered storage-wipe gate (system/backup operator-only, user-data customer-confirmable); eject role-gate; user-data drive enroll/bind into the guest (slice 10 P2/P3); self-heal watchdog + 4-state intent model.
  • Foundations (slice 8–10): per-guest local-API server (internal/localapi, self-scoped endpoints, hashed per-guest token store, pinned self-signed leaf); the /disks data-bearing classifier + signed-job destructive gate (POST /disks/format inspects the device itself; data-bearing → gate → pending_signature refused, caller's claim ignored; blank → benign mkfs); the provisioning back-half (internal/provision: mint token → bootstrap.json 0600 → chown 100000:100000 → pct set bind, no registry cred in guest); host metrics; hub desired-state; operator-signed completion + offline signing CLI; PBS escrow + identity-restore (slice 10D). Runtime dep: proxmox-backup-client.

Layout

cmd/felhom-agent/    main + flag handling + --selftest modes + the daemon entry
internal/config/     JSON config + FELHOM_AGENT_* env overlay; secrets redacted (Redacted())
internal/log/        slog setup
internal/proxmox/    API-first Client + fenced root-CLI Privileged + UPID WaitTask
internal/authz/      operator signed-op verifier (SSHSIG); durable FileNonceStore
internal/hub/        daemon: HostReport collector + Bearer client + resilient Loop
internal/reconcile/  reconcile engine + reversibility gate + op journal + crash recovery + restore-test
internal/storage/    storage-target observer + durable_id + fast-poll watchdog (slice 5)
internal/backup/     vzdump backup runner + restore-test scheduler + report store (slice 6)
internal/pbs/        PBS-API client (fingerprint-pinned) + verify maintenance loop (slice 6 Phase B)

Proxmox model (the load-bearing rules)

• API-first via a scoped FelhomAgent token (16 privileges). Raw root-CLI is fenced to exactly 3 exceptions: keyctl pct create (golden image), USB mount/fstab, SMART/sensors. Client never shells out; Privileged never makes HTTP calls (asserted by tests). Keep that fence.
• Every mutating op is async → returns a UPID → WaitTask asserts exitstatus == "OK". A 200 on the POST is not success; authorization can fail at task execution, not the POST.
• TLS: SHA-256 leaf-cert pinning (the host serves a self-signed cert). No insecure default.
• Privsep token gotcha: a --privsep 1 token's rights = intersection of the backing user's perms AND the token's ACLs — so the role must be granted on both user and token, or every call 403s. (Token provisioning is out-of-band / human-run; the agent only consumes the token.)

Design + platform facts (read before designing)

• Design doc: felhom.eu/documentation/architecture/03-host-agent.md (locked).
• Platform facts: felhom.eu/documentation/proxmox-platform.md + tests/phase{0,1-2,3,4}-findings.md.

Current state

Built in slices, all on main:

• v0.1.0 slice 1 — scaffold + internal/proxmox + internal/config/log + --selftest.
• v0.2.0 slice 2 — internal/authz signed-op verifier.
• v0.3.0 slice 3 — internal/hub: the first daemon loop (no---selftest mode) posting a read-only HostReport to the hub (= the heartbeat). Report's storage/backup/restore/pbs/audit fields are defined-but-empty (slices 5/6); the envelope's desired-state/signed-ops fields are parsed-but-ignored (slice 4).
• v0.3.1 — slice-3 validation follow-ups.
• v0.3.2 — slice-4 pre-check: reversible SetConfig step added to --selftest=task; passed live on guest 9999. Findings: LXC description write is synchronous (empty UPID — dual-mode modeling confirmed); PVE appends a trailing \n to description on read (reconcile must normalize). First live VM.Config.* exercise.
• v0.4.0-rc1 — slice-4 Phase A (structural): internal/reconcile — engine, per-guest serializer (§10), desired-state model + DesiredProvider seam, normalization layer (NormDescription promoted out of main.go), plan/diff engine (benign Start/Stop/SetConfig set), durable op journal + idempotency store. Wired into runDaemon sharing the queue. Runs live but unfed (EmptyProvider → zero mutations until slice 10).
• v0.4.0 — slice-4 Phase B (security core): the benign/destructive classifier (provenance + data-bearing, not by verb; scratch/same-txn provenance is agent-internal, never hub-sourced), the reversibility gate (destructive → pending_signature unless a verified, role-scoped, action-bound operator signature), the signed-op consuming layer over internal/authz (role-scoping per doc 04 §4, op-to-action binding, idempotency-by-nonce, audit), and the crash-recovery consumer (Recover over InFlight(), resume-or-rollback). The gate fronts the queue's executor (every mutation passes it). Inert this slice — no destructive deltas served until slice 10; the destructive path is classified, gated, and adversarially tested but not wired to live execution. authz surface untouched.
• v0.5.0-rc1 — slice-5 Phase A (read-only, live): internal/storage — the StorageTarget wire contract (filled the slice-3 stub), durable_id derivation per type, the Observer, and the storage watchdog (third daemon goroutine; fast-poll → debounced out-of-band report on a known target's attach/disconnect). Hub ingest accepts/persists storage_targets; cross-repo golden byte-identical.
• v0.5.0 — slice-5 Phase B (the host-root surface): the HostOps seam + SudoHostOps (systemd .mount units by fs-UUID, detach, SMART, lvs) behind a strict argument validator (the adversarial matrix is the headline security test — hostile UUID/path/device refused with zero exec); SMART (SATA+NVMe) + thin-pool metadata enrichment; the watchdog's benign re-mount response (off the poll path); the disk-grow executor (pct resize, grow-only, benign) and destructive storage ops through the slice-4 gate (target-scoped; built + tested, inert live); --selftest=storage [-watch]; configs/felhom-agent.sudoers.
• v0.6.0-rc1 — slice-6 Phase A (backup + self-restore-test, local target): proxmox DestroyLXC/Vzdump-notes/LatestBackupVolID; Engine.RunRestoreTest (journaled scratch lifecycle: restore-to-new → net link-down → boot → verify running → defer teardown, all benign); Recover extended to reap a leaked scratch guest (Scratch journal flag, special-cased before the UPID path); internal/backup (runner + bulk-gap + cadence scheduler + report store); hub Backup/RestoreTest filled (cross-repo golden + hub logs a failed restore-test); --selftest=backup/--selftest=restore-test. Live-validated on demo-felhom.
• v0.6.0 — slice-6 Phase B (PBS offsite tier): internal/pbs — a fingerprint-pinned, token-authed PBS-API client (Verify/Snapshots/TaskStatus, node-from-UPID); the verify maintenance loop (own cadence, NOT gated/journaled — like the watchdog); PBSSnapshot reporting filled (cross-repo golden + hub failed-verify WARN); truthful vzdump mode from the task log; --selftest=pbs-verify. Backup/restore-to-PBS reuse Phase A unchanged. Live-validated against the spike's DooPlex PBS.
• Next: slice 7 (provisioning + identity-reset + golden base, §9) — the unified bring-up primitive; restore-overwrite + decommission executors the gate already guards; escrow + host-loss DR.

Demo host (for live tests)

Node demo-felhom, API https://192.168.0.162:8006, PVE 9.2.2; leaf-cert SHA-256 fingerprint starts BA:7C:99:7D:45:D0… (verify it still matches before a live run — the agent pins it). pveum/pct ops need root@pam on the PVE (SSH alias felhom-pve) - available to Claude Code

Selftest modes (run from the build server, pointed at the demo API):

• --selftest / --selftest=read — read-only health checks.
• --selftest=task -vmid N — reversible snapshot→rollback→delete on guest N (gated; never under bare --selftest).
• --selftest=hub — one collect + report round-trip to the hub.
• No flag → the daemon (poll loop); requires hub config.

Conventions

Trunk-based — no branches

All shippable work commits directly to main; main is always equal to what is deployed. Do NOT create feature/fix branches.

• Report-only artifacts (audits, findings, fixspecs, reconciliations) → felhom.eu/documentation/ (audits/, backlog/), committed to main. Never a branch, never left loose at the repo root.
• Risky/supervised fixes (agent / golden / provisioning / destructive) are spec'd, then implemented during the supervised session itself, directly on main — not prepared ahead on a branch. (This is the common case for this repo.)
• Unattended escape hatch: if a fix can't be cleanly verified/shipped, revert it and report it for a supervised redo (or paste the diff into the spec doc in documentation/) — do not park it on a branch.
• This supersedes any older "prepared on branch fix/…, pending review" pattern.

In every repository where you make a change, update both files in that repo:

• CHANGELOG.md — a cumulative log of all changes; newest entry on top.
• REPORT.md — overwrite with a summary of the most recent implementation (or significant validation/operational run) only; not cumulative.

Never write secrets — tokens, passwords, private keys, API keys — into CHANGELOG.md, REPORT.md, or any committed file. Reference them as "stored out-of-band" instead.

• Code quality: verify generated code for bugs/edge cases; add debug logging; ask rather than guess when you'd otherwise invent input/output.

Live validation

Live validation of a user-facing feature must exercise the SERVER-SIDE PIPELINE a real user triggers, end-to-end (e.g. connect → enroll → deploy). The forbidden shortcut is BYPASSING that pipeline — e.g. raw agent guest-attach + hand-set state instead of the enrollment flow (the F9 episode) — which gives false confidence and leaves the system inconsistent. INVOKING THE EXACT ENDPOINT THE UI INVOKES — so the full server pipeline (gates, env injection, pre-create belts, compose generation) runs — is an ACCEPTABLE proxy when a browser-automation tool isn't available: it differs fundamentally from the F9 bypass because no server logic is skipped, only the browser rendering. The residual that proxy does NOT cover is purely client-side (progress panels, card/health rendering, client-side guards like checkBeforeDeploy); for strict end-to-end UI coverage use a real browser tool or a manual click-through — and SAY which was used. Low-level mechanism tests where the direct call IS the mechanism remain exempt.

Workflow & artifacts

• Implement TASK.md / TASK-*.md specs (when placed as TASK.md or told to implement one), then push + CHANGELOG + REPORT.md.
• RUNBOOK-*.md — an operational procedure. CC executes the steps it has access and capability for, including live validation on the demo nodes and the demo Proxmox host (CC has root@felhom-pve SSH + the felhom-agent token). A step is human-only only when it genuinely needs physical presence, a real-world decision, or credentials CC truly lacks — mark those steps HUMAN. Do not decline a whole procedure because it touches a live host or a privileged token. (Judgment still applies: confirm before irreversible ops on real customer data — but demo scratch guests are fair game.)