felhom-agent/CLAUDE.md

# CLAUDE.md — `felhom-agent`

> Place at the repo root (`felhom-agent/CLAUDE.md`). Loads when Claude Code touches this repo. Keep under ~200 lines. The cross-repo orientation lives in the workspace-root `e:\git\CLAUDE.md`; this file is `felhom-agent`-specific.

## What this repo is

`felhom-agent` is the operator-tier **host agent** that runs on each Proxmox host and owns **all** Proxmox interaction: provision/restore guests, host storage, backup/restore orchestration, the hub control loop, and a narrow per-guest local API. It is the **most privilege-sensitive** component.

- It is the renamed former `proxmox-controller` repo.
- **Distinct from `felhom-controller`** — that is the *in-guest* controller (Docker-only, no Proxmox creds). Do not confuse them.
- Control plane, not data plane: if the agent dies, apps keep serving; only management degrades.

## Build / run

- Module `gitea.dooplex.hu/admin/felhom-agent`; binary `felhom-agent` (`cmd/felhom-agent/`).
- **Pure Go stdlib + `golang.org/x/crypto` only** — no web frameworks.
- `go.mod` directive **go 1.25.0**; dep `golang.org/x/crypto v0.52.0` (declares go 1.25, will NOT build on Go 1.24). The **build server (192.168.0.180) runs go1.26.0** (upstream Go on PATH, backward-compatible). Build/run the agent there for live tests (same LAN as the demo host).
- Version: `version` var in `cmd/felhom-agent/main.go`, overridable via `-ldflags "-X main.version=<v>"`; `--version` flag. **Current: v0.6.0** (slice 6 complete: backup + self-restore-test + the PBS offsite tier — verify + PBS-API client + PBSSnapshot reporting). Bump on meaningful changes + add a CHANGELOG entry.

## Layout

```
cmd/felhom-agent/    main + flag handling + --selftest modes + the daemon entry
internal/config/     JSON config + FELHOM_AGENT_* env overlay; secrets redacted (Redacted())
internal/log/        slog setup
internal/proxmox/    API-first Client + fenced root-CLI Privileged + UPID WaitTask
internal/authz/      operator signed-op verifier (SSHSIG); durable FileNonceStore
internal/hub/        daemon: HostReport collector + Bearer client + resilient Loop
internal/reconcile/  reconcile engine + reversibility gate + op journal + crash recovery + restore-test
internal/storage/    storage-target observer + durable_id + fast-poll watchdog (slice 5)
internal/backup/     vzdump backup runner + restore-test scheduler + report store (slice 6)
internal/pbs/        PBS-API client (fingerprint-pinned) + verify maintenance loop (slice 6 Phase B)
```

## Proxmox model (the load-bearing rules)

- **API-first** via a scoped `FelhomAgent` token (16 privileges). Raw root-CLI is **fenced to exactly 3 exceptions**: keyctl `pct create` (golden image), USB mount/fstab, SMART/sensors. `Client` never shells out; `Privileged` never makes HTTP calls (asserted by tests). Keep that fence.
- **Every mutating op is async** → returns a UPID → `WaitTask` asserts `exitstatus == "OK"`. A 200 on the POST is **not** success; authorization can fail at task execution, not the POST.
- **TLS:** SHA-256 leaf-cert pinning (the host serves a self-signed cert). No insecure default.
- **Privsep token gotcha:** a `--privsep 1` token's rights = intersection of the backing user's perms AND the token's ACLs — so the role must be granted on **both** user and token, or every call 403s. (Token provisioning is out-of-band / human-run; the agent only consumes the token.)

## Design + platform facts (read before designing)

- Design doc: `felhom.eu/documentation/architecture/03-host-agent.md` (locked).
- Platform facts: `felhom.eu/documentation/proxmox-platform.md` + `tests/phase{0,1-2,3,4}-findings.md`.

## Current state

Built in slices, all on `main`:
- **v0.1.0** slice 1 — scaffold + `internal/proxmox` + `internal/config`/`log` + `--selftest`.
- **v0.2.0** slice 2 — `internal/authz` signed-op verifier.
- **v0.3.0** slice 3 — `internal/hub`: the first **daemon loop** (no-`--selftest` mode) posting a read-only `HostReport` to the hub (= the heartbeat). Report's storage/backup/restore/pbs/audit fields are **defined-but-empty** (slices 5/6); the envelope's desired-state/signed-ops fields are **parsed-but-ignored** (slice 4).
- **v0.3.1** — slice-3 validation follow-ups.
- **v0.3.2** — slice-4 pre-check: reversible `SetConfig` step added to `--selftest=task`; passed live on guest 9999. Findings: LXC `description` write is **synchronous** (empty UPID — dual-mode modeling confirmed); PVE appends a trailing `\n` to `description` on read (reconcile must normalize). First live `VM.Config.*` exercise.
- **v0.4.0-rc1** — slice-4 **Phase A** (structural): `internal/reconcile` — engine, per-guest serializer (§10), desired-state model + `DesiredProvider` seam, normalization layer (`NormDescription` promoted out of main.go), plan/diff engine (benign Start/Stop/SetConfig set), durable op journal + idempotency store. Wired into `runDaemon` sharing the queue. Runs **live but unfed** (EmptyProvider → zero mutations until slice 10).
- **v0.4.0** — slice-4 **Phase B** (security core): the benign/destructive **classifier** (provenance + data-bearing, not by verb; scratch/same-txn provenance is agent-internal, never hub-sourced), the **reversibility gate** (destructive → `pending_signature` unless a verified, role-scoped, action-bound operator signature), the **signed-op consuming layer** over `internal/authz` (role-scoping per doc 04 §4, op-to-action binding, idempotency-by-nonce, audit), and the **crash-recovery consumer** (`Recover` over `InFlight()`, resume-or-rollback). The gate fronts the queue's executor (every mutation passes it). **Inert this slice** — no destructive deltas served until slice 10; the destructive path is classified, gated, and adversarially tested but not wired to live execution. `authz` surface untouched.
- **v0.5.0-rc1** — slice-5 **Phase A** (read-only, live): `internal/storage` — the `StorageTarget` wire contract (filled the slice-3 stub), `durable_id` derivation per type, the `Observer`, and the **storage watchdog** (third daemon goroutine; fast-poll → debounced out-of-band report on a known target's attach/disconnect). Hub ingest accepts/persists `storage_targets`; cross-repo golden byte-identical.
- **v0.5.0** — slice-5 **Phase B** (the host-root surface): the `HostOps` seam + `SudoHostOps` (systemd `.mount` units by fs-UUID, detach, SMART, lvs) behind a **strict argument validator** (the adversarial matrix is the headline security test — hostile UUID/path/device refused with zero exec); SMART (SATA+NVMe) + thin-pool metadata enrichment; the watchdog's benign **re-mount response** (off the poll path); the **disk-grow executor** (`pct resize`, grow-only, benign) and **destructive storage ops** through the slice-4 gate (target-scoped; built + tested, inert live); `--selftest=storage [-watch]`; `configs/felhom-agent.sudoers`.
- **v0.6.0-rc1** — slice-6 **Phase A** (backup + self-restore-test, local target): proxmox `DestroyLXC`/`Vzdump`-notes/`LatestBackupVolID`; `Engine.RunRestoreTest` (journaled scratch lifecycle: restore-to-new → net link-down → boot → verify running → defer teardown, all benign); `Recover` extended to reap a leaked scratch guest (Scratch journal flag, special-cased before the UPID path); `internal/backup` (runner + bulk-gap + cadence scheduler + report store); hub `Backup`/`RestoreTest` filled (cross-repo golden + hub logs a failed restore-test); `--selftest=backup`/`--selftest=restore-test`. Live-validated on demo-felhom.
- **v0.6.0** — slice-6 **Phase B** (PBS offsite tier): `internal/pbs` — a fingerprint-pinned, token-authed PBS-API client (Verify/Snapshots/TaskStatus, node-from-UPID); the verify maintenance loop (own cadence, NOT gated/journaled — like the watchdog); `PBSSnapshot` reporting filled (cross-repo golden + hub failed-verify WARN); truthful vzdump mode from the task log; `--selftest=pbs-verify`. Backup/restore-to-PBS reuse Phase A unchanged. Live-validated against the spike's DooPlex PBS.
- **Next: slice 7 (provisioning + identity-reset + golden base, §9)** — the unified bring-up primitive; restore-overwrite + decommission executors the gate already guards; escrow + host-loss DR.

## Demo host (for live tests)

Node **`demo-felhom`**, API `https://192.168.0.162:8006`, PVE 9.2.2; leaf-cert SHA-256 fingerprint starts `BA:7C:99:7D:45:D0…` (verify it still matches before a live run — the agent pins it). `pveum`/`pct` ops need `root@pam` on the PVE (SSH alias `felhom-pve`) - available to Claude Code

Selftest modes (run from the build server, pointed at the demo API):
- `--selftest` / `--selftest=read` — read-only health checks.
- `--selftest=task -vmid N` — reversible snapshot→rollback→delete on guest N (gated; never under bare `--selftest`).
- `--selftest=hub` — one collect + report round-trip to the hub.
- No flag → the **daemon** (poll loop); requires `hub` config.

## Conventions

- Push to `main` directly; no feature branches.

> **In every repository where you make a change, update both files in that repo:**
> - **`CHANGELOG.md`** — a cumulative log of **all** changes; newest entry on top.
> - **`REPORT.md`** — **overwrite** with a summary of the **most recent** implementation (or significant validation/operational run) only; not cumulative.
>
> **Never write secrets** — tokens, passwords, private keys, API keys — into `CHANGELOG.md`, `REPORT.md`, or any committed file. Reference them as "stored out-of-band" instead.

- Code quality: verify generated code for bugs/edge cases; add debug logging; **ask rather than guess** when you'd otherwise invent input/output.

## Workflow & artifacts

- Implement **`TASK.md` / `TASK-*.md`** specs (when placed as `TASK.md` or told to implement one), then push + CHANGELOG + REPORT.md.
- **`RUNBOOK-*.md`** — an operational procedure. CC executes the steps it has access and capability for, including live validation on the demo nodes and the demo Proxmox host (CC has root@felhom-pve SSH + the felhom-agent token). A step is human-only only when it genuinely needs physical presence, a real-world decision, or credentials CC truly lacks — mark those steps HUMAN. Do not decline a whole procedure because it touches a live host or a privileged token. (Judgment still applies: confirm before irreversible ops on real customer data — but demo scratch guests are fair game.)