felhom-agent/CLAUDE.md

# CLAUDE.md — `felhom-agent`

> Loads when Claude Code touches this repo. Stable orientation only — **current state lives in
> `CONTEXT.md` and the top of `CHANGELOG.md`**, never here. Cross-repo orientation: workspace-root
> `/mnt/5_hdd/felhom.eu/git/CLAUDE.md`.

## What this repo is

`felhom-agent` is the operator-tier **host agent** that runs on each Proxmox host and owns **all**
Proxmox interaction: provision/restore guests, host storage, backup/restore orchestration, the hub
control loop, and a narrow per-guest local API. It is the **most privilege-sensitive** component.

- Renamed former `proxmox-controller` repo.
- **Distinct from `felhom-controller`** — that is the *in-guest* controller (Docker-only, no Proxmox
  creds). Do not confuse them.
- Control plane, not data plane: if the agent dies, apps keep serving; only management degrades.

## Read before writing code

- **`REUSE.md`** — canonical helpers, format-safety guards, traps, seams. Check it first; update it
  in the same commit that changes a shared helper or pattern.
- `CONTEXT.md` (current state + open threads) and the top `CHANGELOG.md` entry (authoritative history).
- Design doc: `felhom.eu/documentation/architecture/03-host-agent.md` (locked). Platform facts:
  `felhom.eu/documentation/proxmox-platform.md` + `tests/phase{0,1-2,3,4}-findings.md`.

## Layout (verified against the tree)

```
cmd/felhom-agent/    main + flags + --selftest modes + the daemon entry
cmd/felhom-opsign/   offline operator signing CLI (SSHSIG)
internal/authz/      operator signed-op verifier (SSHSIG) + durable FileNonceStore
internal/backup/     vzdump backup runner + restore-test scheduler + report store
internal/capability/ live sudo-policy capability probe (degradation visibility)
internal/config/     JSON config + FELHOM_AGENT_* env overlay; secrets redacted (Redacted())
internal/desired/    hub desired-state syncer (envelope observer)
internal/escrow/     PBS-key escrow (zero-knowledge recovery code)
internal/guesthook/  pre-start self-heal hookscript install
internal/hub/        daemon: HostReport collector + Bearer client + resilient Loop
internal/lanresolver/ split-horizon DNS on guest IP change (dnsmasq RESTART, not reload)
internal/localapi/   per-guest local API: token store, disks/format, guest binds, controller swap,
                     stale-lock recovery, pinned self-signed leaf
internal/log/        slog setup
internal/pbs/        PBS-API client (fingerprint-pinned) + verify maintenance loop
internal/provision/  guest bootstrap back-half (token mint → bootstrap.json → pct bind)
internal/proxmox/    API-first Client + fenced root-CLI Privileged + UPID WaitTask
internal/reconcile/  reconcile engine + reversibility gate + op journal + crash recovery
internal/signedjobs/ operator-signed destructive executors (wipe, decommission)
internal/storage/    storage observer + durable ids + role/claim classifiers + SudoHostOps + watchdog
```

## Build / run

- Module `gitea.dooplex.hu/admin/felhom-agent`; binary `felhom-agent` (`cmd/felhom-agent/`).
- **Pure Go stdlib + `golang.org/x/crypto` only** — no web frameworks. `go.mod` directive go 1.25.0;
  DooPlex (192.168.0.180, where CC runs) has the Go toolchain and is on the same LAN as the demo
  host — build and run live tests locally.
- Version via `-ldflags "-X main.version=<v>"`; `--version` flag. Bump on meaningful changes + CHANGELOG entry.
- **Full build/deploy/publish runbook: use the `felhom-build-deploy` skill.** Summary:

> **Clean-tree gate before any build:** `git status --porcelain` must be empty and
> `git rev-parse HEAD` must equal `git rev-parse origin/main` in the repo being built. An unpushed
> change does not exist — never build a dirty or unpushed tree. The `git pull` in the build step
> stays (it is a no-op when you work in this tree, and load-bearing if anything was pushed from
> elsewhere).

| Step | Where | One-liner |
|---|---|---|
| Build | DooPlex (local) | `cd /mnt/5_hdd/felhom.eu/git/felhom-agent && git pull && go build -ldflags '-X main.version=<v>' -o /tmp/felhom-agent-<v> ./cmd/felhom-agent` |
| Copy | local → felhom-pve | `scp /tmp/felhom-agent-<v> felhom-pve:/tmp/` (one hop) |
| Deploy | felhom-pve | backup `.bak-<old>` → `install -m0755` → `systemctl restart felhom-agent` (non-root `felhom-agent` user, config `/etc/felhom-agent/agent.json`) |
| Ship configs | felhom-pve | sudoers (`/etc/sudoers.d/felhom-agent`) + guarded-mkfs wrapper WITH the binary when `configs/` changed |
| Publish | DooPlex (local) | `scripts/publish-agent.sh <ver> <bin>` (REGISTRY_* creds); hub Day-0 manifest vouch = operator follow-up |
| Verify | felhom-pve | `felhom-agent --version` + journal (clean ReassertGuestBinds, no capability degradation) |

## Proxmox model (the load-bearing rules)

- **API-first** via a scoped `FelhomAgent` token. Raw root-CLI is **fenced to exactly 3 exceptions**:
  keyctl `pct create` (golden image), USB mount/fstab, SMART/sensors. `Client` never shells out;
  `Privileged` never makes HTTP calls (asserted by `routing_test.go`). Keep that fence.
- **Every mutating op is async** → returns a UPID → `WaitTask` asserts `exitstatus == "OK"`. A 200 on
  the POST is **not** success; authorization can fail at task execution.
- **TLS:** SHA-256 leaf-cert pinning (self-signed host cert). No insecure default.
- **Privsep token gotcha:** a `--privsep 1` token's rights = intersection of the backing user's perms
  AND the token's ACLs — the role must be granted on **both**, or every call 403s.
- Destructive ops go through the reconcile gate / signed-jobs path — never call `Client.DestroyLXC`/
  `Vzdump`/`SetConfig` ad-hoc (REUSE.md §3).

## Demo host (for live tests)

Node **`demo-felhom`**, API `https://192.168.0.162:8006`. SSH alias `felhom-pve` (root@pam) —
available to CC as plain `ssh felhom-pve`. The agent pins the served leaf cert — verify the
fingerprint still matches before a live run. Selftest modes (run locally on DooPlex, pointed at the
demo API): `--selftest[=read|task|hub|storage|backup|restore-test|pbs-verify]`; no flag = the daemon.

> **TEMPORARY — felhom-pve is at a remote site (until ~2026-08-02).** The home-LAN literal
> `192.168.0.162` is NOT reachable from DooPlex for the duration. Access via Tailscale:
> felhom-pve = 100.70.170.35; the `Host felhom-pve` entry in `~/.ssh/config` on DooPlex already
> points there (the direct-LAN path stays available as `Host felhom-pve-lan`). Delete this block on
> return. All documented `ssh felhom-pve` / `pct exec` workflows are unchanged. Path is **direct**
> (not DERP), ~37 ms rtt per hop. At the remote site the host is on **DHCP** and currently holds
> `192.168.0.147` — so the PVE API is at `https://192.168.0.147:8006` there, and **the agent does
> not run at all**: `localapi` binds the literal `192.168.0.162` → `bind: cannot assign requested
> address` → the service is `failed` and has never started at the remote site. Fixing it means
> editing `listen_addr` in `/etc/felhom-agent/agent.json` **and** the guest's bootstrap endpoint
> (plus the leaf-cert SAN the controller pins) — Viktor GO required. Details + findings:
> `felhom.eu/documentation/audits/AUDIT-vacation-remote-ops-2026-07-20.md`

> **Legacy: Windows workstation.** Until 2026-07-19 CC ran on Windows 11; `pct` commands over SSH
> needed `export MSYS_NO_PATHCONV=1`, and every remote command used
> `SSH=/c/Windows/System32/OpenSSH/ssh.exe`. Agent deploy was a two-hop copy via the Windows box
> (`cygpath -w` for the local scp path; CRLF hazard on config files).

## Conventions

### Trunk-based — no branches

All shippable work commits **directly to `main`**; `main` equals what is deployed.
- Report-only artifacts (audits, findings, fixspecs) → `felhom.eu/documentation/` (`audits/`, `backlog/`).
- Risky/supervised fixes are spec'd, then implemented **during the supervised session, on `main`**.
- Unattended escape hatch: if a fix can't be cleanly verified/shipped, revert + report — never park on a branch.

> **In every repository where you make a change, update both files in that repo:**
> - **`CHANGELOG.md`** — cumulative log, newest on top.
> - **`REPORT.md`** — **overwrite** with the most recent implementation/validation summary only.
>
> **Never write secrets** into any committed file — reference them as "stored out-of-band".

- Code quality: verify generated code for bugs/edge cases; add debug logging; **ask rather than
  guess** when you'd otherwise invent input/output.
- Update `REUSE.md` if you added/changed/deprecated a shared helper or pattern (same commit).
- Testing doctrine (non-hollow tests, red-proofs, seams): use the `felhom-testing` skill.
- **Logging**: the slog logger fans out to journald (configured level) + the always-DEBUG `applog.Ring`
  (remote pulls) — English, keys-never-values, durations on outcomes; full rules in
  `felhom.eu/documentation/runbooks/logging-conventions.md`.

### Live validation

Exercise the SERVER-SIDE PIPELINE a real user triggers, end-to-end. The forbidden shortcut is
BYPASSING it (the F9 episode: raw guest-attach + hand-set state). Invoking the exact endpoint the UI
invokes is an acceptable proxy when a browser isn't available — say which method was used. Low-level
mechanism tests where the direct call IS the mechanism are exempt.

## Workflow & artifacts

- Implement **`TASK.md` / `TASK-*.md`** specs (when placed as `TASK.md` or told to), then push +
  CHANGELOG + REPORT.md.
- **`RUNBOOK-*.md`** — an operational procedure. CC executes the steps it has access and capability
  for, including live validation on the demo Proxmox host (CC has root@felhom-pve SSH + the
  felhom-agent token). Mark a step HUMAN only when it genuinely needs physical presence, a real-world
  decision, or credentials CC truly lacks. Judgment still applies: confirm before irreversible ops on
  real customer data — demo scratch guests are fair game.