Files
felhom-agent/CHANGELOG.md
T
admin 77b4f21450 v0.5.1: live-validation prep — fix unmounted-dir durable_id mis-id + watchdog UUID memory
Surfaced preparing the live USB validation on demo-felhom:
- observe.go: an unmounted removable dir-storage no longer falls through to the ROOT fs
  for its backing device/UUID — durable_id was becoming uuid:<root-uuid> (a DR mis-id that
  would re-attach the wrong disk). Now derived only from the target's own mountpoint;
  unmounted → no device + stable store:<name> durable_id. Removed containingMountDevice.
- watchdog.go: remember the fs-UUID observed while attached and backfill it onto the
  re-mount target, so re-mount works even if the known-set cache refreshed mid-drop
  (doc 03 §7 "sourced from the existing definition").

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 11:01:40 +02:00

497 lines
34 KiB
Markdown

# Changelog
All notable changes to **felhom-agent** are recorded here. Update on every code
change that gets pushed.
## v0.5.1 — slice 5 live-validation prep: durable_id mis-id fix + re-mount UUID memory (2026-06-09)
Two correctness fixes surfaced while preparing the live USB validation on `demo-felhom`
(a real 1TB USB HDD, sdb1, ext4). Both are DR-load-bearing — exactly the "false-id →
re-attach the wrong disk" failure mode the slice warned about.
### Fixed
- **Unmounted dir-storage no longer inherits the ROOT filesystem's UUID** (`observe.go`).
Previously, when a removable dir-storage was unmounted, the observer fell through to the
*containing* mount (root) for the backing device, so its `durable_id` became
`uuid:<root-uuid>` — a catastrophic DR mis-id (the hub would re-attach the wrong disk).
Now the backing device/UUID/`durable_id` are derived ONLY from the target's OWN
mountpoint; an unmounted target reports no device and a stable `store:<name>` durable_id,
never another filesystem's UUID. (Removed the `containingMountDevice` root-fallthrough.)
- **Watchdog remembers the fs-UUID observed while attached** (`watchdog.go`) so a re-mount
works even after the known-set cache refreshes mid-drop (an unmounted target can't resolve
its own UUID). The re-mount key is backfilled from this memory — aligning with doc 03 §7's
"sourced from the existing definition, no hub manifest needed": the agent learns the UUID
while the target is attached, then re-mounts by it on return.
### Tests
- Observer: an unmounted dir-storage asserts NO `uuid:` durable_id and no backing device.
- Watchdog: a drop where the cache lost the UUID still re-mounts using the remembered UUID.
## v0.5.0 — slice 5 Phase B: the host-root surface (mounts + SMART + grow + destructive gate) (2026-06-09)
The write surface — the agent's first step outside its Proxmox API token into OS-root.
Isolated behind a narrow, argument-validated, adversarially-tested seam, exactly like the
slice-4 gate. Completes slice 5 (Phase A = read-only observe/report/watchdog at v0.5.0-rc1).
### Added
- **`HostOps` seam + `SudoHostOps`** (`internal/storage/hostops.go`) — the one privileged
host surface: persistent mounts via **systemd `.mount` units keyed by fs-UUID** (enabled to
survive reboot), detach (stop+disable), SMART, and thin-pool metadata. Shells out via the
fenced Runner (`sudo -n`, **fixed arg vectors, no shell**); a fake backs the tests (no real
root in the suite). `NoopHostOps` is the safe fallback when the surface is unavailable.
- **The argument validator** (`internal/storage/validate.go`) — the security boundary:
`ValidateUUID` (strict hex), `ValidateMountPath` (absolute, no traversal, no metacharacters),
`ValidateSMARTDevice` (raw-disk whitelist), `ValidateLVMName`, and an in-process
`systemdEscapePath` (no `systemd-escape` shell-out). **Every argument is validated BEFORE a
command is constructed.** Headline test (`validate_test.go`): an adversarial matrix of
shell metacharacters / `../` traversal / malformed inputs is rejected with **zero exec**.
- **SMART** (`internal/storage/smart.go`) — parses `smartctl -a -j` into `StorageTarget.smart`:
**SATA** (reallocated/pending/offline-uncorrectable, temp, power-on-hours) **and NVMe**
(critical_warning, media_errors, percentage_used, temp), degrading to `UNKNOWN` for devices
with no SMART (USB-SATA bridges). **`lvs`** fills the lvmthin thin-pool **metadata** fill
(the value Phase A left null). Wired into the Observer's enrichment (Observe only, not the
watchdog's fast Known path).
- **Watchdog re-mount response** (`internal/storage/watchdog.go`) — on a known mount-backed
target's device returning **unmounted** (a new `DevicePresent` liveness probe), the watchdog
**dispatches a benign by-UUID re-mount off the poll path** (a goroutine, never under the
lock), rate-limited per target to the debounce window. The mount is routed through the gate
as benign (`gateRemounter` in `main.go`, so `storage` stays decoupled from `reconcile`).
- **Disk-grow executor** (`internal/reconcile`) — `ActionResize` (benign `ClassResize`), planned
**grow-only** (desired DiskBytes > actual → `pct resize rootfs +<n>M`; a shrink is refused,
never silently grown) + a defensive executor guard (size must start with `+`). New
`proxmox.Client.ResizeLXC` (API; `VM.Config.Disk`+`Datastore.AllocateSpace`; async→UPID).
Built + fixture-tested; **unfed** live (no hub spec until slice 10).
- **Destructive storage ops through the slice-4 gate** (`internal/reconcile/storage_ops.go`) —
`IntentForStorageMount` (benign) and `IntentForStorageDestructive` (`ClassStorageWipe`/
`ClassDecommission`). Host/target-scoped: the op binds on the storage **target identity**
(carried in `target.guest_id`). Reuses the existing verifier/role-scoping/binding/audit — no
new gate, no new crypto. Storage cases added to the adversarial matrix (`storage_test.go`):
unsigned wipe → `pending_signature`; "wipe A" signature vs "wipe B" → `binding_mismatch`;
valid → accepted. **Inert** live.
- **`--selftest=storage` [`-watch <dur>`]** — the live USB-runbook harness: an observe pass
(full table incl. SMART + thin-pool data+metadata), and a bounded watchdog window with the
re-mount response live. Runs standalone on the Proxmox host (no hub).
- **`configs/felhom-agent.sudoers`** — the documented narrow allowlist (install unit / systemctl
manage / smartctl / lvs), with the agent-side fine validation noted.
- **Config**: `privileged.{unit_dir,stage_dir,systemctl,install,smartctl,lvs}` (paths must match
the sudoers entries).
### Notes
- Daemon still runs cleanly with no removable storage / no signers / no hub manifest, and a
missing/declined sudoers entry degrades with a warning (SMART→UNKNOWN, mount→logged error),
not a crash. `go test -race` passes (the watchdog re-mount dispatches off the poll path).
- Slice-3/4 + Phase-A exported surfaces, goldens, and adversarial tests intact. `authz`
untouched. The destructive-storage executor + grow are built/tested but unfed live until
slice 10.
## v0.5.0-rc1 — slice 5 Phase A: storage observe + report + watchdog (read-only, live) (2026-06-09)
Phase A of the storage slice (doc 03 §7). Read-only and live: the agent now observes every
host storage target, reports it into the host-report's `storage_targets` (previously an empty
stub), and runs a fast-poll watchdog that pushes a disconnect to the hub in seconds. No
host-root writes this phase (mounts/SMART/grow/destructive-gate are Phase B). The hub-owned
desired manifest (class/role/policy/creds) is not served until slice 10, so reconcile against
it is built-but-unfed — this phase ships only the genuinely-useful read-only footprint.
### Added
- **`internal/storage` package** (new):
- **`StorageTarget` wire contract** (`internal/hub/report.go`) — filled the slice-3 stub:
`name`/`type`/`durable_id`/`state`/`reachable`, usage (`total`/`used`/`avail`/
`used_fraction`), `content`, `mount_path`/`backing_device`, `class_hint` (rotational HINT
— never authoritative; class is hub-owned), `role` (empty until slice 10), a `thin_pool`
sub-object (lvmthin data fill; metadata fill is Phase B/`lvs`), and a `smart` sub-object
(`UNKNOWN` until Phase B). Cross-repo golden kept byte-identical with `felhom.eu/hub` and
guarded by the bidirectional key-set test (`contract_test.go`).
- **`durable_id` derivation** (`durableid.go`) — deterministic per type (the DR-load-bearing
re-attach key): fs-UUID (usb/local-dir), `server:export` (nfs/cifs), `repo+fingerprint`
(pbs), `vg/pool` (lvmthin); never empty (falls back to a stable store id).
- **`HostReader` seam + `ProcHostReader`** (`hostread.go`) — non-privileged `/proc/mounts`,
`/dev/disk/by-uuid`, `/sys/.../rotational` + `removable` reads. Root-free by construction.
- **`Observer`** (`observe.go`) — builds `[]hub.StorageTarget` from `ListStorage`/`NodeStorage`
joined with host reads; surfaces the lvmthin thin-pool data fill prominently (warns ≥85%).
- **Storage watchdog** (`watchdog.go`) — a third daemon goroutine fast-polling the *known*
target set (a defined Proxmox storage and/or a previously-seen one) for
`attached↔disconnected` transitions; on a transition it triggers an immediate, **debounced**
out-of-band host-report. Only flags a *known* target's change (never a never-attached
device); coalesces flaps within the debounce window (leading + trailing edge).
`CachingKnownTargets` rate-limits the Proxmox-derived known set; `HostLiveness` probes
device/mount presence (local) + a reachability dial (network), all non-privileged.
- **Proxmox `Storage` type** (`internal/proxmox/types.go`) — additive parse-only config fields
(`server`/`export`/`share`/`datastore`/`fingerprint`/`vgname`/`thinpool`) feeding durable_id.
- **Collector `StorageObserver` seam** (`internal/hub/collect.go`) — populates `storage_targets`
via the observer; a nil observer or an observe error degrades to empty (never sinks the
heartbeat). Hub does not import storage (storage imports hub for the wire type).
- **Out-of-band report trigger** (`internal/hub/loop.go`) — `Loop.SetTrigger`: a watchdog
signal runs one extra collect→report immediately without disturbing the regular cadence.
- **`StorageConfig`** (`internal/config`) — watchdog interval / debounce / known-refresh knobs
(all optional; package defaults otherwise).
- **Hub ingest** (`felhom.eu/hub`) — `hostReportPayload` now parses `storage_targets`
(full mirror struct), persists them via `report_json`, counts + warns on disconnected
targets, and has its own half of the bidirectional golden key-set test.
### Notes
- The daemon still runs cleanly with no removable storage, no signers, and no hub manifest —
the watchdog finds nothing to flag; storage reporting is best-effort.
- `proxmox`/`hub`/`authz`/`reconcile` exported surfaces + their golden/adversarial tests are
intact. No host-root writes, no destructive paths, no SMART this phase (all Phase B).
- Version: **v0.5.0-rc1** at the Phase-A checkpoint; **v0.5.0** when Phase B lands.
## v0.4.0 — slice 4 Phase B: reversibility gate + signed-op consuming layer (2026-06-08)
The security core of slice 4: hub-supplied intent stops being trusted for destructive
change. Layered in front of the per-guest queue's executor — **every** mutation now
passes the gate. Reuses `internal/authz` for all crypto (untouched surface). Inert
this slice: no destructive deltas are served until slice 10, so the destructive path is
classified, gated, and adversarially tested but not wired to live execution.
### Added
- **Classifier (`classify.go`, doc 03 §4)** — benign vs destructive by **provenance +
data-bearing-ness, NOT by verb**. The `OpClass` vocabulary (seeded by the committed
slice-2 `op_blob.json`: `guest_destroy`) is the agent-side contract slice 10 matches.
Destroy/overwrite of customer data is destructive UNLESS **agent-internal**
provenance (same-journaled-transaction create → compensating rollback, or
agent-tagged scratch) makes it benign. `Provenance` is journal-recorded and **never
populated from the hub** (its zero value is the only thing an external intent may
carry). Unknown op class fails safe → destructive.
- **Reversibility gate (`gate.go`)** — `Gate.Authorize(intent, signed)`: benign →
allowed unsigned; destructive → requires a verified, role-authorized, action-bound
operator signature, else refused **`pending_signature`**, never executed. Every
decision is written to an `AuditSink` (audit is a signal, never the guard).
- **Signed-op consuming layer over `authz`** — verifies via `authz.Verifier.Verify`
(the locked pipeline, untouched), then enforces on the `VerifiedOp`:
- **Role-scoping (doc 04 §4)** — recovery key authorizes key-rotation re-pins ONLY;
operational key authorizes ordinary destructive ops + planned rotation.
- **Op-to-action binding** — verified `op` + host + guest + `params` must match the
gated action (a signature for guest X / op A can't authorize guest Y / op B);
params compared semantically (key-order/whitespace independent).
- **Signed-job orchestration (`job.go`)** — `RunSignedJob`: idempotency dedupe (the
op nonce as the journal key — a redelivered completed op is skipped, not re-run),
gate authorization, then journal-wrapped execution via an injected
`DestructiveExecutor` (nil this slice — authorized destructive ops are inert, no
executor wired until 6/7).
- **Crash-recovery consumer (`recover.go`, Note 1 / doc 03 §10)** — `Engine.Recover`
consumes the journal's `InFlight()` at startup: an op that crashed AFTER the Proxmox
POST and BEFORE its terminal record (`OpTaskRunning`, nonce already consumed) is NOT
covered by idempotency dedupe — only this resume-or-rollback resolves it (re-read the
task via the new `TaskStatusOnce`, record the real outcome; a no-task-id op is
abandoned fail-safe). Landed together with the signed-op executor, as Note 1 required.
- **Daemon wiring** — `runDaemon` builds the verifier from `config.Authz.Signers` (a
bad key / missing nonce-store path is a fatal misconfig; **no signers = nil verifier**,
the common slice-4 state), constructs the gate (+ `SlogAudit`), runs `Recover` before
issuing any mutation, and routes every reconcile action through the gate.
### Changed
- **Memory comparison canonicalized (Note 2)** — `desiredMemoryMiB` makes the
desired↔actual memory compare in the same MiB unit that is then written, so a
non-MiB-aligned `MemoryBytes` converges in one pass instead of re-issuing SetConfig
forever (the numeric cousin of the description-newline normalization). Test proves
convergence. Slice 10 should still serve MiB-aligned specs at the source.
### Tests (the security proof — each independently rejected)
- **Adversarial matrix** via the REAL `authz.Verifier` with in-test-minted SSHSIGs
(framing replicated in reconcile's test binary; production authz untouched, no signing
added to the verify-only package): unsigned destructive **job** → pending_signature;
unsigned destructive **desired-state delta** → pending_signature (distrusts hub
desired state, not just jobs); forged/unknown signer → `ErrUnknownSigner`; expired →
`ErrExpired`; **replayed nonce across an agent restart** (durable `FileNonceStore`) →
`ErrReplay`; wrong host → `ErrTarget`; wrong guest / wrong op / wrong params →
binding_mismatch; **recovery key on ordinary destructive** → role_denied;
**hub-supplied "scratch" tag ignored** → still destructive → refused; **valid + role +
target + fresh nonce → accepted**, and a second presentation → `ErrReplay` (nonce
consumed).
- Classifier (benign/destructive/provenance/key-rotation/fail-safe), role-scoping,
params binding, crash-recovery (resume OK / fail / still-running / no-task rollback /
unreadable / one-shot key applied on resume), signed-job idempotency (execute once,
dedupe redelivery, refused-not-executed, no-executor-inert, executor-error).
- Full module **race-clean** (`go test -race`) + vet clean on the Linux build server.
## v0.4.0-rc1 — slice 4 Phase A: reconcile engine (structural; runs live, unfed) (2026-06-08)
The agent-side control core's structural half. **Checkpoint marker**`-rc1` is the
Phase-A push; awaiting validation before Phase B (the reversibility gate + signed-op
consuming layer) lands the final **v0.4.0**. Runs LIVE but UNFED: with no desired-state
provider until slice 10, the live engine computes an empty action set and performs
**zero mutations**.
### Added
- **`internal/reconcile`** package — the engine, the per-guest serializer, the
desired-state model, the normalization layer, and the durable op journal:
- **Per-guest serializer (`Queue`, doc 03 §10)** — the single choke point ALL
mutation sources funnel through. Same-vmid jobs run strictly one-at-a-time in
submit order; independent vmids run in parallel. Each vmid is a cond-var FIFO lane
(unbounded, non-blocking, order-preserving); graceful drain on `Close`.
- **Desired-state model + `DesiredProvider` seam** — `DesiredGuest` (per-field
optional: run-state / `*hub.GuestSpec` / `*description`), `DesiredState`. The only
live provider is **`EmptyProvider`** (slice 4 has no source); `StaticProvider`
feeds fixtures. The seam is where slice 10's hub-serving plugs in — no hub/local
source invented here.
- **Normalization layer (`FieldNormalizers`)** — reconcile compares *normalized*
desired-vs-actual so Proxmox round-trip quirks don't read as drift. `description`'s
trailing newline is the first registered case; the registry takes more (boolean
coercion, list ordering) as discovered. `normDesc` **promoted** out of
`cmd/felhom-agent/main.go` to **`reconcile.NormDescription`**; the `--selftest=task`
description round-trip now uses that shared helper (one source of truth for the quirk).
- **Plan engine (`Plan`, pure function)** — computes the minimal **benign** action set
(`Start`/`Stop`/`SetConfig`) for guests present in both desired and actual, with
normalized comparison, deterministic vmid ordering, config-before-run-state. Skips
provision (desired-absent-in-actual, slice 7) and destroy (actual-absent-in-desired,
gated, slice 10); never writes a config it couldn't first read (`SpecKnown`). Disk
(rootfs grow) intentionally not reconciled here.
- **Reconcile engine (`Engine`)** — reads desired+actual, plans, dispatches each action
onto the shared queue. Every Proxmox op handled per the mutate.go contract: non-empty
UPID → `WaitTask` + assert `exitstatus`; empty UPID → clean **synchronous** success
(slice-4 proven). Per-action failures are counted, not fatal (other guests still
converge).
- **Operation journal (`Journal`)** — durable fsync'd append-only JSONL mirroring
`authz.FileNonceStore`: records each op's lifecycle (started → task_running →
succeeded/failed) with its Proxmox task id (crash mid-op is detected and re-checkable
on restart via `InFlight()`), plus an **idempotency-key store** (`AlreadyApplied`) so
a one-shot op never re-runs across retries/restarts. Reconcile actions carry no
idempotency key (convergent — must re-run on real drift).
- **Daemon wiring (`runDaemon`)** — reconcile runs alongside the hub loop on the poll
cadence, **sharing the per-guest queue**. Journal path is a `journal.log` sibling of the
nonce store. The daemon runs cleanly with **no desired state and no signers** (reconcile
is a logged live no-op; a journal-open failure degrades to journal-less, never crashes).
### Tests
- Serializer: same-guest serialized (max-concurrency 1, submit order preserved) and
different-guests parallel (cross-waiting jobs both complete — would deadlock if not);
error propagation; drain-pending-on-close; submit-after-close.
- Normalization: description round-trip; unknown-field identity; extensibility seam
(synthetic boolean-coercion + list-ordering normalizers).
- Plan: run-state start/stop, spec drift (cores/memory), disk-not-reconciled,
description-newline-not-drift, unmanaged fields, spec-unknown skips config keeps
run-state, desired-absent skipped, combined ordering, empty-desired no-op, deterministic
vmid order.
- Engine: empty-provider zero mutations; async start (WaitTask); synchronous SetConfig
(no WaitTask); WaitTask failure + POST error counted failed; list error = pass failure.
- Journal: lifecycle latest-wins; in-flight survives restart; idempotency dedupe across
restart; failed key not applied; torn-trailing-line skipped.
- Full module **race-clean** (`go test -race`) on the Linux build server; vet clean.
### Not in this phase (Phase B)
- The benign/destructive classifier, the reversibility gate, and the signed-op consuming
layer over `internal/authz` (doc 03 §4 / doc 04) — added next, in front of the queue's
executor, landing **v0.4.0**.
## v0.3.2 — SetConfig selftest extension (slice-4 pre-check) (2026-06-08)
The gate before slice 4: prove `SetConfig` works live under the scoped token before
reconcile is built on it. **Self-gated live run PASSED** on `demo-felhom`/guest 9999.
### Added
- **Reversible `SetConfig` step appended to `--selftest=task`** (`cmd/felhom-agent/main.go`,
`selftestSetConfig`): read `GuestConfig` → write a `description` marker
(`felhom-selftest <RFC3339>`) → verify it landed → restore the original value (or
`delete` the key if it was absent) → verify the restore. Handles PVE's dual-mode
`SetConfig` return per the `mutate.go` contract: empty UPID = synchronous success
(printed `synchronous`); non-empty UPID = `WaitTask` + assert `exitstatus=OK`.
The existing snapshot → rollback → delete-snapshot steps are unchanged. First live
exercise of the **`VM.Config.*`** privilege cluster.
- **`normDesc` / `extraString` helpers** — `extraString` decodes a string-valued key
from `GuestConfig.Extra` (raw JSON); `normDesc` strips the trailing newline PVE
appends to `description` on read, so a written value round-trips equal.
### Finding (live)
- The LXC `description` write returned **synchronous (empty UPID)** — PVE applied it
inline, no task. The agent's dual-mode `SetConfig` modeling is correct: the
empty-string path is real and must not be treated as an error.
- PVE **appends a trailing `\n` to `description`** on read (stored URL-encoded as
`%0A`). A naive exact-match reconcile would see perpetual drift — slice-4 reconcile
must normalize `description` comparisons (hence `normDesc`).
### Ops
- Standing operator token (`felhom-agent@pve!agent`, privsep) **rotated** during this
run (the prior secret was not retrievable); role + both user/token ACL rows
re-confirmed at `/`. New secret stored out-of-band, **not persisted to the repo**.
Guest 9999 left pristine (stopped, no `description`, no leftover snapshot). Version → 0.3.2.
## Docs + live validation — no version bump (2026-06-08)
### Changed
- **Reflowed `CLAUDE.md`** — removed hard mid-paragraph line wraps (prose, list items, blockquotes now single-line, soft-wrapped); code blocks and tables untouched; rendered output unchanged.
- **Unified the REPORT/CHANGELOG convention** in `CLAUDE.md`: `CHANGELOG.md` is the cumulative log (newest on top); `REPORT.md` is overwritten with the most-recent implementation/validation only. Added an explicit **no-secrets** rule (never write tokens/passwords/keys into committed files; reference them as stored out-of-band).
### Added
- **`REPORT.md`** rewritten for the live `--selftest=task` validation on the demo host (`demo-felhom`): snapshot → rollback → delete-snapshot on guest 9999, each polled to `exitstatus=OK` under the `felhom-agent@pve!agent` privsep token (UPIDs name the token actor — privsep path genuinely exercised); 16-privilege `FelhomAgent` role + both user & token ACLs confirmed; `--selftest=read` clean. Closes the slice-1 "mutating ops unit-tested only" gap; `WaitTask` async foundation validated live → **slice 4 unblocked**. (Token secret stored out-of-band, not in the repo.)
## v0.3.1 — slice-3 validation follow-ups (2026-06-08)
### Changed
- **Collector keeps the known run-status on a `GuestConfig` failure** (`internal/hub/collect.go`):
previously a per-guest config-read error forced `status="unknown"`; now the run-status from
`ListLXC` is preserved (only the `spec` is dropped). An empty status is still normalized to
`unknown` (wire value is always `running|stopped|unknown`). Test renamed to
`TestCollect_GuestConfigFailureKeepsStatusOmitsSpec` and asserts the preserved `running` + nil spec.
- **`--selftest` usage** error string now reads `(want read|task|hub)`.
### Added
- **Cross-repo contract fixture** `internal/hub/testdata/host-report.golden.json` +
`TestHostReport_ContractMatchesGolden` — compares the marshaled `HostReport` field-name sets
(top level + `host` + `guests[0]`) against the golden, failing on any json-tag drift. The file is
**kept byte-identical** with felhom-hub's copy (duplicated contract until a shared types module;
revisit when slices 5/6 populate the empty collections). Version → 0.3.1.
## v0.3.0 — hub client + host-report + first daemon loop (slice 3) (2026-06-08)
The agent's first daemon: a periodic read-only host-report POSTed to the hub (the
heartbeat). No Proxmox mutations, no desired-state/signed-op consumption, no
storage/backup collection yet — those are slices 4/5/6.
### Added
- **`internal/hub`** package:
- **`HostReport`** wire contract (`report.go`) shared field-for-field with the hub
ingest: host metrics, guests (`vmid` + spec), `cloudflared` status, and the
`storage_targets`/`backups`/`restore_tests`/`pbs_snapshots`/`audit_tail`
collections **defined but emitted empty** (typed `[]`, slices 5/6 fill them).
- **`Collector`** (`collect.go`) builds the report from a read-only `proxmoxReader`
(adapted to the real `internal/proxmox` surface — node held by the client, value
returns, `proxmox.Guest`) + a `CloudflaredProber`. Partial-failure policy: a
failed `NodeStatus` is a hard error (skip the POST); a failed per-guest
`GuestConfig` degrades that guest to `status="unknown"` (spec omitted) but still
sends; a cloudflared probe failure → `"unknown"`, never fatal.
- **`CloudflaredProber`** + `SystemctlProber` (`systemctl is-active cloudflared`;
read-only — NOT a Privileged/root op; tunnel management is a later slice).
- **`Client`** (`client.go`): `POST /api/v1/host-report` with
`Authorization: Bearer <key>`, standard TLS (system roots or optional `ca_file`;
verification always on). Typed `*TransportError` / `*HTTPError`; the bearer token
never appears in any error.
- **`Loop`** (`loop.go`): the daemon — immediate first report then tick; adopts the
hub's `poll_interval_seconds` clamped to [60,3600]; resilient (a collect/report
error is logged and the loop continues); clean shutdown on context cancel.
- **`ControlEnvelope`**: only `poll_interval_seconds` is acted on; `blocked` /
`desired_generation` / `has_signed_ops` are parsed-but-ignored (logged at most)
pending reconcile (slice 4).
- **Config**: `HubConfig` (url/host_id/api_key/poll_seconds/timeout_seconds/ca_file),
`FELHOM_AGENT_HUB_*` env overlay, `HubConfig.Validate()` (mode-aware — proxmox-only
`--selftest=read|task` still runs without hub config), `WithDefaults()`, and
`Redacted()` now also blanks the hub key. `configs/agent.example.json` gains `hub`
(and `authz`) blocks.
- **`cmd/felhom-agent`**: the no-`--selftest` mode is now the **daemon** (poll loop);
added **`--selftest=hub`** (one collect+report, prints the report + envelope).
Version 0.2.0 → 0.3.0.
### Tests
- Report serialization (field names; empty collections are `[]` not `null`; spec
omitted when unknown); client (Bearer header, non-2xx→`*HTTPError`,
transport→`*TransportError`, **token never in error**); collector (host mapping,
guest spec, per-guest failure degrades-but-still-reports, NodeStatus hard error,
cloudflared error→unknown); loop (immediate first report, continuation after an
injected error, interval adoption + clamp); config (hub validate/redact/env).
### Notes
- `internal/proxmox` and `internal/authz` were **not touched** — no new proxmox
surface was needed (`ListLXC` already exposes status/maxmem/maxdisk; `GuestConfig`
exposes cores). The task's `proxmoxReader` sketch (node-arg/pointer/`LXC`) was
adapted to the real exports as instructed.
- **Defined-but-empty** this slice: `storage_targets`, `backups`, `restore_tests`,
`pbs_snapshots`, `audit_tail` (slices 5/6). **Parsed-but-ignored**: the envelope's
`blocked`/`desired_generation`/`has_signed_ops` (slice 4).
## v0.2.0 — `authz` signed-op verifier (slice 2) (2026-06-08)
Production form of the Phase-4 signing primitive: a key-type-agnostic SSHSIG
verifier for operator-signed destructive ops, with the full anti-replay/
authorization pipeline and a durable, crash-safe nonce store. What slice 4
(reconcile) will call to gate destructive desired-state deltas. No hub, no signing
CLI, no reconcile loop.
### Added
- **`internal/authz``Verifier`**: `New(signers, store, hostID)` + `Verify(blob,
sigArmored) (*VerifiedOp, error)`. Runs the LOCKED pipeline (order is
load-bearing): parse armor → namespace → parse pubkey → allow-list (by key
**material**, `pub.Marshal()` equality, not key_id) → crypto verify (over the
**raw received bytes**, never re-canonicalized) → parse blob → target → time
window → **nonce recorded LAST**. Each post-crypto stage rejects even with a
valid signature.
- **SSHSIG framing** (`sshsig.go`) via `golang.org/x/crypto/ssh` — `pem.Decode` →
strip 6-byte magic → `ssh.Unmarshal` → `ssh.ParsePublicKey` → recompute signed
data with the named hash → `pub.Verify` (dispatches on key algorithm). No
hand-rolled crypto. Key-type-agnostic: ed25519 / **sk-ssh-ed25519 (FIDO2)** /
rsa / ecdsa via the one path.
- **Fixed namespace** `felhom-op-v1` (package constant, never caller-supplied).
- **`OpBlob`** (corrected `host_id`/`guest_id` json tags) + **`VerifiedOp`** (op,
host/guest, params, key_id, matched signer). key_id is advisory/audit only —
never an authz input.
- **Typed errors**: `ErrMalformed, ErrNamespace, ErrUnknownSigner, ErrBadSignature,
ErrTarget, ErrExpired, ErrNotYetValid, ErrReplay` (errors.Is-friendly).
- **`NonceStore`** + two impls: `MemoryNonceStore` (tests) and **`FileNonceStore`**
— durable, crash-safe (fsync'd append log, replayed into an index on open,
periodic compaction, expiry-only pruning). A nonce is fsync'd to disk before
`SeenOrRecord` returns false; replay protection survives restart; I/O failure
fails safe (reports seen=true). Target generalization: host_id matched strictly,
guest_id surfaced for the caller to route.
- **Config**: `AuthzConfig` (nonce-store path + pinned operator `signers` tagged
`operational`/`recovery` with a key_id, as authorized_keys lines).
- **Version 0.2.0.**
### Tests
- Real OpenSSH interop via a committed `ssh-keygen -Y sign` vector (hermetic CI);
per-stage rejection (each with an otherwise-valid sig); the headline
**invalid-sig-does-not-burn-the-nonce** invariant; replay; **persistence across
restart**; synthetic **sk-ssh-ed25519** through the unchanged path; byte-exactness
(a re-serialized blob fails crypto — not re-canonicalized).
### Notes / corrections to the Phase-4 reference
- §7's `Target` lacked json tags (`host_id`/`guest_id`) — fixed.
- The doc paired "Go 1.24.4 / x/crypto v0.52.0", but v0.52.0 declares `go 1.25.0`
and does **not** build on Go 1.24. Resolved by upgrading the build server to
go1.26.0 (backward-compatible; felhom-controller/hub unaffected); the module is
`go 1.25.0` on x/crypto v0.52.0.
- Free function → constructed `Verifier`; returns the full `VerifiedOp`; typed
errors; clock-skew tolerance added; durable nonce store is the net-new work.
- **Shared-contract dependency flagged** (not built): the hub and the `felhom-sign`
CLI must emit byte-identical canonical JSON or signatures won't verify; a shared
canonicalizer both import would be the right home.
## v0.1.0 — Scaffold + `proxmox` interaction layer (slice 1) (2026-06-08)
First slice: stand up the host-agent project and its foundation — the typed
Proxmox interaction layer every other module will call. No reconcile loop, hub
client, signing, or storage/backup orchestration yet (later slices).
### Added
- **Project scaffold**: module `gitea.dooplex.hu/admin/felhom-agent`, binary
`felhom-agent` (`cmd/felhom-agent/`), Go 1.24, zero external dependencies
(pure stdlib). `--version` flag; `version` var overridable via
`-ldflags "-X main.version=<v>"`.
- **`internal/proxmox` — API backend (`Client`)**: hand-rolled REST client over
`https://<host>:8006/api2/json` with `PVEAPIToken` auth. Typed read ops
(`Version`, `Nodes`, `NodeStatus`, `ListLXC`, `GuestStatus`, `GuestConfig`,
`ListStorage`, `NodeStorage`, `StorageContent`) and async mutating ops
returning a UPID (`RestoreLXC` — the primary create path, `Vzdump`, `Snapshot`,
`Rollback`, `DeleteSnapshot`, `SetConfig`, `Start`, `Stop`).
- **`WaitTask`**: polls `GET /nodes/{node}/tasks/{upid}/status` until stopped, then
asserts `exitstatus == "OK"` (authorization can surface at task execution, not
the POST — phase1-2 §1.3). Exponential backoff (1s→5s cap), context
cancellation + timeout. `*APIError` parses the offending privilege from a 403;
`*TaskError` parses it from a failed task exitstatus + log tail.
- **`internal/proxmox` — fenced root-CLI backend (`Privileged`)**: limited to the
three proven OS-root exceptions only — `CreateGoldenLXC` (keyctl `pct create`),
`MountUSBByUUID`, `SMART`, `Sensors`; each cites why it can't be the API. Fence
is structural (Client never shells out, Privileged never makes an HTTP call) and
asserted in tests.
- **TLS trust**: SHA-256 leaf-cert pinning (the host serves a self-signed cert) or
a CA file; an explicitly-named `insecure_skip_verify` that is off by default. No
blanket verification disable.
- **`internal/config`**: JSON config file + `FELHOM_AGENT_*` env overrides; the
token secret is never logged (`Redacted()`).
- **`internal/log`**: slog setup (text, stderr, configurable level).
- **`cmd/felhom-agent --selftest`**: read-only health report against a live host
(version/nodes/status/guests/storage); `--selftest=task --vmid N` exercises
`WaitTask` on a reversible snapshot→rollback→delete op (gated; default selftest
mutates nothing).
- **Tests**: unit tests with a mock HTTP transport + mock runner (UPID parse,
`WaitTask` running→OK / failed-403 / timeout / ctx-cancel, 403→privilege error,
response decoding against shapes captured live from `demo-felhom`, config
redaction, and the API-vs-root routing fence).
### Notes
- Types are grounded in the spike findings
(`felhom.eu/documentation/proxmox-platform.md`, `tests/phase{0,1-2,3}-findings.md`)
and the exact JSON shapes captured live from `demo-felhom` (PVE 9.2.2).
- Verified: `go build/vet/test` green on Go 1.24.4 (build server) and a live
read-only `--selftest` against the demo host with TLS fingerprint pinning.
- The 16-privilege `FelhomAgent` role + privsep token (role on **both** user and
token) is provisioned out-of-band; the agent only consumes the token.