77b4f21450
Surfaced preparing the live USB validation on demo-felhom: - observe.go: an unmounted removable dir-storage no longer falls through to the ROOT fs for its backing device/UUID — durable_id was becoming uuid:<root-uuid> (a DR mis-id that would re-attach the wrong disk). Now derived only from the target's own mountpoint; unmounted → no device + stable store:<name> durable_id. Removed containingMountDevice. - watchdog.go: remember the fs-UUID observed while attached and backfill it onto the re-mount target, so re-mount works even if the known-set cache refreshed mid-drop (doc 03 §7 "sourced from the existing definition"). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
497 lines
34 KiB
Markdown
497 lines
34 KiB
Markdown
# Changelog
|
|
|
|
All notable changes to **felhom-agent** are recorded here. Update on every code
|
|
change that gets pushed.
|
|
|
|
## v0.5.1 — slice 5 live-validation prep: durable_id mis-id fix + re-mount UUID memory (2026-06-09)
|
|
|
|
Two correctness fixes surfaced while preparing the live USB validation on `demo-felhom`
|
|
(a real 1TB USB HDD, sdb1, ext4). Both are DR-load-bearing — exactly the "false-id →
|
|
re-attach the wrong disk" failure mode the slice warned about.
|
|
|
|
### Fixed
|
|
- **Unmounted dir-storage no longer inherits the ROOT filesystem's UUID** (`observe.go`).
|
|
Previously, when a removable dir-storage was unmounted, the observer fell through to the
|
|
*containing* mount (root) for the backing device, so its `durable_id` became
|
|
`uuid:<root-uuid>` — a catastrophic DR mis-id (the hub would re-attach the wrong disk).
|
|
Now the backing device/UUID/`durable_id` are derived ONLY from the target's OWN
|
|
mountpoint; an unmounted target reports no device and a stable `store:<name>` durable_id,
|
|
never another filesystem's UUID. (Removed the `containingMountDevice` root-fallthrough.)
|
|
- **Watchdog remembers the fs-UUID observed while attached** (`watchdog.go`) so a re-mount
|
|
works even after the known-set cache refreshes mid-drop (an unmounted target can't resolve
|
|
its own UUID). The re-mount key is backfilled from this memory — aligning with doc 03 §7's
|
|
"sourced from the existing definition, no hub manifest needed": the agent learns the UUID
|
|
while the target is attached, then re-mounts by it on return.
|
|
|
|
### Tests
|
|
- Observer: an unmounted dir-storage asserts NO `uuid:` durable_id and no backing device.
|
|
- Watchdog: a drop where the cache lost the UUID still re-mounts using the remembered UUID.
|
|
|
|
## v0.5.0 — slice 5 Phase B: the host-root surface (mounts + SMART + grow + destructive gate) (2026-06-09)
|
|
|
|
The write surface — the agent's first step outside its Proxmox API token into OS-root.
|
|
Isolated behind a narrow, argument-validated, adversarially-tested seam, exactly like the
|
|
slice-4 gate. Completes slice 5 (Phase A = read-only observe/report/watchdog at v0.5.0-rc1).
|
|
|
|
### Added
|
|
- **`HostOps` seam + `SudoHostOps`** (`internal/storage/hostops.go`) — the one privileged
|
|
host surface: persistent mounts via **systemd `.mount` units keyed by fs-UUID** (enabled to
|
|
survive reboot), detach (stop+disable), SMART, and thin-pool metadata. Shells out via the
|
|
fenced Runner (`sudo -n`, **fixed arg vectors, no shell**); a fake backs the tests (no real
|
|
root in the suite). `NoopHostOps` is the safe fallback when the surface is unavailable.
|
|
- **The argument validator** (`internal/storage/validate.go`) — the security boundary:
|
|
`ValidateUUID` (strict hex), `ValidateMountPath` (absolute, no traversal, no metacharacters),
|
|
`ValidateSMARTDevice` (raw-disk whitelist), `ValidateLVMName`, and an in-process
|
|
`systemdEscapePath` (no `systemd-escape` shell-out). **Every argument is validated BEFORE a
|
|
command is constructed.** Headline test (`validate_test.go`): an adversarial matrix of
|
|
shell metacharacters / `../` traversal / malformed inputs is rejected with **zero exec**.
|
|
- **SMART** (`internal/storage/smart.go`) — parses `smartctl -a -j` into `StorageTarget.smart`:
|
|
**SATA** (reallocated/pending/offline-uncorrectable, temp, power-on-hours) **and NVMe**
|
|
(critical_warning, media_errors, percentage_used, temp), degrading to `UNKNOWN` for devices
|
|
with no SMART (USB-SATA bridges). **`lvs`** fills the lvmthin thin-pool **metadata** fill
|
|
(the value Phase A left null). Wired into the Observer's enrichment (Observe only, not the
|
|
watchdog's fast Known path).
|
|
- **Watchdog re-mount response** (`internal/storage/watchdog.go`) — on a known mount-backed
|
|
target's device returning **unmounted** (a new `DevicePresent` liveness probe), the watchdog
|
|
**dispatches a benign by-UUID re-mount off the poll path** (a goroutine, never under the
|
|
lock), rate-limited per target to the debounce window. The mount is routed through the gate
|
|
as benign (`gateRemounter` in `main.go`, so `storage` stays decoupled from `reconcile`).
|
|
- **Disk-grow executor** (`internal/reconcile`) — `ActionResize` (benign `ClassResize`), planned
|
|
**grow-only** (desired DiskBytes > actual → `pct resize rootfs +<n>M`; a shrink is refused,
|
|
never silently grown) + a defensive executor guard (size must start with `+`). New
|
|
`proxmox.Client.ResizeLXC` (API; `VM.Config.Disk`+`Datastore.AllocateSpace`; async→UPID).
|
|
Built + fixture-tested; **unfed** live (no hub spec until slice 10).
|
|
- **Destructive storage ops through the slice-4 gate** (`internal/reconcile/storage_ops.go`) —
|
|
`IntentForStorageMount` (benign) and `IntentForStorageDestructive` (`ClassStorageWipe`/
|
|
`ClassDecommission`). Host/target-scoped: the op binds on the storage **target identity**
|
|
(carried in `target.guest_id`). Reuses the existing verifier/role-scoping/binding/audit — no
|
|
new gate, no new crypto. Storage cases added to the adversarial matrix (`storage_test.go`):
|
|
unsigned wipe → `pending_signature`; "wipe A" signature vs "wipe B" → `binding_mismatch`;
|
|
valid → accepted. **Inert** live.
|
|
- **`--selftest=storage` [`-watch <dur>`]** — the live USB-runbook harness: an observe pass
|
|
(full table incl. SMART + thin-pool data+metadata), and a bounded watchdog window with the
|
|
re-mount response live. Runs standalone on the Proxmox host (no hub).
|
|
- **`configs/felhom-agent.sudoers`** — the documented narrow allowlist (install unit / systemctl
|
|
manage / smartctl / lvs), with the agent-side fine validation noted.
|
|
- **Config**: `privileged.{unit_dir,stage_dir,systemctl,install,smartctl,lvs}` (paths must match
|
|
the sudoers entries).
|
|
|
|
### Notes
|
|
- Daemon still runs cleanly with no removable storage / no signers / no hub manifest, and a
|
|
missing/declined sudoers entry degrades with a warning (SMART→UNKNOWN, mount→logged error),
|
|
not a crash. `go test -race` passes (the watchdog re-mount dispatches off the poll path).
|
|
- Slice-3/4 + Phase-A exported surfaces, goldens, and adversarial tests intact. `authz`
|
|
untouched. The destructive-storage executor + grow are built/tested but unfed live until
|
|
slice 10.
|
|
|
|
## v0.5.0-rc1 — slice 5 Phase A: storage observe + report + watchdog (read-only, live) (2026-06-09)
|
|
|
|
Phase A of the storage slice (doc 03 §7). Read-only and live: the agent now observes every
|
|
host storage target, reports it into the host-report's `storage_targets` (previously an empty
|
|
stub), and runs a fast-poll watchdog that pushes a disconnect to the hub in seconds. No
|
|
host-root writes this phase (mounts/SMART/grow/destructive-gate are Phase B). The hub-owned
|
|
desired manifest (class/role/policy/creds) is not served until slice 10, so reconcile against
|
|
it is built-but-unfed — this phase ships only the genuinely-useful read-only footprint.
|
|
|
|
### Added
|
|
- **`internal/storage` package** (new):
|
|
- **`StorageTarget` wire contract** (`internal/hub/report.go`) — filled the slice-3 stub:
|
|
`name`/`type`/`durable_id`/`state`/`reachable`, usage (`total`/`used`/`avail`/
|
|
`used_fraction`), `content`, `mount_path`/`backing_device`, `class_hint` (rotational HINT
|
|
— never authoritative; class is hub-owned), `role` (empty until slice 10), a `thin_pool`
|
|
sub-object (lvmthin data fill; metadata fill is Phase B/`lvs`), and a `smart` sub-object
|
|
(`UNKNOWN` until Phase B). Cross-repo golden kept byte-identical with `felhom.eu/hub` and
|
|
guarded by the bidirectional key-set test (`contract_test.go`).
|
|
- **`durable_id` derivation** (`durableid.go`) — deterministic per type (the DR-load-bearing
|
|
re-attach key): fs-UUID (usb/local-dir), `server:export` (nfs/cifs), `repo+fingerprint`
|
|
(pbs), `vg/pool` (lvmthin); never empty (falls back to a stable store id).
|
|
- **`HostReader` seam + `ProcHostReader`** (`hostread.go`) — non-privileged `/proc/mounts`,
|
|
`/dev/disk/by-uuid`, `/sys/.../rotational` + `removable` reads. Root-free by construction.
|
|
- **`Observer`** (`observe.go`) — builds `[]hub.StorageTarget` from `ListStorage`/`NodeStorage`
|
|
joined with host reads; surfaces the lvmthin thin-pool data fill prominently (warns ≥85%).
|
|
- **Storage watchdog** (`watchdog.go`) — a third daemon goroutine fast-polling the *known*
|
|
target set (a defined Proxmox storage and/or a previously-seen one) for
|
|
`attached↔disconnected` transitions; on a transition it triggers an immediate, **debounced**
|
|
out-of-band host-report. Only flags a *known* target's change (never a never-attached
|
|
device); coalesces flaps within the debounce window (leading + trailing edge).
|
|
`CachingKnownTargets` rate-limits the Proxmox-derived known set; `HostLiveness` probes
|
|
device/mount presence (local) + a reachability dial (network), all non-privileged.
|
|
- **Proxmox `Storage` type** (`internal/proxmox/types.go`) — additive parse-only config fields
|
|
(`server`/`export`/`share`/`datastore`/`fingerprint`/`vgname`/`thinpool`) feeding durable_id.
|
|
- **Collector `StorageObserver` seam** (`internal/hub/collect.go`) — populates `storage_targets`
|
|
via the observer; a nil observer or an observe error degrades to empty (never sinks the
|
|
heartbeat). Hub does not import storage (storage imports hub for the wire type).
|
|
- **Out-of-band report trigger** (`internal/hub/loop.go`) — `Loop.SetTrigger`: a watchdog
|
|
signal runs one extra collect→report immediately without disturbing the regular cadence.
|
|
- **`StorageConfig`** (`internal/config`) — watchdog interval / debounce / known-refresh knobs
|
|
(all optional; package defaults otherwise).
|
|
- **Hub ingest** (`felhom.eu/hub`) — `hostReportPayload` now parses `storage_targets`
|
|
(full mirror struct), persists them via `report_json`, counts + warns on disconnected
|
|
targets, and has its own half of the bidirectional golden key-set test.
|
|
|
|
### Notes
|
|
- The daemon still runs cleanly with no removable storage, no signers, and no hub manifest —
|
|
the watchdog finds nothing to flag; storage reporting is best-effort.
|
|
- `proxmox`/`hub`/`authz`/`reconcile` exported surfaces + their golden/adversarial tests are
|
|
intact. No host-root writes, no destructive paths, no SMART this phase (all Phase B).
|
|
- Version: **v0.5.0-rc1** at the Phase-A checkpoint; **v0.5.0** when Phase B lands.
|
|
|
|
## v0.4.0 — slice 4 Phase B: reversibility gate + signed-op consuming layer (2026-06-08)
|
|
|
|
The security core of slice 4: hub-supplied intent stops being trusted for destructive
|
|
change. Layered in front of the per-guest queue's executor — **every** mutation now
|
|
passes the gate. Reuses `internal/authz` for all crypto (untouched surface). Inert
|
|
this slice: no destructive deltas are served until slice 10, so the destructive path is
|
|
classified, gated, and adversarially tested but not wired to live execution.
|
|
|
|
### Added
|
|
- **Classifier (`classify.go`, doc 03 §4)** — benign vs destructive by **provenance +
|
|
data-bearing-ness, NOT by verb**. The `OpClass` vocabulary (seeded by the committed
|
|
slice-2 `op_blob.json`: `guest_destroy`) is the agent-side contract slice 10 matches.
|
|
Destroy/overwrite of customer data is destructive UNLESS **agent-internal**
|
|
provenance (same-journaled-transaction create → compensating rollback, or
|
|
agent-tagged scratch) makes it benign. `Provenance` is journal-recorded and **never
|
|
populated from the hub** (its zero value is the only thing an external intent may
|
|
carry). Unknown op class fails safe → destructive.
|
|
- **Reversibility gate (`gate.go`)** — `Gate.Authorize(intent, signed)`: benign →
|
|
allowed unsigned; destructive → requires a verified, role-authorized, action-bound
|
|
operator signature, else refused **`pending_signature`**, never executed. Every
|
|
decision is written to an `AuditSink` (audit is a signal, never the guard).
|
|
- **Signed-op consuming layer over `authz`** — verifies via `authz.Verifier.Verify`
|
|
(the locked pipeline, untouched), then enforces on the `VerifiedOp`:
|
|
- **Role-scoping (doc 04 §4)** — recovery key authorizes key-rotation re-pins ONLY;
|
|
operational key authorizes ordinary destructive ops + planned rotation.
|
|
- **Op-to-action binding** — verified `op` + host + guest + `params` must match the
|
|
gated action (a signature for guest X / op A can't authorize guest Y / op B);
|
|
params compared semantically (key-order/whitespace independent).
|
|
- **Signed-job orchestration (`job.go`)** — `RunSignedJob`: idempotency dedupe (the
|
|
op nonce as the journal key — a redelivered completed op is skipped, not re-run),
|
|
gate authorization, then journal-wrapped execution via an injected
|
|
`DestructiveExecutor` (nil this slice — authorized destructive ops are inert, no
|
|
executor wired until 6/7).
|
|
- **Crash-recovery consumer (`recover.go`, Note 1 / doc 03 §10)** — `Engine.Recover`
|
|
consumes the journal's `InFlight()` at startup: an op that crashed AFTER the Proxmox
|
|
POST and BEFORE its terminal record (`OpTaskRunning`, nonce already consumed) is NOT
|
|
covered by idempotency dedupe — only this resume-or-rollback resolves it (re-read the
|
|
task via the new `TaskStatusOnce`, record the real outcome; a no-task-id op is
|
|
abandoned fail-safe). Landed together with the signed-op executor, as Note 1 required.
|
|
- **Daemon wiring** — `runDaemon` builds the verifier from `config.Authz.Signers` (a
|
|
bad key / missing nonce-store path is a fatal misconfig; **no signers = nil verifier**,
|
|
the common slice-4 state), constructs the gate (+ `SlogAudit`), runs `Recover` before
|
|
issuing any mutation, and routes every reconcile action through the gate.
|
|
|
|
### Changed
|
|
- **Memory comparison canonicalized (Note 2)** — `desiredMemoryMiB` makes the
|
|
desired↔actual memory compare in the same MiB unit that is then written, so a
|
|
non-MiB-aligned `MemoryBytes` converges in one pass instead of re-issuing SetConfig
|
|
forever (the numeric cousin of the description-newline normalization). Test proves
|
|
convergence. Slice 10 should still serve MiB-aligned specs at the source.
|
|
|
|
### Tests (the security proof — each independently rejected)
|
|
- **Adversarial matrix** via the REAL `authz.Verifier` with in-test-minted SSHSIGs
|
|
(framing replicated in reconcile's test binary; production authz untouched, no signing
|
|
added to the verify-only package): unsigned destructive **job** → pending_signature;
|
|
unsigned destructive **desired-state delta** → pending_signature (distrusts hub
|
|
desired state, not just jobs); forged/unknown signer → `ErrUnknownSigner`; expired →
|
|
`ErrExpired`; **replayed nonce across an agent restart** (durable `FileNonceStore`) →
|
|
`ErrReplay`; wrong host → `ErrTarget`; wrong guest / wrong op / wrong params →
|
|
binding_mismatch; **recovery key on ordinary destructive** → role_denied;
|
|
**hub-supplied "scratch" tag ignored** → still destructive → refused; **valid + role +
|
|
target + fresh nonce → accepted**, and a second presentation → `ErrReplay` (nonce
|
|
consumed).
|
|
- Classifier (benign/destructive/provenance/key-rotation/fail-safe), role-scoping,
|
|
params binding, crash-recovery (resume OK / fail / still-running / no-task rollback /
|
|
unreadable / one-shot key applied on resume), signed-job idempotency (execute once,
|
|
dedupe redelivery, refused-not-executed, no-executor-inert, executor-error).
|
|
- Full module **race-clean** (`go test -race`) + vet clean on the Linux build server.
|
|
|
|
## v0.4.0-rc1 — slice 4 Phase A: reconcile engine (structural; runs live, unfed) (2026-06-08)
|
|
|
|
The agent-side control core's structural half. **Checkpoint marker** — `-rc1` is the
|
|
Phase-A push; awaiting validation before Phase B (the reversibility gate + signed-op
|
|
consuming layer) lands the final **v0.4.0**. Runs LIVE but UNFED: with no desired-state
|
|
provider until slice 10, the live engine computes an empty action set and performs
|
|
**zero mutations**.
|
|
|
|
### Added
|
|
- **`internal/reconcile`** package — the engine, the per-guest serializer, the
|
|
desired-state model, the normalization layer, and the durable op journal:
|
|
- **Per-guest serializer (`Queue`, doc 03 §10)** — the single choke point ALL
|
|
mutation sources funnel through. Same-vmid jobs run strictly one-at-a-time in
|
|
submit order; independent vmids run in parallel. Each vmid is a cond-var FIFO lane
|
|
(unbounded, non-blocking, order-preserving); graceful drain on `Close`.
|
|
- **Desired-state model + `DesiredProvider` seam** — `DesiredGuest` (per-field
|
|
optional: run-state / `*hub.GuestSpec` / `*description`), `DesiredState`. The only
|
|
live provider is **`EmptyProvider`** (slice 4 has no source); `StaticProvider`
|
|
feeds fixtures. The seam is where slice 10's hub-serving plugs in — no hub/local
|
|
source invented here.
|
|
- **Normalization layer (`FieldNormalizers`)** — reconcile compares *normalized*
|
|
desired-vs-actual so Proxmox round-trip quirks don't read as drift. `description`'s
|
|
trailing newline is the first registered case; the registry takes more (boolean
|
|
coercion, list ordering) as discovered. `normDesc` **promoted** out of
|
|
`cmd/felhom-agent/main.go` to **`reconcile.NormDescription`**; the `--selftest=task`
|
|
description round-trip now uses that shared helper (one source of truth for the quirk).
|
|
- **Plan engine (`Plan`, pure function)** — computes the minimal **benign** action set
|
|
(`Start`/`Stop`/`SetConfig`) for guests present in both desired and actual, with
|
|
normalized comparison, deterministic vmid ordering, config-before-run-state. Skips
|
|
provision (desired-absent-in-actual, slice 7) and destroy (actual-absent-in-desired,
|
|
gated, slice 10); never writes a config it couldn't first read (`SpecKnown`). Disk
|
|
(rootfs grow) intentionally not reconciled here.
|
|
- **Reconcile engine (`Engine`)** — reads desired+actual, plans, dispatches each action
|
|
onto the shared queue. Every Proxmox op handled per the mutate.go contract: non-empty
|
|
UPID → `WaitTask` + assert `exitstatus`; empty UPID → clean **synchronous** success
|
|
(slice-4 proven). Per-action failures are counted, not fatal (other guests still
|
|
converge).
|
|
- **Operation journal (`Journal`)** — durable fsync'd append-only JSONL mirroring
|
|
`authz.FileNonceStore`: records each op's lifecycle (started → task_running →
|
|
succeeded/failed) with its Proxmox task id (crash mid-op is detected and re-checkable
|
|
on restart via `InFlight()`), plus an **idempotency-key store** (`AlreadyApplied`) so
|
|
a one-shot op never re-runs across retries/restarts. Reconcile actions carry no
|
|
idempotency key (convergent — must re-run on real drift).
|
|
- **Daemon wiring (`runDaemon`)** — reconcile runs alongside the hub loop on the poll
|
|
cadence, **sharing the per-guest queue**. Journal path is a `journal.log` sibling of the
|
|
nonce store. The daemon runs cleanly with **no desired state and no signers** (reconcile
|
|
is a logged live no-op; a journal-open failure degrades to journal-less, never crashes).
|
|
|
|
### Tests
|
|
- Serializer: same-guest serialized (max-concurrency 1, submit order preserved) and
|
|
different-guests parallel (cross-waiting jobs both complete — would deadlock if not);
|
|
error propagation; drain-pending-on-close; submit-after-close.
|
|
- Normalization: description round-trip; unknown-field identity; extensibility seam
|
|
(synthetic boolean-coercion + list-ordering normalizers).
|
|
- Plan: run-state start/stop, spec drift (cores/memory), disk-not-reconciled,
|
|
description-newline-not-drift, unmanaged fields, spec-unknown skips config keeps
|
|
run-state, desired-absent skipped, combined ordering, empty-desired no-op, deterministic
|
|
vmid order.
|
|
- Engine: empty-provider zero mutations; async start (WaitTask); synchronous SetConfig
|
|
(no WaitTask); WaitTask failure + POST error counted failed; list error = pass failure.
|
|
- Journal: lifecycle latest-wins; in-flight survives restart; idempotency dedupe across
|
|
restart; failed key not applied; torn-trailing-line skipped.
|
|
- Full module **race-clean** (`go test -race`) on the Linux build server; vet clean.
|
|
|
|
### Not in this phase (Phase B)
|
|
- The benign/destructive classifier, the reversibility gate, and the signed-op consuming
|
|
layer over `internal/authz` (doc 03 §4 / doc 04) — added next, in front of the queue's
|
|
executor, landing **v0.4.0**.
|
|
|
|
## v0.3.2 — SetConfig selftest extension (slice-4 pre-check) (2026-06-08)
|
|
|
|
The gate before slice 4: prove `SetConfig` works live under the scoped token before
|
|
reconcile is built on it. **Self-gated live run PASSED** on `demo-felhom`/guest 9999.
|
|
|
|
### Added
|
|
- **Reversible `SetConfig` step appended to `--selftest=task`** (`cmd/felhom-agent/main.go`,
|
|
`selftestSetConfig`): read `GuestConfig` → write a `description` marker
|
|
(`felhom-selftest <RFC3339>`) → verify it landed → restore the original value (or
|
|
`delete` the key if it was absent) → verify the restore. Handles PVE's dual-mode
|
|
`SetConfig` return per the `mutate.go` contract: empty UPID = synchronous success
|
|
(printed `synchronous`); non-empty UPID = `WaitTask` + assert `exitstatus=OK`.
|
|
The existing snapshot → rollback → delete-snapshot steps are unchanged. First live
|
|
exercise of the **`VM.Config.*`** privilege cluster.
|
|
- **`normDesc` / `extraString` helpers** — `extraString` decodes a string-valued key
|
|
from `GuestConfig.Extra` (raw JSON); `normDesc` strips the trailing newline PVE
|
|
appends to `description` on read, so a written value round-trips equal.
|
|
|
|
### Finding (live)
|
|
- The LXC `description` write returned **synchronous (empty UPID)** — PVE applied it
|
|
inline, no task. The agent's dual-mode `SetConfig` modeling is correct: the
|
|
empty-string path is real and must not be treated as an error.
|
|
- PVE **appends a trailing `\n` to `description`** on read (stored URL-encoded as
|
|
`%0A`). A naive exact-match reconcile would see perpetual drift — slice-4 reconcile
|
|
must normalize `description` comparisons (hence `normDesc`).
|
|
|
|
### Ops
|
|
- Standing operator token (`felhom-agent@pve!agent`, privsep) **rotated** during this
|
|
run (the prior secret was not retrievable); role + both user/token ACL rows
|
|
re-confirmed at `/`. New secret stored out-of-band, **not persisted to the repo**.
|
|
Guest 9999 left pristine (stopped, no `description`, no leftover snapshot). Version → 0.3.2.
|
|
|
|
## Docs + live validation — no version bump (2026-06-08)
|
|
|
|
### Changed
|
|
- **Reflowed `CLAUDE.md`** — removed hard mid-paragraph line wraps (prose, list items, blockquotes now single-line, soft-wrapped); code blocks and tables untouched; rendered output unchanged.
|
|
- **Unified the REPORT/CHANGELOG convention** in `CLAUDE.md`: `CHANGELOG.md` is the cumulative log (newest on top); `REPORT.md` is overwritten with the most-recent implementation/validation only. Added an explicit **no-secrets** rule (never write tokens/passwords/keys into committed files; reference them as stored out-of-band).
|
|
|
|
### Added
|
|
- **`REPORT.md`** rewritten for the live `--selftest=task` validation on the demo host (`demo-felhom`): snapshot → rollback → delete-snapshot on guest 9999, each polled to `exitstatus=OK` under the `felhom-agent@pve!agent` privsep token (UPIDs name the token actor — privsep path genuinely exercised); 16-privilege `FelhomAgent` role + both user & token ACLs confirmed; `--selftest=read` clean. Closes the slice-1 "mutating ops unit-tested only" gap; `WaitTask` async foundation validated live → **slice 4 unblocked**. (Token secret stored out-of-band, not in the repo.)
|
|
|
|
## v0.3.1 — slice-3 validation follow-ups (2026-06-08)
|
|
|
|
### Changed
|
|
- **Collector keeps the known run-status on a `GuestConfig` failure** (`internal/hub/collect.go`):
|
|
previously a per-guest config-read error forced `status="unknown"`; now the run-status from
|
|
`ListLXC` is preserved (only the `spec` is dropped). An empty status is still normalized to
|
|
`unknown` (wire value is always `running|stopped|unknown`). Test renamed to
|
|
`TestCollect_GuestConfigFailureKeepsStatusOmitsSpec` and asserts the preserved `running` + nil spec.
|
|
- **`--selftest` usage** error string now reads `(want read|task|hub)`.
|
|
|
|
### Added
|
|
- **Cross-repo contract fixture** `internal/hub/testdata/host-report.golden.json` +
|
|
`TestHostReport_ContractMatchesGolden` — compares the marshaled `HostReport` field-name sets
|
|
(top level + `host` + `guests[0]`) against the golden, failing on any json-tag drift. The file is
|
|
**kept byte-identical** with felhom-hub's copy (duplicated contract until a shared types module;
|
|
revisit when slices 5/6 populate the empty collections). Version → 0.3.1.
|
|
|
|
## v0.3.0 — hub client + host-report + first daemon loop (slice 3) (2026-06-08)
|
|
|
|
The agent's first daemon: a periodic read-only host-report POSTed to the hub (the
|
|
heartbeat). No Proxmox mutations, no desired-state/signed-op consumption, no
|
|
storage/backup collection yet — those are slices 4/5/6.
|
|
|
|
### Added
|
|
- **`internal/hub`** package:
|
|
- **`HostReport`** wire contract (`report.go`) shared field-for-field with the hub
|
|
ingest: host metrics, guests (`vmid` + spec), `cloudflared` status, and the
|
|
`storage_targets`/`backups`/`restore_tests`/`pbs_snapshots`/`audit_tail`
|
|
collections **defined but emitted empty** (typed `[]`, slices 5/6 fill them).
|
|
- **`Collector`** (`collect.go`) builds the report from a read-only `proxmoxReader`
|
|
(adapted to the real `internal/proxmox` surface — node held by the client, value
|
|
returns, `proxmox.Guest`) + a `CloudflaredProber`. Partial-failure policy: a
|
|
failed `NodeStatus` is a hard error (skip the POST); a failed per-guest
|
|
`GuestConfig` degrades that guest to `status="unknown"` (spec omitted) but still
|
|
sends; a cloudflared probe failure → `"unknown"`, never fatal.
|
|
- **`CloudflaredProber`** + `SystemctlProber` (`systemctl is-active cloudflared`;
|
|
read-only — NOT a Privileged/root op; tunnel management is a later slice).
|
|
- **`Client`** (`client.go`): `POST /api/v1/host-report` with
|
|
`Authorization: Bearer <key>`, standard TLS (system roots or optional `ca_file`;
|
|
verification always on). Typed `*TransportError` / `*HTTPError`; the bearer token
|
|
never appears in any error.
|
|
- **`Loop`** (`loop.go`): the daemon — immediate first report then tick; adopts the
|
|
hub's `poll_interval_seconds` clamped to [60,3600]; resilient (a collect/report
|
|
error is logged and the loop continues); clean shutdown on context cancel.
|
|
- **`ControlEnvelope`**: only `poll_interval_seconds` is acted on; `blocked` /
|
|
`desired_generation` / `has_signed_ops` are parsed-but-ignored (logged at most)
|
|
pending reconcile (slice 4).
|
|
- **Config**: `HubConfig` (url/host_id/api_key/poll_seconds/timeout_seconds/ca_file),
|
|
`FELHOM_AGENT_HUB_*` env overlay, `HubConfig.Validate()` (mode-aware — proxmox-only
|
|
`--selftest=read|task` still runs without hub config), `WithDefaults()`, and
|
|
`Redacted()` now also blanks the hub key. `configs/agent.example.json` gains `hub`
|
|
(and `authz`) blocks.
|
|
- **`cmd/felhom-agent`**: the no-`--selftest` mode is now the **daemon** (poll loop);
|
|
added **`--selftest=hub`** (one collect+report, prints the report + envelope).
|
|
Version 0.2.0 → 0.3.0.
|
|
|
|
### Tests
|
|
- Report serialization (field names; empty collections are `[]` not `null`; spec
|
|
omitted when unknown); client (Bearer header, non-2xx→`*HTTPError`,
|
|
transport→`*TransportError`, **token never in error**); collector (host mapping,
|
|
guest spec, per-guest failure degrades-but-still-reports, NodeStatus hard error,
|
|
cloudflared error→unknown); loop (immediate first report, continuation after an
|
|
injected error, interval adoption + clamp); config (hub validate/redact/env).
|
|
|
|
### Notes
|
|
- `internal/proxmox` and `internal/authz` were **not touched** — no new proxmox
|
|
surface was needed (`ListLXC` already exposes status/maxmem/maxdisk; `GuestConfig`
|
|
exposes cores). The task's `proxmoxReader` sketch (node-arg/pointer/`LXC`) was
|
|
adapted to the real exports as instructed.
|
|
- **Defined-but-empty** this slice: `storage_targets`, `backups`, `restore_tests`,
|
|
`pbs_snapshots`, `audit_tail` (slices 5/6). **Parsed-but-ignored**: the envelope's
|
|
`blocked`/`desired_generation`/`has_signed_ops` (slice 4).
|
|
|
|
## v0.2.0 — `authz` signed-op verifier (slice 2) (2026-06-08)
|
|
|
|
Production form of the Phase-4 signing primitive: a key-type-agnostic SSHSIG
|
|
verifier for operator-signed destructive ops, with the full anti-replay/
|
|
authorization pipeline and a durable, crash-safe nonce store. What slice 4
|
|
(reconcile) will call to gate destructive desired-state deltas. No hub, no signing
|
|
CLI, no reconcile loop.
|
|
|
|
### Added
|
|
- **`internal/authz` — `Verifier`**: `New(signers, store, hostID)` + `Verify(blob,
|
|
sigArmored) (*VerifiedOp, error)`. Runs the LOCKED pipeline (order is
|
|
load-bearing): parse armor → namespace → parse pubkey → allow-list (by key
|
|
**material**, `pub.Marshal()` equality, not key_id) → crypto verify (over the
|
|
**raw received bytes**, never re-canonicalized) → parse blob → target → time
|
|
window → **nonce recorded LAST**. Each post-crypto stage rejects even with a
|
|
valid signature.
|
|
- **SSHSIG framing** (`sshsig.go`) via `golang.org/x/crypto/ssh` — `pem.Decode` →
|
|
strip 6-byte magic → `ssh.Unmarshal` → `ssh.ParsePublicKey` → recompute signed
|
|
data with the named hash → `pub.Verify` (dispatches on key algorithm). No
|
|
hand-rolled crypto. Key-type-agnostic: ed25519 / **sk-ssh-ed25519 (FIDO2)** /
|
|
rsa / ecdsa via the one path.
|
|
- **Fixed namespace** `felhom-op-v1` (package constant, never caller-supplied).
|
|
- **`OpBlob`** (corrected `host_id`/`guest_id` json tags) + **`VerifiedOp`** (op,
|
|
host/guest, params, key_id, matched signer). key_id is advisory/audit only —
|
|
never an authz input.
|
|
- **Typed errors**: `ErrMalformed, ErrNamespace, ErrUnknownSigner, ErrBadSignature,
|
|
ErrTarget, ErrExpired, ErrNotYetValid, ErrReplay` (errors.Is-friendly).
|
|
- **`NonceStore`** + two impls: `MemoryNonceStore` (tests) and **`FileNonceStore`**
|
|
— durable, crash-safe (fsync'd append log, replayed into an index on open,
|
|
periodic compaction, expiry-only pruning). A nonce is fsync'd to disk before
|
|
`SeenOrRecord` returns false; replay protection survives restart; I/O failure
|
|
fails safe (reports seen=true). Target generalization: host_id matched strictly,
|
|
guest_id surfaced for the caller to route.
|
|
- **Config**: `AuthzConfig` (nonce-store path + pinned operator `signers` tagged
|
|
`operational`/`recovery` with a key_id, as authorized_keys lines).
|
|
- **Version 0.2.0.**
|
|
|
|
### Tests
|
|
- Real OpenSSH interop via a committed `ssh-keygen -Y sign` vector (hermetic CI);
|
|
per-stage rejection (each with an otherwise-valid sig); the headline
|
|
**invalid-sig-does-not-burn-the-nonce** invariant; replay; **persistence across
|
|
restart**; synthetic **sk-ssh-ed25519** through the unchanged path; byte-exactness
|
|
(a re-serialized blob fails crypto — not re-canonicalized).
|
|
|
|
### Notes / corrections to the Phase-4 reference
|
|
- §7's `Target` lacked json tags (`host_id`/`guest_id`) — fixed.
|
|
- The doc paired "Go 1.24.4 / x/crypto v0.52.0", but v0.52.0 declares `go 1.25.0`
|
|
and does **not** build on Go 1.24. Resolved by upgrading the build server to
|
|
go1.26.0 (backward-compatible; felhom-controller/hub unaffected); the module is
|
|
`go 1.25.0` on x/crypto v0.52.0.
|
|
- Free function → constructed `Verifier`; returns the full `VerifiedOp`; typed
|
|
errors; clock-skew tolerance added; durable nonce store is the net-new work.
|
|
- **Shared-contract dependency flagged** (not built): the hub and the `felhom-sign`
|
|
CLI must emit byte-identical canonical JSON or signatures won't verify; a shared
|
|
canonicalizer both import would be the right home.
|
|
|
|
## v0.1.0 — Scaffold + `proxmox` interaction layer (slice 1) (2026-06-08)
|
|
|
|
First slice: stand up the host-agent project and its foundation — the typed
|
|
Proxmox interaction layer every other module will call. No reconcile loop, hub
|
|
client, signing, or storage/backup orchestration yet (later slices).
|
|
|
|
### Added
|
|
- **Project scaffold**: module `gitea.dooplex.hu/admin/felhom-agent`, binary
|
|
`felhom-agent` (`cmd/felhom-agent/`), Go 1.24, zero external dependencies
|
|
(pure stdlib). `--version` flag; `version` var overridable via
|
|
`-ldflags "-X main.version=<v>"`.
|
|
- **`internal/proxmox` — API backend (`Client`)**: hand-rolled REST client over
|
|
`https://<host>:8006/api2/json` with `PVEAPIToken` auth. Typed read ops
|
|
(`Version`, `Nodes`, `NodeStatus`, `ListLXC`, `GuestStatus`, `GuestConfig`,
|
|
`ListStorage`, `NodeStorage`, `StorageContent`) and async mutating ops
|
|
returning a UPID (`RestoreLXC` — the primary create path, `Vzdump`, `Snapshot`,
|
|
`Rollback`, `DeleteSnapshot`, `SetConfig`, `Start`, `Stop`).
|
|
- **`WaitTask`**: polls `GET /nodes/{node}/tasks/{upid}/status` until stopped, then
|
|
asserts `exitstatus == "OK"` (authorization can surface at task execution, not
|
|
the POST — phase1-2 §1.3). Exponential backoff (1s→5s cap), context
|
|
cancellation + timeout. `*APIError` parses the offending privilege from a 403;
|
|
`*TaskError` parses it from a failed task exitstatus + log tail.
|
|
- **`internal/proxmox` — fenced root-CLI backend (`Privileged`)**: limited to the
|
|
three proven OS-root exceptions only — `CreateGoldenLXC` (keyctl `pct create`),
|
|
`MountUSBByUUID`, `SMART`, `Sensors`; each cites why it can't be the API. Fence
|
|
is structural (Client never shells out, Privileged never makes an HTTP call) and
|
|
asserted in tests.
|
|
- **TLS trust**: SHA-256 leaf-cert pinning (the host serves a self-signed cert) or
|
|
a CA file; an explicitly-named `insecure_skip_verify` that is off by default. No
|
|
blanket verification disable.
|
|
- **`internal/config`**: JSON config file + `FELHOM_AGENT_*` env overrides; the
|
|
token secret is never logged (`Redacted()`).
|
|
- **`internal/log`**: slog setup (text, stderr, configurable level).
|
|
- **`cmd/felhom-agent --selftest`**: read-only health report against a live host
|
|
(version/nodes/status/guests/storage); `--selftest=task --vmid N` exercises
|
|
`WaitTask` on a reversible snapshot→rollback→delete op (gated; default selftest
|
|
mutates nothing).
|
|
- **Tests**: unit tests with a mock HTTP transport + mock runner (UPID parse,
|
|
`WaitTask` running→OK / failed-403 / timeout / ctx-cancel, 403→privilege error,
|
|
response decoding against shapes captured live from `demo-felhom`, config
|
|
redaction, and the API-vs-root routing fence).
|
|
|
|
### Notes
|
|
- Types are grounded in the spike findings
|
|
(`felhom.eu/documentation/proxmox-platform.md`, `tests/phase{0,1-2,3}-findings.md`)
|
|
and the exact JSON shapes captured live from `demo-felhom` (PVE 9.2.2).
|
|
- Verified: `go build/vet/test` green on Go 1.24.4 (build server) and a live
|
|
read-only `--selftest` against the demo host with TLS fingerprint pinning.
|
|
- The 16-privilege `FelhomAgent` role + privsep token (role on **both** user and
|
|
token) is provisioned out-of-band; the agent only consumes the token.
|