Commit Graph

46 Commits

Author SHA1 Message Date
admin f43697c881 v0.32.0: self-serve decommission endpoint + intent-aware re-assert (B2a)
POST /disks/decommission mirrors eject (withGuest, user-data role gate) — no
operator signature, non-destructive (never formats): sets IntentDecommissioned,
prunes the GuestBindStore entry, unmounts. ReassertGuestBinds is now intent-aware
(skip non-enrolled) so a decommissioned-but-present drive never auto-rebinds on
agent restart — the load-bearing F9-reconnect fix. GuestBindStore.Remove added.
Operator-signed DecommissionExecutor + classify untouched. Non-hollow tests incl.
the intent-aware reassert companion (mutation-proven to fail on intent-blind code).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 19:55:17 +02:00
admin 4777f8a221 F20-BUG3: run mkfs detached (survives request deadline + agent restart); v0.31.0
The format ran mkfs under the HTTP request context, so the controller's 15s client
timeout cancelled it → SIGKILL mid-write → corrupt disk. Now mkfs runs DETACHED off
s.baseCtx (a dropped request can't kill it) via a persisted formatJob record; the handler
still waits to return the synchronous result (backward-compatible with the v0.62.0
controller) but abandoning the wait on client-disconnect leaves the mkfs running to
completion. New GET /disks/format/status surfaces the job (additive). RecoverFormatJob
runs on agent startup: a record left 'running' (agent died mid-format) is re-resolved by
durable-id (anti-retarget — absent/swapped disk NOT re-formatted) and the mkfs re-run; a
blank/path-bound interrupted format is marked failed (retry), never auto-re-run.

Tests: detached run persists running→done + binds durable-id; status endpoint; recovery
re-runs an interrupted durable-id-bound format; skips blank; skips unresolvable durable-id.
Version 0.30.0 → 0.31.0.
2026-06-14 15:16:01 +02:00
admin 4cd1d024e9 F9: auto-re-assert enrolled guest data-drive binds on agent startup
The in-guest bind (pct set -mpN) is config state that a destroy+re-provision drops, and
nothing restored it — so a re-provisioned guest came up with its enrolled HDD unattached
(the live-drive F9 finding). New GuestBindStore persists, per guest, the durable-ids of
enrolled drives (recorded at guest-attach); ReassertGuestBinds runs on agent startup (the
host's bring-up/reconcile trigger) and re-adds any bind a guest is MISSING — but ONLY when
the durable-id still resolves to a present, mounted drive (a swapped/absent drive is never
auto-bound) and the guest lacks it (idempotent). The re-added bind activates on the guest's
next reboot, like the enroll flow. Wired in main.go (store opened beside drive-intents.json;
ReassertGuestBinds called before the local API serves).

Tests: restores a missing bind with no manual call (the operator's real-trigger proof);
skips absent/swapped durable-id; no-op when already bound; store survives reopen (restart).
2026-06-14 15:07:37 +02:00
admin a2a76e7624 F20-BUG2 + F9-reporting: /disks surfaces wipe_durable_id (gate scheme) + guest_attached
F20-BUG2: the /disks list only carried DurableID in the uuid: scheme (for /disks/assign),
but the wipe gate resolves devices in the byid:/byuuid: scheme — so a customer confirming a
wipe with the advertised id was refused (binding_mismatch). Added a shared s.deviceDurableID
seam used by BOTH handleDisks (new DiskInfo.WipeDurableID) and the format gate, so the id the
customer copies from the list is exactly the id the gate accepts. DurableID (uuid:) is unchanged
(still feeds assign).

F9 (reporting half): added DiskInfo.GuestAttached — whether the drive's namespace is actually
bound into THIS guest's config (guestBoundPaths), distinct from mere host presence (State). This
is the signal whose absence made the HDD look available when it wasn't attached, and resolves the
F2 hdd_configured-vs-/disks disagreement.

Tests: wipe_durable_id is the gate scheme + distinct from uuid:; the list's wipe id matches the
gate's device-id binding (no mismatch); guest_attached true iff bound into the guest.
2026-06-14 15:00:56 +02:00
admin d96e5bddd0 fix(AGENT-001): anti-retarget re-resolution for inline customer-confirmed wipe
handleDiskFormat's customer-confirmed branch formatted the mutable req.Device
path; the durable id only bound the confirmation, never the mkfs target. A /dev
reassignment between inspect and mkfs could wipe the wrong physical disk.

Now mirrors signedjobs.WipeExecutor: resolve confirmed durable id -> current
device, re-derive + require exact match, re-inspect (still data-bearing), then
format THAT device. Any refusal -> 409, no mkfs. New antiRetargetResolve helper
(injected deps, unit-tested: mismatch/gone/blank/empty all refuse). Injectable
reresolveWipe seam on Server (defaults to real storage funcs).

BRANCH ONLY — pending supervised review/deploy (see AGENT-001-FIX-NOTES.md).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 19:26:27 +02:00
admin 77fa5af592 v0.29.1: lanresolver restarts dnsmasq on change (SIGHUP doesn't re-read config)
After a guest DHCP IP move, the split-horizon resolver kept serving the old IP:
the drop-in (address=/domain/ip) updated but 'systemctl reload dnsmasq' (SIGHUP)
does NOT re-read /etc/dnsmasq.d config — only /etc/hosts + cache. Changed
reload() -> restartDnsmasq() so address= changes actually take effect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 16:56:39 +02:00
admin d7d68fdd83 v0.29.0: OS/Docker-data storage split — golden + provision (Phase 1)
build-golden.sh bakes a small OS rootfs + a dedicated /var/lib/docker volume
(mp0, backup=1) carrying the baked images, plus Docker log rotation. bringup.go
grows the golden's data volume to the per-customer target (DataVolGrowGB) and
emits backup=1 on data mounts (GuestMount.Backup) — closing the spike-B3 silent
DB-loss trap. CLI gains -rootfs-grow/-datavol-grow/-datavol-mount. New
RUNBOOK-provisioning-storage.md. Phase 2 = felhom-controller v0.58.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 15:38:26 +02:00
admin b66c137aa7 restore-test: add rootfs override alongside bind-mount overrides
PVE refuses a restore carrying mountpoint params unless rootfs is also set ("mount
points configured, but 'rootfs' not set"). Size the rootfs override from the source
rootfs (rootfsSizeGB/sizeToGB, round up). Validated manually on the host: restore of
bind-mounted 9201 with --rootfs + --mp0/--mp9 overrides boots + tears down cleanly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 20:54:46 +02:00
admin 7fb7ef5c3d restore-test: neutralize source bind-mount mountpoints for scratch restore
A slice-10 enrolled guest's data drive is a host bind-mount mp0 that the privsep
token can't vzrestore ("bind mount is only possible for root") — so the restore-test
failed for every enrolled guest regardless of backup tier. The restore-test now reads
the source guest config (vmid from the archive volid) and passes RestoreLXC mp
overrides converting each bind-mount mpN to a throwaway 1G volume on the restore
storage (no root needed; boot-verify doesn't need the data). proxmox.RestoreLXC gains
MountOverrides. + unit tests (archiveVMID, bindMountOverrides).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 20:50:35 +02:00
admin 109dd853a3 v0.28.0: backup re-target → felhom-pbs (offsite DR) + operator-signed decommission
- BackupConfig.BackupTarget() defaults whole-guest backup to felhom-pbs (separate
  hardware = real DR), configurable via backup.local_backup_target; all NewBackupRunner
  sites route through it. PBS round-trip proven live (snapshot marker + restore-test +
  pct-restore) before the re-point.
- signedjobs DecommissionExecutor + ExecutorChain: makes IntentDecommissioned reachable
  ONLY via a verified operator signature (keyed by the watchdog's storage durable-id);
  felhom-opsign builds decommission params from -durable-id. Runner wiring moved below
  the intent-store open.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 20:26:34 +02:00
admin 9580c8c75d fix(P3): record enrolled intent on the idempotent guest-attach path too 2026-06-12 17:51:52 +02:00
admin 237b85f420 agent v0.27.0: slice 10 P3 — self-heal watchdog reconcile + 4-state intent model
IntentStore (durable-id-keyed: new/enrolled/ejected/decommissioned, OnAbsent
replug rule). Watchdog re-mounts only enrolled drives (out-of-band unmount heals;
ejected/decommissioned/new left alone) + exp-backoff flapping guard (alert@4,
cap@8). guest-attach records enrolled; eject records ejected. Non-hollow tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 17:49:25 +02:00
admin bb1692cbfb agent v0.26.0: slice 10 P2 activation — POST /guest/reboot (user-triggered)
Self-scoped guest reboot (pct reboot, detached, 202) so an enrolled-into-running-
guest drive's persisted bind activates at next boot. Tests: accepted + cross-guest
403. Pairs with controller v0.49.0 pending-drive detection + "Újraindítás most".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 17:19:12 +02:00
admin c1d04c28c1 agent v0.25.0: slice 10 P2 — bind enrolled user-data drives into the guest
POST /disks/guest-attach binds an enrolled drive's felhom-data namespace into
the guest (Model A: felhom-data is the bind source mounted at /mnt/<name>, so
only Felhom's namespace crosses in). GuestBinder does mkdir+chown(100000)+pct set
(RW bind) via the fenced runner. Idempotent, free-slot selection, path-validated.
Spike-proven on 9201. Pairs with controller P2C + golden /mnt:rslave (P2B).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 15:38:55 +02:00
admin 7ae82e1d5d agent v0.24.0: role-gate the eject path (system/backup mounts unmount-protected at the agent)
handleDiskEject now resolves the authoritative role of the storage at `where`
and refuses 403 (no Unmount) unless it is user-data. Fails safe to protected on
ambiguity. Adds roleForMountPath + an injectable HostReader seam for testability.
TestEject_RoleGated asserts protected mounts are refused with no Unmount.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 09:29:09 +02:00
admin 832b73e6e8 agent v0.23.0: expose per-drive capacity (total/used/fraction) in GET /disks for the controller capacity bar
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 21:30:38 +02:00
admin 15f7529a1c v0.23.0: device-ROLE classification + tiered storage-wipe gate (user-data customer-confirmable; system/backup operator-only)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 21:29:18 +02:00
admin 4734d4a132 v0.22.0: expose durable_id in GET /disks (enables controller guided storage)
localapi.DiskInfo gains durable_id (from StorageTarget.DurableID, "uuid:<fs-uuid>"
for usb/local-dir). The de-privileged controller can't read a device's fs UUID
but assign mounts strictly by UUID — this read-only field is the only way it
learns the assign key. No new privilege, no gate change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 19:46:50 +02:00
admin a43e9813ad v0.21.0: agent-managed split-horizon LAN resolver (internal/lanresolver)
Host-side dnsmasq the agent manages so LAN clients reach their guest directly
(same hostname + real wildcard cert, no Cloudflare hairpin). Renders local=/
+address=/ per customer (AAAA->NODATA via authoritative zone, wildcard A ->
live guest IP), forwards everything else. Manager ensures dnsmasq+base config,
discovers guest IP (pct exec ip) + domain (controller.yaml), write-if-changed +
reload. Loop (7th daemon goroutine) tracks DHCP IP changes per provisioned
guest. --selftest=lanresolver. FELHOM_DNSMASQ sudoers. Spiked live on felhom-pve.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 18:24:58 +02:00
admin e5a18194f4 v0.19.0: bootstrap contract v2 — relay hub retrieval passphrase (no host key in guest)
Lockstep with felhom-controller v0.40.0. The agent now bakes a v2 bootstrap.json
carrying only what the controller needs to PULL its config from the hub:
customer.id + hub.url + hub.retrieval_password + the per-guest local_api. Stops
baking the agent's host hub key/host_id (and customer name/domain/email) into the
guest — the controller gets the customer-scoped key from the hub pull.

- internal/provision/doc.go: SchemaV2; DocCustomer{id}; DocHub{url,retrieval_password}.
- backhalf.go: render v2; require customer.id + hub.url + hub.retrieval_password.
- cmd/.../main.go --selftest=provision: new required -hub-password flag; stop
  baking APIKey/HostID; -customer-domain/name/email accepted but not baked.
- configs/build-golden.sh: default CONTROLLER_IMAGE off stale :v0.35.0 -> :0.40.0.
- doc_test.go/backhalf_test.go updated to v2 shape.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 13:22:51 +02:00
admin e4dfe5ccc7 slice 10D (agent): DR capstone — identity escrow + restore-mode consumption (v0.18.0)
Identity escrow wraps {tunnel_token,pbs_token} under the SAME R via age
(scrypt+ChaCha20-Poly1305), reusing the K-escrow pty; wrong R fails closed.
escrow.Create optionally emits the identity blob; escrow-create uploads it +
the non-secret directive; identity-consume recovers it (R by hand, never
logged). K-escrow + 10C Consume untouched. Closes slice 10 with hub v0.11.0;
operator-side rotation model (hub holds no Cloudflare write-power).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 09:48:36 +02:00
admin 89e9f98a95 slice 10C: escrow consumption — productionize the spike (v0.17.0)
Add escrow.Consume(blob, R, expectedFingerprint, keyDest): Unwrap -> fingerprint
gate -> atomic 0600 install. Bakes in the spike findings — wrong R fails closed
(no write), the fingerprint gate runs BEFORE any restore (no install on
mismatch), the input blob is read-only (retryable), K is never mutated, R/key
bytes never logged. Zero-knowledge holds: the hub serves all but R (by hand).
--selftest=escrow-consume invokes the real path live. Agent-only; no hub change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 22:17:56 +02:00
admin 588fed2aa9 slice 10B: operator-signed destructive completion (offline key + signing CLI) (v0.16.0)
A destructive op runs ONLY on a pinned-key-verified, nonce-fresh, in-window,
host-bound, durable-id-bound operator signature. New cmd/felhom-opsign signs
canonical OpBlobs offline via ssh-keygen -Y sign (hardware-ready); the signing
key is never in the hub or agent. New internal/signedjobs runner verifies each
queued blob through the gate and only on all-pass runs the WipeExecutor, which
re-resolves the DURABLE device id + re-inspects (8C) before mkfs — closing the
8C data-bearing-wipe pending_signature gap. New storage durable-device
resolution; authz.CanonicalBlob promoted to production. Real-crypto tests assert
valid executes and forged/replay/expired/retarget/non-pinned are rejected
(executor never called).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 20:14:16 +02:00
admin 8ecf8929fb slice 10A: activate the control envelope (Down channel) + hub-backed desired provider (v0.15.0)
The control envelope becomes live: the agent caches the hub's desired-state +
generation and re-fetches GET /hosts/{id}/desired-state only when the
generation advances. A new internal/desired Syncer maps the wire shape into a
reconcile.CachingProvider feeding the engine; benign deltas reconcile, an
explicit guest decommission is gated pending_signature (exec is 10B). Adds the
DesiredStateResponse/WireDesiredState wire types + Client.FetchDesiredState +
the loop EnvelopeObserver seam. Cross-repo golden (envelope + desired-state)
byte-identical with the hub.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 19:02:59 +02:00
admin aa4dfb75ea slice 9: GET /host/metrics + CPU/chassis-temp collector (v0.14.0)
Add a host-wide, token-authed GET /host/metrics local-API endpoint that
re-serves the slice-4 collector's host + per-storage view to the customer
(the de-privileged controller can't read the host itself). Add the one new
collector — CPU/chassis temperature via sysfs hwmon/thermal-zones, graceful-
null — to the shared HostMetrics struct, so the hub report carries cpu_temp_c
too. Cross-repo host-report golden updated byte-identical.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 16:16:03 +02:00
admin 570410cd1a slice 8B.2 (agent): emit snapshotted phase at storage-snapshot moment (v0.13.0)
BackupRunner.BackupWithSnapshotHook tails the task log for the 'create storage
snapshot' marker (snapshot mode only) and fires onSnapshot once; localapi flips
/backup/status to 'snapshotted' before 'done' so the controller resumes early.
Phase 0 validated on PVE 9.2.2: marker confirmed, downtime ~24s->~1s (934MB).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 14:54:18 +02:00
admin c17cfde236 slice 8C Phase A: agent disk endpoints + data-bearing classifier gate + mkfs (v0.12.0)
internal/storage: mkfs executor (Format, device-pinned, narrow FELHOM_FORMAT
sudoers) + data-bearing device inspection (InspectDevice/DeviceProbe via
blkid+lsblk; conservative — ambiguous=data-bearing). internal/localapi: /disks
(+ data-bearing flag), /disks/assign (EnsureMount), /disks/eject (Unmount +
dependent guests), /disks/format. SECURITY CENTERPIECE: the agent inspects the
device itself; data-bearing format -> ClassStorageWipe gate -> pending_signature
refused; the caller's claim is never trusted. Additive (no controller change yet).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 12:52:22 +02:00
admin 33dfd9afb3 slice 8B (agent half): /backup/due cadence policy + /backup/status phases (v0.11.0)
internal/localapi: real /backup/due (cadence; due when no successful backup or
newest older than backup.backup_cadence_seconds; false in-window after success;
failed doesn't count) + /backup/status phases (idle|running|done|failed + job
id) + POST /backup single-flight with job id. Drives the controller quiesce loop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 10:44:50 +02:00
admin 3fecf4c713 slice 8A (agent half): local-API server + provisioning back-half (v0.10.0)
internal/localapi: per-guest local-API server (doc 03 §6) — 7 self-scoped
endpoints, hashed per-guest token store, persisted self-signed leaf with stable
SHA-256 pin, optional 6th daemon goroutine. internal/provision: back-half —
mint token, render bootstrap.json (no registry cred), write 0600, chown
100000:100000, attach pct-set bind mount (host-side, F3, no pct exec).
--selftest=provision. build-golden.sh bakes the controller image + bootstrap
unit. sudoers FELHOM_PROVISION; firewall narrowing artifact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 09:47:42 +02:00
admin 47dd0bd244 slice 7: PBS recovery-code escrow creation (agent, Phase B) (v0.9.0 wip)
internal/escrow: zero-knowledge escrow creation. R = 10 EFF-wordlist words
(crypto/rand, ~129 bits); wrap K under R via PBS-native key change-passphrase
driven over a stdlib pty (x/sys/unix; output discarded so R can't leak, F-A2);
self-verify the blob recovers K (fingerprint match) before shipping. Opt-in (b)
R-wrapped offline copy + (a) raw paperkey. Live K is byte-unchanged (operates on
a copy). --selftest=escrow-create (-storage/-paperkey/-offline/-upload). Posture
config field (zero_knowledge default). PBSEncKeyPath helper. Grounded by the
escrow spike findings.

Tests: R entropy>=128/format/uniqueness; integration round-trip (wrap->unwrap
fingerprint match, wrong-R fails, K byte-unchanged) guarded to linux+pbc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 07:37:03 +02:00
admin 57405c1a99 slice 7 Phase 1: unified bring-up reconcile job (provision + guest-loss DR) (v0.8.0)
The shared front half of provision and guest-loss DR as a journaled reconcile job
(internal/reconcile/bringup.go), mirroring the restore-test's crash-safety but keeping
the guest on success and applying a scenario-specific identity policy. Agent-only; no
hub/wire change. Grounded by the slice-7 bring-up spike (commit 3342993): F1/F3/F4.

- RunBringUp: restore -> reset identity -> size -> attach mounts -> start link-up;
  verdict is liveness (waitRunning), success KEEPS the guest.
- identity policy: provision = fresh MAC (net0 sans hwaddr -> PVE regen) + hostname,
  host-side; machine-id/host-keys regenerate guest-side (systemd + baked golden unit).
  dr_guest_loss = preserve continuity (keep hostname; keep MAC unless KeepMAC=false).
- compensating rollback: mid-flight failure destroys the just-created guest
  (SameTxnCreated provenance, gated); new Rollback journal flag + Recover.recoverBringUp
  reap a half-built guest from a crash.
- F4: coalesced config PUT + bounded retry on the transient PVE config-lock 500 only.
- --selftest=bring-up (mode/archive/vmid/hostname/keep).
- configs/build-golden.sh: validated golden recipe incl. the F3 first-boot host-key unit.
- doc-03 §9 + identity-reset settled/implemented.

Deferred (stated): provisioning back half -> slice 8; host-loss DR + escrow consumption
and the BringUpSpec source (hub desired-state) -> slice 10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:27:49 +02:00
admin 9f6753de0f restore-test wire contract: warnings + warnings_recognized in host-report golden (Phase B)
Adds the two new RestoreTest wire keys to the shared host-report golden and exercises
them in the bidirectional key-set contract test. Kept byte-identical with felhom-hub's
copy (sha256 e6999d77...). Testdata/tests only — the v0.7.0 binary is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 19:41:08 +02:00
admin 6e86483185 restore-test: verdict is liveness, not start-task exitstatus (v0.7.0)
Fixes the crying-wolf false-fail surfaced by the live hub-enrollment runbook:
PVE's guest-start task exits "WARNINGS: 1" for the benign systemd-nesting
advisory, and WaitTask treated any non-OK exitstatus as failure, so the verdict
was decided by an advisory exit code before the real boot check ran. Every
modern-distro restore-test reported pass:false.

- proxmox.WaitOptions.AllowWarnings (opt-in; default keeps all callers strict)
- restore-test start step accepts warnings, surfaces them, verdict stays waitRunning
- RestoreTestResult.StartWarnings/.WarningsRecognized + version-free "enable
  nesting" recognizer (can't rot back at systemd 258+); GuestAPI.TaskLogTail
- hub.RestoreTest.warnings/.warnings_recognized wire fields (consumed by hub v0.7.5)
- scheduler logs clean / passed-with-recognized / passed-with-unrecognized warnings
- tests: WaitTask warnings matrix; restore-test pass/fail-on-liveness; version-free
  regression guard (systemd 256-300)

Single agent bump 0.6.0 -> 0.7.0 covering the agent half of both task phases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 19:30:03 +02:00
admin dcd8a8eff4 fix(pbs): Verify forces ignore-verified=false; restore-test source_tier from storage type
Live PBS runbook surfaced two gaps: (1) PBS verify defaults to ignore-verified=true and
SKIPS already-verified snapshots, so corruption after the first verify is never caught —
the agent's integrity check now POSTs ignore-verified=false to actually re-read+re-check.
(2) restore-test source_tier was hardcoded 'local'; now derived from the source storage
type ('pbs' for a PBS datastore). Adds a form-POST path to the PBS client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 17:00:21 +02:00
admin 766500dfc3 v0.6.0: slice 6 Phase B — PBS offsite tier (verify + PBS-API client + reporting)
Spike-proven that backup/restore-to-PBS reuse Phase A unchanged; the only new code is
the verify capability, a small PBS-API client, and PBSSnapshot reporting.

- internal/pbs: fingerprint-pinned, token-authed PBS-API client (Verify/Snapshots/
  TaskStatus, node-from-UPID; secret read from /etc/pve/priv/storage/<id>.pw at runtime,
  never logged) + the verify maintenance loop (own cadence, default 6h, NOT gated/journaled,
  like the watchdog) + SnapshotStore.
- hub: PBSSnapshot filled (namespace/type/id/time/size/owner/protected/encrypted/
  verify_state/verify_upid); PBSReporter collector seam; cross-repo golden + bidirectional
  key-set tests; hub handler parses pbs_snapshots + logs a failed-verify WARN.
- backup: report the ACTUAL vzdump mode (parsed from the task log; PVE may downgrade
  snapshot->stop). proxmox.Storage.Username. config PBSVerifyCadence/secret-dir.
  --selftest=pbs-verify. Backup/restore-to-PBS unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:53:04 +02:00
admin 280a7f80e5 fix(proxmox): DestroyLXC DELETE takes no body + force=1 (live restore-test teardown)
The slice-6 restore-test teardown failed live: DELETE /lxc/{vmid} with a form body
returns HTTP 501 'Unexpected content for method DELETE', and a booted scratch needs
force=1. Move purge/destroy-unreferenced-disks/force to the query string, send no body.
Regression test locks the contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 13:53:36 +02:00
admin b527430ec7 v0.6.0-rc1: slice 6 Phase A — backup + the self-restore-test (local target)
The guest-level backup layer + the journaled self-restore-test (restore→boot→verify→
teardown) that closes "a backup you haven't restored isn't a backup". All benign
(reuses the slice-4 classifier/gate/journal; no new destructive class/crypto). Local
target only; PBS = Phase B. Restore to a NEW guest only. Backups crash-consistent.

- proxmox: DestroyLXC, VzdumpOptions.Notes (notes-template), LatestBackupVolID.
- reconcile: Engine.RunRestoreTest (journal Scratch entry BEFORE mutation; net link-down
  pre-boot; defer teardown always; benign gated destroy) + Recover extended to reap a
  leaked scratch guest (Scratch flag, special-cased before the UPID path; idempotent).
- internal/backup: runner (vzdump + archive resolve + bulk-gap = backup!=1) + cadence
  scheduler (4th daemon goroutine, default 24h) + in-memory report store.
- hub: Backup/RestoreTest filled; collector seams; cross-repo golden byte-identical +
  bidirectional key-set tests; hub handler logs a FAILED restore-test prominently.
- config BackupConfig (band 990000-990009 default); --selftest=backup / restore-test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 13:49:39 +02:00
admin 77b4f21450 v0.5.1: live-validation prep — fix unmounted-dir durable_id mis-id + watchdog UUID memory
Surfaced preparing the live USB validation on demo-felhom:
- observe.go: an unmounted removable dir-storage no longer falls through to the ROOT fs
  for its backing device/UUID — durable_id was becoming uuid:<root-uuid> (a DR mis-id that
  would re-attach the wrong disk). Now derived only from the target's own mountpoint;
  unmounted → no device + stable store:<name> durable_id. Removed containingMountDevice.
- watchdog.go: remember the fs-UUID observed while attached and backfill it onto the
  re-mount target, so re-mount works even if the known-set cache refreshed mid-drop
  (doc 03 §7 "sourced from the existing definition").

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 11:01:40 +02:00
admin 9d6e49236c v0.5.0: slice 5 Phase B — the host-root surface (mounts + SMART + grow + destructive gate)
The privileged write surface, isolated behind a narrow, arg-validated, adversarially-
tested seam (HostOps), the same discipline as the slice-4 gate. Completes slice 5.

- internal/storage: HostOps seam + SudoHostOps (systemd .mount units by fs-UUID, detach,
  SMART, lvs) via sudoers allowlist + fixed arg vectors, no shell; NoopHostOps fallback.
- validate.go: strict UUID/mount-path/device/LVM validators + in-process systemd-escape.
  Headline test: adversarial matrix (metacharacters/traversal/malformed) refused with
  zero exec.
- smart.go: smartctl SATA + NVMe parse, UNKNOWN-degrade; lvs thin-pool metadata fill.
- observer enrichment (Observe only): fills smart + thin-pool metadata.
- watchdog: benign re-mount response off the poll path (DevicePresent probe, rate-limited).
- reconcile: ActionResize (benign, grow-only) + proxmox.ResizeLXC; destructive storage ops
  (ClassStorageWipe/Decommission) through the slice-4 gate, target-scoped; built+tested,
  inert live.
- --selftest=storage [-watch] live harness; configs/felhom-agent.sudoers; privileged.* knobs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 10:53:38 +02:00
admin 27b68f043b v0.5.0-rc1: slice 5 Phase A — storage observe/report + watchdog (read-only, live)
Fill the slice-3 storage_targets stub and add the fast-poll storage watchdog.
Read-only this phase; the host-root surface (mounts/SMART/grow/destructive gate)
is Phase B. Hub-owned desired manifest is slice 10, so reconcile against it is
built-but-unfed.

- internal/storage: StorageTarget wire contract, durable_id derivation per type,
  HostReader seam (procfs/sysfs, root-free), Observer (storage_targets from
  ListStorage/NodeStorage + host reads, lvmthin thin-pool fill), and the watchdog
  (third daemon goroutine; debounced out-of-band report on a known target's
  attach/disconnect transition).
- proxmox.Storage: additive parse-only config fields (durable_id sources).
- collector StorageObserver seam; Loop.SetTrigger out-of-band report; daemon runs
  the watchdog as a third goroutine; StorageConfig knobs.
- cross-repo golden kept byte-identical with felhom.eu/hub; bidirectional key-set test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 09:59:05 +02:00
admin 1af21a6cac v0.4.0: slice 4 Phase B — reversibility gate + signed-op consuming layer
The security core of slice 4: hub-supplied intent is no longer trusted for
destructive change. The gate fronts the per-guest queue's executor, so every
mutation passes it. Reuses internal/authz for all crypto (surface untouched).

- Classifier (doc 03 §4): benign vs destructive by provenance + data-bearing-
  ness, NOT by verb. Destroy/overwrite of customer data is destructive unless
  agent-internal provenance (same-journaled-txn create, or agent-tagged scratch)
  makes it benign — and that provenance is journal-recorded, NEVER hub-sourced.
  Unknown op class fails safe to destructive.
- Reversibility gate: benign -> allowed unsigned; destructive -> requires a
  verified, role-scoped, action-bound operator signature, else pending_signature
  and never executed. Every decision audited (signal, never the guard).
- Signed-op consuming layer over authz.Verifier.Verify (locked pipeline
  untouched): role-scoping (doc 04 §4 — recovery=rotation only, operational=
  ordinary destructive + planned rotation) + op-to-action binding (op+host+
  guest+params must match the gated action).
- Signed-job orchestration: idempotency dedupe by nonce + journal-wrapped
  execution via an injected DestructiveExecutor (nil this slice — inert).
- Crash recovery (Note 1): Engine.Recover consumes the journal InFlight() set at
  startup (resume-or-rollback) — covers an op that crashed after the POST and
  before its terminal record, which idempotency dedupe alone cannot. Added
  TaskStatusOnce to the GuestAPI seam. Wired into daemon startup.
- Note 2: memory comparison canonicalized to MiB (desiredMemoryMiB) so a
  non-MiB-aligned MemoryBytes converges in one pass, not perpetual drift.
- Daemon: builds the verifier from config signers (none = nil verifier, the
  common slice-4 state), the gate (+SlogAudit), runs Recover before mutating.

Adversarial matrix proven against the REAL authz.Verifier with in-test-minted
SSHSIGs (framing replicated in reconcile's test binary; authz untouched, no
signing added to the verify-only package): unsigned job + unsigned desired-state
delta -> pending_signature; unknown signer/expired/replay-across-restart/wrong
host -> typed authz rejections; wrong guest/op/params -> binding_mismatch;
recovery key on ordinary destructive -> role_denied; hub-supplied scratch tag
ignored -> refused; valid+role+target+fresh nonce -> accepted then replay
rejected. Full module race-clean + vet-clean on the Linux build server.

Inert this slice: no destructive deltas served until slice 10; the destructive
path is classified, gated, and tested but not wired to live execution.

CHECKPOINT: Phase B complete (slice 4 done). Awaiting validation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 23:56:20 +02:00
admin 05c450147c v0.4.0-rc1: slice 4 Phase A — reconcile engine (structural, runs live unfed)
New internal/reconcile package: the agent-side control core's structural half.

- Per-guest serializer Queue (doc 03 §10): the single choke point all mutation
  sources funnel through; same-vmid serial in submit order, different vmids
  parallel (cond-var FIFO lanes).
- Desired-state model + DesiredProvider seam; EmptyProvider is the only live
  source at slice 4 (no hub serving until slice 10) so the live engine computes
  an empty action set and performs zero mutations.
- Normalization layer (FieldNormalizers): normalized desired-vs-actual so
  Proxmox round-trip quirks don't read as drift. normDesc promoted out of
  main.go to reconcile.NormDescription; selftest uses the shared helper.
- Plan (pure diff): minimal benign action set (Start/Stop/SetConfig) for guests
  in both desired and actual; provision/destroy out of scope here.
- Engine: dispatches onto the shared queue; honors the dual-mode SetConfig
  contract (UPID -> WaitTask; empty UPID -> synchronous success).
- Durable op journal + idempotency store (mirrors authz.FileNonceStore):
  in-flight task ids for crash detection + AlreadyApplied dedupe across restart.
- Wired into runDaemon alongside the hub loop, sharing the queue; runs cleanly
  with no desired state and no signers.

Full module race-clean and vet-clean on the Linux build server.

CHECKPOINT: Phase A only. Awaiting validation before Phase B (the reversibility
gate + signed-op consuming layer, landing v0.4.0).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 23:21:55 +02:00
admin e68a7af4d3 fix(agent): slice-3 follow-ups — keep run-status on config fail, selftest usage, contract golden (v0.3.1)
- collect: a per-guest GuestConfig failure preserves the ListLXC run-status (only
  spec dropped); empty status normalized to "unknown". Test asserts preserved
  "running" + nil spec.
- main: --selftest usage error now reads (want read|task|hub).
- contract: testdata/host-report.golden.json + TestHostReport_ContractMatchesGolden
  (field-name key-set check vs golden; byte-identical with the hub copy).
- version 0.3.0 -> 0.3.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:29:05 +02:00
admin ab77fa3544 feat(hub): host-report client + collector + first daemon loop (slice 3, v0.3.0)
internal/hub: the agent's first daemon — a periodic read-only host-report POSTed to
the hub (the heartbeat; no separate ping).

- HostReport wire contract (shared field-for-field with the hub ingest): host
  metrics, guests (vmid + spec), cloudflared status; storage/backups/restore-tests/
  pbs/audit collections DEFINED but emitted empty (slices 5/6 fill).
- Collector over a read-only proxmoxReader (adapted to the real proxmox surface;
  no proxmox changes) + a CloudflaredProber. Partial-failure: NodeStatus fail = hard
  (skip POST); per-guest GuestConfig fail = status "unknown", still report.
- Client: Bearer-auth POST, standard TLS (system roots / optional ca_file), typed
  TransportError/HTTPError, token never in errors.
- Loop: immediate first report, adopt hub poll_interval (clamp [60,3600]), resilient
  to collect/report errors, clean ctx-cancel shutdown.
- ControlEnvelope: only poll_interval_seconds acted on; blocked/desired_generation/
  has_signed_ops parsed-but-ignored (slice 4).
- config: HubConfig + FELHOM_AGENT_HUB_* overlay + mode-aware HubConfig.Validate +
  WithDefaults + hub-key redaction; example config updated.
- main: no-selftest mode is now the daemon; added --selftest=hub. Version -> 0.3.0.

Tests: report serialization, client (incl. token-redaction), collector partial-
failure, loop continuation+interval adoption, config. internal/proxmox + internal/
authz untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 16:20:09 +02:00
admin f0fee7e193 feat(authz): operator signed-op verifier + durable nonce store (slice 2, v0.2.0)
internal/authz: production form of the Phase-4 SSHSIG signing primitive.

- Verifier.New/Verify with the LOCKED pipeline (namespace → allow-list by key
  material → crypto over RAW bytes → target → time → nonce LAST); each post-crypto
  stage rejects even with a valid sig; an invalid sig never burns a nonce.
- SSHSIG framing via x/crypto/ssh (no hand-rolled crypto); key-type-agnostic
  (ed25519 / sk-ssh-ed25519 / rsa / ecdsa via pub.Verify). Fixed namespace
  felhom-op-v1. Typed errors. OpBlob (fixed host_id/guest_id tags) + VerifiedOp.
- NonceStore: MemoryNonceStore + durable crash-safe FileNonceStore (fsync'd append
  log, replay-on-open, compaction, expiry-only pruning; survives restart).
- config.AuthzConfig (nonce path + pinned operational/recovery signer keys).
- Tests (14): real ssh-keygen fixture, per-stage rejection, nonce-not-burned,
  replay, persistence-across-restart, synthetic sk, byte-exactness.

Dep: golang.org/x/crypto v0.52.0 (declares go 1.25 — the Phase-4 doc's "Go 1.24.4 /
x/crypto v0.52.0" pairing doesn't build; build server upgraded to go1.26.0,
backward-compatible). Version 0.1.0 -> 0.2.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:23:02 +02:00
admin a042316d6d feat(agent): scaffold + proxmox interaction layer (slice 1)
Stand up the felhom-agent project (module gitea.dooplex.hu/admin/felhom-agent,
binary felhom-agent) and the internal/proxmox package: the typed library every
other agent module calls to talk to Proxmox.

- API-first Client (hand-rolled REST over net/http, PVEAPIToken auth) with typed
  read ops (version/nodes/status/lxc/config/storage) and async mutating ops
  (restore/vzdump/snapshot/rollback/delete-snapshot/setconfig/start/stop), each
  returning a UPID. WaitTask polls task status until stopped and asserts
  exitstatus OK (authz can surface at task exec, not the POST — phase1-2 §1.3).
- Fenced Privileged (root-CLI) backend for the THREE proven exceptions only
  (keyctl pct create, USB mount/fstab, SMART/sensors); each cites why it can't be
  the API. Fence is structural (Client never shells out, Privileged never HTTPs)
  and asserted in routing_test.go.
- TLS: SHA-256 leaf-cert pinning or CA file; insecure mode explicit + off by
  default. No blanket verification disable.
- 403 -> privilege-named APIError; failed task -> privilege-named TaskError.
- JSON config + env overrides (token never logged); slog logging.
- cmd/felhom-agent --selftest (read-only health report) + gated --selftest=task
  (reversible snapshot/rollback/delete exercise of WaitTask). No daemon loop yet.
- Types grounded in the spike findings and exact JSON shapes captured live from
  demo-felhom (PVE 9.2.2). Unit tests use a mock transport + runner.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 14:34:32 +02:00