The restore-test had GL-5 finding #2's mirror image: its live-source-config
bind-override path tripped PVE's drop-unlisted-mountpoints rule, so scratch
guests boot-verified WITHOUT their storage mpN - weaker verification than
claimed. Params now derive from the ARCHIVE's own embedded config via
ExtractArchiveConfig + drRestoreOverrides (the object under test; full
layout, content genuinely extracted - the added runtime IS the
verification); unreadable/unknown-topology archives refuse up front. NEW
mount-parity assert (2b, pre-start): restored mpN set vs the archive's -
missing/mispathed/undersized/extra mpN fail the test naming the delta, so
constraint (b) can never regress into a green light. MountParity +
MountInventory ride the result + hub wire record (additive). Dead
bindMountOverrides/archiveVMID path deleted with its tests (no reachable
lookalike). DR bring-up untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
renderConf appends a validated, deterministically-SORTED oob_peer_ip /32 so the
operator peer survives self-heal/restart (a runtime wg set was wiped by self-heal).
WireWireguard gains oob_peer_ip (omitempty — absent = byte-identical pre-H1 conf).
Non-hollow tests: sorted+byte-stable, lower-IP-sorts-first, absent-identical,
invalid/v6 rejected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Prerequisite for felhom-sshd (H1). Closes the SPIKE-felhom-sshd §8 lockout: a
second sshd's RuntimeDirectory=sshd removed the SHARED /run/sshd privsep dir and
took stock sshd on :22 down (sessions reset after KEXINIT).
Host artifacts (configs/, installed by felhom-host-install):
- felhom-privsep.tmpfiles: layer 1, boot-persistent /run/sshd owned by no unit
- felhom-mgmt-watchdog.sh/.service/.timer: layer 2, AGENT-INDEPENDENT ~60s heal
(stat-first recreate + reset-failed sshd only if failed + heal-marker); never
RuntimeDirectory=, never restarts stock sshd, never touches a healthy dir.
Go (internal/mgmtplane): read-only Reporter → additive omitempty mgmt_plane
heartbeat stanza (privsep_dir_ok/sshd_reachable/healed_recently/privsep_healed_at),
wired via Collector.SetMgmtPlaneReporter. Non-hollow tests + red-proofs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
- reconcile: ClassAgentUpdate op class; always Destructive (no provenance
blesses replacing the root-adjacent binary). classify test + companion
(TestClassify_AgentUpdateAlwaysDestructive).
- opsign: `-op agent_update` with -agent-version + -sha256 (isHex64-validated);
params {version,sha256}. isHex64 test (Group D).
- config: SelfUpdateConfig{URLTemplate,Username,Token,StateDir,DwellSeconds}
+ WithDefaults + Token redaction.
- internal/selfupdate: Executor (download → verify vs the SIGNED sha → sudo -n
wrapper `apply`; sha is the only integrity root — mismatch refuses + removes,
agent untouched); Manager (startup dwell → `commit`; version-mismatch → no
commit + loud WARN + marker left for report visibility; shutdown-before-dwell
leaves pending). WrapperRunner seam → tests never shell out.
- hub report: additive selfupdate_pending(+version) via SetSelfUpdateReporter
seam; both omitempty (Wireguard precedent) so the cross-repo golden contract
stays byte-stable — no hub change.
- capability manifest: 3 non-critical FELHOM_SELFUPDATE probes.
- main.go: updateExec appended to the executor chain; commit-manager wired to
the report seam + MaybeCommit goroutine after core init.
Tests: Group A (executor happy/sha-mismatch+companion/bad-params/wrapper-fail),
B (agent_update rides the real gate: pinned-key executes, non-pinned +
retarget rejected), C (commit/version-mismatch/no-pending/shutdown), D (opsign).
C2 companion red-proof verified (neutered Go verify → bad binary reaches apply
→ test fails), reverted. Full go test ./... green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
The 2026-07-04 CGNAT smoke test found MTU 1420 silently black-holes bulk TCP
on sub-~1480 paths (mobile ~1400, DS-Lite ~1452): handshake+ping stay healthy,
PBS TLS page (and at S4 the backup itself) drops. Set a fleet-wide, permanent,
family-agnostic client MTU of 1280 (RFC 8200 IPv6-minimum floor; outer 1340 v4
/ 1360 v6 fits every realistic path).
Client-only by construction — interface MTU caps box→PBS, advertised MSS caps
PBS→box; the endpoint's wg0 is untouched (zero live-endpoint risk). New const
clientMTU=1280 as the single home; golden pins exact "MTU = 1280" (red-proofed
against a 1420 flip). Stale report.go comment updated. No wire/JSON change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
key.go: create-once 0600/0700, corrupt-refusal (never overwrite — may be escrowed
identity), clamp for CANONICAL STORED form (x/crypto X25519 clamps derivation
internally — discovered during red-proof (c); the stored-clamped test is the
real anchor). Fixed vectors generated with real wg pubkey (provenance in test).
hub: WireDesiredState.Wireguard + WireguardStatus report stanza + RegisterWG
client (typed errors, token-free). S2 golden copied BYTE-IDENTICAL + field-exact
decode test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
HostReport.LeafFingerprint rides the served fp (from EnsureLeaf) on every report; empty when local
API disabled. Collector.SetLeafFingerprint threads it like Capabilities. Hub watches it for a re-key.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
New internal/capability: Manifest of required sudo -n grants + Prober that LISTS each
via 'sudo -n -l' (never executes) + binary-exists check → ok/degraded snapshot on the hub
report. Build-time test asserts manifest⊆sudoers (red-proof: dropping lxc-info FAILs the
gate). Startup logs N/N ok + ERROR per degraded. Serve-degraded; no allowlist change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
Live PBS coord: new internal/pbs/live_reporter.go (LiveSnapshotReporter implements
hub.PBSReporter via the cheap Client.Snapshots() list with last-known-good fallback,
bounded by an 8s timeout, list-only — never triggers a verify). Closes the gap where
the recipe's pbs block was omitted whenever the verify-loop SnapshotStore was empty
(one-shot collect + the first ~6h after a daemon restart). SnapshotStore.Get added
(per-datastore LKG). Wired into the collector in both runDaemon and runSelftestHub;
the verify loop keeps Recording into the SAME shared store via one hoisted pbsTargets.
v1 host-half drive shape: dropped drives[].role (hub/operator-owned manifest concept,
not host-derivable) and drives[].restic_repo_coord (named a backup tier that doesn't
exist). Drive shape is now {durable_id, mount_path, intent, fs_type?, total_bytes}.
Hub reads drives as json.RawMessage → no hub struct change; goldens re-pinned
byte-identical (agent + hub copies).
Tests: live_reporter_test.go (T1 load-bearing coord-without-verify + T2..T6),
TestDRRecipeHostHalf_V1DriveShape; each companion demonstrated to fail pre-fix then
reverted. go build/vet/test green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DR recipe slice (agent half), grounded in SPIKE-dr-recipe-2026-06-16. Additive
`dr_recipe` host-report section = the non-secret reconstruction scaffolding the
operator must rebuild before PBS bytes can land. Built by pure
BuildDRRecipeHostHalf from facts the report already collects (no new reads):
guests[] sizing, drives[] (user-data by durable_id/role/mount/intent),
pve_storage[] (storage.cfg), pbs coordinates.
BOUNDARY (Phase-1 lesson): every field is an identifier/intent/size/coordinate —
never a key/password/token/hash/ENC:. PBS key stays in escrow; restic password
stays in escrow; the recipe names only the coordinates the restore targets.
Tests: BuildDRRecipeHostHalf selection, NoPBS, NoSecrets (boundary mirror),
dr_recipe key-set in the cross-repo golden contract test. recipe_version=1,
ignore-unknown on read.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A destructive op runs ONLY on a pinned-key-verified, nonce-fresh, in-window,
host-bound, durable-id-bound operator signature. New cmd/felhom-opsign signs
canonical OpBlobs offline via ssh-keygen -Y sign (hardware-ready); the signing
key is never in the hub or agent. New internal/signedjobs runner verifies each
queued blob through the gate and only on all-pass runs the WipeExecutor, which
re-resolves the DURABLE device id + re-inspects (8C) before mkfs — closing the
8C data-bearing-wipe pending_signature gap. New storage durable-device
resolution; authz.CanonicalBlob promoted to production. Real-crypto tests assert
valid executes and forged/replay/expired/retarget/non-pinned are rejected
(executor never called).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The control envelope becomes live: the agent caches the hub's desired-state +
generation and re-fetches GET /hosts/{id}/desired-state only when the
generation advances. A new internal/desired Syncer maps the wire shape into a
reconcile.CachingProvider feeding the engine; benign deltas reconcile, an
explicit guest decommission is gated pending_signature (exec is 10B). Adds the
DesiredStateResponse/WireDesiredState wire types + Client.FetchDesiredState +
the loop EnvelopeObserver seam. Cross-repo golden (envelope + desired-state)
byte-identical with the hub.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a host-wide, token-authed GET /host/metrics local-API endpoint that
re-serves the slice-4 collector's host + per-storage view to the customer
(the de-privileged controller can't read the host itself). Add the one new
collector — CPU/chassis temperature via sysfs hwmon/thermal-zones, graceful-
null — to the shared HostMetrics struct, so the hub report carries cpu_temp_c
too. Cross-repo host-report golden updated byte-identical.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the two new RestoreTest wire keys to the shared host-report golden and exercises
them in the bidirectional key-set contract test. Kept byte-identical with felhom-hub's
copy (sha256 e6999d77...). Testdata/tests only — the v0.7.0 binary is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes the crying-wolf false-fail surfaced by the live hub-enrollment runbook:
PVE's guest-start task exits "WARNINGS: 1" for the benign systemd-nesting
advisory, and WaitTask treated any non-OK exitstatus as failure, so the verdict
was decided by an advisory exit code before the real boot check ran. Every
modern-distro restore-test reported pass:false.
- proxmox.WaitOptions.AllowWarnings (opt-in; default keeps all callers strict)
- restore-test start step accepts warnings, surfaces them, verdict stays waitRunning
- RestoreTestResult.StartWarnings/.WarningsRecognized + version-free "enable
nesting" recognizer (can't rot back at systemd 258+); GuestAPI.TaskLogTail
- hub.RestoreTest.warnings/.warnings_recognized wire fields (consumed by hub v0.7.5)
- scheduler logs clean / passed-with-recognized / passed-with-unrecognized warnings
- tests: WaitTask warnings matrix; restore-test pass/fail-on-liveness; version-free
regression guard (systemd 256-300)
Single agent bump 0.6.0 -> 0.7.0 covering the agent half of both task phases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Spike-proven that backup/restore-to-PBS reuse Phase A unchanged; the only new code is
the verify capability, a small PBS-API client, and PBSSnapshot reporting.
- internal/pbs: fingerprint-pinned, token-authed PBS-API client (Verify/Snapshots/
TaskStatus, node-from-UPID; secret read from /etc/pve/priv/storage/<id>.pw at runtime,
never logged) + the verify maintenance loop (own cadence, default 6h, NOT gated/journaled,
like the watchdog) + SnapshotStore.
- hub: PBSSnapshot filled (namespace/type/id/time/size/owner/protected/encrypted/
verify_state/verify_upid); PBSReporter collector seam; cross-repo golden + bidirectional
key-set tests; hub handler parses pbs_snapshots + logs a failed-verify WARN.
- backup: report the ACTUAL vzdump mode (parsed from the task log; PVE may downgrade
snapshot->stop). proxmox.Storage.Username. config PBSVerifyCadence/secret-dir.
--selftest=pbs-verify. Backup/restore-to-PBS unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fill the slice-3 storage_targets stub and add the fast-poll storage watchdog.
Read-only this phase; the host-root surface (mounts/SMART/grow/destructive gate)
is Phase B. Hub-owned desired manifest is slice 10, so reconcile against it is
built-but-unfed.
- internal/storage: StorageTarget wire contract, durable_id derivation per type,
HostReader seam (procfs/sysfs, root-free), Observer (storage_targets from
ListStorage/NodeStorage + host reads, lvmthin thin-pool fill), and the watchdog
(third daemon goroutine; debounced out-of-band report on a known target's
attach/disconnect transition).
- proxmox.Storage: additive parse-only config fields (durable_id sources).
- collector StorageObserver seam; Loop.SetTrigger out-of-band report; daemon runs
the watchdog as a third goroutine; StorageConfig knobs.
- cross-repo golden kept byte-identical with felhom.eu/hub; bidirectional key-set test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
internal/hub: the agent's first daemon — a periodic read-only host-report POSTed to
the hub (the heartbeat; no separate ping).
- HostReport wire contract (shared field-for-field with the hub ingest): host
metrics, guests (vmid + spec), cloudflared status; storage/backups/restore-tests/
pbs/audit collections DEFINED but emitted empty (slices 5/6 fill).
- Collector over a read-only proxmoxReader (adapted to the real proxmox surface;
no proxmox changes) + a CloudflaredProber. Partial-failure: NodeStatus fail = hard
(skip POST); per-guest GuestConfig fail = status "unknown", still report.
- Client: Bearer-auth POST, standard TLS (system roots / optional ca_file), typed
TransportError/HTTPError, token never in errors.
- Loop: immediate first report, adopt hub poll_interval (clamp [60,3600]), resilient
to collect/report errors, clean ctx-cancel shutdown.
- ControlEnvelope: only poll_interval_seconds acted on; blocked/desired_generation/
has_signed_ops parsed-but-ignored (slice 4).
- config: HubConfig + FELHOM_AGENT_HUB_* overlay + mode-aware HubConfig.Validate +
WithDefaults + hub-key redaction; example config updated.
- main: no-selftest mode is now the daemon; added --selftest=hub. Version -> 0.3.0.
Tests: report serialization, client (incl. token-redaction), collector partial-
failure, loop continuation+interval adoption, config. internal/proxmox + internal/
authz untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
admin