store.GetHostOOBStates parses the agent oob heartbeat stanza. monitor/host_oob:
transition-based oob_degraded/oob_recovered warning (felhom-sshd down while the
operator peer is configured, OR config invalid) — proactive "can the operator get
in right now" signal; unconfigured OOB never alerts. Wired into the 60s sweep.
Non-hollow tests + transitions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
A heal marker is an event, not a baseline: construction seeds pre-existing markers
(startup false-alarm guard) but a newly-observed marker now raises the warning, so
the first auto-heal surfaces (matches the live drill). Added tests for both halves.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Endpoint card + peers table (truncated pubkeys with full-value title attr,
bound peers link to /hosts/<id>); Offsite nav link in all 9 page templates;
render tests for endpoint/peers, empty, and not-configured states.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
POST /hosts/{id}/wg (per-host self-scoped; global = operator/DR path): bind /
re-key-in-place / adopt; generation bump + endpoint push ONLY on real change.
mergeWireguard injects the hub-owned block into served desired-state at READ
time (stored operator blob never touched; fail-safe unmerged on any error;
no-peer = byte-identical pass-through — existing golden test untouched+green).
handleAdminSetDesiredState rejects top-level wireguard (400). Admin DELETE of a
BOUND peer bumps the owning host. NEW golden desired-state-wireguard.golden.json
= the S3 cross-repo contract. Red-proofs a/b/c/d run + reverted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Live S1 validation caught it: a stock multi-hostkey sshd presented ECDSA while
we pin ed25519 → FixedHostKey refused a legitimate server. Regression test with
an in-process dual-hostkey server (fails without the fix — red-proofed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
internal/wgsync: x/crypto/ssh client with ssh.FixedHostKey pin (no insecure
fallback), forced-command exec, ok/applied response contract; Reconciler pushes
the FULL peer list on Trigger or 5-min tick (drift repair by construction).
internal/api/wg.go: PUT/GET /admin/wg/endpoint + POST/DELETE/GET /admin/wg/peers,
global-key-only, pubkey in body (base64 vs URL), sync ok|deferred|disabled.
main.go: WG_ENDPOINT_SSH_* env wiring, disabled-with-INFO when unconfigured.
Groups B/C/D tests incl. in-process SSH server; red-proofs b/c/d run + reverted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
- static/fonts/: the 4 vendored woff2 (byte-copied from
felhom-controller), embedded (embed.go) and served at /static/fonts/
(font/woff2, immutable) mirroring the chart.min.js pattern. No CDN
before, none now.
- templates/icons.html: 12-symbol Lucide sprite partial (icon_sprite),
included at the top of <body> on all 9 pages ({{template}} — the hub
has no shared layout; per-page include is the minimal shared block).
- statusColor now returns v2 semantic tokens (nominal/warn/crit/
neutral) consumed as class suffixes: ok->nominal, warn+stale->warn,
down+fail->crit, pending+disabled->neutral (a not-yet-provisioned or
deliberately paused customer is a normal fleet state), blocked->warn
(intentional operator cut-off, attention-worthy not an outage),
unknown->neutral. The inline style="color: {{statusColor}}" pattern
is dead: dashboard + customer_unified render a class-based
.status-dot-<token>; statusIcon (constant "●") retired from funcmap
and templates.
- Tests (new; the hub web package had no funcmap/template tests):
TestStatusColorTruthTable over the full enumerated status set —
red-proven vs the old implementation (ok returned "#4ade80") — and
TestTemplatesParseWithFuncmap.
Operator picks a version from a Gitea-populated dropdown; the hub reads that
version's sha256 from Gitea itself (files-metadata API, no artifact download) and
vouches it — no hand-copied checksums. New internal/gitea read-only client
(ListVersions + FileSHA256, unit-tested). Configuration UI: version <select>s +
read-only sha display; handleSetArtifacts derives the sha authoritatively and
refuses the save on a Gitea lookup failure. Degrades to manual text entry without
registry creds. go build/vet/test clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Part A (hub): move the global-floor + Day-0-artifacts cards from the Customers
page to the Configuration tab; routes → /configuration/{global-floor,artifacts};
redirects + flashes to /configuration. Customers page back to list + Add.
Part B: online setup command on the customer page (download-then-run, passphrase
at prompt, not templated); serve /scripts/ from the website (sparse-checkout +
nginx location) so felhom.eu/scripts/felhom-host-install.sh resolves; script
passphrase prompt reads < /dev/tty (works for pipe-to-bash too).
Part C (script): --vmid auto-detect — default 9201 in use + no --force → pick the
next free id from pct+qm and confirm; explicit --vmid stays die-unless-force.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ProcessEvent routed only warning/error; a critical-severity event was silently dropped.
Now routes warning/error/critical, logs unrecognized severities, and guards a nil
GetNotificationPrefs (which would panic/crash the hub). host_disk_critical emits its
natural critical severity.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
New HostDiskChecker on the 60s sweep alerts the operator when a Proxmox host root
filesystem crosses warn (90%) / crit (95%). Born/persistent (a disk already full at
hub restart alerts on cycle 1); distinct host_disk_* event types from the guest disk_*;
critical band maps to severity error (the dispatcher only routes warning/error).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
Constructors seed only healthy hosts; an already-degraded/stale host is left unseeded so the first
Check() emits once (cooldown dedups on hub restart). Born-degraded red-proof + staleness test updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
The controller v0.90.0 channel-health checker pushes agent_channel_* events to /api/v1/event;
the allowedEventTypes allowlist rejected them (HTTP 400). Added the 8 types (operator-only).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
Companion to felhom-agent v0.44.0. New monitor.HostCapabilityChecker (sibling of
HostStalenessChecker) reads the capabilities snapshot from the latest host report and emits
agent_capability_degraded/recovered (operator-only, 1h cooldown) on ok<->degraded transitions
for any Critical capability. store.GetHostCapabilities (MAX(id), no migration). Goldens mirror
the new capabilities field.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
Operator sets a minimum controller version (FLOOR), per-customer defaulting to a
global floor; the report ACK returns the effective floor + latest_version so the
controller auto-updates to the floor when below it (latest stays the opt-in button).
- store: min_controller_version column + hub_settings global floor + Effective/
Get/SetGlobal/SetMin resolution + config/env DEFAULT_MIN_CONTROLLER_VERSION
- handler: report ACK {min_controller_version, latest_version}; LatestVersionProvider
- web: global floor editor + per-customer override form + Floor column (English)
- tests: floor resolution + ACK + render; override-precedence red-proof verified
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSZmmSFVzGwEzhYmxbkgBK
New POST /api/v1/host-enroll (handleHostEnroll): X-Retrieval-Password authed,
body {customer_id} -> {host_id, api_key}. Mint-once-reuse (201 first, 200
reuse) so re-running the host-bootstrap never orphans a running agent's key;
auth checked before any mint. Backed by new Store.GetHostByCustomer
(ORDER BY updated_at DESC LIMIT 1, idx_hosts_customer).
GET /config/{id} and global-key POST /admin/hosts left untouched. Exact-match
route (path == "/host-enroll") to avoid the /hosts/ prefix collision.
Tests: host_enroll_test.go (mint/reuse/401-no-mint/404/400) + GetHostByCustomer
store test; companion red-proof verified always-mint fails the reuse assertion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtXesNa2LGbMmE4DNL6SE7
Tracks felhom-agent v0.39.0, which dropped role + restic_repo_coord from the host-half
drives[]. Hub reads drives as json.RawMessage (verbatim passthrough) → no store/handler
change needed. Dropped role from the store golden + drHostHalf fixture; re-synced the api
host-report golden byte-identical with the agent copy (it previously lacked dr_recipe
entirely). Added TestAssembleDRRecipe_V1DriveShape regression guard (demonstrated to fail
when the fixture re-adds role, then reverted). No behavior change; redeploy optional.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DR recipe slice (hub half), grounded in SPIKE-dr-recipe-2026-06-16. The hub
receives two additive dr_recipe halves on the existing report paths (agent
storage/guest/PBS on host-report; controller customer/apps on the controller
report), stores them PLAINTEXT in a DEDICATED dr_recipe table keyed by customer
(each half preserves the other), and AssembleDRRecipe stitches them into one
operator-readable recipe (ignore-unknown + version-skew tolerant).
View: a DR-recipe panel on the customer page + GET /customers/{id}/dr-recipe.json
download (operator-auth, no secrets to redact). Plaintext-at-rest is correct —
the recipe is the clean inverse of the retired infra-backup.
Tests: store round-trip (each half preserves the other), assemble-matches-golden,
ignore-unknown + version skew, partial halves, no-secrets sweep. Manifest tag
bumped to v0.13.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase-1 of SPIKE-infra-backup-2026-06-15. The infra-backup mechanism was dead
since slice 8C yet stored plaintext customer secrets at rest (app-secret key,
restic password, Cloudflare tokens) — a zero-knowledge violation — and its
absence made the daily expected_backup_missed email fire for healthy customers.
- Repoint monitor.CheckBackupDeadlines backup half to the agent host-report's
PBS snapshots (+vzdump): alarm only on no-backup / >26h stale / verify failed.
Keep the db_dump half. No host-report → no backup alarm (liveness owns that).
New store.GetLatestHostReportJSON. Tests incl. a companion that fails pre-fix.
- Remove the infra-backup endpoints, store methods/types, and operator panel;
/recovery now returns config_yaml only.
- migrate(): DROP infra_backup_versions/infra_backups + VACUUM (+wal_checkpoint)
to physically reclaim the plaintext pages, gated on table existence.
Flagged out-of-scope: exposed creds need operator rotation; legacy reports table
holds historical plaintext restic_password rows (separate leak, not purged here).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add DELETE /hosts/{id}/jobs/{job_id} (per-host self-scoped, idempotent) so the
agent clears a job after executing or terminally rejecting it. The hub stores
the operator-signed blobs opaquely (no signing key — cannot forge or open);
the agent verifies + executes. Doc 03 §4/§6/§9 updated (operator-signed path
live; 8C wipe completes; 10B done).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Serve operator intent to authenticated hosts: PUT /admin/hosts/{id}/desired-state
(global key) bumps desired_generation; GET /hosts/{id}/desired-state + /jobs are
per-host self-scoped; the host-report envelope now carries the real generation +
has_signed_ops. New signed_jobs table + store methods. Desired-state stored/served
opaquely (agent owns the schema). Cross-repo golden (envelope + desired-state)
byte-identical with felhom-agent; doc 03 §4/§9 updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update the cross-repo host-report golden byte-identical with felhom-agent
(host.cpu_temp_c). Document GET /host/metrics in doc 03 section 6 and define
slice 9 in the section 9 roadmap. No hub code change / no version bump.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase B (hub half) of the restore-test warning fix. The agent v0.7.0 now passes a
restore-test that emitted a benign start advisory (systemd-nesting) and carries the
warning text on the wire.
- hostRestoreTest gains warnings + warnings_recognized mirror fields (omitempty;
absent recognized => false => louder unrecognized path)
- ingest logs [INFO] passed WITH WARNINGS (recognized), [WARN] for unrecognized;
FAILED still [WARN]
- golden restore_tests[0] gains the keys, byte-identical with felhom-agent (sha256
e6999d77...); bidirectional key-set contract test round-trips them
- no dashboard widget: no host-domain dashboard surface exists yet (log+persist only,
as with pbs_snapshots) -- deferred to slice 10
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Accept + persist the now-populated host-report pbs_snapshots. hostPBSSnapshot mirror in
hostReportPayload (persisted via report_json, no schema change); a FAILED PBS verify is
logged prominently (loudest offsite-DR signal). Shared golden updated byte-identical with
felhom-agent; TestHostPBSSnapshot_GoldenContract added. Build/deploy deferred (backward-compatible).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Accept + persist the now-populated host-report backups/restore_tests. Mirror structs in
hostReportPayload; persisted via report_json (no schema change); a FAILED restore-test is
logged prominently (loudest DR signal). Shared golden updated byte-identical with
felhom-agent; bidirectional key-set tests added. Build/deploy deferred (backward-compatible).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Accept + persist the now-populated host-report storage_targets. Minimal — the
authoritative storage manifest is hub-owned (slice 10); this mirrors what the agent
observes.
- hostReportPayload.StorageTargets: full mirror of the agent's hub.StorageTarget
wire contract; persisted verbatim in report_json (no schema change); count +
WARN on disconnected targets.
- shared host-report golden updated with two populated targets; byte-identical with
felhom-agent's copy.
- TestHostStorageTarget_GoldenContract: hub half of the bidirectional key-set test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- handleHostReport: read maxHostReportBytes+1 (4 MiB const) and reject oversize with
413 instead of silent LimitReader truncation. Controller handleReport (1 MiB) is
unchanged. Test asserts 413.
- contract: hub/internal/api/testdata/host-report.golden.json (byte-identical with
felhom-agent's copy) + TestHostReport_GoldenContract drives the real handler and
asserts 200 + denorm + both guests upserted.
- CHANGELOG v0.7.1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restores notify/templates.go, store/telemetry.go, web/configs.go to upstream —
those were alignment-only churn from a tree-wide gofmt, not part of slice 3. Keeps
the host-domain diff additions-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- hub/internal/web/templatefetcher.go: raw-template URL now points at the renamed
repo (was relying on Gitea's post-rename redirect)
- documentation/ (moved here from the felhom-agent repo): fix controller-source path
refs (deploy-felhom-compose -> felhom-controller) and the platform repo name
(proxmox-controller -> felhom-agent)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the hardcoded 60s meta-refresh with a JavaScript-based timer
and a toggle switch in the page header. The preference persists across
page loads via localStorage (enabled by default).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New infra_backup_versions table with GFS pruning (~14 versions per
customer). Recovery endpoint supports ?version=ID. New /versions API.
Dashboard shows collapsible backup history with app names and disk count.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
admin