Commit Graph

276 Commits

Author SHA1 Message Date
admin 6db68f4e45 host-install v1.3.0: --uninstall (clean revert) + pre-flight guards
Colleague-safety batch #1+#2 (script-only). Adds a guarded, dry-run-aware
--uninstall local host teardown (guest -> agent -> pveum(ACL,token,user,role)
-> golden(opt-in) -> state file), with ours-check on the /etc/felhom-bootstrap
mount, typed vmid confirm, other-Felhom-guests safe-skip, and idempotent
tolerate-absent. Records customer_id + provisioned_vmid into the install state
(_state_put/_state_get) so uninstall resolves its target automatically. Adds
three provision pre-flight guards: multi-node (--node required), archive-storage
exists, and a RAM-floor WARN. Validated dry-run-only on felhom-pve.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 07:47:34 +02:00
admin 83c4f7f794 docs: REPORT for hub v0.28.0 + host-install v1.2.0 (live-validated)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:55:17 +02:00
admin b77df3c761 host-install: bump header comment to v1.2.0 (matches SCRIPT_VERSION)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:46:35 +02:00
admin 629196c66a deploy: hub 0.28.0 (settings→Configuration + online install)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:44:44 +02:00
admin 562c0dfeb6 hub v0.28.0 + host-install v1.2.0: settings→Configuration, online install, vmid auto-detect
Part A (hub): move the global-floor + Day-0-artifacts cards from the Customers
page to the Configuration tab; routes → /configuration/{global-floor,artifacts};
redirects + flashes to /configuration. Customers page back to list + Add.

Part B: online setup command on the customer page (download-then-run, passphrase
at prompt, not templated); serve /scripts/ from the website (sparse-checkout +
nginx location) so felhom.eu/scripts/felhom-host-install.sh resolves; script
passphrase prompt reads < /dev/tty (works for pipe-to-bash too).

Part C (script): --vmid auto-detect — default 9201 in use + no --force → pick the
next free id from pct+qm and confirm; explicit --vmid stays die-unless-force.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:43:29 +02:00
admin 54daaf949a docs: hub REPORT for v0.27.0 (Hosts page, live-validated)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:24:53 +02:00
admin f8b8730c53 deploy: hub 0.27.0 (Hosts page)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:11:25 +02:00
admin 2289fc907c hub v0.27.0: Hosts page — read-only fleet view (audit F-M1)
New Hosts nav section: a fleet list (/hosts) + per-host detail (/hosts/{id}),
read-only (GET only, no host actions). Surfaces identity, agent version,
online/stale status (reusing the HostStalenessChecker threshold), guests,
vitals, storage targets with SMART/thin-pool, and DR/escrow presence.

- store: new ListGuestsForHost reader (reality cols only; omits api_key/
  desired_spec_json) + scanGuest helper.
- web: handleHostsList + handleHostDetail (hosts.go); hosts.html +
  host_detail.html; Hosts nav link on every page; timeAgoPtr helper; routes.
- tests: store getter, both handlers, no-secret (api_key) assertion, 404,
  no-report empty state, status-band mapping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:10:02 +02:00
admin 73b3f6ac71 docs: hub REPORT for v0.26.0 (config delivery + inbound retire, live-validated)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 22:17:40 +02:00
admin d88b43a6e6 deploy: hub 0.26.0 (manifest bump)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 21:53:03 +02:00
admin a3ac6c9488 hub v0.26.0: pull-based config delivery + retire inbound GUI controls
config_version counter (bumped on every config save) advertised in the report
ACK; controller re-pulls + self-restarts on a change. Retire Trigger Update /
Push Config / Pull Config / Show Diff handlers+routes+buttons and the inbound
geo-notify (keep hub->Cloudflare geo removal). Setup command -> host-install;
delete dead customer.html + config_detail.html. Closes AUDIT-hub-gui F-S1/F-S4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 21:49:33 +02:00
admin e51e03bd7b docs(audit): hub GUI vs current architecture findings (2026-06-30)
Categorized STALE/DEAD/MISSING/POLISH/CORRECTNESS findings for the hub web
GUI. No GUI code changed, no version bump — findings doc only, for operator
prioritization.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 20:13:48 +02:00
admin fdcbc04d82 host-install: seed agent backup.local_backup_retention=3 (vzdump keep-last default)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 19:44:08 +02:00
admin 34364b47c6 docs: REPORT for hub v0.25.0 (per-storage fill alerting, live-validated)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 19:08:36 +02:00
admin e1dd06e77a test(temp): restore storage_fill threshold to default after live validation 2026-06-30 19:07:07 +02:00
admin 5a3747f6ab test(temp): lower storage_fill_warn_percent to 20 for live alert validation 2026-06-30 19:04:50 +02:00
admin 063fb0cfd4 deploy: hub image 0.24.0 -> 0.25.0 (per-storage fill alerting)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 19:02:31 +02:00
admin 88073ac464 hub v0.25.0: per-storage worst-fill alerting (StorageFillChecker)
Generalizes host_disk to any reported storage target (dump/backup volume, data drive,
thin pool, PBS). Per-(host,target) state, born/persistent, natural critical severity,
distinct storage_fill_* events; excludes the root-backed builtin (host_disk owns root).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 19:01:20 +02:00
admin cac745ee04 docs: REPORT for hub v0.24.0 (dispatcher critical routing, Part 0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 14:53:53 +02:00
admin 87f0794938 deploy: hub image 0.23.0 -> 0.24.0 (dispatcher critical routing)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 14:19:52 +02:00
admin 0ff1d3c883 hub v0.24.0: dispatcher routes critical severity (+ nil-prefs crash guard)
ProcessEvent routed only warning/error; a critical-severity event was silently dropped.
Now routes warning/error/critical, logs unrecognized severities, and guards a nil
GetNotificationPrefs (which would panic/crash the hub). host_disk_critical emits its
natural critical severity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 14:18:45 +02:00
admin 6882d853ed docs: REPORT for hub v0.23.0 (host disk monitoring, live alert proven)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 13:55:58 +02:00
admin 77c61ff494 deploy: hub image 0.22.1 -> 0.23.0 (host disk monitoring)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 13:51:13 +02:00
admin 897997c164 hub v0.23.0: host root-disk pressure monitoring + alert
New HostDiskChecker on the 60s sweep alerts the operator when a Proxmox host root
filesystem crosses warn (90%) / crit (95%). Born/persistent (a disk already full at
hub restart alerts on cycle 1); distinct host_disk_* event types from the guest disk_*;
critical band maps to severity error (the dispatcher only routes warning/error).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 13:48:55 +02:00
admin f133355e34 audit: §D supervised reboot done — F2-a + F2-b red-proofs PASS (findings closed)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0162BnMpUXscPsUB1cU8Tr6K
2026-06-30 10:04:59 +02:00
admin 06727e3228 docs+config: operator email enabled (Part A, proven) + TESTRUN findings closed
Part A: hub-config operator_email/operator_enabled (committed earlier);
operator + customer email paths proven end-to-end via real Resend sends.
Append the "Findings closed" section to the TESTRUN audit (Part A proof,
F2-a root cause + deploy fix, F2-b fix + live validation, pending §D
supervised reboot) and remove the stale duplicate "Final state" block.
REPORT overwritten. hub/CHANGELOG entry. No hub image change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0162BnMpUXscPsUB1cU8Tr6K
2026-06-30 09:21:17 +02:00
admin 83ed127d53 hub-config: enable operator email alerts (operator_email + operator_enabled)
The dispatcher's operator path was gated off (processOperator returns
unless operatorOn && operatorEmail!=""), so the self-health pipeline
stopped one hop short of the inbox. Set the operator recipient
(admin@felhom.eu — the operator's own address, not a secret) and enable.
Config-only; no image bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0162BnMpUXscPsUB1cU8Tr6K
2026-06-30 09:10:10 +02:00
admin c978ea1951 testrun: coverage gaps RESOLVED — capability->hub alert fired live; 2 reboot findings (doubling NOT self-corrected, backup-lock)
Phase 1: capability->hub operator alert fired live (event captured, English, names cap+feature);
email is operator-config-gated in demo (not a defect). Phase 2: drive re-propagation + zero false
pages PASS, but TWO findings — F2-a the §4 doubling did NOT self-correct (boot script runs as root,
never needed the grant → hypothesis falsified), F2-b a vzdump backup in flight at reboot left a stale
snapshot-delete lock blocking guest autostart. Phase 3 referenced (host_leaf_changed proven on v0.22.1).
Final parity known-good.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:56:54 +02:00
admin f4d88c1ff4 docs: felhom.eu REPORT (HostLeafChecker) + hub README + self-health arc complete (spike doc)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:32:12 +02:00
admin c37658ebb0 hub: CHANGELOG v0.22.1 (HostLeafChecker wiring fix)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:27:31 +02:00
admin 8f3b247e6e manifest: hub 0.22.0 -> 0.22.1 (HostLeafChecker wiring)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:26:37 +02:00
admin 253117b292 hub: WIRE HostLeafChecker into the monitor loop (the wiring was missed in v0.22.0)
The v0.22.0 commit added the checker but the main.go goroutine edit never applied. Live test caught
it (no host_leaf_changed on a leaf regen). Now started on the 60s sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:25:20 +02:00
admin 27aa15fb93 manifest: hub 0.21.0 -> 0.22.0 (HostLeafChecker)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:16:54 +02:00
admin 9c5cf2975f hub: HostLeafChecker — proactive agent re-key detection (host_leaf_changed) v0.22.0
Watches each host's reported local-API leaf fp; alerts on change (trust-on-first-report). Sibling of
HostCapabilityChecker; store.GetHostLeafFingerprints reads report_json (no migration); hub-generated
event (no allowlist change). Change red-proof + first-obs-seed + empty-skip + blocked-drop tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:15:22 +02:00
admin dd2bb5fc01 audit(testrun): F1 RESOLVED — swap verify RestartCount+dwell (agent v0.47.0), live rollback confirmed
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 22:50:17 +02:00
admin cbba20aa73 docs: felhom.eu REPORT (hub F2 + install-script preservation) + TESTRUN F2-closed
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 22:01:57 +02:00
admin 575f62ab7f manifest: hub 0.20.0 -> 0.21.0 (F2 born-degraded alert)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 21:49:48 +02:00
admin a69e06354e host-install: --preserve-state-from + populated-host leaf guard (prevention B.2)
--preserve-state-from DIR carries the prior local-api.{crt,key}+local-tokens.log into the agent
state dir (validates the leaf parses) so the pinned fp stays stable across a reinstall — no
re-bootstrap. Populated-host guard: refuse to proceed leaf-less when the host already has guests
unless --preserve-state-from or --allow-new-leaf is given (converts the 2026-06-28 silent footgun
into a hard stop). bash -n clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 21:46:45 +02:00
admin 12f038618e hub: F2 — alert on host already degraded/stale at (re)start (seed only healthy) v0.21.0
Constructors seed only healthy hosts; an already-degraded/stale host is left unseeded so the first
Check() emits once (cooldown dedups on hub restart). Born-degraded red-proof + staleness test updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 21:46:25 +02:00
admin a297639c58 testrun: full-stack no-mercy campaign — capstone pin-mismatch detection PASS (~25s), swap happy PASS + rollback finding, multi-drive reboot PASS
Headline: leaf-regeneration (original-incident class) DETECTED by channel-health in ~25s + recovered.
Controller-swap happy PASS (byte-exact tee) + negative controls denied; FINDING F1: no-healthcheck
crash-loop image slips the point-in-time swap verify (low sev — real controller has a healthcheck).
Multi-drive guest-reboot re-propagation PASS, no flapping. Stack restored to known-good parity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 21:10:33 +02:00
admin 26a6dde2fb audit(channel-health): IMPLEMENTED — controller v0.90.0 + hub v0.20.0 allowlist; live-validated
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 20:42:38 +02:00
admin d3661319c8 manifest: hub 0.19.0 → 0.20.0 (accept agent_channel_* events)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 20:38:28 +02:00
admin 121405772f hub: accept controller agent_channel_* events (channel-health relay) v0.20.0
The controller v0.90.0 channel-health checker pushes agent_channel_* events to /api/v1/event;
the allowedEventTypes allowlist rejected them (HTTP 400). Added the 8 types (operator-only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 20:37:24 +02:00
admin 0d470b1e05 spike: controller-agent channel health — GO (Q1 classify map; Q2 memoized client self-heals, fresh probe faithful)
Q1: 6 modes classified — pin-mismatch/401/refused/timeout/no-such-host are RUNTIME (per-call),
malformed-fp is a CONSTRUCTION error (latches via sync.Once). Q2: across a clean agent restart the
memoized /api/disks path self-heals in ~1s with NO controller restart (Up 7h); a fresh-client probe
agrees at every phase. Recommend: probe GET /storage via the memoized client, classify, N>=2 debounce
for transient reasons, hub relay. Throwaway probe removed, no config changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 20:06:06 +02:00
admin 6b1c2892d6 docs: controller-swap narrow-grant Option A IMPLEMENTED (agent v0.45.0) — spike + multidrive pointers resolved
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 19:46:17 +02:00
admin 1141d74532 spike: narrow-grant controller-swap mechanics — GO (stdin tee + sudoers * spans spaces, grants stay scoped)
Q1 stdin->pct exec->tee writes byte-identical image\n (PASS); Q2 all 5 narrow grants match,
* spans spaces, negatives denied (PASS); Q3 deployed bootstrap reads via $(cat), tee-written
file consumed identically (PASS). Scratch drop-in + file removed, /etc/sudoers re-validated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 19:27:15 +02:00
admin eea3cd4040 docs: hub REPORT (v0.19.0 capability alert) + multidrive audit countermeasure pointer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 18:57:23 +02:00
admin 2312572411 manifest: hub 0.18.0 → 0.19.0 (agent capability-degraded alert)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 18:53:22 +02:00
admin b7b165bff5 hub: HostCapabilityChecker — operator alert on agent capability-degraded (v0.19.0)
Companion to felhom-agent v0.44.0. New monitor.HostCapabilityChecker (sibling of
HostStalenessChecker) reads the capabilities snapshot from the latest host report and emits
agent_capability_degraded/recovered (operator-only, 1h cooldown) on ok<->degraded transitions
for any Critical capability. store.GetHostCapabilities (MAX(id), no migration). Goldens mirror
the new capabilities field.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 18:50:50 +02:00
admin c6bae9515a audit(multidrive): §8 Outcome — FIXED via sudoers completeness pass; controller-swap surfaced
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 16:06:41 +02:00