Commit Graph

268 Commits

Author SHA1 Message Date
admin 73b3f6ac71 docs: hub REPORT for v0.26.0 (config delivery + inbound retire, live-validated)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 22:17:40 +02:00
admin d88b43a6e6 deploy: hub 0.26.0 (manifest bump)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 21:53:03 +02:00
admin a3ac6c9488 hub v0.26.0: pull-based config delivery + retire inbound GUI controls
config_version counter (bumped on every config save) advertised in the report
ACK; controller re-pulls + self-restarts on a change. Retire Trigger Update /
Push Config / Pull Config / Show Diff handlers+routes+buttons and the inbound
geo-notify (keep hub->Cloudflare geo removal). Setup command -> host-install;
delete dead customer.html + config_detail.html. Closes AUDIT-hub-gui F-S1/F-S4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 21:49:33 +02:00
admin e51e03bd7b docs(audit): hub GUI vs current architecture findings (2026-06-30)
Categorized STALE/DEAD/MISSING/POLISH/CORRECTNESS findings for the hub web
GUI. No GUI code changed, no version bump — findings doc only, for operator
prioritization.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 20:13:48 +02:00
admin fdcbc04d82 host-install: seed agent backup.local_backup_retention=3 (vzdump keep-last default)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 19:44:08 +02:00
admin 34364b47c6 docs: REPORT for hub v0.25.0 (per-storage fill alerting, live-validated)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 19:08:36 +02:00
admin e1dd06e77a test(temp): restore storage_fill threshold to default after live validation 2026-06-30 19:07:07 +02:00
admin 5a3747f6ab test(temp): lower storage_fill_warn_percent to 20 for live alert validation 2026-06-30 19:04:50 +02:00
admin 063fb0cfd4 deploy: hub image 0.24.0 -> 0.25.0 (per-storage fill alerting)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 19:02:31 +02:00
admin 88073ac464 hub v0.25.0: per-storage worst-fill alerting (StorageFillChecker)
Generalizes host_disk to any reported storage target (dump/backup volume, data drive,
thin pool, PBS). Per-(host,target) state, born/persistent, natural critical severity,
distinct storage_fill_* events; excludes the root-backed builtin (host_disk owns root).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 19:01:20 +02:00
admin cac745ee04 docs: REPORT for hub v0.24.0 (dispatcher critical routing, Part 0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 14:53:53 +02:00
admin 87f0794938 deploy: hub image 0.23.0 -> 0.24.0 (dispatcher critical routing)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 14:19:52 +02:00
admin 0ff1d3c883 hub v0.24.0: dispatcher routes critical severity (+ nil-prefs crash guard)
ProcessEvent routed only warning/error; a critical-severity event was silently dropped.
Now routes warning/error/critical, logs unrecognized severities, and guards a nil
GetNotificationPrefs (which would panic/crash the hub). host_disk_critical emits its
natural critical severity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 14:18:45 +02:00
admin 6882d853ed docs: REPORT for hub v0.23.0 (host disk monitoring, live alert proven)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 13:55:58 +02:00
admin 77c61ff494 deploy: hub image 0.22.1 -> 0.23.0 (host disk monitoring)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 13:51:13 +02:00
admin 897997c164 hub v0.23.0: host root-disk pressure monitoring + alert
New HostDiskChecker on the 60s sweep alerts the operator when a Proxmox host root
filesystem crosses warn (90%) / crit (95%). Born/persistent (a disk already full at
hub restart alerts on cycle 1); distinct host_disk_* event types from the guest disk_*;
critical band maps to severity error (the dispatcher only routes warning/error).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 13:48:55 +02:00
admin f133355e34 audit: §D supervised reboot done — F2-a + F2-b red-proofs PASS (findings closed)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0162BnMpUXscPsUB1cU8Tr6K
2026-06-30 10:04:59 +02:00
admin 06727e3228 docs+config: operator email enabled (Part A, proven) + TESTRUN findings closed
Part A: hub-config operator_email/operator_enabled (committed earlier);
operator + customer email paths proven end-to-end via real Resend sends.
Append the "Findings closed" section to the TESTRUN audit (Part A proof,
F2-a root cause + deploy fix, F2-b fix + live validation, pending §D
supervised reboot) and remove the stale duplicate "Final state" block.
REPORT overwritten. hub/CHANGELOG entry. No hub image change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0162BnMpUXscPsUB1cU8Tr6K
2026-06-30 09:21:17 +02:00
admin 83ed127d53 hub-config: enable operator email alerts (operator_email + operator_enabled)
The dispatcher's operator path was gated off (processOperator returns
unless operatorOn && operatorEmail!=""), so the self-health pipeline
stopped one hop short of the inbox. Set the operator recipient
(admin@felhom.eu — the operator's own address, not a secret) and enable.
Config-only; no image bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0162BnMpUXscPsUB1cU8Tr6K
2026-06-30 09:10:10 +02:00
admin c978ea1951 testrun: coverage gaps RESOLVED — capability->hub alert fired live; 2 reboot findings (doubling NOT self-corrected, backup-lock)
Phase 1: capability->hub operator alert fired live (event captured, English, names cap+feature);
email is operator-config-gated in demo (not a defect). Phase 2: drive re-propagation + zero false
pages PASS, but TWO findings — F2-a the §4 doubling did NOT self-correct (boot script runs as root,
never needed the grant → hypothesis falsified), F2-b a vzdump backup in flight at reboot left a stale
snapshot-delete lock blocking guest autostart. Phase 3 referenced (host_leaf_changed proven on v0.22.1).
Final parity known-good.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:56:54 +02:00
admin f4d88c1ff4 docs: felhom.eu REPORT (HostLeafChecker) + hub README + self-health arc complete (spike doc)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:32:12 +02:00
admin c37658ebb0 hub: CHANGELOG v0.22.1 (HostLeafChecker wiring fix)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:27:31 +02:00
admin 8f3b247e6e manifest: hub 0.22.0 -> 0.22.1 (HostLeafChecker wiring)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:26:37 +02:00
admin 253117b292 hub: WIRE HostLeafChecker into the monitor loop (the wiring was missed in v0.22.0)
The v0.22.0 commit added the checker but the main.go goroutine edit never applied. Live test caught
it (no host_leaf_changed on a leaf regen). Now started on the 60s sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:25:20 +02:00
admin 27aa15fb93 manifest: hub 0.21.0 -> 0.22.0 (HostLeafChecker)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:16:54 +02:00
admin 9c5cf2975f hub: HostLeafChecker — proactive agent re-key detection (host_leaf_changed) v0.22.0
Watches each host's reported local-API leaf fp; alerts on change (trust-on-first-report). Sibling of
HostCapabilityChecker; store.GetHostLeafFingerprints reads report_json (no migration); hub-generated
event (no allowlist change). Change red-proof + first-obs-seed + empty-skip + blocked-drop tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 23:15:22 +02:00
admin dd2bb5fc01 audit(testrun): F1 RESOLVED — swap verify RestartCount+dwell (agent v0.47.0), live rollback confirmed
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 22:50:17 +02:00
admin cbba20aa73 docs: felhom.eu REPORT (hub F2 + install-script preservation) + TESTRUN F2-closed
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 22:01:57 +02:00
admin 575f62ab7f manifest: hub 0.20.0 -> 0.21.0 (F2 born-degraded alert)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 21:49:48 +02:00
admin a69e06354e host-install: --preserve-state-from + populated-host leaf guard (prevention B.2)
--preserve-state-from DIR carries the prior local-api.{crt,key}+local-tokens.log into the agent
state dir (validates the leaf parses) so the pinned fp stays stable across a reinstall — no
re-bootstrap. Populated-host guard: refuse to proceed leaf-less when the host already has guests
unless --preserve-state-from or --allow-new-leaf is given (converts the 2026-06-28 silent footgun
into a hard stop). bash -n clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 21:46:45 +02:00
admin 12f038618e hub: F2 — alert on host already degraded/stale at (re)start (seed only healthy) v0.21.0
Constructors seed only healthy hosts; an already-degraded/stale host is left unseeded so the first
Check() emits once (cooldown dedups on hub restart). Born-degraded red-proof + staleness test updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 21:46:25 +02:00
admin a297639c58 testrun: full-stack no-mercy campaign — capstone pin-mismatch detection PASS (~25s), swap happy PASS + rollback finding, multi-drive reboot PASS
Headline: leaf-regeneration (original-incident class) DETECTED by channel-health in ~25s + recovered.
Controller-swap happy PASS (byte-exact tee) + negative controls denied; FINDING F1: no-healthcheck
crash-loop image slips the point-in-time swap verify (low sev — real controller has a healthcheck).
Multi-drive guest-reboot re-propagation PASS, no flapping. Stack restored to known-good parity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 21:10:33 +02:00
admin 26a6dde2fb audit(channel-health): IMPLEMENTED — controller v0.90.0 + hub v0.20.0 allowlist; live-validated
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 20:42:38 +02:00
admin d3661319c8 manifest: hub 0.19.0 → 0.20.0 (accept agent_channel_* events)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
2026-06-29 20:38:28 +02:00
admin 121405772f hub: accept controller agent_channel_* events (channel-health relay) v0.20.0
The controller v0.90.0 channel-health checker pushes agent_channel_* events to /api/v1/event;
the allowedEventTypes allowlist rejected them (HTTP 400). Added the 8 types (operator-only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 20:37:24 +02:00
admin 0d470b1e05 spike: controller-agent channel health — GO (Q1 classify map; Q2 memoized client self-heals, fresh probe faithful)
Q1: 6 modes classified — pin-mismatch/401/refused/timeout/no-such-host are RUNTIME (per-call),
malformed-fp is a CONSTRUCTION error (latches via sync.Once). Q2: across a clean agent restart the
memoized /api/disks path self-heals in ~1s with NO controller restart (Up 7h); a fresh-client probe
agrees at every phase. Recommend: probe GET /storage via the memoized client, classify, N>=2 debounce
for transient reasons, hub relay. Throwaway probe removed, no config changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 20:06:06 +02:00
admin 6b1c2892d6 docs: controller-swap narrow-grant Option A IMPLEMENTED (agent v0.45.0) — spike + multidrive pointers resolved
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 19:46:17 +02:00
admin 1141d74532 spike: narrow-grant controller-swap mechanics — GO (stdin tee + sudoers * spans spaces, grants stay scoped)
Q1 stdin->pct exec->tee writes byte-identical image\n (PASS); Q2 all 5 narrow grants match,
* spans spaces, negatives denied (PASS); Q3 deployed bootstrap reads via $(cat), tee-written
file consumed identically (PASS). Scratch drop-in + file removed, /etc/sudoers re-validated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 19:27:15 +02:00
admin eea3cd4040 docs: hub REPORT (v0.19.0 capability alert) + multidrive audit countermeasure pointer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 18:57:23 +02:00
admin 2312572411 manifest: hub 0.18.0 → 0.19.0 (agent capability-degraded alert)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 18:53:22 +02:00
admin b7b165bff5 hub: HostCapabilityChecker — operator alert on agent capability-degraded (v0.19.0)
Companion to felhom-agent v0.44.0. New monitor.HostCapabilityChecker (sibling of
HostStalenessChecker) reads the capabilities snapshot from the latest host report and emits
agent_capability_degraded/recovered (operator-only, 1h cooldown) on ok<->degraded transitions
for any Critical capability. store.GetHostCapabilities (MAX(id), no migration). Goldens mirror
the new capabilities field.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 18:50:50 +02:00
admin c6bae9515a audit(multidrive): §8 Outcome — FIXED via sudoers completeness pass; controller-swap surfaced
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 16:06:41 +02:00
admin 66a001de0b spike: multi-drive mutual-exclusion — root cause is bound_under_parent detection bug (lxc-info denied under non-root agent), not propagation/gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 15:33:44 +02:00
admin 1777c89fb9 spike(pin-mismatch): §9 Outcome — R1 executed, controller↔agent service restored
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 15:08:58 +02:00
admin 8254b8823e spike(pin-mismatch): follow-up — aside-var-lib holds matching token store; R1 = one-copy fix for pin+token
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 14:44:10 +02:00
admin 2de9659592 spike: controller↔agent TLS leaf-pin mismatch — root→non-root migration regenerated agent leaf (findings)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
2026-06-29 14:38:43 +02:00
admin 0719db0d53 spike: NAS network storage (media automount + restic backup target) — findings
Empirical validation on a DooPlex-simulated NAS (SMB+NFS). Verdict READY for the
production network-storage TASK. Key findings: mount must be host-side (unprivileged
LXC blocks NFS/CIFS mount) + bind-propagated into the guest; NFS soft fails-clean +
auto-recovers (SMB hangs-but-contained); UID mapping = container uid + 100000 LXC
offset; ~128MB/s adequate; write integrity intact (atomic-write apps safe on soft);
restic-SFTP backup/restore works + fails-as-job not hang; systemd automount gives
on-demand + idle-unmount. Distinct network-storage class bypasses the drive lifecycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 11:44:40 +02:00
admin b4ce6c10b5 docs: FINDING — app-email rollout mechanism gaps (calcom/nextcloud/immich)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 10:32:56 +02:00
admin 576f4ca88b docs: REPORT — hub v0.18.0 deployed + live relay validated
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 08:58:06 +02:00
admin 2e749741b8 deploy: hub 0.18.0 (app-email passthrough)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 08:48:08 +02:00