Read-only spike on felhom-pve. Verdicts: (SQ1) customer drives need NOT be PVE
storages — track in the intent registry (durable-id + host mount + bind), the
dir-storage is legacy. (SQ2) an unclaimed-disk filter (OS/LVM-PV/ZFS/PVE-storage/
VM-CT/mounted/RO/registry signals, fail-safe exclude-on-unknown) returns EXACTLY
{/dev/sdd} on the demo, never sda/sdb/sdc. (SQ3) mkfs sudoers is /dev/* — the
agent code is the only guard; DataBearing (wipe-confirm) is insufficient (OS disk
is data-bearing), so a mandatory agent unclaimed-gate + guarded-mkfs-wrapper
sudoers + a shared-box operator gate for self-serve format. (SQ4) new
/disks/candidates raw-scan endpoint feeding both wizards, registry-based
enrollment. (SQ5) migrate existing drives to the registry. Implementation plan +
open risks included. No product code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live supervised validation on felhom-pve: NO 403 on drive management under the
scoped token (agent has no storage-registration/pvesm path — all host-ops +
Datastore.Audit which is now box-wide). 3b-fix confirmed in the real UI (drives
Regisztrálva, no detach alert). A full physical wizard enroll couldn't complete
because a raw new device (SD-card reader, no PVE-storage entry) doesn't surface
in the init wizard (Observe() lists PVE storages only) — ACL-independent; flagged
as a separate new-drive-registration follow-up. No code change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Throwaway-token spike on felhom-pve: the agent's 16 privs scope from / to
/pool/felhom + /storage/<targets> (+Pool.Allocate) without breaking the flow.
CRUX proven: restore a fresh vmid INTO the pool works at /pool/felhom (T1 200).
T1-T7 all pass; blast radius contained (out-of-pool guests 403). Findings:
3-role layout (per-path subsets need separate roles); Datastore.Audit filters
so needn't be box-wide; Store scope MUST include felhom-pbs (offsite default);
restore-test scratch band (9990xx) not in pool → 3b must handle. All spike
objects cleaned up; live FelhomAgent/9201 untouched. No product code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Categorized STALE/DEAD/MISSING/POLISH/CORRECTNESS findings for the hub web
GUI. No GUI code changed, no version bump — findings doc only, for operator
prioritization.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
Part A: hub-config operator_email/operator_enabled (committed earlier);
operator + customer email paths proven end-to-end via real Resend sends.
Append the "Findings closed" section to the TESTRUN audit (Part A proof,
F2-a root cause + deploy fix, F2-b fix + live validation, pending §D
supervised reboot) and remove the stale duplicate "Final state" block.
REPORT overwritten. hub/CHANGELOG entry. No hub image change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0162BnMpUXscPsUB1cU8Tr6K
Phase 1: capability->hub operator alert fired live (event captured, English, names cap+feature);
email is operator-config-gated in demo (not a defect). Phase 2: drive re-propagation + zero false
pages PASS, but TWO findings — F2-a the §4 doubling did NOT self-correct (boot script runs as root,
never needed the grant → hypothesis falsified), F2-b a vzdump backup in flight at reboot left a stale
snapshot-delete lock blocking guest autostart. Phase 3 referenced (host_leaf_changed proven on v0.22.1).
Final parity known-good.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
Q1: 6 modes classified — pin-mismatch/401/refused/timeout/no-such-host are RUNTIME (per-call),
malformed-fp is a CONSTRUCTION error (latches via sync.Once). Q2: across a clean agent restart the
memoized /api/disks path self-heals in ~1s with NO controller restart (Up 7h); a fresh-client probe
agrees at every phase. Recommend: probe GET /storage via the memoized client, classify, N>=2 debounce
for transient reasons, hub relay. Throwaway probe removed, no config changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
Empirical validation on a DooPlex-simulated NAS (SMB+NFS). Verdict READY for the
production network-storage TASK. Key findings: mount must be host-side (unprivileged
LXC blocks NFS/CIFS mount) + bind-propagated into the guest; NFS soft fails-clean +
auto-recovers (SMB hangs-but-contained); UID mapping = container uid + 100000 LXC
offset; ~128MB/s adequate; write integrity intact (atomic-write apps safe on soft);
restic-SFTP backup/restore works + fails-as-job not hang; systemd automount gives
on-demand + idle-unmount. Distinct network-storage class bypasses the drive lifecycle.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verdict READY. Real Vaultwarden test email travelled app → on-box go-smtp
shim → relay-leg → Resend → real inbox, From felhom.eu, TLS/DKIM-aligned,
Resend key never on the demo guest. Q4 central finding: raw-MIME passthrough
via Resend SMTP delivers faithfully; parse-then-API silently drops inline
(CID) images — hub leg should be SMTP passthrough, not the structured-API path.
Probe code throwaway (not committed); cleaned up from guest 9201 + host 180.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NB4fVgvEurdKgyG8KoQSzC
Operator-run Day-0 automation for a fresh PVE host: pveum token -> hub
POST /host-enroll (single secret, option C) -> agent config -> felhom-agent
--selftest=provision -> verify. Idempotent/resumable (state.json), --dry-run,
--resume, --force, --preserve-from; --mode dr is a documented 10D stub.
Live-validated on felhom-pve: authorized wipe of demo guest 9201 + full
re-provision from the golden -> controller config-pull + public tunnel HTTP 200
+ host-report of 9201; idempotent --resume no-op. Found+fixed one ordering bug
during the run (token ACL must be applied AFTER token rotation — token-remove
purges the ACL).
Adds scripts/README.md (operator runbook) + scripts/CHANGELOG.md; REPORT
overwritten; spike doc pointer added. No service code changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtXesNa2LGbMmE4DNL6SE7
New POST /api/v1/host-enroll (handleHostEnroll): X-Retrieval-Password authed,
body {customer_id} -> {host_id, api_key}. Mint-once-reuse (201 first, 200
reuse) so re-running the host-bootstrap never orphans a running agent's key;
auth checked before any mint. Backed by new Store.GetHostByCustomer
(ORDER BY updated_at DESC LIMIT 1, idx_hosts_customer).
GET /config/{id} and global-key POST /admin/hosts left untouched. Exact-match
route (path == "/host-enroll") to avoid the /hosts/ prefix collision.
Tests: host_enroll_test.go (mint/reuse/401-no-mint/404/400) + GetHostByCustomer
store test; companion red-proof verified always-mint fails the reuse assertion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtXesNa2LGbMmE4DNL6SE7
Empirically validates the composed Day-0 chain against the LIVE hub: host
mint (POST /admin/hosts), agent host-auth via the real internal/hub client
(host-report + FetchDesiredState), and controller config-pull
(GET /config/{id} + X-Retrieval-Password). All links + wrong-cases pass.
Option A works end-to-end with zero new code; option C (single-secret enroll)
feasible + recommended for cutover. Throwaway customer/host cleaned up.
No production code, no version bump (spike).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtXesNa2LGbMmE4DNL6SE7
Root cause: controller handleStorageRegister registers the raw /mnt/<name> path
instead of the stable /mnt/felhom-drives/<name> path the agent binds the drive at.
Drives ARE live in guest+container at the stable path; controller watches an empty
rootfs placeholder. Not v0.40.0 mp1; not stale agent state. Read-only diagnosis,
no fix applied. Recommended fix direction documented.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PsnU2ASocYrvzqE82YDYW
Part 0 (live): flash apps on 9201 were down due to an operator pct reboot at
10:26 UTC + a boot-ordering race — dockerd auto-starts unless-stopped flash apps
~18s before the agent re-binds felhom-flash, so the create-time bind mkdir fails
(permission denied) and RestartCount=0 never retries. Drive healthy, data intact,
no USB drop, durable-id fine, drive-gate uninvolved. v0.70.0 self-restart RULED
OUT (container restart, not a guest reboot; +38min after exits). Fix: restarted
the 7 apps via the controller (drive present) — all Up. Flagged the intermediary
mount app-start race as an architectural gap.
Parts 1-3 (cited): characterized escrow (K + identity under recovery code R,
fingerprint-gated, hub zero-knowledge) + PBS whole-CT contents (rootfs/secrets in,
external drives out) + capstone DONE vs PENDING (agent-side recovery orchestration
not wired, syncer.go:92). Defined the secret-free DR recipe (guest sizing + drive
durable-id/role inventory + PVE storage + app bindings + PBS coords), sourced from
facts the agent/controller already hold, landing in the reserved
WireDesiredState.storage_manifest placeholder. Field-by-field boundary proof + a
no-secrets test spec. Fork list + recommendation: spec/emit/store the recipe now,
defer re-enrollment auth to slice 10D, never touch the escrow/PBS secret path.
No code changes, no version bump.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Report-only audit. Confirms the infra-backup push died at slice 8C
(controller v0.37.0) — builder deleted, PushInfraBackup +
NotifyBackupCompleted left as caller-less stubs; agent never
re-implemented it. Same root cause as the daily deadline-check email
(no backup_completed emitter). Live hub records inspected read-only:
plaintext secrets at rest (AES master key + restic pw + Cloudflare
tokens) = zero-knowledge violation; disk_layout is bare-metal-era.
Recommends phased retire-now / revive-concept-later (secret-free DR
recipe) + fork list. No code changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live-validated on demo guest 9201 (felhom-flash throwaway): shared/slave mount
propagation delivers host->guest drive swaps live with no restart and no pct
change; bind source = permanent stable dir => inherently C1-immune (clean boot
with drive absent). nsenter fallback non-viable on unprivileged LXC; ro-tmpfs
placeholder is the CAP_DAC_OVERRIDE-proof fail-close. Report-only, no code change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live unattended UI exercise of migration/decommission/re-enroll/eject +
feature sweep on shipped controller v0.66.2 / agent v0.32.0 (guest 9201).
No production code changed.
Migration engine solid (all moves checksum-verified identical, sources
cleaned, HDD_PATH flipped, apps healthy). Lifecycle gaps found:
- C1 (critical): decommission/eject leave a stale LXC mp (no pct delete)
-> guest fails pre-start hook on next reboot, ALL apps down. Recovered
live via pct set --delete mp1.
- H1: /api/storage/{disconnect,reconnect,restart-apps} are 404 (unrouted)
though settings.js calls them.
- H2: no UI button for /api/storage/decommission.
- H3: no one-click re-enroll for a decommissioned drive.
- M1: decommissioning the default drive doesn't reassign the default.
- M2: eject does NOT auto-remount (intent-gated like decommission).
- M3: userdata/import/calibre created 755 at enroll; migrate-all doesn't
re-assert setgid on pre-existing dirs (16/17 preserved 2775).
- M4: felhom-usb mount unit had stale UUID; M6: test-email fails (no SMTP).
End-state sane: apps healthy on felhom-flash (now default), both data
drives Aktiv, reboot-safe, sda/PBS/controller untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Report-only spike + one live throwaway FORMAT experiment on /dev/sdc.
No production code, no version bump.
- Q1: raw-disk discovery gap confirmed — Observe AND localapi GET /disks
are both built only from PVE storages (ListStorage/NodeStorage); a
not-yet-storage disk is invisible. proxmox client has no /disks/* method.
- Q3: safety classifier (RoleForRawDevice/SystemDisks/RoleForStorage)
already protects sda/felhom-pbs live, but needs two hardening checks:
whole-disk-granularity enrolled-subtraction + LVM-PV membership.
- Q6: FORMAT path proven end-to-end live — pvesh disks/directory create
-> agent Observe -> controller "Regisztralas" enroll. sdc left as the
clean enrolled felhom-flash B3 migration target. sda/sdb untouched.
- Premise correction: felhom-pbs is remote (no local PBS disk).
- Fork list for the build spec in the audit doc + REPORT.md.
Maps current namespace dir creation/ownership (live on 9201), the 9-app/13-mount
media blast radius, run-identity table (FileBrowser=uid1000, komga/audiobookshelf=root),
LXC idmap, and proposes the setgid GID-1000 convention. Flags: MigrateAll's merge walk
drops setgid+owner on userdata. Ends with a 7-item fork list. No code/deploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Relocates the 2026-06-14 live-drive findings + fixspec from the felhom-controller
repo root into documentation/audits/ (alongside the bughunt-reconcile/deep-sweep
records), and preserves the M18/M19 implementable fix-plans (from the deleted
controller fix/m18 + fix/m19 branches) into a new documentation/backlog/. Part of
the trunk-based no-branches reconciliation.
Brings the 2026-06-13 audit + reconciliation under documentation/audits/ with
per-finding remediation status (controller v0.59.0 fixes deployed to demo;
AGENT-001 fix prepared/pending review).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
admin