offbox located each toggled app's recovery unit via AppNamespaceRoot→GetAppDrivePath,
which reads the app's LIVE app.yaml HDD_PATH and silently falls back to systemDataPath
when the app isn't deployed → looked on the wrong drive, backed up nothing, reported
ok/0 (DIAG root cause). Now:
- discoverOffboxUnit/offboxCandidateNSRoots scan the durable storage registry
(schedulable non-decommissioned paths ∪ systemDataPath) for backups/primary/<app>,
independent of deploy state; newest-by-manifest-CreatedAt wins on drive churn.
- RunOffboxBackup: runOffboxInternal returns (backedUp, missing, err); 0-of-N toggled →
hard error + operator alert; partial → ok + new OffboxTarget.LastWarning (shown on
/backups, preserved across config edit).
- AppNamespaceRoot + primary WRITE paths unchanged.
- 6 non-hollow tests (A-E + edge) + both companion red-proofs run (reverted).
- NOT yet live-validated against the Storage Box (spike creds torn down).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
loadAndParse + LoadFromBytes ran os.ExpandEnv over the whole YAML before parse;
a bcrypt hash ($2a$10$...) is full of $word sequences that get replaced with
empty env values, silently corrupting web.password_hash on load (a silent
auth-integrity bug: $2a$10$N9qo8uL... -> "a0"). Remove both ExpandEnv calls;
parse raw bytes. The typed applyEnvOverrides path (FELHOM_WEB_PASSWORD_HASH) is
the sanctioned env mechanism and is unchanged. Tests + red-proof.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Re-adjudicates F4: /backup/restore, /backup/tier2/restore, /backup/offbox/restore
blocked the HTTP request until completion, so through cloudflared's 100s cap a
customer got an error page while the restore succeeded (offbox worse — bounded
on r.Context(), canceling the SFTP restore mid-flight). Convert all three to the
offboxRun async shape: fast-path IsRunning refuse, background goroutine
(offbox ctx off r.Context() -> Background+30m), instant redirect. Add mutex-
guarded op-status (opstatus.go) + GET /api/backup/restore-status + a 3s-polling
backups.html banner (neutral running, red on failure). Restore single-flight
unchanged. Tests + red-proof (sync handler blocks indefinitely vs <500ms async).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
EjectDisk/Decommission switched from c.post (drops non-2xx body) to
postWithStatus + shared refusalError, so the agent's informative 403 body
("…decommission refused (role: X)") reaches the operator instead of a bare
"HTTP 403" (campaign F2 evidence gap). Generic post + other callers untouched.
Tests T-D1/T-D2/T-D3 + ok:false case; T-D1 red-proof shows the pre-fix bare
"HTTP 403". Bundles the v0.101.0 CHANGELOG entry (this + the F3 sync deadline).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
List-removal only deletes the registry entry — it's the undo of a
manual path add. On an enrolled drive (/mnt/felhom-drives/) it leaves a
de-registered-but-still-bound limbo nobody wants; the real lifecycle
there is Biztonságos leválasztás / Végleges leszerelés. New
StoragePathView.IsEnrolled gates the button; manual paths keep it; the
decommissioned-branch removal (final cleanup) is unchanged. Endpoint
untouched.
Two Leválasztás buttons rendered per USB drive (registry safe-disconnect
+ the agent eject the D1 merge carried over like-for-like) and four
near-synonymous labels covered very different operations. The agent
eject now hides when the card offers the registry safe-disconnect (one
detach per card; non-USB + unregistered drives keep eject). Labels are
self-documenting with explanatory title tooltips: Új telepítések
letiltása/engedélyezése, Biztonságos leválasztás, Eltávolítás a
listából, Végleges leszerelés, Formázás…. Endpoints and semantics
unchanged.
The .drive-agent-extra slot stacked the agent tag row and the agent
action row with no vertical gap (and sat flush against the registry
action row). The slot is now a flex column with a .6rem gap + top
margin, hidden when empty; enrichCard's inline margin removed.
Both wizards now source candidates from the agent's Impl-2a raw-device scan
(GET /disks/candidates, proxied) instead of the Observe-based /api/disks — so a
brand-new non-PVE-storage drive is finally discoverable + enrollable end-to-end.
agentapi.ListCandidates + a passthrough proxy (no controller-side filtering; the
agent's unclaimed filter is authoritative). storage_init renders `initialize`,
storage_attach renders `attach`; the enroll flow + Impl-1 guarded mkfs unchanged.
Tests + go build/vet/test clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PushResponse.ConfigVersion from the report ACK; ConfigRefresher reconciles vs.
the last-applied version (settings.applied_config_version) and on a change calls
bootstrap.RefreshConfig (re-pull controller.yaml + re-merge local_api) then
GracefulSelfRestart. First-run records baseline (no restart); unchanged = no-op
(no storm); failed pull keeps config + retries. Companion to hub v0.26.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
Encrypted restic repo over SFTP for the app-data tier (the off-site 3-2-1 leg). A dead
NAS fails fast via -oConnectTimeout (spike Q8), never hangs the runner; secrets are 0600
files (ride DR via PBS whole-CT); init-if-absent, retention forget --prune, restore,
single-flight, per-app toggle + UI. restic re-added to the image.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
Controller-side of NAS network storage, proxying to agent A1 /netstorage/*. Distinct
'network' storage kind (no drive lifecycle); add/list/remove + per-share health UI;
unreachable NAS is a recoverable warning, never the drive missing/stop cascade; SMB
creds pass through to the agent, never persisted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
A channel broken at startup/reseed (e.g. controller boots into pin_mismatch) was dashboard-only,
no operator email ever. New 'alerted' flag drives alerting instead of prev=='': born-down
non-transient alerts cycle 1; transient still N>=2; healthy first-obs silent; recovery re-arms.
Red-proof + companion included.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pg8ANF97SEeKYSN5Jxw3qJ
New internal/channelhealth Checker: ~60s probe via the PRODUCTION memoized client
(Server.ProbeAgentChannel, GET /storage), classifies failures (spike Q1 map), debounces transient
reasons (N>=2; construction error latches distinctly), seeds first obs, alerts operator+dashboard on
transition. Notifier.NotifyAgentChannelDown/Recovered (English, operator-only), AlertManager dashboard
banner (Hungarian). No agent/hub change. Spike-proven.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
Gap 1: third shim listener :2526, plaintext, does NOT advertise STARTTLS (TLSConfig
nil) — for opportunistic-STARTTLS clients with no cert-skip (cal.com, nextcloud).
Gap 2: SMTPMapping tls_mode (picks port 2525/2526/2465) + from_domain_var (split
local-part + domain for nextcloud's MAIL_FROM_ADDRESS/MAIL_DOMAIN). Default keeps
existing apps on 2525. Hub untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The controller honors an operator-enforced minimum version (FLOOR) on the hub
report ACK and auto-updates to the floor when below it (managed default, no click),
reusing the Phase 1 in-guest-pull + agent-swap + rollback. Latest stays the opt-in
button; the floor is the auto-target, never latest.
- pusher.go: PushResponse += min_controller_version, latest_version (existing ACK seam)
- main.go: OnPushResponse → updater.SetFloor + MaybeAutoUpdate (rides report cycle)
- updater.go: SetFloor/GetFloor + MaybeAutoUpdate reusing performUpdate (auto-floor);
no-op at/above floor, floor>latest, dev/no-agent/backup; no flap (in-mem+persisted)
- settings UI (HU): floor display + auto restart-poll during an auto-update
- tests: below/at/floor>latest/no-flap/raised-floor; below-floor red-proof verified
- no agent change (reuses Phase 1 POST /controller/swap)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSZmmSFVzGwEzhYmxbkgBK
syncFileBrowserMounts no longer force-recreates FileBrowser unconditionally:
captures config.yaml+compose before writes, re-reads final content after, and
recreates only when they actually changed (new pure helper fbNeedsRecreate).
Controller restarts / no-op storage syncs now issue a plain up -d and do NOT
bounce the customer's file UI. Restore-mode DB reset still forces a recreate.
Dockerfile: removed the unused restic apt package (disk-tier restic moved to the
host agent; no controller code execs the binary). ResticSchedule/migrateResticToRsync
config+settings paths untouched (still live in the dashboard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FpBYrZCt9sFDqLgbG5GRGD
In the intermediary-mount model an enrolled drive binds live into the running
guest (no reboot), so the "… meghajtó aktiválásra vár / Újraindítás most" banner
was an obsolete relic — also dead since v0.78 (pendingActivationDrives keyed on
the raw MountPath vs the now-stable sp.Path). Removed the banner block +
activatePendingDrives JS (settings.html), the PendingDrives feed (handlers.go),
and the dead pendingActivationDrives helper + its unused internal/system import.
Renamed handleStorageActivate -> HandleServerReboot (split out a testable
serverReboot core), removed the /api/storage/activate case, and mounted the
handler at the new non-storage route /api/server/reboot (RequireAuth+CsrfProtect).
The agent GuestReboot primitive is reused unchanged.
Added the standalone "Kiszolgáló újraindítása" settings card (sibling to the
controller-only "Vezérlő újraindítása"), reusing the pollRestart() loop.
Test: TestHandleServerReboot_CallsGuestReboot (fake diskAgent asserts GuestReboot
invoked once + 202). diskAgent/mockAgent gained GuestReboot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PsnU2ASocYrvzqE82YDYW
The storage card displayed each external drive's raw host PVE mount (/mnt/<name>,
which doesn't exist in the guest) instead of the stable in-guest path
(/mnt/felhom-drives/<name> = guest_path) the registry/HDD_PATH/FileBrowser use.
The eject/wipe buttons also posted the raw path, so they would unmount the drive
but leave the stable registry entry orphaned, and the impact warning found no apps.
Fix: card sub-line + eject/wipe buttons use the stable path (regKey); type-to-confirm
name uses the basename; register keeps the raw path. handleStorageWipe maps to raw
via agentWhere() for the agent eject (matching handleStorageEject). Agent ops
unchanged (same raw paths); display + registry bookkeeping corrected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PsnU2ASocYrvzqE82YDYW
Follow-up to v0.78.0. The disk-view JS (settings.html regBadge/actions) checked
registration by the raw mount_path, but the registry stores the stable path since
v0.78.0 — so enrolled drives showed a spurious "Nem regisztrált" badge + Register
button. Fix: regKey(d) = d.guest_path || d.mount_path (agent reports the stable
guest_path per disk); registerDrive still posts the raw path. Display-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PsnU2ASocYrvzqE82YDYW
handleStorageRegister (the "Regisztrálás" action for an already-mounted,
unregistered drive) registered the raw /mnt/<name> path verbatim, unlike
runStorageInit/runStorageAttach which map to the stable /mnt/felhom-drives/<name>
path the agent actually binds the drive at. The controller then watched an empty
placeholder dir on the guest rootfs → "Rendszermeghajtón" + stuck "activation
pending" banner after a re-provision.
Fix: register stablePathForName(path.Base(req.Where)); attachIntoGuest still uses
the raw path (the agent operates on raw). Test + red-proof added.
Diagnosis: felhom.eu/documentation/audits/DIAGNOSE-drive-bind-after-reprovision-2026-06-23.md
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PsnU2ASocYrvzqE82YDYW
Belt (ensureUserdataMounts) + FileBrowser sync skip ensure/mount when an external
drive root is not a live mountpoint -> no 'mkdir userdata: permission denied' + no
rootfs-shadow during a drive-absent window. System/local path never gated. Reuses
system.IsMountPoint; matches planDriveGates external-only rule. T1-T4 + red-proofs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
agentClient() built a new agentapi.Client (new bare http.Transport, IdleConnTimeout:0)
per call and discarded it without closing idle conns -> one leaked idle ESTABLISHED
socket per call to the agent :8443, exhausting the ephemeral port range after ~5 days
(EADDRNOTAVAIL). Memoize one shared client via sync.Once; harden Transport
(MaxIdleConns/PerHost + IdleConnTimeout 90s). Agent/firewall untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DR recipe slice (controller half), grounded in SPIKE-dr-recipe-2026-06-16. The
controller emitter is the BOUNDARY enforcement point: v1 ships an explicit
allowlist {catalog_ref, enabled, storage_bindings} and reads NOTHING from
AppConfig.Env, so no ENC:/token/password can leak. storage_bindings parsed from
the compose (${HDD_PATH}/${USERDATA_PATH} volume binds -> {container_path,
drive, subpath}).
Load-bearing tests: TestBuildAppRecipe_NoSecrets (synthetic-secret app -> none
leak) + TestBuildAppRecipe_AllowlistIsLoadBearing (red-proof companion) +
TestAppStorageBindings + TestBuildDRRecipeAppHalf. Red-proofed live: forcing the
emitter to dump Env makes the boundary test fail. recipe_version=1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
processGuestBootChange recreated the drive-backed app stacks but never re-synced
FileBrowser (base-infra, no HDD_PATH), so its drive mounts went stale after a
reboot. Now, AFTER pollLiveBinds confirms the live binds and the apps are
recreated, trigger go s.SyncFileBrowserMounts() so FileBrowser converges against
the now-live drives. Refactored into pure recreateDriveBackedApps(stacks, present,
recreate, syncFB). Tests: FB sync runs once after recreate (red-proofed companion);
runs even when nothing recreated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-drilled the HOST reboot (felhom-pve) x2 after the follow-up challenge: both
recover all 8 drive-backed apps automatically; host-btime prefix advances so the
host path triggers processGuestBootChange; the boot-race manifests on host reboots
too (not just guest). Corrected the REPORT's 'host path unaffected' claim.
Regression analysis: the v0.68 sweep genuinely exercised the recovery (it surfaced
a real state-filter bug on the host reboot), so the agent path worked then and
regressed after (controller.yaml reset to golden no-local_api baseline on each
recreate; old MaybeIngest never re-merged). v0.70.0 config round-trip exonerated
(GET /api/config is verbatim). ensureLocalAPI closes the regression on both paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live diagnosis of drive-backed apps stuck Exited after a pct reboot pinned THREE
sub-causes, fixed together (hardening the existing processGuestBootChange, not a
parallel mechanism):
1. Agent-path blocker (live root cause): agentClient() returned "agent not
configured" (cfg.LocalAPI.Endpoint empty), so processGuestBootChange AND the
whole drive gate bailed at the first guard. bootstrap.json had a complete
local_api block, but MaybeIngest returned immediately on "already configured"
so a controller.yaml seeded before local_api existed never got the agent path.
Fix: MaybeIngest now calls ensureLocalAPI on the already-configured path,
merging local_api from bootstrap.json into the existing controller.yaml when
missing (no hub re-pull, config preserved; idempotent + fail-safe).
2. Boot-race readiness gate: processGuestBootChange sampled BoundUnderParent once
during fast startup, racing the ~18s rebind, recreated nothing, burned its
boot-id one-shot. Fix: gate on the REAL live in-guest bind -- driveBindLive
checks /mnt/felhom-drives/<drive> is a mountpoint in the controller's own /mnt
rslave /proc/self/mountinfo; pollLiveBinds waits for it (bounded ~120s) before
recreating via the normal pipeline. shouldRecreateOnBoot stays state-independent
so stuck-Exited create-time-failure apps are included.
3. Single-shot fragility: processGuestBootChange ran only once at startup; a
briefly-unreachable agent right after a guest reboot stranded recovery. Fix:
driveGateLoop runs it every periodic tick too (idempotent, boot-id gated).
Tests (non-hollow, pre-fix companions, red-proofed): pollLiveBinds waits then
reports live / never-live stays absent / single early sample misses; ensureLocalAPI
merges local_api into a configured controller.yaml that lacks it / no-ops when
present. Live-accepted with repeated pct reboot 9201.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sub-cause: on guest pct reboot, in-guest dockerd auto-starts unless-stopped
drive-backed apps ~18s BEFORE the agent re-binds the drive; the create-time
volume bind fails (mkdir /mnt/felhom-drives/<drive>/userdata: permission denied)
and RestartCount=0 means it's never retried -> stuck Exited. The existing
recovery (processGuestBootChange) RAN but raced the rebind: it sampled the
agent's BoundUnderParent once during fast startup (not live yet), recreated
nothing, and persisted the new boot-id -> burned its one-shot. The periodic gate
never recovered them either (first observation after the rebind -> no transition).
Fix (harden the existing mechanism, no parallel one): processGuestBootChange now
gates on the REAL live in-guest bind. driveBindLive checks whether
/mnt/felhom-drives/<drive> is an actual mountpoint in the controller's own /mnt
(rslave) /proc/self/mountinfo -- true only once the agent's bind propagated,
exactly when docker can recreate the app. pollLiveBinds waits for that (bounded
~120s, poll 2s; rebind lands ~18s) and only then recreates via the normal
pipeline, including stuck-Exited create-time-failure apps (shouldRecreateOnBoot
is state-independent). Single-flight; absent-after-window drives left to the gate;
host-reboot path unaffected; guest-only reboot path now covered.
Tests: pollLiveBinds waits through the rebind then reports live (recreate fires);
never-live drive stays absent; pre-fix companion (single early sample misses the
not-yet-live bind). Red-proofed against a no-wait single-sample.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Controller half of Phase-1 Infra Backup retirement (hub v0.12.0;
SPIKE-infra-backup-2026-06-15). Pure dead-code removal, no behaviour change.
- Remove Pusher.PushInfraBackup (caller-less; hub endpoint gone).
- Remove Notifier.NotifyBackupCompleted / backup_completed event (caller-less
since slice 8C; hub deadline check now reads agent host-report PBS snapshots).
- Remove report.BackupReport.ResticPassword — builder never sets it post-8C
(confirmed in source + live), but it historically leaked the restic password
into the hub's plaintext reports store.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The migration journal keeps returning the last completed job; the resume-view
watched any job and migWatch's done-branch reloads the page -> endless reload
loop after any migration. Resume-view now watches only in-progress jobs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
.stack-title-row gains flex:1+min-width:0 so it shrinks/wraps; .stack-state-badge
gains flex-shrink:0 so the nowrap badge is never compressed. Only unhealthy cards
clipped (the route-unpublished warning inflated the title-row).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
E1 caught it: filtering on State!=stopped missed apps docker hadn't auto-restarted
yet at the one-shot instant (5 apps exited after host reboot). Now recreates every
deployed present drive-backed app regardless of state (deployed=should run).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
admin