felhom-controller/CONTEXT.md

CONTEXT.md — Project Memory

This file serves as persistent project memory across Claude Code sessions. It replaces the auto-generated "Memory" from the claude.ai Project. Update this file at the end of each working session with current state, recent decisions, and anything the next session needs to know.

Ask Claude Code: "Please update CONTEXT.md with what we did today"

Last updated: 2026-07-20 (v0.150.0 — green gate restored + F7 export-link fix)

2026-07-20 — v0.150.0. go test ./... on DooPlex is fully green again (23/23 packages, run twice) — no surviving t.Skips, no weakened assertions, no deleted tests. The 7 red TestTier2V2_* / TestSharesTier2* were all one environmental class: Tier-2's off-drive guard uses system.SamePhysicalDevice (st_dev equality), and every t.TempDir() here shares one filesystem, so the fixture's "two drives" looked identical and the guard correctly refused the target (nincs másik fizikai meghajtó). Fixed with a nil-defaulted Manager.samePhysicalDevice seam + sameDevice wrapper — production behaviour is byte-for-byte unchanged (nil → the real check); only the two fixtures inject a fake. All 7 mutation-proved. F7/R-53 shipped: app_export.html built the app URL from {{$.CSRFToken}}; now {{$.Domain}}, with exportPageHandler supplying the key (it bypasses baseData). Host: the orphaned dhclient on the non-existent eth0 was killed and did not respawn. Still open from the arc: R-50 (durable F1, spike-first — its ROADMAP note about needing a new cert SAN was corrected: the pin is a raw leaf-DER SHA-256 with InsecureSkipVerify, so SAN never enters it), R-51, R-52, R-39(b)/F6.

2026-07-20 — remote-site remediation + v0.149.0. F1 is MITIGATED FOR THE WINDOW, not durably fixed: vmbr0 on the demo host is now static 192.168.0.162/24 (was DHCP; the remote router had handed it .147, and the agent binds that literal), applied with ifreload -a; the agent came up clean and the red „a tárolókezelő ügynök nem elérhető" banner is gone. The control plane is still pinned to a LAN literal — the durable fix (host-internal island bridge) is a separate spike-first arc, R-50. Restoring the agent immediately let the quiesce loop run the overdue whole-guest backup by itself (F2 closed), a manual app-data run followed (2 DBs, 3 volume dumps, 43 s), and Immich is back (photos.demo-felhom.eu → 200; it had been left Exited by the pre-transport shutdown, not by the offsite-restore test). v0.149.0 fixes F3 — the dashboard card said „Utolsó mentés: Még nem futott" on every box because dashboardHandler never passed the BackupStatus key the template branches on. F4/F5/F6/F7 are roadmap-only (R-51/R-52/R-53/R-54). Evidence: felhom.eu/documentation/audits/AUDIT-vacation-remote-ops-2026-07-20.md §Remediation.

2026-07-20 — demo box moved to a remote site until ~2026-08-02; ssh felhom-pve = tailnet 100.70.170.35 (direct, ~37 ms). THE HOST AGENT IS DOWN THERE: its localapi binds the literal 192.168.0.162, the host now DHCPs 192.168.0.147 → bind: cannot assign requested address, so the service has never started at the remote site and the controller's agentapi dials of the same literal get no route to host — that is the whole "A tárolókezelő ügynök nem elérhető" banner, and it kills storage/PBS-backup/quiesce/restore-test/DR until fixed (Viktor GO: config and guest bootstrap state). Calibre-Web and immich-server were left Exited by the pre-transport shutdown and never came back despite unless-stopped; Calibre-Web was restarted via the real UI endpoint, Immich deliberately left down. Two code defects found and NOT fixed: the dashboard's "Utolsó mentés: Még nem futott" is a display bug (dashboardHandler never sets BackupStatus, so dashboard.html:116's {{if}} branch is unreachable — it renders on every box regardless of history), and multi-container apps under-alert (IsDownState excludes unhealthy, so Immich's dead primary container produced no banner and no app_start_failed event for 18 h). Full evidence + ranked findings: felhom.eu/documentation/audits/AUDIT-vacation-remote-ops-2026-07-20.md

2026-07-19 — v0.148.0: R-43 + R-44. Viktor deleted 11 immich photos to test offsite restore; both runs flashed success and the photos stayed gone (DIAG-immich-restore-2026-07-19). Two defects. R-43: no offsite path could restore a DATABASE — all three buttons were file-only, so a DB-indexed app got its bytes back and still could not see them. R-44: a manual push shipped whatever dump the 02:30 local run left; that day's predated the customer's account by four hours and held zero users and zero assets inside 52MB of shipped geodata.

Now: every run (manual AND nightly) dumps FIRST, then captures, so each snapshot is a coherent {DB@T, files@T} pair stamped with offsite_run_id; and „Teljes visszaállítás (fájlok + adatbázis)" does safety-dump → stop → overwrite files → start → replay the snapshot's dump. Two invariants: nothing is ever deleted, and the undo is verified on disk before the act. Warn-level honesty surfaces for stale/empty-looking pairs — never gates.

The live acceptance has NOT run yet (§9: upload → push → empty trash for real → one button → photos back). Until it does, the capability-map offsite row stays PARTIAL/scope-contested, the customer-restore row stays MISSING, and R-3 stays DRAFT. Floor raise = Viktor's click, BEFORE the acceptance run. Still open: the 00-capability-map.md:61 ruling — did CAMPAIGN-6D's "immich end-to-end from offsite alone" exercise the DB half, or only the file half?

2026-07-19 — v0.147.0 → v0.147.3: feedback slice 1. The systemic complaint, twice in one evening: you press a button and nothing happens. Three worst offenders fixed on the two patterns that already existed (deploy 3-step panel; storage-init status poll). Deliberately NOT a framework — that is ROADMAP R-45, and the two lessons it must encode are already written down there: a terminal state must be probed, not inferred (compose up -d exits 0 on a crash-loop), and a progress source reporting nothing is normal, not broken (restic reports 0 bytes for a whole incremental run).

4a — the offsite verification restore names its full path in the flash, and /backups/restore lists existing verification copies (app · size · date · path) with a double-confirmed per-copy delete. That delete takes a stack name, never a path; red-proofed (neutralise the name guard and stack:"" resolves to the offsite-restore ROOT and takes every copy with it). backups/offsite-restore now has ONE home, offsiteRestoreRootFor. 4b — Megosztás enable/password no longer reconcile inside the POST; detached job + poll, with „képfájl letöltése" vs „indítás" decided BEFORE the work starts (afterwards the image is always present and the distinction is unrecoverable). 4c — „Távoli mentés most" streams restic --json. Manual only; the nightly stays silent, pinned by a test.

Three of the four versions exist because the cards were watched against real runs on the demo box — none of these would have surfaced from unit tests: (.1) an incremental run reports 0 bytes for its whole duration, so a byte-only bar looks hung in the COMMON case; (.2) restic 0.14 counts a file only when it completes, so one big archive freezes the file counters too — fall back to current file + elapsed; (.3) the run does not end with the last app — the shares leg and forget --prune took 40 of a 57-second run, and the card froze on the last app until phases were added.

Also: infra.Images() + --print-infra-images close the golden/controller infra-image drift at the source. The golden's own copy had already drifted (felhom-samba missing → 3 of 4 baked), which is why enabling Megosztás pulled at runtime in the first place. Effective at the next bake (felhom-agent build-golden v2.1.0); no golden rebuilt. Floor NOT raised — Viktor decides.

Earlier: 2026-07-18 (v0.145.0 — R-7b: share data enters the live backup runs, Model B′)

2026-07-18 — v0.145.0: R-7b — the „Felhőmentés" toggle is now TRUE (Model B′), + samba liveness. Until v0.144.0 a share could be marked „Felhőmentés: bekapcsolva" while its files were in NO backup: both engines are recovery-unit shaped (RunTier2 short-circuits on os.Stat(unitDir); the offsite runner enumerates GetOffboxApps()) and a share-only infra stack has neither. Viktor's ruling was Model B′: a SIBLING shares source — additive job/leg code reusing the proven primitives (tier-2 mirror seam, restic wrappers, quota gate, status recorders) while every per-app engine path stays byte-identical. That invariant is enforced by test in both tiers, red-proofed.

Tier 2 → RunSharesTier2 (legs grouped by SOURCE drive → backups/secondary/_shares/<driveKey>/…, payload at _payload/, marker LAST). Tier 3 → runOffboxSharesLeg, ONE extra restic call tagged _shares, placed after the app loop and BEFORE retention so --group-by host,tags covers it for free. Restore → „Megosztások" on /backups/restore: scratch, then a missing-only merge whose every destination is PREFIX-ASSERTED against live storage roots; definitions merge existing-wins; then ReconcileSamba; then the credential, best-effort.

The payload is the point: a byte-deterministic _shares-manifest.json + a best-effort secret-bearing passdb.tar, so DR returns files + configuration + password, not loose bytes. A quota-blocked offsite push degrades to the MANIFEST ONLY — never to nothing.

Findings: (a) the reserved-name assumption was FALSE — nbNameRe accepted „_shares" as a share name; ValidateSMBShareName now refuses a leading underscore and both run loops skip a _shares stack loudly. (b) The samba-liveness fold-in needed NO alert/e-mail pipeline change and adds no new event type (so the allowedEventTypes gotcha does not apply) — EffectiveProtected just gains a settings-backed dynamic extra watching the CONTAINER name (felhom-samba ≠ stack name samba). (c) A real bug surfaced in review-by-test: shareSourceDrive returned a slash-normalised path, making the target selector's equality check miss so a share group could target its own source drive.

OPEN: Viktor raises the managed-update floor to v0.145.0 (supersedes the 0.144 note) so the N100 rehearsal's day-0 box converges onto the honest version. Pre-existing, untouched: docker_run_volume_path_gate fails on internal/appexport/estimate.go (predates this work).

2026-07-18 — v0.144.0: „Megosztás" LAN SMB sharing (R-7 slice 1), LIVE on demo. SMB ships as an embedded controller feature — the FOURTH protected infra stack (traefik/cloudflared/filebrowser/ samba), NOT a catalog app (it needs network_mode: host per the R-6 spike, its config is a generated share list, and its roots ride the backup classification). New own image felhom-samba:1.0.0 (pinned alpine + smbd + nmbd + wsdd + tini; smb.conf bind-mounted read-only, nothing templated inside, passdb on a volume). nmbd is REQUIRED alongside wsdd — the R-6 spike proved wsdd-only leaves the box visible but the Explorer double-click fails 0x80070035. New top-nav category „Megosztás" → „Hálózati megosztás": enable + one household SMB password (STDIN→smbpasswd, NEVER persisted — only user_set), shares table, and a create flow (new folder under <storage>/shares/ or an existing folder via a guarded browse modal). Every customer path goes through sharingResolvePath (absolute → EvalSymlinks → containment in a registered LIVE storage root → deny-listed system subtree → is-a-dir) with uniform refusals so the picker is never a filesystem oracle; the deny-list is DERIVED from ProtectedHDDPaths (provably a subset). ClassifiedBinds("samba") resolves from the shares registry: Felhőmentés ON → mandatory (offsite+tier-2), OFF → optional (tier-2 only), with ZERO backup-engine edits. OPEN / needs a Viktor ruling (suggested R-7b): the classification seam is correct but share data is not in any live backup run — RunTier2 short-circuits on the missing recovery unit before it reaches the seam, and the offsite runner enumerates GetOffboxApps(). Both engines are recovery-unit shaped; teaching them about a share-only stack is a structural change, so it was reported as a design fork rather than improvised (task STOP clause). Live-validated end-to-end through the real endpoints

• a Windows 11 workstation (445, NetBIOS FELHOM resolves, write/read byte-compare PASS, write to a read-only share REFUSED with no effect, SMB writes land as uid 1000). Explorer leg PASSED (Viktor): both shares open, an interactive Explorer save landed as uid 1000, a write into the read-only share was refused. Slice 1 fully PROVEN-LIVE. Also: samba is protected in CODE (config.alwaysProtectedStacks) because controller.yaml is golden-generated — a side effect is that monitor.EffectiveProtected does NOT monitor samba liveness (deliberate: no false alarms while off; see REPORT §9).

2026-07-17 — v0.143.0: guest RAM resize UI (R-24), LIVE on demo. MinAgent: 0.90.0. The customer sees the guest's current memory + allowed range on the Rendszer settings page and resizes it ("Szerver memória (RAM)" card). The controller proxies + maps the agent's machine code to Hungarian; the AGENT (felhom-agent v0.90.0) enforces every bound and applies live (no reboot — R-24's old hub-desired-state framing is SUPERSEDED by Viktor's controller-direct ruling). agentapi GuestMemory/ResizeMemory (*MemoryRefusedError carries the code); capability FeatureGuestMemoryResize (featureMinAgent 0.90.0, probe type-asserts GuestMemory so the shared SupportProber/netAgent stay untouched); POST /api/system/memory/resize; JS confirm only on a shrink; agent-outdated hides the control, agent-unreachable falls back to the guest's /proc/meminfo. Deployed to BOTH demo guests (felhom-pve 9201 + nested demo-vm-felhom-4846bc 9201). LIVE-validated through the real endpoint on the nested demo (above_max + below_min refusals render the Hungarian message; the agent English never leaks; the capability gate resolves SupportYes via the 0.90.0 version header). A successful grow couldn't be shown on the tiny 4 GB nested host (max<current, correctly refused); the apply is Phase-0-proven at the agent layer. See REPORT.md.

2026-07-17 — v0.142.0: offsite repo continuity (Parts A + C), LIVE on demo. Closes the reinstall-orphaned-repo incident (a recreated data volume mints a new repo passphrase → the offsite repo, keyed under the old one, errors nightly with wrong password or no key found). Part A: ensureOffboxRepo classifies the cat config failure → ORPHANED state + calm Hungarian card (exception color) + offbox_repo_orphaned event (once, not nightly); reset = move-aside (never delete, mv <repo> <repo>.orphaned-<date>) + init — UNCLAIMED auto, CLAIMED reveal-then-confirm. Part C: GET /backup/offbox/status + poll on backups_remote flips Fut→Rendben/Hiba without a manual reload. Pairs with hub v0.60.0 (superseded-escrow retention). Live leg staged for the rehearsal (scratch-target swap would disturb the live escrow state; live repo untouchable) — mechanism covered by 3 fake-based scenarios + 2 red-proofs. Details: REPORT.md.

2026-07-17 — v0.141.0: N100 polish (F6 + F7), LIVE on demo. F6 (MEDIUM): drive "initialize" now ends in a mounted+registered drive even on a client disconnect. POST /api/storage/init runs the format→mount→register chain as a DETACHED single-flight job (web/storage_init_job.go, context.Background(), netAddState shape) the wizard polls via GET /api/storage/init/status (3-step Hungarian progress); register is the last step (marker-last crash-safety). Fork verdict: controller-side, NO agent change (the chain must reach FileBrowser sync = controller-only). Deeper half found on the live leg: a slow mkfs (64 GB USB, ~27 s) outruns the agentapi client's 15 s timeout → the controller now polls the agent's GET /disks/format/status (agentapi.FormatStatus → awaitAgentFormat) then continues. Live-validated on /dev/sdd → /mnt/felhom-drives/scratch1 (mounted+registered). F7 (LOW): storage init/attach Vissza → /storage. Red-proofs for both F6 halves + F7. Security review of the commit flagged the pre-existing format→resolve→assign device-node TOCTOU (agent-guarded destructive step, benign fs-UUID mount) — acknowledged as an Observation, not expanded. Fork/landmarks/live evidence: REPORT.md.

2026-07-16 — v0.140.0: Direction-2 immediate-sync (hub→box) SHIPPED. The reverse of v0.139.0: an OPERATOR action on the hub now reaches the box in seconds. report.Waiter (internal/report/waiter.go) holds a hanging GET {hub}/api/v1/wait?gen=N (same hub URL+key as the pusher — no new config keys) against hub ≥ v0.58.0's in-memory operator-intent generation counter; on a generation CHANGE it fires the v0.139.0 report.Trigger and nothing else, so the report ACK delivers everything through the UNCHANGED machinery (the box pulls even the wake-up). No overall client timeout (held GET); first-observation records-not-fires (no restart echo); same-gen timeout fires nothing; errors incl. a 404 from a pre-v0.58.0 hub back off 5s→5min while the 15-min cycle reconciles. Gated on the SAME hubPusher!=nil && Hub.Enabled as the trigger. Copy soften: backups_remote/escrow "néhány másodperc, legfeljebb 15 perc" (15-min bound stays as the honest worst case). Red-proof: baseline-branch-off → first-obs fires (reverted). Agent-plane poke is PARKED in the OOB arc (spike P4). Deployed to 9201; live-validated (hold + immediacy). Detail: CHANGELOG.md v0.140.0, controller/README.md §9, REPORT.md.

Last updated (prior): 2026-07-16 (v0.139.0 — immediate out-of-cycle hub report, Direction 1)

2026-07-16 — v0.139.0: immediate out-of-cycle hub report on user actions (Direction 1 SHIPPED; Direction 2 pending SPIKE-immediate-sync-transport). Viktor's ruling: user actions with hub-side effects round-trip in seconds. New report.Trigger (internal/report/trigger.go): buffered-1 chan + worker, quiet 2 s → drain → min-interval 15 s → ONE full BuildReport+Claimed+ Push; trailing-edge coalescing (burst ≤ 1+ceil(burst/15 s) pushes, last state always lands), no own retries, failures degrade to the UNTOUCHED 15-min cycle. Generalizes the v0.70.0 geo reportPushNow seam (raw goroutine in main.go replaced by the debounced trigger). Wired: geo save/sync + app deploy/remove/delete (api), and via web.SetReportTrigger/reportTriggerNow (nil-safe, AFTER successful local commit only): escrow recovery-code claim (headline — the v0.138.0 "megerősítésre vár" card now collapses in seconds via the unchanged EscrowAutoConfirmer ACK hash-match), notification-prefs save, app-email toggle, offsite config + per-app toggle, customer claim. hub.enabled:false → seams nil → strict no-op. NO hub change, NO UI copy change ("legfeljebb 15 perc" stays the honest worst case; post-live-proof a soften to "általában néhány másodperc" is a later one-liner). Tests: trigger_test.go (2 red-proofs recorded), report_trigger_seam_test.go, report_trigger_nilsafe_test.go. Known pre-existing Windows-only test failures (appexport df=0, stacks paperless, web fab pipelines) verified failing on base 8f3564c too.

2026-07-16 — v0.138.0: escrow "awaiting hub confirmation" waiting state. Fixes the customer-zero (N100) UX gap: after a completed escrow ceremony the Távoli mentés page kept showing the yellow "Helyreállítási kód szükséges" card for ~15 min until the next hub-report ACK flipped pending→escrowed. Phase-0 diagnosis (read-only) = verdict A (report-cycle lag), already resolved on the box: demo logs show the ceremony claimed 16:13:39, the next hub-report ACK at 16:27:58 auto-confirmed via hash-match (d517ce7f…); settings.json = escrow_state:"escrowed". Hub Hypothesis B verified FALSE → no hub change: SaveHostEscrow's ON CONFLICT … stale_at = NULL already clears stale on upload (store.go:2084); withhold only fires while stale_at != "" (store.go:2150). Part 1 (code): new persisted OffboxTarget.CeremonyCompletedAt (stamped on the recovery-code claim, zeroed on the flip + manual confirm); offboxCeremonyWaitState + escrowCeremonyGraceWindow=35m; backups_remote.html gains an info "megerősítésre vár, legfeljebb 15 perc" card → warn "a megerősítés nem érkezett meg" past the window; backups_escrow.html final step gains a "Mi történik ezután?" note. Test web/escrow_wait_state_test.go + red-proof. No scheduler/agent/hub/endpoint changes. Deploy 0.136.0→0.138.0 to 9201. First "Távoli mentés most" = Viktor's click (NOT done). Note: demo host key changed (box reprovisioned for N100) → known_hosts refreshed.

2026-07-15 — v0.137.0: cleanup bundle (email-wipe guard + carried hygiene). Closes the arc's carried micro-queue. Part 1 (code): settingsNotificationsHandler now REFUSES a save with a blank email box while events are enabled (it would push empty to the hub → wipe the customer's provisioning-seeded alert address — the 2026-07-15 demo incident). Returns before SetNotificationPrefs + sync, Hungarian error, repaints submitted checkboxes; empty+zero-events clear-all still allowed. No HTML required (it would block the legit clear-all). Tests + red-proof in web/notifications_guard_test.go. Deployed 0.137.0 to 9201 (healthy). Part 2 (hygiene): removed the confirmed-older felhom-flash/backups/primary/immich recovery unit (CreatedAt 06-23 < live usb 07-15, 44M); STOPPED audiobookshelf/komga/romm on flash (CreatedAt TIED with usb → tie-break is drive-order-dependent, not confirmable — manual disposition pending). Part 3: campaign6 is a bare empty leftover dir (not a live mount); safe rmdir refused (Permission denied — autofs-ghost/immutable); no mount disturbed → left for Viktor's reboot window. Part 4: tagged the campaign6 6D-audit finding track-only (felhom.eu dee72cd).

2026-07-15 — v0.136.0: .fab exclusion scoping (Task 4). Architecture §2 .fab row + SQ5 verdict + R1-C. SQ6 over-capture FIXED for classified apps: the userdata root tar is exclude-scoped (keeps only ancestors/descendants of a SELECTED bind relpath — the tier2Reconcile keep-rule); no selected userdata bind ⇒ no root tar (radarr state-only). New appbackup.ComputeFabBuckets (shared resolveGuardCollapse pipeline; class buckets; guards over ALL classes; no cross-bucket containment). New appexport/fabplan.go: computeFabPlan (SkipMounts/SkipUserdataTar/UserdataExcludeRels) + tarDirectoryExcluding + fabEstimateSplit. ExportRequest += DeselectOptional/OptInExcluded (both start handlers — two-call-site); mandatory is a SERVER-SIDE floor. Manifest v1 + import UNTOUCHED; legacy apps byte-identical to v0.130.0. Estimate gains an additive class split; export UI shows locked-mandatory / optional-checkboxes / excluded-opt-in + the two-number warning. All 6 §10 red-proofs verified. CAMPAIGN-6D Accept #1 (≥1 GiB .fab full circle) now runs against this shape.

2026-07-15 — v0.135.0: tier-2 engine rework (Task 3b). Architecture §2/§8. New tier2_capture.go: classified apps get the TierSecondary per-bind legs (paperless copy shrinks — export drops); legacy apps keep the byte-identical resolver set. v2 relpath-mirroring layout (backups/secondary/<stack>/{.felhom-tier2-layout marker LAST, recovery-unit/, hdd/<rel>/, userdata/<rel>/}) — N>1 native (flat-appdata refusal + errTier2MultiDir/tier2AppDataName deleted). Migration=delete-and-rebuild + reconcile (prunes dest dirs a bind no longer covers); all os.RemoveAll via tier2SafeRemove (refuses outside backups/secondary/). SSD=state-only tier (unit+mandatory). selectTier2Target never picks NETWORK storage (pinned+auto, F-6C-1). Restore reads v2 behind a marker gate (pre-v2 refused); two-subtree missing-only merge. Part 0: offbox_enlarge_blocked is now a persisted one-time Load seed (OffboxEnlargeNoticeSeeded), NOT a getter append — opt-out STICKS (fixes the 3a-fix un-disableable checkbox). Part 0.5: offsite restore scratch prefers a local (non-network) path. Full v2 test suite + all 10 §10 red-proofs verified. Every destructive write bounded to backups/secondary/.

2026-07-15 — v0.134.1 (+ hub v0.55.0): Task 3a-fix. Placement hardening in offbox_restore.go: F-3a-1a live target uses raw GetStackHDDPath (not AppNamespaceRoot — its systemDataPath fallback would merge userdata onto the SSD; empty ⇒ undeployed ⇒ refuse); F-3a-1b placement headroom gate; F-3a-4 stat PRE-PASS over all placements before any copy (no partial writes); F-3a-3 mapOffsiteRestorePaths refuses the namespace root itself; F-3a-2 scratch removed on success (place button then gone), kept on failure. Enlarge-blocked notification delivery chain: DefaultEnabledEvents + GetNotificationPrefs append-if-absent migration + settings checkbox + handler slice (controller), and hub v0.55.0 allowlists offbox_enlarge_blocked — NO customerMessages entry (raw dynamic message must survive). +8 controller tests, +2 hub; all §10 red-proofs verified. Hub LIVE (ArgoCD synced, :0.55.0). Migration trade-off noted (getter re-enables on opt-out — future persisted marker). 3a deferred list shrinks after §13 live legs.

2026-07-14 — v0.134.0: offsite tier policy engine (Task 3a — FIRST behavior change). Implements architecture §2/§6/§7/§9. Each toggled app's offsite push = ONE multi-path restic snapshot (recovery unit + TierOffsite mandatory userdata via ComputeCaptureSet); legacy/undeployed stay unit-only. New offbox_capture.go (offboxCaptureSet + loud gaps: restic 0.14.0 silently skips missing paths, SP-3.4) + offbox_restore.go (ID-first snapshot introspection, scratch off the rootfs + headroom gate F-A1, RestoreOffboxScratch(full) unit-only default via --include, PlaceOffsiteRestore missing-only merge, pure mapOffsiteRestorePaths). Quota → stats --mode raw-data (SP-1; displayed size drops once after deploy). Pre-push enlargement gate blocks the userdata enlargement over-quota (unit-only push continues; OffboxTarget.EnlargedBlocked; edge-triggered notify). forget --group-by host,tags on both sites (SP-2). UI: /backups/restore three actions (unit / full two-step / place-to-live); /backups/remote per-app blocked note. New route POST /backup/offbox/place. HUB FLAG: offbox_enlarge_blocked event needs hub allowlist+customerMessages for push delivery (in-dashboard LastWarning works now). +13 tests, all 10 §10 red-proofs verified. NOT-live-yet (6D): PlaceOffsiteRestore, large full restore, live enlarge-block, notification delivery, SQ3 immich full-circle. Tier-2 (3b) + .fab (Task 4) untouched.

2026-07-14 — v0.133.0: capture-set computation (Task 3-core, INERT). Task 3-core of the backup-classification-redesign arc (architecture felhom.eu/documentation/architecture/07-backup-architecture.md §3; spike verdicts SPIKE-restic-snapshot-shape-2026-07-14.md). New appbackup/captureset.go: pure ComputeCaptureSet(binds, hasClassification, tier, hddPath) → CaptureSet{HasClassification, Paths []CapturePath, Skipped []SkippedPath}. Pipeline: legacy short-circuit → tier filter (TierOffsite=mandatory only, TierSecondary=mandatory+optional, excluded dropped) → structural guards (traversal / bare HDD drive-root / reserved backups/ → Skipped with English reasons; bare userdata allowed) → equal-Abs collapse (mandatory>optional) → containment dedup (keep ancestor) → sort by Abs. Slash algebra only (no filepath). Pure CrossAppOverlaps advisory (WARN wiring deferred to 3a/3b). Deliberately INERT — no engine consumes it yet; 3a (offsite policy) and 3b (tier-2 rework) are the consumers. ARCHITECTURE IMPACT from the spike (SP-3.4, already in §2.5): restic 0.14.0 does NOT error on a missing source path (exit 0, silent partial snapshot) → the stat-filter in 3a/3b is load-bearing. Wiring test through a real Manager (F-S3 no-seam); all 6 §10 red-proofs verified. felhom.eu §3 docs aligned (8d85da7).

2026-07-14 — v0.132.0: backup classification (Task 2, INERT). Task 2 of the backup-classification-redesign arc (spike: felhom.eu/documentation/audits/SPIKE-backup-classification-2026-07-14.md). Ships the referential-coupling classification as DATA + PARSER + PURE CLASSIFIER, deliberately inert — no backup tier changes behavior. New appbackup/classify.go: BackupSpec/BindSpec (the .felhom.yml backup: block), ComposeBind (${VAR}-relative + :ro), ClassifyBinds (SQ5 two-level default: explicit beats :ro; unlisted writable→mandatory, unlisted :ro→excluded; no block → legacy/false), ValidateBackupSpec (whole-block-reject on any defect). New stacks/classify_binds.go ParseComposeClassifiableBinds (relative-space, keeps :ro — NOT ParseComposeHDDMounts/ExportDataMounts, the classifier-input traps). LoadMetadata is the SINGLE validation choke point (bad catalog block → nil + one [ERROR] within one sync cycle → legacy). Wired seam Manager.ClassifiedBinds + StackDataProvider.GetStackClassifiedBinds (nil-stubbed in every fake) so Task 3 (tier policy engine) consumes a tested seam, not a fresh one. Inertness proven: full pre-existing suite green with ZERO test-logic edits. The 13 catalog backup: blocks ship in the same app-catalog-felhom.eu change (controller deployed FIRST so the parser validates on first sync). audiobookshelf PENDING-VETO: media/audiobooks ruled optional (consistency with komga/romm) pending a Viktor veto to excluded. +14 tests, RP-1..RP-4 confirmed. Next: Task 3 consumes ClassifiedBinds to scope offsite/tier-2/.fab capture by class.

2026-07-14 — v0.131.0: F-S2 + F-S3 (compose-derived appdata dir resolution). Task 1 of the backup-classification-redesign arc (spike: felhom.eu/documentation/audits/SPIKE-backup-classification-2026-07-14.md). The controller assumed appdata/<stackName>; paperless-ngx writes appdata/paperless (stack paperless-ngx). ONE canonical resolver appbackup.AppDataDirNames(hddPath, stackName, mounts) derives the real dir name(s) from compose ${HDD_PATH} binds (deduped/sorted; fallback [stackName]); all consumers use it. F-S2 (spike-proven): RunTier2/Tier2Info/RestoreTier2Files now hit the resolved dir (paperless documents got NO tier-2 copy before — the appdata leg stat-skipped a dir that never existed). F-S3 (NEW, found this session): migrate.go keyed all six per-app appdata legs by stack name; scope="app" has no merge walk, so migrating paperless-ngx copied nothing, verified vacuously, flipped HDD_PATH → empty media dir (scope="all" was saved by the merge walk — data safe, accounting off). All six legs now loop resolved names. Multi-dir (N>1) refusal is defensive (no catalog app hits it today: immich/nextcloud/romm match, paperless mismatches, each app = exactly ONE dir): tier-2 backup/info/restore refuse loudly (Hungarian); migrate supports N naturally. This limitation is deferred to Task 3 (tier-policy engine), which owns the destination layout. Storage page sums resolved dirs. Truth repairs: the v0.130.0 CHANGELOG/CONTEXT "tier-2 copies the namespace wholesale" claim is FALSE — corrected in the v0.131.0 CHANGELOG entry + main.go export-adapter comment; tier-2 copies the recovery unit + resolved appdata/<name> ONLY (NOT userdata — F-S1, unaddressed here). New seam tier2Mirror; migSeams.resolveNames. +9 tests, RP-1..RP-5 all confirmed. Controller-only, no agent/hub coupling. NOT live-validated here: scope="app" migration of a real app between drives (F-S3 live proof — supervised leg, Viktor's session).

2026-07-14 — v0.130.0: CRITICAL C6B-F1 (hollow .fab export) + C6B-F2 (share-removal guard). CAMPAIGN-6B proved .fab export shipped config-only, data-free bundles for 12/13 needs_hdd catalog apps (sonarr 4.17 GB → 2308 B, success, past the v0.125.0 guard). Three compounding fixes (all red-proofed run→fail→revert): (1) stacks.ExportDataMounts — export mount discovery unions ${HDD_PATH} binds + the ${USERDATA_PATH} ROOT (single userdata entry; root-not-per-bind is LOAD-BEARING: the manifest keys tars by basename and the untouched import maps basename → <HDD_PATH>/<subdir> — per-bind subpaths would restore to wrong places; this deviates from the task's literal per-bind+namespaced-names instruction, which could not round-trip without import changes the task forbade); (2) export + estimate are ADDITIVE for needs_hdd apps (HDD data AND named volumes — sonarr_config was silently dropped); (3) anti-hollow guard: needs_hdd manifest with zero data fails loudly. Plus §8: basename collision between mounts = loud Hungarian failure (was silent overwrite). C6B-F2: netstorage/remove refuses 409 while a DEPLOYED app's HDD_PATH is on the share (the orphaned-autofs trigger); resolves via the netAgent seam. Residual flagged for a felhom-agent task: RemoveNetworkMount's tolerate-and-continue stop (felhom-agent netmount.go:434-443) still deletes unit files under a busy mount if some non-product path calls it. Scheduled/tier-2 backup path was NOT affected and is untouched (stackAdapter deliberately unchanged). CAMPAIGN-6C's first acceptance test = the full-circle byte-compare this unblocks.

2026-07-13 night — v0.128.1 + demo storage hygiene (ruling F5). classTag suppresses the rotational class hint for type==='usb' (card already carries the USB tag; hub ClassHint UNCHANGED; pinned by TestStorageTemplate_USBClassBadgeSuppressed + red-proof). Host op on demo-felhom: the two pre-intermediary legacy dir: storages (felhom-usb, felhom-flash, content=Backup, is_mountpoint) RETIRED via pvesm remove after G1/G2/G3 gates all PASSED (agent-owned UUID .mount units; both enrolled in drive-intents.json; zero /etc/pve refs, empty dump/, no customer app on either drive). Post-removal: mounts+binds intact (marker round-trip through the guest), GET /disks shows both registry-sourced (role+durable-id intact, class absent), pvesm status clean. The demo node now matches the fresh-install storage shape — drives are registry+units-sourced only, no legacy PVE dir: storages. v0.128.1 LIVE on demo 9201 (drill guest skipped — optional, no behavioral dependency; it runs 0.128.0).

2026-07-13 night — v0.128.0: CHUNKED BROWSER .FAB UPLOAD on /import (ruling F3: chunked). Step-0 probe on the real tunnel PROVED the Cloudflare edge cap (120 MiB POST → edge 413 with Server: cloudflare on Content-Length alone; 80 MiB → origin 302 /login; local DNS overrides the hostname to the LAN guest, probe needed --resolve onto CF's public IP). Design: JS File.slice 64 MiB strictly-sequential chunks → POST /api/export/upload/{init,chunk,finalize, abort} inside ServeExportAPI (inherits RequireAuth+CsrfProtect; single-flight; offset must equal received else 409+echo; 96 MiB request cap; free-space gate size+1 GiB; finalize = exact-size + fsync + atomic rename, collision → lowest-free "name (N).fab"). Lands in the DEFAULT drive's exports dir — scan/validate/import pipeline untouched. No client hash (deliberate: .fab self-validates). In-memory state: startup GC of *.part-*, 15-min idle abort. §7 A–F tested + 3 red-proofs. appexport.DiskFree exported (REUSE.md row). NOT live-validated: the end-to-end multi-GB browser upload through the real tunnel needs a dashboard login → Viktor's 5-minute leg (export an app to .fab, download, re-upload, import — full circle). Possible follow-up if it itches: per-drive target picker (v1 = default drive only).

2026-07-13 eve — v0.127.0: CUSTOMER-FACING ESCROW CEREMONY WIZARD (/backup/escrow) + Scenario-F stale-blob re-check. MinAgent 0.88.0 (wizard only). LIVE on demo 9201 + drill guest (both healthy). The friend-alpha missing piece: preflight → warnings → password re-auth (login rate limiter) → re-stage-first (abort on failure — the UI can never mint a hash-less blob) → agent job (poll 2 s) → ONE-SHOT R reveal (no-store; R only in the claim XHR + page JS; 10-min TTL → void) → typed-back (two random words) → finish. Ruling F1: R over the CF tunnel once = accepted (threat model in felhom.eu RUNBOOK-escrow-ceremony.md). Scenario F: an ESCROWED box re-checks the ACK hash — mismatch/hash-less ⇒ stale flag (card warning + CTA) + one WARN per hash; never flips, never blocks; fired LIVE on both boxes' hash-less blobs at first ACK (drill = the spike's superseded blob, since REPAIRED via a real ceremony — restic_pw_sha256 now covers; demo = its legacy blob, warning stays until a wizard run). Manual-confirm BUTTON removed (endpoint stays, deprecated). OPEN: one supervised full-browser wizard pass with Viktor's login (re-auth needs the customer-owned password — CC validated everything beneath it endpoint-exact); demo wizard run to clear its stale warning.

2026-07-13 — v0.126.0: UI UNIFORMITY BUNDLE (shared app-list rows + infra identity + restore-form polish + mojibake gate + honest stale line). Presentation-layer only — NO backup/toggle/engine behavior change. MinAgent 0.81 + floor unchanged. (A) templates/app_row.html app_list_row/app_list_row_end is THE canonical list row (icon+name left, caller action right, compact 44px) — dashboard installed-apps, Távoli mentés toggles, Visszaállítás restore-to-verify + .fab lists render through it; the backups-apps expander header is ALIGNED (own markup, allowlisted); gate scripts/app_row_dedup_gate.py (red-proven). funcmap: dict/appHref/infraMeta. (B) inframeta.go: cloudflared → „Cloudflare Tunnel", traefik → „Traefik", filebrowser → „FileBrowser" + Hungarian descriptions

• generic /static/infra-logo.svg fallback; filebrowser = the ONLY Linked infra (files.); render test counts exactly one customer link (red-proven). (C) .fab password field standard („Opcionális jelszó" + helper; import-page input got .form-input). (D) scripts/mojibake_gate.py — templates+Go strict UTF-8, zero Ã/Â/Ă-signature chars, allowlist ZERO (red-proven); source had NO mojibake — the live „Tárhely" is the felhom-usb drive-label DATA, repaired via the label-edit UI (live step). (E) offboxWarningDisplay display pick — stale „nincs mentésre jelölt alkalmazás" run-warning → „A kijelölés módosult…" note once ≥1 app toggled (neutral color); 0 toggled unchanged (red-proven). Housekeeping: one-shot backups_split_move_check.py RETIRED (served its purpose). Live QA fixes: v0.126.1 — .form-input/.form-row had NO CSS rule at all (root cause of the unstyled .fab password field; styled as the .form-control twin). v0.126.2 — CF edge caches /static/style.css 4h → stylesheet link now ?v={{.Version}} (auto-bust per release). 0.126.2 LIVE drill+demo (both healthy). §13 visual QA ran on the DRILL box via a reversible SSH gate-lift (hash restored byte-identical, gate verified back ON) — the demo is customer-claimed and CC does not enter credentials. OPEN human step: demo login → felhom-usb label repair via the label-edit UI (data = 'Tárhely (felhom-usb)' in storage_paths, documented read-only; the Part D gate closed the code side).

2026-07-13 — v0.125.0: .FAB VOLUME PATH-STRAND DATA LOSS FIXED (IA finding 1, HIGH). MinAgent 0.81 unchanged; floor may advance to 0.125.0 next train (must NOT halt above 0.124.0 without this). Both volume legs stream via docker cp (helper container + dockerExec seam — zero shared paths, correct bare-metal AND containerized; §3 live probe first). Export FAILS LOUD on any missing/empty claimed tar (assertBundleDataComplete); import VALIDATES BEFORE it destroys (validateBundleData in step 0 — hollow bundle → refusal, app untouched). Class extinguished by scripts/docker_run_volume_path_gate.py (every "-v" allowlisted with WHY; Tier-1/2 mounts documented host-visible). Live: the exact failed ActualBudget leg round-trips byte-identically (ec8ea6cb… before==after); engine-invalid volume → loud export failure. ASYMMETRY (needs a customer-docs line): .fab bundles exported by containerized ≤0.124.0 controllers are hollow — re-export; the import guard refuses them loudly.

2026-07-13 — v0.124.0: BACKUPS IA RESTRUCTURE. MinAgent 0.81.0 + floor unchanged. Operator decisions (2026-07-13, treat as settled): (1) single active offsite destination per box STANDS — the dual-destination managed_by model is the separate queued Task B; (2) the Felhom-offsite status card NEVER changes anything — display + opt-in pointers only; (3) .fab export/download is PORTABILITY, not a backup tier — no scheduling, no status surface, point-in-time framing. Mechanics: four sub-pages (/backups{,/remote,/apps,/restore}, sections moved VERBATIM — scripts/backups_split_move_check.py gates vs df7ad37), status card (3 states, display-only), .fab download exit (existing exporter + staging dir + guarded stream + 1h TTL; traversal guard red-proven). Live-validated on drill+demo incl. a supervised import round-trip. NEW FINDINGS: (HIGH) containerized .fab export strands Docker-VOLUME tars on the guest host (docker run -v <container-tmp> → host path) — bundle ships empty volumes, import brings the app up EMPTY; fix = host-visible staging + fail-loud post-export assertion. (MEDIUM, agent) legacy-boot PVE (LVM root, no ESP mount) → SystemDisks empty → sysKnown=false → drive wizard offers ZERO candidates ever. Follow-ups: .fab browser-upload; mega-zip parked.

2026-07-13 — v0.123.0: POLISH BATCH (take-two F-15/F-11 + rename + zero-toggle). MinAgent 0.81.0 unchanged; floor 0.122 unchanged. Requires hub v0.52.0 for F-15 (old hub = clean no-op). (1) F-15: the reset-request RESPONSE carries the rotated code hash, applied via the ACK's generation-guarded ClaimSync — emailed codes work immediately (live: 1 s, first-try accept). (2) F-11: zero native confirm() — felhomConfirm/data-confirm inline Igen/Mégse (layout.html); gate scripts/native_confirm_gate.py. (3) Tier-3 customer branding is "Távoli mentés" (NAS-mentés gone; manual form generalized to any SFTP target; gate scripts/offbox_rename_gate.py; "Hálózati tárhely" feature untouched). (4) Zero-toggle honesty: hint + "Sikeres — nincs mentésre jelölt alkalmazás" run copy. Also: atomicPromoteTar O_RDWR fsync (Windows dev-box green gate was permanently red). Deployed drill qm300 + demo 9201. NOT started (separate queued task, operator fork pending): offbox managed_by coexistence.

2026-07-12 — v0.122.0: CUSTOMER-CLAIM PASSWORD GATE (closes DRILL-day0-vm F-4/F-5). MinAgent 0.81.0 unchanged. Requires hub v0.50.0. The dashboard password is CUSTOMER-OWNED via a one-time claim code the hub emails to the registered address — the "no password → open dashboard" default is GONE. Unclaimed box (code hash delivered, no password) → serves ONLY /claim; every other route → claim page (302) or 401 (API). A set password disables the gate (auth wins). Reset rides the same code engine (login "Elfelejtett jelszó"). Legacy-open (no password + no hash) → red transition banner until the hub delivers a hash. internal/web/claim.go (gate + pages + HMAC pre-auth CSRF + 5-try/15-min lockout → claim_lockout event), report/claim_sync.go (ACK cache, idempotent by generation), settings Claimed/ClaimCode*/ClaimConsumedGeneration, config.web.claim_code_* (hub-baked), --print-reset-code root hatch. Gate-coverage signature test + 4 red-proofs. LIVE-PROVEN on drill guest 9201 (0.122.0): gate ON via the real edge (/ → 302 claim page, /api → 401), code emailed to demo-vm-felhom's registered address. Floor raise 0.120→0.122 = operator's LAST step (supervised). Details: felhom.eu/documentation/audits/DRILL-day0-vm-2026-07-12.md (F-4/F-5 RESOLVED).

2026-07-12 — v0.121.0: BACKUPS PAGE TRUTH PASS. MinAgent 0.81.0 unchanged. Controller-only, no agent-API change. Pure UI/data-plumbing on /backups; no backup-engine behavior change. Fixes the self-contradicting live page: (1) removed the dead "Részletek" card (operator decision — redundant; per-app rows + Adatbázisok section already carry the truth) — kills the last uses of the never-set template fields Tier2DriveGroups/ResticPassword, the restic-pw element, and the toggleTier/ toggleResticPw/copyResticPw JS. (2) per-app "3. mentés" row now shows real off-box state via a new pure tier3State (configured→toggle→escrow precedence): unconfigured/off/escrow_pending/active — the "Hamarosan — B2/S3/SFTP" placeholder is gone. (3) SQLite-honest DB messaging via pure dbSectionState(discovered,dumps) → dumps/pending/embedded (embedded-only box shows "–" + "beágyazott DB-k a kötetmentésben", not a bare "0"). (4) dead/raw fields fixed — Tier1LastRun/Tier1LastStatus now populated from ListRestorePoints; Tier-1/Tier-2 labels via timeAgoStr (relative), confirm() dialog keeps raw. (5) terminology split — off-box section = "Távoli mentés (3. mentés)" (+ #offbox-section anchor); whole-guest PBS card = "Távoli rendszermentés" (was both "Távoli mentés"). (6) deploy page gains a "Mentési beállítások →" link. DECISIONS: Részletek removed as redundant (operator-approved); "Távoli mentés (3. mentés)" (app off-box) vs "Távoli rendszermentés" (PBS whole-CT) are two distinct customer-facing names. Pure helpers in internal/web/backup_page_state.go. +9 web tests, 4 red-proofs. Observations: orphaned style.css classes from the Részletek removal left in place (details-tier*, repo-encryption*, restic-pw-field, drive-detail-, tier-empty-state, repo-info-row, repo-tier-title) — noted, not cleaned. Backlog: felhom.eu backup-architecture.md offbox refresh (separate task).

2026-07-12 — v0.120.0: fix-3 + fix-6 → CAMPAIGN-3 CLOSED (LIVE on 9201 + hub 0.48.0). MinAgent 0.81.0 unchanged. fix-3: a deadapp-check job (30s, 90s boot grace) flags a DEPLOYED app in stopped/exited state (stacks.IsDownState) → self-clearing WARN dashboard banner + one app_start_failed hub event per running→down transition (Notifier.NotifyAppStartFailures, in-memory tracker, hub owns cooldown). fix-6: ring cap 1000→5000 (display cap raised too); periodic scheduler/refresh success lines → [TRACE] (ring drops at write-time, failures never TRACE); atomic JSON-lines spill to <DataDir>/debug-ring.log (SSD, survives recreate) every 30s + shutdown, loaded on boot. hub v0.48.0 accepts app_start_failed (allowlist + customerMessages). LIVE: docker stop seerr → banner + ONE hub event across 3 cycles (anti-spam) → docker start → banner self-cleared; ring 0 spam lines + restart PRESERVED the pre-restart window (oldest unchanged, 63KB spill on SSD volume). CAMPAIGN-3 CLOSED (F12/F11/F10/F9/F2/F1→agent 0.85; F7/F6/F5→0.118; F8/F4→0.119; fix-3/6→0.120). Follow-ups: agent-ring persistence; F13 (active-nfs-mp8 rc255); publish train (agent 0.85 + ctrl 0.118/0.119/0.120 + hub 0.48) to Peti. Seams: deadapp scanDeployedAppRunStates, notify.pushFn.

2026-07-12 — v0.119.0: STORAGE-HEALTH COHERENCE (LIVE on 9201). MinAgent 0.81.0 unchanged. Fixes CAMPAIGN-3 F8+F4. F8 (MED): the share row's health came only from the agent's server-level TCP dial (blind to a single unexported share) → it showed benign "Készenlét" while the stacks cards showed the stub — a contradictory UI. networkStorageItems→fuseNetHealth now reuses the SAME system.ClassifyPathFS the stacks stub badge reads (§3 fork = option B, controller-only): a new stub health (badge "Hibás — az alkalmazások nem a NAS-t látják") overrides idle/ok when the namespace sees local disk; unreachable still wins over stub; autofs/network/unknown leave agent health intact (never force-mount). Row + stacks badge now share ONE classifier → can't contradict. F4 (LOW): handleNetStorageAdd range-checks container uid/gid 1..65533 (validMappedID) → friendly 400, nothing installed (was raw agent_error on 101000). LIVE: F8 row=stub matching stacks badge through an exportfs cut, cleared to ok on re-export; F4 uid 101000→400, uid 1000 passes. Seam: s.classifyFSPath. Task D (fix-3 alerting + ring revision) still queued.

2026-07-12 — v0.118.0: BACKUP INTEGRITY (LIVE on 9201). MinAgent 0.81.0 unchanged. Fixes CAMPAIGN-3 backup findings (documentation/audits/CAMPAIGN-3-2026-07-11.md). F7 (HIGH) atomic volume dumps: DumpAppVolumes writes <vol>.tar.tmp → fsync → os.Rename over the .tar only on success (atomicPromoteTar), mirroring dbdump.go DumpOne; a mid-write NFS cut can no longer truncate the last good tar to 0 bytes. F6 (LOW) no single-copy: RunAllTier2 no longer skips volume-only apps (they now get a cross-drive tier-2 copy); sys_drive restore-point label is clear ("Belső SSD (rendszer)"); single-drive box shows an honest SingleCopyWarning banner. F5 (LOW) stale-primary sweep: pruneStalePrimaryDirs removes an orphaned backups/primary/<app> dir on an OLD drive after an HDD_PATH move (guarded: deployed + different-current-drive only, never a restore point). Part 4 locality fork → operator chose (A) keep locality, doc-only (NAS tier-1 stays on the NAS; tier-2 is the off-NAS leg). LIVE: F7 money-shot (all NAS tars byte-identical through a mid-write cut, no 0-byte, success:false); F6 (actualbudget/seerr on felhom-usb/secondary); F5 (seeded stale dir swept, current kept); restore round-trip byte-identical. Seams: tarVolume, perAppTier2. Task C (F8/F4) + Task D (ring/alerting) still queued; Peti reaches 0.118 + agent 0.85 at his next train (agentless-on-proxmox2 gap noted).

2026-07-11 — v0.116.0/0.116.1: OBSERVABILITY PASS (LIVE on 9201; agent v0.83.0 + hub v0.46.0). MinAgent: 0.81.0 unchanged. The debug ring (LogBuffer) now ALWAYS exists — logger = MultiWriter(LevelFilterWriter(stdout, logging.level), ring), so DEBUG detail is remotely readable on an info box while docker logs keep the configured level. New internal/logx leveled helpers = the sweep standard (netstorage_job phases/verdicts/durations, netprobe, validation refusals, orphan WARN, SupportsWithSource gate line, agentapi per-call DEBUG, migrate phases, tier2/offbox unswallowed persists). Report ACK gains controller_log_requested → next report ships controller_log_tail (selftail.go, consume-once, 128 KB; the customer-visible operator log pull served INFO rides in the tail; app-tail wire byte-compatible). Debug page: Vezérlő | Ügynök tabs — the agent tab proxies agent GET /debug/logs (Client.DebugLogs; typed-404 → the "after the agent's next update" notice). v0.116.1 (found by live validation): /debug + /api/debug/* + the nav item were STILL gated on logging.level=debug — ungated (auth unchanged), the incident's actual blind spot. Live-proven at info: a real refused NAS add is fully reconstructable on both tabs (capability gate w/ source=version, phase lines, 502+duration, category). Conventions: felhom.eu/documentation/runbooks/logging-conventions.md. OPEN: operator clicks the hub's two "Request logs" buttons (hub UI password-gated) to close the live bundle round-trip; legacy isDebug() emission sites left for incremental migration. NOT published — Peti stays 0.113/0.81.

2026-07-11 — v0.115.0: version-aware Supports + DSM-validated NAS guidance (LIVE on 9201, pairs with agent v0.82.0 + hub v0.45.0). MinAgent: 0.81.0. Capability detection now compares the agent version from agent v0.82.0's X-Felhom-Agent-Version header (Client.noteAgentVersion captures it on every response, strict semver; features.go featureMinAgent table + version-first Supports) instead of route-probing — the probe stays as the fallback for header-less (≤0.81) agents, so nothing changed for Peti's box. THE one comparator moved to internal/util/version.go (selfupdate aliases it). Part A DSM spike (real DSM 7.2 via virtual-dsm) validated the consumer recipes E2E; the NAS-page NFS guidance gained the verified Synology steps (File Services → NFS → NFSv4.1; "Map all users to admin"; /volume1/<share>); caveat narrowed to QNAP-only. Live-checked on demo: a real add shows capability gate: netstorage_verify=yes via the version compare, zero probes. Q1c (Part E, supervised) FAILED: a NAS automount trigger does NOT survive a guest reboot (guest sees an empty dir; agent has no network-mount reassert) — fix is felhom-agent's, spec'd at felhom.eu/documentation/backlog/FOLLOWUP-nas-automount-guest-reboot-reassert.md (controller health-cross-check follow-on noted there). NOT published (agent 0.82 demo-only; Peti 0.81).

2026-07-11 — v0.114.0: agent-capability gate (option-1) + publish-train rules (option-2). Answer to the 0.81/0.113 train's 9-minute controller-before-agent skew (Peti's box): the box now protects itself. internal/agentapi/features.go — Supports(Feature) route-probes the agent (GET /netstorage/verify-status = the v0.81.0 coupling signal; typed StatusError 404 ⇒ No, 2xx ⇒ Yes, transport/5xx ⇒ Unknown NEVER refused; SupportCache TTL 5m both polarities, Unknown uncached). handleNetStorageAdd refuses on No BEFORE the single-flight claim (412 + agent_outdated + honest Hungarian message); settings page swaps the add form for a banner (list/remove untouched in every state). remove/list NOT gated. NO agent/hub changes; NOT published, floor untouched, Peti stays 0.113.0 — the gate is inert protection until the next train. Rules codified: felhom.eu/documentation/runbooks/publish-train-rules.md (manifest before floor; floor field LAST — hub_settings DB row overrides env + acts immediately; MinAgent fleet gate — CHANGELOG header convention starts with this release; the gate as box-level backstop). Tests T1–T6 + wire-level 404-typing; red-proofs RP1–RP5 in REPORT.md. Roadmap: agent version-in-envelope upgrade of Supports; hub floor-UI separation = its own task. The agent_outdated branch is test-proven only (demo agent is current — downgrade not justified).

2026-07-11 — v0.113.0: NAS verify-before-commit + page redesign (LIVE on 9201, pairs with agent v0.81.0 + host-install v1.13.0). POST /api/storage/netstorage/add no longer registers blind (the bogus-share-at-Készenlét bug is dead): detached single-flight orchestration (internal/web/netstorage_job.go, migrate shape; poll GET .../add/status) = agent add (units + agent-side detached verify with journal classification + auto-rollback) → controller uid-1000 re-exec write probe (--netprobe, SysProcAttr.Credential — catches the squash trap) → register LAST. Any failure = full rollback; verify-lost after agent restart ⇒ controller rollback; unregistered agent shares surface as remove-only "Árva megosztás" rows. §3.2 Hungarian error map server-side (netAddMessage; nfs_export MERGES not-found/not-permitted — NFSv4 identical strings). storage_network.html rebuilt on the storage_attach pattern (form-row/form-input killed), SMB listed first, NFS two-recipe guidance with live computed uid+100000. Agent v0.81.0: NFS retry=0 (dead-NAS access 91 s→3.8 s), ClassifyNetVerifyFailure (Q4-verbatim), unprivileged journal read (systemd-journal group — host-install v1.13.0 adds it; NO new sudoers). Live-validated A–E on 9201 vs an isolated sim NAS (all transcripts + red-proofs in REPORT.md); Route A proven in production (alien-uid squash → server-side 1060:1060). Authoritative doc: felhom.eu/documentation/controller/network-storage-nas.md. NOT published (0.81.0 not in Gitea / Day-0 manifest; Peti untouched — his rollout incl. the usermod one-liner comes with the floor bump). Gotcha for future sessions: the controller container is bridge-only — in-guest API tests need the CONTAINER IP + Host: felhom.demo-felhom.eu (127.0.0.1:8080 is stale advice).

2026-07-10 — v0.112.0: self-update without credentials (LIVE on 9201, pairs with hub v0.43.1). Root cause on Peti's box: the updater refused without Git Sync creds, but the public package is anonymously pullable. queryRegistry now does the Docker v2 anonymous token dance when both creds are empty (realm/service parsed FROM the WWW-Authenticate header — never hardcoded); pullImage skips docker login credential-less; creds path byte-unchanged (private catalogs); half-configured pair = loud misconfig; denial = "registry denied anonymous access — a private registry requires Git Sync credentials". Settings panel gains the mode line "Registry: nyilvános (hitelesítés nélkül) / hitelesített" — credential-less is a supported mode, not an error. Red-proof green (old guard restored → anonymous tests fail with the old message). LIVE-PROVEN on the credential-less demo (git creds are quoted-empty): /api/selfupdate/check → ok, latest=0.112.0, no error; settings shows "nyilvános". PENDING OPERATOR: floor-bump Peti to 0.112.0, then delete his temp Git Sync creds → clean "nyilvános" check. runCommand/runCommandStdin are now package VARS (test seam).

2026-07-10 — v0.111.0: remote app-log diagnostics (LIVE on 9201, pairs with hub v0.43.0). The telemetry scraper now attaches LogIssue.Context (±5 raw lines around the FIRST occurrence of each error-severity issue; ≤11 lines, ≤400 chars/line, 16KB/report budget dropping lowest-count first; warns carry none) and metrics.RedactLine sanitizes EVERY off-box context/tail line (password/token/api-key/ authorization/bearer → [REDACTED], 64-hex → [REDACTED-HEX64]). On-demand log tails ride the ACK pull pattern: hub ACK log_tail_requests → next report log_tails (200 lines via stacks.GetLogs / FetchContainerLogTail, ordered, ≤64KB/app newest-kept, redacted, consume-once drain). Hub v0.43.0 stores context (first-capture-wins + context_customer provenance), renders click-to-expand copyable issues, fixes the period filter on Known Issues, replaces issue deletion with DISMISSAL (dismissed_at, resurface only on last_seen > dismissed_at), adds ?customer= filtered drill-down, and keeps the last 2 tails per app with an ordered viewer + .log download. All red-proofs green (capture, redaction, consume-once ×2, dismissal guard, range filter, context clobber). Live-proven on demo: synthetic error → hub row with ordered 11-line context and password=[REDACTED]. OPERATOR: one click ("Request log tail" on demo felhom-controller) completes the live tail round-trip — hub UI is password-gated, CC cannot.

2026-07-09 — v0.106.0: offsite provisioning SLICE 2 — the apply-bridge (pairs with hub v0.38.0). On startup the controller reconciles the hub-served offsite: descriptor into a key-only offbox target: internal/offsiteapply.Bridge.Reconcile — verify-pin the box host key against host_fingerprint (NO blind TOFU) → generate keypair → consume the one-time password (POST /api/v1/offsite/consume-password/{id}, single-use, never logged) → sshpass ssh-copy-id -s -f install + verify → Manager.ApplyOffsiteTarget (fork-4 enable → EscrowState="pending") → persist a descriptor-hash marker LAST. Idempotent (no re-consume of a spent password) + fail-safe (any step fails → nothing persisted, retry next restart; consumed-but-failed install = loud "reset on the hub"). Seams faked in tests; both red-proofs (no-TOFU, marker-after-success) green. Dockerfile + sshpass. NOT yet live-applied — supervised end-to-end (hub provision → controller apply) is the next runbook, gated on the hub's new scoped HETZNER_TOKEN. NEXT slices: SLICE 3 (escrow auto-confirm), SLICE 4 (soft-quota).

2026-07-09 — v0.105.0: fork-4 offsite password custody (pairs with agent v0.77.0). The restic-offsite repo password now rides the customer-R escrow (age-under-R in the agent IdentityBundle; custody spike febdc56). Enable → controller pushes the password (StageEscrowSecret → agent POST /escrow/stage-secret) → EscrowState="pending". Atomicity gate: no offsite RUN until EscrowState="escrowed" (operator POST /backup/offbox/confirm-escrow after the escrow ceremony) — so no un-recoverable offsite ciphertext exists. DR: POST /backup/offbox/inject-password pre-places the recovered password (honored by WriteOffboxSecrets). DR recipe gains non-secret offsite_restic coords (DRResticCoord); the SFTP key is regenerated at DR (not escrowed). Atomicity + inject companion red-proofs green. NOT yet live-validated — the supervised escrow ceremony (enable→stage→escrow-create→confirm→gated run) is operator-run; NEXT = hub-verified auto-confirm + customer-self-serve enable (provisioning task). Deployed to 9201; see REPORT.

2026-07-09 — v0.104.0: off-box discovery over inference + no-silent-success. The Storage-Box spike found offbox reporting ok/0 snapshots while backing up nothing; DIAG pinned it: offbox resolved each toggled app's recovery unit via AppNamespaceRoot→GetAppDrivePath, which reads the app's live app.yaml HDD_PATH and silently falls back to systemDataPath when the app isn't deployed → it looked on the wrong drive. Decision: DISCOVER, don't infer — scan the durable storage registry (schedulable, non-decommissioned paths ∪ systemDataPath) for backups/primary/<app>, deployment-state independent; newest-by-manifest wins on drive churn. Silent-success closed: 0-of-N toggled → hard error + operator alert; partial → ok + customer LastWarning. Write paths + AppNamespaceRoot untouched. Unit suite + both companion red-proofs green. NEXT: supervised box re-provision + a real offbox→Storage-Box endpoint round-trip (this task did NOT re-point at the live box — spike creds were torn down). Deployed to 9201; see REPORT.md.

2026-07-07 — v0.103.0: F-C2-1 (LIVE on 9201). The config loader ran os.ExpandEnv over the whole YAML before parse, silently corrupting a bcrypt web.password_hash ($2a$10$… → "a0") — a silent auth-integrity bug. Removed both ExpandEnv calls (parse raw bytes); typed FELHOM_WEB_PASSWORD_HASH override unchanged. Live-proven: a bcrypt hash in controller.yaml now loads intact and login succeeds (pre-fix it corrupted → login fail). Behavior change: literal ${VAR} in a value is now preserved verbatim (no repo config depends on the old expansion).

Last updated: 2026-07-06 (v0.102.0 — async restore family; F4 re-adjudicated + fixed)

2026-07-06 — v0.102.0: async restore family (F4 UX fix, LIVE on 9201). All three restore surfaces (/backup/restore, /backup/tier2/restore, /backup/offbox/restore) blocked the HTTP request until completion → through cloudflared's 100s cap a customer got an error page while the restore succeeded (offbox worse: bounded on r.Context(), canceling the SFTP restore mid-flight). Now async (offboxRun shape): fast-path IsRunning refuse → background goroutine (offbox ctx off r.Context()→Background+30m) → instant redirect. New GET /api/backup/restore-status + mutex op-status (opstatus.go) + 3s-polling backups.html banner. Live-proven: restore POST 0.018s internal / 0.235s external (F4 tunnel), canary bit-identical, status transitions. Restore single-flight unchanged. OPEN: op-status is in-memory (no persistence, by design).

2026-07-06 — v0.101.0: no-mercy campaign findings. F3: git subprocess deadline in internal/sync/sync.go (gitCmdTimeout=120s, exec.CommandContext) — a hung remote no longer wedges syncing=true until restart. F2 evidence gap: agentapi.EjectDisk/Decommission now use postWithStatus + refusalError so the agent's "…refused (role: X)" reaches the operator instead of a bare HTTP 403. Companion: catalog d86e256 (F1 vaultwarden _ENABLE_SMTP boot-gate — fresh email-off deploys crash-looped; live-validated Scenarios A/B on 9201). F2 diagnosed to a verdict (REAL finding — roleForMountPath over-refuses an enrolled user-data drive that isn't a PVE storage; fail-safe direction; agent fix DEFERRED). Full triage: felhom.eu/documentation/audits/CAMPAIGN-nomercy-2026-07-06.md addendum. OPEN follow-ups: agent roleForMountPath fallback; the targeted P1–P3 campaign re-run for clean backup/restore coverage.

2026-07-05 — v0.100.0 (TASK C2): drill finding F2 CLOSED — one-click class-C file restore. POST /backup/tier2/restore + "Fájlok visszaállítása" on the Tier-2 row: in-place, ADDITIVE-ONLY (rsync -a --ignore-existing from the recorded Tier-2 copy — never overwrites, never deletes). Serves "I deleted my files"; corruption/point-in-time stays offbox/operator. The C-series (drill findings F1/F2/F3/O4) is now fully closed. Reindex caveat (e.g. Nextcloud occ files:scan) documented in backup-architecture.md.

2026-07-05 — v0.99.0 restore-path fixes (TASK C1): drill findings F1/F3/O4 RESOLVED. F1: GET /api/backup/snapshots implemented (backup.ListRestorePoints) — the restore panel populates and the restore button enables. F3: runVolumeDumps wired into the nightly/manual backup run (volume gate BEFORE DumpAppVolumesSafe; before unit capture). O4: unrecoverable resettable secrets get a generated replacement (GenerateSecretForField + SetSecretGenerator seam); data-key gate untouched. F2 (one-click in-place class-C restore) remains OPEN → TASK C2. O4 residual: restored volume tar with an OLD credential hash may still need a manual in-DB reset.

2026-07-04 — app-data restore drill → see felhom.eu/documentation/audits/DRILL-appdata-restore-2026-07-04.md. Keep-side restore proven live on 9201 (class-A DB replay + fail-closed data-key gate + non-destruction). F1 (HIGH): UI restore is dead — /api/backup/snapshots has no handler, so the restore button never enables. F2: no one-click in-place class-C (HDD bind-mount) restore. F3: named-volume data never backed up (DumpAppVolumes* has no caller). (F1/F3/O4 resolved in v0.99.0, see above.)

2026-07-03 — CLAUDE.md slimmed to stable orientation (full package map, verified 9201 deploy summary, no version-pinned state). Deep runbooks/design/testing doctrine now in the personal skills felhom-build-deploy / felhom-ui-design / felhom-testing (source: felhom.eu/skills/).

2026-07-03 — REUSE.md exists at the repo root (canonical helpers / patterns / traps / seams, code-verified). Check it before writing new code; update it in the same commit that adds/changes a shared helper (maintenance rule now in CLAUDE.md).

2026-07-02 — v0.98.0 (deployed on 9201): Tárhely IA follow-up. User feedback on D1: NAS-add and local-drive enrollment buttons sat side by side — confusing. /storage split into two subpages under the Tárhely nav item (nested sub-links, .nav-links-nested): /storage = Meghajtók (drives + agent view + wizards + manual add) and /storage/network = Hálózati tárhely (NAS) (NAS-megosztások list + add form; own openDialog copy). networkStoragePageData (page key storage-network) split out of storagePageData. No API/storage-semantics change.

2026-07-02 — v0.97.0 (deployed on 9201): TASK-D1 — settings split + Tárhely page (IA only). The 1451-line settings.html monolith is split into four pages: /storage (new main-nav Tárhely: Adattárolók + NAS + unified agent drive view + wizard entry), /settings (Rendszer: konfig, verzió/frissítés, vezérlő+kiszolgáló újraindítás), /settings/notifications (Értesítések

• Alkalmazás-email), /settings/security (Jelszó, Földrajzi korlátozás, Vészhelyzeti információk). Sidebar gains the Tárhely item + a "Beállítások" group with three sub-links. No storage/agent/API behavior change — pages moved, logic didn't; every /api/storage/* endpoint + POST action unchanged. Decisions: (a) storage flashes + wizards moved to /storage; old /settings/storage/{init,attach} 301 to /storage/{init,attach}. (b) The two duplicated drive views MERGED: registry cards render server-side, then JS enriches each connected user-data card in place from the agent /api/disks (role tag, durable-id, agent actions) joined on mount path, plus two groups — Rendszermeghajtók (system/backup, read-only, lock tag, NO actions) and Nem regisztrált (register only); agent-down → one warn note, registry cards still render. (c) Every native confirm()/prompt() on the four pages now routes through a light .confirm-overlay (openDialog; texts verbatim; type-to-confirm kept only where it existed) — the D0 tab-freeze is gone. Contract for future UI: new templates must pass scripts/template_id_gate.py (every JS element-ID resolves in its own template) and scripts/emoji_gate.py (0 emoji — the D0 grep gate false-negatived multibyte emoji on Windows; 8 survivors removed). .badge-lock/.lock-ico CSS deleted; other .badge* still used on secondary pages (D2 to finish). Next: D2 (badge→tag on backups/secondary pages; storage.html is still ~700 lines — NAS-add partial candidate). NOT live-validated: agent-down degradation (don't stop the live agent — static/unit only); destructive storage ops via the moved overlay paths (endpoints unchanged; supervised session).

2026-07-02 — v0.96.0 (deployed on 9201): TASK-D0 — design system v2 re-skin (appearance only). The whole customer UI (dashboard AND setup wizard) now renders in the approved v2 language: navy token palette (--bg-0/1/2, --line, --text-1/2/3, --blue, --warn, --crit), single 2px radius, no shadows; exception-based status color — nominal is blue/neutral, amber/red only on deviation; zone bars → the hairline .meter (neutral 70/85 ticks, warn/crit flag „Fogyóban a hely" / „Kritikusan kevés hely"); pills → .tag / .metarow. Decisions: (a) stateColor semantics changed — stopped/exited = neutral, NOT red (operator-approved; failures surface via unhealthy

• alerts); restarting = warn; outputs are now run/progress/warn/neutral/off and nominal/warn/crit — any new template MUST use these suffixes (truth tables guarded by unit tests, stateLabel copy frozen byte-identical). (b) Fonts (PJS + JBM variable woff2, latin+latin-ext) and a 30-icon Lucide sprite are vendored in the binary (/static/fonts/, templates/icons.html) — no CDN; fonts.googleapis.com, box-shadow, 999px and the emoji set are banned strings (grep gate in REPORT §5, 143→0). (c) Setup wizard handleCSS bug fixed: it read a dataDir-derived path that never exists in the container → always served minimalCSS in production; now serves embedded web.StyleCSS(). (d) Found+fixed pre-existing v0.93.0 bug: /backups 500'd once an off-box backup had run (OffboxTarget.LastRun is an RFC3339 string fed to timeAgo; new timeAgoStr). Canonical reference: felhom.eu/documentation/design/design-system.md. Next: D1 (settings IA split + Tárhely main-nav promotion; also migrate the remaining native confirm() dialogs to the type-to-confirm overlay and finish badge→tag markup on secondary pages). NOT live-validated: setup wizard visuals (unit-only), warn/crit meters on real hardware (demo healthy).

2026-07-01 — v0.95.0 (deployed on 9201): Impl-2b — raw-drive enrollment wizards. Both enrollment wizards (storage_init.html / storage_attach.html) now fetch candidates from the agent's raw-device scan GET /api/disks/candidates (proxy agentDiskCandidatesHandler → agent Impl-2a) instead of the Observe()-based /api/disks, so a brand-new (non-PVE-storage) drive is discoverable + enrollable. agentapi.ListCandidates + resolveEnrollUUID (a raw candidate isn't in /disks → resolve its fs-UUID from the scan). Live end-to-end on felhom-pve: raw /dev/sdd (SD card) → wizard → format (Impl-1 guard) / attach → mount → bind → intent → guest-bind → "Aktív" in the UI (3rd drive, no false detach). Needed a chain of AGENT fixes (v0.56–0.58: durableIDForMount / ReassertGuestBinds / HostReader wiring / /disks GuestPath+BoundUnderParent for registry rows) — the whole enroll+tracking machinery had assumed a PVE-storage drive. Follow-ups: runStorageInit should poll the agent's detached-format status (/disks/format/status) so a SLOW-device format completes in one flow (pre-existing); Impl-3 shared-box operator format gate.

2026-06-30 — v0.94.0 (deployed on 9201): pull-based config-refresh. The hub report ACK now also carries a per-customer config_version (hub v0.26.0; a stored counter bumped on every config save). OnPushResponse → ConfigRefresher.Reconcile (internal/report/config_refresh.go): on a change vs. settings.applied_config_version, bootstrap.RefreshConfig re-pulls controller.yaml (re-merging local_api from bootstrap.json; overwrites controller.yaml, never settings.json) → record → graceful self-restart (api.GracefulSelfRestart). First-run records baseline (no restart); unchanged = no-op (no storm); failed pull keeps config + retries. This is the box-pulls-config replacement for the hub's retired inbound "Push Config" — the hub never connects into the box. Companion hub v0.26.0 also retired Trigger Update / Pull Config / Show Diff and dropped geo-disable's inbound notify (kept the hub→Cloudflare WAF removal), and fixed the stale docker-setup.sh setup command → host-install. Live-validated on 9201: first-run baseline (applied=1, no restart) → DB bump 1→2 → re-pull+self-restart (session_secret rotated, local_api preserved, RestartCount 0→1) → 4 cycles no-loop → apps stayed up.

2026-06-29 — Self-health arc complete (agent v0.48.0 + hub v0.22.1). The hub now proactively watches every agent's served local-API leaf fingerprint fleet-wide: the agent reports leaf_fingerprint on its host report (v0.48.0), and the hub HostLeafChecker (monitor/host_leaf.go) raises host_leaf_changed when it changes (trust-on-first-report baseline; empty fp = unknown, never alerts; reads from report_json, no migration; hub-generated so no allowlist change). Independent of the controller channel-check. Live: a leaf regen raised host_leaf_changed (82078fab…→60b5974d…) AND the controller's agent_channel_pin_mismatch — complementary. NOTE: v0.22.0 shipped the checker but the main.go wiring never applied (concurrent file-touch); the LIVE TEST caught it (no event on regen) → fixed in v0.22.1. The original silent-multi-day-outage incident class is now caught from THREE angles (agent capabilities v0.44.0 / controller channel-check v0.90.0+F2 / hub leaf-fp v0.22.x) AND prevented (agent v0.46.0 loud-regenerate + host-install --preserve-state-from). Backlog: the served-fp-vs-pinned-fp authoritative cross-check; the test-run coverage gaps (capability→hub live, host-reboot doubling).

2026-06-29 — v0.91.0: F2 (channel-health born-down alerting). The v0.90.0 channel-health check alerted only on a live up→down transition; a channel broken at startup/reseed (e.g. the controller boots right after a leaf regen → first observation is pin_mismatch) was dashboard-only, no operator email — forever. Fixed with an alerted flag in channelhealth/checker.go: a confirmed down coming from up/unseeded OR a reason-change re-arms then alerts once; steady-down no dup; recovery re-arms; debounce intact (transient born-down still N≥2; healthy first-obs silent). Live-validated: controller restarted into a regenerated-leaf channel → [channel] DOWN (unseeded->down:pin_mismatch)

• Event pushed: agent_channel_pin_mismatch (~60s) — the capstone's missing half. Companions: hub v0.21.0 (same fix for HostCapabilityChecker/HostStalenessChecker — seed only healthy, born-degraded/ stale emits on first Check, cooldown dedups), agent v0.46.0 (EnsureLeaf loud-WARNs a REGENERATED leaf
• host-install --preserve-state-from/populated-host guard = the prevention for the original incident class). Self-health story now complete: agent watches its capabilities, controller watches its link, detection works at startup, and a reinstall can preserve the leaf. Backlog: F1 swap-rollback RestartCount hardening; full live reinstall (supervised); hub-side leaf-fp fleet comparison.

2026-06-29 — v0.90.0: controller→agent channel health-check (self-health slice). A ~60s scheduler job (internal/channelhealth) probes the local-API channel via the PRODUCTION memoized client (Server.ProbeAgentChannel → s.agentClient() + GET /storage — NOT a fresh client, per the spike: self-heals, mirrors the disk UI, no transport leak). Classifies failures (spike Q1 map: pin_mismatch/unauthorized/unreachable/timeout/misconfigured/construction_error/unknown), debounces transient reasons (refused/timeout need N≥2 consecutive so the ~1s agent-restart blip doesn't page; pin/401/DNS/construction alert first-obs), seeds the first observation silently, and on a transition emits an English operator-only event (Notifier.NotifyAgentChannelDown/Recovered) + a Hungarian dashboard banner (AlertManager.SetAgentChannelAlert). Closes the R1-incident gap (channel was only checked once at startup). Required a hub change (felhom-hub v0.20.0): the agent_channel_* event types were rejected by the hub's allowedEventTypes allowlist (HTTP 400) — controller-pushed events are gated, unlike the hub-generated host_* ones. Live-validated: transient restart → no alert; sustained stop → debounce(1/2) then DOWN transition + dashboard banner; start → recovered. No agent change. Self-health story now: agent watches its own capabilities (v0.44.0) + controller watches its link to the agent (this). Backlog: hub-side leaf-fp comparison.

2026-06-29 — OPS (no code change): controller↔agent TLS leaf-pin mismatch RESOLVED via R1. The 2026-06-28 root→non-root agent migration regenerated the agent's local-API leaf (60b5974d…→ced34036…) and truncated its token store, so the in-guest controller's pinned agentapi client failed every call (TLS pin mismatch) — disk UI dead AND the quiesce app-consistent-backup loop dead. Fix: restored the pre-migration local-api.{crt,key} + local-tokens.log from /root/agent-backup-bundle-test/aside-var-lib/ on felhom-pve (R1 RUNBOOK, supervised). Agent now serves 60b5974d… again (== controller pin + bootstrap.json seed) and the restored token store's 9201 entry matches the controller's current local_api.token — zero in-guest change. Gates passed: leaf loaded (not regenerated), wire fp 60b5974d…, authenticated GET /disks 200/5-disks (bogus token 401), and the next quiesce cycle ran a real backup job. Rollback point kept at /root/agent-rollback-20260629-150137/. OPEN (carried forward): the migration/Day-0 path must carry these three files into the new felhom-agent-owned data dir (or treat their loss as a re-bootstrap trigger), else any re-migration reintroduces this. Full record: felhom.eu/documentation/audits/SPIKE-agent-pin-mismatch-2026-06-29.md (§9 Outcome).

2026-06-27 — v0.86.0 (deployed on 9201): Phase 2 managed updates — controller-version FLOOR. On top of the Phase 1 opt-in "update to latest" button, the controller now honors an operator-enforced minimum version (FLOOR) delivered on the hub report ACK (min_controller_version + latest_version; internal/report/pusher.go PushResponse). OnPushResponse → updater.SetFloor() + updater.MaybeAutoUpdate() (rides the report cycle — no new timer). Below the floor → auto-update to the floor (reuses Phase 1 performUpdate: in-guest pull → agent swap → rollback; initiatedBy="auto-floor"). At/above floor → nothing (does NOT chase latest — that's the button). Guards: dev/no-agent/backup → skip; floor must be pullable (floor ≤ latest; floor>latest → warn+noop); one attempt per below-floor condition (in-mem + persisted state) → no flapping. Floor source

• operator UI are hub-side (felhom-hub v0.15.0: per-customer override + global default + report ACK). No agent change (reuses Phase 1 POST /controller/swap). Day-0 now ships current (golden rebuilt at 0.85.1) AND stays current (floor) — the fleet-currency story is closed. Live: dogfood 0.85.1→0.86.0 via the button, then floor 0.87.0 → auto 0.86.0→0.87.0 (no click).

2026-06-26 — v0.84.0 (deployed on 9201): show an app's auto-generated first-login on its page. General, catalog-driven mechanism: .felhom.yml initial_credentials: {file, format json|regex|plain, username_key/password_key | username_pattern/password_pattern, note}. The controller reads the file live from the container (internal/stacks/initialcreds.go ReadInitialCredentials + pure parseInitialCreds, unit-tested), never persists it, and renders a "Kezdeti belépési adatok" card on /apps/{slug} (masked password + reveal/copy). First consumer: crafty-controller (Crafty writes a random admin password to /crafty/app/config/default-creds.txt). Live-verified on 9201: card shows username admin + the real extracted password. Reuse for any future self-seeding app. Security: same exposure class as the existing post-deploy reveal / default_creds card — relies on the prod dashboard being auth-gated (demo public-unauth is the separate tracked issue). Backlog idea unchanged: a backend_scheme hint for TLS backends (v0.83.0 line).

2026-06-26 — v0.83.0 (deployed on 9201): scoped Traefik backend transport for self-signed HTTPS apps. Crafty is the first/only catalog app with an HTTPS backend (self-signed TLS on :8443, no plain-HTTP port). The crafty healthcheck fix un-withheld its route, exposing a pre-existing 502 (Traefik proxied HTTP to the HTTPS backend). Since traefik v3 forbids insecureSkipVerify via Docker labels, the controller now renders a file-provider dynamic file dynamic/serverstransports.yml defining a named insecure-skip-verify transport (infra.RenderServersTransports + stacks.ensureServersTransports, written from EnsureBaseStack outside the traefik-running guard, write-if-changed, hot-loaded). Apps opt in per-service via catalog labels scheme=https + serverstransport=insecure-skip-verify@file. No global insecureSkipVerify — verification stays ON for every other backend (scoped Option B). Pattern to reuse for any future self-signed HTTPS backend. Live-verified: minecraft.demo-felhom.eu 502→302; filebrowser (HTTP) unaffected. Backlog: a .felhom.yml backend_scheme hint so the catalog convention sets these labels instead of hand-adding.

2026-06-22 — v0.76.0 (deployed on 9201): three campaign-#3 hardening fixes.

• S1: corrupt settings.json no longer crash-loops — save() writes a last-known-good .bak (after the primary rename); Load() recovers from .bak, else preserves the corrupt file as *.corrupt-<ts> + safe defaults. New Settings.LoadWarning → dashboard banner. (main.go Fatalf now only on the IO-unreadable path.)
• F2: web.validStackName gates backupRestoreHandler + apiExportStart (reject / \ .. NUL) before any restore/export — closes the traversal defense-in-depth gap (no escape had occurred, but it relied on downstream map-lookups).
• S3: quiesce.readMarker logs [WARN] + quarantines a corrupt marker to *.corrupt-<ts> instead of silently dropping it. All three live-validated on 9201 (truncate settings → recover from .bak no crash-loop; traversal restore → rejected, /etc intact; corrupt quiesce marker → quarantined). Tests + red-proofs. Still-open campaign-#3 deferrals (NOT done): time-chaos on a dedicated VM, host reboot (supervised), .fab import compose-fuzzing. See felhom.eu/documentation/tests/test-campaign-3-2026-06-22-findings.md.

Last updated: 2026-06-22 (v0.75.0 — gate userdata MkdirAll on a live mountpoint)

2026-06-22 — v0.75.0 (deployed on 9201): userdata MkdirAll gated on a live mountpoint. Both MkdirAll-into-<drive>/userdata sites — the deploy belt (stacks.ensureUserdataMounts) and the FileBrowser sync (web.syncFileBrowserMounts) — now skip when an external drive root (under StableParentDir=/mnt/felhom-drives, != sysDataPath) is not a live mountpoint (system.IsMountPoint). Closes the campaign-#2 hazard where a drive-absent window produced mkdir …/userdata: permission denied

• transient Created flapping AND could write app data onto the guest rootfs (shadowed when the drive returns). The app is held by planDriveGates instead. System/local path is never gated. New Manager.isMountPoint seam + pure web.skipFileBrowserPath helper for tests. Live-proven via a C6 re-run: 0 permission-denied, 0 shadow dirs on the rootfs, apps recover on reconnect.

Boot-ordering design note (NOT implemented — decide separately): the boot-time mkdir … permission denied is a different cause — docker's boot-restore auto-starts drive-backed containers (restart: unless-stopped) before the agent mounts the drives, so docker (not the belt) tries to create the bind source; planDriveGates recovers them after mount convergence (why a reboot ends healthy). Options: (A) accept + suppress (apps self-heal; lowest risk; maybe downgrade the boot-window log level) — recommended now; (B) drive-backed app containers don't docker-auto-start at boot (restart policy no/on-failure) so the controller's gate is the sole starter after mounts converge — cleaner but bigger (must cover crash-restart too). See felhom.eu/documentation/tests/test-campaign-2-finding1-recovery-diagnosis.md (boot-ordering observation).

Last updated: 2026-06-22 (v0.74.0 — controller→agent connection-leak fix)

2026-06-22 — v0.74.0 (deployed on demo guest 9201): fixed the controller→agent connection leak. Server.agentClient() built a new agentapi.Client (new bare http.Transport, IdleConnTimeout:0) per call → one leaked idle ESTABLISHED socket per agent call to 192.168.0.162:8443, exhausting the ephemeral source-port range after ~5 days of controller uptime → EADDRNOTAVAIL, which had taken down storage UI + host-metrics + whole-guest backup (found in the 2026-06-22 unattended campaign). Fix: memoize ONE shared client via sync.Once + harden the Transport (MaxIdleConnsPerHost:2, IdleConnTimeout:90s). Live-proven: idle sockets to :8443 stay flat at 2 across a 120-call burst (pre-fix grew ~1/call → 132). Agent/firewall untouched. Diagnosis + campaign findings live in felhom.eu/documentation/tests/unattended-test-campaign-2026-06-22-*.md. Separate open item: the defense-in-depth host firewall rule scoping :8443 to the guest bridge subnet is still absent (pve-firewall disabled) — to be closed independently.

Last updated: 2026-06-13 (v0.60.0 backlog-Medium cleanup)

Live version: controller v0.60.0 (deployed on demo guest 9201), agent v0.30.0 (AGENT-001 deployed), hub v0.11.0. NOTE: the long-form sections far below this banner are stale (last full pass ~v0.16.1). Current state is the dated entries immediately below + CHANGELOG.md + the now-authoritative central docs at felhom.eu/documentation/controller/ (code-verified) + auto-memory MEMORY.md.

2026-06-13 — v0.60.0 backlog-Medium cleanup (BUGHUNT reconcile):

• M25 (Server.integrationMgr data race — constructor goroutine vs post-construction SetIntegrationManager): FIXED via atomic.Pointer; -race-verified. M4/M5/M6 verified already FIXED (no action).
• M18 (dump re-validation every 5 min — perf) and M19 (deriveStackName misattribution — low incidence) verified LIVE but cross-package-entangled → branches fix/m18-dump-validation-cache / fix/m19-stackname-crossref (notes + fix plan, PENDING REVIEW, not deployed).

2026-06-13 — v0.59.0 audit fixes + documentation centralization:

• Fixed the validated 2026-06-13 audit findings (records: felhom.eu/documentation/audits/): CTRL-001 (.fab import path traversal — manifest segment validator, fail the parse); CTRL-T2-1 (ghost-deployed on crash — app.yaml now persists deployed:false until compose up -d succeeds, flipped true only on success; in-memory flag still true during pull for UX); H10 (plaintext secret on encrypt failure — SaveAppConfig now fail-closed, returns an error); M2 (misleading lock on init-only SetStackProvider removed). All with regression tests.
• AGENT-001 (wrong-disk wipe TOCTOU) fixed on agent branch fix/agent-001-wipe-durable-reresolve — PENDING REVIEW, NOT deployed (supervised merge+golden-rebake reserved; see AGENT-001-FIX-NOTES.md).
• Built + deployed v0.59.0 to demo guest 9201 (bootstrap mechanism); verified healthy, dashboard 200.
• Documentation centralized under felhom.eu/documentation/ (controller subtree + audits + top index), code-verified; controller/README.md banner points there; this CONTEXT banner refreshed.

2026-06-13 — v0.58.0 OS/Docker-data split prevention layer (Phase 2; Phase 1 = agent v0.29.0):

• OS rootfs + Docker data split onto separate local-lvm volumes (golden bakes 32G rootfs + 256G /var/lib/docker, backup=1, overlay2 so images live on the data vol). Infra protected by PREVENTION not placement: system.GetDockerVolumeHeadroom() (statfs "/" = data vol) reserves max(5GB,10%); deployStack refuses HTTP 507 below buffer; deploy.html banner+disable; monitor (warn80/crit90) watches the same vol; log rotation baked into the golden daemon.json.
• Live-validated by DESTROYING + RE-PROVISIONING 9201 from the split golden: split layout, overlay2, images on data vol, lean rootfs, OS isolation (data vol 100% → rootfs healthy), deploy gate 507, ActualBudget volume on data vol, hub config pull, external access via CF. RomM/USB re-enroll = the documented final restore step (RomM data safe on host USB). See memory os-data-split.

2026-06-13 — v0.57.0 UI fixes (Part A of the UI-fixes/storage-spike spec):

• A1: fixed the RIGHT storage list — #host-storage-bars (the JS-filled, agent-PVE-storage list: local/local-lvm/felhom-pbs/felhom-usb), which reordered on every poll. Now enrichHostStorageTargets sorts /api/host-metrics server-side + adds friendly Hungarian labels/purpose. Display-only — PVE storage ids never renamed. (v0.56.0's 4C had sorted the OTHER, server-rendered user-data list.)
• A2: per-app Tier-2 config panel at GET/POST /stacks/{name}/backup; the dead-end "Beállítás" button (was → deploy page) is repointed there. Pin a target drive / toggle Tier 2 off; prefs (UserDisabled/PreferredTarget) persist on CrossDriveBackup and survive the runner's status writes (withTier2Prefs). Always visible incl. single-SSD + non-HDD (PBS-context) apps.
• Part B (storage OS/data split spike) = build-nothing; findings → felhom-agent/REPORT-storage-split-spike.md.
• Live-validated on guest 9201; build/deploy = golden bootstrap (/etc/felhom-controller-image + restart felhom-controller-bootstrap.service).

2026-06-13 — v0.56.0 Phase 4: FileBrowser scoping + UI polish (SLICE COMPLETE):

• 4A: FileBrowser bind scoped to <drive>/appdata (recovery units + Tier 2 copies under backups/ NOT mounted → customer can't browse/delete the restore source). 4B: deploy storage step states files-on-drive / DB-on-fast-SSD. 4C: buildStorageBars stable sort + purpose description on the monitoring list (user-data drives only; agent local/local-lvm/pbs live on the storage page, not here).
• Live-validated (9201): FileBrowser mount /mnt/felhom-usb/appdata -> /srv/felhom-usb (backups hidden); deploy + monitoring text rendered. All 5 phases (1, 2, 2b, 3, 4) shipped + live-validated, v0.52→v0.56.

2026-06-13 — v0.55.0 Phase 3: auto off-drive Tier 2 (rootfs-headroom guard):

• internal/backup/tier2.go: rsync -a --delete of each HDD app's recovery unit + appdata → a DIFFERENT physical disk (<target>/backups/secondary/<app>/). Auto target: prefer another registered drive (off-disk via system.SamePhysicalDevice), else internal SSD for SMALL units only.
• Rootfs-headroom guard (tier2FitsHeadroom, unit-tested): SSD = ~8G guest rootfs, so REFUSE unless the unit fits leaving reserve = max(2G, 20%) free; honest "needs 2nd HDD" status when nothing fits — never fills the rootfs. Status via surviving settings.CrossDriveBackup; "2. mentés" UI card now populated (buildAppBackupRows). Daily tier2-backup 03:30 + POST /api/backup/tier2.
• Live-validated (9201): happy path (RomM → SSD, off felhom-usb, 77KB, "[SSD: DB/config only]"); refuse path (1G userdata dummy → REFUSED with honest msg, rootfs not filled); UI card shows "Sikeres → belső SSD (csak DB/konfiguráció)". Demo cleaned.
• Next: Phase 4 (FileBrowser scoping + deploy-UI DB-on-SSD note + monitoring sort).

2026-06-13 — v0.53.0/v0.53.1 Phase 2: per-app recovery unit (capture side, SECRET-FREE):

• Each app's backups/primary/<app>/ becomes a self-contained recovery unit: compose/ (docker-compose.yml + .felhom.yml + secret-stripped app.yaml) + db-dumps/ + volume-dumps/ + manifest.json (image pins, secret env-var NAMES, data_key names, checksums, secret_source note).
• Secret-free by design. Decided after reading the ACTUAL hub code: hub is zero-knowledge (no app secrets); app.yaml + key live on the guest rootfs → in the PBS whole-guest snapshot. So the unit stores no secret/data-key/image; restore recovers secrets from the guest's app.yaml (live/PBS), regenerates nothing. data_key (DeployField.DataKey; AdventureLog SECRET_KEY marked) = fail-closed restore annotation only.
• Capture needs no decryption (non-secret env is plaintext; excludes secret-named + encrypted keys). Wired into RunDBDumps AND the periodic RefreshCache (idempotent checksum-skip → no USB thrash).
• Deploy mechanism resolved: controller in guest 9201 is golden/bootstrap-managed — felhom-controller-bootstrap.service docker-runs the tag from /etc/felhom-controller-image (gitea anon-pull). Deploy = build+push → anon-pull → update tag file → restart the service.
• Live-validated (9201): RomM unit captured (images=3, secrets=3, data_keys=0), secret-leak grep = NO_LEAK.
• v0.54.0 Phase 2b (restore-from-unit + fail-closed gate): RestoreFromRecoveryUnit recreates an app from its unit + secrets recovered from the GUEST's live app.yaml (RecoverStackSecrets, stacks.RedeployFromEnv), regenerating nothing. reconcileRestoreSecrets (pure, unit-tested) is the fail-closed gate: missing/empty data-key → REFUSE (needs PBS whole-guest restore); missing resettable secret → warn+proceed. Wired into /backup/restore. Gate + orchestration + data_key parsing unit/integration-tested; deployed v0.54.0 healthy.
• LIVE-validated (9201, AdventureLog): unit manifest data_key_env_vars:[SECRET_KEY] (catalog→manifest live); with SECRET_KEY made unrecoverable, POST /backup/restore REFUSED with the exact fail-closed message BEFORE any compose-up. Demo has NO dashboard password → API open (auth+CSRF skipped), driven via public URL. NOTE: full deploy-with-data→restore e2e blocked because AdventureLog images don't fit the 8G guest rootfs ("no space left") — that's the Phase 3 rootfs-headroom concern seen live. Demo left clean (AdventureLog reverted to not-deployed).
• Next: Phase 3 (Tier 2 auto off-drive, rootfs-headroom guard), Phase 4 (FileBrowser + UI).

2026-06-13 — v0.52.0 Phase 1 GATE: deploy-side double-nest fix (catalog) + path-agreement test:

• The felhom-data double-nest lived in the app-catalog compose templates (${HDD_PATH}/felhom-data/appdata/<app>), not in deploy.go. On a Model-A in-guest drive the mount already IS the felhom-data namespace, so it double-nested on disk while the v0.51.0 backup helpers resolved single-nested → divergence. Fixed all four HDD templates (romm, nextcloud, immich, paperless-ngx) → ${HDD_PATH}/appdata/<app>.
• New internal/stacks/hddpath_agreement_test.go locks deploy-resolver (ParseComposeHDDMounts) == backup helper (AppDataDir(NamespaceRoot(.,true))). No controller runtime change → no image rebuild (deployed stays 0.51.0, functionally current; golden not rebaked for a no-op).
• Live (guest 9201): git-sync auto-delivered the fix to all four stack files; RomM migrated (stop→move→verify→redeploy) from /mnt/felhom-usb/felhom-data/appdata/romm → /mnt/felhom-usb/appdata/romm, healthy + HTTP 200, no data loss, old namespace empty. GATE PASSED.
• Next: Phase 2 (per-app recovery unit), Phase 3 (auto-enabled off-drive Tier 2 w/ rootfs-headroom guard), Phase 4 (FileBrowser scoping + deploy-UI DB-on-SSD note + monitoring sort).

2026-06-12 — storage UX polish (v0.45.0), pairs with felhom-agent v0.24.0:

• Agent eject role-gate (Part A, felhom-agent v0.24.0): POST /disks/eject now refuses to unmount system/backup storage at the agent (fail-safe to protected on ambiguity) — the UI hiding the button was never the control. Validated live on guest 9201 (eject /var/lib/vz → 403, no unmount).
• Controller (Part B, v0.45.0): B1 deterministic /api/disks order (user-data→system→backup, alpha within); B2 init wizard excludes mounted drives; B3 Regisztrálás primary action for a mounted-but-unregistered user-data drive (POST /api/storage/register); B4 per-card purpose descriptions + app-backing tags + tiering note (local & local-lvm both kept); B5 eject already names affected apps. All validated live on guest 9201.
• Golden REBAKED to controller 0.45.0 (/root/build-golden.sh; gitea allows anon pull, no creds); archive local:backup/vzdump-lxc-9100-2026_06_12-09_53_03.tar.zst, build guest 9100 purged.

---

About Viktor (project owner)

• Works at Deutsche Telekom (Budapest), building Felhom.eu as a side business
• Felhom.eu: managed home-server service for Hungarian households
• Technical but prefers pragmatic solutions over over-engineering
• Runs all infrastructure on Gitea (gitea.dooplex.hu), k3s cluster for management
• Customer deployments use Docker Compose (not Kubernetes) for simplicity

felhom-controller (this repo)

• Version: v0.16.1
• Phase 1: ✅ COMPLETE — Stack Manager + Deploy Flow
• Phase 2: ✅ COMPLETE — Monitoring & Health (scheduler, CPU/temp, healthchecks.io pings)
• Phase 3: ✅ COMPLETE — Backups (DB dumps, restic integration, manual trigger, dedicated backup page)
• Phase 4: ✅ COMPLETE — Monitoring Page with Metrics Store (SQLite, Chart.js, system + container metrics)
• Phase 5: ✅ COMPLETE — Authentication, Persistence & Settings Page (settings.json, password change, session management)
• Phase 6: ✅ COMPLETE — Monitoring Warnings, Dashboard Alerts & Notification System
• Phase 7: ✅ COMPLETE — Storage Overview, Per-App Backup Toggles & Limited Restore
• Phase A: ✅ COMPLETE — Storage Paths Foundation (registry, auto-discovery, per-app HDD_PATH, deploy dropdown, health monitoring)
• Phase B: ✅ COMPLETE — Storage Management UI Polish & Health Severity Fix (flash messages, label editing, app details, FS info, deploy free space, backup context)
• Phase C: ✅ COMPLETE — Storage Init Wizard, Data Migration & Startup Fix (disk scan/format/mount wizard, rsync-based migration, startup pings)
• v0.11.1 bugfix: ✅ COMPLETE — Storage Scan: system disk detection via host fstab + blkid UUID resolution; FSType enrichment via blkid -o export
• v0.11.2 bugfix: ✅ COMPLETE — /host-dev mount for block device access; HostDevicePath() helper; all format/scan/safety ops use /host-dev
• v0.11.3 bugfix: ✅ COMPLETE — Added fdisk package to Dockerfile (provides sfdisk; not in util-linux on Debian bookworm)
• v0.11.4 bugfix: ✅ COMPLETE — FormatAndMount: fixed sfdisk (wipefs+force+,,), mount (explicit device path), mount propagation (rshared), ASCII label, smart partition skip, findmnt verification
• v0.11.6: ✅ COMPLETE — FileBrowser auto-mount sync (syncFileBrowserMounts()) + 3 UI fixes (badge color, progress bar, button text)
• v0.11.7: ✅ COMPLETE — Stale data cleanup + FileBrowser sync after migration + deploy page title fix
• v0.11.8: ✅ COMPLETE — Per-App Cross-Drive Backup (3-2-1 rule): rsync/restic to secondary drive, deploy page UI, backup page summary, scheduler jobs, API endpoints
• v0.11.9: ✅ COMPLETE — UI Polish Fixes: spacing, tooltip on "Módszer", status dot instead of disabled checkbox, progressive disclosure, emoji cleanup
• First app deployed: Paperless-ngx on demo-felhom.eu (2026-02-13)
• Running on: demo-felhom (N100 mini PC) at 192.168.0.162:8080, felhotest (Proxmox VM) at router.abonet.hu:33022
• All Phase 1-5 features working: deploy, start/stop/restart/update, logs, health-aware states, auth, monitoring, backups, backup detail page, system monitoring page, settings page

Architecture decisions

Decision	Rationale
Go stdlib for web (no Gin/Echo)	Minimal dependencies, single binary, easy to embed templates
Templates as go:embed HTML/CSS files	Zero runtime file dependencies (compiled into binary), but each template is a separate editable file
Docker Compose for customers (not k8s)	Simpler troubleshooting, customers don't need k8s knowledge
k3s for management infra only	Viktor's own services (gitea, monitoring, website) run on k3s
Cloudflare Tunnel for remote access	No port forwarding needed, works behind any NAT
app.yaml per stack	Separates deploy config from compose files, survives git pulls
Password fields require explicit input	Prevents accidental empty-password deployments
Health-aware state from Docker Status field	Docker's State says "running" even for unhealthy containers
Memory limits via deploy.resources.limits	Prevents runaway containers; ~50% headroom over expected usage
System info from /proc/meminfo + statfs	No external dependencies, cheap to read on each page load
mem_request vs mem_limit (K8s-inspired)	Requests = expected usage (hard block), limits = peak (overcommit OK)
384MB reserved for system	Prevents deploying apps that would starve the OS/controller
Logo SVG embedded as Go constant	Same approach as CSS/HTML — zero external file deps
Git sync via os/exec git CLI	No Go git library needed, git is in the container image
SHA-256 for content comparison	Only copy changed files, avoid unnecessary disk writes
30s debounce on manual sync	Prevents spamming the git server
Orphan = deployed but not in catalog	Safe lifecycle: remove from catalog → mark orphaned → user deletes via UI
FileBrowser as infra (not catalog)	Needed even after apps deleted (user browses HDD data); deployed by setup script
Protected HDD paths	Safety net: never delete top-level HDD dirs (media, storage, Dokumentumok, appdata)
Central scheduler (not ad-hoc goroutines)	Single place to register/monitor all periodic tasks, graceful shutdown, skip-if-running
CPU sampling via background goroutine	/proc/stat delta needs two readings — collector runs every 5s, GetInfo() reads cached value
Temperature from /host/sys (Docker mount)	Container can't read host /sys directly — mount /sys:/host/sys:ro, try /host/sys first
Restic password auto-generated	No manual setup needed — generated on first backup run, stored in named volume
DB discovery via docker inspect	No config needed — discovers postgres/mariadb containers by image name + env vars
Backup orchestrator with running flag	Prevents concurrent backups, supports both scheduled and manual trigger
modernc.org/sqlite (pure Go)	No CGO/gcc needed in Docker build stage — keeps CGO_ENABLED=0 static binary
AlertManager state-based refresh	Alerts regenerated every 5min from health report — no persistent storage needed, always reflects current state
Notification relay via hub	Controller → hub → Resend → email. Hub acts as central relay: knows customer email, handles Resend API. Controller only needs hub URL + API key
In-memory notification cooldowns	Per-event-type cooldown map (default 6h). Lost on restart = acceptable (better to re-notify than miss). No persistence needed
Health status change detection	Only notify on degradation (ok→warn, ok→fail, warn→fail). Avoids spam on flapping. First run records baseline, doesn't notify
Resend HTTP API (no SMTP)	Direct POST to api.resend.com — same pattern as website contact-mailer. Simpler than SMTP setup, good deliverability
Preferences sync on save + startup	Controller pushes prefs to hub (not pull). Startup sync handles hub DB rebuild. Local save always succeeds even if sync fails
Chart.js embedded locally	Customer hardware may not have internet — CDN not reliable for offline environments
StackDataProvider interface	backup package needs stack data but can't import stacks (circular). Interface in backup, thin adapter in main.go
Password sync to hub via report	Restic password in Docker named volume on SSD. Hub sync provides redundancy for disaster recovery
App backup via HDD mounts only	Docker volumes at /var/lib/docker/volumes/ not mounted in controller. HDD data is the important user data; DB in volumes covered by nightly dump
Restore uses running mutex	Prevents concurrent backup+restore on same restic repo. Reuses existing m.running flag
Storage paths registry in settings.json	Multi-storage support: each app's HDD_PATH from app.yaml is authoritative. Auto-discovery on startup avoids manual config. Registry enables UI management + health monitoring per path
/mnt:/mnt:rw mount in controller	Replaces per-path HDD_PATH mount. Enables multi-storage + restore writes. All customer HDD mounts are under /mnt/ by convention
Per-app HDD_PATH resolution (app.yaml > global)	App's own env HDD_PATH is Priority 1, registered storage paths as fallback. Eliminates dependency on global controller.yaml hdd_path
Mount-point detection via syscall.Stat_t.Dev	Compares device ID of path vs parent dir — reliable check that path is on separate filesystem. Prevents data writes to SSD
Health severity: mount-point = warning	Non-mount-point is informational, not a service failure. FAIL reserved for genuinely broken things. Avoids false alarms on demo/test environments
FS info via findmnt + sysfs	findmnt -n -o SOURCE,FSTYPE --target <path> for filesystem type/device. /sys/block/<dev>/device/model for disk model. Best-effort, returns nil on failure
Query param flash messages	Stateless, no session store needed. Consistent with backup page pattern. ?storage_msg=success&storage_detail=...
StorageLabels map on stacks page	Separate map passed to template (not modifying Stack struct). Built from deployed apps' HDD_PATH → registered path label lookup
Metrics downsampling via SQL	Bucket-based AVG in GROUP BY keeps Chart.js responsive with up to 30 days of data
60s metrics collection interval	Good balance of resolution vs. storage — ~44K rows/month for system metrics
/etc/os-release mounted read-only	Container can't read host OS info directly — mount to /host/etc/os-release:ro

Key file locations on demo-felhom

/opt/docker/felhom-controller/         # Controller compose + config
  ├── controller.yaml                  # Customer config (domain, auth, paths)
  ├── docker-compose.yml               # Controller's own compose
  └── data/                             # Controller persistent data (named volume)

/opt/docker/stacks/                    # All app stacks
  ├── traefik/                         # Reverse proxy (protected)
  ├── cloudflared/                     # Tunnel (protected)
  ├── paperless-ngx/                   # First deployed app ✅
  │   ├── docker-compose.yml
  │   ├── .felhom.yml                  # App metadata
  │   └── app.yaml                     # Deploy config (env vars, locked fields)
  └── whoami/                          # Test stack (not deployed)

/mnt/hdd_placeholder/storage/          # HDD storage for apps
  └── paperless/
      ├── consume/                     # Drop files here for OCR
      ├── media/                       # Processed documents
      └── export/                      # Backup exports

Related repositories and their state

Repository	Status	Notes
felhom-controller	Active	This repo. Controller code + deploy scripts
app-catalog-felhom.eu	Active	10 app templates, all with .felhom.yml metadata + memory limits
felhom.eu	Active	Website + hub/ subfolder (felhom-hub service) + k8s manifests
homelab-manifests	Stable	k3s cluster running (dooplex.hu services)
misc-scripts	Utility	collect-repo.sh, backup helpers

Gotchas & lessons learned

• docker compose restart ≠ docker compose up -d — restart doesn't pick up new images
• Go maps have random iteration order — always sort slices before displaying
• Docker .State="running" doesn't mean healthy — check .Status for "(health: starting)" / "(unhealthy)"
• Paperless-ngx needs PAPERLESS_OCR_LANGUAGES (plural) to install language packs, PAPERLESS_OCR_LANGUAGE (singular) to select
• In-memory Deployed flag must be set BEFORE docker compose up -d (not after) — compose can take 30-60s for image pulls, during which the UI would show a stale "Telepítés" button
• Cloudflare Tunnel handles *.demo-felhom.eu → Traefik handles Host()-based routing to containers
• BIOS "AC Power Recovery" must be enabled on N100 for auto-restart after power outage
• docker compose up -d returns exit 0 even when containers immediately crash-loop — need post-start status check to detect this
• When logging env vars for debugging, only log keys (not values) to avoid leaking secrets in log files
• Mealie image (ghcr.io/mealie-recipes/mealie) doesn't include wget/curl — use Python TCP socket check for healthcheck
• Mealie DB migrations on first start take ~40s (alembic) — use start_period: 60s to avoid premature unhealthy status
• Alpine-based images (filebrowser, vaultwarden) have wget via BusyBox — healthchecks with wget --spider work fine
• Deploy sed command to update image version must target only the image: line — naive sed 's|name:OLD|name:NEW|' also matches the service name line (e.g., felhom-controller: → felhom-controller:0.2.12), breaking YAML. Use sudo sed -i 's|image:.*felhom-controller:[^ ]*|image: ...felhom-controller:NEW|' or similar scoped pattern
• Hungarian quotation marks „" in YAML: „ (U+201E) is safe inside YAML double-quoted strings, but the closing " must NOT be ASCII " (0x22) — it terminates the YAML string. Use \" escape or Unicode " (U+201D). This caused a silent parse failure for the entire .felhom.yml file
• Never silently swallow parse errors — always log them. Silent failures make debugging impossible (took a dedicated debug session to find a simple quoting issue)