Commit Graph

52 Commits

Author SHA1 Message Date
admin c81df55dcb feat(shares): R-7b Parts 1-2 — shares payload builder + tier-2 shares job (Model B')
Sibling shares source for the local cross-drive tier. Reuses the tier2Mirror seam,
selectTier2TargetFrom (narrow source-drive seam extracted from selectTier2Target),
tier2ReconcileRoots (pure extraction), tier2SafeRemove, the marker-LAST discipline
and the recordTier2* helpers. Per-app paths are untouched.

- shares_payload.go: deterministic _shares-manifest.json + best-effort passdb capture
- tier2_shares.go: per-source-drive legs -> cross-drive target, payload, marker LAST
- infra.SambaContainerName/SambaPassdbVolume/Mount: single source of truth for the
  container identity (renderer, stacks execs, backup execs, monitor all read it)
- RESERVED-NAME finding: ValidateSMBShareName did NOT exclude a leading underscore,
  so "_shares" was an accepted share name. Now refused; RunAllTier2 additionally
  skips a "_shares" stack loudly as defense in depth.
- fix: shareSourceDrive returned a slash-normalised path, which made the target
  selector's source-drive equality check miss (a group could target its own drive)
2026-07-18 12:45:57 +02:00
admin 596505ed64 v0.142.0: offsite repo continuity — orphaned-repo guard (A) + run-status auto-refresh (C)
- Part A: classify restic cat-config failure (wrong-password=orphaned vs no-repo vs other); ORPHANED state + Hungarian card + offbox_repo_orphaned/reset events (once, not nightly); reset = move-aside (never delete) + init, unclaimed auto / claimed confirm. Red-proofs TestOffbox_OrphanDetection_* + ConfirmedReset.
- Part C: GET /backup/offbox/status + poll on backups_remote → flips Fut→Rendben/Hiba without manual reload.
2026-07-17 10:47:30 +02:00
admin 3603d1fc7f Tier-2 engine rework: class-driven legs, v2 layout, NAS-target exclusion (Task 3b, v0.135.0)
tier2_capture.go: classified apps get TierSecondary per-bind legs (paperless copy shrinks — export
drops); legacy apps keep the byte-identical resolver set. v2 relpath-mirroring layout
(backups/secondary/<stack>/{marker LAST, recovery-unit/, hdd/<rel>/, userdata/<rel>/}); N>1 native
(errTier2MultiDir/tier2AppDataName deleted). Migration=delete-and-rebuild + reconcile; all RemoveAll
via tier2SafeRemove (refuses outside backups/secondary/). SSD=state-only tier. selectTier2Target
never picks network storage (pinned+auto, F-6C-1). Restore reads v2 behind a marker gate.
Part 0: offbox_enlarge_blocked is a persisted one-time Load seed (opt-out sticks), not a getter
append. Part 0.5: offsite restore scratch prefers a local (non-network) path.
Full v2 test suite + all 10 §10 red-proofs verified. Destructive writes bounded to backups/secondary/.
2026-07-15 10:10:20 +02:00
admin 2d20859858 Offsite tier policy engine: mandatory userdata, raw-data quota, restore rework (Task 3a, v0.134.0)
Each toggled app's offsite push = one multi-path restic snapshot (recovery unit + TierOffsite
mandatory userdata via ComputeCaptureSet); legacy/undeployed stay unit-only. Loud capture gaps
(SP-3.4: restic 0.14.0 silently skips missing paths). Quota = stats --mode raw-data (SP-1;
displayed size drops once). Pre-push enlargement gate blocks the userdata enlargement over-quota
(unit-only push continues; EnlargedBlocked; edge-triggered notify). forget --group-by host,tags
on both sites (SP-2). Restore reworked: scratch off the rootfs + headroom gate (F-A1), unit-only
default via --include, size-first full, place-to-live missing-only merge (never --delete).
UI: unit/full-two-step/place actions + per-app blocked note; route POST /backup/offbox/place.
HUB FLAG: offbox_enlarge_blocked event needs hub allowlist for push delivery.
+13 tests; all 10 §10 red-proofs verified. No tier-2/.fab/hub/agent changes.
2026-07-14 22:51:54 +02:00
admin 68f0e0cf5c F-S2 + F-S3: compose-derived appdata dir resolution (v0.131.0)
The controller assumed an app's HDD appdata dir is always appdata/<stackName>.
paperless-ngx writes appdata/paperless (stack paperless-ngx), so every consumer
keying by stack name silently missed it via a stat-and-skip. One canonical
resolver appbackup.AppDataDirNames derives the real dir name(s) from the app's
compose ${HDD_PATH} binds; all consumers use it.

- F-S2 (tier-2): RunTier2 mirrors the resolved appdata/<name> (paperless docs
  got NO tier-2 copy before). Tier2Info size + RestoreTier2Files live dir use it.
  WARN when a declared appdata dir is absent. New tier2Mirror seam.
- F-S3 (migrate, NEW): all six per-app appdata legs (collision/size/copy/verify/
  cleanup/skip-set) now loop resolved names. scope="app" migration of paperless
  previously copied nothing and left an empty media dir (scope="all" was saved by
  the merge walk). WARN on missing declared dir in the copy leg.
- Multi-dir (N>1) refusal: tier-2 backup/info/restore refuse loudly (Hungarian);
  migrate supports N. No catalog app hits it today; lifted by Task 3.
- Display: storage page sums resolved dirs.
- Truth repair: the v0.130.0 "tier-2 copies the namespace wholesale" claim is
  false; corrected in CHANGELOG + main.go export-adapter comment.

+9 tests; red-proofs RP-1..RP-5 all confirmed. Controller-only, no agent/hub
coupling. Task 1 of the backup-classification-redesign arc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
2026-07-14 17:45:53 +02:00
admin 40b53047f1 controller: atomicPromoteTar syncs via an O_RDWR handle — read-only fsync is refused on Windows, which kept the dev-box green gate permanently red (Linux behavior unchanged)
Claude-Session: https://claude.ai/code/session_01GzammAMzsJTgpQHqxwM2bC
2026-07-13 08:19:39 +02:00
admin 68b3a3932e controller: F7 atomic volume dumps + F6 no-single-copy + F5 stale-primary sweep (WIP, pre-build)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017CDMFpFx84pfviCTVuGGhf
2026-07-12 09:00:16 +02:00
admin cb9e992c7a v0.110.0: offbox stale-lock self-heal (campaign C2) + crash-truthful status (C1)
resticStep escalates a restic lock error to `unlock --remove-all` + one
retry (safe: single-writer repo — sub-account isolation + single-flight
mutex); plain `unlock` is stale-only and can't clear a crash lock across a
container-hostname change. Pre-run stale unlock hygiene on run+restore.
C1: NewManager flips a persisted LastStatus=running to a truthful error.
Both red-proofed (A reproduces the exact campaign backup failure).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-10 07:02:23 +02:00
admin c529a455af feat(backup): async restore family — no proxy-timeout error page on a succeeding restore (v0.102.0)
Re-adjudicates F4: /backup/restore, /backup/tier2/restore, /backup/offbox/restore
blocked the HTTP request until completion, so through cloudflared's 100s cap a
customer got an error page while the restore succeeded (offbox worse — bounded
on r.Context(), canceling the SFTP restore mid-flight). Convert all three to the
offboxRun async shape: fast-path IsRunning refuse, background goroutine
(offbox ctx off r.Context() -> Background+30m), instant redirect. Add mutex-
guarded op-status (opstatus.go) + GET /api/backup/restore-status + a 3s-polling
backups.html banner (neutral running, red on failure). Restore single-flight
unchanged. Tests + red-proof (sync handler blocks indefinitely vs <500ms async).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-06 20:23:49 +02:00
admin 30b11100e0 feat(backup): C2 Part 1 — RestoreTier2Files: in-place, additive-only class-C file restore engine
Closes the engine half of drill finding F2: user files under appdata/<stack>
had no customer recovery path (operator copy-back only).

- RestoreTier2Files(stack): single-flight with backup/restore; ALL refusals
  before any stop (no Tier-2 record / LastRun empty / copy dir absent →
  "nincs másodlagos fájlmásolat"; Tier-2 drive disconnected; live drive
  disconnected/decommissioned — Hungarian, flash-ready); source is the
  RECORDED CrossDriveBackup.DestinationPath (never a fresh selectTier2Target);
  stop → copy → start → waitForHealthy; copy/restart errors surface (F17).
- rsyncRestoreMissing: rsyncMirror's exec shape with the OPPOSITE-direction
  flags: -a --ignore-existing --itemize-changes — existing live files are
  never overwritten, nothing is ever deleted (the --delete trap this task
  exists to avoid). Count = ">f" itemize lines (pure countRestoredFiles).
- restoreFilesCopier seam so orchestration tests never shell out; the one
  FS-level test of the real rsync is LookPath-guarded (runs on the Linux
  build server + live validation).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-05 13:17:40 +02:00
admin a52851e79e fix(backup): O4 — generate a replacement for unrecoverable resettable secrets on restore
The proceed-path for a missing RESETTABLE secret redeployed the app with the
secret blank (compose "Defaulting to a blank string" → exit 1, live-hit in the
2026-07-04 drill Phase 5). Now the restore generates a fresh credential instead:

- stacks.Manager.GenerateSecretForField: replacement value from the field's
  catalog generate spec via the deploy flow's generateValue (no logic copied);
  refuses data-keys (defense-in-depth), spec-less and non-secret fields.
- backup.Manager.SetSecretGenerator seam (wired in main.go), consulted in
  RestoreFromRecoveryUnit AFTER the untouched fail-closed gate, for missing
  names NOT in DataKeyEnvVars. The generated value rides fullEnv into
  RecreateStackFromUnit → RedeployFromEnv → SaveAppConfig, so it persists
  encrypted in the guest app.yaml and round-trips on the next backup/restore
  (no second write path). reconcileRestoreSecrets stays pure and untouched.
- WARNs now discriminate: "generated replacement for X (credential was reset)"
  vs "X unrecoverable and has no generator — app may fail to start". Values are
  never logged (asserted in test).
- Residual case (documented, not pretended away): if a restored volume tar
  carries the OLD internal credential hash, the app may still fail auth until a
  manual in-DB reset — generation fully fixes only the fresh-init case.

Companion red-proof: pre-fix behaviour (generation skipped) fails
TestRestoreGeneratesMissingResettableSecret on the non-empty DB_PASSWORD
assertion (verified, reverted). Data-key gate proven unreachable by generation
in TestRestoreGenerationNeverReachesDataKeys.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-05 11:52:04 +02:00
admin 73378a812c fix(backup): F3 — wire named-volume dumps into the app-data backup run
DumpAppVolumesSafe had NO production caller: no trigger ever produced
volume-dumps/, so named-volume app data (e.g. nextcloud's html volume) was never
captured into the recovery unit and the granular restore silently restored
nothing for class-B data (drill finding F3).

- runVolumeDumps: per-stack loop in runDBDumpsInternal, BEFORE
  captureAllRecoveryUnits (so manifests enumerate the fresh tars). Gate order is
  load-bearing: protected-stack and volume-check gates precede DumpAppVolumesSafe
  (which stops the stack before its own check — unconditional calls would bounce
  every volume-less app nightly). Disconnected/decommissioned drives skip with
  the same summary style as the DB loop.
- No silent partials: a per-stack failure lands as a FAIL summary entry, flips
  Success, and fails the run ("some backup steps failed: ..."), without aborting
  the other stacks.
- Zero-DB early return removed: volume-bearing apps without a database still get
  their class-B dump + unit refresh.
- dumpVolumesSafe seam (same style as the F17 discoverDBs/importDBDump seams) so
  the gating is unit-tested without Docker. Companion red-proof: neutering the
  volume gate fails TestRunVolumeDumps_GatesPrecedeDump (dump fired for the
  volume-less stack) and _VolumelessNeverStopped (verified, reverted).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-05 11:46:37 +02:00
admin 2a7deadc93 controller v0.93.0: NAS Part B off-box backup target (restic-over-SFTP)
Encrypted restic repo over SFTP for the app-data tier (the off-site 3-2-1 leg). A dead
NAS fails fast via -oConnectTimeout (spike Q8), never hangs the runner; secrets are 0600
files (ride DR via PBS whole-CT); init-if-absent, retention forget --prune, restore,
single-flight, per-app toggle + UI. restic re-added to the image.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
2026-06-30 15:26:38 +02:00
admin b57150e3ab B1: data-migration engine (MigrateAll + MigrateApp) + backup mutual-exclusion
internal/stacks/migrate.go: crash-safe, resumable namespace migration over the
controller's /mnt RW mount. Two entry points (whole-namespace + per-app) share one
journaled pipeline: validate -> stop -> copy (rsync -a --checksum, additive; conflict-
merge walk for non-app content) -> verify -> flip+redeploy (RedeployFromEnv) -> cleanup.
CLEANUP (the only destructive step) is gated on all units verified AND all apps
redeployed. Single-flight; mutual exclusion with the backup orchestrator (Change 3).
Non-hollow tests incl. mutation-proven collision + cleanup-gate companions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 19:42:57 +02:00
admin f8afe5c055 M18: cache DB-dump validation; skip re-validate on unchanged dumps
ListDumpFiles ran ValidateDump (line-by-line scan) for every dump on every ~5-min
RefreshCache cycle — wasted I/O+CPU on large customer dumps. ListDumpFiles now takes
an optional cached(name,size,mod) lookup; on a (size+modtime) match it reuses the
prior result and skips ValidateDump. settings.DBValidationCache gains Size+ModTime;
listAllDumpFiles builds the lookup from the persisted cache and writes back only fresh
validations (cache miss), so an unchanged dump triggers neither a re-validation nor a
settings.json write each cycle. nil cached = legacy validate-always (back-compat).
Tests: cache-hit skips validate (sentinel), cache-miss validates, nil validates.
2026-06-14 14:13:09 +02:00
admin 6bab68b132 M19: deriveStackName cross-references deployed stacks (fix DB-container misattribution)
deriveStackName pure-suffix-stripped on '-' (postgres/db/mariadb/.../cache), so a
stack whose slug ENDS in a role token (e.g. 'my-cache') was misattributed (stripped
to 'my') — filing its DB dump under the wrong/nonexistent stack. Now threads the set
of deployed stack names (m.knownStackNames() <- ListDeployedStacks) into
DiscoverDatabases and cross-references: candidate suffix-strip if known, else the
container name if it IS a known stack, else longest known stack that is a prefix
(handles <stack>_postgres / <stack>-1), else legacy strip. nil/empty known = legacy
behaviour (appexport passes nil). Table test incl. the my-cache case (fails pre-fix).
2026-06-14 14:08:56 +02:00
admin 0b9450e356 F17: per-app restore now replays the captured .sql DB dump (.sql wins)
The restore paths (RestoreFromRecoveryUnit + the RestoreApp fallback) repopulated
Docker volume tars but NEVER replayed the captured <stack>-<dbtype>.sql dump, so
DB-resident data (e.g. rows in a DB whose data dir is a bind mount) did not come
back — the romm marker round-trip in the audit lost the row.

New appbackup.ImportDump (read-side counterpart to DumpOne) replays a .sql/.sql.gz
into the running DB using the live container's OWN discovered credentials (no env
threading; reuses DiscoveredDB + getMariaDBPassword). backup.reimportDBDumps
orchestrates it AFTER volume restore + stack bring-up, so the logical dump WINS
over any volume-tar copy of the DB (operator-chosen precedence). pg_dump
--clean --if-exists and mariadb-dump (default --add-drop-table) make replay
idempotent; psql ON_ERROR_STOP=1 surfaces real import errors.

Also: volume-restore per-volume failures and DB-import failures now SURFACE (the
restore returns an error) instead of a swallowed WARN, so a failed data restore
cannot read as success.

Tests (restore_db_test.go, injectable discover/import seams): imports when dump+DB
present, failure surfaces, no-dump skips discovery, dump-but-no-matching-DB is a
non-fatal skip. Live DB round-trip to be validated post-deploy.
2026-06-14 10:02:58 +02:00
admin 092cbbe804 fix(M2): make backup.Manager.stackProvider init-only contract explicit
The single write (SetStackProvider, main.go:225) was mutex-guarded while all 11
reads were unlocked — the lock implied a runtime concurrency the reads don't
honour. It is called once during single-threaded startup before any goroutine,
so the write happens-before every read and no race exists. Removed the
misleading lock and documented the init-only contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 19:15:38 +02:00
admin d2071430ea v0.55.0: Phase 3 — auto off-drive Tier 2 (rootfs-headroom guard)
Tier 2 rsync-mirrors each HDD app's recovery unit + appdata to a DIFFERENT physical
disk (the only off-drive protection bind-mounted userdata can get; PBS can't reach it).
Auto-enabled, auto-target: prefer another registered drive (different physical disk via
system.SamePhysicalDevice), else the internal SSD for SMALL units only — with a
size-aware headroom guard that REFUSES rather than fill the ~8G guest rootfs, recording
an honest "needs 2nd HDD" status. Status persisted via the surviving CrossDriveBackup;
"2. mentés" UI card now populated. Daily tier2-backup job + POST /api/backup/tier2.

- backup/tier2.go (engine+selection+headroom), tier2_test.go (headroom arithmetic)
- system.SamePhysicalDevice (linux Stat_t.Dev + stub)
- handlers.go Tier2 UI population + tier2DestLabel; backups.html honest no-target reason
- fixed stale TestBackupCopiesOnPath (old felhom-data layout -> in-guest layout)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 13:24:49 +02:00
admin e02292aa1a test: Phase 2b restore orchestration coverage + nil-safe isDebug
Adds an in-process orchestration test for RestoreFromRecoveryUnit: success path
calls recreate with non-secret env + recovered secrets merged; data-key-missing
path is REFUSED and recreate is never called. Makes Manager.isDebug nil-safe
(behavior-neutral in prod; cfg is always set) so the gate/orchestration are testable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 11:17:08 +02:00
admin eefeeabea3 v0.53.1: refresh recovery units on periodic cache cycle (idempotent)
CaptureRecoveryUnit now builds content in memory and skips writes when the unit
is already current (checksum + dump-set + version), so it can run from RefreshCache
(startup + every 5m) without thrashing the USB drive. Units now exist shortly after
startup and track config changes without waiting for the daily DB dump. +idempotency test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 10:27:35 +02:00
admin 70eb521cd0 v0.53.0: Phase 2 capture side — per-app secret-free recovery unit
Each app's on-drive backup becomes a self-contained, recreatable recovery unit:
compose/ (docker-compose.yml + .felhom.yml + secret-stripped app.yaml) alongside
the existing db-dumps/ + volume-dumps/, plus a secret-free manifest.json (image
pins, secret env-var NAMES, data_key names, checksums). The unit stores no secret
value, no data-key, and not the image — secrets are recovered at restore from the
guest's own app.yaml (live/PBS), never regenerated.

- appbackup: RecoveryUnit* path helpers, RecoveryInfo + GetStackRecoveryInfo,
  ParseComposeImages; AppDBDump/Volume refactored onto RecoveryUnitPath.
- backup: recovery_unit.go (manifest + CaptureRecoveryUnit), wired into RunDBDumps;
  capture test proves secret-free.
- stacks: DeployField.DataKey + Metadata.DataKeyEnvVars(); main.go stackAdapter
  implements GetStackRecoveryInfo (excludes secret-named + encrypted values).
- Restore-from-unit recreate + fail-closed gate + live AdventureLog validation: next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 10:20:37 +02:00
admin 63484a0bd4 v0.51.0: offsite-backup UI (felhom-pbs DR) + Model-A double-nest fix
- Backups page: whole-guest backup shown as real DR — target label "Biztonsági szerver –
  külön hardver (PBS)"; app-data "Távoli mentés" card now reflects the PBS offsite tier
  (guestBackupView.Offsite) instead of "nincs beállítva".
- Model-A double-nest fix: appbackup path helpers take a felhom-data NAMESPACE ROOT (no
  internal felhom-data join); backup.Manager.namespaceRoot/AppNamespaceRoot resolve
  HDD-vs-systemDataPath provenance so a drive-resident app's backups land single-nested
  (<drive>/backups/... on the guest = <drive>/felhom-data/backups/... on the host) instead
  of .../felhom-data/felhom-data/.... Writes, deletion (GetStackBackupData/RemoveStack/
  ProtectedHDDPaths), wipe-warning scan, and export updated coherently; legacy double-nest
  dirs kept protected. New appbackup test asserts no doubled segment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 20:26:52 +02:00
admin abe4e8e619 slice 8C Phase B.2 + C.1/C.2: retire disk subsystem + rewire disk mgmt to agent
Retired (~12.3k LOC): internal/storage/* (scan/format/attach/migrate/safety),
backup restic/crossdrive/restore_drives/disk_layout/local_infra/restore_scan/
paths + restore_app, report/infra_backup*/infra_pull, setup/scanner,
monitor/watchdog+pinger, web/storage_handlers+handler_restore. Surgically split
backup.Manager to app-data only (DB dumps + volume tars + app restore; dropped
restic + cross-drive + snapshot history). Fixed router/main/web wiring.
Added agent-backed disk API (web/agent_disk_handlers.go): /api/disks list/
assign/eject/format proxying agentapi; data-bearing format refusal -> HTTP 409
'operator authorization required'. report/config_pull.go keeps the setup
fresh-install config download. go build + go test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 13:57:27 +02:00
admin fb11c3b75a feat: backup safety — stop-before-dump, streaming restore, health check, per-app restic, infra configs (v0.34.0)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 08:56:48 +01:00
admin c929948f27 feat: Docker volume backup, Tier 2 restore, restore dropdown fixes (v0.33.0)
- Add Docker named volume backup to Tier 1 (dump to tar, include in restic)
  and Tier 2 (copy tars to rsync mirror _volumes/ dir)
- Fix volume name resolution: use project-prefixed names (mealie_mealie_data)
- Fix double Tier 1 in restore dropdown: filter snapshots by app's home drive
- Add Tier 2 restore: RestoreAppFromTier2() restores from rsync mirror
- Show Tier 2 entry in restore dropdown when cross-drive backup succeeded
- Add .fab import link in restore section
- Volume-aware restore type banners and backup content labels

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 21:43:02 +01:00
admin af1dd14933 fix: standardize log prefixes, remove duplicates, add missing module tags
Second-pass logging cleanup: consistent [LEVEL] [module] format across
all 41 files. Remove stale prefixes ([CF], [SYNC], [SCHED], [API],
[STORAGE], [HEALTH], [ROLLBACK]). Remove 5 duplicate log lines. Gate
ungated DEBUG lines. Fix wrong log levels (restore start WARN→INFO).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 21:20:09 +01:00
admin 8e61cd7ec4 feat: comprehensive INFO/WARN/ERROR logging across all controller modules
Add structured operational logging at INFO, WARN, and ERROR levels to
every controller module. Standardize custom prefixes ([GEO], [SCHED],
[SYNC]) to use [INFO/WARN/ERROR] [module] format. Fix misleveled logs
(WARN->ERROR for data loss scenarios, WARN->INFO for routine operations).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 19:58:27 +01:00
admin 95c821deb2 feat: comprehensive debug logging across all controller modules
Add detailed [DEBUG] logging to every controller module when
logging.level is set to "debug". Each module with stateful debug
uses SetDebug(bool) wired from main.go. Covers stacks, backup,
cloudflare, integrations, system, monitor, settings, scheduler,
web handlers, storage, metrics, API, selfupdate, and assets.

Also includes the app export/import (.fab bundles) feature from
v0.32.0 and its debug page integration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 18:14:43 +01:00
admin db83db383c fix: deep bug hunt II — concurrency, security & optimization (25 files)
Critical: watchdog mutex panic safety, SetGeoAppOverride nil guard,
SSD-only app DB restore fallback.

High: double deploy race (atomic Deploying flag), delete/remove during
deploy guard, ScanStacks overwrite protection, FileBrowser mount mutex,
PushEvent history, PushOnce error handling, DB dump sync+close before
rename, restic retry fresh context, encrypt failure logging, cross-backup
path traversal validation, deepCopyStack completeness.

Security: constant-time API key comparison, login rate limiting (5/min),
git credential masking in logs, storage path prefix traversal fix.

Concurrency: MigrateEncryption lock ordering, SubdomainInUse I/O outside
lock, scheduler late-registered jobs, SQLite WAL verification, metrics
shutdown context, telemetry scan error logging, asset sync lock scope.

Optimization: streaming file copy for DB dumps, restic stats dedup,
atomic infra config copy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 14:21:09 +01:00
admin 8b8c04a487 fix: P0+P1 critical bug fixes across controller (24 files)
Concurrency fixes:
- Deep-copy stacks in GetStack/GetStacks to prevent shared state mutation (C04)
- Add per-state mutex to watchdog pathProbeState (C05)
- Guard MetricsCollector.Start() with sync.Once against double-start (C06)
- Hold diskJobMu across entire raw mount operation (C07)
- Add mutex to SetEncryptionKey (C08), MigrateEncryption write lock (H03)
- Use sync.Once for sync.Stop() channel close (H08)
- Set syncing=true before releasing lock in TriggerSync (H09)
- Deep-copy lastDBDump/lastBackup in GetFullStatus (H11)
- Add WaitGroup for stderr goroutine in MigrateDrive (H19)
- Add mutex to SetBackupRunningCheck (M18)

Security fixes:
- Validate Bearer token against Hub API key in CSRF middleware (H16)
- Validate backup paths start with expected prefix in RemoveStack (M12)
- Guard uuid[:8] slice with length check (H20)
- Parse fstab fields exactly for mount target matching (H21)

Bug fixes:
- Use decrypted env vars for compose deploy (C01)
- Log decrypt failures in DecryptMap instead of swallowing (C02)
- Move Deployed=false inside lock in runComposeDeploy (C03)
- Fix activeDrives() to skip disconnected drives (H02)
- Fix Snapshot() stderr extraction from exec.ExitError (H01)
- Check unlockCmd.Run() error in restic (H01)
- Buffer template rendering via bytes.Buffer (H07)
- Thread context.Context through cloudflare client (H10)
- Fix leaf-name collision detection in cross-drive backup (H15)
- Add nil check for crossDriveRunner (H17)
- Use strings.TrimSpace instead of slice on command output (H18)
- Make SaveAppConfig atomic with write-to-tmp+rename (H04)
- Pass encKey on deploy failure SaveAppConfig (H05)
- Fix IPv6 address format in TCP health probe

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 13:39:45 +01:00
admin be7803c0ac v0.24.0 — Pre-testing observability: debug logging, diagnostic dump, startup self-test
- Add [DEBUG] logging across all modules (backup, storage, sync, selfupdate,
  monitor, notify, report, assets, setup) gated behind logging.level: "debug"
- Add /api/debug/dump endpoint returning full controller state JSON (debug only)
- Add startup self-test validating 9 subsystems (Docker, dirs, storage, hub,
  restic repos, metrics DB) with pass/warn/fail summary
- New packages: internal/selftest, internal/util
- Constructor/signature changes: debug bool params, logger params on
  RunHealthCheck and BuildReport, smart watchdog probe logging

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 18:32:26 +01:00
admin 99bf3ca7a8 feat: drive migration & Tier 2 restic deprecation (v0.18.0)
Phase 1: Deprecate restic as Tier 2 method (rsync only), auto-migrate on startup
Phase 2: Enhanced per-app migration with backup awareness, DB dump copy, auto-cleanup
Phase 3: Full drive migration with decommissioned state, rollback support, wizard UI
Phase 4: Hub report includes decommissioned drive state

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 21:49:14 +01:00
admin bdbe170a54 feat: storage watchdog — USB disconnect detection, auto-stop, safe eject, auto-reconnect (v0.17.0)
New storage watchdog monitors registered storage paths every 5s. On disconnect
(3 consecutive probe failures), auto-stops affected apps, lazy-unmounts stale
VFS entries, fires alerts/notifications/hub report. On reconnect (UUID detected),
auto-remounts via fstab, cleans stale restic locks, offers app restart.

Safe disconnect UI for USB drives: confirmation dialog, stop apps, sync, unmount.
Disconnected state visible across all pages (dashboard, settings, backups, monitoring)
with hatched red bars and badges. Backup guards skip disconnected drives.

22 files changed (1 new: monitor/watchdog.go), ~1500 lines added.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 19:42:26 +01:00
admin d372454c18 v0.15.2: Fix snapshot stats and DB validation loss on restart 2026-02-19 08:45:37 +01:00
admin 2befa6877b v0.15.1: Backup page Részletek overhaul with per-drive tier sections
Replace Tároló section with collapsible Részletek containing 3 tiers:
- Tier 1: per-drive restic repo stats with storage labels
- Tier 2: cross-drive items grouped by destination, split by method
- Tier 3: remote backup placeholder
Restore UI now shows tier + drive labels in snapshot dropdown.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 08:23:33 +01:00
admin 70d503a902 fix(backup): 4 bug fixes from v0.14.1 code review (v0.14.2)
Bug 1 (HIGH): add --exclude _* to rsync --delete so _db/ and _config/
  directories are never deleted between backup runs (crossdrive.go)

Bug 2 (MEDIUM): refactor RunDBDumps/RunBackup/RunFullBackup to use
  acquireRunning/releaseRunning helpers; extract runDBDumpsInternal and
  runBackupInternal so all three public entry points set m.running and
  RunFullBackup no longer deadlocks calling the public methods (backup.go)

Bug 3 (MEDIUM): log [WARN] when GetDiskUsage returns nil in
  ValidateDestination instead of silently skipping space checks (crossdrive.go)

Bug 4 (MEDIUM): add [WARN] on empty SystemDataPath in NewManager; add
  [ERROR] in GetAppDrivePath; guard DumpStackDB against empty/relative paths
  (backup.go)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 19:46:16 +01:00
admin 563c9515d9 v0.14.0: Per-drive backup architecture + storage path overhaul
Major refactor of backup and storage paths:

- Per-drive restic repos at <drive>/backups/primary/restic/
- Per-app DB dumps at <drive>/backups/primary/<app>/db-dumps/
- Remove global BackupDir, DBDumpDir, ResticRepo config fields
- Add SystemDataPath config (fallback for apps without HDD)
- New backup/paths.go with pure path computation helpers
- Add GetStackHDDPath to StackDataProvider interface
- Restic methods now accept repoPath as parameter
- Cross-drive backup uses new secondary path structure
- Rename storage/ to appdata/ in scripts and compose templates
- Update protected HDD paths (storage → appdata + backups)
- Simplify backup UI (remove global path displays)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 18:47:39 +01:00
admin 6c1762141a v0.12.7: mandatory HDD backup, pre-dump, restore for all apps
Fix 1: HDD data backup is now mandatory for all deployed apps.
resolveAppBackupPaths() iterates ListDeployedStacks() directly — no
longer reads GetAppBackupMap() or checks the Enabled flag. DiscoverAppData()
drops backupPrefs parameter; BackupEnabled is set from HasHDDData.
Five dead settings methods removed: IsAppBackupEnabled, SetAppBackup,
GetAppBackupMap, SetAppBackupBulk, GetAppBackupPrefs.

Fix 2: Cross-drive backup now triggers a fresh DB dump (DumpStackDB)
before running. DBDumper interface added to crossdrive.go; Manager
implements it; SetDBDumper wired in main.go. Non-fatal — proceeds with
user data backup even if DB dump fails.

Fix 3: Restore dropdown shows ALL deployed apps (not just HDD+enabled).
restore.go rewritten: always restores config+DB, adds user data if hasHDD.
UI shows restore type banner (full / config+DB / config only) with
color-coded styling. Snapshot API clarified for non-HDD apps.

Fix 4: "Docker kötetek" → "Konfiguráció" — named volumes are not in
the restic backup paths; compose files + app.yaml are what's backed up.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 10:38:51 +01:00
admin 93d9b474f1 v0.12.3 — Security & correctness bug fixes (33 bugs)
CRITICAL: 10 data race and security fixes — backup.go mutex coverage
(C1-C4), IsSystemDisk 12-bit major/minor (C5), /dev/ path validation
(C6), extractName traversal (C7), TargetPath/DestinationPath against
registered paths (C8-C9), ParseComposeHDDMounts Clean-before-prefix (C10).

HIGH: 17 logic/resource fixes — ValidateDump bufio.Scanner (H1), single
appDirSize() with 30s timeout (H2/H3), snapshot ID regex (H4), cross-drive
restic prune (H5), temp file order (H6), dirSizeBytes errors (H7), atomic
fstab (H8), IsDeviceMounted suffix check (H9), eMMC partition mapping (H10),
bytesCopied mutex (H11), separator-aware migrate prefix (H13), DeleteStack
error on compose-down (H14), docker 60s timeout (H16), NotificationPrefs
deep-copy (H17), wipefs warning (H18), fstab rollback on mount fail (H19).

MEDIUM: 7 code quality fixes — formatBytes dedup (M1), .tmp filter order
(M2), sizeBytes string type (M3), elapsed in message (M6), LoadLocation
fallback (M7), pathCovers separator (M10), cancelEditLabel textContent (M11).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 21:10:55 +01:00
admin 62992e0e04 v0.12.2: restore section simplification — snapshot filtering, auto-stop/restart, UI cleanup
- StackDataProvider interface extended with StopStack/StartStack
- backup.Manager.GetStackHDDMounts() delegates to stackProvider
- RestoreApp() auto-stops app before restic restore, restarts after (even on failure)
- stackAdapter in main.go wires StopStack/StartStack through to stacks.Manager
- GET /api/backup/snapshots?stack={name} filters snapshots by app HDD paths via filterSnapshotsByPaths()
- Restore section simplified: no path list, per-app filtered snapshots, human-friendly timestamp format, single calm warning, empty-result inline message

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 19:19:23 +01:00
admin 1de244646b v0.12.0 — Backup page overhaul: unified app rows, bug fixes, sequential chaining
Bug fixes:
- GetFullStatus() returns deep copy; CrossDriveSummary/UnconfiguredApps/CrossDriveWarnings
  are always nil in the copy so the handler builds them fresh (fixes duplicate-apps bug)
- Replace binary IsMountPoint check with tiered CheckBackupDestination() — path-not-exist,
  not-writable, system-drive (warning), disk >90-95% full; shown as warning vs critical
- Remove dead settingsAppBackupHandler / POST /settings/app-backup route (toggle wrote
  to settings.json but nothing consumed the flag)

Architecture:
- Unified per-app backup rows: new AppBackupRow struct + buildAppBackupRows() replaces
  the two old sections with expandable rows showing all 3 layers per app
- Sequential backup chaining: cross-drive runs immediately after restic (removed
  independent cross-drive-daily/cross-drive-weekly scheduler jobs)
- Deploy page: remove "Csak kézi indítás" schedule option; add weekly consistency note

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 17:56:28 +01:00
admin 1a8036d055 v0.11.8 — Per-App Cross-Drive Backup (3-2-1 rule)
New feature: backup app data to a secondary storage drive to satisfy
the "different media" requirement of the 3-2-1 backup rule.

- settings.go: CrossDriveBackup struct, AppBackupPrefs.CrossDrive field,
  getter/setter methods, GetOrCreateCrossDrivePassword, preserves
  cross-drive config when toggling nightly backup

- crossdrive.go (new): CrossDriveRunner with rsync and restic backends.
  Validates destination (mount point, writable), prevents source/dest
  overlap, per-app concurrency lock, persists last_run/status/size.

- main.go: wire CrossDriveRunner, register cross-drive-daily (03:30)
  and cross-drive-weekly (04:30 Sundays) scheduler jobs

- router.go: 4 new API endpoints — save config, trigger run, get status,
  run-all. Router now accepts Settings and CrossDriveRunner.

- server.go: Server struct accepts CrossDriveRunner, new web route
  POST /settings/cross-backup/{name}

- handlers.go: deployHandler populates CrossDriveConfig, BackupDestPaths,
  BackupDestWarning, AppBackupEnabled. settingsCrossBackupHandler saves
  config. backupsHandler builds CrossDriveSummary, UnconfiguredApps,
  CrossDriveWarnings for backup page.

- deploy.html: "Biztonsági mentés" card with destination/method/schedule
  dropdowns, last-run status, manual trigger button, flash messages.

- backups.html: "Másolatok másik meghajtóra" section with per-app
  status rows, unconfigured app warnings, "Összes futtatása most" button.

- style.css: margin-bottom fix for .deploy-stale-data, new cross-drive
  card and list styles.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 15:45:31 +01:00
admin aca3b8680a v0.9.0: Storage paths registry, per-app HDD_PATH resolution, storage management UI
- Fix backup toggles not appearing (read each app's own HDD_PATH from app.yaml)
- Storage paths registry in settings.json with auto-discovery from deployed apps
- Settings page "Adattárolók" section with disk usage, add/remove/default/schedulable
- Deploy page path field as dropdown of registered storage paths
- Health check storage monitoring (mount point, disk usage alerts)
- Mount-point validation utilities (Linux syscall + cross-platform stubs)
- Controller docker-compose mount changed to /mnt:/mnt:rw for multi-storage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 09:04:28 +01:00
admin 7d801d1094 Phase 3 complete: per-app backup toggles, restore, storage overview
- Storage overview on backup page (SSD/HDD bars, repo stats)
- Restic password visibility + hub sync for disaster recovery
- App data discovery (HDD bind mounts, Docker volumes)
- Per-app backup toggle checkboxes with settings persistence
- Dynamic backup paths: enabled app HDD data included in restic snapshots
- Limited app restore from snapshots (self-service recovery)
- Snapshots API endpoint for restore dropdown
- Version bump to 0.8.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 21:29:56 +01:00
admin 4053245be8 v0.7.0: Phase 1 — Authentication, Persistence & Settings Page
- New settings.json persistence layer (internal/settings/settings.go)
  - Atomic write (tmp + rename), thread-safe with sync.RWMutex
  - Stores password hash overrides and DB validation cache
  - Auto-creates on first save, graceful handling if missing

- Auth improvements
  - Password resolution priority: settings.json > controller.yaml > none
  - Session duration extended to 7 days (was 24h)
  - ?next= redirect after session expiry (returns to original page)
  - Flash messages on login page (used after password change)
  - Conditional logout link (hidden when auth disabled)
  - Session invalidation on password change

- New Settings page (/settings)
  - Read-only system config display (customer, domain, git, backup, monitoring)
  - Password change form with validation (min 8 chars, match check)
  - Sidebar "Beállítások" item pinned to bottom above version

- DB validation persistence
  - Validation results saved to settings.json after each dump
  - Cached data survives container restarts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:26:59 +01:00
admin 9e8b48a32b fix: backup validation display and debug logging
- Add 4-branch template guard to handle zero-value DumpValidation
  (shows "–" instead of misleading "Hiba" when validation never ran)
- Add debug logging to ValidateDump for all code paths
- Add cross-check re-validation in RefreshCache to heal stale
  lastDBDump validation state from disk

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 14:56:34 +01:00
admin 97074e7a0c v0.6.0: healthcheck + hub reporting implementation
- Add heartbeat ping (every 5 min, controller alive signal)
- Add backup integrity check (weekly restic check, Sunday 04:00)
- Add Heartbeat + BackupIntegrity fields to PingUUIDsConfig
- Add HubConfig for central hub reporting
- Add report package (types, builder, pusher) for hub push
- Wire hub reporting into scheduler (configurable interval)
- Update controller.yaml.example with new monitoring + hub sections
- Add monitoring/DEPRECATED.md for legacy bash scripts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 13:19:08 +01:00
admin 3e8baebfa5 v0.5.0: Backup bugfixes + monitoring page with metrics store
- Fix "Helyi mentés" showing "–" after controller restart by synthesizing
  LastBackup from snapshot history and LastDBDump from dump files on disk
- New monitoring page (/monitoring) with system info, metrics charts, and
  container resource overview
- SQLite metrics store (modernc.org/sqlite, pure Go, no CGO) with 60s
  collection interval and 30-day auto-prune
- REST API endpoints: /api/metrics/system, /api/metrics/containers/summary,
  /api/metrics/containers/{name}, /api/metrics/sysinfo
- Chart.js 4.4.7 embedded locally for offline environments
- System info provider reads hostname, OS, kernel, CPU, uptime from /proc
- Docker compose updated with /etc/os-release host mount

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 10:14:46 +01:00
admin 3be989f665 v0.4.7: Protected stack detail pages + backup page caching
Task 1: Protected stacks with .felhom.yml (slug) are now clickable
on both dashboard and stacks pages. "Részletek" button added to
protected stack actions section. Filebrowser .felhom.yml updated
with resources metadata.

Task 2: Backup page now reads from a cached FullBackupStatus that
refreshes every 5 minutes in background + after each backup run.
Page loads instantly instead of blocking on restic/docker subprocesses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 09:02:54 +01:00