# TASK.md — Complete Cross-Drive Backup + Per-Tier UI (v0.12.8) ## Prompt (copy-paste this into Claude Code) ``` Read TASK.md for the full plan. Apply all code changes described, then build and deploy. After all fixes are done: 1. Run `go build ./...` and `go vet ./...` from the controller/ directory — fix any errors 2. Update CHANGELOG.md with a new entry at the top (session 45, v0.12.8) 3. Commit, build, and deploy following the workflow in CLAUDE.md ``` --- ## Context and Goals The cross-drive backup (Tier 2) currently only copies HDD user data (mounts). It does NOT include DB dumps or app config. If the primary drive fails, the customer loses their database and config — even though photos are safe on the second drive. **This is not a viable backup.** Each tier must be a **complete, self-sufficient backup**: | Tier | Contents | Location | Can fully restore? | |------|----------|----------|--------------------| | 1. Nightly restic | DB + Config + User data | Same drive | Yes (not against drive failure) | | 2. Cross-drive | DB + Config + User data | Different drive | **Yes (after this fix)** | | 3. Remote (future) | Everything | Offsite | Yes | This task also restructures the UI from per-layer (DB/Config/UserData rows) to per-tier (1st backup / 2nd backup rows) — matching the customer mental model. --- ## Fix 1: Include DB dumps + config in cross-drive backup ### 1a: Add `dbDumpDir` field to CrossDriveRunner **File:** `internal/backup/crossdrive.go` Add a `dbDumpDir` field to the struct: ```go type CrossDriveRunner struct { sett *settings.Settings stackProvider StackDataProvider dbDumper DBDumper dbDumpDir string // path to DB dump directory (e.g., /srv/backups/db-dumps) logger *log.Logger mu sync.Mutex running map[string]bool } ``` Add setter after `SetDBDumper`: ```go // SetDBDumpDir sets the path to the DB dump directory for cross-drive backups. func (r *CrossDriveRunner) SetDBDumpDir(dir string) { r.dbDumpDir = dir } ``` ### 1b: Add helper to copy DB dump files for a stack **File:** `internal/backup/crossdrive.go`, add helper function: ```go // copyStackDBDumps copies DB dump files for the given stack to destDir. // DB dump files are named _.sql (e.g., immich_postgres.sql). // Small files — uses plain file copy, not rsync. func (r *CrossDriveRunner) copyStackDBDumps(stackName, destDir string) error { if r.dbDumpDir == "" { return nil } entries, err := os.ReadDir(r.dbDumpDir) if err != nil { if os.IsNotExist(err) { return nil } return fmt.Errorf("reading DB dump dir: %w", err) } prefix := stackName + "_" copied := 0 for _, e := range entries { if e.IsDir() || !strings.HasPrefix(e.Name(), prefix) { continue } src := filepath.Join(r.dbDumpDir, e.Name()) dst := filepath.Join(destDir, e.Name()) data, err := os.ReadFile(src) if err != nil { return fmt.Errorf("reading %s: %w", e.Name(), err) } if err := os.WriteFile(dst, data, 0644); err != nil { return fmt.Errorf("writing %s: %w", e.Name(), err) } copied++ } if copied > 0 { r.logger.Printf("[DEBUG] Copied %d DB dump file(s) to %s", copied, destDir) } return nil } ``` ### 1c: Update `runRsyncBackup` to include DB + config **File:** `internal/backup/crossdrive.go`, function `runRsyncBackup` After the existing HDD mount rsync loop (after line 260 `return nil` of the for loop), add these blocks BEFORE the final `return nil`: ```go // --- Copy DB dumps for this stack --- dbDestDir := filepath.Join(destDir, "_db") if err := os.MkdirAll(dbDestDir, 0755); err != nil { return fmt.Errorf("creating DB dump dest dir: %w", err) } if err := r.copyStackDBDumps(stackName, dbDestDir); err != nil { r.logger.Printf("[WARN] Cross-drive DB dump copy failed for %s: %v", stackName, err) // Non-fatal: user data is the primary concern } // --- Rsync app config (compose dir) --- if composePath, ok := r.stackProvider.GetStackComposePath(stackName); ok { configSrcDir := filepath.Dir(composePath) configDestDir := filepath.Join(destDir, "_config") if err := os.MkdirAll(configDestDir, 0755); err != nil { return fmt.Errorf("creating config dest dir: %w", err) } src := strings.TrimRight(configSrcDir, "/") + "/" dst := strings.TrimRight(configDestDir, "/") + "/" cmd := exec.CommandContext(ctx, "rsync", "-a", "--delete", src, dst) r.logger.Printf("[DEBUG] rsync config: %s → %s", src, dst) if out, err := cmd.CombinedOutput(); err != nil { r.logger.Printf("[WARN] Cross-drive config rsync failed for %s: %v (%s)", stackName, err, strings.TrimSpace(string(out))) // Non-fatal } } ``` **Resulting rsync destination layout:** ``` backups/rsync// _db/ ← stackName_postgres.sql, stackName_mariadb.sql _config/ ← compose.yml, app.yaml, .felhom.yml ← existing HDD mount contents (unchanged) ``` The `_` prefix prevents collision with user data directories (HDD mounts never start with `_`). ### 1d: Update `runResticBackup` to include DB + config **File:** `internal/backup/crossdrive.go`, function `runResticBackup` Currently the restic backup only includes `mounts`. Change to also include the config dir and DB dump dir. **Current code (around line 294–301):** ```go args := []string{ "backup", "--repo", repoPath, "--password-file", pwPath, "--tag", stackName, "--tag", "cross-drive", } args = append(args, mounts...) ``` **Replace with:** ```go args := []string{ "backup", "--repo", repoPath, "--password-file", pwPath, "--tag", stackName, "--tag", "cross-drive", } // Include user data (HDD mounts) args = append(args, mounts...) // Include app config dir (compose + app.yaml + .felhom.yml) if composePath, ok := r.stackProvider.GetStackComposePath(stackName); ok { args = append(args, filepath.Dir(composePath)) } // Include DB dump dir (all stacks' dumps — restic deduplicates) if r.dbDumpDir != "" { if _, err := os.Stat(r.dbDumpDir); err == nil { args = append(args, r.dbDumpDir) } } ``` Note: for restic, including the full DB dump dir is fine — restic deduplicates and it's shared across apps. The `--tag ` identifies which app this backup belongs to. ### 1e: Wire up in main.go **File:** `cmd/controller/main.go`, after the existing `crossDriveRunner.SetDBDumper(backupMgr)` line: ```go crossDriveRunner.SetDBDumpDir(cfg.Paths.DBDumpDir) ``` --- ## Fix 2: Restructure UI from per-layer to per-tier The current UI shows 3 rows per app (DB / Konfiguráció / Felhasználói adatok). This maps to implementation details, not the customer mental model. Change to 2 rows per app: - **1. mentés** — nightly restic (mandatory, same drive) - **2. mentés** — cross-drive (opt-in, different drive) ### 2a: Restructure `AppBackupRow` struct **File:** `internal/web/handlers.go` **Replace the current `AppBackupRow` struct (lines 500–533) with:** ```go // AppBackupRow holds per-tier backup information for one app on the backup page. type AppBackupRow struct { StackName string DisplayName string Status string // "green", "yellow", "red", "auto" StatusText string // short Hungarian tooltip // App characteristics HasHDDData bool HasDB bool StorageLabel string HDDSizeHuman string // What this app's backup contains (for display) // e.g., "DB + Konfiguráció + Adatok", "DB + Konfiguráció", "Konfiguráció" BackupContents string // Tier 1: Nightly backup (always exists) Tier1LastRun string // formatted time of last restic snapshot Tier1LastStatus string // "ok", "error", "" Tier1DBStatus string // "ok", "error", "" — separate DB dump status for warning // Tier 2: Cross-drive backup (only for apps with HDD data) Tier2Configured bool Tier2Method string // "rsync", "restic" Tier2MethodLabel string // "rsync", "restic" Tier2Dest string // destination label Tier2Schedule string // "Naponta", "Hetente" Tier2LastRun string Tier2LastStatus string // "ok", "error", "running", "" Tier2LastError string Tier2StatusBadge string // "Sikeres", "Hiba", "Fut...", "—" Tier2SizeHuman string Tier2Browsable bool // true for rsync (plain files), false for restic // Warnings accumulated for this app Warnings []string } ``` ### 2b: Update `buildAppBackupRows` **File:** `internal/web/handlers.go` **Replace the entire `buildAppBackupRows` function** with: ```go // buildAppBackupRows constructs one AppBackupRow per deployed app for the backup page. func (s *Server) buildAppBackupRows( status *backup.FullBackupStatus, crossConfigs map[string]*settings.CrossDriveBackup, destLabels map[string]string, ) []AppBackupRow { loc := getTimezone() // Build DB stack lookup dbStacks := make(map[string]bool) for _, db := range status.DiscoveredDBs { dbStacks[db.StackName] = true } for _, f := range status.DumpFiles { dbStacks[f.StackName] = true } // Tier 1 timestamps (shared across all apps — single nightly job) tier1LastRun := "" tier1LastStatus := "" if status.LastBackup != nil { tier1LastRun = status.LastBackup.LastRun.In(loc).Format("01-02 15:04") if status.LastBackup.Success { tier1LastStatus = "ok" } else { tier1LastStatus = "error" } } tier1DBStatus := "" if status.LastDBDump != nil { if status.LastDBDump.Success { tier1DBStatus = "ok" } else { tier1DBStatus = "error" } } var rows []AppBackupRow for _, app := range status.AppDataInfo { hasDB := dbStacks[app.StackName] || app.HasDBDump // Build backup contents label var parts []string if hasDB { parts = append(parts, "DB") } parts = append(parts, "Konfig") if app.HasHDDData { parts = append(parts, "Adatok") } contents := strings.Join(parts, " + ") row := AppBackupRow{ StackName: app.StackName, DisplayName: app.DisplayName, HasHDDData: app.HasHDDData, HasDB: hasDB, StorageLabel: app.StorageLabel, HDDSizeHuman: app.HDDSizeHuman, BackupContents: contents, Tier1LastRun: tier1LastRun, Tier1LastStatus: tier1LastStatus, Tier1DBStatus: tier1DBStatus, } // Default status = auto (no user data, just config) row.Status = "auto" row.StatusText = "Automatikus mentés" if app.HasHDDData { cfg, hasCfg := crossConfigs[app.StackName] if !hasCfg || cfg == nil || !cfg.Enabled { // HDD data backed up via nightly restic (mandatory), but no second copy row.Tier2Configured = false row.Status = "yellow" row.StatusText = "Nincs második másolat (csak helyi mentés)" } else { row.Tier2Configured = true row.Tier2Method = cfg.Method row.Tier2MethodLabel = cfg.Method // "rsync" or "restic" row.Tier2Browsable = cfg.Method == "rsync" row.Tier2Dest = destLabels[cfg.DestinationPath] if row.Tier2Dest == "" { row.Tier2Dest = cfg.DestinationPath } switch cfg.Schedule { case "daily": row.Tier2Schedule = "Naponta" case "weekly": row.Tier2Schedule = "Hetente" default: row.Tier2Schedule = cfg.Schedule } if cfg.LastRun != "" { if t, err := time.Parse(time.RFC3339, cfg.LastRun); err == nil { row.Tier2LastRun = t.In(loc).Format("01-02 15:04") } } row.Tier2LastStatus = cfg.LastStatus row.Tier2LastError = cfg.LastError row.Tier2SizeHuman = cfg.LastSizeHuman switch cfg.LastStatus { case "ok": row.Tier2StatusBadge = "Sikeres" case "error": row.Tier2StatusBadge = "Hiba" row.Status = "yellow" row.StatusText = "Utolsó mentés sikertelen" case "running": row.Tier2StatusBadge = "Fut..." default: row.Tier2StatusBadge = "—" } // Destination health check if cfg.Enabled && cfg.DestinationPath != "" { if err := s.crossDrive.ValidateDestination(cfg.DestinationPath); err != nil { if strings.Contains(err.Error(), "does not exist") || strings.Contains(err.Error(), "not writable") { row.Status = "red" row.StatusText = "Mentési cél nem elérhető" } else { row.Status = "yellow" row.StatusText = "Figyelmeztetés" } row.Warnings = append(row.Warnings, err.Error()) } else if row.Status != "yellow" { row.Status = "green" row.StatusText = "Mentés rendben" } } } } // DB dump failure warning (affects Tier 1 quality) if hasDB && tier1DBStatus == "error" { if row.Status != "red" { row.Status = "yellow" row.StatusText = "Adatbázis mentés sikertelen" } } rows = append(rows, row) } return rows } ``` **Note:** This requires `strings` import in handlers.go — check it's present. ### 2c: Update template to per-tier display **File:** `internal/web/templates/backups.html` Replace the backup-layers section inside the `app-backup-row-detail` div (lines 265–322). The current code has three `backup-layer-row` divs (DB, Konfiguráció, Felhasználói adatok). **Replace with two tier rows:** ```html ``` ### 2d: Add CSS for new tier elements **File:** `internal/web/templates/style.css` Add near the existing `.layer-*` styles: ```css .tier-label { font-weight: 600; min-width: 5rem; color: var(--text); } .tier-location { color: var(--text-muted); font-size: .85rem; } .tier-contents { color: var(--text-muted); font-size: .8rem; font-style: italic; margin-left: .25rem; } .tier-size { color: var(--text-muted); font-size: .8rem; margin-left: .25rem; } .tier-browsable { font-size: .75rem; margin-left: .15rem; cursor: help; } ``` --- ## Fix 3: Clean up unused code ### 3a: Remove unused `filterSnapshotsByPaths` and `pathCovers` **File:** `internal/api/router.go` Delete the `filterSnapshotsByPaths` function and the `pathCovers` helper — they were left after v0.12.7a removed the call site. No other code references them. ### 3b: Remove `VolumeLastRun` / `VolumeLastStatus` / `DBLastRun` / `DBLastStatus` fields These fields no longer exist in the new `AppBackupRow` struct (replaced by Tier1/Tier2 fields). Verify no template references to the old field names remain. --- ## Summary of all changes | Fix | What | File(s) | |-----|------|---------| | 1a | `dbDumpDir` field + setter on CrossDriveRunner | `crossdrive.go` | | 1b | `copyStackDBDumps` helper for rsync | `crossdrive.go` | | 1c | rsync includes DB dumps + config | `crossdrive.go` | | 1d | restic includes config dir + DB dump dir | `crossdrive.go` | | 1e | Wire `SetDBDumpDir` in main.go | `main.go` | | 2a | `AppBackupRow` restructured to per-tier | `handlers.go` | | 2b | `buildAppBackupRows` rewritten for per-tier | `handlers.go` | | 2c | Template: 2 tier rows instead of 3 layer rows | `backups.html` | | 2d | CSS for tier elements | `style.css` | | 3a | Remove unused `filterSnapshotsByPaths` + `pathCovers` | `router.go` | | 3b | Verify no old field references remain | `backups.html` | ## Files to modify (6) 1. `internal/backup/crossdrive.go` — Fix 1a + 1b + 1c + 1d 2. `cmd/controller/main.go` — Fix 1e 3. `internal/web/handlers.go` — Fix 2a + 2b 4. `internal/web/templates/backups.html` — Fix 2c 5. `internal/web/templates/style.css` — Fix 2d 6. `internal/api/router.go` — Fix 3a ## Architecture after fix ``` Per-app backup tiers: ┌──────────────────────────────────────────────────────┐ │ TIER 1 — Nightly restic (MANDATORY, same drive) │ │ │ │ Contains: │ │ - DB dumps (pg_dump / mariadb-dump) │ │ - Config (compose.yml, app.yaml, .felhom.yml) │ │ - User data (ALL HDD bind mounts — mandatory) │ │ │ │ Protects against: accidental deletion, corruption │ │ Does NOT protect against: drive failure │ │ │ ├──────────────────────────────────────────────────────┤ │ TIER 2 — Cross-drive backup (OPT-IN, second device) │ │ │ │ Contains (COMPLETE — same as Tier 1): │ │ - DB dumps (copied to _db/ subfolder) │ │ - Config (rsynced to _config/ subfolder) │ │ - User data (rsync or restic to destination) │ │ │ │ rsync layout: │ │ backups/rsync// │ │ _db/ ← DB dump files (browsable) │ │ _config/ ← compose + app.yaml (browsable) │ │ ← user data (browsable) │ │ │ │ restic layout: │ │ backups/restic/ ← encrypted repo (not browsable) │ │ │ │ Protects against: drive failure, drive theft │ │ │ ├──────────────────────────────────────────────────────┤ │ TIER 3 — Remote backup (FUTURE) │ │ │ │ Complete offsite copy. Not implemented yet. │ └──────────────────────────────────────────────────────┘ UI per-app display: ┌─────────────────────────────────────────────────────────┐ │ ● Immich Külső tárhely (hdd_1) 63.9 MB │ │ 1. mentés Auto helyi 02-18 03:00 ✓ DB+Konfig+Adat│ │ 2. mentés rsync → hdd_1 Naponta 02-18 10:48 ✓ │ │ DB+Konfig+Adatok 📁 [Beáll][Futtás]│ ├─────────────────────────────────────────────────────────┤ │ ● Mealie Auto │ │ 1. mentés Auto helyi 02-18 03:00 ✓ Konfig │ ├─────────────────────────────────────────────────────────┤ │ ● Paperless-ngx hdd_placeholder 76 B │ │ 1. mentés Auto helyi 02-18 03:00 ✓ Konfig+Adatok │ │ 2. mentés ✓ 1. mentés auto ⚠ Nincs 2. másolat │ └─────────────────────────────────────────────────────────┘ ``` ## Post-fix checklist - [ ] `go build ./...` passes - [ ] `go vet ./...` passes - [ ] Verify no references to old fields: `DBLastRun`, `DBLastStatus`, `VolumeLastRun`, `VolumeLastStatus`, `HasUserData`, `UserDataConfigured`, `UserDataMethod`, `UserDataDest`, `UserDataSchedule`, `UserDataLastRun`, `UserDataLastStatus`, `UserDataLastError`, `UserDataStatusBadge` - [ ] Verify `filterSnapshotsByPaths` and `pathCovers` deleted from router.go - [ ] Update `CHANGELOG.md` — session 45, version **v0.12.8**: - Cross-drive backup now includes DB dumps + app config (complete backup) - rsync layout: `_db/` and `_config/` subdirs alongside user data - restic cross-drive includes config dir + DB dump dir - UI: restructured from per-layer to per-tier display (1. mentés / 2. mentés) - UI: shows backup contents per app (DB + Konfig + Adatok) - UI: rsync backups show browsable indicator (📁) - Cleanup: removed unused filterSnapshotsByPaths code - [ ] Commit, build on 192.168.0.180, deploy on 192.168.0.162 - [ ] Verify with `docker ps` and `docker logs` - [ ] After deploy, verify: - Immich card shows "1. mentés" and "2. mentés" rows (not DB/Konfig/User data rows) - Immich Tier 2 shows "DB + Konfig + Adatok" contents label - Run manual cross-drive backup for Immich - Check destination: `ls backups/rsync/immich/` should show `_db/`, `_config/`, and user data - Verify `_db/` contains `immich_postgres.sql` - Verify `_config/` contains `docker-compose.yml`, `app.yaml` - Mealie shows only "1. mentés" row (no Tier 2 — no HDD data) - Paperless-ngx shows yellow dot with "⚠ Nincs 2. másolat"