From f9c0338894e4e344ab65cb75da3b4441b51b2749 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Wed, 18 Feb 2026 12:07:34 +0100 Subject: [PATCH] Tier 2 for All Apps + Status Dot Update (v0.12.9) --- CLAUDE.md | 26 +- TASK.md | 865 ++++++++++++++++--------------------------- controller/README.md | 63 ++-- 3 files changed, 370 insertions(+), 584 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 65f7945..f9e0d00 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,9 +24,17 @@ Claude in Chrome extension is available — can be used to test web UI on demo-f - Add debug capabilities (logging, verbose output) for easier troubleshooting - If you need more input or troubleshooting command output, ask first — don't guess +## Environment + +| Machine | OS | IP | Purpose | +|---------|----|----|---------| +| **Local (this machine)** | Windows 11 | — | Development, Claude Code runs here. Repos in `E:\git\` | +| **Build server (k3s, infra)** | Debian 13 | 192.168.0.180 | Build + push container images, k3s cluster | +| **Demo node** | Debian 13 | 192.168.0.162 | Test deployment (demo-felhom.eu) | + ## Workspace layout -Claude Code runs on Windows. The working directory is `E:\git\` (mapped as `/e/git/` in Git Bash). +Claude Code runs on Windows 11. The working directory is `E:\git\` (mapped as `/e/git/` in Git Bash). This repo is at: ``` @@ -84,17 +92,17 @@ SSH=/c/Windows/System32/OpenSSH/ssh.exe All SSH commands in this file use `$SSH` — set it at the start of your session or substitute the full path manually. -| Host | IP | User | Role | -|------|----|------|------| -| Build server | 192.168.0.180 | kisfenyo | Build + push container images | -| Demo node | 192.168.0.162 | kisfenyo | Test deployment (demo-felhom.eu) | +| Host | OS | IP | User | Role | +|------|----|----|------|------| +| Build server | Debian 13 | 192.168.0.180 | kisfenyo | Build + push container images | +| Demo node | Debian 13 | 192.168.0.162 | kisfenyo | Test deployment (demo-felhom.eu) | ## Test environments -| Node | Hardware | Domain | IP | Notes | -|------|----------|--------|----|-------| -| demo-felhom | Acemagic N100, 16G RAM, 512G SSD + 1TB HDD | demo-felhom.eu | 192.168.0.162 | Primary test node, Cloudflare Tunnel | -| pi-customer-1 | Raspberry Pi 3B+, 1G RAM, 32G SD | pi-customer-1.local | 192.168.0.161 | Secondary test, not yet active | +| Node | OS | Hardware | Domain | IP | Notes | +|------|-----|----------|--------|----|-------| +| demo-felhom | Debian 13 | Acemagic N100, 16G RAM, 512G SSD + 1TB HDD | demo-felhom.eu | 192.168.0.162 | Primary test node, Cloudflare Tunnel | +| pi-customer-1 | Debian 13 | Raspberry Pi 3B+, 1G RAM, 32G SD | pi-customer-1.local | 192.168.0.161 | Secondary test, not yet active | - Pi-hole DNS on local network forwards `*.demo-felhom.eu` → 192.168.0.162 - External access via Cloudflare Tunnel → Traefik reverse proxy diff --git a/TASK.md b/TASK.md index b1c4316..7783f3f 100644 --- a/TASK.md +++ b/TASK.md @@ -1,4 +1,4 @@ -# TASK.md — Complete Cross-Drive Backup + Per-Tier UI (v0.12.8) +# TASK.md — Tier 2 for All Apps + Status Dot Update (v0.12.9) ## Prompt (copy-paste this into Claude Code) @@ -6,7 +6,7 @@ Read TASK.md for the full plan. Apply all code changes described, then build and deploy. After all fixes are done: 1. Run `go build ./...` and `go vet ./...` from the controller/ directory — fix any errors -2. Update CHANGELOG.md with a new entry at the top (session 45, v0.12.8) +2. Update CHANGELOG.md with a new entry at the top (session 46, v0.12.9) 3. Commit, build, and deploy following the workflow in CLAUDE.md ``` @@ -14,525 +14,304 @@ After all fixes are done: ## Context and Goals -The cross-drive backup (Tier 2) currently only copies HDD user data (mounts). It does NOT -include DB dumps or app config. If the primary drive fails, the customer loses their database -and config — even though photos are safe on the second drive. **This is not a viable backup.** +Currently Tier 2 (cross-drive backup) is only available for apps with HDD data (Immich, +Paperless-ngx, etc.). Apps without HDD data (Mealie, Gokapi, etc.) cannot configure Tier 2 +at all — the config section is hidden, the code rejects empty mounts, and the UI shows them +as "auto" (gray dot). -Each tier must be a **complete, self-sufficient backup**: +**Problem:** These apps have only 1 tier of protection. If the primary drive fails, their DB +and config are lost. The customer should be able to configure Tier 2 for ANY app. -| Tier | Contents | Location | Can fully restore? | -|------|----------|----------|--------------------| -| 1. Nightly restic | DB + Config + User data | Same drive | Yes (not against drive failure) | -| 2. Cross-drive | DB + Config + User data | Different drive | **Yes (after this fix)** | -| 3. Remote (future) | Everything | Offsite | Yes | +**Changes in this version:** -This task also restructures the UI from per-layer (DB/Config/UserData rows) to per-tier -(1st backup / 2nd backup rows) — matching the customer mental model. +1. **Tier 2 for all apps** — Remove all HDD-only gates. Non-HDD apps back up config + DB + dumps to the secondary drive (small, but protects against drive failure). +2. **Status dot update** — Remove "auto" (gray). All apps start as yellow (1 tier only). + Green requires 2+ tiers with successful backups. +3. **Tier 3 placeholder** — Show a disabled "3. mentés" row in the UI (future: remote backup). +4. **Deploy page** — Show cross-drive config form for ALL deployed apps, not just HDD ones. + +**What non-HDD apps back up in Tier 2:** +- App with DB (e.g., Mealie): `_config/` + `_db/mealie_postgres.sql` +- App without DB (e.g., Gokapi): `_config/` only +- Small files — seconds to rsync/restic, but provides drive-failure protection. --- -## Fix 1: Include DB dumps + config in cross-drive backup - -### 1a: Add `dbDumpDir` field to CrossDriveRunner +## Fix 1: Remove empty-mounts gate from RunAppBackup **File:** `internal/backup/crossdrive.go` -Add a `dbDumpDir` field to the struct: +In `RunAppBackup()`, the code currently errors out when no HDD mounts exist (lines 98–103): ```go -type CrossDriveRunner struct { - sett *settings.Settings - stackProvider StackDataProvider - dbDumper DBDumper - dbDumpDir string // path to DB dump directory (e.g., /srv/backups/db-dumps) - logger *log.Logger - mu sync.Mutex - running map[string]bool -} -``` - -Add setter after `SetDBDumper`: - -```go -// SetDBDumpDir sets the path to the DB dump directory for cross-drive backups. -func (r *CrossDriveRunner) SetDBDumpDir(dir string) { - r.dbDumpDir = dir -} -``` - -### 1b: Add helper to copy DB dump files for a stack - -**File:** `internal/backup/crossdrive.go`, add helper function: - -```go -// copyStackDBDumps copies DB dump files for the given stack to destDir. -// DB dump files are named _.sql (e.g., immich_postgres.sql). -// Small files — uses plain file copy, not rsync. -func (r *CrossDriveRunner) copyStackDBDumps(stackName, destDir string) error { - if r.dbDumpDir == "" { - return nil - } - entries, err := os.ReadDir(r.dbDumpDir) - if err != nil { - if os.IsNotExist(err) { - return nil - } - return fmt.Errorf("reading DB dump dir: %w", err) - } - prefix := stackName + "_" - copied := 0 - for _, e := range entries { - if e.IsDir() || !strings.HasPrefix(e.Name(), prefix) { - continue - } - src := filepath.Join(r.dbDumpDir, e.Name()) - dst := filepath.Join(destDir, e.Name()) - data, err := os.ReadFile(src) - if err != nil { - return fmt.Errorf("reading %s: %w", e.Name(), err) - } - if err := os.WriteFile(dst, data, 0644); err != nil { - return fmt.Errorf("writing %s: %w", e.Name(), err) - } - copied++ - } - if copied > 0 { - r.logger.Printf("[DEBUG] Copied %d DB dump file(s) to %s", copied, destDir) - } - return nil -} -``` - -### 1c: Update `runRsyncBackup` to include DB + config - -**File:** `internal/backup/crossdrive.go`, function `runRsyncBackup` - -After the existing HDD mount rsync loop (after line 260 `return nil` of the for loop), -add these blocks BEFORE the final `return nil`: - -```go - // --- Copy DB dumps for this stack --- - dbDestDir := filepath.Join(destDir, "_db") - if err := os.MkdirAll(dbDestDir, 0755); err != nil { - return fmt.Errorf("creating DB dump dest dir: %w", err) - } - if err := r.copyStackDBDumps(stackName, dbDestDir); err != nil { - r.logger.Printf("[WARN] Cross-drive DB dump copy failed for %s: %v", stackName, err) - // Non-fatal: user data is the primary concern - } - - // --- Rsync app config (compose dir) --- - if composePath, ok := r.stackProvider.GetStackComposePath(stackName); ok { - configSrcDir := filepath.Dir(composePath) - configDestDir := filepath.Join(destDir, "_config") - if err := os.MkdirAll(configDestDir, 0755); err != nil { - return fmt.Errorf("creating config dest dir: %w", err) - } - src := strings.TrimRight(configSrcDir, "/") + "/" - dst := strings.TrimRight(configDestDir, "/") + "/" - cmd := exec.CommandContext(ctx, "rsync", "-a", "--delete", src, dst) - r.logger.Printf("[DEBUG] rsync config: %s → %s", src, dst) - if out, err := cmd.CombinedOutput(); err != nil { - r.logger.Printf("[WARN] Cross-drive config rsync failed for %s: %v (%s)", stackName, err, strings.TrimSpace(string(out))) - // Non-fatal - } - } -``` - -**Resulting rsync destination layout:** -``` -backups/rsync// - _db/ ← stackName_postgres.sql, stackName_mariadb.sql - _config/ ← compose.yml, app.yaml, .felhom.yml - ← existing HDD mount contents (unchanged) -``` - -The `_` prefix prevents collision with user data directories (HDD mounts never start with `_`). - -### 1d: Update `runResticBackup` to include DB + config - -**File:** `internal/backup/crossdrive.go`, function `runResticBackup` - -Currently the restic backup only includes `mounts`. Change to also include the config dir -and DB dump dir. - -**Current code (around line 294–301):** -```go - args := []string{ - "backup", "--repo", repoPath, - "--password-file", pwPath, - "--tag", stackName, - "--tag", "cross-drive", - } - args = append(args, mounts...) +// CURRENT CODE — DELETE these 4 lines: + mounts := r.stackProvider.GetStackHDDMounts(stackName) + if len(mounts) == 0 { + r.updateStatus(stackName, "error", "no HDD data paths found for this app", time.Since(start), "") + return fmt.Errorf("no HDD data paths found for %s", stackName) + } ``` **Replace with:** -```go - args := []string{ - "backup", "--repo", repoPath, - "--password-file", pwPath, - "--tag", stackName, - "--tag", "cross-drive", - } - // Include user data (HDD mounts) - args = append(args, mounts...) - // Include app config dir (compose + app.yaml + .felhom.yml) - if composePath, ok := r.stackProvider.GetStackComposePath(stackName); ok { - args = append(args, filepath.Dir(composePath)) - } - // Include DB dump dir (all stacks' dumps — restic deduplicates) - if r.dbDumpDir != "" { - if _, err := os.Stat(r.dbDumpDir); err == nil { - args = append(args, r.dbDumpDir) - } - } -``` - -Note: for restic, including the full DB dump dir is fine — restic deduplicates and it's -shared across apps. The `--tag ` identifies which app this backup belongs to. - -### 1e: Wire up in main.go - -**File:** `cmd/controller/main.go`, after the existing `crossDriveRunner.SetDBDumper(backupMgr)` line: ```go - crossDriveRunner.SetDBDumpDir(cfg.Paths.DBDumpDir) + // Resolve HDD mounts for this app (may be empty for config-only apps) + mounts := r.stackProvider.GetStackHDDMounts(stackName) ``` +**Why this works:** The rest of the function already handles empty mounts correctly: +- Safety overlap check: empty loop = no overlap → passes +- `runRsyncBackup`: mount loop doesn't execute, but DB + config copy still runs +- `runResticBackup`: no mount paths appended, but config dir + DB dump dir still included +- Size calculation: destDir exists and can be measured even without mount data + --- -## Fix 2: Restructure UI from per-layer to per-tier - -The current UI shows 3 rows per app (DB / Konfiguráció / Felhasználói adatok). This -maps to implementation details, not the customer mental model. Change to 2 rows per app: -- **1. mentés** — nightly restic (mandatory, same drive) -- **2. mentés** — cross-drive (opt-in, different drive) - -### 2a: Restructure `AppBackupRow` struct +## Fix 2: Update status dot logic + remove HasHDDData gates from handlers.go **File:** `internal/web/handlers.go` -**Replace the current `AppBackupRow` struct (lines 500–533) with:** +### 2a: Update `AppBackupRow` struct comments + +In the `AppBackupRow` struct, update the Tier 2 comment: ```go -// AppBackupRow holds per-tier backup information for one app on the backup page. -type AppBackupRow struct { - StackName string - DisplayName string - Status string // "green", "yellow", "red", "auto" - StatusText string // short Hungarian tooltip - - // App characteristics - HasHDDData bool - HasDB bool - StorageLabel string - HDDSizeHuman string - - // What this app's backup contains (for display) - // e.g., "DB + Konfiguráció + Adatok", "DB + Konfiguráció", "Konfiguráció" - BackupContents string - - // Tier 1: Nightly backup (always exists) - Tier1LastRun string // formatted time of last restic snapshot - Tier1LastStatus string // "ok", "error", "" - Tier1DBStatus string // "ok", "error", "" — separate DB dump status for warning - - // Tier 2: Cross-drive backup (only for apps with HDD data) - Tier2Configured bool - Tier2Method string // "rsync", "restic" - Tier2MethodLabel string // "rsync", "restic" - Tier2Dest string // destination label - Tier2Schedule string // "Naponta", "Hetente" - Tier2LastRun string - Tier2LastStatus string // "ok", "error", "running", "" - Tier2LastError string - Tier2StatusBadge string // "Sikeres", "Hiba", "Fut...", "—" - Tier2SizeHuman string - Tier2Browsable bool // true for rsync (plain files), false for restic - - // Warnings accumulated for this app - Warnings []string -} + // Tier 2: Cross-drive backup (configurable for all apps) ``` -### 2b: Update `buildAppBackupRows` +(Remove the old "(only for apps with HDD data)" comment.) -**File:** `internal/web/handlers.go` +### 2b: Rewrite `buildAppBackupRows` status + Tier2 section -**Replace the entire `buildAppBackupRows` function** with: +Replace the current status + Tier2 block (lines 605–672): + +**CURRENT CODE:** +```go + // Default status = auto (no user data, just config) + row.Status = "auto" + row.StatusText = "Automatikus mentés" + + if app.HasHDDData { + cfg, hasCfg := crossConfigs[app.StackName] + // ... full Tier2 block ... + } +``` + +**REPLACE WITH:** +```go + // Status dot — start as yellow (1 tier only) + row.Status = "yellow" + row.StatusText = "Csak helyi mentés (1 szint)" + + cfg, hasCfg := crossConfigs[app.StackName] + + if !hasCfg || cfg == nil || !cfg.Enabled { + // Only Tier 1 — no second copy + row.Tier2Configured = false + } else { + row.Tier2Configured = true + row.Tier2Method = cfg.Method + row.Tier2MethodLabel = cfg.Method // "rsync" or "restic" + row.Tier2Browsable = cfg.Method == "rsync" + row.Tier2Dest = destLabels[cfg.DestinationPath] + if row.Tier2Dest == "" { + row.Tier2Dest = cfg.DestinationPath + } + switch cfg.Schedule { + case "daily": + row.Tier2Schedule = "Naponta" + case "weekly": + row.Tier2Schedule = "Hetente" + default: + row.Tier2Schedule = cfg.Schedule + } + if cfg.LastRun != "" { + if t, err := time.Parse(time.RFC3339, cfg.LastRun); err == nil { + row.Tier2LastRun = t.In(loc).Format("01-02 15:04") + } + } + row.Tier2LastStatus = cfg.LastStatus + row.Tier2LastError = cfg.LastError + row.Tier2SizeHuman = cfg.LastSizeHuman + switch cfg.LastStatus { + case "ok": + row.Tier2StatusBadge = "Sikeres" + row.Status = "green" + row.StatusText = "Mentés rendben" + case "error": + row.Tier2StatusBadge = "Hiba" + // Status stays yellow + row.StatusText = "Utolsó mentés sikertelen" + case "running": + row.Tier2StatusBadge = "Fut..." + default: + row.Tier2StatusBadge = "—" + // Tier2 configured but never run — stay yellow + } + + // Destination health check — can downgrade green to yellow/red + if cfg.DestinationPath != "" { + if err := s.crossDriveRunner.ValidateDestination(cfg.DestinationPath); err != nil { + if strings.Contains(err.Error(), "does not exist") || strings.Contains(err.Error(), "not writable") { + row.Status = "red" + row.StatusText = "Mentési cél nem elérhető" + } else if row.Status != "red" { + row.Status = "yellow" + row.StatusText = "Figyelmeztetés" + } + row.Warnings = append(row.Warnings, err.Error()) + } + } + } +``` + +Note: `s.crossDriveRunner` is used instead of `s.crossDrive` — verify the field name +in `server.go` and use whatever the actual field is called. (The current code at the +old line 657 shows `s.crossDriveRunner.ValidateDestination`.) + +### 2c: Remove HasHDDData gate from cross-drive summary builder + +In `backupsHandler`, the cross-drive summary loop (around line 409) has: ```go -// buildAppBackupRows constructs one AppBackupRow per deployed app for the backup page. -func (s *Server) buildAppBackupRows( - status *backup.FullBackupStatus, - crossConfigs map[string]*settings.CrossDriveBackup, - destLabels map[string]string, -) []AppBackupRow { - loc := getTimezone() - - // Build DB stack lookup - dbStacks := make(map[string]bool) - for _, db := range status.DiscoveredDBs { - dbStacks[db.StackName] = true - } - for _, f := range status.DumpFiles { - dbStacks[f.StackName] = true - } - - // Tier 1 timestamps (shared across all apps — single nightly job) - tier1LastRun := "" - tier1LastStatus := "" - if status.LastBackup != nil { - tier1LastRun = status.LastBackup.LastRun.In(loc).Format("01-02 15:04") - if status.LastBackup.Success { - tier1LastStatus = "ok" - } else { - tier1LastStatus = "error" - } - } - tier1DBStatus := "" - if status.LastDBDump != nil { - if status.LastDBDump.Success { - tier1DBStatus = "ok" - } else { - tier1DBStatus = "error" - } - } - - var rows []AppBackupRow - for _, app := range status.AppDataInfo { - hasDB := dbStacks[app.StackName] || app.HasDBDump - - // Build backup contents label - var parts []string - if hasDB { - parts = append(parts, "DB") - } - parts = append(parts, "Konfig") - if app.HasHDDData { - parts = append(parts, "Adatok") - } - contents := strings.Join(parts, " + ") - - row := AppBackupRow{ - StackName: app.StackName, - DisplayName: app.DisplayName, - HasHDDData: app.HasHDDData, - HasDB: hasDB, - StorageLabel: app.StorageLabel, - HDDSizeHuman: app.HDDSizeHuman, - BackupContents: contents, - - Tier1LastRun: tier1LastRun, - Tier1LastStatus: tier1LastStatus, - Tier1DBStatus: tier1DBStatus, - } - - // Default status = auto (no user data, just config) - row.Status = "auto" - row.StatusText = "Automatikus mentés" - - if app.HasHDDData { - cfg, hasCfg := crossConfigs[app.StackName] - - if !hasCfg || cfg == nil || !cfg.Enabled { - // HDD data backed up via nightly restic (mandatory), but no second copy - row.Tier2Configured = false - row.Status = "yellow" - row.StatusText = "Nincs második másolat (csak helyi mentés)" - } else { - row.Tier2Configured = true - row.Tier2Method = cfg.Method - row.Tier2MethodLabel = cfg.Method // "rsync" or "restic" - row.Tier2Browsable = cfg.Method == "rsync" - row.Tier2Dest = destLabels[cfg.DestinationPath] - if row.Tier2Dest == "" { - row.Tier2Dest = cfg.DestinationPath - } - switch cfg.Schedule { - case "daily": - row.Tier2Schedule = "Naponta" - case "weekly": - row.Tier2Schedule = "Hetente" - default: - row.Tier2Schedule = cfg.Schedule - } - if cfg.LastRun != "" { - if t, err := time.Parse(time.RFC3339, cfg.LastRun); err == nil { - row.Tier2LastRun = t.In(loc).Format("01-02 15:04") - } - } - row.Tier2LastStatus = cfg.LastStatus - row.Tier2LastError = cfg.LastError - row.Tier2SizeHuman = cfg.LastSizeHuman - switch cfg.LastStatus { - case "ok": - row.Tier2StatusBadge = "Sikeres" - case "error": - row.Tier2StatusBadge = "Hiba" - row.Status = "yellow" - row.StatusText = "Utolsó mentés sikertelen" - case "running": - row.Tier2StatusBadge = "Fut..." - default: - row.Tier2StatusBadge = "—" - } - - // Destination health check - if cfg.Enabled && cfg.DestinationPath != "" { - if err := s.crossDrive.ValidateDestination(cfg.DestinationPath); err != nil { - if strings.Contains(err.Error(), "does not exist") || strings.Contains(err.Error(), "not writable") { - row.Status = "red" - row.StatusText = "Mentési cél nem elérhető" - } else { - row.Status = "yellow" - row.StatusText = "Figyelmeztetés" - } - row.Warnings = append(row.Warnings, err.Error()) - } else if row.Status != "yellow" { - row.Status = "green" - row.StatusText = "Mentés rendben" - } - } - } - } - - // DB dump failure warning (affects Tier 1 quality) - if hasDB && tier1DBStatus == "error" { - if row.Status != "red" { - row.Status = "yellow" - row.StatusText = "Adatbázis mentés sikertelen" - } - } - - rows = append(rows, row) - } - return rows -} + for _, app := range fullStatus.AppDataInfo { + if !app.HasHDDData { + continue + } ``` -**Note:** This requires `strings` import in handlers.go — check it's present. +**Remove** the `if !app.HasHDDData { continue }` — all apps participate in cross-drive summary. -### 2c: Update template to per-tier display +### 2d: Update the top-level warning logic + +The "NoUserDataBackupWarning" check (around line 473) uses `HasHDDData` — **keep this as-is**. +This warning is specifically about user data (photos, documents) being at risk, which only +applies to HDD apps. The status dot change already incentivizes Tier 2 for all apps. + +--- + +## Fix 3: Update backup page template **File:** `internal/web/templates/backups.html` -Replace the backup-layers section inside the `app-backup-row-detail` div (lines 265–322). -The current code has three `backup-layer-row` divs (DB, Konfiguráció, Felhasználói adatok). - -**Replace with two tier rows:** +### 3a: Remove HasHDDData gate on Tier 2 row +**Find** (around line 284): ```html - ``` -### 2d: Add CSS for new tier elements +**Remove** the `{{if .HasHDDData}}` opening and its matching `{{end}}` (around line 313). +The Tier 2 row should always be shown for all apps. -**File:** `internal/web/templates/style.css` +### 3b: Update header meta badges -Add near the existing `.layer-*` styles: +**Find** (around line 256–261): +```html + {{if .HasHDDData}} + {{if .StorageLabel}}{{.StorageLabel}}{{end}} + {{.HDDSizeHuman}} + {{else}} + Auto + {{end}} +``` -```css -.tier-label { - font-weight: 600; - min-width: 5rem; - color: var(--text); -} -.tier-location { - color: var(--text-muted); - font-size: .85rem; -} -.tier-contents { - color: var(--text-muted); - font-size: .8rem; - font-style: italic; - margin-left: .25rem; -} -.tier-size { - color: var(--text-muted); - font-size: .8rem; - margin-left: .25rem; -} -.tier-browsable { - font-size: .75rem; - margin-left: .15rem; - cursor: help; -} +**Replace with:** +```html + {{if .HasHDDData}} + {{if .StorageLabel}}{{.StorageLabel}}{{end}} + {{.HDDSizeHuman}} + {{else}} + Konfig{{if .HasDB}} + DB{{end}} + {{end}} +``` + +This shows what type of data the app has (instead of meaningless "Auto"). + +### 3c: Add Tier 3 placeholder row + +After the Tier 2 `` (the closing div of the tier-2 backup-layer-row), add: + +```html + +
+ 3. mentés + Hamarosan + távoli (offsite) + B2 / S3 / SFTP — hamarosan elérhető +
+``` + +### 3d: Update "Run all" button text + +**Find** (around line 328): +```html + +``` + +**Replace with:** +```html + ``` --- -## Fix 3: Clean up unused code +## Fix 4: Deploy page — show cross-drive config for all deployed apps -### 3a: Remove unused `filterSnapshotsByPaths` and `pathCovers` +**File:** `internal/web/templates/deploy.html` -**File:** `internal/api/router.go` +### 4a: Remove StorageInfo gate from cross-drive section -Delete the `filterSnapshotsByPaths` function and the `pathCovers` helper — they were -left after v0.12.7a removed the call site. No other code references them. +The cross-drive backup config section is currently double-gated (lines 95–220): -### 3b: Remove `VolumeLastRun` / `VolumeLastStatus` / `DBLastRun` / `DBLastStatus` fields +```html + {{if .AlreadyDeployed}} + {{if .StorageInfo}} ← THIS IS THE GATE — remove it +
+ ... +
+ {{end}} ← remove matching end + {{end}} +``` -These fields no longer exist in the new `AppBackupRow` struct (replaced by Tier1/Tier2 -fields). Verify no template references to the old field names remain. +**Change to** (keep only the AlreadyDeployed gate): + +```html + {{if .AlreadyDeployed}} +
+ ... +
+ {{end}} +``` + +### 4b: Update section text for generality + +**Find** (line 109): +```html +

Másolat másik meghajtóra (felhasználói adatok):

+``` + +**Replace with:** +```html +

2. mentés — másolat másik meghajtóra:

+``` + +**Find** (line 214–216): +```html +
+ A cél meghajtó legyen más fizikai eszköz, mint az alkalmazás adattárolója. +
+``` + +**Replace with:** +```html +
+ A cél meghajtó legyen más fizikai eszköz a meghibásodás elleni védelem érdekében. +
+``` --- @@ -540,108 +319,86 @@ fields). Verify no template references to the old field names remain. | Fix | What | File(s) | |-----|------|---------| -| 1a | `dbDumpDir` field + setter on CrossDriveRunner | `crossdrive.go` | -| 1b | `copyStackDBDumps` helper for rsync | `crossdrive.go` | -| 1c | rsync includes DB dumps + config | `crossdrive.go` | -| 1d | restic includes config dir + DB dump dir | `crossdrive.go` | -| 1e | Wire `SetDBDumpDir` in main.go | `main.go` | -| 2a | `AppBackupRow` restructured to per-tier | `handlers.go` | -| 2b | `buildAppBackupRows` rewritten for per-tier | `handlers.go` | -| 2c | Template: 2 tier rows instead of 3 layer rows | `backups.html` | -| 2d | CSS for tier elements | `style.css` | -| 3a | Remove unused `filterSnapshotsByPaths` + `pathCovers` | `router.go` | -| 3b | Verify no old field references remain | `backups.html` | +| 1 | Remove `len(mounts) == 0` error gate | `crossdrive.go` | +| 2a | Update `AppBackupRow` Tier2 comment | `handlers.go` | +| 2b | Rewrite status + Tier2 block (remove HasHDDData gate, new dot logic) | `handlers.go` | +| 2c | Remove HasHDDData gate from cross-drive summary | `handlers.go` | +| 3a | Remove `{{if .HasHDDData}}` around Tier 2 row | `backups.html` | +| 3b | Update meta badges ("Auto" → "Konfig + DB") | `backups.html` | +| 3c | Add Tier 3 placeholder row | `backups.html` | +| 3d | Rename "Összes HDD mentés" → "Összes 2. mentés" | `backups.html` | +| 4a | Remove `{{if .StorageInfo}}` gate from cross-drive section | `deploy.html` | +| 4b | Update cross-drive section text for generality | `deploy.html` | -## Files to modify (6) +## Files to modify (4) -1. `internal/backup/crossdrive.go` — Fix 1a + 1b + 1c + 1d -2. `cmd/controller/main.go` — Fix 1e -3. `internal/web/handlers.go` — Fix 2a + 2b -4. `internal/web/templates/backups.html` — Fix 2c -5. `internal/web/templates/style.css` — Fix 2d -6. `internal/api/router.go` — Fix 3a +1. `internal/backup/crossdrive.go` — Fix 1 +2. `internal/web/handlers.go` — Fix 2a + 2b + 2c +3. `internal/web/templates/backups.html` — Fix 3a + 3b + 3c + 3d +4. `internal/web/templates/deploy.html` — Fix 4a + 4b + +## Status dot logic after fix + +| Dot color | Meaning | +|-----------|---------| +| Green | 2+ tiers with successful backups + destination healthy | +| Yellow | Only 1 tier, or Tier 2 failing, or Tier 2 configured but never run | +| Red | Tier 2 destination blocked/inaccessible | + +**"auto" (gray) is removed.** Every app now shows yellow or better. ## Architecture after fix ``` -Per-app backup tiers: -┌──────────────────────────────────────────────────────┐ -│ TIER 1 — Nightly restic (MANDATORY, same drive) │ -│ │ -│ Contains: │ -│ - DB dumps (pg_dump / mariadb-dump) │ -│ - Config (compose.yml, app.yaml, .felhom.yml) │ -│ - User data (ALL HDD bind mounts — mandatory) │ -│ │ -│ Protects against: accidental deletion, corruption │ -│ Does NOT protect against: drive failure │ -│ │ -├──────────────────────────────────────────────────────┤ -│ TIER 2 — Cross-drive backup (OPT-IN, second device) │ -│ │ -│ Contains (COMPLETE — same as Tier 1): │ -│ - DB dumps (copied to _db/ subfolder) │ -│ - Config (rsynced to _config/ subfolder) │ -│ - User data (rsync or restic to destination) │ -│ │ -│ rsync layout: │ -│ backups/rsync// │ -│ _db/ ← DB dump files (browsable) │ -│ _config/ ← compose + app.yaml (browsable) │ -│ ← user data (browsable) │ -│ │ -│ restic layout: │ -│ backups/restic/ ← encrypted repo (not browsable) │ -│ │ -│ Protects against: drive failure, drive theft │ -│ │ -├──────────────────────────────────────────────────────┤ -│ TIER 3 — Remote backup (FUTURE) │ -│ │ -│ Complete offsite copy. Not implemented yet. │ -└──────────────────────────────────────────────────────┘ +Per-app Tier 2 availability: +┌──────────────────────────────────────────────────────────┐ +│ App type │ Tier 1 │ Tier 2 (new) │ +│───────────────────│───────────────────│──────────────────│ +│ HDD + DB │ Config+DB+Data │ Config+DB+Data │ +│ HDD, no DB │ Config+Data │ Config+Data │ +│ DB, no HDD │ Config+DB │ Config+DB (new!) │ +│ Config only │ Config │ Config (new!) │ +└──────────────────────────────────────────────────────────┘ -UI per-app display: -┌─────────────────────────────────────────────────────────┐ -│ ● Immich Külső tárhely (hdd_1) 63.9 MB │ -│ 1. mentés Auto helyi 02-18 03:00 ✓ DB+Konfig+Adat│ -│ 2. mentés rsync → hdd_1 Naponta 02-18 10:48 ✓ │ -│ DB+Konfig+Adatok 📁 [Beáll][Futtás]│ -├─────────────────────────────────────────────────────────┤ -│ ● Mealie Auto │ -│ 1. mentés Auto helyi 02-18 03:00 ✓ Konfig │ -├─────────────────────────────────────────────────────────┤ -│ ● Paperless-ngx hdd_placeholder 76 B │ -│ 1. mentés Auto helyi 02-18 03:00 ✓ Konfig+Adatok │ -│ 2. mentés ✓ 1. mentés auto ⚠ Nincs 2. másolat │ -└─────────────────────────────────────────────────────────┘ +UI per-app display after fix: +┌─────────────────────────────────────────────────────────────┐ +│ 🟢 Immich Külső tárhely (hdd_1) 63.9 MB │ +│ 1. mentés Auto helyi 02-18 03:00 ✓ DB+Konfig+Adatok │ +│ 2. mentés rsync → hdd_1 Naponta Sikeres 📁 │ +│ 3. mentés Hamarosan távoli (offsite) │ +├─────────────────────────────────────────────────────────────┤ +│ 🟡 Mealie Konfig + DB │ +│ 1. mentés Auto helyi 02-18 03:00 ✓ DB+Konfig │ +│ 2. mentés ✓ 1. mentés auto ⚠ Nincs 2. másolat │ +│ 3. mentés Hamarosan távoli (offsite) │ +├─────────────────────────────────────────────────────────────┤ +│ 🟡 Gokapi Konfig │ +│ 1. mentés Auto helyi 02-18 03:00 ✓ Konfig │ +│ 2. mentés ⚠ Nincs 2. másolat [Beállítás →] │ +│ 3. mentés Hamarosan távoli (offsite) │ +└─────────────────────────────────────────────────────────────┘ ``` ## Post-fix checklist - [ ] `go build ./...` passes - [ ] `go vet ./...` passes -- [ ] Verify no references to old fields: `DBLastRun`, `DBLastStatus`, `VolumeLastRun`, - `VolumeLastStatus`, `HasUserData`, `UserDataConfigured`, `UserDataMethod`, - `UserDataDest`, `UserDataSchedule`, `UserDataLastRun`, `UserDataLastStatus`, - `UserDataLastError`, `UserDataStatusBadge` -- [ ] Verify `filterSnapshotsByPaths` and `pathCovers` deleted from router.go -- [ ] Update `CHANGELOG.md` — session 45, version **v0.12.8**: - - Cross-drive backup now includes DB dumps + app config (complete backup) - - rsync layout: `_db/` and `_config/` subdirs alongside user data - - restic cross-drive includes config dir + DB dump dir - - UI: restructured from per-layer to per-tier display (1. mentés / 2. mentés) - - UI: shows backup contents per app (DB + Konfig + Adatok) - - UI: rsync backups show browsable indicator (📁) - - Cleanup: removed unused filterSnapshotsByPaths code +- [ ] Verify no references to old "auto" status remain in handlers.go +- [ ] Verify no template references to removed `{{if .HasHDDData}}` gate on Tier 2 +- [ ] Update `CHANGELOG.md` — session 46, version **v0.12.9**: + - Tier 2 cross-drive backup now configurable for ALL apps (not just HDD apps) + - Non-HDD apps back up config + DB dumps to secondary drive + - Status dot: removed "auto" (gray) — all apps start yellow, green requires 2+ tiers + - Tier 3 placeholder row shown in UI + - Deploy page: cross-drive config form visible for all deployed apps + - Updated button text "Összes 2. mentés futtatása most" - [ ] Commit, build on 192.168.0.180, deploy on 192.168.0.162 - [ ] Verify with `docker ps` and `docker logs` - [ ] After deploy, verify: - - Immich card shows "1. mentés" and "2. mentés" rows (not DB/Konfig/User data rows) - - Immich Tier 2 shows "DB + Konfig + Adatok" contents label - - Run manual cross-drive backup for Immich - - Check destination: `ls backups/rsync/immich/` should show `_db/`, `_config/`, and user data - - Verify `_db/` contains `immich_postgres.sql` - - Verify `_config/` contains `docker-compose.yml`, `app.yaml` - - Mealie shows only "1. mentés" row (no Tier 2 — no HDD data) - - Paperless-ngx shows yellow dot with "⚠ Nincs 2. másolat" + - Immich: green dot (Tier 2 configured + successful backup) + - Mealie: yellow dot with "Csak helyi mentés (1 szint)" + - Mealie: Tier 2 row shown with "⚠ Nincs 2. másolat" + "Beállítás →" link + - Mealie deploy page: cross-drive config form visible + - Configure Tier 2 for Mealie → run manual backup → verify dot turns green + - Tier 3 placeholder row shown for all apps (grayed out "Hamarosan") + - Gokapi: yellow dot, Tier 2 configurable diff --git a/controller/README.md b/controller/README.md index 4e7cb8b..d2de07b 100644 --- a/controller/README.md +++ b/controller/README.md @@ -4,7 +4,7 @@ A single, lightweight Go container that replaces Portainer + scattered systemd scripts with a unified, Hungarian-language web dashboard for managing Docker Compose stacks, backups, storage, monitoring, and notifications on customer hardware. -**Current version: v0.12.7** +**Current version: v0.12.9** --- @@ -177,8 +177,19 @@ self-sufficient backup** — any single tier can fully restore an app. automatically. There is no per-app toggle. - Each tier includes **everything** needed to restore: DB dumps, config, and user data. No tier depends on another tier's data. +- **Tier 2 is configurable for ALL apps** — not just apps with HDD data. Non-HDD apps + back up config + DB dumps to the secondary drive (small but protects against drive failure). - The `AppBackupPrefs.Enabled` field in settings.json is legacy and not read by any code. +**Per-app Tier 2 contents by app type:** + +| App type | Tier 2 contents | Example | +|----------|----------------|---------| +| HDD + DB | Config + DB + User data | Immich, Paperless-ngx | +| HDD, no DB | Config + User data | — | +| DB, no HDD | Config + DB | Mealie, Vikunja | +| Config only | Config | Gokapi, Homepage | + #### Tier 1: Nightly Backup (mandatory, same drive) The nightly backup has two phases that run sequentially: @@ -208,13 +219,16 @@ Does NOT protect against drive failure (backup is on the same physical drive). #### Tier 2: Cross-Drive Backup (opt-in, different device) (`internal/backup/crossdrive.go`) -**Complete backup** to a different physical drive — DB dumps + config + user data. +**Complete backup** to a different physical drive. Available for **all apps** — apps with HDD +data back up config + DB + user data; apps without HDD back up config + DB dumps only. - **Two methods:** - **rsync** — Simple mirror with `--delete` (fast, no versioning, **browsable** on disk) - **restic** — Versioned, deduplicated, encrypted (shared repo across apps, not browsable) - Per-app configuration in settings.json: destination path, method, schedule (daily/weekly/manual) - **Pre-backup DB dump:** `DumpStackDB()` runs fresh pg_dump/mariadb-dump before each cross-drive backup; non-fatal on failure (wired via `DBDumper` interface to avoid circular imports) +- **Empty mounts allowed:** `RunAppBackup` accepts apps with no HDD mounts — the rsync + mount loop simply doesn't execute, but DB + config copy still runs - **Drive-type-aware validation** (`ValidateDestination`): | Destination type | Space checks | @@ -227,12 +241,13 @@ Does NOT protect against drive failure (backup is on the same physical drive). backups/rsync// _db/ ← DB dump files (stackName_postgres.sql, etc.) _config/ ← compose.yml, app.yaml, .felhom.yml - ← HDD mount contents (single mount: flat; multi-mount: leaf subfolders) + ← HDD mount contents (only for apps with HDD data) ``` - DB dump files excluded from user data rsync (`--exclude backups/*.sql.gz/sql/dump`) to avoid duplicating app-internal dumps - `_` prefix directories prevent collision with user data -- **Restic backup paths:** includes HDD mounts + config dir + DB dump dir (deduplication handles overlap) -- Safety guards: destination ≠ source, path-overlap check, writable check + - For non-HDD apps, only `_db/` and `_config/` are present (no user data directory) +- **Restic backup paths:** includes HDD mounts (if any) + config dir + DB dump dir (deduplication handles overlap) +- Safety guards: destination ≠ source, path-overlap check (HDD mounts only), writable check - **Chained execution:** runs immediately after nightly restic — daily apps every night, weekly apps on Sundays - Per-app concurrency lock prevents overlapping runs - Status (last_run, duration, size, error) persisted to settings.json @@ -242,6 +257,7 @@ Does NOT protect against drive failure (backup is on the same physical drive). #### Tier 3: Remote Backup (future) Complete offsite backup for disaster recovery. Not yet implemented. +Placeholder shown in UI ("3. mentés — Hamarosan"). #### Restore (`internal/backup/restore.go`) @@ -250,18 +266,18 @@ All deployed apps appear in the restore dropdown — every app has restic snapsh | App type | Config restored | DB restored | User data restored | |----------|----------------|------------|-------------------| -| Has HDD data | ✓ | ✓ | ✓ (always — backup is mandatory) | -| DB only, no HDD | ✓ | ✓ | n/a | -| No DB, no HDD | ✓ | — | n/a | +| Has HDD data | Yes | Yes | Yes (always — backup is mandatory) | +| DB only, no HDD | Yes | Yes | n/a | +| No DB, no HDD | Yes | — | n/a | - **Snapshot API** returns ALL snapshots unfiltered — older snapshots still allow config+DB restore; `RestoreApp` extracts whatever paths are available - **Restore type info** shown per-app when selected in dropdown (Hungarian banners): - - Has HDD: "Teljes visszaállítás: adatbázis + konfiguráció + felhasználói adatok" - - Has DB, no HDD: "Adatbázis és konfiguráció visszaállítása" - - No DB, no HDD: "Csak konfiguráció visszaállítása" + - Has HDD: "Teljes visszaállitas: adatbazis + konfiguracio + felhasznaloi adatok" + - Has DB, no HDD: "Adatbazis es konfiguracio visszaallitasa" + - No DB, no HDD: "Csak konfiguracio visszaallitasa" - **Execution flow:** stop app → `restic restore --target / --include ...` → restart app - Running flag prevents concurrent backup/restore operations -- Snapshot ID validated (8–64 lowercase hex) +- Snapshot ID validated (8-64 lowercase hex) **Note:** Restore currently uses Tier 1 (primary restic repo) only. Restoring from Tier 2 (cross-drive) is a future enhancement. @@ -274,16 +290,18 @@ Unified per-app status table with expandable rows showing **per-tier** backup st | Dot color | Meaning | |-----------|---------| -| Green | Fully covered — cross-drive configured and last run OK | -| Yellow | Warning — no second copy, or last backup failed, or disk space issue | -| Red | Cross-drive destination blocked or inaccessible | -| Gray (auto) | No user data — only config/DB backup (automatic) | +| Green | 2+ tiers configured with successful backups + destination healthy | +| Yellow | Only 1 tier, or Tier 2 failing, or Tier 2 configured but never run | +| Red | Tier 2 destination blocked or inaccessible | -**Per-app backup tiers:** -- **1. mentés** (Tier 1, always present) — Auto badge + "helyi" + last run + contents (e.g., "DB + Konfig + Adatok") -- **2. mentés** (Tier 2, only for apps with HDD data) — one of: - - Configured: method (rsync/restic) + destination + schedule + last run + status + contents + browsable indicator (📁 for rsync) + action buttons - - Not configured: "✓ 1. mentés auto" + "⚠ Nincs 2. másolat" + settings link +Every app starts as yellow (1 tier only). Green requires Tier 2 configured with successful backup. + +**Per-app backup tiers (3 rows per app):** +- **1. mentes** (Tier 1, always present) — Auto badge + "helyi" + last run + contents (e.g., "DB + Konfig + Adatok") +- **2. mentes** (Tier 2, configurable for ALL apps) — one of: + - Configured: method (rsync/restic) + destination + schedule + last run + status + contents + browsable indicator (folder icon for rsync) + action buttons + - Not configured: "1. mentes auto" + "Nincs 2. masolat" + settings link +- **3. mentes** (Tier 3, placeholder) — grayed out "Hamarosan" + "tavoli (offsite)" + future note **Backup contents per app** (shown per tier): - Apps with DB + HDD: "DB + Konfig + Adatok" @@ -291,6 +309,9 @@ Unified per-app status table with expandable rows showing **per-tier** backup st - Apps with HDD, no DB: "Konfig + Adatok" - Apps with neither: "Konfig" +**Deploy page** shows cross-drive (Tier 2) configuration form for **all deployed apps**, +not just those with HDD data. Non-HDD apps can configure destination, method, and schedule. + **Other sections:** - Schedule overview with next run times for DB dump, restic, prune - Snapshot history table (last 20 snapshots with ID, time, files new/changed, data added)