Complete Cross-Drive Backup + Per-Tier UI (v0.12.8)
This commit is contained in:
@@ -1,54 +1,647 @@
|
||||
# TASK.md — Post-deploy fixes (v0.12.7a)
|
||||
# TASK.md — Complete Cross-Drive Backup + Per-Tier UI (v0.12.8)
|
||||
|
||||
## Prompt (copy-paste this into Claude Code)
|
||||
|
||||
```
|
||||
Read TASK.md for context. The code changes are already applied. Build, deploy, and verify.
|
||||
Read TASK.md for the full plan. Apply all code changes described, then build and deploy.
|
||||
After all fixes are done:
|
||||
1. Run `go build ./...` and `go vet ./...` from the controller/ directory — fix any errors
|
||||
2. Update CHANGELOG.md with a new entry at the top (session 45, v0.12.8)
|
||||
3. Commit, build, and deploy following the workflow in CLAUDE.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
## Context and Goals
|
||||
|
||||
v0.12.7 was deployed with two issues discovered during testing:
|
||||
The cross-drive backup (Tier 2) currently only copies HDD user data (mounts). It does NOT
|
||||
include DB dumps or app config. If the primary drive fails, the customer loses their database
|
||||
and config — even though photos are safe on the second drive. **This is not a viable backup.**
|
||||
|
||||
### Fix A: Restore showed "Nincs elérhető mentés" for Immich
|
||||
Each tier must be a **complete, self-sufficient backup**:
|
||||
|
||||
**Root cause:** `filterSnapshotsByPaths` in `router.go` filtered snapshots by HDD mount
|
||||
paths. Older snapshots (taken before v0.12.7 made HDD backup mandatory) don't contain
|
||||
HDD paths, so they got filtered out — leaving zero snapshots for Immich.
|
||||
| Tier | Contents | Location | Can fully restore? |
|
||||
|------|----------|----------|--------------------|
|
||||
| 1. Nightly restic | DB + Config + User data | Same drive | Yes (not against drive failure) |
|
||||
| 2. Cross-drive | DB + Config + User data | Different drive | **Yes (after this fix)** |
|
||||
| 3. Remote (future) | Everything | Offsite | Yes |
|
||||
|
||||
**Fix applied:** Removed the path filtering entirely (`router.go` line 460-469). All
|
||||
snapshots contain config + DB dumps, so they're useful for any app. The `RestoreApp`
|
||||
function extracts whatever paths are available from the snapshot.
|
||||
|
||||
Note: `filterSnapshotsByPaths` and `pathCovers` functions are now unused but kept for
|
||||
potential future use. They won't cause compile errors (Go only errors on unused
|
||||
variables/imports, not functions).
|
||||
|
||||
### Fix B: "Nincs beállítva" warning was misleading
|
||||
|
||||
**Root cause:** With mandatory nightly restic backup, user data IS backed up to the local
|
||||
drive. The missing piece is only the second copy (cross-drive). The old messages implied
|
||||
no backup at all.
|
||||
|
||||
**Changes applied:**
|
||||
1. `handlers.go` line 601-604: Status changed from `"red"` → `"yellow"`, StatusText
|
||||
from `"Felhasználói adatokról nincs mentés"` → `"Nincs második másolat (csak helyi mentés)"`
|
||||
2. `backups.html` line 315: Added `✓ Helyi mentés auto` badge before `⚠ Nincs 2. másolat`
|
||||
3. `style.css`: Added `.layer-auto-ok` class (green text for the auto badge)
|
||||
This task also restructures the UI from per-layer (DB/Config/UserData rows) to per-tier
|
||||
(1st backup / 2nd backup rows) — matching the customer mental model.
|
||||
|
||||
---
|
||||
|
||||
## Steps
|
||||
## Fix 1: Include DB dumps + config in cross-drive backup
|
||||
|
||||
1. Run `go build ./...` and `go vet ./...` from the controller/ directory — fix any errors
|
||||
2. Update CHANGELOG.md: add v0.12.7a entry (session 44):
|
||||
- Fix: restore dropdown now shows snapshots for all apps (removed HDD path filtering)
|
||||
- Fix: user data warning clarified — shows "Helyi mentés auto / Nincs 2. másolat"
|
||||
instead of the misleading "Nincs beállítva"
|
||||
3. Commit, build, and deploy following the workflow in CLAUDE.md
|
||||
4. Verify:
|
||||
- Immich now shows restore snapshots (should list all available snapshots)
|
||||
- Paperless-ngx and RomM show yellow dot (not red) with "✓ Helyi mentés auto · ⚠ Nincs 2. másolat"
|
||||
- Mealie/Gokapi (no HDD) still show correctly with config+DB restore
|
||||
### 1a: Add `dbDumpDir` field to CrossDriveRunner
|
||||
|
||||
**File:** `internal/backup/crossdrive.go`
|
||||
|
||||
Add a `dbDumpDir` field to the struct:
|
||||
|
||||
```go
|
||||
type CrossDriveRunner struct {
|
||||
sett *settings.Settings
|
||||
stackProvider StackDataProvider
|
||||
dbDumper DBDumper
|
||||
dbDumpDir string // path to DB dump directory (e.g., /srv/backups/db-dumps)
|
||||
logger *log.Logger
|
||||
mu sync.Mutex
|
||||
running map[string]bool
|
||||
}
|
||||
```
|
||||
|
||||
Add setter after `SetDBDumper`:
|
||||
|
||||
```go
|
||||
// SetDBDumpDir sets the path to the DB dump directory for cross-drive backups.
|
||||
func (r *CrossDriveRunner) SetDBDumpDir(dir string) {
|
||||
r.dbDumpDir = dir
|
||||
}
|
||||
```
|
||||
|
||||
### 1b: Add helper to copy DB dump files for a stack
|
||||
|
||||
**File:** `internal/backup/crossdrive.go`, add helper function:
|
||||
|
||||
```go
|
||||
// copyStackDBDumps copies DB dump files for the given stack to destDir.
|
||||
// DB dump files are named <stackName>_<dbtype>.sql (e.g., immich_postgres.sql).
|
||||
// Small files — uses plain file copy, not rsync.
|
||||
func (r *CrossDriveRunner) copyStackDBDumps(stackName, destDir string) error {
|
||||
if r.dbDumpDir == "" {
|
||||
return nil
|
||||
}
|
||||
entries, err := os.ReadDir(r.dbDumpDir)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("reading DB dump dir: %w", err)
|
||||
}
|
||||
prefix := stackName + "_"
|
||||
copied := 0
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasPrefix(e.Name(), prefix) {
|
||||
continue
|
||||
}
|
||||
src := filepath.Join(r.dbDumpDir, e.Name())
|
||||
dst := filepath.Join(destDir, e.Name())
|
||||
data, err := os.ReadFile(src)
|
||||
if err != nil {
|
||||
return fmt.Errorf("reading %s: %w", e.Name(), err)
|
||||
}
|
||||
if err := os.WriteFile(dst, data, 0644); err != nil {
|
||||
return fmt.Errorf("writing %s: %w", e.Name(), err)
|
||||
}
|
||||
copied++
|
||||
}
|
||||
if copied > 0 {
|
||||
r.logger.Printf("[DEBUG] Copied %d DB dump file(s) to %s", copied, destDir)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### 1c: Update `runRsyncBackup` to include DB + config
|
||||
|
||||
**File:** `internal/backup/crossdrive.go`, function `runRsyncBackup`
|
||||
|
||||
After the existing HDD mount rsync loop (after line 260 `return nil` of the for loop),
|
||||
add these blocks BEFORE the final `return nil`:
|
||||
|
||||
```go
|
||||
// --- Copy DB dumps for this stack ---
|
||||
dbDestDir := filepath.Join(destDir, "_db")
|
||||
if err := os.MkdirAll(dbDestDir, 0755); err != nil {
|
||||
return fmt.Errorf("creating DB dump dest dir: %w", err)
|
||||
}
|
||||
if err := r.copyStackDBDumps(stackName, dbDestDir); err != nil {
|
||||
r.logger.Printf("[WARN] Cross-drive DB dump copy failed for %s: %v", stackName, err)
|
||||
// Non-fatal: user data is the primary concern
|
||||
}
|
||||
|
||||
// --- Rsync app config (compose dir) ---
|
||||
if composePath, ok := r.stackProvider.GetStackComposePath(stackName); ok {
|
||||
configSrcDir := filepath.Dir(composePath)
|
||||
configDestDir := filepath.Join(destDir, "_config")
|
||||
if err := os.MkdirAll(configDestDir, 0755); err != nil {
|
||||
return fmt.Errorf("creating config dest dir: %w", err)
|
||||
}
|
||||
src := strings.TrimRight(configSrcDir, "/") + "/"
|
||||
dst := strings.TrimRight(configDestDir, "/") + "/"
|
||||
cmd := exec.CommandContext(ctx, "rsync", "-a", "--delete", src, dst)
|
||||
r.logger.Printf("[DEBUG] rsync config: %s → %s", src, dst)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
r.logger.Printf("[WARN] Cross-drive config rsync failed for %s: %v (%s)", stackName, err, strings.TrimSpace(string(out)))
|
||||
// Non-fatal
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Resulting rsync destination layout:**
|
||||
```
|
||||
backups/rsync/<app>/
|
||||
_db/ ← stackName_postgres.sql, stackName_mariadb.sql
|
||||
_config/ ← compose.yml, app.yaml, .felhom.yml
|
||||
<user data> ← existing HDD mount contents (unchanged)
|
||||
```
|
||||
|
||||
The `_` prefix prevents collision with user data directories (HDD mounts never start with `_`).
|
||||
|
||||
### 1d: Update `runResticBackup` to include DB + config
|
||||
|
||||
**File:** `internal/backup/crossdrive.go`, function `runResticBackup`
|
||||
|
||||
Currently the restic backup only includes `mounts`. Change to also include the config dir
|
||||
and DB dump dir.
|
||||
|
||||
**Current code (around line 294–301):**
|
||||
```go
|
||||
args := []string{
|
||||
"backup", "--repo", repoPath,
|
||||
"--password-file", pwPath,
|
||||
"--tag", stackName,
|
||||
"--tag", "cross-drive",
|
||||
}
|
||||
args = append(args, mounts...)
|
||||
```
|
||||
|
||||
**Replace with:**
|
||||
```go
|
||||
args := []string{
|
||||
"backup", "--repo", repoPath,
|
||||
"--password-file", pwPath,
|
||||
"--tag", stackName,
|
||||
"--tag", "cross-drive",
|
||||
}
|
||||
// Include user data (HDD mounts)
|
||||
args = append(args, mounts...)
|
||||
// Include app config dir (compose + app.yaml + .felhom.yml)
|
||||
if composePath, ok := r.stackProvider.GetStackComposePath(stackName); ok {
|
||||
args = append(args, filepath.Dir(composePath))
|
||||
}
|
||||
// Include DB dump dir (all stacks' dumps — restic deduplicates)
|
||||
if r.dbDumpDir != "" {
|
||||
if _, err := os.Stat(r.dbDumpDir); err == nil {
|
||||
args = append(args, r.dbDumpDir)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note: for restic, including the full DB dump dir is fine — restic deduplicates and it's
|
||||
shared across apps. The `--tag <stackName>` identifies which app this backup belongs to.
|
||||
|
||||
### 1e: Wire up in main.go
|
||||
|
||||
**File:** `cmd/controller/main.go`, after the existing `crossDriveRunner.SetDBDumper(backupMgr)` line:
|
||||
|
||||
```go
|
||||
crossDriveRunner.SetDBDumpDir(cfg.Paths.DBDumpDir)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Fix 2: Restructure UI from per-layer to per-tier
|
||||
|
||||
The current UI shows 3 rows per app (DB / Konfiguráció / Felhasználói adatok). This
|
||||
maps to implementation details, not the customer mental model. Change to 2 rows per app:
|
||||
- **1. mentés** — nightly restic (mandatory, same drive)
|
||||
- **2. mentés** — cross-drive (opt-in, different drive)
|
||||
|
||||
### 2a: Restructure `AppBackupRow` struct
|
||||
|
||||
**File:** `internal/web/handlers.go`
|
||||
|
||||
**Replace the current `AppBackupRow` struct (lines 500–533) with:**
|
||||
|
||||
```go
|
||||
// AppBackupRow holds per-tier backup information for one app on the backup page.
|
||||
type AppBackupRow struct {
|
||||
StackName string
|
||||
DisplayName string
|
||||
Status string // "green", "yellow", "red", "auto"
|
||||
StatusText string // short Hungarian tooltip
|
||||
|
||||
// App characteristics
|
||||
HasHDDData bool
|
||||
HasDB bool
|
||||
StorageLabel string
|
||||
HDDSizeHuman string
|
||||
|
||||
// What this app's backup contains (for display)
|
||||
// e.g., "DB + Konfiguráció + Adatok", "DB + Konfiguráció", "Konfiguráció"
|
||||
BackupContents string
|
||||
|
||||
// Tier 1: Nightly backup (always exists)
|
||||
Tier1LastRun string // formatted time of last restic snapshot
|
||||
Tier1LastStatus string // "ok", "error", ""
|
||||
Tier1DBStatus string // "ok", "error", "" — separate DB dump status for warning
|
||||
|
||||
// Tier 2: Cross-drive backup (only for apps with HDD data)
|
||||
Tier2Configured bool
|
||||
Tier2Method string // "rsync", "restic"
|
||||
Tier2MethodLabel string // "rsync", "restic"
|
||||
Tier2Dest string // destination label
|
||||
Tier2Schedule string // "Naponta", "Hetente"
|
||||
Tier2LastRun string
|
||||
Tier2LastStatus string // "ok", "error", "running", ""
|
||||
Tier2LastError string
|
||||
Tier2StatusBadge string // "Sikeres", "Hiba", "Fut...", "—"
|
||||
Tier2SizeHuman string
|
||||
Tier2Browsable bool // true for rsync (plain files), false for restic
|
||||
|
||||
// Warnings accumulated for this app
|
||||
Warnings []string
|
||||
}
|
||||
```
|
||||
|
||||
### 2b: Update `buildAppBackupRows`
|
||||
|
||||
**File:** `internal/web/handlers.go`
|
||||
|
||||
**Replace the entire `buildAppBackupRows` function** with:
|
||||
|
||||
```go
|
||||
// buildAppBackupRows constructs one AppBackupRow per deployed app for the backup page.
|
||||
func (s *Server) buildAppBackupRows(
|
||||
status *backup.FullBackupStatus,
|
||||
crossConfigs map[string]*settings.CrossDriveBackup,
|
||||
destLabels map[string]string,
|
||||
) []AppBackupRow {
|
||||
loc := getTimezone()
|
||||
|
||||
// Build DB stack lookup
|
||||
dbStacks := make(map[string]bool)
|
||||
for _, db := range status.DiscoveredDBs {
|
||||
dbStacks[db.StackName] = true
|
||||
}
|
||||
for _, f := range status.DumpFiles {
|
||||
dbStacks[f.StackName] = true
|
||||
}
|
||||
|
||||
// Tier 1 timestamps (shared across all apps — single nightly job)
|
||||
tier1LastRun := ""
|
||||
tier1LastStatus := ""
|
||||
if status.LastBackup != nil {
|
||||
tier1LastRun = status.LastBackup.LastRun.In(loc).Format("01-02 15:04")
|
||||
if status.LastBackup.Success {
|
||||
tier1LastStatus = "ok"
|
||||
} else {
|
||||
tier1LastStatus = "error"
|
||||
}
|
||||
}
|
||||
tier1DBStatus := ""
|
||||
if status.LastDBDump != nil {
|
||||
if status.LastDBDump.Success {
|
||||
tier1DBStatus = "ok"
|
||||
} else {
|
||||
tier1DBStatus = "error"
|
||||
}
|
||||
}
|
||||
|
||||
var rows []AppBackupRow
|
||||
for _, app := range status.AppDataInfo {
|
||||
hasDB := dbStacks[app.StackName] || app.HasDBDump
|
||||
|
||||
// Build backup contents label
|
||||
var parts []string
|
||||
if hasDB {
|
||||
parts = append(parts, "DB")
|
||||
}
|
||||
parts = append(parts, "Konfig")
|
||||
if app.HasHDDData {
|
||||
parts = append(parts, "Adatok")
|
||||
}
|
||||
contents := strings.Join(parts, " + ")
|
||||
|
||||
row := AppBackupRow{
|
||||
StackName: app.StackName,
|
||||
DisplayName: app.DisplayName,
|
||||
HasHDDData: app.HasHDDData,
|
||||
HasDB: hasDB,
|
||||
StorageLabel: app.StorageLabel,
|
||||
HDDSizeHuman: app.HDDSizeHuman,
|
||||
BackupContents: contents,
|
||||
|
||||
Tier1LastRun: tier1LastRun,
|
||||
Tier1LastStatus: tier1LastStatus,
|
||||
Tier1DBStatus: tier1DBStatus,
|
||||
}
|
||||
|
||||
// Default status = auto (no user data, just config)
|
||||
row.Status = "auto"
|
||||
row.StatusText = "Automatikus mentés"
|
||||
|
||||
if app.HasHDDData {
|
||||
cfg, hasCfg := crossConfigs[app.StackName]
|
||||
|
||||
if !hasCfg || cfg == nil || !cfg.Enabled {
|
||||
// HDD data backed up via nightly restic (mandatory), but no second copy
|
||||
row.Tier2Configured = false
|
||||
row.Status = "yellow"
|
||||
row.StatusText = "Nincs második másolat (csak helyi mentés)"
|
||||
} else {
|
||||
row.Tier2Configured = true
|
||||
row.Tier2Method = cfg.Method
|
||||
row.Tier2MethodLabel = cfg.Method // "rsync" or "restic"
|
||||
row.Tier2Browsable = cfg.Method == "rsync"
|
||||
row.Tier2Dest = destLabels[cfg.DestinationPath]
|
||||
if row.Tier2Dest == "" {
|
||||
row.Tier2Dest = cfg.DestinationPath
|
||||
}
|
||||
switch cfg.Schedule {
|
||||
case "daily":
|
||||
row.Tier2Schedule = "Naponta"
|
||||
case "weekly":
|
||||
row.Tier2Schedule = "Hetente"
|
||||
default:
|
||||
row.Tier2Schedule = cfg.Schedule
|
||||
}
|
||||
if cfg.LastRun != "" {
|
||||
if t, err := time.Parse(time.RFC3339, cfg.LastRun); err == nil {
|
||||
row.Tier2LastRun = t.In(loc).Format("01-02 15:04")
|
||||
}
|
||||
}
|
||||
row.Tier2LastStatus = cfg.LastStatus
|
||||
row.Tier2LastError = cfg.LastError
|
||||
row.Tier2SizeHuman = cfg.LastSizeHuman
|
||||
switch cfg.LastStatus {
|
||||
case "ok":
|
||||
row.Tier2StatusBadge = "Sikeres"
|
||||
case "error":
|
||||
row.Tier2StatusBadge = "Hiba"
|
||||
row.Status = "yellow"
|
||||
row.StatusText = "Utolsó mentés sikertelen"
|
||||
case "running":
|
||||
row.Tier2StatusBadge = "Fut..."
|
||||
default:
|
||||
row.Tier2StatusBadge = "—"
|
||||
}
|
||||
|
||||
// Destination health check
|
||||
if cfg.Enabled && cfg.DestinationPath != "" {
|
||||
if err := s.crossDrive.ValidateDestination(cfg.DestinationPath); err != nil {
|
||||
if strings.Contains(err.Error(), "does not exist") || strings.Contains(err.Error(), "not writable") {
|
||||
row.Status = "red"
|
||||
row.StatusText = "Mentési cél nem elérhető"
|
||||
} else {
|
||||
row.Status = "yellow"
|
||||
row.StatusText = "Figyelmeztetés"
|
||||
}
|
||||
row.Warnings = append(row.Warnings, err.Error())
|
||||
} else if row.Status != "yellow" {
|
||||
row.Status = "green"
|
||||
row.StatusText = "Mentés rendben"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// DB dump failure warning (affects Tier 1 quality)
|
||||
if hasDB && tier1DBStatus == "error" {
|
||||
if row.Status != "red" {
|
||||
row.Status = "yellow"
|
||||
row.StatusText = "Adatbázis mentés sikertelen"
|
||||
}
|
||||
}
|
||||
|
||||
rows = append(rows, row)
|
||||
}
|
||||
return rows
|
||||
}
|
||||
```
|
||||
|
||||
**Note:** This requires `strings` import in handlers.go — check it's present.
|
||||
|
||||
### 2c: Update template to per-tier display
|
||||
|
||||
**File:** `internal/web/templates/backups.html`
|
||||
|
||||
Replace the backup-layers section inside the `app-backup-row-detail` div (lines 265–322).
|
||||
The current code has three `backup-layer-row` divs (DB, Konfiguráció, Felhasználói adatok).
|
||||
|
||||
**Replace with two tier rows:**
|
||||
|
||||
```html
|
||||
<div class="app-backup-row-detail" style="display:none">
|
||||
<div class="backup-layers">
|
||||
<!-- Tier 1: Nightly backup (mandatory, same drive) -->
|
||||
<div class="backup-layer-row">
|
||||
<span class="tier-label">1. mentés</span>
|
||||
<span class="layer-badge">Auto</span>
|
||||
<span class="tier-location">helyi</span>
|
||||
{{if .Tier1LastRun}}
|
||||
<span class="layer-last">Utolsó: {{.Tier1LastRun}}
|
||||
{{if eq .Tier1LastStatus "ok"}}<span class="text-ok">✓</span>
|
||||
{{else if eq .Tier1LastStatus "error"}}<span class="text-error">✗</span>{{end}}
|
||||
</span>
|
||||
{{end}}
|
||||
<span class="tier-contents">{{.BackupContents}}</span>
|
||||
{{if and .HasDB (eq .Tier1DBStatus "error")}}
|
||||
<span class="text-error" style="font-size:.8rem">⚠ DB dump hiba</span>
|
||||
{{end}}
|
||||
</div>
|
||||
<!-- Tier 2: Cross-drive backup (opt-in, different device) -->
|
||||
{{if .HasHDDData}}
|
||||
<div class="backup-layer-row">
|
||||
<span class="tier-label">2. mentés</span>
|
||||
{{if .Tier2Configured}}
|
||||
<span class="layer-method">{{.Tier2MethodLabel}}</span>
|
||||
<span class="layer-dest">→ {{.Tier2Dest}}</span>
|
||||
<span class="layer-schedule">{{.Tier2Schedule}}</span>
|
||||
{{if .Tier2LastRun}}
|
||||
<span class="layer-last">Utolsó: {{.Tier2LastRun}}
|
||||
<span class="{{if eq .Tier2LastStatus "ok"}}text-ok{{else if eq .Tier2LastStatus "error"}}text-error{{else if eq .Tier2LastStatus "running"}}text-muted{{end}}">
|
||||
{{.Tier2StatusBadge}}
|
||||
</span>
|
||||
</span>
|
||||
{{end}}
|
||||
{{if .Tier2SizeHuman}}<span class="tier-size">{{.Tier2SizeHuman}}</span>{{end}}
|
||||
<span class="tier-contents">{{.BackupContents}}</span>
|
||||
{{if .Tier2Browsable}}<span class="tier-browsable" title="A mentés böngészhető fájlrendszerben">📁</span>{{end}}
|
||||
<div class="layer-actions">
|
||||
<a href="/stacks/{{.StackName}}/deploy" class="btn btn-xs btn-outline">Beállítás</a>
|
||||
<button class="btn btn-xs btn-outline"
|
||||
onclick="triggerCrossDriveBackup('{{.StackName}}', this)">
|
||||
Futtatás most</button>
|
||||
</div>
|
||||
{{else}}
|
||||
<span class="layer-auto-ok">✓ 1. mentés auto</span>
|
||||
<span class="layer-unconfigured">⚠ Nincs 2. másolat</span>
|
||||
<a href="/stacks/{{.StackName}}/deploy" class="btn btn-xs">Beállítás →</a>
|
||||
{{end}}
|
||||
</div>
|
||||
{{end}}
|
||||
</div>
|
||||
{{if .Warnings}}
|
||||
<div class="layer-warnings">
|
||||
{{range .Warnings}}
|
||||
<div class="backup-layer-warning">{{.}}</div>
|
||||
{{end}}
|
||||
</div>
|
||||
{{end}}
|
||||
</div>
|
||||
```
|
||||
|
||||
### 2d: Add CSS for new tier elements
|
||||
|
||||
**File:** `internal/web/templates/style.css`
|
||||
|
||||
Add near the existing `.layer-*` styles:
|
||||
|
||||
```css
|
||||
.tier-label {
|
||||
font-weight: 600;
|
||||
min-width: 5rem;
|
||||
color: var(--text);
|
||||
}
|
||||
.tier-location {
|
||||
color: var(--text-muted);
|
||||
font-size: .85rem;
|
||||
}
|
||||
.tier-contents {
|
||||
color: var(--text-muted);
|
||||
font-size: .8rem;
|
||||
font-style: italic;
|
||||
margin-left: .25rem;
|
||||
}
|
||||
.tier-size {
|
||||
color: var(--text-muted);
|
||||
font-size: .8rem;
|
||||
margin-left: .25rem;
|
||||
}
|
||||
.tier-browsable {
|
||||
font-size: .75rem;
|
||||
margin-left: .15rem;
|
||||
cursor: help;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Fix 3: Clean up unused code
|
||||
|
||||
### 3a: Remove unused `filterSnapshotsByPaths` and `pathCovers`
|
||||
|
||||
**File:** `internal/api/router.go`
|
||||
|
||||
Delete the `filterSnapshotsByPaths` function and the `pathCovers` helper — they were
|
||||
left after v0.12.7a removed the call site. No other code references them.
|
||||
|
||||
### 3b: Remove `VolumeLastRun` / `VolumeLastStatus` / `DBLastRun` / `DBLastStatus` fields
|
||||
|
||||
These fields no longer exist in the new `AppBackupRow` struct (replaced by Tier1/Tier2
|
||||
fields). Verify no template references to the old field names remain.
|
||||
|
||||
---
|
||||
|
||||
## Summary of all changes
|
||||
|
||||
| Fix | What | File(s) |
|
||||
|-----|------|---------|
|
||||
| 1a | `dbDumpDir` field + setter on CrossDriveRunner | `crossdrive.go` |
|
||||
| 1b | `copyStackDBDumps` helper for rsync | `crossdrive.go` |
|
||||
| 1c | rsync includes DB dumps + config | `crossdrive.go` |
|
||||
| 1d | restic includes config dir + DB dump dir | `crossdrive.go` |
|
||||
| 1e | Wire `SetDBDumpDir` in main.go | `main.go` |
|
||||
| 2a | `AppBackupRow` restructured to per-tier | `handlers.go` |
|
||||
| 2b | `buildAppBackupRows` rewritten for per-tier | `handlers.go` |
|
||||
| 2c | Template: 2 tier rows instead of 3 layer rows | `backups.html` |
|
||||
| 2d | CSS for tier elements | `style.css` |
|
||||
| 3a | Remove unused `filterSnapshotsByPaths` + `pathCovers` | `router.go` |
|
||||
| 3b | Verify no old field references remain | `backups.html` |
|
||||
|
||||
## Files to modify (6)
|
||||
|
||||
1. `internal/backup/crossdrive.go` — Fix 1a + 1b + 1c + 1d
|
||||
2. `cmd/controller/main.go` — Fix 1e
|
||||
3. `internal/web/handlers.go` — Fix 2a + 2b
|
||||
4. `internal/web/templates/backups.html` — Fix 2c
|
||||
5. `internal/web/templates/style.css` — Fix 2d
|
||||
6. `internal/api/router.go` — Fix 3a
|
||||
|
||||
## Architecture after fix
|
||||
|
||||
```
|
||||
Per-app backup tiers:
|
||||
┌──────────────────────────────────────────────────────┐
|
||||
│ TIER 1 — Nightly restic (MANDATORY, same drive) │
|
||||
│ │
|
||||
│ Contains: │
|
||||
│ - DB dumps (pg_dump / mariadb-dump) │
|
||||
│ - Config (compose.yml, app.yaml, .felhom.yml) │
|
||||
│ - User data (ALL HDD bind mounts — mandatory) │
|
||||
│ │
|
||||
│ Protects against: accidental deletion, corruption │
|
||||
│ Does NOT protect against: drive failure │
|
||||
│ │
|
||||
├──────────────────────────────────────────────────────┤
|
||||
│ TIER 2 — Cross-drive backup (OPT-IN, second device) │
|
||||
│ │
|
||||
│ Contains (COMPLETE — same as Tier 1): │
|
||||
│ - DB dumps (copied to _db/ subfolder) │
|
||||
│ - Config (rsynced to _config/ subfolder) │
|
||||
│ - User data (rsync or restic to destination) │
|
||||
│ │
|
||||
│ rsync layout: │
|
||||
│ backups/rsync/<app>/ │
|
||||
│ _db/ ← DB dump files (browsable) │
|
||||
│ _config/ ← compose + app.yaml (browsable) │
|
||||
│ <data> ← user data (browsable) │
|
||||
│ │
|
||||
│ restic layout: │
|
||||
│ backups/restic/ ← encrypted repo (not browsable) │
|
||||
│ │
|
||||
│ Protects against: drive failure, drive theft │
|
||||
│ │
|
||||
├──────────────────────────────────────────────────────┤
|
||||
│ TIER 3 — Remote backup (FUTURE) │
|
||||
│ │
|
||||
│ Complete offsite copy. Not implemented yet. │
|
||||
└──────────────────────────────────────────────────────┘
|
||||
|
||||
UI per-app display:
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ ● Immich Külső tárhely (hdd_1) 63.9 MB │
|
||||
│ 1. mentés Auto helyi 02-18 03:00 ✓ DB+Konfig+Adat│
|
||||
│ 2. mentés rsync → hdd_1 Naponta 02-18 10:48 ✓ │
|
||||
│ DB+Konfig+Adatok 📁 [Beáll][Futtás]│
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ ● Mealie Auto │
|
||||
│ 1. mentés Auto helyi 02-18 03:00 ✓ Konfig │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ ● Paperless-ngx hdd_placeholder 76 B │
|
||||
│ 1. mentés Auto helyi 02-18 03:00 ✓ Konfig+Adatok │
|
||||
│ 2. mentés ✓ 1. mentés auto ⚠ Nincs 2. másolat │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Post-fix checklist
|
||||
|
||||
- [ ] `go build ./...` passes
|
||||
- [ ] `go vet ./...` passes
|
||||
- [ ] Verify no references to old fields: `DBLastRun`, `DBLastStatus`, `VolumeLastRun`,
|
||||
`VolumeLastStatus`, `HasUserData`, `UserDataConfigured`, `UserDataMethod`,
|
||||
`UserDataDest`, `UserDataSchedule`, `UserDataLastRun`, `UserDataLastStatus`,
|
||||
`UserDataLastError`, `UserDataStatusBadge`
|
||||
- [ ] Verify `filterSnapshotsByPaths` and `pathCovers` deleted from router.go
|
||||
- [ ] Update `CHANGELOG.md` — session 45, version **v0.12.8**:
|
||||
- Cross-drive backup now includes DB dumps + app config (complete backup)
|
||||
- rsync layout: `_db/` and `_config/` subdirs alongside user data
|
||||
- restic cross-drive includes config dir + DB dump dir
|
||||
- UI: restructured from per-layer to per-tier display (1. mentés / 2. mentés)
|
||||
- UI: shows backup contents per app (DB + Konfig + Adatok)
|
||||
- UI: rsync backups show browsable indicator (📁)
|
||||
- Cleanup: removed unused filterSnapshotsByPaths code
|
||||
- [ ] Commit, build on 192.168.0.180, deploy on 192.168.0.162
|
||||
- [ ] Verify with `docker ps` and `docker logs`
|
||||
- [ ] After deploy, verify:
|
||||
- Immich card shows "1. mentés" and "2. mentés" rows (not DB/Konfig/User data rows)
|
||||
- Immich Tier 2 shows "DB + Konfig + Adatok" contents label
|
||||
- Run manual cross-drive backup for Immich
|
||||
- Check destination: `ls backups/rsync/immich/` should show `_db/`, `_config/`, and user data
|
||||
- Verify `_db/` contains `immich_postgres.sql`
|
||||
- Verify `_config/` contains `docker-compose.yml`, `app.yaml`
|
||||
- Mealie shows only "1. mentés" row (no Tier 2 — no HDD data)
|
||||
- Paperless-ngx shows yellow dot with "⚠ Nincs 2. másolat"
|
||||
|
||||
+49
-32
@@ -163,19 +163,23 @@ The `/apps/{slug}` page renders hero section, screenshots, setup guide, and opti
|
||||
|
||||
### 2. Backup System
|
||||
|
||||
The backup system implements a **3-2-1 backup architecture**:
|
||||
The backup system implements a **3-2-1 backup architecture**. Each tier is a **complete,
|
||||
self-sufficient backup** — any single tier can fully restore an app.
|
||||
|
||||
| Rule | What | Where | Status |
|
||||
|------|------|-------|--------|
|
||||
| **1. Nightly backup** | DB dumps + config + ALL user data | Same drive as app | Mandatory, automatic |
|
||||
| **2. Cross-drive backup** | User data copy to secondary drive | Different physical device | Opt-in per app |
|
||||
| **3. Remote backup** | Offsite copy for disaster recovery | Cloud / remote server | Future |
|
||||
| Tier | Contents | Location | Can fully restore? |
|
||||
|------|----------|----------|--------------------|
|
||||
| **1. Nightly restic** | DB + Config + User data | Same drive as app | Yes (not against drive failure) |
|
||||
| **2. Cross-drive** | DB + Config + User data | Different physical device | Yes |
|
||||
| **3. Remote** | Everything | Cloud / remote server | Future |
|
||||
|
||||
**Key principle:** User data backup is **mandatory** — every app with HDD bind mounts
|
||||
is included in the nightly restic snapshot automatically. There is no per-app toggle.
|
||||
The `AppBackupPrefs.Enabled` field in settings.json is legacy and not read by any code.
|
||||
**Key principles:**
|
||||
- User data backup is **mandatory** — every app with HDD bind mounts is included
|
||||
automatically. There is no per-app toggle.
|
||||
- Each tier includes **everything** needed to restore: DB dumps, config, and user data.
|
||||
No tier depends on another tier's data.
|
||||
- The `AppBackupPrefs.Enabled` field in settings.json is legacy and not read by any code.
|
||||
|
||||
#### Rule 1: Nightly Backup (mandatory, same drive)
|
||||
#### Tier 1: Nightly Backup (mandatory, same drive)
|
||||
|
||||
The nightly backup has two phases that run sequentially:
|
||||
|
||||
@@ -199,18 +203,18 @@ The nightly backup has two phases that run sequentially:
|
||||
- Weekly prune on Sundays with configurable retention (keep-daily, keep-weekly, keep-monthly)
|
||||
- Weekly integrity check (`restic check`) on Sunday 04:00
|
||||
|
||||
**What this protects against:** accidental deletion, data corruption, point-in-time rollback.
|
||||
**Protects against:** accidental deletion, data corruption, point-in-time rollback.
|
||||
Does NOT protect against drive failure (backup is on the same physical drive).
|
||||
|
||||
#### Rule 2: Cross-Drive Backup (opt-in, different device) (`internal/backup/crossdrive.go`)
|
||||
#### Tier 2: Cross-Drive Backup (opt-in, different device) (`internal/backup/crossdrive.go`)
|
||||
|
||||
Copies user data to a **different physical drive**, providing the second copy for 3-2-1.
|
||||
**Complete backup** to a different physical drive — DB dumps + config + user data.
|
||||
|
||||
- **Two methods:**
|
||||
- **rsync** — Simple mirror with `--delete` (fast, no versioning)
|
||||
- **restic** — Versioned, deduplicated, encrypted (shared repo across apps, auto-generated password)
|
||||
- **rsync** — Simple mirror with `--delete` (fast, no versioning, **browsable** on disk)
|
||||
- **restic** — Versioned, deduplicated, encrypted (shared repo across apps, not browsable)
|
||||
- Per-app configuration in settings.json: destination path, method, schedule (daily/weekly/manual)
|
||||
- **Pre-backup DB dump:** `DumpStackDB()` runs fresh pg_dump/mariadb-dump before each cross-drive backup to ensure DB state matches user data; non-fatal on failure (wired via `DBDumper` interface to avoid circular imports)
|
||||
- **Pre-backup DB dump:** `DumpStackDB()` runs fresh pg_dump/mariadb-dump before each cross-drive backup; non-fatal on failure (wired via `DBDumper` interface to avoid circular imports)
|
||||
- **Drive-type-aware validation** (`ValidateDestination`):
|
||||
|
||||
| Destination type | Space checks |
|
||||
@@ -218,20 +222,26 @@ Copies user data to a **different physical drive**, providing the second copy fo
|
||||
| External mount (different device than `/`) | Block if <100 MB free |
|
||||
| System drive (same device as `/`) | Require ≥10 GB free AND <90% used; logged warning |
|
||||
|
||||
- **Rsync destination layout:**
|
||||
- Single mount: `backups/rsync/<app>/` (flat, no extra nesting)
|
||||
- Multiple mounts: `backups/rsync/<app>/<leaf>/` per mount; duplicate leaf names get `_N` suffix
|
||||
- DB dump files excluded (`--exclude backups/*.sql.gz/sql/dump`) — already handled by pg_dump
|
||||
- **Rsync destination layout** (complete — can restore app independently):
|
||||
```
|
||||
backups/rsync/<app>/
|
||||
_db/ ← DB dump files (stackName_postgres.sql, etc.)
|
||||
_config/ ← compose.yml, app.yaml, .felhom.yml
|
||||
<user data> ← HDD mount contents (single mount: flat; multi-mount: leaf subfolders)
|
||||
```
|
||||
- DB dump files excluded from user data rsync (`--exclude backups/*.sql.gz/sql/dump`) to avoid duplicating app-internal dumps
|
||||
- `_` prefix directories prevent collision with user data
|
||||
- **Restic backup paths:** includes HDD mounts + config dir + DB dump dir (deduplication handles overlap)
|
||||
- Safety guards: destination ≠ source, path-overlap check, writable check
|
||||
- **Chained execution:** runs immediately after nightly restic — daily apps every night, weekly apps on Sundays
|
||||
- Per-app concurrency lock prevents overlapping runs
|
||||
- Status (last_run, duration, size, error) persisted to settings.json
|
||||
|
||||
**What this protects against:** primary drive failure, drive theft/damage.
|
||||
**Protects against:** primary drive failure, drive theft/damage.
|
||||
|
||||
#### Rule 3: Remote Backup (future)
|
||||
#### Tier 3: Remote Backup (future)
|
||||
|
||||
Offsite backup for disaster recovery. Not yet implemented.
|
||||
Complete offsite backup for disaster recovery. Not yet implemented.
|
||||
|
||||
#### Restore (`internal/backup/restore.go`)
|
||||
|
||||
@@ -244,7 +254,7 @@ All deployed apps appear in the restore dropdown — every app has restic snapsh
|
||||
| DB only, no HDD | ✓ | ✓ | n/a |
|
||||
| No DB, no HDD | ✓ | — | n/a |
|
||||
|
||||
- **Snapshot API** returns ALL snapshots unfiltered — older snapshots (pre-mandatory HDD backup) still allow config+DB restore; `RestoreApp` extracts whatever paths are available
|
||||
- **Snapshot API** returns ALL snapshots unfiltered — older snapshots still allow config+DB restore; `RestoreApp` extracts whatever paths are available
|
||||
- **Restore type info** shown per-app when selected in dropdown (Hungarian banners):
|
||||
- Has HDD: "Teljes visszaállítás: adatbázis + konfiguráció + felhasználói adatok"
|
||||
- Has DB, no HDD: "Adatbázis és konfiguráció visszaállítása"
|
||||
@@ -253,9 +263,12 @@ All deployed apps appear in the restore dropdown — every app has restic snapsh
|
||||
- Running flag prevents concurrent backup/restore operations
|
||||
- Snapshot ID validated (8–64 lowercase hex)
|
||||
|
||||
**Note:** Restore currently uses Tier 1 (primary restic repo) only. Restoring from Tier 2
|
||||
(cross-drive) is a future enhancement.
|
||||
|
||||
#### Backup Page UI (`internal/web/templates/backups.html`)
|
||||
|
||||
Unified per-app status table with expandable rows showing 3 backup layers per app:
|
||||
Unified per-app status table with expandable rows showing **per-tier** backup status:
|
||||
|
||||
**Status dot per app:**
|
||||
|
||||
@@ -266,13 +279,17 @@ Unified per-app status table with expandable rows showing 3 backup layers per ap
|
||||
| Red | Cross-drive destination blocked or inaccessible |
|
||||
| Gray (auto) | No user data — only config/DB backup (automatic) |
|
||||
|
||||
**Three backup layers per app row:**
|
||||
1. **Adatbázis mentés** — Auto badge + last run timestamp + status
|
||||
2. **Konfiguráció** — Auto badge + last restic snapshot timestamp + status
|
||||
3. **Felhasználói adatok** — one of:
|
||||
- Cross-drive configured: method + destination + schedule + last run + status + "Futtatás most" button
|
||||
- HDD data, no cross-drive: "✓ Helyi mentés auto" (green) + "⚠ Nincs 2. másolat" (yellow) + settings link
|
||||
- No HDD data: "— (nincs HDD adat)" (muted)
|
||||
**Per-app backup tiers:**
|
||||
- **1. mentés** (Tier 1, always present) — Auto badge + "helyi" + last run + contents (e.g., "DB + Konfig + Adatok")
|
||||
- **2. mentés** (Tier 2, only for apps with HDD data) — one of:
|
||||
- Configured: method (rsync/restic) + destination + schedule + last run + status + contents + browsable indicator (📁 for rsync) + action buttons
|
||||
- Not configured: "✓ 1. mentés auto" + "⚠ Nincs 2. másolat" + settings link
|
||||
|
||||
**Backup contents per app** (shown per tier):
|
||||
- Apps with DB + HDD: "DB + Konfig + Adatok"
|
||||
- Apps with DB only: "DB + Konfig"
|
||||
- Apps with HDD, no DB: "Konfig + Adatok"
|
||||
- Apps with neither: "Konfig"
|
||||
|
||||
**Other sections:**
|
||||
- Schedule overview with next run times for DB dump, restic, prune
|
||||
|
||||
Reference in New Issue
Block a user