Post-deploy fixes (v0.12.7a)

This commit is contained in:
2026-02-18 11:03:56 +01:00
parent 6c1762141a
commit e4433f07b4
6 changed files with 152 additions and 752 deletions
+49 -695
View File
@@ -1,700 +1,54 @@
# TASK.md — Backup Architecture Overhaul (v0.12.7)
# TASK.md — Post-deploy fixes (v0.12.7a)
## Prompt (copy-paste this into Claude Code)
```
Read TASK.md for the full plan. Apply all code changes described, then build and deploy.
After all fixes are done:
Read TASK.md for context. The code changes are already applied. Build, deploy, and verify.
```
---
## Context
v0.12.7 was deployed with two issues discovered during testing:
### Fix A: Restore showed "Nincs elérhető mentés" for Immich
**Root cause:** `filterSnapshotsByPaths` in `router.go` filtered snapshots by HDD mount
paths. Older snapshots (taken before v0.12.7 made HDD backup mandatory) don't contain
HDD paths, so they got filtered out — leaving zero snapshots for Immich.
**Fix applied:** Removed the path filtering entirely (`router.go` line 460-469). All
snapshots contain config + DB dumps, so they're useful for any app. The `RestoreApp`
function extracts whatever paths are available from the snapshot.
Note: `filterSnapshotsByPaths` and `pathCovers` functions are now unused but kept for
potential future use. They won't cause compile errors (Go only errors on unused
variables/imports, not functions).
### Fix B: "Nincs beállítva" warning was misleading
**Root cause:** With mandatory nightly restic backup, user data IS backed up to the local
drive. The missing piece is only the second copy (cross-drive). The old messages implied
no backup at all.
**Changes applied:**
1. `handlers.go` line 601-604: Status changed from `"red"``"yellow"`, StatusText
from `"Felhasználói adatokról nincs mentés"``"Nincs második másolat (csak helyi mentés)"`
2. `backups.html` line 315: Added `✓ Helyi mentés auto` badge before `⚠ Nincs 2. másolat`
3. `style.css`: Added `.layer-auto-ok` class (green text for the auto badge)
---
## Steps
1. Run `go build ./...` and `go vet ./...` from the controller/ directory — fix any errors
2. Update CHANGELOG.md with a new entry at the top (session 43, v0.12.7)
3. Update controller/README.md backup section with the new architecture
4. Commit, build, and deploy following the workflow in CLAUDE.md
```
---
## Context and Goals
The backup system has accumulated an unnecessary `Enabled` flag that gates user data
inclusion in the nightly restic backup. This was a design mistake — user data backup
must be **mandatory** for all apps that have HDD data. Without it, the service isn't viable.
This task removes the `Enabled` gate, makes HDD data backup automatic, and fixes the
restore feature to show all apps.
**Backup architecture (3-2-1 rule):**
1. **Nightly restic (mandatory, same drive)** — DB dumps + config + ALL user data (HDD).
Every app with data gets backed up automatically. This protects against accidental
deletion and enables point-in-time restore. The backup lives on the same drive as the
app, so it does NOT protect against drive failure.
2. **Cross-drive backup (opt-in, different device)** — rsync or restic to a secondary
physical drive. This is the second copy that protects against drive failure. When not
configured for an app with HDD data, the UI warns that data is only on one drive.
3. **Remote backup (future)** — offsite copy for disaster recovery. Not implemented yet.
**Current problems:**
- `AppBackupPrefs.Enabled` flag gates HDD inclusion in nightly restic — should be automatic
- The flag gets out of sync with cross-drive config
- Restore dropdown only shows apps with `HasHDDData && BackupEnabled` — excludes most apps
- "Docker kötetek: Auto ✓" in UI implies volumes are separately backed up — they're not
- Cross-drive backup doesn't trigger a DB dump first
---
## Fix 1: Make HDD data backup mandatory
Remove the `Enabled` flag as a gate. All apps with HDD data are always included in the
nightly restic backup. No toggle, no opt-in.
### 1a: Rewrite `resolveAppBackupPaths()` — include ALL HDD paths
**File:** `internal/backup/backup.go`, function `resolveAppBackupPaths` (line 429)
**Current:**
```go
func (m *Manager) resolveAppBackupPaths() []string {
if m.stackProvider == nil || m.settings == nil {
return nil
}
appBackupMap := m.settings.GetAppBackupMap()
if len(appBackupMap) == 0 {
return nil
}
var paths []string
seen := make(map[string]bool)
for stackName, enabled := range appBackupMap {
if !enabled {
continue
}
hddMounts := m.stackProvider.GetStackHDDMounts(stackName)
for _, mount := range hddMounts {
if seen[mount] {
continue
}
if _, err := os.Stat(mount); err == nil {
paths = append(paths, mount)
seen[mount] = true
m.logger.Printf("[DEBUG] Including app data: %s (from %s)", mount, stackName)
}
}
}
return paths
}
```
**Replace with:**
```go
// resolveAppBackupPaths returns HDD paths for ALL deployed apps.
// User data backup is mandatory — every app with HDD mounts is included.
func (m *Manager) resolveAppBackupPaths() []string {
if m.stackProvider == nil {
return nil
}
var paths []string
seen := make(map[string]bool)
for _, stack := range m.stackProvider.ListDeployedStacks() {
hddMounts := m.stackProvider.GetStackHDDMounts(stack.Name)
for _, mount := range hddMounts {
if seen[mount] {
continue
}
if _, err := os.Stat(mount); err == nil {
paths = append(paths, mount)
seen[mount] = true
m.logger.Printf("[DEBUG] Including app data: %s (from %s)", mount, stack.Name)
}
}
}
return paths
}
```
Key change: no longer reads `GetAppBackupMap()`, no longer checks `Enabled`. Iterates
all deployed stacks via `ListDeployedStacks()` and includes every HDD mount.
The `m.settings` dependency is also removed from this method.
### 1b: Remove `backupPrefs` parameter from `DiscoverAppData()`
**File:** `internal/backup/appdata.go`, function `DiscoverAppData` (line 61)
**Current:**
```go
func DiscoverAppData(provider StackDataProvider, backupPrefs map[string]bool, discoveredDBs []DiscoveredDB) []AppBackupInfo {
```
and at line 100:
```go
info.BackupEnabled = backupPrefs[stack.Name]
```
**Replace function signature with:**
```go
func DiscoverAppData(provider StackDataProvider, discoveredDBs []DiscoveredDB) []AppBackupInfo {
```
**Replace line 100 with:**
```go
// All apps with HDD data are backed up automatically (mandatory)
info.BackupEnabled = info.HasHDDData
```
### 1c: Update `RefreshCache()` caller
**File:** `internal/backup/backup.go`, in `RefreshCache` (around line 547)
**Current:**
```go
if m.stackProvider != nil {
backupPrefs := m.settings.GetAppBackupMap()
status.AppDataInfo = DiscoverAppData(m.stackProvider, backupPrefs, status.DiscoveredDBs)
```
**Replace with:**
```go
if m.stackProvider != nil {
status.AppDataInfo = DiscoverAppData(m.stackProvider, status.DiscoveredDBs)
```
### 1d: Delete dead settings methods
**File:** `internal/settings/settings.go`
Delete these methods entirely — they are no longer called anywhere:
1. `IsAppBackupEnabled()` (lines 235243) — was used by restore.go, replaced in Fix 3f
2. `SetAppBackup()` (lines 245257) — was never called outside settings.go
3. `GetAppBackupMap()` (lines 259271) — was used by resolveAppBackupPaths + RefreshCache, replaced in 1a + 1c
4. `SetAppBackupBulk()` (lines 273287) — was never called outside settings.go
5. `GetAppBackupPrefs()` (lines 289298) — was never called outside settings.go
**Keep** the `AppBackupPrefs` struct and `AppBackup` field in `Settings` — the JSON field
`"app_backup"` still holds `CrossDrive` configs. The `Enabled` field stays in the struct
for backward compat (existing settings.json won't break on load) but nothing reads it.
---
## Fix 2: DB dump before cross-drive backup
When cross-drive backup runs (scheduled or manual), trigger a fresh DB dump for that
app's databases first. This ensures DB state matches the user data being rsynced.
### 2a: Define `DBDumper` interface
**File:** `internal/backup/crossdrive.go`, add near top (after imports):
```go
// DBDumper can run a database dump for a specific stack.
type DBDumper interface {
DumpStackDB(ctx context.Context, stackName string) error
}
```
### 2b: Add field to `CrossDriveRunner`
**File:** `internal/backup/crossdrive.go`
Add `dbDumper` field to the struct (do NOT change the constructor signature):
```go
type CrossDriveRunner struct {
sett *settings.Settings
stackProvider StackDataProvider
dbDumper DBDumper
logger *log.Logger
mu sync.Mutex
running map[string]bool
}
```
Add setter method after `NewCrossDriveRunner`:
```go
// SetDBDumper sets the DB dumper for pre-backup database dumps.
// Called after backup manager is initialized (avoids circular init dependency).
func (r *CrossDriveRunner) SetDBDumper(d DBDumper) {
r.dbDumper = d
}
```
### 2c: Implement `DumpStackDB` on backup Manager
**File:** `internal/backup/backup.go`, add new method:
```go
// DumpStackDB runs a database dump for containers belonging to a specific stack.
// Used by cross-drive backup to ensure DB state matches user data.
func (m *Manager) DumpStackDB(ctx context.Context, stackName string) error {
dbs, err := DiscoverDatabases(ctx, m.logger)
if err != nil {
return fmt.Errorf("database discovery failed: %w", err)
}
var stackDBs []DiscoveredDB
for _, db := range dbs {
if db.StackName == stackName {
stackDBs = append(stackDBs, db)
}
}
if len(stackDBs) == 0 {
m.logger.Printf("[DEBUG] No databases found for stack %s — skipping pre-backup dump", stackName)
return nil
}
m.logger.Printf("[INFO] Running pre-backup DB dump for %s (%d database(s))", stackName, len(stackDBs))
results := DumpAll(ctx, stackDBs, m.cfg.Paths.DBDumpDir, m.logger)
for _, r := range results {
if r.Error != nil {
return fmt.Errorf("DB dump failed for %s: %w", r.DB.ContainerName, r.Error)
}
m.logger.Printf("[INFO] Pre-backup DB dump OK: %s (%s)", r.DB.ContainerName, humanizeBytes(r.Size))
// Persist validation to settings
if m.settings != nil && r.FilePath != "" {
filename := filepath.Base(r.FilePath)
cache := settings.DBValidationCache{
ValidatedAt: time.Now().Format(time.RFC3339),
TableCount: r.Validation.TableCount,
HasHeader: r.Validation.Valid,
}
if !r.Validation.Valid {
cache.Error = r.Validation.Error
}
_ = m.settings.SetDBValidation(filename, cache)
}
}
return nil
}
```
### 2d: Call DB dump in `RunAppBackup`
**File:** `internal/backup/crossdrive.go`, in `RunAppBackup`, add BEFORE the
`ValidateDestination` call (around line 67, after the "Mark as running" block):
```go
// Trigger fresh DB dump for this app before cross-drive backup
if r.dbDumper != nil {
if err := r.dbDumper.DumpStackDB(ctx, stackName); err != nil {
r.logger.Printf("[WARN] Pre-backup DB dump failed for %s: %v — proceeding with user data backup", stackName, err)
// Non-fatal: user data backup is still valuable without fresh dump
}
}
```
### 2e: Wire up in main.go
**File:** `cmd/controller/main.go`, after both `crossDriveRunner` and `backupMgr` are created
(after line 135):
```go
// Wire cross-drive → backup manager for pre-backup DB dumps
if backupMgr != nil {
crossDriveRunner.SetDBDumper(backupMgr)
}
```
---
## Fix 3: Restore dropdown shows ALL deployed apps
The restore section should show every deployed app. All apps have restic snapshots
(stacks dir + DB dumps are always backed up). Apps with HDD data get full restore.
### 3a: Update template filter
**File:** `internal/web/templates/backups.html`, lines 450456
**Current:**
```html
{{range .Backup.AppDataInfo}}
{{if and .HasHDDData .BackupEnabled}}
<option value="{{.StackName}}">{{.DisplayName}}</option>
{{end}}
{{end}}
```
**Replace with:**
```html
{{range .Backup.AppDataInfo}}
<option value="{{.StackName}}" data-has-hdd="{{.HasHDDData}}" data-has-db="{{.HasDBDump}}">{{.DisplayName}}</option>
{{end}}
```
Note: no `data-backup-enabled` attribute — user data is always backed up when present.
### 3b: Show restore type info when app is selected
**File:** `internal/web/templates/backups.html`
Add a new info div after the snapshot selector (after line 464, before the warning div):
```html
<div id="restore-type-info" class="restore-info" style="display:none;margin-bottom:0.5rem">
</div>
```
### 3c: Update `onRestoreAppChange()` JavaScript
**File:** `internal/web/templates/backups.html`, replace the `onRestoreAppChange` function
(lines 604637) with:
```javascript
function onRestoreAppChange() {
var sel = document.getElementById('restore-app');
var appName = sel.value;
var snapSel = document.getElementById('restore-snapshot');
var noSnaps = document.getElementById('restore-no-snapshots');
var typeInfo = document.getElementById('restore-type-info');
document.getElementById('restore-confirm-cb').checked = false;
document.getElementById('restore-btn').disabled = true;
noSnaps.style.display = 'none';
typeInfo.style.display = 'none';
if (!appName) {
snapSel.innerHTML = '<option value="">— Válasszon alkalmazást —</option>';
return;
}
// Determine restore type from data attributes
var opt = sel.options[sel.selectedIndex];
var hasHDD = opt.getAttribute('data-has-hdd') === 'true';
var hasDB = opt.getAttribute('data-has-db') === 'true';
if (hasHDD) {
typeInfo.innerHTML = '🔄 Teljes visszaállítás: adatbázis + konfiguráció + felhasználói adatok a kiválasztott pillanatképből.';
typeInfo.className = 'restore-info';
} else if (hasDB) {
typeInfo.innerHTML = ' Adatbázis és konfiguráció visszaállítása — az alkalmazásnak nincs külön felhasználói adata.';
typeInfo.className = 'restore-info restore-info-partial';
} else {
typeInfo.innerHTML = ' Csak konfiguráció visszaállítása (compose fájlok, beállítások).';
typeInfo.className = 'restore-info restore-info-partial';
}
typeInfo.style.display = 'block';
snapSel.innerHTML = '<option value="">— Betöltés... —</option>';
fetch('/api/backup/snapshots?stack=' + encodeURIComponent(appName))
.then(function(r) { return r.json(); })
.then(function(data) {
snapSel.innerHTML = '<option value="">— Válasszon —</option>';
if (data.ok && data.data && data.data.length > 0) {
data.data.forEach(function(s) {
var o = document.createElement('option');
o.value = s.short_id;
o.textContent = formatSnapshot(s);
snapSel.appendChild(o);
});
} else {
snapSel.innerHTML = '<option value="">— Nincs elérhető mentés —</option>';
noSnaps.style.display = 'block';
}
});
}
```
Note: the JS is simpler than before — no `backupEnabled` check. If `hasHDD` is true, the
data IS backed up (it's mandatory). So: hasHDD → full restore, hasDB → config+DB, else → config only.
### 3d: Add CSS for restore info
**File:** `internal/web/templates/style.css`, add:
```css
.restore-info {
padding: 0.5rem 0.75rem;
border-radius: 6px;
font-size: 0.85rem;
background: rgba(59, 130, 246, 0.1);
border: 1px solid rgba(59, 130, 246, 0.3);
color: #93c5fd;
}
.restore-info-partial {
background: rgba(251, 191, 36, 0.1);
border-color: rgba(251, 191, 36, 0.3);
color: #fcd34d;
}
```
### 3e: Update snapshot filtering API for non-HDD apps
**File:** `internal/api/router.go`, function `backupSnapshots` (line 448)
**Current code (lines 460466):**
```go
if stackName := req.URL.Query().Get("stack"); stackName != "" {
mounts := r.backupMgr.GetStackHDDMounts(stackName)
if len(mounts) > 0 {
snapshots = filterSnapshotsByPaths(snapshots, mounts)
}
}
```
**Replace with:**
```go
if stackName := req.URL.Query().Get("stack"); stackName != "" {
mounts := r.backupMgr.GetStackHDDMounts(stackName)
if len(mounts) > 0 {
// App has HDD data — filter to snapshots containing those paths
snapshots = filterSnapshotsByPaths(snapshots, mounts)
}
// Apps without HDD mounts: return all snapshots (they all contain
// the stacks dir + DB dumps which cover this app's config and database)
}
```
This is effectively a no-op change (just adding a comment), since `if len(mounts) > 0`
already skips filtering for non-HDD apps. The comment clarifies intent.
### 3f: Update `RestoreApp` to handle all apps
**File:** `internal/backup/restore.go`
**Replace the entire file** with:
```go
package backup
import (
"fmt"
"path/filepath"
"regexp"
)
// snapshotIDRe validates restic snapshot IDs: 8-64 lowercase hex characters.
var snapshotIDRe = regexp.MustCompile(`^[0-9a-f]{8,64}$`)
// RestoreApp restores an app from a restic snapshot.
// All apps get config + DB dump restored. Apps with HDD data also get user data restored.
func (m *Manager) RestoreApp(stackName, snapshotID string) error {
if m.stackProvider == nil {
return fmt.Errorf("stack provider not configured")
}
// Validate snapshot ID format
if !snapshotIDRe.MatchString(snapshotID) {
return fmt.Errorf("invalid snapshot ID: must be 8-64 lowercase hex characters")
}
// Prevent concurrent operations
m.mu.Lock()
if m.running {
m.mu.Unlock()
return fmt.Errorf("backup or restore already in progress")
}
m.running = true
m.mu.Unlock()
defer func() {
m.mu.Lock()
m.running = false
m.mu.Unlock()
}()
// Determine what to restore
hddMounts := m.stackProvider.GetStackHDDMounts(stackName)
hasHDD := len(hddMounts) > 0
// Build list of paths to restore from the snapshot
var restorePaths []string
// Always restore the stack's config dir (compose + app.yaml + .felhom.yml)
composePath, ok := m.stackProvider.GetStackComposePath(stackName)
if ok {
stackDir := filepath.Dir(composePath)
restorePaths = append(restorePaths, stackDir)
}
// Restore DB dump files for this stack
if m.cfg.Paths.DBDumpDir != "" {
restorePaths = append(restorePaths, m.cfg.Paths.DBDumpDir)
}
// Restore HDD data (always included for apps that have it — backup is mandatory)
if hasHDD {
restorePaths = append(restorePaths, hddMounts...)
}
if len(restorePaths) == 0 {
return fmt.Errorf("no restorable paths found for %s", stackName)
}
m.logger.Printf("[WARN] RESTORE starting: stack=%s, snapshot=%s, paths=%v, hasHDD=%v",
stackName, snapshotID, restorePaths, hasHDD)
// Stop the app before restore
if err := m.stackProvider.StopStack(stackName); err != nil {
m.logger.Printf("[WARN] RESTORE could not stop %s: %v (proceeding anyway)", stackName, err)
}
// Execute restore via restic
if err := m.restic.RestoreAppData(snapshotID, restorePaths); err != nil {
m.logger.Printf("[ERROR] RESTORE failed for %s: %v", stackName, err)
if startErr := m.stackProvider.StartStack(stackName); startErr != nil {
m.logger.Printf("[WARN] RESTORE could not restart %s after failure: %v", stackName, startErr)
}
return err
}
// Restart the app
if err := m.stackProvider.StartStack(stackName); err != nil {
m.logger.Printf("[WARN] RESTORE could not restart %s after restore: %v", stackName, err)
}
restoreType := "config+DB"
if hasHDD {
restoreType = "full (config+DB+userdata)"
}
m.logger.Printf("[INFO] RESTORE completed: stack=%s, snapshot=%s, type=%s", stackName, snapshotID, restoreType)
return nil
}
```
Key difference from previous version: no `IsAppBackupEnabled()` check — HDD data is always
backed up and always restorable.
---
## Fix 4: Honest Docker volume UI label
### 4a: Change label in template
**File:** `internal/web/templates/backups.html`, lines 283292
**Current:**
```html
<div class="backup-layer-row">
<span class="layer-label">Docker kötetek</span>
<span class="layer-badge">Auto</span>
```
**Replace** `Docker kötetek` with `Konfiguráció`:
```html
<div class="backup-layer-row">
<span class="layer-label">Konfiguráció</span>
<span class="layer-badge">Auto</span>
```
Why: Docker named volumes (`immich_postgres_data`) live at `/var/lib/docker/volumes/`
which is NOT in the restic backup paths. DB data is protected via pg_dump (separate row).
What's actually backed up is compose files + app.yaml + .felhom.yml = configuration.
---
## Summary of all changes
| Fix | What | File(s) |
|-----|------|---------|
| 1a | `resolveAppBackupPaths()` includes ALL deployed stacks' HDD | `backup.go` |
| 1b | Remove `backupPrefs` parameter from `DiscoverAppData()` | `appdata.go` |
| 1c | Update `RefreshCache()` caller to match new signature | `backup.go` |
| 1d | Delete 5 dead settings methods | `settings.go` |
| 2a | `DBDumper` interface | `crossdrive.go` |
| 2b | `dbDumper` field + `SetDBDumper` on CrossDriveRunner | `crossdrive.go` |
| 2c | `DumpStackDB` method on backup Manager | `backup.go` |
| 2d | Call DB dump before cross-drive backup | `crossdrive.go` |
| 2e | Wire dbDumper in main.go | `main.go` |
| 3a | Remove filter from restore dropdown | `backups.html` |
| 3b | Add restore type info div | `backups.html` |
| 3c | Update JS to show restore type per app | `backups.html` |
| 3d | CSS for restore info banners | `style.css` |
| 3e | Comment clarifying snapshot API for non-HDD apps | `router.go` |
| 3f | `RestoreApp` handles config+DB+HDD restore | `restore.go` |
| 4a | Rename "Docker kötetek" → "Konfiguráció" | `backups.html` |
## Files to modify (8)
1. `internal/backup/backup.go` — Fix 1a + 1c + 2c
2. `internal/backup/appdata.go` — Fix 1b
3. `internal/settings/settings.go` — Fix 1d (delete 5 methods)
4. `internal/backup/crossdrive.go` — Fix 2a + 2b + 2d
5. `internal/backup/restore.go` — Fix 3f (full rewrite)
6. `internal/web/templates/backups.html` — Fix 3a + 3b + 3c + 4a
7. `internal/web/templates/style.css` — Fix 3d
8. `internal/api/router.go` — Fix 3e
9. `cmd/controller/main.go` — Fix 2e
## Architecture after fix
```
Backup layers per app (3-2-1 rule):
┌──────────────────────────────────────────────────────┐
│ RULE 1 — Nightly restic (MANDATORY, same drive) │
│ │
│ 1. Adatbázis mentés (Auto) │
│ - pg_dump / mysqldump → /srv/backups/db-dumps/ │
│ - Runs nightly 02:30 (all discovered DBs) │
│ - Also runs before cross-drive backup (per-app) │
│ │
│ 2. Konfiguráció (Auto) │
│ - compose.yml, app.yaml, .felhom.yml │
│ - In nightly restic snapshot (always) │
│ │
│ 3. Felhasználói adatok (Auto — if app has HDD data) │
│ - HDD bind mounts (photos, documents, etc.) │
│ - Always included in nightly restic │
│ - No toggle — backup is mandatory │
│ │
├──────────────────────────────────────────────────────┤
│ RULE 2 — Cross-drive backup (OPT-IN, second device) │
│ │
│ - rsync or restic to a secondary physical drive │
│ - Protects against primary drive failure │
│ - When NOT configured: UI warns "nincs 2. másolat" │
│ - Triggers fresh DB dump before backup │
│ │
├──────────────────────────────────────────────────────┤
│ RULE 3 — Remote backup (FUTURE) │
│ │
│ - Offsite copy for disaster recovery │
│ - Not implemented yet │
└──────────────────────────────────────────────────────┘
Restore capabilities:
┌─────────────────┬────────────┬──────────┬────────────┐
│ App type │ DB restore │ Config │ User data │
├─────────────────┼────────────┼──────────┼────────────┤
│ Has HDD data │ ✓ │ ✓ │ ✓ (always) │
│ DB only, no HDD │ ✓ │ ✓ │ n/a │
│ No DB, no HDD │ — │ ✓ │ n/a │
└─────────────────┴────────────┴──────────┴────────────┘
```
Note: the "DB + HDD, no backup" row from the previous version is GONE. All apps with
HDD data are always backed up. The restore table is simpler.
## Post-fix checklist
- [ ] `go build ./...` passes
- [ ] `go vet ./...` passes
- [ ] Verify no remaining references to deleted methods (`IsAppBackupEnabled`, `GetAppBackupMap`,
`SetAppBackup`, `SetAppBackupBulk`, `GetAppBackupPrefs`) — use `grep -r` to confirm
- [ ] Update `CHANGELOG.md` — session 43, version **v0.12.7**:
- HDD data backup is now mandatory for all apps (no opt-in toggle)
- Removed dead `Enabled` flag gating from settings
- Cross-drive: triggers pre-backup DB dump for consistency
- Restore: all deployed apps now appear in dropdown
- Restore: shows restore type info (full / config+DB / config only)
- Restore: supports config+DB restore for apps without user data
- UI: "Docker kötetek" renamed to "Konfiguráció" (accuracy)
- [ ] Update `controller/README.md` — backup architecture section:
- 3-2-1 backup rule (nightly mandatory, cross-drive opt-in, remote future)
- Restore capabilities matrix
- Cross-drive pre-backup DB dump flow
- [ ] Commit, build on 192.168.0.180, deploy on 192.168.0.162
- [ ] Verify with `docker ps` and `docker logs`
- [ ] After deploy, verify:
- Immich shows in restore dropdown (HDD data → full restore)
- Gokapi/Mealie also show (DB+config restore)
- Selecting Immich shows "Teljes visszaállítás" banner
- Selecting Gokapi shows "Adatbázis és konfiguráció" banner
- Manual cross-drive backup triggers DB dump first (check logs)
- Backup status page shows "Konfiguráció" instead of "Docker kötetek"
- Nightly restic backup includes Immich HDD paths without any toggle
2. Update CHANGELOG.md: add v0.12.7a entry (session 44):
- Fix: restore dropdown now shows snapshots for all apps (removed HDD path filtering)
- Fix: user data warning clarified — shows "Helyi mentés auto / Nincs 2. másolat"
instead of the misleading "Nincs beállítva"
3. Commit, build, and deploy following the workflow in CLAUDE.md
4. Verify:
- Immich now shows restore snapshots (should list all available snapshots)
- Paperless-ngx and RomM show yellow dot (not red) with "✓ Helyi mentés auto · ⚠ Nincs 2. másolat"
- Mealie/Gokapi (no HDD) still show correctly with config+DB restore
+88 -43
View File
@@ -164,76 +164,121 @@ The `/apps/{slug}` page renders hero section, screenshots, setup guide, and opti
### 2. Backup System
The backup system implements a **3-2-1 backup architecture**:
1. **Nightly restic (mandatory, same drive)** — DB dumps + config + ALL user data (HDD). Every app with data is backed up automatically. No toggles.
2. **Cross-drive backup (opt-in, different device)** — rsync or restic to a secondary physical drive. Protects against drive failure.
3. **Remote backup (future)** — offsite copy for disaster recovery.
#### Layer 1: Database Dumps (`internal/backup/dbdump.go`)
| Rule | What | Where | Status |
|------|------|-------|--------|
| **1. Nightly backup** | DB dumps + config + ALL user data | Same drive as app | Mandatory, automatic |
| **2. Cross-drive backup** | User data copy to secondary drive | Different physical device | Opt-in per app |
| **3. Remote backup** | Offsite copy for disaster recovery | Cloud / remote server | Future |
**Key principle:** User data backup is **mandatory** — every app with HDD bind mounts
is included in the nightly restic snapshot automatically. There is no per-app toggle.
The `AppBackupPrefs.Enabled` field in settings.json is legacy and not read by any code.
#### Rule 1: Nightly Backup (mandatory, same drive)
The nightly backup has two phases that run sequentially:
**Phase 1 — Database Dumps** (`internal/backup/dbdump.go`, scheduled 02:30)
- **Auto-discovery** of PostgreSQL and MariaDB containers via `docker ps` + `docker inspect`
- Dumps via `docker exec pg_dump` / `docker exec mariadb-dump` with 5-minute timeout
- Atomic writes (`.tmp``.sql`) to prevent corruption
- **Validation** after each dump: checks file size, header presence, counts `CREATE TABLE` statements
- **Validation** after each dump: checks file size, header presence, counts `CREATE TABLE`
- Results cached in `settings.json` surviving container restarts
- Scheduled nightly at 02:30
- Also triggered per-app by cross-drive backup before each run (`DumpStackDB`)
#### Layer 2: Restic Snapshots (`internal/backup/restic.go`)
**Phase 2 — Restic Snapshot** (`internal/backup/restic.go`, scheduled 03:00)
- Auto-generated repository password (32 random bytes, base64url)
- Password synced to hub for disaster recovery
- Backs up: stacks dir + DB dump dir + **ALL deployed apps' HDD mount paths** (mandatory, no opt-in)
- `resolveAppBackupPaths()` iterates all deployed stacks via `ListDeployedStacks()` — no `Enabled` flag
- Auto-detects and unlocks stale locks
- Auto-generated repository password (32 random bytes, base64url), synced to hub
- **Paths included in every snapshot:**
- Stacks dir (all compose.yml + app.yaml + .felhom.yml)
- DB dump dir (all `.sql` dump files from Phase 1)
- `controller.yaml` (controller config)
- **ALL deployed apps' HDD mount paths** — discovered via `resolveAppBackupPaths()` which iterates `ListDeployedStacks()`, no `Enabled` flag
- Auto-detects and unlocks stale locks (restic repo lock)
- Weekly prune on Sundays with configurable retention (keep-daily, keep-weekly, keep-monthly)
- Weekly integrity check (`restic check`) on Sunday 04:00
- Scheduled nightly at 03:00 (runs after DB dumps complete)
#### Layer 3: Cross-Drive Backup (`internal/backup/crossdrive.go`)
**What this protects against:** accidental deletion, data corruption, point-in-time rollback.
Does NOT protect against drive failure (backup is on the same physical drive).
Implements the 3-2-1 backup rule by copying data to a different physical drive.
#### Rule 2: Cross-Drive Backup (opt-in, different device) (`internal/backup/crossdrive.go`)
Copies user data to a **different physical drive**, providing the second copy for 3-2-1.
- **Two methods:**
- **rsync** — Simple mirror with `--delete` (fast, no versioning)
- **restic** — Versioned, deduplicated, encrypted (shared repo across apps)
- Per-app configuration: destination path, method, schedule (daily/weekly/manual)
- **Pre-backup DB dump**: `DumpStackDB()` runs before cross-drive backup to ensure DB consistency; non-fatal on failure
- **Drive-type-aware validation** (`ValidateDestination` / `CheckBackupDestination`):
- External mount: block if <100 MB free; warn/block at 90%/95% usage
- System drive (same block device as `/`): require ≥10 GB free AND <90% usage; allowed with logged warning
- **Rsync destination layout** (`runRsyncBackup`):
- Single mount: data goes directly into `backups/rsync/<app>/` (no extra nesting)
- Multiple mounts: each gets `backups/rsync/<app>/<leaf>/` subfolder; duplicate leaf names get `_N` suffix
- DB dump files excluded: `--exclude backups/*.sql.gz/sql/dump` — avoids duplicating pg_dump data
- **restic** — Versioned, deduplicated, encrypted (shared repo across apps, auto-generated password)
- Per-app configuration in settings.json: destination path, method, schedule (daily/weekly/manual)
- **Pre-backup DB dump:** `DumpStackDB()` runs fresh pg_dump/mariadb-dump before each cross-drive backup to ensure DB state matches user data; non-fatal on failure (wired via `DBDumper` interface to avoid circular imports)
- **Drive-type-aware validation** (`ValidateDestination`):
| Destination type | Space checks |
|-----------------|--------------|
| External mount (different device than `/`) | Block if <100 MB free |
| System drive (same device as `/`) | Require ≥10 GB free AND <90% used; logged warning |
- **Rsync destination layout:**
- Single mount: `backups/rsync/<app>/` (flat, no extra nesting)
- Multiple mounts: `backups/rsync/<app>/<leaf>/` per mount; duplicate leaf names get `_N` suffix
- DB dump files excluded (`--exclude backups/*.sql.gz/sql/dump`) — already handled by pg_dump
- Safety guards: destination ≠ source, path-overlap check, writable check
- **Chained execution**: cross-drive runs immediately after nightly restic backup (daily apps every night, weekly apps on Sundays)
- **Chained execution:** runs immediately after nightly restic daily apps every night, weekly apps on Sundays
- Per-app concurrency lock prevents overlapping runs
- Status tracking (last_run, duration, size, error) persisted to settings.json
- Status (last_run, duration, size, error) persisted to settings.json
**What this protects against:** primary drive failure, drive theft/damage.
#### Rule 3: Remote Backup (future)
Offsite backup for disaster recovery. Not yet implemented.
#### Restore (`internal/backup/restore.go`)
All deployed apps appear in the restore dropdown — not just those with HDD data.
All deployed apps appear in the restore dropdown — every app has restic snapshot data
(stacks dir + DB dumps are always backed up).
| App type | DB restored | Config restored | User data restored |
|----------|------------|-----------------|-------------------|
| Has HDD data | ✓ | ✓ | ✓ (always — mandatory) |
| App type | Config restored | DB restored | User data restored |
|----------|----------------|------------|-------------------|
| Has HDD data | ✓ | ✓ | ✓ (always — backup is mandatory) |
| DB only, no HDD | ✓ | ✓ | n/a |
| No DB, no HDD | | | n/a |
| No DB, no HDD | | | n/a |
- Restore type info shown in UI when app selected (Hungarian banner: full / config+DB / config only)
- Snapshot API: apps without HDD mounts return all snapshots (all contain stacks dir + DB dumps)
- **Auto stop/restart**: stops app before `restic restore`, restarts after (even on failure)
- **Snapshot API** returns ALL snapshots unfiltered — older snapshots (pre-mandatory HDD backup) still allow config+DB restore; `RestoreApp` extracts whatever paths are available
- **Restore type info** shown per-app when selected in dropdown (Hungarian banners):
- Has HDD: "Teljes visszaállítás: adatbázis + konfiguráció + felhasználói adatok"
- Has DB, no HDD: "Adatbázis és konfiguráció visszaállítása"
- No DB, no HDD: "Csak konfiguráció visszaállítása"
- **Execution flow:** stop app → `restic restore <id> --target / --include <path>...` → restart app
- Running flag prevents concurrent backup/restore operations
- Snapshot ID validated (864 lowercase hex)
#### Backup Page UI
#### Backup Page UI (`internal/web/templates/backups.html`)
The backups page shows a unified per-app status table:
- **Status dot**: green (fully covered), yellow (warning — failed run, system drive, disk full), red (HDD data without cross-drive), auto (no user data)
- Expandable row per app showing all 3 backup layers (DB, Konfiguráció, user data)
- Schedule overview with next run times
- Snapshot history table (last 20 snapshots with ID, time, data added)
Unified per-app status table with expandable rows showing 3 backup layers per app:
**Status dot per app:**
| Dot color | Meaning |
|-----------|---------|
| Green | Fully covered — cross-drive configured and last run OK |
| Yellow | Warning — no second copy, or last backup failed, or disk space issue |
| Red | Cross-drive destination blocked or inaccessible |
| Gray (auto) | No user data — only config/DB backup (automatic) |
**Three backup layers per app row:**
1. **Adatbázis mentés** — Auto badge + last run timestamp + status
2. **Konfiguráció** — Auto badge + last restic snapshot timestamp + status
3. **Felhasználói adatok** — one of:
- Cross-drive configured: method + destination + schedule + last run + status + "Futtatás most" button
- HDD data, no cross-drive: "✓ Helyi mentés auto" (green) + "⚠ Nincs 2. másolat" (yellow) + settings link
- No HDD data: "— (nincs HDD adat)" (muted)
**Other sections:**
- Schedule overview with next run times for DB dump, restic, prune
- Snapshot history table (last 20 snapshots with ID, time, files new/changed, data added)
- Repository info card (path, size, snapshot count, encryption key with show/copy)
- Restore section with app/snapshot dropdowns and confirmation flow
- Restore section: app dropdown → snapshot dropdown → restore type info → confirmation checkbox → execute
---
+5 -10
View File
@@ -457,16 +457,11 @@ func (r *Router) backupSnapshots(w http.ResponseWriter, req *http.Request) {
return
}
// Filter by stack if requested — only return snapshots that include the app's HDD paths.
if stackName := req.URL.Query().Get("stack"); stackName != "" {
mounts := r.backupMgr.GetStackHDDMounts(stackName)
if len(mounts) > 0 {
// App has HDD data — filter to snapshots containing those paths
snapshots = filterSnapshotsByPaths(snapshots, mounts)
}
// Apps without HDD mounts: return all snapshots (they all contain
// the stacks dir + DB dumps which cover this app's config and database)
}
// All snapshots contain the stacks dir + DB dumps, so they're useful for
// any app (config + DB restore). Apps with HDD data get user data restored
// too — but only from snapshots that include those paths (post-v0.12.7).
// We don't filter here because older snapshots still allow config+DB restore,
// and the RestoreApp function extracts whatever paths are available.
if snapshots == nil {
snapshots = []backup.SnapshotInfo{}
+3 -3
View File
@@ -598,10 +598,10 @@ func (s *Server) buildAppBackupRows(
cfg, hasCfg := crossConfigs[app.StackName]
if !hasCfg || cfg == nil || !cfg.Enabled {
// HDD data but no cross-drive configured → RED
// HDD data backed up via nightly restic (mandatory), but no second copy
row.UserDataConfigured = false
row.Status = "red"
row.StatusText = "Felhasználói adatokról nincs mentés"
row.Status = "yellow"
row.StatusText = "Nincs második másolat (csak helyi mentés)"
} else {
row.UserDataConfigured = true
row.UserDataMethod = cfg.Method
@@ -312,7 +312,8 @@
Futtatás most</button>
</div>
{{else}}
<span class="layer-unconfigured">⚠ Nincs beállítva</span>
<span class="layer-auto-ok">✓ Helyi mentés auto</span>
<span class="layer-unconfigured">⚠ Nincs 2. másolat</span>
<a href="/stacks/{{.StackName}}/deploy" class="btn btn-xs">Beállítás →</a>
{{end}}
{{else}}
@@ -2561,6 +2561,11 @@ a.stat-card:hover {
font-size: .8rem;
margin-left: .25rem;
}
.layer-auto-ok {
color: var(--green);
font-size: .85rem;
margin-right: .5rem;
}
.layer-unconfigured {
color: var(--yellow);
font-weight: 500;