# Per-App Cross-Drive Backup — Design & Task Document ## Overview Extend the controller with per-app user data backup to a **secondary storage drive**. This is distinct from the existing nightly restic snapshot (which backs up to the same drive). The cross-drive backup provides the "second copy on different media" part of the 3-2-1 backup rule. Two mechanisms available (user chooses per app): - **rsync** — Simple file mirror. Easy to browse via FileBrowser. No versioning. - **restic** — Versioned, encrypted, deduplicated snapshots on the secondary drive. ### Current State (what already exists) | Feature | Status | Location | |---------|--------|----------| | Per-app backup toggle | ✅ Exists | Backup page, `settings.json` `app_backup` map | | `resolveAppBackupPaths()` | ✅ Exists | `backup.go` — includes enabled app HDD paths in nightly restic | | `AppBackupInfo` discovery | ✅ Exists | `appdata.go` — discovers HDD mounts, Docker volumes per app | | Storage paths registry | ✅ Exists | `settings.json` — multiple paths with labels, health, default | | Mount health checks | ✅ Exists | `mounts_linux.go` — `IsMountPoint()`, `GetDiskUsage()` | | Scheduler | ✅ Exists | `scheduler.go` — daily cron-style jobs | | Stale data cleanup | ✅ v0.11.7 | Deploy page — delete old data from non-active paths | ### What's New The existing backup toggle (`Enabled bool`) includes app data in the **same-drive** restic snapshot. The new feature adds a completely separate **cross-drive** backup job with its own method, destination, and schedule. --- ## Data Model ### Extended `AppBackupPrefs` in `settings.json` ```go // AppBackupPrefs holds per-app backup configuration. type AppBackupPrefs struct { // Existing: includes app data in nightly restic (same drive) Enabled bool `json:"enabled"` // NEW: Cross-drive backup to secondary storage CrossDrive *CrossDriveBackup `json:"cross_drive,omitempty"` } // CrossDriveBackup configures per-app backup to a secondary drive. type CrossDriveBackup struct { Enabled bool `json:"enabled"` Method string `json:"method"` // "rsync" or "restic" DestinationPath string `json:"destination_path"` // e.g., "/mnt/hdd_1" Schedule string `json:"schedule"` // "daily", "weekly", "manual" // Runtime state (updated by backup runner, persisted for display) LastRun string `json:"last_run,omitempty"` // RFC3339 LastStatus string `json:"last_status,omitempty"` // "ok", "error", "running" LastError string `json:"last_error,omitempty"` LastDuration string `json:"last_duration,omitempty"` // "2m34s" LastSizeHuman string `json:"last_size_human,omitempty"` // "1.2 GB" } ``` Example `settings.json`: ```json { "app_backup": { "immich": { "enabled": true, "cross_drive": { "enabled": true, "method": "rsync", "destination_path": "/mnt/hdd_1", "schedule": "daily", "last_run": "2026-02-17T03:15:00Z", "last_status": "ok", "last_duration": "45s", "last_size_human": "48 MB" } }, "paperless-ngx": { "enabled": true, "cross_drive": { "enabled": true, "method": "restic", "destination_path": "/mnt/hdd_1", "schedule": "weekly" } } } } ``` ### Cross-drive backup directory layout On the destination drive: ``` /mnt/hdd_1/ ├── storage/ # App user data (active apps store data here) │ ├── immich/ │ └── paperless-ngx/ ├── media/ # User media files ├── Dokumentumok/ # User documents └── backups/ # NEW: Cross-drive backups ├── rsync/ # Mirror copies │ ├── immich/ # rsync of /mnt/hdd_placeholder/storage/immich/ │ └── ... └── restic/ # Restic repository for versioned backups ├── config ├── data/ ├── index/ ├── keys/ └── snapshots/ ``` Key decisions: - All cross-drive backups go under `{destination}/backups/` to keep them separate from active app data - rsync method: one directory per app under `backups/rsync/{stackname}/` - restic method: single shared restic repo at `backups/restic/` (dedup benefits from shared repo) - Restic repo on secondary drive uses a **separate password** stored in `settings.json` (not the same as the main backup repo) --- ## Architecture ### New package: `internal/backup/crossdrive.go` ```go // CrossDriveRunner handles per-app backup to secondary storage. type CrossDriveRunner struct { settings *settings.Settings stackProvider StackDataProvider logger *log.Logger mu sync.Mutex running map[string]bool // per-app running state } // RunAppBackup runs cross-drive backup for a single app. func (r *CrossDriveRunner) RunAppBackup(ctx context.Context, stackName string) error // RunAllScheduled runs cross-drive backup for all apps matching the schedule. func (r *CrossDriveRunner) RunAllScheduled(ctx context.Context, schedule string) error // GetAppStatus returns the current cross-drive backup status for an app. func (r *CrossDriveRunner) GetAppStatus(stackName string) *CrossDriveStatus ``` ### rsync backup flow ``` 1. Validate: destination path mounted & writable 2. Resolve app HDD mounts (e.g., /mnt/hdd_placeholder/storage/immich/) 3. Create destination: {dest}/backups/rsync/{stackname}/ 4. For each HDD mount: rsync -a --delete --info=progress2 \ /mnt/hdd_placeholder/storage/immich/ \ /mnt/hdd_1/backups/rsync/immich/storage/immich/ 5. Update settings: last_run, last_status, last_size_human ``` Note: `--delete` mirrors exactly — old files on destination get removed. This is a mirror, not versioned. ### restic backup flow ``` 1. Validate: destination path mounted & writable 2. Ensure shared restic repo initialized at {dest}/backups/restic/ 3. Resolve app HDD mounts 4. restic backup --repo {dest}/backups/restic/ \ --password-file {settings-based} \ --tag {stackname} \ /mnt/hdd_placeholder/storage/immich/ 5. Update settings: last_run, last_status, last_size_human ``` Restic benefits: dedup across apps, versioned snapshots, can restore specific point-in-time. --- ## UI Design ### 1. Deploy/Settings Page — Per-App Section On the deploy page (when `AlreadyDeployed`), after the "Adattárolás" card and before/after the "Korábbi adatok" card, add a new **"Biztonsági mentés"** card: ``` ┌────────────────────────────────────────────────────────┐ │ 🔒 Biztonsági mentés │ │ │ │ ☑ Napi mentésbe foglalás (restic, helyi) │ │ Az alkalmazás adatai bekerülnek az éjszakai │ │ biztonsági mentésbe. │ │ │ │ ───────────────────────────────────────────── │ │ │ │ Másolat másik meghajtóra: │ │ │ │ Cél tárhely: [▼ Külső HDD 1TB (/mnt/hdd_1) ★] │ │ Módszer: [▼ Egyszerű másolat (rsync) ] │ │ Ütemezés: [▼ Naponta ] │ │ │ │ Utolsó futás: 2026-02-17 03:15 — ✅ Sikeres (45s) │ │ Méret: 48 MB │ │ │ │ [Mentés most] [Beállítások mentése] │ │ │ │ ⚠️ A cél meghajtó legyen más fizikai eszköz, mint │ │ az alkalmazás adattárolója. │ └────────────────────────────────────────────────────────┘ ``` **States:** - **No other storage path available:** Card visible but form disabled with message: "Másik adattároló szükséges a másolat készítéséhez. Csatlakoztass egy külső meghajtót a Beállítások oldalon." - **Other path available but not configured:** Dropdowns shown, save button active - **Configured and healthy:** Shows last run status, manual trigger available - **Configured but destination unreachable:** Red warning: "⚠️ A cél tárhely ({path}) nem elérhető! Ellenőrizd a meghajtó csatlakozását." ### 2. Backup Page — Summary Card On the central "Biztonsági mentés" page, add a new section after "Alkalmazás adatok": ``` ┌────────────────────────────────────────────────────────┐ │ Másolatok másik meghajtóra │ │ │ │ ┌──────────────────────────────────────────────────┐ │ │ │ Immich rsync → hdd_1 ✅ 03:15 48MB │ │ │ │ Paperless-ngx restic → hdd_1 ⏰ Heti (V) │ │ │ └──────────────────────────────────────────────────┘ │ │ │ │ ⚠️ 1 alkalmazáshoz nincs beállítva: │ │ RoMM — Beállítás → │ │ │ │ [Összes futtatása most] │ └────────────────────────────────────────────────────────┘ ``` Each row links to the app's deploy/settings page. Shows warnings for: - Apps with HDD data but no cross-drive backup configured - Destinations that are unreachable/unmounted - Last run failures --- ## Routes ### New API endpoints | Method | Path | Auth | Description | |--------|------|------|-------------| | POST | `/api/stacks/{name}/cross-backup` | Yes | Save cross-drive backup config for app | | POST | `/api/stacks/{name}/cross-backup/run` | Yes | Trigger manual run for single app | | GET | `/api/stacks/{name}/cross-backup/status` | Yes | Get current status (for polling) | | POST | `/api/backup/cross-drive/run-all` | Yes | Trigger all scheduled cross-drive backups | ### New web handler | Method | Path | Description | |--------|------|-------------| | POST | `/settings/cross-backup/{name}` | Form POST from deploy page (redirect back) | --- ## Implementation Steps ### Step 0: CSS fix (immediate) Add `margin-bottom: 1.5rem` to `.deploy-stale-data` in `style.css`. ### Step 1: Extend data model **Files:** `settings.go` - Add `CrossDriveBackup` struct - Extend `AppBackupPrefs` with `CrossDrive` field - Add getter/setter methods: - `GetCrossDriveConfig(stackName string) *CrossDriveBackup` - `SetCrossDriveConfig(stackName string, cfg CrossDriveBackup) error` - `GetAllCrossDriveConfigs() map[string]*CrossDriveBackup` - Add `CrossDriveResticPassword` field to Settings for the secondary restic repo - Auto-generate password on first restic cross-drive config save ### Step 2: Cross-drive backup runner **New file:** `internal/backup/crossdrive.go` - `CrossDriveRunner` struct with mutex, settings, stack provider, logger - `RunAppBackup(ctx, stackName)`: 1. Load config from settings 2. Validate destination: `IsMountPoint()` + `IsWritable()` 3. Resolve HDD mounts via `StackDataProvider` 4. Branch on method: - rsync: `runRsyncBackup()` — runs rsync per mount with `--delete --info=progress2` - restic: `runResticBackup()` — ensures repo init, runs `restic backup` with app tag 5. Update settings with result (last_run, last_status, etc.) 6. Log result - `RunAllScheduled(ctx, schedule)`: 1. Iterate all apps with cross_drive enabled 2. Filter by schedule match (daily → every day, weekly → Sunday) 3. Run sequentially (not parallel — disk I/O bound) - `GetAppStatus(stackName)` — returns latest status from settings - `ValidateDestination(path)` — checks mount + writable + free space **rsync specifics:** ```go func (r *CrossDriveRunner) runRsyncBackup(stackName, destBase string, mounts []string) error { destDir := filepath.Join(destBase, "backups", "rsync", stackName) os.MkdirAll(destDir, 0755) for _, srcMount := range mounts { // Preserve directory structure relative to storage root // e.g., /mnt/hdd_placeholder/storage/immich/ → {dest}/backups/rsync/immich/storage/immich/ relPath := strings.TrimPrefix(srcMount, filepath.Dir(filepath.Dir(srcMount))) dstPath := filepath.Join(destDir, relPath) os.MkdirAll(filepath.Dir(dstPath), 0755) cmd := exec.Command("rsync", "-a", "--delete", srcMount+"/", dstPath+"/") if out, err := cmd.CombinedOutput(); err != nil { return fmt.Errorf("rsync failed for %s: %v (%s)", srcMount, err, string(out)) } } return nil } ``` **restic specifics:** ```go func (r *CrossDriveRunner) runResticBackup(stackName, destBase string, mounts []string) error { repoPath := filepath.Join(destBase, "backups", "restic") passwordFile := r.getResticPasswordFile() // from settings or dedicated file // Ensure initialized if !r.isRepoInitialized(repoPath) { cmd := exec.Command("restic", "init", "--repo", repoPath, "--password-file", passwordFile) if out, err := cmd.CombinedOutput(); err != nil { return fmt.Errorf("restic init failed: %v (%s)", err, string(out)) } } // Build args args := []string{"backup", "--repo", repoPath, "--password-file", passwordFile, "--tag", stackName, "--tag", "cross-drive"} args = append(args, mounts...) cmd := exec.Command("restic", args...) if out, err := cmd.CombinedOutput(); err != nil { return fmt.Errorf("restic backup failed: %v (%s)", err, string(out)) } return nil } ``` ### Step 3: Scheduler integration **File:** `main.go` (scheduler registration) Add a new daily job that runs after the existing backup: ```go // Cross-drive backup job — runs at 03:30 (after main backup at 03:00) sched.RegisterDaily("cross_drive_backup", "03:30", func(ctx context.Context) error { return crossDriveRunner.RunAllScheduled(ctx, "daily") }) // Weekly cross-drive job — runs Sundays at 04:00 sched.RegisterWeekly("cross_drive_weekly", time.Sunday, "04:00", func(ctx context.Context) error { return crossDriveRunner.RunAllScheduled(ctx, "weekly") }) ``` Note: If `RegisterWeekly` doesn't exist in the scheduler, we can check the day inside `RunAllScheduled` (like the existing `shouldPrune` pattern). ### Step 4: API endpoints **File:** `internal/api/router.go` Add to the switch: ```go // POST /api/stacks/{name}/cross-backup — save config case hasSuffix(path, "/cross-backup") && req.Method == http.MethodPost: r.saveCrossBackupConfig(w, req, extractName(path, "/cross-backup")) // POST /api/stacks/{name}/cross-backup/run — trigger manual run case hasSuffix(path, "/cross-backup/run") && req.Method == http.MethodPost: r.triggerCrossBackup(w, req, extractNameFromPath(path, "/cross-backup/run")) // GET /api/stacks/{name}/cross-backup/status — poll status case hasSuffix(path, "/cross-backup/status") && req.Method == http.MethodGet: r.getCrossBackupStatus(w, req, extractNameFromPath(path, "/cross-backup/status")) // POST /api/backup/cross-drive/run-all — trigger all case path == "/backup/cross-drive/run-all" && req.Method == http.MethodPost: r.triggerAllCrossBackups(w, req) ``` ### Step 5: Deploy page UI **File:** `internal/web/templates/deploy.html` After the StorageInfo section, add the backup card for deployed apps with HDD data. The card is rendered server-side with current config values. **File:** `internal/web/handlers.go` In `deployHandler`, populate new template data: ```go if alreadyDeployed { // ... existing storageInfo ... // Cross-drive backup config for this app crossCfg := s.settings.GetCrossDriveConfig(name) data["CrossDriveConfig"] = crossCfg // Other storage paths for destination dropdown (exclude current app path) var destPaths []DeployStoragePath for _, sp := range s.settings.GetStoragePaths() { if storageInfo != nil && sp.Path == storageInfo.Path { continue // skip the app's current storage } dp := DeployStoragePath{StoragePath: sp} if di := system.GetDiskUsage(sp.Path); di != nil { dp.FreeHuman = formatFreeSpace(di.AvailGB) dp.FreePercent = di.AvailGB / di.TotalGB * 100 } destPaths = append(destPaths, dp) } data["BackupDestPaths"] = destPaths // Destination health warning if crossCfg != nil && crossCfg.Enabled && crossCfg.DestinationPath != "" { if !system.IsMountPoint(crossCfg.DestinationPath) || !system.IsWritable(crossCfg.DestinationPath) { data["BackupDestWarning"] = fmt.Sprintf( "A cél tárhely (%s) nem elérhető! Ellenőrizd a meghajtó csatlakozását.", crossCfg.DestinationPath, ) } } // Existing nightly backup toggle state appBackupEnabled := false if prefs, ok := s.settings.GetAppBackupPrefs(name); ok { appBackupEnabled = prefs.Enabled } data["AppBackupEnabled"] = appBackupEnabled } ``` ### Step 6: Backup page summary **File:** `internal/web/templates/backups.html` Add section after "Alkalmazás adatok": ```html {{if .Backup.CrossDriveSummary}}

Másolatok másik meghajtóra

Alkalmazás adatok biztonsági másolata külső meghajtóra.

{{if .Backup.CrossDriveWarnings}}
{{range .Backup.CrossDriveWarnings}}
{{.}}
{{end}}
{{end}}
{{range .Backup.CrossDriveSummary}}
{{.DisplayName}}
{{.MethodLabel}} → {{.DestLabel}} {{if eq .LastStatus "ok"}}✅ {{.LastRunShort}} {{else if eq .LastStatus "error"}}❌ Hiba {{else}}⏰ {{.ScheduleLabel}}{{end}} {{if .SizeHuman}}{{.SizeHuman}}{{end}}
{{end}}
{{if .Backup.UnconfiguredApps}}
⚠️ {{len .Backup.UnconfiguredApps}} alkalmazáshoz nincs beállítva: {{range .Backup.UnconfiguredApps}} {{.DisplayName}}{{if not $.Last}}, {{end}} {{end}}
{{end}}
{{end}} ``` **File:** `internal/web/handlers.go` — in backup page handler Populate the summary data: ```go // Cross-drive backup summary type CrossDriveSummaryItem struct { StackName string DisplayName string Method string // "rsync" or "restic" MethodLabel string // "Egyszerű másolat" or "Restic" DestPath string DestLabel string Schedule string ScheduleLabel string // "Naponta" or "Hetente" LastStatus string LastRunShort string SizeHuman string } ``` ### Step 7: Destination health monitoring **File:** `internal/web/handlers.go` or `internal/monitor/health.go` In the periodic health check (runs every 5 min), also check cross-drive destinations: ```go func (s *Server) checkCrossDriveDestinations() []string { var warnings []string configs := s.settings.GetAllCrossDriveConfigs() seen := make(map[string]bool) for stackName, cfg := range configs { if !cfg.Enabled || cfg.DestinationPath == "" || seen[cfg.DestinationPath] { continue } seen[cfg.DestinationPath] = true if !system.IsMountPoint(cfg.DestinationPath) { warnings = append(warnings, fmt.Sprintf("A(z) %s mentési célja (%s) nincs csatlakoztatva!", stackName, cfg.DestinationPath)) } else if !system.IsWritable(cfg.DestinationPath) { warnings = append(warnings, fmt.Sprintf("A(z) %s mentési célja (%s) nem írható!", stackName, cfg.DestinationPath)) } } return warnings } ``` Include warnings in both the backup page and the hub report. --- ## Safety Guards 1. **Destination ≠ Source**: Never allow backup destination to be the same storage path as the app's HDD_PATH 2. **Protected paths**: Use `ProtectedHDDPaths()` — never write to top-level dirs 3. **No parallel runs**: Mutex per app — skip if already running 4. **Free space check**: Before starting, verify destination has sufficient free space (source size × 1.1) 5. **rsync `--delete`**: Clearly warn user that rsync mirror deletes files on destination that were removed from source 6. **Restic password**: Auto-generated, stored in `settings.json`, displayed in backup page for recovery --- ## Files Summary ### New files (3) | File | Purpose | |------|---------| | `internal/backup/crossdrive.go` | Cross-drive backup runner (rsync + restic) | | `internal/backup/crossdrive_test.go` | Unit tests for path validation, config parsing | | (templates inline changes) | | ### Modified files (9) | File | Changes | |------|---------| | `internal/settings/settings.go` | `CrossDriveBackup` struct, getter/setter methods, password storage | | `internal/backup/appdata.go` | Add `CrossDriveConfig` to `AppBackupInfo` for backup page | | `internal/backup/backup.go` | Add `CrossDriveRunner` field, wire into `Manager` | | `internal/api/router.go` | 4 new API routes | | `internal/web/handlers.go` | Deploy page data + backup page cross-drive summary | | `internal/web/server.go` | Wire `CrossDriveRunner` | | `internal/web/templates/deploy.html` | Backup config card for deployed apps | | `internal/web/templates/backups.html` | Cross-drive summary section | | `internal/web/templates/style.css` | Stale data margin fix + cross-drive styles | | `main.go` | Create runner, register scheduler jobs | --- ## Testing Checklist ### rsync method 1. Configure Immich rsync → hdd_1, daily 2. Trigger manual run → verify `/mnt/hdd_1/backups/rsync/immich/` created with data 3. Add a file to immich upload, run again → file appears in backup 4. Delete a file from immich, run again → file removed from backup (--delete) 5. Disconnect hdd_1 → warning shows on deploy page + backup page ### restic method 1. Configure Paperless restic → hdd_1, weekly 2. Trigger manual run → verify repo created at `/mnt/hdd_1/backups/restic/` 3. List snapshots: `restic -r /mnt/hdd_1/backups/restic/ snapshots --tag paperless-ngx` 4. Run again → second snapshot, dedup keeps repo small 5. Test restore: `restic -r /mnt/hdd_1/backups/restic/ restore latest --tag paperless-ngx --target /tmp/test` ### Scheduler 1. Set schedule daily → verify it runs after nightly backup 2. Set schedule weekly → verify it only runs on Sunday 3. Set schedule manual → verify it doesn't auto-run ### Destination monitoring 1. Configure backup to hdd_1 → no warnings 2. Unmount hdd_1 → warning appears on deploy + backup pages 3. Remount → warning clears ### Edge cases 1. App with no HDD data → backup card not shown 2. Only one storage path → "Másik adattároló szükséges" message 3. Source = destination → rejected with error 4. Destination full → error logged, status shows "error" 5. App migrated to new storage → backup source paths update automatically (reads from app.yaml)