Files
deploy-felhom-compose/TASK.md
T

624 lines
25 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Per-App Cross-Drive Backup — Design & Task Document
## Overview
Extend the controller with per-app user data backup to a **secondary storage drive**. This is distinct from the existing nightly restic snapshot (which backs up to the same drive). The cross-drive backup provides the "second copy on different media" part of the 3-2-1 backup rule.
Two mechanisms available (user chooses per app):
- **rsync** — Simple file mirror. Easy to browse via FileBrowser. No versioning.
- **restic** — Versioned, encrypted, deduplicated snapshots on the secondary drive.
### Current State (what already exists)
| Feature | Status | Location |
|---------|--------|----------|
| Per-app backup toggle | ✅ Exists | Backup page, `settings.json` `app_backup` map |
| `resolveAppBackupPaths()` | ✅ Exists | `backup.go` — includes enabled app HDD paths in nightly restic |
| `AppBackupInfo` discovery | ✅ Exists | `appdata.go` — discovers HDD mounts, Docker volumes per app |
| Storage paths registry | ✅ Exists | `settings.json` — multiple paths with labels, health, default |
| Mount health checks | ✅ Exists | `mounts_linux.go``IsMountPoint()`, `GetDiskUsage()` |
| Scheduler | ✅ Exists | `scheduler.go` — daily cron-style jobs |
| Stale data cleanup | ✅ v0.11.7 | Deploy page — delete old data from non-active paths |
### What's New
The existing backup toggle (`Enabled bool`) includes app data in the **same-drive** restic snapshot. The new feature adds a completely separate **cross-drive** backup job with its own method, destination, and schedule.
---
## Data Model
### Extended `AppBackupPrefs` in `settings.json`
```go
// AppBackupPrefs holds per-app backup configuration.
type AppBackupPrefs struct {
// Existing: includes app data in nightly restic (same drive)
Enabled bool `json:"enabled"`
// NEW: Cross-drive backup to secondary storage
CrossDrive *CrossDriveBackup `json:"cross_drive,omitempty"`
}
// CrossDriveBackup configures per-app backup to a secondary drive.
type CrossDriveBackup struct {
Enabled bool `json:"enabled"`
Method string `json:"method"` // "rsync" or "restic"
DestinationPath string `json:"destination_path"` // e.g., "/mnt/hdd_1"
Schedule string `json:"schedule"` // "daily", "weekly", "manual"
// Runtime state (updated by backup runner, persisted for display)
LastRun string `json:"last_run,omitempty"` // RFC3339
LastStatus string `json:"last_status,omitempty"` // "ok", "error", "running"
LastError string `json:"last_error,omitempty"`
LastDuration string `json:"last_duration,omitempty"` // "2m34s"
LastSizeHuman string `json:"last_size_human,omitempty"` // "1.2 GB"
}
```
Example `settings.json`:
```json
{
"app_backup": {
"immich": {
"enabled": true,
"cross_drive": {
"enabled": true,
"method": "rsync",
"destination_path": "/mnt/hdd_1",
"schedule": "daily",
"last_run": "2026-02-17T03:15:00Z",
"last_status": "ok",
"last_duration": "45s",
"last_size_human": "48 MB"
}
},
"paperless-ngx": {
"enabled": true,
"cross_drive": {
"enabled": true,
"method": "restic",
"destination_path": "/mnt/hdd_1",
"schedule": "weekly"
}
}
}
}
```
### Cross-drive backup directory layout
On the destination drive:
```
/mnt/hdd_1/
├── storage/ # App user data (active apps store data here)
│ ├── immich/
│ └── paperless-ngx/
├── media/ # User media files
├── Dokumentumok/ # User documents
└── backups/ # NEW: Cross-drive backups
├── rsync/ # Mirror copies
│ ├── immich/ # rsync of /mnt/hdd_placeholder/storage/immich/
│ └── ...
└── restic/ # Restic repository for versioned backups
├── config
├── data/
├── index/
├── keys/
└── snapshots/
```
Key decisions:
- All cross-drive backups go under `{destination}/backups/` to keep them separate from active app data
- rsync method: one directory per app under `backups/rsync/{stackname}/`
- restic method: single shared restic repo at `backups/restic/` (dedup benefits from shared repo)
- Restic repo on secondary drive uses a **separate password** stored in `settings.json` (not the same as the main backup repo)
---
## Architecture
### New package: `internal/backup/crossdrive.go`
```go
// CrossDriveRunner handles per-app backup to secondary storage.
type CrossDriveRunner struct {
settings *settings.Settings
stackProvider StackDataProvider
logger *log.Logger
mu sync.Mutex
running map[string]bool // per-app running state
}
// RunAppBackup runs cross-drive backup for a single app.
func (r *CrossDriveRunner) RunAppBackup(ctx context.Context, stackName string) error
// RunAllScheduled runs cross-drive backup for all apps matching the schedule.
func (r *CrossDriveRunner) RunAllScheduled(ctx context.Context, schedule string) error
// GetAppStatus returns the current cross-drive backup status for an app.
func (r *CrossDriveRunner) GetAppStatus(stackName string) *CrossDriveStatus
```
### rsync backup flow
```
1. Validate: destination path mounted & writable
2. Resolve app HDD mounts (e.g., /mnt/hdd_placeholder/storage/immich/)
3. Create destination: {dest}/backups/rsync/{stackname}/
4. For each HDD mount:
rsync -a --delete --info=progress2 \
/mnt/hdd_placeholder/storage/immich/ \
/mnt/hdd_1/backups/rsync/immich/storage/immich/
5. Update settings: last_run, last_status, last_size_human
```
Note: `--delete` mirrors exactly — old files on destination get removed. This is a mirror, not versioned.
### restic backup flow
```
1. Validate: destination path mounted & writable
2. Ensure shared restic repo initialized at {dest}/backups/restic/
3. Resolve app HDD mounts
4. restic backup --repo {dest}/backups/restic/ \
--password-file {settings-based} \
--tag {stackname} \
/mnt/hdd_placeholder/storage/immich/
5. Update settings: last_run, last_status, last_size_human
```
Restic benefits: dedup across apps, versioned snapshots, can restore specific point-in-time.
---
## UI Design
### 1. Deploy/Settings Page — Per-App Section
On the deploy page (when `AlreadyDeployed`), after the "Adattárolás" card and before/after the "Korábbi adatok" card, add a new **"Biztonsági mentés"** card:
```
┌────────────────────────────────────────────────────────┐
│ 🔒 Biztonsági mentés │
│ │
│ ☑ Napi mentésbe foglalás (restic, helyi) │
│ Az alkalmazás adatai bekerülnek az éjszakai │
│ biztonsági mentésbe. │
│ │
│ ───────────────────────────────────────────── │
│ │
│ Másolat másik meghajtóra: │
│ │
│ Cél tárhely: [▼ Külső HDD 1TB (/mnt/hdd_1) ★] │
│ Módszer: [▼ Egyszerű másolat (rsync) ] │
│ Ütemezés: [▼ Naponta ] │
│ │
│ Utolsó futás: 2026-02-17 03:15 — ✅ Sikeres (45s) │
│ Méret: 48 MB │
│ │
│ [Mentés most] [Beállítások mentése] │
│ │
│ ⚠️ A cél meghajtó legyen más fizikai eszköz, mint │
│ az alkalmazás adattárolója. │
└────────────────────────────────────────────────────────┘
```
**States:**
- **No other storage path available:** Card visible but form disabled with message: "Másik adattároló szükséges a másolat készítéséhez. Csatlakoztass egy külső meghajtót a Beállítások oldalon."
- **Other path available but not configured:** Dropdowns shown, save button active
- **Configured and healthy:** Shows last run status, manual trigger available
- **Configured but destination unreachable:** Red warning: "⚠️ A cél tárhely ({path}) nem elérhető! Ellenőrizd a meghajtó csatlakozását."
### 2. Backup Page — Summary Card
On the central "Biztonsági mentés" page, add a new section after "Alkalmazás adatok":
```
┌────────────────────────────────────────────────────────┐
│ Másolatok másik meghajtóra │
│ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ Immich rsync → hdd_1 ✅ 03:15 48MB │ │
│ │ Paperless-ngx restic → hdd_1 ⏰ Heti (V) │ │
│ └──────────────────────────────────────────────────┘ │
│ │
│ ⚠️ 1 alkalmazáshoz nincs beállítva: │
│ RoMM — Beállítás → │
│ │
│ [Összes futtatása most] │
└────────────────────────────────────────────────────────┘
```
Each row links to the app's deploy/settings page. Shows warnings for:
- Apps with HDD data but no cross-drive backup configured
- Destinations that are unreachable/unmounted
- Last run failures
---
## Routes
### New API endpoints
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/api/stacks/{name}/cross-backup` | Yes | Save cross-drive backup config for app |
| POST | `/api/stacks/{name}/cross-backup/run` | Yes | Trigger manual run for single app |
| GET | `/api/stacks/{name}/cross-backup/status` | Yes | Get current status (for polling) |
| POST | `/api/backup/cross-drive/run-all` | Yes | Trigger all scheduled cross-drive backups |
### New web handler
| Method | Path | Description |
|--------|------|-------------|
| POST | `/settings/cross-backup/{name}` | Form POST from deploy page (redirect back) |
---
## Implementation Steps
### Step 0: CSS fix (immediate)
Add `margin-bottom: 1.5rem` to `.deploy-stale-data` in `style.css`.
### Step 1: Extend data model
**Files:** `settings.go`
- Add `CrossDriveBackup` struct
- Extend `AppBackupPrefs` with `CrossDrive` field
- Add getter/setter methods:
- `GetCrossDriveConfig(stackName string) *CrossDriveBackup`
- `SetCrossDriveConfig(stackName string, cfg CrossDriveBackup) error`
- `GetAllCrossDriveConfigs() map[string]*CrossDriveBackup`
- Add `CrossDriveResticPassword` field to Settings for the secondary restic repo
- Auto-generate password on first restic cross-drive config save
### Step 2: Cross-drive backup runner
**New file:** `internal/backup/crossdrive.go`
- `CrossDriveRunner` struct with mutex, settings, stack provider, logger
- `RunAppBackup(ctx, stackName)`:
1. Load config from settings
2. Validate destination: `IsMountPoint()` + `IsWritable()`
3. Resolve HDD mounts via `StackDataProvider`
4. Branch on method:
- rsync: `runRsyncBackup()` — runs rsync per mount with `--delete --info=progress2`
- restic: `runResticBackup()` — ensures repo init, runs `restic backup` with app tag
5. Update settings with result (last_run, last_status, etc.)
6. Log result
- `RunAllScheduled(ctx, schedule)`:
1. Iterate all apps with cross_drive enabled
2. Filter by schedule match (daily → every day, weekly → Sunday)
3. Run sequentially (not parallel — disk I/O bound)
- `GetAppStatus(stackName)` — returns latest status from settings
- `ValidateDestination(path)` — checks mount + writable + free space
**rsync specifics:**
```go
func (r *CrossDriveRunner) runRsyncBackup(stackName, destBase string, mounts []string) error {
destDir := filepath.Join(destBase, "backups", "rsync", stackName)
os.MkdirAll(destDir, 0755)
for _, srcMount := range mounts {
// Preserve directory structure relative to storage root
// e.g., /mnt/hdd_placeholder/storage/immich/ → {dest}/backups/rsync/immich/storage/immich/
relPath := strings.TrimPrefix(srcMount, filepath.Dir(filepath.Dir(srcMount)))
dstPath := filepath.Join(destDir, relPath)
os.MkdirAll(filepath.Dir(dstPath), 0755)
cmd := exec.Command("rsync", "-a", "--delete",
srcMount+"/", dstPath+"/")
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("rsync failed for %s: %v (%s)", srcMount, err, string(out))
}
}
return nil
}
```
**restic specifics:**
```go
func (r *CrossDriveRunner) runResticBackup(stackName, destBase string, mounts []string) error {
repoPath := filepath.Join(destBase, "backups", "restic")
passwordFile := r.getResticPasswordFile() // from settings or dedicated file
// Ensure initialized
if !r.isRepoInitialized(repoPath) {
cmd := exec.Command("restic", "init", "--repo", repoPath, "--password-file", passwordFile)
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("restic init failed: %v (%s)", err, string(out))
}
}
// Build args
args := []string{"backup", "--repo", repoPath, "--password-file", passwordFile,
"--tag", stackName, "--tag", "cross-drive"}
args = append(args, mounts...)
cmd := exec.Command("restic", args...)
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("restic backup failed: %v (%s)", err, string(out))
}
return nil
}
```
### Step 3: Scheduler integration
**File:** `main.go` (scheduler registration)
Add a new daily job that runs after the existing backup:
```go
// Cross-drive backup job — runs at 03:30 (after main backup at 03:00)
sched.RegisterDaily("cross_drive_backup", "03:30", func(ctx context.Context) error {
return crossDriveRunner.RunAllScheduled(ctx, "daily")
})
// Weekly cross-drive job — runs Sundays at 04:00
sched.RegisterWeekly("cross_drive_weekly", time.Sunday, "04:00", func(ctx context.Context) error {
return crossDriveRunner.RunAllScheduled(ctx, "weekly")
})
```
Note: If `RegisterWeekly` doesn't exist in the scheduler, we can check the day inside `RunAllScheduled` (like the existing `shouldPrune` pattern).
### Step 4: API endpoints
**File:** `internal/api/router.go`
Add to the switch:
```go
// POST /api/stacks/{name}/cross-backup — save config
case hasSuffix(path, "/cross-backup") && req.Method == http.MethodPost:
r.saveCrossBackupConfig(w, req, extractName(path, "/cross-backup"))
// POST /api/stacks/{name}/cross-backup/run — trigger manual run
case hasSuffix(path, "/cross-backup/run") && req.Method == http.MethodPost:
r.triggerCrossBackup(w, req, extractNameFromPath(path, "/cross-backup/run"))
// GET /api/stacks/{name}/cross-backup/status — poll status
case hasSuffix(path, "/cross-backup/status") && req.Method == http.MethodGet:
r.getCrossBackupStatus(w, req, extractNameFromPath(path, "/cross-backup/status"))
// POST /api/backup/cross-drive/run-all — trigger all
case path == "/backup/cross-drive/run-all" && req.Method == http.MethodPost:
r.triggerAllCrossBackups(w, req)
```
### Step 5: Deploy page UI
**File:** `internal/web/templates/deploy.html`
After the StorageInfo section, add the backup card for deployed apps with HDD data. The card is rendered server-side with current config values.
**File:** `internal/web/handlers.go`
In `deployHandler`, populate new template data:
```go
if alreadyDeployed {
// ... existing storageInfo ...
// Cross-drive backup config for this app
crossCfg := s.settings.GetCrossDriveConfig(name)
data["CrossDriveConfig"] = crossCfg
// Other storage paths for destination dropdown (exclude current app path)
var destPaths []DeployStoragePath
for _, sp := range s.settings.GetStoragePaths() {
if storageInfo != nil && sp.Path == storageInfo.Path {
continue // skip the app's current storage
}
dp := DeployStoragePath{StoragePath: sp}
if di := system.GetDiskUsage(sp.Path); di != nil {
dp.FreeHuman = formatFreeSpace(di.AvailGB)
dp.FreePercent = di.AvailGB / di.TotalGB * 100
}
destPaths = append(destPaths, dp)
}
data["BackupDestPaths"] = destPaths
// Destination health warning
if crossCfg != nil && crossCfg.Enabled && crossCfg.DestinationPath != "" {
if !system.IsMountPoint(crossCfg.DestinationPath) || !system.IsWritable(crossCfg.DestinationPath) {
data["BackupDestWarning"] = fmt.Sprintf(
"A cél tárhely (%s) nem elérhető! Ellenőrizd a meghajtó csatlakozását.",
crossCfg.DestinationPath,
)
}
}
// Existing nightly backup toggle state
appBackupEnabled := false
if prefs, ok := s.settings.GetAppBackupPrefs(name); ok {
appBackupEnabled = prefs.Enabled
}
data["AppBackupEnabled"] = appBackupEnabled
}
```
### Step 6: Backup page summary
**File:** `internal/web/templates/backups.html`
Add section after "Alkalmazás adatok":
```html
<!-- Section 5: Cross-drive backups -->
{{if .Backup.CrossDriveSummary}}
<div class="backup-section-card">
<h3>Másolatok másik meghajtóra</h3>
<p class="backup-section-desc">Alkalmazás adatok biztonsági másolata külső meghajtóra.</p>
{{if .Backup.CrossDriveWarnings}}
<div class="alert alert-warning" style="margin-bottom:1rem">
{{range .Backup.CrossDriveWarnings}}
<div>{{.}}</div>
{{end}}
</div>
{{end}}
<div class="cross-drive-list">
{{range .Backup.CrossDriveSummary}}
<div class="cross-drive-item">
<div class="cross-drive-header">
<a href="/stacks/{{.StackName}}/deploy" class="cross-drive-name">{{.DisplayName}}</a>
<div class="cross-drive-meta">
<span class="meta-badge">{{.MethodLabel}}</span>
<span class="meta-badge meta-badge-storage">→ {{.DestLabel}}</span>
{{if eq .LastStatus "ok"}}<span class="meta-badge meta-badge-ok">✅ {{.LastRunShort}}</span>
{{else if eq .LastStatus "error"}}<span class="meta-badge meta-badge-fail">❌ Hiba</span>
{{else}}<span class="meta-badge">⏰ {{.ScheduleLabel}}</span>{{end}}
{{if .SizeHuman}}<span class="mono" style="font-size:.8rem;color:var(--text-muted)">{{.SizeHuman}}</span>{{end}}
</div>
</div>
</div>
{{end}}
</div>
{{if .Backup.UnconfiguredApps}}
<div style="margin-top:1rem;font-size:.85rem;color:var(--yellow)">
⚠️ {{len .Backup.UnconfiguredApps}} alkalmazáshoz nincs beállítva:
{{range .Backup.UnconfiguredApps}}
<a href="/stacks/{{.StackName}}/deploy" style="color:var(--accent-blue)">{{.DisplayName}}</a>{{if not $.Last}}, {{end}}
{{end}}
</div>
{{end}}
<div class="cross-drive-actions" style="margin-top:1rem">
<button class="btn btn-sm btn-primary" onclick="triggerAllCrossDrive()">Összes futtatása most</button>
</div>
</div>
{{end}}
```
**File:** `internal/web/handlers.go` — in backup page handler
Populate the summary data:
```go
// Cross-drive backup summary
type CrossDriveSummaryItem struct {
StackName string
DisplayName string
Method string // "rsync" or "restic"
MethodLabel string // "Egyszerű másolat" or "Restic"
DestPath string
DestLabel string
Schedule string
ScheduleLabel string // "Naponta" or "Hetente"
LastStatus string
LastRunShort string
SizeHuman string
}
```
### Step 7: Destination health monitoring
**File:** `internal/web/handlers.go` or `internal/monitor/health.go`
In the periodic health check (runs every 5 min), also check cross-drive destinations:
```go
func (s *Server) checkCrossDriveDestinations() []string {
var warnings []string
configs := s.settings.GetAllCrossDriveConfigs()
seen := make(map[string]bool)
for stackName, cfg := range configs {
if !cfg.Enabled || cfg.DestinationPath == "" || seen[cfg.DestinationPath] {
continue
}
seen[cfg.DestinationPath] = true
if !system.IsMountPoint(cfg.DestinationPath) {
warnings = append(warnings,
fmt.Sprintf("A(z) %s mentési célja (%s) nincs csatlakoztatva!",
stackName, cfg.DestinationPath))
} else if !system.IsWritable(cfg.DestinationPath) {
warnings = append(warnings,
fmt.Sprintf("A(z) %s mentési célja (%s) nem írható!",
stackName, cfg.DestinationPath))
}
}
return warnings
}
```
Include warnings in both the backup page and the hub report.
---
## Safety Guards
1. **Destination ≠ Source**: Never allow backup destination to be the same storage path as the app's HDD_PATH
2. **Protected paths**: Use `ProtectedHDDPaths()` — never write to top-level dirs
3. **No parallel runs**: Mutex per app — skip if already running
4. **Free space check**: Before starting, verify destination has sufficient free space (source size × 1.1)
5. **rsync `--delete`**: Clearly warn user that rsync mirror deletes files on destination that were removed from source
6. **Restic password**: Auto-generated, stored in `settings.json`, displayed in backup page for recovery
---
## Files Summary
### New files (3)
| File | Purpose |
|------|---------|
| `internal/backup/crossdrive.go` | Cross-drive backup runner (rsync + restic) |
| `internal/backup/crossdrive_test.go` | Unit tests for path validation, config parsing |
| (templates inline changes) | |
### Modified files (9)
| File | Changes |
|------|---------|
| `internal/settings/settings.go` | `CrossDriveBackup` struct, getter/setter methods, password storage |
| `internal/backup/appdata.go` | Add `CrossDriveConfig` to `AppBackupInfo` for backup page |
| `internal/backup/backup.go` | Add `CrossDriveRunner` field, wire into `Manager` |
| `internal/api/router.go` | 4 new API routes |
| `internal/web/handlers.go` | Deploy page data + backup page cross-drive summary |
| `internal/web/server.go` | Wire `CrossDriveRunner` |
| `internal/web/templates/deploy.html` | Backup config card for deployed apps |
| `internal/web/templates/backups.html` | Cross-drive summary section |
| `internal/web/templates/style.css` | Stale data margin fix + cross-drive styles |
| `main.go` | Create runner, register scheduler jobs |
---
## Testing Checklist
### rsync method
1. Configure Immich rsync → hdd_1, daily
2. Trigger manual run → verify `/mnt/hdd_1/backups/rsync/immich/` created with data
3. Add a file to immich upload, run again → file appears in backup
4. Delete a file from immich, run again → file removed from backup (--delete)
5. Disconnect hdd_1 → warning shows on deploy page + backup page
### restic method
1. Configure Paperless restic → hdd_1, weekly
2. Trigger manual run → verify repo created at `/mnt/hdd_1/backups/restic/`
3. List snapshots: `restic -r /mnt/hdd_1/backups/restic/ snapshots --tag paperless-ngx`
4. Run again → second snapshot, dedup keeps repo small
5. Test restore: `restic -r /mnt/hdd_1/backups/restic/ restore latest --tag paperless-ngx --target /tmp/test`
### Scheduler
1. Set schedule daily → verify it runs after nightly backup
2. Set schedule weekly → verify it only runs on Sunday
3. Set schedule manual → verify it doesn't auto-run
### Destination monitoring
1. Configure backup to hdd_1 → no warnings
2. Unmount hdd_1 → warning appears on deploy + backup pages
3. Remount → warning clears
### Edge cases
1. App with no HDD data → backup card not shown
2. Only one storage path → "Másik adattároló szükséges" message
3. Source = destination → rejected with error
4. Destination full → error logged, status shows "error"
5. App migrated to new storage → backup source paths update automatically (reads from app.yaml)