diff --git a/CLAUDE.md b/CLAUDE.md index c4c9fba..87b429e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -230,6 +230,7 @@ Before ending a session, always: 1. **Commit and push** all code changes 2. **Build, push, and deploy** the new controller image (if controller code changed) -3. **Update CONTEXT.md** with what was done, decisions made, and what's next -4. **Update controller/README.md** if architecture or features changed -5. **Verify** the deployment is working (check `docker ps` and logs) \ No newline at end of file +3. **Update CHANGELOG.md** with what was done +4. **Update CONTEXT.md** with decisions made, update architectural state and what's next +5. **Update controller/README.md** if architecture or features changed +6. **Verify** the deployment is working (check `docker ps` and logs) \ No newline at end of file diff --git a/TASK.md b/TASK.md index 2e6d741..b42d91d 100644 --- a/TASK.md +++ b/TASK.md @@ -1,853 +1,686 @@ -# TASK: Phase 3 — Storage Overview & Per-App Backup Toggles +# TASK: Phase A — Storage Paths Foundation & Backup Toggle Fix -**Version target:** controller 0.8.0 -**Repos:** `deploy-felhom-compose` (controller), `app-catalog-felhom.eu` (metadata) +**Version target:** controller 0.9.0 +**Repos:** `deploy-felhom-compose` (controller), `app-catalog-felhom.eu` (compose mount change) ## Overview -Currently, restic backs up three fixed paths: the stacks directory (compose files), DB dumps, and `controller.yaml`. User data stored on the HDD (Paperless documents, media files, etc.) and Docker named volumes are **not** included. +Per-app backup toggles (implemented in v0.8.0) don't appear on the backup page. Root cause is two-layered: -Phase 3 adds: -1. **Storage overview** on the backup page — SSD, HDD, and backup repo usage at a glance -2. **Per-app backup discovery** — controller discovers each app's user data (HDD bind mounts + Docker volumes) -3. **Per-app backup toggles** — customer enables/disables backup per app on the backup page -4. **Dynamic backup paths** — `RunBackup()` includes enabled app data paths in the restic snapshot -5. **Restic password visibility** — password shown on backup page (behind toggle) + synced to hub for disaster recovery -6. **Limited app restore** — per-app restore from snapshot with warnings, self-service emergency recovery +1. **`controller.yaml` has no `paths.hdd_path`** → `m.cfg.Paths.HDDPath` is `""` → `ParseComposeHDDMounts` returns nil → `DiscoverAppData` finds zero HDD data → no backup toggles shown +2. **Even with a global hdd_path, the parser uses ONE path** → apps deployed with a different `HDD_PATH` (e.g., `/mnt/hdd_placeholder` vs `/mnt/hdd_1`) won't match + +The fix: stop relying on a single global HDD path. Instead, read each app's **own** `HDD_PATH` from its `app.yaml` env section, and introduce a storage paths registry in `settings.json` that auto-discovers paths from deployed apps. + +This phase also lays the foundation for multi-storage management (Phase B: UI polish, Phase C: migration wizard). --- -## 1. Current State +## 1. Root Cause Analysis -### What restic backs up today +### Current broken flow -```go -// In backup.go RunBackup(): -paths := []string{ - m.cfg.Paths.StacksDir, // /opt/docker/stacks (compose files + .felhom.yml) - m.cfg.Paths.DBDumpDir, // /srv/backups/db-dumps (nightly dumps) - "/opt/docker/felhom-controller/controller.yaml", -} +``` +controller.yaml → paths.hdd_path = "" (missing) + ↓ +backup.Manager.stackProvider.GetStackHDDMounts(name) + → stacks.ParseComposeHDDMounts(composePath, hddPath="") + → hddPath == "" → return nil ← BUG: exits early + ↓ +DiscoverAppData → info.HasHDDData = false for ALL apps + ↓ +backups.html → {{if .HasHDDData}} → false → no checkbox rendered ``` -### What's NOT backed up -- HDD user data (e.g., `/mnt/hdd_1/paperless/media`, `/mnt/hdd_1/romm/...`) -- Docker named volumes (e.g., `actualbudget_data`, `docmost-postgres_data`) +### Second layer (would hit even with hdd_path set) -### Existing infrastructure we can reuse -- `parseComposeHDDMounts(composePath, hddPath)` — already discovers HDD bind mount paths per stack by parsing compose files -- `GetStackHDDData(name)` — returns HDD paths + sizes for a stack -- `HDDPath` struct with `Path`, `SizeBytes`, `SizeHuman`, `Exists` -- HDD is already mounted into the controller container (read-only, fine for restic) -- `.felhom.yml` has `resources.needs_hdd: true` flag -- Monitoring page already shows SSD/HDD disk usage - -### Controller container volume mounts (relevant) -```yaml -- /opt/docker/stacks:/opt/docker/stacks # compose files -- /srv/backups:/srv/backups # restic repo + db dumps -- ${HDD_PATH}:${HDD_PATH}:ro # HDD (read-only) -- /var/run/docker.sock:/var/run/docker.sock:ro # Docker socket +``` +controller.yaml → paths.hdd_path = "/mnt/hdd_1" + ↓ +ParseComposeHDDMounts(composePath, "/mnt/hdd_1") + → reads compose: "- ${HDD_PATH}/storage/immich:/usr/src/app/upload" + → resolves ${HDD_PATH} → "/mnt/hdd_1/storage/immich" + → BUT app.yaml has HDD_PATH="/mnt/hdd_placeholder" + → actual data is at /mnt/hdd_placeholder/storage/immich + → os.Stat("/mnt/hdd_1/storage/immich") → not found + → path.Exists = false, SizeBytes = 0 ``` -HDD data is accessible from the controller container for restic reads. Docker volume paths (`/var/lib/docker/volumes/`) are NOT mounted — volume backup requires a different approach (deferred). +### Correct approach + +Read each app's actual `HDD_PATH` from its `app.yaml` env section. The compose template uses `${HDD_PATH}` which gets resolved at `docker compose up` time from the environment. We need to resolve it the same way during discovery. --- -## 2. Storage Overview (Backup Page Enhancement) +## 2. Storage Paths Registry -### 2.1 New section: "Tárhely áttekintés" (Storage Overview) - -Add as the **first section** on the backup page, above the current "Ütemezés" section. Shows storage utilization relevant to backups: - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Tárhely áttekintés │ -│ │ -│ SSD (/) ████████░░░░░░░░░░░░ 42.1 / 512.0 GB (8%) │ -│ Külső HDD ██████████████░░░░░░ 680.2 / 1000.0 GB │ -│ (68%) │ -│ │ -│ Mentési tároló 2.4 GB (/srv/backups/restic-repo) │ -│ DB mentések 142 MB (/srv/backups/db-dumps) │ -│ Pillanatképek 14 │ -└─────────────────────────────────────────────────────────────┘ -``` - -### 2.2 Data source - -Reuse `system.GetInfo()` for SSD/HDD disk stats (already available in `SystemInfo`). Add backup repo stats from existing `RepoStats`. DB dump dir size from `ListDumpFiles()`. - -No new backend code needed — just template work. The `FullBackupStatus` already contains `RepoStats` and the monitoring page already has `SystemInfo`. - -Add `SystemInfo` to the backup page template data if not already present: +### 2.1 New struct in `settings.go` ```go -type BackupPageData struct { - // existing fields... - SystemInfo *system.Info // for SSD/HDD bars +// StoragePath represents a registered external storage location. +type StoragePath struct { + Path string `json:"path"` // e.g., "/mnt/hdd_1" + Label string `json:"label,omitempty"` // e.g., "Külső HDD 1TB" + IsDefault bool `json:"is_default,omitempty"` // new apps use this by default + Schedulable bool `json:"schedulable"` // whether new apps can be deployed here + AddedAt string `json:"added_at"` // RFC3339 } ``` ---- - -## 3. Per-App Data Discovery - -### 3.1 App data classification - -For each deployed app, classify its data into three categories: - -| Category | Example | Discovery method | Backup support | -|---|---|---|---| -| **HDD bind mounts** | `/mnt/hdd_1/paperless/media` | `parseComposeHDDMounts()` (existing) | ✅ This phase | -| **Named Docker volumes** | `paperless-ngx_postgres_data` | Parse compose `volumes:` section | ⏳ Future (not mounted in controller) | -| **Config (stacks dir)** | `/opt/docker/stacks/paperless-ngx/` | Always backed up | ✅ Already done | - -### 3.2 New struct: `AppBackupInfo` - -```go -// AppBackupInfo holds backup-relevant data paths for a deployed app. -type AppBackupInfo struct { - StackName string // e.g., "paperless-ngx" - DisplayName string // e.g., "Paperless-ngx" - NeedsHDD bool // from .felhom.yml resources.needs_hdd - - // HDD bind mounts (backupable now) - HDDPaths []AppDataPath - HDDTotalSize int64 // bytes - HDDSizeHuman string - - // Docker named volumes (info only, not backupable yet) - DockerVolumes []AppDockerVolume - - // Backup state - BackupEnabled bool // from settings.json - HasHDDData bool // HDDPaths is non-empty - HasDBDump bool // app has a database container (already covered by DB dump) -} - -type AppDataPath struct { - HostPath string // e.g., "/mnt/hdd_1/paperless/media" - Exists bool - SizeHuman string - SizeBytes int64 -} - -type AppDockerVolume struct { - Name string // e.g., "paperless-ngx_postgres_data" - Contains string // Human description (from .felhom.yml, or empty) -} -``` - -### 3.3 Discovery: `DiscoverAppData()` - -New function in `internal/backup/appdata.go`: - -```go -// DiscoverAppData discovers backup-relevant data for all deployed apps. -func DiscoverAppData(stackProvider StackDataProvider, hddPath string, backupPrefs map[string]bool, discoveredDBs []DiscoveredDB) []AppBackupInfo { - var result []AppBackupInfo - - for _, stack := range stackProvider.ListDeployedStacks() { - info := AppBackupInfo{ - StackName: stack.Name, - DisplayName: stack.DisplayName, - NeedsHDD: stack.NeedsHDD, - } - - // Discover HDD bind mounts (reuse existing parser) - hddMounts := parseComposeHDDMounts(stack.ComposePath, hddPath) - for _, mount := range hddMounts { - path := AppDataPath{HostPath: mount} - if fi, err := os.Stat(mount); err == nil && fi.IsDir() { - path.Exists = true - path.SizeBytes = getDirSizeBytes(mount) - path.SizeHuman = getDirSizeHuman(mount) - } - info.HDDPaths = append(info.HDDPaths, path) - info.HDDTotalSize += path.SizeBytes - } - info.HDDSizeHuman = humanizeBytes(info.HDDTotalSize) - info.HasHDDData = len(info.HDDPaths) > 0 - - // Discover Docker named volumes (from compose file) - info.DockerVolumes = parseComposeNamedVolumes(stack.ComposePath) - - // Check if app has a DB container (already backed up via DB dump) - for _, db := range discoveredDBs { - if db.StackName == stack.Name { - info.HasDBDump = true - break - } - } - - info.BackupEnabled = backupPrefs[stack.Name] - - if info.HasHDDData || len(info.DockerVolumes) > 0 { - result = append(result, info) - } - } - - return result -} -``` - -### 3.4 Named volume parser: `parseComposeNamedVolumes()` - -```go -// parseComposeNamedVolumes extracts named Docker volumes from a docker-compose.yml. -// Parses the top-level `volumes:` key. Skips external volumes. -func parseComposeNamedVolumes(composePath string) []AppDockerVolume { - data, err := os.ReadFile(composePath) - if err != nil { return nil } - - var compose struct { - Volumes map[string]interface{} `yaml:"volumes"` - } - if err := yaml.Unmarshal(data, &compose); err != nil { return nil } - - var volumes []AppDockerVolume - for name, cfg := range compose.Volumes { - // Skip external volumes (networks like traefik-public) - if cfgMap, ok := cfg.(map[string]interface{}); ok { - if ext, ok := cfgMap["external"]; ok && ext == true { - continue - } - } - volumes = append(volumes, AppDockerVolume{Name: name}) - } - return volumes -} -``` - ---- - -## 4. Per-App Backup Toggles - -### 4.1 Storage in `settings.json` - -Add to the `Settings` struct: +### 2.2 Add to Settings struct ```go type Settings struct { // ... existing fields ... - // Per-app backup preferences - AppBackup map[string]AppBackupPrefs `json:"app_backup,omitempty"` -} - -type AppBackupPrefs struct { - Enabled bool `json:"enabled"` + // Storage paths registry + StoragePaths []StoragePath `json:"storage_paths,omitempty"` } ``` -Add getter/setter methods: +### 2.3 Helper methods on Settings ```go -func (s *Settings) IsAppBackupEnabled(stackName string) bool -func (s *Settings) SetAppBackup(stackName string, enabled bool) error -func (s *Settings) GetAppBackupMap() map[string]bool // stack_name -> enabled +// GetStoragePaths returns all registered storage paths. +func (s *Settings) GetStoragePaths() []StoragePath + +// GetDefaultStoragePath returns the default storage path, or "" if none. +func (s *Settings) GetDefaultStoragePath() string + +// GetSchedulableStoragePaths returns paths where new apps can be deployed. +func (s *Settings) GetSchedulableStoragePaths() []StoragePath + +// AddStoragePath registers a new storage path after validation. +func (s *Settings) AddStoragePath(path, label string, isDefault bool) error + +// RemoveStoragePath removes a path only if no apps reference it. +// Returns error with list of apps using this path if removal is blocked. +func (s *Settings) RemoveStoragePath(path string, appChecker func(string) []string) error + +// SetDefaultStoragePath changes which path is the default. +func (s *Settings) SetDefaultStoragePath(path string) error + +// SetSchedulable enables/disables a path for new deployments. +func (s *Settings) SetSchedulable(path string, schedulable bool) error ``` -### 4.2 Backup page: "Alkalmazás adatok" section +### 2.4 Validation rules (in AddStoragePath) -New section on the backup page, below "Tároló" (repo info), above snapshot history: +1. **Must be a real mount point** — compare device ID of path vs parent using `syscall.Stat`. Reject paths that are just directories on the boot SSD. Error message: "Ez az útvonal nem külön csatlakoztatott meghajtó. Adatok az SSD-re kerülnének." +2. **Must exist and be a directory** — `os.Stat` check +3. **Must be writable** — attempt to create + remove a temp file +4. **No overlapping paths** — reject if new path is a parent or child of an existing path (e.g., `/mnt/hdd_1` and `/mnt/hdd_1/subdir`) +5. **No duplicates** — reject if path already registered (normalized with `filepath.Clean`) +6. **Must be under `/mnt/`** — soft warning, not hard block (log a WARN for edge cases) -``` -┌─────────────────────────────────────────────────────────────┐ -│ Alkalmazás adatok │ -│ │ -│ Az alkalmazások felhasználói adatainak biztonsági mentése. │ -│ │ -│ ┌──────────────────────────────────────────────────────────┐ │ -│ │ [✅] Paperless-ngx 2.4 GB (HDD) │ │ -│ │ /mnt/hdd_1/paperless/media (2.1 GB) │ │ -│ │ /mnt/hdd_1/paperless/consume (312 MB) │ │ -│ │ 📦 Docker kötet: paperless-ngx_data (nem mentett) │ │ -│ ├──────────────────────────────────────────────────────────┤ │ -│ │ [☐] RoMM 8.7 GB (HDD) │ │ -│ │ /mnt/hdd_1/romm/library (8.5 GB) │ │ -│ │ /mnt/hdd_1/romm/assets (200 MB) │ │ -│ ├──────────────────────────────────────────────────────────┤ │ -│ │ [—] ActualBudget │ │ -│ │ 📦 Docker kötet: actualbudget_data (nem mentett) │ │ -│ │ ℹ️ Adatbázis mentés naponta (DB dump) │ │ -│ └──────────────────────────────────────────────────────────┘ │ -│ │ -│ [Mentés] │ -│ │ -│ ⚠️ Docker kötetek mentése jelenleg nem támogatott. │ -│ Az adatbázisokat az automatikus DB dump menti naponta. │ -└─────────────────────────────────────────────────────────────┘ +### 2.5 Auto-discovery on startup + +On controller startup, if `StoragePaths` is empty in settings.json, auto-discover from deployed apps: + +```go +func (s *Settings) AutoDiscoverStoragePaths(stacksDir string, fallbackHDDPath string, logger *log.Logger) { + if len(s.StoragePaths) > 0 { + return // already configured, don't override + } + + // Scan all deployed apps' app.yaml for HDD_PATH values + seen := map[string]bool{} + entries, _ := os.ReadDir(stacksDir) + for _, e := range entries { + if !e.IsDir() { continue } + appCfg := LoadAppConfig(filepath.Join(stacksDir, e.Name())) + if appCfg == nil || !appCfg.Deployed { continue } + if hddPath, ok := appCfg.Env["HDD_PATH"]; ok && hddPath != "" { + cleaned := filepath.Clean(hddPath) + if !seen[cleaned] { + seen[cleaned] = true + } + } + } + + // Also use controller.yaml paths.hdd_path as fallback seed + if fallbackHDDPath != "" { + cleaned := filepath.Clean(fallbackHDDPath) + if !seen[cleaned] { + seen[cleaned] = true + } + } + + // Register discovered paths + isFirst := true + for path := range seen { + sp := StoragePath{ + Path: path, + Label: inferStorageLabel(path), + IsDefault: isFirst, + Schedulable: true, + AddedAt: time.Now().UTC().Format(time.RFC3339), + } + s.StoragePaths = append(s.StoragePaths, sp) + isFirst = false + } + + if len(s.StoragePaths) > 0 { + s.Save() + logger.Printf("[INFO] Auto-discovered %d storage path(s)", len(s.StoragePaths)) + for _, sp := range s.StoragePaths { + logger.Printf("[INFO] %s (%s) default=%v", sp.Path, sp.Label, sp.IsDefault) + } + } +} ``` -**UI behavior:** -- Toggle checkbox per app — only enabled for apps with HDD data -- Apps with only Docker volumes show info state (no checkbox) -- Apps with databases show "DB dump naponta" note -- "Mentés" button saves all toggles at once via `POST /settings/app-backup` -- Sizes update on page refresh (from the cached `RefreshCache`) +Helper `inferStorageLabel(path)`: +- `/mnt/hdd_1` → "Külső tárhely (hdd_1)" +- `/mnt/hdd_placeholder` → "Külső tárhely (hdd_placeholder)" +- Anything else → path basename -### 4.3 Route +Note: `LoadAppConfig` is in the `stacks` package. To avoid circular imports, either pass a scanner function or do the scanning in `main.go` and pass the discovered paths in. Use whichever approach avoids import issues. + +--- + +## 3. Fix `GetStackHDDMounts` — Per-App HDD_PATH Resolution + +### 3.1 Change the adapter + +Currently: +```go +func (a *stackAdapter) GetStackHDDMounts(name string) []string { + s, ok := a.mgr.GetStack(name) + if !ok { return nil } + return stacks.ParseComposeHDDMounts(s.ComposePath, a.hddPath) // ← uses global path +} +``` + +Change to: +```go +func (a *stackAdapter) GetStackHDDMounts(name string) []string { + s, ok := a.mgr.GetStack(name) + if !ok { return nil } + + // Priority 1: Read the app's own HDD_PATH from its app.yaml + stackDir := filepath.Dir(s.ComposePath) + appCfg := stacks.LoadAppConfig(stackDir) + if appCfg != nil && appCfg.Env["HDD_PATH"] != "" { + return stacks.ParseComposeHDDMounts(s.ComposePath, appCfg.Env["HDD_PATH"]) + } + + // Priority 2: Try all registered storage paths (fallback) + var allMounts []string + seen := map[string]bool{} + for _, sp := range a.getStoragePaths() { + mounts := stacks.ParseComposeHDDMounts(s.ComposePath, sp.Path) + for _, m := range mounts { + if !seen[m] { + seen[m] = true + allMounts = append(allMounts, m) + } + } + } + return allMounts +} +``` + +### 3.2 Update adapter struct + +```go +type stackAdapter struct { + mgr *stacks.Manager + getStoragePaths func() []settings.StoragePath // getter from settings +} +``` + +Wire in `main.go`: +```go +adapter := &stackAdapter{ + mgr: stackMgr, + getStoragePaths: func() []settings.StoragePath { return settingsMgr.GetStoragePaths() }, +} +``` + +Remove the old `hddPath string` field from the adapter. + +### 3.3 `resolveAppBackupPaths()` in backup Manager + +No changes needed — it already calls `m.stackProvider.GetStackHDDMounts(stackName)` which we're fixing above. The fix propagates automatically. + +--- + +## 4. Controller Docker Compose Mount Change + +### 4.1 Change in `controller/docker-compose.yml` + +Replace: +```yaml + - ${HDD_PATH:-/mnt/hdd_placeholder}:${HDD_PATH:-/mnt/hdd_placeholder}:ro +``` + +With: +```yaml + # All external storage — covers /mnt/* paths for multi-storage support + restore + - /mnt:/mnt:rw +``` + +**Why `:rw`:** Required for restore feature (already implemented). Also needed for mount-point validation (write test in AddStoragePath). + +**Why `/mnt:/mnt`:** All standard storage mounts are under `/mnt`. Covers current and future drives with one entry. No controller restart when adding new drives. + +### 4.2 Also update `docker-setup.sh` + +If `docker-setup.sh` generates the controller compose, update it to emit `/mnt:/mnt:rw` instead of the old `${HDD_PATH}` line. + +--- + +## 5. Deploy Page: Storage Path Dropdown + +### 5.1 When app has a `path` type deploy field (HDD_PATH) + +Currently renders as a plain text input. Change to dropdown when storage paths exist: + +```html +{{if eq .Type "path"}} + {{if $.StoragePaths}} + + {{else}} + + + Nincs regisztrált adattároló. Adjon hozzá egyet a Beállítások oldalon. + + {{end}} +{{end}} +``` + +### 5.2 Pass storage paths to deploy page template data + +In the deploy handler: +```go +data["StoragePaths"] = settingsMgr.GetSchedulableStoragePaths() +``` + +### 5.3 Already-deployed: show current path read-only + +When `AlreadyDeployed == true`, the field is disabled. Show the actual `HDD_PATH` from `app.yaml` as the displayed value (existing behavior works as-is with a disabled select or text input). + +### 5.4 Edge case: no schedulable paths but app needs HDD + +If `resources.needs_hdd: true` and zero schedulable paths: +- Show warning: "Nincs elérhető adattároló. Csatlakoztasson külső meghajtót és adja hozzá a Beállítások oldalon." +- Disable deploy button (add `data-needs-storage="true"` to form, JS disables submit) + +--- + +## 6. Storage Path Monitoring Integration + +### 6.1 Periodic mount-point check + +Add to existing system health check (runs every `system_health_interval`, default 5m): + +```go +func checkStoragePaths(paths []settings.StoragePath, logger *log.Logger) []string { + var warnings []string + for _, sp := range paths { + // Check 1: path exists and is directory + fi, err := os.Stat(sp.Path) + if err != nil { + warnings = append(warnings, fmt.Sprintf("Adattároló nem elérhető: %s", sp.Path)) + continue + } + if !fi.IsDir() { + warnings = append(warnings, fmt.Sprintf("Adattároló nem mappa: %s", sp.Path)) + continue + } + + // Check 2: still a real mount point (not writing to SSD) + if !isMountPoint(sp.Path) { + warnings = append(warnings, + fmt.Sprintf("FIGYELEM: %s nincs felcsatolva (leválasztva?) — "+ + "az adatok az SSD-re íródnának!", sp.Path)) + } + + // Check 3: disk space (reuse existing threshold config) + usage := getDiskUsage(sp.Path) + if usage.UsedPercent >= 90 { + warnings = append(warnings, + fmt.Sprintf("Adattároló majdnem tele: %s (%.0f%%)", sp.Path, usage.UsedPercent)) + } + } + return warnings +} +``` + +### 6.2 `isMountPoint()` implementation + +New file: `controller/internal/system/mounts.go` + +```go +// IsMountPoint checks if a path is a mount point by comparing device IDs. +// Returns true if path is on a different device than its parent. +func IsMountPoint(path string) bool { + var pathStat, parentStat syscall.Stat_t + if err := syscall.Stat(path, &pathStat); err != nil { + return false + } + parent := filepath.Dir(path) + if err := syscall.Stat(parent, &parentStat); err != nil { + return false + } + return pathStat.Dev != parentStat.Dev +} +``` + +Platform-specific: needs build tag `//go:build linux` (consistent with existing system package). Add stub for non-Linux that always returns true. + +### 6.3 Surface warnings + +- Add storage warnings to monitoring page warnings section (existing infrastructure) +- Include in hub report `SystemWarnings` field (already exists) +- Fire `disk_warning` notification event for enabled customers + +--- + +## 7. Beállítások Page: Storage Section + +Add between "Rendszer konfiguráció" and "Jelszó módosítás" sections. + +### 7.1 Template: "Adattárolók" section + +```html + +
+

Adattárolók

+

+ Külső meghajtók, ahol az alkalmazások felhasználói adatai tárolódnak. +

+ + {{if .StorageError}}
{{.StorageError}}
{{end}} + {{if .StorageSuccess}}
{{.StorageSuccess}}
{{end}} + + {{if .StoragePaths}} +
+ {{range .StoragePaths}} +
+
+
+ {{.Label}} + {{.Path}} +
+
+ {{if .IsDefault}}alapértelmezett{{end}} + {{if not .Schedulable}}nem ütemezhet{{end}} + {{if .DiskInfo}} + + {{.DiskInfo.UsedHuman}} / {{.DiskInfo.TotalHuman}} ({{.DiskInfo.UsedPercent}}%) + + {{end}} +
+
+ {{if .AppCount}} +
+ {{.AppCount}} alkalmazás használja +
+ {{end}} +
+ {{if not .IsDefault}} +
+ + +
+ {{end}} +
+ + + +
+ {{if and (not .IsDefault) (eq .AppCount 0)}} +
+ + +
+ {{end}} +
+
+ {{end}} +
+ {{else}} +
+ Nincs regisztrált adattároló. Csatlakoztasson külső meghajtót és adja hozzá alább. +
+ {{end}} + + +
+ + Új adattároló hozzáadása +
+
+ + + A felcsatolt meghajtó elérési útja (pl. /mnt/hdd_1) +
+
+ + +
+
+ +
+ +
+
+
+``` + +### 7.2 Routes | Method | Path | Auth? | Handler | |--------|------|-------|---------| -| POST | `/settings/app-backup` | Yes | Save per-app backup toggles | +| POST | `/settings/storage/add` | Yes | Add storage path with validation | +| POST | `/settings/storage/remove` | Yes | Remove (blocked if apps use it) | +| POST | `/settings/storage/default` | Yes | Set default storage path | +| POST | `/settings/storage/schedulable` | Yes | Toggle schedulable on/off | -**Handler:** -1. Parse form: checkboxes named `backup_{stack_name}` (checked = enabled) -2. For each app with HDD data: set enabled/disabled in settings -3. Save to `settings.json` -4. Refresh backup cache -5. Redirect to `/backups` with success flash: "Alkalmazás mentési beállítások mentve." +### 7.3 Handler: `/settings/storage/add` + +1. Parse form: `storage_path`, `storage_label`, `storage_default` +2. Clean path with `filepath.Clean` +3. Validate (Section 2.4): + - Exists + is directory → error: "Az útvonal nem létezik vagy nem mappa." + - Is mount point → error: "Ez az útvonal nem külön csatlakoztatott meghajtó. Adatok az SSD-re kerülnének!" + - Writable → error: "Az útvonal nem írható." + - No overlap → error: "Az útvonal átfedi a már regisztrált XYZ útvonalat." + - No duplicate → error: "Ez az útvonal már regisztrálva van." +4. If `storage_default`, unset previous default +5. Add to settings.json via `AddStoragePath()` +6. Redirect to `/settings` with flash: "Adattároló sikeresen hozzáadva: /mnt/hdd_1" +7. On error: redirect with error flash + +### 7.4 Handler: `/settings/storage/remove` + +1. Parse form: `storage_path` +2. Count apps using this path (scan app.yaml files for `HDD_PATH`) +3. If apps found → error: "Nem törölhető: az alábbi alkalmazások használják: Immich, Paperless-ngx" +4. Cannot be the default path → error: "Az alapértelmezett adattároló nem törölhető." +5. If only one path left → error: "Az utolsó adattároló nem törölhető." +6. Remove from settings.json +7. Redirect with success flash + +### 7.5 Template data for settings page + +Add to the settings handler: +```go +type StoragePathView struct { + settings.StoragePath + DiskInfo *DiskUsageInfo // total, used, percent + AppCount int // number of deployed apps with HDD_PATH matching this + IsMounted bool // mount-point check result +} +``` + +Populate by scanning `app.yaml` files for `HDD_PATH` and matching against registered paths. --- -## 5. Dynamic Backup Paths in RunBackup +## 8. Deprecate `paths.hdd_path` from controller.yaml -### 5.1 Modify `RunBackup()` in `backup.go` +### 8.1 Backward compatibility +Keep reading `paths.hdd_path` — use it only as fallback seed for auto-discovery on first run. + +Startup order in `main.go`: ```go -// Base paths (always backed up) -paths := []string{ - m.cfg.Paths.StacksDir, - m.cfg.Paths.DBDumpDir, - "/opt/docker/felhom-controller/controller.yaml", -} +// 1. Load settings.json +settingsMgr, _ := settings.Load(...) -// Per-app HDD data paths (from settings) -appPaths := m.resolveAppBackupPaths() -paths = append(paths, appPaths...) +// 2. Auto-discover storage paths (if settings.json has none) +settingsMgr.AutoDiscoverStoragePaths(cfg.Paths.StacksDir, cfg.Paths.HDDPath, logger) -m.logger.Printf("[INFO] Backup paths (%d total): %v", len(paths), paths) -``` - -### 5.2 Path resolver - -```go -func (m *Manager) resolveAppBackupPaths() []string { - appBackupMap := m.settings.GetAppBackupMap() - if len(appBackupMap) == 0 { - return nil - } - - var paths []string - seen := make(map[string]bool) - - for stackName, enabled := range appBackupMap { - if !enabled { - continue - } - composePath, ok := m.stackProvider.GetStackComposePath(stackName) - if !ok { - m.logger.Printf("[WARN] App backup enabled for %s but stack not found", stackName) - continue - } - hddMounts := parseComposeHDDMounts(composePath, m.cfg.Paths.HDDPath) - for _, mount := range hddMounts { - if seen[mount] { - continue - } - if _, err := os.Stat(mount); err == nil { - paths = append(paths, mount) - seen[mount] = true - m.logger.Printf("[DEBUG] Including app data: %s (from %s)", mount, stackName) - } - } - } - return paths +// 3. Wire adapter with storage paths getter +adapter := &stackAdapter{ + mgr: stackMgr, + getStoragePaths: func() []settings.StoragePath { return settingsMgr.GetStoragePaths() }, } ``` -### 5.3 Stack data provider interface - -To avoid circular imports between `backup` and `stacks` packages: - -```go -// In backup package -type StackDataProvider interface { - GetStackComposePath(name string) (composePath string, ok bool) - ListDeployedStacks() []StackSummary -} - -type StackSummary struct { - Name string - DisplayName string - ComposePath string - NeedsHDD bool -} -``` - -Implement this interface with a thin wrapper in `main.go` or as an adapter: - -```go -// In main.go -type stackAdapter struct{ mgr *stacks.Manager } - -func (a *stackAdapter) GetStackComposePath(name string) (string, bool) { - s, ok := a.mgr.GetStack(name) - if !ok { return "", false } - return s.ComposePath, true -} -func (a *stackAdapter) ListDeployedStacks() []backup.StackSummary { ... } -``` - -### 5.4 Update `FullBackupStatus` - -```go -type FullBackupStatus struct { - // ... existing ... - AppDataInfo []AppBackupInfo // per-app backup info for backup page - AppDataPaths []string // resolved app data paths (for "Mentett útvonalak" display) - AppDataSizeHuman string // total size of enabled app data -} -``` - -Populate `AppDataInfo` during `RefreshCache()` (runs periodically). - ---- - -## 6. App Catalog Metadata Enhancement (Optional) - -### 6.1 Add `backup` section to `.felhom.yml` - -While the controller can discover HDD paths from compose files, explicit metadata gives better descriptions: +### 8.2 Update configs/controller.yaml.example ```yaml -# In .felhom.yml for paperless-ngx -backup: - description: "Dokumentumok, beolvasott fájlok és feldolgozási queue" - data_paths: - - "${HDD_PATH}/paperless/media" - - "${HDD_PATH}/paperless/consume" - - "${HDD_PATH}/paperless/export" - docker_volumes: - - name: "paperless-ngx_data" - contains: "Alkalmazás belső adatai" - - name: "paperless-ngx_postgres_data" - contains: "Adatbázis (DB dump menti)" +paths: + stacks_dir: "/opt/docker/stacks" + # hdd_path is DEPRECATED — storage paths are managed via web UI (Beállítások > Adattárolók) + # Existing value is auto-migrated to settings.json on first startup + # hdd_path: "/mnt/hdd_1" ``` -**Optional** — the controller falls back to `parseComposeHDDMounts()` auto-discovery if the `backup` section doesn't exist. - -### 6.2 Update `Metadata` struct - -```go -type Metadata struct { - // ... existing fields ... - Backup BackupMetadata `yaml:"backup" json:"backup"` -} - -type BackupMetadata struct { - Description string `yaml:"description" json:"description"` - DataPaths []string `yaml:"data_paths" json:"data_paths"` - DockerVolumes []VolumeDescription `yaml:"docker_volumes" json:"docker_volumes"` -} -``` - -### 6.3 Discovery priority - -1. If `.felhom.yml` has `backup.data_paths` → use those (resolve `${HDD_PATH}`) -2. Else → fall back to `parseComposeHDDMounts()` auto-discovery -3. Docker volumes: merge `.felhom.yml` descriptions with parsed compose volumes - --- -## 7. Restic Password Visibility +## 9. Implementation Steps -### 7.1 Problem +### Step 1: Storage paths in settings.json +- Add `StoragePath` struct and `StoragePaths` field to `Settings` +- Add all getter/setter methods +- Add `AutoDiscoverStoragePaths()` — note: may need to pass a scanner function or do scanning in main.go to avoid circular import with stacks package +- Add `inferStorageLabel()` helper +- **Test:** Manually create a `settings.json` with `storage_paths` → Load → verify GetStoragePaths returns them -The restic password is auto-generated on first backup and stored at `/opt/docker/felhom-controller/data/restic-password` inside the `controller-data` Docker named volume. If the SSD dies, that password is gone and ALL backup snapshots become permanently inaccessible. The customer currently has zero visibility into this password. +### Step 2: Mount-point validation utilities +- Create `controller/internal/system/mounts.go` with `IsMountPoint()` (Linux + stub) +- Add overlap check helper: `PathsOverlap(a, b string) bool` +- Add writable check helper: `IsWritable(path string) bool` +- **Test:** `IsMountPoint("/mnt/hdd_1")` → true, `IsMountPoint("/tmp")` → false -### 7.2 Password display on backup page +### Step 3: Wire auto-discovery on startup +- In `main.go`: call `AutoDiscoverStoragePaths` after loading settings +- Pass `cfg.Paths.HDDPath` as fallback +- **Test:** Start controller on demo-felhom with empty settings.json → should auto-discover `/mnt/hdd_placeholder` from deployed apps' app.yaml -Add a "Titkosítási kulcs" (Encryption Key) section on the backup page, within the storage/repo area: +### Step 4: Fix per-app HDD_PATH resolution (THE CORE FIX) +- Update `stackAdapter` struct: replace `hddPath string` with `getStoragePaths func()` +- Update `GetStackHDDMounts()` to read per-app HDD_PATH from app.yaml first, fallback to registered paths +- Wire updated adapter in `main.go` +- **Test:** Backup page → "Alkalmazás adatok" section now shows with correct paths (e.g., `/mnt/hdd_placeholder/storage/immich`). Backup toggles are visible and functional. -``` -┌─────────────────────────────────────────────────────────────┐ -│ Titkosítási kulcs │ -│ │ -│ A mentések titkosítva vannak. A visszaállításhoz szükség │ -│ van erre a kulcsra. │ -│ │ -│ [••••••••••••••••••••••] [👁 Megjelenítés] [📋 Másolás] │ -│ │ -│ ⚠️ Mentse el biztonságos helyre! A kulcs nélkül a │ -│ biztonsági mentések NEM állíthatók vissza. │ -└─────────────────────────────────────────────────────────────┘ -``` +### Step 5: Controller compose mount change +- Change `controller/docker-compose.yml`: `/mnt:/mnt:rw` +- Update `docker-setup.sh` controller compose generation +- **Test:** Recreate controller container → verify both `/mnt/hdd_1` and `/mnt/hdd_placeholder` accessible inside container -**Implementation:** +### Step 6: Beállítások storage section +- Add "Adattárolók" section to `settings.html` +- Add handlers: `POST /settings/storage/add`, `/remove`, `/default`, `/schedulable` +- Add `StoragePathView` type for template data with disk info + app count +- Pass storage data to settings page handler +- Add CSS for storage path items +- **Test:** Add path via UI → validation catches non-mount-point. Remove path with apps → blocked with app list. Set default → badge updates. -```go -// Add to ResticManager -func (r *ResticManager) GetPassword() (string, error) { - data, err := os.ReadFile(r.passwordFile) - if err != nil { - return "", fmt.Errorf("reading restic password: %w", err) - } - return strings.TrimSpace(string(data)), nil -} -``` +### Step 7: Deploy page dropdown +- Modify `deploy.html`: path field becomes dropdown when storage paths exist +- Pass `StoragePaths` to deploy page template data +- Fall back to text input if no paths registered +- Block deploy if app needs HDD but no schedulable paths +- **Test:** Deploy page for Immich → dropdown shows registered paths. Already-deployed shows current path read-only. -**Template behavior:** -- Password field is masked by default (`type="password"`) -- "Megjelenítés" button toggles visibility (JS, `type="text"`) -- "Másolás" button copies to clipboard (JS, `navigator.clipboard.writeText()`) -- Password is loaded in the template data (served over existing auth-protected page) +### Step 8: Storage monitoring integration +- Add `checkStoragePaths()` to system health check +- Surface warnings on monitoring page +- Include in hub report +- Fire `disk_warning` notification for unmounted drives +- **Test:** (Simulated) If path doesn't exist → warning appears on monitoring page within one health check cycle -### 7.3 Password sync to hub - -Add password to the periodic hub report so the operator has a backup copy: - -```go -// In the hub report payload (already sent every 15m) -type ReportPayload struct { - // ... existing fields ... - ResticPassword string `json:"restic_password,omitempty"` // encrypted key for recovery -} -``` - -**Hub side** — store the password in the customer record. The hub is operator-controlled and already stores customer metadata. Add a `restic_password` field to the hub database for each customer. - -**Sync trigger:** -- On every periodic report (piggyback on existing 15-minute push) -- On startup (controller init) - -**Security note:** The hub API is already authenticated with `Bearer` token. The password travels over HTTPS (Cloudflare Tunnel). This is safer than the password being in a single Docker named volume with no redundancy. - ---- - -## 8. Limited App Restore (Self-Service Emergency) - -### 8.1 Approach - -Provide a per-app "Visszaállítás" (Restore) feature on the backup page that restores an entire app's HDD data from a selected snapshot. This is a self-service emergency tool — customers can recover from accidental deletions without waiting for support. - -**Scope:** Only restore HDD bind mount data for apps that have backup enabled. Compose files and controller.yaml are always restorable (they're in every snapshot). - -### 8.2 UI: Restore section on backup page - -Add a "Visszaállítás" section below the snapshot history: - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Visszaállítás │ -│ │ -│ Alkalmazás: [▾ Paperless-ngx ] │ -│ Pillanatkép: [▾ 2026-02-16 03:00 (legutóbbi) ] │ -│ │ -│ Visszaállítandó útvonalak: │ -│ /mnt/hdd_1/paperless/media │ -│ /mnt/hdd_1/paperless/consume │ -│ │ -│ ⚠️ FIGYELMEZTETÉS │ -│ A visszaállítás FELÜLÍRJA a kiválasztott alkalmazás │ -│ jelenlegi adatait a mentés pillanatának állapotával. │ -│ Ez a művelet NEM vonható vissza! │ -│ │ -│ Javasoljuk az alkalmazás leállítását a visszaállítás előtt. │ -│ │ -│ [☐] Megértettem, visszaállítás saját felelősségre. │ -│ │ -│ [🔄 Visszaállítás indítása] (disabled until checkbox) │ -└─────────────────────────────────────────────────────────────┘ -``` - -### 8.3 Restore flow - -1. User selects app + snapshot from dropdowns -2. JS fetches app's HDD paths (already in `AppBackupInfo`) and shows them -3. User checks the "saját felelősségre" checkbox — this enables the button -4. `POST /backup/restore` with `stack_name`, `snapshot_id` -5. Handler validates: - - Stack exists and has backup enabled - - Snapshot ID exists in restic repo - - HDD paths are valid -6. Controller runs `restic restore` for the specific paths -7. Redirect to backup page with result flash message - -### 8.4 Backend: `RestoreAppData()` - -```go -// In ResticManager -func (r *ResticManager) RestoreAppData(snapshotID string, paths []string) error { - // Build --include flags for each HDD path - args := []string{ - "restore", snapshotID, - "--target", "/", // restore to original absolute paths - "--password-file", r.passwordFile, - "--repo", r.repoPath, - "--cache-dir", r.cacheDir, - } - for _, p := range paths { - args = append(args, "--include", p) - } - - r.logger.Printf("[WARN] RESTORE started: snapshot=%s, paths=%v", snapshotID, paths) - - cmd := exec.Command("restic", args...) - output, err := cmd.CombinedOutput() - if err != nil { - r.logger.Printf("[ERROR] Restore failed: %v, output: %s", err, string(output)) - return fmt.Errorf("restic restore failed: %w", err) - } - - r.logger.Printf("[INFO] RESTORE completed: snapshot=%s, paths=%v", snapshotID, paths) - return nil -} -``` - -### 8.5 Backup Manager: `RestoreApp()` - -```go -func (m *Manager) RestoreApp(stackName, snapshotID string) error { - // Validate app has backup enabled - if !m.settings.IsAppBackupEnabled(stackName) { - return fmt.Errorf("backup not enabled for %s", stackName) - } - - // Resolve HDD paths for this app - composePath, ok := m.stackProvider.GetStackComposePath(stackName) - if !ok { - return fmt.Errorf("stack %s not found", stackName) - } - hddMounts := parseComposeHDDMounts(composePath, m.cfg.Paths.HDDPath) - if len(hddMounts) == 0 { - return fmt.Errorf("no HDD data paths found for %s", stackName) - } - - // Validate snapshot exists - snapshots, err := m.restic.ListSnapshots(100) - if err != nil { - return fmt.Errorf("listing snapshots: %w", err) - } - found := false - for _, s := range snapshots { - if s.ID == snapshotID { - found = true - break - } - } - if !found { - return fmt.Errorf("snapshot %s not found", snapshotID) - } - - // Send notification before restore - m.notify("restore_started", fmt.Sprintf("Visszaállítás indult: %s (snapshot: %s)", stackName, snapshotID)) - - // Execute restore - if err := m.restic.RestoreAppData(snapshotID, hddMounts); err != nil { - m.notify("restore_failed", fmt.Sprintf("Visszaállítás sikertelen: %s — %v", stackName, err)) - return err - } - - m.notify("restore_completed", fmt.Sprintf("Visszaállítás kész: %s (snapshot: %s)", stackName, snapshotID)) - return nil -} -``` - -### 8.6 Route - -| Method | Path | Auth? | Handler | -|--------|------|-------|---------| -| POST | `/backup/restore` | Yes | Restore app HDD data from snapshot | -| GET | `/api/backup/snapshots` | Yes | List snapshots (JSON, for restore dropdown) | - -**Restore handler:** -1. Parse form: `stack_name`, `snapshot_id` -2. Validate both fields present -3. Call `backupManager.RestoreApp(stackName, snapshotID)` -4. On success: redirect to `/backups` with flash "✅ {app} visszaállítva ({snapshot})." -5. On error: redirect to `/backups` with error flash "❌ Visszaállítás sikertelen: {error}" - -**Snapshots JSON handler** (for dynamic dropdown population): -```go -func (s *Server) apiBackupSnapshotsHandler(w http.ResponseWriter, r *http.Request) { - snapshots, err := s.backupManager.ListSnapshots(50) - // return as JSON array -} -``` - -### 8.7 Important constraints - -- **HDD must NOT be read-only for restore.** The controller currently mounts HDD as `:ro`. For restore to work, the mount must be `:rw` OR a separate restore path must be used. - - Change HDD mount to `:rw` (simpler, but less safe for normal operation) - -- **Restore only for apps with enabled backup** — if the user never enabled backup for an app, there's no data to restore. - -- **No concurrent restore + backup** — add a mutex/lock to prevent backup and restore running simultaneously. - -- **Notification events:** Add `restore_started`, `restore_completed`, `restore_failed` event types to the notification system. - ---- - -## 9. Implementation Order - -### Step 1: Storage overview on backup page -- Add `SystemInfo` to backup page template data -- Add "Tárhely áttekintés" section to `backups.html` with SSD/HDD bars + backup repo stats -- Reuse existing CSS for storage bars (from monitoring page) -- **Test:** Backup page shows storage usage bars and repo stats - -### Step 2: Restic password visibility -- Add `GetPassword()` to ResticManager -- Add "Titkosítási kulcs" section to `backups.html` with masked password, show/copy buttons -- Add password to hub report payload (`restic_password` field) -- Hub side: store password in customer record (new DB column or JSON field) -- **Test:** Password visible on backup page → copy works → hub DB has password after report cycle - -### Step 3: App data discovery + settings struct -- Create `internal/backup/appdata.go` with `AppBackupInfo`, `DiscoverAppData()`, `parseComposeNamedVolumes()` -- Define `StackDataProvider` interface in backup package -- Add `AppBackupPrefs` + getter/setter to settings.go -- Create stack adapter in main.go -- **Test:** Call `DiscoverAppData()`, verify it finds paperless HDD paths - -### Step 4: Per-app toggles on backup page -- Add "Alkalmazás adatok" section to `backups.html` with toggle checkboxes per app -- Add `POST /settings/app-backup` handler -- Include `AppDataInfo` in backup page template data (populate during `RefreshCache`) -- **Test:** Toggle paperless backup on → save → check settings.json → page shows updated state - -### Step 5: Dynamic backup paths in RunBackup -- Add `resolveAppBackupPaths()` to backup Manager -- Modify `RunBackup()` to include enabled app HDD paths -- Update `BackupPaths` display in `FullBackupStatus` -- **Test:** Enable paperless backup → trigger manual backup → verify restic snapshot includes HDD data paths via `restic snapshots --json` - -### Step 6: Limited app restore -- Change HDD container mount from `:ro` to `:rw` in controller docker-compose.yml -- Add `RestoreAppData()` to ResticManager -- Add `RestoreApp()` to backup Manager with validation and notifications -- Add `POST /backup/restore` handler and `GET /api/backup/snapshots` JSON endpoint -- Add "Visszaállítás" section to `backups.html` with app/snapshot dropdowns, warnings, confirmation checkbox -- Add restore notification events (`restore_started`, `restore_completed`, `restore_failed`) -- Add restore mutex to prevent concurrent backup/restore operations -- **Test:** Enable paperless backup → backup → delete test file → restore → verify file is back - -### Step 7: Metadata enhancement (optional) -- Add `backup` section to `.felhom.yml` for relevant apps in app-catalog repo -- Update `Metadata` struct in `metadata.go` -- Update discovery to prefer metadata over compose parsing -- **Test:** Apps with `backup.data_paths` show correct descriptions and paths - -### Step 8: Cleanup & version bump -- Update CONTEXT.md / CHANGELOG -- Bump to 0.8.0 +### Step 9: Cleanup & version bump +- Deprecate `paths.hdd_path` in controller.yaml.example +- Create new CHANGELOG.md, changelogs will be updated there, not in CONTEXT.md. CONTEXT.md will only have information about architecture decisions, roadmap, information about the project +- Regenerate CONTEXT.md with current architecture, deparating different sections, with detailed descriptions how it should work, what is planned, what is the architecture. Sections like: Storage management, Notifications, Backup management, App management, Monitoring, Infra, Settings management, etc-etc.. +- Update CONTEXT.md / CHANGELOG.md / CLAUDE.md +- Bump to 0.9.0 - Build + deploy -- **Test end-to-end:** Full cycle — enable app backup → nightly backup runs → restore works → password visible and synced to hub +- **Test end-to-end:** Auto-discover → backup toggles visible → enable Immich backup → manual backup → verify HDD data in restic snapshot → storage management in Beállítások works --- ## 10. Files to Create / Modify ### New files: -- `controller/internal/backup/appdata.go` — `AppBackupInfo`, `DiscoverAppData()`, `parseComposeNamedVolumes()`, `StackDataProvider` interface -- `controller/internal/backup/restore.go` — `RestoreAppData()`, `RestoreApp()`, restore mutex +- `controller/internal/system/mounts.go` — `IsMountPoint()`, `IsWritable()`, `PathsOverlap()` + non-Linux stub +- (No new packages — StoragePath goes in existing `settings` package) ### Modified files: -- `controller/internal/backup/backup.go` — `RunBackup()` uses dynamic paths, `RefreshCache()` builds `AppDataInfo`, add `StackDataProvider` field + `resolveAppBackupPaths()`, restore mutex integration -- `controller/internal/backup/restic.go` — Add `GetPassword()`, `RestoreAppData()` methods to ResticManager -- `controller/internal/settings/settings.go` — `AppBackupPrefs` struct, getter/setter methods -- `controller/internal/web/handlers.go` — `POST /settings/app-backup` handler, `POST /backup/restore` handler, `GET /api/backup/snapshots` JSON endpoint, pass `AppDataInfo` + password to backup page -- `controller/internal/web/templates/backups.html` — "Tárhely áttekintés" section, "Titkosítási kulcs" section, "Alkalmazás adatok" section, "Visszaállítás" section -- `controller/internal/web/templates/style.css` — Styles for app backup cards, toggle rows, restore section, password field -- `controller/internal/notify/notifier.go` — Add `restore_started`, `restore_completed`, `restore_failed` event types -- `controller/internal/stacks/metadata.go` — Add `BackupMetadata` to `Metadata` struct (optional) -- `controller/cmd/controller/main.go` — Create stack adapter, wire into backup manager -- Controller `docker-compose.yml` — Change HDD mount from `:ro` to `:rw` -- Hub report payload struct — Add `restic_password` field -- Hub API/DB — Store `restic_password` per customer (hub-side change) - -### App catalog (optional): -- `paperless-ngx/.felhom.yml` — Add `backup` section with data paths and volume descriptions -- Other apps with HDD data as applicable +- `controller/internal/settings/settings.go` — `StoragePath` struct, `StoragePaths` field, all getter/setter methods, `AutoDiscoverStoragePaths()`, `inferStorageLabel()`, validation logic +- `controller/cmd/controller/main.go` — Wire auto-discovery, update stackAdapter struct (replace `hddPath` with `getStoragePaths`), update adapter constructor +- `controller/internal/web/handlers.go` — Storage management handlers (`/settings/storage/*`), pass StoragePaths to deploy + settings templates +- `controller/internal/web/server.go` — Register new storage routes +- `controller/internal/web/templates/settings.html` — New "Adattárolók" section +- `controller/internal/web/templates/deploy.html` — Path field → dropdown, edge case warnings +- `controller/internal/web/templates/style.css` — Storage path item styles, badges +- `controller/internal/monitoring/health.go` (or wherever health checks live) — Add `checkStoragePaths()` +- `controller/docker-compose.yml` — `/mnt:/mnt:rw` +- `controller/configs/controller.yaml.example` — Deprecation comment +- `docker-setup.sh` — Update controller compose generation --- ## 11. Design Decisions & Notes -### Why only HDD data in this phase? +### Why settings.json, not controller.yaml? +controller.yaml is operator-controlled (read-only from customer perspective). User-configurable state lives in settings.json. Clear separation: operator configures infrastructure, customer configures preferences. -Docker named volumes are stored at `/var/lib/docker/volumes/` which is NOT mounted into the controller container. Backing them up would require either: -- Mounting `/var/lib/docker/volumes` into the controller (security concern, large mount) -- Running restic via a temporary Docker container (complex orchestration) -- Using `docker cp` to export data (slow, no incremental) +### Why auto-discover from app.yaml? +Existing deployments already have apps with HDD_PATH set. Auto-discovery makes the upgrade seamless — backup toggles appear without manual intervention after the update. -For most apps, the important user data is on HDD (documents, media, photos). Database data in named volumes is covered by the nightly DB dump. Config-only volumes are small and recoverable from compose + deploy fields. +### Why `/mnt:/mnt:rw` instead of individual mounts? +Single mount covers all current and future storage devices. No controller restart when adding new drives. Required for restore + validation. Trade-off: broader access, but controller already has Docker socket access and is a trusted component. -### Why reuse `parseComposeHDDMounts` over explicit metadata? +### Why validate mount points? +If a USB drive is disconnected but the empty directory remains (created by Docker or `nofail` fstab), data silently writes to the SSD boot drive. Mount-point check (device ID comparison) catches this. This is the #1 gotcha for home server setups. -Auto-discovery from compose files is zero-config for existing apps. The parser is already proven (used in the orphan delete workflow). Adding `backup.data_paths` to `.felhom.yml` is optional polish that gives better descriptions. +### Why no overlapping paths? +If `/mnt/hdd_1` and `/mnt/hdd_1/storage` are both registered, discovery counts files twice and backup may include data twice. One entry per physical mount point keeps it simple. -### Why toggles on the backup page instead of per-app detail? +### Why fallback chain in GetStackHDDMounts? +1. App's own HDD_PATH from app.yaml (most accurate — the actual deployed value) +2. Try all registered storage paths (handles edge cases: missing app.yaml, new template before deploy) +Ensures backup discovery works even in degraded states. -Backup is a cross-cutting concern. Having all toggles in one place on the backup page gives the customer a complete picture of what's protected. Individual app detail pages could show backup status and link to the backup page. +### Why `Schedulable` toggle? +An "unschedulable" path means existing apps stay put but no new apps can use it. Useful for drives that are filling up or being deprecated — prevents new deployments while allowing existing apps to continue operating. -### Backup size awareness - -Enabling app data backup can dramatically increase backup duration and repo size (especially media files with low dedup potential). The repo lives on SSD (`/srv/backups/restic-repo`). Show an info banner: - -"ℹ️ Az alkalmazás adatok mentésének bekapcsolása megnöveli a mentési időt és a tárhelyigényt." - -The existing health check already warns on SSD disk usage > 80%. - -### HDD read-only mount is fine - -The controller mounts HDD as `:ro`. Restic only reads files to create snapshots — read-only access is sufficient and more secure. - -### Circular import avoidance - -The backup package needs stack data but shouldn't import the stacks package. Use a `StackDataProvider` interface defined in the backup package, implemented by a thin adapter in `main.go`. This keeps the dependency graph clean. - -### Future considerations -- **Docker volume backup** — Phase 4: mount `/var/lib/docker/volumes:ro` into controller, or use a sidecar approach -- **Offsite backup** — the "Távoli másolat" placeholder: second restic repo to B2/S3/SFTP -- **Selective file restore** — browse individual files within snapshots (restic `ls` + `restore --include`). Phase 3 does whole-app restore only. -- **Backup scheduling per app** — different retention or frequency for different apps -- **Backup size quota warnings** — alert when backup repo exceeds a configurable threshold - -### Encryption architecture - -All restic backups are encrypted at rest — restic has no unencrypted mode. This is a feature for a managed service ("your backups are encrypted even if someone accesses the drive"). The password is: - -1. Auto-generated (32 random bytes, base64url encoded) on first backup -2. Stored locally at `/opt/docker/felhom-controller/data/restic-password` (Docker named volume) -3. Displayed on the backup page behind a toggle (customer can copy/save it) -4. Synced to hub via periodic report (operator has recovery copy) - -If the SSD fails, the password can be retrieved from hub to access the offsite/HDD backup repo. - -### Why limited restore instead of full file browser? - -A full file browser (restic `ls` per snapshot, pick individual files) adds significant UI complexity: directory tree rendering, file selection, path handling. For Phase 3, whole-app restore covers the primary emergency use case (accidental deletion, corruption). File-level restore can be added later as a power-user feature. - -### Why show the password to the customer? - -The managed service model means the operator (Viktor) has hub access and can recover the password. But showing it to the customer: -- Enables self-service disaster recovery if the customer is technically capable -- Builds trust — the customer isn't locked out of their own backups -- Reduces support burden for simple restore scenarios - -The "behind toggle + clipboard" pattern prevents accidental exposure while making it accessible when needed. - -### Why sync password to hub? - -The password lives in a Docker named volume on the SSD. If the SSD fails: -- Without hub sync: password is permanently lost → all backups inaccessible -- With hub sync: operator retrieves from hub → restores access - -This is the single most critical piece of information for backup recovery. Hub sync provides the necessary redundancy. - -### HDD mount mode for restore - -Changing from `:ro` to `:rw` is acceptable because: -- The controller is a trusted, operator-managed component -- It already has Docker socket access (much more privileged than filesystem write) -- The `:ro` mount was a defense-in-depth measure, not a hard security boundary -- Without `:rw`, restore would require either a sidecar container or host-level orchestration, adding significant complexity for minimal security benefit - -### Restore concurrency - -Backup and restore both use the same restic repository. Running them concurrently can cause lock contention or corruption. A simple `sync.Mutex` in the backup Manager prevents this. The mutex is held for the duration of either operation. \ No newline at end of file +### Future phases (NOT in scope) +- **Phase B:** Storage management UI polish — disk usage bars per path, per-app storage column on apps page +- **Phase C:** Migration wizard — "Mozgatás" button per app, rsync with progress reporting, automatic app.yaml HDD_PATH update + restart, old data cleanup option \ No newline at end of file