# TASK: Phase A — Storage Paths Foundation & Backup Toggle Fix **Version target:** controller 0.9.0 **Repos:** `deploy-felhom-compose` (controller), `app-catalog-felhom.eu` (compose mount change) ## Overview Per-app backup toggles (implemented in v0.8.0) don't appear on the backup page. Root cause is two-layered: 1. **`controller.yaml` has no `paths.hdd_path`** → `m.cfg.Paths.HDDPath` is `""` → `ParseComposeHDDMounts` returns nil → `DiscoverAppData` finds zero HDD data → no backup toggles shown 2. **Even with a global hdd_path, the parser uses ONE path** → apps deployed with a different `HDD_PATH` (e.g., `/mnt/hdd_placeholder` vs `/mnt/hdd_1`) won't match The fix: stop relying on a single global HDD path. Instead, read each app's **own** `HDD_PATH` from its `app.yaml` env section, and introduce a storage paths registry in `settings.json` that auto-discovers paths from deployed apps. This phase also lays the foundation for multi-storage management (Phase B: UI polish, Phase C: migration wizard). --- ## 1. Root Cause Analysis ### Current broken flow ``` controller.yaml → paths.hdd_path = "" (missing) ↓ backup.Manager.stackProvider.GetStackHDDMounts(name) → stacks.ParseComposeHDDMounts(composePath, hddPath="") → hddPath == "" → return nil ← BUG: exits early ↓ DiscoverAppData → info.HasHDDData = false for ALL apps ↓ backups.html → {{if .HasHDDData}} → false → no checkbox rendered ``` ### Second layer (would hit even with hdd_path set) ``` controller.yaml → paths.hdd_path = "/mnt/hdd_1" ↓ ParseComposeHDDMounts(composePath, "/mnt/hdd_1") → reads compose: "- ${HDD_PATH}/storage/immich:/usr/src/app/upload" → resolves ${HDD_PATH} → "/mnt/hdd_1/storage/immich" → BUT app.yaml has HDD_PATH="/mnt/hdd_placeholder" → actual data is at /mnt/hdd_placeholder/storage/immich → os.Stat("/mnt/hdd_1/storage/immich") → not found → path.Exists = false, SizeBytes = 0 ``` ### Correct approach Read each app's actual `HDD_PATH` from its `app.yaml` env section. The compose template uses `${HDD_PATH}` which gets resolved at `docker compose up` time from the environment. We need to resolve it the same way during discovery. --- ## 2. Storage Paths Registry ### 2.1 New struct in `settings.go` ```go // StoragePath represents a registered external storage location. type StoragePath struct { Path string `json:"path"` // e.g., "/mnt/hdd_1" Label string `json:"label,omitempty"` // e.g., "Külső HDD 1TB" IsDefault bool `json:"is_default,omitempty"` // new apps use this by default Schedulable bool `json:"schedulable"` // whether new apps can be deployed here AddedAt string `json:"added_at"` // RFC3339 } ``` ### 2.2 Add to Settings struct ```go type Settings struct { // ... existing fields ... // Storage paths registry StoragePaths []StoragePath `json:"storage_paths,omitempty"` } ``` ### 2.3 Helper methods on Settings ```go // GetStoragePaths returns all registered storage paths. func (s *Settings) GetStoragePaths() []StoragePath // GetDefaultStoragePath returns the default storage path, or "" if none. func (s *Settings) GetDefaultStoragePath() string // GetSchedulableStoragePaths returns paths where new apps can be deployed. func (s *Settings) GetSchedulableStoragePaths() []StoragePath // AddStoragePath registers a new storage path after validation. func (s *Settings) AddStoragePath(path, label string, isDefault bool) error // RemoveStoragePath removes a path only if no apps reference it. // Returns error with list of apps using this path if removal is blocked. func (s *Settings) RemoveStoragePath(path string, appChecker func(string) []string) error // SetDefaultStoragePath changes which path is the default. func (s *Settings) SetDefaultStoragePath(path string) error // SetSchedulable enables/disables a path for new deployments. func (s *Settings) SetSchedulable(path string, schedulable bool) error ``` ### 2.4 Validation rules (in AddStoragePath) 1. **Must be a real mount point** — compare device ID of path vs parent using `syscall.Stat`. Reject paths that are just directories on the boot SSD. Error message: "Ez az útvonal nem külön csatlakoztatott meghajtó. Adatok az SSD-re kerülnének." 2. **Must exist and be a directory** — `os.Stat` check 3. **Must be writable** — attempt to create + remove a temp file 4. **No overlapping paths** — reject if new path is a parent or child of an existing path (e.g., `/mnt/hdd_1` and `/mnt/hdd_1/subdir`) 5. **No duplicates** — reject if path already registered (normalized with `filepath.Clean`) 6. **Must be under `/mnt/`** — soft warning, not hard block (log a WARN for edge cases) ### 2.5 Auto-discovery on startup On controller startup, if `StoragePaths` is empty in settings.json, auto-discover from deployed apps: ```go func (s *Settings) AutoDiscoverStoragePaths(stacksDir string, fallbackHDDPath string, logger *log.Logger) { if len(s.StoragePaths) > 0 { return // already configured, don't override } // Scan all deployed apps' app.yaml for HDD_PATH values seen := map[string]bool{} entries, _ := os.ReadDir(stacksDir) for _, e := range entries { if !e.IsDir() { continue } appCfg := LoadAppConfig(filepath.Join(stacksDir, e.Name())) if appCfg == nil || !appCfg.Deployed { continue } if hddPath, ok := appCfg.Env["HDD_PATH"]; ok && hddPath != "" { cleaned := filepath.Clean(hddPath) if !seen[cleaned] { seen[cleaned] = true } } } // Also use controller.yaml paths.hdd_path as fallback seed if fallbackHDDPath != "" { cleaned := filepath.Clean(fallbackHDDPath) if !seen[cleaned] { seen[cleaned] = true } } // Register discovered paths isFirst := true for path := range seen { sp := StoragePath{ Path: path, Label: inferStorageLabel(path), IsDefault: isFirst, Schedulable: true, AddedAt: time.Now().UTC().Format(time.RFC3339), } s.StoragePaths = append(s.StoragePaths, sp) isFirst = false } if len(s.StoragePaths) > 0 { s.Save() logger.Printf("[INFO] Auto-discovered %d storage path(s)", len(s.StoragePaths)) for _, sp := range s.StoragePaths { logger.Printf("[INFO] %s (%s) default=%v", sp.Path, sp.Label, sp.IsDefault) } } } ``` Helper `inferStorageLabel(path)`: - `/mnt/hdd_1` → "Külső tárhely (hdd_1)" - `/mnt/hdd_placeholder` → "Külső tárhely (hdd_placeholder)" - Anything else → path basename Note: `LoadAppConfig` is in the `stacks` package. To avoid circular imports, either pass a scanner function or do the scanning in `main.go` and pass the discovered paths in. Use whichever approach avoids import issues. --- ## 3. Fix `GetStackHDDMounts` — Per-App HDD_PATH Resolution ### 3.1 Change the adapter Currently: ```go func (a *stackAdapter) GetStackHDDMounts(name string) []string { s, ok := a.mgr.GetStack(name) if !ok { return nil } return stacks.ParseComposeHDDMounts(s.ComposePath, a.hddPath) // ← uses global path } ``` Change to: ```go func (a *stackAdapter) GetStackHDDMounts(name string) []string { s, ok := a.mgr.GetStack(name) if !ok { return nil } // Priority 1: Read the app's own HDD_PATH from its app.yaml stackDir := filepath.Dir(s.ComposePath) appCfg := stacks.LoadAppConfig(stackDir) if appCfg != nil && appCfg.Env["HDD_PATH"] != "" { return stacks.ParseComposeHDDMounts(s.ComposePath, appCfg.Env["HDD_PATH"]) } // Priority 2: Try all registered storage paths (fallback) var allMounts []string seen := map[string]bool{} for _, sp := range a.getStoragePaths() { mounts := stacks.ParseComposeHDDMounts(s.ComposePath, sp.Path) for _, m := range mounts { if !seen[m] { seen[m] = true allMounts = append(allMounts, m) } } } return allMounts } ``` ### 3.2 Update adapter struct ```go type stackAdapter struct { mgr *stacks.Manager getStoragePaths func() []settings.StoragePath // getter from settings } ``` Wire in `main.go`: ```go adapter := &stackAdapter{ mgr: stackMgr, getStoragePaths: func() []settings.StoragePath { return settingsMgr.GetStoragePaths() }, } ``` Remove the old `hddPath string` field from the adapter. ### 3.3 `resolveAppBackupPaths()` in backup Manager No changes needed — it already calls `m.stackProvider.GetStackHDDMounts(stackName)` which we're fixing above. The fix propagates automatically. --- ## 4. Controller Docker Compose Mount Change ### 4.1 Change in `controller/docker-compose.yml` Replace: ```yaml - ${HDD_PATH:-/mnt/hdd_placeholder}:${HDD_PATH:-/mnt/hdd_placeholder}:ro ``` With: ```yaml # All external storage — covers /mnt/* paths for multi-storage support + restore - /mnt:/mnt:rw ``` **Why `:rw`:** Required for restore feature (already implemented). Also needed for mount-point validation (write test in AddStoragePath). **Why `/mnt:/mnt`:** All standard storage mounts are under `/mnt`. Covers current and future drives with one entry. No controller restart when adding new drives. ### 4.2 Also update `docker-setup.sh` If `docker-setup.sh` generates the controller compose, update it to emit `/mnt:/mnt:rw` instead of the old `${HDD_PATH}` line. --- ## 5. Deploy Page: Storage Path Dropdown ### 5.1 When app has a `path` type deploy field (HDD_PATH) Currently renders as a plain text input. Change to dropdown when storage paths exist: ```html {{if eq .Type "path"}} {{if $.StoragePaths}} {{else}} Nincs regisztrált adattároló. Adjon hozzá egyet a Beállítások oldalon. {{end}} {{end}} ``` ### 5.2 Pass storage paths to deploy page template data In the deploy handler: ```go data["StoragePaths"] = settingsMgr.GetSchedulableStoragePaths() ``` ### 5.3 Already-deployed: show current path read-only When `AlreadyDeployed == true`, the field is disabled. Show the actual `HDD_PATH` from `app.yaml` as the displayed value (existing behavior works as-is with a disabled select or text input). ### 5.4 Edge case: no schedulable paths but app needs HDD If `resources.needs_hdd: true` and zero schedulable paths: - Show warning: "Nincs elérhető adattároló. Csatlakoztasson külső meghajtót és adja hozzá a Beállítások oldalon." - Disable deploy button (add `data-needs-storage="true"` to form, JS disables submit) --- ## 6. Storage Path Monitoring Integration ### 6.1 Periodic mount-point check Add to existing system health check (runs every `system_health_interval`, default 5m): ```go func checkStoragePaths(paths []settings.StoragePath, logger *log.Logger) []string { var warnings []string for _, sp := range paths { // Check 1: path exists and is directory fi, err := os.Stat(sp.Path) if err != nil { warnings = append(warnings, fmt.Sprintf("Adattároló nem elérhető: %s", sp.Path)) continue } if !fi.IsDir() { warnings = append(warnings, fmt.Sprintf("Adattároló nem mappa: %s", sp.Path)) continue } // Check 2: still a real mount point (not writing to SSD) if !isMountPoint(sp.Path) { warnings = append(warnings, fmt.Sprintf("FIGYELEM: %s nincs felcsatolva (leválasztva?) — "+ "az adatok az SSD-re íródnának!", sp.Path)) } // Check 3: disk space (reuse existing threshold config) usage := getDiskUsage(sp.Path) if usage.UsedPercent >= 90 { warnings = append(warnings, fmt.Sprintf("Adattároló majdnem tele: %s (%.0f%%)", sp.Path, usage.UsedPercent)) } } return warnings } ``` ### 6.2 `isMountPoint()` implementation New file: `controller/internal/system/mounts.go` ```go // IsMountPoint checks if a path is a mount point by comparing device IDs. // Returns true if path is on a different device than its parent. func IsMountPoint(path string) bool { var pathStat, parentStat syscall.Stat_t if err := syscall.Stat(path, &pathStat); err != nil { return false } parent := filepath.Dir(path) if err := syscall.Stat(parent, &parentStat); err != nil { return false } return pathStat.Dev != parentStat.Dev } ``` Platform-specific: needs build tag `//go:build linux` (consistent with existing system package). Add stub for non-Linux that always returns true. ### 6.3 Surface warnings - Add storage warnings to monitoring page warnings section (existing infrastructure) - Include in hub report `SystemWarnings` field (already exists) - Fire `disk_warning` notification event for enabled customers --- ## 7. Beállítások Page: Storage Section Add between "Rendszer konfiguráció" and "Jelszó módosítás" sections. ### 7.1 Template: "Adattárolók" section ```html
Külső meghajtók, ahol az alkalmazások felhasználói adatai tárolódnak.
{{if .StorageError}}