# TASK.md — v0.14.0 Storage & Backup Architecture Overhaul **Version:** v0.14.0 **Type:** Architecture overhaul — storage paths, backup structure, multi-drive support **Scope:** Controller Go code + app catalog compose files + setup scripts **Note:** Demo node will be reinstalled from scratch — no migration needed --- ## Design Overview ### New directory structure (per drive) Every drive mount (`/mnt/sys_drive`, `/mnt/hdd_1`, `/mnt/hdd_2`, ...) uses the same layout: ``` /mnt// appdata// ← live app data (renamed from "storage") backups/ primary/ /db-dumps/ ← raw DB dumps per app (accessible for testing) restic/ ← per-drive restic repo (all apps on this drive) secondary/ /rsync/ ← rsync copies from apps on OTHER drives restic/ ← restic repo for secondary copies Dokumentumok/ media/ Download/ movies/ series/ music/ audiobooks/ ``` ### Key rules 1. **An app's "home drive"** = the drive from its `HDD_PATH` env var, or `cfg.Paths.SystemDataPath` if no HDD_PATH 2. **Primary backup** lives on the SAME drive as the app — protects against accidental deletion, app bugs 3. **Secondary backup** lives on a DIFFERENT drive — protects against drive failure 4. **One restic repo per drive** (in both primary and secondary) — same password for all repos 5. **DB dumps** are raw SQL files per-app, always on the app's home drive, also included in restic 6. **Compose configs + controller.yaml** go into EVERY primary restic repo (small, ensures self-contained restore) 7. **`storage/` → `appdata/`** rename across all compose templates 8. **Filebrowser** mounts per-drive subdirectories: `media/`, `Dokumentumok/`, `backups/secondary/` (for file recovery) --- ## Phase 1: Config & path helpers ### 1a. `internal/config/config.go` **Add:** - `SystemDataPath string \`yaml:"system_data_path"\`` to `PathsConfig` — default `/mnt/sys_drive` **Remove from struct:** - `BackupDir string` from PathsConfig - `DBDumpDir string` from PathsConfig - `ResticRepo string` from BackupConfig **Keep:** - `ResticPasswordFile string` in BackupConfig (shared across all repos) - `HDDPath string` in PathsConfig (legacy, still used as default storage) **Update `applyDefaults()`:** - Remove: `d(&cfg.Paths.BackupDir, "/srv/backups")` - Remove: `d(&cfg.Paths.DBDumpDir, "/srv/backups/db-dumps")` - Remove: `d(&cfg.Backup.ResticRepo, "/srv/backups/restic-repo")` - Add: `d(&cfg.Paths.SystemDataPath, "/mnt/sys_drive")` **Gotcha:** All code referencing `cfg.Paths.BackupDir`, `cfg.Paths.DBDumpDir`, `cfg.Backup.ResticRepo` will break. Grep for all references and update. ### 1b. New file: `internal/backup/paths.go` Path computation helpers (pure functions, no state): ```go package backup import "path/filepath" func PrimaryBackupPath(drivePath string) string { return filepath.Join(drivePath, "backups", "primary") } func PrimaryResticRepoPath(drivePath string) string { return filepath.Join(drivePath, "backups", "primary", "restic") } func AppDBDumpPath(drivePath, stackName string) string { return filepath.Join(drivePath, "backups", "primary", stackName, "db-dumps") } func SecondaryBackupPath(drivePath string) string { return filepath.Join(drivePath, "backups", "secondary") } func AppSecondaryRsyncPath(drivePath, stackName string) string { return filepath.Join(drivePath, "backups", "secondary", stackName, "rsync") } func SecondaryResticRepoPath(drivePath string) string { return filepath.Join(drivePath, "backups", "secondary", "restic") } func AppDataPath(drivePath, stackName string) string { return filepath.Join(drivePath, "appdata", stackName) } ``` ### 1c. App drive resolution Need a method to determine which drive an app lives on. Add to the backup Manager or StackDataProvider: ```go // GetAppDrivePath returns the drive path for an app. // Uses HDD_PATH from app.yaml if set, otherwise falls back to system data path. func (m *Manager) GetAppDrivePath(stackName string) string { if mounts := m.stackProvider.GetStackHDDMounts(stackName); len(mounts) > 0 { // The HDD_PATH is the mount point — extract the drive from the first mount // e.g., /mnt/hdd_1/appdata/immich → /mnt/hdd_1 // Actually, we need the HDD_PATH itself, not the mounts } return m.systemDataPath } ``` **Gotcha:** `GetStackHDDMounts` returns resolved mount paths (e.g., `/mnt/hdd_1/appdata/immich`), not the raw `HDD_PATH` value. Need a way to get the raw HDD_PATH for a stack. Options: - Add `GetStackHDDPath(name string) string` to `StackDataProvider` interface - Or: derive the drive from mount paths by finding the common `/mnt/` prefix - Best: add to StackDataProvider — clean, explicit ### 1d. StackDataProvider interface update In `internal/backup/appdata.go`, add: ```go type StackDataProvider interface { // ... existing methods ... GetStackHDDPath(name string) string // NEW: raw HDD_PATH from app.yaml } ``` And implement in the `stackAdapter` in `main.go`. --- ## Phase 2: DB dump refactor ### 2a. `internal/backup/backup.go` — DumpAll() Currently dumps all DBs to one global directory (`m.cfg.Paths.DBDumpDir`). **Change to:** For each discovered DB, determine the app's drive, dump to `/backups/primary//db-dumps/`. Key changes: - Remove references to `m.cfg.Paths.DBDumpDir` - Compute dump path per stack: `AppDBDumpPath(m.GetAppDrivePath(stack), stack)` - Create dir if not exists before dumping - Update `DumpResult` to include per-stack dump paths ### 2b. `internal/backup/backup.go` — DumpStackDB() Same refactor for single-stack dump (called by cross-drive before running Tier 2 backup). ### 2c. Status/validation Currently `RefreshCache()` lists all dump files from one directory. Need to scan per-drive dump directories instead. - Scan all registered drives (from settings or deployed stacks) - For each drive, glob `/backups/primary/*/db-dumps/*.sql` - Aggregate results --- ## Phase 3: Restic backup refactor ### 3a. `internal/backup/restic.go` — ResticManager Currently `ResticManager` has a single `repoPath`. Need to support multiple repos. **Option A:** Make ResticManager stateless — pass repoPath per operation. **Option B:** Create multiple ResticManager instances. **Recommend Option A** — cleaner for per-drive operations. Refactor all ResticManager methods to accept `repoPath` as parameter instead of using `r.repoPath`: - `EnsureInitialized(repoPath string) error` - `RunBackup(ctx, repoPath string, paths []string, tags []string) (*SnapshotResult, error)` - `ListSnapshots(repoPath string) ([]SnapshotInfo, error)` - `GetRepoStats(repoPath string) (*RepoStats, error)` - `RunCheck(repoPath string) error` - `RunPrune(repoPath string) error` - etc. Keep `r.passwordFile`, `r.cacheDir`, `r.logger` as instance fields. ### 3b. `internal/backup/backup.go` — RunBackup() Currently: ```go paths := []string{stacksDir, dbDumpDir, controllerYaml} paths = append(paths, appPaths...) // one restic backup ``` **Change to:** ```go func (m *Manager) RunBackup(ctx context.Context) error { // Group deployed stacks by drive driveStacks := m.groupStacksByDrive() infraPaths := []string{ m.cfg.Paths.StacksDir, "/opt/docker/felhom-controller/controller.yaml", } for drivePath, stacks := range driveStacks { repoPath := PrimaryResticRepoPath(drivePath) m.restic.EnsureInitialized(repoPath) var paths []string // Always include infra (compose configs + controller.yaml) in every repo paths = append(paths, infraPaths...) for _, stack := range stacks { // App data (appdata//) appData := AppDataPath(drivePath, stack.Name) if _, err := os.Stat(appData); err == nil { paths = append(paths, appData) } // DB dumps for this stack dumpDir := AppDBDumpPath(drivePath, stack.Name) if _, err := os.Stat(dumpDir); err == nil { paths = append(paths, dumpDir) } } // Tag with drive name for easy filtering tags := []string{filepath.Base(drivePath)} m.restic.RunBackup(ctx, repoPath, paths, tags) } } ``` ### 3c. Prune, check, forget — per drive Currently scheduled as single jobs. Need to loop over all active drive repos: - `RunPrune()` → for each drive, prune that drive's primary restic repo - `RunCheck()` → same - `RunForget()` → same ### 3d. Snapshot listing & stats — aggregate For the backup page UI: - `ListSnapshots()` → list from all primary repos, merge and sort by time - `GetRepoStats()` → aggregate total size and snapshot count across repos - Tag snapshots with drive name so UI can optionally group them ### 3e. Monitoring pings After ALL drive backups complete (not per-drive), send the backup ping. If ANY drive fails, the ping is not sent (or sent as failure). --- ## Phase 4: Cross-drive (secondary) backup refactor ### 4a. `internal/backup/crossdrive.go` — runRsyncBackup() **Current:** `destDir = filepath.Join(destBase, "backups", "rsync", stackName)` **New:** `destDir = AppSecondaryRsyncPath(destBase, stackName)` → `/backups/secondary//rsync/` Update all path computations: - `destDir` construction - `_db/` subdirectory (now under rsync/ too) - `_config/` subdirectory - Size calculation path ### 4b. `internal/backup/crossdrive.go` — runResticBackup() **Current:** `repoPath = filepath.Join(destBase, "backups", "restic")` **New:** `repoPath = SecondaryResticRepoPath(destBase)` → `/backups/secondary/restic/` ### 4c. DB dump source path Currently: `r.dbDumpDir` (global directory) Now: per-app dump dir: `AppDBDumpPath(appDrivePath, stackName)` The cross-drive runner needs to know the app's home drive to find its DB dumps. - Add `GetAppDrivePath` method to CrossDriveRunner (or pass via StackDataProvider) --- ## Phase 5: Protected paths & delete safety ### 5a. `internal/stacks/delete.go` — ProtectedHDDPaths() **Current:** ```go return map[string]bool{ hddPath: true, filepath.Join(hddPath, "media"): true, filepath.Join(hddPath, "storage"): true, filepath.Join(hddPath, "Dokumentumok"): true, filepath.Join(hddPath, "appdata"): true, } ``` **Change to:** ```go return map[string]bool{ hddPath: true, filepath.Join(hddPath, "appdata"): true, filepath.Join(hddPath, "backups"): true, filepath.Join(hddPath, "media"): true, filepath.Join(hddPath, "Dokumentumok"): true, } ``` Remove `storage` (gone), add `backups`. --- ## Phase 6: Filebrowser mount sync ### 6a. `internal/web/handlers.go` — syncFileBrowserMounts() **Current:** Mounts each registered path as one volume: `:/srv/` This exposes EVERYTHING on the drive, including encrypted restic repos and raw appdata. **Change to:** Mount specific subdirectories per drive: ```go for _, sp := range paths { driveName := filepath.Base(sp.Path) // "hdd_1", "sys_drive" // User media mediaPath := filepath.Join(sp.Path, "media") if dirExists(mediaPath) { storageMounts = append(storageMounts, fmt.Sprintf(" - %s:/srv/%s/media", mediaPath, driveName)) } // User documents docsPath := filepath.Join(sp.Path, "Dokumentumok") if dirExists(docsPath) { storageMounts = append(storageMounts, fmt.Sprintf(" - %s:/srv/%s/Dokumentumok", docsPath, driveName)) } // Secondary backup copies (rsync — browseable for file recovery) secPath := filepath.Join(sp.Path, "backups", "secondary") if dirExists(secPath) { storageMounts = append(storageMounts, fmt.Sprintf(" - %s:/srv/%s/backups:ro", secPath, driveName)) } } ``` This gives Filebrowser users access to: - Their media files (movies, music, etc.) - Their documents - Secondary backup copies (rsync) for file recovery - NOT raw appdata (dangerous), NOT restic repos (useless) --- ## Phase 7: App catalog changes ### 7a. Compose file updates (`app-catalog-felhom.eu`) All 11+ apps with `needs_hdd: true`: rename `${HDD_PATH}/storage/` → `${HDD_PATH}/appdata/` in volume mounts. **Apps to update** (grep for `storage/` in compose files): - immich, paperless-ngx, audiobookshelf, calibre-web, emby, jellyfin, komga, navidrome, nextcloud, plex, radarr, romm, sonarr Each compose file's volumes section changes, e.g.: ```yaml # Before: - ${HDD_PATH}/storage/immich:/usr/src/app/upload # After: - ${HDD_PATH}/appdata/immich:/usr/src/app/upload ``` ### 7b. Media-centric apps (Jellyfin, Plex, Emby, Radarr, Sonarr) These apps also mount media directories. Check if they reference `${HDD_PATH}/media/` — if so, that's correct (no rename needed for media/). ### 7c. `.felhom.yml` files The `HDD_PATH` field metadata doesn't reference `storage/` — it just declares the env var. Description says "külső merevlemez elérési útja" which is fine. No changes needed. --- ## Phase 8: Setup script updates ### 8a. `scripts/docker-setup.sh` - Update `install_filebrowser()` volume mounts to use new per-subdirectory pattern - Or: remove Filebrowser initial mounts entirely (controller will sync them on startup) ### 8b. `scripts/hdd-setup.sh` - Update `STORAGE_DIRS` to remove `storage/` entries - Update to use `appdata/` naming - Or: mark as deprecated (controller handles disk init now) --- ## Phase 9: controller.yaml update New controller.yaml for demo node (after OS reinstall with SSD partition): ```yaml paths: stacks_dir: "/opt/docker/stacks" system_data_path: "/mnt/sys_drive" backup: enabled: true restic_password_file: "/opt/docker/felhom-controller/data/restic-password" db_dump_schedule: "02:30" restic_schedule: "03:00" retention: keep_daily: 7 keep_weekly: 4 keep_monthly: 6 prune_schedule: "sunday" ``` No more `restic_repo`, `db_dump_dir`, `backup_dir`. --- ## Phase 10: UI — Tároló section (simple update) The Tároló section on the backup page needs to work with the new multi-drive, multi-repo architecture. Since we already agreed to show combined stats (not paths): - **Tier 1 summary:** Aggregate snapshot count + total size across all primary repos - **Tier 2 summary:** How many apps configured, total size - **Keep:** Encryption key display (same password for all repos) - **Remove:** Path displays, DB dump section (unnecessary detail) --- ## Gotchas & risks 1. **Grep for ALL references** to removed config fields: `BackupDir`, `DBDumpDir`, `ResticRepo`, `cfg.Backup.ResticRepo`, `cfg.Paths.DBDumpDir`, `cfg.Paths.BackupDir` 2. **ResticManager refactor** changes all call sites — grep for `m.restic.` in backup.go 3. **DB dump path in crossdrive.go** — currently `r.dbDumpDir` (global). Needs per-app resolution. 4. **Snapshot aggregation** — merging snapshots from multiple repos for the UI. Need to handle different repo sizes, dedup by timestamp. 5. **New restic repo initialization** — when a new drive is registered and first backup runs, `restic init` must succeed before `restic backup`. The `EnsureInitialized` pattern already exists. 6. **Empty drives** — a drive with no apps deployed yet should NOT get a restic backup (empty paths). Skip drives with zero apps. 7. **The `systemDataPath` as fallback** — SSD-only apps (Mealie, Gokapi) have no HDD_PATH. Their drive is `cfg.Paths.SystemDataPath`. Make sure this path exists and is registered as a storage path. 8. **Compose config files in multiple repos** — `/opt/docker/stacks/` is included in every drive's primary repo. This means the same files are in multiple repos. That's intentional (each repo is self-contained) but uses slightly more storage. 9. **The `ParseComposeHDDMounts` function** references `${HDD_PATH}` with `storage/` subdirs. After rename to `appdata/`, the compose files change, so the parsed mounts change too. The function itself is generic (parses any `${HDD_PATH}` prefix) so it doesn't need code changes — only the compose templates change. 10. **docker-compose.yml volumes in felhom-controller** — currently `- /srv/backups:/srv/backups`. This mount becomes unnecessary since all backups are under `/mnt/`. The `/mnt:/mnt:rshared` mount already provides access. Can remove the `/srv/backups` volume mount from the controller's compose file. --- ## Implementation order 1. **Phase 1** — Config + path helpers + StackDataProvider update (foundation, everything depends on this) 2. **Phase 7** — App catalog compose files (independent, can do in parallel) 3. **Phase 5** — Protected paths (quick, independent) 4. **Phase 2** — DB dump refactor 5. **Phase 3** — Restic backup refactor (depends on Phase 1 + 2) 6. **Phase 4** — Cross-drive backup refactor (depends on Phase 1) 7. **Phase 6** — Filebrowser mount sync 8. **Phase 10** — UI Tároló section 9. **Phase 8** — Setup scripts 10. **Phase 9** — controller.yaml Build, deploy to reinstalled demo node, verify. --- ## Files to modify ### Controller (deploy-felhom-compose/controller/) | File | Phase | Changes | |------|-------|---------| | `internal/config/config.go` | 1a | Add SystemDataPath, remove BackupDir/DBDumpDir/ResticRepo | | `internal/backup/paths.go` | 1b | **NEW FILE** — path computation helpers | | `internal/backup/appdata.go` | 1d | Add GetStackHDDPath to StackDataProvider | | `cmd/controller/main.go` | 1d | Implement GetStackHDDPath in stackAdapter | | `internal/backup/backup.go` | 2+3 | DumpAll, DumpStackDB, RunBackup, RefreshCache, GetFullStatus, RunPrune, RunCheck | | `internal/backup/restic.go` | 3a | Make repoPath a parameter, not instance field | | `internal/backup/crossdrive.go` | 4 | Update destination paths, DB dump source paths | | `internal/stacks/delete.go` | 5 | Update ProtectedHDDPaths | | `internal/web/handlers.go` | 6+10 | syncFileBrowserMounts, backupsHandler | | `internal/web/templates/backups.html` | 10 | Tároló section | ### App catalog (app-catalog-felhom.eu/) | File | Phase | Changes | |------|-------|---------| | `templates/*/docker-compose.yml` (11+ files) | 7 | `storage/` → `appdata/` in volume mounts | ### Scripts | File | Phase | Changes | |------|-------|---------| | `scripts/docker-setup.sh` | 8 | Filebrowser mounts, path references | | `scripts/hdd-setup.sh` | 8 | Directory structure arrays | ### Config | File | Phase | Changes | |------|-------|---------| | Demo node `controller.yaml` | 9 | New paths config | | Demo node `docker-compose.yml` | 10 | Remove `/srv/backups` mount |