feat: Docker volume backup, Tier 2 restore, restore dropdown fixes (v0.33.0)

- Add Docker named volume backup to Tier 1 (dump to tar, include in restic) and Tier 2 (copy tars to rsync mirror _volumes/ dir) - Fix volume name resolution: use project-prefixed names (mealie_mealie_data) - Fix double Tier 1 in restore dropdown: filter snapshots by app's home drive - Add Tier 2 restore: RestoreAppFromTier2() restores from rsync mirror - Show Tier 2 entry in restore dropdown when cross-drive backup succeeded - Add .fab import link in restore section - Volume-aware restore type banners and backup content labels Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 21:43:02 +01:00
parent 5bf13ca19d
commit c929948f27
12 changed files with 655 additions and 45 deletions
@@ -4,7 +4,7 @@

 A single, lightweight Go container that replaces Portainer + scattered systemd scripts with a unified, Hungarian-language web dashboard for managing Docker Compose stacks, backups, storage, monitoring, and notifications on customer hardware.

-**Current version: v0.32.6**
+**Current version: v0.33.0**

 ---

@@ -282,7 +282,9 @@ self-sufficient backup** — any single tier can fully restore an app.
 |----------|----------------|---------|
 | HDD + DB | Config + DB + User data | Immich, Paperless-ngx |
 | HDD, no DB | Config + User data | — |
-| DB, no HDD | Config + DB | Mealie, Vikunja |
+| Docker volumes + DB | Config + DB + Volume data | Tandoor |
+| Docker volumes, no DB | Config + Volume data | Mealie (SQLite) |
+| DB, no HDD/volumes | Config + DB | Vikunja |
 | Config only | Config | Gokapi, Homepage |

 #### Tier 1: Nightly Backup (mandatory, same drive)
@@ -296,8 +298,10 @@ The nightly backup has two phases that run sequentially. All paths are **per-dri
 │   ├── appdata/<app>/          ← app user data
 │   └── backups/
 │       ├── primary/
-│       │   ├── restic/         ← one restic repo per drive (all apps on this drive)
-│       │   └── <app>/db-dumps/ ← per-app DB dump files
+│       │   ├── restic/              ← one restic repo per drive (all apps on this drive)
+│       │   └── <app>/
+│       │       ├── db-dumps/       ← per-app DB dump files
+│       │       └── volume-dumps/   ← per-app Docker volume tars (v0.33.0)
 │       └── secondary/
 │           ├── restic/         ← secondary restic repo (cross-drive)
 │           ├── _infra/         ← infra config mirror
@@ -313,6 +317,7 @@ The nightly backup has two phases that run sequentially. All paths are **per-dri
 Path computation is centralized in `backup/paths.go` via the `FelhomDataDir = "felhom-data"` constant:
 - `PrimaryResticRepoPath(drivePath)` → `<drive>/felhom-data/backups/primary/restic/`
 - `AppDBDumpPath(drivePath, stackName)` → `<drive>/felhom-data/backups/primary/<stack>/db-dumps/`
+- `AppVolumeDumpPath(drivePath, stackName)` → `<drive>/felhom-data/backups/primary/<stack>/volume-dumps/`
 - `AppDataDir(drivePath, stackName)` → `<drive>/felhom-data/appdata/<stack>/`
 - `SecondaryResticRepoPath(drivePath)` → `<drive>/felhom-data/backups/secondary/restic/`
 - `AppSecondaryRsyncPath(drivePath, stackName)` → `<drive>/felhom-data/backups/secondary/<stack>/rsync/`
@@ -328,6 +333,15 @@ Path computation is centralized in `backup/paths.go` via the `FelhomDataDir = "f
 - **Validation** after each dump: checks file size, header presence, counts `CREATE TABLE`
 - Results cached in `settings.json` surviving container restarts

+**Phase 1b — Docker Volume Dumps** (`internal/backup/backup.go`, runs after DB dumps)
+
+- Iterates all deployed stacks that have Docker named volumes (`GetDockerVolumes()`)
+- For each volume: `docker run --rm -v <vol>:/vol:ro -v <dumpDir>:/out alpine tar cf /out/<vol>.tar -C /vol .`
+- 10-minute timeout per volume; warnings on failure (non-fatal)
+- Stale tars cleaned up (volumes that no longer exist)
+- Volume names resolved with project prefix via `ResolveDockerVolumeNames()` (e.g., `mealie_mealie_data`)
+- Dumps written to `AppVolumeDumpPath(appDrive, stackName)`
+
 **Phase 2 — Restic Snapshot** (`internal/backup/restic.go`, scheduled 03:00)

 - Apps are **grouped by drive** via `groupStacksByDrive()` — each drive's apps are backed up to that drive's restic repo
@@ -335,6 +349,7 @@ Path computation is centralized in `backup/paths.go` via the `FelhomDataDir = "f
 - Auto-generated repository password (32 random bytes, base64url), shared across all repos, synced to hub
 - **Paths included in every per-drive snapshot:**
  - Per-app DB dump dirs on that drive
+  - Per-app Docker volume dump dirs (`volume-dumps/*.tar`)
  - Per-app HDD mount paths (user data)
  - Stacks dir (compose.yml + app.yaml + .felhom.yml for all apps)
  - `controller.yaml` (controller config)
@@ -348,7 +363,7 @@ Does NOT protect against drive failure (backup is on the same physical drive).
 #### Tier 2: Cross-Drive Backup (opt-in, different device) (`internal/backup/crossdrive.go`)

 **Complete backup** to a different physical drive. Available for **all apps** — apps with HDD
-data back up config + DB + user data; apps without HDD back up config + DB dumps only.
+data back up config + DB + user data + Docker volumes; apps without HDD back up config + DB dumps + Docker volumes.

 - **Auto-enable for small apps (v0.14.1):** Apps without HDD mounts (config-only, DB-only) are
  automatically configured for daily rsync Tier 2 when ≥2 storage paths are registered.
@@ -362,6 +377,7 @@ data back up config + DB + user data; apps without HDD back up config + DB dumps
  - **restic** — Versioned, deduplicated, encrypted (shared repo across apps, not browsable)
 - Per-app configuration in settings.json: destination path, method, schedule (daily/weekly/manual)
 - **Pre-backup DB dump:** `DumpStackDB()` runs fresh pg_dump/mariadb-dump before each cross-drive backup; non-fatal on failure (wired via `DBDumper` interface to avoid circular imports)
+- **Pre-backup volume dump (v0.33.0):** `DumpAppVolumes()` exports Docker named volumes to tar before each cross-drive backup (wired via `VolumeDumper` interface)
 - **Empty mounts allowed:** `RunAppBackup` accepts apps with no HDD mounts — the rsync
  mount loop simply doesn't execute, but DB + config copy still runs
 - **Drive-type-aware validation** (`ValidateDestination`):
@@ -380,12 +396,13 @@ data back up config + DB + user data; apps without HDD back up config + DB dumps
  ├── <app>/rsync/         ← per-app rsync mirror
  │   ├── _db/             ← DB dump files
  │   ├── _config/         ← compose.yml, app.yaml, .felhom.yml
+  │   ├── _volumes/        ← Docker volume tars (v0.33.0)
  │   └── <user data>      ← HDD mount contents (if app has HDD data)
  └── restic/              ← shared restic repo (all cross-drive apps)
  ```
  - DB dump files read from **per-app home drive** path (`AppDBDumpPath`)
  - `_` prefix directories prevent collision with user data
-  - For non-HDD apps, only `_db/` and `_config/` are present (no user data directory)
+  - For non-HDD apps, only `_db/`, `_config/`, and `_volumes/` (if applicable) are present (no user data directory)
 - **Restic backup paths:** includes HDD mounts (if any) + config dir + per-app DB dump dir from home drive + stacks dir + controller.yaml (infra, v0.14.1)
 - Safety guards: destination ≠ source, path-overlap check (HDD mounts only), writable check
 - **Chained execution:** runs immediately after nightly restic — daily apps every night, weekly apps on Sundays
@@ -402,28 +419,40 @@ Placeholder shown in UI ("3. mentés — Hamarosan").

 #### Restore (`internal/backup/restore.go`)

-All deployed apps appear in the restore dropdown — every app has restic snapshot data
-(stacks dir + DB dumps are always backed up).
+Both **Tier 1** (restic) and **Tier 2** (rsync) restores are supported. All deployed apps
+appear in the restore dropdown with per-app snapshot filtering.

-| App type | Config restored | DB restored | User data restored |
-|----------|----------------|------------|-------------------|
-| Has HDD data | Yes | Yes | Yes (always — backup is mandatory) |
-| DB only, no HDD | Yes | Yes | n/a |
-| No DB, no HDD | Yes | — | n/a |
+| App type | Config restored | DB restored | User data restored | Docker volumes restored |
+|----------|----------------|------------|-------------------|------------------------|
+| Has HDD data | Yes | Yes | Yes (always) | Yes (if present) |
+| Docker volumes, no HDD | Yes | Yes | n/a | Yes |
+| DB only, no HDD/volumes | Yes | Yes | n/a | n/a |
+| Config only | Yes | — | n/a | n/a |

- **Snapshot API** returns ALL snapshots unfiltered — older snapshots still allow config+DB restore; `RestoreApp` extracts whatever paths are available
- **Restore type info** shown per-app when selected in dropdown (Hungarian banners):
-  - Has HDD: "Teljes visszaállitas: adatbazis + konfiguracio + felhasznaloi adatok"
-  - Has DB, no HDD: "Adatbazis es konfiguracio visszaallitasa"
-  - No DB, no HDD: "Csak konfiguracio visszaallitasa"
- **Execution flow:** stop app → resolve app's home drive → `restic restore <id> --target / --include <path>...` from per-drive repo → restart app
- Restic repo resolved via `PrimaryResticRepoPath(appDrivePath)`
- DB dumps restored from `AppDBDumpPath(appDrivePath, stackName)`
+**Snapshot API** (`/api/backup/snapshots?stack=<name>`):
+- Returns snapshots **only from the app's home drive** primary repo (prevents showing irrelevant snapshots from other drives)
+- Appends a synthetic Tier 2 entry (ID `tier2-rsync`) from cross-drive config when last backup was successful
+- Dropdown groups by tier: "1. szint — Helyi mentes" and "2. szint — Masodlagos masolat"
+
+**Restore type info** shown per-app when selected in dropdown (Hungarian banners):
+  - Has HDD or Docker volumes: "Teljes visszaallitas: adatbazis + konfiguracio + felhasznaloi adatok"
+  - Has DB, no user data: "Adatbazis es konfiguracio visszaallitasa"
+  - Config only: "Csak konfiguracio visszaallitasa"
+
+**Tier 1 restore** (`RestoreApp`):
+- Stop app → resolve app's home drive → `restic restore <id> --target / --include <path>...` → populate Docker volumes from restored tars → restart app
+- Restore paths: config dir, DB dump dir, volume dump dir, HDD mounts
+- Docker volumes restored via `restoreDockerVolumes()`: `docker volume rm -f` → `docker volume create` → `docker run alpine tar xf`
+
+**Tier 2 restore** (`RestoreAppFromTier2`):
+- Stop app → rsync config from `_config/` → rsync HDD data (single/multi-mount) → copy DB dumps from `_db/` → restore Docker volumes from `_volumes/` tars → restart app
+- Uses rsync `--delete` for config and HDD data to ensure exact mirror state
+- Single-mount apps: data directly in rsync dir (excluding `_*`); multi-mount: per-leaf subdirectories
+
+**Common:**
 - Running flag prevents concurrent backup/restore operations
- Snapshot ID validated (8-64 lowercase hex)
-
-**Note:** Restore currently uses Tier 1 (primary restic repo on app's home drive) only.
-Restoring from Tier 2 (cross-drive) is a future enhancement.
+- Snapshot ID validated (8-64 lowercase hex, or special `tier2-rsync`)
+- Import from `.fab` bundle link shown in restore section for cross-system migration

 #### Backup Page UI (`internal/web/templates/backups.html`)

@@ -448,6 +477,7 @@ Every app starts as yellow (1 tier only). Green requires Tier 2 configured with

 **Backup contents per app** (shown per tier):
 - Apps with DB + HDD: "DB + Konfig + Adatok"
+- Apps with Docker volumes (no HDD): "Konfig + DB + Adatok" or "Konfig + Adatok"
 - Apps with DB only: "DB + Konfig"
 - Apps with HDD, no DB: "Konfig + Adatok"
 - Apps with neither: "Konfig"
@@ -459,7 +489,7 @@ not just those with HDD data. Non-HDD apps can configure destination, method, an
 - Schedule overview with next run times for DB dump, restic, prune
 - Snapshot history table (last 20 snapshots aggregated from all per-drive repos, sorted by time)
 - Storage overview card (total size across repos, snapshot count, DB dump count/size, encryption key with show/copy)
- Restore section: app dropdown → snapshot dropdown → restore type info → confirmation checkbox → execute
+- Restore section: app dropdown → per-app snapshot dropdown (Tier 1 + Tier 2 grouped) → restore type info → confirmation checkbox → execute → import from `.fab` bundle link

 ---

@@ -1773,7 +1803,7 @@ See `docker-compose.yml` for the full volume configuration.
 ### In Progress / Planned

 - [ ] Update classification and auto-apply (optional/required/security markers)
- [ ] Docker volume backup (`/var/lib/docker/volumes:ro`)
+- [x] Docker volume backup + Tier 2 restore (v0.33.0)
 - [ ] Raspberry Pi testing (pi-customer-1)
 - [x] CSRF protection on POST endpoints (v0.23.0)
 - [x] Verbose debug logging across all modules (v0.24.0)