161 lines
6.0 KiB
Markdown
161 lines
6.0 KiB
Markdown
# TASK.md — Cross-Drive Backup Improvements (v0.12.6)
|
||
|
||
## Prompt (copy-paste this into Claude Code)
|
||
|
||
```
|
||
Read TASK.md for the full plan. Apply all code changes described, then build and deploy.
|
||
After all fixes are done:
|
||
1. Run `go build ./...` and `go vet ./...` from the controller/ directory — fix any errors
|
||
2. Update CHANGELOG.md with a new entry at the top (session 42, v0.12.6)
|
||
3. Commit, build, and deploy following the workflow in CLAUDE.md
|
||
```
|
||
|
||
---
|
||
|
||
## Context
|
||
|
||
The cross-drive backup for Immich was fixed earlier today (mount-point validation + system-drive
|
||
space thresholds). During testing, two more issues were found:
|
||
|
||
1. **Redundant destination folder nesting** — rsync creates
|
||
`backups/rsync/immich/storage/immich/<data>` instead of `backups/rsync/immich/<data>`
|
||
2. **DB backups backed up twice** — Immich stores its own DB dumps in
|
||
`/mnt/hdd_1/storage/immich/backups/` (~16 MB each). The cross-drive rsync copies these as part of the user data, but the controller already handles DB backups separately via pg_dump.
|
||
|
||
|
||
## Fix 1: Simplify rsync destination path structure (crossdrive.go)
|
||
|
||
**File:** `internal/backup/crossdrive.go`, function `runRsyncBackup`, lines ~206–217
|
||
|
||
**Problem:** The path-stripping logic strips only the first 2 segments of the source path
|
||
(e.g., `mnt/hdd_1`) and keeps everything else as a relative subpath:
|
||
|
||
```go
|
||
parts := strings.SplitN(strings.TrimPrefix(srcMount, "/"), "/", 3)
|
||
if len(parts) >= 3 {
|
||
rel = parts[2] // "storage/immich" — redundant nesting!
|
||
}
|
||
```
|
||
|
||
For source `/mnt/hdd_1/storage/immich`, this creates:
|
||
```
|
||
backups/rsync/immich/storage/immich/ ← "storage/immich" repeats context
|
||
```
|
||
|
||
Expected:
|
||
```
|
||
backups/rsync/immich/ ← data goes directly here (single mount)
|
||
```
|
||
|
||
**Fix:** Use `filepath.Base()` as the subdirectory name. If the app has only one mount,
|
||
rsync directly into the stack folder; if multiple, use basenames to keep them separate.
|
||
|
||
Replace the path-stripping block (lines ~206–219) with:
|
||
|
||
```go
|
||
for i, srcMount := range mounts {
|
||
var dstPath string
|
||
if len(mounts) == 1 {
|
||
// Single mount: rsync directly into the stack folder
|
||
dstPath = destDir
|
||
} else {
|
||
// Multiple mounts: use the leaf directory name as subfolder
|
||
// Disambiguate if needed by appending index
|
||
leaf := filepath.Base(srcMount)
|
||
dstPath = filepath.Join(destDir, leaf)
|
||
// Check for duplicate leaf names (unlikely but safe)
|
||
if i > 0 {
|
||
if _, err := os.Stat(dstPath); err == nil {
|
||
dstPath = filepath.Join(destDir, fmt.Sprintf("%s_%d", leaf, i))
|
||
}
|
||
}
|
||
}
|
||
if err := os.MkdirAll(dstPath, 0755); err != nil {
|
||
return fmt.Errorf("creating rsync destination: %w", err)
|
||
}
|
||
|
||
// Ensure trailing slash on source for rsync semantics (copy contents, not the dir itself)
|
||
src := strings.TrimRight(srcMount, "/") + "/"
|
||
dst := strings.TrimRight(dstPath, "/") + "/"
|
||
|
||
cmd := exec.CommandContext(ctx, "rsync", "-a", "--delete", src, dst)
|
||
r.logger.Printf("[DEBUG] rsync: %s → %s", src, dst)
|
||
if out, err := cmd.CombinedOutput(); err != nil {
|
||
return fmt.Errorf("rsync failed for %s: %v (%s)", srcMount, err, strings.TrimSpace(string(out)))
|
||
}
|
||
}
|
||
```
|
||
|
||
Remove the old `rel` variable and the `SplitN` block entirely. Also remove the
|
||
`os.MkdirAll(dstPath, 0755)` that was inside the old loop since it's now in the new block.
|
||
|
||
**Result after fix:**
|
||
```
|
||
/mnt/hdd_placeholder/backups/rsync/immich/
|
||
├── backups/ ← Immich's internal DB dumps (will be excluded in Fix 4)
|
||
├── encoded-video/
|
||
├── library/
|
||
├── profile/
|
||
├── thumbs/
|
||
└── upload/
|
||
```
|
||
|
||
**Impact on existing backups:** The first rsync after this change will create the new
|
||
flat structure. The old nested `storage/immich/` subfolder inside `backups/rsync/immich/`
|
||
will remain orphaned (rsync `--delete` only deletes within the target, not sibling dirs).
|
||
This is fine — no data loss, and the old folder can be cleaned up manually.
|
||
|
||
---
|
||
|
||
## Fix 2: Exclude app-internal DB backups from rsync (crossdrive.go)
|
||
|
||
**File:** `internal/backup/crossdrive.go`, function `runRsyncBackup`
|
||
|
||
**Problem:** Many apps store their own periodic DB dumps inside their data directory:
|
||
- Immich: `storage/immich/backups/` (64 MB of daily postgres dumps)
|
||
- Other apps may follow similar patterns
|
||
|
||
The controller already handles DB backups separately via `pg_dump` (the "Adatbázis mentés"
|
||
feature). Copying the app's internal DB dumps via rsync is redundant and wastes space.
|
||
|
||
Immich's internal backup path: `<data>/backups/*.sql.gz` (created by Immich itself daily).
|
||
|
||
**Fix:** Add `--exclude` flags to the rsync command for common app-internal backup patterns.
|
||
|
||
In the rsync `exec.CommandContext` call, add excludes:
|
||
|
||
```go
|
||
cmd := exec.CommandContext(ctx, "rsync", "-a", "--delete",
|
||
"--exclude", "backups/*.sql.gz",
|
||
"--exclude", "backups/*.sql",
|
||
"--exclude", "backups/*.dump",
|
||
src, dst)
|
||
```
|
||
|
||
This excludes only DB dump files inside `backups/` subdirectories — not the `backups/`
|
||
directory itself (which might contain non-DB files), and not any other `*.sql.gz` files
|
||
outside of `backups/`. This is conservative and safe.
|
||
|
||
**Note:** The `.immich` marker file and the `backups/` directory structure itself are
|
||
preserved — only the large dump files are excluded.
|
||
|
||
---
|
||
|
||
## Files to modify
|
||
|
||
1. `internal/backup/crossdrive.go` — `runRsyncBackup()` (Fix 3 + Fix 4)
|
||
|
||
## Post-fix checklist
|
||
|
||
- [ ] `go build ./...` passes
|
||
- [ ] `go vet ./...` passes
|
||
- [ ] Update `CHANGELOG.md` — session 42, version **v0.12.6**, describe ALL fixes:
|
||
- Fix 1: ValidateDestination allows non-mount-point destinations with warning
|
||
- Fix 2: System-drive space thresholds (10 GB / 90%) in both runner and web UI
|
||
- Fix 3: Simplified rsync destination path (flat structure per app)
|
||
- Fix 4: Exclude app-internal DB dumps from rsync
|
||
- [ ] Update `controller/README.md` backup section with the new architecture
|
||
- [ ] Commit, build on 192.168.0.180, deploy on 192.168.0.162
|
||
- [ ] Verify with `docker ps` and `docker logs`
|
||
- [ ] After deploy, run manual Immich backup and verify new folder structure
|