From 45bff0184f0c567a46203e20f8fd1ccd2509ddb8 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Sun, 12 Jul 2026 09:18:21 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20backup-architecture=20=E2=80=94=20F7=20?= =?UTF-8?q?atomic=20volume=20dumps=20+=20F6=20no-single-copy=20+=20F5=20st?= =?UTF-8?q?ale=20sweep=20+=20NAS=20locality=20(Task=20B)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_017CDMFpFx84pfviCTVuGGhf --- documentation/controller/backup-architecture.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/documentation/controller/backup-architecture.md b/documentation/controller/backup-architecture.md index 940878c..e89ae19 100644 --- a/documentation/controller/backup-architecture.md +++ b/documentation/controller/backup-architecture.md @@ -39,6 +39,10 @@ The agent surface the controller talks to is `internal/agentapi/client.go`: a TL **Volume dumps are real again (F3, v0.99.0).** After the restic removal, `DumpAppVolumesSafe` had no caller — no trigger ever produced `volume-dumps/`, so named-volume app data was never captured (drill finding F3). `runVolumeDumps` now runs inside the same nightly/manual backup run, per deployed stack: skip if protected (`IsProtectedStack`), skip if it has **no named volumes** (this check deliberately precedes `DumpAppVolumesSafe`, which stops the stack before its own check), skip disconnected/decommissioned drives, else stop → tar each volume (`docker run alpine tar`) → restart. It runs BEFORE the unit capture so `manifest.VolumeDumps` enumerates the fresh tars. A per-stack failure lands in the run summary (`FAIL volumes:`), flips the run's Success flag and fails the run — no silent partials. Note the operational consequence: volume-bearing apps are briefly stopped during each nightly backup (the locked stop-first policy — a live tar of a database volume would be torn). +**Atomic volume dumps (F7, v0.118.0, CAMPAIGN-3).** The volume dump now has the SAME crash-safety the DB dump has always had: `DumpAppVolumes` writes the tar to `.tar.tmp` (`docker run … tar cf …/.tar.tmp`), then `atomicPromoteTar` fsyncs the tmp (and best-effort the dir) and `os.Rename`s it over the final `.tar` **only on success**. Before this, tar wrote the `.tar` IN PLACE, so a mid-write NFS cut left a **0-byte tar replacing the last good dump** — and a tier-1 restore is replace-semantics, so the only "restore point" then restored an empty volume (the exact CAMPAIGN-3 F7 failure: a `exportfs -u` during a dump truncated a 247 MB calibre tar to 0 bytes). Now any tar error / timeout / dead-NFS EIO removes only the `.tmp`; the last good `.tar` is byte-untouched. The `.tar.tmp` name ends `.tmp` (not `.tar`), so it is invisible to the restore-point and stale scans; orphan `.tar.tmp` from a killed run is swept on the next dump. **Live-proven 2026-07-12:** an `exportfs -u` mid-volume-dump left all NAS volume tars byte-identical (sha unchanged), no 0-byte file, run `success:false`; the next run produced fresh good tars. + +**Stale-primary sweep (F5, v0.118.0).** After the units are refreshed, `pruneStalePrimaryDirs` removes an orphaned `backups/primary/` dir an app left on an OLD drive when its `HDD_PATH` moved (invisible disk residue). Guards: removes only when the app is deployed AND its current namespace root differs from the dir's drive; never the current-drive dir (the live restore point) or an undeployed app's dir; strictly under a `backups/primary/` prefix. + ## 3. Recovery units (`internal/backup/recovery_unit.go`) A recovery unit is a per-app, **secret-free**, self-contained directory at `/backups/primary//` (`paths.go:42`). It contains: @@ -87,6 +91,12 @@ The portable `.fab` import (`internal/appexport/`) validates every manifest path Tier 2 is the only off-drive protection browsable HDD userdata can get (PBS cannot reach bind mounts, `tier2.go:17-23`). It is an `rsync -a --delete` **mirror** (`rsyncMirror`, `tier2.go:358`) of an HDD app's recovery unit + bulk `appdata/` to `/backups/secondary//{recovery-unit,appdata}/` on a **different physical disk**. +**No single-copy backups (F6, v0.118.0, CAMPAIGN-3).** `RunAllTier2` used to `continue` on every non-HDD app, so a **volume-only** app (no `HDD_PATH`, its dumps on `sys_drive`) got NO tier-2 copy — a single controller-level copy on one device. It now flows through too: its recovery unit (holding the db/volume dumps) gets the cross-drive second copy like any HDD app (live-proven 2026-07-12: `actualbudget`/`seerr` appear under `felhom-usb/backups/secondary/`). A sys_drive app's restore-point drive label is now the clear "Belső SSD (rendszer)", never blank. + +**3-2-1 honesty on a single-drive box (F6).** When NO off-drive target exists (`Manager.hasOffDriveTarget` false — only the system drive, no enrolled second disk), there is genuinely only ONE local copy. `FullBackupStatus.SingleCopyWarning` surfaces an honest Hungarian banner on the backup page ("Csak egy másolat készül (nincs második meghajtó) — a 3-2-1 mentéshez csatlakoztasson egy második meghajtót vagy offsite tárolót.") rather than implying a 3-2-1 guarantee the box cannot keep. + +**NAS backup locality (Part 4 — decision A, v0.118.0).** A NAS-resident app's tier-1 dumps live **on the NAS itself** (`nas-media/backups/primary/`), beside the data. During a NAS outage both the app data AND its freshest tier-1 dump are on the dead device — the tier-2 cross-drive copy to a local drive is the off-NAS leg that saves them, **and only after it has run**. This locality was kept deliberately in the operator fork (over retargeting tier-1 to a local drive, which would have moved restore-point resolution off the drive); the tier-2 copy is the mitigation. Documented here so the outage window is never a surprise. + **Auto-target selection** (`selectTier2Target`, `tier2.go:54`), in order: 1. A customer-pinned target (`PreferredTarget` from the config panel) if it is still registered, schedulable, and off-disk (`tier2.go:62-83`).