From 518a13592dd942293fa17feada0a583e92ebc98b Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Sat, 24 Oct 2026 20:07:16 +0200 Subject: [PATCH] =?UTF-8?q?R-1252=20(P2):=20a=20rejected=20or=20absent=20b?= =?UTF-8?q?ackup=20block=20never=20deletes=20second-drive=20copies=20?= =?UTF-8?q?=E2=80=94=20reconcile=20only=20on=20a=20valid=20classification;?= =?UTF-8?q?=20a=20rejected=20block=20is=20said=20on=20the=20app's=20status?= =?UTF-8?q?=20and=20once=20a=20day=20to=20the=20operator=20(backup=5Frun?= =?UTF-8?q?=5Ffailures);=20kept=20copies=20stay=20kept=20(H2);=20Remove=20?= =?UTF-8?q?drops=20the=20kept=20records=20it=20deleted=20(H9).=20Commit=20?= =?UTF-8?q?only.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG.md | 20 ++ REUSE.md | 1 + .../cmd/controller/appstop_wiring_test.go | 11 + controller/cmd/controller/main.go | 16 ++ controller/internal/backup/backup.go | 2 + .../r1252_rejected_block_keeps_copies_test.go | 189 ++++++++++++++++++ .../internal/backup/r474_remove_mirrors.go | 1 + controller/internal/backup/tier2.go | 18 +- controller/internal/backup/tier2_capture.go | 12 +- controller/internal/backup/tier2_kept.go | 40 ++++ .../internal/backup/tier2_unreadable.go | 56 ++++++ controller/internal/i18n/locales/en.json | 1 + controller/internal/i18n/locales/hu.json | 1 + .../internal/stacks/classify_wiring_test.go | 23 +++ controller/internal/stacks/metadata.go | 14 ++ controller/scripts/i18n_go_keys.json | 1 + 16 files changed, 400 insertions(+), 6 deletions(-) create mode 100644 controller/internal/backup/r1252_rejected_block_keeps_copies_test.go create mode 100644 controller/internal/backup/tier2_unreadable.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 927ccbc..d9c9d8f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,6 +22,26 @@ brief 2026-10-24, Part 4 (commit only; the release rides the next normal train). `…OnlyWhenTheAgentWaitsAndAnAppRuns`, `…QBLimitRuleSkipsAHeldLimit`, `…TheTwoAppsReadAndWriteTheirLimit`, `TestLineTestRunOutlastsTheClientTimeout`, `TestStartUpwatchRecoversTheHoldFirst`, `TestUpwatchHoldGetsBothAgentCalls`. +### A rejected backup block never deletes second-drive copies (R-1252, P2; review round 2 HUNT-2 H1, H2, H9) — 2026-10-24 night, commit only + +- **One wrong line in an app's backup block no longer deletes its second-drive copy.** LoadMetadata rejects the whole + block; the app fell back to the legacy legs (its appdata only) and the night copy reconciled every other dest dir + away — on 9202 the household's audiobooks copy („removed stale dest dir userdata/media"). `RunTier2` now reconciles + ONLY on a valid classification (`tier2CaptureSetFull` returns `classified`): with a rejected or absent block, + nothing is deleted that night. A REJECTED block (`Metadata.BackupRejected`, `Manager.BackupBlockRejected`, read + through the stack adapter) puts „note.tier2.settings_unreadable_kept" on the app's status and tells the operator + once per app per day through the existing operator-only digest (`backup_run_failures`, leg + `tier2-settings-unreadable`, `SetTier2SettingsUnreadableNotify`) — no new event type, no hub change. +- **A D25 kept copy stays kept when its leg leaves the declared set (HUNT-2 H2):** every kept copy recorded inside + the dest tree joins the reconcile's keep set (`keptCopyRels`). +- **Remove with „remove backups" drops the kept-copy records of what it deleted (HUNT-2 H9)** (`dropKeptCopiesUnder` + in `RemoveTier2Mirrors`), so the backups page never lists a copy that is gone. +- Tests (each seen failing with its rule taken out): `TestR1252_ARejectedBlockKeepsEveryCopy` (the review's 9202 steps: + two control runs, then the bogus path; and the once-a-day signal), `…AnAbsentBlockReconcilesNothing`, + `…AValidClassificationStillReconciles` (the control: §8.2's re-classed leg still goes), `…AKeptCopyStaysWhenItsLegLeaves`, + `…RemoveDropsTheKeptRecordsItDeleted`, `TestR1252_BackupBlockRejectedNamesTheReason` (stacks, the real + LoadMetadata), `TestR1252_MainWiresTheUnreadableSettingsSignal` + a compile-time pin on the adapter. + ### A pending put-back blocks every start (R-1251, P1; review round 2 HUNT-1 H1, H2, H7) — 2026-10-24 night, commit only - **A power cut during a restore of a drive-backed app no longer brings old data back later.** The restore's copy of diff --git a/REUSE.md b/REUSE.md index 1b1ba94..347a192 100644 --- a/REUSE.md +++ b/REUSE.md @@ -83,6 +83,7 @@ | `Manager.RestoreFromRecoveryUnitAt` / `RestoreFromRecoveryUnit` + `UnitRestoreResult` (R-353 v0.226.0; R-102 v0.229.0) | controller/internal/backup/restore_unit.go | `(stack, unitDir string) (UnitRestoreResult, error)` / `(stack string) (…)` | THE local recovery-unit restore, and the facts its surface must state | **Returns a RESULT, not just an error** — volumes replayed, DBs replayed, and what the manifest LISTED. The Manifest* counts are load-bearing: zero-replayed has two causes (the backup held no data / the backup listed data that did not come back) and they are opposite news. Pair it with `unitRestoreOutcomeMsg`; do NOT write a new sentence. **A claim about the APP is forbidden on this path** — it has no `SafetyDump` discriminator, unlike the off-site twin (CONTEXT.md ruling, 07-backup-architecture §6.3). `restoreDockerVolumes` now returns `(int, error)`; `restoreDockerVolumesFrom` is unchanged and still the shared implementation. **R-102: `…At` holds the whole body and takes the unit DIRECTORY; the one-argument form is the thin caller naming the primary unit.** THE SOURCE MOVES; THE DESTINATION DOES NOT — `GetAppDrivePath` still resolves where the data lands. The R-47 mutation order, the unit-over-guest secret precedence, the fail-closed data-key gate and the no-unit `RestoreApp` fallback are all pinned and unchanged. The volume leg goes through the R-354 `volumeReplayFrom` seam so the source directory is assertable without Docker | | `Manager.putBackAfterFailedRestore` + `liveVolumeCopy` (R-893 option A, v0.336.0) | controller/internal/backup/offbox_restore_undo.go | `(stack, cause, live, copy, volumesMoved, filesPlaced, restart, holdKey) error` | THE restore undo — off-site (v0.336.0) and, since D22 (unreleased), the LOCAL unit restore (`RestoreFromRecoveryUnitAtWith`, so also the second drive's and a kept load): the live volumes it will replace, measured before the stop (`planLiveVolumeCopy`), copied right after it (`copyLiveVolumes`), poured back on any failure | One copy per restore under `/backups/restore-undo//` (no capture set reads `backups/`); the put-back uses the restore's own `volumeReplayFrom`/`restoreDockerVolumesFrom`, so there is one volume-replay implementation. A put-back that fails sets `keep` and HOLDS (option C) — never start on volumes that are not known to be back. Seams for tests without Docker: `undoVolumeProbeFn`, `undoVolumeTarFn`, `undoVolumeRemoveFn`; operator notice `SetRestorePutBackNotify` (leg `restore-put-back`). Pinned by `r893_putback_test.go` and `r1177_local_putback_test.go` | | `CrashPutBack` + `Manager.persistUndoForCrash` + `AppStopGuard.SetUndo` (D22 / R-1195, v0.338.0) | controller/internal/backup/restore_crash_putback.go | `NewCrashPutBack(stop, writeDef, logger)`; `PutBackInterruptedRestore(stack, undoDir) error` | A restore cut by a power loss is put back at start: the copy (volumes + `definition/` + `undo.json`) is recorded in the app-stop marker once whole; `Recover` pours it back BEFORE the restart | Call `persistUndoForCrash` right after `copyLiveVolumes` in any new restore path. `Recover` runs before the backup manager exists, so the put-back is wired in main.go with `stackMgr.StopStack` and the stack adapter's definition writer. A failed put-back = a FAILED recovery (app not started, marker kept). The household reads `note.restore.interrupted_put_back` (`LoadRestoreRecordAfterRecovery`). Pinned by `TestR1195_*` | +| `tier2CaptureSetFull(...) (legs, missing, warns, classified)` + `keptCopyRels` + `dropKeptCopiesUnder` (R-1252, unreleased) | controller/internal/backup/tier2_capture.go, tier2_kept.go | `classified` = the legs come from a VALID backup block | **The second-drive reconcile deletes only under a valid classification**; a rejected or absent block keeps every dest dir; kept copies (D25) are always in the keep set; a mirror deleted by Remove drops its kept records | A new reconcile caller must take `classified` and the kept rels too — never reconcile against the legacy legs. A rejected block is told by `Manager.BackupBlockRejected` (stacks; via the adapter's optional `BackupBlockRejected` method) and signalled once a day (`SetTier2SettingsUnreadableNotify`). Pinned by `r1252_rejected_block_keeps_copies_test.go` | | `AppStopGuard.ResumePutBack` / `ResumeWaitingPutBacks` + `HoldReasonPutBackPending` (R-1251, unreleased) | controller/internal/backup/appstop_putback_wait.go | `ResumePutBack(stack) bool` (true = nothing owed, the caller may start); `ResumeWaitingPutBacks() []string` (puts back, then starts through the gated starter) | A cut restore's copy on a drive that is not bound is WAITING: Recover writes the restore hold `putback_pending` (copy in `SafetyDump`) and starts nothing; the put-back runs when the drive binds, before any start | **Any new path that starts an app when its drive arrives calls `ResumePutBack` first** (the boot recreate and `restartStacks` do, in web via `Server.resumePutBack`). Every other start path is covered by the hold store it already honours — do not add a second check. A failed put-back (at start or on resume) writes the D20 hold (`restore_mixed`) and drops the app from the marker. Needs `SetHolds` + `SetUndoReachable` before `Recover` (main.go). Pinned by `r1251_putback_waits_test.go`, `TestR1251_DriveStartPathsPutBackBeforeTheyStart`, `TestR1251_MainWiresThePutBackWait` | | `Manager.RestoreTier2Unit` + `Tier2Coverage.CanRestoreUnit` / `Tier2CopyDate` (R-102/R-103, v0.229.0) | controller/internal/backup/tier2_restore.go | `(stack string) (UnitRestoreResult, error)` / `() bool` / `() (string, bool)` | THE full restore from the SECOND DRIVE's mirror, and the predicate that gates it | **Do NOT widen `CanRestore()`** — it answers only "can the additive file restore run?" and one predicate answering two questions is R-356. Fail-closed: `tier2UnitIsOpenable` requires a parseable manifest, because a directory is not a package. The single-writer flag is taken INSIDE `RestoreFromRecoveryUnitAt` — a second `acquireRunning()` here would refuse the restore it guards. `Tier2CopyDate` prefers the SUCCESS anchor over the attempt clock (R-101) and reports which it returned | | `unitRestoreOutcomeMsg` + its three message constants (R-353, v0.226.0) | controller/internal/web/handlers.go | `(app string, res backup.UnitRestoreResult) string` | THE customer sentence for a completed LOCAL restore | Twin of `reconstituteOutcomeMsg`; copy its SHAPE (clauses earned by having done the thing, no filesystem path, base names only), never its text. The constants are named because `r353_unit_outcome_test.go` asserts them verbatim — a silent edit is how an honest message drifts back into a comforting one, which is the documented history of the sentence it replaces | diff --git a/controller/cmd/controller/appstop_wiring_test.go b/controller/cmd/controller/appstop_wiring_test.go index 49a529b..848eb94 100644 --- a/controller/cmd/controller/appstop_wiring_test.go +++ b/controller/cmd/controller/appstop_wiring_test.go @@ -509,3 +509,14 @@ func TestR1251_MainWiresThePutBackWait(t *testing.T) { t.Errorf("func main() must resume the waiting put-backs after Recover (at %d, Recover at %d)", i, rec) } } + +// R-1252: the second-drive copy asks the stack adapter (by an optional interface) whether a block was REJECTED. An +// adapter that lost the method would make every rejected block read as absent — quiet, with no alarm. Pinned at +// compile time here; the decision is tested in internal/backup. +var _ interface{ BackupBlockRejected(string) string } = (*stackAdapter)(nil) + +func TestR1252_MainWiresTheUnreadableSettingsSignal(t *testing.T) { + if indexOfCall(callsInMain(t, mainBody(t)), "SetTier2SettingsUnreadableNotify") < 0 { + t.Fatal("func main() no longer wires SetTier2SettingsUnreadableNotify — a rejected block would keep the copies silently") + } +} diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index 9a133a2..d576be6 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -1437,6 +1437,17 @@ func main() { "the app is not protected on that tier. Check for a hold on a running app (R-704's shape) or a skipped dump.", app, tier), d) }) + // R-1252: an app's backup block is REJECTED — the second-drive copy kept every copy that night instead of + // reconciling against the legacy legs (which deleted the household's file copies on 9202). Same operator-only + // digest, its own leg; the backup package sends it once per app per day. + backupMgr.SetTier2SettingsUnreadableNotify(func(app, why string) { + d := notify.BackupRunFailuresDetails{RunKind: "tier2", Attempted: 1, Failed: 1, + Apps: []notify.RunFailureDetail{{App: app, Leg: "tier2-settings-unreadable", Reason: why}}} + notifier.NotifyBackupRunFailures(fmt.Sprintf( + "Second-drive copy KEPT, settings unreadable: the backup block of %s is rejected (%s). The night copy deleted nothing "+ + "and every existing copy stays, but new folders are not copied until the block is fixed (R-1252). Fix the template.", + app, why), d) + }) backupMgr.SetOffboxGapNotify(func(gaps map[string][]string) { apps := make([]string, 0, len(gaps)) for app := range gaps { @@ -3093,6 +3104,11 @@ func (a *stackAdapter) GetStackClassifiedBinds(name string) ([]backup.Classified return a.mgr.ClassifiedBinds(name) } +// BackupBlockRejected (R-1252) lets the second-drive copy tell a rejected backup block from an absent one. +func (a *stackAdapter) BackupBlockRejected(name string) string { + return a.mgr.BackupBlockRejected(name) +} + func (a *stackAdapter) ListDeployedStacks() []backup.StackSummary { var result []backup.StackSummary for _, s := range a.mgr.GetStacks() { diff --git a/controller/internal/backup/backup.go b/controller/internal/backup/backup.go index b64b5c6..054f99b 100644 --- a/controller/internal/backup/backup.go +++ b/controller/internal/backup/backup.go @@ -185,6 +185,8 @@ type Manager struct { // hollow (Part D, v0.279.0) — the running apps whose newest copy holds no data. See hollow_watch.go. hollow hollowWatch + // tier2Unreadable (R-1252) dedupes the operator signal for a rejected backup block: once per app per day. + tier2Unreadable tier2UnreadableWatch // F17 restore seams — overridable in tests so the .sql re-import orchestration can be unit-tested // without Docker. Default to the real DiscoverDatabases / ImportDump (lazy-init in reimportDBDumps). diff --git a/controller/internal/backup/r1252_rejected_block_keeps_copies_test.go b/controller/internal/backup/r1252_rejected_block_keeps_copies_test.go new file mode 100644 index 0000000..894ae11 --- /dev/null +++ b/controller/internal/backup/r1252_rejected_block_keeps_copies_test.go @@ -0,0 +1,189 @@ +package backup + +// R-1252 (review round 2, HUNT-2 H1, H2, H9): a rejected backup block never deletes second-drive copies. +// +// The first test is the review's 9202 reproduction (audits/review-round-2-2026-10-24/proofs/H2-bad-backup-block/) as a +// unit test: an app with its household files in a userdata leg; two good runs keep the copy (the control); one bogus +// path in the block rejects it → the legacy legs → before the fix „Tier 2 reconcile: removed stale dest dir +// userdata/media". RED-PROOF (2026-10-24, on the parent of this commit): TestR1252_ARejectedBlockKeepsEveryCopy failed +// on „the household's file copy was deleted"; TestR1252_AKeptCopyStaysWhenItsLegLeaves on „the kept copy was +// reconciled away"; TestR1252_RemoveDropsTheKeptRecordsItDeleted on „the record outlived its copy". + +import ( + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// rejectingProvider is the v2 fixture provider that can also say a block was rejected (the stack adapter's +// BackupBlockRejected in production). +type rejectingProvider struct { + *t2v2Provider + rejected map[string]string +} + +func (p *rejectingProvider) BackupBlockRejected(n string) string { return p.rejected[n] } + +func mUD(rel string) ClassifiedBind { + b := oUD(rel) + b.ReadOnly = false + b.Class = ClassMandatory + return b +} + +func TestR1252_ARejectedBlockKeepsEveryCopy(t *testing.T) { + m, src, target, prov := newTier2V2(t, "audiobookshelf") + rp := &rejectingProvider{t2v2Provider: prov, rejected: map[string]string{}} + m.stackProvider = rp + var sent []string + m.SetTier2SettingsUnreadableNotify(func(app, why string) { sent = append(sent, app+": "+why) }) + prov.has["audiobookshelf"] = true + prov.binds["audiobookshelf"] = []ClassifiedBind{mHDD("appdata/audiobookshelf/config"), mUD("media/audiobooks")} + prov.mounts["audiobookshelf"] = []string{filepath.Join(src, "appdata", "audiobookshelf", "config")} + mustWrite(t, filepath.Join(src, "appdata", "audiobookshelf", "config", "db.sqlite"), "DB") + mustWrite(t, filepath.Join(src, "userdata", "media", "audiobooks", "r1252-marker.mp3"), "the household's book") + destBase := filepath.Join(target, "backups", "secondary", "audiobookshelf") + copyOfBook := filepath.Join(destBase, "userdata", "media", "audiobooks", "r1252-marker.mp3") + + for run := 1; run <= 2; run++ { // the positive control: an unchanged block keeps the copy + if err := m.RunTier2("audiobookshelf"); err != nil { + t.Fatal(err) + } + if !exists(copyOfBook) { + t.Fatalf("control run %d: the household's file was not copied", run) + } + } + // one bogus path: LoadMetadata rejects the WHOLE block → no classification, the legacy legs + prov.has["audiobookshelf"] = false + rp.rejected["audiobookshelf"] = `userdata path "media/nope" matches no compose bind` + if err := m.RunTier2("audiobookshelf"); err != nil { + t.Fatal(err) + } + if !exists(copyOfBook) { + t.Fatal("the household's file copy was deleted by the night copy after a rejected block (R-1252)") + } + if cd := m.settings.GetCrossDriveConfig("audiobookshelf"); cd == nil || !strings.Contains(cd.LastWarning, "nem olvashat") { + t.Fatalf("the app's status must say the settings are unreadable and the copies kept: %+v", cd) + } + if len(sent) != 1 || !strings.Contains(sent[0], "media/nope") { + t.Fatalf("the operator must be told once, with the reason: %v", sent) + } + if err := m.RunTier2("audiobookshelf"); err != nil { + t.Fatal(err) + } + if len(sent) != 1 { + t.Fatalf("a second run the same day must not tell the operator again: %v", sent) + } + m.tier2Unreadable.now = func() time.Time { return time.Now().Add(25 * time.Hour) } + if err := m.RunTier2("audiobookshelf"); err != nil { + t.Fatal(err) + } + if len(sent) != 2 { + t.Fatalf("the next day tells the operator again: %v", sent) + } +} + +// An app with no block at all (absent) — the legacy legs — reconciles nothing either, and says nothing. +func TestR1252_AnAbsentBlockReconcilesNothing(t *testing.T) { + m, src, target, prov := newTier2V2(t, "legacyapp") + prov.has["legacyapp"] = false + prov.mounts["legacyapp"] = []string{filepath.Join(src, "appdata", "legacyapp", "x")} + mustWrite(t, filepath.Join(src, "appdata", "legacyapp", "a.txt"), "A") + destBase := filepath.Join(target, "backups", "secondary", "legacyapp") + stray := filepath.Join(destBase, "userdata", "photos", "p.jpg") + mustWrite(t, stray, "a copy from an earlier, classified night") + if err := m.RunTier2("legacyapp"); err != nil { + t.Fatal(err) + } + if !exists(stray) { + t.Fatal("an app without a valid classification must not reconcile away a copy") + } + if cd := m.settings.GetCrossDriveConfig("legacyapp"); cd != nil && strings.Contains(cd.LastWarning, "nem olvashat") { + t.Fatalf("an absent block is not unreadable settings: %q", cd.LastWarning) + } +} + +// The control the other way: a VALID classification still reconciles a leg that was re-classed excluded. +func TestR1252_AValidClassificationStillReconciles(t *testing.T) { + m, src, target, prov := newTier2V2(t, "app") + prov.has["app"] = true + prov.binds["app"] = []ClassifiedBind{mHDD("appdata/app/keep"), mHDD("appdata/app/drop")} + mustWrite(t, filepath.Join(src, "appdata", "app", "keep", "k"), "K") + mustWrite(t, filepath.Join(src, "appdata", "app", "drop", "d"), "D") + if err := m.RunTier2("app"); err != nil { + t.Fatal(err) + } + dropped := filepath.Join(target, "backups", "secondary", "app", "hdd", "appdata", "app", "drop") + if !exists(dropped) { + t.Fatal("precondition: run 1 copied both legs") + } + prov.binds["app"] = []ClassifiedBind{mHDD("appdata/app/keep"), xHDD("appdata/app/drop")} + if err := m.RunTier2("app"); err != nil { + t.Fatal(err) + } + if exists(dropped) { + t.Fatal("a leg re-classed excluded under a VALID block must still be reconciled away (§8.2)") + } +} + +// HUNT-2 H2: a copy D25 keeps (its folder went missing) stays kept when its leg later leaves the declared set. +func TestR1252_AKeptCopyStaysWhenItsLegLeaves(t *testing.T) { + m, src, target, prov := newTier2V2(t, "app") + prov.has["app"] = true + prov.binds["app"] = []ClassifiedBind{mHDD("appdata/app/photos"), mHDD("appdata/app/docs")} + mustWrite(t, filepath.Join(src, "appdata", "app", "photos", "family.jpg"), "PHOTO") + mustWrite(t, filepath.Join(src, "appdata", "app", "docs", "contract.pdf"), "PDF") + if err := m.RunTier2("app"); err != nil { + t.Fatal(err) + } + photo := filepath.Join(target, "backups", "secondary", "app", "hdd", "appdata", "app", "photos", "family.jpg") + if err := os.RemoveAll(filepath.Join(src, "appdata", "app", "photos")); err != nil { + t.Fatal(err) + } + if err := m.RunTier2("app"); err != nil { // the folder is missing → kept + recorded (D25) + t.Fatal(err) + } + if len(m.KeptCopies()) != 1 { + t.Fatalf("precondition: the photos copy is kept, got %+v", m.KeptCopies()) + } + prov.binds["app"] = []ClassifiedBind{mHDD("appdata/app/docs")} // the photos leg leaves the declared set + if err := m.RunTier2("app"); err != nil { + t.Fatal(err) + } + if !exists(photo) { + t.Fatal("the kept copy was reconciled away while its record stays (HUNT-2 H2)") + } + if len(m.KeptCopies()) != 1 { + t.Fatalf("the kept record must stay: %+v", m.KeptCopies()) + } +} + +// HUNT-2 H9: Remove with „remove backups" deletes the mirror — and the kept-copy records inside it go with it. +func TestR1252_RemoveDropsTheKeptRecordsItDeleted(t *testing.T) { + m, src, target, prov := newTier2V2(t, "app") + prov.has["app"] = true + prov.binds["app"] = []ClassifiedBind{mHDD("appdata/app/photos")} + mustWrite(t, filepath.Join(src, "appdata", "app", "photos", "family.jpg"), "PHOTO") + if err := m.RunTier2("app"); err != nil { + t.Fatal(err) + } + if err := os.RemoveAll(filepath.Join(src, "appdata", "app", "photos")); err != nil { + t.Fatal(err) + } + if err := m.RunTier2("app"); err != nil { + t.Fatal(err) + } + if len(m.KeptCopies()) != 1 { + t.Fatalf("precondition: one kept copy, got %+v", m.KeptCopies()) + } + mirror := filepath.Join(target, "backups", "secondary", "app") + removed := m.RemoveTier2Mirrors("app", []string{mirror}) + if len(removed) != 1 || exists(mirror) { + t.Fatalf("precondition: the mirror is deleted, got %v", removed) + } + if len(m.KeptCopies()) != 0 { + t.Fatalf("the record outlived its copy: %+v", m.KeptCopies()) + } +} diff --git a/controller/internal/backup/r474_remove_mirrors.go b/controller/internal/backup/r474_remove_mirrors.go index 15aa06b..8fbc86d 100644 --- a/controller/internal/backup/r474_remove_mirrors.go +++ b/controller/internal/backup/r474_remove_mirrors.go @@ -89,6 +89,7 @@ func (m *Manager) RemoveTier2Mirrors(stackName string, dirs []string) []string { continue } m.logger.Printf("[INFO] [backup] remove %s: Tier-2 mirror deleted: %s (%s)", stackName, d, size) + m.dropKeptCopiesUnder(d) // R-1252 H9: the kept copies inside it are gone too removed = append(removed, fmt.Sprintf("%s (%s)", d, size)) } return removed diff --git a/controller/internal/backup/tier2.go b/controller/internal/backup/tier2.go index 4796e02..a3a5b76 100644 --- a/controller/internal/backup/tier2.go +++ b/controller/internal/backup/tier2.go @@ -367,7 +367,7 @@ func (m *Manager) RunTier2(stackName string) error { } // Class-driven legs (classified) / resolver legs (legacy) + loud gap warnings (§7-H). - legs, missingLegs, warns := m.tier2CaptureSetFull(stackName, sourceNsRoot) + legs, missingLegs, warns, classified := m.tier2CaptureSetFull(stackName, sourceNsRoot) // Two sizes: full = unit + all legs; state-only = unit + mandatory legs (the SSD ceiling). unitSize := dirSizeBytes(unitDir) @@ -488,8 +488,22 @@ func (m *Manager) RunTier2(stackName string) error { filepath.Join(destBase, filepath.FromSlash(lg.DestRel))) } + // HUNT-2 H2 (R-1252): a copy D25 keeps stays kept even when its leg leaves the declared set — its record is the + // household's only handle on it, and only their „Eltávolítás" removes it. + legRels = append(legRels, m.keptCopyRels(destBase)...) // Reconcile stale dest dirs (a bind removed / re-classed excluded), then write the marker LAST. - m.tier2Reconcile(destBase, legRels) + // R-1252: ONLY on a valid classification. A rejected or absent block yields the legacy legs, and reconciling + // against them deleted every copy of the household's files (9202, 2026-10-24). Nothing is reconciled that night; + // a REJECTED block is said on the app's status and to the operator. + if classified { + m.tier2Reconcile(destBase, legRels) + } else if why := m.backupBlockRejected(stackName); why != "" { + m.logger.Printf("[WARN] [backup] Tier 2 %s: the backup block is REJECTED (%s) — nothing reconciled, every existing copy on the second drive is KEPT (R-1252)", stackName, why) + warns = append(warns, m.note("note.tier2.settings_unreadable_kept")) + m.notifyTier2SettingsUnreadable(stackName, why) + } else if m.isDebug() { + m.logger.Printf("[DEBUG] [backup] Tier 2 %s: no backup classification — nothing reconciled (R-1252)", stackName) + } if err := os.WriteFile(markerPath, []byte(tier2LayoutVersion), 0644); err != nil { m.logger.Printf("[WARN] [backup] Tier 2 %s: layout marker write failed (restore will refuse until next run): %v", stackName, err) } diff --git a/controller/internal/backup/tier2_capture.go b/controller/internal/backup/tier2_capture.go index f8f571b..ecafd1a 100644 --- a/controller/internal/backup/tier2_capture.go +++ b/controller/internal/backup/tier2_capture.go @@ -40,14 +40,18 @@ func tier2DestRel(root appbackup.BindRoot, rel string) string { // footprint promise), mapped into the same v2 layout so restore has ONE reader. Missing/refused // mandatory paths are stat-filtered + surfaced loudly (§7-H). func (m *Manager) tier2CaptureSet(stack, nsRoot string) (legs []tier2Leg, warns []string) { - legs, _, warns = m.tier2CaptureSetFull(stack, nsRoot) + legs, _, warns, _ = m.tier2CaptureSetFull(stack, nsRoot) return legs, warns } // tier2CaptureSetFull is tier2CaptureSet plus the DECLARED legs that are missing on disk (mandatory and optional). // D25 / R-1181 (2026-10-21): a missing declared folder keeps its second-drive copy — RunTier2 adds these to the // reconcile's keep set and lists them as kept, instead of pruning the copy that may be the only one left. -func (m *Manager) tier2CaptureSetFull(stack, nsRoot string) (legs, missing []tier2Leg, warns []string) { +// +// classified (R-1252) reports whether the legs come from a VALID backup classification. Only then may the run +// reconcile (delete) dest dirs the legs no longer name: the legacy set is a fallback, and a block that was rejected +// or could not be read falls back to it too — reconciled, that deleted the household's file copies (measured on 9202). +func (m *Manager) tier2CaptureSetFull(stack, nsRoot string) (legs, missing []tier2Leg, warns []string, classified bool) { // Classified path. if m.stackProvider != nil { if binds, has := m.stackProvider.GetStackClassifiedBinds(stack); has { @@ -75,7 +79,7 @@ func (m *Manager) tier2CaptureSetFull(stack, nsRoot string) (legs, missing []tie warns = append(warns, fmt.Sprintf("Figyelmeztetés: a(z) %s alkalmazás egyes adatmappái nem kerültek a másodlagos mentésbe: %s.", stack, strings.Join(gaps, ", "))) } - return legs, missing, warns + return legs, missing, warns, true } } // Legacy path: the resolver appdata dir(s) → hdd/appdata/ legs (mandatory-equivalent for SSD @@ -98,5 +102,5 @@ func (m *Manager) tier2CaptureSetFull(stack, nsRoot string) (legs, missing []tie Class: appbackup.ClassMandatory, }) } - return legs, missing, warns + return legs, missing, warns, false } diff --git a/controller/internal/backup/tier2_kept.go b/controller/internal/backup/tier2_kept.go index f3ab4c8..3ac7c9a 100644 --- a/controller/internal/backup/tier2_kept.go +++ b/controller/internal/backup/tier2_kept.go @@ -57,6 +57,46 @@ func (m *Manager) releaseKeptCopy(path string) { } } +// keptCopyRels (R-1252, HUNT-2 H2) is the slash-form path, relative to destBase, of every kept copy recorded inside +// it — added to the reconcile's keep set so a kept copy is never reconciled away while its record stays. +func (m *Manager) keptCopyRels(destBase string) []string { + if m.settings == nil { + return nil + } + base := filepath.Clean(destBase) + var out []string + for _, k := range m.settings.GetKeptCopies() { + if filepath.Clean(k.DestBase) != base { + continue + } + rel, err := filepath.Rel(base, filepath.Clean(k.Path)) + if err != nil || rel == "." || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) { + continue + } + out = append(out, filepath.ToSlash(rel)) + } + return out +} + +// dropKeptCopiesUnder (R-1252, HUNT-2 H9) drops the records of kept copies inside dir — called after dir itself was +// deleted (Remove with „remove backups"), so the backups page never lists a copy that is gone. +func (m *Manager) dropKeptCopiesUnder(dir string) { + if m.settings == nil { + return + } + d := filepath.Clean(dir) + for _, k := range m.settings.GetKeptCopies() { + p := filepath.Clean(k.Path) + if p == d || strings.HasPrefix(p, d+string(filepath.Separator)) { + if err := m.settings.DropKeptCopy(k.ID); err != nil { + m.logger.Printf("[WARN] [backup] the record of the deleted kept copy %s could not be dropped: %v", p, err) + continue + } + m.logger.Printf("[INFO] [backup] kept copy %s was deleted with its app's backups — record dropped", p) + } + } +} + // KeptCopies lists the kept copies for the backups page. func (m *Manager) KeptCopies() []settings.KeptCopy { if m.settings == nil { diff --git a/controller/internal/backup/tier2_unreadable.go b/controller/internal/backup/tier2_unreadable.go new file mode 100644 index 0000000..2403729 --- /dev/null +++ b/controller/internal/backup/tier2_unreadable.go @@ -0,0 +1,56 @@ +package backup + +import ( + "sync" + "time" +) + +// R-1252 (review round 2, HUNT-2 H1): a rejected backup block never deletes second-drive copies. RunTier2 reconciles +// only on a valid classification (tier2CaptureSetFull's `classified`); for a REJECTED block it says so on the app's +// status (note.tier2.settings_unreadable_kept) and tells the operator once a day through the existing operator-only +// digest (backup_run_failures, leg tier2-settings-unreadable — no new event type, so no hub change). Pinned by +// r1252_rejected_block_keeps_copies_test.go. + +type tier2UnreadableWatch struct { + mu sync.Mutex + notify func(app, why string) + sent map[string]string // app → the UTC day it was last sent + now func() time.Time +} + +// SetTier2SettingsUnreadableNotify wires the operator signal (main.go → notifier). INIT-ONLY. +func (m *Manager) SetTier2SettingsUnreadableNotify(fn func(app, why string)) { + m.tier2Unreadable.mu.Lock() + m.tier2Unreadable.notify = fn + m.tier2Unreadable.mu.Unlock() +} + +// backupBlockRejected asks the stack provider why the app's backup block was rejected ("" = valid or absent). +func (m *Manager) backupBlockRejected(stack string) string { + if r, ok := m.stackProvider.(interface{ BackupBlockRejected(string) string }); ok { + return r.BackupBlockRejected(stack) + } + return "" +} + +// notifyTier2SettingsUnreadable sends the operator signal at most once per app per UTC day. +func (m *Manager) notifyTier2SettingsUnreadable(app, why string) { + w := &m.tier2Unreadable + w.mu.Lock() + now := time.Now + if w.now != nil { + now = w.now + } + day := now().UTC().Format("2006-01-02") + if w.notify == nil || w.sent[app] == day { + w.mu.Unlock() + return + } + if w.sent == nil { + w.sent = map[string]string{} + } + w.sent[app] = day + fn := w.notify + w.mu.Unlock() + fn(app, why) +} diff --git a/controller/internal/i18n/locales/en.json b/controller/internal/i18n/locales/en.json index 6245894..da81c89 100644 --- a/controller/internal/i18n/locales/en.json +++ b/controller/internal/i18n/locales/en.json @@ -1897,6 +1897,7 @@ "note.tier2.same_disk": "the target you picked is on the same physical disk", "note.backup.single_copy_unproven": "Felhom cannot check that the second drive is really a different disk (inside a virtual machine and behind some USB enclosures this is not visible), so it does not count it as a second copy.", "note.backup.single_copy": "Only one local copy is made: there is no second disk, or it is on the same disk as the data. This protects against mistakes, not against a disk failure. A different disk or the off-site copy protects against a disk failure.", + "note.tier2.settings_unreadable_kept": "The backup settings of the app cannot be read now, so no earlier copy on the second drive was deleted. New folders are not copied until then. Felhom support has been told.", "note.tier2.ssd_partial": "Only the settings, the database and the required data fit on the internal SSD — the optional content was not copied.", "note.tier2.unit_preserved": "The data package on the main drive was incomplete, so the complete one already in the second copy was kept. The copy's data package is therefore older than this backup.", "note.tier2_no_coverage": "This app's data cannot be restored from this copy — the app was not stopped. Use the Start restore button on the Backup → Restore page.", diff --git a/controller/internal/i18n/locales/hu.json b/controller/internal/i18n/locales/hu.json index e039114..e7282a7 100644 --- a/controller/internal/i18n/locales/hu.json +++ b/controller/internal/i18n/locales/hu.json @@ -1885,6 +1885,7 @@ "note.tier2.same_disk": "a kiválasztott cél ugyanazon a fizikai lemezen van", "note.backup.single_copy_unproven": "A Felhom nem tudja ellenőrizni, hogy a második meghajtó valóban másik lemez-e (virtuális gépben és egyes USB-házakban ez nem látszik), ezért nem számolja második másolatnak.", "note.backup.single_copy": "Csak egy helyi másolat készül: nincs második lemez, vagy ugyanazon a lemezen van, mint az adatok. Ez a véletlen törlés ellen véd, lemezhiba ellen nem. Lemezhiba ellen egy másik lemez vagy a távoli mentés véd.", + "note.tier2.settings_unreadable_kept": "Az alkalmazás mentési beállításai most nem olvashatók, ezért a második meghajtón egyetlen korábbi másolat sem törlődött. Az új mappák másolása addig szünetel. A Felhom ügyfélszolgálata értesítést kapott.", "note.tier2.ssd_partial": "A belső SSD-n csak a konfiguráció, adatbázis és a kötelező adatok férnek el — a választható tartalom nem került másolásra.", "note.tier2.unit_preserved": "A fő meghajtón lévő adatcsomag hiányos volt, ezért a másodlagos másolatban meglévő, teljes csomagot megőriztük. A másolat adatcsomagja ezért régebbi, mint ez a mentés.", "note.tier2_no_coverage": "Ennek az alkalmazásnak az adatai nem ebből a másolatból állíthatók vissza — az alkalmazás nem állt le. Használd a Visszaállítás indítása gombot a Biztonsági mentés → Visszaállítás oldalon.", diff --git a/controller/internal/stacks/classify_wiring_test.go b/controller/internal/stacks/classify_wiring_test.go index 11628c0..ef50fcd 100644 --- a/controller/internal/stacks/classify_wiring_test.go +++ b/controller/internal/stacks/classify_wiring_test.go @@ -215,3 +215,26 @@ func TestCatalogBlocks_Fidelity(t *testing.T) { }) } } + +// R-1252: the second-drive copy must tell a REJECTED block from an absent one (a rejected one keeps every copy and +// alarms; an absent one is quiet). Through the REAL LoadMetadata path. RED-PROOF: drop the BackupRejected assignment +// in LoadMetadata → the rejected fixture reads "" and this fails. +func TestR1252_BackupBlockRejectedNamesTheReason(t *testing.T) { + compose := `services: + app: + volumes: + - ${USERDATA_PATH}/media/audiobooks:/audiobooks +` + bad := newClassifyManager(t, "display_name: App\nbackup:\n userdata:\n - path: media/nope\n class: mandatory\n", compose) + if why := bad.BackupBlockRejected("app"); why == "" { + t.Fatal("a block whose path matches no compose bind must be reported as rejected, with its reason") + } + good := newClassifyManager(t, "display_name: App\nbackup:\n userdata:\n - path: media/audiobooks\n class: mandatory\n", compose) + if why := good.BackupBlockRejected("app"); why != "" { + t.Fatalf("a valid block is not rejected, got %q", why) + } + none := newClassifyManager(t, "display_name: App\n", compose) + if why := none.BackupBlockRejected("app"); why != "" { + t.Fatalf("an absent block is not rejected, got %q", why) + } +} diff --git a/controller/internal/stacks/metadata.go b/controller/internal/stacks/metadata.go index 89542b3..b150c2f 100644 --- a/controller/internal/stacks/metadata.go +++ b/controller/internal/stacks/metadata.go @@ -92,6 +92,9 @@ type Metadata struct { // block (sets this back to nil + logs one ERROR) on any validation defect, so a bad catalog push // degrades to legacy loudly rather than partially classifying. Consumed by Task 3/4 — INERT today. Backup *appbackup.BackupSpec `yaml:"backup,omitempty" json:"backup,omitempty"` + // BackupRejected (R-1252) is why LoadMetadata rejected the backup block ("" = not rejected). Never read from the + // file: the second-drive copy must tell a REJECTED block (keep every copy, alarm) from an absent one. + BackupRejected string `yaml:"-" json:"-"` // DataPaths is the customer-facing folder ANNOTATION (R-75): role + Hungarian label over paths // that must ALREADY exist as compose binds. It never declares a path. Validated in LoadMetadata // with the Fork-3 asymmetry — a malformed PATH rejects the whole block, an unknown ROLE drops @@ -551,9 +554,11 @@ func loadMetadata(stackDir string, validateBackup bool) Metadata { if composeErr != nil { log.Printf("[ERROR] [stacks] .felhom.yml backup block rejected in %s: docker-compose.yml unreadable: %v", stackDir, composeErr) meta.Backup = nil + meta.BackupRejected = "docker-compose.yml unreadable: " + composeErr.Error() } else if err := appbackup.ValidateBackupSpec(meta.Backup, binds); err != nil { log.Printf("[ERROR] [stacks] .felhom.yml backup block rejected in %s: %v", stackDir, err) meta.Backup = nil + meta.BackupRejected = err.Error() } } // data_paths (R-75) rides the same choke point, with its own asymmetric rules: a malformed @@ -597,6 +602,15 @@ func (m *Manager) ClassifiedBinds(name string) ([]appbackup.ClassifiedBind, bool return appbackup.ClassifyBinds(meta.Backup, binds) } +// BackupBlockRejected (R-1252) is why this app's backup block was rejected, or "" when it is valid or absent. +func (m *Manager) BackupBlockRejected(name string) string { + stack, ok := m.GetStack(name) + if !ok || name == SambaStackName { + return "" + } + return LoadMetadata(filepath.Dir(stack.ComposePath)).BackupRejected +} + // ExcludedDockerVolumes returns the Docker names of the stack's named volumes its template excludes from every app-data backup // (R-49, D21 = B; appbackup.ExcludedDockerVolumes), and a sentence per template entry that was ignored. func (m *Manager) ExcludedDockerVolumes(name string) ([]string, []string) { diff --git a/controller/scripts/i18n_go_keys.json b/controller/scripts/i18n_go_keys.json index e63d204..b31fdb7 100644 --- a/controller/scripts/i18n_go_keys.json +++ b/controller/scripts/i18n_go_keys.json @@ -11,6 +11,7 @@ "err.backup.db_restore_failed_held_mixed": "BORN AS A KEY (R-893, 2026-10-08) -- a NEW sentence for a failed off-site replay whose database rollback worked but whose files/volumes/version had already moved; the app is held. Never a Go literal; pinned by internal/backup/r893_mixed_restore_test.go.", "err.backup.restore_failed_held_mixed": "BORN AS A KEY (R-893, 2026-10-08, security review) -- a restore that stopped half-way (placement, volume replay or DB start failed) after files/volumes/version had moved; the app is HELD. Pinned by internal/backup/r893_mixed_restore_test.go.", "note.restore.interrupted_put_back_waiting": "BORN AS A KEY (R-1251, 2026-10-24) -- the restore record sentence for an interrupted restore whose put-back waits for the app's drive. Never a Go literal; pinned by TestR1251_TheRecordSaysThePutBackWaits.", + "note.tier2.settings_unreadable_kept": "BORN AS A KEY (R-1252, 2026-10-24) -- the second-drive status note when a rejected backup block made the night copy keep every copy. Never a Go literal; pinned by r1252_rejected_block_keeps_copies_test.go.", "hold.putback_pending": "BORN AS A KEY (R-1251, 2026-10-24) -- the hold sentence for an app whose cut restore waits for its drive before its pre-restore data is put back (HoldReasonPutBackPending). Never a Go literal; pinned by internal/backup/r1251_putback_waits_test.go.", "note.reconstitute.held_mixed": "BORN AS A KEY (R-893, 2026-10-08) -- the hold sentence for that held app (HoldReasonRestoreMixed). Never a Go literal; pinned by internal/backup/r893_mixed_restore_test.go.", "recovery.older_unchecked": "BORN AS A KEY (R-304, 2026-10-08) -- a NEW sentence of the recovery screen for the agent's 424 (earlier sealed packages not all checked), never a Go literal; pinned by TestR304_OlderUnchecked_IsNotAWrongCode.",