From 2d63714eca68b92c18e5b67bce169bd1336695ad Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Tue, 6 Oct 2026 11:30:01 +0200 Subject: [PATCH] R-645: the night backup skips an app whose pinned version is not the one it runs (09 decision 142) Every night leg (DB dump, volume dump, recovery-unit capture, Tier-2 mirror) now leaves alone an app whose app.yaml pin (pinned_images) differs from its running record (installed_images) - the state a failed update leaves behind. A hold lifted by hand (--clear-restore-hold + restart) no longer lets the capture write the just-failed definition over the good unit. Unknown (no pin, no record, a service not observed) never skips. The log says it per leg; the backups page shows one amber line, hu + en (backup.status.version_skip). Seam: backup.Manager.SetVersionCheck <- stacks.Manager.PinNotRunning. Tests: TestR645_HandLiftedHoldKeepsTheGoodUnit (whole night run + Tier 2, unit tree fingerprint), TestR645_VersionSkipSentence, TestR645_PinNotRunning_*, TestR645_BackupRowSaysTheNightBackupSkipsIt, TestR645_VersionCheckIsWiredAtStartup. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS --- REUSE.md | 2 +- controller/cmd/controller/main.go | 2 + .../cmd/controller/slice4_wiring_test.go | 9 ++ controller/internal/backup/backup.go | 16 ++ .../internal/backup/r645_version_skip_test.go | 141 ++++++++++++++++++ controller/internal/backup/recovery_unit.go | 11 ++ controller/internal/backup/tier2.go | 5 + controller/internal/backup/update_guard.go | 33 ++++ controller/internal/i18n/locales/en.json | 1 + controller/internal/i18n/locales/hu.json | 1 + controller/internal/stacks/pin.go | 56 +++++++ controller/internal/stacks/pin_r645_test.go | 39 +++++ controller/internal/web/handlers.go | 6 + .../web/r645_version_skip_row_test.go | 34 +++++ controller/scripts/i18n_go_keys.json | 1 + 15 files changed, 356 insertions(+), 1 deletion(-) create mode 100644 controller/internal/backup/r645_version_skip_test.go create mode 100644 controller/internal/stacks/pin_r645_test.go create mode 100644 controller/internal/web/r645_version_skip_row_test.go diff --git a/REUSE.md b/REUSE.md index 80e2f96..542a914 100644 --- a/REUSE.md +++ b/REUSE.md @@ -161,7 +161,7 @@ | `backup.Manager.UpdateRestorePoints` + `CanBackUpApp` (v0.239.0, R-475) | controller/internal/backup/update_guard.go | `(ctx, stack, accept func(UpdateTierPoint) bool) (UpdateTierPoint, bool, []UpdateTierPoint)` | "which backup can this update lean on" — walks Tier 2, 1, 3 and returns the first copy `accept` admits | **The age rule lives in the caller's `accept`** (stacks' `freshRestorePoint`), so it is ONE rule for every tier. **The ORDER is `UpdateTierOrderFor` (v0.241.0, R-479): 2 → 3 → 1 for an app with classified binds (`DataOutsideUnit`), 2 → 1 → 3 otherwise; `UpdateCopyHolds` is the matching phrase for the hold.** Stops at the first accepted copy, so Tier 3 (restic, 15 s bound, unreachable = absent + WARN) is reached only when needed. Seams: `updateTier2PointFn`, `updateTier1PointsFn`, `updateOffsiteTimesFn` (v0.240.0: Tier 3 reads `OffsiteSnapshotTimes` — snapshots only, never the inventory's per-app `stats`). stacks adds R-478's rule: a copy older than `deployed_at` does not count (`usableRestorePoint`). Tier numbers are pinned equal across stacks/backup by `TestR475_TierConstantsAgree` | | `backup.Manager.Tier2MirrorDirsForApp` / `RemoveTier2Mirrors` + `settings.DeleteAppBackupPrefs` (v0.240.0, R-474 / R-486) | controller/internal/backup/r474_remove_mirrors.go | `(stack) []string`; `(stack, dirs) []string` | deleting an app's backups on removal — the Tier-2 mirror lives on ANOTHER drive, outside RemoveStack's per-app base; the same dirs size the backup card (R-485) | Read the mirror dirs BEFORE the prefs are forgotten. Deletes only `/backups/secondary/` for a known root; never `_shares`, never a path that merely cleans to it. **The Tier-2 RECORD goes only with `remove_backups`** (R-486) — a removal that keeps the backups must keep the record, or the mirror is unrestorable. Pinned by `TestR474_RemoveHandlerDeletesUnitMirrorAndPrefs` + `TestR486_RemovalKeepsTheTier2RecordUnlessBackupsGo` | | `appbackup.dbTypeForImage` (R-484, v0.240.0) | controller/internal/appbackup/dbservices.go | `(image) (DBType, bool)` | the ONE place an image is judged a database — nightly dumps, pre-update safety dump, DB-only replay | Derived Postgres images (`postgis`, `pgvector`, `timescaledb`) are Postgres. A new engine image goes HERE and in `dbservices_test.go`'s table, never in a second matcher | -| `backup.Manager.HoldAfterFailedUpdate` / `RunAppBackupNow` / `WriteUpdateSafetyDump` / `UpdateBusy` (v0.237.0) | controller/internal/backup/update_guard.go | see file | the update's hold, per-app backup-now, safety dump, busy check | The hold is `settings.RestoreHold` with `Reason: update_failed` — SAME store and gate as R-379, never a second map. `RunAppBackupNow` composes the nightly legs for ONE app (admission, DB dump, volume dump, capture, Tier-2) — do not write a second backup orchestration. A successful unit restore lifts an UPDATE hold only. **`isHeld` is ALSO true while a guarded update is moving the app (`SetUpdatingCheck`, v0.238.1)** — found live: the periodic capture overwrote a primary unit during a health wait | +| `backup.Manager.HoldAfterFailedUpdate` / `RunAppBackupNow` / `WriteUpdateSafetyDump` / `UpdateBusy` (v0.237.0) | controller/internal/backup/update_guard.go | see file | the update's hold, per-app backup-now, safety dump, busy check | The hold is `settings.RestoreHold` with `Reason: update_failed` — SAME store and gate as R-379, never a second map. `RunAppBackupNow` composes the nightly legs for ONE app (admission, DB dump, volume dump, capture, Tier-2) — do not write a second backup orchestration. A successful unit restore lifts an UPDATE hold only. **`isHeld` is ALSO true while a guarded update is moving the app (`SetUpdatingCheck`, v0.238.1)** — found live: the periodic capture overwrote a primary unit during a health wait. **R-645: every night leg (DB dump, volume dump, capture, Tier-2) also skips an app whose pin is not what it runs (`versionSkip`, wired by `SetVersionCheck` → `stacks.Manager.PinNotRunning`); unknown never skips. The page line is `VersionSkipFor`** | | `stacks.Manager.memoryVerdict` (v0.237.0) | controller/internal/stacks/deploy.go | `(newReq, newLimit, releasedReq, releasedLimit int) (refusal, warning string)` | the deploy's memory check, shared with the update | An update RELEASES the app's current request first. Deploy passes `0, 0` and is byte-identical in wording and log line | | `Syncer.SetRenderPlanFn` + `renderSource` (v0.235.0) | controller/internal/sync/sync.go | `func(appName string) stacks.RenderPlan` | the catalog render table | **NIL-SAFE: no seam = copy verbatim = the old product.** Catalog images == pin → verbatim (fixes flow + self-healing, both deliberately kept); differ → the WHOLE stored definition, **never a ref substitution into a newer template** (`wger 2.6`). `.felhom.yml` always verbatim (R-458). The syncer must NEVER read app.yaml. Re-reads the applied file before writing it — a test caught it writing an empty compose over a live app | | `Stack.CatalogImages` vs `Stack.TemplateImages` (v0.235.0) | controller/internal/stacks/manager.go | both `map[string]string` | badge input vs "what the next `up -d` gives this app" | **THE TRAP: same type, same shape, opposite meaning after the freeze.** `TemplateImages` reads the LIVE (possibly frozen) compose file; `CatalogImages` reads the syncer's clone. `web.compareInstalledToTemplate` MUST use `CatalogImages` or it answers „Naprakész" on exactly the apps that are behind, with every test green. Red-proved | diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index 58ba80c..70f106a 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -600,6 +600,8 @@ func main() { // updated, not only once it is held — found live in Scenario F, see backup.Manager.isHeld. if backupMgr != nil { backupMgr.SetUpdatingCheck(stackMgr.IsUpdating) + // R-645 (09 §3 decision 142): the night backup leaves an app that is not running its pinned version alone. + backupMgr.SetVersionCheck(stackMgr.PinNotRunning) // R-671 (v0.272.0): a restore that lifts an update hold removes the undo copies that hold kept. backupMgr.SetUndoCopyRemover(stackMgr.RemoveUndoCopies) } diff --git a/controller/cmd/controller/slice4_wiring_test.go b/controller/cmd/controller/slice4_wiring_test.go index eb48d29..075a184 100644 --- a/controller/cmd/controller/slice4_wiring_test.go +++ b/controller/cmd/controller/slice4_wiring_test.go @@ -111,3 +111,12 @@ func TestSlice4_UpdatingCheckIsWiredAtStartup(t *testing.T) { t.Fatal("backupMgr.SetUpdatingCheck is never called — the nightly legs cannot see an update in progress") } } + +// R-645 (09 §3 decision 142): the backup manager must be told which apps are not running their pinned +// version, or the night backup writes a just-failed definition over the good unit after a hand-lifted hold. +func TestR645_VersionCheckIsWiredAtStartup(t *testing.T) { + lines, _, _ := slice4CallLines(t) + if len(lines["SetVersionCheck"]) == 0 { + t.Fatal("backupMgr.SetVersionCheck is never called — the night backup cannot see an app off its pin") + } +} diff --git a/controller/internal/backup/backup.go b/controller/internal/backup/backup.go index 30910a4..30d6f36 100644 --- a/controller/internal/backup/backup.go +++ b/controller/internal/backup/backup.go @@ -202,6 +202,8 @@ type Manager struct { // updatingCheck (slice 4) — nil-safe; see isHeld / SetUpdatingCheck. updatingCheck func(stackName string) bool + // versionCheck (R-645) — nil-safe; see versionSkip / SetVersionCheck. + versionCheck func(stackName string) (detail string, mismatch bool) // undoCopyRemover (R-671, v0.272.0) deletes an app's leftover undo copies — stacks.Manager.RemoveUndoCopies, // wired in main.go (SetUndoCopyRemover). nil-safe: without it the copies stay, as before. undoCopyRemover func(stackName string) int @@ -620,6 +622,14 @@ func (m *Manager) runDBDumpsInternal(ctx context.Context) error { continue } + // R-645 (09 §3 decision 142): an app not running its pinned version is left alone — its dump + // would land in the unit beside the good definition and become the restore point. A SKIP: the + // unit keeps the last good copy, and the backups page says why. + if why, skip := m.versionSkip(db.StackName); skip { + m.logger.Printf("[WARN] [backup] Skipping DB dump for %s — it is not running its pinned version (%s); the last good backup is kept (R-645)", db.StackName, why) + summary = append(summary, fmt.Sprintf("SKIP %s (not running its pinned version)", db.ContainerName)) + continue + } // R-181: the reserve, BEFORE the first byte of this app's backup is written. This is usually // where an app's verdict is taken, because the DB leg runs first; the volume leg and the // capture then read the same memo. SKIP, not FAIL — a deliberate hold is not a broken dump, @@ -761,6 +771,12 @@ func (m *Manager) runVolumeDumps() (summary []string, dumped int, allOK bool) { summary = append(summary, fmt.Sprintf("SKIP %s volumes (held)", stack.Name)) continue } + // R-645: never stop, dump or restart an app that is not running its pinned version. + if why, skip := m.versionSkip(stack.Name); skip { + m.logger.Printf("[WARN] [backup] Skipping volume dump for %s — it is not running its pinned version (%s); the last good backup is kept (R-645)", stack.Name, why) + summary = append(summary, fmt.Sprintf("SKIP %s volumes (not running its pinned version)", stack.Name)) + continue + } // Volume check FIRST — a volume-less stack must not be stopped at all (see gate-order note). if len(m.stackProvider.GetDockerVolumes(stack.Name)) == 0 { if m.isDebug() { diff --git a/controller/internal/backup/r645_version_skip_test.go b/controller/internal/backup/r645_version_skip_test.go new file mode 100644 index 0000000..07ef698 --- /dev/null +++ b/controller/internal/backup/r645_version_skip_test.go @@ -0,0 +1,141 @@ +package backup + +import ( + "context" + "log" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// R-645 (09 §3 decision 142) — the night backup skips an app whose pinned version is not the version +// it runs. +// +// THE HAND-LIFT SHAPE, measured 2026-09-23 on 9202: docmost was HELD after a failed 0.95.0 → 0.96.0 +// update; `--clear-restore-hold docmost` + the restart ran, and three seconds later the capture wrote +// the 0.96.0 definition into the unit the hold sentence had named. This drives the whole night run +// (DB leg, volume leg, capture) and then Tier 2 over that state — the hold is gone, the stack dir names +// the failed version, the pin says 0.96.0, the running record says 0.95.0 — and asserts the +// CONSEQUENCE: the good unit's tree is byte-identical afterwards, and its compose still names 0.95.0. +// +// COMPANION RED-PROOF: make versionSkip return ("", false) — the unit's compose/docker-compose.yml then +// names docmost/docmost:0.96.0 and the tree fingerprint differs, and this test fails. +func TestR645_HandLiftedHoldKeepsTheGoodUnit(t *testing.T) { + h := newAdmissionHarness(t, "docmost", "free") + h.m.settings = slice4Settings(t) + ns := h.nsRoot() + + // The good unit: written by an earlier backup of 0.95.0. Its manifest carries no data stamps (a unit + // from before v0.275.0), so nothing else in the capture freezes its definition — this skip is the + // only thing standing between the failed definition and the restore point. + h.seedUnit(t, "docmost", 0) + goodCompose := "services:\n docmost:\n image: docmost/docmost:0.95.0\n" + if err := os.WriteFile(filepath.Join(RecoveryUnitComposePath(ns, "docmost"), "docker-compose.yml"), []byte(goodCompose), 0o644); err != nil { + t.Fatal(err) + } + // The stack dir after the failed update: it names the version that just failed. + for _, app := range []string{"docmost", "free"} { + dir := filepath.Join(h.dir, "stacks", app) + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + img := "docmost/docmost:0.96.0" + if app == "free" { + img = "free/free:1.0" + } + if err := os.WriteFile(filepath.Join(dir, "docker-compose.yml"), []byte("services:\n "+app+":\n image: "+img+"\n"), 0o644); err != nil { + t.Fatal(err) + } + } + + // The hold existed and was lifted by hand — exactly the operator CLI's act. + if err := h.m.HoldAfterFailedUpdate("docmost", time.Now(), time.Now(), UpdateTierLocal); err != nil { + t.Fatal(err) + } + if _, err := h.m.settings.ClearRestoreHold("docmost"); err != nil { + t.Fatal(err) + } + if held, _ := h.m.RestoreHoldFor("docmost"); held { + t.Fatal("setup: the hold must be lifted") + } + + // stacks.Manager.PinNotRunning's answer for this shape (pinned by TestR645_PinNotRunning_* there). + h.m.SetVersionCheck(func(name string) (string, bool) { + if name == "docmost" { + return "docmost runs docmost/docmost:0.95.0, pinned docmost/docmost:0.96.0", true + } + return "", false + }) + + var dumped []string + h.m.discoverDBs = func(context.Context) ([]DiscoveredDB, error) { + return []DiscoveredDB{ + {StackName: "docmost", ContainerName: "docmost-postgres"}, + {StackName: "free", ContainerName: "free-postgres"}, + }, nil + } + h.m.dumpOne = func(_ context.Context, db DiscoveredDB, dumpDir string, _ *log.Logger, _ bool) DumpResult { + dumped = append(dumped, db.StackName) + _ = os.MkdirAll(dumpDir, 0o755) + p := filepath.Join(dumpDir, db.StackName+"-postgres.sql") + _ = os.WriteFile(p, []byte("-- FRESH DUMP OF THE FAILED VERSION\n"), 0o644) + return DumpResult{DB: db, FilePath: p, Size: 36} + } + + unitRoot := RecoveryUnitPath(ns, "docmost") + before := treeFingerprint(t, unitRoot) + + _ = h.m.runDBDumpsInternal(context.Background()) + var mirrored []string + h.m.perAppTier2 = func(name string) error { mirrored = append(mirrored, name); return nil } + h.m.RunAllTier2() + + if after := treeFingerprint(t, unitRoot); after != before { + t.Errorf("the good unit was rewritten by the night run after a hand-lifted hold:\nbefore:\n%s\nafter:\n%s", before, after) + } + b, _ := os.ReadFile(filepath.Join(RecoveryUnitComposePath(ns, "docmost"), "docker-compose.yml")) + if !strings.Contains(string(b), "docmost/docmost:0.95.0") { + t.Errorf("the unit's definition must still be the good 0.95.0 one, got:\n%s", b) + } + for _, list := range [][]string{dumped, h.volDumped, h.prov.stopped, mirrored} { + for _, n := range list { + if n == "docmost" { + t.Errorf("a night leg touched docmost (db=%v vol=%v stopped=%v mirrored=%v)", dumped, h.volDumped, h.prov.stopped, mirrored) + } + } + } + // It says so in the log — the operator's half of the ruling. + if !strings.Contains(h.logs.String(), "Recovery unit NOT captured for docmost — it is not running its pinned version") { + t.Errorf("the skip must be logged, log:\n%s", h.logs.String()) + } + // Positive control: the app that runs its pin is backed up as before. + if len(dumped) != 1 || dumped[0] != "free" || len(h.volDumped) != 1 || h.volDumped[0] != "free" || len(mirrored) != 1 || mirrored[0] != "free" { + t.Errorf("positive control: the other app must still be dumped and mirrored (db=%v vol=%v mirrored=%v)", dumped, h.volDumped, mirrored) + } + if _, err := os.Stat(RecoveryUnitManifestPath(ns, "free")); err != nil { + t.Errorf("positive control: the other app's unit must be captured: %v", err) + } +} + +// The page's half: the household reads one plain sentence, in their language; nothing when the app +// runs its pin or when nothing is wired (unknown never skips). +func TestR645_VersionSkipSentence(t *testing.T) { + m := &Manager{} + if skip, why := m.VersionSkipFor("docmost", "hu"); skip || why != "" { + t.Fatalf("no check wired must read as no skip, got (%v, %q)", skip, why) + } + m.SetVersionCheck(func(name string) (string, bool) { return "x", name == "docmost" }) + skip, hu := m.VersionSkipFor("docmost", "hu") + if !skip || !strings.Contains(hu, "docmost") || !strings.Contains(hu, "jszakai ment") || !strings.Contains(hu, "Vedd fel") { + t.Errorf("hu sentence = %q", hu) + } + _, en := m.VersionSkipFor("docmost", "en") + if !strings.Contains(en, "night backup leaves it out") || strings.Contains(en, "jszakai") { + t.Errorf("en sentence = %q", en) + } + if skip, _ := m.VersionSkipFor("free", "hu"); skip { + t.Error("an app running its pin must not read as skipped") + } +} diff --git a/controller/internal/backup/recovery_unit.go b/controller/internal/backup/recovery_unit.go index d48b6c0..9992f5d 100644 --- a/controller/internal/backup/recovery_unit.go +++ b/controller/internal/backup/recovery_unit.go @@ -458,6 +458,17 @@ func (m *Manager) captureAllRecoveryUnits(dataRun bool) { if m.isHeld(stack.Name) { continue } + // R-645 (09 §3 decision 142): the unit of an app that is not running its pinned version stays + // as it is — the measured case wrote a just-failed definition over the good unit within seconds + // of a hand-lifted hold. Said at WARN on a data run; the periodic refresh repeats it only at DEBUG. + if why, skip := m.versionSkip(stack.Name); skip { + if dataRun { + m.logger.Printf("[WARN] [backup] Recovery unit NOT captured for %s — it is not running its pinned version (%s); the last good unit is kept (R-645)", stack.Name, why) + } else if m.isDebug() { + m.logger.Printf("[DEBUG] [backup] recovery-unit refresh skipped for %s — not running its pinned version (%s)", stack.Name, why) + } + continue + } m.noteAttempted(stack.Name) // The reserve, checked BEFORE anything is written. Per app, and the loop continues. if !m.admitApp(stack.Name) { diff --git a/controller/internal/backup/tier2.go b/controller/internal/backup/tier2.go index 5bd1e63..349f464 100644 --- a/controller/internal/backup/tier2.go +++ b/controller/internal/backup/tier2.go @@ -464,6 +464,11 @@ func (m *Manager) RunAllTier2() { m.logger.Printf("[WARN] [backup] Tier 2 skipped for %s — the app is HELD; its copy is the restore point and is preserved", stack.Name) continue } + // R-645: the mirror of an app not running its pinned version stays as it is. + if why, skip := m.versionSkip(stack.Name); skip { + m.logger.Printf("[WARN] [backup] Tier 2 skipped for %s — it is not running its pinned version (%s); the last good copy is kept (R-645)", stack.Name, why) + continue + } runOne := m.perAppTier2 if runOne == nil { runOne = m.RunTier2 diff --git a/controller/internal/backup/update_guard.go b/controller/internal/backup/update_guard.go index f7110f6..87093a6 100644 --- a/controller/internal/backup/update_guard.go +++ b/controller/internal/backup/update_guard.go @@ -600,6 +600,39 @@ func (m *Manager) isHeld(stackName string) bool { return m.updatingCheck != nil && m.updatingCheck(stackName) } +// versionSkip (R-645, 09 §3 decision 142) reports whether the night backup must leave an app alone +// because the version it is pinned to is NOT the version it runs, and names the difference. +// +// THE MEASURED CASE (9202, 2026-09-23): a failed update left docmost pinned to 0.96.0 while its running +// record said 0.95.0. Lifting the hold by hand (`--clear-restore-hold` + the restart) let the capture +// write the 0.96.0 definition — the version that had just failed — over the recovery unit the hold +// sentence named, within seconds. The hold is what kept the legs off the app (isHeld); once it is gone, +// this is what does. Where a leg also asks isHeld it asks this AFTER it, so a held app keeps its own line. +// +// Nil check or unknown ⇒ false (back the app up as before): a missing backup is the worse failure. +// Pinned by TestR645_HandLiftedHoldKeepsTheGoodUnit (r645_version_skip_test.go). +func (m *Manager) versionSkip(stackName string) (string, bool) { + if m == nil || m.versionCheck == nil { + return "", false + } + return m.versionCheck(stackName) +} + +// VersionSkipFor is versionSkip for the backups page: the household's sentence in lang, or false. +func (m *Manager) VersionSkipFor(stackName, lang string) (bool, string) { + if _, skip := m.versionSkip(stackName); !skip { + return false, "" + } + return true, util.Text(lang, "backup.status.version_skip", stackName) +} + +// SetVersionCheck wires the "is this app running its pinned version" question (stacks.Manager.PinNotRunning). +// INIT-ONLY, in main.go — pinned by TestR645_VersionCheckIsWiredAtStartup. The backup package cannot +// import stacks, which is why it is a seam. +func (m *Manager) SetVersionCheck(fn func(stackName string) (string, bool)) { + m.versionCheck = fn +} + // SetUpdatingCheck wires the "is a guarded update moving this app" question (stacks.Manager.IsUpdating). // INIT-ONLY, in main.go — pinned by TestSlice4_UpdatingCheckIsWiredAtStartup. The backup package cannot // import stacks, which is why it is a seam. diff --git a/controller/internal/i18n/locales/en.json b/controller/internal/i18n/locales/en.json index 46a5da7..fc4ec40 100644 --- a/controller/internal/i18n/locales/en.json +++ b/controller/internal/i18n/locales/en.json @@ -173,6 +173,7 @@ "backup.guest.err.unavailable": "the system backup is not available on this machine", "backup.removed_app": "Removed app — its backup is here and can be restored", "backup.status.db_failed": "The database backup failed", + "backup.status.version_skip": "%s is not running the version it is set to, so the night backup leaves it out and your last good backup stays as it is. Get in touch with us", "backup.status.ok": "App data backup is fine", "backup.target.absent": "The drive for the system backup cannot be reached — until you reconnect it, the full system backup is not made.", "backup.target.degraded": "The system backup is currently on the same disk as the system — so it protects against corrupted files, but not against a disk failure. Attach a second drive for full protection.", diff --git a/controller/internal/i18n/locales/hu.json b/controller/internal/i18n/locales/hu.json index d7f2fb4..b7109cb 100644 --- a/controller/internal/i18n/locales/hu.json +++ b/controller/internal/i18n/locales/hu.json @@ -169,6 +169,7 @@ "backup.guest.err.unavailable": "a rendszermentés nem érhető el ezen a gépen", "backup.removed_app": "Eltávolított alkalmazás — a mentése megvan, visszaállítható", "backup.status.db_failed": "Adatbázis mentés sikertelen", + "backup.status.version_skip": "A(z) %s nem a beállított verzióval fut, ezért az éjszakai mentés kihagyja, és a legutóbbi jó mentésed változatlan marad. Vedd fel velünk a kapcsolatot", "backup.status.ok": "Alkalmazás-adat mentés rendben", "backup.target.absent": "A rendszermentés meghajtója nem érhető el — amíg vissza nem csatlakoztatod, a teljes rendszermentés nem készül el.", "backup.target.degraded": "A rendszermentés jelenleg ugyanazon a lemezen van, mint a rendszer — így hibás fájlok ellen véd, lemezhiba ellen nem. Csatlakoztass egy második meghajtót a teljes védelemhez.", diff --git a/controller/internal/stacks/pin.go b/controller/internal/stacks/pin.go index 4978b6c..52960ed 100644 --- a/controller/internal/stacks/pin.go +++ b/controller/internal/stacks/pin.go @@ -373,3 +373,59 @@ func (m *Manager) advancePinTo(name, stackDir, src, metaSrc string) error { m.logger.Printf("[INFO] [stacks] update %s: pin advanced to %s (%s)", name, src, summarisePin(pin)) return nil } + +// PinNotRunning (R-645, 09 §3 decision 142) answers the night backup's question: is this app RUNNING +// the version it is pinned to? It returns a one-line description of every service whose recorded +// running reference (installed_images, an OBSERVATION) differs from its pin (pinned_images, the +// DECISION), and true when at least one differs. +// +// THE CASE IT EXISTS FOR, measured 2026-09-23 on 9202: a failed update leaves the pin on the new +// version and the running record on the old one (the record is written only after a healthy start). +// An operator lifting that hold by hand let the capture write the FAILED definition over the recovery +// unit the hold sentence pointed the household to, within seconds. The backup leaves such an app alone. +// +// UNKNOWN IS NEVER A MISMATCH: no app.yaml, no pin, no running record, or a pinned service with no +// running entry all answer false. Skipping a household's backup on a guess would trade a missing +// backup for a bookkeeping gap — the same reason a failed installed_images write never refuses a start. +// It reads app.yaml from disk, so a pin or record written a moment ago is seen. +// Pinned by TestR645_PinNotRunning_* (pin_r645_test.go). +func (m *Manager) PinNotRunning(name string) (string, bool) { + m.mu.RLock() + s, ok := m.stacks[name] + var composePath string + if ok { + composePath = s.ComposePath + } + m.mu.RUnlock() + if !ok || composePath == "" { + return "", false + } + cfg := LoadAppConfig(filepath.Dir(composePath)) + if cfg == nil { + return "", false + } + return pinRunDiff(cfg.PinnedImages, cfg.InstalledImages) +} + +// pinRunDiff is PinNotRunning's pure comparison, in deterministic service order. +func pinRunDiff(pinned map[string]string, running map[string]InstalledImage) (string, bool) { + if len(pinned) == 0 || len(running) == 0 { + return "", false + } + svcs := make([]string, 0, len(pinned)) + for svc := range pinned { + svcs = append(svcs, svc) + } + sort.Strings(svcs) + var diffs []string + for _, svc := range svcs { + got, ok := running[svc] + if !ok || got.Ref == "" { + continue // not observed: unknown, never a mismatch + } + if got.Ref != pinned[svc] { + diffs = append(diffs, fmt.Sprintf("%s runs %s, pinned %s", svc, got.Ref, pinned[svc])) + } + } + return strings.Join(diffs, "; "), len(diffs) > 0 +} diff --git a/controller/internal/stacks/pin_r645_test.go b/controller/internal/stacks/pin_r645_test.go new file mode 100644 index 0000000..d9d15fb --- /dev/null +++ b/controller/internal/stacks/pin_r645_test.go @@ -0,0 +1,39 @@ +package stacks + +import ( + "strings" + "testing" +) + +// R-645 (09 §3 decision 142) — the night backup asks "is this app running its pinned version?". +// These pin the answer; r645_version_skip_test.go (backup) pins what the backup does with it. + +// The measured shape (9202, 2026-09-23): a failed update leaves the pin on the new version and the +// running record on the old one. Read from the app.yaml ON DISK, not the in-memory copy. +func TestR645_PinNotRunning_FailedUpdateShapeIsAMismatch(t *testing.T) { + appYAML := "deployed: true\npinned_images:\n web: nextcloud:34.0.1-apache\ninstalled_images:\n web:\n ref: nextcloud:31.0.14-apache\n" + m, _ := newPinManager(t, pinTplNew, "", appYAML) + m.stacks["nextcloud"].AppConfig = nil // the in-memory view must not be what answers + why, skip := m.PinNotRunning("nextcloud") + if !skip || !strings.Contains(why, "web runs nextcloud:31.0.14-apache, pinned nextcloud:34.0.1-apache") { + t.Fatalf("PinNotRunning = (%q, %v), want a mismatch naming both versions", why, skip) + } +} + +func TestR645_PinNotRunning_AgreementAndUnknownAreNotAMismatch(t *testing.T) { + for name, appYAML := range map[string]string{ + "agree": "deployed: true\npinned_images:\n web: nextcloud:31.0.14-apache\ninstalled_images:\n web:\n ref: nextcloud:31.0.14-apache\n", + "unpinned": "deployed: true\ninstalled_images:\n web:\n ref: nextcloud:31.0.14-apache\n", + "no running record": "deployed: true\npinned_images:\n web: nextcloud:34.0.1-apache\n", + "service unobserved": "deployed: true\npinned_images:\n web: nextcloud:34.0.1-apache\ninstalled_images:\n db:\n ref: postgres:16\n", + } { + m, _ := newPinManager(t, pinTplOld, "", appYAML) + if why, skip := m.PinNotRunning("nextcloud"); skip { + t.Errorf("%s: unknown or agreeing must never skip a backup, got (%q, true)", name, why) + } + } + m, _ := newPinManager(t, pinTplOld, "", "deployed: true\n") + if _, skip := m.PinNotRunning("no-such-app"); skip { + t.Error("an unknown app is not a mismatch") + } +} diff --git a/controller/internal/web/handlers.go b/controller/internal/web/handlers.go index ce4e895..7f064c1 100644 --- a/controller/internal/web/handlers.go +++ b/controller/internal/web/handlers.go @@ -1638,6 +1638,12 @@ func (s *Server) buildAppBackupRows(status *backup.FullBackupStatus, lang string // the customer's data may not be intact. Measured 2026-08-22: after a failed MariaDB replay // the app reported `health=healthy, running=true, restarts=0` while its schema-version table // held zero rows. + // R-645 (09 §3 decision 142): an app the night backup leaves alone because it is not running its + // pinned version says so — amber, a deviation, not a failure. Before the hold check, which wins. + if skip, why := s.backupMgr.VersionSkipFor(app.StackName, lang); skip { + row.Status = "yellow" + row.StatusText = why + } if held, why := s.backupMgr.RestoreHoldForLang(app.StackName, lang); held { row.Status = "red" row.StatusText = why diff --git a/controller/internal/web/r645_version_skip_row_test.go b/controller/internal/web/r645_version_skip_row_test.go new file mode 100644 index 0000000..9b10eeb --- /dev/null +++ b/controller/internal/web/r645_version_skip_row_test.go @@ -0,0 +1,34 @@ +package web + +import ( + "strings" + "testing" + + "gitea.dooplex.hu/admin/felhom-controller/internal/backup" +) + +// R-645 (09 §3 decision 142): an app the night backup leaves alone because it is not running its pinned +// version says so on the backups page — amber, in the reader's language — and an app running its pin +// keeps its ordinary row. +// +// Red-proof: delete the VersionSkipFor block in buildAppBackupRows — the docmost row reads green and +// this test fails. +func TestR645_BackupRowSaysTheNightBackupSkipsIt(t *testing.T) { + s, _, m := newOffboxWebServer(t) + m.SetStackProvider(&blockProvider{hdd: t.TempDir()}) + m.SetVersionCheck(func(name string) (string, bool) { return "x", name == "docmost" }) + status := &backup.FullBackupStatus{AppDataInfo: []backup.AppBackupInfo{ + {StackName: "docmost", DisplayName: "Docmost", HasVolumeData: true}, + {StackName: "privatebin", DisplayName: "PrivateBin", HasVolumeData: true}, + }} + for lang, want := range map[string]string{"hu": "jszakai ment", "en": "night backup leaves it out"} { + rows := s.buildAppBackupRows(status, lang) + d := findRow(rows, "docmost") + if d == nil || d.Status != "yellow" || !strings.Contains(d.StatusText, want) { + t.Errorf("%s: docmost row = %+v, want amber with %q", lang, d, want) + } + if p := findRow(rows, "privatebin"); p == nil || p.Status != "green" { + t.Errorf("%s: positive control: privatebin must stay green, got %+v", lang, p) + } + } +} diff --git a/controller/scripts/i18n_go_keys.json b/controller/scripts/i18n_go_keys.json index 4c3b831..786a80b 100644 --- a/controller/scripts/i18n_go_keys.json +++ b/controller/scripts/i18n_go_keys.json @@ -160,6 +160,7 @@ "family_gate.msg.wrong": "BORN AS A KEY, v0.287.0 (the family gate, decisions 63/64) -- a NEW sentence, never a Go literal. Pinned in both languages by TestFamilyGate_MessagesFollowTheReader.", "note.offsite.fail_locked": "BORN AS A KEY (R-104) -- the cause line for a repository lock that survived the self-heal; a NEW sentence, never a Go literal. Pinned in both languages by TestR104_SurvivingLockIsNamed.", "err.backup.restore_drive_gone": "BORN AS A KEY (R-362) -- names the drive a restore could not reach instead of a raw permission error; a NEW sentence, never a Go literal. Pinned in both languages by TestR362_DetachedDriveIsNamed.", + "backup.status.version_skip": "BORN AS A KEY (R-645, `09` decision 142) -- the backups-page line for an app the night backup skips because it is not running its pinned version; a NEW sentence, never a Go literal. Pinned in both languages by TestR645_VersionSkipSentence and TestR645_BackupRowSaysTheNightBackupSkipsIt.", "restore.refuse.files.second_drive_whole": "BORN AS A KEY (R-675) -- the unit-restore refusal names the second drive's WHOLE restore (decision 26); a NEW sentence, never a Go literal. Pinned in both languages by TestR675_RefusalNamesTheWholeCopy.", "flash.login.password_changed": "R-516 item 12 -- REWORDED on purpose: the formal „Kérjük, jelentkezzen be\" became the product's te-form „Jelentkezz be\"; the row is about exactly these bytes, so byte parity with the base literal cannot hold. Pinned by TestR516_FormalFormsAreGone.", "flash.backup.window_invalid_time": "R-516 (2026-10-06) -- REWORDED on purpose: the formal („ön\") verb forms in this sentence became the product's te-form; the row is about exactly these bytes, so byte parity with the base literal cannot hold. Pinned by TestR516_WidenedFormalFormsAreGone.",