diff --git a/CHANGELOG.md b/CHANGELOG.md index ead2be50..a3d0d342 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,43 @@ +## Unreleased (2026-11-15) — the off-site copies of a removed app end 30 days after „delete backup data” (felhom.eu R-1374 part 1; D72, D84) + +**MinAgent: 0.158.1** (unchanged — nothing here asks the agent for anything) + +- **The off-site copies of a removed app end 30 days after „delete backup data” — the box's half (R-1374 part 1; D72 + and D84, `09` §3 decisions 292 and 309; felhom.eu `07-backup-architecture.md` §8.2f). Built, NOT released; no box + has run it. It does nothing without a hub that knows it (felhom.eu `hub/CHANGELOG.md`, Unreleased): an older hub + agrees to no id, and then nothing of a removed app is deleted.** + (1) **The record.** `settings.OffsiteEnd` (`settings.json` → `offsite_ends`): `{app, removed_at, ends_at}`. One + writer, `backup.StartOffsiteEnd`, called by the removal ONLY when „delete backup data” was pressed + (`api.removeStack`) and by „Törlés 30 nap múlva” on the restore page. A second press changes nothing. **No record, + no deletion.** Only on a box with the hub-provisioned (pinned) off-site tier; the household's own NAS target is + unchanged. + (2) **The date before the press, and the way back.** `GET /api/stacks/{name}/backup-data` carries + `offsite_end_date`; the remove dialog shows the sentence with it. The restore page's off-site list shows a marked + app with „Mégis megtartom” (`POST /backup/offsite-end/keep`), a removed app with kept copies with „Törlés 30 nap + múlva” (`POST /backup/offsite-end/start`), and one history line per carried-out end. An install of the same app + deletes the record (`stacks.DeployStack`, at its first write). + (3) **The window.** `backup/offbox_ended.go`: after the fake-snapshot guard has passed the policy's own plan under + the cap exactly as before, the window adds the ids of snapshots whose app tags are ALL due records, that the hub + sent back (`OffsiteWindow.EndedAllowed`) and that the box named in the same request. `_shares`, the box's own + stacks and an installed app are never due (`SetOffsiteEndLiving`, wired in `main.go`; not wired → nothing is due). + The result names them apart (`removed_for_ended_apps`). `OffsiteWindowClient.Open` takes the claims; the request's + `ended` key is absent when there is none. + (4) **The hub report** carries the pending records (`offsite.ended_apps`) so the hub can start its own clock. + (5) **The mail (D84).** `Notifier.NotifyOffsiteEndScheduled` → `offsite_end_scheduled` (info), once per record + written, in both languages with each language's date form; details `subject` = app + the record's time. + **Household-facing texts, new, Hungarian and English — the operator reads them before the release:** + `offsite_end.dialog`, `offsite_end.marked`, `offsite_end.keep_button`, `offsite_end.kept_before`, + `offsite_end.start_button`, `offsite_end.done`, `flash.offsite_end.kept`, `flash.offsite_end.nothing_to_keep`, + `flash.offsite_end.not_possible`, `flash.offsite_end.failed`, `event.offsite_end_scheduled` (the full list with both + languages is in `07` §8.2f). + 143 parity fixtures gained the dialog's one new script line (patched, not regenerated); one new parity case + (`backups_restore_offsite_end`). + Tests `TestD72_*`, `TestD84_*` (settings, backup, stacks, notify, api, web, offsiteapply); 56 mutations seen red, + none redone (felhom.eu `documentation/audits/build-release-4-2026-11-15/A/redproof-ctrl-{record,window,press,page}.txt`). + **A seam, named:** `api.removeStack`'s success path needs Docker, so its hook is pinned by reading the handler's + source (`TestD72_RemovalWritesTheEndRecordOnlyWithDeleteBackupData`); what the hook then does runs on the real code. + restic is the package's fake runner, as in the window's other tests — no real repository. + ## v0.353.0 (2026-11-15) — a clock step is not a new boot (felhom.eu R-1395); HSTS one year (R-1050, decision 249); the web certificate's end date in the report (R-1379) — RELEASED (image `sha256:61cf2a8b…`, from `ae07ed4`, CI run id 2681; rc.1 on scratch guest 9202); per-box floors 10:08 UTC, global floor 2026-11-15 10:26:53 UTC (MinAgent 0.158.1) **MinAgent: 0.158.1** (unchanged — the agent's boot key is optional: with an agent before 0.175.0 the controller behaves as before, so a clock step still recreates the drive-backed apps once on such a box; the fix needs agent 0.175.0 or newer on the same box. rc.1 on scratch guest 9202, 2026-11-15.) diff --git a/REUSE.md b/REUSE.md index 985fb891..78813d44 100644 --- a/REUSE.md +++ b/REUSE.md @@ -282,6 +282,7 @@ | `ConfigRefresher.Reconcile` | controller/internal/report/config_refresh.go | `(ackVersion int)` | Pull-based config refresh | Re-pulls controller.yaml (re-merging local_api), then graceful self-restart; first-run = baseline, no restart | | `offsiteapply.HubRegistrar` / `HubWindowClient` / `PinnedProber` (v0.289.0, decisions 68–69) | controller/internal/offsiteapply/seams.go | `Register(ctx,pub)(fp,err)` · `Confirm` · `MoveAside` · `Open/Close` window · `Probe(ctx,host,user,port,kh,privPEM) bool` | EVERY off-site key install, the hub move-aside, the clean-up window | **The box never handles the sub-account password** — there is no consume path any more. `PinnedProber` is a POSITIVE observable (exit 0 + rclone output); "authenticates" is NOT enough — an unpinned key authenticates and can delete. | | `Manager.offsiteWindowRetention` + `offsiteGuard` (v0.289.0, R-822) | controller/internal/backup/offbox_window.go | `(ctx, base, env, why)` · pure `(all, plan, now, newestAllowed, max) (ids, refuse)` | THE retention step for both callers (after a run, over quota) | Pinned tier: no window → nothing deleted; the guard runs BEFORE any forget; "young" = fewer than `keepDaily` CALENDAR days old (v0.294.0, R-867 — the line is built from the same constants as `retentionPolicy`; never an hour count); a young snapshot superseded the same day is EXCLUDED, any other young removal / future date / plan above `max_remove` REFUSES; tests run restic 0.14.0's policy itself (`restic0140Plan`, proven identical to the binary); forget is by explicit ids, oldest first. A due abandonment goes to `OffsiteAbandonClient` (hub, 7-day wait). NAS tier: the old SP-2 policy, unchanged. | +| `Manager.StartOffsiteEnd` / `KeepOffsiteCopies` + `settings.OffsiteEnd` (D72/D84, felhom.eu `07` §8.2f; built, not released) | controller/internal/backup/offbox_ended.go, controller/internal/settings/offsite_ends.go | `StartOffsiteEnd(app) (rec, created, err)` · pure `dueOffsiteEnds`, `endedCandidates`, `endedToRemove` | The off-site copies of a removed app end 30 days after „delete backup data”: the record, the press, the take-back, and the ended apps' ids in the weekly window | **NO RECORD = NO DELETION.** `StartOffsiteEnd` is the ONLY writer (it sends the household's one mail when it really wrote); an install deletes the record (`stacks.DeployStack`). The window removes an ended app's snapshot only when every app tag on it is a due record, the hub sent its id back (`OffsiteWindow.EndedAllowed`) and the box named it in the same window. `SetOffsiteEndLiving` not wired → nothing is ever due. Only on the hub-provisioned (pinned) tier. | | `offsiteapply.SettleProvider` / `SettleFunc` / `Bridge.AwaitSettle` / `ReconcileWhenSettled` (R-71a, v0.162.0) | controller/internal/offsiteapply/offsiteapply.go + seams.go | `SettleState() (version, floor string, updateRunning, floorKnown bool)` | THE settle-gate: defers the offsite one-time-password consume past a managed day-0 floor-update (the F10 race). Wire the `SettleFunc` adapter over `updater.GetFloor()`/`IsUpdateRunning()` — **the updater's knowledge is the ONE floor source; never fetch the floor a second way**. Gate ONLY the bridge goroutine, and only when an updater exists (nil `Settle` = reconcile immediately). Bounds `settlePoll`/`settleFloorSubBound`/`settleOverallBound`; the floor is in-memory (report-ACK-derived, ~5–10 s), NOT persisted → unknown until the first ACK on any restart. Inject `Now`/`Sleep` in tests (no real sleeps). B′: at/above-floor GOes on the first poll, zero wait. Do NOT touch the consume/persist order or the 404 contract — ordering only | | `bootstrap.MaybeIngest` / `RefreshConfig` | controller/internal/bootstrap/bootstrap.go | bootstrap.json → controller.yaml | Day-0 + refresh | Overwrites controller.yaml, NEVER settings.json | | `api.GracefulSelfRestart` | controller/internal/api/selfrestart.go | `(logger)` | Controller self-restart | Detached exit; bootstrap unit re-runs the image | diff --git a/controller/README.md b/controller/README.md index 3249842a..db251611 100644 --- a/controller/README.md +++ b/controller/README.md @@ -1123,6 +1123,18 @@ The `/apps/{slug}` page renders hero section, screenshots, setup guide, and opti **Remove modal** shows three sections: (1) always-removed items (Docker volumes, app.yaml, cross-drive schedule), (2) optional HDD data deletion with reimport warning, (3) optional backup data deletion (DB dumps + cross-drive rsync) with restic retention note. +**The off-site copies of a removed app (D72/D84, felhom.eu `07-backup-architecture.md` §8.2f — built, not released, no +box has run it).** On a box with the hub-provisioned off-site tier, „Mentési adatok törlése” deletes the local +backups at once and starts 30 days for the app's off-site copies: the dialog names the day before the press +(`offsite_end_date` in `GET /api/stacks/{name}/backup-data`), the box writes an *off-site end record* +(`settings.json` → `offsite_ends`) and the household gets one mail (`offsite_end_scheduled`). The restore page's +off-site list shows „Törlésre kijelölve. Törlés napja: …” with „Mégis megtartom” (`POST /backup/offsite-end/keep` +deletes the record); a removed app with copies and no record shows „Törlés 30 nap múlva” +(`POST /backup/offsite-end/start`). An install of the same app deletes the record. The copies are removed only in +the weekly clean-up window, by snapshot id, for ids the hub agreed to — never without a record, never a snapshot +that also belongs to an installed app, never `_shares` or the box's own stacks. A box whose off-site target is the +household's own NAS keeps a removed app's copies as before. + **Protected stacks** (traefik, cloudflared, felhom-controller) cannot be stopped, removed, or deleted from the UI. Restart is allowed. **Orphan detection**: Deployed stacks with no matching catalog template are marked as orphaned with an "Elavult" badge and can be safely deleted. diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index 39852852..a8e2b268 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -1608,6 +1608,19 @@ func main() { backupMgr.SetOffboxEnlargeBlockedNotifier(func(stack string, estBytes int64, usedGB, quotaGB int) { notifier.NotifyOffboxEnlargeBlocked(stack, appbackup.HumanizeBytes(estBytes), usedGB, quotaGB) // R-585 }) + // D72 / D84 (`07` §8.2f): the off-site copies of a removed app. (1) „Living": installed right now, or one of + // the box's own protected stacks — such an app's end record is never due, and it cannot get one. Without + // this seam nothing is ever due (backup/offbox_ended.go). (2) The household's one mail per press. + backupMgr.SetOffsiteEndLiving(func(app string) bool { + if cfg.IsProtectedStack(app) { + return true + } + st, ok := stackMgr.GetStack(app) + return ok && (st.Deployed || st.Deploying || st.Protected) + }) + backupMgr.SetOffsiteEndNotify(func(rec settings.OffsiteEnd) { + notifier.NotifyOffsiteEndScheduled(rec.App, rec.RemovedAt, rec.EndsAt, rec.Ends().Local()) + }) // v0.142.0 offsite-repo continuity: push a hub event on the orphaned/reset transitions (once per // transition — the Manager guards nightly re-fire). Operator-visible on the customer page. backupMgr.SetOffboxOrphanEvent(func(eventType, renamedTo string) { diff --git a/controller/internal/api/d72_offsite_end_test.go b/controller/internal/api/d72_offsite_end_test.go new file mode 100644 index 00000000..8e8cfe68 --- /dev/null +++ b/controller/internal/api/d72_offsite_end_test.go @@ -0,0 +1,136 @@ +package api + +import ( + "encoding/json" + "go/ast" + "go/parser" + "go/token" + "io" + "log" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/backup" + "gitea.dooplex.hu/admin/felhom-controller/internal/config" + "gitea.dooplex.hu/admin/felhom-controller/internal/i18n" + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// D72 / D84 (`09` §3 decisions 292, 309; `07-backup-architecture.md` §8.2f). +// RED-PROOFS: felhom.eu/documentation/audits/build-release-4-2026-11-15/A/redproof-ctrl-press.txt. + +func endsRouter(t *testing.T, transport string) (*Router, *settings.Settings) { + t.Helper() + r, _ := newR442Router(t, "deployed: true\nenv: {}\n") + lg := log.New(io.Discard, "", 0) + sett, err := settings.Load(filepath.Join(t.TempDir(), "settings.json"), lg) + if err != nil { + t.Fatal(err) + } + if err := sett.SetOffboxTarget(&settings.OffboxTarget{Enabled: true, Host: "h", User: "u", RepoPath: "/r", Transport: transport}); err != nil { + t.Fatal(err) + } + cfg := &config.Config{} + cfg.Paths.DataDir = t.TempDir() + r.backupMgr = backup.NewManager(cfg, sett, lg) + r.sett = sett + return r, sett +} + +// THE DATE IS SHOWN BEFORE THE PRESS (D72): the dialog's data carries the day the off-site copies would be deleted — +// today + 30 days, in the household's date form — only on a box where a press starts the clock. +func TestD72_BackupDataCarriesTheEndDateBeforeThePress(t *testing.T) { + get := func(r *Router, lang string) map[string]any { + w := httptest.NewRecorder() + r.getStackBackupData(w, httptest.NewRequest(http.MethodGet, "/api/stacks/app/backup-data?lang="+lang, nil), "app") + var resp struct { + OK bool `json:"ok"` + Data map[string]any `json:"data"` + } + if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil || !resp.OK { + t.Fatalf("backup-data: %s (%v)", w.Body.String(), err) + } + return resp.Data + } + r, sett := endsRouter(t, settings.TransportRclonePinned) + want := time.Now().Add(settings.OffsiteEndAfter).Local() + for _, lang := range []string{"hu", "en"} { + got, _ := get(r, lang)["offsite_end_date"].(string) + if got != i18n.FormatDate(lang, want) && got != i18n.FormatDate(lang, want.Add(-time.Minute)) { + t.Errorf("[%s] offsite_end_date = %q, want %q (today + 30 days)", lang, got, i18n.FormatDate(lang, want)) + } + } + // reading the dialog's data writes NOTHING: the record is the press's, not the look's + if n := len(sett.GetOffsiteEnds()); n != 0 { + t.Fatalf("opening the dialog wrote %d end record(s)", n) + } + // the household's own NAS (no hub window): the dialog says nothing about off-site copies + r2, _ := endsRouter(t, "") + if v, has := get(r2, "hu")["offsite_end_date"]; has { + t.Fatalf("a box without the hub's tier names an end date: %v", v) + } +} + +// MUST-HOLD 1, at the handler: the end record is written ONLY when the household pressed „delete backup data”. +// removeStack's success path needs Docker, so this reads the handler's source (as R-486/R-491 do): every call of +// StartOffsiteEnd in removeStack sits inside an `if` whose condition names body.RemoveBackups — and there is one. +// What the call then does is pinned on the real code in backup (TestD84_OnePressOneMail and the window tests). +func TestD72_RemovalWritesTheEndRecordOnlyWithDeleteBackupData(t *testing.T) { + fset := token.NewFileSet() + f, err := parser.ParseFile(fset, "router.go", nil, 0) + if err != nil { + t.Fatal(err) + } + calls, guarded := 0, 0 + for _, d := range f.Decls { + fn, ok := d.(*ast.FuncDecl) + if !ok || fn.Name.Name != "removeStack" || fn.Body == nil { + continue + } + isCall := func(n ast.Node) bool { + s, ok := n.(*ast.SelectorExpr) + return ok && s.Sel.Name == "StartOffsiteEnd" + } + ast.Inspect(fn.Body, func(n ast.Node) bool { + if isCall(n) { + calls++ + } + ifs, ok := n.(*ast.IfStmt) + if !ok { + return true + } + names := false + ast.Inspect(ifs.Cond, func(c ast.Node) bool { + if s, ok := c.(*ast.SelectorExpr); ok && s.Sel.Name == "RemoveBackups" { + names = true + } + return true + }) + if names { + ast.Inspect(ifs.Body, func(b ast.Node) bool { + if isCall(b) { + guarded++ + } + return true + }) + } + return true + }) + } + if calls != 1 || guarded != 1 { + t.Fatalf("removeStack: %d call(s) of StartOffsiteEnd, %d inside `if … body.RemoveBackups …` — want exactly one, guarded: "+ + "a removal that KEEPS the backups must write no end record", calls, guarded) + } + raw, err := os.ReadFile("router.go") + if err != nil { + t.Fatal(err) + } + if n := strings.Count(string(raw), "StartOffsiteEnd("); n != 1 { + t.Fatalf("router.go calls StartOffsiteEnd %d time(s) — the removal is its only caller here", n) + } +} diff --git a/controller/internal/api/router.go b/controller/internal/api/router.go index 2d2c4d7b..196dfe97 100644 --- a/controller/internal/api/router.go +++ b/controller/internal/api/router.go @@ -16,6 +16,7 @@ import ( "gitea.dooplex.hu/admin/felhom-controller/internal/backup" cf "gitea.dooplex.hu/admin/felhom-controller/internal/cloudflare" "gitea.dooplex.hu/admin/felhom-controller/internal/config" + "gitea.dooplex.hu/admin/felhom-controller/internal/i18n" "gitea.dooplex.hu/admin/felhom-controller/internal/integrations" "gitea.dooplex.hu/admin/felhom-controller/internal/metrics" "gitea.dooplex.hu/admin/felhom-controller/internal/notify" @@ -963,6 +964,10 @@ func (r *Router) getStackBackupData(w http.ResponseWriter, req *http.Request, na writeJSON(w, http.StatusNotFound, apiResponse{OK: false, Error: r.errText(req, err)}) return } + // D72: the date is on the dialog BEFORE the press. Only where a press would start the 30 days. + if r.backupMgr != nil && r.backupMgr.OffsiteEndOffered() { + resp.OffsiteEndDate = i18n.FormatDate(r.langFor(req), r.backupMgr.OffsiteEndDateIfPressedNow().Local()) + } writeJSON(w, http.StatusOK, apiResponse{OK: true, Data: resp}) } @@ -1101,6 +1106,19 @@ func (r *Router) removeStack(w http.ResponseWriter, req *http.Request, name stri if body.RemoveBackups { resp.BackupPathsRemoved = append(resp.BackupPathsRemoved, r.removeVerificationCopy(name)...) } + // D72 / D84 (`07` §8.2f): „delete backup data” deletes the LOCAL backups now (above) and starts the 30 days of + // the app's OFF-SITE copies: the end record, and the household's one mail. Never on a removal that kept the + // backups; never on a box without the hub-provisioned tier (nothing would carry the deletion out). + if body.RemoveBackups && r.backupMgr != nil { + switch rec, created, serr := r.backupMgr.StartOffsiteEnd(name); { + case errors.Is(serr, backup.ErrOffsiteEndNotOffered): + r.dbg("removeStack: %s: no hub-provisioned off-site tier — no off-site end record", name) + case serr != nil: + r.logger.Printf("[WARN] [api] remove %s: its off-site end record was NOT written (%v) — its off-site copies stay; the Backups page offers „delete in 30 days”", name, serr) + case created: + r.logger.Printf("[INFO] [api] remove %s: its off-site copies end %s (D72)", name, rec.EndsAt) + } + } // R-486 (v0.240.0): the app's backup preferences — and with them the Tier-2 RECORD that // tier2RecordedCopyDir needs — are forgotten ONLY when the customer asked for the backups to be diff --git a/controller/internal/backup/backup.go b/controller/internal/backup/backup.go index 23deadd1..fef79f36 100644 --- a/controller/internal/backup/backup.go +++ b/controller/internal/backup/backup.go @@ -127,6 +127,11 @@ type Manager struct { offsiteMoveAside func(ctx context.Context) (string, error) // offsiteWindow (v0.289.0, decision 68) asks the hub for a clean-up window. nil → no box retention. offsiteWindow OffsiteWindowClient + // offsiteEndLiving (D72): "installed now, or a protected stack" — nil → no ended app is ever due (offbox_ended.go). + offsiteEndLiving func(app string) bool + // offsiteEndNotify (D84) tells the household ONCE per press that an app's off-site copies are marked for + // deletion. Wired in main.go to the notifier; nil-safe. Called only when a record was really written. + offsiteEndNotify func(rec settings.OffsiteEnd) // offsiteAbandon (v0.290.0, decision 74) hands a due set-aside deletion to the hub. offsiteAbandon OffsiteAbandonClient diff --git a/controller/internal/backup/offbox.go b/controller/internal/backup/offbox.go index 1a6ca351..106b8acf 100644 --- a/controller/internal/backup/offbox.go +++ b/controller/internal/backup/offbox.go @@ -1680,9 +1680,14 @@ type OffboxReportStatus struct { // hub cannot see that and must not infer it. It keeps being sent until the ACK stops reporting a // superseded package, so a lost request retries by itself rather than leaving the pair half-gone. // Absent/false on every other box, so a healthy report is byte-identical to v0.205.0's. - AbandonPurgeRequested bool `json:"abandon_purge_requested,omitempty"` - LastRun string `json:"last_run,omitempty"` // RFC3339 - LastStatus string `json:"last_status,omitempty"` // "ok" | "incomplete" (R-203) | "error" | "running" + // EndedApps (D72/D84, `07` §8.2f part 4) — the pending off-site end records: apps the household removed with + // „delete backup data”, whose off-site copies end 30 days later. The hub starts ITS OWN clock from its own + // records (and, for an app it never saw installed, from the first report that names it here); the dates in + // this list decide nothing there. Absent when there is none, so a report without one is byte-identical. + EndedApps []OffsiteEndedApp `json:"ended_apps,omitempty"` + AbandonPurgeRequested bool `json:"abandon_purge_requested,omitempty"` + LastRun string `json:"last_run,omitempty"` // RFC3339 + LastStatus string `json:"last_status,omitempty"` // "ok" | "incomplete" (R-203) | "error" | "running" // LastSuccess (R-100) is the last run that SUCCEEDED — the hub's staleness anchor. Absent on a // pre-v0.181.0 controller, which the hub must degrade on explicitly rather than by accident: // treating absence as failure alarms every un-upgraded box, treating it as success keeps the bug. @@ -1944,7 +1949,8 @@ func (m *Manager) OffboxReportStatus() *OffboxReportStatus { LastProofSnapshot: t.LastProofSnapshot, LastProofResult: t.LastProofResult, LastProofReason: t.LastProofReason, - AbandonPurgeRequested: t.AbandonPurgeRequested, // R-241: declared until the hub drops the package + AbandonPurgeRequested: t.AbandonPurgeRequested, // R-241: declared until the hub drops the package + EndedApps: m.OffsiteEndedAppsForReport(), // D72: the pending off-site end records } } diff --git a/controller/internal/backup/offbox_ended.go b/controller/internal/backup/offbox_ended.go new file mode 100644 index 00000000..6fd6ae23 --- /dev/null +++ b/controller/internal/backup/offbox_ended.go @@ -0,0 +1,257 @@ +package backup + +import ( + "errors" + "sort" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// D72 / D84 (`09` §3 decisions 292 and 309; `07-backup-architecture.md` §8.2f, parts 3 and 4) — THE OFF-SITE COPIES OF +// A REMOVED APP END 30 DAYS AFTER „DELETE BACKUP DATA”, IN THE WEEKLY WINDOW AND NOWHERE ELSE. +// +// What this file adds to the clean-up window (offbox_window.go) and what it must never do: +// +// - NO RECORD, NO DELETION. Only an off-site end record (settings.OffsiteEnd, written by the household's press) +// that is DUE makes a snapshot a candidate. A removal that kept the backups, a rebuilt guest, an app removed +// before this existed: there is no record, so there is no candidate. +// - A LIVING APP'S COPY IS NEVER REMOVED BY ANOTHER APP'S END. A snapshot is a candidate only when EVERY app tag +// it carries belongs to a due record — a snapshot with two apps' tags stays until both have ended. The shares +// pseudo-app (`_shares`), the infrastructure stacks and an app that is installed right now are never due. +// - THE HUB AGREES FIRST. The candidates go to the hub with the window request; the box removes ONLY the ids the +// hub sends back (OffsiteWindow.EndedAllowed) — and of those, only ids it named itself in this same window. +// A hub that does not know this (an older hub) sends none, and then nothing of an ended app is removed. +// - ONE GUARD, NO SECOND PATH. The fake-snapshot guard runs first, on the whole repository and on the policy's own +// plan under the window's cap, exactly as before. If it refuses, the ended apps wait with everything else. +// The ended ids are OUTSIDE the cap (D84) and are named apart in the result (removed_for_ended_apps), so the +// hub's 07:10 audit can explain them. + +// OffsiteEndedClaim is one due end record as the box sends it with the window request: the app, the record's own +// dates (the hub does not decide by them) and the snapshot ids the box would remove. +type OffsiteEndedClaim struct { + App string `json:"app"` + RemovedAt string `json:"removed_at"` + EndsAt string `json:"ends_at"` + SnapshotIDs []string `json:"snapshot_ids"` +} + +// OffsiteEndedRefusal is one refusal the hub sent back, in its own words (logged). +type OffsiteEndedRefusal struct { + App string `json:"app"` + Reason string `json:"reason"` +} + +// OffsiteEndedApp is one pending end record on the hub report (`offsite.ended_apps`): it lets the hub start its own +// clock for an app it never saw installed (`07` §8.2f part 4). The dates are the box's; the hub decides by its own. +type OffsiteEndedApp struct { + App string `json:"app"` + RemovedAt string `json:"removed_at"` + EndsAt string `json:"ends_at"` +} + +// SetOffsiteEndLiving wires "is this app installed on this box right now, or one of the box's own protected stacks". +// An app for which it answers true is never due. NOT WIRED (nil) = NOTHING IS DUE: a box that cannot tell what it +// runs must not delete any app's history. Wired in cmd/controller/main.go; pinned by TestD72_MainWiresTheLivingCheck. +func (m *Manager) SetOffsiteEndLiving(fn func(app string) bool) { m.offsiteEndLiving = fn } + +// appTags returns a snapshot's app tags: every tag but the tier's marker. +func appTags(s guardSnap) []string { + var out []string + for _, t := range s.Tags { + if t != "" && t != OffboxMarkerTag { + out = append(out, t) + } + } + return out +} + +// dueOffsiteEnds is the PURE first half: the records that may be claimed now. living == nil → none. +func dueOffsiteEnds(ends []settings.OffsiteEnd, now time.Time, living func(string) bool) map[string]settings.OffsiteEnd { + due := map[string]settings.OffsiteEnd{} + if living == nil { + return due + } + for _, e := range ends { + if !settings.ValidOffsiteEndApp(e.App) || e.App == SharesPseudoStack || !e.Due(now) || living(e.App) { + continue + } + due[e.App] = e + } + return due +} + +// endedCandidates is the PURE second half: from the repository's snapshots and the due records, the claims to send +// (sorted by app; an app with no candidate is not claimed) and the set of candidate ids. +// +// A snapshot is a candidate only when it carries at least one app tag, EVERY app tag it carries is a due record, +// and it is not younger than any of those records' removal (a snapshot made under that tag AFTER the removal is not +// the removed app's copy — a later install's, or a planted one). +func endedCandidates(snaps []guardSnap, due map[string]settings.OffsiteEnd) (claims []OffsiteEndedClaim, candidate map[string]bool) { + candidate = map[string]bool{} + byApp := map[string][]string{} + for _, s := range snaps { + apps := appTags(s) + if len(apps) == 0 || s.ID == "" { + continue + } + ok := true + for _, a := range apps { + e, isDue := due[a] + if !isDue || s.Time.After(e.Removed().Add(offsiteGuardSkew)) { + ok = false + break + } + } + if !ok { + continue + } + candidate[s.ID] = true + for _, a := range apps { + byApp[a] = append(byApp[a], s.ID) + } + } + for app, ids := range byApp { + sort.Strings(ids) + e := due[app] + claims = append(claims, OffsiteEndedClaim{App: app, RemovedAt: e.RemovedAt, EndsAt: e.EndsAt, SnapshotIDs: ids}) + } + sort.Slice(claims, func(i, j int) bool { return claims[i].App < claims[j].App }) + return claims, candidate +} + +// endedToRemove is the PURE third half: of the ids the hub agreed to, those the box itself named as candidates in +// this window and that the policy's own plan does not already remove (those stay under the cap). Sorted. +func endedToRemove(allowed []string, candidate map[string]bool, policyIDs []string) []string { + inPolicy := make(map[string]bool, len(policyIDs)) + for _, id := range policyIDs { + inPolicy[id] = true + } + seen := map[string]bool{} + var out []string + for _, id := range allowed { + if candidate[id] && !inPolicy[id] && !seen[id] { + seen[id] = true + out = append(out, id) + } + } + sort.Strings(out) + return out +} + +// appStillInStore reports whether any snapshot still carries app's tag. +func appStillInStore(snaps []guardSnap, app string) bool { + for _, s := range snaps { + for _, t := range s.Tags { + if t == app { + return true + } + } + } + return false +} + +// dropEndsWithNoCopies deletes the due records of apps that have NO snapshot in the repository at all: there is +// nothing to end, and a record that can never complete would be asked of the hub every week for ever. It writes no +// history line (nothing was deleted). Called only after a listing that succeeded. +func (m *Manager) dropEndsWithNoCopies(snaps []guardSnap, due map[string]settings.OffsiteEnd) { + for app, e := range due { + if appStillInStore(snaps, app) { + continue + } + if done, err := m.settings.CompleteOffsiteEnd(app, e.RemovedAt, m.now(), 0); err != nil { + m.logger.Printf("[WARN] [offbox] ended app %s: the store holds no copy of it, but its end record could not be closed: %v", app, err) + } else if done { + m.logger.Printf("[INFO] [offbox] ended app %s: the off-site store holds no copy of it — its end record is closed, nothing was deleted", app) + } + delete(due, app) + } +} + +// completeOffsiteEnds closes the record of every app whose copies this window removed: at least one of its claimed +// ids is gone and no snapshot carries its tag any more. An app that still has a snapshot (one shared with a living +// app, or one the hub did not agree to) keeps its record and is asked again at the next window. +func (m *Manager) completeOffsiteEnds(claims []OffsiteEndedClaim, removed []string, after []guardSnap) { + gone := make(map[string]bool, len(removed)) + for _, id := range removed { + gone[id] = true + } + for _, c := range claims { + n := 0 + for _, id := range c.SnapshotIDs { + if gone[id] { + n++ + } + } + if n == 0 || appStillInStore(after, c.App) { + continue + } + if _, err := m.settings.CompleteOffsiteEnd(c.App, c.RemovedAt, m.now(), n); err != nil { + m.logger.Printf("[WARN] [offbox] ended app %s: its %d off-site snapshot(s) are removed, but the end record could not be closed: %v", c.App, n, err) + } + } +} + +// OffsiteEndedAppsForReport returns the pending end records for the hub report. +func (m *Manager) OffsiteEndedAppsForReport() []OffsiteEndedApp { + var out []OffsiteEndedApp + for _, e := range m.settings.GetOffsiteEnds() { + out = append(out, OffsiteEndedApp{App: e.App, RemovedAt: e.RemovedAt, EndsAt: e.EndsAt}) + } + return out +} + +// ── The household's side: the press, the take-back, the date ──────────────────────────────────────────────────────── + +// ErrOffsiteEndNotOffered: this box has no hub-provisioned off-site tier, so there is no weekly window that could +// carry the deletion out — and no record is written. (The household's own NAS target is pruned by the box itself and +// keeps a removed app's copies, as before: `07` §8.2f.) +var ErrOffsiteEndNotOffered = errors.New("no hub-provisioned off-site tier on this box") + +// ErrOffsiteEndLiving: the app is installed right now, is one of the box's own stacks, or the box cannot tell. +var ErrOffsiteEndLiving = errors.New("the app is installed or is not an app a household removes") + +// OffsiteEndOffered reports whether a press can start the 30 days here: the box has the hub-provisioned tier +// (switched on or not — a tier that is off only makes the deletion wait). +func (m *Manager) OffsiteEndOffered() bool { + return m != nil && m.settings != nil && m.settings.GetOffboxTarget().Pinned() +} + +// SetOffsiteEndNotify wires the household's one mail per press (D84). +func (m *Manager) SetOffsiteEndNotify(fn func(rec settings.OffsiteEnd)) { m.offsiteEndNotify = fn } + +// OffsiteEndDateIfPressedNow is the date the remove dialog shows BEFORE the press. +func (m *Manager) OffsiteEndDateIfPressedNow() time.Time { + return m.now().Add(settings.OffsiteEndAfter) +} + +// StartOffsiteEnd is THE press: „delete backup data” at a removal, or „delete in 30 days” on the Backups page. It +// writes the end record and tells the household once. A second press for the same app changes nothing and tells +// nobody (created == false). The ONLY writer of an end record. +func (m *Manager) StartOffsiteEnd(app string) (rec settings.OffsiteEnd, created bool, err error) { + if !m.OffsiteEndOffered() { + return settings.OffsiteEnd{}, false, ErrOffsiteEndNotOffered + } + if !settings.ValidOffsiteEndApp(app) || app == SharesPseudoStack || m.offsiteEndLiving == nil || m.offsiteEndLiving(app) { + return settings.OffsiteEnd{}, false, ErrOffsiteEndLiving + } + rec, created, err = m.settings.StartOffsiteEnd(app, m.now()) + if err != nil || !created { + return rec, false, err + } + if m.offsiteEndNotify != nil { + m.offsiteEndNotify(rec) + } + return rec, true, nil +} + +// KeepOffsiteCopies is „keep it after all”: it deletes the end record. Returns whether there was one. +func (m *Manager) KeepOffsiteCopies(app string) (bool, error) { + return m.settings.CancelOffsiteEnd(app, "the household keeps the copies") +} + +// PendingOffsiteEnds and OffsiteEndsHistory are what the Backups page shows. +func (m *Manager) PendingOffsiteEnds() []settings.OffsiteEnd { return m.settings.GetOffsiteEnds() } +func (m *Manager) OffsiteEndsHistory() []settings.OffsiteEndDone { + return m.settings.GetOffsiteEndsDone() +} diff --git a/controller/internal/backup/offbox_ended_test.go b/controller/internal/backup/offbox_ended_test.go new file mode 100644 index 00000000..153359dd --- /dev/null +++ b/controller/internal/backup/offbox_ended_test.go @@ -0,0 +1,570 @@ +package backup + +import ( + "context" + "os" + "path/filepath" + "sort" + "strings" + "testing" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// D72 / D84 (`09` §3 decisions 292 and 309; `07-backup-architecture.md` §8.2f): the off-site copies of a removed app +// end 30 days after „delete backup data”, in the weekly window, for ids the hub agreed to. restic is the package's +// fake runner (as in offbox_window_test.go); here it keeps state, so a forget really removes what it names. +// RED-PROOFS: felhom.eu/documentation/audits/build-release-4-2026-11-15/A/redproof-ctrl-window.txt. + +// endedRunner: `snapshots` lists what is left, `forget --dry-run` gives the policy's plan, a real `forget` removes +// the ids it names (and is recorded). +type endedRunner struct { + snaps []guardSnap + plan []guardSnap + forgets [][]string +} + +func (e *endedRunner) run(_ context.Context, _ []string, args ...string) ([]byte, error) { + switch { + case contains(args, "forget") && contains(args, "--dry-run"): + return planJSON(e.plan), nil + case contains(args, "forget"): + e.forgets = append(e.forgets, append([]string{}, args...)) + gone := map[string]bool{} + for _, a := range args { + gone[a] = true + } + var left []guardSnap + for _, s := range e.snaps { + if !gone[s.ID] { + left = append(left, s) + } + } + e.snaps = left + return nil, nil + case contains(args, "snapshots"): + return snapJSON(e.snaps), nil + } + return nil, nil +} + +// removedIDs are the snapshot ids every real forget named, sorted. +func (e *endedRunner) removedIDs() []string { + var out []string + for _, f := range e.forgets { + for _, a := range f { + if strings.HasSuffix(a, "-full") { + out = append(out, a) + } + } + } + sort.Strings(out) + return out +} + +func tagged(id string, at time.Time, tags ...string) guardSnap { + s := snap(id, at) + s.Tags = append([]string{OffboxMarkerTag}, tags...) + return s +} + +func ids(names ...string) []string { + out := make([]string, 0, len(names)) + for _, n := range names { + out = append(out, n+"-full") + } + sort.Strings(out) + return out +} + +// agreeingHub is the fake hub: it grants the window and agrees to exactly the ids in `allow` — or, with allowAll, to +// EVERY id in the repository (a hub that would agree to anything: the box must still remove only its own candidates). +type agreeingHub struct { + fakeWindow + allow []string + allowAll func() []string + refuse []OffsiteEndedRefusal + maxRm int +} + +func (h *agreeingHub) Open(ctx context.Context, n int, ended []OffsiteEndedClaim) (OffsiteWindow, error) { + h.opened++ + h.claims = ended + w := OffsiteWindow{Granted: true, ID: 7, NewestAllowed: time.Now(), MaxRemove: h.maxRm, EndedAllowed: h.allow, EndedRefused: h.refuse} + if h.allowAll != nil { + w.EndedAllowed = h.allowAll() + } + return w, nil +} + +// endedBox: a pinned box whose living check says `living` apps are installed. The repository, 60 days old: +// vikunja ×3 (v1..v3), immich ×2 (i1,i2), one snapshot of BOTH (vi), the shares (s1), kimai ×1 (k1). +func endedBox(t *testing.T, living ...string) (*Manager, *settings.Settings, *endedRunner, *agreeingHub) { + t.Helper() + m, sett := newOffboxManager(t) + pinTarget(t, sett) + old := time.Now().Add(-60 * 24 * time.Hour) + er := &endedRunner{snaps: []guardSnap{ + tagged("v1", old, "vikunja"), tagged("v2", old.Add(time.Hour), "vikunja"), tagged("v3", old.Add(2*time.Hour), "vikunja"), + tagged("i1", old, "immich"), tagged("i2", old.Add(time.Hour), "immich"), + tagged("vi", old, "vikunja", "immich"), + tagged("s1", old, SharesPseudoStack), + tagged("k1", old, "kimai"), + }} + m.SetOffboxRunner(er.run) + hub := &agreeingHub{maxRm: 5} + hub.allowAll = func() []string { + var all []string + for _, s := range er.snaps { + all = append(all, s.ID) + } + return all + } + m.SetOffsiteWindowClient(hub) + live := map[string]bool{} + for _, l := range living { + live[l] = true + } + m.SetOffsiteEndLiving(func(app string) bool { return live[app] }) + return m, sett, er, hub +} + +// pressed writes an end record as the household's press `daysAgo` days ago. +func pressed(t *testing.T, sett *settings.Settings, app string, daysAgo int) settings.OffsiteEnd { + t.Helper() + rec, created, err := sett.StartOffsiteEnd(app, time.Now().Add(-time.Duration(daysAgo)*24*time.Hour)) + if err != nil || !created { + t.Fatalf("press for %s: created=%v err=%v", app, created, err) + } + return rec +} + +// MUST-HOLD 1 — NO RECORD, NO DELETION, EVER. The hub here agrees to EVERY id in the repository, and still nothing is +// removed: a removal that kept the backups, a rebuilt guest and an app removed before this was built all look the +// same on the box — there is no record. +func TestD72_NoRecordNoDeletion(t *testing.T) { + for _, name := range []string{ + "removed without „delete backup data”", + "a rebuilt guest (its settings are new)", + "an app removed before this was built", + } { + t.Run(name, func(t *testing.T) { + m, _, er, hub := endedBox(t, "immich") + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if hub.opened != 1 || len(hub.claims) != 0 { + t.Fatalf("opened=%d claims=%+v — with no record the box must claim nothing", hub.opened, hub.claims) + } + if got := er.removedIDs(); len(got) != 0 { + t.Fatalf("with NO end record the window removed %v", got) + } + if len(er.snaps) != 8 { + t.Fatalf("%d snapshots left of 8", len(er.snaps)) + } + if len(hub.closed) != 1 || len(hub.closed[0].RemovedForEndedApps) != 0 { + t.Fatalf("the result names ended removals: %+v", hub.closed) + } + }) + } +} + +// MUST-HOLD 3 and 5 — the due app's own snapshots go, by id, in the window; the snapshot it shares with a living +// app, the living app's snapshots and the shares stay; the result names them apart; the record becomes history. +func TestD72_TheEndedAppsOwnSnapshotsGoAndNothingElse(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + rec := pressed(t, sett, "vikunja", 31) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + + if len(hub.claims) != 1 || hub.claims[0].App != "vikunja" || strings.Join(hub.claims[0].SnapshotIDs, ",") != strings.Join(ids("v1", "v2", "v3"), ",") { + t.Fatalf("the claim sent to the hub = %+v, want vikunja with v1,v2,v3 only (not the snapshot shared with immich)", hub.claims) + } + if hub.claims[0].RemovedAt != rec.RemovedAt || hub.claims[0].EndsAt != rec.EndsAt { + t.Fatalf("the claim does not carry the record's dates: %+v vs %+v", hub.claims[0], rec) + } + if got := er.removedIDs(); strings.Join(got, ",") != strings.Join(ids("v1", "v2", "v3"), ",") { + t.Fatalf("removed %v, want exactly vikunja's own three", got) + } + left := map[string]bool{} + for _, s := range er.snaps { + left[s.ShortID] = true + } + for _, keep := range []string{"i1", "i2", "vi", "s1", "k1"} { + if !left[keep] { + t.Errorf("%s was removed — a living app's copy, the shares, or another app's", keep) + } + } + if len(er.forgets) != 1 || !contains(er.forgets[0], "--prune") { + t.Fatalf("want ONE forget … --prune (the window's own call), got %v", er.forgets) + } + if len(hub.closed) != 1 { + t.Fatalf("the window was not closed once: %+v", hub.closed) + } + r := hub.closed[0] + if r.Outcome != "pruned" || r.CountBefore != 8 || r.CountAfter != 5 || strings.Join(r.RemovedForEndedApps, ",") != strings.Join(ids("v1", "v2", "v3"), ",") { + t.Fatalf("result = %+v, want pruned 8 -> 5 with removed_for_ended_apps = v1,v2,v3", r) + } + // vikunja still has the shared snapshot in the store, so its record stays pending (asked again next week). + if _, ok := sett.GetOffsiteEnd("vikunja"); !ok { + t.Fatal("the record was closed although a snapshot with vikunja's tag is still in the store") + } + if len(sett.GetOffsiteEndsDone()) != 0 { + t.Fatalf("a history line was written while a copy is left: %+v", sett.GetOffsiteEndsDone()) + } +} + +// A snapshot with two apps' tags goes only when BOTH have ended — and then the records become history. +func TestD72_ASharedSnapshotGoesWhenBothHaveEnded(t *testing.T) { + m, sett, er, hub := endedBox(t) // nothing is installed + pressed(t, sett, "vikunja", 40) + pressed(t, sett, "immich", 31) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if got := er.removedIDs(); strings.Join(got, ",") != strings.Join(ids("i1", "i2", "v1", "v2", "v3", "vi"), ",") { + t.Fatalf("removed %v, want both apps' snapshots and the shared one", got) + } + if len(er.snaps) != 2 { + t.Fatalf("left %d, want the shares and kimai", len(er.snaps)) + } + if n := len(sett.GetOffsiteEnds()); n != 0 { + t.Fatalf("%d record(s) still pending after every copy went", n) + } + h := sett.GetOffsiteEndsDone() + if len(h) != 2 { + t.Fatalf("history = %+v, want one line per app", h) + } + got := map[string]int{} + for _, d := range h { + got[d.App] = d.Snapshots + } + if got["vikunja"] != 4 || got["immich"] != 3 { + t.Fatalf("history counts = %v, want vikunja 4 (three + the shared), immich 3", got) + } + _ = hub +} + +// MUST-HOLD 3 — `_shares` and the box's own stacks are never eligible, even with a record that somehow exists. +func TestD72_SharesAndInfraAreNeverEligible(t *testing.T) { + now := time.Now() + old := now.Add(-60 * 24 * time.Hour) + rec := func(app string) settings.OffsiteEnd { + return settings.OffsiteEnd{App: app, RemovedAt: now.Add(-40 * 24 * time.Hour).Format(time.RFC3339), EndsAt: now.Add(-10 * 24 * time.Hour).Format(time.RFC3339)} + } + living := func(app string) bool { return app == "traefik" } // the box's own stacks answer true (main.go: IsProtectedStack) + due := dueOffsiteEnds([]settings.OffsiteEnd{rec(SharesPseudoStack), rec(OffboxMarkerTag), rec("traefik"), rec("vikunja")}, now, living) + if len(due) != 1 || due["vikunja"].App != "vikunja" { + t.Fatalf("due = %v, want vikunja only", due) + } + claims, cand := endedCandidates([]guardSnap{tagged("s1", old, SharesPseudoStack), tagged("t1", old, "traefik"), tagged("v1", old, "vikunja"), + {ID: "u1-full", ShortID: "u1", Time: old, Tags: []string{OffboxMarkerTag}}, {ID: "n1-full", ShortID: "n1", Time: old}}, due) + if len(claims) != 1 || len(cand) != 1 || !cand["v1-full"] { + t.Fatalf("claims=%+v candidates=%v — the shares, an infra stack, a marker-only and an untagged snapshot are never candidates", claims, cand) + } + // and no such record can be written by the press + m, sett, _, _ := endedBox(t, "traefik") + for _, app := range []string{SharesPseudoStack, OffboxMarkerTag, "traefik"} { + if _, created, err := m.StartOffsiteEnd(app); err == nil || created { + t.Errorf("%q got an end record", app) + } + } + if n := len(sett.GetOffsiteEnds()); n != 0 { + t.Fatalf("%d record(s) written", n) + } +} + +// MUST-HOLD 2, the box's half — THE BOX REMOVES ONLY WHAT THE HUB AGREED TO. An older hub (no answer), a refusal, and +// a box whose clock is a month ahead (it asks on day 2; the hub, by its own clock, refuses): nothing of the app goes +// and the record stays for the next window. +func TestD84_TheBoxRemovesOnlyWhatTheHubAgreedTo(t *testing.T) { + for _, c := range []struct { + name string + allow []string + refuse []OffsiteEndedRefusal + want []string + }{ + {"a hub that does not know ended apps answers nothing", nil, nil, nil}, + {"the hub refuses the app (the box's clock is a month ahead: absent 2 days by the hub's clock)", nil, + []OffsiteEndedRefusal{{App: "vikunja", Reason: "the hub has seen the app absent since … — 2 full day(s), 30 needed"}}, nil}, + {"the hub agrees to two of the three ids", ids("v1", "v3"), nil, ids("v1", "v3")}, + {"the hub „agrees” to ids the box never named (a living app's, the shares)", ids("i1", "i2", "s1", "vi", "v2"), nil, ids("v2")}, + } { + t.Run(c.name, func(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + hub.allowAll, hub.allow, hub.refuse = nil, c.allow, c.refuse + pressed(t, sett, "vikunja", 31) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if got := er.removedIDs(); strings.Join(got, ",") != strings.Join(c.want, ",") { + t.Fatalf("removed %v, want %v", got, c.want) + } + if _, ok := sett.GetOffsiteEnd("vikunja"); !ok { + t.Fatal("the record is gone although copies of the app are left — the next window could not finish the job") + } + if got := hub.closed[0].RemovedForEndedApps; strings.Join(got, ",") != strings.Join(c.want, ",") { + t.Fatalf("removed_for_ended_apps = %v, want %v", got, c.want) + } + }) + } +} + +// MUST-HOLD 5 — BEHIND THE SAME GUARD, AND THE CAP'S OWN RULE IS NOT TOUCHED. (a) the guard refuses (a future-dated +// snapshot): the ended apps wait with everything else. (b) the policy's plan is above the cap: refused as always — +// the ended apps do not push it through and do not go themselves. (c) the plan is exactly the cap: it goes, and the +// agreed ended ids go BESIDE it — more than the cap in one window. +func TestD84_OneGuard_AndTheAllowanceIsBesideTheCap(t *testing.T) { + old := time.Now().Add(-60 * 24 * time.Hour) + policy := func(n int) []guardSnap { // n old snapshots of the living app the policy wants to remove + var out []guardSnap + for i := 0; i < n; i++ { + out = append(out, tagged("p"+string(rune('a'+i)), old.Add(time.Duration(i)*time.Minute), "immich")) + } + return out + } + t.Run("the guard refuses: nothing goes, the ended apps wait", func(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + pressed(t, sett, "vikunja", 31) + er.snaps = append(er.snaps, tagged("fut", time.Now().Add(72*time.Hour), "immich")) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if len(er.forgets) != 0 || hub.closed[0].Outcome != "guard-refused" || len(hub.closed[0].RemovedForEndedApps) != 0 { + t.Fatalf("forgets=%v result=%+v — a refused guard must delete nothing", er.forgets, hub.closed[0]) + } + if _, ok := sett.GetOffsiteEnd("vikunja"); !ok { + t.Fatal("the record was lost at a refused window") + } + }) + t.Run("the policy's plan is above the cap: refused, with or without ended apps", func(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + pressed(t, sett, "vikunja", 31) + er.plan = policy(6) // cap 5 + er.snaps = append(er.snaps, er.plan...) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if len(er.forgets) != 0 || hub.closed[0].Outcome != "guard-refused" || !strings.Contains(hub.closed[0].Reason, "more than one week") { + t.Fatalf("forgets=%v result=%+v — the cap's own refusal must stand", er.forgets, hub.closed[0]) + } + }) + t.Run("the plan is exactly the cap: it goes, and the ended ids go beside it", func(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + pressed(t, sett, "vikunja", 31) + er.plan = policy(5) // cap 5 + er.snaps = append(er.snaps, er.plan...) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + got := er.removedIDs() + if len(got) != 8 { + t.Fatalf("removed %d (%v), want the policy's 5 and vikunja's 3", len(got), got) + } + r := hub.closed[0] + if r.Outcome != "pruned" || r.CountBefore-r.CountAfter != 8 || len(r.RemovedForEndedApps) != 3 { + t.Fatalf("result = %+v, want a fall of 8 with 3 named for ended apps (so 5 are under the cap)", r) + } + }) + t.Run("an ended id the policy already removes stays under the cap and is not named twice", func(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + pressed(t, sett, "vikunja", 31) + er.plan = []guardSnap{er.snaps[0]} // v1 is in the policy's own plan + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if got := hub.closed[0].RemovedForEndedApps; strings.Join(got, ",") != strings.Join(ids("v2", "v3"), ",") { + t.Fatalf("removed_for_ended_apps = %v, want v2,v3 (v1 went under the cap)", got) + } + if got := er.removedIDs(); len(got) != 3 { + t.Fatalf("removed %v", got) + } + }) +} + +// WRONG CASE — an ended app that is INSTALLED AGAIN the day before the window. The install deletes the record +// (stacks.DeployStack; TestD72_AnInstallDeletesTheEndRecord). And if a record were still there: an installed app is +// never due — nothing is claimed, nothing goes. +func TestD72_AnAppInstalledAgainIsNeverDue(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich", "vikunja") // vikunja is installed again + pressed(t, sett, "vikunja", 31) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if len(hub.claims) != 0 || len(er.removedIDs()) != 0 { + t.Fatalf("claims=%+v removed=%v — an installed app's history must not be touched", hub.claims, er.removedIDs()) + } +} + +// A box that cannot tell what it runs (the seam is not wired) has nothing due and can write no record. +func TestD72_WithoutTheLivingCheckNothingIsDue(t *testing.T) { + m, sett, er, hub := endedBox(t) + m.SetOffsiteEndLiving(nil) + pressed(t, sett, "vikunja", 31) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if len(hub.claims) != 0 || len(er.removedIDs()) != 0 { + t.Fatalf("claims=%+v removed=%v", hub.claims, er.removedIDs()) + } + if _, created, err := m.StartOffsiteEnd("kimai"); err == nil || created { + t.Fatal("a press wrote a record on a box that cannot tell what is installed") + } +} + +// A record that is not due yet (29 days) claims nothing; a snapshot made under the app's tag AFTER the removal is +// not the removed app's copy and is never a candidate. +func TestD72_NotDueYet_AndASnapshotYoungerThanTheRemoval(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + pressed(t, sett, "vikunja", 29) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if len(hub.claims) != 0 || len(er.removedIDs()) != 0 { + t.Fatalf("29 days: claims=%+v removed=%v", hub.claims, er.removedIDs()) + } + + m2, sett2, er2, hub2 := endedBox(t, "immich") + pressed(t, sett2, "kimai", 31) + er2.snaps = append(er2.snaps, tagged("k2", time.Now().Add(-20*24*time.Hour), "kimai")) // 11 days AFTER the removal + m2.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if got := er2.removedIDs(); strings.Join(got, ",") != strings.Join(ids("k1"), ",") { + t.Fatalf("removed %v, want k1 only — k2 was made after the removal", got) + } + if _, ok := sett2.GetOffsiteEnd("kimai"); !ok { + t.Fatal("the record closed although a snapshot with the tag is left") + } + _ = hub2 +} + +// An app that has NO copy in the store: its due record is closed with no history line and nothing is asked of the hub. +func TestD72_ARecordWithNoCopiesIsClosedQuietly(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + pressed(t, sett, "neverbackedup", 31) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if len(hub.claims) != 0 || len(er.removedIDs()) != 0 { + t.Fatalf("claims=%+v removed=%v", hub.claims, er.removedIDs()) + } + if _, ok := sett.GetOffsiteEnd("neverbackedup"); ok || len(sett.GetOffsiteEndsDone()) != 0 { + t.Fatalf("want the record closed with no history line; pending=%v history=%+v", ok, sett.GetOffsiteEndsDone()) + } +} + +// WRONG CASE — THE OFF-SITE TIER IS SWITCHED OFF AT THE DATE. No run, no window, no call to the store; the record +// stays and does nothing. And the household's own NAS target (no hub window) never removes an ended app's copies. +func TestD72_TierOffAtTheDate_TheRecordWaits(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + rec := pressed(t, sett, "vikunja", 31) + if err := sett.UpdateOffboxStatus(func(o *settings.OffboxTarget) { o.Enabled = false }); err != nil { + t.Fatal(err) + } + _ = m.RunOffboxBackup(context.Background()) + if hub.opened != 0 || len(er.forgets) != 0 { + t.Fatalf("with the tier OFF the box opened %d window(s) and ran %d forget(s)", hub.opened, len(er.forgets)) + } + if got, ok := sett.GetOffsiteEnd("vikunja"); !ok || got != rec { + t.Fatalf("the record changed while the tier was off: %+v ok=%v", got, ok) + } + + // The household's own NAS: the box prunes by the policy alone; no id of an ended app is ever named. + m2, sett2 := newOffboxManager(t) // Transport "" = the NAS + if _, _, err := sett2.StartOffsiteEnd("vikunja", time.Now().Add(-31*24*time.Hour)); err != nil { + t.Fatal(err) + } + er2 := &endedRunner{snaps: []guardSnap{tagged("v1", time.Now().Add(-60*24*time.Hour), "vikunja")}} + m2.SetOffboxRunner(er2.run) + m2.SetOffsiteEndLiving(func(string) bool { return false }) + m2.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if got := er2.removedIDs(); len(got) != 0 { + t.Fatalf("on the household's own NAS the box removed %v by id", got) + } + if _, created, err := m2.StartOffsiteEnd("kimai"); err != ErrOffsiteEndNotOffered || created { + t.Fatalf("a press on a box without the hub's tier: created=%v err=%v, want ErrOffsiteEndNotOffered and no record", created, err) + } +} + +// MUST-HOLD 7 and the wrong case TWO PRESSES — one record, ONE mail; the date does not move. A press for an +// installed app writes nothing and mails nobody. +func TestD84_OnePressOneMail(t *testing.T) { + m, sett, _, _ := endedBox(t, "immich") + var mails []settings.OffsiteEnd + m.SetOffsiteEndNotify(func(rec settings.OffsiteEnd) { mails = append(mails, rec) }) + first, created, err := m.StartOffsiteEnd("vikunja") + if err != nil || !created { + t.Fatalf("first press: created=%v err=%v", created, err) + } + if first.Ends().Sub(first.Removed()) != settings.OffsiteEndAfter { + t.Fatalf("the record's end is not 30 days after the press: %+v", first) + } + second, created2, err := m.StartOffsiteEnd("vikunja") + if err != nil || created2 || second != first { + t.Fatalf("second press: created=%v rec=%+v err=%v — want the first record, unchanged", created2, second, err) + } + if len(mails) != 1 || mails[0] != first { + t.Fatalf("two presses gave %d mail(s): %+v — want ONE, for the record", len(mails), mails) + } + if _, c, err := m.StartOffsiteEnd("immich"); err != ErrOffsiteEndLiving || c { + t.Fatalf("a press for an INSTALLED app: created=%v err=%v", c, err) + } + if len(mails) != 1 || len(sett.GetOffsiteEnds()) != 1 { + t.Fatalf("mails=%d records=%d after a refused press", len(mails), len(sett.GetOffsiteEnds())) + } + // MUST-HOLD 4 — the household takes it back: the record is gone, and the next window claims nothing. + if kept, err := m.KeepOffsiteCopies("vikunja"); err != nil || !kept { + t.Fatalf("keep: %v %v", kept, err) + } + if n := len(sett.GetOffsiteEnds()); n != 0 { + t.Fatalf("%d record(s) after the take-back", n) + } + if len(mails) != 1 { + t.Fatalf("the take-back sent a mail: %d", len(mails)) + } +} + +// The take-back before the window: with the record deleted, the window removes nothing of the app. +func TestD72_AfterTheTakeBackTheWindowRemovesNothing(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + pressed(t, sett, "vikunja", 31) + if kept, err := m.KeepOffsiteCopies("vikunja"); err != nil || !kept { + t.Fatalf("keep: %v %v", kept, err) + } + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if len(hub.claims) != 0 || len(er.removedIDs()) != 0 { + t.Fatalf("claims=%+v removed=%v after „keep it after all”", hub.claims, er.removedIDs()) + } +} + +// The hub report carries the pending records (so the hub can start its own clock for an app it never saw). +func TestD72_TheReportCarriesThePendingRecords(t *testing.T) { + m, sett, _, _ := endedBox(t, "immich") + if st := m.OffboxReportStatus(); st == nil || len(st.EndedApps) != 0 { + t.Fatalf("no record: %+v", st) + } + rec := pressed(t, sett, "vikunja", 3) + st := m.OffboxReportStatus() + if st == nil || len(st.EndedApps) != 1 || st.EndedApps[0].App != "vikunja" || st.EndedApps[0].RemovedAt != rec.RemovedAt || st.EndedApps[0].EndsAt != rec.EndsAt { + t.Fatalf("report = %+v, want the pending record", st) + } +} + +// The living check and the mail are WIRED in main.go — without the first nothing is ever due, without the second no +// household is told. (A seam nobody wires is a feature that does not exist.) +func TestD72_MainWiresTheLivingCheck(t *testing.T) { + raw, err := os.ReadFile(filepath.Join("..", "..", "cmd", "controller", "main.go")) + if err != nil { + t.Fatal(err) + } + src := string(raw) + for _, want := range []string{"backupMgr.SetOffsiteEndLiving(func(app string) bool {", "cfg.IsProtectedStack(app)", "st.Deployed || st.Deploying || st.Protected", + "backupMgr.SetOffsiteEndNotify(func(rec settings.OffsiteEnd) {", "notifier.NotifyOffsiteEndScheduled(rec.App, rec.RemovedAt, rec.EndsAt"} { + if !strings.Contains(src, want) { + t.Errorf("main.go lacks %q", want) + } + } +} + +// The result names for the hub's audit only ids that are REALLY gone: a forget that removed nothing (the provider +// refused, the call failed) names none, and the record stays for the next window. +func TestD84_TheResultNamesOnlyIdsThatAreReallyGone(t *testing.T) { + m, sett, er, hub := endedBox(t, "immich") + pressed(t, sett, "kimai", 31) + base := er.run + m.SetOffboxRunner(func(ctx context.Context, env []string, args ...string) ([]byte, error) { + if contains(args, "forget") && !contains(args, "--dry-run") { + er.forgets = append(er.forgets, append([]string{}, args...)) + return []byte("Fatal: unable to remove"), context.DeadlineExceeded // nothing is removed + } + return base(ctx, env, args...) + }) + m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") + if len(er.forgets) != 1 || len(hub.closed) != 1 { + t.Fatalf("forgets=%d closed=%d", len(er.forgets), len(hub.closed)) + } + r := hub.closed[0] + if r.Outcome != "error" || len(r.RemovedForEndedApps) != 0 || r.CountAfter != r.CountBefore { + t.Fatalf("result = %+v — a failed forget removed nothing, so nothing may be named as removed for ended apps", r) + } + if _, ok := sett.GetOffsiteEnd("kimai"); !ok || len(sett.GetOffsiteEndsDone()) != 0 { + t.Fatal("the record was closed although its copy is still in the store") + } +} diff --git a/controller/internal/backup/offbox_window.go b/controller/internal/backup/offbox_window.go index 6707d077..fb1d6b69 100644 --- a/controller/internal/backup/offbox_window.go +++ b/controller/internal/backup/offbox_window.go @@ -56,6 +56,10 @@ type OffsiteWindow struct { NewestAllowed time.Time MaxRemove int Reason string // why not granted (logged) + // D84 (`09` §3 decision 309): the snapshot ids of ENDED apps the hub agreed to — their own allowance, outside + // MaxRemove — and what it refused. Empty from a hub that does not know ended apps: then none is removed. + EndedAllowed []string + EndedRefused []OffsiteEndedRefusal } // OffsiteWindowResult is what the box reports when it is done (the hub closes the window on it). @@ -66,11 +70,16 @@ type OffsiteWindowResult struct { Removed int `json:"removed"` Outcome string `json:"outcome"` // pruned | nothing | guard-refused | error Reason string `json:"reason,omitempty"` + // RemovedForEndedApps names the snapshots this window removed for ended apps (D84) — only ids the hub agreed + // to, and only those that are really gone. The hub's 07:10 audit explains its vanished files by this list. + RemovedForEndedApps []string `json:"removed_for_ended_apps,omitempty"` } // OffsiteWindowClient is the hub side (offsiteapply.HubWindowClient in production). type OffsiteWindowClient interface { - Open(ctx context.Context, countBefore int) (OffsiteWindow, error) + // Open asks for a window. ended are the box's DUE off-site end records with the ids it would remove (D72/D84); + // the hub answers which ids it agrees to. + Open(ctx context.Context, countBefore int, ended []OffsiteEndedClaim) (OffsiteWindow, error) Close(ctx context.Context, r OffsiteWindowResult) error } @@ -225,7 +234,11 @@ func (m *Manager) offsiteWindowRetention(ctx context.Context, base, env []string m.logger.Printf("[WARN] [offbox] retention skipped (%s): %v", why, err) return } - w, err := m.offsiteWindow.Open(ctx, len(snaps)) + // D72/D84: the ended apps' candidates, from the records that are due. No record → no candidate. + due := dueOffsiteEnds(m.settings.GetOffsiteEnds(), m.now(), m.offsiteEndLiving) + m.dropEndsWithNoCopies(snaps, due) + claims, candidate := endedCandidates(snaps, due) + w, err := m.offsiteWindow.Open(ctx, len(snaps), claims) if err != nil { m.logger.Printf("[WARN] [offbox] retention skipped (%s): asking the hub for a window failed: %v", why, err) return @@ -255,6 +268,15 @@ func (m *Manager) offsiteWindowRetention(ctx context.Context, base, env []string m.logger.Printf("[ERROR] [offbox] clean-up window %d: the fake-snapshot guard REFUSED — nothing deleted: %s (R-822)", w.ID, refuse) return } + for _, r := range w.EndedRefused { + m.logger.Printf("[WARN] [offbox] clean-up window %d: the hub REFUSED the ended app %q — nothing of it is removed: %s", w.ID, r.App, r.Reason) + } + // The ended apps' ids: outside the cap, only what the hub agreed to AND this box named in this window. + endedIDs := endedToRemove(w.EndedAllowed, candidate, ids) + if len(endedIDs) > 0 { + m.logger.Printf("[INFO] [offbox] clean-up window %d: %d snapshot(s) of removed apps are due and agreed by the hub — removed beside the policy's %d (D84)", w.ID, len(endedIDs), len(ids)) + } + ids = append(ids, endedIDs...) if len(ids) == 0 { res.Outcome = "nothing" m.logger.Printf("[INFO] [offbox] clean-up window %d: the policy removes nothing", w.ID) @@ -271,8 +293,19 @@ func (m *Manager) offsiteWindowRetention(ctx context.Context, base, env []string } if after, lerr := m.listGuardSnaps(ctx, base, env); lerr == nil { res.CountAfter = len(after) + // Named apart for the hub's audit — only the agreed ids that are REALLY gone from the repository. + still := make(map[string]bool, len(after)) + for _, s := range after { + still[s.ID] = true + } + for _, id := range endedIDs { + if !still[id] { + res.RemovedForEndedApps = append(res.RemovedForEndedApps, id) + } + } + m.completeOffsiteEnds(claims, res.RemovedForEndedApps, after) } res.Removed = res.CountBefore - res.CountAfter - m.logger.Printf("[INFO] [offbox] clean-up window %d (%s): %d of %d planned snapshot(s) removed, %d -> %d, in %s", - w.ID, why, res.Removed, len(plan), res.CountBefore, res.CountAfter, time.Since(start).Round(time.Second)) + m.logger.Printf("[INFO] [offbox] clean-up window %d (%s): %d of %d planned snapshot(s) removed (%d of them for removed apps), %d -> %d, in %s", + w.ID, why, res.Removed, len(ids), len(res.RemovedForEndedApps), res.CountBefore, res.CountAfter, time.Since(start).Round(time.Second)) } diff --git a/controller/internal/backup/offbox_window_test.go b/controller/internal/backup/offbox_window_test.go index d22881a8..a49349ed 100644 --- a/controller/internal/backup/offbox_window_test.go +++ b/controller/internal/backup/offbox_window_test.go @@ -56,13 +56,15 @@ func (w *windowRunner) run(_ context.Context, _ []string, args ...string) ([]byt } type fakeWindow struct { + claims []OffsiteEndedClaim grant OffsiteWindow opened int closed []OffsiteWindowResult } -func (f *fakeWindow) Open(context.Context, int) (OffsiteWindow, error) { +func (f *fakeWindow) Open(_ context.Context, _ int, ended []OffsiteEndedClaim) (OffsiteWindow, error) { f.opened++ + f.claims = ended return f.grant, nil } func (f *fakeWindow) Close(_ context.Context, r OffsiteWindowResult) error { diff --git a/controller/internal/i18n/locales/en.json b/controller/internal/i18n/locales/en.json index 3f0f0dfa..3ee80544 100644 --- a/controller/internal/i18n/locales/en.json +++ b/controller/internal/i18n/locales/en.json @@ -2888,5 +2888,16 @@ "err.backup.restore_image_gone": "The version of this backup (%s) cannot be downloaded: its maker no longer offers it. We did not touch the app. Choose another backup.", "err.backup.restore_image_unreachable": "The version of this backup (%s) cannot be downloaded right now, so we did not touch the app. Try again later, or choose another backup.", "kept.backup.future_note": "(its date is in the future — the clock of the box that made it may have been wrong)", - "kept.choice.use.future": "Note: the date of this backup (%s) is in the future. The clock of the box that made it may have been wrong. Choose it only if you are sure it is your backup." + "kept.choice.use.future": "Note: the date of this backup (%s) is in the future. The clock of the box that made it may have been wrong. Choose it only if you are sure it is your backup.", + "offsite_end.dialog": "The local backups are deleted now. The off-site copies stay for 30 more days. They are deleted on: {date}. Until then you can take this back on the Backups page.", + "offsite_end.marked": "Marked for deletion. Deleted on: %s.", + "offsite_end.keep_button": "Keep it after all", + "offsite_end.kept_before": "The off-site copies of this app were kept.", + "offsite_end.start_button": "Delete in 30 days", + "offsite_end.done": "The off-site copies were deleted on %s.", + "flash.offsite_end.kept": "The off-site copies are kept.", + "flash.offsite_end.nothing_to_keep": "The off-site copies of this app are not marked for deletion, or are already deleted.", + "flash.offsite_end.not_possible": "The off-site copies of this app cannot be marked for deletion now.", + "flash.offsite_end.failed": "This could not be saved. Try again.", + "event.offsite_end_scheduled": "%s: the off-site copies of this app are marked for deletion. They are deleted on: %s. Until then you can take this back on the Backups page with the “Keep it after all” button." } diff --git a/controller/internal/i18n/locales/hu.json b/controller/internal/i18n/locales/hu.json index a158d658..52fbc082 100644 --- a/controller/internal/i18n/locales/hu.json +++ b/controller/internal/i18n/locales/hu.json @@ -2876,5 +2876,16 @@ "err.backup.restore_image_gone": "Ennek a mentésnek a verzióját (%s) nem lehet letölteni: a készítője már nem kínálja. Az alkalmazáshoz nem nyúltunk. Válassz egy másik mentést.", "err.backup.restore_image_unreachable": "Ennek a mentésnek a verzióját (%s) most nem lehet letölteni, ezért az alkalmazáshoz nem nyúltunk. Próbáld újra később, vagy válassz egy másik mentést.", "kept.backup.future_note": "(a dátuma a jövőben van — a mentést készítő doboz órája rosszul járhatott)", - "kept.choice.use.future": "Figyelem: ennek a mentésnek a dátuma (%s) a jövőben van. A mentést készítő doboz órája rosszul járhatott. Csak akkor válaszd, ha biztosan ez a ti mentésetek." + "kept.choice.use.future": "Figyelem: ennek a mentésnek a dátuma (%s) a jövőben van. A mentést készítő doboz órája rosszul járhatott. Csak akkor válaszd, ha biztosan ez a ti mentésetek.", + "offsite_end.dialog": "A helyi mentések most törlődnek. A távoli másolatok még 30 napig megmaradnak. Törlésük napja: {date} Addig a Mentések oldalon visszavonhatod.", + "offsite_end.marked": "Törlésre kijelölve. Törlés napja: %s", + "offsite_end.keep_button": "Mégis megtartom", + "offsite_end.kept_before": "Ennek az alkalmazásnak a távoli másolatai megmaradtak.", + "offsite_end.start_button": "Törlés 30 nap múlva", + "offsite_end.done": "A távoli másolatok törölve: %s", + "flash.offsite_end.kept": "A távoli másolatok megmaradnak.", + "flash.offsite_end.nothing_to_keep": "Ennek az alkalmazásnak a távoli másolatai nincsenek törlésre kijelölve, vagy már törlődtek.", + "flash.offsite_end.not_possible": "Ennek az alkalmazásnak a távoli másolatait most nem lehet törlésre kijelölni.", + "flash.offsite_end.failed": "Nem sikerült elmenteni. Próbáld újra.", + "event.offsite_end_scheduled": "%s: az alkalmazás távoli másolatai törlésre vannak kijelölve. Törlésük napja: %s Addig a Mentések oldalon a „Mégis megtartom” gombbal visszavonhatod." } diff --git a/controller/internal/notify/d84_offsite_end_scheduled_test.go b/controller/internal/notify/d84_offsite_end_scheduled_test.go new file mode 100644 index 00000000..8bf46737 --- /dev/null +++ b/controller/internal/notify/d84_offsite_end_scheduled_test.go @@ -0,0 +1,71 @@ +package notify + +import ( + "encoding/json" + "io" + "log" + "path/filepath" + "strings" + "testing" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// D84 (`09` §3 decision 309; `07-backup-architecture.md` §8.2f): the household's ONE mail at the press names the app, +// the end date and how to take it back — in the household's language, with the date in that language's form. +// RED-PROOF: felhom.eu/documentation/audits/build-release-4-2026-11-15/A/redproof-ctrl-press.txt. +func TestD84_OffsiteEndScheduledEvent(t *testing.T) { + ends := time.Date(2026, 12, 15, 10, 30, 0, 0, time.UTC) + for _, lang := range []string{"hu", "en"} { + sett, err := settings.Load(filepath.Join(t.TempDir(), "settings.json"), log.New(io.Discard, "", 0)) + if err != nil { + t.Fatal(err) + } + if err := sett.SetLanguage(lang); err != nil { + t.Fatal(err) + } + n := &Notifier{settings: sett, logger: log.New(io.Discard, "", 0)} + var et, sev, msg, cust string + var det []byte + calls := 0 + n.pushFn = func(eventType, severity, message, customer string, details interface{}) { + calls++ + et, sev, msg, cust = eventType, severity, message, customer + det, _ = json.Marshal(details) + } + n.NotifyOffsiteEndScheduled("vikunja", "2026-11-15T10:30:00Z", "2026-12-15T10:30:00Z", ends) + if calls != 1 || et != "offsite_end_scheduled" || sev != "info" { + t.Fatalf("%s: calls=%d type=%q severity=%q", lang, calls, et, sev) + } + // the wire sentence is always Hungarian: the app, the day, the way back + for _, want := range []string{"vikunja", "2026. 12. 15.", "Mentések oldalon", "„Mégis megtartom”"} { + if !strings.Contains(msg, want) { + t.Errorf("%s: the Hungarian sentence %q lacks %q", lang, msg, want) + } + } + if strings.Contains(msg, "15..") || strings.Contains(msg, "%") { + t.Errorf("%s: the Hungarian sentence is malformed: %q", lang, msg) + } + if lang == "hu" { + if cust != "" { + t.Errorf("a Hungarian household sends nothing extra, got %q", cust) + } + } else { + for _, want := range []string{"vikunja", "2026-12-15", "Backups page", "“Keep it after all”"} { + if !strings.Contains(cust, want) { + t.Errorf("the English sentence %q lacks %q", cust, want) + } + } + if strings.Contains(cust, "2026. 12.") || strings.Contains(cust, "%") { + t.Errorf("the English sentence carries the Hungarian date form or a verb: %q", cust) + } + } + // the details: stack_name (the hub's headline names the app from it) and the press's own key + for _, want := range []string{`"stack_name":"vikunja"`, `"removed_at":"2026-11-15T10:30:00Z"`, `"ends_at":"2026-12-15T10:30:00Z"`, `"subject":"vikunja|2026-11-15T10:30:00Z"`} { + if !strings.Contains(string(det), want) { + t.Errorf("%s: details %s lack %s", lang, det, want) + } + } + } +} diff --git a/controller/internal/notify/notifier.go b/controller/internal/notify/notifier.go index 7026a85b..2c9fca7b 100644 --- a/controller/internal/notify/notifier.go +++ b/controller/internal/notify/notifier.go @@ -1604,6 +1604,37 @@ func (n *Notifier) NotifyRestoreCopyDeleted(d RestoreCopyDeletedDetails) { n.pushEventBoth("restore_copy_deleted", "info", msg, "", d) } +// OffsiteEndScheduledDetails is offsite_end_scheduled's record. Subject is the press's own key (app + the record's +// time): the hub mails once per subject, so two apps removed in one hour are two mails and one press delivered twice +// is one. StackName is what the hub's mail headline names the app from. +type OffsiteEndScheduledDetails struct { + StackName string `json:"stack_name"` + RemovedAt string `json:"removed_at"` // RFC 3339, UTC + EndsAt string `json:"ends_at"` // RFC 3339, UTC + Subject string `json:"subject"` +} + +// NotifyOffsiteEndScheduled (D84, `09` §3 decision 309; `07` §8.2f): the household pressed „delete backup data” for +// an app — its off-site copies end on endsOn. ONE mail per press to the household's own address, naming the app, the +// day and the way back; a receipt on the hub (never switched off by a group, never mailed to the operator). `info`: +// nothing is wrong. The day is written in each language's own date form, so both sentences are rendered here. +// Pinned by TestD84_OffsiteEndScheduledEvent. +func (n *Notifier) NotifyOffsiteEndScheduled(app, removedAt, endsAt string, endsOn time.Time) { + const key = "event.offsite_end_scheduled" + d := OffsiteEndScheduledDetails{StackName: app, RemovedAt: removedAt, EndsAt: endsAt, Subject: app + "|" + removedAt} + b, err := i18n.Shared() + if err != nil { + n.logger.Printf("[ERROR] NotifyOffsiteEndScheduled: bundle unavailable: %v", err) + n.pushEventBoth("offsite_end_scheduled", "info", key, "", d) + return + } + household := "" + if lang := n.boxLang(); lang != i18n.Default { + household = b.Msgf(lang, key, app, i18n.FormatDate(lang, endsOn)) + } + n.pushEventBoth("offsite_end_scheduled", "info", b.Msgf(i18n.Default, key, app, i18n.FormatDate(i18n.Default, endsOn)), household, d) +} + // healthSeverity is the event severity a health status would be sent at (R-647): the disabled path // names what it drops, and "warn" is a health STATUS, not a severity the hub knows. func healthSeverity(status string) string { diff --git a/controller/internal/offsiteapply/d84_window_client_test.go b/controller/internal/offsiteapply/d84_window_client_test.go new file mode 100644 index 00000000..a1b7609f --- /dev/null +++ b/controller/internal/offsiteapply/d84_window_client_test.go @@ -0,0 +1,71 @@ +package offsiteapply + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "gitea.dooplex.hu/admin/felhom-controller/internal/backup" +) + +// D72 / D84 (`07-backup-architecture.md` §8.2f): the window request carries the box's due end records, the answer +// carries what the hub agreed to — and only a GRANTED window carries an agreement. +// RED-PROOF: felhom.eu/documentation/audits/build-release-4-2026-11-15/A/redproof-ctrl-window.txt. +func TestD84_WindowClient_SendsTheClaimsAndReadsTheAgreement(t *testing.T) { + var gotPath string + var gotBody map[string]json.RawMessage + answer := `{"granted":true,"window_id":9,"newest_allowed":"2026-12-20T03:00:00Z","max_remove":10,"ended_allowed":["aa","bb"],"ended_refused":[{"app":"immich","reason":"the box's newest report lists the app as installed"}]}` + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotPath = r.URL.Path + b, _ := io.ReadAll(r.Body) + gotBody = map[string]json.RawMessage{} + _ = json.Unmarshal(b, &gotBody) + _, _ = w.Write([]byte(answer)) + })) + defer srv.Close() + c := HubWindowClient{Registrar: HubRegistrar{HubURL: srv.URL, CustomerID: "c1", APIKey: "k"}} + claims := []backup.OffsiteEndedClaim{{App: "vikunja", RemovedAt: "2026-11-15T10:00:00Z", EndsAt: "2026-12-15T10:00:00Z", SnapshotIDs: []string{"aa", "bb", "cc"}}} + w, err := c.Open(context.Background(), 20, claims) + if err != nil { + t.Fatal(err) + } + if gotPath != "/api/v1/offsite/window-open/c1" || string(gotBody["count_before"]) != "20" { + t.Fatalf("request: path=%q body=%v", gotPath, gotBody) + } + var sent []backup.OffsiteEndedClaim + if err := json.Unmarshal(gotBody["ended"], &sent); err != nil || len(sent) != 1 || sent[0].App != "vikunja" || strings.Join(sent[0].SnapshotIDs, ",") != "aa,bb,cc" || sent[0].RemovedAt != "2026-11-15T10:00:00Z" { + t.Fatalf("the hub did not receive the claim: %s (%v)", gotBody["ended"], err) + } + if !w.Granted || w.ID != 9 || w.MaxRemove != 10 || strings.Join(w.EndedAllowed, ",") != "aa,bb" || len(w.EndedRefused) != 1 || w.EndedRefused[0].App != "immich" { + t.Fatalf("window = %+v, want the agreement (aa,bb) and the refusal of immich", w) + } + + // With no due record the body is the one an older hub always saw: no `ended` key at all. + if _, err := c.Open(context.Background(), 20, nil); err != nil { + t.Fatal(err) + } + if _, has := gotBody["ended"]; has { + t.Fatalf("a request with no claim carries `ended`: %v", gotBody) + } + + // A window that is NOT granted carries no agreement, whatever the answer holds. + answer = `{"granted":false,"reason":"not due","ended_allowed":["aa","bb"]}` + w2, err := c.Open(context.Background(), 20, claims) + if err != nil || w2.Granted || len(w2.EndedAllowed) != 0 { + t.Fatalf("not granted: %+v %v — an agreement without a window must not reach the box", w2, err) + } + + // The close names the removed ids for the hub's audit. + answer = `{"closed":true}` + if err := c.Close(context.Background(), backup.OffsiteWindowResult{ID: 9, CountBefore: 20, CountAfter: 18, Removed: 2, Outcome: "pruned", RemovedForEndedApps: []string{"aa", "bb"}}); err != nil { + t.Fatal(err) + } + var removed []string + if err := json.Unmarshal(gotBody["removed_for_ended_apps"], &removed); err != nil || strings.Join(removed, ",") != "aa,bb" || gotPath != "/api/v1/offsite/window-close/c1" { + t.Fatalf("close: path=%q removed_for_ended_apps=%s", gotPath, gotBody["removed_for_ended_apps"]) + } +} diff --git a/controller/internal/offsiteapply/seams.go b/controller/internal/offsiteapply/seams.go index a7448af4..0125d8cb 100644 --- a/controller/internal/offsiteapply/seams.go +++ b/controller/internal/offsiteapply/seams.go @@ -127,17 +127,25 @@ func (c HubRegistrar) MoveAside(ctx context.Context) (string, error) { type HubWindowClient struct{ Registrar HubRegistrar } -func (c HubWindowClient) Open(ctx context.Context, countBefore int) (backup.OffsiteWindow, error) { - raw, err := c.Registrar.post(ctx, "window-open", map[string]int{"count_before": countBefore}) +func (c HubWindowClient) Open(ctx context.Context, countBefore int, ended []backup.OffsiteEndedClaim) (backup.OffsiteWindow, error) { + // D72/D84: the due end records ride the same request; `ended` is left out when there is none, so the body an + // older hub sees is the one it always saw. + req := struct { + CountBefore int `json:"count_before"` + Ended []backup.OffsiteEndedClaim `json:"ended,omitempty"` + }{CountBefore: countBefore, Ended: ended} + raw, err := c.Registrar.post(ctx, "window-open", req) if err != nil { return backup.OffsiteWindow{}, err } var r struct { - Granted bool `json:"granted"` - WindowID int64 `json:"window_id"` - NewestAllowed string `json:"newest_allowed"` - MaxRemove int `json:"max_remove"` - Reason string `json:"reason"` + Granted bool `json:"granted"` + WindowID int64 `json:"window_id"` + NewestAllowed string `json:"newest_allowed"` + MaxRemove int `json:"max_remove"` + Reason string `json:"reason"` + EndedAllowed []string `json:"ended_allowed"` + EndedRefused []backup.OffsiteEndedRefusal `json:"ended_refused"` } if err := json.Unmarshal(raw, &r); err != nil { return backup.OffsiteWindow{}, fmt.Errorf("hub window-open: malformed response") @@ -149,6 +157,8 @@ func (c HubWindowClient) Open(ctx context.Context, countBefore int) (backup.Offs return backup.OffsiteWindow{}, fmt.Errorf("hub window-open: bad newest_allowed") } w.NewestAllowed = t + // Only a GRANTED window carries an agreement. + w.EndedAllowed, w.EndedRefused = r.EndedAllowed, r.EndedRefused } return w, nil } diff --git a/controller/internal/settings/offsite_ends.go b/controller/internal/settings/offsite_ends.go new file mode 100644 index 00000000..6725aea4 --- /dev/null +++ b/controller/internal/settings/offsite_ends.go @@ -0,0 +1,171 @@ +package settings + +import ( + "fmt" + "sort" + "strings" + "time" +) + +// D72 / D84 (`09` §3 decisions 292 and 309; `07-backup-architecture.md` §8.2f, part 1) — THE OFF-SITE END RECORD. +// +// When the household removes an app and presses „delete backup data”, the local backups go at once and the app's +// OFF-SITE copies end 30 days later. This record is the whole of that promise on the box: +// +// NO RECORD = NO DELETION, EVER. +// +// It is written only by StartOffsiteEnd (the household's press), deleted by CancelOffsiteEnd („keep it after all”, +// and every install of the same app), and moved to the short history by CompleteOffsiteEnd when a clean-up window +// has removed the copies. Nothing else starts, moves or shortens the clock. It lives in settings.json, so it +// survives a controller restart; a rebuilt guest has none, and then the copies stay (the safe direction). + +// OffsiteEndAfter is how long the off-site copies of a removed app stay (D72: the number of a restore copy, D62). +const OffsiteEndAfter = 30 * 24 * time.Hour + +// OffsiteEnd is one pending end record. +type OffsiteEnd struct { + App string `json:"app"` + RemovedAt string `json:"removed_at"` // RFC3339, the moment of the press + EndsAt string `json:"ends_at"` // RFC3339, RemovedAt + OffsiteEndAfter +} + +// Removed and Ends parse the record's dates; a date that cannot be read is the zero time, and a record with a zero +// date is never due (OffsiteEndDue). +func (e OffsiteEnd) Removed() time.Time { t, _ := time.Parse(time.RFC3339, e.RemovedAt); return t } +func (e OffsiteEnd) Ends() time.Time { t, _ := time.Parse(time.RFC3339, e.EndsAt); return t } + +// Due reports whether the record's day has come by this box's clock. The HUB decides by its own clock whether it +// agrees (`07` §8.2f part 4); this is only when the box starts to ask. The end is never earlier than +// RemovedAt + OffsiteEndAfter, whatever EndsAt says (a hand-edited or damaged record cannot shorten the wait). +func (e OffsiteEnd) Due(now time.Time) bool { + removed, ends := e.Removed(), e.Ends() + if removed.IsZero() || ends.IsZero() { + return false + } + if min := removed.Add(OffsiteEndAfter); ends.Before(min) { + ends = min + } + return !now.Before(ends) +} + +// OffsiteEndDone is one line of history: the off-site copies of App were removed by a clean-up window. +type OffsiteEndDone struct { + App string `json:"app"` + RemovedAt string `json:"removed_at"` + DeletedAt string `json:"deleted_at"` // RFC3339 + Snapshots int `json:"snapshots"` +} + +// maxOffsiteEndsDone bounds the history list. +const maxOffsiteEndsDone = 20 + +// ValidOffsiteEndApp: a name an end record may carry. The shares pseudo-app (`_shares`), the tier's marker tag and +// anything that is not a plain stack name are never eligible (`07` §8.2f, the edge-case table). Whether the name is +// a PROTECTED infrastructure stack is the caller's check — settings does not know the stack list. +func ValidOffsiteEndApp(app string) bool { + return app != "" && app != "." && app != ".." && !strings.HasPrefix(app, "_") && app != "felhom-offbox" && + !strings.ContainsAny(app, "/\\ \t\n") +} + +// StartOffsiteEnd writes the end record for app at now. created is false when a record already exists: a second +// press changes NOTHING — not the date, and the caller sends no second mail. +func (s *Settings) StartOffsiteEnd(app string, now time.Time) (rec OffsiteEnd, created bool, err error) { + if !ValidOffsiteEndApp(app) { + return OffsiteEnd{}, false, fmt.Errorf("offsite end: %q is not an app a household removes", app) + } + s.mu.Lock() + defer s.mu.Unlock() + if cur, ok := s.OffsiteEnds[app]; ok { + return cur, false, nil + } + now = now.UTC().Truncate(time.Second) + rec = OffsiteEnd{App: app, RemovedAt: now.Format(time.RFC3339), EndsAt: now.Add(OffsiteEndAfter).Format(time.RFC3339)} + if s.OffsiteEnds == nil { + s.OffsiteEnds = map[string]OffsiteEnd{} + } + s.OffsiteEnds[app] = rec + if err := s.save(); err != nil { + delete(s.OffsiteEnds, app) // a record that is not on disk is not a record + return OffsiteEnd{}, false, err + } + if s.log != nil { + s.log.Printf("[INFO] [settings] off-site end record WRITTEN for %s: removed %s, its off-site copies end %s (D72)", app, rec.RemovedAt, rec.EndsAt) + } + return rec, true, nil +} + +// CancelOffsiteEnd deletes app's end record („keep it after all”, or an install of the same app). Returns whether a +// record existed. why is for the log. +func (s *Settings) CancelOffsiteEnd(app, why string) (bool, error) { + s.mu.Lock() + defer s.mu.Unlock() + cur, ok := s.OffsiteEnds[app] + if !ok { + return false, nil + } + delete(s.OffsiteEnds, app) + if err := s.save(); err != nil { + s.OffsiteEnds[app] = cur + return false, err + } + if s.log != nil { + s.log.Printf("[INFO] [settings] off-site end record DELETED for %s (%s) — its off-site copies are kept (was: ends %s)", app, why, cur.EndsAt) + } + return true, nil +} + +// GetOffsiteEnd returns app's pending end record. +func (s *Settings) GetOffsiteEnd(app string) (OffsiteEnd, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + e, ok := s.OffsiteEnds[app] + return e, ok +} + +// GetOffsiteEnds returns the pending end records, by app name. +func (s *Settings) GetOffsiteEnds() []OffsiteEnd { + s.mu.RLock() + defer s.mu.RUnlock() + out := make([]OffsiteEnd, 0, len(s.OffsiteEnds)) + for _, e := range s.OffsiteEnds { + out = append(out, e) + } + sort.Slice(out, func(i, j int) bool { return out[i].App < out[j].App }) + return out +} + +// CompleteOffsiteEnd moves app's record to the history after a clean-up window removed its copies. The record is +// keyed on the app AND its removed_at (`07` §8.2f): a record that was deleted or written anew since the window read +// it is NOT completed. Returns whether it was. +func (s *Settings) CompleteOffsiteEnd(app, removedAt string, deletedAt time.Time, snapshots int) (bool, error) { + s.mu.Lock() + defer s.mu.Unlock() + cur, ok := s.OffsiteEnds[app] + if !ok || cur.RemovedAt != removedAt { + return false, nil + } + delete(s.OffsiteEnds, app) + prevDone := s.OffsiteEndsDone + if snapshots > 0 { + done := append([]OffsiteEndDone{{App: app, RemovedAt: removedAt, DeletedAt: deletedAt.UTC().Format(time.RFC3339), Snapshots: snapshots}}, s.OffsiteEndsDone...) + if len(done) > maxOffsiteEndsDone { + done = done[:maxOffsiteEndsDone] + } + s.OffsiteEndsDone = done + } + if err := s.save(); err != nil { + s.OffsiteEnds[app], s.OffsiteEndsDone = cur, prevDone + return false, err + } + if s.log != nil { + s.log.Printf("[INFO] [settings] off-site end record COMPLETED for %s: %d snapshot(s) removed", app, snapshots) + } + return true, nil +} + +// GetOffsiteEndsDone returns the history, newest first. +func (s *Settings) GetOffsiteEndsDone() []OffsiteEndDone { + s.mu.RLock() + defer s.mu.RUnlock() + return append([]OffsiteEndDone(nil), s.OffsiteEndsDone...) +} diff --git a/controller/internal/settings/offsite_ends_test.go b/controller/internal/settings/offsite_ends_test.go new file mode 100644 index 00000000..5cbd7aea --- /dev/null +++ b/controller/internal/settings/offsite_ends_test.go @@ -0,0 +1,155 @@ +package settings + +import ( + "io" + "log" + "path/filepath" + "testing" + "time" +) + +// D72 / D84 (`09` §3 decisions 292, 309; `07-backup-architecture.md` §8.2f part 1): the off-site end record. +// RED-PROOFS: felhom.eu/documentation/audits/build-release-4-2026-11-15/A/redproof-ctrl-record.txt. + +func endsSettings(t *testing.T) (*Settings, string) { + t.Helper() + p := filepath.Join(t.TempDir(), "settings.json") + s, err := Load(p, log.New(io.Discard, "", 0)) + if err != nil { + t.Fatal(err) + } + return s, p +} + +// The record: removed_at is the press, ends_at is 30 days later, and it is on DISK — a controller restart reads it +// back. (A record that lives only in memory would be lost at the next update, and the copies would stay for ever.) +func TestD72_EndRecord_IsWrittenAndSurvivesARestart(t *testing.T) { + s, path := endsSettings(t) + press := time.Date(2026, 11, 15, 10, 30, 0, 0, time.UTC) + rec, created, err := s.StartOffsiteEnd("vikunja", press) + if err != nil || !created { + t.Fatalf("created=%v err=%v", created, err) + } + if rec.RemovedAt != "2026-11-15T10:30:00Z" || rec.EndsAt != "2026-12-15T10:30:00Z" { + t.Fatalf("record = %+v, want removed 2026-11-15T10:30:00Z and ends exactly 30 days later", rec) + } + again, err := Load(path, log.New(io.Discard, "", 0)) + if err != nil { + t.Fatal(err) + } + got, ok := again.GetOffsiteEnd("vikunja") + if !ok || got != rec { + t.Fatalf("after a restart the record is %+v ok=%v, want %+v", got, ok, rec) + } + if n := len(again.GetOffsiteEnds()); n != 1 { + t.Fatalf("records after a restart: %d", n) + } +} + +// TWO PRESSES FOR THE SAME APP: one record, and the second press changes nothing — not earlier, not later. +func TestD72_EndRecord_ASecondPressChangesNothing(t *testing.T) { + s, _ := endsSettings(t) + press := time.Date(2026, 11, 15, 10, 30, 0, 0, time.UTC) + first, _, _ := s.StartOffsiteEnd("vikunja", press) + for _, at := range []time.Time{press.Add(-10 * 24 * time.Hour), press.Add(10 * 24 * time.Hour)} { + rec, created, err := s.StartOffsiteEnd("vikunja", at) + if err != nil || created || rec != first { + t.Fatalf("a second press at %s: created=%v rec=%+v err=%v — want the first record unchanged (%+v)", at, created, rec, err, first) + } + } + if got, _ := s.GetOffsiteEnd("vikunja"); got != first { + t.Fatalf("stored record moved: %+v, want %+v", got, first) + } + if n := len(s.GetOffsiteEnds()); n != 1 { + t.Fatalf("two presses left %d records", n) + } +} + +// „Keep it after all” (and an install) deletes the record; a second one finds none. +func TestD72_EndRecord_CancelDeletesIt(t *testing.T) { + s, path := endsSettings(t) + _, _, _ = s.StartOffsiteEnd("vikunja", time.Now()) + if ok, err := s.CancelOffsiteEnd("vikunja", "test"); err != nil || !ok { + t.Fatalf("cancel: ok=%v err=%v", ok, err) + } + if _, ok := s.GetOffsiteEnd("vikunja"); ok { + t.Fatal("the record is still there after the take-back") + } + again, _ := Load(path, log.New(io.Discard, "", 0)) + if _, ok := again.GetOffsiteEnd("vikunja"); ok { + t.Fatal("the take-back was not written to disk: a restart brings the record back") + } + if ok, _ := s.CancelOffsiteEnd("vikunja", "test"); ok { + t.Fatal("a second take-back reported a record") + } +} + +// The record is keyed on the app AND its removed_at: a window that read an OLD record must not complete a record +// written anew since (removed, taken back, removed again). And an end with no snapshot leaves no history line. +func TestD72_EndRecord_CompleteNeedsTheSameRecord(t *testing.T) { + s, _ := endsSettings(t) + old, _, _ := s.StartOffsiteEnd("vikunja", time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)) + _, _ = s.CancelOffsiteEnd("vikunja", "test") + cur, _, _ := s.StartOffsiteEnd("vikunja", time.Date(2026, 11, 1, 0, 0, 0, 0, time.UTC)) + if done, err := s.CompleteOffsiteEnd("vikunja", old.RemovedAt, time.Now(), 5); err != nil || done { + t.Fatalf("an old window completed a NEW record: done=%v err=%v", done, err) + } + if got, ok := s.GetOffsiteEnd("vikunja"); !ok || got != cur { + t.Fatalf("the new record is gone or changed: %+v ok=%v", got, ok) + } + at := time.Date(2026, 12, 6, 3, 0, 0, 0, time.UTC) + if done, err := s.CompleteOffsiteEnd("vikunja", cur.RemovedAt, at, 5); err != nil || !done { + t.Fatalf("complete: done=%v err=%v", done, err) + } + if _, ok := s.GetOffsiteEnd("vikunja"); ok { + t.Fatal("a completed record is still pending") + } + h := s.GetOffsiteEndsDone() + if len(h) != 1 || h[0].App != "vikunja" || h[0].Snapshots != 5 || h[0].DeletedAt != "2026-12-06T03:00:00Z" { + t.Fatalf("history = %+v", h) + } + // nothing was deleted → no history line + e, _, _ := s.StartOffsiteEnd("kimai", time.Now()) + if done, _ := s.CompleteOffsiteEnd("kimai", e.RemovedAt, at, 0); !done || len(s.GetOffsiteEndsDone()) != 1 { + t.Fatalf("an end with no snapshot must close with no history line: %+v", s.GetOffsiteEndsDone()) + } +} + +// When a record is DUE by this box's clock: on or after ends_at — and never before removed_at + 30 days, whatever +// the record's ends_at says. Dates that cannot be read are never due. +func TestD72_EndRecord_Due(t *testing.T) { + removed := time.Date(2026, 11, 15, 10, 0, 0, 0, time.UTC) + ok := OffsiteEnd{App: "a", RemovedAt: removed.Format(time.RFC3339), EndsAt: removed.Add(OffsiteEndAfter).Format(time.RFC3339)} + for _, c := range []struct { + name string + e OffsiteEnd + now time.Time + want bool + }{ + {"one second before the end", ok, removed.Add(OffsiteEndAfter - time.Second), false}, + {"at the end", ok, removed.Add(OffsiteEndAfter), true}, + {"a week after the end (the box was off)", ok, removed.Add(OffsiteEndAfter + 7*24*time.Hour), true}, + {"an ends_at EARLIER than 30 days after the removal does not shorten the wait", + OffsiteEnd{App: "a", RemovedAt: removed.Format(time.RFC3339), EndsAt: removed.Add(time.Hour).Format(time.RFC3339)}, removed.Add(29 * 24 * time.Hour), false}, + {"…and is due at the 30 days", OffsiteEnd{App: "a", RemovedAt: removed.Format(time.RFC3339), EndsAt: removed.Add(time.Hour).Format(time.RFC3339)}, removed.Add(OffsiteEndAfter), true}, + {"no removed_at", OffsiteEnd{App: "a", EndsAt: removed.Format(time.RFC3339)}, removed.Add(400 * 24 * time.Hour), false}, + {"an ends_at that cannot be read", OffsiteEnd{App: "a", RemovedAt: removed.Format(time.RFC3339), EndsAt: "soon"}, removed.Add(400 * 24 * time.Hour), false}, + } { + if got := c.e.Due(c.now); got != c.want { + t.Errorf("%s: Due = %v, want %v", c.name, got, c.want) + } + } +} + +// The shares pseudo-app, the tier's marker and anything that is not a plain stack name never get a record. +func TestD72_EndRecord_ReservedNamesGetNone(t *testing.T) { + s, _ := endsSettings(t) + for _, app := range []string{"_shares", "felhom-offbox", "", ".", "..", "a/b", "a b", "_anything"} { + if _, created, err := s.StartOffsiteEnd(app, time.Now()); err == nil || created { + t.Errorf("%q got an end record (created=%v err=%v)", app, created, err) + } + } + if n := len(s.GetOffsiteEnds()); n != 0 { + t.Fatalf("%d record(s) written for reserved names", n) + } +} diff --git a/controller/internal/settings/settings.go b/controller/internal/settings/settings.go index f0551536..19534a59 100644 --- a/controller/internal/settings/settings.go +++ b/controller/internal/settings/settings.go @@ -228,6 +228,11 @@ type Settings struct { // leaving the marker active would have Recover() start the broken app at the next controller // boot, hours later and quietly, which is the outcome the hold exists to prevent. RestoreHolds map[string]RestoreHold `json:"restore_holds,omitempty"` + + // OffsiteEnds (D72/D84, `07` §8.2f) — the pending off-site end records, by app: no record, no deletion. + // OffsiteEndsDone is the short history of ends a clean-up window has carried out. See offsite_ends.go. + OffsiteEnds map[string]OffsiteEnd `json:"offsite_ends,omitempty"` + OffsiteEndsDone []OffsiteEndDone `json:"offsite_ends_done,omitempty"` // UnhealthyStops (v0.269.0, decision 28) — when the box last stopped each app for a crash loop or an // out-of-memory storm (RFC3339). Survives the hold being lifted by Start, so a second stop within // 24 h is known to be a second one. diff --git a/controller/internal/stacks/d72_install_ends_record_test.go b/controller/internal/stacks/d72_install_ends_record_test.go new file mode 100644 index 00000000..21579d90 --- /dev/null +++ b/controller/internal/stacks/d72_install_ends_record_test.go @@ -0,0 +1,71 @@ +package stacks + +import ( + "io" + "log" + "path/filepath" + "testing" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// D72 (`09` §3 decision 292; `07-backup-architecture.md` §8.2f, the edge-case table): AN INSTALL OF THE SAME APP +// DELETES ITS OFF-SITE END RECORD — „the household uses the app; deleting its history a few days later would be a +// surprise”. This is the wrong case „an ended app that is installed again the day before the window”, on the box: +// the record is gone at the install, so the window finds nothing to claim. Through the real DeployStack (the +// composeExecFn seam and gateManager's docker stub: nothing reaches Docker). Another app's record is not touched. +// RED-PROOF: felhom.eu/documentation/audits/build-release-4-2026-11-15/A/redproof-ctrl-press.txt. +func TestD72_AnInstallDeletesTheEndRecord(t *testing.T) { + m := gateManager(t, gatedYml) + sett, err := settings.Load(filepath.Join(t.TempDir(), "settings.json"), log.New(io.Discard, "", 0)) + if err != nil { + t.Fatal(err) + } + m.SetMigrationDeps(sett, func() bool { return false }) + removed := time.Now().Add(-29 * 24 * time.Hour) // the window would run tomorrow + if _, _, err := sett.StartOffsiteEnd("gapp", removed); err != nil { + t.Fatal(err) + } + other, _, err := sett.StartOffsiteEnd("vikunja", removed) + if err != nil { + t.Fatal(err) + } + m.composeExecFn = func(string, map[string]string, ...string) (string, error) { return "", nil } + done := make(chan bool, 1) + m.SetDeployDoneHook(func(_ string, ok bool, _ string) { done <- ok }) + if _, err := m.DeployStack(DeployRequest{StackName: "gapp", Values: map[string]string{}}); err != nil { + t.Fatal(err) + } + select { + case <-done: + case <-time.After(20 * time.Second): + t.Fatal("the deploy never ended") + } + if rec, ok := sett.GetOffsiteEnd("gapp"); ok { + t.Fatalf("the app is installed again and its off-site end record is still there (%+v) — the next window would delete a living app's history", rec) + } + if got, ok := sett.GetOffsiteEnd("vikunja"); !ok || got != other { + t.Fatalf("another app's record changed at this install: %+v ok=%v", got, ok) + } +} + +// A deploy that is REFUSED changes nothing: the record stays. +func TestD72_ARefusedInstallKeepsTheEndRecord(t *testing.T) { + m := gateManager(t, gatedYml) + sett, err := settings.Load(filepath.Join(t.TempDir(), "settings.json"), log.New(io.Discard, "", 0)) + if err != nil { + t.Fatal(err) + } + m.SetMigrationDeps(sett, func() bool { return false }) + rec, _, _ := sett.StartOffsiteEnd("gapp", time.Now()) + m.mu.Lock() + m.stacks["gapp"].Deployed = true // „already deployed” is refused before anything is written + m.mu.Unlock() + if _, err := m.DeployStack(DeployRequest{StackName: "gapp", Values: map[string]string{}}); err == nil { + t.Fatal("fixture: the deploy was not refused") + } + if got, ok := sett.GetOffsiteEnd("gapp"); !ok || got != rec { + t.Fatalf("a refused deploy deleted or changed the record: %+v ok=%v", got, ok) + } +} diff --git a/controller/internal/stacks/delete.go b/controller/internal/stacks/delete.go index 39281e7a..7da2e97c 100644 --- a/controller/internal/stacks/delete.go +++ b/controller/internal/stacks/delete.go @@ -205,6 +205,10 @@ type BackupDataResponse struct { Stack string `json:"stack"` BackupPaths []HDDPath `json:"backup_paths"` // reuses HDDPath (path, size, exists) HasBackups bool `json:"has_backups"` + // OffsiteEndDate (D72, `07` §8.2f) is the day this app's OFF-SITE copies would be deleted if the household + // pressed „delete backup data” now — written in the household's date form. Empty when the box has no + // hub-provisioned off-site tier: then the dialog says nothing about off-site copies. Set by the API handler. + OffsiteEndDate string `json:"offsite_end_date,omitempty"` } // HDDDataResponse holds information about HDD data associated with a stack. diff --git a/controller/internal/stacks/deploy.go b/controller/internal/stacks/deploy.go index 27fdfdba..c0b74408 100644 --- a/controller/internal/stacks/deploy.go +++ b/controller/internal/stacks/deploy.go @@ -565,6 +565,14 @@ func (m *Manager) DeployStack(req DeployRequest) (string, error) { clearDeploying() return "", fmt.Errorf("saving app config: %w", err) } + // D72 (`07` §8.2f): an install of the same app ENDS its pending off-site end record — the household uses the + // app again, and its old copies are restore sources again. Here, at the first write of the install: a deploy + // that was refused above changed nothing. Pinned by TestD72_AnInstallDeletesTheEndRecord. + if m.settings != nil { + if _, cerr := m.settings.CancelOffsiteEnd(req.StackName, "the app is installed again"); cerr != nil { + m.logger.Printf("[WARN] [stacks] deploy %s: its off-site end record could not be deleted (%v) — the clean-up window skips an installed app anyway", req.StackName, cerr) + } + } // Debug: log final env var keys (not values) envKeys := make([]string, 0, len(env)) diff --git a/controller/internal/web/handlers.go b/controller/internal/web/handlers.go index 2ba140aa..da159917 100644 --- a/controller/internal/web/handlers.go +++ b/controller/internal/web/handlers.go @@ -1360,6 +1360,7 @@ func (s *Server) backupsRestoreHandler(w http.ResponseWriter, r *http.Request) { // box has neither, and used to be shown nothing at all while its snapshots sat in the repository. if s.backupMgr != nil { rows, state := s.offsiteRestoreRows(r.Context()) + rows = s.offsiteEndData(r, data, rows) // D72: the end records — „marked for deletion”, the take-back, the history data["OffsiteRestoreRows"] = rows data["OffsiteStoreState"] = string(state) // R-252: a rebuilt box's drives survive but their REGISTRATION does not, and every restore diff --git a/controller/internal/web/i18n_cases_b_test.go b/controller/internal/web/i18n_cases_b_test.go index 5c9e6a20..53a518ed 100644 --- a/controller/internal/web/i18n_cases_b_test.go +++ b/controller/internal/web/i18n_cases_b_test.go @@ -244,6 +244,20 @@ func i18nCasesB() []i18nCase { d["KeptCopies"] = []keptCopyRow{{ID: "0123456789abcdef", Source: "csaladi_fotok", Line: "hdd_3", SizeHuman: "1.2 GB", Since: "2026-10-21T12:00:00Z"}} return d }}, + // D72 (2026-11-15, `07` §8.2f): the off-site list with a marked app („keep it after all”), a removed app whose copies + // were kept („delete in 30 days”) and one carried-out end (a NEW case, so no existing fixture changes). + {"backups_restore_offsite_end", "backups_restore", func() map[string]interface{} { + d := i18nLayoutData("backups-restore", "Visszaállítás a mentésből") + d["Backup"] = i18nBackupStatus(false, false) + d["OffboxConfigured"] = true + d["OffsiteStoreState"] = "known" + d["OffsiteRestoreRows"] = []OffsiteRestoreRow{ + {App: "vikunja", DisplayName: "vikunja", InStore: true, EndDate: "2026. 12. 15."}, + {App: "kimai", DisplayName: "kimai", InStore: true, EndOffered: true}, + } + d["OffsiteEndsDone"] = []offsiteEndDoneRow{{App: "docmost", Line: "2026. 11. 08."}} + return d + }}, // D47 (2026-10-25): a set-aside unit of an earlier install — the row with „Visszaállítás" (a NEW case, so no existing // fixture changes). {"backups_restore_kept_unit", "backups_restore", func() map[string]interface{} { diff --git a/controller/internal/web/offsite_ends.go b/controller/internal/web/offsite_ends.go new file mode 100644 index 00000000..98a685eb --- /dev/null +++ b/controller/internal/web/offsite_ends.go @@ -0,0 +1,128 @@ +package web + +import ( + "errors" + "net/http" + "sort" + "strings" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/backup" + "gitea.dooplex.hu/admin/felhom-controller/internal/i18n" + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// D72 / D84 (`09` §3 decisions 292 and 309; `07-backup-architecture.md` §8.2f, part 2) — THE HOUSEHOLD SEES THE DATE +// AND CAN TAKE IT BACK. The off-site list on the restore page is keyed on the store, not on what is installed +// (R-237), so a removed app stays on it while its copies exist: +// +// - an app with an end record reads „marked for deletion: ” and has ONE button, „keep it after all”; +// - a removed app with copies and NO record (its backups were kept, or it was removed before this existed) reads +// „the off-site copies of this app were kept” and has „delete in 30 days” — the household starts the clock itself; +// - an end a clean-up window has carried out is one line of history. +// +// Nothing here deletes anything: both buttons only write or delete the end record (backup.StartOffsiteEnd, +// backup.KeepOffsiteCopies). The deletion is the weekly window's, after the hub agreed. + +// offsiteEndDoneRow is one history line: „: the off-site copies were deleted on ”. +type offsiteEndDoneRow struct { + App string + Line string +} + +// applyOffsiteEnds puts the end records on the rows. offered = this box has the hub-provisioned tier. +// +// A record whose app has no row (the store could not be read, or its listing is cut) still gets a row: the +// take-back button must not depend on a network read succeeding. +func applyOffsiteEnds(rows []OffsiteRestoreRow, ends []settings.OffsiteEnd, offered bool, date func(e settings.OffsiteEnd) string) []OffsiteRestoreRow { + byApp := make(map[string]settings.OffsiteEnd, len(ends)) + for _, e := range ends { + byApp[e.App] = e + } + seen := map[string]bool{} + for i := range rows { + r := &rows[i] + seen[r.App] = true + if e, ok := byApp[r.App]; ok && !r.Installed { + r.EndDate = date(e) + continue + } + // „delete in 30 days”: only for a removed app whose copies are really there, on a box where a press can + // start the clock. Never for an installed app, never for a row we could not verify. + r.EndOffered = offered && r.InStore && !r.Installed && !r.StoreUnknown + } + var extra []OffsiteRestoreRow + for _, e := range ends { + if !seen[e.App] { + extra = append(extra, OffsiteRestoreRow{App: e.App, DisplayName: e.App, EndDate: date(e)}) + } + } + sort.Slice(extra, func(i, j int) bool { return extra[i].App < extra[j].App }) + return append(rows, extra...) +} + +// offsiteEndData adds the end records to the restore page's data. +func (s *Server) offsiteEndData(r *http.Request, data map[string]interface{}, rows []OffsiteRestoreRow) []OffsiteRestoreRow { + if s.backupMgr == nil { + return rows + } + lang := s.langFor(r) + rows = applyOffsiteEnds(rows, s.backupMgr.PendingOffsiteEnds(), s.backupMgr.OffsiteEndOffered(), func(e settings.OffsiteEnd) string { + return s.msgLang(lang, "offsite_end.marked", i18n.FormatDate(lang, e.Ends().Local())) + }) + var done []offsiteEndDoneRow + for _, d := range s.backupMgr.OffsiteEndsHistory() { + at, perr := time.Parse(time.RFC3339, d.DeletedAt) + if perr != nil { + continue + } + done = append(done, offsiteEndDoneRow{App: d.App, Line: s.msgLang(lang, "offsite_end.done", i18n.FormatDate(lang, at.Local()))}) + } + data["OffsiteEndsDone"] = done + return rows +} + +// offsiteEndKeepHandler is „keep it after all”: POST /backup/offsite-end/keep (app). It deletes the end record. +// After the window has run there is no record, and the page says so. +func (s *Server) offsiteEndKeepHandler(w http.ResponseWriter, r *http.Request) { + if s.backupMgr == nil { + offboxRedirectTo(w, r, "/backups/restore", "flash.offbox.mgr_unreachable", true) + return + } + _ = r.ParseForm() + app := strings.TrimSpace(r.FormValue("app")) + kept, err := s.backupMgr.KeepOffsiteCopies(app) + switch { + case err != nil: + s.logger.Printf("[WARN] [web] keep the off-site copies of %s: the end record could not be deleted: %v", app, err) + offboxRedirectTo(w, r, "/backups/restore", "flash.offsite_end.failed", true) + case !kept: + offboxRedirectTo(w, r, "/backups/restore", "flash.offsite_end.nothing_to_keep", true) + default: + s.logger.Printf("[INFO] [web] the household keeps the off-site copies of %s after all (from %s)", app, r.RemoteAddr) + offboxRedirectTo(w, r, "/backups/restore", "flash.offsite_end.kept", false) + } +} + +// offsiteEndStartHandler is „delete in 30 days”: POST /backup/offsite-end/start (app). The same press as „delete +// backup data” at a removal — the same record, the same one mail — for an app that was removed without one. +func (s *Server) offsiteEndStartHandler(w http.ResponseWriter, r *http.Request) { + if s.backupMgr == nil { + offboxRedirectTo(w, r, "/backups/restore", "flash.offbox.mgr_unreachable", true) + return + } + _ = r.ParseForm() + app := strings.TrimSpace(r.FormValue("app")) + rec, _, err := s.backupMgr.StartOffsiteEnd(app) + switch { + case errors.Is(err, backup.ErrOffsiteEndNotOffered), errors.Is(err, backup.ErrOffsiteEndLiving): + s.logger.Printf("[WARN] [web] delete-in-30-days for %q refused: %v", app, err) + offboxRedirectTo(w, r, "/backups/restore", "flash.offsite_end.not_possible", true) + case err != nil: + s.logger.Printf("[WARN] [web] delete-in-30-days for %q: the end record was not written: %v", app, err) + offboxRedirectTo(w, r, "/backups/restore", "flash.offsite_end.failed", true) + default: + // Also on a second press: the record stands as it was, and the page says its date again. + offboxRedirectTo(w, r, "/backups/restore", "offsite_end.marked", false, i18n.FormatDate(s.langFor(r), rec.Ends().Local())) + } +} diff --git a/controller/internal/web/offsite_ends_test.go b/controller/internal/web/offsite_ends_test.go new file mode 100644 index 00000000..ad72c8dd --- /dev/null +++ b/controller/internal/web/offsite_ends_test.go @@ -0,0 +1,289 @@ +package web + +import ( + "io" + "log" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/backup" + "gitea.dooplex.hu/admin/felhom-controller/internal/config" + "gitea.dooplex.hu/admin/felhom-controller/internal/i18n" + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// D72 / D84 (`09` §3 decisions 292, 309; `07-backup-architecture.md` §8.2f part 2): the household sees the date and +// can take it back. RED-PROOFS: felhom.eu/documentation/audits/build-release-4-2026-11-15/A/redproof-ctrl-page.txt. + +func endsRestoreData(rows []OffsiteRestoreRow) map[string]interface{} { + d := i18nLayoutData("backups-restore", "Visszaállítás a mentésből") + d["Backup"] = i18nBackupStatus(false, false) + d["OffboxConfigured"] = true + d["OffsiteStoreState"] = "known" + d["OffsiteRestoreRows"] = rows + return d +} + +// The page, per branch of its gates (the seam-wiring rule): a marked app shows its line and ONLY „keep it after all”; +// a removed app with kept copies shows the kept line and „delete in 30 days”; an installed app shows neither; a +// carried-out end is one history line. In both languages. +func TestD72_RestorePageShowsTheEndPerBranch(t *testing.T) { + rows := []OffsiteRestoreRow{ + {App: "vikunja", DisplayName: "vikunja", InStore: true, EndDate: "Törlésre kijelölve. Törlés napja: 2026. 12. 15."}, + {App: "kimai", DisplayName: "kimai", InStore: true, EndOffered: true}, + {App: "immich", DisplayName: "Immich", InStore: true, Installed: true}, + } + d := endsRestoreData(rows) + d["OffsiteEndsDone"] = []offsiteEndDoneRow{{App: "docmost", Line: "A távoli másolatok törölve: 2026. 11. 08."}} + html := renderBackupPage(t, "backups_restore", d) + for _, want := range []string{ + `data-offsite-end="marked">Törlésre kijelölve. Törlés napja: 2026. 12. 15.`, + `action="/backup/offsite-end/keep"`, `name="app" value="vikunja"`, "Mégis megtartom", + `data-offsite-end="kept">Ennek az alkalmazásnak a távoli másolatai megmaradtak.`, + `action="/backup/offsite-end/start"`, `name="app" value="kimai"`, "Törlés 30 nap múlva", + `data-offsite-end="done">docmost: A távoli másolatok törölve: 2026. 11. 08.

`, + } { + if !strings.Contains(html, want) { + t.Errorf("the restore page lacks %q", want) + } + } + if n := strings.Count(html, `action="/backup/offsite-end/keep"`); n != 1 { + t.Errorf("%d „keep it after all” forms, want one (the marked app's)", n) + } + if n := strings.Count(html, `action="/backup/offsite-end/start"`); n != 1 { + t.Errorf("%d „delete in 30 days” forms, want one (the removed app with kept copies)", n) + } + for _, bad := range []string{`name="app" value="immich"`} { + if strings.Contains(html, bad) { + t.Errorf("an INSTALLED app is offered an end: %q", bad) + } + } + // no record, nothing removed, no history: the page carries none of it + plain := renderBackupPage(t, "backups_restore", endsRestoreData([]OffsiteRestoreRow{{App: "immich", DisplayName: "Immich", InStore: true, Installed: true}})) + if strings.Contains(plain, `data-offsite-end="marked"`) || strings.Contains(plain, `data-offsite-end="kept"`) || + strings.Contains(plain, `data-offsite-end="done"`) || strings.Contains(plain, "/backup/offsite-end/") { + t.Error("a page with no end record and no removed app shows the end's controls") + } + // English + en := renderBackupPageLang(t, "en", "backups_restore", endsRestoreData([]OffsiteRestoreRow{ + {App: "vikunja", DisplayName: "vikunja", InStore: true, EndDate: "Marked for deletion. Deleted on: 2026-12-15."}, + {App: "kimai", DisplayName: "kimai", InStore: true, EndOffered: true}})) + for _, want := range []string{"Keep it after all", "The off-site copies of this app were kept.", "Delete in 30 days", "Marked for deletion. Deleted on: 2026-12-15."} { + if !strings.Contains(en, want) { + t.Errorf("the English restore page lacks %q", want) + } + } + for _, bad := range []string{"Mégis megtartom", "Törlés 30 nap múlva", "megmaradtak"} { + if strings.Contains(en, bad) { + t.Errorf("Hungarian on the English page: %q", bad) + } + } +} + +// Which row gets which: the pure rule. +func TestD72_ApplyOffsiteEnds(t *testing.T) { + date := func(e settings.OffsiteEnd) string { return "MARKED " + e.App } + ends := []settings.OffsiteEnd{{App: "vikunja"}, {App: "immich"}, {App: "ghost"}} + in := func() []OffsiteRestoreRow { + return []OffsiteRestoreRow{ + {App: "vikunja", InStore: true}, // removed, record → marked + {App: "immich", InStore: true, Installed: true}, // INSTALLED with a stale record → nothing + {App: "kimai", InStore: true}, // removed, no record → the offer + {App: "docmost", InStore: true, Installed: true}, // installed → nothing + {App: "unread", InStore: false, StoreUnknown: true}, // the store could not be read → no offer + {App: "empty", InStore: false}, // nothing in the store → no offer + } + } + got := map[string]OffsiteRestoreRow{} + for _, r := range applyOffsiteEnds(in(), ends, true, date) { + got[r.App] = r + } + if got["vikunja"].EndDate != "MARKED vikunja" || got["vikunja"].EndOffered { + t.Errorf("vikunja: %+v", got["vikunja"]) + } + if got["immich"].EndDate != "" || got["immich"].EndOffered { + t.Errorf("an installed app shows an end: %+v", got["immich"]) + } + if !got["kimai"].EndOffered || got["kimai"].EndDate != "" { + t.Errorf("kimai (removed, copies kept): %+v", got["kimai"]) + } + for _, app := range []string{"docmost", "unread", "empty"} { + if got[app].EndOffered || got[app].EndDate != "" { + t.Errorf("%s must show nothing: %+v", app, got[app]) + } + } + // a record whose app has no row at all still shows — the take-back must not depend on reading the store + if g, ok := got["ghost"]; !ok || g.EndDate != "MARKED ghost" { + t.Errorf("a record with no row is not shown: %+v ok=%v", g, ok) + } + // a box where a press cannot start the clock offers nothing + for _, r := range applyOffsiteEnds(in(), nil, false, date) { + if r.EndOffered { + t.Errorf("%s: offered on a box without the hub's tier", r.App) + } + } +} + +func endsServer(t *testing.T, pinned bool, living ...string) (*Server, *settings.Settings, *[]settings.OffsiteEnd) { + t.Helper() + tmp := t.TempDir() + lg := log.New(io.Discard, "", 0) + sett, err := settings.Load(filepath.Join(tmp, "settings.json"), lg) + if err != nil { + t.Fatal(err) + } + tr := "" + if pinned { + tr = settings.TransportRclonePinned + } + if err := sett.SetOffboxTarget(&settings.OffboxTarget{Enabled: true, Host: "h", User: "u", RepoPath: "/r", Transport: tr}); err != nil { + t.Fatal(err) + } + cfg := &config.Config{} + cfg.Paths.DataDir = tmp + m := backup.NewManager(cfg, sett, lg) + live := map[string]bool{} + for _, l := range living { + live[l] = true + } + m.SetOffsiteEndLiving(func(app string) bool { return live[app] }) + var mails []settings.OffsiteEnd + m.SetOffsiteEndNotify(func(rec settings.OffsiteEnd) { mails = append(mails, rec) }) + s := testServer(t) + s.cfg, s.backupMgr, s.logger, s.settings = cfg, m, lg, sett + return s, sett, &mails +} + +// MUST-HOLD 4 — THE HOUSEHOLD CAN TAKE IT BACK, through the page's own endpoints: „delete in 30 days” writes the +// record (one mail), a second press changes nothing (no second mail), „keep it after all” deletes it, and a second +// „keep” says there is nothing marked. +func TestD72_StartAndKeepHandlers(t *testing.T) { + s, sett, mails := endsServer(t, true, "immich") + post := func(path, form string) string { + rr := httptest.NewRecorder() + req := httptest.NewRequest("POST", path, strings.NewReader(form)) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + if strings.HasSuffix(path, "/keep") { + s.offsiteEndKeepHandler(rr, req) + } else { + s.offsiteEndStartHandler(rr, req) + } + return rr.Header().Get("Location") + } + loc := post("/backup/offsite-end/start", "app=vikunja") + rec, ok := sett.GetOffsiteEnd("vikunja") + if !ok || !strings.Contains(loc, "offsite_end.marked") || strings.Contains(loc, "flash_error") { + t.Fatalf("start → %q, record ok=%v", loc, ok) + } + wantDate := i18n.FormatDate("hu", rec.Ends().Local()) + if !strings.Contains(loc, strings.ReplaceAll(wantDate, " ", "+")) { + t.Fatalf("the flash does not carry the record's day %q: %q", wantDate, loc) + } + if d := rec.Ends().Sub(time.Now()); d < 29*24*time.Hour || d > 31*24*time.Hour { + t.Fatalf("the record ends in %s, want 30 days", d) + } + _ = post("/backup/offsite-end/start", "app=vikunja") + if again, _ := sett.GetOffsiteEnd("vikunja"); again != rec || len(*mails) != 1 { + t.Fatalf("a second press: record %+v (was %+v), %d mail(s) — want unchanged and ONE mail", again, rec, len(*mails)) + } + // an installed app, the shares and a path are refused and write nothing + for _, app := range []string{"immich", "_shares", "../x", ""} { + if loc := post("/backup/offsite-end/start", "app="+app); !strings.Contains(loc, "flash_error=flash.offsite_end.not_possible") { + t.Errorf("start for %q → %q, want the refusal", app, loc) + } + } + if n := len(sett.GetOffsiteEnds()); n != 1 || len(*mails) != 1 { + t.Fatalf("records=%d mails=%d after the refused presses", n, len(*mails)) + } + if loc := post("/backup/offsite-end/keep", "app=vikunja"); !strings.Contains(loc, "flash=flash.offsite_end.kept") { + t.Fatalf("keep → %q", loc) + } + if _, ok := sett.GetOffsiteEnd("vikunja"); ok { + t.Fatal("„keep it after all” left the record — the window would still delete the copies") + } + if loc := post("/backup/offsite-end/keep", "app=vikunja"); !strings.Contains(loc, "flash_error=flash.offsite_end.nothing_to_keep") { + t.Fatalf("a second keep → %q, want the page to say nothing is marked", loc) + } + // a box without the hub's tier: the press is refused, nothing is written, nobody is mailed + s2, sett2, mails2 := endsServer(t, false) + rr := httptest.NewRecorder() + req := httptest.NewRequest("POST", "/backup/offsite-end/start", strings.NewReader("app=vikunja")) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + s2.offsiteEndStartHandler(rr, req) + if !strings.Contains(rr.Header().Get("Location"), "flash.offsite_end.not_possible") || len(sett2.GetOffsiteEnds()) != 0 || len(*mails2) != 0 { + t.Fatalf("no hub tier: %q records=%d mails=%d", rr.Header().Get("Location"), len(sett2.GetOffsiteEnds()), len(*mails2)) + } +} + +// The page data: a record becomes the marked line with the record's own day in the page's language; a carried-out +// end becomes a history line. +func TestD72_OffsiteEndDataFromTheRecords(t *testing.T) { + s, sett, _ := endsServer(t, true) + rec, _, _ := sett.StartOffsiteEnd("vikunja", time.Date(2026, 11, 15, 10, 0, 0, 0, time.UTC)) + old, _, _ := sett.StartOffsiteEnd("docmost", time.Date(2026, 10, 1, 10, 0, 0, 0, time.UTC)) + if _, err := sett.CompleteOffsiteEnd("docmost", old.RemovedAt, time.Date(2026, 11, 8, 3, 0, 0, 0, time.UTC), 4); err != nil { + t.Fatal(err) + } + for lang, want := range map[string][2]string{ + "hu": {"Törlésre kijelölve. Törlés napja: " + i18n.FormatDate("hu", rec.Ends().Local()), "A távoli másolatok törölve: " + i18n.FormatDate("hu", time.Date(2026, 11, 8, 3, 0, 0, 0, time.UTC).Local())}, + "en": {"Marked for deletion. Deleted on: " + i18n.FormatDate("en", rec.Ends().Local()) + ".", "The off-site copies were deleted on " + i18n.FormatDate("en", time.Date(2026, 11, 8, 3, 0, 0, 0, time.UTC).Local()) + "."}, + } { + data := map[string]interface{}{} + req := httptest.NewRequest("GET", "/backups/restore?lang="+lang, nil) + rows := s.offsiteEndData(req, data, []OffsiteRestoreRow{{App: "vikunja", InStore: true}, {App: "kimai", InStore: true}}) + if rows[0].EndDate != want[0] { + t.Errorf("%s: marked line = %q, want %q", lang, rows[0].EndDate, want[0]) + } + if !rows[1].EndOffered { + t.Errorf("%s: a removed app with copies and no record is not offered „delete in 30 days”", lang) + } + done, _ := data["OffsiteEndsDone"].([]offsiteEndDoneRow) + if len(done) != 1 || done[0].App != "docmost" || done[0].Line != want[1] { + t.Errorf("%s: history = %+v, want docmost: %q", lang, done, want[1]) + } + } +} + +// The routes are REACHABLE: both POSTs are in the server's switch (a handler nobody routes to is not a feature). +func TestD72_TheEndRoutesAreWired(t *testing.T) { + raw, err := os.ReadFile("server.go") + if err != nil { + t.Fatal(err) + } + for _, want := range []string{ + "case path == \"/backup/offsite-end/keep\" && r.Method == http.MethodPost:\n\t\ts.offsiteEndKeepHandler(w, r)", + "case path == \"/backup/offsite-end/start\" && r.Method == http.MethodPost:\n\t\ts.offsiteEndStartHandler(w, r)", + } { + if !strings.Contains(string(raw), want) { + t.Errorf("server.go lacks the route %q", want) + } + } + h, err := os.ReadFile("handlers.go") + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(h), "rows = s.offsiteEndData(r, data, rows)") { + t.Error("the restore page does not put the end records on its rows") + } +} + +// THE DATE IS ON THE REMOVE DIALOG BEFORE THE PRESS (D72): the dialog's script shows the sentence, with the day the +// API gave, only when the API gave one — in both languages. +func TestD72_TheRemoveDialogNamesTheDate(t *testing.T) { + for lang, want := range map[string]string{ + "hu": "A helyi mentések most törlődnek. A távoli másolatok még 30 napig megmaradnak. Törlésük napja: {date} Addig a Mentések oldalon visszavonhatod.", + "en": "The local backups are deleted now. The off-site copies stay for 30 more days. They are deleted on: {date}. Until then you can take this back on the Backups page.", + } { + html := renderBackupPageLang(t, lang, "backups_restore", endsRestoreData(nil)) + if !strings.Contains(html, want) { + t.Errorf("[%s] the remove dialog lacks the sentence %q", lang, want) + } + if !strings.Contains(html, "backupResp.data.offsite_end_date ?") || !strings.Contains(html, ".replace('{date}', backupResp.data.offsite_end_date)") { + t.Errorf("[%s] the dialog does not show the sentence only with the API's date", lang) + } + } +} diff --git a/controller/internal/web/offsite_restore_list.go b/controller/internal/web/offsite_restore_list.go index f9c4be0f..f9dedcad 100644 --- a/controller/internal/web/offsite_restore_list.go +++ b/controller/internal/web/offsite_restore_list.go @@ -77,6 +77,12 @@ type OffsiteRestoreRow struct { StoreUnknown bool LatestAt time.Time SizeBytes int64 + // EndDate (D72, `07` §8.2f) — non-empty when the app has an off-site end record: the finished line „marked for + // deletion: ” in the page's language. The row then offers „keep it after all”. + EndDate string + // EndOffered — a removed app whose copies are in the store and that has NO end record, on a box where a press + // can start the 30 days: the row says the copies were kept and offers „delete in 30 days”. + EndOffered bool } // Restorable reports whether a restore may be OFFERED for this row: the store holds it, or we could diff --git a/controller/internal/web/server.go b/controller/internal/web/server.go index 91e022f0..7738aee0 100644 --- a/controller/internal/web/server.go +++ b/controller/internal/web/server.go @@ -846,6 +846,12 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { // DeleteOffsiteRestoreCopy for the prefix guard. case path == "/backup/offbox/verify-copy/delete" && r.Method == http.MethodPost: s.offboxVerifyCopyDeleteHandler(w, r) + // D72 / D84 (`07` §8.2f): the off-site copies of a removed app — „keep it after all” deletes the end record, + // „delete in 30 days” writes one. Neither deletes anything; the weekly window does, after the hub agreed. + case path == "/backup/offsite-end/keep" && r.Method == http.MethodPost: + s.offsiteEndKeepHandler(w, r) + case path == "/backup/offsite-end/start" && r.Method == http.MethodPost: + s.offsiteEndStartHandler(w, r) // D25 (2026-10-21): „Eltávolítás" of ONE kept second-drive copy whose source is gone. case path == "/backup/kept/remove" && r.Method == http.MethodPost: s.keptCopyRemoveHandler(w, r) diff --git a/controller/internal/web/templates/backups_restore.html b/controller/internal/web/templates/backups_restore.html index 1a514848..562d3337 100644 --- a/controller/internal/web/templates/backups_restore.html +++ b/controller/internal/web/templates/backups_restore.html @@ -112,7 +112,17 @@ {{if .OffsiteRestoreRows}}
{{range .OffsiteRestoreRows}} - {{template "app_list_row" dict "Slug" .Slug "Name" .DisplayName}} + {{template "app_list_row" dict "Slug" .Slug "Name" .DisplayName}}{{if .EndDate}} + {{.EndDate}} +
{{$.CSRFField}} + + +
{{else if .EndOffered}} + {{T "offsite_end.kept_before"}} +
{{$.CSRFField}} + + +
{{end}} {{if .Restorable}}