R-86: restore-test follows the backup, not the clock (v0.121.0)
gates / gates (push) Failing after 7s

The ticker survives as the EVALUATION interval only. A tier is DUE when its
newest archive that has settled for `settle` (default 24h) has not been proven:
daily tier -> proved daily on yesterday's archive, weekly tier -> weekly on its
own, newborn -> UNKNOWN.

The trap avoided: the literal reading ("newest archive is >= 24h old") is NEVER
true on a daily tier, so it silently switches restore-testing off where it
matters most. Red-proved at 0 runs over 5 simulated days.

- state records WHICH archive was proven; legacy files keep their time and yield
  no proven archive (each tier due once after the upgrade, deliberately)
- two knobs replace one: restore_test_eval_interval_seconds (6h, measured) and
  restore_test_settle_seconds (24h). The old cadence key keeps its DISABLE
  meaning verbatim and now seeds the settle lag, with a start-up WARN.
- due-check runs BEFORE the heavy-op gate (a frequent poll must not make a
  starting backup record a failure, F-A1)
- candidate picker skips implausible archives (a phantom would be due forever)
- new read-only --selftest=restore-test-due prints the verdict + its cost
This commit is contained in:
2026-08-03 14:54:57 +02:00
parent 1b14cfd0b4
commit 4618169036
13 changed files with 1273 additions and 123 deletions
+24 -3
View File
@@ -135,10 +135,11 @@ func TestBackup_VzdumpFailureReturnsFailedRecord(t *testing.T) {
}
func TestPickRestoreCandidate_NewestOrEmpty(t *testing.T) {
const big = 4 << 30 // a plausible whole-guest archive
api := &fakeBackupAPI{content: []proxmox.StorageContent{
{VolID: "a", Content: "backup", CTime: 10},
{VolID: "b", Content: "backup", CTime: 99},
{VolID: "iso", Content: "iso", CTime: 999}, // not a backup → ignored
{VolID: "a", Content: "backup", CTime: 10, Size: big},
{VolID: "b", Content: "backup", CTime: 99, Size: big},
{VolID: "iso", Content: "iso", CTime: 999, Size: big}, // not a backup → ignored
}}
r := NewBackupRunner(api, "local", "", "", "", quiet())
vol, err := r.PickRestoreCandidate(context.Background())
@@ -152,6 +153,26 @@ func TestPickRestoreCandidate_NewestOrEmpty(t *testing.T) {
}
}
// R-86: the NEWEST entry is not a candidate if it cannot be a complete archive. An incomplete
// artefact (F-CRIT-2's 1-byte phantom, which server-side prune does not collect) would otherwise be
// picked forever, fail its restore forever, never earn proof, and so leave the tier due at every
// evaluation — turning the evaluation interval into the retry rate for a multi-GB restore.
//
// COMPANION RED-PROOF (observed): drop the `archivePlausiblyComplete` guard from
// PickSettledRestoreCandidateOn and this fails with
// `pick = "phantom" want the newest COMPLETE archive 'real'`.
func TestPickRestoreCandidate_SkipsImplausibleArchives(t *testing.T) {
api := &fakeBackupAPI{content: []proxmox.StorageContent{
{VolID: "real", Content: "backup", CTime: 10, Size: 4 << 30},
{VolID: "phantom", Content: "backup", CTime: 99, Size: 1}, // newest, and impossible
}}
r := NewBackupRunner(api, "local", "", "", "", quiet())
vol, err := r.PickRestoreCandidate(context.Background())
if err != nil || vol != "real" {
t.Fatalf("pick = %q,%v want the newest COMPLETE archive 'real'", vol, err)
}
}
// --- scheduler ---
type fakeRTRunner struct {