v0.98.0 — R-82 Slice A fix: per-tier vzdump wait bound (the 30-minute false failure)
Found by live validation on demo-felhom, not by review. The first real PBS-targeted backup ran past the runner's hard-coded 30-minute WaitTask bound. The agent stopped waiting and recorded success=false WHILE THE VZDUMP KEPT RUNNING (still running 72 min later, 2.4 GB uploaded). Consequences: the tier stays permanently due, the next attempt collides with the guest lock the live vzdump holds, and the hub sees a DR tier that never succeeds — R-82's 'applied and empty' fault re-created by a timeout. Measured: ~33 MB/min over wg to Hetzner, so a first FULL ~10 GB snapshot projects to ~5h. - BackupTargetConfig.WaitTimeoutSeconds: per-tier bound. Primary 30m UNCHANGED (a local vzdump hanging 30m IS a real fault); additional tier 6h, sized from the measurement. - backup.NewBackupRunnerWithWait: per-instance (per-tier) bound. NewBackupRunner keeps its signature, so restore-test/selftest are untouched. - localapi.BackupTier.WaitTimeout: the fire-and-forget context is sized from the tier, not a fixed 2h. BOTH bounds had to move — a 6h runner bound under a 2h outer context reproduces the same false failure four hours later. Same direction as restore_test_pbs_restore_timeout_seconds: when in doubt wait LONGER. A slow backup is a slow backup; a false timeout is a corrupt status plus lock contention. Red-proof observed and restored; full suite green.
This commit is contained in:
@@ -24,13 +24,16 @@ func normalizeBackupTiers(tiers []BackupTier, legacy BackupService, cadence time
|
||||
if t.Cadence <= 0 {
|
||||
t.Cadence = cadence
|
||||
}
|
||||
if t.WaitTimeout <= 0 {
|
||||
t.WaitTimeout = 2 * time.Hour
|
||||
}
|
||||
usable = append(usable, t)
|
||||
}
|
||||
if len(usable) == 0 {
|
||||
if legacy == nil {
|
||||
return nil
|
||||
}
|
||||
return []BackupTier{{TargetID: "", Cadence: cadence, Primary: true, Service: legacy}}
|
||||
return []BackupTier{{TargetID: "", Cadence: cadence, WaitTimeout: 2 * time.Hour, Primary: true, Service: legacy}}
|
||||
}
|
||||
primary := -1
|
||||
for i, t := range usable {
|
||||
|
||||
Reference in New Issue
Block a user