v0.98.0 — R-82 Slice A fix: per-tier vzdump wait bound (the 30-minute false failure)

Found by live validation on demo-felhom, not by review.

The first real PBS-targeted backup ran past the runner's hard-coded 30-minute
WaitTask bound. The agent stopped waiting and recorded success=false WHILE THE
VZDUMP KEPT RUNNING (still running 72 min later, 2.4 GB uploaded). Consequences:
the tier stays permanently due, the next attempt collides with the guest lock
the live vzdump holds, and the hub sees a DR tier that never succeeds — R-82's
'applied and empty' fault re-created by a timeout.

Measured: ~33 MB/min over wg to Hetzner, so a first FULL ~10 GB snapshot
projects to ~5h.

- BackupTargetConfig.WaitTimeoutSeconds: per-tier bound. Primary 30m UNCHANGED
  (a local vzdump hanging 30m IS a real fault); additional tier 6h, sized from
  the measurement.
- backup.NewBackupRunnerWithWait: per-instance (per-tier) bound.
  NewBackupRunner keeps its signature, so restore-test/selftest are untouched.
- localapi.BackupTier.WaitTimeout: the fire-and-forget context is sized from the
  tier, not a fixed 2h. BOTH bounds had to move — a 6h runner bound under a 2h
  outer context reproduces the same false failure four hours later.

Same direction as restore_test_pbs_restore_timeout_seconds: when in doubt wait
LONGER. A slow backup is a slow backup; a false timeout is a corrupt status
plus lock contention.

Red-proof observed and restored; full suite green.
This commit is contained in:
Claude Code
2026-07-26 14:53:24 +02:00
parent 68bcebe493
commit a667c269c7
7 changed files with 155 additions and 21 deletions
+7 -6
View File
@@ -1286,18 +1286,19 @@ func buildLocalAPIServer(cfg config.Config, px *proxmox.Client, store *backup.St
if t.KeepLast > 0 {
prune = fmt.Sprintf("keep-last=%d", t.KeepLast)
}
r := backup.NewBackupRunner(px, t.TargetID, "", "felhom local-api", prune, logger)
r := backup.NewBackupRunnerWithWait(px, t.TargetID, "", "felhom local-api", prune, t.WaitTimeout, logger)
if t.Primary {
runner = r
}
apiTiers = append(apiTiers, localapi.BackupTier{
TargetID: t.TargetID,
Cadence: t.Cadence,
Primary: t.Primary,
Service: r,
TargetID: t.TargetID,
Cadence: t.Cadence,
WaitTimeout: t.WaitTimeout,
Primary: t.Primary,
Service: r,
})
logger.Info("backup tier armed", "target", t.TargetID, "cadence", t.Cadence.String(),
"keep_last", t.KeepLast, "primary", t.Primary)
"keep_last", t.KeepLast, "wait_timeout", t.WaitTimeout.String(), "primary", t.Primary)
}
// Guest data-drive passthrough (slice 10 P2): a root-CLI runner for the `pct set` bind + chown
// (same fenced ExecRunner the host-storage + provision back-half use).