reconcile: tier-aware restore-task deadline (S4.1 unattended offsite restore-test)
A WAN (pbs-tier) restore of a large guest exceeds the restore-task wait's 10m default → the wait expired mid-restore, teardown fired against a still-restoring (not-yet-pool-associated) scratch guest → leak + a phantom VM.Allocate 403. - RestoreTestSpec.RestoreTaskTimeout (0→10m default); the restore WaitTask passes it. Local tier unchanged (10m). - config RestoreTestPBSRestoreTimeoutSeconds + accessor (default 120m). - main restoreTaskTimeout(cfg,tier): configured PBS timeout only when tier==pbs, else 0. Both scheduler + selftest spec builds. - Tests + WaitOptions red-proof + accessor contract. The "grant scratch-band VM.Allocate" follow-up is diagnosed not blind-applied: the scratch is restored INTO /pool/felhom (ACL already grants VM.Allocate), so the earlier 403 was a consequence of the timeout. No ACL/host-install change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
@@ -35,6 +35,11 @@ type RestoreTestSpec struct {
|
||||
ScratchMin int // inclusive scratch VMID band (must be > 0)
|
||||
ScratchMax int // inclusive
|
||||
BootTimeout time.Duration // 0 → DefaultBootTimeout
|
||||
// RestoreTaskTimeout bounds the wait on the restore (vzrestore) task. 0 → WaitOptions' 10m
|
||||
// default (fine for a LOCAL restore). A WAN/pbs restore of a large guest runs long, so the
|
||||
// caller sets this generously for the pbs tier — else the wait expires mid-restore, teardown
|
||||
// fires against a still-restoring (not-yet-pool-associated) guest, and the scratch leaks.
|
||||
RestoreTaskTimeout time.Duration
|
||||
}
|
||||
|
||||
// RestoreTestResult is the reconcile-local outcome (the backup package maps it to the
|
||||
@@ -243,7 +248,7 @@ func (e *Engine) runScratchTest(ctx context.Context, vmid int, spec RestoreTestS
|
||||
launched = true
|
||||
e.append(withUPID(base, upid, OpTaskRunning))
|
||||
if upid != "" {
|
||||
if _, err := e.api.WaitTask(ctx, upid, proxmox.WaitOptions{}); err != nil {
|
||||
if _, err := e.api.WaitTask(ctx, upid, proxmox.WaitOptions{Timeout: spec.RestoreTaskTimeout}); err != nil {
|
||||
res.Err = fmt.Errorf("reconcile: restore-test restore task: %w", err)
|
||||
return
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user