admin
a18b18e5de
The gate's behaviour is correct and unchanged. The comment said 'a caller that cannot acquire DEFERS to its next cadence' — true of the restore-test caller, NOT of the backup caller, and it did not say so. The controller recorded the refusal as a tier failure and emailed the operator; fixed controller-side in v0.179.0. Comment only, no behaviour change.
70 lines
2.9 KiB
Go
70 lines
2.9 KiB
Go
package backup
|
|
|
|
import "sync"
|
|
|
|
// InFlight is the host-wide "one heavy guest operation at a time" gate.
|
|
//
|
|
// R-85 (Scenario F). The operator's R-82 ruling was "one backup at a time per guest"; a restore-test
|
|
// must JOIN that single-flight rather than sit outside it. It is not a lock-contention concern —
|
|
// a restore-test uses a scratch VMID, so it never touches the live guest's vzdump lock. It is a
|
|
// LINK concern: an offsite restore PULLS a multi-GB archive while an offsite backup PUSHES one, over
|
|
// the same WireGuard tunnel. On the demo fleet that link runs at ~33 MB/min upstream; running both
|
|
// at once makes each slower and pushes both toward their timeouts, which is how a healthy tier ends
|
|
// up recorded as failed.
|
|
//
|
|
// It is deliberately host-wide and coarse rather than per-guest: these boxes carry one customer
|
|
// guest, and the resource being protected (the uplink) is shared by everything on the host anyway.
|
|
//
|
|
// The gate is ADVISORY in one direction only — it never cancels anything already running. A caller
|
|
// that cannot acquire DEFERS to its next cadence. Deferring a restore-test costs a few hours of
|
|
// coverage; cancelling a running backup costs the backup.
|
|
//
|
|
// CORRECTED 2026-07-28 (F-A1). That "DEFERS" was true of the restore-test caller and NOT of the
|
|
// backup caller, and the comment did not say so. The controller's start path had no 409 branch, so
|
|
// a refusal here was recorded as a tier FAILURE: the R-88 breaker armed and the operator was
|
|
// emailed "Whole-guest backup FAILED" about a backup that was merely waiting its turn. Campaign 8
|
|
// observed it on both demo boxes in the same minute.
|
|
//
|
|
// Fixed on the CONTROLLER side (v0.179.0), which is where the misreading lived — this gate's
|
|
// behaviour was correct throughout and is unchanged. The controller now maps HTTP 409 to a
|
|
// contention path: it defers the tier, keeps it DUE, and alarms only if contention outlives the
|
|
// agent's own restore-test ceiling. Nothing here needs to change; the claim above is simply now
|
|
// true of both callers.
|
|
type InFlight struct {
|
|
mu sync.Mutex
|
|
what string // "" = idle
|
|
}
|
|
|
|
// TryAcquire claims the gate for `what`. ok=false means something else holds it, and `busy` names
|
|
// it — the name matters, because "deferred" with no reason is indistinguishable from "broken".
|
|
func (g *InFlight) TryAcquire(what string) (release func(), busy string, ok bool) {
|
|
if g == nil {
|
|
// Not wired (older call sites, tests) → no gating, previous behaviour.
|
|
return func() {}, "", true
|
|
}
|
|
g.mu.Lock()
|
|
defer g.mu.Unlock()
|
|
if g.what != "" {
|
|
return nil, g.what, false
|
|
}
|
|
g.what = what
|
|
var once sync.Once
|
|
return func() {
|
|
once.Do(func() {
|
|
g.mu.Lock()
|
|
g.what = ""
|
|
g.mu.Unlock()
|
|
})
|
|
}, "", true
|
|
}
|
|
|
|
// Busy reports what currently holds the gate ("" = idle).
|
|
func (g *InFlight) Busy() string {
|
|
if g == nil {
|
|
return ""
|
|
}
|
|
g.mu.Lock()
|
|
defer g.mu.Unlock()
|
|
return g.what
|
|
}
|