v0.99.0 — R-82 operator rulings: 2-week offsite retention + one backup at a time
Ruling 1 (2 weeks of weekly offsite backups): localPruneSpec's blanket PBS refusal is now scoped — an ADDITIONAL tier with an explicit keep_last may prune its PBS target. The refusal still applies in full to the PRIMARY tier, because BackupTarget() defaults to felhom-pbs and KeepLast() defaults to 3, so a box with neither key set would silently prune its offsite DR to 3 restore points. An additional tier cannot have that accident (keep_last defaults to 0). Ruling 3 (first backup runs as long as needed; nothing else starts until done): - additional-tier wait bound 6h -> 12h (measured ~33 MB/min => ~5h for a first full 10 GB snapshot; 12h gives margin but stays bounded so a hung task still surfaces) - ONE BACKUP AT A TIME PER GUEST across all tiers: POST /backup returns 409 when a DIFFERENT tier is in flight, naming the busy tier, with NO data object so nothing is parseable as the caller's own job. Same tier still returns that job (202, unchanged). - snapshotted now counts as in-flight, not just running — after the snapshot the vzdump is still uploading and holding the lock. The old check left a window where a second POST started a real second vzdump. Latent bug, closed. Full suite green (29 packages); red-proof observed and restored.
This commit is contained in:
@@ -1,3 +1,38 @@
|
||||
## v0.99.0 — R-82: operator rulings — 2-week offsite retention + one backup at a time (2026-07-26)
|
||||
|
||||
Implements two operator rulings of 2026-07-26. Both are behaviour changes on the multi-tier path
|
||||
only; a single-tier config is untouched.
|
||||
|
||||
### Ruling 1 — keep two weeks of weekly offsite backups
|
||||
`localPruneSpec` refused to prune ANY PBS target. That blanket refusal is now scoped: an
|
||||
**ADDITIONAL tier with an explicitly configured `keep_last`** may prune its PBS target
|
||||
(`NewBackupRunnerFull(..., allowPBSPrune)`).
|
||||
|
||||
**The refusal still applies in full to the PRIMARY tier, and that is not caution for its own sake:**
|
||||
`BackupTarget()` defaults to `"felhom-pbs"` and `KeepLast()` defaults to 3, so a box with neither key
|
||||
set would silently prune its offsite DR down to 3 restore points. An additional tier cannot have that
|
||||
accident — its `keep_last` defaults to 0 (never prune), so any value there is a deliberate act.
|
||||
|
||||
### Ruling 3 — let the first backup run as long as it needs; nothing else starts until it is done
|
||||
- **Wait bound for an additional tier raised 6h → 12h.** Measured on demo-felhom: ~33 MB/min over
|
||||
the wg link, so a first FULL ~10 GB snapshot projects to ~5h. 12h gives real margin on a slower
|
||||
link while staying BOUNDED — a genuinely hung task must still surface eventually.
|
||||
- **ONE BACKUP AT A TIME PER GUEST, ACROSS ALL TIERS.** `POST /backup` now refuses with **409** when
|
||||
a DIFFERENT tier has a backup in flight, naming the busy tier and job. vzdump holds a guest lock,
|
||||
so a concurrent second backup could not succeed anyway — but without this guard it would be
|
||||
ATTEMPTED, fail on the lock, and record a spurious failure that leaves the tier permanently due.
|
||||
- SAME tier in flight → still returns THAT job (202, idempotent) — unchanged.
|
||||
- DIFFERENT tier in flight → 409 with **no data object**, so nothing is parseable as the caller's
|
||||
own job. Handing back a foreign job id is precisely how a caller comes to believe its backup ran.
|
||||
- **`snapshotted` now counts as in flight**, not just `running`. After the storage snapshot the
|
||||
vzdump is still uploading and still holds the lock; the pre-R-82 check looked at `running` only,
|
||||
leaving a window where a second POST started a real second vzdump. Latent bug, closed here.
|
||||
|
||||
### Tests
|
||||
Full suite green (29 packages). `TestBackupPost_SecondTierRefusedWhileAnotherInFlight`,
|
||||
`…_SecondTierAllowedAfterFirstFinishes`, `…_SnapshottedCountsAsInFlight`, and the per-tier wait-bound
|
||||
test updated to 12h. Red-proof for the wait bound observed and restored.
|
||||
|
||||
## v0.98.0 — R-82 Slice A fix: per-tier vzdump wait bound (the 30-minute false failure) (2026-07-26)
|
||||
|
||||
**Found by live validation on demo-felhom, not by review.** The first real PBS-targeted backup ran
|
||||
|
||||
Reference in New Issue
Block a user