installer 1.25.0: the off-site tier stops asking to prune (R-191)
gates / gates (push) Successful in 8s

Every weekly off-site run uploaded successfully and then failed the job on a prune
the box's token is deliberately refused — R-89 moved off-site pruning server-side
to ep0 and box tokens stay write-only. The 2026-07-26 'two weeks' ruling was not
reversed; where it is enforced moved, and keep_last: 2 did not follow. Now 0,
which the agent's existing guard already reads as 'never prune from the box'.

Verified read-only on ep0 before changing it: both namespaces have a prune job at
03:30 keep-last 2 that has run every day since 2026-07-27 — 18 tasks, all OK, the
newest keeping exactly two. Without that check this would have traded a weekly
false alarm for unbounded growth.

A gate asserts the offsite tier carries no client-side prune. The local tier is
untouched.
This commit is contained in:
2026-08-04 09:46:09 +02:00
parent 33f0ab2f33
commit 51871a7ea6
3 changed files with 75 additions and 4 deletions
+23
View File
@@ -1,3 +1,26 @@
## 1.25.0 — the off-site tier stops asking to prune (2026-08-04, R-191)
**A backup that worked must not report failure.** The off-site tier was written with `keep_last: 2`,
so every weekly run uploaded its snapshot successfully and then failed the whole job on a prune the
box's token is deliberately refused: `prune 'ct/9201': permission check failed - missing
Datastore.Modify|Datastore.Prune``TASK ERROR: job errors``whole_guest_backup_failed` in the
operator's inbox. Every week, on both boxes, about a backup that had already succeeded.
**R-89 moved off-site pruning SERVER-SIDE** — ep0 runs a per-namespace prune job and box tokens stay
write-only, so a box can never delete its own off-site history. The 2026-07-26 "two weeks" ruling was
not reversed; where it is ENFORCED moved, and this value did not follow. The tier now writes
`keep_last: 0`, which the agent's existing guard (`allowPBSPrune = !primary && keep_last > 0`) already
reads as "never prune from the box" — no agent change needed.
**VERIFIED BEFORE CHANGING IT** (read-only on ep0, 2026-08-04): prune jobs `prune-demo-felhom` and
`prune-demo-hp` exist on datastore `felhom-offsite`, one per namespace, schedule 03:30, keep-last 2,
and have run **every day since 2026-07-27 — 18 tasks, all `status=OK`**, the newest showing
`retention options: --ns demo-felhom --max-depth 0 --keep-last 2` and keeping exactly two. Disabling
the client-side prune without that check would have traded a weekly false alarm for unbounded growth.
A gate now asserts the off-site tier carries no client-side prune, so the value cannot drift back
quietly. The local tier's retention is untouched.
## 1.24.0 — a pre-existing backup target is granted too (2026-08-03, R-185)
**`configure_backup_target` has two arms and only one of them granted.** The Case A arm creates the
+26 -4
View File
@@ -184,7 +184,7 @@
set -euo pipefail
SCRIPT_VERSION="1.24.0" # the SINGLE version source (F-1): -h and the run banners follow it.
SCRIPT_VERSION="1.25.0" # the SINGLE version source (F-1): -h and the run banners follow it.
# The hub used to carry a copy for its Setup tab; R-94 DELETED it
# (2026-08-02) because the hub cannot know which version a box runs —
# the Setup command fetches this script at run time. scripts/
@@ -2479,8 +2479,30 @@ for _k,_v in {"unit_dir":"/etc/systemd/system","stage_dir":"/var/lib/felhom-agen
base.setdefault('storage', {"watchdog_interval_seconds":5,"watchdog_debounce_seconds":15,"known_refresh_seconds":20})
# R-82: local DAILY + offsite WEEKLY. The two tiers carry SEPARATE cadences and retentions —
# keep_last=3 is three DAYS on the daily tier and three WEEKS on a weekly one, so one shared knob
# would guarantee that one of them is wrong. keep_last=2 on the offsite tier = two weeks (operator
# ruling 2026-07-26).
# would guarantee that one of them is wrong.
#
# THE OFFSITE TIER CARRIES NO CLIENT-SIDE RETENTION, AND THAT IS THE POINT (R-191, 2026-08-04).
# It used to be written `keep_last: 2` ("two weeks", operator ruling 2026-07-26). **R-89 then moved
# offsite pruning SERVER-SIDE** — ep0 runs a per-namespace prune job and box tokens stay write-only,
# deliberately, so that a box can never delete its own offsite history. The 2026-07-26 ruling was not
# reversed; where it is ENFORCED moved, and this value did not follow.
#
# The consequence was weekly and wrong in the worst direction: vzdump UPLOADED the snapshot fine and
# then failed the whole job on the prune the token is refused —
# `prune 'ct/9201': permission check failed - missing Datastore.Modify|Datastore.Prune`
# → `TASK ERROR: job errors` → `whole_guest_backup_failed` in the operator's inbox. Every week, on
# both boxes, about a backup that had already succeeded. A tier that cries wolf weekly is a tier
# whose real failure nobody will see.
#
# VERIFIED BEFORE CHANGING IT (2026-08-04, read-only on ep0): prune jobs `prune-demo-felhom` and
# `prune-demo-hp` exist on datastore `felhom-offsite`, one per namespace, schedule 03:30, keep-last 2,
# and have run EVERY DAY since 2026-07-27 — 18 tasks, all `status=OK`, the newest showing
# `retention options: --ns demo-felhom --max-depth 0 --keep-last 2` and keeping exactly two. Retention
# happens; it happens THERE. **If that ever stops being true, this zero is unbounded growth** — check
# ep0's prune jobs before assuming the offsite tier is retained.
#
# `keep_last: 0` means "never prune from the box" and is the value the agent's own guard reads
# (allowPBSPrune = !primary && keep_last > 0), so no agent change is needed to honour it.
#
# The offsite tier is written even though `felhom-pbs` does not exist yet: that storage appears only
# when the hub provisions the DR tier. The agent DEFERS a tier whose target storage is absent
@@ -2491,7 +2513,7 @@ base.setdefault('storage', {"watchdog_interval_seconds":5,"watchdog_debounce_sec
# setdefault: an EXISTING box's backup block is preserved WHOLE. Upgrades never gain the tier here —
# they are migrated explicitly (R-82 Slice D.2), so an in-place upgrade can never silently start
# writing to an offsite datastore.
base.setdefault('backup', {"local_backup_target":os.environ.get('BACKUP_TARGET_RESOLVED','local'),"local_backup_retention":3,"restore_storage":"local-lvm","restore_test_cadence_seconds":0,"scratch_vmid_min":990000,"scratch_vmid_max":990009,"pbs_secret_dir":"/etc/pve/priv/storage","backup_cadence_seconds":0,"backup_targets":[{"target_id":"felhom-pbs","cadence_seconds":604800,"keep_last":2}]})
base.setdefault('backup', {"local_backup_target":os.environ.get('BACKUP_TARGET_RESOLVED','local'),"local_backup_retention":3,"restore_storage":"local-lvm","restore_test_cadence_seconds":0,"scratch_vmid_min":990000,"scratch_vmid_max":990009,"pbs_secret_dir":"/etc/pve/priv/storage","backup_cadence_seconds":0,"backup_targets":[{"target_id":"felhom-pbs","cadence_seconds":604800,"keep_last":0}]})
base.setdefault('local_api', {})
base['local_api'].setdefault('enable', True)
base['local_api']['listen_addr'] = os.environ['BRIDGE_ADDR']
+26
View File
@@ -233,6 +233,32 @@ else:
"invisible to the agent, it is never restore-tested, and an empty listing looks exactly "
"like a brand-new tier." % (resolutions, grants))
# ── R-191: the OFFSITE tier must not arm a client-side prune ─────────────────────────────────
#
# R-89 moved offsite pruning SERVER-SIDE — ep0 runs a per-namespace prune job and box tokens stay
# write-only, so the box is REFUSED if it asks. When this default was `keep_last: 2` the effect was a
# weekly lie: vzdump uploaded the snapshot, then failed the whole job on the prune, and the operator
# was told the offsite backup had failed when it had succeeded.
#
# The assertion is on the OFFSITE entry only. The local tier's `local_backup_retention` is untouched
# and must stay untouched — it prunes correctly and is allowed to.
m = re.search(r'"backup_targets":\s*\[(.*?)\]', src, re.S)
if not m:
fail("cannot find backup_targets in the rendered agent.json defaults — the offsite-retention "
"assertion cannot run, and a check that cannot run must never report OK (R-191)")
else:
targets = m.group(1)
kl = re.search(r'"keep_last"\s*:\s*(\d+)', targets)
if not kl:
fail("the offsite backup_target carries no keep_last at all — expected an explicit 0 "
"(R-191: 0 means 'never prune from the box'; absent is not the same statement)")
elif kl.group(1) != "0":
fail("the offsite backup_target arms a CLIENT-SIDE prune (keep_last=%s). R-89 moved offsite "
"pruning server-side to ep0 and box tokens are write-only, so every weekly run will "
"upload successfully and then FAIL the job on a refused prune (R-191)." % kl.group(1))
else:
ok("the offsite tier arms no client-side prune (keep_last=0; retention is ep0's prune jobs)")
print()
if fails:
print("hostinstall gates: %d FAILURE(S)" % len(fails))