F-CRIT-2: a failed backup must not look like a fresh one (v0.106.0)

NewestArchiveTime counted an aborted PBS upload (1 byte, no manifest, NEWEST)
as a successful backup, so the tier reported fresh, went not-due, and was never
retried. On the real 168h offsite cadence that is 7 days of silence, and neither
the R-88 breaker (defers only DUE tiers) nor the hub deadline monitor (reads the
same freshness) can catch it.

R-84's storage-as-ground-truth was right; the bug is that presence was taken for
validity. Now only plausibly-complete entries count, via a measured size floor
(minPlausibleArchiveBytes = 1 MiB). Undecidable => not counted.

Size is the only tier-agnostic discriminator: verification and encrypted are
absent on EVERY local dir archive (and on a good PBS snapshot until verify-new
catches up), so gating on either would reject 100% of local backups and cause
fleet-wide backup thrash. Floor measured against the fleet: smallest real backup
is 612,397,450 B, so 1 MiB leaves 584x headroom — asserted by a test.

Rejections are announced at WARN once per distinct volid, naming snapshot and
reason; per-poll logging would emit ~288 lines/day and bury the signal.

Four red-proofs, all observed failing.
This commit is contained in:
2026-07-28 07:47:22 +02:00
parent 023655370b
commit c9a5cc664a
3 changed files with 380 additions and 3 deletions
+48
View File
@@ -1,5 +1,53 @@
# felhom-agent — Changelog
## v0.106.0 — F-CRIT-2: a failed backup must not look like a fresh one (2026-07-28)
Campaign 8 killed the PBS daemon mid-upload. PBS published the aborted upload into the storage
listing anyway — **1 byte, no manifest, and the NEWEST entry**. `NewestArchiveTime` counted it, so
the tier reported freshly backed up, went **not due**, and was never attempted again. On the real
168h offsite cadence that is **seven days of silence**, and neither backstop helps: the R-88 breaker
only defers tiers that are *due*, and the hub's deadline monitor reads the same freshness.
R-84 replaced remembered state with *"ask the storage, it is ground truth"*. That was right. The bug
is that **presence was taken for validity** — and a phantom is a more convincing lie than an empty
array, because absence at least reads as absence.
**The fix.** `NewestArchiveTime` now counts only entries that are *plausibly complete*, via a
measured size floor (`minPlausibleArchiveBytes` = 1 MiB). Undecidable ⇒ **not** counted: erring
toward "less fresh" costs one extra backup, whereas counting an undecidable entry is the defect.
**Why size and nothing else.** This runner is tier-agnostic — the same predicate runs against PBS
and against a plain `dir` storage. Verified against the live PVE API (2026-07-28):
| candidate | phantom | good PBS | good **local** | verdict |
|---|---|---|---|---|
| `verification` | absent | present | **absent** | unusable — would reject every local backup |
| `encrypted` | absent | present | **absent** | unusable — same |
| `notes` | absent | present | present | too fragile (agent-set only) |
| `size` | **1 B** | 4.35 GB | 1.59 GB | **robust, tier-agnostic** |
The floor is measured, not chosen: the smallest real backup anywhere on the fleet is **612,397,450 B**
(~584 MiB); 1 MiB sits 584x below it and 1,048,576x above the phantom. A test asserts that headroom
so nobody can quietly raise the floor into the thrash zone.
**The opposite risk is real and is a first-class test.** A filter that is too aggressive does not
merely lose safety — the tier reports absent every poll, backs up every cycle, and **R-88 cannot
save it because those backups succeed**. That is a continuous multi-GB write loop across the fleet.
`TestNewestArchiveTime_ValidSnapshotsAreStillCounted` is that guard, red-proofed by making the
filter reject everything.
**A rejected archive is never silent.** WARN, once per distinct volid (not per 5-minute poll —
~288 lines/day would bury it), naming the snapshot and the reason.
Also established while investigating, and worth recording: **server-side prune does NOT count a
phantom toward `keep-last`** — a dry-run with `keep-last 2` against three real snapshots plus a
phantom retained *two real ones plus the phantom*. So there is **no retention/data-loss bug**. But
prune never removes phantoms either, so they accumulate one per aborted upload, forever.
Files: `internal/backup/runner.go`, `internal/backup/archive_completeness_test.go` (new).
No wire/contract change; the controller needs no change — it consumes `age_state` and now simply
receives the truth.
## v0.105.0 — R-88 Part 2: the agent can finally say "unknown" (2026-07-27)
`newestArchiveOn` promised, in its own doc comment, that *"errors and unsupported services degrade to