F-LEAK's first fix (pool adoption) was refuted live and is recorded as such. R-100 is
F-CRIT-2's defect class on the hub for the restic tier: isStale reads only LastRun, which
the controller writes unconditionally on failure. Investigated, not fixed.
Both marked SHIPPED + PROVEN-LIVE in OPEN-ITEMS and the campaign doc. All three
of Campaign 8's alarm findings are now closed (F-CRIT-1, F-CRIT-2, F-A1).
Adds the standing rule earned by this arc to the versioned workspace CLAUDE.md:
a comment asserting an invariant needs a test pinning it, or it is a wish — with
all six shipped-false-guarantee instances catalogued, and the corollary that a
test should assert the CONSEQUENCE (does the alarm fire?) not the MECHANISM
(does suppression expire?).
F-CRIT-2 marked SHIPPED + PROVEN-LIVE: NewestArchiveTime now counts only
plausibly-complete entries (measured 1 MiB floor), verified by replaying
campaign fault 2 on demo-hp in both directions.
Files two follow-ups:
- R-99: prune never REMOVES phantoms. It does NOT count them toward keep-last
(dry-run kept 2 real + the phantom), so there is no retention/data-loss bug —
but they accumulate one per aborted upload.
- F-CRIT-1: filed READY-HIGHEST. Campaign 8's other HIGH finding, untouched by
this fix.
Seam sweep: TieredBackend was the FIRST, not the only one. BackupArchiveLister
has the identical silent-degrade shape and a worse blast radius (it degrades to
the pre-R-84 in-memory-only behaviour), and no compile-time witness existed in
production code anywhere in either repo. No defect found, so no version bump and
no deploy — the witnesses are guards, proven by breaking a signature and watching
go build fail where it previously passed.
Live outage: age_state=unknown captured on real hardware for the first time, with
demo-felhom's local tier genuinely due throughout — the controller deferred and
zero app stacks were stopped. The R-88 breaker did NOT arm and no
whole_guest_backup_failed travelled, because felhom-pbs was not due; recorded as
conditions-did-not-arise rather than claimed as coverage.
Post-boot: the volume changed device name (sdb->sda) across the reboot and the
mount survived only because fstab uses by-id. That was never tested before.
R-88b closed (agent v0.105.0 + controller v0.178.0) — age_state gives 'unknown'
its own representation, with empty meaning legacy rather than unknown so the
first-backup valve keeps working on un-upgraded boxes.
R-97 note updated: hub v0.79.0 (R-97c) replaced a FALSE operator-only comment
with a real register — the comment claimed a guarantee the code did not provide.
Standing rule 4 (R-96): a recommendation that is not followed gets one line
saying why. Added to the live CLAUDE.md and this versioned copy — the live file
is not in a git repo, so committing to it alone would leave the rule as durable
as the chat it came from.
READY re-ranked: R-95 now leads.
R-97 collapsed to its shipped one-liner in ROADMAP and closed in OPEN-ITEMS.
PROMPT-TEMPLATE N.5 now names FOUR coupled artifacts instead of two: the
capability map, ROADMAP, the owning architecture doc (ruled as S-1 in CONTEXT.md
but never reflected in the template CC actually reads, so it bound nobody), and
OPEN-ITEMS.md. Tasks must now report which register rows they opened, closed or
re-ranked.
Ops: R-90 swap done (interim; CX33 still blocked), R-95 mitigation armed but zero
snapshots taken so it moves to WATCHING rather than closed, R-91 gate still not
satisfied. CONTEXT.md datastore path corrected to /mnt/pbs-datastore.
R-88 split: Part 1 (the failure breaker) SHIPPED in controller v0.176.0 and live
on both boxes; Part 2 (unknown != never) stays OPEN and is agent-side.
Phase 0 established the root cause at source: newestArchiveOn's (time.Time, bool)
signature cannot represent 'unknown', so a storage read ERROR collapses into a
positive 'no successful backup recorded yet'. The errored and genuine-never paths
are byte-identical on the wire, which is why Part 2 cannot be done controller-side.
R-97: the whole-guest backup tier has no failure signal to the hub at all —
internal/quiesce never imports internal/notify, so three failed backups and three
app-stack outages produced zero backup_failed events. Its only trace was a
customer-tier Hungarian app_start_failed for an app the backup itself had stopped.
Open state was spread across ROADMAP, CONTEXT.md, four audits/, three runbooks,
per-session REPORT.md files and a chat log. This is the one page to read first:
every row has a state (BLOCKED/READY/WAITING-ON-OPERATOR/WATCHING) and an owner,
and the READY rows are ranked with reasoning.
R-88 is the recommended next task — quiesce's nil-age fail-open stops every app
stack every 5 minutes with no backoff and bypasses the maintenance window, and
its trigger (a PBS read failure) is live given ep0's demonstrated OOM.
CONTEXT.md now records that OPEN-ITEMS.md is authoritative and that REPORT.md is
overwritten per session.
admin