# REPORT — R-331: the operator Backup card said every customer had no backups **Controller v0.225.0 + hub v0.109.0 · 2026-08-30** This is Option B of a two-part request. Option A (the nightly false `app_start_failed` e-mails) shipped earlier today as controller v0.224.0 (R-330) and is recorded in this repo's `CHANGELOG.md`. --- ## 1. What was wrong The hub customer page's **Backup** card read, for **every customer, indefinitely**: ``` Enabled Yes Snapshots 0 Repo Size 0 MB Integrity Unknown ``` Measured on `demo-hp` 2026-08-30, at which moment the truth was: | source | value | |---|---| | the box's own `settings.json` | `snapshot_count: 67, repo_size_bytes: 140829678, stats_known: true` | | that night's controller log | `[offbox] backup OK: 8 app(s) backed up, 67 snapshot(s), 2m14s` | | the hub's **Offsite** page | `0.1 GB` used of a `50 GB` quota — read from the stored report | **A card that reads "no backups" over a working backup is worse than no card.** It is the R-88 direction of failure — degrading to *no backup* rather than to *unknown* — on the one screen an operator consults to answer "is this customer protected?". ## 2. Root cause The card rendered the report's **`backup`** object. Its `snapshot_count`, `repo_size_mb` and `integrity_ok` fields have had **no producer** since disk-tier restic moved to the host agent (slice 8C) — `buildBackupReport` leaves them zero *deliberately* and says so in a comment. So the zeros were not a bug in the controller; they were correct values for dead fields, being rendered as if live. **The data was never missing.** The live numbers ride in the report's **`offsite`** object (`backup.OffboxReportStatus`), which the hub *already* reads for the Offsite page (`offsiteUsageBytes`) and which `monitor.OffsiteChecker` *already* drives fill and staleness alarms from. That the Offsite page rendered demo-hp's real usage from the same stored report, at the same moment the Backup card said `0 MB`, is the proof the bytes were arriving. So the fix is a **render change over an existing feed**, not a new pipeline. ## 3. Why it was not a one-line template swap `snapshot_count: 0` means two opposite things — *this repository holds nothing* and *nobody has ever measured this repository*. **R-225 already measured that exact confusion one layer down**: a rebuilt box rendered „Tarolo meret · 0 pillanatkep" over a store that really held snapshot `f3d9cd67`, and `settings.OffboxTarget.StatsKnown` is what fixed it for the controller's own UI. It was never put on the wire, so the hub was free to make the identical mistake one layer up — and rendering the count without consulting it would have **moved R-225 to the hub instead of fixing anything**. ## 4. What changed **Controller (v0.225.0) — one field, and the dead ones labelled.** - `OffboxReportStatus.StatsKnown` now forwards `settings.OffboxTarget.StatsKnown`. It is `omitempty`, so a controller below this version sends no key, a reader sees `false`, and **false means "cannot answer", never "the answer is zero"**. Absence is ignorance, not emptiness. - The four dead `BackupReport` fields (`SnapshotCount`, `RepoSizeMB`, `LastIntegrityCheck`, `IntegrityOK`) are **kept on the wire** so historical reports in the hub's store keep parsing, but now carry a warning naming R-331 and pointing at `Offsite`. `TestBackupReport_DeadFieldsStayZero` fails the moment a producer appears for any of them — the prompt to update the hub card in the **same** change rather than shipping a field nothing renders. **Hub (v0.109.0) — the card reads `offsite`, resolved in Go.** `backup_card.go` builds a typed view, because the card's whole subject is a distinction a template `{{if}}` chain over `map[string]interface{}` float64s cannot keep: | report state | card shows | |---|---| | no `offsite` object at all | "No off-site data reported" — **and says explicitly this is not the same as "no backups"** | | `enabled:false` + declared `state` | the blocker by name (`needs_credential`) — a different operator action from "not enabled" | | enabled, `stats_known:false` | **—**, plus "never been measured". Never `0` | | enabled, `stats_known:true` | the real count and size, **including a real `0`** — measured empty is knowledge | **The Integrity row is deleted, not re-sourced.** Nothing produces it: the controller runs no integrity check, and `NotifyIntegrityOK` / `NotifyIntegrityFailed` exist and are **called from nowhere**. A row that can only ever read "Unknown" is not information, and one that could read "OK" from an unwritten field would be a lie. `fmtBytesAuto` is new rather than reusing the Offsite page's `fmtBytesGB`: that one is fixed at GB because it renders against GB quotas, and it turns demo-hp's real 140 829 678 bytes into `0.1 GB` — which on a card whose entire defect was under-reporting a real backup reads as "nearly nothing". ## 5. Tests and red-proofs The hub tests assert the **rendered page**, using demo-hp's real reported values, so a regression fails against the same numbers the defect was measured against. The defect lived in the template's choice of source object, so a test one layer below it would have been green against the shipped bug. | test | pins | |---|---| | `TestBackupCard_RendersTheRealSnapshotCount` (hub) | 67 and 134.3 MB are on the page; no `0 MB`; no Integrity row | | `TestBackupCard_ThreeWayRuling` (hub) | unmeasured ≠ measured-empty ≠ no-object, all four branches | | `TestBackupCard_OldControllerDegradesToUnknownNotEmpty` (hub) | upgrading the hub ahead of the fleet must not report every customer as empty | | `TestFmtBytesAuto_...` (hub) | the real byte value renders as 134.3 MB, not 0.1 GB | | `TestOffboxReportStatus_CarriesStatsKnown` (controller) | the field is on the wire | | `TestOffboxReportStatus_MeasuredEmptyIsNotUnmeasured` (controller) | asserted on the **JSON**, not the struct — the hub sees bytes | | `TestOffboxReportStatus_AbsentStatsKnownParsesAsUnknown` (controller) | the fail-safe direction | | `TestBackupReport_DeadFieldsStayZero` (controller) | the dead fields stay dead | **Red-proofs, each printing the pre-fix value:** 1. Restore the pre-fix card markup → **all four hub tests fail**, reporting `67` and `134.3 MB` absent from the rendered page and the `Integrity` row present. 2. Drop `StatsKnown: t.StatsKnown` from `OffboxReportStatus()` → `a MEASURED empty repository reported stats_known=`. Both restored immediately; `git diff` clean afterwards. **Green gates:** controller `go build && go vet && go test ./...` — 28 packages, rc 0. Hub, same command in `hub/` — 18 packages, rc 0. ## 6. Deployment and live verification Recorded in the hub's own `REPORT.md`; the two halves ship together (the hub renders "unknown" for any box still on 0.224.0, which is correct rather than wrong). ## 7. Not done, and why - **No staleness verdict on the card.** `monitor.OffsiteChecker` already owns that and alarms on it. A second verdict on the same data is two things that can disagree, which this codebase has been bitten by before (`LastRun` vs `LastSuccess`, R-100). - **The dead `BackupReport` fields were not removed from the wire.** Removing them would stop historical reports already in the hub's store from parsing, for no gain — nothing renders them now, and a test fails if anything starts producing them.