v0.227.1: the damage classifier matched restic's ordinary progress output
gates / gates (push) Successful in 11s

A patch and not a rebuilt 0.227.0: that tag was already running on demo-hp, and
re-pushing changed bytes under a live tag is the :latest hazard with extra steps.

looksLikeRepositoryDamage matched bare "pack ", "tree ", "snapshot ", "blob ". A
HEALTHY restic check prints "check all packs" and "check snapshots, trees and
blobs" -- so any check that failed for a NON-damage reason, a connection dropped
mid-run for instance, would have been classified as a corrupted repository and
told the customer their backups may be damaged. That is the false alarm that
teaches an operator to ignore the true one.

Caught by the NEGATIVE control, built from the real bytes of a real passing
check on demo-hp. The spec made the negative control mandatory and this is what
it was for: a control that has only ever seen the failing case proves nothing.

Signatures are now phrases from restic's own error wording.

Also in this commit: CONTEXT.md records the three rulings (take the flag and
skip, due-ness not a weekday, publish on OffboxReportStatus not the R-331 dead
fields) plus the measurement a future session would otherwise assume wrongly --
THE STRUCTURE CHECK DOES NOT CATCH SILENT CORRUPTION. README documents the job,
the route and the config, and corrects a line that listed four debug backup
routes when only two exist. REUSE gains three rows, including one that records
R-398 was my own mistake so nobody re-files it.
This commit is contained in:
2026-08-30 21:22:23 +02:00
parent 0d52a42c17
commit 45770f2282
6 changed files with 270 additions and 8 deletions
+3
View File
@@ -53,6 +53,9 @@
| `unitRestoreOutcomeMsg` + its three message constants (R-353, v0.226.0) | controller/internal/web/handlers.go | `(app string, res backup.UnitRestoreResult) string` | THE customer sentence for a completed LOCAL restore | Twin of `reconstituteOutcomeMsg`; copy its SHAPE (clauses earned by having done the thing, no filesystem path, base names only), never its text. The constants are named because `r353_unit_outcome_test.go` asserts them verbatim — a silent edit is how an honest message drifts back into a comforting one, which is the documented history of the sentence it replaces |
| `offsiteNoSpaceMsgFmt` + `offsiteSizeUnknownMsg` (R-357, v0.226.0) | controller/internal/backup/offbox_restore.go | two consts | EVERY headroom refusal on the off-site restore surface | **All four gates share these** (prepare, scratch, place, and the destructive reconstitute). A customer meeting one wording on one path and a different one on another has to work out whether it is the same problem. **The reconstitute gate uses NO ×1.1 margin** — it copies a measured tree; `OffboxRestorePrepareFull`'s ×1.1 predicts a download. **Fail closed when either probe reads ≤ 0**: `free < need` with `need == 0` is FALSE, so an unmeasurable input sails through — a gate present and inert |
| `Manager.OffboxFullScratchReady` + the scratch marker (R-358, v0.226.0) | controller/internal/backup/offbox_restore.go | `(stack) bool`; `.felhom-restore-complete.json` | THE gate for place-to-live and reconstitute | **It answers "did the run FINISH and was it FULL", not "are there files".** The old non-empty check passed a part-copy from a failed restic run, and the old doc comment ("PlaceOffsiteRestore re-validates per-path completeness") is what made it look adequate — that call stats top-level placements, not files. Marker written 0600 atomically AFTER restic returns nil; stale one cleared BEFORE it starts; both orders pinned by an AST test because `resticStep` is not a seam. Anything else — absent, unreadable, wrong schema, `full:false` — is NOT ready, with a WARN naming which. **Unit-only and full restores write the SAME directory**, so `full` is the only separator (R-396) |
| `Manager.CheckOffboxIntegrity` + `IntegrityResult` + `IntegrityDue` (R-359, v0.227.0) | controller/internal/backup/offbox_integrity.go | `(ctx) IntegrityResult`; `(now) (due bool, last time.Time)` | THE off-site integrity check, and the only place `restic check` is run | **It TAKES `acquireRunning` and SKIPS rather than waits — never remove that guard.** `resticStep` escalates to `unlock --remove-all` on a lock error and is only safe because every caller holds the single-flight mutex; a check without it can strip a LIVE prune's lock. **THREE outcomes, not two:** `Skipped`, `Unreachable` and failed are different facts — a timeout is unreachable, NEVER damage, and only a failure notifies. A skip and an unreachable repo do **not** advance due-ness; a failure does. **DUE-NESS, NOT A WEEKDAY** (R-341). **`looksLikeRepositoryDamage` matches PHRASES, not words** — bare `pack `/`tree `/`snapshot ` appear in restic's ordinary progress output and made a healthy run look corrupt |
| `runOffsiteIntegrityCheck` + `integrityFailedMsg` / `integrityOKMsg` (R-397, v0.227.0) | controller/cmd/controller/main.go | `(ctx, mgr, notifier, logger, force) backup.IntegrityResult` | THE one caller of the check — the scheduled job AND the debug button both go through it | ONE function so the hand-run cannot drift from the scheduled one; `force` skips due-ness and **nothing else**. Wired to `NotifyIntegrityOK` / `NotifyIntegrityFailed`, which existed with no caller since the notifier did (sixth built-but-never-wired instance). **`ok` is severity `info` and therefore mails NOBODY by design** — a weekly success e-mail is how alerts stop being read. The customer gets a sentence; restic's output goes to the log truncated (R-379) |
| **`SetOffboxRunner` / `m.runner()` — the restic exec seam, and it has ALWAYS existed** | controller/internal/backup/offbox.go (~L52, ~L386) | `offboxRunner func(ctx, env, args...) ([]byte, error)` | Driving ANY restic-backed path under test | **R-398 claimed there was no such seam and was WRONG — do not re-file it.** `resticStep` is not itself overridable, but the layer it calls is, and tests have driven restic paths through it since the off-site tier shipped. **Use this rather than adding a `resticStepFn`:** replacing `resticStep` would hide its `unlock --remove-all` escalation from exactly the assertions that must see it (R-359's lock-safety tests assert `unlock` never appears in any argv) |
| `Manager.SetOffboxLatestSnapshotFn` (R-357, v0.226.0) | controller/internal/backup/offbox_restore.go | `(fn func(ctx, stack) (id string, paths []string, err error))` INIT/TEST-ONLY | Overriding the restic snapshot lookup in tests | Exists because R-357's gate could not otherwise be tested at the level that matters: reaching it requires getting past `offboxLatestSnapshot`, which shells to restic. **The assertion the seam enables is `StopStack` call count == 0** — a gate placed after the stop returns the right sentence and still takes the outage. Nil in production |
| `CheckPlacement` + `PlacementMismatchMessage` | controller/internal/backup/offbox_placement.go | `(*RecoveryManifest, liveDrive, liveNS) PlacementCheck` / `(stack, PlacementCheck) string` | Comparing where a backup SAYS the data lived against where a restore is about to write | Pure and total — nil/empty/blank manifest all give the same honest "not known, no mismatch". **An UNKNOWN is never a mismatch** (refusing on an absence strands every pre-field unit). Compares the DRIVE only (the namespace root is derived from it), Cleaned, so a trailing slash is not a difference. The message names BOTH values on purpose |
| `RecordedUnitForStack` + `RecordedAddress` | controller/internal/backup/offbox_placement.go | `(stack) (RecordedPlacement, RecordedAddress, bool)` | Reading back the address + data folder a backup recorded, for a reinstall prefill | Local file reads over every readable namespace root — **no network, no restic, no restore**; it exists for the NOT-INSTALLED case where `GetStackHDDPath` is `""`. **`RecordedAddress.Known()` requires BOTH halves:** an absent `SUBDOMAIN` makes the live deploy path fall back to the CATALOG default (`stacks/deploy.go:88-90`), and offering that back as "what your backup says" is a fabricated fact |