v0.231.0 - the box proves its own off-site copy still holds something (R-87)
gates / gates (push) Successful in 11s

R-87 re-scoped by its own spike and built as Option C. MinAgent 0.129.0 unchanged.

THE QUESTION NOTHING ASKED. The weekly check proves the stored bytes are the bytes we
stored; it cannot tell us we stored the WRONG thing. A hollow recovery unit backs up
cleanly, checks cleanly at 100 percent depth, restores cleanly and gives the customer
nothing back - measured on demo-hp 2026-08-31, 120082104 B to 7036 B in one nightly run
recorded as a success (R-403). No tier and no cadence asked it. Now offsite-proof does,
nightly, on one app.

IT DOES NOT prove a restore puts data back into a running app. That stays drill work and
07 section 8 matrix row 4 is NOT moved.

THE ACCEPTANCE RULE HAS TWO PARTS AND THE OBVIOUS ONE IS A TRAP. "Check the unit against
its own packing list" PASSES a hollow unit, because a hollow unit declares nothing. So:
(1) everything declared is present, AND (2) the manifest declares what the app is supposed
to have. Part 2 is the whole value. RED-PROOFED: the naive rule makes the hollow-unit test
read verdict "pass".

THE EXPECTATION COMES FROM INSIDE THE UNIT, never the live box - the snapshot may predate
the app's shape, and GetDockerVolumes describes the running app. Database half is
DBServiceNames, the same discriminator RestoreFromRecoveryUnit uses. Volume half is
ParseComposeNamedVolumes as an EXISTENCE check, not a name match: tars are
<project>_<volume>.tar and ResolveDockerVolumeNames derives the project from the compose
file's parent dir, which inside a unit is the literal string "compose". Measured on all
eight real units on demo-hp the counts match exactly and the naming held every time - but
"held on eight" is not "derivable" (R-355). Half a rule that is true beats a whole rule
that is invented.

THREE OUTCOMES: pass, fail (readable and empty), cannot judge. An app that legitimately
has neither a database nor volumes PASSES. RED-PROOFED: alarming on any empty unit makes
that test read verdict "fail".

IT NEVER WRITES TO THE REPOSITORY and that is asserted on the ARGV as a non-effect:
--no-lock, no unlockStale, and m.runner() rather than resticStep so the unlock --remove-all
escalation is unreachable. RED-PROOFED: routing it the customer path's way makes the test
fail on "unlock" appearing in the argv.

IT TAKES acquireRunning ITSELF and skips rather than waits, because RestoreOffboxScratch
does not take it (R-408) while offbox_integrity.go states that invariant as universal.

DUE-NESS IS PER SNAPSHOT (R-86's model), never per clock. RED-PROOFED: recording a
timestamp fails the stored-value test AND breaks the rotation - night 2 re-picks night 1's
app.

ITS SCRATCH IS A SEPARATE ROOT (backups/offsite-proof) and that is a safety decision, not
tidiness: the job deletes its copy on every path, and sharing backups/offsite-restore/<app>
would mean a nightly background job deleting the verification copy a CUSTOMER is looking
at. It is also invisible to placement, so a proof copy can never be pushed into a live app.

SHARED RATHER THAN FORKED: offboxScratchDirIn parameterises the scratch resolver on its
ROOT builder, and unitOnlyHeadroom extracts the free-space gate, so the customer path and
the proof refuse at the same floor with the same Hungarian sentence. RestoreOffboxScratch's
behaviour is unchanged.

NEW EVENT offsite_proof_empty, severity error, operator-only - deliberately NOT
backup_integrity_failed, whose hub template says the store is DAMAGED. Here the store is
sound and the content is absent: different cause, different action. The hub half shipped
FIRST, in felhom.eu 1aeaa30 (hub v0.110.0, live and verified), because an unallowlisted
type is 400'd and vanishes.

33 new tests, all groups green; full suite 1689 tests, 28 packages, rc=0. All 13 controller
gates OK. Five red-proofs run and recorded in REPORT.md.

A golden carrying 0.231.0 is OWED - the fleet is on 0.230.0. Viktor's call (R-242).
This commit is contained in:
2026-08-31 20:55:34 +02:00
parent 2d802d75e8
commit e43b5ec07d
16 changed files with 1973 additions and 33 deletions
+31 -15
View File
@@ -135,13 +135,13 @@ type Settings struct {
// The full page interrupts while `Epoch > PostponedEpoch`, and the banner reminds while
// `Epoch > OptOutEpoch` — so a fresh entry resets BOTH by arithmetic, with nothing to clear and
// nothing that can be forgotten to clear (§7.1 condition 2).
RecoveryOfferEpoch int `json:"recovery_offer_epoch,omitempty"`
RecoveryOfferActive bool `json:"recovery_offer_active,omitempty"`
RecoveryOfferEpoch int `json:"recovery_offer_epoch,omitempty"`
RecoveryOfferActive bool `json:"recovery_offer_active,omitempty"`
// RecoveryOfferSince (RFC3339) stamps when the CURRENT epoch began — the anchor the undecided
// reminders escalate against (§2.3). Re-stamped on every entry, so a box that settles and is later
// rebuilt starts its reminder ladder again rather than inheriting an old one.
RecoveryOfferSince string `json:"recovery_offer_since,omitempty"`
RecoveryNoticePostponedEpoch int `json:"recovery_notice_postponed_epoch,omitempty"`
RecoveryOfferSince string `json:"recovery_offer_since,omitempty"`
RecoveryNoticePostponedEpoch int `json:"recovery_notice_postponed_epoch,omitempty"`
// RecoveryRemindOptOutEpoch — the customer ticked „ne emlékeztessen újra" in this epoch.
//
// It silences THE BANNER AND NOTHING ELSE (§7.1 condition 3). It is not an abandonment, it starts
@@ -270,7 +270,7 @@ type OffboxTarget struct {
QuotaGB int `json:"quota_gb,omitempty"`
// Runtime status (written by the off-box runner; never holds a secret).
LastRun string `json:"last_run,omitempty"` // RFC3339
LastRun string `json:"last_run,omitempty"` // RFC3339
// LastStatus — "ok" | "incomplete" | "error" | "running". R-203 added "incomplete": the run
// completed and what it captured is real, but a directory the app declares MANDATORY could not be
// captured, so the app is NOT fully protected. Distinct from "error" (the run failed) on purpose;
@@ -288,7 +288,7 @@ type OffboxTarget struct {
// and has failed every night since must keep Monday's stamp, because that stamp is precisely what
// makes the staleness threshold elapse. Clearing it on failure would restore the bug in mirror
// image (an instantly-stale tier on the first blip — the F-A1 noise path).
LastSuccess string `json:"last_success,omitempty"` // RFC3339
LastSuccess string `json:"last_success,omitempty"` // RFC3339
// LastIntegrityCheck / LastIntegrityOK (R-359) record the last time `restic check` actually RAN
// against this repository and what it found. They live HERE, beside LastSuccess, because this is
// where the off-site tier's state already is — one store, one lifetime, one atomic write.
@@ -313,9 +313,25 @@ type OffboxTarget struct {
// than v0.228.0 — and never "structure"; absence means the box cannot answer, exactly as StatsKnown
// below establishes for the counts.
LastIntegrityDepth string `json:"last_integrity_depth,omitempty"`
LastError string `json:"last_error,omitempty"`
LastDuration string `json:"last_duration,omitempty"`
RepoSizeHuman string `json:"repo_size_human,omitempty"`
// ── R-87: the nightly off-site PROOF ────────────────────────────────────────────────────────────
//
// ProvedSnapshots maps a stack name to the SNAPSHOT ID last proved for it — never a timestamp, and
// the distinction is the whole scheduling model (R-86, 07 §3). A timestamp re-proves the same
// snapshot forever and says nothing about the newest one; a snapshot ID makes an app due again the
// moment a new backup lands, and never before. Not a secret (app names + restic short IDs).
ProvedSnapshots map[string]string `json:"proved_snapshots,omitempty"`
// The newest verdict, for the report card. LastProofResult is "" when NO proof has ever reached a
// verdict on this box — a box older than v0.231.0 sends no key at all — and that MUST read as NOT
// RECORDED, never as a failure. Same rule as StatsKnown above and LastIntegrityDepth's reserved
// empty: absence and "no" are opposite news and the wire cannot tell them apart unless we do.
LastProofRun string `json:"last_proof_run,omitempty"` // RFC3339
LastProofStack string `json:"last_proof_stack,omitempty"`
LastProofSnapshot string `json:"last_proof_snapshot,omitempty"`
LastProofResult string `json:"last_proof_result,omitempty"` // "pass" | "fail" | "cannot_judge"
LastProofReason string `json:"last_proof_reason,omitempty"`
LastError string `json:"last_error,omitempty"`
LastDuration string `json:"last_duration,omitempty"`
RepoSizeHuman string `json:"repo_size_human,omitempty"`
// RepoSizeBytes (SLICE 4) is the machine-readable repo size from `restic stats` — the soft-quota
// gate's input (last-known value; a failed stats call keeps the previous one — stale-but-safe).
RepoSizeBytes int64 `json:"repo_size_bytes,omitempty"`
@@ -460,7 +476,7 @@ type CrossDriveBackup struct {
// LastSuccess=="") and every pre-existing row on the fleet would render as never-succeeded on the
// deploy — all 7 rows on the two demo boxes were in exactly that state. Set by every runner write.
SuccessTracked bool `json:"success_tracked,omitempty"`
LastWarning string `json:"last_warning,omitempty"` // Tier-2 3b: capture-gap / state-only notice (Hungarian)
LastWarning string `json:"last_warning,omitempty"` // Tier-2 3b: capture-gap / state-only notice (Hungarian)
// UnitLegSkipped / UnitPackageDate (R-403) — the newest run PRESERVED the copy's recovery unit
// instead of refreshing it, because the source unit carried no data and this one does. Both exist
// so the surface cannot render a preserved package as a fresh one: LastRun/LastSuccess describe
@@ -468,8 +484,8 @@ type CrossDriveBackup struct {
// destination unit's own manifest, so it is a fact about the copy; "" means UNKNOWN, never "now".
UnitLegSkipped bool `json:"unit_leg_skipped,omitempty"`
UnitPackageDate string `json:"unit_package_date,omitempty"`
LastDuration string `json:"last_duration,omitempty"` // "2m34s"
LastSizeHuman string `json:"last_size_human,omitempty"` // "1.2 GB"
LastDuration string `json:"last_duration,omitempty"` // "2m34s"
LastSizeHuman string `json:"last_size_human,omitempty"` // "1.2 GB"
// Customer preference (set from the per-app Tier-2 config panel; PRESERVED across the runner's
// status writes). UserDisabled turns Tier 2 off for this app; PreferredTarget pins a chosen
@@ -1565,10 +1581,10 @@ func InferStorageLabel(path string) string {
// is left with nothing to do.
type RestoreHold struct {
Stack string `json:"stack"`
At string `json:"at"` // RFC3339 UTC
ReplayError string `json:"replay_error,omitempty"` // what the restore hit
At string `json:"at"` // RFC3339 UTC
ReplayError string `json:"replay_error,omitempty"` // what the restore hit
RollbackErr string `json:"rollback_error,omitempty"` // what the rollback then hit
SafetyDump string `json:"safety_dump,omitempty"` // basename of the undo copy that could not be applied
SafetyDump string `json:"safety_dump,omitempty"` // basename of the undo copy that could not be applied
}
// SetRestoreHold records a hold. Modelled on SetDisconnected: a condition, plus what it is holding.