db38f4c800
gates / gates (push) Successful in 17s
R-434 CLOSED — and the row's own "blocked on R-433" verdict was wrong, which is the point.
The fix is a DELETION, not a replacement: withdraw the promise instead of swapping it for a
new one, and the sentence is true under every possible answer to the provider questions, so
it never needs a second rewrite. A replacement would have been blocked; a withdrawal is not.
was: "...still hold the older copy, so this is recoverable file-by-file; it is NOT
confirmed data loss. Check whether a deletion ran on the box before restoring."
now: "...still hold the older copy. The route back out of them is not yet established,
so treat this as neither confirmed data loss nor confirmed recovery. Get in touch
before restoring anything, and check whether a deletion ran on the box."
It must not swing the other way either: "your backups are gone" is still usually false.
Clause (a) — the box cannot WRITE into the snapshot area — stands and is re-confirmed.
Tests: offsite_r434_test.go, three, all driving the production path so they assert the
sentence an operator RECEIVES. ASCII-only fragments, positive and negative controls.
RED-PROOF: restoring the v0.111.0 sentence failed all three, on every fragment, with the
offending sentence printed. TestR431_FiresOnAMassDeletion asserted "NOT confirmed data
loss" and caught this fix correctly; its wording fragment is REMOVED rather than updated,
so the wording keeps ONE home.
R-435 written into the detector's own documentation, no threshold changed: it sees a mass
deletion, not one app being wiped (69 across 9 apps -> ~35 needed, one tag is ~9, and
forget --prune groups by host,tags). Says explicitly not to lower the numbers.
THE STOPPING LINE, in all three places — register, 07 section 8 head, STATUS.md.
Deferred set ENUMERATED, not described: 07 section 8 rows 4, 8, 9, 10, 11 (+11b), 12,
each tagged [BETA-DEFERRED]. A number in the brief was wrong and is corrected in place:
six rows are DEFERRED, ELEVEN carry a blank RTO (4,5,8,9,10,11,11b,12,13,14,15); the other
five are blank for reasons that are not deferred work, and row 15 is an open DEFECT (R-104)
that the stopping line does NOT cover. NO STATUS MOVED — nothing was proven today.
Two provider questions drafted, not sent, no API called (11-D stands):
documentation/runbooks/provider-questions-2026-09-01.md, linked from R-95 and R-433, and
tracked by a dated DUE-CHECKS row (2026-09-15) — the 2026-07-27 check that sat unconfirmed
for 36 days is the scar that block exists for.
R-95, R-433 BLOCKED-ON-PROVIDER. R-95's one-day demotion on a clause that did not hold is
recorded; the proposal to rank it back near the top is stated and NOT acted on. R-430 marked
LATENT with its trigger: it becomes live the moment delete is withdrawn, so it is a
precondition on the R-95 build, not a follow-up. The stale ranking paragraph ("armed",
"zero snapshots") is corrected in place, order unchanged.
Register 621 -> 688 lines; 181 rows throughout; open-state 170 -> 169.
No controller or agent change. No golden owed, no floor change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM
156 lines
6.6 KiB
Go
156 lines
6.6 KiB
Go
package monitor
|
|
|
|
import (
|
|
"strings"
|
|
"testing"
|
|
"time"
|
|
)
|
|
|
|
// R-434 — the snapshot-drop alarm must not promise a recovery that cannot be performed.
|
|
//
|
|
// WHAT WENT WRONG. hub v0.111.0 shipped, on 2026-09-01, an alarm reading "The daily Storage Box
|
|
// snapshots are read-only and still hold the older copy, so this is recoverable file-by-file; it is
|
|
// NOT confirmed data loss." Measured the same day (R-433): NO snapshot is reachable from a
|
|
// sub-account by any name — 777,600 exact names in the vendor format over nine days, zero hits, with
|
|
// a passing control. The promise named a route nobody can walk, in the one message an operator acts
|
|
// on while their customer's off-site history is disappearing.
|
|
//
|
|
// WHY THE FIX IS A DELETION AND NOT A REPLACEMENT. A sentence asserting neither loss nor recovery is
|
|
// true under every possible answer to the outstanding provider question, so it never needs a second
|
|
// rewrite. That is why this test pins the ABSENCE of a promise as hard as it pins the new words:
|
|
// the next person who "improves" this message by putting a route back into it must fail here.
|
|
//
|
|
// THESE TESTS DRIVE THE REAL PATH — saveOffsiteReport -> oc.Check() -> the notify callback — so they
|
|
// assert the CONSEQUENCE (the sentence an operator receives), not the mechanism. Asserting the
|
|
// mechanism one layer below where the damage happens is R-224, entry 9 of the doctrine table.
|
|
//
|
|
// ASCII-ONLY FRAGMENTS. The message contains an em dash. A fragment carrying one has returned 0 for
|
|
// strings that WERE there in this project before, so every fragment below is plain ASCII.
|
|
|
|
// the promise that must never come back, in the shapes it could plausibly return as
|
|
var r434ForbiddenFragments = []string{
|
|
"recoverable file-by-file",
|
|
"recoverable file by file",
|
|
"so this is recoverable",
|
|
}
|
|
|
|
// the withdrawal that replaced it
|
|
var r434RequiredFragments = []string{
|
|
"The route back out of them is not yet established",
|
|
"neither confirmed data loss nor confirmed recovery",
|
|
"Get in touch before restoring anything",
|
|
"still hold the older copy", // clause (a) STANDS and must not be lost with the promise
|
|
}
|
|
|
|
// r434Message drives the production path once and returns the message the operator would receive.
|
|
func r434Message(t *testing.T) string {
|
|
t.Helper()
|
|
st := newDiskStore(t)
|
|
var msgs []string
|
|
saveOffsiteReport(t, st, "victim", dropJSON(69, true, "", "ok"))
|
|
oc := NewOffsiteChecker(st, 48*time.Hour, func(_, et, _, msg, _, _ string) {
|
|
if et == "offsite_snapshots_dropped" {
|
|
msgs = append(msgs, msg)
|
|
}
|
|
}, quietLog())
|
|
|
|
saveOffsiteReport(t, st, "victim", dropJSON(4, true, "", "ok"))
|
|
oc.Check()
|
|
|
|
if len(msgs) != 1 {
|
|
t.Fatalf("setup: want exactly 1 offsite_snapshots_dropped message, got %d", len(msgs))
|
|
}
|
|
return msgs[0]
|
|
}
|
|
|
|
// TestR434_AlarmMakesNoRecoveryPromise — the fix, both directions, with both controls.
|
|
//
|
|
// RED-PROOF (run 2026-09-01, recorded in REPORT.md): restoring the v0.111.0 sentence in
|
|
// emitSnapshotDrop makes this FAIL on the forbidden fragment "recoverable file-by-file" AND on all
|
|
// three required fragments, with the offending sentence printed in the failure message.
|
|
func TestR434_AlarmMakesNoRecoveryPromise(t *testing.T) {
|
|
msg := r434Message(t)
|
|
|
|
// POSITIVE CONTROL — a fragment present in EVERY version of this alarm. If this is missing the
|
|
// test is reading the wrong string and every other assertion below is worthless.
|
|
if !strings.Contains(msg, "off-site backup count fell from") {
|
|
t.Fatalf("positive control failed: not the snapshot-drop message at all: %q", msg)
|
|
}
|
|
// NEGATIVE CONTROL — proves Contains can actually report absence here.
|
|
if strings.Contains(msg, "zzz-no-such-fragment-r434") {
|
|
t.Fatalf("negative control failed: matched a fragment that cannot exist: %q", msg)
|
|
}
|
|
|
|
for _, bad := range r434ForbiddenFragments {
|
|
if strings.Contains(msg, bad) {
|
|
t.Errorf("alarm promises a recovery that cannot be performed (R-433): found %q in %q", bad, msg)
|
|
}
|
|
}
|
|
for _, want := range r434RequiredFragments {
|
|
if !strings.Contains(msg, want) {
|
|
t.Errorf("alarm is missing the withdrawal wording: want %q in %q", want, msg)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestR434_AlarmStillDoesNotClaimDataLoss — the OTHER direction, and the reason the fix is a
|
|
// withdrawal rather than a reversal.
|
|
//
|
|
// After R-433 the temptation is to swing to "your backups are gone". That is still usually FALSE:
|
|
// the snapshots exist and hold the older copy; what is unproven is our route to them. An alarm that
|
|
// over-claims loss sends an operator into a destructive recovery they did not need — which is the
|
|
// failure the v0.111.0 comment was written to prevent, and it must survive its own correction.
|
|
func TestR434_AlarmStillDoesNotClaimDataLoss(t *testing.T) {
|
|
msg := r434Message(t)
|
|
|
|
for _, bad := range []string{
|
|
"data is lost", "backups are gone", "data has been lost", "permanently lost", "unrecoverable",
|
|
} {
|
|
if strings.Contains(msg, bad) {
|
|
t.Errorf("alarm over-claims loss: found %q in %q", bad, msg)
|
|
}
|
|
}
|
|
// The one phrase that must appear NEGATED, never bare. A bare "confirmed data loss" would read
|
|
// as a verdict; the shipped sentence only ever uses it inside "neither ... nor".
|
|
if strings.Contains(msg, "confirmed data loss") &&
|
|
!strings.Contains(msg, "neither confirmed data loss nor confirmed recovery") {
|
|
t.Errorf("the phrase 'confirmed data loss' appears outside its negation: %q", msg)
|
|
}
|
|
}
|
|
|
|
// TestR434_StoredEventCarriesTheSameSentence — the delivered message and the stored one are the same
|
|
// string today, and a future refactor that formats them separately must not let them drift: the
|
|
// operator reads the mail, but every later audit reads the stored row.
|
|
func TestR434_StoredEventCarriesTheSameSentence(t *testing.T) {
|
|
st := newDiskStore(t)
|
|
var delivered string
|
|
saveOffsiteReport(t, st, "victim", dropJSON(69, true, "", "ok"))
|
|
oc := NewOffsiteChecker(st, 48*time.Hour, func(_, et, _, msg, _, _ string) {
|
|
if et == "offsite_snapshots_dropped" {
|
|
delivered = msg
|
|
}
|
|
}, quietLog())
|
|
saveOffsiteReport(t, st, "victim", dropJSON(4, true, "", "ok"))
|
|
oc.Check()
|
|
|
|
evs, err := st.GetRecentEvents("victim", 50)
|
|
if err != nil {
|
|
t.Fatalf("GetRecentEvents: %v", err)
|
|
}
|
|
var stored []string
|
|
for _, e := range evs {
|
|
if e.EventType == "offsite_snapshots_dropped" {
|
|
stored = append(stored, e.Message)
|
|
}
|
|
}
|
|
if len(stored) != 1 {
|
|
t.Fatalf("want exactly 1 stored offsite_snapshots_dropped, got %d", len(stored))
|
|
}
|
|
if stored[0] != delivered {
|
|
t.Errorf("stored and delivered messages have drifted:\n stored: %q\n delivered: %q", stored[0], delivered)
|
|
}
|
|
if strings.Contains(stored[0], "recoverable file-by-file") {
|
|
t.Errorf("the stored row still carries the withdrawn promise: %q", stored[0])
|
|
}
|
|
}
|