R-203 Part 2: a run that missed a MANDATORY directory is not a successful run (v0.197.0)
gates / gates (push) Successful in 8s
gates / gates (push) Successful in 8s
The gap was already detected and warned about, in Hungarian, naming the app and the folders -- that warning is what stopped the R-201 drill. The defect was that the run still reported `ok` beside it, and a warning standing beside a success is read as a success. last_status gains "incomplete": minted, because "ok" | "error" | "running" had nothing meaning "it ran, and this app is not fully protected". NOT "error" -- the rest of the run worked and what was captured is real, so SnapshotCount and the LastSuccess anchor still record it. Half a backup is not no backup. The gaps are now recorded STRUCTURALLY (offboxRunResult.mandatoryGaps), not only as prose, so the verdict has something to act on. It reaches the operator through the EXISTING per-run digest (backup_run_failures) rather than a new event type -- a new type is a two-repo change and the hub drops anything outside allowedEventTypes. The stat-filter gains the ClassMandatory check Tier 2 already had. It is a NO-OP today (TierOffsite admits mandatory only), so no customer-visible warning disappears -- demonstrated by widening the tier filter alone and watching the check hold the line. ANTICIPATED: calibre-web on demo-hp has exactly this gap, so its off-site status becomes incomplete the moment this ships. That is correct and is the point. Red-proofs: my first Scenario-C proof PASSED because the test only reached offboxCaptureSet while the mutation lives in runOffboxInternal -- a mutation the test cannot observe is not a red-proof, and the fix was the test. The run-level test now fails under both mutations (unreachable gap recording; unconditional ok).
This commit is contained in:
@@ -785,6 +785,34 @@ func main() {
|
||||
backupMgr.OffsiteFailureMessage(err, dur))
|
||||
}
|
||||
})
|
||||
// R-203 (operator half): a completed off-site run could NOT capture a directory an app declares
|
||||
// MANDATORY. It is not a failed run — what was captured is real and the snapshot count records
|
||||
// it — but the app is NOT fully protected, and until v0.197.0 that reached nobody: the run said
|
||||
// ok, the card said ok, the hub said ok, and one WARN line inside the container said otherwise.
|
||||
//
|
||||
// It rides the EXISTING per-run digest (backup_run_failures) rather than a new event type: a
|
||||
// new type is a two-repo change (the hub's allowedEventTypes drops anything unlisted), and the
|
||||
// digest is already operator-only and already means "this run did not fully do its job".
|
||||
backupMgr.SetOffboxGapNotify(func(gaps map[string][]string) {
|
||||
apps := make([]string, 0, len(gaps))
|
||||
for app := range gaps {
|
||||
apps = append(apps, app)
|
||||
}
|
||||
sort.Strings(apps) // deterministic message + details
|
||||
d := notify.BackupRunFailuresDetails{RunKind: "offsite", Attempted: len(apps), Failed: len(apps)}
|
||||
var parts []string
|
||||
for _, app := range apps {
|
||||
sort.Strings(gaps[app])
|
||||
reason := "declared MANDATORY data directories not captured: " + strings.Join(gaps[app], ", ")
|
||||
d.Apps = append(d.Apps, notify.RunFailureDetail{App: app, Leg: "offsite-userdata", Reason: reason})
|
||||
parts = append(parts, fmt.Sprintf("%s (%s)", app, strings.Join(gaps[app], ", ")))
|
||||
}
|
||||
notifier.NotifyBackupRunFailures(fmt.Sprintf(
|
||||
"Offsite backup INCOMPLETE: %d app(s) had a directory they declare MANDATORY missing from the snapshot — %s. "+
|
||||
"The run itself succeeded and what it captured is real, but these apps are NOT fully protected off-site. "+
|
||||
"Check that the declared path exists on disk and that the app's data root resolves where the backup looks (R-203).",
|
||||
len(apps), strings.Join(parts, "; ")), d)
|
||||
})
|
||||
// R-158 / R-167 (D-c, operator half): a per-app Tier-1 recovery-unit capture failed. Until
|
||||
// v0.191.0 this was a `[WARN]` line and nothing else — /backups/apps is the page you open to
|
||||
// ask whether ONE app is backed up, and it was the one page that never said. Fires per app;
|
||||
|
||||
Reference in New Issue
Block a user