hub v0.75.0: R-81 — "no signal" is not "bad signal" (anchor the backup deadline check)

Third instance of one class (hub v0.12.0, v0.73.0, this), fixed as a class.
On 2026-07-26 03:00 UTC expected_backup_missed fired on demo-felhom, demo-hp
and drill-r50 at once; the demo-felhom one reached the CUSTOMER channel
claiming "newest backup is 176h0m0s old". Nothing was wrong — three vzdump
archives were on disk. Cause: the agent backup store is in-memory, so the
R-50 fleet restart emptied `backups` until the next run, and the hub read
empty as "no backup exists".

- assessBackupFreshness returns OK/UNKNOWN/MISSED instead of `missed bool`;
  absence is UNKNOWN until it outlives an anchored window. Still pure.
- store.GetHostReportsSince + monitor.newestBackupEvidence read the hubs own
  retained history (bounded 7-day lookback, early-exit on fresh evidence) —
  "when did I last SEE evidence of a backup?" The anchor was free: the hub
  already retains 90 days. No agent change, no new persisted state.
- store.GetFirstHostReportAt anchors absence at first contact, reusing the
  existing 26h threshold as the grace (no new knob, the v0.73.0 shape).
- Deferrals logged + counted; reason strings kept distinct.
- backupStaleAfter untouched; landmine recorded (a weekly PBS snapshot would
  alarm six days in seven) and owned by R-82.

Tests 493->508. Red-proofs A/B/C observed and restored; A reproduces the live
message verbatim. Replayed the real 03:00 reports (600/417/77 rows): all
three now silent.

Source: documentation/audits/DIAG-backup-missed-2026-07-26.md
This commit is contained in:
Claude Code
2026-07-26 11:44:15 +02:00
parent add5b9bbbb
commit f5a5e2b911
9 changed files with 843 additions and 28 deletions
+77
View File
@@ -2825,6 +2825,83 @@ func (s *Store) GetLatestHostReportJSON(customerID string) (string, error) {
return j, nil
}
// HostReportRow is one retained host-report: when the hub received it, and its payload.
type HostReportRow struct {
ReceivedAt time.Time
ReportJSON string
}
// GetHostReportsSince returns the customer's retained host-reports received at or after
// `since`, NEWEST FIRST. The hub keeps ~retention.max_days of history (90 by default), which
// is what makes it possible to ask "when did I last SEE evidence of a backup?" rather than
// only "what does the latest report say?".
//
// R-81: this is the anchor source for the backup-deadline check. The agent's backup record
// store is IN-MEMORY (felhom-agent/internal/backup/store.go — "lost on restart; the cadence
// re-populates"), so a restart empties `backups` in every report until the next backup runs.
// The hub has memory the agent does not; reading across the window is what turns that blind
// window from a false alarm into a correctly-silent verdict. Newest-first so the caller can
// stop as soon as it has seen enough (see monitor.newestBackupEvidence).
func (s *Store) GetHostReportsSince(customerID string, since time.Time) ([]HostReportRow, error) {
rows, err := s.db.Query(
`SELECT received_at, report_json FROM host_reports
WHERE customer_id = ? AND received_at >= ?
ORDER BY received_at DESC`,
customerID, since.UTC().Format("2006-01-02 15:04:05"),
)
if err != nil {
return nil, err
}
defer rows.Close()
var out []HostReportRow
for rows.Next() {
var at, j string
if err := rows.Scan(&at, &j); err != nil {
return nil, err
}
out = append(out, HostReportRow{ReceivedAt: parseSQLiteTime(at).UTC(), ReportJSON: j})
}
return out, rows.Err()
}
// GetFirstHostReportAt returns when the hub received its FIRST retained host-report for the
// customer, or (zero, nil) when it holds none.
//
// R-81: this is the observation anchor. "No backup evidence anywhere" is only meaningful
// relative to how long the hub has been watching — a box registered an hour ago has no
// evidence yet and is NOT failing. Same shape as the v0.73.0 offsite never-ran anchor:
// absence becomes a fault only once it has outlived the existing threshold, measured from a
// point where evidence first became POSSIBLE.
//
// Caveat, deliberately accepted: retention prunes at max_days, so for a host older than the
// window this returns the prune horizon rather than true first-contact. That only makes the
// anchor MORE conservative for long-lived hosts (the window has long since elapsed either
// way), and it never shortens a newborn's grace.
func (s *Store) GetFirstHostReportAt(customerID string) (time.Time, error) {
var at string
err := s.db.QueryRow(
`SELECT received_at FROM host_reports WHERE customer_id = ? ORDER BY received_at ASC LIMIT 1`,
customerID,
).Scan(&at)
if err == sql.ErrNoRows {
return time.Time{}, nil
}
if err != nil {
return time.Time{}, err
}
return parseSQLiteTime(at).UTC(), nil
}
// SetHostReportsReceivedAtForTest back-dates ALL of a customer's existing host_reports rows
// to the given SQLite datetime string, so anchor/window behaviour is testable without
// sleeping. TEST-ONLY — mirrors SetOneTimeSecretTimesForTest. Rows saved AFTER the call keep
// datetime('now'), which is how a test stages "old reports, then a fresh one".
func (s *Store) SetHostReportsReceivedAtForTest(customerID, sqliteTime string) error {
_, err := s.db.Exec(`UPDATE host_reports SET received_at = ? WHERE customer_id = ?`, sqliteTime, customerID)
return err
}
// UpsertGuestFromReport upserts the REALITY columns of a guest. On conflict it
// must NOT clobber the inert columns (api_key / desired_spec_json).
func (s *Store) UpsertGuestFromReport(g *Guest) error {