v0.148.0 — coherent snapshot pairs + an offsite restore that actually restores (R-43 + R-44)
Closes the two findings from DIAG-immich-restore-2026-07-19. Viktor deleted 11
immich photos to test offsite restore; both runs flashed success and the photos
stayed gone. Two independent defects.
R-43 — no offsite path could restore a database. All three buttons were
file-only: the two "visszaállítás" actions staged to a scratch folder and never
touched postgres, and place-to-live merged only MISSING files. For a DB-indexed
app the bytes returned and the app still could not see them. The dump was
carried INTO every snapshot and could never be replayed OUT of one.
New ReconstituteFromOffsite (/backup/offbox/reconstitute): safety dump → stop →
files overwritten to the snapshot version → start → the snapshot's own dump
replayed → health wait. Two invariants:
- nothing is ever deleted (-a, no --ignore-existing, no --delete): a file
created after the snapshot survives as an extra;
- the undo exists before the act — the pre-restore- dump is verified ON DISK
before anything is stopped, overwritten or replayed; if it cannot be taken
the operation refuses with zero changes.
The replay reads the SCRATCH unit: the live unit is never overwritten, so
replaying from it would replay the current DB over itself and restore nothing.
R-44 — a manual push shipped an unrefreshed dump (up to ~24h old). That day's
predated the customer's account by four hours and probed to asset:0/user:0/
album:0 inside 52MB whose bulk was immich's shipped geodata. Every run, manual
AND nightly, now refreshes dumps + units BEFORE capturing. Order is the
mechanism: the gap can only ADD files the DB does not reference yet, never
remove one it does. Manifests carry offsite_run_id + dumps_at, so coherence is
verifiable at restore time rather than assumed; the periodic refresh carries a
prior stamp forward and never invents one.
Honesty surfaces, all warn-level and none a gate: unstamped (pre-v0.148) pairs
report their skew, ValidateDump gained an EXACT-match accounts-table sniff for
customer-empty dumps, the completion flash states an outcome instead of a
mechanism, and the missing-only button now says what it does NOT do.
11 tests; 5 red-proofs run and reverted. Two of those found real test weaknesses
rather than confirming strength — the first undo mutation was caught by a second
guard, and the first table-matching test did not discriminate between the two
matchers at all. Both tests were rewritten to the cases that separate them.
NOT in scope: R-41's catalog invariant check, nightly cadence, retention, quota
math, tier-2, and v0.147.x progress semantics beyond one added phase line.
Live acceptance (§9) has NOT run: no capability-map flip, customer-restore row
stays MISSING, R-3 stays DRAFT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9Nn14TWGzKoqAJAiVwC2s
This commit is contained in:
@@ -51,6 +51,20 @@ type DumpValidation struct {
|
||||
Error string
|
||||
FileSize int64
|
||||
ModTime time.Time
|
||||
// R-44 (v0.148.0) content sniff — a WARN-LEVEL signal, never a gate.
|
||||
//
|
||||
// Structural validity says nothing about whether a dump holds the customer's data. The immich
|
||||
// dump of 2026-07-19 was 52MB, had a valid header and 60+ CREATE TABLEs, and contained zero
|
||||
// users and zero assets: its whole bulk was the geodata reference tables immich ships. Size and
|
||||
// table count are therefore both useless as emptiness heuristics — but an accounts table with
|
||||
// no rows is a strong, cheap, app-agnostic hint that a dump predates the customer entirely.
|
||||
//
|
||||
// Deliberately NOT a refusal: plenty of legitimate apps have no users table (UserTableFound
|
||||
// false → inconclusive → silent), and a false positive that blocked a restore would be far
|
||||
// worse than the skew it guards against. The restore confirm shows it as one extra line.
|
||||
UserTableFound bool
|
||||
UserRows int
|
||||
LooksEmpty bool // UserTableFound && UserRows == 0
|
||||
}
|
||||
|
||||
// DumpFileInfo holds info about a dump file on disk.
|
||||
@@ -363,6 +377,9 @@ func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
lineNum := 0
|
||||
headerFound := false
|
||||
tableCount := 0
|
||||
// R-44 sniff state. inUserCopy tracks a postgres `COPY … FROM stdin;` block for an accounts
|
||||
// table; rows are counted until the `\.` terminator.
|
||||
inUserCopy := false
|
||||
for {
|
||||
lineBytes, isPrefix, err := reader.ReadLine()
|
||||
if err != nil {
|
||||
@@ -374,7 +391,12 @@ func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
break // EOF
|
||||
}
|
||||
if isPrefix {
|
||||
// Line exceeds buffer — skip remainder (COPY data, large INSERTs)
|
||||
// Line exceeds buffer — skip remainder (COPY data, large INSERTs).
|
||||
// A long line inside a user COPY block is still a ROW: count it before discarding it,
|
||||
// or a table whose rows happen to be wide would sniff as empty and raise a false alarm.
|
||||
if inUserCopy {
|
||||
v.UserRows++
|
||||
}
|
||||
for isPrefix && err == nil {
|
||||
_, isPrefix, err = reader.ReadLine()
|
||||
}
|
||||
@@ -384,6 +406,23 @@ func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
line := string(lineBytes)
|
||||
lineNum++
|
||||
|
||||
// R-44 content sniff (warn-level; see DumpValidation).
|
||||
if inUserCopy {
|
||||
if line == `\.` {
|
||||
inUserCopy = false
|
||||
} else {
|
||||
v.UserRows++
|
||||
}
|
||||
} else if isUserCopyStart(line, dbType) {
|
||||
inUserCopy = true
|
||||
v.UserTableFound = true
|
||||
} else if dbType == DBTypeMariaDB && isUserInsert(line) {
|
||||
// mysqldump writes multi-row `INSERT INTO \`users\` VALUES (…),(…);` — the row count is
|
||||
// not worth parsing out of it, and presence alone answers the only question asked here.
|
||||
v.UserTableFound = true
|
||||
v.UserRows++
|
||||
}
|
||||
|
||||
// Header check — scan first 10 lines for expected dump header
|
||||
// MariaDB 11.4+ prepends a sandbox comment before the header line
|
||||
if lineNum <= 10 && !headerFound {
|
||||
@@ -427,10 +466,65 @@ func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
return v
|
||||
}
|
||||
|
||||
v.LooksEmpty = v.UserTableFound && v.UserRows == 0
|
||||
if v.LooksEmpty {
|
||||
log.Printf("[WARN] [backup] ValidateDump: %s is structurally valid (%d tables) but its accounts table has NO rows — the dump may predate the customer's data", filePath, tableCount)
|
||||
}
|
||||
|
||||
v.Valid = true
|
||||
return v
|
||||
}
|
||||
|
||||
// userTableNames are the table names treated as "the accounts table" by the R-44 sniff. Kept
|
||||
// deliberately short: a wider net (anything containing "user") would match join/audit tables like
|
||||
// `user_metadata` or `album_user`, which are legitimately empty on a healthy single-user install
|
||||
// and would produce exactly the false alarm this signal must not raise.
|
||||
var userTableNames = []string{"user", "users", "account", "accounts"}
|
||||
|
||||
// isUserCopyStart reports whether a line opens a postgres `COPY <accounts-table> … FROM stdin;`
|
||||
// block. pg_dump writes the table qualified and optionally quoted — `COPY public."user" (…)`,
|
||||
// `COPY public.users (…)` — so both forms are matched.
|
||||
func isUserCopyStart(line string, dbType DBType) bool {
|
||||
if dbType != DBTypePostgres || !strings.HasPrefix(line, "COPY ") {
|
||||
return false
|
||||
}
|
||||
rest := strings.TrimPrefix(line, "COPY ")
|
||||
sp := strings.IndexByte(rest, ' ')
|
||||
if sp < 0 {
|
||||
return false
|
||||
}
|
||||
return matchesUserTable(rest[:sp])
|
||||
}
|
||||
|
||||
// isUserInsert reports whether a line is a mysqldump INSERT into an accounts table.
|
||||
func isUserInsert(line string) bool {
|
||||
const pfx = "INSERT INTO "
|
||||
if !strings.HasPrefix(line, pfx) {
|
||||
return false
|
||||
}
|
||||
rest := strings.TrimPrefix(line, pfx)
|
||||
sp := strings.IndexByte(rest, ' ')
|
||||
if sp < 0 {
|
||||
return false
|
||||
}
|
||||
return matchesUserTable(rest[:sp])
|
||||
}
|
||||
|
||||
// matchesUserTable strips schema qualification and quoting from a dumped table reference and
|
||||
// reports whether the bare name is an accounts table.
|
||||
func matchesUserTable(ref string) bool {
|
||||
if dot := strings.LastIndexByte(ref, '.'); dot >= 0 {
|
||||
ref = ref[dot+1:]
|
||||
}
|
||||
ref = strings.Trim(ref, "\"`")
|
||||
for _, n := range userTableNames {
|
||||
if strings.EqualFold(ref, n) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ListDumpFiles returns info about SQL dump files on disk.
|
||||
//
|
||||
// M18: ValidateDump scans the dump line-by-line; on a customer with hundreds-of-MB dumps that is wasted
|
||||
|
||||
Reference in New Issue
Block a user