v0.148.0 — coherent snapshot pairs + an offsite restore that actually restores (R-43 + R-44)
Closes the two findings from DIAG-immich-restore-2026-07-19. Viktor deleted 11
immich photos to test offsite restore; both runs flashed success and the photos
stayed gone. Two independent defects.
R-43 — no offsite path could restore a database. All three buttons were
file-only: the two "visszaállítás" actions staged to a scratch folder and never
touched postgres, and place-to-live merged only MISSING files. For a DB-indexed
app the bytes returned and the app still could not see them. The dump was
carried INTO every snapshot and could never be replayed OUT of one.
New ReconstituteFromOffsite (/backup/offbox/reconstitute): safety dump → stop →
files overwritten to the snapshot version → start → the snapshot's own dump
replayed → health wait. Two invariants:
- nothing is ever deleted (-a, no --ignore-existing, no --delete): a file
created after the snapshot survives as an extra;
- the undo exists before the act — the pre-restore- dump is verified ON DISK
before anything is stopped, overwritten or replayed; if it cannot be taken
the operation refuses with zero changes.
The replay reads the SCRATCH unit: the live unit is never overwritten, so
replaying from it would replay the current DB over itself and restore nothing.
R-44 — a manual push shipped an unrefreshed dump (up to ~24h old). That day's
predated the customer's account by four hours and probed to asset:0/user:0/
album:0 inside 52MB whose bulk was immich's shipped geodata. Every run, manual
AND nightly, now refreshes dumps + units BEFORE capturing. Order is the
mechanism: the gap can only ADD files the DB does not reference yet, never
remove one it does. Manifests carry offsite_run_id + dumps_at, so coherence is
verifiable at restore time rather than assumed; the periodic refresh carries a
prior stamp forward and never invents one.
Honesty surfaces, all warn-level and none a gate: unstamped (pre-v0.148) pairs
report their skew, ValidateDump gained an EXACT-match accounts-table sniff for
customer-empty dumps, the completion flash states an outcome instead of a
mechanism, and the missing-only button now says what it does NOT do.
11 tests; 5 red-proofs run and reverted. Two of those found real test weaknesses
rather than confirming strength — the first undo mutation was caught by a second
guard, and the first table-matching test did not discriminate between the two
matchers at all. Both tests were rewritten to the cases that separate them.
NOT in scope: R-41's catalog invariant check, nightly cadence, retention, quota
math, tier-2, and v0.147.x progress semantics beyond one added phase line.
Live acceptance (§9) has NOT run: no capability-map flip, customer-restore row
stays MISSING, R-3 stays DRAFT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9Nn14TWGzKoqAJAiVwC2s
This commit is contained in:
@@ -1,5 +1,79 @@
|
||||
## Changelog
|
||||
|
||||
### v0.148.0 — coherent snapshot pairs + an offsite restore that actually restores (2026-07-19)
|
||||
|
||||
Closes **R-43** and **R-44**, the two findings from `DIAG-immich-restore-2026-07-19`. The short
|
||||
version of that diagnosis: Viktor deleted 11 immich photos to test offsite restore, both restore
|
||||
runs flashed success, and the photos stayed gone. Two independent defects, both fixed here.
|
||||
|
||||
**R-43 — no offsite path could restore a database.** All three offsite buttons were file-only.
|
||||
The two „visszaállítás" actions staged into a scratch folder and never touched postgres; the
|
||||
place-to-live action merged only files MISSING from the live tree and never replayed a dump. For a
|
||||
DB-indexed app — most of the catalog — that combination cannot bring content back: the bytes
|
||||
return and the app still cannot see them, because its index lives in the database. The dump was
|
||||
faithfully carried INTO every snapshot and could never be replayed OUT of one.
|
||||
|
||||
New: **„Teljes visszaállítás (fájlok + adatbázis)"** (`ReconstituteFromOffsite`,
|
||||
`/backup/offbox/reconstitute`). Safety dump → stop → files overwritten to the snapshot's version →
|
||||
start → the snapshot's own dump replayed → health wait. Two invariants:
|
||||
|
||||
- **Nothing is ever deleted.** The full-restore copier is `rsync -a` with NO `--ignore-existing`
|
||||
(a changed file becomes the snapshot's version) and NO `--delete` (a file created after the
|
||||
snapshot survives as an extra). A restore that silently removed newer work would be a data-loss
|
||||
event wearing a recovery button's label.
|
||||
- **The undo exists before the act.** A `pre-restore-` dump of the live database is written and
|
||||
verified on disk BEFORE anything is stopped, overwritten or replayed; if it cannot be taken the
|
||||
whole operation refuses with zero changes. The safety dumps live in the app's own unit and
|
||||
appear in `ListDumpFiles` — an undo the customer cannot see is not much of one.
|
||||
|
||||
The replay reads the SCRATCH unit, not the live one: the live recovery unit is still never
|
||||
overwritten (it is the local restore path's source), so replaying from it would replay the current
|
||||
database back over itself and restore nothing.
|
||||
|
||||
**R-44 — a manual push shipped an unrefreshed dump.** `RunOffboxBackup` went straight to the
|
||||
restic push; dumps came only from the separate 02:30 local run, so a manual push at any other hour
|
||||
shipped a dump up to ~24h old. On 2026-07-19 that dump was taken four hours before the customer's
|
||||
account existed and probed to `asset: 0 / user: 0 / album: 0` — a 52MB file whose entire bulk was
|
||||
immich's shipped geodata tables. Size and table count both called it healthy.
|
||||
|
||||
Every offsite run — **manual and nightly** — now refreshes the dumps and recovery units FIRST, then
|
||||
captures. Order is the mechanism: the gap can only ADD files the DB does not reference yet (a
|
||||
harmless orphan blob), never remove one it does, so the file set is always a superset of what the
|
||||
restored DB points at. This also makes the nightly ordering structural instead of a coincidence of
|
||||
two scheduler entries at 02:30 and 04:15. Each unit manifest carries the run's `offsite_run_id` +
|
||||
`dumps_at`, so a snapshot's coherence is verifiable at restore time rather than assumed.
|
||||
|
||||
**Honesty surfaces** (warn-level, never gates — a false positive that blocked a restore would be
|
||||
worse than the skew it guards against):
|
||||
|
||||
- A pre-v0.148 snapshot has no stamp → the confirm says „Az adatbázis-mentés régebbi (<ts>) — a
|
||||
fájlok és az adatbázis eltérő időpontból származnak." It still restores.
|
||||
- `ValidateDump` gained a content sniff: a structurally valid dump whose accounts table has zero
|
||||
rows raises „A mentett adatbázis üresnek tűnik". Exact table-name matching, deliberately — a
|
||||
substring match on "user" would flag `user_metadata` / `album_user` / `user_audit` on every
|
||||
healthy single-user box and turn the signal into noise.
|
||||
- The completion flash states an OUTCOME, not a mechanism: „A(z) X: N fájl és az adatbázis
|
||||
visszaállítva (mentés: <ts>) — az alkalmazás újraindult." A no-database app says so explicitly
|
||||
rather than borrowing the confident sentence.
|
||||
- The old missing-only button now says what it does NOT do: „Adatbázist nem állít vissza — törölt
|
||||
tartalom ettől nem jelenik meg újra."
|
||||
|
||||
A new `dump` progress phase („Adatbázisok mentése a pillanatképhez…") names the pre-phase, which on
|
||||
a large database dominates the early wall clock and would otherwise read as a hang.
|
||||
|
||||
Tests: 11 new, with **5 red-proofs run and reverted** — replay removed (0 replayed), capture moved
|
||||
before the dump (`[capture dump]`), both undo guards removed (no refusal), substring table matching
|
||||
(join tables mistaken for accounts), buffer-exceeding rows uncounted (a wide row sniffed as empty).
|
||||
Two of those red-proofs found real test weaknesses rather than confirming strength: the first undo
|
||||
mutation was caught by a second guard, and the first table-matching test did not discriminate
|
||||
between the two matchers at all — both tests were rewritten to the cases that actually separate
|
||||
them.
|
||||
|
||||
**NOT in this slice:** the catalog-wide invariant check (stays on R-41), nightly cadence, retention,
|
||||
quota math, tier-2, and the v0.147.x progress semantics beyond the one added phase line. The
|
||||
missing-only place button's own zero-file flash is also untouched — that is the v0.147 feedback arc's
|
||||
item, not R-43/R-44.
|
||||
|
||||
### docs — the workflow moved to DooPlex-local execution (2026-07-19)
|
||||
|
||||
**Docs only, no version bump, no code change.** Claude Code now runs on DooPlex
|
||||
|
||||
Reference in New Issue
Block a user