admin
062357f778
Closes the two findings from DIAG-immich-restore-2026-07-19. Viktor deleted 11
immich photos to test offsite restore; both runs flashed success and the photos
stayed gone. Two independent defects.
R-43 — no offsite path could restore a database. All three buttons were
file-only: the two "visszaállítás" actions staged to a scratch folder and never
touched postgres, and place-to-live merged only MISSING files. For a DB-indexed
app the bytes returned and the app still could not see them. The dump was
carried INTO every snapshot and could never be replayed OUT of one.
New ReconstituteFromOffsite (/backup/offbox/reconstitute): safety dump → stop →
files overwritten to the snapshot version → start → the snapshot's own dump
replayed → health wait. Two invariants:
- nothing is ever deleted (-a, no --ignore-existing, no --delete): a file
created after the snapshot survives as an extra;
- the undo exists before the act — the pre-restore- dump is verified ON DISK
before anything is stopped, overwritten or replayed; if it cannot be taken
the operation refuses with zero changes.
The replay reads the SCRATCH unit: the live unit is never overwritten, so
replaying from it would replay the current DB over itself and restore nothing.
R-44 — a manual push shipped an unrefreshed dump (up to ~24h old). That day's
predated the customer's account by four hours and probed to asset:0/user:0/
album:0 inside 52MB whose bulk was immich's shipped geodata. Every run, manual
AND nightly, now refreshes dumps + units BEFORE capturing. Order is the
mechanism: the gap can only ADD files the DB does not reference yet, never
remove one it does. Manifests carry offsite_run_id + dumps_at, so coherence is
verifiable at restore time rather than assumed; the periodic refresh carries a
prior stamp forward and never invents one.
Honesty surfaces, all warn-level and none a gate: unstamped (pre-v0.148) pairs
report their skew, ValidateDump gained an EXACT-match accounts-table sniff for
customer-empty dumps, the completion flash states an outcome instead of a
mechanism, and the missing-only button now says what it does NOT do.
11 tests; 5 red-proofs run and reverted. Two of those found real test weaknesses
rather than confirming strength — the first undo mutation was caught by a second
guard, and the first table-matching test did not discriminate between the two
matchers at all. Both tests were rewritten to the cases that separate them.
NOT in scope: R-41's catalog invariant check, nightly cadence, retention, quota
math, tier-2, and v0.147.x progress semantics beyond one added phase line.
Live acceptance (§9) has NOT run: no capability-map flip, customer-restore row
stays MISSING, R-3 stays DRAFT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9Nn14TWGzKoqAJAiVwC2s
86 lines
4.7 KiB
Markdown
86 lines
4.7 KiB
Markdown
# REPORT — v0.148.0: coherent snapshot pairs + a real offsite restore (R-43 + R-44)
|
|
|
|
**Date:** 2026-07-19 · **Repo:** felhom-controller (v0.147.3 → **v0.148.0**) · Trunk, pushed to `main`.
|
|
|
|
Closes the two findings from `felhom.eu/documentation/audits/DIAG-immich-restore-2026-07-19.md`.
|
|
|
|
## What was broken
|
|
|
|
Viktor deleted 11 immich photos to test offsite restore. Both restore runs flashed success; the
|
|
photos stayed gone. Two independent defects:
|
|
|
|
- **R-43** — no offsite path could restore a database. All three buttons were file-only, so for a
|
|
DB-indexed app (most of the catalog) the bytes came back and the app still could not see them.
|
|
- **R-44** — a manual push shipped whatever dump the 02:30 local run had left. That day's dump
|
|
predated the customer's account by four hours and contained `asset: 0 / user: 0 / album: 0`, in a
|
|
52 MB file whose bulk was immich's shipped geodata tables.
|
|
|
|
## What shipped
|
|
|
|
**P1 — coherence pre-phase.** Every offsite run (manual AND nightly) refreshes dumps + recovery
|
|
units before capturing. Each manifest carries `offsite_run_id` + `dumps_at`. The periodic refresh
|
|
carries a prior stamp forward and never invents one. A dump failure is a loud WARN that does not
|
|
abort the push (data-first). New `dump` progress phase.
|
|
|
|
**P2 — honesty surfaces.** `ValidateDump` gained an exact-match accounts-table sniff
|
|
(`LooksEmpty`); the confirm shows dump age, an unstamped-pair warning and the empty-dump warning.
|
|
All warn-level, none a gate.
|
|
|
|
**P3 — `ReconstituteFromOffsite`** (`/backup/offbox/reconstitute`): safety dump → stop → files
|
|
overwritten → start → snapshot's dump replayed → health wait. Outcome-shaped flash. Two invariants:
|
|
nothing is ever deleted (`-a`, no `--ignore-existing`, no `--delete`), and the `pre-restore-` undo
|
|
is verified on disk before anything is touched — otherwise refuse with zero changes.
|
|
|
|
**P4 — tests.** 11 new; full suite green: `go build ./... && go vet ./... && go test ./...`.
|
|
Design gates pass: template_id, emoji, native_confirm, offbox_rename, app_row_dedup, mojibake.
|
|
|
|
> `docker_run_volume_path_gate` still fails on `internal/appexport/estimate.go:179`. **Pre-existing
|
|
> on HEAD and untouched by this release** — same finding as the v0.147.x report, same file, not a
|
|
> path this work goes near.
|
|
|
|
## Red-proofs — 5 run and reverted
|
|
|
|
| Mutation | Result |
|
|
|---|---|
|
|
| Replay removed | `0 DB dump(s) replayed` → FAIL |
|
|
| Capture moved before the dump | `got [capture dump]` → FAIL |
|
|
| Both undo guards removed | no refusal → FAIL |
|
|
| Substring table matching | join table mistaken for accounts; `UserRows = 4, want 1` → FAIL |
|
|
| Buffer-exceeding rows uncounted | wide row sniffs as empty → FAIL |
|
|
|
|
**Two of these found real test weaknesses rather than confirming strength**, and both tests were
|
|
rewritten:
|
|
|
|
- The first undo mutation still refused — a second guard (`os.Stat` on the empty path) held. The
|
|
red-proof only goes red when BOTH guards are removed, which is now recorded in REUSE.md.
|
|
- The first table-matching test did not discriminate between the two matchers at all: its aggregate
|
|
row count was identical either way. Rewritten to the case that separates them — an app with no
|
|
accounts table but with `user_metadata` / `album_user` / `user_audit`, where a substring match
|
|
would shout "your backup looks empty" on every restore of a healthy app.
|
|
|
|
## Deliberately NOT done
|
|
|
|
Per §10 of the task: no catalog-wide invariant check (stays R-41), no changes to nightly cadence,
|
|
retention, quota math or tier-2, and no touching v0.147.x progress semantics beyond the one added
|
|
phase line. The missing-only button's own zero-file flash is untouched — that belongs to the v0.147
|
|
feedback arc, not R-43/R-44. Its label now states what it does *not* do, which is in scope.
|
|
|
|
## Live acceptance — NOT YET DONE (§9, needs Viktor)
|
|
|
|
The genuine red-proof is the joint run over the tunnel and has not happened: upload → „Távoli mentés
|
|
most" → empty the trash for real → „Teljes visszaállítás (fájlok + adatbázis)" → photos visible in
|
|
the immich timeline. Until that evidence exists:
|
|
|
|
- the capability-map offsite row stays **PARTIAL / scope-contested** (no PROVEN-LIVE flip),
|
|
- the customer-restore row stays **MISSING**,
|
|
- R-3 stays **DRAFT**.
|
|
|
|
**Floor raise to v0.148.0 is Viktor's click and must come BEFORE the acceptance run.**
|
|
|
|
## Open question for Viktor
|
|
|
|
The capability-map row at `00-capability-map.md:61` claims PROVEN-LIVE citing `CAMPAIGN-6D`'s
|
|
"immich end-to-end from offsite alone". This diagnosis contradicts that phrase. I flagged it
|
|
scope-contested rather than downgrading a proven row — the ruling (did 6D exercise the DB half, or
|
|
only the file half?) is still outstanding and is not something this implementation can settle.
|