14 Commits

Author	SHA1	Message	Date
admin	0d402f711d	v0.237.0: the Update button takes a backup first, and tells the truth (update arc slice 4 — R-448, R-443, R-439) ... POST /api/stacks/{name}/update is now a guarded job answering 202: cheap refusals (hold — R-439, busy, migration, deploying, memory via the deploy's own memoryVerdict, a fixed 2 GB disk floor, and no restorable Tier-2 copy) → backup-first when the proven copy is older than update.backup_max_age (24h) → safety dump BEFORE the pin moves → pin → pull (failure puts the pin back) → up → health (.felhom.yml check or 60 s settle, update.health_timeout 5m). Not healthy → the app is stopped and HELD (RestoreHold reason update_failed, same store and gate as R-379) and the page names the backup to restore from; the pin stays. Success is only ever update_phase=done after health (R-443). UpdateStack is deleted. The restorable-unit predicate is EXTRACTED to backup.Tier2UnitRestorePoint and shared with the backups page (row pinned unchanged). The copy is aged by the last successful Tier-2 copy, not the manifest created_at — measured on demo-hp that created_at moves only on definition changes. Crash safety: update-journal.json before each phase; RecoverUpdates before the boot sweep, ResumeInterruptedUpdates after the guards are wired. Three unattended start paths ignored a hold and now honour it: the drive-return gate (restart + boot recreate) and the nightly volume dump. The nightly capture and Tier-2 run skip held apps so the restore point survives. No automatic rollback — measured per-app; route back = restore. Tests A–H across stacks/backup/api/web/cmd; six red-proofs seen to fail. Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS	2026-09-13 11:41:31 +02:00
admin	b8af72764d	R-353/R-357/R-358/R-360: the restore tells the truth (v0.226.0) ... Four defects on the restore surface, all proven on demo-hp during the 2026-08-21 backup-truth drill, all still in shipped code. They share one acceptance idea: a restore surface must state what it actually did, and must refuse what it cannot do. VERSION NOTE. The task specifying this targeted v0.224.0 against baseline f8c9390. Both were consumed earlier the same day by R-330 (0.224.0) and R-331 (0.225.0). Drift re-confirmed against live Gitea before the first edit, operator authorised proceeding, every symbol the spec named re-verified present at the real baseline e5eee50. R-353 -- a restore that gave back nothing still said it worked. RestoreFromRecoveryUnit returned only error, so the surface printed "<app> visszaallitva (<snapshot>)." -- equally true of a run that returned an entire dataset and one that returned nothing. The count already existed and was discarded one line deep: restoreDockerVolumesFrom always returned it, the wrapper threw it away. Now (UnitRestoreResult, error), carrying replayed counts AND what the manifest LISTED, because zero-replayed has two causes that are opposite news. Three cases, three sentences, and EVERY one is a claim about the BACKUP, never about the app -- this path has no SafetyDump discriminator, and 07-backup-architecture 6.3 records that an absent dump says nothing about the app (R-361 destroyed canonical .sql files for four months). R-357 -- the destructive restore had no free-space gate. offbox_reconstitute.go contained ZERO references to offboxFree; all three existing gates guard non-destructive paths. The gate now sits before mapOffsiteRestorePaths, writeSafetyDump and StopStack, so a refusal costs nothing. Position IS the fix, which is why the test asserts StopStack was never called. No headroom multiplier (matches PlaceOffsiteRestore; the x1.1 elsewhere predicts a download). Fail closed on either probe <= 0 -- otherwise `free < need` with need==0 is FALSE and an unmeasurable scratch sails through: a gate present and inert. R-358 -- a failed download was offered as a good one. The gate answered "the directory exists and is non-empty", which is exactly what a part-way restic run leaves. Now a completion marker written 0600 atomically AFTER restic returns nil, with any stale one cleared BEFORE it starts; both orders pinned by an AST test because resticStep is not a seam. Both handlers refuse server-side: the wizard flags control a button, and a hidden button is not a guard. SCENARIO F ANSWERED, and worse than the question assumed: a unit-only scratch IS reachable through the real flow, by the most ordinary route. "Ellenorzo visszaallitas" (mode=unit, advertised non-destructive) writes the SAME directory -- offboxRestoreScratchDir ignores `full` and --include limits what restic extracts, never where -- so a customer who ran the SAFE restore was then offered the destructive one over a unit-only copy. Filed R-396; the marker closes it. R-360 -- the delete refused only while a BACKUP ran. IsRunning() is FALSE for the whole of a verification restore; the five sibling handlers all use restoreOpBlocked(). Its doc comment claimed it already did this, which is why nobody looked -- corrected in place. Red-proofs, each printing the pre-fix behaviour, in CHANGELOG and REPORT. The first R-357 red-proof exposed a hollow test OF MY OWN and it is recorded rather than quietly fixed: the fixture refused earlier at the placement stat pre-pass, so `stops == 0` passed against the pre-fix code. Fixture corrected, assertions reordered so a removed gate reports the outage rather than "no error returned". Green gate clean: 28 packages, rc 0. All 12 controller gates OK.	2026-08-30 19:31:31 +02:00
admin	92cebb8c95	R-330: stop the backup alarming about the apps it is holding down (v0.224.0) ... Measured live on demo-hp 2026-08-30 (controller 0.223.0): the nightly db-dump and offbox-backup legs stop each stack ~13s to tar its volumes while the deadapp-check job scans every 30s, so the scan caught whichever stack was mid-cycle and pushed app_start_failed to the customer. 61 e-mails about apps that were never broken. The defect is not a missing mechanism. quiesce/suppress.go solved exactly this in v0.179.0 and works -- but classifyRunStates read only the quiesce loop's set, and that loop covers the WHOLE-GUEST backup. The per-app legs stop stacks through Manager.DumpAppVolumesSafe, which registered with nothing. Two mechanisms stop apps on purpose; only one told the alarm. Fifth instance of the "seam built but never wired" class, and the first where the unwired half was a consumer. The suppression now rides AppStopGuard, which already brackets every deliberate stop in the product (Begin before the stop, End after a successful restart) at all three call sites, and which main.go hands as ONE object to the backup manager and the exporter. scanDeployedAppRunStates takes the union of both sets. All three per-app stop paths are covered, not only the reported nightly one. It cannot latch -- End() runs only on a restart that SUCCEEDED, so unlike the quiesce loop an open-ended hold is a real hazard here: 1. ReleaseFailed drops the entry IMMEDIATELY on a restart that broke, wired at every failure path, so the app alarms on the next scan; 2. Begin REPLACES the set (one marker file = one operation); 3. appStopMaxHold (6h) caps a hold nothing released, logged at WARN. Grace is 180s, deliberately quiesce's own constant and derivation. Suppression is NOT persisted: after a crash the guard holds nothing and a down app must alarm. ReleaseFailed keeps the durable crash marker; a test pins that. Three companion red-proofs, each printing the pre-fix value (REPORT.md section 5): - drop markStopped from Begin -> "suppressed at stop = map[]" - drop ReleaseFailed from the dump -> "map[bookstack:true] after a restart that FAILED" - pass nil instead of appStopGuard -> the AST wiring test fails The third is load-bearing: the component was never the broken part, so a suite that only injected it would have been green against the shipped defect. Green gate clean: go build + go vet + go test ./... -- 28 packages, rc 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM	2026-08-30 17:58:15 +02:00
admin	5da11c4480	v0.222.0: ask whether a supervised member is DEAD before whether one is UNHEALTHY (R-384), and stop promising an undo copy nobody looked for (R-383) ... R-384. aggregateState returned StateUnhealthy the moment unhealthy > 0, and the R-51 mixed-case block that asks "is a supervised member dead?" sat below it. A two-container app whose database exits goes unhealthy BECAUSE it cannot reach that database - so the symptom the dead database causes was what suppressed the alarm for it. unhealthy is not a down state, so classifyRunStates never marked the app down and app_start_failed never fired. Measured live on demo-hp 2026-08-22: bookstack-db stopped at 21:27:01 and the F-OBS heartbeat printed "0 currently down" throughout. R-51's 18-hour immich failure, back through a different door. Two things moved, and either alone leaves the defect standing: the supervised test is hoisted above the unhealthy/starting/restarting returns, and "some members are up" now counts ANY member not in the down bucket. The old guard was running > 0, which made the R-51 block unreachable in exactly the case it was written for. IsDownState is byte-identical - unhealthy stays excluded, because an unhealthy container is running and folding it in reintroduces the flapping that exclusion exists to stop. No new state was minted. Only the ORDER changed. The priority comment was rewritten because it asserted an ordering the code no longer has. Three subtests in TestAggregateState_UnchangedBranches were AMENDED: they asserted an unhealthy/starting/restarting member beat an exited peer on unless-stopped, which pinned the defect as settled behaviour. They keep their intent with the down member given a benign policy. R-383. The double-failure message said the previous state's backup EXISTS, built from the returned path without asking the filesystem - and a missing file is one of the two ways that rollback fails. undoCopyPhrase now describes the copy from disk: present, partial, missing (still naming where it should be), or never written. Zero-length counts as missing. Test count 1494 -> 1504. Four red-proofs planted, four seen failing; the two halves of R-384 convict independently.	2026-08-23 07:25:00 +02:00
admin	968c968559	R-361: the safety dump destroyed the app's own database backup ... writeSafetyDump called DumpOne into the app's OWN unit dir and renamed the result to pre-restore-* afterwards. DumpOne writes <stack>-<dbtype>.sql - the app's canonical dump - so every safety dump overwrote the app's real backup and then moved it away, leaving the app with no database backup until the next nightly run. A local restore-from-unit in that window tells the customer the app never had a database. The comment beside it asserted the rename meant it 'can never overwrite the app's real dump'. False as written, and believed for four months. Measured live before the fix: docmost and bookstack each held only pre-restore-* files and no canonical dump. DumpOneTo takes the final path and derives its own .tmp from it. DumpOne keeps its signature and calls it with the canonical name. writeSafetyDump asks for its own name directly; the rename is gone; the comment now states the invariant and how it is enforced. db_dumps no longer lists the undo copies. All three consumers of Manifest.DBDumps were grepped and named - all inside recovery_unit.go, none reads it for recovery. The files are neither deleted nor hidden. Tests 1485 -> 1493. FIVE red-proofs, TWO PASSED first time and both are reported: the behavioural tests inject the dump seam so a mutation inside DumpOneTo was invisible, and 1.3 had no test at all. Guards added at the layer each defect lives in; both mutations then convicted.	2026-08-22 23:39:59 +02:00
admin	5b52a5964d	R-379 fix: the rollback must re-discover the DB container ... Found by v0.220.0's own live walk on its first real run. writeSafetyDump captures its DiscoveredDB before the stop; the DB-only start then re-creates the container with a new id, so the rollback's docker exec hit a dead container and sat in waitDBReady for 30s. The app was held for an infrastructure reason while its data was recoverable. Re-discover and match by {stack, engine} - what reimportDBDumpsFrom already did. Fail closed when the container cannot be found. No unit test caught it because they all inject the import seam and never look at container identity. The new test asserts the identity handed to the import.	2026-08-22 18:14:31 +02:00
admin	2c724c9283	R-379/R-380: put the customer's undo copy back when a database restore fails ... R-379 and R-380 were one failure. Both ended with a half-restored database; the only difference was whether it looked broken. Postgres emptied and crash-looped; MariaDB applied part of the dump and reported health=healthy with a zero-row schema-version table. Measured live on demo-hp 2026-08-22. The undo copy was already taken and already good - proven by hand that day on both engines. Nothing in the product could apply it. Now it does, with the same ImportDump call, before any restart and inside the DB-only window. The WHOLE undo set, matched on this run's stamp. writeSafetyDump returned one path for an app with two databases; a rollback on that would restore one and leave the other half-written. When the rollback also fails the app is HELD STOPPED (operator ruling): a running app on a half-written database lets the customer make the damage permanent. Every start path refuses it - customer button, appstop Recover, boot sweep - via the shared driveStartGate, checked ABOVE its driveless early return because these apps have no drive. The marker is ended so nothing auto-restarts it. The row goes red. Cleared with --clear-restore-hold, an operator CLI route. --single-transaction is a belt on Postgres only; MariaDB DDL is not transactional and that is why the rollback is the fix. R-381: the engine's stderr stops reaching the customer (615 bytes on MariaDB, its middle rows out of their own database) and starts reaching the operator log, which never had it. R-382: the summary log prints the volume count it already held. Undo copies resolve to their own app, are marked IsUndo, and are capped at 3 per app, pruned from the capture side. The reported render-as-an-app symptom did NOT reproduce - the live page was read first and had zero occurrences. Tests 1468 -> 1483. Eight red-proofs; ONE PASSED and is reported: the R-381 behavioural test injected below ImportDump. A guard at that layer now convicts.	2026-08-22 18:03:18 +02:00
admin	08eb1a6e3a	R-356: the off-site restore refused every app that has no data drive ... ReconstituteFromOffsite and PlaceOffsiteRestore both resolved the restore destination with the RAW HDD_PATH and read an empty answer as "the app is not installed". For 40 of the 53 catalogue apps that answer is correctly empty and permanent, so both actions refused forever for a running, healthy app — and told the customer to reinstall it "in the same place", which those apps never offer. Separate the two questions. "Installed?" is asked of ListDeployedStacks via a new Manager.isStackDeployed that fails CLOSED on a nil provider. "Where?" is answered by GetAppDrivePath — the same resolver CaptureRecoveryUnit wrote the snapshot with, so the restore aims at the place the backup came from. The 13 drive apps are unchanged: own drive, mismatch check, ack still required. A third refusal, with its own sentence, covers installed-but-no-resolvable-root. Fixtures that marked an app "installed" by giving it an HDD path now state deployment as its own fact. No assertion weakened.	2026-08-22 13:08:46 +02:00
admin	5ce3a44645	v0.218.0: attribute a DB container by its compose project, and replay volumes on the off-site restore ... R-355 (first, because it is the only one where data can be lost for good). paperless-ngx's PostgreSQL was dumped into backups/primary/paperless/db-dumps/ — a directory for a stack that does not exist, on the system drive — while the app's own unit recorded db_dumps: null. The same misattribution reached writeSafetyDump, so a destructive restore of that app took NO undo copy and the fail-closed refusal was never reached. Fixed by reading the compose project label, which is the stack name by construction (compose runs with cmd.Dir set to the stack dir and no -p). The old derivation stays as the fallback and an unresolvable attribution is now loud. Catalogue sweep, proven able to convict: one affected app of 53. The fix is in the controller, not the catalogue. R-354. ReconstituteFromOffsite skipped every unit placement and the volume archives live inside the unit, so the off-site restore had no volume leg at all — proven live with planted files: calibre-web's 1,422,848-byte config archive was in the unit, the snapshot and the checking folder, and the restore reported success without it. For the 40 of 53 apps that declare no data drive that archive is the whole dataset. restoreDockerVolumesFrom is the local path's own replay with an explicit directory: ONE implementation, two callers. Volumes replay before the database and inside the stopped window. VolumesReplayed reaches the message. The comment beside the skip was half false and is corrected; the half that still holds — the live unit is the local path's source — is named, and scenario D fingerprints the whole live unit across the operation. Seven red-proofs, each asserted applied and reverted. Two found defects in the tests, not the code: scenario D passed with the unit guard removed because the fingerprint had been narrowed and was blind to the unit root.	2026-08-22 09:43:22 +02:00
admin	985388c6e9	R-351: the restore compares where the backup says the data lived; second press cannot start a second run ... Part 3 (not droppable) and the engine half of Part 2. No version bump yet - one bump and one bake at the end of the session. PART 3a - a second press really did start a second run. Established with a test BEFORE any change: both offboxReconstituteHandler and offboxPlaceHandler answered "...elindult" and overwrote the first restore's op/stack. Cause: every restore handler gated on backupMgr.IsRunning() - the CONCURRENCY flag, which the restore goroutine acquires AFTER the handler returns (offbox_reconstitute.go:180, offbox_restore.go:393). Seven sites. The wizard had read the correct flag since v0.154.0 and said so in a comment; the handlers never moved. New Server.restoreOpBlocked() reads BOTH flags - the display flag covers the whole off-box restore, the concurrency flag is the only one the nightly backup holds - and the refusal now names the running app and a route. PART 3b - the page DOES refresh; the defect was the RESULT. backups_shared.html gated the terminal result on a page-local sawRunning flag, so a restore that finished before the page was opened, or inside one 3s poll, was shown to nobody. The 2026-08-21 OpenGist restore took 8.666s and no screen ever said it completed - the answer existed only in docker logs. RestoreOpStatus.LastRecent now carries the server's verdict. The 10-minute window moved to internal/backup as RestoreResultWindow and internal/web's constant is an alias: one expression, two surfaces. Also removed the wizard's self-contradiction, which said the state refreshes automatically AND that you must refresh the page. PART 2 (engine) - every recovery unit manifest has carried drive and namespace_root since schema 1, and NO non-test code read either back. The reconstitution opened the manifest and took only the coherence stamp, then resolved its destination from the live app. A restore into a different destination succeeded silently under a green message. New backup/offbox_placement.go: CheckPlacement (pure, total), PlacementMismatchMessage, recordedPlacementFromScratch. Compared before the safety dump and before the first byte. A mismatch is NAMED and refused; ackPlacementChange lets the customer proceed deliberately - a separate field from confirm=1, because one click must not carry two decisions. An UNKNOWN recording is never a mismatch: refusing on an absence would strand every pre-field unit. The not-installed refusal (R-253) now names the drive the backup recorded. RED-PROOFS, each mutation asserted applied and reverted to 0: B both guards removed (count asserted 2) -> the restore WAS seen starting with no drive attached: no error, full 3.00s run, wrote into /tmp/mutant-destination C Mismatch forced false -> the silent divergent restore returned E Known() forced true -> the fabricated empty prefill appeared D Mismatch forced true -> 8 ordinary reconstitute tests broke, proving reachability both ways Note on D: the existing fixtures write a schema-1 manifest with NO drive, so they are scenario-E shaped. The matching case is covered in the scenario table, not by them. Gates 11/11 OK. Suite 28 packages ok. Hungarian verified as hex, no BOM, no mojibake sentinels. NOT in this commit, still open: Part 2's scenario-A prefill UI, Part 1's deploy-page visibility line, Part 1's specification document, Part 4's measurement.	2026-08-21 21:04:16 +02:00
admin	8dbbc98ff2	v0.207.0 — R-249: the retrieval passphrase leaves the page body; R-252/R-253: two refusals learn to say what to do ... R-249. settings_security.html rendered the passphrase into a display:none span behind a Megjelenit button. That toggle stops a browser DRAWING the value and nothing else — the plaintext was in the response body of every render, so a curl of the page returned it. Found by exactly that: it landed in a session transcript while driving the documented rebuild path. The codebase already stated this rule for the recovery code and this page did not follow it (escrow_handlers.go: 'reveal (claim XHR only — R is NEVER templated server-side into HTML)'). The page now carries only HasRetrievalPassword; the value comes from POST /settings/retrieval-password/reveal — CSRF-covered because POST, no-store, and LOGGED as an act, which reading it off the markup never was. The tests assert the RAW RESPONSE BODY. Every test that asked what the customer sees passed while the bytes carried the secret; that is why this survived. Census: the render-then-hide pattern appears twice more — app_info.html (a real per-install app password in a hidden span) and deploy.html. Filed as R-254, NOT fixed here. R-252. A rebuilt box keeps its drives but loses their REGISTRATION. The restore page now states that before the customer presses anything, says the backups and drives are both still there, and links to Tarhely > Meghajtok. Page and resolver ask ONE question — HasRestoreDestination() reads the same GetSchedulableStoragePaths() the scratch resolver reads. R-253. The list promised 'a visszaallitas elobb ujratelepiti' three lines above a refusal that fired BECAUSE the app was not installed. The promise was the wrong half: reconstitution writes to the app's own GetStackHDDPath, which exists only once the CUSTOMER has chosen a drive at deploy time. Auto-reinstalling would mean the product making that choice for them. Copy now says to install first and routes to /stacks/<app>/deploy. Both notices are conditional — a healthy box renders as before, pinned by a test that fails if either becomes unconditional.	2026-08-07 18:04:26 +02:00
admin	dbcb306fcf	v0.189.0 — desired state + the app-stop crash marker (R-166 / D-b) ... The box stops inferring the customer's intent from a container count and reads what they actually asked for. Part 1 — desired state. AppConfig gains a tri-state `desired_state` (""/running/stopped), written ONLY by the customer's own action: the API action switch, DeployStack, UpdateOptionalConfig's redeploy branch, and the .fab import. Intent is written BEFORE the act and a failed write REFUSES the act. StartStack/StopStack are deliberately not writers — 14 callers, only 2 are the customer. bootrecon.isBootOrphan now reads intent instead of len(Containers)>0, which closes R-157 mechanism B (a power cut or interrupted deploy left an app with zero containers, read as a deliberate stop, and stranded silently). ABSENT MEANS UNKNOWN, NEVER "running": every pre-v0.189.0 app.yaml reads absent, so the legacy fallback is byte-identical to the old rule. A running-only startup backfill converges the unambiguous cases; `stopped` is never inferred. Part 2 — backup.AppStopGuard, a persisted marker over every stop→work→start window (volume dump, offbox reconstitute, .fab export). Its own file, never quiesce's. Written before the stop, cleared only after a restart that succeeded, kept when one fails. Recover() completes before the boot reconciler is launched and returns its outcome, which main.go reports on the existing backup_failed event once the notifier exists. A defer is not the mechanism — a SIGKILL runs none (Campaign 8 fault 10). Also: SaveAppConfig rebuilt AppConfig field-by-field (the R-100 shape) and would have dropped desired_state on every save across nine call sites. Replaced with copy-and-overlay. Measured: app.yaml does not round-trip unknown YAML keys. No hub change, no agent coupling, no user-visible string. 27/27 packages green; 7 red-proofs observed FAIL then restored.	2026-08-02 18:40:17 +02:00
admin	78ff991f1c	v0.153.0 — R-47: the DB replay no longer races the app, on BOTH restore paths ... Closes R-47. No new agent coupling — MinAgent stays 0.90.0. The replay needs a running DB container, so both restore paths started the WHOLE stack first, giving the application a window to rebuild the very schema objects the dump was about to create. Measured live on 2026-07-19 (H4, DIAG-immich-restore-round2): immich-server rebuilt clip_index two seconds before the dump's CREATE INDEX, the replay aborted "already exists" under ON_ERROR_STOP=1, and immich reported schema drift. The data survived only because pg_dump emits COPY before CREATE INDEX. Both paths now open a DB-ONLY window: only the stack's database service(s) come up, the dump is replayed with the app still down, and the full start runs only after the replay exits 0. Fail-closed: a dump with no identifiable DB service refuses BEFORE the first mutation. Every exit from the window still does a best-effort full start, so a failed restore never leaves a box with a database and no application. New: appbackup.DBServiceNames (yaml.v3 services-map parse — never a line scan; immich's top-level volume keys are the decoy) sharing dbTypeForImage with DiscoverDatabases; stacks.Manager.StartStackServices (refuses an empty list — argument-less `up -d` is a full start); RedeployFromEnv split into PersistUnitRedeployConfig + its unchanged tail. StackDataProvider's RecreateStackFromUnit becomes RecreateStackDefinitionFromUnit — the hidden `up -d` inside the old name is what carried the defect on the local path. 19 new tests (ordering plus state-at-replay-time, zero-mutation fail-closed effects, replay-failure bring-up, parser decoys, empty-list refusal); three companion red-proofs run and reverted. 23/23 packages green. Not yet live-validated: STOP-1 supervised reconstitute, golden 0.153.0.	2026-07-20 17:01:52 +02:00
admin	062357f778	v0.148.0 — coherent snapshot pairs + an offsite restore that actually restores (R-43 + R-44) ... Closes the two findings from DIAG-immich-restore-2026-07-19. Viktor deleted 11 immich photos to test offsite restore; both runs flashed success and the photos stayed gone. Two independent defects. R-43 — no offsite path could restore a database. All three buttons were file-only: the two "visszaállítás" actions staged to a scratch folder and never touched postgres, and place-to-live merged only MISSING files. For a DB-indexed app the bytes returned and the app still could not see them. The dump was carried INTO every snapshot and could never be replayed OUT of one. New ReconstituteFromOffsite (/backup/offbox/reconstitute): safety dump → stop → files overwritten to the snapshot version → start → the snapshot's own dump replayed → health wait. Two invariants: - nothing is ever deleted (-a, no --ignore-existing, no --delete): a file created after the snapshot survives as an extra; - the undo exists before the act — the pre-restore- dump is verified ON DISK before anything is stopped, overwritten or replayed; if it cannot be taken the operation refuses with zero changes. The replay reads the SCRATCH unit: the live unit is never overwritten, so replaying from it would replay the current DB over itself and restore nothing. R-44 — a manual push shipped an unrefreshed dump (up to ~24h old). That day's predated the customer's account by four hours and probed to asset:0/user:0/ album:0 inside 52MB whose bulk was immich's shipped geodata. Every run, manual AND nightly, now refreshes dumps + units BEFORE capturing. Order is the mechanism: the gap can only ADD files the DB does not reference yet, never remove one it does. Manifests carry offsite_run_id + dumps_at, so coherence is verifiable at restore time rather than assumed; the periodic refresh carries a prior stamp forward and never invents one. Honesty surfaces, all warn-level and none a gate: unstamped (pre-v0.148) pairs report their skew, ValidateDump gained an EXACT-match accounts-table sniff for customer-empty dumps, the completion flash states an outcome instead of a mechanism, and the missing-only button now says what it does NOT do. 11 tests; 5 red-proofs run and reverted. Two of those found real test weaknesses rather than confirming strength — the first undo mutation was caught by a second guard, and the first table-matching test did not discriminate between the two matchers at all. Both tests were rewritten to the cases that separate them. NOT in scope: R-41's catalog invariant check, nightly cadence, retention, quota math, tier-2, and v0.147.x progress semantics beyond one added phase line. Live acceptance (§9) has NOT run: no capability-map flip, customer-restore row stays MISSING, R-3 stays DRAFT. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P9Nn14TWGzKoqAJAiVwC2s	2026-07-19 12:21:16 +02:00