R-353/R-357/R-358/R-360: the restore tells the truth (v0.226.0)
gates / gates (push) Successful in 11s
gates / gates (push) Successful in 11s
Four defects on the restore surface, all proven on demo-hp during the 2026-08-21 backup-truth drill, all still in shipped code. They share one acceptance idea: a restore surface must state what it actually did, and must refuse what it cannot do. VERSION NOTE. The task specifying this targeted v0.224.0 against baselinef8c9390. Both were consumed earlier the same day by R-330 (0.224.0) and R-331 (0.225.0). Drift re-confirmed against live Gitea before the first edit, operator authorised proceeding, every symbol the spec named re-verified present at the real baselinee5eee50. R-353 -- a restore that gave back nothing still said it worked. RestoreFromRecoveryUnit returned only error, so the surface printed "<app> visszaallitva (<snapshot>)." -- equally true of a run that returned an entire dataset and one that returned nothing. The count already existed and was discarded one line deep: restoreDockerVolumesFrom always returned it, the wrapper threw it away. Now (UnitRestoreResult, error), carrying replayed counts AND what the manifest LISTED, because zero-replayed has two causes that are opposite news. Three cases, three sentences, and EVERY one is a claim about the BACKUP, never about the app -- this path has no SafetyDump discriminator, and 07-backup-architecture 6.3 records that an absent dump says nothing about the app (R-361 destroyed canonical .sql files for four months). R-357 -- the destructive restore had no free-space gate. offbox_reconstitute.go contained ZERO references to offboxFree; all three existing gates guard non-destructive paths. The gate now sits before mapOffsiteRestorePaths, writeSafetyDump and StopStack, so a refusal costs nothing. Position IS the fix, which is why the test asserts StopStack was never called. No headroom multiplier (matches PlaceOffsiteRestore; the x1.1 elsewhere predicts a download). Fail closed on either probe <= 0 -- otherwise `free < need` with need==0 is FALSE and an unmeasurable scratch sails through: a gate present and inert. R-358 -- a failed download was offered as a good one. The gate answered "the directory exists and is non-empty", which is exactly what a part-way restic run leaves. Now a completion marker written 0600 atomically AFTER restic returns nil, with any stale one cleared BEFORE it starts; both orders pinned by an AST test because resticStep is not a seam. Both handlers refuse server-side: the wizard flags control a button, and a hidden button is not a guard. SCENARIO F ANSWERED, and worse than the question assumed: a unit-only scratch IS reachable through the real flow, by the most ordinary route. "Ellenorzo visszaallitas" (mode=unit, advertised non-destructive) writes the SAME directory -- offboxRestoreScratchDir ignores `full` and --include limits what restic extracts, never where -- so a customer who ran the SAFE restore was then offered the destructive one over a unit-only copy. Filed R-396; the marker closes it. R-360 -- the delete refused only while a BACKUP ran. IsRunning() is FALSE for the whole of a verification restore; the five sibling handlers all use restoreOpBlocked(). Its doc comment claimed it already did this, which is why nobody looked -- corrected in place. Red-proofs, each printing the pre-fix behaviour, in CHANGELOG and REPORT. The first R-357 red-proof exposed a hollow test OF MY OWN and it is recorded rather than quietly fixed: the fixture refused earlier at the placement stat pre-pass, so `stops == 0` passed against the pre-fix code. Fixture corrected, assertions reordered so a removed gate reports the outage rather than "no error returned". Green gate clean: 28 packages, rc 0. All 12 controller gates OK.
This commit is contained in:
+113
@@ -1,3 +1,116 @@
|
||||
## v0.226.0 — four ways the restore screen could mislead a customer (2026-08-30, R-353/R-357/R-358/R-360)
|
||||
**MinAgent: 0.129.0** (unchanged — no new agent coupling)
|
||||
|
||||
> **Version note.** The task specifying this work targeted **v0.224.0** against baseline `f8c9390`.
|
||||
> Both were consumed earlier the same day by R-330 (v0.224.0) and R-331 (v0.225.0). The drift was
|
||||
> re-confirmed against live Gitea before the first edit, the operator authorised proceeding, and every
|
||||
> symbol the spec named was re-verified present at the real baseline `e5eee50`. This is that work at
|
||||
> **v0.226.0**.
|
||||
|
||||
All four defects were proven on `demo-hp` during the 2026-08-21 backup-truth drill and all four were
|
||||
still in shipped code. They share one acceptance idea: **a restore surface must state what it actually
|
||||
did, and must refuse what it cannot do.**
|
||||
|
||||
### R-353 — a restore that gave back nothing still said it worked
|
||||
|
||||
`RestoreFromRecoveryUnit` returned only `error`, so the surface reported `<app> visszaállítva
|
||||
(<snapshot>).` — a sentence equally true of a run that returned an app's entire dataset and of one that
|
||||
returned nothing. On 2026-08-21 an `opengist` restore printed it over a unit holding `manifest.json`
|
||||
and `compose/` and nothing else.
|
||||
|
||||
**The count already existed and was thrown away one line deep.** `restoreDockerVolumesFrom` has always
|
||||
returned it; the wrapper `restoreDockerVolumes` discarded the int. It now returns `(int, error)`, and
|
||||
`RestoreFromRecoveryUnit` returns `(UnitRestoreResult, error)` carrying volumes replayed, DBs replayed,
|
||||
**and what the manifest LISTED** — because zero-replayed has two causes and they are opposite news:
|
||||
|
||||
| state | sentence |
|
||||
|---|---|
|
||||
| something came back | „A(z) X: 2 adatkötet és az adatbázis visszaállítva — az alkalmazás újraindult." |
|
||||
| nothing came back, unit listed nothing | „…FIGYELEM: ez a mentés csak a beállításokat tartalmazta, adatot nem." |
|
||||
| nothing came back, unit listed dumps | „…FIGYELEM — a mentés N adatkötetet és M adatbázis-mentést sorol fel, de egyik sem állt vissza. Az adataid változatlanok maradtak." |
|
||||
|
||||
**Every sentence is a claim about THE BACKUP, never about the app.** „ennek az alkalmazásnak nincs
|
||||
adata" is forbidden here and the reason is recorded: the off-site twin has `SafetyDump` as an honest
|
||||
discriminator, this path has none, and 07-backup-architecture §6.3 records that an absent dump has
|
||||
causes that say nothing about the app (R-361 destroyed apps' canonical `.sql` files for four months).
|
||||
That is R-355's rule, now extended to the Tier-1 path.
|
||||
|
||||
The snapshot id is dropped from the sentence deliberately: it named WHICH backup ran and said nothing
|
||||
about what came out of it, which is the question the sentence exists to answer.
|
||||
|
||||
### R-357 — the destructive restore had no free-space gate
|
||||
|
||||
`offbox_reconstitute.go` contained **zero** references to `offboxFree`. All three existing headroom
|
||||
gates guard NON-destructive paths. The one path that stops the customer's app and overwrites their live
|
||||
data had none — and on 2026-08-21 it stopped an app, ran out of disk half-way, left 2 of 5 planted
|
||||
items in place and restarted the app.
|
||||
|
||||
The gate now sits **before `mapOffsiteRestorePaths`, before `writeSafetyDump` and well before
|
||||
`StopStack`**, so a refusal costs the customer nothing. **Position is the whole fix**, which is why the
|
||||
test asserts `StopStack` was never called rather than asserting the error string.
|
||||
|
||||
**No headroom multiplier**, matching `PlaceOffsiteRestore`: this is a local copy whose size is known
|
||||
exactly, unlike `OffboxRestorePrepareFull`'s ×1.1, which is predicting a download. Stated in a comment
|
||||
so it is not "fixed" later. **Fail-closed on either probe returning ≤ 0** — without that, `free < need`
|
||||
with `need == 0` is FALSE and an unmeasurable scratch sailed straight through: a gate present and
|
||||
inert, which is worse than no gate.
|
||||
|
||||
### R-358 — a failed download was offered as a good one
|
||||
|
||||
`OffboxFullScratchReady` answered "the directory exists and is non-empty". A restic run that dies
|
||||
part-way leaves exactly that, so „Teljes visszaállítás indítása" was offered over a part-copy and
|
||||
reported success. Its doc comment — *"PlaceOffsiteRestore re-validates per-path completeness"* — is
|
||||
what made the weak gate look adequate; that call stats top-level placements, not the files inside them.
|
||||
|
||||
`RestoreOffboxScratch` now clears any stale marker **before** restic runs and writes
|
||||
`.felhom-restore-complete.json` (0600, tmp+fsync+rename) **only after** restic returns nil. The gate
|
||||
reads it. Absent, unreadable, wrong schema or `full:false` → **not ready**, with a WARN naming which.
|
||||
|
||||
**Both handlers refuse server-side.** The wizard's `PlaceEnabled`/`RestoreEnabled` flags control a
|
||||
button, and a hidden button is not a guard — a direct POST over a part-copy used to be accepted.
|
||||
|
||||
**Scenario F's open question is answered, and the answer is worse than the question assumed.** The
|
||||
task asked whether a unit-only scratch is reachable through the real UI flow. **It is, by the most
|
||||
ordinary route available:** „Ellenőrző visszaállítás" (`mode=unit`, advertised as non-destructive)
|
||||
writes the SAME directory — `offboxRestoreScratchDir` ignores `full`, and `--include` limits what
|
||||
restic extracts, never where — so a customer who ran the SAFE verification restore was then offered the
|
||||
destructive one over a unit-only copy. Filed as **R-396**; the marker closes it.
|
||||
|
||||
### R-360 — the delete refused only while a BACKUP ran
|
||||
|
||||
`offboxVerifyCopyDeleteHandler` guarded on `s.backupMgr.IsRunning()`, which is FALSE for the whole of a
|
||||
verification restore. Its five siblings on that surface all use `restoreOpBlocked()`, which consults
|
||||
both flags; this one was missed. **Its doc comment claimed it refused during a restore, and that
|
||||
sentence is why nobody looked** — it is corrected in place rather than deleted.
|
||||
|
||||
Observed live 2026-08-21 22:35: `RestoreStatus().Running == true` while `IsRunning() == false`, and the
|
||||
delete of the copy the restore was writing into went through. No app-name comparison was added:
|
||||
refusing during ANY restore is strictly stronger and matches the other five handlers.
|
||||
|
||||
### One new test seam, and why it was necessary
|
||||
|
||||
`SetOffboxLatestSnapshotFn` overrides the restic snapshot lookup. Without it R-357's gate could not be
|
||||
tested at the level that matters — reaching it requires getting past `offboxLatestSnapshot`, which
|
||||
shells out to restic, and a gate proven only by reading the code is the assurance class this project
|
||||
has been burned by. Nil in production.
|
||||
|
||||
### Red-proofs — each printed the pre-fix behaviour
|
||||
|
||||
| mutation | observed failure |
|
||||
|---|---|
|
||||
| revert the outcome to `stackName+" visszaállítva ("+snapshotID+")."` | `THE PRE-FIX SENTENCE REACHED THE CUSTOMER: "opengist visszaállítva (snap-123)."` |
|
||||
| delete the R-357 headroom gate | `THE APP WAS STOPPED (1 call(s)) … (err=<nil>)` on all three gate tests |
|
||||
| restore the old non-empty scratch check | `a part-copy was reported READY` + the unit-only and unreadable-marker cases |
|
||||
| revert the delete guard to `IsRunning()` | `THE VERIFICATION COPY WAS DELETED while a restore was writing into it` |
|
||||
| remove both server-side scratch refusals | both handlers redirected with „elindult" over a part-copy |
|
||||
|
||||
**The first R-357 red-proof exposed a hollow test of my own and is recorded rather than quietly
|
||||
fixed:** the fixture refused earlier, at the placement stat pre-pass, so `stops == 0` passed against
|
||||
the pre-fix code. The scratch is now populated the way a completed download leaves it, and the
|
||||
assertions are ordered so a removed gate reports the outage rather than "no error returned".
|
||||
|
||||
**Green gate:** `go build ./... && go vet ./... && go test ./...` — 28 packages, rc 0.
|
||||
|
||||
## v0.225.0 — the hub could not tell an empty off-site store from an unmeasured one (2026-08-30, R-331)
|
||||
**MinAgent: 0.129.0** (unchanged — no new agent coupling)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user