R-379/R-380: put the customer's undo copy back when a database restore fails
gates / gates (push) Successful in 11s

R-379 and R-380 were one failure. Both ended with a half-restored database; the
only difference was whether it looked broken. Postgres emptied and crash-looped;
MariaDB applied part of the dump and reported health=healthy with a zero-row
schema-version table. Measured live on demo-hp 2026-08-22.

The undo copy was already taken and already good - proven by hand that day on
both engines. Nothing in the product could apply it. Now it does, with the same
ImportDump call, before any restart and inside the DB-only window.

The WHOLE undo set, matched on this run's stamp. writeSafetyDump returned one
path for an app with two databases; a rollback on that would restore one and
leave the other half-written.

When the rollback also fails the app is HELD STOPPED (operator ruling): a running
app on a half-written database lets the customer make the damage permanent. Every
start path refuses it - customer button, appstop Recover, boot sweep - via the
shared driveStartGate, checked ABOVE its driveless early return because these
apps have no drive. The marker is ended so nothing auto-restarts it. The row goes
red. Cleared with --clear-restore-hold, an operator CLI route.

--single-transaction is a belt on Postgres only; MariaDB DDL is not transactional
and that is why the rollback is the fix.

R-381: the engine's stderr stops reaching the customer (615 bytes on MariaDB, its
middle rows out of their own database) and starts reaching the operator log,
which never had it.
R-382: the summary log prints the volume count it already held.
Undo copies resolve to their own app, are marked IsUndo, and are capped at 3 per
app, pruned from the capture side. The reported render-as-an-app symptom did NOT
reproduce - the live page was read first and had zero occurrences.

Tests 1468 -> 1483. Eight red-proofs; ONE PASSED and is reported: the R-381
behavioural test injected below ImportDump. A guard at that layer now convicts.
This commit is contained in:
2026-08-22 18:03:18 +02:00
parent 0f3cf0dbb2
commit 2c724c9283
15 changed files with 1288 additions and 37 deletions
+19
View File
@@ -54,6 +54,18 @@ type Manager struct {
// precedent.
unitNotify func(stackName string, err error, usage *UnitSpace)
// restoreHoldNotify (R-379/R-380), if set, is called ONCE when an app is HELD after a database
// replay failed AND the rollback to the customer's own pre-restore copy also failed. Wired in
// cmd/controller/main.go. Same seam shape as unitNotify above and for the same reason: the
// manager must not import the notifier.
//
// OPERATOR-TIER, and this is the whole reason it is a seam rather than a direct call. A held app
// must NOT reach `NotifyBackupFailed` — that type is customer-enabled by default
// (`settings.DefaultEnabledEvents`) and carries the Hungarian "A biztonsági mentés sikertelen!",
// which would alarm a customer about an app we are DELIBERATELY holding. That is R-171's defect
// one path over, and it is the same distinction `ErrStartRefused` exists to keep.
restoreHoldNotify func(stack string, replayErr, rollbackErr error)
// unitSpaceFn (R-165 / B2), if set, replaces the real statfs behind the capture floor so a test
// can state a filesystem's occupancy as an input. Nil in production → `unitTargetSpace`.
unitSpaceFn func(stackName string) *UnitSpace
@@ -137,6 +149,13 @@ type Manager struct {
discoverDBs func(ctx context.Context) ([]DiscoveredDB, error)
importDBDump func(ctx context.Context, db DiscoveredDB, dumpPath string) error
// rollbackImport (R-379) — the ROLLBACK's ImportDump seam. Deliberately SEPARATE from
// importDBDump above even though both default to ImportDump: the whole point of the rollback is
// what happens when the replay fails, so a test must be able to make the replay fail and the
// rollback succeed (and the reverse). One shared seam cannot express that, and a test that
// cannot express the case cannot pin it.
rollbackImport func(ctx context.Context, db DiscoveredDB, dumpPath string) error
// F3 volume-dump seam — overridable in tests so runVolumeDumps' gating (protected / volume-less /
// disconnected) can be unit-tested without Docker. Nil → the real DumpAppVolumesSafe.
dumpVolumesSafe func(stackName string) error