v0.218.0: attribute a DB container by its compose project, and replay volumes on the off-site restore
gates / gates (push) Successful in 11s

R-355 (first, because it is the only one where data can be lost for good). paperless-ngx's
PostgreSQL was dumped into backups/primary/paperless/db-dumps/ — a directory for a stack that
does not exist, on the system drive — while the app's own unit recorded db_dumps: null. The
same misattribution reached writeSafetyDump, so a destructive restore of that app took NO undo
copy and the fail-closed refusal was never reached. Fixed by reading the compose project label,
which is the stack name by construction (compose runs with cmd.Dir set to the stack dir and no
-p). The old derivation stays as the fallback and an unresolvable attribution is now loud.
Catalogue sweep, proven able to convict: one affected app of 53. The fix is in the controller,
not the catalogue.

R-354. ReconstituteFromOffsite skipped every unit placement and the volume archives live inside
the unit, so the off-site restore had no volume leg at all — proven live with planted files:
calibre-web's 1,422,848-byte config archive was in the unit, the snapshot and the checking
folder, and the restore reported success without it. For the 40 of 53 apps that declare no data
drive that archive is the whole dataset. restoreDockerVolumesFrom is the local path's own replay
with an explicit directory: ONE implementation, two callers. Volumes replay before the database
and inside the stopped window. VolumesReplayed reaches the message.

The comment beside the skip was half false and is corrected; the half that still holds — the
live unit is the local path's source — is named, and scenario D fingerprints the whole live unit
across the operation.

Seven red-proofs, each asserted applied and reverted. Two found defects in the tests, not the
code: scenario D passed with the unit guard removed because the fingerprint had been narrowed
and was blind to the unit root.
This commit is contained in:
2026-08-22 09:43:22 +02:00
parent f94543ee5c
commit 5ce3a44645
10 changed files with 871 additions and 24 deletions
+53 -5
View File
@@ -91,7 +91,17 @@ func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool, know
if debug {
logger.Printf("[DEBUG] DiscoverDatabases: running docker ps to find database containers")
}
cmd := exec.CommandContext(ctx, "docker", "ps", "--format", "{{.ID}}\t{{.Names}}\t{{.Image}}", "--filter", "status=running")
// R-355: the compose PROJECT label is asked for, because it is the stack name BY CONSTRUCTION and
// the container name is only a guess at it. The controller runs compose with `cmd.Dir` set to
// `/opt/docker/stacks/<stack>` and never passes `-p` (stacks/manager.go:1218-1226), so compose
// derives the project from that directory — i.e. from the stack name itself.
//
// The label sits in the MIDDLE of the format on purpose: `strings.TrimSpace` is applied to the
// whole `docker ps` output, so a trailing empty field on the LAST line would be eaten and that
// row would arrive one column short. Image is never empty, so ending on it keeps every row the
// same width.
cmd := exec.CommandContext(ctx, "docker", "ps", "--format",
"{{.ID}}\t{{.Names}}\t{{.Label \"com.docker.compose.project\"}}\t{{.Image}}", "--filter", "status=running")
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("docker ps failed: %w", err)
@@ -108,12 +118,12 @@ func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool, know
if line == "" {
continue
}
parts := strings.SplitN(line, "\t", 3)
if len(parts) < 3 {
parts := strings.SplitN(line, "\t", 4)
if len(parts) < 4 {
continue
}
id, name, image := parts[0], parts[1], strings.ToLower(parts[2])
id, name, project, image := parts[0], parts[1], strings.TrimSpace(parts[2]), strings.ToLower(parts[3])
// R-47: the same predicate that DBServiceNames applies to compose `image:` values, so a dump
// that exists is always attributable to a startable service (see dbservices.go).
@@ -134,7 +144,7 @@ func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool, know
ContainerID: id,
ContainerName: name,
DBType: dbType,
StackName: deriveStackName(name, known),
StackName: resolveStackName(name, project, known, logger, debug),
}
// Get env vars from container
@@ -767,6 +777,44 @@ func getMariaDBPassword(ctx context.Context, containerID string) string {
// the stack list is empty/unavailable, so nothing regresses).
//
// A nil/empty `known` map = the legacy fast path (pure suffix-strip).
// resolveStackName attributes a running DB container to the stack that owns it.
//
// R-355, and the reason this exists rather than more string-munging: the container name is a GUESS at
// the stack name and the compose project label is the ANSWER. `paperless-ngx` runs a container called
// `paperless-postgres`; no amount of suffix-stripping or prefix-matching can bridge that, and
// deriveStackName's last line returns its unresolved candidate as though it had resolved it. The dump
// then went to `backups/primary/paperless/db-dumps/` — a directory for a stack that does not exist, on
// the system drive — while the app's own recovery unit recorded `db_dumps: null`. Nothing collected it,
// nothing off-sited it, nothing restored it, and because writeSafetyDump filters on this same value the
// destructive restore took NO undo copy and told the customer the app had no database. Measured live
// 2026-08-21: 284 617 bytes, 72 tables, valid, and unreachable.
//
// The label is authoritative BY CONSTRUCTION — see the note on the `docker ps` format above — so it is
// preferred whenever it names a stack we know. It is not trusted blindly: a project label naming an
// unknown stack would write into an unknown app's directory, which is the very fault being fixed.
//
// The fallback is unchanged, so every container whose name already resolves keeps its exact previous
// attribution (pinned by TestResolveStackName_UnaffectedAppsAreUnchanged).
//
// When NEITHER route resolves to a known stack we still return the old candidate — refusing here would
// change behaviour for any container legitimately relying on the legacy path — but we say so loudly,
// because the silent version of this line is what hid R-355 for the life of the feature.
func resolveStackName(containerName, composeProject string, known map[string]bool, logger *log.Logger, debug bool) string {
if composeProject != "" && (len(known) == 0 || known[composeProject]) {
if debug && composeProject != deriveStackName(containerName, known) {
logger.Printf("[DEBUG] DiscoverDatabases: %s → stack %q from the compose project label (the container name would have given %q)",
containerName, composeProject, deriveStackName(containerName, known))
}
return composeProject
}
fallback := deriveStackName(containerName, known)
if len(known) > 0 && !known[fallback] {
logger.Printf("[WARN] [backup] DB container %q could not be attributed to any deployed stack (compose project %q, name gives %q) — its dump will be written under %q, which no recovery unit reads. This is the R-355 shape; investigate before trusting that app's backup.",
containerName, composeProject, fallback, fallback)
}
return fallback
}
func deriveStackName(containerName string, known map[string]bool) string {
candidate := suffixStripStackName(containerName)