D5: an app restore works from the drive alone (v0.188.0)

The recovery unit on the customer's drive now carries the PORTABLE secret
class, so Tier-1/Tier-2 restore no longer depends on the whole-guest tier.
A customer needs the drive and nothing else.

Part 0's rulings overturned the brief's recommendation, on evidence:
- the data_key flag is untrustworthy (4+ encryption keys the catalog itself
  labels as such are unflagged) -> R-127
- a DB password is not resettable in practice: POSTGRES_PASSWORD is ignored
  once PGDATA is non-empty, so a regenerated value leaves the app unable to
  authenticate against its own restored rows while the dump replay still
  reports success (proven on a throwaway postgres:16-alpine)

Ruling (operator): type:secret travels, type:password never does, minus the
nonPortableSecrets code register. Plaintext -- withholding the internet-
reachable class is what licenses that, and the two are coupled.

Precedence: the UNIT WINS over the guest -- the unit's secrets were captured
in the same run as the dumps beside them, so they match the data being
restored. The fail-closed data-key gate is unchanged.

Secret values are never logged; the manifest records NAMES only.
This commit is contained in:
2026-07-30 16:33:06 +02:00
parent 2f27a363d5
commit 4ed938cce4
12 changed files with 818 additions and 151 deletions
+84
View File
@@ -1,5 +1,89 @@
## Changelog
### v0.188.0 — D5: an app restore works from the drive alone (2026-07-30) — MinAgent 0.113.0 (unchanged)
**Tier-1/Tier-2 no longer depend on the whole-guest tier.** Until now the recovery unit on the
customer's drive was secret-free, which made the two-lane split *look* independent while it was not: the
app's files sat safely on the drive and could not be brought back, because the secrets that make them
readable went down with the guest. After this, restoring an app needs **the drive and nothing else**
not the server, not the operator, not the offsite copy.
**Part 0 first: the brief's own recommendation did not survive the test it asked for.** It proposed that
only `data_key`-flagged secrets travel. Two findings overturned it, both evidenced before any code:
1. **The `data_key` flag is not a trustworthy classification.** Only 5 fields across 4 apps carry it,
yet the catalog's own Hungarian labels contradict the flag elsewhere: `n8n/N8N_ENCRYPTION_KEY`
(„Titkosítási kulcs"), `wanderer/POCKETBASE_ENCRYPTION_KEY` („Adatbázis titkosítási kulcs"),
`calcom/CALENDSO_ENCRYPTION_KEY`, `bookstack/APP_KEY` — same label as `adventurelog/SECRET_KEY`,
opposite flag. Travelling "only data keys" would omit real data keys, and the fail-closed gate would
not fire for them → a restore that succeeds onto unreadable data. Filed **R-127**.
2. **A DB password is not resettable in practice — proven, not argued.** `DumpAppVolumes` dumps every
compose named volume with no DB exclusion, so `immich_postgres_data.tar` is captured and restored.
Probe on `postgres:16-alpine` (seed → drop container, keep volume → redeploy with a regenerated
password): the **replay succeeded** (`docker exec psql`, no password — verbatim what `ImportDump`
does, and the image's local socket is `trust`), the **app path failed** over the compose network
(`FATAL: password authentication failed`), and the **old** password still worked — `POSTGRES_PASSWORD`
is ignored once PGDATA is non-empty. So the restore reports success, the dump replays, the rows are
there, and the application cannot reach them. 18 DB/root-password fields affected. MariaDB fails
louder: `getMariaDBPassword` reads the regenerated value from container env against a datadir holding
the old hash, so the replay itself gets Access denied (`nextcloud`, `romm`).
**The rulings (operator, 2026-07-30).** `type: secret` travels; `type: password` never does; minus a
code register. Plaintext, as the data already is.
- **TRAVELS (45 fields):** the 5 declared data keys, 18 DB/root passwords, 22 internal signing/encryption
secrets. Every one decrypts data on the SAME drive or authenticates to a container on an internal
compose network with no external listener — possessing it adds nothing to possessing the drive, which
is exactly D2's argument for plaintext DATA.
- **WITHHELD (8):** the 7 `type: password` admin/UI logins + `vaultwarden/ADMIN_TOKEN` via the
`nonPortableSecrets` register. These authenticate against published services, so their reach is NOT
bounded by the drive. **Excluding this class is what licenses the plaintext ruling; the two are coupled
and must not be relaxed independently.** The register is code, not a catalog flag — a boundary a
catalog push can silently move is not a boundary (cf. R-97a).
**What a customer must possess to complete a Tier-1/2 restore after this change: the drive.**
**Implementation** — one place per side, no parallel path. `stacks.PortableSecretEnvVars` is the whole
boundary; `GetStackRecoveryInfo` decrypts the portable class through the SAME `LoadAppConfigDecrypted`
the restore side uses; `buildUnitAppYaml` (was `buildStrippedAppYaml`) writes it at **0600** and names
the withheld class in the header so an operator can see WHY a credential is absent rather than suspect a
capture bug; `readUnitEnv` splits it back using the **manifest's** portable names, never guessed from key
names. `reconcileRestoreSecrets` stays a pure function — the new source arrives as an **argument**.
Manifest → **schema 2** + `portable_secret_env_vars` (NAMES only; the manifest is 0644).
**Precedence: the UNIT WINS.** Not "newest wins". The unit's secrets are captured in the same run as the
dumps beside them (`runVolumeDumps``captureAllRecoveryUnits`), so the unit's value is the one that
matches the data about to be restored; the guest's is merely the most recent. A rotated data key does not
decrypt data encrypted with the old one, and a rotated DB password does not match the hash in the
restored data directory. Pinned in both directions — an undefined precedence between two sources of a
decryption key is a data-loss bug waiting for its first disagreement.
**The fail-closed gate is unchanged and still fail-closed:** a data key in NEITHER source refuses
outright. D5 makes it normally present; "normally" is not a reason to soften a gate.
**Three comments that asserted invariants D5 makes false were corrected, not left to read as settled**
(`CaptureRecoveryUnit` "NEVER writes a secret value", `RestoreFromRecoveryUnit` "no secret is read from
the unit", `appbackup/paths.go` + `appdata.go` "secret-free"), and the O4 WARN that claimed "stored data
is unaffected" for every non-data-key secret now says what is true — finding 2 disproves it for DB
passwords.
**Backward compatible.** A schema-1 unit carries no secrets and still restores from the guest; the next
capture rewrites it (the app.yaml checksum changes). No existing backup changes, no data moves, and the
escrow / whole-guest / offsite tiers are untouched in code — the offsite copy simply carries the secrets
inside the unit it already pushed, encrypted at rest under the customer-owned restic password.
**Tests** — `TestRestoreFromRecoveryUnitWithGuestAbsent` is D5's claim as a test rather than a
description; plus fail-closed-with-both-sources-absent, precedence both directions, the schema-1
no-regression case, `readUnitEnv` splitting, and the wrong-outcome check that the withheld class appears
NOWHERE in the unit. Fixtures come from a unit written by the **real** `CaptureRecoveryUnit`, so the two
sides meet at real bytes. Seam: `Manager.stackProvider` only. **Four red-proofs, each verified to have
landed:** drop the portable merge → the consequence test fails; neuter the gate → 4 failures; flip
precedence → the unit-wins test fails; widen the class to `type: password` → the boundary test fails.
**R-120's gate does not apply to this task** — it sits in `hub/internal/web/configs.go`
`handleSetArtifacts`, the golden-**vouch** form, and never runs on a controller image deploy. Re-baking
the golden is a follow-on so that FRESH installs get D5; it is not a prerequisite here.
### v0.187.0 — R-108: network storage may not host an app's data namespace (2026-07-30) — MinAgent 0.113.0 (unchanged)
**This is D5's precondition, and it is now met.** D5 moves app secrets into the local recovery unit so