D5: an app restore works from the drive alone (v0.188.0)

The recovery unit on the customer's drive now carries the PORTABLE secret
class, so Tier-1/Tier-2 restore no longer depends on the whole-guest tier.
A customer needs the drive and nothing else.

Part 0's rulings overturned the brief's recommendation, on evidence:
- the data_key flag is untrustworthy (4+ encryption keys the catalog itself
  labels as such are unflagged) -> R-127
- a DB password is not resettable in practice: POSTGRES_PASSWORD is ignored
  once PGDATA is non-empty, so a regenerated value leaves the app unable to
  authenticate against its own restored rows while the dump replay still
  reports success (proven on a throwaway postgres:16-alpine)

Ruling (operator): type:secret travels, type:password never does, minus the
nonPortableSecrets code register. Plaintext -- withholding the internet-
reachable class is what licenses that, and the two are coupled.

Precedence: the UNIT WINS over the guest -- the unit's secrets were captured
in the same run as the dumps beside them, so they match the data being
restored. The fail-closed data-key gate is unchanged.

Secret values are never logged; the manifest records NAMES only.
This commit is contained in:
2026-07-30 16:33:06 +02:00
parent 2f27a363d5
commit 4ed938cce4
12 changed files with 818 additions and 151 deletions
+47 -19
View File
@@ -808,36 +808,64 @@ Path computation is centralized in `backup/paths.go` via the `FelhomDataDir = "f
> `AppSecondaryRsyncPath`, `SecondaryInfraPath`) describe the pre-strip layout — restic/cross-drive was
> removed in slice 8C. This section is rewritten when Tier 2 (Phase 3) lands.
#### Per-app recovery unit (Phase 2, v0.53.x) — SECRET-FREE
#### Per-app recovery unit (Phase 2, v0.53.x; secret model rewritten by **D5**, v0.188.0)
Each app's `backups/primary/<app>/` is a self-contained, recreatable **recovery unit**:
```
backups/primary/<app>/
├── compose/ docker-compose.yml + .felhom.yml + a SECRET-STRIPPED app.yaml
├── compose/ docker-compose.yml + .felhom.yml + app.yaml (0600 — CARRIES the portable secrets)
├── db-dumps/ app-consistent DB dump(s)
├── volume-dumps/ named-volume tars
└── manifest.json image pins, secret env-var NAMES, data_key names, checksums, secret_source
└── manifest.json image pins, secret NAMES, data_key names, portable NAMES, checksums, secret_source
```
- **Secret-free by design.** The unit stores **no secret value, no data-encrypting key, and not the
Docker image** — only the pinned image tag(s) (re-pulled on restore) and the *names* of the secret /
`data_key` env vars. Rationale: app.yaml + the encryption key live on the guest rootfs → already in
the PBS whole-guest snapshot, and the hub is deliberately zero-knowledge. Restore recovers the
original secrets from the guest's own app.yaml (live, or via PBS); for a `data_key` app it
**fails closed** (refuse + warn) if the key can't be recovered — data-keys are NEVER generated.
**Resettable secrets (O4, v0.99.0):** an unrecoverable resettable secret (DB password etc.) gets a
**generated replacement** from its catalog `generate` spec (`stacks.GenerateSecretForField` via the
`backup.SetSecretGenerator` seam) instead of redeploying blank (which failed compose-up); the new
value persists encrypted through the normal `RecreateStackDefinitionFromUnit``SaveAppConfig` path. Fields
with no `generate` spec still proceed with a loud "may fail to start" WARN. Residual case: a restored
volume tar carrying the OLD internal credential hash may still need a manual in-DB reset.
- **The secret split (D5, schema 2, operator ruling 2026-07-30).** The unit was secret-free until
v0.188.0, and that made "restore from the drive alone" false: the fast, local, customer-doable
Tier-1/2 restore secretly depended on the slow, operator-driven whole-guest restore, because a
data-encrypting key or a DB password absent from the guest cannot be regenerated without leaving the
restored data unreachable. **Tier-1/2 now needs the drive and nothing else.** What travels is decided
in ONE place, `stacks.PortableSecretEnvVars`:
- **TRAVELS — every `type: secret` field** (45 of 53 across the catalog): the declared `data_key`s,
the 18 DB/root passwords, and the internal signing/encryption secrets. Each of these either
decrypts data sitting on the SAME drive or authenticates to a container on an internal compose
network with no external listener, so possessing it adds nothing to possessing the drive — which is
exactly D2's argument for keeping the DATA plaintext. Written into the unit's app.yaml at **0600**,
plaintext, like the data beside it.
- **WITHHELD — every `type: password` field** (7 admin/UI logins) **plus the `nonPortableSecrets`
register** (`vaultwarden/ADMIN_TOKEN`, whose `/admin` panel is on the app's public web port).
These authenticate against published services, so their blast radius is NOT bounded by the drive.
They stay in the guest and are regenerated on restore (O4). **Excluding this class is what licenses
the plaintext ruling — the two are coupled and must not be relaxed independently.**
- The register is **code, not a catalog flag**, deliberately: a security boundary a catalog push can
silently move is not a boundary (cf. R-97a). Adding an app whose `type: secret` field gates an
internet-reachable login means adding a row there.
- **Fail-closed is unchanged.** A `data_key` missing from **both** the unit and the guest still refuses
the restore outright (never generated). D5 makes the key normally present; "normally" is not a reason
to soften the gate.
- **Precedence: the UNIT WINS** over the guest when both hold a value. Not "newest wins" — the unit's
secrets are captured in the same run as the dumps beside them, so the unit's value is the one that
MATCHES THE DATA BEING RESTORED, while the guest's is merely the most recent. A rotated data key does
not decrypt data encrypted with the old one, and a rotated DB password does not match the hash inside
the restored data directory. Pinned in both directions.
- **Resettable secrets (O4, v0.99.0)** — now the rare path, since the portable class comes from the
unit. An unrecoverable withheld secret gets a **generated replacement** from its catalog `generate`
spec (`stacks.GenerateSecretForField` via the `backup.SetSecretGenerator` seam) rather than redeploying
blank; the value persists encrypted through `RecreateStackDefinitionFromUnit``SaveAppConfig`.
⚠️ **R-127:** a regenerated **database** password is NOT harmless — `POSTGRES_PASSWORD` is ignored once
PGDATA is non-empty, so the restored data dir keeps the old role hash and the app cannot authenticate
against its own rows, while the dump replay (local trust socket) still reports success. The WARN says so.
- Helpers: `RecoveryUnitPath` / `RecoveryUnitComposePath` / `RecoveryUnitManifestPath`
(`internal/appbackup/paths.go`). Capture: `Manager.CaptureRecoveryUnit` (`internal/backup/recovery_unit.go`),
run from the daily DB dump and the periodic `RefreshCache` (idempotent checksum-skip). The non-secret
env comes from `StackDataProvider.GetStackRecoveryInfo` (excludes secret-named + encrypted values, so
the capture never touches a secret). `data_key` fields are marked in `.felhom.yml`
(`DeployField.DataKey`).
run from the daily DB dump and the periodic `RefreshCache` (idempotent checksum-skip); the split itself
is in `buildUnitAppYaml`. The env + portable values come from `StackDataProvider.GetStackRecoveryInfo`,
which keeps `NonSecretEnv` and the secret set disjoint by construction. `data_key` fields are marked in
`.felhom.yml` (`DeployField.DataKey`).
- **A schema-1 (pre-D5) unit carries no secrets** and still restores from the guest — the restore
degrades rather than failing, and the next capture rewrites the unit (the app.yaml checksum changes).
- **Consequence for the other tiers:** the unit is copied by Tier 2 (another customer drive, plaintext,
same reasoning) and pushed offsite by restic (`offbox.go` — encrypted at rest under the customer-owned
repo password). Neither tier's code changed; the secrets simply travel with the unit they already carried.
- **Restore replays the DB dump (F17, v0.61.0; re-sequenced v0.153.0, R-47).** `RestoreFromRecoveryUnit`
(and the `RestoreApp` fallback) stops the app → restores named-volume tars → recreates the compose
definition and persists the recovered env (`RecreateStackDefinitionFromUnit`**starts nothing**)