D5: an app restore works from the drive alone (v0.188.0)
The recovery unit on the customer's drive now carries the PORTABLE secret class, so Tier-1/Tier-2 restore no longer depends on the whole-guest tier. A customer needs the drive and nothing else. Part 0's rulings overturned the brief's recommendation, on evidence: - the data_key flag is untrustworthy (4+ encryption keys the catalog itself labels as such are unflagged) -> R-127 - a DB password is not resettable in practice: POSTGRES_PASSWORD is ignored once PGDATA is non-empty, so a regenerated value leaves the app unable to authenticate against its own restored rows while the dump replay still reports success (proven on a throwaway postgres:16-alpine) Ruling (operator): type:secret travels, type:password never does, minus the nonPortableSecrets code register. Plaintext -- withholding the internet- reachable class is what licenses that, and the two are coupled. Precedence: the UNIT WINS over the guest -- the unit's secrets were captured in the same run as the dumps beside them, so they match the data being restored. The fail-closed data-key gate is unchanged. Secret values are never logged; the manifest records NAMES only.
This commit is contained in:
@@ -1,5 +1,89 @@
|
|||||||
## Changelog
|
## Changelog
|
||||||
|
|
||||||
|
### v0.188.0 — D5: an app restore works from the drive alone (2026-07-30) — MinAgent 0.113.0 (unchanged)
|
||||||
|
|
||||||
|
**Tier-1/Tier-2 no longer depend on the whole-guest tier.** Until now the recovery unit on the
|
||||||
|
customer's drive was secret-free, which made the two-lane split *look* independent while it was not: the
|
||||||
|
app's files sat safely on the drive and could not be brought back, because the secrets that make them
|
||||||
|
readable went down with the guest. After this, restoring an app needs **the drive and nothing else** —
|
||||||
|
not the server, not the operator, not the offsite copy.
|
||||||
|
|
||||||
|
**Part 0 first: the brief's own recommendation did not survive the test it asked for.** It proposed that
|
||||||
|
only `data_key`-flagged secrets travel. Two findings overturned it, both evidenced before any code:
|
||||||
|
|
||||||
|
1. **The `data_key` flag is not a trustworthy classification.** Only 5 fields across 4 apps carry it,
|
||||||
|
yet the catalog's own Hungarian labels contradict the flag elsewhere: `n8n/N8N_ENCRYPTION_KEY`
|
||||||
|
(„Titkosítási kulcs"), `wanderer/POCKETBASE_ENCRYPTION_KEY` („Adatbázis titkosítási kulcs"),
|
||||||
|
`calcom/CALENDSO_ENCRYPTION_KEY`, `bookstack/APP_KEY` — same label as `adventurelog/SECRET_KEY`,
|
||||||
|
opposite flag. Travelling "only data keys" would omit real data keys, and the fail-closed gate would
|
||||||
|
not fire for them → a restore that succeeds onto unreadable data. Filed **R-127**.
|
||||||
|
2. **A DB password is not resettable in practice — proven, not argued.** `DumpAppVolumes` dumps every
|
||||||
|
compose named volume with no DB exclusion, so `immich_postgres_data.tar` is captured and restored.
|
||||||
|
Probe on `postgres:16-alpine` (seed → drop container, keep volume → redeploy with a regenerated
|
||||||
|
password): the **replay succeeded** (`docker exec psql`, no password — verbatim what `ImportDump`
|
||||||
|
does, and the image's local socket is `trust`), the **app path failed** over the compose network
|
||||||
|
(`FATAL: password authentication failed`), and the **old** password still worked — `POSTGRES_PASSWORD`
|
||||||
|
is ignored once PGDATA is non-empty. So the restore reports success, the dump replays, the rows are
|
||||||
|
there, and the application cannot reach them. 18 DB/root-password fields affected. MariaDB fails
|
||||||
|
louder: `getMariaDBPassword` reads the regenerated value from container env against a datadir holding
|
||||||
|
the old hash, so the replay itself gets Access denied (`nextcloud`, `romm`).
|
||||||
|
|
||||||
|
**The rulings (operator, 2026-07-30).** `type: secret` travels; `type: password` never does; minus a
|
||||||
|
code register. Plaintext, as the data already is.
|
||||||
|
|
||||||
|
- **TRAVELS (45 fields):** the 5 declared data keys, 18 DB/root passwords, 22 internal signing/encryption
|
||||||
|
secrets. Every one decrypts data on the SAME drive or authenticates to a container on an internal
|
||||||
|
compose network with no external listener — possessing it adds nothing to possessing the drive, which
|
||||||
|
is exactly D2's argument for plaintext DATA.
|
||||||
|
- **WITHHELD (8):** the 7 `type: password` admin/UI logins + `vaultwarden/ADMIN_TOKEN` via the
|
||||||
|
`nonPortableSecrets` register. These authenticate against published services, so their reach is NOT
|
||||||
|
bounded by the drive. **Excluding this class is what licenses the plaintext ruling; the two are coupled
|
||||||
|
and must not be relaxed independently.** The register is code, not a catalog flag — a boundary a
|
||||||
|
catalog push can silently move is not a boundary (cf. R-97a).
|
||||||
|
|
||||||
|
**What a customer must possess to complete a Tier-1/2 restore after this change: the drive.**
|
||||||
|
|
||||||
|
**Implementation** — one place per side, no parallel path. `stacks.PortableSecretEnvVars` is the whole
|
||||||
|
boundary; `GetStackRecoveryInfo` decrypts the portable class through the SAME `LoadAppConfigDecrypted`
|
||||||
|
the restore side uses; `buildUnitAppYaml` (was `buildStrippedAppYaml`) writes it at **0600** and names
|
||||||
|
the withheld class in the header so an operator can see WHY a credential is absent rather than suspect a
|
||||||
|
capture bug; `readUnitEnv` splits it back using the **manifest's** portable names, never guessed from key
|
||||||
|
names. `reconcileRestoreSecrets` stays a pure function — the new source arrives as an **argument**.
|
||||||
|
Manifest → **schema 2** + `portable_secret_env_vars` (NAMES only; the manifest is 0644).
|
||||||
|
|
||||||
|
**Precedence: the UNIT WINS.** Not "newest wins". The unit's secrets are captured in the same run as the
|
||||||
|
dumps beside them (`runVolumeDumps` → `captureAllRecoveryUnits`), so the unit's value is the one that
|
||||||
|
matches the data about to be restored; the guest's is merely the most recent. A rotated data key does not
|
||||||
|
decrypt data encrypted with the old one, and a rotated DB password does not match the hash in the
|
||||||
|
restored data directory. Pinned in both directions — an undefined precedence between two sources of a
|
||||||
|
decryption key is a data-loss bug waiting for its first disagreement.
|
||||||
|
|
||||||
|
**The fail-closed gate is unchanged and still fail-closed:** a data key in NEITHER source refuses
|
||||||
|
outright. D5 makes it normally present; "normally" is not a reason to soften a gate.
|
||||||
|
|
||||||
|
**Three comments that asserted invariants D5 makes false were corrected, not left to read as settled**
|
||||||
|
(`CaptureRecoveryUnit` "NEVER writes a secret value", `RestoreFromRecoveryUnit` "no secret is read from
|
||||||
|
the unit", `appbackup/paths.go` + `appdata.go` "secret-free"), and the O4 WARN that claimed "stored data
|
||||||
|
is unaffected" for every non-data-key secret now says what is true — finding 2 disproves it for DB
|
||||||
|
passwords.
|
||||||
|
|
||||||
|
**Backward compatible.** A schema-1 unit carries no secrets and still restores from the guest; the next
|
||||||
|
capture rewrites it (the app.yaml checksum changes). No existing backup changes, no data moves, and the
|
||||||
|
escrow / whole-guest / offsite tiers are untouched in code — the offsite copy simply carries the secrets
|
||||||
|
inside the unit it already pushed, encrypted at rest under the customer-owned restic password.
|
||||||
|
|
||||||
|
**Tests** — `TestRestoreFromRecoveryUnitWithGuestAbsent` is D5's claim as a test rather than a
|
||||||
|
description; plus fail-closed-with-both-sources-absent, precedence both directions, the schema-1
|
||||||
|
no-regression case, `readUnitEnv` splitting, and the wrong-outcome check that the withheld class appears
|
||||||
|
NOWHERE in the unit. Fixtures come from a unit written by the **real** `CaptureRecoveryUnit`, so the two
|
||||||
|
sides meet at real bytes. Seam: `Manager.stackProvider` only. **Four red-proofs, each verified to have
|
||||||
|
landed:** drop the portable merge → the consequence test fails; neuter the gate → 4 failures; flip
|
||||||
|
precedence → the unit-wins test fails; widen the class to `type: password` → the boundary test fails.
|
||||||
|
|
||||||
|
**R-120's gate does not apply to this task** — it sits in `hub/internal/web/configs.go`
|
||||||
|
`handleSetArtifacts`, the golden-**vouch** form, and never runs on a controller image deploy. Re-baking
|
||||||
|
the golden is a follow-on so that FRESH installs get D5; it is not a prerequisite here.
|
||||||
|
|
||||||
### v0.187.0 — R-108: network storage may not host an app's data namespace (2026-07-30) — MinAgent 0.113.0 (unchanged)
|
### v0.187.0 — R-108: network storage may not host an app's data namespace (2026-07-30) — MinAgent 0.113.0 (unchanged)
|
||||||
|
|
||||||
**This is D5's precondition, and it is now met.** D5 moves app secrets into the local recovery unit so
|
**This is D5's precondition, and it is now met.** D5 moves app secrets into the local recovery unit so
|
||||||
|
|||||||
@@ -111,7 +111,9 @@
|
|||||||
| `SaveAppConfig` / `LoadAppConfigDecrypted` | controller/internal/stacks/deploy.go | `(stackDir, cfg, encKey, sensitiveVars)` | app.yaml persistence | Encrypts only `SensitiveEnvVars(meta)`; never write app.yaml directly |
|
| `SaveAppConfig` / `LoadAppConfigDecrypted` | controller/internal/stacks/deploy.go | `(stackDir, cfg, encKey, sensitiveVars)` | app.yaml persistence | Encrypts only `SensitiveEnvVars(meta)`; never write app.yaml directly |
|
||||||
| `generateValue` / `randomAlphanumeric` | controller/internal/stacks/deploy.go | `(spec "password:N\|hex:N\|base64key:N\|static:v")` | Auto-generated secrets | crypto/rand-backed; reuse the spec grammar |
|
| `generateValue` / `randomAlphanumeric` | controller/internal/stacks/deploy.go | `(spec "password:N\|hex:N\|base64key:N\|static:v")` | Auto-generated secrets | crypto/rand-backed; reuse the spec grammar |
|
||||||
| `Manager.GenerateSecretForField` | controller/internal/stacks/deploy.go | `(stackName, envVar) (string, bool)` | Replacement value for a RESETTABLE secret from its catalog `generate` spec (O4 restore path via `backup.SetSecretGenerator`) | REFUSES `data_key` fields, spec-less and non-secret fields; never log the value |
|
| `Manager.GenerateSecretForField` | controller/internal/stacks/deploy.go | `(stackName, envVar) (string, bool)` | Replacement value for a RESETTABLE secret from its catalog `generate` spec (O4 restore path via `backup.SetSecretGenerator`) | REFUSES `data_key` fields, spec-less and non-secret fields; never log the value |
|
||||||
| `reconcileRestoreSecrets` | controller/internal/backup/restore_unit.go | `(nonSecretEnv, recoveredSecrets, secretNames, dataKeyNames)` | Recovery-unit restore env merge | Units are secret-FREE by design; secrets come from live app.yaml |
|
| `reconcileRestoreSecrets` | controller/internal/backup/restore_unit.go | `(nonSecretEnv, unitSecrets, guestSecrets, secretNames, dataKeyNames)` | Recovery-unit restore env merge | **Precedence: UNIT WINS over guest** (the unit's secrets match the data being restored; the guest's are merely newest). Pure — new sources arrive as ARGUMENTS. Fail-closed data-key gate lives here |
|
||||||
|
| `stacks.PortableSecretEnvVars` | controller/internal/stacks/deploy.go | `(meta) []string` | **THE D5 secret boundary**: which secrets may travel on a customer drive | `type: secret` travels, `type: password` NEVER, minus the `nonPortableSecrets` code register. Withholding the password class is what licenses plaintext — do not relax one without the other |
|
||||||
|
| `buildUnitAppYaml` / `readUnitEnv` | controller/internal/backup/{recovery_unit,restore_unit}.go | `(info) []byte` / `(path, portableNames)` | The ONE place the unit's app.yaml is written / split back | Split is driven by the MANIFEST's portable names, never guessed from key names; write 0600; empty `portableNames` = schema-1 unit ⇒ everything is plain config |
|
||||||
| `EncryptFile` / `DecryptFile` / `IsEncryptedFAB` | controller/internal/appexport/crypto.go | password-based file crypto | .fab export bundles | scrypt-derived AES+HMAC keys |
|
| `EncryptFile` / `DecryptFile` / `IsEncryptedFAB` | controller/internal/appexport/crypto.go | password-based file crypto | .fab export bundles | scrypt-derived AES+HMAC keys |
|
||||||
| `maskRepoURL` | controller/internal/sync/sync.go | `(url) string` | Logging git URLs | Strips embedded credentials |
|
| `maskRepoURL` | controller/internal/sync/sync.go | `(url) string` | Logging git URLs | Strips embedded credentials |
|
||||||
| `metrics.RedactLine` | controller/internal/metrics/redact.go | `(s string) string` | ANY log line shipped off-box (issue context, log tails) | Masks password/passwd/secret/token/api-key/authorization/bearer values + 64-hex; apply BEFORE the line leaves the box — controller-side redaction is authoritative |
|
| `metrics.RedactLine` | controller/internal/metrics/redact.go | `(s string) string` | ANY log line shipped off-box (issue context, log tails) | Masks password/passwd/secret/token/api-key/authorization/bearer values + 64-hex; apply BEFORE the line leaves the box — controller-side redaction is authoritative |
|
||||||
|
|||||||
+47
-19
@@ -808,36 +808,64 @@ Path computation is centralized in `backup/paths.go` via the `FelhomDataDir = "f
|
|||||||
> `AppSecondaryRsyncPath`, `SecondaryInfraPath`) describe the pre-strip layout — restic/cross-drive was
|
> `AppSecondaryRsyncPath`, `SecondaryInfraPath`) describe the pre-strip layout — restic/cross-drive was
|
||||||
> removed in slice 8C. This section is rewritten when Tier 2 (Phase 3) lands.
|
> removed in slice 8C. This section is rewritten when Tier 2 (Phase 3) lands.
|
||||||
|
|
||||||
#### Per-app recovery unit (Phase 2, v0.53.x) — SECRET-FREE
|
#### Per-app recovery unit (Phase 2, v0.53.x; secret model rewritten by **D5**, v0.188.0)
|
||||||
|
|
||||||
Each app's `backups/primary/<app>/` is a self-contained, recreatable **recovery unit**:
|
Each app's `backups/primary/<app>/` is a self-contained, recreatable **recovery unit**:
|
||||||
|
|
||||||
```
|
```
|
||||||
backups/primary/<app>/
|
backups/primary/<app>/
|
||||||
├── compose/ docker-compose.yml + .felhom.yml + a SECRET-STRIPPED app.yaml
|
├── compose/ docker-compose.yml + .felhom.yml + app.yaml (0600 — CARRIES the portable secrets)
|
||||||
├── db-dumps/ app-consistent DB dump(s)
|
├── db-dumps/ app-consistent DB dump(s)
|
||||||
├── volume-dumps/ named-volume tars
|
├── volume-dumps/ named-volume tars
|
||||||
└── manifest.json image pins, secret env-var NAMES, data_key names, checksums, secret_source
|
└── manifest.json image pins, secret NAMES, data_key names, portable NAMES, checksums, secret_source
|
||||||
```
|
```
|
||||||
|
|
||||||
- **Secret-free by design.** The unit stores **no secret value, no data-encrypting key, and not the
|
- **The secret split (D5, schema 2, operator ruling 2026-07-30).** The unit was secret-free until
|
||||||
Docker image** — only the pinned image tag(s) (re-pulled on restore) and the *names* of the secret /
|
v0.188.0, and that made "restore from the drive alone" false: the fast, local, customer-doable
|
||||||
`data_key` env vars. Rationale: app.yaml + the encryption key live on the guest rootfs → already in
|
Tier-1/2 restore secretly depended on the slow, operator-driven whole-guest restore, because a
|
||||||
the PBS whole-guest snapshot, and the hub is deliberately zero-knowledge. Restore recovers the
|
data-encrypting key or a DB password absent from the guest cannot be regenerated without leaving the
|
||||||
original secrets from the guest's own app.yaml (live, or via PBS); for a `data_key` app it
|
restored data unreachable. **Tier-1/2 now needs the drive and nothing else.** What travels is decided
|
||||||
**fails closed** (refuse + warn) if the key can't be recovered — data-keys are NEVER generated.
|
in ONE place, `stacks.PortableSecretEnvVars`:
|
||||||
**Resettable secrets (O4, v0.99.0):** an unrecoverable resettable secret (DB password etc.) gets a
|
- **TRAVELS — every `type: secret` field** (45 of 53 across the catalog): the declared `data_key`s,
|
||||||
**generated replacement** from its catalog `generate` spec (`stacks.GenerateSecretForField` via the
|
the 18 DB/root passwords, and the internal signing/encryption secrets. Each of these either
|
||||||
`backup.SetSecretGenerator` seam) instead of redeploying blank (which failed compose-up); the new
|
decrypts data sitting on the SAME drive or authenticates to a container on an internal compose
|
||||||
value persists encrypted through the normal `RecreateStackDefinitionFromUnit` → `SaveAppConfig` path. Fields
|
network with no external listener, so possessing it adds nothing to possessing the drive — which is
|
||||||
with no `generate` spec still proceed with a loud "may fail to start" WARN. Residual case: a restored
|
exactly D2's argument for keeping the DATA plaintext. Written into the unit's app.yaml at **0600**,
|
||||||
volume tar carrying the OLD internal credential hash may still need a manual in-DB reset.
|
plaintext, like the data beside it.
|
||||||
|
- **WITHHELD — every `type: password` field** (7 admin/UI logins) **plus the `nonPortableSecrets`
|
||||||
|
register** (`vaultwarden/ADMIN_TOKEN`, whose `/admin` panel is on the app's public web port).
|
||||||
|
These authenticate against published services, so their blast radius is NOT bounded by the drive.
|
||||||
|
They stay in the guest and are regenerated on restore (O4). **Excluding this class is what licenses
|
||||||
|
the plaintext ruling — the two are coupled and must not be relaxed independently.**
|
||||||
|
- The register is **code, not a catalog flag**, deliberately: a security boundary a catalog push can
|
||||||
|
silently move is not a boundary (cf. R-97a). Adding an app whose `type: secret` field gates an
|
||||||
|
internet-reachable login means adding a row there.
|
||||||
|
- **Fail-closed is unchanged.** A `data_key` missing from **both** the unit and the guest still refuses
|
||||||
|
the restore outright (never generated). D5 makes the key normally present; "normally" is not a reason
|
||||||
|
to soften the gate.
|
||||||
|
- **Precedence: the UNIT WINS** over the guest when both hold a value. Not "newest wins" — the unit's
|
||||||
|
secrets are captured in the same run as the dumps beside them, so the unit's value is the one that
|
||||||
|
MATCHES THE DATA BEING RESTORED, while the guest's is merely the most recent. A rotated data key does
|
||||||
|
not decrypt data encrypted with the old one, and a rotated DB password does not match the hash inside
|
||||||
|
the restored data directory. Pinned in both directions.
|
||||||
|
- **Resettable secrets (O4, v0.99.0)** — now the rare path, since the portable class comes from the
|
||||||
|
unit. An unrecoverable withheld secret gets a **generated replacement** from its catalog `generate`
|
||||||
|
spec (`stacks.GenerateSecretForField` via the `backup.SetSecretGenerator` seam) rather than redeploying
|
||||||
|
blank; the value persists encrypted through `RecreateStackDefinitionFromUnit` → `SaveAppConfig`.
|
||||||
|
⚠️ **R-127:** a regenerated **database** password is NOT harmless — `POSTGRES_PASSWORD` is ignored once
|
||||||
|
PGDATA is non-empty, so the restored data dir keeps the old role hash and the app cannot authenticate
|
||||||
|
against its own rows, while the dump replay (local trust socket) still reports success. The WARN says so.
|
||||||
- Helpers: `RecoveryUnitPath` / `RecoveryUnitComposePath` / `RecoveryUnitManifestPath`
|
- Helpers: `RecoveryUnitPath` / `RecoveryUnitComposePath` / `RecoveryUnitManifestPath`
|
||||||
(`internal/appbackup/paths.go`). Capture: `Manager.CaptureRecoveryUnit` (`internal/backup/recovery_unit.go`),
|
(`internal/appbackup/paths.go`). Capture: `Manager.CaptureRecoveryUnit` (`internal/backup/recovery_unit.go`),
|
||||||
run from the daily DB dump and the periodic `RefreshCache` (idempotent checksum-skip). The non-secret
|
run from the daily DB dump and the periodic `RefreshCache` (idempotent checksum-skip); the split itself
|
||||||
env comes from `StackDataProvider.GetStackRecoveryInfo` (excludes secret-named + encrypted values, so
|
is in `buildUnitAppYaml`. The env + portable values come from `StackDataProvider.GetStackRecoveryInfo`,
|
||||||
the capture never touches a secret). `data_key` fields are marked in `.felhom.yml`
|
which keeps `NonSecretEnv` and the secret set disjoint by construction. `data_key` fields are marked in
|
||||||
(`DeployField.DataKey`).
|
`.felhom.yml` (`DeployField.DataKey`).
|
||||||
|
- **A schema-1 (pre-D5) unit carries no secrets** and still restores from the guest — the restore
|
||||||
|
degrades rather than failing, and the next capture rewrites the unit (the app.yaml checksum changes).
|
||||||
|
- **Consequence for the other tiers:** the unit is copied by Tier 2 (another customer drive, plaintext,
|
||||||
|
same reasoning) and pushed offsite by restic (`offbox.go` — encrypted at rest under the customer-owned
|
||||||
|
repo password). Neither tier's code changed; the secrets simply travel with the unit they already carried.
|
||||||
- **Restore replays the DB dump (F17, v0.61.0; re-sequenced v0.153.0, R-47).** `RestoreFromRecoveryUnit`
|
- **Restore replays the DB dump (F17, v0.61.0; re-sequenced v0.153.0, R-47).** `RestoreFromRecoveryUnit`
|
||||||
(and the `RestoreApp` fallback) stops the app → restores named-volume tars → recreates the compose
|
(and the `RestoreApp` fallback) stops the app → restores named-volume tars → recreates the compose
|
||||||
definition and persists the recovered env (`RecreateStackDefinitionFromUnit` — **starts nothing**)
|
definition and persists the recovered env (`RecreateStackDefinitionFromUnit` — **starts nothing**)
|
||||||
|
|||||||
@@ -1440,12 +1440,14 @@ func (a *stackAdapter) GetStackHDDPath(name string) string {
|
|||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
// GetStackRecoveryInfo gathers the SECRET-FREE inputs for an app's recovery unit (Phase 2): the
|
// GetStackRecoveryInfo gathers the inputs for an app's recovery unit: the stack dir, pinned image
|
||||||
// stack dir, pinned image tags, the non-secret env, and the NAMES of secret/data-key env vars.
|
// tags, the non-secret env, the NAMES of every secret/data-key env var, and — since D5 — the decrypted
|
||||||
// It deliberately does NOT decrypt or return any secret value — secret/password fields are stored
|
// VALUES of the PORTABLE secret class.
|
||||||
// encrypted in app.yaml, so excluding them (plus a defensive crypto.IsEncrypted guard) yields a
|
//
|
||||||
// plaintext, secret-free env. The actual secret values are recovered at restore time from the
|
// The non-secret env still excludes every named secret (plus a defensive crypto.IsEncrypted guard), so
|
||||||
// guest's own app.yaml (live, or via the PBS whole-guest snapshot), never from the unit.
|
// the two sets are disjoint by construction and a secret can only reach the unit by being in the
|
||||||
|
// portable class. The EXCLUDED class (`type: password`, and the nonPortableSecrets register) is
|
||||||
|
// name-only here and is recovered from the guest — or regenerated (O4) — exactly as before.
|
||||||
func (a *stackAdapter) GetStackRecoveryInfo(name string) (backup.RecoveryInfo, bool) {
|
func (a *stackAdapter) GetStackRecoveryInfo(name string) (backup.RecoveryInfo, bool) {
|
||||||
s, ok := a.mgr.GetStack(name)
|
s, ok := a.mgr.GetStack(name)
|
||||||
if !ok {
|
if !ok {
|
||||||
@@ -1482,13 +1484,31 @@ func (a *stackAdapter) GetStackRecoveryInfo(name string) (backup.RecoveryInfo, b
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// D5: decrypt the portable class so it can travel in the unit. Reuses the SAME decrypt path as the
|
||||||
|
// restore side (LoadAppConfigDecrypted), so there is one way to turn app.yaml into plaintext, not
|
||||||
|
// two. A name whose value is absent/empty is simply omitted — the restore's fail-closed data-key
|
||||||
|
// gate is what decides whether that is survivable.
|
||||||
|
portableNames := stacks.PortableSecretEnvVars(&meta)
|
||||||
|
portable := make(map[string]string, len(portableNames))
|
||||||
|
if len(portableNames) > 0 {
|
||||||
|
if dec := stacks.LoadAppConfigDecrypted(stackDir, a.encKey); dec != nil {
|
||||||
|
for _, n := range portableNames {
|
||||||
|
if v, ok := dec.Env[n]; ok && v != "" {
|
||||||
|
portable[n] = v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
return backup.RecoveryInfo{
|
return backup.RecoveryInfo{
|
||||||
StackDir: stackDir,
|
StackDir: stackDir,
|
||||||
DisplayName: s.Meta.DisplayName,
|
DisplayName: s.Meta.DisplayName,
|
||||||
ImagePins: backup.ParseComposeImages(s.ComposePath),
|
ImagePins: backup.ParseComposeImages(s.ComposePath),
|
||||||
NonSecretEnv: nonSecret,
|
NonSecretEnv: nonSecret,
|
||||||
SecretEnvVars: secretNames,
|
SecretEnvVars: secretNames,
|
||||||
DataKeyEnvVars: dataKeys,
|
DataKeyEnvVars: dataKeys,
|
||||||
|
PortableSecretEnvVars: portableNames,
|
||||||
|
PortableSecrets: portable,
|
||||||
}, true
|
}, true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -28,10 +28,10 @@ type StackDataProvider interface {
|
|||||||
StopStack(name string) error
|
StopStack(name string) error
|
||||||
StartStack(name string) error
|
StartStack(name string) error
|
||||||
RefreshAndIsRunning(name string) bool
|
RefreshAndIsRunning(name string) bool
|
||||||
// GetStackRecoveryInfo returns the data needed to capture a SECRET-FREE recovery unit
|
// GetStackRecoveryInfo returns the data needed to capture a recovery unit: the stack dir,
|
||||||
// (Phase 2): the stack dir, pinned image tags, the non-secret env, and the NAMES of the
|
// pinned image tags, the non-secret env, the NAMES of the secret/data-key env vars, and (D5)
|
||||||
// secret/data-key env vars (values are NEVER returned — they are recovered at restore time
|
// the decrypted VALUES of the portable class. A WITHHELD secret's value is never returned —
|
||||||
// from the guest's own app.yaml, live or via the PBS whole-guest snapshot). ok=false if the
|
// it is recovered at restore time from the guest's app.yaml, or regenerated. ok=false if the
|
||||||
// stack is unknown.
|
// stack is unknown.
|
||||||
GetStackRecoveryInfo(name string) (RecoveryInfo, bool)
|
GetStackRecoveryInfo(name string) (RecoveryInfo, bool)
|
||||||
|
|
||||||
@@ -62,17 +62,27 @@ type StackDataProvider interface {
|
|||||||
GetStackClassifiedBinds(name string) ([]ClassifiedBind, bool)
|
GetStackClassifiedBinds(name string) ([]ClassifiedBind, bool)
|
||||||
}
|
}
|
||||||
|
|
||||||
// RecoveryInfo carries everything needed to write a secret-free recovery unit for a stack.
|
// RecoveryInfo carries everything needed to write a recovery unit for a stack.
|
||||||
// It deliberately holds NO secret values — only the names of secret/data-key env vars, so the
|
//
|
||||||
// manifest can record what must be recovered from elsewhere (guest app.yaml / PBS) without the
|
// D5: it now carries the VALUES of the PORTABLE secret class (stacks.PortableSecretEnvVars — every
|
||||||
// unit ever storing a secret or a data-encrypting key.
|
// `type: secret` field bar the nonPortableSecrets register), because a Tier-1/2 restore that depends
|
||||||
|
// on the guest for a data-encrypting key or a DB password is not independent of the guest at all: the
|
||||||
|
// data sits safely on the drive and cannot be read back. The EXCLUDED class (`type: password` admin
|
||||||
|
// logins) is still name-only and never leaves the guest.
|
||||||
type RecoveryInfo struct {
|
type RecoveryInfo struct {
|
||||||
StackDir string // dir holding docker-compose.yml + .felhom.yml + app.yaml
|
StackDir string // dir holding docker-compose.yml + .felhom.yml + app.yaml
|
||||||
DisplayName string // app display name
|
DisplayName string // app display name
|
||||||
ImagePins []string // pinned image tags from compose `image:` lines (re-pulled on restore)
|
ImagePins []string // pinned image tags from compose `image:` lines (re-pulled on restore)
|
||||||
NonSecretEnv map[string]string // env with all secret/password/data-key values removed (plaintext only)
|
NonSecretEnv map[string]string // env with ALL secret/password values removed (plaintext only)
|
||||||
SecretEnvVars []string // NAMES of stripped secret/password fields (recovered from guest/PBS)
|
SecretEnvVars []string // NAMES of every secret/password field
|
||||||
DataKeyEnvVars []string // NAMES of data-encrypting-key fields (fail-closed gate on restore)
|
DataKeyEnvVars []string // NAMES of data-encrypting-key fields (fail-closed gate on restore)
|
||||||
|
// PortableSecretEnvVars are the NAMES of the secrets that travel in the unit (D5), and
|
||||||
|
// PortableSecrets their DECRYPTED values. A name present here but absent from PortableSecrets was
|
||||||
|
// unset/empty in the guest's app.yaml — the restore's fail-closed gate decides what that means.
|
||||||
|
// Never logged, never in the manifest's value space: the values reach disk only inside the unit's
|
||||||
|
// 0600 app.yaml.
|
||||||
|
PortableSecretEnvVars []string
|
||||||
|
PortableSecrets map[string]string
|
||||||
}
|
}
|
||||||
|
|
||||||
// ParseComposeImages extracts the pinned image references (`image: repo:tag`) from a
|
// ParseComposeImages extracts the pinned image references (`image: repo:tag`) from a
|
||||||
|
|||||||
@@ -40,9 +40,10 @@ func PrimaryBackupPath(nsRoot string) string {
|
|||||||
// RecoveryUnitPath returns the per-app self-contained recovery-unit ROOT under a namespace root.
|
// RecoveryUnitPath returns the per-app self-contained recovery-unit ROOT under a namespace root.
|
||||||
// It is the existing per-app backup dir (`backups/primary/<stack>/`) — the legacy name is kept so the
|
// It is the existing per-app backup dir (`backups/primary/<stack>/`) — the legacy name is kept so the
|
||||||
// db-dumps/ and volume-dumps/ already written there need no migration; the unit gains compose/ and
|
// db-dumps/ and volume-dumps/ already written there need no migration; the unit gains compose/ and
|
||||||
// manifest.json as siblings, making the whole dir a complete, recreatable unit (Phase 2). The unit is
|
// manifest.json as siblings, making the whole dir a complete, recreatable unit (Phase 2). Since D5 the
|
||||||
// secret-free: secrets/data-keys are recovered from the guest's own app.yaml (live or via PBS), never
|
// unit's compose/app.yaml CARRIES the portable secret class (data keys, DB passwords, internal signing
|
||||||
// stored here. See backup.recoveryUnit / restore for the capture + restore flow.
|
// secrets) at mode 0600, so a Tier-1/2 restore needs the drive and nothing else; internet-reachable
|
||||||
|
// admin logins are still withheld. See backup.recoveryUnit / restore for the capture + restore flow.
|
||||||
func RecoveryUnitPath(nsRoot, stackName string) string {
|
func RecoveryUnitPath(nsRoot, stackName string) string {
|
||||||
return filepath.Join(nsRoot, "backups", "primary", stackName)
|
return filepath.Join(nsRoot, "backups", "primary", stackName)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,20 +15,28 @@ import (
|
|||||||
"gopkg.in/yaml.v3"
|
"gopkg.in/yaml.v3"
|
||||||
)
|
)
|
||||||
|
|
||||||
// RecoveryManifest describes an app's self-contained, SECRET-FREE recovery unit (Phase 2).
|
// RecoveryManifest describes an app's self-contained recovery unit.
|
||||||
//
|
//
|
||||||
// The unit on a drive is `<nsRoot>/backups/primary/<app>/` and contains:
|
// The unit on a drive is `<nsRoot>/backups/primary/<app>/` and contains:
|
||||||
//
|
//
|
||||||
// compose/ docker-compose.yml + .felhom.yml + a SECRET-STRIPPED app.yaml
|
// compose/ docker-compose.yml + .felhom.yml + app.yaml (0600; carries the PORTABLE secrets)
|
||||||
// db-dumps/ app-consistent DB dump(s) (written by the dump flow)
|
// db-dumps/ app-consistent DB dump(s) (written by the dump flow)
|
||||||
// volume-dumps/ named-volume tars (written by the dump flow)
|
// volume-dumps/ named-volume tars (written by the dump flow)
|
||||||
// manifest.json this file
|
// manifest.json this file
|
||||||
//
|
//
|
||||||
// The unit holds NO secret values, NO data-encrypting keys, and NOT the Docker image — only the
|
// D5 (schema 2) changed what the unit holds. Before it held NO secret at all, which made
|
||||||
// pinned image tag(s) (re-pulled on restore) and the NAMES of the secret/data-key env vars. The
|
// "restore from the drive alone" false: the fast, local, customer-doable Tier-1/2 restore secretly
|
||||||
// secret values are recovered at restore time from the guest's own app.yaml (live on the rootfs,
|
// depended on the slow, operator-driven whole-guest restore, because a data-encrypting key or a DB
|
||||||
// or via the PBS whole-guest snapshot) — see Restore. "Restore from the unit alone" is therefore
|
// password absent from the guest cannot be regenerated without rendering the restored data
|
||||||
// honestly "unit + the guest's app.yaml"; SecretSource records that dependency explicitly.
|
// unreachable. The unit now carries the PORTABLE secret class (stacks.PortableSecretEnvVars) in its
|
||||||
|
// 0600 app.yaml, and Tier-1/2 needs the DRIVE AND NOTHING ELSE.
|
||||||
|
//
|
||||||
|
// It still holds NO `type: password` admin login (those are internet-reachable, so their blast radius
|
||||||
|
// is not bounded by the drive — they stay in the guest and are regenerated on restore) and NOT the
|
||||||
|
// Docker image, only the pinned tag(s), re-pulled on restore. SecretSource records the split.
|
||||||
|
//
|
||||||
|
// A schema-1 unit carries no secrets: the restore degrades to the pre-D5 guest-only behaviour rather
|
||||||
|
// than failing, and the next capture rewrites it (the app.yaml checksum changes).
|
||||||
type RecoveryManifest struct {
|
type RecoveryManifest struct {
|
||||||
SchemaVersion int `json:"schema_version"`
|
SchemaVersion int `json:"schema_version"`
|
||||||
AppName string `json:"app_name"`
|
AppName string `json:"app_name"`
|
||||||
@@ -38,13 +46,17 @@ type RecoveryManifest struct {
|
|||||||
Drive string `json:"drive"` // HDD_PATH (in-guest mount)
|
Drive string `json:"drive"` // HDD_PATH (in-guest mount)
|
||||||
NamespaceRoot string `json:"namespace_root"` // resolved felhom-data namespace root
|
NamespaceRoot string `json:"namespace_root"` // resolved felhom-data namespace root
|
||||||
ImagePins []string `json:"image_pins"` // image NOT stored — re-pulled on restore
|
ImagePins []string `json:"image_pins"` // image NOT stored — re-pulled on restore
|
||||||
SecretEnvVars []string `json:"secret_env_vars"` // NAMES only — recovered from guest/PBS
|
SecretEnvVars []string `json:"secret_env_vars"` // NAMES of every secret/password field
|
||||||
DataKeyEnvVars []string `json:"data_key_env_vars"` // fail-closed gate on restore
|
DataKeyEnvVars []string `json:"data_key_env_vars"` // fail-closed gate on restore
|
||||||
SecretSource string `json:"secret_source"` // human note: where secrets come from
|
SecretSource string `json:"secret_source"` // human note: where secrets come from
|
||||||
ConfigFiles []string `json:"config_files"` // captured into compose/
|
ConfigFiles []string `json:"config_files"` // captured into compose/
|
||||||
DBDumps []string `json:"db_dumps"`
|
DBDumps []string `json:"db_dumps"`
|
||||||
VolumeDumps []string `json:"volume_dumps"`
|
VolumeDumps []string `json:"volume_dumps"`
|
||||||
Checksums map[string]string `json:"checksums"` // sha256 of captured compose/ files
|
Checksums map[string]string `json:"checksums"` // sha256 of captured compose/ files
|
||||||
|
// PortableSecretEnvVars (D5) are the NAMES of the secrets this unit's app.yaml CARRIES. Names only
|
||||||
|
// — the manifest is 0644 and never holds a value. The restore reads it to know which app.yaml env
|
||||||
|
// entries are secrets rather than plain config; absent (schema 1) ⇒ the unit carries none.
|
||||||
|
PortableSecretEnvVars []string `json:"portable_secret_env_vars,omitempty"`
|
||||||
// R-43/R-44 (v0.148.0): the coherence stamp. An offsite run refreshes the dumps FIRST and then
|
// R-43/R-44 (v0.148.0): the coherence stamp. An offsite run refreshes the dumps FIRST and then
|
||||||
// captures the unit, so a manifest carrying an OffsiteRunID asserts "the db-dumps/ in this unit
|
// captures the unit, so a manifest carrying an OffsiteRunID asserts "the db-dumps/ in this unit
|
||||||
// were taken by that run" — i.e. the snapshot is an internally coherent {DB@T, files@T} pair.
|
// were taken by that run" — i.e. the snapshot is an internally coherent {DB@T, files@T} pair.
|
||||||
@@ -68,9 +80,11 @@ func (m *Manager) SetTier2Notifier(fn func(stackName, destLabel string, dur time
|
|||||||
m.tier2Notify = fn
|
m.tier2Notify = fn
|
||||||
}
|
}
|
||||||
|
|
||||||
// CaptureRecoveryUnit writes/refreshes an app's secret-free recovery unit: it captures the
|
// CaptureRecoveryUnit writes/refreshes an app's recovery unit: it captures the compose + metadata +
|
||||||
// compose + metadata + a secret-stripped app.yaml into compose/, enumerates the DB/volume dumps
|
// an app.yaml carrying the PORTABLE secret class (D5) into compose/, enumerates the DB/volume dumps
|
||||||
// already present, and writes manifest.json. It NEVER writes a secret value or the Docker image.
|
// already present, and writes manifest.json. It never writes the Docker image (only the pinned tag),
|
||||||
|
// and never writes a WITHHELD secret — the split is decided in buildUnitAppYaml, pinned by
|
||||||
|
// TestCaptureRecoveryUnitCarriesPortableSecretsOnly.
|
||||||
//
|
//
|
||||||
// Idempotent: it builds the captured content in memory first and SKIPS all writes when the unit is
|
// Idempotent: it builds the captured content in memory first and SKIPS all writes when the unit is
|
||||||
// already current (same config checksums, same dump set, same controller version) — so it can run on
|
// already current (same config checksums, same dump set, same controller version) — so it can run on
|
||||||
@@ -107,7 +121,7 @@ func (m *Manager) CaptureRecoveryUnit(stackName string) error {
|
|||||||
checksums[fname] = sha256Hex(data)
|
checksums[fname] = sha256Hex(data)
|
||||||
configFiles = append(configFiles, fname)
|
configFiles = append(configFiles, fname)
|
||||||
}
|
}
|
||||||
appYaml := buildStrippedAppYaml(info)
|
appYaml := buildUnitAppYaml(info)
|
||||||
files = append(files, capFile{"app.yaml", appYaml, 0600})
|
files = append(files, capFile{"app.yaml", appYaml, 0600})
|
||||||
checksums["app.yaml"] = sha256Hex(appYaml)
|
checksums["app.yaml"] = sha256Hex(appYaml)
|
||||||
configFiles = append(configFiles, "app.yaml")
|
configFiles = append(configFiles, "app.yaml")
|
||||||
@@ -151,30 +165,34 @@ func (m *Manager) CaptureRecoveryUnit(stackName string) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
manifest := &RecoveryManifest{
|
manifest := &RecoveryManifest{
|
||||||
SchemaVersion: 1,
|
SchemaVersion: 2, // D5: compose/app.yaml carries the portable secret class
|
||||||
AppName: stackName,
|
AppName: stackName,
|
||||||
DisplayName: info.DisplayName,
|
DisplayName: info.DisplayName,
|
||||||
ControllerVer: version,
|
ControllerVer: version,
|
||||||
CreatedAt: time.Now().UTC().Format(time.RFC3339),
|
CreatedAt: time.Now().UTC().Format(time.RFC3339),
|
||||||
Drive: drivePath,
|
Drive: drivePath,
|
||||||
NamespaceRoot: nsRoot,
|
NamespaceRoot: nsRoot,
|
||||||
ImagePins: info.ImagePins,
|
ImagePins: info.ImagePins,
|
||||||
SecretEnvVars: info.SecretEnvVars,
|
SecretEnvVars: info.SecretEnvVars,
|
||||||
DataKeyEnvVars: info.DataKeyEnvVars,
|
DataKeyEnvVars: info.DataKeyEnvVars,
|
||||||
SecretSource: "guest app.yaml (live rootfs) or PBS whole-guest snapshot — never stored in this unit",
|
PortableSecretEnvVars: info.PortableSecretEnvVars,
|
||||||
ConfigFiles: configFiles,
|
SecretSource: "portable secrets (data keys, DB passwords, internal signing secrets) are IN this unit's compose/app.yaml (0600); internet-reachable admin logins are NOT, and come from the guest's app.yaml or are regenerated on restore",
|
||||||
DBDumps: dbDumps,
|
ConfigFiles: configFiles,
|
||||||
VolumeDumps: volDumps,
|
DBDumps: dbDumps,
|
||||||
Checksums: checksums,
|
VolumeDumps: volDumps,
|
||||||
OffsiteRunID: runID,
|
Checksums: checksums,
|
||||||
DumpsAt: dumpsAt,
|
OffsiteRunID: runID,
|
||||||
|
DumpsAt: dumpsAt,
|
||||||
}
|
}
|
||||||
if err := writeManifest(manifestPath, manifest); err != nil {
|
if err := writeManifest(manifestPath, manifest); err != nil {
|
||||||
return fmt.Errorf("writing manifest: %w", err)
|
return fmt.Errorf("writing manifest: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
m.logger.Printf("[INFO] [backup] Recovery unit captured for %s → %s (images=%d, secrets-referenced=%d, data_keys=%d)",
|
// Counts and NAMES only — never a value (D5 puts more secrets through this path than before).
|
||||||
stackName, RecoveryUnitPath(nsRoot, stackName), len(info.ImagePins), len(info.SecretEnvVars), len(info.DataKeyEnvVars))
|
m.logger.Printf("[INFO] [backup] Recovery unit captured for %s → %s (images=%d, secrets-referenced=%d, data_keys=%d, portable-carried=%d/%d, withheld=%d)",
|
||||||
|
stackName, RecoveryUnitPath(nsRoot, stackName), len(info.ImagePins), len(info.SecretEnvVars),
|
||||||
|
len(info.DataKeyEnvVars), len(info.PortableSecrets), len(info.PortableSecretEnvVars),
|
||||||
|
len(withheldSecretNames(info)))
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -201,29 +219,63 @@ func (m *Manager) versionLocked() string {
|
|||||||
return m.version
|
return m.version
|
||||||
}
|
}
|
||||||
|
|
||||||
// strippedAppYaml is the on-disk shape of the secret-free app.yaml captured into the unit.
|
// strippedAppYaml is the on-disk shape of the app.yaml captured into the unit. The name is historical:
|
||||||
|
// since D5 the `env` map carries the PORTABLE secrets alongside the plain config (see buildUnitAppYaml).
|
||||||
type strippedAppYaml struct {
|
type strippedAppYaml struct {
|
||||||
Deployed bool `yaml:"deployed"`
|
Deployed bool `yaml:"deployed"`
|
||||||
Env map[string]string `yaml:"env"`
|
Env map[string]string `yaml:"env"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// buildStrippedAppYaml renders a secret-free app.yaml (non-secret env only) as bytes. Deterministic:
|
// buildUnitAppYaml renders the unit's app.yaml as bytes: the non-secret env PLUS the portable secret
|
||||||
// yaml.v3 sorts map keys and the secret-name list comes in stable metadata order, so identical input
|
// values (D5). Deterministic: yaml.v3 sorts map keys and the name lists come in stable metadata order,
|
||||||
// yields identical bytes (needed for the checksum-skip guard).
|
// so identical input yields identical bytes (needed for the checksum-skip guard).
|
||||||
func buildStrippedAppYaml(info RecoveryInfo) []byte {
|
//
|
||||||
body, err := yaml.Marshal(strippedAppYaml{Deployed: true, Env: info.NonSecretEnv})
|
// This is the ONE place the capture side decides what does and does not reach the drive — there is no
|
||||||
|
// second path that writes a unit app.yaml. The caller writes the result 0600.
|
||||||
|
func buildUnitAppYaml(info RecoveryInfo) []byte {
|
||||||
|
env := make(map[string]string, len(info.NonSecretEnv)+len(info.PortableSecrets))
|
||||||
|
for k, v := range info.NonSecretEnv {
|
||||||
|
env[k] = v
|
||||||
|
}
|
||||||
|
// Portable secrets last: NonSecretEnv is disjoint from the secret set by construction
|
||||||
|
// (GetStackRecoveryInfo), so this cannot shadow a plain config value.
|
||||||
|
for k, v := range info.PortableSecrets {
|
||||||
|
env[k] = v
|
||||||
|
}
|
||||||
|
body, err := yaml.Marshal(strippedAppYaml{Deployed: true, Env: env})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
body = []byte("deployed: true\nenv: {}\n")
|
body = []byte("deployed: true\nenv: {}\n")
|
||||||
}
|
}
|
||||||
header := "# Captured by felhom-controller recovery unit — SECRET-FREE.\n" +
|
header := "# Captured by felhom-controller recovery unit.\n" +
|
||||||
"# Secret/data-key values are intentionally omitted; recover them at restore from the\n" +
|
"# This file CARRIES SECRETS (D5) so a Tier-1/2 restore needs the drive and nothing else:\n" +
|
||||||
"# guest's own app.yaml (live rootfs, or the PBS whole-guest snapshot). Stripped names:\n"
|
"# data-encrypting keys, database passwords and internal signing secrets. Mode 0600.\n"
|
||||||
if len(info.SecretEnvVars) > 0 {
|
if len(info.PortableSecretEnvVars) > 0 {
|
||||||
header += "# " + strings.Join(info.SecretEnvVars, ", ") + "\n"
|
header += "# Carried: " + strings.Join(info.PortableSecretEnvVars, ", ") + "\n"
|
||||||
|
}
|
||||||
|
// The withheld class is named, not valued — an operator reading the unit must be able to see WHY a
|
||||||
|
// credential is missing rather than suspecting a capture bug.
|
||||||
|
if withheld := withheldSecretNames(info); len(withheld) > 0 {
|
||||||
|
header += "# WITHHELD (internet-reachable logins — stay in the guest, regenerated on restore): " +
|
||||||
|
strings.Join(withheld, ", ") + "\n"
|
||||||
}
|
}
|
||||||
return []byte(header + string(body))
|
return []byte(header + string(body))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// withheldSecretNames returns the secret names deliberately NOT carried by the unit, in stable order.
|
||||||
|
func withheldSecretNames(info RecoveryInfo) []string {
|
||||||
|
portable := make(map[string]bool, len(info.PortableSecretEnvVars))
|
||||||
|
for _, n := range info.PortableSecretEnvVars {
|
||||||
|
portable[n] = true
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, n := range info.SecretEnvVars {
|
||||||
|
if !portable[n] {
|
||||||
|
out = append(out, n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
// writeManifest writes the manifest JSON atomically.
|
// writeManifest writes the manifest JSON atomically.
|
||||||
func writeManifest(dst string, manifest *RecoveryManifest) error {
|
func writeManifest(dst string, manifest *RecoveryManifest) error {
|
||||||
data, err := json.MarshalIndent(manifest, "", " ")
|
data, err := json.MarshalIndent(manifest, "", " ")
|
||||||
|
|||||||
@@ -70,12 +70,21 @@ func (f *fakeRecoveryProvider) StartStackServices(_ string, services []string) e
|
|||||||
return f.startSvcErr
|
return f.startSvcErr
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestCaptureRecoveryUnitIsSecretFree proves the captured unit (a) contains compose+config+manifest,
|
// TestCaptureRecoveryUnitCarriesPortableSecretsOnly proves the captured unit (a) contains
|
||||||
// (b) enumerates the existing dumps, and (c) is SECRET-FREE: a secret value present in the SOURCE
|
// compose+config+manifest, (b) enumerates the existing dumps, and (c) implements the D5 secret split:
|
||||||
// app.yaml does NOT appear anywhere in the unit, because the capture writes the stripped NonSecretEnv
|
// the PORTABLE class is written into the unit's app.yaml, and the WITHHELD class appears NOWHERE in
|
||||||
// (not the raw app.yaml). The manifest records the secret NAMES + data_key flag for recovery-from-guest.
|
// the unit.
|
||||||
func TestCaptureRecoveryUnitIsSecretFree(t *testing.T) {
|
//
|
||||||
const secretVal = "SUPERSECRETVALUE-do-not-leak"
|
// This test replaces TestCaptureRecoveryUnitIsSecretFree, whose global "no secret value appears in the
|
||||||
|
// unit" invariant D5 deliberately overturns for the portable class. The wrong-outcome half — the
|
||||||
|
// withheld value must still leak nowhere — is kept verbatim, because that is the half that is still a
|
||||||
|
// security boundary.
|
||||||
|
func TestCaptureRecoveryUnitCarriesPortableSecretsOnly(t *testing.T) {
|
||||||
|
const (
|
||||||
|
dataKeyVal = "DATAKEY-must-travel-or-the-data-is-unreadable"
|
||||||
|
dbPwVal = "DBPASSWORD-must-travel-or-the-app-cannot-authenticate"
|
||||||
|
withheldVal = "ADMINLOGIN-must-never-reach-the-drive"
|
||||||
|
)
|
||||||
tmp := t.TempDir()
|
tmp := t.TempDir()
|
||||||
stackDir := filepath.Join(tmp, "stack")
|
stackDir := filepath.Join(tmp, "stack")
|
||||||
drive := filepath.Join(tmp, "drive") // in-guest namespace root (basename need not be felhom-data)
|
drive := filepath.Join(tmp, "drive") // in-guest namespace root (basename need not be felhom-data)
|
||||||
@@ -83,25 +92,30 @@ func TestCaptureRecoveryUnitIsSecretFree(t *testing.T) {
|
|||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Source stack files — the raw app.yaml DELIBERATELY holds a secret to prove it's not copied.
|
|
||||||
mustWrite(t, filepath.Join(stackDir, "docker-compose.yml"),
|
mustWrite(t, filepath.Join(stackDir, "docker-compose.yml"),
|
||||||
"services:\n app:\n image: example/app:1.2.3\n")
|
"services:\n app:\n image: example/app:1.2.3\n")
|
||||||
mustWrite(t, filepath.Join(stackDir, ".felhom.yml"), "display_name: Example\n")
|
mustWrite(t, filepath.Join(stackDir, ".felhom.yml"), "display_name: Example\n")
|
||||||
|
// The SOURCE app.yaml holds the withheld admin login too, so the leak check below is not vacuous:
|
||||||
|
// it fails if anything ever copies the raw app.yaml into the unit instead of the generated one.
|
||||||
mustWrite(t, filepath.Join(stackDir, "app.yaml"),
|
mustWrite(t, filepath.Join(stackDir, "app.yaml"),
|
||||||
"deployed: true\nenv:\n DB_PASSWORD: "+secretVal+"\n SUBDOMAIN: example\n")
|
"deployed: true\nenv:\n DB_PASSWORD: "+dbPwVal+"\n ADMIN_PASSWORD: "+withheldVal+
|
||||||
|
"\n SUBDOMAIN: example\n")
|
||||||
|
|
||||||
// Pre-existing dumps (written by the dump flow before capture).
|
// Pre-existing dumps (written by the dump flow before capture).
|
||||||
mustWrite(t, filepath.Join(AppDBDumpPath(drive, "example"), "example-postgres.sql"), "dump")
|
mustWrite(t, filepath.Join(AppDBDumpPath(drive, "example"), "example-postgres.sql"), "dump")
|
||||||
mustWrite(t, filepath.Join(AppVolumeDumpPath(drive, "example"), "example_data.tar"), "tar")
|
mustWrite(t, filepath.Join(AppVolumeDumpPath(drive, "example"), "example_data.tar"), "tar")
|
||||||
|
|
||||||
// RecoveryInfo as the adapter would build it: secret values already stripped from NonSecretEnv.
|
// RecoveryInfo as the adapter builds it: NonSecretEnv holds no secret, PortableSecrets holds the
|
||||||
|
// decrypted portable class, and ADMIN_PASSWORD is named in SecretEnvVars but NOT portable.
|
||||||
info := RecoveryInfo{
|
info := RecoveryInfo{
|
||||||
StackDir: stackDir,
|
StackDir: stackDir,
|
||||||
DisplayName: "Example",
|
DisplayName: "Example",
|
||||||
ImagePins: []string{"example/app:1.2.3"},
|
ImagePins: []string{"example/app:1.2.3"},
|
||||||
NonSecretEnv: map[string]string{"SUBDOMAIN": "example", "HDD_PATH": drive},
|
NonSecretEnv: map[string]string{"SUBDOMAIN": "example", "HDD_PATH": drive},
|
||||||
SecretEnvVars: []string{"DB_PASSWORD", "SECRET_KEY"},
|
SecretEnvVars: []string{"DB_PASSWORD", "SECRET_KEY", "ADMIN_PASSWORD"},
|
||||||
DataKeyEnvVars: []string{"SECRET_KEY"},
|
DataKeyEnvVars: []string{"SECRET_KEY"},
|
||||||
|
PortableSecretEnvVars: []string{"DB_PASSWORD", "SECRET_KEY"},
|
||||||
|
PortableSecrets: map[string]string{"DB_PASSWORD": dbPwVal, "SECRET_KEY": dataKeyVal},
|
||||||
}
|
}
|
||||||
m := &Manager{
|
m := &Manager{
|
||||||
logger: log.New(io.Discard, "", 0),
|
logger: log.New(io.Discard, "", 0),
|
||||||
@@ -136,8 +150,8 @@ func TestCaptureRecoveryUnitIsSecretFree(t *testing.T) {
|
|||||||
if len(man.ImagePins) != 1 || man.ImagePins[0] != "example/app:1.2.3" {
|
if len(man.ImagePins) != 1 || man.ImagePins[0] != "example/app:1.2.3" {
|
||||||
t.Errorf("image pins: %v", man.ImagePins)
|
t.Errorf("image pins: %v", man.ImagePins)
|
||||||
}
|
}
|
||||||
if len(man.SecretEnvVars) != 2 {
|
if len(man.SecretEnvVars) != 3 {
|
||||||
t.Errorf("secret env-var names: %v (want 2)", man.SecretEnvVars)
|
t.Errorf("secret env-var names: %v (want 3)", man.SecretEnvVars)
|
||||||
}
|
}
|
||||||
if len(man.DataKeyEnvVars) != 1 || man.DataKeyEnvVars[0] != "SECRET_KEY" {
|
if len(man.DataKeyEnvVars) != 1 || man.DataKeyEnvVars[0] != "SECRET_KEY" {
|
||||||
t.Errorf("data-key env-vars: %v", man.DataKeyEnvVars)
|
t.Errorf("data-key env-vars: %v", man.DataKeyEnvVars)
|
||||||
@@ -145,21 +159,48 @@ func TestCaptureRecoveryUnitIsSecretFree(t *testing.T) {
|
|||||||
if len(man.DBDumps) != 1 || len(man.VolumeDumps) != 1 {
|
if len(man.DBDumps) != 1 || len(man.VolumeDumps) != 1 {
|
||||||
t.Errorf("dumps enumerated: db=%v vol=%v", man.DBDumps, man.VolumeDumps)
|
t.Errorf("dumps enumerated: db=%v vol=%v", man.DBDumps, man.VolumeDumps)
|
||||||
}
|
}
|
||||||
|
// D5: schema 2 + the carried names, so the restore can tell secrets from plain config.
|
||||||
// app.yaml in the unit must carry the non-secret env but NOT the secret value.
|
if man.SchemaVersion != 2 {
|
||||||
appy := mustRead(t, filepath.Join(composeDir, "app.yaml"))
|
t.Errorf("schema version = %d, want 2 (D5 units carry secrets)", man.SchemaVersion)
|
||||||
if !strings.Contains(appy, "SUBDOMAIN") {
|
}
|
||||||
t.Errorf("stripped app.yaml missing non-secret env: %s", appy)
|
if len(man.PortableSecretEnvVars) != 2 {
|
||||||
|
t.Errorf("portable secret names: %v (want DB_PASSWORD + SECRET_KEY)", man.PortableSecretEnvVars)
|
||||||
|
}
|
||||||
|
// The manifest is 0644 — it must record NAMES, never a value.
|
||||||
|
if s := string(mfData); strings.Contains(s, dbPwVal) || strings.Contains(s, dataKeyVal) {
|
||||||
|
t.Error("SECRET LEAK: a secret VALUE reached manifest.json (names only)")
|
||||||
}
|
}
|
||||||
|
|
||||||
// SECRET-FREE invariant: the secret value must not appear ANYWHERE in the unit.
|
// app.yaml in the unit must carry the non-secret env AND the portable secrets.
|
||||||
|
appyPath := filepath.Join(composeDir, "app.yaml")
|
||||||
|
appy := mustRead(t, appyPath)
|
||||||
|
if !strings.Contains(appy, "SUBDOMAIN") {
|
||||||
|
t.Errorf("unit app.yaml missing non-secret env: %s", appy)
|
||||||
|
}
|
||||||
|
// THE CONSEQUENCE of D5 at capture time: without these two values on the drive, a guest-less
|
||||||
|
// restore cannot read the data sitting beside them.
|
||||||
|
if !strings.Contains(appy, dataKeyVal) {
|
||||||
|
t.Error("data-encrypting key did NOT travel — a guest-less restore would be impossible")
|
||||||
|
}
|
||||||
|
if !strings.Contains(appy, dbPwVal) {
|
||||||
|
t.Error("DB password did NOT travel — the restored app could not authenticate to its own data")
|
||||||
|
}
|
||||||
|
// Secret-bearing ⇒ owner-only.
|
||||||
|
if fi, err := os.Stat(appyPath); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
} else if perm := fi.Mode().Perm(); perm != 0600 {
|
||||||
|
t.Errorf("unit app.yaml mode = %04o, want 0600 (it carries secrets)", perm)
|
||||||
|
}
|
||||||
|
|
||||||
|
// THE WRONG-OUTCOME CHECK: the withheld class must appear NOWHERE in the unit. This is the half of
|
||||||
|
// the old secret-free invariant that D5 does not relax.
|
||||||
unitRoot := RecoveryUnitPath(drive, "example")
|
unitRoot := RecoveryUnitPath(drive, "example")
|
||||||
_ = filepath.WalkDir(unitRoot, func(path string, d fs.DirEntry, err error) error {
|
_ = filepath.WalkDir(unitRoot, func(path string, d fs.DirEntry, err error) error {
|
||||||
if err != nil || d.IsDir() {
|
if err != nil || d.IsDir() {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
if strings.Contains(mustRead(t, path), secretVal) {
|
if strings.Contains(mustRead(t, path), withheldVal) {
|
||||||
t.Errorf("SECRET LEAK: %q found in %s", secretVal, path)
|
t.Errorf("SECRET LEAK: withheld admin login %q found in %s", withheldVal, path)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -11,27 +11,53 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// reconcileRestoreSecrets merges the recovery unit's non-secret env with the secrets recovered from
|
// reconcileRestoreSecrets merges the recovery unit's non-secret env with the secrets recovered from
|
||||||
// the guest's own app.yaml, and applies the FAIL-CLOSED data-key gate. It is the safety-critical heart
|
// the unit itself (D5) and from the guest's own app.yaml, and applies the FAIL-CLOSED data-key gate.
|
||||||
// of Phase 2b and is deliberately a pure function (no I/O) so it can be exhaustively unit-tested.
|
// It is the safety-critical heart of Phase 2b and is deliberately a pure function (no I/O) so it can
|
||||||
|
// be exhaustively unit-tested — the D5 source arrives as an ARGUMENT, not as a read.
|
||||||
//
|
//
|
||||||
// Policy (per the Phase 2 design — see REPORT/CHANGELOG):
|
// Policy:
|
||||||
// - Regenerate NOTHING. Every secret comes from the guest (live rootfs, or PBS whole-guest restore).
|
// - Regenerate NOTHING here. Secrets come from the unit (portable class) or the guest (the rest).
|
||||||
// - A missing DATA-ENCRYPTING key (`dataKeyNames`) is FATAL: regenerating it would render the
|
// - A missing DATA-ENCRYPTING key (`dataKeyNames`) is FATAL: regenerating it would render the
|
||||||
// restored data unreadable, so we refuse and tell the operator to do a PBS whole-guest restore.
|
// restored data unreadable, so we refuse and tell the operator to do a PBS whole-guest restore.
|
||||||
// - A missing resettable secret (DB password, admin password) is NON-fatal: it's returned in
|
// D5 means the key is normally IN the unit — but "normally" is not a reason to soften the gate.
|
||||||
// `missing` so the caller can warn; the app may simply need a credential reset, no data is lost.
|
// - A missing resettable secret is NON-fatal: returned in `missing` so the caller can warn or
|
||||||
func reconcileRestoreSecrets(nonSecretEnv, recoveredSecrets map[string]string, secretNames, dataKeyNames []string) (fullEnv map[string]string, missing []string, err error) {
|
// regenerate it (O4). No data is lost.
|
||||||
|
//
|
||||||
|
// PRECEDENCE — the UNIT WINS over the guest when both hold a value for the same name.
|
||||||
|
//
|
||||||
|
// This is not arbitrary and it is not "newest wins". The unit's secrets are captured in the SAME run
|
||||||
|
// as the dumps beside them (runVolumeDumps → captureAllRecoveryUnits, backup.go), so the unit's value
|
||||||
|
// is the one that MATCHES THE DATA ABOUT TO BE RESTORED, whereas the guest's value is merely the most
|
||||||
|
// recent. Where they disagree the guest's has been rotated since the capture, and preferring it is
|
||||||
|
// precisely the data-loss bug:
|
||||||
|
// - a rotated data-encrypting key does not decrypt data encrypted with the old one;
|
||||||
|
// - a rotated DB password does not match the scram/mysql hash inside the restored data directory
|
||||||
|
// (POSTGRES_PASSWORD is ignored once PGDATA is non-empty), so the app cannot reach its own rows.
|
||||||
|
//
|
||||||
|
// The restore persists fullEnv back to the guest's app.yaml (RecreateStackDefinitionFromUnit), so
|
||||||
|
// unit-wins also leaves the guest consistent with the data now on disk.
|
||||||
|
func reconcileRestoreSecrets(nonSecretEnv, unitSecrets, guestSecrets map[string]string, secretNames, dataKeyNames []string) (fullEnv map[string]string, missing []string, err error) {
|
||||||
fullEnv = make(map[string]string, len(nonSecretEnv)+len(secretNames))
|
fullEnv = make(map[string]string, len(nonSecretEnv)+len(secretNames))
|
||||||
for k, v := range nonSecretEnv {
|
for k, v := range nonSecretEnv {
|
||||||
fullEnv[k] = v
|
fullEnv[k] = v
|
||||||
}
|
}
|
||||||
|
// resolve applies the precedence: unit first, guest only as a fallback.
|
||||||
|
resolve := func(n string) (string, bool) {
|
||||||
|
if v, ok := unitSecrets[n]; ok && v != "" {
|
||||||
|
return v, true
|
||||||
|
}
|
||||||
|
if v, ok := guestSecrets[n]; ok && v != "" {
|
||||||
|
return v, true
|
||||||
|
}
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
have := func(n string) bool {
|
have := func(n string) bool {
|
||||||
v, ok := recoveredSecrets[n]
|
_, ok := resolve(n)
|
||||||
return ok && v != ""
|
return ok
|
||||||
}
|
}
|
||||||
for _, n := range secretNames {
|
for _, n := range secretNames {
|
||||||
if have(n) {
|
if v, ok := resolve(n); ok {
|
||||||
fullEnv[n] = recoveredSecrets[n]
|
fullEnv[n] = v
|
||||||
} else {
|
} else {
|
||||||
missing = append(missing, n)
|
missing = append(missing, n)
|
||||||
}
|
}
|
||||||
@@ -45,24 +71,42 @@ func reconcileRestoreSecrets(nonSecretEnv, recoveredSecrets map[string]string, s
|
|||||||
}
|
}
|
||||||
if len(missingDataKeys) > 0 {
|
if len(missingDataKeys) > 0 {
|
||||||
return nil, missing, fmt.Errorf(
|
return nil, missing, fmt.Errorf(
|
||||||
"refusing to restore: data-encrypting key(s) %v could not be recovered from the guest's app.yaml — "+
|
"refusing to restore: data-encrypting key(s) %v are in NEITHER the recovery unit nor the guest's app.yaml — "+
|
||||||
"a PBS whole-guest restore is required first (regenerating the key would render stored data unreadable)",
|
"a PBS whole-guest restore is required first (regenerating the key would render stored data unreadable)",
|
||||||
missingDataKeys)
|
missingDataKeys)
|
||||||
}
|
}
|
||||||
return fullEnv, missing, nil
|
return fullEnv, missing, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// readStrippedEnv parses the non-secret env from a recovery unit's secret-stripped app.yaml.
|
// readUnitEnv parses a recovery unit's app.yaml and SPLITS it into the plain config env and the
|
||||||
func readStrippedEnv(path string) map[string]string {
|
// secrets the unit carries (D5), using the manifest's portable-secret names as the discriminator.
|
||||||
|
//
|
||||||
|
// The split is driven by the MANIFEST, not by guessing from key names: the manifest and the app.yaml
|
||||||
|
// are captured together and checksummed together, so they cannot disagree about which entries are
|
||||||
|
// secrets. A schema-1 unit has no portable names, so everything lands in nonSecret — exactly the
|
||||||
|
// pre-D5 behaviour, which is what makes an old unit still restorable.
|
||||||
|
func readUnitEnv(path string, portableNames []string) (nonSecret, unitSecrets map[string]string) {
|
||||||
|
nonSecret, unitSecrets = map[string]string{}, map[string]string{}
|
||||||
data, err := os.ReadFile(path)
|
data, err := os.ReadFile(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return map[string]string{}
|
return nonSecret, unitSecrets
|
||||||
}
|
}
|
||||||
var s strippedAppYaml
|
var s strippedAppYaml
|
||||||
if yaml.Unmarshal(data, &s) != nil || s.Env == nil {
|
if yaml.Unmarshal(data, &s) != nil || s.Env == nil {
|
||||||
return map[string]string{}
|
return nonSecret, unitSecrets
|
||||||
}
|
}
|
||||||
return s.Env
|
isPortable := make(map[string]bool, len(portableNames))
|
||||||
|
for _, n := range portableNames {
|
||||||
|
isPortable[n] = true
|
||||||
|
}
|
||||||
|
for k, v := range s.Env {
|
||||||
|
if isPortable[k] {
|
||||||
|
unitSecrets[k] = v
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
nonSecret[k] = v
|
||||||
|
}
|
||||||
|
return nonSecret, unitSecrets
|
||||||
}
|
}
|
||||||
|
|
||||||
// hasReplayableDump reports whether dumpDir holds a .sql dump that the replay could actually use.
|
// hasReplayableDump reports whether dumpDir holds a .sql dump that the replay could actually use.
|
||||||
@@ -85,13 +129,17 @@ func hasReplayableDump(dumpDir string) bool {
|
|||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
// RestoreFromRecoveryUnit recreates an app from its on-drive recovery unit + the guest's own secrets.
|
// RestoreFromRecoveryUnit recreates an app from its on-drive recovery unit.
|
||||||
//
|
//
|
||||||
// It reads the unit manifest, recovers the secret values from the guest's live app.yaml, applies the
|
// It reads the unit manifest, takes the portable secrets from the UNIT and the rest from the guest's
|
||||||
// fail-closed data-key gate, restores the named-volume data from the unit's tars, then restores the
|
// live app.yaml (unit wins — see reconcileRestoreSecrets), applies the fail-closed data-key gate,
|
||||||
// app's definition from the unit and redeploys it with the reconstructed env (re-pulling the pinned
|
// restores the named-volume data from the unit's tars, then restores the app's definition from the unit
|
||||||
// image). No secret is ever regenerated, and no secret is read from the unit. If no unit exists it
|
// and redeploys it with the reconstructed env (re-pulling the pinned image). If no unit exists it falls
|
||||||
// falls back to the legacy volume-only RestoreApp.
|
// back to the legacy volume-only RestoreApp.
|
||||||
|
//
|
||||||
|
// D5: this no longer needs the guest. A restore with the guest's app.yaml absent succeeds, which is
|
||||||
|
// pinned by TestRestoreFromRecoveryUnitWithGuestAbsent — the withheld class is regenerated (O4) and
|
||||||
|
// only a data key missing from BOTH sources still refuses.
|
||||||
func (m *Manager) RestoreFromRecoveryUnit(stackName string) error {
|
func (m *Manager) RestoreFromRecoveryUnit(stackName string) error {
|
||||||
if m.stackProvider == nil {
|
if m.stackProvider == nil {
|
||||||
return fmt.Errorf("stack provider not configured")
|
return fmt.Errorf("stack provider not configured")
|
||||||
@@ -126,11 +174,15 @@ func (m *Manager) RestoreFromRecoveryUnit(stackName string) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
composeDir := RecoveryUnitComposePath(nsRoot, stackName)
|
composeDir := RecoveryUnitComposePath(nsRoot, stackName)
|
||||||
nonSecretEnv := readStrippedEnv(filepath.Join(composeDir, "app.yaml"))
|
nonSecretEnv, unitSecrets := readUnitEnv(filepath.Join(composeDir, "app.yaml"), manifest.PortableSecretEnvVars)
|
||||||
|
|
||||||
// Recover secrets from the GUEST (never the unit), then apply the fail-closed gate.
|
// D5: the unit carries the portable class, so this is the leg that no longer needs the guest. The
|
||||||
recovered := m.stackProvider.RecoverStackSecrets(stackName, manifest.SecretEnvVars)
|
// guest is still consulted for the WITHHELD class (internet-reachable admin logins) and as the
|
||||||
fullEnv, missing, err := reconcileRestoreSecrets(nonSecretEnv, recovered, manifest.SecretEnvVars, manifest.DataKeyEnvVars)
|
// fallback for a schema-1 unit — it returns an empty map when the guest is gone, which is the whole
|
||||||
|
// point: a Tier-1/2 restore must survive that. Precedence is unit-over-guest (see
|
||||||
|
// reconcileRestoreSecrets), then the fail-closed gate.
|
||||||
|
guestSecrets := m.stackProvider.RecoverStackSecrets(stackName, manifest.SecretEnvVars)
|
||||||
|
fullEnv, missing, err := reconcileRestoreSecrets(nonSecretEnv, unitSecrets, guestSecrets, manifest.SecretEnvVars, manifest.DataKeyEnvVars)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
m.logger.Printf("[ERROR] [backup] Restore REFUSED for %s: %v", stackName, err)
|
m.logger.Printf("[ERROR] [backup] Restore REFUSED for %s: %v", stackName, err)
|
||||||
return err
|
return err
|
||||||
@@ -141,6 +193,16 @@ func (m *Manager) RestoreFromRecoveryUnit(stackName string) error {
|
|||||||
// encrypted in the guest app.yaml and round-trips on the next backup/restore. Data-keys are
|
// encrypted in the guest app.yaml and round-trips on the next backup/restore. Data-keys are
|
||||||
// never generated: the fail-closed gate above already refused if one was missing, and the
|
// never generated: the fail-closed gate above already refused if one was missing, and the
|
||||||
// generator itself refuses data-key fields (defense-in-depth). Values are never logged.
|
// generator itself refuses data-key fields (defense-in-depth). Values are never logged.
|
||||||
|
//
|
||||||
|
// D5 shrinks this path to the rare case: the portable class now comes from the unit, so a
|
||||||
|
// generator run means the secret was empty at capture AND absent from the guest.
|
||||||
|
//
|
||||||
|
// It does NOT claim the reset is harmless. R-127: for a DB password it is not — a restored data
|
||||||
|
// directory keeps the OLD role hash (POSTGRES_PASSWORD is ignored once PGDATA is non-empty), so a
|
||||||
|
// regenerated value leaves the app unable to authenticate against its own restored rows while the
|
||||||
|
// dump replay, which uses the container's local trust socket, still reports success. The old wording
|
||||||
|
// here asserted "stored data is unaffected" for every non-data-key secret; that is false for the 18
|
||||||
|
// DB/root-password fields and is now scoped to what is actually true.
|
||||||
if len(missing) > 0 {
|
if len(missing) > 0 {
|
||||||
dataKeySet := make(map[string]bool, len(manifest.DataKeyEnvVars))
|
dataKeySet := make(map[string]bool, len(manifest.DataKeyEnvVars))
|
||||||
for _, dk := range manifest.DataKeyEnvVars {
|
for _, dk := range manifest.DataKeyEnvVars {
|
||||||
@@ -158,7 +220,7 @@ func (m *Manager) RestoreFromRecoveryUnit(stackName string) error {
|
|||||||
unresolved = append(unresolved, name)
|
unresolved = append(unresolved, name)
|
||||||
}
|
}
|
||||||
if len(generated) > 0 {
|
if len(generated) > 0 {
|
||||||
m.logger.Printf("[WARN] [backup] Restore %s: generated replacement for %v — the credential was reset (old value unrecoverable); stored data is unaffected (no data-key involved)",
|
m.logger.Printf("[WARN] [backup] Restore %s: generated replacement for %v — the credential was reset (old value unrecoverable); no data-encrypting key was involved, but a regenerated DATABASE password will not match the restored data directory's stored hash (R-127) — check the app can reach its data",
|
||||||
stackName, generated)
|
stackName, generated)
|
||||||
}
|
}
|
||||||
if len(unresolved) > 0 {
|
if len(unresolved) > 0 {
|
||||||
|
|||||||
@@ -3,10 +3,123 @@ package backup
|
|||||||
import (
|
import (
|
||||||
"io"
|
"io"
|
||||||
"log"
|
"log"
|
||||||
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// captureFixtureUnit writes a real stack tree, runs the REAL CaptureRecoveryUnit over it, and returns
|
||||||
|
// the drive (namespace root) holding the resulting unit.
|
||||||
|
//
|
||||||
|
// Fixtures come from a CAPTURED unit rather than hand-written YAML deliberately: the capture side and
|
||||||
|
// the restore side must meet at real bytes on a real filesystem, so a change to the on-disk shape
|
||||||
|
// (header text, key ordering, the non-secret/portable split) cannot pass by having a test agree with
|
||||||
|
// itself. Everything from buildUnitAppYaml through readUnitEnv is production code here.
|
||||||
|
func captureFixtureUnit(t *testing.T, portable map[string]string) (drive string) {
|
||||||
|
t.Helper()
|
||||||
|
tmp := t.TempDir()
|
||||||
|
stackDir := filepath.Join(tmp, "stack")
|
||||||
|
drive = filepath.Join(tmp, "drive")
|
||||||
|
if err := os.MkdirAll(stackDir, 0755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
mustWrite(t, filepath.Join(stackDir, "docker-compose.yml"),
|
||||||
|
"services:\n app:\n image: example/app:1\n db:\n image: postgres:16\n")
|
||||||
|
mustWrite(t, filepath.Join(stackDir, ".felhom.yml"), "display_name: App\n")
|
||||||
|
mustWrite(t, filepath.Join(stackDir, "app.yaml"), "deployed: true\nenv:\n SUBDOMAIN: trips\n")
|
||||||
|
|
||||||
|
var names []string
|
||||||
|
for _, n := range []string{"DB_PASSWORD", "SECRET_KEY"} {
|
||||||
|
if _, ok := portable[n]; ok {
|
||||||
|
names = append(names, n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
info := RecoveryInfo{
|
||||||
|
StackDir: stackDir,
|
||||||
|
DisplayName: "App",
|
||||||
|
ImagePins: []string{"example/app:1"},
|
||||||
|
NonSecretEnv: map[string]string{"SUBDOMAIN": "trips"},
|
||||||
|
SecretEnvVars: []string{"DB_PASSWORD", "SECRET_KEY"},
|
||||||
|
DataKeyEnvVars: []string{"SECRET_KEY"},
|
||||||
|
PortableSecretEnvVars: names,
|
||||||
|
PortableSecrets: portable,
|
||||||
|
}
|
||||||
|
m := &Manager{
|
||||||
|
logger: log.New(io.Discard, "", 0),
|
||||||
|
systemDataPath: filepath.Join(tmp, "system"),
|
||||||
|
stackProvider: &fakeRecoveryProvider{info: info, hdd: drive},
|
||||||
|
version: "vtest",
|
||||||
|
}
|
||||||
|
if err := m.CaptureRecoveryUnit("app"); err != nil {
|
||||||
|
t.Fatalf("capture fixture: %v", err)
|
||||||
|
}
|
||||||
|
return drive
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRestoreFromRecoveryUnitWithGuestAbsent is D5's entire claim, as a test rather than a
|
||||||
|
// description: a Tier-1/2 restore SUCCEEDS when the guest's app.yaml is unavailable.
|
||||||
|
//
|
||||||
|
// SEAM (R-125): injection is at Manager.stackProvider only — i.e. the docker/compose operations and the
|
||||||
|
// guest's app.yaml decrypt. RecoverStackSecrets returning nil IS the guest being gone: it is exactly
|
||||||
|
// what the real adapter returns when the stack or its app.yaml cannot be read (main.go
|
||||||
|
// GetStack/LoadAppConfigDecrypted nil paths). Everything under test is production code: the unit was
|
||||||
|
// written by the real CaptureRecoveryUnit, read back by the real readUnitEnv, and reconciled by the
|
||||||
|
// real reconcileRestoreSecrets.
|
||||||
|
func TestRestoreFromRecoveryUnitWithGuestAbsent(t *testing.T) {
|
||||||
|
const (
|
||||||
|
dataKey = "deadbeefdeadbeef"
|
||||||
|
dbPw = "pw-from-the-drive"
|
||||||
|
)
|
||||||
|
drive := captureFixtureUnit(t, map[string]string{"DB_PASSWORD": dbPw, "SECRET_KEY": dataKey})
|
||||||
|
|
||||||
|
// The guest is GONE: no secrets recoverable from it at all.
|
||||||
|
fake := &fakeRecoveryProvider{hdd: drive, running: true, secrets: nil}
|
||||||
|
m := &Manager{logger: log.New(io.Discard, "", 0),
|
||||||
|
systemDataPath: filepath.Join(drive, "..", "sys"), stackProvider: fake}
|
||||||
|
|
||||||
|
if err := m.RestoreFromRecoveryUnit("app"); err != nil {
|
||||||
|
t.Fatalf("restore must succeed from the drive alone, got: %v", err)
|
||||||
|
}
|
||||||
|
if fake.gotEnv == nil {
|
||||||
|
t.Fatal("recreate was not called — the restore did not reach the redeploy")
|
||||||
|
}
|
||||||
|
// The consequence: the app is redeployed with the key that decrypts the data beside it.
|
||||||
|
if fake.gotEnv["SECRET_KEY"] != dataKey {
|
||||||
|
t.Errorf("data-encrypting key not recovered from the unit: %q", fake.gotEnv["SECRET_KEY"])
|
||||||
|
}
|
||||||
|
if fake.gotEnv["DB_PASSWORD"] != dbPw {
|
||||||
|
t.Errorf("DB password not recovered from the unit: %q", fake.gotEnv["DB_PASSWORD"])
|
||||||
|
}
|
||||||
|
if fake.gotEnv["SUBDOMAIN"] != "trips" {
|
||||||
|
t.Errorf("plain config lost: %v", fake.gotEnv)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestRestoreFromRecoveryUnitGuestAbsentStillFailsClosed proves D5 did not soften the gate: with the
|
||||||
|
// data key in NEITHER the unit nor the guest, the restore still REFUSES and mutates nothing.
|
||||||
|
func TestRestoreFromRecoveryUnitGuestAbsentStillFailsClosed(t *testing.T) {
|
||||||
|
// Unit carries only the DB password — the data key is absent from both sources.
|
||||||
|
drive := captureFixtureUnit(t, map[string]string{"DB_PASSWORD": "pw"})
|
||||||
|
fake := &fakeRecoveryProvider{hdd: drive, running: true, secrets: nil}
|
||||||
|
m := &Manager{logger: log.New(io.Discard, "", 0),
|
||||||
|
systemDataPath: filepath.Join(drive, "..", "sys"), stackProvider: fake}
|
||||||
|
|
||||||
|
err := m.RestoreFromRecoveryUnit("app")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected fail-closed refusal when the data key is in neither source")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "SECRET_KEY") {
|
||||||
|
t.Errorf("refusal should name the missing data key, got: %v", err)
|
||||||
|
}
|
||||||
|
if fake.gotEnv != nil {
|
||||||
|
t.Errorf("recreate must NOT be called on refusal, got %v", fake.gotEnv)
|
||||||
|
}
|
||||||
|
if fake.stopped {
|
||||||
|
t.Error("the live app must not be stopped when the restore refuses")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestRestoreFromRecoveryUnitOrchestration exercises the full in-process flow: read manifest →
|
// TestRestoreFromRecoveryUnitOrchestration exercises the full in-process flow: read manifest →
|
||||||
// recover secrets → apply gate → recreate with the reconciled env. It proves (a) on success the
|
// recover secrets → apply gate → recreate with the reconciled env. It proves (a) on success the
|
||||||
// recreate is called with non-secret env + recovered secrets merged, and (b) on a missing data-key the
|
// recreate is called with non-secret env + recovered secrets merged, and (b) on a missing data-key the
|
||||||
@@ -15,7 +128,7 @@ func TestRestoreFromRecoveryUnitOrchestration(t *testing.T) {
|
|||||||
newUnit := func(t *testing.T) (drive string) {
|
newUnit := func(t *testing.T) (drive string) {
|
||||||
tmp := t.TempDir()
|
tmp := t.TempDir()
|
||||||
drive = filepath.Join(tmp, "drive")
|
drive = filepath.Join(tmp, "drive")
|
||||||
// stripped (secret-free) app.yaml in the unit
|
// A schema-1 unit: no portable secrets, so the guest is the only source (pre-D5 behaviour).
|
||||||
mustWrite(t, filepath.Join(RecoveryUnitComposePath(drive, "app"), "app.yaml"),
|
mustWrite(t, filepath.Join(RecoveryUnitComposePath(drive, "app"), "app.yaml"),
|
||||||
"deployed: true\nenv:\n SUBDOMAIN: trips\n")
|
"deployed: true\nenv:\n SUBDOMAIN: trips\n")
|
||||||
man := &RecoveryManifest{SchemaVersion: 1, AppName: "app", ControllerVer: "v",
|
man := &RecoveryManifest{SchemaVersion: 1, AppName: "app", ControllerVer: "v",
|
||||||
@@ -49,6 +162,21 @@ func TestRestoreFromRecoveryUnitOrchestration(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
t.Run("schema-1 unit still restores from the guest — no regression", func(t *testing.T) {
|
||||||
|
// An old unit carries nothing; the guest must still be able to supply everything.
|
||||||
|
drive := newUnit(t)
|
||||||
|
fake := &fakeRecoveryProvider{hdd: drive, running: true,
|
||||||
|
secrets: map[string]string{"DB_PASSWORD": "pw", "SECRET_KEY": "deadbeef"}}
|
||||||
|
m := &Manager{logger: log.New(io.Discard, "", 0),
|
||||||
|
systemDataPath: filepath.Join(drive, "..", "sys"), stackProvider: fake}
|
||||||
|
if err := m.RestoreFromRecoveryUnit("app"); err != nil {
|
||||||
|
t.Fatalf("a pre-D5 unit must still restore: %v", err)
|
||||||
|
}
|
||||||
|
if fake.gotEnv["SECRET_KEY"] != "deadbeef" {
|
||||||
|
t.Errorf("guest fallback lost the data key: %v", fake.gotEnv)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
t.Run("data-key unrecoverable — REFUSED, recreate not called", func(t *testing.T) {
|
t.Run("data-key unrecoverable — REFUSED, recreate not called", func(t *testing.T) {
|
||||||
drive := newUnit(t)
|
drive := newUnit(t)
|
||||||
fake := &fakeRecoveryProvider{
|
fake := &fakeRecoveryProvider{
|
||||||
@@ -67,13 +195,82 @@ func TestRestoreFromRecoveryUnitOrchestration(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestReconcileRestoreSecretsPrecedence pins the D5 precedence rule in BOTH directions. An undefined
|
||||||
|
// precedence between two sources of a decryption key is a data-loss bug waiting for its first
|
||||||
|
// disagreement, so this is not a style question.
|
||||||
|
//
|
||||||
|
// The UNIT wins: its secrets were captured in the same run as the dumps beside them, so the unit's
|
||||||
|
// value is the one that matches the data about to be restored. The guest's value is merely the most
|
||||||
|
// recent — and a rotated key does not decrypt data encrypted with the old one.
|
||||||
|
func TestReconcileRestoreSecretsPrecedence(t *testing.T) {
|
||||||
|
nonSecret := map[string]string{"SUBDOMAIN": "trips"}
|
||||||
|
names := []string{"DB_PASSWORD", "SECRET_KEY"}
|
||||||
|
dataKeys := []string{"SECRET_KEY"}
|
||||||
|
|
||||||
|
t.Run("both sources disagree — the UNIT wins", func(t *testing.T) {
|
||||||
|
unit := map[string]string{"DB_PASSWORD": "unit-pw", "SECRET_KEY": "unit-key"}
|
||||||
|
guest := map[string]string{"DB_PASSWORD": "guest-pw", "SECRET_KEY": "guest-key"}
|
||||||
|
full, missing, err := reconcileRestoreSecrets(nonSecret, unit, guest, names, dataKeys)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if len(missing) != 0 {
|
||||||
|
t.Errorf("missing: %v", missing)
|
||||||
|
}
|
||||||
|
if full["SECRET_KEY"] != "unit-key" {
|
||||||
|
t.Errorf("data key: got %q, want the UNIT's value (it matches the restored data)", full["SECRET_KEY"])
|
||||||
|
}
|
||||||
|
if full["DB_PASSWORD"] != "unit-pw" {
|
||||||
|
t.Errorf("DB password: got %q, want the UNIT's value (it matches the restored data dir hash)", full["DB_PASSWORD"])
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("unit silent — the GUEST fills in", func(t *testing.T) {
|
||||||
|
// The withheld class (admin logins) is never in the unit, so this direction must work too.
|
||||||
|
guest := map[string]string{"DB_PASSWORD": "guest-pw", "SECRET_KEY": "guest-key"}
|
||||||
|
full, _, err := reconcileRestoreSecrets(nonSecret, nil, guest, names, dataKeys)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if full["SECRET_KEY"] != "guest-key" || full["DB_PASSWORD"] != "guest-pw" {
|
||||||
|
t.Errorf("guest fallback not applied: %v", full)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("unit present but EMPTY for a name — the guest fills in", func(t *testing.T) {
|
||||||
|
// An empty value is not a value; it must not shadow a good one from the guest.
|
||||||
|
unit := map[string]string{"SECRET_KEY": ""}
|
||||||
|
guest := map[string]string{"SECRET_KEY": "guest-key", "DB_PASSWORD": "guest-pw"}
|
||||||
|
full, _, err := reconcileRestoreSecrets(nonSecret, unit, guest, names, dataKeys)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("an empty unit value must fall through to the guest, got: %v", err)
|
||||||
|
}
|
||||||
|
if full["SECRET_KEY"] != "guest-key" {
|
||||||
|
t.Errorf("empty unit value shadowed the guest: %q", full["SECRET_KEY"])
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("a portable secret never shadows plain config", func(t *testing.T) {
|
||||||
|
// GetStackRecoveryInfo keeps the two sets disjoint; if that ever breaks, the merge order in
|
||||||
|
// buildUnitAppYaml decides silently. Pin the intended outcome.
|
||||||
|
full, _, err := reconcileRestoreSecrets(map[string]string{"DB_PASSWORD": "should-not-win"},
|
||||||
|
map[string]string{"DB_PASSWORD": "unit-pw"}, nil, []string{"DB_PASSWORD"}, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if full["DB_PASSWORD"] != "unit-pw" {
|
||||||
|
t.Errorf("the secret source must win over a stray non-secret entry: %q", full["DB_PASSWORD"])
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// TestReconcileRestoreSecrets covers the safety-critical fail-closed gate + secret reconciliation.
|
// TestReconcileRestoreSecrets covers the safety-critical fail-closed gate + secret reconciliation.
|
||||||
func TestReconcileRestoreSecrets(t *testing.T) {
|
func TestReconcileRestoreSecrets(t *testing.T) {
|
||||||
nonSecret := map[string]string{"SUBDOMAIN": "trips", "DOMAIN": "demo-felhom.eu"}
|
nonSecret := map[string]string{"SUBDOMAIN": "trips", "DOMAIN": "demo-felhom.eu"}
|
||||||
|
|
||||||
t.Run("all recovered, no data_key — full env, no error", func(t *testing.T) {
|
t.Run("all recovered, no data_key — full env, no error", func(t *testing.T) {
|
||||||
recovered := map[string]string{"DB_PASSWORD": "pw", "SECRET_KEY": "deadbeef"}
|
guest := map[string]string{"DB_PASSWORD": "pw", "SECRET_KEY": "deadbeef"}
|
||||||
full, missing, err := reconcileRestoreSecrets(nonSecret, recovered,
|
full, missing, err := reconcileRestoreSecrets(nonSecret, nil, guest,
|
||||||
[]string{"DB_PASSWORD", "SECRET_KEY"}, nil)
|
[]string{"DB_PASSWORD", "SECRET_KEY"}, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("unexpected error: %v", err)
|
t.Fatalf("unexpected error: %v", err)
|
||||||
@@ -87,9 +284,9 @@ func TestReconcileRestoreSecrets(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
t.Run("data_key missing — FAIL CLOSED (refuse)", func(t *testing.T) {
|
t.Run("data_key missing from BOTH sources — FAIL CLOSED (refuse)", func(t *testing.T) {
|
||||||
recovered := map[string]string{"DB_PASSWORD": "pw"} // SECRET_KEY (a data_key) is gone
|
guest := map[string]string{"DB_PASSWORD": "pw"} // SECRET_KEY (a data_key) is gone
|
||||||
full, _, err := reconcileRestoreSecrets(nonSecret, recovered,
|
full, _, err := reconcileRestoreSecrets(nonSecret, nil, guest,
|
||||||
[]string{"DB_PASSWORD", "SECRET_KEY"}, []string{"SECRET_KEY"})
|
[]string{"DB_PASSWORD", "SECRET_KEY"}, []string{"SECRET_KEY"})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected fail-closed error for missing data-encrypting key, got nil")
|
t.Fatal("expected fail-closed error for missing data-encrypting key, got nil")
|
||||||
@@ -99,17 +296,34 @@ func TestReconcileRestoreSecrets(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
t.Run("data_key empty value — FAIL CLOSED", func(t *testing.T) {
|
t.Run("data_key empty in both — FAIL CLOSED", func(t *testing.T) {
|
||||||
recovered := map[string]string{"SECRET_KEY": ""} // present but empty == unrecoverable
|
guest := map[string]string{"SECRET_KEY": ""} // present but empty == unrecoverable
|
||||||
_, _, err := reconcileRestoreSecrets(nonSecret, recovered, []string{"SECRET_KEY"}, []string{"SECRET_KEY"})
|
_, _, err := reconcileRestoreSecrets(nonSecret, map[string]string{"SECRET_KEY": ""}, guest,
|
||||||
|
[]string{"SECRET_KEY"}, []string{"SECRET_KEY"})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("empty data-key value must fail closed")
|
t.Fatal("empty data-key value must fail closed")
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
t.Run("data_key recovered from the UNIT — no refusal", func(t *testing.T) {
|
||||||
|
// The D5 case: the guest is gone but the unit carries the key, so the gate must NOT fire.
|
||||||
|
unit := map[string]string{"SECRET_KEY": "deadbeef", "DB_PASSWORD": "pw"}
|
||||||
|
full, missing, err := reconcileRestoreSecrets(nonSecret, unit, nil,
|
||||||
|
[]string{"DB_PASSWORD", "SECRET_KEY"}, []string{"SECRET_KEY"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("the unit's data key must satisfy the gate: %v", err)
|
||||||
|
}
|
||||||
|
if len(missing) != 0 {
|
||||||
|
t.Errorf("nothing should be missing: %v", missing)
|
||||||
|
}
|
||||||
|
if full["SECRET_KEY"] != "deadbeef" {
|
||||||
|
t.Errorf("data key wrong: %v", full)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
t.Run("resettable secret missing — proceed with warning", func(t *testing.T) {
|
t.Run("resettable secret missing — proceed with warning", func(t *testing.T) {
|
||||||
recovered := map[string]string{"SECRET_KEY": "deadbeef"} // data_key ok; DB_PASSWORD missing
|
guest := map[string]string{"SECRET_KEY": "deadbeef"} // data_key ok; DB_PASSWORD missing
|
||||||
full, missing, err := reconcileRestoreSecrets(nonSecret, recovered,
|
full, missing, err := reconcileRestoreSecrets(nonSecret, nil, guest,
|
||||||
[]string{"DB_PASSWORD", "SECRET_KEY"}, []string{"SECRET_KEY"})
|
[]string{"DB_PASSWORD", "SECRET_KEY"}, []string{"SECRET_KEY"})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("a missing resettable secret must NOT fail closed: %v", err)
|
t.Fatalf("a missing resettable secret must NOT fail closed: %v", err)
|
||||||
@@ -125,3 +339,38 @@ func TestReconcileRestoreSecrets(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestReadUnitEnvSplitsByManifest proves the split is driven by the manifest's portable names, and that
|
||||||
|
// a schema-1 unit (no names) degrades to "everything is plain config" rather than losing entries.
|
||||||
|
func TestReadUnitEnvSplitsByManifest(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
path := filepath.Join(dir, "app.yaml")
|
||||||
|
mustWrite(t, path, "deployed: true\nenv:\n SUBDOMAIN: trips\n DB_PASSWORD: pw\n SECRET_KEY: key\n")
|
||||||
|
|
||||||
|
t.Run("named secrets land in unitSecrets, the rest in nonSecret", func(t *testing.T) {
|
||||||
|
nonSecret, unitSecrets := readUnitEnv(path, []string{"DB_PASSWORD", "SECRET_KEY"})
|
||||||
|
if nonSecret["SUBDOMAIN"] != "trips" || len(nonSecret) != 1 {
|
||||||
|
t.Errorf("nonSecret = %v, want only SUBDOMAIN", nonSecret)
|
||||||
|
}
|
||||||
|
if unitSecrets["DB_PASSWORD"] != "pw" || unitSecrets["SECRET_KEY"] != "key" || len(unitSecrets) != 2 {
|
||||||
|
t.Errorf("unitSecrets = %v, want the two named secrets", unitSecrets)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("schema-1 (no portable names) — everything is plain config, nothing lost", func(t *testing.T) {
|
||||||
|
nonSecret, unitSecrets := readUnitEnv(path, nil)
|
||||||
|
if len(unitSecrets) != 0 {
|
||||||
|
t.Errorf("a schema-1 unit carries no secrets, got %v", unitSecrets)
|
||||||
|
}
|
||||||
|
if len(nonSecret) != 3 {
|
||||||
|
t.Errorf("nonSecret should keep every entry, got %v", nonSecret)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("absent file — empty maps, no panic", func(t *testing.T) {
|
||||||
|
nonSecret, unitSecrets := readUnitEnv(filepath.Join(dir, "nope.yaml"), []string{"DB_PASSWORD"})
|
||||||
|
if len(nonSecret) != 0 || len(unitSecrets) != 0 {
|
||||||
|
t.Errorf("want empty maps, got %v / %v", nonSecret, unitSecrets)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|||||||
@@ -822,6 +822,48 @@ func SensitiveEnvVars(meta *Metadata) []string {
|
|||||||
return vars
|
return vars
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// nonPortableSecrets is the register of secrets that must NEVER travel in an on-drive recovery unit
|
||||||
|
// even though the catalog types them `secret` — i.e. credentials whose reach is NOT bounded by
|
||||||
|
// physical possession of the drive, because they authenticate against a service published to the
|
||||||
|
// internet. Keyed by catalog SLUG (never empty — LoadMetadata falls back to the directory name).
|
||||||
|
//
|
||||||
|
// It is a CODE register, not a catalog flag, deliberately: the D5 ruling is a security boundary, and
|
||||||
|
// a boundary a catalog push can silently move is not a boundary (the R-97a lesson — an invariant that
|
||||||
|
// only configuration enforced). Adding an app whose `type: secret` field gates an internet-reachable
|
||||||
|
// login means adding a row here.
|
||||||
|
//
|
||||||
|
// - vaultwarden/ADMIN_TOKEN gates the /admin panel, served on the app's own public web port.
|
||||||
|
var nonPortableSecrets = map[string]map[string]bool{
|
||||||
|
"vaultwarden": {"ADMIN_TOKEN": true},
|
||||||
|
}
|
||||||
|
|
||||||
|
// PortableSecretEnvVars returns the env-var names of secrets that TRAVEL inside the on-drive recovery
|
||||||
|
// unit (D5), in deterministic metadata order.
|
||||||
|
//
|
||||||
|
// The ruling (operator, 2026-07-30): `type: secret` travels, `type: password` does not, minus
|
||||||
|
// nonPortableSecrets. The line is drawn on REACH, not on whether a secret is nominally resettable:
|
||||||
|
//
|
||||||
|
// - Every `type: secret` field either decrypts data sitting on the SAME drive (the 5 declared
|
||||||
|
// data_keys, plus encryption keys the catalog labels as such but never flagged — see R-127) or
|
||||||
|
// authenticates to a container on an internal compose network with no external listener (the 18
|
||||||
|
// DB/root passwords, and the signing secrets). Possessing it adds nothing to possessing the
|
||||||
|
// drive, which is exactly D2's argument for keeping the DATA plaintext.
|
||||||
|
// - Every `type: password` field is an admin/UI login for a published service, so its blast radius
|
||||||
|
// is NOT bounded by the drive. Those stay in the guest and are regenerated on restore (O4).
|
||||||
|
//
|
||||||
|
// Excluding the `type: password` class is what licenses the plaintext ruling; the two are coupled and
|
||||||
|
// must not be relaxed independently.
|
||||||
|
func PortableSecretEnvVars(meta *Metadata) []string {
|
||||||
|
blocked := nonPortableSecrets[meta.Slug]
|
||||||
|
var vars []string
|
||||||
|
for _, f := range meta.DeployFields {
|
||||||
|
if f.Type == "secret" && !blocked[f.EnvVar] {
|
||||||
|
vars = append(vars, f.EnvVar)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return vars
|
||||||
|
}
|
||||||
|
|
||||||
// --- Secret generation ---
|
// --- Secret generation ---
|
||||||
|
|
||||||
const alphanumChars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
|
const alphanumChars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
|
||||||
|
|||||||
@@ -0,0 +1,76 @@
|
|||||||
|
package stacks
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
// TestPortableSecretEnvVars pins the D5 ruling (operator, 2026-07-30) about WHICH secrets may travel
|
||||||
|
// on the customer's backup drive: `type: secret` travels, `type: password` does not, minus the
|
||||||
|
// nonPortableSecrets register.
|
||||||
|
//
|
||||||
|
// This is a security boundary, so the test asserts the CONSEQUENCE in both directions — what travels
|
||||||
|
// and, more importantly, what must not. A change that widens the class fails here.
|
||||||
|
func TestPortableSecretEnvVars(t *testing.T) {
|
||||||
|
t.Run("type=secret travels, type=password does not", func(t *testing.T) {
|
||||||
|
meta := &Metadata{Slug: "paperless-ngx", DeployFields: []DeployField{
|
||||||
|
{EnvVar: "DB_PASSWORD", Type: "secret"},
|
||||||
|
{EnvVar: "PAPERLESS_SECRET_KEY", Type: "secret"},
|
||||||
|
{EnvVar: "PAPERLESS_ADMIN_PASSWORD", Type: "password"},
|
||||||
|
{EnvVar: "SUBDOMAIN", Type: "subdomain"},
|
||||||
|
}}
|
||||||
|
got := PortableSecretEnvVars(meta)
|
||||||
|
want := []string{"DB_PASSWORD", "PAPERLESS_SECRET_KEY"}
|
||||||
|
if len(got) != len(want) {
|
||||||
|
t.Fatalf("portable = %v, want %v", got, want)
|
||||||
|
}
|
||||||
|
for i := range want {
|
||||||
|
if got[i] != want[i] { // order must be stable metadata order (checksum-skip depends on it)
|
||||||
|
t.Errorf("portable[%d] = %q, want %q (got %v)", i, got[i], want[i], got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("an internet-reachable admin login is WITHHELD even when typed secret", func(t *testing.T) {
|
||||||
|
meta := &Metadata{Slug: "vaultwarden", DeployFields: []DeployField{
|
||||||
|
{EnvVar: "ADMIN_TOKEN", Type: "secret"},
|
||||||
|
}}
|
||||||
|
if got := PortableSecretEnvVars(meta); len(got) != 0 {
|
||||||
|
t.Errorf("vaultwarden/ADMIN_TOKEN must NOT travel (it gates the public /admin panel), got %v", got)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("the register is scoped to its app, not the bare env-var name", func(t *testing.T) {
|
||||||
|
// Another app's ADMIN_TOKEN is not vaultwarden's and must not be caught by the register.
|
||||||
|
meta := &Metadata{Slug: "some-other-app", DeployFields: []DeployField{
|
||||||
|
{EnvVar: "ADMIN_TOKEN", Type: "secret"},
|
||||||
|
}}
|
||||||
|
if got := PortableSecretEnvVars(meta); len(got) != 1 || got[0] != "ADMIN_TOKEN" {
|
||||||
|
t.Errorf("register must be slug-scoped, got %v", got)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("no deploy fields — nothing travels", func(t *testing.T) {
|
||||||
|
if got := PortableSecretEnvVars(&Metadata{Slug: "x"}); len(got) != 0 {
|
||||||
|
t.Errorf("want empty, got %v", got)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("portable is a SUBSET of SensitiveEnvVars", func(t *testing.T) {
|
||||||
|
// The unit's app.yaml splits env into non-secret + portable using two different helpers; if
|
||||||
|
// PortableSecretEnvVars ever returned a name SensitiveEnvVars does not, that name would land in
|
||||||
|
// NonSecretEnv as well and the disjointness GetStackRecoveryInfo relies on would break.
|
||||||
|
meta := &Metadata{Slug: "nextcloud", DeployFields: []DeployField{
|
||||||
|
{EnvVar: "DB_PASSWORD", Type: "secret"},
|
||||||
|
{EnvVar: "MYSQL_ROOT_PASSWORD", Type: "secret"},
|
||||||
|
{EnvVar: "NEXTCLOUD_ADMIN_PASSWORD", Type: "password"},
|
||||||
|
{EnvVar: "DOMAIN", Type: "domain"},
|
||||||
|
}}
|
||||||
|
sensitive := make(map[string]bool)
|
||||||
|
for _, n := range SensitiveEnvVars(meta) {
|
||||||
|
sensitive[n] = true
|
||||||
|
}
|
||||||
|
for _, n := range PortableSecretEnvVars(meta) {
|
||||||
|
if !sensitive[n] {
|
||||||
|
t.Errorf("%q is portable but not sensitive — it would also land in NonSecretEnv", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user