D5 SHIPPED: Tier-1/2 restore no longer depends on the whole-guest tier

Records controller v0.188.0 across the four coupled artifacts.

07-backup-architecture.md is the owning doc:
- new 7.4 = the recovery chain AFTER D5 (7.1 leg 1 superseded; leg 2,
  the living-app dependency, explicitly unchanged so this is not read
  as more than it is)
- 7.3 collapsed to history, with the correction that the target as
  written (data_key-only) was tested in Part 0 and rejected
- 3 records that the two-lane split is now real, not just intended
- matrix rows 3 / 3c (new) / 13; 10.1 D5 itself shipped

Also: new capability-map row, D5 collapsed in ROADMAP + OPEN-ITEMS,
and R-127 filed in both (data_key flag unreliable; O4 can regenerate a
DB password that no longer matches the restored data directory).

The audit is named D5-drive-alone-restore rather than "...secrets..."
because .gitignore blocks *secret* -- a guard worth respecting, not
forcing past.
This commit is contained in:
2026-07-30 16:58:06 +02:00
parent d42d90fed7
commit 0a9bd3829d
5 changed files with 379 additions and 8 deletions
+2
View File
@@ -109,6 +109,8 @@
| R-106 | ~~The DR recipe records the PBS namespace as `"root"` on every box~~ | XS | **SHIPPED** — agent v0.118.1, 2026-07-30 | Namespace resolves from the pbs STORAGE (storage.cfg `namespace`) — the field `vzdump --storage <pbs>` makes PVE read — not from the listed snapshot, whose `ns` PBS omits once the list is `?ns=`-scoped. `namespace_state` distinguishes a genuine root namespace from not knowing. Live on both boxes. Follow-on: **R-124** (the `"root"` spelling is not `--ns`-pasteable). `audits/R106-R109-recipe-completeness-2026-07-30.md`; flipped `07` §4 |
| R-107 | **No offsite action unpacks the named-volume tars Tier-3 captures on every run.** `ReconstituteFromOffsite` skips the unit outright (`internal/backup/offbox_reconstitute.go:284-289`); `PlaceOffsiteRestore` places it only when the live unit is ABSENT (`internal/backup/offbox_restore.go:352-356`). The DB **is** replayed, from the scratch unit's `db-dumps/` | M | READY — 2026-07-28 | 52 of 53 catalog templates keep data in Docker named volumes, so for most apps the offsite tier carries the volume data and cannot itself put it back. A two-step route exists (full restore → place → Tier-1 restore, which does unpack tars) but **no single action does it and no UI routes it** — the R-103 shape one tier over. Sibling of R-102: both are "captured and never read". Flips: the offsite-restore row in map §C; `07` §6.3, §7.2 |
| R-108 | ~~Network storage can host an app's namespace, and FileBrowser binds a network share at its ROOT — this BLOCKS D5~~ | M | **SHIPPED** — controller v0.187.0, 2026-07-30 | **D5 UNBLOCKED.** One fail-closed predicate (`Settings.RefuseAsAppNamespace`) consulted by every placement surface, so no `backups/` tree can exist inside the share-root bind. The bind was deliberately NOT narrowed — it is load-bearing (`:rslave` automount wake, 2026-07-22 probe) and unscopable (apps on a share store at `<share>/<app>`; creating a `userdata/` layer would write Felhom convention onto a customer's NAS). **Five surfaces, not the four this row named** — decommission-with-migrate guarded only its SOURCE; and the boundary is the deploy POST, not the dropdown. Zero apps on network storage fleet-wide, so nothing stranded. The `.fab`-onto-NAS "second effect" is split out as **R-126**. Flipped: `07` §7.3 (D5 → unblocked), §10.1 (CLOSED), `00-capability-map.md` §D (new PROVEN-LIVE row). `audits/R108-network-app-namespace-2026-07-30.md` |
| R-127 | **`data_key: true` is unreliable (4+ encryption keys unflagged, contradicting the catalog's own labels), and O4 can regenerate a DB password that no longer matches the restored data directory** | S/M | READY — NEW 2026-07-30 | Found by **D5's Part 0**, and the reason D5's boundary became `type: secret` rather than `data_key`. Leg (a): flag the missing keys (catalog-only) + pin flag-vs-label agreement; the residual risk after D5 is that the **fail-closed gate** keys on `data_key`, so an unflagged key missing from both sources lets the restore proceed onto undecryptable data. Leg (b): a regenerated DB password is silently wrong — `POSTGRES_PASSWORD` is ignored once PGDATA is non-empty, so the app cannot authenticate while the replay still succeeds over the local trust socket (proven live on `postgres:16-alpine`). v0.188.0 corrected the false "stored data is unaffected" WARN but added no guard. Flips: `07` §7.4 |
| D5 | ~~**Move app secrets into the LOCAL recovery unit** so Tier-1/Tier-2 restore stop needing the guest and stop needing R~~ | M | **SHIPPED + PROVEN-LIVE** — controller v0.188.0, 2026-07-30 | **The arc's architectural centrepiece. Tier-1/2 no longer depend on the whole-guest tier — a customer needs the DRIVE AND NOTHING ELSE.** Part 0 tested this row's own premise and **rejected** it: data-keys-only is both insufficient and unsafe, because `data_key` is unreliable (→ **R-127**) and a DB password is not resettable in practice (`POSTGRES_PASSWORD` is ignored once PGDATA is non-empty, so a regenerated value leaves the app unable to reach its own restored rows while the dump replay still reports success — proven on `postgres:16-alpine`). **Operator ruling: `type: secret` travels (45 fields), `type: password` never (7) plus a code register (`vaultwarden/ADMIN_TOKEN`); plaintext, because withholding the internet-reachable class is what licenses it — the two are coupled.** `stacks.PortableSecretEnvVars` is the single boundary; the register is code, not a catalog flag (R-97a). **Precedence: the UNIT WINS** (its secrets match the data being restored, not merely the newest), pinned both directions. Fail-closed data-key gate UNCHANGED. Manifest schema 2; schema-1 units still restore. Proven live on a scratch drill guest: AdventureLog restored with the guest `app.yaml` moved aside (`secrets recovered=2/2`, 27.6 s) and **the app read the seeded row over TCP with its own credential**; Grafana's admin password withheld with **0 hits** across the backup namespace. 4 red-proofs each verified to land. `audits/D5-drive-alone-restore-2026-07-30.md` Flips `07` §3/§7.1/§7.3/§7.4/§8/§10.1 + a new capability-map row |
| R-126 | **A `.fab` bundle — plaintext secrets, optional password — can be exported ONTO a NAS.** `storageDriveList()` (`internal/web/handler_export.go`) does not filter network paths | S | READY — 2026-07-30 | Split out of R-108 on its closure. NOT a D5 precondition: an explicit customer-chosen export destination, not a browsing surface reaching a backup tree (`07` §7.3 records the reasoning). Fix = filter network paths from the export destination list, or force the bundle password when the destination is a share. Flips: `07` §5 |
| R-109 | ~~The DR recipe records no backup target~~ | XS | **SHIPPED** — agent v0.118.1 + hub v0.83.0, 2026-07-30 | `backup_target` resolves from the PRIMARY tier of `cfg.Backup.BackupTiers()` (the scheduler's own function) + the mountpoint that separates `/mnt/hdd_1` from `/var/lib/vz`; unresolvable is recorded as unresolvable, never defaulted. Required an unscoped HUB half: `AssembleDRRecipe` allow-lists top-level keys → **R-122**, a fourth recipe-completeness defect that had already shipped (`offsite_restic` dropped for the feature's whole life). `audits/R106-R109-recipe-completeness-2026-07-30.md`; flipped `07` §4 |
| E-2 | **Drive-role machinery around the moved vzdump target.** The 2026-07-28 runbook proved the architecture change by hand on both demo boxes; this is the machinery: a **backup-target role** on `StoragePath` beside `Schedulable`/`IsDefault`/`Kind`; **assignment in the storage wizard** (suggest by attribute, refuse the absurd, never decide by transport or `removable` — on the reference hardware demo-felhom's target IS a USB HDD and BOTH drives report `removable=0`); **unassigned drives do nothing automatically**; **stickiness** (never silently retarget); `felhom-host-install.sh` creating the target with `--is_mountpoint 1` **and** issuing the `FelhomAgentStore` ACL; **absent-target policy**; **retention/space accounting** on a drive the customer shares; the honest **single-drive label**; remaining fleet migration | M | READY — 2026-07-28 | Full scope + rationale in `runbooks/RUNBOOK-vzdump-target-move-2026-07-29.md` §7. Two traps already paid for live: the storage `path` must BE the mountpoint or the agent reports the target `disconnected` forever (`internal/storage/observe.go:321`), and the per-storage `FelhomAgentStore` grant is mandatory or every backup 403s. Absent-drive behaviour today is **fail-loudly, no silent retarget** (`is_mountpoint 1` proven live) — which is NOT the intended fall-back-and-alarm design. Flips: matrix row 4 |