R-108 CLOSED — D5's precondition is met (controller v0.187.0)
Four-artifact update per the coupling rule, plus the audit. 07-backup-architecture.md: §10.1 retitled CLOSED with the ruling and the D5 sentence; the FileBrowser network-share row flipped YES->NO, closed at the PLACEMENT rather than at the bind; the exposure chain annotated with the fifth surface (decommission-with-migrate guarded only its source) and the correction that the boundary is the deploy POST, not the dropdown; §7.3 retitled UNBLOCKED; register row collapsed; open question F answered. 00-capability-map.md: new §D row PROVEN-LIVE, with the un-exercised legs named — the deploy-POST and decommission refusals are unit-tested, not live-fired. OPEN-ITEMS.md: R-108 dispositioned; D5 given its OWN row as READY/UNBLOCKED (it had existed only inside other rows' prose — the R-123 thread-loss pattern); R-126 registered. ROADMAP.md: R-108 collapsed to a shipped one-liner; R-126 added. R-126 filed not fixed: a .fab bundle (plaintext secrets, optional password) can be exported ONTO a NAS. Split out of R-108 rather than folded in — it is an explicit customer-chosen export destination, not a browsing surface reaching a backup tree, so it was never part of D5's precondition. Live evidence: same-box before/after on demo-felhom through the real authenticated endpoint, the network-specific refusal on demo-hp, non-effect verified in the registry, and R-67's share-root bind diffed byte-identical across the deploy.
This commit is contained in:
@@ -47,7 +47,9 @@ State: `BLOCKED` · `READY` · `WAITING-ON-OPERATOR` · `WATCHING`. Every row ha
|
||||
| **C9-F3** → **R-104** | An **interrupted offsite run leaves an exclusive restic lock the self-heal cannot reach**: `resticStep` (`offbox.go:634-648`) has `unlock --remove-all`, but `ensureOffboxRepo`'s probe fails first, `classifyResticProbe` (`offbox.go:77-93`) has no lock case → `"other"` → fail-fast. Tier dead until a human unlocks; `ClassifyOffsiteFailure` likewise has no lock case so the operator is told **„A távoli mentés ismeretlen okból nem sikerült"** for a precisely-known, self-healable condition | **READY (MEDIUM)** | — | Add a lock case to both classifiers and let the probe path escalate to `unlock --remove-all`. Answers Phase C item 8: the repo is NOT usable after a killed run. Cleared manually this run; tier proven working again (`ok`, 1m35s). Reachable by any interruption — container restart, OOM, **host reboot mid-backup** | CC |
|
||||
| **C9-F1b** → **R-103** | Tier-2's restore cannot cover 43 of 53 apps; the action that CAN is the keep-side unit restore (`POST /backup/restore` → `RestoreFromRecoveryUnit`, replays volume tars + DB dumps). v0.183.0 NAMES it in the refusal text but does not route to it | **READY** | — | Put the working action in the card the customer already opened. **Deliberately its own task:** it places a DESTRUCTIVE operation (overwrites live data with the backup state) behind a button reached via a NON-destructive one, so the confirm copy must carry that difference — the reason it was not folded into v0.183.0 | CC |
|
||||
| **C9-F4** → **R-102** | **Nothing reads the Tier-2 copy's `recovery-unit/` mirror.** It is written by EVERY Tier-2 run (`tier2.go:369`, „Unit leg (always)") and read by no code path: `RecoveryUnitPath` resolves to `backups/**primary**/` (`appbackup/paths.go:46-48`), and the only reader of the secondary tree is `tier2_restore.go:79`, which reads `hdd/`+`userdata/` only | **READY (potentially > C9-F1)** | — | Tier-2 exists for the case where the PRIMARY drive is lost — and in exactly that case the primary unit is gone while this mirror survives on the second drive, unreachable by any customer action, leaving offsite as the only route. Verified by enumeration: 6 references to `"secondary"` in the tree, one writer, one reader, one wipe-warning lister | CC |
|
||||
| **R-108** | **Network storage can host an app's namespace, and FileBrowser binds a network share at its ROOT.** Local drives are `userdata`-scoped (`web/handlers.go:2450-2460`); network paths are bound at the share root (`:2432`) and served with `download: true`. No `IsNetwork()` filter guards the deploy dropdown (`settings.go:904-914`), the per-app migrate targets (`handlers.go:674-679`), or `handleStorageMigrateApp` (`storage_handlers.go:410-424` — its whole-namespace sibling DOES refuse, `:397`) | **READY — BLOCKS an architectural target** | — | **This is why D5 was not adopted** in the 2026-07-28 `07-backup-architecture.md` rewrite: D5 moves app secrets into the local recovery unit so Tier-1/Tier-2 restore stop needing the guest, and that is safe only if no browsing surface can reach the backup tree. Every other surface was verified clean (SMB both namespace shapes, FileBrowser for drives, `.fab` import + download, `/api/debug/*`, all three `ServeFile` sites, storage-path add) — `07` §10.1 has the full sweep. **Not a leak today** (the unit's `app.yaml` is secret-stripped). Verified LIVE in demo-hp's generated FileBrowser compose | CC |
|
||||
| **R-108** | ~~Network storage can host an app's namespace, and FileBrowser binds a network share at its ROOT~~ | **SHIPPED + PROVEN-LIVE** (controller v0.187.0, 2026-07-30) | — | **D5 IS UNBLOCKED — that is this item's whole point.** `RefuseAsAppNamespace` (`internal/settings/settings.go`) is one fail-closed predicate consulted by every placement surface, so no `backups/` tree can exist inside the share-root bind. **The fix inverted the obvious one:** the bind was NOT narrowed, because it cannot be — the `:rslave` share-ROOT bind is load-bearing (a 2026-07-22 probe proved an in-container access through it wakes the idle automount trigger), there is no `userdata/` layer to scope to (apps on a share store at `<share>/<app>`), and creating one would write Felhom convention onto a customer's own NAS, which R-67 forbids. Operator ruling: refuse the placement, keep the browse bind. **FIVE surfaces, not the four this row named** — `handleStorageDecommission` mode=migrate guarded only its SOURCE, so a whole namespace could be decommissioned ONTO a NAS; and the *boundary* is the deploy **POST**, not the dropdown (it takes any caller-supplied `HDD_PATH`, validated only by `os.Stat`). **Nothing stranded:** zero apps on network storage across all six customers incl. Peti. R-67's bind verified byte-identical before/after. Supersedes the controller README's "NAS backup locality — decision A". `audits/R108-network-app-namespace-2026-07-30.md` | — |
|
||||
| **D5** | **Move app secrets into the LOCAL recovery unit** so Tier-1/Tier-2 restore stop needing the guest and stop needing R (R would then be required for offsite recovery + host identity only) | **READY — UNBLOCKED 2026-07-30** (was blocked by R-108) | — | **The precondition is now met and stated in `07-backup-architecture.md` §7.3 + §10.1:** no browsing surface can reach the backup tree on any storage class. Every other read surface was already clear in the §10.1 sweep (SMB both namespace shapes, FileBrowser for drives, `.fab` import + download, `/api/debug/*`, all three `ServeFile` sites, storage-path add); the network-share bind was the one exception and R-108 closed it at the PLACEMENT rather than at the bind. **Registered as its own row 2026-07-30** — it previously existed only inside R-108's and E-2's prose, which is the R-123 thread-loss pattern. **NOT started:** unblocking is not implementing, and the task that closed R-108 was explicitly fenced from starting D5. Scope note: `.fab` bundles carry plaintext secrets independently of D5 (→ **R-126**) and are not part of its precondition | CC |
|
||||
| **R-126** | **A `.fab` bundle — plaintext secrets, OPTIONAL password — can be exported ONTO a NAS.** `storageDriveList()` (`internal/web/handler_export.go`) does not filter network paths | READY (S) | — | Split out of R-108, which closed without it: this is an explicit customer-chosen **export destination**, not a browsing surface reaching a backup tree, so it was never part of D5's precondition (`07` §7.3 records that reasoning). Was recorded inside R-108's row as its "second effect, independent of D5"; promoted to its own row so it does not vanish with R-108's closure. Fix = filter network paths out of the export destination list, or require the bundle password when the destination is a share | CC |
|
||||
| **F-DIAG** | ~~Four distinct offsite failure causes collapse into two operator-visible strings~~ | **SHIPPED** (controller v0.182.0, 2026-07-28) | — | `ClassifyOffsiteFailure` → quota / orphaned / no_repo / no_units / transport / **unknown**, each with its own Hungarian message. Unclassifiable says so rather than being folded into a neighbour. **Secrets:** the old message was a raw `err.Error()` passthrough carrying `sftp:<user>@<host>:<path>`; redaction is now by the target's **actual** host/user/path (a first regex-only attempt leaked on a bare hostname and its own test caught it). Unit-proven; **not** yet exercised by a live offsite failure of each class | — |
|
||||
| **F-OPS** | ~~A manual `pct restore` inherits the source guest's bind mounts — during a real DR, on a different host, under pressure~~ | **DOCUMENTED** (2026-07-28) | — | `documentation/runbooks/RUNBOOK-manual-guest-restore.md`: which `mpN` are volumes vs host binds, the `mp9` source-VMID trap (it can bind **another guest's bootstrap credentials**), strip-and-re-add before first boot, and a positive pre-start verification. Docs only by design — the agent already neutralises binds on its own restore paths, and a second implementation would drift | — |
|
||||
| **F-REBOOT** | ~~A guest rebooted during its backup does not come back — shutdown completes, start never happens, no self-heal; 9m47s total appliance outage with every alarm silent~~ | **SHIPPED + PROVEN-LIVE** (agent v0.107.0, 2026-07-28) | — | 60 s guest-power watchdog; `onboot` is the deliberate-stop discriminator (already the stale-lock path's, and what `pve-guests` consults), retry bounded 3x/1m-2m-4m then escalates once. Live on demo-hp: **120 s unattended** vs the incident's 587 s with a human; Scenario B proven (an `onboot:0` guest left stopped) | — |
|
||||
|
||||
@@ -108,7 +108,8 @@
|
||||
| R-105 | **Three hub-held DR records are empty on the entire live fleet.** `hosts.dr_record_json` = `{}` on all 3 hosts; `host_escrow.directive_json` = `{}` on both escrowed hosts; `dr_recipe.host_half.drives` = `[]` on every customer **including two with enrolled data drives** (916 GB USB on demo-felhom, 938 GB NVMe on demo-hp) | M | READY — 2026-07-28 | These are exactly the fields a host-loss recovery reads: `05-hub-architecture.md:175-176,186` names the slim DR record as one of four durable sources; `06-offsite-connectivity.md:148-150` says the escrow upload carried the DR directive; `felhom-agent/internal/dr/plan.go:34-35` makes `PlannedDrive` the re-attach-by-`durable_id` wrong-disk guard. **The three may have different causes** — `isUserDataDrive` (`internal/hub/dr_recipe.go:129-136`) requires type `usb`/`local-dir` **and** a non-empty `DurableID` **and** `MountPath`, and which of the three fails was not traced. Evidence: `architecture/_recovery-inventory-2026-07-28.md` Part D2.3. **UPDATE 2026-07-28 (vzdump-target move): the `drives` third is TRACED and now POPULATED on both demo boxes.** Cause: the enrolled data drives were never PVE storages at all — only agent-generated systemd mounts — so they never entered `report.StorageTargets` and `isUserDataDrive` never saw them. Giving each drive a `dir` storage at its own mountpoint supplied all three required fields at once (type `local-dir`, fs-UUID durable id, mount path), and the recipe now emits `uuid:91d2dc2d-…`/`/mnt/nvme-1tb` on demo-hp and `uuid:47a3361a-…`/`/mnt/hdd_1` on demo-felhom. **The other two fields (`hosts.dr_record_json`, `host_escrow.directive_json`) are untouched and still `{}`** — this narrows R-105, it does not close it. See `runbooks/RUNBOOK-vzdump-target-move-2026-07-29.md` §5.4. Flips: `07` §4 |
|
||||
| R-106 | ~~The DR recipe records the PBS namespace as `"root"` on every box~~ | XS | **SHIPPED** — agent v0.118.1, 2026-07-30 | Namespace resolves from the pbs STORAGE (storage.cfg `namespace`) — the field `vzdump --storage <pbs>` makes PVE read — not from the listed snapshot, whose `ns` PBS omits once the list is `?ns=`-scoped. `namespace_state` distinguishes a genuine root namespace from not knowing. Live on both boxes. Follow-on: **R-124** (the `"root"` spelling is not `--ns`-pasteable). `audits/R106-R109-recipe-completeness-2026-07-30.md`; flipped `07` §4 |
|
||||
| R-107 | **No offsite action unpacks the named-volume tars Tier-3 captures on every run.** `ReconstituteFromOffsite` skips the unit outright (`internal/backup/offbox_reconstitute.go:284-289`); `PlaceOffsiteRestore` places it only when the live unit is ABSENT (`internal/backup/offbox_restore.go:352-356`). The DB **is** replayed, from the scratch unit's `db-dumps/` | M | READY — 2026-07-28 | 52 of 53 catalog templates keep data in Docker named volumes, so for most apps the offsite tier carries the volume data and cannot itself put it back. A two-step route exists (full restore → place → Tier-1 restore, which does unpack tars) but **no single action does it and no UI routes it** — the R-103 shape one tier over. Sibling of R-102: both are "captured and never read". Flips: the offsite-restore row in map §C; `07` §6.3, §7.2 |
|
||||
| R-108 | **Network storage can host an app's namespace, and FileBrowser binds a network share at its ROOT — this BLOCKS D5.** For local drives FileBrowser binds `<drive>/userdata` only (`internal/web/handlers.go:2450-2460`); for network paths it binds the share **root** with `:rslave` (`:2432`) and serves it with `download: true` (`internal/infra/infra.go:326`). Nothing stops an app's namespace landing there: `GetSchedulableStoragePaths()` has no `IsNetwork()` filter (`internal/settings/settings.go:904-914`, feeds the deploy dropdown at `handlers.go:462-473`), the per-app migrate target list has none (`handlers.go:674-679`), and `handleStorageMigrateApp` does not call `refuseNetworkLifecycle` though its whole-namespace sibling does (`internal/web/storage_handlers.go:397` vs `:410-424`) | M | **READY — blocks an architectural target** — 2026-07-28 | **Today this is not a secret leak** — the recovery unit's `app.yaml` is secret-stripped (`internal/backup/recovery_unit.go:73`). It becomes one under **D5**, which would put app secrets in the local unit precisely so Tier-1/Tier-2 restore stop needing the guest. Verified LIVE in demo-hp's generated compose, where the asymmetry is visible rather than inferred: `- /mnt/felhom-drives/nvme-1tb/userdata:/srv/nvme-1tb` beside `- /mnt/felhom-drives/Felhom-Share:/srv/Felhom-Share:rslave`. **Second effect, independent of D5:** `.fab` bundles already carry **plaintext** secrets by design with an OPTIONAL password (`internal/appexport/export.go:484,506-511,307`) and `storageDriveList()` does not filter network paths (`internal/web/handler_export.go:377-387`), so one can be exported onto a NAS today. Full sweep of every read surface: `07-backup-architecture.md` §10.1. Flips: `07` §7.3 (D5 target → adopted) once closed |
|
||||
| R-108 | ~~Network storage can host an app's namespace, and FileBrowser binds a network share at its ROOT — this BLOCKS D5~~ | M | **SHIPPED** — controller v0.187.0, 2026-07-30 | **D5 UNBLOCKED.** One fail-closed predicate (`Settings.RefuseAsAppNamespace`) consulted by every placement surface, so no `backups/` tree can exist inside the share-root bind. The bind was deliberately NOT narrowed — it is load-bearing (`:rslave` automount wake, 2026-07-22 probe) and unscopable (apps on a share store at `<share>/<app>`; creating a `userdata/` layer would write Felhom convention onto a customer's NAS). **Five surfaces, not the four this row named** — decommission-with-migrate guarded only its SOURCE; and the boundary is the deploy POST, not the dropdown. Zero apps on network storage fleet-wide, so nothing stranded. The `.fab`-onto-NAS "second effect" is split out as **R-126**. Flipped: `07` §7.3 (D5 → unblocked), §10.1 (CLOSED), `00-capability-map.md` §D (new PROVEN-LIVE row). `audits/R108-network-app-namespace-2026-07-30.md` |
|
||||
| R-126 | **A `.fab` bundle — plaintext secrets, optional password — can be exported ONTO a NAS.** `storageDriveList()` (`internal/web/handler_export.go`) does not filter network paths | S | READY — 2026-07-30 | Split out of R-108 on its closure. NOT a D5 precondition: an explicit customer-chosen export destination, not a browsing surface reaching a backup tree (`07` §7.3 records the reasoning). Fix = filter network paths from the export destination list, or force the bundle password when the destination is a share. Flips: `07` §5 |
|
||||
| R-109 | ~~The DR recipe records no backup target~~ | XS | **SHIPPED** — agent v0.118.1 + hub v0.83.0, 2026-07-30 | `backup_target` resolves from the PRIMARY tier of `cfg.Backup.BackupTiers()` (the scheduler's own function) + the mountpoint that separates `/mnt/hdd_1` from `/var/lib/vz`; unresolvable is recorded as unresolvable, never defaulted. Required an unscoped HUB half: `AssembleDRRecipe` allow-lists top-level keys → **R-122**, a fourth recipe-completeness defect that had already shipped (`offsite_restic` dropped for the feature's whole life). `audits/R106-R109-recipe-completeness-2026-07-30.md`; flipped `07` §4 |
|
||||
| E-2 | **Drive-role machinery around the moved vzdump target.** The 2026-07-28 runbook proved the architecture change by hand on both demo boxes; this is the machinery: a **backup-target role** on `StoragePath` beside `Schedulable`/`IsDefault`/`Kind`; **assignment in the storage wizard** (suggest by attribute, refuse the absurd, never decide by transport or `removable` — on the reference hardware demo-felhom's target IS a USB HDD and BOTH drives report `removable=0`); **unassigned drives do nothing automatically**; **stickiness** (never silently retarget); `felhom-host-install.sh` creating the target with `--is_mountpoint 1` **and** issuing the `FelhomAgentStore` ACL; **absent-target policy**; **retention/space accounting** on a drive the customer shares; the honest **single-drive label**; remaining fleet migration | M | READY — 2026-07-28 | Full scope + rationale in `runbooks/RUNBOOK-vzdump-target-move-2026-07-29.md` §7. Two traps already paid for live: the storage `path` must BE the mountpoint or the agent reports the target `disconnected` forever (`internal/storage/observe.go:321`), and the per-storage `FelhomAgentStore` grant is mandatory or every backup 403s. Absent-drive behaviour today is **fail-loudly, no silent retarget** (`is_mountpoint 1` proven live) — which is NOT the intended fall-back-and-alarm design. Flips: matrix row 4 |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user