felhom.eu/documentation/architecture/07-backup-architecture.md

07 — The recovery model

Status	NOT RATIFIED. Ratification is Viktor's review, not an editor's.
Written	2026-07-28 (full rewrite; supersedes the 2026-07-14 DRAFT entirely)
Verified against	controller v0.183.0 · agent v0.110.0 · hub v0.80.0 · catalog 4252121 · repo HEADs felhom.eu ff050cf, felhom-controller fd50a73, felhom-agent d5c7691
Live fleet at verification	demo-felhom + demo-hp, both guest 9201, both on the versions above
Freshness	CURRENT as of 2026-07-28. Per standing ruling S-2, mark this STALE the moment it is more than a few trains behind. The document it replaces was DRAFT since 2026-07-14, verified against controller v0.132.0 — fifty-one versions stale — and was cited as authoritative throughout that time.

How to read this document. Two kinds of statement appear, and they are always labelled:

• [DESIGN] — a decision taken in the 2026-07-28 architecture discussion (D1–D6, §3–§5, §9). Not derived from code; the code may not implement it yet. Where it does not, §10 says so.
• [FACT] — an observed property, carrying a file:line, a live command output, or a citation to _recovery-inventory-2026-07-28.md (below: INV).

Where the model is silent, this document says OPEN rather than filling the gap.

Primary input: _recovery-inventory-2026-07-28.md (read-only inventory, 2026-07-28) — cited throughout as INV Part n. Every number in §6, §8 and §11 traces back to it.

---

1. Purpose and scope

This document describes how a Felhom customer gets their system back, and who can do it.

It replaces a document that described where copies are written. That was the wrong frame, and §7 explains why: a set of copies is not a set of recovery routes, and the previous document's tier table could be entirely satisfied while a real recovery was impossible.

In scope: the recovery model — trust boundary, lanes, the three-part model, the tiers as inputs to recovery, the dependency chain between them, the failure→recovery matrix, and the encryption policy.

Out of scope, deliberately: implementation specs (they live in task specs), the capture-set algorithm (internal/appbackup/captureset.go and its tests are the source of truth), and per-tier operational runbooks (documentation/runbooks/).

Authority split. This document is authoritative for the failure→recovery matrix (§8). The capability map (00-capability-map.md) stays authoritative for per-capability status. Neither restates the other; §8 rows are cited from the map, not copied into it.

---

2. The trust model

[DESIGN] The operator holds root SSH on every box. That is a fact of the product — the agent is operator-tier, the host is operator-managed, and break-glass exists precisely so the operator can get in when nothing else works. "The operator cannot read customer data" was never the security property, and no part of this document may be read as claiming it.

[FACT] The mechanics that make this concrete:

• The operator's OOB SSH public key is pushed to every box from the hub (hub_settings.oob_operator_ssh_pubkey → /var/lib/felhom-agent/felhom-sshd/authorized_keys.felhom-op, 93 bytes, LIVE on both hosts — INV Part C, row 21).
• The break-glass root@pam console password for every host is stored in the hub and retrievable with the operator's global key (documentation/runbooks/break-glass.md:42-48; LIVE: host_recovery holds 3 rows — INV Part D2.1).
• Guest data is reachable from the host by definition: pct exec, and the data drives are bind mounts on the host (mp8 /mnt/felhom-drives, LIVE pct config 9201 on both hosts).

[DESIGN] What R (the customer's recovery code) does provide, and must keep providing: the hub alone is not enough. A compromised hub yields blobs nobody can open. This property holds only while the operator's own key is not stored in the hub — which is why §11-A is an open decision and why its recommendation on record is "operator key held offline and never in the hub".

[FACT] The escrow is genuinely zero-knowledge today: host_escrow rows carry posture = zero_knowledge, a 383-byte blob and a 572-byte identity blob, and the hub holds only a restic_pw_sha256 hash beside them (LIVE, both hosts — INV Part D2.1). R exists in zero system copies by design (INV Part C, row 24).

So the honest statement of the property is:

The hub cannot open the escrow. The operator can reach a live box. Neither fact substitutes for the other, and R is what keeps the first true.

---

3. The two lanes (D1)

[DESIGN] Recovery is split into two lanes with different owners. This is a product decision, not a limitation.

Lane 1 — the customer, unassisted: files and app data

The customer owns their data and can get it back themselves, through the „Visszaállítás" surfaces, with nothing but their dashboard password. No operator, no ticket, no scheduling.

[FACT] What Lane 1 contains today (INV Part A.1 — seven paths, all behind the controller's RequireAuth gate, which is the customer-owned password: internal/web/auth.go:34-35 puts settings.json → password_hash ahead of the operator-provisioned controller.yaml value):

#	Surface (HU)	Endpoint	Semantics
1	„Visszaállítás indítása"	POST /backup/restore	destructive — rebuilds the app from its Tier-1 unit
2	„Fájlok visszaállítása"	POST /backup/tier2/restore	additive, missing-only
3	„Visszaállítás a távoli tárolóból"	POST /backup/offbox/restore	non-destructive, to a verification copy
4	„Helyreállítás az élő adatok közé"	POST /backup/offbox/place	additive, missing-only
5	„Teljes visszaállítás (fájlok + adatbázis)"	POST /backup/offbox/reconstitute	destructive to files + DB, never deleting
6	„Megosztások visszaállítása" + place	POST /backup/shares/{restore,place}	additive
7	.fab import	POST → apiImportStart	destructive re-import of one app

[FACT] What is proven in Lane 1 (INV Part G.1): paths 2 and 6 are proven live end-to-end; path 5 is proven live through a destructive drill (photos deleted, trash emptied, restored, timeline confirmed); path 3 is proven for bytes only; path 7 is proven for the drive-to-drive circle but its browser-upload leg is not; path 1 is proven to execute but its content recovery after real loss has never been demonstrated — the single most valuable unproven item in the system (audits/CAMPAIGN-9-restore-proof-2026-07-28.md:825-827); path 4 has never been exercised as a distinct action (audits/CAMPAIGN-8-backup-restore-2026-07-27.md:520).

[FACT] One honest residual on the whole lane: every proven run was performed by the operator, not by a customer. 00-capability-map.md:75 still carries "A customer (not the operator) performs a restore via UI alone" as MISSING as evidence — by absence of the run, not by a product gap.

Lane 2 — the operator: guest and host recovery

Rebuilding an LXC guest, rebuilding a host, and re-establishing a box's identity are operator work, by design and by contract. They are not customer-facing and are not going to be.

[FACT] What Lane 2 contains (INV Part A.2): the scheduled restore-test, pct restore, raw proxmox-backup-client restore, raw restic restore, the agent's DR bring-up (--selftest=bring-up -mode dr), the host-loss plan builder, and the escrow-consume ceremony. Each needs root on the host or a CLI flag; none is reachable from any customer surface.

[FACT] What is proven in Lane 2 (INV Parts G.1, G.3): whole-guest pct restore from both tiers is proven with exact mount parity; the unattended restore-test is proven and currently running on both boxes; a corrupted snapshot is proven to fail cleanly. The DR bring-up path has never been executed (CAMPAIGN-8…:522), the host-loss plan executes nothing by construction (felhom-agent/internal/dr/plan.go:1-4), and no host has ever been rebuilt as its former self (INV Part D1).

Why the split is right, stated once

[DESIGN] A customer can reason about "my photos are gone". A customer cannot reason about "restore the LXC and re-attach mp8 by durable_id without binding another guest's bootstrap credentials" — a hazard serious enough to have its own runbook page (runbooks/RUNBOOK-manual-guest-restore.md:45-48, F-OPS). Putting Lane 2 behind the operator is what lets Lane 1 be a button instead of a procedure.

[FACT] And as of D5 (v0.188.0) the split is real, not just intended. Until then Lane 1 only looked independent: an app's files sat on the customer's drive but could not be brought back without secrets that lived solely in the guest, so the fast customer lane was silently conditioned on the slow operator lane (§7.1 leg 1). The unit now carries those secrets, so Lane 1 needs the drive and nothing else for app rebuild (§7.4). The one honest caveat: Lane 1's file-merge surface (Tier-2 additive) still reads its destination from the guest's settings.json, so that surface remains guest-conditioned — D5 removed the secrets leg, not the living-app leg.

---

4. The three-part model (D4)

[DESIGN] Recovery needs three things, and they live in three different places. Losing one is a different problem from losing another, and the matrix in §8 is organised around that.

Part	Holds	Where	Lost when
Recipe	scaffolding — guest sizing, drive inventory, PVE storage definitions, PBS coordinates, app inventory + storage bindings. No secrets, no bytes.	the hub	the hub is lost
Escrow	the host identity key material and the restic repo password	the hub, R-wrapped	the hub is lost, or R is lost
Bytes	Tier-1 / Tier-2 / Tier-3 / whole-guest	the tiers	the relevant medium is lost

[FACT] The Recipe exists and is current. dr_recipe holds 6 rows; demo-felhom's was updated 2026-07-28 17:31:13 and demo-hp's 17:25:31 — i.e. within one report cycle (LIVE, INV Part D2). It carries guest sizing, pve_storage[], and an app half with per-app storage_bindings.

[FACT] The Escrow exists and is zero-knowledge. host_escrow: 2 rows, posture zero_knowledge; the identity bundle's shape is {tunnel_token, pbs_token, wg_private_key, restic_repo_password} (felhom-agent/internal/escrow/identity.go:26-39).

[FACT] Three parts of the Recipe are empty or wrong on the live fleet, and they are exactly the parts a host-loss recovery would read (INV Part D2.3): hosts.dr_record_json is {} on all three hosts; host_escrow.directive_json is {} on both escrowed hosts; dr_recipe.host_half.drives is [] on every customer including two with enrolled data drives; and dr_recipe.host_half.pbs.namespace reads "root" while the real namespaces are demo-felhom / demo-hp. → R-105, R-106.

---

5. Encryption policy (D2)

[DESIGN] Encryption follows the boundary, not the tier.

On the customer's own premises: plaintext

Tier-1 recovery units, Tier-2 cross-drive copies, the app data on the drives, and the local vzdump archives on the host are unencrypted, deliberately.

The reasoning, recorded so nobody "hardens" this later:

1. It buys nothing against a real threat. Someone who can take the second drive can take the first. Local encryption defends against a threat model — physical theft of only the backup medium — that does not describe a home server where both media sit in the same box.
2. It adds a key-loss path that turns a working backup into a brick. Every local encryption key is one more thing that must survive the disaster it exists for, and the system already has one such dependency it is trying to reduce (§7).
3. It would break browsing, which is a feature. FileBrowser and SMB let the household see and use their own files. An encrypted local copy is not browsable, and the „Megosztás" and FileBrowser surfaces are part of the product, not an accident.

[FACT] The local plaintext posture is real and verifiable: dir: local in /etc/pve/storage.cfg carries no encryption-key (LIVE, both hosts), so the daily whole-guest archive is a plain .tar.zst — 5.93 GB on demo-felhom, 1.68 GB on demo-hp (LIVE, INV Part B.3). That archive is the single highest-value object on the box: it contains encryption.key, the restic repo_password, the offbox ssh_key, settings.json and controller.yaml (all under /var/lib/docker = mp0, backup=1). Naming that plainly is part of the policy, not an argument against it.

Leaving the premises: encrypted, and the provider must not be able to read it

[FACT] Both offsite tiers encrypt client-side:

• restic (Tier-3): repo password is a 256-bit hex value generated once on the box and never logged (internal/backup/offbox.go:392-410); restic is invoked with RESTIC_PASSWORD_FILE (:608).
• PBS (whole-guest offsite): /etc/pve/storage.cfg carries a per-customer encryption-key and the fingerprints differ between demo-felhom and demo-hp (LIVE). The live restore command line observed on demo-felhom carries --crypt-mode=encrypt (INV Part A.2.3). Per-tenant encryption is why cross-customer dedup is impossible on the shared datastore — a cost consequence, recorded, not a defect.

The one exception, stated because it is not covered by either rule

[FACT] The .fab bundle writes app.yaml with decrypted plaintext secrets, deliberately, and its password is optional (internal/appexport/export.go:484,506-511; the generated file's first line is literally # Exported by felhom-controller — plaintext secrets, :531; Encrypted: req.Password != "", :307). A .fab can also be written to a registered network share, because storageDriveList() does not filter network paths (internal/web/handler_export.go:377-387). This is a portability artifact, not a tier — but it is the one place where a customer action can put every secret of one app onto a NAS in plaintext. Recorded here so the encryption policy is not read as covering it. → R-108 (same root cause).

---

6. The tiers — what each captures

The tiers are inputs to recovery, not recovery routes. §7 and §8 say what they can actually do.

6.1 The four tiers, as configured on the live fleet

Tier	Location	Captures	Cadence (LIVE)	Retention (LIVE)	Encrypted
Tier-1 recovery unit	<nsRoot>/backups/primary/<app>/ on the app's own drive	compose + .felhom.yml + secret-stripped app.yaml, db-dumps/*.sql, volume-dumps/*.tar, manifest.json	nightly at W, plus a checksum-gated refresh on the periodic status pass	one point per app (restore_points.go:14-18)	no
Tier-2 cross-drive	<target nsRoot>/backups/secondary/<app>/	always a full mirror of the unit (tier2.go:368-369) plus mandatory + optional file legs, v2 layout hdd/<rel> + userdata/<rel>	nightly at W+60m	mirror (rsync)	no
Tier-3 restic offsite	Hetzner Storage Box over SFTP, one multi-path snapshot per app per run	the unit plus mandatory legs only; a separate _shares snapshot	nightly at W+105m	--keep-daily 7 --keep-weekly 4 --keep-monthly 6	yes (restic)
Plane-2 whole-guest, local	local: → /var/lib/vz/dump on the host	rootfs + mp0 /var/lib/docker + mp1 /mnt/sys_drive	24 h (backup_cadence_seconds: 0)	local_backup_retention: 3	no
Plane-2 whole-guest, offsite	felhom-pbs: → ep0 datastore felhom-offsite, per-customer namespace, over WireGuard	same contents	7 days (604800)	server-side prune on ep0, keep-last 2 at 03:30	yes (per-customer key)

[FACT] The three nightly legs derive from one customer-settable window start W at fixed offsets — db-dump at W, Tier-2 at W+60m, offsite at W+105m — so they can never be misordered (cmd/controller/main.go:604-607). Both boxes run W = 02:30.

[FACT] What the whole-guest tiers do NOT carry. mp8 /mnt/felhom-drives and mp9 /etc/felhom-bootstrap are host bind mounts and are out of vzdump scope entirely (LIVE pct config 9201, both hosts). So the whole-guest tiers carry the guest and none of the customer's data drives — 916 GB on demo-felhom, 938 GB on demo-hp.

6.2 Coverage per app class — and an unresolved count

[FACT] Of 53 catalog templates, 52 keep data in Docker named volumes; exactly 13 carry a backup: block, and those 13 are exactly the templates that bind ${HDD_PATH} / ${USERDATA_PATH} / ${IMPORT_PATH} at all; 14 have a database service (INV Part B.1, computed against catalog 4252121).

Applying the classifier's documented two-level default (internal/appbackup/classify.go: explicit entry wins; else a :ro reader is excluded; else a writable bind is mandatory):

tier filter	templates with at least one file leg
Tier-3 (mandatory only)	4 — calibre-web, immich, nextcloud, paperless-ngx
Tier-2 (mandatory + optional)	7 — the 4 above plus audiobookshelf, komga, romm
legacy resolver path (apps with no block)	0 — no non-block template binds a namespace path

⚠️ UNRESOLVED — two counts of the same thing disagree

source	count
C9-F1 Phase 0, as shipped	A = 9 / B = 43 / C = 1 — felhom.eu/REPORT.md:17-24, restated backlog/OPEN-ITEMS.md:31
INV Part B.1, independent enumeration at catalog 4252121	A = 7 / B = 45 / C = 1

Both use the same definition ("templates whose Tier-2 copy can hold a readable file leg"). The difference is two apps and neither number is adopted here. The Phase-0 enumeration is described in prose but the script is not committed, so the two methods cannot be diffed from the repo. This must be resolved before either figure is used to size anything.

[FACT] The class-B consequence is real regardless of which count is right. For an app whose data lives entirely in named volumes, the Tier-2 copy holds a full recovery-unit/ and no readable leg — LIVE on demo-felhom, backups/secondary/bookstack/ and .../docmost/ contain recovery-unit and nothing else, at 156 MB and 86 MB (INV Part B.2). Since v0.183.0 the button refuses before stopping the app and names the action that works.

6.3 Where capture and restore disagree

[FACT] Three asymmetries, each source-cited:

tier	captured	read back by that tier's restore	gap
Tier-1	unit incl. volume tars + DB dumps	all of it	none
Tier-2	unit mirror + file legs	hdd/ and userdata/ only (tier2_restore.go:101-104)	the unit mirror is read by nothing — RecoveryUnitPath resolves to backups/primary/ (appbackup/paths.go:46-48) → R-102
Tier-3	unit (incl. volume tars) + mandatory legs	files + DB replay; the unit is skipped on the way to live (offbox_reconstitute.go:284-289; placed only if the live unit is absent, offbox_restore.go:352-356)	no offsite action unpacks the named-volume tars it captures → R-107

[FACT] Tier-2's gap is the sharper one because of when it bites: Tier-2 exists for the case where the primary drive is lost — and in exactly that case the primary unit is gone while this mirror survives on the second drive, unreachable by any customer action.

---

7. The recovery chain (D3) — the reason this document exists

[DESIGN] 3-2-1 describes copies. It does not describe recovery.

Three copies on two media with one offsite is a statement about bytes surviving. It says nothing about whether the bytes can be turned back into a working system, and a system can satisfy 3-2-1 completely while having no executable recovery route for a given failure. That is not a hypothetical here — §8 has rows where it is the actual state.

7.1 The dependency graph

[FACT] SUPERSEDED IN PART BY D5 — SHIPPED 2026-07-30 (controller v0.188.0). Leg 1 below (secrets) is no longer true for Tier-1 and Tier-2: the local recovery unit now carries the portable secret class, so those two tiers need the drive and nothing else. The graph and leg 1 are kept as written because they are the model everything downstream was derived from, and because leg 2 (the living app) is unchanged and still binds Tier-2's file leg and Tier-3. Read §7.4 for the current chain; the corrected rows are 3, 4 and 9 in §8.

[FACT — as of the R-108 survey, before D5] Every app-tier restore depends on the guest, for one of two reasons:

                       ┌──────────────────────────────────────────┐
                       │  the LIVE GUEST                          │
                       │   · settings.json (tier-2 destination)   │
                       │   · encryption.key  (32 B)               │
                       │   · app.yaml  (ENC: under that key)      │
                       │   · the deployed app itself              │
                       └───────────┬──────────────────────────────┘
                                   │ required by
        ┌──────────────────────────┼──────────────────────────┬─────────────────────┐
        │                          │                          │                     │
   Tier-1 restore            Tier-2 restore            Tier-3 reconstitute    Tier-3 scratch
   (rebuild the app)         (fill in files)           (overwrite + replay)   (verification copy)
        │                          │                          │                     │
   needs SECRETS              needs the app             needs the app          needs only the
   from app.yaml              running + the             deployed + a DB        repo password
   (restore_unit.go           recorded dest             service identifiable   (also in the guest)
    :130-132)                 (tier2_restore.go
                               :114-116)
        │                          │                          │                     │
        └──────────────────────────┴──────────────────────────┴─────────────────────┘
                                   │
                       ┌───────────▼──────────────────────────────┐
                       │  WHOLE-GUEST TIER  (local vzdump / PBS)  │   ← Lane 2, operator only
                       └───────────┬──────────────────────────────┘
                                   │ required by
                       ┌───────────▼──────────────────────────────┐
                       │  THE HOST  (agent.json, bootstrap.json,  │   ← in NO backup at all
                       │   vmbr9, sudoers, /etc/pve/priv, WG)     │      (INV Part D1)
                       └──────────────────────────────────────────┘

[FACT] The two legs of the dependency, precisely:

1. Secrets. The recovery unit is secret-free by design — "It NEVER writes a secret value" (internal/backup/recovery_unit.go:73). RestoreFromRecoveryUnit recovers secrets from the guest, never from the unit (restore_unit.go:130-132), and the policy is stated outright at :18-22: "Regenerate NOTHING… A missing DATA-ENCRYPTING key is FATAL: regenerating it would render the restored data unreadable, so we refuse and tell the operator to do a PBS whole-guest restore." Those secrets are encrypted under encryption.key, a 32-byte file that exists only inside the guest (LIVE, both boxes — INV Part C, row 9).
2. The living app. Tier-2's restore reads the destination recorded in the guest's settings.json (tier2_restore.go:114-116), and Tier-3's reconstitution refuses outright when the app is not deployed — „a(z) %s nincs telepítve — előbb állítsd helyre az alkalmazást, utána az adatokat" (offbox_reconstitute.go:198-201).

[FACT] So all three app tiers are conditioned on the whole-guest tier, which is Lane 2. The customer's own recovery lane rests on an operator-only tier, and today nothing tells the customer that. — This sentence was the reason D5 exists. It is now false for Tier-1 and Tier-2; see §7.4.

7.4 The chain AFTER D5 (controller v0.188.0, 2026-07-30) — the current model

[FACT] Tier-1 and Tier-2 no longer depend on the whole-guest tier. The unit's compose/app.yaml (mode 0600) carries the portable secret class — every catalog type: secret field, i.e. the declared data_keys, the 18 DB/root passwords and the internal signing secrets — so neither encryption.key nor the guest's app.yaml is required to rebuild an app:

   Tier-1 restore            Tier-2 restore            Tier-3 reconstitute    Tier-3 scratch
   (rebuild the app)         (fill in files)           (overwrite + replay)   (verification copy)
        │                          │                          │                     │
   needs ONLY THE DRIVE       needs the app             needs the app          needs only the
   (unit carries the          running + the             deployed + a DB        repo password
    secrets; guest is         recorded dest             service identifiable
    consulted only for
    the withheld class)       └────────── still conditioned on the guest (leg 2, unchanged) ──────┘
        │
   ✅ INDEPENDENT of the whole-guest tier

[FACT] What still needs the guest, precisely — so this is not read as more than it is:

1. Leg 2 is untouched. Tier-2's file restore still reads the destination from the guest's settings.json (tier2_restore.go:114-116) and Tier-3's reconstitution still refuses when the app is not deployed (offbox_reconstitute.go:198-201). D5 removed the secrets leg, not the living-app leg. So "Tier-2 no longer depends on the guest" is true of the unit-restore path it shares with Tier-1, and false of its additive file-merge path.
2. The withheld class. The 7 type: password admin logins and vaultwarden/ADMIN_TOKEN are deliberately NOT on the drive; they come from the guest, or are regenerated (O4). Their absence costs a credential reset, never data.
3. The fail-closed gate remains. A data_key in neither the unit nor the guest still refuses the restore outright. D5 makes it normally present; it does not soften the gate.

[DESIGN] Why plaintext is the right answer here, and what it is coupled to. Every travelling secret either decrypts data on the same drive or authenticates to a container on an internal compose network with no external listener — so possessing it adds nothing to possessing the drive, which is exactly D2's argument for keeping the DATA plaintext. That argument holds only because the internet-reachable class is withheld. The exclusion is what licenses the plaintext; the two must not be relaxed independently. stacks.PortableSecretEnvVars is the single boundary, with a code-level nonPortableSecrets register (not a catalog flag — a boundary a catalog push can move is not a boundary, cf. R-97a).

[FACT] Precedence, because two sources now exist. The unit wins over the guest. Not "newest wins": the unit's secrets are captured in the same run as the dumps beside them, so the unit's value matches the data being restored, whereas the guest's is merely the most recent. Preferring the guest is the data-loss bug — a rotated data key does not decrypt data encrypted with the old one, and a rotated DB password does not match the hash inside the restored data directory.

[FACT] Proven live 2026-07-30, on a scratch drill guest, through the real endpoints: AdventureLog (SECRET_KEY data_key + DB_PASSWORD) restored with the guest's app.yaml moved aside — secrets recovered=2/2, 27.6 s — and then the application itself read the seeded customer row over TCP with its own credential, which is the observable that matters (a restore that returns success onto unreadable data is a failure that looks like a pass). The discriminator held: the pre-backup row came back, a row added after the backup was gone, so the volume tar was genuinely restored. There was no .sql dump in that unit, so the DB came back from the volume tar — the exact case where a regenerated password would have failed. Separately, Grafana's type: password admin login was proven withheld: the sentinel value was live in the container and encrypted in the guest, and appeared in 0 files anywhere under the backup namespace. Full record: audits/D5-drive-alone-restore-2026-07-30.md.

7.2 A tier whose prerequisites cannot be met in the failure it exists for

[FACT] Two instances, both current:

• Tier-2 vs primary-drive loss. Tier-2's stated purpose is surviving the loss of the primary drive. In that failure the primary recovery unit is gone; the surviving mirror on the second drive is backups/secondary/<app>/recovery-unit/, which no code path reads (§6.3). For the 45-or-43 class-B apps the restore is a guaranteed no-op in exactly its designed scenario. → R-102
• Tier-3 vs guest loss. Tier-3 holds the volume tars and the DB dump. Reconstitution requires the app to be deployed and skips the unit; the tars are unpacked only by the Tier-1 path, which requires the guest's secrets. So offsite alone cannot rebuild an app onto a fresh guest. → R-107

7.3 What D5 would change — and why it was blocked — D5 SHIPPED 2026-07-30 (v0.188.0)

This section is history. D5 is implemented and proven live; the current chain is §7.4. Kept because the reasoning below is why the precondition was required, and because the last paragraph (.fab / R-126) is still open and still not part of D5.

One correction to the target as stated below. It assumed the class that must travel is the data_key-flagged secrets. Part 0 of the D5 task tested that and it did not survive: the flag is unreliable (four encryption keys the catalog itself labels as such are unflagged → R-127), and a DB password is not resettable in practice — POSTGRES_PASSWORD is ignored once PGDATA is non-empty, so a regenerated value leaves the app unable to authenticate against its own restored rows while the dump replay still reports success. The shipped boundary is therefore all type: secret minus the register, not data_key alone.

[DESIGN, TARGET — SUPERSEDED BY §7.4] The intended fix is to make app secrets travel with the local recovery unit, so Tier-1 and Tier-2 restore work without the guest and without R. Offsite already encrypts everything, so secrets travelling offsite would be covered by the escrowed repo password. R would then be required for offsite recovery and host identity only — losing R would cost the offsite route, not local recovery.

The premise D5 rests on is now established. §2 of the task that produced this document required it to be proven, not assumed: the backup tree must be unreachable from every browsing, download and export surface. When this document was written it was not — the FileBrowser network-share bind reached it. R-108 closed that on 2026-07-30 (controller v0.187.0) by refusing app namespaces on network storage, so no backups/ tree can exist under the share-root bind; every other surface was already clear (§10.1's table). D5's precondition is therefore MET and D5 may be adopted.

Still true, and not part of D5's precondition: a .fab bundle carries plaintext secrets by design with an optional password, and storageDriveList() does not filter network paths, so a bundle can be exported onto a NAS (§5, → R-126). That is an export destination the customer chooses explicitly, not a browsing surface reaching a backup tree, and it is unchanged by D5 — D5 moves secrets into the local recovery unit, not into .fab. It is tracked separately rather than folded in.

Until D5 is actually implemented, §7.1's chain stands as the model.

---

7.5 Where a recovery unit is RETAINED — and the size bound it puts on Lane 1

Added 2026-08-02 from audits/SPIKE-recovery-unit-space-2026-08-02.md and audits/CAMPAIGN-10-closeout-2026-08-02.md. §6.1 says a Tier-1 unit lives "on the app's own drive", which is true and is the whole story for a drive-resident app. It is not the whole story for an app with no data drive, and that case was undocumented until now.

The unit is RETAINED, not staged. GetAppDrivePath (felhom-controller/internal/backup/backup.go:245-255) returns the app's HDD_PATH if it has one and systemDataPath otherwise — what internal/appbackup/paths.go:26-27 calls "the SSD-only system-data fallback". Nothing deletes a unit after it is copied onward: the only prune is F5 (backup.go:1053-1112), which removes residue on an old drive when an app moves. So for every app without a data drive, mp1 holds the kept copy indefinitely.

What a unit contains, which bounds the problem (internal/backup/recovery_unit.go:20-25): compose/ + db-dumps/ + volume-dumps/ (named-volume tars) + manifest.json. It never contains mp8 / HDD_PATH userdata — a photo library on a 1 TB drive is not in a unit, and cannot make one overflow.

The resulting mismatch, on a default appliance:

	size	holds
mp0 /var/lib/docker	50 G	every app's live volumes — the thing a unit copies
mp1 /mnt/sys_drive	20 G	the retained units of every driveless app

A DB-backed app's unit is up to ~2× its data, because it carries the volume tar and the SQL dump (measured: 21.1 GB of app data → a 40.2 GB unit). A file-only app's is 1.00× (measured: homebox 2305 MB → 2305 MB). --sysdata-grow (felhom-agent/cmd/felhom-agent/main.go:178) defaults to 0, is not derived from the physical drive, and no observed box passes it — demo-hp's production guest 9201 ships mp0 50G / mp1 20G.

mp1 gates the entire app-data chain, not just Tier 1. Tier-2 mirrors the unit "(always)" from RecoveryUnitPath (internal/backup/tier2.go:302,368) and Tier-3 carries it too, so a unit that cannot be written to mp1 leaves Tier-2 and Tier-3 with nothing to copy.

The bound this places on Lane 1 (§3). Lane 1's promise is that a customer can restore an app from the drive alone, without the operator and without the guest — D5 (§7.4) made that true by putting the portable secrets in the unit. That independence is bounded by app size, and the bound is:

On a default appliance, an app can be restored by Lane 1 only while its recovery unit still fits the retained space — ≈ 19 GB of app data for a file-only app, ≈ 10 GB for a DB-backed one. Past that the unit cannot be captured, and the app falls back to Lane 2's operator-driven whole-guest route.

AS OF 2026-08-02 SOMETHING NOW WARNS, AND THE ALERTING IS PART OF THIS CONTRACT (R-167 / R-158, decision D-c; controller v0.191.x + hub v0.89.0). The last sentence of this section used to end "nothing warns when an app crosses the line". Two signals now exist and both are PROVEN-LIVE:

• To the CUSTOMER, before anything fails — internal/fillwatch warns per FILESYSTEM (never per app: one full disk holding ten apps would fire ten times) on whichever trips first, used ≥ 85% or free < 5 GiB, critical at 95% / 2 GiB, clearing at 75% / 7 GiB. Two terms, because a percentage alone lies at both ends of the range this section itself documents: 85% of a 20 G mp1 leaves 3 G — less than one DB-backed app's unit — while 85% of a 4 TB drive leaves 600 G. It watches the app-data volume, the system-data volume and every registered drive, which the previous health_degraded signal did not. Edge-triggered against persisted state; the hub owns cooldown.
• To the OPERATOR, when a capture actually fails — recovery_unit_capture_failed, per app, with the target filesystem's used/free bytes at the moment of failure, so the why needs no login. It is operator-tier (notify.operatorOnlyEvents) and deliberately not backup_failed: a customer can take no action on a capture failure.

7.5.1 — THE CEILING THIS SECTION DESCRIBES HAS BEEN REMOVED (2026-08-03, R-165 / decision D-a)

Everything above describes the SPLIT layout, which is now the legacy shape. A golden built by build-golden.sh v3.0.0 ships one data volume; mp1 does not exist. Both consumer paths are binds of subdirectories of it (variant V-c):

mp0 -> /var/lib/felhom      ├─ docker/    --bind--> /var/lib/docker
                            └─ sys_drive/ --bind--> /mnt/sys_drive

So the size bound below no longer applies to a box built from that golden. A driveless app's recovery unit is limited by the box's actual free space, not by a partition set at build time. The mismatch table above (mp0 50 G vs mp1 20 G) describes what a merged box no longer has.

R-175, fixed here rather than left standing. The bound below was stated as the fleet's and was one box's: it is derived from mp1 = 20 G, which is demo-hp exactly and never was demo-felhom (mp0 200G / mp1 50G, where the same arithmetic gives ≈ 49 GB / ≈ 24 GB), nor the golden (16 G / 8 G before provision grew them). Read it as a function of mp1, and only for a box still on the split layout. Measured: audits/SPIKE-r165-mp1-merge-2026-08-02.md M1.

What replaced the partition's second job — the reserve. mp1 was also a BULKHEAD: an overflow was refused per app with the last good unit byte-identical, and it could not reach /var/lib/docker, because that was a different filesystem. On a merged box it can. Decision B2, shipped in controller v0.192.0, is that bulkhead made deliberate — a two-term reserve (97% used or 1 GiB free) in internal/fillwatch's shape, sitting beyond its critical band so the customer is always warned first. It refuses per app and never deletes: nothing on this filesystem is generational, so pruning could only destroy a different app's only local copy.

THE CONTRACT, stated as what the code provides (controller v0.193.0, R-181). The reserve is a per-app, per-run ADMISSION decision, not a capture check. It is taken once for an app, immediately before that app's FIRST write of the run, and it covers all three write legs — the database dump, the volume dump and the recovery-unit capture. Those three write under one per-app root (backups/primary/<app>), which is what makes one verdict able to cover them honestly.

• What it guarantees. A refused app has nothing written for it in that run, its previous unit is byte-identical, it is not stopped, nothing anywhere is deleted, and the operator gets exactly one alert naming the app, the term that bound and the disk figures.
• Two terms, two questions. Headroom: is the filesystem already below the reserve? Size: would THIS app's write take it below? The size estimate is the app's previous .sql + .tar on disk; with no history the decision degrades to headroom alone, deliberately — otherwise the first backup is the one that can never happen.
• Why it is decided lazily and not once per run. Space changes during a run: app A's dump can put app B under the reserve, so a verdict taken at run start reads a disk that no longer exists.
• Why it is never re-decided between an app's own legs. That is precisely the shape v0.192.0 had — the two dump legs unguarded and only the capture refused — under which the reserve was consumed by the very write it exists to bound, and the refusal's "the previous unit is untouched" was measured false. Proven live on demo-hp 2026-08-03 (R-181), fixed the same day, and re-proven by filling the box for each of the two terms.
• It sits ahead of DumpAppVolumesSafe, which stops the stack as its first act — a refusal decided inside it would already have bounced the app it is refusing to back up.

Status caveat, deliberately explicit: every box in the field that has not been reinstalled is still on the split layout and everything above still describes them exactly. This subsection describes what a box built from golden ≥ 0.192.0 gets. Both demo boxes were reinstalled from it on 2026-08-03 (R-178).

Two things are deliberately not recorded here. The sizing ratio is the operator's ruling (R-163) — this section states the constraint, not a number. And the same-device placement is intended, not a defect: a driveless app's unit sits on the same SSD as its volumes, but Tier-2's cross-drive mirror, the whole-guest tier (which covers rootfs + mp0 + mp1, §6.1) and Tier-3 all cover device loss. What was missing was that this case existed at all, and that nothing warns when an app crosses the line — R-158.

8. The failure → recovery matrix

This is the core artifact. One row per failure. It is authoritative for recovery routes; 00-capability-map.md stays authoritative for per-capability status.

How to read the numbers.

• RTO — only measured durations from INV Part F appear here. A blank cell means nobody has ever measured it, and a blank is a finding, not an omission.
• RPO — no RPO has ever been measured from an incident. These cells carry the configured cadence that bounds RPO, read live from the box, labelled (cadence). A blank means no cadence governs the row.
• Status — PROVEN (live, cited) · PARTIAL (some legs proven) · IMPLEMENTED (code + tests, never exercised) · NONE (no route exists).

#	Failure	What survives	Recovery route	Invocable by	RTO (measured)	RPO (cadence)	Status	Evidence
1	Customer deletes files	everything else	„Fájlok visszaállítása" — Tier-2 additive merge	customer	39 s (6 files)	24 h	PROVEN	CAMPAIGN-9 A1 — byte-identical returns, both non-destruction promises kept, bytes re-served by paperless's own API
2	An app's data directory is destroyed	the guest, the other tiers	same route	customer	46 s (43 files)	24 h	PROVEN	CAMPAIGN-9 A3 — loss verified by a positive observable (doc download 200 → 404), then 16/16 documents usable
3	An app's DB and named volumes are lost	the guest, the unit	„Visszaállítás indítása" — Tier-1 unit restore (the only path that unpacks volume tars)	customer	18.25 s (path execution) · 27.6 s (D5 drill, guest app.yaml absent)	24 h	PROVEN (content recovery proven 2026-07-30)	CAMPAIGN-9 A2 proved the path executes; D5 v0.188.0 closed the content gap — after a restore with the guest's app.yaml moved aside, the app read the seeded row over TCP with its own credential, the pre-backup row returned and a post-backup row was gone (so the tar was really restored). No .sql dump in the unit ⇒ the DB came back from the volume tar
3c	same, with the GUEST GONE (secrets unavailable)	the drive	Tier-1 unit restore — the unit carries the portable secrets	customer	27.6 s	24 h	PROVEN	D5, §7.4. Before v0.188.0 this row was NONE: the data-key gate refused and the customer's own copy of their own data was not a recovery
3b	same, for a class-B app via Tier-2	—	no route — Tier-2 never reads the unit mirror	—			NONE	§6.3; R-102
4	Primary drive dies	Tier-2 copy on the second drive; Tier-3 offsite; the guest	Tier-2 for file legs (7 or 9 of 53 apps); Tier-3 reconstitute for files + DB; the volume tars in either copy are unreachable	customer (both)		24 h	PARTIAL	§7.2; R-102, R-107
5	Secondary drive dies	everything the customer uses	none needed — Tier-2 is a derived copy, rebuilt on the next run (07 §8 migration rule: "Migration = rebuild, not preserve")	automatic		24 h	PROVEN (by construction)	tier2 v2 layout marker + rebuild, internal/backup/tier2.go:359-393
6	Guest lost or corrupted	the host, both whole-guest tiers, the data drives (they are host binds)	pct restore from local: or felhom-pbs:	operator (SSH)	84–112 s local · 1101 s PBS (both = restore-test into a scratch guest, boot + verify + teardown)	24 h local · 7 d offsite	PROVEN	CAMPAIGN-2 T-P9; CAMPAIGN-8 Phase C (exact mount parity, unprivileged: 1 preserved); LIVE restore-tests on both boxes this session
7	Guest stopped and does not come back	everything	guest-power watchdog (60 s, onboot as the deliberate-stop discriminator)	automatic	120 s		PROVEN	agent v0.107.0 replay — 120 s unattended vs the incident's 587 s with a human
8	Host dies (hardware), drives intact	the data drives; ep0's PBS namespace; the hub's Recipe + Escrow	install a new host, then --selftest=bring-up -mode dr per guest, then re-attach drives by durable_id	operator (SSH)		7 d	IMPLEMENTED	bring-up code exists and has never been executed (CAMPAIGN-8…:522); the drive half of the plan is empty on every box (R-105)
9	Whole box lost (fire/theft) — host and drives gone	ep0 PBS namespace; the restic repo; the hub's Recipe + Escrow	new hardware → day-0 → escrow-consume with R → restore guests from PBS → app data from Tier-3	operator + customer (R)		7 d (guest) · 24 h (app data)	IMPLEMENTED / UNPROVEN	every leg exists; the composed path has never been run. The destructive S5 drill is operator-gated and unrun (06-offsite-connectivity.md:327)
10	Ransomware / malicious deletion inside the guest	PBS offsite (the box cannot delete its own snapshots); the restic repo is NOT protected the same way	whole-guest restore from PBS to a point before the event	operator (SSH)		7 d	PARTIAL	R-89 proved the box is refused when deleting its own PBS snapshot (CAMPAIGN-8…:514-515). R-95 (open, ranked #1): the restic credential can delete — readonly=False, forget --prune runs from the box, and SFTP cannot express append-only (OPEN-ITEMS.md:13)
11	Hub lost	every box's data plane, every tier, every Lane-1 route	none needed for recovery of a box; the hub itself restores from its Longhorn volume backup	operator (kubectl)		24 h + weekly (Longhorn RETAIN 1 each)	UNPROVEN	a hub restore has never been performed. The backup target is nfs://192.168.0.180 — DooPlex itself — and exactly 2 restore points exist (LIVE, INV Part D2.2)
11b	consequences while the hub is gone	—	—	—			[FACT]	day: nothing customer-visible breaks; events queue (settings.go:1466-1485). week: the operator alarm plane is dark, no claim/reset codes, no config or floor convergence, no PBS-secret re-issue. permanently: escrow custody and break-glass credentials are gone (INV Part D2.4)
12	Offsite provider lost (Hetzner)	everything on-premises: both drives, both whole-guest tiers	none needed — on-premises recovery is unaffected. Re-provision a new offsite target.	operator			[FACT]	restic and PBS share the provider (INV Part E.1). Whether they share an account and payment method is UNKNOWN → §11-D
13	Customer loses R	every byte, every tier, the Recipe	on-premises recovery is unaffected — Tier-1/2/3 and the whole-guest tiers all work while the guest and host live. What is lost: the ability to re-establish host identity and to open the escrowed offsite key material after a host loss	—			NONE for host-loss	R exists in zero system copies by design (INV Part C, row 24). Whether the operator should be able to recover is open → §11-A, §11-B. D5 narrows this further: Tier-1/2 app recovery needs only the drive, so losing R now costs the offsite route and host identity, never local app recovery (§7.4)
14	Host SSH management plane dies	everything	3-layer break-glass: tmpfiles → 60 s watchdog → hub-vaulted root@pam on the PVE console	operator			PROVEN	runbooks/break-glass.md; the original incident and its fix
15	Interrupted offsite run leaves a stale lock	everything	manual restic unlock --remove-all	operator (SSH)			DEFECT	the self-heal exists (offbox.go:634-648) but the probe fails first and classifyResticProbe has no lock case → fail-fast, operator told „ismeretlen okból" → R-104

8.1 The blank cells, listed explicitly

Per the rule that a blank is a finding, here they are:

row	blank	why
3b	RTO, RPO	no route exists to time
4	RTO	no drive-loss recovery has ever been timed
5	RTO	never timed; the rebuild is a normal Tier-2 run
6	—	RTO present, but it is a restore into a scratch guest on the same host; a restore to a different host has never been timed
8	RTO	a host has never been rebuilt as itself (INV Part D1)
9	RTO	the composed whole-box path has never been run
10	RTO	no ransomware-shaped recovery has ever been run
11	RTO	a hub restore has never been performed
12	RTO, RPO	no provider-loss recovery has ever been run
13	RTO, RPO	not a timed recovery; a capability loss
14	RTO	the break-glass path is proven but was never timed
15	RTO	the manual unlock was performed but not timed

Also unmeasured, and not representable as a row (INV Part F.3): any restore larger than 155.5 MB from restic; any restore of a customer data drive (no tier holds one whole); the weekly PBS incremental; .fab import wall-clock; and time-to-first-byte for a customer restore over a home uplink — no customer has ever driven a restore.

---

9. What the model implies for the tiers (recorded, not new design)

[DESIGN] Three consequences follow from §3–§7 and are stated so they are not re-derived:

1. Tier-2 is a drive-loss tier, not a second chance at Tier-1. Its job is to survive one drive dying. That is why it mirrors the unit at all — and why the unit mirror being unreadable (§6.3) defeats the tier rather than degrading it.
2. Tier-3 is a premises-loss tier. It is the only copy that survives fire, theft and ransomware-with-host-access, which is why it is the only tier whose credential exposure is ranked as the largest open data risk (R-95).
3. The whole-guest tiers are the system tier, and they sit under everything else (§7.1). Treating them as "one more copy" is precisely the 3-2-1 framing this document rejects.

---

10. Known gaps

Every divergence between the model above and the system as it is, each with an ID.

10.1 D5 is BLOCKED — precondition CLOSED by R-108 (v0.187.0); D5 ITSELF SHIPPED (v0.188.0)

D5 IS DONE — 2026-07-30, controller v0.188.0. The precondition below was met by R-108, and D5 was then implemented and proven live the same day: Tier-1/Tier-2 no longer depend on the whole-guest tier, and a customer needs the drive and nothing else. The current recovery chain is §7.4; the corrected matrix rows are 3, 3c and 13. This section's remaining value is the precondition analysis, which is why the table below is kept.

D5's PRECONDITION IS MET. An app's data namespace can no longer be placed on network storage, so no backups/ tree can exist inside FileBrowser's share-root bind, and the browsing surface therefore cannot reach the backup tree on ANY storage class. Every other read surface in the table below was already NO. D5 may be adopted — nothing in this section blocks it.

The fix inverted the obvious one, and that is the durable lesson here. The share-root bind was not narrowed, because it cannot be: (a) the :rslave share-ROOT bind is load-bearing — a Phase-0 probe (2026-07-22) proved an in-container access through it wakes the idle automount trigger, so narrowing it breaks NAS access itself; (b) there is no userdata/ layer to scope to, since apps on a share store at <share>/<app>; and (c) creating one would write Felhom's directory convention onto a customer's own NAS, which R-67 forbids outright. The browsing surface being immovable is precisely why the backup tree must never be placed under it. Tier 2 had already reached the same conclusion for its own targets (F-6C-1); R-108 closes the PRIMARY namespace, which was the last remaining route.

Operator ruling 2026-07-30: refuse the placement, keep the browse bind. RefuseAsAppNamespace (internal/settings/settings.go) is the single predicate; all placement surfaces consult it. It fails closed — /mnt/felhom-drives holds both storage kinds in-guest, so a path prefix cannot classify and Kind exists only on a REGISTERED path; an unregistered path under that root is therefore un-classifiable and is refused rather than assumed to be a drive.

Nothing was stranded: zero apps on network storage across all six hub customers including Peti. R-67's browse capability is byte-for-byte unchanged (verified by diffing demo-hp's generated compose before and after the deploy). This also supersedes the controller README's "NAS backup locality — decision A" (v0.118.0), which deliberately kept a NAS-resident app's Tier-1 artifacts on the NAS: that case can no longer arise.

Evidence: audits/R108-network-app-namespace-2026-07-30.md. The pre-fix analysis below is retained verbatim as the record of what was wrong.

10.1 (historical) The exposure as it stood before v0.187.0

[FACT] The verification and its result. Every surface that can read a file was checked:

surface	can it reach backups/?	evidence
SMB share creation	NO	sharingResolvePath (internal/web/sharing_handlers.go:52-81) resolves symlinks before containment, then refuses any path within SharingDeniedRoots(root); that set covers <root>/backups and the legacy <root>/felhom-data + <root>/felhom-data/backups (internal/stacks/samba.go:46-65, derived from ProtectedHDDPaths, delete.go:59-78) — both namespace shapes
SMB browse (folder picker)	NO	same deny set applied per child (sharing_handlers.go:521-537)
SMB ensureImportShare (the store-direct bypass)	NO	writes one controller-generated constant, GetImportRoot() = <system ns>/userdata/import (sharing_handlers.go:564-587)
FileBrowser — local drives	NO	the bind is appbackup.UserdataDir(sp.Path) only, and the comment says why (internal/web/handlers.go:2450-2460)
FileBrowser — network shares	YES → NO (R-108, v0.187.0)	the bind is still the share ROOT (- %s:/srv/%s:rslave, handlers.go:2437) and deliberately so — but no app namespace, hence no backups/ tree, can exist on a share, so the root bind reaches only the customer's own files. The reachability is closed at the PLACEMENT, not at the bind
.fab import path validation	NO	confined to <root>/exports (handler_export.go:400-408, estimate.go:215-217)
.fab browser download	NO	name-pattern + parent-must-be-the-staging-dir double guard (handler_export_download.go:36-45,120-140)
/api/debug/*	NO	no file-serving branch (handler_debug.go:46-92)
http.ServeFile (3 sites)	NO	assets only, filepath.Base-normalised (server.go:720-771)
log bundles	NO producer found	no filesystem-walk bundle producer exists on the box; searched felhom-controller/internal, felhom-agent/internal, cmd/
registering the backup dir as a drive	NO	the manual add requires system.IsMountPoint(path) (handlers.go:2091-2095); <drive>/backups is not a mount point

The exposure, end to end. All six links are source-cited and the precondition is live today:

1. A NAS share is registerable as a storage path and lands Schedulable: true — LIVE on demo-hp: {"path":"/mnt/felhom-drives/Felhom-Share","kind":"network","schedulable":true}.
2. GetSchedulableStoragePaths() has no IsNetwork() filter (internal/settings/settings.go:904-914), so that share appears in the deploy dropdown (handlers.go:462-473).
3. The per-app migrate target list filters only current / decommissioned / disconnected / schedulable — also no network filter (handlers.go:674-679).
4. handleStorageMigrateApp does not call refuseNetworkLifecycle, unlike its whole-namespace sibling which does (storage_handlers.go:397 vs :410-424), and startMigration has no guard either (internal/stacks/migrate.go:214-255).
5. With HDD_PATH on the share, namespaceRoot() returns it as-is under Model A (internal/backup/backup.go:262-263), so the app's Tier-1 unit is written to <share>/backups/primary/<app>/compose/app.yaml.
6. FileBrowser binds that share at its root and serves it with download: true (internal/infra/infra.go:326).

Observed live on demo-hp, in the generated compose — the asymmetry is visible, not inferred:

- /mnt/felhom-drives/nvme-1tb/userdata:/srv/nvme-1tb            <- drive:   userdata-scoped
- /mnt/felhom-drives/Felhom-Share:/srv/Felhom-Share:rslave      <- network: ROOT-bound
- /mnt/sys_drive/felhom-data/userdata/import:/srv/beolvasas

Today this is not a secret leak, because the unit's app.yaml is secret-stripped (recovery_unit.go:73). D5 would make it one. That is exactly the test §2 set, and D5 therefore does not hold as written. → R-108

CLOSED (v0.187.0). Links 2, 3 and 4 of the chain above are now guarded, and a fifth surface the chain did not list was found and guarded too: handleStorageDecommission mode=migrate checked only req.Where (the SOURCE) via refuseNetworkLifecycle, so a whole namespace could be decommissioned ONTO a NAS. Link 2's framing also understated the problem — the deploy dropdown is only a UI list; the boundary is the deploy POST (internal/api/router.go), which accepts any caller-supplied HDD_PATH and whose only other validation is os.Stat existence (internal/stacks/deploy.go). Filtering the list alone would have left the surface open. Links 5 and 6 are unchanged and still true — they simply can no longer be reached.

10.2 The gap register

ID	Gap	Consequence
R-102	Tier-2 writes a full recovery-unit/ mirror on every run and no code path reads it	Tier-2 is defeated in the drive-loss scenario it exists for (§7.2). Was C9-F4
R-103	The Tier-2 no-coverage refusal names the working action but does not route to it	45-or-43 of 53 apps dead-end at a message. Was C9-F1b
R-104	An interrupted offsite run leaves an exclusive restic lock the existing self-heal cannot reach, reported as „ismeretlen okból"	the offsite tier stays dead until a human unlocks. Was C9-F3
R-105	Three hub-held DR records are empty on the whole live fleet: hosts.dr_record_json, host_escrow.directive_json, dr_recipe.host_half.drives	the Recipe (§4) is incomplete in exactly the fields host-loss recovery reads. Causes may differ per field
R-106	dr_recipe.host_half.pbs.namespace records "root" on every box	the recorded restore coordinate is wrong; real namespaces are per-customer
R-107	No offsite action unpacks the named-volume tars Tier-3 captures on every run	offsite alone cannot rebuild a named-volume app (§7.2)
R-108	Network storage can host an app's namespace	CLOSED 2026-07-30, controller v0.187.0 — D5 UNBLOCKED. An app namespace may no longer be placed on network storage (5 surfaces guarded by one fail-closed predicate); the share-root bind is deliberately UNCHANGED because it is load-bearing and unscopable (§10.1). audits/R108-network-app-namespace-2026-07-30.md
R-126	A .fab bundle — plaintext secrets, optional password — can be exported ONTO a NAS: storageDriveList() (internal/web/handler_export.go) does not filter network paths	split out of R-108, which closed without it. NOT a D5 precondition: an explicit customer-chosen export destination, not a browsing surface reaching a backup tree (§5, §7.3)
R-95 (open)	The restic offsite credential can delete — the box can forget --prune its own repo	the tier holding the customer's documents and photos is the one whose credential can destroy it (matrix row 10)
R-86 (open)	Restore-tests are interval-scheduled, not backup-aligned	a tier's proof cadence is unrelated to when its archives are written
R-87 (open)	The restic tier is never restore-tested	matrix row 4's route has no unattended proof

10.3 Divergences that are documented elsewhere and are not re-opened here

[FACT] 06-offsite-connectivity.md:19-21 describes the operator's public edge as a Cloudflare Tunnel and states DooPlex has no public IP. Live DNS resolves hub.felhom.eu through a no-ip DynDNS CNAME straight to DooPlex's own public address (INV Part E.3). That is a topology-document issue, not a recovery-model one; recorded so the discrepancy is not lost.

---

11. Open decisions — for the operator

Recorded, deliberately not answered.

A. Escrow custody. Split custody (R or an offline operator key) versus a 2-of-3 threshold across customer / hub / box drives. Recommendation on record: *split custody, operator key held offline and never in the hub. Note the constraint from §2: the moment the operator key lives in the hub, "a compromised hub yields blobs nobody can open" stops being true.

B. Lost-R policy. Under split custody the operator can recover. Is that the stated policy — and if so, what identity check gates it? Today matrix row 13 has no route at all, and the customer is not told that losing R costs them the host-loss route.

C. RTO / RPO targets per scenario. None have ever been stated. Without them §8 cannot judge whether, for example, offsite-only recovery is acceptable for drive loss, or whether the 7-day PBS cadence is adequate for row 8. The measured column is the input; the target column does not exist.

D. Hetzner as a single failure domain. restic (Storage Box u629488, sub-accounts per customer) and PBS (Cloud server ep0 + a Cloud Volume) are both Hetzner. Whether they share an account, login and payment method is unverified (INV Unknown U-1) — the question was deliberately not answered by calling the provider API with the production token. Accept explicitly, or mitigate.

E. local vzdump shares a physical device with the guest it backs up. LIVE on both hosts: /var/lib/vz (the archive target) and local-lvm (the guest's rootfs and both backup=1 mountpoints) are both on /dev/sda3 → VG pve. The tier therefore protects against corruption and operator error only, never against disk failure. Accept and name it honestly in the customer- facing description, or move the target.

F (added by §10.1, not in the original list). — ANSWERED 2026-07-30. D5 could not be adopted until R-108 closed. Closing R-108 was the intended path, and it is done (controller v0.187.0): the operator ruled to refuse app namespaces on network storage rather than narrow the browsing surface, because the share-root bind is load-bearing and cannot be scoped. D5 is no longer blocked. Whether to now implement D5 remains an open scheduling decision, not a blocked one.

---

12. Evidence index

Claim	Grade	Source
Tier-2 file restore, gap-fill and after total loss	PROVEN-LIVE	CAMPAIGN-9 A1/A3
Tier-2 refuses without an outage for a no-coverage app	PROVEN-LIVE	v0.183.0 replay, felhom.eu/REPORT.md:60
Tier-1 unit restore executes	PROVEN-LIVE	CAMPAIGN-9 A2
Tier-1 content recovery after loss	UNPROVEN	CAMPAIGN-9…:825-827
restic restore of app data (bytes)	PROVEN-LIVE	CAMPAIGN-8 R-87, 155.5 MB, 6/7 byte-identical
offsite reconstitution of a DB-indexed app	PROVEN-LIVE	destructive immich drill 2026-07-20, 00-capability-map.md:66,75
offsite place-to-live as a distinct action	UNPROVEN	CAMPAIGN-8…:520
shares restore (files + definitions + credential)	PROVEN-LIVE	2026-07-18, 00-capability-map.md:96
.fab drive-to-drive round trip	PROVEN-LIVE	CAMPAIGN-6D P-FAB, 1.7 GB
.fab browser upload leg	UNPROVEN	00-capability-map.md:67
whole-guest restore, local and PBS, exact mount parity	PROVEN-LIVE	CAMPAIGN-2 T-P9; CAMPAIGN-8 Phase C
corrupted PBS snapshot fails cleanly	PROVEN-LIVE	CAMPAIGN-8 fault 17
the box cannot delete its own PBS snapshots	PROVEN-LIVE	CAMPAIGN-8, R-89
unattended restore-test across tiers	IMPLEMENTED (rotation not observed across consecutive cadences)	00-capability-map.md:41; LIVE per-tier timestamps this session
guest-power watchdog	PROVEN-LIVE	agent v0.107.0, 120 s
quiesce crash recovery	PROVEN-LIVE	CAMPAIGN-8 fault 10, 1 s, by SIGKILL
break-glass	PROVEN-LIVE	runbooks/break-glass.md
agent DR bring-up (ModeDRGuestLoss)	NEVER EXECUTED	CAMPAIGN-8…:522
host-loss plan → an actual restore	EXECUTES NOTHING BY CONSTRUCTION	felhom-agent/internal/dr/plan.go:1-4
host rebuilt as its former self	NEVER DONE	INV Part D1
escrow consume in a real recovery	SPIKE-LEVEL ONLY	06-offsite-connectivity.md:327
hub DB restore from its Longhorn backup	NEVER DONE	INV Part D2.2
a customer performing a restore unassisted	MISSING AS EVIDENCE	00-capability-map.md:75

---

13. What this document deliberately does not do

• It does not restate capability status — §8 cites the map, the map cites §8.
• It does not resolve the 7/53-vs-9/43/1 count (§6.2). Both are on the record; neither is adopted.
• It does not estimate a single RTO or RPO. Every blank in §8 is a real gap.
• It does not answer §11. Those are the operator's.
• It does not claim ratification.