# SPIKE — referential-coupling backup classification (2026-07-14)

**Class:** spike (findings only — no production code, no schema edits, no repo mutations beyond this doc).
**Versions at spike time (recorded live):** controller **0.130.0** on demo guest 9201 (healthy),
agent **0.88.0** on felhom-pve (active), catalog @ HEAD (55 templates). Offsite repo: Hetzner
Storage Box sub-account, 12 snapshots, last run 2026-07-14 02:15 UTC, status ok, all inspection
**read-only** (`--no-lock`; no prune/forget/write).
**Empirical core:** a scratch Immich (catalog-pinned images v2.5.5) deployed in an isolated
namespace on 9201, exercised through the DB-present/library-absent restore scenario, then fully
removed.

Product decision under test: classify each userdata bind by **referential coupling** —
**mandatory** (COUPLED: DB/config references it; restoring state without it yields a *broken* app),
**optional** (DECOUPLED-precious: user-specific, not re-downloadable, app re-derives on restore →
empty-not-broken), **excluded** (DECOUPLED-bulk: large, replaceable, cross-app).

## Verdict up front

1. **Offsite carries NO userdata today — confirmed at source and live, no exception.** Every restic
   snapshot path is `backups/primary/<app>` (unit only). The latent gap is real: **immich is
   offbox-toggled on demo, yet its photo library is in none of its 5 snapshots** (§SQ1).
2. **The coupling model is VALIDATED by the Immich experiment** — DB-only restore is not merely
   empty, it is broken at two escalating levels: the server **refuses to boot** (its own
   folder-check), and once coaxed past that, the timeline still advertises every asset while **every
   thumbnail/original fetch is HTTP 500**. No self-heal, no re-import path for managed uploads
   (§SQ3).
3. **Two NEW capture-gap findings** beyond the task baselines: (F-S1) tier-2 does **not** copy bulk
   userdata — the tier2.go header comment is stale; browsable HDD userdata has **zero automated
   off-drive protection today**; (F-S2) tier-2's appdata leg keys on the **stack name**, so
   paperless-ngx (`appdata/paperless`) is **silently skipped** — live-proven (§SQ1).
4. **Round-trip verdict: exclusion scoping** (tar the userdata root, skip excluded subtrees) is the
   only shape that round-trips through the v1 `.fab` manifest without format changes; inclusion
   scoping (per-bind tars) cannot carry nested paths in `HDDSubdirs` (CTRL-001 forbids separators)
   (§SQ5).
5. **The migration-default premise as stated does NOT hold** and needs a two-level default —
   detailed in §SQ5.

---

## SQ1 — Capture map per tier (source + live)

### Source-confirmed mechanics

| Tier | What it captures | Code evidence |
|---|---|---|
| Recovery unit (primary) | `compose/` (docker-compose.yml, .felhom.yml, secret-stripped app.yaml) + enumerated `db-dumps/*.sql` + `volume-dumps/*.tar` + manifest.json. **No userdata, no HDD appdata.** | `backup/recovery_unit.go:91-106` (capture list is exactly the two config files + built app.yaml; dumps only *enumerated* from dirs the dump flow wrote) |
| Offsite (restic) | The recovery unit dir **only** — `discoverOffboxUnit` returns `backups/primary/<app>`; one `restic backup <unitdir>` per toggled app. | `backup/offbox.go:372` (`RunOffboxBackup`), `offbox.go:559-586` (`runOffboxInternal`: `src` = the unit path, tagged `felhom-offbox,<app>`) |
| Tier-2 (cross-drive rsync) | Unit → `backups/secondary/<stack>/recovery-unit` **plus** `appdata/<stack>` → `.../appdata`. **NOT `userdata/`** — the only two `rsyncMirror` call sites in the tree are these (`tier2.go:169,177`). | `backup/tier2.go:140-184`; `AppDataDir` = `<nsRoot>/appdata/<stack>` (`appbackup/paths.go:68`) |
| Manual `.fab` (v0.130.0) | Config + DB dump + volume tars + `ExportDataMounts` = `${HDD_PATH}` binds ∪ the **userdata ROOT** as one tar. | `stacks/delete.go:575` (`ExportDataMounts`), `appexport/export.go:594-638` |
| PBS whole-guest | Guest rootfs + docker volumes; bind-mounted drive content is out of reach (platform docs; not re-derived here). | — |

### F-S1 (NEW, HIGH for the redesign): tier-2 does not copy bulk userdata — the comment lies

`tier2.go:17` says *"recovery unit + bulk userdata … the ONLY off-drive protection that browsable
HDD userdata can get"*, and `cmd/controller/main.go:1360-1361` says the tier-2 path *"copies the
felhom-data namespace wholesale"*. **The code copies neither**: `RunTier2` rsyncs exactly
`RecoveryUnitPath` and `AppDataDir(nsRoot, stack)` — `UserdataDir` appears nowhere in the backup
package (grep: only the two comment lines). Consequence for the model: **the shared media tree has
no automated off-drive copy at any tier today**; its only protection is a manual `.fab` export.
The task baseline "Tier-2 DOES copy bulk userdata" traced to these stale comments, not to behavior.

Live proof (demo 9201): `backups/secondary/radarr/` and `backups/secondary/jellyfin/` on felhom-usb
contain **only** `recovery-unit/` — no media, while `nas-media/userdata/media/` holds 96 MB the
apps actively use.

### F-S2 (NEW): tier-2 appdata leg silently skips apps whose appdata dir ≠ stack name

`RunTier2` copies `appdata/<stackName>`; the catalog binds `${HDD_PATH}/appdata/paperless/...` for
stack **paperless-ngx** (slug `paperless-ngx`). `AppDataDir(nsRoot,"paperless-ngx")` doesn't exist →
the `os.Stat` guard (`tier2.go:176`) skips the copy **silently** (status still records ok).
Live contrast on demo: `secondary/nextcloud/` contains `appdata/` (name matches);
`secondary/paperless-ngx/` (felhom-flash) contains **only** `recovery-unit/` while
`appdata/paperless/` exists on the same source drive. Affected today: **paperless-ngx** (its
document originals+archive!). immich/nextcloud/romm happen to match. Any future app with a nested
or differently-named appdata bind inherits the gap. The per-bind classification metadata fixes this
class of bug structurally (capture what the metadata declares, not what a name-convention guesses).

### Live capture map (per catalog app; source-derived, spot-verified live where deployed)

Legend: unit = config+dumps; ✔=captured, ✘=not captured. "app-named HDD appdata" = `${HDD_PATH}/appdata/<x>` bind.

| app | needs_hdd | offsite (restic) | tier-2 (cross-drive) | manual .fab (v0.130.0) |
|---|---|---|---|---|
| audiobookshelf | true | unit ✔ / userdata ✘ | unit ✔ / userdata ✘ | unit ✔ + userdata ROOT ✔ |
| calibre-web | true | unit ✔ / userdata ✘ | unit ✔ / userdata ✘ | unit ✔ + userdata ROOT ✔ |
| emby, jellyfin, plex | true | unit ✔ / media tree ✘ | unit ✔ / media tree ✘ | unit ✔ + userdata ROOT ✔ (RO shared tree over-captured) |
| immich | true | unit ✔ / **photo library ✘** | unit ✔ + `appdata/immich` ✔ | unit ✔ + `appdata/immich` ✔ + userdata ROOT ✔ |
| komga, navidrome, radarr, sonarr, romm | true | unit ✔ / userdata ✘ | unit ✔ / userdata ✘ (romm: + `appdata/romm` ✔) | unit ✔ + userdata ROOT ✔ (romm: + appdata bind ✔) |
| nextcloud | true | unit ✔ / **data dir ✘** | unit ✔ + `appdata/nextcloud` ✔ | unit ✔ + `appdata/nextcloud` ✔ |
| paperless-ngx | true | unit ✔ / **documents ✘** | unit ✔ / **`appdata/paperless` ✘ (F-S2)** | unit ✔ + appdata binds ✔ + userdata ROOT ✔ |
| 42 volume-only apps | false | unit ✔ (volume dumps ride in the unit) | unit ✔ (F6: they flow through tier-2 too) | unit ✔ (volume tars) |

Live legs run: restic `snapshots` + `ls` (below); `find` on `backups/primary/radarr` (unit =
compose/ + volume-dumps/ + manifest.json exactly) and `backups/secondary/{radarr,jellyfin,nextcloud,paperless-ngx}`.

### The offsite "no userdata" verdict — live evidence

Offbox-toggled apps on demo: audiobookshelf, calibre-web, immich (settings.json `app_backup`).
`restic snapshots --no-lock`: 12 snapshots, **every** path is
`/mnt/felhom-drives/{felhom-usb,nas-media}/backups/primary/<app>`. Full `restic ls` of the newest
immich snapshot (`81927042`, 2026-07-14 02:15):

```
.../backups/primary/immich/compose/{.felhom.yml,app.yaml,docker-compose.yml}
.../backups/primary/immich/db-dumps/immich-postgres.sql
.../backups/primary/immich/manifest.json
```

That is the complete file list. **The photo library (`appdata/immich`, 129 MB on the same drive) is
absent** → an offsite-only Immich restore today reproduces exactly the broken state measured in SQ3.
This is the "looks backed up but isn't" gap the redesign exists to close: the customer toggled
immich offsite ON, the status page says ok, and the photos are not there.

---

## SQ2 — Per-bind classification inventory (all 55 catalog apps)

Catalog ground truth (full read of all 55 templates): **13 apps** have `${USERDATA_PATH}`/
`${HDD_PATH}` binds; **42 are named-volume-only** (classification moot — their state already rides
in the unit's volume dumps; the only host bind among them is homepage's RO docker.sock). No
`.felhom.yml` anywhere has a backup-classification field (grep across all 55: zero hits) —
greenfield confirmed. `needs_hdd` lives under `resources:`.

Proposed class per bind (reason = coupling argument):

| app | bind | class | reason |
|---|---|---|---|
| immich | `${HDD_PATH}/appdata/immich` → `/usr/src/app/upload` | **mandatory** | Managed upload library. DB-authoritative: asset rows + `mountChecks` state reference these paths; **SQ3 proves restore-without = broken, not empty** |
| immich | `${USERDATA_PATH}/media/photos` → `/external/photos:ro` | **optional** | External library: RO, registered post-deploy, re-scanned on demand → empty-not-broken if absent; but content is user-precious (not re-downloadable) → customer-selectable |
| paperless-ngx | `${HDD_PATH}/appdata/paperless/media` | **mandatory** | Document originals + archive; DB rows carry content hashes/paths → restore-without = broken documents (coupled by design; empirical leg deferred, see SQ3 note) |
| paperless-ngx | `${HDD_PATH}/appdata/paperless/export` | **excluded** | Transient export target, re-creatable |
| paperless-ngx | `${USERDATA_PATH}/import/paperless` | **excluded** | Consume inbox — transient by contract (paperless deletes after ingest) |
| nextcloud | `${HDD_PATH}/appdata/nextcloud` → `/var/www/html/data` | **mandatory** | THE user files + `oc_filecache`/shares reference them; not re-derivable content |
| calibre-web | `${USERDATA_PATH}/media/books` → `/calibre-library` | **mandatory** | The Calibre library dir contains `metadata.db` **inside** the tree — DB and book files are one referential unit; restore-without = broken library |
| calibre-web | `${USERDATA_PATH}/import/calibre` | **excluded** | Ingest inbox, transient |
| audiobookshelf | `${USERDATA_PATH}/media/audiobooks`, `media/podcasts` | **excluded** | ABS re-scans folders; DB keeps progress/metadata keyed by rescan-match; podcasts are re-downloadable by definition. (Judgment call on audiobooks — see shared-tree note) |
| komga | `${USERDATA_PATH}/media/comics` → `/data` | **optional** | Komga re-scans (empty-not-broken), but comic collections are typically hand-curated/irreplaceable → precious, customer-selectable |
| navidrome | `${USERDATA_PATH}/media/music:ro` | **excluded** | Classic re-rippable bulk; navidrome re-scans; shared tree |
| radarr | `${USERDATA_PATH}/media/movies` | **excluded** | Re-downloadable; radarr re-scans and re-acquires by design |
| radarr / sonarr | `${USERDATA_PATH}/downloads` (SAME host path, both apps) | **excluded** | Transient download queue; also cross-app shared — backing it up per-app would double-capture |
| sonarr | `${USERDATA_PATH}/media/tv` | **excluded** | As movies |
| emby / jellyfin / plex | `${USERDATA_PATH}/media` (whole tree) `:ro` | **excluded** | Pure READERS of the shared tree (watch-state/config in volumes). See consistency rule below |
| romm | `${USERDATA_PATH}/roms` → `/romm/library` | **optional** | Romm re-scans (empty-not-broken); curated ROM sets are often not re-acquirable → precious |
| romm | `${HDD_PATH}/appdata/romm/resources` | **excluded** | Scraped artwork/metadata cache, re-derivable from scrapers |

**Shared-media-tree consistency (the model survives, with one rule).** emby/jellyfin/plex mount the
**whole** `media/` tree RO while seven apps own subtrees of it, and radarr+sonarr bind the identical
`downloads/` path. A naive per-app class assignment contradicts itself (books are *mandatory* to
calibre-web but sit inside the streamers' *excluded* tree; a per-app tar of `downloads` would
double-capture). The rule that dissolves both: **a class belongs to the BIND (the host path), and
the app that owns/writes the path is the classification authority; read-only binds are never
backup-relevant for the mounting app** (the streamers back up nothing from `media/`; calibre-web's
`media/books` = mandatory regardless of who else reads it). Overlaps of the same host path across
apps must be deduplicated at capture time by path, not by app — this already matches
`ExportDataMounts`' containment-dedup philosophy.

**Flagged judgment calls** (not blockers, spec should confirm with the operator): audiobooks
(excluded vs optional), comics/roms (optional chosen — precious-decoupled). The mandatory class is
NOT in doubt anywhere — every mandatory above is DB-referenced content.

---

## SQ3 — Immich DB-only restore (the empirical core)

**Setup** (scratch, isolated): compose project `spikeimmich` in `/root/spike-immich/` on 9201's
rootfs — catalog-pinned images (`immich-server:v2.5.5`, `postgres:16-vectorchord0.3.0`,
`redis:7-alpine`), ML disabled, loopback port 127.0.0.1:12283 only, no traefik, library bind
`/root/spike-immich/library` → `/usr/src/app/upload`, postgres in `/root/spike-immich/pgdata`.
**No felhom namespace touched** (registered drives, real apps, controller state all untouched;
controller verified healthy before/after).

**Healthy baseline:** admin created + logged in via API; 3 generated PNGs (640×480/800×600,
0.8–1.2 MB) uploaded via `POST /api/assets` → 3 assets `status:created`; after job settle, all 3
thumbnails HTTP 200 (8–13 KB webp/jpeg) and originals HTTP 200. Library tree: `upload/` (3
originals), `thumbs/` (preview+thumbnail per asset), plus Immich's own `.immich` marker files
(timestamp content) in all six subdirs. Sizes: library 3.6 MB, pgdata 302 MB.

**The coupled-broken scenario** — stop server → `mv library library-hidden` → empty `library/`
→ start (DB fully intact = "restored state without userdata"):

- **Level 1 — refuses to boot.** Server exits 1 in a crash-loop. Its StorageService logs
  `Verifying system mount folder checks, current state: {"mountChecks":{"thumbs":true,"upload":true,
  "backups":true,"library":true,"profile":true,"encoded-video":true}}` — i.e. **the DB itself
  remembers the folders existed** — then
  `Failed to read (/usr/src/app/upload/encoded-video/.immich): ENOENT` and
  `microservices worker exited with code 1` (pointer to
  docs.immich.app/administration/system-integrity#folder-checks). The coupling is not incidental:
  Immich *actively encodes* it (markers written on first run, presence recorded in the DB,
  verified on every boot).
- **Level 2 — skeleton-recreated restore is a lying gallery.** Recreating the six subdirs + copying
  the `.immich` markers back (what a well-meaning restore script would do) lets the server boot.
  Then: `GET /api/timeline/buckets` still reports `[{"timeBucket":"2026-07-01","count":3}]` — the
  UI advertises all 3 photos — while **all 3 thumbnail fetches and all 3 original fetches return
  HTTP 500** (`ENOENT` → `GlobalExceptionFilter Unknown error` per request in the server log).
- **Worse-than-empty verdict: CONFIRMED.** A clean install boots quietly with an empty timeline. The
  DB-only restore either doesn't boot at all, or boots into a gallery where every remembered photo
  errors — "looks recoverable but isn't", plus error spam.
- **No self-heal / re-import:** no repair API at `/api/repair` (404); job queues idle-clean; the
  admin "repair" surface in Immich identifies orphans but cannot resurrect managed uploads whose
  bytes are gone (nothing exists to re-import *from* — the library WAS the source). Reverse
  direction (files present, DB absent) is a different scenario and irrelevant to offsite restore.
- **Control leg:** restoring the hidden library and restarting → all 3 thumbnails + originals
  HTTP 200 again. The files alone were the broken link → **mandatory=coupled validated**.

**Paperless second candidate: deferred** (time-boxed; Immich was decisive). Paperless's coupling is
design-documented (content-hash-indexed originals in `media/documents/originals` + archive
versions); recommend a 15-minute repeat of this protocol during the implementation spec's
acceptance run rather than blocking the model on it.

**Cleanup:** done and verified — see §Cleanup.

---

## SQ4 — Size split on live data

Measured on demo (du; drive namespaces on felhom-flash/felhom-usb/nas-media):

| quantity | example values |
|---|---|
| (a) app state = recovery unit (config + DB dump + volume dumps) | navidrome 548 K · jellyfin 1.0 M · radarr 4.0 M · paperless-ngx 71 M (usb) · immich 44 M · calibre-web 248 M · nextcloud 933 M |
| (b) coupled/precious data (HDD appdata) | immich `appdata/immich` 129 M · nextcloud `appdata/nextcloud` 37 M · paperless `appdata/paperless` 56 K (demo-tiny) |
| (c) bulk shared tree | nas-media `userdata/media` 96 M (movies 57 M, music 37 M, books 12 M) + `downloads` 8.1 M; felhom-flash `userdata/navidrome` 2.9 G (campaign canary .bins — debris, see Observations) |

Ratios even at demo scale: jellyfin state (1 M) vs the media tree it reads (96 M) ≈ **1:100**;
radarr state (4 M) vs movies (57 M) ≈ 1:14. Production libraries are 100 GB–10 TB against
state+coupled in the 0.1–5 GB band → the two-number estimate ("state+coupled" vs "+bulk") separates
by 2–4 orders of magnitude and is exactly the number the opt-in warning needs. Counter-example that
justifies *measuring* rather than assuming: scratch Immich's postgres (302 MB) dwarfed its 3-photo
library (3.6 MB) — state is not always the small number; and nextcloud's unit (933 MB, volume dumps
carrying `nextcloud_html`+DB) exceeds its appdata (37 MB) on demo.

**Where the numbers come from (reuse seams):** tier-2 already sizes with `du -sb`
(`tier2.go dirSizeBytes`); the export estimate already does a **per-mount** du loop
(`appexport/estimate.go:49-57 duBytes`) + the v0.129.0 F-A container-view `volumeSizer` for named
volumes, with the `SizeUnknown`-never-renders-as-fits guard. A classification-driven two-number
estimate needs **zero new measurement plumbing** — split the existing per-mount loop by the bind's
class and sum into two buckets.

---

## SQ5 — Metadata schema + round-trip feasibility

### Round-trip mechanics as-built (code evidence)

- The `.fab` manifest keys HDD tars by **basename**: `HDDSubdirs []string`
  (`appexport/manifest.go:27`); export tars each mount as `<basename>.tar` with a fatal
  basename-collision guard (`export.go:611-619`).
- **CTRL-001 hard-blocks nested paths in the v1 field:** `UnmarshalManifest` →
  `validateManifestPaths` → `ValidateSegment` rejects any segment containing a path separator
  (`validate.go:35-36`), and restore re-checks per subdir (`restore.go:604`). `hdd_subdirs:
  ["userdata/media/tv"]` fails the parse *by security design*.
- Import maps a basename → a resolved `${HDD_PATH}` mount (note: `resolveHDDMounts` matches only
  literal `${HDD_PATH}` references — `restore.go:670` — never `${USERDATA_PATH}`), else **fallback**
  `<HDD_PATH>/<basename>` (`restore.go:616-626`). The v0.130.0 userdata ROOT round-trips purely via
  this fallback: basename `userdata` → `<HDD_PATH>/userdata` = `USERDATA_PATH`. Pinned by
  `stacks/export_mounts_test.go:55-59`.

### Feasibility verdict: **exclusion scoping wins**

- **Inclusion scoping** (tar each mandatory bind separately) requires nested-path knowledge on the
  import side → a **manifest v2** field (e.g. `hdd_entries: [{rel_path, tar}]`), a new multi-segment
  validator (per-segment CTRL-001 checks, no `..`/absolute), import-side placement logic, and a
  compatibility story for old-controller-imports-new-bundle (v1 readers would silently ignore the
  new field → hollow restores — the exact C6B-F1 failure class resurrected). Feasible but expensive
  and risk-shaped.
- **Exclusion scoping** (tar the userdata ROOT, **skip excluded-class subtrees**; per-class
  variants: offsite/state-only captures nothing of userdata unless a mandatory bind exists) keeps
  the manifest byte-identical in shape: still one `userdata` entry in `HDDSubdirs`, still restored
  by the existing basename fallback, **zero import changes, old controllers import new bundles
  correctly** (they just extract a root tar that happens to contain less). The only new code is an
  exclude list on the export tar call — and mandatory-bind capture for the *other* tiers (offsite
  unit enrichment) reuses the same per-bind path resolution `ParseComposeUserdataMounts` already
  provides (`stacks/delete.go:513`). The v0.130.0 root-capture comment's dedup philosophy carries
  over unchanged.
- Same-drive HDD-appdata binds (`appdata/immich` etc.) already round-trip per-bind today (their
  basenames are direct `${HDD_PATH}` children) — classification of those needs no tar-shape change
  at all.

### Proposed `.felhom.yml` addition

```yaml
backup:
  userdata:
    - path: "media/photos"    # relative to USERDATA_PATH
      class: optional
  hdd:                        # optional block for ${HDD_PATH} binds (appdata/*)
    - path: "appdata/immich"  # relative to HDD_PATH
      class: mandatory
```

Semantics: `class ∈ {mandatory, optional, excluded}`; a bind **listed in the compose but absent
from the block defaults to `mandatory`** (C6B-F1 lesson: back up rather than silently drop);
read-only binds (`:ro`) are exempt from that default (reader rule, §SQ2) and default `excluded`.
The `${HDD_PATH}/appdata/*` binds default `mandatory` even without a block — they are app-managed
stores (immich/nextcloud/paperless all mandatory; only romm's scraper cache warrants an explicit
`excluded`).

### The migration default — the task's premise needs one correction

The premise *"missing field → mandatory default means today's behavior (offsite=config-only) does
NOT auto-start shipping media"* is **internally contradictory as stated**: if a missing field
defaults each bind to mandatory AND the new offsite policy ships mandatory binds, then an
unmigrated sonarr ships its whole `media/tv` offsite on day one — a surprise size/cost regression
(demo quota is 50 GB; a modest TV library is 10–100× that; the SLICE-4 soft-quota gate would refuse
runs and alert-spam). The reasoning **holds with a two-level default**:

- **No `backup:` block at all → legacy behavior per tier** (offsite = unit only, .fab = full root
  capture as v0.130.0). No unmigrated app changes its offsite footprint or cost.
- **Block present → unlisted writable binds default `mandatory`** (within an app that has opted
  into classification, the safe direction is capture).

Since all 13 hdd apps get explicit classes in the same catalog change that ships the feature, the
block-absent branch exists only as a safety net for stragglers/third-party templates — and it fails
toward *today's* behavior, not toward a quota blow-up. The 42 volume-only apps need no block.

---

## SQ6 — `.fab` over-capture (derived + quantified; live export leg blocked)

**Code-proven:** `ExportDataMounts` returns the userdata **ROOT** for any app with a
`${USERDATA_PATH}` bind (`delete.go:575-598`; pinned by `TestExportDataMounts_UserdataConvention`),
so a sonarr export tars every sibling's content in the shared namespace by construction — the
question is only *how much*.

**Quantified on the live demo tree (nas-media, the drive radarr/jellyfin/navidrome/calibre-web
share):** a radarr `.fab` today captures the whole `userdata/` root ≈ **105 MB** (media 96 MB +
downloads 8.1 MB + import/documents/roms), of which radarr's own binds (`media/movies` 57 MB +
`downloads` 8.1 MB) are ~65 MB → **~1.6× over-capture even on a small demo tree**, carrying
navidrome's music, calibre-web's books (with its `metadata.db`!), and jellyfin's whole view. Under
the proposed classes, radarr's default `.fab` = state only (both its binds are excluded); the
over-capture factor vs today is unbounded (grows with everyone else's library).

**Live byte-level leg (deploy sonarr+jellyfin one-drive, marker files, export, inspect tar): NOT
run.** Triggering a real export requires the dashboard session (CSRF-protected UI flow) and the
demo box's login is customer-owned (Viktor) — the same blocker as the open `.fab` full-circle
upload test. Protocol for when it runs: drop `SPIKE-marker-sonarr.txt` in `media/tv` and
`SPIKE-marker-jelly.txt` in `media/movies`, export sonarr, `tar tf` the bundle's
`data/hdd/userdata.tar` for both markers (expected today: both present). Suitable as the
before-picture acceptance test of the implementation spec.

---

## Recommendations for the implementation spec

1. **Schema:** the `backup:` block above (per-bind, path-relative, three classes; reader-rule for
   `:ro`; two-level default). Ship explicit classes for all 13 hdd apps in the same catalog commit.
2. **Per-tier policy:**
   - **Offsite (restic)** = unit + **mandatory** binds, not deselectable. Fixes the live-proven
     immich gap. Size impact is bounded because mandatory ≈ DB-coupled stores, not media trees.
   - **Tier-2** = unit + mandatory + **optional** (it's the browsable local copy; a 2nd HDD has the
     room) — and fix F-S2 by driving the appdata leg from the declared binds instead of the
     stack-name guess; fix the F-S1/stale comments either by implementing the claimed userdata copy
     for mandatory+optional or by correcting the comments in the same change.
   - **Manual `.fab`** = state + mandatory always; optional pre-selected; excluded opt-in behind the
     two-number size warning ("state+coupled ≈ X MB / +bulk ≈ Y GB", reusing the estimate.go
     per-mount du split). UI states plainly: bulk media's intended path is FileBrowser, coupled data
     rides in the `.fab`.
3. **Round-trip scoping: exclusion** (root tar minus excluded subtrees) — no manifest change, no
   import change, backward-compatible both directions (§SQ5). Do NOT introduce per-bind nested tars
   under manifest v1.
4. **Capture the class at deploy/unit time** into the recovery-unit manifest (the unit already
   carries `.felhom.yml` — the offsite runner can read the classes without new sync paths).
5. **Acceptance tests to carry over:** the SQ3 protocol (immich DB-only restore must become
   restorable end-to-end from offsite alone), the SQ6 marker test (sonarr `.fab` must stop carrying
   jellyfin's markers), a paperless repeat of SQ3, and an F-S2 red-proof (paperless-ngx tier-2 must
   copy `appdata/paperless`).
6. **Migration:** catalog commit with explicit classes for the 13 → controller feature gates on
   block presence → stragglers keep legacy behavior. No `MinAgent` coupling expected (all
   controller-side), but the offsite unit-content change should bump the recovery-unit
   `SchemaVersion` consideration in the spec.

## Cleanup checklist (all verified live)

- [x] Scratch Immich: `docker compose down -v` (containers + volumes + `spikeimmich_spikenet`
      network removed; `docker ps -a --filter name=spike` → 0).
- [x] `/root/spike-immich/` (library, pgdata, photos, token), `/root/spike-immich-compose.yml`,
      `/root/genpng.py` deleted.
- [x] Pulled images removed (`immich-server:v2.5.5`, `postgres:16-vectorchord0.3.0`);
      `redis:7-alpine` left in place (cannot prove it wasn't pre-existing; no container uses it;
      zero-risk residue, noted here for honesty).
- [x] No restic write/prune/forget — inspection used `--no-lock` reads only; repo status unchanged
      (12 snapshots before and after).
- [x] No tier-2 target, drive namespace, controller setting, or deployed app touched; controller
      0.130.0 healthy at spike end (`Up`, healthy).
- [x] No secret values recorded anywhere in this doc (scratch instance used throwaway credentials,
      destroyed with the instance).

## Observations (out of scope, documented not acted on)

- **Userdata debris on demo drives:** felhom-usb `userdata/` contains non-skeleton per-app dirs
  (`actualbudget`, `bookstack`, `mealie`, `navidrome`, `paperless-ngx`, `uptime-kuma`,
  `vaultwarden`) and felhom-flash `userdata/navidrome/` holds 2.9 GB of campaign canary `.bin`
  files — leftovers from earlier campaigns; harmless but they'd inflate any root-capture `.fab`
  and the future size estimates. Candidate for a demo-hygiene pass.
- **audiobookshelf binds `media/podcasts`, which is not in `UserdataSkeleton()`** — the deploy belt
  pre-creates it from the compose scan, so no bug, but skeleton and catalog have drifted.
- **immich's current recovery unit carries no volume-dumps** (db-dump only) — consistent with the
  dump flow not having run for its named volumes since the last teardown; worth a look during the
  spec's offsite enrichment work.
- calibre-web's tier-2 status records an error against destination `/mnt/felhom-drives/teszt_enroll`
  (a stale pinned target from an old enroll test) — the pin-invalidation fallback may deserve a
  glance.
- The task baseline described the catalog as "~13 apps"; it is **55** (13 hdd + 42 volume-only) —
  the classification workload is smaller than "all apps" implies but the migration commit touches
  13 files, not 55.
