From ebf3c0cbea37955b87a80c26ed63dba2af4124d3 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Sun, 14 Jun 2026 21:15:24 +0200 Subject: [PATCH] docs(spike): userdata/ layout + shared-storage ownership mapping (report-only gate) Maps current namespace dir creation/ownership (live on 9201), the 9-app/13-mount media blast radius, run-identity table (FileBrowser=uid1000, komga/audiobookshelf=root), LXC idmap, and proposes the setgid GID-1000 convention. Flags: MigrateAll's merge walk drops setgid+owner on userdata. Ends with a 7-item fork list. No code/deploy. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../SPIKE-userdata-layout-2026-06-14.md | 276 ++++++++++++++++++ 1 file changed, 276 insertions(+) create mode 100644 documentation/audits/SPIKE-userdata-layout-2026-06-14.md diff --git a/documentation/audits/SPIKE-userdata-layout-2026-06-14.md b/documentation/audits/SPIKE-userdata-layout-2026-06-14.md new file mode 100644 index 0000000..46a436d --- /dev/null +++ b/documentation/audits/SPIKE-userdata-layout-2026-06-14.md @@ -0,0 +1,276 @@ +# SPIKE — userdata/ layout + shared-storage ownership convention (2026-06-14) + +**Report-only. NO code, NO catalog edits, NO deploy, NO version bump.** This is the GATE before any +implementation; it MAPS the current state and PROPOSES the change with file:line + live evidence, and +ends with a FORK LIST for the operator. + +**Goal (operator-decided):** introduce a customer-facing `userdata/` dir under each drive's felhom-data +namespace, sibling to `appdata/` and `backups/`. FileBrowser mounts `userdata/`. Apps that write to +`${HDD_PATH}/media/...` move into the userdata layout. A shared-ownership convention must let FileBrowser ++ apps collaborate on userdata without permission collisions. + +**Access path used (operator can't login to the LXC):** `ssh root@felhom-pve` (the Proxmox host, +192.168.0.162) → `pct exec 9201 -- `; in-guest apps are Docker containers reached via `pct exec 9201 +-- docker exec …`. All "live:" evidence below was captured this way on guest **9201**. + +--- + +## 1. Current layout (LIVE) + who creates what + +### Live evidence + +In-guest (Model A — `/mnt/felhom-usb` IS the felhom-data namespace root): +``` +$ pct exec 9201 -- ls -la /mnt/felhom-usb/ +drwxr-xr-x 3 root root … appdata +drwxr-xr-x 3 root root … backups +drwxr-xr-x 3 root root … media ← media/books (calibre-web), root:root mode 755 +$ pct exec 9201 -- ls -la /mnt/felhom-usb/media/ → drwxr-xr-x root root books +$ pct exec 9201 -- ls -la /mnt/felhom-usb/appdata/ → drwxr-xr-x root root romm +``` +Host view of the same drive (host `/mnt/felhom-usb/felhom-data/` = guest `/mnt/felhom-usb/`): +``` +$ ls -la /mnt/felhom-usb/felhom-data/ +drwxr-xr-x 100000 100000 … appdata / backups / media ← all guest-root (uid 0 → host 100000) +``` +System drive (in guest): namespace root is `/mnt/sys_drive/felhom-data/` (NOT `/mnt/sys_drive`): +``` +$ pct exec 9201 -- ls -la /mnt/sys_drive/felhom-data/ → appdata / backups / exports (all root:root) +``` +**No `userdata/` exists anywhere yet** (`find /mnt -maxdepth 3 -name userdata` → none). + +### Who creates each dir (code) + +| Dir | Created by | Where | Owner / mode | +|---|---|---|---| +| `/felhom-data` (additional drive) | **AGENT** at guest-attach | `felhom-agent/internal/localapi/guestbind.go:55-61`: `mkdir -p /felhom-data` then `chown 100000:100000` (NOT `-R`) | guest-root 0:0, 0755 | +| system-drive `felhom-data/…` | provisioning + controller | `scripts/docker-setup.sh` does **NOT** create namespace dirs; the controller creates `appdata` on FileBrowser sync | guest-root 0:0, 0755 | +| `appdata/` | **CONTROLLER** | `internal/web/handlers.go:~1464` `syncFileBrowserMounts`: `os.MkdirAll(/appdata, 0755)` | guest-root 0:0, 0755 | +| `appdata/`, `media/…`, `backups/` | **Docker auto-create** (bind-mount source) on first app deploy, OR the backup code | docker daemon (runs as guest-root) | **guest-root 0:0, 0755** | + +**Why `media/` is guest-root (confirmed):** the agent creates only the `felhom-data` ROOT (comment +`guestbind.go:20-21`: "per-app subdirs are chowned at deploy — NOT here"); nothing pre-creates `media/`, +so when calibre-web first deployed, the **Docker daemon auto-created** `${HDD_PATH}/media/books/...` as +its bind-mount source — owned by the daemon (guest-root). This is the collision root cause: a +guest-uid-1000 app (FileBrowser, calibre) gets only `r-x` on a 0755 root-owned dir, never write. + +--- + +## 2. HDD_PATH + namespace resolution + +- `appbackup/paths.go:11` `const FelhomDataDir = "felhom-data"`; `:24` `NamespaceRoot(drivePath, + inGuestDrive)` → returns `drivePath` as-is when `inGuestDrive` (Model-A user drive: the in-guest mount + IS the namespace root), else `filepath.Join(drivePath, "felhom-data")` (SSD/system). `:68` + `AppDataDir(nsRoot, app)` = `/appdata/`. +- **`${HDD_PATH}` at deploy time = the registered StoragePath, which IS the in-guest namespace root.** + Live: `romm/app.yaml` → `HDD_PATH: /mnt/felhom-usb`; SSD apps (mealie, actualbudget) have **no + HDD_PATH** (named volumes only). So `${HDD_PATH}/media` expands to: + - user drive: `/mnt/felhom-usb/media` (guest) = host `/mnt/felhom-usb/felhom-data/media` + - system drive (a media app pinned there): `/mnt/sys_drive/felhom-data/media` +- Host→guest bind (`guestbind.go:64-66`): `pct set -mpN /felhom-data,mp=/mnt/` — the + `felhom-data` segment is consumed by the bind, so the guest sees the namespace at `/mnt/`. +- **Therefore `${HDD_PATH}/userdata` works uniformly** on both drive types (it's always namespace-relative). + +--- + +## 3. App-catalog blast radius (the apps that must change) + +Grepped `app-catalog-felhom.eu/templates/*/docker-compose.yml`. Apps writing customer content to +`${HDD_PATH}/media/...` (the ones the userdata move touches): + +| App | file:line | Mount | RW/RO | +|---|---|---|---| +| jellyfin | jellyfin/docker-compose.yml:20 | `${HDD_PATH}/media:/media` | **:ro** | +| emby | emby/docker-compose.yml:21 | `${HDD_PATH}/media:/media` | **:ro** | +| plex | plex/docker-compose.yml:22 | `${HDD_PATH}/media:/media` | **:ro** | +| navidrome | navidrome/docker-compose.yml:22 | `${HDD_PATH}/media/music:/music` | **:ro** | +| sonarr | sonarr/docker-compose.yml:21-22 | `${HDD_PATH}/media:/media` + `${HDD_PATH}/downloads:/downloads` | RW | +| radarr | radarr/docker-compose.yml:21-22 | `${HDD_PATH}/media:/media` + `${HDD_PATH}/downloads:/downloads` | RW | +| audiobookshelf | audiobookshelf/docker-compose.yml:20-21 | `${HDD_PATH}/media/audiobooks` + `${HDD_PATH}/media/podcasts` | RW | +| calibre-web | calibre-web/docker-compose.yml:29-30 | `${HDD_PATH}/media/books/ingest` + `${HDD_PATH}/media/books/library` | RW | +| komga | komga/docker-compose.yml:19 | `${HDD_PATH}/media/comics:/data` | RW | + +**Total: 9 apps / ~13 media mounts** must change (`${HDD_PATH}/media` → the userdata layout). + +**NOT in scope (app-exclusive `appdata/` data, not shared media):** immich +(`${HDD_PATH}/appdata/immich`), nextcloud (`${HDD_PATH}/appdata/nextcloud`), paperless-ngx +(`${HDD_PATH}/appdata/paperless/{media,consume,export}`), romm (`${HDD_PATH}/appdata/romm/{library, +resources}`). These are single-app subtrees under `appdata/` — moving them to userdata is a *separate* +decision (see ROMM + fork list). `sonarr/radarr` also use `${HDD_PATH}/downloads` (staging, not final +library) — decide whether downloads moves under userdata too. + +**ROMM (flagged):** `romm/docker-compose.yml:65-66` → +`${HDD_PATH}/appdata/romm/library:/romm/library` + `…/resources:/romm/resources`. **Compose-path only — +no in-app/env override** (`.felhom.yml` only exposes `HDD_PATH`). The library lives under `appdata/` +today, so it is **not customer-browsable**. The operator wants a ROM import/library location *under +userdata* — that means repointing the `library` mount to `${HDD_PATH}/userdata/<…>` (fork: exact path). +`resources` (cover art/metadata) is app-internal — leave under appdata. + +--- + +## 4. App run-identity (decides the shared-group design) + +| App | image | run identity | writes shared media? | +|---|---|---|---| +| **filebrowser** | gtstef/filebrowser:1.3.3-stable | **uid 1000 (filebrowser)** by image default — LIVE `id`=`uid=1000 gid=1000`; compose sets no `user:` | RW (the customer's file manager) | +| calibre-web | crocodilestick/calibre-web-automated | PUID/PGID **1000** (linuxserver-style) | RW (books) | +| sonarr | lscr.io/linuxserver/sonarr | PUID/PGID **1000** | RW (media+downloads) | +| radarr | lscr.io/linuxserver/radarr | PUID/PGID **1000** | RW (media+downloads) | +| emby | emby/embyserver | UID/GID **1000** | :ro | +| paperless-ngx | paperless-ngx | USERMAP_UID/GID **1000** | (appdata only) | +| jellyfin | jellyfin/jellyfin | **root** (no PUID) | :ro | +| plex | plexinc/pms-docker | **root** (no PUID) | :ro | +| navidrome | deluan/navidrome | **root** (no PUID) | :ro | +| komga | gotson/komga | **root** (no PUID) — LIVE pattern | **RW (comics)** | +| audiobookshelf | advplyr/audiobookshelf | **root** (no PUID) | **RW (audiobooks/podcasts)** | +| romm | rommapp/romm | **root** — LIVE `docker exec romm id`=`uid=0(root)`, `Config.User`="" | (appdata; would be RW if moved) | + +**The 1000 cluster is large and natural:** FileBrowser + the 5 PUID/PGID-1000 apps already share GID +1000. The :ro media apps (jellyfin/plex/emby/navidrome) only READ — they need group/other **read**, no +write concern. **The problem set = root-running apps that WRITE shared media: komga + audiobookshelf** +(and romm if its library moves to userdata). + +--- + +## 5. FileBrowser — run UID + mount-change plan + +- **LIVE:** `docker exec filebrowser id` → `uid=1000(filebrowser) gid=1000(filebrowser)`; + `Config.User`=`filebrowser`. The gtstef image bakes uid 1000 even though the controller compose sets no + `user:` — so FileBrowser is **already gid 1000**. +- Current mounts (LIVE `docker inspect`): `/mnt/sys_drive/felhom-data/appdata → /srv/felhom-data` and + `/mnt/felhom-usb/appdata → /srv/felhom-usb`. Renderers: `internal/web/handlers.go:~1464` + `syncFileBrowserMounts` (`os.MkdirAll(/appdata,0755)` + bind `…/appdata:/srv/`); + `internal/infra/infra.go:~116` `RenderFileBrowserCompose` (image const `FileBrowserImage = + "gtstef/filebrowser:1.3.3-stable"`, **no `user:`**, no PUID), `:~211` `RenderFileBrowserConfig`. +- **To mount `userdata/` instead of `appdata/`:** change `syncFileBrowserMounts` to + `/userdata:/srv/` (and `MkdirAll userdata`). (Scoping note: it was deliberately scoped to + `appdata/` in Phase 4A to hide backups/recovery-units; pointing at `userdata/` keeps backups hidden and + exposes only customer media — strictly better.) +- **To run as the shared GID:** FileBrowser is already uid/gid 1000. If the shared GID is 1000, **no + change needed**. If a dedicated felhom GID is chosen, add `user: "1000:"` (or `group_add`) to + `RenderFileBrowserCompose`. With `userdata` dirs at `:1000`/`:GID` mode 2775, FileBrowser then has + full RW. + +--- + +## 6. LXC userns mapping + +- **LIVE** `/etc/pve/lxc/9201.conf`: `unprivileged: 1`, `features: nesting=1,keyctl=1`, **no custom + `lxc.idmap`** → the default unprivileged map applies: **guest uid/gid 0 → host 100000**, **guest 1000 → + host 101000** (range 65536). Confirmed by the host `ls` showing guest-root dirs as `100000:100000`. +- The convention only needs to be consistent **inside the guest** (guest uids/gids); the host-side numbers + are the +100000 image. The agent, running as host-root, can `chown` to any host uid:gid, i.e. it can + stamp guest `0:1000` as host `100000:101000`. + +--- + +## 7. Ownership-convention recommendation (+ umask caveats) + +**Recommended convention:** `userdata/` and its subtree owned `root:` (guest), **mode 2775** +(setgid + group-rwx). Setgid makes every file/dir created inside inherit `` regardless of who +creates it. Members of ``: FileBrowser + every customer-content app. **Recommend +` = 1000`** — it is already FileBrowser's gid and the PUID/PGID of calibre/sonarr/radarr/emby/ +paperless, so 5 apps + FileBrowser need zero identity change. + +**The setgid bit fixes the GROUP; it does NOT fix the WRITE BIT — umask still applies.** Caveats per app: +- **linuxserver apps (calibre-web, sonarr, radarr):** run as 1000 already; default `UMASK=022` → files + 0644 (group can't write). Set **`UMASK=002`** (linuxserver supports it) so group keeps write. +- **root-running RW-to-shared apps (komga, audiobookshelf):** write as root with umask 022 → group-read + only. Setgid stamps gid 1000 but FileBrowser still can't modify/delete. Each needs **either** a + `user: "1000:1000"` pin **or** a `UMASK`/umask-equivalent (komga: `user:` works; audiobookshelf: no + PUID — `user: "1000:1000"` is the lever). Flag: verify each image starts cleanly when pinned to 1000. +- **:ro media apps (jellyfin/plex/emby/navidrome):** read-only mounts — group/other **read** is enough; + no umask concern. +- **romm (if library moves to userdata):** runs as root; same caveat as komga/audiobookshelf. + +**Pre-create userdata so Docker never root-auto-creates it again** (the #1 fix). Creation points to +patch: +- **Additional drives:** `felhom-agent/internal/localapi/guestbind.go:55-66` — after creating + `felhom-data`, also `mkdir -p /felhom-data/userdata/{media subdirs}` + `chown 100000:101000` + + `chmod 2775` (agent is host-root → can set guest `0:1000`). +- **System drive + per-deploy safety net:** the controller — at app deploy (`internal/stacks/deploy.go`) + and/or the FileBrowser-sync `MkdirAll` site (`handlers.go:~1464`) — ensure `userdata/` exist + with the shared GID + setgid **before** `docker compose up` so the daemon never auto-creates them root. + (The controller runs as guest-root, so it can chown to `0:1000` + chmod 2775.) + +--- + +## 8. Migration interaction (controller v0.65.0) — ⚠ FLAG + +`MigrateAll` copies "everything under felhom-data", so `userdata/` is covered. **But ownership/setgid +survival is split:** +- **App subtrees** (`appdata/` + recovery unit) go via `rsyncCopy` = `rsync -a --checksum` — + `-a` includes `-o -g -p`, and the controller runs **as root in the guest** (Dockerfile has no `USER`), + so owner/group **and the setgid bit are preserved**. ✓ +- **Non-app content** (which is exactly `userdata/`, `media/`, customer dirs) goes via the **custom + conflict-merge walk** (`migrate.go` `walkMerge`/`copyFile`), NOT rsync. That code does + `os.MkdirAll(dst, 0o755)` for dirs (**drops the setgid bit + sets root:root**) and `copyFile` preserves + only `fi.Mode().Perm()` (**`.Perm()` masks off setgid/setuid**) and does **not** chown. **So a + MigrateAll would land userdata/ at the target as `root:root` 0755 — the shared-ownership convention + would NOT survive a whole-drive migration.** This must be fixed (re-apply the convention post-copy, or + teach `walkMerge`/`copyFile` to preserve setgid+group) as part of implementation. Per-app `MigrateApp` + doesn't touch userdata (it only moves the app's appdata+recovery-unit), so it's unaffected. + +--- + +## 9. Transition (existing demo has `media/` in the old layout) + +- Live: `/mnt/felhom-usb/media/books` (calibre-web) already exists, guest-root 0755. New deploys would + use `userdata/` but existing media stays at `media/`. +- Options: **(a) new-layout-for-new-deploys-only**, leave existing `media/` (simplest; existing apps keep + working until re-deployed). **(b) one-time move** `media/ → userdata/media/` + re-chown to the + convention + redeploy affected apps. The just-shipped per-app migrate does NOT do this (it moves + *appdata* between drives, not `media→userdata` within a drive); it would need a small dedicated + migration step (mv + chown -R + chmod -R, then redeploy). **Recommend (a) for the cutover** + a small + opt-in one-time fixup script for the demo, executed under supervision — not auto-run. + +--- + +## Summary table + +| # | Finding | Evidence | +|---|---|---| +| 1 | `media/`/`appdata/` are **guest-root 0:0 mode 755**; Docker auto-creates bind sources as root (calibre's `media/books` proves it) — the collision cause | live `ls -la` (guest + host 100000) | +| 2 | `${HDD_PATH}` = the in-guest namespace root (`/mnt/felhom-usb`); `${HDD_PATH}/userdata` works uniformly | `romm/app.yaml HDD_PATH=/mnt/felhom-usb`; `paths.go:11,24,68` | +| 3 | **9 apps / ~13 mounts** use `${HDD_PATH}/media`; ROMM library is compose-path-only under appdata | catalog grep (table above) | +| 4 | FileBrowser + 5 apps already at **uid/gid 1000**; only komga + audiobookshelf (root, RW) are the problem set | live `id` + catalog | +| 5 | FileBrowser **uid 1000** by image default, mounts `appdata/` today; switch to `userdata/` is a one-line mount change | live `docker exec filebrowser id`; `handlers.go:~1464`, `infra.go:~116` | +| 6 | unprivileged LXC, default idmap (0→100000, 1000→101000); convention is guest-internal | `/etc/pve/lxc/9201.conf` | +| 7 | Convention: `userdata` `:1000` mode **2775** (setgid); umask caveat → komga/audiobookshelf need `user:1000` or UMASK; linuxserver apps need `UMASK=002` | synthesis | +| 8 | ⚠ MigrateAll's **non-app merge walk drops setgid+owner** on userdata — convention won't survive a migration as-is | `migrate.go walkMerge`/`copyFile` | +| 9 | Existing `media/` stays; recommend new-layout-for-new + supervised one-time fixup for the demo | live | + +--- + +## FORK LIST (operator's call — GATE for implementation) + +1. **Shared GID value** — **`1000`** (recommended: already FileBrowser's gid + the PUID/PGID of + calibre/sonarr/radarr/emby/paperless; zero identity change for 6 of the players) **vs** a dedicated + `felhom` group (cleaner semantics, but every app needs `group_add`/PUID set to it). +2. **Common-group-only vs one-UID-for-all-apps** (operator's open question): + - *Common-group (setgid 2775 + per-app UMASK/user tweaks):* preserves each app's native identity + (linuxserver PUID dance, root-init images work), but needs the umask fix on root-RW apps + (komga/audiobookshelf) and `UMASK=002` on linuxserver apps. More per-app tuning, fewer image + surprises. + - *One-UID-for-all (`user: "1000:1000"` on every content app):* simplest permissions (everything is + 1000, no setgid/umask reasoning), but **breaks images that must initialize as root** — linuxserver + images run their PUID/PGID setup as root then drop; pinning `user:` skips that; some official images + (jellyfin/plex) and root-only apps may misbehave. Higher image-compat risk. + - *Recommendation:* **common-group (setgid) with GID 1000**, pin `user:`/UMASK only on the few + root-RW-to-shared apps. +3. **userdata subfolder set** — confirm `media/{movies, "tv shows", music, audiobooks, books}`. The space + in `"tv shows"` is cosmetic and workable (quote in compose/scripts); a hyphen/underscore avoids + quoting. Decide: keep the space, or `tv-shows`. Also: do `downloads/` (sonarr/radarr staging) and + `comics/` (komga) live under `userdata/media/` too? +4. **ROMM import location under userdata** — repoint `library` to e.g. `${HDD_PATH}/userdata/roms` + (browsable) while keeping `resources` under `appdata/romm`? Confirm the exact path + that RomM tolerates + an empty/new library dir. +5. **Media path scheme** — edit each compose to `${HDD_PATH}/userdata/media/...` **vs** introduce a single + **`USERDATA_PATH`** env (one var, set at deploy like HDD_PATH; cleaner, one place to change, and makes a + future layout change a one-liner). *Recommendation: `USERDATA_PATH`.* +6. **Transition** — new-layout-for-new-deploys-only (recommended) **vs** migrate existing `media/ → + userdata/media/` now (needs a supervised one-time fixup; the v0.65.0 per-app migrate does NOT cover + this in-drive move). +7. **(Surfaced, needs a call)** **Fix MigrateAll's merge walk to preserve setgid+group** (or re-stamp the + convention post-migration) — otherwise the shared-ownership convention silently degrades to root:root + on any whole-drive migration (#8). Decide whether this rides in the same slice.