docs: REPORT + README — intermediary-mount model (controller half)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,76 +1,42 @@
|
|||||||
# REPORT — userdata arc close-out: FileBrowser umask + Immich/Paperless deploy → controller v0.66.2 + catalog (2026-06-15)
|
# REPORT — intermediary-mount re-architecture (controller half) → v0.67.5 (2026-06-15)
|
||||||
|
|
||||||
Finished the userdata arc: (1) FileBrowser-created folders are now group-writable; (2) Immich deployed
|
## Type
|
||||||
with its external photo library registered + verified; (3) Paperless deployed and consume verified
|
Controller half of the intermediary-mount re-architecture (pairs with agent v0.35.1 + catalog repoint).
|
||||||
end-to-end. Trunk-based on `main` (controller + app-catalog). Built + deployed + live-verified on demo
|
Built + Linux-gated (`go vet`/`go test ./...` green) + deployed to demo guest 9201, which was then
|
||||||
guest 9201 (now 12 GiB RAM).
|
migrated live. Drives are now visible in-guest ONLY at the STABLE path `/mnt/felhom-drives/<name>` (the
|
||||||
|
host swaps the backing drive underneath it — no per-drive `pct` mp, no guest reboot for attach/detach).
|
||||||
|
Design: `felhom.eu/documentation/audits/SPIKE-intermediary-mount-2026-06-15.md`.
|
||||||
|
|
||||||
> Validate against pushed source at file:line, not this report.
|
## What changed (internal/web/intermediary.go + storage_handlers.go + api/router.go + settings.go)
|
||||||
|
- **HDD_PATH repoint.** The registered storage path + every app's HDD_PATH + FileBrowser source are now
|
||||||
|
the stable `/mnt/felhom-drives/<name>`; the AGENT still operates on the raw `/mnt/<name>` (mapped via
|
||||||
|
`agentWhere()` at the assign/attach/eject/decommission call sites). Enroll binds-under-the-parent
|
||||||
|
BEFORE register (the controller only sees the drive at the stable path post-attach). New
|
||||||
|
`settings.RepointStoragePath`; `agentapi.DiskInfo` gains `GuestPath`/`BoundUnderParent`.
|
||||||
|
- **Drive-absent GATE** (`planDriveGates` pure core + `driveGateLoop` 30s, replacing the retired
|
||||||
|
watchdog). An ABSENT external drive (only `/mnt/felhom-drives/<name>` paths — never SSD/system) stops +
|
||||||
|
blocks its apps (`StoppedStacks` = gate-stopped set); a RETURNED drive re-attaches + auto-restarts
|
||||||
|
them. `present` keys on `BoundUnderParent` (guest-usable), not host-mount. Start-gate in `actionStack`
|
||||||
|
refuses to start an app whose drive is absent.
|
||||||
|
- **Guest-reboot convergence** (`recreateBootStaleApps`, one-time at startup after a bounded stack-scan
|
||||||
|
wait): docker auto-starts app containers on a guest reboot possibly before the agent re-propagates the
|
||||||
|
drive, so they bind the empty stable dir; the controller (which restarts with the guest) recreates
|
||||||
|
present drive-backed apps that are boot-stale/exited/restarting onto the populated path.
|
||||||
|
- **H1 endpoints routed** (were 404): `POST /api/storage/{disconnect,reconnect,restart-apps}` → host-side
|
||||||
|
eject/reconnect (no guest reboot).
|
||||||
|
|
||||||
## Confirmed baseline
|
Version path v0.67.0 → .5 (`.1` gate-only-externals, `.2` BoundUnderParent, `.3`/`.4`/`.5` startup
|
||||||
- Controller `main` was `d0718e8` = **v0.66.1** → shipped **v0.66.2**.
|
recreate hardening). Non-hollow tests + companions: `TestPlanDriveGates` (5 states incl. SSD-never-gated),
|
||||||
- Guest 9201: `pct config` memory **12288** (12 GiB) — confirmed before the deploy phases (was 2 GiB).
|
`TestAgentWhere`, `TestRunStorageInit_Success` (agent gets RAW, registry gets STABLE).
|
||||||
|
|
||||||
## Commit hashes
|
## Live migration (guest 9201) + verification
|
||||||
| Repo | Commit | Scope |
|
8 deployed apps (all on felhom-flash) repointed (app.yaml HDD_PATH rewrite + controller-driven
|
||||||
|---|---|---|
|
down/up recreate, which decrypts the `ENC:` secrets) + registry repointed + FileBrowser re-synced + the
|
||||||
| controller | `98d5504` | v0.66.2 — FileBrowser umask 002 entrypoint wrapper + test |
|
legacy `mp2` deleted. **A guest reboot self-converges**: the startup recreate logged + rebound the
|
||||||
| controller | (this) | docs: README FB umask + REPORT |
|
drive-backed apps onto `/mnt/felhom-drives/felhom-flash` with no manual intervention. End state: **25
|
||||||
| app-catalog | `67037cc` | paperless OCR lang-pack fix (space-separated `PAPERLESS_OCR_LANGUAGES`) |
|
containers healthy** (komga healthcheck pre-existing quirk), confinement + data integrity verified.
|
||||||
|
|
||||||
## Phase 1 — FileBrowser umask (mechanism: ENTRYPOINT WRAPPER, not UMASK env)
|
## Known residuals (documented)
|
||||||
Verified the gtstef/filebrowser:1.3.3-stable image is a single Go binary (`entrypoint ./filebrowser`,
|
- Startup recreate is state-sampled; an app cleanly stopped at the exact sample instant may need one
|
||||||
runs uid 1000) that does **NOT** honor a `UMASK` env — live test: `docker run -e UMASK=002 … umask` →
|
manual start (1/8 once). Deterministic fix = a guest-boot-id signal from the agent (future).
|
||||||
`0022`. So `RenderFileBrowserCompose` (`internal/infra/infra.go`) wraps the entrypoint:
|
- felhom-usb left on its legacy bind (no deployed apps + stale `decommissioned` intent).
|
||||||
`["sh","-c","umask 002; exec /home/filebrowser/filebrowser"]`. Test asserts the rendered compose carries
|
|
||||||
it. **Live proof on 9201** (after deploying v0.66.2 → FileBrowser re-rendered): PID1 `Umask: 0002`; a
|
|
||||||
folder created at that umask under the setgid parent → **2775 group 1000**; a uid/gid-1000 app process
|
|
||||||
(busybox `--user 1000:1000`) wrote a file INTO that customer folder (664 group 1000) → `APP_WRITE_OK`;
|
|
||||||
FileBrowser then deleted it. The pre-fix `Teszt` folder (2755) was the only stale dir — normalized with a
|
|
||||||
one-time `find -type d -exec chmod g+rwxs` (no data); no 2755 dirs remain.
|
|
||||||
|
|
||||||
## Phase 2 — Immich: DEPLOYED + EXTERNAL LIBRARY REGISTERED + verified
|
|
||||||
- Deployed via the controller deploy flow (the dashboard's `/api/stacks/immich/deploy` over the public
|
|
||||||
URL — mem-gate + USERDATA_PATH + belt all run; NOT a raw compose/CLI shortcut). **Mem-gate PASSED**
|
|
||||||
(it refused at 2 GiB; cleared at 12 GiB). All 4 containers **healthy**; memory: server 916 MiB, ML
|
|
||||||
231 MiB, postgres 340 MiB, redis 5 MiB (~1.5 GiB total; 8.7 GiB available after).
|
|
||||||
- External mount confirmed: `/mnt/felhom-usb/userdata/media/photos → /external/photos (ro)`, 2775 setgid.
|
|
||||||
- **External library REGISTERED via the Immich API** (no browser tool in this env — see Note): admin
|
|
||||||
account created (creds stored out-of-band), `POST /api/libraries` with `importPaths:["/external/photos"]`.
|
|
||||||
- **End-to-end VERIFIED**: a JPEG placed in `media/photos` by a uid-1000 process → `POST /api/libraries/{id}/scan`
|
|
||||||
→ `GET …/statistics` returned `{"photos":1,"total":1,"usage":631}`. The photo appears in Immich.
|
|
||||||
|
|
||||||
## Phase 3 — Paperless: DEPLOYED (UID 1000) + consume verified + catalog OCR bug fixed
|
|
||||||
- Deployed via the controller deploy flow; mem-gate passed. Runs as **uid 1000 (USERMAP_UID/GID 1000)**;
|
|
||||||
consume mount `/mnt/felhom-usb/userdata/import/paperless → /usr/src/paperless/consume`.
|
|
||||||
- **Found + fixed a real catalog bug** (`67037cc`): the default `PAPERLESS_OCR_LANGUAGE=hun+eng` crash-looped
|
|
||||||
— the compose fed the `+`-joined value into `PAPERLESS_OCR_LANGUAGES` (the install list, which must be
|
|
||||||
space-separated), so the image tried to apt-install a bogus `tesseract-ocr-hun+eng` → `hun` pack missing
|
|
||||||
→ Django check crash-loop. Confirmed it's a name bug not network (`tesseract-ocr-hun` installs cleanly).
|
|
||||||
Fix: `PAPERLESS_OCR_LANGUAGES=eng hun deu` (fixed space-separated superset). **Verified**: redeployed with
|
|
||||||
the default `hun+eng` → boots **healthy**, `tesseract --list-langs` = `deu eng hun`.
|
|
||||||
- **Consume VERIFIED end-to-end** (on the final hun+eng deploy): a PDF dropped into `import/paperless` by a
|
|
||||||
uid-1000 process → within ~45 s `consume_files_left=0` (paperless **read + deleted** it from the 2775
|
|
||||||
shared dir) and `doc_count=1` (ingested/OCR'd). Proves write+delete in the shared dir works.
|
|
||||||
|
|
||||||
## Phase 4 — close-out
|
|
||||||
Controller **v0.66.2 Up (healthy)** on 9201; FileBrowser re-rendered with the umask wrapper; clean logs;
|
|
||||||
8.5 GiB free. Full userdata collaboration story now holds with the heavy apps live: customer-created
|
|
||||||
folder usable by apps (P1), Immich photos via external library (P2), Paperless consume (P3).
|
|
||||||
|
|
||||||
## Tests (Linux gate green on 192.168.0.180)
|
|
||||||
`internal/infra`: `RenderFileBrowserCompose` emits the umask-002 entrypoint wrapper. Full
|
|
||||||
`go build && go vet && go test ./...` green.
|
|
||||||
|
|
||||||
## Note on the deploy method (transparency)
|
|
||||||
The task specified deploying via the dashboard with Claude-in-Chrome, but no browser-automation tool was
|
|
||||||
available in this environment. I used the controller's **real deploy endpoint** (the same JSON API the
|
|
||||||
dashboard's deploy button calls, over the public URL through traefik/cloudflared) — so the mem-gate,
|
|
||||||
USERDATA_PATH injection, and deploy belt all executed; this is NOT a raw `docker compose`/CLI shortcut.
|
|
||||||
The Immich external-library registration used the Immich REST API (admin sign-up → login → create library
|
|
||||||
→ scan), fully completed and verified, not left as a documented manual step.
|
|
||||||
|
|
||||||
## Pending / follow-ups (noted, not done)
|
|
||||||
- **komga** reports `unhealthy` via its `/api/v1/actuator/health` probe on komga 1.20 despite serving
|
|
||||||
(pre-existing healthcheck-definition issue, unrelated to userdata) — fix the healthcheck.
|
|
||||||
- Demo admin accounts created for Immich + Paperless to drive verification (creds out-of-band).
|
|
||||||
|
|||||||
@@ -621,6 +621,24 @@ not just those with HDD data. Non-HDD apps can configure destination, method, an
|
|||||||
|
|
||||||
### 4. Storage Management
|
### 4. Storage Management
|
||||||
|
|
||||||
|
> **⚠️ INTERMEDIARY-MOUNT model (v0.67.x, pairs with agent v0.35.x).** External data drives are now
|
||||||
|
> visible in the guest at a STABLE path **`/mnt/felhom-drives/<name>`**, NOT the raw `/mnt/<name>`. The
|
||||||
|
> agent keeps a single permanent parent bind `/mnt/felhom-drives` in the guest and swaps each drive's
|
||||||
|
> `felhom-data` namespace **underneath it host-side** (`mount --bind`), so the guest sees attach/detach
|
||||||
|
> **live with no reboot** (mount propagation), the bind source never disappears (C1-immune), and only
|
||||||
|
> `felhom-data` crosses in (confinement). The per-drive `pct set -mpN` bind is **deprecated**.
|
||||||
|
> - The registered storage path + every app's **`HDD_PATH`** + the FileBrowser source = the stable
|
||||||
|
> `/mnt/felhom-drives/<name>`. The controller maps it back to the raw `/mnt/<name>` (`agentWhere()`)
|
||||||
|
> only for agent calls (assign/attach/eject/decommission). `GET /api/disks` carries `guest_path` +
|
||||||
|
> `bound_under_parent` (the guest-visible signal).
|
||||||
|
> - **Drive-absent gate** (`internal/web/intermediary.go`, `driveGateLoop` 30s): an absent
|
||||||
|
> `/mnt/felhom-drives/<name>` drive stops + blocks its apps (`StoppedStacks` = the gate-stopped set);
|
||||||
|
> a returned drive re-attaches + auto-restarts them; `actionStack` refuses to start an app whose drive
|
||||||
|
> is absent ("tárhely nem elérhető"). SSD/system paths are never gated.
|
||||||
|
> - **H1 endpoints** `POST /api/storage/{disconnect,reconnect,restart-apps}` drive the host-side
|
||||||
|
> eject/reconnect (no guest reboot). A guest reboot self-converges (`recreateBootStaleApps` recreates
|
||||||
|
> boot-stale drive-backed apps onto the re-propagated drive once the agent re-binds it).
|
||||||
|
>
|
||||||
> **⚠️ Rebuilt on the agent-delegated disk model (v0.43.0), made ROLE-AWARE in v0.44.0, UX-polished in
|
> **⚠️ Rebuilt on the agent-delegated disk model (v0.43.0), made ROLE-AWARE in v0.44.0, UX-polished in
|
||||||
> v0.45.0.** After the 8C
|
> v0.45.0.** After the 8C
|
||||||
> de-privileging, the controller holds **no Proxmox/disk credentials and no destructive authority** — disk
|
> de-privileging, the controller holds **no Proxmox/disk credentials and no destructive authority** — disk
|
||||||
|
|||||||
Reference in New Issue
Block a user