diff --git a/documentation/architecture/00-capability-map.md b/documentation/architecture/00-capability-map.md index a834aa2..0e4d7cc 100644 --- a/documentation/architecture/00-capability-map.md +++ b/documentation/architecture/00-capability-map.md @@ -93,7 +93,7 @@ | Documents/OCR | paperless-ngx (classified) | **PROVEN-LIVE** | `CAMPAIGN-6C` 4P-1 (deploy paperless-ngx, ingest 3 docs via consume flow, OCR + PDF/A ~90s) + 4P-2/3/5 | Consume-folder ingestion awkward without SMB → R-7 | | Files from Windows Explorer / Mac Finder (SMB **server**) | controller v0.145.0 + `felhom-samba:1.0.0` | **PROVEN-LIVE** | `felhom-controller` REPORT.md (v0.144.0) + `controller/sharing.md`; transport verdict `audits/SPIKE-lan-discovery-2026-07-18.md` | „Megosztás" page: enable + one household password + shares (new folder or picked existing, per-share read-only). Fourth protected infra stack (host-net, smbd+**nmbd**+wsdd). Live on demo: 445 reachable, NetBIOS `FELHOM` resolves, write/read byte-compare PASS, write to a read-only share REFUSED, SMB writes land as uid 1000. **Explorer leg PASSED 2026-07-18 (Viktor):** Network → FELHOM → both shares open; a real Explorer *save* into `dokumentumok` landed owned uid 1000, and a write into the read-only `filmek` was refused by Windows with the folder left untouched. Share data RIDES BOTH BACKUP TIERS (R-7b, controller v0.145.0, Model B′ sibling shares source): tier-2 cross-drive legs + an offsite `_shares` restic snapshot carrying the share definitions and the credential copy, with a „Megosztások" restore. **All four legs PROVEN-LIVE on demo 2026-07-18** — tier-2 tree md5-verified; offsite snapshots `e0b9d723` (Viktor 12:18:16Z) and `4e2b15ec` both carrying manifest + `passdb.tar`; restore round-trip returned a deleted probe file byte-identical and a deleted share DEFINITION with its original flags without overwriting live files; samba liveness → hub-accepted `health_critical`. Remaining human leg: SMB positive auth with the real household password | | Media to TV via DLNA | — | **MISSING** | — | Jellyfin app exists; DLNA/SSDP unvalidated → R-6, R-8 | -| File access via browser | FileBrowser (infra app, auto-mount sync) | **IMPLEMENTED** | FileBrowser runs healthy + userdata-bound (`storage-lifecycle-acceptance-2026-06-15`, `CAMPAIGN-3`) | Actual browse/download **through FileBrowser** is exercised in no doc. (Cited `CAMPAIGN-2` T-PAGE-ALL renders only the controller dashboard pages, not FileBrowser.) Demoted | +| File access via browser | FileBrowser (infra app, auto-mount sync) | **IMPLEMENTED** | FileBrowser runs healthy + userdata-bound (`storage-lifecycle-acceptance-2026-06-15`, `CAMPAIGN-3`) | Actual browse/download **through FileBrowser** is exercised in no doc. (Cited `CAMPAIGN-2` T-PAGE-ALL renders only the controller dashboard pages, not FileBrowser.) Demoted. **2026-07-26, controller v0.172.0 (R-75) — status DELIBERATELY UNCHANGED.** The canonical drop-zone now has its own FileBrowser source („Beolvasás" → `/srv/beolvasas`, a separate bind of `/userdata/import`) and the app page carries a per-app deep link into it. Verified live on demo-hp: the source and bind are in the generated config, the app page renders `https://files.enkisfelhom.hu/files/Beolvas%C3%A1s/paperless`, and a file written through FileBrowser's OWN mount was consumed and deleted by paperless in ~30 s. **That is still not a browse.** Nothing in this arc drove the FileBrowser HTTP UI — no browser exists on DooPlex — so the row's standing caveat survives intact and the upgrade to PROVEN-LIVE remains unearned. What it would take: a human click-through, or an authenticated `/api/resources` round-trip against the live instance. See `controller/import-and-data-paths.md` | | **Indítópult (app launcher) — one-tap grid of the household's openable apps** | controller v0.163.0 | **IMPLEMENTED** | New FIRST sidebar page `/launcher`: colored tiles (deterministic slug color or `.felhom.yml` `brand_color`) + white glyph/monogram, one per openable app (tile ⟺ „Megnyitás" — subdomain presence is the single criterion; controller excluded). Operational → `` to the public URL; stopped → greyed + state badge, no link. `/` stays the Vezérlőpult. Endpoint-level + render-test verified; `felhom-controller/REPORT.md` (2026-07-24) | Live operator click-through of a real tile → app pending (browser automation not available on DooPlex). Follow-up: curate `brand_color` for top catalog apps (R-72). Sharing the launcher outside the household is now the capability-URL guest link — see the row below | | **Indítópult megosztás (vendég link)** — capability URL `/s/` serves a standalone read-only guest launcher (no account, no admin session); optional per-share password; QR | controller v0.165.0 | **IMPLEMENTED** | 160-bit `crypto/rand` token, constant-time match (empty stored = disabled = byte-identical to the mux default 404); guest headers `noindex`/`no-referrer`/`no-store`; optional SEPARATE bcrypt share password + its own per-IP attempt map; signed cookie `HMAC(token`\|`passwordHash)` keyed with `session_secret` (rotate-token OR change-password invalidates all cookies); token redacted in logs (`/s/`). Groups A–G (14 tests) + 3 red-proofs; **§13 endpoint-level live validation on 9201 all-pass** (`felhom-controller/REPORT.md` 2026-07-24). **Design ruling: member accounts SUPERSEDED by this capability-URL model; per-member tile visibility parked under the SSO arc (R-15).** | Full operator browser click-through + a validation doc pending → then PROVEN-LIVE. Accepted residuals: link-preview crawlers fetch once (noindex prevents indexing); reverse-proxy/CF access logs hold the path (ops-tier); the modal link carries the request Host (LAN-IP admin ⇒ LAN-IP link) | | Forgot dashboard password → instant reset code | controller v0.123, hub | **PROVEN-LIVE** | `DRILL-day0-take2-2026-07-12` F-15 (live re-run of the exact failure path: hash applied 1s after request, code accepted first try) | | diff --git a/documentation/backlog/ROADMAP.md b/documentation/backlog/ROADMAP.md index eab3f49..bdae190 100644 --- a/documentation/backlog/ROADMAP.md +++ b/documentation/backlog/ROADMAP.md @@ -104,7 +104,7 @@ | R-72 | Curate `brand_color` for the top catalog apps | XS | idea | Parked follow-up to the v0.163.0 launcher. `.felhom.yml` `brand_color` (`#rgb`/`#rrggbb`) overrides the deterministic slug-hash tile color; no catalog app sets it yet. Pick brand-accurate colors for the most-installed apps so their launcher tiles match their real brand. Catalog-only change (`app-catalog-felhom.eu`), `brand_color` is already `omitempty` and consumed by the controller | | R-73 | **SMART history / trending (hub-side)** | M | idea (parked) | Parked follow-up to the v0.169.0 disk-health card + degradation alert (agent v0.94.0 serializes SMART into `/disks`; controller renders „Lemezek állapota" + fires `disk_health_degraded` on a degradation). Today the verdict is point-in-time and the baseline is in-memory (a controller restart re-baselines silently). Hub-side would let the operator SEE a disk's SMART counters trend over time (reallocated/pending sectors climbing, NVMe wear %), catch slow degradation the 6h transition check can miss between reboots, and persist the baseline across restarts. Needs a hub ingest surface for periodic SMART snapshots + a small time-series store + an operator trend view. **No demo hardware exposes real SMART** (virtualized), so this only earns evidence on a real box | | R-74 | **Island control plane on a CLUSTER (Peti's 2 nodes)** — bring R-50's island bridge to a multi-node PVE cluster. | M | idea (Phase C of R-50, parked) | R-50 shipped the island for the ONE-host fleet (demo-hp, demo-felhom). A cluster needs **bridge parity on every node**: either per-node identical `/etc/network/interfaces` `vmbr9` stanzas (simplest, drift-prone) or — preferred at ≥2 nodes — a Proxmox **SDN zone/vnet** defined cluster-wide (one definition, auto-applied per node). The guest island IP is per-guest + node-independent; the **agent-follows-guest** rule holds (each node's agent binds its own `vmbr9` `169.254.253.1`). Migration order per the spike: drill-proven → demo (done) → **Peti (this row)**. Its own supervised runbook, coordinated with Peti (a live customer). Completes the capability-map "site/network change" row for clustered installs. Source: `audits/SPIKE-island-bridge-2026-07-25.md` (cluster-parity finding) + `RUNBOOK-island-migration.md` (single-host procedure to generalise) | -| R-75 | **Catalog-derived userdata skeleton + import surfaces** — replace the hardcoded `appbackup.UserdataSkeleton()` Go list with a set DERIVED from the catalog, and give the customer a way to find an app's folders: a named FileBrowser sidebar source for the `import/` tree, a per-app deep link from the controller's app page, and `data_paths:` in `.felhom.yml` as a role+Hungarian-label ANNOTATION over paths that must already exist as userdata-rooted compose binds. | M | **spiked (2026-07-26) — GO, one mandatory constraint** | `audits/SPIKE-catalog-data-paths-2026-07-26.md`. **Phase 0 HOLDS:** the path set is fully derivable from `ParseComposeClassifiableBinds` (`stacks/classify_binds.go:38`) and `ValidateBackupSpec`'s refusal set (`appbackup/classify.go:97`) already covers the path-safety class `data_paths:` needs — annotation-only introduces **no new filesystem-write primitive**. Every non-`${VAR}` host token in all 53 templates is a named docker volume or the docker socket; **no** app's folder is reachable only via env indirection; **zero** templates use long `type: bind` syntax. **The derivation is ALREADY LIVE at deploy time** — `stacks/manager.go:183` `ensureUserdataMounts` ("the deploy belt") derives userdata dirs from compose and applies 2775/GID-1000 per app; proven by a clean two-box control (P5: `media/podcasts` exists on demo-felhom where audiobookshelf is deployed and NOT on demo-hp, and it is in no skeleton). So this item is mostly *promoting a shipped lazy mechanism to an eager, annotated one*. **Diff is tiny:** derived set 14 dirs vs hardcoded 14 — `+media/podcasts`, `−documents`; `documents` is implied by no app, exists on BOTH boxes, and is the entire removal risk → **zero-removals invariant is mandatory**, proven against the P5 before-state. Only **2 of 53** apps have an `import/*` path (paperless-ngx, calibre-web), and **both are `class: excluded`** — any "put your files here" copy must say temporary+unbacked, drivable from data since every catalog class is `explicit`. **MANDATORY CONSTRAINT (P6):** the naive map-based derivation produced **20 distinct outputs from 20 identical runs** (1/20 identical); `fbNeedsRecreate` force-recreates on any byte diff, so that is a guaranteed FileBrowser restart loop on all ~14 sync call sites — fleet-wide, v0.151-class. `sort.Strings` → 20/20. **The spec must state the sort + a determinism red-proof as a requirement.** Mitigating: the catalog sync does **not** currently trigger `SyncFileBrowserMounts` (`main.go:215` wires only `ScanStacks` + `InjectMissingFields`, and even that is change-gated), so the 15-min storm only appears if the impl wires them together. **FileBrowser Quantum 1.3.3 carries the UI half:** 4 sources incl. nested + accented/spaced/`&` names all index `ready` with no warning; deep-link template `{baseURL}files/{encodeURIComponent(source)}/{encodeURIComponent'd segments}` is constructible in Go from `(sourceName, relPath)` alone, and the router guard preserves `?redirect=fullPath` so a cold deep link survives login. Use `url.PathEscape` — **not** `QueryEscape` (space→`+` breaks a path segment). Scale safe to 12 sources server-side. **Share guard needs no change:** `userdata/import` is not in `SharingDeniedRoots` and the real `sharingResolvePath` accepts it while still refusing appdata/root/relative/traversal/symlink-escape (P4, all green, no live share created). **Two pre-existing defects surfaced, deliberately NOT fixed here** — see R-76. Flips capability-map row *File access via browser* (line 96, currently **IMPLEMENTED** with the honest caveat "actual browse/download through FileBrowser is exercised in no doc") toward PROVEN-LIVE, and is the natural home for R-7 slice 2's "curated app-folder presets (one-click paperless `consume/`)". **Design forks recorded, awaiting operator ruling:** per-drive vs canonical import root (rec: per-drive); all-apps vs deployed-only (rec: all-apps skeleton, deployed-only in the UI); unknown-role fails OPEN per the R-57 lifecycle precedent vs whole-block reject per the `backup:` precedent (rec: OPEN for roles, reject for malformed paths); backup-class collision copy (rec: data-driven from the class) | +| R-75 | **Catalog-derived userdata skeleton + import surfaces** | M | **SHIPPED (controller v0.172.0 + catalog, 2026-07-26)** | `${IMPORT_PATH}` = `/userdata/import` — ONE canonical drop-zone per box on the system drive, with **no per-drive fallback** (unresolvable leaves the var unset so compose fails loudly). *Operator ruling overriding the spike's Fork-1 per-drive recommendation:* each drop-zone app has exactly one ingest bind, so a per-drive `import/` would put a dead lookalike on every other drive — unbacked too, since `import/*` is `class: excluded`. Third `BindRoot` (`RootImport`) threaded through the classifier AND every resolver (`resolveAbs` would otherwise have named a directory on the WRONG DRIVE); catalog-derived skeleton, **sorted** (the naive map-order derivation measured 20 distinct outputs from 20 identical runs — a fleet-wide FileBrowser restart loop via `fbNeedsRecreate`); one authoritative compose parser; FileBrowser „Beolvasás" source; app-page „Hova tegyem a fájlokat?" with `PathEscape` deep links and **class-driven** copy; `data_paths:` annotation (malformed path ⇒ whole-block reject, unknown role ⇒ fails OPEN); system-owned `beolvasas` SMB share refused server-side at handler AND store. **Live-proven on demo-hp:** consume bind resolves to the system drive, a file dropped through FileBrowser's mount was ingested and deleted in ~30 s, the share auto-appeared on enable and survived a direct delete POST, and the before/after directory diff on BOTH boxes shows **zero removals** (demo-hp gained `media/podcasts`, the one genuinely derived entry). Two defects found and fixed during the live legs: `EnsureImportRoot` left the parent `userdata` at 755, and the carry-list initially re-created the per-drive drop-zone forever. Capability-map row *File access via browser* **deliberately NOT upgraded** — nothing drove the FileBrowser HTTP UI. Docs: `controller/import-and-data-paths.md`, spike `audits/SPIKE-catalog-data-paths-2026-07-26.md` | | R-76 | **FileBrowser-created folders break the setgid chain, and a drop-zone's mode is not stable** | S | idea (surfaced by the R-75 spike, 2026-07-26) | Two related findings from `audits/SPIKE-catalog-data-paths-2026-07-26.md` P3/P5, both **pre-existing** and deliberately left alone by that spike. **(a)** FileBrowser Quantum 1.3.3 creates files `0644` and folders `0755` and does **not** propagate the setgid bit — even though the entrypoint wrapper's `umask 002` really is in effect (`/proc/1/status` `Umask: 0002`). Group inheritance itself works (a file uploaded into a 2775 group-100 dir landed group 100, not the process gid 1000), so the convention's *group* half holds and only its *mode* half is lost. The consequence is proven with a control: inside a UI-created `0755` folder a gid-1000 process's file landed group **1000**, while the identical write into the 2775 parent landed group **100**. So **any folder a customer creates through FileBrowser breaks the shared-group chain one level down.** Latent today — every userdata-touching catalog app that declares an identity declares uid/gid **1000**, the same uid FileBrowser runs as, so owner permissions mask it; it bites the day a content app runs as a different non-root uid with gid 1000. The comment at `infra/infra.go:156` is right that the image ignores `-e UMASK` but does not say the wrapper fails to achieve the intended mode either. **(b)** `import/calibre` is live on demo-felhom at `755 1000:1000` where every sibling is `2775 root:1000` — with `media/books` on the same box, same app, same deploy, at `2775` as the control, and no parser asymmetry (checked: `ParseComposeUserdataMounts` picks up both calibre-web binds). Consistent with the consuming app rewriting the mode of its own ingest dir after the deploy belt sets it; **not confirmed causally**. Together they mean **a drop-zone directory's mode is not stable against either the customer or the consuming app** — verify before building anything that assumes `import/*` stays 2775. Flips no capability-map row today (latent); would become customer-visible the moment a non-1000 content app enters the catalog | | R-21 | **Bare-metal Felhom ISO** — per-PVE-release auto-install ISO for blank customer hardware → first-boot wrapper (invokes `felhom-host-install.sh`) → universal secret-free / operator-bind (option C) | XL | **SHIPPED + PHYSICALLY CLOSED (slices A+B+C; rehearsal executed 2026-07-18)** | **PHYSICAL CLOSURE 2026-07-18** (`tests/VALIDATION-n100-rehearsal-2026-07-18.md`): the generic pairing ISO v1.20.0 (`--loader mkimage`, SB off) **booted the very AMI board that F1 blocked**, installed unattended, and the box self-registered as an unclaimed appliance the same second it first booted (16:17:14) → self-bind → credential → day-0 SUCCESS 16:32:32 → floor-lifted to current. **F1 is closed on physical hardware and the rehearsal dependency on this item is discharged.** Two residual notes stay open, neither blocking: PXE/network-boot is still unbuilt (the third F1 option, for boards where even USB-mkimage fails), and mkimage remains unsigned → **SB must be OFF**. The installer's GRUB menu still offers interactive installers → new item **R-38**. — Prior: **PHYSICAL RUN 2026-07-16 (`tests/VALIDATION-n100-baremetal-2026-07-16.md`):** demo N100 reinstalled clean-slate from a pipeline ISO → chain reached **rc-0 first try on real hardware** (closes slice A's operator-gated boundary), serial-filter safety proven on metal, PBS-DR reconciler self-healed on the reused peer, DMI verdict = key on MAC+UUID. **F1 (HIGH, slice-B input):** this cheap AMI `AN3PLUS 0.01` firmware won't UEFI-boot the ISO's GRUB from USB (`relocation 0x0`) — SB-off/shim-bypass don't help; worked around live with a `grub-mkimage` loader built from the box's own GRUB. Pipeline must ship a firmware-compatible loader / PXE path. Reused-customer edges (F2 claim re-issue, F3 offsite re-issue, F4 non-default-storage-id ACL 403) feed R-1/Peti. UX: F6 drive-init doesn't mount+attach, F5 guest-RAM not configurable, F7 back-route. — **Slice A (build pipeline + first-boot bootstrap) DONE + validated on VM 310:** build gate/red-proof, disk-filter fail-safe, stub→retry-unit→real public-channel host-install fetch+invoke→retry, resume-decision, exactly-once, no-net retry+recovery all GREEN. Operator-gated remainder: host-install rc-0 terminal success (drill customer needs the password-gated create-UI). **Slice B — SHIPPED (scripts v1.18.0, 2026-07-17):** the F1 firmware fix is now a first-class pipeline mode `build-felhom-iso.sh --loader shim|mkimage` (default shim; `mkimage` = monolithic grub-mkimage loader from the ISO's own GRUB, recipe from the run evidence). RUNBOOK-B legs on nested VM 311 proved it: shim boots+installs under OVMF SB-enforcing + SeaBIOS; mkimage boots+installs under OVMF SB-off; mkimage under SB-enforcing FAILS `Access Denied` (unsigned → **SB must be OFF**, documented); surgery byte-identical payload. **Physical N100 boot on the real board still pending** → folds into the supervised rehearsal (R-1; an `n100-safety` match-nothing ISO is built + sha-recorded for a zero-risk pre-flight). **PXE/network-boot** (the third F1 option, for boards where even USB-mkimage fails) stays a deferred note under this item — not built. **Slice C — SHIPPED (hub v0.62.0 + scripts v1.19.0, 2026-07-17):** the GENERIC secret-free universal ISO (`build-felhom-iso.sh --pairing`). The box self-registers as an unclaimed appliance (keyed by SMBIOS-uuid + MAC set — the DMI-verdict tiebreaker), the operator BINDS it to a customer on the Hosts page, and the hub delivers customer-id + retrieval passphrase ONCE (`/api/v1/appliance/register` + one-shot poll, 404-no-oracle — all live-verified through the public ingress); the bootstrap then falls through to the slice-A direct path. **One unit, two modes** (direct = byte-identical, regression-proven zero-appliance-calls). Artifact proven secret-free (baked env = hub URL only; manifest `secret-bearing: no`). The **bind is operator-password-gated** → the live boot→register→bind→day-0 composition (with a Viktor-created drill customer) + the physical N100 boot fold into the supervised rehearsal — **which now runs the COMPLETE final product flow in one pass: RESET the demo → boot the generic ISO → bind → day-0**. SSH-host-key pinning: stored + fingerprints displayed (attaching to the host on bind = future, no clean hand-off surface today). **Customer-facing self-bind page = R-27 (future).** Origin spike `audits/SPIKE-baremetal-iso-2026-07-16.md`: every mechanism GREEN on nested virt (VM 310 on felhom-pve). Zero-touch install BIOS **and** UEFI incl. **Secure Boot enforcing** (no MOK/keypress); first-boot hook `fully-up` = root + working pvesh/pct, exactly-once via `pending-first-boot-setup` flag; post-install **webhook** carries SMBIOS-UUID + management-MAC + host SSH keys → the unclaimed-appliance record; disk-filter installs only the target (canary byte-identical) and **fails-safe** on match-nothing / bad disk; `from-url` + `cert-fingerprint` **fails CLOSED**. OPEN (needs ONE real bare-metal run): vendor DMI serials (empty on virt), real firmware/NIC quirks, a pre-existing-LVM wipe step. Pipeline notes: assistant pairs to the ISO by Debian codename, ~11.5 s/ISO on DooPlex, **gate on `validate-answer` output not `$?`** (exit 0 on failure). Would flip a new capability-map MISSING row "customer self-installs on bare hardware" once spec'd. *(brief called this R-22)* | **SECOND-HARDWARE PROOF 2026-07-21 (slice C, demo-hp): the pairing flow is no longer a one-board result.** The universal secret-free ISO was booted on a completely different machine — an **HP t740 (Ryzen V1756B, AMI M42 firmware)** versus the N100 it was proven on — and the whole chain ran on **virgin hardware in one pass**: armed install → self-registration as an unclaimed appliance → operator bind → day-0, ending with a running customer guest 9201 and agent 0.92.1 checking in as host `demo-hp-bb76ea`. **Two things generalise from the second board specifically:** the **shim** loader booted with **Secure Boot ENABLED** (`mokutil --sb-state` → `SecureBoot enabled`), confirming the mkimage/SB-off dance is an N100-firmware workaround and NOT a Felhom requirement; and the exact-serial disk filter selected the SanDisk system SSD while leaving the box's **1TB NVMe untouched and unenrolled** (its prior NTFS partition is still intact, unmounted, in no LVM/ZFS) — the destructive path stayed inside its filter on hardware it had never seen. **Not clean, and the failures are filed:** the install got no DHCP on the 4-port NIC and baked a static fallback rather than aborting (**R-59**), which cost a cable move and a hand-repair; and the console was unreachable because the baked root password is unknowable (**R-61**) | R-29 | **The design-v2 green gates are not enforced anywhere — one has been RED for 16 releases.** `controller/scripts/docker_run_volume_path_gate.py` has failed continuously since **2026-07-14 (v0.129.0)** and nobody noticed until R-7b's close-out ran it by hand at v0.145.0. Two separable parts. **(a) The finding itself is benign and the fix is 3 lines.** The flagged call is `internal/appexport/estimate.go:179` `docker run --rm -v :/vol:ro alpine du` — a **NAMED-VOLUME** mount, i.e. daemon-side with no host path, which is the *safe* shape and byte-for-byte the same pattern as three entries already on the gate's ALLOWLIST (`export.go` `volName+":/vol"`, `backup.go` `volName+":/vol:ro"`, `restore.go` `volName+":/vol"`). It is NOT the v0.124.0 path-strand class the gate exists to catch — the author of the v0.129.0 F-A fix explicitly avoided that class (see the function's own comment) and simply never added the allowlist entry. So the fix is an ALLOWLIST addition WITH ITS WHY, **not** a docker-cp rewrite; anyone who 'fixes' this by rewriting the call has misread the gate. **(b) The systemic half is the real item:** the gates run only when a human remembers to run them, so a gate can sit red across 16 releases while every REPORT says 'green'. This is the SECOND instance of the class — cf. the v0.123.0 note *'Windows green gate silently red (read-only fsync)'*. Decide where they run (pre-push hook, `build.sh` step, or a CI job) and make a red gate block the train the way the Go green gate does. | S (a) / M (b) | idea | Origin: R-7b close-out, `felhom-controller` REPORT §4(f) — CC correctly left it alone as out-of-scope and pre-existing, and verified by stashing that it fails identically on the unmodified tree. Flips no capability-map row (engineering hygiene, no customer-visible behaviour). Affected gates to audit for the same rot: controller `template_id_gate` / `emoji_gate` / `native_confirm_gate` / `offbox_rename_gate` / `mojibake_gate` / `app_row_dedup_gate` / `docker_run_volume_path_gate`, hub `hub_confirm_gate`, manifests `manifest_bearer_gate`, website `site_gates`. **Do not bundle (a) into an unrelated feature commit** — it is a one-line behavioural claim about a mount's safety and deserves its own reviewed diff. **2026-07-18 rehearsal note:** the run's finding list independently re-raised "assign the pre-existing `docker_run_volume_path_gate` failure its ID so red stops normalizing" — **that is this item; no second ID was minted.** | diff --git a/documentation/controller/import-and-data-paths.md b/documentation/controller/import-and-data-paths.md new file mode 100644 index 0000000..c825b70 --- /dev/null +++ b/documentation/controller/import-and-data-paths.md @@ -0,0 +1,117 @@ +# Import root & `data_paths` (R-75) + +Shipped in controller **v0.172.0** (2026-07-26). Evidence: +`audits/SPIKE-catalog-data-paths-2026-07-26.md`. + +--- + +## 1. The canonical import root + +``` +${IMPORT_PATH} = /userdata/import + = /mnt/sys_drive/felhom-data/userdata/import (both demo boxes) +``` + +**One drop-zone per box, on the system drive — never per data drive.** It is resolved from +`cfg.Paths.SystemDataPath` via `appbackup.NamespaceRoot(sys, false)` (the system drive holds a +`felhom-data` SUBDIR, so it is the `inGuestDrive=false` case) and exposed as +`stacks.Manager.GetImportRoot()`. + +**Why canonical, not per-drive** — this overrides the spike's Fork-1 recommendation, on a stronger +argument the spike did not weigh. Each drop-zone app has exactly **one** ingest bind. On a two-drive +box a per-drive `import/` would therefore put a folder that *looks* like a drop-zone on every drive +while only one of them does anything; and because `import/*` is `class: excluded`, files stranded in +a dead one are never backed up either. A canonical root is the only shape with no dead drop-zone. + +**No fallback, deliberately.** If the system root is unresolvable, `IMPORT_PATH` is left **unset** and +compose fails loudly on `${IMPORT_PATH}`. It must never fall back to `/userdata/import` — +that silently recreates the shape this design removes. + +**The system drive is NOT a registered `StoragePath`** (verified on both demo boxes, 2026-07-26). +Registering it would make a 50 GB volume holding the recovery units a customer-visible drive, a deploy +target and a wipe candidate. Everything that must reach the import root therefore does so explicitly: + +| Surface | How it reaches the import root | +|---|---| +| Deploy belt | `ensureUserdataMounts` — an import loop that is **not** drive-absent-gated | +| Skeleton | `Manager.EnsureImportRoot()`, not the per-storage-path skeleton loop | +| FileBrowser | an explicit extra bind `:/srv/beolvasas`, outside `buildFileBrowserPaths` | +| SMB share | written directly to the store by `ensureImportShare` | + +That last one matters: `sharingResolvePath` **refuses** the import root, correctly — it validates +paths a *customer* supplied through the picker and requires containment in a registered live root. A +controller-generated constant is a different trust class. (`TestImportRoot_NotReachableViaTheCustomerPicker` +pins both halves.) + +--- + +## 2. The three roles + +`data_paths:` is a **pure annotation** over paths that already exist as compose binds. It can never +declare a path — `ValidateDataPaths` rejects any entry matching no bind. That is the property which +keeps the design cheap: **no new filesystem-write primitive is introduced from catalog data.** + +```yaml +data_paths: + - path: paperless # relative to the bind's root + root: import # import | userdata | hdd + role: import # import | library | export + label: "Beolvasandó dokumentumok" +``` + +| role | meaning | rendered? | +|---|---|---| +| `import` | drop-zone; the app consumes and usually DELETES from it | yes, + free space | +| `library` | where the customer's files live | yes | +| `export` | the app writes results here for the customer | yes | + +`root: hdd` entries are **not** rendered: `appdata/` is app-internal, FileBrowser does not mount it, +and a link would 404. + +### Validation is asymmetric, and that is a decision + +| defect | outcome | precedent | +|---|---|---| +| malformed **path** (absolute, `..`, backslash, non-`Clean`'d, matches no bind, duplicate, unknown root) | **whole-block reject**, one ERROR, no folder card | `backup:` — paths govern DATA HANDLING | +| unknown **role** | **fails OPEN** — that entry is not surfaced, one WARN, the rest renders | `Lifecycle` — roles govern PRESENTATION; a typo must never brick a template | + +The path rules reuse `appbackup.ValidateRelPath`, the single refusal set shared with `backup:`. +**Do not write a second path validator.** + +--- + +## 3. The class-driven copy rule + +The consequence sentence on the app page is driven by the **derived backup class**, never by the role +and never by a per-app string, so the promise the UI makes cannot drift from what the backup engines +actually do: + +| class | Hungarian copy | +|---|---| +| `excluded` + `role: import` | „Ide másold a feldolgozandó fájlokat. Az alkalmazás beolvassa, majd törli innen — ez a mappa átmeneti, és nem készül róla biztonsági mentés." | +| `excluded` (other) | „Ez a mappa átmeneti, és nem készül róla biztonsági mentés." | +| `mandatory` / `optional` | „Itt tárolódnak a fájljaid. Biztonsági mentés készül róla." | +| unclassified | **nothing** — an unverified backup promise is worse than no sentence | + +Both catalog import paths are `class: excluded`, so both must carry the temporary-and-unbacked +warning. `TestConsequenceIsClassDriven` pins it. + +--- + +## 4. Invariants a future change must not break + +1. **Zero removals.** `UserdataSkeletonCarry()` is the v0.171.0 hardcoded list, retained forever, so + derivation can only ADD. `documents` is implied by no catalog app and exists on both demo boxes. +2. **The skeleton set is SORTED.** `fbNeedsRecreate` force-recreates FileBrowser on any byte diff + across ~14 `SyncFileBrowserMounts` call sites; the naive map-order derivation measured **20 + distinct outputs from 20 identical runs**. +3. **The catalog sync is NOT wired to `SyncFileBrowserMounts`.** Connecting them turns a 15-minute + catalog cycle into a 15-minute container-restart cycle. +4. **`url.PathEscape` per path segment for deep links — never `url.QueryEscape`** (it encodes a space + as `+`, which in a path segment is a literal plus). Let `html/template` do the attribute escaping. +5. **Sharing stays opt-in.** The `beolvasas` share is created when sharing is ENABLED, never as a side + effect of deploying an app. +6. **`${IMPORT_PATH}` binds do not resolve against `hddPath`.** Any new consumer of `BindRoot` needs + an explicit `RootImport` case with the import root passed in. +7. **Do not assume an `import/*` directory stays `2775`** — see R-76: FileBrowser creates `0755` + without propagating setgid, and a live `import/calibre` was found at `755`.