diff --git a/REPORT.md b/REPORT.md index beab578..0699732 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,34 +1,283 @@ -# REPORT — v0.171.0: disk-health card device-model label (2026-07-25) +# REPORT — R-75: canonical import root, catalog-derived skeleton, import surfaces (2026-07-26) -**Overwritten** per the standing rule. Pairs with agent v0.95.0 (SMART coverage); the R-50 -island-bridge spike lives in felhom.eu. +**Overwritten** per the standing rule. Controller **v0.172.0**, live on demo-felhom 9201 and +demo-hp 9201. MinAgent unchanged. -## Baseline → target -felhom-controller `de14eed` (v0.170.0) → committed **`f6a8249`** (v0.171.0). Deployed to guest 9201. +--- -## What shipped (additive) -- `agentapi.SmartSummary` gains `ModelName` (mirrors agent v0.95.0 `model_name`, omitempty). -- `diskDisplayLabel` (disk_health.go) now **prefers the device model** over the raw storage name/UUID, - falling back to `Name` (+ speed hint) on an older agent or a modelless disk. Old-agent payloads render - exactly as before. -- Files: `internal/agentapi/client.go`, `internal/web/disk_health.go`, `+_test.go`, CHANGELOG. +## 1. Baselines used -## Tests + red-proof -`go build/vet/test ./...` — **27 packages pass, 0 fail**; template/emoji/mojibake gates OK. -`TestDiskDisplayLabel_PrefersModel`: model present → model label; old agent / nil model → Name(+hint) -(A4 tolerance). Red-proof: drop the fallback (always return ModelName) → the old-agent/nil cases return -"" and the test fails. +| Repo | start | end | +|---|---|---| +| felhom-controller | `3b672ba7` (v0.171.0) | `8fadbd9` (v0.172.0) | +| app-catalog-felhom.eu | `3067a946` | `4252121` | +| felhom.eu | `2d2050c3` | see §10 | +| felhom-agent | `dfd5d731` (v0.96.0) | untouched | -## LIVE acceptance (guest 9201, with agent v0.95.0) — the operator's complaint reversed -Deployed `0.171.0` → Up (healthy). The "Lemezek állapota" card now shows BOTH physical disks with -**real verdicts + human model labels**: -- **"AirDisk 512GB SSD" → Rendben, 34 °C** (the system SSD — Fix A resolved the LVM root). -- **"TOSHIBA MQ04ABF100" → Rendben, 30 °C** (the USB — Fix B + the model label). +--- -No raw UUIDs, no "Nincs adat". `/disks` carries `smart.health=PASSED` + `model_name` for both (see the -agent REPORT). The capability-map row was upgraded to PROVEN-LIVE (healthy path + delivery). +## 2. Part 1.0 probe result — and the STOP it triggered -## Observations -- A live FAILING/Figyelmeztetés verdict still needs a genuinely degrading disk (both demo disks are - healthy) — that arm stays unit + red-proof covered; the notification delivery path is separately - live-proven (hub 400→200 + email, 2026-07-24). +The brief anticipated an ambiguity between `cfg.Paths.SystemDataPath` and the *registered* path. The +probe found something bigger, and I stopped before Part 1.1 as instructed. + +| | demo-felhom 9201 | demo-hp 9201 | +|---|---|---| +| `paths.system_data_path` | `/mnt/sys_drive` | `/mnt/sys_drive` | +| Registered storage paths | `/mnt/felhom-drives/hdd_1` ("USB HDD 1TB", default) | `/mnt/felhom-drives/nvme-1tb` ("NVME 1TB", default); `/mnt/felhom-drives/Felhom-Share` (network) | +| **System drive registered?** | **No** | **No** | + +`/mnt/sys_drive` is a real mountpoint (50 G, 4 % used) holding only +`felhom-data/backups/{primary,secondary}` and `lost+found`. **Neither `/mnt/sys_drive/userdata` nor +`/mnt/sys_drive/felhom-data/userdata` existed** on either box — the userdata convention had never been +exercised on the system drive. + +**Which string is the namespace root:** `/mnt/sys_drive/felhom-data`, per +`appbackup.NamespaceRoot(drivePath, inGuestDrive=false)`, confirmed by the on-disk +`felhom-data/backups/primary`. **But `withUserdataPath` disagrees** — it computes `USERDATA_PATH` as +`/userdata`, not `NamespaceRoot(hdd)/userdata`, so an app on the system drive would get +`/mnt/sys_drive/userdata` while its backups go to `/mnt/sys_drive/felhom-data/backups/`. Latent, and +left untouched; recorded in CONTEXT.md. + +**Why it blocked Part 6** — verified against the real guard with a passing control, not read from +source: + +``` +sysroot NamespaceRoot shape -> REFUSED (Ez a mappa nem osztható meg.) +sysroot withUserdataPath shape -> REFUSED (Ez a mappa nem osztható meg.) +user drive (control) -> ACCEPTED (…/felhom-drives/hdd_1/userdata/import) +``` + +And the obvious fix had a trap: if the system drive *were* registered, `SharingDeniedRoots` denies +`/felhom-data`, so the namespace-consistent shape is refused anyway. + +``` +SharingDeniedRoots("…/sys_drive") = [appdata backups felhom-data felhom-data/appdata felhom-data/backups] + NamespaceRoot shape (felhom-data) -> REFUSED + withUserdataPath shape -> ACCEPTED +``` + +**Operator ruling:** leave the system drive unregistered; the controller writes the `beolvasas` share +directly. `sharingResolvePath` validates CUSTOMER-supplied picker paths — a controller-generated +constant is a different trust class — so **no guard was weakened, and none was bypassed for user +input**. + +--- + +## 3. Exhaustive-switch audit + +| Site | Verdict | Action | +|---|---|---| +| `stacks/classify_binds.go:20` `composeVarRoots` | needs third entry | `{"${IMPORT_PATH}", RootImport}` added | +| `appbackup/classify.go` `ValidateBackupSpec` / `ClassifyBinds` | needs third root | `Import []BindSpec` + both loops extended | +| `appbackup/captureset.go` `resolveAbs` | **BROKEN for import — the sharp one** | fell through to `path.Join(hddPath, relPath)` → a directory on the WRONG DRIVE. Now takes `importRoot`; `RootImport` is its own case | +| `appbackup/captureset.go` `structuralGuard` | needs a case | an unresolvable import root is refused into `Skipped` (`reasonNoImportRoot`) rather than joined onto `""`. The `RootHDD` bare-root/`backups` rules deliberately do NOT transfer — `/userdata/import` nests no backups tree | +| `ComputeCaptureSet` / `ComputeFabBuckets` | needed the root threaded | `importRoot` param added; compile-forced at all 4 call sites (2 backup, 2 appexport) | +| `appexport/fabplan.go:53` | needs a case | import binds fell into `selectedHDD` and would tar from the wrong root. `excluded` makes them default-out, but `OptInExcluded` can pull them in | +| `appexport/fabplan.go:78` `classifiedHDD` | **correctly indifferent** | import binds are not `RootHDD`, so they are not listed | +| `backup/tier2_capture.go` `tier2DestRel` | **indifferent, but only by class** | TierSecondary is mandatory+optional and import is excluded. Left as-is and recorded rather than relied on silently | +| `stacks/samba_classify.go:44` | produces binds, does not switch | `shareRelPath` falls back to path-minus-leading-slash for a share under no registered root. **Known gap, not fixed here** — see §9.2 | +| `appexport/export.go:629` | **indifferent** | compares resolved paths, not the enum | +| `backup/appbackup_bridge.go` | **indifferent** | pure re-export shim | + +`GetImportRoot()` added to both provider interfaces (`appbackup.StackDataProvider`, +`appexport.ExportStackProvider`) and both adapters in `cmd/controller/main.go`. + +--- + +## 4. Commits + +| Repo | Commit | What | +|---|---|---| +| felhom-controller | `2958946` | v0.172.0 main implementation | +| felhom-controller | `4773809` | fixup 1 — `EnsureImportRoot` parent convention | +| felhom-controller | `8fadbd9` | fixup 2 — drop `import/*` from the carry-list | +| app-catalog-felhom.eu | `4252121` | ingest binds → `${IMPORT_PATH}`, `backup:` moved, `data_paths:` added | + +**Push ordering was deliberate and load-bearing:** the controller was built and deployed to BOTH boxes +*before* the catalog was pushed. A catalog carrying `${IMPORT_PATH}` reaches a running v0.171.0 +controller within 15 minutes and would resolve the variable to `""`, binding a bogus root-owned dir at +the container root on the next deploy or restart. + +**Files:** 10 new (`stacks/{datapaths,skeleton_derive}.go`, `web/{datapath_card,filebrowser_link}.go`, +6 test files), 43 modified. + +--- + +## 5. Tests + the three mandatory red-proofs + +**915 → 951 test functions; 27 packages; `go build ./... && go vet ./... && go test ./...` all green.** + +### Red-proof B — classification regression +Pre-fix shape: revert the third-root plumbing in `ValidateBackupSpec`/`ClassifyBinds`. +``` +--- FAIL: TestScenarioB_ImportMoveKeepsClassification (0.00s) + import_root_classify_test.go:107: import/paperless: class = "mandatory", want "excluded" — the moved ingest bind must classify under the import root + import_root_classify_test.go:110: import/paperless: origin = "default_writable", want "explicit" — the moved ingest bind must classify under the import root +``` +The failure is *worse* than "degrades to legacy": the drop-zone silently becomes `mandatory`, i.e. a +transient consume inbox would be shipped OFFSITE. + +### Red-proof C — determinism (the P6 gate) +Pre-fix shape: remove `sort.Strings` from `BuildUserdataSkeleton`. +``` +--- FAIL: TestScenarioC_SkeletonDeterminism (0.00s) + skeleton_determinism_test.go:28: generation 2/20 differs — a non-deterministic skeleton force-recreates FileBrowser on every sync pass + first: [a/b media media/audiobooks media/books media/photos import import/paperless roms a/b/c …] + got: [a/b media media/music media/comics media/photos downloads import/calibre media/tv …] +``` +Failed on generation **2 of 20**. + +### Red-proof E — server-side share refusal +Pre-fix shape: remove BOTH server-side refusals (handler + store), leaving only the template gate. +``` +--- FAIL: TestScenarioE_SystemShareDeleteRefusedServerSide (0.07s) + import_share_test.go:63: the system share was DELETED by a direct POST — the refusal is not server-side +--- FAIL: TestScenarioE_StoreLayerRefusesSystemShare (0.06s) + import_share_test.go:84: RemoveSMBShare must refuse a System share +``` + +**Design gates:** `template_id_gate`, `emoji_gate`, `mojibake_gate`, `native_confirm_gate`, +`app_row_dedup_gate`, `offbox_rename_gate` — all rc=0. +`docker_run_volume_path_gate` **rc=1, PRE-EXISTING** (`internal/appexport/estimate.go:179`, a file +this change does not touch — `git status --porcelain` on it is empty). That is R-29(a), red since +v0.129.0; not fixed here. + +--- + +## 6. Live validation (Part 8) + +Deployed `gitea.dooplex.hu/admin/felhom-controller:0.172.0` — both boxes `Up … (healthy)`. +Method: **endpoint-level** — the exact endpoints the UI invokes, driven with an authed session against +the controller container IP with the `Host:` header. No browser exists on DooPlex. + +**Leg 1 — canonical bind.** Catalog synced through the real button endpoint (`POST /api/sync` → +`updated: [calibre-web, paperless-ngx, romm]`), then `POST /api/stacks/paperless-ngx/update`: +``` +/mnt/sys_drive/felhom-data/userdata/import/paperless -> /usr/src/paperless/consume +drwxrwsr-x 2 1000 1000 /mnt/sys_drive/felhom-data/userdata/import/paperless +``` +**PASS** — 2775, group 1000, on the system drive. + +**Leg 2 — ingest round-trip.** File written into `/srv/beolvasas/paperless` through FileBrowser's own +mount as uid 1000; paperless saw it as `paperless:paperless`, then: +``` +INGESTED+DELETED after ~30s +[paperless.consumer] Consuming r75-teszt.txt +[paperless.consumer] Document 2026-07-26 r75-teszt consumption finished +[paperless.tasks] ConsumeTaskPlugin completed with: Success. New document id 16 created +final drop-zone contents: (empty) +``` +**PASS** — the consume-and-delete contract holds across the new bind. + +**Leg 3 — app-page deep link**, rendered live on demo-hp: +```html +Beolvasandó dokumentumok ↗ +18.5 GB szabad +

Ide másold a feldolgozandó fájlokat. Az alkalmazás beolvassa, majd törli innen + — ez a mappa átmeneti, és nem készül róla biztonsági mentés.

+``` +**PASS.** *Honest scope:* the link was verified as rendered and correctly encoded, **not followed in a +browser** — see §8. + +**Leg 4 — the share.** `POST /sharing/enable` → the `beolvasas` share auto-appeared at +`/mnt/sys_drive/felhom-data/userdata/import`, row marked `rendszer` with no delete button. A direct +`POST /sharing/shares/delete name=beolvasas` returned 303 and the share **survived**: +``` +[('beolvasas', True, '/mnt/sys_drive/felhom-data/userdata/import')] +``` +**PASS.** Note `felhom-samba` is **not running** — sharing is on but no household password is set, so +the stack correctly stays undeployed. Enabling the toggle therefore did **not** put SMB on the LAN. + +**Leg 5 — zero removals, both boxes.** + +| Box | paths removed | paths added | +|---|---|---| +| demo-felhom | **none** | none | +| demo-hp | **none** | `media/podcasts` | + +`documents` intact on both. `media/podcasts` is the one genuinely derived addition (audiobookshelf's +second bind, in no hardcoded list) — and its appearance on demo-hp but not demo-felhom, where it +already existed, is the clean control that the derivation works. System-drive trees on both boxes: +`userdata` and `userdata/import` at `2775 root:1000`, plus `import/paperless` at `2775 1000:1000` on +demo-hp where paperless runs. + +--- + +## 7. Two defects found DURING the live legs, fixed in the same version + +1. **`EnsureImportRoot` left the parent at 755.** `EnsureUserdataDir` MkdirAll's intermediates at plain + `0755` and chmods only the leaf, so `/userdata` came out `755 root:root` — the one userdata + root on the box outside the 2775/gid-1000 convention. Both now carry it + (`TestEnsureImportRoot_ParentCarriesTheConvention`). +2. **The carry-list re-created the dead drop-zone forever.** With `import`, `import/paperless` and + `import/calibre` carried, the derived skeleton would re-create a per-drive drop-zone on every drive + in perpetuity — the exact lookalike this arc removes, and unbacked (`class: excluded`). Dropped from + the carry-list. **Not a zero-removals violation:** nothing deletes what a box already has, and both + boxes' old drop-zones were verified to hold **zero files** first. + `TestSkeletonNeverCreatesAPerDriveDropZone` pins it; `TestUserdataSkeleton_List` was updated to + assert their absence (a deliberate behaviour change, not a weakened assertion). + +**One latent 500 caught before it shipped:** the sharing template's row struct was function-local, so +adding `{{if .System}}` would have failed at render for every share row. `ShareRow` is now +package-level and the render test constructs the handler's own type. + +--- + +## 8. NOT yet live-validated + +- **No browse through the FileBrowser UI.** The source, the bind and the deep-link URL are verified; + nothing drove FileBrowser's HTTP UI or API. The capability-map row *File access via browser* is + therefore **deliberately left at IMPLEMENTED**, with a note. Needs a human click-through. +- **The deep link was not followed.** Rendered and encoding-verified only; the cold-link → login → + redirect path is proven from the shipped router source (spike P2), not exercised live. +- **calibre-web's ingest bind** was not redeployed — calibre-web runs on demo-felhom, and leg 1 + exercised paperless-ngx on demo-hp. The catalog change is identical in shape. +- **SMB access to the `beolvasas` share** — the samba stack is not deployed on demo-hp (no household + password), so no Windows/Explorer leg ran. Config-level only. +- **`data_paths` roles `library` and `export`** were not seen live: paperless declares only `import`. + calibre-web (`library`) and romm (`library`) are unit-tested but not rendered on a box. +- **A multi-drive box.** Both demo boxes have one non-network data drive, so the canonical root's + central benefit — no dead drop-zone on drive #2 — is proven by construction and unit test, not in + the field. +- **The `.fab` export path with an opted-in import bind** — `resolveAbs`'s third case is unit-tested, + never exercised through a real export. +- **A migration off the system drive** (the `appDataSkipSet` exclusion) — unit-tested only. + +--- + +## 9. Observations — noticed, not acted on + +1. **`withUserdataPath` puts userdata outside the namespace on the system drive** (`/userdata` vs + `NamespaceRoot(hdd)/userdata`). Latent — no app with a userdata bind has been deployed there — but + it is a real inconsistency and would surface the day one is. +2. **`samba_classify.shareRelPath` has no case for a share under no registered root.** It falls back to + path-minus-leading-slash, which for the system share yields a bogus `RootHDD` relpath. Only reached + for backup classification of shares, and the share is `Offsite: false`, so it is inert today. Left + alone deliberately (out of scope); worth its own item. +3. **A customer with pending files in an OLD per-drive drop-zone would find them never ingested** after + this upgrade — the bind moved, and nothing migrates the files. Both demo boxes were verified empty + so no action was needed here, but a real box would need an operator-run move. **No destructive or + move operation was added anywhere in this arc.** +4. **`docker_run_volume_path_gate` is still red** (R-29(a), since v0.129.0), in a file this change does + not touch. +5. **R-76 untouched** (FileBrowser's `0755`/no-setgid, and `import/calibre` at 755 on demo-felhom — + still `755 1000:1000` after this deploy, since nothing here chmods it). Nothing built in this arc + assumes an `import/*` directory stays 2775. +6. **Sharing is now ENABLED on demo-hp** (leg 4, explicitly authorised by the brief). No household + password is set, so `felhom-samba` is not deployed and SMB is not on the LAN. Reverting would be a + further unrequested state change, so it was left as-is — flagging for the operator's decision. +7. **I re-hit the accented-Hungarian grep trap myself** while checking leg 3: an accented pattern + through `ssh → pct exec → bash -c` returned nothing and briefly read as "the consequence line is + missing". It was rendering correctly all along. ASCII-only gating patterns, every time. + +--- + +## 10. felhom.eu docs + +- `documentation/controller/import-and-data-paths.md` — NEW: the canonical root, the three roles, the + `data_paths` contract, the class-driven copy rule, and the seven invariants a future change must not + break. +- `documentation/architecture/00-capability-map.md` — *File access via browser* note added, **status + unchanged** (§8). +- `documentation/backlog/ROADMAP.md` — R-75 collapsed to its shipped one-liner; **R-76 left open**.