af98c53c82
Both guests live+healthy on 0.131.0. Live functional legs (paperless deploy → tier-2 backup → marker restore → storage page → F-S3 migration) deferred to Viktor's supervised session — paperless-ngx is undeployed on the demo and the API is container-internal; documented with rationale. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
127 lines
8.0 KiB
Markdown
127 lines
8.0 KiB
Markdown
# REPORT — F-S2 + F-S3: compose-derived appdata dir resolution (controller v0.131.0)
|
|
|
|
## Summary
|
|
|
|
The controller assumed an app's HDD appdata dir is always `appdata/<stackName>`. paperless-ngx binds
|
|
`${HDD_PATH}/appdata/paperless/...` (stack `paperless-ngx`, dir `paperless`), so every consumer that
|
|
keyed by stack name silently missed it via a stat-and-skip. One canonical resolver
|
|
(`appbackup.AppDataDirNames`) now derives the real dir name(s) from the app's compose `${HDD_PATH}`
|
|
binds; all consumers use it. Task 1 of the backup-classification-redesign arc
|
|
(`felhom.eu/documentation/audits/SPIKE-backup-classification-2026-07-14.md`), independent of the
|
|
classification schema.
|
|
|
|
## Confirmed baselines
|
|
|
|
| Repo | `main` @ start | Version | → |
|
|
|------|----------------|---------|---|
|
|
| felhom-controller | `b42904b` (verified 2026-07-14) | v0.130.0 | **v0.131.0** (commit `68f0e0c`, pushed to `main`) |
|
|
|
|
## Files created / modified
|
|
|
|
**Part 1 — resolver + helpers**
|
|
- `controller/internal/appbackup/paths.go` — NEW pure `AppDataDirNames(hddPath, stackName, hddMounts) []string`
|
|
(dedup+sort; fallback `[stackName]`) + `AppDataBindsPresent(hddPath, hddMounts) bool` (WARN predicate);
|
|
`AppDataDir` doc updated.
|
|
- `controller/internal/backup/tier2.go` — `appDataDirNames` / `tier2AppDataName` (the one place the N>1
|
|
refusal error `errTier2MultiDir` is built) / `tier2AppDataBindsPresent`.
|
|
- `controller/internal/backup/appbackup_bridge.go` — forwarders for the two new appbackup funcs.
|
|
- `controller/internal/stacks/migrate.go` — `ResolveAppDataDirNames` (exported; used by handlers) +
|
|
`resolveAppDataDirs` (names + `declared` bool for the WARN); `migSeams.resolveNames` test seam.
|
|
|
|
**Part 2 — tier-2 backup/info/restore**
|
|
- `controller/internal/backup/tier2.go` — `RunTier2` resolves the appdata dir name (L146 site), refuses
|
|
N>1 (honest `no_target` status + `[ERROR]`), WARNs on a declared-but-absent dir; new `tier2Mirror`
|
|
seam on both rsync legs; rewrote the lying package header (no userdata/no-wholesale truth).
|
|
- `controller/internal/backup/backup.go` — `tier2Mirror` seam field.
|
|
- `controller/internal/backup/tier2_restore.go` — `RestoreTier2Files` targets the resolved live dir;
|
|
`errTier2MultiDirRestore` sentinel, refused BEFORE `StopStack`.
|
|
|
|
**Part 3 — migrate (F-S3)** — `controller/internal/stacks/migrate.go`: all six per-app appdata legs
|
|
(collision `:328`, size `:354`, copy `:449`, verify `:491`, cleanup `:556`, skip-set `:605`) loop the
|
|
resolved name(s); copy leg WARNs on a missing declared dir. Merge walk / scope gating / journal untouched.
|
|
|
|
**Part 4 — display** — `controller/internal/web/handlers.go`: `appDetailsForPath` sums the resolved
|
|
dir(s); `dirSizeHuman` split into `dirSizeBytesWalk` + `humanizeDirBytes`.
|
|
|
|
**Part 5 — truth repairs** — `controller/cmd/controller/main.go:1357` export-adapter comment corrected;
|
|
the v0.130.0 CHANGELOG's false "copies the namespace wholesale" sentence is explicitly noted false in the
|
|
v0.131.0 entry (old entry left intact as the record).
|
|
|
|
**Docs** — `CHANGELOG.md` (v0.131.0, newest-on-top, incl. v0.130.0 correction), `REUSE.md`
|
|
(AppDataDirNames/BindsPresent + tier2Mirror/resolveNames seams), `CONTEXT.md`, `controller/README.md`.
|
|
|
|
## Commits on `main`
|
|
|
|
| Hash | Contents |
|
|
|------|----------|
|
|
| `68f0e0c` | F-S2 + F-S3 code + all tests + docs (single commit) |
|
|
|
|
## Tests
|
|
|
|
`go build ./... && go vet ./... && go test ./...` — **all green** (every package `ok`; vet clean).
|
|
**+9 test functions** (before → after in the touched packages):
|
|
|
|
- `internal/appbackup/appdatadirnames_test.go` (NEW, +2): `TestAppDataDirNames` (Group A table:
|
|
paperless mismatch, matching name, two-name sorted, media+export dedupe, non-appdata + foreign-drive
|
|
filtered, whole-root bind ignored, empty→fallback, unclean path), `TestAppDataBindsPresent`.
|
|
- `internal/backup/tier2_appdata_test.go` (NEW, +6): `TestRunTier2_PaperlessShape` (Scenario A),
|
|
`TestRunTier2_LegacyShape` (Scenario B: match + no-binds fallback), `TestRunTier2_MultiDirRefusal`
|
|
(Scenario C, exact Hungarian reason, mirror-count 0), `TestTier2Info_MultiDirRefusal`,
|
|
`TestRestoreTier2Files_ResolvedLiveDir` (Scenario E), `TestRestoreTier2Files_MultiDirRefusal`
|
|
(refused before StopStack). `t2rFakeProvider` extended with configurable `GetStackHDDMounts`.
|
|
- `internal/stacks/migrate_fs3_test.go` (NEW, +1): `TestMigrate_PaperlessShape_ScopeApp` (Scenario D:
|
|
copy+verify captured src/dst = appdata/paperless; size/collision/cleanup/skip-set all on the resolved
|
|
dir; never appdata/paperless-ngx).
|
|
|
|
### Red-proofs (run → fail → revert → green) — all confirmed
|
|
|
|
| RP | Mutation | Failing assertion observed | Reverted |
|
|
|----|----------|----------------------------|----------|
|
|
| RP-1 | `AppDataDirNames` → `return []string{stackName}` | `TestAppDataDirNames` paperless/two-name/unclean cases FAIL | green |
|
|
| RP-2 | RunTier2 L146 → `AppDataDir(nsRoot, stackName)` | `TestRunTier2_PaperlessShape`: appdata leg never mirrored | green |
|
|
| RP-3 | restore `liveDir` → stack-name keying | `TestRestoreTier2Files_ResolvedLiveDir`: dst = appdata/app not appdata/paperless | green |
|
|
| RP-4 | migCopy site → key by `app` | `TestMigrate_PaperlessShape_ScopeApp`: copy seam never saw appdata/paperless | green |
|
|
| RP-5 | delete N>1 guard (`>1` → `>999`) | all three MultiDir refusal tests FAIL (mirror fired, no refusal) | green |
|
|
|
|
## Deploy + verify
|
|
|
|
- Built `v0.131.0` on 180 (`build.sh 0.131.0 --push`, image 145M, digest `sha256:7bba3b54…`).
|
|
- Deployed via the bootstrap mechanism to **BOTH guests**:
|
|
- **demo 9201** (felhom-pve): `felhom-controller:0.131.0 … Up (healthy)`; logs show a clean status
|
|
refresh (9 containers / 56 stacks) + hub report pushed.
|
|
- **drill guest 9201** (nested PVE `drill-day0`, 192.168.0.152): `felhom-controller:0.131.0 … Up
|
|
(healthy)`; logs clean (scheduler + status refresh).
|
|
|
|
## NOT live-validated here (deferred to Viktor's supervised session)
|
|
|
|
The functional live legs all require a **deployed paperless-ngx** (the only catalog app with the
|
|
appdata mismatch shape). On the current demo, paperless-ngx is **undeployed** — only leftover skeleton
|
|
data (`appdata/paperless`, ~16 KB) + a stale recovery unit remain from a prior session; every
|
|
currently-deployed HDD app (jellyfin/radarr/navidrome/calibre-web) uses userdata/media binds, not
|
|
`appdata/`. Standing up paperless-ngx is a 5-container, Postgres-backed deploy, and the controller's
|
|
API (`:8080`) is container-internal (reachable only via traefik + Host header), so a clean unsupervised
|
|
deploy→backup→restore→teardown was disproportionate/risky on the shared demo. Deferred, to run through
|
|
the real dashboard UI under supervision:
|
|
|
|
1. **Tier-2 backup (F-S2 core):** deploy paperless-ngx on felhom-usb, trigger "2. mentés", verify
|
|
`…/secondary/paperless-ngx/appdata/` mirrors `appdata/paperless` (count + size).
|
|
2. **Marker restore leg:** inject a marker in `appdata/paperless`, re-run tier-2, delete it live, run the
|
|
UI file-restore, verify byte-identical return + app healthy.
|
|
3. **Storage page:** paperless-ngx shows a non-empty size.
|
|
4. **F-S3 migration leg:** scope="app" migration of a real paperless-ngx between drives (supervised —
|
|
destructive cleanup step).
|
|
|
|
Confidence for these rests on the exhaustive unit coverage + RP-1..RP-5 above and the confirmed-live
|
|
compose shape (paperless-ngx's on-box `docker-compose.yml` binds `${HDD_PATH}/appdata/paperless/media`
|
|
and `.../export` — the exact mismatch the fix resolves).
|
|
|
|
## Observations (noticed, NOT acted on)
|
|
|
|
- **F-S1 (userdata not backed up at any tier)** remains open — deliberately out of scope; owned by the
|
|
classification redesign (Task 2/3), which this task's tier-2 header + comments now point to.
|
|
- **Multi-dir (N>1) limitation** is a deliberate tier-2 refusal to be lifted by the tier-policy engine
|
|
(Task 3, which owns the flat destination layout). No catalog app hits it today.
|
|
- Leftover undeployed paperless/immich/nextcloud/romm skeleton dirs + a stale
|
|
`felhom-flash/backups/secondary/paperless-ngx` exist on the demo drives (prior-session residue) — not
|
|
touched.
|