admin/felhom-controller
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: REPORT — v0.132.0 backup classification (INERT) deploy + live sync evidence

Browse Source 
Both guests healthy on 0.132.0; catalog 21e8df1 (13 blocks) synced live
with ZERO backup-block rejections. 13-app cross-check table (every bind
resolves explicit). RP-1..RP-4 confirmed. audiobookshelf PENDING-VETO =
optional pending Viktor's ruling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
This commit is contained in:
authentik Default Admin
2026-07-14 18:54:34 +02:00
parent 0649f9a3e6
commit 95f3180ab4
1 changed files with 100 additions and 99 deletions
Download Patch File Download Diff File Expand all files Collapse all files
REPORT.md
+100 -99
View File
@@ -1,126 +1,127 @@
# REPORT — F-S2 + F-S3: compose-derived appdata dir resolution (controller v0.131.0)
# REPORT — Backup classification: schema + parser + classifier + catalog migration (controller v0.132.0)
## Summary
The controller assumed an app's HDD appdata dir is always `appdata/<stackName>`. paperless-ngx binds
`${HDD_PATH}/appdata/paperless/...` (stack `paperless-ngx`, dir `paperless`), so every consumer that
keyed by stack name silently missed it via a stat-and-skip. One canonical resolver
(`appbackup.AppDataDirNames`) now derives the real dir name(s) from the app's compose `${HDD_PATH}`
binds; all consumers use it. Task 1 of the backup-classification-redesign arc
(`felhom.eu/documentation/audits/SPIKE-backup-classification-2026-07-14.md`), independent of the
classification schema.
Task 2 of the backup-classification-redesign arc
(`felhom.eu/documentation/audits/SPIKE-backup-classification-2026-07-14.md`). Ships the
referential-coupling classification as **DATA + PARSER + PURE CLASSIFIER — deliberately INERT**: no
backup tier changes behavior. The `backup:` block now lands in `.felhom.yml` for all 13 bind-bearing
catalog apps; the controller parses + validates it loudly and a pure classifier resolves per-bind
classes with the SQ5 two-level default. Task 3 (tier policy engine) and Task 4 (manual `.fab` UI)
consume this — nothing does yet except a validation log pass.
## Confirmed baselines
| Repo | `main` @ start | Version | → |
|------|----------------|---------|---|
| felhom-controller | `b42904b` (verified 2026-07-14) | v0.130.0 | **v0.131.0** (commit `68f0e0c`, pushed to `main`) |
| Repo | `main` @ start | → | Commit(s) on `main` |
|------|----------------|---|---------------------|
| felhom-controller | `af98c53` (v0.131.0) | **v0.132.0** | `0649f9a` (code+tests+docs) |
| app-catalog-felhom.eu | `365a017` | (no version scheme) | `21e8df1` (13 blocks + CHANGELOG) |
Trunk-based, direct to `main`. Controller committed + deployed FIRST, then the catalog blocks (so the
parser validates them on first sync).
## Files created / modified
**Part 1 — resolver + helpers**
- `controller/internal/appbackup/paths.go` — NEW pure `AppDataDirNames(hddPath, stackName, hddMounts) []string`
(dedup+sort; fallback `[stackName]`) + `AppDataBindsPresent(hddPath, hddMounts) bool` (WARN predicate);
`AppDataDir` doc updated.
- `controller/internal/backup/tier2.go` — `appDataDirNames` / `tier2AppDataName` (the one place the N>1
refusal error `errTier2MultiDir` is built) / `tier2AppDataBindsPresent`.
- `controller/internal/backup/appbackup_bridge.go` — forwarders for the two new appbackup funcs.
- `controller/internal/stacks/migrate.go` — `ResolveAppDataDirNames` (exported; used by handlers) +
`resolveAppDataDirs` (names + `declared` bool for the WARN); `migSeams.resolveNames` test seam.
**Controller (`0649f9a`)**
- `internal/appbackup/classify.go` (NEW) — schema (`BindClass`, `BindRoot`, `BackupSpec`, `BindSpec`,
`ComposeBind`, `ClassOrigin`, `ClassifiedBind`) + `ValidateBackupSpec` (whole-block-reject) +
`ClassifyBinds` (two-level default; explicit beats `:ro`; nil spec → legacy/false).
- `internal/stacks/classify_binds.go` (NEW) — `ParseComposeClassifiableBinds`: `${VAR}`-relative binds
+ `:ro` flag, `ParseComposeUserdataMounts` scanner shape, dedupe (first-`:ro`-wins).
- `internal/stacks/metadata.go` — `Metadata.Backup` field + the LoadMetadata validation choke point
(bad block → nil + one `[ERROR]` → legacy) + `Manager.ClassifiedBinds`.
- `internal/appbackup/appdata.go` — `StackDataProvider.GetStackClassifiedBinds`.
- `internal/backup/appbackup_bridge.go` — `ClassifiedBind` alias.
- `cmd/controller/main.go` — `stackAdapter.GetStackClassifiedBinds` delegation.
- 5 test fakes (`recovery_unit_test.go`, `tier2_restore_test.go`, `volume_dumps_test.go`,
`api/backup_snapshots_test.go`, `web/async_restore_test.go`) — nil-stub method ONLY (no logic edits).
- Tests: `appbackup/classify_test.go`, `stacks/classify_binds_test.go`, `stacks/classify_wiring_test.go`.
- Docs: `CHANGELOG.md`, `CONTEXT.md`, `REUSE.md`, `controller/README.md`.
**Part 2 — tier-2 backup/info/restore**
- `controller/internal/backup/tier2.go` — `RunTier2` resolves the appdata dir name (L146 site), refuses
N>1 (honest `no_target` status + `[ERROR]`), WARNs on a declared-but-absent dir; new `tier2Mirror`
seam on both rsync legs; rewrote the lying package header (no userdata/no-wholesale truth).
- `controller/internal/backup/backup.go` — `tier2Mirror` seam field.
- `controller/internal/backup/tier2_restore.go` — `RestoreTier2Files` targets the resolved live dir;
`errTier2MultiDirRestore` sentinel, refused BEFORE `StopStack`.
**Part 3 — migrate (F-S3)** — `controller/internal/stacks/migrate.go`: all six per-app appdata legs
(collision `:328`, size `:354`, copy `:449`, verify `:491`, cleanup `:556`, skip-set `:605`) loop the
resolved name(s); copy leg WARNs on a missing declared dir. Merge walk / scope gating / journal untouched.
**Part 4 — display** — `controller/internal/web/handlers.go`: `appDetailsForPath` sums the resolved
dir(s); `dirSizeHuman` split into `dirSizeBytesWalk` + `humanizeDirBytes`.
**Part 5 — truth repairs** — `controller/cmd/controller/main.go:1357` export-adapter comment corrected;
the v0.130.0 CHANGELOG's false "copies the namespace wholesale" sentence is explicitly noted false in the
v0.131.0 entry (old entry left intact as the record).
**Docs** — `CHANGELOG.md` (v0.131.0, newest-on-top, incl. v0.130.0 correction), `REUSE.md`
(AppDataDirNames/BindsPresent + tier2Mirror/resolveNames seams), `CONTEXT.md`, `controller/README.md`.
## Commits on `main`
| Hash | Contents |
|------|----------|
| `68f0e0c` | F-S2 + F-S3 code + all tests + docs (single commit) |
**Catalog (`21e8df1`)** — `backup:` block added after `resources:` in 13 `.felhom.yml` files (immich,
paperless-ngx, nextcloud, calibre-web, audiobookshelf, komga, navidrome, radarr, sonarr, emby,
jellyfin, plex, romm) + `CHANGELOG.md`.
## Tests
`go build ./... && go vet ./... && go test ./...` — **all green** (every package `ok`; vet clean).
**+9 test functions** (before → after in the touched packages):
`go build ./... && go vet ./... && go test ./...` — **all green**, both repos' content parses clean.
**+14 test functions** (before → after in the touched packages; a 9-case validation subtable inside):
- `internal/appbackup/appdatadirnames_test.go` (NEW, +2): `TestAppDataDirNames` (Group A table:
paperless mismatch, matching name, two-name sorted, media+export dedupe, non-appdata + foreign-drive
filtered, whole-root bind ignored, empty→fallback, unclean path), `TestAppDataBindsPresent`.
- `internal/backup/tier2_appdata_test.go` (NEW, +6): `TestRunTier2_PaperlessShape` (Scenario A),
`TestRunTier2_LegacyShape` (Scenario B: match + no-binds fallback), `TestRunTier2_MultiDirRefusal`
(Scenario C, exact Hungarian reason, mirror-count 0), `TestTier2Info_MultiDirRefusal`,
`TestRestoreTier2Files_ResolvedLiveDir` (Scenario E), `TestRestoreTier2Files_MultiDirRefusal`
(refused before StopStack). `t2rFakeProvider` extended with configurable `GetStackHDDMounts`.
- `internal/stacks/migrate_fs3_test.go` (NEW, +1): `TestMigrate_PaperlessShape_ScopeApp` (Scenario D:
copy+verify captured src/dst = appdata/paperless; size/collision/cleanup/skip-set all on the resolved
dir; never appdata/paperless-ngx).
- **Group A (classifier):** `TestClassify_ImmichShape` (explicit beats `:ro`), `_TwoLevelDefault`
(unlisted writable→mandatory, `:ro`→excluded), `_NilSpecLegacy`, `_BareRootFallsToDefault`,
`_SameRelPathBothRoots`.
- **Group B (validation):** `TestValidateBackupSpec_Defects` (9 cases: unknown/empty class, empty/
absolute/`..`/backslash/non-clean path, duplicate, no-matching-bind — each names the entry) + `_ValidAndNil`.
- **Group C (parser):** `TestParseClassifiableBinds_Variants` (ro/rw/ro,z/z, dedupe, bare root, both
roots, non-volume sections), `_CleanAndSubpaths` (false-prefix rejected), `_Missing`.
- **Group D (end-to-end, NO seams):** `TestClassifiedBinds_ValidFixture`, `_RejectedFixture` (RP-4),
`_NoBlockLegacy` — real files through the REAL LoadMetadata path.
- **Group E (catalog fidelity):** `TestCatalogBlocks_Fidelity` (immich, paperless-ngx, sonarr exact class sets).
**Inertness proof (Scenario C):** the entire pre-existing suite is green with **zero test-logic
edits** — only the mandated nil-stub method was added to each fake (verified: `+14 −1` across the 5
fake files, the `−1` being a gofmt re-alignment).
### Red-proofs (run → fail → revert → green) — all confirmed
| RP | Mutation | Failing assertion observed | Reverted |
|----|----------|----------------------------|----------|
| RP-1 | `AppDataDirNames` → `return []string{stackName}` | `TestAppDataDirNames` paperless/two-name/unclean cases FAIL | green |
| RP-2 | RunTier2 L146 → `AppDataDir(nsRoot, stackName)` | `TestRunTier2_PaperlessShape`: appdata leg never mirrored | green |
| RP-3 | restore `liveDir` → stack-name keying | `TestRestoreTier2Files_ResolvedLiveDir`: dst = appdata/app not appdata/paperless | green |
| RP-4 | migCopy site → key by `app` | `TestMigrate_PaperlessShape_ScopeApp`: copy seam never saw appdata/paperless | green |
| RP-5 | delete N>1 guard (`>1` → `>999`) | all three MultiDir refusal tests FAIL (mirror fired, no refusal) | green |
| RP | Mutation | Failing assertion | Reverted |
|----|----------|-------------------|----------|
| RP-1 | disable the class-value check | `TestValidateBackupSpec_Defects/unknown_class` + `/empty_class` accept silently | green |
| RP-2 | `:ro` default overrides explicit | `TestClassify_ImmichShape`: media/photos forced excluded | green |
| RP-3 | flip unlisted-writable default to excluded | `TestClassify_TwoLevelDefault`: unlisted writable ≠ mandatory | green |
| RP-4 | LoadMetadata accepts `Backup` without validating | `TestClassifiedBinds_RejectedFixture`: bad block not degraded to legacy | green |
## Deploy + verify
- Built `v0.131.0` on 180 (`build.sh 0.131.0 --push`, image 145M, digest `sha256:7bba3b54…`).
- Deployed via the bootstrap mechanism to **BOTH guests**:
- **demo 9201** (felhom-pve): `felhom-controller:0.131.0 … Up (healthy)`; logs show a clean status
refresh (9 containers / 56 stacks) + hub report pushed.
- **drill guest 9201** (nested PVE `drill-day0`, 192.168.0.152): `felhom-controller:0.131.0 … Up
(healthy)`; logs clean (scheduler + status refresh).
- Built `v0.132.0` on 180 (`build.sh 0.132.0 --push`, image 145M, digest `sha256:00c6f83e…`).
- Deployed via the bootstrap mechanism to **BOTH guests** — `felhom-controller:0.132.0 … Up (healthy)`
on demo 9201 (felhom-pve) and the drill guest 9201 (nested PVE `drill-day0`, 192.168.0.152).
## NOT live-validated here (deferred to Viktor's supervised session)
## Live sync check (demo guest — the real server-side pipeline)
The functional live legs all require a **deployed paperless-ngx** (the only catalog app with the
appdata mismatch shape). On the current demo, paperless-ngx is **undeployed** — only leftover skeleton
data (`appdata/paperless`, ~16 KB) + a stale recovery unit remain from a prior session; every
currently-deployed HDD app (jellyfin/radarr/navidrome/calibre-web) uses userdata/media binds, not
`appdata/`. Standing up paperless-ngx is a 5-container, Postgres-backed deploy, and the controller's
API (`:8080`) is container-internal (reachable only via traefik + Host header), so a clean unsupervised
deploy→backup→restore→teardown was disproportionate/risky on the shared demo. Deferred, to run through
the real dashboard UI under supervision:
Catalog `21e8df1` pushed AFTER the controller was live, then the controller's `doSync` pipeline
(same code path as the "Sablonok frissítése" button) was triggered by a bootstrap restart:
1. **Tier-2 backup (F-S2 core):** deploy paperless-ngx on felhom-usb, trigger "2. mentés", verify
`…/secondary/paperless-ngx/appdata/` mirrors `appdata/paperless` (count + size).
2. **Marker restore leg:** inject a marker in `appdata/paperless`, re-run tier-2, delete it live, run the
UI file-restore, verify byte-identical return + app healthy.
3. **Storage page:** paperless-ngx shows a non-empty size.
4. **F-S3 migration leg:** scope="app" migration of a real paperless-ngx between drives (supervised —
destructive cleanup step).
- `[sync] Updated <app>/.felhom.yml` for **exactly the 13 apps**, then `[sync] Catalog sync complete`.
- **`backup block rejected` count: 0.** Zero `[ERROR] [stacks]` lines anywhere in the log. Controller
healthy. → **live proof that all 13 committed blocks parse, validate, and match their real composes.**
Confidence for these rests on the exhaustive unit coverage + RP-1..RP-5 above and the confirmed-live
compose shape (paperless-ngx's on-box `docker-compose.yml` binds `${HDD_PATH}/appdata/paperless/media`
and `.../export` — the exact mismatch the fix resolves).
## 13-app cross-check (shipped parser against the real catalog clone — every bind resolved `explicit`)
| App | Resolved class set (root/relpath = class) |
|-----|-------------------------------------------|
| immich | hdd/appdata/immich=mandatory · userdata/media/photos=optional |
| paperless-ngx | hdd/appdata/paperless/media=mandatory · hdd/appdata/paperless/export=excluded · userdata/import/paperless=excluded |
| nextcloud | hdd/appdata/nextcloud=mandatory |
| calibre-web | userdata/media/books=mandatory · userdata/import/calibre=excluded |
| audiobookshelf | userdata/media/audiobooks=optional · userdata/media/podcasts=excluded |
| komga | userdata/media/comics=optional |
| navidrome | userdata/media/music=excluded |
| radarr | userdata/media/movies=excluded · userdata/downloads=excluded |
| sonarr | userdata/media/tv=excluded · userdata/downloads=excluded |
| emby | userdata/media=excluded |
| jellyfin | userdata/media=excluded |
| plex | userdata/media=excluded |
| romm | userdata/roms=optional · hdd/appdata/romm/resources=excluded |
Every bind resolves with origin `explicit` (full coverage — no writable bind left to the mandatory
default). Verified via a throwaway `go run` harness over the shipped `LoadMetadata →
ParseComposeClassifiableBinds → ClassifyBinds`, then removed (not committed).
## PENDING-VETO resolution
**audiobookshelf `media/audiobooks` = `optional`** (spike SQ2 proposed `excluded`). Committed optional
for consistency with komga/romm (curated, often personally-ripped → precious-decoupled). If Viktor
rules `excluded`, flip that ONE line in `templates/audiobookshelf/.felhom.yml`.
## Observations (noticed, NOT acted on)
- **F-S1 (userdata not backed up at any tier)** remains open — deliberately out of scope; owned by the
classification redesign (Task 2/3), which this task's tier-2 header + comments now point to.
- **Multi-dir (N>1) limitation** is a deliberate tier-2 refusal to be lifted by the tier-policy engine
(Task 3, which owns the flat destination layout). No catalog app hits it today.
- Leftover undeployed paperless/immich/nextcloud/romm skeleton dirs + a stale
`felhom-flash/backups/secondary/paperless-ngx` exist on the demo drives (prior-session residue) — not
touched.
- The classification is INERT by contract — offsite/tier-2/`.fab` still capture per today's legacy
behavior. **Task 3** (tier policy engine) is the next step: consume `Manager.ClassifiedBinds` /
`GetStackClassifiedBinds` to scope offsite/tier-2/`.fab` capture by class (mandatory→offsite,
excluded→never, optional→customer-selectable). The seam is wired + tested for it.
- Recovery units already carry `.felhom.yml` and git-sync already whitelists it, so the block reaches
deployed stacks + units with zero plumbing changes; no recovery-unit SchemaVersion bump (Task 3 owns
any content-semantics bump).
- All 13 catalog apps list every classifiable bind explicitly — the two-level *default* branch
(unlisted writable→mandatory) is exercised only by tests today, never by a real catalog app.