Tier-2 engine rework: class-driven legs, v2 layout, NAS-target exclusion (Task 3b, v0.135.0)
tier2_capture.go: classified apps get TierSecondary per-bind legs (paperless copy shrinks — export
drops); legacy apps keep the byte-identical resolver set. v2 relpath-mirroring layout
(backups/secondary/<stack>/{marker LAST, recovery-unit/, hdd/<rel>/, userdata/<rel>/}); N>1 native
(errTier2MultiDir/tier2AppDataName deleted). Migration=delete-and-rebuild + reconcile; all RemoveAll
via tier2SafeRemove (refuses outside backups/secondary/). SSD=state-only tier. selectTier2Target
never picks network storage (pinned+auto, F-6C-1). Restore reads v2 behind a marker gate.
Part 0: offbox_enlarge_blocked is a persisted one-time Load seed (opt-out sticks), not a getter
append. Part 0.5: offsite restore scratch prefers a local (non-network) path.
Full v2 test suite + all 10 §10 red-proofs verified. Destructive writes bounded to backups/secondary/.
This commit is contained in:
@@ -1,143 +1,75 @@
|
||||
# REPORT — Placement hardening + enlarge-blocked delivery chain (Task 3a-fix) — controller v0.134.1 / hub v0.55.0
|
||||
# REPORT — Tier-2 engine rework (Task 3b) — controller v0.135.0
|
||||
|
||||
## Summary
|
||||
|
||||
Two follow-ups on Task 3a: (1) four hardening fixes to the not-yet-live place-to-live flow surfaced
|
||||
by reviewer source-validation of v0.134.0, and (2) the three-link delivery chain for the
|
||||
`offbox_enlarge_blocked` notification (hub ingestion + the customer whitelist/migration/checkbox on
|
||||
the controller). No new architecture.
|
||||
Task 3b: the tier-2 column of the matrix (architecture §2/§8). The tier-2 appdata leg becomes
|
||||
**class-driven** for classified apps (per-bind mandatory + optional; paperless's copy shrinks as
|
||||
`export` drops out), legacy apps keep a byte-identical resolver set — both mapped into a new **v2
|
||||
relpath-mirroring layout** that lifts the N>1 refusal structurally and gains a reconcile step; the SSD
|
||||
fallback is an enforced **state-only** tier; and `selectTier2Target` gains the F-6C-1 **network-target
|
||||
exclusion** (pinned + auto). Restore reads v2 behind a marker gate. Plus **Part 0** (persisted prefs
|
||||
seed — fixes the 3a-fix un-disableable checkbox) and **Part 0.5** (offsite scratch prefers local).
|
||||
Every destructive write lands only under `backups/secondary/<stack>/` (code-asserted).
|
||||
|
||||
## Baselines (live-verified at session start)
|
||||
|
||||
| Repo | `main` @ start | Version | → |
|
||||
|---|---|---|---|
|
||||
| felhom-controller | `482d0d9` | v0.134.0 | **v0.134.1** |
|
||||
| felhom.eu (hub) | `8d85da7` | hub v0.54.0 | **hub v0.55.0** |
|
||||
|
||||
## WIP fence (§9.0) — recorded
|
||||
|
||||
The felhom.eu clone was **CLEAN** at session start (`git status` empty; `hub/internal/claim/` is
|
||||
**tracked/committed** at `8d85da7`, not WIP). The ~215-line foreign WIP the prompt warned about was
|
||||
already resolved (committed) — **no fence trigger**. After my edits, `git status` showed only the 5
|
||||
task files (4 named + `hub/internal/notify/templates_offbox_test.go`, see note below); no foreign WIP
|
||||
appeared or was touched. Staged with explicit per-file `git add`; pulled with `--rebase --autostash`.
|
||||
|
||||
**Deviation noted transparently:** Part 11 / §15 explicitly require a `FormatCustomerEmail` fallback
|
||||
assertion, which can only live in `hub/internal/notify/`. §9.0(b)/§12 restrict hub edits to 4 named
|
||||
files and say "do not touch internal/notify/" — but that prohibition's stated rationale (WIP zone +
|
||||
the customerMessages trap) is void here: the WIP is absent and the test adds NO `customerMessages`
|
||||
entry and touches NO notify logic (a new isolated file). I added it to satisfy the explicit
|
||||
deliverable; it locks in the deliberate non-change. Flagged here for the reviewer's judgment.
|
||||
| felhom-controller | `1245a6c` | v0.134.1 | **v0.135.0** |
|
||||
|
||||
## Files
|
||||
|
||||
**felhom-controller (v0.134.1):**
|
||||
- `internal/backup/offbox_restore.go` — F-3a-1a/1b/2/4 in `PlaceOffsiteRestore`; F-3a-3 in `mapOffsiteRestorePaths`.
|
||||
- `internal/settings/settings.go` — `DefaultEnabledEvents += offbox_enlarge_blocked`; `GetNotificationPrefs` append-if-absent migration + `appendIfAbsent` helper.
|
||||
- `internal/web/handlers.go` — `offbox_enlarge_blocked` in the prefs single-event slice.
|
||||
- `internal/web/templates/settings_notifications.html` — the new checkbox.
|
||||
- **new** `internal/backup/offbox_place_test.go` (5 tests) · **new** `internal/settings/notif_migration_test.go` (3 tests).
|
||||
- CHANGELOG / REPORT / CONTEXT.
|
||||
- **new** `internal/backup/tier2_capture.go` — `tier2CaptureSet` (TierSecondary legs + gaps), `tier2Leg`, `tier2DestRel`.
|
||||
- **mod** `internal/backup/tier2.go` — `selectTier2Target` (network exclusion, two sizes, `StateOnly`), `RunTier2` v2 body (migration/mirror/reconcile/marker-last), `tier2SafeRemove` + `tier2Reconcile` + `classifyTier2Rel`, `Tier2Info` rewired, header + `RunTier2` comments truthful; deleted `errTier2MultiDir`/`tier2AppDataName`.
|
||||
- **mod** `internal/backup/tier2_restore.go` — v2 marker gate + two-subtree missing-only merge; deleted `errTier2MultiDirRestore`.
|
||||
- **mod** `internal/backup/offbox_restore.go` — Part 0.5 scratch local-preference.
|
||||
- **mod** `internal/backup/backup.go` — `tier2SSDFits` seam.
|
||||
- **mod** `internal/backup/appbackup_bridge.go` — `ClassMandatory/Optional/Excluded` aliases.
|
||||
- **mod** `internal/settings/settings.go` — Part 0 seed (`OffboxEnlargeNoticeSeeded` + `seedOffboxEnlargeNotice` at Load; getter append removed); `CrossDriveBackup.LastWarning`.
|
||||
- **mod** `internal/web/handlers.go` + `templates/backups_apps.html` — surface `Tier2LastWarning`.
|
||||
- **tests** new `tier2_v2_test.go`, `offbox_scratch_test.go`; rewrote `notif_migration_test.go` (Part 0), `tier2_restore_test.go` fixture (v2), `async_restore_test.go` fixture (v2), `tier2_test.go` signatures; **deleted** obsolete `tier2_appdata_test.go`.
|
||||
- CHANGELOG / REPORT / CONTEXT / README.
|
||||
|
||||
**felhom.eu (hub v0.55.0):**
|
||||
- `hub/internal/api/handler.go` — `offbox_enlarge_blocked` in `allowedEventTypes` (+ gofmt realignment).
|
||||
- `hub/internal/api/event_test.go` — acceptance case (+ the 400 red-proof target).
|
||||
- **new** `hub/internal/notify/templates_offbox_test.go` — raw-message fallback assertion.
|
||||
- `hub/CHANGELOG.md` · `manifests/hub.yaml` (image → :0.55.0).
|
||||
|
||||
Untouched: hub `internal/notify/dispatcher.go`/`templates.go`/`store.go`; no `customerMessages` entry;
|
||||
placement stays non-auto-deploying; no engine changes beyond `offbox_restore.go`.
|
||||
|
||||
## Commits
|
||||
|
||||
- felhom.eu hub: `08fef48`
|
||||
- felhom-controller: `0cfcc42`
|
||||
Untouched: offbox engine (except `offboxRestoreScratchDir`), appexport, hub, agent; `rsyncMirror`/`rsyncRestoreMissing` unchanged.
|
||||
|
||||
## Tests — results
|
||||
|
||||
`go build ./... && go vet ./... && go test ./...` — **green, both repos.** Controller +8 tests, hub
|
||||
+2 tests.
|
||||
`go build ./... && go vet ./... && go test ./...` — **green.** New: v2 suite (§7 A–H + reconcile
|
||||
keep/remove + safe-remove boundary proof + `classifyTier2Rel` table), Part 0 (idempotent seed +
|
||||
opt-out-sticks + fresh-customer), Part 0.5 (local-preference + network-only WARN).
|
||||
|
||||
### §10 red-proofs (mutate → FAIL → revert), all verified
|
||||
|
||||
| ID | Mutation | Test |
|
||||
|---|---|---|
|
||||
| A | restore AppNamespaceRoot fallback (neuter undeployed guard) | `TestPlace_UndeployedRefused` (copier ran) |
|
||||
| B | delete placement headroom gate | `TestPlace_HeadroomRefused` (copier ran) |
|
||||
| C | neuter the stat pre-pass | `TestPlace_IncompleteScratchRefusedNoCopies` (copies > 0) |
|
||||
| D | restore `p != oldNs &&` escape condition | `TestMapOffsiteRestorePaths_RefusesNamespaceRoot` (junk placement accepted) |
|
||||
| E | drop post-success scratch cleanup | `TestPlace_ScratchLifecycle` (scratch survived) |
|
||||
| F2 | unconditional append | `TestGetNotificationPrefs_AlreadyPresentNoDuplicate` (duplicate) |
|
||||
| Hub | remove the allowlist entry | `TestHandleEvent_OffboxEnlargeBlockedAccepted` (400) |
|
||||
| A | classified app routed to the legacy resolver | `ClassifiedPaperless` (`export` appears in dest) |
|
||||
| B | legacy routed through ComputeCaptureSet | `LegacyMultiDir` (empty legs) |
|
||||
| C | marker written before the legs | `MigrationAndMarkerLast` (marker present after a leg failure) |
|
||||
| D-reconcile | disable stale detection | `Reconcile` (stale dir survives) |
|
||||
| D-boundary | neuter the safe-remove prefix assert | `Reconcile` (removal outside destBase not refused) |
|
||||
| E | SSD copies optional (no state-only filter) | `SSDStateOnly` (optional on the SSD) |
|
||||
| F1 | drop the pinned IsNetwork check | `NetworkExclusion/pinned` (NAS pin honored) |
|
||||
| F2 | drop the auto IsNetwork check (NFS iterated first) | `NetworkExclusion/auto` (NAS selected) |
|
||||
| G2 | neuter the restore marker gate | `RestoreRefusesOldLayout` (merge runs on old layout) |
|
||||
| Part 0 | don't persist the seed marker | `SeedOffboxEnlargeNotice_OptOutSticks` (type re-enabled) |
|
||||
|
||||
All reverted; post-revert both suites green; no mutation residue.
|
||||
All reverted; post-revert full suite green; no residue.
|
||||
|
||||
## Deploy / verify
|
||||
|
||||
- **Hub:** built + pushed `felhom-hub:0.55.0` on 180; `manifests/hub.yaml` → :0.55.0 (commit `08fef48`);
|
||||
ArgoCD hard-refresh + patch-sync → **Synced/Healthy**, `deploy/hub` rolled out to image `:0.55.0`,
|
||||
startup `Listening on :8080`.
|
||||
- **Controller:** built + pushed `felhom-controller:0.134.1` on 180; deployed to guest 9201 →
|
||||
`Up (healthy)`. Commit `0cfcc42`.
|
||||
`<filled after deploy>`
|
||||
|
||||
## §13 live validation — ALL LEGS PASS
|
||||
## §13 live validation
|
||||
|
||||
The dashboard session was established with `180:~/.config/credentials` `C4_PASSWORD` (the value is
|
||||
**single-quoted** in the file — stripping the quotes is required; an earlier run that kept the quotes
|
||||
produced a false `Hibás jelszó`, since corrected). Login → 302 + `felhom_session` cookie; the flow ran
|
||||
entirely on 180 (LAN → dashboard) so the password was never transferred or echoed.
|
||||
`<filled after the live legs>`
|
||||
|
||||
- **Leg 2 — manual run (LIVE):** POST `/backup/offbox/run` → 302; run log:
|
||||
`backed up calibre-web (…, 1 mandatory path(s))` (enlarged shape), `audiobookshelf`/`immich`
|
||||
`not deployed — offsite push is unit-only` (the §2.4 undeployed WARN firing live), `backup OK: 3
|
||||
app(s) backed up, 15 snapshot(s), 49s`. **Leg 2a** confirmed by `snapshots --no-lock --json`:
|
||||
calibre-web newest = `[…/backups/primary/calibre-web, …/userdata/media/books]` (unit + mandatory).
|
||||
- **Leg 2b — raw-data quota (LIVE):** stored `repo_size_bytes = 280,932,890` (267.9 MB) == live
|
||||
`stats --mode raw-data` — down from 744 MB modeless pre-3a.
|
||||
- **Leg 2c — forget grouping:** both call sites carry `--group-by host,tags` (deployed + unit-tested);
|
||||
a successful forget is logged SILENTLY by design (only failures log), so no live line — the ~40s gap
|
||||
before "backup OK" + the stable 15-snapshot retention evidence it ran.
|
||||
- **Leg 3 — unit-only scratch restore (LIVE):** POST `mode=unit` → scratch landed on the DATA DRIVE
|
||||
`/mnt/felhom-drives/nas-media/backups/offsite-restore/calibre-web` (NOT the rootfs — F-A1); the
|
||||
reconstructed tree contains ONLY `…/backups/primary/calibre-web/{compose,volume-dumps}` (unit-only
|
||||
scope, `--include` working); rootfs `DataDir/offbox-restore` absent.
|
||||
- **Leg 4 — full restore two-step (LIVE):** step 1 `mode=full` → 302 to
|
||||
`/backups/restore?full_prep=calibre-web&full_size=258.3+MB` (size shown BEFORE start); step 2
|
||||
`mode=full&confirm=1` → the scratch now also holds `…/userdata/media/books/.calnotes/…` (whole
|
||||
snapshot, SP-3.1 abs-path reconstruction).
|
||||
- **Leg 5 — prefs round-trip (LIVE):** `/settings/notifications` renders `event_offbox_enlarge_blocked`
|
||||
**checked** (migration surfaced it enabled); a save round-trip → the type **survived** (F3
|
||||
checkbox-drop trap avoided); the hub logged `Notification preferences updated for demo-felhom:
|
||||
events=[… offbox_enlarge_blocked …]` — the migrated type reached the hub whitelist end-to-end.
|
||||
- **Leg 6 — delivery (LIVE, one synthetic event):** pushed `offbox_enlarge_blocked`/warning through the
|
||||
real controller→hub path → **hub_status 200** (was 400 before v0.55.0); hub logged
|
||||
`Event from demo-felhom: offbox_enlarge_blocked (warning)` + `Operator email sent for
|
||||
demo-felhom/offbox_enlarge_blocked`. The message was the debug endpoint's generic text (it fixes the
|
||||
body); the dynamic two-number message survival is unit-proven (`templates_offbox_test.go`) — no
|
||||
`customerMessages` entry, so `FormatCustomerEmail` falls back to the raw message.
|
||||
- **Leg 7 — hygiene:** the calibre-web scratch (legs 3–4) removed; 180 session temp files removed;
|
||||
read-only inspection scripts removed from the container. `PlaceOffsiteRestore` was NOT run (6D).
|
||||
## 6D-pending
|
||||
|
||||
## NOT yet live-validated — awaiting CAMPAIGN-6D (supervised)
|
||||
`<updated after §13>`
|
||||
|
||||
- `PlaceOffsiteRestore` against live data (the STOP boundary — merges into live app data).
|
||||
- Organic enlarge-block firing (demo repo 268 MB / quota 50 GB won't trip; unit-tested + the delivery
|
||||
chain proven live via the synthetic event).
|
||||
- The SQ3 immich offsite-only full-circle restore-and-boot.
|
||||
## Observations
|
||||
|
||||
## Observations (documented, not acted on)
|
||||
|
||||
- **Getter-based migration trade-off:** the append-if-absent migration lives in `GetNotificationPrefs`
|
||||
(per §2.2's explicit instruction). Consequence: because the getter always surfaces the type, a
|
||||
customer who later unchecks *this one warning* and saves will see it re-enabled on the next page
|
||||
load — the getter can't distinguish "never had it" from "opted out" without a persisted
|
||||
migration-marker. Acceptable for a first delivery (a quota warning), but a future one-time persisted
|
||||
migration would honor a deliberate opt-out. Noted, not changed (the task specified the getter).
|
||||
- The stale `documentation/controller/backup-architecture.md` ("restic is gone from the controller")
|
||||
remains — flagged in the v0.134.0 report; still its own task.
|
||||
- **Demo-state touches during §13 (benign, noted):** the leg-5 prefs save persisted the demo
|
||||
customer's `enabled_events` to explicitly include `offbox_enlarge_blocked` (previously getter-migrated
|
||||
— idempotent, no behavior change); leg 6 sent one operator test email. Legs 3–4 scratch dirs were
|
||||
removed (leg 7). No live app data was modified (`PlaceOffsiteRestore` not run).
|
||||
- **Credential note:** `C4_PASSWORD` in `180:~/.config/credentials` is **single-quoted** — strip the
|
||||
quotes before use (an unstripped value yields a false `Hibás jelszó`). The credential is valid.
|
||||
- The `Tier2LastWarning` surfacing is minimal (one line on the app's tier-2 row); the SSD state-only
|
||||
hint was already rendered via `EffectiveIsSSD`/`EffectiveDesc` (no new template needed there).
|
||||
- `tier2SSDFits` seam added because `system.GetDiskUsage` is Linux-only (nil on the Windows test host)
|
||||
— production uses the real predicate.
|
||||
|
||||
Reference in New Issue
Block a user