REPORT — Offsite tier policy engine (Task 3a) — controller v0.134.0

Summary

Task 3a of the backup-classification-redesign arc — the first behavior-changing task. Each offsite-toggled app's push becomes ONE multi-path restic snapshot (recovery unit + the app's mandatory userdata from Task 3-core ComputeCaptureSet), with SP-proven quota accounting (--mode raw-data), retention grouping (--group-by host,tags), a pre-push enlargement gate (block + notify; unit-only push continues), and a reworked restore side (scratch off the rootfs with a headroom gate, unit-only default via --include, size-first full restore, and a place-to-live missing-only merge). Implements felhom.eu/documentation/architecture/07-backup-architecture.md §2/§6/§7/§9 verbatim.

Baselines (live-verified at session start)

Repo	`main` @ start	Version	This task
felhom-controller	`0c6e151`	v0.133.0 → v0.134.0	backup engine + settings + web + main.go
felhom.eu	`8d85da7`	—	reference only (not committed)

Files created / modified

new internal/backup/offbox_capture.go — offboxCaptureSet (TierOffsite resolution + loud gaps), offboxBlocked.
new internal/backup/offbox_restore.go — snapshot introspection (ID-first), scratch relocation + headroom, RestoreOffboxScratch, mapOffsiteRestorePaths (pure) + PlaceOffsiteRestore, OffboxRestorePrepareFull, OffboxFullScratchReady, free-space seam.
new internal/backup/offbox_3a_test.go — 13 tests (Scenarios A–G + extras).
mod internal/backup/offbox.go — multi-path argv + gate in runOffboxInternal; raw-data stats; --group-by host,tags on both forget sites; warns/EnlargedBlocked/edge-notify in RunOffboxBackup; setters + sizer.
mod internal/backup/backup.go — 4 new Manager seams (sizer, enlarge-blocked notifier, place copier, free-fn).
mod internal/settings/settings.go — OffboxTarget.EnlargedBlocked []string.
mod internal/notify/notifier.go — NotifyOffboxEnlargeBlocked (event offbox_enlarge_blocked, warning).
mod cmd/controller/main.go — enlarge-blocked notifier wiring (+ appbackup import for HumanizeBytes).
mod internal/web/offbox_handlers.go — restore two-step (mode unit/full) + offboxPlaceHandler; EnlargedBlocked/RepoSizeBytes preserved across a config edit.
mod internal/web/server.go — route POST /backup/offbox/place.
mod internal/web/handlers.go — restore-page reveal params + scratch-ready map; blocked-set map for the remote page.
mod internal/web/templates/backups_restore.html, backups_remote.html — Hungarian actions + notes (nested-if guards so render tests without the new keys don't crash).
mod CHANGELOG / REPORT / CONTEXT / README.

Untouched by design: RunTier2/tier-2 (3b), .fab/appexport (Task 4), the hub, the agent, discoverOffboxUnit, escrow gates, the ≥100% run-refusal.

Tests — results

go build ./... && go vet ./... && go test ./... — all green. +13 new tests (offbox_3a_test.go); existing offbox/web/settings suites unchanged and green.

§10 red-proofs (mutation → FAIL → revert), every one verified

ID	Mutation	Test that failed
A	tier filter admits optional to offsite	`EnlargedPush_MandatoryOnly` (optional `:ro` in argv)
B	force classified handling for a legacy app	`LegacyUnitOnly` (resolved `appdata/sonarr` in argv — SQ5 regression)
C	delete the enlargement gate	`EnlargementGateBlocks` (immich enlarged + EnlargedBlocked empty)
D	drop the stat-filter	`CaptureGapsAreLoud` (missing `ghost` present in argv)
E	revert stats to modeless	`StatsRawDataMode` (no `--mode raw-data`)
F-main	drop `--group-by` from `runOffboxInternal` forget	`ForgetGrouping_MainRun`
F-prune	drop `--group-by` from `offboxPruneOnly` forget	`ForgetGrouping_OverQuotaPrune`
G-anchor	break the unit-path anchor trim	`MapOffsiteRestorePaths` (wrong dest)
G-escape	remove the namespace-escape refusal	`MapOffsiteRestorePaths` (out-of-ns path accepted)
Headroom	proceed on size-unknown	`FullRestoreRefusesOnSizeUnknown` (restore attempted)

All reverted; post-revert full suite green; no RP- residue in source (one explanatory test comment only).

Deploy / verify

Built + pushed felhom-controller:0.134.0 on 180 (digest sha256:0f6ae90…a0447); deployed to demo guest 9201 → Up (healthy), startup log Controller elindult (0.134.0), [selfupdate] Current version 0.134.0 is up to date. Code commit 2d20859.

§13 live validation (read-only, against the REAL Hetzner Storage Box repo — 12 snapshots):

restic version parity: restic 0.14.0 inside the deployed container (matches the spike binary).
Current snapshot shape (baseline): every latest snapshot is UNIT-ONLY — paths = ["…/backups/primary/<app>"] for immich / audiobookshelf / calibre-web (tags felhom-offbox,<app>). This is the pre-3a shape; the next 0.134.0 run will enlarge classified apps to unit + mandatory userdata.
SP-1 raw-data switch, PROVEN on the real repo: stats --mode raw-data --json → total_size 271,709,746 (~271 MB, what v0.134.0 records); stats --json (modeless) → total_size 744,763,144 (710.3 MB, the OLD measurement — matches the stored repo_size_bytes). The displayed remote-backup size will drop 710 MB → ~271 MB on the first 0.134.0 stats refresh — the documented one-time drop, confirmed (2.7× restore-size multiplication across 12 snapshots).
The manual-run endpoint (POST /backup/offbox/run) is customer-password-gated (302 → login) — a credential CC lacks — so the run-TRIGGERED validations below are deferred (they occur on the next scheduled 04:15 run or a supervised login).

Hub flag (rule 9.2 — flagged, not made)

The enlarge-blocked push notification needs the hub to add offbox_enlarge_blocked to allowedEventTypes + customerMessages (a hub-side task). Until then the hub 400s/drops the event and the message reaches the customer only in-dashboard (the LastWarning line + the /backups/remote per-app note, both live). No existing event type fit a "warning, not failure" semantics, and adding a hub event type is out of this task's scope.

The new multi-path snapshot shape live (run is auth-gated; occurs on the next 04:15 run or a supervised trigger — the read-only baseline above confirms the repo is reachable and currently unit-only, so the change will be visible).
The forget --group-by host,tags log line + the raw-data RepoSizeBytes UI refresh (both fire on the next run; raw-data value already proven read-only = ~271 MB).
Unit-only scratch restore via the real endpoint (auth-gated).
PlaceOffsiteRestore against live data (unit-tested only; STOP boundary).
A full restore of a large set (unit-tested; the demo apps are small).
The enlarge-blocked path firing live (demo quota is 50 GB / repo 271 MB — won't trip; unit-tested).
Notification delivery end-to-end (blocked on the hub flag above).
The SQ3 immich offsite-only full-circle restore.

Observations (documented, NOT acted on)

Stale feature doc: felhom.eu/documentation/controller/backup-architecture.md (v0.59.0-era) states "Restic is gone from the controller" — already false before 3a (offbox restic-SFTP has run since ~v0.68). It is globally stale about the offsite leg; a targeted 3a patch would be incoherent beside that claim. The authoritative offsite home is architecture/07-backup-architecture.md (aligned in Task 3-core). A full rewrite of the controller feature doc is its own task.
offsite capture resolution uses the raw GetStackHDDPath, not GetAppDrivePath — the latter's systemDataPath fallback would resolve userdata onto the SSD (wrong drive). Empty HDD ⇒ undeployed ⇒ unit-only + WARN (§2.4). This is a deliberate deviation from the §5 reuse-table hint, driven by the §8 edge-table semantics.
offboxRecordStats still reuses one probe-timeout context for both snapshots and stats (pre-existing, noted in §12 of the architecture doc) — untouched here.
Free-space probe uses df via an injectable seam (SetOffboxFreeFn) so tests run on the Windows host where df is absent; production uses the real df on the Linux guest.

8.3 KiB Raw Blame History Unescape Escape