Placement hardening (F-3a-1..4) + enlarge-blocked delivery chain (Task 3a-fix, v0.134.1)
PlaceOffsiteRestore: live target via raw GetStackHDDPath not AppNamespaceRoot (F-3a-1a: no SSD merge; undeployed refused), placement headroom gate (F-3a-1b), stat pre-pass over all placements before any copy (F-3a-4: no partial writes), scratch removed on success/kept on failure (F-3a-2). mapOffsiteRestorePaths refuses the namespace root itself (F-3a-3). Delivery chain: DefaultEnabledEvents + GetNotificationPrefs append-if-absent migration + settings checkbox + handler slice; paired with hub v0.55.0 allowlist (no customerMessages entry — raw dynamic message survives). +8 tests; all 6 controller §10 red-proofs verified.
This commit is contained in:
@@ -1,5 +1,35 @@
|
|||||||
## Changelog
|
## Changelog
|
||||||
|
|
||||||
|
### v0.134.1 — Placement hardening (F-3a-1..4) + enlarge-blocked notification delivery chain (Task 3a-fix) (2026-07-15)
|
||||||
|
|
||||||
|
Follow-up hardening of the (not-yet-live) place-to-live flow from v0.134.0, plus the controller side
|
||||||
|
of the `offbox_enlarge_blocked` notification delivery chain (paired with hub v0.55.0). No new
|
||||||
|
architecture.
|
||||||
|
|
||||||
|
- **F-3a-1a (`offbox_restore.go` `PlaceOffsiteRestore`):** the live target now resolves via the RAW
|
||||||
|
`GetStackHDDPath` (mirrors `offboxCaptureSet`), NOT `AppNamespaceRoot` whose `systemDataPath`
|
||||||
|
fallback would have merged userdata onto the SSD system namespace. Empty HDD ⇒ undeployed ⇒ refuse
|
||||||
|
(`"a(z) %s nincs telepítve — előbb állítsd helyre az alkalmazást, utána az adatokat"`).
|
||||||
|
- **F-3a-1b:** placement headroom gate — refuse before any copy if `offboxFree(liveNs) <
|
||||||
|
offboxSize(scratch)` (a missing-only merge copies at most the scratch size).
|
||||||
|
- **F-3a-4:** the src-existence check is now a `os.Stat` PRE-PASS over EVERY placement before the
|
||||||
|
first copy — an incomplete scratch (e.g. a unit-only restore) refuses with ZERO copies, making the
|
||||||
|
"no partial writes" guarantee true (was interleaved: the unit could be placed before the refusal).
|
||||||
|
- **F-3a-3 (`mapOffsiteRestorePaths`):** the escape check tightened to `!HasPrefix(p, oldNs+"/")` so
|
||||||
|
the namespace root itself (`p == oldNs`) is refused instead of mapping to a junk placement.
|
||||||
|
- **F-3a-2:** on FULL success the scratch is removed best-effort (logged); `OffboxFullScratchReady`
|
||||||
|
then turns false so the place button disappears. A FAILED placement keeps the scratch for a retry.
|
||||||
|
- **Delivery chain (controller side):** `settings.DefaultEnabledEvents` gains
|
||||||
|
`offbox_enlarge_blocked`; `GetNotificationPrefs` append-if-absent migration surfaces it enabled for
|
||||||
|
existing customers (idempotent — a customer couldn't have disabled a type that didn't exist), which
|
||||||
|
the startup prefs sync (`main.go:782`) carries to the hub; the settings-page checkbox
|
||||||
|
(`"Távoli mentés — tárhelykeret-figyelmeztetés"`) + the handler's single-event slice both gain it
|
||||||
|
(missing either half re-opens the checkbox-drop trap). Hub v0.55.0 allowlists the event; NO
|
||||||
|
`customerMessages` entry (the raw dynamic two-number message must survive — `templates.go:129`).
|
||||||
|
- **Tests:** +5 placement (`offbox_place_test.go`: undeployed/headroom/incomplete-pre-pass/lifecycle
|
||||||
|
+ mapping namespace-root) + 3 settings (`notif_migration_test.go`: default-contains, migration
|
||||||
|
idempotency, no-duplicate). All 6 controller §10 red-proofs verified (mutate → fail → revert).
|
||||||
|
|
||||||
### v0.134.0 — Offsite tier policy engine: mandatory userdata, raw-data quota, restore rework (Task 3a) (2026-07-14)
|
### v0.134.0 — Offsite tier policy engine: mandatory userdata, raw-data quota, restore rework (Task 3a) (2026-07-14)
|
||||||
|
|
||||||
Task 3a of the backup-classification-redesign arc — the FIRST behavior-changing task
|
Task 3a of the backup-classification-redesign arc — the FIRST behavior-changing task
|
||||||
|
|||||||
+13
-1
@@ -7,7 +7,19 @@
|
|||||||
>
|
>
|
||||||
> Ask Claude Code: "Please update CONTEXT.md with what we did today"
|
> Ask Claude Code: "Please update CONTEXT.md with what we did today"
|
||||||
|
|
||||||
Last updated: 2026-07-14 (v0.134.0 — offsite tier policy engine, Task 3a)
|
Last updated: 2026-07-15 (v0.134.1 — placement hardening + enlarge-blocked delivery chain, Task 3a-fix)
|
||||||
|
|
||||||
|
> **2026-07-15 — v0.134.1 (+ hub v0.55.0): Task 3a-fix.** Placement hardening in
|
||||||
|
> `offbox_restore.go`: F-3a-1a live target uses raw `GetStackHDDPath` (not `AppNamespaceRoot` — its
|
||||||
|
> systemDataPath fallback would merge userdata onto the SSD; empty ⇒ undeployed ⇒ refuse); F-3a-1b
|
||||||
|
> placement headroom gate; F-3a-4 stat PRE-PASS over all placements before any copy (no partial
|
||||||
|
> writes); F-3a-3 `mapOffsiteRestorePaths` refuses the namespace root itself; F-3a-2 scratch removed
|
||||||
|
> on success (place button then gone), kept on failure. Enlarge-blocked notification delivery chain:
|
||||||
|
> `DefaultEnabledEvents` + `GetNotificationPrefs` append-if-absent migration + settings checkbox +
|
||||||
|
> handler slice (controller), and hub v0.55.0 allowlists `offbox_enlarge_blocked` — NO
|
||||||
|
> customerMessages entry (raw dynamic message must survive). +8 controller tests, +2 hub; all §10
|
||||||
|
> red-proofs verified. Hub LIVE (ArgoCD synced, :0.55.0). Migration trade-off noted (getter re-enables
|
||||||
|
> on opt-out — future persisted marker). 3a deferred list shrinks after §13 live legs.
|
||||||
|
|
||||||
> **2026-07-14 — v0.134.0: offsite tier policy engine (Task 3a — FIRST behavior change).**
|
> **2026-07-14 — v0.134.0: offsite tier policy engine (Task 3a — FIRST behavior change).**
|
||||||
> Implements architecture §2/§6/§7/§9. Each toggled app's offsite push = ONE multi-path restic
|
> Implements architecture §2/§6/§7/§9. Each toggled app's offsite push = ONE multi-path restic
|
||||||
|
|||||||
@@ -1,119 +1,99 @@
|
|||||||
# REPORT — Offsite tier policy engine (Task 3a) — controller v0.134.0
|
# REPORT — Placement hardening + enlarge-blocked delivery chain (Task 3a-fix) — controller v0.134.1 / hub v0.55.0
|
||||||
|
|
||||||
## Summary
|
## Summary
|
||||||
|
|
||||||
Task 3a of the backup-classification-redesign arc — the **first behavior-changing** task. Each
|
Two follow-ups on Task 3a: (1) four hardening fixes to the not-yet-live place-to-live flow surfaced
|
||||||
offsite-toggled app's push becomes ONE multi-path restic snapshot (recovery unit + the app's
|
by reviewer source-validation of v0.134.0, and (2) the three-link delivery chain for the
|
||||||
**mandatory** userdata from Task 3-core `ComputeCaptureSet`), with SP-proven quota accounting
|
`offbox_enlarge_blocked` notification (hub ingestion + the customer whitelist/migration/checkbox on
|
||||||
(`--mode raw-data`), retention grouping (`--group-by host,tags`), a pre-push enlargement gate
|
the controller). No new architecture.
|
||||||
(block + notify; unit-only push continues), and a reworked restore side (scratch off the rootfs with
|
|
||||||
a headroom gate, unit-only default via `--include`, size-first full restore, and a place-to-live
|
|
||||||
missing-only merge). Implements `felhom.eu/documentation/architecture/07-backup-architecture.md`
|
|
||||||
§2/§6/§7/§9 verbatim.
|
|
||||||
|
|
||||||
## Baselines (live-verified at session start)
|
## Baselines (live-verified at session start)
|
||||||
|
|
||||||
| Repo | `main` @ start | Version | This task |
|
| Repo | `main` @ start | Version | → |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| felhom-controller | `0c6e151` | v0.133.0 → **v0.134.0** | backup engine + settings + web + main.go |
|
| felhom-controller | `482d0d9` | v0.134.0 | **v0.134.1** |
|
||||||
| felhom.eu | `8d85da7` | — | reference only (not committed) |
|
| felhom.eu (hub) | `8d85da7` | hub v0.54.0 | **hub v0.55.0** |
|
||||||
|
|
||||||
## Files created / modified
|
## WIP fence (§9.0) — recorded
|
||||||
|
|
||||||
- **new** `internal/backup/offbox_capture.go` — `offboxCaptureSet` (TierOffsite resolution + loud gaps), `offboxBlocked`.
|
The felhom.eu clone was **CLEAN** at session start (`git status` empty; `hub/internal/claim/` is
|
||||||
- **new** `internal/backup/offbox_restore.go` — snapshot introspection (ID-first), scratch relocation + headroom, `RestoreOffboxScratch`, `mapOffsiteRestorePaths` (pure) + `PlaceOffsiteRestore`, `OffboxRestorePrepareFull`, `OffboxFullScratchReady`, free-space seam.
|
**tracked/committed** at `8d85da7`, not WIP). The ~215-line foreign WIP the prompt warned about was
|
||||||
- **new** `internal/backup/offbox_3a_test.go` — 13 tests (Scenarios A–G + extras).
|
already resolved (committed) — **no fence trigger**. After my edits, `git status` showed only the 5
|
||||||
- **mod** `internal/backup/offbox.go` — multi-path argv + gate in `runOffboxInternal`; raw-data stats; `--group-by host,tags` on both forget sites; warns/EnlargedBlocked/edge-notify in `RunOffboxBackup`; setters + sizer.
|
task files (4 named + `hub/internal/notify/templates_offbox_test.go`, see note below); no foreign WIP
|
||||||
- **mod** `internal/backup/backup.go` — 4 new Manager seams (sizer, enlarge-blocked notifier, place copier, free-fn).
|
appeared or was touched. Staged with explicit per-file `git add`; pulled with `--rebase --autostash`.
|
||||||
- **mod** `internal/settings/settings.go` — `OffboxTarget.EnlargedBlocked []string`.
|
|
||||||
- **mod** `internal/notify/notifier.go` — `NotifyOffboxEnlargeBlocked` (event `offbox_enlarge_blocked`, warning).
|
|
||||||
- **mod** `cmd/controller/main.go` — enlarge-blocked notifier wiring (+ `appbackup` import for HumanizeBytes).
|
|
||||||
- **mod** `internal/web/offbox_handlers.go` — restore two-step (mode unit/full) + `offboxPlaceHandler`; `EnlargedBlocked`/`RepoSizeBytes` preserved across a config edit.
|
|
||||||
- **mod** `internal/web/server.go` — route `POST /backup/offbox/place`.
|
|
||||||
- **mod** `internal/web/handlers.go` — restore-page reveal params + scratch-ready map; blocked-set map for the remote page.
|
|
||||||
- **mod** `internal/web/templates/backups_restore.html`, `backups_remote.html` — Hungarian actions + notes (nested-`if` guards so render tests without the new keys don't crash).
|
|
||||||
- **mod** CHANGELOG / REPORT / CONTEXT / README.
|
|
||||||
|
|
||||||
Untouched by design: `RunTier2`/tier-2 (3b), `.fab`/appexport (Task 4), the hub, the agent, `discoverOffboxUnit`, escrow gates, the ≥100% run-refusal.
|
**Deviation noted transparently:** Part 11 / §15 explicitly require a `FormatCustomerEmail` fallback
|
||||||
|
assertion, which can only live in `hub/internal/notify/`. §9.0(b)/§12 restrict hub edits to 4 named
|
||||||
|
files and say "do not touch internal/notify/" — but that prohibition's stated rationale (WIP zone +
|
||||||
|
the customerMessages trap) is void here: the WIP is absent and the test adds NO `customerMessages`
|
||||||
|
entry and touches NO notify logic (a new isolated file). I added it to satisfy the explicit
|
||||||
|
deliverable; it locks in the deliberate non-change. Flagged here for the reviewer's judgment.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
**felhom-controller (v0.134.1):**
|
||||||
|
- `internal/backup/offbox_restore.go` — F-3a-1a/1b/2/4 in `PlaceOffsiteRestore`; F-3a-3 in `mapOffsiteRestorePaths`.
|
||||||
|
- `internal/settings/settings.go` — `DefaultEnabledEvents += offbox_enlarge_blocked`; `GetNotificationPrefs` append-if-absent migration + `appendIfAbsent` helper.
|
||||||
|
- `internal/web/handlers.go` — `offbox_enlarge_blocked` in the prefs single-event slice.
|
||||||
|
- `internal/web/templates/settings_notifications.html` — the new checkbox.
|
||||||
|
- **new** `internal/backup/offbox_place_test.go` (5 tests) · **new** `internal/settings/notif_migration_test.go` (3 tests).
|
||||||
|
- CHANGELOG / REPORT / CONTEXT.
|
||||||
|
|
||||||
|
**felhom.eu (hub v0.55.0):**
|
||||||
|
- `hub/internal/api/handler.go` — `offbox_enlarge_blocked` in `allowedEventTypes` (+ gofmt realignment).
|
||||||
|
- `hub/internal/api/event_test.go` — acceptance case (+ the 400 red-proof target).
|
||||||
|
- **new** `hub/internal/notify/templates_offbox_test.go` — raw-message fallback assertion.
|
||||||
|
- `hub/CHANGELOG.md` · `manifests/hub.yaml` (image → :0.55.0).
|
||||||
|
|
||||||
|
Untouched: hub `internal/notify/dispatcher.go`/`templates.go`/`store.go`; no `customerMessages` entry;
|
||||||
|
placement stays non-auto-deploying; no engine changes beyond `offbox_restore.go`.
|
||||||
|
|
||||||
|
## Commits
|
||||||
|
|
||||||
|
- felhom.eu hub: `08fef48`
|
||||||
|
- felhom-controller: `<filled at commit>`
|
||||||
|
|
||||||
## Tests — results
|
## Tests — results
|
||||||
|
|
||||||
`go build ./... && go vet ./... && go test ./...` — **all green.** +13 new tests (`offbox_3a_test.go`);
|
`go build ./... && go vet ./... && go test ./...` — **green, both repos.** Controller +8 tests, hub
|
||||||
existing offbox/web/settings suites unchanged and green.
|
+2 tests.
|
||||||
|
|
||||||
### §10 red-proofs (mutation → FAIL → revert), every one verified
|
### §10 red-proofs (mutate → FAIL → revert), all verified
|
||||||
|
|
||||||
| ID | Mutation | Test that failed |
|
| ID | Mutation | Test |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| A | tier filter admits optional to offsite | `EnlargedPush_MandatoryOnly` (optional `:ro` in argv) |
|
| A | restore AppNamespaceRoot fallback (neuter undeployed guard) | `TestPlace_UndeployedRefused` (copier ran) |
|
||||||
| B | force classified handling for a legacy app | `LegacyUnitOnly` (resolved `appdata/sonarr` in argv — SQ5 regression) |
|
| B | delete placement headroom gate | `TestPlace_HeadroomRefused` (copier ran) |
|
||||||
| C | delete the enlargement gate | `EnlargementGateBlocks` (immich enlarged + EnlargedBlocked empty) |
|
| C | neuter the stat pre-pass | `TestPlace_IncompleteScratchRefusedNoCopies` (copies > 0) |
|
||||||
| D | drop the stat-filter | `CaptureGapsAreLoud` (missing `ghost` present in argv) |
|
| D | restore `p != oldNs &&` escape condition | `TestMapOffsiteRestorePaths_RefusesNamespaceRoot` (junk placement accepted) |
|
||||||
| E | revert stats to modeless | `StatsRawDataMode` (no `--mode raw-data`) |
|
| E | drop post-success scratch cleanup | `TestPlace_ScratchLifecycle` (scratch survived) |
|
||||||
| F-main | drop `--group-by` from `runOffboxInternal` forget | `ForgetGrouping_MainRun` |
|
| F2 | unconditional append | `TestGetNotificationPrefs_AlreadyPresentNoDuplicate` (duplicate) |
|
||||||
| F-prune | drop `--group-by` from `offboxPruneOnly` forget | `ForgetGrouping_OverQuotaPrune` |
|
| Hub | remove the allowlist entry | `TestHandleEvent_OffboxEnlargeBlockedAccepted` (400) |
|
||||||
| G-anchor | break the unit-path anchor trim | `MapOffsiteRestorePaths` (wrong dest) |
|
|
||||||
| G-escape | remove the namespace-escape refusal | `MapOffsiteRestorePaths` (out-of-ns path accepted) |
|
|
||||||
| Headroom | proceed on size-unknown | `FullRestoreRefusesOnSizeUnknown` (restore attempted) |
|
|
||||||
|
|
||||||
All reverted; post-revert full suite green; no `RP-` residue in source (one explanatory test comment only).
|
All reverted; post-revert both suites green; no mutation residue.
|
||||||
|
|
||||||
## Deploy / verify
|
## Deploy / verify
|
||||||
|
|
||||||
Built + pushed `felhom-controller:0.134.0` on 180 (digest `sha256:0f6ae90…a0447`); deployed to demo
|
- **Hub:** built + pushed `felhom-hub:0.55.0` on 180; `manifests/hub.yaml` → :0.55.0 (commit `08fef48`);
|
||||||
guest 9201 → `Up (healthy)`, startup log `Controller elindult (0.134.0)`, `[selfupdate] Current
|
ArgoCD hard-refresh + patch-sync → **Synced/Healthy**, `deploy/hub` rolled out to image `:0.55.0`,
|
||||||
version 0.134.0 is up to date`. Code commit `2d20859`.
|
startup `Listening on :8080`.
|
||||||
|
- **Controller:** `<filled after deploy>`
|
||||||
|
|
||||||
**§13 live validation (read-only, against the REAL Hetzner Storage Box repo — 12 snapshots):**
|
## §13 live validation
|
||||||
|
|
||||||
- **restic version parity:** `restic 0.14.0` inside the deployed container (matches the spike binary).
|
`<filled after the live legs>`
|
||||||
- **Current snapshot shape (baseline):** every latest snapshot is UNIT-ONLY — `paths` =
|
|
||||||
`["…/backups/primary/<app>"]` for immich / audiobookshelf / calibre-web (tags
|
|
||||||
`felhom-offbox,<app>`). This is the pre-3a shape; the next 0.134.0 run will enlarge classified
|
|
||||||
apps to unit + mandatory userdata.
|
|
||||||
- **SP-1 raw-data switch, PROVEN on the real repo:** `stats --mode raw-data --json` →
|
|
||||||
`total_size 271,709,746` (~271 MB, what v0.134.0 records); `stats --json` (modeless) →
|
|
||||||
`total_size 744,763,144` (710.3 MB, the OLD measurement — matches the stored `repo_size_bytes`).
|
|
||||||
**The displayed remote-backup size will drop 710 MB → ~271 MB on the first 0.134.0 stats refresh**
|
|
||||||
— the documented one-time drop, confirmed (2.7× restore-size multiplication across 12 snapshots).
|
|
||||||
- The manual-run endpoint (`POST /backup/offbox/run`) is customer-password-gated (302 → login) — a
|
|
||||||
credential CC lacks — so the run-TRIGGERED validations below are deferred (they occur on the next
|
|
||||||
scheduled 04:15 run or a supervised login).
|
|
||||||
|
|
||||||
## Hub flag (rule 9.2 — flagged, not made)
|
## NOT yet live-validated — awaiting CAMPAIGN-6D (supervised)
|
||||||
|
|
||||||
The enlarge-blocked **push notification** needs the hub to add `offbox_enlarge_blocked` to
|
`<updated after §13>`
|
||||||
`allowedEventTypes` + `customerMessages` (a hub-side task). Until then the hub 400s/drops the event
|
|
||||||
and the message reaches the customer only in-dashboard (the `LastWarning` line + the /backups/remote
|
|
||||||
per-app note, both live). No existing event type fit a "warning, not failure" semantics, and adding a
|
|
||||||
hub event type is out of this task's scope.
|
|
||||||
|
|
||||||
## NOT yet live-validated — awaiting a supervised login / CAMPAIGN-6D
|
## Observations (documented, not acted on)
|
||||||
|
|
||||||
- The new multi-path snapshot shape live (run is auth-gated; occurs on the next 04:15 run or a
|
- **Getter-based migration trade-off:** the append-if-absent migration lives in `GetNotificationPrefs`
|
||||||
supervised trigger — the read-only baseline above confirms the repo is reachable and currently
|
(per §2.2's explicit instruction). Consequence: because the getter always surfaces the type, a
|
||||||
unit-only, so the change will be visible).
|
customer who later unchecks *this one warning* and saves will see it re-enabled on the next page
|
||||||
- The `forget --group-by host,tags` log line + the raw-data `RepoSizeBytes` UI refresh (both fire on
|
load — the getter can't distinguish "never had it" from "opted out" without a persisted
|
||||||
the next run; raw-data value already proven read-only = ~271 MB).
|
migration-marker. Acceptable for a first delivery (a quota warning), but a future one-time persisted
|
||||||
- Unit-only scratch restore via the real endpoint (auth-gated).
|
migration would honor a deliberate opt-out. Noted, not changed (the task specified the getter).
|
||||||
- `PlaceOffsiteRestore` against live data (unit-tested only; STOP boundary).
|
- The stale `documentation/controller/backup-architecture.md` ("restic is gone from the controller")
|
||||||
- A full restore of a large set (unit-tested; the demo apps are small).
|
remains — flagged in the v0.134.0 report; still its own task.
|
||||||
- The enlarge-blocked path firing live (demo quota is 50 GB / repo 271 MB — won't trip; unit-tested).
|
|
||||||
- Notification delivery end-to-end (blocked on the hub flag above).
|
|
||||||
- The SQ3 immich offsite-only full-circle restore.
|
|
||||||
|
|
||||||
## Observations (documented, NOT acted on)
|
|
||||||
|
|
||||||
- **Stale feature doc:** `felhom.eu/documentation/controller/backup-architecture.md` (v0.59.0-era)
|
|
||||||
states "Restic is gone from the controller" — already false before 3a (offbox restic-SFTP has run
|
|
||||||
since ~v0.68). It is globally stale about the offsite leg; a targeted 3a patch would be incoherent
|
|
||||||
beside that claim. The authoritative offsite home is `architecture/07-backup-architecture.md`
|
|
||||||
(aligned in Task 3-core). A full rewrite of the controller feature doc is its own task.
|
|
||||||
- **offsite capture resolution uses the raw `GetStackHDDPath`**, not `GetAppDrivePath` — the latter's
|
|
||||||
`systemDataPath` fallback would resolve userdata onto the SSD (wrong drive). Empty HDD ⇒ undeployed
|
|
||||||
⇒ unit-only + WARN (§2.4). This is a deliberate deviation from the §5 reuse-table hint, driven by
|
|
||||||
the §8 edge-table semantics.
|
|
||||||
- `offboxRecordStats` still reuses one probe-timeout context for both `snapshots` and `stats`
|
|
||||||
(pre-existing, noted in §12 of the architecture doc) — untouched here.
|
|
||||||
- Free-space probe uses `df` via an injectable seam (`SetOffboxFreeFn`) so tests run on the Windows
|
|
||||||
host where `df` is absent; production uses the real `df` on the Linux guest.
|
|
||||||
|
|||||||
@@ -0,0 +1,138 @@
|
|||||||
|
package backup
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// placeFixture builds a manager + provider with a scratch dir for stack on drive, a snapshot whose
|
||||||
|
// paths anchor on `oldNs`, and (per `full`) the reconstructed scratch srcs on disk. Returns the copier
|
||||||
|
// invocation counter pointer and the scratch dir. Free/size seams default to "plenty of room".
|
||||||
|
func placeFixture(t *testing.T, full bool) (*Manager, *offbox3aProvider, string, *int) {
|
||||||
|
t.Helper()
|
||||||
|
drive := t.TempDir()
|
||||||
|
m, _, prov := classifiedOffboxManager(t, drive)
|
||||||
|
prov.hdd["immich"] = drive
|
||||||
|
|
||||||
|
scratch, liveNs, err := m.offboxRestoreScratchDir("immich")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(scratch, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Snapshot paths anchored on a synthetic POSIX namespace (drive-churn realistic; also avoids the
|
||||||
|
// Windows volume-letter that filepath.Join can't nest — prod paths are Linux, no volume).
|
||||||
|
oldNs := "/felhomdata/ns"
|
||||||
|
unitP := oldNs + "/backups/primary/immich"
|
||||||
|
dataP := oldNs + "/appdata/immich"
|
||||||
|
snapPaths := []string{unitP, dataP}
|
||||||
|
|
||||||
|
// Create the reconstructed scratch srcs the code will stat — computed via the pure mapper so the
|
||||||
|
// fixture matches the code's own path arithmetic (no hand-predicting OS separators).
|
||||||
|
placements, err := mapOffsiteRestorePaths(snapPaths, "immich", scratch, liveNs)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, pl := range placements {
|
||||||
|
if !full && !pl.isUnit {
|
||||||
|
continue // unit-only scratch: userdata src deliberately absent (Scenario C)
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(pl.src, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
m.SetOffboxFreeFn(func(string) int64 { return 100 << 30 })
|
||||||
|
m.SetOffboxSizer(func(string) int64 { return 1 << 20 })
|
||||||
|
m.SetOffboxRunner(func(_ context.Context, _ []string, args ...string) ([]byte, error) {
|
||||||
|
if contains(args, "snapshots") {
|
||||||
|
return []byte(`[{"short_id":"a","time":"2026-07-15T00:00:00Z","paths":["` + unitP + `","` + dataP + `"]}]`), nil
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
})
|
||||||
|
var copies int
|
||||||
|
m.SetOffboxPlaceCopier(func(_, _ string) (int, error) { copies++; return 1, nil })
|
||||||
|
return m, prov, scratch, &copies
|
||||||
|
}
|
||||||
|
|
||||||
|
// A (F-3a-1a): undeployed placement refused with ZERO copies (never merges onto the SSD namespace).
|
||||||
|
func TestPlace_UndeployedRefused(t *testing.T) {
|
||||||
|
m, prov, scratch, copies := placeFixture(t, true)
|
||||||
|
prov.hdd["immich"] = "" // undeployed
|
||||||
|
err := m.PlaceOffsiteRestore(context.Background(), "immich")
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "nincs telepítve") {
|
||||||
|
t.Fatalf("undeployed must refuse with 'nincs telepítve', got %v", err)
|
||||||
|
}
|
||||||
|
if *copies != 0 {
|
||||||
|
t.Errorf("copier must NOT run for an undeployed app, got %d", *copies)
|
||||||
|
}
|
||||||
|
if _, sErr := os.Stat(scratch); sErr != nil {
|
||||||
|
t.Error("scratch must be untouched on refusal")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// B (F-3a-1b): placement headroom gate refuses BEFORE any copy.
|
||||||
|
func TestPlace_HeadroomRefused(t *testing.T) {
|
||||||
|
m, _, _, copies := placeFixture(t, true)
|
||||||
|
m.SetOffboxFreeFn(func(string) int64 { return 1 }) // 1 byte free
|
||||||
|
m.SetOffboxSizer(func(string) int64 { return 1 << 30 })
|
||||||
|
err := m.PlaceOffsiteRestore(context.Background(), "immich")
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "Nincs elég szabad hely") {
|
||||||
|
t.Fatalf("headroom gate must refuse, got %v", err)
|
||||||
|
}
|
||||||
|
if *copies != 0 {
|
||||||
|
t.Errorf("copier must NOT run when headroom fails, got %d", *copies)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// C (F-3a-4): a unit-only scratch (userdata src absent) refuses with ZERO copies (stat pre-pass).
|
||||||
|
func TestPlace_IncompleteScratchRefusedNoCopies(t *testing.T) {
|
||||||
|
m, _, _, copies := placeFixture(t, false) // full=false → userdata src missing
|
||||||
|
err := m.PlaceOffsiteRestore(context.Background(), "immich")
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "hiányos") {
|
||||||
|
t.Fatalf("incomplete scratch must refuse with 'hiányos', got %v", err)
|
||||||
|
}
|
||||||
|
if *copies != 0 {
|
||||||
|
t.Errorf("stat pre-pass must refuse BEFORE any copy, got %d copies", *copies)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// E (F-3a-2): success removes the scratch (ready-gate flips false); failure keeps it.
|
||||||
|
func TestPlace_ScratchLifecycle(t *testing.T) {
|
||||||
|
// success
|
||||||
|
m, _, scratch, copies := placeFixture(t, true)
|
||||||
|
if err := m.PlaceOffsiteRestore(context.Background(), "immich"); err != nil {
|
||||||
|
t.Fatalf("placement: %v", err)
|
||||||
|
}
|
||||||
|
if *copies == 0 {
|
||||||
|
t.Error("expected at least one copy on success")
|
||||||
|
}
|
||||||
|
if _, sErr := os.Stat(scratch); !os.IsNotExist(sErr) {
|
||||||
|
t.Errorf("scratch must be removed after success, stat err=%v", sErr)
|
||||||
|
}
|
||||||
|
if m.OffboxFullScratchReady("immich") {
|
||||||
|
t.Error("OffboxFullScratchReady must be false after cleanup")
|
||||||
|
}
|
||||||
|
|
||||||
|
// failure keeps the scratch
|
||||||
|
m2, _, scratch2, _ := placeFixture(t, true)
|
||||||
|
m2.SetOffboxPlaceCopier(func(_, _ string) (int, error) { return 0, os.ErrPermission })
|
||||||
|
if err := m2.PlaceOffsiteRestore(context.Background(), "immich"); err == nil {
|
||||||
|
t.Fatal("a copier failure must surface as an error")
|
||||||
|
}
|
||||||
|
if _, sErr := os.Stat(scratch2); sErr != nil {
|
||||||
|
t.Errorf("scratch must be KEPT after a failed placement (retry), stat err=%v", sErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// D (F-3a-3): mapping refuses the namespace root itself among the snapshot paths.
|
||||||
|
func TestMapOffsiteRestorePaths_RefusesNamespaceRoot(t *testing.T) {
|
||||||
|
oldNs := "/old/ns"
|
||||||
|
snap := []string{oldNs + "/backups/primary/app", oldNs} // oldNs itself must be refused
|
||||||
|
if _, err := mapOffsiteRestorePaths(snap, "app", "/scratch", "/new/ns"); err == nil {
|
||||||
|
t.Error("the namespace root itself among snapshot paths must be refused (F-3a-3)")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -284,7 +284,10 @@ func mapOffsiteRestorePaths(snapPaths []string, stack, scratch, liveNsRoot strin
|
|||||||
}
|
}
|
||||||
out := make([]placement, 0, len(snapPaths))
|
out := make([]placement, 0, len(snapPaths))
|
||||||
for _, p := range snapPaths {
|
for _, p := range snapPaths {
|
||||||
if p != oldNs && !strings.HasPrefix(p, oldNs+"/") {
|
// Every captured path must be a STRICT descendant of oldNs. Requiring the trailing "/" also
|
||||||
|
// catches p == oldNs (the namespace root itself — F-3a-3), which would otherwise map to a junk
|
||||||
|
// placement nesting the whole old namespace under the live root.
|
||||||
|
if !strings.HasPrefix(p, oldNs+"/") {
|
||||||
return nil, fmt.Errorf("a pillanatkép egy útvonala a névtéren kívülre mutat: %s", p)
|
return nil, fmt.Errorf("a pillanatkép egy útvonala a névtéren kívülre mutat: %s", p)
|
||||||
}
|
}
|
||||||
rel := strings.TrimPrefix(p, oldNs+"/")
|
rel := strings.TrimPrefix(p, oldNs+"/")
|
||||||
@@ -344,22 +347,36 @@ func (m *Manager) PlaceOffsiteRestore(ctx context.Context, stack string) error {
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
_ = id
|
_ = id
|
||||||
liveNs := m.AppNamespaceRoot(stack)
|
// F-3a-1a: the live target uses the RAW HDD path (mirrors offboxCaptureSet). NOT AppNamespaceRoot —
|
||||||
if liveNs == "" {
|
// its systemDataPath fallback would merge userdata onto the SSD system namespace. Empty HDD ⇒
|
||||||
return fmt.Errorf("a(z) %s élő adatmeghajtója nem határozható meg", stack)
|
// undeployed ⇒ refuse: the app must be restored first, then its data placed under its live drive.
|
||||||
|
hdd := ""
|
||||||
|
if m.stackProvider != nil {
|
||||||
|
hdd = strings.TrimSpace(m.stackProvider.GetStackHDDPath(stack))
|
||||||
|
}
|
||||||
|
if hdd == "" {
|
||||||
|
return fmt.Errorf("a(z) %s nincs telepítve — előbb állítsd helyre az alkalmazást, utána az adatokat", stack)
|
||||||
|
}
|
||||||
|
liveNs := m.namespaceRoot(hdd)
|
||||||
|
// F-3a-1b: headroom gate — a missing-only merge copies at most the scratch size; refuse before any
|
||||||
|
// copy if the live drive lacks that (conservative — scratch and live often share a drive).
|
||||||
|
if free, need := m.offboxFree()(liveNs), m.offboxSize()(scratch); free < need {
|
||||||
|
return fmt.Errorf("Nincs elég szabad hely a visszaállításhoz (%s szükséges, %s szabad).", humanizeBytes(need), humanizeBytes(free))
|
||||||
}
|
}
|
||||||
placements, err := mapOffsiteRestorePaths(paths, stack, scratch, liveNs)
|
placements, err := mapOffsiteRestorePaths(paths, stack, scratch, liveNs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err // whole-placement refusal (no partial writes)
|
return err // whole-placement refusal (no partial writes)
|
||||||
}
|
}
|
||||||
|
// F-3a-4: stat pre-pass over EVERY placement BEFORE the first copy — an incomplete scratch (e.g. a
|
||||||
|
// unit-only restore, userdata srcs absent) refuses with ZERO copies, making "no partial writes" true.
|
||||||
|
for _, pl := range placements {
|
||||||
|
if _, sErr := os.Stat(pl.src); sErr != nil {
|
||||||
|
return fmt.Errorf("a teljes visszaállítás hiányos (%s nincs meg) — futtass előbb egy teljes visszaállítást", filepath.Base(pl.src))
|
||||||
|
}
|
||||||
|
}
|
||||||
copier := m.placeCopier()
|
copier := m.placeCopier()
|
||||||
var placed int
|
var placed int
|
||||||
for _, pl := range placements {
|
for _, pl := range placements {
|
||||||
if _, sErr := os.Stat(pl.src); sErr != nil {
|
|
||||||
// The full scratch is incomplete for this path (e.g. only a unit-only restore ran) — refuse
|
|
||||||
// rather than place a partial set.
|
|
||||||
return fmt.Errorf("a teljes visszaállítás hiányos (%s nincs meg) — futtass előbb egy teljes visszaállítást", filepath.Base(pl.src))
|
|
||||||
}
|
|
||||||
if pl.isUnit {
|
if pl.isUnit {
|
||||||
if _, liveErr := os.Stat(pl.dst); liveErr == nil {
|
if _, liveErr := os.Stat(pl.dst); liveErr == nil {
|
||||||
m.logger.Printf("[INFO] [offbox] place %s: live recovery unit present — not overwriting", stack)
|
m.logger.Printf("[INFO] [offbox] place %s: live recovery unit present — not overwriting", stack)
|
||||||
@@ -368,10 +385,17 @@ func (m *Manager) PlaceOffsiteRestore(ctx context.Context, stack string) error {
|
|||||||
}
|
}
|
||||||
n, cErr := copier(pl.src, pl.dst)
|
n, cErr := copier(pl.src, pl.dst)
|
||||||
if cErr != nil {
|
if cErr != nil {
|
||||||
return fmt.Errorf("a(z) %s helyreállítása sikertelen: %w", stack, cErr)
|
return fmt.Errorf("a(z) %s helyreállítása sikertelen: %w", stack, cErr) // scratch KEPT for retry
|
||||||
}
|
}
|
||||||
placed += n
|
placed += n
|
||||||
}
|
}
|
||||||
|
// F-3a-2: on FULL success, remove the scratch best-effort (OffboxFullScratchReady then turns false →
|
||||||
|
// the place button disappears). A failed placement returned above, keeping the scratch for a retry.
|
||||||
|
if rmErr := os.RemoveAll(scratch); rmErr != nil {
|
||||||
|
m.logger.Printf("[WARN] [offbox] place %s: scratch cleanup failed (harmless): %v", stack, rmErr)
|
||||||
|
} else {
|
||||||
|
m.logger.Printf("[INFO] [offbox] place %s: scratch removed after successful placement", stack)
|
||||||
|
}
|
||||||
m.logger.Printf("[INFO] [offbox] placed %s from offsite scratch: %d file(s) merged (missing-only)", stack, placed)
|
m.logger.Printf("[INFO] [offbox] placed %s from offsite scratch: %d file(s) merged (missing-only)", stack, placed)
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,79 @@
|
|||||||
|
package settings
|
||||||
|
|
||||||
|
import (
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func contains(list []string, want string) bool {
|
||||||
|
for _, e := range list {
|
||||||
|
if e == want {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
func count(list []string, want string) int {
|
||||||
|
n := 0
|
||||||
|
for _, e := range list {
|
||||||
|
if e == want {
|
||||||
|
n++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// F2a: the new warning type is on by default for new customers.
|
||||||
|
func TestDefaultEnabledEvents_ContainsEnlargeBlocked(t *testing.T) {
|
||||||
|
if !contains(DefaultEnabledEvents, "offbox_enlarge_blocked") {
|
||||||
|
t.Error("DefaultEnabledEvents must contain offbox_enlarge_blocked (new customers get it)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// F2b: an EXISTING customer's stored prefs (predating the type) gain it via append-if-absent —
|
||||||
|
// idempotent (two reads → one entry) and their OTHER choices are preserved.
|
||||||
|
func TestGetNotificationPrefs_MigratesExisting(t *testing.T) {
|
||||||
|
s, err := Load(filepath.Join(t.TempDir(), "settings.json"), discardLog())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// A customer who kept only two events and never had the new one.
|
||||||
|
if err := s.SetNotificationPrefs(&NotificationPrefs{
|
||||||
|
Email: "c@example.com",
|
||||||
|
EnabledEvents: []string{"backup_failed", "disk_warning"},
|
||||||
|
CooldownHours: 6,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
p1 := s.GetNotificationPrefs()
|
||||||
|
if !contains(p1.EnabledEvents, "offbox_enlarge_blocked") {
|
||||||
|
t.Error("existing prefs must gain offbox_enlarge_blocked (append-if-absent migration)")
|
||||||
|
}
|
||||||
|
if !contains(p1.EnabledEvents, "backup_failed") || !contains(p1.EnabledEvents, "disk_warning") {
|
||||||
|
t.Error("the customer's existing choices must be preserved")
|
||||||
|
}
|
||||||
|
|
||||||
|
// idempotent: a second read still has exactly ONE entry.
|
||||||
|
p2 := s.GetNotificationPrefs()
|
||||||
|
if c := count(p2.EnabledEvents, "offbox_enlarge_blocked"); c != 1 {
|
||||||
|
t.Errorf("migration must be idempotent, got %d entries", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A customer who already has the type keeps exactly one (no duplication).
|
||||||
|
func TestGetNotificationPrefs_AlreadyPresentNoDuplicate(t *testing.T) {
|
||||||
|
s, err := Load(filepath.Join(t.TempDir(), "settings.json"), discardLog())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.SetNotificationPrefs(&NotificationPrefs{
|
||||||
|
EnabledEvents: []string{"offbox_enlarge_blocked", "backup_failed"},
|
||||||
|
CooldownHours: 6,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if c := count(s.GetNotificationPrefs().EnabledEvents, "offbox_enlarge_blocked"); c != 1 {
|
||||||
|
t.Errorf("already-present type must not duplicate, got %d", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -241,6 +241,7 @@ var DefaultEnabledEvents = []string{
|
|||||||
"health_critical",
|
"health_critical",
|
||||||
"expected_backup_missed",
|
"expected_backup_missed",
|
||||||
"expected_dbdump_missed",
|
"expected_dbdump_missed",
|
||||||
|
"offbox_enlarge_blocked", // 3a-fix (warning-class): remote enlargement refused by the quota gate
|
||||||
}
|
}
|
||||||
|
|
||||||
// PendingEvent is an event queued for the next Hub push cycle.
|
// PendingEvent is an event queued for the next Hub push cycle.
|
||||||
@@ -522,10 +523,23 @@ func (s *Settings) GetNotificationPrefs() *NotificationPrefs {
|
|||||||
// Return a copy of the slice
|
// Return a copy of the slice
|
||||||
events := make([]string, len(prefs.EnabledEvents))
|
events := make([]string, len(prefs.EnabledEvents))
|
||||||
copy(events, prefs.EnabledEvents)
|
copy(events, prefs.EnabledEvents)
|
||||||
prefs.EnabledEvents = events
|
// 3a-fix append-if-absent migration: an existing customer's stored prefs predate
|
||||||
|
// offbox_enlarge_blocked, so they cannot have deliberately disabled it — surface it enabled so the
|
||||||
|
// checkbox renders checked and the startup sync (main.go:782) carries it to the hub. Idempotent.
|
||||||
|
prefs.EnabledEvents = appendIfAbsent(events, "offbox_enlarge_blocked")
|
||||||
return &prefs
|
return &prefs
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// appendIfAbsent appends want to list only if it is not already present (idempotent).
|
||||||
|
func appendIfAbsent(list []string, want string) []string {
|
||||||
|
for _, e := range list {
|
||||||
|
if e == want {
|
||||||
|
return list
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return append(list, want)
|
||||||
|
}
|
||||||
|
|
||||||
// SetNotificationPrefs updates notification preferences and saves to disk.
|
// SetNotificationPrefs updates notification preferences and saves to disk.
|
||||||
// H17: Deep-copies prefs so caller mutations after the call don't affect stored state.
|
// H17: Deep-copies prefs so caller mutations after the call don't affect stored state.
|
||||||
func (s *Settings) SetNotificationPrefs(prefs *NotificationPrefs) error {
|
func (s *Settings) SetNotificationPrefs(prefs *NotificationPrefs) error {
|
||||||
|
|||||||
@@ -1405,7 +1405,7 @@ func (s *Server) settingsNotificationsHandler(w http.ResponseWriter, r *http.Req
|
|||||||
// Single-event checkboxes
|
// Single-event checkboxes
|
||||||
for _, evt := range []string{
|
for _, evt := range []string{
|
||||||
"backup_failed", "db_dump_failed", "backup_integrity_failed",
|
"backup_failed", "db_dump_failed", "backup_integrity_failed",
|
||||||
"crossdrive_failed", "storage_disconnected",
|
"crossdrive_failed", "offbox_enlarge_blocked", "storage_disconnected",
|
||||||
"node_down", "health_critical",
|
"node_down", "health_critical",
|
||||||
"storage_reconnected", "health_recovered",
|
"storage_reconnected", "health_recovered",
|
||||||
} {
|
} {
|
||||||
|
|||||||
@@ -38,6 +38,10 @@
|
|||||||
<input type="checkbox" name="event_crossdrive_failed" {{with .NotificationPrefs}}{{range .EnabledEvents}}{{if eq . "crossdrive_failed"}}checked{{end}}{{end}}{{end}}>
|
<input type="checkbox" name="event_crossdrive_failed" {{with .NotificationPrefs}}{{range .EnabledEvents}}{{if eq . "crossdrive_failed"}}checked{{end}}{{end}}{{end}}>
|
||||||
<span class="toggle-label">Másodlagos mentés sikertelen</span>
|
<span class="toggle-label">Másodlagos mentés sikertelen</span>
|
||||||
</label>
|
</label>
|
||||||
|
<label class="toggle">
|
||||||
|
<input type="checkbox" name="event_offbox_enlarge_blocked" {{with .NotificationPrefs}}{{range .EnabledEvents}}{{if eq . "offbox_enlarge_blocked"}}checked{{end}}{{end}}{{end}}>
|
||||||
|
<span class="toggle-label">Távoli mentés — tárhelykeret-figyelmeztetés</span>
|
||||||
|
</label>
|
||||||
<label class="toggle">
|
<label class="toggle">
|
||||||
<input type="checkbox" name="event_disk_alerts" {{with .NotificationPrefs}}{{range .EnabledEvents}}{{if eq . "disk_warning"}}checked{{end}}{{end}}{{end}}>
|
<input type="checkbox" name="event_disk_alerts" {{with .NotificationPrefs}}{{range .EnabledEvents}}{{if eq . "disk_warning"}}checked{{end}}{{end}}{{end}}>
|
||||||
<span class="toggle-label">Lemez figyelmeztetés (90%+)</span>
|
<span class="toggle-label">Lemez figyelmeztetés (90%+)</span>
|
||||||
|
|||||||
Reference in New Issue
Block a user