docs: v0.124.0 backups IA restructure — CHANGELOG + REPORT + CONTEXT (decisions 1-3) + README page map + REUSE; records the HIGH containerized-.fab volume-strand finding and the agent SystemDisks legacy-boot finding

Claude-Session: https://claude.ai/code/session_01GzammAMzsJTgpQHqxwM2bC
2026-07-13 10:17:58 +02:00
parent 753cd83456
commit 811a0ef7a7
5 changed files with 151 additions and 57 deletions
@@ -1,5 +1,45 @@
 ## Changelog

+### v0.124.0 — backups IA restructure: four sub-pages, Felhom-offsite status card, .fab browser download (2026-07-13) — MinAgent: 0.81.0
+
+The nine-section backups page split into four sub-pages (operator review: customers got lost);
+plus the offsite status card and the .fab portability exit. Controller-only; floor untouched.
+Operator decisions recorded in CONTEXT: single active offsite destination stands; the status
+card never changes state; .fab is portability, NOT a backup tier.
+
+- **IA split** (`backups{,_remote,_apps,_restore}.html` + `backups_shared.html` partials):
+  **Áttekintés** `/backups` (storage overview, Rendszermentés, stat cards, single-copy warning),
+  **Távoli mentés** `/backups/remote` (status card + toggles + manual-target form,
+  `#offbox-section` anchor), **Alkalmazások** `/backups/apps` (schedule, Adatbázisok, per-app
+  1./2./3. rows), **Visszaállítás** `/backups/restore` (restore panel, the RELOCATED offbox
+  restore-to-verify, the .fab loop). Sections MOVED verbatim — `scripts/backups_split_move_check.py`
+  compares all 15 blocks against the v0.123.0 baseline (red-proven). Shared data builder
+  extracted (`backupsCommonData` + `backupsOffboxData`) — no duplicated computation. Old links
+  survive: `/backups` = Áttekintés; tier-3 row actions → `/backups/remote#offbox-section`;
+  offbox/restore/tier2 flash redirects + the tier2-config back-link retargeted per page.
+- **Felhom-offsite status card** (top of Távoli mentés; local data only, DISPLAY-ONLY — no form,
+  no button, unit-enforced): (1) no target → "Felhom offsite tárhely — igényelhető szolgáltatás.
+  …Érdeklődj az üzemeltetőnél."; (2) applied + zero toggled → "Aktív — nincs kijelölt
+  alkalmazás" (+ the v0.123.0 hint below); (3) applied + toggled → no card (the status block is
+  the state). States 2 and 3 live-proven on the drill box.
+- **.fab browser download** (`handler_export_download.go`): the EXISTING async export pipeline
+  with dest = a staging dir under the data dir (`fab-downloads/`; same producer → byte-identical
+  bundle), estimate shown BEFORE start, then a guarded streaming exit
+  (`GET /api/export/download?file=` — basename-shape + dir-containment guard, red-proven against
+  prefix-only matching; `io.Copy`, `Content-Disposition: attachment`, post-stream removal, 1h TTL
+  sweep on startup + each start). Batch UI downloads apps ONE AT A TIME (no mega-zip). Import
+  stays drive-scan; portability copy on the section ("Hordozható pillanatfelvétel…").
+  Unit round-trip: real export → import → content equality; a corrupted bundle is REFUSED
+  (gzip CRC — there is no per-file checksum; documented).
+- **FINDINGS from the live §13 run** (both pre-existing, recorded for follow-up tasks):
+  **(HIGH)** containerized-controller .fab export of Docker-VOLUME apps strands the volume tar on
+  the GUEST host (`docker run -v <container-tmp>:/out` resolves against the host FS) → the bundle
+  ships an EMPTY `data/volumes`, export reports success, import brings the app up EMPTY. HDD-data
+  apps unaffected. Live-hit on demo (ActualBudget; data restored from the stranded tar by hand).
+  **(MEDIUM)** agent-side: on a legacy-boot PVE with LVM root, `SystemDisks` resolves NO raw
+  system disk → `sysKnown=false` → every disk classified system → the drive wizard can never
+  offer a candidate (drill box; hot-added disk invisible).
+
 ### v0.123.0 — polish batch: F-15 instant reset codes, F-11 inline confirms, Tier-3 "Távoli mentés" rename, zero-toggle honesty (2026-07-13) — MinAgent: 0.81.0

 Four independent fixes from the take-two drill + operator review. Live-validated on the drill box
@@ -7,7 +7,22 @@
 >
 > Ask Claude Code: "Please update CONTEXT.md with what we did today"

-Last updated: 2026-07-13 (v0.123.0 — polish batch)
+Last updated: 2026-07-13 (v0.124.0 — backups IA restructure)
+
+> **2026-07-13 — v0.124.0: BACKUPS IA RESTRUCTURE. MinAgent 0.81.0 + floor unchanged.
+> Operator decisions (2026-07-13, treat as settled):** (1) single active offsite destination per
+> box STANDS — the dual-destination `managed_by` model is the separate queued Task B;
+> (2) the Felhom-offsite status card NEVER changes anything — display + opt-in pointers only;
+> (3) .fab export/download is PORTABILITY, not a backup tier — no scheduling, no status surface,
+> point-in-time framing. Mechanics: four sub-pages (`/backups{,/remote,/apps,/restore}`, sections
+> moved VERBATIM — `scripts/backups_split_move_check.py` gates vs df7ad37), status card (3 states,
+> display-only), .fab download exit (existing exporter + staging dir + guarded stream + 1h TTL;
+> traversal guard red-proven). Live-validated on drill+demo incl. a supervised import round-trip.
+> **NEW FINDINGS:** **(HIGH)** containerized .fab export strands Docker-VOLUME tars on the guest
+> host (`docker run -v <container-tmp>` → host path) — bundle ships empty volumes, import brings
+> the app up EMPTY; fix = host-visible staging + fail-loud post-export assertion. **(MEDIUM,
+> agent)** legacy-boot PVE (LVM root, no ESP mount) → SystemDisks empty → sysKnown=false → drive
+> wizard offers ZERO candidates ever. Follow-ups: .fab browser-upload; mega-zip parked.

 > **2026-07-13 — v0.123.0: POLISH BATCH (take-two F-15/F-11 + rename + zero-toggle). MinAgent
 > 0.81.0 unchanged; floor 0.122 unchanged. Requires hub v0.52.0 for F-15 (old hub = clean no-op).**
@@ -1,64 +1,86 @@
-# REPORT — v0.123.0: polish batch (F-15 instant reset codes, F-11 inline confirms, "Távoli mentés" rename, zero-toggle honesty)
+# REPORT — v0.124.0: backups IA restructure (four sub-pages, Felhom-offsite status card, .fab browser download)

-**Date:** 2026-07-13 · **Version:** controller v0.123.0 (from v0.122.0) · **MinAgent:** 0.81.0 (UNCHANGED) · **Companion:** hub v0.52.0 (F-15 response; old hub = clean no-op) · **Floor:** 0.122 UNCHANGED (no raise in this task)
+**Date:** 2026-07-13 · **Version:** controller v0.124.0 (from v0.123.0) · **MinAgent:** 0.81.0 (UNCHANGED) · **Floor:** untouched (rides the next train) · Controller-only.

-## What shipped (commits `b4e0a19`, `40b5304`, `8987ce0`, `a00afcc`, `b482860`)
+## What shipped (commits `0ece2ba`, `aa967fb`, `753cd83`)

-1. **F-15 — reset code works the moment the email lands** (`internal/web/claim.go`). The
-   reset-request POST now reads the hub's response (`claim {code_hash, generation, issued_at}`,
-   hub ≥0.52.0) and applies it through the SAME generation-guarded consumer as the report ACK
-   (`report.ClaimSync.Reconcile` — one consumer, two feeds). Replayed/older generations can never
-   downgrade the active hash; a legacy hub's bare `{"status":"ok"}` is a no-op.
-2. **F-11 — inline confirms** (`layout.html` helper + 7 call sites). `felhomConfirm(el, q, onYes)`
-   swaps the trigger IN PLACE to "kérdés + Igen/Mégse"; form buttons opt in with `data-confirm`
-   (delegated listener + `requestSubmit`). Zero native `confirm()`/`prompt()` remain —
-   enforced by the new committed gate `controller/scripts/native_confirm_gate.py`. The deploy
-   stale-data delete keeps its double acknowledgement (two chained inline steps).
-3. **Tier-3 rename** — customer-facing "NAS-mentés" → **"Távoli mentés"** across `backups.html`,
-   `offbox_handlers.go` flashes, and the `backup/offbox.go` quota copy; the manual-target form is
-   generalized to any SFTP target. Executed as a count-guarded python sweep (multibyte rule);
-   enforced by `controller/scripts/offbox_rename_gate.py`. "Hálózati tárhely" (NAS network
-   storage) is a different feature and was deliberately untouched.
-4. **Zero-toggle honesty** — configured+escrowed offbox with no app selected: muted hint
-   "Nincs távoli mentésre jelölt alkalmazás — jelölj ki legalább egyet." + the run reports
-   "Sikeres — nincs mentésre jelölt alkalmazás" (LastWarning; status stays ok).
-5. **Dev-box green-gate fix** (`backup.go` `atomicPromoteTar`): fsync via O_RDWR — read-only
-   fsync is refused on Windows; the two F7 atomic-dump tests were permanently red on the dev box
-   since CAMPAIGN-3 (Linux unchanged).
+1. **IA split** — `/backups` (Áttekintés), `/backups/remote` (Távoli mentés), `/backups/apps`
+   (Alkalmazások), `/backups/restore` (Visszaállítás), sidebar children (Tárhely pattern).
+   Sections MOVED verbatim from the v0.123.0 page — the committed one-shot
+   `scripts/backups_split_move_check.py` verifies all 15 blocks against baseline `df7ad37`
+   (whitespace-normalized; the two allowed divergences — tier-3 anchor retarget + the
+   restore-to-verify relocation — are encoded explicitly). Shared handler builders extracted
+   (`backupsCommonData`/`backupsOffboxData`); flash redirects, tier-3 anchors and the
+   tier2-config back-link retargeted per page.
+2. **Felhom-offsite status card** — three honest states from local data, display-only by
+   construction (no form/button inside the card; unit-enforced). Decision 2 honored: the card
+   never changes anything.
+3. **.fab browser download** — existing export pipeline + staging dest + guarded streaming exit
+   with estimate-first UX, post-stream cleanup and a 1h TTL sweep; per-app sequential batch;
+   portability framing everywhere (decision 3). Import stays drive-scan.

-## Tests + red-proofs (all run → FAIL → restored)
+## Tests + red-proofs (run → FAIL → restored)

- `TestResetRequest_ResponseHashAppliedImmediately` (+ generation-guard + legacy-hub no-op
-  tests). **Red-proof:** response handling disabled → FAIL `gen=1 hash-is-new=false`.
- `native_confirm_gate.py` **red-proof:** one `onsubmit="return confirm('redproof')"`
-  reintroduced → exit 1; restored → 0.
- `offbox_rename_gate.py` **red-proof:** one 'A NAS-mentés…' flash reintroduced → exit 1.
- `TestOffbox_ZeroToggledRunReportsEmptiness` (+ covered-run negative). **Red-proof:** emptiness
-  branch disabled → FAIL `LastWarning = ""`.
- Full green gate (build+vet+test, all packages) + template_id/emoji/native_confirm/offbox_rename
-  gates green before every commit.
+- **Move check red-proof:** one moved block reworded → exit 1 ("1 block(s) rewritten"); restored → 0.
+- **Traversal guard red-proof:** guard loosened to raw prefix-matching → `../decoy.fab` came back
+  **200 serving the decoy** — the test caught it (got 200, want 400). Restored → green.
+- **Round-trip + corruption (data-mutation red-proof):** real `executeExport` → bundle →
+  real `executeImport` → byte-identical restored content; a mid-file-corrupted copy is REFUSED
+  (job fails, app not started, nothing restored). Integrity = gzip CRC + manifest segment
+  validation — there is NO per-file checksum (documented; the spec's "manifest checksum" wording
+  adapted to reality).
+- Nav/state tests (marker per section, present on exactly one page; `.Page` ids), status-card
+  state tests, TTL sweep test (fake mtimes/now). Full green gate + all five template gates green
+  per commit.

-## Live validation (drill box qm300/guest 9201 + demo 9201, controller 0.123.0 both, healthy)
+## §13 live validation

- **F-15 exact-failure-path re-run:** "Visszaállító kód kérése" on felhom.enkisfelhom.hu/claim →
-  controller log `claim-sync: hub claim code cached (generation 3)` **1 second** after the POST →
-  Viktor entered the emailed code: **accepted immediately, first try** (take-two failure closed).
- **F-11 automation-completes proof:** offbox restore-to-verify driven END-TO-END by browser
-  automation — inline strip rendered, Igen → restore ran ("A távoli visszaállítás elindult…",
-  banner "Távoli visszaállítás folyamatban: actualbudget…"), zero freezing.
- **Rename:** /backups renders with ZERO 'NAS-mentés' strings (screenshots in session record);
-  only the allowlisted "saját NAS vagy Felhom offsite tárhely" intro mention remains.
- **Zero-toggle:** untoggled ActualBudget → hint appeared → run → "Sikeres — nincs mentésre
-  jelölt alkalmazás" + status Rendben → re-toggled (box left as found).
+- **Deployed:** 0.124.0 on the drill guest (qm300) AND demo 9201, both healthy.
+- **Four-page walk (drill, screenshots in session record):** every moved control clicked once —
+  tier-2 Beállítás (+ retargeted back-link), tier-3 toggle (off→on), restore-to-verify from its
+  NEW home (`/backups/restore` flash), manual-target form open/close. Old `/backups` bookmark →
+  Áttekintés on both boxes; sidebar active states correct.
+- **Status card:** state 3 (no card) and state 2 ("Aktív — nincs kijelölt alkalmazás") both
+  live-rendered on the drill box; state 1 by unit render test (demo has a managed target — spec
+  says do NOT unconfigure).
+- **.fab download:** drill ActualBudget — estimate 3.6 KB shown first, bundle 2102 B streamed,
+  staging bundle removed (server log). Demo ActualBudget — downloaded to the operator machine,
+  sha256-identical after placement on `felhom-usb/exports` (`e2a1233f…` both ends).
+- **Supervised import (Viktor GO):** ON DEMO (see finding 2 for why not drill): scan found the
+  bundle, manifest read (exported-by 0.124.0), "Importálás kész! Az alkalmazás sikeresen
+  visszaállítva.", ActualBudget Up (healthy). **The round-trip surfaced finding 1.**
+
+## FINDINGS (both pre-existing, surfaced by this task's first real containerized round-trip)
+
+1. **(HIGH — data-loss trap) Containerized .fab export ships EMPTY volume data.**
+   `exportVolumeData` runs `docker run -v <volDir>:/out alpine tar …` where `<volDir>` is the
+   controller-CONTAINER's `os.MkdirTemp` path — the guest's docker daemon resolves it against the
+   HOST filesystem, so the tar lands stranded on the guest host (`/tmp/felhom-export-*/…`, found
+   with ActualBudget's 66 KB tar in it) and the bundle's `data/volumes` is empty while the
+   manifest claims volume data and the export reports SUCCESS. Import then wipes the app's
+   volumes and brings it up EMPTY. Affects every golden/bootstrap (containerized) box for
+   volume-backed apps; HDD-data apps are unaffected (direct FS copy). Demo's ActualBudget was
+   repaired from the stranded tar; both temp dir and staged bundle cleaned. **Fix direction:**
+   stage the docker-run output under a host-visible bind (e.g. the controller data volume) or use
+   `docker cp`/stdout streaming; plus a post-export assertion that manifest-claimed volume tars
+   exist in the bundle (fail LOUD).
+2. **(MEDIUM — agent-side, out of this controller-only task) Drive wizard offers zero candidates
+   on a legacy-boot PVE.** With LVM root and no ESP mount, every system mount resolves to
+   device-mapper, `storage.SystemDisks` finds no raw disk → `sysKnown=false` → `isSystemBacked`
+   fail-safes EVERYTHING to system → `ListCandidateDisks` is always empty (drill box; a hot-added
+   blank 5G disk stayed invisible). The fail-safe is right to exist but overreaches here — needs
+   an agent ruling (e.g. resolve device-mapper parents to their PVs' disks).

 ## Observations

- The queued **customer-target coexistence** task (`managed_by` on OffboxTarget) remains a
-  SEPARATE task with a pending operator fork — not started, per spec.
- NAS-string inventory beyond backups.html: the two customer-facing quota strings in
-  `internal/backup/offbox.go` (renamed here); everything else is the distinct "Hálózati tárhely"
-  feature or English comments/tests (untouched by design).
- Windows-only pre-existing failure found+fixed: read-only-handle fsync in `atomicPromoteTar`
-  (the F7 tests were red on the dev box, green on Linux — a silent hole in the local green gate).
- Browser-automation note: `find`-ref clicks on plain submit buttons focused but did not submit
-  (coordinate clicks did); the data-confirm buttons worked via refs. Cosmetic, tooling-side.
+- **.fab browser-UPLOAD** remains the noted follow-up (import is drive-scan only — downloaded
+  bundles return via a drive/share, exactly as exercised here).
+- **Combined mega-zip** stays parked (decision; per-app sequential batch shipped instead).
+- **appexport packages LIVE state** (fresh config copy + DB dump + live data), NOT the tier-1
+  recovery unit — §8's "stale unit" concern is moot; no "futtass előbb mentést" needed.
+- The import page's drive-label mojibake ("TÃ˘rhely (felhom-usb)") is the KNOWN open fix-5 item.
+- `NoUserDataBackupWarning` is referenced by the per-app section but never set by any handler
+  (dead conditional, pre-existing); `DBDumpTotalBytes` was computed but never rendered — dropped
+  in the split.
+- Drill box left as found: hot-added disk detached + LV removed; ActualBudget re-toggled; demo
+  exports dir + stray export temp cleaned.
@@ -208,6 +208,8 @@ Cross-repo edges:
 - **New background job**: `sched.Every`/`sched.Daily` registration block in controller/cmd/controller/main.go.
 - **New template function**: `Server.templateFuncMap` (controller/internal/web/funcmap.go) — obey v2 state-suffix vocabulary.
 - **New page/nav item**: `baseData` + sidebar in controller/internal/web/templates/ (nested sub-links pattern `.nav-links-nested`); must pass `controller/scripts/template_id_gate.py` + `controller/scripts/emoji_gate.py` + `controller/scripts/native_confirm_gate.py` + `controller/scripts/offbox_rename_gate.py`.
+- **Guarded file download (v0.124.0)**: `handler_export_download.go` — the canonical shape for streaming a server-side file to the browser: accept a BASENAME only (shape regexp + no separators/`..`), `filepath.Join` then assert `filepath.Dir(path) == dir`, `io.Copy` (never ReadAll), `Content-Disposition: attachment`, remove after a successful stream, TTL sweep (`sweepFabDownloads(dir, now, maxAge, logger)` — now injected for tests). Red-proof the guard by loosening to prefix-matching (the `..` case must fail).
+- **Backups sub-page data**: `backupsCommonData(page, title, r)` + `backupsOffboxData(data)` (handlers.go) — the ONLY builders for the four `/backups*` pages; a new backups section extends these, never re-derives in a page handler. Section moves are gated by `scripts/backups_split_move_check.py` (one-shot vs df7ad37).
 - **Consequential-action confirm (LIGHT)**: `felhomConfirm(el, question, onYes)` in layout.html (v0.123.0) — the trigger swaps in place to "kérdés + Igen/Mégse"; form buttons opt in with `data-confirm="…"` (delegated listener, `requestSubmit` keeps formaction/name-value). NEVER native `confirm()`/`prompt()` (OS-modals freeze browser automation — drill F-11; `native_confirm_gate.py` enforces). Heavy destructive flows keep the `.confirm-overlay` `openDialog` pattern.
 - **New hub event**: typed `Notify*` wrapper on Notifier + hub allowlist entry (cross-repo).
 - **New app integration**: `integrations.Manager.RegisterHandler` with `IntegrationKey(provider, target)`.
@@ -343,9 +343,11 @@ Per-app export creates a self-contained `.fab` file (tar.gz, optionally encrypte

 **Architecture:** `internal/appexport/` package with `ExportStackProvider` adapter interface (same pattern as `backup.StackDataProvider`). `exportAdapter` in `main.go` bridges `stacks.Manager` to the provider.

-**API endpoints:** `/api/export/estimate`, `/api/export/start`, `/api/export/status`, `/api/export/bundles`, `/api/export/manifest`, `/api/export/import`, `/api/export/import/status`.
+**API endpoints:** `/api/export/estimate`, `/api/export/start`, `/api/export/status`, `/api/export/bundles`, `/api/export/manifest`, `/api/export/import`, `/api/export/import/status`; **browser download (v0.124.0):** `/api/export/download/estimate`, `/api/export/download/start`, `GET /api/export/download?file=<basename>`.

-**UI:** Export button on app info page, standalone import page at `/import` accessible from the stacks page header.
+**Browser download (v0.124.0 — portability, NOT a backup tier):** the same export pipeline runs with dest = `<DataDir>/fab-downloads/` (same producer → byte-identical bundle), then streams via a guarded endpoint (basename-shape + dir-containment guard; `Content-Disposition: attachment`; `io.Copy`; the staged bundle is removed after the stream and a 1h TTL sweep runs on startup + each start). Estimate is shown BEFORE starting; the batch UI downloads apps one at a time (no combined archive). Import remains drive-scan only — a downloaded bundle returns via a drive/share. `handler_export_download.go`.
+
+**UI:** Export button on app info page, the "Hordozható mentéscsomag (.fab)" section on `/backups/restore` (per-app download + batch), standalone import page at `/import`.

 ---

@@ -354,6 +356,19 @@ Per-app export creates a self-contained `.fab` file (tar.gz, optionally encrypte
 The backup system implements a **3-2-1 backup architecture**. Each tier is a **complete,
 self-sufficient backup** — any single tier can fully restore an app.

+**Page map (v0.124.0 IA split — sidebar children under Biztonsági mentés):**
+
+| Route | Page | Sections |
+|-------|------|----------|
+| `/backups` | Áttekintés | storage overview, whole-guest Rendszermentés, status stat cards, single-copy warning |
+| `/backups/remote` | Távoli mentés | Felhom-offsite status card (3 states, display-only), tier-3 status block + quota, participation toggles (+ zero-toggle hint), manual-target form (`#offbox-section`) |
+| `/backups/apps` | Alkalmazások | schedule, Adatbázisok table, per-app 1./2./3. tier rows (tier-2 config entry; tier-3 actions deep-link to `/backups/remote#offbox-section`) |
+| `/backups/restore` | Visszaállítás | restore panel, offbox restore-to-verify list, .fab download/import loop |
+
+Shared data builders: `backupsCommonData` (chrome + full-status + flash) + `backupsOffboxData`
+(offbox target/toggles) in `handlers.go`; shared partials in `templates/backups_shared.html`.
+The split is MOVE-only vs v0.123.0 — `scripts/backups_split_move_check.py` gates it.
+
 | Tier | Contents | Location | Can fully restore? |
 |------|----------|----------|--------------------|
 | **1. Nightly restic** | DB + Config + User data | Same drive as app | Yes (not against drive failure) |