docs: v0.99.0 — CHANGELOG/CONTEXT/REUSE/README for the restore-path fixes (F1/F3/O4)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6

CHANGELOG.md

@@ -1,5 +1,40 @@
 ## Changelog
 
+### v0.99.0 — restore-path fixes: dead restore UI + volume dumps + blank-secret redeploy (2026-07-05)
+
+TASK C1 — fixes F1/F3/O4 from the 2026-07-04 restore drill
+(`felhom.eu/documentation/audits/DRILL-appdata-restore-2026-07-04.md`). F2 (one-click in-place
+class-C restore) deliberately NOT included — product-design work (C2).
+
+- **F1 (HIGH — the restore panel was dead):** `GET /api/backup/snapshots?stack=` now exists
+  (`internal/api/router.go` + `backup.Manager.ListRestorePoints`, `internal/backup/restore_points.go`).
+  The backups.html restore panel fetched this restic-era route, got the catch-all 404, so the
+  snapshot dropdown never populated and "Visszaállítás indítása" could never enable. Returns the
+  ONE honest keep-side restore point (the current recovery unit): `time` = newest artifact mtime
+  (manifest / db-dumps / volume-dumps), `short_id:"helyi"`, `tier:1` always (Tier-2 copies are NOT
+  restorable via POST /backup/restore — never listed), `drive_label` from the storage registry.
+  Guards: traversal/empty → 400 (`validStackParam`), unknown stack → 404, no unit yet → `ok:true, data:[]`.
+  No template change needed — the JS payload contract was honoured server-side.
+- **F3 — named-volume data was never backed up:** `DumpAppVolumesSafe` had no production caller.
+  New `runVolumeDumps` loop in `runDBDumpsInternal` (`internal/backup/backup.go`), running BEFORE
+  `captureAllRecoveryUnits` so manifests enumerate the fresh tars. Gate order is load-bearing:
+  protected-stack + has-volumes checks precede the Safe call (which stops the stack before its own
+  check — unconditional calls would bounce every volume-less app nightly); disconnected/decommissioned
+  drives skip like the DB loop. Failures land in the run summary and fail the run (no silent
+  partial). Zero-DB early return removed (volume-only apps still get dumps + unit refresh).
+  Test seam: `dumpVolumesSafe` func field (F17-style).
+- **O4 — missing resettable secret redeployed blank:** the restore proceed-path now generates a
+  replacement credential via the catalog field's `generate` spec (`stacks.Manager.GenerateSecretForField`
+  → `backup.SetSecretGenerator` seam, wired in main.go), persisted encrypted through the existing
+  `RecreateStackFromUnit` → `SaveAppConfig` path. Data-keys are NEVER generated (gate untouched +
+  generator refuses `data_key` fields); values never logged. No-generator fields keep proceeding
+  with an upgraded "may fail to start" WARN. Residual case documented: a restored volume tar
+  carrying the OLD internal credential hash may still need a manual in-DB reset.
+
+Tests: +12 (api snapshots ×3, backup restore-points ×4, volume-dump gating ×3, secret-gen ×2 files);
+all three fixes companion-red-proofed (hollow `[]` endpoint / removed volume gate / no-generation
+each fail their test). Full `go build && go vet && go test ./...` green.
+
 ### docs — CLAUDE.md refresh: slim-down to stable orientation (2026-07-03)
 
 No code change, no version bump. CLAUDE.md 338 → ~160 lines: full 30-package layout map (was 7);

CONTEXT.md

@@ -9,8 +9,16 @@
 
 Last updated: 2026-07-03 (docs: CLAUDE.md refreshed — stable orientation; runbooks live in the felhom skills)
 
+> **2026-07-05 — v0.99.0 restore-path fixes (TASK C1): drill findings F1/F3/O4 RESOLVED.**
+> F1: `GET /api/backup/snapshots` implemented (`backup.ListRestorePoints`) — the restore panel
+> populates and the restore button enables. F3: `runVolumeDumps` wired into the nightly/manual
+> backup run (volume gate BEFORE DumpAppVolumesSafe; before unit capture). O4: unrecoverable
+> resettable secrets get a generated replacement (`GenerateSecretForField` + `SetSecretGenerator`
+> seam); data-key gate untouched. **F2 (one-click in-place class-C restore) remains OPEN → TASK C2.**
+> O4 residual: restored volume tar with an OLD credential hash may still need a manual in-DB reset.
+
 > **2026-07-04 — app-data restore drill** → see `felhom.eu/documentation/audits/DRILL-appdata-restore-2026-07-04.md`.
-> Keep-side restore proven live on 9201 (class-A DB replay + fail-closed data-key gate + non-destruction). **F1 (HIGH): UI restore is dead — `/api/backup/snapshots` has no handler, so the restore button never enables.** F2: no one-click in-place class-C (HDD bind-mount) restore. F3: named-volume data never backed up (`DumpAppVolumes*` has no caller).
+> Keep-side restore proven live on 9201 (class-A DB replay + fail-closed data-key gate + non-destruction). **F1 (HIGH): UI restore is dead — `/api/backup/snapshots` has no handler, so the restore button never enables.** F2: no one-click in-place class-C (HDD bind-mount) restore. F3: named-volume data never backed up (`DumpAppVolumes*` has no caller). *(F1/F3/O4 resolved in v0.99.0, see above.)*
 
 > **2026-07-03 — CLAUDE.md slimmed to stable orientation** (full package map, verified 9201 deploy
 > summary, no version-pinned state). Deep runbooks/design/testing doctrine now in the personal

REUSE.md

@@ -67,7 +67,8 @@
 | `resolveContainerState` / `aggregateState` | controller/internal/stacks/manager.go | `(dockerState, dockerStatus)` / `([]ContainerInfo)` | State classification | `.State` says "running" even when unhealthy — `.Status` parse is the fix |
 | `Manager.logPostStartStatus` | controller/internal/stacks/manager.go | `(name, stackDir, env)` | Async post-start verification | compose up exits 0 on crash-loops; this is the detection. Goroutine + 3s, never blocks |
 | `Manager.EnsureBaseStack` | controller/internal/stacks/infra.go | `() error` | Traefik/cloudflared/FileBrowser infra convergence | Renders from `internal/infra` templates |
-| `backup.Manager.DumpAppVolumesSafe` | controller/internal/backup/backup.go | `(stackName) error` | Volume tar of a live app | Stops → dumps → restarts; surfaces BOTH errors (app may be left stopped) |
+| `backup.Manager.DumpAppVolumesSafe` | controller/internal/backup/backup.go | `(stackName) error` | Volume tar of a live app | Stops → dumps → restarts; surfaces BOTH errors (app may be left stopped). Check `GetDockerVolumes()!=0` + `IsProtectedStack` BEFORE calling — it stops the stack before its own volume check (see `runVolumeDumps`) |
+| `backup.Manager.ListRestorePoints` | controller/internal/backup/restore_points.go | `(stackName) ([]RestorePoint, bool)` | Restorable keep-side backups (the /api/backup/snapshots payload) | ONE point per app (the current unit); tier always 1 — never list Tier-2 (not restorable via /backup/restore) |
 | `Manager.acquireRunning`/`releaseRunning`, `acquireMigrating` | controller/internal/backup/backup.go, controller/internal/stacks/migrate.go | `() error` | Single-flight for long ops | Copy this mutex-flag pattern for any new long-running manager op |
 
 ### Secrets hygiene
@@ -78,6 +79,7 @@
 | `crypto.LoadOrCreateKey` | controller/internal/crypto/crypto.go | `(path) ([]byte, error)` | The 32-byte key file (0600) | — |
 | `SaveAppConfig` / `LoadAppConfigDecrypted` | controller/internal/stacks/deploy.go | `(stackDir, cfg, encKey, sensitiveVars)` | app.yaml persistence | Encrypts only `SensitiveEnvVars(meta)`; never write app.yaml directly |
 | `generateValue` / `randomAlphanumeric` | controller/internal/stacks/deploy.go | `(spec "password:N\|hex:N\|base64key:N\|static:v")` | Auto-generated secrets | crypto/rand-backed; reuse the spec grammar |
+| `Manager.GenerateSecretForField` | controller/internal/stacks/deploy.go | `(stackName, envVar) (string, bool)` | Replacement value for a RESETTABLE secret from its catalog `generate` spec (O4 restore path via `backup.SetSecretGenerator`) | REFUSES `data_key` fields, spec-less and non-secret fields; never log the value |
 | `reconcileRestoreSecrets` | controller/internal/backup/restore_unit.go | `(nonSecretEnv, recoveredSecrets, secretNames, dataKeyNames)` | Recovery-unit restore env merge | Units are secret-FREE by design; secrets come from live app.yaml |
 | `EncryptFile` / `DecryptFile` / `IsEncryptedFAB` | controller/internal/appexport/crypto.go | password-based file crypto | .fab export bundles | scrypt-derived AES+HMAC keys |
 | `maskRepoURL` | controller/internal/sync/sync.go | `(url) string` | Logging git URLs | Strips embedded credentials |
@@ -174,6 +176,8 @@
 | `integrations.Handler` + `StackProvider` | controller/internal/integrations/integrations.go + manager.go | OnlyOffice handlers | table-driven tests in package |
 | `bootstrap.PullFunc` | controller/internal/bootstrap/bootstrap.go | `report.PullConfig` | injected in bootstrap tests |
 | `offboxRunner` (func) | controller/internal/backup/offbox.go | `defaultOffboxRunner` (restic exec) | `SetOffboxRunner` injection point |
+| `dumpVolumesSafe` (func seam) | controller/internal/backup/backup.go | nil → real `DumpAppVolumesSafe` | injected in controller/internal/backup/volume_dumps_test.go (gating tests without Docker) |
+| `generateSecret` (func seam) | controller/internal/backup/backup.go | `stacks.Manager.GenerateSecretForField` via `SetSecretGenerator` (main.go) | injected in controller/internal/backup/restore_secrets_gen_test.go |
 
 Cross-repo edges:
 - `controller/internal/agentapi/client.go` ↔ **felhom-agent** local API (`/storage`, `/disks*`, `/backup*`, `/netstorage*`, `/guest/*`): pinned leaf SHA-256 + per-guest bearer token from bootstrap.json.
@@ -205,6 +209,6 @@ Cross-repo edges:
 | CSRF ×2 | controller/internal/web/csrf.go (session HMAC) vs controller/internal/setup/csrf.go (cookie double-submit) — intentional (pre-auth wizard) but unlabeled |
 | Budapest timezone loader ×2 | controller/internal/scheduler/scheduler.go `getBudapestLocation` vs controller/internal/web/funcmap.go `getTimezone` |
 | JSON writers ×5, 3 envelope shapes | api `writeJSON`; web `writeDiskJSON`, `jsonResponse`/`jsonError`, `writeDebugJSON` |
-| Safe-name validators ×3 | controller/internal/web/validate.go `validStackName`; controller/internal/backup/offbox.go `isSafeStackName`; controller/internal/appexport/validate.go `ValidateSegment` (strictest) |
+| Safe-name validators ×4 | controller/internal/web/validate.go `validStackName`; controller/internal/api/router.go `validStackParam` (same body — api↔web import cycle); controller/internal/backup/offbox.go `isSafeStackName`; controller/internal/appexport/validate.go `ValidateSegment` (strictest) |
 | DB wait/import ×2 | controller/internal/appbackup/dbdump.go `waitDBReady`/`ImportDump` vs controller/internal/appexport/restore.go `waitForDB`/`importDBDump` |
 | compose exec ×2 | controller/internal/stacks/manager.go `composeExecCustomEnv` vs controller/internal/appexport/restore.go `composeExecEnv` (the latter has ctx+timeout; the former has the userdata belt) |

controller/README.md

@@ -441,8 +441,14 @@ backups/primary/<app>/
   Docker image** — only the pinned image tag(s) (re-pulled on restore) and the *names* of the secret /
   `data_key` env vars. Rationale: app.yaml + the encryption key live on the guest rootfs → already in
   the PBS whole-guest snapshot, and the hub is deliberately zero-knowledge. Restore recovers the
-  original secrets from the guest's own app.yaml (live, or via PBS) and **regenerates nothing**; for a
-  `data_key` app it **fails closed** (refuse + warn) if the key can't be recovered.
+  original secrets from the guest's own app.yaml (live, or via PBS); for a `data_key` app it
+  **fails closed** (refuse + warn) if the key can't be recovered — data-keys are NEVER generated.
+  **Resettable secrets (O4, v0.99.0):** an unrecoverable resettable secret (DB password etc.) gets a
+  **generated replacement** from its catalog `generate` spec (`stacks.GenerateSecretForField` via the
+  `backup.SetSecretGenerator` seam) instead of redeploying blank (which failed compose-up); the new
+  value persists encrypted through the normal `RecreateStackFromUnit` → `SaveAppConfig` path. Fields
+  with no `generate` spec still proceed with a loud "may fail to start" WARN. Residual case: a restored
+  volume tar carrying the OLD internal credential hash may still need a manual in-DB reset.
 - Helpers: `RecoveryUnitPath` / `RecoveryUnitComposePath` / `RecoveryUnitManifestPath`
   (`internal/appbackup/paths.go`). Capture: `Manager.CaptureRecoveryUnit` (`internal/backup/recovery_unit.go`),
   run from the daily DB dump and the periodic `RefreshCache` (idempotent checksum-skip). The non-secret
@@ -492,15 +498,23 @@ re-validates the pin off-disk at run time. `Manager.Tier2Info(stackName)` is the
 - **Validation** after each dump: checks file size, header presence, counts `CREATE TABLE`
 - Results cached in `settings.json` surviving container restarts
 
-**Phase 1b — Docker Volume Dumps** (`internal/backup/backup.go`, runs after DB dumps)
+**Phase 1b — Docker Volume Dumps** (`internal/backup/backup.go` `runVolumeDumps`, part of the same run)
 
-- Iterates all deployed stacks that have Docker named volumes (`GetDockerVolumes()`)
-- **v0.34.0:** Each stack is stopped before dump, restarted after (`DumpAppVolumesSafe()`) — prevents inconsistent tars of live databases. Protected stacks (traefik, etc.) that reject StopStack are skipped with a warning.
+- **F3 (v0.99.0):** re-wired into the nightly/manual app-data backup run (`runDBDumpsInternal`) —
+  after the restic removal `DumpAppVolumesSafe` had no caller, so `volume-dumps/` was never produced.
+  Runs AFTER the DB dumps and BEFORE `captureAllRecoveryUnits` so the manifests enumerate fresh tars.
+- Gate order (load-bearing): protected-stack (`cfg.IsProtectedStack`) and has-volumes
+  (`GetDockerVolumes()`) checks come BEFORE `DumpAppVolumesSafe` — the Safe variant stops the stack
+  before its own volume check, so unconditional calls would bounce every volume-less app nightly.
+  Disconnected/decommissioned drives skip with the same summary style as the DB loop.
+- Each volume-bearing stack is stopped before dump, restarted after (`DumpAppVolumesSafe()`) —
+  prevents inconsistent tars of live databases.
 - For each volume: `docker run --rm -v <vol>:/vol:ro -v <dumpDir>:/out alpine tar cf /out/<vol>.tar -C /vol .`
-- 10-minute timeout per volume; warnings on failure (non-fatal)
+- 10-minute timeout per volume; a per-stack failure lands in the run summary as `FAIL <app> volumes:`,
+  flips the run's Success flag and fails the run (no silent partial) — other stacks still proceed
 - Stale tars cleaned up (volumes that no longer exist)
 - Volume names resolved with project prefix via `ResolveDockerVolumeNames()` (e.g., `mealie_mealie_data`)
-- Dumps written to `AppVolumeDumpPath(appDrive, stackName)`
+- Dumps written to `AppVolumeDumpPath(nsRoot, stackName)`
 
 **Phase 2 — Restic Snapshot** (`internal/backup/restic.go`, scheduled 03:00)
 
@@ -589,10 +603,17 @@ appear in the restore dropdown with per-app snapshot filtering.
 | DB only, no HDD/volumes | Yes | Yes | n/a | n/a |
 | Config only | Yes | — | n/a | n/a |
 
-**Snapshot API** (`/api/backup/snapshots?stack=<name>`):
-- Returns snapshots **only from the app's home drive** primary repo (prevents showing irrelevant snapshots from other drives)
-- Appends a synthetic Tier 2 entry (ID `tier2-rsync`) from cross-drive config when last backup was successful
-- Dropdown groups by tier: "1. szint — Helyi mentes" and "2. szint — Masodlagos masolat"
+**Snapshot API** (`GET /api/backup/snapshots?stack=<name>` — F1, v0.99.0):
+- Backed by `backup.Manager.ListRestorePoints` (`internal/backup/restore_points.go`). The keep-side
+  restore has exactly **one** restore point per app — the current recovery unit — so the endpoint
+  returns at most one entry: `time` = newest artifact mtime (manifest / db-dumps / volume-dumps),
+  `short_id:"helyi"`, `tier:1`, `drive_label` from the storage registry (empty on the SSD fallback)
+- **Never emits tier-2 entries**: Tier-2 copies are not restorable via `POST /backup/restore` (it
+  only reads the primary unit) — listing them would silently restore tier-1 data while claiming tier-2
+- Guards: empty/traversal stack name → 400 (`validStackParam`), unknown stack → 404, known stack
+  with no unit yet → `ok:true, data:[]` (the UI shows "Nincs elérhető mentés")
+- History: the route was a restic-era leftover fetched by the template but unregistered — the
+  dropdown could never populate and the restore button never enabled (drill finding F1)
 
 **Restore type info** shown per-app when selected in dropdown (Hungarian banners):
   - Has HDD or Docker volumes: "Teljes visszaallitas: adatbazis + konfiguracio + felhasznaloi adatok"