Commit Graph

5 Commits

Author SHA1 Message Date
admin 78ff991f1c v0.153.0 — R-47: the DB replay no longer races the app, on BOTH restore paths
Closes R-47. No new agent coupling — MinAgent stays 0.90.0.

The replay needs a running DB container, so both restore paths started the
WHOLE stack first, giving the application a window to rebuild the very schema
objects the dump was about to create. Measured live on 2026-07-19 (H4,
DIAG-immich-restore-round2): immich-server rebuilt clip_index two seconds
before the dump's CREATE INDEX, the replay aborted "already exists" under
ON_ERROR_STOP=1, and immich reported schema drift. The data survived only
because pg_dump emits COPY before CREATE INDEX.

Both paths now open a DB-ONLY window: only the stack's database service(s)
come up, the dump is replayed with the app still down, and the full start
runs only after the replay exits 0. Fail-closed: a dump with no identifiable
DB service refuses BEFORE the first mutation. Every exit from the window
still does a best-effort full start, so a failed restore never leaves a box
with a database and no application.

New: appbackup.DBServiceNames (yaml.v3 services-map parse — never a line
scan; immich's top-level volume keys are the decoy) sharing dbTypeForImage
with DiscoverDatabases; stacks.Manager.StartStackServices (refuses an empty
list — argument-less `up -d` is a full start); RedeployFromEnv split into
PersistUnitRedeployConfig + its unchanged tail. StackDataProvider's
RecreateStackFromUnit becomes RecreateStackDefinitionFromUnit — the hidden
`up -d` inside the old name is what carried the defect on the local path.

19 new tests (ordering plus state-at-replay-time, zero-mutation fail-closed
effects, replay-failure bring-up, parser decoys, empty-list refusal); three
companion red-proofs run and reverted. 23/23 packages green.

Not yet live-validated: STOP-1 supervised reconstitute, golden 0.153.0.
2026-07-20 17:01:52 +02:00
admin 0649f9a3e6 Backup classification: schema + parser + pure classifier (INERT, v0.132.0)
Task 2 of the backup-classification-redesign arc. Ships the referential-
coupling classification as DATA + PARSER + PURE CLASSIFIER, deliberately
inert — no backup tier changes behavior. Task 3 (tier policy engine) and
Task 4 (manual .fab UI) consume it.

- appbackup/classify.go: BackupSpec/BindSpec/ComposeBind/ClassifiedBind;
  ClassifyBinds (SQ5 two-level default — explicit beats :ro; unlisted
  writable→mandatory, unlisted :ro→excluded; nil spec→legacy/false);
  ValidateBackupSpec (whole-block-reject on any defect, first defect named).
- stacks/classify_binds.go: ParseComposeClassifiableBinds — ${VAR}-relative
  binds + :ro flag (NOT ParseComposeHDDMounts/ExportDataMounts, the traps).
- Metadata.Backup + LoadMetadata as the single validation choke point (bad
  catalog block → nil + one ERROR → legacy, within one sync cycle).
- Manager.ClassifiedBinds + StackDataProvider.GetStackClassifiedBinds seam
  (delegated by stackAdapter, nil-stubbed in every fake) — wired + tested
  now so Task 3 consumes a tested seam.

INERT: full pre-existing suite green with zero test-logic edits. +14 tests;
red-proofs RP-1..RP-4 confirmed. The 13 catalog backup: blocks ship in the
same app-catalog change (this controller deploys first).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
2026-07-14 18:46:32 +02:00
admin 7863e62f29 v0.54.0: Phase 2b — restore-from-recovery-unit + fail-closed data-key gate
Restore recreates an app from its on-drive unit + the guest's own secrets,
regenerating nothing. reconcileRestoreSecrets (pure, unit-tested) merges the unit's
non-secret env with secrets recovered from the live app.yaml and FAILS CLOSED if a
data-encrypting key is unrecoverable (refuse — a PBS whole-guest restore is needed —
rather than regenerate and corrupt). Resettable secrets missing → warn + proceed.

- backup: RestoreFromRecoveryUnit (manifest -> recover secrets -> gate -> restore
  volumes -> recreate definition + redeploy w/ re-pull); falls back to volume-only.
- seams: RecoverStackSecrets/RecreateStackFromUnit (adapter +encKey),
  stacks.RedeployFromEnv. Wired into /backup/restore.
- tests: gate (refuse/proceed/verbatim) + data_key parsing.

Gate + reconcile + data_key parsing unit-tested; capture live-validated (v0.53.1).
Full readable-data e2e vs AdventureLog needs the auth-gated dashboard restore — pending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 11:12:43 +02:00
admin 70eb521cd0 v0.53.0: Phase 2 capture side — per-app secret-free recovery unit
Each app's on-drive backup becomes a self-contained, recreatable recovery unit:
compose/ (docker-compose.yml + .felhom.yml + secret-stripped app.yaml) alongside
the existing db-dumps/ + volume-dumps/, plus a secret-free manifest.json (image
pins, secret env-var NAMES, data_key names, checksums). The unit stores no secret
value, no data-key, and not the image — secrets are recovered at restore from the
guest's own app.yaml (live/PBS), never regenerated.

- appbackup: RecoveryUnit* path helpers, RecoveryInfo + GetStackRecoveryInfo,
  ParseComposeImages; AppDBDump/Volume refactored onto RecoveryUnitPath.
- backup: recovery_unit.go (manifest + CaptureRecoveryUnit), wired into RunDBDumps;
  capture test proves secret-free.
- stacks: DeployField.DataKey + Metadata.DataKeyEnvVars(); main.go stackAdapter
  implements GetStackRecoveryInfo (excludes secret-named + encrypted values).
- Restore-from-unit recreate + fail-closed gate + live AdventureLog validation: next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 10:20:37 +02:00
admin a4de90def3 refactor: extract app-data-backup into internal/appbackup (no behaviour change)
Extract the stateless, keep-side app-data backup primitives out of
internal/backup/ into a new self-contained internal/appbackup/ package:
- dbdump.go: DB dump discovery/execution (DiscoverDatabases, DumpOne, ...)
- appdata.go: StackDataProvider + app-data/volume discovery, HumanizeBytes
- paths.go: keep-side path helpers (AppDBDumpPath, AppVolumeDumpPath, AppDataDir)

backup/ keeps every name available via type/const aliases + one-line function
forwarders (appbackup_bridge.go), so the still-present delete-side code
(restic, cross-drive, drive-mount) and the both-side consumers (web/api/report)
compile unchanged. The keep-only consumers appexport and storage are rewired to
import appbackup directly and no longer import backup.

This is the Part-2 prerequisite for the Proxmox port: appbackup has zero
references to restic/cross-drive/drive-mount and does not import backup, so the
delete-side can later be removed without breaking app-data backup or appexport.

Behaviour-preserving: pure move + import/qualifier rewrites, no logic edits.
The four Manager methods (RunDBDumps/DumpAppVolumes/DumpAppVolumesSafe share the
delete-side mutex/status state; RestoreAppFromTier2 reads the cross-drive mirror)
intentionally stay on Manager and delegate to appbackup — for the re-platform step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 11:01:39 +02:00