Commit Graph

9 Commits

Author SHA1 Message Date
admin 4ed938cce4 D5: an app restore works from the drive alone (v0.188.0)
The recovery unit on the customer's drive now carries the PORTABLE secret
class, so Tier-1/Tier-2 restore no longer depends on the whole-guest tier.
A customer needs the drive and nothing else.

Part 0's rulings overturned the brief's recommendation, on evidence:
- the data_key flag is untrustworthy (4+ encryption keys the catalog itself
  labels as such are unflagged) -> R-127
- a DB password is not resettable in practice: POSTGRES_PASSWORD is ignored
  once PGDATA is non-empty, so a regenerated value leaves the app unable to
  authenticate against its own restored rows while the dump replay still
  reports success (proven on a throwaway postgres:16-alpine)

Ruling (operator): type:secret travels, type:password never does, minus the
nonPortableSecrets code register. Plaintext -- withholding the internet-
reachable class is what licenses that, and the two are coupled.

Precedence: the UNIT WINS over the guest -- the unit's secrets were captured
in the same run as the dumps beside them, so they match the data being
restored. The fail-closed data-key gate is unchanged.

Secret values are never logged; the manifest records NAMES only.
2026-07-30 16:33:06 +02:00
admin 2958946517 v0.172.0 — R-75: canonical import root, catalog-derived skeleton, import surfaces
${IMPORT_PATH} = <system namespace root>/userdata/import — ONE drop-zone per box,
on the system drive, injected at BOTH compose-env builders with NO per-drive
fallback (unresolvable leaves it unset so compose fails loudly rather than
quietly building a second, dead drop-zone).

Third BindRoot (RootImport) + Import list in BackupSpec, extended through
ValidateBackupSpec/ClassifyBinds. Load-bearing: a stale `userdata: import/<app>`
entry against the moved bind would be a WHOLE-BLOCK reject, taking the app's
mandatory hdd classification with it.

Exhaustive-root audit: resolveAbs/structuralGuard/ComputeCaptureSet/
ComputeFabBuckets now take importRoot explicitly (an import bind resolved
against hddPath would name a directory on the wrong drive); unresolvable is
refused loudly into Skipped. GetImportRoot added to both provider interfaces.

Catalog-derived skeleton: UserdataSkeleton() -> UserdataSkeletonCarry() +
BuildUserdataSkeleton(), SORTED. The carry-list makes zero-removals true by
construction (`documents` is in no catalog app but on both boxes) and is the
fresh-box floor. The sort is not tidiness: the naive map-order derivation
measured 20 distinct outputs from 20 identical runs, which with fbNeedsRecreate
is a fleet-wide FileBrowser restart loop.

One authoritative compose parser: ParseComposeUserdataMounts now delegates to
ParseComposeClassifiableBinds. Import root excluded from per-app migration.

Surfaces: FileBrowser /srv/beolvasas source; app-page "Hova tegyem a fajlokat?"
with PathEscape deep links (never QueryEscape) and class-driven copy;
data_paths: annotation with the Fork-3 asymmetry; system-owned beolvasas SMB
share refused server-side at handler AND store, button omitted in template.

Caught on the way: the sharing template's row struct was function-local, so
adding {{if .System}} would have 500'd every share row. ShareRow is now
package-level and the render test uses the handler's own type.

Tests 915 -> 949, all green. MinAgent unchanged.
2026-07-26 08:12:57 +02:00
admin 78ff991f1c v0.153.0 — R-47: the DB replay no longer races the app, on BOTH restore paths
Closes R-47. No new agent coupling — MinAgent stays 0.90.0.

The replay needs a running DB container, so both restore paths started the
WHOLE stack first, giving the application a window to rebuild the very schema
objects the dump was about to create. Measured live on 2026-07-19 (H4,
DIAG-immich-restore-round2): immich-server rebuilt clip_index two seconds
before the dump's CREATE INDEX, the replay aborted "already exists" under
ON_ERROR_STOP=1, and immich reported schema drift. The data survived only
because pg_dump emits COPY before CREATE INDEX.

Both paths now open a DB-ONLY window: only the stack's database service(s)
come up, the dump is replayed with the app still down, and the full start
runs only after the replay exits 0. Fail-closed: a dump with no identifiable
DB service refuses BEFORE the first mutation. Every exit from the window
still does a best-effort full start, so a failed restore never leaves a box
with a database and no application.

New: appbackup.DBServiceNames (yaml.v3 services-map parse — never a line
scan; immich's top-level volume keys are the decoy) sharing dbTypeForImage
with DiscoverDatabases; stacks.Manager.StartStackServices (refuses an empty
list — argument-less `up -d` is a full start); RedeployFromEnv split into
PersistUnitRedeployConfig + its unchanged tail. StackDataProvider's
RecreateStackFromUnit becomes RecreateStackDefinitionFromUnit — the hidden
`up -d` inside the old name is what carried the defect on the local path.

19 new tests (ordering plus state-at-replay-time, zero-mutation fail-closed
effects, replay-failure bring-up, parser decoys, empty-list refusal); three
companion red-proofs run and reverted. 23/23 packages green.

Not yet live-validated: STOP-1 supervised reconstitute, golden 0.153.0.
2026-07-20 17:01:52 +02:00
admin 0649f9a3e6 Backup classification: schema + parser + pure classifier (INERT, v0.132.0)
Task 2 of the backup-classification-redesign arc. Ships the referential-
coupling classification as DATA + PARSER + PURE CLASSIFIER, deliberately
inert — no backup tier changes behavior. Task 3 (tier policy engine) and
Task 4 (manual .fab UI) consume it.

- appbackup/classify.go: BackupSpec/BindSpec/ComposeBind/ClassifiedBind;
  ClassifyBinds (SQ5 two-level default — explicit beats :ro; unlisted
  writable→mandatory, unlisted :ro→excluded; nil spec→legacy/false);
  ValidateBackupSpec (whole-block-reject on any defect, first defect named).
- stacks/classify_binds.go: ParseComposeClassifiableBinds — ${VAR}-relative
  binds + :ro flag (NOT ParseComposeHDDMounts/ExportDataMounts, the traps).
- Metadata.Backup + LoadMetadata as the single validation choke point (bad
  catalog block → nil + one ERROR → legacy, within one sync cycle).
- Manager.ClassifiedBinds + StackDataProvider.GetStackClassifiedBinds seam
  (delegated by stackAdapter, nil-stubbed in every fake) — wired + tested
  now so Task 3 consumes a tested seam.

INERT: full pre-existing suite green with zero test-logic edits. +14 tests;
red-proofs RP-1..RP-4 confirmed. The 13 catalog backup: blocks ship in the
same app-catalog change (this controller deploys first).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
2026-07-14 18:46:32 +02:00
admin 68b3a3932e controller: F7 atomic volume dumps + F6 no-single-copy + F5 stale-primary sweep (WIP, pre-build)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017CDMFpFx84pfviCTVuGGhf
2026-07-12 09:00:16 +02:00
admin e02292aa1a test: Phase 2b restore orchestration coverage + nil-safe isDebug
Adds an in-process orchestration test for RestoreFromRecoveryUnit: success path
calls recreate with non-secret env + recovered secrets merged; data-key-missing
path is REFUSED and recreate is never called. Makes Manager.isDebug nil-safe
(behavior-neutral in prod; cfg is always set) so the gate/orchestration are testable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 11:17:08 +02:00
admin 7863e62f29 v0.54.0: Phase 2b — restore-from-recovery-unit + fail-closed data-key gate
Restore recreates an app from its on-drive unit + the guest's own secrets,
regenerating nothing. reconcileRestoreSecrets (pure, unit-tested) merges the unit's
non-secret env with secrets recovered from the live app.yaml and FAILS CLOSED if a
data-encrypting key is unrecoverable (refuse — a PBS whole-guest restore is needed —
rather than regenerate and corrupt). Resettable secrets missing → warn + proceed.

- backup: RestoreFromRecoveryUnit (manifest -> recover secrets -> gate -> restore
  volumes -> recreate definition + redeploy w/ re-pull); falls back to volume-only.
- seams: RecoverStackSecrets/RecreateStackFromUnit (adapter +encKey),
  stacks.RedeployFromEnv. Wired into /backup/restore.
- tests: gate (refuse/proceed/verbatim) + data_key parsing.

Gate + reconcile + data_key parsing unit-tested; capture live-validated (v0.53.1).
Full readable-data e2e vs AdventureLog needs the auth-gated dashboard restore — pending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 11:12:43 +02:00
admin eefeeabea3 v0.53.1: refresh recovery units on periodic cache cycle (idempotent)
CaptureRecoveryUnit now builds content in memory and skips writes when the unit
is already current (checksum + dump-set + version), so it can run from RefreshCache
(startup + every 5m) without thrashing the USB drive. Units now exist shortly after
startup and track config changes without waiting for the daily DB dump. +idempotency test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 10:27:35 +02:00
admin 70eb521cd0 v0.53.0: Phase 2 capture side — per-app secret-free recovery unit
Each app's on-drive backup becomes a self-contained, recreatable recovery unit:
compose/ (docker-compose.yml + .felhom.yml + secret-stripped app.yaml) alongside
the existing db-dumps/ + volume-dumps/, plus a secret-free manifest.json (image
pins, secret env-var NAMES, data_key names, checksums). The unit stores no secret
value, no data-key, and not the image — secrets are recovered at restore from the
guest's own app.yaml (live/PBS), never regenerated.

- appbackup: RecoveryUnit* path helpers, RecoveryInfo + GetStackRecoveryInfo,
  ParseComposeImages; AppDBDump/Volume refactored onto RecoveryUnitPath.
- backup: recovery_unit.go (manifest + CaptureRecoveryUnit), wired into RunDBDumps;
  capture test proves secret-free.
- stacks: DeployField.DataKey + Metadata.DataKeyEnvVars(); main.go stackAdapter
  implements GetStackRecoveryInfo (excludes secret-named + encrypted values).
- Restore-from-unit recreate + fail-closed gate + live AdventureLog validation: next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 10:20:37 +02:00