Task 2 of the backup-classification-redesign arc. Ships the referential-
coupling classification as DATA + PARSER + PURE CLASSIFIER, deliberately
inert — no backup tier changes behavior. Task 3 (tier policy engine) and
Task 4 (manual .fab UI) consume it.
- appbackup/classify.go: BackupSpec/BindSpec/ComposeBind/ClassifiedBind;
ClassifyBinds (SQ5 two-level default — explicit beats :ro; unlisted
writable→mandatory, unlisted :ro→excluded; nil spec→legacy/false);
ValidateBackupSpec (whole-block-reject on any defect, first defect named).
- stacks/classify_binds.go: ParseComposeClassifiableBinds — ${VAR}-relative
binds + :ro flag (NOT ParseComposeHDDMounts/ExportDataMounts, the traps).
- Metadata.Backup + LoadMetadata as the single validation choke point (bad
catalog block → nil + one ERROR → legacy, within one sync cycle).
- Manager.ClassifiedBinds + StackDataProvider.GetStackClassifiedBinds seam
(delegated by stackAdapter, nil-stubbed in every fake) — wired + tested
now so Task 3 consumes a tested seam.
INERT: full pre-existing suite green with zero test-logic edits. +14 tests;
red-proofs RP-1..RP-4 confirmed. The 13 catalog backup: blocks ship in the
same app-catalog change (this controller deploys first).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
The controller assumed an app's HDD appdata dir is always appdata/<stackName>.
paperless-ngx writes appdata/paperless (stack paperless-ngx), so every consumer
keying by stack name silently missed it via a stat-and-skip. One canonical
resolver appbackup.AppDataDirNames derives the real dir name(s) from the app's
compose ${HDD_PATH} binds; all consumers use it.
- F-S2 (tier-2): RunTier2 mirrors the resolved appdata/<name> (paperless docs
got NO tier-2 copy before). Tier2Info size + RestoreTier2Files live dir use it.
WARN when a declared appdata dir is absent. New tier2Mirror seam.
- F-S3 (migrate, NEW): all six per-app appdata legs (collision/size/copy/verify/
cleanup/skip-set) now loop resolved names. scope="app" migration of paperless
previously copied nothing and left an empty media dir (scope="all" was saved by
the merge walk). WARN on missing declared dir in the copy leg.
- Multi-dir (N>1) refusal: tier-2 backup/info/restore refuse loudly (Hungarian);
migrate supports N. No catalog app hits it today; lifted by Task 3.
- Display: storage page sums resolved dirs.
- Truth repair: the v0.130.0 "tier-2 copies the namespace wholesale" claim is
false; corrected in CHANGELOG + main.go export-adapter comment.
+9 tests; red-proofs RP-1..RP-5 all confirmed. Controller-only, no agent/hub
coupling. Task 1 of the backup-classification-redesign arc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
ListDumpFiles ran ValidateDump (line-by-line scan) for every dump on every ~5-min
RefreshCache cycle — wasted I/O+CPU on large customer dumps. ListDumpFiles now takes
an optional cached(name,size,mod) lookup; on a (size+modtime) match it reuses the
prior result and skips ValidateDump. settings.DBValidationCache gains Size+ModTime;
listAllDumpFiles builds the lookup from the persisted cache and writes back only fresh
validations (cache miss), so an unchanged dump triggers neither a re-validation nor a
settings.json write each cycle. nil cached = legacy validate-always (back-compat).
Tests: cache-hit skips validate (sentinel), cache-miss validates, nil validates.
deriveStackName pure-suffix-stripped on '-' (postgres/db/mariadb/.../cache), so a
stack whose slug ENDS in a role token (e.g. 'my-cache') was misattributed (stripped
to 'my') — filing its DB dump under the wrong/nonexistent stack. Now threads the set
of deployed stack names (m.knownStackNames() <- ListDeployedStacks) into
DiscoverDatabases and cross-references: candidate suffix-strip if known, else the
container name if it IS a known stack, else longest known stack that is a prefix
(handles <stack>_postgres / <stack>-1), else legacy strip. nil/empty known = legacy
behaviour (appexport passes nil). Table test incl. the my-cache case (fails pre-fix).
The restore paths (RestoreFromRecoveryUnit + the RestoreApp fallback) repopulated
Docker volume tars but NEVER replayed the captured <stack>-<dbtype>.sql dump, so
DB-resident data (e.g. rows in a DB whose data dir is a bind mount) did not come
back — the romm marker round-trip in the audit lost the row.
New appbackup.ImportDump (read-side counterpart to DumpOne) replays a .sql/.sql.gz
into the running DB using the live container's OWN discovered credentials (no env
threading; reuses DiscoveredDB + getMariaDBPassword). backup.reimportDBDumps
orchestrates it AFTER volume restore + stack bring-up, so the logical dump WINS
over any volume-tar copy of the DB (operator-chosen precedence). pg_dump
--clean --if-exists and mariadb-dump (default --add-drop-table) make replay
idempotent; psql ON_ERROR_STOP=1 surfaces real import errors.
Also: volume-restore per-volume failures and DB-import failures now SURFACE (the
restore returns an error) instead of a swallowed WARN, so a failed data restore
cannot read as success.
Tests (restore_db_test.go, injectable discover/import seams): imports when dump+DB
present, failure surfaces, no-dump skips discovery, dump-but-no-matching-DB is a
non-fatal skip. Live DB round-trip to be validated post-deploy.
Restore recreates an app from its on-drive unit + the guest's own secrets,
regenerating nothing. reconcileRestoreSecrets (pure, unit-tested) merges the unit's
non-secret env with secrets recovered from the live app.yaml and FAILS CLOSED if a
data-encrypting key is unrecoverable (refuse — a PBS whole-guest restore is needed —
rather than regenerate and corrupt). Resettable secrets missing → warn + proceed.
- backup: RestoreFromRecoveryUnit (manifest -> recover secrets -> gate -> restore
volumes -> recreate definition + redeploy w/ re-pull); falls back to volume-only.
- seams: RecoverStackSecrets/RecreateStackFromUnit (adapter +encKey),
stacks.RedeployFromEnv. Wired into /backup/restore.
- tests: gate (refuse/proceed/verbatim) + data_key parsing.
Gate + reconcile + data_key parsing unit-tested; capture live-validated (v0.53.1).
Full readable-data e2e vs AdventureLog needs the auth-gated dashboard restore — pending.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Backups page: whole-guest backup shown as real DR — target label "Biztonsági szerver –
külön hardver (PBS)"; app-data "Távoli mentés" card now reflects the PBS offsite tier
(guestBackupView.Offsite) instead of "nincs beállítva".
- Model-A double-nest fix: appbackup path helpers take a felhom-data NAMESPACE ROOT (no
internal felhom-data join); backup.Manager.namespaceRoot/AppNamespaceRoot resolve
HDD-vs-systemDataPath provenance so a drive-resident app's backups land single-nested
(<drive>/backups/... on the guest = <drive>/felhom-data/backups/... on the host) instead
of .../felhom-data/felhom-data/.... Writes, deletion (GetStackBackupData/RemoveStack/
ProtectedHDDPaths), wipe-warning scan, and export updated coherently; legacy double-nest
dirs kept protected. New appbackup test asserts no doubled segment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the stateless, keep-side app-data backup primitives out of
internal/backup/ into a new self-contained internal/appbackup/ package:
- dbdump.go: DB dump discovery/execution (DiscoverDatabases, DumpOne, ...)
- appdata.go: StackDataProvider + app-data/volume discovery, HumanizeBytes
- paths.go: keep-side path helpers (AppDBDumpPath, AppVolumeDumpPath, AppDataDir)
backup/ keeps every name available via type/const aliases + one-line function
forwarders (appbackup_bridge.go), so the still-present delete-side code
(restic, cross-drive, drive-mount) and the both-side consumers (web/api/report)
compile unchanged. The keep-only consumers appexport and storage are rewired to
import appbackup directly and no longer import backup.
This is the Part-2 prerequisite for the Proxmox port: appbackup has zero
references to restic/cross-drive/drive-mount and does not import backup, so the
delete-side can later be removed without breaking app-data backup or appexport.
Behaviour-preserving: pure move + import/qualifier rewrites, no logic edits.
The four Manager methods (RunDBDumps/DumpAppVolumes/DumpAppVolumesSafe share the
delete-side mutex/status state; RestoreAppFromTier2 reads the cross-drive mirror)
intentionally stay on Manager and delegate to appbackup — for the re-platform step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
admin