v0.172.0 — R-75: canonical import root, catalog-derived skeleton, import surfaces
${IMPORT_PATH} = <system namespace root>/userdata/import — ONE drop-zone per box,
on the system drive, injected at BOTH compose-env builders with NO per-drive
fallback (unresolvable leaves it unset so compose fails loudly rather than
quietly building a second, dead drop-zone).
Third BindRoot (RootImport) + Import list in BackupSpec, extended through
ValidateBackupSpec/ClassifyBinds. Load-bearing: a stale `userdata: import/<app>`
entry against the moved bind would be a WHOLE-BLOCK reject, taking the app's
mandatory hdd classification with it.
Exhaustive-root audit: resolveAbs/structuralGuard/ComputeCaptureSet/
ComputeFabBuckets now take importRoot explicitly (an import bind resolved
against hddPath would name a directory on the wrong drive); unresolvable is
refused loudly into Skipped. GetImportRoot added to both provider interfaces.
Catalog-derived skeleton: UserdataSkeleton() -> UserdataSkeletonCarry() +
BuildUserdataSkeleton(), SORTED. The carry-list makes zero-removals true by
construction (`documents` is in no catalog app but on both boxes) and is the
fresh-box floor. The sort is not tidiness: the naive map-order derivation
measured 20 distinct outputs from 20 identical runs, which with fbNeedsRecreate
is a fleet-wide FileBrowser restart loop.
One authoritative compose parser: ParseComposeUserdataMounts now delegates to
ParseComposeClassifiableBinds. Import root excluded from per-app migration.
Surfaces: FileBrowser /srv/beolvasas source; app-page "Hova tegyem a fajlokat?"
with PathEscape deep links (never QueryEscape) and class-driven copy;
data_paths: annotation with the Fork-3 asymmetry; system-owned beolvasas SMB
share refused server-side at handler AND store, button omitted in template.
Caught on the way: the sharing template's row struct was function-local, so
adding {{if .System}} would have 500'd every share row. ShareRow is now
package-level and the render test uses the handler's own type.
Tests 915 -> 949, all green. MinAgent unchanged.
This commit is contained in:
@@ -2,7 +2,10 @@ package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Customer-facing userdata layout + the shared-storage ownership convention (v0.66.0).
|
||||
@@ -28,9 +31,41 @@ func UserdataDir(nsRoot string) string {
|
||||
return filepath.Join(nsRoot, "userdata")
|
||||
}
|
||||
|
||||
// UserdataSkeleton is the standard subtree created on every storage path (relative to UserdataDir).
|
||||
// ImportDirName is the single import (drop-zone) subtree name under a userdata root.
|
||||
const ImportDirName = "import"
|
||||
|
||||
// ImportDir returns the CANONICAL drop-zone root under a namespace root (R-75).
|
||||
//
|
||||
// Unlike every other userdata dir, this one is drive-INDEPENDENT: the caller resolves it against the
|
||||
// SYSTEM drive's namespace root, never against the app's own HDD_PATH, so a multi-drive box has
|
||||
// exactly ONE import tree. That is the whole point. Each drop-zone app has exactly one ingest bind,
|
||||
// so a per-drive import/ would put a folder that LOOKS like a drop-zone on every drive while only
|
||||
// one of them does anything — and because import paths are `class: excluded`, files stranded in a
|
||||
// dead one are never backed up either.
|
||||
//
|
||||
// It deliberately stays INSIDE the userdata tree, so the 2775/setgid/GID-1000 convention, the
|
||||
// FileBrowser mount and the ownership rules all apply to it unchanged.
|
||||
func ImportDir(nsRoot string) string {
|
||||
return filepath.Join(UserdataDir(nsRoot), ImportDirName)
|
||||
}
|
||||
|
||||
// UserdataSkeletonCarry is the explicit NON-DERIVED carry-list: every entry the v0.171.0 hardcoded
|
||||
// skeleton created, retained verbatim and forever.
|
||||
//
|
||||
// It exists so the catalog-derived skeleton (R-75) can only ever ADD. That makes the zero-removals
|
||||
// invariant true BY CONSTRUCTION rather than by review, and it is not hypothetical:
|
||||
//
|
||||
// - `documents` is implied by NO catalog app (SPIKE P0(a)) yet exists on both demo boxes and is
|
||||
// customer-visible — it may hold customer files. Derivation alone would drop it.
|
||||
// - `import/paperless` and `import/calibre` moved to the canonical system-drive root in R-75, so
|
||||
// derivation no longer implies them under a data drive either. The pre-existing ones stay put;
|
||||
// nothing in this arc deletes a directory.
|
||||
//
|
||||
// It doubles as the fresh-box floor: on a box whose catalog has not synced yet the derived set is
|
||||
// empty, and the customer still gets the full standard tree instead of a nearly-empty one.
|
||||
//
|
||||
// ASCII, no spaces (flows through ${} interpolation, shell, and the rsync merge walk).
|
||||
func UserdataSkeleton() []string {
|
||||
func UserdataSkeletonCarry() []string {
|
||||
return []string{
|
||||
"media", "media/movies", "media/tv", "media/music", "media/audiobooks",
|
||||
"media/books", "media/comics", "media/photos",
|
||||
@@ -41,6 +76,41 @@ func UserdataSkeleton() []string {
|
||||
}
|
||||
}
|
||||
|
||||
// BuildUserdataSkeleton merges the catalog-derived dirs with the carry-list into the final, SORTED
|
||||
// set. Each entry is expanded to its ancestor chain ("media/podcasts" implies "media"), deduped, and
|
||||
// sorted.
|
||||
//
|
||||
// SORTING IS A HARD REQUIREMENT, not tidiness. The FileBrowser config is regenerated from this set
|
||||
// and fbNeedsRecreate force-recreates the container on ANY byte difference. Go randomises map
|
||||
// iteration, and the spike measured the naive map-order derivation producing 20 DISTINCT outputs from
|
||||
// 20 identical runs (SPIKE P6) — which across SyncFileBrowserMounts' ~14 call sites is a fleet-wide
|
||||
// FileBrowser restart loop. TestSkeletonDeterminism pins this.
|
||||
func BuildUserdataSkeleton(derived []string) []string {
|
||||
set := make(map[string]bool, len(derived)+16)
|
||||
addChain := func(rel string) {
|
||||
rel = path.Clean(strings.TrimPrefix(filepath.ToSlash(rel), "/"))
|
||||
if rel == "" || rel == "." || rel == ".." || strings.HasPrefix(rel, "../") {
|
||||
return // never let a traversal or an empty entry become a directory to create
|
||||
}
|
||||
parts := strings.Split(rel, "/")
|
||||
for i := range parts {
|
||||
set[strings.Join(parts[:i+1], "/")] = true
|
||||
}
|
||||
}
|
||||
for _, d := range UserdataSkeletonCarry() {
|
||||
addChain(d)
|
||||
}
|
||||
for _, d := range derived {
|
||||
addChain(d)
|
||||
}
|
||||
out := make([]string, 0, len(set))
|
||||
for d := range set { // map order is RANDOM — the sort below is what makes this deterministic
|
||||
out = append(out, d)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// EnsureDirOwned creates path (idempotent) and enforces the convention: mode 2775 via an explicit
|
||||
// Chmod incl. setgid (MkdirAll cannot) + group = gid. Setting an arbitrary group needs CAP_CHOWN —
|
||||
// the in-guest controller runs as root, so this succeeds in production. Returns the first hard error.
|
||||
@@ -60,7 +130,11 @@ func EnsureUserdataDir(path string) error { return EnsureDirOwned(path, SharedCo
|
||||
// EnsureUserdataSkeleton creates the full userdata tree under a namespace root with the convention.
|
||||
// It creates ALL dirs even if one errors (so a single chown/chmod hiccup doesn't truncate the tree),
|
||||
// returning the first error seen for the caller to log.
|
||||
func EnsureUserdataSkeleton(nsRoot string) error {
|
||||
//
|
||||
// dirs is the merged, sorted set from BuildUserdataSkeleton. This function only ever CREATES: there
|
||||
// is no removal path here or anywhere in R-75, so a directory the current catalog no longer implies
|
||||
// simply stays where it is (Scenario D).
|
||||
func EnsureUserdataSkeleton(nsRoot string, dirs []string) error {
|
||||
base := UserdataDir(nsRoot)
|
||||
var firstErr error
|
||||
rec := func(e error) {
|
||||
@@ -69,7 +143,7 @@ func EnsureUserdataSkeleton(nsRoot string) error {
|
||||
}
|
||||
}
|
||||
rec(EnsureUserdataDir(base))
|
||||
for _, sub := range UserdataSkeleton() {
|
||||
for _, sub := range dirs {
|
||||
rec(EnsureUserdataDir(filepath.Join(base, sub)))
|
||||
}
|
||||
return firstErr
|
||||
|
||||
Reference in New Issue
Block a user