# REUSE.md — felhom-controller > Before writing new code, check here. Canonical helpers, patterns to copy, traps to avoid. > Maintenance: update in the SAME commit that adds/changes/deprecates a shared helper. > Entries cite file + symbol. Line numbers are landmarks only — reconfirm before editing. ## 1. Canonical helpers (MUST reuse — do not reinvent) ### Paths & namespaces (felhom-data layout) | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `NamespaceRoot` | controller/internal/appbackup/paths.go | `(drivePath string, inGuestDrive bool) string` | Resolve felhom-data root for a drive | `inGuestDrive=true` returns path AS-IS (Model A: guest mount IS the ns root); false appends `felhom-data`. Never double-nest | | `PrimaryBackupPath` / `RecoveryUnitPath` / `RecoveryUnitComposePath` / `RecoveryUnitManifestPath` | controller/internal/appbackup/paths.go | `(nsRoot[, stackName]) string` | All backup dir layout | Take the NAMESPACE ROOT, not a bare drive path. Since v0.229.0 the last three are thin wrappers over the unit-directory-relative primitives below and return byte-identical strings (`TestR102_PathWrappersAreByteIdenticalToToday`) | | `UnitComposeDir` / `UnitManifestFile` / `UnitDBDumpDir` / `UnitVolumeDumpDir` (R-102, v0.229.0) | controller/internal/appbackup/paths.go (re-exported by `backup/appbackup_bridge.go`) | `(unitDir string) string` | naming a leg inside a recovery unit that is NOT under `backups/primary/` | **These take the recovery-unit DIRECTORY itself.** The `(nsRoot, stackName)` form joins a hard-coded `primary`, which was the MECHANISM of R-102: the Tier-2 mirror at `/backups/secondary//recovery-unit/` was written nightly and readable by nothing. Use these whenever the unit's location is an argument; do NOT add a second copy of a layout join | | `AppDBDumpPath` / `AppVolumeDumpPath` / `AppDataDir` | controller/internal/appbackup/paths.go | `(nsRoot, stackName) string` | Per-app dump/data dirs | Same nsRoot contract. `AppDataDir`'s final segment is the app's real appdata dir NAME — NOT always the stack name (paperless-ngx → `paperless`); resolve via `AppDataDirNames` first (F-S2/F-S3) | | `AppDataDirNames` / `AppDataBindsPresent` | controller/internal/appbackup/paths.go | `(hddPath, stackName string, hddMounts []string) []string` / `(hddPath, hddMounts) bool` | Resolve the real `appdata/` dir(s) from compose `${HDD_PATH}` binds (F-S2/F-S3) | `hddMounts` = ParseComposeHDDMounts shape. Deduped+sorted; falls back to `[stackName]` when no appdata bind. Tier-2 (`backup.Manager.tier2AppDataName`) refuses N>1; migrate (`stacks.Manager.ResolveAppDataDirNames`) loops N. `BindsPresent` drives the WARN-on-missing-declared-dir | | `UserdataDir` / `ImportDir` / `EnsureUserdataSkeleton` / `EnsureDirOwned` | controller/internal/appbackup/userdata.go | `(nsRoot)` / `(nsRoot)` / `(nsRoot, dirs []string)` / `(path, gid int)` | userdata/ tree w/ 2775 setgid gid-1000 convention. **R-75:** `ImportDir` is the CANONICAL drop-zone (`/userdata/import`) and callers MUST resolve it against the SYSTEM namespace, never an app's HDD_PATH — use `stacks.Manager.GetImportRoot()`. `EnsureUserdataSkeleton` now takes the dir set: build it with `BuildUserdataSkeleton(DeriveUserdataDirs(stacksDir))`, or via `Manager.EnsureUserdataSkeleton` / `web.Server.ensureUserdataSkeleton`. | Linux-only chown via build-tag twin userdata_linux.go. **The set MUST stay sorted** — `fbNeedsRecreate` force-recreates FileBrowser on any byte diff and the naive map-order derivation measured 20/20 distinct (SPIKE P6). `UserdataSkeletonCarry()` is the old hardcoded list, retained forever so derivation can only ADD (zero removals). | | `BuildUserdataSkeleton` / `UserdataSkeletonCarry` / `DeriveUserdataDirs` | appbackup/userdata.go, stacks/skeleton_derive.go | `([]string)` / `()` / `(stacksDir)` | catalog-derived userdata skeleton (R-75) | Derives `${USERDATA_PATH}` binds only — `${IMPORT_PATH}` is NOT part of a drive skeleton (one root, system drive, `Manager.EnsureImportRoot`). Do NOT wire the catalog sync to `SyncFileBrowserMounts`. | | `appbackup.ValidateRelPath` / `ValidRoot` | controller/internal/appbackup/classify.go | `(root, path)` / `(root)` | THE single path-safety refusal set for every `${VAR}`-relative catalog path | Shared by `backup:` and `data_paths:`. **Do not write a second path validator.** | | `stacks.ValidateDataPaths` | controller/internal/stacks/datapaths.go | `(entries, binds, appName, logger)` | `data_paths:` annotation validation | ASYMMETRIC on purpose (Fork-3): malformed PATH ⇒ whole-block reject (data handling, `backup:` precedent); unknown ROLE ⇒ fails OPEN, one WARN (presentation, `Lifecycle` precedent). | | `web.fileBrowserLink` / `importFolderLink` | controller/internal/web/filebrowser_link.go | `(domain, sourceName, relPath)` | FileBrowser Quantum deep link | Template read out of the shipped router (SPIKE P2). **`url.PathEscape` per segment — NEVER `QueryEscape`** (space→`+` is a literal plus in a path). Let `html/template` do the attribute escaping; do not pre-escape. | | `HumanizeBytes` | controller/internal/appbackup/appdata.go | `(b int64) string` | Human byte sizes | Exported canonical; private clones exist (§6) | | `stablePathForName` / `agentWhere` | controller/internal/web/intermediary.go | `(name/registeredPath) string` | Map registry stable path `/mnt/felhom-drives/` ↔ raw agent mount | Registry stores STABLE path; agent ops take the RAW mount — always convert | | `offsiteRestoreRootFor` | controller/internal/backup/offbox_verify_copies.go | `(drivePath string) string` | THE only place `backups/offsite-restore` is spelled | `offboxRestoreScratchDir` builds on it — the listing/delete surface MUST resolve byte-identical paths to what the restore wrote. Do not re-hardcode the segments (they were open-coded in 3 places before v0.147.0) | | `ProtectedHDDPaths` | controller/internal/stacks/delete.go | `(hddPath string) map[string]bool` | Never-delete set (root, appdata, backups, media, kept, legacy felhom-data) | Consult before ANY recursive delete under a drive | | `stacks.OldAppDataPaths` / `Manager.ListKept` / `KeepAside` / `DeleteKept` / `FindKept` | controller/internal/stacks/kept.go | `(composePath, hdd)` / `(drives)` / … | Kept data (`09` §3 decision 36): what counts as an app's old data (ONLY `/appdata/…` binds), the list, start-fresh, the household's delete | **An action names a kept item by path only through `FindKept`** — `DeleteKept` refuses anything not listed. `KeepAside` is a rename on one drive; never copy, never `RemoveAll` in a rollback (`removeEmptyDirs`) | | `stacks.RunAfterInstall` / `expandAfterInstall` / `web.defaultLoginInEffect` (v0.279.0, decision 45) | controller/internal/stacks/after_install.go · controller/internal/web/known_login.go | `(name, wait)` / `(cmd, allowed, env)` / `(meta, cfg, installed)` | A fresh install replaces a known default login; the page says when a default is still in effect | **Only from the deploy-done hook** — never after a restore/kept load (R-694). A `success:` marker is required (exit 0 lies). Never log the expanded command | | `stacks.OpenSetupGate` / `SetupGateTick` / `SetupGateHost` · `web.ServeGateAuth` / `ServeGateStart` (v0.280.0, decision 46) | controller/internal/stacks/setup_gate.go · controller/internal/web/setup_gate.go | `(name, by)` / `()` / `(host)` · handlers | The setup gate: a `setup_gate: true` install is closed to everyone but the household until its probe or the household's press opens it | **Write the gate BEFORE the first start** (spike F2). Open = record first, then remove the file. Never widen the dashboard cookie — the handshake mints a host-bound one-use token | | `stacks.OpenSignupWindow` / `SignupBlocked` / `SetupGateProbe` · `web.ServeSignupClosed` (v0.281.0, decision 47) | controller/internal/stacks/signup_block.go · controller/internal/web/setup_gate.go | `(name)` | Sign-up closed at the app's own address once the gate opens; the household's 15-minute window; the press asks the probe | **The block goes up BEFORE the gate comes down** (a failed write keeps the gate closed). Never on an app this box did not gate | | `family.Store` · `stacks.FamilyGateHost` / `familyGateTick` / `FamilyExceptRegexp` · `web.ServeFamilyGateAuth` / `ServeFamilyStart` / `ServeFamilyLogin` / `ServeFamilyLogout` (v0.287.0, decisions 63/64) | controller/internal/family/family.go · controller/internal/stacks/family_gate.go · controller/internal/web/family_gate.go | store `Add/Reset/Remove/Verify/NewSession/Valid/EndSession` · `(host)` / `()` / `(prefix)` · handlers | The PERMANENT family gate: family members with their own logins in front of a `family_gate: true` app | **A family cookie never opens the dashboard** (RequireAuth reads only `felhom_session`); the app cookie names a STORE session, so the store is asked on every request (reset/remove/logout end access at once); **every exception goes through `FamilyExceptRegexp`** (anchored — never a hand-written PathPrefix); door written before the first start, like the setup gate. `familyStoreOverride` is the test seam. Fifth atomic-write helper (family.json, fsync) — see §6 | | `stacks.OpenInstallHold` / `installHoldTick` (v0.284.0, R-741) | controller/internal/stacks/install_hold.go | `(name, by)` / `()` | An `after_install` app held behind the setup gate's door until its known login is replaced | **Written before the first start**, like the gate; opens on `after_install` success or the household's "I changed it"; the door (`SetupGateHost`) reads holds first | | `stacks.RetainImagesAfterUpdate` / `RetainImagesAfterRemove` / `RunImageRetentionOnce` (→ `(deleted, done)` since v0.294.0) / `RunImageRetentionOnceUntilDone` · seam `imageDocker` (v0.284.0, decision 53) | controller/internal/stacks/image_retention.go | `(name, previous)` / `(name, repos)` / `()` | Deletes an app's images older than its running + previous one | **The keep set is box-wide and read at delete time** (containers, installed composes, installed/previous records); exact id, never forced or pruned; skipped while any update runs AND while any image-pulling compose command runs (R-863, `dockerexec.TryImageCleanup`); the one-time pass writes its marker only after a pass that RAN; tests use the `imageDocker` seam, never Docker | | `dockerexec.BeginImageWork(args)` / `TryImageCleanup()` / `ImagePulling(args)` (v0.294.0, R-863) | controller/internal/dockerexec/imagework.go | `defer dockerexec.BeginImageWork(args)()` | **Every NEW place that runs `compose up/pull/create/run` must hold it** (the stacks compose helpers, appexport's restore and the FileBrowser sync already do) | An image pulled by compose is named by no container until compose creates one; a clean-up pass in that window deleted it (measured, R-863). The pass takes the lock exclusively without waiting; pulling commands wait for a running pass (seconds) | | `nightchain.Ledger` / `CatchUp` / `ResumeWatch` / `ComputeBanner` (v0.295.0, R-871) | controller/internal/nightchain/ | `Open(path, now)`, `MarkEnded(leg, t)`, `Missed(now, W, loc)`, `Evaluate(ctx, why)` | "was a night missed?" and making it up; the missed-backup banner's rule | The ledger is an ATTEMPT record (ran to its end) — never evidence a backup exists; a NEW nightly backup leg must be wrapped by `withLeg` in main.go and listed in `nightchain.Order`, or the catch-up never makes it up; never add the update leg to `catchUpLegs` | | `scheduler.DailyLateLimit` (v0.295.0) | controller/internal/scheduler/scheduler.go | — | a daily timer firing > 60 min late (host suspend) is skipped | do not "fix" a late fire by running it: the app-update leg would run at noon | | `stacks.RetainControllerImages(ControllerImageRecord)` · `selfupdate.UpdateState.RecordedPrevious` (v0.285.0, decision 56) | controller/internal/stacks/controller_image_retention.go | `({Repo, Running, Previous})` | Deletes controller images older than the running + previous one | The previous comes from the SWAP RECORD (success onto the running version), version order only as the fallback; versions above the running one and non-version tags are kept; skipped while the controller swaps itself; same `imageDocker` seam | | `stacks.CloseSignupNow` / `CloseSignupOffered` / `applyNativeLock` (v0.282.0, decisions 47/49) | controller/internal/stacks/after_setup.go | `(name)` | The app's own sign-up switch after the setup; "close sign-up now" for an app installed before the rule | **Check the installed compose reads the variable** (an old install carries the old compose until its next update) — never record a lock that is not there. One run per app at a time (`nativeLockBusy`) | | `backup.judgeCopy` / `HollowCopies` / `SetHollowCopyNotify` (Part D, v0.279.0) | controller/internal/backup/hollow_watch.go | `(app, tier, unitDir)` | A RUNNING app whose newest copy holds no data → operator digest once/day + page sentence | Uses `unitCarriesData` (the manifest, never size); a stopped held app is never flagged | | `web.nightChain` (R-705, v0.279.0) | controller/internal/web/night_chain.go | `POST /api/debug/backup/night-chain` | The night's four legs now, in order | Refuses while any op/update/chain runs; the leg uses `RunUpdateLegNow` | | `Router.dropLeftoverHold` + `settings.ClearUpdateHold` (R-704, v0.278.0) | controller/internal/api/router.go · controller/internal/settings/settings.go | `(name, why)` / `(stack) (bool, error)` | A new install (plain or "use my kept data") and a removal clear the update / crash-loop hold of the app's install | **A hold belongs to an INSTALL; the name is all the next install shares with it.** Never clears an R-379 restore hold (operator-only) | | `backup.KeptBestCopy` / `KeptDBCopy` / `KeptOffsiteCopies` / `KeptCopyAt` / `LoadKeptApp` / `LoadKeptOffsite` / `KeptCopyKey` | controller/internal/backup/kept_load.go | `(ctx, app, drive)` / `(app, drive)` / `(ctx, apps)` / `(unitDir, drive, tier)` / … | Which copy can load kept files (local tiers + since v0.277.0 the off-site copy, R-691 (2)), the load as a restore op, the copy's name for the page | A copy counts only with data (DB dump or volume tar) AND its app.yaml `HDD_PATH` = this drive. Installed apps are never offered. **The off-site copy is judged only after its unit is downloaded** — `LoadKeptOffsite` refuses an unversioned unit (`07` §6.6) BEFORE `prepare` (a dated folder's move-back); ask the repository once per page (`KeptOffsiteCopies`), never per row. Both pages name a copy through `KeptCopyKey` | ### Subprocess + timeout + exit-code discipline | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `Manager.composeExec` / `composeExecCustomEnv` | controller/internal/stacks/manager.go | `(dir string, [env,] args...) (string, error)` | ALL docker-compose invocations | Logs env KEYS only (secrets safety), truncates output to 500, extracts exit code; `up` triggers userdata pre-create belt. NO timeout — see §3 | | `rsyncCopy` | controller/internal/stacks/migrate.go | `(ctx, src, dst, onBytes)` | Additive copy (migration/moves) | `-a --checksum`, NEVER `--delete`; progress2 byte callback; ctx timeout | | `rsyncVerify` | controller/internal/stacks/migrate.go | `(ctx, src, dst) error` | Post-copy verification | Dry-run `-ani`; fails on any pending content transfer; attr-only lines ignored | | `walkMerge` | controller/internal/stacks/migrate.go | `(lg, srcNS, dstNS, skip, assertOnly, onBytes)` | Collision-safe userdata merge | Renames to lowest-free sibling on content mismatch; additive | | `runCommand` / `runCommandStdin` | controller/internal/selfupdate/updater.go | `(name, args...) (string, error)` | docker CLI in updater | stdin variant for `docker login --password-stdin` (no secret in argv); package VARS since v0.112.0 — override in tests (fakeRunner in registry_anon_test.go) | | `parseWWWAuthenticate` + `fetchAnonymousToken` | controller/internal/selfupdate/updater.go | Bearer-challenge parse + anonymous Docker v2 token | Any credential-free registry API access | realm comes FROM THE HEADER (never hardcode a token URL); denial = errAnonymousDenied, never "credentials missing" | | `Syncer.runGit` / `runGitInDir` | controller/internal/sync/sync.go | `(args...) error` | git CLI ops | Credentials masked in logs via `maskRepoURL` | ### Error kinds — never branch on a customer-facing sentence (R-553) | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `util.KindErrorf` / `util.KindError` | controller/internal/util/errkind.go | `(kind error, format string, a ...interface{}) error` | ANY refusal a caller must tell apart: build the message exactly as `fmt.Errorf` would AND carry a sentinel for `errors.Is` | The message bytes are unchanged (pinned by tests); never `fmt.Errorf("%w: …")`, which would prepend the sentinel's own text to the customer's sentence | | `stacks.ErrAlreadyDeployed` / `ErrRequiredField` / `ErrPathMissing` / `ErrNotEnoughMemory` | controller/internal/stacks/deploy_errors.go | sentinels | the API's deploy status code (`api.deployStatusFor`) | 409 / 400 / 400 / 400. Do NOT add a text signature beside them | | `backup.ErrOffsiteQuota` | controller/internal/backup/offbox.go | sentinel | `ClassifyOffsiteFailure` telling a quota over-run apart | The other arms of that switch stay TEXT matches on purpose — they are restic's and ssh's own English output, which we neither write nor translate | | `monitor.WarnKind*` + `HealthReport.addWarning` / `WarningKindAt` | controller/internal/monitor/healthcheck.go | `(text, kind string)` | a health warning whose PLACEMENT the dashboard decides | Internal only: `internal/report/builder.go` copies Status/Issues/Warnings, so kinds never reach the hub (pinned) | | `settings.OffboxTarget.LastWarningKind` + `backup.OffboxWarnNoAppsSelected` | controller/internal/settings/settings.go | persisted string | the Távoli mentés page's stale-note substitution | Written and cleared with `LastWarning`; the text fallback in `offboxWarningDisplay` is LEGACY only (kind == "") and is removed when R-570 closes | ### HTTP/JSON envelopes + flash messages | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `writeJSON` | controller/internal/api/router.go | `(w, status, v)` | REST API (`/api/*` router) responses | Pair with `apiResponse{OK,Data,Error}` envelope | | `writeDiskJSON` | controller/internal/web/agent_disk_handlers.go | `(w, status, ok, errMsg, data)` | Storage/disk web-API responses | The `{ok,error,data}` envelope the storage JS expects | | `jsonResponse` / `jsonError` | controller/internal/web/handler_export.go | `(w, v)` / `(w, msg, code)` | Export/import API | Third envelope shape — keep within export surface | | `limitBody` | controller/internal/api/router.go | `(w, req)` | Bound request bodies (1MB) | Apply before decode on any new POST | | `offboxRedirect` | controller/internal/web/offbox_handlers.go | `(w, r, msg string, isErr bool)` | Flash-message redirects | Flash = `?flash=` / `?flash_error=` query params, read by page handlers | | `offboxRedirectTo` | controller/internal/web/offbox_handlers.go | `(w, r, page, msg string, isErr bool)` | Same, to an EXPLICIT page | **TRAP (fixed v0.154.0): the separator is chosen, not `"?"`.** Targets may already carry a query — the R-48 wizard is `/backups/restore/app?name=` — and a hardcoded `"?"` buries the flash inside the previous parameter's value | | `restoreOpInFlight` + `hasRecentRestoreResult` | controller/internal/web/restore_wizard.go | `(backup.RestoreOpStatus) bool` / `(st, app, now) bool` | THE "is a restore running / did one just finish" display reads | **TRAP (v0.154.0 shipped this bug): `Manager` has TWO running flags.** `IsRunning()` reads the CONCURRENCY flag, acquired inside the goroutine — and `RestoreOffboxScratch` never acquires it, so it is false for the whole verification restore. Display must read `RestoreStatus().Running` (set synchronously by `BeginRestoreOp`). Read the status ONCE per render or the strip and the suppression can disagree. `hasRecentRestoreResult` is app-bound and window-bounded — a process-wide result must not light another app's „Eredmény" | | `restoreWizardPath` / `deriveWizardStep` / `resolveWizardApp` | controller/internal/web/restore_wizard.go | `(app) string` / `(restoreWizardInput) restoreWizardView` / `([]OffboxAppRow, name) *OffboxAppRow` | R-48 offsite restore wizard: URL builder + the PURE step/unlock derivation + the app-resolution refusals | The step is **never** taken from the request. Precedence is load-bearing: op-running outranks a stale `?full_prep=`, else a commit button reappears mid-restore. Truth table + red-proof: `restore_wizard_test.go`. Adding a form here that posts anywhere new breaks `TestRestoreWizard_NoNewMutationEndpoints` **by design** — R-48 adds no mutation surface | | `restoreOpBlocked` | controller/internal/web/restore_wizard.go | `() (msg string, blocked bool)` | THE refusal gate before starting ANY restore | **Use this, never a bare `IsRunning()`.** It reads BOTH flags: `RestoreStatus().Running` (set synchronously by `BeginRestoreOp`, true for the whole off-box restore) and `IsRunning()` (the concurrency flag, the only one the nightly backup holds). **R-351: all seven handlers read only `IsRunning()`, which the goroutine acquires AFTER the handler returns — a second press started a second run and was told „…elindult".** Returns the Hungarian refusal, which names the running app and a route | **R-360 (v0.226.0): `offboxVerifyCopyDeleteHandler` was the LAST holdout and its doc comment claimed it already did this — the sentence is why nobody looked. `IsRunning()` is FALSE for the whole of a verification restore, so the copy a restore was writing into could be deleted from the UI (observed live 2026-08-21 22:35). No app-name comparison: refusing during ANY restore is stronger and uniform.** | `Manager.RestoreFromRecoveryUnitAt` / `RestoreFromRecoveryUnit` + `UnitRestoreResult` (R-353 v0.226.0; R-102 v0.229.0) | controller/internal/backup/restore_unit.go | `(stack, unitDir string) (UnitRestoreResult, error)` / `(stack string) (…)` | THE local recovery-unit restore, and the facts its surface must state | **Returns a RESULT, not just an error** — volumes replayed, DBs replayed, and what the manifest LISTED. The Manifest* counts are load-bearing: zero-replayed has two causes (the backup held no data / the backup listed data that did not come back) and they are opposite news. Pair it with `unitRestoreOutcomeMsg`; do NOT write a new sentence. **A claim about the APP is forbidden on this path** — it has no `SafetyDump` discriminator, unlike the off-site twin (CONTEXT.md ruling, 07-backup-architecture §6.3). `restoreDockerVolumes` now returns `(int, error)`; `restoreDockerVolumesFrom` is unchanged and still the shared implementation. **R-102: `…At` holds the whole body and takes the unit DIRECTORY; the one-argument form is the thin caller naming the primary unit.** THE SOURCE MOVES; THE DESTINATION DOES NOT — `GetAppDrivePath` still resolves where the data lands. The R-47 mutation order, the unit-over-guest secret precedence, the fail-closed data-key gate and the no-unit `RestoreApp` fallback are all pinned and unchanged. The volume leg goes through the R-354 `volumeReplayFrom` seam so the source directory is assertable without Docker | | `Manager.RestoreTier2Unit` + `Tier2Coverage.CanRestoreUnit` / `Tier2CopyDate` (R-102/R-103, v0.229.0) | controller/internal/backup/tier2_restore.go | `(stack string) (UnitRestoreResult, error)` / `() bool` / `() (string, bool)` | THE full restore from the SECOND DRIVE's mirror, and the predicate that gates it | **Do NOT widen `CanRestore()`** — it answers only "can the additive file restore run?" and one predicate answering two questions is R-356. Fail-closed: `tier2UnitIsOpenable` requires a parseable manifest, because a directory is not a package. The single-writer flag is taken INSIDE `RestoreFromRecoveryUnitAt` — a second `acquireRunning()` here would refuse the restore it guards. `Tier2CopyDate` prefers the SUCCESS anchor over the attempt clock (R-101) and reports which it returned | | `unitRestoreOutcomeMsg` + its three message constants (R-353, v0.226.0) | controller/internal/web/handlers.go | `(app string, res backup.UnitRestoreResult) string` | THE customer sentence for a completed LOCAL restore | Twin of `reconstituteOutcomeMsg`; copy its SHAPE (clauses earned by having done the thing, no filesystem path, base names only), never its text. The constants are named because `r353_unit_outcome_test.go` asserts them verbatim — a silent edit is how an honest message drifts back into a comforting one, which is the documented history of the sentence it replaces | | `offsiteNoSpaceMsgFmt` + `offsiteSizeUnknownMsg` (R-357, v0.226.0) | controller/internal/backup/offbox_restore.go | two consts | EVERY headroom refusal on the off-site restore surface | **All four gates share these** (prepare, scratch, place, and the destructive reconstitute). A customer meeting one wording on one path and a different one on another has to work out whether it is the same problem. **The reconstitute gate uses NO ×1.1 margin** — it copies a measured tree; `OffboxRestorePrepareFull`'s ×1.1 predicts a download. **Fail closed when either probe reads ≤ 0**: `free < need` with `need == 0` is FALSE, so an unmeasurable input sails through — a gate present and inert | | `Manager.OffboxFullScratchReady` + the scratch marker (R-358, v0.226.0) | controller/internal/backup/offbox_restore.go | `(stack) bool`; `.felhom-restore-complete.json` | THE gate for place-to-live and reconstitute | **It answers "did the run FINISH and was it FULL", not "are there files".** The old non-empty check passed a part-copy from a failed restic run, and the old doc comment ("PlaceOffsiteRestore re-validates per-path completeness") is what made it look adequate — that call stats top-level placements, not files. Marker written 0600 atomically AFTER restic returns nil; stale one cleared BEFORE it starts; both orders pinned by an AST test because `resticStep` is not a seam. Anything else — absent, unreadable, wrong schema, `full:false` — is NOT ready, with a WARN naming which. **Unit-only and full restores write the SAME directory**, so `full` is the only separator (R-396) | | `JudgeRestoredUnit` + `UnitProofResult` (R-87, v0.231.0) | controller/internal/backup/r403_hollow.go | `(unitDir string) UnitProofResult` | THE question "does this app's backup contain what THIS APP should have" | **The rule has TWO parts and part 1 alone is the trap:** "everything declared is present" passes a HOLLOW unit, because a hollow unit declares nothing — the exact shape it exists to catch. Part 2 is the expectation, and it comes from the unit's **own** captured compose file (`UnitComposeDir(unitDir)` + the compose filename), NEVER from live Docker (`GetDockerVolumes` describes the running app; the snapshot may predate it). Database half is `DBServiceNames`, the same discriminator `RestoreFromRecoveryUnit` uses, so this cannot disagree with the restore path about what an app is. **The volume half is an EXISTENCE check, not a name match** — `ResolveDockerVolumeNames` derives the project from the compose file's parent dir, which inside a unit is the literal string `compose`. **THREE outcomes:** pass / fail / **cannot judge**, and the third is never collapsed. Size is never consulted (`TestR87_SizeIsNeverConsulted`) | | `Manager.ProveOffboxUnit` + `ProofResult` (R-87, v0.231.0) | controller/internal/backup/offbox_proof.go | `(ctx) ProofResult` | THE nightly off-site content proof — one app, its newest snapshot, restored read-only and judged | **It NEVER writes to the repository and that is asserted on the ARGV:** `--no-lock`, no `unlockStale`, and `m.runner()` rather than `resticStep` so the `unlock --remove-all` escalation is unreachable. **It takes `acquireRunning` ITSELF** because `RestoreOffboxScratch` does not (R-408) — do not remove that. **Due-ness is per SNAPSHOT** (`ProvedSnapshots[stack]`), never a timestamp: a timestamp re-proves the same snapshot forever AND breaks the rotation. **Its scratch is a SEPARATE root** (`offsiteProofRootFor`, `backups/offsite-proof`) — sharing the customer's `offsite-restore` root would let a nightly job delete a copy the customer is looking at. A skip, a missing snapshot and a restore error reach NO verdict and do not advance due-ness | | `unitOnlyHeadroom` + `offboxScratchDirIn` (R-87, v0.231.0) | controller/internal/backup/offbox_restore.go | `(free int64) error`; `(stack, rootFor)` | The unit-only free-space gate and the drive-preference resolver, **shared** by the customer restore and the nightly proof | Extracted rather than forked so the two paths cannot drift on the parts that must not differ — the floor, the Hungarian refusal (`offsiteNoSpaceMsgFmt`), the network-storage refusal and the R-252 wording. `rootFor` is the ONLY difference between the two scratch paths. **Fail-closed on an unmeasurable probe:** `offboxFree` returns 0 when it cannot read, and `0 < floor` refuses — the inverse of the R-357 shape where a gate went inert | | `Manager.CheckOffboxIntegrity` + `IntegrityResult` + `IntegrityDue` (R-359, v0.227.0) | controller/internal/backup/offbox_integrity.go | `(ctx) IntegrityResult`; `(now) (due bool, last time.Time)` | THE off-site integrity check, and the only place `restic check` is run | **It TAKES `acquireRunning` and SKIPS rather than waits — never remove that guard.** `resticStep` escalates to `unlock --remove-all` on a lock error and is only safe because every caller holds the single-flight mutex; a check without it can strip a LIVE prune's lock. **THREE outcomes, not two:** `Skipped`, `Unreachable` and failed are different facts — a timeout is unreachable, NEVER damage, and only a failure notifies. A skip and an unreachable repo do **not** advance due-ness; a failure does. **DUE-NESS, NOT A WEEKDAY** (R-341). **`looksLikeRepositoryDamage` matches PHRASES, not words** — bare `pack `/`tree `/`snapshot ` appear in restic's ordinary progress output and made a healthy run look corrupt | | `runOffsiteIntegrityCheck` + `integrityFailedMsg` / `integrityOKMsg` (R-397, v0.227.0) | controller/cmd/controller/main.go | `(ctx, mgr, notifier, logger, force) backup.IntegrityResult` | THE one caller of the check — the scheduled job AND the debug button both go through it | ONE function so the hand-run cannot drift from the scheduled one; `force` skips due-ness and **nothing else**. Wired to `NotifyIntegrityOK` / `NotifyIntegrityFailed`, which existed with no caller since the notifier did (sixth built-but-never-wired instance). **`ok` is severity `info` and therefore mails NOBODY by design** — a weekly success e-mail is how alerts stop being read. The customer gets a sentence; restic's output goes to the log truncated (R-379) | | **`SetOffboxRunner` / `m.runner()` — the restic exec seam, and it has ALWAYS existed** | controller/internal/backup/offbox.go (~L52, ~L386) | `offboxRunner func(ctx, env, args...) ([]byte, error)` | Driving ANY restic-backed path under test | **R-398 claimed there was no such seam and was WRONG — do not re-file it.** `resticStep` is not itself overridable, but the layer it calls is, and tests have driven restic paths through it since the off-site tier shipped. **Use this rather than adding a `resticStepFn`:** replacing `resticStep` would hide its `unlock --remove-all` escalation from exactly the assertions that must see it (R-359's lock-safety tests assert `unlock` never appears in any argv) | | `Manager.SetOffboxLatestSnapshotFn` (R-357, v0.226.0) | controller/internal/backup/offbox_restore.go | `(fn func(ctx, stack) (id string, paths []string, err error))` INIT/TEST-ONLY | Overriding the restic snapshot lookup in tests | Exists because R-357's gate could not otherwise be tested at the level that matters: reaching it requires getting past `offboxLatestSnapshot`, which shells to restic. **The assertion the seam enables is `StopStack` call count == 0** — a gate placed after the stop returns the right sentence and still takes the outage. Nil in production | | `CheckPlacement` + `PlacementMismatchMessage` | controller/internal/backup/offbox_placement.go | `(*RecoveryManifest, liveDrive, liveNS) PlacementCheck` / `(stack, PlacementCheck) string` | Comparing where a backup SAYS the data lived against where a restore is about to write | Pure and total — nil/empty/blank manifest all give the same honest "not known, no mismatch". **An UNKNOWN is never a mismatch** (refusing on an absence strands every pre-field unit). Compares the DRIVE only (the namespace root is derived from it), Cleaned, so a trailing slash is not a difference. The message names BOTH values on purpose | | `RecordedUnitForStack` + `RecordedAddress` | controller/internal/backup/offbox_placement.go | `(stack) (RecordedPlacement, RecordedAddress, bool)` | Reading back the address + data folder a backup recorded, for a reinstall prefill | Local file reads over every readable namespace root — **no network, no restic, no restore**; it exists for the NOT-INSTALLED case where `GetStackHDDPath` is `""`. **`RecordedAddress.Known()` requires BOTH halves:** an absent `SUBDOMAIN` makes the live deploy path fall back to the CATALOG default (`stacks/deploy.go:88-90`), and offering that back as "what your backup says" is a fabricated fact | | **`Metadata.For(lang)` + `LocalizeStacks` / `LocalizeStackPtr` / `Stack.MetaFor` (R-560, v0.257.0)** | controller/internal/stacks/metadata_i18n.go | `(lang string) Metadata`; `([]Stack, lang) []Stack`; `(*Stack, lang) *Stack` | THE only way a page may reach catalog COPY — the app description, tagline, use cases, first steps, prerequisites, deploy-field labels and descriptions, optional-config text, integration labels, data-path labels and the initial-credentials note | Reading `stack.Meta.` off the manager renders HUNGARIAN to an English household, with no error and nothing wrong-looking on the page — `TestNoDirectMetaCopyReadOnPages` keeps the named, reasoned allow-list of every direct read in `internal/web`. **`For("hu")` is the parsed struct with `I18n` cleared** and nothing else (pinned over all 53 real catalog files), so a translation cannot change the Hungarian product. **Fallback is FIELD BY FIELD** — absent or blank English shows Hungarian, which is what makes a half-translated app shippable. **Lists replace WHOLE; every other list is matched by its own key** (`env_var`, option `value`, `match_group`, `target`, `path`) — position matching mistranslates silently the first time a field is inserted. **It never writes through:** the metadata is shared by concurrent requests, so an in-place merge leaks one household's language into another's page | | `Metadata.HasDeployField` | controller/internal/stacks/metadata.go | `(envVar string) bool` | "Does this app have somewhere to PUT a recorded value?" | **13 of 53 templates declare `HDD_PATH`; 40 do not** (measured 2026-08-21). For the 40 a recorded placement is a FACT TO STATE, never a value to write into a field that does not exist | | `redirectTier2` | controller/internal/web/tier2_config_handler.go | `(w, r, name, flash, flashErr)` | Tier2 page flash redirects | Same convention | | `validStackName` | controller/internal/web/validate.go | `(name string) bool` | Any stack name from a request | Single-segment, no `/ \ ..` — blocks path traversal into stacks/userdata | | `ValidateSegment` | controller/internal/appexport/validate.go | `(kind, s string) error` | Any attacker-controlled path segment (.fab manifest fields) | CTRL-001 guard; deliberately NOT for dotfile ConfigFiles | | `validateSubdomain` / `SubdomainInUse` | controller/internal/stacks/deploy.go | `(s)` / `(subdomain, excludeStack)` | Subdomain fields on deploy | — | ### Crash-safe journal / atomic writes | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `Manager.OffboxReportStatus` + `OffboxReportStatus.StatsKnown` (R-331, v0.225.0) | controller/internal/backup/offbox.go (~L1627) | `() *OffboxReportStatus` | THE off-site summary the hub reports on — snapshots, repo size, quota, last SUCCESSFUL run, declared state | **This object, not `report.BackupReport`, is where the hub gets live backup numbers.** `BackupReport`'s `SnapshotCount`/`RepoSizeMB`/`IntegrityOK` have had no producer since slice 8C; the hub rendered them anyway and told every operator every customer had `Snapshots 0` (R-331, measured on demo-hp over a repository holding 67). **`StatsKnown` must ride with the counts or they are unreadable**: zero is what an unread repository and a genuinely empty one both look like, and they are opposite news (R-225 measured that live). It is `omitempty`, so an older controller sends no key and a reader MUST degrade to "unknown", never to "empty". nil = nothing to say, EXCEPT the declared `needs_credential`/`awaiting_recovery_key` states, which report an object while disabled | | `backup.ErrOffboxSealedPackageHeld` + `IsOffboxSealedPackageHeld` + `sealedPackageHeld` + `OffboxAwaitingRecoveryKey` (R-241, v0.206.0) | controller/internal/backup/offbox.go | sentinel; `(error) bool`; `() bool`; `() bool` | **THE MINT GUARD** — a box never creates a repository key while the hub holds a sealed package for it | **The guard is a CONJUNCTION** (package held AND no key present). Widening it to "never mint" leaves a first-time box unable to start, waiting for a package that will never exist — pinned by `TestR241_ScenarioB_FirstTimeBoxStillMints`. **The refusal is a HOLDING state, not a failure:** `ApplyOffsiteTarget` catches the sentinel and still writes the transport, so `/recovery`'s synchronous tier-up (R-219) can bring the tier up the instant the key arrives; returning the error instead leaves `needsOffsiteCredential` true and the hub re-staging a consumed credential for ever. `OffboxAwaitingRecoveryKey` is **DERIVED, never stored** — and **`t.Enabled` is load-bearing in it**: a customer who switched off-site OFF is not awaiting anything (the Scenario-E carve-out `needsOffsiteCredential` makes two functions above; the first draft omitted it and an existing test caught it). A nil settings store reads as "no package held" — a transient read failure must never become a permanently-held tier | | `settings.HubEscrowKeySHA256` + `SetHubEscrowKeySHA256` / `GetHubEscrowKeySHA256`, and `OffsiteRecoveryOffer` **shape (c)** (R-241, v0.206.0) | controller/internal/settings/settings.go, controller/internal/backup/offbox.go | `(sha, checkedAt string) error` / `() (string, string)` | **THE DISCRIMINATOR the recovery screen asks** — does the hub hold a package for a key other than the one we use? | **The comparison was ALREADY computed on every ACK since SLICE 3 and persisted nowhere** — that is R-241's second half. Wire the recorder in `main.go`'s `EscrowAutoConfirmer` literal or shape (c) reads an empty hash for ever and the fix ships INERT (pinned by `TestMainWiresRecordEscrowKeyHash`). **§7.2 staleness, decided:** a KNOWN DIFFERENCE offers **however old the reading** — age is deliberately NOT gated on, because gating makes a box offline from the hub silently stop offering; an **ABSENT hash falls back to (a)/(b)** and does NOT offer, because `""` is the hub positively saying its package seals no key (legacy hash-less escrow), not an unknown. `CheckedAt` is for diagnosis, never a gate | | `backup.AbandonStatus` / `AbandonSweep` / `CancelAbandon` / `ClearAbandonPurgeIfConfirmed` / `ExtendAbandon` / `StopAbandon` + `AbandonGraceDays` (R-241, v0.206.0) | controller/internal/backup/offbox_abandon.go | see file | **The 14-day abandonment countdown** — the ONLY thing in the product that deletes a customer's off-site history | **BOTH HALVES OR NEITHER.** The set-aside store and the sealed package that protects it are two halves of one thing; removing only one leaves a package that opens nothing, or ciphertext nobody can decrypt. Not atomic across two machines, so it is a **two-phase commit**: delete the store, set `AbandonPurgeRequested`, and keep declaring it until the hub's ACK stops reporting a superseded package — the confirmation rides the SAME ACK as the request. **The countdown starts in `ResetOrphanedRepo`, NOT in the shared `resetOrphanedRepo`** — the helper is also the UNCLAIMED auto-reset, where nobody decided anything. **The recovery offer stays reachable for the whole grace** (a grace in which recovery is impossible is decorative). **Drive it with `SetOffboxClock`, never a shortened live timer** (§7.4). A transport failure leaves the countdown DUE so tomorrow retries; the operator levers REFUSE rather than no-op when nothing is running or the store is already gone | | `settings.SyncRecoveryOfferEpoch` / `PostponeRecoveryNoticeForEpoch` / `OptOutRecoveryRemindersForEpoch` + `web.recoveryBannerCookie` (R-241, v0.206.0) | controller/internal/settings/settings.go, controller/internal/web/recovery_handlers.go | `(offered bool, now) (RecoveryOfferView, error)` | **The offer EPOCH** — "once per entry into the offered state", not once ever | **Sync the epoch FIRST and UNCONDITIONALLY in `recoveryInterrupts`.** The first draft returned early when the offer was false, so the FALLING edge was never recorded, `RecoveryOfferActive` stayed true through a settled period, and the next entry counted as a continuation — **the exact defect the epoch exists to fix, reintroduced inside the fix**. Dismissals are recorded against the epoch they were made in, so a fresh entry resets them **by arithmetic**, with nothing to clear. **Three levers, three scopes, and NONE removes the entry point on `/backups/remote`:** the banner cookie is a browser SESSION cookie (no MaxAge — cleared on login) and persists nothing; the reminder opt-out is durable but silences the BANNER ONLY; "most nem" suppresses the full page only | | `atomicWrite` | controller/internal/backup/recovery_unit.go | `(path, data, perm) error` | Atomic file writes (backup pkg) | tmp+rename; no dir creation, no fallback | | `writeFileAtomic` | controller/internal/bootstrap/bootstrap.go | `(path, b) error` | controller.yaml writes from bootstrap | Always 0600 (holds local-api token + hub key) | | `writeConfig0600` | controller/internal/api/router.go | `(path, body) error` | config writes via API | ALWAYS chmods 0600 even pre-existing (F8); direct-write fallback on bind-mount EBUSY (non-atomic!) | | `atomicWriteFile` | controller/internal/setup/handlers.go | `(path, data, perm) error` | setup-wizard writes | Same bind-mount fallback caveat | | `Settings.save` (unexported) | controller/internal/settings/settings.go | via mutator methods only | ALL settings.json persistence | tmp+rename, then `.bak` last-known-good AFTER rename succeeds. Never write settings.json by hand | | `settings.Load` | controller/internal/settings/settings.go | `(path, logger) (*Settings, error)` | Startup load | Corruption recovery: `.bak` restore → else preserve `.corrupt-` + safe defaults; never crash-loops | | `Manager.writeJournal` / `loadJournal` | controller/internal/stacks/migrate.go | `(j *MigrationJob)` | Migration crash journal | Enables `RecoverMigration` at startup | | `backup.SharesPseudoStack` / `DisplayStackName` | controller/internal/backup/shares_payload.go | `"_shares"` / `(key) string` | THE reserved key for the shares source (restic tag, `backups/secondary/_shares`, CrossDriveBackup record) + its display mapping | NEVER let the raw key reach a Hungarian surface — map at the notification/prose boundary ONLY; the persisted `EnlargedBlocked` set and the templates index by the RAW key | | `Manager.buildSharesPayload` / `classifiedShares` | controller/internal/backup/shares_payload.go | `() (dir, passdbOK, error)` / `() []classifiedShare` | the definitions+credential payload and the availability-filtered share set both tiers read | payload is SECRET-BEARING (0600 passdb.tar) — never log its bytes/name at INFO. `classifiedShares` is the single place a dead mount is dropped, so both jobs agree | | `Manager.selectTier2TargetFrom` | controller/internal/backup/tier2.go | `(stack, sourceDrive, fullSize, stateOnlySize) (*Tier2Target, error)` | tier-2 target choice with the source drive supplied EXPLICITLY | the seam the shares job reuses — NEVER fork the headroom math; `selectTier2Target` is now a thin wrapper over it | | `Manager.tier2ReconcileRoots` | controller/internal/backup/tier2.go | `(destBase, roots, legRels)` | staleness pruning with explicit dest roots | pure extraction from `tier2Reconcile` (which now calls it with `hdd`/`userdata`); reuse it rather than writing a second pruner | | `Manager.liveShareRootOK` / `scratchJoin` | controller/internal/backup/shares_restore.go | `(dst) bool` / `(scratch, abs) string` | THE place guard for shares restore + scratch path reconstruction | a snapshot is UNTRUSTED layout input: require a STRICT descendant of a live registered root, refuse `..` and the drive root itself. `scratchJoin` strips the volume name — plain `filepath.Join` splices a drive letter mid-path | | `infra.SambaContainerName` / `SambaPassdbVolume` / `SambaPassdbMount` | controller/internal/infra/samba.go | consts | single source of truth for the samba container identity | the compose renderer interpolates them; stacks/backup/monitor read them. The CONTAINER name (`felhom-samba`) is NOT the stack name (`samba`) — `EffectiveProtected` needs the container one | | `sambaWriteAtomic` | controller/internal/stacks/samba.go | `(path, data, mode) error` | samba smb.conf/compose writes | tmp+**fsync**+rename (the only one of these that fsyncs). Fourth atomic-write helper in the tree — see §6 | | `Loop.writeMarker` / `Recover` | controller/internal/quiesce/quiesce.go | `(m Marker)` / `()` | Quiesce crash-safety | Marker written BEFORE stopping stacks; Recover restarts stranded stacks at boot | | `quiesce.TieredBackend` + `Loop.resolveDueTiers` / `quiesceAndPollTiers` | controller/internal/quiesce/tiers.go, quiesce.go | `Tiers/DueFor/StartBackupFor/BackupStatusFor`; `resolveDueTiers(ctx) ([]dueTier,bool,error)` | THE R-82 multi-tier backup schedule — several whole-guest tiers (local daily + PBS weekly) reconciled into ONE quiesce window | **Both tiers due ⇒ ONE stop/start pair**, never two (two = two app outages for one night). Tiers run SEQUENTIALLY (vzdump holds a guest lock) and the app stays down until the LAST tier snapshots — resuming earlier loses app-consistency on the DR tier. Order is fast-first (agent advertises primary first) or downtime blows up. `ErrTiersUnsupported` (route 404) ⇒ pre-R-82 agent ⇒ degrade to the untargeted path and **STILL BACK UP** — never read it as "nothing due". | | `quiesce.failureBreaker` + `Loop.dropBackedOffTiers` / `noteTierFailure` / `noteTierSuccess` | controller/internal/quiesce/breaker.go, quiesce.go | `blocked/recordFailure/recordSuccess(target, now)`; `backoffFor(n) time.Duration` | **R-88** — a tier whose backups keep failing stops re-quiescing. Backoff 15m→30m→1h→2h→4h (cap), reset on success | **It gates the QUIESCE, not the backup** — the harm was never the failing backup, it was the app outage taken to attempt it, so backed-off tiers are dropped from the due set BEFORE any stack is stopped. **Per TARGET** — a broken offsite tier must never suppress a healthy local one (`TestBreaker_OneFailingTierDoesNotSuppressAHealthyOne`). **Never permanent** — the cap bounds the retry INTERVAL, it never stops retrying; a latched breaker is a silent backup outage, worse than the loop it replaces. **`TriggerNow` is never gated** (it already bypasses due-ness and the window gate), though a manual run still RECORDS its outcome. **`stillRunning` is NOT a failure** — a first full offsite snapshot legitimately runs for hours. State is **in-memory on purpose**: a restart forgets the backoff and re-attempts, which is the cheap direction to fail. Log the deferral ONCE when armed, never per tick. | | `quiesce.TierNotifier` + `Loop.SetTierNotifier` / `noteTierFailure` / `noteTierSuccess` | controller/internal/quiesce/breaker.go, quiesce.go | `BackupFailed(tier,msg,err)` / `BackupRecovered(tier,msg)`; `SetTierNotifier(n)` INIT-ONLY | **R-97a** — the whole-guest backup tier reports its outcome to the hub | A **seam, not an import** — quiesce keeps no dependency on `internal/notify` (same reason `windowStartFn` is injected). Wired by a setter because main.go builds the notifier AFTER the loop; `nil` = unprovisioned guest, not an error. **Edge-triggered:** failure fires only when the breaker ARMS (`n == 1`), never per retry — the cadence is 15m/30m/1h/2h/4h and an event per attempt is an inbox nobody reads. Recovery rides `recordSuccess`'s existing bool. **Event types are OPERATOR-ONLY** (`whole_guest_backup_failed`/`_recovered`, hub >= v0.78.0) — NOT `backup_failed`, which has a customerMessages entry AND sits in live `enabled_events`, so it would email the CUSTOMER about a backup they cannot act on. `WholeGuestBackupDetails.Tier` is load-bearing: the hub keys its per-tier cooldown on it. | | `quiesce.Loop.SuppressedStacks` + `markQuiesced` / `markUnquiesced` | controller/internal/quiesce/suppress.go | `() map[string]bool` (nil-safe on a nil *Loop) | **R-97b** — an app THIS controller stopped for a backup is not a fault | Consumed at the SINGLE derivation point `classifyRunStates` (which computes both the banner dead-list and the notifier Down-set — keep it one place). **Cycle-keyed, not state-based:** v0.164.0's `!= StateStopped` filter cannot see an app caught MID-RESTART (`starting`/`unhealthy`), which is how BookStack alarmed on 2026-07-27. The window (`quiesceAlarmGrace` = 180 s, derived from the deploy flow's 120 s health timeout and Mealie's 60 s start_period) **EXPIRES** — permanent suppression turns a loud false alarm into a silent real one. Open-ended while the cycle runs (a first offsite snapshot legitimately takes hours). **This set alone is NOT the whole answer** — see `AppStopGuard.SuppressedStacks` (R-330) for the per-app operations; `classifyRunStates` consumes the union of both. | | `agentapi.BackupTiers` / `BackupDueFor` / `StartBackupFor` / `BackupStatusFor` | controller/internal/agentapi/backup_tiers.go | `(ctx[, target]) (…, error)` | The per-tier agent surface (agent >= v0.97.0) | `targetQuery("")` returns an EMPTY suffix so an untargeted call hits the pre-R-82 route byte-for-byte. `BackupTiers` maps a 404 to `ErrTiersUnsupported` — the documented ROUTE-PROBE capability signal, NOT a `featureProbes` row (the loop needs the tier LIST, not a yes/no). | ### Compose ops / stack lifecycle | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `Manager.DeployStack` | controller/internal/stacks/deploy.go | `(req DeployRequest) (string, error)` | Full deploy flow | Sets in-memory `Deployed` BEFORE compose up (slow-pull race), reverts on failure | | `DeclaredVolumeNames` (R-658, v0.268.0) | controller/internal/stacks/undo.go | `(composePath) (own, external []string, err)` | THE app's named volumes as Docker names them: `name:` else `_`, project = top-level `name:` else the stack dir | **Never select an app's volumes by the `com.docker.compose.project` label** — a restore before v0.268.0 made volumes without it, and the undo then copied nothing (R-658). The label is a cross-check only | | `renderDigests` / `RenderWithLadderDigests` / `CarryDigests` / `StripDigest` (v0.269.0/.1) | controller/internal/stacks/digest.go | compose bytes → the same with `ref@sha256` from the ladder entry for its refs | writing a compose that RUNS (update; the sync for an app NOT yet installed) — an INSTALLED app's sync uses `CarryDigests` (v0.269.1), which keeps the digest the app runs | Pins and records are digest-free by construction (`parseComposeImagesBytes` and the installed record strip) — never compare a raw `.Config.Image` against a pin | | `mergeRestoreFiles` / `RestoreTier2Whole` (v0.269.0) | controller/internal/backup/tier2_whole.go | mirror subtree → live subtree, four rules, counts | ANY restore that brings files back over live ones | Never `rsyncMirror` (--delete) in the restore direction; never overwrite a newer live file; a replaced file's old copy stays beside | | `nextLadderStep` / `StepKey` / `StepFile` (v0.268.0) | controller/internal/stacks/ladder.go | `(templateDir, pinned) (LadderStep, error)` | which definition ONE guarded-update press pins (`09` §3 decision 14) | `StepKey` must equal the catalog's `ladder.step_key` (TestLadder_StepKeyMatchesTheCatalog). A missing/wrong step file is an ERROR, never a jump | | `Manager.RedeployFromEnv` | controller/internal/stacks/deploy.go | `(name, env map[string]string) error` | Re-up with changed env (migration flip, config edits) | `compose up -d`, never `restart` (restart won't pick up images/env) | | `Manager.PersistUnitRedeployConfig` (R-47, v0.153.0) | controller/internal/stacks/deploy.go | `(name, env map[string]string) error` | the PERSIST half of `RedeployFromEnv` — app.yaml + locked fields + in-memory flags, **starts nothing** | **TRAP: the restore paths must use THIS, never `RedeployFromEnv`.** RedeployFromEnv ends in a full `up -d`, which before the replay IS the H4 race. RedeployFromEnv is now literally this + the unchanged up-and-report tail | | `Manager.StartStackServices` (R-47, v0.153.0) | controller/internal/stacks/manager.go | `(name string, services []string) error` | scoped `compose up -d ...` — the DB-only window a dump is replayed in | **REFUSES an empty list** (argument-less `up -d` is a FULL start — the one silent fall-through that would reintroduce the race). No `logPostStartStatus`: the app containers are absent on purpose. Never `RestartStack` here — it is a full up in disguise | | `appbackup.DBServiceNames` / `dbTypeForImage` (R-47, v0.153.0) | controller/internal/appbackup/dbservices.go | `(composePath string) ([]string, error)` | naming the compose SERVICE(s) holding a database, sorted | yaml.v3 `services:` MAP parse — **never a line scan** (immich's top-level `immich_ml_cache:` / `immich_postgres_data:` volume keys look exactly like services). `dbTypeForImage` is shared with `DiscoverDatabases`, which is what makes "a dump exists ⇒ a service can be named" hold. An error means CANNOT-TELL, never "no database" — callers refuse when a dump exists | | `Manager.StartStack/StopStack/RestartStack` | controller/internal/stacks/manager.go | `(name string) error` | Lifecycle | **`UpdateStack` was DELETED in v0.237.0 — use `StartGuardedUpdate` (update.go).** Protected stacks refuse stop; all funnel through composeExec. **NOT writers of desired state (R-166)** — 14 call sites, only 2 are the customer; recording intent here would make a nightly backup indistinguishable from the customer pressing Stop. Use `SetDesiredState` at the intent point instead | | `Manager.SetDesiredState` / `DesiredStateOf` / `BackfillDesiredState` (R-166, v0.189.0) | controller/internal/stacks/desiredstate.go | `(name, desired string) error` / `(Stack) string` / `() int` | THE customer-intent record — `app.yaml` `desired_state`, tri-state `""`/`running`/`stopped` | **ONE OWNER: the customer's action.** Writers are the API action switch, `DeployStack`, `UpdateOptionalConfig`'s redeploy branch, and the `.fab` restore adapter — nothing else, ever. **`""` (absent) means UNKNOWN, never "running"**: every pre-v0.189.0 app.yaml reads absent, so treating it as running would start every deliberately-stopped app on upgrade. Write intent BEFORE the act and REFUSE the act if it fails (§8.2). Backfill is **running-only** — never infer `stopped` from zero containers, that inference IS the defect | | `Manager.DriveLive` (R-171, v0.190.0) | controller/internal/stacks/deploy.go | `(hddPath string) bool` | is an app's data drive a live mountpoint RIGHT NOW | Wraps the **same** `isMountPoint` seam the userdata belt uses (`manager.go`) — never write a second liveness check, the two would drift invisibly. The system/local path is legitimately not a mountpoint and returns true | | `bootrecon.StartGate` (R-171, v0.190.0) | controller/internal/bootrecon/bootrecon.go | `MayStart(stack) (bool, reason)` | THE one question the boot sweep asks before starting anything | **Fail-safe: cannot determine ⇒ return FALSE.** One seam for all three holders (absent drive · quiesce · an in-flight app-data operation) because they differ only in the reason string. Implemented in `main.go` (`bootDriveGate`) reusing `quiesce.SuppressedStacks()`, `AppStopGuard.HeldStacks()` and `Manager.DriveLive` — never re-derive any of them. Held apps go to `Result.HeldByDrive`, **never** `StillDown` (that is the dead-app alarm's bucket) | | the boot settle window (R-157 A, v0.190.0) | controller/cmd/controller/main.go | `bootReconcileSample` / `StableFor` / `Budget` | sample the fleet until it stops changing, then sweep ONCE | **settle + budget + one `DefaultRetryDelay` must stay under `deadAppBootGrace`** — pinned by `TestBootWindow_CommonCaseFitsInsideTheDeadAppGrace`, which is why the budget is 50 s and not 60 s. Sampling is READ-ONLY; sweeping per sample would never see a settled fleet (the sweep's own StartStack changes it). A late recovery is REPORTED (`recordLateRecovery`), never hidden by widening the grace | | `backup.AppStopGuard` (`Begin`/`End`/`Recover`) (R-166, v0.189.0) | controller/internal/backup/appstop_marker.go | `(opID, reason, stacks) error` / `()` / `() *AppStopRecovery` | THE crash marker for stop→work→start windows (volume dump, offbox reconstitute, `.fab` export) | Its **own** file (`appstop-state.json`), never quiesce's — one file, one writer. **A `defer` is NOT the mechanism** (Campaign 8 fault 10: SIGKILL runs no defer); the marker is. Written BEFORE the stop, cleared ONLY after a restart that succeeded; a FAILED restart deliberately KEEPS it. `Recover` RETURNS its outcome rather than notifying, because it must complete before the boot reconciler while the notifier does not exist yet | | `backup.AppStopGuard.SuppressedStacks` + `markStopped` / `releaseStarted` / `ReleaseFailed` (R-330, v0.224.0) | controller/internal/backup/appstop_suppress.go | `() map[string]bool` (nil-safe on a nil *AppStopGuard); `ReleaseFailed(stacks ...string)` | **R-330** — an app a PER-APP operation is holding stopped (nightly volume dump, offbox reconstitute, `.fab` export) is not a fault | The **twin** of `quiesce.Loop.SuppressedStacks` above, and the two are unioned by `unionSuppressed` in main.go before `classifyRunStates` — **consult BOTH or the bug comes back**: R-330 shipped because the alarm read only the quiesce set while the per-app legs stopped apps through a different path. Rides `Begin`/`End`, so all three call sites got it with no call-site change. Grace is `appStopAlarmGrace` = 180 s, deliberately the SAME constant and derivation as quiesce's — two windows over one alarm that disagreed would be a bug on whichever path used the shorter one. **It must never latch**, and unlike quiesce's loop `End()` runs ONLY on a restart that succeeded: (1) every failure path calls `ReleaseFailed`, which drops the entry IMMEDIATELY so the app alarms on the next scan; (2) `Begin` REPLACES the set (one marker file = one operation); (3) `appStopMaxHold` (6 h) caps an open-ended hold and logs at WARN. **Deliberately NOT persisted** — after a crash the guard holds nothing and a down app must alarm. `ReleaseFailed` drops the suppression and KEEPS the durable marker; the two are independent and a test pins that | | `backup.ErrStartRefused` + `AppStopRecovery.Refused`/`Alarming()` (R-174, v0.191.0) | controller/internal/backup/appstop_marker.go | `errors.Is(err, ErrStartRefused)` / `() bool` | THE refusal-vs-failure split in the app-stop crash recovery | **A gated starter's refusal is NOT a restart failure.** `Recover`'s starter MUST be the gated `gatedAppStopStarter` (cmd/controller/main.go), never the raw `stacks.Manager` — that was the v0.189.0 defect, which started apps onto ABSENT drives at boot (R-171 one path over). A refusal goes to `Refused` (marker KEPT, silent), a real error to `Failed` (marker kept, ALARMS). Collapsing them routes a deliberate hold into `NotifyBackupFailed`, a customer-enabled type — the R-171 false alarm again. `main.go` must guard the notify with `Alarming()`, not `!= nil` | | `Manager.DeleteStack` / `RemoveStack` | controller/internal/stacks/delete.go | `(name, removeHDDData[, backupPaths])` | THE guarded removal paths | Orphan/protected/deploying/running checks + ProtectedHDDPaths filter before any RemoveAll. **R-442 (v0.236.0): the drive is the app's OWN `app.yaml` `HDD_PATH` (`appHDDPath`), never `cfg.Paths.HDDPath`; a data removal that cannot be resolved returns a typed `*RemoveRefusedError` BEFORE `compose down` — handlers `errors.As` it to 409 + `Message`** | | `resolveContainerState` / `aggregateState` | controller/internal/stacks/manager.go | `(dockerState, dockerStatus)` / `([]ContainerInfo)` | State classification | `.State` says "running" even when unhealthy — `.Status` parse is the fix | | `Manager.recordInstalledImages` (v0.233.0) | controller/internal/stacks/installed.go | `(name, stackDir string, env []string)` | writing down what each compose SERVICE is ACTUALLY running, into `app.yaml.installed_images` | Called after a successful compose up from `StartStack`/`RestartStack`/`UpdateStack`/`runComposeDeploy`. **Reads the CONTAINER, never `docker-compose.yml`** — that file is the value the syncer has already moved (spike §3: 25 minutes of disagreement). **A failed write NEVER refuses the action** — the deliberate OPPOSITE of `SetDesiredState`: intent refused, observation logged at ERROR. **NOT from `StartStackServices`** (the R-47 DB-only window would overwrite a complete record with a partial one). Skips the write when ref+digest are unchanged, and carries `at` forward so it means "running since". Its OWN seam (`installedExecFn`) with a **context + 30 s timeout** — the two existing exec helpers have neither | | `Manager.SetPin` / `AdoptPins` / `RenderPlanFor` / `AppliedComposePath` (v0.235.0) | controller/internal/stacks/pin.go | `SetPin(name, stackDir, pin, composeSrc) error` | THE version freeze — `app.yaml.pinned_images` + the stored `applied-compose.yml` | **`PinnedImages` is INTENT, `InstalledImages` is an OBSERVATION — never feed one from the other** (the R-166 category error, one field over). Four writers only: deploy, the guarded update (via `advancePinToCatalog`, which advances the pin and re-renders BEFORE the pull, and REFUSES the update if the pin cannot be written; a failed pull puts the pin BACK via `SetPin`), the restore adapter (this is what closes R-441), and `AdoptPins`. `AdoptPins` reuses `observationCoversTemplate` — do NOT write a second completeness rule — and skips loudly rather than inventing a pin. Absent pin = pre-v0.235.0 behaviour | | `Manager.UpdatePreflight` / `StartGuardedUpdate` / `RecoverUpdates` / `ResumeInterruptedUpdates` (v0.237.0) | controller/internal/stacks/update.go | `UpdatePreflight(name) *UpdateRefusal`; `StartGuardedUpdate(name) error` | THE update — refusals, then a 202 job with phases on `Stack.Updating/UpdatePhase/UpdateError` | **Never report an update complete before health is known (R-443).** Every cheap refusal runs BEFORE the intent write. Safety dump BEFORE the pin moves; pin BEFORE pull; pull failure → pin back; health failure → stop + HOLD, pin stays. Journal-before-mutate (`update-journal.json`); `RecoverUpdates` MUST run before the boot sweep and `ResumeInterruptedUpdates` AFTER `SetUpdateGuards`. Seams: `updateComposeFn`, `updateHealthFn`, `updateMemoryFn`, `updateDiskFreeFn`, `updateNowFn` (R-457: the age check and the test read ONE clock). Unwired guards ⇒ every update refused | | `stacks.UpdateGuards` + `updateGuardsAdapter` (v0.237.0; tiers v0.239.0) | controller/internal/stacks/update.go, controller/cmd/controller/main.go | `HoldFor`, `Busy`, `RestorePoints`, `CanBackUp`, `BackupNow`, `SafetyDump`, `HoldAfterFailedUpdate` | the ONLY bridge from the update job to the backup side (stacks cannot import backup) | Wired by `stackMgr.SetUpdateGuards` — pinned by `TestSlice4_UpdateGuardsAreWiredAtStartup`. Add a guard HERE, never by importing backup into stacks | | `backup.Manager.Tier2UnitRestorePoint` + `Tier2RestorePoint.ProvenCopyTime` (v0.237.0) | controller/internal/backup/update_guard.go | `(stack) (Tier2RestorePoint, error)` | "can this app be restored from Tier 2, and from when" — the predicate that gates BOTH the „Teljes visszaállítás" action and an update | **ONE predicate, two callers** (extracted from `buildAppBackupRows`, not copied). `CopyDate` is what the page NAMES (the package date, R-403); `ProvenCopyTime` is how OLD the data is — the last successful copy, because the manifest's `created_at` moves only when the DEFINITION changes (measured: a fresh dump under a 22-h-older manifest). Do not age a copy by `CopyDate`. **Since v0.239.0 the UPDATE no longer calls it directly** — it goes through `UpdateRestorePoints` (next row); the page still does | | `backup.Manager.UpdateRestorePoints` + `CanBackUpApp` (v0.239.0, R-475) | controller/internal/backup/update_guard.go | `(ctx, stack, accept func(UpdateTierPoint) bool) (UpdateTierPoint, bool, []UpdateTierPoint)` | "which backup can this update lean on" — walks Tier 2, 1, 3 and returns the first copy `accept` admits | **The age rule lives in the caller's `accept`** (stacks' `freshRestorePoint`), so it is ONE rule for every tier. **The ORDER is `UpdateTierOrderFor` (v0.241.0, R-479): 2 → 3 → 1 for an app with classified binds (`DataOutsideUnit`), 2 → 1 → 3 otherwise; `UpdateCopyHolds` is the matching phrase for the hold.** Stops at the first accepted copy, so Tier 3 (restic, 15 s bound, unreachable = absent + WARN) is reached only when needed. Seams: `updateTier2PointFn`, `updateTier1PointsFn`, `updateOffsiteTimesFn` (v0.240.0: Tier 3 reads `OffsiteSnapshotTimes` — snapshots only, never the inventory's per-app `stats`). stacks adds R-478's rule: a copy older than `deployed_at` does not count (`usableRestorePoint`). Tier numbers are pinned equal across stacks/backup by `TestR475_TierConstantsAgree` | | `backup.Manager.Tier2MirrorDirsForApp` / `RemoveTier2Mirrors` + `settings.DeleteAppBackupPrefs` (v0.240.0, R-474 / R-486) | controller/internal/backup/r474_remove_mirrors.go | `(stack) []string`; `(stack, dirs) []string` | deleting an app's backups on removal — the Tier-2 mirror lives on ANOTHER drive, outside RemoveStack's per-app base; the same dirs size the backup card (R-485) | Read the mirror dirs BEFORE the prefs are forgotten. Deletes only `/backups/secondary/` for a known root; never `_shares`, never a path that merely cleans to it. **The Tier-2 RECORD goes only with `remove_backups`** (R-486) — a removal that keeps the backups must keep the record, or the mirror is unrestorable. Pinned by `TestR474_RemoveHandlerDeletesUnitMirrorAndPrefs` + `TestR486_RemovalKeepsTheTier2RecordUnlessBackupsGo` | | `appbackup.dbTypeForImage` (R-484, v0.240.0) | controller/internal/appbackup/dbservices.go | `(image) (DBType, bool)` | the ONE place an image is judged a database — nightly dumps, pre-update safety dump, DB-only replay | Derived Postgres images (`postgis`, `pgvector`, `timescaledb`) are Postgres. A new engine image goes HERE and in `dbservices_test.go`'s table, never in a second matcher | | `backup.Manager.HoldAfterFailedUpdate` / `RunAppBackupNow` / `WriteUpdateSafetyDump` / `UpdateBusy` (v0.237.0) | controller/internal/backup/update_guard.go | see file | the update's hold, per-app backup-now, safety dump, busy check | The hold is `settings.RestoreHold` with `Reason: update_failed` — SAME store and gate as R-379, never a second map. `RunAppBackupNow` composes the nightly legs for ONE app (admission, DB dump, volume dump, capture, Tier-2) — do not write a second backup orchestration. A successful unit restore lifts an UPDATE hold only. **`isHeld` is ALSO true while a guarded update is moving the app (`SetUpdatingCheck`, v0.238.1)** — found live: the periodic capture overwrote a primary unit during a health wait | | `stacks.Manager.memoryVerdict` (v0.237.0) | controller/internal/stacks/deploy.go | `(newReq, newLimit, releasedReq, releasedLimit int) (refusal, warning string)` | the deploy's memory check, shared with the update | An update RELEASES the app's current request first. Deploy passes `0, 0` and is byte-identical in wording and log line | | `Syncer.SetRenderPlanFn` + `renderSource` (v0.235.0) | controller/internal/sync/sync.go | `func(appName string) stacks.RenderPlan` | the catalog render table | **NIL-SAFE: no seam = copy verbatim = the old product.** Catalog images == pin → verbatim (fixes flow + self-healing, both deliberately kept); differ → the WHOLE stored definition, **never a ref substitution into a newer template** (`wger 2.6`). `.felhom.yml` always verbatim (R-458). The syncer must NEVER read app.yaml. Re-reads the applied file before writing it — a test caught it writing an empty compose over a live app | | `Stack.CatalogImages` vs `Stack.TemplateImages` (v0.235.0) | controller/internal/stacks/manager.go | both `map[string]string` | badge input vs "what the next `up -d` gives this app" | **THE TRAP: same type, same shape, opposite meaning after the freeze.** `TemplateImages` reads the LIVE (possibly frozen) compose file; `CatalogImages` reads the syncer's clone. `web.compareInstalledToTemplate` MUST use `CatalogImages` or it answers „Naprakész" on exactly the apps that are behind, with every test green. Red-proved | | `Manager.BackfillInstalledImages` (v0.234.0) | controller/internal/stacks/installed.go | `() int` | seeding `installed_images` for apps that have NO record — call ONCE at startup | Beside `BackfillDesiredState` in `cmd/controller/main.go`, after it and BEFORE the boot reconciler (pinned by an AST-walking test that asserts the ORDER). **READS only** — starts nothing, writes no compose file. **Never overwrites an existing record** (an app that has one is not even observed). **REFUSES a partial observation** (`observationCoversTemplate`): `web.compareInstalledToTemplate` reads a service-count mismatch as BEHIND, so seeding a degraded app from what is visible renders „Frissítés elérhető" over an app that is current. The bring-up paths may write a partial because they follow a SUCCESSFUL `up -d` where a gap is real news; a backfill meets any state and must be stricter | | `stacks.ParseComposeImages` (v0.233.0) | controller/internal/stacks/installed.go | `(composePath string) (map[string]string, error)` | compose SERVICE name -> the image the FILE pins; feeds `Stack.TemplateImages` and the update badge | yaml.v3 `services:` MAP parse, never a line scan (same reason as `DBServiceNames`). An error means CANNOT-TELL — `ScanStacks` leaves `TemplateImages` nil and the badge renders NOTHING, never "current" | | `stacks.Manager.AnyUpdating` / `SetSelfUpdatingCheck` + `selfupdate.Updater.SetAppUpdatingCheck` (v0.261.0, R-608) | controller/internal/stacks/update.go, internal/selfupdate/updater.go, wired in cmd/controller/main.go | `() bool` callbacks, both directions | THE two-way lock between the CONTROLLER's own swap and a guarded APP update | **Wire BOTH halves or neither** — they are wired together in `main.go`, the only place holding both objects; **`stacks` must never import `selfupdate`**. The app-update side is a sibling of `SetBackupRunningCheck` and is consulted in the SAME three places (dry run, `TriggerUpdate`, `maybeAutoUpdate`) — do not add a fourth pattern. **`AnyUpdating` MUST answer false for a HELD app** (`Stack.Updating` is cleared on done/failed/held): a latching gate would block the controller's own updates for ever, including the one that fixes the hold. Nil callbacks are SAFE and mean pre-v0.261.0 behaviour — never fail closed on an unwired gate | | `stacks.CatalogOrder` / `CompareImageRefs` (v0.260.0, R-524) | controller/internal/stacks/updateorder.go | `(Stack) UpdateOrder` — Unknown/Current/Behind/**Ahead** | THE one "how does this app stand against the catalog?" verdict | **Both the badge AND `Manager.UpdatePreflight`'s `downgrade` refusal read it — never re-implement the comparison.** `web.compareInstalledToTemplate` is a thin wrapper. Ahead is NARROW: every differing service must be orderable AND newer, else Behind. Ordering is `util.Version.Compare` behind a tag normaliser (`X.Y`/`X.Y.Z`, optional `v`, suffix must be IDENTICAL on both sides) — **never add a second comparator**. Queries NO registry; absent record = Unknown, never „Naprakész" | | `web.updateBadge` / `updateBadgeAt` / `Metadata.CatalogSince` + `CatalogSinceAge` (v0.233.0) | controller/internal/web/updatebadge.go, controller/internal/stacks/metadata.go | `(stacks.Stack) *MetaBadge` | THE "is this app current?" label — „Naprakész" / „Frissítés elérhető — N napja" | The SECOND `*MetaBadge` user the type was built for: existing `meta_badge` partial, **no new markup or CSS**. **NO RECORD RENDERS NOTHING — absent means UNKNOWN, never current** (R-166 applied to an observation; red-proved). **No version number reaches the customer** and **no registry is queried**. `catalog_since` is tolerant in the `lifecycle` style — absent/empty/malformed/**future** all degrade to a badge with no age + one WARN. LIMITATION: for the **10** floating pins (recounted 2026-09-21; the old "23" matched no definition the catalog supports) the ref can match while the image has moved — 6 of the 7 measurable ones HAVE moved — so those read „Naprakész" when they may not be | | `Manager.logPostStartStatus` | controller/internal/stacks/manager.go | `(name, stackDir, env)` | Async post-start verification | compose up exits 0 on crash-loops; this is the detection. Goroutine + 3s, never blocks | | `Manager.EnsureBaseStack` | controller/internal/stacks/infra.go | `() error` | Traefik/cloudflared/FileBrowser infra convergence | Renders from `internal/infra` templates | | `appbackup.ClassifyBinds` / `ValidateBackupSpec` | controller/internal/appbackup/classify.go | `(spec, binds) ([]ClassifiedBind, bool)` / `(spec, binds) error` | Backup-classification (Task 2, referential coupling) — pure | Two-level default: explicit wins over `:ro`; unlisted writable→mandatory, unlisted `:ro`→excluded; nil spec→legacy/false. Validate REJECTS the WHOLE block on any defect (whole-block semantics). INERT — no tier consumes it yet | | `ParseComposeClassifiableBinds` | controller/internal/stacks/classify_binds.go | `(composePath) []appbackup.ComposeBind` | `${VAR}`-relative binds + `:ro` for classification | Do NOT use `ParseComposeHDDMounts`/`ExportDataMounts` as classifier input (§traps) — they resolve absolutes, drop `:ro`, or union the userdata ROOT. Short-syntax only | | `Metadata.EffectiveLifecycle` / `CanInstall` / `IsAbandoned` + `web.lifecycleBadge` / `web.visibleCatalogStacks` | controller/internal/stacks/metadata.go, controller/internal/web/metabadge.go, controller/internal/web/handlers.go | `meta.CanInstall() bool` | app lifecycle: `available` / `hidden` / `abandoned` (v0.158.0) | THE single interpretation of `.felhom.yml` `lifecycle:` — every surface must go through these, never compare the raw string. Listing drops `!Deployed && !Protected && !CanInstall()`; `api.deployStack` refuses server-side BEFORE any mutation (hiding a button is not a gate), `stacks.DeployStack` repeats it for non-API callers. **Unknown value fails OPEN** (→ available + one WARN) — opposite to the gate on purpose: a typo must never pull a working app out of every catalog. **NEVER let lifecycle reach orphan detection** (`getCatalogTemplateSlugs`) — a withdrawn template stays in the tree, or every deployed instance reads as `Elavult` and gets a Törlés button. Badges: `MetaBadge` + `meta_badge` partial, built generic for R-56 difficulty labels | | `Manager.ClassifiedBinds` + `StackDataProvider.GetStackClassifiedBinds` | controller/internal/stacks/metadata.go, appbackup/appdata.go | `(name) ([]appbackup.ClassifiedBind, bool)` | Per-stack classification through the REAL LoadMetadata validate path | The wired seam Task 3 consumes; LoadMetadata is the SINGLE validation choke point (bad block → nil + one ERROR → legacy) | | `backup.Manager.DumpAppVolumesSafe` | controller/internal/backup/backup.go | `(stackName) error` | Volume tar of a live app | Stops → dumps → restarts; surfaces BOTH errors (app may be left stopped). Check `GetDockerVolumes()!=0` + `IsProtectedStack` BEFORE calling — it stops the stack before its own volume check (see `runVolumeDumps`) | | `backup.Manager.ListRestorePoints` | controller/internal/backup/restore_points.go | `(stackName) ([]RestorePoint, bool)` | Restorable keep-side backups (the /api/backup/snapshots payload) | ONE point per app (the current unit); tier always 1 — never list Tier-2 (not restorable via /backup/restore) | | `backup.Manager.stampDataFile` / `foldUnitData` / `unitNewestArtifact` / `RecoveryManifest.Data` (v0.275.0, R-696) | controller/internal/backup/data_versions.go, removed_units.go | `(stack, unitDir, rel)` / … / `(unitDir) (time.Time, bool)` | A backup's data and its version travel together: the leg stamps each file it writes, the capture folds the stamps into `data` and keeps the definition the data belongs to; `unitNewestArtifact` is THE unit data time for every tier and list | **A new writer of a unit's data file MUST call `stampDataFile` right after the file reaches its final name** — an unstamped file makes the whole unit's versions unknown (restores as before, WARN). Never date a unit by its manifest; never count `pre-restore-*` as data. `definitionPins` is the ONE source of "current pins" (the stack's compose parse) | | `backup.unitVersionCheck` / `ErrUnitVersionMismatch` (v0.275.0) | controller/internal/backup/restore_unit.go | `(stack, *RecoveryManifest, composeDir) ([]string, error)` | A restore never starts data with a definition it does not belong to — the check every unit restore and the off-site restore make before the first mutation | Branch with `errors.Is`; the sentence is the bundle's (`err.backup.unit_version_mismatch` / `_versions_mixed`). Unknown (`nil, nil`) restores as before | | `backup.Manager.RestoreTier2Files` | controller/internal/backup/tier2_restore.go | `(stackName) (filesRestored int, err error)` | In-place ADDITIVE-ONLY class-C file restore from the recorded Tier-2 copy (`POST /backup/tier2/restore`) | Never overwrites/deletes live files; refusals (Hungarian) before any stop; source = recorded `DestinationPath`, never re-selected. **C9-F1 (v0.183.0): reads `hdd/` + `userdata/` ONLY — never `recovery-unit/`.** For 43 of 53 catalog apps that is a guaranteed no-op, so it now refuses with `ErrTier2NoRestorableData` BEFORE stopping the app. Ask `Tier2RestoreCoverage` first | | `backup.Manager.Tier2RestoreCoverage` | controller/internal/backup/tier2_restore.go | `(stackName) (Tier2Coverage{Legs, HasUnit}, error)` | Answers what a Tier-2 restore CAN and CANNOT return for an app, from the RECORDED copy on disk | **C9-F1.** `Legs` = subtrees the restore reads; `HasUnit` = the copy also holds DB dumps + volume tarballs it will NEVER read. Use it to refuse up front and to decide whether the success message must disclose uncovered data. Judged from the copy, not the catalog, so a retemplated app is judged by what it actually has | | `Manager.acquireRunning`/`releaseRunning`, `acquireMigrating` | controller/internal/backup/backup.go, controller/internal/stacks/migrate.go | `() error` | Single-flight for long ops | Copy this mutex-flag pattern for any new long-running manager op | | `stacks.Manager.WantsStopped` (v0.283.0, R-721) | controller/internal/stacks/desiredstate.go | `(name) bool` | The household's recorded intent is „stopped" | ASK IT before any machine restarts an app it stopped for its own purpose (quiesce, volume dump, update leg) — a Stop pressed meanwhile must hold. Never gate the household's own Start on it | | `settings.DefaultOffboxOnForNewApp` / `AppsWithoutOffbox` / `EnableOffboxFor` (v0.283.0, decision 50) | controller/internal/settings/offbox_default.go | `(stack) (bool, why)` | A fresh install joins the off-site copy when the customer has off-site | Call only from the deploy-done hook; it never overrides an existing per-app choice | ### Secrets hygiene | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `crypto.Encrypt/Decrypt/IsEncrypted/DecryptMap` | controller/internal/crypto/crypto.go | AES-256-GCM, `ENC:` prefix | app.yaml sensitive values | `Decrypt` errors on non-ENC input — use `DecryptMap` for whole env maps (passes through + warns) | | `crypto.LoadOrCreateKey` | controller/internal/crypto/crypto.go | `(path) ([]byte, error)` | The 32-byte key file (0600) | — | | `SaveAppConfig` / `LoadAppConfigDecrypted` | controller/internal/stacks/deploy.go | `(stackDir, cfg, encKey, sensitiveVars)` | app.yaml persistence | Encrypts only `SensitiveEnvVars(meta)`; never write app.yaml directly | | `generateValue` / `randomAlphanumeric` | controller/internal/stacks/deploy.go | `(spec "password:N\|hex:N\|base64key:N\|static:v")` | Auto-generated secrets | crypto/rand-backed; reuse the spec grammar | | `Manager.GenerateSecretForField` | controller/internal/stacks/deploy.go | `(stackName, envVar) (string, bool)` | Replacement value for a RESETTABLE secret from its catalog `generate` spec (O4 restore path via `backup.SetSecretGenerator`) | REFUSES `data_key` fields, spec-less and non-secret fields; never log the value | | `reconcileRestoreSecrets` | controller/internal/backup/restore_unit.go | `(nonSecretEnv, unitSecrets, guestSecrets, secretNames, dataKeyNames)` | Recovery-unit restore env merge | **Precedence: UNIT WINS over guest** (the unit's secrets match the data being restored; the guest's are merely newest). Pure — new sources arrive as ARGUMENTS. Fail-closed data-key gate lives here | | `stacks.PortableSecretEnvVars` | controller/internal/stacks/deploy.go | `(meta) []string` | **THE D5 secret boundary**: which secrets may travel on a customer drive | `type: secret` travels, `type: password` NEVER, minus the `nonPortableSecrets` code register. Withholding the password class is what licenses plaintext — do not relax one without the other | | `buildUnitAppYaml` / `readUnitEnv` | controller/internal/backup/{recovery_unit,restore_unit}.go | `(info) []byte` / `(path, portableNames)` | The ONE place the unit's app.yaml is written / split back | Split is driven by the MANIFEST's portable names, never guessed from key names; write 0600; empty `portableNames` = schema-1 unit ⇒ everything is plain config | | `EncryptFile` / `DecryptFile` / `IsEncryptedFAB` | controller/internal/appexport/crypto.go | password-based file crypto | .fab export bundles | scrypt-derived AES+HMAC keys | | `maskRepoURL` | controller/internal/sync/sync.go | `(url) string` | Logging git URLs | Strips embedded credentials | | `metrics.RedactLine` | controller/internal/metrics/redact.go | `(s string) string` | ANY log line shipped off-box (issue context, log tails) | Masks password/passwd/secret/token/api-key/authorization/bearer values + 64-hex; apply BEFORE the line leaves the box — controller-side redaction is authoritative | | `settingsRetrievalPasswordRevealHandler` | controller/internal/web/handlers.go | `POST /settings/retrieval-password/reveal` | **THE PATTERN for showing a secret in the UI** — an XHR that returns only the value | **Never template a secret into a page and hide it with CSS.** `display:none` / `hidden` / `type="password"` stop a browser DRAWING the value; the plaintext is still in the response body, so a `curl` of the page returns it, and it reaches caches, history and any screen-share of the source. R-249 shipped exactly that for two months and was found by it landing in a transcript. The page carries a **boolean** (`HasRetrievalPassword`); the value comes from a POST (CSRF-covered, uncacheable) and the reveal is **logged as an act**. `escrow_handlers.go` states the same rule for R. **Test on the RESPONSE BODY** — a test asserting what the customer *sees* cannot see this class at all. **Both R-254 sites are now FIXED the same way** — `POST /apps//initial-credentials/reveal` (re-reads the container, never a cached copy) and `POST /stacks//auto-field/reveal` (authorised on the field being a `type: secret` auto-field of that stack). **Per-secret, never one generic reveal-any-named-secret endpoint.** The PRE-DEPLOY hidden input is deliberate and untouched — a form must carry what it submits (README §318). Enforced by `scripts/secret_in_markup_gate.py`, whose measured blind spot (a secret under a neutral page-data key) is in its docstring; runtime body-assertion covers 4 of 27 pages — R-255. | ### Storage registry + mount detection | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `Settings.AddStoragePath/RemoveStoragePath/RepointStoragePath` | controller/internal/settings/settings.go | registry CRUD | ALL drive registration | `AddStoragePath` dedupes (double-register is clean no-op); `AutoDiscoverStoragePaths` never re-adds a known-in-any-state path | | `Settings.SetDisconnected/ClearDisconnected/SetDecommissioned` | controller/internal/settings/settings.go | state flags + stopped-stacks memo | Drive lifecycle state | Records `stoppedStacks` so reconnect restarts exactly those | | `registerStoragePath` | controller/internal/web/storage_handlers.go | `(where, label, setDefault) error` | Post-enroll registration | The single funnel used by init/attach/manual-add | | `system.IsMountPoint` / `IsWritable` / `PathsOverlap` | controller/internal/system/mounts_linux.go | `(path) bool` | Mount checks | `_other.go` stubs return permissive values — Linux behavior is the real one | | `system.CheckBackupDestination` | controller/internal/system/mounts_linux.go | `(path) DestinationHealth` | Tier2/offbox target vetting | Detects same-physical-device (`SamePhysicalDevice`) | | `system.ProbeStoragePath` | controller/internal/system/mounts_linux.go | `(path) ProbeResult` | Disconnect detection | — | | `appexport.DiskFree` | controller/internal/appexport/estimate.go | `(path) int64` | Free bytes for space gates (df-based, 0 on any error) | Exported v0.128.0 for the browser-upload gate; test seam = `web.uploadDiskFree` package var | | `stacks.ExportDataMounts` | controller/internal/stacks/delete.go | `(composePath, hddPath) []string` | THE .fab-export mount discovery (v0.130.0 C6B-F1) | Unions `${HDD_PATH}` binds + the `${USERDATA_PATH}` ROOT (single `userdata` entry — basename must round-trip the import's `/` mapping; NEVER return per-bind userdata subpaths). Containment-deduped. Backup-side `stackAdapter` deliberately does NOT use it | | `Server.deployedAppsOnPath` | controller/internal/web/netstorage_handlers.go | `(base) []string` | Deployed stacks whose HDD_PATH is base or a subpath | The C6B-F2 share-removal guard; nil-safe on stackMgr | | `planDriveGates` / `Server.ReconcileDriveGates` | controller/internal/web/intermediary.go | pure plan + executor | Drive appear/disappear reactions | `planDriveGates` is PURE (unit-testable); loop at `driveGateLoop` | | `Server.runStorageInit` / `runStorageAttach` | controller/internal/web/storage_handlers.go | wizard pipelines | New-drive enroll / re-attach | Format goes through the agent's two-step confirm (below) | | `Server.sharingResolvePath` / `sharingResolveStorageRoot` | controller/internal/web/sharing_handlers.go | `(raw) (string, error)` | THE guard for every customer-supplied SMB share path | resolvePath validates a share TARGET (refuses the drive root); resolveStorageRoot validates the new-folder PARENT (accepts exactly a registered live root). Refusals are UNIFORM (no filesystem oracle). Never add a second deny-list — `stacks.SharingDeniedRoots` derives from `ProtectedHDDPaths` | ### Agent local-API client (cross-repo edge) | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `agentapi.New` | controller/internal/agentapi/client.go | `(endpoint, token, fingerprintHex) (*Client, error)` | Pinned-TLS client to felhom-agent | Leaf-DER SHA-256 pin replaces chain verify; fails closed. Bounded idle pool (leak fix) | | `Server.agentClient` | controller/internal/web/agent_disk_handlers.go | `() (*agentapi.Client, error)` | THE memoized client accessor | Always use this, never a fresh `agentapi.New` per request (§3) | | `Server.ProbeAgentChannel` | controller/internal/web/agent_disk_handlers.go | `(ctx) (constructionErr bool, err error)` | Channel health probe | Probes via the PRODUCTION client on purpose (self-heals, mirrors UI) | | `Client.FormatDisk` | controller/internal/agentapi/client.go | `(ctx, device, fstype, confirmed, durableID)` | ONLY format/wipe entry | Sentinels: `ErrNeedsConfirmation` (user-data, resubmit confirmed+durableID) / `ErrFormatRefused` (system/backup — operator opsign only). Agent re-checks role server-side | | `Client.EjectDisk` / `Decommission` / `AssignDisk` / `GuestAttach` / `ListCandidates` | controller/internal/agentapi/client.go | disk lifecycle | Delegate ALL disk ops to agent | Controller holds no Proxmox creds — never shell out to disk tools in-guest | | `Client.AddNetStorage/ListNetStorage/RemoveNetStorage` | controller/internal/agentapi/client.go | NAS mounts (A1) | Network storage | Password passes through to agent's 0600 cred file; controller NEVER persists it | | `agentapi.StatusError` | controller/internal/agentapi/client.go | `{Path, Code}` typed non-2xx GET error | Distinguishing HTTP statuses from transport errors (`errors.As`) | NEVER string-match agent error text — the capability probe keys on `Code==404` | | `SupportCache.Supports` / `Client.Supports` | controller/internal/agentapi/features.go | `(ctx, prober, Feature) SupportState` | Agent-capability gate for COUPLED features (route probe, TTL 5m) | 404 ⇒ No; transport/5xx ⇒ Unknown (NEVER refuse on Unknown). New coupled feature = new `featureProbes` row + gate call at the entry point + `MinAgent:` in the CHANGELOG header (publish-train-rules.md). Web layer: `Server.netFeatures` through the `netAgent` seam | | `agentapi.DiskVerdictFor` / `DiskVerdict.Label` / `DegradedAttributes` / `UncorrectableSectors` / `DiskPrior` / `TemperatureFailC` | controller/internal/agentapi/diskverdict.go | `(*SmartSummary, DiskPrior) DiskVerdict` | THE shared disk-health verdict (card chip + hourly check) — v0.169.0, 14-row ladder v0.215.0 | Pure — no clock, no I/O; history arrives as `DiskPrior`. nil/UNKNOWN → `DiskVerdictUnknown` (Nincs adat, NEVER alarms, row 1 is first for that reason). **Never trust `smart_status.passed`**: attrs 187/197/198 carry `thresh: 0`, so it cannot fail on unreadable sectors. A zero `DiskPrior` is the fail-safe (first sighting can only reach Figyelmeztetés). **Four labels, no fifth** — predicted failure is „Hiba". Do NOT recompute the verdict inline anywhere, and do NOT re-literal 60 °C — use `TemperatureFailC` | | `Server.resolveBackupTargetState` / `backupTargetView` | controller/internal/web/backup_target_offer.go | `(ctx)` → state / `*BackupTargetView` (nil = render nothing) | The whole-system backup-target answer: healthy · degraded-never-configured · **TargetAbsent** (configured, drive gone) · unknown | Test seams `Server.tiersFn` + `Server.disksFn` (nil → the real client). **`degradedMessageFor` is the ONE place that decides customer copy** — add a state there, never in a template. `backupTargetView` returns **nil** for healthy AND unknown so a template typo cannot decorate a working box. R-112: this state had NO consumer for two releases; the render is server-side on `backups.html`, and the seam test drives `backupsHandler` and asserts rendered HTML | | `Server.cachedDisks` / `RunDiskHealthCheck` | controller/internal/web/disk_health.go | `(ctx)` | Card fetch (60s TTL) / the hourly degradation check | Card uses the 60s TTL cache (anti-smartctl-storm); the CHECK fetches FRESH (`fetchDisks`). Test seams: `Server.disksFn` (source) + `Server.diskNotifyFn(notify.DiskAlert)` (sink). State is PERSISTED (v0.215.0) — a restart no longer re-baselines | | `diskAlertDecision` / `diskAlertKindFor` / `Server.priorFor` / `Server.cardPriorFor` | controller/internal/web/disk_health_state.go | pure + `(key) agentapi.DiskPrior` | Whether an observation emits, and which message shape | Compares against the **last ALERTED** verdict, not the last observed — that is what collapses a flap to one alert. Re-alert needs doubling **AND** 24h (an AND). **`priorFor` is for the CHECK, `cardPriorFor` for the CARD** — they differ by one observation and mixing them makes the chip read one level more severe than the email | | `diskRecord` / `writeDiskState` / `Server.loadDiskStateLocked` | controller/internal/web/disk_health_state.go | `disk-health-state.json` in `cfg.Paths.DataDir` | Persisted per-disk observation + alert history | Atomic tmp+rename (the `selfupdate.SaveState` shape, copied not imported). Missing file = normal; corrupt = LOG and fall back to no-prior, **never fatal**. Written ONCE per check run. Keyed by `diskKey`. **One record per disk, NOT a sample series** — history is Phase 2/3 in `metrics.MetricsStore` | ### Notifications / hub sync | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `Notifier.PushEvent` | controller/internal/notify/notifier.go | `(eventType, severity, message, details)` | Hub events | Async goroutine, 3 attempts/3s backoff. NEW event types MUST be added to hub `allowedEventTypes` or POST /event 400s. **SEVERITY IS AN EXACT WIRE CONTRACT: `{"info","warning","error","critical"}` and nothing else.** The hub silently COERCES any other string to `"info"` (`hub/internal/api/handler.go`, the ingest severity switch) and `severityNotifies` (`hub/internal/notify/dispatcher.go`) emails only warning/error/critical — so a typo'd severity is stored and delivered to NOBODY, with no error anywhere. **`"warn"` is not a severity.** It shipped on `disk_health_degraded` (fixed v0.215.0, R-328) and is STILL live on `app_start_failed` (R-329) | | `notify.DiskAlert` / `DiskAlertKind` / `DiskAlertKind.Severity()` | controller/internal/notify/notifier.go | `NotifyDiskHealthDegraded(DiskAlert)` | The disk-health alert payload + its five Hungarian message shapes | The notifier owns customer copy — pass a `DiskAlert`, never a pre-formatted string, or Hungarian scatters across packages. `Severity()` is the ONE mapping kind→hub severity and is exported so any package can assert the contract instead of duplicating the literal | | `Notifier.Notify*` convenience methods | controller/internal/notify/notifier.go | typed wrappers (backup/DB/storage/channel/DR…) | Standard events | Add a typed wrapper rather than raw PushEvent calls | | `report.BuildReport` / `Pusher.Push` | controller/internal/report/builder.go + pusher.go | periodic hub report | Box→hub reporting | ACK carries `config_version` → `ConfigRefresher.Reconcile` | | `report.Trigger` (`NewTrigger`/`Fire`/`Run`) | controller/internal/report/trigger.go | `Fire()` after a hub-relevant user action | THE out-of-cycle report push (v0.139.0) — fire via `api.Router.reportPushNow` / `web.Server.reportTriggerNow`, both nil-safe | Coalesce-and-eventually-fire (trailing edge; quiet 2s, min spacing 15s). NEVER add retries (Pusher owns them); NEVER reuse the `internal/sync` REFUSE-debounce for hub pushes (a refused fire loses the update until the next cycle). Fire only AFTER a successful local commit | | `report.SetPendingLogTails` + `buildLogTailsSection` | controller/internal/report/logtail.go | ACK `log_tail_requests` → next report `log_tails` | THE pull-based ACK-flag pattern (hub asks, controller pushes next cycle) — copy for any new hub→box request | Consume-once drain at BuildReport; failed push re-arms from the hub's still-pending request; NEVER add a hub→controller push channel | | `metrics.FetchContainerLogTail` | controller/internal/metrics/logscanner.go | `(name, tailLines) (string, error)` | Raw per-container `docker logs --tail=N` | 15s timeout; caller caps/redacts (capTailLines) | | `ConfigRefresher.Reconcile` | controller/internal/report/config_refresh.go | `(ackVersion int)` | Pull-based config refresh | Re-pulls controller.yaml (re-merging local_api), then graceful self-restart; first-run = baseline, no restart | | `offsiteapply.HubRegistrar` / `HubWindowClient` / `PinnedProber` (v0.289.0, decisions 68–69) | controller/internal/offsiteapply/seams.go | `Register(ctx,pub)(fp,err)` · `Confirm` · `MoveAside` · `Open/Close` window · `Probe(ctx,host,user,port,kh,privPEM) bool` | EVERY off-site key install, the hub move-aside, the clean-up window | **The box never handles the sub-account password** — there is no consume path any more. `PinnedProber` is a POSITIVE observable (exit 0 + rclone output); "authenticates" is NOT enough — an unpinned key authenticates and can delete. | | `Manager.offsiteWindowRetention` + `offsiteGuard` (v0.289.0, R-822) | controller/internal/backup/offbox_window.go | `(ctx, base, env, why)` · pure `(all, plan, now, newestAllowed, max) (ids, refuse)` | THE retention step for both callers (after a run, over quota) | Pinned tier: no window → nothing deleted; the guard runs BEFORE any forget; "young" = fewer than `keepDaily` CALENDAR days old (v0.294.0, R-867 — the line is built from the same constants as `retentionPolicy`; never an hour count); a young snapshot superseded the same day is EXCLUDED, any other young removal / future date / plan above `max_remove` REFUSES; tests run restic 0.14.0's policy itself (`restic0140Plan`, proven identical to the binary); forget is by explicit ids, oldest first. A due abandonment goes to `OffsiteAbandonClient` (hub, 7-day wait). NAS tier: the old SP-2 policy, unchanged. | | `offsiteapply.SettleProvider` / `SettleFunc` / `Bridge.AwaitSettle` / `ReconcileWhenSettled` (R-71a, v0.162.0) | controller/internal/offsiteapply/offsiteapply.go + seams.go | `SettleState() (version, floor string, updateRunning, floorKnown bool)` | THE settle-gate: defers the offsite one-time-password consume past a managed day-0 floor-update (the F10 race). Wire the `SettleFunc` adapter over `updater.GetFloor()`/`IsUpdateRunning()` — **the updater's knowledge is the ONE floor source; never fetch the floor a second way**. Gate ONLY the bridge goroutine, and only when an updater exists (nil `Settle` = reconcile immediately). Bounds `settlePoll`/`settleFloorSubBound`/`settleOverallBound`; the floor is in-memory (report-ACK-derived, ~5–10 s), NOT persisted → unknown until the first ACK on any restart. Inject `Now`/`Sleep` in tests (no real sleeps). B′: at/above-floor GOes on the first poll, zero wait. Do NOT touch the consume/persist order or the 404 contract — ordering only | | `bootstrap.MaybeIngest` / `RefreshConfig` | controller/internal/bootstrap/bootstrap.go | bootstrap.json → controller.yaml | Day-0 + refresh | Overwrites controller.yaml, NEVER settings.json | | `api.GracefulSelfRestart` | controller/internal/api/selfrestart.go | `(logger)` | Controller self-restart | Detached exit; bootstrap unit re-runs the image | | `Settings.AddPendingEvent/DrainPendingEvents` | controller/internal/settings/settings.go | offline event queue | Events while hub unreachable | — | | `Manager.SetUnitNotify` + `UnitSpace` (R-158/R-167, v0.191.0) | controller/internal/backup/recovery_unit.go | `(func(stack string, err error, *UnitSpace))` | THE per-app Tier-1 recovery-unit capture failure alert — fires PER APP from `captureAllRecoveryUnits`, loop continues | **OPERATOR-TIER** (`recovery_unit_capture_failed`, in the hub's `operatorOnlyEvents`). **NEVER route it to `backup_failed`** — that type is in `DefaultEnabledEvents` and carries Hungarian copy, so it emails the CUSTOMER about a failure they cannot act on (D-c; R-158's own proposal said `backup_failed` and D-c overrides it). `UnitSpace` is **nil when the target filesystem is unreadable** and renders as *"unavailable"*, never as zeros — "0 GB free" and "we could not look" are opposite diagnoses. No controller-side cooldown: the hub owns it | | `Manager.beginRunSummary` / `noteFailure` / `noteAttempted` / `emitRunSummary` / `SetRunSummaryNotify` (R-182, v0.194.0) | controller/internal/backup/runsummary.go | `(kind, runID) func()` / `(app, leg, reason)` / `(RunSummary)` | **THE per-run operator digest.** One `backup_run_failures` event at the end of a run listing every failed app, its leg and its reason — emitted ONLY when something failed | **The RECORD and the NOTIFICATION are different things and must stay so.** The per-app `recovery_unit_capture_failed` event is the record (hub routes it *record-only*, stored + logged every time); this digest is the notification. Before R-182 one event was both, and did neither: nine arrived, two were mailed, seven vanished before `LogNotification`. **Lifetime is `admissionSet`'s exactly** — absent collector means "no run in flight", never a stale answer. **A refusal is noted ONCE, inside `admitApp` where the verdict is taken**, not at the three legs that consult it: R-181's one-verdict-covers-all-three contract makes per-leg noting produce "2 of 1 apps failed". **Deliberate skips (disconnected / decommissioned) must NEVER be noted** — they have their own alert and a nightly digest about an unplugged drive is an ignored digest. **A clean run emits NOTHING**; silence is safe only because the hub's deadline check (`monitor/deadline.go:396,417`) raises a missed backup from report freshness independently — if that is ever weakened this design loses its footing. **`run_id` is unique per real run** (so the hub's 1-h cooldown cannot collapse a manual run into the nightly one) and **deliberately EMPTY on the periodic refresh sweep**, which must stay under that cooldown or a polled status page becomes a mail flood | | `Manager.admitApp` / `beginAdmissionRun` / `decideAdmission` / `estimatedWriteBytes` (R-181, v0.193.0) | controller/internal/backup/admission.go | `(stackName) bool` / `() func()` | **THE reserve gate. Call it before ANY per-app backup write** — one verdict per app per run, covering the DB dump, the volume dump and the unit capture (all three write under one per-app root) | **Decided LAZILY at the app's first write, never once at run start** — app A's dump can put app B under the reserve, so a run-start verdict reads a disk that no longer exists. **Never re-decided between an app's own legs**: that is exactly the split R-181 closed (bulk written, capture refused). **Reset per run** via the closer `beginAdmissionRun` returns. **Must sit ahead of `DumpAppVolumesSafe`**, which stops the stack as its first act — a refusal decided inside it has already bounced the app. Fires **exactly one** `unitNotify` per refused app per run. Nil admission set (periodic status refresh) → decides fresh, which is still once per app per sweep. Wiring pinned by an **AST walk** in `TestAdmission_IsWiredIntoEveryProductionWriteLeg`, not `strings.Contains` | | `Manager.floorVerdict` + `FloorUsedPercent`/`FloorFreeGiB` / `ErrCaptureFloor` / `floorReason` (R-165 B2 v0.192.0, size term R-181 v0.193.0) | controller/internal/backup/recovery_unit.go | `(*UnitSpace, estGiB float64) (*UnitSpace, floorReason)` | The pure two-question predicate behind `admitApp`: is the filesystem already below the reserve (`floorHeadroom`), and would THIS app's write take it below (`floorSize`)? | **Headroom is about the FILESYSTEM, never a per-unit cap** — a size cap is R-163 rebuilt inside one volume; the size term bounds the *delta*, not the unit. **REFUSES, never deletes:** nothing here is generational (a unit is one fixed path per app, a DB dump one fixed name), so pruning could only destroy a DIFFERENT app's only local copy — **never repurpose `pruneStalePrimaryDirs`**, which removes ORPHANED dirs from an app that moved drives and has no notion of age. Two terms (97% / 1 GiB) in `fillwatch`'s shape, deliberately BEYOND its critical band (95% / 2 GiB) so the customer is always warned first — pinned by `TestFloorSitsBelowTheCriticalWarningBand`. **`estGiB == 0` degrades to headroom-only on purpose** — refusing an app with no history makes the FIRST backup the one that can never happen. A nil reading neither refuses nor warns (§8.4). Inject `unitSpaceFn` in tests rather than manufacturing occupancy on a real disk | | `fillwatch.Watcher` (`New`/`SetNotify`/`Check`) (R-167, v0.191.0) | controller/internal/fillwatch/fillwatch.go | `(statePath, logger, targetsFn, usageFn)` → `Check() error` | THE customer fill warning — warns BEFORE a filesystem fills, per FILESYSTEM (never per app: one full disk holding ten apps would fire ten times) | Emits the **pre-existing** `disk_warning`/`disk_critical` pair, which was allowlisted + copy'd + default-enabled with **no producer in any repo** until now — do NOT mint a new type beside it. **Two threshold terms, whichever trips first** (85% / 5 GiB; critical 95% / 2 GiB) because a percentage alone lies at both ends of this fleet's size range. **Edge-triggered on ESCALATION ONLY**, state persisted; de-escalation is silent and re-arms. Hysteresis dead zone between clear (75% / 7 GiB) and warn — pinned by `TestThresholdsKeepTheirHysteresisGap`. **A nil usage read is NEVER a warning** (§8.4). The hub has **no `customerMessages` entry** for either type on purpose — an entry would override the dynamic message and discard the drive label + free space | ### Scheduler / time / UI | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| | `Scheduler.Every` / `Daily` | controller/internal/scheduler/scheduler.go | `(name, interval/"HH:MM", fn)` | ALL background jobs | Daily is Europe/Budapest, DST-safe (`nextDailyRun` avoids Add(24h)); register in main.go block (§5) | | `Scheduler.UpdateDaily` | controller/internal/scheduler/scheduler.go | `(name, "HH:MM") bool` | Retime a daily job at runtime (no restart) | Per-job buffered `resched` chan + select case in `runDailyJob`; false (WARN) on invalid time / unknown-or-non-daily name; read `Schedule` under the mutex in the loop | | `backupwindow.*` (LegTimes / GateWindow / EffectiveWindow / ParseHHMM / FmtHHMM / Valid) | controller/internal/backupwindow/backupwindow.go | pure `string`↔`int` | Backup-window arithmetic (v0.168.0) | Offsets (W+60m/W+105m, gate W+2h..W+6h) are CONSTANTS — derived, never stored; wrap-safe modulo 1440; `EffectiveWindow(settings, yaml)` = settings>yaml>"02:30" | | `getBudapestLocation` | controller/internal/scheduler/scheduler.go | `() *time.Location` | Local-time math | web has its own `getTimezone` (§6); quiesce has its own `budapestLocation` (window gate) — 3rd copy, see §6 | | `Server.templateFuncMap` | controller/internal/web/funcmap.go | template.FuncMap | ALL template functions | `stateColor` outputs v2 suffixes `run/progress/warn/neutral/off`; stopped = NEUTRAL not red (operator-approved); `stateLabel` copy is frozen byte-identical (unit-tested) | | `i18n.Bundle` (`Shared` / `Msg` / `Plural` / `Expand`) | controller/internal/i18n/i18n.go | `Msg(lang, key) string` | ALL customer copy that has been converted (v0.247.0) | hu.json is authoritative; a key missing in en shows Hungarian (counted), never blank; `Expand` runs BEFORE template parse — never add a runtime `T` func (escaping would change Hungarian bytes). A template edit to converted copy goes in hu.json + en.json, then `TestI18nParity` must still pass | | `huTemplateSource` (test) | controller/internal/web/i18n_wiring_test.go | `(name) ([]byte, error)` | A test that asserts a template's Hungarian WORDING | Reads the expansion, not the raw file — since slice 1 the wording lives in hu.json. Never use `templateFS.ReadFile` for a wording assertion | | `Server.langFor` / `templatesFor` / `addLanguageData` | controller/internal/web/server.go + i18n_web.go | request → lang → template set | Rendering a page in the household's language | Only `executeTemplate` is language-aware; `s.tmpl` is the Hungarian set; Go-side titles via `data["TitleKey"]` | | `timeAgoStr` | controller/internal/web/funcmap.go | `(s RFC3339 string) string` | Ago-format for STRING timestamps | Exists because `timeAgo(time.Time)` 500'd on strings (v0.93 bug) | | `Server.baseData` / `executeTemplate` | controller/internal/web/handlers.go + server.go | page-data plumbing | New pages | baseData injects nav/alerts/version; templates must pass `controller/scripts/template_id_gate.py` + `controller/scripts/emoji_gate.py` | | `Server.RequireAuth` / `CsrfProtect` / `csrfField` | controller/internal/web/auth.go + csrf.go | middleware | Any new authed route/form | csrfField emits the hidden input; setup wizard has its OWN csrf (§6) | | `LogBuffer` + `Lines(maxBytes)` | controller/internal/web/logbuffer.go | ring buffer io.Writer | In-memory log capture for the debug UI + the report `controller_log_tail` source | v0.116.0: ALWAYS constructed (any logging.level) — the logger is `MultiWriter(LevelFilterWriter(stdout, level), ring)`; `Lines` drops OLDEST to honor the byte budget | | `logx.Debugf/Infof/Warnf/Errorf` | controller/internal/logx/logx.go | `(l *log.Logger, format, args…)` | ALL NEW leveled log lines (the v0.116.0 sweep standard) | routing is the WRITER's job — Debugf always reaches the ring, stdout filters; nil logger = no-op; caller-attributed (Output calldepth 3) | | `web.LevelFilterWriter` | controller/internal/web/levelfilter.go | `NewLevelFilterWriter(w, minLevel)` | stdout leveling under the always-on ring | untagged lines parse INFO; always reports full length written | | `monitor.RunHealthCheck` / `EffectiveProtected` | controller/internal/monitor/healthcheck.go | system health report | Health + protected-container list | — | | `util.TruncateStr` | controller/internal/util/strings.go | `(s, maxLen) string` | Rune-safe truncation | The intended shared helper; stacks still uses its byte-based twin (§6) | ## 2. Canonical patterns (copy structure from THE named file) | Pattern | Canonical file | Key traits | |---|---|---| | Agent-proxy web handler | controller/internal/web/agent_disk_handlers.go | memoized `s.agentClient()` → typed client call → `writeDiskJSON` envelope, Hungarian error strings, 502/503 mapping | | Two-step confirmed destructive op | controller/internal/web/storage_handlers.go `handleStorageWipe` | server-side type-to-confirm + probe(unconfirmed) → sentinel error → resubmit bound to agent durable-id; agent re-checks role regardless | | Crash-safe long job (journal + recover) | controller/internal/stacks/migrate.go | state machine + `writeJournal` per transition + `RecoverMigration` at startup + single-flight acquire/release + done-hook | | Quiesce/marker loop | controller/internal/quiesce/quiesce.go | marker BEFORE side effects, guaranteed undo (defer + max bound), `Recover()` once at startup, `TriggerNow` 409 single-flight | | Settings mutator | controller/internal/settings/settings.go (any Set*/Add*) | Lock → mutate → `s.save()`; getters return copies; never expose internal slices | | Channel-health checker w/ born-down alerting | controller/internal/channelhealth/checker.go | classify → debounce N≥2 → `alerted` flag re-armed on reason change (F2) | | Platform split | controller/internal/system/mounts_linux.go + mounts_other.go | `_linux.go`/`_other.go` twins; other = permissive no-op stubs for dev on Windows | | Debounced trigger + status (REFUSE-style — a too-soon fire is refused/lost) | controller/internal/sync/sync.go | `TriggerSync` 30s debounce, `Status()` snapshot struct, post-sync hook fan-out | | Coalescing trigger (trailing edge — a burst collapses but the LAST state always fires) | controller/internal/report/trigger.go | buffered-1 chan + non-blocking `Fire()` + single worker (quiet window → drain → min-interval → fire once); shape from hub `wgsync/reconciler.go` | | Detached job + status poll (single-flight, phase strings) | controller/internal/web/storage_init_job.go | acquire/release/set/**deep-copied** snapshot; phases mapped to Hungarian in the template; 1–3 s poll; terminal state **PROBED, not inferred**. Clones: `netstorage_job.go`, `samba_ensure_job.go` (v0.147.0). **Five of these now exist and agree on nothing — R-45 will unify them; prefer extending an existing one over a sixth** | | Streaming subprocess progress | controller/internal/backup/offbox_progress.go | `offboxStreamRunner` seam (stdout scanned line-by-line, stderr buffered, output tail-bounded) + a PURE line parser + a mutex-guarded published snapshot. Traps it encodes: a source reporting nothing is **normal** (restic sends 0 bytes for a whole incremental run) and the progress source may only update on unit completion — degrade bytes → files → current item + elapsed, never fake a percentage | | Post-start async verification | controller/internal/stacks/manager.go `logPostStartStatus` | goroutine + sleep, INFO log, never blocks/fails the operation | | Startup wiring order | controller/cmd/controller/main.go | init-only setters (`SetStackProvider` M2 contract: exactly once, before scheduler/HTTP), scheduler registration block | ## 3. Dangerous lookalikes — do NOT reuse | Trap | Why it bites | Use instead | |---|---|---| | `rsyncMirror` (controller/internal/backup/tier2.go) | `rsync -a --delete` — DESTROYS anything extra at dst; correct only for tier-2 mirror dirs (backup DIRECTION). In the tier2→live restore direction it would erase every live file created since the last copy | `rsyncCopy` + `rsyncVerify` (controller/internal/stacks/migrate.go) for any move/copy; `rsyncRestoreMissing` (controller/internal/backup/tier2_restore.go, `-a --ignore-existing`) for the additive-only restore direction | | raw `os.RemoveAll` on drive/HDD paths | Bypasses the protected-set; wipes appdata/backups/media | `Manager.DeleteStack`/`RemoveStack` (controller/internal/stacks/delete.go) — gated by `ProtectedHDDPaths` + orphan/protected/running checks | | fresh `agentapi.New` per request | Idle-conn leak → EADDRNOTAVAIL, port exhaustion (live incident, fixed ctrl v0.74.0) | `Server.agentClient()` memoized accessor | | `timeAgo` on an RFC3339 string field | Template 500 (OffboxTarget.LastRun bug, fixed v0.96.0) | `timeAgoStr` | | `backup.Manager.DumpAppVolumes` on a running DB app | Inconsistent tar of live DB volume | `DumpAppVolumesSafe` (stop → dump → restart, both errors surfaced) | | `stacks.Manager.execCommand` / `composeExecCustomEnv` for NEW long-running calls | No context/timeout — a hung docker CLI blocks forever | `exec.CommandContext` + explicit timeout (copy `rsyncCopy` or appexport `composeExecEnv`) | | `config.LoadPermissive` | Skips validation — setup-mode only (customer.id/domain may be unset) | `config.Load` everywhere else | | `ExportDataMounts` / `ParseComposeHDDMounts` as **backup-classification** input | `ExportDataMounts` unions the `${USERDATA_PATH}` ROOT (export-capture logic, not per-bind); `ParseComposeHDDMounts` resolves absolutes AND drops the `:ro` flag — classification needs `${VAR}`-relative paths + read-only awareness | `ParseComposeClassifiableBinds` (controller/internal/stacks/classify_binds.go) | | `docker compose restart` (any wrapper) | Does not pick up new images or env | `RedeployFromEnv` / composeExec `up -d` | | `backup.WholeOnTier` vs `UpdateCopyHolds` (R-659) | `UpdateCopyHolds` says what a copy HOLDS; it does not say the restore will ACCEPT it — a file app's second-drive copy holds the files and its unit restore still refuses | `WholeOnTier` (asks `DeclaredDriveFileLegs`, the refusal's own predicate) before a sentence names a copy as a way back | ## 4. Seams & interfaces (testing + cross-repo) | Interface | Defined in | Implemented by | Fakes/tests at | |---|---|---|---| | `diskAgent` | controller/internal/web/storage_handlers.go | `*agentapi.Client` | `mockAgent` in controller/internal/web/storage_handlers_test.go | | `netAgent` + `Server.netAgentFn/netProbeFn/netListFn` | controller/internal/web/netstorage_job.go (+ server.go fields) | `*agentapi.Client` / `runNetProbe` (linux re-exec) / `agent.ListNetStorage` | `fakeNetAgent` + fn injections in controller/internal/web/netstorage_job_test.go — the NAS add orchestration never shells/TLS-dials in tests | | `Server.agentLogsFn` (func seam) | controller/internal/web/server.go | nil → `agentClient().DebugLogs` (agent GET /debug/logs) | injected in controller/internal/web/observability_test.go (incl. the pre-0.83 typed-404 notice path) | | `escrowAgent` + `Server.escrowAgentFn/escrowStageFn/escrowStaleFn` | controller/internal/web/escrow_handlers.go (+ server.go fields) | `*agentapi.Client` / `PushOffboxPasswordForEscrow` / `report.EscrowAutoConfirmer.StaleBlob` (SetEscrowStale) | `fakeEscrowAgent` + fn injections in escrow_wizard_test.go — call-ORDER assertions (stage BEFORE trigger) + agent-never-called gates. The claim leg is the ONLY surface R crosses: no-store, never logged, never templated | | `offboxCeremonyWaitState` + `escrowCeremonyGraceWindow` | controller/internal/web/handlers.go | pure pick: (awaiting, timedOut) from `OffboxTarget.{EscrowState,CeremonyCompletedAt}` — the v0.138.0 "megerősítésre vár" card. Stamp SET on claim (escrow_handlers.go), CLEARED on the flip (main.go Flip + offbox_handlers.go manual confirm) | escrow_wait_state_test.go truth table (escrowed/unstamped/unparseable → plain CTA; boundary via `>=`) | | `Manager.sambaUpFn` / `sambaPasswdFn` / `sambaRunFn` / `sambaAddrFn` (func seams) | controller/internal/stacks/manager.go (fields) + samba.go | nil → `composeUp` / `docker exec smbpasswd` (STDIN) / `containerRunning("felhom-samba")` / `docker exec felhom-samba ip -4 -o addr show eth0` | injected in controller/internal/stacks/samba_test.go — the idempotency test asserts the up-seam is called **zero** times when config is unchanged; the passwd seam means no unit test ever handles a real secret or touches docker. **`sambaRunFn` has an EXPORTED setter (`SetSambaRunProbe`)** — internal/web's status-contract tests need a live-container world from another package. `sambaAddrFn` backs `SambaLANAddress()` (v0.151.0); its parse is separately pinned in samba_lanaddr_test.go and it returns "" on any failure — the page omits a line rather than printing a wrong address | | `volumeCopier` + `Manager.undoCopier` / `updateUndoHealthFn` (v0.263.0) | controller/internal/stacks/undo.go | nil → `dockerVolumeCopier` (alpine helper: `cp -a` named volume → `.pre-update-`, finished-marker LAST; restore re-checks the marker in the same shell) / nil → `waitUpdateHealthyMeta` with the OLD `.felhom.yml` | `fakeCopier` in controller/internal/stacks/undo_test.go — volume CONTENT as strings, so "the data came back" is a compare; a cut-off copy is a copy without its marker. **Judge a copy by the helper's own exit + the marker, never by the client** (killing `docker run` leaves the container copying — measured) | | `pgConverter` + `Manager.pgConv` / `convertFreeFn` (v0.273.0) | controller/internal/stacks/pgconvert.go | nil → `dockerPGConverter` (docker exec psql/pg_dumpall over 127.0.0.1 — the entrypoint's temporary init server listens on the socket only; `Empty` re-checks the undo copy's marker in the same helper) / nil → statfs of the stack dir | `fakePG` in controller/internal/stacks/pgconvert_test.go — works on `fakeCopier`'s volume strings. **Never convert without the ladder's `engine_conversion` mark** (`planEngineConversion` refuses a PostgreSQL major move without it); `isPostgresImage` must match `appbackup.dbTypeForImage` (pinned by a source-reading test) | | `Manager.composeExecFn` / `stacks.setupGateProbeGet` / `Server.gateClock` (v0.280.0) | controller/internal/stacks/manager.go + setup_gate.go · controller/internal/web/server.go | nil → the real compose call / an HTTP GET (5 s) / `time.Now` | controller/internal/stacks/setup_gate_test.go (with a docker STUB on PATH — R-650) · controller/internal/web/setup_gate_test.go | | `Manager.SambaLANAddress()` | controller/internal/stacks/samba.go | `() string` — the guest's LAN IPv4 for the Megosztás connect card (v0.151.0, S-2) | Read from the SAMBA container's netns (`network_mode: host`), never `net.InterfaceAddrs()` — the controller is on a docker BRIDGE and would answer 172.x (the same trap `setup.DetectLocalIPs` needs `HOST_IP` for). **NEVER cache/persist it** — the guest holds it by DHCP (S-5); callers re-derive per render. `""` = omit the line | | `Server.sambaAddrFn` (func seam) | controller/internal/web/server.go (field) + sharing_handlers.go `sambaLANAddress()` | nil → `stackMgr.SambaLANAddress()` | The web-side half of the connect card. Tests inject a COUNTED fn — the fresh-per-render assertion is what stops anyone memoizing a DHCP lease | | `Manager.guestNetExecFn` (func seam) + `GuestGateway()` / `GuestNetSnapshot()` | controller/internal/stacks/manager.go (field) + guestnet.go | nil → `docker exec felhom-samba ` — ONE seam for all R-66 guest-netns reads (route/link/addr/resolv.conf); tests script canned outputs per argv | guestnet_test.go. **The netns door rule:** the controller's OWN netns is the docker bridge, so any in-process read (`net.Interfaces`, `/proc/net/route`, its own `/etc/resolv.conf` = 127.0.0.11) is the S-2 wrong answer — guest-net reads MUST go through the samba (`network_mode: host`) exec door. Megosztás off ⇒ door closed ⇒ "" / per-item error strings; NEVER substitute an in-process value. Same S-5 law as SambaLANAddress: live per render, never cached/persisted. Parsers (`parseDefaultRoute`, `parseGuestInterfaces`, `parseResolvConf`) are pure + separately pinned | | `buildFileBrowserPaths` + `fbPathDeps` (R-67, v0.160.0) | controller/internal/web/handlers.go | pure assembly of one FileBrowser sync pass: (mount lines, config source paths) from the registry, with per-kind gates | filebrowser_network_test.go. **Two storage classes, two DIFFERENT gates:** drives keep the drive-absent gate + userdata scoping + skeleton (byte-identical to pre-R-67 — tested); network shares bind the share ROOT `:rslave` with the STUB gate instead (`classifyFSPath`; stub ⇒ excluded from mounts AND sources — an exposed stub swallows uploads the real mount later shadows; idle autofs / unknown ⇒ include, fail open). NEVER call `EnsureUserdataSkeleton` toward a network path (red-proven); never force-wake an idle trigger in the sync (doctrine) | | `Settings.RefuseAsAppNamespace` (R-108, v0.187.0) | controller/internal/settings/settings.go | `(path) (refuse bool, hungarianReason string)` — may an app's DATA NAMESPACE live here? | **THE single predicate for every placement surface** (deploy POST `api/router.go`, per-app migrate list + `handleStorageMigrateApp`, `handleStorageDecommission` mode=migrate TARGET). **Network storage is refused** because an app's namespace root IS its backup root (`namespaceRoot` returns a non-system drive path as-is → `/backups/primary//`), and on a share that lands inside FileBrowser's share-ROOT `download:true` bind — which CANNOT be narrowed (R-67 `:rslave` = automount wake; and apps on a share store at `/`, so there is no `userdata/` to scope to and creating one would write Felhom convention onto a customer's NAS). **DISTINCT from `refuseNetworkLifecycle`** — that asks "may a DRIVE lifecycle op run on this path" and is applied to the op's SUBJECT; this asks "may an app live here" and is applied to a placement TARGET. Migrate needs BOTH. **FAILS CLOSED:** `/mnt/felhom-drives` holds both kinds, so a path prefix cannot classify — `Kind` exists only on a REGISTERED path, therefore an unregistered path under that root is un-classifiable and REFUSES. Empty path = SSD-resident = allowed; nil receiver refuses. network_app_namespace_test.go, 4 red-proofs | | `Server.guestGatewayFn` / `guestNetFn` (func seams) | controller/internal/web/server.go (fields) + sharing_handlers.go accessors | nil → `stackMgr.GuestGateway` / `stackMgr.GuestNetSnapshot` | network_card_test.go — the counted-fn freshness test (2 renders ⇒ 2 resolves) is what stops anyone memoizing a DHCP lease; the Hálózati név row is gated on `smb.Enabled` (red-proven: gate dropped ⇒ \\FELHOM rendered while samba is down) | | `sambaEnsureState.consumeIfRunning()` | controller/internal/web/samba_ensure_job.go | serve-once `snapshot()` for terminal `running` only | `/sharing/status` carries a job EDGE (`phase`) and a service LEVEL (`running`) in one envelope — never let a level reach the phase channel, and never re-serve a consumed edge: the client answers `phase=="running"` with `location.reload()`, so both mistakes produce an infinite page reload (S-1/S-4, DIAG-sharing-2026-07-20.md). `failed`/`needs_password`/in-flight are NOT consumed | | `infra.SambaHostInterface` | controller/internal/infra/samba.go | the guest LAN nic name (`eth0`) | Single source for smb.conf's `interfaces =`, the container's `FELHOM_IFACE`, and the LAN-address read — if they name different nics, the service and the address the page prints drift apart | | `Manager.installedExecFn` (func seam, v0.233.0) | controller/internal/stacks/manager.go (field) + installed.go | nil → `defaultExecRunner` (`exec.CommandContext`, 30 s) | the installed-images recorder's ONLY process boundary — `docker compose ps` + `docker inspect` + `docker image inspect` | Deliberately NOT `execFn`/`composeExecCustomEnv`: those are already load-bearing elsewhere and **neither carries a context or a timeout**, and a bookkeeping read must never wedge a lifecycle action. Injected in controller/internal/stacks/installed_test.go, which FAILS the test on an argv it does not recognise, so a change to the commands issued cannot pass silently. The wiring test stubs the compose binary on PATH instead, so `RestartStack` is reached for real | | `Manager.sambaImgFn` (func seam) | controller/internal/stacks/manager.go (field) + samba.go | nil → `docker image inspect ` | drives the 4b card's pulling-vs-starting decision, which MUST be taken before `compose up` (afterwards the image is always present) | | `Manager.offboxStreamRunner` + `SetOffboxStreamRunner` | controller/internal/backup/offbox_progress.go | nil → `defaultOffboxStreamRunner` (real `restic`, stdout scanned live) | streaming sibling of `offboxRunner`; fakes emit canned `--json` status lines in offbox_progress_test.go, so the whole progress path runs with no restic, network or repo | | `Manager.offsitePreDumpFn` + `SetOffsitePreDumpFn` (R-44, v0.148.0) | controller/internal/backup/offbox_reconstitute.go (seam) + offbox.go (call site) | nil → `runDBDumpsInternal` under the SAME running flag | THE dumps-before-capture ordering seam. Extracted so the order is observable without Docker/restic — an ordering guarantee no test can see is one refactor from silently reverting to the DIAG-immich-restore-2026-07-19 behaviour. Red-proof: moving the capture first yields `[capture dump]` | | `Manager.offboxFullPlaceCopier` + `SetOffboxFullPlaceCopier` (R-43) | controller/internal/backup/offbox_reconstitute.go | nil → `rsyncRestoreOverwrite` (`-a --itemize-changes`; **no** `--ignore-existing`, **no** `--delete`) | **TRAP: do NOT reuse `offboxPlaceCopier` here.** The two copiers have OPPOSITE semantics for an existing file — `--ignore-existing` is exactly what a full restore must not do, and conflating them is how a missing-only merge came to be labelled a restore. Never `rsyncMirror` (`--delete`) in any restore direction | | `Manager.safetyDumpFn` + `SetSafetyDumpFn` (R-43) | controller/internal/backup/offbox_reconstitute.go | nil → `DumpOne` | the pre-restore undo. Invariant: the `pre-restore-`-prefixed dump must be verified ON DISK before anything is stopped/overwritten/replayed; failure ⇒ refuse with zero changes. Red-proof requires removing BOTH guards (the `err != nil` return and the `os.Stat`) — removing one leaves the other holding | | `reimportDBDumpsFrom(ctx, stack, dumpDir)` | controller/internal/backup/restore_db.go | explicit-dir sibling of `reimportDBDumps` (which passes `AppDBDumpPath`) | offsite reconstitution replays from the SCRATCH unit: the live unit is deliberately never overwritten, so replaying from it would replay the current DB over itself and restore nothing | | The DB-only replay window (R-47, v0.153.0) | controller/internal/backup/{offbox_reconstitute,restore_unit}.go | both restore paths: stop → place/volumes → `StartStackServices(dbServices)` → replay → `StartStack` (full) | **THE ordering invariant.** Replaying while the whole stack is up lets the app's own schema management race the dump — measured at 2 s on 2026-07-19 (H4), replay aborted `already exists`. Fail-closed: a dump with NO identifiable DB service refuses BEFORE the first mutation. Every exit from the window (replay error, DB-only start error) MUST still do a best-effort full start, or a failed restore becomes an outage. `hasReplayableDump` excludes `pre-restore-` safety dumps — counting them would arm the window for an app with nothing to replay | | `Manager.OffsiteScratchPair` / `OffsitePairInfo` | controller/internal/backup/offbox_reconstitute.go | reads the restored scratch unit's manifest (`offsite_run_id` / `dumps_at`) + the R-44 sniff | the confirm-dialog honesty surface. All warn-level: a pre-v0.148 (unstamped) pair and an empty-looking dump are SURFACED, never blocked — a false positive that refused a legitimate restore would be worse than the skew | | `appbackup.DumpValidation.LooksEmpty` (R-44 sniff) | controller/internal/appbackup/dbdump.go | computed in ValidateDump's existing single pass; `userTableNames` is EXACT-match | size and table count are both useless as emptiness heuristics (the 2026-07-19 dump: 52MB, 60+ tables, zero users — all geodata). **TRAP: never widen to a substring match on "user"** — it would flag `user_metadata` / `album_user` / `user_audit` on every healthy single-user box. A row wider than the read buffer still counts as a row | | `Manager.execFn` (func seam) + `restartPolicyLookup` / `inspectRestartPolicyFn` (R-51, v0.156.0) | controller/internal/stacks/manager.go | nil → real `exec.Command` / `docker inspect -f {{.HostConfig.RestartPolicy.Name}}` | `scriptedDocker` in controller/internal/stacks/degraded_test.go drives the WHOLE production path (docker ps → aggregateState → docker inspect) — an aggregateState-only test proves the function, not the caller. Policy answers are cached per container+state and pruned to the live `docker ps` set; a FAILED inspect is deliberately never cached (a hiccup must not pin a container to "unknown") and reads as SUPERVISED, i.e. fail-closed — the opposite of `IsDownState`'s fail-open, because there the state is ambiguous while here a member is known dead | | `bootrecon.StackProvider` (R-52, v0.156.0) | controller/internal/bootrecon/bootrecon.go | `*stacks.Manager` (GetStacks/StartStack/RefreshStatus) | `fakeStacks` counts StartStack per app; the load-bearing assertion is the NEGATIVE — a zero-container stack (a UI Stop = `compose down` = containers removed) must record **0** starts, while a boot orphan (containers present, Exited) records exactly 1. `Reconciler.sleep` is injected so the 30 s gap costs nothing | | `bootReconcileFn` + `runBootReconcile` (package-main seam, v0.156.0) | controller/cmd/controller/main.go | `bootrecon.New(mgr, logger).Run` | controller/cmd/controller/bootrecon_wiring_test.go. **The wiring itself is asserted by an AST walk** over `func main()`, not a `strings.Contains` — the substring version passed its own red-proof because a commented-out call still contains the string. Comments are not callers | | `classifyRunStates` (pure fix-3 derivation, v0.164.0) | controller/cmd/controller/main.go | `([]stacks.Stack, quiesced, failedRestart map[string]bool, now time.Time)` → `(dead []web.DeadApp, states []notify.AppRunState)` | classify_runstates_test.go. **THE single fix-3 rule: down = `(IsDownState(st.State) || st.CrashLooping(now)) && !userStopped && !quiesced`.** **`quiesced` is a UNION of TWO suppression sets** (R-330, v0.224.0): `quiesce.Loop.SuppressedStacks()` (whole-guest vzdump/PBS) and `backup.AppStopGuard.SuppressedStacks()` (per-app volume dump / offbox reconstitute / `.fab` export), merged by `unionSuppressed` in `scanDeployedAppRunStates`. **Adding a third way to stop an app means adding its set here** — R-330 was 61 false customer e-mails caused by exactly that omission, with a working suppressor sitting three lines away. C9-F2 (v0.183.0) added the crash-loop term: `restarting` is NOT in `IsDownState` and must not be — adding it alarms on every deploy and update fleet-wide — so a SUSTAINED restarting run (`stacks.crashLoopAfter` = 5 m, above the 120 s deploy timeout, Mealie's 60 s start_period AND R-97b's 180 s grace) becomes down instead. `now` is injected so the threshold is a testable contract. A deliberate UI stop (`compose down` → zero containers → StateStopped, I1) must not alarm — banner OR email — while faults (Exited/Degraded) alarm byte-identically; I2 (P2 census: all catalog services `unless-stopped`) is why a crash never rests at stopped. **Do NOT touch `IsDownState`** (other callers rely on stopped=down) and do NOT filter in `buildDeadAppAlerts`/`NotifyAppStartFailures` — one derivation point. If I1 or I2 changes, revisit the suppression | | `report.SetPendingControllerLog` / `SetControllerLogSource` | controller/internal/report/selftail.go | ACK-armed consume-once self-log pull (the logtail.go shape) | selftail_test.go; source = `logBuffer.Lines`, wired once in main.go | | `util.ParseVersion` / `util.Version.Compare` | controller/internal/util/version.go | THE one semver comparator (house rule: never a second) — selfupdate aliases it; agentapi's MinAgent comparison uses it | rejects pre-release/dev/latest (callers fall back, never trust); numeric compare (0.100 > 0.81) | | `agentapi.AgentVersionReporter` + `featureMinAgent` | controller/internal/agentapi/features.go | version-first Supports (v0.82.0 header channel); probe = fallback for header-less agents | a coupled feature adds BOTH a featureProbes row AND a featureMinAgent row; v0.116.0: `SupportsWithSource` also reports HOW the verdict was reached (version/probe-cache/probe) for the gate log line | | `netProbeReadBack` (package var) | controller/internal/web/netprobe.go | `os.ReadFile` | overridden in TestNetProbeChild (nonce-tamper + cleanup-fail rows); package var because the child is a RE-EXEC'd process in production | | `system.ClassifyPathFS(Timeout)` + `netProbeFSClass` / `Server.classifyFSPath` / `Router.classifyFSPath` | controller/internal/system/fsclass*.go (+ web/netprobe.go, web/server.go, api/router.go seams) | statfs f_type → network/autofs/stub/unknown in THIS namespace (RCA fix 2) | idle autofs = HEALTHY, never force-mount; unknown = fail OPEN; seams injected in netprobe_stub_test.go / networkstub_test.go / deploygate_test.go | | `quiesce.Backend` / `quiesce.Stacks` | controller/internal/quiesce/quiesce.go | adapter over `*agentapi.Client` / `*stacks.Manager` | `fakeBackend`/`fakeStacks` in controller/internal/quiesce/quiesce_test.go | | `channelhealth.Probe` (func) + `Sink` | controller/internal/channelhealth/checker.go | `Server.ProbeAgentChannel` / notifier adapter | `fakeSink` in controller/internal/channelhealth/checker_test.go | | `appbackup.StackDataProvider` | controller/internal/appbackup/appdata.go | `*stacks.Manager` (via `backup.SetStackProvider`) | `fakeRecoveryProvider` in controller/internal/backup/recovery_unit_test.go | | `appexport.ExportStackProvider` | controller/internal/appexport/provider.go | `*stacks.Manager` | exercised in appexport tests | | `selfupdate.AgentSwapper` | controller/internal/selfupdate/updater.go | `*agentapi.Client` (SwapController/SwapStatus) | `fakeAgent` in controller/internal/selfupdate/updater_test.go | | `mailrelay.Forwarder` | controller/internal/mailrelay/forward.go | `HubForwarder` (hub relay endpoint) | `fakeForwarder` in controller/internal/mailrelay/mailrelay_test.go | | `integrations.Handler` + `StackProvider` | controller/internal/integrations/integrations.go + manager.go | OnlyOffice handlers | table-driven tests in package | | `bootstrap.PullFunc` | controller/internal/bootstrap/bootstrap.go | `report.PullConfig` | injected in bootstrap tests | | `offboxRunner` (func) | controller/internal/backup/offbox.go | `defaultOffboxRunner` (restic exec) | `SetOffboxRunner` injection point | | `dumpVolumesSafe` (func seam) | controller/internal/backup/backup.go | nil → real `DumpAppVolumesSafe` | injected in controller/internal/backup/volume_dumps_test.go (gating tests without Docker) | | `generateSecret` (func seam) | controller/internal/backup/backup.go | `stacks.Manager.GenerateSecretForField` via `SetSecretGenerator` (main.go) | injected in controller/internal/backup/restore_secrets_gen_test.go | | `restoreFilesCopier` (func seam) | controller/internal/backup/backup.go | nil → real `rsyncRestoreMissing` | injected in controller/internal/backup/tier2_restore_test.go (orchestration without rsync) | | `tier2Mirror` (func seam) | controller/internal/backup/backup.go | nil → real `rsyncMirror` | both RunTier2 rsync legs; injected in controller/internal/backup/tier2_test.go (resolve→mirror without rsync) | | `migSeams.resolveNames` (func seam) | controller/internal/stacks/migrate.go | nil → real `ResolveAppDataDirNames` (compose-derived) | injected in controller/internal/stacks/migrate_fs3_test.go (F-S3 appdata dir-name resolution) | Cross-repo edges: - `controller/internal/agentapi/client.go` ↔ **felhom-agent** local API (`/storage`, `/disks*`, `/backup*`, `/netstorage*`, `/guest/*`): pinned leaf SHA-256 + per-guest bearer token from bootstrap.json. - `controller/internal/report/pusher.go` → **hub** `/api/v1/report` ingest; ACK `config_version` drives config_refresh.go; `controller/internal/notify/notifier.go` → hub `/api/v1/event` (hub-side `allowedEventTypes` allowlist must include new types). - `controller/internal/sync/sync.go` ← **app-catalog-felhom.eu**: copies ONLY `docker-compose.yml` + `.felhom.yml` per app (SHA-256 change detection); NEVER overwrites `app.yaml` (deployed secrets). ## 5. Extension points (where new features plug in) - **New storage web endpoint**: switch in `ServeStorageAPI` (controller/internal/web/storage_handlers.go); disk ops in `ServeDiskAPI` (controller/internal/web/agent_disk_handlers.go); backup in `ServeBackupAPI`; export in `ServeExportAPI`; debug in `handleDebugAPI` (debug-mode gated). - **A gate that must REPORT and never REFUSE (R-404, 2026-09-01)**: register it in `controller/scripts/controller_gates.py` with the fifth field `blocking=False` — its exit code then never reaches the runner's verdict and it prints as `ADVISORY`. `golden_notice.py` is the only one, and `test_golden_notice.py` asserts it stays the only one. **Use this instead of giving a notice the power to refuse a push**: the golden notice must fire at the moment a release is committed, when the golden legitimately cannot exist yet. A cross-repo gate IMPORTS the sibling's script (see `golden_notice.py` loading `felhom.eu/scripts/golden_currency_gate.py`) and never copies it — a copy recreates the drift these gates exist to detect. - **New debug-page control (R-400, v0.228.0)**: the control in `controller/internal/web/templates/debug.html` AND the `subpath == "…"` case in `controller/internal/web/handler_debug.go` are ONE change — `controller/scripts/debug_route_gate.py` fails on either half alone, in both directions (a reference with no case, and a case with no reference). Keep the dispatcher's EXACT-match switch with its `http.NotFound` default; a prefix match is what would have hidden the original defect. Handler shape: `debugTriggerDBDump` for a fire-and-forget trigger, `debugRunIntegrityCheck` for one the operator pressed to learn an ANSWER (synchronous). **Before this gate the page referenced 24 addresses and 17 were answered, and three of the seven dead ones fetched on page LOAD** — those panels were permanently blank on the page an operator opens when something is already wrong. - **New REST endpoint**: path dispatch in `Router.ServeHTTP` (controller/internal/api/router.go); use `writeJSON` + `limitBody`. - **New background job**: `sched.Every`/`sched.Daily` registration block in controller/cmd/controller/main.go. - **New template function**: `Server.templateFuncMap` (controller/internal/web/funcmap.go) — obey v2 state-suffix vocabulary. - **New page/nav item**: `baseData` + sidebar in controller/internal/web/templates/ (nested sub-links pattern `.nav-links-nested`); must pass `controller/scripts/template_id_gate.py` + `controller/scripts/emoji_gate.py` + `controller/scripts/native_confirm_gate.py` + `controller/scripts/offbox_rename_gate.py` + `controller/scripts/app_row_dedup_gate.py` + `controller/scripts/mojibake_gate.py`. - **Docker volume tar streaming (v0.125.0)**: `appexport.dockerExec` (seam, package var) + `withVolumeHelper`/`exportVolumeTar`/`importVolumeTar` — stream volume content via `docker cp` through a stopped helper container. NEVER `docker run -v ` — the daemon resolves `-v` host-side and strands the data when the controller is containerized (the v0.124.0 HIGH finding); `controller/scripts/docker_run_volume_path_gate.py` enforces (every `"-v"` allowlisted with its WHY). - **Guarded file download (v0.124.0)**: `handler_export_download.go` — the canonical shape for streaming a server-side file to the browser: accept a BASENAME only (shape regexp + no separators/`..`), `filepath.Join` then assert `filepath.Dir(path) == dir`, `io.Copy` (never ReadAll), `Content-Disposition: attachment`, remove after a successful stream, TTL sweep (`sweepFabDownloads(dir, now, maxAge, logger)` — now injected for tests). Red-proof the guard by loosening to prefix-matching (the `..` case must fail). - **Backups sub-page data**: `backupsCommonData(page, title, r)` + `backupsOffboxData(data)` (handlers.go) — the ONLY builders for the four `/backups*` pages; a new backups section extends these, never re-derives in a page handler. (The one-shot v0.124.0 move gate `backups_split_move_check.py` was retired in v0.126.0.) - **App-list row (v0.126.0)**: `app_list_row`/`app_list_row_end` in `controller/internal/web/templates/app_row.html` is THE canonical list pattern — icon+name(+secondary) left, caller action block right; open with `dict "Slug" ... "Name" ...` (optional `Secondary`/`RowClass`/`Href`/`FallbackIcon`), close with `app_list_row_end`. Do NOT hand-roll app rows — `controller/scripts/app_row_dedup_gate.py` enforces single-sourcing (the backups_apps expander header is the one allowlisted aligned copy). Infra display identity: `inframeta.go` map + `infraMeta` func (filebrowser is the only Linked stack). - **Consequential-action confirm (LIGHT)**: `felhomConfirm(el, question, onYes)` in layout.html (v0.123.0) — the trigger swaps in place to "kérdés + Igen/Mégse"; form buttons opt in with `data-confirm="…"` (delegated listener, `requestSubmit` keeps formaction/name-value). NEVER native `confirm()`/`prompt()` (OS-modals freeze browser automation — drill F-11; `native_confirm_gate.py` enforces). Heavy destructive flows keep the `.confirm-overlay` `openDialog` pattern. - **New hub event**: typed `Notify*` wrapper on Notifier + hub allowlist entry (cross-repo). - **New app integration**: `integrations.Manager.RegisterHandler` with `IntegrationKey(provider, target)`. - **New startup self-check**: append check fn in `selftest.Run` (controller/internal/selftest/selftest.go). - **Converting a template to the bundle (i18n, v0.247.0)**: `python3 controller/scripts/i18n_extract.py