docs: centralize controller documentation + top-level index (code-verified, v0.59.0)

New documentation/controller/ subtree (module map + deploy/stack-lifecycle, backup,
storage/monitoring/metrics, auth/hub/sync/integrations) grounded in current source;
top-level documentation/README.md index across controller/agent/platform/hub/audits;
REORG-NOTES with the verification ledger + flagged doc-gaps. Supersedes (keeps) the
v0.33 controller planning map. Additive only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

documentation/controller/README.md

@@ -0,0 +1,35 @@
+# Felhom Controller — Documentation
+
+The **in-guest controller** (`felhom-controller`): one per customer LXC, Docker-only, Hungarian web
+dashboard. It manages the customer's app stacks and app-data backups, reports to the hub, and delegates
+all Proxmox/disk operations to the host agent. Current version **v0.59.0**.
+
+These docs are **code-verified against current source** (`felhom-controller/controller/internal/`) and
+are the authoritative architecture reference. The repo-local `controller/README.md` is a build/dev
+quickstart that points here; operational working files (`CLAUDE.md`, `CONTEXT.md`, `CHANGELOG.md`,
+`BUGHUNT.md`) stay in the controller repo.
+
+## Index
+
+- [module-map.md](module-map.md) — per-package map of the current controller (supersedes the v0.33
+  planning map in `../architecture/02-controller-module-map.md`).
+- [deploy-and-stack-lifecycle.md](deploy-and-stack-lifecycle.md) — stack model, the deploy flow
+  (incl. the v0.59 crash-safe `deployed` persistence + fail-closed secret encryption), protected
+  stacks, base-infra bring-up (`EnsureBaseStack`).
+- [backup-architecture.md](backup-architecture.md) — app-data backup: DB dumps, per-app recovery
+  units, restore + the fail-closed data-key gate, Tier-2 off-drive copies, and the controller↔agent/PBS
+  split for whole-guest backup.
+- [storage-monitoring-metrics.md](storage-monitoring-metrics.md) — storage registry, disk topology &
+  operations delegated to the agent, the v0.58 Docker-data headroom prevention layer, host metrics,
+  the SQLite metrics store, and health monitoring.
+- [auth-hub-sync-integrations.md](auth-hub-sync-integrations.md) — auth/CSRF/sessions, the pre-auth
+  setup wizard + bootstrap ingestion, hub reporting & notifications, catalog sync, app-to-app
+  integrations, geo-restriction, self-update, and asset sync.
+
+## Cross-references
+
+- Host agent architecture: [`../architecture/03-host-agent.md`](../architecture/03-host-agent.md)
+- Control-plane authorization (signing/escrow): [`../architecture/04-control-plane-authorization.md`](../architecture/04-control-plane-authorization.md)
+- Hub: [`../architecture/05-hub-architecture.md`](../architecture/05-hub-architecture.md)
+- Topology & trust: [`../architecture/01-topology-and-trust.md`](../architecture/01-topology-and-trust.md)
+- Security audits & remediation: [`../audits/`](../audits/)

documentation/controller/auth-hub-sync-integrations.md

@@ -0,0 +1,121 @@
+# Controller control-plane & app-domain glue
+
+Source of truth: felhom-controller `internal/web` (auth/csrf/setup), `internal/report`, `internal/notify`, `internal/sync`, `internal/integrations`, `internal/cloudflare`, `internal/selfupdate`, `internal/assets` at v0.59.0.
+
+This document covers the in-guest controller's link to the operator control plane (the hub), its first-run onboarding, and the app-domain glue (catalog sync, app-to-app integrations, geo-restriction, self-update, asset sync). Every claim below is verified against current source; file:line citations are in the verification ledger of the implementing task.
+
+---
+
+## 1. Auth & sessions
+
+The web tier is gated by two middlewares, `RequireAuth` and `CsrfProtect`, both methods on `web.Server`.
+
+**Password & hash priority.** `effectivePasswordHash()` resolves the active hash in priority order: (1) `settings.json` `password_hash` (customer-changed), then (2) `controller.yaml` `web.password_hash` (operator-provisioned). Empty from both sources means no auth. Hashes are bcrypt (`bcrypt.DefaultCost`); login compares with `bcrypt.CompareHashAndPassword`.
+
+**Sessions.** On successful login `createSession()` mints a 256-bit random hex session token AND a separate 256-bit hex CSRF token, stored server-side in an in-memory map (`s.sessions`). The session cookie is `felhom_session`, `MaxAge` 7 days, `HttpOnly`, `SameSite=Lax`, and `Secure` only when the request arrived over TLS or carried `X-Forwarded-Proto: https`. A background goroutine sweeps expired sessions every 15 minutes; `invalidateAllSessions()` clears them all after a password change. There is no server-side persistence — a controller restart drops all sessions (re-login required).
+
+**Demo / no-password open mode (verified).** When `authEnabled()` is false (no hash from either source), `RequireAuth` passes every request through, and `CsrfProtect` returns before any token check. So a controller with no dashboard password is fully open — both auth and CSRF are skipped. This is the demo posture (guest 9201). `/api/health` is always exempt from auth even when enabled.
+
+**CSRF.** Double-submit using the per-session CSRF token. Safe methods (GET/HEAD/OPTIONS) pass through. For unsafe methods, the submitted token is read from the `_csrf` form field first, then the `X-CSRF-Token` header, and compared to the session's stored token with `subtle.ConstantTimeCompare`. Two exemptions: auth disabled (above), and a valid `Authorization: Bearer <api_key>` header — the bearer is constant-time-compared against `cfg.Hub.APIKey` and, if it matches, CSRF is skipped (this is how the hub's signed callbacks and API-key clients reach mutating endpoints). An invalid bearer falls through to normal CSRF validation. Rejections return JSON 403 for `/api/` paths, plain text 403 otherwise.
+
+**Login rate-limit.** Per-IP: max 5 failed attempts in a 1-minute sliding window; the 6th within the window is refused with a Hungarian "too many attempts" message. The window resets after a minute of no failures, and a successful login clears the counter for that IP. **XFF caveat:** the rate-limit key (and login-failure logging) takes the client IP from the first `X-Forwarded-For` entry when present, falling back to `RemoteAddr`. Behind the per-guest reverse proxy this is the real client; a spoofable upstream XFF would let an attacker rotate the key, so the limit assumes a trusted proxy sets XFF.
+
+---
+
+## 2. Setup wizard + first-run bootstrap
+
+Both run **pre-auth** — there is no password yet. They are mutually exclusive paths to a configured `controller.yaml`.
+
+**Gating.** `setup.NeedsSetup(cfg)` is true when `cfg.Customer.ID == ""`, or when a `.needs-setup` marker file exists in the data dir (a debug re-trigger). `main.go` calls `bootstrap.MaybeIngest(...)` first, then checks `NeedsSetup`; if still unconfigured it launches the setup server instead of the normal app.
+
+**Bootstrap (`internal/bootstrap`, schema `felhom.bootstrap/v2`).** The host agent's provisioning back-half writes a read-only `bootstrap.json` (default mount `/etc/felhom-bootstrap/bootstrap.json`, overridable via `FELHOM_BOOTSTRAP_PATH`). It carries ONLY: `customer.id`, `hub.url`, a per-customer `hub.retrieval_password` (secret), and the `local_api` block (`endpoint`, `fingerprint`, `token`; token secret). On an unconfigured controller, `MaybeIngest`:
+
+1. Is idempotent — if `cfg.Customer.ID` is already set, it never reads the file, never pulls, never clobbers `controller.yaml`.
+2. Fail-safe — an absent/malformed file, a non-v2 schema, a missing required field, a missing pull function, or a pull that ultimately fails all leave cfg unchanged and log a WARN; the controller falls back to the setup wizard. A hub outage at first boot must not brick the guest.
+3. On success, **pulls** the full `controller.yaml` from the hub (`PullFunc`, wired to `report.PullConfig`) authenticated by the retrieval passphrase, **merges** the per-guest `local_api` block into the pulled YAML at the map level (the hub never knows per-guest Proxmox internals), writes `controller.yaml` atomically at 0600, reloads it, and comes up configured — skipping the wizard.
+
+Transient pull failures (`ErrPullTransient`, wrapping `report.ErrHubUnreachable`) are retried with a 2s/4s/8s backoff (4 attempts total); permanent failures (auth/not-found) fail fast.
+
+**Setup wizard (`internal/setup`, pre-auth HTTP server).** Offered when bootstrap did not configure the guest. The wizard server (`setup.NewServer`) serves routes with no auth middleware (`/setup`, `/setup/fresh`, `/setup/manual`, `/setup/failed`). Disk-recovery wizardry was removed in slice 8C; today there are two modes:
+
+- **Fresh-from-hub** (`processFreshHub` / `autoProcessFreshHub`): the customer (or pre-seeded env vars `FELHOM_SETUP_CUSTOMER_ID` / `FELHOM_SETUP_PASSWORD` from `docker-setup.sh --hub-customer`) supplies customer-id + retrieval passphrase; `report.PullConfig` downloads the generated `controller.yaml`; `writeFreshConfig` writes it.
+- **Manual** (`processManual`): `generateManualConfig()` builds a `controller.yaml` from form fields locally (no hub call).
+
+Both end in `writeFreshConfig`, which writes `/opt/docker/felhom-controller/controller.yaml` atomically at 0600, and — into `settings.json` — bcrypt-hashes the dashboard password (if the form supplied one) via `SetPasswordHash`, and stores the retrieval passphrase via `SetRetrievalPassword`. Wizard progress is persisted to `setup-state.json` (atomic write) so a browser crash mid-wizard survives.
+
+---
+
+## 3. Hub reporting & notifications
+
+The controller reports **state**, never app secrets — consistent with the zero-knowledge posture (the hub holds only non-secret config plus a wrapped PBS key it can't decrypt; app secrets live encrypted on the guest rootfs, never sent to the hub).
+
+**Report (`report.BuildReport` → `report.Pusher`).** `BuildReport` assembles a snapshot: customer id/name, controller version, a SHA-256 hash of the on-disk `controller.yaml` (for hub drift comparison, not the content), controller URL (`https://felhom.<domain>`), system info (host/OS/kernel/CPU/mem/temp/load), storage (root fs + each registered user-data path, with decommissioned/disconnected markers), container counts + per-container cpu/mem, app-data backup status (DB-dump last-run only — disk-tier restic/snapshot fields are zero since that moved to the agent in slice 8C), health (status/issues/warnings), deployed/available stack lists, per-app telemetry, and geo-restriction status. **No app secrets, env, or volume contents are in the report.**
+
+`Pusher.Push` POSTs the JSON to `<hub>/api/v1/report` with `Authorization: Bearer <hub.api_key>`, retrying 3 times with 5s backoff; it tracks last-attempt/last-success/consecutive-failures and parses the response for a `customer_blocked` flag (relayed via `OnPushResponse`). Cadence: `sched.Every("hub-report", cfg.Hub.PushInterval)`, default 15m when unparseable, only when `cfg.Hub.Enabled`. A separate `PushInfraBackup` POSTs to `/api/v1/infra-backup`; `PushOnce` sends a single report regardless of the enabled flag (e.g. a reporting-disabled startup notice).
+
+**Notifications (`notify.Notifier`).** Structured events POSTed to `<hub>/api/v1/event` with the bearer key. Enabled only when both hub URL and API key are set. Sends are non-blocking (goroutine), 3 attempts with 3s backoff; **cooldown/dedup is the hub's job** — the controller sends unconditionally. Event types include `controller_started`, `controller_updated`, `backup_completed`/`backup_failed`, `db_dump_completed`/`db_dump_failed`, `backup_integrity_ok`/`backup_integrity_failed`, `crossdrive_completed`/`crossdrive_failed`, `health_critical`/`health_degraded`/`health_recovered` (driven by status-rank change detection), `storage_disconnected`/`storage_reconnected`, `app_deployed`/`app_removed`, `disaster_recovery_started`/`disaster_recovery_completed`, plus `test`. Each event carries `customer_id`, severity, a Hungarian message, and an optional typed `details` object. A 50-entry ring buffer (`GetEventHistory`) records each send (type/severity/message/hub-status/error) for the debug page. `SyncPreferences` POSTs email + enabled-event list + cooldown to `/api/v1/preferences` (synchronous). A legacy `Notify` → `/api/v1/notify` path is retained for old hubs.
+
+---
+
+## 4. Catalog sync (`internal/sync`)
+
+A periodic git pull of the app catalog into the local stacks dir. `Syncer.Start` clones (`--depth 1 --branch <branch>`) into `<dataDir>/catalog-cache` on first run, else `fetch --depth 1` + `reset --hard origin/<branch>`; it runs an initial sync on startup and then on `cfg.Git.SyncInterval` (default 15m). Disabled (manual mode) when `cfg.Git.RepoURL` is empty. `TriggerSync` debounces to once per 30s. Credentials, if configured, are injected into the HTTPS URL and **masked in all logs**.
+
+**Safety — never overwrites app.yaml.** `copyTemplates` syncs only `docker-compose.yml` and `.felhom.yml` per app directory; `app.yaml` (the per-guest deployed config carrying encrypted secrets) is never in the sync set and is never touched. Copies are **content-hash gated** (`copyIfChanged` / `copyTemplates` compare SHA-256 of src vs dst and skip identical files), so an unchanged catalog produces no writes and no USB thrash. New app dirs are reported as `new_apps`, changed ones as `updated`; a rescan + a post-sync hook (missing-deploy-field injection) fire only when something changed.
+
+---
+
+## 5. App-to-app integrations (`internal/integrations`)
+
+Lets one app configure another (e.g. enable OnlyOffice editing inside FileBrowser).
+
+**Adapter pattern.** `integrations.Manager` depends on a `StackProvider` interface (`GetStack`/`GetStacks`/`RestartStack`); `integrationStackAdapter` in `main.go` bridges `stacks.Manager` to it, breaking the import cycle. Handlers are keyed `"provider:target"` and registered at construction. Integration enable/disable state lives in `settings.json` (`SetIntegrationState`/`GetIntegrationState`); apply/revoke are serialized under a mutex.
+
+**Concrete handlers (registered in `NewManager`):**
+- `onlyoffice:filebrowser` — `Apply` reads OnlyOffice's `JWT_SECRET` and subdomain from the provider's decrypted `app.yaml` env, then patches FileBrowser's `config.yaml` with an `integrations.office` block (public office URL, internal `http://onlyoffice:80`, the shared JWT secret) and restarts FileBrowser. `Revoke` strips that block and restarts. Writes are atomic (tmp+rename). FileBrowser is treated as always-present infrastructure (not gated on "deployed").
+- `onlyoffice:nextcloud` — patches Nextcloud via occ commands (separate handler).
+
+**Lifecycle hooks.** `Toggle` validates that provider (and target, unless it's filebrowser) are deployed and running before applying. The manager exposes:
+- `OnStackStop` — revokes active integrations where the stack is provider or target, but keeps `enabled=true` and marks status `provider_stopped`/`target_unavailable` so they can come back.
+- `OnStackStart` — after a 5s delay (so the 30s stack-state refresh catches up), re-applies previously-enabled integrations once both sides are running/starting.
+- `OnStackRemove` — permanently revokes (best-effort) and deletes integration state for the removed stack.
+
+`ReapplyConfigForTarget` re-applies config without restarting (the caller, e.g. FileBrowser mount sync, handles restart).
+
+---
+
+## 6. Geo-restriction (`internal/cloudflare`)
+
+The controller enforces country-based access via Cloudflare custom-WAF rules using a CF API token; this is the controller's own CF-API geo layer (distinct from any hub-side config).
+
+**Rule identity.** All managed rules carry the `[felhom-geo]` description prefix — the global rule is exactly `[felhom-geo] Global`, per-app rules are `[felhom-geo] app:<name>`. `GetFelhomRules` filters the zone's `http_request_firewall_custom` ruleset to that prefix, so the controller only ever touches its own rules. Rules use the `block` action (plain 403; custom response bodies need a paid plan). Expressions are built from allowed ISO country codes: global = `(not ip.src.country in {...})` with per-app hostnames excluded; per-app = `(http.host eq "<host>" and not ip.src.country in {...})`. Zero allowed countries means block-all.
+
+**Sync.** `GeoSyncManager.Sync` resolves zone-id and ruleset-id (creating the custom ruleset if absent, caching ids in settings), lists existing felhom rules, builds the desired set from `settings.GeoRestriction` + deployed hostnames (via the `StackLister`/`geoStackAdapter`), and diffs: create new, update changed-expression, delete obsolete. When geo is nil or disabled, `deleteAllRules` removes every felhom rule. Sync state (zone/ruleset ids, last error) is saved back to settings.
+
+**Cadence & hooks.** A delayed (15s) non-blocking initial sync at startup; periodic `geo-verify` every 6h; and `OnGeoRelevantChange` (wired to the API router) re-syncs on app deploy/remove — each guarded so it only runs when geo is enabled.
+
+---
+
+## 7. Self-update (`internal/selfupdate`)
+
+**Version check.** `CheckForUpdate` queries the Gitea Docker Registry V2 tags API (`/v2/<owner>/<repo>/tags/list`) with basic auth from the git credentials, parses tags as semver (skipping `latest`/`dev`/non-semver), and reports the highest. A `dev` running version can't check. Scheduled as `selfupdate-check` on `cfg.SelfUpdate.CheckInterval` (default 6h) — **check only, never triggers an update**. The running version is injected at build time via ldflags (`-X main.Version`); the source default is `"dev"`.
+
+**Update mechanism (compose-managed path).** `TriggerUpdate` guards against concurrent updates, dev versions, an in-progress backup (`backupRunning` callback), and an inaccessible compose file, then runs `performUpdate` in a goroutine: write `pending` state → `docker pull <image>:<version>` → rewrite the `image:` line in `docker-compose.yml` (atomic tmp+rename, regex-pinned to `gitea.dooplex.hu/admin/felhom-controller:<tag>`) → `docker compose up -d` (which replaces the running container). On next boot, `VerifyStartup` reads the pending state and marks it `success` if the running version equals the target, else `failed`. Auto-update is a separate daily job (`selfupdate-auto` at `cfg.SelfUpdate.AutoUpdateTime`) gated on `cfg.SelfUpdate.AutoUpdate`.
+
+**Never `:latest`.** The image reference is always `<image>:<version>` with a concrete semver tag — both the registry query and the compose rewrite pin an explicit version; `latest`/`dev` tags are explicitly skipped during version selection.
+
+> Deployment note: on the bootstrap-managed golden guest (e.g. 9201) the controller has no `docker-compose.yml` — it is started by `felhom-controller-bootstrap.service` from `/etc/felhom-controller-image`. There the compose-rewrite self-update path does not apply; version drift is handled by the bootstrap service pulling the pinned tag. The compose path above is the `/opt/docker/felhom-controller` deployment model.
+
+---
+
+## 8. Asset sync (`internal/assets`)
+
+Branding/app assets are sourced from the hub with a baked-in fallback. `Syncer.Sync` GETs `<hub>/api/v1/assets/manifest` (bearer key), compares each manifest entry's SHA-256 against the local cache (`<dataDir>/assets`), downloads only new/changed files (atomic tmp+rename), and removes local files absent from the manifest; it saves a local copy of the manifest. Cadence: an initial sync on startup plus a daily `asset-sync` job at `cfg.Assets.SyncSchedule`. `Resolve(filename)` returns the cached path if present, else the baked-in fallback (`/usr/share/felhom/assets`), else the cached path (so the caller emits a clean 404). Filenames are base-sanitized to prevent path traversal.
+
+---
+
+## Cross-references
+
+- Hub API contract and zero-knowledge config generation: hub `internal/api`, `internal/configgen` (felhom.eu repo).
+- Agent-side bootstrap emission and provisioning: `felhom-agent` `internal/provision` / `internal/localapi`.
+- Backup/recovery-unit and Tier-2 mechanics: see the controller backup documentation.

documentation/controller/backup-architecture.md

@@ -0,0 +1,109 @@
+# Controller backup architecture
+
+Source of truth: felhom-controller `internal/appbackup/`, `internal/backup/`, `internal/quiesce/` at v0.59.0; whole-guest backup is the agent's (`felhom-agent`).
+
+This document describes what the in-guest controller actually does for backup and restore at v0.59.0. The controller was de-privileged in slice 8C: restic, cross-drive-to-arbitrary-disks execution, drive-recovery and infra-backup were removed from the controller and now live in the host agent + PBS. The `restic` string still appears in the backup package only inside comments that record its removal (`backup.go:20`, `restore.go:14`, `restore.go:19`); there is no restic execution in the controller.
+
+## 1. The split: controller vs agent/PBS
+
+The controller owns the **app-data domain** only. The agent owns the **whole-LXC domain**.
+
+| Domain | Owner | What |
+|---|---|---|
+| Per-app DB dumps | controller | `pg_dump` / `mariadb-dump` via `docker exec` |
+| Per-app Docker-volume tars | controller | `docker run alpine tar` of named volumes |
+| Per-app secret-free recovery unit | controller | compose + dumps + `manifest.json` on the app's drive |
+| Tier-2 off-drive copy | controller | `rsync` mirror of an HDD app's unit + userdata to a different physical disk |
+| Whole-LXC vzdump snapshot | agent | `POST /backup`, crash- or stop-consistent vzdump to local / PBS |
+| Offsite / encrypted backup, verify, restore-test | agent + PBS | reported read-only to the controller |
+| Whole-guest restore (rootfs, secrets, keys) | agent + PBS | the controller refuses any restore that needs this |
+
+The boundary is precise: app data that lives on an **external user-data drive** (an HDD bind mount) is NOT inside the PBS whole-guest snapshot — PBS cannot reach bind mounts. That data is protected only by the controller's recovery unit + Tier-2 copy. App data that lives on the **rootfs** (non-HDD apps) IS inside the PBS whole-guest snapshot, so it gets no Tier-2 copy (`tier2.go:197-218`). The encrypted `app.yaml` and the controller's encryption key live on the rootfs, so they are inside PBS too — which is exactly why the restore path recovers secrets from the guest rather than storing them off-rootfs (see §3, §4).
+
+The agent surface the controller talks to is `internal/agentapi/client.go`: a TLS client pinning the agent's self-signed leaf by SHA-256 (`client.go:69-82`) and authenticating with a per-guest bearer token. Backup-relevant calls: `BackupDue` (`GET /backup/due`), `StartBackup` (`POST /backup`), `BackupStatus` (`GET /backup/status`), `RestoreTestStatus` (`GET /restore-test/status`). The whole-guest backup record (`BackupRecord`) and restore-test record are rendered read-only — the comment at `client.go:135-136` states the controller does not own whole-guest backup.
+
+## 2. Database dumps (`internal/appbackup/dbdump.go`)
+
+**Discovery** is `docker ps`-driven. `DiscoverDatabases` (`dbdump.go:66`) runs `docker ps --format {{.ID}}\t{{.Names}}\t{{.Image}} --filter status=running` and classifies each container by image substring: `postgres` → Postgres, `mariadb`/`mysql` → MariaDB; everything else is skipped. The stack name is derived by stripping a known DB suffix from the container name (`deriveStackName`, `dbdump.go:536` — `postgres`/`db`/`mariadb`/`mysql`/`database`/`redis`/`cache`). Connection details come from the container's own env (`populateDBEnv`, `dbdump.go:477`): `POSTGRES_USER`/`POSTGRES_DB` (defaults `postgres`), or `MYSQL_DATABASE`/`MARIADB_DATABASE` with root.
+
+**Per-DB dump** is `DumpOne` (`dbdump.go:162`), one container at a time, 5-minute timeout each. It re-checks the container is still running, then:
+
+- Postgres: `docker exec <id> pg_dump -U <user> -d <db> --clean --if-exists --no-owner --no-privileges` (`dbdump.go:202`).
+- MariaDB: `docker exec <id> mariadb-dump -u root -p<pw> --single-transaction --routines --triggers <db>` (`dbdump.go:228`). The root password is read from the container env (`MYSQL_ROOT_PASSWORD`/`MARIADB_ROOT_PASSWORD`, `dbdump.go:516`); the password is never logged (a redacted `-p***` form is built only for the debug log line).
+
+**The tmpfile safety (H8).** The dump streams to `<name>.sql.tmp`, then before the rename to the final `.sql`: `tmpFile.Sync()` then `tmpFile.Close()` are called explicitly, each removing the tmp and failing the dump on error (`dbdump.go:266-278`). Only after a non-empty stat (`dbdump.go:281`) is the tmp atomically `os.Rename`d to the final path (`dbdump.go:293`). This guarantees the data is flushed to disk before the rename makes the dump "visible", so a crash never leaves a half-written `.sql`. Stale `.tmp` files older than one hour are reaped at the start of each run (`cleanupTmpFiles`, `dbdump.go:553`). Each finished dump is structurally validated (`ValidateDump`, `dbdump.go:320`): header line + at least one `CREATE TABLE`, scanned line-by-line with a bounded `bufio.Reader` so large data lines do not allocate (H1).
+
+**Where dumps are stored.** `Manager.GetAppDrivePath(stack)` (`backup.go:88`) returns the app's `HDD_PATH` if it has one, else falls back to the configured `systemDataPath` (the internal SSD) for SSD-only apps. That drive path is mapped to its felhom-data namespace root by `namespaceRoot` (`backup.go:105` — Model A: an in-guest drive mount IS the namespace root, so it is used as-is; only the system-data fallback gets the `felhom-data` subdir appended). Dumps land in `AppDBDumpPath(nsRoot, stack)` = `<nsRoot>/backups/primary/<stack>/db-dumps/` (`paths.go:58`). The SSD-only fallback is the case that the C3 DR finding was about — and it is handled here: SSD-only apps get a correct path via `systemDataPath`, so there is no path gap. (The fallback only warns when `systemDataPath` itself is unconfigured, `backup.go:75`/`backup.go:94`.)
+
+`RunDBDumps` (`backup.go:143`) acquires the running flag, discovers, dumps each DB to its app's path (skipping drives marked disconnected/decommissioned), persists each validation result to `settings.json`, and finally refreshes every recovery unit (`captureAllRecoveryUnits`, `backup.go:247`) — even on partial DB failure, so units never go stale.
+
+## 3. Recovery units (`internal/backup/recovery_unit.go`)
+
+A recovery unit is a per-app, **secret-free**, self-contained directory at `<nsRoot>/backups/primary/<app>/` (`paths.go:42`). It contains:
+
+- `compose/` — `docker-compose.yml` + `.felhom.yml` + a **secret-stripped** `app.yaml`.
+- `db-dumps/` — the `.sql` dumps from §2.
+- `volume-dumps/` — named-volume `.tar` archives.
+- `manifest.json` — the `RecoveryManifest` (`recovery_unit.go:31`).
+
+What the unit **excludes**: it holds no secret values, no data-encrypting keys, and not the Docker image. The manifest stores only the pinned image tag(s) (`ImagePins` — re-pulled on restore), the **names** of the secret env vars (`SecretEnvVars`), and the names of the data-key env vars (`DataKeyEnvVars`); `SecretSource` records in plain text that the values come from "guest app.yaml (live rootfs) or PBS whole-guest snapshot — never stored in this unit" (`recovery_unit.go:141`). The stripped `app.yaml` carries only non-secret env, with a header naming the omitted secrets (`buildStrippedAppYaml`, `recovery_unit.go:188`).
+
+`CaptureRecoveryUnit` (`recovery_unit.go:68`) pulls the app's `RecoveryInfo` from the stack provider (`GetStackRecoveryInfo`), builds the captured content in memory, and is **idempotent**: it skips all drive writes when the existing manifest matches the current controller version, config checksums (sha256 of each captured file), and the DB/volume dump set (`recovery_unit.go:112-118`). This is what lets it run on the 5-minute status refresh without thrashing a spinning USB drive. Writes are atomic (`atomicWrite`, `recovery_unit.go:270` — tmp + rename). `DataKeyEnvVars` is a fail-closed restore annotation only (see §4); it does not affect capture.
+
+The `.fab` portable export/import path (`internal/appexport/`) is a separate, operator-driven mechanism documented elsewhere — cross-reference that doc; it is not the periodic recovery unit.
+
+## 4. Restore
+
+Two keep-side restore entry points exist; neither does a whole-guest restore (that is the agent's).
+
+**`RestoreApp(stack, snapshotID)`** (`restore.go:21`) — the legacy volume-only path. Stops the stack, re-imports the named-volume `.tar` dumps (`restoreDockerVolumes`, `restore.go:85`: `docker volume rm -f` + `create` + `docker run alpine tar xf`), restarts, and health-checks. `snapshotID` is retained only for signature/logging compatibility now that restic is gone (`restore.go:19-20`).
+
+**`RestoreFromRecoveryUnit(stack)`** (`restore_unit.go:74`) — the recovery-unit path. It reads the unit manifest (falling back to `RestoreApp` if no unit exists, `restore_unit.go:99`), recovers the secret values from the **guest's own live `app.yaml`** via `stackProvider.RecoverStackSecrets` (never from the unit), reconciles them, restores the named-volume data, then `RecreateStackFromUnit` rebuilds the app's definition from `compose/` and redeploys with the reconstructed env (re-pulling the pinned image). Nothing is regenerated; no secret is read from the unit.
+
+**The fail-closed data-key gate** is `reconcileRestoreSecrets` (`restore_unit.go:22`) — a pure, unit-tested function. It merges non-secret env with recovered secrets, then:
+
+- A missing **resettable** secret (DB password, admin password) is non-fatal: returned in `missing`, the caller warns and proceeds (`restore_unit.go:117`).
+- A missing **data-encrypting key** (`DataKeyEnvVars`) is **fatal**: the restore is refused with an explicit error directing the operator to do a PBS whole-guest restore first, because regenerating the key would render the stored data unreadable (`restore_unit.go:45-50`). This is the safety centerpiece: the controller never silently recreates an app whose data it can no longer decrypt.
+
+### 4b. Security note — `.fab` import path validation (CTRL-001, v0.59.0)
+
+The portable `.fab` import (`internal/appexport/`) validates every manifest path segment before it reaches a `filepath.Join` against a trusted base (`appexport.ValidateSegment`, `validate.go:28`; `validateManifestPaths`, `validate.go:51`, called from `UnmarshalManifest`). The attacker-controllable `AppName` / `HDDSubdirs` / `VolumeNames` are rejected on any `..`, path separator, or absolute path, closing the v0.59.0 path-traversal finding. This is cross-referenced here; the detail lives in the appexport doc and the v0.59.0 audit record.
+
+## 5. Tier-2 cross-drive (`internal/backup/tier2.go`)
+
+Tier 2 is the only off-drive protection browsable HDD userdata can get (PBS cannot reach bind mounts, `tier2.go:17-23`). It is an `rsync -a --delete` **mirror** (`rsyncMirror`, `tier2.go:358`) of an HDD app's recovery unit + bulk `appdata/` to `<target>/backups/secondary/<app>/{recovery-unit,appdata}/` on a **different physical disk**.
+
+**Auto-target selection** (`selectTier2Target`, `tier2.go:54`), in order:
+
+1. A customer-pinned target (`PreferredTarget` from the config panel) if it is still registered, schedulable, and off-disk (`tier2.go:62-83`).
+2. Another registered user-data drive on a different physical disk — can hold bulk userdata (`tier2.go:86-102`).
+3. The internal SSD (system data path) — **small units only**, headroom-guarded.
+
+Off-disk-ness is decided by `system.SamePhysicalDevice` (a `Stat_t.Dev` compare). If the only candidate is the same physical disk, `errNoOffDiskTarget` is returned.
+
+**The rootfs-headroom guard** is the key safety. The internal SSD is the ~8 GB guest rootfs, so option 3 refuses rather than fills: `tier2FitsSystemDrive` (`tier2.go:121`) → `tier2FitsHeadroom` (`tier2.go:43`) requires the copy to leave a reserve of `max(2 GB, 20% of total)` free, else returns `errSSDNoHeadroom`. A non-fitting or single-drive case is recorded as an honest `no_target` status (`recordTier2NoTarget`, `tier2.go:331`) with a Hungarian "needs a 2nd HDD" reason — the rootfs is never filled. When the SSD target is used, it is labelled DB/config-only (`tier2.go:116`, log suffix `[SSD: DB/config only]`).
+
+`RunAllTier2` (`tier2.go:199`) iterates deployed stacks, processes only those with an `HDD_PATH` (non-HDD apps are skipped — they are already in PBS), and skips disconnected/decommissioned drives. Status is persisted into `settings.CrossDriveBackup` (method `rsync`), with the customer-preference fields (`UserDisabled`, `PreferredTarget`) preserved across runner writes by `withTier2Prefs` (`tier2.go:290`). The config-panel view is `Tier2Info` (`tier2.go:245`, read-only).
+
+## 6. Concurrency, scheduling, and the quiesce loop
+
+**Single-flight / running mutex.** `Manager` guards a `running` flag with a mutex; `acquireRunning`/`releaseRunning` (`backup.go:373`) reject a second backup or restore with "already in progress". `RestoreApp` and `RestoreFromRecoveryUnit` take the same flag (`restore.go:31`, `restore_unit.go:79`).
+
+**Scheduling** (wired in `cmd/controller/main.go`, Europe/Budapest):
+
+- `db-dump` daily at `cfg.Backup.DBDumpSchedule` (default `02:30`, `config.go:271`) → `RunDBDumps` (which also refreshes recovery units) (`main.go:328`).
+- `tier2-backup` daily at `03:30` → `RunAllTier2` (`main.go:360`).
+- `RefreshCache` runs on the 5-minute status refresh, re-scanning dump files and (idempotently) re-capturing recovery units (`backup.go:477`).
+
+Manual triggers exist via the API router (`RunDBDumps`, `RunAllTier2` launched as goroutines, `router.go:764`/`781`).
+
+**The quiesce loop** (`internal/quiesce/quiesce.go`) drives the whole-guest backup app-consistently. The agent's vzdump is crash-consistent only (an LXC has no fsfreeze), so the controller stops the app stacks first. `Loop.Run` polls `GET /backup/due`; when due, `quiesceAndPoll` (`quiesce.go:205`):
+
+1. Writes a persisted marker (atomic, `0600`) listing the stacks it is about to stop — **before** stopping anything (`quiesce.go:207`).
+2. Stops the running app stacks.
+3. `POST /backup`, records the job id, polls `GET /backup/status`.
+4. Resumes early at the `snapshotted` phase (8B.2 downtime optimization — the storage snapshot has captured the stopped state, so the app may come back up; the loop keeps polling to `done`/`failed`), or at `done` in stop/downgraded mode (`quiesce.go:256-279`).
+
+Unquiesce is **guaranteed**: a deferred closure restarts exactly the stopped stacks and clears the marker on every exit path — backup error, status-poll error, the `MaxQuiesce` bound (default 30 min, restarts the app while the backup continues on the agent), or context cancellation (`quiesce.go:213-225`, `quiesce.go:243-249`). On startup, `Recover` (`quiesce.go:113`) restarts any stacks left stopped by a mid-quiesce crash, then clears the marker. Single-flight is enforced both within the process (a `TryLock` mutex shared by the scheduled loop and the manual `TriggerNow`, `quiesce.go:149`/`quiesce.go:181`) and across restarts (the active marker — a cycle refuses to start on top of one, `quiesce.go:157`). `TriggerNow` runs the same flow asynchronously for the manual "Mentés most" action, returning `ErrBackupInProgress` if a cycle is already running.
+
+**Restic is gone from the controller.** All disk-tier backup (restic snapshots, cross-drive-to-other-disks, drive recovery, infra backup) is the agent's; the controller's only remaining off-drive copy is the Tier-2 rsync mirror in §5.

documentation/controller/deploy-and-stack-lifecycle.md

@@ -0,0 +1,256 @@
+# Controller: deploy & stack lifecycle
+
+> Source of truth: felhom-controller `internal/stacks/`, `internal/infra`, `internal/sync` at v0.59.0.
+
+This document describes how the in-guest controller represents application stacks, deploys
+them, runs their lifecycle (update/stop/start/restart/remove/delete), and brings up the base
+infrastructure (traefik / cloudflared / filebrowser) that everything else depends on. Every
+claim below is verified against the current source.
+
+## 1. The stack model
+
+A *stack* is one customer app, represented on disk as a directory under
+`/opt/docker/stacks/<name>/` (config key `paths.stacks_dir`, default `/opt/docker/stacks`)
+containing:
+
+- `docker-compose.yml` — the compose template (synced from the catalog, see §7).
+- `.felhom.yml` — app metadata (`Metadata`): display name, deploy fields, resource
+  requests/limits, optional config, healthcheck spec, integrations.
+- `app.yaml` — the per-app deployment record (`AppConfig`), written **only after** the app
+  is first deployed.
+
+In memory each stack is a `Stack` struct (`internal/stacks/manager.go:62`):
+
+| Field | Meaning |
+|---|---|
+| `State` (`ContainerState`) | aggregated container state |
+| `Deployed` | has an `app.yaml` with `deployed: true` |
+| `Deploying` | a `compose up` is in progress (image pull, etc.) |
+| `Protected` | a system stack that must never be stopped/removed (§4) |
+| `Orphaned` | deployed but no longer present in the synced catalog |
+| `DeployError` | last async deploy error, surfaced to the UI |
+| `HealthProbe` | latest controller-side probe result (§6) |
+
+`ContainerState` values (`manager.go:23`): `running`, `starting` (`health: starting`),
+`unhealthy`, `stopped`, `restarting`, `exited`, `paused`, `unknown`, `not_deployed`,
+`deploying`, `orphaned`. The effective state is derived by `resolveContainerState`
+(combines Docker `State` + the health hint in `Status`) and then `aggregateState` across
+all of a stack's containers, with priority `unhealthy > starting > restarting > all-running
+> stopped` (`manager.go:437`, `:468`).
+
+`ScanStacks` (`manager.go:250`) discovers stacks by walking `stacks_dir`, reading metadata
+and `app.yaml`, marking `Protected` via `cfg.IsProtectedStack`, and computing `Orphaned`
+against the synced catalog cache (`getCatalogTemplateSlugs`, `manager.go:1078`). While an
+async deploy is in flight it deliberately does **not** overwrite `Deployed`/`AppConfig` —
+the deploy goroutine owns those (H3 fix, `manager.go:293`).
+
+### `AppConfig` (`app.yaml`)
+
+```go
+type AppConfig struct {
+    Deployed     bool              `yaml:"deployed"`
+    DeployedAt   string            `yaml:"deployed_at"`
+    Env          map[string]string `yaml:"env"`
+    LockedFields []string          `yaml:"locked_fields"`
+}
+```
+(`deploy.go:98`). `Env` holds the resolved deploy-field values; sensitive ones are stored
+**encrypted** (§3). `LockedFields` lists env vars marked `locked_after_deploy` that
+`UpdateStackConfig` refuses to change (`deploy.go:434`).
+
+## 2. Deploy flow
+
+Entry point: `DeployStack(req)` (`deploy.go:121`), driven by the deploy form ("Telepítés").
+
+1. **Atomic check-and-set of `Deploying` (H1).** Under the manager lock, the stack must
+   exist, must not already be `Deploying`, and must not already be `Deployed`; then
+   `Deploying` is set true (`deploy.go:122`). This single critical section prevents two
+   concurrent deploys of the same app. Any later validation failure clears the flag via
+   `clearDeploying()`.
+
+2. **Memory gate.** `system.GetMemoryMB()` vs the app's `mem_request` and the configured
+   `reserved_memory_mb`. A hard block (real used + new request > usable) aborts with a
+   Hungarian error; a soft over-commit case returns a non-fatal warning string
+   (`deploy.go:159`).
+
+3. **Field resolution.** For each `DeployField`: `domain` is auto-filled from
+   `customer.domain`; `subdomain` is validated (DNS-safe, not reserved, not already in
+   use); `secret` uses the value previewed to the user or generates one; `password` is
+   **required** (never silently generated — the user must know it); `path` must exist on
+   the host. Required-but-empty fails (`deploy.go:206`–`292`).
+
+4. **`app.yaml` is persisted with `deployed: false` first (CTRL-T2-1).** The in-memory
+   `AppConfig` carries `Deployed: true` (for UX), but a clone with `Deployed: false` is what
+   `SaveAppConfig` writes to disk at this point (`deploy.go:302`–`314`). The durable record
+   is intentionally *not* marked deployed yet.
+
+5. **In-memory state flips to deployed** (`s.Deployed = true`, `s.AppConfig = appCfg`),
+   so the UI immediately stops showing a stale "Telepítés" button during the pull
+   (`deploy.go:331`).
+
+6. **Async compose.** `go m.runComposeDeploy(...)` runs `docker compose up -d` off the
+   request path, so the HTTP handler returns at once and the UI polls for progress (an
+   image pull can take 30–60s). The compose command injects the resolved env plus `DOMAIN`
+   (`deploy.go:338`, `composeExecWithEnv` `manager.go:505`).
+
+### CTRL-T2-1 crash-safety (the key current behaviour)
+
+The on-disk `app.yaml` records `deployed: true` **only after** `docker compose up -d`
+actually succeeds. `runComposeDeploy` (`deploy.go:346`):
+
+- On compose **failure**: reverts in-memory state (`Deployed=false`, `Deploying=false`,
+  records `DeployError`, `AppConfig=nil`) and re-saves the reverted `AppConfig`
+  (still `deployed:false`) to disk (`deploy.go:350`).
+- On compose **success**: re-runs `SaveAppConfig` with `appCfg.Deployed == true`, flipping
+  the durable record to deployed (`deploy.go:371`). If *that* save fails, it reverts in
+  memory so the customer can cleanly redeploy rather than be stuck half-recorded
+  (`deploy.go:375`).
+- Only then is `Deploying` cleared and `RefreshStatus()` run.
+
+Why it matters: a crash or power loss **during the image-pull window** leaves `app.yaml`
+at `deployed: false`. On restart, `ScanStacks` therefore sees the app as *not deployed*
+(no ghost-deployed stack with no containers), and `DeployStack` will cleanly redeploy
+instead of refusing with "already deployed".
+
+The UI's three-step progress (form submit → "pulling/deploying" via the `deploying`
+state and `DeployError` → settled `running`/`unhealthy`) is driven entirely by this
+in-memory state, which the front-end polls.
+
+## 3. `SaveAppConfig` and secret encryption (fail-closed, H10)
+
+`SaveAppConfig(stackDir, cfg, encKey, sensitiveVars)` (`deploy.go:669`) clones the env and,
+for each var named in `sensitiveVars` (the `secret`/`password` deploy fields, computed by
+`SensitiveEnvVars`, `deploy.go:741`), encrypts the value with the AES-256 key
+(`crypto.Encrypt`) unless it is empty or already encrypted.
+
+**Fail-closed (H10, v0.59.0):** if encryption of a sensitive value errors, `SaveAppConfig`
+**returns the error and writes nothing** — it never falls through to a plaintext write
+(`deploy.go:683`–`691`). Earlier code logged a WARN and persisted plaintext, which leaked
+the secret to disk; that path is gone. Callers (`DeployStack`, `UpdateStackConfig`, etc.)
+already propagate this error, so the deploy fails cleanly with no secret on disk.
+
+The write itself is atomic: write to `app.yaml.tmp` then `rename` (H04, `deploy.go:711`),
+mode `0600`. Decryption for compose happens lazily via `LoadAppConfigDecrypted` /
+`stackEnv` (`deploy.go:727`, `manager.go:809`), which injects `DOMAIN` plus the decrypted
+env into the compose process environment. A one-time startup `MigrateEncryption`
+(`manager.go:128`) re-saves any deployed app still holding plaintext sensitive values.
+
+## 4. Lifecycle: update / stop / start / restart / remove / delete
+
+All operations resolve the stack dir, build env via `stackEnv` (decrypted), and shell out
+to compose. Notably **start/restart use `up -d`, not bare `restart`**, so env changes and
+template changes (new images, healthchecks) are picked up (`manager.go:709`, `:643`).
+
+- **Start** (`StartStack`, `manager.go:643`): `compose up -d`; clears the stale health
+  probe so the next probe runs fresh.
+- **Stop** (`StopStack`, `manager.go:682`): refuses protected stacks, then `compose down`.
+- **Restart** (`RestartStack`, `manager.go:709`): `compose up -d`; clears health probe.
+- **Update** (`UpdateStack`, `manager.go:746`): `compose pull` then
+  `compose up -d --remove-orphans`.
+- **Config update** (`UpdateStackConfig`, `deploy.go:405`): rejects locked fields, re-saves
+  `app.yaml` (encrypting secrets), then `up -d` with the decrypted env.
+- **Remove** (`RemoveStack`, `delete.go:283`): for a *deployed* (non-orphaned) stack —
+  `compose down --volumes` (keeps images for redeploy), optional HDD-data and backup-path
+  cleanup, then deletes **only** `app.yaml`, reverting the stack to "not deployed". Compose
+  template files are preserved so the user can redeploy.
+- **Delete** (`DeleteStack`, `delete.go:80`): for an *orphaned* stack only —
+  `compose down --rmi local --volumes`, optional HDD-data removal, then removes the whole
+  stack directory.
+
+### Protected-stack enforcement (server-side, defence in depth)
+
+The protected set is configured under `stacks.protected` (default
+`traefik`, `cloudflared`, `felhom-controller`, `filebrowser` — see
+`configs/controller.yaml.example:55` and the setup writer `internal/setup/handlers.go:476`).
+`config.IsProtectedStack` matches case-insensitively (`config.go:347`). Enforcement exists
+at three layers:
+
+- **Router/API:** `actionStack` blocks any action other than `restart` on a protected
+  stack with HTTP 403 (`internal/api/router.go:412`).
+- **Manager:** `StopStack` (`manager.go:683`), `RemoveStack` (`delete.go:289`) and
+  `DeleteStack` (`delete.go:86`) each independently refuse protected stacks, so the guard
+  holds even if a caller bypasses the router.
+- The quiesce loop's `RunningAppStacks` (`manager.go:232`) also excludes protected stacks,
+  so the controller never stops its own tunnel/proxy or itself for a backup.
+
+### Deploying-guard on remove/delete (H2)
+
+Both `RemoveStack` (`delete.go:309`) and `DeleteStack` (`delete.go:106`) refuse while the
+stack is `Deploying`, and both also require the stack to be stopped (not
+running/starting/restarting). HDD-path deletion is gated by `ProtectedHDDPaths`
+(`delete.go:58`), which refuses to wipe top-level namespace directories (`appdata/`,
+`backups/`, `media/`, etc.), and `ParseComposeHDDMounts` cleans paths before the prefix
+check to block traversal (C10, `delete.go:555`).
+
+## 5. Base-infra bring-up: `EnsureBaseStack`
+
+`EnsureBaseStack` (`internal/stacks/infra.go:27`) renders and deploys the routing/access
+infrastructure. Properties (verified):
+
+- **Single-flight:** guarded by `infraMu.TryLock()` (`infra.go:28`). It is fired both at
+  first boot and on every `system-health` tick (self-heal); a second concurrent invocation
+  returns immediately rather than racing a `compose up` on the same dir.
+- **Idempotent:** each component is skipped when its container is already running
+  (`containerRunning`, `infra.go:256`), so the healthy-state re-run is a cheap trio of
+  `docker inspect` calls.
+- **Non-fatal by contract:** per-component failures are collected into a joined error for
+  the caller to *log* — it must never crash the controller (`infra.go:67`).
+
+Deploy order is load-bearing (the composes declare `traefik-public` as
+`external: true`, so it must exist first):
+
+1. `ensureTraefikNetwork` — creates the external `traefik-public` docker network if absent,
+   tolerating a create/inspect race (`infra.go:211`). A failure here is fatal to the run
+   (every stack `up` would fail without it).
+2. `ensureTraefik` — prepares `dynamic/`, `certs/`, a `0600` `acme.json`, renders the
+   traefik config from `customer.email` + `infrastructure.cf_api_token`, writes it, and
+   `compose up -d` (`infra.go:73`).
+3. `wireController` — writes the file-provider route `Host(felhom.<domain>) →
+   http://felhom-controller:8080` and joins the `felhom-controller` container to
+   `traefik-public`. Both steps are idempotent (route rewritten only when content changed,
+   so the traefik file watcher doesn't reload every tick; network-connect skipped when
+   already attached). A missing domain is a logged no-op, not an error (`infra.go:160`).
+4. `ensureCloudflared` — **only when `infrastructure.cf_tunnel_token` is set**; a LAN-only
+   node legitimately runs without it, and `monitor.EffectiveProtected` mirrors this
+   condition so such a node doesn't report cloudflared as a perpetually-missing protected
+   container (`infra.go:55`, `monitor/healthcheck.go:252`).
+5. `ensureFileBrowser` — **preserves an existing compose.** If
+   `filebrowser/docker-compose.yml` already exists, it `compose up -d` *without
+   regenerating*, so the storage mounts that `web.SyncFileBrowserMounts` manages are kept
+   intact. Only on first provision does it render the initial compose + `config.yaml` with
+   no mounts (`infra.go:122`).
+
+## 6. Health probes
+
+`RunHealthProbes` (`internal/stacks/healthprobe.go:23`) is a scheduler-driven, controller-side
+probe (independent of Docker's own healthcheck). It only probes stacks in `running`/`unhealthy`
+state that declare a `healthcheck` in metadata and whose interval has elapsed. The interval is
+adaptive: a fast **10s** retry while unhealthy, otherwise the configured interval (default 5m);
+when `HealthProbe` is nil (just started) it probes immediately (`healthprobe.go:41`).
+
+Targets are collected under a read-lock, probed **concurrently** with the lock released
+(`healthprobe.go:91`), and results applied under the write-lock. Check types: `tcp`
+(dial), `http` (any response = healthy), and `api` (validate expected status / body
+substring) — `runSingleCheck`, `probeTCP`, `probeHTTP` (`healthprobe.go:169`+). A failed
+probe overrides a Docker-`running` stack to `unhealthy`; a passing probe clears that
+override back to `running`. `refreshStatusLocked` re-applies the last probe result so a
+status refresh never resurrects a stale healthy state (`manager.go:418`).
+
+## 7. How stacks get their templates (catalog git-sync)
+
+The compose templates and metadata come from the app catalog via `internal/sync`
+(`Syncer`, `sync.go:22`). On a configurable interval (`git.sync_interval`, default 15m,
+with an initial sync at startup) it shallow-clones/pulls `git.repo_url` (branch
+`git.branch`) into `<data_dir>/catalog-cache` via `git fetch --depth 1` +
+`reset --hard origin/<branch>` (`sync.go:247`). It then copies **only** `docker-compose.yml`
+and `.felhom.yml` from each `templates/<app>/` into `stacks_dir/<app>/`, hashing content
+to skip unchanged files and **never** overwriting `app.yaml` (`copyTemplates`,
+`sync.go:311`; `copyIfChanged`, `sync.go:395`).
+
+After a sync that changed anything, it triggers a rescan (`ScanStacks`) and, for *updated*
+stacks, a post-sync hook that runs `InjectMissingFields` — auto-generating values for any
+new `secret`/`domain`/`subdomain` deploy fields that aren't yet in a deployed app's
+`app.yaml` (`sync.go:207`, `deploy.go:801`). `TriggerSync` (manual) debounces to one run
+per 30s. Sync is disabled (manual mode) when `git.repo_url` is empty (`sync.go:79`). The
+catalog cache also feeds orphan detection (`getCatalogTemplateSlugs`): a deployed stack
+whose slug is no longer in the cache is flagged `Orphaned`.

documentation/controller/module-map.md

@@ -0,0 +1,65 @@
+# Felhom Controller — Module Map (current)
+
+**Source of truth:** `felhom-controller/controller/internal/` at v0.59.0 (2026-06-13).
+**Supersedes** the planning map `../architecture/02-controller-module-map.md` (a v0.33 KEEP/PORT/DELETE
+plan written before slice 8C). That plan is now **executed**: the disk/storage/restic/watchdog
+subsystems were deleted or moved to the host agent; the controller is Docker-only and holds no Proxmox
+credentials. The old doc is kept for history; this is the live map.
+
+## What the controller is (and is not)
+
+The in-guest controller is **one per customer LXC, Docker-only**. It owns the **app domain**: stack
+lifecycle, the Hungarian web UI, app-data backup (DB dumps + volume tars + recovery units + Tier-2
+off-drive copies), metrics/telemetry, hub reporting, catalog sync, integrations, geo-restriction, and
+self-update. It does **not** touch Proxmox or raw disks: whole-guest backup (PBS vzdump), disk
+classification, and destructive storage ops live in the **host agent** (`felhom-agent`), reached over a
+TLS-leaf-pinned local API (`internal/agentapi`).
+
+## Package map
+
+| Package | Role | Notes / doc |
+|---|---|---|
+| `cmd/controller` | entry point, wiring, schedulers | `Version` via ldflags |
+| `internal/stacks` | stack model + deploy/start/stop/update/remove lifecycle, protected-stack enforcement | [deploy-and-stack-lifecycle](deploy-and-stack-lifecycle.md) |
+| `internal/infra` | base-infra bring-up (`EnsureBaseStack`: traefik/cloudflared/filebrowser), self-heal | [deploy-and-stack-lifecycle](deploy-and-stack-lifecycle.md) |
+| `internal/sync` | catalog git-sync (compose/.felhom.yml; never overwrites app.yaml) | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/appbackup` | app-data backup primitives: DB dumps, recovery units, restore | [backup-architecture](backup-architecture.md) |
+| `internal/backup` | backup orchestration: scheduling, Tier-2 cross-drive, running mutex | [backup-architecture](backup-architecture.md) |
+| `internal/appexport` | `.fab` encrypted app export/import (AES-CTR+HMAC); path-segment validation (CTRL-001) | [backup-architecture](backup-architecture.md) §export |
+| `internal/quiesce` | app-consistent quiesce loop (stop→backup→restart, crash-safe) | [backup-architecture](backup-architecture.md) |
+| `internal/agentapi` | TLS-leaf-pinned client to the host agent (disks, host-metrics, whole-guest backup) | [storage-monitoring-metrics](storage-monitoring-metrics.md) |
+| `internal/settings` | `settings.json` persistence (password hash, storage registry, caches) | RWMutex; atomic writes |
+| `internal/config` | `controller.yaml` loading + defaults; `IsProtectedStack` | — |
+| `internal/crypto` | AES-256-GCM at-rest secret encryption | — |
+| `internal/web` | dashboard UI + JSON API: auth, CSRF, storage/disk handlers, host-metrics, templates | [auth-hub-sync-integrations](auth-hub-sync-integrations.md), [storage-monitoring-metrics](storage-monitoring-metrics.md) |
+| `internal/api` | REST router (`/api/*`) | route→CSRF/auth coverage |
+| `internal/setup` | first-run setup wizard (pre-auth) | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/bootstrap` | first-run `bootstrap.json` ingestion (seed config, skip setup) | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/report` | hub report builder + pusher (zero-knowledge; no app secrets) | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/notify` | event notifications + history | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/integrations` | app-to-app integrations (adapter pattern; OnlyOffice↔FileBrowser/Nextcloud) | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/cloudflare` | geo-restriction via CF WAF (`[felhom-geo]` rules, geosync) | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/selfupdate` | self-update (version check; pinned tags, never `:latest`) | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/assets` | hub-sourced asset sync (cache + baked-in fallback) | [auth-hub-sync-integrations](auth-hub-sync-integrations.md) |
+| `internal/metrics` | SQLite (WAL) metrics store, docker-stats collector, log scanner/telemetry | [storage-monitoring-metrics](storage-monitoring-metrics.md) |
+| `internal/monitor` | health checks (`healthcheck.go`). **watchdog/pinger deleted in 8C** | [storage-monitoring-metrics](storage-monitoring-metrics.md) |
+| `internal/system` | system info; `dockervol.go` = OS/Docker-data-split prevention layer (v0.58) | [storage-monitoring-metrics](storage-monitoring-metrics.md) |
+| `internal/scheduler` | cron-like job scheduler (Europe/Budapest); late-registration safe; per-job `recover()` | — |
+| `internal/recovery` | recovery-file generation | — |
+| `internal/selftest` | startup self-test (hub reachability, metrics DB, etc.) | — |
+| `internal/util` | shared helpers | — |
+
+## Cross-repo boundary
+
+- **Host agent (`felhom-agent`):** owns Proxmox + all destructive storage. The controller calls it via
+  `internal/agentapi`; the agent enforces the data-bearing gate + operator signatures. See
+  `../architecture/03-host-agent.md` and `../audits/deep-sweep-2026-06-13.md` (agent destructive-path).
+- **Hub (`felhom.eu/hub/`):** operator backend; the controller pushes zero-knowledge reports/events to
+  it. See `../architecture/05-hub-architecture.md`.
+
+## De-privileging — what is GONE from the controller (executed in 8C)
+
+`internal/storage/*` (scan/format/migrate), restic, cross-drive-to-other-disks execution,
+`monitor/watchdog.go` + `pinger.go`, drive-restore, infra-backup, the storage UI's raw-disk operations.
+The controller now **delegates** all of these to the agent. Do not document the controller as performing
+them.

documentation/controller/storage-monitoring-metrics.md

@@ -0,0 +1,256 @@
+# Controller — storage, monitoring and metrics
+
+Source of truth: felhom-controller `internal/agentapi/`, `internal/web` storage handlers,
+`internal/metrics/`, `internal/monitor/`, `internal/system/dockervol.go` at v0.59.0; disk
+classification/execution is the host agent's.
+
+This document describes what the **in-guest controller** actually does with storage, host
+health and metrics. The controller is **de-privileged** (slice 8C): it is Docker-only, holds
+no Proxmox/disk credentials, and does not scan, format, mount or classify disks itself. Disk
+topology and classification come from the host **agent**; the controller calls the agent over a
+pinned local API, displays the agent's view, and keeps a small registry of the user-data drives
+it cares about.
+
+---
+
+## 1. Storage model in the controller
+
+There are two distinct storage views in the controller, and they must not be conflated:
+
+### 1a. The registered user-data drive registry (`settings.StoragePath`)
+
+The controller persists a list of user-data drives it manages, in `settings.json` as
+`StoragePaths` (`internal/settings/settings.go:97`, the `StoragePath` struct). Each entry is a
+mount the customer's apps deploy their large files onto:
+
+| Field | Meaning |
+|---|---|
+| `Path` | the in-guest mount, e.g. `/mnt/felhom-usb` |
+| `Label` | friendly Hungarian name (`Külső HDD 1TB`) |
+| `IsDefault` | new apps default here |
+| `Schedulable` | new apps may be deployed here |
+| `Disconnected` / `Decommissioned` | lifecycle flags |
+| `MigratedTo` | target path after a decommission |
+
+`GetStoragePaths()` (`settings.go:439`) returns a defensive copy under `RWMutex`. This registry
+is **only the external/user-data drives** — the agent's system disk (local-lvm), `local`
+templates dir and PBS targets are **never** in it (they live host-side and are surfaced via the
+agent's host-metrics view, §4). All public settings methods take the mutex; writes are atomic
+(write `.tmp`, rename).
+
+### 1b. The agent's authoritative disk topology
+
+A "storage target" / "disk" in the operational sense is whatever the **agent** reports. The
+controller learns topology by *asking the agent*, never by inspecting devices:
+
+- `Client.Disks()` → agent `GET /disks` → `[]DiskInfo` (`internal/agentapi/client.go:227`,
+  `:306`). Each `DiskInfo` carries the agent's **authoritative** `Role`
+  (`system | backup | user-data`), `DataBearing`, `DataReason`, capacity, and a `DurableID`
+  (`uuid:<fs-uuid>` for usb/local-dir). `DiskInfo.FSUUID()` (`client.go:252`) strips the
+  `uuid:` prefix — this is the only way the de-privileged controller learns a mount key it
+  cannot read off the device itself.
+- `Client.HostMetrics()` → agent `GET /host/metrics` → host health + `[]StorageTarget`
+  (`client.go:459`, `:492`).
+
+Classification is the agent's; the controller **displays** it. The UI is driven from
+`DiskInfo.Role`: `system`/`backup` get a lock badge and no destructive controls; `user-data` is
+customer-manageable (`client.go:235-237`).
+
+---
+
+## 2. Disk operations are delegated to the agent
+
+All disk execution (list/assign/eject/format) goes through `agentapi.Client` to the agent's
+`/disks` endpoints. The controller's web handlers are **thin proxies**
+(`internal/web/agent_disk_handlers.go`), wired behind `RequireAuth + CsrfProtect`:
+
+| Route | Proxies to | Notes |
+|---|---|---|
+| `GET /api/disks` | agent `GET /disks` | list; sorted Go-side for stable order |
+| `POST /api/disks/assign` | agent `POST /disks/assign` | benign mount of an existing fs |
+| `POST /api/disks/eject` | agent `POST /disks/eject` | safe-unmount (data preserved) |
+| `POST /api/disks/format` | agent `POST /disks/format` | data-bearing gated agent-side |
+
+`agentClient()` (`agent_disk_handlers.go:43`) builds a pinned client from `cfg.LocalAPI`
+(endpoint/token/fingerprint); it returns "agent not configured" on an unprovisioned guest.
+
+**The data-bearing gate is enforced on the agent, not the controller.** `FormatDisk()`
+(`client.go:372`) sends the caller's `device/fstype/confirmed/durable_id`; the agent inspects
+the device itself, tiers it by role (its own classification — the controller's claim is
+ignored), and:
+
+- blank device → formatted;
+- **user-data**, data-bearing, not confirmed → `ErrNeedsConfirmation` with the durable id to
+  type-to-confirm against (a *customer* confirmation, not an operator signature);
+- **system/backup**, data-bearing → `ErrFormatRefused` with a `PendingOp` carrying the exact
+  offline `felhom-opsign` command (`PendingOp.OpsignCommand()`, `client.go:292`).
+
+The controller holds **no destructive authority** — there is no force-format path. The format
+handler surfaces both refusals as HTTP 409 (`agent_disk_handlers.go:192-200`); the deep gate
+mechanics live on the agent side (cross-ref the agent's destructive-path / AGENT-001 docs).
+
+### Eject and the guided init/wipe flows
+
+`internal/web/storage_handlers.go` orchestrates the guided init/attach/wipe over the same agent
+endpoints plus the local registry:
+
+- **Init** (`runStorageInit`, `storage_handlers.go:83`): format → (confirm/refuse?) → resolve
+  the *new* fs UUID by re-listing disks → benign `assign` → register in the StoragePath
+  registry → `guest-attach` into this guest. On any refusal it performs no further destructive
+  or mount action.
+- **Wipe** (`handleStorageWipe`, `:357`): customer-confirmed wipe of a **user-data** drive only;
+  server-side type-to-confirm (the typed name must equal the mount basename). A
+  system/backup-protected device is refused by the agent even though the controller sends
+  `confirmed:true` (`:402-406`).
+- **Eject** (`handleStorageEject`, `:506`): benign unmount via the agent (data preserved) +
+  deregister the StoragePath + resync FileBrowser mounts. The eject **role-gate** is implicit:
+  eject is non-destructive, but the destructive wipe behind it is the agent's role-tiered gate.
+  The agent's `EjectResult` returns `DependentGuests` so the UI can warn about other guests
+  bound to the drive (`client.go:343`).
+- Restricted to `/mnt/<name>` (validated by `mountNameRe`, `storage_handlers.go:39`); only
+  `ext4`/`xfs` offered (the agent re-validates).
+
+---
+
+## 3. The v0.58 infra-protection prevention layer (`internal/system/dockervol.go`)
+
+After the OS/Docker-data split, `/var/lib/docker` is a dedicated volume holding **all** images,
+overlay and named volumes — both infra (controller/traefik/cloudflared/filebrowser) and customer
+apps. Infra is protected by **prevention, not placement**: a reserved buffer the controller
+refuses to deploy into.
+
+- `DockerVolumePath = "/"` (`dockervol.go:16`). The controller container's root is an overlay
+  whose upperdir lives on the guest's `/var/lib/docker` volume, so `statfs("/")` reports **that
+  volume's** capacity/free (true with the golden's overlay2 driver; pre-split it was the rootfs —
+  correct either way).
+- `DockerVolumeReserveGB(totalGB)` (`dockervol.go:23`) = `max(5 GB, 10% of total)`. (10% rather
+  than the Tier-2 guard's 20%: 20% of a large data volume would reserve an absurd amount.)
+- `GetDockerVolumeHeadroom()` (`dockervol.go:43`) measures the volume and returns
+  `DockerVolumeHeadroom{TotalGB, AvailGB, ReserveGB, BelowReserve, OK}`. `BelowReserve` is
+  `AvailGB <= ReserveGB` (`:53`). **`OK=false` when stats are unreadable** — callers MUST
+  fail-open (a transient measurement error must not block all deploys; the buffer is a safety
+  net, not a security control).
+
+**Deploy gate** (`internal/api/router.go:353`): `deployStack` refuses a new deploy with
+**HTTP 507** + a Hungarian message when `hr.OK && hr.BelowReserve`. The fail-open is explicit —
+the gate only engages when `hr.OK` is true.
+
+**UI pre-warning** (`internal/web/handlers.go:339`): for a NEW deploy only (an existing app's
+config save consumes no fresh image space), the deploy page sets `DockerBelowReserve` +
+human-readable free/reserve, and `deploy.html` shows a warning banner and disables the
+"Telepítés indítása" button when below reserve.
+
+---
+
+## 4. Host metrics (agent-sourced host health view)
+
+The de-privileged controller sees only its own cgroup, so it cannot read host health itself.
+`ServeHostMetricsAPI` (`internal/web/agent_host_metrics_handler.go:20`, behind `RequireAuth`,
+read-only) proxies `GET /api/host-metrics` → agent `GET /host/metrics` and returns the host-wide
+view: cpu%/mem/load/uptime/cpu-temp (`HostMetrics`, `client.go:431`) plus per-storage capacity +
+SMART/thin-pool health (`StorageTarget`, `client.go:459`).
+
+**v0.57 server-side enrichment** (`enrichHostStorageTargets`, `agent_host_metrics_handler.go:51`):
+the agent enumerates storages via `pvesm` in non-deterministic order, so the
+`#host-storage-bars` list reordered on every poll. The handler now:
+
+1. Sorts `StorageTargets` stably by `storageTypeRank` (`:67`): user-data drives (usb/local-dir)
+   → internal SSD (lvmthin/lvm) → `local` templates+backups → backup targets (pbs/nfs/cifs) →
+   other; alphabetical by id within a tier.
+2. Attaches a friendly Hungarian `Label` + one-line `Purpose` per entry
+   (`storageLabelAndPurpose`, `:84`). These are **display-only** controller-side fields
+   (`StorageTarget.Label/Purpose`, `client.go:476-480`) — the raw PVE storage id stays in `Name`
+   and is never renamed.
+
+The disk-overview list (`GET /api/disks`) is similarly sorted Go-side by role
+(`sortDisksForView`, `agent_disk_handlers.go:87`) for a stable, user-data-first order.
+
+**`buildStorageBars`** (`internal/web/handlers.go:50`) is the separate monitoring-page
+"Tárolók kapacitása" list. It iterates the **registered user-data StoragePaths** (not the agent
+view), skips decommissioned drives, reads local `system.GetDiskUsage` per path, and sorts by
+`Path`. Every bar carries the same `storageBarPurpose` text (`:46`) because this list is
+all-user-data by construction — the agent's system/PBS storage is not in this registry (it is on
+the storage-management page via the agent host-metrics view).
+
+---
+
+## 5. Metrics subsystem (`internal/metrics/`)
+
+### SQLite store (`store.go`)
+
+`NewMetricsStore(dbPath)` opens `modernc.org/sqlite` at
+`/opt/docker/felhom-controller/data/metrics.db` and **verifies WAL mode took effect** — it
+errors out if `PRAGMA journal_mode=WAL` does not return `"wal"` (`store.go:27-34`), then sets
+`synchronous=NORMAL` and `busy_timeout=5000`. Two tables: `system_metrics` and
+`container_metrics`, with ts indices (`:48-77`). Queries downsample into time buckets
+(`QuerySystemMetrics`/`QueryContainerMetrics`, default resolution 200 points). `Prune` deletes
+rows older than a cutoff.
+
+### Collector (`collector.go`)
+
+`MetricsCollector.Start(ctx)` runs a **60-second** loop guarded by `sync.Once`
+(`collector.go:38,53`), wired in `cmd/controller/main.go:203` and stopped on shutdown. Each tick:
+
+- **System sample**: `system.GetInfo(hddPath, cpuCollector)` → cpu%, mem, temp, loadavg, SSD +
+  HDD usage (`sampleSystem`, `:81`).
+- **Container sample** (`sampleContainers`, `:99`): runs `docker stats --no-stream` under a
+  **cancellable 30s timeout context** derived from the loop ctx (`:100`), parses the
+  tab-separated cpu/mem/net/block columns, and batch-inserts.
+
+### Telemetry + log scanner
+
+- `GetContainerTelemetry(since)` (`telemetry.go:20`) aggregates per-container avg/peak memory +
+  avg cpu over a window from the DB, then patches in the most-recent memory per container.
+- `ScanContainerLogs(names, since, logger)` (`logscanner.go:52`) scans each container's
+  `docker logs --since=<m> --tail=1000` **sequentially** (to avoid load spikes), under a
+  per-container 10s timeout (`:87`). It classifies lines on the first 5 words against
+  error/warn keyword sets (`error|fatal|panic|crit|oom|killed|exception|traceback`;
+  `warn|warning`), deduplicates by a normalized fingerprint (ANSI/timestamp stripped; UUIDs,
+  long hex and 6+ digit runs collapsed), and caps each container at its 10 most-frequent issues.
+
+These two feed the **hub report**, not a live UI: `internal/report/telemetry.go:18`
+(`buildAppTelemetrySection`) collects 15-minute telemetry + a log scan for every non-protected,
+deployed, running stack **plus the controller container itself** (`controllerContainerName =
+"felhom-controller"`), merges per-app issues (capped at 10), and emits `[]AppTelemetry` pushed to
+the hub.
+
+---
+
+## 6. Monitoring (`internal/monitor/healthcheck.go`)
+
+`RunHealthCheck(cfg, cpuCollector, storagePaths, logger)` (`healthcheck.go:26`) produces a
+`HealthReport{Status: ok|warn|fail, Issues, Warnings, Info}`. It is invoked at startup, on a
+schedule, and from the debug endpoint (`cmd/controller/main.go:273,558`;
+`internal/web/handler_debug.go:255`). Checks:
+
+1. **SSD disk usage** — `sysInfo.DiskPercent` is `statfs("/")`, i.e. the same Docker-data volume
+   the prevention layer guards; warn at `DiskWarnPercent` / crit at `DiskCritPercent`. These trip
+   **above** the 10%-free reserved buffer, so the customer is warned before the deploy gate
+   engages (`:47-76`).
+2. **HDD usage** (when `HDDConfigured`), **memory**, **CPU**, **temperature** — each against its
+   configured threshold (`:79-145`).
+3. **Docker** reachable (`docker info`, `:148`).
+4. **Protected containers** running — using `EffectiveProtected(cfg)` (`:252`), which drops
+   `cloudflared` when no tunnel token is configured so a LAN-only node is not perpetually FAIL.
+5. **Storage paths** (`checkStoragePaths`, `:279`): per registered StoragePath, warn on
+   disconnected/inaccessible, warn (not fail) when not on a separate mount point, warn ≥90% /
+   issue ≥95% usage. All messages Hungarian.
+
+Status rolls up to `fail` if any issue, else `warn` if any warning.
+
+### Removed: watchdog and pinger (slice 8C)
+
+The old storage **watchdog** (disk disconnect/reconnect detection) and the **pinger** were
+deleted when the controller was de-privileged — disk detection moved to the host agent
+(`cmd/controller/main.go:435` notes the watchdog moved host-side; `monitor/` now contains only
+`healthcheck.go` + tests). Healthchecks pinging is likewise retired: ping UUIDs in config are
+logged as no-ops and the hub now owns monitoring (`cmd/controller/main.go:209-214`).
+
+---
+
+## Cross-references
+
+- Agent disk classification, the data-bearing gate and the signed destructive path: the host
+  agent's storage / AGENT-001 docs.
+- Backup capture/restore and Tier-2 off-drive copies: `controller/backup-architecture.md`.
+- Deploy flow and the headroom gate in context: `controller/deploy-and-stack-lifecycle.md`.