- 05-hub-architecture.md: stale 'felhom-hub v0.6.3' -> v0.11.0 (design-draft note). - auth-hub-sync-integrations.md: full onlyoffice:nextcloud occ command sequence. - storage-monitoring-metrics.md: metricsDBPath verified to coincide with the volume-backed data_dir on the bootstrap guest (persists; hardcoding is latent fragility only). - REORG-NOTES: gaps 2/3 CLOSED, gap 5 partially closed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
17 KiB
Controller control-plane & app-domain glue
Source of truth: felhom-controller internal/web (auth/csrf/setup), internal/report, internal/notify, internal/sync, internal/integrations, internal/cloudflare, internal/selfupdate, internal/assets at v0.59.0.
This document covers the in-guest controller's link to the operator control plane (the hub), its first-run onboarding, and the app-domain glue (catalog sync, app-to-app integrations, geo-restriction, self-update, asset sync). Every claim below is verified against current source; file:line citations are in the verification ledger of the implementing task.
1. Auth & sessions
The web tier is gated by two middlewares, RequireAuth and CsrfProtect, both methods on web.Server.
Password & hash priority. effectivePasswordHash() resolves the active hash in priority order: (1) settings.json password_hash (customer-changed), then (2) controller.yaml web.password_hash (operator-provisioned). Empty from both sources means no auth. Hashes are bcrypt (bcrypt.DefaultCost); login compares with bcrypt.CompareHashAndPassword.
Sessions. On successful login createSession() mints a 256-bit random hex session token AND a separate 256-bit hex CSRF token, stored server-side in an in-memory map (s.sessions). The session cookie is felhom_session, MaxAge 7 days, HttpOnly, SameSite=Lax, and Secure only when the request arrived over TLS or carried X-Forwarded-Proto: https. A background goroutine sweeps expired sessions every 15 minutes; invalidateAllSessions() clears them all after a password change. There is no server-side persistence — a controller restart drops all sessions (re-login required).
Demo / no-password open mode (verified). When authEnabled() is false (no hash from either source), RequireAuth passes every request through, and CsrfProtect returns before any token check. So a controller with no dashboard password is fully open — both auth and CSRF are skipped. This is the demo posture (guest 9201). /api/health is always exempt from auth even when enabled.
CSRF. Double-submit using the per-session CSRF token. Safe methods (GET/HEAD/OPTIONS) pass through. For unsafe methods, the submitted token is read from the _csrf form field first, then the X-CSRF-Token header, and compared to the session's stored token with subtle.ConstantTimeCompare. Two exemptions: auth disabled (above), and a valid Authorization: Bearer <api_key> header — the bearer is constant-time-compared against cfg.Hub.APIKey and, if it matches, CSRF is skipped (this is how the hub's signed callbacks and API-key clients reach mutating endpoints). An invalid bearer falls through to normal CSRF validation. Rejections return JSON 403 for /api/ paths, plain text 403 otherwise.
Login rate-limit. Per-IP: max 5 failed attempts in a 1-minute sliding window; the 6th within the window is refused with a Hungarian "too many attempts" message. The window resets after a minute of no failures, and a successful login clears the counter for that IP. XFF caveat: the rate-limit key (and login-failure logging) takes the client IP from the first X-Forwarded-For entry when present, falling back to RemoteAddr. Behind the per-guest reverse proxy this is the real client; a spoofable upstream XFF would let an attacker rotate the key, so the limit assumes a trusted proxy sets XFF.
2. Setup wizard + first-run bootstrap
Both run pre-auth — there is no password yet. They are mutually exclusive paths to a configured controller.yaml.
Gating. setup.NeedsSetup(cfg) is true when cfg.Customer.ID == "", or when a .needs-setup marker file exists in the data dir (a debug re-trigger). main.go calls bootstrap.MaybeIngest(...) first, then checks NeedsSetup; if still unconfigured it launches the setup server instead of the normal app.
Bootstrap (internal/bootstrap, schema felhom.bootstrap/v2). The host agent's provisioning back-half writes a read-only bootstrap.json (default mount /etc/felhom-bootstrap/bootstrap.json, overridable via FELHOM_BOOTSTRAP_PATH). It carries ONLY: customer.id, hub.url, a per-customer hub.retrieval_password (secret), and the local_api block (endpoint, fingerprint, token; token secret). On an unconfigured controller, MaybeIngest:
- Is idempotent — if
cfg.Customer.IDis already set, it never reads the file, never pulls, never clobberscontroller.yaml. - Fail-safe — an absent/malformed file, a non-v2 schema, a missing required field, a missing pull function, or a pull that ultimately fails all leave cfg unchanged and log a WARN; the controller falls back to the setup wizard. A hub outage at first boot must not brick the guest.
- On success, pulls the full
controller.yamlfrom the hub (PullFunc, wired toreport.PullConfig) authenticated by the retrieval passphrase, merges the per-guestlocal_apiblock into the pulled YAML at the map level (the hub never knows per-guest Proxmox internals), writescontroller.yamlatomically at 0600, reloads it, and comes up configured — skipping the wizard.
Transient pull failures (ErrPullTransient, wrapping report.ErrHubUnreachable) are retried with a 2s/4s/8s backoff (4 attempts total); permanent failures (auth/not-found) fail fast.
Setup wizard (internal/setup, pre-auth HTTP server). Offered when bootstrap did not configure the guest. The wizard server (setup.NewServer) serves routes with no auth middleware (/setup, /setup/fresh, /setup/manual, /setup/failed). Disk-recovery wizardry was removed in slice 8C; today there are two modes:
- Fresh-from-hub (
processFreshHub/autoProcessFreshHub): the customer (or pre-seeded env varsFELHOM_SETUP_CUSTOMER_ID/FELHOM_SETUP_PASSWORDfromdocker-setup.sh --hub-customer) supplies customer-id + retrieval passphrase;report.PullConfigdownloads the generatedcontroller.yaml;writeFreshConfigwrites it. - Manual (
processManual):generateManualConfig()builds acontroller.yamlfrom form fields locally (no hub call).
Both end in writeFreshConfig, which writes /opt/docker/felhom-controller/controller.yaml atomically at 0600, and — into settings.json — bcrypt-hashes the dashboard password (if the form supplied one) via SetPasswordHash, and stores the retrieval passphrase via SetRetrievalPassword. Wizard progress is persisted to setup-state.json (atomic write) so a browser crash mid-wizard survives.
3. Hub reporting & notifications
The controller reports state, never app secrets — consistent with the zero-knowledge posture (the hub holds only non-secret config plus a wrapped PBS key it can't decrypt; app secrets live encrypted on the guest rootfs, never sent to the hub).
Report (report.BuildReport → report.Pusher). BuildReport assembles a snapshot: customer id/name, controller version, a SHA-256 hash of the on-disk controller.yaml (for hub drift comparison, not the content), controller URL (https://felhom.<domain>), system info (host/OS/kernel/CPU/mem/temp/load), storage (root fs + each registered user-data path, with decommissioned/disconnected markers), container counts + per-container cpu/mem, app-data backup status (DB-dump last-run only — disk-tier restic/snapshot fields are zero since that moved to the agent in slice 8C), health (status/issues/warnings), deployed/available stack lists, per-app telemetry, and geo-restriction status. No app secrets, env, or volume contents are in the report.
Pusher.Push POSTs the JSON to <hub>/api/v1/report with Authorization: Bearer <hub.api_key>, retrying 3 times with 5s backoff; it tracks last-attempt/last-success/consecutive-failures and parses the response for a customer_blocked flag (relayed via OnPushResponse). Cadence: sched.Every("hub-report", cfg.Hub.PushInterval), default 15m when unparseable, only when cfg.Hub.Enabled. A separate PushInfraBackup POSTs to /api/v1/infra-backup; PushOnce sends a single report regardless of the enabled flag (e.g. a reporting-disabled startup notice).
Notifications (notify.Notifier). Structured events POSTed to <hub>/api/v1/event with the bearer key. Enabled only when both hub URL and API key are set. Sends are non-blocking (goroutine), 3 attempts with 3s backoff; cooldown/dedup is the hub's job — the controller sends unconditionally. Event types include controller_started, controller_updated, backup_completed/backup_failed, db_dump_completed/db_dump_failed, backup_integrity_ok/backup_integrity_failed, crossdrive_completed/crossdrive_failed, health_critical/health_degraded/health_recovered (driven by status-rank change detection), storage_disconnected/storage_reconnected, app_deployed/app_removed, disaster_recovery_started/disaster_recovery_completed, plus test. Each event carries customer_id, severity, a Hungarian message, and an optional typed details object. A 50-entry ring buffer (GetEventHistory) records each send (type/severity/message/hub-status/error) for the debug page. SyncPreferences POSTs email + enabled-event list + cooldown to /api/v1/preferences (synchronous). A legacy Notify → /api/v1/notify path is retained for old hubs.
4. Catalog sync (internal/sync)
A periodic git pull of the app catalog into the local stacks dir. Syncer.Start clones (--depth 1 --branch <branch>) into <dataDir>/catalog-cache on first run, else fetch --depth 1 + reset --hard origin/<branch>; it runs an initial sync on startup and then on cfg.Git.SyncInterval (default 15m). Disabled (manual mode) when cfg.Git.RepoURL is empty. TriggerSync debounces to once per 30s. Credentials, if configured, are injected into the HTTPS URL and masked in all logs.
Safety — never overwrites app.yaml. copyTemplates syncs only docker-compose.yml and .felhom.yml per app directory; app.yaml (the per-guest deployed config carrying encrypted secrets) is never in the sync set and is never touched. Copies are content-hash gated (copyIfChanged / copyTemplates compare SHA-256 of src vs dst and skip identical files), so an unchanged catalog produces no writes and no USB thrash. New app dirs are reported as new_apps, changed ones as updated; a rescan + a post-sync hook (missing-deploy-field injection) fire only when something changed.
5. App-to-app integrations (internal/integrations)
Lets one app configure another (e.g. enable OnlyOffice editing inside FileBrowser).
Adapter pattern. integrations.Manager depends on a StackProvider interface (GetStack/GetStacks/RestartStack); integrationStackAdapter in main.go bridges stacks.Manager to it, breaking the import cycle. Handlers are keyed "provider:target" and registered at construction. Integration enable/disable state lives in settings.json (SetIntegrationState/GetIntegrationState); apply/revoke are serialized under a mutex.
Concrete handlers (registered in NewManager):
onlyoffice:filebrowser—Applyreads OnlyOffice'sJWT_SECRETand subdomain from the provider's decryptedapp.yamlenv, then patches FileBrowser'sconfig.yamlwith anintegrations.officeblock (public office URL, internalhttp://onlyoffice:80, the shared JWT secret) and restarts FileBrowser.Revokestrips that block and restarts. Writes are atomic (tmp+rename). FileBrowser is treated as always-present infrastructure (not gated on "deployed").onlyoffice:nextcloud—Apply(internal/integrations/onlyoffice_nextcloud.go) drives Nextcloud's OnlyOffice connector entirely throughdocker exec -u www-data nextcloud php occcommands, in order:app:install onlyoffice→app:enable onlyoffice→config:app:set onlyoffice DocumentServerUrl(public office URL) /DocumentServerInternalUrl(http://onlyoffice:80) /jwt_secret(the shared JWT secret read from OnlyOffice's env) /StorageUrl(http://nextcloud/) →config:system:set trusted_domains 10 nextcloud. Eachocccall runs under a context timeout; a failure aborts and is surfaced (Hungarian error).Revokerunsocc app:disable onlyoffice(a not-installed result is treated as already-revoked). No config-file patching — Nextcloud owns its own state viaocc.
Lifecycle hooks. Toggle validates that provider (and target, unless it's filebrowser) are deployed and running before applying. The manager exposes:
OnStackStop— revokes active integrations where the stack is provider or target, but keepsenabled=trueand marks statusprovider_stopped/target_unavailableso they can come back.OnStackStart— after a 5s delay (so the 30s stack-state refresh catches up), re-applies previously-enabled integrations once both sides are running/starting.OnStackRemove— permanently revokes (best-effort) and deletes integration state for the removed stack.
ReapplyConfigForTarget re-applies config without restarting (the caller, e.g. FileBrowser mount sync, handles restart).
6. Geo-restriction (internal/cloudflare)
The controller enforces country-based access via Cloudflare custom-WAF rules using a CF API token; this is the controller's own CF-API geo layer (distinct from any hub-side config).
Rule identity. All managed rules carry the [felhom-geo] description prefix — the global rule is exactly [felhom-geo] Global, per-app rules are [felhom-geo] app:<name>. GetFelhomRules filters the zone's http_request_firewall_custom ruleset to that prefix, so the controller only ever touches its own rules. Rules use the block action (plain 403; custom response bodies need a paid plan). Expressions are built from allowed ISO country codes: global = (not ip.src.country in {...}) with per-app hostnames excluded; per-app = (http.host eq "<host>" and not ip.src.country in {...}). Zero allowed countries means block-all.
Sync. GeoSyncManager.Sync resolves zone-id and ruleset-id (creating the custom ruleset if absent, caching ids in settings), lists existing felhom rules, builds the desired set from settings.GeoRestriction + deployed hostnames (via the StackLister/geoStackAdapter), and diffs: create new, update changed-expression, delete obsolete. When geo is nil or disabled, deleteAllRules removes every felhom rule. Sync state (zone/ruleset ids, last error) is saved back to settings.
Cadence & hooks. A delayed (15s) non-blocking initial sync at startup; periodic geo-verify every 6h; and OnGeoRelevantChange (wired to the API router) re-syncs on app deploy/remove — each guarded so it only runs when geo is enabled.
7. Self-update (internal/selfupdate)
Version check. CheckForUpdate queries the Gitea Docker Registry V2 tags API (/v2/<owner>/<repo>/tags/list) with basic auth from the git credentials, parses tags as semver (skipping latest/dev/non-semver), and reports the highest. A dev running version can't check. Scheduled as selfupdate-check on cfg.SelfUpdate.CheckInterval (default 6h) — check only, never triggers an update. The running version is injected at build time via ldflags (-X main.Version); the source default is "dev".
Update mechanism (compose-managed path). TriggerUpdate guards against concurrent updates, dev versions, an in-progress backup (backupRunning callback), and an inaccessible compose file, then runs performUpdate in a goroutine: write pending state → docker pull <image>:<version> → rewrite the image: line in docker-compose.yml (atomic tmp+rename, regex-pinned to gitea.dooplex.hu/admin/felhom-controller:<tag>) → docker compose up -d (which replaces the running container). On next boot, VerifyStartup reads the pending state and marks it success if the running version equals the target, else failed. Auto-update is a separate daily job (selfupdate-auto at cfg.SelfUpdate.AutoUpdateTime) gated on cfg.SelfUpdate.AutoUpdate.
Never :latest. The image reference is always <image>:<version> with a concrete semver tag — both the registry query and the compose rewrite pin an explicit version; latest/dev tags are explicitly skipped during version selection.
Deployment note: on the bootstrap-managed golden guest (e.g. 9201) the controller has no
docker-compose.yml— it is started byfelhom-controller-bootstrap.servicefrom/etc/felhom-controller-image. There the compose-rewrite self-update path does not apply; version drift is handled by the bootstrap service pulling the pinned tag. The compose path above is the/opt/docker/felhom-controllerdeployment model.
8. Asset sync (internal/assets)
Branding/app assets are sourced from the hub with a baked-in fallback. Syncer.Sync GETs <hub>/api/v1/assets/manifest (bearer key), compares each manifest entry's SHA-256 against the local cache (<dataDir>/assets), downloads only new/changed files (atomic tmp+rename), and removes local files absent from the manifest; it saves a local copy of the manifest. Cadence: an initial sync on startup plus a daily asset-sync job at cfg.Assets.SyncSchedule. Resolve(filename) returns the cached path if present, else the baked-in fallback (/usr/share/felhom/assets), else the cached path (so the caller emits a clean 404). Filenames are base-sanitized to prevent path traversal.
Cross-references
- Hub API contract and zero-knowledge config generation: hub
internal/api,internal/configgen(felhom.eu repo). - Agent-side bootstrap emission and provisioning:
felhom-agentinternal/provision/internal/localapi. - Backup/recovery-unit and Tier-2 mechanics: see the controller backup documentation.