An absent off-site object has four meanings — never configured, mid-restart, a
transient config read failure, and rebuilt-and-stranded — and the hub cannot tell
them apart. The box can, from two local facts it holds with certainty, so it says
so instead of leaving the hub to deduce it from a silence (operator ruling).
The ACK's identity_blob_present is now recorded on EVERY ACK, before the gates
that used to discard it: on a box with no off-site target the auto-confirm returns
immediately, which is exactly a rebuilt box, so the one fact distinguishing it from
a box that never had off-site backups was thrown away every cycle.
The declaration needs BOTH halves — a fresh data area AND a hub-held recovery
package. Freshness alone is a box that never had off-site backups; dropping that
condition makes the whole fleet ask for credentials, which is what the Scenario B
test exists to catch.
The object carries enabled:false and zero sizes, which is what makes it inert to
the hub's existing fill and staleness checkers and to a pre-upgrade hub. A
configured box's JSON is byte-identical to v0.198.0's.
gofmt -w across the controller tree (46 files) so gofmt -l is empty — disarms the
formatting landmine where a targeted edit + accidental gofmt -w swept ~46 unrelated
files. Pure formatting: whitespace + gofmt's optional-semicolon removal in reflowed
inline closures. One doc comment reworded ('' -> 'the empty string') to avoid gofmt's
Go-1.19 doc-comment typographic substitition ('' -> curly quote) muddying its meaning.
No build/vet/test behavior change.
R-51: aggregateState's mixed branch returned StateRunning ("partial"), so a stack whose
MAIN container was dead behind live helpers alerted on nothing — immich-server sat Exited
for 18 h, 100 % unreachable, no banner and no app_start_failed (audit F4). New
StateDegraded: a DOWN member whose docker restart policy is always/unless-stopped is a
fault (degraded, a down state); no/on-failure is a finished one-shot and stays benign; an
unreadable policy fails CLOSED. The unhealthy/restarting/paused/unknown exclusions are
byte-identical — folding unhealthy into down is the flapping fix-3 avoided.
R-52: new internal/bootrecon — one bounded start-once sweep at startup (2 attempts, 30 s
apart) for apps an interrupted boot left behind, inside the 90 s boot grace so a success
is silent and a failure still alerts. A zero-container stack is NEVER touched: the UI's
Stop is compose down, so a deliberate stop survives a reboot.
Both features carry a production-path wiring test (the v0.154.0 / v0.91.0 inert-seam
class). The main() assertion is an AST walk, not strings.Contains — the substring version
passed its own red-proof, because a commented-out call still contains the string.
Red-proofs run and restored: mix branch reverted -> "running" on the immich fixture;
boot hook commented out -> wiring test fails; zero-container gate dropped -> the
user-stopped app gets started.
NOTE: controller/cmd/controller/ is matched by .gitignore's `controller` entry, so new
files there need `git add -f` (and ripgrep silently skips main.go without --no-ignore).
Part 4 — restore: RestoreSharesScratch + PlaceSharesRestore as SIBLINGS of the
per-app scratch/place pair. Files merged missing-only (never overwriting), each
destination PREFIX-ASSERTED against registered LIVE storage roots; definitions
merged with existing-wins; ReconcileSamba via a seam (backup must not import
stacks); credential restored best-effort into the samba named volume.
New routes POST /backup/shares/{restore,place} + a restore-page entry that renders
'Megosztasok', never the raw reserved key.
Also adds scratchJoin: reconstructing an absolute captured path under a scratch
must strip the volume name rather than rely on filepath.Join.
Part 5 — liveness: EffectiveProtected gains a settings-backed dynamic extra so the
samba CONTAINER (not the stack name — they differ) is watched exactly while sharing
is on. FINDING: the issue -> health 'fail' -> existing health_critical event ->
alert -> Hungarian degradation e-mail path needs NO further change, and introduces
no new event type, so the allowlist gotcha does not apply.
Part 6 — UI: per-tier backup status lines on the Megosztas page (amber only on
deviation). Verified the two warning-prose sites (offbox_capture/tier2_capture)
only ever receive per-app stack names, so no mapping is needed there.
RED-PROOFS RUN AND REVERTED (both fired):
4. prefix-assert removed -> place-guard traversal test FAILS
5. dynamic samba extra removed -> Scenario E enabled-case FAILS
report.Waiter holds a hanging GET against hub /api/v1/wait?gen=N (same hub
URL+key as the pusher). On a generation change it fires the v0.139.0
report.Trigger — nothing else; the report ACK delivers everything through the
unchanged machinery. No overall client timeout (held GET); per-request ctx
bounds a dead connection. First-observation records-not-fires; same-gen
timeout fires nothing; errors (incl. 404 from a pre-v0.58.0 hub) back off
5s->5min while the 15-min cycle reconciles. Wired beside the trigger under the
same hubPusher!=nil && Hub.Enabled gate.
Red-proof: disable the baseline branch -> first observation fires
(TestWaiter_FirstObservationRecordsNoFire), run-fail-reverted.
Copy soften: backups_remote/escrow "néhány másodperc, legfeljebb 15 perc".
Pairs with hub v0.58.0. Grounding:
felhom.eu/documentation/audits/SPIKE-immediate-sync-transport-2026-07-16.md
The customer sets + owns the dashboard password via a hub-emailed one-time
claim code. An unclaimed box (code hash present, no password) serves ONLY the
claim page — every other route → claim page (302) or 401, so a Day-0 box is
never open on the internet. A set password disables the gate (auth wins).
Reset rides the same code engine (login "Elfelejtett jelszó"). Legacy-open
(no password, no hash) shows a red transition banner until the hub delivers a
hash. Report ACK caches the code state idempotently by generation; report
carries claimed (set-only). --print-reset-code root escape hatch. Requires
hub v0.50.0. Gate-coverage signature test + 4 red-proofs proven.
QuotaGB rides the descriptor into OffboxTarget; RepoSizeBytes persisted
from restic stats. Pre-run gate: >=100% refuses NEW backups (Hungarian
notice + operator alert) but prune STILL runs (red-proofed) and restore is
never gated; >=80% warns. /backups usage bar (quota>0 only). The hub
report gains the non-secret offsite status object for the OffsiteChecker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
EscrowAutoConfirmer flips pending->escrowed ONLY when sha256(local repo
password) matches the ACK's restic_pw_sha256 (blob-presence alone never
confirms — red-proofed). Mismatch warns once per hash naming the ceremony;
never un-confirms; wipes the staged secret on flip. Pinned cross-repo hash
vector; manual confirm deprecated to a legacy-blob fallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Pairs with agent v0.77.0. StageEscrowSecret pushes the repo password to the
agent (POST /escrow/stage-secret) at offsite-enable → EscrowState="pending".
Atomicity gate: RunOffboxBackup (scheduler + handler) refuses until
EscrowState="escrowed" (operator POST /backup/offbox/confirm-escrow after the
escrow ceremony) — no un-recoverable offsite ciphertext can exist. DR:
POST /backup/offbox/inject-password pre-places a recovered 64-hex password 0600
(honored by WriteOffboxSecrets' IsNotExist guard; refuses clobber without
force). DR recipe gains non-secret offsite_restic coords (DRResticCoord); SFTP
key regenerated at DR, not escrowed. New settings.OffboxTarget.EscrowState.
Tests + atomicity & inject companion red-proofs green; UI gates pass. NOT yet
live-validated (supervised ceremony).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
PushResponse.ConfigVersion from the report ACK; ConfigRefresher reconciles vs.
the last-applied version (settings.applied_config_version) and on a change calls
bootstrap.RefreshConfig (re-pull controller.yaml + re-merge local_api) then
GracefulSelfRestart. First-run records baseline (no restart); unchanged = no-op
(no storm); failed pull keeps config + retries. Companion to hub v0.26.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
The controller honors an operator-enforced minimum version (FLOOR) on the hub
report ACK and auto-updates to the floor when below it (managed default, no click),
reusing the Phase 1 in-guest-pull + agent-swap + rollback. Latest stays the opt-in
button; the floor is the auto-target, never latest.
- pusher.go: PushResponse += min_controller_version, latest_version (existing ACK seam)
- main.go: OnPushResponse → updater.SetFloor + MaybeAutoUpdate (rides report cycle)
- updater.go: SetFloor/GetFloor + MaybeAutoUpdate reusing performUpdate (auto-floor);
no-op at/above floor, floor>latest, dev/no-agent/backup; no flap (in-mem+persisted)
- settings UI (HU): floor display + auto restart-poll during an auto-update
- tests: below/at/floor>latest/no-flap/raised-floor; below-floor red-proof verified
- no agent change (reuses Phase 1 POST /controller/swap)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSZmmSFVzGwEzhYmxbkgBK
DR recipe slice (controller half), grounded in SPIKE-dr-recipe-2026-06-16. The
controller emitter is the BOUNDARY enforcement point: v1 ships an explicit
allowlist {catalog_ref, enabled, storage_bindings} and reads NOTHING from
AppConfig.Env, so no ENC:/token/password can leak. storage_bindings parsed from
the compose (${HDD_PATH}/${USERDATA_PATH} volume binds -> {container_path,
drive, subpath}).
Load-bearing tests: TestBuildAppRecipe_NoSecrets (synthetic-secret app -> none
leak) + TestBuildAppRecipe_AllowlistIsLoadBearing (red-proof companion) +
TestAppStorageBindings + TestBuildDRRecipeAppHalf. Red-proofed live: forcing the
emitter to dump Env makes the boundary test fail. recipe_version=1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Geo settings save and manual geo sync now fire an out-of-band, non-blocking hub
report push (Router.reportPushNow seam, wired in main.go to BuildReport+Push in a
goroutine) so the hub reflects the new geo state / clears a stale last_sync_error
within seconds instead of after the next ~15-min cycle. Scope: geo handlers only.
- builder.go always populates report.GeoRestriction (Enabled=false, empty countries
when nil/disabled) via new buildGeoRestrictionReport helper, so the hub always
renders the geo section ("Inaktív" when off) instead of hiding it via omitempty.
- Tests: geo save success → push once; invalid country → no push (companion);
buildGeoRestrictionReport(nil) → non-nil disabled (companion vs old nil-omit).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Controller half of Phase-1 Infra Backup retirement (hub v0.12.0;
SPIKE-infra-backup-2026-06-15). Pure dead-code removal, no behaviour change.
- Remove Pusher.PushInfraBackup (caller-less; hub endpoint gone).
- Remove Notifier.NotifyBackupCompleted / backup_completed event (caller-less
since slice 8C; hub deadline check now reads agent host-report PBS snapshots).
- Remove report.BackupReport.ResticPassword — builder never sets it post-8C
(confirmed in source + live), but it historically leaked the restic password
into the hub's plaintext reports store.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the felhom-controller container as a special entry in the
app_telemetry array sent to the hub. This reuses all existing hub
infrastructure (storage, aggregation, UI) with zero hub-side changes.
The controller's memory/CPU metrics and log warnings/errors are now
collected alongside app telemetry, giving the hub visibility into
controller health, memory trends, and known issues.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add structured operational logging at INFO, WARN, and ERROR levels to
every controller module. Standardize custom prefixes ([GEO], [SCHED],
[SYNC]) to use [INFO/WARN/ERROR] [module] format. Fix misleveled logs
(WARN->ERROR for data loss scenarios, WARN->INFO for routine operations).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hub: GFS retention (7d/4w/3m, ~14 versions) in new infra_backup_versions
table. Recovery endpoint supports ?version=ID. New /versions API endpoint.
Dashboard shows backup history.
Controller: local drive backups rotated into history/ (last 5 versions).
Setup wizard shows version picker for Hub restores when multiple versions
exist. Scan results enriched with app names, disk count, history badge.
Local restore supports historical versions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug fixes:
- Add applyEnvOverrides to LoadFromBytes (M05)
- Set state=failed on compose-up failure in selfupdate (M16)
- Clamp usableMB to min 0 in memory check (M22)
- Remove "manual" schedule from triggerAllCrossBackups (M23)
- Add mmcblk device handling for partition paths (M21)
- Fix stripPartition for mmcblk devices (L25)
- Fix TruncateStr for UTF-8 and negative maxLen (L05/L06)
- Fix AllDone to return false for empty restore plans (L14)
- Fix PushOnce to return actual errors (L39)
- Restore pending events on save failure in DrainPendingEvents (M03)
- Add duplicate check in AddStoragePath (M04)
- Call CleanupTempMounts after drive scan (H13)
- Log SetStep save errors (M25)
Hardening:
- Guard scheduler Start() against double-start (M14)
- Acquire mutex in scheduler Stop() before reading cancel (L24)
- Cap log lines parameter to 10000 (L31)
- Require POST for logout (L32)
- Use sync.Once for Server.Close() (L49)
- Panic on crypto/rand.Read failure in setup CSRF (L40)
- Validate Bearer token against Hub API key in CSRF (H16 fix)
- Replace custom hasPrefix with strings.HasPrefix (L13)
- Replace simpleHash with crc32.ChecksumIEEE (L48)
Cleanup:
- Remove dead imageName function (L02)
- Remove dead detectHostIPViaRoute function (L03)
- Rename shadowed copy variable to cp (L07)
- Copy DefaultEnabledEvents in GetNotificationPrefs early return (L09)
- Update BUGHUNT.md with comprehensive audit results
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add GeoRestrictionReport to report types and builder, so Hub can
display geo-blocking status on customer detail pages
- Update all 5 BuildReport() call sites with new geoRestriction param
- Add /api/geo/ to selfUpdateAuthMiddleware (Hub Bearer token auth)
- Replace embedded logo SVG with updated logo.svg (white text variant)
- Add FelhomFaviconSVG constant + /static/favicon.svg route
- Update layout.html and catchall.html favicon links
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Passwords and secrets from deploy fields (type: password/secret) are now
encrypted at rest in app.yaml using a per-node 32-byte key. Values stored
as ENC:base64(nonce+ciphertext), decrypted transparently for docker-compose
and web UI. Key included in infra backup bundle for disaster recovery.
Existing plaintext values migrated automatically on startup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deployed-but-stopped apps were included in telemetry reports with all-zero
memory/CPU values, dragging down hub-side averages. Now isStackRunning()
filters to only running/starting/unhealthy/restarting states.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New GET /api/debug/telemetry endpoint runs full telemetry pipeline on-demand
- GetTelemetryPreview callback added to DebugCallbacks, wired in main.go
- BuildAppTelemetryForDebug() exported wrapper in report/telemetry.go
- Debug page: new collapsible section with per-app table (memory, CPU, log errors/warnings, issues) and raw JSON viewer
- Available regardless of hub configuration
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New internal/metrics/telemetry.go: MetricsStore.GetContainerTelemetry()
aggregates container memory/CPU from SQLite over the last 15 min
- New internal/metrics/logscanner.go: ScanContainerLogs() scans docker logs
for errors/warnings, deduplicates via fingerprinting (strips timestamps,
replaces 6+ digit numbers, hex strings, UUIDs)
- New internal/report/telemetry.go: buildAppTelemetrySection() assembles
per-stack AppTelemetry by aggregating container metrics and log summaries
- internal/report/types.go: added AppTelemetry field to Report struct plus
AppTelemetry type with memory/CPU/log fields and LogIssue references
- internal/report/builder.go: calls buildAppTelemetrySection() in BuildReport()
- Backward-compatible: old Hub versions silently ignore app_telemetry field
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New controller features:
- Web-based setup wizard replaces docker-setup.sh interactive config
- Dual listener: :8080 (Traefik) + :8081 (direct HTTP for LAN)
- Drive scanner finds .felhom-infra-backup/ on all block devices
- Hub recovery pull (GET /api/v1/recovery/{id}) with retrieval password
- Fresh install: Hub config download or manual wizard
- CSRF protection, state persistence, Hungarian UI
- Local infra backup written to all connected drives after each backup cycle
- .felhom-infra-backup/backup.json + metadata.json with SHA256 checksum
- Hub verification: parse customer_blocked from report push response
- Limited mode after 7 days without verification
- Recovery info page on Settings + recovery-info.txt file generation
- Pending events queue: DR events sent to Hub on next report push
- docker-setup.sh v6.0.0: removed interactive wizard, minimal controller.yaml only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- POST /api/config/apply: accepts YAML body from Hub, validates and
writes controller.yaml atomically (tmp+rename)
- GET /api/config/hash: returns SHA256 hash of current config file
- Report payload now includes config_hash field for Hub comparison
- Config endpoints use same dual auth as self-update (session OR Bearer)
- config.LoadFromBytes() for validation without file I/O
- config.FileHash() helper for SHA256 computation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 1: Deprecate restic as Tier 2 method (rsync only), auto-migrate on startup
Phase 2: Enhanced per-app migration with backup awareness, DB dump copy, auto-cleanup
Phase 3: Full drive migration with decommissioned state, rollback support, wizard UI
Phase 4: Hub report includes decommissioned drive state
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Controller now includes its external URL in periodic hub reports so the
hub can trigger self-updates remotely via the /api/selfupdate/update endpoint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix backup toggles not appearing (read each app's own HDD_PATH from app.yaml)
- Storage paths registry in settings.json with auto-discovery from deployed apps
- Settings page "Adattárolók" section with disk usage, add/remove/default/schedulable
- Deploy page path field as dropdown of registered storage paths
- Health check storage monitoring (mount point, disk usage alerts)
- Mount-point validation utilities (Linux syscall + cross-platform stubs)
- Controller docker-compose mount changed to /mnt:/mnt:rw for multi-storage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
admin