2026-04-13T19:38:28Z INFO [infra] filebrowser compose exists — starting without regenerating
2026-04-13T19:38:29Z INFO [web] sessions: restored 124 dashboard session(s) from disk (dropped 0 expired, 0 malformed)
2026-04-13T19:38:29Z INFO [web] Auth: using password from settings.json
2026-04-13T19:38:29Z INFO [scheduler] Registered periodic job: disk-health-check (every 1h0m0s)
2026-04-13T19:38:29Z INFO [scheduler] Registered periodic job: agent-channel-health (every 1m0s)
2026-04-13T19:38:29Z INFO Web UI listening on :8080
2026-04-13T19:38:29Z INFO [web] FileBrowser: drive /mnt/felhom-drives/hdd_2 not mounted — skipping userdata skeleton
2026-04-13T19:38:29Z INFO [backup] Discovered 4 databases
2026-04-13T19:38:30Z INFO [monitor] Health check: status=fail
2026-04-13T19:38:30Z INFO [web] FileBrowser sync — no config/compose change, ensured running without recreate (3 storage path(s))
2026-04-13T19:38:30Z INFO [infra] connected felhom-controller to felhom-filebrowser (the FileBrowser default-login check, R-1126)
2026-04-13T19:38:31Z INFO [backup] Backup status cache refreshed
2026-04-13T19:38:32Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:33Z INFO [report] Building system report
2026-04-13T19:38:35Z INFO [samba] applying samba stack: shares=2 config_changed=false
2026-04-13T19:38:36Z INFO [gate] boot 1776109082-1457: waiting (≤2m0s) for live drive bind(s) [/mnt/felhom-drives/hdd_1] before recreating drive-backed apps
2026-04-13T19:38:36Z INFO [gate] boot 1776109082-1457: live bind confirmed — recreating drive-backed app immich (state=degraded) onto /mnt/felhom-drives/hdd_1
2026-04-13T19:38:36Z INFO [stacks] Stopping stack: immich
2026-04-13T19:38:37Z INFO [stacks] Stack immich stopped successfully (took 1.0s)
2026-04-13T19:38:37Z INFO [stacks] Status refresh: 15 containers across 68 stacks
2026-04-13T19:38:37Z INFO [stacks] Starting stack: immich
2026-04-13T19:38:37Z INFO [stacks] Status refresh: 15 containers across 68 stacks
2026-04-13T19:38:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:38Z INFO Health probes: 2 ok (of 2 probed)
2026-04-13T19:38:39Z WARN Event push failed after 3 attempts (controller_started/info): Post "https://hub.felhom.eu/api/v1/event": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:38:39Z is before 2026-10-14T07:36:58Z
2026-04-13T19:38:42Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:47Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:48Z INFO [monitor] Health check: status=ok
2026-04-13T19:38:48Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:48Z INFO Health probes: 1 ok (of 1 probed)
2026-04-13T19:38:48Z INFO [stacks] Stack immich started successfully (took 11.1s)
2026-04-13T19:38:49Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:49Z INFO [gate] boot 1776109082-1457: live bind confirmed — recreating drive-backed app nextcloud (state=degraded) onto /mnt/felhom-drives/hdd_1
2026-04-13T19:38:49Z INFO [stacks] Stopping stack: nextcloud
2026-04-13T19:38:49Z INFO [stacks] Stack nextcloud stopped successfully (took 0.4s)
2026-04-13T19:38:49Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:38:49Z INFO [stacks] Starting stack: nextcloud
2026-04-13T19:38:52Z INFO [stacks] Stack immich post-start status:
2026-04-13T19:38:52Z INFO [stacks]   immich-machine-learning   ghcr.io/immich-app/immich-machine-learning:v3.2.4@sha256:e16c2f166a8174901959fdf85e2e4c7bd1ebc4b37e0b6655de97c41408a260c4                running   Up 13 seconds (healthy)
2026-04-13T19:38:52Z INFO [stacks]   immich-postgres           ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0@sha256:1a078b237c1d9b420b0ee59147386b4aa60d3a07a8e6a402fc84a57e41b043a4   running   Up 13 seconds (healthy)
2026-04-13T19:38:52Z INFO [stacks]   immich-redis              redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499                                                   running   Up 13 seconds (healthy)
2026-04-13T19:38:52Z INFO [stacks]   immich-server             ghcr.io/immich-app/immich-server:v3.2.4@sha256:d317916b28090c33eb36b308464ea391f8b7df1d850fcfea227a39ec879718c2                          running   Up 3 seconds (health: starting)
2026-04-13T19:38:52Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:56Z INFO [stacks] Stack nextcloud started successfully (took 6.5s)
2026-04-13T19:38:56Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:56Z INFO [gate] boot 1776109082-1457: live bind confirmed — recreating drive-backed app paperless-ngx (state=degraded) onto /mnt/felhom-drives/hdd_1
2026-04-13T19:38:56Z INFO [stacks] Stopping stack: paperless-ngx
2026-04-13T19:38:56Z INFO [stacks] Stack paperless-ngx stopped successfully (took 0.5s)
2026-04-13T19:38:56Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:38:56Z INFO [stacks] Starting stack: paperless-ngx
2026-04-13T19:38:58Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:58Z INFO Health probes: 1 ok (of 1 probed)
2026-04-13T19:38:58Z INFO [deadapp] boot grace 1m30s: the host's last boot was clean (R-856)
2026-04-13T19:38:58Z INFO [aftercrash] a normal (or unknown) boot — no app repair after an unclean boot (R-1007)
2026-04-13T19:38:58Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:38:59Z WARN [report] Push failed: Post "https://hub.felhom.eu/api/v1/report": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:38:59Z is before 2026-10-14T07:36:58Z
2026-04-13T19:38:59Z WARN Startup hub report attempt 1/3 failed: hub push failed after 3 attempts: Post "https://hub.felhom.eu/api/v1/report": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:38:59Z is before 2026-10-14T07:36:58Z
2026-04-13T19:38:59Z INFO [stacks] Stack nextcloud post-start status:
2026-04-13T19:38:59Z INFO [stacks]   nextcloud         nextcloud:34.0.4-apache@sha256:37b109885aa3cba3e056362a899556a625c986f2c17cd2c0cc157d21c789f53b   running   Up 3 seconds (health: starting)
2026-04-13T19:38:59Z INFO [stacks]   nextcloud-db      mariadb:12.3@sha256:805c8e104bd563d5bfa24fadd3f31cd419ea859cb5277f32b5dbf2db714f9ed1              running   Up 9 seconds (healthy)
2026-04-13T19:38:59Z INFO [stacks]   nextcloud-redis   redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499            running   Up 9 seconds (healthy)
2026-04-13T19:39:03Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:07Z INFO [stacks] Stack paperless-ngx started successfully (took 10.9s)
2026-04-13T19:39:07Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:07Z INFO [gate] boot 1776109082-1457: re-syncing FileBrowser mounts against the live binds
2026-04-13T19:39:07Z INFO [settings] Settings saved
2026-04-13T19:39:07Z INFO [web] FileBrowser: drive /mnt/felhom-drives/hdd_2 not mounted — skipping userdata skeleton
2026-04-13T19:39:08Z INFO [web] FileBrowser sync — no config/compose change, ensured running without recreate (3 storage path(s))
2026-04-13T19:39:08Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:08Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:08Z INFO Health probes: 3 ok (of 3 probed)
2026-04-13T19:39:11Z INFO [stacks] Stack paperless-ngx post-start status:
2026-04-13T19:39:11Z INFO [stacks]   paperless-postgres    postgres:18-alpine@sha256:77f585114c32fbca283dc835b0596f4e52b51b4c6662d7810b2f4084f60a1873                            running   Up 13 seconds (healthy)
2026-04-13T19:39:11Z INFO [stacks]   paperless-redis       redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499                                running   Up 13 seconds (healthy)
2026-04-13T19:39:11Z INFO [stacks]   paperless-webserver   ghcr.io/paperless-ngx/paperless-ngx:2.20.15@sha256:6c86cad803970ea782683a8e80e7403444c5bf3cf70de63b4d3c8e87500db92f   running   Up 3 seconds (health: starting)
2026-04-13T19:39:13Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:18Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:18Z INFO [bootrecon] boot window: fleet settled after 46s (3 identical samples 5s apart) — sweeping
2026-04-13T19:39:18Z INFO [bootrecon] Boot reconciliation: no boot-orphaned apps (nothing to start)
2026-04-13T19:39:18Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:24Z WARN [report] Push failed: Post "https://hub.felhom.eu/api/v1/report": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:39:24Z is before 2026-10-14T07:36:58Z
2026-04-13T19:39:24Z WARN Startup hub report attempt 2/3 failed: hub push failed after 3 attempts: Post "https://hub.felhom.eu/api/v1/report": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:39:24Z is before 2026-10-14T07:36:58Z
2026-04-13T19:39:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:39:36Z INFO [scheduler] Job agent-channel-health completed (took 7.226s)
2026-04-13T19:39:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:48Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:39:49Z WARN [report] Push failed: Post "https://hub.felhom.eu/api/v1/report": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:39:49Z is before 2026-10-14T07:36:58Z
2026-04-13T19:39:49Z WARN Startup hub report attempt 3/3 failed: hub push failed after 3 attempts: Post "https://hub.felhom.eu/api/v1/report": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:39:49Z is before 2026-10-14T07:36:58Z
2026-04-13T19:39:49Z WARN Startup hub report failed after 3 attempts — next scheduled push in 15m
2026-04-13T19:39:57Z INFO [fillwatch] checked 4 filesystem(s), 0 unreadable/skipped, 0 notification(s); bands: all ok
2026-04-13T19:39:57Z WARN [offsite-apply] settle-gate: GO — floor still unknown after 1m30s; a hub that cannot serve a floor cannot serve a consume (no burn risk)
2026-04-13T19:39:58Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:40:08Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:40:14Z WARN [selfupdate] Registry check failed: HTTP request failed: Get "https://gitea.dooplex.hu/v2/admin/felhom-controller/tags/list": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:40:14Z is before 2026-10-05T03:52:51Z
2026-04-13T19:40:18Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:40:28Z INFO [scheduler] Running job: stack-scan
2026-04-13T19:40:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:40:28Z INFO [stacks] ScanStacks complete: 68 stacks found (8 deployed, 60 available)
2026-04-13T19:40:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:40:28Z INFO [scheduler] Job stack-scan completed (took 419ms)
2026-04-13T19:40:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:40:36Z INFO [scheduler] Job agent-channel-health completed (took 7.212s)
2026-04-13T19:40:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:40:48Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:40:58Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:41:08Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:41:18Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:41:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:41:28Z INFO [stacks] controller image retention: pass over 2 controller image(s) — running 0.336.0, previous "0.331.0" (the self-update's record), 0 candidate(s), 0 deleted, the rest kept
2026-04-13T19:41:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:41:36Z INFO [scheduler] Job agent-channel-health completed (took 7.217s)
2026-04-13T19:41:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:41:48Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:41:58Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:42:08Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:42:18Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:42:28Z INFO [scheduler] Running job: stack-scan
2026-04-13T19:42:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:42:28Z INFO [stacks] ScanStacks complete: 68 stacks found (8 deployed, 60 available)
2026-04-13T19:42:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:42:28Z INFO [scheduler] Job stack-scan completed (took 209ms)
2026-04-13T19:42:28Z WARN [deadapp] paperless-ngx: crash_loop — 7 in 10m0s; STOPPING it (decision 28)
2026-04-13T19:42:28Z INFO [stacks] Stopping stack: paperless-ngx
2026-04-13T19:42:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:42:36Z INFO [scheduler] Job agent-channel-health completed (took 7.22s)
2026-04-13T19:42:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-04-13T19:42:39Z INFO [stacks] Stack paperless-ngx stopped successfully (took 10.8s)
2026-04-13T19:42:39Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:42:39Z WARN [settings] restore hold SET for paperless-ngx — the app stays stopped until it is cleared
2026-04-13T19:42:39Z INFO [settings] Settings saved
2026-04-13T19:42:39Z INFO [settings] Settings saved
2026-04-13T19:42:39Z WARN [backup] paperless-ngx is STOPPED by the box: crash_loop (trip 1 within 24h0m0s) — Start gives it one more try (decision 28)
2026-04-13T19:42:45Z WARN Event push failed after 3 attempts (app_stopped_unhealthy/warning): Post "https://hub.felhom.eu/api/v1/event": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-04-13T19:42:45Z is before 2026-10-14T07:36:58Z
2026-04-13T19:42:48Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:42:58Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:43:08Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:43:18Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:43:28Z INFO [scheduler] Running job: backup-cache
2026-04-13T19:43:28Z INFO [scheduler] Running job: system-health
2026-04-13T19:43:28Z INFO [scheduler] Running job: docker-socket-users
2026-04-13T19:43:28Z INFO [scheduler] Running job: offsite-credential-retry
2026-04-13T19:43:28Z INFO [scheduler] Job offsite-credential-retry completed (took 1ms)
2026-04-13T19:43:28Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:43:28Z INFO [backup] Found 7 DB dump files across drives
2026-04-13T19:43:28Z INFO [backup] Discovered 3 databases
2026-04-13T19:43:28Z INFO [scheduler] Job docker-socket-users completed (took 271ms)
2026-04-13T19:43:28Z INFO [monitor] Health check: status=ok
2026-04-13T19:43:28Z INFO [scheduler] Job system-health completed (took 356ms)
2026-04-13T19:43:28Z INFO [infra] cloudflared skipped — no cf_tunnel_token configured (LAN-only node)
2026-04-13T19:43:29Z INFO [backup] Backup status cache refreshed
2026-04-13T19:43:29Z INFO [scheduler] Job backup-cache completed (took 675ms)
2026-04-13T19:43:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:43:37Z INFO [scheduler] Job agent-channel-health completed (took 7.986s)
2026-04-13T19:43:38Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:43:38Z INFO Health probes: 2 ok (of 2 probed)
2026-04-13T19:43:48Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:43:58Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:43:58Z INFO Health probes: 2 ok (of 2 probed)
2026-04-13T19:44:08Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:44:18Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:44:18Z INFO Health probes: 3 ok (of 3 probed)
2026-04-13T19:44:28Z INFO [scheduler] Running job: stack-scan
2026-04-13T19:44:28Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:44:28Z INFO [stacks] ScanStacks complete: 68 stacks found (8 deployed, 60 available)
2026-04-13T19:44:28Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:44:28Z INFO [scheduler] Job stack-scan completed (took 177ms)
2026-04-13T19:44:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:44:36Z INFO [scheduler] Job agent-channel-health completed (took 7.199s)
2026-04-13T19:44:38Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:44:48Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:44:58Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:45:08Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:45:18Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:45:28Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:45:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:45:36Z INFO [scheduler] Job agent-channel-health completed (took 7.219s)
2026-04-13T19:45:38Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:45:48Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:45:58Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:46:08Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:46:18Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:46:28Z INFO [scheduler] Running job: stack-scan
2026-04-13T19:46:28Z INFO [stacks] ScanStacks complete: 68 stacks found (8 deployed, 60 available)
2026-04-13T19:46:28Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:46:28Z INFO [scheduler] Job stack-scan completed (took 137ms)
2026-04-13T19:46:28Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:46:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:46:36Z INFO [scheduler] Job agent-channel-health completed (took 7.226s)
2026-04-13T19:46:38Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:46:48Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:46:58Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:47:08Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:47:18Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:47:28Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:47:29Z INFO [scheduler] Running job: agent-channel-health
2026-04-13T19:47:36Z INFO [scheduler] Job agent-channel-health completed (took 7.246s)
2026-04-13T19:47:38Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:47:48Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-04-13T19:47:49Z INFO [web] Login from 192.168.0.180