installer 1.36.1: http_code prints one status (R-1284, red-proved; committed, not published); R-1285 (BYO drop-in removed one bundle late), R-1286 (demo-hp kimai held since 10-14); release-2026-10-26 evidence: BYO proof, the three boxes on 0.343.0, Homepage gate on the drill
gates / gates (push) Successful in 12m58s

This commit is contained in:
admin committed 2026-10-25 18:32:04 +01:00
1 parent ade357a14e
commit 36ef8dcdec
20 files changed
+1750 -9

No files matched your search

+3 -3
View File
@@ -24,7 +24,7 @@
- DooPlex: something sets the shared checkout to „bare" (R-935).
## What we work on
- Dated checks, run by CC. 2026-10-26: tonight's database copy reaches the restic sweep (R-232); each hub daily job ran once on the clock-change day (R-1183). 2026-10-27: the weekly clean-up window (R-435; the logs of the last one were lost to restarts). 2026-10-30: D35, website and installer to Cloudflare, starts (R-1228, ruled yes).
- Dated checks, run by CC. 2026-10-26: each hub daily job ran once on the clock-change day (R-1183). 2026-10-27: the weekly clean-up window (R-435; the logs of the last one were lost to restarts). 2026-10-30: D35, website and installer to Cloudflare, starts (R-1228, ruled yes).
## Waiting on you — decide
@@ -50,13 +50,13 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te
6. BIOS: on the N100, read „After Power Loss"; on the HP, set „Bypass F1 Prompt on Configuration Changes" to Enable. Since 2026-10-14.
7. Pin the first Facebook post (R-917), and look at the new cover in the Facebook phone app (R-919). Since 2026-10-12.
**Monday 2026-10-26, release list (attended).** 1. Controller v0.343.0: the restore and backup fixes (R-1251–R-1253, R-1269–R-1271), R-1250. If not: the P1 stays open. 2. Agent v0.165.0 + your vouch: BYO keeps its SSH (R-1257). 3. Hub v0.170.0: the recovery package in one step (R-1254). 4. Catalog: n8n usage data off; OpenGist 256 MB.
**Day, 2026-10-23 — your do list.**
1. Print the break-glass sheet (`documentation/runbooks/break-glass-sheet.md`, the part from „The sheet”, one A4), fill it by hand, keep it outside the home (D32; R-923, R-924). If not: after a DooPlex loss no off-site copy opens.
2. On the hub's Hosts page: Discard appliance 49, and delete host Tester4-aa9466. If not: two dead entries alarm.
2. On the hub's Hosts page: Discard appliance 49 (host Tester4-aa9466 is gone). If not: a dead entry alarms.
3. Give the homelab session R-1241, R-1242, the lists at the end of `documentation/audits/plex2shell-felhom-impact-2026-10-23/README.md` and `documentation/audits/dooplex-alarms-2026-10-24/README.md`, and an upload alarm on DooPlex's `eno1` (~100 Mbit/s line; the flood peaked at 905). If not: the next hacked app on DooPlex can read the hub's backups.
**The rest**
- System poster needs a refresh: the new 01:40 „release copy” arrow from DooPlex to ep0 (D33); the hub-database label already says every 6 hours. If not: the drawing misses one copy.
- Publish installer 1.36.1 (tag + webpage sync): a wrong passphrase then reads „passphrase REJECTED", not a doubled code (R-1284).
- Rotate the reused homelab logins and decide the Gitea repo's visibility (R-925, P1). Since 2026-10-09.
- Two DooPlex leftovers: `leftovers/2026-10-14/first-post/` (keep or delete); the `deny`-list edit in homelab-manifests' `.claude/settings.json` (commit or drop). Since 2026-10-14.
- Before the first paying customer: Tandoor's permission (R-789); a lawyer for the licence list and the terms (R-802, R-813); SparkyFitness by e-mail (R-784).
@@ -0,0 +1,161 @@
2026/10/25 17:16:37 [INFO] local-api: channel up (agent 169.254.253.1:8443) — guest 9201, 3 mount(s) visible
2026/10/25 17:16:37 [INFO] local-api: mount mp0 → /var/lib/felhom (storage=local-lvm, class=, backup=true)
2026/10/25 17:16:37 [INFO] local-api: mount mp8 → /mnt/felhom-drives (storage=/mnt/felhom-drives, class=, backup=false)
2026/10/25 17:16:37 [INFO] local-api: mount mp9 → /etc/felhom-bootstrap (storage=/var/lib/felhom-agent/guests/9201/bootstrap, class=, backup=false)
2026/10/25 17:16:37 [INFO] felhom-controller 0.343.0 starting (customer: demo-felhom, domain: demo-felhom.eu)
2026/10/25 17:16:37 [INFO] [settings] Loaded settings from /opt/docker/felhom-controller/data/settings.json
2026/10/25 17:16:37 [INFO] Encryption key loaded from /opt/docker/felhom-controller/data/encryption.key
2026/10/25 17:16:37 [INFO] [stacks] Using compose command: docker compose
2026/10/25 17:16:37 [INFO] [stacks] ScanStacks complete: 68 stacks found (5 deployed, 63 available)
2026/10/25 17:16:37 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:16:37 [INFO] [stacks] InjectMissingFields: processed 5 stacks
2026/10/25 17:16:37 [INFO] [stacks] Encryption migration: no stacks needed migration
2026/10/25 17:16:37 [INFO] [quiesce] loop started (poll 5m0s, max-quiesce 30m0s)
2026/10/25 17:16:37 [INFO] [stacks] desired-state backfill: 0 app(s) recorded as running, 0 left unrecorded (state ambiguous — legacy boot behaviour retained)
2026/10/25 17:16:37 [INFO] [stacks] installed-images backfill: 0 app(s) recorded, 5 already had a record, 0 left unrecorded (could not be observed completely — unknown, which renders nothing)
2026/10/25 17:16:37 [INFO] [stacks] pin adoption: 0 pinned, 5 already pinned, 0 left unpinned (0 not completely observed, 0 running something the template no longer offers)
2026/10/25 17:16:37 [INFO] [stacks] applied-meta backfill (R-646): recorded 0 []; skipped 0 [] — not current with the catalog, their pinned version's .felhom.yml is no longer on the box
2026/10/25 17:16:37 [INFO] [sync] Starting catalog sync (repo: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git, interval: 15m0s)
2026/10/25 17:16:37 [INFO] [sync] Starting catalog sync
2026/10/25 17:16:37 [INFO] Metrics store opened at /opt/docker/felhom-controller/data/metrics.db
2026/10/25 17:16:37 [INFO] Metrics collector started (60s interval)
2026/10/25 17:16:37 [INFO] [sync] Pulling latest from https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git (branch: main)
2026/10/25 17:16:37 [INFO] [tier2] start-up: no leftover second-drive copy container
2026/10/25 17:16:37 [INFO] [backup] restore record wired: /opt/docker/felhom-controller/data/restore-status.json (interrupted at startup: false)
2026/10/25 17:16:37 [INFO] [backup] app-data run record wired: /opt/docker/felhom-controller/data/appdata-run.json (an earlier cut still shown: false)
2026/10/25 17:16:37 [INFO] Notifier enabled (hub: https://hub.felhom.eu)
2026/10/25 17:16:37 [INFO] [upwatch] the host's weekly line test: 92.6 Mbit/s (measured 2026-10-24T08:02:45Z)
2026/10/25 17:16:37 [INFO] [upwatch] upload and CPU watch started (line 93 Mbit/s, qBittorrent share 50 %)
2026/10/25 17:16:37 [INFO] [selfupdate] Post-update startup: update successful (0.342.0 → 0.343.0)
2026/10/25 17:16:37 [INFO] Self-update enabled (check every 6h, auto-update: false, auto-update time: 04:30)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: status-refresh (every 10s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: stack-scan (every 2m0s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: docker-socket-watch (every 15s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: docker-socket-users (every 5m0s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: health-probes (every 10s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: system-health (every 5m0s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: after-unclean-boot (every 30s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: deadapp-check (every 30s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: ring-spill (every 30s)
2026/10/25 17:16:37 [INFO] [offsite-apply] settle-gate: awaiting floor knowledge (first report ACK) before offsite apply
2026/10/25 17:16:37 [INFO] [scheduler] Daily job db-dump scheduled for 2026-10-26 02:30 CET
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: offsite-credential-retry (every 5m0s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: backup-cache (every 5m0s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: conversion-copy-release (every 1h0m0s)
2026/10/25 17:16:37 [INFO] [scheduler] Daily job tier2-backup scheduled for 2026-10-26 03:30 CET
2026/10/25 17:16:37 [INFO] [scheduler] Daily job offbox-backup scheduled for 2026-10-26 04:15 CET
2026/10/25 17:16:37 [INFO] [catch-up] controller start: no backup leg missed its last scheduled time — nothing to make up
2026/10/25 17:16:37 [INFO] [scheduler] Daily job offsite-abandon-sweep scheduled for 2026-10-26 05:10 CET
2026/10/25 17:16:37 [INFO] [scheduler] Daily job offsite-integrity scheduled for 2026-10-26 06:00 CET
2026/10/25 17:16:37 [INFO] [scheduler] Daily job offsite-proof scheduled for 2026-10-26 05:30 CET
2026/10/25 17:16:37 [INFO] [scheduler] Daily job metrics-prune scheduled for 2026-10-26 04:00 CET
2026/10/25 17:16:37 [INFO] [scheduler] Daily job fill-watch scheduled for 2026-10-26 03:30 CET
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: fill-watch-interval (every 10m0s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: hub-report (every 15m0s)
2026/10/25 17:16:37 [INFO] Hub reporting enabled (every 15m0s to https://hub.felhom.eu)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
2026/10/25 17:16:37 [INFO] ========== Startup Self-Test ==========
2026/10/25 17:16:37 [INFO] Event pushed: controller_updated (info) [hu-only] — Controller frissítve: 0.342.0 → 0.343.0
2026/10/25 17:16:37 [INFO] [PASS] Docker socket: reachable (v29.9.2)
2026/10/25 17:16:37 [INFO] [PASS] Stacks directory: /opt/docker/stacks
2026/10/25 17:16:37 [INFO] [PASS] Data directory: /opt/docker/felhom-controller/data (writable)
2026/10/25 17:16:37 [INFO] [PASS] System data path: /mnt/sys_drive
2026/10/25 17:16:37 [INFO] [PASS] Storage paths: 1 connected, 0 disconnected
2026/10/25 17:16:37 [INFO] [PASS] Git catalog: 65 app definitions found
2026/10/25 17:16:37 [INFO] [PASS] Hub connectivity: https://hub.felhom.eu reachable (HTTP 200)
2026/10/25 17:16:37 [INFO] [PASS] Metrics DB: 30.3 MB
2026/10/25 17:16:37 [INFO] ========================================
2026/10/25 17:16:37 [INFO] Self-test complete: 8 passed, 0 warnings, 0 failed
2026/10/25 17:16:37 [INFO] [scheduler] Starting scheduler with 23 jobs
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: geo-verify (every 6h0m0s)
2026/10/25 17:16:37 [INFO] Geo-restriction support enabled (CF API token configured)
2026/10/25 17:16:37 [INFO] [report] hub wait channel active (hold ≤240s)
2026/10/25 17:16:37 [INFO] Notification preferences synced to hub: source=household status=ok email=doodoo21@freemail.hu, events=[app_stopped_unhealthy app_update_held app_update_undone backup_failed db_dump_failed disk_critical disk_warning expected_backup_missed expected_dbdump_missed health_critical node_down offbox_enlarge_blocked storage_disconnected], cooldown=6h, email_cleared=false
2026/10/25 17:16:37 [INFO] [backup] Found 0 DB dump files across drives
2026/10/25 17:16:37 [INFO] [infra] connected felhom-controller to traefik-public
2026/10/25 17:16:37 [INFO] [backup] Discovered 0 databases
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/opengist/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/prowlarr/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/radarr/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/sonarr/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/qbittorrent/docker-compose.yml
2026/10/25 17:16:37 [INFO] [backup] Discovered app data: 5 apps
2026/10/25 17:16:37 [INFO] [web] sessions: restored 1 dashboard session(s) from disk (dropped 0 expired, 0 malformed)
2026/10/25 17:16:37 [INFO] [web] Auth: using password from settings.json
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: disk-health-check (every 1h0m0s)
2026/10/25 17:16:37 [INFO] [scheduler] Registered periodic job: agent-channel-health (every 1m0s)
2026/10/25 17:16:37 [INFO] Web UI listening on :8080
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/radarr/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/sonarr/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/qbittorrent/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/grimmory/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/romm/docker-compose.yml
2026/10/25 17:16:37 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/opengist — 2 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:37 [INFO] [stacks] SaveAppConfig: saved config for opengist
2026/10/25 17:16:37 [INFO] [stacks] opengist: install id 75d74fb92a0d5fcd recorded (an app installed before D47)
2026/10/25 17:16:37 [INFO] [backup] Recovery unit captured for opengist → /mnt/sys_drive/felhom-data/backups/primary/opengist (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:37 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/prowlarr — 2 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:37 [INFO] [stacks] SaveAppConfig: saved config for prowlarr
2026/10/25 17:16:37 [INFO] [stacks] prowlarr: install id 4a473060b914b0e2 recorded (an app installed before D47)
2026/10/25 17:16:37 [INFO] [backup] Recovery unit captured for prowlarr → /mnt/sys_drive/felhom-data/backups/primary/prowlarr (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:37 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radarr — 3 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:37 [INFO] [stacks] SaveAppConfig: saved config for radarr
2026/10/25 17:16:37 [INFO] [stacks] radarr: install id 55bfad619c700564 recorded (an app installed before D47)
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/grimmory/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/romm/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/grimmory/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/romm/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/grimmory/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:37 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/romm/docker-compose.yml
2026/10/25 17:16:37 [INFO] [backup] Recovery unit captured for radarr → /mnt/felhom-drives/hdd_1/backups/primary/radarr (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:37 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/sonarr — 3 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:37 [INFO] [stacks] SaveAppConfig: saved config for sonarr
2026/10/25 17:16:37 [INFO] [stacks] sonarr: install id c0df4f095aa926d7 recorded (an app installed before D47)
2026/10/25 17:16:37 [INFO] [backup] Recovery unit captured for sonarr → /mnt/felhom-drives/hdd_1/backups/primary/sonarr (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:37 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/qbittorrent — 4 env vars, 0 encrypted, 1 sensitive fields
2026/10/25 17:16:37 [INFO] [stacks] SaveAppConfig: saved config for qbittorrent
2026/10/25 17:16:37 [INFO] [stacks] qbittorrent: install id f82c26e291976a7f recorded (an app installed before D47)
2026/10/25 17:16:37 [INFO] [sync] Catalog sync complete
2026/10/25 17:16:37 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
2026/10/25 17:16:37 [INFO] [infra] connected felhom-controller to felhom-filebrowser (the FileBrowser default-login check, R-1126)
2026/10/25 17:16:37 [INFO] [backup] Recovery unit captured for qbittorrent → /mnt/felhom-drives/hdd_1/backups/primary/qbittorrent (images=1, secrets-referenced=1, data_keys=0, portable-carried=0/0, withheld=1)
2026/10/25 17:16:37 [INFO] [backup] Backup status cache refreshed
2026/10/25 17:16:37 [INFO] [web] FileBrowser sync — no config/compose change, ensured running without recreate (1 storage path(s))
2026/10/25 17:16:37 [INFO] [monitor] Health check: status=ok
2026/10/25 17:16:42 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:16:42 [INFO] [report] Building system report
2026/10/25 17:16:42 [INFO] Event pushed: controller_started (info) [hu-only] — Controller elindult (0.343.0)
2026/10/25 17:16:43 [INFO] [monitor] Health check: status=ok
2026/10/25 17:16:43 [INFO] [report] Hub report pushed successfully (5496 bytes)
2026/10/25 17:16:43 [INFO] [settings] Settings saved
2026/10/25 17:16:43 [INFO] Startup hub report sent
2026/10/25 17:16:47 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:16:47 [INFO] [offsite-apply] settle-gate: GO — at/above floor 0.343.0 (we are 0.343.0), no managed update running
2026/10/25 17:16:47 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:16:47 [INFO] Health probes: 5 ok (of 5 probed)
2026/10/25 17:16:52 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:16:52 [INFO] [bootrecon] boot window: fleet settled after 10s (3 identical samples 5s apart) — sweeping
2026/10/25 17:16:52 [INFO] [bootrecon] Boot reconciliation: no boot-orphaned apps (nothing to start)
2026/10/25 17:16:57 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:17:07 [INFO] [deadapp] boot grace 1m30s: the host's last boot was clean (R-856)
2026/10/25 17:17:07 [INFO] [aftercrash] a normal (or unknown) boot — no app repair after an unclean boot (R-1007)
2026/10/25 17:17:07 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:17:08 [INFO] [selfupdate] Current version 0.343.0 is up to date
2026/10/25 17:17:17 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:17:27 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:17:37 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/25 17:17:37 [INFO] [scheduler] Running job: agent-channel-health
2026/10/25 17:17:38 [INFO] [scheduler] Job agent-channel-health completed (took 859ms)
2026/10/25 17:17:47 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
@@ -0,0 +1,731 @@
2026/10/25 17:16:53 main.go:3990: [INFO] local-api: channel up (agent 169.254.253.1:8443) — guest 9201, 3 mount(s) visible
2026/10/25 17:16:53 main.go:3993: [INFO] local-api: mount mp8 → /mnt/felhom-drives (storage=/mnt/felhom-drives, class=, backup=false)
2026/10/25 17:16:53 main.go:3993: [INFO] local-api: mount mp0 → /var/lib/felhom (storage=local-lvm, class=, backup=true)
2026/10/25 17:16:53 main.go:3993: [INFO] local-api: mount mp9 → /etc/felhom-bootstrap (storage=/var/lib/felhom-agent/guests/9201/bootstrap, class=, backup=false)
2026/10/25 17:16:53 main.go:355: [INFO] felhom-controller 0.343.0 starting (customer: demo-hp, domain: enkisfelhom.hu)
2026/10/25 17:16:53 settings.go:813: [INFO] [settings] Loaded settings from /opt/docker/felhom-controller/data/settings.json
2026/10/25 17:16:53 settings.go:1861: [DEBUG] [settings] AutoDiscoverStoragePaths discovered=[/mnt/felhom-drives/hdd_1] existing=1
2026/10/25 17:16:53 main.go:381: [INFO] Encryption key loaded from /opt/docker/felhom-controller/data/encryption.key
2026/10/25 17:16:53 manager.go:396: [INFO] [stacks] Using compose command: docker compose
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=true composePath=/opt/docker/stacks/adventurelog/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=true composePath=/opt/docker/stacks/bentopdf/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=true composePath=/opt/docker/stacks/bookstack/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=true composePath=/opt/docker/stacks/calibre-web/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "cloudflared" deployed=false composePath=/opt/docker/stacks/cloudflared/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "dawarich" deployed=false composePath=/opt/docker/stacks/dawarich/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=true composePath=/opt/docker/stacks/docmost/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grimmory" deployed=false composePath=/opt/docker/stacks/grimmory/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grocy" deployed=false composePath=/opt/docker/stacks/grocy/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=true composePath=/opt/docker/stacks/home-assistant/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=false composePath=/opt/docker/stacks/immich/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "karakeep" deployed=false composePath=/opt/docker/stacks/karakeep/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=true composePath=/opt/docker/stacks/kimai/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "lubelogger" deployed=false composePath=/opt/docker/stacks/lubelogger/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "metube" deployed=false composePath=/opt/docker/stacks/metube/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nzbget" deployed=false composePath=/opt/docker/stacks/nzbget/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=true composePath=/opt/docker/stacks/opengist/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=true composePath=/opt/docker/stacks/plex/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "prowlarr" deployed=false composePath=/opt/docker/stacks/prowlarr/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "qbittorrent" deployed=false composePath=/opt/docker/stacks/qbittorrent/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radicale" deployed=false composePath=/opt/docker/stacks/radicale/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=true composePath=/opt/docker/stacks/romm/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sabnzbd" deployed=false composePath=/opt/docker/stacks/sabnzbd/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "samba" deployed=false composePath=/opt/docker/stacks/samba/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=false composePath=/opt/docker/stacks/tandoor/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "transmission" deployed=false composePath=/opt/docker/stacks/transmission/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml
2026/10/25 17:16:53 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml
2026/10/25 17:16:53 manager.go:1960: [DEBUG] [stacks] getCatalogTemplateSlugs: found 65 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates
2026/10/25 17:16:53 manager.go:757: [DEBUG] [stacks] ScanStacks: catalog has 65 template slugs for orphan detection
2026/10/25 17:16:53 manager.go:788: [INFO] [stacks] ScanStacks complete: 69 stacks found (12 deployed, 57 available)
2026/10/25 17:16:54 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:16:54 deploy.go:1389: [DEBUG] [stacks] InjectMissingFields: checking 12 stacks
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack privatebin — 2 deploy fields, 2 existing env vars
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack calibre-web — 5 deploy fields, 4 existing env vars
2026/10/25 17:16:54 deploy.go:1470: [WARN] [stacks] Stack calibre-web: new field ADMIN_PASSWORD (type=password) requires manual configuration
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack home-assistant — 2 deploy fields, 2 existing env vars
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack bentopdf — 2 deploy fields, 2 existing env vars
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack plex — 4 deploy fields, 3 existing env vars
2026/10/25 17:16:54 deploy.go:1470: [WARN] [stacks] Stack plex: new field PLEX_CLAIM (type=text) requires manual configuration
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack romm — 6 deploy fields, 6 existing env vars
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack bookstack — 5 deploy fields, 4 existing env vars
2026/10/25 17:16:54 deploy.go:1470: [WARN] [stacks] Stack bookstack: new field ADMIN_PASSWORD (type=password) requires manual configuration
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack paperless-ngx — 8 deploy fields, 8 existing env vars
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack opengist — 2 deploy fields, 2 existing env vars
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack docmost — 4 deploy fields, 4 existing env vars
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack adventurelog — 4 deploy fields, 5 existing env vars
2026/10/25 17:16:54 deploy.go:1411: [DEBUG] [stacks] InjectMissingFields: checking stack kimai — 5 deploy fields, 5 existing env vars
2026/10/25 17:16:54 deploy.go:1482: [INFO] [stacks] InjectMissingFields: processed 12 stacks
2026/10/25 17:16:54 manager.go:535: [DEBUG] [stacks] MigrateEncryption: checking 12 deployed stacks for plaintext sensitive values
2026/10/25 17:16:54 manager.go:555: [DEBUG] [stacks] MigrateEncryption: checking stack "adventurelog" (2 sensitive fields)
2026/10/25 17:16:54 manager.go:555: [DEBUG] [stacks] MigrateEncryption: checking stack "kimai" (2 sensitive fields)
2026/10/25 17:16:54 manager.go:555: [DEBUG] [stacks] MigrateEncryption: checking stack "calibre-web" (2 sensitive fields)
2026/10/25 17:16:54 manager.go:555: [DEBUG] [stacks] MigrateEncryption: checking stack "romm" (3 sensitive fields)
2026/10/25 17:16:54 manager.go:555: [DEBUG] [stacks] MigrateEncryption: checking stack "bookstack" (3 sensitive fields)
2026/10/25 17:16:54 manager.go:555: [DEBUG] [stacks] MigrateEncryption: checking stack "paperless-ngx" (3 sensitive fields)
2026/10/25 17:16:54 manager.go:555: [DEBUG] [stacks] MigrateEncryption: checking stack "docmost" (2 sensitive fields)
2026/10/25 17:16:54 manager.go:579: [INFO] [stacks] Encryption migration: no stacks needed migration
2026/10/25 17:16:54 quiesce.go:192: [INFO] [quiesce] loop started (poll 5m0s, max-quiesce 30m0s)
2026/10/25 17:16:54 desiredstate.go:137: [INFO] [stacks] desired-state backfill: 0 app(s) recorded as running, 0 left unrecorded (state ambiguous — legacy boot behaviour retained)
2026/10/25 17:16:54 installed.go:537: [INFO] [stacks] installed-images backfill: 0 app(s) recorded, 12 already had a record, 0 left unrecorded (could not be observed completely — unknown, which renders nothing)
2026/10/25 17:16:54 pin.go:290: [INFO] [stacks] pin adoption: 0 pinned, 12 already pinned, 0 left unpinned (0 not completely observed, 0 running something the template no longer offers)
2026/10/25 17:16:54 undo.go:676: [INFO] [stacks] applied-meta backfill (R-646): recorded 0 []; skipped 0 [] — not current with the catalog, their pinned version's .felhom.yml is no longer on the box
2026/10/25 17:16:54 sync.go:137: [INFO] [sync] Starting catalog sync (repo: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git, interval: 15m0s)
2026/10/25 17:16:54 sync.go:228: [INFO] [sync] Starting catalog sync
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] CPUCollector.Start: sampleRate=5s
2026/10/25 17:16:54 main.go:570: [INFO] Metrics store opened at /opt/docker/felhom-controller/data/metrics.db
2026/10/25 17:16:54 main.go:579: [INFO] Metrics collector started (60s interval)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
2026/10/25 17:16:54 sync.go:358: [INFO] [sync] Pulling latest from https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git (branch: main)
2026/10/25 17:16:54 sync.go:360: [DEBUG] [sync] git fetch --depth 1 origin main in /opt/docker/felhom-controller/data/catalog-cache
2026/10/25 17:16:54 sync.go:787: [DEBUG] [sync] Running: git fetch --depth 1 origin main
2026/10/25 17:16:54 [INFO] [tier2] start-up: no leftover second-drive copy container
2026/10/25 17:16:54 restore_record_wiring.go:37: [INFO] [backup] restore record wired: /opt/docker/felhom-controller/data/restore-status.json (interrupted at startup: false)
2026/10/25 17:16:54 run_record_wiring.go:29: [INFO] [backup] app-data run record wired: /opt/docker/felhom-controller/data/appdata-run.json (an earlier cut still shown: false)
2026/10/25 17:16:54 notifier.go:81: [INFO] Notifier enabled (hub: https://hub.felhom.eu)
2026/10/25 17:16:54 service.go:79: [INFO] [upwatch] the host's weekly line test: 94.4 Mbit/s (measured 2026-10-24T08:00:33Z)
2026/10/25 17:16:54 upwatch_wiring.go:149: [INFO] [upwatch] upload and CPU watch started (line 94 Mbit/s, qBittorrent share 50 %)
2026/10/25 17:16:54 updater.go:104: [DEBUG] [selfupdate] VerifyStartup: checking update state in /opt/docker/felhom-controller/data
2026/10/25 17:16:54 updater.go:104: [DEBUG] [selfupdate] VerifyStartup: pending update found — target=0.343.0 previous=0.342.0
2026/10/25 17:16:54 updater.go:855: [INFO] [selfupdate] Post-update startup: update successful (0.342.0 → 0.343.0)
2026/10/25 17:16:54 main.go:864: [INFO] Self-update enabled (check every 6h, auto-update: false, auto-update time: 04:30)
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: status-refresh (every 10s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="status-refresh" interval=10s totalJobs=1
2026/10/25 17:16:54 notifier.go:317: [DEBUG] PushEvent: type=controller_updated severity=info url=https://hub.felhom.eu/api/v1/event
2026/10/25 17:16:54 offsiteapply.go:148: [INFO] [offsite-apply] settle-gate: awaiting floor knowledge (first report ACK) before offsite apply
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: stack-scan (every 2m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="stack-scan" interval=2m0s totalJobs=2
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: docker-socket-watch (every 15s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="docker-socket-watch" interval=15s totalJobs=3
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: docker-socket-users (every 5m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="docker-socket-users" interval=5m0s totalJobs=4
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: health-probes (every 10s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="health-probes" interval=10s totalJobs=5
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: system-health (every 5m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="system-health" interval=5m0s totalJobs=6
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: after-unclean-boot (every 30s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="after-unclean-boot" interval=30s totalJobs=7
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: deadapp-check (every 30s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="deadapp-check" interval=30s totalJobs=8
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: ring-spill (every 30s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="ring-spill" interval=30s totalJobs=9
2026/10/25 17:16:54 scheduler.go:150: [INFO] [scheduler] Daily job db-dump scheduled for 2026-10-26 02:30 CET
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="db-dump" schedule="02:30" nextRun=2026-10-26T02:30:00+01:00 totalJobs=10
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: offsite-credential-retry (every 5m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="offsite-credential-retry" interval=5m0s totalJobs=11
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: backup-cache (every 5m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="backup-cache" interval=5m0s totalJobs=12
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: conversion-copy-release (every 1h0m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="conversion-copy-release" interval=1h0m0s totalJobs=13
2026/10/25 17:16:54 scheduler.go:150: [INFO] [scheduler] Daily job tier2-backup scheduled for 2026-10-26 03:30 CET
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="tier2-backup" schedule="03:30" nextRun=2026-10-26T03:30:00+01:00 totalJobs=14
2026/10/25 17:16:54 scheduler.go:150: [INFO] [scheduler] Daily job offbox-backup scheduled for 2026-10-26 04:15 CET
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="offbox-backup" schedule="04:15" nextRun=2026-10-26T04:15:00+01:00 totalJobs=15
2026/10/25 17:16:54 nightchain.go:284: [INFO] [catch-up] controller start: no backup leg missed its last scheduled time — nothing to make up
2026/10/25 17:16:54 scheduler.go:150: [INFO] [scheduler] Daily job offsite-abandon-sweep scheduled for 2026-10-26 05:10 CET
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="offsite-abandon-sweep" schedule="05:10" nextRun=2026-10-26T05:10:00+01:00 totalJobs=16
2026/10/25 17:16:54 scheduler.go:150: [INFO] [scheduler] Daily job offsite-integrity scheduled for 2026-10-26 06:00 CET
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="offsite-integrity" schedule="06:00" nextRun=2026-10-26T06:00:00+01:00 totalJobs=17
2026/10/25 17:16:54 scheduler.go:150: [INFO] [scheduler] Daily job offsite-proof scheduled for 2026-10-26 05:30 CET
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="offsite-proof" schedule="05:30" nextRun=2026-10-26T05:30:00+01:00 totalJobs=18
2026/10/25 17:16:54 scheduler.go:150: [INFO] [scheduler] Daily job metrics-prune scheduled for 2026-10-26 04:00 CET
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="metrics-prune" schedule="04:00" nextRun=2026-10-26T04:00:00+01:00 totalJobs=19
2026/10/25 17:16:54 scheduler.go:150: [INFO] [scheduler] Daily job fill-watch scheduled for 2026-10-26 03:30 CET
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="fill-watch" schedule="03:30" nextRun=2026-10-26T03:30:00+01:00 totalJobs=20
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: fill-watch-interval (every 10m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="fill-watch-interval" interval=10m0s totalJobs=21
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: hub-report (every 15m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="hub-report" interval=15m0s totalJobs=22
2026/10/25 17:16:54 main.go:1823: [INFO] Hub reporting enabled (every 15m0s to https://hub.felhom.eu)
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="selfupdate-check" interval=6h0m0s totalJobs=23
2026/10/25 17:16:54 selftest.go:50: [INFO] ========== Startup Self-Test ==========
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈16630MB avail≈9268MB
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=10990616 → total=25898MB avail=9268MB used=16630MB (64.2%)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=20.8GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=225.2GB avail=665.0GB (25.3%)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.71 1.15 1.44 14/1581 105" → 1m=1.71 5m=1.15 15m=1.44
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 58.6°C (hwmon2)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetInfo done in 116ms — mem=16630MB/25898MB (64.2%), rootDisk=20.8GB/68.7GB (31.8%), load=1.71/1.15/1.44, temp=58.6°C (hwmon2), cpu=0.0%
2026/10/25 17:16:54 notifier.go:343: [DEBUG] PushEvent: controller_updated pushed OK (HTTP 200)
2026/10/25 17:16:54 selftest.go:68: [INFO] [PASS] Docker socket: reachable (v29.9.2)
2026/10/25 17:16:54 notifier.go:355: [INFO] Event pushed: controller_updated (info) [hu-only] — Controller frissítve: 0.342.0 → 0.343.0
2026/10/25 17:16:54 selftest.go:68: [INFO] [PASS] Stacks directory: /opt/docker/stacks
2026/10/25 17:16:54 selftest.go:68: [INFO] [PASS] Data directory: /opt/docker/felhom-controller/data (writable)
2026/10/25 17:16:54 selftest.go:68: [INFO] [PASS] System data path: /mnt/sys_drive
2026/10/25 17:16:54 selftest.go:68: [INFO] [PASS] Storage paths: 1 connected, 0 disconnected
2026/10/25 17:16:54 selftest.go:68: [INFO] [PASS] Git catalog: 65 app definitions found
2026/10/25 17:16:54 selftest.go:68: [INFO] [PASS] Hub connectivity: https://hub.felhom.eu reachable (HTTP 200)
2026/10/25 17:16:54 selftest.go:68: [INFO] [PASS] Metrics DB: 107.4 MB
2026/10/25 17:16:54 selftest.go:70: [INFO] ========================================
2026/10/25 17:16:54 selftest.go:71: [INFO] Self-test complete: 8 passed, 0 warnings, 0 failed
2026/10/25 17:16:54 scheduler.go:224: [INFO] [scheduler] Starting scheduler with 23 jobs
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] scheduler started: periodic=15 daily=8
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: geo-verify (every 6h0m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="geo-verify" interval=6h0m0s totalJobs=24
2026/10/25 17:16:54 main.go:2088: [INFO] Geo-restriction support enabled (CF API token configured)
2026/10/25 17:16:54 server.go:318: [DEBUG] [web] NewServer: initializing web server v0.343.0
2026/10/25 17:16:54 server.go:319: [DEBUG] [web] NewServer: backup=true scheduler=true alertMgr=true notifier=true updater=true
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job offsite-abandon-sweep: next run at 2026-10-26 05:10:00 CET (waiting 10h53m6s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job db-dump: next run at 2026-10-26 02:30:00 CET (waiting 8h13m6s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job tier2-backup: next run at 2026-10-26 03:30:00 CET (waiting 9h13m6s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job offbox-backup: next run at 2026-10-26 04:15:00 CET (waiting 9h58m6s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job offsite-integrity: next run at 2026-10-26 06:00:00 CET (waiting 11h43m6s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job offsite-proof: next run at 2026-10-26 05:30:00 CET (waiting 11h13m6s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job metrics-prune: next run at 2026-10-26 04:00:00 CET (waiting 9h43m6s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] daily job fill-watch: next run at 2026-10-26 03:30:00 CET (waiting 9h13m6s)
2026/10/25 17:16:54 waiter.go:95: [INFO] [report] hub wait channel active (hold ≤240s)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
2026/10/25 17:16:54 backup.go:540: [DEBUG] groupStacksByDrive: /mnt/sys_drive → [adventurelog, bentopdf, bookstack, docmost, home-assistant, kimai, opengist, privatebin]
2026/10/25 17:16:54 backup.go:540: [DEBUG] groupStacksByDrive: /mnt/felhom-drives/hdd_1 → [calibre-web, paperless-ngx, plex, romm]
2026/10/25 17:16:54 backup.go:1246: [INFO] [backup] Found 19 DB dump files across drives
2026/10/25 17:16:54 waiter.go:132: [DEBUG] [report] wait baseline generation=1
2026/10/25 17:16:54 dbdump.go:130: [DEBUG] DiscoverDatabases: running docker ps to find database containers
2026/10/25 17:16:54 server.go:399: [DEBUG] [web] loadTemplates: lang=hu loaded 99 templates, 2085 markers expanded in 63.1108ms
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈16652MB avail≈9246MB
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=10965152 → total=25898MB avail=9246MB used=16652MB (64.3%)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=20.8GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=225.2GB avail=665.0GB (25.3%)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.71 1.15 1.44 15/1599 194" → 1m=1.71 5m=1.15 15m=1.44
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 59.0°C (hwmon2)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetInfo done in 90ms — mem=16652MB/25898MB (64.3%), rootDisk=20.8GB/68.7GB (31.8%), load=1.71/1.15/1.44, temp=59.0°C (hwmon2), cpu=0.0%
2026/10/25 17:16:54 healthcheck.go:128: [DEBUG] [monitor] Raw values: disk=31.8%, hdd=25.3% (configured=true), mem=64.3% (16652MB/25898MB), cpu=0.0%, temp=59.0°C (hwmon2)
2026/10/25 17:16:54 healthcheck.go:409: [DEBUG] [monitor] SSD disk: OK (32%)
2026/10/25 17:16:54 healthcheck.go:445: [DEBUG] [monitor] Memory: OK (64%)
2026/10/25 17:16:54 healthcheck.go:483: [DEBUG] [monitor] Temperature: OK (59°C)
2026/10/25 17:16:54 infra.go:378: [INFO] [infra] connected felhom-controller to traefik-public
2026/10/25 17:16:54 server.go:399: [DEBUG] [web] loadTemplates: lang=en loaded 99 templates, 2085 markers expanded in 50.250005ms
2026/10/25 17:16:54 session_store.go:113: [INFO] [web] sessions: restored 38 dashboard session(s) from disk (dropped 0 expired, 0 malformed)
2026/10/25 17:16:54 server.go:339: [INFO] [web] Auth: using password from settings.json
2026/10/25 17:16:54 sync.go:787: [DEBUG] [sync] Running: git reset --hard origin/main
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: disk-health-check (every 1h0m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="disk-health-check" interval=1h0m0s totalJobs=25
2026/10/25 17:16:54 scheduler.go:120: [INFO] [scheduler] Registered periodic job: agent-channel-health (every 1m0s)
2026/10/25 17:16:54 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="agent-channel-health" interval=1m0s totalJobs=26
2026/10/25 17:16:54 dbdump.go:149: [DEBUG] DiscoverDatabases: docker ps output: 496564edc900 felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.343.0
ef0e57fa0965 romm romm rommapp/romm:5.3.1
01aca2e96906 romm-redis romm redis:7-alpine
d3c8ecc410f7 romm-db romm mariadb:11.8
0a833b20453d privatebin privatebin privatebin/pdo:2.0.6
ffbe0c7120c6 plex plex plexinc/pms-docker:1.43.5.11029-6bf8af3f0
a3375a5404e3 paperless-webserver paperless-ngx ghcr.io/paperless-ngx/paperless-ngx:2.20.15
9619a5283d77 paperless-postgres paperless-ngx postgres:18-alpine
b4082466c0fe pape...
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container felhom-controller (image=gitea.dooplex.hu/admin/felhom-controller:0.343.0, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container romm (image=rommapp/romm:5.3.1, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container romm-redis (image=redis:7-alpine, not a database)
2026/10/25 17:16:54 dbdump.go:178: [DEBUG] DiscoverDatabases: found mariadb container: romm-db (id=d3c8ecc410f7)
2026/10/25 17:16:54 server.go:983: [DEBUG] [web] CatchAllMiddleware: controller host=felhom.enkisfelhom.hu
2026/10/25 17:16:54 main.go:2364: [INFO] Web UI listening on :8080
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] actualbudget/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] actualbudget/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/calibre-web/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/plex/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/romm/docker-compose.yml
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] adventurelog/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] adventurelog/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] audiobookshelf/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] audiobookshelf/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] bentopdf/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] bentopdf/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] bookstack/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] bookstack/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] calcom/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] calcom/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] calibre-web/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] calibre-web/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 manager.go:258: [DEBUG] [integrations] ReapplyConfigForTarget: target=filebrowser integrations=0
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] claper/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] claper/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] code-server/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] code-server/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] crafty-controller/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] crafty-controller/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] dawarich/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] dawarich/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 dbdump.go:198: [DEBUG] DiscoverDatabases: romm-db → stack=romm, dbUser=root, dbName=romm
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container privatebin (image=privatebin/pdo:2.0.6, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container plex (image=plexinc/pms-docker:1.43.5.11029-6bf8af3f0, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-webserver (image=ghcr.io/paperless-ngx/paperless-ngx:2.20.15, not a database)
2026/10/25 17:16:54 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: paperless-postgres (id=9619a5283d77)
2026/10/25 17:16:54 dbdump.go:915: [DEBUG] DiscoverDatabases: paperless-postgres → stack "paperless-ngx" from the compose project label (the container name would have given "paperless")
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] docmost/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] docmost/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] emby/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] emby/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] ghost/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] ghost/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] gitea/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] gitea/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] glance/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] glance/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] gokapi/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] gokapi/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] grafana/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] grafana/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] gramps-web/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] gramps-web/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] grimmory/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] grimmory/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] grocy/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] grocy/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] home-assistant/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] home-assistant/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] homebox/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] homebox/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] homepage/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 dbdump.go:198: [DEBUG] DiscoverDatabases: paperless-postgres → stack=paperless-ngx, dbUser=paperless, dbName=paperless
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-redis (image=redis:7-alpine, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container opengist (image=ghcr.io/thomiceli/opengist:1.15, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container home-assistant (image=ghcr.io/home-assistant/home-assistant:2026.9.4, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container docmost (image=docmost/docmost:0.96.0, not a database)
2026/10/25 17:16:54 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: docmost-postgres (id=dd941d6240e4)
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] homepage/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] immich/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] immich/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] jellyfin/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] jellyfin/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] karakeep/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] karakeep/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 healthcheck.go:145: [DEBUG] [monitor] Docker daemon: OK
2026/10/25 17:16:54 healthcheck.go:153: [DEBUG] [monitor] Checking 4 protected containers: [traefik cloudflared felhom-controller filebrowser]
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] kimai/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] kimai/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] komga/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] komga/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] lubelogger/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] lubelogger/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] mealie/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] mealie/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] metube/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] metube/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] n8n/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] n8n/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] navidrome/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] navidrome/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] nextcloud/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] nextcloud/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] nzbget/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] nzbget/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 dbdump.go:198: [DEBUG] DiscoverDatabases: docmost-postgres → stack=docmost, dbUser=docmost, dbName=docmost
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container docmost-redis (image=redis:7-alpine, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container calibre-web (image=crocodilestick/calibre-web-automated:v4.0.8, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container bookstack (image=lscr.io/linuxserver/bookstack:26.09.1, not a database)
2026/10/25 17:16:54 dbdump.go:178: [DEBUG] DiscoverDatabases: found mariadb container: bookstack-db (id=fbe94713ce78)
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] onlyoffice/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] onlyoffice/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] opengist/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] opengist/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] outline/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] outline/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] paperless-ngx/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] paperless-ngx/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] papra/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] papra/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] plant-it/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] plant-it/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] plex/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] plex/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] privatebin/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] privatebin/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] prowlarr/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] prowlarr/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] qbittorrent/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] qbittorrent/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] radarr/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] radarr/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] radicale/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] radicale/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] rallly/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] rallly/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] recipe-importer/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] recipe-importer/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] romm/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] romm/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] sabnzbd/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] sabnzbd/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] seerr/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] seerr/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] sonarr/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] sonarr/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] sparkyfitness/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] sparkyfitness/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] tandoor/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] tandoor/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] termix/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] termix/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 dbdump.go:198: [DEBUG] DiscoverDatabases: bookstack-db → stack=bookstack, dbUser=root, dbName=bookstack
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container bentopdf (image=ghcr.io/alam00000/bentopdf:v2.8.6, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container adventurelog (image=ghcr.io/seanmorley15/adventurelog-backend:v0.13.0, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container adventurelog-frontend (image=ghcr.io/seanmorley15/adventurelog-frontend:v0.13.0, not a database)
2026/10/25 17:16:54 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: adventurelog-postgres (id=c84ab95505dc)
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] transmission/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] transmission/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] uptime-kuma/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] uptime-kuma/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] vaultwarden/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] vaultwarden/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] vikunja/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] vikunja/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] wanderer/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] wanderer/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] wger/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] wger/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] wishlist/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] wishlist/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] zipline/docker-compose.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:559: [DEBUG] [sync] zipline/.felhom.yml: hash match, skipped
2026/10/25 17:16:54 sync.go:299: [INFO] [sync] Catalog sync complete
2026/10/25 17:16:54 sync.go:142: [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
2026/10/25 17:16:54 dbdump.go:198: [DEBUG] DiscoverDatabases: adventurelog-postgres → stack=adventurelog, dbUser=adventurelog, dbName=adventurelog
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container filebrowser (image=gtstef/filebrowser:1.5.6-stable, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container cloudflared (image=cloudflare/cloudflared:2026.9.3, not a database)
2026/10/25 17:16:54 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container traefik (image=traefik:v3.7.13, not a database)
2026/10/25 17:16:54 dbdump.go:205: [DEBUG] DiscoverDatabases: found 5 database(s), skipped 19 non-DB container(s)
2026/10/25 17:16:54 dbdump.go:208: [INFO] [backup] Discovered 5 databases
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/adventurelog/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bentopdf/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/calibre-web/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/docmost/docker-compose.yml
2026/10/25 17:16:54 handlers.go:3942: [INFO] [web] FileBrowser sync — no config/compose change, ensured running without recreate (1 storage path(s))
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/home-assistant/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/kimai/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/opengist/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/plex/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/romm/docker-compose.yml
2026/10/25 17:16:54 [INFO] [backup] Discovered app data: 12 apps
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.7 GB used=20.8 GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/adventurelog — 5 env vars, 0 encrypted, 2 sensitive fields
2026/10/25 17:16:54 setup_gate.go:511: [DEBUG] [stacks] romm: setup probe unreadable (Get "http://romm:8080/api/heartbeat": dial tcp: lookup romm on 192.168.0.250:53: no such host) — gate stays closed
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for adventurelog
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] adventurelog: install id 2c99f493b77aed71 recorded (an app installed before D47)
2026/10/25 17:16:54 watch.go:237: [DEBUG] [upwatch] plex: the app did not answer (resolving the plex container: lookup plex on 192.168.0.250:53: no such host) — treated as no number this minute
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for adventurelog → /mnt/sys_drive/felhom-data/backups/primary/adventurelog (images=3, secrets-referenced=2, data_keys=1, portable-carried=2/2, withheld=0)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.7 GB used=20.8 GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/bentopdf — 2 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for bentopdf
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] bentopdf: install id 17f35f4c0d87a32f recorded (an app installed before D47)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for bentopdf → /mnt/sys_drive/felhom-data/backups/primary/bentopdf (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.7 GB used=20.8 GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/bookstack — 4 env vars, 0 encrypted, 3 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for bookstack
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] bookstack: install id e172a289fd47c51d recorded (an app installed before D47)
2026/10/25 17:16:54 healthcheck.go:161: [DEBUG] [monitor] All protected containers running
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/hdd_1" total=937.8 GB used=225.2 GB avail=665.0GB (25.3%)
2026/10/25 17:16:54 healthcheck.go:176: [INFO] [monitor] Health check: status=ok
2026/10/25 17:16:54 healthcheck.go:180: [DEBUG] [monitor] Final status: ok (issues=0, warnings=0, info=4)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for bookstack → /mnt/sys_drive/felhom-data/backups/primary/bookstack (images=2, secrets-referenced=3, data_keys=1, portable-carried=2/2, withheld=1)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/hdd_1" total=937.8 GB used=225.2 GB avail=665.0GB (25.3%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/calibre-web — 4 env vars, 0 encrypted, 2 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for calibre-web
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] calibre-web: install id a3ee4c39e2439d7b recorded (an app installed before D47)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for calibre-web → /mnt/felhom-drives/hdd_1/backups/primary/calibre-web (images=1, secrets-referenced=2, data_keys=0, portable-carried=1/1, withheld=1)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.7 GB used=20.8 GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/docmost — 4 env vars, 0 encrypted, 2 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for docmost
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] docmost: install id 7f50f75f42a6eab1 recorded (an app installed before D47)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for docmost → /mnt/sys_drive/felhom-data/backups/primary/docmost (images=3, secrets-referenced=2, data_keys=0, portable-carried=2/2, withheld=0)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.7 GB used=20.8 GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/home-assistant — 2 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for home-assistant
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] home-assistant: install id 919730e70cb01fe9 recorded (an app installed before D47)
2026/10/25 17:16:54 filebrowser_gate.go:68: [INFO] [infra] connected felhom-controller to felhom-filebrowser (the FileBrowser default-login check, R-1126)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for home-assistant → /mnt/sys_drive/felhom-data/backups/primary/home-assistant (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.7 GB used=20.8 GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/opengist — 2 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for opengist
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] opengist: install id d956c5ae9ac7b528 recorded (an app installed before D47)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for opengist → /mnt/sys_drive/felhom-data/backups/primary/opengist (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/hdd_1" total=937.8 GB used=225.2 GB avail=665.0GB (25.3%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/paperless-ngx — 8 env vars, 0 encrypted, 3 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for paperless-ngx
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] paperless-ngx: install id a1c1bb5a5103b37d recorded (an app installed before D47)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for paperless-ngx → /mnt/felhom-drives/hdd_1/backups/primary/paperless-ngx (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/hdd_1" total=937.8 GB used=225.2 GB avail=665.0GB (25.3%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/plex — 3 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for plex
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] plex: install id 8a3ddf75968d32ab recorded (an app installed before D47)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for plex → /mnt/felhom-drives/hdd_1/backups/primary/plex (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.7 GB used=20.8 GB avail=44.5GB (31.8%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/privatebin — 2 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for privatebin
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] privatebin: install id 19340734630f84cd recorded (an app installed before D47)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for privatebin → /mnt/sys_drive/felhom-data/backups/primary/privatebin (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:54 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/hdd_1" total=937.8 GB used=225.2 GB avail=665.0GB (25.3%)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/romm — 6 env vars, 0 encrypted, 3 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for romm
2026/10/25 17:16:54 deploy.go:1682: [INFO] [stacks] romm: install id e4637a49405a2049 recorded (an app installed before D47)
2026/10/25 17:16:54 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for romm → /mnt/felhom-drives/hdd_1/backups/primary/romm (images=3, secrets-referenced=3, data_keys=0, portable-carried=3/3, withheld=0)
2026/10/25 17:16:54 backup.go:1317: [INFO] [backup] Backup status cache refreshed
2026/10/25 17:16:55 client.go:67: [DEBUG] [agentapi] agent version seen: 0.165.0 (was "")
2026/10/25 17:16:55 client.go:1276: [DEBUG] [agentapi] GET /disks -> 200 (862ms)
2026/10/25 17:16:56 client.go:1276: [DEBUG] [agentapi] GET /disks -> 200 (840ms)
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] CPUCollector: first sample — cpu=61.9% (idle=1549 total=4063)
2026/10/25 17:16:59 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:16:59 builder.go:37: [INFO] [report] Building system report
2026/10/25 17:16:59 builder.go:40: [DEBUG] [report] BuildReport: starting — version=0.343.0, storagePaths=1
2026/10/25 17:16:59 notifier.go:317: [DEBUG] PushEvent: type=controller_started severity=info url=https://hub.felhom.eu/api/v1/event
2026/10/25 17:16:59 builder.go:62: [DEBUG] [report] BuildReport: configHash=042b71a3123d... (2989 bytes)
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
2026/10/25 17:16:59 notifier.go:343: [DEBUG] PushEvent: controller_started pushed OK (HTTP 200)
2026/10/25 17:16:59 notifier.go:355: [INFO] Event pushed: controller_started (info) [hu-only] — Controller elindult (0.343.0)
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈16857MB avail≈9041MB
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=10722164 → total=25898MB avail=9041MB used=16857MB (65.1%)
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=20.8GB avail=44.5GB (31.8%)
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=225.2GB avail=665.0GB (25.3%)
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.89 1.20 1.45 12/1545 459" → 1m=1.89 5m=1.20 15m=1.45
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 61.6°C (hwmon2)
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] GetInfo done in 112ms — mem=16857MB/25898MB (65.1%), rootDisk=20.8GB/68.7GB (31.8%), load=1.89/1.20/1.45, temp=61.6°C (hwmon2), cpu=61.9%
2026/10/25 17:16:59 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/hdd_1" total=937.8 GB used=225.2 GB avail=665.0GB (25.3%)
2026/10/25 17:16:59 builder.go:128: [DEBUG] [report] BuildReport: system info collected — cpu=61.9%, mem=16857/25898MB, temp=61.6C
2026/10/25 17:16:59 builder.go:130: [DEBUG] [report] BuildReport: storage entries=2
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈16793MB avail≈9105MB
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=10797116 → total=25898MB avail=9105MB used=16793MB (64.8%)
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=20.8GB avail=44.5GB (31.8%)
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=225.2GB avail=665.0GB (25.3%)
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.89 1.20 1.45 11/1541 475" → 1m=1.89 5m=1.20 15m=1.45
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 62.5°C (hwmon2)
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] GetInfo done in 61ms — mem=16793MB/25898MB (64.8%), rootDisk=20.8GB/68.7GB (31.8%), load=1.89/1.20/1.45, temp=62.5°C (hwmon2), cpu=61.9%
2026/10/25 17:17:00 healthcheck.go:128: [DEBUG] [monitor] Raw values: disk=31.8%, hdd=25.3% (configured=true), mem=64.8% (16793MB/25898MB), cpu=61.9%, temp=62.5°C (hwmon2)
2026/10/25 17:17:00 healthcheck.go:409: [DEBUG] [monitor] SSD disk: OK (32%)
2026/10/25 17:17:00 healthcheck.go:445: [DEBUG] [monitor] Memory: OK (65%)
2026/10/25 17:17:00 healthcheck.go:464: [DEBUG] [monitor] CPU: OK (62%)
2026/10/25 17:17:00 healthcheck.go:483: [DEBUG] [monitor] Temperature: OK (62°C)
2026/10/25 17:17:00 healthcheck.go:145: [DEBUG] [monitor] Docker daemon: OK
2026/10/25 17:17:00 healthcheck.go:153: [DEBUG] [monitor] Checking 4 protected containers: [traefik cloudflared felhom-controller filebrowser]
2026/10/25 17:17:00 healthcheck.go:161: [DEBUG] [monitor] All protected containers running
2026/10/25 17:17:00 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/hdd_1" total=937.8 GB used=225.2 GB avail=665.0GB (25.3%)
2026/10/25 17:17:00 healthcheck.go:176: [INFO] [monitor] Health check: status=ok
2026/10/25 17:17:00 healthcheck.go:180: [DEBUG] [monitor] Final status: ok (issues=0, warnings=0, info=5)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: starting log scan for 21 containers (since 15m0s)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned adventurelog: errors=0 warnings=0 issues=0 (took 35ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned adventurelog-frontend: errors=0 warnings=0 issues=0 (took 33ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned adventurelog-postgres: errors=0 warnings=0 issues=0 (took 31ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned bentopdf: errors=0 warnings=0 issues=0 (took 34ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned bookstack: errors=0 warnings=0 issues=0 (took 29ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned bookstack-db: errors=0 warnings=0 issues=0 (took 28ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned calibre-web: errors=0 warnings=0 issues=0 (took 31ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned docmost: errors=0 warnings=0 issues=0 (took 35ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned docmost-postgres: errors=0 warnings=0 issues=0 (took 32ms)
2026/10/25 17:17:00 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned docmost-redis: errors=0 warnings=0 issues=0 (took 41ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned home-assistant: errors=0 warnings=3 issues=1 (took 37ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned opengist: errors=0 warnings=0 issues=0 (took 31ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned paperless-webserver: errors=0 warnings=0 issues=0 (took 33ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned paperless-postgres: errors=0 warnings=0 issues=0 (took 28ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned paperless-redis: errors=0 warnings=0 issues=0 (took 37ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned plex: errors=0 warnings=0 issues=0 (took 26ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned privatebin: errors=0 warnings=0 issues=0 (took 34ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned romm: errors=0 warnings=0 issues=0 (took 35ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned romm-redis: errors=0 warnings=0 issues=0 (took 34ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned romm-db: errors=0 warnings=0 issues=0 (took 30ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned felhom-controller: errors=0 warnings=3 issues=3 (took 42ms)
2026/10/25 17:17:01 logscanner.go:69: [DEBUG] [metrics] logscanner: log scan completed: 21 containers in 694ms
2026/10/25 17:17:01 builder.go:204: [DEBUG] [report] BuildReport: complete — containers=20, health=ok, deployed=12, available=53, app_telemetry=12
2026/10/25 17:17:01 pusher.go:106: [DEBUG] [report] Push: url=https://hub.felhom.eu/api/v1/report payload=9977 bytes
2026/10/25 17:17:01 pusher.go:142: [INFO] [report] Hub report pushed successfully (9977 bytes)
2026/10/25 17:17:01 settings.go:970: [DEBUG] [settings] saved to /opt/docker/felhom-controller/data/settings.json (23056 bytes)
2026/10/25 17:17:01 settings.go:973: [INFO] [settings] Settings saved
2026/10/25 17:17:01 updater.go:104: [DEBUG] [selfupdate] SetFloor: floor "" → "0.343.0"
2026/10/25 17:17:01 updater.go:104: [DEBUG] [selfupdate] maybeAutoUpdate: current 0.343.0 >= floor 0.343.0 — no action
2026/10/25 17:17:01 main.go:1924: [INFO] Startup hub report sent
2026/10/25 17:17:04 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:17:04 offsiteapply.go:148: [INFO] [offsite-apply] settle-gate: GO — at/above floor 0.343.0 (we are 0.343.0), no managed update running
2026/10/25 17:17:04 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting
2026/10/25 17:17:04 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 11 targets (0 skipped not due, 0 skipped no container)
2026/10/25 17:17:04 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe romm: HTTP GET :8080/ → 200 (2ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe calibre-web: HTTP GET :8083/ → 302 (8ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe paperless-ngx: HTTP GET :8000/ → 302 (8ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe plex: API GET :32400/identity → 200 (8ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe opengist: API GET :6157/healthcheck → 200 (8ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe adventurelog: API GET :8000/api/ → 200 (9ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe privatebin: HTTP GET :8080/ → 200 (10ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe docmost: HTTP GET :3000/ → 200 (10ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe home-assistant: API GET :8123/api/ → 401 (10ms)
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe bentopdf: HTTP GET :8080/ → 200 (10ms)
2026/10/25 17:17:04 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:17:04 healthprobe.go:187: [DEBUG] Health probe bookstack: HTTP GET :80/ → 302 (44ms)
2026/10/25 17:17:04 healthprobe.go:167: [INFO] Health probes: 11 ok (of 11 probed)
2026/10/25 17:17:09 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:17:09 main.go:2844: [INFO] [bootrecon] boot window: fleet settled after 10s (3 identical samples 5s apart) — sweeping
2026/10/25 17:17:09 bootrecon.go:236: [INFO] [bootrecon] "kimai" is a boot orphan by intent but is HELD (held after a failed restore whose rollback also failed (2026-10-14T09:06:31Z) — clear the hold to start it) — NOT starting it; whatever is holding it owns its recovery
2026/10/25 17:17:09 bootrecon.go:251: [INFO] [bootrecon] Boot reconciliation: nothing to start — 1 app(s) held (absent drive / quiesce / app-data operation): [kimai]
2026/10/25 17:17:09 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting
2026/10/25 17:17:14 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bentopdf — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bookstack — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping docmost — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping adventurelog — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping home-assistant — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:14 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (11 skipped not due, 0 skipped no container)
2026/10/25 17:17:14 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:17:14 setup_gate.go:516: [DEBUG] [stacks] romm: setup probe SYSTEM.SHOW_SETUP_WIZARD = "true" (done when "false")
2026/10/25 17:17:24 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting
2026/10/25 17:17:24 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting
2026/10/25 17:17:24 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting
2026/10/25 17:17:24 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting
2026/10/25 17:17:24 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting
2026/10/25 17:17:24 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting
2026/10/25 17:17:24 crashboot.go:147: [INFO] [deadapp] boot grace 1m30s: the host's last unclean boot (2026-10-23T15:32:02Z) is not the one this start followed (R-856)
2026/10/25 17:17:24 aftercrash.go:106: [INFO] [aftercrash] a normal (or unknown) boot — no app repair after an unclean boot (R-1007)
2026/10/25 17:17:24 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping home-assistant — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bentopdf — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bookstack — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping docmost — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping adventurelog — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:24 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (11 skipped not due, 0 skipped no container)
2026/10/25 17:17:26 updater.go:104: [DEBUG] [selfupdate] queryRegistry: anonymous mode, url=https://gitea.dooplex.hu/v2/admin/felhom-controller/tags/list
2026/10/25 17:17:26 updater.go:104: [DEBUG] [selfupdate] queryRegistry: anonymous plain GET HTTP 401
2026/10/25 17:17:26 updater.go:104: [DEBUG] [selfupdate] queryRegistry: anonymous token request: https://gitea.dooplex.hu/v2/token?scope=repository%3Aadmin%2Ffelhom-controller%3Apull&service=container_registry
2026/10/25 17:17:26 updater.go:104: [DEBUG] [selfupdate] queryRegistry: anonymous tags/list with Bearer HTTP 200
2026/10/25 17:17:26 updater.go:104: [DEBUG] [selfupdate] queryRegistry: 15 tags returned: [0.331.0 0.341.0 0.341.0-rc.1 0.342.0 0.342.0-rc.1 0.342.0-rc.2 0.343.0 0.343.0-rc.1 0.343.0-rc.2 0.343.0-rc.3 0.343.0-rc.4 0.343.0-rc.5 0.343.0-rc.6 0.343.0-rc.7 latest]
2026/10/25 17:17:26 updater.go:196: [INFO] [selfupdate] Current version 0.343.0 is up to date
2026/10/25 17:17:26 updater.go:104: [DEBUG] [selfupdate] version comparison: current=0.343.0 (0.343.0), latest=0.343.0 (0.343.0), cmp=0, updateAvailable=false
2026/10/25 17:17:26 client.go:1276: [DEBUG] [agentapi] GET /disks -> 200 (796ms)
2026/10/25 17:17:27 client.go:1276: [DEBUG] [agentapi] GET /disks -> 200 (806ms)
2026/10/25 17:17:34 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping docmost — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping adventurelog — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping home-assistant — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bentopdf — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bookstack — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:34 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (11 skipped not due, 0 skipped no container)
2026/10/25 17:17:34 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting
2026/10/25 17:17:34 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:17:34 setup_gate.go:516: [DEBUG] [stacks] romm: setup probe SYSTEM.SHOW_SETUP_WIZARD = "true" (done when "false")
2026/10/25 17:17:39 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting
2026/10/25 17:17:44 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting
2026/10/25 17:17:44 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping adventurelog — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping home-assistant — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bentopdf — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bookstack — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping docmost — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:44 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (11 skipped not due, 0 skipped no container)
2026/10/25 17:17:44 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈16531MB avail≈9367MB
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=11108764 → total=25898MB avail=9367MB used=16531MB (63.8%)
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=20.8GB avail=44.5GB (31.8%)
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=225.2GB avail=665.0GB (25.3%)
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.24 1.12 1.41 7/1529 938" → 1m=1.24 5m=1.12 15m=1.41
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 50.0°C (hwmon0)
2026/10/25 17:17:54 info.go:13: [DEBUG] [system] GetInfo done in 72ms — mem=16531MB/25898MB (63.8%), rootDisk=20.8GB/68.7GB (31.8%), load=1.24/1.12/1.41, temp=50.0°C (hwmon0), cpu=7.6%
2026/10/25 17:17:54 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting
2026/10/25 17:17:54 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting
2026/10/25 17:17:54 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping home-assistant — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bentopdf — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bookstack — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping docmost — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping adventurelog — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/25 17:17:54 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (11 skipped not due, 0 skipped no container)
2026/10/25 17:17:54 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting
2026/10/25 17:17:54 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting
2026/10/25 17:17:54 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting
2026/10/25 17:17:54 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:17:54 setup_gate.go:516: [DEBUG] [stacks] romm: setup probe SYSTEM.SHOW_SETUP_WIZARD = "true" (done when "false")
2026/10/25 17:17:54 scheduler.go:450: [INFO] [scheduler] Running job: agent-channel-health
2026/10/25 17:17:54 scheduler.go:85: [DEBUG] [scheduler] job agent-channel-health: execution starting
2026/10/25 17:17:55 client.go:1276: [DEBUG] [agentapi] GET /storage -> 200 (705ms)
2026/10/25 17:17:55 scheduler.go:467: [INFO] [scheduler] Job agent-channel-health completed (took 705ms)
2026/10/25 17:17:56 client.go:1276: [DEBUG] [agentapi] GET /disks -> 200 (794ms)
2026/10/25 17:17:57 client.go:1276: [DEBUG] [agentapi] GET /disks -> 200 (798ms)
2026/10/25 17:18:04 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting
2026/10/25 17:18:04 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting
2026/10/25 17:18:04 manager.go:839: [INFO] [stacks] Status refresh: 23 containers across 69 stacks
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping docmost — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping adventurelog — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping home-assistant — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bentopdf — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping bookstack — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/25 17:18:04 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (11 skipped not due, 0 skipped no container)
@@ -0,0 +1,33 @@
control NEGATIVE (a stub answering 200 over plain http): convicted, as it must (200 -)
== tester-1 (enkicsifelhom.hu): 6 public name(s)
OK control POSITIVE http://felhom.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkicsifelhom.hu/r1033-check?q=1
OK http://files.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://files.enkicsifelhom.hu/r1033-check?q=1
OK http://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1
OK http://paperless.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paperless.enkicsifelhom.hu/r1033-check?q=1
OK http://paste.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paste.enkicsifelhom.hu/r1033-check?q=1
OK http://wiki.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkicsifelhom.hu/r1033-check?q=1
== demo-hp (enkisfelhom.hu): 14 public name(s)
OK http://arcade.enkisfelhom.hu/r1033-check?q=1 -> 301 https://arcade.enkisfelhom.hu/r1033-check?q=1
OK http://books.enkisfelhom.hu/r1033-check?q=1 -> 301 https://books.enkisfelhom.hu/r1033-check?q=1
OK http://docs.enkisfelhom.hu/r1033-check?q=1 -> 301 https://docs.enkisfelhom.hu/r1033-check?q=1
OK http://felhom.enkisfelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkisfelhom.hu/r1033-check?q=1
OK http://files.enkisfelhom.hu/r1033-check?q=1 -> 301 https://files.enkisfelhom.hu/r1033-check?q=1
OK http://gist.enkisfelhom.hu/r1033-check?q=1 -> 301 https://gist.enkisfelhom.hu/r1033-check?q=1
OK http://ha.enkisfelhom.hu/r1033-check?q=1 -> 301 https://ha.enkisfelhom.hu/r1033-check?q=1
OK http://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1
OK http://papir.enkisfelhom.hu/r1033-check?q=1 -> 301 https://papir.enkisfelhom.hu/r1033-check?q=1
OK http://paste.enkisfelhom.hu/r1033-check?q=1 -> 301 https://paste.enkisfelhom.hu/r1033-check?q=1
OK http://pdf.enkisfelhom.hu/r1033-check?q=1 -> 301 https://pdf.enkisfelhom.hu/r1033-check?q=1
OK http://plex.enkisfelhom.hu/r1033-check?q=1 -> 301 https://plex.enkisfelhom.hu/r1033-check?q=1
OK http://travel.enkisfelhom.hu/r1033-check?q=1 -> 301 https://travel.enkisfelhom.hu/r1033-check?q=1
OK http://wiki.enkisfelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkisfelhom.hu/r1033-check?q=1
== demo-felhom (demo-felhom.eu): 8 public name(s)
OK http://felhom.demo-felhom.eu/r1033-check?q=1 -> 301 https://felhom.demo-felhom.eu/r1033-check?q=1
OK http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1
OK http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1
OK http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1
OK http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1
OK http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1
OK http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1
OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
== every public name redirects http -> https
@@ -0,0 +1,139 @@
2026/10/25 17:16:53 [INFO] local-api: channel up (agent 169.254.253.1:8443) — guest 9201, 3 mount(s) visible
2026/10/25 17:16:53 [INFO] local-api: mount mp9 → /etc/felhom-bootstrap (storage=/var/lib/felhom-agent/guests/9201/bootstrap, class=, backup=false)
2026/10/25 17:16:53 [INFO] local-api: mount mp8 → /mnt/felhom-drives (storage=/mnt/felhom-drives, class=, backup=false)
2026/10/25 17:16:53 [INFO] local-api: mount mp0 → /var/lib/felhom (storage=local-lvm, class=, backup=true)
2026/10/25 17:16:53 [INFO] felhom-controller 0.343.0 starting (customer: tester-1, domain: enkicsifelhom.hu)
2026/10/25 17:16:53 [INFO] [settings] Loaded settings from /opt/docker/felhom-controller/data/settings.json
2026/10/25 17:16:53 [INFO] Encryption key loaded from /opt/docker/felhom-controller/data/encryption.key
2026/10/25 17:16:53 [INFO] [stacks] Using compose command: docker compose
2026/10/25 17:16:53 [INFO] [stacks] ScanStacks complete: 68 stacks found (3 deployed, 65 available)
2026/10/25 17:16:53 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:16:53 [INFO] [stacks] InjectMissingFields: processed 3 stacks
2026/10/25 17:16:53 [INFO] [stacks] Encryption migration: no stacks needed migration
2026/10/25 17:16:53 [INFO] [quiesce] loop started (poll 5m0s, max-quiesce 30m0s)
2026/10/25 17:16:53 [INFO] [stacks] desired-state backfill: 0 app(s) recorded as running, 0 left unrecorded (state ambiguous — legacy boot behaviour retained)
2026/10/25 17:16:53 [INFO] [stacks] installed-images backfill: 0 app(s) recorded, 3 already had a record, 0 left unrecorded (could not be observed completely — unknown, which renders nothing)
2026/10/25 17:16:53 [INFO] [stacks] pin adoption: 0 pinned, 3 already pinned, 0 left unpinned (0 not completely observed, 0 running something the template no longer offers)
2026/10/25 17:16:53 [INFO] [stacks] applied-meta backfill (R-646): recorded 0 []; skipped 0 [] — not current with the catalog, their pinned version's .felhom.yml is no longer on the box
2026/10/25 17:16:53 [INFO] [sync] Starting catalog sync (repo: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git, interval: 15m0s)
2026/10/25 17:16:53 [INFO] [sync] Starting catalog sync
2026/10/25 17:16:53 [INFO] Metrics store opened at /opt/docker/felhom-controller/data/metrics.db
2026/10/25 17:16:53 [INFO] Metrics collector started (60s interval)
2026/10/25 17:16:53 [INFO] [sync] Pulling latest from https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git (branch: main)
2026/10/25 17:16:53 [INFO] [tier2] start-up: no leftover second-drive copy container
2026/10/25 17:16:53 [INFO] [backup] restore record wired: /opt/docker/felhom-controller/data/restore-status.json (interrupted at startup: false)
2026/10/25 17:16:53 [INFO] [backup] app-data run record wired: /opt/docker/felhom-controller/data/appdata-run.json (an earlier cut still shown: false)
2026/10/25 17:16:53 [INFO] Notifier enabled (hub: https://hub.felhom.eu)
2026/10/25 17:16:53 [INFO] [upwatch] the host's weekly line test: 94.3 Mbit/s (measured 2026-10-24T08:00:58Z)
2026/10/25 17:16:53 [INFO] [upwatch] upload and CPU watch started (line 94 Mbit/s, qBittorrent share 50 %)
2026/10/25 17:16:53 [INFO] [selfupdate] Post-update startup: update successful (0.342.0 → 0.343.0)
2026/10/25 17:16:53 [INFO] Self-update enabled (check every 6h, auto-update: false, auto-update time: 04:30)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: status-refresh (every 10s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: stack-scan (every 2m0s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: docker-socket-watch (every 15s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: docker-socket-users (every 5m0s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: health-probes (every 10s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: system-health (every 5m0s)
2026/10/25 17:16:53 [INFO] [offsite-apply] settle-gate: awaiting floor knowledge (first report ACK) before offsite apply
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: after-unclean-boot (every 30s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: deadapp-check (every 30s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: ring-spill (every 30s)
2026/10/25 17:16:53 [INFO] [scheduler] Daily job db-dump scheduled for 2026-10-26 02:30 CET
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: offsite-credential-retry (every 5m0s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: backup-cache (every 5m0s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: conversion-copy-release (every 1h0m0s)
2026/10/25 17:16:53 [INFO] [scheduler] Daily job tier2-backup scheduled for 2026-10-26 03:30 CET
2026/10/25 17:16:53 [INFO] [scheduler] Daily job offbox-backup scheduled for 2026-10-26 04:15 CET
2026/10/25 17:16:53 [INFO] [catch-up] controller start: no backup leg missed its last scheduled time — nothing to make up
2026/10/25 17:16:53 [INFO] [scheduler] Daily job offsite-abandon-sweep scheduled for 2026-10-26 05:10 CET
2026/10/25 17:16:53 [INFO] [scheduler] Daily job offsite-integrity scheduled for 2026-10-26 06:00 CET
2026/10/25 17:16:53 [INFO] [scheduler] Daily job offsite-proof scheduled for 2026-10-26 05:30 CET
2026/10/25 17:16:53 [INFO] [scheduler] Daily job metrics-prune scheduled for 2026-10-26 04:00 CET
2026/10/25 17:16:53 [INFO] [scheduler] Daily job fill-watch scheduled for 2026-10-26 03:30 CET
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: fill-watch-interval (every 10m0s)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: hub-report (every 15m0s)
2026/10/25 17:16:53 [INFO] Hub reporting enabled (every 15m0s to https://hub.felhom.eu)
2026/10/25 17:16:53 [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
2026/10/25 17:16:53 [INFO] ========== Startup Self-Test ==========
2026/10/25 17:16:54 [INFO] Event pushed: controller_updated (info) [hu-only] — Controller frissítve: 0.342.0 → 0.343.0
2026/10/25 17:16:54 [INFO] [PASS] Docker socket: reachable (v29.8.2)
2026/10/25 17:16:54 [INFO] [PASS] Stacks directory: /opt/docker/stacks
2026/10/25 17:16:54 [INFO] [PASS] Data directory: /opt/docker/felhom-controller/data (writable)
2026/10/25 17:16:54 [INFO] [PASS] System data path: /mnt/sys_drive
2026/10/25 17:16:54 [INFO] [PASS] Storage paths: 1 connected, 0 disconnected
2026/10/25 17:16:54 [INFO] [PASS] Git catalog: 65 app definitions found
2026/10/25 17:16:54 [INFO] [PASS] Hub connectivity: https://hub.felhom.eu reachable (HTTP 200)
2026/10/25 17:16:54 [INFO] [PASS] Metrics DB: 35.9 MB
2026/10/25 17:16:54 [INFO] ========================================
2026/10/25 17:16:54 [INFO] Self-test complete: 8 passed, 0 warnings, 0 failed
2026/10/25 17:16:54 [INFO] [scheduler] Starting scheduler with 23 jobs
2026/10/25 17:16:54 [INFO] [scheduler] Registered periodic job: geo-verify (every 6h0m0s)
2026/10/25 17:16:54 [INFO] Geo-restriction support enabled (CF API token configured)
2026/10/25 17:16:54 [INFO] [report] hub wait channel active (hold ≤240s)
2026/10/25 17:16:54 [INFO] [backup] Found 2 DB dump files across drives
2026/10/25 17:16:54 [INFO] [infra] connected felhom-controller to traefik-public
2026/10/25 17:16:54 [INFO] [web] sessions: restored 12 dashboard session(s) from disk (dropped 0 expired, 0 malformed)
2026/10/25 17:16:54 [INFO] [web] Auth: using password from settings.json
2026/10/25 17:16:54 [INFO] [scheduler] Registered periodic job: disk-health-check (every 1h0m0s)
2026/10/25 17:16:54 [INFO] [scheduler] Registered periodic job: agent-channel-health (every 1m0s)
2026/10/25 17:16:54 [INFO] Web UI listening on :8080
2026/10/25 17:16:54 [INFO] [web] FileBrowser: drive /mnt/felhom-drives/media_a not mounted — skipping userdata skeleton
2026/10/25 17:16:54 [INFO] [web] FileBrowser: drive /mnt/felhom-drives/media_b not mounted — skipping userdata skeleton
2026/10/25 17:16:54 [INFO] [web] FileBrowser: drive /mnt/felhom-drives/media_c not mounted — skipping userdata skeleton
2026/10/25 17:16:54 [INFO] [web] FileBrowser: drive /mnt/felhom-drives/media_d not mounted — skipping userdata skeleton
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:54 [INFO] [backup] Discovered 2 databases
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/10/25 17:16:54 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/10/25 17:16:54 [INFO] [backup] Discovered app data: 3 apps
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/bookstack — 5 env vars, 0 encrypted, 3 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for bookstack
2026/10/25 17:16:54 [INFO] [stacks] bookstack: install id 27a3384799fbbc3f recorded (an app installed before D47)
2026/10/25 17:16:54 [INFO] [backup] Recovery unit captured for bookstack → /mnt/sys_drive/felhom-data/backups/primary/bookstack (images=2, secrets-referenced=3, data_keys=1, portable-carried=2/2, withheld=1)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/paperless-ngx — 8 env vars, 0 encrypted, 3 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for paperless-ngx
2026/10/25 17:16:54 [INFO] [stacks] paperless-ngx: install id 694723ce3390099b recorded (an app installed before D47)
2026/10/25 17:16:54 [INFO] [backup] Recovery unit captured for paperless-ngx → /mnt/felhom-drives/adatlemez/backups/primary/paperless-ngx (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1)
2026/10/25 17:16:54 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/privatebin — 2 env vars, 0 encrypted, 0 sensitive fields
2026/10/25 17:16:54 [INFO] [stacks] SaveAppConfig: saved config for privatebin
2026/10/25 17:16:54 [INFO] [stacks] privatebin: install id 6b00861f43793b20 recorded (an app installed before D47)
2026/10/25 17:16:54 [INFO] [backup] Recovery unit captured for privatebin → /mnt/sys_drive/felhom-data/backups/primary/privatebin (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/10/25 17:16:54 [INFO] [backup] Backup status cache refreshed
2026/10/25 17:16:54 [INFO] [sync] Catalog sync complete
2026/10/25 17:16:54 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
2026/10/25 17:16:54 [INFO] [web] FileBrowser sync — no config/compose change, ensured running without recreate (5 storage path(s))
2026/10/25 17:16:54 [INFO] [monitor] Health check: status=ok
2026/10/25 17:16:54 [INFO] [infra] connected felhom-controller to felhom-filebrowser (the FileBrowser default-login check, R-1126)
2026/10/25 17:16:58 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:16:59 [INFO] [report] Building system report
2026/10/25 17:16:59 [INFO] Event pushed: controller_started (info) [hu-only] — Controller elindult (0.343.0)
2026/10/25 17:17:00 [INFO] [monitor] Health check: status=ok
2026/10/25 17:17:00 [INFO] [report] Hub report pushed successfully (5247 bytes)
2026/10/25 17:17:00 [INFO] [settings] Settings saved
2026/10/25 17:17:00 [INFO] Startup hub report sent
2026/10/25 17:17:03 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:17:03 [INFO] [offsite-apply] settle-gate: GO — at/above floor 0.343.0 (we are 0.343.0), no managed update running
2026/10/25 17:17:04 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:17:04 [INFO] Health probes: 3 ok (of 3 probed)
2026/10/25 17:17:08 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:17:08 [INFO] [bootrecon] boot window: fleet settled after 10s (3 identical samples 5s apart) — sweeping
2026/10/25 17:17:08 [INFO] [bootrecon] Boot reconciliation: no boot-orphaned apps (nothing to start)
2026/10/25 17:17:14 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:17:24 [INFO] [deadapp] boot grace 1m30s: the host's last unclean boot (2026-10-24T17:44:02Z) is not the one this start followed (R-856)
2026/10/25 17:17:24 [INFO] [aftercrash] a normal (or unknown) boot — no app repair after an unclean boot (R-1007)
2026/10/25 17:17:24 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:17:25 [INFO] [selfupdate] Current version 0.343.0 is up to date
2026/10/25 17:17:34 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:17:44 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:17:54 [INFO] [stacks] Status refresh: 9 containers across 68 stacks
2026/10/25 17:17:54 [INFO] [scheduler] Running job: agent-channel-health
2026/10/25 17:17:54 [INFO] [scheduler] Job agent-channel-health completed (took 551ms)
@@ -0,0 +1,159 @@
[INFO] felhom-host-install v1.36.0 — mode=byo customer=Tester4 vmid=9201
[STEP] 1/8 pre-flight
[INFO] pve-manager/9.2.2/b9984c6d90a4bd80 (running kernel: 7.0.2-6-pve)
[INFO] node: byo-test (explicit)
[INFO] agent config: /etc/felhom-agent/agent.json
[INFO] agent: not installed yet — will be fetched + installed in step 5/8
[INFO] local-lvm free: ~29 GiB
[WARN] local-lvm free ~29 GiB is below the recommended 120 GiB — the install continues; proceed only if you sized the grows deliberately
[INFO] free RAM: ~6285 MiB
[INFO] existing guests on this host: 0 (pct+qm)
[INFO] archive storage 'local' present
[INFO] acl storages all present or pre-positioned: local local-lvm
[INFO] dnsmasq: not present before Felhom — recorded; uninstall will remove it again if we install it
[INFO] host DNS (:53): free
[INFO] hub reachable (https://hub.felhom.eu)
[OK] customer 'Tester4' exists + passphrase valid
[INFO] golden: none local — will fetch + verify from Gitea in step 7/8
[STEP] BYO disclosure — installing Felhom on a host the operator does NOT own will create:
pveum: roles FelhomAgentBase/FelhomAgentGuest/FelhomAgentStore + user felhom-agent@pve + token felhom-agent@pve!agent
+ scoped ACL grants (Base@/, Guest@/pool/felhom, Store@[local local-lvm]) + the 'felhom' pool
agent: service user felhom-agent + /usr/local/bin/felhom-agent (+ .bak-* backups; A/B slots via the guarded self-update)
+ /etc/systemd/system/felhom-agent.service (+ felhom-agent-rollback.service + start-limit drop-in)
+ /etc/sudoers.d/felhom-agent + state dir /var/lib/felhom-agent + config /etc/felhom-agent/agent.json (0600)
host: /usr/local/sbin/felhom-mkfs-guarded + /usr/local/sbin/felhom-selfupdate-guarded
+ /usr/local/sbin/felhom-pbs-apply (PBS-DR apply wrapper — DR capability is baked
on every install; ACTIVATION stays a hub flag, off = zero effect on this host)
+ felhom-mgmt-watchdog service+timer+script + /etc/tmpfiles.d/felhom-privsep.conf
+ /usr/local/sbin/felhom-priv-apply + the guest-hook snippet under /var/lib/vz/snippets/
(both from the agent's config bundle)
+ the 'sudo' and 'age' packages if absent + install state dir /var/lib/felhom-install
wg: an OUTBOUND WireGuard tunnel to the Felhom hub (wg_tunnel.enabled=true — base
infrastructure like the cloudflared tunnel; hands-free peer registration; the
operator can disable it per-customer hub-side on request)
guest: the provisioned Felhom LXC (vmid 9201, capped 2 cores / 4096 MiB) + its volumes
+ the golden vzdump imported onto storage 'local'
reuse: a golden already on 'local' is used only if its sha256 or its controller version
matches the hub's vouched golden; otherwise the vouched one is fetched (a --golden you name is refused)
update: operator-signed self-update authority: felhom-op-1
NOT touched in byo mode: root@pam (no break-glass), host DNS (:53), OOB sshd.
Type this host's short hostname (byo-test) to acknowledge the byo install: [OK] byo install acknowledged for host byo-test
[OK] pre-flight passed
[STEP] 2/8 Proxmox API token
[OK] token minted (secret captured, not logged)
[OK] scoped ACL applied (Base@/, Guest@/pool/felhom + /vms/990000..990009, Store@[local local-lvm])
[SKIP] old broad role FelhomAgent already absent
[STEP] 3/8 compute volume grows
[INFO] auto-computed from a 29 GiB thin pool (16 §7: 50/100/130 GiB)
[INFO] grows: rootfs +0G (->32G), data +26G (->50G, ONE volume)
[STEP] 4/8 host enrollment (POST /host-enroll)
[OK] host MINTED (first enroll)
[INFO] host_id: Tester4-f967d4 (api_key captured, not logged)
[SKIP] break-glass (step 4b) SKIPPED (byo: the host owner manages break-glass; root@pam untouched, nothing vaulted)
[STEP] 5/8 agent install (fetch + verify + install)
[SKIP] apt repo alignment skipped (byo — the owner manages repos)
[WARN] no git credential in controller.yaml — fetching artifacts ANONYMOUSLY (they are world-readable; sha256 verification unchanged)
[INFO] manifest: agent v0.164.0 (sha 1b803d8b70f8afbe…), golden v0.331.0
[INFO] fetching agent binary v0.164.0 from Gitea …
[OK] verified sha256 1b803d8b70f8afbe… matches the hub manifest
[OK] installed /usr/local/bin/felhom-agent (felhom-agent 0.164.0)
[INFO] installing the 'sudo' package (required for the non-root agent model) …
[OK] sudo installed (Sudo version 1.9.16p2)
[INFO] installing the 'age' package (escrow ceremony identity-wrap dependency) …
[OK] age installed (1.2.1)
[OK] created service user felhom-agent
[OK] added felhom-agent to systemd-journal (unprivileged journal read for NAS verify)
[OK] verified sha256 1d7b54d39d0c1aff… matches the hub manifest
os-apply: BUNDLE START agent=0.164.0 sha=1d7b54d39d0c1aff authority=installer files=32 write=26 same=2 kept=0 skipped=4
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mkfs-guarded (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-selfupdate-guarded (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-pbs-apply (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-backup-target-apply (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-vzdump-partial-clean (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-thin-check (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-internal-place (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-crash-guard (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-priv-apply (new)
os-apply: BUNDLE WROTE /var/lib/vz/snippets/felhom-guest-hook.sh (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-shared-parent.sh (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-shared-parent.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.timer (new)
os-apply: BUNDLE WROTE /etc/felhom/crash-guard.conf (new)
os-apply: BUNDLE WROTE /etc/grub.d/01_felhom_oneshot (new)
os-apply: BUNDLE WROTE /etc/grub.d/42_felhom_oneshot (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent-rollback.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service.d/felhom-agent-limits.conf (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mgmt-watchdog (new)
os-apply: BUNDLE WROTE /etc/tmpfiles.d/felhom-privsep.conf (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.timer (new)
os-apply: BUNDLE WROTE /etc/sudoers.d/felhom-agent (new)
os-apply: BUNDLE DONE agent=0.164.0 written=26 same=2 self-check=ok signers-created=False
[OK] installed every root-owned file from config bundle v0.164.0 (sha 1d7b54d39d0c1aff…; checked, previous copies kept)
[SKIP] OOB (felhom-sshd) off (byo, or appliance --no-oob) — the bundle skipped its files
[STEP] 5b/8 internal data place (Belső tárhely)
[SKIP] byo: the host owner's thin pool is not ours to divide — no internal place
[STEP] 6/8 agent config + service
[WARN] backup target: DEGRADED — no eligible second drive, so the whole-system backup stays on the SYSTEM drive.
[WARN] It protects against file corruption but NOT against a disk failure. Attach a second drive and assign it in the dashboard.
[INFO] node=byo-test local_api=192.168.0.151:8443 tls_fp=7E:77:41:86:26:92…
[OK] wrote /etc/felhom-agent/agent.json (0600 felhom-agent)
[OK] byo config asserts passed (lan_resolver.enable / oob.enabled off; deployment_mode=byo; wg_tunnel intentionally ON — base infrastructure)
[OK] agent --selftest (read-only) passed
[OK] felhom-agent service active (non-root felhom-agent reads the config OK)
[OK] wrote /etc/felhom/os-trust.json + operator-signers (felhom-op-1) — the root-owned slow-lane anchors
[STEP] 7/8 golden archive
[WARN] no git credential in controller.yaml — fetching artifacts ANONYMOUSLY (they are world-readable; sha256 verification unchanged)
[INFO] fetching golden v0.331.0 from Gitea → /var/lib/vz/dump/vzdump-lxc-9100-2026_10_25-17_52_14.tar.zst
[OK] verified sha256 3881de62b0986ccb… matches the hub manifest
[OK] golden imported + verified: local:backup/vzdump-lxc-9100-2026_10_25-17_52_14.tar.zst
[STEP] 8/8 provision guest 9201
[SKIP] pool felhom already exists
=== felhom-agent 0.164.0 selftest=provision (vmid=9201 customer=Tester4 hostname=tester4) ===
--- front half: bring-up (provision) local:backup/vzdump-lxc-9100-2026_10_25-17_52_14.tar.zst → vmid 9201 ---
time=2026-10-25T17:53:51.729+01:00 level=INFO msg="bring-up: pool membership re-asserted" vmid=9201 pool=felhom
[OK] front half: vmid 9201 up (boot+running) in 1m26s; MAC=BC:24:11:4B:04:4E
--- back half: mint per-guest token + populate bootstrap config mount ---
time=2026-10-25T17:53:56.751+01:00 level=INFO msg="provision: back-half complete" vmid=9201 mount=mp9 guest_path=/etc/felhom-bootstrap endpoint=192.168.0.151:8443
[OK] back half: bootstrap mount mp9 → /etc/felhom-bootstrap on vmid 9201 (host dir /var/lib/felhom-agent/guests/9201/bootstrap)
local-api endpoint 192.168.0.151:8443 · leaf fp f09a20d4c01686516db562b6acc9aa7e95581d69c2318fccf7894ed772ef4a8e · token: minted (not printed)
=== selftest=provision OK — guest 9201 provisioned + bootstrap-mounted (KEPT) ===
next: reboot the guest → the golden's baked controller-bootstrap unit deploys the controller,
which PULLS its controller.yaml from the hub (retrieval passphrase) and merges in this local_api.
[OK] provision completed
[INFO] rebooting guest 9201 so the baked controller-bootstrap unit picks up the mount
[STEP] verify
[OK] pct status: running
[OK] onboot: 1
mp0: local-lvm:vm-9201-disk-1,mp=/var/lib/felhom,backup=1,size=50G
mp8: /mnt/felhom-drives,mp=/mnt/felhom-drives
rootfs: local-lvm:vm-9201-disk-0,size=32G
[OK] pool: guest 9201 is a member of felhom
[OK] acl: FelhomAgentBase@/ present (user+token)
[OK] acl: FelhomAgentGuest@/pool/felhom present (user+token)
[OK] acl: FelhomAgentStore@/storage/local present (user+token)
[OK] acl: FelhomAgentStore@/storage/local-lvm present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990000 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990001 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990002 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990003 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990004 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990005 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990006 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990007 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990008 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990009 present (user+token)
[OK] authz signers: 2 (operator-signed self-update armed)
[OK] controller: Up 5 seconds (healthy) (after ~0s)
[INFO] controller image: gitea.dooplex.hu/admin/felhom-controller:0.342.0
[WARN] cloudflared not visible yet
[INFO] (confirm in the hub UI that host Tester4-f967d4 reports guest 9201)
[OK] Day-0 provision SUCCESS — vmid=9201 host_id=Tester4-f967d4 customer=Tester4 golden=local:backup/vzdump-lxc-9100-2026_10_25-17_52_14.tar.zst
@@ -0,0 +1,73 @@
# BYO test host byo-test (VM 9371 on demo-hp), host_id Tester4-f967d4, 2026-10-25
## planted before the install: our 0.164.0 drop-in, sha256 f0485da5… (copied from Tester 1)
felhom-agent 0.165.0
root 644 /var/lib/felhom-install/state.json
install mode: byo
## sshd_config.d now:
total 8
drwxr-xr-x 2 root root 4096 Oct 25 18:22 .
drwxr-xr-x 4 root root 4096 Oct 25 17:37 ..
permitrootlogin yes
passwordauthentication yes
## journal (os-apply + agent, 17:40–18:25)
Oct 25 17:52:07 byo-test felhom-os-apply[4082]: os-apply: BUNDLE START agent=0.164.0 sha=1d7b54d39d0c1aff authority=installer files=32 write=26 same=2 kept=0 skipped=4
Oct 25 17:52:07 byo-test felhom-os-apply[4083]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mkfs-guarded (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4084]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-selfupdate-guarded (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4085]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-pbs-apply (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4086]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-backup-target-apply (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4087]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-vzdump-partial-clean (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4088]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-thin-check (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4089]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-internal-place (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4090]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-crash-guard (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4091]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-priv-apply (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4092]: os-apply: BUNDLE WROTE /var/lib/vz/snippets/felhom-guest-hook.sh (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4093]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-shared-parent.sh (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4094]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-shared-parent.service (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4095]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard.service (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4096]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.service (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4097]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.timer (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4098]: os-apply: BUNDLE WROTE /etc/felhom/crash-guard.conf (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4099]: os-apply: BUNDLE WROTE /etc/grub.d/01_felhom_oneshot (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4100]: os-apply: BUNDLE WROTE /etc/grub.d/42_felhom_oneshot (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4101]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4102]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent-rollback.service (new)
Oct 25 17:52:07 byo-test felhom-os-apply[4103]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service.d/felhom-agent-limits.conf (new)
Oct 25 17:52:08 byo-test felhom-os-apply[4104]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mgmt-watchdog (new)
Oct 25 17:52:08 byo-test felhom-os-apply[4105]: os-apply: BUNDLE WROTE /etc/tmpfiles.d/felhom-privsep.conf (new)
Oct 25 17:52:08 byo-test felhom-os-apply[4106]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.service (new)
Oct 25 17:52:08 byo-test felhom-os-apply[4107]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.timer (new)
Oct 25 17:52:08 byo-test felhom-os-apply[4108]: os-apply: BUNDLE WROTE /etc/sudoers.d/felhom-agent (new)
Oct 25 17:52:09 byo-test felhom-os-apply[4235]: os-apply: BUNDLE DONE agent=0.164.0 written=26 same=2 self-check=ok signers-created=False
Oct 25 17:52:11 byo-test felhom-os-apply[4417]: os-apply: REFUSED: R12 this box was installed as 'byo', not appliance — its host belongs to its owner
Oct 25 17:52:11 byo-test felhom-agent[4389]: time=2026-10-25T17:52:11.013+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: REFUSED: R12 this box was installed as 'byo', not appliance — its host belongs to its owner"
Oct 25 17:52:12 byo-test felhom-agent[4389]: time=2026-10-25T17:52:12.970+01:00 level=INFO msg="capabilities self-check" ok=70 total=76 degraded=1 inactive=5
Oct 25 18:02:41 byo-test felhom-agent[4389]: time=2026-10-25T18:02:41.293+01:00 level=INFO msg="osupdate: docker step skipped — ring 1 takes an engine set only inside a signed operator job (`11` §5.8)" run=20261025T170128Z vmid=9201 trigger=night ring=1 enabled=true
Oct 25 18:02:41 byo-test felhom-agent[4389]: time=2026-10-25T18:02:41.293+01:00 level=INFO msg="osupdate: pve step skipped — ring 1 takes a Proxmox set only inside a signed operator job (`11` §5.10)" run=20261025T170128Z vmid=9201 trigger=night ring=1 enabled=true
Oct 25 18:07:19 byo-test felhom-agent[4389]: time=2026-10-25T18:07:19.468+01:00 level=INFO msg="desired: hub reports pending signed ops (fetch/execute is slice 10B)"
Oct 25 18:07:19 byo-test felhom-agent[4389]: time=2026-10-25T18:07:19.475+01:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=1c622afc4aec7208 op=agent_update key_id=felhom-op-1 nonce=8abd1cd5faf79f4ce21dfc5b55d49d84
Oct 25 18:07:20 byo-test felhom-os-apply[27473]: os-apply: AGENT-UPDATE signed by the operator: version=0.165.0 sha=e9d04a47ee85222c — handing the root copy to the A/B wrapper
Oct 25 18:07:20 byo-test felhom-agent[4389]: time=2026-10-25T18:07:20.446+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=1c622afc4aec7208 op=agent_update
Oct 25 18:07:20 byo-test felhom-agent[4389]: time=2026-10-25T18:07:20.497+01:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=d885aae390b02162 op=agent_config_update key_id=felhom-op-1 nonce=2cfa2580f3917ea3ed018d0c2233f81f
Oct 25 18:07:20 byo-test felhom-os-apply[27523]: os-apply: BUNDLE START agent=0.165.0 sha=8dea1f34d77ba40a authority=signed files=32 write=1 same=26 kept=1 skipped=4
Oct 25 18:07:20 byo-test felhom-os-apply[27527]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-os-apply (replaced)
Oct 25 18:07:21 byo-test felhom-os-apply[27603]: os-apply: BUNDLE DONE agent=0.165.0 written=1 same=26 self-check=ok signers-created=False
Oct 25 18:07:21 byo-test felhom-agent[4389]: time=2026-10-25T18:07:21.547+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE START agent=0.165.0 sha=8dea1f34d77ba40a authority=signed files=32 write=1 same=26 kept=1 skipped=4"
Oct 25 18:07:21 byo-test felhom-agent[4389]: time=2026-10-25T18:07:21.547+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE WROTE /usr/local/sbin/felhom-os-apply (replaced)"
Oct 25 18:07:21 byo-test felhom-agent[4389]: time=2026-10-25T18:07:21.547+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.165.0 written=1 same=26 self-check=ok signers-created=False"
Oct 25 18:07:22 byo-test felhom-agent[4389]: time=2026-10-25T18:07:22.499+01:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=71 total=76 degraded=pve:store-grant:felhom-pbs
Oct 25 18:07:22 byo-test felhom-agent[4389]: time=2026-10-25T18:07:22.500+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=d885aae390b02162 op=agent_config_update
Oct 25 18:07:23 byo-test felhom-os-apply[27793]: os-apply: REFUSED: R12 this box was installed as 'byo', not appliance — its host belongs to its owner
Oct 25 18:07:23 byo-test felhom-agent[27763]: time=2026-10-25T18:07:23.572+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: REFUSED: R12 this box was installed as 'byo', not appliance — its host belongs to its owner"
Oct 25 18:07:24 byo-test felhom-agent[27763]: time=2026-10-25T18:07:24.350+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
Oct 25 18:08:25 byo-test felhom-agent[27763]: time=2026-10-25T18:08:25.030+01:00 level=WARN msg="selfupdate: update committed" version=0.165.0 wrapper=""
Oct 25 18:22:34 byo-test felhom-agent[27763]: time=2026-10-25T18:22:34.999+01:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=cc915dd2020e42b0 op=agent_config_update key_id=felhom-op-1 nonce=8083346e293da88779c3c975d4e8abae
Oct 25 18:22:35 byo-test felhom-os-apply[39714]: os-apply: BUNDLE /etc/ssh/sshd_config.d/10-felhom-root-key-only.conf: a BYO host keeps its own settings (D43) — removing the copy we wrote
Oct 25 18:22:35 byo-test felhom-os-apply[39715]: os-apply: BUNDLE START agent=0.165.0 sha=8dea1f34d77ba40a authority=signed files=32 write=0 same=26 kept=1 skipped=4
Oct 25 18:22:35 byo-test felhom-os-apply[39716]: os-apply: BUNDLE REMOVED /etc/ssh/sshd_config.d/10-felhom-root-key-only.conf (ours, on a BYO host — D43)
Oct 25 18:22:36 byo-test felhom-os-apply[39816]: os-apply: BUNDLE DONE agent=0.165.0 written=0 same=26 self-check=ok signers-created=False
Oct 25 18:22:36 byo-test felhom-agent[27763]: time=2026-10-25T18:22:36.194+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE /etc/ssh/sshd_config.d/10-felhom-root-key-only.conf: a BYO host keeps its own settings (D43) — removing the copy we wrote"
Oct 25 18:22:36 byo-test felhom-agent[27763]: time=2026-10-25T18:22:36.194+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE START agent=0.165.0 sha=8dea1f34d77ba40a authority=signed files=32 write=0 same=26 kept=1 skipped=4"
Oct 25 18:22:36 byo-test felhom-agent[27763]: time=2026-10-25T18:22:36.194+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE REMOVED /etc/ssh/sshd_config.d/10-felhom-root-key-only.conf (ours, on a BYO host — D43)"
Oct 25 18:22:36 byo-test felhom-agent[27763]: time=2026-10-25T18:22:36.194+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.165.0 written=0 same=26 self-check=ok signers-created=False"
Oct 25 18:22:37 byo-test felhom-agent[27763]: time=2026-10-25T18:22:37.238+01:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=76 total=76 degraded=""
Oct 25 18:22:37 byo-test felhom-agent[27763]: time=2026-10-25T18:22:37.238+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=cc915dd2020e42b0 op=agent_config_update
@@ -0,0 +1,162 @@
[INFO] felhom-host-install v1.36.0 — mode=byo customer=Tester4 vmid=9201
[STEP] 1/8 pre-flight
[INFO] pve-manager/9.2.2/b9984c6d90a4bd80 (running kernel: 7.0.2-6-pve)
[INFO] node: byo-test (explicit)
[INFO] agent config: /etc/felhom-agent/agent.json
[INFO] agent: not installed yet — will be fetched + installed in step 5/8
[INFO] local-lvm free: ~29 GiB
[WARN] local-lvm free ~29 GiB is below the recommended 120 GiB — the install continues; proceed only if you sized the grows deliberately
[INFO] free RAM: ~6285 MiB
[INFO] existing guests on this host: 0 (pct+qm)
[INFO] archive storage 'local' present
[INFO] acl storages all present or pre-positioned: local local-lvm
[INFO] dnsmasq: not present before Felhom — recorded; uninstall will remove it again if we install it
[INFO] host DNS (:53): free
[INFO] hub reachable (https://hub.felhom.eu)
[OK] customer 'Tester4' exists + passphrase valid
[INFO] golden: none local — will fetch + verify from Gitea in step 7/8
byo-test
[STEP] BYO disclosure — installing Felhom on a host the operator does NOT own will create:
pveum: roles FelhomAgentBase/FelhomAgentGuest/FelhomAgentStore + user felhom-agent@pve + token felhom-agent@pve!agent
+ scoped ACL grants (Base@/, Guest@/pool/felhom, Store@[local local-lvm]) + the 'felhom' pool
agent: service user felhom-agent + /usr/local/bin/felhom-agent (+ .bak-* backups; A/B slots via the guarded self-update)
+ /etc/systemd/system/felhom-agent.service (+ felhom-agent-rollback.service + start-limit drop-in)
+ /etc/sudoers.d/felhom-agent + state dir /var/lib/felhom-agent + config /etc/felhom-agent/agent.json (0600)
host: /usr/local/sbin/felhom-mkfs-guarded + /usr/local/sbin/felhom-selfupdate-guarded
+ /usr/local/sbin/felhom-pbs-apply (PBS-DR apply wrapper — DR capability is baked
on every install; ACTIVATION stays a hub flag, off = zero effect on this host)
+ felhom-mgmt-watchdog service+timer+script + /etc/tmpfiles.d/felhom-privsep.conf
+ /usr/local/sbin/felhom-priv-apply + the guest-hook snippet under /var/lib/vz/snippets/
(both from the agent's config bundle)
+ the 'sudo' and 'age' packages if absent + install state dir /var/lib/felhom-install
wg: an OUTBOUND WireGuard tunnel to the Felhom hub (wg_tunnel.enabled=true — base
infrastructure like the cloudflared tunnel; hands-free peer registration; the
operator can disable it per-customer hub-side on request)
guest: the provisioned Felhom LXC (vmid 9201, capped 2 cores / 4096 MiB) + its volumes
+ the golden vzdump imported onto storage 'local'
reuse: a golden already on 'local' is used only if its sha256 or its controller version
matches the hub's vouched golden; otherwise the vouched one is fetched (a --golden you name is refused)
update: operator-signed self-update authority: felhom-op-1
NOT touched in byo mode: root@pam (no break-glass), host DNS (:53), OOB sshd.
Type this host's short hostname (byo-test) to acknowledge the byo install: [OK] byo install acknowledged for host byo-test
[OK] pre-flight passed
[STEP] 2/8 Proxmox API token
[OK] token minted (secret captured, not logged)
[OK] scoped ACL applied (Base@/, Guest@/pool/felhom + /vms/990000..990009, Store@[local local-lvm])
[SKIP] old broad role FelhomAgent already absent
[STEP] 3/8 compute volume grows
[INFO] auto-computed from a 29 GiB thin pool (16 §7: 50/100/130 GiB)
[INFO] grows: rootfs +0G (->32G), data +26G (->50G, ONE volume)
[STEP] 4/8 host enrollment (POST /host-enroll)
[OK] host MINTED (first enroll)
[INFO] host_id: Tester4-f967d4 (api_key captured, not logged)
[SKIP] break-glass (step 4b) SKIPPED (byo: the host owner manages break-glass; root@pam untouched, nothing vaulted)
[STEP] 5/8 agent install (fetch + verify + install)
[SKIP] apt repo alignment skipped (byo — the owner manages repos)
[WARN] no git credential in controller.yaml — fetching artifacts ANONYMOUSLY (they are world-readable; sha256 verification unchanged)
[INFO] manifest: agent v0.164.0 (sha 1b803d8b70f8afbe…), golden v0.331.0
[INFO] fetching agent binary v0.164.0 from Gitea …
[OK] verified sha256 1b803d8b70f8afbe… matches the hub manifest
[OK] installed /usr/local/bin/felhom-agent (felhom-agent 0.164.0)
[INFO] installing the 'sudo' package (required for the non-root agent model) …
[OK] sudo installed (Sudo version 1.9.16p2)
[INFO] installing the 'age' package (escrow ceremony identity-wrap dependency) …
[OK] age installed (1.2.1)
[OK] created service user felhom-agent
[OK] added felhom-agent to systemd-journal (unprivileged journal read for NAS verify)
[OK] verified sha256 1d7b54d39d0c1aff… matches the hub manifest
os-apply: BUNDLE START agent=0.164.0 sha=1d7b54d39d0c1aff authority=installer files=32 write=26 same=2 kept=0 skipped=4
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mkfs-guarded (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-selfupdate-guarded (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-pbs-apply (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-backup-target-apply (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-vzdump-partial-clean (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-thin-check (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-internal-place (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-crash-guard (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-priv-apply (new)
os-apply: BUNDLE WROTE /var/lib/vz/snippets/felhom-guest-hook.sh (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-shared-parent.sh (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-shared-parent.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.timer (new)
os-apply: BUNDLE WROTE /etc/felhom/crash-guard.conf (new)
os-apply: BUNDLE WROTE /etc/grub.d/01_felhom_oneshot (new)
os-apply: BUNDLE WROTE /etc/grub.d/42_felhom_oneshot (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent-rollback.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service.d/felhom-agent-limits.conf (new)
os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mgmt-watchdog (new)
os-apply: BUNDLE WROTE /etc/tmpfiles.d/felhom-privsep.conf (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.service (new)
os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.timer (new)
os-apply: BUNDLE WROTE /etc/sudoers.d/felhom-agent (new)
os-apply: BUNDLE DONE agent=0.164.0 written=26 same=2 self-check=ok signers-created=False
[OK] installed every root-owned file from config bundle v0.164.0 (sha 1d7b54d39d0c1aff…; checked, previous copies kept)
[SKIP] OOB (felhom-sshd) off (byo, or appliance --no-oob) — the bundle skipped its files
[STEP] 5b/8 internal data place (Belső tárhely)
[SKIP] byo: the host owner's thin pool is not ours to divide — no internal place
[STEP] 6/8 agent config + service
[WARN] backup target: DEGRADED — no eligible second drive, so the whole-system backup stays on the SYSTEM drive.
[WARN] It protects against file corruption but NOT against a disk failure. Attach a second drive and assign it in the dashboard.
[INFO] node=byo-test local_api=192.168.0.151:8443 tls_fp=7E:77:41:86:26:92…
[OK] wrote /etc/felhom-agent/agent.json (0600 felhom-agent)
[OK] byo config asserts passed (lan_resolver.enable / oob.enabled off; deployment_mode=byo; wg_tunnel intentionally ON — base infrastructure)
[OK] agent --selftest (read-only) passed
[OK] felhom-agent service active (non-root felhom-agent reads the config OK)
[OK] wrote /etc/felhom/os-trust.json + operator-signers (felhom-op-1) — the root-owned slow-lane anchors
[STEP] 7/8 golden archive
[WARN] no git credential in controller.yaml — fetching artifacts ANONYMOUSLY (they are world-readable; sha256 verification unchanged)
[INFO] fetching golden v0.331.0 from Gitea → /var/lib/vz/dump/vzdump-lxc-9100-2026_10_25-17_52_14.tar.zst
[OK] verified sha256 3881de62b0986ccb… matches the hub manifest
[OK] golden imported + verified: local:backup/vzdump-lxc-9100-2026_10_25-17_52_14.tar.zst
[STEP] 8/8 provision guest 9201
[SKIP] pool felhom already exists
=== felhom-agent 0.164.0 selftest=provision (vmid=9201 customer=Tester4 hostname=tester4) ===
--- front half: bring-up (provision) local:backup/vzdump-lxc-9100-2026_10_25-17_52_14.tar.zst → vmid 9201 ---
time=2026-10-25T17:53:51.729+01:00 level=INFO msg="bring-up: pool membership re-asserted" vmid=9201 pool=felhom
[OK] front half: vmid 9201 up (boot+running) in 1m26s; MAC=BC:24:11:4B:04:4E
--- back half: mint per-guest token + populate bootstrap config mount ---
time=2026-10-25T17:53:56.751+01:00 level=INFO msg="provision: back-half complete" vmid=9201 mount=mp9 guest_path=/etc/felhom-bootstrap endpoint=192.168.0.151:8443
[OK] back half: bootstrap mount mp9 → /etc/felhom-bootstrap on vmid 9201 (host dir /var/lib/felhom-agent/guests/9201/bootstrap)
local-api endpoint 192.168.0.151:8443 · leaf fp f09a20d4c01686516db562b6acc9aa7e95581d69c2318fccf7894ed772ef4a8e · token: minted (not printed)
=== selftest=provision OK — guest 9201 provisioned + bootstrap-mounted (KEPT) ===
next: reboot the guest → the golden's baked controller-bootstrap unit deploys the controller,
which PULLS its controller.yaml from the hub (retrieval passphrase) and merges in this local_api.
[OK] provision completed
[INFO] rebooting guest 9201 so the baked controller-bootstrap unit picks up the mount
[STEP] verify
[OK] pct status: running
[OK] onboot: 1
mp0: local-lvm:vm-9201-disk-1,mp=/var/lib/felhom,backup=1,size=50G
mp8: /mnt/felhom-drives,mp=/mnt/felhom-drives
rootfs: local-lvm:vm-9201-disk-0,size=32G
[OK] pool: guest 9201 is a member of felhom
[OK] acl: FelhomAgentBase@/ present (user+token)
[OK] acl: FelhomAgentGuest@/pool/felhom present (user+token)
[OK] acl: FelhomAgentStore@/storage/local present (user+token)
[OK] acl: FelhomAgentStore@/storage/local-lvm present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990000 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990001 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990002 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990003 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990004 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990005 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990006 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990007 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990008 present (user+token)
[OK] acl: FelhomAgentGuest@/vms/990009 present (user+token)
[OK] authz signers: 2 (operator-signed self-update armed)
[OK] controller: Up 5 seconds (healthy) (after ~0s)
[INFO] controller image: gitea.dooplex.hu/admin/felhom-controller:0.342.0
[WARN] cloudflared not visible yet
[INFO] (confirm in the hub UI that host Tester4-f967d4 reports guest 9201)
[OK] Day-0 provision SUCCESS — vmid=9201 host_id=Tester4-f967d4 customer=Tester4 golden=local:backup/vzdump-lxc-9100-2026_10_25-17_52_14.tar.zst
INSTALL_RC=0
Connection to 192.168.0.151 closed.
@@ -0,0 +1,10 @@
18:05:26 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
18:06:01 [1] deployed, controller state=running, pinned={'homepage': 'ghcr.io/gethomepage/homepage:v1.13.2'}
18:06:01 family gate: homepage is family-gated — passed as the household (cookie set)
18:06:01 deployed: True, state=running
18:06:01 STRANGER (text/html): 302 location=https://felhom.enkisfelhom.hu/__family/start?rd=https%3A%2F%2Fhomepage.enkisfelhom.hu%2F
18:06:01 STRANGER (application/json): 401 location=
18:06:01 HOUSEHOLD (dashboard session -> family cookie): 200, body 48349 B, homepage marker=True
18:06:02 [X] stop -> 200 {'ok': True, 'message': 'Stack homepage stop completed'}
18:06:34 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'homepage', 'volumes_removed': ['homepage_homepage_config'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd_note': 'A
18:06:42 [X] after remove: deployed=False leftovers=''
@@ -0,0 +1,10 @@
2026/10/25 17:04:28 sync.go:535: [INFO] [sync] Updated plex/docker-compose.yml
2026/10/25 17:04:28 sync.go:535: [INFO] [sync] Updated plex/.felhom.yml
2026/10/25 17:04:28 sync.go:261: [INFO] [sync] catalog moved 227a54498f76 -> 7e3f9543ae72 (0 new, 4 stack dir(s) changed)
2026/10/25 17:04:28 sync.go:299: [INFO] [sync] Catalog sync complete
2026/10/25 17:04:28 sync.go:142: [INFO] [sync] Initial sync: Sablonok frissítve — frissítve: homepage, n8n, opengist, plex
expect n8n/docker-compose.yml ∋ N8N_DIAGNOSTICS_ENABLED=false: 1
expect opengist/docker-compose.yml ∋ 256M: 2
expect homepage/.felhom.yml ∋ family_gate: true: 1
drill main 7e3f954, box cache 7e3f954: DRILL-CONFIRMED
@@ -0,0 +1,6 @@
26: repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
-rw------- 1 root root 1994 Oct 22 14:47 /var/lib/docker/volumes/felhom-controller-data/_data/controller.yaml.pre-release1026-drill
mirror main 7e3f9543
cache HEAD 227a544
https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
@@ -0,0 +1,5 @@
# installer http_code: the new test against the OLD helper (felhom-host-install.sh 1.36.0), 2026-10-25
FAIL test_http_code_prints_one_status_on_a_refusal -- http_code printed '[401401]' for a 401 (the preflight's case never matches)
PASS test_http_code_prints_the_status_on_success
41 passed, 1 failed
# live, the BYO VM's preflight with installer 1.36.0: 'unexpected hub status 404404' (wrong id) and '401401' (wrong passphrase) — byo/ in this folder
@@ -0,0 +1,22 @@
#!/usr/bin/env python3
"""homepage_gate.py — D46 on the drill catalog (9202): Homepage deployed with `family_gate: true` stops a stranger
(no cookie → a redirect to /__family/start) and lets the household in (dashboard session → felhom_famgate → 200).
The same household pass on the same box is the positive control. Removed through the product at the end."""
import os, re, sys, time
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../catalog-sweep-2026-10-24/tools"))
os.environ.setdefault("TARGET", "9202")
import box_walk as w
APP = SUB = "homepage"
w.login()
try:
ok = w.deploy(APP, SUB) and w.wait_app(SUB, "/", tries=60, delay=5)
w.say(f"deployed: {ok}, state={w.stack(APP).get('state')}")
for accept in ("text/html", "application/json"):
r = w.sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", f"Accept: {accept}", "-H", f"Host: {SUB}.{w.DOMAIN}", f"{w.BASE}/"])
st = re.match(r"HTTP/\S+ (\d+)", r.stdout or "")
w.say(f"STRANGER ({accept}): {st.group(1) if st else '?'} location={w._loc(r.stdout)[:90]}")
rc, code, body = w.app_curl(SUB, "/")
w.say(f"HOUSEHOLD (dashboard session -> family cookie): {code}, body {len(body)} B, homepage marker={'homepage' in body.lower()}")
finally:
w.remove(APP)
@@ -0,0 +1,85 @@
#!/usr/bin/env python3
"""repoint.py drill|live|status — (release 2026-10-26 copy: TARGET=9202, Homepage family_gate expected) point the box's controller at the drill catalog, or put the saved
controller.yaml back byte for byte.
The drill repo on Gitea is PRIVATE (R-992) and the box holds no catalog credential, so the drill is MIRRORED onto the
box: a bare git repository in the controller's own data volume, loaded from a git bundle, and
`git.repo_url: file:///opt/docker/felhom-controller/drill-catalog.git` — the method of
`audits/burndown-3-2026-10-17/tools/repoint.py`. The first version of this file set repo_url to the Gitea URL and
believed the repo public; the box's clone then failed („could not read Username", 21:18:11) and the controller ran on
the templates it already had — the first drill pass proved nothing (README, „Drill fixes").
`drill` ends with a POSITIVE check, not the absence of an error: the box's catalog cache must be at the drill's main
commit, and every `EXPECT` line must be in the box's copy of that template. Prints repo_url, never a token."""
import os, subprocess, sys, time
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../catalog-sweep-2026-10-24/tools"))
os.environ.setdefault("TARGET", "9202")
import box_walk as w
DRILL = "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill"
VOL = "/var/lib/docker/volumes/felhom-controller-data/_data" # guest path of the controller's data volume
CVOL = "/opt/docker/felhom-controller" # the same, inside the controller container
SAVE = f"{VOL}/controller.yaml.pre-release1026-drill"
MIRROR_URL = f"file://{CVOL}/drill-catalog.git"
EXPECT = [("n8n/docker-compose.yml", "N8N_DIAGNOSTICS_ENABLED=false"), ("opengist/docker-compose.yml", "256M"), ("homepage/.felhom.yml", "family_gate: true")]
def status():
print(w.guest(f"""grep -n 'repo_url' {VOL}/controller.yaml; ls -l {SAVE} 2>&1 | tail -1
docker exec felhom-controller sh -c 'git -C {CVOL}/drill-catalog.git log -1 --format="mirror main %h" main 2>&1;
git -C {CVOL}/data/catalog-cache log -1 --format="cache HEAD %h" 2>&1; git -C {CVOL}/data/catalog-cache remote get-url origin'"""))
def mirror():
b = f"/tmp/drill-rel1026-{os.getpid()}.bundle"
subprocess.run(["git", "-C", DRILL, "bundle", "create", b, "main"], check=True, capture_output=True)
subprocess.run(["scp", "-q", "-o", "StrictHostKeyChecking=accept-new", b, f"{w._T['ssh']}:{b}"], check=True)
subprocess.run(w.ssh_args() + [f"pct push {w.GUEST} {b} {VOL}/{os.path.basename(b)} && rm -f {b}"], check=True)
print(w.guest(f"""set -e
docker exec felhom-controller sh -c 'test -d {CVOL}/drill-catalog.git || git init -q --bare -b main {CVOL}/drill-catalog.git
git -C {CVOL}/drill-catalog.git fetch -q {CVOL}/{os.path.basename(b)} +main:main && rm -f {CVOL}/{os.path.basename(b)}
git -C {CVOL}/drill-catalog.git log -1 --format="mirror main %h" main'"""))
os.remove(b)
def restart_sync():
w.guest("docker restart felhom-controller >/dev/null")
for _ in range(60):
time.sleep(3)
try:
w.login()
break
except SystemExit:
continue
w.sync_rescan()
time.sleep(5)
print(w.guest("docker logs --since 3m felhom-controller 2>&1 | grep -E '\\[sync\\]' | grep -v DEBUG | tail -6 | cut -c1-200"))
def drill():
head = subprocess.run(["git", "-C", DRILL, "rev-parse", "--short=7", "main"], capture_output=True, text=True).stdout.strip()
mirror()
print(w.guest(f"""set -e
test -f {SAVE} || cp -p {VOL}/controller.yaml {SAVE}
sed -i -E 's#^(\\s+repo_url: ).*$#\\1{MIRROR_URL}#' {VOL}/controller.yaml
grep -n 'repo_url' {VOL}/controller.yaml"""))
restart_sync()
cache = w.guest(f"docker exec felhom-controller git -C {CVOL}/data/catalog-cache log -1 --format=%h").strip()
ok = cache.startswith(head)
for f, want in EXPECT:
got = w.guest(f"docker exec felhom-controller grep -c -- '{want}' {CVOL}/data/catalog-cache/templates/{f} || true").strip()
print(f" expect {f} ∋ {want}: {got}")
ok = ok and got not in ("", "0")
print(f"drill main {head}, box cache {cache}: " + ("DRILL-CONFIRMED" if ok else "DRILL-NOT-CONFIRMED"))
sys.exit(0 if ok else 1)
def live():
print(w.guest(f"""set -e
cp -p {SAVE} {VOL}/controller.yaml
cmp {SAVE} {VOL}/controller.yaml && echo RESTORED-IDENTICAL; grep -n 'repo_url' {VOL}/controller.yaml"""))
restart_sync()
status()
{"drill": drill, "live": live, "status": status}[sys.argv[1]]()
@@ -0,0 +1,100 @@
#!/usr/bin/env python3
"""t1probe.py — controller 0.343.0 on Tester 1 (release brief Part 5): one local unit restore and one kept-data cycle with
a THROWAWAY Tandoor, through the endpoints the pages use. Each restore: the seeded item back, then the undo copy
released after the stable window — read from the controller's own line „ran stably for … — the restore copy is
released" (the positive observable) and the restore-undo folder gone. Removed through the product at the end.
usage: SC=<dir with .ctlpw> SWEEP_EV=<evidence dir> TARGET=tester-1 python3 t1probe.py"""
import json, os, re, sys, time
TOOLS = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../catalog-sweep-2026-10-24/tools")
sys.path.insert(0, TOOLS)
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../restore-truth-2026-10-25/tools"))
import sweep as sw # noqa: E402
import unitprobe2 as up # noqa: E402
w, fxm = sw.w, sw.fxm
APP = "tandoor"
def wait_release(since, say, label, cap=600):
for _ in range(cap // 15):
time.sleep(15)
line = sw.g(f"docker logs --since {int(since)} felhom-controller 2>&1 | grep -E '{APP}: ran stably for .* restore copy is released' | tail -1").strip()
if line:
undo = sw.g(f"ls -d /mnt/*/felhom-data/backups/restore-undo/{APP} /mnt/felhom-drives/*/felhom-data/backups/restore-undo/{APP} 2>/dev/null").strip()
say(f" [{label}] released: {line[:200]} | undo folder now: {undo or 'gone'}")
return {"line": line, "undo_left": undo}
say(f" [{label}] NO release line within {cap}s")
return {"line": "", "undo_left": "?"}
def main():
d = os.path.join(sw.EV, "_t1probe")
os.makedirs(d, exist_ok=True)
log = open(os.path.join(d, "walk.log"), "a")
def say(*a):
line = time.strftime("%H:%M:%SZ", time.gmtime()) + " " + " ".join(str(x) for x in a)
print(line, flush=True)
log.write(sw.scrub(line) + "\n")
log.flush()
w.say = say
fx = fxm.FIXTURES[APP]
sub = getattr(fx, "sub", APP)
rec = {"app": APP}
w.login()
up.w.SC = w.SC
t_start = int(time.time())
try:
rec["install"] = w.deploy(APP, sub) and w.wait_app(sub, "/", tries=90, delay=5)
tok = fx.seed(w, sub, say)
rec["seed"] = bool(tok)
sw.backup_now(rec, say)
# 1. a plain local unit restore
t1 = time.time()
sw.restore_unit(APP, rec, say)
rec["unit_restore"] = rec.pop("restore")
say(" [1] unit restore: " + rec["unit_restore"].get("status", "")[:300])
time.sleep(20)
rec["unit_restore_item"] = bool(tok) and bool(fx.verify(w, sub, tok, say))
say(f" [1] item after the unit restore: {rec['unit_restore_item']}")
rec["unit_restore_release"] = wait_release(t1, say, "1")
# 2. the kept-data cycle: remove keeping the backups, reinstall, restore the earlier install's set-aside unit
sw.remove(APP, keep=True, say=say)
t_re = time.time()
rec["reinstall"] = w.deploy(APP, sub, dict(w.GENERATED.get(APP) or {}))
aside = ""
for _ in range(60):
time.sleep(10)
aside = sw.g(f"docker logs --since {int(t_re)} felhom-controller 2>&1 | grep -E '{APP}: the recovery unit of an earlier install' | cut -c1-300")
if aside.strip():
break
rec["set_aside_line"] = aside.strip()
say(" [2] set aside: " + (aside.strip()[:260] or "NONE within 10 min"))
html = up.page("/backups/restore")
m = re.search(r'action="/backup/kept/restore".*?name="id" value="([0-9a-f]{16})"', html, re.S)
rec["restore_page_offers"] = bool(m)
if not m:
say(" [2] the restore page offers NO restore of the set-aside unit")
return
t2 = time.time()
head, loc = up.post_form("/backup/kept/restore", {"id": m.group(1), "confirm": "1"})
say(f" [2] POST /backup/kept/restore -> {head} {loc[:1]}")
up.wait_restore("kept_restore", rec)
say(" [2] kept restore: " + rec["kept_restore"]["status"][:300])
time.sleep(20)
rec["kept_restore_item"] = bool(tok) and bool(fx.verify(w, sub, tok, say))
say(f" [2] item after the kept-data restore: {rec['kept_restore_item']}")
rec["kept_restore_release"] = wait_release(t2, say, "2")
finally:
rec["controller_lines"] = sw.scrub(sw.g(
f"docker logs --since {t_start} felhom-controller 2>&1 | grep -E '{APP}' | grep -iE 'set aside|restor|stabl|install id|D47|D48|R-1270|refused|released|KEPT' | cut -c1-300"))
sw.remove(APP, keep=False, say=say)
rec["after_remove"] = sw.leftovers(APP, sub)
rec["set_aside_after_remove"] = sw.g(f"ls -1 /mnt/*/felhom-data/backups/set-aside/{APP} 2>&1 | head -3").strip()
json.dump(rec, open(os.path.join(d, "record.json"), "w"), ensure_ascii=False, indent=1)
if __name__ == "__main__":
main()
+6 -3
View File
@@ -113,7 +113,7 @@ match what the reader sees is how an instrument stops being believed (R-421). It
**nothing was proven on the day this line was drawn** and a stopping line that moves a status is a
stopping line that lies.
## Install & onboarding — 7 rows (P3 5, P4 2)
## Install & onboarding — 8 rows (P3 6, P4 2)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -123,6 +123,7 @@ stopping line that lies.
| **R-1273** | Install & onboarding | P3 | **An install that fails on the image registry's rate limit tells the household only „exit code 1”.** Catalog sweep 2026-10-24, code-server 16:01:37: `lscr.io … toomanyrequests` → the product cleaned up right (R-649) but the event and the card say „Alkalmazás telepítése nem sikerült: Code-Server — exit code 1”; nothing says the download server is busy and a later try works. The same install went through later (`audits/catalog-sweep-2026-10-24/code-server/`). | OPEN | — | Recognise `toomanyrequests` (and a pull timeout) in the deploy error and say „the app's download server is busy — try again in an hour”; a test with the stderr line | CC |
| **R-494** | Install & onboarding | P4 | **NARROWED 2026-09-14 by operator ruling → [P3-LOW] the hub COULD create the tunnel at customer creation, for a domain already on Cloudflare. Not blocking: every customer has their own domain and the operator creates the tunnel per day-0 A.1 (`architecture/01-topology-and-trust.md`).** *Original finding, kept:* **[P1-HIGH] A new customer's dashboard has NO reachable address unless the operator hand-makes a Cloudflare tunnel — the link in the setup-code mail is dead.** MEASURED 2026-09-14 on a fresh install from the public ISO (drill intervention **I1**): the claim mail points at `https://felhom.drill0242.felhom.eu`; that name has **no A and no AAAA** record (`dig @1.1.1.1`, control `felhom.enkisfelhom.hu` resolves); the hub has **no tunnel- or DNS-creation code** (`hub/internal/cloudflare/` holds only geo-rule removal; `cf_tunnel_token` is a pasted, optional form field, `configs.go:1478`) — day-0 runbook A.1 makes it a manual Cloudflare-dashboard step that nothing on the customer-create page asks for; the box's own split-horizon resolver on the appliance LAN IP answered `google.com` but not the dashboard name at 13:27:39Z; the agent applied the record at **13:27:44Z** (`lanresolver: applied split-horizon record … ip=192.168.0.158`, 3 m 46 s after the controller started), so the box CAN answer the name — **but only to a device that uses the box as its DNS server, and no document, screen or mail tells a household to do that**; the router and the installer-offered DNS answer nothing. The page was reachable only at the guest's LAN address with the name forced (`curl --resolve …:443:192.168.0.158`). **A volunteer could not have done that.** **What it needs:** an operator ruling — the hub creates the tunnel and DNS at customer creation, or the product gives a household a LAN address that works with no DNS change. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: operator-side automation; the operator creates the tunnel by hand per the day-0 runbook.** | — | — | CC |
| **R-504** | Install & onboarding | P4 | **[P3-LOW] `iso.felhom.eu` cannot show an index page on its own — its root returns 404, and the download page lives on the website instead.** MEASURED 2026-09-14: `https://iso.felhom.eu/` and `/index.html` → 404; only named objects answer. The host is an R2 bucket behind a custom domain; whether R2 would serve an uploaded `index.html` at `/` was **not measured** (uploading anything to the public bucket is a publication). The ISO v1.27.0 task puts the Hungarian download page at `felhom.eu/letoltes` (published with the ISO, after the operator's yes). **Remaining:** a redirect from `iso.felhom.eu/` to that page needs a Cloudflare rule the session has no credential for. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (Cloudflare rule)** **Re-ranked 2026-10-03: P3→P4: households are sent to the website's download page; the bare address is cosmetic.** | — | — | operator |
| **R-1284** | Install & onboarding | P3 | **The installer's preflight never words a refusal: `http_code` printed the status twice („401401"), so the 401 („passphrase REJECTED"), 404 („customer not found") and 409 (R-1167's live-box sentence) branches never matched — every refusal read „unexpected hub status 401401".** Seen live 2026-10-25 on the BYO test VM (attended release run) with installer 1.36.0: a wrong id read `404404`, a wrong passphrase `401401`. R-1167 was CLOSED 2026-10-20 on a publish whose 409 sentence could not show. With `-f`, curl prints `-w` and exits 22 on a 4xx, and the fallback curl printed the code again (line 1833 already worked around it by keeping the last 3 characters). | OPEN — **fixed on main, NOT published:** installer 1.36.1 (`http_code`: one curl without `-f`, never failing); `test_http_code_prints_one_status_on_a_refusal`, seen failing on the old helper (`audits/release-2026-10-26/redproof/installer-http-code-doubled.txt`). | the publish is a DooPlex change (tag + webpage git-sync pins + the webpage sync) — the operator's | publish installer 1.36.1 (cut `installer-v1.36.1`, bump both `--ref` in `manifests/webpage.yaml`, sync the webpage alone); then a preflight with a wrong passphrase reads „passphrase REJECTED (401)"; then close | operator |
## Apps & catalog — 17 rows (P3 7, P4 10)
@@ -158,7 +159,7 @@ stopping line that lies.
| **R-683** | App updates | P3 | **[P3-LOW] Watch: after a power cut during an update's health check, the hold named an HOUR-OLD second-drive copy, not the one the update's own backup should have just made.** 2026-09-24 chaos round 3 (nextcloud, `backup_max_age: 1m`): no `backing-up` phase was seen and the hold named Tier 2 at 13:04 for an update pressed at 14:04; the pre-cut controller log was lost with the container (the runner now saves it at arm time — R-320). Round 11, the same action without a power cut, named a fresh 14:34 copy and logged the Tier-2 copy. The sentence was TRUE (it named the copy it offered); the question is why the update did not back up first. Not reproduced; watch the next power-cut drill. `audits/night-2026-09-24/E/round-03*.json`, `E/round-11-controller-pre.log` | **OPEN — P3; owner: CC (watch)** | — | — | CC |
| **R-785** | App updates | P3 | **[P3-LOW] SparkyFitness is pinned 11 releases and a major behind upstream (v0.17.3; upstream v1.7.3, v1.6.0 dated 2026-07-24).** READ 2026-10-01 (`audits/visitors-2026-10-01/C/bench/C1-previous-tag.txt`). **Needs:** an update walk 0.17 → 1.x through the ladder (bench + box), after R-784 is decided. | **OPEN — rank P3-LOW; owner: CC (after R-784)** | — | — | CC |
## Backup & restore — 59 rows (P1 1, P2 14, P3 26, P4 18)
## Backup & restore — 60 rows (P1 1, P2 14, P3 26, P4 19)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -221,6 +222,7 @@ stopping line that lies.
| **R-1235** | Backup & restore | P3 | **The release copy on ep0 (D33) is proven to load, not yet to restore a registry — and it follows the newest versions, not the ones the fleet needs.** Built 2026-10-23 (`scripts/release-copy/`, `audits/day-2026-10-23/08-release-copy-D33.txt`). (a) `total-loss-of-dooplex.md` „Restore the release copy” steps 2 (skopeo push back, same digest) and 3 (agent packages back) were never walked into a real new registry — only a `docker load` on 9202. (b) The copy keeps the newest 3 image tags and 5 agent versions; it does not ask the hub for the floor and the vouched agent and always include them — a vouch more than 5 releases behind would fall out. (c) The golden (`felhom-golden`, 637 MB for 0.331.0) is not copied: a new install after a DooPlex loss needs it (the brief named images + agent packages only; one golden fits under the 3 GB cap). | OPEN | — | (a) a throwaway registry on demo-hp, push one tag back, compare digests; (b) read `/configuration`'s floor + vouched agent; (c) the operator says whether the golden goes in | CC |
| **R-1263** | Backup & restore | P4 | **Small items from review round 2:** a LOCAL restore's put-back mails the operator „its off-site restore FAILED” (`felhom-controller/controller/cmd/controller/main.go:1514-1518`, shared via `restore_unit.go:517-518`); Remove — even „keep my data” — deletes undeclared volumes labelled with the app's project, which no tier holds (`internal/stacks/delete.go:763-764`, R-1117); the capped copy's `--delete` reads the source in the daemon's mount view, the emptiness check in the controller's (`tier2.go:834` vs `:464`); stale comments — `felhom-agent/internal/localapi/backup_cap.go:15` („keep-last”) and `restore_crash_putback.go:32` (names a test file that does not exist); the qBittorrent password goes to whatever answers the DNS name `qbittorrent` (`upwatch/explain.go:94,108-113`); kept-copy Remove has a check-to-remove gap (`tier2_kept.go:92-111`); the release-copy restore check trusts hashes stored on ep0 (`runbooks/total-loss-of-dooplex.md`); rrsync without `-munge`/`-wo` (`audits/day-2026-10-23/08-release-copy-D33.txt:6`). | OPEN | — | fix in each repo's next attended pass | CC |
| **R-1283** | Backup & restore | P3 | **MeTube's downloads are excluded from every backup — make them `optional` (D44 #25).** Catalog `c50adfc` (R-1019) marks the download folders of qBittorrent, Transmission, NZBGet, SABnzbd and MeTube `excluded` (`app-catalog-felhom.eu/templates/metube/.felhom.yml:33`). Risk: a MeTube video may be gone from its source site, so after a drive failure it cannot be fetched again. Pick: MeTube's `media/metube` becomes `optional` — the second-drive copy carries it, the off-site copy does not; NZBGet and SABnzbd stay excluded (re-downloadable). Cost: second-drive space for MeTube's folder. Operator ruling D44 = all picks, 2026-10-25. Note: `architecture/15-media-library.md` §7. | OPEN | — | change the class in the catalog template, run the catalog gates (the download-binds gate and its decoys must accept MeTube as `optional`), check on a Tier-0 box that a second-drive copy carries the folder and the off-site run does not | CC |
| **R-1286** | Backup & restore | P4 | **demo-hp's kimai has been HELD since 2026-10-14 09:06Z („held after a failed restore whose rollback also failed") and no row names it.** Seen 2026-10-25 by the D47 upgrade count (attended release run): kimai is the one app of 12 on demo-hp without an install id, because the start-up step skips a held app; `app.yaml` says deployed, no container exists, boot reconciliation leaves it held, the night update leg has skipped it since at least 2026-10-16 (`audits/burndown-apps-2026-10-16/P1/night-chain-log.txt`). A test box; the hold is the designed fail-closed state (R-379), but 11 days with no record is how a held household app would go unseen too. | OPEN | — | read why the 2026-10-14 restore and its rollback failed (demo-hp's controller log of that day, if kept; the hold record); then clear the hold (`--clear-restore-hold kimai` + a restart) or remove the app through the product; check that a held app raises a standing operator signal, else file that | CC |
## Storage & devices — 14 rows (P3 5, P4 9)
@@ -273,7 +275,7 @@ stopping line that lies.
| **R-1280** | Security & access | P3 | **A signed operation's window has no maximum length — cap it at 24 h (D44 #22).** The agent (`felhom-agent/internal/authz/verifier.go`) and the root wrapper (`configs/felhom-os-apply`) check `now ∈ [issued_at, expires_at]` with a 2-minute skew on `issued_at`, but neither checks `expires_at − issued_at`; `felhom-opsign` defaults to 30 min (`cmd/felhom-opsign/main.go:57`) and the nonce store keeps a record until its expiry. Risk: a signed job made with a long window by mistake stays usable for that whole time. Pick: the agent and the root wrapper refuse a window longer than 24 h. Operator ruling D44 = all picks, 2026-10-25. Note: `architecture/04-control-plane-authorization.md` §2. | OPEN | — | one agent change in both checkers + a test that a 24 h + 1 s window is refused (red first) and a 24 h one is accepted; rides an agent release (agent releases stop for the operator's vouch) | CC |
| **R-1281** | Security & access | P2 | **Every app container can reach FileBrowser directly; the only guard is the secret header name — take FileBrowser off the app networks (D44 #23).** FileBrowser trusts its proxy header (`auth.methods.proxy`); every app container can reach `filebrowser:80` without traefik; the guard is that the header NAME is a per-box secret that traefik's forwardAuth strips from client requests (`felhom-controller/controller/internal/infra/filebrowser_sso.go:15-22`). Risk: an app that learns the name (a hacked app on the box) reads and writes every household file. Pick: FileBrowser joins only the network traefik uses, so no app container reaches it directly. Operator ruling D44 = all picks, 2026-10-25. Note: `architecture/13-threat-model.md` §5. | OPEN | — | a compose/network change in the controller's infra render + a test on the rendered networks; a live check on a Tier-0 box that a forged header from an app container no longer reaches FileBrowser (the 2026-10-18 measurement, now refused) while the dashboard's file manager still signs in; one controller release | CC |
## Box system & updates — 12 rows (P2 1, P3 7, P4 4)
## Box system & updates — 13 rows (P2 1, P3 7, P4 5)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -289,6 +291,7 @@ stopping line that lies.
| **R-1004** | Box system & updates | P4 | **After an unclean stop every guest ext4 volume waits ~43 s in multi-mount protection before it mounts, so each restart after a power cut takes ~1.5 min longer.** Kernel: „EXT4-fs warning (device dm-6): ext4_multi_mount_protect: MMP interval 42 higher than expected, please wait" on dm-6/dm-7 and loop0/loop1, demo-hp and Tester 1, every round of `audits/night-power-2026-10-13/` (R1 `repairs.txt`). | **READY** — filed 2026-10-14 | — | CC: find where MMP is switched on for the guest volumes (Proxmox `pct` create, or our own mkfs), whether it protects anything on a one-host disk, and measure the restart without it on a scratch guest | CC |
| **R-1013** | Box system & updates | P4 | **After a power cut the agent's start-up stale-lock pass logs an ERROR „pct unlock failed … no lock found" when Proxmox's own start-all removed the lock in the same second.** The outcome is the goal (no lock), so the line is a false alarm in the log. Seen on demo-hp 2026-10-14 12:07:44–45 UTC (`audits/power-cut-fixes-2026-10-14/E/after-boot.txt`; Proxmox: „removed left over backup lock from '9201'!"). Small, but the agent's release for this session is spent (v0.156.1). | **OPEN** — filed 2026-10-14 | — | CC: treat „no lock found" from `pct unlock` as already clear (INFO), with a test; ride the next agent release | CC |
| **R-1061** | Box system & updates | P4 | **A per-customer floor reached Tester 1 only at its next 15-minute report, not in seconds.** Set 2026-10-15 12:55:51 UTC for tester-1, demo-hp and demo-felhom together (hub 0.153.0, CC's token): demo-hp and demo-felhom ran 0.314.0 within 13 s (the hub's wake); Tester 1 moved at 13:09:35, its regular report (`audits/gate-restore-2026-10-15/release/03-boxes-move.txt`). It had restarted at 12:54:29 onto rc.1 and logged „hub wait channel active". Why the wake did not reach it is not read (the wake line is DEBUG). No harm: the floor arrived. | **OPEN** — filed 2026-10-15 | — | Read Tester 1's DEBUG ring at the next floor change; compare with the hub's bump log | CC |
| **R-1285** | Box system & updates | P4 | **D43's removal of our SSH drop-in from a BYO host happens one bundle LATE: the bundle run that delivers agent 0.165.0 is planned by the box's OLD (0.164.0) wrapper, which writes the drop-in as before („same"), and only then replaces itself.** Measured 2026-10-25 on the BYO test host (VM 9371 on demo-hp, `Tester4-f967d4`; attended release run): 0.164.0 day-0 → signed `agent_update` + `agent_config_update` 0.165.0 → the planted drop-in still there; a second `agent_config_update` (same bundle) → `BUNDLE REMOVED … (ours, on a BYO host — D43)`, nothing written, `sshd -T` the owner's own `permitrootlogin yes` (`audits/release-2026-10-26/byo/byo-proof.txt`). No box is affected today (the fleet's only BYO host is this test host); a fresh BYO install on ≥ 0.165.0 runs the new wrapper from the start. | OPEN | — | either the wrapper plans again after it replaced itself (one agent change + a test that a 0.164.0→0.165.0 bundle on a BYO record removes our copy in ONE run, red first), or the release procedure sends a BYO box's config bundle twice; until then a BYO box upgraded from ≤ 0.164.0 keeps our drop-in until its next bundle | CC |
## Monitoring & notifications — 22 rows (P2 1, P3 15, P4 6)
+4
View File
@@ -7,6 +7,10 @@ Moved here byte for byte on 2026-10-14 (night session), newest first, so that `S
## Moved from STATUS on 2026-10-25 (attended release run)
The Monday list, run on 2026-10-25 (attended; results in `REPORT-release-2026-10-26.md`):
**Monday 2026-10-26, release list (attended).** 1. Controller v0.343.0: the restore and backup fixes (R-1251–R-1253, R-1269–R-1271), R-1250. If not: the P1 stays open. 2. Agent v0.165.0 + your vouch: BYO keeps its SSH (R-1257). 3. Hub v0.170.0: the recovery package in one step (R-1254). 4. Catalog: n8n usage data off; OpenGist 256 MB.
Decided: **D44 = all picks**, **D45 = all picks** (operator, 2026-10-25; R-1265 and R-1266 closed, the four code picks are R-1280–R-1283).
| D45 | R-1266 | Seven places where a design document says one thing and the code does another. One line each, with a pick: `documentation/audits/review-round-2-2026-10-24/D44-D45-picks.md`. Answer „all picks" or name the ones you disagree with. | **Yes — the text follows the code** (each is how the system works today; you read every changed sentence) | the code follows the text: six code changes, one of them to the box's door for you | the documents keep saying what is not true | — |
+10
View File
@@ -1,3 +1,13 @@
## 2026-10-25 — installer 1.36.1: the preflight words a refusal again (committed, NOT published)
- **`felhom-host-install.sh` `http_code`** printed the status twice on a 4xx (`curl -f` prints `-w` and exits 22, and the
`|| curl …` fallback printed it again: „401401"), so the preflight's 401 („passphrase REJECTED"), 404 („customer not
found") and 409 (R-1167's „this customer already has a live box") branches never matched — every refusal read
„unexpected hub status 401401". Seen live on the BYO test VM (attended release run 2026-10-25). Now one curl without
`-f`, `|| true`. `test_hostinstall.py`: `test_http_code_prints_one_status_on_a_refusal` (seen failing on the old helper,
`documentation/audits/release-2026-10-26/redproof/installer-http-code-doubled.txt`). **Not published:** the public
script follows the `installer-v*` tag; a publish needs the tag and the webpage sync (felhom.eu R-1284).
## 2026-10-24 — `ep0-iperf3/`: the tunnel-only line-speed test server on ep0 (R-1247; operator ruling 2026-10-24 07:26)
- **`ep0-iperf3/felhom-iperf3.service`** (new, installed by hand on ep0 2026-10-24): Debian's `iperf3 --server` bound to
+5 -3
View File
@@ -194,7 +194,7 @@
set -euo pipefail
SCRIPT_VERSION="1.36.0" # the SINGLE version source (F-1): -h and the run banners follow it.
SCRIPT_VERSION="1.36.1" # the SINGLE version source (F-1): -h and the run banners follow it.
# The hub used to carry a copy for its Setup tab; R-94 DELETED it
# (2026-08-02) because the hub cannot know which version a box runs —
# the Setup command fetches this script at run time. scripts/
@@ -453,8 +453,10 @@ _state_get() {
STATE_FILE="$STATE_FILE" python3 -c "import json,os,sys;d=json.load(open(os.environ['STATE_FILE']));print(d.get(sys.argv[1],''))" "$1" 2>/dev/null
}
http_code() { # GET, prints status code only (read-only preflight)
curl -fsS -o /dev/null -w '%{http_code}' "$@" 2>/dev/null || curl -sS -o /dev/null -w '%{http_code}' "$@" 2>/dev/null
http_code() { # GET, prints ONE status code (read-only preflight); 000 when nothing answered
# No `-f`: with it curl prints the code AND exits 22 on a 4xx, and the old `|| curl …` fallback printed it a second
# time („401401") — the preflight's 401/404/409 branches never matched (test_http_code_prints_one_status_on_a_refusal).
curl -sS -o /dev/null -w '%{http_code}' "$@" 2>/dev/null || true
}
#-------------------------------------------------------------------------------
+26
View File
@@ -700,6 +700,32 @@ def test_internal_place_runs_after_the_bundle_and_before_provision():
assert -1 < i_bundle < i_place < i_prov, (i_bundle, i_place, i_prov)
# ── http_code prints ONE status (found 2026-10-25, attended release run) ─────────────────────────────────────────────
# With `-f`, curl prints -w's code AND exits 22 on a 4xx, so the old `curl -f … || curl …` printed the code twice
# („401401"): the preflight's 401 / 404 / 409 branches never matched and every refusal read „unexpected hub status".
def _http_code_with(status):
sb = Sandbox()
try:
# a curl stub that answers like the real one: -w prints the status; -f makes a 4xx/5xx exit 22
body = ('f=0; for a in "$@"; do case "$a" in --*) ;; -*f*) f=1;; esac; done\n'
'printf "%s" STATUS\n[ "$f" = 1 ] && [ STATUS -ge 400 ] && exit 22\nexit 0').replace("STATUS", status)
sb.stub("curl", body)
rc, out = sb.run(func("http_code") + 'c=$(http_code https://hub.invalid/api/v1/config/x -H "X: y"); printf "[%s]" "$c"')
return out
finally:
sb.close()
def test_http_code_prints_one_status_on_a_refusal():
for status in ("401", "404", "409"):
out = _http_code_with(status)
assert out.endswith("[%s]" % status), "http_code printed %r for a %s (the preflight's case never matches)" % (out, status)
def test_http_code_prints_the_status_on_success():
assert _http_code_with("200").endswith("[200]")
def main():
tests = [(n, f) for n, f in sorted(globals().items()) if n.startswith("test_") and callable(f)]
fails = 0