# guest-night demo-hp taken 2026-11-11T10:09:24Z
log first line: 2026-11-10T23:15:05.505129813Z
log last line:  2026-11-11T10:09:20.389434141Z

=== scheduler jobs (UTC): runs, first start, last start, last three "took" ===
agent-channel-health               runs=654   first=2026-11-10T23:16:05 last=2026-11-11T10:09:05 [completed (took 664ms)] [completed (took 647ms)] [completed (took 669ms)]
backup-cache                       runs=131   first=2026-11-10T23:17:05 last=2026-11-11T10:07:05 [completed (took 674ms)] [completed (took 970ms)] [completed (took 655ms)]
conversion-copy-release            runs=11    first=2026-11-10T23:17:05 last=2026-11-11T09:17:05 [completed (took 77ms)] [completed (took 342ms)] [completed (took 266ms)]
db-dump                            runs=1     first=2026-11-11T01:30:00 last=2026-11-11T01:30:00 [completed (took 2m33.216s)]
disk-health-check                  runs=11    first=2026-11-10T23:17:05 last=2026-11-11T09:17:05 [completed (took 962ms)] [completed (took 907ms)] [completed (took 876ms)]
docker-socket-users                runs=131   first=2026-11-10T23:17:05 last=2026-11-11T10:07:05 [completed (took 305ms)] [completed (took 414ms)] [completed (took 337ms)]
fill-watch                         runs=1     first=2026-11-11T02:30:00 last=2026-11-11T02:30:00 [completed (took 0s)]
fill-watch-interval                runs=66    first=2026-11-10T23:17:05 last=2026-11-11T10:07:05 [completed (took 0s)] [completed (took 0s)] [completed (took 0s)]
geo-verify                         runs=2     first=2026-11-11T03:17:05 last=2026-11-11T09:17:05 [completed (took 0s)] [completed (took 0s)]
hold-report                        runs=654   first=2026-11-10T23:16:05 last=2026-11-11T10:09:05 [completed (took 0s)] [completed (took 0s)] [completed (took 0s)]
hub-report                         runs=44    first=2026-11-10T23:17:05 last=2026-11-11T10:02:05 [completed (took 2.129s)] [completed (took 2.298s)] [completed (took 2.126s)]
metrics-prune                      runs=1     first=2026-11-11T03:00:00 last=2026-11-11T03:00:00 [completed (took 419ms)]
offbox-backup                      runs=1     first=2026-11-11T03:15:00 last=2026-11-11T03:15:00 [completed (took 3m8.568s)]
offsite-abandon-sweep              runs=1     first=2026-11-11T04:10:00 last=2026-11-11T04:10:00 [completed (took 0s)]
offsite-credential-retry           runs=131   first=2026-11-10T23:17:05 last=2026-11-11T10:07:05 [completed (took 0s)] [completed (took 0s)] [completed (took 0s)]
offsite-integrity                  runs=1     first=2026-11-11T05:00:00 last=2026-11-11T05:00:00 [completed (took 0s)]
offsite-proof                      runs=1     first=2026-11-11T04:30:00 last=2026-11-11T04:30:00 [completed (took 5.282s)]
restore-copy-sweep                 runs=2     first=2026-11-11T03:17:05 last=2026-11-11T09:17:05 [completed (took 0s)] [completed (took 0s)]
selfupdate-check                   runs=2     first=2026-11-11T03:17:05 last=2026-11-11T09:17:05 [completed (took 139ms)] [completed (took 134ms)]
stack-scan                         runs=327   first=2026-11-10T23:17:05 last=2026-11-11T10:09:05 [completed (took 136ms)] [completed (took 274ms)] [completed (took 158ms)]
system-health                      runs=131   first=2026-11-10T23:17:05 last=2026-11-11T10:07:05 [completed (took 396ms)] [completed (took 511ms)] [completed (took 415ms)]
tier2-backup                       runs=1     first=2026-11-11T02:30:00 last=2026-11-11T02:30:00 [completed (took 38.939s)]

=== jobs that ran fewer than 30 times: every start and end (UTC) ===
2026/11/10 23:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/10 23:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/10 23:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 106ms)
2026/11/10 23:17:06 [INFO] [scheduler] Job disk-health-check completed (took 833ms)
2026/11/11 00:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 00:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 00:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 366ms)
2026/11/11 00:17:06 [INFO] [scheduler] Job disk-health-check completed (took 868ms)
2026/11/11 01:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 01:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 01:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 396ms)
2026/11/11 01:17:06 [INFO] [scheduler] Job disk-health-check completed (took 847ms)
2026/11/11 01:30:00 [INFO] [scheduler] Running job: db-dump
2026/11/11 01:32:33 [INFO] [scheduler] Job db-dump completed (took 2m33.216s)
2026/11/11 02:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 02:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 84ms)
2026/11/11 02:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 02:17:06 [INFO] [scheduler] Job disk-health-check completed (took 911ms)
2026/11/11 02:30:00 [INFO] [scheduler] Running job: tier2-backup
2026/11/11 02:30:00 [INFO] [scheduler] Running job: fill-watch
2026/11/11 02:30:00 [INFO] [scheduler] Job fill-watch completed (took 0s)
2026/11/11 02:30:38 [INFO] [scheduler] Job tier2-backup completed (took 38.939s)
2026/11/11 03:00:00 [INFO] [scheduler] Running job: metrics-prune
2026/11/11 03:00:00 [INFO] [scheduler] Job metrics-prune completed (took 419ms)
2026/11/11 03:15:00 [INFO] [scheduler] Running job: offbox-backup
2026/11/11 03:17:05 [INFO] [scheduler] Running job: geo-verify
2026/11/11 03:17:05 [INFO] [scheduler] Job geo-verify completed (took 0s)
2026/11/11 03:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 03:17:05 [INFO] [scheduler] Running job: selfupdate-check
2026/11/11 03:17:05 [INFO] [scheduler] Running job: restore-copy-sweep
2026/11/11 03:17:05 [INFO] [scheduler] Job restore-copy-sweep completed (took 0s)
2026/11/11 03:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 03:17:05 [INFO] [scheduler] Job selfupdate-check completed (took 139ms)
2026/11/11 03:17:06 [INFO] [scheduler] Job conversion-copy-release completed (took 834ms)
2026/11/11 03:17:06 [INFO] [scheduler] Job disk-health-check completed (took 1.069s)
2026/11/11 03:18:08 [INFO] [scheduler] Job offbox-backup completed (took 3m8.568s)
2026/11/11 04:10:00 [INFO] [scheduler] Running job: offsite-abandon-sweep
2026/11/11 04:10:00 [INFO] [scheduler] Job offsite-abandon-sweep completed (took 0s)
2026/11/11 04:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 04:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 04:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 285ms)
2026/11/11 04:17:06 [INFO] [scheduler] Job disk-health-check completed (took 839ms)
2026/11/11 04:30:00 [INFO] [scheduler] Running job: offsite-proof
2026/11/11 04:30:05 [INFO] [scheduler] Job offsite-proof completed (took 5.282s)
2026/11/11 05:00:00 [INFO] [scheduler] Running job: offsite-integrity
2026/11/11 05:00:00 [INFO] [scheduler] Job offsite-integrity completed (took 0s)
2026/11/11 05:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 05:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 62ms)
2026/11/11 05:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 05:17:06 [INFO] [scheduler] Job disk-health-check completed (took 816ms)
2026/11/11 06:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 06:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 06:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 323ms)
2026/11/11 06:17:06 [INFO] [scheduler] Job disk-health-check completed (took 853ms)
2026/11/11 07:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 07:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 77ms)
2026/11/11 07:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 07:17:06 [INFO] [scheduler] Job disk-health-check completed (took 962ms)
2026/11/11 08:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 08:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 08:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 342ms)
2026/11/11 08:17:06 [INFO] [scheduler] Job disk-health-check completed (took 907ms)
2026/11/11 09:17:05 [INFO] [scheduler] Running job: selfupdate-check
2026/11/11 09:17:05 [INFO] [scheduler] Running job: conversion-copy-release
2026/11/11 09:17:05 [INFO] [scheduler] Running job: geo-verify
2026/11/11 09:17:05 [INFO] [scheduler] Job geo-verify completed (took 0s)
2026/11/11 09:17:05 [INFO] [scheduler] Running job: restore-copy-sweep
2026/11/11 09:17:05 [INFO] [scheduler] Job restore-copy-sweep completed (took 0s)
2026/11/11 09:17:05 [INFO] [scheduler] Running job: disk-health-check
2026/11/11 09:17:05 [INFO] [scheduler] Job selfupdate-check completed (took 134ms)
2026/11/11 09:17:05 [INFO] [scheduler] Job conversion-copy-release completed (took 266ms)
2026/11/11 09:17:06 [INFO] [scheduler] Job disk-health-check completed (took 876ms)

=== night lines 19:00-08:00 UTC, INFO/WARN/ERROR, backup words (first 2 per message shape) ===
2026/11/10 23:17:05 fillwatch.go:284: [INFO] [fillwatch] checked 2 filesystem(s), 0 unreadable/skipped, 0 notification(s); bands: all ok
2026/11/10 23:17:05 backup.go:1252: [INFO] [backup] Found 21 DB dump files across drives
2026/11/10 23:17:05 healthcheck.go:176: [INFO] [monitor] Health check: status=ok
2026/11/10 23:17:06 dbdump.go:208: [INFO] [backup] Discovered 6 databases
2026/11/10 23:17:06 [INFO] [backup] Discovered app data: 12 apps
2026/11/10 23:17:06 disk_health.go:277: [INFO] [web] disk-health check complete: 2 disk(s) evaluated, 0 alert(s)
2026/11/10 23:17:06 backup.go:1323: [INFO] [backup] Backup status cache refreshed
2026/11/10 23:17:06 healthcheck.go:176: [INFO] [monitor] Health check: status=ok
2026/11/10 23:18:05 main.go:2430: [INFO] [deadapp] check alive: 960 scans since boot, 12 deployed app(s) evaluated, 0 currently down
2026/11/10 23:22:05 backup.go:1252: [INFO] [backup] Found 21 DB dump files across drives
2026/11/10 23:22:05 dbdump.go:208: [INFO] [backup] Discovered 6 databases
2026/11/10 23:22:05 [INFO] [backup] Discovered app data: 12 apps
2026/11/10 23:22:05 backup.go:1323: [INFO] [backup] Backup status cache refreshed
2026/11/10 23:27:05 fillwatch.go:284: [INFO] [fillwatch] checked 2 filesystem(s), 0 unreadable/skipped, 0 notification(s); bands: all ok
2026/11/10 23:28:05 main.go:2430: [INFO] [deadapp] check alive: 980 scans since boot, 12 deployed app(s) evaluated, 0 currently down
2026/11/11 00:17:06 disk_health.go:277: [INFO] [web] disk-health check complete: 2 disk(s) evaluated, 0 alert(s)
2026/11/11 01:30:00 backup.go:601: [INFO] [backup] Starting database dump run
2026/11/11 01:30:00 backup.go:640: [INFO] [backup] Discovered 6 database(s): romm-db(mariadb), paperless-postgres(postgres), kimai-db(mariadb), docmost-postgres(postgres), bookstack-db(mariadb), adventurelog-postgres(postgres)
2026/11/11 01:30:00 dbdump.go:429: [INFO] [backup] DB dump: romm-db → romm-mariadb.sql (136.3 KB, 459ms, 40 tables)
2026/11/11 01:30:01 dbdump.go:429: [INFO] [backup] DB dump: paperless-postgres → paperless-ngx-postgres.sql (733.5 KB, 575ms, 75 tables)
2026/11/11 01:30:01 dbdump.go:429: [INFO] [backup] DB dump: kimai-db → kimai-mariadb.sql (51.3 KB, 346ms, 35 tables)
2026/11/11 01:30:02 dbdump.go:429: [INFO] [backup] DB dump: docmost-postgres → docmost-postgres.sql (164.7 KB, 563ms, 52 tables)
2026/11/11 01:30:02 dbdump.go:429: [INFO] [backup] DB dump: bookstack-db → bookstack-mariadb.sql (57.8 KB, 377ms, 42 tables)
2026/11/11 01:30:03 dbdump.go:429: [INFO] [backup] DB dump: adventurelog-postgres → adventurelog-postgres.sql (12.5 MB, 922ms, 51 tables)
2026/11/11 01:30:03 backup.go:1044: [INFO] [backup] Stopping adventurelog for safe volume dump
2026/11/11 01:30:05 busymark.go:81: [INFO] [busymark] busy: a backup, a restore or an off-site run is running — an operator reboot of the box is refused until it ends
2026/11/11 01:30:08 backup.go:941: [INFO] [backup] Volume dump: adventurelog/adventurelog_adventurelog_media → 90.1 MB
2026/11/11 01:30:14 backup.go:941: [INFO] [backup] Volume dump: adventurelog/adventurelog_adventurelog_postgres_data → 273.4 MB
2026/11/11 01:30:14 backup.go:1060: [INFO] [backup] Restarting adventurelog after volume dump
2026/11/11 01:30:20 backup.go:1044: [INFO] [backup] Stopping bookstack for safe volume dump
2026/11/11 01:30:26 backup.go:941: [INFO] [backup] Volume dump: bookstack/bookstack_bookstack_config → 6.4 MB
2026/11/11 01:30:28 backup.go:941: [INFO] [backup] Volume dump: bookstack/bookstack_bookstack_db_data → 153.6 MB
2026/11/11 01:30:28 backup.go:1060: [INFO] [backup] Restarting bookstack after volume dump
2026/11/11 01:30:35 backup.go:1044: [INFO] [backup] Stopping calibre-web for safe volume dump
2026/11/11 01:30:41 backup.go:941: [INFO] [backup] Volume dump: calibre-web/calibre-web_calibre_web_config → 921.5 KB
2026/11/11 01:30:41 backup.go:1060: [INFO] [backup] Restarting calibre-web after volume dump
2026/11/11 01:30:42 backup.go:1044: [INFO] [backup] Stopping docmost for safe volume dump
2026/11/11 01:30:46 backup.go:941: [INFO] [backup] Volume dump: docmost/docmost_docmost_storage → 2.5 KB
2026/11/11 01:30:49 backup.go:941: [INFO] [backup] Volume dump: docmost/docmost_docmost_postgres_data → 66.8 MB
2026/11/11 01:30:50 backup.go:1060: [INFO] [backup] Restarting docmost after volume dump
2026/11/11 01:31:02 backup.go:1044: [INFO] [backup] Stopping home-assistant for safe volume dump
2026/11/11 01:31:13 backup.go:941: [INFO] [backup] Volume dump: home-assistant/home-assistant_homeassistant_config → 17.8 MB
2026/11/11 01:31:13 backup.go:1060: [INFO] [backup] Restarting home-assistant after volume dump
2026/11/11 01:31:13 backup.go:1044: [INFO] [backup] Stopping kimai for safe volume dump
2026/11/11 01:31:16 backup.go:941: [INFO] [backup] Volume dump: kimai/kimai_kimai_var → 18.3 MB
2026/11/11 01:31:20 backup.go:941: [INFO] [backup] Volume dump: kimai/kimai_kimai_db_data → 154.5 MB
2026/11/11 01:31:20 backup.go:1060: [INFO] [backup] Restarting kimai after volume dump
2026/11/11 01:31:26 backup.go:1044: [INFO] [backup] Stopping opengist for safe volume dump
2026/11/11 01:31:28 backup.go:941: [INFO] [backup] Volume dump: opengist/opengist_opengist_data → 181.5 KB
2026/11/11 01:31:28 backup.go:1060: [INFO] [backup] Restarting opengist after volume dump
2026/11/11 01:31:28 backup.go:1044: [INFO] [backup] Stopping paperless-ngx for safe volume dump
2026/11/11 01:31:38 backup.go:941: [INFO] [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_data → 21.0 MB
2026/11/11 01:31:40 backup.go:941: [INFO] [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_postgres_data → 70.2 MB
2026/11/11 01:31:41 backup.go:1060: [INFO] [backup] Restarting paperless-ngx after volume dump
2026/11/11 01:31:52 backup.go:1044: [INFO] [backup] Stopping plex for safe volume dump
2026/11/11 01:32:03 backup.go:941: [INFO] [backup] Volume dump: plex/plex_plex_config → 9.9 MB
2026/11/11 01:32:03 backup.go:941: [INFO] [backup] Volume dump: plex/plex_plex_transcode → 3.0 KB
2026/11/11 01:32:03 backup.go:1060: [INFO] [backup] Restarting plex after volume dump
2026/11/11 01:32:04 backup.go:1044: [INFO] [backup] Stopping privatebin for safe volume dump
2026/11/11 01:32:04 backup.go:941: [INFO] [backup] Volume dump: privatebin/privatebin_privatebin_data → 2.0 MB
2026/11/11 01:32:04 backup.go:1060: [INFO] [backup] Restarting privatebin after volume dump
2026/11/11 01:32:07 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for adventurelog → /mnt/sys_drive/felhom-data/backups/primary/adventurelog (images=3, secrets-referenced=2, data_keys=1, portable-carried=2/2, with
2026/11/11 01:32:07 backup.go:1044: [INFO] [backup] Stopping romm for safe volume dump
2026/11/11 01:32:07 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for bookstack → /mnt/sys_drive/felhom-data/backups/primary/bookstack (images=2, secrets-referenced=3, data_keys=1, portable-carried=2/2, withheld=1
2026/11/11 01:32:17 backup.go:941: [INFO] [backup] Volume dump: romm/romm_romm_redis_data → 61.6 MB
2026/11/11 01:32:17 backup.go:941: [INFO] [backup] Volume dump: romm/romm_romm_config → 2.5 KB
2026/11/11 01:32:20 backup.go:1060: [INFO] [backup] Restarting romm after volume dump
2026/11/11 01:32:32 backup.go:743: [INFO] [backup] App-data backup completed: 6 databases (13.6 MB total), 11 volume dump(s) (2m32.713s)
2026/11/11 01:32:33 notifier.go:357: [INFO] Event pushed: db_dump_completed (info) [hu-only] — Adatbázis mentés elkészült
2026/11/11 01:32:35 busymark.go:81: [INFO] [busymark] idle again (was: a backup, a restore or an off-site run is running) — an operator reboot of the box is no longer refused for it
2026/11/11 02:30:08 tier2.go:525: [INFO] [backup] Tier 2 copied adventurelog → /mnt/felhom-drives/hdd_1/backups/secondary/adventurelog (404.1 MB, 0 leg(s), 7s)
2026/11/11 02:30:10 tier2.go:525: [INFO] [backup] Tier 2 copied bentopdf → /mnt/felhom-drives/hdd_1/backups/secondary/bentopdf (12.8 KB, 0 leg(s), 0s)
2026/11/11 02:30:14 tier2.go:525: [INFO] [backup] Tier 2 copied bookstack → /mnt/felhom-drives/hdd_1/backups/secondary/bookstack (160.3 MB, 0 leg(s), 3s)
2026/11/11 02:30:16 tier2.go:525: [INFO] [backup] Tier 2 copied calibre-web → /mnt/sys_drive/felhom-data/backups/secondary/calibre-web (5.6 MB, 1 leg(s), 1s) [SSD: state-only]
2026/11/11 02:30:20 tier2.go:525: [INFO] [backup] Tier 2 copied docmost → /mnt/felhom-drives/hdd_1/backups/secondary/docmost (120.1 MB, 0 leg(s), 2s)
2026/11/11 02:30:21 tier2.go:525: [INFO] [backup] Tier 2 copied home-assistant → /mnt/felhom-drives/hdd_1/backups/secondary/home-assistant (17.8 MB, 0 leg(s), 0s)
2026/11/11 02:30:26 tier2.go:525: [INFO] [backup] Tier 2 copied kimai → /mnt/felhom-drives/hdd_1/backups/secondary/kimai (173.0 MB, 0 leg(s), 4s)
2026/11/11 02:30:28 tier2.go:525: [INFO] [backup] Tier 2 copied opengist → /mnt/felhom-drives/hdd_1/backups/secondary/opengist (189.7 KB, 0 leg(s), 0s)
2026/11/11 02:30:31 tier2.go:525: [INFO] [backup] Tier 2 copied paperless-ngx → /mnt/sys_drive/felhom-data/backups/secondary/paperless-ngx (155.1 MB, 1 leg(s), 2s) [SSD: state-only]
2026/11/11 02:30:33 tier2.go:525: [INFO] [backup] Tier 2 copied plex → /mnt/sys_drive/felhom-data/backups/secondary/plex (9.9 MB, 0 leg(s), 0s) [SSD: state-only]
2026/11/11 02:30:35 tier2.go:525: [INFO] [backup] Tier 2 copied privatebin → /mnt/felhom-drives/hdd_1/backups/secondary/privatebin (2.0 MB, 0 leg(s), 0s)
2026/11/11 02:30:38 tier2.go:525: [INFO] [backup] Tier 2 copied romm → /mnt/sys_drive/felhom-data/backups/secondary/romm (217.2 MB, 0 leg(s), 2s) [SSD: state-only]
2026/11/11 02:30:38 tier2.go:587: [INFO] [backup] Tier 2 run complete: 12 app(s) processed (incl. volume-only — F6)
2026/11/11 03:00:00 main.go:1759: [INFO] Pruned 31667 old metric rows
2026/11/11 03:15:00 offbox.go:1094: [INFO] [offbox] backup run started (15 app(s) toggled)
2026/11/11 03:15:00 backup.go:601: [INFO] [backup] Starting database dump run
2026/11/11 03:15:00 backup.go:640: [INFO] [backup] Discovered 6 database(s): romm-db(mariadb), paperless-postgres(postgres), kimai-db(mariadb), docmost-postgres(postgres), bookstack-db(mariadb), adventurelog-postgres(postgres)
2026/11/11 03:15:00 dbdump.go:429: [INFO] [backup] DB dump: romm-db → romm-mariadb.sql (136.3 KB, 360ms, 40 tables)
2026/11/11 03:15:00 dbdump.go:429: [INFO] [backup] DB dump: paperless-postgres → paperless-ngx-postgres.sql (736.3 KB, 356ms, 75 tables)
2026/11/11 03:15:01 dbdump.go:429: [INFO] [backup] DB dump: kimai-db → kimai-mariadb.sql (51.3 KB, 266ms, 35 tables)
awk: write failure (Broken pipe)
awk: close failed on file "/dev/stdout" (Broken pipe)
RC=0
