admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

BIGNIGHT phase 4 done: tiers run, guarded update PASS, catalog reverted; box logs
gates / gates (push) Successful in 18s
Details

Browse Source
This commit is contained in:
authentik Default Admin
2026-09-14 21:18:02 +02:00
parent 6aaa3a4a34
commit e46e525ef6
6 changed files with 1010 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
documentation/audits/evidence-bignight-2026-09-14/box-logs-phase4/agent-journal.txt
+200
View File
@@ -0,0 +1,200 @@
Sep 14 21:17:01 felhom sudo[169314]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:01 felhom sudo[169318]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives/hdd_1
Sep 14 21:17:01 felhom sudo[169318]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:01 felhom sudo[169318]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:01 felhom sudo[169321]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:01 felhom sudo[169321]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:01 felhom sudo[169321]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:01 felhom felhom-agent[4303]: time=2026-09-14T21:17:01.724+02:00 level=INFO msg="reconcile: enrolled drive bound under shared parent (live, no reboot)" vmid=9201 where=/mnt/hdd_1 guest_path=/mnt/felhom-drives/hdd_1 durable_id=uuid:c4b530fd-d410-424c-917e-c8ecfdde9d8b
Sep 14 21:17:01 felhom sudo[169327]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:17:01 felhom sudo[169327]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:01 felhom sudo[169327]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:01 felhom sudo[169330]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:17:01 felhom sudo[169330]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:01 felhom sudo[169330]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:01 felhom sudo[169333]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:01 felhom sudo[169333]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:01 felhom sudo[169333]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:01 felhom sudo[169338]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sdb
Sep 14 21:17:01 felhom sudo[169338]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:01 felhom sudo[169338]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:01 felhom sudo[169341]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:01 felhom sudo[169341]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:01 felhom sudo[169341]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:02 felhom sudo[169351]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:17:02 felhom sudo[169351]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:02 felhom sudo[169351]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:02 felhom sudo[169356]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:17:02 felhom sudo[169356]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:02 felhom sudo[169356]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:02 felhom sudo[169380]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:02 felhom sudo[169380]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:02 felhom sudo[169380]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:02 felhom sudo[169388]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sdb
Sep 14 21:17:02 felhom sudo[169388]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:02 felhom sudo[169388]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:02 felhom sudo[169396]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:02 felhom sudo[169396]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:02 felhom sudo[169396]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:21 felhom sudo[170249]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives
Sep 14 21:17:21 felhom sudo[170249]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:21 felhom sudo[170249]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:21 felhom sudo[170256]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:17:21 felhom sudo[170256]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:21 felhom sudo[170256]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:21 felhom sudo[170259]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:17:21 felhom sudo[170259]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:21 felhom sudo[170259]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:21 felhom sudo[170281]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/hdd_1/felhom-data
Sep 14 21:17:21 felhom sudo[170281]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:21 felhom sudo[170281]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:21 felhom sudo[170285]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/chown 100000:100000 /mnt/hdd_1/felhom-data
Sep 14 21:17:21 felhom sudo[170285]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:21 felhom sudo[170285]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:21 felhom sudo[170293]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives/hdd_1
Sep 14 21:17:21 felhom sudo[170293]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:21 felhom sudo[170293]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:21 felhom sudo[170296]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:21 felhom sudo[170296]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:21 felhom sudo[170296]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:21 felhom felhom-agent[4303]: time=2026-09-14T21:17:21.739+02:00 level=INFO msg="reconcile: enrolled drive bound under shared parent (live, no reboot)" vmid=9201 where=/mnt/hdd_1 guest_path=/mnt/felhom-drives/hdd_1 durable_id=uuid:c4b530fd-d410-424c-917e-c8ecfdde9d8b
Sep 14 21:17:31 felhom sudo[170604]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:17:31 felhom sudo[170604]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:31 felhom sudo[170604]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:31 felhom sudo[170607]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:17:31 felhom sudo[170607]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:31 felhom sudo[170607]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:31 felhom sudo[170610]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:31 felhom sudo[170610]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:31 felhom sudo[170610]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:31 felhom sudo[170613]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sdb
Sep 14 21:17:31 felhom sudo[170613]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:31 felhom sudo[170613]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:31 felhom sudo[170616]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:31 felhom sudo[170616]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:31 felhom sudo[170616]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:32 felhom sudo[170623]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:17:32 felhom sudo[170623]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:32 felhom sudo[170623]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:32 felhom sudo[170626]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:17:32 felhom sudo[170626]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:32 felhom sudo[170626]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:32 felhom sudo[170629]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:32 felhom sudo[170629]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:32 felhom sudo[170629]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:32 felhom sudo[170632]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sdb
Sep 14 21:17:32 felhom sudo[170632]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:32 felhom sudo[170632]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:32 felhom sudo[170635]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:32 felhom sudo[170635]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:32 felhom sudo[170635]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:41 felhom sudo[171010]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives
Sep 14 21:17:41 felhom sudo[171010]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:41 felhom sudo[171010]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:41 felhom felhom-agent[4303]: time=2026-09-14T21:17:41.484+02:00 level=INFO msg="stale-lock: scanning pool guests" pool=felhom listed=1 scanned=1
Sep 14 21:17:41 felhom sudo[171029]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct exec 9201 -- ip -4 -o addr show dev eth0
Sep 14 21:17:41 felhom sudo[171029]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:41 felhom felhom-agent[4303]: time=2026-09-14T21:17:41.504+02:00 level=INFO msg="stale-lock: scanning pool guests" pool=felhom listed=1 scanned=1
Sep 14 21:17:41 felhom felhom-agent[4303]: time=2026-09-14T21:17:41.504+02:00 level=INFO msg="guest-power: watchdog alive" sweeps_since_boot=80 guests_evaluated=1 currently_stopped=0
Sep 14 21:17:41 felhom sudo[171043]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:17:41 felhom sudo[171043]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:41 felhom sudo[171043]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:41 felhom sudo[171046]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:17:41 felhom sudo[171046]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:41 felhom sudo[171046]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:41 felhom sudo[171049]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/hdd_1/felhom-data
Sep 14 21:17:41 felhom sudo[171049]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:41 felhom sudo[171049]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:41 felhom sudo[171052]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/chown 100000:100000 /mnt/hdd_1/felhom-data
Sep 14 21:17:41 felhom sudo[171052]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:41 felhom sudo[171052]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:41 felhom sudo[171055]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives/hdd_1
Sep 14 21:17:41 felhom sudo[171055]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:41 felhom sudo[171055]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:41 felhom sudo[171058]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:17:41 felhom sudo[171058]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:41 felhom sudo[171058]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:41 felhom felhom-agent[4303]: time=2026-09-14T21:17:41.722+02:00 level=INFO msg="reconcile: enrolled drive bound under shared parent (live, no reboot)" vmid=9201 where=/mnt/hdd_1 guest_path=/mnt/felhom-drives/hdd_1 durable_id=uuid:c4b530fd-d410-424c-917e-c8ecfdde9d8b
Sep 14 21:17:42 felhom sudo[171062]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nft list set inet felhom_oob ssh_port
Sep 14 21:17:42 felhom sudo[171062]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:42 felhom sudo[171062]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:42 felhom sudo[171065]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nft list set inet felhom_oob operator_ips
Sep 14 21:17:42 felhom sudo[171065]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:42 felhom sudo[171065]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:42 felhom sudo[171072]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/wg show wg-felhom latest-handshakes
Sep 14 21:17:42 felhom sudo[171072]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:42 felhom sudo[171072]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:42 felhom sudo[171029]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:42 felhom sudo[171085]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct exec 9201 -- ip route show default
Sep 14 21:17:42 felhom sudo[171085]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:43 felhom sudo[171085]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:43 felhom sudo[171121]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct exec 9201 -- cat /etc/network/interfaces
Sep 14 21:17:43 felhom sudo[171121]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:44 felhom sudo[171121]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:44 felhom sudo[171128]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct exec 9201 -- pgrep -x dhclient
Sep 14 21:17:44 felhom sudo[171128]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:45 felhom sudo[171128]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:46 felhom sudo[171283]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct exec 9201 -- ip -4 -o addr show dev eth0
Sep 14 21:17:46 felhom sudo[171283]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:47 felhom sudo[171283]: pam_unix(sudo:session): session closed for user root
Sep 14 21:17:47 felhom sudo[171335]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct exec 9201 -- docker exec felhom-controller cat /opt/docker/felhom-controller/controller.yaml
Sep 14 21:17:47 felhom sudo[171335]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:17:48 felhom sudo[171335]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171914]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:18:01 felhom sudo[171914]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171914]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171917]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:18:01 felhom sudo[171917]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171917]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171920]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives
Sep 14 21:18:01 felhom sudo[171920]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171920]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171927]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:18:01 felhom sudo[171927]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171927]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171948]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:18:01 felhom sudo[171948]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171948]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171952]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/hdd_1/felhom-data
Sep 14 21:18:01 felhom sudo[171952]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171952]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171955]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/chown 100000:100000 /mnt/hdd_1/felhom-data
Sep 14 21:18:01 felhom sudo[171955]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171955]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171959]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives/hdd_1
Sep 14 21:18:01 felhom sudo[171959]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171959]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171962]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:18:01 felhom sudo[171962]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171962]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom felhom-agent[4303]: time=2026-09-14T21:18:01.733+02:00 level=INFO msg="reconcile: enrolled drive bound under shared parent (live, no reboot)" vmid=9201 where=/mnt/hdd_1 guest_path=/mnt/felhom-drives/hdd_1 durable_id=uuid:c4b530fd-d410-424c-917e-c8ecfdde9d8b
Sep 14 21:18:01 felhom sudo[171968]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:18:01 felhom sudo[171968]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171968]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171971]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:18:01 felhom sudo[171971]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171971]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171974]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:18:01 felhom sudo[171974]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171974]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171977]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sdb
Sep 14 21:18:01 felhom sudo[171977]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171977]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:01 felhom sudo[171980]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:18:01 felhom sudo[171980]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:01 felhom sudo[171980]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:02 felhom sudo[171987]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
Sep 14 21:18:02 felhom sudo[171987]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:02 felhom sudo[171987]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:02 felhom sudo[171990]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
Sep 14 21:18:02 felhom sudo[171990]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:02 felhom sudo[171990]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:02 felhom sudo[171993]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:18:02 felhom sudo[171993]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:02 felhom sudo[171993]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:02 felhom sudo[171996]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sdb
Sep 14 21:18:02 felhom sudo[171996]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:02 felhom sudo[171996]: pam_unix(sudo:session): session closed for user root
Sep 14 21:18:02 felhom sudo[171999]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/lxc-info -n 9201 -p -H
Sep 14 21:18:02 felhom sudo[171999]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
Sep 14 21:18:02 felhom sudo[171999]: pam_unix(sudo:session): session closed for user root
documentation/audits/evidence-bignight-2026-09-14/box-logs-phase4/controller.log
+730
View File
@@ -0,0 +1,730 @@
2026/09/14 18:59:00 [INFO] [sync] Starting catalog sync
2026/09/14 18:59:00 [INFO] [sync] Pulling latest from https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git (branch: main)
2026/09/14 18:59:01 [INFO] [sync] Catalog sync complete
2026/09/14 18:59:01 [INFO] [sync] Periodic sync: Sablonok naprakészek — nincs változás
2026/09/14 18:59:01 [INFO] [scheduler] Running job: backup-cache
2026/09/14 18:59:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 18:59:01 [INFO] [scheduler] Running job: hub-report
2026/09/14 18:59:01 [INFO] [report] Building system report
2026/09/14 18:59:01 [INFO] [scheduler] Running job: system-health
2026/09/14 18:59:01 [INFO] [scheduler] Running job: offsite-credential-retry
2026/09/14 18:59:01 [INFO] [scheduler] Job offsite-credential-retry completed (took 0s)
2026/09/14 18:59:01 [INFO] [scheduler] Running job: disk-health-check
2026/09/14 18:59:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 18:59:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 18:59:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 18:59:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 18:59:01 [INFO] [scheduler] Job stack-scan completed (took 129ms)
2026/09/14 18:59:01 [INFO] [backup] Found 0 DB dump files across drives
2026/09/14 18:59:01 [INFO] [scheduler] Job agent-channel-health completed (took 275ms)
2026/09/14 18:59:01 [INFO] [monitor] Health check: status=ok
2026/09/14 18:59:01 [INFO] [scheduler] Job system-health completed (took 331ms)
2026/09/14 18:59:01 [INFO] [backup] Discovered 6 databases
2026/09/14 18:59:01 [INFO] [web] disk-health check complete: 0 disk(s) evaluated, 0 alert(s)
2026/09/14 18:59:01 [INFO] [scheduler] Job disk-health-check completed (took 356ms)
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/adventurelog/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/docmost/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/gokapi/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/jellyfin/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/mealie/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/uptime-kuma/docker-compose.yml
2026/09/14 18:59:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/vaultwarden/docker-compose.yml
2026/09/14 18:59:01 [INFO] [backup] Discovered app data: 12 apps
2026/09/14 18:59:01 [INFO] [backup] Backup status cache refreshed
2026/09/14 18:59:01 [INFO] [scheduler] Job backup-cache completed (took 445ms)
2026/09/14 18:59:01 [INFO] [monitor] Health check: status=ok
2026/09/14 18:59:02 [INFO] [report] Hub report pushed successfully (17815 bytes)
2026/09/14 18:59:02 [INFO] [settings] Settings saved
2026/09/14 18:59:02 [INFO] [scheduler] Job hub-report completed (took 1.235s)
2026/09/14 18:59:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 18:59:11 [INFO] Health probes: 1 ok (of 1 probed)
2026/09/14 18:59:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 18:59:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 18:59:31 [INFO] Health probes: 2 ok (of 2 probed)
2026/09/14 18:59:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 18:59:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 18:59:55 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/09/14 18:59:55 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/09/14 19:00:01 [INFO] [deadapp] check alive: 120 scans since boot, 12 deployed app(s) evaluated, 0 currently down
2026/09/14 19:00:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:00:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:00:01 [INFO] Health probes: 1 ok (of 1 probed)
2026/09/14 19:00:01 [INFO] [scheduler] Job agent-channel-health completed (took 197ms)
2026/09/14 19:00:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:00:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:00:26 [INFO] [api] Manual app-data backup (DB dump) triggered
2026/09/14 19:00:26 [INFO] [backup] Starting database dump run
2026/09/14 19:00:26 [INFO] [backup] Discovered 6 databases
2026/09/14 19:00:26 [INFO] [backup] Discovered 6 database(s): adventurelog-postgres(postgres), paperless-postgres(postgres), immich-postgres(postgres), nextcloud-db(mariadb), docmost-postgres(postgres), bookstack-db(mariadb)
2026/09/14 19:00:26 [INFO] [backup] DB dump: adventurelog-postgres → adventurelog-postgres.sql (152.6 KB, 413ms, 47 tables)
2026/09/14 19:00:26 [INFO] [settings] Settings saved
2026/09/14 19:00:27 [INFO] [backup] DB dump: paperless-postgres → paperless-ngx-postgres.sql (298.5 KB, 356ms, 72 tables)
2026/09/14 19:00:27 [INFO] [settings] Settings saved
2026/09/14 19:00:28 [INFO] [backup] DB dump: immich-postgres → immich-postgres.sql (51.0 MB, 831ms, 66 tables)
2026/09/14 19:00:28 [INFO] [settings] Settings saved
2026/09/14 19:00:28 [INFO] [backup] DB dump: nextcloud-db → nextcloud-mariadb.sql (1.1 MB, 596ms, 131 tables)
2026/09/14 19:00:28 [INFO] [settings] Settings saved
2026/09/14 19:00:29 [INFO] [backup] DB dump: docmost-postgres → docmost-postgres.sql (139.9 KB, 304ms, 42 tables)
2026/09/14 19:00:29 [INFO] [settings] Settings saved
2026/09/14 19:00:29 [INFO] [backup] DB dump: bookstack-db → bookstack-mariadb.sql (66.4 KB, 297ms, 41 tables)
2026/09/14 19:00:29 [INFO] [settings] Settings saved
2026/09/14 19:00:29 [INFO] [backup] Stopping adventurelog for safe volume dump
2026/09/14 19:00:29 [INFO] [stacks] Stopping stack: adventurelog
2026/09/14 19:00:31 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:00:32 [INFO] [stacks] Stack adventurelog stopped successfully (took 3.0s)
2026/09/14 19:00:32 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:00:34 [INFO] [backup] Volume dump: adventurelog/adventurelog_adventurelog_media → 2.0 KB
2026/09/14 19:00:35 [INFO] [backup] Volume dump: adventurelog/adventurelog_adventurelog_postgres_data → 112.0 MB
2026/09/14 19:00:35 [INFO] [backup] Restarting adventurelog after volume dump
2026/09/14 19:00:35 [INFO] [stacks] Starting stack: adventurelog
2026/09/14 19:00:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:00:41 [INFO] [stacks] Stack adventurelog started successfully (took 6.3s)
2026/09/14 19:00:42 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:00:42 [INFO] [backup] Stopping bookstack for safe volume dump
2026/09/14 19:00:42 [INFO] [stacks] Stopping stack: bookstack
2026/09/14 19:00:45 [INFO] [stacks] Stack adventurelog post-start status:
2026/09/14 19:00:45 [INFO] [stacks] adventurelog ghcr.io/seanmorley15/adventurelog-backend:v0.12.1 running Up 3 seconds (health: starting)
2026/09/14 19:00:45 [INFO] [stacks] adventurelog-frontend ghcr.io/seanmorley15/adventurelog-frontend:v0.12.1 running Up 9 seconds (healthy)
2026/09/14 19:00:45 [INFO] [stacks] adventurelog-postgres postgis/postgis:16-3.5-alpine running Up 9 seconds (healthy)
2026/09/14 19:00:47 [INFO] [stacks] Stack bookstack stopped successfully (took 4.9s)
2026/09/14 19:00:47 [INFO] [stacks] Status refresh: 26 containers across 57 stacks
2026/09/14 19:00:47 [INFO] [backup] Volume dump: bookstack/bookstack_bookstack_config → 1.4 MB
2026/09/14 19:00:48 [INFO] [backup] Volume dump: bookstack/bookstack_bookstack_db_data → 153.4 MB
2026/09/14 19:00:48 [INFO] [backup] Restarting bookstack after volume dump
2026/09/14 19:00:48 [INFO] [stacks] Starting stack: bookstack
2026/09/14 19:00:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:00:54 [INFO] [stacks] Stack bookstack started successfully (took 6.2s)
2026/09/14 19:00:54 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:00:54 [INFO] [backup] Stopping docmost for safe volume dump
2026/09/14 19:00:54 [INFO] [stacks] Stopping stack: docmost
2026/09/14 19:00:56 [INFO] [stacks] Stack docmost stopped successfully (took 1.1s)
2026/09/14 19:00:56 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:00:56 [INFO] [backup] Volume dump: docmost/docmost_docmost_storage → 1.5 KB
2026/09/14 19:00:57 [INFO] [backup] Volume dump: docmost/docmost_docmost_postgres_data → 49.9 MB
2026/09/14 19:00:57 [INFO] [backup] Volume dump: docmost/docmost_docmost_redis_data → 219.5 KB
2026/09/14 19:00:57 [INFO] [backup] Restarting docmost after volume dump
2026/09/14 19:00:57 [INFO] [stacks] Starting stack: docmost
2026/09/14 19:00:58 [INFO] [stacks] Stack bookstack post-start status:
2026/09/14 19:00:58 [INFO] [stacks] bookstack lscr.io/linuxserver/bookstack:26.05.2 running Up 3 seconds (health: starting)
2026/09/14 19:00:58 [INFO] [stacks] bookstack-db mariadb:12.3 running Up 9 seconds (healthy)
2026/09/14 19:01:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:01:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:01:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:01:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:01 [INFO] [scheduler] Job stack-scan completed (took 125ms)
2026/09/14 19:01:01 [INFO] Health probes: 2 ok (of 2 probed)
2026/09/14 19:01:01 [INFO] [scheduler] Job agent-channel-health completed (took 214ms)
2026/09/14 19:01:08 [INFO] [stacks] Stack docmost started successfully (took 11.2s)
2026/09/14 19:01:09 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:09 [INFO] [backup] Stopping gokapi for safe volume dump
2026/09/14 19:01:09 [INFO] [stacks] Stopping stack: gokapi
2026/09/14 19:01:09 [INFO] [stacks] Stack gokapi stopped successfully (took 0.3s)
2026/09/14 19:01:09 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:01:09 [INFO] [backup] Volume dump: gokapi/gokapi_gokapi_config → 3.5 KB
2026/09/14 19:01:10 [INFO] [backup] Volume dump: gokapi/gokapi_gokapi_data → 51.4 MB
2026/09/14 19:01:10 [INFO] [backup] Restarting gokapi after volume dump
2026/09/14 19:01:10 [INFO] [stacks] Starting stack: gokapi
2026/09/14 19:01:10 [INFO] [stacks] Stack gokapi started successfully (took 0.4s)
2026/09/14 19:01:10 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:10 [INFO] [backup] Stopping immich for safe volume dump
2026/09/14 19:01:10 [INFO] [stacks] Stopping stack: immich
2026/09/14 19:01:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:11 [ERROR] [stacks] execCommand failed: exit status 1
2026/09/14 19:01:11 [WARN] [stacks] restart-policy inspect failed for container "immich-machine-learning": exec docker inspect -f {{.HostConfig.RestartPolicy.Name}} immich-machine-learning: exit status 1
stderr: error: no such object: immich-machine-learning
(treating as supervised)
2026/09/14 19:01:11 [INFO] Health probes: 1 ok (of 1 probed)
2026/09/14 19:01:11 [INFO] [stacks] Stack immich stopped successfully (took 0.9s)
2026/09/14 19:01:11 [INFO] [stacks] Status refresh: 24 containers across 57 stacks
2026/09/14 19:01:12 [INFO] [stacks] Stack docmost post-start status:
2026/09/14 19:01:12 [INFO] [stacks] docmost docmost/docmost:0.95.0 running Up 3 seconds (health: starting)
2026/09/14 19:01:12 [INFO] [stacks] docmost-postgres postgres:16-alpine running Up 14 seconds (healthy)
2026/09/14 19:01:12 [INFO] [stacks] docmost-redis redis:7-alpine running Up 14 seconds (healthy)
2026/09/14 19:01:13 [INFO] [stacks] Stack gokapi post-start status:
2026/09/14 19:01:13 [INFO] [stacks] gokapi f0rc3/gokapi:v1.9.6 running Up 3 seconds (health: starting)
2026/09/14 19:01:14 [INFO] [backup] Volume dump: immich/immich_immich_ml_cache → 785.5 MB
2026/09/14 19:01:17 [INFO] [backup] Volume dump: immich/immich_immich_postgres_data → 328.1 MB
2026/09/14 19:01:17 [INFO] [backup] Volume dump: immich/immich_immich_redis_data → 3.2 MB
2026/09/14 19:01:17 [INFO] [backup] Restarting immich after volume dump
2026/09/14 19:01:17 [INFO] [stacks] Starting stack: immich
2026/09/14 19:01:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:28 [INFO] [stacks] Stack immich started successfully (took 11.0s)
2026/09/14 19:01:28 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:28 [INFO] [backup] Stopping jellyfin for safe volume dump
2026/09/14 19:01:28 [INFO] [stacks] Stopping stack: jellyfin
2026/09/14 19:01:29 [INFO] [stacks] Stack jellyfin stopped successfully (took 0.4s)
2026/09/14 19:01:29 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:01:29 [INFO] [backup] Volume dump: jellyfin/jellyfin_jellyfin_cache → 3.0 KB
2026/09/14 19:01:30 [INFO] [backup] Volume dump: jellyfin/jellyfin_jellyfin_config → 594.0 KB
2026/09/14 19:01:30 [INFO] [backup] Restarting jellyfin after volume dump
2026/09/14 19:01:30 [INFO] [stacks] Starting stack: jellyfin
2026/09/14 19:01:30 [INFO] [stacks] Stack jellyfin started successfully (took 0.4s)
2026/09/14 19:01:30 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:30 [INFO] [backup] Stopping mealie for safe volume dump
2026/09/14 19:01:30 [INFO] [stacks] Stopping stack: mealie
2026/09/14 19:01:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:31 [INFO] Health probes: 3 ok (of 3 probed)
2026/09/14 19:01:31 [INFO] [stacks] Stack immich post-start status:
2026/09/14 19:01:31 [INFO] [stacks] immich-machine-learning ghcr.io/immich-app/immich-machine-learning:v3.0.3 running Up 13 seconds (healthy)
2026/09/14 19:01:31 [INFO] [stacks] immich-postgres ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0 running Up 13 seconds (healthy)
2026/09/14 19:01:31 [INFO] [stacks] immich-redis redis:7-alpine running Up 13 seconds (healthy)
2026/09/14 19:01:31 [INFO] [stacks] immich-server ghcr.io/immich-app/immich-server:v3.0.3 running Up 3 seconds (health: starting)
2026/09/14 19:01:33 [INFO] [stacks] Stack jellyfin post-start status:
2026/09/14 19:01:33 [INFO] [stacks] jellyfin jellyfin/jellyfin:10.11.11 running Up 3 seconds (health: starting)
2026/09/14 19:01:34 [INFO] [stacks] Stack mealie stopped successfully (took 3.3s)
2026/09/14 19:01:34 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:01:34 [INFO] [backup] Volume dump: mealie/mealie_mealie_data → 1.4 MB
2026/09/14 19:01:34 [INFO] [backup] Restarting mealie after volume dump
2026/09/14 19:01:34 [INFO] [stacks] Starting stack: mealie
2026/09/14 19:01:35 [INFO] [stacks] Stack mealie started successfully (took 0.4s)
2026/09/14 19:01:35 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:01:35 [INFO] [backup] Stopping nextcloud for safe volume dump
2026/09/14 19:01:35 [INFO] [stacks] Stopping stack: nextcloud
2026/09/14 19:01:37 [INFO] [stacks] Stack nextcloud stopped successfully (took 2.5s)
2026/09/14 19:01:37 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:01:38 [INFO] [stacks] Stack mealie post-start status:
2026/09/14 19:01:38 [INFO] [stacks] mealie ghcr.io/mealie-recipes/mealie:v3.20.1 running Up 3 seconds (health: starting)
2026/09/14 19:01:38 [INFO] [backup] Volume dump: nextcloud/nextcloud_nextcloud_db_data → 173.5 MB
2026/09/14 19:01:41 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:01:51 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:01:56 [INFO] [backup] Volume dump: nextcloud/nextcloud_nextcloud_html → 745.1 MB
2026/09/14 19:01:56 [INFO] [backup] Volume dump: nextcloud/nextcloud_nextcloud_redis_data → 2.6 MB
2026/09/14 19:01:56 [INFO] [backup] Restarting nextcloud after volume dump
2026/09/14 19:01:56 [INFO] [stacks] Starting stack: nextcloud
2026/09/14 19:02:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:02:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:01 [INFO] [scheduler] Job agent-channel-health completed (took 197ms)
2026/09/14 19:02:03 [INFO] [stacks] Stack nextcloud started successfully (took 6.4s)
2026/09/14 19:02:03 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:03 [INFO] [backup] Stopping paperless-ngx for safe volume dump
2026/09/14 19:02:03 [INFO] [stacks] Stopping stack: paperless-ngx
2026/09/14 19:02:06 [INFO] [stacks] Stack nextcloud post-start status:
2026/09/14 19:02:06 [INFO] [stacks] nextcloud nextcloud:34.0.1-apache running Up 3 seconds (health: starting)
2026/09/14 19:02:06 [INFO] [stacks] nextcloud-db mariadb:11.6 running Up 9 seconds (healthy)
2026/09/14 19:02:06 [INFO] [stacks] nextcloud-redis redis:7-alpine running Up 9 seconds (healthy)
2026/09/14 19:02:11 [INFO] [stacks] Status refresh: 26 containers across 57 stacks
2026/09/14 19:02:11 [INFO] Health probes: 4 ok (of 4 probed)
2026/09/14 19:02:11 [INFO] [stacks] Stack paperless-ngx stopped successfully (took 7.9s)
2026/09/14 19:02:11 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:02:11 [INFO] [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_data → 126.0 KB
2026/09/14 19:02:12 [INFO] [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_postgres_data → 68.2 MB
2026/09/14 19:02:12 [INFO] [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_redis_data → 135.0 KB
2026/09/14 19:02:12 [INFO] [backup] Restarting paperless-ngx after volume dump
2026/09/14 19:02:12 [INFO] [stacks] Starting stack: paperless-ngx
2026/09/14 19:02:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:24 [INFO] [stacks] Stack paperless-ngx started successfully (took 11.2s)
2026/09/14 19:02:24 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:24 [INFO] [backup] Stopping privatebin for safe volume dump
2026/09/14 19:02:24 [INFO] [stacks] Stopping stack: privatebin
2026/09/14 19:02:24 [INFO] [stacks] Stack privatebin stopped successfully (took 0.6s)
2026/09/14 19:02:25 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:02:25 [INFO] [backup] Volume dump: privatebin/privatebin_privatebin_data → 18.5 KB
2026/09/14 19:02:25 [INFO] [backup] Restarting privatebin after volume dump
2026/09/14 19:02:25 [INFO] [stacks] Starting stack: privatebin
2026/09/14 19:02:26 [INFO] [stacks] Stack privatebin started successfully (took 0.5s)
2026/09/14 19:02:26 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:26 [INFO] [backup] Stopping uptime-kuma for safe volume dump
2026/09/14 19:02:26 [INFO] [stacks] Stopping stack: uptime-kuma
2026/09/14 19:02:27 [INFO] [stacks] Stack paperless-ngx post-start status:
2026/09/14 19:02:27 [INFO] [stacks] paperless-postgres postgres:16-alpine running Up 14 seconds (healthy)
2026/09/14 19:02:27 [INFO] [stacks] paperless-redis redis:7-alpine running Up 14 seconds (healthy)
2026/09/14 19:02:27 [INFO] [stacks] paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 running Up 3 seconds (health: starting)
2026/09/14 19:02:29 [INFO] [stacks] Stack privatebin post-start status:
2026/09/14 19:02:29 [INFO] [stacks] privatebin privatebin/pdo:2.0.5 running Up 3 seconds (health: starting)
2026/09/14 19:02:31 [INFO] [stacks] Stack uptime-kuma stopped successfully (took 4.5s)
2026/09/14 19:02:31 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:02:31 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:02:31 [INFO] [backup] Volume dump: uptime-kuma/uptime-kuma_uptime_kuma_data → 356.5 KB
2026/09/14 19:02:31 [INFO] [backup] Restarting uptime-kuma after volume dump
2026/09/14 19:02:31 [INFO] [stacks] Starting stack: uptime-kuma
2026/09/14 19:02:32 [INFO] [stacks] Stack uptime-kuma started successfully (took 0.4s)
2026/09/14 19:02:32 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:32 [INFO] [backup] Stopping vaultwarden for safe volume dump
2026/09/14 19:02:32 [INFO] [stacks] Stopping stack: vaultwarden
2026/09/14 19:02:32 [INFO] [stacks] Stack vaultwarden stopped successfully (took 0.4s)
2026/09/14 19:02:32 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:02:33 [INFO] [backup] Volume dump: vaultwarden/vaultwarden_vaultwarden_data → 739.0 KB
2026/09/14 19:02:33 [INFO] [backup] Restarting vaultwarden after volume dump
2026/09/14 19:02:33 [INFO] [stacks] Starting stack: vaultwarden
2026/09/14 19:02:33 [INFO] [stacks] Stack vaultwarden started successfully (took 0.5s)
2026/09/14 19:02:34 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:34 [INFO] [backup] App-data backup completed: 6 databases (52.7 MB total), 12 volume dump(s) (2m8.275s)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for adventurelog → /mnt/sys_drive/felhom-data/backups/primary/adventurelog (images=3, secrets-referenced=2, data_keys=1, portable-carried=2/2, withheld=0)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for bookstack → /mnt/sys_drive/felhom-data/backups/primary/bookstack (images=2, secrets-referenced=2, data_keys=0, portable-carried=2/2, withheld=0)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for docmost → /mnt/sys_drive/felhom-data/backups/primary/docmost (images=3, secrets-referenced=2, data_keys=0, portable-carried=2/2, withheld=0)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for gokapi → /mnt/sys_drive/felhom-data/backups/primary/gokapi (images=1, secrets-referenced=1, data_keys=0, portable-carried=0/0, withheld=1)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for immich → /mnt/felhom-drives/hdd_1/backups/primary/immich (images=4, secrets-referenced=1, data_keys=0, portable-carried=1/1, withheld=0)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for jellyfin → /mnt/felhom-drives/hdd_1/backups/primary/jellyfin (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for mealie → /mnt/sys_drive/felhom-data/backups/primary/mealie (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for nextcloud → /mnt/felhom-drives/hdd_1/backups/primary/nextcloud (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for paperless-ngx → /mnt/felhom-drives/hdd_1/backups/primary/paperless-ngx (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for privatebin → /mnt/sys_drive/felhom-data/backups/primary/privatebin (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for uptime-kuma → /mnt/sys_drive/felhom-data/backups/primary/uptime-kuma (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/09/14 19:02:34 [INFO] [backup] Recovery unit captured for vaultwarden → /mnt/sys_drive/felhom-data/backups/primary/vaultwarden (images=1, secrets-referenced=1, data_keys=0, portable-carried=0/0, withheld=1)
2026/09/14 19:02:35 [INFO] [stacks] Stack uptime-kuma post-start status:
2026/09/14 19:02:35 [INFO] [stacks] uptime-kuma louislam/uptime-kuma:2.4.0 running Up 3 seconds (health: starting)
2026/09/14 19:02:37 [INFO] [stacks] Stack vaultwarden post-start status:
2026/09/14 19:02:37 [INFO] [stacks] vaultwarden vaultwarden/server:1.36.0-alpine running Up 3 seconds (health: starting)
2026/09/14 19:02:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:41 [INFO] Health probes: 3 ok (of 3 probed)
2026/09/14 19:02:49 [INFO] [api] Manual Tier 2 (off-drive) backup triggered
2026/09/14 19:02:49 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/adventurelog/docker-compose.yml
2026/09/14 19:02:49 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/adventurelog/docker-compose.yml
2026/09/14 19:02:50 [INFO] [settings] Settings saved
2026/09/14 19:02:50 [INFO] [backup] Tier 2 copied adventurelog → /mnt/felhom-drives/hdd_1/backups/secondary/adventurelog (112.2 MB, 0 leg(s), 1s)
2026/09/14 19:02:50 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/09/14 19:02:50 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/09/14 19:02:50 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: adventurelog
2026/09/14 19:02:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:02:53 [INFO] [settings] Settings saved
2026/09/14 19:02:53 [INFO] [backup] Tier 2 copied bookstack → /mnt/felhom-drives/hdd_1/backups/secondary/bookstack (154.9 MB, 0 leg(s), 3s)
2026/09/14 19:02:53 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/docmost/docker-compose.yml
2026/09/14 19:02:53 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/docmost/docker-compose.yml
2026/09/14 19:02:53 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: bookstack
2026/09/14 19:02:53 [INFO] [settings] Settings saved
2026/09/14 19:02:53 [INFO] [backup] Tier 2 copied docmost → /mnt/felhom-drives/hdd_1/backups/secondary/docmost (50.3 MB, 0 leg(s), 0s)
2026/09/14 19:02:53 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/gokapi/docker-compose.yml
2026/09/14 19:02:53 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/gokapi/docker-compose.yml
2026/09/14 19:02:53 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: docmost
2026/09/14 19:02:53 [INFO] [settings] Settings saved
2026/09/14 19:02:53 [INFO] [backup] Tier 2 copied gokapi → /mnt/felhom-drives/hdd_1/backups/secondary/gokapi (51.4 MB, 0 leg(s), 0s)
2026/09/14 19:02:53 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: gokapi
2026/09/14 19:03:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:03:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:03:01 [INFO] [settings] Settings saved
2026/09/14 19:03:01 [INFO] [backup] Tier 2 copied immich → /mnt/sys_drive/felhom-data/backups/secondary/immich (1.7 GB, 1 leg(s), 7s) [SSD: state-only]
2026/09/14 19:03:01 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: immich
2026/09/14 19:03:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:03:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:03:01 [INFO] [scheduler] Job stack-scan completed (took 276ms)
2026/09/14 19:03:01 [INFO] [scheduler] Job agent-channel-health completed (took 296ms)
2026/09/14 19:03:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:03:01 [INFO] [settings] Settings saved
2026/09/14 19:03:01 [INFO] [backup] Tier 2 copied jellyfin → /mnt/sys_drive/felhom-data/backups/secondary/jellyfin (614.7 KB, 0 leg(s), 0s) [SSD: state-only]
2026/09/14 19:03:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/mealie/docker-compose.yml
2026/09/14 19:03:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/mealie/docker-compose.yml
2026/09/14 19:03:01 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: jellyfin
2026/09/14 19:03:01 [INFO] [settings] Settings saved
2026/09/14 19:03:01 [INFO] [backup] Tier 2 copied mealie → /mnt/felhom-drives/hdd_1/backups/secondary/mealie (1.4 MB, 0 leg(s), 0s)
2026/09/14 19:03:01 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: mealie
2026/09/14 19:03:05 [INFO] [settings] Settings saved
2026/09/14 19:03:05 [INFO] [backup] Tier 2 copied nextcloud → /mnt/sys_drive/felhom-data/backups/secondary/nextcloud (1.3 GB, 1 leg(s), 4s) [SSD: state-only]
2026/09/14 19:03:05 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: nextcloud
2026/09/14 19:03:05 [INFO] [settings] Settings saved
2026/09/14 19:03:05 [INFO] [backup] Tier 2 copied paperless-ngx → /mnt/sys_drive/felhom-data/backups/secondary/paperless-ngx (68.8 MB, 1 leg(s), 0s) [SSD: state-only]
2026/09/14 19:03:05 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/09/14 19:03:05 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/09/14 19:03:05 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: paperless-ngx
2026/09/14 19:03:05 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/uptime-kuma/docker-compose.yml
2026/09/14 19:03:05 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/uptime-kuma/docker-compose.yml
2026/09/14 19:03:05 [INFO] [settings] Settings saved
2026/09/14 19:03:05 [INFO] [backup] Tier 2 copied privatebin → /mnt/felhom-drives/hdd_1/backups/secondary/privatebin (34.9 KB, 0 leg(s), 0s)
2026/09/14 19:03:05 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: privatebin
2026/09/14 19:03:06 [INFO] [settings] Settings saved
2026/09/14 19:03:06 [INFO] [backup] Tier 2 copied uptime-kuma → /mnt/felhom-drives/hdd_1/backups/secondary/uptime-kuma (373.4 KB, 0 leg(s), 0s)
2026/09/14 19:03:06 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/vaultwarden/docker-compose.yml
2026/09/14 19:03:06 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/vaultwarden/docker-compose.yml
2026/09/14 19:03:06 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: uptime-kuma
2026/09/14 19:03:06 [INFO] [settings] Settings saved
2026/09/14 19:03:06 [INFO] [backup] Tier 2 copied vaultwarden → /mnt/felhom-drives/hdd_1/backups/secondary/vaultwarden (759.3 KB, 0 leg(s), 0s)
2026/09/14 19:03:06 [INFO] [backup] Tier 2 run complete: 12 app(s) processed (incl. volume-only — F6)
2026/09/14 19:03:06 [WARN] [shares] tier-2: share beolvasas is not under a registered storage root — skipped
2026/09/14 19:03:06 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: vaultwarden
2026/09/14 19:03:06 [INFO] [shares] payload staged: 2 share definition(s), credential copy=true
2026/09/14 19:03:06 [INFO] [shares] tier-2 copied 1 share(s) from /mnt/felhom-drives/hdd_1 → /mnt/sys_drive/felhom-data/backups/secondary/_shares (94.3 MB)
2026/09/14 19:03:06 [INFO] [settings] Settings saved
2026/09/14 19:03:06 [INFO] [shares] tier-2 run complete: 1 share leg(s), 94.3 MB, 0s
2026/09/14 19:03:06 [INFO] Event pushed: crossdrive_completed (info) — Másodlagos mentés elkészült: Megosztások
2026/09/14 19:03:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:03:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:03:23 [INFO] [web] manual whole-guest backup triggered (quiesce loop)
2026/09/14 19:03:23 [INFO] [quiesce] manual backup requested — quiescing now
2026/09/14 19:03:23 [INFO] [quiesce] backup due on 2 tier(s) — quiescing 12 stack(s): [adventurelog bookstack docmost gokapi immich jellyfin mealie nextcloud paperless-ngx privatebin uptime-kuma vaultwarden]
2026/09/14 19:03:23 [INFO] [stacks] Stopping stack: adventurelog
2026/09/14 19:03:27 [INFO] [stacks] Stack adventurelog stopped successfully (took 4.0s)
2026/09/14 19:03:27 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:03:27 [INFO] [stacks] Stopping stack: bookstack
2026/09/14 19:03:30 [INFO] [stacks] Stack bookstack stopped successfully (took 3.8s)
2026/09/14 19:03:30 [INFO] [stacks] Status refresh: 23 containers across 57 stacks
2026/09/14 19:03:30 [INFO] [stacks] Stopping stack: docmost
2026/09/14 19:03:31 [INFO] [stacks] Status refresh: 23 containers across 57 stacks
2026/09/14 19:03:32 [INFO] [stacks] Stack docmost stopped successfully (took 1.1s)
2026/09/14 19:03:32 [INFO] [stacks] Status refresh: 20 containers across 57 stacks
2026/09/14 19:03:32 [INFO] [stacks] Stopping stack: gokapi
2026/09/14 19:03:32 [INFO] [stacks] Stack gokapi stopped successfully (took 0.2s)
2026/09/14 19:03:32 [INFO] [stacks] Status refresh: 19 containers across 57 stacks
2026/09/14 19:03:32 [INFO] [stacks] Stopping stack: immich
2026/09/14 19:03:33 [INFO] [stacks] Stack immich stopped successfully (took 0.8s)
2026/09/14 19:03:33 [INFO] [stacks] Status refresh: 15 containers across 57 stacks
2026/09/14 19:03:33 [INFO] [stacks] Stopping stack: jellyfin
2026/09/14 19:03:33 [INFO] [stacks] Stack jellyfin stopped successfully (took 0.5s)
2026/09/14 19:03:33 [INFO] [stacks] Status refresh: 14 containers across 57 stacks
2026/09/14 19:03:33 [INFO] [stacks] Stopping stack: mealie
2026/09/14 19:03:35 [INFO] [stacks] Stack mealie stopped successfully (took 2.0s)
2026/09/14 19:03:35 [INFO] [stacks] Status refresh: 13 containers across 57 stacks
2026/09/14 19:03:35 [INFO] [stacks] Stopping stack: nextcloud
2026/09/14 19:03:37 [INFO] [stacks] Stack nextcloud stopped successfully (took 1.8s)
2026/09/14 19:03:37 [INFO] [stacks] Status refresh: 10 containers across 57 stacks
2026/09/14 19:03:37 [INFO] [stacks] Stopping stack: paperless-ngx
2026/09/14 19:03:41 [INFO] [stacks] Status refresh: 10 containers across 57 stacks
2026/09/14 19:03:44 [INFO] [stacks] Stack paperless-ngx stopped successfully (took 6.9s)
2026/09/14 19:03:44 [INFO] [stacks] Status refresh: 7 containers across 57 stacks
2026/09/14 19:03:44 [INFO] [stacks] Stopping stack: privatebin
2026/09/14 19:03:44 [INFO] [stacks] Stack privatebin stopped successfully (took 0.3s)
2026/09/14 19:03:44 [INFO] [stacks] Status refresh: 6 containers across 57 stacks
2026/09/14 19:03:44 [INFO] [stacks] Stopping stack: uptime-kuma
2026/09/14 19:03:49 [INFO] [stacks] Stack uptime-kuma stopped successfully (took 4.3s)
2026/09/14 19:03:49 [INFO] [stacks] Status refresh: 5 containers across 57 stacks
2026/09/14 19:03:49 [INFO] [stacks] Stopping stack: vaultwarden
2026/09/14 19:03:49 [INFO] [stacks] Stack vaultwarden stopped successfully (took 0.3s)
2026/09/14 19:03:49 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:03:49 [INFO] [quiesce] tier local: backup job backup-9201-1789412629454125782 started — polling
2026/09/14 19:03:51 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:04:00 [INFO] [quiesce] a backup cycle is already running — skipping this scheduled check
2026/09/14 19:04:01 [INFO] [scheduler] Running job: backup-cache
2026/09/14 19:04:01 [INFO] [scheduler] Running job: system-health
2026/09/14 19:04:01 [INFO] [scheduler] Running job: offsite-credential-retry
2026/09/14 19:04:01 [INFO] [scheduler] Job offsite-credential-retry completed (took 0s)
2026/09/14 19:04:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:04:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:04:01 [INFO] [backup] Found 6 DB dump files across drives
2026/09/14 19:04:01 [INFO] [backup] Discovered 0 databases
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/adventurelog/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/docmost/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/gokapi/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/jellyfin/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/mealie/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/uptime-kuma/docker-compose.yml
2026/09/14 19:04:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/vaultwarden/docker-compose.yml
2026/09/14 19:04:01 [INFO] [backup] Discovered app data: 12 apps
2026/09/14 19:04:01 [INFO] [backup] Backup status cache refreshed
2026/09/14 19:04:01 [INFO] [scheduler] Job backup-cache completed (took 181ms)
2026/09/14 19:04:01 [INFO] [scheduler] Job agent-channel-health completed (took 242ms)
2026/09/14 19:04:01 [INFO] [monitor] Health check: status=ok
2026/09/14 19:04:01 [INFO] [scheduler] Job system-health completed (took 332ms)
2026/09/14 19:04:11 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:04:21 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:04:31 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:04:41 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:04:51 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:05:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:05:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:05:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:05:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:05:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:05:01 [INFO] [scheduler] Job stack-scan completed (took 109ms)
2026/09/14 19:05:01 [INFO] [scheduler] Job agent-channel-health completed (took 202ms)
2026/09/14 19:05:11 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:05:21 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:05:31 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:05:41 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:05:51 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:06:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:06:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:06:01 [INFO] [scheduler] Job agent-channel-health completed (took 206ms)
2026/09/14 19:06:11 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:06:21 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:06:31 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:06:41 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:06:51 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:07:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:07:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:07:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:07:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:07:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:07:01 [INFO] [scheduler] Job stack-scan completed (took 83ms)
2026/09/14 19:07:01 [INFO] [scheduler] Job agent-channel-health completed (took 188ms)
2026/09/14 19:07:11 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:07:21 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:07:31 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:07:41 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:07:51 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:08:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:08:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:08:01 [INFO] [scheduler] Job agent-channel-health completed (took 212ms)
2026/09/14 19:08:11 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:08:21 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:08:31 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:08:41 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:08:51 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:09:00 [INFO] [quiesce] a backup cycle is already running — skipping this scheduled check
2026/09/14 19:09:01 [INFO] [scheduler] Running job: backup-cache
2026/09/14 19:09:01 [INFO] [scheduler] Running job: offsite-credential-retry
2026/09/14 19:09:01 [INFO] [scheduler] Job offsite-credential-retry completed (took 0s)
2026/09/14 19:09:01 [INFO] [scheduler] Running job: system-health
2026/09/14 19:09:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:09:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:09:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:09:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:09:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:09:01 [INFO] [scheduler] Job stack-scan completed (took 123ms)
2026/09/14 19:09:01 [INFO] [backup] Found 6 DB dump files across drives
2026/09/14 19:09:01 [INFO] [backup] Discovered 0 databases
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/adventurelog/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/docmost/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/gokapi/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
2026/09/14 19:09:01 [INFO] [scheduler] Job agent-channel-health completed (took 265ms)
2026/09/14 19:09:01 [INFO] [monitor] Health check: status=ok
2026/09/14 19:09:01 [INFO] [scheduler] Job system-health completed (took 375ms)
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/jellyfin/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/mealie/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/uptime-kuma/docker-compose.yml
2026/09/14 19:09:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/vaultwarden/docker-compose.yml
2026/09/14 19:09:01 [INFO] [backup] Discovered app data: 12 apps
2026/09/14 19:09:01 [INFO] [backup] Backup status cache refreshed
2026/09/14 19:09:01 [INFO] [scheduler] Job backup-cache completed (took 522ms)
2026/09/14 19:09:11 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:09:21 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:09:31 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:09:41 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:09:51 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:09:59 [INFO] [quiesce] tier local: backup job backup-9201-1789412629454125782 done — next tier may start (app still quiesced)
2026/09/14 19:09:59 [INFO] [quiesce] tier felhom-pbs: backup job backup-9201-felhom-pbs-1789412999526947801 started — polling
2026/09/14 19:10:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:10:01 [INFO] [stacks] Status refresh: 4 containers across 57 stacks
2026/09/14 19:10:01 [INFO] [deadapp] check alive: 140 scans since boot, 12 deployed app(s) evaluated, 0 currently down
2026/09/14 19:10:01 [INFO] [scheduler] Job agent-channel-health completed (took 222ms)
2026/09/14 19:10:09 [INFO] [quiesce] unquiescing (backup failed): restarting 12 stack(s)
2026/09/14 19:10:09 [INFO] [stacks] Starting stack: adventurelog
2026/09/14 19:10:11 [INFO] [stacks] Status refresh: 7 containers across 57 stacks
2026/09/14 19:10:16 [INFO] [stacks] Stack adventurelog started successfully (took 6.6s)
2026/09/14 19:10:16 [INFO] [stacks] Status refresh: 7 containers across 57 stacks
2026/09/14 19:10:16 [INFO] [stacks] Starting stack: bookstack
2026/09/14 19:10:19 [INFO] [stacks] Stack adventurelog post-start status:
2026/09/14 19:10:19 [INFO] [stacks] adventurelog ghcr.io/seanmorley15/adventurelog-backend:v0.12.1 running Up 3 seconds (health: starting)
2026/09/14 19:10:19 [INFO] [stacks] adventurelog-frontend ghcr.io/seanmorley15/adventurelog-frontend:v0.12.1 running Up 9 seconds (healthy)
2026/09/14 19:10:19 [INFO] [stacks] adventurelog-postgres postgis/postgis:16-3.5-alpine running Up 9 seconds (healthy)
2026/09/14 19:10:21 [INFO] [stacks] Status refresh: 9 containers across 57 stacks
2026/09/14 19:10:22 [INFO] [stacks] Stack bookstack started successfully (took 6.4s)
2026/09/14 19:10:22 [INFO] [stacks] Status refresh: 9 containers across 57 stacks
2026/09/14 19:10:22 [INFO] [stacks] Starting stack: docmost
2026/09/14 19:10:26 [INFO] [stacks] Stack bookstack post-start status:
2026/09/14 19:10:26 [INFO] [stacks] bookstack lscr.io/linuxserver/bookstack:26.05.2 running Up 3 seconds (health: starting)
2026/09/14 19:10:26 [INFO] [stacks] bookstack-db mariadb:12.3 running Up 9 seconds (healthy)
2026/09/14 19:10:31 [INFO] [stacks] Status refresh: 12 containers across 57 stacks
2026/09/14 19:10:34 [INFO] [stacks] Stack docmost started successfully (took 11.7s)
2026/09/14 19:10:34 [INFO] [stacks] Status refresh: 12 containers across 57 stacks
2026/09/14 19:10:34 [INFO] [stacks] Starting stack: gokapi
2026/09/14 19:10:35 [INFO] [stacks] Stack gokapi started successfully (took 0.5s)
2026/09/14 19:10:35 [INFO] [stacks] Status refresh: 13 containers across 57 stacks
2026/09/14 19:10:35 [INFO] [stacks] Starting stack: immich
2026/09/14 19:10:37 [INFO] [stacks] Stack docmost post-start status:
2026/09/14 19:10:37 [INFO] [stacks] docmost docmost/docmost:0.95.0 running Up 3 seconds (health: starting)
2026/09/14 19:10:37 [INFO] [stacks] docmost-postgres postgres:16-alpine running Up 14 seconds (healthy)
2026/09/14 19:10:37 [INFO] [stacks] docmost-redis redis:7-alpine running Up 14 seconds (healthy)
2026/09/14 19:10:38 [INFO] [stacks] Stack gokapi post-start status:
2026/09/14 19:10:38 [INFO] [stacks] gokapi f0rc3/gokapi:v1.9.6 running Up 3 seconds (health: starting)
2026/09/14 19:10:41 [INFO] [stacks] Status refresh: 17 containers across 57 stacks
2026/09/14 19:10:41 [INFO] Health probes: 2 ok (of 2 probed)
2026/09/14 19:10:46 [INFO] [stacks] Stack immich started successfully (took 11.3s)
2026/09/14 19:10:47 [INFO] [stacks] Status refresh: 17 containers across 57 stacks
2026/09/14 19:10:47 [INFO] [stacks] Starting stack: jellyfin
2026/09/14 19:10:47 [INFO] [stacks] Stack jellyfin started successfully (took 0.5s)
2026/09/14 19:10:47 [INFO] [stacks] Status refresh: 18 containers across 57 stacks
2026/09/14 19:10:47 [INFO] [stacks] Starting stack: mealie
2026/09/14 19:10:48 [INFO] [stacks] Stack mealie started successfully (took 0.6s)
2026/09/14 19:10:48 [INFO] [stacks] Status refresh: 19 containers across 57 stacks
2026/09/14 19:10:48 [INFO] [stacks] Starting stack: nextcloud
2026/09/14 19:10:50 [INFO] [stacks] Stack immich post-start status:
2026/09/14 19:10:50 [INFO] [stacks] immich-machine-learning ghcr.io/immich-app/immich-machine-learning:v3.0.3 running Up 14 seconds (healthy)
2026/09/14 19:10:50 [INFO] [stacks] immich-postgres ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0 running Up 14 seconds (healthy)
2026/09/14 19:10:50 [INFO] [stacks] immich-redis redis:7-alpine running Up 14 seconds (healthy)
2026/09/14 19:10:50 [INFO] [stacks] immich-server ghcr.io/immich-app/immich-server:v3.0.3 running Up 3 seconds (health: starting)
2026/09/14 19:10:51 [INFO] [stacks] Stack jellyfin post-start status:
2026/09/14 19:10:51 [INFO] [stacks] jellyfin jellyfin/jellyfin:10.11.11 running Up 3 seconds (health: starting)
2026/09/14 19:10:51 [INFO] [stacks] Status refresh: 22 containers across 57 stacks
2026/09/14 19:10:51 [INFO] Health probes: 1 ok (of 1 probed)
2026/09/14 19:10:51 [INFO] [stacks] Stack mealie post-start status:
2026/09/14 19:10:51 [INFO] [stacks] mealie ghcr.io/mealie-recipes/mealie:v3.20.1 running Up 3 seconds (health: starting)
2026/09/14 19:10:55 [INFO] [stacks] Stack nextcloud started successfully (took 6.9s)
2026/09/14 19:10:56 [INFO] [stacks] Status refresh: 22 containers across 57 stacks
2026/09/14 19:10:56 [INFO] [stacks] Starting stack: paperless-ngx
2026/09/14 19:10:59 [INFO] [stacks] Stack nextcloud post-start status:
2026/09/14 19:10:59 [INFO] [stacks] nextcloud nextcloud:34.0.1-apache running Up 3 seconds (health: starting)
2026/09/14 19:10:59 [INFO] [stacks] nextcloud-db mariadb:11.6 running Up 9 seconds (healthy)
2026/09/14 19:10:59 [INFO] [stacks] nextcloud-redis redis:7-alpine running Up 9 seconds (healthy)
2026/09/14 19:11:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:11:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:11:01 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:11:01 [INFO] [scheduler] Job agent-channel-health completed (took 248ms)
2026/09/14 19:11:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:11:01 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:11:01 [INFO] [scheduler] Job stack-scan completed (took 410ms)
2026/09/14 19:11:01 [INFO] Health probes: 1 ok (of 1 probed)
2026/09/14 19:11:08 [INFO] [stacks] Stack paperless-ngx started successfully (took 12.2s)
2026/09/14 19:11:08 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
2026/09/14 19:11:08 [INFO] [stacks] Starting stack: privatebin
2026/09/14 19:11:09 [INFO] [stacks] Stack privatebin started successfully (took 0.6s)
2026/09/14 19:11:09 [INFO] [stacks] Status refresh: 26 containers across 57 stacks
2026/09/14 19:11:09 [INFO] [stacks] Starting stack: uptime-kuma
2026/09/14 19:11:10 [INFO] [stacks] Stack uptime-kuma started successfully (took 1.0s)
2026/09/14 19:11:11 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:11:11 [INFO] [stacks] Starting stack: vaultwarden
2026/09/14 19:11:11 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
2026/09/14 19:11:11 [INFO] Health probes: 2 ok (of 2 probed)
2026/09/14 19:11:12 [INFO] [stacks] Stack paperless-ngx post-start status:
2026/09/14 19:11:12 [INFO] [stacks] paperless-postgres postgres:16-alpine running Up 15 seconds (healthy)
2026/09/14 19:11:12 [INFO] [stacks] paperless-redis redis:7-alpine running Up 15 seconds (healthy)
2026/09/14 19:11:12 [INFO] [stacks] paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 running Up 4 seconds (health: starting)
2026/09/14 19:11:12 [INFO] [stacks] Stack vaultwarden started successfully (took 1.1s)
2026/09/14 19:11:12 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:11:12 [WARN] [quiesce] tier felhom-pbs: backup job backup-9201-felhom-pbs-1789412999526947801 failed
2026/09/14 19:11:12 [WARN] [quiesce] tier felhom-pbs has now failed 1 time(s) in a row — deferring its next quiesce by 15m0s (cap 4h0m0s) so the apps are not stopped again for a backup that cannot succeed
2026/09/14 19:11:12 [INFO] Event pushed: whole_guest_backup_failed (error) — Whole-guest backup FAILED on the felhom-pbs tier — retrying with backoff (next attempt in 15m0s)
2026/09/14 19:11:12 [INFO] [stacks] Stack privatebin post-start status:
2026/09/14 19:11:12 [INFO] [stacks] privatebin privatebin/pdo:2.0.5 running Up 3 seconds (health: starting)
2026/09/14 19:11:14 [INFO] [stacks] Stack uptime-kuma post-start status:
2026/09/14 19:11:14 [INFO] [stacks] uptime-kuma louislam/uptime-kuma:2.4.0 running Up 4 seconds (health: starting)
2026/09/14 19:11:15 [INFO] [stacks] Stack vaultwarden post-start status:
2026/09/14 19:11:15 [INFO] [stacks] vaultwarden vaultwarden/server:1.36.0-alpine running Up 4 seconds (health: starting)
2026/09/14 19:11:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:11:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:11:31 [INFO] Health probes: 4 ok (of 4 probed)
2026/09/14 19:11:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:11:41 [INFO] Health probes: 1 ok (of 1 probed)
2026/09/14 19:11:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:12:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:12:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:12:01 [INFO] [scheduler] Job agent-channel-health completed (took 205ms)
2026/09/14 19:12:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:12:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:12:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:12:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:12:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:13:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:13:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:13:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:13:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:13:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:13:01 [INFO] [scheduler] Job stack-scan completed (took 132ms)
2026/09/14 19:13:01 [INFO] [scheduler] Job agent-channel-health completed (took 233ms)
2026/09/14 19:13:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:13:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:13:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:13:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:13:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:14:00 [INFO] [sync] Starting catalog sync
2026/09/14 19:14:00 [INFO] [sync] Pulling latest from https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git (branch: main)
2026/09/14 19:14:01 [INFO] [sync] Updated privatebin/.felhom.yml
2026/09/14 19:14:01 [INFO] [scheduler] Running job: hub-report
2026/09/14 19:14:01 [INFO] [report] Building system report
2026/09/14 19:14:01 [INFO] [scheduler] Running job: backup-cache
2026/09/14 19:14:01 [INFO] [scheduler] Running job: offsite-credential-retry
2026/09/14 19:14:01 [INFO] [scheduler] Job offsite-credential-retry completed (took 0s)
2026/09/14 19:14:01 [INFO] [scheduler] Running job: system-health
2026/09/14 19:14:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:14:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:14:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:14:01 [INFO] [stacks] InjectMissingFields: processed 1 stacks
2026/09/14 19:14:01 [INFO] [sync] Catalog sync complete
2026/09/14 19:14:01 [INFO] [sync] Periodic sync: Sablonok frissítve — frissítve: privatebin
2026/09/14 19:14:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:14:01 [INFO] [backup] Found 6 DB dump files across drives
2026/09/14 19:14:01 [INFO] [scheduler] Job agent-channel-health completed (took 232ms)
2026/09/14 19:14:01 [INFO] [backup] Discovered 6 databases
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/adventurelog/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/docmost/docker-compose.yml
2026/09/14 19:14:01 [INFO] [monitor] Health check: status=ok
2026/09/14 19:14:01 [INFO] [scheduler] Job system-health completed (took 406ms)
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/gokapi/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/jellyfin/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/mealie/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/uptime-kuma/docker-compose.yml
2026/09/14 19:14:01 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/vaultwarden/docker-compose.yml
2026/09/14 19:14:01 [INFO] [backup] Discovered app data: 12 apps
2026/09/14 19:14:01 [INFO] [monitor] Health check: status=ok
2026/09/14 19:14:01 [INFO] [backup] Recovery unit captured for privatebin → /mnt/sys_drive/felhom-data/backups/primary/privatebin (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026/09/14 19:14:01 [INFO] [backup] Backup status cache refreshed
2026/09/14 19:14:01 [INFO] [scheduler] Job backup-cache completed (took 492ms)
2026/09/14 19:14:02 [INFO] [report] Hub report pushed successfully (18715 bytes)
2026/09/14 19:14:02 [INFO] [settings] Settings saved
2026/09/14 19:14:02 [INFO] [scheduler] Job hub-report completed (took 1.455s)
2026/09/14 19:14:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:14:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:14:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:14:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:14:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:15:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:15:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:15:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:15:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:15:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:15:01 [INFO] [scheduler] Job stack-scan completed (took 87ms)
2026/09/14 19:15:01 [INFO] [scheduler] Job agent-channel-health completed (took 200ms)
2026/09/14 19:15:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:15:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:15:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:15:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:15:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:15:51 [INFO] Health probes: 3 ok (of 3 probed)
2026/09/14 19:16:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:16:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:01 [INFO] [scheduler] Job agent-channel-health completed (took 208ms)
2026/09/14 19:16:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:11 [INFO] Health probes: 1 ok (of 1 probed)
2026/09/14 19:16:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:21 [INFO] Health probes: 2 ok (of 2 probed)
2026/09/14 19:16:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:33 [INFO] [api] update requested for stack: privatebin
2026/09/14 19:16:33 [INFO] [stacks] Memory check: total=11828MB, reserved=384MB, usable=11444MB, committed_used=4096MB, new_req=30MB, remaining=7318MB
2026/09/14 19:16:33 [INFO] [stacks] Memory check: total=11828MB, reserved=384MB, usable=11444MB, committed_used=4096MB, new_req=30MB, remaining=7318MB
2026/09/14 19:16:33 [INFO] [stacks] update privatebin: accepted — guarded update started
2026/09/14 19:16:33 [INFO] [stacks] update privatebin: phase checking
2026/09/14 19:16:33 [INFO] [stacks] update privatebin: precondition met — Tier 2 (second drive) copy from 2026-09-14T19:03:05Z (13m0s old, limit 24h0m0s)
2026/09/14 19:16:33 [INFO] [stacks] update privatebin: phase safety-dump
2026/09/14 19:16:34 [INFO] [backup] Discovered 6 databases
2026/09/14 19:16:34 [INFO] [backup] update safety dump for privatebin: the app has no database — nothing to copy (no-op)
2026/09/14 19:16:34 [INFO] [stacks] update privatebin: safety dump done (0 file(s)) []
2026/09/14 19:16:34 [INFO] [stacks] update privatebin: phase pinning
2026/09/14 19:16:34 [INFO] [stacks] update privatebin: pin advanced to the catalog's current definition (privatebin=privatebin/pdo:2.0.6)
2026/09/14 19:16:34 [INFO] [stacks] SaveAppConfig: saved config for privatebin
2026/09/14 19:16:34 [INFO] [stacks] update privatebin: phase pulling
2026/09/14 19:16:38 [INFO] [stacks] update privatebin: phase starting
2026/09/14 19:16:39 [INFO] [stacks] update privatebin: phase verifying
2026/09/14 19:16:39 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:41 [INFO] Health probes: 3 ok (of 3 probed)
2026/09/14 19:16:44 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:44 [INFO] [stacks] update privatebin: healthy after 5s (the app's health check passed)
2026/09/14 19:16:44 [INFO] [stacks] SaveAppConfig: saved config for privatebin
2026/09/14 19:16:44 [INFO] [stacks] installed-images privatebin: recorded 1 service(s) (privatebin=privatebin/pdo:2.0.6 (sha256:4c141b2326f8…))
2026/09/14 19:16:44 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:44 [INFO] [stacks] update privatebin: DONE in 11s
2026/09/14 19:16:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:16:51 [INFO] Health probes: 1 ok (of 1 probed)
2026/09/14 19:17:01 [INFO] [scheduler] Running job: stack-scan
2026/09/14 19:17:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:17:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:17:01 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
2026/09/14 19:17:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:17:01 [INFO] [scheduler] Job stack-scan completed (took 89ms)
2026/09/14 19:17:01 [INFO] [scheduler] Job agent-channel-health completed (took 225ms)
2026/09/14 19:17:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:17:21 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:17:31 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:17:41 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:17:51 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:18:01 [INFO] [scheduler] Running job: agent-channel-health
2026/09/14 19:18:01 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
2026/09/14 19:18:01 [INFO] [scheduler] Job agent-channel-health completed (took 231ms)
documentation/audits/evidence-bignight-2026-09-14/journal.md
+38
View File
@@ -429,3 +429,41 @@ Tier 2 result (controller log, `phase4/controller-log-t1-t2.txt`): `Tier 2 run c
1.3 GB, paperless 68.8 MB, jellyfin 0.6 MB) — definitions and databases, not the photos, as the Nextcloud page said
(„csak az adatbázis és a konfiguráció másolódik"). The status API's `running` flag covers only the DB leg, so it read
false during Tier 2 (the harness loop exited on it; the log is the observable).
**Whole-system „Mentés most"** (`POST /api/guest-backup/trigger` 19:03:23Z; `phase4/guest-backup-run.txt`,
`guest-backup-quiesce-log.txt`): the controller quiesced **all 12 apps** („backup due on 2 tier(s)"); tier `local`
19:03:49 → 19:09:59Z, **8 877 619 753 B, 362 s, success**; then tier `felhom-pbs` started with the apps still stopped,
failed (`storage 'felhom-pbs' does not exist` — the DR tier that never provisioned, R-511), unquiesce 19:10:09Z, last app
started 19:11:12Z → **apps down ≈ 7 m 45 s** under a page line promising „csak néhány másodpercre" → **R-518 (P2)**.
Backoff logged: next PBS attempt in 15 min „so the apps are not stopped again for a backup that cannot succeed".
Harness: the status API kept the previous job's `done` until the new job id appeared; the first poll stopped on it.
**Backup pages, read as a first-timer after the run** (`phase4/backup-pages-after.txt`, 19:15:36Z):
- Áttekintés: „✗ · Utolsó teljes mentés 2026-09-14 21:09 (5 perce) · **0 B** · Biztonsági szerver – külön hardver
(PBS) · **Naprakész**" and „✓ **Távoli rendszermentés — külön hardveren (PBS)**" — **false on both counts**; the real
8.9 GB local backup is not shown → **R-517 (P1)**. Dates are local time (21:09 CEST) ✓. „DB mentések 6 fájl" ✓.
„Következő mentés — 0 órája" (reads backwards, seen in the 0242 drill). Both honest warnings about one disk remain
although a data drive is now enrolled — one of them offers „Adatlemez · Kijelölöm" as the whole-system target ✓.
- Alkalmazások: every app „1. mentés Auto helyi Utolsó: …"; drive apps' second copy state-only as said.
- `/stacks/bookstack/backup` still claims PBS coverage (**R-499**).
- **Off-site integrity check: not possible on this record** — „Távoli mentés": „Még nincs beállítva távoli mentési cél";
`/backup/offbox/status` → `snapshots 0, status ""`. Verdict and duration: **none — no tier to check** (recorded, not run).
**Hub after the routines** (`phase4/hub-check.txt`, 19:15:31Z): „Tester 1 · **ok** · Controller 0.242.0 · Last report
2 min ago · Containers **24/24**" (all 12 apps' containers listed), storage „SSD 33 % · **Adatlemez 4 %**", „Last DB dump
13 min ago", „No off-site data reported … this page cannot say". 13 × `crossdrive_completed (info)`; **21:11:12
`whole_guest_backup_failed (error)` + operator e-mail — a TRUE alarm** (the tier really cannot succeed), with no word of
the local success. The hub page shows no whole-system backup line at all.
**Catalog travel:** the box synced at 18:59:00Z (36 s before the push) and picked the bump up at **19:14:01Z**
(„Sablonok frissítve — frissítve: privatebin") — 14 m 25 s after the push. App page: „Frissítés elérhető — **ma**"
(title „Újabb változat érhető el …"). PrivateBin before the update: 2 pastes decrypt equal; the 10-minute paste
answers „Document does not exist, has expired or has been deleted." — honest expiry.
**The guarded Update, privatebin 2.0.5 → 2.0.6** (`phase4/privatebin-update.txt`): `POST /api/stacks/privatebin/update`
19:16:33Z → „Frissítés elindult – az állapot a kártyán követhető". Phases: checking → **precondition met — Tier 2
(second drive) copy from 19:03:05Z (13 m old, limit 24 h)** → safety-dump (no database, no-op) → **pinning** („pin
advanced to the catalog's current definition") → pulling → starting → verifying → „healthy after 5s" → installed-images
recorded `privatebin/pdo:2.0.6 (sha256:4c141b23…)` → **DONE in 11 s**. Container `privatebin/pdo:2.0.6 (healthy)`. App
page „Fut · **Naprakész**". **Data read back after: both surviving pastes decrypt equal**; the expired one still
expired. PASS. (Observation: `template_images` in the API still read 2.0.5 for ≈ 15 s after `updating` went false.)
documentation/audits/evidence-bignight-2026-09-14/phase4/privatebin-after-update.txt
+7
View File
@@ -0,0 +1,7 @@
=== after update 19:17:17
19:17:17 16b07513ec2946f2 1month decrypts equal True
19:17:17 eb1a716a315d7329 never decrypts equal True
19:17:17 32e4908873822f0e 10min status 1 Document does not exist, has expired or has been deleted.
19:17:17 pastes in seed file 3 read back equal 2
19:17:17 page version marker []
privatebin privatebin/pdo:2.0.6 Up 39 seconds (healthy)
documentation/audits/evidence-bignight-2026-09-14/phase4/privatebin-before-update.txt
+6 -6
View File
@@ -1,6 +1,6 @@
(seed file holds the three pastes of the last seeding run: 1month, never, 10min; the first two pastes' keys were not kept by the harness)
19:04:23 16b07513ec2946f2 1month not JSON — HTTP 404 (expired or gone)
19:04:23 eb1a716a315d7329 never not JSON — HTTP 404 (expired or gone)
19:04:23 32e4908873822f0e 10min not JSON — HTTP 404 (expired or gone)
19:04:23 pastes in seed file 3 read back equal 0
19:04:23 page version marker []
(re-read 19:16Z: the 19:04Z 404s were the apps stopped by the whole-system backup, not lost pastes)
19:16:20 16b07513ec2946f2 1month decrypts equal True
19:16:20 eb1a716a315d7329 never decrypts equal True
19:16:20 32e4908873822f0e 10min status 1 Document does not exist, has expired or has been deleted.
19:16:20 pastes in seed file 3 read back equal 2
19:16:20 page version marker []
documentation/audits/evidence-bignight-2026-09-14/phase4/privatebin-update.txt
+29
View File
@@ -0,0 +1,29 @@
POST /api/stacks/privatebin/update 19:16:33
{"ok":true,"data":{"accepted":true,"completed":false},"message":"Frissítés elindult – az állapot a kártyán követhető"}
19:16:34 running updating=True privatebin/pdo:2.0.5
19:16:40 starting updating=True privatebin/pdo:2.0.5
19:16:46 running updating=False privatebin/pdo:2.0.5
19:17:01 running updating=False privatebin/pdo:2.0.6
--- app page tags
Fut
Naprakész
--- controller log
2026/09/14 19:16:33 [INFO] [api] update requested for stack: privatebin
2026/09/14 19:16:33 [INFO] [stacks] update privatebin: accepted — guarded update started
2026/09/14 19:16:33 [INFO] [stacks] update privatebin: phase checking
2026/09/14 19:16:33 [INFO] [stacks] update privatebin: precondition met — Tier 2 (second drive) copy from 2026-09-14T19:03:05Z (13m0s old, limit 24h0m0s)
2026/09/14 19:16:33 [INFO] [stacks] update privatebin: phase safety-dump
2026/09/14 19:16:34 [INFO] [backup] Discovered 6 databases
2026/09/14 19:16:34 [INFO] [backup] update safety dump for privatebin: the app has no database — nothing to copy (no-op)
2026/09/14 19:16:34 [INFO] [stacks] update privatebin: safety dump done (0 file(s)) []
2026/09/14 19:16:34 [INFO] [stacks] update privatebin: phase pinning
2026/09/14 19:16:34 [INFO] [stacks] update privatebin: pin advanced to the catalog's current definition (privatebin=privatebin/pdo:2.0.6)
2026/09/14 19:16:34 [INFO] [stacks] SaveAppConfig: saved config for privatebin
2026/09/14 19:16:34 [INFO] [stacks] update privatebin: phase pulling
2026/09/14 19:16:38 [INFO] [stacks] update privatebin: phase starting
2026/09/14 19:16:39 [INFO] [stacks] update privatebin: phase verifying
2026/09/14 19:16:44 [INFO] [stacks] update privatebin: healthy after 5s (the app's health check passed)
2026/09/14 19:16:44 [INFO] [stacks] SaveAppConfig: saved config for privatebin
2026/09/14 19:16:44 [INFO] [stacks] installed-images privatebin: recorded 1 service(s) (privatebin=privatebin/pdo:2.0.6 (sha256:4c141b2326f8…))
2026/09/14 19:16:44 [INFO] [stacks] update privatebin: DONE in 11s