BIGNIGHT F1 power cut: PASS, all 12 back on the same images in 4m03s, no false alarm
gates / gates (push) Successful in 20s
gates / gates (push) Successful in 20s
This commit is contained in:
@@ -467,3 +467,36 @@ advanced to the catalog's current definition") → pulling → starting → veri
|
||||
recorded `privatebin/pdo:2.0.6 (sha256:4c141b23…)` → **DONE in 11 s**. Container `privatebin/pdo:2.0.6 (healthy)`. App
|
||||
page „Fut · **Naprakész**". **Data read back after: both surviving pastes decrypt equal**; the expired one still
|
||||
expired. PASS. (Observation: `template_images` in the API still read 2.0.5 for ≈ 15 s after `updating` went false.)
|
||||
|
||||
**Catalog revert** `a161ccb` pushed 19:17:55Z (image back to 2.0.5; `catalog_since` stays 2026-09-14 by the gate).
|
||||
Phase 4 evidence pulled off the box 19:18Z (`box-logs-phase4/`).
|
||||
|
||||
## Phase 5 — the accidents (`phase5/`)
|
||||
|
||||
Method, the same for every fault: a **family loop** (`family.py`) keeps three apps busy (BookStack page loads each
|
||||
second, a Nextcloud note written every 2 s, Mealie API each second) and logs ok/fail per 5 s; `watch_recovery.py`
|
||||
records from the fault's T0 when the dashboard's `/api/health` answers and when each of the 12 apps is `running` again
|
||||
with the **same images** as the Phase 4 baseline; `alarms.sh` reads the hub's events, staleness changes and operator
|
||||
mails for tester-1; the Gmail connector is read for what reached the customer's and the operator's mailbox. For each
|
||||
fault: what the customer saw · what the box did alone · time to steady state · which alarm fired and was it true ·
|
||||
which alarm should have fired and did not. Evidence off the box before the next fault.
|
||||
|
||||
**Before F1 — does the failed PBS tier stop the apps again?** The controller deferred the tier 15 min at 19:11:12Z
|
||||
(„so the apps are not stopped again for a backup that cannot succeed"). Read at 19:27:40Z (below).
|
||||
|
||||
**Quiesce retry check (19:27:40Z):** no quiesce or stack stop between 19:18 and 19:27Z — the deferred PBS tier did
|
||||
not stop the apps again. The agent's own attempt still failed at 21:27:43 CEST (hub `WARN host … backup FAILED:
|
||||
target=felhom-pbs … storage 'felhom-pbs' does not exist`), no operator mail for it.
|
||||
|
||||
### F1 — power cut while the family uses three apps (`phase5/F1-power-cut.txt`, `phase5/F1/`, `family-activity-F1.txt`)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| how | `qm stop 333` 19:28:08Z (hard) · 61 s off · `qm start` 19:29:09Z |
|
||||
| what the customer saw | from 19:28:12Z every request to wiki / cloud / recipes fails (connection refused/timeouts) for ≈ 3 m 30 s; BookStack answers again 19:31:37Z, Nextcloud 19:31:42Z, Mealie after 19:32:02Z; the dashboard asks for login again |
|
||||
| what the box did alone | dashboard `/api/health` 200 at **+133 s**; privatebin, vaultwarden, jellyfin +134–135 s … immich, adventurelog +191 s; **paperless-ngx left „boot-orphaned"** and started by the boot reconciler at 19:32:26Z („1 app(s) recovered in 1 attempt(s)"); **all 12 running at +243 s, every one on the same images** as before |
|
||||
| time to steady state | **4 m 03 s** from power-on |
|
||||
| alarms fired | hub: `controller_started (info) — Controller elindult (0.242.0)` only — **true**. Re-read at +10 min (past the 90 s dead-app grace): **no `app_start_failed`, no false alarm** |
|
||||
| should have fired, did not | none — a 61 s outage is under the hub's 30-minute staleness threshold, by design |
|
||||
|
||||
Verdict **PASS**. Jellyfin's health probe answered 503 / refused for ≈ 45 s after its start (WARN lines), then healthy.
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
F1 qm stop at 19:28:08
|
||||
status: stopped
|
||||
F1 qm start at 19:29:09 (power off 61 s)
|
||||
status: running
|
||||
F1 + 133s dashboard /api/health 200
|
||||
F1 + 134s privatebin running same images
|
||||
F1 + 135s vaultwarden running same images
|
||||
F1 + 135s jellyfin running same images
|
||||
F1 + 141s gokapi running same images
|
||||
F1 + 146s bookstack running same images
|
||||
F1 + 147s nextcloud running same images
|
||||
F1 + 152s uptime-kuma running same images
|
||||
F1 + 186s docmost running same images
|
||||
F1 + 186s mealie running same images
|
||||
F1 + 191s immich running same images
|
||||
F1 + 192s adventurelog running same images
|
||||
F1 + 243s paperless-ngx running same images
|
||||
F1 + 243s ALL 12 running
|
||||
--- hub alarms since F1
|
||||
2026/09/14 21:27:43 [WARN] host tester-1-a61396 backup FAILED: target=felhom-pbs vmid=9201 err="proxmox: task UPID:felhom:00023B4C:0007D509:6AA84687:vzdump:9201:felhom-agent@pve!agent: failed: exitstatus \"could not activate storage 'felhom-pbs': storage 'felhom-pbs' does not exist could not get storage information for 'felhom-pbs': storage 'felhom-pbs' does not exist\""
|
||||
2026/09/14 21:31:25 [INFO] Event from tester-1: controller_started (info) — Controller elindult (0.242.0)
|
||||
@@ -0,0 +1,61 @@
|
||||
Sep 14 21:29:33 felhom systemd[1]: Started felhom-agent.service - Felhom host agent (Proxmox host tier; hub control loop + PBS verify + storage watchdog).
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.650+02:00 level=INFO msg="felhom-agent daemon starting" version=0.130.0 host_id=tester-1-a61396 hub_url=https://hub.felhom.eu interval_s=900
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.655+02:00 level=INFO msg="daemon: operator signers pinned" count=2
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.659+02:00 level=INFO msg="local-api leaf LOADED" fingerprint_sha256=f7853a846c980c99337618d5a4618f17fc6b88ffdc00e29d4d422060b0ebf4c2 cert=/var/lib/felhom-agent/local-api.crt
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.659+02:00 level=INFO msg="backup tier armed" target=local cadence=24h0m0s keep_last=3 wait_timeout=30m0s prune_pbs_allowed=false primary=true
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.659+02:00 level=INFO msg="backup tier armed" target=felhom-pbs cadence=168h0m0s keep_last=0 wait_timeout=12h0m0s prune_pbs_allowed=false primary=false
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.659+02:00 level=INFO msg="lanresolver: enabled" host_ip=192.168.0.136 upstreams="[1.1.1.1 8.8.8.8]" interval_s=300
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.659+02:00 level=INFO msg="wgtunnel: enabled" interval_s=60 state_dir=/var/lib/felhom-agent
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.659+02:00 level=INFO msg="poke: agent-plane sync listener enabled" port=51822
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.659+02:00 level=INFO msg="felhomsshd (OOB): enabled" interval_s=60 state_dir=/var/lib/felhom-agent
|
||||
Sep 14 21:29:33 felhom felhom-agent[1141]: time=2026-09-14T21:29:33.659+02:00 level=INFO msg="pbsdr: bridge enabled (hub-driven; no-op until a pbs_dr descriptor arrives)"
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.164+02:00 level=INFO msg="capabilities self-check" ok=65 total=70 degraded=0 inactive=5
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.164+02:00 level=INFO msg="storage: watchdog starting" interval=5s debounce=15s
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.168+02:00 level=INFO msg="poke: listening for hub sync-pokes (WG-confined, contentless)" addr=10.77.0.5:51822
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.168+02:00 level=INFO msg="backup: restore-test scheduler starting (per-archive due-check)" eval_interval=6h0m0s settle=24h0m0s
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.168+02:00 level=INFO msg="pbs: verify loop starting" cadence=6h0m0s
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.168+02:00 level=INFO msg="fast-tick armed: 30s out-of-band cadence while desired-state is unapplied" interval=30s
|
||||
Sep 14 21:29:36 felhom sudo[1295]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives
|
||||
Sep 14 21:29:36 felhom sudo[1295]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1295]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1307]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
Sep 14 21:29:36 felhom sudo[1307]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1307]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1312]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
|
||||
Sep 14 21:29:36 felhom sudo[1312]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1313]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
Sep 14 21:29:36 felhom sudo[1313]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1312]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1313]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1318]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/hdd_1/felhom-data
|
||||
Sep 14 21:29:36 felhom sudo[1318]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1318]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1319]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -a -j /dev/sda
|
||||
Sep 14 21:29:36 felhom sudo[1319]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1322]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/chown 100000:100000 /mnt/hdd_1/felhom-data
|
||||
Sep 14 21:29:36 felhom sudo[1322]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1322]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1319]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1327]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives/hdd_1
|
||||
Sep 14 21:29:36 felhom sudo[1327]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1327]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1331]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/mount --bind /mnt/hdd_1/felhom-data /mnt/felhom-drives/hdd_1
|
||||
Sep 14 21:29:36 felhom sudo[1331]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1331]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.423+02:00 level=INFO msg="guest-attach: drive bound under shared parent (normalized to one bind, live)" vmid=9201 where=/mnt/hdd_1 stable=/mnt/felhom-drives/hdd_1 prior_binds=0
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.423+02:00 level=INFO msg="reconcile: enrolled drive bound under shared parent (live, no reboot)" vmid=9201 where=/mnt/hdd_1 guest_path=/mnt/felhom-drives/hdd_1 durable_id=uuid:c4b530fd-d410-424c-917e-c8ecfdde9d8b
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.424+02:00 level=INFO msg="selfheal: node watchdog starting" mode=appliance interval_s=60
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.426+02:00 level=INFO msg="guestnet: watchdog starting" interval=1m0s min_heal_interval=10m0s max_heals_per_hour=3 settle=3m0s
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.449+02:00 level=INFO msg="stale-lock: scanning pool guests" pool=felhom listed=1 scanned=1
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.454+02:00 level=INFO msg="stale-lock: scanning pool guests" pool=felhom listed=1 scanned=1
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.458+02:00 level=INFO msg="guest-power: watchdog started" interval=1m0s max_attempts=3
|
||||
Sep 14 21:29:36 felhom felhom-agent[1141]: time=2026-09-14T21:29:36.458+02:00 level=INFO msg="local-api server listening" addr=169.254.253.1:8443
|
||||
Sep 14 21:29:36 felhom sudo[1344]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/systemctl enable --now dnsmasq
|
||||
Sep 14 21:29:36 felhom sudo[1344]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1350]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/sshd -t -f /var/lib/felhom-agent/felhom-sshd/sshd_config
|
||||
Sep 14 21:29:36 felhom sudo[1350]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1350]: pam_unix(sudo:session): session closed for user root
|
||||
Sep 14 21:29:36 felhom sudo[1360]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/install -o root -g root -m 0644 -- /var/lib/felhom-agent/felhom-sshd/sshd_config /etc/felhom-sshd/sshd_config
|
||||
Sep 14 21:29:36 felhom sudo[1360]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=999)
|
||||
Sep 14 21:29:36 felhom sudo[1360]: pam_unix(sudo:session): session closed for user root
|
||||
2026-09-14 21:29:24
|
||||
@@ -0,0 +1,194 @@
|
||||
2026/09/14 19:31:18 [INFO] local-api: channel up (agent 169.254.253.1:8443) — guest 9201, 3 mount(s) visible
|
||||
2026/09/14 19:31:18 [INFO] local-api: mount mp9 → /etc/felhom-bootstrap (storage=/var/lib/felhom-agent/guests/9201/bootstrap, class=, backup=false)
|
||||
2026/09/14 19:31:18 [INFO] local-api: mount mp0 → /var/lib/felhom (storage=local-lvm, class=, backup=true)
|
||||
2026/09/14 19:31:18 [INFO] local-api: mount mp8 → /mnt/felhom-drives (storage=/mnt/felhom-drives, class=, backup=false)
|
||||
2026/09/14 19:31:18 [INFO] felhom-controller 0.242.0 starting (customer: tester-1, domain: enkicsifelhom.hu)
|
||||
2026/09/14 19:31:18 [INFO] [settings] Loaded settings from /opt/docker/felhom-controller/data/settings.json
|
||||
2026/09/14 19:31:18 [INFO] Encryption key loaded from /opt/docker/felhom-controller/data/encryption.key
|
||||
2026/09/14 19:31:18 [INFO] [stacks] Using compose command: docker compose
|
||||
2026/09/14 19:31:18 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
|
||||
2026/09/14 19:31:18 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:18 [INFO] [stacks] InjectMissingFields: processed 12 stacks
|
||||
2026/09/14 19:31:18 [INFO] [stacks] Encryption migration: no stacks needed migration
|
||||
2026/09/14 19:31:18 [INFO] [stacks] desired-state backfill: 0 app(s) recorded as running, 0 left unrecorded (state ambiguous — legacy boot behaviour retained)
|
||||
2026/09/14 19:31:18 [INFO] [stacks] installed-images backfill: 0 app(s) recorded, 12 already had a record, 0 left unrecorded (could not be observed completely — unknown, which renders nothing)
|
||||
2026/09/14 19:31:18 [INFO] [stacks] pin adoption: 0 pinned, 12 already pinned, 0 left unpinned (0 not completely observed, 0 running something the template no longer offers)
|
||||
2026/09/14 19:31:18 [INFO] [sync] Starting catalog sync (repo: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git, interval: 15m0s)
|
||||
2026/09/14 19:31:18 [INFO] [quiesce] loop started (poll 5m0s, max-quiesce 30m0s)
|
||||
2026/09/14 19:31:18 [INFO] [sync] Starting catalog sync
|
||||
2026/09/14 19:31:18 [INFO] [sync] Pulling latest from https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git (branch: main)
|
||||
2026/09/14 19:31:18 [INFO] Metrics store opened at /opt/docker/felhom-controller/data/metrics.db
|
||||
2026/09/14 19:31:18 [INFO] Metrics collector started (60s interval)
|
||||
2026/09/14 19:31:18 [INFO] Notifier enabled (hub: https://hub.felhom.eu)
|
||||
2026/09/14 19:31:18 [INFO] Self-update enabled (check every 6h, auto-update: false, auto-update time: 04:30)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: status-refresh (every 10s)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: stack-scan (every 2m0s)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: health-probes (every 10s)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: system-health (every 5m0s)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: deadapp-check (every 30s)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: ring-spill (every 30s)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Daily job db-dump scheduled for 2026-09-15 02:30 CEST
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: offsite-credential-retry (every 5m0s)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: backup-cache (every 5m0s)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Daily job tier2-backup scheduled for 2026-09-15 03:30 CEST
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Daily job offbox-backup scheduled for 2026-09-15 04:15 CEST
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Daily job offsite-abandon-sweep scheduled for 2026-09-15 05:10 CEST
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Daily job offsite-integrity scheduled for 2026-09-15 06:00 CEST
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Daily job offsite-proof scheduled for 2026-09-15 05:30 CEST
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Daily job metrics-prune scheduled for 2026-09-15 04:00 CEST
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Daily job fill-watch scheduled for 2026-09-15 03:30 CEST
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: hub-report (every 15m0s)
|
||||
2026/09/14 19:31:18 [INFO] Hub reporting enabled (every 15m0s to https://hub.felhom.eu)
|
||||
2026/09/14 19:31:18 [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
|
||||
2026/09/14 19:31:18 [INFO] ========== Startup Self-Test ==========
|
||||
2026/09/14 19:31:19 [INFO] [PASS] Docker socket: reachable (v29.8.0)
|
||||
2026/09/14 19:31:19 [INFO] [PASS] Stacks directory: /opt/docker/stacks
|
||||
2026/09/14 19:31:19 [INFO] [PASS] Data directory: /opt/docker/felhom-controller/data (writable)
|
||||
2026/09/14 19:31:19 [INFO] [PASS] System data path: /mnt/sys_drive
|
||||
2026/09/14 19:31:19 [INFO] [PASS] Storage paths: 1 connected, 0 disconnected
|
||||
2026/09/14 19:31:19 [INFO] [PASS] Git catalog: 53 app definitions found
|
||||
2026/09/14 19:31:19 [INFO] [infra] connected felhom-controller to traefik-public
|
||||
2026/09/14 19:31:20 [INFO] [PASS] Hub connectivity: https://hub.felhom.eu reachable (HTTP 200)
|
||||
2026/09/14 19:31:20 [INFO] [PASS] Metrics DB: 0.0 MB
|
||||
2026/09/14 19:31:20 [INFO] ========================================
|
||||
2026/09/14 19:31:20 [INFO] Self-test complete: 8 passed, 0 warnings, 0 failed
|
||||
2026/09/14 19:31:20 [INFO] [scheduler] Starting scheduler with 18 jobs
|
||||
2026/09/14 19:31:20 [INFO] [scheduler] Registered periodic job: geo-verify (every 6h0m0s)
|
||||
2026/09/14 19:31:20 [INFO] Geo-restriction support enabled (CF API token configured)
|
||||
2026/09/14 19:31:20 [INFO] [web] Auth: using password from settings.json
|
||||
2026/09/14 19:31:20 [INFO] [scheduler] Registered periodic job: disk-health-check (every 1h0m0s)
|
||||
2026/09/14 19:31:20 [INFO] [report] hub wait channel active (hold ≤240s)
|
||||
2026/09/14 19:31:20 [INFO] [scheduler] Registered periodic job: agent-channel-health (every 1m0s)
|
||||
2026/09/14 19:31:20 [INFO] [backup] Found 6 DB dump files across drives
|
||||
2026/09/14 19:31:20 [INFO] Web UI listening on :8080
|
||||
2026/09/14 19:31:20 [INFO] [gate] boot 1789414164-10084: waiting (≤2m0s) for live drive bind(s) [/mnt/felhom-drives/hdd_1] before recreating drive-backed apps
|
||||
2026/09/14 19:31:20 [INFO] [gate] boot 1789414164-10084: live bind confirmed — recreating drive-backed app immich (state=starting) onto /mnt/felhom-drives/hdd_1
|
||||
2026/09/14 19:31:20 [INFO] [stacks] Stopping stack: immich
|
||||
2026/09/14 19:31:21 [WARN] [api] Unauthorized request to /api/stacks/bookstack from 172.18.0.14:41806
|
||||
2026/09/14 19:31:22 [INFO] [sync] Catalog sync complete
|
||||
2026/09/14 19:31:22 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||
2026/09/14 19:31:22 [INFO] [web] FileBrowser sync — no config/compose change, ensured running without recreate (1 storage path(s))
|
||||
2026/09/14 19:31:22 [INFO] [web] Login from 172.18.0.14:41806
|
||||
2026/09/14 19:31:22 [INFO] [monitor] Health check: status=ok
|
||||
2026/09/14 19:31:23 [INFO] [backup] Discovered 6 databases
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/adventurelog/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/bookstack/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/docmost/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/gokapi/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/jellyfin/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/mealie/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/nextcloud/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/uptime-kuma/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/vaultwarden/docker-compose.yml
|
||||
2026/09/14 19:31:23 [INFO] [backup] Discovered app data: 12 apps
|
||||
2026/09/14 19:31:23 [INFO] [backup] Backup status cache refreshed
|
||||
2026/09/14 19:31:23 [INFO] [stacks] Stack immich stopped successfully (took 2.8s)
|
||||
2026/09/14 19:31:23 [INFO] [stacks] Status refresh: 24 containers across 57 stacks
|
||||
2026/09/14 19:31:23 [INFO] [stacks] Starting stack: immich
|
||||
2026/09/14 19:31:23 [INFO] [stacks] Status refresh: 24 containers across 57 stacks
|
||||
2026/09/14 19:31:25 [INFO] [report] Building system report
|
||||
2026/09/14 19:31:25 [INFO] Event pushed: controller_started (info) — Controller elindult (0.242.0)
|
||||
2026/09/14 19:31:26 [INFO] [monitor] Health check: status=ok
|
||||
2026/09/14 19:31:27 [INFO] [report] Hub report pushed successfully (12853 bytes)
|
||||
2026/09/14 19:31:27 [INFO] [settings] Settings saved
|
||||
2026/09/14 19:31:27 [INFO] Startup hub report sent
|
||||
2026/09/14 19:31:29 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:30 [WARN] Health probe jellyfin: API GET :8096/health → Get "http://jellyfin:8096/health": dial tcp 172.18.0.6:8096: connect: connection refused
|
||||
2026/09/14 19:31:30 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:30 [WARN] Health probes: 3 ok, 1 unhealthy (of 4 probed)
|
||||
2026/09/14 19:31:34 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:36 [INFO] [stacks] Stack immich started successfully (took 13.0s)
|
||||
2026/09/14 19:31:37 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:37 [INFO] [gate] boot 1789414164-10084: live bind confirmed — recreating drive-backed app jellyfin (state=starting) onto /mnt/felhom-drives/hdd_1
|
||||
2026/09/14 19:31:37 [INFO] [stacks] Stopping stack: jellyfin
|
||||
2026/09/14 19:31:39 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:40 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:40 [INFO] [stacks] Stack immich post-start status:
|
||||
2026/09/14 19:31:40 [INFO] [stacks] immich-machine-learning ghcr.io/immich-app/immich-machine-learning:v3.0.3 running Up 15 seconds (health: starting)
|
||||
2026/09/14 19:31:40 [INFO] [stacks] immich-postgres ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0 running Up 15 seconds (healthy)
|
||||
2026/09/14 19:31:40 [INFO] [stacks] immich-redis redis:7-alpine running Up 15 seconds (healthy)
|
||||
2026/09/14 19:31:40 [INFO] [stacks] immich-server ghcr.io/immich-app/immich-server:v3.0.3 running Up 4 seconds (health: starting)
|
||||
2026/09/14 19:31:41 [WARN] Health probe jellyfin: API GET :8096/health → expected status 200, got 503
|
||||
2026/09/14 19:31:41 [WARN] Health probes: 2 ok, 1 unhealthy (of 3 probed)
|
||||
2026/09/14 19:31:44 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:47 [INFO] [stacks] Stack jellyfin stopped successfully (took 10.7s)
|
||||
2026/09/14 19:31:47 [INFO] [stacks] Status refresh: 27 containers across 57 stacks
|
||||
2026/09/14 19:31:47 [INFO] [stacks] Starting stack: jellyfin
|
||||
2026/09/14 19:31:48 [INFO] [stacks] Stack jellyfin started successfully (took 0.9s)
|
||||
2026/09/14 19:31:49 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:49 [INFO] [gate] boot 1789414164-10084: live bind confirmed — recreating drive-backed app nextcloud (state=starting) onto /mnt/felhom-drives/hdd_1
|
||||
2026/09/14 19:31:49 [INFO] [stacks] Stopping stack: nextcloud
|
||||
2026/09/14 19:31:49 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:50 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:31:50 [INFO] Health probes: 1 ok (of 1 probed)
|
||||
2026/09/14 19:31:52 [INFO] [selfupdate] Current version 0.242.0 is up to date
|
||||
2026/09/14 19:31:52 [INFO] [stacks] Stack jellyfin post-start status:
|
||||
2026/09/14 19:31:52 [INFO] [stacks] jellyfin jellyfin/jellyfin:10.11.11 running Up 4 seconds (health: starting)
|
||||
2026/09/14 19:31:54 [INFO] [stacks] Status refresh: 26 containers across 57 stacks
|
||||
2026/09/14 19:31:54 [INFO] [stacks] Stack nextcloud stopped successfully (took 5.7s)
|
||||
2026/09/14 19:31:55 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
|
||||
2026/09/14 19:31:55 [INFO] [stacks] Starting stack: nextcloud
|
||||
2026/09/14 19:31:59 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:00 [WARN] Health probe jellyfin: API GET :8096/health → Get "http://jellyfin:8096/health": dial tcp 172.18.0.6:8096: connect: connection refused
|
||||
2026/09/14 19:32:00 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:00 [WARN] Health probes: 0 ok, 1 unhealthy (of 1 probed)
|
||||
2026/09/14 19:32:02 [INFO] [stacks] Stack nextcloud started successfully (took 7.2s)
|
||||
2026/09/14 19:32:02 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:02 [INFO] [gate] boot 1789414164-10084: live bind confirmed — recreating drive-backed app paperless-ngx (state=starting) onto /mnt/felhom-drives/hdd_1
|
||||
2026/09/14 19:32:02 [INFO] [stacks] Stopping stack: paperless-ngx
|
||||
2026/09/14 19:32:05 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:07 [INFO] [stacks] Stack nextcloud post-start status:
|
||||
2026/09/14 19:32:07 [INFO] [stacks] nextcloud nextcloud:34.0.1-apache running Up 5 seconds (health: starting)
|
||||
2026/09/14 19:32:07 [INFO] [stacks] nextcloud-db mariadb:11.6 running Up 11 seconds (healthy)
|
||||
2026/09/14 19:32:07 [INFO] [stacks] nextcloud-redis redis:7-alpine running Up 11 seconds (healthy)
|
||||
2026/09/14 19:32:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:11 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:12 [WARN] Health probe jellyfin: API GET :8096/health → expected status 200, got 503
|
||||
2026/09/14 19:32:12 [WARN] Health probes: 2 ok, 1 unhealthy (of 3 probed)
|
||||
2026/09/14 19:32:14 [INFO] [stacks] Stack paperless-ngx stopped successfully (took 12.1s)
|
||||
2026/09/14 19:32:14 [INFO] [stacks] Status refresh: 25 containers across 57 stacks
|
||||
2026/09/14 19:32:14 [INFO] [stacks] Starting stack: paperless-ngx
|
||||
2026/09/14 19:32:16 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:16 [INFO] [bootrecon] boot window: budget 50s exhausted while the fleet was still changing — sweeping anyway
|
||||
2026/09/14 19:32:16 [INFO] [bootrecon] Boot reconciliation: 1 boot-orphaned app(s) found: [paperless-ngx] — up to 2 attempt(s)
|
||||
2026/09/14 19:32:16 [INFO] [stacks] Starting stack: paperless-ngx
|
||||
2026/09/14 19:32:20 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:20 [INFO] Health probes: 1 ok (of 1 probed)
|
||||
2026/09/14 19:32:20 [INFO] [scheduler] Running job: agent-channel-health
|
||||
2026/09/14 19:32:20 [INFO] [scheduler] Job agent-channel-health completed (took 186ms)
|
||||
2026/09/14 19:32:26 [INFO] [stacks] Stack paperless-ngx started successfully (took 9.9s)
|
||||
2026/09/14 19:32:26 [INFO] [stacks] Stack paperless-ngx started successfully (took 11.5s)
|
||||
2026/09/14 19:32:26 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:26 [INFO] [bootrecon] Boot reconciliation attempt 1/2: started "paperless-ngx" (took 10.2s)
|
||||
2026/09/14 19:32:26 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:26 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:26 [INFO] [gate] boot 1789414164-10084: re-syncing FileBrowser mounts against the live binds
|
||||
2026/09/14 19:32:26 [INFO] [settings] Settings saved
|
||||
2026/09/14 19:32:26 [INFO] [bootrecon] Boot reconciliation complete: 1 app(s) recovered in 1 attempt(s): [paperless-ngx]
|
||||
2026/09/14 19:32:26 [INFO] [web] FileBrowser sync — no config/compose change, ensured running without recreate (1 storage path(s))
|
||||
2026/09/14 19:32:29 [INFO] [stacks] Stack paperless-ngx post-start status:
|
||||
2026/09/14 19:32:29 [INFO] [stacks] paperless-postgres postgres:16-alpine running Up 14 seconds (healthy)
|
||||
2026/09/14 19:32:29 [INFO] [stacks] paperless-redis redis:7-alpine running Up 14 seconds (healthy)
|
||||
2026/09/14 19:32:29 [INFO] [stacks] paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 running Up 3 seconds (health: starting)
|
||||
2026/09/14 19:32:29 [INFO] [stacks] Stack paperless-ngx post-start status:
|
||||
2026/09/14 19:32:29 [INFO] [stacks] paperless-postgres postgres:16-alpine running Up 14 seconds (healthy)
|
||||
2026/09/14 19:32:29 [INFO] [stacks] paperless-redis redis:7-alpine running Up 14 seconds (healthy)
|
||||
2026/09/14 19:32:29 [INFO] [stacks] paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 running Up 3 seconds (health: starting)
|
||||
2026/09/14 19:32:30 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:30 [INFO] Health probes: 3 ok (of 3 probed)
|
||||
2026/09/14 19:32:40 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:32:48 [INFO] [fillwatch] checked 3 filesystem(s), 0 unreadable/skipped, 0 notification(s); bands: all ok
|
||||
2026/09/14 19:32:50 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:33:00 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:33:10 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:33:20 [INFO] [scheduler] Running job: stack-scan
|
||||
2026/09/14 19:33:20 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:33:20 [INFO] [stacks] ScanStacks complete: 57 stacks found (12 deployed, 45 available)
|
||||
2026/09/14 19:33:20 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
2026/09/14 19:33:20 [INFO] [scheduler] Job stack-scan completed (took 105ms)
|
||||
2026/09/14 19:33:20 [INFO] [scheduler] Running job: agent-channel-health
|
||||
2026/09/14 19:33:20 [INFO] [scheduler] Job agent-channel-health completed (took 193ms)
|
||||
2026/09/14 19:33:30 [INFO] [stacks] Status refresh: 28 containers across 57 stacks
|
||||
@@ -0,0 +1,255 @@
|
||||
19:18:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:18:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:18:57 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:07 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:12 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:19:17 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:19:32 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:37 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:42 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:19:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:52 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:19:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:20:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:20:07 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:20:12 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:20:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:20:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:20:27 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:20:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:20:37 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:20:42 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:20:47 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:20:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:20:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:07 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:12 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok1/fail0
|
||||
19:21:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:22 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:37 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:42 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:21:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:21:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:21:57 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:07 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:22:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:17 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:22:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:37 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:22:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:52 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:22:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:23:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:23:07 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:23:12 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:23:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:23:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:23:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:23:32 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:23:37 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:23:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:23:47 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:23:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:23:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:07 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok1/fail0
|
||||
19:24:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:22 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:37 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:24:42 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:24:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:24:57 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:25:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:25:07 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:25:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:25:17 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:25:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:25:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:25:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:25:37 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:25:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:25:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:25:52 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:25:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:07 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:26:12 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:26:27 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:37 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:26:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:47 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:26:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:27:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:07 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:27:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:22 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:37 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok1/fail0
|
||||
19:27:42 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:27:57 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:28:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:28:07 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:28:12 bookstack:ok2/fail0 mealie:ok2/fail0 nextcloud:ok0/fail0
|
||||
19:28:17 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:28:22 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail0
|
||||
19:28:27 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:28:32 bookstack:ok0/fail0 mealie:ok0/fail0 nextcloud:ok0/fail0
|
||||
19:28:37 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:28:42 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail0
|
||||
19:28:47 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:28:52 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail0
|
||||
19:28:57 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:02 bookstack:ok0/fail1 mealie:ok0/fail2 nextcloud:ok0/fail1
|
||||
19:29:07 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:12 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:17 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:22 bookstack:ok0/fail2 mealie:ok0/fail2 nextcloud:ok0/fail2
|
||||
19:29:27 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:32 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:37 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:42 bookstack:ok0/fail2 mealie:ok0/fail2 nextcloud:ok0/fail2
|
||||
19:29:47 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:52 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:29:57 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:02 bookstack:ok0/fail2 mealie:ok0/fail2 nextcloud:ok0/fail2
|
||||
19:30:07 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:12 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:17 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:22 bookstack:ok0/fail2 mealie:ok0/fail2 nextcloud:ok0/fail2
|
||||
19:30:27 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:32 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:37 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:42 bookstack:ok0/fail2 mealie:ok0/fail2 nextcloud:ok0/fail2
|
||||
19:30:47 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:52 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:30:57 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:31:02 bookstack:ok0/fail2 mealie:ok0/fail2 nextcloud:ok0/fail2
|
||||
19:31:07 bookstack:ok0/fail1 mealie:ok0/fail1 nextcloud:ok0/fail1
|
||||
19:31:12 bookstack:ok0/fail4 mealie:ok0/fail4 nextcloud:ok0/fail2
|
||||
19:31:17 bookstack:ok0/fail5 mealie:ok0/fail5 nextcloud:ok0/fail3
|
||||
19:31:22 bookstack:ok0/fail4 mealie:ok0/fail4 nextcloud:ok0/fail2
|
||||
19:31:27 bookstack:ok0/fail5 mealie:ok0/fail5 nextcloud:ok0/fail2
|
||||
19:31:32 bookstack:ok0/fail4 mealie:ok0/fail5 nextcloud:ok0/fail3
|
||||
19:31:37 bookstack:ok2/fail3 mealie:ok0/fail4 nextcloud:ok0/fail0
|
||||
19:31:42 bookstack:ok3/fail0 mealie:ok0/fail5 nextcloud:ok1/fail0
|
||||
19:31:47 bookstack:ok5/fail0 mealie:ok0/fail5 nextcloud:ok1/fail0
|
||||
19:31:52 bookstack:ok4/fail0 mealie:ok0/fail5 nextcloud:ok1/fail0
|
||||
19:31:57 bookstack:ok4/fail0 mealie:ok0/fail4 nextcloud:ok1/fail2
|
||||
19:32:02 bookstack:ok4/fail0 mealie:ok0/fail5 nextcloud:ok0/fail2
|
||||
19:32:07 bookstack:ok5/fail0 mealie:ok0/fail5 nextcloud:ok0/fail3
|
||||
19:32:12 bookstack:ok4/fail0 mealie:ok1/fail3 nextcloud:ok1/fail0
|
||||
19:32:17 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:32:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:32:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:32:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:32:37 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:32:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:32:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:32:52 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:32:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:33:02 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:33:07 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:27 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:32 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok1/fail0
|
||||
19:33:37 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:42 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:33:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:34:02 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:34:07 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:34:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:34:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:34:22 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:34:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:34:32 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:34:37 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:34:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:34:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:34:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:34:57 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:02 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:35:07 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:17 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:35:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:32 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:35:37 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:52 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:35:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:36:07 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:36:12 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:27 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:37 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok1/fail0
|
||||
19:36:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:47 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:36:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:37:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:37:07 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:37:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:37:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:37:22 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:37:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:37:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:37:37 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:37:42 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:37:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:37:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:37:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:02 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:07 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:38:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:17 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:22 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:38:32 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:37 bookstack:ok4/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:38:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:52 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:38:57 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:39:02 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:39:07 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:39:12 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:39:17 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:39:22 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:39:27 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:39:32 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:39:37 bookstack:ok5/fail0 mealie:ok4/fail0 nextcloud:ok2/fail0
|
||||
19:39:42 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:39:47 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok1/fail0
|
||||
19:39:52 bookstack:ok4/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
19:39:57 bookstack:ok5/fail0 mealie:ok5/fail0 nextcloud:ok2/fail0
|
||||
Reference in New Issue
Block a user