diff --git a/documentation/tests/campaign10-evidence-2026-07-31/r117-q7-abort-in-place.md b/documentation/tests/campaign10-evidence-2026-07-31/r117-q7-abort-in-place.md new file mode 100644 index 0000000..3a8a140 --- /dev/null +++ b/documentation/tests/campaign10-evidence-2026-07-31/r117-q7-abort-in-place.md @@ -0,0 +1,53 @@ +# R-117 Q7 — filesystem aborted IN PLACE, device present: PASS (2026-08-01) + +The case R-117 was filed for and the one its spike called *"the worse half"*: a drive dies with **no +detach/return cycle at all**, so the gate never runs and — before agent v0.117.0 — *"nothing is emitted +on any channel, indefinitely"* (R-117a). + +Box runs **agent 0.119.0**, so the v0.117.0 `bindUsable` fix is in. Proven by hand before automation. + +## Method + +The spike used a `dmsetup` error table; these drives are raw SCSI passthrough, so the equivalent state +was produced with ext4's own abort, which lands in exactly the vocabulary `bindLiveness` matches +(`shutdown` / `emergency_ro`): + + # mount -o remount,abort /mnt/adatok + /dev/sde /mnt/adatok ext4 rw,relatime,abort,emergency_ro 0 0 <-- fs dead + lsblk: sda sde sdf sr0 <-- DEVICE STILL PRESENT + +## Result — it surfaces, and the gate acts + +Stable across three polls (~2 min), so not a transient: + +| Signal | Value | Meaning | +|---|---|---| +| `bound_under_parent` | **false** | the abort surfaced — the fix's stated mechanism | +| `state` | `attached` | correct: the drive *is* still physically there | +| hub event | `storage_disconnected (error) — Meghajtó váratlanul leválasztva: Adatok` | **an event fired** — R-117a's silence is gone | +| storage page | `Adatok … Leválasztva … **Leállított alkalmazások: calibre-web**` | the customer is told which app was stopped | +| `calibre-web` | **not running** | the gate STOPPED the app on the dead namespace rather than restarting it onto one | + +`calibre-web`'s library is a bind onto this drive +(`/mnt/felhom-drives/adatok/userdata/media/books → /calibre-library`), which is why it is the app that +must stop. Before the fix this state left it running on a namespace that `EIO`s every byte. + +## Recovery — a remount is NOT enough, exactly as the fix intends + +`umount -l` + `mount UUID=…` left `abort,emergency_ro` in `/proc/mounts`: the lazy umount leaves the +aborted superblock held open by the stale bind (the spike hit the same thing as +`dmsetup remove --force` → *Device or resource busy*). The agent deliberately does **not** auto-repair +here — `BindAborted ⇒ quiet no-op`, because a re-bind lands on the same aborted superblock and this +runs every 20 s, so repairing would be an infinite silent retry. Clearing it is an operator decision. + +A full device close — detach and re-attach the virtual disk, i.e. re-seating the drive — recovered it +in one agent tick: + + /dev/sdb /mnt/adatok ext4 rw,relatime <-- abort flags gone, letter recycled after the close + bound_under_parent = True ; calibre-web = Up + +The recycled device letter matches the spike's own observation that releasing the bind lets the letter +be reused. + +**Verdict: I4's second clause ("an aborted filesystem surfaces and does not silently retry") and I3 +("no app is ever restarted onto a dead namespace") both hold on the hardest case in the register.** diff --git a/documentation/tests/campaign10-evidence-2026-07-31/state/journal-run1.tsv b/documentation/tests/campaign10-evidence-2026-07-31/state/journal-run1.tsv new file mode 100644 index 0000000..8a4d97c --- /dev/null +++ b/documentation/tests/campaign10-evidence-2026-07-31/state/journal-run1.tsv @@ -0,0 +1,587 @@ +cycle utc invariant verdict detail +1 2026-08-01T07:20:12Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +1 2026-08-01T07:20:15Z I3 PASS adatok bound_under_parent=False while absent +1 2026-08-01T07:22:27Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +1 2026-08-01T07:22:30Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +1 2026-08-01T07:23:14Z REDEPLOY PASS grafana redeployed +1 2026-08-01T07:23:56Z BACKUP PASS {"db_dump": {"count": 1, "duration": "29.235715924s", "last_run": "2026-08-01T07:23:45.622922949Z", "success": true}, "enabled": true, "running": false} +1 2026-08-01T07:24:48Z BACKUP PASS {"db_dump": {"count": 1, "duration": "34.037544623s", "last_run": "2026-08-01T07:24:42.576575317Z", "success": true}, "enabled": true, "running": false} +1 2026-08-01T07:25:51Z I7 PASS app=rallly want=C10-C001-A-092356 got=C10-C001-A-092356 snap=helyi restore_ok=True rto=42.3s +1 2026-08-01T07:28:07Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +1 2026-08-01T07:28:10Z I3 PASS mentes bound_under_parent=False while absent +1 2026-08-01T07:30:22Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +1 2026-08-01T07:30:25Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +1 2026-08-01T07:31:14Z KILL-CTRL PASS controller recovered after kill +1 2026-08-01T07:31:17Z I5 PASS degraded=False absent_events=0 target=felhom-backup +1 2026-08-01T07:31:17Z I6 PASS healthy state must carry no degraded message; got=None +1 2026-08-01T07:31:19Z I11 PASS leaked=[] +1 2026-08-01T07:35:53Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +1 2026-08-01T07:35:56Z I3 PASS adatok bound_under_parent=False while absent +1 2026-08-01T07:38:09Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +1 2026-08-01T07:38:12Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +1 2026-08-01T07:38:56Z REDEPLOY PASS homebox redeployed +1 2026-08-01T07:39:38Z BACKUP PASS {"db_dump": {"count": 1, "duration": "29.454266341s", "last_run": "2026-08-01T07:39:28.065882881Z", "success": true}, "enabled": true, "running": false} +1 2026-08-01T07:40:31Z BACKUP PASS {"db_dump": {"count": 1, "duration": "32.574200326s", "last_run": "2026-08-01T07:40:23.804874002Z", "success": true}, "enabled": true, "running": false} +1 2026-08-01T07:41:34Z I7 PASS app=rallly want=C10-C001-A-093938 got=C10-C001-A-093938 snap=helyi restore_ok=True rto=41.8s +1 2026-08-01T07:43:49Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +1 2026-08-01T07:43:52Z I3 PASS mentes bound_under_parent=False while absent +1 2026-08-01T07:46:05Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +1 2026-08-01T07:46:07Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +1 2026-08-01T07:46:56Z KILL-CTRL PASS controller recovered after kill +1 2026-08-01T07:46:59Z I5 PASS degraded=False absent_events=0 target=felhom-backup +1 2026-08-01T07:46:59Z I6 PASS healthy state must carry no degraded message; got=None +1 2026-08-01T07:47:01Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +1 2026-08-01T07:47:01Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +1 2026-08-01T07:47:03Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +1 2026-08-01T07:47:05Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +1 2026-08-01T07:47:07Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +1 2026-08-01T07:47:09Z I11 PASS leaked=[] +2 2026-08-01T07:48:01Z KILL-CTRL PASS controller recovered after kill +2 2026-08-01T07:50:16Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +2 2026-08-01T07:50:19Z I3 PASS adatok bound_under_parent=False while absent +2 2026-08-01T07:52:32Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +2 2026-08-01T07:52:35Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +2 2026-08-01T07:53:17Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.724847172s", "last_run": "2026-08-01T07:53:05.968779815Z", "success": true}, "enabled": true, "running": false} +2 2026-08-01T07:54:01Z REDEPLOY PASS grafana redeployed +2 2026-08-01T07:54:54Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.880675052s", "last_run": "2026-08-01T07:54:43.377582192Z", "success": true}, "enabled": true, "running": false} +2 2026-08-01T07:55:56Z I7 PASS app=rallly want=C10-C002-A-095401 got=C10-C002-A-095401 snap=helyi restore_ok=True rto=41.7s +2 2026-08-01T07:58:12Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +2 2026-08-01T07:58:14Z I3 PASS mentes bound_under_parent=False while absent +2 2026-08-01T08:00:27Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +2 2026-08-01T08:00:30Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +2 2026-08-01T08:00:32Z I5 PASS degraded=False absent_events=0 target=felhom-backup +2 2026-08-01T08:00:32Z I6 PASS healthy state must carry no degraded message; got=None +2 2026-08-01T08:00:34Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +2 2026-08-01T08:00:34Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +2 2026-08-01T08:00:36Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +2 2026-08-01T08:00:38Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +2 2026-08-01T08:00:40Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +2 2026-08-01T08:00:42Z I11 PASS leaked=[] +3 2026-08-01T08:03:00Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +3 2026-08-01T08:03:03Z I3 PASS mentes bound_under_parent=False while absent +3 2026-08-01T08:05:15Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +3 2026-08-01T08:05:18Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +3 2026-08-01T08:06:02Z REDEPLOY PASS homebox redeployed +3 2026-08-01T08:06:44Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.249083164s", "last_run": "2026-08-01T08:06:32.499808867Z", "success": true}, "enabled": true, "running": false} +3 2026-08-01T08:09:00Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +3 2026-08-01T08:09:02Z I3 PASS adatok bound_under_parent=False while absent +3 2026-08-01T08:11:15Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +3 2026-08-01T08:11:18Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +3 2026-08-01T08:12:07Z KILL-CTRL PASS controller recovered after kill +3 2026-08-01T08:13:00Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.907036117s", "last_run": "2026-08-01T08:12:49.360548351Z", "success": true}, "enabled": true, "running": false} +3 2026-08-01T08:14:04Z I7 PASS app=rallly want=C10-C003-A-101207 got=C10-C003-A-101207 snap=helyi restore_ok=True rto=44.3s +3 2026-08-01T08:14:07Z I5 PASS degraded=False absent_events=0 target=felhom-backup +3 2026-08-01T08:14:07Z I6 PASS healthy state must carry no degraded message; got=None +3 2026-08-01T08:14:09Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +3 2026-08-01T08:14:09Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +3 2026-08-01T08:14:11Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +3 2026-08-01T08:14:13Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +3 2026-08-01T08:14:15Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +3 2026-08-01T08:14:17Z I11 PASS leaked=[] +4 2026-08-01T08:16:35Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +4 2026-08-01T08:16:38Z I3 PASS adatok bound_under_parent=False while absent +4 2026-08-01T08:18:51Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +4 2026-08-01T08:18:54Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +4 2026-08-01T08:19:35Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.425451162s", "last_run": "2026-08-01T08:19:24.403091262Z", "success": true}, "enabled": true, "running": false} +4 2026-08-01T08:20:20Z REDEPLOY PASS grafana redeployed +4 2026-08-01T08:21:12Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.411530632s", "last_run": "2026-08-01T08:21:00.742317925Z", "success": true}, "enabled": true, "running": false} +4 2026-08-01T08:22:14Z I7 PASS app=rallly want=C10-C004-A-102020 got=C10-C004-A-102020 snap=helyi restore_ok=True rto=42.0s +4 2026-08-01T08:24:30Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +4 2026-08-01T08:24:32Z I3 PASS mentes bound_under_parent=False while absent +4 2026-08-01T08:26:45Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +4 2026-08-01T08:26:48Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +4 2026-08-01T08:27:37Z KILL-CTRL PASS controller recovered after kill +4 2026-08-01T08:27:39Z I5 PASS degraded=False absent_events=0 target=felhom-backup +4 2026-08-01T08:27:39Z I6 PASS healthy state must carry no degraded message; got=None +4 2026-08-01T08:27:41Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +4 2026-08-01T08:27:41Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +4 2026-08-01T08:27:43Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +4 2026-08-01T08:27:46Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +4 2026-08-01T08:27:47Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +4 2026-08-01T08:27:50Z I11 PASS leaked=[] +5 2026-08-01T08:28:36Z REDEPLOY PASS grafana redeployed +5 2026-08-01T08:30:51Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +5 2026-08-01T08:30:54Z I3 PASS adatok bound_under_parent=False while absent +5 2026-08-01T08:33:07Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +5 2026-08-01T08:33:10Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +5 2026-08-01T08:33:59Z KILL-CTRL PASS controller recovered after kill +5 2026-08-01T08:36:15Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +5 2026-08-01T08:36:18Z I3 PASS mentes bound_under_parent=False while absent +5 2026-08-01T08:38:30Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +5 2026-08-01T08:38:33Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +5 2026-08-01T08:39:15Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.537566027s", "last_run": "2026-08-01T08:39:03.884190323Z", "success": true}, "enabled": true, "running": false} +5 2026-08-01T08:40:08Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.819105251s", "last_run": "2026-08-01T08:39:56.870257525Z", "success": true}, "enabled": true, "running": false} +5 2026-08-01T08:41:13Z I7 PASS app=rallly want=C10-C005-A-103915 got=C10-C005-A-103915 snap=helyi restore_ok=True rto=43.0s +5 2026-08-01T08:41:16Z I5 PASS degraded=False absent_events=0 target=felhom-backup +5 2026-08-01T08:41:16Z I6 PASS healthy state must carry no degraded message; got=None +5 2026-08-01T08:41:18Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +5 2026-08-01T08:41:18Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +5 2026-08-01T08:41:20Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +5 2026-08-01T08:41:22Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +5 2026-08-01T08:41:24Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +5 2026-08-01T08:41:26Z I11 PASS leaked=[] +6 2026-08-01T08:42:18Z KILL-CTRL PASS controller recovered after kill +6 2026-08-01T08:43:10Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.890401885s", "last_run": "2026-08-01T08:42:58.392385368Z", "success": true}, "enabled": true, "running": false} +6 2026-08-01T08:44:12Z I7 PASS app=rallly want=C10-C006-A-104218 got=C10-C006-A-104218 snap=helyi restore_ok=True rto=41.6s +6 2026-08-01T08:46:28Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +6 2026-08-01T08:46:30Z I3 PASS mentes bound_under_parent=False while absent +6 2026-08-01T08:48:44Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +6 2026-08-01T08:48:46Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +6 2026-08-01T08:51:02Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +6 2026-08-01T08:51:05Z I3 PASS adatok bound_under_parent=False while absent +6 2026-08-01T08:53:18Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +6 2026-08-01T08:53:20Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +6 2026-08-01T08:54:02Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.101250195s", "last_run": "2026-08-01T08:53:50.441271314Z", "success": true}, "enabled": true, "running": false} +6 2026-08-01T08:54:46Z REDEPLOY PASS grafana redeployed +6 2026-08-01T08:54:49Z I5 PASS degraded=False absent_events=0 target=felhom-backup +6 2026-08-01T08:54:49Z I6 PASS healthy state must carry no degraded message; got=None +6 2026-08-01T08:54:51Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +6 2026-08-01T08:54:51Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +6 2026-08-01T08:54:53Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +6 2026-08-01T08:54:55Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +6 2026-08-01T08:54:57Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +6 2026-08-01T08:54:59Z I11 PASS leaked=[] +7 2026-08-01T08:57:17Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +7 2026-08-01T08:57:20Z I3 PASS adatok bound_under_parent=False while absent +7 2026-08-01T08:59:33Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +7 2026-08-01T08:59:35Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +7 2026-08-01T09:01:50Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +7 2026-08-01T09:01:54Z I3 PASS mentes bound_under_parent=False while absent +7 2026-08-01T09:04:07Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +7 2026-08-01T09:04:09Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) +7 2026-08-01T09:04:58Z KILL-CTRL PASS controller recovered after kill +7 2026-08-01T09:05:43Z REDEPLOY PASS homebox redeployed +7 2026-08-01T09:06:35Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.192784423s", "last_run": "2026-08-01T09:06:24.336193131Z", "success": true}, "enabled": true, "running": false} +7 2026-08-01T09:07:38Z I7 PASS app=rallly want=C10-C007-A-110543 got=C10-C007-A-110543 snap=helyi restore_ok=True rto=42.2s +7 2026-08-01T09:08:21Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.754755491s", "last_run": "2026-08-01T09:08:08.67970607Z", "success": true}, "enabled": true, "running": false} +7 2026-08-01T09:09:00Z REBOOT PASS controller back after guest reboot +7 2026-08-01T09:09:03Z I5 PASS degraded=False absent_events=0 target=felhom-backup +7 2026-08-01T09:09:03Z I6 PASS healthy state must carry no degraded message; got=None +7 2026-08-01T09:09:05Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +7 2026-08-01T09:09:05Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +7 2026-08-01T09:09:07Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +7 2026-08-01T09:09:09Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +7 2026-08-01T09:09:11Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +7 2026-08-01T09:09:13Z I11 PASS leaked=[] +8 2026-08-01T09:09:59Z REDEPLOY PASS homebox redeployed +8 2026-08-01T09:10:48Z KILL-CTRL PASS controller recovered after kill +8 2026-08-01T09:11:41Z BACKUP PASS {"db_dump": {"count": 1, "duration": "29.417683334s", "last_run": "2026-08-01T09:11:30.326680315Z", "success": true}, "enabled": true, "running": false} +8 2026-08-01T09:12:45Z I7 PASS app=rallly want=C10-C008-A-111048 got=C10-C008-A-111048 snap=helyi restore_ok=True rto=42.3s +8 2026-08-01T09:15:01Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +8 2026-08-01T09:15:04Z I3 PASS adatok bound_under_parent=False while absent +8 2026-08-01T09:17:16Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +8 2026-08-01T09:17:19Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +8 2026-08-01T09:19:35Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +8 2026-08-01T09:19:38Z I3 PASS mentes bound_under_parent=False while absent +8 2026-08-01T09:21:51Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +8 2026-08-01T09:21:53Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +8 2026-08-01T09:22:35Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.112766197s", "last_run": "2026-08-01T09:22:23.959739494Z", "success": true}, "enabled": true, "running": false} +8 2026-08-01T09:22:38Z I5 PASS degraded=False absent_events=0 target=felhom-backup +8 2026-08-01T09:22:38Z I6 PASS healthy state must carry no degraded message; got=None +8 2026-08-01T09:22:40Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +8 2026-08-01T09:22:40Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +8 2026-08-01T09:22:42Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +8 2026-08-01T09:22:44Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +8 2026-08-01T09:22:46Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +8 2026-08-01T09:22:48Z I11 PASS leaked=[] +9 2026-08-01T09:23:39Z KILL-CTRL PASS controller recovered after kill +9 2026-08-01T09:24:19Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.120915244s", "last_run": "2026-08-01T09:24:19.437409908Z", "success": true}, "enabled": true, "running": false} +9 2026-08-01T09:25:22Z I7 PASS app=rallly want=C10-C009-A-112339 got=C10-C009-A-112339 snap=helyi restore_ok=True rto=41.9s +9 2026-08-01T09:26:06Z REDEPLOY PASS grafana redeployed +9 2026-08-01T09:28:22Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +9 2026-08-01T09:28:25Z I3 PASS adatok bound_under_parent=False while absent +9 2026-08-01T09:30:38Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +9 2026-08-01T09:30:40Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +9 2026-08-01T09:31:10Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.234736705s", "last_run": "2026-08-01T09:31:09.977587344Z", "success": true}, "enabled": true, "running": false} +9 2026-08-01T09:33:26Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +9 2026-08-01T09:33:29Z I3 PASS mentes bound_under_parent=False while absent +9 2026-08-01T09:35:42Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +9 2026-08-01T09:35:44Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +9 2026-08-01T09:35:47Z I5 PASS degraded=False absent_events=0 target=felhom-backup +9 2026-08-01T09:35:47Z I6 PASS healthy state must carry no degraded message; got=None +9 2026-08-01T09:35:49Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +9 2026-08-01T09:35:49Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +9 2026-08-01T09:35:51Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +9 2026-08-01T09:35:53Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +9 2026-08-01T09:35:55Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +9 2026-08-01T09:35:57Z I11 PASS leaked=[] +10 2026-08-01T09:38:14Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +10 2026-08-01T09:38:17Z I3 PASS adatok bound_under_parent=False while absent +10 2026-08-01T09:40:30Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +10 2026-08-01T09:40:33Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +10 2026-08-01T09:41:17Z REDEPLOY PASS homebox redeployed +10 2026-08-01T09:42:06Z KILL-CTRL PASS controller recovered after kill +10 2026-08-01T09:44:22Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +10 2026-08-01T09:44:25Z I3 PASS mentes bound_under_parent=False while absent +10 2026-08-01T09:46:38Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +10 2026-08-01T09:46:40Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +10 2026-08-01T09:47:21Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.035320075s", "last_run": "2026-08-01T09:47:21.013177918Z", "success": true}, "enabled": true, "running": false} +10 2026-08-01T09:48:24Z I7 PASS app=rallly want=C10-C010-A-114640 got=C10-C010-A-114640 snap=helyi restore_ok=True rto=41.6s +10 2026-08-01T09:48:54Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.198265248s", "last_run": "2026-08-01T09:48:54.018758879Z", "success": true}, "enabled": true, "running": false} +10 2026-08-01T09:48:57Z I5 PASS degraded=False absent_events=0 target=felhom-backup +10 2026-08-01T09:48:57Z I6 PASS healthy state must carry no degraded message; got=None +10 2026-08-01T09:48:59Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +10 2026-08-01T09:48:59Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +10 2026-08-01T09:49:01Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +10 2026-08-01T09:49:03Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +10 2026-08-01T09:49:05Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +10 2026-08-01T09:49:07Z I11 PASS leaked=[] +11 2026-08-01T09:50:02Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.500198373s", "last_run": "2026-08-01T09:49:50.111560186Z", "success": true}, "enabled": true, "running": false} +11 2026-08-01T09:51:05Z I7 PASS app=rallly want=C10-C011-A-114910 got=C10-C011-A-114910 snap=helyi restore_ok=True rto=41.7s +11 2026-08-01T09:53:20Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +11 2026-08-01T09:53:23Z I3 PASS adatok bound_under_parent=False while absent +11 2026-08-01T09:55:35Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +11 2026-08-01T09:55:38Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +11 2026-08-01T09:57:54Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +11 2026-08-01T09:57:57Z I3 PASS mentes bound_under_parent=False while absent +11 2026-08-01T10:00:10Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +11 2026-08-01T10:00:13Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +11 2026-08-01T10:00:55Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.274827764s", "last_run": "2026-08-01T10:00:42.619398629Z", "success": true}, "enabled": true, "running": false} +11 2026-08-01T10:01:38Z REDEPLOY PASS homebox redeployed +11 2026-08-01T10:02:28Z KILL-CTRL PASS controller recovered after kill +11 2026-08-01T10:02:30Z I5 PASS degraded=False absent_events=0 target=felhom-backup +11 2026-08-01T10:02:30Z I6 PASS healthy state must carry no degraded message; got=None +11 2026-08-01T10:02:32Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +11 2026-08-01T10:02:32Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +11 2026-08-01T10:02:34Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +11 2026-08-01T10:02:36Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +11 2026-08-01T10:02:38Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +11 2026-08-01T10:02:40Z I11 PASS leaked=[] +12 2026-08-01T10:03:26Z REDEPLOY PASS homebox redeployed +12 2026-08-01T10:05:42Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +12 2026-08-01T10:05:45Z I3 PASS adatok bound_under_parent=False while absent +12 2026-08-01T10:07:57Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +12 2026-08-01T10:08:00Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +12 2026-08-01T10:08:49Z KILL-CTRL PASS controller recovered after kill +12 2026-08-01T10:11:04Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +12 2026-08-01T10:11:07Z I3 PASS mentes bound_under_parent=False while absent +12 2026-08-01T10:13:20Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +12 2026-08-01T10:13:24Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +12 2026-08-01T10:13:54Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.82906082s", "last_run": "2026-08-01T10:13:53.343511281Z", "success": true}, "enabled": true, "running": false} +12 2026-08-01T10:14:47Z BACKUP PASS {"db_dump": {"count": 1, "duration": "34.794492507s", "last_run": "2026-08-01T10:14:42.108803281Z", "success": true}, "enabled": true, "running": false} +12 2026-08-01T10:15:50Z I7 PASS app=rallly want=C10-C012-A-121354 got=C10-C012-A-121354 snap=helyi restore_ok=True rto=41.6s +12 2026-08-01T10:15:53Z I5 PASS degraded=False absent_events=0 target=felhom-backup +12 2026-08-01T10:15:53Z I6 PASS healthy state must carry no degraded message; got=None +12 2026-08-01T10:15:54Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +12 2026-08-01T10:15:54Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +12 2026-08-01T10:15:56Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +12 2026-08-01T10:15:58Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +12 2026-08-01T10:16:00Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +12 2026-08-01T10:16:02Z I11 PASS leaked=[] +13 2026-08-01T10:16:54Z KILL-CTRL PASS controller recovered after kill +13 2026-08-01T10:17:38Z REDEPLOY PASS grafana redeployed +13 2026-08-01T10:18:18Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.066037384s", "last_run": "2026-08-01T10:18:18.41231692Z", "success": true}, "enabled": true, "running": false} +13 2026-08-01T10:19:22Z I7 PASS app=rallly want=C10-C013-A-121738 got=C10-C013-A-121738 snap=helyi restore_ok=True rto=42.2s +13 2026-08-01T10:20:03Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.463205132s", "last_run": "2026-08-01T10:19:51.680188929Z", "success": true}, "enabled": true, "running": false} +13 2026-08-01T10:22:19Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +13 2026-08-01T10:22:22Z I3 PASS mentes bound_under_parent=False while absent +13 2026-08-01T10:24:35Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +13 2026-08-01T10:24:37Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +13 2026-08-01T10:26:53Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +13 2026-08-01T10:26:56Z I3 PASS adatok bound_under_parent=False while absent +13 2026-08-01T10:29:09Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +13 2026-08-01T10:29:12Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +13 2026-08-01T10:29:14Z I5 PASS degraded=False absent_events=0 target=felhom-backup +13 2026-08-01T10:29:14Z I6 PASS healthy state must carry no degraded message; got=None +13 2026-08-01T10:29:16Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +13 2026-08-01T10:29:16Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +13 2026-08-01T10:29:18Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +13 2026-08-01T10:29:20Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +13 2026-08-01T10:29:22Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +13 2026-08-01T10:29:25Z I11 PASS leaked=[] +14 2026-08-01T10:29:56Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.80615982s", "last_run": "2026-08-01T10:29:55.907589946Z", "success": true}, "enabled": true, "running": false} +14 2026-08-01T10:30:50Z BACKUP PASS {"db_dump": {"count": 1, "duration": "33.858181263s", "last_run": "2026-08-01T10:30:43.782817699Z", "success": true}, "enabled": true, "running": false} +14 2026-08-01T10:31:54Z I7 PASS app=rallly want=C10-C014-A-122956 got=C10-C014-A-122956 snap=helyi restore_ok=True rto=42.4s +14 2026-08-01T10:34:09Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +14 2026-08-01T10:34:12Z I3 PASS adatok bound_under_parent=False while absent +14 2026-08-01T10:36:25Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +14 2026-08-01T10:36:28Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +14 2026-08-01T10:37:12Z REDEPLOY PASS grafana redeployed +14 2026-08-01T10:38:01Z KILL-CTRL PASS controller recovered after kill +14 2026-08-01T10:40:17Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +14 2026-08-01T10:40:20Z I3 PASS mentes bound_under_parent=False while absent +14 2026-08-01T10:42:33Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +14 2026-08-01T10:42:36Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) +14 2026-08-01T10:43:14Z REBOOT PASS controller back after guest reboot +14 2026-08-01T10:43:17Z I5 PASS degraded=False absent_events=0 target=felhom-backup +14 2026-08-01T10:43:17Z I6 PASS healthy state must carry no degraded message; got=None +14 2026-08-01T10:43:19Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +14 2026-08-01T10:43:19Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +14 2026-08-01T10:43:21Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +14 2026-08-01T10:43:24Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +14 2026-08-01T10:43:26Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +14 2026-08-01T10:43:28Z I11 PASS leaked=[] +15 2026-08-01T10:45:45Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +15 2026-08-01T10:45:48Z I3 PASS mentes bound_under_parent=False while absent +15 2026-08-01T10:48:01Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +15 2026-08-01T10:48:03Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +15 2026-08-01T10:48:46Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.270428557s", "last_run": "2026-08-01T10:48:34.163245256Z", "success": true}, "enabled": true, "running": false} +15 2026-08-01T10:49:36Z KILL-CTRL PASS controller recovered after kill +15 2026-08-01T10:50:20Z REDEPLOY PASS homebox redeployed +15 2026-08-01T10:51:01Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.421623567s", "last_run": "2026-08-01T10:51:00.95301014Z", "success": true}, "enabled": true, "running": false} +15 2026-08-01T10:52:04Z I7 PASS app=rallly want=C10-C015-A-125020 got=C10-C015-A-125020 snap=helyi restore_ok=True rto=41.8s +15 2026-08-01T10:54:20Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +15 2026-08-01T10:54:23Z I3 PASS adatok bound_under_parent=False while absent +15 2026-08-01T10:56:36Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +15 2026-08-01T10:56:39Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +15 2026-08-01T10:56:42Z I5 PASS degraded=False absent_events=0 target=felhom-backup +15 2026-08-01T10:56:42Z I6 PASS healthy state must carry no degraded message; got=None +15 2026-08-01T10:56:45Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +15 2026-08-01T10:56:45Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +15 2026-08-01T10:56:47Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +15 2026-08-01T10:56:49Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +15 2026-08-01T10:56:51Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +15 2026-08-01T10:56:54Z I11 PASS leaked=[] +16 2026-08-01T10:57:27Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.683550606s", "last_run": "2026-08-01T10:57:26.606568663Z", "success": true}, "enabled": true, "running": false} +16 2026-08-01T10:59:43Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +16 2026-08-01T10:59:48Z I3 PASS mentes bound_under_parent=False while absent +16 2026-08-01T11:02:02Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +16 2026-08-01T11:02:08Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +16 2026-08-01T11:04:31Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +16 2026-08-01T11:04:40Z I3 PASS adatok bound_under_parent=False while absent +16 2026-08-01T11:06:58Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +16 2026-08-01T11:07:01Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +16 2026-08-01T11:08:29Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.894128293s", "last_run": "2026-08-01T11:08:14.025870917Z", "success": true}, "enabled": true, "running": false} +16 2026-08-01T11:09:59Z I7 PASS app=rallly want=C10-C016-A-130701 got=C10-C016-A-130701 snap=helyi restore_ok=True rto=38.8s +16 2026-08-01T11:10:54Z KILL-CTRL PASS controller recovered after kill +16 2026-08-01T11:11:53Z REDEPLOY PASS grafana redeployed +16 2026-08-01T11:12:03Z I5 PASS degraded=False absent_events=0 target=felhom-backup +16 2026-08-01T11:12:03Z I6 PASS healthy state must carry no degraded message; got=None +16 2026-08-01T11:12:11Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +16 2026-08-01T11:12:11Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +16 2026-08-01T11:12:19Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +16 2026-08-01T11:12:27Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +16 2026-08-01T11:12:36Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +16 2026-08-01T11:12:43Z I11 PASS leaked=[] +17 2026-08-01T11:13:43Z REDEPLOY PASS grafana redeployed +17 2026-08-01T11:16:04Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +17 2026-08-01T11:16:08Z I3 PASS adatok bound_under_parent=False while absent +17 2026-08-01T11:18:21Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +17 2026-08-01T11:18:23Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +17 2026-08-01T11:18:53Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.838575753s", "last_run": "2026-08-01T11:18:52.693573505Z", "success": true}, "enabled": true, "running": false} +17 2026-08-01T11:19:42Z KILL-CTRL PASS controller recovered after kill +17 2026-08-01T11:20:23Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.317455253s", "last_run": "2026-08-01T11:20:23.002386465Z", "success": true}, "enabled": true, "running": false} +17 2026-08-01T11:21:26Z I7 PASS app=rallly want=C10-C017-A-131942 got=C10-C017-A-131942 snap=helyi restore_ok=True rto=41.5s +17 2026-08-01T11:23:42Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +17 2026-08-01T11:23:45Z I3 PASS mentes bound_under_parent=False while absent +17 2026-08-01T11:25:57Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +17 2026-08-01T11:26:00Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +17 2026-08-01T11:26:02Z I5 PASS degraded=False absent_events=0 target=felhom-backup +17 2026-08-01T11:26:02Z I6 PASS healthy state must carry no degraded message; got=None +17 2026-08-01T11:26:04Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +17 2026-08-01T11:26:04Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +17 2026-08-01T11:26:06Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +17 2026-08-01T11:26:08Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +17 2026-08-01T11:26:10Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +17 2026-08-01T11:26:12Z I11 PASS leaked=[] +18 2026-08-01T11:28:30Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +18 2026-08-01T11:28:33Z I3 PASS mentes bound_under_parent=False while absent +18 2026-08-01T11:30:45Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +18 2026-08-01T11:30:48Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +18 2026-08-01T11:31:38Z KILL-CTRL PASS controller recovered after kill +18 2026-08-01T11:33:53Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +18 2026-08-01T11:33:57Z I3 PASS adatok bound_under_parent=False while absent +18 2026-08-01T11:36:10Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +18 2026-08-01T11:36:12Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +18 2026-08-01T11:36:42Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.287771352s", "last_run": "2026-08-01T11:36:41.854058541Z", "success": true}, "enabled": true, "running": false} +18 2026-08-01T11:37:27Z REDEPLOY PASS grafana redeployed +18 2026-08-01T11:38:08Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.269077889s", "last_run": "2026-08-01T11:38:07.785076049Z", "success": true}, "enabled": true, "running": false} +18 2026-08-01T11:39:12Z I7 PASS app=rallly want=C10-C018-A-133727 got=C10-C018-A-133727 snap=helyi restore_ok=True rto=42.0s +18 2026-08-01T11:39:15Z I5 PASS degraded=False absent_events=0 target=felhom-backup +18 2026-08-01T11:39:15Z I6 PASS healthy state must carry no degraded message; got=None +18 2026-08-01T11:39:17Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +18 2026-08-01T11:39:17Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +18 2026-08-01T11:39:19Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +18 2026-08-01T11:39:21Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +18 2026-08-01T11:39:24Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +18 2026-08-01T11:39:26Z I11 PASS leaked=[] +19 2026-08-01T11:40:18Z KILL-CTRL PASS controller recovered after kill +19 2026-08-01T11:41:02Z REDEPLOY PASS homebox redeployed +19 2026-08-01T11:41:44Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.240196615s", "last_run": "2026-08-01T11:41:43.294094532Z", "success": true}, "enabled": true, "running": false} +19 2026-08-01T11:42:48Z I7 PASS app=rallly want=C10-C019-A-134102 got=C10-C019-A-134102 snap=helyi restore_ok=True rto=42.1s +19 2026-08-01T11:45:03Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +19 2026-08-01T11:45:06Z I3 PASS adatok bound_under_parent=False while absent +19 2026-08-01T11:47:20Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +19 2026-08-01T11:47:22Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +19 2026-08-01T11:49:38Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +19 2026-08-01T11:49:41Z I3 PASS mentes bound_under_parent=False while absent +19 2026-08-01T11:51:54Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +19 2026-08-01T11:51:56Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +19 2026-08-01T11:52:25Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.074516344s", "last_run": "2026-08-01T11:52:25.638816606Z", "success": true}, "enabled": true, "running": false} +19 2026-08-01T11:52:28Z I5 PASS degraded=False absent_events=0 target=felhom-backup +19 2026-08-01T11:52:28Z I6 PASS healthy state must carry no degraded message; got=None +19 2026-08-01T11:52:30Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +19 2026-08-01T11:52:30Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +19 2026-08-01T11:52:32Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +19 2026-08-01T11:52:34Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +19 2026-08-01T11:52:36Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +19 2026-08-01T11:52:39Z I11 PASS leaked=[] +20 2026-08-01T11:53:24Z BACKUP PASS {"db_dump": {"count": 1, "duration": "33.58133879s", "last_run": "2026-08-01T11:53:17.277403988Z", "success": true}, "enabled": true, "running": false} +20 2026-08-01T11:55:40Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +20 2026-08-01T11:55:43Z I3 PASS mentes bound_under_parent=False while absent +20 2026-08-01T11:57:56Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +20 2026-08-01T11:57:58Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +20 2026-08-01T12:00:14Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +20 2026-08-01T12:00:17Z I3 PASS adatok bound_under_parent=False while absent +20 2026-08-01T12:02:30Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +20 2026-08-01T12:02:33Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +20 2026-08-01T12:03:17Z REDEPLOY PASS grafana redeployed +20 2026-08-01T12:03:58Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.633748751s", "last_run": "2026-08-01T12:03:57.032814639Z", "success": true}, "enabled": true, "running": false} +20 2026-08-01T12:05:02Z I7 PASS app=rallly want=C10-C020-A-140317 got=C10-C020-A-140317 snap=helyi restore_ok=True rto=42.2s +20 2026-08-01T12:05:52Z KILL-CTRL PASS controller recovered after kill +20 2026-08-01T12:05:55Z I5 PASS degraded=False absent_events=0 target=felhom-backup +20 2026-08-01T12:05:55Z I6 PASS healthy state must carry no degraded message; got=None +20 2026-08-01T12:05:57Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +20 2026-08-01T12:05:57Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +20 2026-08-01T12:05:59Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +20 2026-08-01T12:06:01Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +20 2026-08-01T12:06:03Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +20 2026-08-01T12:06:05Z I11 PASS leaked=[] +21 2026-08-01T12:06:37Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.742931297s", "last_run": "2026-08-01T12:06:36.352784544Z", "success": true}, "enabled": true, "running": false} +21 2026-08-01T12:07:26Z KILL-CTRL PASS controller recovered after kill +21 2026-08-01T12:09:42Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +21 2026-08-01T12:09:45Z I3 PASS adatok bound_under_parent=False while absent +21 2026-08-01T12:11:58Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +21 2026-08-01T12:12:01Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +21 2026-08-01T12:12:45Z REDEPLOY PASS homebox redeployed +21 2026-08-01T12:15:01Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +21 2026-08-01T12:15:04Z I3 PASS mentes bound_under_parent=False while absent +21 2026-08-01T12:17:17Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +21 2026-08-01T12:17:19Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) +21 2026-08-01T12:18:01Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.215452442s", "last_run": "2026-08-01T12:18:01.043730281Z", "success": true}, "enabled": true, "running": false} +21 2026-08-01T12:19:03Z I7 PASS app=rallly want=C10-C021-A-141719 got=C10-C021-A-141719 snap=helyi restore_ok=True rto=41.4s +21 2026-08-01T12:19:43Z REBOOT PASS controller back after guest reboot +21 2026-08-01T12:19:46Z I5 PASS degraded=False absent_events=0 target=felhom-backup +21 2026-08-01T12:19:46Z I6 PASS healthy state must carry no degraded message; got=None +21 2026-08-01T12:19:48Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +21 2026-08-01T12:19:48Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +21 2026-08-01T12:19:50Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +21 2026-08-01T12:19:52Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +21 2026-08-01T12:19:54Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +21 2026-08-01T12:19:56Z I11 PASS leaked=[] +22 2026-08-01T12:20:51Z BACKUP PASS {"db_dump": {"count": 1, "duration": "35.295073244s", "last_run": "2026-08-01T12:20:46.946104153Z", "success": true}, "enabled": true, "running": false} +22 2026-08-01T12:21:54Z I7 PASS app=rallly want=C10-C022-A-141958 got=C10-C022-A-141958 snap=helyi restore_ok=True rto=41.9s +22 2026-08-01T12:22:38Z REDEPLOY PASS homebox redeployed +22 2026-08-01T12:23:20Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.506763469s", "last_run": "2026-08-01T12:23:08.027088655Z", "success": true}, "enabled": true, "running": false} +22 2026-08-01T12:25:35Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +22 2026-08-01T12:25:38Z I3 PASS adatok bound_under_parent=False while absent +22 2026-08-01T12:27:51Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +22 2026-08-01T12:27:54Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +22 2026-08-01T12:28:43Z KILL-CTRL PASS controller recovered after kill +22 2026-08-01T12:30:58Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +22 2026-08-01T12:31:01Z I3 PASS mentes bound_under_parent=False while absent +22 2026-08-01T12:33:14Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +22 2026-08-01T12:33:17Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +22 2026-08-01T12:33:20Z I5 PASS degraded=False absent_events=0 target=felhom-backup +22 2026-08-01T12:33:20Z I6 PASS healthy state must carry no degraded message; got=None +22 2026-08-01T12:33:22Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +22 2026-08-01T12:33:22Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +22 2026-08-01T12:33:24Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +22 2026-08-01T12:33:26Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +22 2026-08-01T12:33:28Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +22 2026-08-01T12:33:30Z I11 PASS leaked=[] +23 2026-08-01T12:34:02Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.296490438s", "last_run": "2026-08-01T12:34:02.179324267Z", "success": true}, "enabled": true, "running": false} +23 2026-08-01T12:36:18Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +23 2026-08-01T12:36:21Z I3 PASS mentes bound_under_parent=False while absent +23 2026-08-01T12:38:34Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +23 2026-08-01T12:38:37Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +23 2026-08-01T12:39:17Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.836521386s", "last_run": "2026-08-01T12:39:17.164422568Z", "success": true}, "enabled": true, "running": false} +23 2026-08-01T12:40:21Z I7 PASS app=rallly want=C10-C023-A-143837 got=C10-C023-A-143837 snap=helyi restore_ok=True rto=41.9s +23 2026-08-01T12:41:11Z KILL-CTRL PASS controller recovered after kill +23 2026-08-01T12:41:56Z REDEPLOY PASS grafana redeployed +23 2026-08-01T12:44:11Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +23 2026-08-01T12:44:14Z I3 PASS adatok bound_under_parent=False while absent +23 2026-08-01T12:46:27Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +23 2026-08-01T12:46:30Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +23 2026-08-01T12:46:32Z I5 PASS degraded=False absent_events=0 target=felhom-backup +23 2026-08-01T12:46:32Z I6 PASS healthy state must carry no degraded message; got=None +23 2026-08-01T12:46:34Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +23 2026-08-01T12:46:34Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +23 2026-08-01T12:46:36Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +23 2026-08-01T12:46:38Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +23 2026-08-01T12:46:40Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +23 2026-08-01T12:46:43Z I11 PASS leaked=[] +24 2026-08-01T12:47:26Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.434160964s", "last_run": "2026-08-01T12:47:25.920976261Z", "success": true}, "enabled": true, "running": false} +24 2026-08-01T12:48:31Z I7 PASS app=rallly want=C10-C024-A-144645 got=C10-C024-A-144645 snap=helyi restore_ok=True rto=42.5s +24 2026-08-01T12:49:02Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.477758487s", "last_run": "2026-08-01T12:49:00.748171817Z", "success": true}, "enabled": true, "running": false} +24 2026-08-01T12:51:17Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +24 2026-08-01T12:51:20Z I3 PASS mentes bound_under_parent=False while absent +24 2026-08-01T12:53:33Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +24 2026-08-01T12:53:36Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +24 2026-08-01T12:54:25Z KILL-CTRL PASS controller recovered after kill +24 2026-08-01T12:56:40Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +24 2026-08-01T12:56:44Z I3 PASS adatok bound_under_parent=False while absent +24 2026-08-01T12:58:57Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +24 2026-08-01T12:59:00Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +24 2026-08-01T12:59:45Z REDEPLOY PASS grafana redeployed +24 2026-08-01T12:59:47Z I5 PASS degraded=False absent_events=0 target=felhom-backup +24 2026-08-01T12:59:47Z I6 PASS healthy state must carry no degraded message; got=None +24 2026-08-01T12:59:50Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +24 2026-08-01T12:59:50Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +24 2026-08-01T12:59:52Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +24 2026-08-01T12:59:54Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +24 2026-08-01T12:59:56Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +24 2026-08-01T12:59:59Z I11 PASS leaked=[] +25 2026-08-01T13:02:17Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +25 2026-08-01T13:02:20Z I3 PASS adatok bound_under_parent=False while absent +25 2026-08-01T13:04:33Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +25 2026-08-01T13:04:36Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +25 2026-08-01T13:05:26Z KILL-CTRL PASS controller recovered after kill +25 2026-08-01T13:07:41Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +25 2026-08-01T13:07:45Z I3 PASS mentes bound_under_parent=False while absent +25 2026-08-01T13:09:58Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +25 2026-08-01T13:10:00Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +25 2026-08-01T13:10:40Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.129870607s", "last_run": "2026-08-01T13:10:40.81939141Z", "success": true}, "enabled": true, "running": false} +25 2026-08-01T13:11:44Z I7 PASS app=rallly want=C10-C025-A-151000 got=C10-C025-A-151000 snap=helyi restore_ok=True rto=42.0s +25 2026-08-01T13:12:26Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.439198049s", "last_run": "2026-08-01T13:12:14.38822499Z", "success": true}, "enabled": true, "running": false} +25 2026-08-01T13:13:10Z REDEPLOY PASS homebox redeployed +25 2026-08-01T13:13:13Z I5 PASS degraded=False absent_events=0 target=felhom-backup +25 2026-08-01T13:13:13Z I6 PASS healthy state must carry no degraded message; got=None +25 2026-08-01T13:13:15Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +25 2026-08-01T13:13:15Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +25 2026-08-01T13:13:17Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +25 2026-08-01T13:13:19Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +25 2026-08-01T13:13:21Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +25 2026-08-01T13:13:24Z I11 PASS leaked=[] +26 2026-08-01T13:14:10Z REDEPLOY PASS grafana redeployed +26 2026-08-01T13:16:26Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +26 2026-08-01T13:16:29Z I3 PASS adatok bound_under_parent=False while absent +26 2026-08-01T13:18:42Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +26 2026-08-01T13:18:44Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +26 2026-08-01T13:19:25Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.181611353s", "last_run": "2026-08-01T13:19:24.6968582Z", "success": true}, "enabled": true, "running": false} +26 2026-08-01T13:20:31Z I7 PASS app=rallly want=C10-C026-A-151844 got=C10-C026-A-151844 snap=helyi restore_ok=True rto=42.9s +26 2026-08-01T13:21:20Z KILL-CTRL PASS controller recovered after kill +26 2026-08-01T13:23:36Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +26 2026-08-01T13:23:39Z I3 PASS mentes bound_under_parent=False while absent +26 2026-08-01T13:25:52Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +26 2026-08-01T13:25:55Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +26 2026-08-01T13:26:24Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.266368272s", "last_run": "2026-08-01T13:26:24.55559188Z", "success": true}, "enabled": true, "running": false} +26 2026-08-01T13:26:27Z I5 PASS degraded=False absent_events=0 target=felhom-backup +26 2026-08-01T13:26:27Z I6 PASS healthy state must carry no degraded message; got=None +26 2026-08-01T13:26:29Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +26 2026-08-01T13:26:29Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +26 2026-08-01T13:26:31Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +26 2026-08-01T13:26:33Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +26 2026-08-01T13:26:35Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +26 2026-08-01T13:26:37Z I11 PASS leaked=[] +27 2026-08-01T13:27:22Z BACKUP PASS {"db_dump": {"count": 1, "duration": "34.006310162s", "last_run": "2026-08-01T13:27:15.871339683Z", "success": true}, "enabled": true, "running": false} +27 2026-08-01T13:28:14Z BACKUP PASS {"db_dump": {"count": 1, "duration": "33.534168063s", "last_run": "2026-08-01T13:28:08.289273214Z", "success": true}, "enabled": true, "running": false} +27 2026-08-01T13:29:17Z I7 PASS app=rallly want=C10-C027-A-152722 got=C10-C027-A-152722 snap=helyi restore_ok=True rto=41.8s +27 2026-08-01T13:31:33Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) +27 2026-08-01T13:31:36Z I3 PASS mentes bound_under_parent=False while absent +27 2026-08-01T13:33:49Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) +27 2026-08-01T13:33:52Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +27 2026-08-01T13:34:41Z KILL-CTRL PASS controller recovered after kill +27 2026-08-01T13:36:57Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) +27 2026-08-01T13:37:00Z I3 PASS adatok bound_under_parent=False while absent +27 2026-08-01T13:39:14Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) +27 2026-08-01T13:39:16Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) +27 2026-08-01T13:40:00Z REDEPLOY PASS homebox redeployed +27 2026-08-01T13:40:03Z I5 PASS degraded=False absent_events=0 target=felhom-backup +27 2026-08-01T13:40:03Z I6 PASS healthy state must carry no degraded message; got=None +27 2026-08-01T13:40:04Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True +27 2026-08-01T13:40:04Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True +27 2026-08-01T13:40:07Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True +27 2026-08-01T13:40:09Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False +27 2026-08-01T13:40:11Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True +27 2026-08-01T13:40:13Z I11 PASS leaked=[] diff --git a/documentation/tests/campaign10-evidence-2026-07-31/state/journal.tsv b/documentation/tests/campaign10-evidence-2026-07-31/state/journal.tsv index 8a4d97c..c375132 100644 --- a/documentation/tests/campaign10-evidence-2026-07-31/state/journal.tsv +++ b/documentation/tests/campaign10-evidence-2026-07-31/state/journal.tsv @@ -1,587 +1 @@ cycle utc invariant verdict detail -1 2026-08-01T07:20:12Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -1 2026-08-01T07:20:15Z I3 PASS adatok bound_under_parent=False while absent -1 2026-08-01T07:22:27Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -1 2026-08-01T07:22:30Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -1 2026-08-01T07:23:14Z REDEPLOY PASS grafana redeployed -1 2026-08-01T07:23:56Z BACKUP PASS {"db_dump": {"count": 1, "duration": "29.235715924s", "last_run": "2026-08-01T07:23:45.622922949Z", "success": true}, "enabled": true, "running": false} -1 2026-08-01T07:24:48Z BACKUP PASS {"db_dump": {"count": 1, "duration": "34.037544623s", "last_run": "2026-08-01T07:24:42.576575317Z", "success": true}, "enabled": true, "running": false} -1 2026-08-01T07:25:51Z I7 PASS app=rallly want=C10-C001-A-092356 got=C10-C001-A-092356 snap=helyi restore_ok=True rto=42.3s -1 2026-08-01T07:28:07Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -1 2026-08-01T07:28:10Z I3 PASS mentes bound_under_parent=False while absent -1 2026-08-01T07:30:22Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -1 2026-08-01T07:30:25Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -1 2026-08-01T07:31:14Z KILL-CTRL PASS controller recovered after kill -1 2026-08-01T07:31:17Z I5 PASS degraded=False absent_events=0 target=felhom-backup -1 2026-08-01T07:31:17Z I6 PASS healthy state must carry no degraded message; got=None -1 2026-08-01T07:31:19Z I11 PASS leaked=[] -1 2026-08-01T07:35:53Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -1 2026-08-01T07:35:56Z I3 PASS adatok bound_under_parent=False while absent -1 2026-08-01T07:38:09Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -1 2026-08-01T07:38:12Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -1 2026-08-01T07:38:56Z REDEPLOY PASS homebox redeployed -1 2026-08-01T07:39:38Z BACKUP PASS {"db_dump": {"count": 1, "duration": "29.454266341s", "last_run": "2026-08-01T07:39:28.065882881Z", "success": true}, "enabled": true, "running": false} -1 2026-08-01T07:40:31Z BACKUP PASS {"db_dump": {"count": 1, "duration": "32.574200326s", "last_run": "2026-08-01T07:40:23.804874002Z", "success": true}, "enabled": true, "running": false} -1 2026-08-01T07:41:34Z I7 PASS app=rallly want=C10-C001-A-093938 got=C10-C001-A-093938 snap=helyi restore_ok=True rto=41.8s -1 2026-08-01T07:43:49Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -1 2026-08-01T07:43:52Z I3 PASS mentes bound_under_parent=False while absent -1 2026-08-01T07:46:05Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -1 2026-08-01T07:46:07Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -1 2026-08-01T07:46:56Z KILL-CTRL PASS controller recovered after kill -1 2026-08-01T07:46:59Z I5 PASS degraded=False absent_events=0 target=felhom-backup -1 2026-08-01T07:46:59Z I6 PASS healthy state must carry no degraded message; got=None -1 2026-08-01T07:47:01Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -1 2026-08-01T07:47:01Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -1 2026-08-01T07:47:03Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -1 2026-08-01T07:47:05Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -1 2026-08-01T07:47:07Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -1 2026-08-01T07:47:09Z I11 PASS leaked=[] -2 2026-08-01T07:48:01Z KILL-CTRL PASS controller recovered after kill -2 2026-08-01T07:50:16Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -2 2026-08-01T07:50:19Z I3 PASS adatok bound_under_parent=False while absent -2 2026-08-01T07:52:32Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -2 2026-08-01T07:52:35Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -2 2026-08-01T07:53:17Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.724847172s", "last_run": "2026-08-01T07:53:05.968779815Z", "success": true}, "enabled": true, "running": false} -2 2026-08-01T07:54:01Z REDEPLOY PASS grafana redeployed -2 2026-08-01T07:54:54Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.880675052s", "last_run": "2026-08-01T07:54:43.377582192Z", "success": true}, "enabled": true, "running": false} -2 2026-08-01T07:55:56Z I7 PASS app=rallly want=C10-C002-A-095401 got=C10-C002-A-095401 snap=helyi restore_ok=True rto=41.7s -2 2026-08-01T07:58:12Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -2 2026-08-01T07:58:14Z I3 PASS mentes bound_under_parent=False while absent -2 2026-08-01T08:00:27Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -2 2026-08-01T08:00:30Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -2 2026-08-01T08:00:32Z I5 PASS degraded=False absent_events=0 target=felhom-backup -2 2026-08-01T08:00:32Z I6 PASS healthy state must carry no degraded message; got=None -2 2026-08-01T08:00:34Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -2 2026-08-01T08:00:34Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -2 2026-08-01T08:00:36Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -2 2026-08-01T08:00:38Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -2 2026-08-01T08:00:40Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -2 2026-08-01T08:00:42Z I11 PASS leaked=[] -3 2026-08-01T08:03:00Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -3 2026-08-01T08:03:03Z I3 PASS mentes bound_under_parent=False while absent -3 2026-08-01T08:05:15Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -3 2026-08-01T08:05:18Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -3 2026-08-01T08:06:02Z REDEPLOY PASS homebox redeployed -3 2026-08-01T08:06:44Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.249083164s", "last_run": "2026-08-01T08:06:32.499808867Z", "success": true}, "enabled": true, "running": false} -3 2026-08-01T08:09:00Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -3 2026-08-01T08:09:02Z I3 PASS adatok bound_under_parent=False while absent -3 2026-08-01T08:11:15Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -3 2026-08-01T08:11:18Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -3 2026-08-01T08:12:07Z KILL-CTRL PASS controller recovered after kill -3 2026-08-01T08:13:00Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.907036117s", "last_run": "2026-08-01T08:12:49.360548351Z", "success": true}, "enabled": true, "running": false} -3 2026-08-01T08:14:04Z I7 PASS app=rallly want=C10-C003-A-101207 got=C10-C003-A-101207 snap=helyi restore_ok=True rto=44.3s -3 2026-08-01T08:14:07Z I5 PASS degraded=False absent_events=0 target=felhom-backup -3 2026-08-01T08:14:07Z I6 PASS healthy state must carry no degraded message; got=None -3 2026-08-01T08:14:09Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -3 2026-08-01T08:14:09Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -3 2026-08-01T08:14:11Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -3 2026-08-01T08:14:13Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -3 2026-08-01T08:14:15Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -3 2026-08-01T08:14:17Z I11 PASS leaked=[] -4 2026-08-01T08:16:35Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -4 2026-08-01T08:16:38Z I3 PASS adatok bound_under_parent=False while absent -4 2026-08-01T08:18:51Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -4 2026-08-01T08:18:54Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -4 2026-08-01T08:19:35Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.425451162s", "last_run": "2026-08-01T08:19:24.403091262Z", "success": true}, "enabled": true, "running": false} -4 2026-08-01T08:20:20Z REDEPLOY PASS grafana redeployed -4 2026-08-01T08:21:12Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.411530632s", "last_run": "2026-08-01T08:21:00.742317925Z", "success": true}, "enabled": true, "running": false} -4 2026-08-01T08:22:14Z I7 PASS app=rallly want=C10-C004-A-102020 got=C10-C004-A-102020 snap=helyi restore_ok=True rto=42.0s -4 2026-08-01T08:24:30Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -4 2026-08-01T08:24:32Z I3 PASS mentes bound_under_parent=False while absent -4 2026-08-01T08:26:45Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -4 2026-08-01T08:26:48Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -4 2026-08-01T08:27:37Z KILL-CTRL PASS controller recovered after kill -4 2026-08-01T08:27:39Z I5 PASS degraded=False absent_events=0 target=felhom-backup -4 2026-08-01T08:27:39Z I6 PASS healthy state must carry no degraded message; got=None -4 2026-08-01T08:27:41Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -4 2026-08-01T08:27:41Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -4 2026-08-01T08:27:43Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -4 2026-08-01T08:27:46Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -4 2026-08-01T08:27:47Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -4 2026-08-01T08:27:50Z I11 PASS leaked=[] -5 2026-08-01T08:28:36Z REDEPLOY PASS grafana redeployed -5 2026-08-01T08:30:51Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -5 2026-08-01T08:30:54Z I3 PASS adatok bound_under_parent=False while absent -5 2026-08-01T08:33:07Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -5 2026-08-01T08:33:10Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -5 2026-08-01T08:33:59Z KILL-CTRL PASS controller recovered after kill -5 2026-08-01T08:36:15Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -5 2026-08-01T08:36:18Z I3 PASS mentes bound_under_parent=False while absent -5 2026-08-01T08:38:30Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -5 2026-08-01T08:38:33Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -5 2026-08-01T08:39:15Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.537566027s", "last_run": "2026-08-01T08:39:03.884190323Z", "success": true}, "enabled": true, "running": false} -5 2026-08-01T08:40:08Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.819105251s", "last_run": "2026-08-01T08:39:56.870257525Z", "success": true}, "enabled": true, "running": false} -5 2026-08-01T08:41:13Z I7 PASS app=rallly want=C10-C005-A-103915 got=C10-C005-A-103915 snap=helyi restore_ok=True rto=43.0s -5 2026-08-01T08:41:16Z I5 PASS degraded=False absent_events=0 target=felhom-backup -5 2026-08-01T08:41:16Z I6 PASS healthy state must carry no degraded message; got=None -5 2026-08-01T08:41:18Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -5 2026-08-01T08:41:18Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -5 2026-08-01T08:41:20Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -5 2026-08-01T08:41:22Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -5 2026-08-01T08:41:24Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -5 2026-08-01T08:41:26Z I11 PASS leaked=[] -6 2026-08-01T08:42:18Z KILL-CTRL PASS controller recovered after kill -6 2026-08-01T08:43:10Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.890401885s", "last_run": "2026-08-01T08:42:58.392385368Z", "success": true}, "enabled": true, "running": false} -6 2026-08-01T08:44:12Z I7 PASS app=rallly want=C10-C006-A-104218 got=C10-C006-A-104218 snap=helyi restore_ok=True rto=41.6s -6 2026-08-01T08:46:28Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -6 2026-08-01T08:46:30Z I3 PASS mentes bound_under_parent=False while absent -6 2026-08-01T08:48:44Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -6 2026-08-01T08:48:46Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -6 2026-08-01T08:51:02Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -6 2026-08-01T08:51:05Z I3 PASS adatok bound_under_parent=False while absent -6 2026-08-01T08:53:18Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -6 2026-08-01T08:53:20Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -6 2026-08-01T08:54:02Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.101250195s", "last_run": "2026-08-01T08:53:50.441271314Z", "success": true}, "enabled": true, "running": false} -6 2026-08-01T08:54:46Z REDEPLOY PASS grafana redeployed -6 2026-08-01T08:54:49Z I5 PASS degraded=False absent_events=0 target=felhom-backup -6 2026-08-01T08:54:49Z I6 PASS healthy state must carry no degraded message; got=None -6 2026-08-01T08:54:51Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -6 2026-08-01T08:54:51Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -6 2026-08-01T08:54:53Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -6 2026-08-01T08:54:55Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -6 2026-08-01T08:54:57Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -6 2026-08-01T08:54:59Z I11 PASS leaked=[] -7 2026-08-01T08:57:17Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -7 2026-08-01T08:57:20Z I3 PASS adatok bound_under_parent=False while absent -7 2026-08-01T08:59:33Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -7 2026-08-01T08:59:35Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -7 2026-08-01T09:01:50Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -7 2026-08-01T09:01:54Z I3 PASS mentes bound_under_parent=False while absent -7 2026-08-01T09:04:07Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -7 2026-08-01T09:04:09Z I4 PASS guest init boot before=1785531262 after=1785531262 (must be equal) -7 2026-08-01T09:04:58Z KILL-CTRL PASS controller recovered after kill -7 2026-08-01T09:05:43Z REDEPLOY PASS homebox redeployed -7 2026-08-01T09:06:35Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.192784423s", "last_run": "2026-08-01T09:06:24.336193131Z", "success": true}, "enabled": true, "running": false} -7 2026-08-01T09:07:38Z I7 PASS app=rallly want=C10-C007-A-110543 got=C10-C007-A-110543 snap=helyi restore_ok=True rto=42.2s -7 2026-08-01T09:08:21Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.754755491s", "last_run": "2026-08-01T09:08:08.67970607Z", "success": true}, "enabled": true, "running": false} -7 2026-08-01T09:09:00Z REBOOT PASS controller back after guest reboot -7 2026-08-01T09:09:03Z I5 PASS degraded=False absent_events=0 target=felhom-backup -7 2026-08-01T09:09:03Z I6 PASS healthy state must carry no degraded message; got=None -7 2026-08-01T09:09:05Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -7 2026-08-01T09:09:05Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -7 2026-08-01T09:09:07Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -7 2026-08-01T09:09:09Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -7 2026-08-01T09:09:11Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -7 2026-08-01T09:09:13Z I11 PASS leaked=[] -8 2026-08-01T09:09:59Z REDEPLOY PASS homebox redeployed -8 2026-08-01T09:10:48Z KILL-CTRL PASS controller recovered after kill -8 2026-08-01T09:11:41Z BACKUP PASS {"db_dump": {"count": 1, "duration": "29.417683334s", "last_run": "2026-08-01T09:11:30.326680315Z", "success": true}, "enabled": true, "running": false} -8 2026-08-01T09:12:45Z I7 PASS app=rallly want=C10-C008-A-111048 got=C10-C008-A-111048 snap=helyi restore_ok=True rto=42.3s -8 2026-08-01T09:15:01Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -8 2026-08-01T09:15:04Z I3 PASS adatok bound_under_parent=False while absent -8 2026-08-01T09:17:16Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -8 2026-08-01T09:17:19Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -8 2026-08-01T09:19:35Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -8 2026-08-01T09:19:38Z I3 PASS mentes bound_under_parent=False while absent -8 2026-08-01T09:21:51Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -8 2026-08-01T09:21:53Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -8 2026-08-01T09:22:35Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.112766197s", "last_run": "2026-08-01T09:22:23.959739494Z", "success": true}, "enabled": true, "running": false} -8 2026-08-01T09:22:38Z I5 PASS degraded=False absent_events=0 target=felhom-backup -8 2026-08-01T09:22:38Z I6 PASS healthy state must carry no degraded message; got=None -8 2026-08-01T09:22:40Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -8 2026-08-01T09:22:40Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -8 2026-08-01T09:22:42Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -8 2026-08-01T09:22:44Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -8 2026-08-01T09:22:46Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -8 2026-08-01T09:22:48Z I11 PASS leaked=[] -9 2026-08-01T09:23:39Z KILL-CTRL PASS controller recovered after kill -9 2026-08-01T09:24:19Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.120915244s", "last_run": "2026-08-01T09:24:19.437409908Z", "success": true}, "enabled": true, "running": false} -9 2026-08-01T09:25:22Z I7 PASS app=rallly want=C10-C009-A-112339 got=C10-C009-A-112339 snap=helyi restore_ok=True rto=41.9s -9 2026-08-01T09:26:06Z REDEPLOY PASS grafana redeployed -9 2026-08-01T09:28:22Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -9 2026-08-01T09:28:25Z I3 PASS adatok bound_under_parent=False while absent -9 2026-08-01T09:30:38Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -9 2026-08-01T09:30:40Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -9 2026-08-01T09:31:10Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.234736705s", "last_run": "2026-08-01T09:31:09.977587344Z", "success": true}, "enabled": true, "running": false} -9 2026-08-01T09:33:26Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -9 2026-08-01T09:33:29Z I3 PASS mentes bound_under_parent=False while absent -9 2026-08-01T09:35:42Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -9 2026-08-01T09:35:44Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -9 2026-08-01T09:35:47Z I5 PASS degraded=False absent_events=0 target=felhom-backup -9 2026-08-01T09:35:47Z I6 PASS healthy state must carry no degraded message; got=None -9 2026-08-01T09:35:49Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -9 2026-08-01T09:35:49Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -9 2026-08-01T09:35:51Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -9 2026-08-01T09:35:53Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -9 2026-08-01T09:35:55Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -9 2026-08-01T09:35:57Z I11 PASS leaked=[] -10 2026-08-01T09:38:14Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -10 2026-08-01T09:38:17Z I3 PASS adatok bound_under_parent=False while absent -10 2026-08-01T09:40:30Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -10 2026-08-01T09:40:33Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -10 2026-08-01T09:41:17Z REDEPLOY PASS homebox redeployed -10 2026-08-01T09:42:06Z KILL-CTRL PASS controller recovered after kill -10 2026-08-01T09:44:22Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -10 2026-08-01T09:44:25Z I3 PASS mentes bound_under_parent=False while absent -10 2026-08-01T09:46:38Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -10 2026-08-01T09:46:40Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -10 2026-08-01T09:47:21Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.035320075s", "last_run": "2026-08-01T09:47:21.013177918Z", "success": true}, "enabled": true, "running": false} -10 2026-08-01T09:48:24Z I7 PASS app=rallly want=C10-C010-A-114640 got=C10-C010-A-114640 snap=helyi restore_ok=True rto=41.6s -10 2026-08-01T09:48:54Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.198265248s", "last_run": "2026-08-01T09:48:54.018758879Z", "success": true}, "enabled": true, "running": false} -10 2026-08-01T09:48:57Z I5 PASS degraded=False absent_events=0 target=felhom-backup -10 2026-08-01T09:48:57Z I6 PASS healthy state must carry no degraded message; got=None -10 2026-08-01T09:48:59Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -10 2026-08-01T09:48:59Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -10 2026-08-01T09:49:01Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -10 2026-08-01T09:49:03Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -10 2026-08-01T09:49:05Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -10 2026-08-01T09:49:07Z I11 PASS leaked=[] -11 2026-08-01T09:50:02Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.500198373s", "last_run": "2026-08-01T09:49:50.111560186Z", "success": true}, "enabled": true, "running": false} -11 2026-08-01T09:51:05Z I7 PASS app=rallly want=C10-C011-A-114910 got=C10-C011-A-114910 snap=helyi restore_ok=True rto=41.7s -11 2026-08-01T09:53:20Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -11 2026-08-01T09:53:23Z I3 PASS adatok bound_under_parent=False while absent -11 2026-08-01T09:55:35Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -11 2026-08-01T09:55:38Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -11 2026-08-01T09:57:54Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -11 2026-08-01T09:57:57Z I3 PASS mentes bound_under_parent=False while absent -11 2026-08-01T10:00:10Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -11 2026-08-01T10:00:13Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -11 2026-08-01T10:00:55Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.274827764s", "last_run": "2026-08-01T10:00:42.619398629Z", "success": true}, "enabled": true, "running": false} -11 2026-08-01T10:01:38Z REDEPLOY PASS homebox redeployed -11 2026-08-01T10:02:28Z KILL-CTRL PASS controller recovered after kill -11 2026-08-01T10:02:30Z I5 PASS degraded=False absent_events=0 target=felhom-backup -11 2026-08-01T10:02:30Z I6 PASS healthy state must carry no degraded message; got=None -11 2026-08-01T10:02:32Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -11 2026-08-01T10:02:32Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -11 2026-08-01T10:02:34Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -11 2026-08-01T10:02:36Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -11 2026-08-01T10:02:38Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -11 2026-08-01T10:02:40Z I11 PASS leaked=[] -12 2026-08-01T10:03:26Z REDEPLOY PASS homebox redeployed -12 2026-08-01T10:05:42Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -12 2026-08-01T10:05:45Z I3 PASS adatok bound_under_parent=False while absent -12 2026-08-01T10:07:57Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -12 2026-08-01T10:08:00Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -12 2026-08-01T10:08:49Z KILL-CTRL PASS controller recovered after kill -12 2026-08-01T10:11:04Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -12 2026-08-01T10:11:07Z I3 PASS mentes bound_under_parent=False while absent -12 2026-08-01T10:13:20Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -12 2026-08-01T10:13:24Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -12 2026-08-01T10:13:54Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.82906082s", "last_run": "2026-08-01T10:13:53.343511281Z", "success": true}, "enabled": true, "running": false} -12 2026-08-01T10:14:47Z BACKUP PASS {"db_dump": {"count": 1, "duration": "34.794492507s", "last_run": "2026-08-01T10:14:42.108803281Z", "success": true}, "enabled": true, "running": false} -12 2026-08-01T10:15:50Z I7 PASS app=rallly want=C10-C012-A-121354 got=C10-C012-A-121354 snap=helyi restore_ok=True rto=41.6s -12 2026-08-01T10:15:53Z I5 PASS degraded=False absent_events=0 target=felhom-backup -12 2026-08-01T10:15:53Z I6 PASS healthy state must carry no degraded message; got=None -12 2026-08-01T10:15:54Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -12 2026-08-01T10:15:54Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -12 2026-08-01T10:15:56Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -12 2026-08-01T10:15:58Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -12 2026-08-01T10:16:00Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -12 2026-08-01T10:16:02Z I11 PASS leaked=[] -13 2026-08-01T10:16:54Z KILL-CTRL PASS controller recovered after kill -13 2026-08-01T10:17:38Z REDEPLOY PASS grafana redeployed -13 2026-08-01T10:18:18Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.066037384s", "last_run": "2026-08-01T10:18:18.41231692Z", "success": true}, "enabled": true, "running": false} -13 2026-08-01T10:19:22Z I7 PASS app=rallly want=C10-C013-A-121738 got=C10-C013-A-121738 snap=helyi restore_ok=True rto=42.2s -13 2026-08-01T10:20:03Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.463205132s", "last_run": "2026-08-01T10:19:51.680188929Z", "success": true}, "enabled": true, "running": false} -13 2026-08-01T10:22:19Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -13 2026-08-01T10:22:22Z I3 PASS mentes bound_under_parent=False while absent -13 2026-08-01T10:24:35Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -13 2026-08-01T10:24:37Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -13 2026-08-01T10:26:53Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -13 2026-08-01T10:26:56Z I3 PASS adatok bound_under_parent=False while absent -13 2026-08-01T10:29:09Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -13 2026-08-01T10:29:12Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -13 2026-08-01T10:29:14Z I5 PASS degraded=False absent_events=0 target=felhom-backup -13 2026-08-01T10:29:14Z I6 PASS healthy state must carry no degraded message; got=None -13 2026-08-01T10:29:16Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -13 2026-08-01T10:29:16Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -13 2026-08-01T10:29:18Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -13 2026-08-01T10:29:20Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -13 2026-08-01T10:29:22Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -13 2026-08-01T10:29:25Z I11 PASS leaked=[] -14 2026-08-01T10:29:56Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.80615982s", "last_run": "2026-08-01T10:29:55.907589946Z", "success": true}, "enabled": true, "running": false} -14 2026-08-01T10:30:50Z BACKUP PASS {"db_dump": {"count": 1, "duration": "33.858181263s", "last_run": "2026-08-01T10:30:43.782817699Z", "success": true}, "enabled": true, "running": false} -14 2026-08-01T10:31:54Z I7 PASS app=rallly want=C10-C014-A-122956 got=C10-C014-A-122956 snap=helyi restore_ok=True rto=42.4s -14 2026-08-01T10:34:09Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -14 2026-08-01T10:34:12Z I3 PASS adatok bound_under_parent=False while absent -14 2026-08-01T10:36:25Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -14 2026-08-01T10:36:28Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -14 2026-08-01T10:37:12Z REDEPLOY PASS grafana redeployed -14 2026-08-01T10:38:01Z KILL-CTRL PASS controller recovered after kill -14 2026-08-01T10:40:17Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -14 2026-08-01T10:40:20Z I3 PASS mentes bound_under_parent=False while absent -14 2026-08-01T10:42:33Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -14 2026-08-01T10:42:36Z I4 PASS guest init boot before=1785575320 after=1785575320 (must be equal) -14 2026-08-01T10:43:14Z REBOOT PASS controller back after guest reboot -14 2026-08-01T10:43:17Z I5 PASS degraded=False absent_events=0 target=felhom-backup -14 2026-08-01T10:43:17Z I6 PASS healthy state must carry no degraded message; got=None -14 2026-08-01T10:43:19Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -14 2026-08-01T10:43:19Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -14 2026-08-01T10:43:21Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -14 2026-08-01T10:43:24Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -14 2026-08-01T10:43:26Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -14 2026-08-01T10:43:28Z I11 PASS leaked=[] -15 2026-08-01T10:45:45Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -15 2026-08-01T10:45:48Z I3 PASS mentes bound_under_parent=False while absent -15 2026-08-01T10:48:01Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -15 2026-08-01T10:48:03Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -15 2026-08-01T10:48:46Z BACKUP PASS {"db_dump": {"count": 1, "duration": "28.270428557s", "last_run": "2026-08-01T10:48:34.163245256Z", "success": true}, "enabled": true, "running": false} -15 2026-08-01T10:49:36Z KILL-CTRL PASS controller recovered after kill -15 2026-08-01T10:50:20Z REDEPLOY PASS homebox redeployed -15 2026-08-01T10:51:01Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.421623567s", "last_run": "2026-08-01T10:51:00.95301014Z", "success": true}, "enabled": true, "running": false} -15 2026-08-01T10:52:04Z I7 PASS app=rallly want=C10-C015-A-125020 got=C10-C015-A-125020 snap=helyi restore_ok=True rto=41.8s -15 2026-08-01T10:54:20Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -15 2026-08-01T10:54:23Z I3 PASS adatok bound_under_parent=False while absent -15 2026-08-01T10:56:36Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -15 2026-08-01T10:56:39Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -15 2026-08-01T10:56:42Z I5 PASS degraded=False absent_events=0 target=felhom-backup -15 2026-08-01T10:56:42Z I6 PASS healthy state must carry no degraded message; got=None -15 2026-08-01T10:56:45Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -15 2026-08-01T10:56:45Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -15 2026-08-01T10:56:47Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -15 2026-08-01T10:56:49Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -15 2026-08-01T10:56:51Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -15 2026-08-01T10:56:54Z I11 PASS leaked=[] -16 2026-08-01T10:57:27Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.683550606s", "last_run": "2026-08-01T10:57:26.606568663Z", "success": true}, "enabled": true, "running": false} -16 2026-08-01T10:59:43Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -16 2026-08-01T10:59:48Z I3 PASS mentes bound_under_parent=False while absent -16 2026-08-01T11:02:02Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -16 2026-08-01T11:02:08Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -16 2026-08-01T11:04:31Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -16 2026-08-01T11:04:40Z I3 PASS adatok bound_under_parent=False while absent -16 2026-08-01T11:06:58Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -16 2026-08-01T11:07:01Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -16 2026-08-01T11:08:29Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.894128293s", "last_run": "2026-08-01T11:08:14.025870917Z", "success": true}, "enabled": true, "running": false} -16 2026-08-01T11:09:59Z I7 PASS app=rallly want=C10-C016-A-130701 got=C10-C016-A-130701 snap=helyi restore_ok=True rto=38.8s -16 2026-08-01T11:10:54Z KILL-CTRL PASS controller recovered after kill -16 2026-08-01T11:11:53Z REDEPLOY PASS grafana redeployed -16 2026-08-01T11:12:03Z I5 PASS degraded=False absent_events=0 target=felhom-backup -16 2026-08-01T11:12:03Z I6 PASS healthy state must carry no degraded message; got=None -16 2026-08-01T11:12:11Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -16 2026-08-01T11:12:11Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -16 2026-08-01T11:12:19Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -16 2026-08-01T11:12:27Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -16 2026-08-01T11:12:36Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -16 2026-08-01T11:12:43Z I11 PASS leaked=[] -17 2026-08-01T11:13:43Z REDEPLOY PASS grafana redeployed -17 2026-08-01T11:16:04Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -17 2026-08-01T11:16:08Z I3 PASS adatok bound_under_parent=False while absent -17 2026-08-01T11:18:21Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -17 2026-08-01T11:18:23Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -17 2026-08-01T11:18:53Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.838575753s", "last_run": "2026-08-01T11:18:52.693573505Z", "success": true}, "enabled": true, "running": false} -17 2026-08-01T11:19:42Z KILL-CTRL PASS controller recovered after kill -17 2026-08-01T11:20:23Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.317455253s", "last_run": "2026-08-01T11:20:23.002386465Z", "success": true}, "enabled": true, "running": false} -17 2026-08-01T11:21:26Z I7 PASS app=rallly want=C10-C017-A-131942 got=C10-C017-A-131942 snap=helyi restore_ok=True rto=41.5s -17 2026-08-01T11:23:42Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -17 2026-08-01T11:23:45Z I3 PASS mentes bound_under_parent=False while absent -17 2026-08-01T11:25:57Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -17 2026-08-01T11:26:00Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -17 2026-08-01T11:26:02Z I5 PASS degraded=False absent_events=0 target=felhom-backup -17 2026-08-01T11:26:02Z I6 PASS healthy state must carry no degraded message; got=None -17 2026-08-01T11:26:04Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -17 2026-08-01T11:26:04Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -17 2026-08-01T11:26:06Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -17 2026-08-01T11:26:08Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -17 2026-08-01T11:26:10Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -17 2026-08-01T11:26:12Z I11 PASS leaked=[] -18 2026-08-01T11:28:30Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -18 2026-08-01T11:28:33Z I3 PASS mentes bound_under_parent=False while absent -18 2026-08-01T11:30:45Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -18 2026-08-01T11:30:48Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -18 2026-08-01T11:31:38Z KILL-CTRL PASS controller recovered after kill -18 2026-08-01T11:33:53Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -18 2026-08-01T11:33:57Z I3 PASS adatok bound_under_parent=False while absent -18 2026-08-01T11:36:10Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -18 2026-08-01T11:36:12Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -18 2026-08-01T11:36:42Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.287771352s", "last_run": "2026-08-01T11:36:41.854058541Z", "success": true}, "enabled": true, "running": false} -18 2026-08-01T11:37:27Z REDEPLOY PASS grafana redeployed -18 2026-08-01T11:38:08Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.269077889s", "last_run": "2026-08-01T11:38:07.785076049Z", "success": true}, "enabled": true, "running": false} -18 2026-08-01T11:39:12Z I7 PASS app=rallly want=C10-C018-A-133727 got=C10-C018-A-133727 snap=helyi restore_ok=True rto=42.0s -18 2026-08-01T11:39:15Z I5 PASS degraded=False absent_events=0 target=felhom-backup -18 2026-08-01T11:39:15Z I6 PASS healthy state must carry no degraded message; got=None -18 2026-08-01T11:39:17Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -18 2026-08-01T11:39:17Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -18 2026-08-01T11:39:19Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -18 2026-08-01T11:39:21Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -18 2026-08-01T11:39:24Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -18 2026-08-01T11:39:26Z I11 PASS leaked=[] -19 2026-08-01T11:40:18Z KILL-CTRL PASS controller recovered after kill -19 2026-08-01T11:41:02Z REDEPLOY PASS homebox redeployed -19 2026-08-01T11:41:44Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.240196615s", "last_run": "2026-08-01T11:41:43.294094532Z", "success": true}, "enabled": true, "running": false} -19 2026-08-01T11:42:48Z I7 PASS app=rallly want=C10-C019-A-134102 got=C10-C019-A-134102 snap=helyi restore_ok=True rto=42.1s -19 2026-08-01T11:45:03Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -19 2026-08-01T11:45:06Z I3 PASS adatok bound_under_parent=False while absent -19 2026-08-01T11:47:20Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -19 2026-08-01T11:47:22Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -19 2026-08-01T11:49:38Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -19 2026-08-01T11:49:41Z I3 PASS mentes bound_under_parent=False while absent -19 2026-08-01T11:51:54Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -19 2026-08-01T11:51:56Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -19 2026-08-01T11:52:25Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.074516344s", "last_run": "2026-08-01T11:52:25.638816606Z", "success": true}, "enabled": true, "running": false} -19 2026-08-01T11:52:28Z I5 PASS degraded=False absent_events=0 target=felhom-backup -19 2026-08-01T11:52:28Z I6 PASS healthy state must carry no degraded message; got=None -19 2026-08-01T11:52:30Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -19 2026-08-01T11:52:30Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -19 2026-08-01T11:52:32Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -19 2026-08-01T11:52:34Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -19 2026-08-01T11:52:36Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -19 2026-08-01T11:52:39Z I11 PASS leaked=[] -20 2026-08-01T11:53:24Z BACKUP PASS {"db_dump": {"count": 1, "duration": "33.58133879s", "last_run": "2026-08-01T11:53:17.277403988Z", "success": true}, "enabled": true, "running": false} -20 2026-08-01T11:55:40Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -20 2026-08-01T11:55:43Z I3 PASS mentes bound_under_parent=False while absent -20 2026-08-01T11:57:56Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -20 2026-08-01T11:57:58Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -20 2026-08-01T12:00:14Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -20 2026-08-01T12:00:17Z I3 PASS adatok bound_under_parent=False while absent -20 2026-08-01T12:02:30Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -20 2026-08-01T12:02:33Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -20 2026-08-01T12:03:17Z REDEPLOY PASS grafana redeployed -20 2026-08-01T12:03:58Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.633748751s", "last_run": "2026-08-01T12:03:57.032814639Z", "success": true}, "enabled": true, "running": false} -20 2026-08-01T12:05:02Z I7 PASS app=rallly want=C10-C020-A-140317 got=C10-C020-A-140317 snap=helyi restore_ok=True rto=42.2s -20 2026-08-01T12:05:52Z KILL-CTRL PASS controller recovered after kill -20 2026-08-01T12:05:55Z I5 PASS degraded=False absent_events=0 target=felhom-backup -20 2026-08-01T12:05:55Z I6 PASS healthy state must carry no degraded message; got=None -20 2026-08-01T12:05:57Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -20 2026-08-01T12:05:57Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -20 2026-08-01T12:05:59Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -20 2026-08-01T12:06:01Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -20 2026-08-01T12:06:03Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -20 2026-08-01T12:06:05Z I11 PASS leaked=[] -21 2026-08-01T12:06:37Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.742931297s", "last_run": "2026-08-01T12:06:36.352784544Z", "success": true}, "enabled": true, "running": false} -21 2026-08-01T12:07:26Z KILL-CTRL PASS controller recovered after kill -21 2026-08-01T12:09:42Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -21 2026-08-01T12:09:45Z I3 PASS adatok bound_under_parent=False while absent -21 2026-08-01T12:11:58Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -21 2026-08-01T12:12:01Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -21 2026-08-01T12:12:45Z REDEPLOY PASS homebox redeployed -21 2026-08-01T12:15:01Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -21 2026-08-01T12:15:04Z I3 PASS mentes bound_under_parent=False while absent -21 2026-08-01T12:17:17Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -21 2026-08-01T12:17:19Z I4 PASS guest init boot before=1785580975 after=1785580975 (must be equal) -21 2026-08-01T12:18:01Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.215452442s", "last_run": "2026-08-01T12:18:01.043730281Z", "success": true}, "enabled": true, "running": false} -21 2026-08-01T12:19:03Z I7 PASS app=rallly want=C10-C021-A-141719 got=C10-C021-A-141719 snap=helyi restore_ok=True rto=41.4s -21 2026-08-01T12:19:43Z REBOOT PASS controller back after guest reboot -21 2026-08-01T12:19:46Z I5 PASS degraded=False absent_events=0 target=felhom-backup -21 2026-08-01T12:19:46Z I6 PASS healthy state must carry no degraded message; got=None -21 2026-08-01T12:19:48Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -21 2026-08-01T12:19:48Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -21 2026-08-01T12:19:50Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -21 2026-08-01T12:19:52Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -21 2026-08-01T12:19:54Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -21 2026-08-01T12:19:56Z I11 PASS leaked=[] -22 2026-08-01T12:20:51Z BACKUP PASS {"db_dump": {"count": 1, "duration": "35.295073244s", "last_run": "2026-08-01T12:20:46.946104153Z", "success": true}, "enabled": true, "running": false} -22 2026-08-01T12:21:54Z I7 PASS app=rallly want=C10-C022-A-141958 got=C10-C022-A-141958 snap=helyi restore_ok=True rto=41.9s -22 2026-08-01T12:22:38Z REDEPLOY PASS homebox redeployed -22 2026-08-01T12:23:20Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.506763469s", "last_run": "2026-08-01T12:23:08.027088655Z", "success": true}, "enabled": true, "running": false} -22 2026-08-01T12:25:35Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -22 2026-08-01T12:25:38Z I3 PASS adatok bound_under_parent=False while absent -22 2026-08-01T12:27:51Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -22 2026-08-01T12:27:54Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -22 2026-08-01T12:28:43Z KILL-CTRL PASS controller recovered after kill -22 2026-08-01T12:30:58Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -22 2026-08-01T12:31:01Z I3 PASS mentes bound_under_parent=False while absent -22 2026-08-01T12:33:14Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -22 2026-08-01T12:33:17Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -22 2026-08-01T12:33:20Z I5 PASS degraded=False absent_events=0 target=felhom-backup -22 2026-08-01T12:33:20Z I6 PASS healthy state must carry no degraded message; got=None -22 2026-08-01T12:33:22Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -22 2026-08-01T12:33:22Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -22 2026-08-01T12:33:24Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -22 2026-08-01T12:33:26Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -22 2026-08-01T12:33:28Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -22 2026-08-01T12:33:30Z I11 PASS leaked=[] -23 2026-08-01T12:34:02Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.296490438s", "last_run": "2026-08-01T12:34:02.179324267Z", "success": true}, "enabled": true, "running": false} -23 2026-08-01T12:36:18Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -23 2026-08-01T12:36:21Z I3 PASS mentes bound_under_parent=False while absent -23 2026-08-01T12:38:34Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -23 2026-08-01T12:38:37Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -23 2026-08-01T12:39:17Z BACKUP PASS {"db_dump": {"count": 1, "duration": "26.836521386s", "last_run": "2026-08-01T12:39:17.164422568Z", "success": true}, "enabled": true, "running": false} -23 2026-08-01T12:40:21Z I7 PASS app=rallly want=C10-C023-A-143837 got=C10-C023-A-143837 snap=helyi restore_ok=True rto=41.9s -23 2026-08-01T12:41:11Z KILL-CTRL PASS controller recovered after kill -23 2026-08-01T12:41:56Z REDEPLOY PASS grafana redeployed -23 2026-08-01T12:44:11Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -23 2026-08-01T12:44:14Z I3 PASS adatok bound_under_parent=False while absent -23 2026-08-01T12:46:27Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -23 2026-08-01T12:46:30Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -23 2026-08-01T12:46:32Z I5 PASS degraded=False absent_events=0 target=felhom-backup -23 2026-08-01T12:46:32Z I6 PASS healthy state must carry no degraded message; got=None -23 2026-08-01T12:46:34Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -23 2026-08-01T12:46:34Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -23 2026-08-01T12:46:36Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -23 2026-08-01T12:46:38Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -23 2026-08-01T12:46:40Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -23 2026-08-01T12:46:43Z I11 PASS leaked=[] -24 2026-08-01T12:47:26Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.434160964s", "last_run": "2026-08-01T12:47:25.920976261Z", "success": true}, "enabled": true, "running": false} -24 2026-08-01T12:48:31Z I7 PASS app=rallly want=C10-C024-A-144645 got=C10-C024-A-144645 snap=helyi restore_ok=True rto=42.5s -24 2026-08-01T12:49:02Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.477758487s", "last_run": "2026-08-01T12:49:00.748171817Z", "success": true}, "enabled": true, "running": false} -24 2026-08-01T12:51:17Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -24 2026-08-01T12:51:20Z I3 PASS mentes bound_under_parent=False while absent -24 2026-08-01T12:53:33Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -24 2026-08-01T12:53:36Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -24 2026-08-01T12:54:25Z KILL-CTRL PASS controller recovered after kill -24 2026-08-01T12:56:40Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -24 2026-08-01T12:56:44Z I3 PASS adatok bound_under_parent=False while absent -24 2026-08-01T12:58:57Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -24 2026-08-01T12:59:00Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -24 2026-08-01T12:59:45Z REDEPLOY PASS grafana redeployed -24 2026-08-01T12:59:47Z I5 PASS degraded=False absent_events=0 target=felhom-backup -24 2026-08-01T12:59:47Z I6 PASS healthy state must carry no degraded message; got=None -24 2026-08-01T12:59:50Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -24 2026-08-01T12:59:50Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -24 2026-08-01T12:59:52Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -24 2026-08-01T12:59:54Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -24 2026-08-01T12:59:56Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -24 2026-08-01T12:59:59Z I11 PASS leaked=[] -25 2026-08-01T13:02:17Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -25 2026-08-01T13:02:20Z I3 PASS adatok bound_under_parent=False while absent -25 2026-08-01T13:04:33Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -25 2026-08-01T13:04:36Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -25 2026-08-01T13:05:26Z KILL-CTRL PASS controller recovered after kill -25 2026-08-01T13:07:41Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -25 2026-08-01T13:07:45Z I3 PASS mentes bound_under_parent=False while absent -25 2026-08-01T13:09:58Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -25 2026-08-01T13:10:00Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -25 2026-08-01T13:10:40Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.129870607s", "last_run": "2026-08-01T13:10:40.81939141Z", "success": true}, "enabled": true, "running": false} -25 2026-08-01T13:11:44Z I7 PASS app=rallly want=C10-C025-A-151000 got=C10-C025-A-151000 snap=helyi restore_ok=True rto=42.0s -25 2026-08-01T13:12:26Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.439198049s", "last_run": "2026-08-01T13:12:14.38822499Z", "success": true}, "enabled": true, "running": false} -25 2026-08-01T13:13:10Z REDEPLOY PASS homebox redeployed -25 2026-08-01T13:13:13Z I5 PASS degraded=False absent_events=0 target=felhom-backup -25 2026-08-01T13:13:13Z I6 PASS healthy state must carry no degraded message; got=None -25 2026-08-01T13:13:15Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -25 2026-08-01T13:13:15Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -25 2026-08-01T13:13:17Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -25 2026-08-01T13:13:19Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -25 2026-08-01T13:13:21Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -25 2026-08-01T13:13:24Z I11 PASS leaked=[] -26 2026-08-01T13:14:10Z REDEPLOY PASS grafana redeployed -26 2026-08-01T13:16:26Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -26 2026-08-01T13:16:29Z I3 PASS adatok bound_under_parent=False while absent -26 2026-08-01T13:18:42Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -26 2026-08-01T13:18:44Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -26 2026-08-01T13:19:25Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.181611353s", "last_run": "2026-08-01T13:19:24.6968582Z", "success": true}, "enabled": true, "running": false} -26 2026-08-01T13:20:31Z I7 PASS app=rallly want=C10-C026-A-151844 got=C10-C026-A-151844 snap=helyi restore_ok=True rto=42.9s -26 2026-08-01T13:21:20Z KILL-CTRL PASS controller recovered after kill -26 2026-08-01T13:23:36Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -26 2026-08-01T13:23:39Z I3 PASS mentes bound_under_parent=False while absent -26 2026-08-01T13:25:52Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -26 2026-08-01T13:25:55Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -26 2026-08-01T13:26:24Z BACKUP PASS {"db_dump": {"count": 1, "duration": "27.266368272s", "last_run": "2026-08-01T13:26:24.55559188Z", "success": true}, "enabled": true, "running": false} -26 2026-08-01T13:26:27Z I5 PASS degraded=False absent_events=0 target=felhom-backup -26 2026-08-01T13:26:27Z I6 PASS healthy state must carry no degraded message; got=None -26 2026-08-01T13:26:29Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -26 2026-08-01T13:26:29Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -26 2026-08-01T13:26:31Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -26 2026-08-01T13:26:33Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -26 2026-08-01T13:26:35Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -26 2026-08-01T13:26:37Z I11 PASS leaked=[] -27 2026-08-01T13:27:22Z BACKUP PASS {"db_dump": {"count": 1, "duration": "34.006310162s", "last_run": "2026-08-01T13:27:15.871339683Z", "success": true}, "enabled": true, "running": false} -27 2026-08-01T13:28:14Z BACKUP PASS {"db_dump": {"count": 1, "duration": "33.534168063s", "last_run": "2026-08-01T13:28:08.289273214Z", "success": true}, "enabled": true, "running": false} -27 2026-08-01T13:29:17Z I7 PASS app=rallly want=C10-C027-A-152722 got=C10-C027-A-152722 snap=helyi restore_ok=True rto=41.8s -27 2026-08-01T13:31:33Z I1 PASS mentes absent -> got=backup_target_absent sev=error (want backup_target_absent/error) -27 2026-08-01T13:31:36Z I3 PASS mentes bound_under_parent=False while absent -27 2026-08-01T13:33:49Z I1-pair PASS mentes return -> got=backup_target_restored sev=info (want backup_target_restored/info) -27 2026-08-01T13:33:52Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -27 2026-08-01T13:34:41Z KILL-CTRL PASS controller recovered after kill -27 2026-08-01T13:36:57Z I2 PASS adatok absent -> got=storage_disconnected sev=error (want storage_disconnected/error) -27 2026-08-01T13:37:00Z I3 PASS adatok bound_under_parent=False while absent -27 2026-08-01T13:39:14Z I2-pair PASS adatok return -> got=storage_reconnected sev=info (want storage_reconnected/info) -27 2026-08-01T13:39:16Z I4 PASS guest init boot before=1785586762 after=1785586762 (must be equal) -27 2026-08-01T13:40:00Z REDEPLOY PASS homebox redeployed -27 2026-08-01T13:40:03Z I5 PASS degraded=False absent_events=0 target=felhom-backup -27 2026-08-01T13:40:03Z I6 PASS healthy state must carry no degraded message; got=None -27 2026-08-01T13:40:04Z I10 PASS rallly/SECRET_PASSWORD type=secret must travel; in_unit=True -27 2026-08-01T13:40:04Z I10 PASS rallly/DB_PASSWORD type=secret must travel; in_unit=True -27 2026-08-01T13:40:07Z I10 PASS homebox/HBOX_AUTH_API_KEY_PEPPER type=secret must travel; in_unit=True -27 2026-08-01T13:40:09Z I10 PASS grafana/GF_SECURITY_ADMIN_PASSWORD type=password must NEVER travel; in_unit=False -27 2026-08-01T13:40:11Z I10 PASS papra/AUTH_SECRET type=secret must travel; in_unit=True -27 2026-08-01T13:40:13Z I11 PASS leaked=[] diff --git a/documentation/tests/campaign10-evidence-2026-07-31/state/rto.tsv b/documentation/tests/campaign10-evidence-2026-07-31/state/rto-run1.tsv similarity index 100% rename from documentation/tests/campaign10-evidence-2026-07-31/state/rto.tsv rename to documentation/tests/campaign10-evidence-2026-07-31/state/rto-run1.tsv diff --git a/documentation/tests/campaign10-evidence-2026-07-31/state/runner-run1.log b/documentation/tests/campaign10-evidence-2026-07-31/state/runner-run1.log new file mode 100644 index 0000000..958594b --- /dev/null +++ b/documentation/tests/campaign10-evidence-2026-07-31/state/runner-run1.log @@ -0,0 +1,60 @@ +09:17:54 === Campaign 10 Phase B start — max 1 cycles, deadline 10:17 === +09:17:56 cycle 1: detach_nontarget -> redeploy_app -> backup -> restore_verify -> detach_target -> kill_controller +09:31:19 cycle 1 done — violations so far: 0 +09:31:19 === finished after 1 cycles, 0 violations === +09:33:35 === Campaign 10 Phase B start — max 40 cycles, deadline 15:33 === +09:33:37 cycle 1: detach_nontarget -> redeploy_app -> backup -> restore_verify -> detach_target -> kill_controller +09:47:09 cycle 1 done — violations so far: 0 +09:47:11 cycle 2: kill_controller -> detach_nontarget -> backup -> redeploy_app -> restore_verify -> detach_target +10:00:42 cycle 2 done — violations so far: 0 +10:00:44 cycle 3: detach_target -> redeploy_app -> backup -> detach_nontarget -> kill_controller -> restore_verify +10:14:17 cycle 3 done — violations so far: 0 +10:14:19 cycle 4: detach_nontarget -> backup -> redeploy_app -> restore_verify -> detach_target -> kill_controller +10:27:50 cycle 4 done — violations so far: 0 +10:27:52 cycle 5: redeploy_app -> detach_nontarget -> kill_controller -> detach_target -> backup -> restore_verify +10:41:26 cycle 5 done — violations so far: 0 +10:41:28 cycle 6: kill_controller -> restore_verify -> detach_target -> detach_nontarget -> backup -> redeploy_app +10:54:59 cycle 6 done — violations so far: 0 +10:55:01 cycle 7: detach_nontarget -> detach_target -> kill_controller -> redeploy_app -> restore_verify -> backup -> reboot_guest +11:09:13 cycle 7 done — violations so far: 0 +11:09:15 cycle 8: redeploy_app -> kill_controller -> restore_verify -> detach_nontarget -> detach_target -> backup +11:22:48 cycle 8 done — violations so far: 0 +11:22:50 cycle 9: kill_controller -> restore_verify -> redeploy_app -> detach_nontarget -> backup -> detach_target +11:35:57 cycle 9 done — violations so far: 0 +11:35:59 cycle 10: detach_nontarget -> redeploy_app -> kill_controller -> detach_target -> restore_verify -> backup +11:49:07 cycle 10 done — violations so far: 0 +11:49:10 cycle 11: restore_verify -> detach_nontarget -> detach_target -> backup -> redeploy_app -> kill_controller +12:02:40 cycle 11 done — violations so far: 0 +12:02:43 cycle 12: redeploy_app -> detach_nontarget -> kill_controller -> detach_target -> backup -> restore_verify +12:16:02 cycle 12 done — violations so far: 0 +12:16:05 cycle 13: kill_controller -> redeploy_app -> restore_verify -> backup -> detach_target -> detach_nontarget +12:29:25 cycle 13 done — violations so far: 0 +12:29:27 cycle 14: backup -> restore_verify -> detach_nontarget -> redeploy_app -> kill_controller -> detach_target -> reboot_guest +12:43:28 cycle 14 done — violations so far: 0 +12:43:30 cycle 15: detach_target -> backup -> kill_controller -> redeploy_app -> restore_verify -> detach_nontarget +12:56:54 cycle 15 done — violations so far: 0 +12:56:56 cycle 16: backup -> detach_target -> detach_nontarget -> restore_verify -> kill_controller -> redeploy_app +13:12:43 cycle 16 done — violations so far: 0 +13:12:49 cycle 17: redeploy_app -> detach_nontarget -> backup -> kill_controller -> restore_verify -> detach_target +13:26:12 cycle 17 done — violations so far: 0 +13:26:14 cycle 18: detach_target -> kill_controller -> detach_nontarget -> backup -> redeploy_app -> restore_verify +13:39:26 cycle 18 done — violations so far: 0 +13:39:28 cycle 19: kill_controller -> redeploy_app -> restore_verify -> detach_nontarget -> detach_target -> backup +13:52:39 cycle 19 done — violations so far: 0 +13:52:41 cycle 20: backup -> detach_target -> detach_nontarget -> redeploy_app -> restore_verify -> kill_controller +14:06:05 cycle 20 done — violations so far: 0 +14:06:07 cycle 21: backup -> kill_controller -> detach_nontarget -> redeploy_app -> detach_target -> restore_verify -> reboot_guest +14:19:56 cycle 21 done — violations so far: 0 +14:19:58 cycle 22: restore_verify -> redeploy_app -> backup -> detach_nontarget -> kill_controller -> detach_target +14:33:30 cycle 22 done — violations so far: 0 +14:33:32 cycle 23: backup -> detach_target -> restore_verify -> kill_controller -> redeploy_app -> detach_nontarget +14:46:43 cycle 23 done — violations so far: 0 +14:46:45 cycle 24: restore_verify -> backup -> detach_target -> kill_controller -> detach_nontarget -> redeploy_app +14:59:59 cycle 24 done — violations so far: 0 +15:00:01 cycle 25: detach_nontarget -> kill_controller -> detach_target -> restore_verify -> backup -> redeploy_app +15:13:24 cycle 25 done — violations so far: 0 +15:13:26 cycle 26: redeploy_app -> detach_nontarget -> restore_verify -> kill_controller -> detach_target -> backup +15:26:37 cycle 26 done — violations so far: 0 +15:26:39 cycle 27: backup -> restore_verify -> detach_target -> kill_controller -> detach_nontarget -> redeploy_app +15:40:13 cycle 27 done — violations so far: 0 +15:40:13 === finished after 27 cycles, 0 violations === diff --git a/documentation/tests/campaign10-evidence-2026-07-31/state/runner.log b/documentation/tests/campaign10-evidence-2026-07-31/state/runner.log index 958594b..9395814 100644 --- a/documentation/tests/campaign10-evidence-2026-07-31/state/runner.log +++ b/documentation/tests/campaign10-evidence-2026-07-31/state/runner.log @@ -1,60 +1,2 @@ -09:17:54 === Campaign 10 Phase B start — max 1 cycles, deadline 10:17 === -09:17:56 cycle 1: detach_nontarget -> redeploy_app -> backup -> restore_verify -> detach_target -> kill_controller -09:31:19 cycle 1 done — violations so far: 0 -09:31:19 === finished after 1 cycles, 0 violations === -09:33:35 === Campaign 10 Phase B start — max 40 cycles, deadline 15:33 === -09:33:37 cycle 1: detach_nontarget -> redeploy_app -> backup -> restore_verify -> detach_target -> kill_controller -09:47:09 cycle 1 done — violations so far: 0 -09:47:11 cycle 2: kill_controller -> detach_nontarget -> backup -> redeploy_app -> restore_verify -> detach_target -10:00:42 cycle 2 done — violations so far: 0 -10:00:44 cycle 3: detach_target -> redeploy_app -> backup -> detach_nontarget -> kill_controller -> restore_verify -10:14:17 cycle 3 done — violations so far: 0 -10:14:19 cycle 4: detach_nontarget -> backup -> redeploy_app -> restore_verify -> detach_target -> kill_controller -10:27:50 cycle 4 done — violations so far: 0 -10:27:52 cycle 5: redeploy_app -> detach_nontarget -> kill_controller -> detach_target -> backup -> restore_verify -10:41:26 cycle 5 done — violations so far: 0 -10:41:28 cycle 6: kill_controller -> restore_verify -> detach_target -> detach_nontarget -> backup -> redeploy_app -10:54:59 cycle 6 done — violations so far: 0 -10:55:01 cycle 7: detach_nontarget -> detach_target -> kill_controller -> redeploy_app -> restore_verify -> backup -> reboot_guest -11:09:13 cycle 7 done — violations so far: 0 -11:09:15 cycle 8: redeploy_app -> kill_controller -> restore_verify -> detach_nontarget -> detach_target -> backup -11:22:48 cycle 8 done — violations so far: 0 -11:22:50 cycle 9: kill_controller -> restore_verify -> redeploy_app -> detach_nontarget -> backup -> detach_target -11:35:57 cycle 9 done — violations so far: 0 -11:35:59 cycle 10: detach_nontarget -> redeploy_app -> kill_controller -> detach_target -> restore_verify -> backup -11:49:07 cycle 10 done — violations so far: 0 -11:49:10 cycle 11: restore_verify -> detach_nontarget -> detach_target -> backup -> redeploy_app -> kill_controller -12:02:40 cycle 11 done — violations so far: 0 -12:02:43 cycle 12: redeploy_app -> detach_nontarget -> kill_controller -> detach_target -> backup -> restore_verify -12:16:02 cycle 12 done — violations so far: 0 -12:16:05 cycle 13: kill_controller -> redeploy_app -> restore_verify -> backup -> detach_target -> detach_nontarget -12:29:25 cycle 13 done — violations so far: 0 -12:29:27 cycle 14: backup -> restore_verify -> detach_nontarget -> redeploy_app -> kill_controller -> detach_target -> reboot_guest -12:43:28 cycle 14 done — violations so far: 0 -12:43:30 cycle 15: detach_target -> backup -> kill_controller -> redeploy_app -> restore_verify -> detach_nontarget -12:56:54 cycle 15 done — violations so far: 0 -12:56:56 cycle 16: backup -> detach_target -> detach_nontarget -> restore_verify -> kill_controller -> redeploy_app -13:12:43 cycle 16 done — violations so far: 0 -13:12:49 cycle 17: redeploy_app -> detach_nontarget -> backup -> kill_controller -> restore_verify -> detach_target -13:26:12 cycle 17 done — violations so far: 0 -13:26:14 cycle 18: detach_target -> kill_controller -> detach_nontarget -> backup -> redeploy_app -> restore_verify -13:39:26 cycle 18 done — violations so far: 0 -13:39:28 cycle 19: kill_controller -> redeploy_app -> restore_verify -> detach_nontarget -> detach_target -> backup -13:52:39 cycle 19 done — violations so far: 0 -13:52:41 cycle 20: backup -> detach_target -> detach_nontarget -> redeploy_app -> restore_verify -> kill_controller -14:06:05 cycle 20 done — violations so far: 0 -14:06:07 cycle 21: backup -> kill_controller -> detach_nontarget -> redeploy_app -> detach_target -> restore_verify -> reboot_guest -14:19:56 cycle 21 done — violations so far: 0 -14:19:58 cycle 22: restore_verify -> redeploy_app -> backup -> detach_nontarget -> kill_controller -> detach_target -14:33:30 cycle 22 done — violations so far: 0 -14:33:32 cycle 23: backup -> detach_target -> restore_verify -> kill_controller -> redeploy_app -> detach_nontarget -14:46:43 cycle 23 done — violations so far: 0 -14:46:45 cycle 24: restore_verify -> backup -> detach_target -> kill_controller -> detach_nontarget -> redeploy_app -14:59:59 cycle 24 done — violations so far: 0 -15:00:01 cycle 25: detach_nontarget -> kill_controller -> detach_target -> restore_verify -> backup -> redeploy_app -15:13:24 cycle 25 done — violations so far: 0 -15:13:26 cycle 26: redeploy_app -> detach_nontarget -> restore_verify -> kill_controller -> detach_target -> backup -15:26:37 cycle 26 done — violations so far: 0 -15:26:39 cycle 27: backup -> restore_verify -> detach_target -> kill_controller -> detach_nontarget -> redeploy_app -15:40:13 cycle 27 done — violations so far: 0 -15:40:13 === finished after 27 cycles, 0 violations === +17:10:45 === Campaign 10 Phase B start — max 45 cycles, deadline 07:10 === +17:10:47 cycle 1: detach_nontarget -> kill_agent_mid_backup -> backup -> restore_verify -> detach_target -> redeploy_app -> kill_controller diff --git a/documentation/tests/campaign10-evidence-2026-07-31/state/status.txt b/documentation/tests/campaign10-evidence-2026-07-31/state/status.txt index 89cbf48..b204e91 100644 --- a/documentation/tests/campaign10-evidence-2026-07-31/state/status.txt +++ b/documentation/tests/campaign10-evidence-2026-07-31/state/status.txt @@ -1,4 +1,4 @@ -cycle=27 -phase=finished -utc=2026-08-01T13:40:13Z +cycle=1 +phase=atom_detach_nontarget +utc=2026-08-01T15:10:47Z violations=0