Decisions 157-158 recorded; R-892 corrected (VM 341), R-894 filed (unreadable off-site storage reads due after an agent restart); Part C/E/F evidence; nodes.md: the Tester 1 box; 11 §5.8: the memory-kill check
gates / gates (push) Successful in 2m47s
gates / gates (push) Successful in 2m47s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -0,0 +1,133 @@
|
||||
Sep 22 04:29:29 demo-hp felhom-agent[1195783]: time=2026-09-22T04:29:29.642+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-15T04:11:51Z) is already proven; local: newest settled archive (landed 2026-09-20T04:43:47Z) is already proven"
|
||||
Sep 22 06:12:46 demo-hp felhom-agent[1195783]: time=2026-09-22T06:12:46.231+02:00 level=INFO msg="local-api: backup reached snapshotted (app may resume)" vmid=9201 target=felhom-pbs job=backup-9201-felhom-pbs-1790050365164112326
|
||||
Sep 22 06:16:22 demo-hp felhom-agent[1195783]: time=2026-09-22T06:16:22.067+02:00 level=INFO msg="backup: completed" vmid=9201 target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-22T04:12:45Z size_bytes=18418647089 uncovered_volumes=2
|
||||
Sep 22 06:16:22 demo-hp felhom-agent[1195783]: time=2026-09-22T06:16:22.067+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=felhom-pbs job=backup-9201-felhom-pbs-1790050365164112326 archive=felhom-pbs:backup/ct/9201/2026-09-22T04:12:45Z
|
||||
Sep 22 06:58:40 demo-hp felhom-agent[1195783]: time=2026-09-22T06:58:40.652+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_09_22-06_52_48.tar.zst size_bytes=6267269544 uncovered_volumes=2
|
||||
Sep 22 06:58:40 demo-hp felhom-agent[1195783]: time=2026-09-22T06:58:40.652+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790052768442108602 archive=local:backup/vzdump-lxc-9201-2026_09_22-06_52_48.tar.zst
|
||||
Sep 22 10:29:29 demo-hp felhom-agent[1195783]: time=2026-09-22T10:29:29.346+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=local archive=local:backup/vzdump-lxc-9201-2026_09_21-06_47_30.tar.zst landed=2026-09-21T04:47:30Z reason="newest settled archive (landed 2026-09-21T04:47:30Z) has not been proven (last proven archive was a dif
|
||||
Sep 22 10:34:49 demo-hp felhom-agent[1195783]: time=2026-09-22T10:34:49.417+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=local:backup/vzdump-lxc-9201-2026_09_21-06_47_30.tar.zst duration_s=320.064619459
|
||||
Sep 22 16:29:29 demo-hp felhom-agent[1195783]: time=2026-09-22T16:29:29.663+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-15T04:11:51Z) is already proven; local: newest settled archive (landed 2026-09-21T04:47:30Z) is already proven"
|
||||
Sep 22 22:29:29 demo-hp felhom-agent[1195783]: time=2026-09-22T22:29:29.659+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-15T04:11:51Z) is already proven; local: newest settled archive (landed 2026-09-21T04:47:30Z) is already proven"
|
||||
Sep 23 04:29:29 demo-hp felhom-agent[1195783]: time=2026-09-23T04:29:29.655+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-15T04:11:51Z) is already proven; local: newest settled archive (landed 2026-09-21T04:47:30Z) is already proven"
|
||||
Sep 23 07:02:53 demo-hp felhom-agent[1195783]: time=2026-09-23T07:02:53.924+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_09_23-06_55_25.tar.zst size_bytes=7417540996 uncovered_volumes=2
|
||||
Sep 23 07:02:53 demo-hp felhom-agent[1195783]: time=2026-09-23T07:02:53.924+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790139325067362818 archive=local:backup/vzdump-lxc-9201-2026_09_23-06_55_25.tar.zst
|
||||
Sep 23 10:29:29 demo-hp felhom-agent[1195783]: time=2026-09-23T10:29:29.358+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-22T04:12:45Z landed=2026-09-22T04:12:45Z reason="newest settled archive (landed 2026-09-22T04:12:45Z) has not been proven (last proven archive was a differen
|
||||
Sep 23 10:37:41 demo-hp felhom-agent[1195783]: time=2026-09-23T10:37:41.197+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=felhom-pbs:backup/ct/9201/2026-09-22T04:12:45Z duration_s=491.830022319
|
||||
Sep 23 16:29:29 demo-hp felhom-agent[1195783]: time=2026-09-23T16:29:29.364+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=local archive=local:backup/vzdump-lxc-9201-2026_09_22-06_52_48.tar.zst landed=2026-09-22T04:52:48Z reason="newest settled archive (landed 2026-09-22T04:52:48Z) has not been proven (last proven archive was a dif
|
||||
Sep 23 16:35:15 demo-hp felhom-agent[1195783]: time=2026-09-23T16:35:15.142+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=local:backup/vzdump-lxc-9201-2026_09_22-06_52_48.tar.zst duration_s=345.775360191
|
||||
Sep 23 22:29:29 demo-hp felhom-agent[1195783]: time=2026-09-23T22:29:29.632+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-22T04:12:45Z) is already proven; local: newest settled archive (landed 2026-09-22T04:52:48Z) is already proven"
|
||||
Sep 24 04:29:29 demo-hp felhom-agent[1195783]: time=2026-09-24T04:29:29.641+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-22T04:12:45Z) is already proven; local: newest settled archive (landed 2026-09-22T04:52:48Z) is already proven"
|
||||
Sep 24 07:05:52 demo-hp felhom-agent[1195783]: time=2026-09-24T07:05:52.441+02:00 level=ERROR msg="local-api: backup job failed" vmid=9201 target=local job=backup-9201-1790225968108254908 err="backup: vzdump task vmid 9201: proxmox: task UPID:demo-hp:001C2729:11475EED:6AB4AE30:vzdump:9201:felhom-agent@pve!agent: failed: exitstatus \"job errors\""
|
||||
Sep 24 07:24:21 demo-hp felhom-agent[1195783]: time=2026-09-24T07:24:21.352+02:00 level=ERROR msg="local-api: backup job failed" vmid=9201 target=local job=backup-9201-1790227073555655065 err="backup: vzdump task vmid 9201: proxmox: task UPID:demo-hp:001D29F4:11490EBD:6AB4B281:vzdump:9201:felhom-agent@pve!agent: failed: exitstatus \"job errors\""
|
||||
Sep 24 07:49:16 demo-hp felhom-agent[1195783]: time=2026-09-24T07:49:16.829+02:00 level=ERROR msg="local-api: backup job failed" vmid=9201 target=local job=backup-9201-1790228574136769346 err="backup: vzdump task vmid 9201: proxmox: task UPID:demo-hp:001E7D73:114B58E7:6AB4B85E:vzdump:9201:felhom-agent@pve!agent: failed: exitstatus \"job errors\""
|
||||
Sep 24 08:22:57 demo-hp felhom-agent[1195783]: time=2026-09-24T08:22:57.653+02:00 level=ERROR msg="local-api: backup job failed" vmid=9201 target=local job=backup-9201-1790230975578595274 err="backup: vzdump task vmid 9201: proxmox: task UPID:demo-hp:0020B26F:114F02F7:6AB4C1BF:vzdump:9201:felhom-agent@pve!agent: failed: exitstatus \"job errors\""
|
||||
Sep 24 10:29:29 demo-hp felhom-agent[1195783]: time=2026-09-24T10:29:29.327+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=local archive=local:backup/vzdump-lxc-9201-2026_09_23-06_55_25.tar.zst landed=2026-09-23T04:55:25Z reason="newest settled archive (landed 2026-09-23T04:55:25Z) has not been proven (last proven archive was a dif
|
||||
Sep 24 10:36:30 demo-hp felhom-agent[1195783]: time=2026-09-24T10:36:30.331+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=local:backup/vzdump-lxc-9201-2026_09_23-06_55_25.tar.zst err="reconcile: restore-test start task: proxmox: task UPID:demo-hp:002789C4:115B3A8C:6AB4E106:vzstart:990000:felhom-agent@pve!agent: failed: exitstatus \"startup for container '990000' failed\""
|
||||
Sep 24 12:47:34 demo-hp felhom-agent[1195783]: time=2026-09-24T12:47:34.646+02:00 level=WARN msg="local-api: could not read the backup storage for the due-check — falling back to the in-memory record" vmid=9201 target=felhom-pbs err="proxmox: GET /nodes/demo-hp/storage/felhom-pbs/content -> HTTP 500: {\"data\":null,\"message\":\"felhom-pbs: error fetching datastores - 500 Can't connect to 10.77.
|
||||
Sep 24 13:10:04 demo-hp felhom-agent[3134463]: time=2026-09-24T13:10:04.268+02:00 level=INFO msg="stale-lock: removed dangling vzdump snapshot" vmid=9201
|
||||
Sep 24 22:06:23 demo-hp felhom-agent[756657]: time=2026-09-24T22:06:23.065+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_09_24-21_59_25.tar.zst size_bytes=8182759056 uncovered_volumes=2
|
||||
Sep 24 22:06:23 demo-hp felhom-agent[756657]: time=2026-09-24T22:06:23.065+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790279965190292701 archive=local:backup/vzdump-lxc-9201-2026_09_24-21_59_25.tar.zst
|
||||
Sep 24 22:06:26 demo-hp felhom-agent[756657]: time=2026-09-24T22:06:26.321+02:00 level=INFO msg="local-api: backup reached snapshotted (app may resume)" vmid=9201 target=felhom-pbs job=backup-9201-felhom-pbs-1790280385256228451
|
||||
Sep 24 22:06:45 demo-hp felhom-agent[756657]: time=2026-09-24T22:06:45.066+02:00 level=WARN msg="backup: ignoring an INCOMPLETE archive when computing tier freshness — it is not a successful backup" target=felhom-pbs vmid=9201 volid=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z size_bytes=1 reason="size 1 B is below the 1048576 B plausibility floor — an aborted/incomplete archive, not a succe
|
||||
Sep 24 22:07:45 demo-hp felhom-agent[756657]: time=2026-09-24T22:07:45.839+02:00 level=INFO msg="janitor: stale-lock sweep deferred — a heavy operation is in flight" busy=backup:felhom-pbs
|
||||
Sep 24 22:10:57 demo-hp felhom-agent[756657]: time=2026-09-24T22:10:57.339+02:00 level=INFO msg="backup: completed" vmid=9201 target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z size_bytes=23236593106 uncovered_volumes=2
|
||||
Sep 24 22:10:57 demo-hp felhom-agent[756657]: time=2026-09-24T22:10:57.339+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=felhom-pbs job=backup-9201-felhom-pbs-1790280385256228451 archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z
|
||||
Sep 25 22:57:53 demo-hp felhom-agent[995644]: time=2026-09-25T22:57:53.343+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different
|
||||
Sep 25 22:57:54 demo-hp felhom-agent[995644]: time=2026-09-25T22:57:54.955+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620982 avail_bytes=23108536763 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has
|
||||
Sep 25 22:57:54 demo-hp felhom-agent[995644]: time=2026-09-25T22:57:54.955+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.5 GiB"
|
||||
Sep 26 04:38:34 demo-hp felhom-agent[995644]: time=2026-09-26T04:38:34.211+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_09_26-04_30_46.tar.zst size_bytes=8384567204 uncovered_volumes=2
|
||||
Sep 26 04:38:34 demo-hp felhom-agent[995644]: time=2026-09-26T04:38:34.211+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790389845916353490 archive=local:backup/vzdump-lxc-9201-2026_09_26-04_30_46.tar.zst
|
||||
Sep 26 04:57:53 demo-hp felhom-agent[995644]: time=2026-09-26T04:57:53.350+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different
|
||||
Sep 26 04:57:55 demo-hp felhom-agent[995644]: time=2026-09-26T04:57:55.007+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620981 avail_bytes=23079614940 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has
|
||||
Sep 26 04:57:55 demo-hp felhom-agent[995644]: time=2026-09-26T04:57:55.007+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.5 GiB"
|
||||
Sep 26 10:57:53 demo-hp felhom-agent[995644]: time=2026-09-26T10:57:53.346+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different
|
||||
Sep 26 10:57:54 demo-hp felhom-agent[995644]: time=2026-09-26T10:57:54.951+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620982 avail_bytes=23068046210 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has
|
||||
Sep 26 10:57:54 demo-hp felhom-agent[995644]: time=2026-09-26T10:57:54.951+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.5 GiB"
|
||||
Sep 26 16:57:53 demo-hp felhom-agent[995644]: time=2026-09-26T16:57:53.359+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different
|
||||
Sep 26 16:57:55 demo-hp felhom-agent[995644]: time=2026-09-26T16:57:55.016+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620982 avail_bytes=23056477481 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has
|
||||
Sep 26 16:57:55 demo-hp felhom-agent[995644]: time=2026-09-26T16:57:55.017+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.5 GiB"
|
||||
Sep 26 22:57:53 demo-hp felhom-agent[995644]: time=2026-09-26T22:57:53.370+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different
|
||||
Sep 26 22:57:55 demo-hp felhom-agent[995644]: time=2026-09-26T22:57:55.013+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620982 avail_bytes=23044908752 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has
|
||||
Sep 26 22:57:55 demo-hp felhom-agent[995644]: time=2026-09-26T22:57:55.013+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.5 GiB"
|
||||
Sep 27 04:43:29 demo-hp felhom-agent[995644]: time=2026-09-27T04:43:29.138+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_09_27-04_35_47.tar.zst size_bytes=8380885161 uncovered_volumes=2
|
||||
Sep 27 04:43:29 demo-hp felhom-agent[995644]: time=2026-09-27T04:43:29.138+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790476546353426458 archive=local:backup/vzdump-lxc-9201-2026_09_27-04_35_47.tar.zst
|
||||
Sep 27 04:57:53 demo-hp felhom-agent[995644]: time=2026-09-27T04:57:53.352+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different
|
||||
Sep 27 04:57:54 demo-hp felhom-agent[995644]: time=2026-09-27T04:57:54.985+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620981 avail_bytes=22663140685 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has
|
||||
Sep 27 04:57:54 demo-hp felhom-agent[995644]: time=2026-09-27T04:57:54.985+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.1 GiB"
|
||||
Sep 27 10:57:53 demo-hp felhom-agent[995644]: time=2026-09-27T10:57:53.351+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different
|
||||
Sep 27 10:57:54 demo-hp felhom-agent[995644]: time=2026-09-27T10:57:54.968+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620981 avail_bytes=22657356320 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has
|
||||
Sep 27 10:57:54 demo-hp felhom-agent[995644]: time=2026-09-27T10:57:54.968+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.1 GiB"
|
||||
Sep 27 20:13:17 demo-hp felhom-agent[1370926]: time=2026-09-27T20:13:17.853+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a differen
|
||||
Sep 27 20:13:19 demo-hp felhom-agent[1370926]: time=2026-09-27T20:13:19.466+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620981 avail_bytes=22657356320 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, ha
|
||||
Sep 27 20:13:19 demo-hp felhom-agent[1370926]: time=2026-09-27T20:13:19.467+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.1 GiB"
|
||||
Sep 28 02:13:17 demo-hp felhom-agent[1370926]: time=2026-09-28T02:13:17.870+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a differen
|
||||
Sep 28 02:13:19 demo-hp felhom-agent[1370926]: time=2026-09-28T02:13:19.630+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620982 avail_bytes=22657356320 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, ha
|
||||
Sep 28 02:13:19 demo-hp felhom-agent[1370926]: time=2026-09-28T02:13:19.630+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.1 GiB"
|
||||
Sep 28 04:44:58 demo-hp felhom-agent[1370926]: time=2026-09-28T04:44:58.683+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_09_28-04_37_06.tar.zst size_bytes=8825181955 uncovered_volumes=2
|
||||
Sep 28 04:44:58 demo-hp felhom-agent[1370926]: time=2026-09-28T04:44:58.683+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790563025947283530 archive=local:backup/vzdump-lxc-9201-2026_09_28-04_37_06.tar.zst
|
||||
Sep 28 08:13:17 demo-hp felhom-agent[1370926]: time=2026-09-28T08:13:17.866+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a differen
|
||||
Sep 28 08:13:19 demo-hp felhom-agent[1370926]: time=2026-09-28T08:13:19.655+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620982 avail_bytes=21801270353 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, ha
|
||||
Sep 28 08:13:19 demo-hp felhom-agent[1370926]: time=2026-09-28T08:13:19.655+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 20.3 GiB"
|
||||
Sep 28 14:13:17 demo-hp felhom-agent[1370926]: time=2026-09-28T14:13:17.848+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a differen
|
||||
Sep 28 14:13:19 demo-hp felhom-agent[1370926]: time=2026-09-28T14:13:19.532+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620982 avail_bytes=25653657207 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, ha
|
||||
Sep 28 14:13:19 demo-hp felhom-agent[1370926]: time=2026-09-28T14:13:19.533+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 23.9 GiB"
|
||||
Sep 28 22:25:23 demo-hp felhom-agent[2355282]: time=2026-09-28T22:25:23.861+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a differen
|
||||
Sep 28 22:33:59 demo-hp felhom-agent[2355282]: time=2026-09-28T22:33:59.872+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z duration_s=516.008133181
|
||||
Sep 29 04:25:24 demo-hp felhom-agent[2355282]: time=2026-09-29T04:25:24.198+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven"
|
||||
Sep 29 04:50:53 demo-hp felhom-agent[2355282]: time=2026-09-29T04:50:53.815+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_09_29-04_42_11.tar.zst size_bytes=9516315002 uncovered_volumes=2
|
||||
Sep 29 04:50:53 demo-hp felhom-agent[2355282]: time=2026-09-29T04:50:53.815+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790649730829915100 archive=local:backup/vzdump-lxc-9201-2026_09_29-04_42_11.tar.zst
|
||||
Sep 29 10:25:24 demo-hp felhom-agent[2355282]: time=2026-09-29T10:25:24.184+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven"
|
||||
Sep 29 16:25:24 demo-hp felhom-agent[2355282]: time=2026-09-29T16:25:24.219+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven"
|
||||
Sep 29 22:25:24 demo-hp felhom-agent[2355282]: time=2026-09-29T22:25:24.194+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven"
|
||||
Sep 30 04:25:24 demo-hp felhom-agent[2355282]: time=2026-09-30T04:25:24.194+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven"
|
||||
Sep 30 10:25:23 demo-hp felhom-agent[2355282]: time=2026-09-30T10:25:23.855+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=local archive=local:backup/vzdump-lxc-9201-2026_09_29-04_42_11.tar.zst landed=2026-09-29T02:42:11Z reason="newest settled archive (landed 2026-09-29T02:42:11Z) has not been proven (last proven archive was a dif
|
||||
Sep 30 10:27:48 demo-hp felhom-agent[2355282]: time=2026-09-30T10:27:48.373+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=local:backup/vzdump-lxc-9201-2026_09_29-04_42_11.tar.zst duration_s=144.515446972
|
||||
Sep 30 13:47:12 demo-hp felhom-agent[2135127]: time=2026-09-30T13:47:12.666+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_09_30-13_37_38.tar.zst size_bytes=9978959837 uncovered_volumes=2
|
||||
Sep 30 13:47:12 demo-hp felhom-agent[2135127]: time=2026-09-30T13:47:12.666+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790768258239656621 archive=local:backup/vzdump-lxc-9201-2026_09_30-13_37_38.tar.zst
|
||||
Sep 30 16:40:32 demo-hp felhom-agent[2135127]: time=2026-09-30T16:40:32.449+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven; local: no settled archive yet — nothing to prove (newborn or still settling)"
|
||||
Sep 30 22:40:32 demo-hp felhom-agent[2135127]: time=2026-09-30T22:40:32.547+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven; local: no settled archive yet — nothing to prove (newborn or still settling)"
|
||||
Oct 01 04:40:32 demo-hp felhom-agent[2135127]: time=2026-10-01T04:40:32.442+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven; local: no settled archive yet — nothing to prove (newborn or still settling)"
|
||||
Oct 01 08:11:08 demo-hp felhom-agent[2135127]: time=2026-10-01T08:11:08.130+02:00 level=WARN msg="local-api: storage view unavailable for the backup-target presence check — assuming present" target=felhom-pbs err="storage: NodeStorage: proxmox: GET /nodes/demo-hp/storage: Get \"https://127.0.0.1:8006/api2/json/nodes/demo-hp/storage\": context canceled"
|
||||
Oct 01 08:11:08 demo-hp felhom-agent[2135127]: time=2026-10-01T08:11:08.130+02:00 level=WARN msg="local-api: could not read the backup storage for the due-check — falling back to the in-memory record" vmid=9201 target=felhom-pbs err="proxmox: GET /nodes/demo-hp/storage/felhom-pbs/content: Get \"https://127.0.0.1:8006/api2/json/nodes/demo-hp/storage/felhom-pbs/content\": context canceled"
|
||||
Oct 01 10:40:32 demo-hp felhom-agent[2135127]: time=2026-10-01T10:40:32.422+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven; local: no settled archive yet — nothing to prove (newborn or still settling)"
|
||||
Oct 01 16:40:32 demo-hp felhom-agent[2135127]: time=2026-10-01T16:40:32.078+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=local archive=local:backup/vzdump-lxc-9201-2026_09_30-13_37_38.tar.zst landed=2026-09-30T11:37:38Z reason="newest settled archive (landed 2026-09-30T11:37:38Z) has not been proven (last proven archive was a dif
|
||||
Oct 01 16:43:11 demo-hp felhom-agent[2135127]: time=2026-10-01T16:43:11.704+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=local:backup/vzdump-lxc-9201-2026_09_30-13_37_38.tar.zst duration_s=159.623154174
|
||||
Oct 01 22:15:25 demo-hp felhom-agent[2135127]: time=2026-10-01T22:15:25.563+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_01-22_10_48.tar.zst size_bytes=4681052311 uncovered_volumes=2
|
||||
Oct 01 22:15:25 demo-hp felhom-agent[2135127]: time=2026-10-01T22:15:25.563+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790885448142698430 archive=local:backup/vzdump-lxc-9201-2026_10_01-22_10_48.tar.zst
|
||||
Oct 01 22:15:30 demo-hp felhom-agent[2135127]: time=2026-10-01T22:15:30.441+02:00 level=INFO msg="local-api: backup reached snapshotted (app may resume)" vmid=9201 target=felhom-pbs job=backup-9201-felhom-pbs-1790885728244905026
|
||||
Oct 01 22:18:31 demo-hp felhom-agent[2135127]: time=2026-10-01T22:18:31.284+02:00 level=INFO msg="backup: completed" vmid=9201 target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-10-01T20:15:29Z size_bytes=15223817181 uncovered_volumes=2
|
||||
Oct 01 22:18:31 demo-hp felhom-agent[2135127]: time=2026-10-01T22:18:31.284+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=felhom-pbs job=backup-9201-felhom-pbs-1790885728244905026 archive=felhom-pbs:backup/ct/9201/2026-10-01T20:15:29Z
|
||||
Oct 01 22:40:32 demo-hp felhom-agent[2135127]: time=2026-10-01T22:40:32.400+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven; local: no settled archive yet — nothing to prove (newborn or still settling)"
|
||||
Oct 02 04:40:32 demo-hp felhom-agent[2135127]: time=2026-10-02T04:40:32.387+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven; local: no settled archive yet — nothing to prove (newborn or still settling)"
|
||||
Oct 02 10:40:32 demo-hp felhom-agent[2135127]: time=2026-10-02T10:40:32.460+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven; local: no settled archive yet — nothing to prove (newborn or still settling)"
|
||||
Oct 02 16:40:32 demo-hp felhom-agent[2135127]: time=2026-10-02T16:40:32.406+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-09-24T20:06:25Z) is already proven; local: no settled archive yet — nothing to prove (newborn or still settling)"
|
||||
Oct 02 22:40:32 demo-hp felhom-agent[2135127]: time=2026-10-02T22:40:32.065+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-10-01T20:15:29Z landed=2026-10-01T20:15:29Z reason="newest settled archive (landed 2026-10-01T20:15:29Z) has not been proven (last proven archive was a differen
|
||||
Oct 02 22:46:02 demo-hp felhom-agent[2135127]: time=2026-10-02T22:46:02.535+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=felhom-pbs:backup/ct/9201/2026-10-01T20:15:29Z duration_s=330.466115271
|
||||
Oct 03 04:38:20 demo-hp felhom-agent[2135127]: time=2026-10-03T04:38:20.266+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_03-04_33_33.tar.zst size_bytes=4674947408 uncovered_volumes=2
|
||||
Oct 03 04:38:20 demo-hp felhom-agent[2135127]: time=2026-10-03T04:38:20.266+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1790994812809499660 archive=local:backup/vzdump-lxc-9201-2026_10_03-04_33_33.tar.zst
|
||||
Oct 03 04:40:32 demo-hp felhom-agent[2135127]: time=2026-10-03T04:40:32.407+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 03 10:40:32 demo-hp felhom-agent[2135127]: time=2026-10-03T10:40:32.425+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 03 16:40:32 demo-hp felhom-agent[2135127]: time=2026-10-03T16:40:32.455+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 03 22:40:32 demo-hp felhom-agent[2135127]: time=2026-10-03T22:40:32.411+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 04 04:39:41 demo-hp felhom-agent[2135127]: time=2026-10-04T04:39:41.809+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_04-04_34_55.tar.zst size_bytes=4679944932 uncovered_volumes=2
|
||||
Oct 04 04:39:41 demo-hp felhom-agent[2135127]: time=2026-10-04T04:39:41.809+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1791081294394153713 archive=local:backup/vzdump-lxc-9201-2026_10_04-04_34_55.tar.zst
|
||||
Oct 04 04:40:32 demo-hp felhom-agent[2135127]: time=2026-10-04T04:40:32.412+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 05 01:49:22 demo-hp felhom-agent[447632]: time=2026-10-05T01:49:22.786+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 05 04:40:16 demo-hp felhom-agent[447632]: time=2026-10-05T04:40:16.647+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_05-04_35_24.tar.zst size_bytes=4882309520 uncovered_volumes=2
|
||||
Oct 05 04:40:16 demo-hp felhom-agent[447632]: time=2026-10-05T04:40:16.647+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1791167723831350602 archive=local:backup/vzdump-lxc-9201-2026_10_05-04_35_24.tar.zst
|
||||
Oct 05 06:25:36 demo-hp felhom-agent[2031788]: time=2026-10-05T06:25:36.366+02:00 level=WARN msg="local-api: could not read the backup storage for the due-check — falling back to the in-memory record" vmid=9201 target=felhom-pbs err="proxmox: GET /nodes/demo-hp/storage/felhom-pbs/content -> HTTP 500: {\"data\":null,\"message\":\"felhom-pbs: error fetching datastores - 500 Can't connect to 10.77.
|
||||
Oct 05 06:26:23 demo-hp felhom-agent[2031788]: time=2026-10-05T06:26:23.344+02:00 level=ERROR msg="local-api: backup job failed" vmid=9201 target=felhom-pbs job=backup-9201-felhom-pbs-1791174359051131871 err="backup: vzdump task vmid 9201: proxmox: task UPID:demo-hp:0022DC8C:00483400:6AC326DF:vzdump:9201:felhom-agent@pve!agent: failed: exitstatus \"could not activate storage 'felhom-pbs': felhom-p
|
||||
Oct 05 06:30:36 demo-hp felhom-agent[2031788]: time=2026-10-05T06:30:36.395+02:00 level=WARN msg="local-api: could not read the backup storage for the due-check — falling back to the in-memory record" vmid=9201 target=felhom-pbs err="proxmox: GET /nodes/demo-hp/storage/felhom-pbs/content -> HTTP 500: {\"message\":\"felhom-pbs: error fetching datastores - 500 Can't connect to 10.77.0.1:8007 (Conn
|
||||
Oct 05 06:35:36 demo-hp felhom-agent[2031788]: time=2026-10-05T06:35:36.377+02:00 level=WARN msg="local-api: could not read the backup storage for the due-check — falling back to the in-memory record" vmid=9201 target=felhom-pbs err="proxmox: GET /nodes/demo-hp/storage/felhom-pbs/content -> HTTP 500: {\"data\":null,\"message\":\"felhom-pbs: error fetching datastores - 500 Can't connect to 10.77.
|
||||
Oct 05 10:26:59 demo-hp felhom-agent[1323]: time=2026-10-05T10:26:59.170+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 05 11:24:01 demo-hp felhom-agent[1323]: time=2026-10-05T11:24:01.884+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_05-11_19_30.tar.zst size_bytes=4880501245 uncovered_volumes=2
|
||||
Oct 05 11:24:01 demo-hp felhom-agent[1323]: time=2026-10-05T11:24:01.884+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1791191969558324187 archive=local:backup/vzdump-lxc-9201-2026_10_05-11_19_30.tar.zst
|
||||
Oct 05 11:24:09 demo-hp felhom-agent[1323]: time=2026-10-05T11:24:09.634+02:00 level=INFO msg="local-api: backup refused — a heavy operation is already in flight" vmid=9201 requested_target=felhom-pbs busy=backup:local
|
||||
Oct 05 12:57:08 demo-hp felhom-agent[453394]: time=2026-10-05T12:57:08.743+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 05 18:57:09 demo-hp felhom-agent[453394]: time=2026-10-05T18:57:09.755+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 05 19:42:24 demo-hp felhom-agent[1625894]: time=2026-10-05T19:42:24.219+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 06 01:12:37 demo-hp felhom-agent[2556733]: time=2026-10-06T01:12:37.224+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 06 07:12:38 demo-hp felhom-agent[2556733]: time=2026-10-06T07:12:38.110+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven"
|
||||
Oct 06 12:21:17 demo-hp felhom-agent[297545]: time=2026-10-06T12:21:17.652+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=local archive=local:backup/vzdump-lxc-9201-2026_10_05-11_19_30.tar.zst landed=2026-10-05T09:19:30Z reason="newest settled archive (landed 2026-10-05T09:19:30Z) has not been proven (last proven archive was a diff
|
||||
Oct 06 12:22:46 demo-hp felhom-agent[297545]: time=2026-10-06T12:22:46.987+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=local:backup/vzdump-lxc-9201-2026_10_05-11_19_30.tar.zst duration_s=89.331269652
|
||||
Oct 06 18:22:47 demo-hp felhom-agent[297545]: time=2026-10-06T18:22:47.786+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven; local: newest settled archive (landed 2026-10-05T09:19:30Z) is already proven"
|
||||
@@ -0,0 +1,14 @@
|
||||
Tester-2 ct 9201 2026-10-04T16:31:13Z 2096481498 ok
|
||||
<string>:5: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
|
||||
demo-felhom ct 9201 2026-09-22T04:12:20Z 5840921438 ok
|
||||
demo-felhom ct 9201 2026-09-29T04:16:43Z 6956268440 ok
|
||||
demo-felhom ct 9201 2026-10-06T04:21:09Z 2749038678 ok
|
||||
<string>:5: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
|
||||
<string>:5: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
|
||||
demo-hp ct 9201 2026-09-24T20:06:25Z 23236593218 ok
|
||||
demo-hp ct 9201 2026-10-01T20:15:29Z 15223817295 ok
|
||||
operator host dooplex-hub 2026-10-05T14:23:25Z 369808250 ok
|
||||
operator host dooplex-hub 2026-10-06T00:32:21Z 372921211 ok
|
||||
<string>:5: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
|
||||
tester-1 ct 9201 2026-10-04T19:57:18Z 4913374015 ok
|
||||
<string>:5: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
|
||||
@@ -0,0 +1,48 @@
|
||||
events columns: ['id', 'customer_id', 'event_type', 'severity', 'message', 'details_json', 'source', 'created_at']
|
||||
('2026-10-05 04:27:31', 'whole_guest_backup_failed', 'error', 'Whole-guest backup FAILED on the felhom-pbs tier — retrying with backoff (next attempt in 15m0s)')
|
||||
('2026-10-05 05:16:22', 'controller_updated', 'info', 'Controller frissítve: 0.293.0 → 0.294.0')
|
||||
('2026-10-05 05:16:27', 'controller_started', 'info', 'Controller elindult (0.294.0)')
|
||||
('2026-10-05 06:09:05', 'os_update_applied', 'info', 'System security fixes installed on the box (13 package(s)).')
|
||||
('2026-10-05 06:11:02', 'os_update_applied', 'info', 'System security fixes installed on the box (13 package(s)).')
|
||||
('2026-10-05 06:16:39', 'controller_started', 'info', 'Controller elindult (0.294.0)')
|
||||
('2026-10-05 06:18:03', 'os_update_failed', 'error', 'OS update (guest) on demo-hp-bb76ea failed: {"dpkg_audit":"clean","rc":100,"tail":["E: dpkg was interrupted, you must manually run \'sudo dpkg --configure -a\' to correct the problem."]} no health readi')
|
||||
('2026-10-05 06:19:35', 'os_update_applied', 'info', 'System security fixes installed on the box (12 package(s)).')
|
||||
('2026-10-05 06:29:57', 'host_restarted_after_crash', 'info', 'Your box restarted by itself after an unexpected stop. You do not need to do anything.')
|
||||
('2026-10-05 06:29:57', 'host_crash_restart', 'warning', 'demo-hp-bb76ea restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T06:14:33Z — 4 such restart(s) in 24 h. kernel.panic now 10.')
|
||||
('2026-10-05 07:29:43', 'controller_updated', 'info', 'Controller frissítve: 0.294.0 → 0.295.0')
|
||||
('2026-10-05 07:29:48', 'controller_started', 'info', 'Controller elindult (0.295.0)')
|
||||
('2026-10-05 07:58:50', 'controller_started', 'info', 'Controller elindult (0.295.0)')
|
||||
('2026-10-05 08:00:42', 'os_update_applied', 'info', 'System security fixes installed on the box (12 package(s)).')
|
||||
('2026-10-05 08:12:06', 'host_restarted_after_crash', 'info', 'Your box restarted by itself after an unexpected stop. You do not need to do anything.')
|
||||
('2026-10-05 08:12:06', 'host_crash_restart', 'warning', 'demo-hp-bb76ea restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T07:56:41Z — 5 such restart(s) in 24 h. kernel.panic now 10.')
|
||||
('2026-10-05 09:16:15', 'recovery_credential_revealed', 'info', 'Az üzemeltető lekérte a géped konzolos hozzáférési jelszavát (távoli hibaelhárítás).')
|
||||
('2026-10-05 10:25:08', 'controller_updated', 'info', 'Controller frissítve: 0.295.0 → 0.296.0')
|
||||
('2026-10-05 10:25:13', 'controller_started', 'info', 'Controller elindult (0.296.0)')
|
||||
('2026-10-05 10:28:08', 'agent_capability_degraded', 'warning', 'Host demo-hp-bb76ea: agent privileged capability degraded — wg-conf-install (impairs: wg-felhom conf install (root content check, R-861))')
|
||||
('2026-10-05 11:13:08', 'agent_capability_recovered', 'info', 'Host demo-hp-bb76ea: agent privileged capabilities recovered (all required grants restored)')
|
||||
== notification_log demo-hp 2026-10-05:
|
||||
['id', 'customer_id', 'event_type', 'severity', 'message', 'status', 'error_message', 'created_at', 'channel']
|
||||
(1084, 'demo-hp', 'whole_guest_backup_failed', 'error', 'Whole-guest backup FAILED on the felhom-pbs tier — retrying with backoff (next attempt in 15m0s)', 'failed', 'sending request: Post "https://api.resend.com/emails": context deadline exceeded (Client.Timeout exceeded while awaiting headers)', '
|
||||
(1085, 'demo-hp', 'whole_guest_backup_failed', 'error', 'Whole-guest backup FAILED on the felhom-pbs tier — retrying with backoff (next attempt in 15m0s)', 'skipped', 'operator_only', '2026-10-05 04:27:41', 'customer')
|
||||
(1086, 'demo-hp', 'os_update_failed', 'error', 'OS update (guest) on demo-hp-bb76ea failed: {"dpkg_audit":"clean","rc":100,"tail":["E: dpkg was interrupted, you must manually run \'sudo dpkg --configure -a\' to correct the problem."]} no health reading', 'sent', '', '2026-10-05 06:18:03', 'operator'
|
||||
(1087, 'demo-hp', 'os_update_failed', 'error', 'OS update (guest) on demo-hp-bb76ea failed: {"dpkg_audit":"clean","rc":100,"tail":["E: dpkg was interrupted, you must manually run \'sudo dpkg --configure -a\' to correct the problem."]} no health reading', 'skipped', 'operator_only', '2026-10-05 06:18
|
||||
(1088, 'demo-hp', 'host_crash_restart', 'warning', 'demo-hp-bb76ea restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T06:14:33Z — 4 such restart(s) in 24 h. kernel.panic now 10.', 'sent', '', '2026-10-05 06:29:57', 'operator')
|
||||
(1089, 'demo-hp', 'host_crash_restart', 'warning', 'demo-hp-bb76ea restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T06:14:33Z — 4 such restart(s) in 24 h. kernel.panic now 10.', 'skipped', 'operator_only', '2026-10-05 06:29:57', 'customer')
|
||||
(1090, 'tester-1', 'node_stale', 'warning', 'No report received for 45m', 'sent', '', '2026-10-05 06:46:41', 'operator')
|
||||
(1091, 'tester-1', 'host_stale', 'warning', 'Host tester-1-d70be4: no report for 45m', 'sent', '', '2026-10-05 06:56:41', 'operator')
|
||||
(1092, 'tester-1', 'node_down', 'error', 'No report received for 1h30m', 'sent', '', '2026-10-05 07:31:41', 'operator')
|
||||
(1093, 'tester-1', 'host_recovered', 'info', 'Host tester-1-d70be4: reports resumed (was stale for 35m)', 'sent', '', '2026-10-05 07:31:42', 'operator')
|
||||
(1094, 'tester-1', 'host_crash_restart', 'warning', 'tester-1-d70be4 restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T07:31:20Z — 1 such restart(s) in 24 h. kernel.panic now 10.', 'sent', '', '2026-10-05 07:33:12', 'operator')
|
||||
(1095, 'tester-1', 'host_crash_restart', 'warning', 'tester-1-d70be4 restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T07:31:20Z — 1 such restart(s) in 24 h. kernel.panic now 10.', 'skipped', 'operator_only', '2026-10-05 07:33:12', 'customer')
|
||||
(1096, 'tester-1', 'node_recovered', 'info', 'Reports resumed (was down for 46m)', 'sent', '', '2026-10-05 07:33:41', 'operator')
|
||||
(1097, 'Tester-2', 'host_down', 'error', 'Host Tester-2-be8404: no report for 13h45m', 'sent', '', '2026-10-05 07:51:34', 'operator')
|
||||
(1098, 'demo-hp', 'host_crash_restart', 'warning', 'demo-hp-bb76ea restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T07:56:41Z — 5 such restart(s) in 24 h. kernel.panic now 10.', 'sent', '', '2026-10-05 08:12:06', 'operator')
|
||||
(1099, 'demo-hp', 'host_crash_restart', 'warning', 'demo-hp-bb76ea restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T07:56:41Z — 5 such restart(s) in 24 h. kernel.panic now 10.', 'skipped', 'operator_only', '2026-10-05 08:12:06', 'customer')
|
||||
(1100, 'tester-1', 'host_crash_guard_tripped', 'error', 'The crash guard of tester-1-d70be4 TRIPPED at 2026-10-05T07:57:17Z: 2 unclean boots within 60 minutes — the next crash leaves the box off (limit 3). The next crash leaves the box OFF until someone switches it on. Re-arm: `felhom-crash-guard re
|
||||
(1101, 'tester-1', 'host_crash_guard_tripped', 'error', 'The crash guard of tester-1-d70be4 TRIPPED at 2026-10-05T07:57:17Z: 2 unclean boots within 60 minutes — the next crash leaves the box off (limit 3). The next crash leaves the box OFF until someone switches it on. Re-arm: `felhom-crash-guard re
|
||||
(1102, 'tester-1', 'host_crash_restart', 'warning', 'tester-1-d70be4 restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T07:57:12Z — 2 such restart(s) in 24 h. kernel.panic now 0.', 'sent', '', '2026-10-05 08:12:32', 'operator')
|
||||
(1103, 'tester-1', 'host_crash_restart', 'warning', 'tester-1-d70be4 restarted by itself after an unexpected stop (a kernel crash, a power cut or a hard reset) at 2026-10-05T07:57:12Z — 2 such restart(s) in 24 h. kernel.panic now 0.', 'skipped', 'operator_only', '2026-10-05 08:12:32', 'customer')
|
||||
(1104, 'Tester-2', 'host_down', 'error', 'Host Tester-2-be8404: no report for 15h10m', 'sent', '', '2026-10-05 09:16:10', 'operator')
|
||||
(1105, 'demo-hp', 'agent_capability_degraded', 'warning', 'Host demo-hp-bb76ea: agent privileged capability degraded — wg-conf-install (impairs: wg-felhom conf install (root content check, R-861))', 'sent', '', '2026-10-05 10:28:08', 'operator')
|
||||
(1106, 'tester-1', 'agent_capability_degraded', 'warning', 'Host tester-1-d70be4: agent privileged capability degraded — wg-conf-install (impairs: wg-felhom conf install (root content check, R-861))', 'sent', '', '2026-10-05 10:28:09', 'operator')
|
||||
(1107, 'demo-felhom', 'agent_capability_degraded', 'warning', 'Host demo-felhom-8363b5: agent privileged capability degraded — wg-conf-install (impairs: wg-felhom conf install (root content check, R-861))', 'sent', '', '2026-10-05 10:39:09', 'operator')
|
||||
@@ -0,0 +1,6 @@
|
||||
all notification rows: (1112, '2026-02-16 18:46:11', '2026-10-06 09:56:51')
|
||||
by channel/status: [('customer', 'sent', 155), ('customer', 'skipped', 166), ('operator', 'failed', 1), ('operator', 'recorded', 22), ('operator', 'refused', 7), ('operator', 'sent', 691), ('operator', 'suppressed', 70)]
|
||||
failed OPERATOR mails, by severity: [('error', 1)]
|
||||
failed operator mails, error/critical, newest 15:
|
||||
('2026-10-05 04:27:41', 'demo-hp', 'whole_guest_backup_failed', 'error', 'sending request: Post "https://api.resend.com/emails": context deadline exceeded (Client.Timeout exceeded whil')
|
||||
failed CUSTOMER mails: (0,)
|
||||
@@ -0,0 +1,27 @@
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/cmd/hub 0.077s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/cmd/hubdb-check 0.092s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/api 5.785s
|
||||
? gitea.dooplex.hu/admin/felhom-hub/internal/assets [no test files]
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/claim 3.336s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/cloudflare 0.008s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/configgen 0.031s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/dbsnap 0.154s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/gitea 0.013s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/hetznerapi 0.053s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/i18n 0.009s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/intent 0.092s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/mailrelay 0.009s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/monitor 38.751s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/notify 1.624s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/offsite 0.761s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/offsiteheal 0.269s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys 0.245s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/osupdates 1.325s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/pbsdrheal 0.622s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/poke 0.009s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/semver 0.007s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/store 4.224s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/sysfacts 0.005s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/tenantsync 0.042s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/web 25.654s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/wgsync 0.237s
|
||||
@@ -0,0 +1,12 @@
|
||||
$ go test ./internal/notify/ -run TestOperatorMailRetry -v (processOperator without the retryOperatorEmail call)
|
||||
=== RUN TestOperatorMailRetry_FailedOperatorMailIsRetried
|
||||
r895_operator_retry_test.go:34: the failed mail was never sent again (tries=1)
|
||||
--- FAIL: TestOperatorMailRetry_FailedOperatorMailIsRetried (0.03s)
|
||||
=== RUN TestOperatorMailRetry_RetriesAreBounded
|
||||
r895_operator_retry_test.go:69: tries=1, want 4
|
||||
--- FAIL: TestOperatorMailRetry_RetriesAreBounded (0.03s)
|
||||
=== RUN TestOperatorMailRetry_SuccessIsNotRetried
|
||||
--- PASS: TestOperatorMailRetry_SuccessIsNotRetried (0.03s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/notify 0.104s
|
||||
FAIL
|
||||
@@ -0,0 +1,7 @@
|
||||
host_reports row for tester-1-d70be4 at 2026-10-06 16:51:29
|
||||
host.node = felhom
|
||||
guest_net.guests.ip = 192.168.0.101
|
||||
customer tester-1 domain: []
|
||||
VM 341 (demo-hp) PVE certificate: CN=felhom.enkicsifelhom.hu, SAN IP 192.168.0.154, DNS felhom
|
||||
felhom.enkicsifelhom.hu -> 200
|
||||
felhom.enkisfelhom.hu -> 404
|
||||
@@ -0,0 +1,10 @@
|
||||
Tue Oct 6 17:12:17 UTC 2026
|
||||
containers before: 21
|
||||
image: gitea.dooplex.hu/admin/felhom-controller:0.301.0
|
||||
Killed
|
||||
run rc=137
|
||||
inspect: true 137
|
||||
oom events:
|
||||
removed felhom-oomcheck-byhand-3226729
|
||||
left: 0; containers after: 21
|
||||
engine 29.8.2
|
||||
@@ -0,0 +1,7 @@
|
||||
run rc=137
|
||||
inspect: true 137 id=0f65ef3dde661a4f19
|
||||
A container=NAME event=oom, until T1+1: [oom felhom-oomcheck-byhand2-3227326 ]
|
||||
B event=oom only: [oom felhom-oomcheck-byhand2-3227326 ]
|
||||
C every event of the container: [create attach start oom die ]
|
||||
removed
|
||||
left: 0
|
||||
@@ -0,0 +1,63 @@
|
||||
== R-528 agent tests, clone /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528 branch r528-oom-check base de812bc + working tree, 2026-10-06T17:13:46Z
|
||||
$ python3 -W ignore configs/test_felhom_os_apply.py
|
||||
----------------------------------------------------------------------
|
||||
Ran 99 tests in 0.605s
|
||||
|
||||
OK
|
||||
rc=0
|
||||
$ python3 -W ignore configs/test_felhom_config_bundle.py
|
||||
----------------------------------------------------------------------
|
||||
Ran 55 tests in 1.085s
|
||||
|
||||
OK
|
||||
rc=0
|
||||
$ go build ./...
|
||||
rc=0
|
||||
$ go vet ./...
|
||||
rc=0
|
||||
$ go test ./...
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/cmd/felhom-agent 0.154s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/cmd/felhom-opsign 0.007s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/authz 0.018s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/backup 0.193s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/capability 0.178s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/config 0.006s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/desired 0.009s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/dr 0.009s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/escrow 54.459s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/fasttick 0.004s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/felhomsshd 0.243s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/fstrim 0.013s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/guesthook 0.012s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/guestnet 0.025s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/httpx 0.006s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/hub 1.150s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/lanresolver 0.149s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/localapi 1.611s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/log 0.006s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/mgmtplane 0.010s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/osupdate 2.266s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/pbs 0.263s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/pbsdr 0.066s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/poke 5.012s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/provision 0.013s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/proxmox 0.061s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/reconcile 0.278s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/restorespace 0.010s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/selfheal 0.005s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/selfupdate 0.018s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/signedjobs 0.015s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/storage 0.452s
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/wgtunnel 0.225s
|
||||
rc=0
|
||||
$ python3 scripts/agent_gates.py --fast
|
||||
==============================================================================
|
||||
== summary
|
||||
==============================================================================
|
||||
reuse-refs OK (exit 0)
|
||||
instructions OK (exit 0)
|
||||
release-complete OK (exit 0)
|
||||
observations OK (exit 0)
|
||||
|
||||
all agent gates OK
|
||||
rc=0
|
||||
@@ -0,0 +1,27 @@
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/cmd/hub 0.067s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/cmd/hubdb-check 0.074s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/api 5.720s
|
||||
? gitea.dooplex.hu/admin/felhom-hub/internal/assets [no test files]
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/claim 3.295s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/cloudflare 0.007s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/configgen 0.031s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/dbsnap 0.151s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/gitea 0.012s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/hetznerapi 0.054s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/i18n 0.005s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/intent 0.091s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/mailrelay 0.005s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/monitor 38.738s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/notify 1.684s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/offsite 0.820s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/offsiteheal 0.322s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys 0.255s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/osupdates 1.476s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/pbsdrheal 0.670s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/poke 0.012s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/semver 0.004s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/store 4.240s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/sysfacts 0.007s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/tenantsync 0.044s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/web 26.271s
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/wgsync 0.238s
|
||||
@@ -0,0 +1,17 @@
|
||||
R-528 red-proof red-go-kept: the copy of oom_check removed from internal/osupdate/unsent.go
|
||||
mutation: `rep.OOMCheck = rawOrNil(wr.OOMCheck)` -> `_ = rawOrNil`
|
||||
$ go test ./internal/osupdate/ -run ^TestR868_KeptCopyCarriesTheOOMCheck$ -count=1
|
||||
rc=1
|
||||
2026/10/06 19:13:38 INFO osupdate: sending a kept report late (the agent was stopped mid-pass or the hub was away, R-868) run=20261007T020000Z layer=docker vmid=9201 ring=0 trigger=night path=/tmp/TestR868_KeptCopyCarriesTheOOMCheck2908158053/001/report-20261007T020000Z-docker-apply.json
|
||||
2026/10/06 19:13:38 INFO osupdate: DONE run=20261007T020000Z layer=docker vmid=9201 ring=0 trigger=night outcome=applied healthy=true reason="sent late — kept on the box until the hub could take it (R-868, R-875)" upgraded=1 pending=0 not_covered=0 restart_needed=0 reboot_needed=false wrapper_seconds=0
|
||||
--- FAIL: TestR868_KeptCopyCarriesTheOOMCheck (0.00s)
|
||||
unsent_test.go:171: oom_check lost or changed: ""
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-agent/internal/osupdate 0.008s
|
||||
FAIL
|
||||
|
||||
=== control: restored source ===
|
||||
$ go test ./internal/osupdate/ -run ^TestR868_KeptCopyCarriesTheOOMCheck$ -count=1
|
||||
rc=0
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/osupdate 0.009s
|
||||
VERDICT: RED as expected, GREEN restored
|
||||
@@ -0,0 +1,26 @@
|
||||
R-528 red-proof red-go-runlayer: the copy of oom_check removed from internal/osupdate/leg.go
|
||||
mutation: `rep.OOMCheck = rawOrNil(wr.OOMCheck)` -> `_ = rawOrNil`
|
||||
$ go test ./internal/osupdate/ -run ^TestDocker_OOMCheckReachesTheHubUnchanged$ -count=1
|
||||
rc=1
|
||||
2026/10/06 19:13:37 INFO osupdate: START run=20261004T040000Z layer=guest vmid=9201 ring=0 trigger=night enabled=true release=ring0-20261004T040000Z
|
||||
2026/10/06 19:13:37 INFO osupdate: wrapper line="os-apply: DONE rc=0"
|
||||
2026/10/06 19:13:37 INFO osupdate: DONE run=20261004T040000Z layer=guest vmid=9201 ring=0 trigger=night outcome=nothing healthy=true reason="" upgraded=0 pending=0 not_covered=0 restart_needed=0 reboot_needed=false wrapper_seconds=0
|
||||
2026/10/06 19:13:37 INFO osupdate: START run=20261004T040000Z layer=host vmid=9201 ring=0 trigger=night enabled=true release=ring0-20261004T040000Z
|
||||
2026/10/06 19:13:37 INFO osupdate: wrapper line="os-apply: DONE rc=0"
|
||||
2026/10/06 19:13:37 INFO osupdate: DONE run=20261004T040000Z layer=host vmid=9201 ring=0 trigger=night outcome=nothing healthy=true reason="" upgraded=0 pending=0 not_covered=0 restart_needed=0 reboot_needed=false wrapper_seconds=0
|
||||
2026/10/06 19:13:37 INFO osupdate: wrapper line="os-apply: DONE rc=0"
|
||||
2026/10/06 19:13:37 INFO osupdate: live-restore vmid=9201 result=null
|
||||
2026/10/06 19:13:37 INFO osupdate: START run=20261004T040000Z layer=docker vmid=9201 ring=0 trigger=night enabled=true release=ring0-20261004T040000Z
|
||||
2026/10/06 19:13:37 INFO osupdate: wrapper line="os-apply: DONE rc=0"
|
||||
2026/10/06 19:13:37 INFO osupdate: DONE run=20261004T040000Z layer=docker vmid=9201 ring=0 trigger=night outcome=applied healthy=true reason="" upgraded=1 pending=0 not_covered=0 restart_needed=0 reboot_needed=false wrapper_seconds=0
|
||||
--- FAIL: TestDocker_OOMCheckReachesTheHubUnchanged (0.00s)
|
||||
leg_test.go:540: no oom_check in the docker report: {"run_id":"20261004T040000Z","layer":"docker","trigger":"night","mode":"apply","ring":0,"release_id":"ring0-20261004T040000Z","outcome":"applied","healthy":true,"vmid":9201,"upgraded":[{"name":"docker-ce","version":"5:29.8.2-1~debian.13~trixie","origin":""}],"docker_engine":"29.8.2","authority":"ring0"}
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-agent/internal/osupdate 0.010s
|
||||
FAIL
|
||||
|
||||
=== control: restored source ===
|
||||
$ go test ./internal/osupdate/ -run ^TestDocker_OOMCheckReachesTheHubUnchanged$ -count=1
|
||||
rc=0
|
||||
ok gitea.dooplex.hu/admin/felhom-agent/internal/osupdate 0.009s
|
||||
VERDICT: RED as expected, GREEN restored
|
||||
@@ -0,0 +1,34 @@
|
||||
R-528 red-proof red-health-mode: the check also runs in docker health mode
|
||||
mutation: replace
|
||||
---
|
||||
self.report["health"] = self.health()
|
||||
return 0
|
||||
---
|
||||
with
|
||||
---
|
||||
self.report["health"] = self.health()
|
||||
self.report["oom_check"] = self.oom_check()
|
||||
return 0
|
||||
---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-health-mode-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_never_on_guest_or_host_or_in_health_mode
|
||||
rc=1
|
||||
F
|
||||
======================================================================
|
||||
FAIL: test_never_on_guest_or_host_or_in_health_mode (__main__.OOMCheck.test_never_on_guest_or_host_or_in_health_mode)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1301, in test_never_on_guest_or_host_or_in_health_mode
|
||||
self.assertNotIn("oom_check", r)
|
||||
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
|
||||
AssertionError: 'oom_check' unexpectedly found in {'health': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'running'}, 'felhom-controller': {'health': 'healthy', 'id': 'aaa111', 'state': 'running'}}, 'controller': 'healthy', 'controller_docker_ok': True, 'docker_ok': True, 'network_ok': True}, 'layer': 'docker', 'mode': 'health', 'oom_check': {'detail': 'the engine reported the memory kill: OOMKilled=true and the oom event', 'exit_code': 137, 'image': 'gitea.dooplex.hu/admin/felhom-controller:0.300.0', 'oom_event': True, 'oom_killed': True, 'result': 'pass'}, 'pass_seconds': 0.0, 'refused': None, 'release_id': 'os-docker-t1', 'vmid': 9201}
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.016s
|
||||
|
||||
FAILED (failures=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_never_on_guest_or_host_or_in_health_mode
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,23 @@
|
||||
$ go test ./internal/osupdates/ -run TestDockerApproval -v -count=1 (oomCheckWaiting returns "")
|
||||
=== RUN TestDockerApproval_AFailedMemoryKillCheckBlocks
|
||||
[INFO] [store] app-update event types added to 0 household(s)' notification prefs (one-time, add-only): []
|
||||
[INFO] [store] app_stopped_unhealthy added to 0 household(s)' notification prefs (one-time, add-only): []
|
||||
docker_test.go:106: a set whose engine missed a memory kill was approvable: <nil>
|
||||
--- FAIL: TestDockerApproval_AFailedMemoryKillCheckBlocks (0.04s)
|
||||
=== RUN TestDockerApproval_AMissingMemoryKillCheckBlocks
|
||||
[INFO] [store] app-update event types added to 0 household(s)' notification prefs (one-time, add-only): []
|
||||
[INFO] [store] app_stopped_unhealthy added to 0 household(s)' notification prefs (one-time, add-only): []
|
||||
docker_test.go:121: a set with no memory-kill check was approvable: <nil>
|
||||
--- FAIL: TestDockerApproval_AMissingMemoryKillCheckBlocks (0.04s)
|
||||
=== RUN TestDockerApproval_AnErroredMemoryKillCheckBlocks
|
||||
[INFO] [store] app-update event types added to 0 household(s)' notification prefs (one-time, add-only): []
|
||||
[INFO] [store] app_stopped_unhealthy added to 0 household(s)' notification prefs (one-time, add-only): []
|
||||
docker_test.go:137: a set whose memory-kill check errored was approvable
|
||||
--- FAIL: TestDockerApproval_AnErroredMemoryKillCheckBlocks (0.04s)
|
||||
=== RUN TestDockerApproval_APassingCheckOnEveryBoxAllows
|
||||
[INFO] [store] app-update event types added to 0 household(s)' notification prefs (one-time, add-only): []
|
||||
[INFO] [store] app_stopped_unhealthy added to 0 household(s)' notification prefs (one-time, add-only): []
|
||||
--- PASS: TestDockerApproval_APassingCheckOnEveryBoxAllows (0.04s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/osupdates 0.160s
|
||||
FAIL
|
||||
@@ -0,0 +1,35 @@
|
||||
R-528 red-proof red-image-error: an unreadable image does not stop the check (the early 'error' return removed)
|
||||
mutation: replace
|
||||
---
|
||||
log(f"os-apply: OOM-CHECK error — {res['detail']}")
|
||||
return res
|
||||
---
|
||||
with
|
||||
---
|
||||
log(f"os-apply: OOM-CHECK error — {res['detail']}")
|
||||
---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-image-error-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_image_unreadable_is_error
|
||||
rc=1
|
||||
F
|
||||
======================================================================
|
||||
FAIL: test_image_unreadable_is_error (__main__.OOMCheck.test_image_unreadable_is_error)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1272, in test_image_unreadable_is_error
|
||||
self.assertEqual(oc["result"], "error", oc)
|
||||
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
AssertionError: 'pass' != 'error'
|
||||
- pass
|
||||
+ error
|
||||
: {'detail': 'the engine reported the memory kill: OOMKilled=true and the oom event', 'exit_code': 137, 'image': '', 'oom_event': True, 'oom_killed': True, 'result': 'pass'}
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.016s
|
||||
|
||||
FAILED (failures=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_image_unreadable_is_error
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,32 @@
|
||||
R-528 red-proof red-mode-only: mode oom-check does not stop after the check (falls through to apt)
|
||||
mutation: replace
|
||||
---
|
||||
self.report["oom_check"] = self.oom_check()
|
||||
return 0
|
||||
self.who---
|
||||
with
|
||||
---
|
||||
self.report["oom_check"] = self.oom_check()
|
||||
self.who---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-mode-only-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_mode_oom_check_runs_only_the_check
|
||||
rc=1
|
||||
F
|
||||
======================================================================
|
||||
FAIL: test_mode_oom_check_runs_only_the_check (__main__.OOMCheck.test_mode_oom_check_runs_only_the_check)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1311, in test_mode_oom_check_runs_only_the_check
|
||||
self.assertFalse(any("apt-get" in c[-1] or "dpkg-query" in c[-1] for c in f.calls), f.calls)
|
||||
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
AssertionError: True is not false : [('host', ['/usr/sbin/pct', 'status', '9201']), ('guest', 9201, ['docker', 'inspect', '-f', '{{.Config.Image}}', 'felhom-controller']), ('guest', 9201, ['date', '+%s']), ('guest', 9201, ['docker', 'run', '--name', 'felhom-oomcheck-3263952-df7cff79', '--pull', 'never', '--network', 'none', '--memory', '64m', '--memory-swap', '64m', '--label', 'felhom.oomcheck=1', '--entrypoint', 'sh', 'gitea.dooplex.hu/admin/felhom-controller:0.300.0', '-c', 'dd if=/dev/zero of=/dev/null bs=200M count=1']), ('guest', 9201, ['date', '+%s']), ('guest', 9201, ['docker', 'inspect', '-f', '{{.State.OOMKilled}} {{.State.ExitCode}}', 'felhom-oomcheck-3263952-df7cff79']), ('guest', 9201, ['docker', 'events', '--since', '1791115199', '--until', '1791115204', '--filter', 'container=felhom-oomcheck-3263952-df7cff79', '--filter', 'event=oom', '--format', '{{.Action}}']), ('guest', 9201, ['docker', 'rm', '-f', 'felhom-oomcheck-3263952-df7cff79']), ('guest', 9201, ['fuser', '/var/lib/dpkg/lock-frontend', '/var/lib/dpkg/lock']), ('guest', 9201, ['docker', 'ps', '-a', '--no-trunc', '--format', '{{.Names}}\t{{.State}}\t{{.Status}}\t{{.ID}}']), ('guest', 9201, ['getent', 'hosts', 'deb.debian.org']), ('guest', 9201, ['docker', 'exec', 'felhom-controller', 'docker', 'version', '--format', '{{.Server.Version}}']), ('guest', 9201, ['env', 'DEBIAN_FRONTEND=noninteractive', 'APT_LISTCHANGES_FRONTEND=none', 'NEEDRESTART_MODE=l', 'LC_ALL=C', 'apt-get', '-q', 'update']), ('guest', 9201, ['dpkg-query', '-W', '-f', '${Package}\t${Version}\t${db:Status-Abbrev}\n']), ('guest', 9201, ['apt-cache', 'policy', 'bash', 'containerd.io', 'docker-ce', 'libc6', 'openssl']), ('guest', 9201, ['env', 'DEBIAN_FRONTEND=noninteractive', 'APT_LISTCHANGES_FRONTEND=none', 'NEEDRESTART_MODE=l', 'LC_ALL=C', 'apt-get', '-s', '-q', 'dist-upgrade']), ('guest', 9201, ['sh', '-c', 'for p in /proc/[0-9]*; do grep -q "(deleted)" $p/maps 2>/dev/null || continue; grep -q "docker" $p/cgroup 2>/dev/null && continue; echo "${p#/proc/} $(cat $p/comm 2>/dev/null)"; done']), ('guest', 9201, ['docker', 'version', '--format', '{{.Server.Version}}']), ('guest', 9201, ['docker', 'ps', '-a', '--no-trunc', '--format', '{{.Names}}\t{{.State}}\t{{.Status}}\t{{.ID}}']), ('guest', 9201, ['getent', 'hosts', 'deb.debian.org']), ('guest', 9201, ['docker', 'exec', 'felhom-controller', 'docker', 'version', '--format', '{{.Server.Version}}'])]
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.002s
|
||||
|
||||
FAILED (failures=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_mode_oom_check_runs_only_the_check
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,32 @@
|
||||
R-528 red-proof red-no-event: pass decided on OOMKilled alone (the oom event not required)
|
||||
mutation: replace
|
||||
---
|
||||
if res["oom_killed"] and res["oom_event"]:---
|
||||
with
|
||||
---
|
||||
if res["oom_killed"]:---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-no-event-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_no_event_is_fail
|
||||
rc=1
|
||||
F
|
||||
======================================================================
|
||||
FAIL: test_no_event_is_fail (__main__.OOMCheck.test_no_event_is_fail)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1265, in test_no_event_is_fail
|
||||
self.assertEqual(oc["result"], "fail", oc)
|
||||
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
AssertionError: 'pass' != 'fail'
|
||||
- pass
|
||||
+ fail
|
||||
: {'detail': 'the engine reported the memory kill: OOMKilled=true and the oom event', 'exit_code': 137, 'image': 'gitea.dooplex.hu/admin/felhom-controller:0.300.0', 'oom_event': False, 'oom_killed': True, 'result': 'pass'}
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.016s
|
||||
|
||||
FAILED (failures=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_no_event_is_fail
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,33 @@
|
||||
R-528 red-proof red-oom-pass: the docker apply no longer runs the check (the key line in run() removed)
|
||||
mutation: replace
|
||||
---
|
||||
self.report["oom_check"] = self.oom_check()
|
||||
return 0
|
||||
---
|
||||
with
|
||||
---
|
||||
pass
|
||||
return 0
|
||||
---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-oom-pass-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_pass_when_oomkilled_and_the_event
|
||||
rc=1
|
||||
E
|
||||
======================================================================
|
||||
ERROR: test_pass_when_oomkilled_and_the_event (__main__.OOMCheck.test_pass_when_oomkilled_and_the_event)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1217, in test_pass_when_oomkilled_and_the_event
|
||||
oc = rep["oom_check"]
|
||||
~~~^^^^^^^^^^^^^
|
||||
KeyError: 'oom_check'
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.014s
|
||||
|
||||
FAILED (errors=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_pass_when_oomkilled_and_the_event
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,32 @@
|
||||
R-528 red-proof red-oomkilled-false: pass decided on the oom event alone (OOMKilled not required)
|
||||
mutation: replace
|
||||
---
|
||||
if res["oom_killed"] and res["oom_event"]:---
|
||||
with
|
||||
---
|
||||
if res["oom_event"]:---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-oomkilled-false-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_oomkilled_false_is_fail
|
||||
rc=1
|
||||
F
|
||||
======================================================================
|
||||
FAIL: test_oomkilled_false_is_fail (__main__.OOMCheck.test_oomkilled_false_is_fail)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1257, in test_oomkilled_false_is_fail
|
||||
self.assertEqual(oc["result"], "fail", oc)
|
||||
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
AssertionError: 'pass' != 'fail'
|
||||
- pass
|
||||
+ fail
|
||||
: {'detail': 'the engine reported the memory kill: OOMKilled=true and the oom event', 'exit_code': 0, 'image': 'gitea.dooplex.hu/admin/felhom-controller:0.300.0', 'oom_event': True, 'oom_killed': False, 'result': 'pass'}
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.014s
|
||||
|
||||
FAILED (failures=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_oomkilled_false_is_fail
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,31 @@
|
||||
R-528 red-proof red-other-layers: the check runs on every layer's apply (the docker-layer condition removed)
|
||||
mutation: replace
|
||||
---
|
||||
if self.layer == "docker" and self.mode == "apply":
|
||||
# R-528---
|
||||
with
|
||||
---
|
||||
if self.mode == "apply":
|
||||
# R-528---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-other-layers-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_never_on_guest_or_host_or_in_health_mode
|
||||
rc=1
|
||||
F
|
||||
======================================================================
|
||||
FAIL: test_never_on_guest_or_host_or_in_health_mode (__main__.OOMCheck.test_never_on_guest_or_host_or_in_health_mode)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1301, in test_never_on_guest_or_host_or_in_health_mode
|
||||
self.assertNotIn("oom_check", r)
|
||||
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
|
||||
AssertionError: 'oom_check' unexpectedly found in {'docker_restart_needed': False, 'health_after': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'running'}, 'felhom-controller': {'health': 'healthy', 'id': 'aaa111', 'state': 'running'}}, 'controller': 'healthy', 'controller_docker_ok': True, 'docker_ok': True, 'network_ok': True}, 'health_before': {'containers': {'app': {'health': 'healthy', 'id': 'bbb222', 'state': 'running'}, 'felhom-controller': {'health': 'healthy', 'id': 'aaa111', 'state': 'running'}}, 'controller': 'healthy', 'controller_docker_ok': True, 'docker_ok': True, 'network_ok': True}, 'installed': [{'name': 'bash', 'origin': 'Debian', 'version': '5.2.37-2+b9'}, {'name': 'libc6', 'origin': 'Debian', 'version': '2.41-12+deb13u4'}, {'name': 'openssl', 'origin': 'Debian', 'version': '3.5.7-1~deb13u3'}], 'layer': 'guest', 'mode': 'apply', 'oom_check': {'detail': 'the engine reported the memory kill: OOMKilled=true and the oom event', 'exit_code': 137, 'image': 'gitea.dooplex.hu/admin/felhom-controller:0.300.0', 'oom_event': True, 'oom_killed': True, 'result': 'pass'}, 'pass_seconds': 0.0, 'pending': [{'from': '5.2.37-2+b9', 'name': 'bash', 'origin': ['Debian'], 'to': '5.2.37-2+b10'}], 'plan': {'already': 0, 'from_snapshot': 0, 'not_installed': 0, 'upgrade': 2}, 'reboot_needed': False, 'reboot_scanned': True, 'refused': None, 'release_id': 'os-t1', 'repair': {'clean_after': True, 'fixed': 0, 'half_configured_before': 0, 'journal_before': 0}, 'restart_needed': [], 'seconds': 0.0, 'upgraded': [{'name': 'libc6', 'version': '2.41-12+deb13u4'}, {'name': 'openssl', 'version': '3.5.7-1~deb13u3'}], 'vmid': 9201}
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.020s
|
||||
|
||||
FAILED (failures=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_never_on_guest_or_host_or_in_health_mode
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,33 @@
|
||||
R-528 red-proof red-rm-finally: the removal is no longer in a finally (skipped when the check raised)
|
||||
mutation: replace
|
||||
---
|
||||
finally:
|
||||
try:
|
||||
mrc---
|
||||
with
|
||||
---
|
||||
if res["result"] != "error":
|
||||
try:
|
||||
mrc---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-rm-finally-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_container_removed_even_when_inspect_raises
|
||||
rc=1
|
||||
E
|
||||
======================================================================
|
||||
ERROR: test_container_removed_even_when_inspect_raises (__main__.OOMCheck.test_container_removed_even_when_inspect_raises)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1281, in test_container_removed_even_when_inspect_raises
|
||||
self.assertEqual(len(f.oom_removed), 1, "the container must be removed even when inspect raised")
|
||||
^^^^^^^^^^^^^
|
||||
AttributeError: 'Fake' object has no attribute 'oom_removed'
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.017s
|
||||
|
||||
FAILED (errors=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_container_removed_even_when_inspect_raises
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,29 @@
|
||||
R-528 red-proof red-settle-wait: no wait after the run before the end epoch is read (the measured miss, F1/F2)
|
||||
mutation: replace
|
||||
---
|
||||
self.r.sleep(OOMCHECK_SETTLE) # the engine---
|
||||
with
|
||||
---
|
||||
pass # self.r.sleep(OOMCHECK_SETTLE) # the engine---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-settle-wait-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_events_window_ends_after_the_settle_wait
|
||||
rc=1
|
||||
F
|
||||
======================================================================
|
||||
FAIL: test_events_window_ends_after_the_settle_wait (__main__.OOMCheck.test_events_window_ends_after_the_settle_wait)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1246, in test_events_window_ends_after_the_settle_wait
|
||||
self.assertTrue(waits and waits[0][1] >= 2, f"no settle wait after the run: {getattr(f, 'sleeps', None)}")
|
||||
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
AssertionError: [] is not true : no settle wait after the run: None
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.016s
|
||||
|
||||
FAILED (failures=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_events_window_ends_after_the_settle_wait
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,29 @@
|
||||
R-528 red-proof red-until-bound: the events window ends at the epoch before the run, not after the wait
|
||||
mutation: replace
|
||||
---
|
||||
"--until", str(t1 + 1),---
|
||||
with
|
||||
---
|
||||
"--until", str(t0 + 1),---
|
||||
$ OSAPPLY_UNDER_TEST=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/r528-mut/red-until-bound-felhom-os-apply python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_events_window_ends_after_the_settle_wait
|
||||
rc=1
|
||||
F
|
||||
======================================================================
|
||||
FAIL: test_events_window_ends_after_the_settle_wait (__main__.OOMCheck.test_events_window_ends_after_the_settle_wait)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py", line 1250, in test_events_window_ends_after_the_settle_wait
|
||||
self.assertEqual(until, f.oom_epochs[-1] + 1, "until = the guest epoch read after the wait, + 1")
|
||||
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
AssertionError: 1791115201 != 1791115204 : until = the guest epoch read after the wait, + 1
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.016s
|
||||
|
||||
FAILED (failures=1)
|
||||
|
||||
=== control: the same test against the real wrapper ===
|
||||
$ python3 -W ignore /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/da0e2df0-3072-4d3e-bf10-c475ac1e5ae4/scratchpad/agent-r528/configs/test_felhom_os_apply.py OOMCheck.test_events_window_ends_after_the_settle_wait
|
||||
rc=0
|
||||
OK
|
||||
VERDICT: RED as expected, GREEN on the real wrapper
|
||||
@@ -0,0 +1,63 @@
|
||||
$ python3 scripts/test_upgrade_bench.py (ssh_args without the -J jump)
|
||||
test_all_three_conditions_allow (__main__.BenchAdminSeedGuard.test_all_three_conditions_allow) ... ok
|
||||
test_each_condition_alone_refuses (__main__.BenchAdminSeedGuard.test_each_condition_alone_refuses) ... ok
|
||||
test_the_bench_seeds_through_the_admin_invite_and_never_shows_the_token (__main__.BenchAdminSeedGuard.test_the_bench_seeds_through_the_admin_invite_and_never_shows_the_token) ... /mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts/test_upgrade_bench.py:177: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/.felhom-h-gr4ipisf' mode='r' encoding='utf-8'>
|
||||
headers.append(open(extra[i + 1][1:]).read().strip())
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
ok
|
||||
test_the_bench_venue_names_itself (__main__.BenchAdminSeedGuard.test_the_bench_venue_names_itself) ... ok
|
||||
test_the_box_walk_never_signs_in_as_admin (__main__.BenchAdminSeedGuard.test_the_box_walk_never_signs_in_as_admin) ... ok
|
||||
test_a_household_shaped_box_is_never_seeded (__main__.BoxAdminSeedGuard.test_a_household_shaped_box_is_never_seeded) ... ok
|
||||
test_a_refused_invite_is_inconclusive (__main__.BoxAdminSeedGuard.test_a_refused_invite_is_inconclusive) ... ok
|
||||
test_all_conditions_allow (__main__.BoxAdminSeedGuard.test_all_conditions_allow) ... ok
|
||||
test_each_condition_alone_refuses (__main__.BoxAdminSeedGuard.test_each_condition_alone_refuses) ... ok
|
||||
test_only_a_drill_address_is_invited (__main__.BoxAdminSeedGuard.test_only_a_drill_address_is_invited) ... ok
|
||||
test_the_test_box_seeds_through_the_invite_inside_the_box (__main__.BoxAdminSeedGuard.test_the_test_box_seeds_through_the_invite_inside_the_box) ... ok
|
||||
test_the_tester_1_box_is_a_test_box_and_demo_hp_9201_is_not (__main__.BoxAdminSeedGuard.test_the_tester_1_box_is_a_test_box_and_demo_hp_9201_is_not) ... ok
|
||||
test_an_unknown_target_is_refused (__main__.BoxWalkTargets.test_an_unknown_target_is_refused) ... ok
|
||||
test_every_row_is_complete (__main__.BoxWalkTargets.test_every_row_is_complete) ... ok
|
||||
test_the_default_stays_scratch_9202 (__main__.BoxWalkTargets.test_the_default_stays_scratch_9202) ... ok
|
||||
test_the_old_guest_switch_still_selects_demo_hp_9201 (__main__.BoxWalkTargets.test_the_old_guest_switch_still_selects_demo_hp_9201) ... ok
|
||||
test_the_tester_1_box_is_reached_through_the_hp_box (__main__.BoxWalkTargets.test_the_tester_1_box_is_reached_through_the_hp_box) ... ERROR
|
||||
test_names_changed_added_removed_and_nothing_else (__main__.ChangedFiles.test_names_changed_added_removed_and_nothing_else) ... ok
|
||||
test_never_a_bare_root_never_outside (__main__.ClearScratch.test_never_a_bare_root_never_outside) ... /mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts/test_upgrade_bench.py:52: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/bench-scratch-b_aqb9ob/hdd/appdata/x/config/config.php' mode='w' encoding='utf-8'>
|
||||
open(p, "w").write("last run")
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
ok
|
||||
test_the_apps_own_folders_are_cleared_and_said (__main__.ClearScratch.test_the_apps_own_folders_are_cleared_and_said) ... /mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts/test_upgrade_bench.py:52: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/bench-scratch-msfdhygg/hdd/appdata/nextcloud/config/config.php' mode='w' encoding='utf-8'>
|
||||
open(p, "w").write("last run")
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts/test_upgrade_bench.py:52: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/bench-scratch-msfdhygg/hdd/appdata/immich/config/config.php' mode='w' encoding='utf-8'>
|
||||
open(p, "w").write("last run")
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
ok
|
||||
test_nothing_new_is_refused (__main__.DefinitionEdge.test_nothing_new_is_refused) ... ok
|
||||
test_the_edge_names_the_new_service (__main__.DefinitionEdge.test_the_edge_names_the_new_service) ... ok
|
||||
test_the_to_render_reads_the_new_definition (__main__.DefinitionEdge.test_the_to_render_reads_the_new_definition) ... ok
|
||||
test_answers_of_any_code_are_the_app_answering (__main__.LoadVerdict.test_answers_of_any_code_are_the_app_answering) ... ok
|
||||
test_every_request_errored_is_inconclusive (__main__.LoadVerdict.test_every_request_errored_is_inconclusive) ... ok
|
||||
test_under_half_is_inconclusive_and_none_is_inconclusive (__main__.LoadVerdict.test_under_half_is_inconclusive_and_none_is_inconclusive) ... ok
|
||||
test_a_listed_name_that_grew_or_vanished_still_marks (__main__.MarkerIgnore.test_a_listed_name_that_grew_or_vanished_still_marks) ... ok
|
||||
test_a_moved_tree_the_file_walk_cannot_name_keeps_the_mark (__main__.MarkerIgnore.test_a_moved_tree_the_file_walk_cannot_name_keeps_the_mark) ... ok
|
||||
test_an_unlisted_changed_file_still_marks (__main__.MarkerIgnore.test_an_unlisted_changed_file_still_marks) ... ok
|
||||
test_every_entry_has_a_reason (__main__.MarkerIgnore.test_every_entry_has_a_reason) ... ok
|
||||
test_the_list_belongs_to_its_app (__main__.MarkerIgnore.test_the_list_belongs_to_its_app) ... ok
|
||||
test_the_six_measured_markers_do_not_mark_and_each_carries_a_reason (__main__.MarkerIgnore.test_the_six_measured_markers_do_not_mark_and_each_carries_a_reason) ... ok
|
||||
test_env_is_0600_and_shredded (__main__.SecretHygiene.test_env_is_0600_and_shredded) ... ok
|
||||
test_evidence_files_are_redacted (__main__.SecretHygiene.test_evidence_files_are_redacted) ... ok
|
||||
test_redact_longest_first (__main__.SecretHygiene.test_redact_longest_first) ... ok
|
||||
test_secret_values_are_the_generated_fields_and_the_seed_password (__main__.SecretHygiene.test_secret_values_are_the_generated_fields_and_the_seed_password) ... ok
|
||||
|
||||
======================================================================
|
||||
ERROR: test_the_tester_1_box_is_reached_through_the_hp_box (__main__.BoxWalkTargets.test_the_tester_1_box_is_reached_through_the_hp_box)
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts/test_upgrade_bench.py", line 432, in test_the_tester_1_box_is_reached_through_the_hp_box
|
||||
self.assertEqual(a[a.index("-J") + 1], "demo-hp")
|
||||
~~~~~~~^^^^^^
|
||||
ValueError: '-J' is not in list
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 36 tests in 0.043s
|
||||
|
||||
FAILED (errors=1)
|
||||
@@ -0,0 +1,9 @@
|
||||
wger's ladder step (catalog cf1ed43) reads memory_cgroup_peak_pct 100.0. From the bench's own verdict
|
||||
(audits/design-build-2026-10-06/F/bench/evidence/MV-wger/verdict.json), read 2026-10-06:
|
||||
wger: limit 402653184, cgroup peak 402653184 (1.0), anon peak 202338304 (0.503), oom_kills 0 (the kernel's
|
||||
memory.events counter), restarts 0, OOMKilled flag false
|
||||
the memory-watch log: no sample with kills>0 or rs>0
|
||||
=> FILE CACHE, not a kill: the entrypoint's collectstatic writes ~283 MB of static files at every start
|
||||
(F/bench-static-check.txt: "static volume: 283M; files: 22725"); page cache counts in the cgroup peak
|
||||
(memory note harness-memory-peak-includes-page-cache). The step's mark is judged on anon (50.3 %, decision 22).
|
||||
wger's memory hint (416M) stays.
|
||||
Reference in New Issue
Block a user