- R-528: NOT closed. demo-felhom's night oom_check failed once (2026-11-11 04:40 UTC, Docker 29.9.3, containerd 2.4.2-3); 56 others passed; all four app_oom events since 10-12 explained. Due date 2026-11-28. This clears the due-checks gate that kept nightly CI red (runs 2628, 2630, 2632). - R-1391: Go 1.27.3 (hub, controller, agent) has GO-2026-6859; fixed in 1.27.4. - R-1392: the hub's "root files behind" compares the whole bundle sha, which moves with every agent release even when configs/ did not change; the os_config_bundle_behind alarm is due ~11-17 on all three boxes. - STATUS: D81 + the 11-14 do line; old Updated paragraph moved to STATUS-HISTORY. REPORT overwritten. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
4.0 KiB
REPORT — health check after a few unattended days (2026-11-14, morning, attended)
Plain words first. The three test boxes are healthy. Backups ran last night on all three, off-site too, and each restore proof passed. The night CI mail was red for three nights for one reason: a dated check (R-528) was due. The check is now read and written down, and CI is green for it. Two new rows are filed. Nothing was changed on any box.
Method. Read only. The hub's own pages with CC's token (Dashboard, Hosts, System, Offsite); one copy of the hub
database with its WAL (newest host report 7 minutes old, quick_check ok; the copy was shredded after the read);
the hub log's deadline lines; the operator mailbox (label Monitoring, last 5 days); the CI run log of run id 2632
(job 2636) through the Gitea API.
| Boxes | demo-felhom, demo-hp, Tester 1: controller 0.352.0, agent 0.173.0, ONLINE, all containers running, crash guard armed, 0 crash restarts |
| Backups last night | DB dumps 03:15–03:17 UTC; off-site ok 03:15–03:18 UTC; off-site restore proof pass 04:30 UTC (bentopdf, opengist, bookstack); whole-guest backups success; host restore tests pass (demo-hp from PBS 2026-11-14 02:43 UTC) |
| CI red ×3 nights (runs 2628, 2630, 2632) | only gate due-checks: R-528 was due 2026-11-12. Read and recorded (below), date moved to 2026-11-28 with the reason in the row |
| R-528 reading | NOT closed. One failed oom_check — demo-felhom 2026-11-11 04:40:30 UTC, Docker 29.9.3 / containerd 2.4.2-3: exit 137, OOMKilled=false, no oom event. 56 others passed. The four app_oom events since 2026-10-12 are all explained (a freeze, a squeeze test, a release proof) |
| Tester4 | Host deleted by the operator 07:54:31 UTC. The pairing-link mail at the same second is by design (a host delete that keeps the customer). The 05:00 expected_dbdump_missed came before the delete; the host's reports are gone with it, so the deadline check reads „not judged yet" from tomorrow (as Tester 2 does) |
| demo-hp blips | agent_channel_unknown 2026-11-11 13:52 → recovered 14:00 UTC (the certificate-pin spike on demo-hp that afternoon); health_degraded 2026-11-12 03:17 → recovered 03:22 UTC (a controller restart in the night step) |
| Security watch | New: 4 FileBrowser notices (one high, arbitrary file write) fixed only on 2.1.0 — belongs to R-1305; Go 1.27.3 GO-2026-6859 had no row → R-1391; cloudflared is R-1358 |
| Register | 248 rows at the start, 250 at the end. Opened R-1391 (Go 1.27.4), R-1392 (root files amber after every agent release). Closed 0. Dated line on R-528 |
Findings
- R-1392 — the yellow „root files behind the vouched agent's" on all three boxes is a false amber, and it becomes
three operator alarms around 2026-11-17. The files on the boxes are the vouched ones (
configs/did not change between 0.172.1 and 0.173.0), but the hub compares the whole bundle's sha256, which carries the version. - R-1391 — Go 1.27.3 in hub, controller and agent; fixed in 1.27.4. Reach not measured.
- CI turns red again after 2026-11-15: R-1050 (HSTS one year) is due that day and needs a controller release.
Waiting on the operator
- Rotate
GITEA_GIT_TOKEN(the secret sweep mail of 2026-11-12 found it in 2 session records), thensec put. - Approve the Proxmox set 9.2.26 and kernel 7.0.14-28 on the System page when you want Tester 1 to follow.
- R-1392: A or B (in the row). Sending the 0.173.0 bundle to the three boxes clears today's amber either way.
Fixed without a row: the local instructions gate (and its decoy test) failed because the memory index was 201
lines / 25 725 bytes (ceilings 200 / 25 600). Two index lines were merged; now 199 / 25 571. Not a repo file.
Not done: no box touched, no signed job sent, no hub change, no release. No area CHANGELOG entry: the change is
register-only (hub/, scripts/, website/, marketing/ untouched). Provisioned nothing; teardown: the hub DB copy
in the scratchpad was shredded.