Files
felhom.eu/REPORT.md
T
adminandClaude Opus 5.5 ef8e23f42a
gates / gates (push) Successful in 2m5s
Health check 2026-11-14: R-528 dated check read (one failed oom_check, date moved to 11-28), R-1391 + R-1392 filed
- R-528: NOT closed. demo-felhom's night oom_check failed once (2026-11-11 04:40 UTC, Docker 29.9.3,
  containerd 2.4.2-3); 56 others passed; all four app_oom events since 10-12 explained. Due date 2026-11-28.
  This clears the due-checks gate that kept nightly CI red (runs 2628, 2630, 2632).
- R-1391: Go 1.27.3 (hub, controller, agent) has GO-2026-6859; fixed in 1.27.4.
- R-1392: the hub's "root files behind" compares the whole bundle sha, which moves with every agent release
  even when configs/ did not change; the os_config_bundle_behind alarm is due ~11-17 on all three boxes.
- STATUS: D81 + the 11-14 do line; old Updated paragraph moved to STATUS-HISTORY. REPORT overwritten.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-11-14 09:11:16 +01:00

43 lines
4.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# REPORT — health check after a few unattended days (2026-11-14, morning, attended)
**Plain words first.** The three test boxes are healthy. Backups ran last night on all three, off-site too, and each
restore proof passed. The night CI mail was red for three nights for one reason: a dated check (R-528) was due. The
check is now read and written down, and CI is green for it. Two new rows are filed. Nothing was changed on any box.
**Method.** Read only. The hub's own pages with CC's token (Dashboard, Hosts, System, Offsite); one copy of the hub
database **with its WAL** (newest host report 7 minutes old, `quick_check` ok; the copy was shredded after the read);
the hub log's deadline lines; the operator mailbox (label Monitoring, last 5 days); the CI run log of run id 2632
(job 2636) through the Gitea API.
| | |
|---|---|
| Boxes | demo-felhom, demo-hp, Tester 1: controller 0.352.0, agent 0.173.0, ONLINE, all containers running, crash guard armed, 0 crash restarts |
| Backups last night | DB dumps 03:15–03:17 UTC; off-site `ok` 03:15–03:18 UTC; off-site restore proof `pass` 04:30 UTC (bentopdf, opengist, bookstack); whole-guest backups `success`; host restore tests `pass` (demo-hp from PBS 2026-11-14 02:43 UTC) |
| CI red ×3 nights (runs 2628, 2630, 2632) | only gate `due-checks`: R-528 was due 2026-11-12. Read and recorded (below), date moved to 2026-11-28 with the reason in the row |
| R-528 reading | NOT closed. One failed `oom_check` — demo-felhom 2026-11-11 04:40:30 UTC, Docker 29.9.3 / containerd 2.4.2-3: exit 137, `OOMKilled=false`, no `oom` event. 56 others passed. The four `app_oom` events since 2026-10-12 are all explained (a freeze, a squeeze test, a release proof) |
| Tester4 | Host deleted by the operator 07:54:31 UTC. The pairing-link mail at the same second is by design (a host delete that keeps the customer). The 05:00 `expected_dbdump_missed` came before the delete; the host's reports are gone with it, so the deadline check reads „not judged yet" from tomorrow (as Tester 2 does) |
| demo-hp blips | `agent_channel_unknown` 2026-11-11 13:52 → recovered 14:00 UTC (the certificate-pin spike on demo-hp that afternoon); `health_degraded` 2026-11-12 03:17 → recovered 03:22 UTC (a controller restart in the night step) |
| Security watch | New: 4 FileBrowser notices (one high, arbitrary file write) fixed only on 2.1.0 — belongs to R-1305; Go 1.27.3 GO-2026-6859 had no row → R-1391; cloudflared is R-1358 |
| Register | 248 rows at the start, 250 at the end. Opened R-1391 (Go 1.27.4), R-1392 (root files amber after every agent release). Closed 0. Dated line on R-528 |
## Findings
1. **R-1392 — the yellow „root files behind the vouched agent's" on all three boxes is a false amber, and it becomes
three operator alarms around 2026-11-17.** The files on the boxes are the vouched ones (`configs/` did not change
between 0.172.1 and 0.173.0), but the hub compares the whole bundle's sha256, which carries the version.
2. **R-1391 — Go 1.27.3 in hub, controller and agent; fixed in 1.27.4.** Reach not measured.
3. **CI turns red again after 2026-11-15:** R-1050 (HSTS one year) is due that day and needs a controller release.
## Waiting on the operator
- Rotate `GITEA_GIT_TOKEN` (the secret sweep mail of 2026-11-12 found it in 2 session records), then `sec put`.
- Approve the Proxmox set 9.2.26 and kernel 7.0.14-28 on the System page when you want Tester 1 to follow.
- R-1392: A or B (in the row). Sending the 0.173.0 bundle to the three boxes clears today's amber either way.
**Fixed without a row:** the local `instructions` gate (and its decoy test) failed because the memory index was 201
lines / 25 725 bytes (ceilings 200 / 25 600). Two index lines were merged; now 199 / 25 571. Not a repo file.
**Not done:** no box touched, no signed job sent, no hub change, no release. No area CHANGELOG entry: the change is
register-only (`hub/`, `scripts/`, `website/`, `marketing/` untouched). Provisioned nothing; teardown: the hub DB copy
in the scratchpad was shredded.