diff --git a/REPORT.md b/REPORT.md index 54356414..2c35e791 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,46 +1,24 @@ -# REPORT — test-box check after a week away, and the old website copy retired (2026-11-07, attended) +# REPORT — morning check, and hub 0.179.1 (2026-11-08, attended) -**Plain words first.** All three test boxes (N100, HP, Tester 1) run, report to the hub, and all apps are healthy. -Two faults were found. demo-hp made no local whole-box copy for 9 days because its system disk was too full; CC freed -space with the operator's yes. The N100 refused its nightly Proxmox updates three nights in a row; agent 0.172.1 fixes -it and waits for the operator's vouch and signed jobs (commands in `felhom-agent/REPORT.md`). +**Plain words first.** Last night's fixes worked. The N100 installed its Proxmox updates (no refusal) and a new kernel, +and restarted itself. demo-hp made its first local whole-box copy in 10 days. The hub still said the N100 „needs a +restart" after it had restarted; hub 0.179.1 fixes that. demo-hp does need a restart; the operator chose to wait for its +kernel step. -**Method.** Two channels: the hub's database copied from the running pod WITH its WAL (events since 2026-10-31, -`hosts`, `reports`, OS releases), and SSH to each box (agent journal, `pct list`, `docker ps` in guest 9201, `df`, -`apt-get -s`). The two agree. The DB copy was deleted from the scratchpad after reading. +**Box facts (SSH).** demo-felhom: pve step `DONE upgraded=33` (05:32), kernel 7.0.14-28 one-shot boot 05:35, `KERNEL GOOD` +(default now 7.0.14-28), 0 host processes on replaced libraries, guest 9201 running. demo-hp: local backup `completed` +02:43 (`local:backup/vzdump-lxc-9201-2026_11_08-02_30_38`), pve step `upgraded=35`, no kernel step yet (7.0.14-28 pending), +systemd/sshd/kvm/zed/rrdcached on replaced libraries. -**What the week shows.** -- Every night: DB dumps and drive copies done on all three. Both hosts rebooted 2026-10-29 ~05:00 for the kernel step. -- 2026-11-01 14:19: all three hosts stale together for ~1 h, then recovered — same minute on all three, so a line or - hub-side gap, not a box fault. -- Tester4 (2026-11-03): CC's proof VM, its host already deleted from the hub — the STATUS do-item is removed. -- Tester 1 (2026-11-02): radicale stopped as unhealthy twice, and a disk warning at 89 %; it is at 15 % now. -- All three: „Root files behind" (bundle 0.170.0). The 0.172.1 bundle job clears it. +**Hub 0.179.1** (`osupdates.layerLine`): the host's last boot (crash guard `last_boot_at`) cuts the host layer's +reboot-needed run, less one minute of clock skew; a claim after the boot starts a new run. Feeds the System page and +the 14-day alarm. Test `TestRebootNeeded_ClearedByAHostBootAfterTheClaim`, red on two mutants; `go test ./...` green. +Deployed: image built from `ddbcabeb`, manifest `6b91a215`, ArgoCD sync of `Deployment/hub` only, rollout ok, pod +logs `felhom-hub 0.179.1 starting`. **Live (endpoint method — `GET /system` with CC's token, the page the operator +opens):** only demo-hp is listed. Evidence: `documentation/audits/hub-reboot-flag-2026-11-08/`. -**Done.** -- demo-hp: whole journal saved off-git (`/mnt/5_hdd/felhom.eu/evidence/demo-hp-space-2026-11-07/`), then - `journalctl --vacuum-size=500M` (freed 3.3 GB) and `pvesm free` of the 2026-08-21 archive of guest 9100: `/` 9.5 → 14 GB - free; the preflight asks 12.4 GiB. Recorded under R-548. The journal will grow back (journald default cap). -- felhom-agent v0.172.1 released; the operator vouched it and sent both signed jobs; all three boxes run it (bundle - self-check ok, capability probe 77/77). See that repo's REPORT. -- **R-1228 closed (operator yes):** DooPlex's copy of the website retired. The installer pin moved first to - `INSTALLER-TAG` (publish script + `hostinstall_gates.py` 6c, red on five mutants; dry run = the public installer's - sha256). Then a deliberate ArgoCD sync pruned exactly six objects (webpage Deployment, Service, Ingress, two - ConfigMaps, and the orphan `felhom.eu/api` Ingress — a `number: 80---` typo had glued it to the next YAML document); - live objects saved first (`/mnt/5_hdd/felhom.eu/evidence/r1228-retire-2026-11-07/`); the unused `felhom-webpage-tls` - Secret deleted. After: felhom.eu 200, `api.felhom.eu` preflight 204. -- **Security review follow-up:** the old nginx sent `X-Frame-Options: SAMEORIGIN` and `/scripts/` as `text/plain`; - Cloudflare did neither since 2026-10-30. `website/_headers` restores both; uploaded; measured from outside (both - headers present, installer sha256 unchanged `e18cdd17…`, `/_headers` itself 404). -- N100 diary entry; STATUS updated (new do-list, the Tester4 item removed). +**Remote restart buttons (the operator's question).** The host page has „Restart controller" and „Reboot the guest" +per guest (agent actions, D65). There is no button that reboots the Proxmox host; the host reboots only by the kernel +step. Not a defect — the agent's action list is closed by design. -**Not yet observed.** demo-hp's next local-tier try (every ~4 h) has not run since the cleanup. Positive observable: -`backup: completed … target=local` in its agent journal. - -**Registers.** R-1228 closed (to CLOSED-ITEMS); none opened (the GRUB fault is fixed in the session, size rule). Evidence: -`documentation/audits/demo-check-2026-11-07/`. System poster: the facts file updated (the old copy retired); the drawing refresh was already listed in STATUS. - -**Instruction files edited** (stale facts, no rule loosened): `CLAUDE.md` R-110 fence (where the tag is pinned, what -gate 6 checks), `.claude/rules/manifests.md` (section replaced: no installer in a manifest), `.claude/rules/website.md` -(the way back retired), the `felhom-build-deploy` skill (same). Why: the pin moved from `manifests/webpage.yaml` to -`INSTALLER-TAG`. +**Registers.** No row opened or closed (the hub fault fixed in the session, size rule). System poster: no fact changed. diff --git a/STATUS.md b/STATUS.md index 8b1c56b6..a30c0765 100644 --- a/STATUS.md +++ b/STATUS.md @@ -2,7 +2,7 @@ **Ready for Tester 2? Yes — tell the friend go (`documentation/pilot/TESTER2-handover.md`).** -**Updated 2026-11-07 (late).** The three test boxes run agent 0.172.1 (you vouched and signed it): the N100's nightly Proxmox updates are no longer refused, and demo-hp has space for its local whole-box copy again. DooPlex's old copy of the website is retired; the installer's version pin now lives in `INSTALLER-TAG`, and two browser headers are back on the public site. +**Updated 2026-11-08.** The night went right: the N100 installed its Proxmox updates and a new kernel and restarted itself; demo-hp made its first local whole-box copy in 10 days. Hub 0.179.1 no longer says „needs a restart" for a box that has restarted; demo-hp still needs one (its kernel step, likely tonight). ## What works - **The Plex attack did not reach Felhom's secrets** (R-1241). diff --git a/documentation/audits/hub-reboot-flag-2026-11-08/README.md b/documentation/audits/hub-reboot-flag-2026-11-08/README.md new file mode 100644 index 00000000..ec858dd3 --- /dev/null +++ b/documentation/audits/hub-reboot-flag-2026-11-08/README.md @@ -0,0 +1,7 @@ +# Hub 0.179.1 — a host boot ends „the host needs a restart" (2026-11-08) + +- Before (operator's screenshot, morning): demo-felhom and demo-hp both „the host needs a restart since 2026-11-08". +- Box facts (SSH, a different channel): demo-felhom booted 2026-11-08 05:35:18 CET (kernel step), 0 host processes map a + replaced library; demo-hp up since 2026-10-29, systemd/sshd/kvm/zed/rrdcached still map replaced libraries. +- Hub facts (DB copy with WAL): last_boot_at demo-felhom 2026-11-08T04:35:18Z, demo-hp 2026-10-29T04:10:08Z. +- After the deploy: `system-page-after-0.179.1.txt` — only demo-hp is listed. diff --git a/documentation/audits/hub-reboot-flag-2026-11-08/system-page-after-0.179.1.txt b/documentation/audits/hub-reboot-flag-2026-11-08/system-page-after-0.179.1.txt new file mode 100644 index 00000000..1e594905 --- /dev/null +++ b/documentation/audits/hub-reboot-flag-2026-11-08/system-page-after-0.179.1.txt @@ -0,0 +1,2 @@ +GET /system (hub 0.179.1, CC token, 2026-11-08 ~07:31 CET) — the Needs attention block: +Needs attention One line per box that is amber or red, and why. Amber: worth a look. Red: an operator alarm fires (`08` §6.3). look demo-hp-bb76ea Demo HP the host needs a restart since 2026-11-08 Waiting for you Update sets that only you approve: the kernel, diff --git a/documentation/backlog/STATUS-HISTORY.md b/documentation/backlog/STATUS-HISTORY.md index 67836561..cebe8eb2 100644 --- a/documentation/backlog/STATUS-HISTORY.md +++ b/documentation/backlog/STATUS-HISTORY.md @@ -5,6 +5,10 @@ Moved here byte for byte on 2026-10-14 (night session), newest first, so that `S **Updated 2026-10-13 (night): demo-hp, demo-felhom and Tester 1 run agent 0.155.0; demo-felhom runs controller 0.310.0, the other two 0.309.0 until the floor is raised. The open-items list is at 127. Reports: `REPORT-poweroff.md`, `REPORT-plugmeter.md`, `REPORT-usenet-2026-10-12.md`, `REPORT-newapps-2026-10-12.md`, `REPORT-disk-evening-2026-10-12.md`, `REPORT-answers-2026-10-12.md`, `REPORT-m3-2026-10-11.md`, `REPORT-redesign-2026-10-10.md`, `REPORT-testers-2026-10-10.md`, `REPORT-register-shrink-2026-10-10.md`, `REPORT-new-apps-2026-10-10.md`, `REPORT-release-2026-10-10.md`, `REPORT-break-the-circle-2026-10-09.md`, `REPORT-dooplex-survival-2026-10-09.md`, `REPORT-day4-2026-10-09.md`.** +## Moved from STATUS on 2026-11-08 + +**Updated 2026-11-07 (late).** The three test boxes run agent 0.172.1 (you vouched and signed it): the N100's nightly Proxmox updates are no longer refused, and demo-hp has space for its local whole-box copy again. DooPlex's old copy of the website is retired; the installer's version pin now lives in `INSTALLER-TAG`, and two browser headers are back on the public site. + ## Moved from STATUS on 2026-11-07 **Updated 2026-11-07.** CC checked the three test boxes: all run and all apps are healthy. demo-hp had no local whole-box copy for 9 days (its disk was full); CC freed space. The N100 refused its nightly Proxmox updates (a GRUB package in the wrong group); agent 0.172.1 fixes it and waits for your vouch and signed jobs.