diff --git a/REPORT-hub-system-page.md b/REPORT-hub-system-page.md new file mode 100644 index 00000000..05c05802 --- /dev/null +++ b/REPORT-hub-system-page.md @@ -0,0 +1,89 @@ +# REPORT — the hub's System page, made readable (2026-10-10) + +**Before:** 3 cards plus one box table with 30 columns that scrolled sideways. Release ids, cancelled TEST approvals and +package counts came first. The red „Approve now (guest + host)" sat in the middle of the page. +**After:** 4 parts, in this order: +1. Needs attention. +2. Waiting for you. +3. Boxes: one 7-column table. Each row opens in place. +4. Details: closed by default. + +No sideways scroll at 1280 px or at 390 px (measured: `scrollWidth == clientWidth` at both widths). + +Built and tested only. **No deploy, no release, no box touched.** It ships with the next hub release. + +## Baseline + +- `felhom.eu` `main` @ `ec5605d4` after `git pull --rebase`. The last hub release in `hub/CHANGELOG.md` is v0.145.0. +- Architecture read: `05-hub-architecture.md` §5 (floors, vouched agent), `11-os-updates.md` §5.7, §5.8 and §5.11 + (System page, lanes, kernel approval), `08-alarm-ladder.md` §6.3 (the colours). + +## What changed + +- `hub/internal/web/system_view.go` (new) is the view model: + - `buildSystemPage` takes all the inputs and is pure, so the fixture can feed it. + - `summariseRow` makes the box mark and the plain-words reasons, from the same cell colours as before. + - `buildWaiting` makes the cards, with the same gate the buttons always had: a set exists, it is not approved, and + nothing is waiting. +- `hub/internal/web/system.go`: the handler fills `systemInput`. It now also reads the controller version per box + (`GetCustomers`). It reads the candidate package list and the healthy-night count, both only to describe a card. +- `hub/internal/web/templates/system.html`: the new layout and its page CSS. It uses the hub's colour tokens. The nav + wraps on a phone (this page's CSS only). +- Docs: `05` (a short paragraph) and `11` §5.7 (the new layout). `hub/CHANGELOG.md`: an „Unreleased" entry. + +## Where each old item went (the Baselines contract) + +| Old item | Now | +|---|---| +| Flash / error line | unchanged, at the top | +| Per box: the 30 columns, the ring button, the updates switch | Boxes: the row opens to all of them. Ring, health, controller + agent, OS updates, kernel and last night are also in the row | +| Approved releases (ids, packages, by, TEST) | **Details** | +| „Approve now (guest + host)" | **Details**, beside the releases. It now asks: „Approve the guest and host sets now, without the usual 24 h and one night?" | +| Cancelled approvals (last 7 days) | **Details** | +| What ring 0 runs now (counts, first seen, the wait reason) | **Details**. A set still being tested is also listed under Waiting for you, with the hub's reason | +| Approve Docker / Proxmox / kernel set | Waiting for you: one card each, when the button is allowed | +| Version floors, global floor, vouched agent | **Details** (the `#version-floors` link opens it) | +| kernel.panic, oops, crash restarts, crash guard, root files | in each box's row, and also as a table in **Details** | +| Legend („unknown" = could not read, never a guess) | under the Boxes title | + +Same routes and the same `_csrf` field on every form. Clicking to open works without JavaScript (`
`). + +## Tests + +- `go build ./... && go vet ./... && go test ./...` in `hub/`: see the push section. +- New file `system_layout_test.go`. Each test below was **red-proofed: I broke the code on purpose and saw the test fail**. + - Every old item is still on the page. Red: I dropped the containerd line → `Boxes lacks ">containerd"`. + - Every form carries `_csrf` and posts only to the old routes. Red: I removed `_csrf` from a card → + `form /os/approve-pve lacks the CSRF or return field`. + - Needs attention: a green box is not listed; amber and red boxes are, with reasons; all green gives one line. Red: I + skipped the cells → `the amber box (agent behind) is missing`. + - Waiting for you: a card per lane, the not-ready line, nothing once approved, guest/host never a card, and the empty + line. Red: I dropped the wait gate → `kernel not ready: cards [...]`. + - Approve now is inside Details and asks first. +- Changed old assertion: `system_trim_test.go` now looks for `>Last disk trim` (it was ``). The label moved from + a table header into the box panel. + +## Screenshots (made-up fixture shaped like today: 4 boxes, kernel approved, Docker still testing, Tester 2 „unknown") + +All are in `documentation/audits/hub-system-page-2026-10-10/`. They were taken with a headless Chromium in the +scratchpad, from `system-fixture*.html`. Those pages are written by `SYSTEM_PAGE_FIXTURE_OUT= go test +./internal/web/ -run TestSystemPage_WriteFixture`. + +- `01-1280.png` — the page at 1280 px. +- `02-390.png` — the page at 390 px (phone). +- `03-1280-details-open.png` — Details open. +- `04-1280-box-open.png` — one box (Tester 1) open. +- `05-1280-waiting-card.png` — a Proxmox set ready, as a card with its button. + +## Not done / notes + +- Guest and host sets get no card: they approve themselves. Their urgent override („Approve now") is in Details. +- The flash after an approval still says `approved `. That text comes from the `/os/` route, not from this + page, so I left it. +- Teardown: provisioned nothing. No machine, no host, no hub change. + +## Questions for the operator + +1. Look at the five screenshots. Is the order right (problems first, then approvals, then boxes)? OK, or what to change? +2. Tester 2 („agent older than v0.142.0") shows red for „agent behind" plus three amber reasons. Do you want an old + agent as one line only, or is the full list useful? diff --git a/documentation/architecture/05-hub-architecture.md b/documentation/architecture/05-hub-architecture.md index 635fdefe..ae579df6 100644 --- a/documentation/architecture/05-hub-architecture.md +++ b/documentation/architecture/05-hub-architecture.md @@ -152,6 +152,11 @@ box's agent against the vouched one ("0.142.0 → 0.145.0 (since …)", amber, r vouched agent for 7 days raises `agent_behind` (warning, operator-only; `OS_ALARM_AGENT_BEHIND_AFTER`; the clock starts when the hub first sees the box behind). `[DESIGN — CC 2026-10-05, operator may reverse; `09` decision 119]` +**The System page's layout (2026-10-10).** It answers „is anything wrong?" (Needs attention) and „is anything waiting +for me?" (Waiting for you) first, then the boxes in one narrow table that opens per box, and folds release ids, +cancelled approvals, ring-0 package counts, the floors, the crash guard and root files into a closed „Details". The +page's content list is `11` §5.7; the contract that nothing was dropped is `web/system_layout_test.go`. + ## 6. Authorization — signed-op queue + editing flow Implements Part 4's gate on the hub side. The hub holds **no signing key**. diff --git a/documentation/architecture/11-os-updates.md b/documentation/architecture/11-os-updates.md index 31de7484..a02f18a8 100644 --- a/documentation/architecture/11-os-updates.md +++ b/documentation/architecture/11-os-updates.md @@ -397,6 +397,11 @@ must never overlap a backup, a restore-test or a self-update.~~ reboot needed / `kernel.panic` / oops / crash restarts / the guard, guest Debian / release / pending / restart needed, Docker engine / containerd / live-restore / release, the last leg — and above it the releases, what ring 0 runs, "Approve now" and "Approve Docker set". Colours are the alarm thresholds (decision 94). Hosts shows Proxmox / kernel too. + **Layout from 2026-10-10 (hub, unreleased at writing):** the page opens with „Needs attention" (one line per amber or + red box, the reasons in plain words, from the same colours) and „Waiting for you" (one card per kernel / Docker / + Proxmox set the button may approve now, with the version and the ring-0 evidence). The boxes follow in one narrow + table whose rows open to every value above; release ids, cancelled approvals, what ring 0 runs, the floors, the + crash guard and root files, and „Approve now" (which asks first) sit in a closed „Details". Same data and buttons. - **Household:** one line on the timeline in both languages, informal voice: what was updated and whether the box restarted. Telling households in advance that the box may restart at night is a **promise to users**. That is the operator's decision when the slow lane is built. diff --git a/documentation/audits/hub-system-page-2026-10-10/01-1280.png b/documentation/audits/hub-system-page-2026-10-10/01-1280.png new file mode 100644 index 00000000..80748e6f Binary files /dev/null and b/documentation/audits/hub-system-page-2026-10-10/01-1280.png differ diff --git a/documentation/audits/hub-system-page-2026-10-10/02-390.png b/documentation/audits/hub-system-page-2026-10-10/02-390.png new file mode 100644 index 00000000..10e72065 Binary files /dev/null and b/documentation/audits/hub-system-page-2026-10-10/02-390.png differ diff --git a/documentation/audits/hub-system-page-2026-10-10/03-1280-details-open.png b/documentation/audits/hub-system-page-2026-10-10/03-1280-details-open.png new file mode 100644 index 00000000..59d74553 Binary files /dev/null and b/documentation/audits/hub-system-page-2026-10-10/03-1280-details-open.png differ diff --git a/documentation/audits/hub-system-page-2026-10-10/04-1280-box-open.png b/documentation/audits/hub-system-page-2026-10-10/04-1280-box-open.png new file mode 100644 index 00000000..600ac4be Binary files /dev/null and b/documentation/audits/hub-system-page-2026-10-10/04-1280-box-open.png differ diff --git a/documentation/audits/hub-system-page-2026-10-10/05-1280-waiting-card.png b/documentation/audits/hub-system-page-2026-10-10/05-1280-waiting-card.png new file mode 100644 index 00000000..a3a14425 Binary files /dev/null and b/documentation/audits/hub-system-page-2026-10-10/05-1280-waiting-card.png differ diff --git a/documentation/audits/hub-system-page-2026-10-10/system-fixture-card.html b/documentation/audits/hub-system-page-2026-10-10/system-fixture-card.html new file mode 100644 index 00000000..5fbb2518 --- /dev/null +++ b/documentation/audits/hub-system-page-2026-10-10/system-fixture-card.html @@ -0,0 +1,1657 @@ + + + + + + System — Felhom Hub + + + + + + + + +
+
+

Felhom Hub

+ +
+ + +

System — versions and OS updates

+ + + + +
+

Needs attention

+

One line per box that is amber or red, and why. Amber: worth a look. Red: an operator alarm fires (`08` §6.3).

+ + +
    + +
  • look + tester1-0f1e2d Tester 1 + agent behind: 0.155.0 → 0.156.0 (since 2026-10-08)
    • + +
  • alarm + tester2-9a8b7c Tester 2 + agent behind: 0.141.0 → 0.156.0 (since 2026-10-01) · no versions reported (agent older than v0.142.0) · crash guard not installed
    • + +
+ + +
+ +
+

Waiting for you

+

Update sets that only you approve: the kernel, the Docker engine and the Proxmox packages. Guest and host Debian updates approve themselves after 24 h and one night on the ring-0 boxes.

+ +
+ +
+

Proxmox packages 9.0.12

+

Ran on every ring-0 box — demo-felhom-a1b2c3: 2 healthy night(s), demo-hp-d4e5f6: 2 healthy night(s).

+

first seen 2 days ago (2026-10-08 13:17 UTC). Approving installs nothing by itself: a ring-1 box takes it only through a signed operator job.

+
+ + +
+
+ +
+ + +
    +
  • Docker engine 29.8.3-1 — still being tested: demo-hp-d4e5f6 has 1 of 2 healthy night Docker step(s) with this set
    • +
+ +
+ + +
+

Boxes

+

Click a box to see every value it reports, and its ring and update switches. "unknown": the box could not read the value — never a guess.

+
Box
Ring
Health
Controller · agent
OS updates
Kernel
Last night
+ +
+ +
demo-felhom-a1b2c3Demo N100
+
0
+
fine
+
0.232.0agent 0.156.0
+
on · up to date
+
7.0.14-23-pve
+
ok · 12 h ago
+ +
+
+ +
Ring and updates
+
+
Ring
ring 0 +
+ + + +
+
OS updates
updates ON +
+ + + +
+
Tunnel
running
+
Controller
0.232.0
+
Agent
0.156.0
+
Last OS leg
12 h ago · applied · 41 s
+
+
Docker engine
+
+
Docker
29.8.2
+
containerd
2.3.6-1~debian.13~trixie
+
live-restore
on
+
Docker release
os-docker-20261001-031500
+
+
+
+
Host
+
+
Proxmox
9.0.11
+
Kernel (running)
7.0.14-23-pve
+
Kernel (next boot)
7.0.14-23-pve
+
Kernel (default)
7.0.14-23-pve
+
Kernel step
7.0.14-23-pve applied 30 h ago
+
Debian
13.7
+
Felhom release
os-host-20261009-031500
+
Pending
0
+
Not covered
0
+
Held
none
+
Reboot needed
no
+
kernel.panic
10 s
+
Oops
no
+
Crash restarts 24 h
0
+
Crash guard
armed
+
Root files
0.155.0
+
+
+
+
Guest
+
+
Guest Debian
13.7
+
Felhom release
os-guest-20261009-031500
+
Pending
0
+
Restart needed
0
+
Last disk trim
2 days ago · 3.0 GiB
+
+
+
+
+ +
+ +
demo-hp-d4e5f6Demo HP
+
0
+
fine
+
0.232.0agent 0.156.0
+
on · up to date
+
7.0.14-23-pve
+
ok · 12 h ago
+ +
+
+ +
Ring and updates
+
+
Ring
ring 0 +
+ + + +
+
OS updates
updates ON +
+ + + +
+
Tunnel
running
+
Controller
0.232.0
+
Agent
0.156.0
+
Last OS leg
12 h ago · applied · 41 s
+
+
Docker engine
+
+
Docker
29.8.2
+
containerd
2.3.6-1~debian.13~trixie
+
live-restore
on
+
Docker release
os-docker-20261001-031500
+
+
+
+
Host
+
+
Proxmox
9.0.11
+
Kernel (running)
7.0.14-23-pve
+
Kernel (next boot)
7.0.14-23-pve
+
Kernel (default)
7.0.14-23-pve
+
Kernel step
7.0.14-23-pve applied 30 h ago
+
Debian
13.7
+
Felhom release
os-host-20261009-031500
+
Pending
0
+
Not covered
0
+
Held
none
+
Reboot needed
no
+
kernel.panic
10 s
+
Oops
no
+
Crash restarts 24 h
0
+
Crash guard
armed
+
Root files
0.155.0
+
+
+
+
Guest
+
+
Guest Debian
13.7
+
Felhom release
os-guest-20261009-031500
+
Pending
0
+
Restart needed
0
+
Last disk trim
3 days ago · 3.0 GiB
+
+
+
+
+ +
+ +
tester1-0f1e2dTester 1
+
1
+
look
+
0.231.0agent 0.155.0 → 0.156.0 (since 2026-10-08)
+
on · guest behind
+
7.0.14-20-pve
+
ok · 12 h ago
+ +
+
+ +
Ring and updates
+
+
Ring
ring 1 +
+ + +
+
OS updates
updates ON +
+ + + +
+
Tunnel
running
+
Controller
0.231.0
+
Agent
0.155.0 → 0.156.0 (since 2026-10-08)
+
Last OS leg
12 h ago · nothing · 41 s
+
+
Docker engine
+
+
Docker
29.8.2
+
containerd
2.3.6-1~debian.13~trixie
+
live-restore
on
+
Docker release
—
+
+
+
+
Host
+
+
Proxmox
9.0.11
+
Kernel (running)
7.0.14-20-pve
+
Kernel (next boot)
7.0.14-20-pve
+
Kernel (default)
7.0.14-20-pve
+
Kernel step
due 7.0.14-23-pve — not told yet (mails 09–20 h)
+
Debian
13.7
+
Felhom release
os-host-20261009-031500
+
Pending
0
+
Not covered
0
+
Held
none
+
Reboot needed
no
+
kernel.panic
10 s
+
Oops
no
+
Crash restarts 24 h
0
+
Crash guard
armed
+
Root files
0.155.0
+
+
+
+
Guest
+
+
Guest Debian
13.7
+
Felhom release
os-guest-20261008-031500
+
Pending
0
+
Restart needed
0
+
Last disk trim
4 days ago · 3.0 GiB
+
+
+
+
+ +
+ +
tester2-9a8b7cTester 2
+
1
+
alarm
+
0.229.0agent 0.141.0 → 0.156.0 (since 2026-10-01)
+
on · up to date
+
unknown
+
ok · 12 h ago
+ +
+
+

no versions reported (agent older than v0.142.0)

+
Ring and updates
+
+
Ring
ring 1 +
+ + +
+
OS updates
updates ON +
+ + + +
+
Tunnel
running
+
Controller
0.229.0
+
Agent
0.141.0 → 0.156.0 (since 2026-10-01)
+
Last OS leg
12 h ago · nothing · 41 s
+
+
Docker engine
+
+
Docker
unknown
+
containerd
unknown
+
live-restore
unknown
+
Docker release
—
+
+
+
+
Host
+
+
Proxmox
unknown
+
Kernel (running)
unknown
+
Kernel (next boot)
unknown
+
Kernel (default)
unknown
+
Kernel step
—
+
Debian
unknown
+
Felhom release
os-host-20261009-031500
+
Pending
0
+
Not covered
0
+
Held
unknown
+
Reboot needed
no
+
kernel.panic
unknown
+
Oops
no
+
Crash restarts 24 h
unknown
+
Crash guard
not installed
+
Root files
unknown
+
+
+
+
Guest
+
+
Guest Debian
unknown
+
Felhom release
os-guest-20261009-031500
+
Pending
0
+
Restart needed
0
+
Last disk trim
—
+
+
+
+
+ +
+ + +
+ Details +

Release ids, cancelled approvals, what ring 0 runs, the version floors, and every box's crash guard and root files.

+ +

Approved releases

+

The newest approved set of each layer. A TEST approval is amber.

+
+ +
guest: os-guest-20261009-031500
+ 214 packages · 2026-10-09 09:17 UTC · by auto +
+ +
host: os-host-20261009-031500
+ 389 packages · 2026-10-09 09:17 UTC · by auto +
+ +
docker: os-docker-20261001-031500
+ 4 packages · 2026-10-09 09:17 UTC · by auto +
+ +
pve: os-pve-20261007-090000
+ 31 packages · 2026-10-09 09:17 UTC · by auto +
+ +
kernel: os-kernel-20261010-091200
+ 2 packages · 2026-10-09 09:17 UTC · by operator +
+ +
+
+ + + An urgent fix only — normally the hub approves after 24 h and one night. +
+ +

Cancelled approvals (last 7 days)

+
+ +
guest: os-guest-20261006-100000
+ cancelled 2026-10-07 08:00:00 UTC — a TEST approval
+ no further box installs it; boxes that installed it keep it
+ +
host: os-host-20261006-110000
+ cancelled 2026-10-07 08:00:00 UTC — a TEST approval
+ no further box installs it; boxes that installed it keep it
+ +
docker: os-docker-20261006-120000
+ cancelled 2026-10-07 08:00:00 UTC — a TEST approval
+ no further box installs it; boxes that installed it keep it
+ +
pve: os-pve-20261006-130000
+ cancelled 2026-10-07 08:00:00 UTC — a TEST approval
+ no further box installs it; boxes that installed it keep it
+ +
+ + +

What ring 0 runs now

+
+ +
guest: + 214 packages, first seen 2026-10-09 03:17 UTC
+ approved as os-guest-20261009-031500 + +
+ +
host: + 389 packages, first seen 2026-10-09 03:17 UTC
+ approved as os-host-20261009-031500 + +
+ +
docker: + 4 packages, first seen 2026-10-09 19:17 UTC
+ demo-hp-d4e5f6 has 1 of 2 healthy night Docker step(s) with this set + +
+ +
pve: + 33 packages, first seen 2026-10-08 13:17 UTC
+ ready — see "Waiting for you" +
+ +
kernel: + 2 packages, first seen 2026-10-09 13:17 UTC
+ approved as os-kernel-20261010-091200 + +
+ +
+ +

Version floors

+

Global controller floor: 0.229.0 · vouched agent: 0.156.0

+

What is a floor? · What is a vouched agent?

+ +
+ + + + + + + + + + + + +
CustomerOwn floorSetGlobal floor moves it?
c-tester1
Tester 1		0.231.05 days ago (2026-10-05)no — its own floor applies (at or above the global)
+
+ + + +

Crash guard and root files

+
+ + + + + + + + + + + + + +
Boxkernel.panicOopsCrash restarts 24 hCrash guardRoot files
demo-felhom-a1b2c310 sno0armed0.155.0
demo-hp-d4e5f610 sno0armed0.155.0
tester1-0f1e2d10 sno0armed0.155.0
tester2-9a8b7cunknownnounknownnot installedunknown
+
+ +
+ +
+ Felhom Hub test +
+
+ + + + + diff --git a/documentation/audits/hub-system-page-2026-10-10/system-fixture.html b/documentation/audits/hub-system-page-2026-10-10/system-fixture.html new file mode 100644 index 00000000..b53306f8 --- /dev/null +++ b/documentation/audits/hub-system-page-2026-10-10/system-fixture.html @@ -0,0 +1,1644 @@ + + + + + + System — Felhom Hub + + + + + + + + +
+
+

Felhom Hub

+ +
+ + +

System — versions and OS updates

+ + + + +
+

Needs attention

+

One line per box that is amber or red, and why. Amber: worth a look. Red: an operator alarm fires (`08` §6.3).

+ + +
    + +
  • look + tester1-0f1e2d Tester 1 + agent behind: 0.155.0 → 0.156.0 (since 2026-10-08)
    • + +
  • alarm + tester2-9a8b7c Tester 2 + agent behind: 0.141.0 → 0.156.0 (since 2026-10-01) · no versions reported (agent older than v0.142.0) · crash guard not installed
    • + +
+ + +
+ +
+

Waiting for you

+

Update sets that only you approve: the kernel, the Docker engine and the Proxmox packages. Guest and host Debian updates approve themselves after 24 h and one night on the ring-0 boxes.

+

Nothing waits for your approval.

+ +
    +
  • Docker engine 29.8.3-1 — still being tested: demo-hp-d4e5f6 has 1 of 2 healthy night Docker step(s) with this set
    • +
+ +
+ + +
+

Boxes

+

Click a box to see every value it reports, and its ring and update switches. "unknown": the box could not read the value — never a guess.

+
Box
Ring
Health
Controller · agent
OS updates
Kernel
Last night
+ +
+ +
demo-felhom-a1b2c3Demo N100
+
0
+
fine
+
0.232.0agent 0.156.0
+
on · up to date
+
7.0.14-23-pve
+
ok · 12 h ago
+ +
+
+ +
Ring and updates
+
+
Ring
ring 0 +
+ + + +
+
OS updates
updates ON +
+ + + +
+
Tunnel
running
+
Controller
0.232.0
+
Agent
0.156.0
+
Last OS leg
12 h ago · applied · 41 s
+
+
Docker engine
+
+
Docker
29.8.2
+
containerd
2.3.6-1~debian.13~trixie
+
live-restore
on
+
Docker release
os-docker-20261001-031500
+
+
+
+
Host
+
+
Proxmox
9.0.11
+
Kernel (running)
7.0.14-23-pve
+
Kernel (next boot)
7.0.14-23-pve
+
Kernel (default)
7.0.14-23-pve
+
Kernel step
7.0.14-23-pve applied 30 h ago
+
Debian
13.7
+
Felhom release
os-host-20261009-031500
+
Pending
0
+
Not covered
0
+
Held
none
+
Reboot needed
no
+
kernel.panic
10 s
+
Oops
no
+
Crash restarts 24 h
0
+
Crash guard
armed
+
Root files
0.155.0
+
+
+
+
Guest
+
+
Guest Debian
13.7
+
Felhom release
os-guest-20261009-031500
+
Pending
0
+
Restart needed
0
+
Last disk trim
2 days ago · 3.0 GiB
+
+
+
+
+ +
+ +
demo-hp-d4e5f6Demo HP
+
0
+
fine
+
0.232.0agent 0.156.0
+
on · up to date
+
7.0.14-23-pve
+
ok · 12 h ago
+ +
+
+ +
Ring and updates
+
+
Ring
ring 0 +
+ + + +
+
OS updates
updates ON +
+ + + +
+
Tunnel
running
+
Controller
0.232.0
+
Agent
0.156.0
+
Last OS leg
12 h ago · applied · 41 s
+
+
Docker engine
+
+
Docker
29.8.2
+
containerd
2.3.6-1~debian.13~trixie
+
live-restore
on
+
Docker release
os-docker-20261001-031500
+
+
+
+
Host
+
+
Proxmox
9.0.11
+
Kernel (running)
7.0.14-23-pve
+
Kernel (next boot)
7.0.14-23-pve
+
Kernel (default)
7.0.14-23-pve
+
Kernel step
7.0.14-23-pve applied 30 h ago
+
Debian
13.7
+
Felhom release
os-host-20261009-031500
+
Pending
0
+
Not covered
0
+
Held
none
+
Reboot needed
no
+
kernel.panic
10 s
+
Oops
no
+
Crash restarts 24 h
0
+
Crash guard
armed
+
Root files
0.155.0
+
+
+
+
Guest
+
+
Guest Debian
13.7
+
Felhom release
os-guest-20261009-031500
+
Pending
0
+
Restart needed
0
+
Last disk trim
3 days ago · 3.0 GiB
+
+
+
+
+ +
+ +
tester1-0f1e2dTester 1
+
1
+
look
+
0.231.0agent 0.155.0 → 0.156.0 (since 2026-10-08)
+
on · guest behind
+
7.0.14-20-pve
+
ok · 12 h ago
+ +
+
+ +
Ring and updates
+
+
Ring
ring 1 +
+ + +
+
OS updates
updates ON +
+ + + +
+
Tunnel
running
+
Controller
0.231.0
+
Agent
0.155.0 → 0.156.0 (since 2026-10-08)
+
Last OS leg
12 h ago · nothing · 41 s
+
+
Docker engine
+
+
Docker
29.8.2
+
containerd
2.3.6-1~debian.13~trixie
+
live-restore
on
+
Docker release
—
+
+
+
+
Host
+
+
Proxmox
9.0.11
+
Kernel (running)
7.0.14-20-pve
+
Kernel (next boot)
7.0.14-20-pve
+
Kernel (default)
7.0.14-20-pve
+
Kernel step
due 7.0.14-23-pve — not told yet (mails 09–20 h)
+
Debian
13.7
+
Felhom release
os-host-20261009-031500
+
Pending
0
+
Not covered
0
+
Held
none
+
Reboot needed
no
+
kernel.panic
10 s
+
Oops
no
+
Crash restarts 24 h
0
+
Crash guard
armed
+
Root files
0.155.0
+
+
+
+
Guest
+
+
Guest Debian
13.7
+
Felhom release
os-guest-20261008-031500
+
Pending
0
+
Restart needed
0
+
Last disk trim
4 days ago · 3.0 GiB
+
+
+
+
+ +
+ +
tester2-9a8b7cTester 2
+
1
+
alarm
+
0.229.0agent 0.141.0 → 0.156.0 (since 2026-10-01)
+
on · up to date
+
unknown
+
ok · 12 h ago
+ +
+
+

no versions reported (agent older than v0.142.0)

+
Ring and updates
+
+
Ring
ring 1 +
+ + +
+
OS updates
updates ON +
+ + + +
+
Tunnel
running
+
Controller
0.229.0
+
Agent
0.141.0 → 0.156.0 (since 2026-10-01)
+
Last OS leg
12 h ago · nothing · 41 s
+
+
Docker engine
+
+
Docker
unknown
+
containerd
unknown
+
live-restore
unknown
+
Docker release
—
+
+
+
+
Host
+
+
Proxmox
unknown
+
Kernel (running)
unknown
+
Kernel (next boot)
unknown
+
Kernel (default)
unknown
+
Kernel step
—
+
Debian
unknown
+
Felhom release
os-host-20261009-031500
+
Pending
0
+
Not covered
0
+
Held
unknown
+
Reboot needed
no
+
kernel.panic
unknown
+
Oops
no
+
Crash restarts 24 h
unknown
+
Crash guard
not installed
+
Root files
unknown
+
+
+
+
Guest
+
+
Guest Debian
unknown
+
Felhom release
os-guest-20261009-031500
+
Pending
0
+
Restart needed
0
+
Last disk trim
—
+
+
+
+
+ +
+ + +
+ Details +

Release ids, cancelled approvals, what ring 0 runs, the version floors, and every box's crash guard and root files.

+ +

Approved releases

+

The newest approved set of each layer. A TEST approval is amber.

+
+ +
guest: os-guest-20261009-031500
+ 214 packages · 2026-10-09 09:17 UTC · by auto +
+ +
host: os-host-20261009-031500
+ 389 packages · 2026-10-09 09:17 UTC · by auto +
+ +
docker: os-docker-20261001-031500
+ 4 packages · 2026-10-09 09:17 UTC · by auto +
+ +
pve: os-pve-20261007-090000
+ 31 packages · 2026-10-09 09:17 UTC · by auto +
+ +
kernel: os-kernel-20261010-091200
+ 2 packages · 2026-10-09 09:17 UTC · by operator +
+ +
+
+ + + An urgent fix only — normally the hub approves after 24 h and one night. +
+ +

Cancelled approvals (last 7 days)

+
+ +
guest: os-guest-20261006-100000
+ cancelled 2026-10-07 08:00:00 UTC — a TEST approval
+ no further box installs it; boxes that installed it keep it
+ +
host: os-host-20261006-110000
+ cancelled 2026-10-07 08:00:00 UTC — a TEST approval
+ no further box installs it; boxes that installed it keep it
+ +
docker: os-docker-20261006-120000
+ cancelled 2026-10-07 08:00:00 UTC — a TEST approval
+ no further box installs it; boxes that installed it keep it
+ +
pve: os-pve-20261006-130000
+ cancelled 2026-10-07 08:00:00 UTC — a TEST approval
+ no further box installs it; boxes that installed it keep it
+ +
+ + +

What ring 0 runs now

+
+ +
guest: + 214 packages, first seen 2026-10-09 03:17 UTC
+ approved as os-guest-20261009-031500 + +
+ +
host: + 389 packages, first seen 2026-10-09 03:17 UTC
+ approved as os-host-20261009-031500 + +
+ +
docker: + 4 packages, first seen 2026-10-09 19:17 UTC
+ demo-hp-d4e5f6 has 1 of 2 healthy night Docker step(s) with this set + +
+ +
pve: + 31 packages, first seen 2026-10-06 15:17 UTC
+ approved as os-pve-20261007-090000 + +
+ +
kernel: + 2 packages, first seen 2026-10-09 13:17 UTC
+ approved as os-kernel-20261010-091200 + +
+ +
+ +

Version floors

+

Global controller floor: 0.229.0 · vouched agent: 0.156.0

+

What is a floor? · What is a vouched agent?

+ +
+ + + + + + + + + + + + +
CustomerOwn floorSetGlobal floor moves it?
c-tester1
Tester 1		0.231.05 days ago (2026-10-05)no — its own floor applies (at or above the global)
+
+ + + +

Crash guard and root files

+
+ + + + + + + + + + + + + +
Boxkernel.panicOopsCrash restarts 24 hCrash guardRoot files
demo-felhom-a1b2c310 sno0armed0.155.0
demo-hp-d4e5f610 sno0armed0.155.0
tester1-0f1e2d10 sno0armed0.155.0
tester2-9a8b7cunknownnounknownnot installedunknown
+
+ +
+ +
+ Felhom Hub test +
+
+ + + + + diff --git a/hub/CHANGELOG.md b/hub/CHANGELOG.md index 82212451..eab77b60 100644 --- a/hub/CHANGELOG.md +++ b/hub/CHANGELOG.md @@ -1,3 +1,22 @@ +## Unreleased — the System page answers „is anything wrong?" and „is anything waiting for me?" first (2026-10-10) + +Not released, not deployed: it ships with the next hub release. View only — the same data, buttons, routes and CSRF +field; no change to what the hub does. + +- **Layout** (`templates/system.html`, view model `internal/web/system_view.go`): four parts, in this order. + **Needs attention** — one line per amber or red box with the reasons in plain words, built from the same cell + colours (`08` §6.3), or „All boxes look fine." **Waiting for you** — one card per operator lane (kernel, Docker, + Proxmox) the button may approve now (the same gate as before), with the version, which ring-0 boxes ran it and how, + and when it was first seen; sets still being tested are listed with the hub's own reason; or „Nothing waits for your + approval." **Boxes** — one 7-column table (box, ring, health, controller · agent, OS updates, kernel, last night); + each row is a `
` that opens to every value of the old 30-column table and the ring / update switches. + **Details** (closed) — approved releases with ids, „Approve now (guest + host)" (now asks first), cancelled + approvals, what ring 0 runs, version floors, crash guard and root files. +- No sideways scroll at 1280 px or 390 px (the nav wraps on a phone, page CSS only). Works without JavaScript. +- Tests: `system_layout_test.go` — every old item still on the page, every form carries `_csrf` and posts only the + old routes, Needs attention (green / amber / red / all fine), Waiting cards per lane and the empty line; each + red-proofed (observed). Screenshots: `documentation/audits/hub-system-page-2026-10-10/`. + ## v0.145.0 — a SECURITY fix (one box's key acted for any household), R-922 (a household's clear deletes its address), MAIL-HOLD (a restored hub starts quiet), and the kernel approval rule (2026-10-10) Released 2026-10-10 (operator present for the deploy). diff --git a/hub/internal/web/system.go b/hub/internal/web/system.go index 27840b93..910470a9 100644 --- a/hub/internal/web/system.go +++ b/hub/internal/web/system.go @@ -1,6 +1,7 @@ package web import ( + "encoding/json" "fmt" "net/http" "sort" @@ -46,6 +47,10 @@ type systemRow struct { Engine, Containerd, LiveRestore, DockerRelease cell // last leg LastLeg cell + // the narrow table and "Needs attention" (system_view.go) + Controller, Mark, Updates, KernelShort, LastNight cell + KernelNext string + Reasons []reason } // OSSystemView is what the System page needs beyond OSUpdateAdmin (implemented by *osupdates.Service). @@ -58,8 +63,8 @@ type OSSystemView interface { BundleThreshold() time.Duration AgentThreshold() time.Duration ApproveDocker() (string, error) - ApprovePVE() (string, error) // R-812 option A: the Proxmox package set - ApproveKernel() (string, error) // R-836: the kernel set + ApprovePVE() (string, error) // R-812 option A: the Proxmox package set + ApproveKernel() (string, error) // R-836: the kernel set KernelLineFor(hostID string) osupdates.KernelLine // R-836: one box's kernel step and day-before mail } @@ -383,6 +388,7 @@ func buildSystemRows(lines []osupdates.FleetLine, facts map[string]sysfacts.Syst } else if last.LastOutcome == "health_failed" || last.LastOutcome == "failed" || last.LastOutcome == "refused" { r.LastLeg.Class = "warn" } + r.LastNight = lastNightCell(l.Enabled, last.LastOutcome, last.LastAt, r.LastLeg, now) rows = append(rows, r) } sort.Slice(rows, func(i, j int) bool { return rows[i].HostID < rows[j].HostID }) @@ -409,44 +415,61 @@ func (s *Server) handleSystem(w http.ResponseWriter, r *http.Request) { return } hosts, _ := s.store.ListHosts() - facts, names := map[string]sysfacts.System{}, map[string]string{} - for _, h := range hosts { - names[h.HostID] = s.customerName(h.CustomerID) - if rj, _ := s.store.GetLatestHostReportJSON(h.CustomerID); rj != "" { - facts[h.HostID] = sysfacts.Parse(rj) + now := time.Now() + in := systemInput{Lines: lines, Facts: map[string]sysfacts.System{}, Names: map[string]string{}, Controllers: map[string]string{}, + Agents: map[string]string{}, KernelLines: map[string]osupdates.KernelLine{}, BundleSince: map[string]time.Time{}, + AgentSince: map[string]time.Time{}, BundleAfter: view.BundleThreshold(), AgentAfter: view.AgentThreshold(), + GlobalFloor: s.store.GetGlobalMinControllerVersion(), Releases: view.Releases(), Cancelled: view.CancelledReleases(), + Candidates: view.Candidates(), Now: now} + in.Stale, in.Reboot, in.NotCov = view.Thresholds() + ctrl := map[string]string{} + if cs, cerr := s.store.GetCustomers(); cerr != nil { + s.logger.Printf("[ERROR] system page: customers: %v", cerr) + } else { + for _, c := range cs { + ctrl[c.CustomerID] = c.ControllerVersion } } - stale, reboot, notCov := view.Thresholds() - rows := buildSystemRows(lines, facts, names, stale, reboot, notCov, time.Now()) - man := s.store.GetArtifactManifest() - agents := map[string]string{} for _, h := range hosts { - agents[h.HostID] = h.AgentVersion + in.Names[h.HostID] = s.customerName(h.CustomerID) + in.Agents[h.HostID] = h.AgentVersion + in.Controllers[h.HostID] = ctrl[h.CustomerID] + if rj, _ := s.store.GetLatestHostReportJSON(h.CustomerID); rj != "" { + in.Facts[h.HostID] = sysfacts.Parse(rj) + } + in.KernelLines[h.HostID] = view.KernelLineFor(h.HostID) + in.BundleSince[h.HostID] = s.store.BundleBehindSince(h.HostID) + in.AgentSince[h.HostID] = s.store.AgentBehindSince(h.HostID) } - for i := range rows { - rows[i].KernelDefault, rows[i].KernelStep = kernelCells(facts[rows[i].HostID], view.KernelLineFor(rows[i].HostID), time.Now()) - rows[i].Bundle = bundleCell(facts[rows[i].HostID], man.AgentVersion, man.BundleSHA256, - s.store.BundleBehindSince(rows[i].HostID), view.BundleThreshold(), time.Now()) - rows[i].Agent = agentCell(agents[rows[i].HostID], man.AgentVersion, - s.store.AgentBehindSince(rows[i].HostID), view.AgentThreshold(), time.Now()) - } - global := s.store.GetGlobalMinControllerVersion() + man := s.store.GetArtifactManifest() + in.VouchedAgent, in.VouchedBundle = man.AgentVersion, man.BundleSHA256 ovs, oerr := s.store.CustomerFloorOverrides() if oerr != nil { s.logger.Printf("[ERROR] system page: floor overrides: %v", oerr) } - data := map[string]interface{}{ - "Rows": rows, - "GlobalFloor": global, - "VouchedAgent": man.AgentVersion, - "Floors": buildFloorRows(ovs, global, time.Now()), - "Releases": view.Releases(), - "Cancelled": view.CancelledReleases(), - "Candidates": view.Candidates(), - "Flash": r.URL.Query().Get("flash"), - "FlashErr": r.URL.Query().Get("err"), - "CSRFToken": s.getCSRFToken(r), + in.Floors = ovs + in.Packages = func(fp string) []osupdates.Package { + var list []osupdates.Package + if pj, _ := s.store.OSCandidatePackages(fp); pj != "" { + _ = json.Unmarshal([]byte(pj), &list) + } + return list } + // A healthy night run of the layer since the set was first seen: the same count the approval rule makes. + in.Nights = func(hostID, layer string, since time.Time) int { + reps, _ := s.store.OSReportsSince(hostID, layer, since) + n := 0 + for _, rep := range reps { + if rep.Trigger == "night" && rep.Healthy && rep.Outcome != "failed" && rep.Outcome != "refused" && rep.Outcome != "health_failed" { + n++ + } + } + return n + } + data := buildSystemPage(in) + data["Flash"] = r.URL.Query().Get("flash") + data["FlashErr"] = r.URL.Query().Get("err") + data["CSRFToken"] = s.getCSRFToken(r) if err := s.templates.ExecuteTemplate(w, "system.html", data); err != nil { s.logger.Printf("[ERROR] system.html template: %v", err) } diff --git a/hub/internal/web/system_layout_test.go b/hub/internal/web/system_layout_test.go new file mode 100644 index 00000000..8e02cd66 --- /dev/null +++ b/hub/internal/web/system_layout_test.go @@ -0,0 +1,334 @@ +package web + +import ( + "bytes" + "fmt" + "os" + "path/filepath" + "regexp" + "strings" + "testing" + "time" + + "gitea.dooplex.hu/admin/felhom-hub/internal/osupdates" + "gitea.dooplex.hu/admin/felhom-hub/internal/store" + "gitea.dooplex.hu/admin/felhom-hub/internal/sysfacts" +) + +// The System page's 2026-10-10 layout: Needs attention, Waiting for you, Boxes, Details. These tests render the page +// from a fixture shaped like the fleet on 2026-10-10 (made-up host ids, no real key or address). + +var fixNow = time.Date(2026, 10, 10, 15, 17, 0, 0, time.UTC) + +func fixFacts(kernel, nextBoot string, trimAgo time.Duration) sysfacts.System { + trim := fixNow.Add(-trimAgo).Format(time.RFC3339) + return sysfacts.Parse(fmt.Sprintf(`{"host":{"cpu_percent":1},"guest_disk_trim":{"schedule":"weekly","guests":[{"vmid":9201, +"last_attempt_at":%q,"last_ok_at":%q,"ok":true,"bytes_trimmed":3221225472}]}, +"system":{"pve_version":"pve-manager/9.0.11/abc","kernel_version":"Linux %s #1","vmid":9201, +"config_bundle":{"version":"0.155.0","bundle_sha256":"vouched-sha"}, +"facts":{"host":{"debian":"13.7","kernel_running":%q,"kernel_next_boot":%q,"kernel_next_boot_source":"saved default","held":[], +"kernel_panic":10,"oops_this_boot":false,"crash_guard":{"armed":true,"tripped":false,"unclean_boots_24h":0}, +"kernel_lane":{"running":%q,"default":%q,"phase":"none"}}, +"guest":{"debian":"13.7","docker_engine":"29.8.2","containerd":"2.3.6-1~debian.13~trixie","live_restore":"on"}}}}`, + trim, trim, kernel, kernel, nextBoot, kernel, nextBoot)) +} + +func fixLeg(rel, outcome string, ago time.Duration) osupdates.LayerLine { + at := fixNow.Add(-ago) + return osupdates.LayerLine{ReleaseID: rel, LastOutcome: outcome, LastAt: at, LastSuccessfulLeg: at, WrapperPassSeconds: 41} +} + +// fixtureInput is today's real situation, made up: two ring-0 demo boxes, two testers; the kernel set approved, the +// Docker set still being tested, Tester 2 on an agent that reports no versions ("unknown"); four TEST approvals +// cancelled. readyPVE adds a Proxmox set ready to approve (a "Waiting for you" card). +func fixtureInput(readyPVE bool) systemInput { + g, h := "os-guest-20261009-031500", "os-host-20261009-031500" + lines := []osupdates.FleetLine{ + {HostID: "demo-felhom-a1b2c3", Ring: 0, Enabled: true, Tunnel: "running", Guest: fixLeg(g, "applied", 12*time.Hour), Host: fixLeg(h, "nothing", 12*time.Hour), Docker: fixLeg("os-docker-20261001-031500", "nothing", 12*time.Hour)}, + {HostID: "demo-hp-d4e5f6", Ring: 0, Enabled: true, Tunnel: "running", Guest: fixLeg(g, "applied", 12*time.Hour), Host: fixLeg(h, "applied", 12*time.Hour), Docker: fixLeg("os-docker-20261001-031500", "nothing", 12*time.Hour)}, + {HostID: "tester1-0f1e2d", Ring: 1, Enabled: true, Tunnel: "running", Guest: fixLeg("os-guest-20261008-031500", "applied", 36*time.Hour), Host: fixLeg(h, "nothing", 12*time.Hour)}, + {HostID: "tester2-9a8b7c", Ring: 1, Enabled: true, Tunnel: "running", Guest: fixLeg(g, "nothing", 12*time.Hour), Host: fixLeg(h, "nothing", 12*time.Hour)}, + } + k := "7.0.14-23-pve" + facts := map[string]sysfacts.System{ + "demo-felhom-a1b2c3": fixFacts(k, k, 2*24*time.Hour), + "demo-hp-d4e5f6": fixFacts(k, k, 3*24*time.Hour), + "tester1-0f1e2d": fixFacts("7.0.14-20-pve", "7.0.14-20-pve", 4*24*time.Hour), + "tester2-9a8b7c": sysfacts.Parse(`{"host":{"cpu_percent":1}}`), + } + approved := func(layer, id string, n int, test bool) osupdates.ReleaseInfo { + return osupdates.ReleaseInfo{Layer: layer, ID: id, ApprovedAt: fixNow.Add(-30 * time.Hour), ApprovedBy: "auto", Packages: n, Test: test} + } + rels := []osupdates.ReleaseInfo{approved("guest", g, 214, false), approved("host", h, 389, false), + approved("docker", "os-docker-20261001-031500", 4, false), approved("pve", "os-pve-20261007-090000", 31, false), + approved("kernel", "os-kernel-20261010-091200", 2, false)} + rels[4].ApprovedBy = "operator" + var cancelled []osupdates.ReleaseInfo + for i, l := range []string{"guest", "host", "docker", "pve"} { + c := approved(l, fmt.Sprintf("os-%s-20261006-1%d0000", l, i), 3, true) + c.Cancelled = "2026-10-07 08:00:00" + cancelled = append(cancelled, c) + } + cands := []osupdates.Status{ + {Layer: "guest", Fingerprint: "fp-g", FirstSeen: fixNow.Add(-36 * time.Hour), Packages: 214, Approved: g}, + {Layer: "host", Fingerprint: "fp-h", FirstSeen: fixNow.Add(-36 * time.Hour), Packages: 389, Approved: h}, + {Layer: "docker", Fingerprint: "fp-d", FirstSeen: fixNow.Add(-20 * time.Hour), Packages: 4, + Waiting: "demo-hp-d4e5f6 has 1 of 2 healthy night Docker step(s) with this set"}, + {Layer: "pve", Fingerprint: "fp-p", FirstSeen: fixNow.Add(-4 * 24 * time.Hour), Packages: 31, Approved: "os-pve-20261007-090000", Waiting: "already approved"}, + {Layer: "kernel", Fingerprint: "fp-k", FirstSeen: fixNow.Add(-26 * time.Hour), Packages: 2, Approved: "os-kernel-20261010-091200", Waiting: "already approved"}, + } + if readyPVE { + cands[3] = osupdates.Status{Layer: "pve", Fingerprint: "fp-p2", FirstSeen: fixNow.Add(-50 * time.Hour), Packages: 33} + } + pkgs := map[string][]osupdates.Package{ + "fp-d": {{Name: "containerd.io", Version: "2.3.7-1"}, {Name: "docker-ce", Version: "5:29.8.3-1~debian.13~trixie"}}, + "fp-p2": {{Name: "pve-manager", Version: "9.0.12"}, {Name: "libpve-common-perl", Version: "9.0.8"}}, + "fp-k": {{Name: "proxmox-kernel-7.0", Version: "7.0.14-23"}}, + } + return systemInput{ + Lines: lines, Facts: facts, + Names: map[string]string{"demo-felhom-a1b2c3": "Demo N100", "demo-hp-d4e5f6": "Demo HP", "tester1-0f1e2d": "Tester 1", "tester2-9a8b7c": "Tester 2"}, + Controllers: map[string]string{"demo-felhom-a1b2c3": "0.232.0", "demo-hp-d4e5f6": "0.232.0", "tester1-0f1e2d": "0.231.0", "tester2-9a8b7c": "0.229.0"}, + Agents: map[string]string{"demo-felhom-a1b2c3": "0.156.0", "demo-hp-d4e5f6": "0.156.0", "tester1-0f1e2d": "0.155.0", "tester2-9a8b7c": "0.141.0"}, + KernelLines: map[string]osupdates.KernelLine{ + "demo-felhom-a1b2c3": {LastOutcome: "applied", LastKernel: k, LastAt: fixNow.Add(-30 * time.Hour)}, + "demo-hp-d4e5f6": {LastOutcome: "applied", LastKernel: k, LastAt: fixNow.Add(-30 * time.Hour)}, + "tester1-0f1e2d": {Due: k}, + }, + BundleSince: map[string]time.Time{}, AgentSince: map[string]time.Time{"tester1-0f1e2d": fixNow.Add(-2 * 24 * time.Hour), "tester2-9a8b7c": fixNow.Add(-9 * 24 * time.Hour)}, + Stale: 7 * 24 * time.Hour, Reboot: 14 * 24 * time.Hour, NotCov: 14 * 24 * time.Hour, + BundleAfter: 7 * 24 * time.Hour, AgentAfter: 7 * 24 * time.Hour, + VouchedAgent: "0.156.0", VouchedBundle: "vouched-sha", GlobalFloor: "0.229.0", + Floors: []store.CustomerFloorOverride{{CustomerID: "c-tester1", CustomerName: "Tester 1", Version: "0.231.0", SetAt: fixNow.Add(-5 * 24 * time.Hour)}}, + Releases: rels, Cancelled: cancelled, Candidates: cands, + Packages: func(fp string) []osupdates.Package { return pkgs[fp] }, + Nights: func(string, string, time.Time) int { return 2 }, + Now: fixNow, + } +} + +func renderSystem(t *testing.T, in systemInput) string { + t.Helper() + s, _ := newTestServer(t) + data := buildSystemPage(in) + data["CSRFToken"] = "csrf-fixture-token" + var b bytes.Buffer + if err := s.templates.ExecuteTemplate(&b, "system.html", data); err != nil { + t.Fatal(err) + } + return b.String() +} + +// sysSection returns the page text between two section ids, so a check is made where the item is meant to be. +func sysSection(page, id string) string { + i := strings.Index(page, `id="`+id+`"`) + if i < 0 { + return "" + } + rest := page[i:] + end := len(rest) + for _, m := range []string{"= 0 && j+1 < end { + end = j + 1 + } + } + return rest[:end] +} + +// The contract: every item and button the page had before 2026-10-10 is still on it — in the main part or in Details. +// COMPANION RED-PROOF (observed): drop the Docker-engine group from the box panel → "lacks \">containerd\"". +func TestSystemPage_EveryItemStillOnThePage(t *testing.T) { + page := renderSystem(t, fixtureInput(true)) + boxes, details := sysSection(page, "boxes"), sysSection(page, "details") + for _, want := range []string{ + // per box, in the box panel (every column of the old wide table) + `href="/hosts/demo-hp-d4e5f6"`, "Demo HP", `action="/os/ring/demo-hp-d4e5f6"`, `action="/os/enabled/tester1-0f1e2d"`, + ">Tunnel", ">Proxmox", ">Kernel (running)", ">Kernel (next boot)", ">Kernel (default)", + ">Kernel step", ">Debian", ">Felhom release", ">Pending", ">Not covered", ">Held", + ">Reboot needed", ">kernel.panic", ">Oops", ">Crash restarts 24 h", ">Crash guard", + ">Root files", ">Agent", ">Guest Debian", ">Restart needed", ">Last disk trim", + ">Docker", ">containerd", ">live-restore", ">Docker release", ">Last OS leg", + "no versions reported (agent older than v0.142.0)", "9.0.11", "29.8.2", "2.3.6-1~debian.13~trixie", + } { + if !strings.Contains(boxes, want) { + t.Errorf("Boxes lacks %q", want) + } + } + for _, want := range []string{ + "Approved releases", "os-kernel-20261010-091200", "214 packages", "by operator", + "Cancelled approvals (last 7 days)", "os-docker-20261006-120000", "a TEST approval", "boxes that installed it keep it", + "What ring 0 runs now", "first seen", "approved as os-guest-20261009-031500", "1 of 2 healthy night Docker step", + `action="/os/approve-now"`, "Version floors", "Global controller floor: 0.229.0", "vouched agent: 0.156.0", + `href="/customers/c-tester1"`, "Global floor moves it?", "Crash guard and root files", + } { + if !strings.Contains(details, want) { + t.Errorf("Details lacks %q", want) + } + } + if !strings.Contains(sysSection(page, "waiting"), `action="/os/approve-pve"`) { + t.Error("the ready Proxmox set has no button in Waiting for you") + } + if strings.Count(page, ">unknown<") < 6 { + t.Errorf("Tester 2's values must read unknown, got %d", strings.Count(page, ">unknown<")) + } + //
carries the click-to-open parts: the page works without JavaScript, and Details is closed by default. + if !strings.Contains(page, `
`) || strings.Contains(page, `id="details" open`) { + t.Error("Details must be a
element, closed by default") + } + if strings.Count(page, `
`).FindAllStringSubmatch(page, -1) + if len(forms) < 10 { + t.Fatalf("only %d forms on the page", len(forms)) + } + allowed := regexp.MustCompile(`^/os/(ring/[a-z0-9-]+|enabled/[a-z0-9-]+|approve-now|approve-docker|approve-pve|approve-kernel)$`) + for _, f := range forms { + if !strings.Contains(f[0], `name="_csrf" value="csrf-fixture-token"`) || !strings.Contains(f[0], `name="return" value="/system"`) { + t.Errorf("form %s lacks the CSRF or return field", f[1]) + } + if !allowed.MatchString(f[1]) { + t.Errorf("form posts to a route the page never had: %s", f[1]) + } + } +} + +func attentionOf(t *testing.T, in systemInput) string { + t.Helper() + return sysSection(renderSystem(t, in), "attention") +} + +// Needs attention: a green box is not listed, an amber and a red one are, each with its reason; all green → one line. +// COMPANION RED-PROOF (observed): skip the cells in summariseRow → "the amber box (agent behind) is missing or has no reason". +func TestSystemPage_NeedsAttention(t *testing.T) { + in := fixtureInput(false) + att := attentionOf(t, in) + if strings.Contains(att, "demo-felhom-a1b2c3") { + t.Error("a green box is listed") + } + if !strings.Contains(att, "tester1-0f1e2d") || !strings.Contains(att, "agent behind: 0.155.0 → 0.156.0") { + t.Errorf("the amber box (agent behind) is missing or has no reason:\n%s", att) + } + if !strings.Contains(att, `class="mark c-bad"`) || !strings.Contains(att, "tester2-9a8b7c") { + t.Errorf("the red box (agent behind 9 days) is missing:\n%s", att) + } + if !strings.Contains(att, "no versions reported") { + t.Error("Tester 2's unknown values have no plain reason") + } + if strings.Contains(att, "All boxes look fine") { + t.Error("says all fine while two boxes are not") + } + + // A red cell of its own: the kernel step's failed revert. + in.KernelLines["demo-hp-d4e5f6"] = osupdates.KernelLine{LastOutcome: "revert_failed", LastKernel: "7.0.14-23-pve", LastAt: fixNow.Add(-9 * time.Hour)} + att = attentionOf(t, in) + if !strings.Contains(att, "kernel step: 7.0.14-23-pve revert failed 9 h ago") { + t.Errorf("the kernel step's failure has no plain reason:\n%s", att) + } + // Updates switched off: amber, in plain words. + in.Lines[0].Enabled = false + if att = attentionOf(t, in); !strings.Contains(att, "OS updates switched off") { + t.Error("a box with updates off is not listed") + } + + // All green. + green := fixtureInput(false) + green.Lines, green.Facts = green.Lines[:2], map[string]sysfacts.System{"demo-felhom-a1b2c3": green.Facts["demo-felhom-a1b2c3"], "demo-hp-d4e5f6": green.Facts["demo-hp-d4e5f6"]} + if att = attentionOf(t, green); !strings.Contains(att, "All boxes look fine.") || strings.Contains(att, `class="att"`) { + t.Errorf("all-green fleet:\n%s", att) + } +} + +// Waiting for you: one card per operator lane that the button may approve (the same gate as before), the empty line, and +// guest/host never as a card (they approve themselves). +// COMPANION RED-PROOF (observed): drop `c.Waiting != ""` from buildWaiting's test → "kernel not ready: cards [...]" (a card before the rule allows it). +func TestSystemPage_WaitingCards(t *testing.T) { + ring0 := []string{"demo-felhom-a1b2c3", "demo-hp-d4e5f6"} + pk := func(fp string) []osupdates.Package { + return map[string][]osupdates.Package{ + "k": {{Name: "proxmox-kernel-7.0.14-23-pve-signed", Version: "7.0.14-23"}}, + "d": {{Name: "docker-ce", Version: "5:29.8.3-1"}}, + "p": {{Name: "pve-manager", Version: "9.0.12"}}, + }[fp] + } + two := func(string, string, time.Time) int { return 2 } + for _, c := range []struct { + layer, fp, what, action, evidence string + }{ + {"kernel", "k", "Kernel 7.0.14-23", "/os/approve-kernel", "Started without problems after a night step on demo-felhom-a1b2c3 and demo-hp-d4e5f6."}, + {"docker", "d", "Docker engine 29.8.3-1", "/os/approve-docker", "demo-hp-d4e5f6: 2 healthy night(s)"}, + {"pve", "p", "Proxmox packages 9.0.12", "/os/approve-pve", "demo-felhom-a1b2c3: 2 healthy night(s)"}, + } { + cards, testing := buildWaiting([]osupdates.Status{{Layer: c.layer, Fingerprint: c.fp, FirstSeen: fixNow.Add(-26 * time.Hour), Packages: 2}}, ring0, pk, two, fixNow) + if len(cards) != 1 || len(testing) != 0 || cards[0].What != c.what || cards[0].Action != c.action || !strings.Contains(cards[0].Evidence, c.evidence) { + t.Errorf("%s: cards %+v testing %+v", c.layer, cards, testing) + } + // Not ready yet → a "still being tested" line, no card. + cards, testing = buildWaiting([]osupdates.Status{{Layer: c.layer, Fingerprint: c.fp, Waiting: "needs another night"}}, ring0, pk, two, fixNow) + if len(cards) != 0 || len(testing) != 1 || testing[0].Why != "needs another night" { + t.Errorf("%s not ready: cards %+v testing %+v", c.layer, cards, testing) + } + // Approved → neither. + cards, testing = buildWaiting([]osupdates.Status{{Layer: c.layer, Fingerprint: c.fp, Approved: "os-x", Waiting: "already approved"}}, ring0, pk, two, fixNow) + if len(cards)+len(testing) != 0 { + t.Errorf("%s approved: cards %+v testing %+v", c.layer, cards, testing) + } + } + if cards, _ := buildWaiting([]osupdates.Status{{Layer: "guest", Fingerprint: "g", Packages: 3}, {Layer: "host", Fingerprint: "h", Packages: 3}}, ring0, pk, two, fixNow); len(cards) != 0 { + t.Errorf("guest/host became cards: %+v", cards) + } + // Rendered: the card with its button, and the empty line. + page := renderSystem(t, fixtureInput(true)) + w := sysSection(page, "waiting") + if !strings.Contains(w, "Proxmox packages 9.0.12") || !strings.Contains(w, "Approve Proxmox set") { + t.Errorf("the ready card is not rendered:\n%s", w) + } + if !strings.Contains(w, "Docker engine 29.8.3-1") || !strings.Contains(w, "still being tested") { + t.Errorf("the Docker set still being tested is not shown:\n%s", w) + } + if w = sysSection(renderSystem(t, fixtureInput(false)), "waiting"); !strings.Contains(w, "Nothing waits for your approval.") || strings.Contains(w, `); a no-op otherwise. +func TestSystemPage_WriteFixture(t *testing.T) { + dir := os.Getenv("SYSTEM_PAGE_FIXTURE_OUT") + if dir == "" { + t.Skip("set SYSTEM_PAGE_FIXTURE_OUT to write the fixture pages") + } + css, err := os.ReadFile("templates/style.css") + if err != nil { + t.Fatal(err) + } + fonts, err := filepath.Abs("static/fonts") + if err != nil { + t.Fatal(err) + } + css = bytes.ReplaceAll(css, []byte("url('/static/fonts/"), []byte("url('file://"+fonts+"/")) + link := regexp.MustCompile(``) + for name, ready := range map[string]bool{"system-fixture.html": false, "system-fixture-card.html": true} { + page := link.ReplaceAllString(renderSystem(t, fixtureInput(ready)), "") + if err := os.WriteFile(filepath.Join(dir, name), []byte(page), 0o644); err != nil { + t.Fatal(err) + } + } +} diff --git a/hub/internal/web/system_trim_test.go b/hub/internal/web/system_trim_test.go index 1eb8c1c9..38de998c 100644 --- a/hub/internal/web/system_trim_test.go +++ b/hub/internal/web/system_trim_test.go @@ -66,7 +66,7 @@ func TestSystemPage_LastDiskTrim(t *testing.T) { t.Fatal(err) } b := getSystem(t, s) - if !strings.Contains(b, ">Last disk trim") { + if !strings.Contains(b, ">Last disk trim") { t.Error("the System page lacks the Last disk trim column") } if !strings.Contains(b, "20 days ago · 30.2 GiB") { diff --git a/hub/internal/web/system_view.go b/hub/internal/web/system_view.go new file mode 100644 index 00000000..78f93eec --- /dev/null +++ b/hub/internal/web/system_view.go @@ -0,0 +1,344 @@ +package web + +import ( + "fmt" + "sort" + "strings" + "time" + + "gitea.dooplex.hu/admin/felhom-hub/internal/osupdates" + "gitea.dooplex.hu/admin/felhom-hub/internal/store" + "gitea.dooplex.hu/admin/felhom-hub/internal/sysfacts" +) + +// The System page's layout (2026-10-10): it answers "is anything wrong?" (Needs attention) and "is anything waiting for +// me?" (Waiting for you) first, then the boxes in one narrow table whose rows open to every value the old wide table +// showed, and folds the rest (release ids, cancelled approvals, ring-0 package counts, floors, the crash guard and root +// files) into a closed "Details". The SAME data, buttons, routes and CSRF field as before — only the view changed. +// The contract (every old item still on the page) is pinned by TestSystemPage_EveryItemStillOnThePage. + +// reason is one plain-words line of a box's "Needs attention" entry; its class is the cell's colour. +type reason struct { + Class, Text, Title string +} + +// systemInput is everything the page shows, read by the handler (or written by a test fixture). buildSystemPage is pure. +type systemInput struct { + Lines []osupdates.FleetLine + Facts map[string]sysfacts.System + Names, Controllers, Agents map[string]string // by host id + KernelLines map[string]osupdates.KernelLine + BundleSince, AgentSince map[string]time.Time + Stale, Reboot, NotCov time.Duration + BundleAfter, AgentAfter time.Duration + VouchedAgent, VouchedBundle string + GlobalFloor string + Floors []store.CustomerFloorOverride + Releases, Cancelled []osupdates.ReleaseInfo + Candidates []osupdates.Status + Packages func(fingerprint string) []osupdates.Package + Nights func(hostID, layer string, since time.Time) int + Now time.Time +} + +func buildSystemPage(in systemInput) map[string]interface{} { + rows := buildSystemRows(in.Lines, in.Facts, in.Names, in.Stale, in.Reboot, in.NotCov, in.Now) + latest := map[string]string{} + for _, rel := range in.Releases { + latest[rel.Layer] = rel.ID + } + var attention []systemRow + for i := range rows { + id := rows[i].HostID + rows[i].KernelDefault, rows[i].KernelStep = kernelCells(in.Facts[id], in.KernelLines[id], in.Now) + rows[i].Bundle = bundleCell(in.Facts[id], in.VouchedAgent, in.VouchedBundle, in.BundleSince[id], in.BundleAfter, in.Now) + rows[i].Agent = agentCell(in.Agents[id], in.VouchedAgent, in.AgentSince[id], in.AgentAfter, in.Now) + rows[i].Controller = unknownCell(in.Controllers[id]) + summariseRow(&rows[i], latest) + if rows[i].Mark.Class != "" { + attention = append(attention, rows[i]) + } + } + var ring0 []string + for _, l := range in.Lines { + if l.Ring == 0 && l.Enabled { + ring0 = append(ring0, l.HostID) + } + } + sort.Strings(ring0) + cards, testing := buildWaiting(in.Candidates, ring0, in.Packages, in.Nights, in.Now) + return map[string]interface{}{ + "Rows": rows, + "Attention": attention, + "Cards": cards, + "Testing": testing, + "GlobalFloor": in.GlobalFloor, + "VouchedAgent": in.VouchedAgent, + "Floors": buildFloorRows(in.Floors, in.GlobalFloor, in.Now), + "Releases": in.Releases, + "Cancelled": in.Cancelled, + "Candidates": in.Candidates, + } +} + +// labelled names every coloured per-box cell the way "Needs attention" says it. A new cell that can turn amber or red +// is added here, or the box's mark would miss it. +func (r *systemRow) labelled() []struct { + label string + c cell +} { + type lc = struct { + label string + c cell + } + return []lc{ + {"tunnel", r.Tunnel}, {"Proxmox version", r.PVE}, {"running kernel", r.KernelRunning}, {"next-boot kernel", r.KernelNextBoot}, + {"default kernel", r.KernelDefault}, {"kernel step", r.KernelStep}, {"host Debian", r.HostDebian}, + {"host updates not covered", r.HostNotCovered}, {"held packages", r.Held}, {"host restart", r.RebootSince}, + {"kernel.panic", r.KernelPanic}, {"kernel oops", r.Oops}, {"crash restarts", r.CrashRestarts24h}, {"crash guard", r.Guard}, + {"root files", r.Bundle}, {"agent", r.Agent}, {"controller", r.Controller}, {"guest Debian", r.GuestDebian}, + {"Docker engine", r.Engine}, {"containerd", r.Containerd}, {"Docker live-restore", r.LiveRestore}, + {"disk trim", r.Trim}, {"last OS run", r.LastLeg}, + } +} + +func phrase(label string, c cell) string { + switch label { + case "tunnel": + return "tunnel " + strings.ReplaceAll(c.Text, "_", " ") + case "next-boot kernel": + return "the next boot changes the kernel to " + c.Text + case "default kernel": + return "a one-shot boot is set: " + c.Text + case "kernel step": + return "kernel step: " + strings.ReplaceAll(c.Text, "_", " ") + case "host updates not covered": + return c.Text + " host update(s) that no approved release covers" + case "held packages": + return "packages held by hand: " + c.Text + case "host restart": + return "the host needs a restart " + c.Text + case "kernel.panic": + return "kernel.panic is 0: a crashed box stays off" + case "kernel oops": + return "a kernel oops this boot" + case "crash restarts": + return c.Text + " crash restart(s) in the last 24 h" + case "crash guard": + if strings.HasPrefix(c.Text, "TRIPPED") { + return "crash guard tripped: the next crash leaves the box off" + } + return "crash guard " + c.Text + case "root files": + if strings.Contains(c.Text, "changed by hand") { + return "root files changed by hand" + } + return "root files behind the vouched agent's" + case "agent": + return "agent behind: " + c.Text + case "Docker live-restore": + return "Docker live-restore is off: a Docker step is refused" + case "disk trim": + return "disk trim: " + c.Text + case "last OS run": + if c.Class == "bad" { + return "no successful OS update run for 7 days or more" + } + return "the last OS update run did not succeed: " + c.Text + } + return label + ": " + c.Text +} + +func worse(a, b string) string { + if a == "bad" || b == "bad" { + return "bad" + } + if a == "warn" || b == "warn" { + return "warn" + } + return "" +} + +// summariseRow fills the narrow table's cells and the box's mark and reasons from the cells buildSystemRows and the +// handler computed — the colours are the same thresholds (`08` §6.3), never a second definition. +func summariseRow(r *systemRow, latestRelease map[string]string) { + var reasons []reason + var unknown []string + mark := "" + if !r.Enabled { + reasons = append(reasons, reason{Class: "warn", Text: "OS updates switched off"}) + mark = "warn" + } + if r.FactsNote != "" { + reasons = append(reasons, reason{Class: "warn", Text: r.FactsNote}) + mark = worse(mark, "warn") + } + for _, x := range r.labelled() { + if x.c.Class == "" { + continue + } + mark = worse(mark, x.c.Class) + if x.c.Text == "unknown" { + unknown = append(unknown, x.label) + continue + } + reasons = append(reasons, reason{Class: x.c.Class, Text: phrase(x.label, x.c), Title: x.c.Title}) + } + if len(unknown) > 0 && r.HasFacts { + reasons = append(reasons, reason{Class: "warn", Text: "could not read: " + strings.Join(unknown, ", "), + Title: "the box could not read these values (agent older than v0.142.0, or the guest is down) — never a guess"}) + } + sort.SliceStable(reasons, func(i, j int) bool { return reasons[i].Class == "bad" && reasons[j].Class != "bad" }) + r.Reasons = reasons + switch mark { + case "bad": + r.Mark = cell{Text: "alarm", Class: "bad", Title: "an operator alarm fires for this box"} + case "warn": + r.Mark = cell{Text: "look", Class: "warn", Title: "worth a look"} + default: + r.Mark = cell{Text: "fine", Title: "nothing amber or red"} + } + + // OS updates: on/off, and whether the box runs the newest approved guest and host releases. + if !r.Enabled { + r.Updates = cell{Text: "off", Class: "warn", Title: "the box keeps reporting and installs nothing"} + } else { + var behind []string + for _, l := range []struct{ layer, has string }{{osupdates.LayerGuest, r.GuestRelease.Text}, {osupdates.LayerHost, r.HostRelease.Text}} { + if want := latestRelease[l.layer]; want != "" && l.has != want { + behind = append(behind, l.layer) + } + } + if len(behind) == 0 { + r.Updates = cell{Text: "on · up to date", Title: "runs the newest approved guest and host releases"} + } else { + r.Updates = cell{Text: "on · " + strings.Join(behind, " + ") + " behind", + Title: "not on the newest approved release yet — a box takes it at its next night run"} + } + } + r.KernelShort = r.KernelRunning + if r.KernelNextBoot.Class != "" && r.KernelNextBoot.Text != "unknown" { + r.KernelNext = r.KernelNextBoot.Text + } +} + +// lastNightCell is the narrow table's "Last night": the newest OS run in a word (ok / failed / skipped) and when. +func lastNightCell(enabled bool, outcome string, at time.Time, leg cell, now time.Time) cell { + c := cell{Title: leg.Text + " — " + leg.Title} + switch { + case !enabled: + c.Text = "skipped (updates off)" + case outcome == "": + c.Text = "no run reported" + case outcome == "applied" || outcome == "nothing": + c.Text = "ok · " + ago(at, now) + case outcome == "failed" || outcome == "health_failed" || outcome == "refused": + c.Text, c.Class = strings.ReplaceAll(outcome, "_", " ")+" · "+ago(at, now), "warn" + default: + c.Text = strings.ReplaceAll(outcome, "_", " ") + " · " + ago(at, now) + } + if leg.Class == "bad" { + c.Class = "bad" + c.Text += " · no success for 7+ days" + } + return c +} + +// waitCard is one "Waiting for you" card: a set the operator's button may approve now. +type waitCard struct { + Layer, What, Evidence, FirstSeen, After string + Action, Button, Confirm string +} + +// testingLine is a set ring 0 runs that is not ready for approval yet, with the hub's own reason. +type testingLine struct { + What, Why string +} + +type lane struct { + name, pkg, action, button, confirm, after string +} + +// lanes are the OPERATOR-approved sets (guest and host approve themselves). The button, route and question are the +// ones the page had before 2026-10-10. +var lanes = map[string]lane{ + osupdates.LayerKernel: {"Kernel", "proxmox-kernel-", "/os/approve-kernel", "Approve kernel set", + "Approve this kernel set? Every ring-0 box booted it healthily after a night step. Ring-1 boxes take it only through a signed os_kernel_step, and restart only on a night their household was told about.", + "Approving installs nothing by itself: a ring-1 box takes it only through a signed kernel step, and restarts only on a night its household was told about."}, + osupdates.LayerDocker: {"Docker engine", "docker-ce", "/os/approve-docker", "Approve Docker set", + "Approve this Docker engine set? Ring-1 boxes take it only through a signed operator job.", + "Approving installs nothing by itself: a ring-1 box takes it only through a signed operator job."}, + osupdates.LayerPVE: {"Proxmox packages", "pve-manager", "/os/approve-pve", "Approve Proxmox set", + "Approve this Proxmox package set (no kernel)? Ring-1 boxes take it only through a signed operator job.", + "Approving installs nothing by itself: a ring-1 box takes it only through a signed operator job."}, + osupdates.LayerGuest: {name: "Guest Debian updates"}, + osupdates.LayerHost: {name: "Host Debian updates"}, +} + +func setVersion(ln lane, pkgs []osupdates.Package, count int) string { + if ln.pkg != "" { + for _, p := range pkgs { + if p.Name == ln.pkg || (strings.HasSuffix(ln.pkg, "-") && strings.HasPrefix(p.Name, ln.pkg)) { + v := p.Version + if i := strings.Index(v, ":"); i >= 0 && i < 3 { + v = v[i+1:] // a Debian epoch ("5:29.8.2-1") is not the version a person reads + } + if i := strings.Index(v, "~"); i > 0 { + v = v[:i] // nor is the distribution suffix ("~debian.13~trixie") + } + return ln.name + " " + v + } + } + } + return fmt.Sprintf("%s (%d packages)", ln.name, count) +} + +// buildWaiting splits ring 0's candidate sets into cards (the button may approve now — the SAME gate the page's +// buttons always had: a set, not yet approved, nothing waiting) and lines still being tested. +func buildWaiting(cands []osupdates.Status, ring0 []string, pkgsOf func(string) []osupdates.Package, + nights func(string, string, time.Time) int, now time.Time) ([]waitCard, []testingLine) { + var cards []waitCard + var testing []testingLine + for _, c := range cands { + ln, known := lanes[c.Layer] + if !known || c.Fingerprint == "" || c.Approved != "" { + continue + } + var pkgs []osupdates.Package + if pkgsOf != nil { + pkgs = pkgsOf(c.Fingerprint) + } + what := setVersion(ln, pkgs, c.Packages) + if c.Waiting != "" || ln.action == "" { + why := c.Waiting + if why == "" { + why = "the hub approves it by itself" + } + testing = append(testing, testingLine{What: what, Why: why}) + continue + } + card := waitCard{Layer: c.Layer, What: what, After: ln.after, Action: ln.action, Button: ln.button, Confirm: ln.confirm, + FirstSeen: "first seen " + ago(c.FirstSeen, now) + " (" + c.FirstSeen.UTC().Format("2006-01-02 15:04") + " UTC)"} + switch { + case len(ring0) == 0: + card.Evidence = "no ring-0 box" + case c.Layer == osupdates.LayerKernel: + card.Evidence = "Started without problems after a night step on " + strings.Join(ring0, " and ") + "." + default: + var per []string + for _, h := range ring0 { + n := 0 + if nights != nil { + n = nights(h, c.Layer, c.FirstSeen) + } + per = append(per, fmt.Sprintf("%s: %d healthy night(s)", h, n)) + } + card.Evidence = "Ran on every ring-0 box — " + strings.Join(per, ", ") + "." + if c.Layer == osupdates.LayerDocker { + card.Evidence += " The memory-kill check passed." + } + } + cards = append(cards, card) + } + return cards, testing +} diff --git a/hub/internal/web/templates/system.html b/hub/internal/web/templates/system.html index 8cf9e551..2da88321 100644 --- a/hub/internal/web/templates/system.html +++ b/hub/internal/web/templates/system.html @@ -6,12 +6,65 @@ System — Felhom Hub @@ -37,8 +90,138 @@ {{if .Flash}}
{{.Flash}}
{{end}} {{if .FlashErr}}
{{.FlashErr}}
{{end}} -
-

Approved releases

+
+

Needs attention

+

One line per box that is amber or red, and why. Amber: worth a look. Red: an operator alarm fires (`08` §6.3).

+ {{if .Rows}} + {{if .Attention}} +
    + {{range .Attention}} +
  • {{.Mark.Text}} + {{.HostID}}{{if .CustomerName}} {{.CustomerName}}{{end}} + {{range $i, $r := .Reasons}}{{if $i}} · {{end}}{{$r.Text}}{{end}}
    • + {{end}} +
+ {{else}}

All boxes look fine.

{{end}} + {{else}}

No boxes yet.

{{end}} +
+ +
+

Waiting for you

+

Update sets that only you approve: the kernel, the Docker engine and the Proxmox packages. Guest and host Debian updates approve themselves after 24 h and one night on the ring-0 boxes.

+ {{if .Cards}} +
+ {{range .Cards}} +
+

{{.What}}

+

{{.Evidence}}

+

{{.FirstSeen}}. {{.After}}

+
+ + +
+
+ {{end}} +
+ {{else}}

Nothing waits for your approval.

{{end}} + {{if .Testing}} +
    + {{range .Testing}}
  • {{.What}} — still being tested: {{.Why}}
    • {{end}} +
+ {{end}} +
+ + {{if .Rows}} +
+

Boxes

+

Click a box to see every value it reports, and its ring and update switches. "unknown": the box could not read the value — never a guess.

+
Box
Ring
Health
Controller · agent
OS updates
Kernel
Last night
+ {{range .Rows}} +
+ +
{{.HostID}}{{if .CustomerName}}{{.CustomerName}}{{end}}
+
{{.Ring}}
+
{{.Mark.Text}}
+
{{.Controller.Text}}agent {{.Agent.Text}}
+
{{.Updates.Text}}
+
{{.KernelShort.Text}}{{if .KernelNext}}next boot: {{.KernelNext}}{{end}}
+
{{.LastNight.Text}}
+ +
+
+ {{if .FactsNote}}

{{.FactsNote}}

{{end}} +
Ring and updates
+
+
Ring
ring {{.Ring}} +
+ + {{if eq .Ring 0}} + {{else}}{{end}} +
+
OS updates
updates {{if .Enabled}}ON{{else}}OFF{{end}} +
+ + {{if .Enabled}} + {{else}}{{end}} +
+
Tunnel
{{template "sys_dd" .Tunnel}} +
Controller
{{template "sys_dd" .Controller}} +
Agent
{{template "sys_dd" .Agent}} +
Last OS leg
{{template "sys_dd" .LastLeg}} +
+
Docker engine
+
+
Docker
{{template "sys_dd" .Engine}} +
containerd
{{template "sys_dd" .Containerd}} +
live-restore
{{template "sys_dd" .LiveRestore}} +
Docker release
{{template "sys_dd" .DockerRelease}} +
+
+
+
Host
+
+
Proxmox
{{template "sys_dd" .PVE}} +
Kernel (running)
{{template "sys_dd" .KernelRunning}} +
Kernel (next boot)
{{template "sys_dd" .KernelNextBoot}} +
Kernel (default)
{{template "sys_dd" .KernelDefault}} +
Kernel step
{{template "sys_dd" .KernelStep}} +
Debian
{{template "sys_dd" .HostDebian}} +
Felhom release
{{template "sys_dd" .HostRelease}} +
Pending
{{template "sys_dd" .HostPending}} +
Not covered
{{template "sys_dd" .HostNotCovered}} +
Held
{{template "sys_dd" .Held}} +
Reboot needed
{{template "sys_dd" .RebootSince}} +
kernel.panic
{{template "sys_dd" .KernelPanic}} +
Oops
{{template "sys_dd" .Oops}} +
Crash restarts 24 h
{{template "sys_dd" .CrashRestarts24h}} +
Crash guard
{{template "sys_dd" .Guard}} +
Root files
{{template "sys_dd" .Bundle}} +
+
+
+
Guest
+
+
Guest Debian
{{template "sys_dd" .GuestDebian}} +
Felhom release
{{template "sys_dd" .GuestRelease}} +
Pending
{{template "sys_dd" .GuestPending}} +
Restart needed
{{template "sys_dd" .GuestRestart}} +
Last disk trim
{{template "sys_dd" .Trim}} +
+
+
+
+ {{end}} +
+ {{else}} +

No boxes yet.

+ {{end}} + +
+ Details +

Release ids, cancelled approvals, what ring 0 runs, the version floors, and every box's crash guard and root files.

+ +

Approved releases

+

The newest approved set of each layer. A TEST approval is amber.

{{range .Releases}}
{{.Layer}}: {{.ID}}
@@ -46,6 +229,11 @@ {{if .Test}}
TEST approval{{end}}
{{else}}
No release approved yet.
{{end}}
+
+ + + An urgent fix only — normally the hub approves after 24 h and one night. +
{{if .Cancelled}}

Cancelled approvals (last 7 days)

@@ -56,42 +244,24 @@ {{end}}
{{end}} +

What ring 0 runs now

{{range .Candidates}}
{{.Layer}}: {{if .Fingerprint}}{{.Packages}} packages, first seen {{.FirstSeen.UTC.Format "2006-01-02 15:04"}} UTC{{else}}—{{end}}
{{if .Approved}}approved as {{.Approved}} - {{else if .Waiting}}{{.Waiting}}{{end}} - {{if and (eq .Layer "docker") .Fingerprint (not .Approved) (eq .Waiting "")}} -
- - -
{{end}} - {{if and (eq .Layer "pve") .Fingerprint (not .Approved) (eq .Waiting "")}} -
- - -
{{end}} - {{if and (eq .Layer "kernel") .Fingerprint (not .Approved) (eq .Waiting "")}} -
- - -
{{end}} + {{else if .Waiting}}{{.Waiting}} + {{else if .Fingerprint}}ready — see "Waiting for you"{{end}}
{{end}}
-
- - - An urgent fix only — normally the hub approves after 24 h and one night. -
-
-
-

Version floors

+

Version floors

Global controller floor: {{if .GlobalFloor}}{{.GlobalFloor}}{{else}}none{{end}} · vouched agent: {{if .VouchedAgent}}{{.VouchedAgent}}{{else}}none{{end}}

+

What is a floor? · What is a vouched agent?

{{if .Floors}} +
@@ -105,66 +275,33 @@ {{end}}
CustomerOwn floorSetGlobal floor moves it?
+
{{else}}

No per-customer floors: every box follows the global floor.

{{end}} -
- {{if .Rows}} -
+ {{if .Rows}} +

Crash guard and root files

+
- - - - - - - - - - + {{range .Rows}} - - - - {{template "sys_cell" .Tunnel}} - - {{template "sys_cell" .KernelRunning}}{{template "sys_cell" .KernelNextBoot}}{{template "sys_cell" .KernelDefault}}{{template "sys_cell" .KernelStep}}{{template "sys_cell" .HostDebian}} - {{template "sys_cell" .HostRelease}}{{template "sys_cell" .HostPending}}{{template "sys_cell" .HostNotCovered}} - {{template "sys_cell" .Held}}{{template "sys_cell" .RebootSince}}{{template "sys_cell" .KernelPanic}}{{template "sys_cell" .Oops}} - {{template "sys_cell" .CrashRestarts24h}}{{template "sys_cell" .Guard}}{{template "sys_cell" .Bundle}}{{template "sys_cell" .Agent}} - - {{template "sys_cell" .GuestRelease}}{{template "sys_cell" .GuestPending}}{{template "sys_cell" .GuestRestart}}{{template "sys_cell" .Trim}} - - {{template "sys_cell" .Containerd}}{{template "sys_cell" .LiveRestore}}{{template "sys_cell" .DockerRelease}} - - + {{template "sys_cell" .KernelPanic}}{{template "sys_cell" .Oops}}{{template "sys_cell" .CrashRestarts24h}}{{template "sys_cell" .Guard}}{{template "sys_cell" .Bundle}} {{end}}
BoxRing / updatesTunnelProxmoxKernel (running)Kernel (next boot)Kernel (default)Kernel stepDebianFelhom releasePendingNot coveredHeldReboot neededkernel.panicOopsCrash restarts 24 hCrash guardRoot filesAgentGuest DebianFelhom releasePendingRestart neededLast disk trimDockercontainerdlive-restoreDocker releaseLast OS leg
hostguestDocker engine
Boxkernel.panicOopsCrash restarts 24 hCrash guardRoot files
{{.HostID}}{{if .CustomerName}}
{{.CustomerName}}{{end}} - {{if .FactsNote}}
{{.FactsNote}}{{end}}		 - ring {{.Ring}} -
- - {{if eq .Ring 0}} - {{else}}{{end}} -

- updates {{if .Enabled}}ON{{else}}OFF{{end}} -
- - {{if .Enabled}} - {{else}}{{end}} -
- 		{{.PVE.Text}}{{.GuestDebian.Text}}{{.Engine.Text}}{{.LastLeg.Text}}
{{.HostID}}
-
-

Amber: worth a look. Red: an operator alarm fires (`08` §6.3). "unknown": the box could not read the value — never a guess.

- {{else}} -

No boxes yet.

- {{end}} + + {{end}} +
Felhom Hub {{hubVersion}}
+ {{define "sys_cell"}}{{.Text}}{{end}} +{{define "sys_dd"}}{{.Text}}{{end}}