slice 9: GET /host/metrics + CPU/chassis-temp collector (v0.14.0)

Add a host-wide, token-authed GET /host/metrics local-API endpoint that
re-serves the slice-4 collector's host + per-storage view to the customer
(the de-privileged controller can't read the host itself). Add the one new
collector — CPU/chassis temperature via sysfs hwmon/thermal-zones, graceful-
null — to the shared HostMetrics struct, so the hub report carries cpu_temp_c
too. Cross-repo host-report golden updated byte-identical.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 16:16:03 +02:00
parent 9a0e7e168b
commit aa4dfb75ea
13 changed files with 664 additions and 55 deletions
+43 -43
View File
@@ -1,53 +1,53 @@
# REPORT — slice 8B.2 (agent half): quiesce downtime optimization — `snapshotted` phase (v0.13.0) (2026-06-10)
# REPORT — slice 9 (agent half): host metrics to the controller (v0.14.0) (2026-06-10)
> Overwrite-latest report. Cumulative history: [CHANGELOG.md](CHANGELOG.md). Implements the agent
> half of `TASK — Slice 8B.2`. The controller early-resume is `felhom-controller` v0.38.0. No hub change.
> Overwrite-latest report. Cumulative history: [CHANGELOG.md](CHANGELOG.md).
## Outcome
## What was implemented
In snapshot mode an LXC vzdump only needs the app-stopped state captured at the **storage-snapshot
moment**; after that it reads from the snapshot and the app can safely resume. The agent now emits a
**`snapshotted`** phase on `GET /backup/status` when the snapshot is taken, so the controller resumes
its app early — app downtime drops from *whole-backup* to *until-snapshot* with no loss of
app-consistency.
The agent half of **slice 9** — re-serving the host's health to the customer's controller, plus the
one new collector (CPU/chassis temperature). The de-privileged controller (slice 8C) sees only its
own cgroup, so it cannot read host metrics itself; the agent already collects host CPU/mem/loadavg/
uptime + per-storage targets for the hub, and slice 9 exposes that to the customer over the local API.
## Phase 0 (validated first, on the demo — PVE 9.2.2)
### `internal/hub/cputemp.go` — CPU/chassis-temp collector (new)
- `TempReader` seam + `SysfsTempReader`: reads the CPU package temperature from **sysfs** — hwmon
(`coretemp`/`k10temp`/`zenpower`/`cpu_thermal`, preferring the `Package id 0` input) then thermal
zones (preferring `x86_pkg_temp`/`coretemp`/`cpu-thermal`, falling back to `acpitz`).
- **No external binary, no privilege** (sysfs is world-readable) → the root-CLI fence is untouched.
- **Graceful-null**: a missing sensor, an unsupported board, an implausible reading (outside
5150 °C), or any read error all degrade to `nil` ("n/a") — never fails the report. Same nullable
contract as the per-disk `SmartSummary.TemperatureC`.
- **Snapshot mode is in effect** on local-lvm (lvmthin) — `vzdump --mode snapshot` logs `backup mode:
snapshot`, not downgraded to stop.
- **Marker:** `INFO: create storage snapshot 'vzdump'` — the vzdump-level step after which the backup
reads from the snapshot (generic across snapshot-capable storage; only appears in snapshot mode).
- **Downtime delta:** snapshot created in the first ~1s; archive runs ~23s after. So 8B (resume at
`done`) = ~24s app-down; 8B.2 (resume at `snapshotted`) = ~1s — **~95% cut** for a 934 MB guest;
the delta grows with guest size.
### `internal/hub` — shared wire field + collector reuse
- `HostMetrics` gains **`CPUTempC *int` (`cpu_temp_c`)** — nullable, on the **shared** struct, so the
**hub report carries it too** (operator freebie). Cross-repo host-report golden updated
**byte-identical** with the hub's copy.
- `Collector` gains a nil-safe `temp TempReader` (defaults to the real `SysfsTempReader`;
`SetTempReader` injects a fake in tests). `Collect()` now sets `cpu_temp_c` on the report.
- **`Collector.HostMetricsNow(ctx)`** — a fresh `NodeStatus` + CPU-temp read returning just the host
block; the source for the local API (current cpu%/temp, not the 15-min hub snapshot).
## What landed
### `internal/localapi` — `GET /host/metrics` (new endpoint)
- `host_metrics.go`: host-wide health (cpu%/mem/load/uptime/`cpu_temp_c`) + per-storage capacity
(total/used/fraction, thin-pool, SMART temp+wear). Token-authed via `withGuest` (host-wide data; a
cross-guest `?vmid=` still 403). Best-effort on storage (a view error still returns the host
block). Served only when the `HostMetrics` provider (the shared collector) is wired in
`buildLocalAPIServer` — else 503 "not configured".
- **`BackupRunner.BackupWithSnapshotHook(ctx, vmid, onSnapshot)`** — while the vzdump runs, a watcher
tails the task log (`TaskLogTail`) for the `create storage snapshot` marker and fires `onSnapshot`
**once**. It bails on `backup mode: stop` and the marker never appears in stop mode, so it **never
fires in stop/downgraded mode**. `Backup` keeps its signature (scheduler/selftest); both share one
body. `snapshotWatchInterval` is a package var (prod 1s; tests shrink it).
- **`/backup/status` phase `snapshotted`** (between `running` and `done`): `handleBackup` passes the
hook → `markSnapshotted` flips the running job. `done`/`failed` unchanged; `snapshotted` is additive.
## Tests (all green)
- `cputemp_test.go`, `hostmetrics_test.go`, `host_metrics_test.go`: hwmon/thermal-zone selection +
**graceful-null**, `HostMetricsNow` populate/null/hard-error, endpoint populated + `cpu_temp_c:null`
serialization + **401 without a token** + 403 cross-guest + 503 not-configured.
- `go test ./...` green; `go vet ./internal/hub ./internal/localapi` clean.
## Tests
## Versioning / docs
- Version `0.13.0 → 0.14.0`; `CHANGELOG.md` updated. Doc 03 §6 (local-API surface) + §9 (roadmap +
changelog) updated.
`go test ./...` green; `-race` green (build server). localapi: snapshot mode → `snapshotted` before
`done`; stop mode → `snapshotted` never emitted. runner: the watcher fires on the marker; stop-mode
log never fires.
## Assumption (noted, not built)
- **One customer per host** (home-server model): `/host/metrics` is host-wide. A multi-customer host
would leak cross-customer CPU/mem → revisit then.
## Live validation (demo-felhom)
End-to-end on a provisioned guest (controller v0.38.0 + a postgres stack): the agent logged `backup
reached snapshotted (app may resume)` mid-backup, the controller resumed the app at that point, and
the backup proceeded to `done`. **App downtime ~3s** (quiesce→snapshotted) vs **~23s** if it had
waited for `done` (~87% cut). The snapshot backup restored **clean** (`database system was shut down`,
no WAL replay) — the early resume did not compromise app-consistency. Deployed to the demo service as
v0.13.0.
## Deferred / dependency
Snapshot-capable storage (lvm-thin/ZFS) is required for the win; on stop/downgraded storage the
controller falls back to resume-at-`done` (8B). No consistency-contract or crash-safety change. No
secrets committed.
## Pending
- **Live validation** on the demo (build + deploy agent v0.14.0; controller monitoring page → real
N100 CPU%/temp + per-storage, cross-checked vs `pvesh`/`free`/`df`).