docs(report): guest_net confirmed on the wire hub-side; STOP-3 done; damping state is in-memory only
This commit is contained in:
@@ -230,14 +230,40 @@ One bad probe, **no action**, then recovery. The two-consecutive-probes rule and
|
||||
did exactly what they exist for — a booting guest was not injected with a dhclient — and this was a
|
||||
genuine transient, not a scripted one.
|
||||
|
||||
## 6c. What could NOT be verified here
|
||||
## 6c. The stanza ON THE WIRE — confirmed hub-side after STOP-3
|
||||
|
||||
The `guest_net` stanza **on the wire**. `--selftest=hub` builds its own one-shot collector and never
|
||||
wires the guestnet reporter, so it would print a misleading absence — using it as evidence would be
|
||||
worse than having none. (That divergence between the selftest collector and the daemon's is itself
|
||||
worth a small fix; the same comment already exists in the code for the pbs reporter.) The stanza is
|
||||
proven through the production `Collect` path by test, and `SetGuestNetReporter` is proven called by
|
||||
the AST wiring test, but seeing it in a real hub report needs the operator UI.
|
||||
Initially unverifiable here: `--selftest=hub` builds its own one-shot collector and never wires the
|
||||
guestnet reporter, so it would have printed a misleading absence — worse evidence than none. (That
|
||||
divergence between the selftest collector and the daemon's is worth a small fix; the same caveat is
|
||||
already commented in the code for the pbs reporter.)
|
||||
|
||||
Closed instead by a **read-only** query of the hub's own store (`kubectl cp` of `/data/hub.db`,
|
||||
opened `mode=ro`, copy deleted afterwards). The `guest_net` stanza is present, complete, and
|
||||
round-trips every field:
|
||||
|
||||
```json
|
||||
"guest_net": { "checked_at": "2026-07-21T11:32:10Z", "guests": [
|
||||
{ "vmid": 9201, "state": "healthy", "mode": "dhcp", "ip": "192.168.0.104",
|
||||
"has_route": true, "dhclient_alive": true, "checked_at": "2026-07-21T11:31:07Z",
|
||||
"message": "address, default route and dhclient all present" } ] }
|
||||
```
|
||||
|
||||
And the report history tells the whole story of this session from the hub's side:
|
||||
|
||||
| received (UTC) | agent | state | evidence |
|
||||
|---|---|---|---|
|
||||
| 10:30:27 | **0.92.0** | **`unknown`** | the sudoers-blind window — reported honestly as unknown, never as a false healthy and never as a false dead. The fail-safe is visible fleet-side, and this is independent confirmation that 0.92.0 was genuinely blind (so superseding it was right) |
|
||||
| 10:48:12 | 0.92.1 | `healthy` | **`last_heal_at=2026-07-21T10:45:12Z`, `heals_last_hour=1`** — the STOP-2 heal, surfaced to the hub |
|
||||
| 10:49:14 / 11:04:14 | 0.92.1 | `healthy` | same heal stamp, counter still 1 |
|
||||
| 11:32:10 | 0.92.1 | `healthy` | heal history absent — see the limitation below |
|
||||
|
||||
**A limitation this surfaced, worth stating plainly: the damping state is in-memory only.** The
|
||||
counters vanished from the 11:32 report because the agent was restarted at 11:17Z (the `log_level`
|
||||
revert), which resets `heals`/`lastHealAt`. So the "≥10 min apart, ≤3 per hour" ceilings hold within
|
||||
one agent lifetime, not across restarts. In practice the exposure is small — a restart also re-arms
|
||||
the 3-minute settle window and the two-consecutive-bad-probes rule — but a crash-looping agent could
|
||||
heal more often than the ceiling implies. Not worth persisting state for today; worth knowing before
|
||||
anyone quotes the ceiling as a hard guarantee.
|
||||
|
||||
## 7. Deliverables
|
||||
|
||||
@@ -250,9 +276,10 @@ the AST wiring test, but seeing it in a real hub report needs the operator UI.
|
||||
## 8. Operator actions outstanding
|
||||
|
||||
1. ~~STOP-2~~ — **DONE 2026-07-21, passed** (§6b).
|
||||
2. **STOP-3** — Day-0 manifest Agent → **0.92.1** / `7424bc1c3c533eff9157e15a18d4635c624931f5a479a48126de77a94e6a3d4d`
|
||||
(wrapper sha unchanged — `configs/felhom-pbs-apply` was not touched). MinAgent is the operator's
|
||||
call; 0.92.1 is safe once felhom-pve has converged, which it has.
|
||||
2. ~~STOP-3~~ — **DONE 2026-07-21.** Manifest Agent → **0.92.1**, sha matches the published artifact
|
||||
byte for byte; MinAgent → 0.92.1; PBS wrapper sha unchanged (`104db0a4…`, correct — the wrapper
|
||||
was not touched); controller floor → 0.156.0. The host page shows all four `guestnet-*`
|
||||
capability rows **ok**, which is the fleet-visible proof of the sudoers grant.
|
||||
3. ~~Revert `log_level` to `info`~~ — **DONE** (13:17 CEST, after the quiet window closed; agent
|
||||
restarted clean, caps `68/68 ok, degraded=0`, watchdog back up). The temporary raise is recorded
|
||||
here only so the journald volume change is explainable; `/root/agent.json.bak-debuglevel` remains
|
||||
|
||||
Reference in New Issue
Block a user