71 lines
7.7 KiB
Markdown
71 lines
7.7 KiB
Markdown
# Turning a Felhom machine off — Part A spike on the N100, 2026-10-13
|
||
|
||
By hand, no product change. Bench: N100 `demo-felhom` (host `felhom-pve`, guest 9201), on the Tuya meter plug
|
||
(`scripts/bench/plugmeter.py`, 2 s). Times are UTC. No key, device id or token here; the household's e-mail address in
|
||
the hub page captures is replaced by `<household e-mail>`.
|
||
|
||
## Results
|
||
|
||
| Step | What happened | File |
|
||
|---|---|---|
|
||
| A0 `plugswitch.py` | `off` refused while the N100 ran: „no standby_w" and „the machine answered a ping" (rc 2). First live `off` refused „only 16 s of readings" — slow plug reads shortened the window (a tool bug, fixed: read until the readings span 20 s); the second was allowed: 5 readings over 22 s at 0.0 W, no ping | `A4-plug.log` |
|
||
| A1 at rest | 10 min, 212 rows: **8.46 W mean, 7.2 W median**, sd 2.76, 6.3–19.1 W | `A1-rest.csv` |
|
||
| A1 at work | Immich deployed through the product (deploy 202 → running in 98 s), 200 photos uploaded (200/0), every job queue empty after 442 s: **20.7 W mean, 24.0 W busiest 60 s, 25.2 W max**. Removed through the product: stop 200, remove 200, no leftovers; photo set and bench folder deleted | `A1-work*.{csv,json,txt,log}` |
|
||
| A2 checks | no active Proxmox task, no backup/restore/update/swap line in the controller log for 20 min | `A2-before.txt` |
|
||
| A2 guest | `pct shutdown 9201 --timeout 120` at 16:43:35 → stopped, **12.3 s**, rc 0 | `A2-steps.log` |
|
||
| A2 host | `systemctl poweroff` at 16:43:53 → no ping 16:44:06 (13 s) → plug **0.0 W** from 16:44:15 | `A2-A4-power.csv` |
|
||
| A2 standby | **0.0 W** for 18 min, 254 readings (below the plug's 0.1 W step) | `A2-A4-power.csv` |
|
||
| A3 hub | 18 min after power-off: hosts page still **ONLINE**, cloudflared „running (connected)" (the last report), „Last Report 18 min ago", box connection „last connected 16:45 (16 min ago)". **No event, no notification** — the customer's last-10 list unchanged. Operator mailbox (a second channel): no mail for demo-felhom after 13:58; the 13:58 `host_crash_restart` mail shows the channel delivers | `A3-hub-*` |
|
||
| A3 stays off | no ping, 0.0 W for the whole 18 min | `A2-A4-power.csv` |
|
||
| A4 plug off/on | plug off 17:04:17, on 17:04:27. **The N100 did NOT start** (operator: no monitor at hand, BIOS not read) | `A4-plug.log` |
|
||
| A4 power button | operator pressed it: first watts 17:06:03 → ping 17:06:33 (30 s) → **dashboard 200 at 17:06:45 (42 s)**; boot peak 20.4 W | `A4-boot-*` |
|
||
| A4 after | crash guard `last_boot_unclean: false`; guest running, `onboot: 1`, mounts as before; the same 5 containers healthy; scheduler started with 22 jobs. Hub: report 2 min later, no event (no crash, nothing to recover) | `A4-after.txt` |
|
||
|
||
## What this means
|
||
|
||
- A clean guest shutdown + host power-off on command works, and costs nothing at the plug.
|
||
- **Today the hub does not notice a machine that is off for 18 minutes.** Code (05 §4, `hub.yaml` 45 m): `host_stale` at
|
||
45 min, `host_down` at 90 min; the controller checker `node_stale`/`node_down` too; and every morning at 05:00 the
|
||
`expected_backup_missed` / `expected_dbdump_missed` pair — the operator's mailbox shows exactly this pair for Tester 2,
|
||
which is off (2026-10-13 05:00).
|
||
- **Plug off/on did not start the N100**, but at 13:42 the same day it DID restart by itself after an unplanned power cut
|
||
(hub `host_crash_restart`). Consistent with the BIOS „after power loss = last state": a machine that was off stays off.
|
||
Not read in the BIOS (no monitor) — an inference. That is the behaviour the household text promises („press the power
|
||
button"), so the spike passes for the button; the plug cannot restart a deliberately-off N100 on the bench.
|
||
|
||
# Part B — the button, proven live on the N100, 2026-10-13
|
||
|
||
Released: agent v0.155.0 + config bundle (signed jobs), hub v0.149.0 (ArgoCD), controller v0.310.0 (hand-deployed on
|
||
guest 9201 only — the floor is the operator's). Method: endpoint-level, the exact `POST /api/server/poweroff` the
|
||
dashboard card calls (no browser on DooPlex). Hub read through the operator UI; mail read in the operator mailbox.
|
||
|
||
| Step | What happened | File |
|
||
|---|---|---|
|
||
| sudo on the box | real `sudo -l` checks on the N100: the 2 new lines allowed (`pct shutdown <id> --timeout <n>`, `systemctl poweroff`); 6 attack lines refused (`--force`, `-ff`, `--forceStop 1`, no timeout, `reboot`, `halt`) | `B-sudo-live-n100.txt` |
|
||
| sudo on the fleet | the same checks on demo-hp and Tester 1 (agent 0.155.0, bundle installed): the 2 lines allowed; `--force`, `--forceStop 1`, `reboot` refused | `B-sudo-live-demo-hp.txt`, `B-sudo-live-tester-1.txt` |
|
||
| refusals | no confirm word / a wrong word → 400; `GET /api/server/reboot` → 405 | session log |
|
||
| press | 18:52:34Z → **202** `powering_off: true`; no ping 18:53:02 (28 s); **0.0 W from 18:53:07** | `B-press.log`, `B-power.csv` |
|
||
| hub at once | event `host_deliberate_off` (info) 18:52; host page „Power: Kikapcsolva a háztartás által, 18:52 UTC — riasztás szünetel eddig: 2026-10-27 18:52 UTC" from 18:53 | `B-hub-watch.log` |
|
||
| hub at 33 min | controller staleness `ok → deliberately_off (household power-off, no event)` 19:25 | hub log |
|
||
| hub at 46 min | hosts badge **KIKAPCSOLVA** from 19:38 (before that: ONLINE plus the Power line — R-1002) | `B-hub-watch.log` |
|
||
| hub at 92–95 min | no `host_stale`, `host_down`, `node_stale`, `node_down` event; no notification in the customer's last 10 | `B-hub-*-at-9xmin.txt` |
|
||
| mailbox | no demo-felhom mail after 13:58 for the whole 2 h 56 min. Positive control: the HP power cut's `host_crash_restart` / `host_crash_guard_tripped` mails arrived 20:29 | operator mailbox |
|
||
| off | **0.0 W** for 2 h 56 min, 2346 readings, max 0.0 | `B-power.csv` |
|
||
| power button | operator pressed it: first watts 21:48:43 → boot 21:48:55 → **dashboard 200 by 21:49:33** (first check, ≤ 50 s) | `B-power.csv`, `B-after.txt` |
|
||
| hub clears | 21:49:22 `deliberate power-off ended (was 18:52:28Z)` → event `host_deliberate_off_ended` (info); 21:50 staleness `deliberately_off → ok`; badge ONLINE | hub log, `B-hub-watch.log` |
|
||
| after | crash guard `last_boot_unclean: false` (no crash mail, correct); guest 9201 running; controller 0.310.0; the same 5 containers healthy; no `*_recovered` mail | `B-after.txt` |
|
||
| next report | the first report after boot (21:49:22) was sent before the guest started: „Guests 0/2", cloudflared „unknown"; the next one (22:04:27, 15 min later) read 1/2 and „running (connected)" | hub log |
|
||
|
||
# HP t740 (demo-hp) on the second plug — operator-ordered, 2026-10-13
|
||
|
||
| Step | What happened | File |
|
||
|---|---|---|
|
||
| clean shutdown by hand | VM 341 113 s, guest 9202 99 s, guest 9201 104 s, then `poweroff`; no ping 19:23:10 | `HP-poweroff.log` |
|
||
| off, plug on | **0.0–3.2 W** (flickers; NIC/BIOS standby) → `standby_w` 3.2 in the private plug file | `HP-cut-power.csv` |
|
||
| cut 1 (20:05:06, 15 s) | the HP did **not** start: BIOS „After Power Loss" was Off. Operator set „Previous State" | `HP-cut.log` |
|
||
| cut 2 (20:13:16, 15 s) | started by itself: ping +38 s, controller healthy 20:16:09, all 3 guests running, 22 containers, 0 unhealthy | `HP-cut2*.{log,txt}` |
|
||
| crash guard | 2 unclean boots in 60 min → tripped, `kernel.panic=0`, re-arms in 24 h — the guard working, not a defect; mails at 20:29 | `HP-cut2.log` |
|
||
| idle watts | running, 3 guests: mean 25.2 W, median 23.0 W, max 45.5 W | `HP-cut-power.csv` |
|
||
| Tester 1 (VM 341 on the HP) | the cut killed it too: `host_crash_restart` + `host_crash_guard_tripped` for `tester-1-d70be4`, mails 20:29 | operator mailbox |
|
||
| meter note | from 20:13 two `plugmeter --plug hp` readers ran at once (the first was not stopped), so `HP-cut-power.csv` and `HP-cut2-power.csv` overlap and hold near-duplicate rows; the figures above are from one file | — |
|