Power-off live proof complete (R-998, R-999 closed): the hub held 2 h 56 min, the power button brought the N100 back, the hold ended; HP plug + BIOS; R-1002 filed; F11 PROVEN-LIVE
gates / gates (push) Successful in 6m45s

This commit is contained in:
admin committed 2026-10-14 00:08:56 +02:00
1 parent 95d293b911
commit 50bd3f168b
28 files changed
+8341 -9

No files matched your search

@@ -0,0 +1,17 @@
Tue Oct 13 21:50:00 UTC 2026
23:50:00 up 1 min, 1 user, load average: 1.12, 0.43, 0.15
felhom-agent 0.155.0
"last_boot_at": "2026-10-13T21:48:55Z",
"last_boot_unclean": false,
"tripped": false,
"unclean_boots": [
"unclean_boots_24h": 1,
"unclean_boots_in_window": 0,
VMID Status Lock Name
9201 running demo-felhom
gitea.dooplex.hu/admin/felhom-controller:0.310.0
felhom-controller Up 45 seconds (healthy)
opengist Up 46 seconds (healthy)
cloudflared Up 46 seconds (healthy)
filebrowser Up 46 seconds (healthy)
traefik Up 46 seconds
@@ -0,0 +1 @@
Demo Ügyfél — Felhom Hub Dashboard Customers Apps Hosts System Offsite Configuration ← All Customers Demo Ügyfél Kikapcsolva a háztartás által, 2026-10-13 18:52 UTC — riasztás szünetel eddig: 2026-10-27 18:52 UTC Last report: 2h ago · Controller 0.310.0 Auto-refresh (paused) Demo Ügyfél off Kikapcsolva a háztartás által, 2026-10-13 18:52 UTC — riasztás szünetel eddig: 2026-10-27 18:52 UTC Controller 0.310.0 Last report 2h ago Containers 1/1 Overview Applications Setup Edit Backup & DR Events Notifications Host Customer Info Customer ID demo-felhom Name Demo Ügyfél Domain demo-felhom.eu Email <e-mail> Config MANAGED Config Created 235d ago Health Status: ok System Hostname demo-felhom OS Debian GNU/Linux 12 (bookworm) Kernel 7.0.14-23-pve CPU Intel(R) N100 (4 cores) CPU 8% Memory 13% Storage SSD 1% 1.6 / 245.0 GB Backup App data (local) Enabled Last DB dump — Off-site snapshots 13 Repo size 165.7 KB Last successful run 18h ago Soft quota 50 GB Containers (1/1) Name State CPU Memory opengist running 0.0% 66 MB App Telemetry (last 7 days) App Memory (current) Memory (avg 7d) Memory (peak 7d) Catalog Limit Errors Warnings Logs Immich 2455 MB 2457 MB 3022 MB 4480M 1 8 Request log tail Cal.com 837 MB 905 MB 941 MB 1792M 63 21 Request log tail Tandoor Recipes 868 MB 859 MB 869 MB 1280M 96 14 Request log tail RomM 742 MB 679 MB 743 MB 1280M 0 6 Request log tail Paperless-ngx 589 MB 624 MB 688 MB 1664M 2 2 Request log tail Grimmory 459 MB 502 MB 627 MB 1408M 0 84 Request log tail Dawarich 592 MB 483 MB 599 MB 2688M 12 30 Request log tail Karakeep 359 MB 462 MB 599 MB 2816M 28 7 Request log tail Docmost 471 MB 410 MB 481 MB 896M 0 24 Request log tail Komga 443 MB 367 MB 443 MB 768M 0 4 Request log tail AdventureLog 400 MB 356 MB 401 MB 896M 10 10 Request log tail SparkyFitness 305 MB 346 MB 405 MB 1792M 3 6 Request log tail Nextcloud 333 MB 330 MB 334 MB 1664M 0 4 Request log tail OnlyOffice 364 MB 323 MB 428 MB 2048M 4 48 Request log tail Home Assistant 303 MB 321 MB 353 MB 1024M 0 5 Request log tail Rallly 366 MB 310 MB 366 MB 1024M 0 8 Request log tail Gramps Web 326 MB 283 MB 336 MB 1024M 12 16 Request log tail Papra 264 MB 265 MB 295 MB 768M 0 1 Request log tail Outline 283 MB 259 MB 337 MB 1152M 20 70 Request log tail n8n 326 MB 251 MB 351 MB 1536M 147 12 Request log tail Jellyfin 189 MB 205 MB 365 MB 2048M 0 0 Request log tail Mealie 228 MB 197 MB 228 MB 1000M 8 8 Request log tail Kimai 171 MB 188 MB 219 MB 640M 0 16 Request log tail Claper 248 MB 185 MB 248 MB 640M 0 15 Request log tail Calibre-Web Automated 168 MB 170 MB 201 MB 768M 2 5 Request log tail Zipline 199 MB 168 MB 218 MB 768M 0 4 Request log tail Plex 232 MB 155 MB 234 MB 2048M 6 0 Request log tail Grafana 215 MB 155 MB 254 MB 512M 0 45 Request log tail BookStack 162 MB 141 MB 164 MB 768M 6 30 Request log tail Jellyseerr 168 MB 141 MB 168 MB 384M 0 0 Request log tail Ghost 151 MB 130 MB 158 MB 512M 10 10 Request log tail Wishlist 120 MB 120 MB 169 MB 512M 0 4 Request log tail Emby 139 MB 114 MB 145 MB 2048M 0 0 Request log tail Wanderer 147 MB 103 MB 151 MB 1024M 0 3 Request log tail Uptime Kuma 117 MB 100 MB 117 MB 256M 0 0 Request log tail Termix 101 MB 87 MB 115 MB 128M 0 0 Request log tail Homepage 82 MB 83 MB 84 MB 256M 0 0 Request log tail Radarr 95 MB 78 MB 95 MB 512M 0 0 Request log tail Crafty Controller 78 MB 78 MB 78 MB 2048M 0 0 Request log tail Sonarr 79 MB 75 MB 79 MB 512M 0 1 Request log tail SABnzbd 68 MB 68 MB 68 MB 1024M 0 0 Request log tail Felhom Controller 36 MB 66 MB 156 MB — 281 83 Request log tail MeTube 62 MB 62 MB 62 MB 768M 0 0 Request log tail Code-Server 60 MB 59 MB 60 MB 1024M 0 0 Request log tail Gitea 73 MB 59 MB 89 MB 512M 0 0 Request log tail Recipe Importer 63 MB 54 MB 63 MB 128M 0 0 Request log tail LubeLogger 61 MB 44 MB 61 MB 512M 0 3 Request log tail OpenGist 66 MB 43 MB 73 MB 128M 0 0 Request log tail Audiobookshelf 48 MB 40 MB 52 MB 512M 0 0 Request log tail ActualBudget 44 MB 32 MB 46 MB 256M 0 0 Request log tail Grocy 30 MB 26 MB 30 MB 384M 0 0 Request log tail Radicale 27 MB 23 MB 27 MB 128M 0 4 Request log tail qBittorrent 22 MB 21 MB 22 MB 1024M 0 0 Request log tail Vaultwarden 21 MB 20 MB 21 MB 256M 0 0 Request log tail Vikunja 20 MB 20 MB 20 MB 256M 0 2 Request log tail PrivateBin 24 MB 18 MB 24 MB 128M 0 0 Request log tail Navidrome 13 MB 16 MB 19 MB 256M 0 6 Request log tail Homebox 17 MB 14 MB 20 MB 256M 0 0 Request log tail Glance 6 MB 6 MB 6 MB 128M 0 0 Request log tail NZBGet 5 MB 5 MB 6 MB 512M 2 0 Request log tail Gokapi 7 MB 5 MB 7 MB 128M 0 10 Request log tail BentoPDF 5 MB 4 MB 5 MB 384M 0 0 Request log tail Transmission 6 MB 4 MB 6 MB 512M 0 0 Request log tail App Log Tails (on-demand, last 2 per app kept) App Collected Received Lines felhom-controller 2026-07-10 14:52:34 (95d ago) 95d ago 200 View Download .log Credentials Retrieval Password •••••••••••••••• Reveal ⎘ The per-customer secret that fetches the whole config Line truncated
@@ -0,0 +1 @@
demo-felhom-8363b5 — Felhom Hub Felhom Hub Dashboard Customers Apps Hosts System Offsite Configuration ← Hosts demo-felhom-8363b5 KIKAPCSOLVA Host ID demo-felhom-8363b5 Customer Demo Ügyfél Agent Version 0.155.0 Enrolled 87d ago Last Report 2h ago Power Kikapcsolva a háztartás által, 2026-10-13 18:52 UTC — riasztás szünetel eddig: 2026-10-27 18:52 UTC Box connection last connected 2026-10-13 18:55 UTC (1h ago) Desired Generation 21 Vitals CPU 0% Memory 14% Disk (root fs) 33% Cloudflared running (connected) Guests 1/2 running Guests VMID Name Status Controller Last Seen 9201 demo-felhom running — 2h ago 990000 CT990000 stopped — 71d ago Storage Targets Name Role Type State Fill Thin Pool SMART Temp Wear felhom-backup — local-dir attached 1% — PASSED 34°C — felhom-pbs — pbs attached 0% — UNKNOWN — — local — local attached 33% — PASSED 34°C — local-lvm — lvmthin attached 15% 15% UNKNOWN — — Capabilities Capability Status Feature / reason controllerswap-image-inspect critical ok controller-swap / managed auto-update controllerswap-inspect critical ok controller-swap / managed auto-update controllerswap-read critical ok controller-swap / managed auto-update controllerswap-restart critical ok controller-swap / managed auto-update controllerswap-write critical ok controller-swap / managed auto-update disk-blkid critical ok disk data-bearing classify (format gate) disk-lsblk critical ok disk topology read (format gate) disk-lvs ok thin-pool usage read disk-mkfs-ext4 critical ok guarded format (ext4) disk-mkfs-xfs ok guarded format (xfs) disk-smart ok disk SMART health read dnsmasq-enable ok dnsmasq enable dnsmasq-guest-domain ok guest domain discovery dnsmasq-guest-ip ok guest LAN IP discovery dnsmasq-install ok dnsmasq package install dnsmasq-reload ok dnsmasq reload dnsmasq-restart ok dnsmasq restart (LAN-DNS self-heal) dnsmasq-rm ok dnsmasq drop-in remove (decommission) dnsmasq-write ok dnsmasq drop-in write (root content check, R-861) drive-bind critical ok drive attach (felhom-data bind under parent) drive-umount critical ok drive detach (fail-closed unmount) drives-chown-data ok felhom-data guest-root chown drives-mkdir-data ok felhom-data namespace create drives-mkdir-parent ok stable parent dir create drives-mkdir-sub ok per-drive stable dir create escrow-ceremony critical ok customer recovery-code ceremony (controller-driven) guest-fstrim ok weekly guest disk trim (thin-pool reclaim, R-444) guest-init-pid critical ok drive-gate guest-sees check (multi-drive concurrency) guest-reboot ok enroll activate-binds reboot guest-shutdown ok household power-off: guest shutdown (R-998) guesthook-delete-mp ok dead mountpoint slot delete (C1 net) guesthook-register ok pre-start hook register guestnet-dhclient-probe ok guest DHCP-client liveness probe guestnet-heal ok guest DHCP-client restart (the 2026-07-20 heal) guestnet-ifaces ok guest interface-mode read guestnet-route ok guest default-route probe host-poweroff ok household power-off: host power-off (R-998) mount-daemon-reload ok systemd reload after unit write mount-unit-disable ok mount unit disable mount-unit-enable ok mount unit enable mount-unit-install ok fs-UUID mount unit install (root content check, R-861) mount-unit-stop ok mount unit stop netmount-reset-failed ok NAS automount re-arm after start-limit (F10) netmount-rmdir ok removed-share mountpoint cleanup (F1) osapply-run ok OS update wrapper (guest fast lane) parent-bind-mp8 ok parent bind into guest at provision parent-make-private critical ok intermediary shared-parent peer-group isolation parent-make-shared critical ok intermediary shared-parent propagation parent-self-bind critical ok intermediary shared-parent self-bind parent-unit-enable ok shared-parent boot-persistence enable pbsdr-create ok PBS DR storage-entry create (K autogen) pbsdr-grant ok PBS DR storage ACL self-grant pbsdr-read ok PBS DR credential read (verify-loop auth probe) pbsdr-reconcile ok PBS DR storage-entry reconcile (set-only) provision-chown ok bootstrap mount guest-root chown provision-config-mount ok bootstrap config bind mount provision-onboot ok customer guest autostart (onboot) pve:pool-read ok stale-lock recovery scoping (pool ownership check) pve:store-grant:felhom-backup critical ok backup tier felhom-backup readable by the agent (archive listing, restore-test candidacy) pve:store-grant:felhom-pbs critical ok backup tier felhom-pbs readable by the agent (archive listing, restore-test candidacy) selfheal-networking-start ok appliance networking recovery at boot (F12 defense in depth) selfupdate-commit ok agent self-update commit selfupdate-rollback ok agent self-update rollback stalelock-unlock critical ok reboot-during-backup stale-lock recovery wg-conf-install critical ok wg-felhom conf install (root content check, R-861) wg-disable ok wg-quick@wg-felhom disable (revocation) wg-enable critical ok wg-quick@wg-felhom enable wg-handshake-read critical ok tunnel handshake-Line truncated
@@ -0,0 +1 @@
Hosts — Felhom Hub Felhom Hub Dashboard Customers Apps Hosts System Offsite Configuration Hosts Host Customer Agent Proxmox / kernel Status Guests CPU Mem Disk Cloudflared Worst Storage Tester-2-be8404 Tester 2 0.142.0 9.2.2 7.0.2-6-pve DOWN 1/1 4% 13% 6% running (connected) 6% local demo-felhom-8363b5 Demo Ügyfél 0.155.0 9.2.21 7.0.14-23-pve KIKAPCSOLVA Kikapcsolva a háztartás által, 2026-10-13 18:52 UTC — riasztás szünetel eddig: 2026-10-27 18:52 UTC 1/2 0% 14% 33% running (connected) 33% local demo-hp-bb76ea Demo HP 0.155.0 9.2.21 7.0.14-23-pve ONLINE 0/2 0% 6% 58% unknown (no running customer guest) 58% local tester-1-d70be4 Tester 1 0.155.0 9.2.2 7.0.14-22-pve ONLINE 0/2 0% 18% 21% unknown (no running customer guest) 21% local Felhom Hub 0.149.0
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
Whitespace-only changes.
@@ -0,0 +1,3 @@
press 2026-10-13T18:52:34Z
POST /api/server/poweroff -> 202 {'data': {'powering_off': True}, 'ok': True}
no_ping 2026-10-13T18:53:02Z
@@ -0,0 +1,5 @@
ALLOWED : /usr/bin/systemctl poweroff
ALLOWED : /usr/sbin/pct shutdown 9201 --timeout 120
refused : /usr/bin/systemctl poweroff --force
refused : /usr/sbin/pct shutdown 9201 --timeout 120 --forceStop 1
refused : /usr/bin/systemctl reboot
@@ -0,0 +1,8 @@
ALLOWED : /usr/bin/systemctl poweroff
ALLOWED : /usr/sbin/pct shutdown 9201 --timeout 120
refused : /usr/bin/systemctl poweroff --force
refused : /usr/bin/systemctl poweroff -ff
refused : /usr/sbin/pct shutdown 9201 --timeout 120 --forceStop 1
refused : /usr/sbin/pct shutdown 9201
refused : /usr/bin/systemctl reboot
refused : /usr/bin/systemctl halt
@@ -0,0 +1,6 @@
felhom-agent 0.155.0
ALLOWED : /usr/bin/systemctl poweroff
ALLOWED : /usr/sbin/pct shutdown 9201 --timeout 120
refused : /usr/bin/systemctl poweroff --force
refused : /usr/sbin/pct shutdown 9201 --timeout 120 --forceStop 1
refused : /usr/bin/systemctl reboot
@@ -0,0 +1,8 @@
20:04:49
22:04:49 up 39 min, 1 user, load average: 0.91, 0.71, 0.74
341 night1004-tester1 running 8192 200.00 1827
9201 running demo-hp
9202 running demo-hp-scratch
9401 stopped upgrade-harness
{'last_boot_at': '2026-10-13T19:25:35Z', 'last_boot_unclean': False, 'unclean_boots_24h': 0, 'tripped': False}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,10 @@
2026-10-13T21:00:39.092Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:00:39.193Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:13:49.897Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:13:50.014Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:22:04.745Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:27:46.025Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:35:53.483Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:41:46.819Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:45:11.332Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:45:11.413Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
@@ -0,0 +1,3 @@
20:05:06 CUT: plug off (operator-ordered outage test)
20:05:21 RESTORE: plug on
20:05:21 switch on = True
@@ -0,0 +1 @@
{'last_boot_at': '2026-10-13T20:10:53Z', 'last_boot_unclean': True, 'unclean_boots_24h': 1, 'unclean_boots_in_window': 1, 'tripped': False}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,4 @@
2026-10-13T20:46:31.792Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T20:46:31.922Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:19:55.520Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
2026-10-13T21:27:46.028Z plugmeter: read failed (RuntimeError: tuya error 914 Check device key or version)
@@ -0,0 +1,11 @@
20:13:16 CUT: plug off
20:13:31 RESTORE: plug on
20:13:31 switch on = True
20:14:09 HP answers ping (+38s after power back)
20:16:09 controller healthy
341 night1004-tester1 running 8192
9201 running demo-hp
9202 running demo-hp-scratch
9401 stopped upgrade-harness
containers: 22 running, 0 unhealthy
{'last_boot_at': '2026-10-13T20:13:54Z', 'last_boot_unclean': True, 'unclean_boots_in_window': 2, 'tripped': True}
@@ -0,0 +1,5 @@
19:19:14 qm shutdown 341 rc=0 took 113s -> status: stopped
19:20:54 pct shutdown 9202 rc=0 took 99s -> status: stopped
19:22:39 pct shutdown 9201 rc=0 took 104s -> status: stopped
19:22:45 host poweroff sent
19:23:10 no ping from demo-hp
@@ -32,3 +32,39 @@ the hub page captures is replaced by `<household e-mail>`.
(hub `host_crash_restart`). Consistent with the BIOS „after power loss = last state": a machine that was off stays off.
Not read in the BIOS (no monitor) — an inference. That is the behaviour the household text promises („press the power
button"), so the spike passes for the button; the plug cannot restart a deliberately-off N100 on the bench.
# Part B — the button, proven live on the N100, 2026-10-13
Released: agent v0.155.0 + config bundle (signed jobs), hub v0.149.0 (ArgoCD), controller v0.310.0 (hand-deployed on
guest 9201 only — the floor is the operator's). Method: endpoint-level, the exact `POST /api/server/poweroff` the
dashboard card calls (no browser on DooPlex). Hub read through the operator UI; mail read in the operator mailbox.
| Step | What happened | File |
|---|---|---|
| sudo on the box | real `sudo -l` checks on the N100: the 2 new lines allowed (`pct shutdown <id> --timeout <n>`, `systemctl poweroff`); 6 attack lines refused (`--force`, `-ff`, `--forceStop 1`, no timeout, `reboot`, `halt`) | `B-sudo-live-n100.txt` |
| sudo on the fleet | the same checks on demo-hp and Tester 1 (agent 0.155.0, bundle installed): the 2 lines allowed; `--force`, `--forceStop 1`, `reboot` refused | `B-sudo-live-demo-hp.txt`, `B-sudo-live-tester-1.txt` |
| refusals | no confirm word / a wrong word → 400; `GET /api/server/reboot` → 405 | session log |
| press | 18:52:34Z → **202** `powering_off: true`; no ping 18:53:02 (28 s); **0.0 W from 18:53:07** | `B-press.log`, `B-power.csv` |
| hub at once | event `host_deliberate_off` (info) 18:52; host page „Power: Kikapcsolva a háztartás által, 18:52 UTC — riasztás szünetel eddig: 2026-10-27 18:52 UTC" from 18:53 | `B-hub-watch.log` |
| hub at 33 min | controller staleness `ok → deliberately_off (household power-off, no event)` 19:25 | hub log |
| hub at 46 min | hosts badge **KIKAPCSOLVA** from 19:38 (before that: ONLINE plus the Power line — R-1002) | `B-hub-watch.log` |
| hub at 92–95 min | no `host_stale`, `host_down`, `node_stale`, `node_down` event; no notification in the customer's last 10 | `B-hub-*-at-9xmin.txt` |
| mailbox | no demo-felhom mail after 13:58 for the whole 2 h 56 min. Positive control: the HP power cut's `host_crash_restart` / `host_crash_guard_tripped` mails arrived 20:29 | operator mailbox |
| off | **0.0 W** for 2 h 56 min, 2346 readings, max 0.0 | `B-power.csv` |
| power button | operator pressed it: first watts 21:48:43 → boot 21:48:55 → **dashboard 200 by 21:49:33** (first check, ≤ 50 s) | `B-power.csv`, `B-after.txt` |
| hub clears | 21:49:22 `deliberate power-off ended (was 18:52:28Z)` → event `host_deliberate_off_ended` (info); 21:50 staleness `deliberately_off → ok`; badge ONLINE | hub log, `B-hub-watch.log` |
| after | crash guard `last_boot_unclean: false` (no crash mail, correct); guest 9201 running; controller 0.310.0; the same 5 containers healthy; no `*_recovered` mail | `B-after.txt` |
| next report | the first report after boot (21:49:22) was sent before the guest started: „Guests 0/2", cloudflared „unknown"; the next one (22:04:27, 15 min later) read 1/2 and „running (connected)" | hub log |
# HP t740 (demo-hp) on the second plug — operator-ordered, 2026-10-13
| Step | What happened | File |
|---|---|---|
| clean shutdown by hand | VM 341 113 s, guest 9202 99 s, guest 9201 104 s, then `poweroff`; no ping 19:23:10 | `HP-poweroff.log` |
| off, plug on | **0.0–3.2 W** (flickers; NIC/BIOS standby) → `standby_w` 3.2 in the private plug file | `HP-cut-power.csv` |
| cut 1 (20:05:06, 15 s) | the HP did **not** start: BIOS „After Power Loss" was Off. Operator set „Previous State" | `HP-cut.log` |
| cut 2 (20:13:16, 15 s) | started by itself: ping +38 s, controller healthy 20:16:09, all 3 guests running, 22 containers, 0 unhealthy | `HP-cut2*.{log,txt}` |
| crash guard | 2 unclean boots in 60 min → tripped, `kernel.panic=0`, re-arms in 24 h — the guard working, not a defect; mails at 20:29 | `HP-cut2.log` |
| idle watts | running, 3 guests: mean 25.2 W, median 23.0 W, max 45.5 W | `HP-cut-power.csv` |
| Tester 1 (VM 341 on the HP) | the cut killed it too: `host_crash_restart` + `host_crash_guard_tripped` for `tester-1-d70be4`, mails 20:29 | operator mailbox |
| meter note | from 20:13 two `plugmeter --plug hp` readers ran at once (the first was not stopped), so `HP-cut-power.csv` and `HP-cut2-power.csv` overlap and hold near-duplicate rows; the figures above are from one file | — |