After a power cut: remove the dead half-written backup (R-1006); a backup task from before the boot is not in flight (R-1008)
gates / gates (push) Successful in 35s
gates / gates (push) Successful in 35s
This commit is contained in:
@@ -1,32 +1,30 @@
|
||||
# REPORT — v0.155.0 released and delivered: the household can turn the machine off (2026-10-13)
|
||||
# REPORT — v0.156.0: after a power cut the box clears up (R-1006, R-1008) — built, not yet released (2026-10-14)
|
||||
|
||||
Baseline `86adbca` (v0.154.0). Code `3403c87`, `f38b03c`, `c6d9435`, `bdf25f0`; release entry `68a2130`; tag `v0.155.0`.
|
||||
Task: „The turn the machine off button" (R-998). Design: felhom.eu `documentation/architecture/14-host-power-off.md`.
|
||||
Evidence: felhom.eu `documentation/audits/power-cut-fixes-2026-10-14/` — the spike (C1 the night's leftover, C2 a killed
|
||||
worker, C3 a real power cut, C4 a full disk), the 17 red-proofs.
|
||||
|
||||
## What it does
|
||||
## What the spike showed
|
||||
|
||||
`POST /host/poweroff` (local API, token-scoped): record → hub told (a report carrying `deliberate_off`, accepted) →
|
||||
`pct shutdown <vmid> --timeout 120` → `systemctl poweroff`. Refused in the first 10 min after boot, a second time in a
|
||||
boot, while any other guest or VM runs, and while any backup/restore/OS/controller-update/drive/escrow job or a vzdump
|
||||
runs. After acceptance the heavy gate is held; before each step the jobs and the guests are read again (wait ≤ 15 min,
|
||||
or cancel). Every check fails closed. Sudoers: two exact vectors (`FELHOM_POWEROFF`).
|
||||
- A power cut mid-backup leaves `vzdump-lxc-<vmid>-<date>.tar.dat` + `.tmp/` (root, in `/var/lib/vz/dump`); Proxmox never
|
||||
lists it as a backup; nothing removes it; every later backup is skipped „not enough space".
|
||||
- The dead vzdump task stays in Proxmox's active list ~30–60 s after the boot — long enough for the agent's start-up pass
|
||||
to call it „genuinely in flight" (R-1008). Short, so R-1008's harm is small.
|
||||
- A killed worker leaves `tar`/`zstd` writing into the `.tar.dat` for minutes, none named „vzdump" — the reason the root
|
||||
helper checks open files, not only process names.
|
||||
- On „No space left" vzdump removes its own partial (both paths): the preflight's sizing stays. But a full host root breaks
|
||||
LVM's own metadata writes (a half-removed snapshot; the janitor cannot clear it — new row).
|
||||
|
||||
## Proof
|
||||
## What changed
|
||||
|
||||
- Tests: `internal/localapi/hostpoweroff_test.go` (11), `internal/poweroff/state_test.go`, `internal/hub/deliberate_off_test.go`,
|
||||
`internal/capability` (9 injection cases, new `TestEveryAliasIsGranted`). Each guard red-proofed by removing it.
|
||||
- Three security-review passes the same evening; every finding fixed and red-proofed (other guests, gate not held,
|
||||
fail-open, jobs and guests starting after acceptance).
|
||||
- Fixed without a row: an escrow-ceremony reply data race (`go test -race` red before, green 3/3 after).
|
||||
- Live, real sudo on the N100 after the bundle: the two new vectors ALLOWED; `poweroff --force`, `-ff`, `reboot`, `halt`,
|
||||
`pct shutdown` with `--forceStop` or without `--timeout` refused.
|
||||
- Live press on the N100 through the dashboard endpoint: 202 at 18:52:34Z, no ping 18:53:02Z, 0 W from 18:53:07Z.
|
||||
- Back on: power button 21:48:43Z, clean boot (crash guard `last_boot_unclean: false`); the first report carried no
|
||||
`deliberate_off`, and the hub ended the hold at 21:49:22Z (`host_deliberate_off_ended`). No crash or recovered mail.
|
||||
See `CHANGELOG.md` v0.156.0: `vzdumpInFlight` behind the one `BackupRunning`; `internal/vzpartial` at start-up and before
|
||||
each backup; the root helper `felhom-vzdump-partial-clean` + one sudoers line + the bundle entry.
|
||||
|
||||
## Delivered
|
||||
## Gates
|
||||
|
||||
Signed `agent_update` 0.155.0 + `agent_config_update` (bundle `795f2ab4…`) to demo-felhom, demo-hp, Tester 1. N100:
|
||||
committed 18:37:45Z, bundle installed 18:51:54Z (capability probe 70/70). **Not vouched** — the operator's act.
|
||||
demo-hp and Tester 1: agent 0.155.0, bundle installed — real `sudo -l` checks there allow the two lines and refuse
|
||||
`--force`, `--forceStop 1`, `reboot` (`felhom.eu/documentation/audits/poweroff-2026-10-13/B-sudo-live-*.txt`).
|
||||
`go build ./... && go vet` clean; `go test ./... -count=1` rc 0; `agent_gates.py` all OK; `visudo -cf` parsed OK; the
|
||||
config bundle tests (55) and the helper's tests (21) pass. 17 mutations, 17 red.
|
||||
|
||||
## Next
|
||||
|
||||
Release (tag, publish, step bundle), the operator's vouch + floor, then the live proof on demo-hp (a cut mid-backup) and
|
||||
delivery to Tester 1 and the N100.
|
||||
|
||||
Reference in New Issue
Block a user