REPORT + CONTEXT: F-REBOOT shipped, F-LEAK's first fix refuted and replaced, v0.109.0 observable

This commit is contained in:
2026-07-28 11:21:42 +02:00
parent 88b3cf03dd
commit ff7f68e089
2 changed files with 170 additions and 109 deletions
+33
View File
@@ -5,6 +5,39 @@
## Current
- **2026-07-28 — v0.107.0: F-REBOOT fixed — a guest rebooted mid-backup now comes back by itself.**
New `internal/localapi/guestpower.go`: a 60 s watchdog that starts a guest which is `onboot:1`,
stopped, unlocked, and has no vzdump in flight. It closes the two narrow gaps that let
`RecoverStaleLockedGuests` miss campaign fault 11 — that recovery acts only on a **stale vzdump
lock** (fault 11's guest was unlocked) and runs **once at agent startup** (fault 11's guest went
down while the agent was already up). `onboot` is the deliberate-stop discriminator and is *not*
invented here: it is already what `stalelock.go` uses for this decision, it is 0 on scratch/golden
guests, and it is what `pve-guests` consults at host boot — so the agent agrees with the platform
instead of keeping a second private definition of "should be running". Retry bounded at 3
(1m/2m/4m) then escalates **once**; an unbounded silent retry loop is the over-correction here.
Live on demo-hp: **120 s unattended** recovery vs the incident's **587 s** with a human; Scenario B
proven (an `onboot:0` guest left stopped throughout). Detail: `REPORT.md`.
- **2026-07-28 — v0.108.0: the F-LEAK pool-adoption fix was WRONG and was removed.** v0.107.0 also
shipped a teardown fallback that adopted a stranded scratch guest into the `felhom` pool and retried
the destroy. Its own live replay refuted it: `PUT /pools/felhom -> HTTP 500: permission denied at
/vms/990000 (missing privilege ...)`. **`PUT /pools/{pool}` also requires `VM.Allocate` on the VM
being added, so pool membership cannot bootstrap its own authority.** Removed rather than left in —
a path that provably cannot work looks like a fix. **The real fix is `felhom-host-install.sh`
v1.21.0** granting `FelhomAgentGuest` at each `/vms/990000..990009`; the cause was structural (the
role is granted at `/pool/felhom`, and a guest joins the pool only when its restore *completes*, so
a *failed* restore-test leaves a pool-less guest out of reach). Applied on **both** boxes. Proven by
live A/B on the same guest: grant removed → `403 (/vms/990000, VM.Allocate)`; granted → `200
UPID:...vzdestroy`. Still refused at `/vms/100` and `/vms/990010` — and since PVE checks
**permission before existence**, those 403s are real refusals, not artifacts.
- **2026-07-28 — v0.109.0: the guest-power watchdog got the observable it shipped without.** A
self-correction: v0.107.0's watchdog logged only at startup and when it *acted*, so on a healthy box
its health could be read only from **absence** — F-OBS's exact shape, shipped in the same session
F-OBS was fixed in the controller. Now an INFO summary every 10th sweep carrying
`sweeps_since_boot`/`guests_evaluated`/`currently_stopped`. An **aborted** sweep (unproven
ownership) does not count, or the heartbeat would claim liveness for a watchdog examining nothing.
- **2026-07-28 — v0.106.0: F-CRIT-2 fixed — a failed backup no longer looks like a fresh one.**
`NewestArchiveTime` counted an aborted PBS upload (1 byte, manifest-less, NEWEST) as a successful
backup, so the tier reported fresh, went **not due**, and was never retried — 7 days of silence on