selfupdate: the staging folder keeps what it needs and no more (R-1368 part 2) — not released

Each good update left its staged binary in <state dir>/selfupdate for ever
(49 files / 740 MB on demo-hp). After the wrapper's commit returned success,
MaybeCommit now removes every staged binary but the running version's.

Nothing is removed while a pending marker exists or cannot be read, at a
plain start, or after a rollback. Only a regular file with the exact name
Execute writes is removed. The rollback reads pending.json and
felhom-agent.prev, no staged binary: pinned by a test on the wrapper's text
and by a test that runs the wrapper's real commit and rollback verbs in a
sandbox. Red seen three times.

No version bump, no build, no vouch. /usr/local/bin/felhom-agent.bak-* is
not reached (root-owned; R-1368 part 4).

Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS

CHANGELOG.md

@@ -1,4 +1,80 @@
-## Unreleased — „Reboot guest" from the hub (D65 = A): `reboot_guest` joins the agent's closed list
+## Unreleased (2026-11-10) — R-1368 part 2: the self-update folder keeps what it needs and no more — COMMITTED, NOT RELEASED (no version, no build, no vouch; it waits for a release session)
+
+- **The defect (measured 2026-11-10, space list place H12).** `Execute` downloads each new agent to
+  `<state dir>/selfupdate/felhom-agent-<version>` and removed that file only when the download or its checksum failed.
+  A good update left it for ever: 49 / 48 / 33 binaries = 740 / 724 / 509 MB on demo-hp, demo-felhom and Tester 1.
+- **What the folder is needed for (read first: `configs/felhom-selfupdate-guarded`).** The root wrapper reads the staged
+  file once, copies it to its own root-owned folder and flips from that copy. The rollback needs `pending.json` (in the
+  folder) and `/usr/local/bin/felhom-agent.prev` (outside it); the commit needs `pending.json`. No staged binary.
+- **The fix (`internal/selfupdate/prune.go`).** After the wrapper's `commit` returned success, `MaybeCommit` removes
+  every staged binary but the running version's. Nothing is removed while a pending marker exists or cannot be read,
+  at a plain start, after a rollback (the failed version's binary stays to be read), or when the running version is
+  not a version this code can name a file for. Only a regular file with exactly the name `Execute` writes is removed,
+  one `os.Remove` per name; a failed removal is logged and never fails the commit. A download and a prune take turns
+  (`stagingMu`).
+- **Tests `TestPrune_*` (6).** After six updates the folder holds one binary and every bystander (unknown names, a
+  folder and a symlink that carry a staged name); a pending or unreadable marker refuses; only a healthy commit prunes;
+  the name rule follows `versionRe`; the wrapper's `commit` and `rollback` blocks name no staged binary; and the
+  wrapper's REAL `commit` and `rollback` verbs, run from a sandboxed copy, still put the previous binary back after
+  four pruned updates. **Red seen three times:** no prune call; no pending refusal; no regular-file check.
+- **Not reached.** `/usr/local/bin/felhom-agent.bak-*` (place H13): root-owned, in a folder the agent cannot write, made
+  by a person. R-1368 part 4 stays open. A box gets the clean-up at its first good update AFTER the one that brings
+  this code; nothing was run on a box.
+
+## v0.172.1 (2026-11-07) — proxmox-grub joins the boot-chain lane: the nightly pve step stops being refused R6 — RELEASED and VOUCHED 2026-11-07 (release-agent.sh from `39db030`; agent sha256 505a0594a2fb117f359e329a0b7ae2c7a6902b74c51f52c1883145f3c2ac72f8, bundle 3e7acaee4500893bfaf567b1d427790ef89bf41f08cb6a1419ee528aeb40ddc3)
+
+- **The defect (measured on demo-felhom, 2026-11-05, 11-06, 11-07).** Proxmox published `proxmox-grub` 2.12-9+pmx4. It
+  depends on `grub-efi-amd64` of the same version. `HOST_SLOW_RE` matches `grub` only at the START of a name, so
+  `proxmox-grub` landed in the ring-0 pve plan; the simulation then pulled `grub-efi-amd64`, which is not in the plan,
+  and `felhom-os-apply` refused the WHOLE pve step (R6 „the plan would touch grub-efi-amd64") every night — 34 Proxmox
+  userspace upgrades held with it, and an `os_update_failed` error mail each morning. The same class as the 2026-10-09
+  `proxmox-secure-boot-support` fix. Read-only proof on the box: `apt-get -s install --only-upgrade proxmox-grub` →
+  `Inst grub-efi-amd64 …pmx4`.
+- **The fix.** `proxmox-grub` joins `HOST_SLOW_RE`: GRUB's meta-package now waits with the rest of the boot chain (the
+  slow lane, the operator's signed step), and the pve step runs without it.
+- **Test** `test_ring0_pending_pve_leaves_proxmox_grub_out`; red seen before the fix (proxmox-grub entered the plan), green
+  after; the wrapper suite 149 of 149.
+- **Delivered 2026-11-07 (operator, signed jobs):** demo-felhom, demo-hp, Tester 1 — `agent_update` committed
+  (23:05/23:13/23:13 CET), then the bundle: `BUNDLE DONE agent=0.172.1 written=1 same=30 self-check=ok`, capability probe
+  77/77 on each. `pinned.day0_agent` = 0.172.1 (D50).
+- **Delivery.** `configs/` changed, so a box needs BOTH signed jobs: `agent_update` 0.172.1, then `agent_config_update`
+  0.172.1 with the bundle sha. Not vouched by CC. Bundle vs 0.170.0 (what the boxes run): no path added or removed, only
+  `felhom-os-apply` changed — so no step bundle is needed (R-880). Both downloads re-checked by sha.
+
+## v0.172.0 (2026-11-03) — R-1302: the first grant of a tier is no loss — RELEASED and VOUCHED 10:18:10 CET (release-agent.sh from a clean clone of `ecbb0cc`; agent sha256 2c0222cd01713f65c627d6d17c18b4a8b2b1e0c39881db0df7ccb17de48e24c7, config bundle cc34b0bc3346ffd6a4ba80fe9269852dced8c56c9503c8b14a50d9ded2961cce; configs/ unchanged since 0.171.0, so the signed delivery was agent_update alone)
+
+- **R-1302 — the first grant of a tier is no loss.** A fresh install arms the whole-guest tier before its storage exists, so the
+  first probe found no grant, repaired it and logged ERROR „GRANT WAS MISSING … investigate the loss (R-190)" plus a DEGRADED
+  capability, on every new box (VM 9375, 2026-10-26). The agent now remembers which tiers have EVER held their grant
+  (`<state dir>/store-grant-seen.json`, written by a healthy probe). A first grant logs INFO „granted" and reports healthy; a
+  grant that existed and vanished keeps R-190's ERROR and its one degraded cycle. A doubt keeps the ERROR: an unreadable or
+  corrupt file, or no file path. A box updated to this agent records its tiers at the first healthy probe, so a loss in the
+  one probe before that would read as a first grant. Tests `TestR1302_*`; red seen with the first-grant branch removed and on
+  three more mutants.
+
+## v0.171.0 (2026-10-30) — R-1344 decided from the night's measurement: a controller that runs but does not answer is RESTARTED — RELEASED (release-agent.sh from a clean clone of `bd18347`; agent sha256 c888ca7ba22a1f689059081575814a725e2edcecbb6bcc56829e9d9de88202a5, config bundle 95f977f6dc827185f2b5a2aa9e1c829db41a9f425d49eb40f317286d13c83d2b; binary-only signed agent_update on Tester 1, demo-hp, demo-felhom; vouched by the operator 2026-10-30 10:47:36 CET; `pinned.day0_agent` = 0.171.0). Live on Tester 1: the controller frozen (SIGSTOP) 09:24:50 UTC → Docker „unhealthy" 09:26:36 → the alarm 09:29:21 → the restart 09:31:22 → healthy 09:31:24; the hub mailed `controller_unresponsive` and `controller_unresponsive_restart` (hub 0.178.0), not `controller_restarted_by_agent`
+
+- **The measurement (operator brief 2026-10-29 Part 2, read 2026-10-30).** Docker's own health record of the controller
+  on Tester 1 and demo-hp, every probe from 2026-10-29 11:12 to 2026-10-30 07:00 UTC: 4 733 probes through the night
+  chain (demo-hp's PBS vzdump, Tester 1's local vzdump, both DB dumps, tier-2, the off-site push, a restore test, two
+  controller updates). No failed probe, no 5-s timeout; slowest 303 ms (demo-hp, during the PBS vzdump). The only
+  „unhealthy" was the controller stopped by hand for a proof. A busy night never came near ONE failed probe.
+- **So the alarm becomes a restart.** At `controllerUnresponsiveConfirm` = 6 unhealthy sweeps the alarm is raised, as
+  before. At `controllerUnresponsiveRestart` = 10 (2 minutes later; ~6.5 minutes after the first failed probe) the agent
+  restarts the controller's bootstrap unit — unless the controller's busy file is fresh (a backup, a restore, an
+  update, an export) or cannot be read, or the brake refuses (3 restarts in 15 minutes → the crash-loop pause, which
+  the hub mails). A swap, a parked guest, a locked guest and a vzdump keep resetting the count, as before. A refusal
+  keeps the alarm and is logged once per reason. After a restart the count starts again from zero.
+- **The stanza** carries `unresponsive_restart_after_s` (120: the hub's alarm text says when the restart comes) and
+  `unresponsive_restart_at` (equals `last_restart_at` for that act; the hub mails `controller_unresponsive_restart`).
+- **Refactor, no behaviour change:** the not-running restart's brake and its record are now `supervisorBrakeAllows`
+  and `recordSupervisorRestart`, shared by both restarts. The busy-file stale log line no longer says „operator reboot".
+- Tests `TestSupervisor_UnresponsiveIsAlarmedThenRestarted`, `…Guards` (now also: never restarted),
+  `…RestartRefusals` (busy fresh, busy unreadable, the brake, a crash-loop pause), `…StaleBusyFileDoesNotShield` —
+  red first (build failure, then 3 behaviour failures); 7 mutants, each killed (one survivor — the count reset after a
+  restart — found and pinned).
+
+## v0.170.0 (2026-10-29) — „Reboot guest" from the hub (D65 = A): `reboot_guest` joins the agent's closed list — RELEASED (release-agent.sh from a clean clone of `ca15a31`; agent sha256 da8d12a05ef607c338b9ba307e7a81b18253a3188721bbd8421ba36658a22404, config bundle 03f2ff6442fc3ca600b1f25f26767ce4da466a770a258f3e362cf95f6e76b543; signed agent_config_update + agent_update on Tester 1, demo-hp, demo-felhom; vouched by the operator 2026-10-29 18:57:48 CET; `pinned.day0_agent` = 0.170.0)
 
 - **D65 = A (operator ruling 2026-10-29 15:50; felhom.eu `04` §6.1, `03`).** The agent's closed operator-action list is
   now `restart_controller` and `reboot_guest` (arg: the vmid). No signature: a reboot destroys nothing. The act is the

CONTEXT.md

@@ -1,5 +1,29 @@
 # CONTEXT — felhom-agent working state
 
+> **2026-11-10 (day, operator brief) — COMMITTED, NOT RELEASED: the self-update folder cleans itself (R-1368 part 2).** After a healthy `commit` the agent removes every staged binary but the running version's (`internal/selfupdate/prune.go`; tests `TestPrune_*`, the wrapper's real `rollback` verb run in a sandbox). No version bump, no build, no vouch: it rides the next agent release. A box cleans its folder at the first good update AFTER that release. Not reached: `/usr/local/bin/felhom-agent.bak-*` (root-owned, R-1368 part 4). The old-kernel clean-up is a design only (felhom.eu `11-os-updates.md` §5.11.1, proven by hand on demo-hp); nothing of it is in this repo.
+
+> **2026-11-07 (attended) — v0.172.1 RELEASED and VOUCHED; on the three test boxes (agent + bundle, 77/77).** `proxmox-grub` joins `HOST_SLOW_RE` (the N100's ring-0 pve step was refused R6 three nights: proxmox-grub pmx4 pulls grub-efi-amd64). sha `505a0594…`, bundle `3e7acaee…` (only `felhom-os-apply` changed vs 0.170.0, no step bundle). Done: vouch, both signed jobs on the three boxes, `pinned.day0_agent` = 0.172.1. Next: the N100's pve step on its next night (positive line: `os-apply: DONE` for `layer=pve:9201`, no R6).
+
+> **2026-11-03 (day, attended) — v0.172.0 RELEASED and VOUCHED (10:18:10 CET), R-1302.** `2c0222cd…`, bundle `cc34b0bc…`, `pinned.day0_agent` = 0.172.0; on the three test boxes by signed `agent_update`. The appliance path pre-positions the grant, so a plain first start logs neither line; staged live both ways (felhom.eu `audits/release-2026-11-03/part3/`).
+>
+> **2026-10-30 (day, attended) — v0.171.0 RELEASED and VOUCHED (10:47:36 CET): R-1344 decided — a controller that runs
+> but does not answer is RESTARTED.** The night's measurement (4 733 probes, slowest 303 ms, no failure) licensed it:
+> alarm at `controllerUnresponsiveConfirm` = 6 unhealthy sweeps, restart at `controllerUnresponsiveRestart` = 10 —
+> refused by a fresh/unreadable busy file (`controllerBusy`, shared with D65's reboot) or the brake
+> (`supervisorBrakeAllows`, now shared with the not-running restart; `recordSupervisorRestart` records both). The stanza
+> carries `unresponsive_restart_after_s` and `unresponsive_restart_at` (hub 0.178.0 mails
+> `controller_unresponsive_restart`). Built from `bd18347` (`c888ca7b…`, bundle `95f977f6…`), binary-only signed
+> updates on the three test boxes; live on Tester 1 (SIGSTOP → alarm 4.5 min → restart 6.5 min → healthy).
+> `pinned.day0_agent` = 0.171.0. felhom.eu `REPORT-release-2026-10-30.md`.
+
+> **2026-10-29 (evening, attended) — v0.170.0 RELEASED and VOUCHED (18:57:48 CET): `reboot_guest` (D65 = A).**
+> `da8d12a0…`, bundle `03f2ff64…`, tag `v0.170.0`, from a clean clone of `ca15a31`; configs/ changed (one new read-only
+> sudo vector: `cat` of the controller's `data/ctl/busy`), so per box a signed `agent_config_update` + `agent_update`
+> (Tester 1, demo-hp, demo-felhom; capability probe 76/76). The agent's closed list is `restart_controller` +
+> `reboot_guest` (`opreboot.go`; guards in felhom.eu `03`/`04` §6.1). Proven on Tester 1: refused during an off-site run
+> (the controller's busy file), done in 67 s / 79 s, refused a second time inside the hour. `pinned.day0_agent` = 0.170.0.
+> **R-1344 decided 2026-10-30: RESTART** (see the 2026-10-30 entry above).
+
 > **2026-10-29 (afternoon, attended) — v0.169.0 RELEASED and VOUCHED (14:06:09 CET).** `fc066de6…`, bundle `3ca33f9c…`,
 > from `637c92c`; `pinned.day0_agent` 0.169.0 (`db25eb0`). The agent's own closed operator-action list:
 > `restart_controller <vmid>` (host-report reply → result on the next host report; the supervisor's guards; the brake

REPORT.md

@@ -1,29 +1,23 @@
-# REPORT — v0.169.0 released and vouched: „Restart controller" from the hub (2026-10-29, attended)
+# REPORT — the self-update folder cleans itself (R-1368 part 2) — committed, NOT released (2026-11-10, operator brief)
 
-**Plain words first.** The operator can now restart a box's controller from the hub's host page. The agent does it, so it
-works when the controller is stopped or stuck. It uses the supervisor's own guards and brake: 3 restarts in 15 minutes,
-then presses are refused for 30 minutes. A controller that runs but does not answer now raises an operator alarm
-(`controller_unresponsive`); it is not restarted by itself yet — that waits for the night's measurement (R-1344).
+**Plain words first.** Each agent update left its downloaded file on the box for ever: 49 files and 740 MB on demo-hp.
+The agent now removes the old files after an update that ended well. It keeps the file of the version that runs.
+This is a commit only. No version, no build, no vouch. No box has it. It waits for the next agent release.
 
-## Commits
+The full report of the run is in `felhom.eu`: `REPORT-endings-kernels-fixes-2026-11-10.md`.
 
-| Commit | CI (job id) | What |
-|---|---|---|
-| `637c92c` | 2504, 2507 green | The agent's closed list: `restart_controller <vmid>` on the host-report reply (`ControlEnvelope.OperatorActions`), the result on the next host report (`OperatorActionResults`). Guards: provisioned, running, parked, swap, lock, vzdump. The brake counts operator restarts with the supervisor's (`opBrakeUntil`). One audit line, one result per id, re-sent until the hub records it; a report is asked for at once. R-1344: 6 unhealthy sweeps → `unresponsive_since` in the stanza. Tests in `internal/localapi/opactions_test.go`, `supervisor_unresponsive_test.go`, `opactions_wiring_test.go`, each seen failing by its mutation. |
-| `db25eb0` | 2513 green | v0.169.0 CHANGELOG; `pinned.day0_agent` = 0.169.0 (D50 pin), after the vouch. |
+| | |
+|---|---|
+| Change | `internal/selfupdate/prune.go` (new); the call in `MaybeCommit` (`commit.go`); one lock in `Execute` (`executor.go`) |
+| Rule | After the wrapper's `commit` returned success: remove every staged binary but the running version's |
+| Never removed | anything while `pending.json` exists or cannot be read; anything at a plain start or after a rollback; `pending.json`; a folder; a symlink; a name the code does not know |
+| Rollback | Needs `pending.json` and `/usr/local/bin/felhom-agent.prev`. It reads no staged binary (`configs/felhom-selfupdate-guarded`, read first) |
+| Tests | `TestPrune_*`, 6 tests. One runs the wrapper's real `commit` and `rollback` verbs from a sandboxed copy |
+| Red seen | 3 times: no prune call; no pending refusal; no regular-file check |
+| Gates | `python3 scripts/agent_gates.py --fast`: all OK |
+| Not reached | `/usr/local/bin/felhom-agent.bak-*`: root-owned, made by a person. R-1368 part 4 stays open |
+| Not proven | On a box. A box cleans its folder at the first good update AFTER the release that brings this code |
+| Machines | None touched by this change |
 
-## Release
-
-- `release-agent.sh 0.169.0` from a clean clone: sha256 `fc066de640007038ecfaed6a23e4301a27fc486f1d7851c36177acdd26c2df61`,
-  bundle `3ca33f9cbab3bdd4c1b7f8ac90aa21e43b64209f1cb45c77ac72b3405956041b`, tag `v0.169.0`, verified by download.
-- Signed `agent_update` to the three test boxes; each committed.
-- Vouched by the operator 14:06:09 CET (the page answered 504; the save went through — R-1346).
-
-## Live proofs (Tester 1, through the hub endpoint the button posts, CC's `cc-hosts` token)
-
-- 13:01–13:02 CET on rc builds: press 1 → the controller answered after 9 s; press 2 with the controller stopped by hand →
-  restarted 5 s after the stop; press 3 done; press 4 refused („3 restarts in 15 minutes").
-- 14:49 CET on the release builds (agent 0.169.0, controller 0.348.0, hub 0.176.1): one press → „done — restarted; the
-  controller answered after 9 s", on the hub 16 s after the press; the box showed the controller „Up 12 seconds".
-
-Evidence: felhom.eu `documentation/audits/release-2026-10-29b/` (`p1-restart/`, `final/`).
+**Next.** A release session builds and vouches it. After two updates on a test box, read the folder: it must hold one
+binary.

REUSE.md

@@ -35,7 +35,7 @@
 | `SudoHostOps.ListCandidateDisks` | internal/storage/candidates.go | `ListCandidateDisks(ctx) ([]CandidateDisk, error)` | enroll-candidate discovery | Fail-safe: omits anything not provably unclaimed |
 | `antiRetargetResolveExpect` (+ `antiRetargetResolve`, `antiRetargetResolveBlank`) | internal/localapi/wipe_reresolve.go | `(durableID, expectDataBearing, resolve, derive, inspect) (device, err)` | pre-mkfs anti-retarget: resolve durable id → re-derive+match → re-inspect | AGENT-001 + audit D3; refuses path-only bindings; wired via `Server.reresolveWipe`/`reresolveBlank` (test-injectable) |
 | `signedjobs.WipeExecutor.Execute` | internal/signedjobs/wipe.go | `Execute(ctx, op, params) error` | operator-signed data-bearing wipe | Durable-id bound; nonce burned by gate BEFORE execute; refuses no-longer-data-bearing targets |
-| `selfupdate.Executor` / `selfupdate.Manager` | internal/selfupdate/{executor,commit}.go | `NewExecutor(Config)` / `NewManager(ManagerConfig)` | operator-signed agent self-update (D1): download+verify-vs-signed-sha → wrapper `apply`; startup dwell → `commit` | sha is the ONLY integrity root; wrapper (`felhom-selfupdate-guarded`) re-verifies as root + does the A/B flip; NEVER rolls back (systemd + wrapper do). `WrapperRunner` seam. Report seam `SelfUpdatePending()` |
+| `selfupdate.Executor` / `selfupdate.Manager` | internal/selfupdate/{executor,commit}.go | `NewExecutor(Config)` / `NewManager(ManagerConfig)` | operator-signed agent self-update (D1): download+verify-vs-signed-sha → wrapper `apply`; startup dwell → `commit` | sha is the ONLY integrity root; wrapper (`felhom-selfupdate-guarded`) re-verifies as root + does the A/B flip; NEVER rolls back (systemd + wrapper do). `WrapperRunner` seam. Report seam `SelfUpdatePending()`. **The staging folder cleans itself** (`prune.go`, R-1368): after a healthy `commit` only the running version's staged binary stays; a new file kept in `<state dir>/selfupdate/` must NOT be named `felhom-agent-<version>` or the prune removes it |
 | `Gate.AuthorizeStorageWipe` | internal/reconcile/gate.go | `AuthorizeStorageWipe(StorageWipeAuthz, *SignedOp) Decision` | tiered wipe authz | user-data ⇒ customer confirm bound to agent's DeviceDurableID; system/backup ⇒ operator signature only, `Confirmed` IGNORED by role |
 | `Gate.Authorize` | internal/reconcile/gate.go | `Authorize(Intent, *SignedOp) Decision` | every destructive intent | role-scoping (`roleAuthorizes`) + op-to-action binding; benign passes unsigned; audits every decision |
 | `storage.DeviceDurableID` / `ResolveDurableDevice` | internal/storage/durable_device.go | `DeviceDurableID(device) (string, error)` | WIPE-binding ids (`byid:`/`byuuid:`) | Single seam for /disks list AND gate (F20-BUG2); `ResolveDurableDevice` refuses bare paths |
@@ -88,8 +88,8 @@
 | `EnsureLeaf` | internal/localapi/cert.go | `EnsureLeaf(certPath, keyPath, host) (cert, fingerprint, generated, err)` | pinned self-signed leaf | `generated=true` invalidates every issued bootstrap pin — log LOUD (B.1) |
 | `Server.RecoverStaleLockedGuests` | internal/localapi/stalelock.go | `RecoverStaleLockedGuests(ctx)` | startup stale vzdump-lock heal (F2-b) | Clears ONLY `backup`/`snapshot-delete`, only when no vzdump in-flight; A1 RESOLVED (v0.62.0): scan is pool-intersected (`ListLXC` ∩ `Client.Pool`), fail-safe skip on pool-read failure |
 | `ControllerSwapper.Swap` + `ValidControllerImage` | internal/localapi/controllerswap.go | `Swap(ctx, vmid, target) *ControllerSwapState` | agent-owned controller image swap + rollback | Strict image regex (repo + 3-part semver); state file written BEFORE swap; no-healthcheck images need `verifyDwell` |
-| `Server.ControllerSupervisorTick` + `ControllerParkedMarker` | internal/localapi/controllersupervisor.go | `ControllerSupervisorTick(ctx)` | R-523: restart a provisioned guest's not-running controller via its bootstrap unit | Two-sweep confirm; honours swapInFlight, the host-side park marker, guest lock + vzdump; 3 restarts/15 min → 30 min pause; record rides the report as `controller_supervisor` (the hub mints the events — the agent has no event channel) |
-| `Server.OnEnvelope` / `OperatorActionResults` (operator actions) | internal/localapi/opactions.go | `OnEnvelope(ctx, env)`, `OperatorActionResults() []hub.OperatorActionResult` | 2026-10-29: the operator's actions for THIS agent from the host-report reply — a CLOSED list (`restart_controller` only, `TestOpActions_AgentClosedList`), once per id, the result re-sent until the reply stops listing it, the report requested at once (`SetOperatorActionTrigger`). The restart is the supervisor's act with its guards; the 3-in-15-min brake counts operator restarts too (30-min refusal, `opBrakeUntil`). A new verb is an operator decision AND a hub `store.hostOperatorActionNames` entry in the same release |
+| `Server.ControllerSupervisorTick` + `ControllerParkedMarker` | internal/localapi/controllersupervisor.go | `ControllerSupervisorTick(ctx)` | R-523: restart a provisioned guest's not-running controller via its bootstrap unit | Two-sweep confirm; honours swapInFlight, the host-side park marker, guest lock + vzdump; 3 restarts/15 min → 30 min pause (`supervisorBrakeAllows` — every supervisor restart passes it; `recordSupervisorRestart` puts one on the record + R-539's slow counter); R-1344 (2026-10-30): a RUNNING controller whose Docker health reads „unhealthy" is alarmed at 6 sweeps and restarted at 10 (`restartUnresponsive`: the controller's busy file, then the brake); record rides the report as `controller_supervisor` (the hub mints the events — the agent has no event channel) |
+| `Server.OnEnvelope` / `OperatorActionResults` (operator actions) | internal/localapi/opactions.go | `OnEnvelope(ctx, env)`, `OperatorActionResults() []hub.OperatorActionResult` | 2026-10-29: the operator's actions for THIS agent from the host-report reply — a CLOSED list (`restart_controller`, `reboot_guest` since D65 — `TestOpActions_AgentClosedList`), once per id, the result re-sent until the reply stops listing it, the report requested at once (`SetOperatorActionTrigger`). The restart is the supervisor's act with its guards; the 3-in-15-min brake counts operator restarts too (30-min refusal, `opBrakeUntil`). A new verb is an operator decision AND a hub `store.hostOperatorActionNames` entry in the same release |
 | `MemoryOps` + `Server.readMemoryBounds` | internal/localapi/guestmemory.go | `readMemoryBounds(ctx, vmid) (memoryBounds, err)` | guest RAM resize (v0.90.0, R-24): GET/POST /guest/memory | NEW narrow seam (never extend `GuestAPI` — it breaks every fake); the AGENT is the boundary — bounds recomputed FRESH per request (min 2048 / max host_total−2048 / shrink floor max(2048, usage+512)); §8 UNITS TRAP (config `memory`=MB, status/node=bytes); verify maxmem==target after `SetConfig` before claiming success; SetConfig NEVER called on a refusal path |
 
 ### Proxmox client / hub / PBS / provisioning

cmd/felhom-agent/main.go

@@ -521,6 +521,66 @@ type storeGrantRepairer struct {
 	mu       sync.Mutex
 	last     map[string]time.Time // target id → last ATTEMPT (success or failure)
 	repaired map[string]time.Time // target id → last CONFIRMED repair (drives the report latch)
+	// seenPath (R-1302) is the file that remembers which tiers have EVER held the grant: only those can LOSE it. "" = no
+	// memory, so every missing grant reads as a loss — the old, loud behaviour.
+	seenPath string
+}
+
+// everHeld reports whether this tier's grant was ever seen on this box (R-1302). The direction of a doubt is „yes": an
+// unreadable or corrupt file keeps the loss ERROR, so a real loss is never quietened by a broken record.
+func (r *storeGrantRepairer) everHeld(target string) bool {
+	if r == nil || r.seenPath == "" {
+		return true
+	}
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	seen, err := readGrantSeen(r.seenPath)
+	return err != nil || seen[target]
+}
+
+// noteHeld records that this tier holds the grant. A write that fails is logged and costs one false „first grant" at most
+// after a restart — the old file, or none, stands.
+func (r *storeGrantRepairer) noteHeld(target string) {
+	if r == nil || r.seenPath == "" {
+		return
+	}
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	seen, err := readGrantSeen(r.seenPath)
+	if err != nil {
+		seen = map[string]bool{} // a corrupt file is replaced by what is true now
+	}
+	if seen[target] {
+		return
+	}
+	seen[target] = true
+	b, _ := json.Marshal(seen)
+	tmp := r.seenPath + ".tmp"
+	if werr := os.WriteFile(tmp, b, 0o600); werr == nil {
+		werr = os.Rename(tmp, r.seenPath)
+		if werr == nil {
+			return
+		}
+		err = werr
+	} else {
+		err = werr
+	}
+	repairLogger(r).Warn("store-grant: could not record that this tier holds its grant (R-1302)", "target", target, "err", err)
+}
+
+func readGrantSeen(path string) (map[string]bool, error) {
+	b, err := os.ReadFile(path)
+	if os.IsNotExist(err) {
+		return map[string]bool{}, nil
+	}
+	if err != nil {
+		return nil, err
+	}
+	seen := map[string]bool{}
+	if err := json.Unmarshal(b, &seen); err != nil {
+		return nil, err
+	}
+	return seen, nil
 }
 
 // noteRepaired latches a confirmed repair so it is reported for storeGrantRepairReportWindow.
@@ -631,6 +691,7 @@ func storeGrantStatus(ctx context.Context, px *proxmox.Client, targetID string,
 		// host-report has certainly carried it. Without this latch the repairing probe may be a
 		// log-only one and the hub never learns anything happened (measured live, see the window's
 		// comment).
+		repair.noteHeld(targetID) // R-1302: a tier that holds its grant can lose it from now on
 		return storeGrantHealthyVerdict(targetID, critical, s, repair.recentlyRepaired(targetID, time.Now()))
 	}
 
@@ -679,8 +740,25 @@ func storeGrantStatus(ctx context.Context, px *proxmox.Client, targetID string,
 	// produces exactly one alert pair and the operator learns of it. NOTHING NEW WAS BUILT: no wire
 	// change, no hub change, no new event type. The `Feature` text carries the explanation because
 	// that is the field the hub puts in the operator's e-mail (the Reason does not travel).
+	return storeGrantAfterRepair(targetID, critical, repair, s)
+}
+
+// storeGrantAfterRepair is the verdict once a repair is confirmed (split out for the tests). R-1302: the FIRST grant of a
+// tier that never held one is no loss — on a fresh install the whole-guest tier is armed before its storage exists, so
+// every new box logged „investigate the loss" and a DEGRADED capability at its first start, which teaches the reader to
+// skip the line. That grant logs INFO and reports healthy. A tier seen holding its grant, or one this box cannot tell
+// about, keeps R-190's ERROR and its one degraded cycle. Pinned by TestR1302_*.
+func storeGrantAfterRepair(targetID string, critical bool, repair *storeGrantRepairer, healthy capability.Status) capability.Status {
+	if !repair.everHeld(targetID) {
+		repair.noteHeld(targetID)
+		repairLogger(repair).Info("store-grant: granted — the first grant of this tier, nothing was lost (R-1302)",
+			"target", targetID, "privilege", storeGrantRequiredPriv,
+			"action", "felhom-backup-target-apply grant "+targetID, "confirmed_by", "re-read")
+		healthy.Status, healthy.Reason = capability.StatusOK, ""
+		return healthy
+	}
 	repair.noteRepaired(targetID, time.Now())
-	s = storeGrantRepairedVerdict(targetID, critical)
+	s := storeGrantRepairedVerdict(targetID, critical)
 	repairLogger(repair).Error("store-grant: GRANT WAS MISSING AND HAS BEEN SELF-REPAIRED — investigate the loss (R-190)",
 		"target", targetID, "privilege", storeGrantRequiredPriv,
 		"action", "felhom-backup-target-apply grant "+targetID, "confirmed_by", "re-read")
@@ -853,8 +931,9 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
 	// The runner is the DIRECT one for the same reason the sudo prober uses it: the wrapper is
 	// invoked through the privileged path, which prepends sudo itself.
 	grantRepairer := &storeGrantRepairer{
-		run: (&proxmox.ExecRunner{Mode: proxmox.RunnerMode(cfg.Privileged.Mode)}).Run,
-		log: logger,
+		run:      (&proxmox.ExecRunner{Mode: proxmox.RunnerMode(cfg.Privileged.Mode)}).Run,
+		log:      logger,
+		seenPath: filepath.Join(agentStateDir(cfg), "store-grant-seen.json"), // R-1302
 	}
 	// R-973: the local-API backup cadence cap's refusals ride the report as a CRITICAL status, so the hub's
 	// operator-only whole_guest_backup_refused event (hub v0.159.0, R-1150) tells the operator. Late-bound like

cmd/felhom-agent/storegrant_test.go

@@ -1,11 +1,14 @@
 package main
 
 import (
+	"bytes"
 	"context"
 	"errors"
 	"go/ast"
 	"io"
 	"log/slog"
+	"os"
+	"path/filepath"
 	"strings"
 	"testing"
 	"time"
@@ -415,3 +418,101 @@ func TestGrantRepair_TransitionSurvivesALaterProbe(t *testing.T) {
 			"be missed entirely", storeGrantRepairReportWindow)
 	}
 }
+
+// ── R-1302 — the first grant of a tier is no loss ──────────────────────────────────────
+//
+// A fresh install arms the whole-guest tier before its storage exists, so its first probe found no grant, repaired it and
+// logged „GRANT WAS MISSING … investigate the loss" with a DEGRADED capability — on every new box (VM 9375, 2026-10-26).
+//
+// RED-PROOF (2026-11-02): seen failing before storeGrantAfterRepair existed (build), and with its first-grant branch removed
+// TestR1302_TheFirstGrantIsNoLoss fails on „a first grant must not report a loss".
+func r1302Repairer(t *testing.T, path string) (*storeGrantRepairer, *bytes.Buffer) {
+	t.Helper()
+	var buf bytes.Buffer
+	return &storeGrantRepairer{run: (&fakeRepairRunner{}).Run, seenPath: path,
+		log: slog.New(slog.NewTextHandler(&buf, nil))}, &buf
+}
+
+func TestR1302_TheFirstGrantIsNoLoss(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "store-grant-seen.json")
+	r, buf := r1302Repairer(t, path)
+	healthy := capability.Status{Name: "pve:store-grant:felhom-pbs", Status: capability.StatusDegraded, Reason: "missing", Critical: true}
+	s := storeGrantAfterRepair("felhom-pbs", true, r, healthy)
+	if s.Status != capability.StatusOK || s.Reason != "" {
+		t.Fatalf("a first grant must not report a loss: status %q reason %q", s.Status, s.Reason)
+	}
+	got := buf.String()
+	if strings.Contains(got, "investigate the loss") || strings.Contains(got, "level=ERROR") {
+		t.Fatalf("a first grant must not log the loss ERROR: %s", got)
+	}
+	if !strings.Contains(got, "level=INFO") || !strings.Contains(got, "granted") || !strings.Contains(got, "felhom-pbs") {
+		t.Fatalf("a first grant says so at INFO: %s", got)
+	}
+	if r.recentlyRepaired("felhom-pbs", time.Now()) {
+		t.Fatal("a first grant must not arm the loss report's latch")
+	}
+}
+
+func TestR1302_AGrantThatExistedAndVanishedKeepsTheError(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "store-grant-seen.json")
+	first, _ := r1302Repairer(t, path)
+	first.noteHeld("felhom-pbs") // a healthy probe, in an earlier run of the agent
+
+	r, buf := r1302Repairer(t, path) // the agent restarted: the memory is the file, not the process
+	s := storeGrantAfterRepair("felhom-pbs", true, r, capability.Status{})
+	if s.Status != capability.StatusDegraded || !strings.Contains(s.Feature, "R-190") {
+		t.Fatalf("a grant that existed and vanished must keep its degraded cycle; got %q / %q", s.Status, s.Feature)
+	}
+	if got := buf.String(); !strings.Contains(got, "level=ERROR") || !strings.Contains(got, "investigate the loss") {
+		t.Fatalf("a grant that existed and vanished must keep its ERROR: %s", got)
+	}
+	if !r.recentlyRepaired("felhom-pbs", time.Now()) {
+		t.Fatal("the loss must arm the report's latch, or the hub may never hear it")
+	}
+}
+
+// After a first grant the tier counts as held: the NEXT disappearance is a loss.
+func TestR1302_AfterTheFirstGrantALossIsALoss(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "store-grant-seen.json")
+	r, _ := r1302Repairer(t, path)
+	storeGrantAfterRepair("felhom-pbs", true, r, capability.Status{})
+	s := storeGrantAfterRepair("felhom-pbs", true, r, capability.Status{})
+	if s.Status != capability.StatusDegraded {
+		t.Fatalf("the second disappearance is a loss; got %q", s.Status)
+	}
+}
+
+// The direction of a doubt: a broken record, a repairer with no record, or no repairer at all keep the loud way.
+func TestR1302_ADoubtKeepsTheError(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "store-grant-seen.json")
+	if err := os.WriteFile(path, []byte("{not json"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	r, _ := r1302Repairer(t, path)
+	if s := storeGrantAfterRepair("felhom-pbs", true, r, capability.Status{}); s.Status != capability.StatusDegraded {
+		t.Fatalf("a corrupt record must not quieten a loss; got %q", s.Status)
+	}
+	noMemory, _ := r1302Repairer(t, "")
+	if s := storeGrantAfterRepair("felhom-pbs", true, noMemory, capability.Status{}); s.Status != capability.StatusDegraded {
+		t.Fatalf("no record must not quieten a loss; got %q", s.Status)
+	}
+	if s := storeGrantAfterRepair("felhom-pbs", true, nil, capability.Status{}); s.Status != capability.StatusDegraded {
+		t.Fatalf("no repairer must not quieten a loss; got %q", s.Status)
+	}
+}
+
+// The wiring, read from the source: production names the file, and a healthy probe records the tier.
+func TestR1302_TheMemoryIsWired(t *testing.T) {
+	b, err := os.ReadFile("main.go")
+	if err != nil {
+		t.Fatal(err)
+	}
+	src := string(b)
+	if !strings.Contains(src, `seenPath: filepath.Join(agentStateDir(cfg), "store-grant-seen.json")`) {
+		t.Fatal("main.go must give the repairer its file, or every first grant reads as a loss")
+	}
+	start := strings.Index(src, "func storeGrantStatus(")
+	if start < 0 || !strings.Contains(src[start:start+3000], "repair.noteHeld(targetID)") {
+		t.Fatal("a healthy probe must record the tier, or a real loss would read as a first grant")
+	}
+}

configs/felhom-os-apply

@@ -97,9 +97,12 @@ OOMCHECK_SETTLE = 2
 # so it belongs with them. Measured 2026-10-09 on demo-hp (Secure Boot on): left in the pve lane, its upgrade pulled
 # shim-signed-common, the whole pve step was refused R6, and the night's kernel step was skipped with it
 # (`audits/kernel-night-2026-10-08/`). Pinned by test_ring0_pending_pve_leaves_the_secure_boot_meta_out.
+# proxmox-grub is GRUB's Proxmox meta-package (it depends on grub-efi-amd64 of the same version); the `grub` above is
+# anchored at the start, so it missed it. Measured 2026-11-05..07 on demo-felhom: its upgrade pulled grub-efi-amd64 and
+# the whole pve step was refused R6 every night. Pinned by test_ring0_pending_pve_leaves_proxmox_grub_out.
 HOST_SLOW_RE = re.compile(r"^(linux-(image|headers|kbuild|modules|base)|proxmox-kernel|proxmox-default-kernel|pve-kernel|"
                           r"pve-firmware|firmware-|grub|shim|systemd-boot|intel-microcode|amd64-microcode|efibootmgr|"
-                          r"proxmox-secure-boot-support)")
+                          r"proxmox-secure-boot-support|proxmox-grub)")
 # restart_needed() leaves out processes whose cgroup line matches (grep basic regex). Host: the LXC guests' own
 # processes (`0::/lxc/<vmid>/...`) -- NOT lxc-start itself, whose cgroup is `0::/lxc.monitor/<vmid>` (measured
 # 2026-10-04 on demo-felhom: the old pattern "lxc" hid lxc-start with 20 deleted maps, so "reboot needed" stayed false

configs/test_felhom_os_apply.py

@@ -1494,6 +1494,24 @@ class PVELane(unittest.TestCase):
         self.assertEqual(sorted(u["name"] for u in rep["upgraded"]), ["pve-manager"])
         self.assertEqual(f.installed["proxmox-secure-boot-support"], "9.0.1")
 
+    # 2026-11-05..07, demo-felhom (EFI): proxmox-grub 2.12-9+pmx2 -> pmx4 depends on grub-efi-amd64 of the same version;
+    # HOST_SLOW_RE's `grub` is anchored at the start, so proxmox-grub fell into the pve plan, the simulation pulled
+    # grub-efi-amd64 and the whole pve step was refused R6 every night. It is a boot-chain package: left out.
+    # RED-PROOF: drop proxmox-grub from HOST_SLOW_RE -> it is in the plan -> this test fails.
+    def test_ring0_pending_pve_leaves_proxmox_grub_out(self):
+        f = pve_fake(ring0=True)
+        f.plan["select"], f.plan["packages"] = "pending-pve", []
+        f.installed.update({"proxmox-grub": "2.12-9+pmx2"})
+        f.pending_sim = [
+            "Inst pve-manager [9.2.2] (9.2.21 Proxmox Debian Repository:stable [amd64])",
+            "Inst proxmox-grub [2.12-9+pmx2] (2.12-9+pmx4 Proxmox Debian Repository:stable [amd64])",
+            "Inst grub-efi-amd64 [2.12-9+pmx2] (2.12-9+pmx4 Proxmox Debian Repository:stable [amd64])",
+        ]
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        self.assertEqual(sorted(u["name"] for u in rep["upgraded"]), ["pve-manager"])
+        self.assertEqual(f.installed["proxmox-grub"], "2.12-9+pmx2")
+
     def test_select_pending_pve_needs_the_pve_layer(self):
         f = Fake()
         f.plan["layer"], f.plan["select"], f.plan["packages"] = "host", "pending-pve", []

internal/hub/report.go

@@ -218,6 +218,12 @@ type ControllerSupervisorGuest struct {
 	// looks like it). UnresponsiveSince is the episode's start; the hub keys on it MOVING (controller_unresponsive).
 	Unresponsive      bool   `json:"unresponsive"`
 	UnresponsiveSince string `json:"unresponsive_since,omitempty"` // RFC3339
+	// R-1344 restart (decided 2026-10-30 from the night's measurement): UnresponsiveRestartAfterS is how long after the
+	// alarm the agent restarts a controller that still does not answer (0 / absent = an older agent: alarm only — the
+	// hub's message says which). UnresponsiveRestartAt is the last such restart; the hub keys on it MOVING
+	// (controller_unresponsive_restart) and it equals LastRestartAt for that act.
+	UnresponsiveRestartAfterS int    `json:"unresponsive_restart_after_s,omitempty"`
+	UnresponsiveRestartAt     string `json:"unresponsive_restart_at,omitempty"` // RFC3339
 }
 
 // PBSDRStatus is the per-heartbeat PBS-DR-tier bridge state (slice 2). States:

internal/localapi/controllersupervisor.go

@@ -3,6 +3,7 @@ package localapi
 import (
 	"context"
 	"encoding/json"
+	"fmt"
 	"os"
 	"path/filepath"
 	"sort"
@@ -87,10 +88,16 @@ const (
 	// R-1344 (operator brief 2026-10-29, Part 2) — a controller that RUNS but does not answer. Docker's own HEALTHCHECK
 	// (`curl -f localhost:8080/api/health`, every 30 s, timeout 5 s) reads „unhealthy" only after 3 failed probes (~90 s);
 	// controllerUnresponsiveConfirm more sweeps (3 minutes) make it an episode: logged, carried by the stanza, mailed by
-	// the hub (controller_unresponsive). ALARM ONLY — it is NOT restarted for this until the measurement (R-1344) shows a
-	// busy night (backups, updates) never reads unhealthy that long. Not counted during a swap, a parked guest, a locked
-	// guest or a vzdump.
+	// the hub (controller_unresponsive). Not counted during a swap, a parked guest, a locked guest or a vzdump.
 	controllerUnresponsiveConfirm = 6
+	// controllerUnresponsiveRestart (decided 2026-10-30 from the measurement): at this many consecutive unhealthy sweeps
+	// (5 minutes after Docker's „unhealthy", ~6.5 minutes after the first failed probe; 2 minutes after the alarm) the
+	// agent restarts the controller — the not-running restart's guards and brake, plus the controller's busy file. The
+	// numbers: Tester 1 and demo-hp, 2026-10-29 11:12 → 2026-10-30 07:00 UTC, 4 733 probes through the night chain (a
+	// PBS and a local vzdump, the DB dumps, tier-2, the off-site push, a restore test, two controller updates): no failed
+	// probe, slowest 303 ms against the 5 s timeout. A busy night never came near even ONE failed probe; ten sweeps is
+	// the margin. Pinned by TestSupervisor_UnresponsiveIsAlarmedThenRestarted.
+	controllerUnresponsiveRestart = 10
 )
 
 // controllerSupState is one guest's supervisor record. In-memory on purpose (the guest-power
@@ -119,6 +126,10 @@ type controllerSupState struct {
 	unhealthySeen     int
 	unresponsive      bool
 	unresponsiveSince time.Time
+	// unresponsiveRestartAt: the last restart for not answering (kept; the hub keys on it MOVING). unrespRefusal: the
+	// last reason the restart was refused, so a refusal is logged once per change, not every sweep.
+	unresponsiveRestartAt time.Time
+	unrespRefusal         string
 }
 
 type controllerSupervisor struct {
@@ -354,9 +365,27 @@ func (s *Server) superviseOneController(ctx context.Context, vmid int, guestStat
 		return false
 	}
 
-	// Backoff.
+	if !s.supervisorBrakeAllows(vmid, now) {
+		return false
+	}
+
+	reason := "controller container " + status + " on " + strconv.Itoa(controllerSupervisorConfirm) + " consecutive sweeps"
+	s.logger.Warn("controller-supervisor: controller is NOT running — restarting the bootstrap unit",
+		"vmid", vmid, "status", status, "unit", bootstrapUnit)
+	if _, err := s.guestExec.GuestExec(ctx, vmid, "systemctl", "restart", bootstrapUnit); err != nil {
+		s.logger.Error("controller-supervisor: bootstrap restart failed", "vmid", vmid, "err", err)
+		reason += "; restart FAILED: " + err.Error()
+	}
+	s.recordSupervisorRestart(vmid, now, reason, nil)
+	return false
+}
+
+// supervisorBrakeAllows is the brake every supervisor restart passes (the not-running one and R-1344's): a crash-loop
+// pause in force refuses; controllerCrashloopMax restarts inside controllerCrashloopWindow start one (logged, carried by
+// the stanza, mailed by the hub as controller_crashloop) and refuse.
+func (s *Server) supervisorBrakeAllows(vmid int, now time.Time) bool {
 	s.ctrlSup.mu.Lock()
-	st = s.supState(vmid)
+	st := s.supState(vmid)
 	if !st.crashloopSince.IsZero() {
 		if now.Sub(st.crashloopSince) < controllerCrashloopPause {
 			s.ctrlSup.mu.Unlock()
@@ -379,22 +408,23 @@ func (s *Server) superviseOneController(ctx context.Context, vmid int, guestStat
 		return false
 	}
 	s.ctrlSup.mu.Unlock()
+	return true
+}
 
-	reason := "controller container " + status + " on " + strconv.Itoa(controllerSupervisorConfirm) + " consecutive sweeps"
-	s.logger.Warn("controller-supervisor: controller is NOT running — restarting the bootstrap unit",
-		"vmid", vmid, "status", status, "unit", bootstrapUnit)
-	if _, err := s.guestExec.GuestExec(ctx, vmid, "systemctl", "restart", bootstrapUnit); err != nil {
-		s.logger.Error("controller-supervisor: bootstrap restart failed", "vmid", vmid, "err", err)
-		reason += "; restart FAILED: " + err.Error()
-	}
+// recordSupervisorRestart puts a restart the supervisor performed on the record: the brake's window, the totals, the
+// reason, and R-539's slow counter (persisted; raised at most once per 24 hours — the hub mails on the raise MOVING).
+// mark, when set, runs under the same lock (R-1344 stamps its own record there).
+func (s *Server) recordSupervisorRestart(vmid int, now time.Time, reason string, mark func(st *controllerSupState)) {
 	s.ctrlSup.mu.Lock()
-	st = s.supState(vmid)
+	st := s.supState(vmid)
 	st.restarts = append(st.restarts, now)
 	st.restartsTotal++
 	st.lastRestartAt = now
 	st.lastReason = reason
 	st.notRunningSeen = 0
-	// R-539: the slow counter. Raise at most once per 24 hours — the hub mails on the raise MOVING.
+	if mark != nil {
+		mark(st)
+	}
 	st.restarts24h = append(pruneBefore(st.restarts24h, now.Add(-controllerSlowCrashloopWindow)), now)
 	n24 := len(st.restarts24h)
 	raised := false
@@ -410,7 +440,6 @@ func (s *Server) superviseOneController(ctx context.Context, vmid int, guestStat
 		s.logger.Warn("controller-supervisor: SLOW CRASH-LOOP — the controller keeps dying; still restarting it, raising controller_slow_crashloop",
 			"vmid", vmid, "restarts_24h", n24, "window", controllerSlowCrashloopWindow.String(), "threshold", controllerSlowCrashloopMax)
 	}
-	return false
 }
 
 // observeControllerHealth is R-1344's count for a RUNNING controller: Docker's health status, one read per sweep over
@@ -452,11 +481,60 @@ func (s *Server) observeControllerHealth(ctx context.Context, vmid int, parked b
 	if start {
 		st.unresponsive, st.unresponsiveSince = true, now
 	}
+	since := st.unresponsiveSince
 	s.ctrlSup.mu.Unlock()
 	if start {
-		s.logger.Error("controller-supervisor: the controller RUNS but does not answer its health check — raising controller_unresponsive; NOT restarting it (R-1344: alarm only until measured)",
-			"vmid", vmid, "unhealthy_sweeps", seen, "interval", controllerSupervisorInterval.String())
+		s.logger.Error("controller-supervisor: the controller RUNS but does not answer its health check — raising controller_unresponsive; restarting it if it stays so",
+			"vmid", vmid, "unhealthy_sweeps", seen, "interval", controllerSupervisorInterval.String(),
+			"restart_after_sweeps", controllerUnresponsiveRestart)
 	}
+	if seen >= controllerUnresponsiveRestart {
+		s.restartUnresponsive(ctx, vmid, seen, since, now)
+	}
+}
+
+// restartUnresponsive is R-1344's restart (decided 2026-10-30): the controller's busy file (fresh = a backup, a
+// restore, an update or an export is running; unreadable = cannot be ruled out) refuses, then the brake every
+// supervisor restart passes. The swap, the parked marker, the lock and the vzdump never reach here — they reset the
+// count. A refusal keeps the alarm and is logged once per reason; the next sweep asks again.
+func (s *Server) restartUnresponsive(ctx context.Context, vmid, seen int, since, now time.Time) {
+	refuse := func(why string) {
+		s.ctrlSup.mu.Lock()
+		st := s.supState(vmid)
+		changed := st.unrespRefusal != why
+		st.unrespRefusal = why
+		s.ctrlSup.mu.Unlock()
+		if changed {
+			s.logger.Warn("controller-supervisor: the controller does not answer, but the restart is refused — the alarm stays",
+				"vmid", vmid, "why", why, "unhealthy_sweeps", seen)
+		}
+	}
+	busy, what, err := s.controllerBusy(ctx, vmid)
+	if err != nil {
+		refuse("the controller's busy file cannot be read: " + err.Error())
+		return
+	}
+	if busy {
+		refuse("the controller is busy: " + what)
+		return
+	}
+	if !s.supervisorBrakeAllows(vmid, now) {
+		refuse("the restart brake")
+		return
+	}
+	reason := fmt.Sprintf("the controller ran but did not answer its health check for %d sweeps (since %s)",
+		seen, since.UTC().Format(time.RFC3339))
+	s.logger.Warn("controller-supervisor: the controller does not answer — restarting the bootstrap unit",
+		"vmid", vmid, "unhealthy_sweeps", seen, "since", since.UTC().Format(time.RFC3339), "unit", bootstrapUnit)
+	if _, err := s.guestExec.GuestExec(ctx, vmid, "systemctl", "restart", bootstrapUnit); err != nil {
+		s.logger.Error("controller-supervisor: bootstrap restart failed", "vmid", vmid, "err", err)
+		reason += "; restart FAILED: " + err.Error()
+	}
+	s.recordSupervisorRestart(vmid, now, reason, func(st *controllerSupState) {
+		st.unresponsiveRestartAt = now
+		st.unhealthySeen = 0
+		st.unrespRefusal = ""
+	})
 }
 
 func (s *Server) resetNotRunning(vmid int) {
@@ -517,6 +595,10 @@ func (s *Server) ControllerSupervisorStatus(_ context.Context) *hub.ControllerSu
 		if !st.unresponsiveSince.IsZero() {
 			g.UnresponsiveSince = st.unresponsiveSince.UTC().Format(time.RFC3339)
 		}
+		g.UnresponsiveRestartAfterS = (controllerUnresponsiveRestart - controllerUnresponsiveConfirm) * int(controllerSupervisorInterval/time.Second)
+		if !st.unresponsiveRestartAt.IsZero() {
+			g.UnresponsiveRestartAt = st.unresponsiveRestartAt.UTC().Format(time.RFC3339)
+		}
 		out.Guests = append(out.Guests, g)
 	}
 	sort.Slice(out.Guests, func(i, j int) bool { return out.Guests[i].VMID < out.Guests[j].VMID })

internal/localapi/opreboot.go

@@ -137,7 +137,8 @@ func (s *Server) rebooter() interface {
 
 // controllerBusy reads the controller's busy file. busy=true with the controller's reason when the file is fresh;
 // a missing file is „not busy" (an idle controller, or one older than the file); any other read failure is an error
-// (the caller refuses — it cannot prove no backup runs).
+// (the caller refuses — it cannot prove no backup runs). Two callers: the operator's reboot (D65) and R-1344's restart
+// of a controller that does not answer (controllersupervisor.go restartUnresponsive).
 func (s *Server) controllerBusy(ctx context.Context, vmid int) (bool, string, error) {
 	out, err := s.guestExec.GuestExec(ctx, vmid, "cat", ControllerBusyMarkerGuestPath)
 	if err != nil {
@@ -152,7 +153,7 @@ func (s *Server) controllerBusy(ctx context.Context, vmid int) (bool, string, er
 		return false, "", fmt.Errorf("the controller's busy file does not start with a time: %q", lines[0])
 	}
 	if s.clock().Sub(at) > controllerBusyFresh {
-		s.logger.Info("operator reboot: the controller's busy file is stale — the controller stopped writing it; not busy",
+		s.logger.Info("controller-busy: the controller's busy file is stale — the controller stopped writing it; not busy",
 			"vmid", vmid, "written", at.UTC().Format(time.RFC3339))
 		return false, "", nil
 	}

internal/localapi/supervisor_unresponsive_test.go

@@ -2,19 +2,25 @@ package localapi
 
 import (
 	"context"
+	"errors"
+	"strings"
 	"testing"
+	"time"
 )
 
-// R-1344 (operator brief 2026-10-29, Part 2): a controller that RUNS but whose Docker health check reads „unhealthy" on
-// controllerUnresponsiveConfirm consecutive sweeps is reported unresponsive (the stanza; the hub raises
-// controller_unresponsive). ALARM ONLY: it is never restarted for it until the measurement licenses that.
-// RED-PROOF: seen failing before the stanza carried it.
+// R-1344 (operator brief 2026-10-29, Part 2; decided 2026-10-30 from the night's measurement): a controller that RUNS
+// but whose Docker health check reads „unhealthy" on controllerUnresponsiveConfirm consecutive sweeps is reported
+// unresponsive (the stanza; the hub raises controller_unresponsive). If it still reads „unhealthy" at
+// controllerUnresponsiveRestart sweeps, the agent restarts it — with the not-running restart's guards and brake, plus
+// the controller's own busy file. RED-PROOF: the restart tests seen failing before the restart existed.
 
-func unrespServer(t *testing.T, health string) (*Server, *opExec, *fakeGuestPowerCtl) {
+func unrespServer(t *testing.T, health string) (*Server, *rbExec, *fakeGuestPowerCtl, *supClock) {
 	t.Helper()
-	s, ex, ctl, _, _, _ := opServer(t, "running")
+	s, ex, ctl, clk, _, _ := opServer(t, "running")
 	ex.health = health
-	return s, ex, ctl
+	rb := &rbExec{opExec: ex}
+	s.guestExec = rb
+	return s, rb, ctl, clk
 }
 
 func sweep(s *Server, n int) {
@@ -23,8 +29,8 @@ func sweep(s *Server, n int) {
 	}
 }
 
-func TestSupervisor_UnresponsiveIsAlarmedNeverRestarted(t *testing.T) {
-	s, ex, _ := unrespServer(t, "unhealthy")
+func TestSupervisor_UnresponsiveIsAlarmedThenRestarted(t *testing.T) {
+	s, ex, _, _ := unrespServer(t, "unhealthy")
 	sweep(s, controllerUnresponsiveConfirm-1)
 	if g := s.ControllerSupervisorStatus(context.Background()).Guests; len(g) != 1 || g[0].Unresponsive {
 		t.Fatalf("unresponsive before %d sweeps: %+v", controllerUnresponsiveConfirm, g)
@@ -34,41 +40,117 @@ func TestSupervisor_UnresponsiveIsAlarmedNeverRestarted(t *testing.T) {
 	if len(g) != 1 || !g[0].Unresponsive || g[0].UnresponsiveSince == "" {
 		t.Fatalf("after %d unhealthy sweeps: %+v", controllerUnresponsiveConfirm, g)
 	}
+	if g[0].UnresponsiveRestartAfterS != (controllerUnresponsiveRestart-controllerUnresponsiveConfirm)*int(controllerSupervisorInterval/time.Second) {
+		t.Fatalf("the stanza must tell the hub when the restart comes: %+v", g[0])
+	}
 	since := g[0].UnresponsiveSince
-	sweep(s, 10)
+	sweep(s, controllerUnresponsiveRestart-controllerUnresponsiveConfirm-1)
 	if n := ex.count(9201); n != 0 {
-		t.Fatalf("an unresponsive controller was RESTARTED (%d) — the restart waits for the measurement", n)
+		t.Fatalf("restarted after %d unhealthy sweeps — the restart waits for %d", controllerUnresponsiveRestart-1, controllerUnresponsiveRestart)
 	}
 	if g := s.ControllerSupervisorStatus(context.Background()).Guests; g[0].UnresponsiveSince != since {
 		t.Fatal("unresponsive_since moved while it stayed unresponsive — the hub would mail again")
 	}
-	// it answers again
-	ex.health = "healthy"
+	sweep(s, 1)
+	if n := ex.count(9201); n != 1 {
+		t.Fatalf("after %d unhealthy sweeps: %d restarts, want 1", controllerUnresponsiveRestart, n)
+	}
+	g = s.ControllerSupervisorStatus(context.Background()).Guests
+	if g[0].UnresponsiveRestartAt == "" || g[0].UnresponsiveRestartAt != g[0].LastRestartAt || g[0].Restarts24h != 1 ||
+		!strings.Contains(g[0].LastReason, "health check") {
+		t.Fatalf("the restart is not on the record the hub reads: %+v", g[0])
+	}
+	// A restart that did not help (still „unhealthy" on the next sweep) is not repeated at once: the count starts again.
+	sweep(s, 1)
+	if n := ex.count(9201); n != 1 {
+		t.Fatalf("restarted again on the sweep after a restart: %d restarts", n)
+	}
+	// The restarted controller starts, then answers: the episode ends, its record stays.
+	ex.health = "starting"
 	sweep(s, 1)
 	if g := s.ControllerSupervisorStatus(context.Background()).Guests; g[0].Unresponsive || g[0].UnresponsiveSince != since {
-		t.Fatalf("after it answered: %+v (unresponsive must clear; since stays as the record)", g[0])
+		t.Fatalf("after the restart: %+v (unresponsive must clear; since stays as the record)", g[0])
+	}
+	// A second episode needs the whole count again — never a restart on the first unhealthy read after one.
+	ex.health = "unhealthy"
+	sweep(s, controllerUnresponsiveRestart-1)
+	if n := ex.count(9201); n != 1 {
+		t.Fatalf("a second episode restarted early: %d restarts", n)
 	}
 }
 
-// Slow but answering (healthy / starting), a swap, a vzdump or a parked guest never counts.
+// Slow but answering (healthy / starting), a swap, a vzdump, a locked or a parked guest never counts — and so is never
+// restarted.
 func TestSupervisor_UnresponsiveGuards(t *testing.T) {
 	for _, tc := range []struct {
 		name  string
-		setup func(s *Server, ex *opExec, ctl *fakeGuestPowerCtl)
+		setup func(s *Server, ex *rbExec, ctl *fakeGuestPowerCtl)
 	}{
-		{"healthy", func(_ *Server, ex *opExec, _ *fakeGuestPowerCtl) { ex.health = "healthy" }},
-		{"starting", func(_ *Server, ex *opExec, _ *fakeGuestPowerCtl) { ex.health = "starting" }},
-		{"swap", func(s *Server, _ *opExec, _ *fakeGuestPowerCtl) { s.swapInFlight[9201] = true }},
-		{"vzdump", func(_ *Server, _ *opExec, ctl *fakeGuestPowerCtl) { ctl.backupRun = map[int]bool{9201: true} }},
-		{"locked", func(_ *Server, _ *opExec, ctl *fakeGuestPowerCtl) { ctl.locks[9201] = "backup" }},
+		{"healthy", func(_ *Server, ex *rbExec, _ *fakeGuestPowerCtl) { ex.health = "healthy" }},
+		{"starting", func(_ *Server, ex *rbExec, _ *fakeGuestPowerCtl) { ex.health = "starting" }},
+		{"swap", func(s *Server, _ *rbExec, _ *fakeGuestPowerCtl) { s.swapInFlight[9201] = true }},
+		{"vzdump", func(_ *Server, _ *rbExec, ctl *fakeGuestPowerCtl) { ctl.backupRun = map[int]bool{9201: true} }},
+		{"locked", func(_ *Server, _ *rbExec, ctl *fakeGuestPowerCtl) { ctl.locks[9201] = "backup" }},
 	} {
 		t.Run(tc.name, func(t *testing.T) {
-			s, ex, ctl := unrespServer(t, "unhealthy")
+			s, ex, ctl, _ := unrespServer(t, "unhealthy")
 			tc.setup(s, ex, ctl)
-			sweep(s, controllerUnresponsiveConfirm+2)
+			sweep(s, controllerUnresponsiveRestart+2)
 			if g := s.ControllerSupervisorStatus(context.Background()).Guests; len(g) == 1 && g[0].Unresponsive {
 				t.Fatalf("%s counted as unresponsive: %+v", tc.name, g[0])
 			}
+			if n := ex.count(9201); n != 0 {
+				t.Fatalf("%s: restarted %d times", tc.name, n)
+			}
 		})
 	}
 }
+
+// At the restart moment: the controller's busy file (fresh, or unreadable) and the brake refuse it; the alarm stays.
+// RED-PROOF: each seen failing by deleting its guard.
+func TestSupervisor_UnresponsiveRestartRefusals(t *testing.T) {
+	for _, tc := range []struct {
+		name      string
+		setup     func(s *Server, ex *rbExec, clk *supClock)
+		crashloop bool
+	}{
+		{"busy file fresh", func(_ *Server, ex *rbExec, clk *supClock) {
+			ex.busy = clk.t.Add(-20*time.Second).Format(time.RFC3339) + "\na backup, a restore or an off-site run is running\n"
+		}, false},
+		{"busy file unreadable", func(_ *Server, ex *rbExec, _ *supClock) { ex.busyErr = errors.New("pct exec: exit status 255") }, false},
+		{"brake: 3 restarts in 15 minutes", func(s *Server, _ *rbExec, clk *supClock) {
+			st := s.supState(9201)
+			st.restarts = []time.Time{clk.t.Add(-10 * time.Minute), clk.t.Add(-5 * time.Minute), clk.t.Add(-time.Minute)}
+		}, true},
+		{"crash-loop pause in force", func(s *Server, _ *rbExec, clk *supClock) {
+			s.supState(9201).crashloopSince = clk.t.Add(-5 * time.Minute)
+		}, true},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			s, ex, _, clk := unrespServer(t, "unhealthy")
+			tc.setup(s, ex, clk)
+			sweep(s, controllerUnresponsiveRestart+3)
+			if n := ex.count(9201); n != 0 {
+				t.Fatalf("%s: restarted %d times", tc.name, n)
+			}
+			g := s.ControllerSupervisorStatus(context.Background()).Guests
+			if len(g) != 1 || !g[0].Unresponsive || g[0].UnresponsiveRestartAt != "" {
+				t.Fatalf("%s: the alarm must stay and no restart be recorded: %+v", tc.name, g)
+			}
+			if tc.crashloop && !g[0].Crashloop {
+				t.Fatalf("%s: the brake did not raise the crash-loop: %+v", tc.name, g[0])
+			}
+		})
+	}
+}
+
+// A busy file the controller stopped writing (older than controllerBusyFresh) is not „busy" — a hung controller must
+// not shield itself with its last word.
+func TestSupervisor_UnresponsiveStaleBusyFileDoesNotShield(t *testing.T) {
+	s, ex, _, clk := unrespServer(t, "unhealthy")
+	ex.busy = clk.t.Add(-10*time.Minute).Format(time.RFC3339) + "\nan app update is running\n"
+	sweep(s, controllerUnresponsiveRestart)
+	if n := ex.count(9201); n != 1 {
+		t.Fatalf("a stale busy file blocked the restart: %d restarts", n)
+	}
+}

internal/selfupdate/commit.go

@@ -128,4 +128,11 @@ func (m *Manager) MaybeCommit(ctx context.Context) {
 	if m.onCommitted != nil {
 		m.onCommitted(ctx)
 	}
+	// R-1368 part 2: the update ended healthy — the folder keeps the running version's binary and no other
+	// (prune.go). A refusal or a failed removal is logged and changes nothing about the commit.
+	if removed, err := pruneStaged(m.stateDir, m.runningVersion, m.logger); err != nil {
+		m.logger.Warn("selfupdate: old staged binaries NOT removed after the commit", "err", err)
+	} else if len(removed) > 0 {
+		m.logger.Info("selfupdate: old staged binaries removed after the commit", "count", len(removed), "kept", "felhom-agent-"+m.runningVersion)
+	}
 }

internal/selfupdate/executor.go

@@ -103,6 +103,9 @@ func (e *Executor) Execute(ctx context.Context, op string, params json.RawMessag
 	}
 	e.logger.Debug("agent_update: op invariants passed (semver + sha format)", "version", p.Version)
 
+	// The staged file is this call's from the download to the hand-off: a prune (prune.go) waits for it.
+	stagingMu.Lock()
+	defer stagingMu.Unlock()
 	dir := stagingDir(e.stateDir)
 	if err := os.MkdirAll(dir, 0o750); err != nil {
 		return fmt.Errorf("agent_update: staging dir: %w", err)

internal/selfupdate/prune.go

@@ -0,0 +1,96 @@
+package selfupdate
+
+import (
+	"fmt"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"sync"
+)
+
+// The self-update folder keeps what it needs and no more (R-1368 part 2; space list 2026-11-10, place H12).
+//
+// WHAT WAS WRONG. Execute downloads each new agent to <StateDir>/selfupdate/felhom-agent-<version> and removes that
+// file only when the download or its checksum fails. A good update left it there for ever: measured 2026-11-10, 49 /
+// 48 / 33 binaries = 740 / 724 / 509 MB on demo-hp, demo-felhom and Tester 1 — one for each version the box ever got.
+//
+// WHAT THE FOLDER IS NEEDED FOR. The root wrapper (felhom-os-apply, mode agent_update) reads the staged file ONCE,
+// copies it to its own root-owned folder and flips from that copy. After that hand-off nothing reads the staged file:
+//   - the rollback (felhom-selfupdate-guarded `rollback`) needs two files and neither is a staged binary:
+//     `pending.json` in this folder, and `/usr/local/bin/felhom-agent.prev` outside it;
+//   - the commit needs `pending.json` only.
+//
+// So a staged binary is dead weight as soon as its update ended. Pinned by TestPrune_RollbackReadsNoStagedBinary (the
+// wrapper's text) and TestPrune_RealRollbackStillWorksAfterNUpdates (the wrapper's rollback verb, run for real).
+//
+// THE RULE. After an update ended HEALTHY (the wrapper's `commit` returned success), remove every staged binary but
+// the one of the running version. Nothing is removed:
+//   - while a pending marker exists or cannot be read — an update is in flight, and that is the only time a rollback
+//     can fire;
+//   - after a rollback, or at a plain start — only the commit path calls this, so the binary of a version that failed
+//     stays for the operator to look at until the next good update;
+//   - that is not a regular file with exactly the name Execute gives a staged binary. `pending.json`, a folder, a
+//     symlink, and any name this code does not know are never touched.
+//
+// NOT REACHED BY THIS: `/usr/local/bin/felhom-agent.bak-*` (space list H13). Those files are root-owned, in a folder
+// the agent cannot write, and no code of the agent makes them (a person's copy before a hand deploy). R-1368 part 4.
+
+// stagedNameRe is the exact name Execute gives a staged binary: "felhom-agent-" + a version that versionRe accepts.
+// Kept in step with versionRe by TestPrune_NameShapeFollowsTheVersionRule.
+var stagedNameRe = regexp.MustCompile(`^felhom-agent-([0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.]+)?)$`)
+
+// stagingMu makes a download and a prune take turns. Without it a prune that runs while Execute downloads the NEXT
+// version would remove that half-written file (its name is a staged binary's and it is not the running version).
+var stagingMu sync.Mutex
+
+// pruneStaged removes the staged binaries the folder no longer needs and returns the names it removed, sorted. It
+// refuses (removes nothing, returns an error) while an update is in flight. A file it cannot remove is logged and
+// skipped: a failed clean-up must never fail the commit that came before it.
+func pruneStaged(stateDir, runningVersion string, logger *slog.Logger) ([]string, error) {
+	logger = orDefaultLogger(logger)
+	stagingMu.Lock()
+	defer stagingMu.Unlock()
+
+	if !versionRe.MatchString(runningVersion) {
+		return nil, fmt.Errorf("selfupdate prune: refusing — running version %q is not a version this code knows; it could not name the file to keep", runningVersion)
+	}
+	pm, err := readPending(stateDir)
+	if err != nil {
+		return nil, fmt.Errorf("selfupdate prune: refusing — the pending marker cannot be read: %w", err)
+	}
+	if pm != nil {
+		return nil, fmt.Errorf("selfupdate prune: refusing — an update to %s is pending (a rollback may still fire)", pm.NewVersion)
+	}
+	dir := stagingDir(stateDir)
+	entries, err := os.ReadDir(dir)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, fmt.Errorf("selfupdate prune: read %s: %w", dir, err)
+	}
+	keep := "felhom-agent-" + runningVersion
+	var removed []string
+	for _, ent := range entries {
+		name := ent.Name()
+		if name == keep || !stagedNameRe.MatchString(name) {
+			continue // the running version's binary, pending.json, or a name this code does not know
+		}
+		if !ent.Type().IsRegular() {
+			logger.Warn("selfupdate prune: a staged-binary name that is not a regular file — left alone", "name", name, "type", ent.Type().String())
+			continue
+		}
+		// name came from ReadDir and matched stagedNameRe, so it holds no path separator: this is one exact file.
+		if err := os.Remove(filepath.Join(dir, name)); err != nil {
+			logger.Warn("selfupdate prune: cannot remove an old staged binary — left for the next good update", "name", name, "err", err)
+			continue
+		}
+		logger.Debug("selfupdate prune: removed an old staged binary", "name", name)
+		removed = append(removed, name)
+	}
+	sort.Strings(removed)
+	logger.Info("selfupdate prune: the self-update folder holds what it needs", "kept", keep, "removed", len(removed))
+	return removed, nil
+}

internal/selfupdate/prune_test.go

@@ -0,0 +1,373 @@
+package selfupdate
+
+import (
+	"bytes"
+	"context"
+	"fmt"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"strings"
+	"testing"
+)
+
+// R-1368 part 2 — the self-update folder keeps what it needs and no more (prune.go).
+//
+// RED-PROOFS (each observed 2026-11-10, then restored):
+//   - drop the pruneStaged call from MaybeCommit → TestPrune_AfterNUpdatesTheFolderHoldsOneBinary fails with
+//     "after update 1 (0.170.0) the folder holds [… felhom-agent-0.169.0 felhom-agent-0.170.0 …]", and
+//     TestPrune_RealRollbackStillWorksAfterNUpdates fails too;
+//   - drop the pending-marker refusal from pruneStaged → TestPrune_RefusesWhileAnUpdateIsPending fails with
+//     "removed [...] while an update was pending";
+//   - drop the IsRegular check → TestPrune_AfterNUpdatesTheFolderHoldsOneBinary fails with "a symlink with a
+//     staged-binary name was removed".
+
+// committingWrapper is the guarded wrapper's `commit` as far as this package can see it: it clears the pending marker.
+type committingWrapper struct {
+	stateDir string
+	calls    int
+}
+
+func (w *committingWrapper) Run(_ context.Context, _ string, args ...string) ([]byte, []byte, error) {
+	if len(args) == 1 && args[0] == "commit" {
+		w.calls++
+		if err := os.Remove(pendingPath(w.stateDir)); err != nil {
+			return nil, []byte(err.Error()), err
+		}
+	}
+	return []byte("ok"), nil, nil
+}
+
+func stage(t *testing.T, stateDir, version string) {
+	t.Helper()
+	if err := os.MkdirAll(stagingDir(stateDir), 0o750); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(stagingDir(stateDir), "felhom-agent-"+version), []byte("binary "+version), 0o755); err != nil {
+		t.Fatal(err)
+	}
+}
+
+func folder(t *testing.T, stateDir string) []string {
+	t.Helper()
+	ents, err := os.ReadDir(stagingDir(stateDir))
+	if err != nil {
+		t.Fatal(err)
+	}
+	var out []string
+	for _, e := range ents {
+		out = append(out, e.Name())
+	}
+	sort.Strings(out)
+	return out
+}
+
+// The consequence: after N good updates the folder holds ONE staged binary — the running version's — and every file
+// that is not a staged binary is still there.
+func TestPrune_AfterNUpdatesTheFolderHoldsOneBinary(t *testing.T) {
+	stateDir := t.TempDir()
+	dir := stagingDir(stateDir)
+	if err := os.MkdirAll(dir, 0o750); err != nil {
+		t.Fatal(err)
+	}
+	// Bystanders: names this code does not know, and two things that carry a staged binary's NAME but are not files.
+	outside := filepath.Join(t.TempDir(), "precious")
+	if err := os.WriteFile(outside, []byte("not ours"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	for _, n := range []string{"notes.txt", "felhom-agent-0.1.0.partial", "felhom-agent.bak-0.139.0"} {
+		if err := os.WriteFile(filepath.Join(dir, n), []byte("x"), 0o644); err != nil {
+			t.Fatal(err)
+		}
+	}
+	if err := os.Mkdir(filepath.Join(dir, "felhom-agent-9.9.9"), 0o750); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.Symlink(outside, filepath.Join(dir, "felhom-agent-8.8.8")); err != nil {
+		t.Fatal(err)
+	}
+	bystanders := []string{"felhom-agent-0.1.0.partial", "felhom-agent-8.8.8", "felhom-agent-9.9.9", "felhom-agent.bak-0.139.0", "notes.txt"}
+
+	versions := []string{"0.170.0", "0.171.0", "0.172.0", "0.172.1", "0.173.0-rc.1", "0.173.0"}
+	prev := "0.169.0"
+	stage(t, stateDir, prev) // what an old agent left behind
+	for i, v := range versions {
+		stage(t, stateDir, v)                                                                                   // Execute downloaded it
+		writePending(t, stateDir, PendingMarker{OldVersion: prev, NewVersion: v, SHA256: "ab", AppliedAt: "t"}) // the wrapper flipped
+		wrap := &committingWrapper{stateDir: stateDir}
+		m := newMgr(t, stateDir, v, wrap, nil)
+		m.MaybeCommit(context.Background())
+		if wrap.calls != 1 {
+			t.Fatalf("update %d (%s): commit calls = %d, want 1", i+1, v, wrap.calls)
+		}
+		want := append([]string{"felhom-agent-" + v}, bystanders...)
+		sort.Strings(want)
+		got := folder(t, stateDir)
+		if strings.Join(got, " ") != strings.Join(want, " ") {
+			if _, err := os.Lstat(filepath.Join(dir, "felhom-agent-8.8.8")); err != nil {
+				t.Fatalf("a symlink with a staged-binary name was removed: %v", err)
+			}
+			t.Fatalf("after update %d (%s) the folder holds %v, want %v", i+1, v, got, want)
+		}
+		prev = v
+	}
+	if b, err := os.ReadFile(outside); err != nil || string(b) != "not ours" {
+		t.Fatalf("the file behind the symlink changed: %q %v", b, err)
+	}
+}
+
+// While an update is pending — the only time a rollback can fire — nothing is removed.
+func TestPrune_RefusesWhileAnUpdateIsPending(t *testing.T) {
+	stateDir := t.TempDir()
+	for _, v := range []string{"0.170.0", "0.171.0", "0.172.0"} {
+		stage(t, stateDir, v)
+	}
+	writePending(t, stateDir, PendingMarker{OldVersion: "0.171.0", NewVersion: "0.172.0", SHA256: "ab", AppliedAt: "t"})
+	before := folder(t, stateDir)
+
+	removed, err := pruneStaged(stateDir, "0.172.0", nil)
+	if err == nil || len(removed) != 0 {
+		t.Fatalf("removed %v while an update was pending (err=%v) — want a refusal", removed, err)
+	}
+	if got := folder(t, stateDir); strings.Join(got, " ") != strings.Join(before, " ") {
+		t.Fatalf("removed %v while an update was pending; folder now %v", before, got)
+	}
+
+	// The same through the commit path: a `commit` that reports success but leaves the marker (fakeWrapper does) must
+	// not lead to a prune.
+	m := newMgr(t, stateDir, "0.172.0", &fakeWrapper{}, nil)
+	m.MaybeCommit(context.Background())
+	if got := folder(t, stateDir); strings.Join(got, " ") != strings.Join(before, " ") {
+		t.Fatalf("the commit path pruned with the marker still there; folder now %v", got)
+	}
+
+	// A marker nobody can read is a refusal too.
+	if err := os.WriteFile(pendingPath(stateDir), []byte("{not json"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	if removed, err := pruneStaged(stateDir, "0.172.0", nil); err == nil || len(removed) != 0 {
+		t.Fatalf("an unreadable marker must refuse; removed %v err=%v", removed, err)
+	}
+}
+
+// Only a commit prunes: a plain start, a version mismatch and a failed commit remove nothing — so the binary of a
+// version that was rolled back stays for the operator.
+func TestPrune_OnlyAHealthyCommitPrunes(t *testing.T) {
+	cases := []struct {
+		name    string
+		pending *PendingMarker
+		running string
+		wrap    WrapperRunner
+	}{
+		{"plain start, no marker (also the state after a rollback)", nil, "0.171.0", &fakeWrapper{}},
+		{"marker names another version", &PendingMarker{OldVersion: "0.171.0", NewVersion: "0.172.0"}, "0.171.0", &fakeWrapper{}},
+		{"commit fails", &PendingMarker{OldVersion: "0.171.0", NewVersion: "0.172.0"}, "0.172.0", &fakeWrapper{err: fmt.Errorf("sudo: no")}},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			stateDir := t.TempDir()
+			for _, v := range []string{"0.170.0", "0.171.0", "0.172.0"} {
+				stage(t, stateDir, v)
+			}
+			if c.pending != nil {
+				writePending(t, stateDir, *c.pending)
+			}
+			before := folder(t, stateDir)
+			newMgr(t, stateDir, c.running, c.wrap, nil).MaybeCommit(context.Background())
+			if got := folder(t, stateDir); strings.Join(got, " ") != strings.Join(before, " ") {
+				t.Fatalf("folder changed without a healthy commit: %v → %v", before, got)
+			}
+		})
+	}
+	// A running version this code cannot name a file for: refuse, do not guess.
+	stateDir := t.TempDir()
+	stage(t, stateDir, "0.170.0")
+	if removed, err := pruneStaged(stateDir, "dev", nil); err == nil || len(removed) != 0 {
+		t.Fatalf("running version \"dev\" must refuse; removed %v err=%v", removed, err)
+	}
+}
+
+// The name rule and the version rule say the same thing: every name Execute can write is a name the prune knows, and
+// nothing else is.
+func TestPrune_NameShapeFollowsTheVersionRule(t *testing.T) {
+	for _, v := range []string{"0.70.1", "0.172.1", "10.0.0", "0.70.2-crash", "0.173.0-rc.1"} {
+		if !versionRe.MatchString(v) {
+			t.Fatalf("test input %q is not a version versionRe accepts", v)
+		}
+		if !stagedNameRe.MatchString("felhom-agent-" + v) {
+			t.Errorf("Execute can stage felhom-agent-%s but the prune does not know that name", v)
+		}
+	}
+	for _, n := range []string{"pending.json", "felhom-agent", "felhom-agent-", "felhom-agent-1.2", "felhom-agent-0.1.0.partial",
+		"felhom-agent.bak-0.139.0", "felhom-agent.prev", "felhom-agent-0.1.0/x", "../felhom-agent-0.1.0", "felhom-agent-0.1.0 ", "xfelhom-agent-0.1.0"} {
+		if stagedNameRe.MatchString(n) {
+			t.Errorf("the prune would treat %q as a staged binary", n)
+		}
+	}
+	m := stagedNameRe.FindStringSubmatch("felhom-agent-0.173.0-rc.1")
+	if m == nil || !versionRe.MatchString(m[1]) {
+		t.Fatalf("the version part of a staged name must satisfy versionRe; got %v", m)
+	}
+}
+
+func wrapperText(t *testing.T) string {
+	t.Helper()
+	b, err := os.ReadFile(filepath.Join("..", "..", "configs", "felhom-selfupdate-guarded"))
+	if err != nil {
+		t.Fatalf("the guarded wrapper is the other half of this proof and must be readable: %v", err)
+	}
+	return string(b)
+}
+
+// The stated dependency, pinned: the wrapper's `rollback` and `commit` read the pending marker and `.prev`, and no
+// staged binary. If either verb starts to read the staging folder, this fails and prune.go's rule must be looked at.
+func TestPrune_RollbackReadsNoStagedBinary(t *testing.T) {
+	s := wrapperText(t)
+	for _, want := range []string{"\nBIN=/usr/local/bin/felhom-agent\n", "\nPREV=$BIN.prev\n", "\nSTAGING=/var/lib/felhom-agent/selfupdate\n", "\nPENDING=$STAGING/pending.json\n"} {
+		if !strings.Contains(s, want) {
+			t.Fatalf("the wrapper no longer holds the line %q — the folder's rule (prune.go) was written against it", strings.TrimSpace(want))
+		}
+	}
+	for _, verb := range []string{"commit", "rollback"} {
+		i := strings.Index(s, "\n"+verb+")\n")
+		if i < 0 {
+			t.Fatalf("no %s) block in the wrapper", verb)
+		}
+		block := s[i:]
+		j := strings.Index(block, "\n    ;;\n")
+		if j < 0 {
+			t.Fatalf("no end of the %s) block", verb)
+		}
+		block = block[:j]
+		if !strings.Contains(block, "$PENDING") {
+			t.Fatalf("%s) does not read $PENDING any more", verb)
+		}
+		code := regexp.MustCompile(`(?m)^\s*#.*$`).ReplaceAllString(block, "")
+		for _, bad := range []string{"STAGING", "staged", "felhom-agent-"} {
+			if strings.Contains(code, bad) {
+				t.Fatalf("%s) now mentions %q — it may read a staged binary, and the prune removes those", verb, bad)
+			}
+		}
+	}
+	if stagedNameRe.MatchString("pending.json") || stagedNameRe.MatchString("felhom-agent.prev") {
+		t.Fatal("the prune would remove a file the rollback needs")
+	}
+}
+
+// The consequence, with the wrapper's own code: after N good updates (each committed by the REAL `commit` verb and
+// pruned), update N+1 goes bad and the REAL `rollback` verb puts the previous binary back. The wrapper runs from a copy
+// whose four fixed paths point into a temp folder; systemctl, logger and chown are stubs.
+func TestPrune_RealRollbackStillWorksAfterNUpdates(t *testing.T) {
+	sh, err := exec.LookPath("sh")
+	if err != nil {
+		t.Skip("no sh on this machine")
+	}
+	root := t.TempDir()
+	stateDir := filepath.Join(root, "state")
+	binDir := filepath.Join(root, "bin")
+	stubs := filepath.Join(root, "stubs")
+	for _, d := range []string{stagingDir(stateDir), binDir, stubs, filepath.Join(root, "rootstaging")} {
+		if err := os.MkdirAll(d, 0o750); err != nil {
+			t.Fatal(err)
+		}
+	}
+	live := filepath.Join(binDir, "felhom-agent")
+	s := wrapperText(t)
+	for old, repl := range map[string]string{
+		"\nBIN=/usr/local/bin/felhom-agent\n":                    "\nBIN=" + live + "\n",
+		"\nSTAGING=/var/lib/felhom-agent/selfupdate\n":           "\nSTAGING=" + stagingDir(stateDir) + "\n",
+		"\nROOT_STAGING=/var/lib/felhom-os-apply/agent-update\n": "\nROOT_STAGING=" + filepath.Join(root, "rootstaging") + "\n",
+	} {
+		if strings.Count(s, old) != 1 {
+			t.Fatalf("the wrapper's path line %q is not there exactly once — this test cannot sandbox it", strings.TrimSpace(old))
+		}
+		s = strings.Replace(s, old, repl, 1)
+	}
+	// Safety: no assignment in the copy may still point at a real system path (PREV and PENDING are built from BIN and
+	// STAGING, so they follow).
+	for _, line := range strings.Split(s, "\n") {
+		if regexp.MustCompile(`^[A-Z_]+=/`).MatchString(line) && !strings.Contains(line, root) {
+			t.Fatalf("the sandboxed wrapper still holds a real path: %q", line)
+		}
+	}
+	script := filepath.Join(root, "wrapper.sh")
+	if err := os.WriteFile(script, []byte(s), 0o755); err != nil {
+		t.Fatal(err)
+	}
+	stubLog := filepath.Join(root, "stub.log")
+	for _, n := range []string{"systemctl", "logger", "chown", "systemd-run"} {
+		body := "#!/bin/sh\necho \"" + n + " $*\" >> '" + stubLog + "'\nexit 0\n"
+		if err := os.WriteFile(filepath.Join(stubs, n), []byte(body), 0o755); err != nil {
+			t.Fatal(err)
+		}
+	}
+	run := func(verb string) (string, error) {
+		cmd := exec.Command(sh, script, verb)
+		cmd.Env = []string{"PATH=" + stubs + ":/usr/bin:/bin"}
+		out, err := cmd.CombinedOutput()
+		return string(out), err
+	}
+	// applied is what the wrapper's `apply` leaves behind (it needs root, so the test writes the same four things).
+	applied := func(oldV, newV string) {
+		cur, err := os.ReadFile(live)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if err := os.WriteFile(live+".prev", cur, 0o755); err != nil {
+			t.Fatal(err)
+		}
+		if err := os.WriteFile(live, []byte("binary "+newV), 0o755); err != nil {
+			t.Fatal(err)
+		}
+		writePending(t, stateDir, PendingMarker{OldVersion: oldV, NewVersion: newV, SHA256: "ab", AppliedAt: "t"})
+	}
+	if err := os.WriteFile(live, []byte("binary 0.169.0"), 0o755); err != nil {
+		t.Fatal(err)
+	}
+
+	prev := "0.169.0"
+	for _, v := range []string{"0.170.0", "0.171.0", "0.172.0", "0.172.1"} {
+		stage(t, stateDir, v)
+		applied(prev, v)
+		m := newMgr(t, stateDir, v, runnerFunc(func(args ...string) ([]byte, []byte, error) {
+			out, err := run(args[0])
+			return []byte(out), nil, err
+		}), nil)
+		m.MaybeCommit(context.Background()) // the REAL commit verb, then the prune
+		if got := folder(t, stateDir); len(got) != 1 || got[0] != "felhom-agent-"+v {
+			t.Fatalf("after the good update to %s the folder holds %v", v, got)
+		}
+		prev = v
+	}
+
+	// Update N+1 is applied and goes bad: systemd's OnFailure runs the wrapper's rollback.
+	stage(t, stateDir, "0.173.0")
+	applied("0.172.1", "0.173.0")
+	out, err := run("rollback")
+	if err != nil {
+		t.Fatalf("the real rollback verb failed after the prunes: %v\n%s", err, out)
+	}
+	if b, _ := os.ReadFile(live); string(b) != "binary 0.172.1" {
+		t.Fatalf("after the rollback the live binary is %q, want the previous version's", b)
+	}
+	if p, _ := readPending(stateDir); p != nil {
+		t.Fatal("the rollback left the pending marker")
+	}
+	if b, _ := os.ReadFile(stubLog); !bytes.Contains(b, []byte("systemctl restart felhom-agent.service")) {
+		t.Fatalf("the rollback did not restart the agent; stub log:\n%s", b)
+	}
+	// The rolled-back agent starts: no marker, so no commit and no prune — the bad version's binary stays to be read.
+	newMgr(t, stateDir, "0.172.1", &fakeWrapper{}, nil).MaybeCommit(context.Background())
+	if got := folder(t, stateDir); strings.Join(got, " ") != "felhom-agent-0.172.1 felhom-agent-0.173.0" {
+		t.Fatalf("after the rollback the folder holds %v, want the running version and the one that failed", got)
+	}
+}
+
+// runnerFunc adapts a function to WrapperRunner; it gets the verb arguments only.
+type runnerFunc func(args ...string) ([]byte, []byte, error)
+
+func (f runnerFunc) Run(_ context.Context, _ string, args ...string) ([]byte, []byte, error) {
+	return f(args...)
+}

scripts/retention-policy.json

@@ -24,10 +24,10 @@
   "readers": [
     "scripts/check-published-versions.py — asserts the newest N versions plus every pinned version"
   ],
-  "recorded": "2026-10-29",
-  "recorded_by": "CC 2026-10-29: day0_agent 0.169.0 (operator vouch 2026-10-29 14:06:09 CET, read from the hub log: agent=0.169.0, bundle 3ca33f9c… = the release; his browser got a 504 but the save went through); earlier 0.168.0 (vouch 10:21:56 CET); 0.167.0 (vouch 2026-10-27); 0.166.0 (vouch 2026-10-26 10:04:55 CET); D50 = A (operator 2026-10-25); earlier history: R-287 (2026-08-09), R-291 (2026-10-05)",
+  "recorded": "2026-11-03",
+  "recorded_by": "CC 2026-11-07: day0_agent 0.172.1 (operator vouch 2026-11-07 ~23:00 CET, read from the hub DB with its WAL: artifact_agent_version=0.172.1, bundle 3e7acaee… = the release); earlier 0.172.0 (CC 2026-11-03) (operator vouch 2026-11-03 10:18:10 CET, read from the hub log and DB: agent=0.172.0, bundle cc34b0bc… = the release); earlier 0.171.0 (CC 2026-10-30) (operator vouch 2026-10-30 10:47:36 CET, read from the hub log: agent=0.171.0, bundle 95f977f6… = the release; the save took 132 ms with hub 0.178.0, R-1346); earlier 0.170.0 (operator vouch 2026-10-29 18:57:48 CET, read from the hub log: agent=0.170.0, bundle 03f2ff64… = the release; the save took 38.3 s, R-1346); earlier 0.169.0 (operator vouch 2026-10-29 14:06:09 CET, read from the hub log: agent=0.169.0, bundle 3ca33f9c… = the release; his browser got a 504 but the save went through); earlier 0.168.0 (vouch 10:21:56 CET); 0.167.0 (vouch 2026-10-27); 0.166.0 (vouch 2026-10-26 10:04:55 CET); D50 = A (operator 2026-10-25); earlier history: R-287 (2026-08-09), R-291 (2026-10-05)",
   "pinned": {
     "hub_min_agent": "0.158.1",
-    "day0_agent": "0.169.0"
+    "day0_agent": "0.172.1"
   }
 }