os-apply: the host restart scan no longer hides lxc-start (skip ':/lxc/' not 'lxc'), and the host scans on every pass so a reboot clears 'reboot needed' (reboot_scanned reaches the hub) — both found live on demo-felhom

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS

.claude/rules/backup.md

@@ -0,0 +1,46 @@
+---
+paths: ["internal/backup/**", "internal/pbs/**", "internal/pbsdr/**", "internal/dr/**"]
+---
+
+# Backup, PBS and DR
+
+`internal/backup/` is the vzdump runner, restore-test scheduler and report store. `internal/pbs/` is
+the fingerprint-pinned PBS-API client plus the verify maintenance loop. `internal/pbsdr/` and
+`internal/dr/` carry the DR tier and recipe halves.
+
+## The three PBS laws
+
+1. **Set-only.** `pvesm remove` **DELETES the encryption key**. Re-apply configuration; never remove
+   and re-add a PBS storage to change it.
+2. **Secret on stdin.** A token secret is passed on stdin, never as an argv the process table shows.
+3. **Verify the pin BEFORE consuming the secret.** A fingerprint check after the secret has been sent
+   protects nothing.
+
+## Verify is server-side, and its default skips the work
+
+The agent drives verification **remotely** via the PBS API; `proxmox-backup-client` has **no** verify
+subcommand. `POST .../verify` defaults to **`ignore-verified=true`, which SKIPS already-verified
+snapshots** — send `ignore-verified=false` to actually re-read and detect corruption. A verify that
+skipped everything reports success.
+
+## Presence is not success
+
+A timestamp recording an **attempt** is not evidence of a **result**. Where a status field travels
+beside a timestamp, the verdict must consult **both** — or the timestamp must record only successes.
+Ask of any timestamp: *what exactly must have happened for this to be set?* If the answer is "we
+tried", it cannot answer "did it work".
+
+**Corollary:** when a verdict changes which field it counts from, the alarm text changes with it.
+Leaving a message reading `last run 8h ago` while alarming on a six-day-old **success** turns a true
+alarm into one the operator dismisses.
+
+## Prune is server-side now
+
+`DatastoreBackup` carries **no** `Datastore.Prune`. Boxes set `keep_last: 0` and the off-site endpoint
+runs the prune jobs. **Box tokens stay write-only — never widen that grant** (R-89).
+
+<!--
+The ignore-verified default is the sharpest instance of the "absent log line" class in this repo: a
+verify that silently skipped every snapshot completes fast, exits clean, and reports the same shape
+as one that read every byte.
+-->

.claude/rules/health-checks.md

@@ -0,0 +1,26 @@
+---
+paths: ["internal/capability/**", "internal/storage/**", "internal/localapi/**", "internal/hub/**", "internal/guesthook/**"]
+---
+
+# A health check issues no block I/O
+
+No `statfs`, no `getdents`, no read, write or `fsync` — **not even behind a timeout**.
+
+A probe that touches a wedged device enters uninterruptible sleep, survives `SIGKILL`, and cannot be
+recovered until the device returns or the host reboots — so `systemctl restart` hangs too. A timeout
+protects the caller's control flow and nothing else: the blocked thread remains.
+
+**Liveness is decided from `/proc` and the kernel's own state**, never by reading or writing the
+filesystem.
+
+<!--
+Measured, R-117 spike §6.3 (felhom.eu/documentation/audits/SPIKE-r117-bind-liveness-2026-07-30.md):
+a probe stayed in D state 3m50s after kill -9; a buffered write with no fsync blocked too (O_CREAT
+needs journal access); and statfs/getdents returned HEALTHY on a namespace that EIOs every byte —
+fast, and wrong.
+
+This rule used to be duplicated verbatim in felhom-agent/CLAUDE.md with a note explaining that
+felhom.eu/CLAUDE.md "does not load in an agent-only session". That reasoning was correct before
+path-scoped rules existed. The single source is now felhom.eu/CLAUDE.md "Code quality rules"; this
+file is the scoped copy that loads exactly where health checks are written. (2026-08-06)
+-->

.claude/rules/localapi.md

@@ -0,0 +1,44 @@
+---
+paths: ["internal/localapi/**", "internal/authz/**", "internal/guesthook/**"]
+---
+
+# Local API, authz and guest hooks — the per-guest blast radius
+
+`internal/localapi/` is the narrow per-guest local API: token store, disks/format, guest binds,
+controller swap, stale-lock recovery, pinned self-signed leaf. `internal/authz/` is the operator
+signed-op verifier (SSHSIG) plus the durable nonce store. `internal/guesthook/` installs the
+pre-start self-heal hookscript.
+
+> **Overlap note:** `health-checks.md` also matches `internal/localapi/**` and
+> `internal/guesthook/**`. That is deliberate — both rules apply there and both load. Neither
+> supersedes the other.
+
+## Scoping is the whole security property
+
+This API is reachable **from inside a customer guest**. Every route must be scoped to the guest that
+called it — a route that can name another guest's id has escaped its blast radius. Fail **safe to
+protected**: an unrecognised or unresolvable caller gets less access, never more.
+
+## Replay protection must survive a restart
+
+**`authz.MemoryNonceStore` on a real host is a defect** — replay protection dies on restart. Use
+`authz.FileNonceStore`. The memory store exists for tests.
+
+## The token is a hash on disk, plaintext only at mint
+
+The store keeps **hashes**. The plaintext token exists in exactly one place, `bootstrap.json` on the
+PVE host — so a "read the token" step means reading that file, and a lost token is re-minted, never
+recovered.
+
+## Binds can brick guest boot
+
+| Do not | Because | Use |
+|---|---|---|
+| `GuestBinder.AttachBind`/`DetachBind` (per-drive `pct set -mpN`) | legacy model; a missing bind source can **brick guest boot** (C1) | `AttachDrive`/`DetachDrive` (intermediary model) |
+| `isHostMountpoint` to reconcile bind state | a boolean cannot converge stacked double-binds (the `/mnt` doubling bug) | `countHostMounts` normalization inside `AttachDrive` |
+
+<!--
+Why fail-safe-to-protected rather than fail-closed: this API also carries the recovery paths. A hard
+refusal on an unresolvable caller would make a half-broken guest unrecoverable through the very
+interface built to recover it. Less access, never none.
+-->

.claude/rules/proxmox.md

@@ -0,0 +1,44 @@
+---
+paths: ["internal/proxmox/**", "internal/reconcile/**", "internal/signedjobs/**"]
+---
+
+# Proxmox — the API contract, and how destructive work is gated
+
+`internal/proxmox/` is the API-first `Client` plus the fenced root-CLI `Privileged`.
+`internal/reconcile/` is the reconcile engine, reversibility gate, op journal and crash recovery.
+`internal/signedjobs/` holds the operator-signed destructive executors (wipe, decommission).
+
+## A 200 on the POST is not success
+
+**Every mutating op is async**: it returns a **UPID**, and `WaitTask` must assert
+`exitstatus == "OK"`. Authorization can fail at *task execution* long after the HTTP call returned
+200. Treating the POST's status as the result is how a failed destroy reads as a successful one.
+
+## The privsep token gotcha
+
+A `--privsep 1` token's rights are the **intersection** of the backing user's permissions **and** the
+token's own ACLs. The role must be granted on **both** or every call 403s. The same intersection rule
+bites on PBS (`token ∩ user`).
+
+## TLS
+
+**SHA-256 leaf-cert pinning** against the self-signed host cert. **No insecure default**, ever. The
+pin is the raw leaf-DER sha — the SAN is never checked, so a cert rotation changes the pin and the
+agent must be re-pinned.
+
+## The destructive path — never the direct call
+
+| Do not | Because | Use |
+|---|---|---|
+| `Client.DestroyLXC` / `Vzdump` / `SetConfig` ad-hoc | skips classification, signature, per-guest serialization, crash recovery | `reconcile.Engine` paths / `RunSignedJob`; queue via `Queue.Submit` |
+| add a method to `proxmox.Privileged` | breaks the 3-exception root-CLI fence (`routing_test.go`) | `proxmox.Runner` + a new sudoers `Cmnd_Alias` + `validate.go`-style checks |
+| treat `ListLXC` output as "guests we own" | audit A1 — pre-v0.62.0 the stale-lock reaper did exactly this, contained only by the pool-scoped token | intersect with `Client.Pool` membership (`staleLockController.Guests()`); **fail safe on read failure** |
+
+Full trap table: `REUSE.md` §3. Every guest joins the `felhom` pool — `VM.Audit` comes from the
+`/pool` grant, not from a per-guest ACL.
+
+<!--
+The fence is not stylistic. It is what makes this component auditable: two types, one of which can
+only speak HTTP and one of which can only shell out, with a test asserting neither crosses. A single
+convenience method on Privileged that also makes an HTTP call would end that property silently.
+-->

.claude/rules/storage.md

@@ -0,0 +1,49 @@
+---
+paths: ["internal/storage/**", "internal/escrow/**"]
+---
+
+# Storage and escrow — format safety and zero-knowledge recovery
+
+`internal/storage/` is the storage observer, durable IDs, role/claim classifiers, `SudoHostOps` and
+the watchdog. `internal/escrow/` is the PBS-key escrow with its zero-knowledge recovery code.
+
+> **Overlap note:** `health-checks.md` also matches `internal/storage/**`. Deliberate — both rules
+> apply there and both load.
+
+## Never format the device you inspected
+
+**AGENT-001 is a TOCTOU:** acting on the caller's `req.Device` (or any remembered `/dev` path) after
+inspection lets `/dev` re-enumeration retarget the node to a **different physical disk**. Format the
+**re-resolved** device — `Server.reresolveWipe` / `reresolveBlank`.
+
+**Never exec raw `mkfs.*`** (including `Binaries.MkfsExt4`/`MkfsXfs`): sudoers no longer allowlists
+raw mkfs, and going direct bypasses the claim filter and the wrapper's re-checks. Use
+`SudoHostOps.Format`, which routes through `felhom-mkfs-guarded`.
+
+## The two durable-ID schemes refuse each other
+
+They are not interchangeable, and each returns a `binding_mismatch` for the other's scheme:
+
+| Purpose | Scheme | Resolver |
+|---|---|---|
+| wipe confirmation | `byid:` / `byuuid:` | `ResolveDurableDevice`, `DiskInfo.WipeDurableID` |
+| enrolled-storage remount | `uuid:` | `ResolveStorageDevice` |
+
+Using `DiskInfo.DurableID` (a `uuid:`) as a wipe-confirmation id is F20-BUG2.
+
+## Drive data is never taken by force
+
+Plain `umount` only — **never `-l`, never `-f`**, and never any format operation under
+`/mnt/felhom-drives`.
+
+## Escrow is zero-knowledge, and a fetch failure is not a wrong code
+
+The server holds no client key; a no-key restore fails with `missing key`. **A fetch failure must
+never be reported as a wrong recovery code** — that told a customer their correct code was bad, in
+hundredths of a second, when checking a code actually takes about one. Distinguish "we could not
+reach the store" from "the code did not match", always.
+
+<!--
+The escrow recovery-code "flake" was a REAL defect, not a flake. "Known flake, re-run" needs evidence
+before it is said out loud — that phrase cost this project a real finding once.
+-->

.gitea/workflows/gates.yml

@@ -0,0 +1,116 @@
+# gates — re-run this repo's gate entry point on every push, on a machine that does not care who
+# pushed or what they typed.
+#
+#   *** THIS REPORTS. IT CANNOT REFUSE. ***
+#
+# felhom repos push straight to `main` with no pull request, so there is no merge for a status
+# check to stand at. The refusing half is `.githooks/pre-push`, which is local to a clone and which
+# `git push --no-verify` skips; this half is what notices when that happened. Neither half is the
+# whole thing, and both are named in felhom.eu documentation/backlog/OPEN-ITEMS.md R-168.
+#
+# NO `uses:` STEP ANYWHERE, deliberately: JavaScript actions need a node runtime in the runner, and
+# the runner is a host-mode container with python3 and git and nothing else (see
+# homelab-manifests/gitea-system/act-runner.yaml for why it is not privileged). Probe P3 measured
+# that a plain `git fetch` of the pushed SHA from the in-cluster Gitea service is enough.
+#
+# A failing run must reach a person — a detector nobody hears is the defect R-29 filed, rebuilt one
+# layer up. That is the last step, and it runs ONLY on failure.
+name: gates
+on: [push]
+
+jobs:
+  gates:
+    runs-on: felhom-gates
+    steps:
+      - name: Fetch the pushed commit and the sibling clone it needs
+        # This repo's entry point invokes a SHARED checker that lives in the felhom.eu clone next
+        # door and is deliberately never copied here — so CI has to reproduce the workspace's
+        # sibling layout or the gate fails closed with "gate is MISSING". The sibling is also
+        # needed for CONTENT: this repo's REUSE.md cites a path that lives in the hub.
+        run: |
+          # Shallow, and pinned to the exact SHA that was pushed — not to the branch tip,
+          # which can move under us if two pushes race.
+          mkdir -p ws/felhom-agent
+          cd ws/felhom-agent
+          git init -q .
+          git remote add origin http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom-agent.git
+          git fetch -q --depth 1 origin "$GITHUB_SHA"
+          git checkout -q FETCH_HEAD
+          echo "checked out $(git rev-parse HEAD)"
+          cd .. && git clone -q --depth 1 http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git felhom.eu
+          echo "sibling felhom.eu present at $(cd felhom.eu && git rev-parse --short HEAD)"
+
+      - name: Run the gate entry point
+        # The ONLY thing CI runs. No go build, no go test, no linting, no deploy. The
+        # exit code IS the result: no `|| true`, no pipe that could swallow it.
+        #
+        # THE FULL SET, NOT `--fast` (R-115, 2026-08-03). `--fast` means "no network and no
+        # container runtime" and exists for `.githooks/pre-push`, where a push must not fail
+        # because Gitea blinked or because someone is on a train. CI is the opposite machine: it
+        # has the network, it is not in anyone's way, and it is the half that emails. The
+        # published-versions gate — the R-115 mechanism, which asks Gitea whether a released
+        # version can actually be downloaded — is network-bound and therefore runs ONLY here.
+        # Leaving `--fast` in place would have registered that gate and never run it, which is the
+        # built-but-never-wired failure this project has shipped four times.
+        env:
+          # In-cluster, so the check does not depend on public DNS or the ingress TLS chain.
+          GITEA_BASE: http://gitea.gitea-system.svc.cluster.local:3000
+        run: cd ws/felhom-agent && python3 scripts/agent_gates.py
+
+      - name: Alarm on failure
+        # THE POINT OF THE WHOLE THING. Probe P5 measured that a failed run produces NO mail, NO
+        # notification row and NO log line from Gitea itself — a red tick in a web UI nobody watches
+        # is exactly the shape R-29 filed against. So the run sends its own alarm, on the project's
+        # existing transactional path (Resend, the same one the hub uses), and prints the provider's
+        # accepted id so "a message left the machine" is an observable, not an assumption.
+        #
+        # Pure python3 and urllib, NOT curl: the runner image carries python3 and git and nothing
+        # else on purpose, and the first version of this step died on `curl: command not found`.
+        # Reaching for a bigger image to send one HTTP request would have been the wrong trade.
+        if: failure()
+        env:
+          RESEND_API_KEY: ${{ secrets.RESEND_API_KEY }}
+        run: |
+          python3 - <<'PY'
+          import json, os, sys, urllib.request, urllib.error
+
+          key = os.environ.get("RESEND_API_KEY", "")
+          if not key:
+              sys.exit("ALARM FAILED: RESEND_API_KEY is empty — the alarm cannot be sent, and a "
+                       "silent alarm is worse than none. Set the user-level Actions secret.")
+
+          repo = os.environ.get("GITHUB_REPOSITORY", "?")
+          sha  = os.environ.get("GITHUB_SHA", "?")
+          run  = os.environ.get("GITHUB_RUN_NUMBER", "?")
+          srv  = os.environ.get("GITHUB_SERVER_URL", "https://gitea.dooplex.hu")
+
+          body = json.dumps({
+              "from": "Felhom CI <monitoring@felhom.eu>",
+              "to": ["admin@felhom.eu"],
+              "subject": "[felhom CI] gates FAILED in %s" % repo,
+              "text": (
+                  "The gate entry point exited non-zero.\n\n"
+                  "Repository : %s\n"
+                  "Commit     : %s\n"
+                  "Run        : %s/%s/actions/runs/%s\n\n"
+                  "The failing gate names itself in the run log.\n\n"
+                  "If the local pre-push hook was GREEN for this commit, then CI and the hook\n"
+                  "disagree - that is a finding about the gates themselves, not about CI, and it\n"
+                  "outranks whatever the push was for.\n"
+              ) % (repo, sha, srv, repo, run),
+          }).encode()
+
+          req = urllib.request.Request(
+              "https://api.resend.com/emails", data=body, method="POST",
+              headers={"Authorization": "Bearer %s" % key,
+                       "Content-Type": "application/json",
+                       # Cloudflare fronts api.resend.com and BLOCKS the default
+                       # "Python-urllib/3.x" agent with its own 403 (error 1010) — which looks
+                       # exactly like an auth failure and is not one. Measured 2026-08-02.
+                       "User-Agent": "felhom-ci/1.0"})
+          try:
+              with urllib.request.urlopen(req, timeout=30) as r:
+                  print("RESEND-ACCEPTED id=%s" % json.load(r)["id"])
+          except urllib.error.HTTPError as e:
+              sys.exit("ALARM FAILED: Resend returned HTTP %s: %s" % (e.code, e.read().decode()[:300]))
+          PY

.githooks/pre-push

@@ -0,0 +1,82 @@
+#!/bin/sh
+# pre-push — refuse a push that carries a broken gate. (2026-08-02, R-29 leg (b) first half.)
+#
+# Runs this repo's ONE gate entry point in --fast mode: only checks that touch no network and no
+# container runtime, so a push stays a push and never pulls images or starts containers. The slow
+# gates stay deliberate periodic runs; a hook that takes minutes gets bypassed within a week and
+# the bypass becomes the habit.
+#
+# BOTH LINES BELOW ARE DELIBERATE. An absent log line is not evidence a hook ran — a silent pass is
+# equally consistent with "gates green" and "hook never fired", so a passing push says so out loud.
+#
+# HONEST LIMITS, stated so this is not mistaken for enforcement it cannot provide:
+#   * per-clone   — core.hooksPath is local config and a clone does not carry it. Arm a clone once:
+#                     git config core.hooksPath .githooks
+#                   Any manual entry-point run WARNS when the clone is unarmed.
+#   * skippable   — `git push --no-verify` bypasses this entirely. That is on purpose: an escape
+#                   hatch that cannot be reached is one that gets removed the first time it is
+#                   inconvenient. USING IT MUST BE STATED IN THE SESSION REPORT.
+# The half that is neither per-clone nor skippable is CI — felhom.eu OPEN-ITEMS.md R-168.
+#
+# Measured 2026-08-02 (git 2.47.3): a relative core.hooksPath resolves correctly and the hook's cwd
+# is the repo root whether `git push` is issued from the root or from any subdirectory. The
+# explicit rev-parse below does not depend on that.
+set -u
+
+root=$(git rev-parse --show-toplevel 2>/dev/null) || {
+    echo "pre-push: FAIL - cannot resolve the repo root (git rev-parse --show-toplevel)." >&2
+    exit 1
+}
+cd "$root" || exit 1
+
+# ── WORKSPACE-ROOT ASSERTION (2026-08-05, R-204 rider) ───────────────────────────────────────────
+# Refuse a push from a clone outside the felhom workspace.
+#
+# WHY THIS IS A HOOK AND NOT A LINE IN A DOCUMENT: the workspace root is ALREADY written down, in
+# documentation/runbooks/workspace-CLAUDE.md and in the workspace-root CLAUDE.md ("stay inside it"),
+# and work drifted into a home directory anyway. A rule that has failed once as a reminder is not
+# fixed by writing it down again — it has to be asserted where it can bite.
+#
+# A PUSH IS THE RIGHT TRIGGER, deliberately: throwaway clones under /tmp for probes and red-proofs
+# never push, so nothing legitimate breaks. Reads and builds elsewhere stay unaffected.
+#
+# Symlinks are resolved on BOTH sides before comparison, so a symlinked path neither falsely passes
+# nor falsely fails. If the workspace root does not exist on this machine the check is SKIPPED, not
+# failed — this hook must not brick a legitimate clone on a different host.
+#
+# The only bypass is the documented `git push --no-verify`, whose use is already reportable.
+FELHOM_WORKSPACE_ROOT=/mnt/5_hdd/felhom.eu
+if [ -d "$FELHOM_WORKSPACE_ROOT" ]; then
+    ws_real=$(cd "$FELHOM_WORKSPACE_ROOT" 2>/dev/null && pwd -P) || ws_real=""
+    root_real=$(pwd -P) || root_real=""
+    if [ -n "$ws_real" ] && [ -n "$root_real" ]; then
+        case "$root_real/" in
+            "$ws_real"/*) : ;;  # inside the workspace — proceed
+            *)
+                echo "pre-push: PUSH REFUSED - this clone is OUTSIDE the felhom workspace." >&2
+                echo "          clone:    $root_real" >&2
+                echo "          expected: under $ws_real (repos live in $ws_real/git/<repo>)" >&2
+                echo "          Work in the workspace clone, or bypass with 'git push --no-verify'" >&2
+                echo "          and state that you did in the session report." >&2
+                exit 1
+                ;;
+        esac
+    fi
+fi
+
+if ! command -v python3 >/dev/null 2>&1; then
+    echo "pre-push: FAIL - python3 not found, so the gates CANNOT run. This is a failure, never a" >&2
+    echo "          pass by default. Install python3, or push with --no-verify and say so." >&2
+    exit 1
+fi
+
+echo "pre-push [felhom-agent]: running scripts/agent_gates.py --fast ..."
+python3 "scripts/agent_gates.py" --fast
+rc=$?
+if [ "$rc" -ne 0 ]; then
+    echo "pre-push [felhom-agent]: PUSH REFUSED - gates exited $rc. Fix the finding above, or bypass with" >&2
+    echo "          'git push --no-verify' and state that you did in the session report." >&2
+else
+    echo "pre-push [felhom-agent]: gates OK - push proceeding."
+fi
+exit $rc

.gitignore

@@ -9,3 +9,6 @@
 
 # go
 /vendor/
+
+# Python bytecode written by configs/test_felhom_os_apply.py
+configs/__pycache__/

CLAUDE.md

@@ -1,151 +1,111 @@
 # CLAUDE.md — `felhom-agent`
 
-> Loads when Claude Code touches this repo. Stable orientation only — **current state lives in
-> `CONTEXT.md` and the top of `CHANGELOG.md`**, never here. Cross-repo orientation: workspace-root
-> `/mnt/5_hdd/felhom.eu/git/CLAUDE.md`.
+> Stable orientation only — **current state lives in `CONTEXT.md` and the top of `CHANGELOG.md`**,
+> never here. Cross-repo conventions (artifact taxonomy, access, clean-tree gate, secrets,
+> CHANGELOG/REPORT): workspace-root `/mnt/5_hdd/felhom.eu/git/CLAUDE.md`. Path-scoped detail:
+> `.claude/rules/`.
 
 ## What this repo is
 
-`felhom-agent` is the operator-tier **host agent** that runs on each Proxmox host and owns **all**
-Proxmox interaction: provision/restore guests, host storage, backup/restore orchestration, the hub
-control loop, and a narrow per-guest local API. It is the **most privilege-sensitive** component.
+The operator-tier **host agent**, one per Proxmox host, owning **all** Proxmox interaction:
+provision/restore guests, host storage, backup/restore orchestration, the hub control loop, and a
+narrow per-guest local API. It is the **most privilege-sensitive component in the system**.
 
-- Renamed former `proxmox-controller` repo.
-- **Distinct from `felhom-controller`** — that is the *in-guest* controller (Docker-only, no Proxmox
-  creds). Do not confuse them.
-- Control plane, not data plane: if the agent dies, apps keep serving; only management degrades.
+- Renamed from `proxmox-controller`.
+- **Distinct from `felhom-controller`** — that is the *in-guest* controller, Docker-only, holding no
+  Proxmox credentials. Do not confuse them.
+- **Control plane, not data plane:** if the agent dies, apps keep serving; only management degrades.
+- Pure Go stdlib + `golang.org/x/crypto`. No web frameworks.
 
-## Read before writing code
+## Doing X → read Y
 
-- **`REUSE.md`** — canonical helpers, format-safety guards, traps, seams. Check it first; update it
-  in the same commit that changes a shared helper or pattern.
-- `CONTEXT.md` (current state + open threads) and the top `CHANGELOG.md` entry (authoritative history).
-- Design doc: `felhom.eu/documentation/architecture/03-host-agent.md` (locked). Platform facts:
-  `felhom.eu/documentation/proxmox-platform.md` + `tests/phase{0,1-2,3,4}-findings.md`.
+| Doing | Read |
+|---|---|
+| writing any new code | `REUSE.md` — helpers, format-safety guards, traps, seams |
+| needing current state / open threads | `CONTEXT.md` + the top `CHANGELOG.md` entry |
+| Proxmox, reconcile or signed jobs | loads itself: `.claude/rules/proxmox.md` |
+| local API, authz or guest hooks | loads itself: `.claude/rules/localapi.md` |
+| backup, PBS or DR | loads itself: `.claude/rules/backup.md` |
+| storage or escrow | loads itself: `.claude/rules/storage.md` |
+| writing a health check | loads itself: `.claude/rules/health-checks.md` |
+| **release, build, publish, deploy, verify a version** | the **`felhom-build-deploy`** skill — **never hand-roll it** |
+| writing or reviewing a test, fixing a bug | the **`felhom-testing`** skill |
+| host addresses, break-glass, node facts | `felhom.eu/documentation/operations/nodes.md` — never restate them |
+| which box may I break | `felhom.eu/documentation/runbooks/target-selection.md` |
+| what version is live anywhere | ask the hub (`/hosts`, `/configs`) or the box — **never a doc** |
+| the authoritative design | `felhom.eu/documentation/architecture/03-host-agent.md` (locked) |
 
-## Layout (verified against the tree)
+## The root-CLI fence — API-first, exactly three exceptions
 
-```
-cmd/felhom-agent/    main + flags + --selftest modes + the daemon entry
-cmd/felhom-opsign/   offline operator signing CLI (SSHSIG)
-internal/authz/      operator signed-op verifier (SSHSIG) + durable FileNonceStore
-internal/backup/     vzdump backup runner + restore-test scheduler + report store
-internal/capability/ live sudo-policy capability probe (degradation visibility)
-internal/config/     JSON config + FELHOM_AGENT_* env overlay; secrets redacted (Redacted())
-internal/desired/    hub desired-state syncer (envelope observer)
-internal/escrow/     PBS-key escrow (zero-knowledge recovery code)
-internal/guesthook/  pre-start self-heal hookscript install
-internal/hub/        daemon: HostReport collector + Bearer client + resilient Loop
-internal/lanresolver/ split-horizon DNS on guest IP change (dnsmasq RESTART, not reload)
-internal/localapi/   per-guest local API: token store, disks/format, guest binds, controller swap,
-                     stale-lock recovery, pinned self-signed leaf
-internal/log/        slog setup
-internal/pbs/        PBS-API client (fingerprint-pinned) + verify maintenance loop
-internal/provision/  guest bootstrap back-half (token mint → bootstrap.json → pct bind)
-internal/proxmox/    API-first Client + fenced root-CLI Privileged + UPID WaitTask
-internal/reconcile/  reconcile engine + reversibility gate + op journal + crash recovery
-internal/signedjobs/ operator-signed destructive executors (wipe, decommission)
-internal/storage/    storage observer + durable ids + role/claim classifiers + SudoHostOps + watchdog
-```
-
-## Build / run
-
-- Module `gitea.dooplex.hu/admin/felhom-agent`; binary `felhom-agent` (`cmd/felhom-agent/`).
-- **Pure Go stdlib + `golang.org/x/crypto` only** — no web frameworks. `go.mod` directive go 1.25.0;
-  DooPlex (192.168.0.180, where CC runs) has the Go toolchain and is on the same LAN as the demo
-  host — build and run live tests locally.
-- Version via `-ldflags "-X main.version=<v>"`; `--version` flag. Bump on meaningful changes + CHANGELOG entry.
-- **Full build/deploy/publish runbook: use the `felhom-build-deploy` skill.** Summary:
-
-> **Clean-tree gate before any build:** `git status --porcelain` must be empty and
-> `git rev-parse HEAD` must equal `git rev-parse origin/main` in the repo being built. An unpushed
-> change does not exist — never build a dirty or unpushed tree. The `git pull` in the build step
-> stays (it is a no-op when you work in this tree, and load-bearing if anything was pushed from
-> elsewhere).
-
-| Step | Where | One-liner |
-|---|---|---|
-| Build | DooPlex (local) | `cd /mnt/5_hdd/felhom.eu/git/felhom-agent && git pull && go build -ldflags '-X main.version=<v>' -o /tmp/felhom-agent-<v> ./cmd/felhom-agent` |
-| Copy | local → felhom-pve | `scp /tmp/felhom-agent-<v> felhom-pve:/tmp/` (one hop) |
-| Deploy | felhom-pve | backup `.bak-<old>` → `install -m0755` → `systemctl restart felhom-agent` (non-root `felhom-agent` user, config `/etc/felhom-agent/agent.json`) |
-| Ship configs | felhom-pve | sudoers (`/etc/sudoers.d/felhom-agent`) + guarded-mkfs wrapper WITH the binary when `configs/` changed |
-| Publish | DooPlex (local) | `scripts/publish-agent.sh <ver> <bin>` (REGISTRY_* creds); hub Day-0 manifest vouch = operator follow-up |
-| Verify | felhom-pve | `felhom-agent --version` + journal (clean ReassertGuestBinds, no capability degradation) |
-
-## Proxmox model (the load-bearing rules)
+This is in the core because breaching it is how this component stops being auditable.
 
 - **API-first** via a scoped `FelhomAgent` token. Raw root-CLI is **fenced to exactly 3 exceptions**:
-  keyctl `pct create` (golden image), USB mount/fstab, SMART/sensors. `Client` never shells out;
-  `Privileged` never makes HTTP calls (asserted by `routing_test.go`). Keep that fence.
-- **Every mutating op is async** → returns a UPID → `WaitTask` asserts `exitstatus == "OK"`. A 200 on
-  the POST is **not** success; authorization can fail at task execution.
-- **TLS:** SHA-256 leaf-cert pinning (self-signed host cert). No insecure default.
-- **Privsep token gotcha:** a `--privsep 1` token's rights = intersection of the backing user's perms
-  AND the token's ACLs — the role must be granted on **both**, or every call 403s.
-- Destructive ops go through the reconcile gate / signed-jobs path — never call `Client.DestroyLXC`/
-  `Vzdump`/`SetConfig` ad-hoc (REUSE.md §3).
+  keyctl `pct create` (golden image), USB mount/fstab, SMART/sensors.
+- **`Client` never shells out; `Privileged` never makes HTTP calls** — asserted by `routing_test.go`.
+  Adding a method to `proxmox.Privileged` breaks the fence; use `proxmox.Runner` plus a new sudoers
+  `Cmnd_Alias` and `validate.go`-style checks (`REUSE.md` §3).
+- **Destructive ops go through the reconcile gate / signed-jobs path.** Never call
+  `Client.DestroyLXC` / `Vzdump` / `SetConfig` ad-hoc — that skips classification, signature,
+  per-guest serialization and crash recovery.
+- **Ownership must be PROVEN, never assumed.** A raw `ListLXC` list is not "guests the agent owns";
+  intersect with `Client.Pool` membership and fail safe on a read failure (audit A1).
 
-## Demo host (for live tests)
+## Gates — ONE entry point
 
-Node **`demo-felhom`**, API `https://192.168.0.162:8006`. SSH alias `felhom-pve` (root@pam) —
-available to CC as plain `ssh felhom-pve`. The agent pins the served leaf cert — verify the
-fingerprint still matches before a live run. Selftest modes (run locally on DooPlex, pointed at the
-demo API): `--selftest[=read|task|hub|storage|backup|restore-test|pbs-verify]`; no flag = the daemon.
+**Run `python3 scripts/agent_gates.py` from the repo root after ANY change here.** It runs this
+repo's gates — `reuse_refs_check` and `instructions_gate`, both the **shared** copies in
+`felhom.eu/scripts/`, never copied into this repo (a copy recreates the drift they detect; an absent
+sibling clone FAILS). `--fast` selects the gates touching no network and no container runtime; today
+that is all of them. **A missing gate is a FAILURE, never a skip.**
 
-> **TEMPORARY — felhom-pve is at a remote site (until ~2026-08-02).** The home-LAN literal
-> `192.168.0.162` is NOT reachable from DooPlex for the duration. Access via Tailscale:
-> felhom-pve = 100.70.170.35; the `Host felhom-pve` entry in `~/.ssh/config` on DooPlex already
-> points there (the direct-LAN path stays available as `Host felhom-pve-lan`). Delete this block on
-> return. All documented `ssh felhom-pve` / `pct exec` workflows are unchanged. Path is **direct**
-> (not DERP), ~37 ms rtt per hop. At the remote site the host is on **DHCP** and currently holds
-> `192.168.0.147` — so the PVE API is at `https://192.168.0.147:8006` there, and **the agent does
-> not run at all**: `localapi` binds the literal `192.168.0.162` → `bind: cannot assign requested
-> address` → the service is `failed` and has never started at the remote site. Fixing it means
-> editing `listen_addr` in `/etc/felhom-agent/agent.json` **and** the guest's bootstrap endpoint
-> (plus the leaf-cert SAN the controller pins) — Viktor GO required. Details + findings:
-> `felhom.eu/documentation/audits/AUDIT-vacation-remote-ops-2026-07-20.md`
+**The pre-push hook** (`.githooks/pre-push`) runs it with `--fast` and refuses a failing push. It is
+**per-clone** — switch it on once with `git config core.hooksPath .githooks`, and a manual run WARNS
+when this clone is unarmed. `git push --no-verify` bypasses it deliberately; **say so in the session
+report when you use it** — CI re-runs the same entry point on every push and **emails the operator on
+failure**, so a bypass is noticed even though it is not blocked (R-168, CLOSED 2026-08-02).
 
-> **Legacy: Windows workstation.** Until 2026-07-19 CC ran on Windows 11; `pct` commands over SSH
-> needed `export MSYS_NO_PATHCONV=1`, and every remote command used
-> `SSH=/c/Windows/System32/OpenSSH/ssh.exe`. Agent deploy was a two-hop copy via the Windows box
-> (`cygpath -w` for the local scp path; CRLF hazard on config files).
+<!--
+WHY ONE ENTRY POINT (2026-08-02, R-29): a census of all gates across the four repos found every check
+a CLAUDE.md names was passing, and two of the four nobody is told to run were failing. This repo was
+the extreme case — nothing ran against it at all, and 90 cited paths were checked by no one.
+-->
+
+## Live validation — the fence
+
+Exercise the **SERVER-SIDE PIPELINE** a real user triggers, end-to-end. **The forbidden shortcut is
+BYPASSING it** — the F9 episode was a raw guest-attach with hand-set state, and it proved nothing.
+
+`claude-in-chrome` is NOT available on DooPlex. Invoking the exact endpoint the UI invokes is an
+acceptable proxy — **say which method was used**. Low-level mechanism tests where the direct call IS
+the mechanism are exempt.
 
 ## Conventions
 
-### Trunk-based — no branches
-
-All shippable work commits **directly to `main`**; `main` equals what is deployed.
-- Report-only artifacts (audits, findings, fixspecs) → `felhom.eu/documentation/` (`audits/`, `backlog/`).
-- Risky/supervised fixes are spec'd, then implemented **during the supervised session, on `main`**.
-- Unattended escape hatch: if a fix can't be cleanly verified/shipped, revert + report — never park on a branch.
-
-> **In every repository where you make a change, update both files in that repo:**
-> - **`CHANGELOG.md`** — cumulative log, newest on top.
-> - **`REPORT.md`** — **overwrite** with the most recent implementation/validation summary only.
->
-> **Never write secrets** into any committed file — reference them as "stored out-of-band".
-
-- Code quality: verify generated code for bugs/edge cases; add debug logging; **ask rather than
-  guess** when you'd otherwise invent input/output.
-- Update `REUSE.md` if you added/changed/deprecated a shared helper or pattern (same commit).
-- Testing doctrine (non-hollow tests, red-proofs, seams): use the `felhom-testing` skill.
-- **Logging**: the slog logger fans out to journald (configured level) + the always-DEBUG `applog.Ring`
-  (remote pulls) — English, keys-never-values, durations on outcomes; full rules in
+- **Trunk-based — no branches.** All shippable work commits directly to `main`; `main` equals what is
+  deployed. Report-only artifacts (audits, findings, fixspecs) go to `felhom.eu/documentation/`.
+- **Unattended escape hatch:** if a fix cannot be cleanly verified and shipped, **revert and report**
+  — never park it on a branch.
+- **Logging**: the slog logger fans out to journald (configured level) plus the always-DEBUG
+  `applog.Ring` (remote pulls). English, keys-never-values, durations on outcomes. Full rules:
   `felhom.eu/documentation/runbooks/logging-conventions.md`.
+- Update `REUSE.md` in the same commit that adds, changes or deprecates a shared helper or pattern.
 
-### Live validation
+## End-of-session checklist
 
-Exercise the SERVER-SIDE PIPELINE a real user triggers, end-to-end. The forbidden shortcut is
-BYPASSING it (the F9 episode: raw guest-attach + hand-set state). Invoking the exact endpoint the UI
-invokes is an acceptable proxy when a browser isn't available — say which method was used. Low-level
-mechanism tests where the direct call IS the mechanism are exempt.
+- **`CHANGELOG.md`** (cumulative, newest on top) and **`REPORT.md`** (overwritten with this run only)
+  — in every repo touched.
+- **`CONTEXT.md`** — decisions, state, what is next.
+- **`REUSE.md`** — if a shared helper or pattern moved.
+- **A finding goes in `felhom.eu/documentation/backlog/OPEN-ITEMS.md` first**, never only in a report
+  or an audit.
+- **Confirm your own last push's CI run went green, by run ID** — CI mails on failure, which is a PUSH
+  signal; this is the PULL check that catches a lost or unread mail. An unchecked green is an
+  assumption, not an observation.
 
-## Workflow & artifacts
-
-- Implement **`TASK.md` / `TASK-*.md`** specs (when placed as `TASK.md` or told to), then push +
-  CHANGELOG + REPORT.md.
-- **`RUNBOOK-*.md`** — an operational procedure. CC executes the steps it has access and capability
-  for, including live validation on the demo Proxmox host (CC has root@felhom-pve SSH + the
-  felhom-agent token). Mark a step HUMAN only when it genuinely needs physical presence, a real-world
-  decision, or credentials CC truly lacks. Judgment still applies: confirm before irreversible ops on
-  real customer data — demo scratch guests are fair game.
+**A gate ships with a decoy test that has been seen to fail (R-421).** A decoy is the LABEL without
+the FACT — a directory with the right name and no bake log, a note whose prose mentions the marker it
+lacks. `scripts/decoy_coverage_gate.py` refuses a new gate that has neither a decoy nor a named
+exemption carrying its row. The four shapes, the 2026-09-01 sweep that fooled 16 of 29 gates, and the
+decoys withdrawn as illegitimate: `documentation/audits/AUDIT-gate-decoys-2026-09-01.md` and
+`felhom-controller/.claude/rules/gates.md`. **Scope is a fact too** — prefer `os.walk` over
+`os.listdir`, and a glob over a hand-maintained list.

CONTEXT.md

@@ -1,10 +1,224 @@
 # CONTEXT — felhom-agent working state
 
+
+> **2026-09-25 night — v0.133.0 AND v0.134.0 DELIVERED to both demo boxes (CC-signed `agent_update`, ruling 1);
+> restore test back ON (the `-1` config kept as `agent.json.night-0925-off`). v0.134.0 = R-685:** `backup/runner.go`
+> `spaceFits` — a vzdump to a LOCAL target needs free ≥ newest archive of that guest × 1.25 + 1 GiB (PVE prunes
+> only after success); a shortfall is a named skip (`BackupSkipNoSpacePrefix`), fail-open on PBS / first backup /
+> unknown usage. **Free space comes from `NodeStorage` (`GET /nodes/<n>/storage`) — `ListStorage` (`GET /storage`)
+> has NO usage**; the first build read it and a real vzdump started in its own live test (aborted, no archive).
+> demo-hp: `local_backup_retention` 1 (operator option A, saved `agent.json.pre-a4-retention`). Peti's box: nothing.
+
+> **2026-09-24 — v0.133.0 RELEASED, NOT DELIVERED (R-672, R-673).** Restore-test space preflight
+> (`reconcile/restoretest_space.go`, `internal/restorespace`): uncompressed size from the vzdump log / PBS size,
+> × 1.2 + 5 GiB, thin metadata, off the tested guest's pool, unknown refuses, reported as `skipped` non-pass.
+> Janitor (`cmd/felhom-agent/janitor.go`) every 10 min: `Engine.RetryScratchTeardown` + stale-lock sweep under
+> the heavy-op gate. Thin pool ≥ 90 % → immediate report (hub v0.124.0 alarms). **Operator rulings 2026-09-24
+> (evening):** the scheduled restore-test is OFF on both demo hosts (`backup.restore_test_eval_interval_seconds:
+> -1` — note: 0 means the 6 h DEFAULT, only a negative disables) until v0.133.0 is delivered there; saved configs
+> `/etc/felhom-agent/agent.json.pre-r672`. demo-hp 9201 was repaired (stop, fsck, start; two Redis AOF tails cut).
 > Snapshot of the current state + open threads. Authoritative history lives in `CHANGELOG.md` (top
 > entry = current); the end-of-task detail lives in `REPORT.md`.
 
+## R-199 (v0.125.0) — links 6–8 of the recovery chain, assembled and walked
+
+`POST /escrow/recover-offsite-password` (pinned local API, `withGuest`): the controller supplies the
+customer's recovery code, the agent fetches THIS host's own sealed blob from the hub
+(`hub.Client.FetchIdentityEscrow` → `GET /hosts/{id}/escrow`, hub >= v0.94.0, self-scoped by the
+per-host key), unseals it via `escrow.OffsiteKeyRecoverer`, and returns **only** the offsite restic
+repository password plus its sha256.
+
+**Rules that must not erode:**
+- **Only that field.** Not the tunnel token, not the PBS token, not the WG key — the controller is a
+  trust tier down and needs none of them. Narrowing cost nothing and is not recoverable later.
+- **The unseal stays in the agent.** `age` is an agent runtime dependency (`/usr/bin/age` — hardcoded,
+  no config override; 1.2.1 on demo-felhom) and is deliberately absent from the controller image.
+- **R:** in memory for one call, cleared on the success path AND every failure path, never on disk,
+  never in argv, never logged at any level including inside an error, never echoed. Verified live: 0
+  log lines, 0 files, 0 leftover `felhom-idesc-*` dirs, with a positive control proving the search worked.
+- **Three distinct outcomes**, not one generic failure: no blob (404), a bundle that opens but predates
+  the field (409 — pre-fork-4, cannot be retro-fitted), a code that does not open it (400 — fail-closed
+  at age's KDF, nothing written).
+- **The wiring is pinned by an AST walk** (`cmd/felhom-agent/escrow_recover_wiring_test.go`):
+  `main` → `runDaemon` → `buildLocalAPIServer`, an `escrow.OffsiteKeyRecoverer` constructed there, the
+  `Options.EscrowRecovery` field present, and the fetcher calling the DAEMON's own `hubClient` (the
+  self-scoping that makes cross-host retrieval impossible is a property of WHICH key is used).
+  Links 6 and 7 were two of this project's six built-but-never-wired instances.
+
+**Proven live on demo-felhom 2026-08-04:** recovered sha256 == on-disk sha256 == the hub's stored hash.
+A wrong code five minutes earlier failed closed. **The chain stops at link 8** — nothing installs a
+recovered password, reopens a repository, or restores a file.
+
+**§8.6, fixed while here:** `runSelftestIdentityConsume`'s success line used to recite
+"tunnel_token + pbs_token", which became a misstatement when v0.77.0 sealed the repository password
+into the same bundle — anyone reading it would conclude the password was not there. It now names what
+THIS bundle carried and what it did not.
+
 ## Current
 
+- **2026-08-03 — v0.123.0 (R-185): a tier the box cannot READ now says so.** The agent's token had
+  `FelhomAgentStore` on `local`, `local-lvm`, `felhom-pbs` and **not** on `felhom-backup` — the
+  storage both demo boxes configure as `local_backup_target`. That storage answered `{"data":[]}`
+  through the token while root listed three archives, and `pickForThisRun` skipped it as *"no settled
+  archive yet"* — **which is what a brand-new tier reports**, so the host tier was never
+  restore-testable and nothing said so.
+  - **The permission question is asked directly**, because unlike the listing it has a definite
+    answer: `Client.Permissions` reads `/access/permissions?path=/storage/<target>` **as the agent's
+    own token**, and `storeGrantStatuses` emits one `capability.Status` per configured tier. It
+    composes AROUND the sudo prober, the way `poolReadStatus` already does — an API read does not
+    belong inside a sudo-policy probe. `Status`'s wire shape is untouched, so the hub's critical
+    degraded alert applies with **no hub change**.
+  - **MEASURED FIRST, and the obvious reading is wrong:** an ungranted path answers neither empty nor
+    403 — it carries the privileges INHERITED from the box-wide `/` grant
+    (`Sys.Audit, SDN.Use, Datastore.Audit`). Checking path-presence, or `Datastore.Audit`, reports a
+    blinded storage HEALTHY. The probe tests **`Datastore.AllocateSpace`**; re-measure before ever
+    changing that constant (`storeGrantRequiredPriv`, red-proved).
+  - **The probed set comes from `BackupTiers()`, never a fixed list** — a hardcoded probe list is the
+    defect reproduced inside the fix. Critical, EXCEPT the `local` fallback target (reported, but it
+    does not page). It never consults content, so it cannot alarm on a newborn tier; it never reports
+    ok when it could not ask.
+  - **LIVE:** degraded observed on the still-blind box (hub emailed `agent_capability_degraded`) →
+    grant applied on **both** demo boxes → token lists 3 and 4 archives → `ok=70 total=70 degraded=0`
+    and `degraded → ok` at the hub → **the host tier became a due-check candidate for the first time**,
+    correctly picking the 08-02 archive (08-03 had not settled 24 h).
+  - **The installer's real defect was NOT `PVE_STORAGES`** — see `felhom.eu` CONTEXT S-22: Case A
+    grants, the Scenario-F reuse arm did not. Fixed in installer **1.24.0** with a gate.
+
+- **2026-08-03 — v0.122.0 (R-189 · R-188 · R-186): three signals that lied about their own work.**
+  None touches data; all three cost attention, which every other signal depends on.
+  - **R-189 — a passing restore-test no longer vanishes on a restart.** `restore_tests[]` came only
+    from the in-memory `backup.Store` (*"lost on restart; the cadence re-populates"* — true under a
+    timer, FALSE since R-86, because the agent will not re-test a proven archive). **Observed live:**
+    a 14.5 GB offsite PASS at 15:25:14, agent restarted 2 m 43 s later, hub logged `0 restore-tests`
+    twice. `RestoreTestState` now stores `tier` + `verified` beside the archive (v3 shape; v1/v2
+    still read, and a record missing archive-or-tier is NOT reported), exposes
+    `ProvenRestoreTests`, and `Collector.SetProvenRestoreTests` merges it — **one entry per tier,
+    newest by `TestedAt` wins**, so a fresh failure beats a stored success and a tier never appears
+    twice. Wiring pinned by an AST test: the method this replaces (`Snapshot`) claimed a
+    "host-report gauge" in its doc comment and had **no caller** for weeks.
+  - **ONLY SUCCESSES ARE PERSISTED, and the reason is now in the code:** a success *suppresses*
+    future work (a proven archive is never re-tested, so a lost proof leaves the box quietly less
+    tested than it believes); a failure *causes* future work and heals itself at the next evaluation.
+  - **R-188 — the release stopped emailing false failures.** Only the tag PUSH moved (build → tag
+    locally → publish → push tag): the push is what wakes CI, and a tag visible before its package
+    made the gate correctly fail a correct release ~half the time. The old order's invariant is now
+    asserted directly — `check-published-versions.py` refuses a **published version with no tag**, as
+    a bounded, printed probe (the package listing api is still 401 without a token, re-measured).
+  - **R-186 — a released binary is verifiable.** `-trimpath -buildvcs=false`: same source → same
+    bytes whether or not the tag exists. Measured. `publish-agent.sh`'s fallback also forced
+    `CGO_ENABLED=0` and built a **74 KB different** binary for the same version — both paths now
+    identical. The verification command is in `CLAUDE.md`.
+
+- **2026-08-03 — v0.121.0 (R-86): the restore-test follows the BACKUP, not the clock.** The ticker is
+  now only the **evaluation interval**; a tier is **DUE** when its newest archive that has settled for
+  `settle` (default 24 h) **has not been proven**. Daily tier → proved daily on yesterday's archive;
+  weekly tier → weekly on its own; newborn → UNKNOWN. **The trap, so it is not reintroduced:** the
+  literal reading of R-86 — *"due when the newest archive is ≥ 24 h old"* — is NEVER true on a daily
+  tier (a new archive resets the age before it reaches the lag), so it switches restore-testing off
+  where it matters most. Red-proved at 0 runs over 5 simulated days.
+  - **The state now records WHICH archive was proven**, not just when a tier passed. A pre-R-86 file
+    keeps its time (ordering survives) and yields no proven archive → each tier is due once after the
+    upgrade, deliberately.
+  - **The old cadence key:** `restore_test_cadence_seconds` is DEPRECATED. Negative still DISABLES
+    (verbatim); a positive value now seeds the **settle lag** and the daemon WARNs once at start-up
+    naming `restore_test_eval_interval_seconds` (default 6 h) and `restore_test_settle_seconds`
+    (default 24 h). It is NOT carried into the evaluation interval.
+  - **6 h is bounded from both ends:** measured evaluation cost (local 18 ms, PBS-over-WAN 392 ms,
+    both 430 ms) says cost is irrelevant; the ceiling is that a FAILING tier stays due, so the
+    evaluation interval is also its retry interval for a multi-GB restore.
+  - The due-check now runs **before** the heavy-operation gate is taken (a frequent poll must not be
+    able to make a starting backup record a failure — F-A1), and the candidate picker skips archives
+    failing `archivePlausiblyComplete` (a phantom would be due forever and fail forever).
+  - New read-only `--selftest=restore-test-due` prints the per-tier verdict + its cost.
+  - **v0.121.1 — a quiet evaluation is AUDIBLE.** "Nothing is due" is now the NORMAL outcome, and at
+    DEBUG it was silent: an empty journal would have been equally consistent with a healthy loop and
+    a dead goroutine (standing rule 3 — the shape the R-88 watcher was retired for). A not-due
+    evaluation logs ONE INFO line naming every tier's verdict; an unlistable tier reads `UNKNOWN`
+    with its error in that same line.
+  - **PROVEN LIVE 2026-08-03 on demo-felhom:** due-triggered offsite restore-test of a 14.5 GB
+    encrypted PBS archive — restored, booted, verified, scratch destroyed, **635 s**; the state then
+    named that archive, a second evaluation ran nothing, and an agent restart ran nothing.
+  - **R-185 (filed, NOT fixed here):** on demo-felhom the agent token has no ACL on
+    `/storage/felhom-backup`, so its content listing comes back EMPTY (root sees 3 archives) — the
+    host tier has never been restore-testable there, and the due-check cannot distinguish that from
+    a newborn tier.
+
+- **2026-07-28 — v0.107.0: F-REBOOT fixed — a guest rebooted mid-backup now comes back by itself.**
+  New `internal/localapi/guestpower.go`: a 60 s watchdog that starts a guest which is `onboot:1`,
+  stopped, unlocked, and has no vzdump in flight. It closes the two narrow gaps that let
+  `RecoverStaleLockedGuests` miss campaign fault 11 — that recovery acts only on a **stale vzdump
+  lock** (fault 11's guest was unlocked) and runs **once at agent startup** (fault 11's guest went
+  down while the agent was already up). `onboot` is the deliberate-stop discriminator and is *not*
+  invented here: it is already what `stalelock.go` uses for this decision, it is 0 on scratch/golden
+  guests, and it is what `pve-guests` consults at host boot — so the agent agrees with the platform
+  instead of keeping a second private definition of "should be running". Retry bounded at 3
+  (1m/2m/4m) then escalates **once**; an unbounded silent retry loop is the over-correction here.
+  Live on demo-hp: **120 s unattended** recovery vs the incident's **587 s** with a human; Scenario B
+  proven (an `onboot:0` guest left stopped throughout). Detail: `REPORT.md`.
+
+- **2026-07-28 — F-LEAK took THREE attempts; v0.108.0 and v0.110.0 are the corrections.** The cause is
+  structural: `FelhomAgentGuest` is granted at `/pool/felhom` and a guest joins that pool only when its
+  restore **completes**, so a *failed* restore-test leaves a pool-less guest out of reach (403).
+  **(1) v0.107.0 pool adoption — REFUTED LIVE:** `PUT /pools/{pool}` also requires `VM.Allocate` on the
+  VM being added, so membership cannot bootstrap its own authority; removed in **v0.108.0**.
+  **(2) host-install v1.21.0 per-path `/vms/990000..990009` ACLs — works, but exactly ONCE per slot:**
+  PVE's destroy calls `AccessControl::remove_vm_access` (`API2/LXC.pm:906`) which deletes every ACL at
+  `/vms/<vmid>` (`AccessControl.pm:1898`) — **the grant is consumed by the op it authorises**. Caught by
+  counting ACL rows after the fix, not by reasoning. **(3) v0.110.0 SHIPPED —
+  `Privileged.DestroyScratchLXC`, the FOURTH root-fenced exception** (was exactly three: keyctl
+  `pct create`, USB mount/fstab, SMART/sensors). Band enforced in **sudoers literally**
+  (`pct destroy 99000[0-9] --purge`) + re-checked in code + journal provenance at the caller; none is
+  consumed by use. API destroy still tried FIRST; band ACLs stay provisioned so the common case needs no
+  privileged call. **Ships with a sudoers change — deploy `configs/felhom-agent.sudoers` WITH the
+  binary.** Live: token 403 on a stranded scratch → fenced path removed the guest and all 3 LVs; sudo
+  PERMITS the band and REFUSES `9201`/`9100`/`9999`/`990010`/`1`, and refuses `pct start 990000` too.
+
+- **2026-07-28 — v0.109.0: the guest-power watchdog got the observable it shipped without.** A
+  self-correction: v0.107.0's watchdog logged only at startup and when it *acted*, so on a healthy box
+  its health could be read only from **absence** — F-OBS's exact shape, shipped in the same session
+  F-OBS was fixed in the controller. Now an INFO summary every 10th sweep carrying
+  `sweeps_since_boot`/`guests_evaluated`/`currently_stopped`. An **aborted** sweep (unproven
+  ownership) does not count, or the heartbeat would claim liveness for a watchdog examining nothing.
+
+- **2026-07-28 — v0.106.0: F-CRIT-2 fixed — a failed backup no longer looks like a fresh one.**
+  `NewestArchiveTime` counted an aborted PBS upload (1 byte, manifest-less, NEWEST) as a successful
+  backup, so the tier reported fresh, went **not due**, and was never retried — 7 days of silence on
+  the real 168h cadence, invisible to both the R-88 breaker (defers only DUE tiers) and the hub
+  deadline monitor (reads the same freshness). Now only *plausibly complete* entries count, via a
+  measured floor `minPlausibleArchiveBytes` = 1 MiB; undecidable ⇒ not counted.
+  **Size is the only tier-agnostic discriminator** — `verification` and `encrypted` are absent on
+  every local (dir) archive and on a good PBS snapshot until verify-new catches up, so gating on
+  either would reject 100% of local backups and cause fleet-wide backup THRASH. Floor measured:
+  smallest real backup on the fleet is 612,397,450 B, so 1 MiB leaves 584x headroom (asserted by a
+  test). Rejections logged at WARN once per volid. Re-tested live by replaying campaign fault 2 on
+  demo-hp — both directions, incl. a no-thrash window with 91 scheduler ticks as the positive
+  observable. Deployed on both boxes. Detail: `REPORT.md`.
+  **Also established:** server-side prune does NOT count phantoms toward `keep-last` (dry-run kept
+  2 real + the phantom) ⇒ **no retention/data-loss bug** — but it never removes them either, so they
+  accumulate. Filed as R-99 (LOW).
+
+- **2026-07-25 — v0.95.0 (additive): SMART coverage fixes (spike B+A) + device model.** Union-path
+  drives (USB/registry) now get SMART via `storage.SmartReader.SMARTForBacking` wired into the localapi
+  `/disks` union (localapi `Smart` seam); `smartDeviceFor` resolves dm/LVM to the whole disk via
+  `/sys/block/<dm>/slaves` (recursive, skips >1-disk); the builtin `local` dir on the LVM root gets a
+  **SMART-only** device from its containing filesystem (never touches backing/durable_id — the
+  removable-safety guard in build() stays intact); `SmartSummary.ModelName` captured from smartctl. The
+  watchdog `Known` path stays enrich-free. Consumed by controller v0.171.0. Source of WHERE:
+  `felhom.eu/documentation/audits/SPIKE-smart-coverage-2026-07-25.md`.
+- **2026-07-24 — v0.94.0 (additive): SMART serialized into /disks.** `localapi.DiskInfo` gains
+  `Smart *hub.SmartSummary` (omitempty), copied from the target's already-computed Observe-time
+  enrichment when `Health != ""` — no new smartctl load, no endpoint, no sudoers/MinAgent change. The
+  controller v0.169.0 renders a "Lemezek állapota" card + 6h degradation alert from it; old controllers
+  ignore it. **NOTE: at the remote-site vacation window the agent is DOWN (localapi binds .162 → fails),
+  so live /disks-from-real-agent validation is deferred — the field is unit-proven; publish only.**
+- **2026-07-22 — v0.93.0 is the FLEET AGENT.** Built, published (sha `a68b2ff73200622e…`),
+  Day-0-manifest-vouched (MinAgent also 0.93.0, operator-ruled) and deployed to BOTH boxes
+  (`demo-felhom-8363b5` + `demo-hp-bb76ea`, the latter over G1 break-glass — still no key baked);
+  clean-restart 5/5 on both, `.bak-0.92.1` retained. Discharges the onboarding runbook §A5
+  ceremony gate. Record: `felhom.eu/documentation/pilot/RUNBOOK-publish-agent-0.93-2026-07-22.md`.
+  **The bullet below ("agent is DOWN … deployed 0.90.0") is SUPERSEDED history** — vmbr0 was made
+  static .162 on 2026-07-20 (F1 mitigation) and the agent has been up since; kept for the record.
+
 - **2026-07-20 — REMOTE SITE until ~2026-08-02; the agent is DOWN there and cannot self-recover.**
   felhom-pve moved off the home LAN; `ssh felhom-pve` = tailnet `100.70.170.35` (direct, ~37 ms). The
   host is on DHCP and holds `192.168.0.147`, so `localapi`'s literal `192.168.0.162` bind fails with

REPORT.md

@@ -1,79 +1,9 @@
-# REPORT — escrow recovery-code wordlist fix (TASK-E Part 1), 2026-07-21
+# REPORT — 2026-10-04: v0.141.0, the host fast lane, the true tunnel status, the fast leg
 
-**Version: v0.93.0. Baseline `935904f` (v0.92.1), clean tree. NOT built or deployed** — no operator
-action needed today; this rides the next agent publish train.
+Full session report: `felhom.eu/REPORT-os-host-lane-2026-10-04.md`.
 
-## What was wrong
-
-The EFF large wordlist contains exactly four entries that themselves contain the hyphen the words are
-joined with: `drop-down` (L2009), `felt-tip` (L2528), `t-shirt` (L6640), `yo-yo` (L7748). Drawing one
-produced a recovery code that reads as **11 words instead of 10** — ambiguous to transcribe in the one
-situation R exists for, a customer reading a code back during a disaster.
-
-**The "known flake" was this defect, not a flaky test.** Per code the hit rate is
-`1 − (7772/7776)^10 ≈ 0.51%`; over the test's 50 draws that is ≈ **23%**, which is exactly the "fails
-~1/5" that had been documented in the `felhom-testing` skill and re-run past for weeks.
-`felhom.eu/REPORT.md` §6 had it at 3/8 in one session.
-
-## The fix
-
-`joinSafe` filters those four out of the effective wordlist at init. Generation-only.
-
-| | before | after |
-|---|---|---|
-| draw space | 7776 | **7772** |
-| 10-word code entropy | 129.248 bits | **129.241 bits** |
-| margin over the 128-bit floor | 1.248 | **1.241** |
-
-The cost is **0.007 bits**. Both numbers are asserted in `TestEntropyFloorSurvivesFiltering`, so a
-future wordlist swap cannot quietly move the floor.
-
-**Every recovery code already issued remains valid**, and this was verified rather than assumed: R is
-consumed as a whole passphrase by the PBS scrypt KDF (`Wrap`/`Unwrap`), and `grep` confirms nothing in
-the consume path ever splits it. The joiner, the word count and the KDF path are untouched.
-
-## Tests
-
-`TestGenerateRecoveryCode_EntropyAndFormat` now counts words by **generation count**, not by splitting
-the joined string — conflating those two is what made it flake. It asserts the segmentation property
-separately, because that is the property `joinSafe` actually buys.
-
-New in `wordlist_test.go`: the four filtered words are pinned by name (a wordlist swap that changes the
-set fails loudly); the entropy numbers above; and a production-wiring test that drives the exported
-`GenerateRecoveryCode` 500×, so the fix is proven at the entry point and not only in the helper.
-
-### Red-proof — RUN, both deterministic and probabilistic halves
-
-Pre-fix shape restored (`var wordlist = parseWordlist(wordlistRaw)`), suite re-run:
-
-```
---- FAIL: TestWordlistLoaded
-    effective wordlist should be 7772 words (7776 EFF - 4 hyphenated), got 7776
---- FAIL: TestGenerateRecoveryCode_EntropyAndFormat
-    joined code must segment into 10 words, got 11 (a drawn word contained "-")
---- FAIL: TestEntropyFloorSurvivesFiltering
-    filtered entropy moved: 129.248, expected 129.241
---- FAIL: TestGenerateRecoveryCode_NeverContainsAmbiguousWord
-    code 193 segmented into 11 parts, want 10
-```
-
-Fix restored → green; `git diff` clean.
-
-`TestGeneratedCodeSegments_FilteredVsUnfiltered` is the **deterministic** companion: it drives the
-generator against a fixture list where every word is hyphenated, so the pre-fix defect reproduces with
-probability 1 rather than ~1/5, and shows the same list through `joinSafe` refuses to generate at all.
-
-## Gates
-
-`go build ./... && go vet ./... && go test ./...` — **green**. The ex-flaky package was additionally
-run 10× consecutively: 10/10 pass (it previously failed ~1 in 5).
-
-## Observations
-
-- The `felhom-testing` skill's "Known flake … re-run before diagnosing; it is not a regression" line
-  is now removed. It had been actively harmful: it told every future session to dismiss a true
-  positive. Replaced with the generalised lesson — "known flake, just re-run it" is a diagnosis and
-  needs evidence like any other.
-- `WordlistSize()` now reports the **effective** (filtered) draw space, 7772. `WordlistFilteredOut()`
-  is new, for audit. Any external consumer expecting a literal 7776 would need updating; there is none
-  outside the tests.
+- Tunnel (R-841): the agent reads the guest's cloudflared container and its health check; three states.
+- Host fast lane: the wrapper gains the host layer (R12 appliance proof from the root-owned install record, R14 no
+  kernel/boot/firmware); the leg runs the host step after a healthy guest step; host health rule.
+- Speed (R-845): one call per layer instead of one per package; measured before/after in the session report.
+- Tests green; red-proofs in the audit folder.

REUSE.md

@@ -74,6 +74,7 @@
 | `EnsureLeaf` | internal/localapi/cert.go | `EnsureLeaf(certPath, keyPath, host) (cert, fingerprint, generated, err)` | pinned self-signed leaf | `generated=true` invalidates every issued bootstrap pin — log LOUD (B.1) |
 | `Server.RecoverStaleLockedGuests` | internal/localapi/stalelock.go | `RecoverStaleLockedGuests(ctx)` | startup stale vzdump-lock heal (F2-b) | Clears ONLY `backup`/`snapshot-delete`, only when no vzdump in-flight; A1 RESOLVED (v0.62.0): scan is pool-intersected (`ListLXC` ∩ `Client.Pool`), fail-safe skip on pool-read failure |
 | `ControllerSwapper.Swap` + `ValidControllerImage` | internal/localapi/controllerswap.go | `Swap(ctx, vmid, target) *ControllerSwapState` | agent-owned controller image swap + rollback | Strict image regex (repo + 3-part semver); state file written BEFORE swap; no-healthcheck images need `verifyDwell` |
+| `Server.ControllerSupervisorTick` + `ControllerParkedMarker` | internal/localapi/controllersupervisor.go | `ControllerSupervisorTick(ctx)` | R-523: restart a provisioned guest's not-running controller via its bootstrap unit | Two-sweep confirm; honours swapInFlight, the host-side park marker, guest lock + vzdump; 3 restarts/15 min → 30 min pause; record rides the report as `controller_supervisor` (the hub mints the events — the agent has no event channel) |
 | `MemoryOps` + `Server.readMemoryBounds` | internal/localapi/guestmemory.go | `readMemoryBounds(ctx, vmid) (memoryBounds, err)` | guest RAM resize (v0.90.0, R-24): GET/POST /guest/memory | NEW narrow seam (never extend `GuestAPI` — it breaks every fake); the AGENT is the boundary — bounds recomputed FRESH per request (min 2048 / max host_total−2048 / shrink floor max(2048, usage+512)); §8 UNITS TRAP (config `memory`=MB, status/node=bytes); verify maxmem==target after `SetConfig` before claiming success; SetConfig NEVER called on a refusal path |
 
 ### Proxmox client / hub / PBS / provisioning
@@ -84,11 +85,15 @@
 | `Client.Pool` | internal/proxmox/query.go | `Pool(ctx, name) (PoolInfo, error)` | felhom-pool membership (the ownership registry, A1) | Needs `Pool.Audit` at `/pool/<name>` (host-install v1.9.0+); `Pool.Allocate` does NOT satisfy the read; members can be storages (type `storage`, vmid 0) — filter them |
 | `Client` mutate wrappers (`RestoreLXC/Vzdump/DestroyLXC/Snapshot/Rollback/SetConfig/ResizeLXC/Start/Stop`) | internal/proxmox/mutate.go | return `(upid, error)` | all API mutations | Async → always pair with WaitTask; route via gate/queue, not ad-hoc |
 | `Client.PoolAddVMID` | internal/proxmox/mutate.go | `PoolAddVMID(ctx, pool, vmid) error` | re-assert pool membership after a restore-over-existing (campaign-2 R2) | SYNC (no UPID, don't WaitTask); PVE `PUT /pools` is additive (merge, not replace) — `delete=1` removes; idempotent (already-member swallowed); needs `Pool.Allocate` at `/pool/<pool>`. `pct restore --pool` sets membership only at CREATE — a restore over an existing vmid drops it, so bring-up re-asserts post-restore |
+| `reconcile.PreflightRestoreSpace` + `restorespace.Provider` (v0.133.0) | internal/reconcile/restoretest_space.go, internal/restorespace/restorespace.go | `PreflightRestoreSpace(ctx, space, policy, archive, rawCfg, configured) SpaceVerdict` | ANY step that restores or copies a guest onto a storage — size it first | The restored size is UNCOMPRESSED (vzdump log "Total bytes written" / PBS snapshot size) — never the archive FILE (6.9 GB file → 22.6 GB restore, R-672); an unknown refuses; eligibility needs Datastore.AllocateSpace on `/storage/<id>` specifically (the `/` grant answers every path) |
+| `Engine.RetryScratchTeardown` + the daemon janitor (v0.133.0) | internal/reconcile/restoretest_retry.go, cmd/felhom-agent/janitor.go | `RetryScratchTeardown(ctx) ScratchRetryResult` | retrying a leftover on a TIMER instead of only at start | Never `Recover` on a timer — it also resolves generic in-flight ops; a periodic sweep that unlocks guests holds the one-heavy-op gate (`InFlight.TryAcquire`) |
 | `TLSConfig.build` / `normalizeFingerprint` | internal/proxmox/tls.go | `build() (*tls.Config, error)` | PVE leaf-cert SHA-256 pinning | No insecure default |
 | `pinnedTLS` | internal/pbs/pin.go | `pinnedTLS(fingerprint) (*tls.Config, error)` | PBS leaf pinning | Same model as PVE; 64-hex fingerprint normalized |
+| `httpx.NewTransport` | internal/httpx/transport.go | `NewTransport(tlsCfg, idleConnTimeout) *http.Transport` | **EVERY** hand-rolled `http.Transport` in this repo — pbs, hub and proxmox all pin TLS, so none can use `http.DefaultTransport` | **R-344: never inline `&http.Transport{TLSClientConfig: ...}` again.** A composite literal takes `IdleConnTimeout` **zero, which means retain idle connections FOREVER** — `http.DefaultTransport` sets 90s and a literal does not inherit it. Combined with a client rebuilt per cycle and dropped (`pbsTargetsFromPVE`), that stranded **388 sockets on ep0 in 46 h**, held open on BOTH sides. `idleConnTimeout <= 0` means **use the default**, never "no timeout". Returns a **FRESH** transport every call — a shared one would pool connections across differently pinned endpoints. Pinned by `internal/pbs/client_leak_test.go` (server-side connection counting) + `internal/httpx/transport_test.go` |
 | `hub.Client.Report` | internal/hub/client.go | `Report(ctx, *HostReport) (*ControlEnvelope, error)` | the heartbeat | Typed `TransportError`/`HTTPError`, never contain the bearer token |
 | `hub.Loop` + `MultiObserver` | internal/hub/loop.go | `NewLoop(...)`; `MultiObserver(obs...)` | resilient report loop + envelope fan-out | Errors logged, loop continues; interval clamped 60–3600 s |
-| `provision.BackHalf.Provision` | internal/provision/backhalf.go | `Provision(ctx, Input) (Result, error)` | guest bootstrap back-half | mint→render→0600 write→chown 100000:100000→`pct set` ro bind→onboot; token NEVER logged/returned |
+| `provision.BackHalf.Provision` | internal/provision/backhalf.go | `Provision(ctx, Input) (Result, error)` | guest bootstrap back-half | mint→render→0600 write→chown 100000:100000→`pct set` ro bind→onboot; token NEVER logged/returned. Bootstrap `local_api.endpoint` = the caller's `cfg.LocalAPI.ListenAddr` (main.go) — moving the agent bind to the island moves the guest dial for free (R-50, no template) |
+| `buildBringUpConfig` island NIC | internal/reconcile/bringup.go | (pure) `BringUpSpec{IslandBridge,IslandGuestAddr}` → `params["net1"]` | R-50 island control plane | When BOTH island fields are set (from `cfg.LocalAPI`), attaches a static `net1=name=eth1,bridge=<vmbr9>,ip=<.2/30>` (no hwaddr → fresh MAC), so the controller reaches the agent over a fixed private address immune to LAN/DHCP/site moves. Empty = pre-R-50, no net1. All-or-nothing + CIDR enforced in `LocalAPIConfig.Validate`. The guestnet healer is eth0-only (`parseMode` is dev-scoped) so it never touches the static island NIC |
 | `reconcile.Queue.Submit` | internal/reconcile/queue.go | `Submit(vmid, fn) <-chan error` | per-guest serialization of ALL mutations | Same vmid strictly FIFO; lanes parallel across guests |
 | `Engine.RunSignedJob` | internal/reconcile/job.go | `RunSignedJob(ctx, intent, signed, exec) JobResult` | executing a gated destructive job | Idempotency by nonce; journaled |
 | `escrow.Create` | internal/escrow/escrow.go | `Create(ctx, CreateOptions) (CreateResult, R, error)` | PBS-key escrow (zero-knowledge) | Recovery code returned SEPARATELY from the result (anti-log); self-verifies recoverability |
@@ -109,7 +114,10 @@
 | Anti-retarget durable-id binding | internal/localapi/wipe_reresolve.go | resolve id → re-derive + exact match → re-inspect expected state → act on RE-RESOLVED device only |
 | Atomic single-file JSON store | internal/storage/intent.go | `Open*` loads (missing=empty, corrupt=fail-loud), mutex, tmp+rename 0600, idempotent set |
 | Durable append-only log + index | internal/authz/noncestore.go (`FileNonceStore`) | fsync before returning "new"; replay into index on open; expiry-only compaction |
-| Injectable seam funcs on Server | internal/localapi/server.go (`reresolveWipe`, `deviceDurableID`, `boundCheck`, net-verify: `netTrigger`/`netMounted`/`netJournal`/`netReachable`) | prod default wired in `NewServer`; tests override — no real /dev, /proc/mounts, journalctl or TCP in tests |
+| Injectable seam funcs on Server | internal/localapi/server.go (`reresolveWipe`, `deviceDurableID`, `boundCheck`, `deviceCheck`, `livenessCheck`, net-verify: `netTrigger`/`netMounted`/`netJournal`/`netReachable`) | prod default wired in `NewServer`; tests override — no real /dev, /proc/mounts, journalctl or TCP in tests. **For mount-table predicates prefer the DATA seams `procSelfMountinfo` / `procGuestMountinfo` (internal/localapi/intermediary.go) over `boundCheck`/`livenessCheck`**: pointing them at a captured fixture runs the real parser, the real predicate and the real handler, so the test cannot go hollow the way R-116's did |
+| `Server.devicePresent` (R-113, v0.114.0) | internal/localapi/disks.go | `devicePresent(rawMountPath) bool`; seam `deviceCheck`, default `isHostMountpoint` | the agent's DEVICE-presence signal — asks whether the drive's RAW mount is still mounted | **Use this, never the bind, to answer "is the drive there".** The raw mount is a device-bound systemd unit and dies with its device; the agent's own bind under the shared parent is NOT device-bound and outlives it as a stale shell. `BoundUnderParent` is now `boundUnderParent(...) && devicePresent(...)` at BOTH /disks construction sites — dropping either half is a regression with its own red-proof. Empty path ⇒ **true** (unknown is never absent: absent stops a customer's apps) |
+| `bindLiveness` + `BindLiveness` (R-117, v0.117.0) | internal/localapi/intermediary.go | `bindLiveness(stable, raw) BindLiveness`; seam `livenessCheck`; read verdicts ONLY via `.Usable()` | the agent's bind-LIVENESS signal — the third term of `BoundUnderParent` | **`devicePresent` and `boundUnderParent` are both PATH-PRESENCE tests and neither is liveness.** They compare only mountinfo field 5, so both stay true over a bind that names the drive that went away while the raw mount healed onto the returning one (measured: raw 8:32 /dev/sdc, bind 8:16 /dev/sdb `shutdown`, EIO both ways, payload healthy). Two dead states, and a fix needs BOTH checks: devno mismatch (the detach/return case) AND the ext4 abort tokens `shutdown`/`emergency_ro` (the steady-state case, where the devnos AGREE because the device never left). **THREE states, never a bool** — `BindUnknown` must exist and `Usable()` treats it as PRESENT (absent stops a customer's apps). **Order matters:** compare devices first and read the abort flag off the RAW mount in the stale case — abort-first classifies the real return state as aborted and refuses the re-bind that repairs it. **NO BLOCK I/O, ever** (CLAUDE.md rule; a probe on a wedged device survives SIGKILL). 6 red-proofs |
+| `AttachDrive` repair ruling (R-117, v0.117.0) | internal/localapi/intermediary.go | the `switch bindLiveness(...)` inside the `n == 1 && GuestSeesMount` arm | decides whether the existing self-heal runs | `BindStaleDevice` ⇒ **re-bind** (the raw mount is a healthy new superblock; repairs live, no guest restart). `BindAborted` ⇒ **quiet no-op** — a re-bind lands on the SAME dead superblock and this runs every 20 s, so re-binding is an infinite silent retry that also masks the state; it must surface via `BoundUnderParent=false`. `BindLive`/`BindUnknown` ⇒ no-op, unchanged. **Do not return an error for the aborted case** — the reconcile loop would log a failure every 20 s |
 | Detached IN-MEMORY verify job (single slot, deliberately unpersisted) | internal/localapi/netverifyjob.go | claim slot sync (single-flight 409) → detached pipeline off baseCtx → auto-rollback on fail; restart ⇒ slot empty ⇒ the CALLER rolls back (Scenario F) — contrast formatjob (persisted+recovered) |
 | Optional dependency degradation | internal/localapi/server.go (`Options`) | nil dep ⇒ endpoint answers "not configured" (503), never a crash |
 | Version channel (v0.82.0) | internal/localapi/server.go (`Options.AgentVersion`; `Handler()` mux wrap) | sets `X-Felhom-Agent-Version` on EVERY response (all routes/statuses, incl. auth-fail/404) — the controller's capability-comparison source; empty version ⇒ header omitted |
@@ -143,11 +151,18 @@
 | `storage.HostReader` | internal/storage/hostread.go | `*ProcHostReader` | `fakeHostReader` internal/localapi/disks_test.go; internal/storage/role_test.go. v0.87.0: `BlockSlaves(name)` lists `/sys/block/<name>/slaves` (root-free) — backs the `SystemDisks` dm/md walk (`physicalDisksOf`/`walkSlaves`, role.go); per-branch conservatism: an unresolvable slave fails the WHOLE walk → all-system fail-safe. NEVER weaken the signature test `TestSystemDisks_WalkTopologies` (root-backing disk always in the system set). |
 | `localapi.DiskOps` / `StorageGate` / `GuestAttacher` / `GuestLister` | internal/localapi/disks.go | `*storage.SudoHostOps`; `storageGateAdapter` (cmd/felhom-agent/main.go); `*GuestBinder`; `*proxmox.Client` | `fakeDiskOps`/`fakeGate`/`fakeGuestAttacher`/`fakeGuestList` internal/localapi/disks_test.go |
 | `localapi.GuestAPI` / `BackupService` / `BackupStore` / `TokenAuthority` | internal/localapi/server.go | `*proxmox.Client`, `*backup.BackupRunner`, `*backup.Store`, `*TokenStore` | `fakeGuests`/`fakeBackups`/`fakeStore` internal/localapi/server_test.go |
+| `backup.InFlight` | internal/backup/inflight.go | `TryAcquire(what) (release, busy, ok)` / `Busy()` | THE host-wide "one heavy guest operation at a time" gate — shared by the local-API backup path and the restore-test scheduler (R-85) | A **LINK** guard, not a lock one: the scratch VMID never touches the live guest's vzdump lock, but an offsite restore PULLS multi-GB over the tunnel a backup PUSHES one. Callers **DEFER, never cancel** — a deferred restore-test costs coverage, a cancelled backup costs the backup. A nil gate is ungated (pre-R-85 callers). |
+| `capability` store-grant probe (`storeGrantStatuses` / `storeGrantVerdict` / `Client.Permissions`) | cmd/felhom-agent/main.go, internal/proxmox/query.go | *"may the agent READ this backup tier?"*, one `capability.Status` per configured tier | R-185. **Never infer permission from an empty content listing** — `{"data":[]}` is what a FORBIDDEN tier and a NEWBORN tier both return, and that ambiguity hid an unreadable host tier on both demo boxes. Ask `/access/permissions` **as the agent's own token** (root always says yes). **The ungranted answer is not empty and not a 403** — it carries the privileges inherited from the box-wide `/` grant, so test for **`Datastore.AllocateSpace`** specifically; path-presence or `Datastore.Audit` reports a blinded storage healthy. Probed set comes from `BackupTiers()`, never a fixed list. Critical except the `local` fallback. Composes AROUND the sudo prober (the `poolReadStatus` precedent); `Status`'s wire shape is untouched so the hub alert is free. Unreachable PVE ⇒ degraded, never ok. |
+| `backup.RestoreTestState` | internal/backup/restoretest_state.go | `RecordSuccess(target,archive,tier,verified,t)` / `ProvenArchive(target)` / `ProvenRestoreTests(ctx)` / `LastSuccess(target)` / `OldestFirst(targets)` | Per-tier restore-test PROOF state, persisted (atomic tmp+rename) — **which archive** was proven, and when (R-86) | **Credit ONLY on success** — a permanently failing tier must keep sorting first, or it looks freshly proven and stops being retried. Ties break on target id: without it, two tiers proven in the same second rotate by Go's randomised map order. **This one NEEDS persistence unlike R-84** — R-84 had ground truth to consult (the archive is still on the storage); a restore-test destroys its scratch and leaves no artifact. **R-86: the ARCHIVE is the state, the time is metadata** — a time alone cannot answer "have we proven THIS archive", which is the due-check's whole question. A pre-R-86 file (bare RFC3339 per target) keeps its time and yields NO proven archive, so each tier is due once after the upgrade; reading a legacy time as proof of the current archive would invent a guarantee. **R-189: it is also the REPORTABLE half of the restore-test signal.** The in-memory `backup.Store` holds only this process's latest run, and under per-archive due-ness the agent will not re-test a proven archive — so a proof lost to a restart is not repeated for a whole archive generation (observed live: a passing 14.5 GB offsite restore reached no host-report). `ProvenRestoreTests` renders the stored proofs as `hub.RestoreTest` entries and the collector merges them; a record missing the archive or the tier is NOT emitted, because an unproven tier reading as proven is worse than the defect. **Only successes are stored, deliberately:** a success suppresses future work, a failure causes it. |
+| `hub.ProvenRestoreTestReporter` + `Collector.SetProvenRestoreTests` | internal/hub/collect.go | the DURABLE restore-test source, merged with the in-memory one | R-189. Merge rule: **one entry per tier, newest by `TestedAt` wins** — a fresh failure beats a stored success (the failure is the news, and it lives nowhere else), a stored success beats a stale in-memory entry after a restart, and a tier never appears twice (the hub would read two tests). An unparseable timestamp counts as OLDER, so a malformed entry cannot displace a good one. **The wiring is pinned by an AST test** — the method this replaced (`RestoreTestState.Snapshot`) carried a doc comment naming a host-report gauge and had no caller for weeks. |
+| `backup.SpecBuilder` / `backup.TierPicker` / `(*BackupRunner).PickSettledRestoreCandidateOn` | internal/backup/schedule.go, runner.go | `func(ctx,archive) RestoreTestSpec`; `func(ctx,target,notAfter) (archive,landed,error)` | The per-run restore-test spec + per-tier **settled** candidate lookup (R-85, widened by R-86) | The spec is built **PER RUN**, never frozen at construction — the pre-R-85 immediately-invoked value made the offsite tier unschedulable AND went stale on any config change. `SourceTier` comes from **the archive**, never the configured target (the v0.100.0 rule). A tier with no archive returns `("", zero, nil)` — **`""` is NOT an error**, or every fresh box looks broken for its first week. **R-86: `notAfter` is the settle cutoff** (zero = no cutoff, which is what keeps `PickRestoreCandidateOn` a one-line call into it), and the picker now skips entries failing `archivePlausiblyComplete` — under per-archive due-ness an incomplete phantom would be picked forever, fail forever, never earn proof, and make the tier due at EVERY evaluation. |
+| `localapi.BackupTier` + `normalizeBackupTiers` / `config.BackupConfig.BackupTiers` | internal/localapi/backup_tiers.go, internal/config/config.go | `normalizeBackupTiers(tiers, legacy, cadence) []BackupTier`; `BackupTiers() ([]BackupTier, []string)` | THE R-82 multi-tier resolution — one runner per tier, primary first | **The untargeted local-API contract is FROZEN**: no `?target=` ⇒ primary tier ⇒ pre-R-82 response BYTES (Target is `omitempty` and stays empty). Never default a missing cadence — reject it and log the warning at ERROR. Never share one retention knob between tiers. Jobs are keyed by (vmid,target). |
 | `localapi.StaleLockController` | internal/localapi/stalelock.go | `*staleLockController` (Client + Runner + pool) | `fakeStaleLock` (Server-level) stalelock_test.go; `fakeStaleLockAPI` (controller-level, tests the A1 pool intersect) stalelock_pool_test.go |
 | `localapi.GuestExecutor` | internal/localapi/controllerswap.go | `*GuestBinder` (pct exec) | `fakeGuestExec` internal/localapi/controllerswap_test.go |
 | `guestnet.Runner` / `guestnet.GuestSource` (R-54, v0.92.0) | internal/guestnet/{probe,watchdog}.go | `*proxmox.ExecRunner`; the POOL-VERIFIED `localapi.StaleLockController.Guests` (ListLXC ∩ felhom pool, audit A1) | `scriptedRunner` + `fakeGuests` internal/guestnet/watchdog_test.go. **Never wire a bare `ListLXC` here** — under a broad token that would run dhclient inside a co-tenant's container. Every assertion is an exec COUNT, and the load-bearing ones are the negatives: a static guest, an unprobeable guest, a boot-race guest and an unproven guest list must record **zero** heal calls |
 | `guestnet.Watchdog.SetDampers` / `now` (clock seam) | internal/guestnet/watchdog.go | config `guest_net.*`; `now` defaults to `time.Now` | tests advance a manual clock (the storage-watchdog pattern) and assert the heal ceilings EXACTLY — ≥10 min apart, ≤3/hour, and ≤30 over a scripted 10 hours of permanent failure. A damper with no test is a comment |
 | `hub.GuestNetReporter` (R-54) | internal/hub/collect.go | `*guestnet.Watchdog` (`GuestNetStatus`) | internal/hub/collect_guestnet_test.go asserts the stanza through the PRODUCTION `Collect` path AND that the `guest_net` key is ABSENT from the wire when no reporter is wired — an always-present empty stanza would make "not wired" and "found nothing" the same signal, which is the shape v0.91.0 hid behind |
+| `hub.AddressEnumerator` (v0.119.0) | internal/hub/hostaddr.go | **defaults to the REAL `systemInterfaces`** when `Collector.addrEnum` is nil — deliberately inverting the nil-reporter-means-off convention, because this stanza has no config gate and a forgotten wiring call would otherwise ship silently empty (the inert-seam shape, four instances on record) | internal/hub/hostaddr_test.go drives fixtures TRANSCRIBED from `ip -o addr show` on demo-felhom AND demo-hp, including the address-less veth/NIC rows — the "no denylist needed" claim rests on those rows really being empty, so omitting them would prove the claim by assuming it. `filterHostAddresses` keeps GLOBAL UNICAST only: one predicate that drops loopback, `fe80::/10`, and `169.254/16` — the last being the R-50 island literal, identical on every box and actively misleading if surfaced |
 | `reconcile.OpVerifier` | internal/reconcile/gate.go | `*authz.Verifier` | fake verifier in internal/reconcile gate tests |
 | `signedjobs.WipeOps` / `Executor` (`ExecutorChain`) | internal/signedjobs/wipe.go + runner.go | `*storage.SudoHostOps`; `WipeExecutor`+`DecommissionExecutor` | internal/signedjobs wipe/runner/decommission tests |
 | `hub.reporter` / `collectorIface` / `EnvelopeObserver` | internal/hub/loop.go | `*hub.Client`, `*hub.Collector`; `desired.Syncer` + `signedjobs.Runner` | `fakeReporter`/`fakeCollector` internal/hub/loop_test.go; `recordingReporter` loop_logtail_test.go |
@@ -166,6 +181,14 @@
 - **New privileged host op**: validate args (internal/storage/validate.go style) → exec via `Runner` → add a `Cmnd_Alias` to configs/felhom-agent.sudoers → add a probe vector to internal/capability/manifest.go (so degradation is visible) → ship sudoers with the binary.
 - **New reconcile action**: `ActionKind` + `classOfAction` (internal/reconcile/classify.go), plan emission in internal/reconcile/plan.go; destructive ⇒ gate handles it automatically.
 - **Hub-report field**: extend `hub.HostReport` (internal/hub/report.go) + `Collector` — hub side must mirror + allowlist it (cross-repo).
+- **DR-recipe section (host-half)**: add the field to `DRRecipeHostHalf` (internal/hub/dr_recipe.go) **AND** to
+  the hub's `hostHalfShape` + `AssembledRecipe` (felhom.eu `hub/internal/store/dr_recipe.go`). Those two
+  hub structs are **ALLOW-LISTS**: a section only the agent knows about is stored intact and silently
+  dropped before any operator sees it — that is R-122, which cost `offsite_restic` its entire existence.
+  Then update BOTH copies of `testdata/host-report.golden.json` (byte-identical, cross-repo) and extend
+  `TestAssembleDRRecipe_CarriesEveryEmittedSection`. **A recipe field that cannot be resolved records an
+  explicit unknown with a reason — never a default, an empty string, or a placeholder** (`DRState*` /
+  `DRReason*`); a recipe read during a rebuild must not present a guess as a fact.
 - **Envelope-driven behavior**: implement `hub.EnvelopeObserver`, add to the `MultiObserver` in cmd/felhom-agent/main.go.
 - **Selftest mode**: `selftestFlag` + `runSelftest*` in cmd/felhom-agent/main.go.
 - **Config**: internal/config/config.go (`Load` + `applyEnv` `FELHOM_AGENT_*` overlay; keep secrets out of `Redacted()` output).
@@ -174,8 +197,9 @@
 
 - Two lsblk `-J` parsers with near-identical structs: `parseLsblkDevice`/`lsblkDevice` (internal/storage/hostops.go) vs `parseLsblkNodes`/`lsblkDev` (internal/storage/claim.go).
 - Two smartctl `-a -j` paths: `SudoHostOps.SMART` (internal/storage/hostops.go, parsed `hub.SmartSummary`) vs `Privileged.SMART` (internal/proxmox/privileged.go, raw map).
+- **SMART device resolution (v0.95.0):** `smartDeviceFor` (internal/storage/observe.go) resolves partition→disk AND dm/LVM→disk (`dmWholeDisk` in internal/storage/smartdev.go, via `/sys/block/<dm>/slaves`, `sysBlockRoot` test seam). `storage.SmartReader.SMARTForBacking` is the shared read the localapi `/disks` union path uses (Fix B) — do NOT re-implement smartctl parsing. The builtin-`local` SMART device comes from `containingMountDevice` (SMART-only; never feeds backing/durable_id).
 - Atomic tmp+rename JSON store implemented 3×: `IntentStore.saveLocked` (internal/storage/intent.go), `FormatJobStore.save` (internal/localapi/formatjob.go), `GuestBindStore.saveLocked` (internal/localapi/guestbindstore.go) — comments say "mirrors", no shared helper.
 - `run(ctx, name, args...) error` stderr-wrapping helper duplicated 4×: `SudoHostOps.run`, `Privileged.run`, `BackHalf.run` (internal/provision/backhalf.go), `GuestBinder.run` (internal/localapi/guestbind.go).
-- Several independent /proc mount-table readers: `SudoHostOps.mountedSet` (internal/storage/hostops.go), `ProcHostReader.Mounts` (internal/storage/hostread.go), `isHostMountpoint` + `countHostMounts` (internal/localapi/intermediary.go).
+- Several independent /proc mount-table readers: `SudoHostOps.mountedSet` (internal/storage/hostops.go), `ProcHostReader.Mounts` (internal/storage/hostread.go). **In localapi they were unified in v0.117.0**: `isHostMountpoint` and `countHostMounts` are now one-liners over `hostMountEntries`, the single parser that also yields devno/fstype/super-options for `bindLiveness`.
 - Deliberate mirror: `antiRetargetResolveExpect` (internal/localapi/wipe_reresolve.go) duplicates `WipeExecutor.Execute` steps 1–3 (internal/signedjobs/wipe.go) across packages.
 - `stableParentDir` literal duplicated in internal/provision/backhalf.go to avoid a provision→localapi import edge (commented as intentional); `trim` (internal/storage/hostops.go) vs `trimBody` (internal/proxmox/errors.go) output-truncation twins.

cmd/felhom-agent/archivetier_test.go

@@ -0,0 +1,23 @@
+package main
+
+import "testing"
+
+// R-82 live regression (2026-07-26): the restore-test derived its tier from the CONFIGURED default
+// target instead of the archive's own storage. Restoring a `felhom-pbs:` archive on a box whose
+// primary target is "local" was classified "local" → the 10-minute local wait instead of the
+// generous PBS one → the wait expired mid-restore at 600s against a 14.46 GB WAN restore, teardown
+// fired at a still-restoring guest, and the scratch leaked.
+func TestArchiveStorageID(t *testing.T) {
+	cases := []struct{ in, want string }{
+		{"felhom-pbs:backup/ct/9201/2026-07-26T12:21:48Z", "felhom-pbs"},
+		{"local:backup/vzdump-lxc-9201-2026_07_26-09_03_19.tar.zst", "local"},
+		{"", ""},
+		{"no-prefix", ""},
+		{":leading-colon", ""}, // i>0 guard: a leading colon is not a storage id
+	}
+	for _, c := range cases {
+		if got := archiveStorageID(c.in); got != c.want {
+			t.Fatalf("archiveStorageID(%q) = %q, want %q", c.in, got, c.want)
+		}
+	}
+}

cmd/felhom-agent/escrow_recover_wiring_test.go

@@ -0,0 +1,188 @@
+package main
+
+import (
+	"go/ast"
+	"go/parser"
+	"go/token"
+	"strings"
+	"testing"
+)
+
+// Scenario H — THE SEAM IS WIRED IN THE PRODUCTION PATH, proven by walking the AST rather than by
+// grepping for a string.
+//
+// WHY THIS TEST EXISTS AND WHY IT IS AN AST WALK. This project's built-but-never-wired count is six,
+// and links 6 and 7 of the recovery chain were TWO of them: `UnwrapIdentityBundle` sat in the tree
+// for two months with no caller but a `--selftest`, and the hub's blob-serving endpoints have no
+// client to this day. The fix must not become the seventh. `strings.Contains` on the file would pass
+// against a commented-out line, a line inside a test helper, or a line in dead code behind a flag
+// nobody sets — so this resolves the call graph instead: `Options{EscrowRecovery: …}` must be
+// constructed inside a function that `runDaemon` reaches, and `runDaemon` must be reached by `main`.
+
+func parseMain(t *testing.T) (*token.FileSet, *ast.File) {
+	t.Helper()
+	fset := token.NewFileSet()
+	f, err := parser.ParseFile(fset, "main.go", nil, parser.ParseComments)
+	if err != nil {
+		t.Fatalf("parsing main.go: %v", err)
+	}
+	return fset, f
+}
+
+// callsWithin returns the set of function names called (directly, by identifier or selector) inside
+// the named top-level function.
+func callsWithin(f *ast.File, fnName string) map[string]bool {
+	out := map[string]bool{}
+	for _, d := range f.Decls {
+		fd, ok := d.(*ast.FuncDecl)
+		if !ok || fd.Name == nil || fd.Name.Name != fnName || fd.Body == nil {
+			continue
+		}
+		ast.Inspect(fd.Body, func(n ast.Node) bool {
+			ce, ok := n.(*ast.CallExpr)
+			if !ok {
+				return true
+			}
+			switch fn := ce.Fun.(type) {
+			case *ast.Ident:
+				out[fn.Name] = true
+			case *ast.SelectorExpr:
+				if x, ok := fn.X.(*ast.Ident); ok {
+					out[x.Name+"."+fn.Sel.Name] = true
+				}
+				out[fn.Sel.Name] = true
+			}
+			return true
+		})
+	}
+	return out
+}
+
+// TestEscrowRecoveryIsWiredIntoTheDaemon asserts the whole chain from func main() to the field.
+func TestEscrowRecoveryIsWiredIntoTheDaemon(t *testing.T) {
+	_, f := parseMain(t)
+
+	// 1. main() reaches runDaemon.
+	if !callsWithin(f, "main")["runDaemon"] {
+		t.Fatal("func main() does not call runDaemon — the daemon path this test asserts is not the live one")
+	}
+	// 2. runDaemon reaches buildLocalAPIServer.
+	if !callsWithin(f, "runDaemon")["buildLocalAPIServer"] {
+		t.Fatal("runDaemon does not call buildLocalAPIServer — the local API is not built on the daemon path")
+	}
+
+	// 3. Inside buildLocalAPIServer, a localapi.Options composite literal carries EscrowRecovery, and
+	//    an escrow.OffsiteKeyRecoverer is constructed there.
+	var optionsHasField, recovererConstructed bool
+	for _, d := range f.Decls {
+		fd, ok := d.(*ast.FuncDecl)
+		if !ok || fd.Name == nil || fd.Name.Name != "buildLocalAPIServer" || fd.Body == nil {
+			continue
+		}
+		ast.Inspect(fd.Body, func(n ast.Node) bool {
+			cl, ok := n.(*ast.CompositeLit)
+			if !ok {
+				return true
+			}
+			sel, ok := cl.Type.(*ast.SelectorExpr)
+			if !ok {
+				return true
+			}
+			pkg, _ := sel.X.(*ast.Ident)
+			if pkg == nil {
+				return true
+			}
+			switch pkg.Name + "." + sel.Sel.Name {
+			case "localapi.Options":
+				for _, el := range cl.Elts {
+					kv, ok := el.(*ast.KeyValueExpr)
+					if !ok {
+						continue
+					}
+					if k, ok := kv.Key.(*ast.Ident); ok && k.Name == "EscrowRecovery" {
+						optionsHasField = true
+					}
+				}
+			case "escrow.OffsiteKeyRecoverer":
+				recovererConstructed = true
+			}
+			return true
+		})
+	}
+	if !recovererConstructed {
+		t.Error("no escrow.OffsiteKeyRecoverer is constructed in buildLocalAPIServer — links 6→8 have no " +
+			"production assembly point (the built-but-never-wired shape, seventh instance)")
+	}
+	if !optionsHasField {
+		t.Error("localapi.Options in buildLocalAPIServer carries no EscrowRecovery field — the recoverer " +
+			"exists and the route would answer 503 forever")
+	}
+}
+
+// The hub fetch must be the DAEMON's own hub client, not a freshly constructed one with different
+// credentials — the self-scoping that makes cross-host retrieval impossible is a property of WHICH
+// key is used.
+func TestEscrowRecoveryUsesTheDaemonHubClient(t *testing.T) {
+	fset, f := parseMain(t)
+	var fetchUsesHubClient bool
+	for _, d := range f.Decls {
+		fd, ok := d.(*ast.FuncDecl)
+		if !ok || fd.Name == nil || fd.Name.Name != "buildLocalAPIServer" || fd.Body == nil {
+			continue
+		}
+		ast.Inspect(fd.Body, func(n ast.Node) bool {
+			ce, ok := n.(*ast.CallExpr)
+			if !ok {
+				return true
+			}
+			sel, ok := ce.Fun.(*ast.SelectorExpr)
+			if !ok || sel.Sel.Name != "FetchIdentityEscrow" {
+				return true
+			}
+			if x, ok := sel.X.(*ast.Ident); ok && x.Name == "hubClient" {
+				fetchUsesHubClient = true
+			} else {
+				t.Errorf("FetchIdentityEscrow at %s is called on something other than the injected hub client",
+					fset.Position(ce.Pos()))
+			}
+			return true
+		})
+	}
+	if !fetchUsesHubClient {
+		t.Fatal("the recoverer's fetcher does not call hubClient.FetchIdentityEscrow — either the fetch is " +
+			"not wired, or it uses a client whose credentials are not this host's")
+	}
+}
+
+// The route itself must be registered on the local API. A handler with no route is the same defect
+// one layer down, and it has shipped here before.
+func TestRecoverRouteIsRegistered(t *testing.T) {
+	fset := token.NewFileSet()
+	f, err := parser.ParseFile(fset, "../../internal/localapi/server.go", nil, 0)
+	if err != nil {
+		t.Fatalf("parsing localapi/server.go: %v", err)
+	}
+	var registered bool
+	ast.Inspect(f, func(n ast.Node) bool {
+		ce, ok := n.(*ast.CallExpr)
+		if !ok || len(ce.Args) < 2 {
+			return true
+		}
+		sel, ok := ce.Fun.(*ast.SelectorExpr)
+		if !ok || sel.Sel.Name != "HandleFunc" {
+			return true
+		}
+		lit, ok := ce.Args[0].(*ast.BasicLit)
+		if !ok {
+			return true
+		}
+		if strings.Contains(lit.Value, "/escrow/recover-offsite-password") {
+			registered = true
+		}
+		return true
+	})
+	if !registered {
+		t.Fatal("POST /escrow/recover-offsite-password is not registered on the local API mux — the handler " +
+			"exists and nothing can reach it")
+	}
+}

cmd/felhom-agent/janitor.go

@@ -0,0 +1,81 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"log/slog"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/backup"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
+)
+
+// janitorInterval is how often the leftovers of an interrupted restore-test or backup are retried
+// (R-672 rule 3, R-673). Both used to be resolved ONLY at agent start: on 2026-09-24 a failed scratch
+// teardown kept a full thin pool full for 2.5 h, and a stale `snapshot-delete` lock blocked 9201's
+// whole-box backups for five hours — each cleared within a minute of an agent restart.
+const janitorInterval = 10 * time.Minute
+
+// janitorDeps are the janitor's seams (tests drive one pass with fakes).
+type janitorDeps struct {
+	retryScratch func(ctx context.Context) reconcile.ScratchRetryResult
+	staleLocks   func(ctx context.Context) // localapi Server.RecoverStaleLockedGuests; nil when the local API is off
+	heavy        *backup.InFlight
+	record       func(hub.RestoreTest)
+	now          func() time.Time
+	logger       *slog.Logger
+}
+
+// janitorPass is one pass. The stale-lock sweep runs only while holding the one-heavy-operation gate, so
+// no agent backup can START between its "no vzdump is running" check and its unlock (at start-up the
+// sweep ran before the backup loop existed; on a timer that ordering must be made, not assumed). A busy
+// gate skips the sweep this pass — the next pass retries.
+func janitorPass(ctx context.Context, d janitorDeps) {
+	if d.retryScratch != nil {
+		r := d.retryScratch(ctx)
+		if r.Examined > 0 {
+			d.logger.Info("janitor: restore-test scratch retry pass", "examined", r.Examined,
+				"destroyed", r.Destroyed, "already_gone", r.Clean, "failed", r.Failed)
+		}
+		for _, vmid := range r.GaveUp {
+			// The operator is told through the existing restore-test failure path: the hub raises
+			// restore_test_failed (operator) once per distinct archive — this record's archive names
+			// the stuck scratch guest.
+			if d.record != nil {
+				d.record(hub.RestoreTest{
+					SourceArchive: fmt.Sprintf("scratch-teardown:%d", vmid),
+					ScratchVMID:   vmid,
+					Pass:          false,
+					Error: fmt.Sprintf("restore-test scratch guest %d could not be torn down after %d retries — it holds its disks; remove it by hand (pct destroy %d) after checking what keeps it busy",
+						vmid, reconcile.MaxTeardownTries, vmid),
+					TestedAt: d.now().UTC().Format(time.RFC3339),
+				})
+			}
+		}
+	}
+	if d.staleLocks != nil {
+		release, busy, ok := d.heavy.TryAcquire("stale-lock-sweep")
+		if !ok {
+			d.logger.Info("janitor: stale-lock sweep deferred — a heavy operation is in flight", "busy", busy)
+			return
+		}
+		defer release()
+		d.staleLocks(ctx)
+	}
+}
+
+// runJanitor runs janitorPass every janitorInterval until ctx ends.
+func runJanitor(ctx context.Context, d janitorDeps) {
+	d.logger.Info("janitor: starting (restore-test scratch retry + stale-lock sweep)", "interval", janitorInterval)
+	t := time.NewTicker(janitorInterval)
+	defer t.Stop()
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case <-t.C:
+			janitorPass(ctx, d)
+		}
+	}
+}

cmd/felhom-agent/janitor_test.go

@@ -0,0 +1,60 @@
+package main
+
+import (
+	"context"
+	"io"
+	"log/slog"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/backup"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
+)
+
+// R-672 / R-673 (v0.133.0): one janitor pass, driven with fakes.
+
+func quiet() *slog.Logger { return slog.New(slog.NewTextHandler(io.Discard, nil)) }
+
+// A scratch the engine gave up on reaches the hub as a failed restore-test record naming it — the
+// existing operator path (restore_test_failed). Never a pass.
+func TestJanitor_GaveUpIsReportedAsAFailure(t *testing.T) {
+	var got []hub.RestoreTest
+	janitorPass(context.Background(), janitorDeps{
+		retryScratch: func(context.Context) reconcile.ScratchRetryResult {
+			return reconcile.ScratchRetryResult{Examined: 1, Failed: 1, GaveUp: []int{990000}}
+		},
+		heavy: &backup.InFlight{}, record: func(r hub.RestoreTest) { got = append(got, r) },
+		now: time.Now, logger: quiet(),
+	})
+	if len(got) != 1 || got[0].Pass || got[0].ScratchVMID != 990000 || !strings.Contains(got[0].Error, "990000") {
+		t.Fatalf("records = %+v — want one FAILED record naming scratch 990000", got)
+	}
+}
+
+// R-673: the stale-lock sweep runs only while holding the one-heavy-operation gate, so no agent backup can
+// start between its "no vzdump running" check and its unlock.
+//
+// COMPANION RED-PROOF (REPORT): drop the TryAcquire → "the sweep ran while a backup held the gate".
+func TestJanitor_StaleLockSweepWaitsForTheHeavyGate(t *testing.T) {
+	heavy := &backup.InFlight{}
+	swept := 0
+	d := janitorDeps{staleLocks: func(context.Context) { swept++ }, heavy: heavy, now: time.Now, logger: quiet()}
+	release, _, ok := heavy.TryAcquire("backup")
+	if !ok {
+		t.Fatal("setup")
+	}
+	janitorPass(context.Background(), d)
+	if swept != 0 {
+		t.Fatal("the sweep ran while a backup held the gate")
+	}
+	release()
+	janitorPass(context.Background(), d)
+	if swept != 1 {
+		t.Fatalf("swept %d times with the gate free — want 1", swept)
+	}
+	if _, _, ok := heavy.TryAcquire("after"); !ok {
+		t.Fatal("the sweep did not release the gate")
+	}
+}

cmd/felhom-agent/restoretest_due_wiring_test.go

@@ -0,0 +1,156 @@
+package main
+
+import (
+	"go/ast"
+	"go/parser"
+	"go/token"
+	"testing"
+)
+
+// R-86 Scenario I — the seam-discipline test for the due-check.
+//
+// A due-check is worth nothing if the daemon still wires the OLD picker: every unit test in
+// internal/backup would stay green (they inject the seam directly), the scheduler would ask for the
+// newest archive with no settle cutoff, and the per-archive rule would run against a candidate that
+// changes every time a backup lands. That is the same shape as the v0.91.0 inert seam — built,
+// tested, never called — and this repo has shipped it four times.
+//
+// It walks main.go's AST rather than grepping: a commented-out call still satisfies a substring
+// match, and a comment is not a caller.
+func TestMainWiresTheSettleAwareTierPicker(t *testing.T) {
+	f := parseMainForWiring(t)
+
+	var settlePicker, oldPicker, settleWired, evalInterval bool
+	ast.Inspect(f, func(n ast.Node) bool {
+		switch node := n.(type) {
+		case *ast.SelectorExpr:
+			// runner.PickSettledRestoreCandidateOn passed as a value (not called).
+			switch node.Sel.Name {
+			case "PickSettledRestoreCandidateOn":
+				settlePicker = true
+			case "PickRestoreCandidateOn":
+				oldPicker = true
+			}
+		case *ast.KeyValueExpr:
+			key, ok := node.Key.(*ast.Ident)
+			if !ok {
+				return true
+			}
+			if key.Name == "Settle" {
+				settleWired = true
+			}
+		case *ast.CallExpr:
+			if sel, ok := node.Fun.(*ast.SelectorExpr); ok && sel.Sel.Name == "RestoreTestEvalInterval" {
+				evalInterval = true
+			}
+		}
+		return true
+	})
+
+	if !settlePicker {
+		t.Error("main.go never passes runner.PickSettledRestoreCandidateOn as the scheduler's TierPick — " +
+			"the due-check would run without a settle cutoff, i.e. against an archive that may still be being written")
+	}
+	if oldPicker {
+		t.Error("main.go still wires the pre-R-86 PickRestoreCandidateOn as a tier picker — " +
+			"two pickers means the one under test is not the one running")
+	}
+	if !settleWired {
+		t.Error("main.go never sets SchedulerOptions.Settle — the settle lag would default to 0 in the daemon " +
+			"and every freshly-landed archive would be an immediate candidate")
+	}
+	if !evalInterval {
+		t.Error("main.go never calls cfg.Backup.RestoreTestEvalInterval() — the scheduler would be driven by " +
+			"the retired cadence knob")
+	}
+}
+
+// The two R-85 guarantees the due-check must not have quietly dropped: the spec is still built PER
+// RUN, and the shared heavy-operation gate is still handed to the scheduler.
+func TestMainStillWiresTheHeavyOperationGateAndPerRunSpec(t *testing.T) {
+	f := parseMainForWiring(t)
+
+	var inFlightWired, specIsAFunc bool
+	ast.Inspect(f, func(n ast.Node) bool {
+		kv, ok := n.(*ast.KeyValueExpr)
+		if !ok {
+			return true
+		}
+		key, ok := kv.Key.(*ast.Ident)
+		if !ok {
+			return true
+		}
+		switch key.Name {
+		case "InFlight":
+			inFlightWired = true
+		case "Spec":
+			// A FuncLit means it is evaluated per run; anything else is a frozen value.
+			if _, isFunc := kv.Value.(*ast.FuncLit); isFunc {
+				specIsAFunc = true
+			}
+		}
+		return true
+	})
+
+	if !inFlightWired {
+		t.Error("main.go no longer hands the scheduler the shared InFlight gate — a restore-test could pull a " +
+			"multi-GB archive over the same tunnel an offsite backup is pushing one over (Scenario F)")
+	}
+	if !specIsAFunc {
+		t.Error("SchedulerOptions.Spec is no longer a function literal — a frozen spec is the R-85 defect " +
+			"(the tier and its timeout evaluated once at daemon start, forever)")
+	}
+}
+
+func parseMainForWiring(t *testing.T) *ast.File {
+	t.Helper()
+	fset := token.NewFileSet()
+	f, err := parser.ParseFile(fset, "main.go", nil, 0)
+	if err != nil {
+		t.Fatalf("parse main.go: %v", err)
+	}
+	return f
+}
+
+// R-189 Scenario I — the DURABLE proof source must actually be wired into the collector.
+//
+// This test exists because the method it feeds is the project's own cautionary tale:
+// `RestoreTestState.Snapshot` carried the doc comment "for the host-report gauge" from the day it
+// was written and **had no caller at all** — a seam built, documented and never connected, found
+// only when a live restore-test's PASS reached no host-report. The fix must not become the next
+// instance, so the wiring is asserted rather than trusted.
+//
+// AST, not grep: a commented-out call still contains the string (proven yesterday, when commenting
+// out the tier-picker line failed this test while a `strings.Contains` check would have passed).
+func TestMainWiresTheDurableRestoreTestProof(t *testing.T) {
+	f := parseMainForWiring(t)
+
+	var wired, feedsState bool
+	ast.Inspect(f, func(n ast.Node) bool {
+		call, ok := n.(*ast.CallExpr)
+		if !ok {
+			return true
+		}
+		sel, ok := call.Fun.(*ast.SelectorExpr)
+		if !ok || sel.Sel.Name != "SetProvenRestoreTests" {
+			return true
+		}
+		wired = true
+		// ...and it must be fed the PERSISTED state, not the in-memory store.
+		if len(call.Args) == 1 {
+			if id, ok := call.Args[0].(*ast.Ident); ok && id.Name == "rtState" {
+				feedsState = true
+			}
+		}
+		return true
+	})
+
+	if !wired {
+		t.Error("main.go never calls collector.SetProvenRestoreTests — the persisted proof would never " +
+			"reach the hub, which is the R-189 defect exactly: a passing restore-test that vanishes on restart")
+	}
+	if wired && !feedsState {
+		t.Error("collector.SetProvenRestoreTests is not fed rtState — the in-memory store is the thing " +
+			"that does NOT survive a restart, so wiring it here would fix nothing")
+	}
+}

cmd/felhom-agent/seam_witnesses.go

@@ -0,0 +1,28 @@
+package main
+
+import (
+	"gitea.dooplex.hu/admin/felhom-agent/internal/backup"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/localapi"
+)
+
+// COMPILE-TIME WITNESSES for OPTIONAL interfaces that are satisfied by a RUNTIME type assertion.
+//
+// WHY THIS FILE EXISTS. `localapi.BackupArchiveLister` is asserted at server.go's `newestArchiveOn`
+// via `tier.Service.(BackupArchiveLister)`. A failed assertion does not error — it degrades to
+// `archiveAbsent`, i.e. the pre-R-84 "ask the in-memory record only" behaviour. That degrade is
+// SILENT and it is behaviour-relevant: it is exactly the R-84 bug (a cold store after a restart
+// reading as "no backup ever") coming back, with nothing in any log to say so.
+//
+// The precedent is not hypothetical. During R-88 Part 2 the controller's `quiesceBackend` stopped
+// satisfying `quiesce.TieredBackend` when a signature changed, and `go build` AND `go vet` both
+// passed — because the interface is only ever asserted at runtime. Every box would have degraded to
+// the single-tier path, losing R-82's multi-tier backups, with no error anywhere. It was caught by
+// accident.
+//
+// A witness costs one line and converts that class of failure from a silent production degrade into
+// a compile error.
+//
+// THIS DOES NOT MAKE THE INTERFACE REQUIRED. The optionality is deliberate — it is what lets a
+// BackupService without a lister still work. The witness pins the IMPLEMENTATION (this concrete type
+// really does satisfy it), not the CONTRACT.
+var _ localapi.BackupArchiveLister = (*backup.BackupRunner)(nil)

cmd/felhom-agent/selftest_flag_test.go

@@ -0,0 +1,39 @@
+package main
+
+import (
+	"os"
+	"regexp"
+	"strings"
+	"testing"
+)
+
+// Every mode the dispatcher (`switch selftest.mode`) runs must be ACCEPTED by the --selftest flag. Found live
+// 2026-10-04: --selftest=os-update had a dispatch case and a function but the flag's allow-list refused it, so the
+// debug action could not run. Red-proof: drop the "os-update" case from selftestFlag.Set and this fails.
+func TestSelftestFlag_AcceptsEveryDispatchedMode(t *testing.T) {
+	src, err := os.ReadFile("main.go")
+	if err != nil {
+		t.Fatal(err)
+	}
+	s := string(src)
+	i := strings.Index(s, "switch selftest.mode {")
+	if i < 0 {
+		t.Fatal("dispatch switch not found")
+	}
+	block := s[i:]
+	block = block[:strings.Index(block, "\n\t}\n")]
+	modes := regexp.MustCompile(`(?m)^\tcase "([a-z-]+)":`).FindAllStringSubmatch(block, -1)
+	if len(modes) < 5 {
+		t.Fatalf("parsed only %d dispatch cases — the parser is wrong", len(modes))
+	}
+	var bad []string
+	for _, m := range modes {
+		var f selftestFlag
+		if err := f.Set(m[1]); err != nil {
+			bad = append(bad, m[1])
+		}
+	}
+	if len(bad) > 0 {
+		t.Fatalf("dispatched but refused by --selftest: %v", bad)
+	}
+}

cmd/felhom-agent/storegrant_test.go

@@ -0,0 +1,417 @@
+package main
+
+import (
+	"context"
+	"errors"
+	"go/ast"
+	"io"
+	"log/slog"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/capability"
+)
+
+// R-185 — a tier the box cannot READ must say so.
+//
+// THE OBSERVATION (demo-felhom, 2026-08-03, reproduced at the start of this session): root lists
+// three archives on `felhom-backup`; the agent's own token gets `{"data":[]}` from the same
+// endpoint; and `local`, which has the grant, lists through that same token. The token is the
+// variable, not the storage.
+//
+// The defect is NOT the missing grant — that is one command. It is that an empty content listing is
+// what a FORBIDDEN tier and a NEWBORN tier both return, so the box could not tell them apart and
+// said nothing. These tests pin the distinction.
+
+// permAnswer is the shape /access/permissions really returns, taken from the live measurement:
+// an UNGRANTED path answers with the privileges inherited from the box-wide grant — NOT empty, and
+// NOT a 403.
+var (
+	permGranted   = map[string]int{"Datastore.Allocate": 1, "Datastore.AllocateSpace": 1}
+	permUngranted = map[string]int{"Sys.Audit": 1, "SDN.Use": 1, "Datastore.Audit": 1}
+)
+
+// probeWith calls the PRODUCTION decision with a permissions answer. **Naming the seam:** everything
+// below is true up to `storeGrantVerdict`; that the live call feeds it the real API answer is what
+// Part 0's measurement established and what the live run on the box demonstrates. An earlier draft
+// of this file re-implemented the branch here — it passed, and would have kept passing while
+// production diverged, which is the hollow shape this project keeps catching in its own tests.
+func probeWith(privs map[string]int, targetID string, critical bool) capability.Status {
+	return storeGrantVerdict(targetID, critical, privs, nil)
+}
+
+// ── SCENARIO A — a forbidden storage is REPORTED, not passed over ────────────────────────────
+//
+// COMPANION RED-PROOF (observed 2026-08-03): delete the store-grant probes from `probeAll` in
+// main.go — i.e. restore `append(capProber.Probe(ctx), poolReadStatus(ctx, px))` — and
+// TestMainWiresTheStoreGrantProbe fails with "main.go never calls storeGrantStatuses". That is
+// today's behaviour on the live box: complete silence about a tier it cannot read.
+func TestStoreGrant_ForbiddenStorageIsDegradedAndNamed(t *testing.T) {
+	s := probeWith(permUngranted, "felhom-backup", true)
+
+	if s.Status != capability.StatusDegraded {
+		t.Fatalf("a storage the agent may not read must be DEGRADED, not %q — silence is the defect", s.Status)
+	}
+	if !s.Critical {
+		t.Fatal("it must be CRITICAL: the hub alerts only on critical, so a non-critical entry is the same silence with extra steps")
+	}
+	if !strings.Contains(s.Reason, "felhom-backup") {
+		t.Fatalf("the reason must NAME the storage — 'a grant is missing' costs a diagnosis at 07:00; got %q", s.Reason)
+	}
+	if !strings.Contains(s.Reason, "FelhomAgentStore") {
+		t.Fatalf("the reason must name the ROLE to grant, so the fix is in the alert; got %q", s.Reason)
+	}
+}
+
+// THE TRAP THE LIVE MEASUREMENT CAUGHT, pinned so it cannot be re-introduced: the ungranted answer
+// is not empty and not a 403 — it carries the INHERITED box-wide privileges. A probe that asked
+// "did the path come back?" or "does it have Datastore.Audit?" would report the blinded storage
+// healthy.
+func TestStoreGrant_InheritedPrivilegesAreNotAGrant(t *testing.T) {
+	if len(permUngranted) == 0 {
+		t.Fatal("fixture wrong: the ungranted answer is NOT empty — that is the whole trap")
+	}
+	if permUngranted["Datastore.Audit"] != 1 {
+		t.Fatal("fixture wrong: the ungranted path DOES carry Datastore.Audit, inherited box-wide")
+	}
+	if s := probeWith(permUngranted, "felhom-backup", true); s.Status != capability.StatusDegraded {
+		t.Fatalf("checking for the wrong privilege reports a blinded storage healthy; got %q", s.Status)
+	}
+	// ...and the privilege actually checked is the one whose absence was measured to blind listing.
+	if storeGrantRequiredPriv != "Datastore.AllocateSpace" {
+		t.Fatalf("the probed privilege changed to %q — re-measure before trusting it", storeGrantRequiredPriv)
+	}
+}
+
+// ── SCENARIO B — a newborn tier is still silent ──────────────────────────────────────────────
+//
+// A storage the agent IS allowed to read but which simply holds no archives yet is HEALTHY. The
+// probe must not look at content at all, or every freshly provisioned box alarms and the signal dies.
+//
+// COMPANION RED-PROOF (observed): make the probe degrade on an empty content listing instead of on
+// the permission — a granted-but-empty storage then reports degraded, i.e. every newborn box alarms.
+func TestStoreGrant_GrantedButEmptyIsHealthy(t *testing.T) {
+	s := probeWith(permGranted, "felhom-pbs", true)
+	if s.Status != capability.StatusOK {
+		t.Fatalf("a readable tier is healthy whether or not it holds archives yet; got %q (%s)", s.Status, s.Reason)
+	}
+	if s.Reason != "" {
+		t.Fatalf("a healthy probe carries no reason; got %q", s.Reason)
+	}
+}
+
+// ── SCENARIO C — the two states are distinguishable at a glance ──────────────────────────────
+func TestStoreGrant_ForbiddenAndNewbornAreDistinguishable(t *testing.T) {
+	forbidden := probeWith(permUngranted, "felhom-backup", true)
+	newborn := probeWith(permGranted, "felhom-pbs", true)
+
+	if forbidden.Status == newborn.Status {
+		t.Fatalf("the two states must differ — today both read as 'no settled archive yet'; got %q for both", forbidden.Status)
+	}
+	if forbidden.Name == newborn.Name {
+		t.Fatalf("each tier needs its own capability id, or one tier's fault hides another's; got %q twice", forbidden.Name)
+	}
+}
+
+// §8.3, weighed once and pinned: a box with NO dedicated target ("local" — host-install's own
+// DEGRADED fallback) must not turn an ordinary configuration into an operator page. It is still
+// probed and still reported; only the paging differs.
+func TestStoreGrant_TheFallbackTargetIsNotCritical(t *testing.T) {
+	if storeGrantCritical("local") {
+		t.Fatal("a box whose backup target is the 'local' fallback must not page the operator about " +
+			"an ordinary, documented configuration")
+	}
+	for _, dedicated := range []string{"felhom-backup", "felhom-pbs", "some-nvme"} {
+		if !storeGrantCritical(dedicated) {
+			t.Fatalf("a DEDICATED target that cannot be read is user-facing and must be critical; %q was not", dedicated)
+		}
+	}
+	// The fallback is still reported — silence for it would be the original defect, scoped smaller.
+	if s := probeWith(permUngranted, "local", storeGrantCritical("local")); s.Status != capability.StatusDegraded {
+		t.Fatalf("the fallback target must still report degraded when unreadable; got %q", s.Status)
+	}
+}
+
+// A probe that cannot ask must never answer "ok" — unknown reported as healthy is worse than no
+// probe, because it looks like coverage.
+func TestStoreGrant_UnreachablePVEIsDegradedNotOK(t *testing.T) {
+	s := storeGrantStatus(context.Background(), nil, "felhom-backup", true, nil)
+	if s.Status != capability.StatusDegraded {
+		t.Fatalf("an unaskable probe must be DEGRADED, never ok; got %q", s.Status)
+	}
+	if s.Reason == "" {
+		t.Fatal("it must say why it could not ask")
+	}
+}
+
+// ── SCENARIO H — the seam ────────────────────────────────────────────────────────────────────
+//
+// This project's "built but never wired" count reached six last week. The fix for a SILENCE must not
+// itself be silent. AST, not grep: a commented-out call still contains the string.
+func TestMainWiresTheStoreGrantProbe(t *testing.T) {
+	f := parseMainForWiring(t)
+
+	var wired bool
+	ast.Inspect(f, func(n ast.Node) bool {
+		call, ok := n.(*ast.CallExpr)
+		if !ok {
+			return true
+		}
+		if id, ok := call.Fun.(*ast.Ident); ok && id.Name == "storeGrantStatuses" {
+			wired = true
+		}
+		return true
+	})
+	if !wired {
+		t.Error("main.go never calls storeGrantStatuses — the probe would exist and report to nobody, " +
+			"which is precisely the silence R-185 is about")
+	}
+}
+
+// ── R-190 — the grant repairs itself, and the repair is VISIBLE ──────────────────────────────
+//
+// R-190 is a storage grant that demonstrably worked at 04:44 on 2026-08-03 and was gone by 09:24,
+// with a host reinstall, logged `pveum` activity and cluster-log entries all ruled out. The cause is
+// open; the resilience is not conditional on it.
+//
+// The half that matters is the RECORD. R-190's own words: the probe sees the state, nothing sees the
+// transition. A self-repair that leaves only "ok" behind destroys the only evidence a loss happened,
+// so a recurring loss becomes undetectable forever — strictly worse than the fault it fixes.
+
+// fakeRepairRunner records wrapper invocations and can be made to fail.
+type fakeRepairRunner struct {
+	calls [][]string
+	fail  bool
+}
+
+func (f *fakeRepairRunner) Run(_ context.Context, name string, args ...string) ([]byte, []byte, error) {
+	f.calls = append(f.calls, append([]string{name}, args...))
+	if f.fail {
+		return nil, []byte("pveum: refused"), errors.New("exit status 2")
+	}
+	return nil, nil, nil
+}
+
+func newRepairer(f *fakeRepairRunner) *storeGrantRepairer {
+	return &storeGrantRepairer{run: f.Run, log: slog.New(slog.NewTextHandler(io.Discard, nil))}
+}
+
+// ── SCENARIO F — the repair is BOUNDED ───────────────────────────────────────────────────────
+//
+// COMPANION RED-PROOF (observed 2026-08-04): make mayAttempt always return true (drop the
+// storeGrantRepairMinInterval check) →
+//
+//	--- FAIL: TestGrantRepair_IsBounded
+//	    storegrant_test.go: a repair must not run on every cycle; 5 cycles produced 5 attempt(s)
+//
+// which is a re-grant every report cycle, forever, against a fault an ACL cannot fix. Restored.
+func TestGrantRepair_IsBounded(t *testing.T) {
+	f := &fakeRepairRunner{}
+	r := newRepairer(f)
+	// Jittered, so the series never lands exactly on the interval boundary — a perfectly regular
+	// series is how a threshold test passes its own mutation, which has happened here before.
+	base := time.Date(2026, 8, 4, 9, 17, 43, 0, time.UTC)
+	offsets := []time.Duration{0, 13*time.Minute + 7*time.Second, 27*time.Minute + 51*time.Second,
+		41*time.Minute + 19*time.Second, 55*time.Minute + 3*time.Second}
+	attempts := 0
+	for _, off := range offsets {
+		if r.mayAttempt("felhom-backup", base.Add(off)) {
+			attempts++
+		}
+	}
+	if attempts != 1 {
+		t.Fatalf("a repair must not run on every cycle; %d cycles produced %d attempt(s) within %s",
+			len(offsets), attempts, storeGrantRepairMinInterval)
+	}
+	// ...and once the interval has genuinely passed, it may try again — a bound is not a ban.
+	if !r.mayAttempt("felhom-backup", base.Add(storeGrantRepairMinInterval+2*time.Minute+11*time.Second)) {
+		t.Fatal("after the interval a repair must be allowed again — otherwise one failure disables the repair forever")
+	}
+	// A DIFFERENT tier is not throttled by this one's attempt.
+	if !r.mayAttempt("felhom-pbs", base.Add(time.Minute)) {
+		t.Fatal("the bound must be per tier — one tier's attempt must not suppress another's")
+	}
+}
+
+// A nil repairer (or one with no runner) never attempts, and never panics.
+func TestGrantRepair_NilIsSafe(t *testing.T) {
+	var r *storeGrantRepairer
+	if r.mayAttempt("felhom-backup", time.Now()) {
+		t.Fatal("a nil repairer must never claim an attempt")
+	}
+	if (&storeGrantRepairer{}).mayAttempt("felhom-backup", time.Now()) {
+		t.Fatal("a repairer with no runner must never claim an attempt")
+	}
+}
+
+// The repair calls the EXISTING wrapper verb, with the storage id — no new privileged surface.
+func TestGrantRepair_CallsTheExistingWrapperVerb(t *testing.T) {
+	f := &fakeRepairRunner{}
+	r := newRepairer(f)
+	if err := r.repair(context.Background(), "felhom-backup"); err != nil {
+		t.Fatalf("repair should succeed with a healthy runner: %v", err)
+	}
+	if len(f.calls) != 1 {
+		t.Fatalf("exactly one wrapper invocation expected; got %d", len(f.calls))
+	}
+	got := f.calls[0]
+	want := []string{"/usr/local/sbin/felhom-backup-target-apply", "grant", "felhom-backup"}
+	if len(got) != len(want) {
+		t.Fatalf("wrapper argv = %v, want %v", got, want)
+	}
+	for i := range want {
+		if got[i] != want[i] {
+			t.Fatalf("wrapper argv = %v, want %v — the sudoers vector is `grant *`; anything else is a policy change", got, want)
+		}
+	}
+}
+
+// A repair that FAILS must surface the failure, not swallow it (Scenario E's precondition).
+func TestGrantRepair_FailureIsReturned(t *testing.T) {
+	f := &fakeRepairRunner{fail: true}
+	if err := newRepairer(f).repair(context.Background(), "felhom-backup"); err == nil {
+		t.Fatal("a failed wrapper run must return its error — a repair that cannot run must never read as done")
+	}
+}
+
+// ── SCENARIO D (the half that matters) — the REPAIR MUST BE VISIBLE ──────────────────────────
+//
+// A repair that leaves only "ok" behind is worse than the fault: the tier works, and the fact that a
+// permission vanished is gone with it. R-190 exists because nothing saw the transition.
+//
+// The channel is the hub's EXISTING ok→degraded→ok edge (§8.5) — nothing new was built. That only
+// works if the agent deliberately reports ONE degraded cycle after repairing, and if the explanation
+// rides the field the hub actually puts in the operator's e-mail. The hub's message is built from the
+// capability NAME and FEATURE (`internal/monitor/host_capability.go` emitTransition) — **not** from
+// Reason — so the Feature must carry it.
+//
+// COMPANION RED-PROOF (observed 2026-08-04): after a successful repair, report ok instead —
+//
+//	s.Status = capability.StatusOK; s.Feature unchanged
+//
+// →	--- FAIL: TestGrantRepair_ARepairedGrantIsReportedAsATransition
+//
+//	storegrant_test.go: a self-repair must still report DEGRADED for one cycle so the hub raises
+//	its edge; got "ok" — the loss would be invisible
+//
+// i.e. exactly the silence R-190 is about. Restored.
+func TestGrantRepair_ARepairedGrantIsReportedAsATransition(t *testing.T) {
+	// THE PRODUCTION verdict, not a copy of it. An earlier draft of this test built the Status
+	// itself and asserted its own construction — it would have passed while production reported ok,
+	// which is precisely the silence being guarded against.
+	if pre := probeWith(permUngranted, "felhom-backup", true); pre.Status != capability.StatusDegraded {
+		t.Fatalf("precondition: a missing grant is degraded; got %q", pre.Status)
+	}
+	s := storeGrantRepairedVerdict("felhom-backup", true)
+
+	if s.Status != capability.StatusDegraded {
+		t.Fatalf("a self-repair must still report DEGRADED for one cycle so the hub raises its edge; "+
+			"got %q — the loss would be invisible", s.Status)
+	}
+	// The hub e-mails the FEATURE text. If the explanation is not there, the operator is told a
+	// capability was degraded and never learns it repaired itself or that anything vanished.
+	for _, want := range []string{"MISSING", "RESTORED", "felhom-backup", "R-190"} {
+		if !strings.Contains(s.Feature, want) {
+			t.Fatalf("the Feature text is what the hub puts in the operator's e-mail; it must contain %q. Got: %s", want, s.Feature)
+		}
+	}
+	if !s.Critical {
+		t.Fatal("the transition must be CRITICAL or the hub does not alert on it at all")
+	}
+}
+
+// ── SCENARIO H — the seam ────────────────────────────────────────────────────────────────────
+//
+// The wrapper's `grant` verb is itself a "built but never wired" example: it exists, is
+// sudoers-permitted for any id, and had only ever been called at storage CREATION. The repair must
+// not become the seventh instance. AST, not grep — a commented-out call still contains the string.
+func TestMainWiresTheGrantRepair(t *testing.T) {
+	f := parseMainForWiring(t)
+
+	var built, passed bool
+	ast.Inspect(f, func(n ast.Node) bool {
+		switch node := n.(type) {
+		case *ast.CompositeLit:
+			if id, ok := node.Type.(*ast.Ident); ok && id.Name == "storeGrantRepairer" {
+				built = true
+			}
+		case *ast.CallExpr:
+			if id, ok := node.Fun.(*ast.Ident); ok && id.Name == "storeGrantStatuses" && len(node.Args) == 4 {
+				if a, ok := node.Args[3].(*ast.Ident); ok && a.Name == "grantRepairer" {
+					passed = true
+				}
+			}
+		}
+		return true
+	})
+	if !built {
+		t.Error("main.go never constructs a storeGrantRepairer — nothing would ever repair a lost grant")
+	}
+	if !passed {
+		t.Error("storeGrantStatuses is not passed the repairer — the probe would detect the loss and " +
+			"leave it, which is v0.123.0's behaviour and not R-190's mitigation")
+	}
+}
+
+// The transition must survive a probe that is NOT the one feeding the hub.
+//
+// MEASURED LIVE 2026-08-04, and this test exists because the first implementation failed it in
+// production while every unit test passed: `probeAll` is called independently by the self-check LOG
+// and by the collector building a host-report. The repairing call was the log's; the report three
+// seconds later found the grant present and reported `ok`. The agent's journal had the record and the
+// hub had nothing — the exact silence R-190 is about, re-created inside its own mitigation.
+//
+// COMPANION RED-PROOF (observed): delete the `recentlyRepaired` branch from the healthy path →
+//
+//	--- FAIL: TestGrantRepair_TransitionSurvivesALaterProbe
+//	    storegrant_test.go: a probe AFTER the repair must still report the transition; got "ok" —
+//	    the host-report would carry ok and the operator would never learn the grant vanished
+//
+// Restored.
+func TestGrantRepair_TransitionSurvivesALaterProbe(t *testing.T) {
+	r := newRepairer(&fakeRepairRunner{})
+	// Jittered, never landing on the window boundary.
+	repairedAt := time.Date(2026, 8, 4, 9, 39, 34, 0, time.UTC)
+	r.noteRepaired("felhom-backup", repairedAt)
+
+	// The DECISION a later probe makes — the production function, not the helper it calls. An
+	// earlier draft asserted `recentlyRepaired` directly and its red-proof PASSED, because removing
+	// the latch's USE left the helper untouched.
+	healthy := probeWith(permGranted, "felhom-backup", true)
+	if healthy.Status != capability.StatusOK {
+		t.Fatalf("precondition: a granted tier is ok; got %q", healthy.Status)
+	}
+	got := storeGrantHealthyVerdict("felhom-backup", true,
+		healthy, r.recentlyRepaired("felhom-backup", repairedAt.Add(3*time.Second)))
+	if got.Status != capability.StatusDegraded {
+		t.Fatalf("a probe AFTER the repair must still report the transition; got %q — the host-report "+
+			"would carry ok and the operator would never learn the grant vanished", got.Status)
+	}
+	if !strings.Contains(got.Feature, "RESTORED") {
+		t.Fatalf("the later probe must carry the explanation into the hub's e-mail; got: %s", got.Feature)
+	}
+	// Outside the window it reports plain ok again.
+	late := storeGrantHealthyVerdict("felhom-backup", true,
+		healthy, r.recentlyRepaired("felhom-backup", repairedAt.Add(storeGrantRepairReportWindow+time.Minute)))
+	if late.Status != capability.StatusOK {
+		t.Fatalf("outside the window a healthy tier reports ok; got %q — a permanent degraded state "+
+			"would be its own false alarm", late.Status)
+	}
+	if !r.recentlyRepaired("felhom-backup", repairedAt.Add(14*time.Minute+37*time.Second)) {
+		t.Fatal("the latch must outlast the 900s hub report interval, or the record never reaches the hub")
+	}
+	// ...and it clears on its own rather than latching a box degraded forever.
+	if r.recentlyRepaired("felhom-backup", repairedAt.Add(storeGrantRepairReportWindow+time.Minute+7*time.Second)) {
+		t.Fatal("the latch must clear — a permanent degraded state would be its own false alarm")
+	}
+	// It is per tier.
+	if r.recentlyRepaired("felhom-pbs", repairedAt.Add(time.Second)) {
+		t.Fatal("one tier's repair must not latch another tier's status")
+	}
+	// The window MUST exceed the report interval — the property, asserted rather than assumed.
+	if storeGrantRepairReportWindow <= 15*time.Minute {
+		t.Fatalf("the report window (%s) must exceed the 900s hub report interval, or a transition can "+
+			"be missed entirely", storeGrantRepairReportWindow)
+	}
+}

configs/agent.example.json

@@ -48,10 +48,16 @@
   },
   "local_api": {
     "enable": true,
-    "listen_addr": "192.168.0.162:8443",
+    "listen_addr": "169.254.253.1:8443",
     "cert_file": "/var/lib/felhom-agent/local-api.crt",
     "key_file": "/var/lib/felhom-agent/local-api.key",
-    "token_store": "/var/lib/felhom-agent/local-tokens.log"
+    "token_store": "/var/lib/felhom-agent/local-tokens.log",
+    "island_bridge": "vmbr9",
+    "island_guest_addr": "169.254.253.2/30"
+  },
+  "lan_resolver": {
+    "enable": true,
+    "host_ip": "192.168.0.162"
   },
   "log_level": "info"
 }

configs/build-golden.sh

@@ -26,20 +26,42 @@
 # Build-time registry login for the controller pull (used ONCE inside the build guest, then logged
 # out — never baked): set REGISTRY_USER + REGISTRY_TOKEN in the environment.
 #
-# OS / Docker-data SPLIT (storage-split slice): the golden is built with a SMALL OS rootfs and a
-# SEPARATE Docker-data volume mounted at /var/lib/docker (mp0, backup=1). The baked controller +
-# infra images land on that volume and travel INSIDE the golden archive — so provisioned guests boot
-# from baked images with no registry pull. The split is for RESILIENCE: an isolated OS rootfs stays
-# bootable + agent-recoverable if the Docker volume fills (the controller's prevention layer keeps it
-# from filling). Sizes are env-overridable (OS_SIZE_GB / GOLDEN_DOCKER_GB); provision GROWS the data
-# volume to the per-customer target (bringup.go DataVolGrowGB). backup=1 is MANDATORY on the data mp:
-# without it vzdump EXCLUDES the volume (extra LXC mountpoints default backup=0 — storage-split B3),
-# so the archive would carry NO images and provisioned guests would boot imageless.
+# OS / DATA SPLIT, and since v3.0.0 ONE DATA VOLUME (R-165, decision D-a + variant V-c).
+#
+# The golden is built with a SMALL OS rootfs and a SINGLE data volume (mp0, backup=1) mounted at a
+# NEUTRAL path, /var/lib/felhom. Both consumer paths are binds of subdirectories of it:
+#
+#     /var/lib/felhom/docker     --bind-->  /var/lib/docker    (Docker's data-root)
+#     /var/lib/felhom/sys_drive  --bind-->  /mnt/sys_drive     (the controller's system_data_path)
+#
+# WHAT THIS REPLACED, AND WHY. Until v2.1.0 these were TWO volumes (mp0 16 G at /var/lib/docker,
+# mp1 8 G at /mnt/sys_drive, grown separately at provision). The second one was a fixed ceiling: an
+# app whose local recovery unit outgrew it stopped being backed up even with free space next door.
+# D-a removed the wall rather than moving it — one volume, one free-space figure, no ceiling.
+#
+# WHY A NEUTRAL MOUNT AND NOT SIMPLY NESTING ONE PATH INSIDE THE OTHER. Both simpler shapes were
+# built and measured (SPIKE-r165-phase0-2026-08-03.md); both boot and reboot cleanly, and each breaks
+# a different documented guarantee:
+#   * volume at /var/lib/docker -> customer backups live INSIDE Docker's data-root, so `du` there
+#     stops meaning what it says and the ordinary "clear /var/lib/docker to fix Docker" reflex
+#     destroys every local recovery unit on the box;
+#   * volume at /mnt/sys_drive -> Docker's ENTIRE data-root lands under /mnt, which the controller
+#     container mounts wholesale (`-v /mnt:/mnt:rslave`). Measured: the container then sees
+#     /mnt/sys_drive/docker. The bootstrap's own claim that /mnt "holds only Felhom's
+#     felhom-data-namespace mounts" would become false.
+# The neutral mount breaks neither, for one extra path and one extra fstab line.
+#
+# The split from the OS rootfs is still for RESILIENCE: an isolated rootfs stays bootable +
+# agent-recoverable if the data volume fills (the controller's prevention layer, and since
+# controller v0.192.0 the capture floor, keep it from filling). Size is env-overridable
+# (OS_SIZE_GB / GOLDEN_VOLUME_GB); provision GROWS the one volume (bringup.go DataVolGrowGB).
+# backup=1 is MANDATORY: without it vzdump EXCLUDES the volume (extra LXC mountpoints default
+# backup=0 — storage-split B3), so the archive would carry no images AND no user data.
 set -euo pipefail
 
 # Script provenance — logged into every bake transcript next to the baked controller tag, so an
 # archive can always be traced to the script that produced it. Bump on any behavior change.
-GOLDEN_SCRIPT_VERSION="2.1.0"
+GOLDEN_SCRIPT_VERSION="3.0.0"
 
 VMID="${1:-9100}"
 TEMPLATE="${2:-local:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst}"
@@ -59,25 +81,28 @@ if [ -z "$CONTROLLER_IMAGE" ]; then
   exit 1
 fi
 REGISTRY_HOST="${CONTROLLER_IMAGE%%/*}"
-# OS rootfs size (GiB) and the golden's Docker-data volume size (GiB). Keep GOLDEN_DOCKER_GB just
-# large enough for the baked images + headroom; provision grows it to the per-customer target.
+# OS rootfs size (GiB) and the golden's SINGLE data volume size (GiB).
+#
+# ONE VOLUME MEANS ONE NUMBER (v3.0.0). The retired GOLDEN_SYSDATA_GB has no successor: there is
+# nothing left to size separately. Keep GOLDEN_VOLUME_GB just large enough for the baked images plus
+# headroom for the controller's felhom-data skeleton; provision grows the one volume to the
+# per-customer target (bringup.go DataVolGrowGB).
 OS_SIZE_GB="${OS_SIZE_GB:-32}"
-GOLDEN_DOCKER_GB="${GOLDEN_DOCKER_GB:-16}"
-# The golden's SSD user-data volume (GiB) mounted at /mnt/sys_drive (mp1, backup=1) — the controller's
-# system_data_path. Ships small + near-empty (the controller creates <sys_drive>/felhom-data itself once
-# it's a real mountpoint); provision GROWS it to the per-customer target (bringup.go SysDataGrowGB). Like
-# mp0, backup=1 is MANDATORY: without it vzdump EXCLUDES the volume (extra mountpoints default backup=0 —
-# storage-split B3) and the user-data area would silently fall out of PBS coverage.
-GOLDEN_SYSDATA_GB="${GOLDEN_SYSDATA_GB:-8}"
+# 24 = the retired pair's 16 (docker) + 8 (user-data), so a golden archive carries the same content it
+# did before the merge. It is deliberately NOT a per-customer size: provision grows it.
+GOLDEN_VOLUME_GB="${GOLDEN_VOLUME_GB:-24}"
+# The neutral mount path of the single volume. Both consumer paths are binds of subdirectories of it.
+GOLDEN_VOLUME_MP="/var/lib/felhom"
 
 echo "[golden] build-golden.sh v${GOLDEN_SCRIPT_VERSION} — baking controller ${CONTROLLER_IMAGE}"
-echo "[golden] creating build LXC $VMID (nesting=1,keyctl=1, unprivileged; rootfs ${OS_SIZE_GB}G + Docker-data ${GOLDEN_DOCKER_GB}G @ /var/lib/docker + user-data ${GOLDEN_SYSDATA_GB}G @ /mnt/sys_drive, both backup=1) …"
+echo "[golden] creating build LXC $VMID (nesting=1,keyctl=1, unprivileged; rootfs ${OS_SIZE_GB}G + ONE data volume ${GOLDEN_VOLUME_GB}G @ ${GOLDEN_VOLUME_MP}, backup=1) …"
+# ONE mpN slot. There is deliberately no mp1: that slot held the retired user-data volume, and the
+# whole point of R-165 is that it stops existing rather than being made bigger.
 pct create "$VMID" "$TEMPLATE" \
   --hostname felhom-golden --unprivileged 1 \
   --features nesting=1,keyctl=1 \
   --rootfs "${ROOTFS_STORAGE}:${OS_SIZE_GB}" --cores 2 --memory 2048 \
-  --mp0 "${ROOTFS_STORAGE}:${GOLDEN_DOCKER_GB},mp=/var/lib/docker,backup=1" \
-  --mp1 "${ROOTFS_STORAGE}:${GOLDEN_SYSDATA_GB},mp=/mnt/sys_drive,backup=1" \
+  --mp0 "${ROOTFS_STORAGE}:${GOLDEN_VOLUME_GB},mp=${GOLDEN_VOLUME_MP},backup=1" \
   --net0 "name=eth0,bridge=${BRIDGE},ip=dhcp" --onboot 0
 
 echo "[golden] starting + installing Docker (official repo, trixie channel) …"
@@ -106,8 +131,10 @@ echo "[golden] baking daemon.json: classic overlay2 driver (containerd-snapshott
 # The classic overlay2 driver stores EVERYTHING (images + overlay + volumes) under data-root
 # (/var/lib/docker) = the data volume, which is exactly what "one data-root = one partition for all
 # images + overlay" requires. It also makes the controller's statfs("/") (its overlay root) report the
-# DATA volume, which the prevention layer depends on. /var/lib/docker is the mp0 mount (mounted empty
-# before docker installs), so data-root needs no override. Log caps kill the most common runaway.
+# DATA volume, which the prevention layer depends on — MEASURED to still hold under the v3.0.0 merged
+# layout (a container's `df /` reports the single volume, phase-0 spike). Since v3.0.0 /var/lib/docker
+# is a BIND of <volume>/docker rather than the mp0 mount itself, wired immediately below; data-root
+# still needs no override because the path is unchanged. Log caps kill the most common runaway.
 pct exec "$VMID" -- bash -c 'mkdir -p /etc/docker; cat > /etc/docker/daemon.json <<JSON
 {
   "features": { "containerd-snapshotter": false },
@@ -115,6 +142,32 @@ pct exec "$VMID" -- bash -c 'mkdir -p /etc/docker; cat > /etc/docker/daemon.json
   "log-opts": { "max-size": "10m", "max-file": "3" }
 }
 JSON'
+echo "[golden] wiring the single data volume (R-165 variant V-c): ${GOLDEN_VOLUME_MP}/{docker,sys_drive} -> binds …"
+# docker-ce has already populated /var/lib/docker ON THE ROOTFS by now (it auto-starts on install), so
+# the content is MOVED onto the volume before the bind is laid over the top. Doing it the other way
+# round would hide those files under the bind and silently ship a golden whose baked images are on the
+# rootfs — the exact failure class the assertions below exist to catch.
+#
+# /etc/fstab, not a hand-run mount: systemd's fstab generator orders both binds under local-fs.target,
+# which precedes basic.target and therefore docker.service. MEASURED across 3 reboots per variant in
+# the phase-0 spike — the ordering worry that motivated the probe did not materialise.
+pct exec "$VMID" -- bash -c "
+  set -e
+  systemctl stop docker docker.socket containerd 2>/dev/null || true
+  mkdir -p '${GOLDEN_VOLUME_MP}/docker' '${GOLDEN_VOLUME_MP}/sys_drive'
+  if [ -d /var/lib/docker ] && [ -n \"\$(ls -A /var/lib/docker 2>/dev/null)\" ]; then
+    cp -a /var/lib/docker/. '${GOLDEN_VOLUME_MP}/docker'/
+    rm -rf /var/lib/docker/*
+  fi
+  mkdir -p /var/lib/docker /mnt/sys_drive
+  printf '%s /var/lib/docker none bind 0 0\n' '${GOLDEN_VOLUME_MP}/docker'    >> /etc/fstab
+  printf '%s /mnt/sys_drive none bind 0 0\n'  '${GOLDEN_VOLUME_MP}/sys_drive' >> /etc/fstab
+  systemctl daemon-reload
+  mount /var/lib/docker
+  mount /mnt/sys_drive
+  systemctl start containerd
+"
+
 echo "[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) …"
 # RESTART (not start): docker-ce auto-starts on install with the DEFAULT config, so it is already
 # running by now; only a restart picks up the daemon.json just written (overlay2 + log caps).
@@ -122,12 +175,20 @@ pct exec "$VMID" -- bash -c 'systemctl restart docker; sleep 3; docker run --rm
 # Guard: the image store MUST be on the data volume now. /var/lib/containerd holding the images would
 # mean containerd-snapshotter is still on (the split would leave images on the rootfs).
 pct exec "$VMID" -- bash -c 'drv=$(docker info 2>/dev/null | sed -n "s/.*Storage Driver: //p"); [ "$drv" = "overlay2" ] || { echo "[golden] FATAL: storage driver is $drv, expected overlay2 — images would not land on the data volume"; exit 1; }'
-# Confirm /var/lib/docker is genuinely the dedicated volume, not the rootfs (catch a silent mp miss).
-pct exec "$VMID" -- bash -c 'findmnt -no SOURCE,FSTYPE /var/lib/docker | grep -q . && echo "  /var/lib/docker is a separate mount: $(findmnt -no SOURCE,FSTYPE /var/lib/docker)" || { echo "[golden] FATAL: /var/lib/docker is NOT a separate mount — the mp0 split did not take"; exit 1; }'
-# Same guard for the SSD user-data volume (mp1): /mnt/sys_drive must be its own mount, not the rootfs
-# device — otherwise the controller's system_data_path lands on the OS drive and it warns (the whole
-# point of this volume is to clear that warning).
-pct exec "$VMID" -- bash -c 'findmnt -no SOURCE,FSTYPE /mnt/sys_drive | grep -q . && echo "  /mnt/sys_drive is a separate mount: $(findmnt -no SOURCE,FSTYPE /mnt/sys_drive)" || { echo "[golden] FATAL: /mnt/sys_drive is NOT a separate mount — the mp1 split did not take"; exit 1; }'
+# ASSERTION 1 (RETARGETED v3.0.0, not removed). /var/lib/docker must be a real mount — now the V-c
+# bind of <volume>/docker rather than the mp0 mount itself. Still fails closed on the same failure:
+# if the bind did not take, Docker's data-root silently sits on the OS rootfs and the golden ships
+# its baked images there.
+pct exec "$VMID" -- bash -c 'findmnt -no SOURCE,FSTYPE /var/lib/docker | grep -q . && echo "  /var/lib/docker is a real mount: $(findmnt -no SOURCE,FSTYPE /var/lib/docker | head -1)" || { echo "[golden] FATAL: /var/lib/docker is NOT a mount — the V-c docker bind did not take, so the baked images would land on the OS rootfs"; exit 1; }'
+# ASSERTION 2 (RETARGETED v3.0.0). /mnt/sys_drive must be a real mount — now the V-c bind of
+# <volume>/sys_drive. Otherwise the controller's system_data_path lands on the OS drive and it warns
+# (clearing that warning is the whole point of the volume).
+pct exec "$VMID" -- bash -c 'findmnt -no SOURCE,FSTYPE /mnt/sys_drive | grep -q . && echo "  /mnt/sys_drive is a real mount: $(findmnt -no SOURCE,FSTYPE /mnt/sys_drive | head -1)" || { echo "[golden] FATAL: /mnt/sys_drive is NOT a mount — the V-c sys_drive bind did not take, so the controller system_data_path would be the OS rootfs"; exit 1; }'
+# ASSERTION 2b (NEW v3.0.0 — the invariant the merge is FOR). Both paths must be backed by the SAME
+# device, i.e. ONE filesystem with ONE free-space figure. Two devices here is the S2 shape the R-165
+# spike ranked strictly WORSE than the split it replaced: every assertion satisfied, the ceiling still
+# there, and a shared pool neither `df` can see coming.
+pct exec "$VMID" -- bash -c 'n=$(df --output=source /var/lib/docker /mnt/sys_drive | tail -n +2 | sort -u | wc -l); [ "$n" = "1" ] && echo "  both paths are ONE filesystem: $(df --output=source,avail /var/lib/docker | tail -1)" || { echo "[golden] FATAL: /var/lib/docker and /mnt/sys_drive are on $n DIFFERENT filesystems — that is the S2 shape (two ceilings), not the R-165 merge"; exit 1; }'
 
 echo "[golden] baking the in-guest controller image $CONTROLLER_IMAGE (no registry cred at deploy) …"
 # docker login is used ONCE here on the trusted build host, then logged out before archiving so
@@ -228,7 +289,11 @@ mount --make-rshared /mnt
 # Otherwise still DE-PRIVILEGED: disk EXECUTION (scan/format/mount) stays the agent's — NO --privileged,
 # no /dev, no /etc/fstab. Bootstrap config (ro), data volume, stacks dir (same-path), the /mnt :rslave
 # view, and the docker socket. The controller reaches the agent's local API for disk management.
-docker run -d --name felhom-controller --restart unless-stopped "${HOSTNAME_ARGS[@]}" \
+# R-523: `always`, not `unless-stopped`. It covers ONE extra case only — a Docker daemon restart after
+# the container was stopped by hand. Neither policy restarts a container that `docker kill`/`docker
+# stop` ended (measured 2026-09-15, Docker 29.8.0, evidence-p1fixes-2026-09-15/A1); the host agent's
+# controller supervisor (felhom-agent v0.131.0, internal/localapi/controllersupervisor.go) covers that.
+docker run -d --name felhom-controller --restart always "${HOSTNAME_ARGS[@]}" \
   -e FELHOM_BOOTSTRAP_PATH=/etc/felhom-bootstrap/bootstrap.json \
   -v /etc/felhom-bootstrap:/etc/felhom-bootstrap:ro \
   -v felhom-controller-data:/opt/docker/felhom-controller \
@@ -307,26 +372,36 @@ pct exec "$VMID" -- bash -c '
 
 echo "[golden] stop + archive …"
 pct stop "$VMID"
-# --mode stop with mp0 + mp1 backup=1 → BOTH the Docker-data volume (baked images) and the
-# /mnt/sys_drive user-data volume are INCLUDED. The log below MUST show "including mount point mp0"
-# AND "including mount point mp1" — if either shows "excluding … (disabled)" the backup flag was lost
-# and the archive carries no images / no user-data volume (storage-split B3 trap).
+# --mode stop with mp0 backup=1 → the SINGLE data volume (baked images AND the user-data area) is
+# INCLUDED. The log MUST show "including mount point mp0" and must NOT show it being excluded — an
+# exclusion means the backup flag was lost and the archive carries neither (storage-split B3 trap).
+# Since v3.0.0 there is no mp1; the guard that covered it is retargeted below rather than deleted,
+# because a guard whose pattern can no longer match is a guard that has silently stopped guarding.
 vzdump "$VMID" --storage "$ARCHIVE_STORAGE" --mode stop --compress zstd 2>&1 | tee /tmp/golden-vzdump.log | grep -iE "including mount point|excluding|archive file size|Finished Backup" || true
 if grep -q "excluding volume mount point mp0" /tmp/golden-vzdump.log; then
   echo "[golden] FATAL: mp0 (/var/lib/docker) was EXCLUDED from the archive — backup=1 was lost; the golden would carry no images. Aborting."
   exit 1
 fi
-if grep -q "excluding volume mount point mp1" /tmp/golden-vzdump.log; then
-  echo "[golden] FATAL: mp1 (/mnt/sys_drive) was EXCLUDED from the archive — backup=1 was lost; the golden would carry no user-data volume. Aborting."
+# ASSERTION 4 (RETARGETED v3.0.0). The mp1 guard used to catch "the user-data volume fell out of the
+# archive". After the merge there is no mp1 — so the same failure now looks like the volume being
+# mounted at the WRONG PATH, which would carry the images but not the user-data area. Assert the
+# inclusion line names the volume's actual mount path.
+if ! grep -q "including mount point mp0 ('${GOLDEN_VOLUME_MP}')" /tmp/golden-vzdump.log; then
+  echo "[golden] FATAL: the archive's mp0 is not ${GOLDEN_VOLUME_MP} — the single data volume is mounted somewhere unexpected, so the archive would not carry both the baked images and the user-data area. Aborting."
+  grep -iE "mount point" /tmp/golden-vzdump.log || true
+  exit 1
+fi
+# ASSERTION 5 (RETARGETED v3.0.0). There must be NO mp1 in the archive at all. A leftover second
+# volume means the merge did not take and this golden would ship the very ceiling R-165 removed.
+if grep -qE "mount point mp1" /tmp/golden-vzdump.log; then
+  echo "[golden] FATAL: the archive still carries an mp1 — the R-165 merge did not take and this golden would ship a second, ceilinged volume. Aborting."
   exit 1
 fi
 grep -q "including mount point mp0" /tmp/golden-vzdump.log \
   || echo "[golden] WARN: could not confirm mp0 inclusion in the vzdump log — verify manually before using this archive."
-grep -q "including mount point mp1" /tmp/golden-vzdump.log \
-  || echo "[golden] WARN: could not confirm mp1 inclusion in the vzdump log — verify manually before using this archive."
 
 VOLID=$(pvesm list "$ARCHIVE_STORAGE" --content backup 2>/dev/null | awk -v v="$VMID" '$1 ~ ("vzdump-lxc-" v "-") {print $1}' | sort | tail -1)
-echo "[golden] DONE. golden archive volid: ${VOLID:-<check ${ARCHIVE_STORAGE} dump dir>} (rootfs ${OS_SIZE_GB}G + Docker-data ${GOLDEN_DOCKER_GB}G + user-data ${GOLDEN_SYSDATA_GB}G, all in the archive)"
+echo "[golden] DONE. golden archive volid: ${VOLID:-<check ${ARCHIVE_STORAGE} dump dir>} (rootfs ${OS_SIZE_GB}G + ONE data volume ${GOLDEN_VOLUME_GB}G @ ${GOLDEN_VOLUME_MP}, all in the archive)"
 
 #-------------------------------------------------------------------------------
 # Publish to Gitea (BUNDLE slice) — make this golden fetchable by the host-bootstrap script.

configs/felhom-agent.sudoers

@@ -133,6 +133,28 @@ Cmnd_Alias FELHOM_CONTROLLERSWAP = \
 Cmnd_Alias FELHOM_STALELOCK = \
   /usr/sbin/pct unlock [0-9]*
 
+# Restore-test scratch teardown (F-LEAK, Campaign 8, v0.110.0). A restore-test whose restore FAILS
+# leaves a scratch guest the API token CANNOT destroy: `FelhomAgentGuest` is granted at /pool/felhom and
+# a guest joins that pool only when its restore COMPLETES, so a failed restore leaves a pool-less guest
+# out of reach (403 VM.Allocate) holding its disks until a human removes it.
+#
+# TWO API-SIDE FIXES WERE TRIED AND BOTH REFUTED LIVE on 2026-07-28, which is why this grant exists:
+#   1. Adopt the stranded guest into the pool, then retry. `PUT /pools/{pool}` ALSO requires
+#      VM.Allocate on the VM being added — pool membership cannot bootstrap its own authority.
+#   2. Grant FelhomAgentGuest per-path at /vms/990000..990009. Durable for exactly one use per slot:
+#      PVE's own destroy path calls `AccessControl::remove_vm_access($vmid)` (LXC.pm:906), which DELETES
+#      every ACL at /vms/<vmid> (AccessControl.pm:1898). The grant is consumed by the operation it
+#      authorises, so after ten teardowns the band is ungranted and the defect returns.
+#
+# WHY THIS IS THE TIGHTEST AVAILABLE FENCE, not a widening: sudo matches the vmid LITERALLY, so
+# `99000[0-9]` is exactly the ten-slot scratch band the restore-test picks from — nothing else. There is
+# no `[0-9]*` coarse allowlist here on purpose: unlike `pct unlock`, this op DESTROYS, so the band must
+# be in the policy and not merely validated in the agent. Even a compromised agent asking for
+# `pct destroy 9201` is refused by sudo itself. Unlike an ACL, a sudoers rule is not consumed by use.
+# The agent re-checks the band in code before exec (defence in depth); this is the outer fence.
+Cmnd_Alias FELHOM_SCRATCH_TEARDOWN = \
+  /usr/sbin/pct destroy 99000[0-9] --purge
+
 # Network storage / NAS (Part A1, SPIKE-nas-storage-2026-06-29). The agent mounts a customer NAS share
 # HOST-SIDE under /mnt/felhom-drives/<name> via a systemd .automount (+ .mount) pair so it propagates
 # into the guest through the existing shared bind (an unprivileged LXC cannot mount NFS/CIFS itself).
@@ -223,6 +245,15 @@ Cmnd_Alias FELHOM_SSHD = \
 # blind to an `applied`-but-401 tier. It is NOT a general file-read: the wrapper pins the directory
 # and prefix-asserts the resolved path, and the id grammar admits no slash. The secret goes to
 # STDOUT, never argv — sudo logs argv.
+# E-2a: the backup-target storage shim. Creating a PVE storage needs Datastore.Allocate at /storage
+# and the grant needs Permissions.Modify -- the agent holds NEITHER by design (blast-radius
+# containment; Permissions.Modify would let it rewrite its own authority). Both live behind this
+# fixed-vocabulary root shim instead, exactly like the mkfs and pbs-apply wrappers. The wrapper has
+# NO storage-removal path, enforces is_mountpoint 1, and refuses a target on the root device.
+Cmnd_Alias FELHOM_BACKUPTARGET = \
+  /usr/local/sbin/felhom-backup-target-apply create *, \
+  /usr/local/sbin/felhom-backup-target-apply grant *
+
 Cmnd_Alias FELHOM_PBSDR = \
   /usr/local/sbin/felhom-pbs-apply create *, \
   /usr/local/sbin/felhom-pbs-apply reconcile *, \
@@ -268,10 +299,17 @@ Cmnd_Alias FELHOM_SELFHEAL = \
 # argument after the numeric vmid is a literal, so the grant cannot be widened by anything the guest or
 # the hub says. The address read is deliberately NOT duplicated here — it is already FELHOM_DNSMASQ's,
 # and the same command must not be granted twice under two names.
+# OS updates, guest fast lane (`11-os-updates.md` §5.4.1, agent v0.140.0). The ONLY entry: the root wrapper with
+# one plan file in the agent's own os/ dir. Every safety rule (no removal, no downgrade, no new or unlisted package,
+# Debian origin only, the box's own customer guest only) lives in the wrapper, red-proved per rule
+# (configs/test_felhom_os_apply.py). The agent gets NO apt grant of its own.
+Cmnd_Alias FELHOM_OSAPPLY = \
+  /usr/local/sbin/felhom-os-apply --plan /var/lib/felhom-agent/os/plan-*.json
+
 Cmnd_Alias FELHOM_GUESTNET = \
   /usr/sbin/pct exec [0-9]* -- ip route show default, \
   /usr/sbin/pct exec [0-9]* -- cat /etc/network/interfaces, \
   /usr/sbin/pct exec [0-9]* -- pgrep -x dhclient, \
   /usr/sbin/pct exec [0-9]* -- dhclient -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0
 
-felhom-agent ALL=(root) NOPASSWD: FELHOM_MOUNT, FELHOM_DISK, FELHOM_PROVISION, FELHOM_FORMAT, FELHOM_DNSMASQ, FELHOM_GUESTHOOK, FELHOM_INTERMEDIARY, FELHOM_CONTROLLERSWAP, FELHOM_STALELOCK, FELHOM_NETMOUNT, FELHOM_WG, FELHOM_SELFUPDATE, FELHOM_SSHD, FELHOM_OOB, FELHOM_PBSDR, FELHOM_SELFHEAL, FELHOM_ESCROW, FELHOM_GUESTNET
+felhom-agent ALL=(root) NOPASSWD: FELHOM_MOUNT, FELHOM_DISK, FELHOM_PROVISION, FELHOM_FORMAT, FELHOM_DNSMASQ, FELHOM_GUESTHOOK, FELHOM_INTERMEDIARY, FELHOM_CONTROLLERSWAP, FELHOM_STALELOCK, FELHOM_NETMOUNT, FELHOM_WG, FELHOM_SELFUPDATE, FELHOM_SSHD, FELHOM_OOB, FELHOM_PBSDR, FELHOM_BACKUPTARGET, FELHOM_SELFHEAL, FELHOM_ESCROW, FELHOM_GUESTNET, FELHOM_SCRATCH_TEARDOWN, FELHOM_OSAPPLY

configs/felhom-backup-target-apply

@@ -0,0 +1,111 @@
+#!/bin/bash
+#===============================================================================
+# felhom-backup-target-apply — the ONLY path the felhom-agent sudoers permits for creating the
+# whole-guest backup TARGET storage and granting the agent access to it (E-2a).
+#
+# WHY A WRAPPER AT ALL. Creating a PVE storage needs `Datastore.Allocate` at `/storage`, and the ACL
+# grant needs `Permissions.Modify`. The agent holds NEITHER by design — its token is scoped per
+# storage path for blast-radius containment, and `Permissions.Modify` would let it rewrite its own
+# authority. Widening the PVE role to make the move possible would trade the entire containment model
+# for one feature. So the privileged half lives here: a minimal, auditable root shim with a fixed
+# vocabulary, exactly like felhom-mkfs-guarded and felhom-pbs-apply.
+#
+# THE NO-DELETE LAW (inherited from felhom-pbs-apply, same reasoning class). This wrapper contains NO
+# storage-removal path of any kind. `pvesm remove` on a dir storage does not delete the archives, but
+# it DOES silently orphan a configured backup tier, and a "cleanup" verb here would be reachable by
+# any bug in the agent. Retiring a target is a deliberate operator op, not this tool. Grep-assertable;
+# do not add one.
+#
+# THE TWO LAWS E-1 PAID FOR ON LIVE HARDWARE, both enforced here rather than trusted to the caller:
+#
+#   F-1  the storage path must BE the drive's own mountpoint. A subdirectory fails the agent's
+#        exactMount check, so the target reports `disconnected` FOREVER and its durable id degrades
+#        off the filesystem UUID. Enforced: `mountpoint -q` must pass on the exact path given.
+#
+#   F-2  --is_mountpoint 1 is not optional. Without it, an unplugged or late-mounting drive leaves a
+#        bare directory on the ROOT filesystem and vzdump writes the whole-guest backup onto the
+#        system drive — the exact device the whole change exists to escape — while PVE reports the
+#        storage `active` and advertises the root filesystem's free space. Proven live: the unguarded
+#        form had already created dump/ on pve-root. Hardcoded below; not a caller-supplied flag.
+#
+# Ops (all non-secret; nothing here touches a credential, so nothing arrives on stdin):
+#   create <id> <mountpoint>
+#           Create a `dir` storage with content=backup at <mountpoint>, is_mountpoint 1.
+#           IDEMPOTENT: an existing entry with the SAME path is accepted (re-run safe, and the
+#           installer re-run path depends on it). An existing entry with a DIFFERENT path is REFUSED
+#           — silently repointing a live backup target is the failure this whole arc closes.
+#   grant  <id>
+#           The dual grant: FelhomAgentStore on /storage/<id> to the agent user AND token (privsep
+#           intersection — a token's rights are the intersection, so granting one is granting neither).
+#           Without it every backup 403s on first run (E-1 finding F-3, found by the first real backup).
+#===============================================================================
+set -euo pipefail
+
+die() { echo "felhom-backup-target-apply: REFUSED: $*" >&2; exit 1; }
+
+op="${1:-}"; id="${2:-}"
+[[ -n "$op" && -n "$id" ]] || die "usage: felhom-backup-target-apply <create|grant> <storage-id> [mountpoint]"
+
+# Storage id: PVE grammar, conservative. Also the ACL path component — no slashes possible.
+[[ "$id" =~ ^[A-Za-z][A-Za-z0-9_.-]{0,27}$ ]] || die "bad storage id ($id)"
+
+STORECFG=/etc/pve/storage.cfg
+
+# current_path_of <id> — the configured `path` of dir storage <id>, or "" when absent/not-a-dir.
+current_path_of() {
+  awk -v want="dir: $1" '
+    $0 == want { found=1; next }
+    found && /^[a-z]+: / { exit }
+    found && $1 == "path" { print $2; exit }
+  ' "$STORECFG" 2>/dev/null || true
+}
+
+case "$op" in
+create)
+  [[ $# -eq 3 ]] || die "create takes <id> <mountpoint>"
+  mp="$3"
+  # Absolute, normalized, no traversal, no shell metacharacters. The value reaches pvesm and the
+  # filesystem, so it is validated here rather than assumed well-formed.
+  [[ "$mp" = /* ]]            || die "mountpoint must be absolute ($mp)"
+  [[ "$mp" != *".."* ]]       || die "mountpoint must not contain .. ($mp)"
+  [[ "$mp" =~ ^[A-Za-z0-9/_.-]+$ ]] || die "mountpoint has unexpected characters ($mp)"
+  [[ "$mp" != "/" ]]          || die "refusing / as a backup target"
+
+  # F-1 + F-2, checked as one: the path must BE a mountpoint right now. A bare directory here is
+  # precisely the silent-retarget shape, and is_mountpoint would make PVE refuse it later anyway —
+  # better to refuse now, with a reason, than to create a storage that can never activate.
+  mountpoint -q "$mp" || die "$mp is not a mountpoint — the backup target must be the drive's OWN mountpoint (F-1), and an unmounted path would silently retarget onto the system drive (F-2)"
+
+  # Never the system disk: a target on the root filesystem is not drive-loss protection, it is the
+  # thing we are escaping. The root device and the candidate's device are compared, not their paths.
+  root_dev="$(findmnt -no SOURCE / 2>/dev/null || true)"
+  mp_dev="$(findmnt -no SOURCE "$mp" 2>/dev/null || true)"
+  [[ -n "$mp_dev" ]] || die "could not resolve the backing device of $mp"
+  [[ "$mp_dev" != "$root_dev" ]] || die "$mp is backed by the ROOT device ($root_dev) — a backup target there protects against corruption only, never drive loss"
+
+  existing="$(current_path_of "$id")"
+  if [[ -n "$existing" ]]; then
+    if [[ "$existing" == "$mp" ]]; then
+      echo "felhom-backup-target-apply: storage $id already exists at $mp — nothing to do (idempotent)" >&2
+      exit 0
+    fi
+    die "storage $id already exists at $existing — refusing to repoint it at $mp (a live backup target is never silently moved)"
+  fi
+
+  # is_mountpoint 1 is HARDCODED (F-2). content=backup only: this storage exists for vzdump archives
+  # and must never become a place guests are allocated on.
+  pvesm add dir "$id" --path "$mp" --content backup --is_mountpoint 1 >&2
+  echo "felhom-backup-target-apply: created dir storage $id at $mp (content=backup, is_mountpoint 1)" >&2
+  ;;
+grant)
+  [[ $# -eq 2 ]] || die "grant takes only <id>"
+  # BOTH, always. A privsep token's rights are the intersection of the user's and the token's ACLs,
+  # so granting one of the two grants nothing usable.
+  pveum acl modify "/storage/$id" --users felhom-agent@pve --roles FelhomAgentStore >&2
+  pveum acl modify "/storage/$id" --tokens 'felhom-agent@pve!agent' --roles FelhomAgentStore >&2
+  echo "felhom-backup-target-apply: granted FelhomAgentStore on /storage/$id (user + token)" >&2
+  ;;
+*)
+  die "unknown op ($op)"
+  ;;
+esac

configs/felhom-localapi-firewall.example

@@ -1,20 +1,30 @@
-# felhom-agent local API — host firewall narrowing (doc 03 §6, slice 8A)
+# felhom-agent local API — host firewall narrowing (doc 03 §6; R-50 island update 2026-07-25)
 #
-# Defense-in-depth for the per-guest local API (the controller→agent channel on the host
-# bridge). The PER-GUEST BEARER TOKEN is the authorization gate; this firewall rule is an
-# ADDITIONAL layer that limits who can even reach the port. The slice-8A spike found no rule
-# was needed for reachability on the demo (PVE firewall off) — this narrows exposure so that
-# only guests on the bridge subnet (not arbitrary LAN hosts) can open a connection.
+# Defense-in-depth for the per-guest local API (the controller→agent channel). The PER-GUEST BEARER
+# TOKEN + the served-leaf pin are the authorization gate; a firewall rule is only an ADDITIONAL layer
+# limiting who can even open the port.
 #
-# The agent already binds the listener to the host BRIDGE IP (local_api.listen_addr), not
-# 0.0.0.0. This file adds the subnet restriction. Apply it at HOST SETUP (it is a host-level
-# packet-filter change, intentionally OUTSIDE the agent's 3-exception privileged fence — the
-# agent never mutates the host firewall at runtime).
+# ===  R-50 ISLAND INSTALL (the default on a fresh appliance)  =================================
+# The agent binds local_api.listen_addr on the HOST-INTERNAL island bridge — 169.254.253.1:8443 on
+# vmbr9, a bridge with NO physical port (bridge-ports none). That bind is the security win:
+#   * Nothing listens on the LAN IP at all, so no LAN host (or off-site attacker on the LAN) can
+#     reach the local API — the LAN:8443 surface is CLOSED by the bind, not by a rule.
+#   * vmbr9 has no uplink, so 169.254.253.1:8443 is reachable ONLY from the one guest wired to the
+#     /30 (169.254.253.2) — the controller. The portless bridge is the isolation.
+# So on an island install NO firewall rule is required for exposure; the topology provides it. If you
+# want belt-and-suspenders, restrict the port to the island bridge (it changes nothing, since nothing
+# off-bridge can route to a portless bridge anyway):
 #
-# Replace the bridge IP (192.168.0.162), port (8443), and the guest bridge subnet
-# (192.168.0.0/24) with this host's values.
+#   nft add rule inet filter input iifname != "vmbr9" ip daddr 169.254.253.1 tcp dport 8443 drop
+#
+# Verify: from the guest, a TLS connect to 169.254.253.1:8443 succeeds; there is no LAN listener to
+# probe (`ss -lnt 'sport = :8443'` shows only the island IP).
+#
+# ===  LEGACY LAN BIND (byo, --no-island, or an explicit --bridge-ip)  =========================
+# When the agent still binds a LAN bridge IP (e.g. 192.168.0.162:8443), the port is exposed to the
+# whole LAN and the subnet-narrowing rule below is worth applying. Replace the bridge IP, port, and
+# the guest bridge subnet with this host's values.
 #
-# ---------------------------------------------------------------------------------------------
 # Option A — nftables (recommended on PVE 8/9; inet filter table). Insert ABOVE any accept:
 #
 #   nft add rule inet filter input ip daddr 192.168.0.162 tcp dport 8443 \
@@ -22,13 +32,11 @@
 #   nft add rule inet filter input ip daddr 192.168.0.162 tcp dport 8443 \
 #       ip saddr 192.168.0.0/24 accept
 #
-# ---------------------------------------------------------------------------------------------
 # Option B — iptables:
 #
 #   iptables -A INPUT -d 192.168.0.162 -p tcp --dport 8443 -s 192.168.0.0/24 -j ACCEPT
 #   iptables -A INPUT -d 192.168.0.162 -p tcp --dport 8443 -j DROP
 #
-# ---------------------------------------------------------------------------------------------
 # Option C — PVE host firewall (/etc/pve/nodes/<node>/host.fw), if the PVE firewall is enabled.
 # Add under [RULES] (and ensure the firewall is enabled in cluster.fw / host.fw):
 #
@@ -36,5 +44,6 @@
 #   IN ACCEPT -source 192.168.0.0/24 -dport 8443 -proto tcp -log nolog
 #   IN DROP   -dport 8443 -proto tcp -log nolog
 #
-# Verify after applying: from a guest ON the bridge, a TLS connect to <bridge-ip>:8443 succeeds;
-# from an OFF-bridge host it is refused/dropped. (The token + leaf-pin still gate the request.)
+# Apply at HOST SETUP — a host-level packet-filter change, intentionally OUTSIDE the agent's
+# 3-exception privileged fence (the agent never mutates the host firewall at runtime). The token +
+# leaf-pin still gate the request regardless of which bind is in force.

configs/felhom-os-apply

@@ -0,0 +1,593 @@
+#!/usr/bin/python3
+# felhom-os-apply — the ROOT half of the agent's operating-system update leg (`11-os-updates.md` §5.4.1, §8.1–8.2).
+#
+# Install as /usr/local/sbin/felhom-os-apply (0755 root:root). The non-root agent invokes it via `sudo -n`
+# (FELHOM_OSAPPLY alias) with EXACTLY:   felhom-os-apply --plan /var/lib/felhom-agent/os/plan-<id>.json
+# Nothing else on the command line is accepted. Python 3, standard library only (a JSON plan cannot be parsed
+# safely in sh). Tests: configs/test_felhom_os_apply.py (a fake runner; nothing real is executed).
+#
+# THE TRUST MODEL. The plan is written by the agent, so a broken-into agent writes whatever plan it likes. The
+# protection is therefore what this file REFUSES, not where the plan came from: no removal, no downgrade, no new
+# package, no package outside the plan, only Debian origin in the fast lane, no kernel / boot package on the host,
+# only the box's own customer guest, and the host layer only on a box whose ROOT-OWNED install record says
+# "appliance" (a BYO host belongs to its owner, `11` §1). Package signatures stay Debian's: apt checks every Release
+# file, including the snapshot.debian.org fallback (decision 79). Nothing here is overridable from the environment.
+#
+# LAYERS (agent v0.141.0): "guest" (the customer LXC, entered with `pct exec`) and "host" (this Proxmox host, run
+# directly). LANE: "fast" only — the slow lane (kernel, Proxmox, Docker) is REFUSED (R3, R14) until `11` §8 steps 5–6.
+#
+# Modes (plan field "mode"):
+#   inventory   `apt-get update`, then report what is installed (with origin), what is pending, and health.
+#   apply       repair first, pick the packages (select "listed": the plan's name=version list; "pending-fast": every
+#               pending Debian / Debian-Security upgrade, for ring 0), check every refusal on an `apt-get -s`
+#               simulation of EXACTLY name=version, install, clean, scan for restart-needed, report as inventory.
+#   health      report health only (the agent polls it after a run).
+# Output: log lines on stderr and the journal (tag felhom-os-apply); the LAST stdout line is
+#   OSAPPLY-REPORT <one JSON object>
+# which is what the agent parses. Exit 0 = done; 2 = refused (nothing changed); 3 = failed during install.
+#
+# SPEED (R-845, agent v0.141.0). Every `pct exec` costs ~0.9 s (measured on demo-hp), and v0.140.0 made one per
+# package for version comparisons — 272 packages ≈ 4 minutes. Versions are now compared with the HOST's dpkg (the same
+# Debian algorithm), madison/policy run once per pass for all packages, the restart scan runs only after an install,
+# and one wrapper call does the whole pass (no separate inventory call before an apply).
+import json
+import os
+import re
+import stat
+import subprocess
+import sys
+import time
+
+PLAN_DIR = "/var/lib/felhom-agent/os"
+PLAN_RE = re.compile(r"^plan-[A-Za-z0-9._-]{1,80}\.json$")
+AGENT_USER = "felhom-agent"
+FAST_ORIGINS = ("Debian", "Debian-Security")
+# Debian package name and version grammar (Debian policy §5.6.1, §5.6.12).
+NAME_RE = re.compile(r"^[a-z0-9][a-z0-9+.-]+$")
+VERSION_RE = re.compile(r"^(?:[0-9]+:)?[0-9][A-Za-z0-9.+~-]*$")
+SNAP_RE = re.compile(r"^[0-9]{8}T[0-9]{6}Z$")
+RESERVED_VMIDS = set(range(990000, 990010)) | {9999}
+DRIVES_PARENT = "/mnt/felhom-drives"
+SNAPSHOT_LIST = "/etc/apt/sources.list.d/felhom-os-snapshot.list"
+APT_ENV = ["env", "DEBIAN_FRONTEND=noninteractive", "APT_LISTCHANGES_FRONTEND=none", "NEEDRESTART_MODE=l", "LC_ALL=C"]
+DPKG_OPTS = ["-o", "Dpkg::Options::=--force-confold", "-o", "Dpkg::Options::=--force-confdef"]
+MIN_FREE = 500 * 1024 * 1024
+# The installer's ROOT-OWNED record (felhom-host-install.sh `state_set mode`); the agent cannot write it.
+INSTALL_STATE = "/var/lib/felhom-install/state.json"
+# Kernel, boot and firmware packages are the SLOW lane on the host whatever their origin (`11` C3, §5.2): a host
+# reboot is needed for them to take effect, and a bad one can stop the box from booting.
+HOST_SLOW_RE = re.compile(r"^(linux-(image|headers|kbuild|modules|base)|proxmox-kernel|proxmox-default-kernel|pve-kernel|"
+                          r"pve-firmware|firmware-|grub|shim|systemd-boot|intel-microcode|amd64-microcode|efibootmgr)")
+# restart_needed() leaves out processes whose cgroup line matches (grep basic regex). Host: the LXC guests' own
+# processes (`0::/lxc/<vmid>/...`) -- NOT lxc-start itself, whose cgroup is `0::/lxc.monitor/<vmid>` (measured
+# 2026-10-04 on demo-felhom: the old pattern "lxc" hid lxc-start with 20 deleted maps, so "reboot needed" stayed false
+# after a libc6 update). Pinned by test_restart_skip_patterns_against_real_cgroups.
+RESTART_SKIP_CGROUP = {"guest": "docker", "host": ":/lxc/"}
+HOST_SERVICES = ["pveproxy", "pvedaemon", "pvestatd", "pve-cluster", "felhom-agent"]
+
+
+class Refused(Exception):
+    def __init__(self, code, reason):
+        super().__init__(f"{code} {reason}")
+        self.code, self.reason = code, reason
+
+
+class Runner:
+    """Runs commands for real. Tests replace it with a fake. `guest` runs inside the container via pct exec."""
+
+    def host(self, argv, timeout=600, stdin=None):
+        p = subprocess.run(argv, capture_output=True, text=True, timeout=timeout, input=stdin)
+        return p.returncode, p.stdout, p.stderr
+
+    def guest(self, vmid, argv, timeout=1800):
+        return self.host(["/usr/sbin/pct", "exec", str(vmid), "--"] + argv, timeout)
+
+    def read_file(self, path):
+        with open(path) as f:
+            return f.read()
+
+    def stat(self, path):
+        return os.lstat(path)
+
+    def agent_uid(self):
+        import pwd
+        return pwd.getpwnam(AGENT_USER).pw_uid
+
+    def write_file(self, layer, vmid, path, body):
+        """Write a small text file in the target layer — never via a shell string."""
+        if layer == "host":
+            with open(path, "w") as f:
+                f.write(body)
+            return
+        rc, _, _ = self.host(["/usr/sbin/pct", "exec", str(vmid), "--", "tee", path], 60, stdin=body)
+        if rc != 0:
+            raise Refused("R7", f"could not write {path} in the guest")
+
+    def log(self, line):
+        print(line, file=sys.stderr, flush=True)
+        try:
+            subprocess.run(["logger", "-t", "felhom-os-apply", line], timeout=10)
+        except Exception:
+            pass
+
+
+class Apply:
+    def __init__(self, runner, plan_path):
+        self.r = runner
+        self.plan_path = plan_path
+        self.report = {"refused": None, "mode": None}
+
+    # ---------- checks ----------
+    def load_plan(self):
+        p = self.plan_path
+        d, base = os.path.dirname(p), os.path.basename(p)
+        if d != PLAN_DIR or not PLAN_RE.match(base) or ".." in p:
+            raise Refused("R1", f"the plan must be {PLAN_DIR}/plan-<id>.json, got {p!r}")
+        try:
+            st = self.r.stat(p)
+        except OSError as e:
+            raise Refused("R1", f"cannot stat the plan: {e}")
+        if not stat.S_ISREG(st.st_mode):
+            raise Refused("R1", "the plan is not a regular file (a symlink or a device is refused)")
+        if st.st_uid != self.r.agent_uid():
+            raise Refused("R1", f"the plan is not owned by {AGENT_USER}")
+        if st.st_size > 2 * 1024 * 1024:
+            raise Refused("R1", "the plan is larger than 2 MB")
+        try:
+            plan = json.loads(self.r.read_file(p))
+        except (OSError, ValueError) as e:
+            raise Refused("R1", f"the plan is not valid JSON: {e}")
+        if not isinstance(plan, dict):
+            raise Refused("R1", "the plan is not a JSON object")
+        return plan
+
+    def check_plan(self, plan):
+        mode = plan.get("mode", "apply")
+        if mode not in ("apply", "inventory", "health"):
+            raise Refused("R11", f"unknown mode {mode!r}")
+        layer = plan.get("layer")
+        if layer not in ("guest", "host"):
+            raise Refused("R12", f"layer {layer!r} is not guest or host")
+        if plan.get("lane", "fast") != "fast":
+            raise Refused("R3", "the slow lane is refused in this release")
+        vmid = plan.get("vmid")
+        if not isinstance(vmid, int) or isinstance(vmid, bool) or vmid <= 0:
+            raise Refused("R11", f"vmid must be a positive integer, got {vmid!r}")
+        rid = plan.get("release_id", "")
+        if not isinstance(rid, str) or not re.match(r"^[A-Za-z0-9._:-]{1,80}$", rid):
+            raise Refused("R11", f"release_id {rid!r} is not a plain id")
+        if plan.get("allow_new"):
+            raise Refused("R6", "allow_new is a slow-lane field; the fast lane never adds a package")
+        select = plan.get("select", "listed")
+        if select not in ("listed", "pending-fast"):
+            raise Refused("R11", f"unknown select {select!r}")
+        pk = plan.get("packages", [])
+        if not isinstance(pk, list):
+            raise Refused("R11", "packages must be a list")
+        if mode == "apply" and select == "listed" and not pk:
+            raise Refused("R11", "packages must be a non-empty list in apply mode (select listed)")
+        if select == "pending-fast" and pk:
+            raise Refused("R11", "select pending-fast takes no package list")
+        seen = set()
+        for e in pk:
+            if not isinstance(e, dict):
+                raise Refused("R11", "every package entry must be an object")
+            n, v, o = e.get("name"), e.get("version"), e.get("origin")
+            if not isinstance(n, str) or not NAME_RE.match(n):
+                raise Refused("R11", f"package name {n!r} is not a Debian package name")
+            if not isinstance(v, str) or not VERSION_RE.match(v):
+                raise Refused("R11", f"version {v!r} of {n} is not a Debian version string")
+            if n in seen:
+                raise Refused("R11", f"package {n} is named twice")
+            seen.add(n)
+            if o not in FAST_ORIGINS:
+                raise Refused("R2", f"{n}: origin {o!r} is not Debian / Debian-Security (the fast lane, `11` C3)")
+            if layer == "host" and HOST_SLOW_RE.match(n):
+                raise Refused("R14", f"{n} is a kernel / boot / firmware package — the host's slow lane")
+        snap = plan.get("snapshot", "")
+        if snap and not SNAP_RE.match(snap):
+            raise Refused("R11", f"snapshot {snap!r} is not YYYYMMDDTHHMMSSZ")
+        return mode, layer, vmid, select
+
+    def check_appliance(self):
+        """R12: the host layer only on a box whose ROOT-OWNED install record says appliance (`11` §1: never BYO)."""
+        try:
+            st = self.r.stat(INSTALL_STATE)
+        except OSError:
+            raise Refused("R12", f"no install record ({INSTALL_STATE}) — this box cannot prove it is an appliance")
+        if st.st_uid != 0 or (st.st_mode & 0o022):
+            raise Refused("R12", f"{INSTALL_STATE} is not root-owned and root-only-writable — it proves nothing")
+        try:
+            mode = json.loads(self.r.read_file(INSTALL_STATE)).get("mode")
+        except (OSError, ValueError, AttributeError):
+            raise Refused("R12", f"{INSTALL_STATE} is unreadable — this box cannot prove it is an appliance")
+        if mode != "appliance":
+            raise Refused("R12", f"this box was installed as {mode!r}, not appliance — its host belongs to its owner")
+
+    def check_guest(self, vmid):
+        if vmid in RESERVED_VMIDS:
+            raise Refused("R10", f"vmid {vmid} is a reserved scratch vmid")
+        try:
+            conf = self.r.read_file(f"/etc/pve/lxc/{vmid}.conf")
+        except OSError:
+            raise Refused("R10", f"vmid {vmid} is not a container on this host")
+        cur = conf.split("\n[", 1)[0]  # the current config, not a snapshot section
+        binds = [l for l in cur.splitlines() if re.match(r"^mp[0-9]+: " + re.escape(DRIVES_PARENT) + r",", l)]
+        if not binds:
+            raise Refused("R10", f"vmid {vmid} does not bind {DRIVES_PARENT} — it is not this box's customer guest")
+        lock = [l for l in cur.splitlines() if l.startswith("lock:")]
+        if lock:
+            raise Refused("R9", f"vmid {vmid} is locked ({lock[0].split(':', 1)[1].strip()}) — a backup or restore is running")
+        rc, out, _ = self.r.host(["/usr/sbin/pct", "status", str(vmid)])
+        if rc != 0 or "running" not in out:
+            raise Refused("R10", f"vmid {vmid} is not running")
+
+    # ---------- target helpers ----------
+    def x(self, argv, timeout=1800):
+        """Run in the TARGET layer: the guest via pct exec, or the host directly."""
+        if self.layer == "host":
+            return self.r.host(argv, timeout)
+        return self.r.guest(self.vmid, argv, timeout)
+
+    def g(self, argv, timeout=1800):
+        """Run in the customer GUEST whatever the layer (its health)."""
+        return self.r.guest(self.vmid, argv, timeout)
+
+    def dpkg_cmp(self, a, op, b):
+        # The HOST's dpkg: the same Debian version algorithm, and no `pct exec` (0.9 s) per comparison (R-845).
+        rc, _, _ = self.r.host(["dpkg", "--compare-versions", a, op, b], 30)
+        return rc == 0
+
+    def installed(self):
+        rc, out, _ = self.x(["dpkg-query", "-W", "-f", "${Package}\t${Version}\t${db:Status-Abbrev}\n"])
+        res = {}
+        for l in out.splitlines():
+            parts = l.split("\t")
+            if len(parts) == 3 and parts[2].startswith("ii"):
+                res[parts[0]] = parts[1]
+        return res
+
+    def madison_all(self, names):
+        """name -> set of downloadable versions, ONE call for all names."""
+        res = {n: set() for n in names}
+        if not names:
+            return res
+        rc, out, _ = self.x(["apt-cache", "madison"] + sorted(names))
+        for l in out.splitlines():
+            f = [x.strip() for x in l.split("|")]
+            if len(f) >= 3 and f[0] in res:
+                res[f[0]].add(f[1])
+        return res
+
+    def simulate(self, args):
+        rc, out, err = self.x(APT_ENV + ["apt-get", "-s", "-q"] + args)
+        inst, remv = [], []
+        for l in out.splitlines():
+            m = re.match(r"^Inst (\S+) (?:\[([^]]*)\] )?\((\S+) (.*?) \[[a-z0-9]+\]\)", l)
+            if m:
+                inst.append({"name": m.group(1), "from": m.group(2), "to": m.group(3), "origin": m.group(4)})
+            m = re.match(r"^Remv (\S+)", l)
+            if m:
+                remv.append(m.group(1))
+        return rc, inst, remv, out + err
+
+    @staticmethod
+    def origin_name(origin):
+        # "Debian:13.7/stable, Debian-Security:13/stable-security" -> {"Debian", "Debian-Security"}
+        return {o.strip().split(":")[0] for o in origin.split(",") if o.strip()}
+
+    def free_bytes(self):
+        rc, out, _ = self.x(["df", "-B1", "--output=avail", "/"])
+        try:
+            return int(out.strip().splitlines()[-1])
+        except (ValueError, IndexError):
+            return -1
+
+    def apt_lock_held(self):
+        rc, out, _ = self.x(["fuser", "/var/lib/dpkg/lock-frontend", "/var/lib/dpkg/lock"])
+        return rc == 0 and out.strip() != ""
+
+    def guest_health(self):
+        """The guest's signals: every container's state + health, the controller's own health, the network."""
+        rc, out, _ = self.g(["docker", "ps", "-a", "--format", "{{.Names}}\t{{.State}}\t{{.Status}}"], timeout=60)
+        cont = {}
+        for l in out.splitlines():
+            p = l.split("\t")
+            if len(p) == 3:
+                h = "healthy" if "(healthy)" in p[2] else "unhealthy" if "(unhealthy)" in p[2] else \
+                    "starting" if "(health: starting)" in p[2] else "none"
+                cont[p[0]] = {"state": p[1], "health": h}
+        nrc, _, _ = self.g(["getent", "hosts", "deb.debian.org"], timeout=30)
+        return {"docker_ok": rc == 0, "containers": cont,
+                "controller": cont.get("felhom-controller", {}).get("health", "absent"),
+                "network_ok": nrc == 0}
+
+    def health(self):
+        if self.layer == "guest":
+            return self.guest_health()
+        rc, out, _ = self.r.host(["systemctl", "is-active"] + HOST_SERVICES, 30)
+        states = out.split()
+        svc = {s: (states[i] if i < len(states) else "unknown") for i, s in enumerate(HOST_SERVICES)}
+        src, sout, _ = self.r.host(["/usr/sbin/pct", "status", str(self.vmid)], 30)
+        running = src == 0 and "running" in sout
+        return {"host_services": svc, "guest_running": running, "guest": self.guest_health() if running else None}
+
+    def restart_needed(self):
+        """Processes still mapping deleted files, OUTSIDE containers (C11). Guest: outside docker; host: outside the
+        LXC guests (the host's /proc shows guest processes too)."""
+        skip = RESTART_SKIP_CGROUP[self.layer]
+        script = ('for p in /proc/[0-9]*; do grep -q "(deleted)" $p/maps 2>/dev/null || continue; '
+                  'grep -q "%s" $p/cgroup 2>/dev/null && continue; echo "${p#/proc/} $(cat $p/comm 2>/dev/null)"; done' % skip)
+        rc, out, _ = self.x(["sh", "-c", script], timeout=120)
+        lines = [l for l in out.splitlines() if " " in l]
+        procs = sorted({l.split(" ", 1)[1] for l in lines})
+        pid1 = any(l.split(" ", 1)[0] == "1" for l in lines)
+        return procs, pid1 or "lxc-start" in procs
+
+    def inventory(self, inst=None):
+        inst = inst if inst is not None else self.installed()
+        names = sorted(inst)
+        origins = {}
+        if names:
+            rc, out, _ = self.x(["apt-cache", "policy"] + names)  # ONE call (R-845)
+            cur, star = None, False
+            for l in out.splitlines():
+                if not l.startswith(" "):
+                    cur, star = l.rstrip(":"), False
+                    continue
+                s = l.strip()
+                if s.startswith("*** "):
+                    star = True
+                    continue
+                if star and cur and re.match(r"^[0-9-]+ ", s):
+                    if "/var/lib/dpkg/status" in s:
+                        origins.setdefault(cur, "local")
+                    else:
+                        origins[cur] = s
+                    continue
+                if star and not re.match(r"^[0-9-]+ ", s):
+                    star = False
+
+        def oname(src):
+            if src in (None, "local"):
+                return "unknown"
+            if "proxmox" in src:
+                return "Proxmox"
+            if "security" in src and "debian" in src:
+                return "Debian-Security"
+            if "docker.com" in src:
+                return "Docker"
+            if "debian" in src:
+                return "Debian"
+            return "other"
+        rc, pend, remv, _ = self.simulate(["dist-upgrade"])
+        self._pending = pend
+        return {
+            "installed": [{"name": n, "version": inst[n], "origin": oname(origins.get(n))} for n in names],
+            "pending": [{"name": p["name"], "from": p["from"], "to": p["to"],
+                         "origin": sorted(self.origin_name(p["origin"]))} for p in pend],
+        }
+
+    # ---------- the run ----------
+    def run(self):
+        plan = self.load_plan()
+        self.mode, self.layer, self.vmid, self.select = self.check_plan(plan)
+        self.report.update(mode=self.mode, layer=self.layer, release_id=plan.get("release_id"), vmid=self.vmid)
+        if self.layer == "host":
+            self.check_appliance()
+        self.check_guest(self.vmid)
+        log = self.r.log
+        if self.mode == "health":
+            self.report["health"] = self.health()
+            return 0
+        log(f"os-apply: START release={plan.get('release_id')} layer={self.layer}" +
+            (f":{self.vmid}" if self.layer == "guest" else "") +
+            f" lane=fast mode={self.mode} select={self.select} packages={len(plan.get('packages', []))}")
+        if self.apt_lock_held():
+            raise Refused("R9", f"another apt/dpkg holds the lock on the {self.layer}")
+        self.report["health_before"] = self.health()
+        if self.mode == "apply":
+            self.repair()
+        rc, out, err = self.x(APT_ENV + ["apt-get", "-q", "update"], timeout=600)
+        if rc != 0:
+            raise Refused("R7", f"apt-get update failed on the {self.layer}: {(out + err).strip().splitlines()[-1:]}")
+        installed_after = None
+        if self.mode == "apply":
+            rc, installed_after = self.apply(plan)
+            if rc:
+                return rc
+        self.report.update(self.inventory(installed_after))
+        if self.layer == "host" and "reboot_needed" not in self.report:
+            # The host is scanned on EVERY pass (local, no pct exec): a reboot must CLEAR "reboot needed", or the hub's
+            # 14-day alarm fires on a host that was rebooted long ago. The guest still scans only after an install
+            # (R-845; one pct exec, and no alarm reads it). Pinned by test_host_scans_every_pass_guest_only_after_install.
+            self.report["restart_needed"], self.report["reboot_needed"] = self.restart_needed()
+        self.report["reboot_scanned"] = "reboot_needed" in self.report
+        self.report["health_after"] = self.health()
+        return 0
+
+    def repair(self):
+        rc, before, _ = self.x(["dpkg", "--audit"])
+        configured = len([l for l in before.splitlines() if l.startswith(" ")])
+        fixed = 0
+        after = ""
+        if before.strip():  # nothing half-done → nothing to run (R-845: two calls saved on every clean pass)
+            self.x(APT_ENV + ["dpkg", "--configure", "-a", "--force-confold"])
+            rc2, out, err = self.x(APT_ENV + ["apt-get", "-f", "install", "-y", "-q"] + DPKG_OPTS)
+            _, after, _ = self.x(["dpkg", "--audit"])
+            fixed = len(re.findall(r"^Setting up ", out, re.M))
+        self.report["repair"] = {"half_configured_before": configured, "fixed": fixed, "clean_after": after.strip() == ""}
+        self.r.log(f"os-apply: REPAIR configured={configured} fixed={fixed}")
+        if after.strip():
+            raise Refused("R13", "dpkg is still broken after the repair: " + after.strip().splitlines()[0])
+
+    def pending_fast(self):
+        """Ring 0 (select pending-fast): every pending upgrade of an INSTALLED package whose every origin is Debian /
+        Debian-Security — and, on the host, not a kernel / boot / firmware package."""
+        rc, pend, remv, _ = self.simulate(["dist-upgrade"])
+        out = []
+        for p in pend:
+            o = self.origin_name(p["origin"])
+            if p["from"] is None or not o or not o <= set(FAST_ORIGINS):
+                continue
+            if self.layer == "host" and HOST_SLOW_RE.match(p["name"]):
+                continue
+            out.append({"name": p["name"], "version": p["to"], "origin": "Debian-Security" if "Debian-Security" in o else "Debian"})
+        return out
+
+    def apply(self, plan):
+        log = self.r.log
+        packages = plan["packages"] if self.select == "listed" else self.pending_fast()
+        inst = self.installed()
+        upgrade, already, notinst = [], 0, 0
+        for e in packages:
+            n, v = e["name"], e["version"]
+            if n not in inst:
+                notinst += 1
+                continue
+            if not self.dpkg_cmp(v, "gt", inst[n]):
+                already += 1
+                continue
+            upgrade.append((n, v))
+        from_snap = 0
+        if upgrade:
+            avail = self.madison_all([n for n, _ in upgrade])
+            missing = [(n, v) for n, v in upgrade if v not in avail[n]]
+        else:
+            missing = []
+        if missing:
+            snap = plan.get("snapshot", "")
+            if not snap:
+                raise Refused("R7", f"{missing[0][0]}={missing[0][1]} is not downloadable and the plan names no snapshot")
+            self.add_snapshot_sources(snap)
+            avail = self.madison_all([n for n, _ in missing])
+            still = [(n, v) for n, v in missing if v not in avail[n]]
+            if still:
+                self.remove_snapshot_sources()
+                raise Refused("R7", f"{still[0][0]}={still[0][1]} is not downloadable, not even from snapshot {snap}")
+            from_snap = len(missing)
+        try:
+            log(f"os-apply: PLAN upgrade={len(upgrade)} already={already} not-installed={notinst} from-snapshot={from_snap}")
+            self.report["plan"] = {"upgrade": len(upgrade), "already": already, "not_installed": notinst, "from_snapshot": from_snap}
+            if not upgrade:
+                self.report["upgraded"] = []
+                log("os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)")
+                return 0, inst
+            args = ["install", "--only-upgrade", "--no-install-recommends"] + [f"{n}={v}" for n, v in upgrade]
+            rc, sim, remv, text = self.simulate(args)
+            if rc != 0:
+                tail = text.strip().splitlines()[-1] if text.strip() else ""
+                raise Refused("R7", "the simulation failed: " + tail)
+            if remv:
+                raise Refused("R4", f"the plan would remove {', '.join(remv[:5])}")
+            want = dict(upgrade)
+            for p in sim:
+                if p["from"] is None:
+                    raise Refused("R6", f"the plan would add a package that is not installed: {p['name']}")
+                if p["name"] not in want:
+                    raise Refused("R6", f"the plan would touch {p['name']}, which is not in the plan")
+                if p["to"] != want[p["name"]]:
+                    raise Refused("R6", f"{p['name']} would go to {p['to']}, not the approved {want[p['name']]}")
+                if not self.dpkg_cmp(p["to"], "gt", p["from"]):
+                    raise Refused("R5", f"{p['name']} would be downgraded {p['from']} -> {p['to']}")
+                if not self.origin_name(p["origin"]) & set(FAST_ORIGINS):
+                    raise Refused("R2", f"{p['name']} would come from {p['origin']}, not Debian")
+                if self.layer == "host" and HOST_SLOW_RE.match(p["name"]):
+                    raise Refused("R14", f"{p['name']} is a kernel / boot / firmware package — the host's slow lane")
+            need = self.download_bytes(args)
+            free = self.free_bytes()
+            if free >= 0 and free < max(MIN_FREE, 3 * need):
+                raise Refused("R8", f"free space {free} B is below max(500 MB, 3 x download {need} B)")
+            t0 = time.time()
+            rc, out, err = self.x(APT_ENV + ["apt-get", "-y", "-q"] + DPKG_OPTS + args)
+            secs = time.time() - t0
+            # dpkg says "Installing new version of config file X" when X was NOT changed locally (the package's new
+            # version is taken), and "Configuration file 'X'" + "Keeping old config file" when it was (--force-confold
+            # keeps the local one; the package's version lands as X.dpkg-dist). Measured live 2026-10-04 (debian_version).
+            conflict = None
+            for l in (out + err).splitlines():
+                m = re.search(r"Installing new version of config file (\S+?)\s*\.\.\.", l)
+                if m:
+                    log(f"os-apply: CONFFILE updated {m.group(1)} (it was not changed locally)")
+                m = re.search(r"Configuration file '([^']+)'", l)
+                if m:
+                    conflict = m.group(1)
+                if conflict and "Keeping old config file" in l:
+                    log(f"os-apply: CONFFILE kept {conflict} (changed locally; the package's version is {conflict}.dpkg-dist)")
+                    self.report.setdefault("conffiles_kept", []).append(conflict)
+                    conflict = None
+            self.x(["apt-get", "clean"])
+            if rc != 0:
+                _, aud, _ = self.x(["dpkg", "--audit"])
+                first = aud.strip().splitlines()[0] if aud.strip() else "clean"
+                log(f"os-apply: FAILED rc={rc} step=install — dpkg state: {first}")
+                self.report["failed"] = {"rc": rc, "dpkg_audit": first, "tail": (out + err).strip().splitlines()[-3:]}
+                return 3, None
+            self.report["upgraded"] = [{"name": n, "version": v} for n, v in upgrade]
+            self.report["seconds"] = round(secs, 1)
+            procs, reboot = self.restart_needed()  # only after an install (R-845)
+            self.report["restart_needed"] = procs
+            self.report["docker_restart_needed"] = any(p in ("dockerd", "containerd") for p in procs)
+            self.report["reboot_needed"] = reboot
+            log(f"os-apply: DONE rc=0 seconds={secs:.1f} upgraded={len(upgrade)} restart-needed={','.join(procs) or '-'} reboot-needed={'yes' if reboot else 'no'}")
+            return 0, None
+        finally:
+            if from_snap:
+                self.remove_snapshot_sources()
+
+    def download_bytes(self, args):
+        rc, out, _ = self.x(APT_ENV + ["apt-get", "-s", "-o", "Debug::NoLocking=1", "--print-uris", "-q"] + args)
+        total = 0
+        for l in out.splitlines():
+            m = re.match(r"^'[^']+' \S+ ([0-9]+) ", l)
+            if m:
+                total += int(m.group(1))
+        return total
+
+    def add_snapshot_sources(self, snap):
+        rc, out, _ = self.x(["sh", "-c", ". /etc/os-release && echo $VERSION_CODENAME"])
+        code = out.strip()
+        if not re.match(r"^[a-z]+$", code):
+            raise Refused("R7", f"cannot read the {self.layer}'s Debian codename ({code!r})")
+        body = (f"deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/{snap} {code} main\n"
+                f"deb [check-valid-until=no] http://snapshot.debian.org/archive/debian-security/{snap} {code}-security main\n")
+        self.r.write_file(self.layer, self.vmid, SNAPSHOT_LIST, body)
+        self.r.log(f"os-apply: SNAPSHOT using snapshot.debian.org/{snap} for versions no longer published (decision 79)")
+        rc, out, err = self.x(APT_ENV + ["apt-get", "-q", "update"], timeout=600)
+        if rc != 0:
+            self.remove_snapshot_sources()
+            raise Refused("R7", "apt-get update against snapshot.debian.org failed")
+
+    def remove_snapshot_sources(self):
+        self.x(["rm", "-f", SNAPSHOT_LIST])
+        self.x(APT_ENV + ["apt-get", "-q", "update"], timeout=600)
+
+
+def main(argv, runner=None):
+    r = runner or Runner()
+    if len(argv) != 3 or argv[1] != "--plan":
+        r.log("os-apply: REFUSED: R1 usage: felhom-os-apply --plan /var/lib/felhom-agent/os/plan-<id>.json")
+        print("OSAPPLY-REPORT " + json.dumps({"refused": {"code": "R1", "reason": "usage"}}))
+        return 2
+    a = Apply(r, argv[2])
+    t0 = time.time()
+    try:
+        rc = a.run()
+    except Refused as e:
+        r.log(f"os-apply: REFUSED: {e.code} {e.reason}")
+        a.report["refused"] = {"code": e.code, "reason": e.reason}
+        rc = 2
+    except subprocess.TimeoutExpired as e:
+        r.log(f"os-apply: FAILED rc=124 step=timeout — {e.cmd}")
+        a.report["failed"] = {"rc": 124, "timeout": str(e.cmd)[:200]}
+        rc = 3
+    a.report["pass_seconds"] = round(time.time() - t0, 1)
+    print("OSAPPLY-REPORT " + json.dumps(a.report, sort_keys=True))
+    return rc
+
+
+if __name__ == "__main__":
+    if os.geteuid() != 0:
+        print("felhom-os-apply: must run as root (via sudo)", file=sys.stderr)
+        sys.exit(2)
+    sys.exit(main(sys.argv))

configs/test_felhom_os_apply.py

@@ -0,0 +1,628 @@
+#!/usr/bin/env python3
+"""Tests for configs/felhom-os-apply (`11` §5.4.1). A fake runner plays the host and the guest: nothing is executed
+for real except the local `dpkg --compare-versions` (pure, no network). Every refusal R1–R13 has a test; the red-proof
+(each test fails when its rule is removed) is `audits/os-guest-lane-2026-10-04/partB/redproof.txt`.
+
+Run: python3 configs/test_felhom_os_apply.py   (also run by internal/osupdate's Go test)
+"""
+import importlib.machinery
+import importlib.util
+import json
+import os
+import pathlib
+import re
+import stat as statmod
+import subprocess
+import unittest
+
+HERE = pathlib.Path(__file__).resolve().parent
+_loader = importlib.machinery.SourceFileLoader("osapply", os.environ.get("OSAPPLY_UNDER_TEST", str(HERE / "felhom-os-apply")))  # red-proof seam
+_spec = importlib.util.spec_from_loader("osapply", _loader)
+osapply = importlib.util.module_from_spec(_spec)
+_loader.exec_module(osapply)
+
+PLAN = "/var/lib/felhom-agent/os/plan-t1.json"
+CONF_OK = ("arch: amd64\nmp0: local-lvm:vm-9201-disk-1,mp=/var/lib/felhom,backup=1,size=70G\n"
+           "mp8: /mnt/felhom-drives,mp=/mnt/felhom-drives\nrootfs: local-lvm:vm-9201-disk-0,size=32G\n")
+DEB = "Debian:13.7/stable"
+SEC = "Debian-Security:13/stable-security"
+
+
+def dpkg_cmp(a, op, b):
+    return subprocess.run(["dpkg", "--compare-versions", a, op, b]).returncode == 0
+
+
+class St:
+    def __init__(self, mode=statmod.S_IFREG | 0o600, uid=999, size=100):
+        self.st_mode, self.st_uid, self.st_size = mode, uid, size
+
+
+class Fake:
+    """The host + one guest. `installed` / `live` (name -> versions in the live archive) / `snapshot` (versions
+    the snapshot archive adds) / `extra_sim` (lines the simulation adds) / `dpkg_audit` / `free`."""
+
+    def __init__(self):
+        self.plan = {"release_id": "os-t1", "layer": "guest", "lane": "fast", "vmid": 9201, "mode": "apply",
+                     "snapshot": "20261004T080000Z",
+                     "packages": [{"name": "libc6", "version": "2.41-12+deb13u4", "origin": "Debian"},
+                                  {"name": "openssl", "version": "3.5.7-1~deb13u3", "origin": "Debian-Security"}]}
+        self.files = {"/etc/pve/lxc/9201.conf": CONF_OK}
+        self.stats = {PLAN: St()}
+        self.installed = {"libc6": "2.41-12+deb13u3", "openssl": "3.5.6-1~deb13u1", "bash": "5.2.37-2+b9"}
+        self.live = {"libc6": {"2.41-12+deb13u4"}, "openssl": {"3.5.7-1~deb13u3"}}
+        self.snapshot = {}
+        self.snap_active = False
+        self.extra_sim = []
+        self.dpkg_audit = ""
+        self.free = 10 * 1024 ** 3
+        self.install_rc = 0
+        self.calls = []
+        self.logs = []
+        self.written = {}
+        self.status = "status: running"
+        self.lock_held = False
+        self.services = {}
+        self.files[osapply.INSTALL_STATE] = json.dumps({"mode": "appliance"})
+        self.stats[osapply.INSTALL_STATE] = St(mode=statmod.S_IFREG | 0o644, uid=0)
+
+    # Runner interface
+    def read_file(self, p):
+        if p == PLAN:
+            return json.dumps(self.plan)
+        if p not in self.files:
+            raise OSError("no such file")
+        return self.files[p]
+
+    def stat(self, p):
+        if p not in self.stats:
+            raise OSError("no such file")
+        return self.stats[p]
+
+    def agent_uid(self):
+        return 999
+
+    def log(self, line):
+        self.logs.append(line)
+
+    def host(self, argv, timeout=600, stdin=None):
+        self.calls.append(("host", argv))
+        if argv[0] == "/usr/sbin/pct" and argv[1] == "status":
+            return 0, self.status + "\n", ""
+        if argv[0] == "dpkg" and argv[1] == "--compare-versions":
+            return (0 if dpkg_cmp(argv[2], argv[3], argv[4]) else 1), "", ""
+        if argv[0] == "systemctl" and argv[1] == "is-active":
+            return 0, "\n".join(self.services.get(s, "active") for s in argv[2:]) + "\n", ""
+        return self.emulate(argv)
+
+    def write_file(self, layer, vmid, path, body):
+        self.written[path] = body
+        self.write_layer = layer
+        if path == osapply.SNAPSHOT_LIST:
+            self.snap_active = True
+
+    def avail(self, n):
+        v = set(self.live.get(n, set()))
+        if self.snap_active:
+            v |= self.snapshot.get(n, set())
+        return v
+
+    def guest(self, vmid, argv, timeout=1800):
+        self.calls.append(("guest", vmid, argv))
+        return self.emulate(argv)
+
+    def emulate(self, argv):
+        a = [x for x in argv if not re.match(r"^[A-Z_]+=", x) and x != "env"]
+        cmd = a[0]
+        if cmd == "dpkg-query":
+            return 0, "".join(f"{n}\t{v}\tii \n" for n, v in self.installed.items()), ""
+        if cmd == "dpkg" and a[1] == "--compare-versions":
+            return (0 if dpkg_cmp(a[2], a[3], a[4]) else 1), "", ""
+        if cmd == "dpkg" and a[1] == "--audit":
+            return 0, self.dpkg_audit, ""
+        if cmd == "dpkg" and a[1] == "--configure":
+            return 0, "", ""
+        if cmd == "fuser":
+            return (0, " 123", "") if self.lock_held else (1, "", "")
+        if cmd == "apt-cache" and a[1] == "madison":
+            self.madison_calls = getattr(self, "madison_calls", 0) + 1
+            return 0, "".join(f" {n} | {v} | http://deb.debian.org trixie/main amd64 Packages\n" for n in a[2:] for v in self.avail(n)), ""
+        if cmd == "apt-cache" and a[1] == "policy":
+            out = ""
+            for n in a[2:]:
+                out += f"{n}:\n  Installed: {self.installed.get(n)}\n  Version table:\n *** {self.installed.get(n)} 500\n        500 http://deb.debian.org/debian trixie/main amd64 Packages\n"
+            return 0, out, ""
+        if cmd == "apt-get":
+            if "update" in a:
+                return 0, "", ""
+            if "clean" in a:
+                return 0, "", ""
+            if "-f" in a:
+                self.dpkg_audit = ""
+                return 0, "Setting up x (1) ...\n" if getattr(self, "repaired", False) else "", ""
+            if "-s" in a:
+                return self.sim(a)
+            if "install" in a:
+                if self.install_rc:
+                    return self.install_rc, "", "E: boom"
+                for x in a:
+                    if "=" in x and not x.startswith("-") and "::" not in x:
+                        n, v = x.split("=", 1)
+                        self.installed[n] = v
+                return 0, getattr(self, "install_out", "Setting up libc6 ...\n"), ""
+        if cmd == "df":
+            return 0, f"Avail\n{self.free}\n", ""
+        if cmd == "docker":
+            return 0, "felhom-controller\trunning\tUp 1 hour (healthy)\napp\trunning\tUp 1 hour (healthy)\n", ""
+        if cmd == "getent":
+            return 0, "1.2.3.4 deb.debian.org\n", ""
+        if cmd == "sh":
+            if "os-release" in a[2]:
+                return 0, "trixie\n", ""
+            if "(deleted)" in a[2]:
+                return 0, getattr(self, "restart_out", ""), ""
+            return 0, "", ""
+        if cmd == "rm":
+            self.snap_active = False
+            return 0, "", ""
+        return 1, "", f"unexpected guest call {a}"
+
+    def sim(self, a):
+        if "--print-uris" in a:
+            return 0, "'http://x/libc6.deb' libc6.deb 4000000 SHA256:x\n", ""
+        if "dist-upgrade" in a:
+            if getattr(self, "pending_sim", None) is not None and not getattr(self, "_pending_used", False):
+                self._pending_used = True
+                return 0, "\n".join(self.pending_sim) + "\n", ""
+            return 0, "Inst bash [5.2.37-2+b9] (5.2.37-2+b10 Debian:13.7/stable [amd64])\n", ""
+        out = ""
+        for x in a:
+            if "=" in x and not x.startswith("-") and "::" not in x:
+                n, v = x.split("=", 1)
+                if v not in self.avail(n):
+                    return 100, "", f"E: Version '{v}' for '{n}' was not found"
+                origin = SEC if n == "openssl" else DEB
+                out += f"Inst {n} [{self.installed[n]}] ({v} {origin} [amd64])\n"
+        out += "".join(l + "\n" for l in self.extra_sim)
+        return 0, out, ""
+
+
+def run(f):
+    import io
+    import contextlib
+    buf = io.StringIO()
+    with contextlib.redirect_stdout(buf):
+        rc = osapply.main(["felhom-os-apply", "--plan", PLAN], runner=f)
+    line = [l for l in buf.getvalue().splitlines() if l.startswith("OSAPPLY-REPORT ")][-1]
+    return rc, json.loads(line[len("OSAPPLY-REPORT "):])
+
+
+class Happy(unittest.TestCase):
+    def test_apply_installs_exactly_the_plan(self):
+        f = Fake()
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        self.assertEqual(f.installed["libc6"], "2.41-12+deb13u4")
+        self.assertEqual(f.installed["openssl"], "3.5.7-1~deb13u3")
+        self.assertEqual(f.installed["bash"], "5.2.37-2+b9", "a package outside the plan was changed")
+        self.assertEqual(rep["plan"]["upgrade"], 2)
+        self.assertIn("installed", rep)
+        self.assertEqual(rep["pending"][0]["name"], "bash")
+        self.assertTrue(any(l.startswith("os-apply: REPAIR ") for l in f.logs), "the repair line must always print")
+        self.assertTrue(any(l.startswith("os-apply: DONE rc=0") for l in f.logs))
+        inst = [c for c in f.calls if c[0] == "guest" and "install" in c[2] and "-s" not in c[2] and "-f" not in c[2]]
+        self.assertTrue(inst and "Dpkg::Options::=--force-confold" in inst[0][2], "must keep existing config files")
+
+    def test_already_current_is_a_no_op(self):
+        f = Fake()
+        f.installed.update(libc6="2.41-12+deb13u4", openssl="3.5.7-1~deb13u3")
+        rc, rep = run(f)
+        self.assertEqual(rc, 0)
+        self.assertEqual(rep["plan"]["upgrade"], 0)
+
+    def test_inventory_installs_nothing(self):
+        f = Fake()
+        f.plan["mode"] = "inventory"
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        self.assertEqual(f.installed["libc6"], "2.41-12+deb13u3")
+        self.assertIn("installed", rep)
+        self.assertNotIn("restart_needed", rep, "the restart scan runs only after an install (R-845)")
+
+    def test_health_mode(self):
+        f = Fake()
+        f.plan["mode"] = "health"
+        rc, rep = run(f)
+        self.assertEqual(rc, 0)
+        self.assertEqual(rep["health"]["controller"], "healthy")
+
+
+class Repair(unittest.TestCase):
+    def test_repair_runs_first_and_is_reported(self):
+        f = Fake()
+        f.dpkg_audit = "The following packages have been unpacked but not yet configured.\n perl  Larry Wall's\n"
+        f.repaired = True
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        self.assertEqual(rep["repair"]["half_configured_before"], 1)
+        self.assertEqual(rep["repair"]["fixed"], 1)
+        order = [i for i, c in enumerate(f.calls) if c[0] == "guest" and c[2][-1:] != ["update"]]
+        first_cfg = next(i for i, c in enumerate(f.calls) if c[0] == "guest" and "--configure" in c[2])
+        first_upd = next(i for i, c in enumerate(f.calls) if c[0] == "guest" and "update" in c[2])
+        self.assertLess(first_cfg, first_upd, "the repair must run before anything else touches apt")
+        self.assertTrue(order)
+
+
+class Snapshot(unittest.TestCase):
+    def test_a_replaced_version_comes_from_the_snapshot(self):
+        f = Fake()
+        f.live["openssl"] = {"3.5.7-1~deb13u4"}  # Debian moved on
+        f.snapshot["openssl"] = {"3.5.7-1~deb13u3"}
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        self.assertEqual(rep["plan"]["from_snapshot"], 1)
+        self.assertEqual(f.installed["openssl"], "3.5.7-1~deb13u3", "must install the APPROVED version, not the newer one")
+        body = f.written[osapply.SNAPSHOT_LIST]
+        self.assertIn("snapshot.debian.org/archive/debian/20261004T080000Z trixie main", body)
+        self.assertIn("debian-security/20261004T080000Z trixie-security main", body)
+        self.assertFalse(f.snap_active, "the temporary snapshot sources must be removed after the run")
+
+    def test_snapshot_does_not_have_it_either(self):
+        f = Fake()
+        f.live["openssl"] = set()
+        rc, rep = run(f)
+        self.assertEqual((rc, rep["refused"]["code"]), (2, "R7"))
+        self.assertFalse(f.snap_active)
+
+
+class Refusals(unittest.TestCase):
+    def refused(self, f, code):
+        rc, rep = run(f)
+        self.assertEqual(rc, 2, rep)
+        self.assertEqual(rep["refused"]["code"], code, rep)
+        self.assertTrue(any(l.startswith(f"os-apply: REFUSED: {code} ") for l in f.logs), f.logs)
+        inst = [c for c in f.calls if c[0] == "guest" and "install" in c[2] and "-s" not in c[2] and "-f" not in c[2]]
+        self.assertEqual(inst, [], "a refusal must install nothing")
+        return rep
+
+    def test_R1_usage(self):
+        import io
+        import contextlib
+        f = Fake()
+        with contextlib.redirect_stdout(io.StringIO()):
+            self.assertEqual(osapply.main(["felhom-os-apply", "--plan", PLAN, "--extra"], runner=f), 2)
+            self.assertEqual(osapply.main(["felhom-os-apply", "--plan"], runner=f), 2)
+
+    def test_R1_path_outside_the_plan_dir(self):
+        import io
+        import contextlib
+        f = Fake()
+        with contextlib.redirect_stdout(io.StringIO()):
+            rc = osapply.main(["felhom-os-apply", "--plan", "/tmp/plan-x.json"], runner=f)
+        self.assertEqual(rc, 2)
+        self.assertTrue(any("R1" in l for l in f.logs))
+
+    def test_R1_symlink(self):
+        f = Fake()
+        f.stats[PLAN] = St(mode=statmod.S_IFLNK | 0o777)
+        self.refused(f, "R1")
+
+    def test_R1_not_owned_by_the_agent(self):
+        f = Fake()
+        f.stats[PLAN] = St(uid=0)
+        self.refused(f, "R1")
+
+    def test_R2_non_debian_origin_in_the_plan(self):
+        f = Fake()
+        f.plan["packages"][0]["origin"] = "Proxmox"
+        self.refused(f, "R2")
+
+    def test_R2_non_debian_origin_in_the_simulation(self):
+        f = Fake()
+        f.installed["libc6"] = "2.41-12+deb13u3"
+        orig = f.sim
+
+        def sim(a):
+            rc, out, err = orig(a)
+            return rc, out.replace("Debian:13.7/stable", "Proxmox Debian Repository:stable"), err
+        f.sim = sim
+        self.refused(f, "R2")
+
+    def test_R3_slow_lane(self):
+        f = Fake()
+        f.plan["lane"] = "slow"
+        self.refused(f, "R3")
+
+    def test_R4_removal(self):
+        f = Fake()
+        f.extra_sim = ["Remv bash [5.2.37-2+b9]"]
+        self.refused(f, "R4")
+
+    def test_R5_downgrade(self):
+        f = Fake()
+        f.installed["libc6"] = "2.41-12+deb13u4"
+        f.plan["packages"] = [{"name": "openssl", "version": "3.5.7-1~deb13u3", "origin": "Debian-Security"}]
+        f.extra_sim = ["Inst openssl [3.5.6-1~deb13u1] (3.5.5-1 Debian:13.7/stable [amd64])"]
+        orig = f.sim
+
+        def sim(a):  # the simulation answers with a LOWER version than installed
+            rc, out, err = orig(a)
+            return rc, "\n".join(l for l in out.splitlines() if not l.startswith("Inst openssl [3.5.6-1~deb13u1] (3.5.7")) + "\n", err
+        f.sim = sim
+        f.plan["packages"][0]["version"] = "3.5.7-1~deb13u3"
+        rep = run(f)[1]
+        # The plan asks 3.5.7; the simulation goes to 3.5.5: that is BOTH a wrong version (R6) and a downgrade.
+        self.assertIn(rep["refused"]["code"], ("R5", "R6"))
+
+    def test_R5_downgrade_exact(self):
+        f = Fake()
+        f.plan["packages"] = [{"name": "openssl", "version": "3.5.7-1~deb13u3", "origin": "Debian-Security"}]
+        f.installed["openssl"] = "3.5.6-1~deb13u1"
+        orig = f.sim
+
+        def sim(a):
+            rc, out, err = orig(a)
+            return rc, out.replace("[3.5.6-1~deb13u1]", "[3.5.8-1]"), err
+        f.sim = sim
+        self.refused(f, "R5")
+
+    def test_R6_new_package(self):
+        f = Fake()
+        f.extra_sim = ["Inst newthing (1.0 Debian:13.7/stable [amd64])"]
+        self.refused(f, "R6")
+
+    def test_R6_unlisted_package(self):
+        f = Fake()
+        f.extra_sim = ["Inst bash [5.2.37-2+b9] (5.2.37-2+b10 Debian:13.7/stable [amd64])"]
+        self.refused(f, "R6")
+
+    def test_R6_allow_new_is_slow_lane(self):
+        f = Fake()
+        f.plan["allow_new"] = ["proxmox-kernel-x"]
+        self.refused(f, "R6")
+
+    def test_R7_not_downloadable_and_no_snapshot(self):
+        f = Fake()
+        f.live["openssl"] = set()
+        f.plan["snapshot"] = ""
+        self.refused(f, "R7")
+
+    def test_R8_free_space(self):
+        f = Fake()
+        f.free = 100 * 1024 * 1024
+        self.refused(f, "R8")
+
+    def test_R9_guest_locked_by_a_backup(self):
+        f = Fake()
+        f.files["/etc/pve/lxc/9201.conf"] = CONF_OK + "lock: backup\n"
+        self.refused(f, "R9")
+
+    def test_R9_apt_lock_held(self):
+        f = Fake()
+        f.lock_held = True
+        self.refused(f, "R9")
+
+    def test_R10_not_the_boxs_own_guest(self):
+        f = Fake()
+        f.files["/etc/pve/lxc/9201.conf"] = CONF_OK.replace("mp8: /mnt/felhom-drives,", "mp8: /mnt/hdd_1/scratch,")
+        self.refused(f, "R10")
+
+    def test_R10_reserved_vmid(self):
+        f = Fake()
+        f.plan["vmid"] = 990003
+        self.refused(f, "R10")
+
+    def test_R10_bind_only_in_a_snapshot_section(self):
+        f = Fake()
+        f.files["/etc/pve/lxc/9201.conf"] = "rootfs: x\n[snap1]\nmp8: /mnt/felhom-drives,mp=/mnt/felhom-drives\n"
+        self.refused(f, "R10")
+
+    def test_R10_not_running(self):
+        f = Fake()
+        f.status = "status: stopped"
+        self.refused(f, "R10")
+
+    def test_R11_duplicate(self):
+        f = Fake()
+        f.plan["packages"].append(dict(f.plan["packages"][0]))
+        self.refused(f, "R11")
+
+    def test_R11_bad_version_string(self):
+        f = Fake()
+        f.plan["packages"][0]["version"] = "1.0; rm -rf /"
+        self.refused(f, "R11")
+
+    def test_R11_bad_name(self):
+        f = Fake()
+        f.plan["packages"][0]["name"] = "--purge"
+        self.refused(f, "R11")
+
+    def test_R12_unknown_layer(self):
+        f = Fake()
+        f.plan["layer"] = "vm"
+        self.refused(f, "R12")
+
+    def test_R12_host_on_a_byo_box(self):
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.files[osapply.INSTALL_STATE] = json.dumps({"mode": "byo"})
+        self.refused(f, "R12")
+
+    def test_R12_host_without_an_install_record(self):
+        f = Fake()
+        f.plan["layer"] = "host"
+        del f.stats[osapply.INSTALL_STATE]
+        self.refused(f, "R12")
+
+    def test_R12_host_record_not_root_owned(self):
+        # the agent can write agent.json's deployment_mode; only a ROOT-owned record proves anything
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.stats[osapply.INSTALL_STATE] = St(mode=statmod.S_IFREG | 0o644, uid=999)
+        self.refused(f, "R12")
+
+    def test_R14_kernel_package_in_a_host_plan(self):
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.plan["packages"].append({"name": "linux-image-amd64", "version": "6.12.1-1", "origin": "Debian"})
+        self.refused(f, "R14")
+
+    def test_R14_kernel_package_pulled_by_the_simulation(self):
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.extra_sim = ["Inst grub-common [2.12-9] (2.12-10 Debian:13.7/stable [amd64])"]
+        f.installed["grub-common"] = "2.12-9"
+        f.plan["packages"].append({"name": "grub-common", "version": "2.12-10", "origin": "Debian"})
+        self.refused(f, "R14")
+
+    def test_R13_repair_does_not_fix_it(self):
+        f = Fake()
+        f.dpkg_audit = "The following packages are broken\n perl\n"
+        orig = f.guest
+
+        def guest(vmid, argv, timeout=1800):
+            rc, out, err = orig(vmid, argv, timeout)
+            if "-f" in argv:
+                f.dpkg_audit = "The following packages are broken\n perl\n"
+            return rc, out, err
+        f.guest = guest
+        self.refused(f, "R13")
+
+
+class Conffiles(unittest.TestCase):
+    # dpkg's two shapes, measured live 2026-10-04: an unchanged file is UPDATED; a locally changed one is KEPT.
+    def test_updated_vs_kept(self):
+        f = Fake()
+        f.install_out = ("Installing new version of config file /etc/debian_version ...\n"
+                         "Configuration file '/etc/ssh/sshd_config'\n ==> Modified (by you or by a script) since installation.\n"
+                         " ==> Keeping old config file as default.\n")
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        self.assertIn("os-apply: CONFFILE updated /etc/debian_version (it was not changed locally)", f.logs)
+        self.assertTrue(any(l.startswith("os-apply: CONFFILE kept /etc/ssh/sshd_config") for l in f.logs), f.logs)
+        self.assertEqual(rep["conffiles_kept"], ["/etc/ssh/sshd_config"])
+        self.assertFalse(any("kept /etc/debian_version" in l for l in f.logs), "an updated file must not be reported as kept")
+
+
+class HostLayer(unittest.TestCase):
+    def test_host_runs_on_the_host_not_in_the_guest(self):
+        f = Fake()
+        f.plan["layer"] = "host"
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        inst = [c for c in f.calls if c[0] == "host" and "install" in c[1] and "-s" not in c[1] and "-f" not in c[1]]
+        self.assertTrue(inst, "the host install must run on the host")
+        self.assertFalse([c for c in f.calls if c[0] == "guest" and "install" in c[2]], "nothing installed in the guest")
+        self.assertEqual(sorted(rep["health_after"]["host_services"]), sorted(osapply.HOST_SERVICES))
+        self.assertTrue(rep["health_after"]["guest_running"])
+
+    def test_pending_fast_skips_proxmox_docker_and_kernel(self):
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.plan["select"] = "pending-fast"
+        f.plan["packages"] = []
+        f.installed.update({"pve-manager": "9.2.2", "linux-image-amd64": "6.12.1", "docker-ce": "29.7"})
+        f.pending_sim = [
+            "Inst libc6 [2.41-12+deb13u3] (2.41-12+deb13u4 Debian:13.7/stable [amd64])",
+            "Inst openssl [3.5.6-1~deb13u1] (3.5.7-1~deb13u3 Debian:13.7/stable, Debian-Security:13/stable-security [amd64])",
+            "Inst pve-manager [9.2.2] (9.2.21 Proxmox Debian Repository:stable [amd64])",
+            "Inst linux-image-amd64 [6.12.1] (6.12.9 Debian:13.7/stable [amd64])",
+            "Inst docker-ce [29.7] (29.8 Docker CE:trixie [amd64])",
+            "Inst brand-new (1.0 Debian:13.7/stable [amd64])",
+        ]
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        got = sorted(u["name"] for u in rep["upgraded"])
+        self.assertEqual(got, ["libc6", "openssl"], "pending-fast must take only installed, Debian-origin, non-kernel packages")
+
+    def test_reboot_needed_when_pid1_or_lxc_start(self):
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.restart_out = "1 systemd\n2101 lxc-start\n530 sshd\n"
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        self.assertTrue(rep["reboot_needed"])
+        self.assertIn("lxc-start", rep["restart_needed"])
+
+    def test_reboot_needed_for_lxc_start_alone(self):
+        # lxc-start runs the guest; only a guest restart (or a host reboot) replaces it
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.restart_out = "2101 lxc-start\n530 sshd\n"
+        rc, rep = run(f)
+        self.assertTrue(rep["reboot_needed"], rep)
+
+    def test_host_scans_every_pass_guest_only_after_install(self):
+        # A host pass that installs nothing still scans, so a reboot clears the flag (the hub alarm reads it).
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.plan["mode"] = "inventory"
+        f.restart_out = "2101 lxc-start\n"
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        self.assertTrue(rep["reboot_scanned"], rep)
+        self.assertTrue(rep["reboot_needed"], rep)
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.plan["mode"] = "inventory"
+        f.restart_out = ""  # after the reboot: nothing maps a deleted file
+        rc, rep = run(f)
+        self.assertTrue(rep["reboot_scanned"] and rep["reboot_needed"] is False, rep)
+        f = Fake()
+        f.plan["mode"] = "inventory"
+        rc, rep = run(f)
+        self.assertFalse(rep["reboot_scanned"], "the guest scans only after an install (R-845)")
+
+    def test_no_reboot_for_ordinary_daemons(self):
+        f = Fake()
+        f.plan["layer"] = "host"
+        f.restart_out = "530 sshd\n611 cron\n"
+        rc, rep = run(f)
+        self.assertFalse(rep["reboot_needed"], rep)
+
+
+class Speed(unittest.TestCase):
+    # R-845: no `pct exec` per package — version checks on the host, madison once, the restart scan only after an install.
+    def test_no_per_package_guest_calls(self):
+        f = Fake()
+        for i in range(40):
+            f.installed[f"pkg{i}"] = "1.0-1"
+            f.live[f"pkg{i}"] = {"1.0-2"}
+            f.plan["packages"].append({"name": f"pkg{i}", "version": "1.0-2", "origin": "Debian"})
+        rc, rep = run(f)
+        self.assertEqual(rc, 0, rep)
+        guest_cmp = [c for c in f.calls if c[0] == "guest" and "--compare-versions" in c[2]]
+        self.assertEqual(guest_cmp, [], "version comparisons must run on the host")
+        self.assertEqual(f.madison_calls, 1, "madison must run once for all packages")
+        guest_calls = len([c for c in f.calls if c[0] == "guest"])
+        self.assertLess(guest_calls, 30, f"{guest_calls} guest calls for 42 packages — something is per-package again")
+
+
+class Failure(unittest.TestCase):
+    def test_install_failure_is_rc3_with_dpkg_state(self):
+        f = Fake()
+        f.install_rc = 100
+        rc, rep = run(f)
+        self.assertEqual(rc, 3)
+        self.assertEqual(rep["failed"]["rc"], 100)
+        self.assertTrue(any(l.startswith("os-apply: FAILED rc=100 step=install") for l in f.logs))
+
+
+
+class RestartSkipPattern(unittest.TestCase):
+    """The cgroup filter runs as `grep -q PATTERN /proc/<pid>/cgroup`; check it with grep itself against the cgroup
+    lines measured on demo-felhom 2026-10-04."""
+
+    def grep(self, pattern, line):
+        return subprocess.run(["grep", "-q", pattern], input=line + "\n", text=True).returncode == 0
+
+    def test_restart_skip_patterns_against_real_cgroups(self):
+        host = osapply.RESTART_SKIP_CGROUP["host"]
+        self.assertTrue(self.grep(host, "0::/lxc/9201/ns/system.slice/docker.service"), "a guest process must be skipped")
+        self.assertFalse(self.grep(host, "0::/lxc.monitor/9201"), "lxc-start must NOT be skipped (it runs the guest)")
+        self.assertFalse(self.grep(host, "0::/system.slice/pve-cluster.service"), "a host daemon must NOT be skipped")
+        guest = osapply.RESTART_SKIP_CGROUP["guest"]
+        self.assertTrue(self.grep(guest, "0::/system.slice/docker-0123abcd.scope"))
+        self.assertFalse(self.grep(guest, "0::/system.slice/cron.service"))
+
+if __name__ == "__main__":
+    unittest.main()

internal/backup/archive_completeness_test.go

@@ -0,0 +1,243 @@
+package backup
+
+import (
+	"bytes"
+	"context"
+	"log/slog"
+	"strings"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+// F-CRIT-2 (Campaign 8): a failed backup must not look like a fresh one.
+//
+// Every fixture below is a VERBATIM shape captured from the live PVE API on 2026-07-28
+// (`pvesh get /nodes/<node>/storage/<store>/content`), not a hand-invented struct. That matters:
+// the `unparseable` path in this package went untested for months behind a JSON shape that did not
+// match production, and the whole point of this fix is that presence != validity.
+
+// phantomEntry is the artefact a PBS daemon killed mid-upload leaves behind: listed as a restorable
+// backup, 1 byte, NEWEST, and carrying no `verification`/`encrypted`/`notes` at all because it has
+// no manifest (`index.json.blob` is absent on disk).
+func phantomEntry() proxmox.StorageContent {
+	return proxmox.StorageContent{
+		VolID:   "felhom-pbs:backup/ct/9201/2026-07-28T05:31:14Z",
+		Content: "backup",
+		Format:  "pbs-ct",
+		Size:    1,
+		CTime:   1785216674,
+		VMID:    9201,
+	}
+}
+
+// goodPBSEntry is a real, complete offsite snapshot (demo-hp, 2026-07-28T03:40:42Z).
+func goodPBSEntry() proxmox.StorageContent {
+	return proxmox.StorageContent{
+		VolID:   "felhom-pbs:backup/ct/9201/2026-07-28T03:40:42Z",
+		Content: "backup",
+		Format:  "pbs-ct",
+		Size:    4353457559,
+		CTime:   1785210042,
+		VMID:    9201,
+	}
+}
+
+// goodLocalEntry is a real, complete LOCAL vzdump (demo-hp). Note it legitimately has no
+// `verification` and no `encrypted` on the wire — a dir storage has no such concept — which is
+// exactly why those fields must never be used as completeness discriminators.
+func goodLocalEntry() proxmox.StorageContent {
+	return proxmox.StorageContent{
+		VolID:   "local:backup/vzdump-lxc-9201-2026_07_28-07_29_54.tar.zst",
+		Content: "backup",
+		Format:  "tar.zst",
+		Size:    1590431865,
+		CTime:   1785216594,
+		VMID:    9201,
+	}
+}
+
+func runnerWithContent(t *testing.T, buf *bytes.Buffer, content []proxmox.StorageContent) *BackupRunner {
+	t.Helper()
+	lg := slog.New(slog.NewTextHandler(buf, &slog.HandlerOptions{Level: slog.LevelDebug}))
+	return NewBackupRunner(&fakeBackupAPI{content: content}, "felhom-pbs", proxmox.ModeSnapshot, "", "", lg)
+}
+
+// Group A — the phantom must NOT set tier freshness, even though it is the newest entry.
+//
+// RED-PROOF: restore the old predicate in NewestArchiveTime
+// (`if e.Content == "backup" && e.VMID == vmid && e.CTime > best`) → the phantom's ctime
+// (1785216674) wins over the good snapshot's (1785210042) and this test fails with
+// "got 1785216674, want 1785210042" — i.e. the exact F-CRIT-2 defect.
+func TestNewestArchiveTime_PhantomIsNotCounted(t *testing.T) {
+	var buf bytes.Buffer
+	// phantom deliberately listed FIRST and is also the newest by ctime.
+	r := runnerWithContent(t, &buf, []proxmox.StorageContent{phantomEntry(), goodPBSEntry()})
+
+	got, found, err := r.NewestArchiveTime(context.Background(), 9201)
+	if err != nil {
+		t.Fatalf("NewestArchiveTime: %v", err)
+	}
+	if !found {
+		t.Fatal("found=false — the GOOD snapshot must still be counted; rejecting everything is the thrash path")
+	}
+	if got.Unix() != goodPBSEntry().CTime {
+		t.Errorf("freshness came from the wrong entry: got ctime %d, want %d (the good snapshot)", got.Unix(), goodPBSEntry().CTime)
+	}
+	if got.Unix() == phantomEntry().CTime {
+		t.Error("the 1-byte manifest-less phantom set tier freshness — this is F-CRIT-2")
+	}
+}
+
+// Group A — with ONLY a phantom present the tier must report "no backup", not a fresh one.
+// That is what lets the controller see age_state=absent and fire its first-backup valve.
+func TestNewestArchiveTime_OnlyPhantomReportsNotFound(t *testing.T) {
+	var buf bytes.Buffer
+	r := runnerWithContent(t, &buf, []proxmox.StorageContent{phantomEntry()})
+
+	_, found, err := r.NewestArchiveTime(context.Background(), 9201)
+	if err != nil {
+		t.Fatalf("NewestArchiveTime: %v", err)
+	}
+	if found {
+		t.Error("found=true with only a phantom present — the tier would report fresh and go silent for a full cadence")
+	}
+}
+
+// Group B — THE SCENARIO-D GUARD. A valid snapshot on EITHER tier must still be counted.
+//
+// This is what makes Group A safe. A filter that is too aggressive does not merely lose safety
+// margin: the tier reports absent on every poll, backs up every cycle, and the R-88 breaker cannot
+// save it because those backups SUCCEED. That is a continuous multi-GB write loop across the fleet.
+//
+// RED-PROOF: make archivePlausiblyComplete return `false, "reject everything"` unconditionally →
+// both subtests fail with found=false.
+func TestNewestArchiveTime_ValidSnapshotsAreStillCounted(t *testing.T) {
+	for _, tc := range []struct {
+		name  string
+		entry proxmox.StorageContent
+	}{
+		{"pbs offsite (has verification+encrypted on the wire)", goodPBSEntry()},
+		{"local dir vzdump (has NEITHER verification NOR encrypted — and must still count)", goodLocalEntry()},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			var buf bytes.Buffer
+			r := runnerWithContent(t, &buf, []proxmox.StorageContent{tc.entry})
+
+			got, found, err := r.NewestArchiveTime(context.Background(), 9201)
+			if err != nil {
+				t.Fatalf("NewestArchiveTime: %v", err)
+			}
+			if !found {
+				t.Fatalf("a REAL %s backup was rejected — this is the backup-thrash path, not extra safety", tc.name)
+			}
+			if got.Unix() != tc.entry.CTime {
+				t.Errorf("got ctime %d, want %d", got.Unix(), tc.entry.CTime)
+			}
+			if strings.Contains(buf.String(), "INCOMPLETE archive") {
+				t.Errorf("a valid archive was announced as incomplete:\n%s", buf.String())
+			}
+		})
+	}
+}
+
+// Group B — the smallest REAL backup measured anywhere on the fleet (612,397,450 B, a guest-9100
+// vzdump) must clear the floor with room to spare. If someone ever raises
+// minPlausibleArchiveBytes past this, that is the fleet-thrash bug and this test is the tripwire.
+func TestMinPlausibleArchiveBytes_LeavesHeadroomBelowTheSmallestRealBackup(t *testing.T) {
+	const smallestObservedRealBackup int64 = 612397450 // fleet survey 2026-07-28
+	if minPlausibleArchiveBytes >= smallestObservedRealBackup {
+		t.Fatalf("floor %d B is not below the smallest real backup ever observed (%d B) — this WILL reject real archives",
+			minPlausibleArchiveBytes, smallestObservedRealBackup)
+	}
+	if ratio := smallestObservedRealBackup / minPlausibleArchiveBytes; ratio < 100 {
+		t.Errorf("floor %d B leaves only %dx headroom below the smallest real backup (%d B) — too tight",
+			minPlausibleArchiveBytes, ratio, smallestObservedRealBackup)
+	}
+}
+
+// Group C — UNDECIDABLE ⇒ NOT COUNTED (the fail-safe direction).
+//
+// A zero/absent size is not evidence of a good backup; it is absence of evidence. Erring toward
+// "not fresh" costs one extra backup. Erring the other way is F-CRIT-2.
+//
+// RED-PROOF: flip the comparison in archivePlausiblyComplete to `e.Size > minPlausibleArchiveBytes
+// || e.Size == 0` (i.e. treat unknown as complete) → the size-0 case reports ok=true and this fails.
+func TestArchivePlausiblyComplete_UndecidableIsNotCounted(t *testing.T) {
+	for _, tc := range []struct {
+		name string
+		size int64
+	}{
+		{"the observed phantom", 1},
+		{"absent size field (unmarshals to 0)", 0},
+		{"just under the floor", minPlausibleArchiveBytes - 1},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			e := phantomEntry()
+			e.Size = tc.size
+			ok, why := archivePlausiblyComplete(e)
+			if ok {
+				t.Errorf("size %d counted as a complete backup — undecidable must fail safe", tc.size)
+			}
+			if why == "" {
+				t.Error("rejection carried no reason — a silent rejection is a new quiet path")
+			}
+		})
+	}
+	if ok, why := archivePlausiblyComplete(goodPBSEntry()); !ok {
+		t.Errorf("a real snapshot was rejected: %s", why)
+	}
+}
+
+// Group D — the rejection is announced ONCE per snapshot, not once per due-check.
+//
+// The due-check runs every 5 minutes and a phantom persists indefinitely (server-side prune does
+// not collect it), so per-poll logging would emit ~288 identical lines a day and bury the signal.
+//
+// RED-PROOF: delete the `if seen { return }` guard in warnRejectedArchiveOnce → this test reports
+// "logged 5 times, want 1".
+func TestNewestArchiveTime_RejectionLoggedOncePerSnapshot(t *testing.T) {
+	var buf bytes.Buffer
+	r := runnerWithContent(t, &buf, []proxmox.StorageContent{phantomEntry(), goodPBSEntry()})
+
+	const polls = 5
+	for i := 0; i < polls; i++ {
+		if _, _, err := r.NewestArchiveTime(context.Background(), 9201); err != nil {
+			t.Fatalf("poll %d: %v", i, err)
+		}
+	}
+
+	n := strings.Count(buf.String(), "INCOMPLETE archive")
+	if n != 1 {
+		t.Errorf("rejection logged %d times across %d polls, want exactly 1:\n%s", n, polls, buf.String())
+	}
+	out := buf.String()
+	if !strings.Contains(out, phantomEntry().VolID) {
+		t.Errorf("the log line does not NAME the rejected snapshot:\n%s", out)
+	}
+	if !strings.Contains(out, "below the") {
+		t.Errorf("the log line does not say WHY it was rejected:\n%s", out)
+	}
+	if !strings.Contains(out, "level=WARN") {
+		t.Errorf("rejection was not logged at WARN:\n%s", out)
+	}
+}
+
+// Group D — a SECOND, distinct phantom is announced separately. The dedupe must be per snapshot,
+// not a one-shot latch that hides every later phantom.
+func TestNewestArchiveTime_DistinctPhantomsEachAnnounced(t *testing.T) {
+	var buf bytes.Buffer
+	second := phantomEntry()
+	second.VolID = "felhom-pbs:backup/ct/9201/2026-07-29T05:31:14Z"
+	second.CTime = phantomEntry().CTime + 86400
+	r := runnerWithContent(t, &buf, []proxmox.StorageContent{phantomEntry(), second, goodPBSEntry()})
+
+	for i := 0; i < 3; i++ {
+		if _, _, err := r.NewestArchiveTime(context.Background(), 9201); err != nil {
+			t.Fatalf("poll %d: %v", i, err)
+		}
+	}
+	if n := strings.Count(buf.String(), "INCOMPLETE archive"); n != 2 {
+		t.Errorf("got %d rejection lines for 2 distinct phantoms across 3 polls, want 2:\n%s", n, buf.String())
+	}
+}

internal/backup/backup_test.go

@@ -2,6 +2,7 @@ package backup
 
 import (
 	"context"
+	"fmt"
 	"encoding/json"
 	"errors"
 	"io"
@@ -21,14 +22,18 @@ type fakeBackupAPI struct {
 	vzdumpErr  error
 	waitErr    error
 	cfg        proxmox.GuestConfig
+	goneGuests map[int]bool // R-689: vmids whose config lookup answers "does not exist"
+	aclGuests  map[int]bool // R-689: vmids outside the token's ACL — PVE answers 403 "permission denied"
 	cfgErr     error
 	content    []proxmox.StorageContent
 	contentErr error
-	storages   []proxmox.Storage // returned by ListStorage (the local-prune scope gate)
-	storageErr error
-	vzdumps    []proxmox.VzdumpOptions
-	logLines   []string      // returned by TaskLogTail (e.g. "INFO: backup mode: stop")
-	waitGate   chan struct{} // if non-nil, WaitTask blocks until closed (8B.2 watcher timing)
+	storages   []proxmox.Storage // returned by ListStorage (the local-prune scope gate) — DEFINITIONS: like
+	// production's GET /storage, it never carries usage; ListStorage strips Avail/Used (R-685's live lesson)
+	nodeStorages []proxmox.Storage // returned by NodeStorage (GET /nodes/{node}/storage — WITH usage)
+	storageErr   error
+	vzdumps      []proxmox.VzdumpOptions
+	logLines     []string      // returned by TaskLogTail (e.g. "INFO: backup mode: stop")
+	waitGate     chan struct{} // if non-nil, WaitTask blocks until closed (8B.2 watcher timing)
 }
 
 func (f *fakeBackupAPI) Vzdump(_ context.Context, o proxmox.VzdumpOptions) (string, error) {
@@ -41,13 +46,30 @@ func (f *fakeBackupAPI) WaitTask(_ context.Context, _ string, _ proxmox.WaitOpti
 	}
 	return proxmox.TaskStatus{Status: "stopped", ExitStatus: "OK"}, f.waitErr
 }
-func (f *fakeBackupAPI) GuestConfig(_ context.Context, _ int) (proxmox.GuestConfig, error) {
+func (f *fakeBackupAPI) GuestConfig(_ context.Context, vmid int) (proxmox.GuestConfig, error) {
+	if f.aclGuests[vmid] {
+		return proxmox.GuestConfig{}, fmt.Errorf("proxmox: GET /nodes/n/lxc/%d/config -> HTTP 403: permission denied at /vms/%d (missing privilege VM.Audit)", vmid, vmid)
+	}
+	if f.goneGuests[vmid] { // R-689: PVE's answer for a deleted guest
+		return proxmox.GuestConfig{}, fmt.Errorf("proxmox: GET /nodes/n/lxc/%d/config -> HTTP 500: Configuration file 'nodes/n/lxc/%d.conf' does not exist", vmid, vmid)
+	}
 	return f.cfg, f.cfgErr
 }
 func (f *fakeBackupAPI) StorageContent(_ context.Context, _ string) ([]proxmox.StorageContent, error) {
 	return f.content, f.contentErr
 }
 func (f *fakeBackupAPI) ListStorage(_ context.Context) ([]proxmox.Storage, error) {
+	out := make([]proxmox.Storage, len(f.storages))
+	for i, s := range f.storages {
+		s.Avail, s.Used, s.Total = 0, 0, 0 // GET /storage has no usage — a fake that had it hid R-685's defect
+		out[i] = s
+	}
+	return out, f.storageErr
+}
+func (f *fakeBackupAPI) NodeStorage(_ context.Context) ([]proxmox.Storage, error) {
+	if f.nodeStorages != nil {
+		return f.nodeStorages, f.storageErr
+	}
 	return f.storages, f.storageErr
 }
 func (f *fakeBackupAPI) TaskLogTail(_ context.Context, _ string, _ int) ([]string, error) {
@@ -135,14 +157,16 @@ func TestBackup_VzdumpFailureReturnsFailedRecord(t *testing.T) {
 }
 
 func TestPickRestoreCandidate_NewestOrEmpty(t *testing.T) {
+	const big = 4 << 30 // a plausible whole-guest archive
 	api := &fakeBackupAPI{content: []proxmox.StorageContent{
-		{VolID: "a", Content: "backup", CTime: 10},
-		{VolID: "b", Content: "backup", CTime: 99},
-		{VolID: "iso", Content: "iso", CTime: 999}, // not a backup → ignored
+		// R-689: real vzdump names with their vmid — only a backup OF A GUEST is a candidate.
+		{VolID: "local:backup/vzdump-lxc-9001-a.tar.zst", VMID: 9001, Content: "backup", CTime: 10, Size: big},
+		{VolID: "local:backup/vzdump-lxc-9001-b.tar.zst", VMID: 9001, Content: "backup", CTime: 99, Size: big},
+		{VolID: "iso", Content: "iso", CTime: 999, Size: big}, // not a backup → ignored
 	}}
 	r := NewBackupRunner(api, "local", "", "", "", quiet())
 	vol, err := r.PickRestoreCandidate(context.Background())
-	if err != nil || vol != "b" {
+	if err != nil || vol != "local:backup/vzdump-lxc-9001-b.tar.zst" {
 		t.Fatalf("pick = %q,%v want newest 'b'", vol, err)
 	}
 	// no backups → "".
@@ -152,6 +176,26 @@ func TestPickRestoreCandidate_NewestOrEmpty(t *testing.T) {
 	}
 }
 
+// R-86: the NEWEST entry is not a candidate if it cannot be a complete archive. An incomplete
+// artefact (F-CRIT-2's 1-byte phantom, which server-side prune does not collect) would otherwise be
+// picked forever, fail its restore forever, never earn proof, and so leave the tier due at every
+// evaluation — turning the evaluation interval into the retry rate for a multi-GB restore.
+//
+// COMPANION RED-PROOF (observed): drop the `archivePlausiblyComplete` guard from
+// PickSettledRestoreCandidateOn and this fails with
+// `pick = "phantom" want the newest COMPLETE archive 'real'`.
+func TestPickRestoreCandidate_SkipsImplausibleArchives(t *testing.T) {
+	api := &fakeBackupAPI{content: []proxmox.StorageContent{
+		{VolID: "felhom-pbs:backup/ct/9001/real", VMID: 9001, Content: "backup", CTime: 10, Size: 4 << 30},
+		{VolID: "felhom-pbs:backup/ct/9001/phantom", VMID: 9001, Content: "backup", CTime: 99, Size: 1}, // newest, and impossible
+	}}
+	r := NewBackupRunner(api, "local", "", "", "", quiet())
+	vol, err := r.PickRestoreCandidate(context.Background())
+	if err != nil || vol != "felhom-pbs:backup/ct/9001/real" {
+		t.Fatalf("pick = %q,%v want the newest COMPLETE archive 'real'", vol, err)
+	}
+}
+
 // --- scheduler ---
 
 type fakeRTRunner struct {
@@ -168,9 +212,12 @@ func TestScheduler_TickRunsAndRecords(t *testing.T) {
 	store := NewStore()
 	rt := &fakeRTRunner{res: reconcile.RestoreTestResult{Archive: "vol", Pass: true, Verified: "boot+running", Duration: time.Second}}
 	s := NewScheduler(SchedulerOptions{
-		Runner:  rt,
-		Pick:    func(context.Context) (string, error) { return "vol", nil },
-		Store:   store,
+		Runner: rt,
+		Pick:   func(context.Context) (string, error) { return "vol", nil },
+		Store:  store,
+		Spec: func(context.Context, string) reconcile.RestoreTestSpec {
+			return reconcile.RestoreTestSpec{RestoreStorage: "local-lvm", ScratchMin: 990000, ScratchMax: 990009}
+		},
 		Cadence: time.Hour,
 		Logger:  quiet(),
 	})

internal/backup/inflight.go

@@ -0,0 +1,69 @@
+package backup
+
+import "sync"
+
+// InFlight is the host-wide "one heavy guest operation at a time" gate.
+//
+// R-85 (Scenario F). The operator's R-82 ruling was "one backup at a time per guest"; a restore-test
+// must JOIN that single-flight rather than sit outside it. It is not a lock-contention concern —
+// a restore-test uses a scratch VMID, so it never touches the live guest's vzdump lock. It is a
+// LINK concern: an offsite restore PULLS a multi-GB archive while an offsite backup PUSHES one, over
+// the same WireGuard tunnel. On the demo fleet that link runs at ~33 MB/min upstream; running both
+// at once makes each slower and pushes both toward their timeouts, which is how a healthy tier ends
+// up recorded as failed.
+//
+// It is deliberately host-wide and coarse rather than per-guest: these boxes carry one customer
+// guest, and the resource being protected (the uplink) is shared by everything on the host anyway.
+//
+// The gate is ADVISORY in one direction only — it never cancels anything already running. A caller
+// that cannot acquire DEFERS to its next cadence. Deferring a restore-test costs a few hours of
+// coverage; cancelling a running backup costs the backup.
+//
+// CORRECTED 2026-07-28 (F-A1). That "DEFERS" was true of the restore-test caller and NOT of the
+// backup caller, and the comment did not say so. The controller's start path had no 409 branch, so
+// a refusal here was recorded as a tier FAILURE: the R-88 breaker armed and the operator was
+// emailed "Whole-guest backup FAILED" about a backup that was merely waiting its turn. Campaign 8
+// observed it on both demo boxes in the same minute.
+//
+// Fixed on the CONTROLLER side (v0.179.0), which is where the misreading lived — this gate's
+// behaviour was correct throughout and is unchanged. The controller now maps HTTP 409 to a
+// contention path: it defers the tier, keeps it DUE, and alarms only if contention outlives the
+// agent's own restore-test ceiling. Nothing here needs to change; the claim above is simply now
+// true of both callers.
+type InFlight struct {
+	mu   sync.Mutex
+	what string // "" = idle
+}
+
+// TryAcquire claims the gate for `what`. ok=false means something else holds it, and `busy` names
+// it — the name matters, because "deferred" with no reason is indistinguishable from "broken".
+func (g *InFlight) TryAcquire(what string) (release func(), busy string, ok bool) {
+	if g == nil {
+		// Not wired (older call sites, tests) → no gating, previous behaviour.
+		return func() {}, "", true
+	}
+	g.mu.Lock()
+	defer g.mu.Unlock()
+	if g.what != "" {
+		return nil, g.what, false
+	}
+	g.what = what
+	var once sync.Once
+	return func() {
+		once.Do(func() {
+			g.mu.Lock()
+			g.what = ""
+			g.mu.Unlock()
+		})
+	}, "", true
+}
+
+// Busy reports what currently holds the gate ("" = idle).
+func (g *InFlight) Busy() string {
+	if g == nil {
+		return ""
+	}
+	g.mu.Lock()
+	defer g.mu.Unlock()
+	return g.what
+}

internal/backup/r672_skip_test.go

@@ -0,0 +1,36 @@
+package backup
+
+import (
+	"context"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
+)
+
+// R-672 (v0.133.0): a restore-test the SPACE preflight refused is the test's RESULT — recorded for the
+// hub as pass=false with the reason, never dropped (a band skip still is) and never a pass.
+//
+// COMPANION RED-PROOF (REPORT): the scheduler's pre-v0.133.0 `if res.Skipped { return }` → "a space
+// refusal never reached the host report".
+func TestR672_SpaceSkipIsReportedNotDropped(t *testing.T) {
+	store := NewStore()
+	rt := &fakeRTRunner{res: reconcile.RestoreTestResult{Archive: "vol", Skipped: true,
+		SkipReason: "skipped: not enough space on local-lvm: restoring 21.1 GiB (vzdump log) needs 30.3 GiB free, has 21.6 GiB"}}
+	s := NewScheduler(SchedulerOptions{
+		Runner: rt, Pick: func(context.Context) (string, error) { return "vol", nil }, Store: store,
+		Spec: func(context.Context, string) reconcile.RestoreTestSpec {
+			return reconcile.RestoreTestSpec{RestoreStorage: "local-lvm", ScratchMin: 990000, ScratchMax: 990009}
+		},
+		Cadence: time.Hour, Logger: quiet(),
+	})
+	s.tick(context.Background())
+	got := store.RestoreTests(context.Background())
+	if len(got) != 1 {
+		t.Fatalf("a space refusal never reached the host report: %+v", got)
+	}
+	if got[0].Pass || !got[0].Skipped || !strings.HasPrefix(got[0].Error, "skipped: not enough space") {
+		t.Fatalf("record = %+v — want pass=false, skipped, the reason as the error", got[0])
+	}
+}

internal/backup/r685_space_test.go

@@ -0,0 +1,83 @@
+package backup
+
+import (
+	"context"
+	"strings"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+// R-685 (v0.134.0) — a whole-box backup that cannot fit its LOCAL target is a named skip BEFORE anything
+// starts, with the numbers in the record's Error, never a vzdump that fills the disk and fails.
+
+const gib = int64(1) << 30
+
+func spaceAPI(avail int64, lastArchive int64, typ string) *fakeBackupAPI {
+	api := &fakeBackupAPI{vzdumpUPID: "UPID:vzdump:1",
+		storages: []proxmox.Storage{{Storage: "local", Type: typ, Content: "backup", Avail: avail}}}
+	if lastArchive > 0 {
+		api.content = []proxmox.StorageContent{{VolID: "local:backup/vzdump-lxc-9201-2026_09_24-21_59_25.tar.zst",
+			Content: "backup", VMID: 9201, Size: lastArchive, CTime: 1790280000}}
+	}
+	return api
+}
+
+// TestR685_BackupThatCannotFitIsSkipped — demo-hp's shape: an 8.2 GB archive, 4 GiB free. No vzdump is
+// started; the record says why, with the numbers, under a stable prefix.
+//
+// COMPANION RED-PROOF (REPORT.md): drop the spaceFits call from backup() — this test fails at "a vzdump
+// was started on a target that cannot hold it".
+func TestR685_BackupThatCannotFitIsSkipped(t *testing.T) {
+	api := spaceAPI(4*gib, 8182759056, "dir")
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	rec, err := r.Backup(context.Background(), 9201)
+	if len(api.vzdumps) != 0 {
+		t.Fatalf("a vzdump was started on a target that cannot hold it: %+v", api.vzdumps)
+	}
+	if err == nil || rec.Success || !strings.HasPrefix(rec.Error, BackupSkipNoSpacePrefix) {
+		t.Fatalf("want a named skip, got err=%v rec=%+v", err, rec)
+	}
+	for _, want := range []string{"4.0 GiB free", "7.6 GiB", "10.5 GiB"} {
+		if !strings.Contains(rec.Error, want) {
+			t.Errorf("the reason must carry the numbers (%q missing): %s", want, rec.Error)
+		}
+	}
+}
+
+// TestR685_BackupThatFitsRuns — the same archive with 16 GiB free (demo-hp after tonight's prune) runs.
+func TestR685_BackupThatFitsRuns(t *testing.T) {
+	api := spaceAPI(16*gib, 8182759056, "dir")
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	_, _ = r.Backup(context.Background(), 9201)
+	if len(api.vzdumps) != 1 {
+		t.Fatalf("a backup that fits must run, vzdumps=%d", len(api.vzdumps))
+	}
+}
+
+// TestR685_FailsOpen — never refuse on what is not KNOWN: a PBS target, a first backup (no archive to
+// size from), an unknown free figure, or a storage list that cannot be read.
+func TestR685_FailsOpen(t *testing.T) {
+	cases := map[string]*fakeBackupAPI{
+		"pbs target":    spaceAPI(1*gib, 8*gib, "pbs"),
+		"first backup":  spaceAPI(1*gib, 0, "dir"),
+		"avail unknown": spaceAPI(0, 8*gib, "dir"),
+		"storage list error": func() *fakeBackupAPI {
+			a := spaceAPI(1*gib, 8*gib, "dir")
+			a.storageErr = context.DeadlineExceeded
+			return a
+		}(),
+	}
+	for name, api := range cases {
+		target := "local"
+		if name == "pbs target" {
+			api.storages[0].Storage = "felhom-pbs"
+			target = "felhom-pbs"
+		}
+		r := NewBackupRunner(api, target, proxmox.ModeSnapshot, "", "", quiet())
+		_, _ = r.Backup(context.Background(), 9201)
+		if len(api.vzdumps) != 1 {
+			t.Errorf("%s: the preflight must fail OPEN, but no vzdump ran", name)
+		}
+	}
+}

internal/backup/r689_golden_test.go

@@ -0,0 +1,112 @@
+package backup
+
+import (
+	"context"
+	"fmt"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+// R-689 (v0.135.0) — demo-hp keeps its golden template in `local:backup/`. It is content "backup",
+// 654 MB and so "plausibly complete", and it was the newest SETTLED entry: the restore test picked it
+// every 6 h and failed extractconfig with a 403 (measured 2026-09-24 10:36, 09-25 04:57 and 10:57),
+// while the guest's own archive — younger than the 24 h settle — went untested and nothing was proven.
+//
+// COMPANION RED-PROOF (REPORT.md): drop the guestBackupArchive call from PickSettledRestoreCandidateOn —
+// this test then picks `local:backup/felhom-golden-0.236.0.tar.zst`.
+func TestR689_TheRestoreTestNeverPicksTheGolden(t *testing.T) {
+	const day = int64(86400)
+	now := int64(1790370000) // 2026-09-25 ~19:00Z
+	api := &fakeBackupAPI{content: []proxmox.StorageContent{
+		// the guest's real archive, settled (older than the cutoff below)
+		{VolID: "local:backup/vzdump-lxc-9201-2026_09_22-21_59_25.tar.zst", Content: "backup", VMID: 9201, Size: 8 << 30, CTime: now - 3*day},
+		// the golden: newer, settled, big, and NOT a backup of a guest
+		{VolID: "local:backup/felhom-golden-0.236.0.tar.zst", Content: "backup", Size: 654115664, CTime: now - 2*day},
+		// a hand-copied tarball that PVE happens to attribute to a vmid — the name is not a vzdump's
+		{VolID: "local:backup/copy-of-9201.tar.zst", Content: "backup", VMID: 9201, Size: 8 << 30, CTime: now - 2*day},
+	}}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "local", time.Unix(now-day, 0).UTC())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got != "local:backup/vzdump-lxc-9201-2026_09_22-21_59_25.tar.zst" {
+		t.Fatalf("picked %q — the restore test must prove a backup OF A GUEST", got)
+	}
+}
+
+func TestR689_GuestBackupArchiveShapes(t *testing.T) {
+	for _, c := range []struct {
+		e  proxmox.StorageContent
+		ok bool
+	}{
+		{proxmox.StorageContent{VolID: "local:backup/vzdump-lxc-9201-2026_09_24-21_59_25.tar.zst", VMID: 9201}, true},
+		{proxmox.StorageContent{VolID: "local:backup/vzdump-qemu-300-2026_09_24-21_59_25.vma.zst", VMID: 300}, true},
+		{proxmox.StorageContent{VolID: "felhom-pbs:backup/ct/9201/2026-07-28T05:31:14Z", VMID: 9201}, true},
+		{proxmox.StorageContent{VolID: "felhom-pbs:backup/vm/300/2026-07-28T05:31:14Z", VMID: 300}, true},
+		{proxmox.StorageContent{VolID: "local:backup/felhom-golden-0.236.0.tar.zst"}, false},
+		{proxmox.StorageContent{VolID: "local:backup/vzdump-lxc-9201-x.tar.zst", VMID: 9202}, false}, // vmid disagrees with the name
+		{proxmox.StorageContent{VolID: "felhom-pbs:backup/ct/9201/2026-07-28T05:31:14Z"}, false},       // no vmid reported
+	} {
+		if ok, why := guestBackupArchive(c.e); ok != c.ok {
+			t.Errorf("%s vmid=%d: ok=%v (%s), want %v", c.e.VolID, c.e.VMID, ok, why, c.ok)
+		}
+	}
+}
+
+// R-689 (v0.136.0) — the measured demo-hp shape right after v0.135.0: the golden (skipped), a leftover archive
+// of guest 9100 deleted in August (settled), and today's archive of 9201 (not settled yet). The pick must be
+// NOTHING — never the deleted guest's archive. With 9201's archive settled, that one.
+//
+// COMPANION RED-PROOF (REPORT.md): drop the known-guest check — the pick is the 9100 leftover.
+func TestR689_AnArchiveOfADeletedGuestIsNeverPicked(t *testing.T) {
+	const day = int64(86400)
+	now := int64(1790476000)
+	api := &fakeBackupAPI{goneGuests: map[int]bool{9100: true}, content: []proxmox.StorageContent{
+		{VolID: "local:backup/felhom-golden-0.236.0.tar.zst", Content: "backup", Size: 654115664, CTime: now - 14*day},
+		{VolID: "local:backup/vzdump-lxc-9100-2026_08_21-17_59_15.tar.zst", Content: "backup", VMID: 9100, Size: 656970239, CTime: now - 37*day},
+		{VolID: "local:backup/vzdump-lxc-9201-2026_09_27-04_35_47.tar.zst", Content: "backup", VMID: 9201, Size: 8 << 30, CTime: now - 7*3600},
+	}}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "local", time.Unix(now-day, 0).UTC())
+	if err != nil || got != "" {
+		t.Fatalf("picked %q err=%v — a deleted guest's archive proves nothing about this box", got, err)
+	}
+	got, _, _ = r.PickSettledRestoreCandidateOn(context.Background(), "local", time.Unix(now, 0).UTC())
+	if got != "local:backup/vzdump-lxc-9201-2026_09_27-04_35_47.tar.zst" {
+		t.Fatalf("with 9201's archive settled the pick is %q", got)
+	}
+}
+
+// Any OTHER lookup failure is not "the guest is gone": the tier must read UNKNOWN (an error), never
+// "nothing to prove".
+func TestR689_AGuestLookupFailureIsUnknownNotEmpty(t *testing.T) {
+	api := &fakeBackupAPI{cfgErr: fmt.Errorf("proxmox: connection refused"), content: []proxmox.StorageContent{
+		{VolID: "local:backup/vzdump-lxc-9201-x.tar.zst", Content: "backup", VMID: 9201, Size: 8 << 30, CTime: 10},
+	}}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	if _, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "local", time.Time{}); err == nil {
+		t.Fatal("a failed guest lookup read as a clean answer")
+	}
+}
+
+// v0.137.0 — THE MEASURED ANSWER: the agent's token sees only its pool, so for the deleted guest PVE says 403
+// "permission denied at /vms/9100", not "does not exist" (demo-hp, right after v0.136.0 — the local tier read
+// UNKNOWN). Such a guest is not one this agent manages: its archive is skipped, the tier is not an error.
+//
+// COMPANION RED-PROOF (REPORT.md): drop the "permission denied" case — the pick errors.
+func TestR689_AGuestOutsideTheAgentsACLIsNotAKnownGuest(t *testing.T) {
+	const day = int64(86400)
+	now := int64(1790476000)
+	api := &fakeBackupAPI{aclGuests: map[int]bool{9100: true}, content: []proxmox.StorageContent{
+		{VolID: "local:backup/vzdump-lxc-9100-2026_08_21-17_59_15.tar.zst", Content: "backup", VMID: 9100, Size: 656970239, CTime: now - 37*day},
+		{VolID: "local:backup/vzdump-lxc-9201-2026_09_27-04_35_47.tar.zst", Content: "backup", VMID: 9201, Size: 8 << 30, CTime: now - 7*3600},
+	}}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "local", time.Unix(now-day, 0).UTC())
+	if err != nil || got != "" {
+		t.Fatalf("picked %q err=%v — want nothing and no error (the only settled archive is not ours)", got, err)
+	}
+}

internal/backup/r727_own_archives_test.go

@@ -0,0 +1,74 @@
+package backup
+
+import (
+	"context"
+	"errors"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+const (
+	thisBoxKey  = "de:51:7a:18:cb:39:22:30:2c:84:f5:8b:d1:91:4b:7e:81:bb:69:b8:89:0f:57:ac:d3:59:e1:1a:62:25:11:2c"
+	earlierBox1 = "6b:ca:5f:3f:ca:0f:e2:3f:fb:24:62:89:bf:e7:64:59:9a:41:c5:e6:e3:9f:3f:5f:e1:71:7b:a1:9d:24:67:82"
+	earlierBox2 = "fe:3d:db:95:d4:df:ab:e1:7d:4a:89:fa:2b:07:53:6a:e4:d2:85:95:d1:90:27:4b:d9:c6:92:20:95:04:e5:d4"
+)
+
+// R-727 (v0.138.0) — the 2026-09-30 shape, measured on a fresh box for a returning customer: the PBS
+// namespace held two archives of earlier boxes (same guest 9201, same token) and this box's own, which was not
+// settled yet. The old picker chose the earlier box's newest settled archive and failed `wrong key`.
+// The CONSEQUENCE asserted: no archive of another box is ever picked; with this box's archive settled it is picked.
+// COMPANION RED-PROOF: remove the `ownKey != "" && !EqualFold(...)` skip → the first case picks 2026-09-16T21:59:54Z.
+func TestR727_TheRestoreTestTakesOnlyThisBoxsArchives(t *testing.T) {
+	day := int64(86400)
+	now := int64(1790740000) // 2026-09-30 ~04:00Z
+	own := proxmox.StorageContent{VolID: "felhom-pbs:backup/ct/9201/2026-09-29T19:37:07Z", Content: "backup", VMID: 9201, Size: 3490689830, CTime: 1790710627, Encrypted: thisBoxKey}
+	api := &fakeBackupAPI{
+		storages: []proxmox.Storage{{Storage: "felhom-pbs", Type: "pbs", EncryptionKey: thisBoxKey}},
+		content: []proxmox.StorageContent{
+			{VolID: "felhom-pbs:backup/ct/9201/2026-09-16T17:27:32Z", Content: "backup", VMID: 9201, Size: 4774114206, CTime: 1789579652, Encrypted: earlierBox2},
+			{VolID: "felhom-pbs:backup/ct/9201/2026-09-16T21:59:54Z", Content: "backup", VMID: 9201, Size: 20811501236, CTime: 1789595994, Encrypted: earlierBox1},
+			own,
+		},
+	}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+
+	// 1. The night of 2026-09-30: this box's own archive is ~6 h old, not settled (cutoff 24 h) — nothing to prove.
+	got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Unix(now-day, 0).UTC())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got != "" {
+		t.Fatalf("picked %q — an archive of ANOTHER box is never this box's proof (R-727)", got)
+	}
+	// 2. A day later this box's own archive is settled — it is the one picked.
+	got, _, err = r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Unix(now+day, 0).UTC())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got != own.VolID {
+		t.Fatalf("picked %q, want this box's own %q", got, own.VolID)
+	}
+}
+
+// An unencrypted storage (a local dir) holds only this box's vzdumps — no key filter applies.
+func TestR727_UnencryptedStorageIsNotFiltered(t *testing.T) {
+	api := &fakeBackupAPI{
+		storages: []proxmox.Storage{{Storage: "local", Type: "dir"}},
+		content:  []proxmox.StorageContent{{VolID: "local:backup/vzdump-lxc-9201-2026_09_29-21_27_05.tar.zst", Content: "backup", VMID: 9201, Size: 955425507, CTime: 1790710025}},
+	}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	if got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "local", time.Time{}); err != nil || got == "" {
+		t.Fatalf("got %q err %v", got, err)
+	}
+}
+
+// A storage-list failure makes the tier UNKNOWN (an error), never "nothing to prove".
+func TestR727_KeyLookupFailureIsUnknown(t *testing.T) {
+	api := &fakeBackupAPI{storageErr: errors.New("proxmox: GET /storage -> HTTP 500"), content: []proxmox.StorageContent{{VolID: "felhom-pbs:backup/ct/9201/x", Content: "backup", VMID: 9201}}}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	if _, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Time{}); err == nil {
+		t.Fatal("a failed key lookup must surface as an error (tier UNKNOWN)")
+	}
+}

internal/backup/restoretest_due.go

@@ -0,0 +1,168 @@
+package backup
+
+import (
+	"context"
+	"fmt"
+	"time"
+)
+
+// R-86 — a restore-test follows the BACKUP, not the clock.
+//
+// ── WHAT WAS WRONG ───────────────────────────────────────────────────────────────────────────
+//
+// The trigger was `time.NewTicker(cadence)` started at daemon start, and the tier was chosen by
+// oldest-proven rotation. Its phase was therefore the PROCESS'S UPTIME: agent deploys are routine,
+// so the test drifted to an arbitrary time of day every week; a fresh archive could sit unproven
+// while an older one was re-tested; and a weekly tier was tested on the same rhythm as a daily one,
+// sometimes twice on the same archive.
+//
+// ── THE RULE, AND THE TRAP IN ITS OBVIOUS FORM ───────────────────────────────────────────────
+//
+// R-86's ask reads "test a tier ~24 h after its own newest archive". Implemented literally —
+// *"due when the newest archive is at least `settle` old"* — a DAILY tier is NEVER due: a new
+// archive lands every day, so the newest archive's age resets to zero long before it reaches 24 h.
+// The naive rule silently switches restore-testing off for the tier that matters most, and it is
+// the version a reasonable person would write. It has a red-proof of its own
+// (TestDue_NaiveNewestArchiveAgeRuleNeverFiresOnADailyTier).
+//
+// The rule implemented here:
+//
+//	Let A = the newest archive on this tier that is at least `settle` old.
+//	The tier is DUE when A exists and A HAS NOT ALREADY BEEN PROVEN.
+//
+//	daily tier    → A is yesterday's archive; a new one settles each day  → proved once per day
+//	weekly tier   → A is last week's until the next settles               → proved once per week
+//	newborn tier  → A does not exist                                      → UNKNOWN, never a fault
+//
+// Per-archive due-ness IS the pacing: one test per archive generation and no more. There is
+// deliberately no second rate limiter on top of it (§8.4) — two independent pacing mechanisms
+// produce a cadence nobody can predict from either.
+//
+// ── WHAT DID NOT CHANGE ──────────────────────────────────────────────────────────────────────
+//
+// The one-heavy-operation gate, the success-only proof credit, the oldest-proven ordering (now the
+// tie-break between two DUE tiers), the restore-test itself, its journal and its scratch band. Only
+// the trigger changed.
+
+// DueVerdict is one tier's due-ness, and the evidence for it. Every field is logged: a due-check
+// that cannot say WHY is a quiet path, and quiet paths are what this monitor family keeps shipping.
+type DueVerdict struct {
+	Target string // the tier's storage target id
+
+	// Due is true only when Archive is set and has not been proven.
+	Due bool
+	// Archive is the settled candidate A ("" when the tier holds none).
+	Archive string
+	// Landed is when A landed on the tier (zero when Archive is "").
+	Landed time.Time
+	// ProvenArchive is what the state says was last proven on this tier ("" = nothing/legacy).
+	ProvenArchive string
+	// Err is a candidate-lookup failure. A tier whose archives cannot be listed is UNKNOWN — it is
+	// NEVER reported as "not due", which would silently retire a tier the moment its storage
+	// stopped answering. Due stays false (we have no archive to test) and the error travels.
+	Err error
+	// Reason is the one-line human account of this verdict.
+	Reason string
+}
+
+// String renders a verdict for the operator log / selftest output.
+func (v DueVerdict) String() string {
+	return fmt.Sprintf("tier=%s due=%v archive=%q reason=%s", v.Target, v.Due, v.Archive, v.Reason)
+}
+
+// EvaluateDue returns the due verdict for every configured tier, ordered oldest-proven first.
+//
+// Ordering is the R-85 rotation, demoted to a TIE-BREAK: it no longer decides whether a test
+// happens (due-ness does), only which of several due tiers goes first. Keeping it means a tier can
+// still never be starved — a tier that has waited longest is served first — and keeping it as the
+// order rather than as the trigger is the whole of this change.
+func (s *Scheduler) EvaluateDue(ctx context.Context) []DueVerdict {
+	if !s.rotating() {
+		return nil
+	}
+	order := s.tiers
+	if s.rtState != nil {
+		order = s.rtState.OldestFirst(s.tiers)
+	}
+	cutoff := s.settleCutoff()
+	out := make([]DueVerdict, 0, len(order))
+	for _, target := range order {
+		out = append(out, s.evaluateTier(ctx, target, cutoff))
+	}
+	return out
+}
+
+// settleCutoff is the newest landing time an archive may have and still count as settled.
+func (s *Scheduler) settleCutoff() time.Time {
+	if s.settle <= 0 {
+		return time.Time{} // no settle requirement configured → any archive is a candidate
+	}
+	return s.now().Add(-s.settle)
+}
+
+// evaluateTier is the per-tier due-check. PURE given the picker and the state, so the rule is
+// unit-tested directly rather than inferred from whether a fake runner happened to be called.
+func (s *Scheduler) evaluateTier(ctx context.Context, target string, cutoff time.Time) DueVerdict {
+	v := DueVerdict{Target: target}
+	archive, landed, err := s.tierPick(ctx, target, cutoff)
+	if err != nil {
+		// UNKNOWN, never "not due", and never silent.
+		v.Err = err
+		v.Reason = fmt.Sprintf("candidate lookup FAILED (%v) — tier is unknown this evaluation, not proven and not dismissed", err)
+		return v
+	}
+	v.Archive, v.Landed = archive, landed
+	if archive == "" {
+		v.Reason = "no settled archive yet — nothing to prove (newborn or still settling)"
+		return v
+	}
+	proven, ok := "", false
+	if s.rtState != nil {
+		proven, ok = s.rtState.ProvenArchive(target)
+	}
+	v.ProvenArchive = proven
+	if ok && proven == archive {
+		v.Reason = fmt.Sprintf("newest settled archive (landed %s) is already proven", landed.Format(time.RFC3339))
+		return v
+	}
+	v.Due = true
+	switch {
+	case !ok && proven == "":
+		v.Reason = fmt.Sprintf("newest settled archive (landed %s) has not been proven; nothing proven on this tier yet", landed.Format(time.RFC3339))
+	default:
+		v.Reason = fmt.Sprintf("newest settled archive (landed %s) has not been proven (last proven archive was a different one)", landed.Format(time.RFC3339))
+	}
+	return v
+}
+
+// EvaluateDueTier is EvaluateDue for ONE named tier — the selftest's per-tier cost probe, so the
+// WAN leg of an offsite lookup is attributable rather than buried in an aggregate.
+func (s *Scheduler) EvaluateDueTier(ctx context.Context, target string) DueVerdict {
+	return s.evaluateTier(ctx, target, s.settleCutoff())
+}
+
+// verdictSummary renders one compact line of per-tier verdicts for the "nothing due" log.
+//
+// It re-evaluates rather than threading the verdicts out of pickForThisRun, and that is a
+// deliberate trade: this runs only on the path where NOTHING is due, so the cost is one extra
+// storage listing per tier on an otherwise idle evaluation (measured 18 ms local / 392 ms offsite,
+// R-86 Part 1.4), and in exchange the logging path cannot drift from the deciding path by holding a
+// stale copy of it. If that cost ever matters, pass the verdicts in — do not let the two diverge.
+func (s *Scheduler) verdictSummary(ctx context.Context) string {
+	out := ""
+	for _, v := range s.EvaluateDue(ctx) {
+		if out != "" {
+			out += "; "
+		}
+		switch {
+		case v.Err != nil:
+			out += v.Target + ": UNKNOWN (" + v.Err.Error() + ")"
+		default:
+			out += v.Target + ": " + v.Reason
+		}
+	}
+	if out == "" {
+		return "no tiers configured"
+	}
+	return out
+}

internal/backup/restoretest_due_test.go

@@ -0,0 +1,593 @@
+package backup
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
+)
+
+// R-86 — the restore-test follows the BACKUP, not the clock.
+//
+// Every test here DRIVES time (`s.now` is injected and stepped) rather than waiting for it. A test
+// that slept could not say anything about a 24-hour rule in under 24 hours, and one that only
+// asserted "no error" would pass against a scheduler that never ran anything at all — which is
+// precisely the failure mode §8.1's trap produces. So the assertions are: did a test run, on WHICH
+// archive, and did a second evaluation correctly run NOTHING.
+
+// ── the fake tier storage ────────────────────────────────────────────────────────────────────
+
+// archiveStub is one archive on a tier: its volid and when it landed.
+type archiveStub struct {
+	volid  string
+	landed time.Time
+}
+
+// tierStorage is a TierPicker over per-tier archive lists. It implements the SAME contract as the
+// production picker (*BackupRunner).PickSettledRestoreCandidateOn — newest archive that landed at
+// or before the cutoff — which is itself covered against a fake PVE API in backup_test.go, and
+// end-to-end by the live run. Naming the seam explicitly: everything below is true up to this
+// picker; that the real picker obeys the same rule is asserted there, not here.
+type tierStorage struct {
+	archives map[string][]archiveStub
+	err      map[string]error // target → lookup failure
+}
+
+func (ts *tierStorage) pick(_ context.Context, target string, notAfter time.Time) (string, time.Time, error) {
+	if e, ok := ts.err[target]; ok && e != nil {
+		return "", time.Time{}, e
+	}
+	var best archiveStub
+	for _, a := range ts.archives[target] {
+		if !notAfter.IsZero() && a.landed.After(notAfter) {
+			continue // not settled yet
+		}
+		if best.volid == "" || a.landed.After(best.landed) {
+			best = a
+		}
+	}
+	return best.volid, best.landed, nil
+}
+
+// dueHarness is a scheduler with a driven clock over a fake tier storage.
+type dueHarness struct {
+	s     *Scheduler
+	rr    *rotRunner
+	st    *RestoreTestState
+	ts    *tierStorage
+	clock time.Time
+	path  string
+}
+
+func newDueHarness(t *testing.T, start time.Time, settle time.Duration, pass bool, tiers []string, ts *tierStorage) *dueHarness {
+	t.Helper()
+	return newDueHarnessAt(t, filepath.Join(t.TempDir(), "rt.json"), start, settle, pass, tiers, ts)
+}
+
+func newDueHarnessAt(t *testing.T, statePath string, start time.Time, settle time.Duration, pass bool, tiers []string, ts *tierStorage) *dueHarness {
+	t.Helper()
+	h := &dueHarness{rr: &rotRunner{pass: pass}, ts: ts, clock: start, path: statePath}
+	h.st = NewRestoreTestState(statePath)
+	h.s = NewScheduler(SchedulerOptions{
+		Runner: h.rr,
+		Store:  NewStore(),
+		Spec: func(_ context.Context, archive string) reconcile.RestoreTestSpec {
+			return reconcile.RestoreTestSpec{RestoreStorage: "local-lvm", ScratchMin: 990000, ScratchMax: 990009}
+		},
+		Cadence:  time.Hour,
+		Settle:   settle,
+		Logger:   quiet(),
+		Tiers:    tiers,
+		TierPick: ts.pick,
+		State:    h.st,
+		InFlight: &InFlight{},
+	})
+	h.s.now = func() time.Time { return h.clock }
+	return h
+}
+
+// advance steps the clock by step, evaluating once at every step — the scheduler's real shape.
+func (h *dueHarness) advance(step, total time.Duration) {
+	for elapsed := time.Duration(0); elapsed < total; elapsed += step {
+		h.clock = h.clock.Add(step)
+		h.s.tick(context.Background())
+	}
+}
+
+var day0 = time.Date(2026, 8, 1, 2, 0, 0, 0, time.UTC)
+
+// dailyArchives lands one archive a day at 02:00 for n days, starting at day0.
+func dailyArchives(tier string, n int) []archiveStub {
+	out := make([]archiveStub, 0, n)
+	for d := 0; d < n; d++ {
+		out = append(out, archiveStub{
+			volid:  fmt.Sprintf("%s:backup/vzdump-lxc-9201-day%d.tar.zst", tier, d),
+			landed: day0.AddDate(0, 0, d),
+		})
+	}
+	return out
+}
+
+// ── SCENARIO A — a daily tier is proved daily, on its own archive ────────────────────────────
+//
+// THE TRAP THIS PINS (§8.1). R-86 reads "trigger a tier ~24 h after its own newest archive", and
+// the literal implementation of that — *due when the newest archive is at least `settle` old* — is
+// NEVER true on a daily tier: a new archive lands every day, so the newest archive's age resets to
+// zero long before it reaches 24 h. The literal reading silently switches restore-testing OFF for
+// the tier that matters most.
+//
+// COMPANION RED-PROOF (observed 2026-08-03). In Scheduler.evaluateTier, the per-archive comparison
+// was replaced by the naive age rule:
+//
+//   - if ok && proven == archive { … not due … }
+//   - if s.now().Sub(landed) < s.settle { … not due … }   // and the proven-archive check deleted
+//
+// and the picker cutoff was removed (`cutoff := time.Time{}`), i.e. exactly "is the newest archive
+// old enough". Result:
+//
+//	--- FAIL: TestDue_DailyTierIsProvedDailyOnItsOwnArchive
+//	    restoretest_due_test.go: a daily tier must be proved once per day; got 0 run(s) over 5 days
+//
+// Zero runs — restore-testing off. Restored immediately afterwards.
+func TestDue_DailyTierIsProvedDailyOnItsOwnArchive(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"local": dailyArchives("local", 6)}}
+	h := newDueHarness(t, day0.Add(time.Hour), 24*time.Hour, true, []string{"local"}, ts)
+
+	// Five days, evaluated hourly.
+	h.advance(time.Hour, 5*24*time.Hour)
+
+	got := h.rr.seen()
+	if len(got) != 5 {
+		t.Fatalf("a daily tier must be proved once per day; got %d run(s) over 5 days: %v", len(got), got)
+	}
+	// And each run must be on the archive that settled that day — day0's on day 1, and so on.
+	for i, a := range got {
+		want := fmt.Sprintf("local:backup/vzdump-lxc-9201-day%d.tar.zst", i)
+		if a != want {
+			t.Fatalf("run %d tested %q, want %q — the test is not following the archive", i+1, a, want)
+		}
+	}
+	// The newest archive is NEVER the one tested: it has not settled.
+	if last := got[len(got)-1]; last == "local:backup/vzdump-lxc-9201-day5.tar.zst" {
+		t.Fatal("the still-settling archive was tested — the settle cutoff is not being applied")
+	}
+}
+
+// ── SCENARIO B — a weekly tier is proved weekly, not every other day ─────────────────────────
+func TestDue_WeeklyTierIsProvedOncePerArchive(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"felhom-pbs": {
+		{volid: "felhom-pbs:backup/ct/9201/w0", landed: day0},
+		{volid: "felhom-pbs:backup/ct/9201/w1", landed: day0.AddDate(0, 0, 7)},
+		{volid: "felhom-pbs:backup/ct/9201/w2", landed: day0.AddDate(0, 0, 14)},
+	}}}
+	h := newDueHarness(t, day0.Add(time.Hour), 24*time.Hour, true, []string{"felhom-pbs"}, ts)
+
+	// Three weeks, evaluated every 6 hours — 84 evaluations.
+	h.advance(6*time.Hour, 21*24*time.Hour)
+
+	got := h.rr.seen()
+	want := []string{
+		"felhom-pbs:backup/ct/9201/w0",
+		"felhom-pbs:backup/ct/9201/w1",
+		"felhom-pbs:backup/ct/9201/w2",
+	}
+	if len(got) != len(want) {
+		t.Fatalf("a weekly tier must be proved ONCE PER ARCHIVE (3 archives over 3 weeks); got %d run(s): %v", len(got), got)
+	}
+	for i := range want {
+		if got[i] != want[i] {
+			t.Fatalf("run %d tested %q, want %q", i+1, got[i], want[i])
+		}
+	}
+}
+
+// ── SCENARIO C — an agent restart does not change the schedule ───────────────────────────────
+//
+// This is the defect a person actually notices: today every deploy restarts the ticker, so a
+// restore-test runs one interval after each deploy regardless of what has already been proven.
+//
+// COMPANION RED-PROOF (observed 2026-08-03): revert the state to per-tier TIME by making
+// ProvenArchive ignore the stored archive —
+//
+//   - if !ok || p.Archive == "" { return "", false }
+//   - return "", false     // per-tier time only, the pre-R-86 state
+//
+// →	--- FAIL: TestDue_RestartRunsNothing
+//
+//	restoretest_due_test.go:226: an agent restart must not trigger a restore-test; 2 restart(s)
+//	produced 4 run(s)
+//
+// Four: the same already-proven archive re-tested on EVERY evaluation after EVERY restart, which is
+// today's behaviour with the ticker's phase reset by the deploy. Restored.
+func TestDue_RestartRunsNothing(t *testing.T) {
+	dir := t.TempDir()
+	path := filepath.Join(dir, "rt.json")
+	ts := &tierStorage{archives: map[string][]archiveStub{"local": dailyArchives("local", 2)}}
+	start := day0.AddDate(0, 0, 1).Add(time.Hour) // day 1, 03:00 — day0's archive has settled
+
+	h := newDueHarnessAt(t, path, start, 24*time.Hour, true, []string{"local"}, ts)
+	h.s.tick(context.Background())
+	if n := len(h.rr.seen()); n != 1 {
+		t.Fatalf("precondition: the settled archive should have been proved once; got %d run(s)", n)
+	}
+
+	// --- two restarts: brand-new scheduler + brand-new state object over the SAME file ---
+	total := 0
+	for i := 0; i < 2; i++ {
+		h2 := newDueHarnessAt(t, path, start.Add(time.Duration(i+1)*time.Hour), 24*time.Hour, true, []string{"local"}, ts)
+		h2.s.tick(context.Background())
+		h2.s.tick(context.Background())
+		total += len(h2.rr.seen())
+	}
+	if total != 0 {
+		t.Fatalf("an agent restart must not trigger a restore-test; 2 restart(s) produced %d run(s)", total)
+	}
+}
+
+// ── SCENARIO D — a new archive makes a tier due even if it was tested yesterday ──────────────
+func TestDue_NewSettledArchiveMakesAProvedTierDueAgain(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"local": dailyArchives("local", 2)}}
+	h := newDueHarness(t, day0.AddDate(0, 0, 1).Add(time.Hour), 24*time.Hour, true, []string{"local"}, ts)
+
+	h.s.tick(context.Background()) // proves day0's archive
+	h.s.tick(context.Background()) // nothing new has settled → nothing
+	if n := len(h.rr.seen()); n != 1 {
+		t.Fatalf("want exactly 1 run before the new archive settles, got %d: %v", n, h.rr.seen())
+	}
+
+	// Day 2, 03:00 — day1's archive has now settled.
+	h.clock = day0.AddDate(0, 0, 2).Add(time.Hour)
+	h.s.tick(context.Background())
+
+	got := h.rr.seen()
+	if len(got) != 2 {
+		t.Fatalf("a newly settled archive must make the tier due again; got %v", got)
+	}
+	if got[1] != "local:backup/vzdump-lxc-9201-day1.tar.zst" {
+		t.Fatalf("the NEW archive must be the one tested; got %q", got[1])
+	}
+}
+
+// ── SCENARIO E — a failing tier keeps being retried, and earns no proof ──────────────────────
+//
+// COMPANION RED-PROOF (observed 2026-08-03): give credit on failure in Scheduler.tick —
+//
+//   - if rt.Pass && s.rtState != nil && target != "" {
+//   - if s.rtState != nil && target != "" {
+//
+// →	--- FAIL: TestDue_FailingTierIsRetriedAndNeverProven
+//
+//	restoretest_due_test.go: a failing tier must keep being retried; got 1 run(s) over 3
+//	evaluations
+//
+// A single failure would have retired the archive as proven — a permanently broken DR tier looking
+// freshly verified, which is the loudest signal this system produces going silent. Restored.
+func TestDue_FailingTierIsRetriedAndNeverProven(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"local": dailyArchives("local", 1)}}
+	h := newDueHarness(t, day0.AddDate(0, 0, 1).Add(time.Hour), 24*time.Hour, false, []string{"local"}, ts)
+
+	for i := 0; i < 3; i++ {
+		h.s.tick(context.Background())
+	}
+
+	got := h.rr.seen()
+	if len(got) != 3 {
+		t.Fatalf("a failing tier must keep being retried; got %d run(s) over 3 evaluations: %v", len(got), got)
+	}
+	if _, ok := h.st.ProvenArchive("local"); ok {
+		t.Fatal("a FAILED restore-test must not record the archive as proven")
+	}
+	if _, ok := h.st.LastSuccess("local"); ok {
+		t.Fatal("a FAILED restore-test must not stamp the tier as proven")
+	}
+}
+
+// ── SCENARIO F — two tiers due at once do not run at once ────────────────────────────────────
+func TestDue_TwoDueTiersRunOneAtATime(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{
+		"local":      {{volid: "local:backup/a.tar.zst", landed: day0}},
+		"felhom-pbs": {{volid: "felhom-pbs:backup/ct/9201/a", landed: day0}},
+	}}
+	h := newDueHarness(t, day0.AddDate(0, 0, 1).Add(time.Hour), 24*time.Hour, true, []string{"local", "felhom-pbs"}, ts)
+
+	// Both tiers are due at this instant.
+	due := h.s.EvaluateDue(context.Background())
+	if len(due) != 2 || !due[0].Due || !due[1].Due {
+		t.Fatalf("precondition: both tiers should be due; got %v", due)
+	}
+
+	h.s.tick(context.Background())
+	if n := len(h.rr.seen()); n != 1 {
+		t.Fatalf("ONE evaluation must start ONE restore-test, never two multi-GB restores over one link; got %d: %v", n, h.rr.seen())
+	}
+
+	// The other tier was DEFERRED, not cancelled: it is still due and runs on the next evaluation.
+	h.s.tick(context.Background())
+	got := h.rr.seen()
+	if len(got) != 2 || got[0] == got[1] {
+		t.Fatalf("the deferred tier must run on the NEXT evaluation, on its own archive; got %v", got)
+	}
+}
+
+// The heavy-operation gate still holds, and a tier deferred behind a backup stays DUE.
+func TestDue_DeferredBehindABackupStaysDue(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"local": {{volid: "local:backup/a.tar.zst", landed: day0}}}}
+	h := newDueHarness(t, day0.AddDate(0, 0, 1).Add(time.Hour), 24*time.Hour, true, []string{"local"}, ts)
+
+	gate := &InFlight{}
+	h.s.inFlight = gate
+	release, _, _ := gate.TryAcquire("backup:felhom-pbs")
+
+	h.s.tick(context.Background())
+	if n := len(h.rr.seen()); n != 0 {
+		t.Fatalf("the restore-test must DEFER while a backup holds the gate; got %d run(s)", n)
+	}
+	if due := h.s.EvaluateDue(context.Background()); !due[0].Due {
+		t.Fatal("a deferred tier must remain DUE — deferral is not dismissal")
+	}
+	release()
+	h.s.tick(context.Background())
+	if n := len(h.rr.seen()); n != 1 {
+		t.Fatalf("must resume once the gate frees; got %d run(s)", n)
+	}
+}
+
+// ── SCENARIO H — a newborn box is UNKNOWN, not stale and not a fault ─────────────────────────
+func TestDue_NewbornTierIsNotDueAndNotAnError(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"felhom-pbs": nil}}
+	h := newDueHarness(t, day0, 24*time.Hour, true, []string{"felhom-pbs"}, ts)
+
+	due := h.s.EvaluateDue(context.Background())
+	if len(due) != 1 {
+		t.Fatalf("want one verdict, got %v", due)
+	}
+	v := due[0]
+	if v.Due || v.Err != nil || v.Archive != "" {
+		t.Fatalf("a tier with no archive is UNKNOWN — not due, not an error; got %+v", v)
+	}
+	if v.Reason == "" {
+		t.Fatal("every verdict must carry a reason — a due-check that cannot say why is a quiet path")
+	}
+	h.s.tick(context.Background())
+	if n := len(h.rr.seen()); n != 0 {
+		t.Fatalf("a newborn tier must not be restore-tested; got %d run(s)", n)
+	}
+}
+
+// An archive that exists but has NOT settled yet is not a candidate — and that is not an error.
+func TestDue_UnsettledArchiveIsNotACandidate(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"local": {{volid: "local:backup/fresh.tar.zst", landed: day0}}}}
+	h := newDueHarness(t, day0.Add(2*time.Hour), 24*time.Hour, true, []string{"local"}, ts)
+
+	if v := h.s.EvaluateDue(context.Background())[0]; v.Due || v.Archive != "" {
+		t.Fatalf("an archive 2h old must not be a candidate under a 24h settle lag; got %+v", v)
+	}
+	h.s.tick(context.Background())
+	if n := len(h.rr.seen()); n != 0 {
+		t.Fatalf("nothing settled → no run; got %d", n)
+	}
+}
+
+// A tier whose archives cannot be LISTED is UNKNOWN — never silently "not due", and never silent.
+// Treating a lookup failure as "not due" would retire a tier the moment its storage stopped
+// answering, which is the same absence-is-not-evidence error this monitor family keeps making.
+func TestDue_LookupFailureIsUnknownNotNotDue(t *testing.T) {
+	boom := errors.New("storage unreachable")
+	ts := &tierStorage{
+		archives: map[string][]archiveStub{"local": {{volid: "local:backup/a.tar.zst", landed: day0}}},
+		err:      map[string]error{"felhom-pbs": boom},
+	}
+	h := newDueHarness(t, day0.AddDate(0, 0, 1), 24*time.Hour, true, []string{"local", "felhom-pbs"}, ts)
+
+	var pbs DueVerdict
+	for _, v := range h.s.EvaluateDue(context.Background()) {
+		if v.Target == "felhom-pbs" {
+			pbs = v
+		}
+	}
+	if pbs.Err == nil {
+		t.Fatal("a lookup failure must travel in the verdict, not be swallowed")
+	}
+	if pbs.Due {
+		t.Fatal("a tier we could not list must not be reported DUE — we have no archive to test")
+	}
+	if pbs.Reason == "" {
+		t.Fatal("the failure must be explained, not merely flagged")
+	}
+
+	// And the OTHER tier still runs: one tier's storage being unreadable must not cost the other
+	// tier its proof.
+	h.s.tick(context.Background())
+	if got := h.rr.seen(); len(got) != 1 || got[0] != "local:backup/a.tar.zst" {
+		t.Fatalf("the readable tier must still be proved; got %v", got)
+	}
+}
+
+// ── the state's migration (§8.2) ─────────────────────────────────────────────────────────────
+
+// A pre-R-86 state file carries a TIME and no archive. It must keep its time (rotation ordering
+// survives the upgrade) and yield NO proven archive, so each tier is due exactly once. Reading a
+// legacy time as proof of the CURRENT archive would mark an unproven archive proven — a guarantee
+// invented by a migration.
+func TestRestoreTestState_LegacyFileMigratesToNothingProven(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "rt.json")
+	legacy := `{"local":"2026-08-01T02:00:00Z","felhom-pbs":"2026-07-30T02:00:00Z"}`
+	if err := writeFileForTest(path, legacy); err != nil {
+		t.Fatal(err)
+	}
+
+	st := NewRestoreTestState(path)
+	if _, ok := st.ProvenArchive("local"); ok {
+		t.Fatal("a legacy record names no archive — it must NOT be read as proof of the current one")
+	}
+	at, ok := st.LastSuccess("local")
+	if !ok || !at.Equal(time.Date(2026, 8, 1, 2, 0, 0, 0, time.UTC)) {
+		t.Fatalf("the legacy TIME must survive (rotation ordering depends on it); got %v ok=%v", at, ok)
+	}
+	// Ordering still works off the legacy times.
+	if got := st.OldestFirst([]string{"local", "felhom-pbs"}); got[0] != "felhom-pbs" {
+		t.Fatalf("oldest-first must still order legacy records; got %v", got)
+	}
+}
+
+// The new shape round-trips, archive and all.
+func TestRestoreTestState_ArchiveRoundTrips(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "rt.json")
+	now := time.Now().UTC().Truncate(time.Second)
+	st := NewRestoreTestState(path)
+	if err := st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/x", "pbs", "boot+running", now); err != nil {
+		t.Fatal(err)
+	}
+	re := NewRestoreTestState(path)
+	got, ok := re.ProvenArchive("felhom-pbs")
+	if !ok || got != "felhom-pbs:backup/ct/9201/x" {
+		t.Fatalf("the proven ARCHIVE must survive a restart; got %q ok=%v", got, ok)
+	}
+	at, ok := re.LastSuccess("felhom-pbs")
+	if !ok || !at.Equal(now) {
+		t.Fatalf("the proven TIME must survive too; got %v ok=%v", at, ok)
+	}
+}
+
+// writeFileForTest is a tiny helper so the legacy-migration fixture reads clearly above.
+func writeFileForTest(path, content string) error {
+	return os.WriteFile(path, []byte(content), 0o600)
+}
+
+// Standing rule 3: an absent log line is not evidence. "Nothing is due" is now the NORMAL outcome of
+// an evaluation, so it must produce a POSITIVE observable naming each tier's verdict — otherwise a
+// quiet journal is equally consistent with a healthy loop and a dead goroutine.
+//
+// COMPANION RED-PROOF (observed 2026-08-03): drop the summary back to a bare
+// `s.logger.Debug("backup: restore-test not due this evaluation")` and this fails with
+// "a not-due evaluation must name each tier's verdict; got \"\"" — i.e. nothing at INFO at all.
+func TestDue_NothingDueStillNamesEveryTiersVerdict(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{
+		"local":      {{volid: "local:backup/a.tar.zst", landed: day0}},
+		"felhom-pbs": nil, // no archive at all
+	}}
+	h := newDueHarness(t, day0.AddDate(0, 0, 1), 24*time.Hour, true, []string{"local", "felhom-pbs"}, ts)
+	// Prove the local tier so NOTHING is due.
+	if err := h.st.RecordSuccess("local", "local:backup/a.tar.zst", "local", "boot+running", h.clock); err != nil {
+		t.Fatal(err)
+	}
+
+	// Assert what the SCHEDULER emits on a real evaluation, not what a helper returns — a helper
+	// test would pass against a tick that never calls it.
+	var logbuf strings.Builder
+	h.s.logger = slog.New(slog.NewTextHandler(&logbuf, &slog.HandlerOptions{Level: slog.LevelInfo}))
+	h.s.tick(context.Background())
+	got := logbuf.String()
+	for _, want := range []string{"local", "felhom-pbs", "already proven", "no settled archive"} {
+		if !strings.Contains(got, want) {
+			t.Fatalf("a not-due evaluation must name each tier's verdict; got %q (missing %q)", got, want)
+		}
+	}
+}
+
+// A tier whose storage cannot be listed must say UNKNOWN in that same line — a lookup failure that
+// reads as "nothing due" is the silence this rule exists to prevent.
+func TestDue_VerdictSummaryNamesAnUnknownTier(t *testing.T) {
+	ts := &tierStorage{
+		archives: map[string][]archiveStub{"local": nil},
+		err:      map[string]error{"felhom-pbs": errors.New("storage unreachable")},
+	}
+	h := newDueHarness(t, day0, 24*time.Hour, true, []string{"local", "felhom-pbs"}, ts)
+	got := h.s.verdictSummary(context.Background())
+	if !strings.Contains(got, "UNKNOWN") || !strings.Contains(got, "storage unreachable") {
+		t.Fatalf("an unlistable tier must read as UNKNOWN with its error; got %q", got)
+	}
+}
+
+// ── R-189 — the persisted proof must be REPORTABLE, and must refuse to lie ───────────────────
+//
+// A proof held only in the in-memory store dies with the process, and under per-archive due-ness the
+// agent will not repeat the work. So the persisted record has to be able to become a host-report
+// entry — without inventing anything it does not know.
+//
+// COMPANION RED-PROOF (observed 2026-08-03): drop the `reportable()` filter from
+// ProvenRestoreTests, so a pre-R-189 record (archive but no tier) is emitted →
+//
+//	--- FAIL: TestProvenRestoreTests_RefusesToReportWhatItCannotDescribe
+//	    restoretest_due_test.go: a record with no TIER must not be reported (the hub keys its
+//	    per-tier proof on it); got [{... SourceTier: ...}]
+//
+// Restored.
+func TestProvenRestoreTests_RefusesToReportWhatItCannotDescribe(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "rt.json")
+	// v1 (a bare time), v2 (archive, no tier) and v3 (complete) side by side — every shape this
+	// file has ever had, which is what a real box carries after two upgrades.
+	legacy := `{
+	  "old-v1":     "2026-07-30T02:11:07Z",
+	  "old-v2":     {"archive":"felhom-backup:backup/vzdump-lxc-9201-a.tar.zst","proven_at":"2026-08-01T04:41:58Z"},
+	  "felhom-pbs": {"archive":"felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z","tier":"pbs","verified":"boot+running","proven_at":"2026-08-03T13:25:14Z"}
+	}`
+	if err := writeFileForTest(path, legacy); err != nil {
+		t.Fatal(err)
+	}
+
+	got := NewRestoreTestState(path).ProvenRestoreTests(context.Background())
+	if len(got) != 1 {
+		t.Fatalf("only the record that can be described honestly may be reported; got %d: %+v", len(got), got)
+	}
+	e := got[0]
+	if e.SourceTier != "pbs" {
+		t.Fatalf("a record with no TIER must not be reported (the hub keys its per-tier proof on it); got %+v", got)
+	}
+	if e.SourceArchive != "felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z" || !e.Pass {
+		t.Fatalf("the reported entry must be the stored proof, unchanged; got %+v", e)
+	}
+	if e.TestedAt != "2026-08-03T13:25:14Z" {
+		t.Fatalf("the entry must carry the time the run passed, not now(); got %q", e.TestedAt)
+	}
+	if e.Verified != "boot+running" {
+		t.Fatalf("what the run verified must survive the round trip; got %q", e.Verified)
+	}
+	// Run mechanics are NOT invented: an absent duration is not a claim, a fabricated one would be.
+	if e.DurationSeconds != 0 || e.ScratchVMID != 0 {
+		t.Fatalf("the re-report must not invent run mechanics it never stored; got duration=%v scratch=%d",
+			e.DurationSeconds, e.ScratchVMID)
+	}
+	// The legacy records still serve the DUE-check, which is a separate question from reporting.
+	if _, ok := NewRestoreTestState(path).ProvenArchive("old-v2"); !ok {
+		t.Fatal("a v2 record must still answer the due-check even though it cannot be reported")
+	}
+}
+
+// A tier proved through the SCHEDULER (not by hand) lands in the state complete enough to report —
+// the production path, not a hand-built fixture.
+func TestScheduler_ProofIsRecordedReportably(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"felhom-pbs": {{volid: "felhom-pbs:backup/ct/9201/w0", landed: day0}}}}
+	h := newDueHarness(t, day0.AddDate(0, 0, 1).Add(97*time.Minute), 24*time.Hour, true, []string{"felhom-pbs"}, ts)
+	// The fake runner echoes the spec's tier; give the spec a tier the way main.go does.
+	h.s.spec = func(_ context.Context, archive string) reconcile.RestoreTestSpec {
+		return reconcile.RestoreTestSpec{RestoreStorage: "local-lvm", ScratchMin: 990000, ScratchMax: 990009, SourceTier: "pbs"}
+	}
+	h.s.tick(context.Background())
+
+	got := h.st.ProvenRestoreTests(context.Background())
+	if len(got) != 1 {
+		t.Fatalf("a scheduled pass must leave a REPORTABLE proof; got %d: %+v", len(got), got)
+	}
+	if got[0].SourceTier != "pbs" || got[0].SourceArchive != "felhom-pbs:backup/ct/9201/w0" {
+		t.Fatalf("the proof must name the tier and the archive the run used; got %+v", got[0])
+	}
+}
+
+// A FAILED run leaves nothing to report — the asymmetry of §8.1, asserted rather than assumed.
+func TestScheduler_AFailureLeavesNoPersistedProof(t *testing.T) {
+	ts := &tierStorage{archives: map[string][]archiveStub{"felhom-pbs": {{volid: "felhom-pbs:backup/ct/9201/w0", landed: day0}}}}
+	h := newDueHarness(t, day0.AddDate(0, 0, 1).Add(97*time.Minute), 24*time.Hour, false, []string{"felhom-pbs"}, ts)
+	h.s.tick(context.Background())
+	if got := h.st.ProvenRestoreTests(context.Background()); len(got) != 0 {
+		t.Fatalf("a FAILED run must persist nothing — a failing tier is retried, and a stored failure "+
+			"would outlive the fault; got %+v", got)
+	}
+}

internal/backup/restoretest_state.go

@@ -0,0 +1,289 @@
+package backup
+
+import (
+	"context"
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"sort"
+	"sync"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+)
+
+// RestoreTestState persists the last SUCCESSFUL restore-test per backup tier.
+//
+// R-85 (1.4). This one genuinely needs PERSISTENCE, unlike R-84 — and the difference is worth
+// stating, because the two look like the same problem and are not:
+//
+//   - R-84 (backup freshness) had a GROUND TRUTH to consult: the archive is still on the storage,
+//     so the agent could ask "when did a backup last land?" and never persist anything. That is
+//     strictly better, because a pruned archive correctly stops counting.
+//   - A restore-test leaves NO artifact — the scratch guest is destroyed as its final act. There is
+//     nothing to query. "Did we prove this tier restores?" exists only as remembered state, so it
+//     must be written down or it is lost.
+//
+// Why it must survive a restart: rotation is oldest-first (the operator ruling), so an in-memory map
+// would reset every tier to "never tested" on each restart. Ordering would then depend on map
+// iteration order, and one tier could be starved indefinitely while the other is re-tested — with
+// agent deploys as routine as they are, that is not a corner case.
+//
+// Only SUCCESS is recorded. A failed run must not satisfy rotation, or a tier that fails every time
+// would look freshly proven and stop being retried — the same "a failure satisfies the cadence"
+// trap the backup due-check avoids. R-86 keeps that property unchanged and gives it a second job:
+// the due-check reads this state, so a failure that recorded proof would ALSO stop the tier from
+// ever becoming due again. The rule earns its keep twice now.
+//
+// R-86 (1.2) — WHICH ARCHIVE, not just when.
+//
+// A timestamp alone cannot answer the question the due-check asks. "This tier passed at 04:00" is
+// consistent both with "yesterday's archive is proven" and with "an archive from a week ago is
+// proven and nothing since has been looked at". Restore-testing is now per ARCHIVE GENERATION —
+// a tier is due once it holds a settled archive that has not been proven — so the identity of the
+// proven archive is the state, and the time is metadata (rotation ordering, operator reporting).
+//
+// This is the same class as the workspace rule "a timestamp records an ATTEMPT, not a RESULT":
+// here it records a result, but not WHICH result, and that is just as unable to answer the question
+// being asked of it.
+type RestoreTestState struct {
+	path string
+	mu   sync.Mutex
+	last map[string]provenTier // target id → what was last PROVEN on that tier
+}
+
+// provenTier is one tier's proof: the archive that passed, which tier it was, what was verified,
+// and when.
+//
+// R-189 added `Tier` and `Verified`. Until then this record could answer the DUE-check but could not
+// be REPORTED, and being reportable is what closes R-189: a proof held only in the in-memory result
+// store vanishes on restart, and under per-archive due-ness the box will not repeat the work, so the
+// hub can stay ignorant of a real success until the next archive generation.
+//
+// `Tier` is stored rather than derived because it is known for certain at proof time (the run's own
+// spec used it to choose the restore timeout) and deriving it later would need a storage-type lookup
+// at report-building time — a network call that can fail, on a path where failing means mis-labelling
+// a proof. Store what you knew when you knew it.
+type provenTier struct {
+	Archive  string    // volid of the archive that PASSED; "" = a legacy record with no archive
+	Tier     string    // "local" | "pbs" — as the run reported it; "" = pre-R-189 record
+	Verified string    // what the run verified (e.g. "boot+running"); "" = pre-R-189 record
+	At       time.Time // when that run passed (UTC)
+}
+
+// reportable reports whether this record can be re-reported to the hub as a restore-test result.
+//
+// It needs BOTH the archive and the tier: the hub keys its edge-triggered failure state on the
+// archive and its per-tier proof lookup on the tier, so an entry missing either is not a usable
+// proof — and emitting one anyway would be a report the hub cannot act on, dressed as evidence.
+// A pre-R-189 record is therefore silently not reported; the tier's next real proof fills it in.
+func (p provenTier) reportable() bool { return p.Archive != "" && p.Tier != "" }
+
+// provenTierJSON is the on-disk shape. Two older shapes are read and neither is written:
+//
+//	v1 (pre-R-86)  "<target>": "<RFC3339>"                     — a time, no archive
+//	v2 (R-86)      "<target>": {archive, proven_at}            — due-check usable, not reportable
+//	v3 (R-189)     "<target>": {archive, tier, verified, …}    — both
+//
+// Fields absent in an older file unmarshal to "", which is exactly the "no usable proof" signal the
+// readers above test for — the migration needs no version number because the absence IS the answer.
+type provenTierJSON struct {
+	Archive  string `json:"archive"`
+	Tier     string `json:"tier,omitempty"`
+	Verified string `json:"verified,omitempty"`
+	ProvenAt string `json:"proven_at"`
+}
+
+// NewRestoreTestState opens (or creates) the state at path. A missing or unreadable file is NOT an
+// error: it degrades to "nothing proven yet", which is the correct starting point and keeps a
+// corrupt file from wedging the daemon.
+//
+// MIGRATION (R-86). The pre-R-86 file is `{"<target>": "<RFC3339>"}` — a time and no archive. A
+// legacy record keeps its TIME (rotation ordering survives a deploy, which is why the file exists
+// at all) but yields NO proven archive, so every tier is due exactly once on first evaluation after
+// the upgrade. One extra restore-test per tier, once, is the safe direction: the alternative is to
+// read a legacy time as proof of whatever archive happens to be current, which would mark an
+// unproven archive proven — inventing a guarantee out of a migration.
+func NewRestoreTestState(path string) *RestoreTestState {
+	s := &RestoreTestState{path: path, last: map[string]provenTier{}}
+	data, err := os.ReadFile(path)
+	if err != nil {
+		return s
+	}
+	var raw map[string]json.RawMessage
+	if json.Unmarshal(data, &raw) != nil {
+		return s
+	}
+	for target, msg := range raw {
+		// Legacy shape: a bare RFC3339 string.
+		var legacy string
+		if json.Unmarshal(msg, &legacy) == nil {
+			if t, perr := time.Parse(time.RFC3339, legacy); perr == nil {
+				s.last[target] = provenTier{At: t.UTC()} // no archive → due once, deliberately
+			}
+			continue
+		}
+		var cur provenTierJSON
+		if json.Unmarshal(msg, &cur) != nil {
+			continue // one unreadable entry must not lose the others
+		}
+		t, perr := time.Parse(time.RFC3339, cur.ProvenAt)
+		if perr != nil {
+			continue
+		}
+		s.last[target] = provenTier{Archive: cur.Archive, Tier: cur.Tier, Verified: cur.Verified, At: t.UTC()}
+	}
+	return s
+}
+
+// RecordSuccess stamps a tier as proven at t, naming the ARCHIVE that passed, the TIER the run
+// reported, and what it verified. Only call this for a PASSING restore-test — the archive is what
+// makes the tier not-due, so recording one for a failed run would retire the archive unproven.
+//
+// ONLY SUCCESSES ARE PERSISTED, AND THE ASYMMETRY IS DELIBERATE (R-189 §8.1). Say it here because
+// the next reader will notice failures are absent and try to "fix" it:
+//
+//	a SUCCESS suppresses future work — a proven archive is never re-tested, so a lost proof leaves
+//	the system quietly less tested than it believes. It must survive a restart.
+//
+//	a FAILURE causes future work — a failing tier stays due and is retried at the next evaluation,
+//	so a lost failure heals itself within one interval. Persisting it would do the opposite of
+//	helping: a healed tier would keep reporting a failure that is no longer true.
+func (s *RestoreTestState) RecordSuccess(target, archive, tier, verified string, t time.Time) error {
+	if target == "" {
+		return nil
+	}
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.last[target] = provenTier{Archive: archive, Tier: tier, Verified: verified, At: t.UTC()}
+	return s.saveLocked()
+}
+
+// LastSuccess returns when this tier was last proven (ok=false = never).
+func (s *RestoreTestState) LastSuccess(target string) (time.Time, bool) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	p, ok := s.last[target]
+	return p.At, ok
+}
+
+// ProvenArchive returns the archive last PROVEN on this tier (ok=false = none — either never tested,
+// or a legacy record carrying only a time). It is the due-check's whole question: an archive that is
+// not this one has not been proven.
+func (s *RestoreTestState) ProvenArchive(target string) (string, bool) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	p, ok := s.last[target]
+	if !ok || p.Archive == "" {
+		return "", false
+	}
+	return p.Archive, true
+}
+
+// Snapshot returns a copy of the last-proven TIMES.
+//
+// It carried the comment "for the host-report gauge" from the day it was written and **had no caller
+// at all** until R-189 — a seam built and never wired, and an invariant asserted in a comment with
+// nothing pinning it, in one method. The host report is now fed by ProvenRestoreTests below, which
+// carries the archive and the tier that a bare timestamp cannot. This stays for callers that want
+// only the times; if it acquires none, delete it rather than let it claim a purpose again.
+func (s *RestoreTestState) Snapshot() map[string]time.Time {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	out := make(map[string]time.Time, len(s.last))
+	for k, v := range s.last {
+		out[k] = v.At
+	}
+	return out
+}
+
+// ProvenRestoreTests renders the persisted proofs as host-report entries — the R-189 fix.
+//
+// It satisfies hub.RestoreTestReporter's shape, so the collector can merge these with the in-memory
+// results. What it emits is a RE-REPORT of a run that really happened, not a synthesis:
+//
+//   - `Pass` is true because ONLY successes are stored (RecordSuccess is the sole writer);
+//   - `SourceArchive`, `SourceTier`, `Verified` and `TestedAt` are the values that run reported;
+//   - the run mechanics (scratch VMID, duration, warnings) are NOT re-invented. An absent duration
+//     is not a claim; a fabricated one would be.
+//
+// A record that cannot be reported honestly is omitted rather than padded — see provenTier.reportable.
+// **A tier with no usable proof produces NO entry**: an unproven tier reading as proven would be a
+// worse defect than the one this fixes.
+func (s *RestoreTestState) ProvenRestoreTests(context.Context) []hub.RestoreTest {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	out := make([]hub.RestoreTest, 0, len(s.last))
+	for _, p := range s.last {
+		if !p.reportable() {
+			continue
+		}
+		out = append(out, hub.RestoreTest{
+			SourceArchive: p.Archive,
+			SourceTier:    p.Tier,
+			Pass:          true,
+			Verified:      p.Verified,
+			TestedAt:      p.At.UTC().Format(time.RFC3339),
+		})
+	}
+	// Deterministic order: the report is compared byte-wise by the contract test, and Go's map
+	// iteration is randomised.
+	sort.Slice(out, func(i, j int) bool { return out[i].SourceTier < out[j].SourceTier })
+	return out
+}
+
+// OldestFirst orders targets by "least recently proven first"; never-proven sorts FIRST.
+//
+// This is the operator's 2026-07-26 ruling (Option 1): self-balancing, no new config knob, and it
+// naturally prioritises a tier that has never been restore-tested at all — which on this fleet was
+// the offsite tier, unproven for its entire existence.
+//
+// Ties break on target id so the order is deterministic; without that, two tiers proven in the same
+// second would rotate by map iteration order, which is randomised in Go and would make the
+// behaviour untestable and occasionally starving.
+func (s *RestoreTestState) OldestFirst(targets []string) []string {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	out := append([]string(nil), targets...)
+	sort.SliceStable(out, func(i, j int) bool {
+		pi, oki := s.last[out[i]]
+		pj, okj := s.last[out[j]]
+		ti, tj := pi.At, pj.At
+		switch {
+		case !oki && !okj:
+			return out[i] < out[j] // both never proven → deterministic
+		case !oki:
+			return true // never proven wins
+		case !okj:
+			return false
+		case !ti.Equal(tj):
+			return ti.Before(tj)
+		default:
+			return out[i] < out[j]
+		}
+	})
+	return out
+}
+
+func (s *RestoreTestState) saveLocked() error {
+	raw := make(map[string]provenTierJSON, len(s.last))
+	for target, p := range s.last {
+		raw[target] = provenTierJSON{
+			Archive: p.Archive, Tier: p.Tier, Verified: p.Verified,
+			ProvenAt: p.At.UTC().Format(time.RFC3339),
+		}
+	}
+	data, err := json.MarshalIndent(raw, "", "  ")
+	if err != nil {
+		return err
+	}
+	if err := os.MkdirAll(filepath.Dir(s.path), 0o755); err != nil {
+		return err
+	}
+	tmp := s.path + ".tmp"
+	if err := os.WriteFile(tmp, data, 0o600); err != nil {
+		os.Remove(tmp)
+		return err
+	}
+	return os.Rename(tmp, s.path)
+}

internal/backup/rotation_test.go

@@ -0,0 +1,354 @@
+package backup
+
+import (
+	"context"
+	"os"
+	"path/filepath"
+	"sync"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
+)
+
+// R-85 Phase 2 — tier rotation, persisted state, and the one-heavy-operation gate.
+//
+// The failure this prevents is not hypothetical: demo-hp's DR tier reported `applied` with ZERO
+// snapshots for five days and nobody noticed, because the scheduler could only ever see the primary
+// tier. Rotation is what makes the offsite tier testable at all.
+
+// rotRunner records which archives it was asked to restore.
+type rotRunner struct {
+	mu       sync.Mutex
+	archives []string
+	pass     bool
+}
+
+func (r *rotRunner) RunRestoreTest(_ context.Context, spec reconcile.RestoreTestSpec) reconcile.RestoreTestResult {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	r.archives = append(r.archives, spec.Archive)
+	return reconcile.RestoreTestResult{
+		Archive: spec.Archive, SourceTier: spec.SourceTier,
+		Pass: r.pass, Verified: "boot+running",
+	}
+}
+func (r *rotRunner) seen() []string {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	return append([]string(nil), r.archives...)
+}
+
+// testLanded is a landing time old enough to be settled under any cutoff these tests use. R-86
+// widened the TierPicker seam with the archive's landing time; the rotation tests below are about
+// tier ORDER and the heavy-operation gate, not about settling, so they hold it constant.
+var testLanded = time.Date(2026, 7, 1, 0, 0, 0, 0, time.UTC)
+
+// archiveFor is a TierPicker over a fixed map: target → archive ("" = that tier holds none).
+func archiveFor(m map[string]string) TierPicker {
+	return func(_ context.Context, target string, _ time.Time) (string, time.Time, error) {
+		a := m[target]
+		if a == "" {
+			return "", time.Time{}, nil
+		}
+		return a, testLanded, nil
+	}
+}
+
+func rotScheduler(t *testing.T, rr *rotRunner, st *RestoreTestState, pick TierPicker, gate *InFlight) *Scheduler {
+	t.Helper()
+	return NewScheduler(SchedulerOptions{
+		Runner: rr,
+		Store:  NewStore(),
+		Spec: func(_ context.Context, archive string) reconcile.RestoreTestSpec {
+			return reconcile.RestoreTestSpec{RestoreStorage: "local-lvm", ScratchMin: 990000, ScratchMax: 990009}
+		},
+		Cadence:  time.Hour,
+		Logger:   quiet(),
+		Tiers:    []string{"local", "felhom-pbs"},
+		TierPick: pick,
+		State:    st,
+		InFlight: gate,
+	})
+}
+
+// ── SCENARIO A — both tiers get tested, each ONCE per archive ────────────────────────────────
+//
+// R-86 CHANGED THIS TEST'S CONTRACT, deliberately, and the old assertion is worth recording because
+// it was a faithful statement of the defect. It read:
+//
+//	4 ticks → 4 runs, and consecutive runs must hit different tiers
+//
+// i.e. every tick produced a heavy restore-test, because the ticker WAS the trigger. Under R-86 a
+// tick is an EVALUATION: both tiers are still exercised (rotation is intact), but a tier whose
+// newest settled archive is already proven is not re-tested just because time passed. So the
+// assertion is now 2 runs across 4 evaluations — one per tier, one per archive — which is a
+// STRICTLY STRONGER statement: it pins both the coverage R-85 won and the pacing R-86 adds.
+//
+// COMPANION RED-PROOF (observed): restore the single-target picker — set `Tiers`/`TierPick` to nil
+// so `pickForThisRun` falls back to `s.pick` on the primary runner — and this fails with
+// "both tiers must be exercised; got [local:…]", i.e. the offsite tier never appears. That is
+// pre-R-85 behaviour, and it is why demo-hp's DR tier went unproven for its entire existence.
+func TestRotation_BothTiersExercisedOncePerArchive(t *testing.T) {
+	rr := &rotRunner{pass: true}
+	st := NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json"))
+	s := rotScheduler(t, rr, st, archiveFor(map[string]string{
+		"local":      "local:backup/vzdump-lxc-9201-x.tar.zst",
+		"felhom-pbs": "felhom-pbs:backup/ct/9201/2026-07-26T15:42:42Z",
+	}), &InFlight{})
+	s.now = func() time.Time { return time.Now().UTC() }
+
+	for i := 0; i < 4; i++ {
+		s.tick(context.Background())
+	}
+
+	got := rr.seen()
+	var sawLocal, sawPBS bool
+	for _, a := range got {
+		if len(a) >= 5 && a[:5] == "local" {
+			sawLocal = true
+		}
+		if len(a) >= 10 && a[:10] == "felhom-pbs" {
+			sawPBS = true
+		}
+	}
+	if !sawLocal || !sawPBS {
+		t.Fatalf("both tiers must be exercised; got %v", got)
+	}
+	// Exactly one run per tier: the archives never changed, so nothing became due a second time.
+	if len(got) != 2 {
+		t.Fatalf("want 2 runs across 4 evaluations (one per archive generation), got %d: %v", len(got), got)
+	}
+	if got[0] == got[1] {
+		t.Fatalf("the two runs must be different tiers — oldest-first is not ordering due tiers: %v", got)
+	}
+}
+
+// A tier with NO archive is skipped, not failed, and the other tier still runs. A brand-new offsite
+// tier legitimately has nothing to restore; turning that into a failure would make every fresh box
+// look broken for its first week.
+func TestRotation_EmptyTierSkippedNotFailed(t *testing.T) {
+	rr := &rotRunner{pass: true}
+	st := NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json"))
+	s := rotScheduler(t, rr, st, archiveFor(map[string]string{
+		"local":      "local:backup/vzdump-lxc-9201-x.tar.zst",
+		"felhom-pbs": "", // provisioned but empty
+	}), &InFlight{})
+
+	s.tick(context.Background())
+	got := rr.seen()
+	if len(got) != 1 || got[0][:5] != "local" {
+		t.Fatalf("an empty tier must be skipped and the testable one still run; got %v", got)
+	}
+}
+
+// Nothing testable anywhere → a clean no-op, not an error and not a run.
+func TestRotation_NoArchivesAnywhereIsANoOp(t *testing.T) {
+	rr := &rotRunner{pass: true}
+	s := rotScheduler(t, rr, NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json")),
+		archiveFor(map[string]string{}), &InFlight{})
+	s.tick(context.Background())
+	if got := rr.seen(); len(got) != 0 {
+		t.Fatalf("no archives anywhere → no run; got %v", got)
+	}
+}
+
+// A FAILED restore-test must NOT earn rotation credit, or a tier that fails every time would look
+// freshly proven and quietly stop being retried.
+func TestRotation_FailureEarnsNoCredit(t *testing.T) {
+	rr := &rotRunner{pass: false}
+	st := NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json"))
+	s := rotScheduler(t, rr, st, archiveFor(map[string]string{
+		"local":      "local:backup/x.tar.zst",
+		"felhom-pbs": "felhom-pbs:backup/ct/9201/y",
+	}), &InFlight{})
+	s.tick(context.Background())
+	if _, ok := st.LastSuccess("local"); ok {
+		t.Fatal("a FAILED restore-test must not stamp the tier as proven")
+	}
+	if _, ok := st.LastSuccess("felhom-pbs"); ok {
+		t.Fatal("a FAILED restore-test must not stamp the tier as proven")
+	}
+}
+
+// ── SCENARIO E — rotation survives a restart ─────────────────────────────────────────────────
+//
+// COMPANION RED-PROOF (observed): make the state in-memory (construct a fresh
+// `NewRestoreTestState` on a DIFFERENT path for the second scheduler, i.e. lose the file) and this
+// fails with "after a restart the OTHER tier must be next; got felhom-pbs" — the same tier repeats
+// and the other is starved indefinitely, which with agent deploys as routine as they are is not a
+// corner case.
+func TestRotation_SurvivesRestart(t *testing.T) {
+	dir := t.TempDir()
+	path := filepath.Join(dir, "rt.json")
+	picks := archiveFor(map[string]string{
+		"local":      "local:backup/x.tar.zst",
+		"felhom-pbs": "felhom-pbs:backup/ct/9201/y",
+	})
+
+	// First process: the OFFSITE tier is tested (never-proven sorts first).
+	rr1 := &rotRunner{pass: true}
+	st1 := NewRestoreTestState(path)
+	s1 := rotScheduler(t, rr1, st1, picks, &InFlight{})
+	s1.tick(context.Background())
+	first := rr1.seen()
+	if len(first) != 1 {
+		t.Fatalf("want one run, got %v", first)
+	}
+
+	// --- restart: brand-new state object reading the SAME file ---
+	rr2 := &rotRunner{pass: true}
+	st2 := NewRestoreTestState(path)
+	s2 := rotScheduler(t, rr2, st2, picks, &InFlight{})
+	s2.tick(context.Background())
+	second := rr2.seen()
+	if len(second) != 1 {
+		t.Fatalf("want one run after restart, got %v", second)
+	}
+
+	if second[0] == first[0] {
+		t.Fatalf("after a restart the OTHER tier must be next; got %s twice (rotation state was lost)", second[0])
+	}
+}
+
+// ── SCENARIO F — no collision with a backup ──────────────────────────────────────────────────
+//
+// COMPANION RED-PROOF (observed): drop the TryAcquire guard from `tick` and this fails with
+// "the restore-test must DEFER while a backup holds the gate; concurrent operations = 2" — the
+// count is the assertion, since "both completed" would pass against a fully concurrent
+// implementation.
+func TestRotation_DefersWhileABackupHoldsTheGate(t *testing.T) {
+	gate := &InFlight{}
+	release, _, ok := gate.TryAcquire("backup:felhom-pbs")
+	if !ok {
+		t.Fatal("precondition: the gate should have been free")
+	}
+	defer release()
+
+	rr := &rotRunner{pass: true}
+	s := rotScheduler(t, rr, NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json")),
+		archiveFor(map[string]string{"local": "local:backup/x.tar.zst"}), gate)
+
+	s.tick(context.Background())
+
+	concurrent := 1 + len(rr.seen()) // the backup holding the gate, plus anything the tick started
+	if concurrent != 1 {
+		t.Fatalf("the restore-test must DEFER while a backup holds the gate; concurrent operations = %d", concurrent)
+	}
+}
+
+// Once the backup releases, the next cadence proceeds — deferral must not be permanent.
+func TestRotation_ResumesAfterTheGateFrees(t *testing.T) {
+	gate := &InFlight{}
+	release, _, _ := gate.TryAcquire("backup:local")
+
+	rr := &rotRunner{pass: true}
+	s := rotScheduler(t, rr, NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json")),
+		archiveFor(map[string]string{"local": "local:backup/x.tar.zst"}), gate)
+
+	s.tick(context.Background())
+	if len(rr.seen()) != 0 {
+		t.Fatal("should have deferred while held")
+	}
+	release()
+	s.tick(context.Background())
+	if len(rr.seen()) != 1 {
+		t.Fatalf("must resume once the gate frees; got %v", rr.seen())
+	}
+}
+
+// The gate itself: one holder at a time, named, and release is idempotent.
+func TestInFlight_Semantics(t *testing.T) {
+	g := &InFlight{}
+	rel, busy, ok := g.TryAcquire("backup:local")
+	if !ok || busy != "" {
+		t.Fatalf("first acquire must succeed; ok=%v busy=%q", ok, busy)
+	}
+	if _, busy2, ok2 := g.TryAcquire("restore-test"); ok2 || busy2 != "backup:local" {
+		t.Fatalf("second acquire must fail and NAME the holder; ok=%v busy=%q", ok2, busy2)
+	}
+	rel()
+	rel() // idempotent — a double release must not free someone else's later claim
+	if g.Busy() != "" {
+		t.Fatalf("gate should be idle after release; busy=%q", g.Busy())
+	}
+	if _, _, ok3 := g.TryAcquire("restore-test"); !ok3 {
+		t.Fatal("gate must be reusable after release")
+	}
+}
+
+// A nil gate means "not wired" → no gating, pre-R-85 behaviour. Keeps every existing caller working.
+func TestInFlight_NilIsUngated(t *testing.T) {
+	var g *InFlight
+	rel, _, ok := g.TryAcquire("x")
+	if !ok {
+		t.Fatal("a nil gate must not block")
+	}
+	rel()
+	if g.Busy() != "" {
+		t.Fatal("a nil gate is never busy")
+	}
+}
+
+// ── oldest-first ordering ────────────────────────────────────────────────────────────────────
+
+func TestOldestFirst_Ordering(t *testing.T) {
+	st := NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json"))
+	now := time.Now().UTC()
+
+	// Never-proven sorts FIRST — the case that matters, since the offsite tier starts there.
+	if got := st.OldestFirst([]string{"local", "felhom-pbs"}); got[0] != "felhom-pbs" {
+		// both never proven → deterministic tie-break by id
+		if got[0] != "felhom-pbs" && got[0] != "local" {
+			t.Fatalf("unexpected: %v", got)
+		}
+	}
+	_ = st.RecordSuccess("local", "local:backup/a.tar.zst", "local", "boot+running", now)
+	if got := st.OldestFirst([]string{"local", "felhom-pbs"}); got[0] != "felhom-pbs" {
+		t.Fatalf("a never-proven tier must sort before a proven one; got %v", got)
+	}
+	_ = st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/b", "pbs", "boot+running", now.Add(time.Hour))
+	if got := st.OldestFirst([]string{"local", "felhom-pbs"}); got[0] != "local" {
+		t.Fatalf("the least recently proven must sort first; got %v", got)
+	}
+}
+
+// Ordering must be DETERMINISTIC for equal timestamps, or two tiers proven in the same second would
+// rotate by Go's randomised map iteration — untestable, and occasionally starving.
+func TestOldestFirst_DeterministicOnTies(t *testing.T) {
+	st := NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json"))
+	now := time.Now().UTC()
+	_ = st.RecordSuccess("b-tier", "b:archive", "local", "boot+running", now)
+	_ = st.RecordSuccess("a-tier", "a:archive", "local", "boot+running", now)
+	for i := 0; i < 20; i++ {
+		if got := st.OldestFirst([]string{"b-tier", "a-tier"}); got[0] != "a-tier" {
+			t.Fatalf("tie-break must be deterministic; iteration %d gave %v", i, got)
+		}
+	}
+}
+
+// The state file round-trips, and a corrupt file degrades to "nothing proven" rather than wedging.
+func TestRestoreTestState_PersistenceAndCorruption(t *testing.T) {
+	dir := t.TempDir()
+	path := filepath.Join(dir, "rt.json")
+	now := time.Now().UTC().Truncate(time.Second)
+
+	st := NewRestoreTestState(path)
+	if err := st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/x", "pbs", "boot+running", now); err != nil {
+		t.Fatal(err)
+	}
+	reopened := NewRestoreTestState(path)
+	got, ok := reopened.LastSuccess("felhom-pbs")
+	if !ok || !got.Equal(now) {
+		t.Fatalf("state must round-trip; got %v ok=%v want %v", got, ok, now)
+	}
+
+	bad := filepath.Join(dir, "corrupt.json")
+	if err := os.WriteFile(bad, []byte("{{{not json"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	c := NewRestoreTestState(bad)
+	if _, ok := c.LastSuccess("felhom-pbs"); ok {
+		t.Fatal("a corrupt state file must degrade to 'nothing proven', not invent a timestamp")
+	}
+}

internal/backup/runner.go

@@ -5,7 +5,9 @@ import (
 	"fmt"
 	"log/slog"
 	"sort"
+	"strconv"
 	"strings"
+	"sync"
 	"time"
 
 	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
@@ -21,6 +23,10 @@ type BackupAPI interface {
 	StorageContent(ctx context.Context, store string) ([]proxmox.StorageContent, error)
 	// ListStorage enumerates storages (name+type) — used to scope local-only retention (never prune PBS).
 	ListStorage(ctx context.Context) ([]proxmox.Storage, error)
+	// NodeStorage is GET /nodes/{node}/storage — the storages WITH live usage (avail/used). R-685's space
+	// preflight reads free space HERE: ListStorage (GET /storage) is the cluster DEFINITIONS and carries no
+	// usage at all — measured live 2026-09-24 on demo-hp, where reading it let a backup through.
+	NodeStorage(ctx context.Context) ([]proxmox.Storage, error)
 	// TaskLogTail reads trailing task-log lines — used to read the ACTUAL vzdump mode
 	// (PVE may downgrade a requested snapshot to stop for a stopped guest — spike B1).
 	TaskLogTail(ctx context.Context, upid string, limit int) ([]string, error)
@@ -38,21 +44,57 @@ type BackupRunner struct {
 	// each successful backup, so the agent's own backups can't pile up and refill root. Empty → no prune
 	// (the legacy behaviour; restore-test/selftest runners pass ""). NEVER applied to a PBS target.
 	retention string
-	logger    *slog.Logger
-	now       func() time.Time
+	// waitTimeout bounds the WaitTask poll on this runner's vzdump. Per-TIER since R-82: 30m is
+	// right for a local vzdump and badly wrong for an offsite PBS upload (see the 2026-07-26 live
+	// failure recorded on config.BackupTargetConfig.WaitTimeoutSeconds). 0 → 30m (legacy).
+	waitTimeout time.Duration
+	// allowPBSPrune permits `--prune-backups` on a PBS-type target. OFF by default and ON only for
+	// an ADDITIONAL tier whose keep_last was set explicitly (operator ruling 2026-07-26: keep two
+	// weeks of weekly offsite backups).
+	//
+	// The blanket PBS refusal it replaces existed for a real reason and still applies to the
+	// PRIMARY tier: BackupTarget() DEFAULTS to "felhom-pbs" and KeepLast() DEFAULTS to 3, so a box
+	// with neither key set would silently prune its offsite DR to 3 restore points. An additional
+	// tier cannot have that accident — its keep_last defaults to 0 (never prune), so any value
+	// there is a deliberate act.
+	allowPBSPrune bool
+	logger        *slog.Logger
+	now           func() time.Time
+	// rejected remembers the volids already announced by warnRejectedArchiveOnce, so an incomplete
+	// archive is reported ONCE rather than on every 5-minute due-check. Bounded in practice: one
+	// entry per aborted upload, and a process restart clears it. Guarded by rejectedMu because the
+	// due-check is served from the local-API handler goroutines.
+	rejectedMu sync.Mutex
+	rejected   map[string]struct{}
 }
 
 // NewBackupRunner builds a runner. mode defaults to snapshot (works for a stopped guest and
 // for lvm-thin); the caller may pass ModeStop for storages without snapshot support. retention is the
 // per-run prune spec ("keep-last=N", or "" to never prune) — only the periodic local backup sets it.
 func NewBackupRunner(api BackupAPI, target string, mode proxmox.BackupMode, notes, retention string, logger *slog.Logger) *BackupRunner {
+	return NewBackupRunnerWithWait(api, target, mode, notes, retention, 0, logger)
+}
+
+// NewBackupRunnerWithWait is NewBackupRunner plus an explicit vzdump wait bound (0 → 30m).
+func NewBackupRunnerWithWait(api BackupAPI, target string, mode proxmox.BackupMode, notes, retention string, waitTimeout time.Duration, logger *slog.Logger) *BackupRunner {
+	return NewBackupRunnerFull(api, target, mode, notes, retention, waitTimeout, false, logger)
+}
+
+// NewBackupRunnerFull is the full constructor. allowPBSPrune must be true ONLY for an additional
+// tier with an explicitly configured keep_last — see BackupRunner.allowPBSPrune.
+func NewBackupRunnerFull(api BackupAPI, target string, mode proxmox.BackupMode, notes, retention string, waitTimeout time.Duration, allowPBSPrune bool, logger *slog.Logger) *BackupRunner {
 	if mode == "" {
 		mode = proxmox.ModeSnapshot
 	}
 	if logger == nil {
 		logger = slog.Default()
 	}
-	return &BackupRunner{api: api, target: target, mode: mode, notes: notes, retention: retention, logger: logger, now: func() time.Time { return time.Now().UTC() }}
+	if waitTimeout <= 0 {
+		waitTimeout = 30 * time.Minute
+	}
+	return &BackupRunner{api: api, target: target, mode: mode, notes: notes, retention: retention,
+		waitTimeout: waitTimeout, allowPBSPrune: allowPBSPrune, logger: logger,
+		now: func() time.Time { return time.Now().UTC() }}
 }
 
 // localPruneSpec returns the `--prune-backups` spec to apply to THIS backup, or "" to skip pruning. It
@@ -71,8 +113,11 @@ func (r *BackupRunner) localPruneSpec(ctx context.Context) string {
 	}
 	for _, s := range stores {
 		if s.Storage == r.target {
-			if s.Type == "pbs" {
-				return "" // PBS retention is out of scope — never prune the offsite DR
+			if s.Type == "pbs" && !r.allowPBSPrune {
+				// Not opted in → never prune the offsite DR (the pre-R-82 rule, and still the rule
+				// for the primary tier, whose target+retention both DEFAULT and could prune by
+				// accident).
+				return ""
 			}
 			return r.retention
 		}
@@ -130,6 +175,16 @@ func (r *BackupRunner) backup(ctx context.Context, vmid int, onSnapshot func())
 		rec.UncoveredVolumes = []string{}
 	}
 
+	// R-685 (v0.134.0): will the new archive FIT on a local target? Asked before anything runs, so a
+	// target that cannot hold it is a named SKIP with the numbers, not a nightly "No space left on
+	// device" that only the vzdump log explains (demo-hp, every night from 2026-09-23 — R-684).
+	if ok, why := r.spaceFits(ctx, vmid); !ok {
+		rec.Error = BackupSkipNoSpacePrefix + why
+		rec.DurationSeconds = time.Since(start).Seconds()
+		r.logger.Warn("backup SKIPPED by the space preflight (R-685) — nothing was started", "vmid", vmid, "target", r.target, "reason", why)
+		return rec, fmt.Errorf("backup: %s", rec.Error)
+	}
+
 	upid, err := r.api.Vzdump(ctx, proxmox.VzdumpOptions{
 		VMID: vmid, Storage: r.target, Mode: r.mode, Notes: r.notes,
 		PruneBackups: r.localPruneSpec(ctx), // local target → keep-last=N; PBS/unknown → "" (no prune)
@@ -147,7 +202,7 @@ func (r *BackupRunner) backup(ctx context.Context, vmid int, onSnapshot func())
 			defer stopWatch()
 			go r.watchForSnapshot(watchCtx, upid, onSnapshot)
 		}
-		if _, err := r.api.WaitTask(ctx, upid, proxmox.WaitOptions{Timeout: 30 * time.Minute}); err != nil {
+		if _, err := r.api.WaitTask(ctx, upid, proxmox.WaitOptions{Timeout: r.waitTimeout}); err != nil {
 			rec.Error = err.Error()
 			rec.DurationSeconds = time.Since(start).Seconds()
 			return rec, fmt.Errorf("backup: vzdump task vmid %d: %w", vmid, err)
@@ -177,6 +232,56 @@ func (r *BackupRunner) backup(ctx context.Context, vmid int, onSnapshot func())
 	return rec, nil
 }
 
+// BackupSkipNoSpacePrefix starts a backup record's Error when the space preflight refused (R-685) — a
+// stable prefix the controller's page and the hub can key on.
+const BackupSkipNoSpacePrefix = "skipped: not enough space: "
+
+// Space preflight margins (R-685): the new archive is predicted as the newest archive of this guest on the
+// target × backupSpaceGrowth, plus backupSpaceFloorBytes of headroom for the host. MEASURED 2026-09-24:
+// demo-hp 9201's archives grew 5.8 → 6.2 → 6.9 → 7.6 GB in four nights (+10 % a night at worst), so 1.25
+// covers two nights' growth. PVE prunes old archives only AFTER a successful backup, so the free space
+// must hold the new archive while every kept one still exists.
+const (
+	backupSpaceGrowth     = 1.25
+	backupSpaceFloorBytes = int64(1) << 30
+)
+
+// spaceFits answers whether a new archive of vmid fits on a LOCAL (non-PBS) target. It FAILS OPEN — a
+// backup is the thing being protected, so an unreadable storage, an unknown type or a first backup (no
+// previous archive to size from) proceeds and says so; only a POSITIVE "it does not fit" refuses.
+func (r *BackupRunner) spaceFits(ctx context.Context, vmid int) (bool, string) {
+	// NodeStorage, never ListStorage: only the node view carries avail (see BackupAPI.NodeStorage).
+	stores, err := r.api.NodeStorage(ctx)
+	if err != nil {
+		r.logger.Warn("backup: space preflight could not read storage usage — proceeding (fail-open)", "target", r.target, "err", err)
+		return true, ""
+	}
+	var st *proxmox.Storage
+	for i := range stores {
+		if stores[i].Storage == r.target {
+			st = &stores[i]
+			break
+		}
+	}
+	if st == nil || st.Type == "pbs" || st.Avail <= 0 {
+		return true, "" // PBS dedups and has its own lifecycle; an unknown avail never refuses
+	}
+	_, last, err := r.latestArchive(ctx, vmid)
+	if err != nil || last <= 0 {
+		r.logger.Info("backup: space preflight has no previous archive to size from — proceeding", "vmid", vmid, "target", r.target)
+		return true, ""
+	}
+	need := int64(float64(last)*backupSpaceGrowth) + backupSpaceFloorBytes
+	if st.Avail >= need {
+		r.logger.Info("backup: space preflight passed", "vmid", vmid, "target", r.target, "last_archive_bytes", last, "need_bytes", need, "avail_bytes", st.Avail)
+		return true, ""
+	}
+	return false, fmt.Sprintf("%s has %s free; the last archive of guest %d was %s, so a new one needs about %s (old archives are removed only after a successful backup)",
+		r.target, humanGiB(st.Avail), vmid, humanGiB(last), humanGiB(need))
+}
+
+func humanGiB(b int64) string { return fmt.Sprintf("%.1f GiB", float64(b)/(1<<30)) }
+
 // watchForSnapshot polls the running backup's task log until it sees the storage-snapshot marker
 // (→ onSnapshot once) or the requested mode is reported as `stop` (→ downgraded; the marker will
 // never come, so stop watching) or ctx is cancelled (backup finished). Best-effort: a log-read
@@ -210,18 +315,115 @@ func (r *BackupRunner) watchForSnapshot(ctx context.Context, upid string, onSnap
 // PickRestoreCandidate returns the newest backup archive on the target (any guest), or ""
 // when there is none — the restore-test then no-ops cleanly.
 func (r *BackupRunner) PickRestoreCandidate(ctx context.Context) (string, error) {
-	contents, err := r.api.StorageContent(ctx, r.target)
+	return r.PickRestoreCandidateOn(ctx, r.target)
+}
+
+// PickRestoreCandidateOn is PickRestoreCandidate for an ARBITRARY tier's storage (R-85 1.2), so the
+// scheduler can rotate across tiers instead of only ever seeing this runner's own target.
+//
+// Contract preserved: "" + nil error when the storage holds no archive. **A tier with nothing to
+// restore is not an error** — a brand-new offsite tier legitimately has nothing yet, and turning
+// that into a failure would make every fresh box look broken for its first week.
+func (r *BackupRunner) PickRestoreCandidateOn(ctx context.Context, target string) (string, error) {
+	archive, _, err := r.PickSettledRestoreCandidateOn(ctx, target, time.Time{})
+	return archive, err
+}
+
+// PickSettledRestoreCandidateOn is the R-86 due-check's picker: the newest archive on target that
+// landed AT OR BEFORE notAfter (the settle cutoff), with the time it landed. A zero notAfter means
+// "no cutoff" — that is the pre-R-86 behaviour, which is why PickRestoreCandidateOn is now a
+// one-line call into this and its contract is untouched (one scan, one owner).
+//
+// WHY A CUTOFF AT ALL. An archive that landed minutes ago may still be settling — R-71a's
+// settle-gate exists because the offsite tier's day-0 consume raced its own floor update — and
+// restore-testing the archive a backup is still writing proves nothing about the backup that
+// finished. The due-check therefore asks about the newest SETTLED archive, and §8.1's rule is built
+// on that: the tier is due when a settled archive exists that has not been proven.
+//
+// The plausibility floor is applied here and not in the old path on purpose. Under R-86 the picked
+// archive becomes the tier's due-ness: an incomplete 1-byte phantom (F-CRIT-2's artefact — server
+// prune does NOT collect it) would be selected forever, fail its restore forever, never earn proof,
+// and so make the tier due at EVERY evaluation. Skipping it is what keeps the retry rate bounded by
+// the archive generation rather than by the evaluation interval.
+//
+// Contract preserved: ("", zero, nil) when the storage holds no eligible archive. **A tier with
+// nothing to restore is not an error** — a brand-new offsite tier legitimately has nothing yet, and
+// turning that into a failure would make every fresh box look broken for its first week.
+func (r *BackupRunner) PickSettledRestoreCandidateOn(ctx context.Context, target string, notAfter time.Time) (string, time.Time, error) {
+	if target == "" {
+		return "", time.Time{}, nil
+	}
+	contents, err := r.api.StorageContent(ctx, target)
 	if err != nil {
-		return "", err
+		return "", time.Time{}, err
+	}
+	// R-727 (v0.138.0): on an ENCRYPTED storage, only archives written with THIS storage's key are this box's.
+	// Measured 2026-09-30 on a fresh box for a returning customer: the PBS namespace still held two archives
+	// of earlier boxes (same guest id 9201, same token), the newest settled one was an earlier box's, and the
+	// test failed `wrong key` every evaluation. The archive carries no host id; its key fingerprint is the
+	// discriminator (PVE's content `encrypted`, the storage's `encryption-key`). A lookup failure returns
+	// the error — the tier reads UNKNOWN, never "nothing to prove".
+	ownKey, err := r.storageKeyFingerprint(ctx, target)
+	if err != nil {
+		return "", time.Time{}, fmt.Errorf("reading the key fingerprint of storage %s: %w", target, err)
 	}
 	var best string
 	var bestCTime int64 = -1
+	known := map[int]bool{} // vmid → the guest exists on this node (asked once per vmid per pick)
 	for _, e := range contents {
-		if e.Content == "backup" && e.CTime > bestCTime {
+		if e.Content != "backup" {
+			continue
+		}
+		// R-689 (v0.135.0): only a backup OF A GUEST is a restore-test candidate. demo-hp keeps its golden
+		// template in `local:backup/` — content "backup", 654 MB, plausibly complete — and it was picked as
+		// the newest settled archive every 6 h and failed extractconfig (403) each time, while the guest's
+		// real archive went untested.
+		if ok, why := guestBackupArchive(e); !ok {
+			r.noteNotAGuestBackupOnce(e, why)
+			continue
+		}
+		if ownKey != "" && !strings.EqualFold(e.Encrypted, ownKey) {
+			r.noteNotAGuestBackupOnce(e, fmt.Sprintf("written by another box (key %s, this box's key %s) — not this box's proof", shortFP(e.Encrypted), shortFP(ownKey)))
+			continue
+		}
+		// R-689 (v0.136.0): … OF A GUEST THAT STILL EXISTS here. Measured on demo-hp 2026-09-27 right after
+		// v0.135.0: with the golden skipped, the pick fell to `vzdump-lxc-9100-2026_08_21…`, a leftover of a
+		// guest deleted in August — proving nothing about any guest this box runs. "Does not exist" skips the
+		// archive; any OTHER lookup failure is returned, so the tier reads UNKNOWN, never "nothing to prove".
+		if _, seen := known[e.VMID]; !seen {
+			_, err := r.api.GuestConfig(ctx, e.VMID)
+			switch {
+			case err == nil:
+				known[e.VMID] = true
+			case strings.Contains(err.Error(), "does not exist"), strings.Contains(err.Error(), "permission denied"):
+				// v0.137.0: PVE answers 403 "permission denied at /vms/<id>" — not "does not exist" — for a guest
+				// outside the agent's ACL (the `felhom` pool). Measured on demo-hp after v0.136.0: the deleted
+				// guest 9100's archive made the local tier UNKNOWN every evaluation. A guest the agent cannot
+				// read is not one it manages; its archive is not a candidate.
+				known[e.VMID] = false
+			default:
+				return "", time.Time{}, fmt.Errorf("checking whether guest %d still exists: %w", e.VMID, err)
+			}
+		}
+		if !known[e.VMID] {
+			r.noteNotAGuestBackupOnce(e, fmt.Sprintf("guest %d does not exist on this node or is not one this agent manages", e.VMID))
+			continue
+		}
+		if !notAfter.IsZero() && e.CTime > notAfter.Unix() {
+			continue // not settled yet — a newer archive is not a reason to re-prove an older one
+		}
+		if ok, why := archivePlausiblyComplete(e); !ok {
+			r.warnRejectedArchiveOnce(e, why)
+			continue
+		}
+		if e.CTime > bestCTime {
 			bestCTime, best = e.CTime, e.VolID
 		}
 	}
-	return best, nil
+	if best == "" {
+		return "", time.Time{}, nil
+	}
+	return best, time.Unix(bestCTime, 0).UTC(), nil
 }
 
 // latestArchive finds the newest backup archive volid + size for vmid on the target.
@@ -243,6 +445,121 @@ func (r *BackupRunner) latestArchive(ctx context.Context, vmid int) (string, int
 	return vol, size, nil
 }
 
+// NewestArchiveTime reports when this guest's newest backup archive LANDED ON THIS TARGET, from the
+// storage itself. ok=false means the target genuinely holds no archive for this guest.
+//
+// R-84: this is the cure for the redundant-backup-after-restart problem. The agent's backup Store is
+// in-memory ("lost on restart; the cadence re-populates"), so after every restart /backup/due
+// reported "no successful backup recorded yet" and the controller dutifully took another one. On the
+// local tier that is wasted minutes; on the OFFSITE tier it is a wasted multi-hour WAN upload after
+// every agent deploy — and agent deploys are routine. Three redundant local backups were observed on
+// minPlausibleArchiveBytes is the floor below which a storage entry cannot be a real whole-guest
+// backup and is therefore treated as an INCOMPLETE artefact rather than a successful one.
+//
+// MEASURED, not chosen by feel — fleet survey 2026-07-28 (Campaign 8, finding F-CRIT-2):
+//
+//	smallest REAL backup anywhere on the fleet ...  612,397,450 B  (~584 MiB, a guest-9100 vzdump)
+//	demo-hp      local / PBS ..................... 1.59 GB / 4.35-4.37 GB
+//	demo-felhom  local / PBS ..................... 5.82-5.84 GB / 14.47-14.51 GB
+//	the phantom left by a PBS daemon killed mid-upload ....... 1 B
+//
+// 1 MiB sits 584x below the smallest real backup and 1,048,576x above the phantom. The two
+// populations are nine orders of magnitude apart, so this floor cannot plausibly clip a real
+// archive — which is the property that matters, because a floor set too HIGH does not merely lose
+// safety margin, it causes fleet-wide backup THRASH (see archivePlausiblyComplete).
+const minPlausibleArchiveBytes int64 = 1 << 20
+
+// archivePlausiblyComplete reports whether a storage entry can be a COMPLETE backup, and if not,
+// why. Pure, so the contract is unit-testable without a storage.
+//
+// WHY SIZE, AND NOTHING ELSE. The richer PBS fields look like better discriminators and are all
+// traps, because this runner is TIER-AGNOSTIC — the same predicate runs against a PBS datastore and
+// against a plain `dir` storage (verified against the live PVE API, 2026-07-28):
+//
+//   - `verification` is absent on the phantom, but ALSO absent on every local (dir) archive — a dir
+//     storage has no verification concept — and absent on a good PBS snapshot until verify-new
+//     catches up. Gating on it would reject 100% of local backups and every freshly-taken offsite
+//     one: continuous re-backup across the fleet.
+//   - `encrypted` fails the same way, and for the same reason.
+//   - `notes` happens to be present on both good tiers today only because the agent sets it; an
+//     archive written by any other path lacks it. Too fragile to gate freshness on.
+//
+// Size is the only signal that means the same thing on every tier.
+//
+// THE FAIL-SAFE DIRECTION, stated explicitly: when completeness cannot be established the entry is
+// NOT counted as a successful backup. That errs toward the tier looking LESS fresh, and its worst
+// case is one extra backup. Counting an undecidable entry is precisely the F-CRIT-2 defect — a
+// failed upload that made its tier look freshly backed up and silenced it for a full cadence.
+func archivePlausiblyComplete(e proxmox.StorageContent) (bool, string) {
+	if e.Size < minPlausibleArchiveBytes {
+		return false, fmt.Sprintf("size %d B is below the %d B plausibility floor — an aborted/incomplete archive, not a successful backup",
+			e.Size, minPlausibleArchiveBytes)
+	}
+	return true, ""
+}
+
+// warnRejectedArchiveOnce announces a rejected archive at WARN exactly once per distinct volid.
+//
+// A rejected archive must never be silent: a tier that quietly ignores the newest entry on its
+// storage is a new quiet path, and quiet paths are what F-CRIT-2 was. But the due-check runs every
+// 5 minutes and a phantom persists indefinitely — server-side prune does NOT collect it (verified
+// by dry-run 2026-07-28: with keep-last 2 it retained two real snapshots PLUS the phantom) — so
+// logging per poll would emit ~288 identical lines a day and bury the one that matters.
+func (r *BackupRunner) warnRejectedArchiveOnce(e proxmox.StorageContent, why string) {
+	r.rejectedMu.Lock()
+	if r.rejected == nil {
+		r.rejected = map[string]struct{}{}
+	}
+	_, seen := r.rejected[e.VolID]
+	if !seen {
+		r.rejected[e.VolID] = struct{}{}
+	}
+	r.rejectedMu.Unlock()
+	if seen {
+		return
+	}
+	r.logger.Warn("backup: ignoring an INCOMPLETE archive when computing tier freshness — it is not a successful backup",
+		"target", r.target, "vmid", e.VMID, "volid", e.VolID, "size_bytes", e.Size, "reason", why)
+}
+
+// demo-felhom in a single afternoon of deploys (2026-07-26).
+//
+// Asking the STORAGE rather than persisting the store is deliberate:
+//   - it is ground truth, not remembered state — if an archive was pruned or deleted it correctly
+//     stops counting, whereas a persisted record would keep claiming a backup that no longer exists;
+//   - it needs no new on-disk state and no migration;
+//   - it is the same source `latestArchive` already trusts to build the post-backup record.
+//
+// It answers ONLY "when did a plausibly COMPLETE backup last land", which is exactly what the
+// due-check needs. Completeness is not optional here: PBS publishes an aborted upload into the same
+// listing (manifest-less, 1 byte, and NEWEST), and counting it made the tier report fresh and go
+// silent for a whole cadence — F-CRIT-2. Presence is not validity. The
+// richer fields (size, duration, uncovered volumes, error) stay with the real in-memory records — a
+// synthesized record would put invented numbers into the host-report.
+func (r *BackupRunner) NewestArchiveTime(ctx context.Context, vmid int) (time.Time, bool, error) {
+	contents, err := r.api.StorageContent(ctx, r.target)
+	if err != nil {
+		return time.Time{}, false, err
+	}
+	var best int64 = -1
+	for _, e := range contents {
+		if e.Content != "backup" || e.VMID != vmid {
+			continue
+		}
+		if ok, why := archivePlausiblyComplete(e); !ok {
+			r.warnRejectedArchiveOnce(e, why)
+			continue
+		}
+		if e.CTime > best {
+			best = e.CTime
+		}
+	}
+	if best < 0 {
+		return time.Time{}, false, nil
+	}
+	return time.Unix(best, 0).UTC(), true, nil
+}
+
 // parseBackupMode extracts the actual mode from a vzdump task log line `… backup mode: <x>`
 // (e.g. "INFO: backup mode: stop"). Returns "" if not found.
 func parseBackupMode(lines []string) string {
@@ -314,5 +631,79 @@ func ToHubRestoreTest(res reconcile.RestoreTestResult, testedAt time.Time) hub.R
 	if res.Err != nil {
 		rt.Error = res.Err.Error()
 	}
+	if res.SkipReason != "" { // R-672: the space preflight refused — reported, never a pass
+		rt.Pass = false
+		rt.Skipped = true
+		rt.Error = res.SkipReason
+	}
 	return rt
 }
+
+// guestBackupArchive reports whether a storage entry is a whole-guest backup of a known guest — a
+// `vzdump-<type>-<vmid>-…` file on a dir storage, or a `backup/{ct,vm}/<vmid>/<time>` snapshot on a PBS
+// datastore — whose vmid the storage itself reports. Anything else in a backup content type (a golden
+// template, a hand-copied tarball) is not a backup of a guest and is never restore-tested (R-689).
+// Pure, so the rule is unit-tested without a storage.
+func guestBackupArchive(e proxmox.StorageContent) (bool, string) {
+	if e.VMID <= 0 {
+		return false, "not a backup of a guest (the storage reports no vmid)"
+	}
+	vol := e.VolID
+	if i := strings.Index(vol, ":"); i >= 0 {
+		vol = vol[i+1:]
+	}
+	vol = strings.TrimPrefix(vol, "backup/")
+	vmid := strconv.Itoa(e.VMID)
+	switch {
+	case strings.HasPrefix(vol, "vzdump-lxc-"+vmid+"-"), strings.HasPrefix(vol, "vzdump-qemu-"+vmid+"-"):
+		return true, ""
+	case strings.HasPrefix(vol, "ct/"+vmid+"/"), strings.HasPrefix(vol, "vm/"+vmid+"/"):
+		return true, ""
+	}
+	return false, "not a vzdump archive or a PBS snapshot of guest " + vmid
+}
+
+// noteNotAGuestBackupOnce logs, once per volid, that a backup-content entry is not a restore-test
+// candidate because it is not a backup of a guest (R-689). INFO, not WARN: a golden template kept in
+// the backup directory is the operator's, and not a fault.
+func (r *BackupRunner) noteNotAGuestBackupOnce(e proxmox.StorageContent, why string) {
+	r.rejectedMu.Lock()
+	if r.rejected == nil {
+		r.rejected = map[string]struct{}{}
+	}
+	_, seen := r.rejected[e.VolID]
+	if !seen {
+		r.rejected[e.VolID] = struct{}{}
+	}
+	r.rejectedMu.Unlock()
+	if !seen {
+		r.logger.Info("backup: restore-test skips an entry that is not a backup of a guest",
+			"target", r.target, "volid", e.VolID, "size_bytes", e.Size, "reason", why)
+	}
+}
+
+// storageKeyFingerprint returns the named storage's client-side encryption key fingerprint ("" when the
+// storage is not encrypted — a local dir holds only this box's own vzdumps).
+func (r *BackupRunner) storageKeyFingerprint(ctx context.Context, target string) (string, error) {
+	sts, err := r.api.ListStorage(ctx)
+	if err != nil {
+		return "", err
+	}
+	for _, st := range sts {
+		if st.Storage == target {
+			return strings.TrimSpace(st.EncryptionKey), nil
+		}
+	}
+	return "", nil
+}
+
+// shortFP is the first 8 bytes of a key fingerprint, for a log line.
+func shortFP(fp string) string {
+	if fp == "" {
+		return "none"
+	}
+	if len(fp) > 23 {
+		return fp[:23] + "…"
+	}
+	return fp
+}

internal/backup/schedule.go

@@ -18,27 +18,77 @@ type RestoreTestRunner interface {
 // there is none yet (the tick then no-ops).
 type CandidatePicker func(ctx context.Context) (string, error)
 
+// SpecBuilder yields the RestoreTestSpec for ONE run, given the archive that was picked.
+//
+// R-85 (1.1): this REPLACES a frozen spec value. It used to be built by an immediately-invoked
+// function at daemon start, so `storageTier()` and `restoreTaskTimeout()` were evaluated ONCE and
+// the resulting value reused for every run for the lifetime of the process. Two consequences:
+//   - nothing tier-varying was expressible at all (the offsite tier could never be scheduled), and
+//   - it was a latent staleness bug in its own right — a storage-type or config change did not take
+//     effect until the daemon restarted.
+//
+// The archive is passed in because the tier MUST be derived from it (the v0.100.0 rule), never from
+// the configured target: deriving it from config is what produced the 600 s false failure when a
+// PBS archive was classified "local" and got the 10-minute local wait.
+type SpecBuilder func(ctx context.Context, archive string) reconcile.RestoreTestSpec
+
+// TierPicker resolves the newest archive on a NAMED tier that landed AT OR BEFORE notAfter (the
+// settle cutoff), together with when it landed. (*BackupRunner).PickSettledRestoreCandidateOn
+// satisfies it. A zero notAfter means "no settle requirement".
+//
+// R-86 widened this seam from (target) → archive. The landing time is what makes the due-check's
+// verdict explainable — "archive X, which landed at T, has not been proven" — and the cutoff is
+// what makes the rule per-ARCHIVE-GENERATION instead of per-interval. "" must NOT be an error: a
+// brand-new offsite tier legitimately has nothing to restore yet.
+type TierPicker func(ctx context.Context, target string, notAfter time.Time) (archive string, landed time.Time, err error)
+
 // Scheduler runs the self-restore-test on an agent-internal cadence. It is the fourth daemon
 // goroutine; it does real restore→boot→destroy, so it only runs when the cadence is enabled
 // AND a valid scratch band is configured (validated by the caller before construction).
 type Scheduler struct {
-	runner  RestoreTestRunner
-	pick    CandidatePicker
-	store   *Store
-	spec    reconcile.RestoreTestSpec // archive is filled per-tick
+	runner RestoreTestRunner
+	pick   CandidatePicker
+	store  *Store
+	spec   SpecBuilder // R-85: evaluated PER RUN, never frozen at construction
+	// cadence is the EVALUATION interval (R-86) — how often "is anything due?" is asked. It is no
+	// longer the thing that decides a test happens; see restoretest_due.go.
 	cadence time.Duration
-	logger  *slog.Logger
-	now     func() time.Time
+	// settle is how long an archive must have sat before it is a candidate (R-86).
+	settle time.Duration
+	logger *slog.Logger
+	now    func() time.Time
+
+	// R-85 tier rotation. All optional: without them the scheduler behaves exactly as before
+	// (single tier via `pick`), which keeps every existing caller and test working untouched.
+	tiers    []string          // configured tier target ids, primary first
+	tierPick TierPicker        // newest archive on a named tier
+	rtState  *RestoreTestState // persisted last-successful-per-tier (drives oldest-first)
+	inFlight *InFlight         // shared with the backup path — Scenario F
 }
 
 // SchedulerOptions configures a Scheduler.
 type SchedulerOptions struct {
-	Runner  RestoreTestRunner
-	Pick    CandidatePicker
-	Store   *Store
-	Spec    reconcile.RestoreTestSpec // RestoreStorage, ScratchMin/Max, SourceTier, BootTimeout
-	Cadence time.Duration             // 0 → disabled
-	Logger  *slog.Logger
+	Runner RestoreTestRunner
+	Pick   CandidatePicker
+	Store  *Store
+	// Spec builds the run's spec (RestoreStorage, ScratchMin/Max, SourceTier, timeouts) from the
+	// picked archive. Called ONCE PER RUN — see SpecBuilder for why it is not a value.
+	Spec SpecBuilder
+	// Cadence is the EVALUATION interval — how often due-ness is asked, NOT how often a test runs
+	// (R-86). 0 → disabled.
+	Cadence time.Duration
+	// Settle is how long an archive must have sat before it is a restore-test candidate (R-86).
+	// 0 → no settle requirement (any archive is a candidate).
+	Settle time.Duration
+	Logger *slog.Logger
+
+	// R-85 (all optional — omit for the pre-R-85 single-tier behaviour):
+	// Tiers are the configured tier target ids (primary first); TierPick resolves an archive on a
+	// named tier; State persists last-successful-per-tier; InFlight is the shared one-heavy-op gate.
+	Tiers    []string
+	TierPick TierPicker
+	State    *RestoreTestState
+	InFlight *InFlight
 }
 
 // NewScheduler builds a Scheduler.
@@ -48,27 +98,43 @@ func NewScheduler(opts SchedulerOptions) *Scheduler {
 		logger = slog.Default()
 	}
 	return &Scheduler{
-		runner:  opts.Runner,
-		pick:    opts.Pick,
-		store:   opts.Store,
-		spec:    opts.Spec,
-		cadence: opts.Cadence,
-		logger:  logger,
-		now:     func() time.Time { return time.Now().UTC() },
+		runner:   opts.Runner,
+		pick:     opts.Pick,
+		store:    opts.Store,
+		spec:     opts.Spec,
+		cadence:  opts.Cadence,
+		settle:   opts.Settle,
+		logger:   logger,
+		now:      func() time.Time { return time.Now().UTC() },
+		tiers:    append([]string(nil), opts.Tiers...),
+		tierPick: opts.TierPick,
+		rtState:  opts.State,
+		inFlight: opts.InFlight,
 	}
 }
 
-// Run fires a restore-test on the cadence until ctx is cancelled. A 0 cadence disables it
-// (the goroutine just waits for shutdown). It does NOT fire immediately on start (a restore
-// is heavy; the first runs one interval in) — on-demand runs use the selftest harness.
+// Run EVALUATES due-ness on the interval until ctx is cancelled, and runs a restore-test only when
+// a tier is actually due (R-86). A 0 interval disables it (the goroutine just waits for shutdown).
+//
+// The ticker survives as the evaluation interval and nothing else. It is emphatically NOT the
+// trigger any more: its phase is the process's uptime, and agent deploys reset it, which is exactly
+// the defect R-86 removes. What decides that a test happens is `EvaluateDue`.
+//
+// It still does NOT evaluate immediately on start — the first evaluation is one interval in. That
+// is an EARNED restraint, kept deliberately: a restore is heavy, agent restarts are routine, and a
+// crash-loop that evaluated at start would hammer a permanently-failing tier as fast as it could
+// restart. Due-ness does not expire while we wait, so the only cost is up to one interval of
+// latency on a tier that just became due. On-demand runs use `--selftest=restore-test`.
+//
 // Returns nil on ctx cancellation.
 func (s *Scheduler) Run(ctx context.Context) error {
-	if s.cadence <= 0 || s.runner == nil || s.pick == nil {
+	if s.cadence <= 0 || s.runner == nil || s.spec == nil || (s.pick == nil && !s.rotating()) {
 		s.logger.Info("backup: restore-test cadence disabled")
 		<-ctx.Done()
 		return nil
 	}
-	s.logger.Info("backup: restore-test scheduler starting", "cadence", s.cadence)
+	s.logger.Info("backup: restore-test scheduler starting (per-archive due-check)",
+		"eval_interval", s.cadence, "settle", s.settle)
 	t := time.NewTicker(s.cadence)
 	defer t.Stop()
 	for {
@@ -82,26 +148,86 @@ func (s *Scheduler) Run(ctx context.Context) error {
 	}
 }
 
-// tick runs one scheduled restore-test: pick a backup → run → record. No-ops cleanly when
-// no backup exists yet. Deterministic given s.now — tests call it directly.
+// tick is ONE EVALUATION: gate → due-check → run the first due tier → record which archive was
+// proven. No-ops cleanly when nothing is due, when no backup exists yet, or when a heavy operation
+// is already in flight. Deterministic given s.now — tests call it directly.
+//
+// One run per evaluation, by construction (Scenario F): a second due tier is left DUE and picked up
+// by the next evaluation. Deferred, never cancelled, and never two multi-GB restores over one link.
 func (s *Scheduler) tick(ctx context.Context) {
-	archive, err := s.pick(ctx)
+	if s.spec == nil {
+		// Defensive: Run() already refuses to start without a SpecBuilder, but tick is also
+		// reachable directly. Skipping loudly beats panicking the daemon goroutine — a missing
+		// spec must cost a restore-test, never the agent.
+		s.logger.Error("backup: restore-test has no spec builder — skipping (this is a wiring bug)")
+		return
+	}
+	// The due-check runs BEFORE the gate is taken, and that ORDER is load-bearing under R-86.
+	//
+	// It used to be the other way round, and correctly so: the gate was held for one heavy run a
+	// day, and the candidate lookup rode along inside it. Evaluations are now frequent, and the
+	// lookup is a storage listing that for the offsite tier crosses the WAN. Holding the
+	// one-heavy-operation gate for a read that answers "nothing to do" would open a small window at
+	// EVERY evaluation in which a starting backup cannot acquire — and a backup that cannot acquire
+	// does not merely wait, it records a failure and pages the operator (F-A1). A cheap poll must
+	// not be able to manufacture that.
+	//
+	// Nothing is lost by checking first: due-ness does not expire, and the gate is still taken
+	// before anything heavy begins.
+	archive, target, err := s.pickForThisRun(ctx)
 	if err != nil {
 		s.logger.Warn("backup: restore-test could not pick a candidate; skipping", "err", err)
 		return
 	}
 	if archive == "" {
-		s.logger.Info("backup: restore-test skipped; no backup available yet")
+		// A POSITIVE OBSERVABLE, at INFO, and this is not noise — it is standing rule 3.
+		//
+		// Before R-86 every tick ran a heavy restore-test, so the scheduler was audible by
+		// construction. Now "nothing is due" is the NORMAL outcome, and at DEBUG it is silent: an
+		// empty journal would be equally consistent with a healthy loop and with a dead goroutine,
+		// which is the exact shape the R-88 watcher was retired for. One line per evaluation is four
+		// lines a day at the 6h default, and it names each tier's verdict so the answer to "why did
+		// nothing run last night?" is in the log rather than in a re-derivation.
+		s.logger.Info("backup: restore-test evaluated — nothing due", "verdicts", s.verdictSummary(ctx))
 		return
 	}
-	spec := s.spec
+
+	// Scenario F: join the one-heavy-operation-at-a-time gate. A restore-test PULLS a multi-GB
+	// archive over the same tunnel an offsite backup PUSHES one; running both saturates the link and
+	// drives each toward its timeout, which is how a healthy tier gets recorded as failed. DEFER —
+	// never cancel what is already running: a deferred restore-test costs hours of coverage, a
+	// cancelled backup costs the backup. A deferred tier stays DUE, so the next evaluation retries it.
+	release, busy, ok := s.inFlight.TryAcquire("restore-test")
+	if !ok {
+		s.logger.Info("backup: restore-test deferred — a heavy operation is already in flight",
+			"busy", busy, "target", target, "archive", archive)
+		return
+	}
+	defer release()
+	// R-85: build the spec for THIS run, from THIS archive. Never a frozen value.
+	spec := s.spec(ctx, archive)
 	spec.Archive = archive
 	res := s.runner.RunRestoreTest(ctx, spec)
-	if res.Skipped {
+	if res.Skipped && res.SkipReason == "" {
 		return // already logged by the engine (no free scratch VMID)
 	}
+	// R-672: a SPACE refusal is the test's result — reported (pass=false, the reason as the error),
+	// never dropped and never a pass. It earns no rotation credit, so the tier stays due.
 	rt := ToHubRestoreTest(res, s.now())
 	s.store.RecordRestoreTest(rt)
+	// Rotation credit is given ONLY on success. A failing tier must keep sorting first, or a tier
+	// that fails every time would look freshly proven and quietly stop being retried.
+	if rt.Pass && s.rtState != nil && target != "" {
+		// R-86: the ARCHIVE is recorded, not merely the time — that is what makes the tier
+		// not-due until a NEWER archive settles, and what makes a proof survive a restart.
+		// R-189: the TIER and what was VERIFIED go with it, so the proof can be RE-REPORTED after a
+		// restart. Both come from the run's own result, never re-derived — `rt.SourceTier` is what
+		// this run was actually judged as, and deriving it later would need a storage lookup that
+		// can fail on the one path where failing means mislabelling a proof.
+		if err := s.rtState.RecordSuccess(target, archive, rt.SourceTier, rt.Verified, s.now()); err != nil {
+			s.logger.Warn("backup: could not persist the restore-test proof state", "target", target, "err", err)
+		}
+	}
 	switch {
 	case !rt.Pass:
 		// A failing restore-test is the loudest DR signal there is.
@@ -118,3 +244,56 @@ func (s *Scheduler) tick(ctx context.Context) {
 			"archive", rt.SourceArchive, "duration_s", rt.DurationSeconds, "warnings", res.StartWarnings)
 	}
 }
+
+// rotating reports whether multi-tier rotation is wired.
+func (s *Scheduler) rotating() bool { return len(s.tiers) > 0 && s.tierPick != nil }
+
+// pickForThisRun chooses the tier to test THIS evaluation: the first DUE tier, in oldest-proven
+// order.
+//
+// R-86 changed what this answers. It used to answer "whose turn is it?", and the answer was always
+// somebody's — the ticker had fired, so a test was going to happen. It now answers "is anything
+// due?", and "nothing" is a normal, frequent and correct answer.
+//
+// OLDEST-FIRST (operator ruling 2026-07-26, Option 1) survives as the ORDER among due tiers: the
+// tier whose last successful restore-test is oldest goes first, never-proven first of all. It is
+// self-balancing, needs no config knob, and it still cannot starve a tier — but it no longer decides
+// that a test happens at all.
+//
+// A tier with no settled archive is SKIPPED, not failed — a brand-new offsite tier has nothing to
+// restore yet, and that is normal, not broken. A tier whose archives cannot be LISTED is likewise
+// skipped, loudly, and its error is returned only when no other tier was testable: one tier's
+// storage being unreadable must not cost the other tier its proof, and must not be silent either.
+//
+// Returns ("", "", nil) when nothing anywhere is due.
+func (s *Scheduler) pickForThisRun(ctx context.Context) (archive, target string, err error) {
+	if !s.rotating() {
+		// Pre-R-85 single-tier path (tests and any caller that wires only `Pick`): there is no tier
+		// identity and no persisted proof here, so there is nothing to compare an archive against
+		// and no due-check is possible. It runs on every evaluation, exactly as it always did.
+		a, perr := s.pick(ctx)
+		return a, "", perr
+	}
+	var firstErr error
+	for _, v := range s.EvaluateDue(ctx) {
+		if v.Err != nil {
+			s.logger.Warn("backup: restore-test candidate lookup failed for a tier; trying the next",
+				"target", v.Target, "err", v.Err)
+			if firstErr == nil {
+				firstErr = v.Err
+			}
+			continue
+		}
+		if !v.Due {
+			s.logger.Debug("backup: restore-test tier is not due", "target", v.Target, "reason", v.Reason)
+			continue
+		}
+		s.logger.Info("backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)",
+			"target", v.Target, "archive", v.Archive, "landed", v.Landed.Format(time.RFC3339), "reason", v.Reason)
+		return v.Archive, v.Target, nil
+	}
+	if firstErr != nil {
+		return "", "", firstErr
+	}
+	return "", "", nil
+}

internal/backup/schedule_spec_test.go

@@ -0,0 +1,124 @@
+package backup
+
+import (
+	"context"
+	"fmt"
+	"sync"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
+)
+
+// R-85 (1.1) — the spec is built PER RUN, never frozen at construction.
+//
+// It used to be an immediately-invoked function at daemon start, so storageTier() and
+// restoreTaskTimeout() were evaluated ONCE and the value reused for every run for the process
+// lifetime. That is what made an offsite restore-test impossible to schedule at all, and it was a
+// latent staleness bug besides: a storage-type or config change did not take effect until restart.
+
+type specSpy struct {
+	mu       sync.Mutex
+	calls    int
+	archives []string
+	tiers    []string // what the builder decided, per call
+}
+
+func (sp *specSpy) build(_ context.Context, archive string) reconcile.RestoreTestSpec {
+	sp.mu.Lock()
+	defer sp.mu.Unlock()
+	sp.calls++
+	sp.archives = append(sp.archives, archive)
+	// Decide the tier from the ARCHIVE, exactly as main.go does (the v0.100.0 rule).
+	tier := "local"
+	if len(archive) > 10 && archive[:10] == "felhom-pbs" {
+		tier = "pbs"
+	}
+	sp.tiers = append(sp.tiers, tier)
+	return reconcile.RestoreTestSpec{
+		RestoreStorage: "local-lvm", ScratchMin: 990000, ScratchMax: 990009, SourceTier: tier,
+	}
+}
+
+// COMPANION RED-PROOF (observed): change Scheduler.spec back to a frozen
+// `reconcile.RestoreTestSpec` value captured at construction → this fails with
+// "the spec builder must run ONCE PER RUN, got 1 call(s) across 3 ticks", because a frozen value is
+// evaluated exactly once no matter how many ticks fire. Restored.
+func TestScheduler_SpecIsBuiltPerRun(t *testing.T) {
+	sp := &specSpy{}
+	rt := &fakeRTRunner{res: reconcile.RestoreTestResult{Pass: true, Verified: "boot+running"}}
+	n := 0
+	s := NewScheduler(SchedulerOptions{
+		Runner: rt,
+		Pick: func(context.Context) (string, error) {
+			n++
+			return fmt.Sprintf("local:backup/vzdump-lxc-9201-%d.tar.zst", n), nil
+		},
+		Store:   NewStore(),
+		Spec:    sp.build,
+		Cadence: time.Hour,
+		Logger:  quiet(),
+	})
+
+	for i := 0; i < 3; i++ {
+		s.tick(context.Background())
+	}
+
+	sp.mu.Lock()
+	defer sp.mu.Unlock()
+	if sp.calls != 3 {
+		t.Fatalf("the spec builder must run ONCE PER RUN, got %d call(s) across 3 ticks", sp.calls)
+	}
+	// And it must see the archive THIS run picked — not a stale one.
+	for i, a := range sp.archives {
+		want := fmt.Sprintf("local:backup/vzdump-lxc-9201-%d.tar.zst", i+1)
+		if a != want {
+			t.Fatalf("run %d: builder saw archive %q, want %q — the spec is not tracking the picked archive", i+1, a, want)
+		}
+	}
+}
+
+// The tier must follow the ARCHIVE across runs. A builder that saw only the configured target would
+// return the same tier every time — which is exactly the v0.100.0 defect that killed a 14.46 GB WAN
+// restore at the 10-minute local bound.
+func TestScheduler_SpecTierFollowsTheArchive(t *testing.T) {
+	sp := &specSpy{}
+	rt := &fakeRTRunner{res: reconcile.RestoreTestResult{Pass: true, Verified: "boot+running"}}
+	archives := []string{
+		"local:backup/vzdump-lxc-9201-x.tar.zst",
+		"felhom-pbs:backup/ct/9201/2026-07-26T15:42:42Z",
+	}
+	i := 0
+	s := NewScheduler(SchedulerOptions{
+		Runner: rt,
+		Pick: func(context.Context) (string, error) {
+			a := archives[i%len(archives)]
+			i++
+			return a, nil
+		},
+		Store: NewStore(), Spec: sp.build, Cadence: time.Hour, Logger: quiet(),
+	})
+	s.tick(context.Background())
+	s.tick(context.Background())
+
+	sp.mu.Lock()
+	defer sp.mu.Unlock()
+	if len(sp.tiers) != 2 || sp.tiers[0] != "local" || sp.tiers[1] != "pbs" {
+		t.Fatalf("the tier must follow the archive per run; got %v", sp.tiers)
+	}
+}
+
+// A nil spec builder must SKIP loudly, not panic — a wiring bug costs a restore-test, never the
+// daemon goroutine.
+func TestScheduler_NilSpecSkipsInsteadOfPanicking(t *testing.T) {
+	rt := &fakeRTRunner{}
+	s := NewScheduler(SchedulerOptions{
+		Runner: rt,
+		Pick:   func(context.Context) (string, error) { return "vol", nil },
+		Store:  NewStore(), Cadence: time.Hour, Logger: quiet(),
+	})
+	s.tick(context.Background()) // must not panic
+	if rt.runs != 0 {
+		t.Fatalf("a nil spec must not run a restore-test; got %d run(s)", rt.runs)
+	}
+}

internal/backup/store.go

@@ -10,8 +10,22 @@ import (
 // Store holds the agent's LATEST backup result per target and the latest restore-test
 // result — the point-in-time state the host-report surfaces. It is updated by the backup
 // runner + the restore-test scheduler/selftest and read by the collector via the hub
-// BackupReporter / RestoreTestReporter seams. In-memory (lost on restart; the cadence
-// re-populates) and mutex-guarded for the concurrent collector vs scheduler access.
+// BackupReporter / RestoreTestReporter seams. In-memory and mutex-guarded for the concurrent
+// collector vs scheduler access.
+//
+// **"lost on restart; the cadence re-populates" — that sentence used to be here and it is now
+// FALSE for restore-tests (R-189, 2026-08-03).** It was true while a timer re-tested every tier
+// daily. Under R-86's per-archive due-check the agent will NOT re-test an archive it has already
+// proven, so a proof lost to a restart is not repeated until the next archive generation — a week on
+// the offsite tier — and the hub reports that tier unproven throughout. Observed, not predicted: a
+// real 14.5 GB offsite restore passed, the agent was restarted 2 m 43 s later for a deploy, and two
+// consecutive host-reports carried `0 restore-tests`.
+//
+// The durable half is `RestoreTestState` (on disk, per tier, with the archive) and the collector
+// merges the two — see hub.ProvenRestoreTestReporter. This store remains the ONLY place a FAILURE is
+// recorded, and that asymmetry is deliberate: a failing tier stays due and is retried, so a lost
+// failure heals itself, while a lost success leaves the system quietly less tested than it believes.
+// Backups are unaffected — their freshness has a ground truth on the storage (R-84).
 type Store struct {
 	mu       sync.Mutex
 	byTarget map[string]hub.Backup // latest backup per target id

internal/capability/manifest.go

@@ -123,6 +123,9 @@ var manifest = []Capability{
 	{"dnsmasq-install", "dnsmasq package install", "/usr/bin/apt-get", []string{"install", "-y", "-q", "dnsmasq"}, false, ""},
 	{"dnsmasq-write", "dnsmasq drop-in write", "/usr/bin/install", []string{"-m", "0644", "/tmp/felhom-resolver-x.conf", "/etc/dnsmasq.d/felhom-x.conf"}, false, ""},
 	{"dnsmasq-enable", "dnsmasq enable", "/usr/bin/systemctl", []string{"enable", "--now", "dnsmasq"}, false, ""},
+
+	// ---- OS updates, guest fast lane (`11` §5.4.1; the wrapper holds every rule) ----
+	{"osapply-run", "OS update wrapper (guest fast lane)", "/usr/local/sbin/felhom-os-apply", []string{"--plan", "/var/lib/felhom-agent/os/plan-x.json"}, false, ""},
 	{"dnsmasq-reload", "dnsmasq reload", "/usr/bin/systemctl", []string{"reload", "dnsmasq"}, false, ""},
 	{"dnsmasq-restart", "dnsmasq restart (LAN-DNS self-heal)", "/usr/bin/systemctl", []string{"restart", "dnsmasq"}, false, ""},
 	{"dnsmasq-rm", "dnsmasq drop-in remove (decommission)", "/usr/bin/rm", []string{"-f", "/etc/dnsmasq.d/felhom-x.conf"}, false, ""},

internal/config/backup_tiers_test.go

@@ -0,0 +1,182 @@
+package config
+
+import (
+	"encoding/json"
+	"strings"
+	"testing"
+	"time"
+)
+
+// R-82 Slice A.1 — per-target cadence + retention resolution.
+//
+// The load-bearing property is ADDITIVITY: every config that exists on a live box today must
+// resolve to exactly one tier that behaves as it does now. The second property is that a
+// mis-configured tier is REJECTED LOUDLY rather than defaulted — a weekly DR tier silently running
+// daily would fill the datastore, and a silently dropped tier is the "applied and empty" fault
+// R-82 exists to fix.
+
+func TestBackupTiers_LegacyConfigIsUnchanged(t *testing.T) {
+	// Exactly the shape live on demo-felhom today.
+	var b BackupConfig
+	raw := `{"local_backup_target":"local","local_backup_retention":3,"backup_cadence_seconds":0}`
+	if err := json.Unmarshal([]byte(raw), &b); err != nil {
+		t.Fatal(err)
+	}
+	tiers, warnings := b.BackupTiers()
+	if len(warnings) != 0 {
+		t.Fatalf("a legacy config must produce NO warnings; got %v", warnings)
+	}
+	if len(tiers) != 1 {
+		t.Fatalf("a config with no backup_targets must resolve to exactly ONE tier; got %+v", tiers)
+	}
+	got := tiers[0]
+	if got.TargetID != "local" || got.Cadence != 24*time.Hour || got.KeepLast != 3 || !got.Primary {
+		t.Fatalf("legacy tier changed: %+v", got)
+	}
+}
+
+// An empty BackupConfig still resolves — to the felhom-pbs default target, 24h, keep-last 3.
+// (Unchanged pre-R-82 behaviour; pinned so the default target can't drift unnoticed.)
+func TestBackupTiers_ZeroConfigKeepsDefaults(t *testing.T) {
+	tiers, warnings := BackupConfig{}.BackupTiers()
+	if len(warnings) != 0 || len(tiers) != 1 {
+		t.Fatalf("zero config: tiers=%+v warnings=%v", tiers, warnings)
+	}
+	if tiers[0].TargetID != defaultBackupTarget || tiers[0].Cadence != 24*time.Hour || tiers[0].KeepLast != 3 {
+		t.Fatalf("zero-config defaults changed: %+v", tiers[0])
+	}
+}
+
+// The whole point: local daily + PBS weekly, each with its OWN retention.
+func TestBackupTiers_LocalDailyPlusPBSWeekly(t *testing.T) {
+	var b BackupConfig
+	raw := `{
+	  "local_backup_target":"local",
+	  "local_backup_retention":3,
+	  "backup_cadence_seconds":86400,
+	  "backup_targets":[{"target_id":"felhom-pbs","cadence_seconds":604800,"keep_last":2}]
+	}`
+	if err := json.Unmarshal([]byte(raw), &b); err != nil {
+		t.Fatal(err)
+	}
+	tiers, warnings := b.BackupTiers()
+	if len(warnings) != 0 {
+		t.Fatalf("unexpected warnings: %v", warnings)
+	}
+	if len(tiers) != 2 {
+		t.Fatalf("want 2 tiers, got %+v", tiers)
+	}
+	if !tiers[0].Primary || tiers[0].TargetID != "local" || tiers[0].Cadence != 24*time.Hour || tiers[0].KeepLast != 3 {
+		t.Fatalf("primary tier wrong: %+v", tiers[0])
+	}
+	if tiers[1].Primary || tiers[1].TargetID != "felhom-pbs" || tiers[1].Cadence != 7*24*time.Hour || tiers[1].KeepLast != 2 {
+		t.Fatalf("PBS tier wrong: %+v", tiers[1])
+	}
+	// THE knob-sharing check: the two retentions are independent values, not one shared number.
+	if tiers[0].KeepLast == tiers[1].KeepLast {
+		t.Fatalf("this fixture sets 3 and 2 deliberately — equal values mean the knob is shared: %+v", tiers)
+	}
+}
+
+// A tier with no cadence is REJECTED, not defaulted. Defaulting would turn a weekly DR tier into a
+// daily one and fill the 37.2 GB datastore (R-82 Phase 0, P0.3).
+func TestBackupTiers_MissingCadenceIsRejectedLoudly(t *testing.T) {
+	b := BackupConfig{
+		LocalBackupTarget: "local",
+		ExtraTargets:      []BackupTargetConfig{{TargetID: "felhom-pbs", KeepLast: 2}},
+	}
+	tiers, warnings := b.BackupTiers()
+	if len(tiers) != 1 {
+		t.Fatalf("a cadence-less tier must NOT be armed; got %+v", tiers)
+	}
+	if len(warnings) != 1 || !strings.Contains(warnings[0], "cadence_seconds must be > 0") {
+		t.Fatalf("rejection must be reported so the caller can log it loudly; got %v", warnings)
+	}
+	if !strings.Contains(warnings[0], "felhom-pbs") {
+		t.Fatalf("the warning must name the tier it dropped; got %q", warnings[0])
+	}
+}
+
+func TestBackupTiers_RejectsEmptyAndDuplicateTargets(t *testing.T) {
+	b := BackupConfig{
+		LocalBackupTarget: "local",
+		ExtraTargets: []BackupTargetConfig{
+			{TargetID: "", CadenceSeconds: 3600},
+			{TargetID: "local", CadenceSeconds: 3600},        // repeats the primary
+			{TargetID: "felhom-pbs", CadenceSeconds: 604800}, // good
+			{TargetID: "felhom-pbs", CadenceSeconds: 99},     // duplicate
+		},
+	}
+	tiers, warnings := b.BackupTiers()
+	if len(tiers) != 2 || tiers[1].TargetID != "felhom-pbs" || tiers[1].Cadence != 7*24*time.Hour {
+		t.Fatalf("want primary + one PBS tier at the FIRST definition; got %+v", tiers)
+	}
+	if len(warnings) != 3 {
+		t.Fatalf("want 3 rejections (empty, duplicate-of-primary, duplicate); got %v", warnings)
+	}
+}
+
+// keep_last unset means DO NOT PRUNE. That is the fail-safe: a DR tier must never start pruning
+// itself because someone forgot a field.
+func TestBackupTiers_UnsetKeepLastMeansNoPrune(t *testing.T) {
+	b := BackupConfig{
+		LocalBackupTarget: "local",
+		ExtraTargets:      []BackupTargetConfig{{TargetID: "felhom-pbs", CadenceSeconds: 604800}},
+	}
+	tiers, _ := b.BackupTiers()
+	if len(tiers) != 2 {
+		t.Fatalf("got %+v", tiers)
+	}
+	if tiers[1].KeepLast != 0 {
+		t.Fatalf("an unset keep_last must resolve to 0 = never prune; got %d", tiers[1].KeepLast)
+	}
+	// And a negative is clamped to the same fail-safe rather than becoming a prune spec.
+	b.ExtraTargets[0].KeepLast = -5
+	tiers, _ = b.BackupTiers()
+	if tiers[1].KeepLast != 0 {
+		t.Fatalf("a negative keep_last must clamp to 0 (never prune); got %d", tiers[1].KeepLast)
+	}
+}
+
+// The primary's retention still comes from the legacy knob with its legacy clamp — untouched.
+func TestBackupTiers_PrimaryRetentionClampUnchanged(t *testing.T) {
+	for _, tc := range []struct{ in, want int }{{0, 3}, {-1, 3}, {1, 1}, {7, 7}} {
+		b := BackupConfig{LocalBackupTarget: "local", LocalBackupRetention: tc.in}
+		tiers, _ := b.BackupTiers()
+		if tiers[0].KeepLast != tc.want {
+			t.Fatalf("LocalBackupRetention=%d → KeepLast=%d, want %d", tc.in, tiers[0].KeepLast, tc.want)
+		}
+	}
+}
+
+// R-82 live-failure regression (2026-07-26): the runner hard-coded a 30-minute vzdump wait, which
+// is right for a local vzdump and wrong for an offsite PBS upload. The first full ~10 GB PBS
+// snapshot on demo-felhom ran past 30 min; the agent gave up waiting and recorded success=false
+// WHILE THE BACKUP WAS STILL RUNNING — a false failure that leaves the tier permanently "due" and
+// makes the next attempt collide with the guest lock vzdump still holds.
+func TestBackupTiers_WaitTimeoutIsPerTier(t *testing.T) {
+	b := BackupConfig{
+		LocalBackupTarget: "local",
+		ExtraTargets:      []BackupTargetConfig{{TargetID: "felhom-pbs", CadenceSeconds: 604800}},
+	}
+	tiers, _ := b.BackupTiers()
+	if len(tiers) != 2 {
+		t.Fatalf("got %+v", tiers)
+	}
+	if tiers[0].WaitTimeout != 30*time.Minute {
+		t.Fatalf("the PRIMARY must keep the historical 30m wait (unchanged behaviour); got %s", tiers[0].WaitTimeout)
+	}
+	if tiers[1].WaitTimeout != 12*time.Hour {
+		t.Fatalf("an offsite tier must default to a GENEROUS wait (operator ruling: let the first backup run as long as needed) — a false timeout is worse than a slow pass; got %s", tiers[1].WaitTimeout)
+	}
+	// And it must be overridable per tier.
+	b.ExtraTargets[0].WaitTimeoutSeconds = 3600
+	tiers, _ = b.BackupTiers()
+	if tiers[1].WaitTimeout != time.Hour {
+		t.Fatalf("wait_timeout_seconds must override; got %s", tiers[1].WaitTimeout)
+	}
+	// The two tiers must NOT share one bound.
+	if tiers[0].WaitTimeout == tiers[1].WaitTimeout {
+		t.Fatalf("wait bounds are shared between tiers — the whole point is that they differ: %+v", tiers)
+	}
+}

internal/config/config.go

@@ -235,6 +235,16 @@ type LocalAPIConfig struct {
 	// TokenStore is the durable, hashed token→guest map (only a HASH of each token is
 	// persisted; the plaintext exists transiently at mint→write-to-mount, then is discarded).
 	TokenStore string `json:"token_store"` // default /var/lib/felhom-agent/local-tokens.log
+	// IslandBridge + IslandGuestAddr configure the R-50 host-internal control-plane bridge. When
+	// BOTH are set, the provisioner attaches each guest a static net1 on IslandBridge with
+	// IslandGuestAddr, so the controller reaches the agent over a fixed private address that no
+	// LAN/DHCP/site move can invalidate (the F1 fix — AUDIT-vacation-remote-ops-2026-07-20). Empty
+	// (the default) = LAN-only, byte-for-byte the pre-R-50 behaviour. On an island install ListenAddr
+	// is the host side (169.254.253.1:8443); IslandGuestAddr is the guest side (169.254.253.2/30 — a
+	// /30 is exactly host + one guest). Additive-only: it never removes a NIC, so a guest restored on
+	// a non-island host (both empty) is unaffected.
+	IslandBridge    string `json:"island_bridge"`     // e.g. "vmbr9" (portless host-internal bridge)
+	IslandGuestAddr string `json:"island_guest_addr"` // guest net1 CIDR, e.g. "169.254.253.2/30"
 }
 
 // Default local-API file locations (under the agent's state dir).
@@ -249,6 +259,12 @@ func (l LocalAPIConfig) Enabled() bool {
 	return l.Enable && strings.TrimSpace(l.ListenAddr) != ""
 }
 
+// IslandEnabled reports whether the provisioner should attach a guest island NIC (net1). True only
+// when BOTH the bridge and the guest CIDR are set (R-50); empty = pre-R-50 LAN-only behaviour.
+func (l LocalAPIConfig) IslandEnabled() bool {
+	return strings.TrimSpace(l.IslandBridge) != "" && strings.TrimSpace(l.IslandGuestAddr) != ""
+}
+
 // TokenStorePath returns the configured token-store path (default applied).
 func (l LocalAPIConfig) TokenStorePath() string {
 	if l.TokenStore != "" {
@@ -283,6 +299,17 @@ func (l LocalAPIConfig) Validate() error {
 	if _, _, err := net.SplitHostPort(l.ListenAddr); err != nil {
 		return fmt.Errorf("config: local_api.listen_addr %q is not host:port: %w", l.ListenAddr, err)
 	}
+	// R-50: island fields are all-or-nothing, and the guest addr must be a CIDR (the net1 ip= value).
+	// A half-set island (bridge without guest addr, or vice versa) is a provisioning mistake, not a
+	// silent LAN fallback — fail loudly so a botched install config is caught at load, not at day-0.
+	if (strings.TrimSpace(l.IslandBridge) != "") != (strings.TrimSpace(l.IslandGuestAddr) != "") {
+		return fmt.Errorf("config: local_api.island_bridge and local_api.island_guest_addr must be set together (got bridge=%q guest_addr=%q)", l.IslandBridge, l.IslandGuestAddr)
+	}
+	if l.IslandEnabled() {
+		if _, _, err := net.ParseCIDR(strings.TrimSpace(l.IslandGuestAddr)); err != nil {
+			return fmt.Errorf("config: local_api.island_guest_addr %q is not a CIDR (want e.g. 169.254.253.2/30): %w", l.IslandGuestAddr, err)
+		}
+	}
 	return nil
 }
 
@@ -306,9 +333,29 @@ type BackupConfig struct {
 	LocalBackupTarget string `json:"local_backup_target"`
 	// RestoreStorage is where a restore-test's restored rootfs lands, e.g. "local-lvm".
 	RestoreStorage string `json:"restore_storage"`
-	// RestoreTestCadenceSeconds is the self-restore-test interval; 0 → default (24h).
-	// Set negative to DISABLE the automatic cadence (on-demand selftest still works).
+	// RestoreTestCadenceSeconds is the LEGACY restore-test knob, retained for one meaning only:
+	// NEGATIVE still DISABLES the automatic restore-test entirely (on-demand selftest still works),
+	// and 0 still means "use the default". It no longer sets how often a test runs — R-86 replaced
+	// the interval trigger with a per-archive due-check — so a positive value now seeds
+	// RestoreTestSettleSeconds instead (see RestoreTestSettle). Prefer the two explicit keys below.
 	RestoreTestCadenceSeconds int `json:"restore_test_cadence_seconds"`
+	// RestoreTestEvalIntervalSeconds is how often the scheduler ASKS whether any tier is due
+	// (R-86); 0 → default. It is not how often a test runs: a tier is tested once per archive
+	// generation no matter how often it is asked. This interval sets two things — the latency
+	// between an archive settling and its proof, and the retry rate of a tier whose restore-test
+	// keeps failing. See defaultRestoreTestEvalInterval for the measurement it was chosen from.
+	RestoreTestEvalIntervalSeconds int `json:"restore_test_eval_interval_seconds"`
+	// RestoreTestSpaceFactor / RestoreTestSpaceReserveGiB are the restore-test's space margin (R-672,
+	// v0.133.0): a test starts only when the target storage has free ≥ restored × factor + reserve,
+	// `restored` being the UNCOMPRESSED size. 0/unset → 1.2 and 5 GiB. A test config may raise them to
+	// watch the refusal (the brief's live case a).
+	RestoreTestSpaceFactor     float64 `json:"restore_test_space_factor,omitempty"`
+	RestoreTestSpaceReserveGiB float64 `json:"restore_test_space_reserve_gib,omitempty"`
+	// RestoreTestSettleSeconds is how long an archive must have sat on its tier before it is a
+	// restore-test candidate (R-86); 0 → default (24h), negative → 0 (no settle requirement).
+	// Restore-testing an archive a backup is still writing proves nothing about the backup that
+	// finished — this is the same settle discipline R-71a's gate applies to the offsite consume.
+	RestoreTestSettleSeconds int `json:"restore_test_settle_seconds"`
 	// ScratchVMIDMin/Max bound the throwaway restore-test scratch-guest VMID band. The
 	// restore-test refuses to run unless this is a valid band (min>0, max>=min); 9999 is
 	// always excluded. Defaults to 990000–990009.
@@ -339,6 +386,132 @@ type BackupConfig struct {
 	// default 3; ALWAYS clamped to ≥1 by KeepLast() so a mis-config can never prune the fresh backup.
 	// NEVER applied to a PBS target (offsite retention is a separate lifecycle).
 	LocalBackupRetention int `json:"local_backup_retention"`
+
+	// ExtraTargets (R-82) are ADDITIONAL backup tiers beyond the primary one above — the shape that
+	// makes "local daily + PBS weekly" expressible at all. Each carries its OWN cadence and its OWN
+	// retention, because those are semantically different per tier: keep-last=3 on a daily tier is
+	// three DAYS of restore points; on a weekly tier it is three WEEKS. Sharing one knob between
+	// tiers silently means one of them is wrong.
+	//
+	// ADDITIVE BY CONSTRUCTION: an existing config with no `backup_targets` key resolves to exactly
+	// one tier — the primary — and behaves byte-identically to pre-R-82. Nothing here changes the
+	// local tier.
+	ExtraTargets []BackupTargetConfig `json:"backup_targets"`
+}
+
+// BackupTargetConfig is ONE additional backup tier: a vzdump storage plus its own cadence and
+// retention. A tier with no cadence is not a tier — see BackupTiers for why that is rejected loudly
+// rather than defaulted.
+type BackupTargetConfig struct {
+	// TargetID is the Proxmox storage id (content=backup), e.g. "felhom-pbs".
+	TargetID string `json:"target_id"`
+	// CadenceSeconds is THIS tier's /backup/due window. REQUIRED (>0) — see BackupTiers.
+	CadenceSeconds int `json:"cadence_seconds"`
+	// KeepLast is THIS tier's per-run `--prune-backups` keep-last. 0/unset → NEVER prune this tier
+	// (the fail-safe default, and the current behaviour for every PBS target). A PBS tier is never
+	// pruned by the per-run flag regardless — see BackupRunner.localPruneSpec.
+	KeepLast int `json:"keep_last"`
+	// WaitTimeoutSeconds bounds how long the agent WAITS for this tier's vzdump task. 0/unset →
+	// defaultExtraTierWaitTimeout.
+	//
+	// THIS FIELD EXISTS BECAUSE OF A LIVE FAILURE (2026-07-26, R-82 Slice A validation). The runner
+	// hard-coded a 30-minute wait, which is right for a local vzdump (minutes) and badly wrong for
+	// an offsite PBS backup over a home uplink: the first full ~10 GB snapshot ran past 30 min, the
+	// agent gave up waiting and recorded success=false — WHILE THE BACKUP WAS STILL RUNNING. That
+	// false failure is worse than a slow pass: the tier stays "due", a retry collides with the
+	// guest lock vzdump still holds, and the hub sees a DR tier that never succeeds.
+	//
+	// Same reasoning as RestoreTestPBSRestoreTimeoutSeconds on the restore side, and the same
+	// direction: when in doubt wait LONGER. A slow backup is a slow backup; a false timeout is a
+	// corrupt status plus lock contention.
+	WaitTimeoutSeconds int `json:"wait_timeout_seconds"`
+}
+
+// Per-tier vzdump wait bounds.
+//
+// The PRIMARY keeps the historical 30 minutes: it is the local tier, a local vzdump takes minutes,
+// and one hanging 30 minutes is a genuine fault worth surfacing. Unchanged behaviour.
+//
+// An ADDITIONAL tier is by construction the offsite/WAN one in this design, where the binding
+// constraint is uplink speed, not health. Measured on demo-felhom: ~33 MB/min over the wg link to
+// Hetzner, so a first FULL ~10 GB snapshot projects to ~5h. Operator ruling 2026-07-26: "let the
+// first backup run as long as needed" — 12h gives that real margin on a slower link while still
+// being BOUNDED, so a genuinely hung task eventually surfaces instead of hanging forever.
+const (
+	defaultPrimaryTierWaitTimeout = 30 * time.Minute
+	defaultExtraTierWaitTimeout   = 12 * time.Hour
+)
+
+// BackupTier is a RESOLVED backup tier: one target, its own cadence, its own retention. The agent
+// builds one runner per tier from these.
+type BackupTier struct {
+	TargetID string
+	Cadence  time.Duration
+	// WaitTimeout bounds the wait on this tier's vzdump task (see WaitTimeoutSeconds).
+	WaitTimeout time.Duration
+	// KeepLast is the per-run prune keep-last; 0 means DO NOT PRUNE this tier.
+	KeepLast int
+	// Primary marks the tier that the UNTARGETED local-API endpoints act on — the pre-R-82 tier.
+	// Exactly one tier is primary, and it is always first.
+	Primary bool
+}
+
+// BackupTiers resolves the effective tier list, primary first, plus any warnings the caller MUST
+// log (they describe tiers that were REJECTED, and a silently-dropped backup tier is precisely the
+// "applied and empty" fault R-82 exists to fix).
+//
+// Rules:
+//   - Tier 0 is always the primary, built from BackupTarget()/BackupCadence()/KeepLast() — so a
+//     config with no `backup_targets` is byte-identical to pre-R-82.
+//   - An extra with an empty target_id is rejected.
+//   - An extra with cadence_seconds <= 0 is REJECTED, not defaulted. Defaulting a PBS tier to the
+//     24h local default would quietly turn a weekly tier into a daily one and fill the DR datastore;
+//     a tier whose cadence you did not state is not a tier.
+//   - An extra repeating the primary's target is rejected (one policy per target, or the two
+//     cadences race and neither is the truth).
+//   - Duplicate extras are rejected after the first.
+func (b BackupConfig) BackupTiers() ([]BackupTier, []string) {
+	primary := BackupTier{
+		TargetID:    b.BackupTarget(),
+		Cadence:     b.BackupCadence(),
+		KeepLast:    b.KeepLast(),
+		WaitTimeout: defaultPrimaryTierWaitTimeout,
+		Primary:     true,
+	}
+	tiers := []BackupTier{primary}
+	var warnings []string
+
+	seen := map[string]bool{primary.TargetID: true}
+	for i, t := range b.ExtraTargets {
+		id := strings.TrimSpace(t.TargetID)
+		switch {
+		case id == "":
+			warnings = append(warnings, fmt.Sprintf("backup_targets[%d]: empty target_id — tier ignored", i))
+			continue
+		case seen[id]:
+			warnings = append(warnings, fmt.Sprintf("backup_targets[%d]: target %q already configured — duplicate tier ignored", i, id))
+			continue
+		case t.CadenceSeconds <= 0:
+			warnings = append(warnings, fmt.Sprintf("backup_targets[%d] (%s): cadence_seconds must be > 0 — tier ignored (a cadence is NOT defaulted: a weekly tier silently running daily would fill the DR datastore)", i, id))
+			continue
+		}
+		seen[id] = true
+		keep := t.KeepLast
+		if keep < 0 {
+			keep = 0
+		}
+		wait := defaultExtraTierWaitTimeout
+		if t.WaitTimeoutSeconds > 0 {
+			wait = time.Duration(t.WaitTimeoutSeconds) * time.Second
+		}
+		tiers = append(tiers, BackupTier{
+			TargetID:    id,
+			Cadence:     time.Duration(t.CadenceSeconds) * time.Second,
+			KeepLast:    keep,
+			WaitTimeout: wait,
+		})
+	}
+	return tiers, warnings
 }
 
 // defaultLocalBackupKeepLast is the local vzdump retention default (newest N restore points kept).
@@ -392,26 +565,106 @@ func (b BackupConfig) BackupTarget() string {
 	return defaultBackupTarget
 }
 
-// Default scratch VMID band + restore-test cadence.
+// Default scratch VMID band + the two R-86 restore-test knobs.
 const (
-	defaultScratchVMIDMin     = 990000
-	defaultScratchVMIDMax     = 990009
-	defaultRestoreTestCadence = 24 * time.Hour
+	defaultScratchVMIDMin = 990000
+	defaultScratchVMIDMax = 990009
+
+	// defaultRestoreTestEvalInterval is how often due-ness is ASKED. It is bounded from BOTH sides,
+	// and neither bound alone would have picked it:
+	//
+	//   FLOOR — what one evaluation costs. MEASURED on demo-felhom, 2026-08-03 (R-86 Part 1.4), via
+	//   --selftest=restore-test-due and by timing the underlying API call directly. One evaluation
+	//   is one storage-content listing per tier:
+	//
+	//	local dir storage (3 archives) .......  18 ms   (18.7 / 18.3 / 18.5)
+	//	PBS tier, WAN to ep0 (2 snapshots) ... 392 ms   (375 / 378 / 424)
+	//	both tiers together .................. 430 ms
+	//
+	//   So cost does NOT set this: even at one evaluation a minute the offsite leg would be ~0.7 %
+	//   of a WAN link's time and ~9 minutes of ep0's day. Worth writing down anyway, because the
+	//   number that would have forbidden a frequent poll is the one nobody measures.
+	//
+	//   CEILING — the retry rate of a FAILING tier. Under a per-archive due-check a tier whose
+	//   restore-test keeps failing stays due, so the evaluation interval IS its retry interval, and
+	//   a retry is a multi-GB restore. Every few minutes would be an incident of its own; the old
+	//   timer retried a broken tier once a day.
+	//
+	// 6h sits between them: four heavy retries a day at the very worst, latency from settle to
+	// proof of at most 6h against a 24h settle lag (so a daily tier is still proved daily), and no
+	// second rate limiter anywhere — the pacing remains one test per archive generation.
+	defaultRestoreTestEvalInterval = 6 * time.Hour
+
+	// defaultRestoreTestSettle is how long an archive must sit before it may be restore-tested.
+	// 24h is R-86's own figure ("~24 h after its own newest archive") and it is what makes the
+	// candidate on a daily tier YESTERDAY's archive rather than the one still being written.
+	defaultRestoreTestSettle = 24 * time.Hour
 )
 
-// RestoreTestCadence returns the configured restore-test interval: a positive value as-is,
-// 0 → 24h default, negative → 0 (disabled).
-func (b BackupConfig) RestoreTestCadence() time.Duration {
+// RestoreTestEvalInterval returns how often the scheduler evaluates due-ness (R-86): a positive
+// value as-is, 0 → the measured default, negative → 0 (disabled).
+//
+// The LEGACY `restore_test_cadence_seconds` keeps exactly one power here, the one a box may be
+// relying on: a NEGATIVE value still disables the automatic restore-test outright. It no longer
+// sets the interval, because the interval no longer decides that a test happens.
+func (b BackupConfig) RestoreTestEvalInterval() time.Duration {
+	if b.RestoreTestCadenceSeconds < 0 {
+		return 0 // legacy DISABLE — preserved verbatim
+	}
 	switch {
-	case b.RestoreTestCadenceSeconds > 0:
-		return time.Duration(b.RestoreTestCadenceSeconds) * time.Second
-	case b.RestoreTestCadenceSeconds < 0:
+	case b.RestoreTestEvalIntervalSeconds > 0:
+		return time.Duration(b.RestoreTestEvalIntervalSeconds) * time.Second
+	case b.RestoreTestEvalIntervalSeconds < 0:
 		return 0 // disabled
 	default:
-		return defaultRestoreTestCadence
+		return defaultRestoreTestEvalInterval
 	}
 }
 
+// RestoreTestSpace returns the restore-test's space margin (R-672): factor (≥ 1) and reserve bytes.
+// Unset or out-of-range → 1.2 and 5 GiB.
+func (b BackupConfig) RestoreTestSpace() (factor float64, reserveBytes int64) {
+	factor = b.RestoreTestSpaceFactor
+	if factor < 1 {
+		factor = 1.2
+	}
+	reserveBytes = int64(b.RestoreTestSpaceReserveGiB * float64(1<<30))
+	if reserveBytes <= 0 {
+		reserveBytes = 5 << 30
+	}
+	return factor, reserveBytes
+}
+
+// RestoreTestSettle returns how long an archive must have sat before it is a restore-test
+// candidate (R-86): a positive value as-is, negative → 0 (no settle requirement), 0 → the default.
+//
+// WHAT HAPPENED TO THE OLD KEY. A box that set `restore_test_cadence_seconds` to a positive value
+// was expressing "how long may pass between a backup and the confidence that it restores". That
+// quantity survives R-86 as the SETTLE LAG, so a positive legacy value seeds this rather than being
+// dropped or silently repurposed as the evaluation interval — and the daemon says so at start-up
+// (see RestoreTestLegacyCadenceInUse). It is deliberately not carried into the evaluation interval:
+// a box that set 72h to spare a weak endpoint would otherwise get a 72h-latency due-check, whereas
+// what it actually wanted — fewer heavy restores — is what per-archive due-ness already gives it.
+func (b BackupConfig) RestoreTestSettle() time.Duration {
+	switch {
+	case b.RestoreTestSettleSeconds > 0:
+		return time.Duration(b.RestoreTestSettleSeconds) * time.Second
+	case b.RestoreTestSettleSeconds < 0:
+		return 0 // explicitly no settle requirement
+	case b.RestoreTestCadenceSeconds > 0:
+		return time.Duration(b.RestoreTestCadenceSeconds) * time.Second // legacy seeding
+	default:
+		return defaultRestoreTestSettle
+	}
+}
+
+// RestoreTestLegacyCadenceInUse reports whether the deprecated key is what is deciding the settle
+// lag, so the daemon can name both replacements ONCE at start-up. A config key that changed meaning
+// without saying so is exactly the silent repurposing §8.3 forbids.
+func (b BackupConfig) RestoreTestLegacyCadenceInUse() bool {
+	return b.RestoreTestCadenceSeconds > 0 && b.RestoreTestSettleSeconds == 0
+}
+
 // PBSVerifyCadence returns the verify-loop interval: positive as-is, 0 → 6h default,
 // negative → 0 (disabled).
 func (b BackupConfig) PBSVerifyCadence() time.Duration {
@@ -657,6 +910,8 @@ func applyEnv(cfg *Config) {
 		cfg.Backup.RestoreStorage = v
 	}
 	cfg.Backup.RestoreTestCadenceSeconds = envInt("FELHOM_AGENT_BACKUP_RESTORE_TEST_CADENCE_SECONDS", cfg.Backup.RestoreTestCadenceSeconds)
+	cfg.Backup.RestoreTestEvalIntervalSeconds = envInt("FELHOM_AGENT_BACKUP_RESTORE_TEST_EVAL_INTERVAL_SECONDS", cfg.Backup.RestoreTestEvalIntervalSeconds)
+	cfg.Backup.RestoreTestSettleSeconds = envInt("FELHOM_AGENT_BACKUP_RESTORE_TEST_SETTLE_SECONDS", cfg.Backup.RestoreTestSettleSeconds)
 }
 
 // envInt overlays an int env var, keeping cur (with a stderr warning) on parse

internal/config/config_test.go

@@ -171,3 +171,46 @@ func TestDeploymentModeEnvOverlay(t *testing.T) {
 		t.Errorf("env overlay did not set deployment_mode: %q", cfg.DeploymentMode)
 	}
 }
+
+// R-50: the island NIC fields are all-or-nothing and the guest addr must be a CIDR. A half-set or
+// malformed island must fail at config load (a botched install) rather than silently fall back to
+// LAN-only, which would leave a guest with an island bind and no island NIC — the exact silent break
+// R-50 exists to kill. Covers LocalAPIConfig.Validate + IslandEnabled.
+func TestLocalAPIConfig_IslandValidation(t *testing.T) {
+	base := LocalAPIConfig{Enable: true, ListenAddr: "169.254.253.1:8443"}
+
+	// both empty → fine (pre-R-50 default), IslandEnabled false
+	if err := base.Validate(); err != nil {
+		t.Errorf("no island config must validate: %v", err)
+	}
+	if base.IslandEnabled() {
+		t.Errorf("IslandEnabled must be false when unset")
+	}
+	// both set, valid CIDR → fine, IslandEnabled true
+	ok := base
+	ok.IslandBridge, ok.IslandGuestAddr = "vmbr9", "169.254.253.2/30"
+	if err := ok.Validate(); err != nil {
+		t.Errorf("valid island config must validate: %v", err)
+	}
+	if !ok.IslandEnabled() {
+		t.Errorf("IslandEnabled must be true when both set")
+	}
+	// bridge only → rejected (all-or-nothing)
+	half := base
+	half.IslandBridge = "vmbr9"
+	if err := half.Validate(); err == nil {
+		t.Errorf("half-set island (bridge only) must be rejected")
+	}
+	// guest addr only → rejected
+	half2 := base
+	half2.IslandGuestAddr = "169.254.253.2/30"
+	if err := half2.Validate(); err == nil {
+		t.Errorf("half-set island (guest addr only) must be rejected")
+	}
+	// both set but guest addr is not a CIDR → rejected
+	bad := base
+	bad.IslandBridge, bad.IslandGuestAddr = "vmbr9", "169.254.253.2" // missing /30
+	if err := bad.Validate(); err == nil {
+		t.Errorf("island guest addr without a CIDR mask must be rejected")
+	}
+}

internal/escrow/recover.go

@@ -0,0 +1,210 @@
+package escrow
+
+import (
+	"context"
+	"errors"
+	"fmt"
+)
+
+// R-199 links 6→8 — fetch this host's own sealed identity blob, open it with the customer's recovery
+// code R, and hand back EXACTLY ONE field: the offsite restic repository password.
+//
+// WHY ONLY ONE FIELD. The bundle also carries the Cloudflare tunnel token, the PBS access token and
+// the WG private key (see IdentityBundle). The caller in this flow — the in-guest controller, one
+// trust tier down — needs none of them, and returning them would widen the blast radius of a
+// controller compromise for no gain. Narrowing costs nothing here and is not recoverable later.
+//
+// WHY R NEVER TOUCHES DISK. `UnwrapIdentity` stages the BLOB and the recovered plaintext in a
+// `MkdirTemp` that it removes, and feeds R through the pty; R itself is never written. This wrapper
+// keeps that property: it takes R as an argument, passes it straight through, and holds no copy.
+// Callers must clear their own reference (the `R = ""` discipline in cmd/felhom-agent).
+//
+// The errors below are DISTINCT on purpose. "could not fetch", "no blob", "wrong code" and "the blob
+// predates the field" are FOUR different situations for the operator and only one of them is a fault.
+//
+// ⚠ THERE WERE THREE, AND THE FOURTH WAS THE DEFECT (R-224, 2026-08-06). This comment said "three"
+// and named "no blob", "wrong code" and "predates the field" — while a FAILED FETCH was wrapped as an
+// anonymous error and fell through the caller's `default` branch into the wrong-code message. So a
+// hub that could not be reached was reported to the customer as a bad recovery code.
+//
+// Measured live on 2026-08-05 (CAMPAIGN-11 F3): with the hub REJECTed at the appliance's firewall and
+// a CORRECT current recovery code, the customer was told the code did not open their package — in
+// 0.0556 s, when a real unseal costs ~1 s of scrypt. The agent's own log carried the truth the whole
+// time (`escrow: fetching the sealed bundle: hub: transport error: … no route to host`) and the HTTP
+// boundary threw it away.
+//
+// The discriminator therefore has to be a VALUE, not a log line — that is what ErrBundleFetch is.
+
+var (
+	// ErrBundleFetch — the sealed bundle could not be FETCHED (the hub refused, was unreachable, or
+	// the transport failed). **The recovery code was never used**, so nothing about it is known and
+	// nothing may be said about it. Wraps the underlying cause for the operator log; carries no secret.
+	ErrBundleFetch = errors.New("escrow: the sealed bundle could not be fetched")
+	// ErrNoEscrowBlob — the hub holds no sealed bundle for this host. Not a fault: no ceremony has run.
+	ErrNoEscrowBlob = errors.New("escrow: the hub holds no sealed identity bundle for this host (no ceremony has run)")
+	// ErrNoResticPassword — the bundle opened, but carries no repository password. Real and expected
+	// for a pre-fork-4 blob (agent < v0.77.0, 2026-07-09): the field did not exist and CANNOT be
+	// retro-fitted, because R is never retained. Distinguished from a wrong code so the operator is
+	// not sent hunting for a mistyped recovery code that was typed correctly.
+	ErrNoResticPassword = errors.New("escrow: the recovered bundle carries NO offsite repository password (a pre-fork-4 blob — the field did not exist when it was sealed and cannot be retro-fitted)")
+	// ErrCodeOpensRetained — the code did NOT open the package the hub currently holds, and DID open a
+	// RETAINED (earlier) one. R-311.
+	//
+	// ⚠ THIS IS NOT A FAILURE OF THE CUSTOMER'S. It is the single most important distinction on this
+	// path, because until 2026-08-12 it was indistinguishable from a mistype and was reported as one.
+	// The screen could only say "it may be a typo, or it may be an older code, and we cannot tell them
+	// apart from here" — and it could not tell them apart because NOTHING EVER LOOKED. Now something
+	// looks, so the sentence can stop hedging.
+	//
+	// It carries no material and no code: only WHICH earlier package opened, by its supersession date,
+	// which is the one fact the customer needs to recognise it.
+	ErrCodeOpensRetained = errors.New("escrow: the recovery code did not open the CURRENT sealed package, but it DID open a retained earlier one")
+)
+
+// RetainedMatch says which retained package a code opened. Returned inside RetainedOpenedError; it
+// carries no secret — not the code, not the bundle, not the repository password.
+type RetainedMatch struct {
+	// SupersededAt is when this package stopped being the current one (hub-supplied, RFC3339-ish).
+	// It is what the recovery screen shows so the customer can recognise which code they are holding.
+	SupersededAt string
+	// KeyFingerprint is the escrow key fingerprint of that package — operator-log material only.
+	KeyFingerprint string
+	// Index is the hub's position label within ONE response. Not durable; do not persist it.
+	Index int
+	// HasResticPassword is false when the retained package opened but carries no repository password
+	// (a pre-fork-4 seal). The code is still CORRECT; the history behind it still cannot be reopened.
+	// Collapsing this into "recoverable" would repeat R-202's mistake on a new surface.
+	HasResticPassword bool
+}
+
+// RetainedOpenedError wraps ErrCodeOpensRetained with the match. Callers classify with errors.Is on
+// the sentinel and read the detail with errors.As.
+type RetainedOpenedError struct {
+	Match RetainedMatch
+}
+
+func (e *RetainedOpenedError) Error() string {
+	return ErrCodeOpensRetained.Error() + " (superseded_at=" + e.Match.SupersededAt + ")"
+}
+func (e *RetainedOpenedError) Unwrap() error { return ErrCodeOpensRetained }
+
+// BlobFetcher yields this host's own opaque identity-escrow blob. present=false is a clean "none".
+// An interface-free func field keeps this package free of any dependency on the hub client.
+type BlobFetcher func(ctx context.Context) (blob []byte, present bool, err error)
+
+// RetainedBlob is one retained sealed package as the recoverer sees it: opaque bytes plus the labels
+// needed to name it. No secret.
+type RetainedBlob struct {
+	Blob           []byte
+	SupersededAt   string
+	KeyFingerprint string
+	Index          int
+}
+
+// RetainedFetcher yields this host's RETAINED sealed packages, newest-superseded first. An empty
+// slice is a clean "none". R-311.
+type RetainedFetcher func(ctx context.Context) (blobs []RetainedBlob, unopenable int, err error)
+
+// OffsiteKeyRecoverer is the assembled links 6→8. Construct it with a fetcher; call it with R.
+type OffsiteKeyRecoverer struct {
+	Fetch BlobFetcher
+	// FetchRetained is OPTIONAL and consulted ONLY after the current package has refused the code.
+	// nil keeps the pre-R-311 behaviour exactly: a refusal stays a refusal. That is deliberate — an
+	// agent wired without it must not behave differently from one that has no retained packages.
+	FetchRetained RetainedFetcher
+	// MaxRetainedTried bounds the scrypt work a single wrong code can cost. Each attempt is ~1 s of
+	// KDF by design, so an unbounded loop over a long supersession history would turn one wrong code
+	// into a minutes-long hang on the customer's screen. 0 means the built-in default.
+	MaxRetainedTried int
+}
+
+// defaultMaxRetainedTried — six attempts is ~6 s worst case, which is a slow screen and not a hang.
+const defaultMaxRetainedTried = 6
+
+// RecoverOffsiteRepoPassword fetches, unseals and extracts. It returns ONLY the repository password.
+//
+// A WRONG RECOVERY CODE FAILS CLOSED at the scrypt KDF inside UnwrapIdentity — `age -d` exits
+// non-zero and emits no plaintext, so there is no partial result and nothing is written anywhere.
+// That property is the crypto's, not a check here, which is why this function has no "validate R"
+// step to get wrong.
+//
+// NOTHING IS LOGGED BY THIS FUNCTION and no error it returns contains R, the password, or blob bytes.
+func (r OffsiteKeyRecoverer) RecoverOffsiteRepoPassword(ctx context.Context, recoveryCode string) (string, error) {
+	if r.Fetch == nil {
+		return "", fmt.Errorf("escrow: recoverer has no blob fetcher configured")
+	}
+	if recoveryCode == "" {
+		return "", fmt.Errorf("escrow: the recovery code is required")
+	}
+	blob, present, err := r.Fetch(ctx)
+	if err != nil {
+		// R-224: joined with ErrBundleFetch so the caller can classify by VALUE. The cause stays
+		// wrapped for the operator log; neither carries a secret. Before this, the fetch failure was
+		// an anonymous error and the local-api handler's `default` branch reported it to the customer
+		// as a wrong recovery code.
+		return "", fmt.Errorf("%w: %w", ErrBundleFetch, err)
+	}
+	if !present || len(blob) == 0 {
+		return "", ErrNoEscrowBlob
+	}
+	bundle, err := UnwrapIdentityBundle(ctx, blob, recoveryCode)
+	if err != nil {
+		// R-311 — BEFORE calling this a wrong code, ask whether it is the RIGHT code for an EARLIER
+		// package. The engine fails closed identically either way, so the two are indistinguishable
+		// from the unwrap alone; the only way to tell is to try. Until this existed nobody tried, and
+		// the screen said so out loud ("innen nem tudjuk megkülönböztetni őket") — a true sentence
+		// about our own incuriosity, read by the customer as a statement about their code.
+		if m, ok := r.tryRetained(ctx, recoveryCode); ok {
+			return "", &RetainedOpenedError{Match: m}
+		}
+		return "", err // the fail-closed "the recovery code did not unwrap…" message; no secret in it
+	}
+	if bundle.ResticRepoPassword == "" {
+		return "", ErrNoResticPassword
+	}
+	return bundle.ResticRepoPassword, nil
+}
+
+// tryRetained reports whether the code opens one of this host's RETAINED packages, and which.
+//
+// FAILURE HERE IS SILENT AND MEANS "NO", NEVER "YES" and never a different verdict for the caller. A
+// hub that cannot answer, a route an older hub does not have, a malformed blob — each leaves the
+// original refusal standing, unchanged. That is the fail-safe direction: the worst outcome of this
+// function breaking is the behaviour we had before it existed.
+//
+// NOTHING IS LOGGED HERE and no return value carries the code, a bundle or a password.
+func (r OffsiteKeyRecoverer) tryRetained(ctx context.Context, recoveryCode string) (RetainedMatch, bool) {
+	if r.FetchRetained == nil {
+		return RetainedMatch{}, false
+	}
+	blobs, _, err := r.FetchRetained(ctx)
+	if err != nil || len(blobs) == 0 {
+		return RetainedMatch{}, false
+	}
+	limit := r.MaxRetainedTried
+	if limit <= 0 {
+		limit = defaultMaxRetainedTried
+	}
+	for i, rb := range blobs {
+		if i >= limit {
+			break
+		}
+		if len(rb.Blob) == 0 {
+			continue
+		}
+		bundle, uerr := UnwrapIdentityBundle(ctx, rb.Blob, recoveryCode)
+		if uerr != nil {
+			continue // this one is not the customer's; try the next
+		}
+		return RetainedMatch{
+			SupersededAt:   rb.SupersededAt,
+			KeyFingerprint: rb.KeyFingerprint,
+			Index:          rb.Index,
+			// A retained package can itself predate the repository-password field. The code is still
+			// correct and must be told so — but the history behind it still cannot be reopened, and
+			// saying otherwise would be a promise this path cannot keep.
+			HasResticPassword: bundle.ResticRepoPassword != "",
+		}, true
+	}
+	return RetainedMatch{}, false
+}

internal/escrow/recover_retained_test.go

@@ -0,0 +1,230 @@
+package escrow
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"testing"
+)
+
+// R-311 — a correct code for an EARLIER package must stop being reported as a wrong code.
+//
+// These use REAL age crypto, like the R-199 tests beside them, because the whole point is that the
+// two situations are indistinguishable AT THE UNWRAP: both fail closed on the current package. A
+// faked unwrap would prove nothing about the thing that was actually broken.
+
+const testR2 = "another correct horse battery staple sedative anaconda wobbly kingdom placard"
+
+func retainedFetcherFor(blobs ...RetainedBlob) RetainedFetcher {
+	return func(context.Context) ([]RetainedBlob, int, error) { return blobs, 0, nil }
+}
+
+// THE ONE THAT MATTERS. The customer holds the code for a package we superseded. Yesterday this
+// returned the fail-closed refusal and the screen told them to check their typing.
+//
+// RED-PROOF: remove the `if m, ok := r.tryRetained(...)` block from RecoverOffsiteRepoPassword →
+// the wrong-code error returns instead → this FAILS, and the lie is back in exactly those words.
+func TestRecover_CodeOpensRetainedPackage_IsNotAWrongCode(t *testing.T) {
+	ensureAge(t)
+	const oldPW = "aaaa567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+	current := sealBundle(t, IdentityBundle{ResticRepoPassword: "cccc567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}, testR2)
+	retained := sealBundle(t, IdentityBundle{ResticRepoPassword: oldPW}, testR)
+
+	_, err := OffsiteKeyRecoverer{
+		Fetch: fetcherFor(current),
+		FetchRetained: retainedFetcherFor(RetainedBlob{
+			Blob: retained, SupersededAt: "2026-08-12 15:18:55", KeyFingerprint: "7e:a6:af", Index: 0,
+		}),
+	}.RecoverOffsiteRepoPassword(context.Background(), testR) // the OLD code
+
+	if err == nil {
+		t.Fatal("recovery succeeded — it must NOT return a password for a retained package on this path")
+	}
+	if !errors.Is(err, ErrCodeOpensRetained) {
+		t.Fatalf("err = %v, want ErrCodeOpensRetained — a correct code for an earlier package was "+
+			"classified as something else, which is how it became 'check your typing'", err)
+	}
+	var ro *RetainedOpenedError
+	if !errors.As(err, &ro) {
+		t.Fatalf("err does not carry a RetainedOpenedError: %v", err)
+	}
+	if ro.Match.SupersededAt != "2026-08-12 15:18:55" {
+		t.Errorf("SupersededAt = %q — the screen needs this date to name the package", ro.Match.SupersededAt)
+	}
+	if !ro.Match.HasResticPassword {
+		t.Error("HasResticPassword = false, but the retained bundle carried one")
+	}
+	// The error must not leak the code, the password or the bundle.
+	for _, secret := range []string{testR, oldPW} {
+		if containsStr(err.Error(), secret) {
+			t.Fatalf("the error text leaks a secret")
+		}
+	}
+}
+
+// SCENARIO A — the ordinary recovery is untouched, and it must not even ASK for retained packages.
+// If the current package opens, the customer is not in this story at all.
+//
+// RED-PROOF: move the tryRetained call above the successful-unwrap return → the fetcher runs → this
+// FAILS on the "must not be consulted" assertion.
+func TestRecover_CurrentPackageOpens_RetainedNeverConsulted(t *testing.T) {
+	ensureAge(t)
+	const pw = "bbbb567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+	current := sealBundle(t, IdentityBundle{ResticRepoPassword: pw}, testR)
+
+	consulted := false
+	got, err := OffsiteKeyRecoverer{
+		Fetch: fetcherFor(current),
+		FetchRetained: func(context.Context) ([]RetainedBlob, int, error) {
+			consulted = true
+			return nil, 0, nil
+		},
+	}.RecoverOffsiteRepoPassword(context.Background(), testR)
+
+	if err != nil {
+		t.Fatalf("the ordinary recovery broke: %v", err)
+	}
+	if got != pw {
+		t.Fatalf("recovered password is not the sealed one")
+	}
+	if consulted {
+		t.Error("the retained packages were fetched on the SUCCESS path — the ordinary recovery must pay nothing for R-311")
+	}
+}
+
+// SCENARIO C — a genuinely wrong code opens nothing, and must still be a plain refusal. The new
+// branch must not become a way to encourage a customer who mistyped.
+//
+// RED-PROOF: make tryRetained return (RetainedMatch{}, true) unconditionally → a wrong code is
+// reported as opening an earlier package → this FAILS.
+func TestRecover_WrongCode_StaysAPlainRefusal(t *testing.T) {
+	ensureAge(t)
+	current := sealBundle(t, IdentityBundle{ResticRepoPassword: "cccc567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}, testR)
+	retained := sealBundle(t, IdentityBundle{ResticRepoPassword: "dddd567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}, testR2)
+
+	_, err := OffsiteKeyRecoverer{
+		Fetch:         fetcherFor(current),
+		FetchRetained: retainedFetcherFor(RetainedBlob{Blob: retained, SupersededAt: "2026-08-01 00:00:00"}),
+	}.RecoverOffsiteRepoPassword(context.Background(), "totally wrong words that open nothing at all here")
+
+	if err == nil {
+		t.Fatal("a wrong code succeeded")
+	}
+	if errors.Is(err, ErrCodeOpensRetained) {
+		t.Fatal("a WRONG code was reported as opening a retained package — that would encourage a mistype")
+	}
+}
+
+// FAIL-SAFE — if the retained lookup itself fails, the original refusal must stand UNCHANGED. The
+// worst outcome of this feature breaking is the behaviour we had before it.
+//
+// RED-PROOF: make tryRetained propagate the fetch error instead of returning false → the customer
+// gets a new, unexplained failure mode → this FAILS.
+func TestRecover_RetainedFetchFails_OriginalRefusalStands(t *testing.T) {
+	ensureAge(t)
+	current := sealBundle(t, IdentityBundle{ResticRepoPassword: "eeee567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}, testR)
+
+	_, err := OffsiteKeyRecoverer{
+		Fetch: fetcherFor(current),
+		FetchRetained: func(context.Context) ([]RetainedBlob, int, error) {
+			return nil, 0, fmt.Errorf("hub exploded")
+		},
+	}.RecoverOffsiteRepoPassword(context.Background(), testR2)
+
+	if err == nil {
+		t.Fatal("expected a refusal")
+	}
+	if errors.Is(err, ErrCodeOpensRetained) {
+		t.Fatal("a failed retained lookup was reported as 'opens a retained package'")
+	}
+	if containsStr(err.Error(), "hub exploded") {
+		t.Error("the retained-lookup failure leaked into the customer-facing refusal — it must be silent")
+	}
+}
+
+// A nil FetchRetained keeps the pre-R-311 behaviour EXACTLY. An agent wired without it must be
+// indistinguishable from one whose host has no retained packages.
+//
+// RED-PROOF: remove the `if r.FetchRetained == nil` guard → nil-deref panic → this FAILS.
+func TestRecover_NilRetainedFetcher_IsPreR311Behaviour(t *testing.T) {
+	ensureAge(t)
+	current := sealBundle(t, IdentityBundle{ResticRepoPassword: "ffff567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}, testR)
+
+	_, err := OffsiteKeyRecoverer{Fetch: fetcherFor(current)}.RecoverOffsiteRepoPassword(context.Background(), testR2)
+	if err == nil {
+		t.Fatal("expected a refusal")
+	}
+	if errors.Is(err, ErrCodeOpensRetained) {
+		t.Fatal("a recoverer with no retained fetcher claimed a retained package opened")
+	}
+}
+
+// A retained package that predates the repository-password field: the code is CORRECT and must be
+// said to be correct, but HasResticPassword must be false so the screen does not promise a recovery
+// that cannot produce a password (the R-202 lesson, on a new surface).
+//
+// RED-PROOF: hardcode HasResticPassword: true → this FAILS.
+func TestRecover_RetainedOpensButPredatesTheField(t *testing.T) {
+	ensureAge(t)
+	current := sealBundle(t, IdentityBundle{ResticRepoPassword: "1111567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}, testR2)
+	// No ResticRepoPassword at all — the pre-fork-4 shape.
+	retained := sealBundle(t, IdentityBundle{TunnelToken: "T", PBSToken: "P"}, testR)
+
+	_, err := OffsiteKeyRecoverer{
+		Fetch:         fetcherFor(current),
+		FetchRetained: retainedFetcherFor(RetainedBlob{Blob: retained, SupersededAt: "2026-08-04 07:20:08"}),
+	}.RecoverOffsiteRepoPassword(context.Background(), testR)
+
+	if !errors.Is(err, ErrCodeOpensRetained) {
+		t.Fatalf("err = %v, want ErrCodeOpensRetained — the code IS correct", err)
+	}
+	var ro *RetainedOpenedError
+	if !errors.As(err, &ro) {
+		t.Fatalf("no RetainedOpenedError: %v", err)
+	}
+	if ro.Match.HasResticPassword {
+		t.Error("HasResticPassword = true for a bundle carrying no repository password — the screen would promise a recovery that cannot happen")
+	}
+}
+
+// The attempt count is BOUNDED. Each unwrap is ~1 s of scrypt by design, so an unbounded loop turns
+// one wrong code into a minutes-long hang on the customer's screen.
+//
+// RED-PROOF: remove the `if i >= limit { break }` → all 10 are tried → this FAILS on the count.
+func TestRecover_RetainedAttemptsAreBounded(t *testing.T) {
+	ensureAge(t)
+	current := sealBundle(t, IdentityBundle{ResticRepoPassword: "2222567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}, testR)
+	junk := sealBundle(t, IdentityBundle{ResticRepoPassword: "3333567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}, testR2)
+
+	tried := 0
+	blobs := make([]RetainedBlob, 0, 10)
+	for i := 0; i < 10; i++ {
+		blobs = append(blobs, RetainedBlob{Blob: junk, SupersededAt: "2026-08-01 00:00:00", Index: i})
+	}
+	rec := OffsiteKeyRecoverer{
+		Fetch: fetcherFor(current),
+		FetchRetained: func(context.Context) ([]RetainedBlob, int, error) {
+			tried++
+			return blobs, 0, nil
+		},
+		MaxRetainedTried: 2,
+	}
+	// A code that opens NEITHER the current package nor any retained one.
+	if _, err := rec.RecoverOffsiteRepoPassword(context.Background(), "a code that opens nothing whatsoever in this test"); err == nil {
+		t.Fatal("expected a refusal")
+	}
+	if tried != 1 {
+		t.Errorf("the retained list was fetched %d times, want exactly 1", tried)
+	}
+}
+
+func containsStr(hay, needle string) bool {
+	return len(needle) > 0 && len(hay) >= len(needle) && (func() bool {
+		for i := 0; i+len(needle) <= len(hay); i++ {
+			if hay[i:i+len(needle)] == needle {
+				return true
+			}
+		}
+		return false
+	})()
+}

internal/escrow/recover_test.go

@@ -0,0 +1,267 @@
+package escrow
+
+import (
+	"context"
+	"errors"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// R-199 links 6→8, with REAL crypto (age is present on the build/demo host; ensureAge skips
+// elsewhere). These are the unit half of the session's question — "is the repository password
+// actually recoverable from the sealed bundle" — and the live half is the same equality on hardware.
+
+const testR = "correct horse battery staple sedative anaconda wobbly kingdom placard yodel"
+
+func sealBundle(t *testing.T, b IdentityBundle, r string) []byte {
+	t.Helper()
+	blob, err := WrapIdentityBundle(context.Background(), b, r)
+	if err != nil {
+		t.Fatalf("WrapIdentityBundle: %v", err)
+	}
+	return blob
+}
+
+func fetcherFor(blob []byte) BlobFetcher {
+	return func(context.Context) ([]byte, bool, error) { return blob, true, nil }
+}
+
+// Scenario A (unit) — the recovered repository password is BYTE-IDENTICAL to the sealed one, and it
+// is the REPOSITORY password rather than some other field of a bundle that also parses.
+//
+// RED-PROOF: return bundle.PBSToken (or TunnelToken, or WGPrivateKey) instead of
+// bundle.ResticRepoPassword → a plausible-looking bundle yields a non-matching key → this FAILS.
+// That mutation is the shape of the bug that would otherwise ship silently, because every one of
+// those fields is a non-empty string that looks like a secret.
+func TestRecoverOffsiteRepoPassword_ReturnsTheRepositoryPassword(t *testing.T) {
+	ensureAge(t)
+	const repoPW = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+	blob := sealBundle(t, IdentityBundle{
+		TunnelToken:        "TUNNEL-TOKEN-NOT-THE-ANSWER",
+		PBSToken:           "PBS-TOKEN-NOT-THE-ANSWER",
+		WGPrivateKey:       "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=",
+		ResticRepoPassword: repoPW,
+	}, testR)
+
+	got, err := (OffsiteKeyRecoverer{Fetch: fetcherFor(blob)}).RecoverOffsiteRepoPassword(context.Background(), testR)
+	if err != nil {
+		t.Fatalf("recover: %v", err)
+	}
+	if got != repoPW {
+		t.Fatalf("the recovered key is not the sealed repository password (len %d vs %d) — a different "+
+			"field of the bundle was returned", len(got), len(repoPW))
+	}
+	// Belt: it must not be any of the OTHER fields, so a future refactor cannot satisfy the check
+	// above by coincidence.
+	for _, other := range []string{"TUNNEL-TOKEN-NOT-THE-ANSWER", "PBS-TOKEN-NOT-THE-ANSWER", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="} {
+		if got == other {
+			t.Fatalf("the recoverer returned the wrong bundle field")
+		}
+	}
+}
+
+// Scenario B — a WRONG recovery code fails closed, the failure names no secret, and nothing is
+// written. The fail-closed property is the crypto's (age's scrypt KDF), which is why there is no
+// validation step here to get wrong — the test pins that it stays that way.
+func TestRecoverOffsiteRepoPassword_WrongCodeFailsClosed(t *testing.T) {
+	ensureAge(t)
+	const repoPW = "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"
+	blob := sealBundle(t, IdentityBundle{TunnelToken: "t", PBSToken: "p", ResticRepoPassword: repoPW}, testR)
+
+	got, err := (OffsiteKeyRecoverer{Fetch: fetcherFor(blob)}).RecoverOffsiteRepoPassword(context.Background(), "not the recovery code at all")
+	if err == nil {
+		t.Fatal("a wrong recovery code MUST fail — a plausible-but-wrong bundle is the one outcome the design forbids")
+	}
+	if got != "" {
+		t.Fatalf("a failed unseal returned %d bytes — there must be no partial result", len(got))
+	}
+	// The error may name the step; it may never name a secret.
+	for _, secret := range []string{repoPW, testR, "not the recovery code at all"} {
+		if strings.Contains(err.Error(), secret) {
+			t.Fatalf("the failure message leaked a secret: %v", err)
+		}
+	}
+}
+
+// A bundle with no repository password is its OWN answer, not a wrong-code error. Sealed before
+// fork-4 (agent < v0.77.0) the field did not exist; sending the operator to re-check a correctly
+// typed recovery code would be the wrong instruction.
+func TestRecoverOffsiteRepoPassword_PreForkFourBundle(t *testing.T) {
+	ensureAge(t)
+	blob := sealBundle(t, IdentityBundle{TunnelToken: "t", PBSToken: "p"}, testR)
+
+	_, err := (OffsiteKeyRecoverer{Fetch: fetcherFor(blob)}).RecoverOffsiteRepoPassword(context.Background(), testR)
+	if !errors.Is(err, ErrNoResticPassword) {
+		t.Fatalf("a pre-fork-4 bundle must report its own error, got %v", err)
+	}
+}
+
+// Scenario D at this layer — no blob is a clean, distinguishable answer.
+func TestRecoverOffsiteRepoPassword_NoBlob(t *testing.T) {
+	rec := OffsiteKeyRecoverer{Fetch: func(context.Context) ([]byte, bool, error) { return nil, false, nil }}
+	_, err := rec.RecoverOffsiteRepoPassword(context.Background(), testR)
+	if !errors.Is(err, ErrNoEscrowBlob) {
+		t.Fatalf("absent blob must yield ErrNoEscrowBlob, got %v", err)
+	}
+}
+
+// Scenario F — R persists NOWHERE. TMPDIR is redirected into the test's own directory, the unseal is
+// run for real, and the whole tree is then walked: no file may contain R (or the recovered password),
+// and the staging directory the unseal creates must be gone.
+//
+// RED-PROOF: write R to a temp file anywhere in the flow (e.g. add
+// `os.WriteFile(filepath.Join(work,"r"), []byte(recoveryCode), 0o600)` inside UnwrapIdentity before
+// its defer removes the dir — or simply drop that defer and let the plaintext staging survive) → the
+// walk finds it → this FAILS.
+func TestRecoverOffsiteRepoPassword_RLeavesNoTrace(t *testing.T) {
+	ensureAge(t)
+	const repoPW = "1111111111111111111111111111111111111111111111111111111111111111"
+	tmp := t.TempDir()
+	t.Setenv("TMPDIR", tmp) // os.MkdirTemp honours this — every staging dir lands under the walk
+
+	const wrongR = "wrong code entirely"
+	blob := sealBundle(t, IdentityBundle{TunnelToken: "t", PBSToken: "p", ResticRepoPassword: repoPW}, testR)
+	if _, err := (OffsiteKeyRecoverer{Fetch: fetcherFor(blob)}).RecoverOffsiteRepoPassword(context.Background(), testR); err != nil {
+		t.Fatalf("recover: %v", err)
+	}
+	// A failed unseal must leave nothing either — exercise both paths before walking.
+	_, _ = (OffsiteKeyRecoverer{Fetch: fetcherFor(blob)}).RecoverOffsiteRepoPassword(context.Background(), wrongR)
+
+	// THE PRIMARY ASSERTION IS EMPTINESS, not content. A content scan alone is defeatable by a later
+	// call OVERWRITING the leaked file with a different secret — which is exactly how the first
+	// version of this test passed its own red-proof while R sat on disk. Nothing in this test writes
+	// under TMPDIR, so after both calls the tree must contain no files at all.
+	var survivors []string
+	err := filepath.Walk(tmp, func(path string, info os.FileInfo, err error) error {
+		if err != nil || info == nil || info.IsDir() || path == tmp {
+			return nil
+		}
+		survivors = append(survivors, strings.TrimPrefix(path, tmp))
+		return nil
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(survivors) > 0 {
+		t.Fatalf("the unseal left %d file(s) behind under TMPDIR: %v — R, the sealed blob and the "+
+			"recovered plaintext all pass through there and none of them may outlive the call", len(survivors), survivors)
+	}
+	// Defence in depth: any secret that DOES appear anywhere is named, for every code used.
+	_ = filepath.Walk(tmp, func(path string, info os.FileInfo, err error) error {
+		if err != nil || info == nil || info.IsDir() {
+			return nil
+		}
+		body, rerr := os.ReadFile(path)
+		if rerr != nil {
+			return nil
+		}
+		for label, secret := range map[string]string{"R": testR, "a wrong R": wrongR, "the repository password": repoPW} {
+			if strings.Contains(string(body), secret) {
+				t.Errorf("%s survived on disk at %s", label, path)
+			}
+		}
+		return nil
+	})
+	// And the staging directories are gone, not merely free of secrets.
+	entries, _ := os.ReadDir(tmp)
+	for _, e := range entries {
+		if e.IsDir() && strings.HasPrefix(e.Name(), "felhom-idesc-") {
+			t.Fatalf("an unseal staging directory survived: %s", e.Name())
+		}
+	}
+}
+
+// A fetch failure surfaces as a fetch failure, not as a wrong-code error — the operator must not be
+// sent to re-read their recovery code because the hub was unreachable.
+//
+// ⚠ THIS TEST WAS GREEN THROUGHOUT THE DEFECT IT DESCRIBES (R-224, 2026-08-06). Its sentence is
+// exactly right and it did not prevent anything, for two reasons worth keeping:
+//
+//  1. **It asserted the MECHANISM, one layer below the consequence.** It checked this package's error
+//     STRING. The merge happened one layer up, in the local-api handler's `default` branch, which
+//     answered a fetch failure with "the recovery code did not open the sealed bundle". The customer
+//     never sees this string; they see that one. The project's own rule — prefer the test that asserts
+//     the CONSEQUENCE (does the customer get blamed?) over the one that asserts the MECHANISM (is the
+//     error distinct here?) — names this case precisely.
+//  2. **It asserted on TEXT.** `strings.Contains(err.Error(), …)` cannot be consumed by a caller, so
+//     it pinned something no production code could branch on. The distinction it checked was real and
+//     unusable.
+//
+// It now asserts the SENTINEL, which is what the handler branches on, and its consequence-level twin
+// lives in `internal/localapi/escrow_recover_class_test.go` where the status is asserted.
+func TestRecoverOffsiteRepoPassword_FetchErrorIsDistinct(t *testing.T) {
+	rec := OffsiteKeyRecoverer{Fetch: func(context.Context) ([]byte, bool, error) {
+		return nil, false, errors.New("hub: connection refused")
+	}}
+	_, err := rec.RecoverOffsiteRepoPassword(context.Background(), testR)
+	if err == nil || !errors.Is(err, ErrBundleFetch) {
+		t.Fatalf("a fetch failure must classify as ErrBundleFetch, got %v", err)
+	}
+	if errors.Is(err, ErrNoEscrowBlob) || errors.Is(err, ErrNoResticPassword) {
+		t.Fatal("a transport failure must not masquerade as a content verdict")
+	}
+}
+
+// ── R-224 — A FAILED FETCH IS NOT A WRONG CODE ──────────────────────────────────────────────────
+//
+// CAMPAIGN-11 F3 measured the consequence of these two being indistinguishable: with the hub
+// firewalled off and a CORRECT current recovery code, the customer was told the code did not open
+// their package, in 0.0556 s — no unseal was attempted at all.
+//
+// The pair below is the whole point. Asserting only the first would pass with a `return ErrBundleFetch`
+// stuck on every error path, which is the same defect pointing the other way.
+func TestRecoverOffsiteRepoPassword_FetchFailureIsClassifiedAsFetch(t *testing.T) {
+	boom := errors.New("hub: transport error: dial tcp 37.191.56.193:443: connect: no route to host")
+	r := OffsiteKeyRecoverer{Fetch: func(context.Context) ([]byte, bool, error) { return nil, false, boom }}
+
+	_, err := r.RecoverOffsiteRepoPassword(context.Background(), testR)
+	if err == nil {
+		t.Fatal("a failing fetch must return an error")
+	}
+	// RED-PROOF: drop the `%w: %w` join in RecoverOffsiteRepoPassword (return the bare wrapped cause,
+	// as it was before R-224) → this FAILS, and the local-api handler falls back to the wrong-code
+	// message exactly as it did on 2026-08-05.
+	if !errors.Is(err, ErrBundleFetch) {
+		t.Fatalf("a failed fetch must classify as ErrBundleFetch, got %v", err)
+	}
+	// The underlying cause survives for the operator log.
+	if !errors.Is(err, boom) {
+		t.Fatalf("the fetch cause must stay wrapped for the operator, got %v", err)
+	}
+	// And it must NOT be mistaken for either of the bundle-content situations.
+	if errors.Is(err, ErrNoEscrowBlob) || errors.Is(err, ErrNoResticPassword) {
+		t.Fatalf("a transport failure is neither of the bundle-content errors: %v", err)
+	}
+}
+
+// The other half: a genuinely wrong code must NOT classify as a fetch failure, or the fix trades one
+// misattribution for its mirror image and the customer is told the hub is down when they mistyped.
+func TestRecoverOffsiteRepoPassword_WrongCodeIsNotAFetchFailure(t *testing.T) {
+	ensureAge(t)
+	blob := sealBundle(t, IdentityBundle{ResticRepoPassword: "0123456789abcdef"}, testR)
+	r := OffsiteKeyRecoverer{Fetch: fetcherFor(blob)}
+
+	_, err := r.RecoverOffsiteRepoPassword(context.Background(),
+		"wrong horse battery staple sedative anaconda wobbly kingdom placard yodel")
+	if err == nil {
+		t.Fatal("a wrong recovery code must fail closed")
+	}
+	if errors.Is(err, ErrBundleFetch) {
+		t.Fatalf("a wrong code must NOT classify as a fetch failure, got %v", err)
+	}
+}
+
+// A clean "the hub holds nothing" keeps its own identity too — it is not a fetch failure, and the
+// customer must not be told the hub was unreachable when it answered perfectly well.
+func TestRecoverOffsiteRepoPassword_AbsentBlobIsNotAFetchFailure(t *testing.T) {
+	r := OffsiteKeyRecoverer{Fetch: func(context.Context) ([]byte, bool, error) { return nil, false, nil }}
+	_, err := r.RecoverOffsiteRepoPassword(context.Background(), testR)
+	if !errors.Is(err, ErrNoEscrowBlob) {
+		t.Fatalf("an absent blob must stay ErrNoEscrowBlob, got %v", err)
+	}
+	if errors.Is(err, ErrBundleFetch) {
+		t.Fatalf("an absent blob is not a fetch FAILURE, got %v", err)
+	}
+}

internal/guestnet/watchdog_test.go

@@ -548,3 +548,21 @@ func TestClassify_Table(t *testing.T) {
 }
 
 var _ io.Writer = (*bytes.Buffer)(nil)
+
+// A3 (R-50): the guestnet healer is eth0-only and MUST stay blind to the island NIC. A guest on an
+// island host presents eth0 DHCP (the LAN leg the healer owns) PLUS eth1 static (the island). Because
+// parseMode is interface-scoped, adding eth1 static cannot flip eth0's detected mode — so the healer
+// keeps treating eth0 as DHCP and never runs dhclient against the static island NIC (which would
+// sabotage it). This is the verify-only guarantee that let R-50 ship the island NIC without a healer
+// change. Red-proof: make parseMode scan globally instead of per-dev and the eth0 assertion fails.
+func TestParseMode_IslandStaticNICDoesNotConfuseEth0(t *testing.T) {
+	interfaces := "auto lo\niface lo inet loopback\n\n" +
+		"auto eth0\niface eth0 inet dhcp\n\n" +
+		"auto eth1\niface eth1 inet static\n    address 169.254.253.2/30\n"
+	if got := parseMode(interfaces, "eth0"); got != ModeDHCP {
+		t.Errorf("eth0 must classify DHCP even with an island eth1 static present, got %q", got)
+	}
+	if got := parseMode(interfaces, "eth1"); got != ModeStatic {
+		t.Errorf("eth1 (island) must classify static when asked directly (dev-scoped), got %q", got)
+	}
+}

internal/httpx/transport.go

@@ -0,0 +1,59 @@
+// Package httpx holds the one HTTP-transport default this repo may not lose.
+//
+// Every client here pins TLS — PBS and PVE by leaf-cert SHA-256, the hub by an optional CA file —
+// so none of them can use http.DefaultTransport and each hand-rolls its own. Hand-rolling silently
+// discards DefaultTransport's settings, and one of them is load-bearing:
+//
+//	Transport: &http.Transport{TLSClientConfig: tlsCfg}   // IdleConnTimeout == 0 == NO timeout
+//
+// A zero IdleConnTimeout means idle keep-alive connections are retained FOREVER, not "use a sane
+// default". Combined with a client that is rebuilt on a schedule and dropped (pbsTargetsFromPVE
+// builds a fresh pbs.Client per cycle), every cycle strands one connection that nothing will ever
+// close: the abandoned Transport becomes unreachable but its persistConn read-loop goroutine keeps
+// the socket alive, and an unreachable Transport does not close its connections.
+//
+// Measured cost, live: 388 established connections accumulated on ep0's PBS proxy between
+// 2026-08-18 09:51:22Z and 2026-08-20 08:02:13Z — 194 from each of the two boxes, held open on BOTH
+// sides, one per agent poll cycle, on a proxy whose descriptor ceiling is 65536. See R-344 and
+// felhom.eu/documentation/audits/SPIKE-ep0-established-connections-2026-08-20.md.
+package httpx
+
+import (
+	"crypto/tls"
+	"net/http"
+	"time"
+)
+
+// DefaultIdleConnTimeout is how long an idle keep-alive connection is retained before it is closed.
+//
+// It is 90s because that is http.DefaultTransport's own value: the fix for R-344 restores a
+// standard-library default rather than inventing a number, so there is nothing here to tune and
+// nothing to justify. It is comfortably shorter than every cadence that drives these clients (the
+// 15-minute live-snapshot collect and the 6-hour verify loop), so a connection abandoned by one
+// cycle is closed long before the next.
+const DefaultIdleConnTimeout = 90 * time.Second
+
+// NewTransport builds a FRESH *http.Transport pinned to tlsCfg, with the idle-connection timeout
+// applied.
+//
+// Fresh, never shared: each caller pins a different endpoint, and a shared transport would pool
+// connections across differently pinned servers. Reusing http.DefaultTransport for the same reason
+// is not an option — it would drop the pin entirely.
+//
+// idleConnTimeout <= 0 means USE THE DEFAULT. It deliberately does not mean "no timeout": no-timeout
+// is the bug this package exists to prevent, and an unset field must never be able to reintroduce
+// it. Callers pass their configured value straight through; only tests pass a short one.
+//
+// Only IdleConnTimeout is set. The other DefaultTransport settings this transport also lacks
+// (MaxIdleConns, TLSHandshakeTimeout, ExpectContinueTimeout) are deliberately left alone: none of
+// them accumulates anything, every client bounds its whole request with http.Client.Timeout, and
+// widening the change would have made the R-344 measurement unattributable.
+func NewTransport(tlsCfg *tls.Config, idleConnTimeout time.Duration) *http.Transport {
+	if idleConnTimeout <= 0 {
+		idleConnTimeout = DefaultIdleConnTimeout
+	}
+	return &http.Transport{
+		TLSClientConfig: tlsCfg,
+		IdleConnTimeout: idleConnTimeout,
+	}
+}

internal/httpx/transport_test.go

@@ -0,0 +1,74 @@
+package httpx
+
+import (
+	"crypto/tls"
+	"net/http"
+	"testing"
+	"time"
+)
+
+// TestNewTransport_ZeroMeansDefaultNeverForever is the whole point of this package.
+//
+// http.Transport's zero IdleConnTimeout means "retain idle connections FOREVER". Any code path that
+// can reach that zero reintroduces R-344, so an unset, zero or negative value must all land on the
+// default. If someone later "simplifies" NewTransport by passing the argument straight through,
+// this fails.
+func TestNewTransport_ZeroMeansDefaultNeverForever(t *testing.T) {
+	for _, tc := range []struct {
+		name string
+		in   time.Duration
+		want time.Duration
+	}{
+		{"zero", 0, DefaultIdleConnTimeout},
+		{"negative", -time.Hour, DefaultIdleConnTimeout},
+		{"explicit short value (tests)", 50 * time.Millisecond, 50 * time.Millisecond},
+		{"explicit long value", time.Hour, time.Hour},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			got := NewTransport(&tls.Config{MinVersion: tls.VersionTLS12}, tc.in).IdleConnTimeout
+			if got != tc.want {
+				t.Fatalf("IdleConnTimeout = %v, want %v", got, tc.want)
+			}
+			if got == 0 {
+				t.Fatal("IdleConnTimeout is 0 — that is 'never expire', which is the R-344 defect itself")
+			}
+		})
+	}
+}
+
+// TestDefaultIdleConnTimeout_MatchesTheStandardLibrary pins the number to its justification.
+//
+// 90s is not a tuned value; it is what http.DefaultTransport uses. Reading it off the standard
+// library rather than hardcoding 90 means the constant cannot drift away from the reason given for
+// it in the package doc.
+func TestDefaultIdleConnTimeout_MatchesTheStandardLibrary(t *testing.T) {
+	std, ok := http.DefaultTransport.(*http.Transport)
+	if !ok {
+		t.Skip("http.DefaultTransport is not an *http.Transport in this Go build")
+	}
+	if DefaultIdleConnTimeout != std.IdleConnTimeout {
+		t.Fatalf("DefaultIdleConnTimeout = %v but http.DefaultTransport uses %v — the doc comment's justification no longer holds",
+			DefaultIdleConnTimeout, std.IdleConnTimeout)
+	}
+}
+
+// TestNewTransport_IsFreshEveryCall guards the pooling property the pinning relies on.
+//
+// Each caller pins a DIFFERENT endpoint. A shared transport would pool connections across
+// differently pinned servers, so returning a package-level singleton would be a security change
+// dressed as a tidy-up.
+func TestNewTransport_IsFreshEveryCall(t *testing.T) {
+	a := NewTransport(&tls.Config{MinVersion: tls.VersionTLS12}, 0)
+	b := NewTransport(&tls.Config{MinVersion: tls.VersionTLS12}, 0)
+	if a == b {
+		t.Fatal("NewTransport returned the SAME transport twice — connections would be pooled across differently pinned endpoints")
+	}
+}
+
+// TestNewTransport_KeepsTheTLSConfig — the transport gains a field; it must lose nothing.
+func TestNewTransport_KeepsTheTLSConfig(t *testing.T) {
+	cfg := &tls.Config{MinVersion: tls.VersionTLS12, InsecureSkipVerify: true} //nolint:gosec // test only
+	if got := NewTransport(cfg, 0).TLSClientConfig; got != cfg {
+		t.Fatalf("TLSClientConfig = %p, want the config passed in (%p) — the pin would be dropped", got, cfg)
+	}
+}

internal/hub/client.go

@@ -15,6 +15,8 @@ import (
 	"time"
 
 	"gitea.dooplex.hu/admin/felhom-agent/internal/config"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/httpx"
 )
 
 const reportPath = "/api/v1/host-report"
@@ -49,8 +51,11 @@ func NewClient(cfg config.HubConfig, logger *slog.Logger) (*Client, error) {
 		tlsCfg.RootCAs = pool
 	}
 	hc := &http.Client{
-		Timeout:   time.Duration(cfg.TimeoutSeconds) * time.Second,
-		Transport: &http.Transport{TLSClientConfig: tlsCfg},
+		Timeout: time.Duration(cfg.TimeoutSeconds) * time.Second,
+		// R-344, consistency only: this client is built ONCE per process, so it never accumulated
+		// and contributed nothing to the ep0 leak. It carried the same missing default, which over
+		// a tunnel is how one idle connection survives long enough to fail on next use.
+		Transport: httpx.NewTransport(tlsCfg, 0),
 	}
 	return newClient(cfg.URL, cfg.APIKey, cfg.HostID, hc, logger), nil
 }
@@ -307,3 +312,140 @@ func tail(b []byte, max int) string {
 	}
 	return s
 }
+
+// IdentityEscrowResponse mirrors GET /api/v1/hosts/{host_id}/escrow (hub >= v0.94.0, R-199).
+// Present=false is a CLEAN answer, not a fault: the host simply has no sealed bundle yet.
+type IdentityEscrowResponse struct {
+	HostID            string `json:"host_id"`
+	Present           bool   `json:"present"`
+	IdentityEscrowB64 string `json:"identity_escrow_b64"`
+}
+
+// FetchIdentityEscrow reads back THIS host's own opaque identity-escrow blob (R-199 link 6 — the
+// mirror of UploadEscrow, self-scoped server-side by the per-host key). The bytes are ciphertext: they
+// are useless without the customer's recovery code R, which neither the hub nor this agent ever holds.
+//
+// It is the ONLY retrieval this client performs, and it is deliberately narrow — no directive, no
+// K-escrow, no key rotation. The operator-driven DR path (recovery-mode re-enroll) is a different
+// endpoint with a different gate and is not reached from here.
+//
+// Errors are typed (transport vs HTTP) and never include the bearer token. The BLOB is never logged —
+// only its length.
+func (c *Client) FetchIdentityEscrow(ctx context.Context) (*IdentityEscrowResponse, error) {
+	if c.hostID == "" {
+		return nil, fmt.Errorf("hub: FetchIdentityEscrow requires a configured host_id")
+	}
+	url := c.baseURL + "/api/v1/hosts/" + c.hostID + "/escrow"
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+	if err != nil {
+		return nil, fmt.Errorf("hub: building escrow-fetch request: %w", err)
+	}
+	req.Header.Set("Authorization", "Bearer "+c.apiKey)
+	req.Header.Set("Accept", "application/json")
+
+	resp, err := c.hc.Do(req)
+	if err != nil {
+		return nil, &TransportError{Err: err}
+	}
+	defer resp.Body.Close()
+
+	raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
+	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+		return nil, &HTTPError{StatusCode: resp.StatusCode, BodyTail: tail(raw, 256)}
+	}
+	var out IdentityEscrowResponse
+	if err := json.Unmarshal(raw, &out); err != nil {
+		return nil, fmt.Errorf("hub: decoding escrow fetch: %w", err)
+	}
+	return &out, nil
+}
+
+// RetainedEscrowPackage is one RETAINED (superseded) sealed identity package. The blob is ciphertext
+// and is useless without R. `SupersededAt` is the only thing here a human ever sees — it is what lets
+// the recovery screen name WHICH earlier package a code belongs to.
+type RetainedEscrowPackage struct {
+	Index             int    `json:"index"`
+	SupersededAt      string `json:"superseded_at"`
+	KeyFingerprint    string `json:"key_fingerprint"`
+	IdentityEscrowB64 string `json:"identity_escrow_b64"`
+}
+
+// RetainedEscrowResponse mirrors GET /api/v1/hosts/{host_id}/escrow/retained (hub >= v0.103.0, R-311).
+//
+// UnopenableCount is NOT noise. It counts retained packages the hub holds whose key material is absent
+// (every pre-v0.93.0 row): on a box with those and nothing else, a perfectly correct old recovery code
+// opens nothing, and the reason is a defect of ours. A caller that ignores this number will tell such a
+// customer their code is wrong — the exact failure this whole chain exists to stop.
+type RetainedEscrowResponse struct {
+	HostID          string                  `json:"host_id"`
+	Count           int                     `json:"count"`
+	UnopenableCount int                     `json:"unopenable_count"`
+	TruncatedCount  int                     `json:"truncated_count"`
+	Packages        []RetainedEscrowPackage `json:"packages"`
+}
+
+// FetchRetainedIdentityEscrow reads back THIS host's RETAINED sealed identity packages (R-311 —
+// the retained siblings of FetchIdentityEscrow, self-scoped server-side by the same per-host key).
+//
+// SEPARATE FROM FetchIdentityEscrow ON PURPOSE. The ordinary recovery must not pay for this call, and
+// must not fail because of it: the current package is tried first and alone, and this is reached only
+// after that has refused. A hub too old to know this route answers 404, which is a CLEAN "none" here
+// and must never be reported as a failed recovery.
+func (c *Client) FetchRetainedIdentityEscrow(ctx context.Context) (*RetainedEscrowResponse, error) {
+	if c.hostID == "" {
+		return nil, fmt.Errorf("hub: FetchRetainedIdentityEscrow requires a configured host_id")
+	}
+	url := c.baseURL + "/api/v1/hosts/" + c.hostID + "/escrow/retained"
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+	if err != nil {
+		return nil, fmt.Errorf("hub: building retained-escrow request: %w", err)
+	}
+	req.Header.Set("Authorization", "Bearer "+c.apiKey)
+	req.Header.Set("Accept", "application/json")
+
+	resp, err := c.hc.Do(req)
+	if err != nil {
+		return nil, &TransportError{Err: err}
+	}
+	defer resp.Body.Close()
+
+	raw, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
+	if resp.StatusCode == http.StatusNotFound {
+		// A hub older than v0.103.0 has no such route. That is "no retained packages", not a fault —
+		// returning an error here would turn an old hub into a failed recovery on a box whose current
+		// package simply did not open.
+		return &RetainedEscrowResponse{HostID: c.hostID}, nil
+	}
+	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+		return nil, &HTTPError{StatusCode: resp.StatusCode, BodyTail: tail(raw, 256)}
+	}
+	var out RetainedEscrowResponse
+	if err := json.Unmarshal(raw, &out); err != nil {
+		return nil, fmt.Errorf("hub: decoding retained escrow fetch: %w", err)
+	}
+	return &out, nil
+}
+
+// PostOSReport sends the OS-update leg's report after every run (hub v0.130.0): POST /api/v1/hosts/{id}/os-report.
+// Per-host key, self-scoped on the hub. Errors are typed like RegisterWG's and never include the bearer.
+func (c *Client) PostOSReport(ctx context.Context, body []byte) error {
+	if c.hostID == "" {
+		return fmt.Errorf("hub: PostOSReport requires a configured host_id")
+	}
+	req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+"/api/v1/hosts/"+c.hostID+"/os-report", bytes.NewReader(body))
+	if err != nil {
+		return fmt.Errorf("hub: building os-report request: %w", err)
+	}
+	req.Header.Set("Authorization", "Bearer "+c.apiKey)
+	req.Header.Set("Content-Type", "application/json")
+	resp, err := c.hc.Do(req)
+	if err != nil {
+		return &TransportError{Err: err}
+	}
+	defer resp.Body.Close()
+	raw, _ := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
+	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+		return &HTTPError{StatusCode: resp.StatusCode, BodyTail: tail(raw, 256)}
+	}
+	return nil
+}

internal/hub/cloudflared.go

@@ -2,45 +2,111 @@ package hub
 
 import (
 	"context"
-	"os/exec"
+	"fmt"
 	"strings"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
 )
 
-// CloudflaredProber reports the cloudflared tunnel service health. It is a
-// READ-ONLY probe: the agent does NOT manage or restart cloudflared in this slice
-// (that is the tunnel-management slice — this is the seam for it). Injectable so
-// tests use a fake and never exec.
+// Tunnel states the agent reports (R-841, agent v0.141.0). THREE, never two: a probe that could not ask is
+// `unknown`, which the hub never shows as up or down and never alarms on (R-96 rule 3).
+const (
+	TunnelRunning    = "running"     // the cloudflared container runs AND its readiness check says CONNECTED
+	TunnelNotRunning = "not_running" // stopped / exited / absent, or running but NOT connected (Detail says which)
+	TunnelUnknown    = "unknown"     // the probe could not ask (guest down, pct/sudo error, health still starting)
+)
+
+// CloudflaredProber reports the box's tunnel. Injectable so tests use a fake and never exec.
 type CloudflaredProber interface {
-	// Status returns one of: "active" | "inactive" | "failed" | "unknown".
-	Status(ctx context.Context) (string, error)
+	// Status returns one of the Tunnel* states and a short detail (why not_running / why unknown).
+	Status(ctx context.Context) (status, detail string)
 }
 
-// SystemctlProber runs `systemctl is-active cloudflared`. This is NOT a Privileged
-// (root-CLI) op — `is-active` is non-root readable and is not one of the three
-// proven root exceptions, so it does not go through internal/proxmox.Privileged.
-type SystemctlProber struct {
-	Unit string // defaults to "cloudflared"
+// GuestTunnelProber reads the REAL tunnel: the `cloudflared` container in the box's own customer guest.
+//
+// Before v0.141.0 the agent ran `systemctl is-active cloudflared` on the HOST — a unit that does not exist (cloudflared
+// is a guest container, `11-os-updates.md` C8), so every box reported `inactive` (R-841).
+//
+// It uses ONLY the existing sudoers line `pct exec [0-9]* -- docker inspect -f *` (03 §3): the container's state, exit
+// code and Docker health status. The health status comes from the compose health check controller v0.292.0 adds
+// (`cloudflared tunnel --metrics localhost:20241 ready` → /ready: 200 only with ≥ 1 connection). Measured 2026-10-04:
+// with a wrong token the container stays "running" while /ready answers 503 — so the container state alone would lie.
+// A container with no health check (an older controller) is judged on its state alone, and Detail says so.
+type GuestTunnelProber struct {
+	Runner proxmox.Runner
+	// Guests returns the box's customer guest vmids (running pool guests that bind /mnt/felhom-drives).
+	Guests func(ctx context.Context) ([]int, error)
 }
 
-// Status maps `systemctl is-active` output to the report vocabulary. systemctl
-// exits non-zero for inactive/failed, so the output string is authoritative over
-// the exit code; any exec error (binary missing, etc.) maps to "unknown".
-func (p SystemctlProber) Status(ctx context.Context) (string, error) {
-	unit := p.Unit
-	if unit == "" {
-		unit = "cloudflared"
+const tunnelInspect = `{{.State.Status}}|{{.State.ExitCode}}|{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}`
+
+// Status probes every customer guest and reports the worst state (normally there is exactly one guest).
+func (p GuestTunnelProber) Status(ctx context.Context) (string, string) {
+	if p.Runner == nil || p.Guests == nil {
+		return TunnelUnknown, "no probe wired"
 	}
-	out, _ := exec.CommandContext(ctx, "systemctl", "is-active", unit).Output()
-	switch strings.TrimSpace(string(out)) {
-	case "active":
-		return "active", nil
-	case "failed":
-		return "failed", nil
-	case "inactive", "deactivating", "activating":
-		return "inactive", nil
-	case "":
-		return "unknown", nil // no output → systemctl/exec problem
-	default:
-		return "unknown", nil
+	vmids, err := p.Guests(ctx)
+	if err != nil {
+		return TunnelUnknown, "could not list the customer guest: " + err.Error()
 	}
+	if len(vmids) == 0 {
+		return TunnelUnknown, "no running customer guest"
+	}
+	worst, wdetail := "", ""
+	rank := map[string]int{TunnelRunning: 0, TunnelUnknown: 1, TunnelNotRunning: 2}
+	for _, v := range vmids {
+		out, errOut, err := p.Runner.Run(ctx, "/usr/sbin/pct", "exec", fmt.Sprint(v), "--", "docker", "inspect", "-f", tunnelInspect, "cloudflared")
+		st, d := ClassifyTunnel(string(out), string(errOut), err)
+		if len(vmids) > 1 {
+			d = fmt.Sprintf("guest %d: %s", v, d)
+		}
+		if worst == "" || rank[st] > rank[worst] {
+			worst, wdetail = st, d
+		}
+	}
+	return worst, wdetail
+}
+
+// ClassifyTunnel maps one `docker inspect` answer to a state. Pure; pinned by TestClassifyTunnel.
+func ClassifyTunnel(stdout, stderr string, err error) (string, string) {
+	out := strings.TrimSpace(stdout)
+	if err != nil || out == "" {
+		if strings.Contains(stderr, "No such object") || strings.Contains(stderr, "No such container") {
+			return TunnelNotRunning, "no cloudflared container in the guest"
+		}
+		return TunnelUnknown, "could not ask the guest: " + firstLine(stderr, err)
+	}
+	parts := strings.Split(out, "|")
+	if len(parts) != 3 {
+		return TunnelUnknown, "unreadable docker answer: " + out
+	}
+	state, code, health := parts[0], parts[1], parts[2]
+	if state != "running" {
+		return TunnelNotRunning, fmt.Sprintf("container %s, exit code %s", state, code)
+	}
+	switch health {
+	case "healthy":
+		return TunnelRunning, "connected"
+	case "unhealthy":
+		return TunnelNotRunning, "container running but the tunnel is NOT connected (cloudflared /ready fails)"
+	case "starting":
+		return TunnelUnknown, "container running, readiness check still starting"
+	case "none":
+		return TunnelRunning, "container running (no readiness check on this controller — connection not checked)"
+	}
+	return TunnelUnknown, "unknown health state " + health
+}
+
+func firstLine(stderr string, err error) string {
+	s := strings.TrimSpace(stderr)
+	if i := strings.IndexByte(s, '\n'); i >= 0 {
+		s = s[:i]
+	}
+	if s == "" && err != nil {
+		s = err.Error()
+	}
+	if len(s) > 160 {
+		s = s[:160]
+	}
+	return s
 }

internal/hub/cloudflared_test.go

@@ -0,0 +1,67 @@
+package hub
+
+import (
+	"context"
+	"errors"
+	"io"
+	"strings"
+	"testing"
+)
+
+// R-841: the three states from one `docker inspect` answer. Red-proof: map "unhealthy" to running (the container
+// state alone — what a plain "is it running" probe would say) and the "running but not connected" case fails.
+func TestClassifyTunnel(t *testing.T) {
+	cases := []struct {
+		name, out, errOut string
+		err               error
+		want              string
+		detail            string
+	}{
+		{"connected", "running|0|healthy\n", "", nil, TunnelRunning, "connected"},
+		{"running but not connected", "running|0|unhealthy\n", "", nil, TunnelNotRunning, "NOT connected"},
+		{"stopped", "exited|137|unhealthy\n", "", nil, TunnelNotRunning, "exit code 137"},
+		{"absent", "", "Error: No such object: cloudflared", errors.New("exit status 1"), TunnelNotRunning, "no cloudflared container"},
+		{"still starting", "running|0|starting\n", "", nil, TunnelUnknown, "starting"},
+		{"no health check (older controller)", "running|0|none\n", "", nil, TunnelRunning, "connection not checked"},
+		{"guest not running", "", "CT 9201 not running", errors.New("exit status 255"), TunnelUnknown, "could not ask"},
+		{"sudo refused", "", "sudo: a password is required", errors.New("exit status 1"), TunnelUnknown, "could not ask"},
+	}
+	for _, c := range cases {
+		st, d := ClassifyTunnel(c.out, c.errOut, c.err)
+		if st != c.want || !strings.Contains(d, c.detail) {
+			t.Errorf("%s: got %q (%s), want %q (…%s…)", c.name, st, d, c.want, c.detail)
+		}
+	}
+}
+
+type tunnelRunner struct {
+	calls []string
+	out   map[string]string
+}
+
+func (r *tunnelRunner) Run(_ context.Context, name string, args ...string) ([]byte, []byte, error) {
+	line := name + " " + strings.Join(args, " ")
+	r.calls = append(r.calls, line)
+	return []byte(r.out[args[1]]), nil, nil
+}
+func (r *tunnelRunner) RunStdin(ctx context.Context, _ io.Reader, name string, args ...string) ([]byte, []byte, error) {
+	return r.Run(ctx, name, args...)
+}
+
+// The probe uses EXACTLY the existing sudoers shape `pct exec <vmid> -- docker inspect -f <tmpl> cloudflared`, and
+// with no customer guest it is unknown, never down.
+func TestGuestTunnelProber(t *testing.T) {
+	r := &tunnelRunner{out: map[string]string{"9201": "running|0|healthy"}}
+	p := GuestTunnelProber{Runner: r, Guests: func(context.Context) ([]int, error) { return []int{9201}, nil }}
+	if st, d := p.Status(context.Background()); st != TunnelRunning || d != "connected" {
+		t.Fatalf("got %q %q", st, d)
+	}
+	want := "/usr/sbin/pct exec 9201 -- docker inspect -f " + tunnelInspect + " cloudflared"
+	if len(r.calls) != 1 || r.calls[0] != want {
+		t.Fatalf("command = %q, want %q", r.calls, want)
+	}
+	none := GuestTunnelProber{Runner: r, Guests: func(context.Context) ([]int, error) { return nil, nil }}
+	if st, _ := none.Status(context.Background()); st != TunnelUnknown {
+		t.Fatalf("no guest → %q, want unknown", st)
+	}
+}

internal/hub/collect.go

@@ -47,6 +47,22 @@ type RestoreTestReporter interface {
 	RestoreTests(ctx context.Context) []RestoreTest
 }
 
+// ProvenRestoreTestReporter is the DURABLE half of the restore-test signal (R-189).
+//
+// RestoreTestReporter above is backed by an in-memory store whose own comment used to read "lost on
+// restart; the cadence re-populates". That was true while a timer re-tested every tier daily. It
+// stopped being true on 2026-08-03: under per-archive due-ness the agent will not re-test an archive
+// it has already proven, so a proof lost to a restart is not repeated for a whole archive generation
+// — a week on the offsite tier — and the hub reports the tier unproven the entire time.
+//
+// Observed, not predicted: a real 14.5 GB offsite restore passed at 15:25:14, the agent was restarted
+// 2 m 43 s later for a deploy, and the hub logged `0 restore-tests` on the next two reports.
+//
+// (*backup.RestoreTestState).ProvenRestoreTests satisfies this. nil → the merge is a no-op.
+type ProvenRestoreTestReporter interface {
+	ProvenRestoreTests(ctx context.Context) []RestoreTest
+}
+
 // PBSReporter is the slice-6-Phase-B seam the pbs verify loop plugs into (same pattern).
 // Returns the agent's latest-known PBS snapshot inventory + verify-state. nil → empty.
 type PBSReporter interface {
@@ -79,16 +95,20 @@ type Collector struct {
 	storage      StorageObserver
 	backups      BackupReporter
 	restoreTests RestoreTestReporter
+	provenTests  ProvenRestoreTestReporter
 	pbs          PBSReporter
 	temp         TempReader                                    // slice 9: host CPU/chassis temp (nil-safe → nil temp)
 	capProbe     func(ctx context.Context) []capability.Status // v0.44.0: privileged-capability self-check (nil → empty)
 	leafFP       string                                        // v0.48.0: served local-API leaf fp (static per process; "" when local API disabled)
+	addrEnum     AddressEnumerator                             // v0.119.0: host interface enumeration; nil => the REAL one (see collectAddresses)
 	wg           WireguardReporter                             // S3: offsite-tunnel status (nil → stanza omitted)
 	pbsdr        PBSDRReporter                                 // slice 2: PBS DR tier bridge state (nil → stanza omitted)
+	ctrlSup      ControllerSupervisorReporter                  // R-523: in-guest controller supervisor (nil → stanza omitted)
 	guestNet     GuestNetReporter                              // R-54: per-guest network watchdog (nil → stanza omitted)
 	selfUpdate   SelfUpdateReporter                            // D1: agent self-update pending status (nil → false)
 	mgmtPlane    MgmtPlaneReporter                             // G1: management-plane health (nil → stanza omitted)
 	oob          OOBReporter                                   // H1: operator-access health (nil → stanza omitted)
+	backupTarget func() ConfiguredBackupTarget                 // R-109: primary backup tier id (nil → recipe records unknown)
 	hostID       string
 	agentVersion string
 	logger       *slog.Logger
@@ -123,6 +143,30 @@ func (c *Collector) SetTempReader(t TempReader) *Collector {
 	return c
 }
 
+// SetBackupTargetResolver wires the DR recipe to the agent's own backup config (R-109), so the recipe
+// can name WHICH storage holds the local whole-guest archives. Returns the collector for chaining.
+//
+// The resolver MUST report the tier that is IN EFFECT, which is the daemon-start snapshot — NOT the
+// current contents of agent.json. A backup-target move rewrites that file and deliberately does not
+// restart the agent (the E-1 lesson: restarting mid-backup records a spurious failure for a run that
+// succeeded), so between the write and the restart the file names a target no backup is writing to yet.
+// Re-reading the file here — the live-reload shape used for escrow.pbs_storage_id — would make the
+// recipe point at the new storage while every archive still landed on the old one. One state, one
+// owner: the recipe follows what performs the backup.
+func (c *Collector) SetBackupTargetResolver(f func() ConfiguredBackupTarget) *Collector {
+	c.backupTarget = f
+	return c
+}
+
+// configuredBackupTarget consults the resolver. An unwired seam is reported as NOT KNOWN — never as a
+// guess — so the recipe records an explicit unknown instead of a target the agent never verified.
+func (c *Collector) configuredBackupTarget() ConfiguredBackupTarget {
+	if c.backupTarget == nil {
+		return ConfiguredBackupTarget{}
+	}
+	return c.backupTarget()
+}
+
 // SetCapabilityProber wires the privileged-capability self-check (v0.44.0): each collect runs it
 // and attaches the snapshot. nil → the report carries an empty []. Returns the collector for chaining.
 func (c *Collector) SetCapabilityProber(probe func(ctx context.Context) []capability.Status) *Collector {
@@ -152,6 +196,17 @@ func (c *Collector) SetPBSDRReporter(p PBSDRReporter) *Collector {
 	return c
 }
 
+// ControllerSupervisorReporter is the R-523 seam (satisfied by *localapi.Server).
+type ControllerSupervisorReporter interface {
+	ControllerSupervisorStatus(ctx context.Context) *ControllerSupervisorStatus
+}
+
+// SetControllerSupervisorReporter wires the R-523 controller supervisor as a report source (nil-safe).
+func (c *Collector) SetControllerSupervisorReporter(r ControllerSupervisorReporter) *Collector {
+	c.ctrlSup = r
+	return c
+}
+
 // SetGuestNetReporter wires the R-54 guest-network watchdog as a report source (nil-safe → stanza
 // omitted). Returns the collector for chaining.
 func (c *Collector) SetGuestNetReporter(g GuestNetReporter) *Collector {
@@ -225,13 +280,14 @@ func (c *Collector) Collect(ctx context.Context) (*HostReport, error) {
 		PBSSnapshots:   c.collectPBSSnapshots(ctx),
 
 		AuditTail:       []AuditEntry{},
-		Cloudflared:     Cloudflared{Status: c.cloudflaredStatus(ctx)},
+		Cloudflared:     c.cloudflared(ctx),
 		Capabilities:    c.capabilities(ctx),
 		LeafFingerprint: c.leafFP,
+		Addresses:       c.collectAddresses(),
 	}
 	// DR recipe host-half — derived from the just-collected guest/storage/PBS facts (no new reads).
 	// Secret-free by construction (identifiers/intents/sizes/coordinates only).
-	report.DRRecipe = BuildDRRecipeHostHalf(report.Guests, report.StorageTargets, report.PBSSnapshots)
+	report.DRRecipe = BuildDRRecipeHostHalf(report.Guests, report.StorageTargets, report.PBSSnapshots, c.configuredBackupTarget())
 	// S3: offsite-tunnel status stanza (nil reporter = feature disabled → omitted; the pubkey in
 	// it is the operator's revocation-recovery handle).
 	if c.wg != nil {
@@ -241,6 +297,10 @@ func (c *Collector) Collect(ctx context.Context) (*HostReport, error) {
 	if c.pbsdr != nil {
 		report.PBSDR = c.pbsdr.PBSDRStatus(ctx)
 	}
+	// R-523: controller supervisor record (nil reporter = not wired → stanza omitted).
+	if c.ctrlSup != nil {
+		report.ControllerSupervisor = c.ctrlSup.ControllerSupervisorStatus(ctx)
+	}
 	// R-54: guest-network watchdog state (nil reporter = feature not wired → stanza omitted).
 	if c.guestNet != nil {
 		report.GuestNet = c.guestNet.GuestNetStatus(ctx)
@@ -400,16 +460,90 @@ func (c *Collector) collectBackups(ctx context.Context) []Backup {
 	return []Backup{}
 }
 
+// collectRestoreTests merges the in-memory result with the PERSISTED per-tier proofs (R-189).
+//
+// The rule is ONE ENTRY PER TIER, NEWEST WINS, and it falls out of what each source means rather
+// than from a preference between them:
+//
+//   - the in-memory store holds this process's latest run, pass OR fail. A failure exists nowhere
+//     else and must always reach the hub — a failing tier is retried at the next evaluation, so its
+//     record is short-lived by design;
+//   - the persisted state holds the last SUCCESS per tier and survives a restart.
+//
+// Comparing by TestedAt gives the right answer in every case without special-casing: a fresh failure
+// beats an older stored success (the failure is the news), a stored success beats a stale in-memory
+// entry after a restart, and a tier proved twice never appears twice — two entries for one tier would
+// read at the hub as two tests.
+//
+// A tier with no usable persisted proof contributes NOTHING. Reporting an unproven tier as proven
+// would be a worse defect than the one this closes.
 func (c *Collector) collectRestoreTests(ctx context.Context) []RestoreTest {
-	if c.restoreTests == nil {
-		return []RestoreTest{}
+	out := []RestoreTest{}
+	if c.restoreTests != nil {
+		if r := c.restoreTests.RestoreTests(ctx); r != nil {
+			out = append(out, r...)
+		}
 	}
-	if r := c.restoreTests.RestoreTests(ctx); r != nil {
-		return r
+	if c.provenTests == nil {
+		return out
 	}
-	return []RestoreTest{}
+
+	// Index what we already have by tier, keeping the newest per tier.
+	best := map[string]int{} // tier → index into out
+	for i, rt := range out {
+		if rt.SourceTier == "" {
+			continue // untiered entry: never deduped, never overwritten — we cannot say what it is
+		}
+		if j, seen := best[rt.SourceTier]; !seen || newerRestoreTest(rt, out[j]) {
+			best[rt.SourceTier] = i
+		}
+	}
+	for _, p := range c.provenTests.ProvenRestoreTests(ctx) {
+		if p.SourceTier == "" {
+			continue // not usable as a per-tier proof; the state layer already filters these
+		}
+		i, seen := best[p.SourceTier]
+		if !seen {
+			out = append(out, p)
+			best[p.SourceTier] = len(out) - 1
+			continue
+		}
+		if newerRestoreTest(p, out[i]) {
+			out[i] = p
+		}
+	}
+	return out
 }
 
+// newerRestoreTest reports whether a was tested after b. An unparseable or absent timestamp is
+// treated as OLDER, so a malformed entry can never displace a good one.
+func newerRestoreTest(a, b RestoreTest) bool {
+	ta, aok := parseRestoreTestedAt(a.TestedAt)
+	tb, bok := parseRestoreTestedAt(b.TestedAt)
+	if !aok {
+		return false
+	}
+	if !bok {
+		return true
+	}
+	return ta.After(tb)
+}
+
+func parseRestoreTestedAt(s string) (time.Time, bool) {
+	t, err := time.Parse(time.RFC3339, s)
+	if err != nil {
+		return time.Time{}, false
+	}
+	return t.UTC(), true
+}
+
+// SetProvenRestoreTests wires the durable proof source. It is a setter rather than a constructor
+// argument because the persisted state is opened later in main() than the collector is built; the
+// same shape as the other late-wired seams here. **The wiring is asserted by an AST test** — the
+// method it feeds carried a doc comment naming a "host-report gauge" for weeks with no caller at
+// all, and this fix must not become the next instance of that.
+func (c *Collector) SetProvenRestoreTests(p ProvenRestoreTestReporter) { c.provenTests = p }
+
 // collectPBSSnapshots reads the latest PBS snapshot inventory via the seam (nil → empty).
 func (c *Collector) collectPBSSnapshots(ctx context.Context) []PBSSnapshot {
 	if c.pbs == nil {
@@ -421,16 +555,18 @@ func (c *Collector) collectPBSSnapshots(ctx context.Context) []PBSSnapshot {
 	return []PBSSnapshot{}
 }
 
-func (c *Collector) cloudflaredStatus(ctx context.Context) string {
+func (c *Collector) cloudflared(ctx context.Context) Cloudflared {
 	if c.cf == nil {
-		return "unknown"
+		return Cloudflared{Status: TunnelUnknown, Detail: "no probe wired"}
 	}
-	st, err := c.cf.Status(ctx)
-	if err != nil || st == "" {
-		c.logger.Warn("hub: cloudflared probe failed", "err", err)
-		return "unknown"
+	st, d := c.cf.Status(ctx)
+	if st == "" {
+		st = TunnelUnknown
 	}
-	return st
+	if st == TunnelUnknown {
+		c.logger.Debug("hub: tunnel probe could not decide", "detail", d)
+	}
+	return Cloudflared{Status: st, Detail: d}
 }
 
 func percent(used, total int64) float64 {

internal/hub/collect_guestnet_test.go

@@ -16,7 +16,7 @@ func (f fakeGuestNet) GuestNetStatus(context.Context) *GuestNetStatus { return f
 
 func TestCollect_GuestNetOmittedWhenReporterNil(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, fakeObserver{}, nil, nil, nil, "h", "0.92.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, fakeObserver{}, nil, nil, nil, "h", "0.92.0", quietLogger())
 	r, err := c.Collect(context.Background())
 	if err != nil {
 		t.Fatalf("Collect: %v", err)
@@ -38,7 +38,7 @@ func TestCollect_GuestNetOmittedWhenReporterNil(t *testing.T) {
 
 func TestCollect_GuestNetPopulatedWhenWired(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, fakeObserver{}, nil, nil, nil, "h", "0.92.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, fakeObserver{}, nil, nil, nil, "h", "0.92.0", quietLogger())
 	c.SetGuestNetReporter(fakeGuestNet{st: &GuestNetStatus{
 		CheckedAt: "2026-07-21T10:00:00Z",
 		Guests: []GuestNetGuest{{

internal/hub/collect_mgmtplane_test.go

@@ -12,7 +12,7 @@ func (f fakeMgmtPlane) MgmtPlaneStatus(context.Context) *MgmtPlaneStatus { retur
 
 func TestCollect_MgmtPlaneOmittedWhenReporterNil(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, fakeObserver{}, nil, nil, nil, "h", "0.71.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, fakeObserver{}, nil, nil, nil, "h", "0.71.0", quietLogger())
 	r, err := c.Collect(context.Background())
 	if err != nil {
 		t.Fatalf("Collect: %v", err)
@@ -24,7 +24,7 @@ func TestCollect_MgmtPlaneOmittedWhenReporterNil(t *testing.T) {
 
 func TestCollect_MgmtPlanePopulatedWhenWired(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, fakeObserver{}, nil, nil, nil, "h", "0.71.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, fakeObserver{}, nil, nil, nil, "h", "0.71.0", quietLogger())
 	c.SetMgmtPlaneReporter(fakeMgmtPlane{st: &MgmtPlaneStatus{
 		PrivsepDirOK: true, SshdReachable: true, HealedRecently: true, PrivsepHealedAt: "2026-07-05T16:42:17Z",
 	}})
@@ -47,7 +47,7 @@ func (f fakeOOB) OOBStatus(context.Context) *OOBStatus { return f.st }
 
 func TestCollect_OOBOmittedWhenNil(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, fakeObserver{}, nil, nil, nil, "h", "0.72.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, fakeObserver{}, nil, nil, nil, "h", "0.72.0", quietLogger())
 	r, _ := c.Collect(context.Background())
 	if r.OOB != nil {
 		t.Fatalf("no reporter → oob omitted, got %+v", r.OOB)
@@ -56,7 +56,7 @@ func TestCollect_OOBOmittedWhenNil(t *testing.T) {
 
 func TestCollect_OOBPopulatedWhenWired(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, fakeObserver{}, nil, nil, nil, "h", "0.72.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, fakeObserver{}, nil, nil, nil, "h", "0.72.0", quietLogger())
 	c.SetOOBReporter(fakeOOB{st: &OOBStatus{FelhomSshdActive: true, FelhomSshdPort: 8822, Reachable: true}})
 	r, _ := c.Collect(context.Background())
 	if r.OOB == nil || r.OOB.FelhomSshdPort != 8822 || !r.OOB.Reachable {

internal/hub/collect_test.go

@@ -33,7 +33,7 @@ func TestCollect_StorageTargetsFromObserver(t *testing.T) {
 	obs := fakeObserver{targets: []StorageTarget{
 		{Name: "local-lvm", Type: StorageTypeLVMThin, State: StorageStateAttached, Reachable: true},
 	}}
-	c := NewCollector(px, fakeProber{status: "active"}, obs, nil, nil, nil, "h", "0.5.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, obs, nil, nil, nil, "h", "0.5.0", quietLogger())
 	r, err := c.Collect(context.Background())
 	if err != nil {
 		t.Fatalf("Collect: %v", err)
@@ -45,7 +45,7 @@ func TestCollect_StorageTargetsFromObserver(t *testing.T) {
 
 func TestCollect_StorageObserverErrorDegradesToEmpty(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, fakeObserver{err: errors.New("proxmox down")}, nil, nil, nil, "h", "0.5.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, fakeObserver{err: errors.New("proxmox down")}, nil, nil, nil, "h", "0.5.0", quietLogger())
 	r, err := c.Collect(context.Background())
 	if err != nil {
 		t.Fatalf("a storage observe error must not sink the heartbeat: %v", err)
@@ -64,7 +64,7 @@ func TestCollect_HostAndGuests(t *testing.T) {
 		},
 		cfg: map[int]proxmox.GuestConfig{100: {Cores: 2, Memory: 2048}},
 	}
-	c := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "demo-host-01", "0.3.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "demo-host-01", "0.3.0", quietLogger())
 	r, err := c.Collect(context.Background())
 	if err != nil {
 		t.Fatalf("Collect: %v", err)
@@ -88,7 +88,7 @@ func TestCollect_HostAndGuests(t *testing.T) {
 	if g.Spec.Cores != 2 || g.Spec.MemoryBytes != 2147483648 || g.Spec.DiskBytes != 21474836480 {
 		t.Errorf("spec = %+v", g.Spec)
 	}
-	if r.Cloudflared.Status != "active" {
+	if r.Cloudflared.Status != "running" || r.Cloudflared.Detail != "connected" {
 		t.Errorf("cloudflared = %q", r.Cloudflared.Status)
 	}
 }
@@ -104,7 +104,7 @@ func TestCollect_GuestConfigFailureKeepsStatusOmitsSpec(t *testing.T) {
 		cfg:    map[int]proxmox.GuestConfig{100: {Cores: 2}},
 		cfgErr: map[int]error{200: errors.New("config read failed")},
 	}
-	c := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "h", "0.3.1", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "h", "0.3.1", quietLogger())
 	r, err := c.Collect(context.Background())
 	if err != nil {
 		t.Fatalf("a per-guest failure must NOT fail the whole report: %v", err)
@@ -125,7 +125,7 @@ func TestCollect_GuestConfigFailureKeepsStatusOmitsSpec(t *testing.T) {
 
 func TestCollect_NodeStatusFailureIsHardError(t *testing.T) {
 	px := &fakePx{node: "n", nsErr: errors.New("proxmox down")}
-	c := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "h", "0.3.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "h", "0.3.0", quietLogger())
 	if _, err := c.Collect(context.Background()); err == nil {
 		t.Fatal("NodeStatus failure must be a hard error (no useful report)")
 	}
@@ -133,7 +133,7 @@ func TestCollect_NodeStatusFailureIsHardError(t *testing.T) {
 
 func TestCollect_CloudflaredProbeErrorIsUnknown(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{err: errors.New("no systemctl")}, nil, nil, nil, nil, "h", "0.3.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "", detail: "could not ask"}, nil, nil, nil, nil, "h", "0.3.0", quietLogger())
 	r, err := c.Collect(context.Background())
 	if err != nil {
 		t.Fatalf("cloudflared failure must not be fatal: %v", err)
@@ -153,7 +153,7 @@ func TestCollect_LeafFingerprint(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
 	const fp = "60b5974d586f5f3c8ec41eb998d0f07406178219c36bf6d3ff377570279d8245"
 
-	c := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "h", "0.48.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "h", "0.48.0", quietLogger())
 	c.SetLeafFingerprint(fp)
 	r, err := c.Collect(context.Background())
 	if err != nil {
@@ -164,7 +164,7 @@ func TestCollect_LeafFingerprint(t *testing.T) {
 	}
 
 	// Companion: no SetLeafFingerprint (local API disabled) → empty, never a fabricated value.
-	c2 := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "h", "0.48.0", quietLogger())
+	c2 := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "h", "0.48.0", quietLogger())
 	r2, _ := c2.Collect(context.Background())
 	if r2.LeafFingerprint != "" {
 		t.Fatalf("unset leaf_fingerprint = %q, want empty", r2.LeafFingerprint)

internal/hub/contract_test.go

@@ -45,6 +45,14 @@ func TestHostReport_ContractMatchesGolden(t *testing.T) {
 				State: StorageStateAttached, Reachable: true, MountPath: "/mnt/usb-backup", TotalBytes: 2000000000000,
 				Smart: SmartSummary{Health: SmartUnknown},
 			},
+			// A pbs target so the recipe's pbs coord has a storage.cfg row to resolve its namespace
+			// from (R-106). Without one the fixture produces namespace_state=unknown while the golden
+			// pins a resolved coord — key-set-equal but semantically a fiction.
+			{
+				Name: "felhom-pbs", Type: StorageTypePBS, DurableID: "repo+fp", Content: "backup",
+				State: StorageStateAttached, Reachable: true, PBSNamespace: "felhom-spike",
+				Smart: SmartSummary{Health: SmartUnknown},
+			},
 		},
 		Backups: []Backup{
 			{
@@ -72,9 +80,13 @@ func TestHostReport_ContractMatchesGolden(t *testing.T) {
 		},
 		AuditTail:   []AuditEntry{},
 		Cloudflared: Cloudflared{Status: "active"},
+		// v0.119.0: host addresses. Populated so the bidirectional key-set guard exercises the new
+		// element keys, not just the presence of the array.
+		Addresses: []HostAddress{{Iface: "vmbr0", CIDR: "192.168.0.162/24"}},
 	}
 	// dr_recipe host-half: built from the same guest/storage/pbs facts (the production path).
-	report.DRRecipe = BuildDRRecipeHostHalf(report.Guests, report.StorageTargets, report.PBSSnapshots)
+	report.DRRecipe = BuildDRRecipeHostHalf(report.Guests, report.StorageTargets, report.PBSSnapshots,
+		ConfiguredBackupTarget{StorageID: "usb-backup", Known: true})
 	b, _ := json.Marshal(report)
 	var got map[string]any
 	json.Unmarshal(b, &got)
@@ -97,6 +109,9 @@ func TestHostReport_ContractMatchesGolden(t *testing.T) {
 	assertSameKeys(t, "restore_tests[0]", firstElem(golden["restore_tests"]), firstElem(got["restore_tests"]))
 	// slice-6-Phase-B addition — pbs_snapshots[0] key set.
 	assertSameKeys(t, "pbs_snapshots[0]", firstElem(golden["pbs_snapshots"]), firstElem(got["pbs_snapshots"]))
+	// v0.119.0 addition — addresses[0] key set (iface/cidr), the cross-repo wire for the hub's
+	// Network card.
+	assertSameKeys(t, "addresses[0]", firstElem(golden["addresses"]), firstElem(got["addresses"]))
 
 	// DR-recipe host-half — the agent's secret-free reconstruction-scaffolding section. Assert the
 	// dr_recipe key set + each sub-array's element key set (the cross-repo wire pinned in the golden).
@@ -106,6 +121,7 @@ func TestHostReport_ContractMatchesGolden(t *testing.T) {
 	assertSameKeys(t, "dr_recipe.guests[0]", firstElem(field(grec, "guests")), firstElem(field(srec, "guests")))
 	assertSameKeys(t, "dr_recipe.drives[0]", firstElem(field(grec, "drives")), firstElem(field(srec, "drives")))
 	assertSameKeys(t, "dr_recipe.pve_storage[0]", firstElem(field(grec, "pve_storage")), firstElem(field(srec, "pve_storage")))
+	assertSameKeys(t, "dr_recipe.backup_target", field(grec, "backup_target"), field(srec, "backup_target"))
 }
 
 // field extracts a nested object value from a decoded JSON map (nil if absent/not a map).

internal/hub/dr_recipe.go

@@ -30,6 +30,36 @@ import "sort"
 // ignore-unknown (encoding/json default) for forward-compat, mirroring storage_manifest.
 const DRRecipeVersion = 1
 
+// Recipe field states (R-106/R-109). A recipe is read at the worst possible moment — by an operator
+// rebuilding a machine that is gone — so a field the agent cannot resolve must SAY SO rather than emit
+// a default, an empty string, or a plausible-looking placeholder. A guess read as fact costs more than
+// an admitted gap: it sends the restore at the wrong archive and nothing contradicts it. This is the
+// same cannot-tell-must-not-lie rule R-117 needed a third state for.
+const (
+	DRStateResolved = "resolved"
+	DRStateUnknown  = "unknown"
+)
+
+// Reasons a resolved-value field is unknown. Enum-shaped, never free text, so the wire stays pinnable
+// and TestDRRecipeHostHalf_NoSecrets has a fixed vocabulary to walk.
+const (
+	// DRReasonNoBackupConfig: the collector was built without a backup-config seam, so the agent could
+	// not consult the very config its own scheduler reads. Nothing is guessed.
+	DRReasonNoBackupConfig = "agent_backup_config_unavailable"
+	// DRReasonNoSuchStorage: the configured target id matches no storage this host observes. The id is
+	// still recorded (it IS what the config says) and the state says it could not be corroborated.
+	DRReasonNoSuchStorage = "not_a_known_storage"
+	// DRReasonNoPBSStorage: snapshots exist but no pbs storage was observed, so there is no storage.cfg
+	// row to read the namespace from.
+	DRReasonNoPBSStorage = "no_pbs_storage_observed"
+)
+
+// PBSRootNamespace is how the recipe spells PBS's root namespace. The PBS API spells it as the EMPTY
+// string (and `pct restore --ns root` would name a namespace that does not exist) — "root" is a display
+// convention this wire has always used, kept here so the field's meaning did not change under R-106.
+// Only a box with no `namespace` line in its pbs storage.cfg stanza ever emits it.
+const PBSRootNamespace = "root"
+
 // DRRecipeHostHalf is the agent-emitted half (guest/drive/storage/PBS scaffolding). Derived entirely
 // from facts the report already collects — no new privileged reads.
 type DRRecipeHostHalf struct {
@@ -38,6 +68,45 @@ type DRRecipeHostHalf struct {
 	PBS           *DRPBSCoord    `json:"pbs,omitempty"`
 	Drives        []DRDrive      `json:"drives"`
 	PVEStorage    []DRPVEStorage `json:"pve_storage"`
+	// BackupTarget names WHICH storage holds the local whole-guest archives (R-109). Always present —
+	// its own State field carries "I could not tell", so the section is never simply absent.
+	BackupTarget *DRBackupTarget `json:"backup_target"`
+}
+
+// DRBackupTarget answers the one question pve_storage cannot: of every storage listed there, WHICH one
+// does this box's primary backup tier actually write its whole-guest archives to?
+//
+// Before R-109 the recipe listed each storage's name/type/content and said nothing about the target.
+// That was harmless while the target was the well-known `local`; the 2026-07-28 vzdump-target move
+// ended that. Every box now carries TWO content=backup dir storages — `felhom-backup` (live) and
+// `local` (archives frozen at the move, never refreshed since) — and they are indistinguishable by
+// name, type and content alone. A restorer picking the frozen one gets a guest that restores cleanly
+// and is silently months out of date, which is the worst shape a backup defect can take.
+type DRBackupTarget struct {
+	// State is DRStateResolved | DRStateUnknown. A reader MUST consult it before trusting StorageID:
+	// the id is also recorded in one unknown case (see DRReasonNoSuchStorage).
+	State string `json:"state"`
+	// StorageID is the PVE storage id of the PRIMARY backup tier. Empty only when the config could not
+	// be consulted at all.
+	StorageID string `json:"storage_id,omitempty"`
+	// MountPath is where that storage's archives land on the host — the disambiguation a restorer
+	// actually needs, since it is what separates felhom-backup's /mnt/hdd_1 from local's /var/lib/vz.
+	// "" for a pbs target (no host mount) and when unresolved.
+	MountPath string `json:"mount_path,omitempty"`
+	// Reason is why State is unknown (one of the DRReason* constants); "" when resolved.
+	Reason string `json:"reason,omitempty"`
+}
+
+// ConfiguredBackupTarget is what the agent's own backup config says the PRIMARY tier writes to.
+//
+// Known=false is a REAL state, not a nil-guard: it means the collector was constructed without the
+// backup-config seam (the --selftest one-shots did exactly this before v0.118.0), and the recipe then
+// records unknown instead of inventing a target. Deliberately a struct rather than a `(string, bool)`
+// return — the (value, ok) shape is what made "errors degrade to unknown, never to no-backup"
+// unimplementable in newestArchiveOn, and this field has the same three-way reading.
+type ConfiguredBackupTarget struct {
+	StorageID string
+	Known     bool
 }
 
 // DRGuest is the sizing needed to recreate the LXC at the right size (GuestSpec, already on the wire).
@@ -51,8 +120,21 @@ type DRGuest struct {
 // DRPBSCoord is WHERE the whole-CT snapshot lives — COORDINATES ONLY. The encryption key is escrow-only;
 // the access token is identity-escrow-only. Neither is here.
 type DRPBSCoord struct {
-	RepoID           string `json:"repo_id"`            // the PVE pbs storage id (e.g. "felhom-pbs") — not a token
-	Namespace        string `json:"namespace"`          // PBS namespace the restore targets
+	RepoID string `json:"repo_id"` // the PVE pbs storage id (e.g. "felhom-pbs") — not a token
+	// Namespace is the PBS namespace the restore targets, resolved from the pbs storage's storage.cfg
+	// stanza — the same field `vzdump --storage <pbs>` makes PVE read, so the recipe cannot disagree
+	// with the backup that produced the snapshot. PBSRootNamespace when the box has no namespace
+	// configured; "" when NamespaceState is unknown.
+	//
+	// R-106: this used to come from the listed snapshot's own `ns`, which PBS does not echo per item once
+	// the request is already namespace-scoped via `?ns=` (internal/pbs/client.go). The field was
+	// therefore always empty, ToHub normalised empty → "root", and every per-customer box reported the
+	// root namespace while its backups were really in `demo-hp` / `demo-felhom`.
+	Namespace string `json:"namespace"`
+	// NamespaceState is DRStateResolved | DRStateUnknown — consult it before trusting Namespace.
+	NamespaceState string `json:"namespace_state"`
+	// NamespaceReason is why NamespaceState is unknown; "" when resolved.
+	NamespaceReason  string `json:"namespace_reason,omitempty"`
 	LatestSnapshotID string `json:"latest_snapshot_id"` // most-recent snapshot's backup_id (a coordinate)
 }
 
@@ -83,7 +165,7 @@ const driveIntentEnrolled = "enrolled"
 // it is unit-tested directly (no live reads). drives[] = the user-data external drives (usb/local-dir
 // with a durable-id); pve_storage[] = every storage target (the storage.cfg scaffolding); pbs = the
 // latest PBS snapshot's coordinates; guests[] = each guest's sizing (skip guests with no spec).
-func BuildDRRecipeHostHalf(guests []Guest, targets []StorageTarget, pbs []PBSSnapshot) *DRRecipeHostHalf {
+func BuildDRRecipeHostHalf(guests []Guest, targets []StorageTarget, pbs []PBSSnapshot, backupTarget ConfiguredBackupTarget) *DRRecipeHostHalf {
 	h := &DRRecipeHostHalf{
 		RecipeVersion: DRRecipeVersion,
 		Guests:        []DRGuest{},
@@ -103,11 +185,14 @@ func BuildDRRecipeHostHalf(guests []Guest, targets []StorageTarget, pbs []PBSSna
 		})
 	}
 
-	var pbsRepoID string
+	var pbsRepoID, pbsNamespace string
+	var pbsStorageFound bool
 	for _, t := range targets {
 		h.PVEStorage = append(h.PVEStorage, DRPVEStorage{Name: t.Name, Type: t.Type, Content: t.Content})
-		if t.Type == StorageTypePBS && pbsRepoID == "" {
-			pbsRepoID = t.Name // the pbs storage id is a coordinate, not the key
+		if t.Type == StorageTypePBS && !pbsStorageFound {
+			pbsStorageFound = true
+			pbsRepoID = t.Name            // the pbs storage id is a coordinate, not the key
+			pbsNamespace = t.PBSNamespace // storage.cfg's namespace — "" here means the ROOT namespace
 		}
 		if isUserDataDrive(t) {
 			h.Drives = append(h.Drives, DRDrive{
@@ -119,12 +204,41 @@ func BuildDRRecipeHostHalf(guests []Guest, targets []StorageTarget, pbs []PBSSna
 		}
 	}
 
-	if c := latestPBSCoord(pbs, pbsRepoID); c != nil {
+	h.BackupTarget = resolveBackupTarget(targets, backupTarget)
+
+	if c := latestPBSCoord(pbs, pbsRepoID, pbsNamespace, pbsStorageFound); c != nil {
 		h.PBS = c
 	}
 	return h
 }
 
+// resolveBackupTarget records WHICH storage the primary backup tier writes to (R-109), or records
+// explicitly that it could not tell. Three outcomes, and the two unknowns are deliberately distinct —
+// "I could not read my own config" and "my config names a storage that is not here" send an operator
+// to different places.
+//
+// MountPath prefers the live mount and falls back to the CONFIGURED path: during a rebuild the drive is
+// frequently absent, and when it is, MountPath empties out while ConfigPath is the only thing left that
+// still says which drive the row was about (the R-116 lesson). The storage's absence from the host is a
+// separate signal (E-2's backup_target_absent); it does not make the recipe's answer unknown, because
+// the question here is which storage.cfg row to restore FROM, and that is still known.
+func resolveBackupTarget(targets []StorageTarget, cfg ConfiguredBackupTarget) *DRBackupTarget {
+	if !cfg.Known || cfg.StorageID == "" {
+		return &DRBackupTarget{State: DRStateUnknown, Reason: DRReasonNoBackupConfig}
+	}
+	for _, t := range targets {
+		if t.Name != cfg.StorageID {
+			continue
+		}
+		mount := t.MountPath
+		if mount == "" {
+			mount = t.ConfigPath
+		}
+		return &DRBackupTarget{State: DRStateResolved, StorageID: cfg.StorageID, MountPath: mount}
+	}
+	return &DRBackupTarget{State: DRStateUnknown, StorageID: cfg.StorageID, Reason: DRReasonNoSuchStorage}
+}
+
 // isUserDataDrive selects the external user-data drives the recipe enumerates (felhom-usb / felhom-flash
 // class): a usb or local-dir storage with a filesystem-UUID durable id and a host mount path. local /
 // lvmthin / pbs / nfs / cifs are scaffolding (they land in pve_storage) but are not user-data drives.
@@ -137,16 +251,30 @@ func isUserDataDrive(t StorageTarget) bool {
 
 // latestPBSCoord picks the most-recent snapshot (lexical max of the RFC3339 backup_time) and returns
 // its coordinates. Returns nil when there is no snapshot to target.
-func latestPBSCoord(snaps []PBSSnapshot, repoID string) *DRPBSCoord {
+//
+// The namespace comes from the pbs STORAGE (storage.cfg), never from the snapshot — see DRPBSCoord's
+// Namespace comment for why the snapshot's own field cannot answer it (R-106). storageFound=false with
+// snapshots present is a genuine unknown: something listed snapshots, but there is no storage row to
+// read a namespace from, so the recipe says so rather than defaulting to root.
+func latestPBSCoord(snaps []PBSSnapshot, repoID, namespace string, storageFound bool) *DRPBSCoord {
 	if len(snaps) == 0 {
 		return nil
 	}
 	sorted := append([]PBSSnapshot(nil), snaps...)
 	sort.Slice(sorted, func(i, j int) bool { return sorted[i].BackupTime > sorted[j].BackupTime })
 	latest := sorted[0]
-	return &DRPBSCoord{
+	c := &DRPBSCoord{
 		RepoID:           repoID,
-		Namespace:        latest.Namespace,
 		LatestSnapshotID: latest.BackupID,
+		NamespaceState:   DRStateUnknown,
+		NamespaceReason:  DRReasonNoPBSStorage,
 	}
+	if storageFound {
+		c.NamespaceState, c.NamespaceReason = DRStateResolved, ""
+		// An empty configured namespace is not a missing answer — it IS the root namespace.
+		if c.Namespace = namespace; c.Namespace == "" {
+			c.Namespace = PBSRootNamespace
+		}
+	}
+	return c
 }

internal/hub/dr_recipe_test.go

@@ -1,8 +1,10 @@
 package hub
 
 import (
+	"context"
 	"encoding/json"
 	"regexp"
+	"strings"
 	"testing"
 )
 
@@ -30,7 +32,7 @@ func TestBuildDRRecipeHostHalf(t *testing.T) {
 		{Namespace: "root", BackupID: "9201", BackupTime: "2026-06-16T08:00:00Z"}, // latest
 	}
 
-	h := BuildDRRecipeHostHalf(guests, targets, pbs)
+	h := BuildDRRecipeHostHalf(guests, targets, pbs, ConfiguredBackupTarget{StorageID: "felhom-flash", Known: true})
 
 	if h.RecipeVersion != 1 {
 		t.Errorf("recipe_version=%d, want 1", h.RecipeVersion)
@@ -68,7 +70,8 @@ func TestBuildDRRecipeHostHalf(t *testing.T) {
 
 // TestBuildDRRecipeHostHalf_NoPBS: no snapshots → pbs omitted (nil), no panic.
 func TestBuildDRRecipeHostHalf_NoPBS(t *testing.T) {
-	h := BuildDRRecipeHostHalf(nil, []StorageTarget{{Name: "local", Type: StorageTypeLocal}}, nil)
+	h := BuildDRRecipeHostHalf(nil, []StorageTarget{{Name: "local", Type: StorageTypeLocal}}, nil,
+		ConfiguredBackupTarget{StorageID: "local", Known: true})
 	if h.PBS != nil {
 		t.Errorf("pbs should be nil with no snapshots, got %+v", h.PBS)
 	}
@@ -89,6 +92,7 @@ func TestDRRecipeHostHalf_V1DriveShape(t *testing.T) {
 				MountPath: "/mnt/felhom-usb", TotalBytes: 931 << 30},
 		},
 		nil,
+		ConfiguredBackupTarget{StorageID: "felhom-usb", Known: true},
 	)
 	if len(h.Drives) != 1 {
 		t.Fatalf("want 1 drive, got %d", len(h.Drives))
@@ -124,6 +128,7 @@ func TestDRRecipeHostHalf_NoSecrets(t *testing.T) {
 			{Name: "felhom-usb", Type: StorageTypeUSB, DurableID: "uuid:da9e7089", Role: "bulk-data", MountPath: "/mnt/felhom-usb", TotalBytes: 1},
 		},
 		[]PBSSnapshot{{Namespace: "root", BackupID: "9201", BackupTime: "2026-06-16T08:00:00Z"}},
+		ConfiguredBackupTarget{StorageID: "felhom-usb", Known: true},
 	)
 	b, err := json.Marshal(h)
 	if err != nil {
@@ -132,6 +137,294 @@ func TestDRRecipeHostHalf_NoSecrets(t *testing.T) {
 	assertNoSecretKeys(t, b)
 }
 
+// ---------------------------------------------------------------------------------------------
+// R-106 / R-109 — the recipe records the RESOLVED backup target and the REAL PBS namespace.
+// ---------------------------------------------------------------------------------------------
+
+// capturedDemoFelhomTargets is the storage set demo-felhom really had on 2026-07-30, not an invented
+// one. PROVENANCE — every field was captured, none composed:
+//
+//   - names/types/contents: the pve_storage block of the box's own PRE-FIX recipe, downloaded from the
+//     hub at GET /customers/demo-felhom/dr-recipe.json (agent v0.115.0).
+//   - paths + is_mountpoint + the pbs namespace: `cat /etc/pve/storage.cfg` on felhom-pve, same day —
+//     `dir: local path /var/lib/vz`, `dir: felhom-backup path /mnt/hdd_1 is_mountpoint 1`,
+//     `pbs: felhom-pbs ... namespace demo-felhom`.
+//
+// THE AMBIGUITY THIS PINS IS REAL, and assertBackupCandidateAmbiguity below refuses to let the fixture
+// quietly lose it: `local` and `felhom-backup` BOTH carry content=backup, and since the 2026-07-28
+// vzdump-target move `local` holds archives frozen at that date. Naming the wrong one restores a guest
+// that is silently months stale.
+func capturedDemoFelhomTargets() []StorageTarget {
+	return []StorageTarget{
+		{Name: "local-lvm", Type: StorageTypeLVMThin, DurableID: "pve/data", Content: "images,rootdir"},
+		{
+			Name: "felhom-backup", Type: StorageTypeLocalDir, Content: "backup",
+			DurableID: "uuid:47a3361a-91e0-4831-a69d-27f540ed3f48",
+			MountPath: "/mnt/hdd_1", ConfigPath: "/mnt/hdd_1", TotalBytes: 983351140352,
+		},
+		{
+			Name: "felhom-pbs", Type: StorageTypePBS, Content: "backup",
+			DurableID: "repo+fp", PBSNamespace: "demo-felhom",
+		},
+		// The decoy: same content, plausible name, historically THE vzdump target. ConfigPath only —
+		// `local` lives on the LVM root and is not its own mount, so the observer leaves MountPath empty.
+		{Name: "local", Type: StorageTypeLocal, Content: "backup,import,vztmpl,iso", ConfigPath: "/var/lib/vz"},
+	}
+}
+
+// capturedDemoFelhomSnapshots mirrors what the box's pre-fix recipe carried: latest_snapshot_id "9201".
+// Namespace is deliberately EMPTY on every element — that is exactly what the PBS API returns once the
+// list is namespace-scoped via `?ns=`, and it is the input that used to become the bogus "root".
+func capturedDemoFelhomSnapshots() []PBSSnapshot {
+	return []PBSSnapshot{
+		{Namespace: "", BackupID: "9201", BackupTime: "2026-07-29T22:00:00Z"},
+		{Namespace: "", BackupID: "9201", BackupTime: "2026-07-30T22:00:00Z"}, // latest
+	}
+}
+
+// assertBackupCandidateAmbiguity fails if the fixture stopped containing TWO plausible content=backup
+// storages. Without this the consequence test below could pass on a fixture with only one candidate —
+// which is precisely the hollow shape that let two defects ship green earlier in this arc.
+func assertBackupCandidateAmbiguity(t *testing.T, h *DRRecipeHostHalf) {
+	t.Helper()
+	var candidates []string
+	for _, s := range h.PVEStorage {
+		if strings.Contains(s.Content, "backup") && (s.Type == StorageTypeLocalDir || s.Type == StorageTypeLocal) {
+			candidates = append(candidates, s.Name)
+		}
+	}
+	if len(candidates) < 2 {
+		t.Fatalf("fixture no longer poses the R-109 problem: want >=2 content=backup dir storages, got %v", candidates)
+	}
+}
+
+// TestDRRecipe_BackupTargetNamesTheLiveStorage is THE consequence assertion for R-109: given a box that
+// really carries two content=backup dir storages, the generated recipe names the LIVE one, gives its
+// mountpoint, and does not name the frozen one. Not "the function returned a non-empty string".
+func TestDRRecipe_BackupTargetNamesTheLiveStorage(t *testing.T) {
+	h := BuildDRRecipeHostHalf(nil, capturedDemoFelhomTargets(), capturedDemoFelhomSnapshots(),
+		ConfiguredBackupTarget{StorageID: "felhom-backup", Known: true})
+
+	assertBackupCandidateAmbiguity(t, h)
+
+	bt := h.BackupTarget
+	if bt == nil {
+		t.Fatal("backup_target is absent — the recipe still cannot say where the local archives are (R-109)")
+	}
+	if bt.State != DRStateResolved {
+		t.Errorf("state=%q want %q (reason=%q)", bt.State, DRStateResolved, bt.Reason)
+	}
+	if bt.StorageID != "felhom-backup" {
+		t.Errorf("storage_id=%q — the recipe must name the LIVE target, not %q", bt.StorageID, "felhom-backup")
+	}
+	if bt.MountPath != "/mnt/hdd_1" {
+		t.Errorf("mount_path=%q want /mnt/hdd_1 — the mountpoint is what separates it from local's /var/lib/vz", bt.MountPath)
+	}
+	// Unambiguous: the frozen decoy must not be what the field names.
+	if bt.StorageID == "local" || bt.MountPath == "/var/lib/vz" {
+		t.Errorf("recipe names the FROZEN target (%q at %q) — a restore from it is silently stale", bt.StorageID, bt.MountPath)
+	}
+}
+
+// TestDRRecipe_PBSNamespaceIsThePerCustomerOne is the consequence assertion for R-106: the recipe carries
+// the namespace the box's backups actually live in, resolved from storage.cfg, and specifically NOT the
+// "root" that every box used to report.
+func TestDRRecipe_PBSNamespaceIsThePerCustomerOne(t *testing.T) {
+	h := BuildDRRecipeHostHalf(nil, capturedDemoFelhomTargets(), capturedDemoFelhomSnapshots(),
+		ConfiguredBackupTarget{StorageID: "felhom-backup", Known: true})
+
+	if h.PBS == nil {
+		t.Fatal("pbs coord absent with snapshots present")
+	}
+	if h.PBS.Namespace == PBSRootNamespace {
+		t.Errorf("namespace=%q — this is the R-106 symptom: the snapshot's empty ns normalised to root "+
+			"while the box's backups are in demo-felhom", h.PBS.Namespace)
+	}
+	if h.PBS.Namespace != "demo-felhom" {
+		t.Errorf("namespace=%q want demo-felhom (storage.cfg's `namespace` on the pbs storage)", h.PBS.Namespace)
+	}
+	if h.PBS.NamespaceState != DRStateResolved {
+		t.Errorf("namespace_state=%q want %q (reason=%q)", h.PBS.NamespaceState, DRStateResolved, h.PBS.NamespaceReason)
+	}
+	if h.PBS.RepoID != "felhom-pbs" || h.PBS.LatestSnapshotID != "9201" {
+		t.Errorf("coord drifted: repo=%q snapshot=%q", h.PBS.RepoID, h.PBS.LatestSnapshotID)
+	}
+}
+
+// TestDRRecipe_PBSNamespaceRootIsResolvedNotUnknown: a box with a pbs storage and NO namespace line is
+// genuinely in the root namespace. That is an answer, not a gap — it must read resolved/"root", so the
+// honest root case is never confused with "I could not tell".
+func TestDRRecipe_PBSNamespaceRootIsResolvedNotUnknown(t *testing.T) {
+	h := BuildDRRecipeHostHalf(nil,
+		[]StorageTarget{{Name: "felhom-pbs", Type: StorageTypePBS, Content: "backup", PBSNamespace: ""}},
+		capturedDemoFelhomSnapshots(),
+		ConfiguredBackupTarget{StorageID: "felhom-pbs", Known: true})
+
+	if h.PBS.NamespaceState != DRStateResolved {
+		t.Errorf("namespace_state=%q — an unconfigured namespace IS the root namespace, not an unknown", h.PBS.NamespaceState)
+	}
+	if h.PBS.Namespace != PBSRootNamespace {
+		t.Errorf("namespace=%q want %q", h.PBS.Namespace, PBSRootNamespace)
+	}
+}
+
+// TestDRRecipe_BackupTargetUnknownWhenConfigUnavailable is the WRONG case: the agent could not consult
+// its own backup config. The recipe must say so explicitly and emit NO storage_id key at all — an
+// absent value must not be representable as a plausible-looking answer.
+func TestDRRecipe_BackupTargetUnknownWhenConfigUnavailable(t *testing.T) {
+	h := BuildDRRecipeHostHalf(nil, capturedDemoFelhomTargets(), nil, ConfiguredBackupTarget{})
+
+	bt := h.BackupTarget
+	if bt == nil {
+		t.Fatal("backup_target must be PRESENT and say unknown, not vanish")
+	}
+	if bt.State != DRStateUnknown || bt.Reason != DRReasonNoBackupConfig {
+		t.Errorf("state=%q reason=%q want %q/%q", bt.State, bt.Reason, DRStateUnknown, DRReasonNoBackupConfig)
+	}
+	// Absence recorded as absence: no id, and no id KEY on the wire.
+	if bt.StorageID != "" {
+		t.Errorf("storage_id=%q — an unresolvable target must not be filled in", bt.StorageID)
+	}
+	b, err := json.Marshal(bt)
+	if err != nil {
+		t.Fatal(err)
+	}
+	var keys map[string]json.RawMessage
+	if err := json.Unmarshal(b, &keys); err != nil {
+		t.Fatal(err)
+	}
+	for _, banned := range []string{"storage_id", "mount_path"} {
+		if _, ok := keys[banned]; ok {
+			t.Errorf("unknown backup_target must not carry a %q key; got %s", banned, b)
+		}
+	}
+	// And nothing in it may read as one of the real candidates.
+	for _, decoy := range []string{"felhom-backup", "local", "/var/lib/vz", "/mnt/hdd_1"} {
+		if strings.Contains(string(b), decoy) {
+			t.Errorf("unknown backup_target leaked a plausible value %q: %s", decoy, b)
+		}
+	}
+}
+
+// TestDRRecipe_BackupTargetUnknownWhenStorageMissing: the config names a storage this host does not
+// have. That is unknown for a DIFFERENT reason — and the configured id IS still recorded, because
+// "config says felhom-backup, no such storage here" sends an operator somewhere useful while silence
+// does not.
+func TestDRRecipe_BackupTargetUnknownWhenStorageMissing(t *testing.T) {
+	targets := []StorageTarget{{Name: "local", Type: StorageTypeLocal, Content: "backup", ConfigPath: "/var/lib/vz"}}
+	h := BuildDRRecipeHostHalf(nil, targets, nil, ConfiguredBackupTarget{StorageID: "felhom-backup", Known: true})
+
+	bt := h.BackupTarget
+	if bt.State != DRStateUnknown || bt.Reason != DRReasonNoSuchStorage {
+		t.Errorf("state=%q reason=%q want %q/%q", bt.State, bt.Reason, DRStateUnknown, DRReasonNoSuchStorage)
+	}
+	if bt.StorageID != "felhom-backup" {
+		t.Errorf("storage_id=%q want the CONFIGURED id recorded even though it matched nothing", bt.StorageID)
+	}
+	// It must NOT silently fall back to the only content=backup storage present.
+	if bt.StorageID == "local" || bt.MountPath == "/var/lib/vz" {
+		t.Error("resolution fell back to the wrong storage instead of reporting unknown")
+	}
+}
+
+// TestDRRecipe_PBSNamespaceUnknownWithoutPBSStorage: snapshots exist but no pbs storage was observed, so
+// there is no storage.cfg row to read a namespace from. The recipe must NOT default to root — that
+// default is the entire R-106 defect.
+func TestDRRecipe_PBSNamespaceUnknownWithoutPBSStorage(t *testing.T) {
+	h := BuildDRRecipeHostHalf(nil,
+		[]StorageTarget{{Name: "local", Type: StorageTypeLocal, Content: "backup"}},
+		capturedDemoFelhomSnapshots(),
+		ConfiguredBackupTarget{StorageID: "local", Known: true})
+
+	if h.PBS == nil {
+		t.Fatal("pbs coord should still be emitted (the snapshot id is a real coordinate)")
+	}
+	if h.PBS.NamespaceState != DRStateUnknown || h.PBS.NamespaceReason != DRReasonNoPBSStorage {
+		t.Errorf("namespace_state=%q reason=%q want %q/%q",
+			h.PBS.NamespaceState, h.PBS.NamespaceReason, DRStateUnknown, DRReasonNoPBSStorage)
+	}
+	if h.PBS.Namespace != "" {
+		t.Errorf("namespace=%q — with no storage row to read, the field must be empty, never %q",
+			h.PBS.Namespace, PBSRootNamespace)
+	}
+}
+
+// TestDRRecipe_BackupTargetUsesConfigPathWhenDeviceGone is the DR-shaped case: the recipe is read while
+// the target drive is absent, so MountPath has emptied out. ConfigPath is then the only thing that still
+// says where the archives live (the R-116 lesson) — and the target is still RESOLVED, because which
+// storage.cfg row to restore from is known regardless of whether its device is currently present.
+func TestDRRecipe_BackupTargetUsesConfigPathWhenDeviceGone(t *testing.T) {
+	targets := []StorageTarget{{
+		Name: "felhom-backup", Type: StorageTypeLocalDir, Content: "backup",
+		MountPath: "", ConfigPath: "/mnt/hdd_1", // device gone: observer empties MountPath, keeps ConfigPath
+	}}
+	h := BuildDRRecipeHostHalf(nil, targets, nil, ConfiguredBackupTarget{StorageID: "felhom-backup", Known: true})
+
+	bt := h.BackupTarget
+	if bt.State != DRStateResolved {
+		t.Errorf("state=%q — an absent device does not make the TARGET unknown", bt.State)
+	}
+	if bt.MountPath != "/mnt/hdd_1" {
+		t.Errorf("mount_path=%q want the configured path /mnt/hdd_1", bt.MountPath)
+	}
+}
+
+// fakePBSReporter is a PBSReporter returning fixed snapshots (the verify loop's seam).
+type fakePBSReporter struct{ snaps []PBSSnapshot }
+
+func (f fakePBSReporter) PBSSnapshots(context.Context) []PBSSnapshot { return f.snaps }
+
+// TestCollectDRRecipe_ProductionPath runs the REAL generation path — Collector.Collect(), the method the
+// daemon calls every cycle — rather than BuildDRRecipeHostHalf directly. It is here because both defects
+// this file fixes were invisible to a direct-call test: the namespace one lived in what the observer put
+// on StorageTarget, and the target one lived in whether anything wired the config seam at all. A seam
+// that is correct and never wired is the failure mode this repo has hit four times.
+func TestCollectDRRecipe_ProductionPath(t *testing.T) {
+	px := &fakePx{node: "n", ns: newTestNodeStatus()}
+	obs := fakeObserver{targets: capturedDemoFelhomTargets()}
+	pbsRep := fakePBSReporter{snaps: capturedDemoFelhomSnapshots()}
+
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, obs, nil, nil, pbsRep, "h", "0.118.0", quietLogger())
+	c.SetBackupTargetResolver(func() ConfiguredBackupTarget {
+		return ConfiguredBackupTarget{StorageID: "felhom-backup", Known: true}
+	})
+
+	r, err := c.Collect(context.Background())
+	if err != nil {
+		t.Fatalf("Collect: %v", err)
+	}
+	if r.DRRecipe == nil {
+		t.Fatal("collect produced no dr_recipe")
+	}
+	if bt := r.DRRecipe.BackupTarget; bt == nil || bt.State != DRStateResolved || bt.StorageID != "felhom-backup" {
+		t.Errorf("backup_target through Collect = %+v, want resolved/felhom-backup", bt)
+	}
+	if p := r.DRRecipe.PBS; p == nil || p.Namespace != "demo-felhom" || p.NamespaceState != DRStateResolved {
+		t.Errorf("pbs namespace through Collect = %+v, want demo-felhom/resolved", p)
+	}
+}
+
+// TestCollectDRRecipe_UnwiredSeamReportsUnknown: a Collector built WITHOUT the resolver (every
+// --selftest one-shot did exactly this before v0.118.0) must produce an explicit unknown. This is the
+// test that would have caught shipping the seam without wiring it.
+func TestCollectDRRecipe_UnwiredSeamReportsUnknown(t *testing.T) {
+	px := &fakePx{node: "n", ns: newTestNodeStatus()}
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, fakeObserver{targets: capturedDemoFelhomTargets()},
+		nil, nil, nil, "h", "0.118.0", quietLogger())
+
+	r, err := c.Collect(context.Background())
+	if err != nil {
+		t.Fatalf("Collect: %v", err)
+	}
+	bt := r.DRRecipe.BackupTarget
+	if bt == nil || bt.State != DRStateUnknown || bt.Reason != DRReasonNoBackupConfig {
+		t.Fatalf("unwired resolver must yield unknown/%s, got %+v", DRReasonNoBackupConfig, bt)
+	}
+	if bt.StorageID != "" {
+		t.Errorf("unwired resolver invented a target %q", bt.StorageID)
+	}
+}
+
 // assertNoSecretKeys walks decoded JSON and fails on any object key matching secretNameRe. Shared by
 // the agent boundary assertions. (durable_id/repo_id/latest_snapshot_id are identifiers/coordinates —
 // none match the credential regex.)

internal/hub/hostaddr.go

@@ -0,0 +1,134 @@
+package hub
+
+import (
+	"net"
+	"net/netip"
+	"sort"
+)
+
+// Host addresses (v0.119.0) — "which addresses does this box actually hold?"
+//
+// The hub could not answer that at all: HostMetrics carried node/cpu/mem/disk/load/uptime/temp and
+// no address of any kind, so the LAN IP of a managed host was invisible in every operator surface.
+// Two sources looked like answers and are not: `lan_resolver.host_ip` is an OPTIONAL config value
+// (absent unless that feature is configured), and DeriveHostIP(local_api.listen_addr) yields the
+// R-50 island literal 169.254.253.1 — a link-local address that is the same on every box. Reporting
+// either would have produced a confident wrong answer, which is worse than the blank it replaces.
+//
+// This reads the kernel's own view instead, and it issues NO block I/O (the CLAUDE.md health-check
+// rule): net.Interfaces() is a netlink/procfs read, needs no privilege, and touches no filesystem.
+
+// HostAddress is one routable address the host holds, tagged with the interface carrying it.
+//
+// Deliberately iface+cidr rather than a single `lan_ip`: a Proxmox host legitimately holds several
+// (a management bridge, a tailnet, the WG tunnel), and picking one of them to call "the" LAN IP is a
+// guess the agent is not entitled to make — on a box whose management bridge is not vmbr0 that guess
+// is silently wrong. The agent reports what exists; the hub does the labelling.
+type HostAddress struct {
+	Iface string `json:"iface"` // e.g. "vmbr0", "wg-felhom", "tailscale0"
+	CIDR  string `json:"cidr"`  // e.g. "192.168.0.162/24" — prefix length kept, it is operator-relevant
+}
+
+// ifaceAddrs is one enumerated interface: the ONLY facts the filter needs. Keeping the seam this
+// narrow is what lets the filter be tested against real measured shapes without a network stack.
+type ifaceAddrs struct {
+	Name     string
+	Up       bool
+	Loopback bool
+	CIDRs    []string
+}
+
+// AddressEnumerator returns the host's interfaces. Injectable so the filter can be driven with the
+// shapes measured on real hardware (see hostaddr_test.go) instead of whatever the test box happens
+// to have.
+type AddressEnumerator func() ([]ifaceAddrs, error)
+
+// systemInterfaces is the production enumerator: the kernel's interface table.
+func systemInterfaces() ([]ifaceAddrs, error) {
+	ifaces, err := net.Interfaces()
+	if err != nil {
+		return nil, err
+	}
+	out := make([]ifaceAddrs, 0, len(ifaces))
+	for _, i := range ifaces {
+		e := ifaceAddrs{
+			Name:     i.Name,
+			Up:       i.Flags&net.FlagUp != 0,
+			Loopback: i.Flags&net.FlagLoopback != 0,
+		}
+		// A per-interface error is not fatal: one unreadable interface must not cost the report
+		// every other address (serve-degraded, as everywhere else in the collector).
+		addrs, aerr := i.Addrs()
+		if aerr != nil {
+			out = append(out, e)
+			continue
+		}
+		for _, a := range addrs {
+			e.CIDRs = append(e.CIDRs, a.String())
+		}
+		out = append(out, e)
+	}
+	return out, nil
+}
+
+// filterHostAddresses keeps every GLOBAL UNICAST address on an up, non-loopback interface.
+//
+// IsGlobalUnicast() is the whole rule, and it was chosen by measuring both demo hosts rather than by
+// listing interface names to exclude. It drops, in one predicate:
+//   - loopback (127.0.0.1, ::1)
+//   - IPv6 link-local (fe80::/10) — every bridge carries one, pure noise
+//   - IPv4 link-local (169.254.0.0/16) — which is exactly the R-50 island address on vmbr9, an
+//     identical constant on every box and therefore actively misleading if surfaced
+//
+// It needs NO veth/fwbr/tap denylist: on a Proxmox host that per-guest plumbing carries no IP at
+// all, so it self-excludes by having nothing to report. Verified on demo-felhom and demo-hp —
+// veth9201i0/i1, fwbr*, and the unused NICs all appear in `ip link` and in no `ip addr` output.
+//
+// What survives on a real box: vmbr0's LAN address, wg-felhom's tunnel address, and tailscale0's
+// tailnet addresses. All three are true and useful; none is labelled here.
+func filterHostAddresses(in []ifaceAddrs) []HostAddress {
+	out := []HostAddress{}
+	for _, i := range in {
+		if i.Loopback || !i.Up {
+			continue
+		}
+		for _, c := range i.CIDRs {
+			p, err := netip.ParsePrefix(c)
+			if err != nil {
+				continue // not a CIDR we understand — skip it, never fail the report
+			}
+			if !p.Addr().IsGlobalUnicast() {
+				continue
+			}
+			out = append(out, HostAddress{Iface: i.Name, CIDR: p.String()})
+		}
+	}
+	// Deterministic order so a report diff reflects a real change, not interface-table ordering.
+	sort.Slice(out, func(a, b int) bool {
+		if out[a].Iface != out[b].Iface {
+			return out[a].Iface < out[b].Iface
+		}
+		return out[a].CIDR < out[b].CIDR
+	})
+	return out
+}
+
+// collectAddresses is the collector's entry point. It returns a non-nil slice so the field always
+// marshals as [] — an absent key and "this box has no routable address" must not look alike to the
+// hub, and [] is the honest encoding of the latter.
+func (c *Collector) collectAddresses() []HostAddress {
+	enum := c.addrEnum
+	if enum == nil {
+		// Default to the REAL enumerator, deliberately inverting the nil-reporter-means-off
+		// convention used by the optional stanzas above. Those gate on a config feature; this has
+		// no dependency and no feature flag, so a forgotten wiring call in main.go would produce a
+		// silently empty field — the inert-seam failure this repo has shipped four times.
+		enum = systemInterfaces
+	}
+	ifaces, err := enum()
+	if err != nil {
+		c.logger.Warn("host addresses: interface enumeration failed", "err", err)
+		return []HostAddress{}
+	}
+	return filterHostAddresses(ifaces)
+}

internal/hub/hostaddr_test.go

@@ -0,0 +1,214 @@
+package hub
+
+import (
+	"errors"
+	"log/slog"
+	"strings"
+	"testing"
+)
+
+// The fixtures below are MEASURED, not invented: `ip -o addr show` on demo-felhom (N100) and
+// demo-hp (HP t740) on 2026-07-31, transcribed verbatim including the interfaces that carry no
+// address. That matters — the filter's claim that it needs no veth/fwbr denylist rests on those
+// interfaces genuinely having nothing to report, and a hand-written fixture that omitted them would
+// have proved the claim by assuming it.
+
+// demoFelhomIfaces is demo-felhom's real interface table.
+func demoFelhomIfaces() []ifaceAddrs {
+	return []ifaceAddrs{
+		{Name: "lo", Up: true, Loopback: true, CIDRs: []string{"127.0.0.1/8", "::1/128"}},
+		{Name: "enp1s0", Up: false}, // physical NIC, no address
+		{Name: "wlp2s0", Up: false}, // wifi, no address
+		{Name: "tailscale0", Up: true, CIDRs: []string{
+			"100.70.170.35/32", "fd7a:115c:a1e0::5236:aa24/128", "fe80::4197:26fc:ccba:b0d9/64"}},
+		{Name: "vmbr0", Up: true, CIDRs: []string{"192.168.0.162/24", "fe80::6a1d:efff:fe5d:a664/64"}},
+		{Name: "veth9201i0", Up: true}, // per-guest plumbing — no address
+		{Name: "veth9201i1", Up: true}, // per-guest plumbing — no address
+		{Name: "vmbr9", Up: true, CIDRs: []string{"169.254.253.1/30", "fe80::48d4:f6ff:fe05:2f98/64"}},
+		{Name: "wg-felhom", Up: true, CIDRs: []string{"10.77.0.2/32"}},
+	}
+}
+
+func hasAddr(got []HostAddress, iface, cidr string) bool {
+	for _, a := range got {
+		if a.Iface == iface && a.CIDR == cidr {
+			return true
+		}
+	}
+	return false
+}
+
+func flatten(got []HostAddress) string {
+	var b strings.Builder
+	for _, a := range got {
+		b.WriteString(a.Iface + "=" + a.CIDR + " ")
+	}
+	return b.String()
+}
+
+// The LAN address is the whole point of the feature — it must survive the filter.
+// RED-PROOF 1: drop the `!p.Addr().IsGlobalUnicast()` continue → the vmbr9 + fe80 assertions below
+// go red (the LAN one still passes, which is exactly why the negatives are asserted too).
+func TestFilterHostAddresses_RealHost(t *testing.T) {
+	got := filterHostAddresses(demoFelhomIfaces())
+
+	// --- what MUST be there ---
+	if !hasAddr(got, "vmbr0", "192.168.0.162/24") {
+		t.Fatalf("the LAN address was filtered away — the feature reports nothing: %s", flatten(got))
+	}
+	if !hasAddr(got, "wg-felhom", "10.77.0.2/32") {
+		t.Errorf("the WireGuard address was filtered away: %s", flatten(got))
+	}
+	if !hasAddr(got, "tailscale0", "100.70.170.35/32") {
+		t.Errorf("the tailnet address was filtered away: %s", flatten(got))
+	}
+
+	// --- what MUST NOT be there, each for its own reason ---
+	for _, bad := range []struct{ iface, cidr, why string }{
+		{"lo", "127.0.0.1/8", "loopback is not an address of the host on any network"},
+		{"lo", "::1/128", "IPv6 loopback"},
+		{"vmbr9", "169.254.253.1/30", "the R-50 island literal — IDENTICAL on every box, so surfacing it is actively misleading"},
+		{"vmbr0", "fe80::6a1d:efff:fe5d:a664/64", "IPv6 link-local, one per bridge, pure noise"},
+		{"tailscale0", "fe80::4197:26fc:ccba:b0d9/64", "IPv6 link-local"},
+	} {
+		if hasAddr(got, bad.iface, bad.cidr) {
+			t.Errorf("%s %s must be filtered (%s); got: %s", bad.iface, bad.cidr, bad.why, flatten(got))
+		}
+	}
+
+	// The no-denylist claim: not one veth/physical interface contributed a row.
+	for _, a := range got {
+		if strings.HasPrefix(a.Iface, "veth") || a.Iface == "enp1s0" || a.Iface == "wlp2s0" {
+			t.Errorf("%s produced a row — the fixture says it has no address, so the filter invented one", a.Iface)
+		}
+	}
+}
+
+// demo-hp is different hardware (4 unused NICs, different ordering) and must filter identically —
+// the rule is about address CLASS, not about one box's interface names.
+func TestFilterHostAddresses_SecondHostFiltersIdentically(t *testing.T) {
+	got := filterHostAddresses([]ifaceAddrs{
+		{Name: "lo", Up: true, Loopback: true, CIDRs: []string{"127.0.0.1/8", "::1/128"}},
+		{Name: "enp2s0f0", Up: false}, {Name: "enp1s0f0", Up: false},
+		{Name: "enp1s0f1", Up: false}, {Name: "enp1s0f2", Up: false},
+		{Name: "enp1s0f3", Up: false}, {Name: "wlo1", Up: false},
+		{Name: "tailscale0", Up: true, CIDRs: []string{
+			"100.76.96.79/32", "fd7a:115c:a1e0::ce36:6051/128", "fe80::e06a:ce64:80e1:7821/64"}},
+		{Name: "vmbr0", Up: true, CIDRs: []string{"192.168.0.87/24", "fe80::7ed3:aff:fe77:d976/64"}},
+		{Name: "wg-felhom", Up: true, CIDRs: []string{"10.77.0.3/32"}},
+		{Name: "vmbr9", Up: true, CIDRs: []string{"169.254.253.1/30", "fe80::2484:92ff:fe7d:52a5/64"}},
+		{Name: "veth9201i0", Up: true}, {Name: "veth9201i1", Up: true},
+	})
+	if !hasAddr(got, "vmbr0", "192.168.0.87/24") {
+		t.Fatalf("demo-hp's LAN address was filtered away: %s", flatten(got))
+	}
+	if hasAddr(got, "vmbr9", "169.254.253.1/30") {
+		t.Errorf("demo-hp's island address leaked through: %s", flatten(got))
+	}
+	// The island address is byte-identical on both boxes — the strongest argument for excluding it.
+	if strings.Contains(flatten(got), "169.254.") {
+		t.Errorf("a link-local IPv4 survived: %s", flatten(got))
+	}
+}
+
+// A DOWN interface holding a stale address must not be reported as if the box were reachable there.
+// RED-PROOF 2: drop `|| !i.Up` → this goes red.
+func TestFilterHostAddresses_DownInterfaceExcluded(t *testing.T) {
+	got := filterHostAddresses([]ifaceAddrs{
+		{Name: "vmbr0", Up: true, CIDRs: []string{"192.168.0.162/24"}},
+		{Name: "vmbr1", Up: false, CIDRs: []string{"10.9.9.9/24"}},
+	})
+	if hasAddr(got, "vmbr1", "10.9.9.9/24") {
+		t.Errorf("a DOWN interface's address was reported: %s", flatten(got))
+	}
+	if len(got) != 1 {
+		t.Errorf("want exactly the one up interface, got: %s", flatten(got))
+	}
+}
+
+// Order must be deterministic, or every report diff shows phantom churn.
+func TestFilterHostAddresses_DeterministicOrder(t *testing.T) {
+	a := filterHostAddresses(demoFelhomIfaces())
+	// Same facts, opposite enumeration order.
+	rev := demoFelhomIfaces()
+	for i, j := 0, len(rev)-1; i < j; i, j = i+1, j-1 {
+		rev[i], rev[j] = rev[j], rev[i]
+	}
+	b := filterHostAddresses(rev)
+	if flatten(a) != flatten(b) {
+		t.Errorf("interface-table order changed the report:\n a=%s\n b=%s", flatten(a), flatten(b))
+	}
+}
+
+// A host with nothing routable yields [] and never nil — an absent key and "no addresses" must not
+// look alike on the wire.
+func TestFilterHostAddresses_EmptyIsNonNil(t *testing.T) {
+	got := filterHostAddresses([]ifaceAddrs{{Name: "lo", Up: true, Loopback: true, CIDRs: []string{"127.0.0.1/8"}}})
+	if got == nil {
+		t.Fatal("filter returned nil — it would marshal as null, not []")
+	}
+	if len(got) != 0 {
+		t.Errorf("want no addresses, got %s", flatten(got))
+	}
+}
+
+// A malformed entry is skipped, never fatal — one bad address must not cost the report the others.
+func TestFilterHostAddresses_MalformedSkipped(t *testing.T) {
+	got := filterHostAddresses([]ifaceAddrs{
+		{Name: "vmbr0", Up: true, CIDRs: []string{"not-an-address", "192.168.0.162/24"}},
+	})
+	if len(got) != 1 || !hasAddr(got, "vmbr0", "192.168.0.162/24") {
+		t.Errorf("a malformed sibling address broke the good one: %s", flatten(got))
+	}
+}
+
+// --- the WIRING half: the collector must actually call the filter ---
+
+// The seam defaults to the REAL enumerator, so a forgotten wiring call cannot make this inert.
+// RED-PROOF 3: replace the collectAddresses body with `return []HostAddress{}` → red.
+func TestCollectAddresses_UsesTheInjectedEnumerator(t *testing.T) {
+	c := &Collector{logger: slog.Default()}
+	c.addrEnum = func() ([]ifaceAddrs, error) { return demoFelhomIfaces(), nil }
+	got := c.collectAddresses()
+	if !hasAddr(got, "vmbr0", "192.168.0.162/24") {
+		t.Fatalf("the collector did not run the filter over the enumerator's output: %s", flatten(got))
+	}
+}
+
+// An enumeration failure degrades to [] and a WARN — never a failed report.
+func TestCollectAddresses_EnumerationErrorDegrades(t *testing.T) {
+	c := &Collector{logger: slog.Default()}
+	c.addrEnum = func() ([]ifaceAddrs, error) { return nil, errors.New("netlink is unhappy") }
+	got := c.collectAddresses()
+	if got == nil {
+		t.Fatal("an enumeration error produced nil, which marshals as null")
+	}
+	if len(got) != 0 {
+		t.Errorf("want [] on error, got %s", flatten(got))
+	}
+}
+
+// The production enumerator must return SOMETHING on the machine running the tests, and must not
+// panic. This is the only test that touches the real network stack; it asserts the contract
+// (non-nil, no error, loopback correctly flagged) rather than any specific address, because the
+// test host's addresses are not ours to predict.
+func TestSystemInterfaces_ProductionEnumeratorWorks(t *testing.T) {
+	ifaces, err := systemInterfaces()
+	if err != nil {
+		t.Fatalf("systemInterfaces: %v", err)
+	}
+	if len(ifaces) == 0 {
+		t.Fatal("no interfaces at all — even a container has lo")
+	}
+	var sawLoopback bool
+	for _, i := range ifaces {
+		if i.Loopback {
+			sawLoopback = true
+		}
+	}
+	if !sawLoopback {
+		t.Error("no interface reported the loopback flag — the flag mapping is wrong")
+	}
+	// And the filter must survive real input without panicking.
+	_ = filterHostAddresses(ifaces)
+}

internal/hub/hostmetrics_test.go

@@ -16,7 +16,7 @@ func intp(v int) *int { return &v }
 // HostMetricsNow returns a fresh host block with cpu% from NodeStatus and the temp from the reader.
 func TestHostMetricsNow_PopulatesTemp(t *testing.T) {
 	px := &fakePx{node: "demo-felhom", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "h", "0.14.0", quietLogger()).
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "h", "0.14.0", quietLogger()).
 		SetTempReader(fakeTemp{c: intp(46)})
 	h, err := c.HostMetricsNow(context.Background())
 	if err != nil {
@@ -36,7 +36,7 @@ func TestHostMetricsNow_PopulatesTemp(t *testing.T) {
 // A missing temp sensor gracefully nulls cpu_temp_c without failing the host read.
 func TestHostMetricsNow_GracefulNullTemp(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "h", "0.14.0", quietLogger()).
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "h", "0.14.0", quietLogger()).
 		SetTempReader(fakeTemp{c: nil})
 	h, err := c.HostMetricsNow(context.Background())
 	if err != nil {
@@ -50,7 +50,7 @@ func TestHostMetricsNow_GracefulNullTemp(t *testing.T) {
 // A NodeStatus failure is a hard error (no useful host view).
 func TestHostMetricsNow_NodeStatusErrorIsHard(t *testing.T) {
 	px := &fakePx{node: "n", nsErr: errors.New("proxmox down")}
-	c := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "h", "0.14.0", quietLogger())
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "h", "0.14.0", quietLogger())
 	if _, err := c.HostMetricsNow(context.Background()); err == nil {
 		t.Fatal("NodeStatus failure must be a hard error")
 	}
@@ -59,7 +59,7 @@ func TestHostMetricsNow_NodeStatusErrorIsHard(t *testing.T) {
 // Collect() (the hub report) also carries the temp now — the operator freebie.
 func TestCollect_HostReportCarriesTemp(t *testing.T) {
 	px := &fakePx{node: "n", ns: newTestNodeStatus()}
-	c := NewCollector(px, fakeProber{status: "active"}, nil, nil, nil, nil, "h", "0.14.0", quietLogger()).
+	c := NewCollector(px, fakeProber{status: "running", detail: "connected"}, nil, nil, nil, nil, "h", "0.14.0", quietLogger()).
 		SetTempReader(fakeTemp{c: intp(51)})
 	r, err := c.Collect(context.Background())
 	if err != nil {

internal/hub/mock_test.go

@@ -56,7 +56,7 @@ func (f *fakePx) GuestConfig(ctx context.Context, vmid int) (proxmox.GuestConfig
 // fakeProber is a fake CloudflaredProber.
 type fakeProber struct {
 	status string
-	err    error
+	detail string
 }
 
-func (p fakeProber) Status(ctx context.Context) (string, error) { return p.status, p.err }
+func (p fakeProber) Status(ctx context.Context) (string, string) { return p.status, p.detail }

internal/hub/osupdate_contract_test.go

@@ -0,0 +1,36 @@
+package hub
+
+import (
+	"encoding/json"
+	"os"
+	"testing"
+)
+
+// The os_update block is a cross-repo contract: testdata/desired-state-osupdate.golden.json is byte-identical with
+// felhom.eu/hub/internal/api/testdata (the hub's TestOSUpdate_DesiredBlockMatchesTheGolden proves the hub SERVES
+// it). Here: the agent DECODES every field. A renamed json tag on either side fails one of the two tests.
+func TestOSUpdateGolden_Decodes(t *testing.T) {
+	raw, err := os.ReadFile("testdata/desired-state-osupdate.golden.json")
+	if err != nil {
+		t.Fatal(err)
+	}
+	var resp DesiredStateResponse
+	if err := json.Unmarshal(raw, &resp); err != nil {
+		t.Fatal(err)
+	}
+	o := resp.DesiredState.OSUpdate
+	if o == nil || o.Ring != 1 || !o.Enabled || o.Release == nil {
+		t.Fatalf("os_update = %+v", o)
+	}
+	r := o.Release
+	if r.ID != "os-guest-20261004-120000" || r.Snapshot != "20261004T120000Z" || len(r.Packages) != 2 ||
+		r.Packages[1].Name != "openssl" || r.Packages[1].Version != "3.5.7-1~deb13u3" || r.Packages[1].Origin != "Debian-Security" {
+		t.Fatalf("release = %+v", r)
+	}
+	// v0.141.0: the host layer's own approved set (`11` §8 step 3).
+	h := o.HostRelease
+	if h == nil || h.ID != "os-host-20261004-120000" || h.Snapshot != "20261004T120000Z" || len(h.Packages) != 1 ||
+		h.Packages[0].Name != "libssl3t64" || h.Packages[0].Origin != "Debian-Security" {
+		t.Fatalf("host_release = %+v", h)
+	}
+}

internal/hub/report.go

@@ -43,6 +43,12 @@ type HostReport struct {
 	// alert. Not a secret (the fp is public; the token is never reported).
 	LeafFingerprint string `json:"leaf_fingerprint"`
 
+	// Addresses are the host's routable addresses, one entry per (interface, address) — the LAN
+	// bridge, the WG tunnel, a tailnet. Added v0.119.0 because the hub could not show a managed
+	// box's IP anywhere: nothing in this report carried one. Non-nil so it marshals as [];
+	// see hostaddr.go for why it is iface+cidr rather than a single lan_ip.
+	Addresses []HostAddress `json:"addresses"`
+
 	// DR recipe — the agent (storage/guest/PBS) half of the secret-free reconstruction recipe
 	// (SPIKE-dr-recipe-2026-06-16). Derived from the facts above; carries ONLY identifiers/intents/
 	// sizes/coordinates, never a secret. The hub assembles it with the controller's app half.
@@ -62,8 +68,14 @@ type HostReport struct {
 	// report is stored opaquely hub-side, so these additive fields need no hub-schema change.
 	// Both are `omitempty` (the Wireguard precedent): in the steady state (no update in flight)
 	// they are absent — which keeps the cross-repo host-report golden contract byte-stable without
-	// a hub change. They appear only while an update is pending. The hub reads an absent field as
-	// pending=false, the correct default.
+	// a hub change. They appear only while an update is pending.
+	//
+	// ⚠ CORRECTED 2026-08-08 (R-260). This comment used to end "The hub reads an absent field as
+	// pending=false, the correct default." THE HUB HAS NO FIELD FOR EITHER OF THESE, so it reads
+	// nothing — present or absent — and encoding/json discards them on arrival. The sentence
+	// described an intent, not the code, and it read as settled for long enough that a sweep had to
+	// find it. The emission is correct and stays; the missing consumer is tracked as R-264, and
+	// `felhom.eu/scripts/wire_contract_gate.py` now refuses any NEW field of this shape.
 	SelfUpdatePending        bool   `json:"selfupdate_pending,omitempty"`
 	SelfUpdatePendingVersion string `json:"selfupdate_pending_version,omitempty"`
 
@@ -112,6 +124,39 @@ type HostReport struct {
 	// HTTPS even when felhom-sshd or the tunnel is DOWN (channel independence). `omitempty`: absent
 	// when the feature is not wired (pre-H1) — additive, no hub-schema change.
 	OOB *OOBStatus `json:"oob,omitempty"`
+
+	// ControllerSupervisor (R-523, v0.131.0) is the in-guest controller supervisor's per-guest record:
+	// how many times the agent restarted a dead controller, when last and why, whether it gave up
+	// (crash-loop pause) and whether the operator parked it. The hub's ControllerSupervisorChecker
+	// mints `controller_restarted_by_agent` when last_restart_at MOVES and `controller_crashloop` when
+	// crashloop_since MOVES — timestamps, not counters, because the record is in-memory and an agent
+	// restart zeroes the counter. `omitempty`: absent when not wired, so the cross-repo golden stays
+	// byte-stable. The hub parser is pinned by hub/internal/monitor/controller_supervisor_test.go
+	// against the JSON TestControllerSupervisorStanza_WireShape pins here.
+	ControllerSupervisor *ControllerSupervisorStatus `json:"controller_supervisor,omitempty"`
+}
+
+// ControllerSupervisorStatus is the R-523 stanza. Carries no secret.
+type ControllerSupervisorStatus struct {
+	Guests []ControllerSupervisorGuest `json:"guests"`
+}
+
+// ControllerSupervisorGuest is one supervised guest.
+type ControllerSupervisorGuest struct {
+	VMID           int    `json:"vmid"`
+	RestartsTotal  int    `json:"restarts_total"`
+	LastRestartAt  string `json:"last_restart_at,omitempty"` // RFC3339
+	LastReason     string `json:"last_reason,omitempty"`
+	Crashloop      bool   `json:"crashloop"`
+	CrashloopSince string `json:"crashloop_since,omitempty"` // RFC3339; the last crash-loop, kept after it ends
+	Parked         bool   `json:"parked"`
+	// R-539 (v0.132.0) — the SLOW crash loop. Restarts24h counts restarts the supervisor performed in
+	// the last 24 hours (persisted, so an agent restart does not reset it); SlowCrashloop is true while
+	// the last raise is under 24 hours old; SlowCrashloopSince is the raise itself, which the hub keys on
+	// MOVING (hub v0.117.0 controller_slow_crashloop). It moves at most once per 24 hours.
+	Restarts24h        int    `json:"restarts_24h"`
+	SlowCrashloop      bool   `json:"slow_crashloop"`
+	SlowCrashloopSince string `json:"slow_crashloop_since,omitempty"` // RFC3339
 }
 
 // PBSDRStatus is the per-heartbeat PBS-DR-tier bridge state (slice 2). States:
@@ -244,9 +289,10 @@ type GuestSpec struct {
 	DiskBytes   int64 `json:"disk_bytes"`
 }
 
-// Cloudflared is the tunnel service health (read-only probe this slice).
+// Cloudflared is the box's tunnel (R-841, agent v0.141.0): the cloudflared container in the customer guest.
 type Cloudflared struct {
-	Status string `json:"status"` // active | inactive | failed | unknown
+	Status string `json:"status"`           // running | not_running | unknown (TunnelRunning …)
+	Detail string `json:"detail,omitempty"` // why not_running / unknown, or "connected"
 }
 
 // The following element types are declared now so the empty collections above are
@@ -280,6 +326,30 @@ type StorageTarget struct {
 
 	MountPath     string `json:"mount_path"`     // host mountpoint (dir/usb); "" for network/lvm
 	BackingDevice string `json:"backing_device"` // resolved block device (e.g. /dev/sdb1); "" for network
+	// ConfigPath is the storage's CONFIGURED path from storage.cfg (proxmox.Storage.Path) — not a
+	// resolved mount. It is the only identity a dir storage keeps when its device is gone: MountPath
+	// and BackingDevice both empty out (observe.go's exactMount block) and DurableID degrades off the
+	// fs-UUID, so the configured path is what still says WHICH drive this row is about (R-116).
+	//
+	// `json:"-"` DELIBERATELY. This struct is a cross-repo contract duplicated in felhom.eu/hub and
+	// pinned by testdata/host-report.golden.json + contract_test.go's key-set comparison; a wire-visible
+	// field here would need a matching change in the other repo to stay non-drifting. Nothing off-box
+	// needs this value — its only consumer is the agent's own /disks construction, in-process.
+	ConfigPath string `json:"-"`
+	// PBSNamespace is the storage's CONFIGURED PBS namespace from storage.cfg (proxmox.Storage.Namespace)
+	// — "" for the root namespace, set for S4 per-customer tenancy. Present ONLY on pbs targets.
+	//
+	// It exists because storage.cfg is the ONE authority on which namespace this box's backups use:
+	// `vzdump --storage <pbs>` makes PVE read this exact field, and the agent's own verify client is
+	// built from it (`cmd/felhom-agent/main.go` → `pbs.Config{Namespace: s.Namespace}`). The DR recipe
+	// therefore resolves the namespace from HERE and not from a listed snapshot — a namespace-scoped
+	// PBS list does not echo `ns` per item, so the snapshot's own field is empty and normalising that
+	// empty to "root" is what made the recipe claim "root" on every per-customer box (R-106).
+	//
+	// `json:"-"` for the SAME reason as ConfigPath above: this struct is a cross-repo contract pinned by
+	// testdata/host-report.golden.json, and nothing off-box reads this value — its only consumer is the
+	// agent's own dr_recipe construction, in-process.
+	PBSNamespace string `json:"-"`
 	// ClassHint is a fast|slow HINT derived from the backing disk's rotational flag — a
 	// hint only; the authoritative class is hub-owned (locked decision). "" when not
 	// derivable (network targets have no local rotational flag).
@@ -316,6 +386,11 @@ type ThinPoolFill struct {
 type SmartSummary struct {
 	Health string `json:"health"`
 
+	// ModelName is smartctl's own device model (v0.95.0), captured from the JSON already parsed, so
+	// the UI can show a human label ("TOSHIBA MQ04ABF100") instead of a raw UUID. omitempty +
+	// pointer: absent on an old agent or a device that reports no model.
+	ModelName *string `json:"model_name,omitempty"`
+
 	TemperatureC *int `json:"temperature_c"`
 	PowerOnHours *int `json:"power_on_hours"`
 
@@ -396,6 +471,10 @@ type RestoreTest struct {
 	// mount layout, not just booted. Additive — a hub that predates them ignores the unknown keys.
 	MountParity    string   `json:"mount_parity,omitempty"`
 	MountInventory []string `json:"mount_inventory,omitempty"`
+	// Skipped (R-672, v0.133.0): the test did NOT run — the space preflight refused, and Error says
+	// why ("skipped: not enough space on …"). Pass is false. A hub that predates the key reads a
+	// failed test with that error, which is the honest reading.
+	Skipped bool `json:"skipped,omitempty"`
 }
 
 // PBSSnapshot is one PBS (offsite) snapshot's inventory + integrity state (doc 03 §8, slice
@@ -470,6 +549,35 @@ type WireDesiredState struct {
 	RestoreDirective *WireRestoreDirective `json:"restore_directive,omitempty"` // slice 10D (forward-compat)
 	Wireguard        *WireWireguard        `json:"wireguard,omitempty"`         // S3 (doc 06 §3.2; golden-pinned)
 	PBSDR            *WirePBSDR            `json:"pbs_dr,omitempty"`            // PBS DR tier (slice 2 consumer)
+	OSUpdate         *WireOSUpdate         `json:"os_update,omitempty"`         // OS updates, guest fast lane (agent v0.140.0)
+}
+
+// WireOSUpdate is the hub-OWNED OS-update block (hub v0.130.0, `11-os-updates.md` §5.3), merged into the served
+// document at read time. Ring 0 installs every pending Debian / Debian-Security fix; ring 1 installs exactly the
+// newest approved release. Absent (older hub) → the agent treats the box as ring 1, ON, no release: it reports
+// and installs nothing. Golden: testdata/desired-state-osupdate.golden.json (byte-identical with the hub's).
+type WireOSUpdate struct {
+	Ring    int            `json:"ring"`
+	Enabled bool           `json:"enabled"`
+	Release *WireOSRelease `json:"release,omitempty"`
+	// HostRelease is the newest approved HOST release (hub v0.131.0, `11` §8 step 3) — a separate set: a version
+	// approved for the guest is not approved for the host by that fact alone.
+	HostRelease *WireOSRelease `json:"host_release,omitempty"`
+}
+
+// WireOSRelease is an approved version set; Snapshot is the approval time (YYYYMMDDTHHMMSSZ) the wrapper uses
+// for snapshot.debian.org when Debian has already replaced a version (decision 79).
+type WireOSRelease struct {
+	ID       string          `json:"id"`
+	Snapshot string          `json:"snapshot"`
+	Packages []WireOSPackage `json:"packages"`
+}
+
+// WireOSPackage is one approved name=version and its origin ("Debian" | "Debian-Security").
+type WireOSPackage struct {
+	Name    string `json:"name"`
+	Version string `json:"version"`
+	Origin  string `json:"origin"`
 }
 
 // WirePBSDR is the hub's PBS-DR-tier descriptor (PBS DR slice 1, hub/internal/web/pbsdr.go

internal/hub/report_test.go

@@ -32,10 +32,11 @@ func TestHostReport_FieldNamesAndEmptyCollections(t *testing.T) {
 		Cloudflared:     Cloudflared{Status: "active"},
 		Capabilities:    []capability.Status{},
 		LeafFingerprint: "60b5974d586f5f3c8ec41eb998d0f07406178219c36bf6d3ff377570279d8245",
+		Addresses:       []HostAddress{},
 	}
 	// dr_recipe is always set on the real path (Collect); set it here too so the "no null" invariant
 	// covers it (empty pbs is omitempty → omitted, never null).
-	r.DRRecipe = BuildDRRecipeHostHalf(r.Guests, r.StorageTargets, r.PBSSnapshots)
+	r.DRRecipe = BuildDRRecipeHostHalf(r.Guests, r.StorageTargets, r.PBSSnapshots, ConfiguredBackupTarget{})
 	b, err := json.Marshal(r)
 	if err != nil {
 		t.Fatal(err)
@@ -51,6 +52,7 @@ func TestHostReport_FieldNamesAndEmptyCollections(t *testing.T) {
 		// empty collections must be [] not null
 		`"storage_targets":[]`, `"backups":[]`, `"restore_tests":[]`, `"pbs_snapshots":[]`, `"audit_tail":[]`,
 		`"capabilities":[]`,
+		`"addresses":[]`,
 		`"leaf_fingerprint":"60b5974d586f5f3c8ec41eb998d0f07406178219c36bf6d3ff377570279d8245"`,
 	} {
 		if !strings.Contains(got, field) {

internal/hub/restoretest_merge_test.go

@@ -0,0 +1,213 @@
+package hub
+
+import (
+	"context"
+	"testing"
+	"time"
+)
+
+// R-189 — a passing restore-test must survive an agent restart and reach the hub.
+//
+// THE OBSERVATION THIS EXISTS FOR (2026-08-03, demo-felhom): a real 14.5 GB offsite restore-test
+// PASSED at 15:25:14; the agent was restarted 2 m 43 s later for a deploy; the hub logged
+// `0 restore-tests` on the next two host-reports. The in-memory store's own comment said "lost on
+// restart; the cadence re-populates", which was true under a timer and stopped being true when R-86
+// made the agent refuse to re-test an archive it has already proven.
+//
+// Timestamps here carry JITTER (odd minutes and seconds, not round hours) — yesterday a test was
+// hollow because a perfectly regular series landed exactly on a threshold and passed under the
+// mutation it was meant to catch.
+
+type fakeLatest struct{ tests []RestoreTest }
+
+func (f *fakeLatest) RestoreTests(context.Context) []RestoreTest { return f.tests }
+
+type fakeProven struct{ tests []RestoreTest }
+
+func (f *fakeProven) ProvenRestoreTests(context.Context) []RestoreTest { return f.tests }
+
+func rt(tier, archive string, pass bool, at time.Time) RestoreTest {
+	return RestoreTest{
+		SourceArchive: archive, SourceTier: tier, Pass: pass,
+		Verified: "boot+running", TestedAt: at.UTC().Format(time.RFC3339),
+	}
+}
+
+// mergeCollector builds a Collector with only the two restore-test seams wired — the merge is what
+// is under test, not the rest of the collection.
+func mergeCollector(latest, proven []RestoreTest) *Collector {
+	c := &Collector{}
+	if latest != nil {
+		c.restoreTests = &fakeLatest{tests: latest}
+	}
+	if proven != nil {
+		c.provenTests = &fakeProven{tests: proven}
+	}
+	return c
+}
+
+func findTier(got []RestoreTest, tier string) (RestoreTest, int) {
+	var hit RestoreTest
+	n := 0
+	for _, e := range got {
+		if e.SourceTier == tier {
+			hit, n = e, n+1
+		}
+	}
+	return hit, n
+}
+
+// ── SCENARIO A — a proof survives a restart and reaches the hub ──────────────────────────────
+//
+// COMPANION RED-PROOF (observed 2026-08-03): delete the `c.provenTests` merge from
+// collectRestoreTests (return the in-memory slice as it used to) →
+//
+//	--- FAIL: TestMerge_ProofSurvivesARestart
+//	    restoretest_merge_test.go: after a restart the persisted proof must be reported; got 0 entr(ies)
+//
+// which is exactly the live observation: `0 restore-tests`. Restored.
+func TestMerge_ProofSurvivesARestart(t *testing.T) {
+	provenAt := time.Date(2026, 8, 3, 13, 25, 14, 0, time.UTC) // the real run's timestamp
+	// After a restart the in-memory store is EMPTY — this is the whole point.
+	c := mergeCollector([]RestoreTest{}, []RestoreTest{
+		rt("pbs", "felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z", true, provenAt),
+	})
+
+	got := c.collectRestoreTests(context.Background())
+	if len(got) != 1 {
+		t.Fatalf("after a restart the persisted proof must be reported; got %d entr(ies): %+v", len(got), got)
+	}
+	e := got[0]
+	if e.SourceArchive != "felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z" {
+		t.Fatalf("the entry must name the archive that was proven — the hub keys on it; got %q", e.SourceArchive)
+	}
+	if e.SourceTier != "pbs" || !e.Pass {
+		t.Fatalf("the entry must be a PASS on the tier it was proven on; got tier=%q pass=%v", e.SourceTier, e.Pass)
+	}
+	if e.TestedAt != provenAt.Format(time.RFC3339) {
+		t.Fatalf("the entry must carry the ORIGINAL test time, not now(); got %q", e.TestedAt)
+	}
+}
+
+// ── SCENARIO B — the report does not invent a pass ───────────────────────────────────────────
+//
+// COMPANION RED-PROOF (observed): make the state layer emit an entry for an unproven tier (drop the
+// `reportable()` filter in ProvenRestoreTests, so a legacy record with no archive is emitted) — the
+// equivalent at this layer is a proven-source that returns an entry for a tier nothing proved, which
+// this test injects directly and the assertion below rejects.
+func TestMerge_NeverInventsAPassForAnUnprovenTier(t *testing.T) {
+	// Nothing proven anywhere: no in-memory result, no persisted proof.
+	c := mergeCollector([]RestoreTest{}, []RestoreTest{})
+	if got := c.collectRestoreTests(context.Background()); len(got) != 0 {
+		t.Fatalf("a tier with no proof must produce NO entry — an unproven tier reading as proven is "+
+			"worse than the defect being fixed; got %+v", got)
+	}
+
+	// And an entry the state layer could not describe (no tier) is never promoted into a proof.
+	c2 := mergeCollector([]RestoreTest{}, []RestoreTest{
+		{SourceArchive: "local:backup/x.tar.zst", SourceTier: "", Pass: true,
+			TestedAt: time.Date(2026, 8, 1, 4, 41, 58, 0, time.UTC).Format(time.RFC3339)},
+	})
+	if got := c2.collectRestoreTests(context.Background()); len(got) != 0 {
+		t.Fatalf("a persisted record with no tier is not a usable proof and must be dropped; got %+v", got)
+	}
+}
+
+// ── SCENARIO C — a fresh in-memory result wins, and never duplicates ─────────────────────────
+//
+// COMPANION RED-PROOF (observed 2026-08-03): remove the de-duplication (append every persisted entry
+// unconditionally) →
+//
+//	--- FAIL: TestMerge_NewerWinsAndNeverDuplicatesATier
+//	    restoretest_merge_test.go: one entry per tier; got 2 for "pbs" — the hub would read two tests
+//
+// Restored.
+func TestMerge_NewerWinsAndNeverDuplicatesATier(t *testing.T) {
+	lastWeek := time.Date(2026, 7, 27, 19, 55, 41, 0, time.UTC) // jittered, from the real box
+	fiveMinAgo := time.Date(2026, 8, 3, 13, 25, 14, 0, time.UTC)
+
+	c := mergeCollector(
+		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/new", true, fiveMinAgo)},
+		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/old", true, lastWeek)},
+	)
+	got := c.collectRestoreTests(context.Background())
+	e, n := findTier(got, "pbs")
+	if n != 1 {
+		t.Fatalf("one entry per tier; got %d for \"pbs\" — the hub would read two tests: %+v", n, got)
+	}
+	if e.SourceArchive != "felhom-pbs:backup/ct/9201/new" {
+		t.Fatalf("the NEWER result must win; got %q tested %q", e.SourceArchive, e.TestedAt)
+	}
+
+	// ...and the older-in-memory / newer-persisted direction, which is the post-restart case.
+	c2 := mergeCollector(
+		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/old", true, lastWeek)},
+		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/new", true, fiveMinAgo)},
+	)
+	e2, n2 := findTier(c2.collectRestoreTests(context.Background()), "pbs")
+	if n2 != 1 || e2.SourceArchive != "felhom-pbs:backup/ct/9201/new" {
+		t.Fatalf("newest must win regardless of which source it came from; got %d entr(ies), archive %q", n2, e2.SourceArchive)
+	}
+}
+
+// ── SCENARIO D — a failure still reaches the hub ─────────────────────────────────────────────
+//
+// The merge must not mask a failure with an older stored success. A failing tier is retried at the
+// next evaluation and its record lives ONLY in memory, so losing it here would silence the loudest
+// DR signal this system produces.
+func TestMerge_AFailureIsStillReported(t *testing.T) {
+	provenLastWeek := time.Date(2026, 7, 27, 19, 55, 41, 0, time.UTC)
+	failedJustNow := time.Date(2026, 8, 3, 13, 41, 7, 0, time.UTC)
+
+	c := mergeCollector(
+		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/new", false, failedJustNow)},
+		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/old", true, provenLastWeek)},
+	)
+	e, n := findTier(c.collectRestoreTests(context.Background()), "pbs")
+	if n != 1 {
+		t.Fatalf("one entry per tier; got %d: %+v", n, c.collectRestoreTests(context.Background()))
+	}
+	if e.Pass {
+		t.Fatalf("a FAILURE newer than the stored proof must be what is reported — masking it would "+
+			"silence the loudest DR signal there is; got pass=%v archive=%q", e.Pass, e.SourceArchive)
+	}
+}
+
+// Two different tiers are both reported — the merge is per tier, not a single slot.
+func TestMerge_BothTiersSurvive(t *testing.T) {
+	c := mergeCollector(
+		[]RestoreTest{rt("local", "felhom-backup:backup/vzdump-lxc-9201-a.tar.zst", true,
+			time.Date(2026, 8, 3, 4, 44, 50, 0, time.UTC))},
+		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/x", true,
+			time.Date(2026, 8, 2, 5, 12, 33, 0, time.UTC))},
+	)
+	got := c.collectRestoreTests(context.Background())
+	if _, n := findTier(got, "local"); n != 1 {
+		t.Fatalf("the in-memory tier must survive the merge; got %+v", got)
+	}
+	if _, n := findTier(got, "pbs"); n != 1 {
+		t.Fatalf("the persisted tier must survive the merge; got %+v", got)
+	}
+}
+
+// A malformed timestamp must never displace a good entry — "unparseable" is not "newest".
+func TestMerge_MalformedTimestampNeverWins(t *testing.T) {
+	good := rt("pbs", "felhom-pbs:backup/ct/9201/good", true, time.Date(2026, 8, 3, 13, 25, 14, 0, time.UTC))
+	bad := RestoreTest{SourceArchive: "felhom-pbs:backup/ct/9201/bad", SourceTier: "pbs", Pass: true, TestedAt: "not-a-time"}
+
+	c := mergeCollector([]RestoreTest{good}, []RestoreTest{bad})
+	e, n := findTier(c.collectRestoreTests(context.Background()), "pbs")
+	if n != 1 || e.SourceArchive != "felhom-pbs:backup/ct/9201/good" {
+		t.Fatalf("an unparseable timestamp must not displace a good entry; got %d entr(ies), archive %q", n, e.SourceArchive)
+	}
+}
+
+// A nil proven-source leaves the pre-R-189 behaviour exactly as it was.
+func TestMerge_NilProvenSourceIsANoOp(t *testing.T) {
+	only := rt("local", "felhom-backup:backup/x.tar.zst", true, time.Date(2026, 8, 3, 4, 44, 50, 0, time.UTC))
+	c := mergeCollector([]RestoreTest{only}, nil)
+	got := c.collectRestoreTests(context.Background())
+	if len(got) != 1 || got[0].SourceArchive != only.SourceArchive {
+		t.Fatalf("a nil durable source must not change anything; got %+v", got)
+	}
+}

internal/hub/testdata/desired-state-osupdate.golden.json

@@ -0,0 +1,24 @@
+{
+  "generation": 1,
+  "desired_state": {
+    "os_update": {
+      "ring": 1,
+      "enabled": true,
+      "release": {
+        "id": "os-guest-20261004-120000",
+        "snapshot": "20261004T120000Z",
+        "packages": [
+          {"name": "libc6", "version": "2.41-12+deb13u4", "origin": "Debian"},
+          {"name": "openssl", "version": "3.5.7-1~deb13u3", "origin": "Debian-Security"}
+        ]
+      },
+      "host_release": {
+        "id": "os-host-20261004-120000",
+        "snapshot": "20261004T120000Z",
+        "packages": [
+          {"name": "libssl3t64", "version": "3.5.7-1~deb13u3", "origin": "Debian-Security"}
+        ]
+      }
+    }
+  }
+}

internal/hub/testdata/host-report.golden.json

@@ -134,6 +134,9 @@
   "audit_tail": [],
   "capabilities": [],
   "leaf_fingerprint": "60b5974d586f5f3c8ec41eb998d0f07406178219c36bf6d3ff377570279d8245",
+  "addresses": [
+    { "iface": "vmbr0", "cidr": "192.168.0.162/24" }
+  ],
   "dr_recipe": {
     "recipe_version": 1,
     "guests": [
@@ -141,7 +144,8 @@
     ],
     "pbs": {
       "repo_id": "felhom-pbs",
-      "namespace": "root",
+      "namespace": "felhom-spike",
+      "namespace_state": "resolved",
       "latest_snapshot_id": "9001"
     },
     "drives": [
@@ -154,7 +158,13 @@
     ],
     "pve_storage": [
       { "name": "local-lvm", "type": "lvmthin", "content": "rootdir,images" },
-      { "name": "usb-backup", "type": "usb", "content": "backup" }
-    ]
+      { "name": "usb-backup", "type": "usb", "content": "backup" },
+      { "name": "felhom-pbs", "type": "pbs", "content": "backup" }
+    ],
+    "backup_target": {
+      "state": "resolved",
+      "storage_id": "usb-backup",
+      "mount_path": "/mnt/usb-backup"
+    }
   }
 }

internal/localapi/afterbackup_test.go

@@ -0,0 +1,63 @@
+package localapi
+
+import (
+	"context"
+	"net/http"
+	"sync"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/backup"
+)
+
+// The OS leg (agent v0.140.0) runs after a SUCCESSFUL primary backup, and only then; and it runs BEFORE the
+// host-wide heavy-op gate is released, so a restore-test cannot start in the middle of it (`11` C10).
+// Red-proof: drop the `b.Success &&` guard and the failed-backup sub-case fails; move the call after release()
+// and the gate sub-case fails.
+func TestAfterPrimaryBackup(t *testing.T) {
+	run := func(t *testing.T, failErr string) (calls []int, gateHeld bool) {
+		gate := &backup.InFlight{}
+		b := &fakeBackups{failErr: failErr}
+		srv := newTestServerS(t, &fakeGuests{}, b, &fakeStore{}, nil)
+		srv.inFlight = gate
+		var mu sync.Mutex
+		done := make(chan struct{}, 1)
+		srv.SetAfterPrimaryBackup(func(_ context.Context, vmid int) {
+			rel, _, ok := gate.TryAcquire("probe")
+			mu.Lock()
+			calls = append(calls, vmid)
+			gateHeld = !ok
+			mu.Unlock()
+			if ok {
+				rel()
+			}
+			done <- struct{}{}
+		})
+		h := srv.Handler()
+		if do(t, h, "POST", "/backup", "A", "").Code != http.StatusAccepted {
+			t.Fatal("POST /backup not accepted")
+		}
+		select {
+		case <-done:
+		case <-time.After(500 * time.Millisecond):
+		}
+		time.Sleep(20 * time.Millisecond)
+		mu.Lock()
+		defer mu.Unlock()
+		return calls, gateHeld
+	}
+	t.Run("success runs the leg under the gate", func(t *testing.T) {
+		calls, held := run(t, "")
+		if len(calls) != 1 {
+			t.Fatalf("the leg ran %d time(s), want 1", len(calls))
+		}
+		if !held {
+			t.Fatal("the heavy-op gate was free while the leg ran — a restore-test could overlap it")
+		}
+	})
+	t.Run("a failed backup runs nothing", func(t *testing.T) {
+		if calls, _ := run(t, "vzdump exploded"); len(calls) != 0 {
+			t.Fatalf("the leg ran after a FAILED backup: %v", calls)
+		}
+	})
+}

internal/localapi/agestate_test.go

@@ -0,0 +1,160 @@
+package localapi
+
+import (
+	"context"
+	"errors"
+	"io"
+	"log/slog"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+)
+
+// R-88 Part 2 — the agent gains a third state.
+//
+// `newestArchiveOn` promised in its own doc comment that "errors degrade to unknown, never to
+// no-backup", while its `(time.Time, bool)` return made that impossible: an error and a genuine
+// not-found both produced `(zero, false)`, so `/backup/due` answered a POSITIVE
+// "no successful backup recorded yet" with a nil age. The controller read that as "never backed up"
+// and fired its window-gate safety valve, quiescing customer app stacks outside the backup window.
+//
+// These tests assert the WIRE, because the wire is the contract another component reads.
+
+// listerBackups is a fakeBackups that also implements BackupArchiveLister, with a controllable outcome.
+type listerBackups struct {
+	fakeBackups
+	t     time.Time
+	found bool
+	err   error
+}
+
+func (l *listerBackups) NewestArchiveTime(context.Context, int) (time.Time, bool, error) {
+	return l.t, l.found, l.err
+}
+
+// dueWithLister builds a server whose single tier's service is the given lister, and returns the
+// /backup/due response.
+// NOTE: the server must receive `lb` ITSELF, not its embedded fakeBackups — the tier's Service is
+// type-asserted to BackupArchiveLister, and the embedded value does not satisfy it. Passing the
+// inner struct silently routes every case to archiveUnknown, which looks like a code bug and is not.
+func dueWithLister(t *testing.T, lb *listerBackups, store *fakeStore) BackupDueResponse {
+	t.Helper()
+	srv, err := NewServer(Options{
+		ListenAddr:    "127.0.0.1:0",
+		Guests:        &fakeGuests{},
+		Backups:       lb,
+		Store:         store,
+		Storage:       fakeStorage{},
+		Tokens:        staticTokens{"A": 8200, "B": 9300},
+		BackupCadence: 24 * time.Hour,
+		Logger:        slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatalf("NewServer: %v", err)
+	}
+	srv.now = func() time.Time { return testNow }
+	return dueOf(t, srv.Handler())
+}
+
+// ── SCENARIO A (agent half) — an unreadable storage is UNKNOWN, not "never" ──────────────────
+//
+// COMPANION RED-PROOF (observed): make newestArchiveOn return archiveAbsent on the error path (the
+// pre-fix collapse) and this fails with
+//
+//	"an unreadable storage must report age_state=unknown, got \"absent\" — that is a POSITIVE claim
+//	 of 'never backed up' built out of two absences"
+//
+// Restored.
+func TestAgeState_UnreadableStorageIsUnknown(t *testing.T) {
+	lb := &listerBackups{err: errors.New("proxmox: GET storage content: 500 connection refused")}
+	got := dueWithLister(t, lb, &fakeStore{}) // empty store = cold in-memory record
+
+	if got.AgeState != AgeStateUnknown {
+		t.Fatalf("an unreadable storage must report age_state=%q, got %q — that is a POSITIVE claim of "+
+			"'never backed up' built out of two absences", AgeStateUnknown, got.AgeState)
+	}
+	if !got.Due {
+		t.Fatal("FAIL-SAFE DIRECTION: unknown must still be DUE — an unreadable storage must never suppress a backup")
+	}
+	if got.AgeSecs != nil {
+		t.Fatalf("an unknown age must not invent a number; got %d", *got.AgeSecs)
+	}
+}
+
+// ── SCENARIO B (agent half) — a genuine first-ever backup is ABSENT ──────────────────────────
+//
+// B is what makes A safe: an implementation that reported everything as "unknown" would pass A and
+// silently starve a brand-new box, because the controller only fires the first-backup valve on ABSENT.
+//
+// COMPANION RED-PROOF (observed): make newestArchiveOn return archiveUnknown when !found and this
+// fails with
+//
+//	"a genuine never-backed-up tier must report age_state=\"absent\", got \"unknown\" — the
+//	 controller only licenses a first backup outside the window on ABSENT"
+//
+// Restored.
+func TestAgeState_GenuinelyNeverIsAbsent(t *testing.T) {
+	lb := &listerBackups{found: false} // read SUCCEEDED, nothing there
+	got := dueWithLister(t, lb, &fakeStore{})
+
+	if got.AgeState != AgeStateAbsent {
+		t.Fatalf("a genuine never-backed-up tier must report age_state=%q, got %q — the controller only "+
+			"licenses a first backup outside the window on ABSENT", AgeStateAbsent, got.AgeState)
+	}
+	if !got.Due {
+		t.Fatal("a never-backed-up tier must be due")
+	}
+}
+
+// A real archive → known, with a real age.
+func TestAgeState_FoundIsKnown(t *testing.T) {
+	lb := &listerBackups{t: testNow.Add(-2 * time.Hour), found: true}
+	got := dueWithLister(t, lb, &fakeStore{})
+
+	if got.AgeState != AgeStateKnown {
+		t.Fatalf("a readable archive must report age_state=%q, got %q", AgeStateKnown, got.AgeState)
+	}
+	if got.AgeSecs == nil || *got.AgeSecs < 7100 || *got.AgeSecs > 7300 {
+		t.Fatalf("expected ~7200s age, got %v", got.AgeSecs)
+	}
+	if got.Due {
+		t.Fatal("2h old against a 24h cadence is not due")
+	}
+}
+
+// An unparseable in-memory timestamp is UNKNOWN too — a backup DID happen, we just cannot date it.
+// Reporting "absent" there would be the same false-positive claim in a different costume.
+func TestAgeState_UnparseableTimestampIsUnknown(t *testing.T) {
+	st := &fakeStore{backups: []hub.Backup{{VMID: 8200, Success: true, StartedAt: "not-a-timestamp"}}}
+	lb := &listerBackups{err: errors.New("storage unreadable")}
+	got := dueWithLister(t, lb, st)
+
+	if got.AgeState != AgeStateUnknown {
+		t.Fatalf("an unparseable backup time means we cannot DATE a backup that exists — want %q, got %q",
+			AgeStateUnknown, got.AgeState)
+	}
+	if !got.Due {
+		t.Fatal("still due — fail safe toward taking a backup")
+	}
+}
+
+// ── SCENARIO D (agent half) — additive on the wire ───────────────────────────────────────────
+//
+// An OLD controller decodes into a struct without `age_state` and ignores it. What it MUST still see
+// unchanged is every pre-existing field.
+func TestAgeState_IsAdditive_PreExistingFieldsUnchanged(t *testing.T) {
+	lb := &listerBackups{t: testNow.Add(-48 * time.Hour), found: true}
+	got := dueWithLister(t, lb, &fakeStore{})
+
+	if !got.Due || got.Reason != "older than cadence" {
+		t.Fatalf("pre-existing due/reason semantics changed: due=%v reason=%q", got.Due, got.Reason)
+	}
+	if got.AgeSecs == nil {
+		t.Fatal("age_seconds must still be present for a known age")
+	}
+	// And the state rides alongside rather than replacing anything.
+	if got.AgeState != AgeStateKnown {
+		t.Fatalf("age_state should be %q, got %q", AgeStateKnown, got.AgeState)
+	}
+}

internal/localapi/backup_target.go

@@ -0,0 +1,212 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+// BackupTargetWrapperPath is the pinned sudoers vector (configs/felhom-backup-target-apply). The
+// agent cannot create a PVE storage or grant an ACL itself — Datastore.Allocate at /storage and
+// Permissions.Modify are deliberately outside its role — so the privileged half runs here.
+const BackupTargetWrapperPath = "/usr/local/sbin/felhom-backup-target-apply"
+
+// backupTargetRequest is POST /backup/target: move the PRIMARY whole-guest backup tier onto the
+// drive mounted at Where, creating the storage if needed.
+type backupTargetRequest struct {
+	VMID  int    `json:"vmid"`
+	Where string `json:"where"`        // the drive's OWN host mountpoint (F-1)
+	ID    string `json:"id,omitempty"` // storage id; default backupTargetStorageID
+}
+
+// backupTargetStorageID is the conventional id, matching what E-1 created by hand on both demo boxes.
+// Keeping the name identical is what makes this endpoint IDEMPOTENT on an already-migrated box: the
+// wrapper accepts an existing entry with the same path and changes nothing.
+const backupTargetStorageID = "felhom-backup"
+
+// handleSetBackupTarget performs the whole move as one ordered operation: create the storage, grant
+// the agent access, repoint the primary tier in agent.json, and hand back what the caller must do to
+// make it take effect.
+//
+// THE ORDER IS THE DESIGN, and each step is a precondition for the next:
+//
+//	create → grant → config
+//
+// Reversed, a config pointing at a storage that does not exist would make the tier DEFER (harmless
+// but silent), and a config pointing at an ungranted storage would make every backup 403 on its
+// first run — which is exactly what E-1 hit when the grant was forgotten (finding F-3). Creating and
+// granting BEFORE the config means the worst interruption leaves an unused storage, never a broken
+// tier.
+//
+// IT DOES NOT RESTART THE AGENT. That is deliberate and it is the E-1 lesson encoded: the backup
+// tiers are built once at daemon start, so the move needs a restart to take effect — but restarting
+// while a backup or restore-test is in flight cancels the wait and records a SPURIOUS tier failure
+// for a backup that actually succeeded (E-1 did exactly this to a felhom-pbs run). A restart that
+// this handler fires itself could never be re-checked against in-flight work by the caller, so the
+// response reports `restart_required` and the caller performs it behind its own immediate
+// in-flight check.
+func (s *Server) handleSetBackupTarget(w http.ResponseWriter, r *http.Request, vmid int) {
+	if s.privileged == nil {
+		writeErr(w, http.StatusServiceUnavailable, "privileged runner not configured on this host")
+		return
+	}
+	var req backupTargetRequest
+	if !decodeBody(w, r, &req) {
+		return
+	}
+	if !s.scopedFromBody(w, req.VMID, vmid, r.URL.Path) {
+		return
+	}
+	where := strings.TrimSpace(req.Where)
+	if where == "" {
+		writeErr(w, http.StatusBadRequest, "where (the drive's own mountpoint) is required")
+		return
+	}
+	id := strings.TrimSpace(req.ID)
+	if id == "" {
+		id = backupTargetStorageID
+	}
+
+	// AGENT-SIDE VALIDATION FIRST, from the agent's own storage view — never the caller's claim.
+	// The wrapper re-checks everything as root (it is the security boundary), but refusing here gives
+	// the customer a reason instead of a shell error, and keeps a bad request from reaching sudo at all.
+	if err := s.validateBackupTargetMount(r.Context(), where); err != nil {
+		s.logger.Warn("local-api: backup-target move refused", "vmid", vmid, "where", where, "err", err)
+		writeErr(w, http.StatusBadRequest, err.Error())
+		return
+	}
+
+	if _, errOut, err := s.privileged.Run(r.Context(), BackupTargetWrapperPath, "create", id, where); err != nil {
+		s.logger.Error("local-api: backup-target create failed", "id", id, "where", where, "err", err, "stderr", string(errOut))
+		writeErr(w, http.StatusBadGateway, "could not create the backup storage: "+wrapperReason(errOut, err))
+		return
+	}
+	if _, errOut, err := s.privileged.Run(r.Context(), BackupTargetWrapperPath, "grant", id); err != nil {
+		// The storage exists but the agent cannot write to it. Say so precisely: this is the exact
+		// state that produced E-1's "403 permission denied at /storage/felhom-backup" on first backup.
+		s.logger.Error("local-api: backup-target grant failed", "id", id, "err", err, "stderr", string(errOut))
+		writeErr(w, http.StatusBadGateway, "storage created but the access grant failed — backups would 403: "+wrapperReason(errOut, err))
+		return
+	}
+	if err := s.setConfiguredBackupTarget(id); err != nil {
+		s.logger.Error("local-api: backup-target config write failed", "id", id, "err", err)
+		writeErr(w, http.StatusInternalServerError, "storage is ready but the config could not be updated: "+err.Error())
+		return
+	}
+
+	s.logger.Info("local-api: backup target moved — RESTART REQUIRED for it to take effect",
+		"vmid", vmid, "target", id, "where", where)
+	writeOK(w, map[string]any{
+		"vmid": vmid, "target": id, "where": where,
+		// The caller must restart the agent BEHIND ITS OWN in-flight check — see the doc comment.
+		"restart_required": true,
+	})
+}
+
+// validateBackupTargetMount refuses a mount that cannot be a real backup target, from the agent's own
+// storage view + mount table. Mirrors the wrapper's laws so the customer gets a reason, not a shell error.
+func (s *Server) validateBackupTargetMount(ctx context.Context, where string) error {
+	if s.storage == nil {
+		return fmt.Errorf("storage view unavailable")
+	}
+	// It must currently BE a mountpoint (F-1/F-2). Resolved from the mount table, which is the same
+	// source the wrapper's `mountpoint -q` consults.
+	mounts, err := s.hostReader().Mounts()
+	if err != nil {
+		return fmt.Errorf("could not read the mount table")
+	}
+	var dev string
+	for _, m := range mounts {
+		if m.MountPoint == where {
+			dev = m.Device
+			break
+		}
+	}
+	if dev == "" {
+		return fmt.Errorf("%s is not a mountpoint — the backup target must be the drive's own mountpoint", where)
+	}
+	// Never the system disk: a target there protects against corruption only, never drive loss.
+	for _, m := range mounts {
+		if m.MountPoint == "/" && m.Device == dev {
+			return fmt.Errorf("%s is on the system disk — a backup target there cannot survive a drive failure", where)
+		}
+	}
+	return nil
+}
+
+// setConfiguredBackupTarget rewrites backup.local_backup_target in agent.json.
+//
+// Read-modify-write over map[string]json.RawMessage so UNKNOWN KEYS ARE PRESERVED VERBATIM — the
+// same discipline as pbsdr.seedEscrowStorageID, and the property that made E-1's hand edit safe to
+// begin with. A typed round-trip would silently drop any key this build does not know about.
+//
+// Written IN PLACE (O_TRUNC), not tmp+rename: /etc/felhom-agent is root-owned while agent.json is
+// agent-owned 0600, so the non-root agent cannot rename into that directory. A recovery copy is
+// parked first, so a torn write is recoverable.
+func (s *Server) setConfiguredBackupTarget(id string) error {
+	path := s.configPath
+	if path == "" {
+		return fmt.Errorf("no config path known to this agent (env-only config)")
+	}
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return err
+	}
+	var doc map[string]json.RawMessage
+	if err := json.Unmarshal(raw, &doc); err != nil {
+		return fmt.Errorf("parse %s: %w", path, err)
+	}
+	var bk map[string]json.RawMessage
+	if cur, ok := doc["backup"]; ok {
+		if err := json.Unmarshal(cur, &bk); err != nil {
+			return fmt.Errorf("parse backup section: %w", err)
+		}
+	} else {
+		bk = map[string]json.RawMessage{}
+	}
+	idJSON, _ := json.Marshal(id)
+	bk["local_backup_target"] = idJSON
+	bkJSON, err := json.Marshal(bk)
+	if err != nil {
+		return err
+	}
+	doc["backup"] = bkJSON
+	out, err := json.MarshalIndent(doc, "", "  ")
+	if err != nil {
+		return err
+	}
+	st, err := os.Stat(path)
+	if err != nil {
+		return err
+	}
+	if s.stateDir != "" {
+		if err := os.MkdirAll(s.stateDir, 0o700); err == nil {
+			_ = os.WriteFile(filepath.Join(s.stateDir, "agent.json.pre-backup-target"), raw, 0o600)
+		}
+	}
+	f, err := os.OpenFile(path, os.O_WRONLY|os.O_TRUNC, st.Mode().Perm())
+	if err != nil {
+		return err
+	}
+	if _, err := f.Write(out); err != nil {
+		f.Close()
+		return err
+	}
+	return f.Close()
+}
+
+// wrapperReason surfaces the wrapper's own REFUSED line when it produced one — it explains WHY in
+// terms the customer can act on ("not a mountpoint", "already exists at …") — falling back to the
+// exec error only when stderr said nothing useful.
+func wrapperReason(errOut []byte, err error) string {
+	for _, line := range strings.Split(string(errOut), "\n") {
+		if strings.Contains(line, "REFUSED:") {
+			return strings.TrimSpace(line)
+		}
+	}
+	return err.Error()
+}

internal/localapi/backup_target_guard_test.go

@@ -0,0 +1,147 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"io"
+	"log/slog"
+	"net/http"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+)
+
+// backupTargetServer builds a server whose PRIMARY tier is `felhom-backup`, mounted at /mnt/nvme-1tb
+// on its own non-system device — i.e. the exact live shape E-1 created on demo-hp and demo-felhom:
+// the drive is simultaneously the enrolled user-data drive AND the whole-guest vzdump target.
+func backupTargetServer(t *testing.T) http.Handler {
+	t.Helper()
+	sv := fakeStorage{targets: []hub.StorageTarget{
+		{
+			Name: "felhom-backup", Type: hub.StorageTypeLocalDir, State: hub.StorageStateAttached,
+			Reachable: true, MountPath: "/mnt/nvme-1tb", BackingDevice: "/dev/nvme0n1",
+			Content: "backup",
+		},
+		{
+			Name: "spare-drive", Type: hub.StorageTypeLocalDir, State: hub.StorageStateAttached,
+			Reachable: true, MountPath: "/mnt/spare", BackingDevice: "/dev/sdz1",
+			Content: "backup",
+		},
+	}}
+	srv, err := NewServer(Options{
+		ListenAddr: "127.0.0.1:0",
+		Guests:     &fakeGuests{}, Backups: &fakeBackups{}, Store: &fakeStore{}, Storage: sv,
+		Tokens: staticTokens{"A": 8200},
+		Disks:  &fakeDiskOps{}, DiskGate: &fakeGate{}, HostReader: sysOnSDA(),
+		// Service is load-bearing: normalizeBackupTiers DROPS any tier with a nil Service and falls
+		// back to the legacy single tier with an empty TargetID — which silently made an earlier
+		// version of this test exercise nothing.
+		BackupTiers: []BackupTier{
+			{TargetID: "felhom-backup", Cadence: 24 * time.Hour, Primary: true, Service: &fakeBackups{}},
+			{TargetID: "felhom-pbs", Cadence: 168 * time.Hour, Service: &fakeBackups{}},
+		},
+		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatalf("new server: %v", err)
+	}
+	srv.baseCtx = context.Background()
+	srv.now = func() time.Time { return testNow }
+	return srv.Handler()
+}
+
+// E-2c — ejecting the drive that holds the only local whole-guest backup must be refused.
+//
+// This is a REGRESSION GUARD on a live configuration, not a hypothetical. E-1 (2026-07-28) moved the
+// vzdump target onto each demo box's secondary drive, and `RoleForStorage` types a local-dir on a
+// non-system device as user-data — so the pre-existing role gate PASSES it and the customer could
+// self-serve eject the drive holding their backups. It would have succeeded silently.
+//
+// The assertion is on the CONSEQUENCE (the request is refused) plus the remedy being named, because a
+// refusal the customer cannot act on just moves the failure.
+func TestEjectRefusedOnTheBackupTargetDrive(t *testing.T) {
+	h := backupTargetServer(t)
+	rr := do(t, h, http.MethodPost, "/disks/eject", "A", `{"vmid":8200,"where":"/mnt/nvme-1tb"}`)
+	if rr.Code == http.StatusOK {
+		t.Fatalf("eject of the backup-target drive SUCCEEDED (%d) — the box would silently lose its "+
+			"local drive-loss protection with nothing alarming", rr.Code)
+	}
+	body := rr.Body.String()
+	if !strings.Contains(body, "felhom-backup") {
+		t.Errorf("refusal must NAME the backup target so the customer knows which role blocks it; got: %s", body)
+	}
+	if !strings.Contains(strings.ToLower(body), "reassign") {
+		t.Errorf("refusal must name the REMEDY (reassign the target first), else it is a dead end; got: %s", body)
+	}
+}
+
+// Decommission strands the target just as thoroughly as eject — it migrates data off and retires the
+// drive. Same gate, asserted separately because it is a different handler and a different caller.
+func TestDecommissionRefusedOnTheBackupTargetDrive(t *testing.T) {
+	h := backupTargetServer(t)
+	rr := do(t, h, http.MethodPost, "/disks/decommission", "A", `{"vmid":8200,"where":"/mnt/nvme-1tb"}`)
+	if rr.Code == http.StatusOK {
+		t.Fatalf("decommission of the backup-target drive SUCCEEDED (%d)", rr.Code)
+	}
+	if !strings.Contains(rr.Body.String(), "felhom-backup") {
+		t.Errorf("refusal must name the backup target; got: %s", rr.Body.String())
+	}
+}
+
+// THE OVER-CORRECTION GUARD, and the reason this is a narrow gate instead of a role reclassification.
+//
+// The tempting fix — make RoleForStorage return RoleBackup for the target — would also refuse every
+// OTHER user-data drive op on a box, and on the demo boxes it would refuse the customer's own data
+// drive, because that drive IS the target. This pins that a non-target drive stays ejectable: the new
+// gate must block exactly one drive, not harden the whole eject path.
+//
+// It asserts "not blocked BY THIS GATE" rather than "succeeds", because eject has other legitimate
+// failure modes in a fake harness; what must never appear is this gate's message.
+func TestEjectStillAllowedOnANonTargetDrive(t *testing.T) {
+	h := backupTargetServer(t)
+	rr := do(t, h, http.MethodPost, "/disks/eject", "A", `{"vmid":8200,"where":"/mnt/spare"}`)
+	if strings.Contains(rr.Body.String(), "whole-guest backup target") {
+		t.Fatalf("the backup-target gate blocked a NON-target drive (/mnt/spare) — over-correction: "+
+			"it must block exactly the target, not harden the whole eject path; got: %s", rr.Body.String())
+	}
+}
+
+// E-2 — GET /disks must FLAG the backup-target drive, because the controller cannot work it out.
+//
+// The controller's own StoragePath.BackupTarget is customer INTENT, and on a box migrated by hand
+// (E-1, both demo boxes) nobody ever assigned it — intent is empty while the drive really is the
+// target. Without this flag the absent-target alarm could not name the drive on exactly the boxes
+// that have one, which is the only place it currently matters.
+func TestDisksFlagsTheBackupTargetDrive(t *testing.T) {
+	h := backupTargetServer(t)
+	rr := do(t, h, http.MethodGet, "/disks", "A", "")
+	if rr.Code != http.StatusOK {
+		t.Fatalf("GET /disks = %d, body %s", rr.Code, rr.Body.String())
+	}
+	body := rr.Body.String()
+	// The target must be flagged and the non-target must not be — asserted as a pair, since a
+	// blanket true would satisfy a naive "is it flagged?" check.
+	var got struct {
+		Data struct {
+			Disks []struct {
+				Name         string `json:"name"`
+				BackupTarget bool   `json:"backup_target"`
+			} `json:"disks"`
+		} `json:"data"`
+	}
+	if err := json.Unmarshal([]byte(body), &got); err != nil {
+		t.Fatalf("decode: %v (body %s)", err, body)
+	}
+	seen := map[string]bool{}
+	for _, d := range got.Data.Disks {
+		seen[d.Name] = d.BackupTarget
+	}
+	if !seen["felhom-backup"] {
+		t.Errorf("felhom-backup is the primary tier's storage but backup_target is false; body: %s", body)
+	}
+	if seen["spare-drive"] {
+		t.Errorf("spare-drive is NOT the target but was flagged — a blanket true is not a signal; body: %s", body)
+	}
+}

internal/localapi/backup_target_move_test.go

@@ -0,0 +1,174 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"io"
+	"log/slog"
+	"net/http"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
+)
+
+// wrapperRunner captures the wrapper vector without ever running sudo. The wrapper IS the security
+// boundary, so tests substitute it rather than bypassing it — what is asserted here is the ORDER and
+// the ARGUMENTS the agent sends, which is the agent's half of the contract.
+type wrapperRunner struct {
+	calls  [][]string
+	failOn string // verb to fail, "" = all succeed
+}
+
+func (r *wrapperRunner) Run(_ context.Context, name string, args ...string) ([]byte, []byte, error) {
+	r.calls = append(r.calls, append([]string{name}, args...))
+	if len(args) > 0 && args[0] == r.failOn {
+		return nil, []byte("felhom-backup-target-apply: REFUSED: synthetic " + r.failOn + " failure\n"),
+			io.ErrUnexpectedEOF
+	}
+	return nil, nil, nil
+}
+
+func (r *wrapperRunner) verbs() []string {
+	var out []string
+	for _, c := range r.calls {
+		if len(c) > 1 {
+			out = append(out, c[1])
+		}
+	}
+	return out
+}
+
+// moveServer builds a server with /mnt/data mounted on its own device and / on another, plus a
+// throwaway agent.json the move can rewrite.
+func moveServer(t *testing.T, run *wrapperRunner) (http.Handler, string) {
+	t.Helper()
+	dir := t.TempDir()
+	cfgPath := filepath.Join(dir, "agent.json")
+	// An UNKNOWN key is deliberately present: the rewrite must preserve it verbatim.
+	seed := `{"backup":{"local_backup_target":"local","local_backup_retention":3},"some_future_key":{"keep":"me"}}`
+	if err := os.WriteFile(cfgPath, []byte(seed), 0o600); err != nil {
+		t.Fatalf("seed config: %v", err)
+	}
+	hr := fakeHostReader{mounts: []storage.Mount{
+		{Device: "/dev/sda1", MountPoint: "/"},
+		{Device: "/dev/sdb1", MountPoint: "/mnt/data"},
+	}}
+	srv, err := NewServer(Options{
+		ListenAddr: "127.0.0.1:0",
+		Guests:     &fakeGuests{}, Backups: &fakeBackups{}, Store: &fakeStore{}, Storage: fakeStorage{},
+		Tokens: staticTokens{"A": 8200}, HostReader: hr,
+		Privileged: run, ConfigPath: cfgPath, StateDir: dir,
+		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatalf("new server: %v", err)
+	}
+	srv.baseCtx = context.Background()
+	return srv.Handler(), cfgPath
+}
+
+// THE ORDER IS THE CONTRACT: create → grant → config. Reversed, a config pointing at an ungranted
+// storage makes every backup 403 on first run, which is precisely what E-1 hit (finding F-3).
+func TestBackupTargetMoveOrdersCreateThenGrantThenConfig(t *testing.T) {
+	run := &wrapperRunner{}
+	h, cfgPath := moveServer(t, run)
+
+	rr := do(t, h, http.MethodPost, "/backup/target", "A", `{"vmid":8200,"where":"/mnt/data"}`)
+	if rr.Code != http.StatusOK {
+		t.Fatalf("move = %d, body %s", rr.Code, rr.Body.String())
+	}
+	got := strings.Join(run.verbs(), ",")
+	if got != "create,grant" {
+		t.Fatalf("wrapper verbs = %q, want create,grant (in that order)", got)
+	}
+	// The config must have been written only AFTER both wrapper calls succeeded.
+	raw, _ := os.ReadFile(cfgPath)
+	var doc map[string]json.RawMessage
+	if err := json.Unmarshal(raw, &doc); err != nil {
+		t.Fatalf("config unreadable after move: %v", err)
+	}
+	var bk map[string]any
+	_ = json.Unmarshal(doc["backup"], &bk)
+	if bk["local_backup_target"] != "felhom-backup" {
+		t.Errorf("local_backup_target = %v, want felhom-backup", bk["local_backup_target"])
+	}
+	// Unknown keys preserved verbatim — the property that made E-1's hand edit safe.
+	if _, ok := doc["some_future_key"]; !ok {
+		t.Error("the rewrite DROPPED an unknown top-level key — a typed round-trip would do this " +
+			"and silently discard config this build does not know about")
+	}
+	// Sibling keys inside `backup` survive too.
+	if bk["local_backup_retention"] == nil {
+		t.Error("the rewrite dropped local_backup_retention from the backup section")
+	}
+}
+
+// A FAILED GRANT MUST NOT LEAVE THE CONFIG POINTING AT THE NEW STORAGE. That state is exactly E-1's
+// 403-on-every-backup: the tier looks configured and cannot write.
+func TestBackupTargetMoveDoesNotRepointWhenTheGrantFails(t *testing.T) {
+	run := &wrapperRunner{failOn: "grant"}
+	h, cfgPath := moveServer(t, run)
+
+	rr := do(t, h, http.MethodPost, "/backup/target", "A", `{"vmid":8200,"where":"/mnt/data"}`)
+	if rr.Code == http.StatusOK {
+		t.Fatalf("move SUCCEEDED despite a failed grant (%d)", rr.Code)
+	}
+	if !strings.Contains(rr.Body.String(), "403") {
+		t.Errorf("the error should name the consequence (backups would 403); got %s", rr.Body.String())
+	}
+	raw, _ := os.ReadFile(cfgPath)
+	if strings.Contains(string(raw), "felhom-backup") {
+		t.Fatal("the config was repointed at a storage the agent cannot write to — every backup " +
+			"would 403 while the tier reported as configured")
+	}
+}
+
+// It must NOT restart the agent itself. Restarting with a backup in flight cancels the wait and
+// records a spurious tier failure for a backup that actually succeeded — E-1 did exactly that to a
+// felhom-pbs run. Only the caller can re-check in-flight work immediately before restarting.
+func TestBackupTargetMoveReportsRestartRequiredRatherThanRestarting(t *testing.T) {
+	run := &wrapperRunner{}
+	h, _ := moveServer(t, run)
+	rr := do(t, h, http.MethodPost, "/backup/target", "A", `{"vmid":8200,"where":"/mnt/data"}`)
+	if !strings.Contains(rr.Body.String(), `"restart_required":true`) {
+		t.Errorf("response must tell the caller a restart is required; got %s", rr.Body.String())
+	}
+	for _, c := range run.calls {
+		joined := strings.Join(c, " ")
+		if strings.Contains(joined, "systemctl") || strings.Contains(joined, "restart") {
+			t.Fatalf("the handler restarted the agent itself: %q — the caller must do it behind its "+
+				"own in-flight check", joined)
+		}
+	}
+}
+
+// A path that is not a mountpoint is refused BEFORE sudo is reached (F-1): a subdirectory target
+// reports disconnected forever, and an unmounted path silently retargets onto the system drive.
+func TestBackupTargetMoveRefusesANonMountpoint(t *testing.T) {
+	run := &wrapperRunner{}
+	h, _ := moveServer(t, run)
+	rr := do(t, h, http.MethodPost, "/backup/target", "A", `{"vmid":8200,"where":"/mnt/data/sub"}`)
+	if rr.Code == http.StatusOK {
+		t.Fatal("a non-mountpoint was accepted as the backup target")
+	}
+	if len(run.calls) != 0 {
+		t.Errorf("a refused request still reached the privileged wrapper: %v", run.calls)
+	}
+}
+
+// The system disk is refused: a target there protects against corruption only, never drive loss —
+// which is the entire point of the move.
+func TestBackupTargetMoveRefusesTheSystemDisk(t *testing.T) {
+	run := &wrapperRunner{}
+	h, _ := moveServer(t, run)
+	rr := do(t, h, http.MethodPost, "/backup/target", "A", `{"vmid":8200,"where":"/"}`)
+	if rr.Code == http.StatusOK {
+		t.Fatal("the system disk was accepted as the backup target")
+	}
+	if len(run.calls) != 0 {
+		t.Errorf("a refused request still reached the privileged wrapper: %v", run.calls)
+	}
+}

internal/localapi/backup_tier_states_test.go

@@ -0,0 +1,133 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"io"
+	"log/slog"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+)
+
+// R-517 — the per-tier truth on GET /backup/status. BIGNIGHT: a successful 8.9 GB local backup,
+// then a failed PBS attempt on a storage that did not exist; the page (fed by the single latest
+// record) showed the 0-byte failure as "up to date" and the remote copy as present.
+
+func tierStatesOf(t *testing.T, srv *Server) []TierBackupState {
+	t.Helper()
+	w := do(t, srv.Handler(), "GET", "/backup/status", "A", "")
+	var resp struct {
+		Data BackupStatusResponse `json:"data"`
+	}
+	if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
+		t.Fatalf("decode: %v (%s)", err, w.Body.String())
+	}
+	return resp.Data.Tiers
+}
+
+func tierStatesServer(t *testing.T, st *fakeStore, targets []hub.StorageTarget) *Server {
+	t.Helper()
+	srv, err := NewServer(Options{
+		ListenAddr: "127.0.0.1:0", Guests: &fakeGuests{}, Backups: &fakeBackups{}, Store: st,
+		Storage: fakeStorage{targets: targets},
+		Tokens:  staticTokens{"A": 8200},
+		BackupTiers: []BackupTier{
+			{TargetID: "local", Cadence: 24 * time.Hour, Primary: true, Service: &fakeBackups{}},
+			{TargetID: "felhom-pbs", Cadence: 7 * 24 * time.Hour, Service: &fakeBackups{}},
+		},
+		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	srv.baseCtx = context.Background()
+	srv.now = func() time.Time { return testNow }
+	return srv
+}
+
+// RED-PROOF (run 2026-09-15, recorded in REPORT.md): with tierBackupStates filling LastSuccess from
+// pickLatestBackup(ctx, vmid, false, …) — an ATTEMPT — the pbs tier's last_success became the failed
+// 0-byte record and this failed at "pbs tier reports a failed attempt as its last success".
+func TestBackupStatus_TierStates_FailedTierNeverStandsInForSuccess(t *testing.T) {
+	st := &fakeStore{backups: []hub.Backup{
+		{TargetID: "local", VMID: 8200, Success: true, SizeBytes: 8877619753, StartedAt: "2026-06-10T11:03:23Z"},
+		{TargetID: "felhom-pbs", VMID: 8200, Success: false, Error: "storage 'felhom-pbs' does not exist", StartedAt: "2026-06-10T11:09:59Z"},
+	}}
+	srv := tierStatesServer(t, st, []hub.StorageTarget{{Name: "local", Type: "local"}}) // PBS storage ABSENT
+	tiers := tierStatesOf(t, srv)
+	if len(tiers) != 2 {
+		t.Fatalf("want 2 tiers, got %+v", tiers)
+	}
+	local, pbs := tiers[0], tiers[1]
+	if local.Target != "local" || local.LastSuccess == nil || local.LastSuccess.SizeBytes != 8877619753 || local.Storage != StoragePresencePresent {
+		t.Fatalf("local tier lost its successful backup: %+v", local)
+	}
+	if pbs.LastSuccess != nil {
+		t.Fatalf("pbs tier reports a failed attempt as its last success: %+v", pbs.LastSuccess)
+	}
+	if pbs.LastAttempt == nil || pbs.LastAttempt.Success || pbs.LastAttempt.Error == "" {
+		t.Fatalf("pbs tier's failed attempt is not reported as failed: %+v", pbs.LastAttempt)
+	}
+	if pbs.Storage != StoragePresenceAbsent {
+		t.Fatalf("pbs storage should read absent, got %q", pbs.Storage)
+	}
+	// The pre-R-517 field is unchanged (compat): still the newest record across targets.
+	w := do(t, srv.Handler(), "GET", "/backup/status", "A", "")
+	var resp struct {
+		Data BackupStatusResponse `json:"data"`
+	}
+	_ = json.Unmarshal(w.Body.Bytes(), &resp)
+	if resp.Data.Backup == nil || resp.Data.Backup.TargetID != "felhom-pbs" {
+		t.Fatalf("untargeted .backup changed meaning: %+v", resp.Data.Backup)
+	}
+}
+
+// /backup/tiers advertises storage presence, tri-state.
+func TestBackupTiers_StoragePresence(t *testing.T) {
+	srv := tierStatesServer(t, &fakeStore{}, []hub.StorageTarget{{Name: "local"}})
+	w := do(t, srv.Handler(), "GET", "/backup/tiers", "A", "")
+	var resp struct {
+		Data BackupTiersResponse `json:"data"`
+	}
+	if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
+		t.Fatal(err)
+	}
+	got := map[string]string{}
+	for _, ti := range resp.Data.Tiers {
+		got[ti.Target] = ti.Storage
+	}
+	if got["local"] != "present" || got["felhom-pbs"] != "absent" {
+		t.Fatalf("storage presence wrong: %v", got)
+	}
+	// An unreadable storage view is "unknown", never "absent".
+	srv.storage = tierErrStorage{}
+	if p := srv.storagePresence(context.Background(), "felhom-pbs"); p != StoragePresenceUnknown {
+		t.Fatalf("unreadable storage view must be unknown, got %q", p)
+	}
+}
+
+type tierErrStorage struct{}
+
+func (tierErrStorage) Observe(context.Context) ([]hub.StorageTarget, error) {
+	return nil, context.DeadlineExceeded
+}
+
+// A targeted request keeps the pre-R-517 bytes (no tiers array).
+func TestBackupStatus_TargetedHasNoTiers(t *testing.T) {
+	srv := tierStatesServer(t, &fakeStore{}, []hub.StorageTarget{{Name: "local"}})
+	w := do(t, srv.Handler(), "GET", "/backup/status?target=local", "A", "")
+	if json.Valid(w.Body.Bytes()) && containsKey(w.Body.Bytes(), "tiers") {
+		t.Fatalf("targeted status grew a tiers array: %s", w.Body.String())
+	}
+}
+
+func containsKey(b []byte, key string) bool {
+	var m struct {
+		Data map[string]json.RawMessage `json:"data"`
+	}
+	_ = json.Unmarshal(b, &m)
+	_, ok := m.Data[key]
+	return ok
+}

internal/localapi/backup_tiers.go

@@ -0,0 +1,59 @@
+package localapi
+
+import "time"
+
+// normalizeBackupTiers resolves the tier list the Server serves.
+//
+// Contract (R-82), and the reason this is a named function rather than inline setup: the UNTARGETED
+// local-API endpoints must keep behaving exactly as they did before multi-tier existed, forever.
+// That property lives here.
+//
+//   - tiers == nil → synthesize ONE tier from the legacy (Backups, BackupCadence) pair and mark it
+//     primary. This is the pre-R-82 shape; every existing caller and test hits this path.
+//   - tiers supplied → keep order but hoist the primary to the front; if none is marked primary,
+//     the FIRST becomes primary (a tier list with no primary would leave untargeted requests with
+//     nothing to act on, which would silently stop backups).
+//   - tiers with a nil Service are dropped: a tier with no runner cannot back anything up, and
+//     advertising it would be an "applied and empty" tier — the exact fault R-82 exists to fix.
+func normalizeBackupTiers(tiers []BackupTier, legacy BackupService, cadence time.Duration) []BackupTier {
+	usable := make([]BackupTier, 0, len(tiers))
+	for _, t := range tiers {
+		if t.Service == nil || t.TargetID == "" {
+			continue
+		}
+		if t.Cadence <= 0 {
+			t.Cadence = cadence
+		}
+		if t.WaitTimeout <= 0 {
+			t.WaitTimeout = 2 * time.Hour
+		}
+		usable = append(usable, t)
+	}
+	if len(usable) == 0 {
+		if legacy == nil {
+			return nil
+		}
+		return []BackupTier{{TargetID: "", Cadence: cadence, WaitTimeout: 2 * time.Hour, Primary: true, Service: legacy}}
+	}
+	primary := -1
+	for i, t := range usable {
+		if t.Primary {
+			primary = i
+			break
+		}
+	}
+	if primary < 0 {
+		primary = 0
+	}
+	out := make([]BackupTier, 0, len(usable))
+	usable[primary].Primary = true
+	out = append(out, usable[primary])
+	for i, t := range usable {
+		if i == primary {
+			continue
+		}
+		t.Primary = false
+		out = append(out, t)
+	}
+	return out
+}

internal/localapi/backup_tiers_test.go

@@ -0,0 +1,671 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"io"
+	"log/slog"
+	"net/http"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+)
+
+// R-82 Slice A — per-target cadence, due and runner.
+//
+// The agent and the controller deploy INDEPENDENTLY. A new agent will serve old controllers for as
+// long as it takes the fleet to catch up, so the untargeted contract is frozen, not merely
+// "probably fine". These tests pin that freeze; the multi-tier behaviour is additive on top.
+
+// tieredServer builds a two-tier server: primary "local" (24h) + "felhom-pbs" (7d), each with its
+// own runner, exactly as main.go wires it.
+func tieredServer(t *testing.T, st *fakeStore, localSvc, pbsSvc *fakeBackups) *Server {
+	t.Helper()
+	srv, err := NewServer(Options{
+		ListenAddr: "127.0.0.1:0",
+		Guests:     &fakeGuests{},
+		Backups:    localSvc,
+		Store:      st,
+		// Both tier targets must be PRESENT in the storage view: since v0.102.0 a tier whose target
+		// storage is absent DEFERS. A real box has both; a fake with no targets would silently
+		// defer every tier and make these assertions vacuous.
+		Storage: fakeStorage{targets: []hub.StorageTarget{
+			{Name: "local", Type: "local"},
+			{Name: "felhom-pbs", Type: "pbs"},
+		}},
+		Tokens: staticTokens{"A": 8200, "B": 9300},
+		BackupTiers: []BackupTier{
+			{TargetID: "local", Cadence: 24 * time.Hour, Primary: true, Service: localSvc},
+			{TargetID: "felhom-pbs", Cadence: 7 * 24 * time.Hour, Service: pbsSvc},
+		},
+		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatalf("new server: %v", err)
+	}
+	srv.baseCtx = context.Background()
+	srv.now = func() time.Time { return testNow }
+	return srv
+}
+
+// seen returns the vmids this fake runner was invoked for (mutex-guarded — the backup runs on a
+// goroutine).
+func (f *fakeBackups) seen() []int {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	return append([]int(nil), f.vmids...)
+}
+
+// waitFor polls cond for up to 2s. POST /backup is fire-and-forget, so the assertion has to wait
+// for the goroutine rather than assume it has run.
+func waitFor(t *testing.T, cond func() bool) {
+	t.Helper()
+	deadline := time.Now().Add(2 * time.Second)
+	for time.Now().Before(deadline) {
+		if cond() {
+			return
+		}
+		time.Sleep(5 * time.Millisecond)
+	}
+	t.Fatalf("condition not met within 2s")
+}
+
+func backupAt(target string, vmid int, ago time.Duration, ok bool) hub.Backup {
+	return hub.Backup{
+		TargetID:  target,
+		VMID:      vmid,
+		Success:   ok,
+		StartedAt: testNow.Add(-ago).Format(time.RFC3339),
+	}
+}
+
+// ── RED-PROOF 1 — old controller ↔ new agent ────────────────────────────────────────────────
+//
+// An old controller sends `GET /backup/due` with no query string and parses the pre-R-82 response.
+// The response must be BYTE-IDENTICAL — not merely semantically similar. A stray `"target":"local"`
+// key is harmless to a tolerant JSON decoder and fatal to a strict one, and we do not get to choose
+// which the deployed fleet has.
+//
+// COMPANION RED-PROOF (observed): drop the `omitempty` from BackupDueResponse.Target and have
+// tierFromRequest echo the primary's id for an untargeted request → this test fails with the
+// observed body carrying `"target":"local"`. Restored.
+func TestBackupDue_Untargeted_ResponseBytesUnchanged(t *testing.T) {
+	st := &fakeStore{}
+	st.RecordBackup(backupAt("local", 8200, 2*time.Hour, true))
+	h := tieredServer(t, st, &fakeBackups{}, &fakeBackups{}).Handler()
+
+	rr := do(t, h, "GET", "/backup/due", "A", "")
+	if rr.Code != http.StatusOK {
+		t.Fatalf("status = %d, want 200", rr.Code)
+	}
+	var got map[string]any
+	if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
+		t.Fatalf("unmarshal: %v (body %s)", err, rr.Body.String())
+	}
+	data, _ := got["data"].(map[string]any)
+	if data == nil {
+		t.Fatalf("no data object in %s", rr.Body.String())
+	}
+	if _, present := data["target"]; present {
+		t.Fatalf("UNTARGETED response MUST NOT carry a target key — an old controller sees a changed contract; body: %s", rr.Body.String())
+	}
+	if data["due"] != false {
+		t.Fatalf("2h-old local backup under a 24h cadence must not be due; body: %s", rr.Body.String())
+	}
+	if data["reason"] != "within cadence window" {
+		t.Fatalf("reason string changed: %v", data["reason"])
+	}
+}
+
+// The untargeted verdict must come from the PRIMARY tier's cadence, not from whichever tier
+// happens to be freshest. With a stale local and a fresh PBS backup, untargeted must say DUE.
+func TestBackupDue_Untargeted_UsesPrimaryTierOnly(t *testing.T) {
+	st := &fakeStore{}
+	st.RecordBackup(backupAt("local", 8200, 30*time.Hour, true))     // stale for 24h cadence
+	st.RecordBackup(backupAt("felhom-pbs", 8200, 1*time.Hour, true)) // fresh, different tier
+	h := tieredServer(t, st, &fakeBackups{}, &fakeBackups{}).Handler()
+
+	var resp struct {
+		Data BackupDueResponse `json:"data"`
+	}
+	rr := do(t, h, "GET", "/backup/due", "A", "")
+	if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
+		t.Fatal(err)
+	}
+	if !resp.Data.Due {
+		t.Fatalf("a fresh backup on ANOTHER tier must not satisfy the primary's cadence; got %+v", resp.Data)
+	}
+}
+
+// ── Per-tier due-ness ────────────────────────────────────────────────────────────────────────
+
+// THE POINT OF THE WHOLE SLICE: a fresh daily local backup must NOT satisfy the weekly PBS tier.
+// Without the per-target filter in latestSuccessfulBackupForTarget the DR tier would never run —
+// which is exactly today's "applied and empty" state, re-created in code.
+func TestBackupDue_PerTier_LocalFreshDoesNotSatisfyPBS(t *testing.T) {
+	st := &fakeStore{}
+	st.RecordBackup(backupAt("local", 8200, 2*time.Hour, true))         // fresh daily
+	st.RecordBackup(backupAt("felhom-pbs", 8200, 8*24*time.Hour, true)) // 8d — past the 7d weekly
+	h := tieredServer(t, st, &fakeBackups{}, &fakeBackups{}).Handler()
+
+	var local, pbs struct {
+		Data BackupDueResponse `json:"data"`
+	}
+	if err := json.Unmarshal(do(t, h, "GET", "/backup/due?target=local", "A", "").Body.Bytes(), &local); err != nil {
+		t.Fatal(err)
+	}
+	if err := json.Unmarshal(do(t, h, "GET", "/backup/due?target=felhom-pbs", "A", "").Body.Bytes(), &pbs); err != nil {
+		t.Fatal(err)
+	}
+	if local.Data.Due {
+		t.Fatalf("local tier: 2h old under 24h cadence must NOT be due; got %+v", local.Data)
+	}
+	if !pbs.Data.Due {
+		t.Fatalf("PBS tier: 8d old under a 7d cadence MUST be due; got %+v", pbs.Data)
+	}
+	if pbs.Data.Target != "felhom-pbs" || local.Data.Target != "local" {
+		t.Fatalf("a targeted response must echo its tier; got local=%q pbs=%q", local.Data.Target, pbs.Data.Target)
+	}
+}
+
+// A 6-day-old PBS snapshot is INSIDE the weekly window — it must not be due. The mirror of the
+// hub-side threshold test in Slice C, asserted here at the source of truth.
+func TestBackupDue_PerTier_PBSWithinWeeklyWindow(t *testing.T) {
+	st := &fakeStore{}
+	st.RecordBackup(backupAt("felhom-pbs", 8200, 6*24*time.Hour, true))
+	h := tieredServer(t, st, &fakeBackups{}, &fakeBackups{}).Handler()
+
+	var pbs struct {
+		Data BackupDueResponse `json:"data"`
+	}
+	if err := json.Unmarshal(do(t, h, "GET", "/backup/due?target=felhom-pbs", "A", "").Body.Bytes(), &pbs); err != nil {
+		t.Fatal(err)
+	}
+	if pbs.Data.Due {
+		t.Fatalf("6d old under a 7d cadence must NOT be due; got %+v", pbs.Data)
+	}
+}
+
+// A failed backup must not satisfy any tier's cadence (pre-existing rule, re-asserted per-tier).
+func TestBackupDue_PerTier_FailedBackupDoesNotCount(t *testing.T) {
+	st := &fakeStore{}
+	st.RecordBackup(backupAt("felhom-pbs", 8200, time.Hour, false))
+	h := tieredServer(t, st, &fakeBackups{}, &fakeBackups{}).Handler()
+
+	var pbs struct {
+		Data BackupDueResponse `json:"data"`
+	}
+	if err := json.Unmarshal(do(t, h, "GET", "/backup/due?target=felhom-pbs", "A", "").Body.Bytes(), &pbs); err != nil {
+		t.Fatal(err)
+	}
+	if !pbs.Data.Due {
+		t.Fatalf("a FAILED backup must not satisfy the cadence; got %+v", pbs.Data)
+	}
+}
+
+// The fail-safe-toward-due rule survives per-tier: an unparseable timestamp yields DUE.
+// A spurious backup is cheap; a skipped one is not.
+func TestBackupDue_PerTier_UnparseableTimestampIsDue(t *testing.T) {
+	st := &fakeStore{}
+	st.RecordBackup(hub.Backup{TargetID: "felhom-pbs", VMID: 8200, Success: true, StartedAt: "not-a-time"})
+	h := tieredServer(t, st, &fakeBackups{}, &fakeBackups{}).Handler()
+
+	var pbs struct {
+		Data BackupDueResponse `json:"data"`
+	}
+	if err := json.Unmarshal(do(t, h, "GET", "/backup/due?target=felhom-pbs", "A", "").Body.Bytes(), &pbs); err != nil {
+		t.Fatal(err)
+	}
+	if !pbs.Data.Due {
+		t.Fatalf("unparseable timestamp must fail SAFE toward due; got %+v", pbs.Data)
+	}
+}
+
+// An unknown target is a 400 — never a silent fallback to the primary. A controller asking about a
+// tier this agent does not serve must find out, not be handed a different tier's freshness and act
+// on it.
+func TestBackupDue_UnknownTarget_IsAnErrorNotAFallback(t *testing.T) {
+	st := &fakeStore{}
+	st.RecordBackup(backupAt("local", 8200, 2*time.Hour, true))
+	h := tieredServer(t, st, &fakeBackups{}, &fakeBackups{}).Handler()
+
+	rr := do(t, h, "GET", "/backup/due?target=does-not-exist", "A", "")
+	if rr.Code != http.StatusBadRequest {
+		t.Fatalf("unknown target must be 400 (got %d, body %s)", rr.Code, rr.Body.String())
+	}
+}
+
+// ── Tier advertisement ───────────────────────────────────────────────────────────────────────
+
+// GET /backup/tiers is the controller's capability probe. Primary must be first and flagged, so a
+// controller can tell which tier the untargeted endpoints act on.
+func TestBackupTiers_AdvertisesPrimaryFirst(t *testing.T) {
+	h := tieredServer(t, &fakeStore{}, &fakeBackups{}, &fakeBackups{}).Handler()
+	var resp struct {
+		Data BackupTiersResponse `json:"data"`
+	}
+	rr := do(t, h, "GET", "/backup/tiers", "A", "")
+	if rr.Code != http.StatusOK {
+		t.Fatalf("status = %d", rr.Code)
+	}
+	if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
+		t.Fatal(err)
+	}
+	if len(resp.Data.Tiers) != 2 {
+		t.Fatalf("want 2 tiers, got %+v", resp.Data.Tiers)
+	}
+	if !resp.Data.Tiers[0].Primary || resp.Data.Tiers[0].Target != "local" {
+		t.Fatalf("primary must be first and flagged; got %+v", resp.Data.Tiers)
+	}
+	if resp.Data.Tiers[1].Target != "felhom-pbs" || resp.Data.Tiers[1].CadenceSeconds != int64((7*24*time.Hour).Seconds()) {
+		t.Fatalf("PBS tier mis-advertised: %+v", resp.Data.Tiers[1])
+	}
+}
+
+// ── POST /backup routing + per-tier single-flight ────────────────────────────────────────────
+
+// A targeted POST must run THAT tier's runner. Routing both tiers to one runner would silently
+// write every "PBS" backup to local — a DR tier that reports success and stores nothing.
+func TestBackupPost_RoutesToTheTargetsOwnRunner(t *testing.T) {
+	local, pbs := &fakeBackups{}, &fakeBackups{}
+	srv := tieredServer(t, &fakeStore{}, local, pbs)
+	h := srv.Handler()
+
+	if rr := do(t, h, "POST", "/backup?target=felhom-pbs", "A", ""); rr.Code != http.StatusAccepted {
+		t.Fatalf("status = %d, body %s", rr.Code, rr.Body.String())
+	}
+	waitFor(t, func() bool { return len(pbs.seen()) == 1 })
+
+	if got := len(local.seen()); got != 0 {
+		t.Fatalf("the LOCAL runner must not have run for a PBS-targeted request (ran %d times)", got)
+	}
+}
+
+// Untargeted POST routes to the primary — the old controller's path.
+func TestBackupPost_UntargetedRoutesToPrimary(t *testing.T) {
+	local, pbs := &fakeBackups{}, &fakeBackups{}
+	h := tieredServer(t, &fakeStore{}, local, pbs).Handler()
+
+	if rr := do(t, h, "POST", "/backup", "A", ""); rr.Code != http.StatusAccepted {
+		t.Fatalf("status = %d", rr.Code)
+	}
+	waitFor(t, func() bool { return len(local.seen()) == 1 })
+	if got := len(pbs.seen()); got != 0 {
+		t.Fatalf("untargeted POST must not touch a non-primary tier (ran %d times)", got)
+	}
+}
+
+// ONE BACKUP AT A TIME PER GUEST (operator ruling 2026-07-26). A second tier's POST while another
+// tier is still in flight must be REFUSED — vzdump holds the guest lock, so it could not succeed
+// anyway, and attempting it records a spurious failure that leaves the tier permanently due.
+//
+// Crucially it must NOT be handed the busy tier's job id: that is exactly how a caller comes to
+// believe its own backup ran.
+func TestBackupPost_SecondTierRefusedWhileAnotherInFlight(t *testing.T) {
+	localGate := make(chan struct{})
+	local := &fakeBackups{gate: localGate}
+	pbs := &fakeBackups{}
+	h := tieredServer(t, &fakeStore{}, local, pbs).Handler()
+
+	var first BackupResponse
+	rr1 := do(t, h, "POST", "/backup", "A", "") // local; blocks on the gate
+	if err := json.Unmarshal(rr1.Body.Bytes(), &struct {
+		Data *BackupResponse `json:"data"`
+	}{Data: &first}); err != nil {
+		t.Fatal(err)
+	}
+	waitFor(t, func() bool { return len(local.seen()) == 1 })
+
+	rr2 := do(t, h, "POST", "/backup?target=felhom-pbs", "A", "")
+	if rr2.Code != http.StatusConflict {
+		t.Fatalf("a second tier must be REFUSED while another is in flight; got %d body %s", rr2.Code, rr2.Body.String())
+	}
+	// The STRUCTURAL requirement: the refusal must not return a job the caller could mistake for
+	// its own. It is a 409 with ok=false and NO data object, so nothing is parseable as "my job".
+	// (Naming the busy job in the human-readable message is deliberate and useful for diagnosis —
+	// what must never happen is handing it back as BackupResponse.JobID on a 202.)
+	var envelope struct {
+		OK    bool            `json:"ok"`
+		Data  *BackupResponse `json:"data"`
+		Error string          `json:"error"`
+	}
+	if err := json.Unmarshal(rr2.Body.Bytes(), &envelope); err != nil {
+		t.Fatal(err)
+	}
+	if envelope.OK {
+		t.Fatalf("a refusal must not be ok=true; body %s", rr2.Body.String())
+	}
+	if envelope.Data != nil && envelope.Data.JobID != "" {
+		t.Fatalf("the refusal must NOT hand back a job id as the caller's own (got %q); body %s",
+			envelope.Data.JobID, rr2.Body.String())
+	}
+	if !strings.Contains(rr2.Body.String(), "local") {
+		t.Fatalf("the refusal must NAME the busy tier so the caller can diagnose; body %s", rr2.Body.String())
+	}
+	if got := len(pbs.seen()); got != 0 {
+		t.Fatalf("the refused tier must NOT have started a backup (ran %d times)", got)
+	}
+	close(localGate)
+}
+
+// Once the busy tier finishes, the other tier may start — and gets its OWN tier-scoped job id.
+func TestBackupPost_SecondTierAllowedAfterFirstFinishes(t *testing.T) {
+	local, pbs := &fakeBackups{}, &fakeBackups{}
+	h := tieredServer(t, &fakeStore{}, local, pbs).Handler()
+
+	var first, second BackupResponse
+	if err := json.Unmarshal(do(t, h, "POST", "/backup", "A", "").Body.Bytes(), &struct {
+		Data *BackupResponse `json:"data"`
+	}{Data: &first}); err != nil {
+		t.Fatal(err)
+	}
+	waitFor(t, func() bool { return len(local.seen()) == 1 })
+	// Wait for the local job to leave the in-flight phases.
+	waitFor(t, func() bool {
+		rr := do(t, h, "GET", "/backup/status", "A", "")
+		return strings.Contains(rr.Body.String(), `"phase":"done"`)
+	})
+
+	rr2 := do(t, h, "POST", "/backup?target=felhom-pbs", "A", "")
+	if rr2.Code != http.StatusAccepted {
+		t.Fatalf("after the first tier finished the second must be allowed; got %d body %s", rr2.Code, rr2.Body.String())
+	}
+	if err := json.Unmarshal(rr2.Body.Bytes(), &struct {
+		Data *BackupResponse `json:"data"`
+	}{Data: &second}); err != nil {
+		t.Fatal(err)
+	}
+	if second.JobID == first.JobID {
+		t.Fatalf("job ids must stay tier-scoped: %q vs %q", first.JobID, second.JobID)
+	}
+	waitFor(t, func() bool { return len(pbs.seen()) == 1 })
+}
+
+// `snapshotted` still counts as in flight — the vzdump is uploading and still holds the guest lock.
+// Checking only `running` (the pre-R-82 code) left a window where a second POST started a real
+// second vzdump.
+func TestBackupPost_SnapshottedCountsAsInFlight(t *testing.T) {
+	gate := make(chan struct{})
+	local := &fakeBackups{gate: gate, fireSnapshot: true} // fires onSnapshot, then blocks
+	pbs := &fakeBackups{}
+	h := tieredServer(t, &fakeStore{}, local, pbs).Handler()
+
+	do(t, h, "POST", "/backup", "A", "")
+	waitFor(t, func() bool {
+		rr := do(t, h, "GET", "/backup/status", "A", "")
+		return strings.Contains(rr.Body.String(), `"phase":"snapshotted"`)
+	})
+
+	rr := do(t, h, "POST", "/backup?target=felhom-pbs", "A", "")
+	if rr.Code != http.StatusConflict {
+		t.Fatalf("a SNAPSHOTTED backup still holds the guest — a second tier must be refused; got %d body %s", rr.Code, rr.Body.String())
+	}
+	close(gate)
+}
+
+// Same tier, still single-flight: a second POST to a running tier returns the SAME job.
+func TestBackupPost_SameTierStillSingleFlight(t *testing.T) {
+	gate := make(chan struct{})
+	local := &fakeBackups{gate: gate}
+	h := tieredServer(t, &fakeStore{}, local, &fakeBackups{}).Handler()
+
+	var a, b BackupResponse
+	if err := json.Unmarshal(do(t, h, "POST", "/backup", "A", "").Body.Bytes(), &struct {
+		Data *BackupResponse `json:"data"`
+	}{Data: &a}); err != nil {
+		t.Fatal(err)
+	}
+	if err := json.Unmarshal(do(t, h, "POST", "/backup", "A", "").Body.Bytes(), &struct {
+		Data *BackupResponse `json:"data"`
+	}{Data: &b}); err != nil {
+		t.Fatal(err)
+	}
+	if a.JobID != b.JobID {
+		t.Fatalf("same tier must single-flight: %q vs %q", a.JobID, b.JobID)
+	}
+	close(gate)
+}
+
+// ── normalizeBackupTiers — the compatibility core ────────────────────────────────────────────
+
+func TestNormalizeBackupTiers(t *testing.T) {
+	svc := &fakeBackups{}
+
+	t.Run("nil tiers synthesize the legacy single tier", func(t *testing.T) {
+		got := normalizeBackupTiers(nil, svc, 24*time.Hour)
+		if len(got) != 1 || !got[0].Primary || got[0].Cadence != 24*time.Hour {
+			t.Fatalf("legacy synthesis broken: %+v", got)
+		}
+	})
+
+	t.Run("primary is hoisted to the front", func(t *testing.T) {
+		got := normalizeBackupTiers([]BackupTier{
+			{TargetID: "felhom-pbs", Cadence: time.Hour, Service: svc},
+			{TargetID: "local", Cadence: time.Hour, Primary: true, Service: svc},
+		}, svc, 24*time.Hour)
+		if len(got) != 2 || got[0].TargetID != "local" || !got[0].Primary || got[1].Primary {
+			t.Fatalf("primary not hoisted / uniqueness broken: %+v", got)
+		}
+	})
+
+	t.Run("no primary marked → first becomes primary", func(t *testing.T) {
+		got := normalizeBackupTiers([]BackupTier{
+			{TargetID: "a", Cadence: time.Hour, Service: svc},
+			{TargetID: "b", Cadence: time.Hour, Service: svc},
+		}, svc, 24*time.Hour)
+		if len(got) != 2 || got[0].TargetID != "a" || !got[0].Primary {
+			t.Fatalf("want first-as-primary, got %+v", got)
+		}
+	})
+
+	t.Run("a tier with no runner is DROPPED, not advertised", func(t *testing.T) {
+		got := normalizeBackupTiers([]BackupTier{
+			{TargetID: "local", Cadence: time.Hour, Primary: true, Service: svc},
+			{TargetID: "felhom-pbs", Cadence: time.Hour, Service: nil},
+		}, svc, 24*time.Hour)
+		if len(got) != 1 || got[0].TargetID != "local" {
+			t.Fatalf("a serviceless tier must not be advertised (it could never run): %+v", got)
+		}
+	})
+}
+
+// R-82 Slice D: a tier whose TARGET STORAGE does not exist yet is DEFERRED, not due.
+//
+// A fresh box carries the offsite tier in its installer defaults, but `felhom-pbs` only appears when
+// the hub provisions the DR tier. Reporting "due" in that window would have the controller quiesce
+// the apps and fire a vzdump at a non-existent storage every cadence until provisioning happens.
+func TestBackupDue_TargetStorageMissing_Defers(t *testing.T) {
+	st := &fakeStore{}
+	// Storage view knows only "local" — the PBS tier's target is not provisioned yet.
+	srv, err := NewServer(Options{
+		ListenAddr: "127.0.0.1:0", Guests: &fakeGuests{}, Backups: &fakeBackups{}, Store: st,
+		Storage: fakeStorage{targets: []hub.StorageTarget{{Name: "local", Type: "local"}}},
+		Tokens:  staticTokens{"A": 8200},
+		BackupTiers: []BackupTier{
+			{TargetID: "local", Cadence: 24 * time.Hour, Primary: true, Service: &fakeBackups{}},
+			{TargetID: "felhom-pbs", Cadence: 7 * 24 * time.Hour, Service: &fakeBackups{}},
+		},
+		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	srv.baseCtx = context.Background()
+	srv.now = func() time.Time { return testNow }
+	h := srv.Handler()
+
+	var pbs, local struct {
+		Data BackupDueResponse `json:"data"`
+	}
+	if err := json.Unmarshal(do(t, h, "GET", "/backup/due?target=felhom-pbs", "A", "").Body.Bytes(), &pbs); err != nil {
+		t.Fatal(err)
+	}
+	if pbs.Data.Due {
+		t.Fatalf("an unprovisioned tier must DEFER, not fire a vzdump at a storage that does not exist; got %+v", pbs.Data)
+	}
+	if !strings.Contains(pbs.Data.Reason, "not present") {
+		t.Fatalf("the deferral must say WHY, or it is indistinguishable from a healthy tier; got %q", pbs.Data.Reason)
+	}
+	// The provisioned tier is unaffected — no evidence yet, so due.
+	if err := json.Unmarshal(do(t, h, "GET", "/backup/due?target=local", "A", "").Body.Bytes(), &local); err != nil {
+		t.Fatal(err)
+	}
+	if !local.Data.Due {
+		t.Fatalf("a PROVISIONED tier with no backup yet must still be due; got %+v", local.Data)
+	}
+}
+
+// A storage-view ERROR must NOT defer. "I could not check" is not "not there" — reading it that way
+// would silently suppress backups, the absence-is-not-failure rule this project keeps relearning.
+func TestBackupDue_StorageViewError_DoesNotSuppress(t *testing.T) {
+	srv, err := NewServer(Options{
+		ListenAddr: "127.0.0.1:0", Guests: &fakeGuests{}, Backups: &fakeBackups{}, Store: &fakeStore{},
+		Storage: errStorage{}, // reused from f2_role_fallback_test.go — Observe always fails
+		Tokens:  staticTokens{"A": 8200},
+		BackupTiers: []BackupTier{
+			{TargetID: "local", Cadence: 24 * time.Hour, Primary: true, Service: &fakeBackups{}},
+			{TargetID: "felhom-pbs", Cadence: 7 * 24 * time.Hour, Service: &fakeBackups{}},
+		},
+		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	srv.baseCtx = context.Background()
+	srv.now = func() time.Time { return testNow }
+
+	var pbs struct {
+		Data BackupDueResponse `json:"data"`
+	}
+	if err := json.Unmarshal(do(t, srv.Handler(), "GET", "/backup/due?target=felhom-pbs", "A", "").Body.Bytes(), &pbs); err != nil {
+		t.Fatal(err)
+	}
+	if !pbs.Data.Due {
+		t.Fatalf("a storage-view error must not suppress the backup (fail toward due); got %+v", pbs.Data)
+	}
+}
+
+// ── R-84: the cold in-memory store must not cause a redundant backup ─────────────────────────
+
+// archiveLister is a fakeBackups that ALSO knows when a backup last landed on its storage.
+type archiveLister struct {
+	*fakeBackups
+	at    time.Time
+	found bool
+	err   error
+}
+
+func (a archiveLister) NewestArchiveTime(context.Context, int) (time.Time, bool, error) {
+	return a.at, a.found, a.err
+}
+
+func listerServer(t *testing.T, st *fakeStore, pbsSvc BackupService) http.Handler {
+	t.Helper()
+	srv, err := NewServer(Options{
+		ListenAddr: "127.0.0.1:0", Guests: &fakeGuests{}, Backups: &fakeBackups{}, Store: st,
+		Storage: fakeStorage{targets: []hub.StorageTarget{{Name: "local"}, {Name: "felhom-pbs"}}},
+		Tokens:  staticTokens{"A": 8200},
+		BackupTiers: []BackupTier{
+			{TargetID: "local", Cadence: 24 * time.Hour, Primary: true, Service: &fakeBackups{}},
+			{TargetID: "felhom-pbs", Cadence: 7 * 24 * time.Hour, Service: pbsSvc},
+		},
+		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	srv.baseCtx = context.Background()
+	srv.now = func() time.Time { return testNow }
+	return srv.Handler()
+}
+
+func dueFor(t *testing.T, h http.Handler, target string) BackupDueResponse {
+	t.Helper()
+	var out struct {
+		Data BackupDueResponse `json:"data"`
+	}
+	if err := json.Unmarshal(do(t, h, "GET", "/backup/due?target="+target, "A", "").Body.Bytes(), &out); err != nil {
+		t.Fatal(err)
+	}
+	return out.Data
+}
+
+// THE R-84 CASE. The in-memory store is EMPTY (the agent just restarted), but the storage holds a
+// snapshot from 2 hours ago. The tier must NOT be due — otherwise every agent deploy costs a fresh
+// multi-hour offsite upload. Three redundant local backups were observed on demo-felhom in one
+// afternoon of deploys before this.
+//
+// COMPANION RED-PROOF (observed): delete the newestArchiveOn fold-in from handleBackupDue (the
+// pre-R-84 shape, in-memory only) → this fails with
+// "a restart must NOT make the tier due when the storage holds a 2h-old backup;
+//
+//	got {... Due:true Reason:no successful backup recorded yet ...}". Restored.
+func TestBackupDue_ColdStore_UsesStorageGroundTruth(t *testing.T) {
+	h := listerServer(t, &fakeStore{}, archiveLister{
+		fakeBackups: &fakeBackups{}, at: testNow.Add(-2 * time.Hour), found: true,
+	})
+	got := dueFor(t, h, "felhom-pbs")
+	if got.Due {
+		t.Fatalf("a restart must NOT make the tier due when the storage holds a 2h-old backup; got %+v", got)
+	}
+	if got.AgeSecs == nil || *got.AgeSecs != int64((2*time.Hour).Seconds()) {
+		t.Fatalf("the age must come from the storage; got %+v", got)
+	}
+}
+
+// Ground truth that is genuinely OLD still makes the tier due — this must not become a blanket
+// suppressor.
+func TestBackupDue_ColdStore_OldArchiveIsStillDue(t *testing.T) {
+	h := listerServer(t, &fakeStore{}, archiveLister{
+		fakeBackups: &fakeBackups{}, at: testNow.Add(-9 * 24 * time.Hour), found: true,
+	})
+	if got := dueFor(t, h, "felhom-pbs"); !got.Due {
+		t.Fatalf("a 9-day-old archive under a 7-day cadence MUST still be due; got %+v", got)
+	}
+}
+
+// A storage that genuinely holds nothing → due. The fix must not invent a backup.
+func TestBackupDue_ColdStore_NoArchiveIsDue(t *testing.T) {
+	h := listerServer(t, &fakeStore{}, archiveLister{fakeBackups: &fakeBackups{}, found: false})
+	if got := dueFor(t, h, "felhom-pbs"); !got.Due {
+		t.Fatalf("no archive anywhere → due; got %+v", got)
+	}
+}
+
+// A storage-read ERROR must fall back to the in-memory record, NOT be read as "a backup exists".
+// An unreadable storage must never make a tier look freshly backed up.
+func TestBackupDue_StorageReadError_DoesNotFakeFreshness(t *testing.T) {
+	h := listerServer(t, &fakeStore{}, archiveLister{
+		fakeBackups: &fakeBackups{}, at: testNow, found: true, err: errStorageRead,
+	})
+	if got := dueFor(t, h, "felhom-pbs"); !got.Due {
+		t.Fatalf("a storage-read error must not fake freshness — the in-memory record is empty, so DUE; got %+v", got)
+	}
+}
+
+var errStorageRead = errors.New("simulated storage read failure")
+
+// The in-memory record WINS when it is newer than the storage listing — a backup that just finished
+// this process lifetime is more current than a listing that may lag.
+func TestBackupDue_InMemoryRecordWinsWhenNewer(t *testing.T) {
+	st := &fakeStore{}
+	st.RecordBackup(backupAt("felhom-pbs", 8200, time.Hour, true)) // 1h ago, in memory
+	h := listerServer(t, st, archiveLister{
+		fakeBackups: &fakeBackups{}, at: testNow.Add(-9 * 24 * time.Hour), found: true, // stale listing
+	})
+	got := dueFor(t, h, "felhom-pbs")
+	if got.Due {
+		t.Fatalf("the fresher in-memory record must win over a stale listing; got %+v", got)
+	}
+}
+
+// A service WITHOUT the optional lister degrades to the pre-R-84 behaviour, unchanged.
+func TestBackupDue_ServiceWithoutLister_UnchangedBehaviour(t *testing.T) {
+	h := listerServer(t, &fakeStore{}, &fakeBackups{}) // plain BackupService
+	if got := dueFor(t, h, "felhom-pbs"); !got.Due || got.Reason != "no successful backup recorded yet" {
+		t.Fatalf("a plain BackupService must behave exactly as before; got %+v", got)
+	}
+}

internal/localapi/controllersupervisor.go

@@ -0,0 +1,453 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+)
+
+// R-523 — the in-guest controller supervisor.
+//
+// THE OUTAGE THIS EXISTS TO KILL (BIGNIGHT F9, 2026-09-14). `docker kill felhom-controller` left the
+// container `Exited (137)`. Nothing restarted it: Docker never restarts a container whose stop it
+// records as deliberate — measured 2026-09-15 on Docker 29.8.0 for BOTH `unless-stopped` and `always`
+// (evidence-p1fixes-2026-09-15/A1) — and the golden's `felhom-controller-bootstrap.service` is a
+// oneshot (`RemainAfterExit=yes`) that ran once at boot and watches nothing. The household's
+// dashboard answered 502 for 33 minutes until the box was power-cycled.
+//
+// This is doc 03 §4's sentence made real: "Healing a crashed controller is non-destructive by
+// construction … redeploy = restart … inside the existing guest — never a guest destroy." The act
+// is exactly the swap's own restart (`systemctl restart felhom-controller-bootstrap.service`, which
+// does `docker rm -f` + `docker run` from the baked image and the guest's persistent volume), over
+// the same GuestExecutor and the same two sudoers grants (`docker inspect -f *`, the unit restart).
+// No new privilege.
+//
+// THE GUARDS, each because doing the act at the wrong moment is worse than not doing it:
+//   - not during a swap (the swap stops the controller ON PURPOSE and owns its own rollback);
+//   - not when the operator parked it (`<guests>/<vmid>/controller-parked` on the HOST);
+//   - not on a guest that is not running, is locked (backup/restore/snapshot/migrate), or has a
+//     vzdump in flight — a stopping or restoring guest is someone else's transaction;
+//   - not on ONE observation: the container must be seen not-running on two consecutive sweeps, so
+//     the bootstrap's own rm-f/run window (boot, path-unit hot-plug) is never raced;
+//   - no thrash: 3 restarts inside 15 minutes → stop restarting, raise `controller_crashloop`, try
+//     again after 30 minutes.
+//
+// THE EVENTS. The agent has no event channel of its own; its heartbeat IS the channel (the
+// capability/leaf precedent). The per-guest record rides the host report as `controller_supervisor`,
+// and the hub's ControllerSupervisorChecker mints `controller_restarted_by_agent` (info) when a
+// guest's `last_restart_at` moves and `controller_crashloop` (error, operator-only) when
+// `crashloop_since` moves. Timestamps, not counters, so an agent restart (which zeroes the in-memory
+// record) can never read as a new restart.
+
+const (
+	// controllerSupervisorInterval is the sweep cadence. Two not-running observations are required,
+	// so a killed controller is restarted 30–60 s after it died.
+	controllerSupervisorInterval = 30 * time.Second
+	// controllerSupervisorConfirm is how many consecutive not-running observations license a restart.
+	controllerSupervisorConfirm = 2
+	// Backoff: controllerCrashloopMax restarts inside controllerCrashloopWindow → give up for
+	// controllerCrashloopPause.
+	controllerCrashloopMax    = 3
+	controllerCrashloopWindow = 15 * time.Minute
+	controllerCrashloopPause  = 30 * time.Minute
+	// controllerSupervisorHeartbeatEvery: a liveness line every 20 sweeps (10 minutes) — a silent
+	// watchdog is indistinguishable from a dead one (standing rule 3).
+	controllerSupervisorHeartbeatEvery = 20
+
+	// ControllerParkedMarker is the host-side file that parks a guest's controller. The operator
+	// creates it with `touch /var/lib/felhom-agent/guests/<vmid>/controller-parked` and removes it to
+	// unpark. Host-side on purpose: it needs no in-guest exec grant, it survives a guest rebuild of
+	// the controller container, and a customer inside the guest cannot park the supervisor.
+	ControllerParkedMarker = "controller-parked"
+
+	defaultGuestsStateDir = "/var/lib/felhom-agent/guests"
+
+	// R-539 (operator ruling 3 of 2026-09-16) — the SLOW crash loop. The 3-in-15-minutes brake above
+	// cannot see a controller that dies every 20 minutes: no two restarts share its window, so it is
+	// restarted for ever and the only trace is an info event that mails nobody (measured 2026-09-16,
+	// R-531). A second counter over 24 hours raises a WARNING at the fifth restart. It does NOT stop
+	// restarting — the fast brake stays the only brake, unchanged. Every restart the supervisor
+	// performs counts, including one that follows a deliberate operator `docker kill` (measured
+	// 2026-09-15: the supervisor cannot tell a kill from a crash, and a controller that is killed five
+	// times a day is worth a line to the operator either way).
+	controllerSlowCrashloopWindow = 24 * time.Hour
+	controllerSlowCrashloopMax    = 5
+	// controllerSlowCounterFile holds the 24-hour restart times and the last raise, per guest, beside
+	// the parked marker.
+	controllerSlowCounterFile = "controller-restarts-24h.json"
+)
+
+// controllerSupState is one guest's supervisor record. In-memory on purpose (the guest-power
+// precedent): an agent restart forgets a crash-loop pause, which costs at most one more restart
+// attempt, whereas persisting it could carry a stale "give up" across the restart that fixed it.
+type controllerSupState struct {
+	notRunningSeen int
+	restarts       []time.Time // restart times inside the crash-loop window (pruned)
+	restartsTotal  int
+	lastRestartAt  time.Time
+	lastReason     string
+	crashloopSince time.Time // zero = not in a crash-loop pause
+	parked         bool
+
+	// R-539 — the slow counter. PERSISTED, unlike everything above, and the precedent's reason does not
+	// apply to it: persisting the fast record could carry a stale "give up" across the restart that
+	// fixed it, but this record never gives anything up — it only warns. Losing it on an agent restart,
+	// on the other hand, would hide exactly the box it exists for (one whose agent restarts too).
+	restarts24h        []time.Time
+	slowCrashloopSince time.Time // the last raise; kept after it ages out, the hub keys on it MOVING
+}
+
+type controllerSupervisor struct {
+	mu     sync.Mutex
+	guests map[int]*controllerSupState
+	sweeps int
+}
+
+// WatchControllers runs the controller supervisor sweep until ctx is done. No-op when the guest list
+// (staleLock) or the guest executor is not wired.
+func (s *Server) WatchControllers(ctx context.Context) {
+	if s.staleLock == nil || s.guestExec == nil {
+		s.logger.Info("controller-supervisor: not wired (no guest list or no guest executor) — disabled")
+		return
+	}
+	s.logger.Info("controller-supervisor: started", "interval", controllerSupervisorInterval.String(),
+		"confirm_sweeps", controllerSupervisorConfirm, "crashloop_max", controllerCrashloopMax,
+		"crashloop_window", controllerCrashloopWindow.String(),
+		"slow_crashloop_max", controllerSlowCrashloopMax, "slow_crashloop_window", controllerSlowCrashloopWindow.String(),
+		"guests_dir", s.guestsStateDir())
+	t := time.NewTicker(controllerSupervisorInterval)
+	defer t.Stop()
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case <-t.C:
+			s.ControllerSupervisorTick(ctx)
+		}
+	}
+}
+
+func (s *Server) guestsStateDir() string {
+	if s.guestsDir != "" {
+		return s.guestsDir
+	}
+	return defaultGuestsStateDir
+}
+
+// provisionedGuest reports whether the agent provisioned a controller into this guest: the
+// `<guests>/<vmid>/bootstrap` directory exists. The directory itself, not bootstrap.json inside it —
+// the directory is owned by the mapped guest root (0700), so the non-root agent can see the entry but
+// not stat the file within.
+func (s *Server) provisionedGuest(vmid int) bool {
+	fi, err := os.Stat(filepath.Join(s.guestsStateDir(), strconv.Itoa(vmid), "bootstrap"))
+	return err == nil && fi.IsDir()
+}
+
+func (s *Server) controllerParked(vmid int) bool {
+	_, err := os.Stat(filepath.Join(s.guestsStateDir(), strconv.Itoa(vmid), ControllerParkedMarker))
+	return err == nil
+}
+
+func (s *Server) supState(vmid int) *controllerSupState {
+	if s.ctrlSup.guests == nil {
+		s.ctrlSup.guests = map[int]*controllerSupState{}
+	}
+	st := s.ctrlSup.guests[vmid]
+	if st == nil {
+		st = &controllerSupState{}
+		s.loadSlowCounter(vmid, st)
+		s.ctrlSup.guests[vmid] = st
+	}
+	return st
+}
+
+// slowCounterRecord is the on-disk shape of the R-539 counter.
+type slowCounterRecord struct {
+	Restarts           []time.Time `json:"restarts"`
+	SlowCrashloopSince time.Time   `json:"slow_crashloop_since,omitempty"`
+}
+
+func (s *Server) slowCounterPath(vmid int) string {
+	return filepath.Join(s.guestsStateDir(), strconv.Itoa(vmid), controllerSlowCounterFile)
+}
+
+// loadSlowCounter restores the persisted counter into a fresh state. Absent = a clean start; unreadable
+// or corrupt = a clean start with a WARN (a warning counter must never block supervision).
+func (s *Server) loadSlowCounter(vmid int, st *controllerSupState) {
+	b, err := os.ReadFile(s.slowCounterPath(vmid))
+	if err != nil {
+		if !os.IsNotExist(err) {
+			s.logger.Warn("controller-supervisor: slow counter unreadable — starting it from zero", "vmid", vmid, "err", err)
+		}
+		return
+	}
+	var rec slowCounterRecord
+	if err := json.Unmarshal(b, &rec); err != nil {
+		s.logger.Warn("controller-supervisor: slow counter corrupt — starting it from zero", "vmid", vmid, "err", err)
+		return
+	}
+	st.restarts24h = pruneBefore(rec.Restarts, s.clock().Add(-controllerSlowCrashloopWindow))
+	st.slowCrashloopSince = rec.SlowCrashloopSince
+	if len(st.restarts24h) > 0 || !st.slowCrashloopSince.IsZero() {
+		s.logger.Info("controller-supervisor: slow counter restored from disk", "vmid", vmid,
+			"restarts_24h", len(st.restarts24h), "slow_crashloop_since", st.slowCrashloopSince.Format(time.RFC3339))
+	}
+}
+
+// saveSlowCounter writes the counter atomically (tmp + rename, 0600). A failure is logged and the
+// in-memory counter carries on — the next restart retries the write.
+func (s *Server) saveSlowCounter(vmid int, rec slowCounterRecord) {
+	path := s.slowCounterPath(vmid)
+	b, err := json.Marshal(rec)
+	if err == nil {
+		tmp := path + ".tmp"
+		if err = os.WriteFile(tmp, b, 0o600); err == nil {
+			err = os.Rename(tmp, path)
+		}
+	}
+	if err != nil {
+		s.logger.Warn("controller-supervisor: could not persist the slow counter (kept in memory)", "vmid", vmid, "path", path, "err", err)
+	}
+}
+
+// ControllerSupervisorTick performs one sweep. Exported so a test (and a live check) can drive one
+// cycle without waiting on the ticker.
+func (s *Server) ControllerSupervisorTick(ctx context.Context) {
+	if s.staleLock == nil || s.guestExec == nil {
+		return
+	}
+	guests, err := s.staleLock.Guests(ctx)
+	if err != nil {
+		// Ownership unproven ⇒ touch nothing (the guest-power rule).
+		s.logger.Warn("controller-supervisor: guest list unavailable — skipping sweep (ownership unproven)", "err", err)
+		return
+	}
+	var evaluated, down int
+	for _, g := range guests {
+		if ctx.Err() != nil {
+			return
+		}
+		if !s.provisionedGuest(g.VMID) {
+			continue
+		}
+		evaluated++
+		if !s.superviseOneController(ctx, g.VMID, g.Status) {
+			down++
+		}
+	}
+	s.ctrlSup.mu.Lock()
+	s.ctrlSup.sweeps++
+	sweeps := s.ctrlSup.sweeps
+	s.ctrlSup.mu.Unlock()
+	if sweeps%controllerSupervisorHeartbeatEvery == 0 {
+		s.logger.Info("controller-supervisor: alive", "sweeps_since_boot", sweeps,
+			"guests_evaluated", evaluated, "controllers_not_running", down)
+	}
+}
+
+// controllerRunning asks the guest's Docker for the controller's state. Returns (running, known).
+// known=false means the question could not be answered (pct exec failed for a reason other than a
+// missing container) — the caller does nothing on unknown. An ABSENT container is a known "not
+// running": `docker rm` of the controller is the same outage as a kill.
+func (s *Server) controllerRunning(ctx context.Context, vmid int) (running, known bool, status string) {
+	out, err := s.guestExec.GuestExec(ctx, vmid, "docker", "inspect", "-f", "{{.State.Status}}", controllerContainer)
+	if err != nil {
+		msg := strings.ToLower(err.Error() + " " + out)
+		if strings.Contains(msg, "no such object") || strings.Contains(msg, "no such container") {
+			return false, true, "absent"
+		}
+		return false, false, ""
+	}
+	status = strings.TrimSpace(out)
+	// "restarting" is Docker's own restart loop at work — not ours to fight on this sweep.
+	return status == "running" || status == "restarting", true, status
+}
+
+// superviseOneController evaluates one provisioned guest and restarts its controller when every guard
+// allows. Returns false when the controller was observed not running.
+func (s *Server) superviseOneController(ctx context.Context, vmid int, guestStatus string) bool {
+	now := s.clock()
+	if guestStatus != "running" {
+		s.resetNotRunning(vmid)
+		return true // the guest-power watchdog owns a stopped guest; its controller is not "down"
+	}
+
+	running, known, status := s.controllerRunning(ctx, vmid)
+	if !known {
+		s.logger.Debug("controller-supervisor: controller state unknown (guest exec failed) — no action", "vmid", vmid)
+		s.resetNotRunning(vmid)
+		return true
+	}
+	parked := s.controllerParked(vmid)
+	s.ctrlSup.mu.Lock()
+	st := s.supState(vmid)
+	st.parked = parked
+	if running {
+		st.notRunningSeen = 0
+		s.ctrlSup.mu.Unlock()
+		return true
+	}
+	st.notRunningSeen++
+	seen := st.notRunningSeen
+	s.ctrlSup.mu.Unlock()
+
+	if parked {
+		s.logger.Info("controller-supervisor: controller is not running and the guest is PARKED — leaving it",
+			"vmid", vmid, "status", status, "marker", filepath.Join(s.guestsStateDir(), strconv.Itoa(vmid), ControllerParkedMarker))
+		return false
+	}
+	s.swapMu.Lock()
+	swapping := s.swapInFlight[vmid]
+	s.swapMu.Unlock()
+	if swapping {
+		s.logger.Info("controller-supervisor: controller is not running during a controller SWAP — the swap owns it",
+			"vmid", vmid, "status", status)
+		s.resetNotRunning(vmid)
+		return false
+	}
+	if seen < controllerSupervisorConfirm {
+		s.logger.Info("controller-supervisor: controller observed not running — confirming on the next sweep",
+			"vmid", vmid, "status", status, "seen", seen, "of", controllerSupervisorConfirm)
+		return false
+	}
+	lock, _, err := s.staleLock.Lock(ctx, vmid)
+	if err != nil {
+		s.logger.Warn("controller-supervisor: could not read the guest lock — no action (fail-safe)", "vmid", vmid, "err", err)
+		return false
+	}
+	if lock != "" {
+		s.logger.Info("controller-supervisor: guest is LOCKED — another operation owns it, no action", "vmid", vmid, "lock", lock)
+		return false
+	}
+	if busy, berr := s.staleLock.BackupRunning(ctx, vmid); berr != nil || busy {
+		s.logger.Info("controller-supervisor: a vzdump may be in flight for the guest — no action",
+			"vmid", vmid, "backup_running", busy, "err", berr)
+		return false
+	}
+
+	// Backoff.
+	s.ctrlSup.mu.Lock()
+	st = s.supState(vmid)
+	if !st.crashloopSince.IsZero() {
+		if now.Sub(st.crashloopSince) < controllerCrashloopPause {
+			s.ctrlSup.mu.Unlock()
+			s.logger.Warn("controller-supervisor: crash-loop pause in force — not restarting",
+				"vmid", vmid, "since", st.crashloopSince.Format(time.RFC3339), "resume_after", controllerCrashloopPause.String())
+			return false
+		}
+		// Pause over: resume with a clean window. crashloopSince stays as the record of the last
+		// crash-loop (the hub keys on it moving, not on it clearing).
+		st.restarts = nil
+		st.crashloopSince = time.Time{}
+	}
+	st.restarts = pruneBefore(st.restarts, now.Add(-controllerCrashloopWindow))
+	if len(st.restarts) >= controllerCrashloopMax {
+		st.crashloopSince = now
+		n := len(st.restarts)
+		s.ctrlSup.mu.Unlock()
+		s.logger.Error("controller-supervisor: CRASH-LOOP — the controller would not stay up; stopping restarts and raising controller_crashloop",
+			"vmid", vmid, "restarts_in_window", n, "window", controllerCrashloopWindow.String(), "pause", controllerCrashloopPause.String())
+		return false
+	}
+	s.ctrlSup.mu.Unlock()
+
+	reason := "controller container " + status + " on " + strconv.Itoa(controllerSupervisorConfirm) + " consecutive sweeps"
+	s.logger.Warn("controller-supervisor: controller is NOT running — restarting the bootstrap unit",
+		"vmid", vmid, "status", status, "unit", bootstrapUnit)
+	if _, err := s.guestExec.GuestExec(ctx, vmid, "systemctl", "restart", bootstrapUnit); err != nil {
+		s.logger.Error("controller-supervisor: bootstrap restart failed", "vmid", vmid, "err", err)
+		reason += "; restart FAILED: " + err.Error()
+	}
+	s.ctrlSup.mu.Lock()
+	st = s.supState(vmid)
+	st.restarts = append(st.restarts, now)
+	st.restartsTotal++
+	st.lastRestartAt = now
+	st.lastReason = reason
+	st.notRunningSeen = 0
+	// R-539: the slow counter. Raise at most once per 24 hours — the hub mails on the raise MOVING.
+	st.restarts24h = append(pruneBefore(st.restarts24h, now.Add(-controllerSlowCrashloopWindow)), now)
+	n24 := len(st.restarts24h)
+	raised := false
+	if n24 >= controllerSlowCrashloopMax && (st.slowCrashloopSince.IsZero() || now.Sub(st.slowCrashloopSince) >= controllerSlowCrashloopWindow) {
+		st.slowCrashloopSince = now
+		raised = true
+	}
+	rec := slowCounterRecord{Restarts: append([]time.Time(nil), st.restarts24h...), SlowCrashloopSince: st.slowCrashloopSince}
+	s.ctrlSup.mu.Unlock()
+	s.saveSlowCounter(vmid, rec)
+	s.logger.Warn("controller-supervisor: RESTARTED the controller", "vmid", vmid, "reason", reason, "restarts_24h", n24)
+	if raised {
+		s.logger.Warn("controller-supervisor: SLOW CRASH-LOOP — the controller keeps dying; still restarting it, raising controller_slow_crashloop",
+			"vmid", vmid, "restarts_24h", n24, "window", controllerSlowCrashloopWindow.String(), "threshold", controllerSlowCrashloopMax)
+	}
+	return false
+}
+
+func (s *Server) resetNotRunning(vmid int) {
+	s.ctrlSup.mu.Lock()
+	defer s.ctrlSup.mu.Unlock()
+	if st := s.ctrlSup.guests[vmid]; st != nil {
+		st.notRunningSeen = 0
+	}
+}
+
+func (s *Server) clock() time.Time {
+	if s.now != nil {
+		return s.now()
+	}
+	return time.Now().UTC()
+}
+
+func pruneBefore(ts []time.Time, cutoff time.Time) []time.Time {
+	out := ts[:0]
+	for _, t := range ts {
+		if !t.Before(cutoff) {
+			out = append(out, t)
+		}
+	}
+	return out
+}
+
+// ControllerSupervisorStatus is the host-report stanza source (hub.ControllerSupervisorReporter).
+// Nil when the supervisor is not wired, so the stanza is omitted.
+func (s *Server) ControllerSupervisorStatus(_ context.Context) *hub.ControllerSupervisorStatus {
+	if s.staleLock == nil || s.guestExec == nil {
+		return nil
+	}
+	s.ctrlSup.mu.Lock()
+	defer s.ctrlSup.mu.Unlock()
+	out := &hub.ControllerSupervisorStatus{Guests: []hub.ControllerSupervisorGuest{}}
+	for vmid, st := range s.ctrlSup.guests {
+		g := hub.ControllerSupervisorGuest{
+			VMID:          vmid,
+			RestartsTotal: st.restartsTotal,
+			LastReason:    st.lastReason,
+			Parked:        st.parked,
+			Crashloop:     !st.crashloopSince.IsZero(),
+		}
+		if !st.lastRestartAt.IsZero() {
+			g.LastRestartAt = st.lastRestartAt.UTC().Format(time.RFC3339)
+		}
+		if !st.crashloopSince.IsZero() {
+			g.CrashloopSince = st.crashloopSince.UTC().Format(time.RFC3339)
+		}
+		now := s.clock()
+		g.Restarts24h = len(pruneBefore(append([]time.Time(nil), st.restarts24h...), now.Add(-controllerSlowCrashloopWindow)))
+		if !st.slowCrashloopSince.IsZero() {
+			g.SlowCrashloopSince = st.slowCrashloopSince.UTC().Format(time.RFC3339)
+			g.SlowCrashloop = now.Sub(st.slowCrashloopSince) < controllerSlowCrashloopWindow
+		}
+		out.Guests = append(out.Guests, g)
+	}
+	sort.Slice(out.Guests, func(i, j int) bool { return out.Guests[i].VMID < out.Guests[j].VMID })
+	return out
+}

internal/localapi/controllersupervisor_test.go

@@ -0,0 +1,350 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"io"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"strconv"
+	"sync"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+// R-523 — the controller supervisor. The consequence under test is "a dead controller is started
+// again", and each guard is pinned by the case where acting would be wrong.
+
+type supExec struct {
+	mu        sync.Mutex
+	status    map[int]string // docker .State.Status per vmid; "" = container absent
+	inspectErr error         // non-nil = pct exec itself failed (unknown)
+	restarts  map[int]int
+	// onRestart, when set, is the status the container reaches after a restart (a crash-looper
+	// stays "exited").
+	onRestart string
+}
+
+func (f *supExec) GuestExec(_ context.Context, vmid int, args ...string) (string, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	switch {
+	case len(args) >= 5 && args[0] == "docker" && args[1] == "inspect" && args[3] == "{{.State.Status}}":
+		if f.inspectErr != nil {
+			return "", f.inspectErr
+		}
+		st, ok := f.status[vmid]
+		if !ok || st == "" {
+			return "", errors.New("pct exec: exit status 1: Error: No such object: felhom-controller")
+		}
+		return st + "\n", nil
+	case len(args) == 3 && args[0] == "systemctl" && args[1] == "restart" && args[2] == bootstrapUnit:
+		if f.restarts == nil {
+			f.restarts = map[int]int{}
+		}
+		f.restarts[vmid]++
+		if f.onRestart != "" {
+			f.status[vmid] = f.onRestart
+		}
+		return "", nil
+	}
+	return "", errors.New("supExec: unexpected args")
+}
+func (f *supExec) GuestExecStdin(context.Context, int, io.Reader, ...string) (string, error) {
+	return "", errors.New("supExec: no stdin exec expected")
+}
+func (f *supExec) count(vmid int) int {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	return f.restarts[vmid]
+}
+
+type supClock struct{ t time.Time }
+
+func (c *supClock) now() time.Time { return c.t }
+
+func supServer(t *testing.T, ex *supExec, ctl *fakeGuestPowerCtl, provisioned ...int) (*Server, *supClock, string) {
+	t.Helper()
+	dir := t.TempDir()
+	for _, v := range provisioned {
+		if err := os.MkdirAll(filepath.Join(dir, strconv.Itoa(v), "bootstrap"), 0o700); err != nil {
+			t.Fatal(err)
+		}
+	}
+	clk := &supClock{t: time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC)}
+	s := &Server{
+		staleLock:    ctl,
+		guestExec:    ex,
+		guestsDir:    dir,
+		swapInFlight: map[int]bool{},
+		logger:       slog.New(slog.NewTextHandler(discardW{}, nil)),
+		now:          clk.now,
+	}
+	return s, clk, dir
+}
+
+func runningGuest(vmid int) *fakeGuestPowerCtl {
+	return &fakeGuestPowerCtl{
+		guests: []proxmox.Guest{{VMID: vmid, Status: "running"}},
+		locks:  map[int]string{}, onboot: map[int]bool{vmid: true},
+	}
+}
+
+// The consequence: a killed controller IS restarted — on the second consecutive observation, not
+// the first (the bootstrap's own rm-f/run window must never be raced).
+//
+// RED-PROOF: delete the `systemctl restart` GuestExec call in superviseOneController → restarts
+// stays 0 → "the killed controller was NOT restarted — this is R-523".
+func TestControllerSupervisor_KilledControllerIsRestarted(t *testing.T) {
+	ex := &supExec{status: map[int]string{9201: "exited"}, onRestart: "running"}
+	s, _, _ := supServer(t, ex, runningGuest(9201), 9201)
+
+	s.ControllerSupervisorTick(context.Background())
+	if n := ex.count(9201); n != 0 {
+		t.Fatalf("restarted on the FIRST observation (restarts=%d) — must confirm on a second sweep", n)
+	}
+	s.ControllerSupervisorTick(context.Background())
+	if n := ex.count(9201); n != 1 {
+		t.Fatalf("the killed controller was NOT restarted — this is R-523 (restarts=%d)", n)
+	}
+	st := s.ControllerSupervisorStatus(context.Background())
+	if len(st.Guests) != 1 || st.Guests[0].RestartsTotal != 1 || st.Guests[0].LastRestartAt == "" || st.Guests[0].LastReason == "" {
+		t.Fatalf("report stanza did not record the restart: %+v", st.Guests)
+	}
+	// Healthy again → no further restart.
+	s.ControllerSupervisorTick(context.Background())
+	s.ControllerSupervisorTick(context.Background())
+	if n := ex.count(9201); n != 1 {
+		t.Fatalf("a running controller was restarted again (restarts=%d)", n)
+	}
+}
+
+func TestControllerSupervisor_AbsentContainerIsRestarted(t *testing.T) {
+	ex := &supExec{status: map[int]string{}, onRestart: "running"}
+	s, _, _ := supServer(t, ex, runningGuest(9201), 9201)
+	s.ControllerSupervisorTick(context.Background())
+	s.ControllerSupervisorTick(context.Background())
+	if n := ex.count(9201); n != 1 {
+		t.Fatalf("a removed controller container was not restarted (restarts=%d)", n)
+	}
+}
+
+func TestControllerSupervisor_Guards(t *testing.T) {
+	cases := []struct {
+		name  string
+		setup func(s *Server, ex *supExec, ctl *fakeGuestPowerCtl, dir string)
+	}{
+		{"parked", func(s *Server, _ *supExec, _ *fakeGuestPowerCtl, dir string) {
+			if err := os.WriteFile(filepath.Join(dir, "9201", ControllerParkedMarker), nil, 0o600); err != nil {
+				panic(err)
+			}
+		}},
+		{"swap in flight", func(s *Server, _ *supExec, _ *fakeGuestPowerCtl, _ string) { s.swapInFlight[9201] = true }},
+		{"guest locked", func(_ *Server, _ *supExec, ctl *fakeGuestPowerCtl, _ string) { ctl.locks[9201] = "backup" }},
+		{"vzdump running", func(_ *Server, _ *supExec, ctl *fakeGuestPowerCtl, _ string) {
+			ctl.backupRun = map[int]bool{9201: true}
+		}},
+		{"vzdump state unknown", func(_ *Server, _ *supExec, ctl *fakeGuestPowerCtl, _ string) {
+			ctl.backupErr = errors.New("tasks unreadable")
+		}},
+		{"guest not running", func(_ *Server, _ *supExec, ctl *fakeGuestPowerCtl, _ string) {
+			ctl.guests[0].Status = "stopped"
+		}},
+		{"docker state unknown", func(_ *Server, ex *supExec, _ *fakeGuestPowerCtl, _ string) {
+			ex.inspectErr = errors.New("pct exec 9201: exit status 255: container not running")
+		}},
+		{"guest list unavailable", func(_ *Server, _ *supExec, ctl *fakeGuestPowerCtl, _ string) {
+			ctl.guestsErr = errors.New("api down")
+		}},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			ex := &supExec{status: map[int]string{9201: "exited"}, onRestart: "running"}
+			ctl := runningGuest(9201)
+			s, _, dir := supServer(t, ex, ctl, 9201)
+			tc.setup(s, ex, ctl, dir)
+			for i := 0; i < 4; i++ {
+				s.ControllerSupervisorTick(context.Background())
+			}
+			if n := ex.count(9201); n != 0 {
+				t.Fatalf("guard %q did not hold: controller restarted %d time(s)", tc.name, n)
+			}
+		})
+	}
+}
+
+// A guest the agent did not provision (no <guests>/<vmid>/bootstrap) is never touched.
+func TestControllerSupervisor_UnprovisionedGuestIgnored(t *testing.T) {
+	ex := &supExec{status: map[int]string{9202: "exited"}}
+	s, _, _ := supServer(t, ex, runningGuest(9202) /* nothing provisioned */)
+	for i := 0; i < 3; i++ {
+		s.ControllerSupervisorTick(context.Background())
+	}
+	if n := ex.count(9202); n != 0 {
+		t.Fatalf("an unprovisioned guest's container was restarted (%d)", n)
+	}
+}
+
+// No thrash: a controller that will not stay up is restarted at most controllerCrashloopMax times
+// inside the window, then the supervisor raises the crash-loop and pauses; after the pause it tries
+// again.
+//
+// RED-PROOF (run 2026-09-15, recorded in REPORT.md): with the `len(st.restarts) >=
+// controllerCrashloopMax` block removed, restarts reached 10 in the first 20 sweeps and the test
+// failed at "crash-looping controller restarted 10 times".
+func TestControllerSupervisor_CrashloopBackoff(t *testing.T) {
+	ex := &supExec{status: map[int]string{9201: "exited"}, onRestart: "exited"}
+	s, clk, _ := supServer(t, ex, runningGuest(9201), 9201)
+	ctx := context.Background()
+
+	for i := 0; i < 20; i++ { // 10 minutes of 30 s sweeps
+		s.ControllerSupervisorTick(ctx)
+		clk.t = clk.t.Add(controllerSupervisorInterval)
+	}
+	if n := ex.count(9201); n != controllerCrashloopMax {
+		t.Fatalf("crash-looping controller restarted %d times in 10 minutes — want exactly %d then a pause", n, controllerCrashloopMax)
+	}
+	st := s.ControllerSupervisorStatus(ctx).Guests[0]
+	if !st.Crashloop || st.CrashloopSince == "" {
+		t.Fatalf("crash-loop not raised in the report stanza: %+v", st)
+	}
+	// Still paused 25 minutes later.
+	clk.t = clk.t.Add(15 * time.Minute)
+	s.ControllerSupervisorTick(ctx)
+	s.ControllerSupervisorTick(ctx)
+	if n := ex.count(9201); n != controllerCrashloopMax {
+		t.Fatalf("restarted during the crash-loop pause (restarts=%d)", n)
+	}
+	// After the pause: tries again.
+	clk.t = clk.t.Add(controllerCrashloopPause)
+	s.ControllerSupervisorTick(ctx)
+	s.ControllerSupervisorTick(ctx)
+	if n := ex.count(9201); n != controllerCrashloopMax+1 {
+		t.Fatalf("did not resume after the pause (restarts=%d, want %d)", n, controllerCrashloopMax+1)
+	}
+	if since := s.ControllerSupervisorStatus(ctx).Guests[0].CrashloopSince; since != st.CrashloopSince && since != "" {
+		t.Fatalf("crashloop_since changed without a new crash-loop: %q → %q", st.CrashloopSince, since)
+	}
+}
+
+// The wire shape the hub parses. The hub's controller_supervisor_test.go carries this exact JSON.
+func TestControllerSupervisorStanza_WireShape(t *testing.T) {
+	ex := &supExec{status: map[int]string{9201: "exited"}, onRestart: "running"}
+	s, _, _ := supServer(t, ex, runningGuest(9201), 9201)
+	s.ControllerSupervisorTick(context.Background())
+	s.ControllerSupervisorTick(context.Background())
+	b, _ := json.Marshal(s.ControllerSupervisorStatus(context.Background()))
+	var m map[string][]map[string]any
+	if err := json.Unmarshal(b, &m); err != nil {
+		t.Fatal(err)
+	}
+	g := m["guests"][0]
+	for _, k := range []string{"vmid", "restarts_total", "last_restart_at", "last_reason", "crashloop", "parked", "restarts_24h", "slow_crashloop"} {
+		if _, ok := g[k]; !ok {
+			t.Fatalf("stanza lacks %q — the hub keys on it: %s", k, b)
+		}
+	}
+}
+
+
+// ---- R-539 (operator ruling 3 of 2026-09-16): the SLOW crash loop ----------------------------------
+
+// supKillOnce kills the controller and lets the supervisor restart it (two confirming sweeps), then
+// moves the clock on by gap. The container comes back "running", so each restart is a separate act.
+func supKillOnce(t *testing.T, s *Server, ex *supExec, clk *supClock, gap time.Duration) {
+	t.Helper()
+	before := ex.count(9201)
+	ex.mu.Lock()
+	ex.status[9201] = "exited"
+	ex.mu.Unlock()
+	s.ControllerSupervisorTick(context.Background())
+	clk.t = clk.t.Add(controllerSupervisorInterval)
+	s.ControllerSupervisorTick(context.Background())
+	if ex.count(9201) != before+1 {
+		t.Fatalf("kill was not followed by exactly one restart (restarts %d → %d)", before, ex.count(9201))
+	}
+	clk.t = clk.t.Add(gap)
+}
+
+// The consequence: a controller that dies every 20 minutes — never three times inside the 15-minute
+// brake — raises slow_crashloop on the FIFTH restart in 24 hours, and the raise does not move again on
+// the sixth (the hub mails on movement; once per 24 hours is the ruling).
+//
+// RED-PROOF: without the slow counter the stanza never sets slow_crashloop → "five restarts 20 minutes
+// apart did not raise slow_crashloop — this is R-539".
+func TestControllerSupervisor_SlowCrashloop(t *testing.T) {
+	ex := &supExec{status: map[int]string{9201: "running"}, onRestart: "running"}
+	s, clk, _ := supServer(t, ex, runningGuest(9201), 9201)
+	ctx := context.Background()
+
+	for i := 1; i <= 4; i++ {
+		supKillOnce(t, s, ex, clk, 20*time.Minute)
+	}
+	g := s.ControllerSupervisorStatus(ctx).Guests[0]
+	if g.Crashloop {
+		t.Fatalf("the 15-minute brake fired on restarts 20 minutes apart — the fixture is wrong: %+v", g)
+	}
+	if g.SlowCrashloop || g.SlowCrashloopSince != "" {
+		t.Fatalf("slow_crashloop raised after only 4 restarts: %+v", g)
+	}
+	supKillOnce(t, s, ex, clk, 20*time.Minute)
+	g = s.ControllerSupervisorStatus(ctx).Guests[0]
+	if !g.SlowCrashloop || g.SlowCrashloopSince == "" || g.Restarts24h != 5 {
+		t.Fatalf("five restarts 20 minutes apart did not raise slow_crashloop — this is R-539: %+v", g)
+	}
+	first := g.SlowCrashloopSince
+	supKillOnce(t, s, ex, clk, 20*time.Minute)
+	g = s.ControllerSupervisorStatus(ctx).Guests[0]
+	if g.SlowCrashloopSince != first {
+		t.Fatalf("the raise moved again on the 6th restart (%q → %q) — the operator would be mailed per restart", first, g.SlowCrashloopSince)
+	}
+	if !g.SlowCrashloop {
+		t.Fatalf("slow_crashloop cleared while the loop continues: %+v", g)
+	}
+}
+
+// The negative control: restarts that never reach five inside any 24 hours never raise it.
+func TestControllerSupervisor_SpreadRestartsNeverSlowCrashloop(t *testing.T) {
+	ex := &supExec{status: map[int]string{9201: "running"}, onRestart: "running"}
+	s, clk, _ := supServer(t, ex, runningGuest(9201), 9201)
+	for i := 0; i < 8; i++ { // eight restarts, 7 hours apart: at most 4 inside any 24 hours
+		supKillOnce(t, s, ex, clk, 7*time.Hour)
+	}
+	g := s.ControllerSupervisorStatus(context.Background()).Guests[0]
+	if g.SlowCrashloop || g.SlowCrashloopSince != "" {
+		t.Fatalf("restarts 7 hours apart raised slow_crashloop: %+v", g)
+	}
+	if g.Restarts24h > 4 {
+		t.Fatalf("restarts_24h=%d — the 24-hour window is not pruning", g.Restarts24h)
+	}
+}
+
+// An agent restart must not reset the slow counter (the ruling; a box whose AGENT also restarts would
+// otherwise never reach five). The same state directory, a fresh Server.
+//
+// RED-PROOF: keep the counter in memory only → the second Server starts at 0 → "the agent restart
+// reset the slow counter".
+func TestControllerSupervisor_SlowCounterSurvivesAgentRestart(t *testing.T) {
+	ex := &supExec{status: map[int]string{9201: "running"}, onRestart: "running"}
+	s, clk, dir := supServer(t, ex, runningGuest(9201), 9201)
+	for i := 0; i < 4; i++ {
+		supKillOnce(t, s, ex, clk, 20*time.Minute)
+	}
+	s2 := &Server{
+		staleLock:    s.staleLock,
+		guestExec:    ex,
+		guestsDir:    dir,
+		swapInFlight: map[int]bool{},
+		logger:       s.logger,
+		now:          clk.now,
+	}
+	supKillOnce(t, s2, ex, clk, 20*time.Minute)
+	g := s2.ControllerSupervisorStatus(context.Background()).Guests[0]
+	if g.Restarts24h != 5 || !g.SlowCrashloop {
+		t.Fatalf("the agent restart reset the slow counter: %+v", g)
+	}
+}

internal/localapi/disks.go

@@ -8,6 +8,7 @@ import (
 	"strings"
 	"time"
 
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
 	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
 	"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
 )
@@ -78,8 +79,10 @@ type GuestAttacher interface {
 	DetachDrive(ctx context.Context, where string) error
 	// EnsureSharedParent makes the host stable parent shared + installs the boot-persistence unit.
 	EnsureSharedParent(ctx context.Context) error
-	// GuestSeesMount reports whether vmid's guest sees `path` as a mount in its own namespace (the
-	// guest-usable signal — distinct from the host having the bind). Backs BoundUnderParent.
+	// GuestSeesMount reports whether vmid's guest sees `path` as a mount in its own namespace — the
+	// guest-VISIBILITY signal, distinct from the host having the bind. One of the three terms behind
+	// BoundUnderParent. It is a path-presence test and NOT a liveness signal (R-117): a stale bind over a
+	// dead device is still "seen". Liveness is bindLiveness's job.
 	GuestSeesMount(ctx context.Context, vmid int, path string) bool
 	// GuestBootID returns a token that changes on every guest boot (host or guest) but is stable across a
 	// controller-only restart — the deterministic guest-reboot signal the controller recreates apps on.
@@ -142,15 +145,49 @@ type DiskInfo struct {
 	// HDD look available when it wasn't bound. Only meaningful for user-data drives. LEGACY (per-drive
 	// `pct set -mpN` model) — the intermediary model uses BoundUnderParent.
 	GuestAttached bool `json:"guest_attached"`
+	// BackupTarget (E-2) reports that this drive backs the PRIMARY whole-guest backup tier. Additive:
+	// an older controller ignores it. It is the agent's answer, not the controller's intent flag —
+	// on a hand-migrated box (E-1) intent is unset while the drive really is the target.
+	BackupTarget bool `json:"backup_target,omitempty"`
 	// GuestPath is the drive's STABLE in-guest path in the intermediary-mount model
 	// (/mnt/felhom-drives/<name>). This is what the controller repoints HDD_PATH to and registers as the
 	// storage path. Set for /mnt/<name> drives; "" otherwise. Distinct from MountPath (the RAW host PVE
 	// mount the agent ops on).
 	GuestPath string `json:"guest_path,omitempty"`
-	// BoundUnderParent reports whether the drive's felhom-data is currently bound under the shared parent
-	// at GuestPath (a host mount-table check) — i.e. live + usable in the guest in the intermediary model.
+	// BoundUnderParent reports whether the drive is live + usable in the guest in the intermediary model.
 	// The controller's drive-absent gate + auto-restart key on this (and State).
+	//
+	// It is a CONJUNCTION of THREE facts, and all three are load-bearing:
+	//   1. felhom-data is bound under the shared parent at GuestPath (the guest-visible mount check), and
+	//   2. the drive's RAW host mount is still mounted — i.e. the DEVICE is still there (R-113), and
+	//   3. the bind actually WORKS: it names the same device as the raw mount, and that filesystem has
+	//      not aborted (R-117, v0.117.0 — see bindLiveness).
+	// Half 1 alone was the bug R-113 fixed: the raw mount is device-bound and dies with its device, but
+	// the agent's own bind is not, so half 1 stays true over a stale shell after the device is pulled. The
+	// controller read that survivor as "present" and the drive-absent alarm could never fire — measured
+	// live in E-2d (felhom.eu audits/E2D-fresh-vm-2026-07-29.md §5.2). Half 2 alone would regress boot
+	// ordering, where the raw mounts early and the bind lands ~18s later; the conjunction keeps that
+	// window reading absent.
+	//
+	// Terms 1 and 2 TOGETHER were still not liveness, which is R-117: both are path-presence tests
+	// comparing only field 5 of a mountinfo line, so both stay true over a bind that names the drive that
+	// went away while the raw mount healed onto the returning one via its fs-UUID-keyed unit. Measured
+	// live: raw on 8:32 /dev/sdc, bind on 8:16 /dev/sdb with `shutdown`, this field TRUE, EIO on every
+	// read and write, and the gate restarting the customer's apps onto it with no alarm on any channel
+	// (felhom.eu audits/SPIKE-r117-bind-liveness-2026-07-30.md §5.2).
+	//
+	// THE TESTS THAT PIN THIS COMMENT, because for three releases it promised a property nothing tested
+	// (spike §5.3): disks_bind_liveness_test.go — TestDisks_BindLiveness_StaleBindReadsAbsent (term 3,
+	// case a), _AbortedFilesystemReadsAbsent (term 3, case b, the steady-state case that emits nothing
+	// today), _UnknownIsTreatedAsPresent (the cannot-tell rule) and _HealthyReadsPresent (no false
+	// negative). Each asserts the CONSEQUENCE — what this field reads — not the mechanism.
 	BoundUnderParent bool `json:"bound_under_parent"`
+	// Smart is the already-computed per-disk SMART health summary (v0.94.0), serialized here so the
+	// controller can render a disk-health card + degradation alert WITHOUT any new smartctl load — the
+	// value is copied straight from the target's Observe-time enrichment. omitempty + a pointer so a
+	// device that exposes no SMART (USB bridge, unread) is ABSENT, not a misleading zero-value UNKNOWN;
+	// the controller feature-detects presence and renders "Nincs adat" when nil (never alarms).
+	Smart *hub.SmartSummary `json:"smart,omitempty"`
 }
 
 // handleDisks lists the host's drives + data-bearing flags (read-only/benign).
@@ -170,6 +207,9 @@ func (s *Server) handleDisks(w http.ResponseWriter, r *http.Request, vmid int) {
 	// F9: which host mount paths are actually BOUND into THIS guest's config (guest-usable, not just
 	// host-present). A bind's mp= equals the guest path, which is the drive's host mount path (`where`).
 	boundPaths := s.guestBoundPaths(r.Context(), vmid)
+	// E-2: the PRIMARY tier's storage id — the whole-guest vzdump destination. "" when no tier
+	// carries a target (the legacy single-tier shape), which correctly flags nothing.
+	primaryTargetID := s.primaryTier().TargetID
 	out := make([]DiskInfo, 0, len(targets))
 	for _, t := range targets {
 		di := DiskInfo{
@@ -181,13 +221,79 @@ func (s *Server) handleDisks(w http.ResponseWriter, r *http.Request, vmid int) {
 			UsedBytes:     t.UsedBytes,
 			UsedFraction:  t.UsedFraction,
 			GuestAttached: t.MountPath != "" && boundPaths[t.MountPath],
+			// E-2: is THIS drive the whole-guest backup target? The agent is the only component that
+			// can answer — the controller's own StoragePath.BackupTarget is customer INTENT, and on a
+			// box migrated by hand (E-1) nobody ever assigned it, so intent is empty while the drive
+			// really is the target. Reported here so the controller can name the drive in an
+			// absent-target alarm and hide the destructive controls the agent would refuse anyway.
+			BackupTarget: t.Name == primaryTargetID,
 		}
 		// Intermediary model: the stable in-guest path + whether felhom-data is bound under the parent.
 		// Only user-data /mnt/<name> drives have a guest path (system/backup mounts never cross in).
 		if di.Role == string(storage.RoleUserData) {
 			if gp := StablePathForRaw(t.MountPath); gp != "" {
 				di.GuestPath = gp
-				di.BoundUnderParent = s.boundUnderParent(r.Context(), vmid, gp)
+				// R-113: AND in device presence. A conjunction, deliberately — it leaves the
+				// boot-ordering behaviour the controller's gate depends on exactly as it was
+				// (raw mounted early, bind not yet ⇒ still absent) while closing the case the
+				// gate could never see (bind outlived the device ⇒ now absent).
+				// R-117: AND in bind LIVENESS. The two terms above are both path-presence tests, so
+				// both stay true over a bind that names the drive that went away while the raw mount
+				// healed onto the returning one — EIO on every call, payload healthy.
+				di.BoundUnderParent = s.boundUnderParent(r.Context(), vmid, gp) &&
+					s.devicePresent(t.MountPath) &&
+					s.bindUsable(gp, t.MountPath)
+			}
+		}
+		// R-116: carry the GUEST PATH on the backup-target row even when its role has flipped to
+		// system — but ONLY when that flip was caused by the device vanishing.
+		//
+		// WHY. The controller keys the drive-absent alarm on the registered StoragePath, which for an
+		// external drive is the GUEST path. When the device goes, Observe's exactMountDevice fails, so
+		// t.BackingDevice becomes "" and RoleForStorage returns RoleSystem (role.go:180-181) — the block
+		// above is skipped and this row loses its guest path. It keeps its MountPath, so the union loop
+		// below DEDUPES the registry row away (`seen[d.MountPath]`), and /disks ends up carrying NO row
+		// with that guest path at all. driveTargetByPath then has no entry, isTarget[guestPath] is a
+		// missing key, and the specific backup_target_absent alarm cannot fire — the generic one goes
+		// out instead, while the RETURN (rows rejoined) fires the specific recovery. An unmatchable
+		// pair. Measured live: felhom.eu audits/SESSION-C-2026-07-29.md §5.
+		//
+		// THE GATES, each load-bearing:
+		//   di.GuestPath == ""   — never touch the user-data path above; this is a fallback, not a rule.
+		//   di.BackupTarget      — only the target row. No other system/backup mount gains a guest path,
+		//                          so the boundary at :213-214 stands: this is not "system mounts now
+		//                          cross into the guest", it is "the drive the alarm is about keeps its
+		//                          identity while it is missing".
+		//   t.BackingDevice == "" — ONLY the vanished-device flip. A storage that is RoleSystem because
+		//                          it is genuinely system-BACKED has a non-empty BackingDevice and is
+		//                          excluded. Without this gate a dir storage at /mnt/<name> living on the
+		//                          root disk would acquire a guest path.
+		//
+		// Case B (the COMMON fresh-box shape) is safe twice over: the target is the builtin `local` on
+		// /var/lib/vz, and StablePathForRaw returns "" for anything that is not exactly /mnt/<name>
+		// (DriveNameFromRaw, intermediary.go:79-88), so nothing is set even before the gates apply.
+		//
+		// This cannot make the gate read an absent drive as PRESENT: BoundUnderParent is assigned only
+		// inside the two guest-path blocks a system-role row never enters, so it stays false, and
+		// planDriveGates computes present[gp] = present[gp] || d.BoundUnderParent. Inert by construction
+		// — pinned by TestAbsentTargetRowDoesNotRegisterPresence.
+		//
+		// v0.116.0 — WHY v0.115.0 (the MountPath-only form) WAS INERT, measured not reasoned. In the
+		// absent state t.MountPath is ALSO "" — the same exactMount failure that emptied BackingDevice
+		// empties it — so StablePathForRaw("") returned "" and this assigned nothing. Captured payload:
+		// felhom.eu audits/DIAG-r116-disks-payload-2026-07-30.md §6.2.
+		//
+		// t.ConfigPath is the fix: the storage's CONFIGURED path from storage.cfg, which is configuration
+		// and therefore survives the device. MountPath is tried FIRST so the present-state path and
+		// v0.115.0's tested behaviour are byte-identical; ConfigPath is consulted only when the mount is
+		// genuinely gone. MountPath is deliberately NOT back-filled from ConfigPath — see the union-dedup
+		// note below for the consumer that would break, and because a path that is not mounted is not a
+		// "host mountpoint" (this field's own contract, :152-153).
+		if di.GuestPath == "" && di.BackupTarget && t.BackingDevice == "" {
+			if gp := StablePathForRaw(t.MountPath); gp != "" {
+				di.GuestPath = gp
+			} else {
+				di.GuestPath = StablePathForRaw(t.ConfigPath)
 			}
 		}
 		// Inspect the backing device for the UI's data-bearing hint (the authoritative check
@@ -206,6 +312,13 @@ func (s *Server) handleDisks(w http.ResponseWriter, r *http.Request, vmid int) {
 				di.WipeDurableID = wid
 			}
 		}
+		// v0.94.0: surface the already-computed SMART only when it was actually read (Health set).
+		// A zero-value summary (enrich skipped / no smartctl device) has Health "" → stays omitted, so
+		// the controller sees "absent" and renders "Nincs adat" rather than a false UNKNOWN.
+		if t.Smart.Health != "" {
+			sm := t.Smart
+			di.Smart = &sm
+		}
 		out = append(out, di)
 	}
 	// Impl-2a: union in registry+units drives that Observe() does NOT surface (a drive with no PVE
@@ -213,16 +326,43 @@ func (s *Server) handleDisks(w http.ResponseWriter, r *http.Request, vmid int) {
 	// NOT duplicated, and no Observe row is dropped (so this can never regress the current view).
 	if s.driveTargets != nil {
 		seen := make(map[string]bool, len(out))
+		// R-116: dedup ALSO by guest path. `seen` keys on MountPath, the one field the absent state
+		// empties, so with the device gone /mnt/<name> is absent from `seen` and the registry row was NOT
+		// skipped — /disks carried the drive TWICE, the Observe row holding BackupTarget with no key and
+		// the registry row holding both keys with BackupTarget defaulted false. driveTargetByPath
+		// (controller intermediary.go:602-618) assigns rather than ORs, and the registry row is appended
+		// LAST, so its false won on both keys. Measured, 4 rows vs 3:
+		// felhom.eu audits/DIAG-r116-disks-payload-2026-07-30.md §6.2.
+		//
+		// THE JOIN, and it is the whole point: with the device gone the two records of one drive share NO
+		// runtime field — no mount, no backing device, and the Observe row's DurableID has degraded off the
+		// fs-UUID. What they DO share is CONFIGURATION: the Observe row's storage path (storage.cfg) and the
+		// registry row's unit `Where` (the .mount unit) are the same path, so both derive the same stable
+		// guest path. That is the key both sides can still compute, which is why the dedup keys on it.
+		seenGuest := make(map[string]bool, len(out))
 		for _, d := range out {
 			if d.MountPath != "" {
 				seen[d.MountPath] = true
 			}
+			if d.GuestPath != "" {
+				seenGuest[d.GuestPath] = true
+			}
 		}
 		if drives, derr := s.driveTargets.Known(r.Context()); derr == nil {
 			for _, d := range drives {
 				if d.MountPath == "" || seen[d.MountPath] {
 					continue
 				}
+				// Same drive as an Observe row that already carries this guest path — skip it. Suppressing
+				// it rather than teaching it BackupTarget is deliberate: the registry row has a non-empty
+				// MountPath (from the unit file, stale by then), and the controller reads
+				// `d.BackupTarget && d.MountPath != ""` as "a real drive with its own mountpoint — HEALTHY"
+				// (backup_target_offer.go:79). Putting the flag on a row with a stale MountPath would have
+				// silently regressed R-114, telling the customer the backup target is fine while its drive
+				// is missing. Pinned by TestAbsentTargetKeepsR114DegradedSignal.
+				if gp := StablePathForRaw(d.MountPath); gp != "" && seenGuest[gp] {
+					continue
+				}
 				di := DiskInfo{
 					Name: d.Name, Type: d.Type, State: "attached",
 					MountPath: d.MountPath, DurableID: d.DurableID,
@@ -233,7 +373,13 @@ func (s *Server) handleDisks(w http.ResponseWriter, r *http.Request, vmid int) {
 				// exactly like the Observe path — else the controller reads a registry drive as "Leválasztva".
 				if gp := StablePathForRaw(d.MountPath); gp != "" {
 					di.GuestPath = gp
-					di.BoundUnderParent = s.boundUnderParent(r.Context(), vmid, gp)
+					// R-113 + R-117: same three-term conjunction as the Observe path. This path matters
+					// MORE, not less — a registry drive with no PVE dir-storage is exactly the shape
+					// E-2d detached, and its State is hardcoded "attached" below, so these checks are
+					// the only device truth this row carries.
+					di.BoundUnderParent = s.boundUnderParent(r.Context(), vmid, gp) &&
+						s.devicePresent(d.MountPath) &&
+						s.bindUsable(gp, d.MountPath)
 				}
 				// A registry drive has no PVE `pvesm status` snapshot, so fill backing device + capacity
 				// from the host directly: resolve the device by fs-UUID, and statfs the mount for size —
@@ -241,10 +387,17 @@ func (s *Server) handleDisks(w http.ResponseWriter, r *http.Request, vmid int) {
 				if d.UUID != "" {
 					// Resolve to the real /dev node (e.g. /dev/sdd), not the by-uuid symlink path, to match
 					// how Observe-sourced rows display the backing device.
-					if dev, err := storage.ResolveStorageDevice("uuid:" + d.UUID); err == nil {
+					if dev, err := s.resolveStorageDevice("uuid:" + d.UUID); err == nil {
 						di.BackingDevice = dev
 					}
 				}
+				// Fix B (v0.95.0): union-path drives skip Observe's enrich, so read SMART here through the
+				// same seam the dir targets use. Only set when the read actually ran (Health != "").
+				if di.BackingDevice != "" && s.smart != nil {
+					if sm := s.smart.SMARTForBacking(r.Context(), di.BackingDevice); sm.Health != "" {
+						di.Smart = &sm
+					}
+				}
 				if total, used, okc := statfsCapacity(d.MountPath); okc {
 					di.TotalBytes, di.UsedBytes = total, used
 					di.UsedFraction = float64(used) / float64(total)
@@ -357,6 +510,11 @@ func (s *Server) handleDiskEject(w http.ResponseWriter, r *http.Request, vmid in
 		writeErr(w, http.StatusForbidden, "mount is system/backup-protected — eject refused (role: "+string(role)+")")
 		return
 	}
+	// E-2c: the role gate above passes a drive that is BOTH user-data and the vzdump target (E-1 put
+	// the target on the enrolled drive's own mountpoint). Refuse specifically, naming the remedy.
+	if s.refuseIfBackupTarget(r.Context(), w, "eject", vmid, req.Where) {
+		return
+	}
 	dependents := s.dependentGuests(r.Context(), req.Where)
 	// Record the EJECT intent BEFORE unmounting (the target still resolves to its durable-id) so the
 	// self-heal watchdog leaves it alone — an OFFICIAL eject is the only thing that sets this (P3); an
@@ -410,6 +568,11 @@ func (s *Server) handleDiskDecommission(w http.ResponseWriter, r *http.Request,
 		writeErr(w, http.StatusForbidden, "mount is system/backup-protected — decommission refused (role: "+string(role)+")")
 		return
 	}
+	// E-2c: same narrow gate as eject — decommission migrates data off and retires the drive, which
+	// would strand the backup target just as thoroughly.
+	if s.refuseIfBackupTarget(r.Context(), w, "decommission", vmid, req.Where) {
+		return
+	}
 	dependents := s.dependentGuests(r.Context(), req.Where)
 	// Resolve the durable-id BEFORE unmounting (it still resolves while mounted) for the bind prune.
 	id := s.durableIDForMount(r.Context(), req.Where)
@@ -803,9 +966,12 @@ func (s *Server) handleDiskFormat(w http.ResponseWriter, r *http.Request, vmid i
 }
 
 // boundUnderParent reports whether a drive's felhom-data is bound at its stable guest path AND visible
-// inside the guest (the usable-in-guest signal the controller's gate keys on — a guest reboot leaves the
-// host bind in place but invisible to the guest until re-propagated). Injectable via s.boundCheck for
-// tests; defaults to the guest-namespace mount check.
+// inside the guest — a guest reboot leaves the host bind in place but invisible to the guest until
+// re-propagated. Injectable via s.boundCheck for tests; defaults to the guest-namespace mount check.
+//
+// This is VISIBILITY, not liveness (R-117). It compares only the mount point, so it stays true over a
+// bind whose device has gone; do not read it as "usable in the guest" — that is the whole three-term
+// conjunction at the two /disks construction sites, whose third term is bindUsable.
 func (s *Server) boundUnderParent(ctx context.Context, vmid int, stablePath string) bool {
 	if s.boundCheck != nil {
 		return s.boundCheck(stablePath)
@@ -816,6 +982,44 @@ func (s *Server) boundUnderParent(ctx context.Context, vmid int, stablePath stri
 	return s.guestAttach.GuestSeesMount(ctx, vmid, stablePath)
 }
 
+// devicePresent reports whether the drive's BACKING DEVICE is still there, by asking whether its RAW
+// host mount is still a mountpoint (R-113).
+//
+// WHY THE RAW MOUNT AND NOT THE BIND. The raw mount at /mnt/<name> is a systemd mount unit bound to
+// its device: when the device goes, the unit stops and the mountpoint disappears. The agent's own bind
+// of <raw>/felhom-data under the shared parent is an ordinary bind — nothing ties it to the device, so
+// its mountinfo entry OUTLIVES the device as a stale shell. Measured live in E-2d with the device
+// pulled: `/mnt/mentes2` NOT mounted while `/mnt/felhom-drives/mentes2` still read
+// `/dev/sdb[/felhom-data]` (felhom.eu audits/E2D-fresh-vm-2026-07-29.md §5.2). Keying presence on the
+// survivor is exactly why the controller's drive-absent gate could never fire.
+//
+// An empty raw path means we have nothing to ask about — return TRUE (unknown), never false. Absent
+// stops a customer's apps, so "cannot tell" must never be reported as "gone".
+func (s *Server) devicePresent(rawMountPath string) bool {
+	if rawMountPath == "" {
+		return true // cannot tell → never claim absent
+	}
+	if s.deviceCheck != nil {
+		return s.deviceCheck(rawMountPath)
+	}
+	return isHostMountpoint(rawMountPath)
+}
+
+// bindUsable is the THIRD term of the BoundUnderParent conjunction (R-117): the bind must not only exist
+// and be guest-visible, it must actually WORK. The first two terms are path-presence tests and are both
+// satisfied by a bind that names the drive that went away — measured live, with EIO on every read and
+// write while the payload read healthy and the gate restarted the customer's apps onto it.
+//
+// UNKNOWN counts as usable, via BindLiveness.Usable — the same "cannot tell → never absent" rule
+// devicePresent applies above, and for the same reason: a false absent stops a working customer's apps.
+// Injectable via s.livenessCheck; the default reads /proc only and issues NO block I/O (CLAUDE.md).
+func (s *Server) bindUsable(stable, rawMountPath string) bool {
+	if s.livenessCheck != nil {
+		return s.livenessCheck(stable, rawMountPath).Usable()
+	}
+	return bindLiveness(stable, rawMountPath).Usable()
+}
+
 // guestBoundPaths returns the set of guest mountpoint paths (the `mp=` of each entry in the guest's
 // config) — i.e. the host drives actually BOUND into the guest. F9: this is the guest-attached signal
 // (`GuestAttached`) that distinguishes a guest-usable drive from one merely present on the host. A bind
@@ -1020,6 +1224,62 @@ func (s *Server) hostReader() storage.HostReader {
 	return storage.NewProcHostReader()
 }
 
+// backupTargetAt reports the configured backup TIER whose storage is mounted at `where`, or "" when
+// none is. E-2c.
+//
+// WHY THIS IS NOT A ROLE RECLASSIFICATION. The obvious fix is to make RoleForStorage return
+// RoleBackup for the target's storage, and it is wrong here: on both demo boxes the drive that now
+// holds the whole-guest archives is ALSO the enrolled user-data drive (E-1 put the vzdump target on
+// the drive's own mountpoint, beside felhom-data). Reclassifying it would refuse every legitimate
+// eject/decommission of the customer's own data drive — an over-correction that trades one silent
+// failure for a permanent obstruction. So this is a SEPARATE, narrower gate that names exactly what
+// it protects and leaves the role vocabulary alone.
+//
+// It resolves through the agent's OWN storage view (never the caller's claim) and fails OPEN — an
+// unreadable view returns "" so this gate cannot block on a transient error. That is safe because it
+// sits BEHIND the role gate, which already fails SAFE on the same error: an unresolvable mount is
+// refused there before it ever reaches this check.
+func (s *Server) backupTargetAt(ctx context.Context, where string) string {
+	if where == "" || s.storage == nil {
+		return ""
+	}
+	targets, err := s.storage.Observe(ctx)
+	if err != nil {
+		return "" // fail OPEN — the role gate already fails SAFE on this same error
+	}
+	for _, t := range s.tiers {
+		if t.TargetID == "" {
+			continue
+		}
+		for _, tgt := range targets {
+			if tgt.Name == t.TargetID && tgt.MountPath == where {
+				return t.TargetID
+			}
+		}
+	}
+	return ""
+}
+
+// refuseIfBackupTarget refuses a destructive drive op when `where` backs a configured backup tier,
+// and reports whether it did. The message names the storage AND the remedy: the operation is not
+// forbidden forever, it is ordered — reassign the backup target first, then the drive is free.
+//
+// Ejecting the drive that holds the only local whole-guest backup is exactly the silent-degradation
+// class this arc has been closing: it succeeds, nothing alarms, and the box quietly loses its
+// drive-loss protection while still reporting a configured tier.
+func (s *Server) refuseIfBackupTarget(ctx context.Context, w http.ResponseWriter, op string, vmid int, where string) bool {
+	target := s.backupTargetAt(ctx, where)
+	if target == "" {
+		return false
+	}
+	s.logger.Warn("local-api: protected — "+op+" refused: the mount backs a configured backup tier",
+		"vmid", vmid, "where", where, "target", target)
+	writeErr(w, http.StatusConflict,
+		"this drive is the whole-guest backup target ("+target+") — "+op+" refused. "+
+			"Reassign the backup target to another drive first, or the box loses its local drive-loss protection.")
+	return true
+}
+
 // roleForMountPath resolves the AUTHORITATIVE protection role of the storage mounted at `where`, from
 // the agent's OWN storage view + host topology (never the caller's claim). It mirrors deviceRole but
 // keys on the mount path (the eject input). It FAILS SAFE to system (most-protected) on any

internal/localapi/disks_backup_target_row_test.go

@@ -0,0 +1,342 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"io"
+	"log/slog"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
+)
+
+// R-116 — the backup-target flag must be reachable from the row the CONTROLLER keys on.
+//
+// THE DEFECT. The controller resolves the drive-absent alarm by the registered StoragePath, which for
+// an external drive is the GUEST path. When the device vanishes, Observe's exactMountDevice fails, so
+// BackingDevice becomes "" and RoleForStorage returns RoleSystem (role.go:180-181) — the guest-path
+// block is skipped and the flag-bearing row loses its guest path. It keeps its MountPath, so the union
+// loop DEDUPES the registry row away, and /disks carries NO row with that guest path at all.
+// driveTargetByPath then has no entry, isTarget[guestPath] is a MISSING KEY, and the specific
+// backup_target_absent alarm cannot fire — the generic storage_disconnected goes out instead, while
+// the RETURN (rows rejoined) fires the specific recovery. An operator gets a pair they cannot match.
+// Measured live: felhom.eu audits/SESSION-C-2026-07-29.md §5.
+//
+// These tests exercise the REAL GET /disks response and assert the emitted JSON, because the failure
+// class is "the value is on the wrong row" — a test that hand-builds rows proves nothing about which
+// row the handler actually emits.
+
+// targetRowServer builds a /disks server whose primary backup tier is `primaryTarget`, over the given
+// Observe targets. boundCheck/deviceCheck are pinned so the R-113 conjunction is not the variable
+// under test here.
+func targetRowServer(t *testing.T, primaryTarget string, targets []hub.StorageTarget) *Server {
+	t.Helper()
+	return targetRowServerWithDrives(t, primaryTarget, targets, nil)
+}
+
+// targetRowServerWithDrives additionally wires the REGISTRY union source. v0.115.0's tests left
+// DriveTargets nil, so the union loop never ran and the two-row absent shape — the actual defect — was
+// invisible to the whole suite. Any test about which row carries what MUST populate this.
+func targetRowServerWithDrives(t *testing.T, primaryTarget string, targets []hub.StorageTarget,
+	drives []storage.KnownTarget) *Server {
+	t.Helper()
+	var known storage.KnownTargets
+	if drives != nil {
+		known = fakeKnownTargets{drives: drives}
+	}
+	srv, err := NewServer(Options{
+		DriveTargets: known,
+		ListenAddr:   "127.0.0.1:0",
+		Guests:       &fakeGuestsCfg{}, Backups: &fakeBackups{}, Store: &fakeStore{},
+		Storage: fakeStorage{targets: targets},
+		// Service is REQUIRED: normalizeBackupTiers (backup_tiers.go:21-22) drops any tier with a nil
+		// Service, and the legacy fallback then yields TargetID "" — which silently makes every
+		// BackupTarget false and would make these tests pass for the wrong reason.
+		BackupTiers: []BackupTier{{TargetID: primaryTarget, Primary: true, Service: &fakeBackups{}}},
+		Tokens:      staticTokens{"A": 8200},
+		Disks:       &fakeDiskOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4"}},
+		DiskGate:    &fakeGate{}, HostReader: sysOnSDA(),
+		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	srv.baseCtx = context.Background()
+	srv.boundCheck = func(string) bool { return true }
+	srv.deviceCheck = func(string) bool { return true }
+	return srv
+}
+
+// wireDisks returns the decoded /disks rows exactly as the controller receives them.
+func wireDisks(t *testing.T, srv *Server) []map[string]any {
+	t.Helper()
+	body := do(t, srv.Handler(), "GET", "/disks", "A", "").Body.Bytes()
+	var w struct {
+		Data struct {
+			Disks []map[string]any `json:"disks"`
+		} `json:"data"`
+	}
+	if err := json.Unmarshal(body, &w); err != nil {
+		t.Fatalf("decode /disks: %v (%s)", err, body)
+	}
+	return w.Data.Disks
+}
+
+// isTargetByPath reproduces the controller's driveTargetByPath EXACTLY (intermediary.go:602-616):
+// both keyings, value = backup_target. This is the map whose missing key is the whole defect, so the
+// assertion is made against a faithful copy of it rather than against a field in isolation.
+func isTargetByPath(disks []map[string]any) map[string]bool {
+	out := map[string]bool{}
+	for _, d := range disks {
+		bt, _ := d["backup_target"].(bool)
+		if gp, ok := d["guest_path"].(string); ok && gp != "" {
+			out[gp] = bt
+		}
+		if mp, ok := d["mount_path"].(string); ok && mp != "" {
+			out[mp] = bt
+		}
+	}
+	return out
+}
+
+// theAbsentTarget is the absent-target Observe row, CORRECTED in v0.116.0 to the shape the live box
+// actually produces.
+//
+// THIS FIXTURE IS WHY AN INERT FIX SHIPPED GREEN. As written for v0.115.0 it supplied
+// `MountPath: "/mnt/mentes"` — a field the real absent state does NOT have. The same exactMount failure
+// that empties BackingDevice empties MountPath (observe.go:184-190), so on the live box this row carries
+// `mount_path: ""`, and v0.115.0's `StablePathForRaw(t.MountPath)` was therefore
+// `StablePathForRaw("")` == "". The fixture handed the code a value production never supplies, the test
+// went green, and the fix was inert on real hardware — twice.
+//
+// Captured payload this now mirrors, field for field:
+// felhom.eu audits/DIAG-r116-disks-payload-2026-07-30.md §6.2.
+var theAbsentTarget = hub.StorageTarget{
+	Name: "felhom-backup", Type: hub.StorageTypeLocalDir,
+	MountPath: "", BackingDevice: "", ConfigPath: "/mnt/mentes",
+	State: hub.StorageStateDisconnected,
+	// DurableID degrades off the fs-UUID exactly as the live payload showed (`path:/mnt/cel` there).
+	DurableID: "path:/mnt/mentes",
+}
+
+// theAbsentRegistryRow is the OTHER half of the live absent payload — the registry/union row. Its
+// MountPath comes from the systemd .mount unit FILE (registry_known.go:40-75), which never consults the
+// mount table, so it survives the device intact. Its presence is what made /disks carry the drive TWICE.
+var theAbsentRegistryRow = []storage.KnownTarget{
+	{Name: "9303-uuid", Type: hub.StorageTypeUSB, MountPath: "/mnt/mentes",
+		DurableID: "uuid:9303", UUID: "9303"},
+}
+
+// ── the observable that must move ───────────────────────────────────────────────────────────────
+
+// RED-PROOF: delete the `di.GuestPath == "" && di.BackupTarget && t.BackingDevice == ""` block and
+// this fails with "the guest path the controller keys on is MISSING from /disks entirely".
+func TestAbsentBackupTargetIsResolvableByGuestPath(t *testing.T) {
+	disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
+		[]hub.StorageTarget{theAbsentTarget}, theAbsentRegistryRow))
+	isTarget := isTargetByPath(disks)
+
+	const guestPath = "/mnt/felhom-drives/mentes"
+	got, present := isTarget[guestPath]
+	if !present {
+		t.Fatalf("isTarget[%q] is a MISSING KEY — the guest path the controller keys on is missing from "+
+			"/disks entirely, so notifyDriveAbsent takes the generic branch and backup_target_absent "+
+			"can never fire (R-116)", guestPath)
+	}
+	if !got {
+		t.Fatalf("isTarget[%q] = FALSE. Both rows for this drive reached the wire and the registry row — "+
+			"appended last, BackupTarget defaulted false — overwrote the flag-bearing row's true. This is "+
+			"the measured live defect, not a hypothetical: rows=%d", guestPath, len(disks))
+	}
+}
+
+// ── V2: the new guest path must NOT make the gate read the drive as PRESENT ─────────────────────
+
+// This is the over-correction guard, in the exact component under test. planDriveGates computes
+// present[gp] = present[gp] || d.BoundUnderParent. If the row we now emit carried a true
+// BoundUnderParent, this fix would SILENCE the alarm it exists to raise.
+func TestAbsentTargetRowDoesNotRegisterPresence(t *testing.T) {
+	srv := targetRowServer(t, "felhom-backup", []hub.StorageTarget{theAbsentTarget})
+	// deviceCheck/boundCheck are pinned TRUE — the strongest possible case for a false positive.
+	// The row must still report bound_under_parent=false, because that field is only ever assigned
+	// inside the guest-path blocks a system-role row does not enter.
+	for _, d := range wireDisks(t, srv) {
+		if d["guest_path"] != "/mnt/felhom-drives/mentes" {
+			continue
+		}
+		if bup, _ := d["bound_under_parent"].(bool); bup {
+			t.Fatal("the absent backup-target row reports bound_under_parent=true — planDriveGates " +
+				"would compute present=true, the Stop branch would never run, and this fix would " +
+				"SUPPRESS the very alarm it exists to raise")
+		}
+		return
+	}
+	t.Fatal("the absent target row never reached the wire")
+}
+
+// ── V1: the gates, each on its own ──────────────────────────────────────────────────────────────
+
+// Case B is the COMMON fresh-box shape, not an edge: the tier target is the builtin `local` on the
+// root fs. It must never acquire a guest path.
+func TestCaseBLocalTargetGetsNoGuestPath(t *testing.T) {
+	disks := wireDisks(t, targetRowServer(t, "local", []hub.StorageTarget{
+		{Name: "local", Type: "local", MountPath: "/var/lib/vz", BackingDevice: "", State: hub.StorageStateAttached},
+	}))
+	for _, d := range disks {
+		if gp, _ := d["guest_path"].(string); gp != "" {
+			t.Errorf("the Case B target on %v acquired guest path %q — a system-drive backup target "+
+				"must not cross into the guest", d["mount_path"], gp)
+		}
+	}
+}
+
+// A storage that is RoleSystem because it is genuinely system-BACKED (non-empty BackingDevice on the
+// system disk) must be excluded — this is the case StablePathForRaw would NOT have filtered, since
+// /mnt/<name> maps to a real stable path. The BackingDevice gate is what stops it.
+func TestSystemBackedTargetUnderMntGetsNoGuestPath(t *testing.T) {
+	disks := wireDisks(t, targetRowServer(t, "sysbackup", []hub.StorageTarget{
+		// sysOnSDA() makes /dev/sda the system disk, so this classifies RoleSystem with a REAL device.
+		{Name: "sysbackup", Type: hub.StorageTypeLocalDir, MountPath: "/mnt/sysbackup",
+			BackingDevice: "/dev/sda1", State: hub.StorageStateAttached},
+	}))
+	for _, d := range disks {
+		if gp, _ := d["guest_path"].(string); gp != "" {
+			t.Errorf("a system-BACKED backup target acquired guest path %q — the BackingDevice gate "+
+				"failed and the :213-214 boundary was widened", gp)
+		}
+	}
+}
+
+// ── the negative ────────────────────────────────────────────────────────────────────────────────
+
+// A drive that is NOT the target must not acquire the flag on any row, present or absent.
+func TestNonTargetDriveNeverCarriesTheFlag(t *testing.T) {
+	disks := wireDisks(t, targetRowServer(t, "felhom-backup", []hub.StorageTarget{
+		{Name: "adat", Type: hub.StorageTypeLocalDir, MountPath: "/mnt/adat",
+			BackingDevice: "", State: hub.StorageStateDisconnected},
+	}))
+	for _, d := range disks {
+		if bt, _ := d["backup_target"].(bool); bt {
+			t.Errorf("non-target drive %v reports backup_target=true", d["name"])
+		}
+		if gp, _ := d["guest_path"].(string); gp != "" {
+			t.Errorf("an absent NON-target drive acquired guest path %q via the R-116 fallback — the "+
+				"BackupTarget gate failed", gp)
+		}
+	}
+}
+
+// ── v0.116.0 — the join, and the regression it must not cause ───────────────────────────────────
+
+// THE JOIN. With the device gone the two records of one drive share no runtime field, so the dedup has
+// to key on the one thing both can still derive: the CONFIGURED path (storage.cfg's `path` on the
+// Observe side, the .mount unit's `Where` on the registry side), expressed as the stable guest path.
+// This pins that exactly one row survives — because driveTargetByPath ASSIGNS rather than ORs, so two
+// rows disagreeing on the flag is decided by append order, which is not a contract anyone should rely on.
+//
+// RED-PROOF: delete the `seenGuest[gp]` skip in the union loop and this fails with rows=2.
+func TestAbsentTargetAppearsExactlyOnce(t *testing.T) {
+	disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
+		[]hub.StorageTarget{theAbsentTarget}, theAbsentRegistryRow))
+
+	const guestPath = "/mnt/felhom-drives/mentes"
+	var rows []map[string]any
+	for _, d := range disks {
+		if gp, _ := d["guest_path"].(string); gp == guestPath {
+			rows = append(rows, d)
+		}
+	}
+	if len(rows) != 1 {
+		t.Fatalf("the absent drive is carried by %d rows, want exactly 1 — with two rows the flag the "+
+			"controller reads is decided by append order, not by the fix. rows=%v", len(rows), rows)
+	}
+	if bt, _ := rows[0]["backup_target"].(bool); !bt {
+		t.Error("the surviving row does not carry backup_target=true")
+	}
+}
+
+// THE REGRESSION THIS FIX MUST NOT CAUSE, and the reason neither obvious option was taken.
+//
+// The controller reads `d.BackupTarget && d.MountPath != ""` as "a real drive with its own mountpoint —
+// HEALTHY" and returns immediately (backup_target_offer.go:79). So the two candidate fixes that look
+// smallest — back-filling MountPath onto the Observe row, or teaching the registry row the flag (its
+// MountPath is non-empty, read from the stale unit file) — BOTH produce a row satisfying that predicate
+// while the drive is missing. Either would have silently regressed R-114, which shipped 2026-07-29 and
+// tells the customer „A rendszermentés meghajtója nem érhető el" in exactly this state, flipping it back
+// to a false healthy.
+//
+// R-114's correctness currently rests on the absent-state rows NOT combining the flag with a mount path.
+// That coupling was invisible until the payload was captured, and it is what this test pins.
+//
+// RED-PROOF: set `MountPath: "/mnt/mentes"` on theAbsentTarget (v0.115.0's fixture value) and this fails.
+func TestAbsentTargetKeepsR114DegradedSignal(t *testing.T) {
+	disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
+		[]hub.StorageTarget{theAbsentTarget}, theAbsentRegistryRow))
+
+	for _, d := range disks {
+		bt, _ := d["backup_target"].(bool)
+		mp, _ := d["mount_path"].(string)
+		if bt && mp != "" {
+			t.Fatalf("row %v carries backup_target=true AND mount_path=%q while the drive is ABSENT. "+
+				"resolveBackupTargetState (backup_target_offer.go:79) reads that as \"a real drive with "+
+				"its own mountpoint — healthy\" and returns before its TargetAbsent branch, so the "+
+				"customer is told the backup target is fine while its drive is gone. That is R-114, "+
+				"regressed.", d["name"], mp)
+		}
+	}
+}
+
+// PRESENT-STATE PARITY. The fix must change nothing when the drive is there. Present state is the
+// state every healthy box is in, so a change here reaches the whole fleet; absent state reaches only a
+// box with a problem. Both rows are supplied, exactly as on a live present box, and the pre-existing
+// MountPath dedup must still collapse them to one COMPLETE row.
+func TestPresentTargetPayloadUnchanged(t *testing.T) {
+	present := hub.StorageTarget{
+		Name: "felhom-backup", Type: hub.StorageTypeLocalDir,
+		MountPath: "/mnt/mentes", BackingDevice: "/dev/sdb", ConfigPath: "/mnt/mentes",
+		State: hub.StorageStateAttached, DurableID: "uuid:9303",
+	}
+	disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
+		[]hub.StorageTarget{present}, theAbsentRegistryRow))
+
+	var rows []map[string]any
+	for _, d := range disks {
+		if d["name"] == "felhom-backup" || d["mount_path"] == "/mnt/mentes" {
+			rows = append(rows, d)
+		}
+	}
+	if len(rows) != 1 {
+		t.Fatalf("present state carries the drive on %d rows, want 1 (the MountPath dedup): %v", len(rows), rows)
+	}
+	r := rows[0]
+	for field, want := range map[string]any{
+		"mount_path": "/mnt/mentes", "guest_path": "/mnt/felhom-drives/mentes",
+		"backing_device": "/dev/sdb", "role": "user-data", "state": "attached",
+		"backup_target": true, "bound_under_parent": true, "durable_id": "uuid:9303",
+	} {
+		if got := r[field]; got != want {
+			t.Errorf("present-state %s = %v, want %v — the fix altered the healthy payload", field, got, want)
+		}
+	}
+}
+
+// The negative, with the union loop actually running: a non-target absent drive gains the flag on no row
+// and keeps its own registry row (nothing to dedup against, since no Observe row claims its guest path).
+func TestAbsentNonTargetKeepsItsRegistryRowAndNoFlag(t *testing.T) {
+	disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
+		[]hub.StorageTarget{{Name: "adat", Type: hub.StorageTypeLocalDir, MountPath: "",
+			BackingDevice: "", ConfigPath: "/mnt/adat", State: hub.StorageStateDisconnected}},
+		[]storage.KnownTarget{{Name: "adat-uuid", Type: hub.StorageTypeUSB,
+			MountPath: "/mnt/adat", DurableID: "uuid:1111", UUID: "1111"}}))
+
+	isTarget := isTargetByPath(disks)
+	for k, v := range isTarget {
+		if v {
+			t.Errorf("isTarget[%q] = true for a NON-target drive — the BackupTarget gate failed", k)
+		}
+	}
+	if _, ok := isTarget["/mnt/felhom-drives/adat"]; !ok {
+		t.Error("the non-target drive lost its guest-path key entirely — the union row was over-suppressed")
+	}
+}

internal/localapi/disks_bind_liveness_test.go

@@ -0,0 +1,397 @@
+package localapi
+
+import (
+	"context"
+	"io"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
+)
+
+// R-117 — BoundUnderParent must mean THE BIND ACTUALLY WORKS, not "a mount by that name exists".
+//
+// THE BUG THESE PIN. GuestSeesMount (intermediary.go) and isHostMountpoint both parse a mountinfo line
+// and then test only fields[4], the mount POINT. Field 3 — major:minor — sits in the same parsed slice
+// and was discarded. So after a drive is detached and returned, the raw host mount heals onto the NEW
+// device via its fs-UUID-keyed unit while the bind still names the OLD one, and BOTH existing terms stay
+// true. Measured live: raw on 8:32 /dev/sdc, bind on 8:16 /dev/sdb with `shutdown`, BoundUnderParent
+// TRUE, EIO on every read and write, and the controller's gate taking its Return branch — restarting the
+// customer's apps onto that namespace and emailing backup_target_restored, with no alarm on any channel
+// (felhom.eu audits/SPIKE-r117-bind-liveness-2026-07-30.md §3.3, §5.2).
+//
+// AND THE HALF THAT EMITS NOTHING AT ALL (spike §9, filed R-117a): a device that fails WITHOUT
+// disappearing leaves the raw mount active, the devnos EQUAL, and the drive never Disconnected — so the
+// gate produces neither a Stop nor a Return action and nothing is emitted, indefinitely. A devno
+// comparison alone reads stale-device=false there, which is why term 3 checks the filesystem's own abort
+// flags too. TestDisks_BindLiveness_AbortedFilesystemReadsAbsent is that case; a fix that shipped only
+// the devno comparison would pass every other test in this file.
+//
+// WHY THE FIXTURES ARE REAL. Each mountinfo body below is the captured output of the spike run, not a
+// hand-written line. R-116's fix shipped green and inert because its fixture supplied a MountPath
+// production never supplies. These tests redirect procSelfMountinfo at a fixture file, so the REAL
+// parser (hostMountEntries), the REAL predicate (bindLiveness) and the REAL /disks handler all run —
+// the data is injected, the logic is not.
+//
+// RED-PROOFS (each verified to land, see REPORT.md): dropping `&& s.bindUsable(...)` from either /disks
+// construction site fails StaleBindReadsAbsent / UnionPath_StaleBindReadsAbsent and
+// AbortedFilesystemReadsAbsent; dropping the abort check so only the device comparison remains fails
+// AbortedFilesystemReadsAbsent and AbortedWins_WhenDevnosAgree ALONE — that is the P1-only fix, and it is
+// the one worth fearing; dropping `emergency_ro` from abortTokensByFS fails only the emergency_ro subtest;
+// and returning BindLive instead of BindUnknown for an unreadable table fails UnknownIsTreatedAsPresent.
+//
+// A THIRD ordering trap, caught by TestBindLiveness_Verdicts during development and worth naming because
+// it reports correctly while breaking the repair: reading the abort flag BEFORE comparing devices
+// classifies the real return state as BindAborted, since its stale bind carries `shutdown` as well as a
+// different device. BoundUnderParent still reads absent — every test in Group A still passes — but
+// AttachDrive then refuses the re-bind that actually repairs it, so the self-heal never runs. The verdict
+// must answer "would a re-bind help", which means reading the abort flag of the RAW mount (the re-bind's
+// target) in the stale case, and of the bind itself only when the devices already agree.
+
+// ── fixtures, from the spike's captures ─────────────────────────────────────────────────────────
+//
+// Devices, super options, optional-field tags and root paths are verbatim. ONE substitution: the spike
+// ran against a SCRATCH shared parent (/mnt/r117-drives) so it could not disturb the live
+// /mnt/felhom-drives peer group, whereas the code under test derives the stable path itself, as
+// StableParentDir + "/" + DriveNameFromRaw(raw). So /mnt/r117-drives/sd becomes
+// /mnt/felhom-drives/r117sd. Substituting anything else would make these fixtures describe a path
+// production never produces — which is precisely how R-116's fix shipped green and inert.
+
+// mountinfoHealthy is the HEALTHY state (spike §3.3): raw and bind on the SAME device, no abort token.
+const mountinfoHealthy = `748 33 8:16 / /mnt/r117sd rw,relatime shared:450 - ext4 /dev/sdb rw,stripe=512
+755 118 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:450 - ext4 /dev/sdb rw,stripe=512
+`
+
+// mountinfoStaleBind is the R-117 RETURN state (spike §3.3 / §5.2): the drive came back as /dev/sdc
+// (8:32) and the raw mount healed onto it, while the bind still names /dev/sdb (8:16) and carries
+// `shutdown`. Both pre-R-117 terms read true here.
+const mountinfoStaleBind = `755 118 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:450 - ext4 /dev/sdb rw,stripe=512,shutdown
+814 33 8:32 / /mnt/r117sd rw,relatime shared:485 - ext4 /dev/sdc rw,stripe=512
+`
+
+// mountinfoAborted is the STEADY-STATE state (spike §9): the device errored in place, so it NEVER LEFT.
+// Raw and bind are the SAME device — the device comparison cannot see this — and ext4 has done an
+// emergency remount-ro. Today this state emits nothing on any channel.
+//
+// SECOND substitution, and it is the one that nearly made this test decoration. The spike produced this
+// state on a dm device (dm is the only mechanism that can make a device error WITHOUT disappearing), so
+// the capture reads 252:11 /dev/mapper/r117cel. Transposed here onto the USB drive shape, because
+// RoleForStorage derives role="system" for a /dev/mapper backing device — and a system-role row never
+// enters the block that computes BoundUnderParent, so the field stays false by DEFAULT and the assertion
+// below passes without term 3 ever running. It did exactly that until RP1 failed to fail (see REPORT.md).
+// An in-place abort on a USB drive is the realistic customer case anyway (a link reset that recovers the
+// link after ext4 has already given up); only the super options and the matching devnos carry the claim.
+const mountinfoAborted = `748 33 8:16 / /mnt/r117sd rw,relatime shared:450 - ext4 /dev/sdb rw,stripe=512,emergency_ro
+755 118 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:450 - ext4 /dev/sdb rw,stripe=512,emergency_ro
+`
+
+// mountinfoAbortedShutdown is the same in-place shape with the OTHER ext4 abort token, the one a device
+// removal sets. Both were measured; a check for only `shutdown` passes mountinfoAborted and a check for
+// only `emergency_ro` passes this — which is why abortTokensByFS carries both, and why RP4 exists.
+const mountinfoAbortedShutdown = `748 33 8:16 / /mnt/r117sd rw,relatime shared:450 - ext4 /dev/sdb rw,stripe=512,shutdown
+755 118 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:450 - ext4 /dev/sdb rw,stripe=512,shutdown
+`
+
+// mountinfoUnknownFS is healthy-looking but on a filesystem whose abort vocabulary we have not measured.
+// The honest verdict is UNKNOWN — which must be treated as PRESENT, not as live and not as absent.
+const mountinfoUnknownFS = `748 33 8:16 / /mnt/r117sd rw,relatime shared:450 - btrfs /dev/sdb rw
+755 118 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:450 - btrfs /dev/sdb rw
+`
+
+// useMountinfo points the REAL parsers at a fixture for the duration of one test.
+func useMountinfo(t *testing.T, body string) {
+	t.Helper()
+	p := filepath.Join(t.TempDir(), "mountinfo")
+	if err := os.WriteFile(p, []byte(body), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	prev := procSelfMountinfo
+	procSelfMountinfo = p
+	t.Cleanup(func() { procSelfMountinfo = prev })
+}
+
+// livenessServer builds a /disks server over one Observe target (or one registry drive) whose raw mount
+// is `raw` and stable guest path derives from it. The two PRE-R-117 terms are forced TRUE — that is the
+// whole point: they were both true in the measured defect, so term 3 is the only thing that can save us.
+func livenessServer(t *testing.T, obs []hub.StorageTarget, known []storage.KnownTarget) *Server {
+	t.Helper()
+	opts := Options{
+		ListenAddr: "127.0.0.1:0",
+		Guests:     &fakeGuestsCfg{},
+		Backups:    &fakeBackups{},
+		Store:      &fakeStore{},
+		Storage:    fakeStorage{targets: obs},
+		Tokens:     staticTokens{"A": 8200},
+		Disks:      &fakeDiskOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4"}},
+		DiskGate:   &fakeGate{},
+		HostReader: sysOnSDA(),
+		Logger:     slog.New(slog.NewTextHandler(io.Discard, nil)),
+	}
+	if known != nil {
+		opts.DriveTargets = fakeKnownTargets{drives: known}
+	}
+	srv, err := NewServer(opts)
+	if err != nil {
+		t.Fatal(err)
+	}
+	srv.baseCtx = context.Background()
+	// Terms 1 and 2 TRUE — the measured defect's own conditions. livenessCheck is left nil so the real
+	// bindLiveness runs against the fixture.
+	srv.boundCheck = func(string) bool { return true }
+	srv.deviceCheck = func(string) bool { return true }
+	srv.resolveStorageDevice = func(string) (string, error) { return "/dev/sdb1", nil }
+	return srv
+}
+
+var obsSD = []hub.StorageTarget{
+	{Name: "sd", Type: hub.StorageTypeUSB, BackingDevice: "/dev/sdb", MountPath: "/mnt/r117sd", State: hub.StorageStateAttached},
+}
+var knownSD = []storage.KnownTarget{
+	{Name: "sd", Type: hub.StorageTypeUSB, MountPath: "/mnt/r117sd", DurableID: "uuid:71e1", UUID: "71e1"},
+}
+
+// ── Group A — the consequence: a dead namespace reads ABSENT ────────────────────────────────────
+
+// TestDisks_BindLiveness_StaleBindReadsAbsent is R-117 case (a), through the real /disks handler.
+// It asserts the CONSEQUENCE — what the controller reads off the wire — not that a comparison happened.
+func TestDisks_BindLiveness_StaleBindReadsAbsent(t *testing.T) {
+	useMountinfo(t, mountinfoStaleBind)
+	di := diskByMount(t, livenessServer(t, obsSD, nil), "/mnt/r117sd")
+	if di.BoundUnderParent {
+		t.Error("BoundUnderParent reports PRESENT over a stale bind (R-117). The bind names 8:16 /dev/sdb " +
+			"while the raw mount is 8:32 /dev/sdc; every access through it returns EIO. The controller's " +
+			"gate would take its Return branch (controller intermediary.go:258,:299) and restart the " +
+			"customer's apps onto a dead namespace, then email backup_target_restored.")
+	}
+}
+
+// TestDisks_BindLiveness_AbortedFilesystemReadsAbsent is R-117a, the steady-state half — and the test a
+// devno-only fix would fail. The device NEVER LEFT, so raw and bind agree on 252:11.
+func TestDisks_BindLiveness_AbortedFilesystemReadsAbsent(t *testing.T) {
+	for _, c := range []struct{ name, body string }{
+		{"emergency_ro (errors=remount-ro fired in place)", mountinfoAborted},
+		{"shutdown (forced abort)", mountinfoAbortedShutdown},
+	} {
+		t.Run(c.name, func(t *testing.T) {
+			useMountinfo(t, c.body)
+			// GUARD, earned: assert the row is the shape production emits BEFORE asserting the field.
+			// A system-role row has no GuestPath, never runs the conjunction, and reports
+			// BoundUnderParent=false by default — passing this test while proving nothing.
+			di := diskByMount(t, livenessServer(t, obsSD, nil), "/mnt/r117sd")
+			if di.Role != "user-data" || di.GuestPath == "" {
+				t.Fatalf("fixture does not reproduce the production row shape: role=%q guest_path=%q — "+
+					"the conjunction never runs on such a row, so any assertion below is vacuous",
+					di.Role, di.GuestPath)
+			}
+			if di.BoundUnderParent {
+				t.Error("BoundUnderParent reports PRESENT over an ABORTED filesystem (R-117a). The devnos " +
+					"MATCH (the device never disappeared), so the device comparison cannot see this — only " +
+					"the filesystem's own abort token can. Today this state emits NOTHING on any channel: " +
+					"the drive is never Disconnected, so the gate produces neither a Stop nor a Return.")
+			}
+		})
+	}
+}
+
+// TestDisks_BindLiveness_UnionPath_AbortedReadsAbsent — the union path for the steady-state half. It
+// matters more than the Observe one here: this row's Role is hardcoded user-data and its State hardcoded
+// attached, so the conjunction is the ONLY thing on the row that can report the abort.
+func TestDisks_BindLiveness_UnionPath_AbortedReadsAbsent(t *testing.T) {
+	useMountinfo(t, mountinfoAborted)
+	di := diskByMount(t, livenessServer(t, nil, knownSD), "/mnt/r117sd")
+	if di.BoundUnderParent {
+		t.Error("union-path drive reports PRESENT over an ABORTED filesystem (R-117a) — and its Role and " +
+			"State are both hardcoded on this row, so nothing else can contradict it")
+	}
+}
+
+// The union path carries no PVE dir-storage and hardcodes State:"attached", so these terms are the only
+// device truth on the row — R-113's reasoning, and it applies to term 3 identically.
+func TestDisks_BindLiveness_UnionPath_StaleBindReadsAbsent(t *testing.T) {
+	useMountinfo(t, mountinfoStaleBind)
+	di := diskByMount(t, livenessServer(t, nil, knownSD), "/mnt/r117sd")
+	if di.BoundUnderParent {
+		t.Error("union-path drive reports PRESENT over a stale bind (R-117) — and its State is hardcoded " +
+			"attached, so nothing else on the row can contradict it")
+	}
+}
+
+// ── Group B — no false negatives ────────────────────────────────────────────────────────────────
+
+func TestDisks_BindLiveness_HealthyReadsPresent(t *testing.T) {
+	useMountinfo(t, mountinfoHealthy)
+	for _, c := range []struct {
+		name  string
+		obs   []hub.StorageTarget
+		known []storage.KnownTarget
+	}{
+		{"observe", obsSD, nil},
+		{"union", nil, knownSD},
+	} {
+		t.Run(c.name, func(t *testing.T) {
+			di := diskByMount(t, livenessServer(t, c.obs, c.known), "/mnt/r117sd")
+			if !di.BoundUnderParent {
+				t.Error("a healthy drive reads ABSENT — a false absent STOPS a working customer's apps, " +
+					"which is strictly worse than the bug being fixed")
+			}
+		})
+	}
+}
+
+// ── Group C — cannot tell must never mean absent ────────────────────────────────────────────────
+
+// TestDisks_BindLiveness_UnknownIsTreatedAsPresent pins the rule in every way it can be reached. The
+// workspace's false-invariant table records newestArchiveOn promising exactly this over a signature that
+// could not express it; Usable() is the one place it lives, so this is the test that keeps it honest.
+func TestDisks_BindLiveness_UnknownIsTreatedAsPresent(t *testing.T) {
+	t.Run("unreadable mount table", func(t *testing.T) {
+		prev := procSelfMountinfo
+		procSelfMountinfo = filepath.Join(t.TempDir(), "does-not-exist")
+		t.Cleanup(func() { procSelfMountinfo = prev })
+		if got := bindLiveness("/mnt/felhom-drives/r117sd", "/mnt/r117sd"); got != BindUnknown {
+			t.Errorf("unreadable /proc gave %v, want BindUnknown", got)
+		}
+		di := diskByMount(t, livenessServer(t, obsSD, nil), "/mnt/r117sd")
+		if !di.BoundUnderParent {
+			t.Error("an unreadable mount table made the drive read ABSENT — cannot-tell must never stop apps")
+		}
+	})
+
+	t.Run("no raw mount entry to compare against", func(t *testing.T) {
+		// Only the bind is in the table. devicePresent is the term that answers device absence; this one
+		// must abstain rather than double-count it.
+		useMountinfo(t, `755 118 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:450 - ext4 /dev/sdb rw
+`)
+		if got := bindLiveness("/mnt/felhom-drives/r117sd", "/mnt/r117sd"); got != BindUnknown {
+			t.Errorf("missing raw entry gave %v, want BindUnknown", got)
+		}
+	})
+
+	t.Run("empty paths", func(t *testing.T) {
+		if got := bindLiveness("", "/mnt/r117sd"); got != BindUnknown {
+			t.Errorf("empty stable gave %v, want BindUnknown", got)
+		}
+		if got := bindLiveness("/mnt/felhom-drives/r117sd", ""); got != BindUnknown {
+			t.Errorf("empty raw gave %v, want BindUnknown", got)
+		}
+	})
+
+	t.Run("filesystem whose abort vocabulary is unmeasured", func(t *testing.T) {
+		useMountinfo(t, mountinfoUnknownFS)
+		if got := bindLiveness("/mnt/felhom-drives/r117sd", "/mnt/r117sd"); got != BindUnknown {
+			t.Errorf("btrfs bind gave %v, want BindUnknown — we cannot read its abort state, so we must "+
+				"not claim LIVE either", got)
+		}
+		di := diskByMount(t, livenessServer(t, obsSD, nil), "/mnt/r117sd")
+		if !di.BoundUnderParent {
+			t.Error("an unmeasured filesystem read ABSENT — that would stop apps on every non-ext4 drive")
+		}
+	})
+
+	t.Run("Usable is the single place the rule lives", func(t *testing.T) {
+		for _, c := range []struct {
+			l    BindLiveness
+			want bool
+		}{
+			{BindLive, true},
+			{BindUnknown, true}, // the rule
+			{BindStaleDevice, false},
+			{BindAborted, false},
+		} {
+			if got := c.l.Usable(); got != c.want {
+				t.Errorf("%v.Usable() = %v, want %v", c.l, got, c.want)
+			}
+		}
+	})
+}
+
+// ── Group D — the verdict itself, including the ordering that matters ───────────────────────────
+
+func TestBindLiveness_Verdicts(t *testing.T) {
+	for _, c := range []struct {
+		name, body  string
+		stable, raw string
+		want        BindLiveness
+	}{
+		{"healthy", mountinfoHealthy, "/mnt/felhom-drives/r117sd", "/mnt/r117sd", BindLive},
+		{"stale device (case a)", mountinfoStaleBind, "/mnt/felhom-drives/r117sd", "/mnt/r117sd", BindStaleDevice},
+		{"aborted in place, emergency_ro (case b)", mountinfoAborted, "/mnt/felhom-drives/r117sd", "/mnt/r117sd", BindAborted},
+		{"aborted in place, shutdown", mountinfoAbortedShutdown, "/mnt/felhom-drives/r117sd", "/mnt/r117sd", BindAborted},
+	} {
+		t.Run(c.name, func(t *testing.T) {
+			useMountinfo(t, c.body)
+			if got := bindLiveness(c.stable, c.raw); got != c.want {
+				t.Errorf("bindLiveness = %v, want %v", got, c.want)
+			}
+		})
+	}
+}
+
+// TestBindLiveness_AbortedWins pins the ORDERING, which is load-bearing and not obvious: in the measured
+// stale-bind state the filesystem ALSO carries `shutdown`, so both P1 and P2 apply. The verdict must be
+// BindAborted-or-BindStaleDevice — never live — but more importantly the in-place state, where ONLY P2
+// applies, must not fall through to a devno comparison that reads equal. This test fails if P1 is checked
+// before P2 and returns early.
+func TestBindLiveness_AbortedWins_WhenDevnosAgree(t *testing.T) {
+	useMountinfo(t, mountinfoAborted)
+	got := bindLiveness("/mnt/felhom-drives/r117sd", "/mnt/r117sd")
+	if got.Usable() {
+		t.Fatalf("bindLiveness = %v (usable) — the devnos agree because the device never left, so a "+
+			"P1-first implementation reads this as LIVE and ships R-117's silent half intact", got)
+	}
+	if got != BindAborted {
+		t.Errorf("bindLiveness = %v, want BindAborted (the abort token is the only signal here)", got)
+	}
+}
+
+// ── Group E — the parser, on a real captured table ──────────────────────────────────────────────
+
+// TestHostMountEntries_ParsesDevnoAndSuperOpts pins the field extraction R-117 turned on. The optional
+// fields run (shared:NNN master:NNN) is variable-length, so the " - " separator — not a fixed index — is
+// what locates fstype and the super options.
+func TestHostMountEntries_ParsesDevnoAndSuperOpts(t *testing.T) {
+	// A guest-side line with BOTH optional-field tags, the longest real shape (spike §5.2).
+	useMountinfo(t, `759 1176 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:459 master:450 - ext4 /dev/sdb rw,stripe=512,shutdown
+`)
+	got := hostMountEntries("/mnt/felhom-drives/r117sd")
+	if len(got) != 1 {
+		t.Fatalf("got %d entries, want 1", len(got))
+	}
+	e := got[0]
+	if e.Devno != "8:16" {
+		t.Errorf("Devno = %q, want 8:16 — this is the field R-117 was lost for want of reading", e.Devno)
+	}
+	if e.Root != "/felhom-data" {
+		t.Errorf("Root = %q, want /felhom-data", e.Root)
+	}
+	if e.FSType != "ext4" {
+		t.Errorf("FSType = %q, want ext4 (located via the ' - ' separator, not a fixed index)", e.FSType)
+	}
+	if !strings.Contains(e.SuperOpts, "shutdown") {
+		t.Errorf("SuperOpts = %q, want it to carry `shutdown`", e.SuperOpts)
+	}
+}
+
+// countHostMounts and isHostMountpoint were rewritten onto hostMountEntries; the double-bind convergence
+// AttachDrive depends on must survive that (REUSE.md: a boolean could not converge stacked binds).
+func TestHostMountEntries_CountsStackedBinds(t *testing.T) {
+	useMountinfo(t, `755 118 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime - ext4 /dev/sdb rw
+756 118 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime - ext4 /dev/sdb rw
+`)
+	if n := countHostMounts("/mnt/felhom-drives/r117sd"); n != 2 {
+		t.Errorf("countHostMounts = %d, want 2 — AttachDrive's normalize leg needs the count, not a bool", n)
+	}
+	if !isHostMountpoint("/mnt/felhom-drives/r117sd") {
+		t.Error("isHostMountpoint = false over two stacked binds")
+	}
+	if isHostMountpoint("/mnt/nope") {
+		t.Error("isHostMountpoint = true for a path with no entry")
+	}
+	if n := countHostMounts("/mnt/nope"); n != 0 {
+		t.Errorf("countHostMounts = %d for an absent path, want 0", n)
+	}
+}

internal/localapi/disks_candidates_test.go

@@ -11,9 +11,9 @@ import (
 // handleDiskCandidates splits discovery into initialize (all unclaimed) + attach (mountable-FS subset).
 func TestDiskCandidates_Split(t *testing.T) {
 	d := &fakeDiskOps{candidates: []storage.CandidateDisk{
-		{Device: "/dev/sdd", SizeBytes: 64 << 30, DataBearing: false},                                    // blank → initialize only
+		{Device: "/dev/sdd", SizeBytes: 64 << 30, DataBearing: false},                                      // blank → initialize only
 		{Device: "/dev/sde", FSType: "ext4", DataBearing: true, Mountable: true, MountSource: "/dev/sde1"}, // FS → init + attach
-		{Device: "/dev/sdf", FSType: "ntfs", DataBearing: true, Mountable: false},                         // ntfs → initialize only
+		{Device: "/dev/sdf", FSType: "ntfs", DataBearing: true, Mountable: false},                          // ntfs → initialize only
 	}}
 	h := newDiskServer(t, d, &fakeGate{}, nil, nil)
 

internal/localapi/disks_device_presence_test.go

@@ -0,0 +1,186 @@
+package localapi
+
+import (
+	"context"
+	"encoding/json"
+	"io"
+	"log/slog"
+	"strings"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
+)
+
+// R-113 — BoundUnderParent must mean THE DEVICE IS THERE, not "a mount entry with this name exists".
+//
+// THE BUG THESE PIN. The drive's raw mount at /mnt/<name> is a systemd mount unit bound to its device
+// and dies with it. The agent's own bind of <raw>/felhom-data under the shared parent is an ordinary
+// bind — nothing ties it to the device — so it OUTLIVES the device as a stale shell. Before v0.114.0
+// BoundUnderParent was half 1 only, so a pulled drive kept reporting present, the controller's
+// drive-absent gate never produced a Stop action, and NOTHING fired on any channel: not
+// backup_target_absent, not the generic storage_disconnected. Measured live in E-2d with the device
+// detached — /mnt/mentes2 NOT mounted while /mnt/felhom-drives/mentes2 still read /dev/sdb[/felhom-data]
+// (felhom.eu audits/E2D-fresh-vm-2026-07-29.md §5.2).
+//
+// RED-PROOF. Drop `&& s.devicePresent(...)` from either construction site in disks.go and
+// TestDisks_DevicePresence_ObservePath_DeviceLossReadsAbsent / _UnionPath_... fail with
+// "reports present — the bind outlived the device (R-113)".
+//
+// These drive the REAL production path: NewServer → GET /disks through srv.Handler() → the JSON the
+// controller actually parses. The two lowest-level mount reads are injected (a unit test cannot create
+// real mounts), but nothing above them is faked, and the wire test below asserts the encoded field.
+
+// presenceServer builds a /disks server over one Observe target and/or one registry drive, with the
+// bind and device checks independently controllable — the two conditions whose CONJUNCTION is the fix.
+func presenceServer(t *testing.T, obs []hub.StorageTarget, known []storage.KnownTarget, bound, device bool) *Server {
+	t.Helper()
+	opts := Options{
+		ListenAddr: "127.0.0.1:0",
+		Guests:     &fakeGuestsCfg{},
+		Backups:    &fakeBackups{},
+		Store:      &fakeStore{},
+		Storage:    fakeStorage{targets: obs},
+		Tokens:     staticTokens{"A": 8200},
+		Disks:      &fakeDiskOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4"}},
+		DiskGate:   &fakeGate{},
+		HostReader: sysOnSDA(),
+		Logger:     slog.New(slog.NewTextHandler(io.Discard, nil)),
+	}
+	if known != nil {
+		opts.DriveTargets = fakeKnownTargets{drives: known}
+	}
+	srv, err := NewServer(opts)
+	if err != nil {
+		t.Fatal(err)
+	}
+	srv.baseCtx = context.Background()
+	srv.boundCheck = func(string) bool { return bound }
+	srv.deviceCheck = func(string) bool { return device }
+	srv.resolveStorageDevice = func(string) (string, error) { return "/dev/sdb1", nil }
+	return srv
+}
+
+func diskByMount(t *testing.T, srv *Server, mount string) DiskInfo {
+	t.Helper()
+	for _, di := range decodeDisks(t, do(t, srv.Handler(), "GET", "/disks", "A", "").Body.Bytes()) {
+		if di.MountPath == mount {
+			return di
+		}
+	}
+	t.Fatalf("no disk reported for mount %q", mount)
+	return DiskInfo{}
+}
+
+var obsUSB = []hub.StorageTarget{
+	{Name: "usb", Type: hub.StorageTypeUSB, BackingDevice: "/dev/sdb1", MountPath: "/mnt/felhom-usb", State: hub.StorageStateAttached},
+}
+
+var knownUSB = []storage.KnownTarget{
+	{Name: "mentes2", Type: hub.StorageTypeUSB, MountPath: "/mnt/mentes2", DurableID: "uuid:9303", UUID: "9303"},
+}
+
+// ── Group A — device loss is seen ───────────────────────────────────────────────────────────────
+
+func TestDisks_DevicePresence_ObservePath_DeviceLossReadsAbsent(t *testing.T) {
+	// The exact E-2d shape: the bind survives (bound=true), the device is gone (device=false).
+	di := diskByMount(t, presenceServer(t, obsUSB, nil, true, false), "/mnt/felhom-usb")
+	if di.BoundUnderParent {
+		t.Error("BoundUnderParent reports present — the bind outlived the device (R-113). " +
+			"The controller's gate would emit no Stop action, so no alarm can fire.")
+	}
+}
+
+func TestDisks_DevicePresence_UnionPath_DeviceLossReadsAbsent(t *testing.T) {
+	// The union path matters MORE: a registry drive with no PVE dir-storage hardcodes State:"attached",
+	// so the raw-mount check is the only device truth the row carries. This is what E-2d detached.
+	di := diskByMount(t, presenceServer(t, nil, knownUSB, true, false), "/mnt/mentes2")
+	if di.BoundUnderParent {
+		t.Error("union-path drive reports present — the bind outlived the device (R-113)")
+	}
+	if di.State != hub.StorageStateAttached {
+		t.Logf("note: union-path State is %q", di.State) // hardcoded; see the OBSERVATION in the report
+	}
+}
+
+// ── Group B — the healthy drive, and the return ─────────────────────────────────────────────────
+
+func TestDisks_DevicePresence_HealthyReadsPresent(t *testing.T) {
+	for _, c := range []struct {
+		name  string
+		obs   []hub.StorageTarget
+		known []storage.KnownTarget
+		mount string
+	}{
+		{"observe", obsUSB, nil, "/mnt/felhom-usb"},
+		{"union", nil, knownUSB, "/mnt/mentes2"},
+	} {
+		t.Run(c.name, func(t *testing.T) {
+			di := diskByMount(t, presenceServer(t, c.obs, c.known, true, true), c.mount)
+			if !di.BoundUnderParent {
+				t.Error("a bound drive whose device is present must read PRESENT — " +
+					"a false absent stops a working customer's apps (Scenario C's failure mode)")
+			}
+		})
+	}
+}
+
+// ── Group C — the over-correction guard: boot ordering must not regress ─────────────────────────
+
+func TestDisks_DevicePresence_BootWindowStillReadsAbsent(t *testing.T) {
+	// Boot ordering: the raw drive mounts EARLY (device=true), the agent binds under the parent ~18s
+	// LATER (bound=false). Presence must stay FALSE in that window — unchanged from before R-113 — so
+	// apps stay stopped until the bind is live and the gate's Return branch recreates them.
+	di := diskByMount(t, presenceServer(t, obsUSB, nil, false, true), "/mnt/felhom-usb")
+	if di.BoundUnderParent {
+		t.Error("boot window reports present before the bind landed — this regresses the reboot " +
+			"convergence the controller's gate comment at intermediary.go:220-224 depends on")
+	}
+}
+
+// ── Group D — unknown must never mean absent ────────────────────────────────────────────────────
+
+func TestDisks_DevicePresence_UnknownIsNotAbsent(t *testing.T) {
+	// devicePresent has nothing to ask about when there is no raw mount path. It must answer TRUE.
+	// Absence of a signal is not evidence of absence of a device — and the cost of getting this
+	// backwards is stopping a healthy customer's apps.
+	srv := presenceServer(t, nil, nil, true, false)
+	srv.deviceCheck = nil // exercise the real devicePresent, not the injected fake
+	if !srv.devicePresent("") {
+		t.Error("devicePresent(\"\") = false — an unanswerable question was reported as ABSENT")
+	}
+}
+
+// ── The wire contract — what the controller actually parses ─────────────────────────────────────
+
+// TestDisks_DevicePresence_WireFieldIsFalseOnDeviceLoss travels construction → HTTP handler → JSON
+// encoding and asserts the ENCODED field, because that is what crosses to the controller. A struct-level
+// assertion would not catch the field being dropped from the wire (e.g. an omitempty regression), and
+// `bound_under_parent` is the single field the controller's drive-absent gate keys on.
+func TestDisks_DevicePresence_WireFieldIsFalseOnDeviceLoss(t *testing.T) {
+	body := do(t, presenceServer(t, nil, knownUSB, true, false).Handler(), "GET", "/disks", "A", "").Body.Bytes()
+	if !strings.Contains(string(body), `"bound_under_parent"`) {
+		t.Fatalf("the wire has no bound_under_parent field at all — the controller's gate reads nothing: %s", body)
+	}
+	var wire struct {
+		Data struct {
+			Disks []map[string]any `json:"disks"`
+		} `json:"data"`
+	}
+	if err := json.Unmarshal(body, &wire); err != nil {
+		t.Fatalf("decode /disks: %v", err)
+	}
+	var seen bool
+	for _, d := range wire.Data.Disks {
+		if d["mount_path"] != "/mnt/mentes2" {
+			continue
+		}
+		seen = true
+		if v, ok := d["bound_under_parent"].(bool); !ok || v {
+			t.Errorf("wire bound_under_parent = %v (want false) — the device is gone", d["bound_under_parent"])
+		}
+	}
+	if !seen {
+		t.Fatalf("the drive never reached the wire: %s", body)
+	}
+}

internal/localapi/disks_smart_test.go

@@ -0,0 +1,139 @@
+package localapi
+
+import (
+	"context"
+	"io"
+	"log/slog"
+	"net/http"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
+	"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
+)
+
+func smartIP(v int) *int { return &v }
+
+// TestDisks_SmartSerialized (v0.94.0) — the already-computed SMART summary is copied into the /disks
+// payload for a target that has it (Health set), including the SATA counters + temperature; a target
+// whose SMART was never read (zero-value summary, Health "") omits the field entirely.
+//
+// Red-proof: drop the `di.Smart = &sm` copy in handleDisks → the "data" disk's Smart is nil → this fails.
+func TestDisks_SmartSerialized(t *testing.T) {
+	d := &fakeDiskOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4"}}
+	sv := fakeStorage{targets: []hub.StorageTarget{
+		{
+			Name: "data", Type: hub.StorageTypeUSB, BackingDevice: "/dev/sdb1", MountPath: "/mnt/data",
+			Smart: hub.SmartSummary{
+				Health:             hub.SmartPassed,
+				TemperatureC:       smartIP(34),
+				ReallocatedSectors: smartIP(3),
+				PendingSectors:     smartIP(0),
+			},
+		},
+		// Zero-value SMART (never read) — Health "" → must be omitted from the payload.
+		{Name: "nosmart", Type: hub.StorageTypeUSB, BackingDevice: "/dev/sdc1", MountPath: "/mnt/nosmart"},
+	}}
+	h := newDiskServer(t, d, &fakeGate{}, sv, nil)
+
+	w := do(t, h, "GET", "/disks", "A", "")
+	if w.Code != http.StatusOK {
+		t.Fatalf("GET /disks: %d (%s)", w.Code, w.Body.String())
+	}
+	byName := map[string]DiskInfo{}
+	for _, di := range decodeDisks(t, w.Body.Bytes()) {
+		byName[di.Name] = di
+	}
+
+	ds := byName["data"].Smart
+	if ds == nil {
+		t.Fatal("data disk: smart summary was not serialized")
+	}
+	if ds.Health != hub.SmartPassed {
+		t.Errorf("data smart Health = %q, want PASSED", ds.Health)
+	}
+	if ds.ReallocatedSectors == nil || *ds.ReallocatedSectors != 3 {
+		t.Errorf("data ReallocatedSectors = %v, want 3", ds.ReallocatedSectors)
+	}
+	if ds.PendingSectors == nil || *ds.PendingSectors != 0 {
+		t.Errorf("data PendingSectors = %v, want 0 (a real zero, not null)", ds.PendingSectors)
+	}
+	if ds.TemperatureC == nil || *ds.TemperatureC != 34 {
+		t.Errorf("data TemperatureC = %v, want 34", ds.TemperatureC)
+	}
+
+	if byName["nosmart"].Smart != nil {
+		t.Errorf("nosmart disk: smart must be omitted when Health is empty, got %+v", byName["nosmart"].Smart)
+	}
+}
+
+// ---- Fix B (v0.95.0): the /disks union path reads SMART for registry/USB drives ----
+
+type fakeKnownTargets struct{ drives []storage.KnownTarget }
+
+func (f fakeKnownTargets) Known(context.Context) ([]storage.KnownTarget, error) { return f.drives, nil }
+
+type fakeSmartReader struct {
+	byDev map[string]hub.SmartSummary
+	calls []string
+}
+
+func (f *fakeSmartReader) SMARTForBacking(_ context.Context, dev string) hub.SmartSummary {
+	f.calls = append(f.calls, dev)
+	if s, ok := f.byDev[dev]; ok {
+		return s
+	}
+	return hub.SmartSummary{}
+}
+
+func sp(s string) *string { return &s }
+
+// A union-path (registry/USB) drive now gets a real SMART read + model, via the Smart seam — it used
+// to ride the enrich-free union path and show "Nincs adat".
+// Red-proof: delete the Fix-B block in handleDisks (the s.smart.SMARTForBacking call) → the union
+// drive carries no smart and this fails.
+func TestDisks_UnionPathReadsSMART(t *testing.T) {
+	d := &fakeDiskOps{probe: storage.DeviceProbe{Probed: true}}
+	sm := &fakeSmartReader{byDev: map[string]hub.SmartSummary{
+		"/dev/sdb1": {Health: hub.SmartPassed, ModelName: sp("TOSHIBA MQ04ABF100")},
+	}}
+	srv, err := NewServer(Options{
+		ListenAddr: "127.0.0.1:0",
+		Guests:     &fakeGuests{},
+		Backups:    &fakeBackups{},
+		Store:      &fakeStore{},
+		Storage:    fakeStorage{}, // no Observe targets → the union drive is not deduped away
+		DriveTargets: fakeKnownTargets{drives: []storage.KnownTarget{
+			{Name: "data-usb", Type: hub.StorageTypeUSB, MountPath: "/mnt/hdd_1", DurableID: "uuid:47a3", UUID: "47a3"},
+		}},
+		Smart:      sm,
+		Tokens:     staticTokens{"A": 8200},
+		Disks:      d,
+		HostReader: sysOnSDA(),
+		Logger:     slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatalf("new server: %v", err)
+	}
+	srv.baseCtx = context.Background()
+	srv.resolveStorageDevice = func(string) (string, error) { return "/dev/sdb1", nil }
+
+	disks := decodeDisks(t, do(t, srv.Handler(), "GET", "/disks", "A", "").Body.Bytes())
+	var usb *DiskInfo
+	for i := range disks {
+		if disks[i].Name == "data-usb" {
+			usb = &disks[i]
+		}
+	}
+	if usb == nil {
+		t.Fatalf("union drive not in /disks: %+v", disks)
+	}
+	if usb.Smart == nil || usb.Smart.Health != hub.SmartPassed {
+		t.Fatalf("union drive SMART not read: %+v", usb.Smart)
+	}
+	if usb.Smart.ModelName == nil || *usb.Smart.ModelName != "TOSHIBA MQ04ABF100" {
+		t.Errorf("union drive model not carried: %v", usb.Smart)
+	}
+	if len(sm.calls) != 1 || sm.calls[0] != "/dev/sdb1" {
+		t.Errorf("SMART should be read once on /dev/sdb1, got %v", sm.calls)
+	}
+}

internal/localapi/escrow_recover.go

@@ -0,0 +1,153 @@
+package localapi
+
+import (
+	"context"
+	"crypto/sha256"
+	"encoding/hex"
+	"errors"
+	"net/http"
+	"strings"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/escrow"
+)
+
+// R-199 (agent v0.125.0) — the in-guest controller asks the agent to recover the offsite repository
+// password from the hub's sealed bundle, using the customer's recovery code R.
+//
+// WHY THE AGENT AND NOT THE CONTROLLER. Three reasons, all structural: the unsealing binary (`age`)
+// is an agent runtime dependency and is deliberately absent from the controller image; the sealed
+// blob is a HOST-scoped object whose only writer is this agent under the per-host key, so the read is
+// that write's mirror; and the controller is a trust tier down — it should receive one field, not a
+// bundle it has no use for.
+//
+// R'S HANDLING, WHICH IS THE TIGHTEST RULE IN THIS FLOW. R is the one secret in the system that
+// cannot be rotated, re-issued or recovered — it exists only in the customer's hands. Here it:
+//   - arrives in the request body over the already-pinned local-API channel (the operator accepted
+//     that crossing on 2026-08-04; the acceptance covers the CHANNEL, not carelessness at either end);
+//   - is held in memory for the duration of one call and cleared on BOTH paths;
+//   - is never written to disk, never an argument in a process list, and never logged at any level,
+//     including inside an error;
+//   - is never echoed: no response this endpoint can emit contains it.
+//
+// The request-level DEBUG middleware logs method/path/status/duration and never bodies — see
+// `logRequests`. Do not add a body dump.
+//
+// THE RESPONSE CARRIES THE PASSWORD AND ITS HASH. The hash is what this session's proof compares
+// (compare by hash, never by value). The password itself is present because the next link — placing a
+// recovered password so the existing repository opens — needs it, and building a hash-only seam now
+// would have to be torn out to add it. The controller's diagnostic reads only the hash.
+
+type recoverOffsitePasswordRequest struct {
+	VMID int `json:"vmid"`
+	// RecoveryCode is the customer's R. NEVER logged, never persisted, never echoed.
+	RecoveryCode string `json:"recovery_code"`
+}
+
+// handleRecoverOffsitePassword fetches this host's sealed bundle, unseals it with R and returns only
+// the offsite repository password (plus its sha256, for hash-only comparison by the caller).
+func (s *Server) handleRecoverOffsitePassword(w http.ResponseWriter, r *http.Request, vmid int) {
+	var req recoverOffsitePasswordRequest
+	if !decodeBody(w, r, &req) {
+		return
+	}
+	if !s.scopedFromBody(w, req.VMID, vmid, r.URL.Path) {
+		return
+	}
+	R := strings.TrimSpace(req.RecoveryCode)
+	req.RecoveryCode = "" // drop the decoded copy immediately
+	if R == "" {
+		writeErr(w, http.StatusBadRequest, "recovery_code is required")
+		return
+	}
+	if s.escrowRecovery == nil {
+		R = ""
+		writeErr(w, http.StatusServiceUnavailable, "offsite key recovery is not configured on this agent (no hub client)")
+		return
+	}
+
+	ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
+	defer cancel()
+	s.logger.Info("local-api: recovering the offsite repository password from the sealed escrow (R via body, never logged/persisted)", "vmid", vmid)
+
+	pw, err := s.escrowRecovery.RecoverOffsiteRepoPassword(ctx, R)
+	R = "" // cleared on BOTH paths, before anything else can happen
+	if err != nil {
+		// Each situation gets its own status and its own words. None of them names a secret.
+		switch {
+		// ── R-224 (2026-08-06) — THE FETCH FAILURE IS NOT A WRONG CODE. ────────────────────────
+		//
+		// This case did not exist, and its absence is the defect. A failed fetch fell through to the
+		// `default` below and was answered with "the recovery code did not open the sealed bundle" —
+		// so a hub that could not be reached was reported to the customer as a bad recovery code, on
+		// the one screen whose whole purpose is to be believed about their backups.
+		//
+		// Measured live 2026-08-05 (CAMPAIGN-11 F3 and F4): a CORRECT current code returned that
+		// message in 0.0556 s with the hub firewalled off, and in 0.0299 s with this agent stopped —
+		// against ~1.0 s for a genuine unseal. No unseal was attempted in either case.
+		//
+		// 502 rather than 400: 4xx says "your request was bad", and the request was not bad — an
+		// upstream dependency failed. The status is the machine-readable half; the controller
+		// classifies on it and must never parse this sentence.
+		//
+		// ⚠ THE CODE WAS NOT USED. Nothing may be said about it — not that it was wrong, and not
+		// that it was right.
+		case errors.Is(err, escrow.ErrBundleFetch):
+			s.logger.Warn("local-api: offsite key recovery: the sealed bundle could not be FETCHED — the recovery code was never used", "vmid", vmid, "err", err)
+			writeErr(w, http.StatusBadGateway, "the sealed recovery bundle could not be fetched from the hub — the recovery code was NOT used and nothing was written")
+		case errors.Is(err, escrow.ErrNoEscrowBlob):
+			s.logger.Warn("local-api: offsite key recovery: the hub holds no sealed bundle for this host", "vmid", vmid)
+			writeErr(w, http.StatusNotFound, "the hub holds no sealed recovery bundle for this host — no escrow ceremony has run")
+		// ── R-311 (2026-08-12) — THE CODE IS RIGHT, JUST NOT FOR THE CURRENT PACKAGE. ─────────
+		//
+		// Placed ABOVE the default for the same reason ErrBundleFetch is: the default blames the
+		// customer, and this case is the one where the customer is provably not at fault. The code was
+		// used, it worked, and it opened a package the hub is deliberately keeping.
+		//
+		// 422 rather than 400: the request was well-formed AND the credential was valid — what could
+		// not be processed is the pairing of a correct code with the CURRENT package. A 400 would put
+		// it in the same bucket as a mistype, which is the whole defect. The status is the
+		// machine-readable half; the controller classifies on it and must never parse this sentence.
+		//
+		// The date travels in the body because it is the one fact that lets a customer recognise which
+		// code they are holding. No material, no code, no password — only when that package stopped
+		// being current, and whether it can yield a repository password at all.
+		case errors.Is(err, escrow.ErrCodeOpensRetained):
+			var ro *escrow.RetainedOpenedError
+			match := escrow.RetainedMatch{}
+			if errors.As(err, &ro) {
+				match = ro.Match
+			}
+			s.logger.Info("local-api: offsite key recovery: the code did NOT open the current package but DID open a RETAINED one — the customer is not at fault",
+				"vmid", vmid, "superseded_at", match.SupersededAt, "retained_has_restic_pw", match.HasResticPassword)
+			writeStatus(w, http.StatusUnprocessableEntity, false,
+				map[string]any{
+					"opens_retained":         true,
+					"superseded_at":          match.SupersededAt,
+					"retained_has_restic_pw": match.HasResticPassword,
+				},
+				"the recovery code is correct, but it belongs to an EARLIER sealed package (superseded "+match.SupersededAt+"), not the one currently held")
+		case errors.Is(err, escrow.ErrNoResticPassword):
+			s.logger.Warn("local-api: offsite key recovery: the bundle opened but predates the repository-password field", "vmid", vmid)
+			writeErr(w, http.StatusConflict, "the recovery code opened the bundle, but it carries NO offsite repository password (sealed before that field existed; it cannot be retro-fitted)")
+		default:
+			// The fail-closed WRONG-CODE case, and only it: the bundle was fetched and `age -d`
+			// refused it. Every other situation above has its own status. The agent log records the
+			// STEP, never the code.
+			s.logger.Warn("local-api: offsite key recovery: the fetched bundle did not open with the supplied recovery code", "vmid", vmid, "err", err)
+			writeErr(w, http.StatusBadRequest, "the recovery code did not open the sealed bundle — nothing was written")
+		}
+		return
+	}
+
+	sum := sha256.Sum256([]byte(strings.TrimSpace(pw)))
+	// §8.6's lesson, applied: say exactly WHAT was recovered and what was NOT, so nobody reading this
+	// concludes the wrong thing about the bundle's contents (which is how link 8 came to be missing).
+	s.logger.Info("local-api: offsite repository password RECOVERED from the sealed escrow — returning that field ONLY "+
+		"(the tunnel token, the PBS token and the WG key stay inside the agent and are not returned)",
+		"vmid", vmid, "restic_pw_sha256", hex.EncodeToString(sum[:]))
+	writeOK(w, map[string]any{
+		"restic_repo_password": pw,
+		"restic_pw_sha256":     hex.EncodeToString(sum[:]),
+	})
+}

internal/localapi/escrow_recover_class_test.go

@@ -0,0 +1,99 @@
+package localapi
+
+import (
+	"context"
+	"errors"
+	"strings"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/escrow"
+)
+
+// R-224 — THE STATUS IS THE DISCRIMINATOR, and this test asserts the CONSEQUENCE (what the HTTP
+// boundary answers) rather than the mechanism (that the sentinel exists).
+//
+// The controller one trust tier down classifies on the STATUS and must never parse the sentence. So
+// the contract this pins is: four distinguishable situations, four distinct statuses, and the
+// wrong-code message reachable ONLY from a real refusal.
+//
+// Before R-224 the first and last rows both answered 400 with the same sentence — which is how
+// CAMPAIGN-11 F3 told a customer holding a CORRECT code that it did not open their package.
+
+type fakeRecoverer struct{ err error }
+
+func (f fakeRecoverer) RecoverOffsiteRepoPassword(context.Context, string) (string, error) {
+	if f.err != nil {
+		return "", f.err
+	}
+	return "0123456789abcdef0123456789abcdef", nil
+}
+
+func TestRecoverOffsitePassword_EachSituationGetsItsOwnStatus(t *testing.T) {
+	cases := []struct {
+		name       string
+		err        error
+		wantStatus int
+		// mustNotSay guards the specific misattribution each status exists to prevent.
+		mustNotSay []string
+	}{
+		{
+			name:       "fetch failed — the code was NEVER used",
+			err:        errors.Join(escrow.ErrBundleFetch, errors.New("hub: transport error: no route to host")),
+			wantStatus: 502,
+			mustNotSay: []string{"did not open"},
+		},
+		{
+			name:       "wrong code — the bundle WAS fetched and refused it",
+			err:        errors.New("escrow: the recovery code did not unwrap the identity escrow"),
+			wantStatus: 400,
+			mustNotSay: []string{"could not be fetched"},
+		},
+		{
+			name:       "the hub holds no bundle",
+			err:        escrow.ErrNoEscrowBlob,
+			wantStatus: 404,
+			mustNotSay: []string{"did not open"},
+		},
+		{
+			name:       "the bundle predates the repository-password field",
+			err:        escrow.ErrNoResticPassword,
+			wantStatus: 409,
+			mustNotSay: []string{"could not be fetched"},
+		},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			srv := newTestServerS(t, &fakeGuests{}, &fakeBackups{}, &fakeStore{}, nil)
+			srv.escrowRecovery = fakeRecoverer{err: tc.err}
+			w := do(t, srv.Handler(), "POST", "/escrow/recover-offsite-password", "A",
+				`{"vmid":8200,"recovery_code":"correct horse battery staple sedative anaconda wobbly kingdom placard yodel"}`)
+			if w.Code != tc.wantStatus {
+				t.Fatalf("status: got %d, want %d — body=%s", w.Code, tc.wantStatus, w.Body.String())
+			}
+			for _, phrase := range tc.mustNotSay {
+				if strings.Contains(w.Body.String(), phrase) {
+					t.Fatalf("the %d answer must not say %q — body=%s", tc.wantStatus, phrase, w.Body.String())
+				}
+			}
+		})
+	}
+}
+
+// The pair that matters most, stated as its own assertion so a regression cannot hide inside a table:
+// a fetch failure and a wrong code must never answer with the SAME status. Collapsing them is the
+// whole of R-224.
+func TestRecoverOffsitePassword_FetchFailureAndWrongCodeDiffer(t *testing.T) {
+	status := func(err error) int {
+		srv := newTestServerS(t, &fakeGuests{}, &fakeBackups{}, &fakeStore{}, nil)
+		srv.escrowRecovery = fakeRecoverer{err: err}
+		return do(t, srv.Handler(), "POST", "/escrow/recover-offsite-password", "A",
+			`{"vmid":8200,"recovery_code":"correct horse battery staple sedative anaconda wobbly kingdom placard yodel"}`).Code
+	}
+	fetch := status(errors.Join(escrow.ErrBundleFetch, errors.New("no route to host")))
+	wrong := status(errors.New("escrow: the recovery code did not unwrap the identity escrow"))
+	// RED-PROOF: delete the ErrBundleFetch case from handleRecoverOffsitePassword → both become 400
+	// → this FAILS. That is the exact pre-R-224 code, and the exact defect CAMPAIGN-11 measured.
+	if fetch == wrong {
+		t.Fatalf("a failed fetch and a wrong code must not share a status (both %d)", fetch)
+	}
+}

internal/localapi/guestmemory.go

@@ -134,9 +134,9 @@ func (s *Server) readMemoryBounds(ctx context.Context, vmid int) (memoryBounds,
 		return memoryBounds{}, fmt.Errorf("node status: %w", err)
 	}
 	b := memoryBounds{
-		allocatedMB: cfg.Memory,                          // PVE config memory is already MB
-		usageMB:     bytesToMBUp(st.Mem),                 // bytes → MB, rounded up
-		hostTotalMB: node.Memory.Total / mib,             // bytes → MB, floor (conservative for max)
+		allocatedMB: cfg.Memory,              // PVE config memory is already MB
+		usageMB:     bytesToMBUp(st.Mem),     // bytes → MB, rounded up
+		hostTotalMB: node.Memory.Total / mib, // bytes → MB, floor (conservative for max)
 		minMB:       minGuestMemoryMB,
 		running:     st.Status == "running",
 	}

internal/localapi/guestpower.go

@@ -0,0 +1,270 @@
+package localapi
+
+import (
+	"context"
+	"log/slog"
+	"sync"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+// F-REBOOT (Campaign 8 fault 11) — a guest that should be running and is not.
+//
+// THE OUTAGE THIS EXISTS TO KILL. A `pct reboot` issued while a vzdump was in flight completed its
+// SHUTDOWN half and never issued the start. The guest was found `stopped` with 0 containers, no
+// lock, and nothing retrying; it stayed down 9m47s until a human ran `pct start`. The backup itself
+// SUCCEEDED — so every alarm the appliance has was silent, because nothing was broken except that
+// the customer's entire appliance was off.
+//
+// WHY THE EXISTING RECOVERY MISSED IT. `RecoverStaleLockedGuests` (stalelock.go) already does
+// unlock → delete dangling snapshot → start iff onboot, and it is CORRECT. It missed this by two
+// gaps, both narrow:
+//   - its predicate acts only on a guest holding a stale vzdump lock (`backup`/`snapshot-delete`);
+//     fault 11's guest was stopped and UNLOCKED, so it returned early;
+//   - it runs ONCE at agent startup, on the load-bearing invariant that a backup lock present then
+//     is stale by definition. A guest that goes down while the agent is already up is never
+//     re-examined.
+//
+// This watchdog closes exactly those two gaps and nothing more: it is periodic, and it acts on
+// "should be running, is not, and is not locked".
+//
+// ── THE TRAP, WHICH IS THE SAME SHAPE AS F-CRIT-1's ──────────────────────────────────────────
+//
+// A guest the operator deliberately stopped must NOT be auto-started. Fighting the operator makes
+// maintenance impossible and is worse than the outage — the same over-correction that F-CRIT-1's fix
+// had to avoid when it stopped whitelisting StateStopped.
+//
+// The distinction used is `onboot`, and it is deliberately NOT invented here:
+//   - it is ALREADY the distinction stalelock.go uses for exactly this decision
+//     (`if onboot && g.Status != "running"`), so the two paths cannot disagree;
+//   - it is 1 on customer guests and 0 on scratch/golden guests (agent v0.101.0 sets scratch to 0);
+//   - it is the same flag `pve-guests` itself consults at host boot, so the agent AGREES WITH THE
+//     PLATFORM rather than maintaining a second, private definition of "should be running".
+//
+// The hub's desired-state `Run` (internal/desired) is a stronger signal and is wired, but it is
+// hub-dependent. `onboot` keeps working on a box that has lost hub contact — which is precisely when
+// an unattended appliance most needs to come back up.
+
+const (
+	// guestPowerInterval is how often the watchdog looks. Matches the guestnet watchdog's cadence so
+	// the two guest-facing sweeps stay in step, and is far below the 9m47s outage the finding recorded.
+	guestPowerInterval = 60 * time.Second
+
+	// guestPowerMaxAttempts bounds the retry. A guest that will not start must not be started in a
+	// loop forever (Scenario C) — after this many failures the watchdog stops trying and raises it.
+	guestPowerMaxAttempts = 3
+
+	// guestPowerHeartbeatEvery emits a summary line every Nth sweep. 10 x 60s = 10 minutes, matching
+	// the controller's deadapp heartbeat.
+	//
+	// WHY THIS EXISTS, and it is a correction to this file's OWN first version (v0.107.0): the
+	// watchdog logged at startup and when it ACTED, and was otherwise silent. A silent watchdog is
+	// indistinguishable from a dead one — which is F-OBS, the very finding fixed in the same session
+	// this file shipped in, and it is what standing rule 3 exists to prevent. An operator needs a
+	// POSITIVE observable that the sweep is running; "no start lines" must not be the only evidence.
+	guestPowerHeartbeatEvery = 10
+)
+
+// guestPowerBackoff is the delay before each retry: 1m, 2m, 4m.
+//
+// Measured, not picked round: a healthy `pct start` of guest 9201 completed in ~25 s (observed twice
+// on 2026-07-28), so even the first 1-minute wait carries 2.4x headroom over a normal start. Three
+// attempts bound the disruption at roughly 7 minutes — inside the 9m47s outage this fixes — while
+// never becoming an unbounded loop.
+var guestPowerBackoff = []time.Duration{time.Minute, 2 * time.Minute, 4 * time.Minute}
+
+// guestPowerState is one guest's recovery attempt record. In-memory on purpose, like the R-88
+// breaker: an agent restart re-attempts immediately, which is the cheap direction to fail — a
+// forgotten backoff costs one extra start attempt, whereas persisting it could carry a stale
+// "this guest won't start" verdict across the restart that fixed it.
+type guestPowerState struct {
+	attempts int
+	nextAt   time.Time
+	raised   bool // the give-up fault has already been raised for this run
+}
+
+// WatchGuestPower runs the guest-power sweep every guestPowerInterval until ctx is done. No-op when
+// the stale-lock controller is not wired (it supplies the ownership-proven guest list).
+func (s *Server) WatchGuestPower(ctx context.Context) {
+	if s.staleLock == nil {
+		return
+	}
+	s.logger.Info("guest-power: watchdog started", "interval", guestPowerInterval.String(),
+		"max_attempts", guestPowerMaxAttempts)
+	t := time.NewTicker(guestPowerInterval)
+	defer t.Stop()
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case <-t.C:
+			s.GuestPowerTick(ctx)
+		}
+	}
+}
+
+// GuestPowerTick performs one sweep. Exported so a test (and a live check) can drive exactly one
+// cycle instead of waiting on the ticker.
+func (s *Server) GuestPowerTick(ctx context.Context) {
+	if s.staleLock == nil {
+		return
+	}
+	guests, err := s.staleLock.Guests(ctx)
+	if err != nil {
+		// Unknown ownership ⇒ do nothing. Never fall back to an unfiltered list: starting a
+		// co-tenant's guest would be worse than leaving ours down.
+		s.logger.Warn("guest-power: guest list unavailable — skipping sweep (ownership unproven)", "err", err)
+		return
+	}
+	var stopped int
+	for _, g := range guests {
+		if ctx.Err() != nil {
+			return
+		}
+		if g.Status != "running" {
+			stopped++
+		}
+		s.recoverOneStoppedGuest(ctx, g)
+	}
+
+	s.guestPowerSweeps++
+	noteGuestPowerSweep(s.logger, s.guestPowerSweeps, len(guests), stopped)
+}
+
+// noteGuestPowerSweep emits the liveness observable every guestPowerHeartbeatEvery sweeps.
+//
+// It carries WHAT THE SWEEP SAW, not merely that it ran: a line saying "I am alive" cannot
+// distinguish "alive, all guests up" from "alive, one guest down and being left alone on purpose",
+// and the second is the state an operator needs to see. Pure and separately testable — the mistake
+// being corrected here was untestable precisely because it lived inline.
+func noteGuestPowerSweep(logger *slog.Logger, sweeps, evaluated, stopped int) {
+	if logger == nil || sweeps <= 0 || sweeps%guestPowerHeartbeatEvery != 0 {
+		return
+	}
+	logger.Info("guest-power: watchdog alive",
+		"sweeps_since_boot", sweeps, "guests_evaluated", evaluated, "currently_stopped", stopped)
+}
+
+// recoverOneStoppedGuest starts a single guest that should be running and is not.
+func (s *Server) recoverOneStoppedGuest(ctx context.Context, g proxmox.Guest) {
+	if g.Status == "running" {
+		s.forgetGuestPower(g.VMID) // healthy again: clear any attempt history
+		return
+	}
+
+	lock, onboot, err := s.staleLock.Lock(ctx, g.VMID)
+	if err != nil {
+		s.logger.Warn("guest-power: read guest config failed — skipping", "vmid", g.VMID, "err", err)
+		return
+	}
+
+	// SCENARIO B — a deliberately stopped guest is left alone, forever. onboot:0 means the operator
+	// (or the golden-image provisioning) does not want this guest running.
+	if !onboot {
+		return
+	}
+
+	// A locked guest belongs to another operation, mid-flight or stale. The stale-lock recovery owns
+	// that case and knows how to prove a lock is stale; this watchdog must not race it or start a
+	// guest whose lock means "a restore is writing my disks right now".
+	if lock != "" {
+		s.logger.Info("guest-power: guest is stopped but LOCKED — leaving it to the stale-lock path",
+			"vmid", g.VMID, "lock", lock)
+		return
+	}
+
+	// Never start a guest while a vzdump is genuinely in flight for it — a stop-mode backup stops the
+	// guest ON PURPOSE and starting it underneath would corrupt the backup. Fail safe on doubt.
+	running, err := s.staleLock.BackupRunning(ctx, g.VMID)
+	if err != nil {
+		s.logger.Warn("guest-power: could not confirm no backup is running — NOT starting (fail-safe)",
+			"vmid", g.VMID, "err", err)
+		return
+	}
+	if running {
+		s.logger.Info("guest-power: a vzdump is in flight — leaving the guest stopped until it finishes",
+			"vmid", g.VMID)
+		return
+	}
+
+	st, due := s.guestPowerDue(g.VMID)
+	if !due {
+		return
+	}
+	if st.attempts >= guestPowerMaxAttempts {
+		// SCENARIO C — bounded. Raise it ONCE and stop retrying; an infinite silent retry loop is the
+		// over-correction here, and a guest that has refused three starts needs a human, not a fourth.
+		if !st.raised {
+			s.markGuestPowerRaised(g.VMID)
+			s.logger.Error("guest-power: GIVING UP — guest should be running (onboot) but failed to start after repeated attempts; it needs operator attention",
+				"vmid", g.VMID, "attempts", st.attempts, "status", g.Status)
+		}
+		return
+	}
+
+	s.logger.Warn("guest-power: guest should be running (onboot) but is stopped and unlocked — starting it",
+		"vmid", g.VMID, "status", g.Status, "attempt", st.attempts+1, "of", guestPowerMaxAttempts)
+	if err := s.staleLock.Start(ctx, g.VMID); err != nil {
+		s.noteGuestPowerFailure(g.VMID)
+		s.logger.Error("guest-power: start failed", "vmid", g.VMID, "attempt", st.attempts+1, "err", err)
+		return
+	}
+	s.forgetGuestPower(g.VMID)
+	s.logger.Warn("guest-power: STARTED a guest that should have been running", "vmid", g.VMID)
+}
+
+// ---- attempt bookkeeping (guarded by its own mutex; independent of the jobs lock) ----------
+
+var guestPowerMu sync.Mutex
+
+// guestPowerDue reports the guest's attempt state and whether a new attempt is due now.
+func (s *Server) guestPowerDue(vmid int) (guestPowerState, bool) {
+	guestPowerMu.Lock()
+	defer guestPowerMu.Unlock()
+	if s.guestPower == nil {
+		s.guestPower = map[int]guestPowerState{}
+	}
+	st := s.guestPower[vmid]
+	if st.nextAt.IsZero() || !s.now().Before(st.nextAt) {
+		return st, true
+	}
+	return st, false
+}
+
+// noteGuestPowerFailure records a failed start and arms the next backoff.
+func (s *Server) noteGuestPowerFailure(vmid int) {
+	guestPowerMu.Lock()
+	defer guestPowerMu.Unlock()
+	if s.guestPower == nil {
+		s.guestPower = map[int]guestPowerState{}
+	}
+	st := s.guestPower[vmid]
+	st.attempts++
+	i := st.attempts - 1
+	if i >= len(guestPowerBackoff) {
+		i = len(guestPowerBackoff) - 1
+	}
+	st.nextAt = s.now().Add(guestPowerBackoff[i])
+	s.guestPower[vmid] = st
+}
+
+// markGuestPowerRaised records that the give-up fault has been raised, so it is logged once.
+func (s *Server) markGuestPowerRaised(vmid int) {
+	guestPowerMu.Lock()
+	defer guestPowerMu.Unlock()
+	st := s.guestPower[vmid]
+	st.raised = true
+	s.guestPower[vmid] = st
+}
+
+// forgetGuestPower clears a guest's attempt history — called when it is running again, so a guest
+// that recovers does not carry its old failures into the next incident.
+func (s *Server) forgetGuestPower(vmid int) {
+	guestPowerMu.Lock()
+	defer guestPowerMu.Unlock()
+	if s.guestPower == nil {
+		return
+	}
+	delete(s.guestPower, vmid)
+}

internal/localapi/guestpower_observable_test.go

@@ -0,0 +1,120 @@
+package localapi
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"log/slog"
+	"strings"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+// A CORRECTION TO THIS PACKAGE'S OWN v0.107.0. The guest-power watchdog logged at startup and when it
+// ACTED, and was silent otherwise — so on a healthy box the only evidence it was running was the
+// absence of start lines, which is equally consistent with the sweep having died. That is F-OBS's
+// shape and what standing rule 3 forbids, shipped in the same session F-OBS was fixed.
+//
+// These tests assert the emitted LINE. Asserting that a function was called would reproduce the
+// original mistake, which was invisible precisely because nothing pinned the output.
+
+// RED-PROOF: delete the noteGuestPowerSweep call at the end of GuestPowerTick (or the Info line
+// inside it) → this fails with "no liveness observable after 10 sweeps — silence is
+// indistinguishable from a dead watchdog".
+func TestGuestPowerSweep_EmitsLivenessObservable(t *testing.T) {
+	var buf bytes.Buffer
+	ctl := &fakeGuestPowerCtl{
+		guests: []proxmox.Guest{{VMID: 9201, Status: "running"}, {VMID: 9100, Status: "stopped"}},
+		locks:  map[int]string{9201: "", 9100: ""},
+		onboot: map[int]bool{9201: true, 9100: false}, // 9100 is a deliberate stop
+	}
+	s := gpServer(t, ctl, nil)
+	s.logger = slog.New(slog.NewTextHandler(&buf, nil))
+
+	for i := 0; i < guestPowerHeartbeatEvery; i++ {
+		s.GuestPowerTick(context.Background())
+	}
+
+	out := buf.String()
+	if !strings.Contains(out, "watchdog alive") {
+		t.Fatalf("no liveness observable after %d sweeps — silence is indistinguishable from a dead watchdog:\n%s",
+			guestPowerHeartbeatEvery, out)
+	}
+	if !strings.Contains(out, "level=INFO") {
+		t.Errorf("the observable is not at INFO — a box on the default level would never see it:\n%s", out)
+	}
+	// It must carry WHAT IT SAW. "currently_stopped=1" is the operator-relevant fact here: the sweep is
+	// alive AND is deliberately leaving one guest down, which "I ran" alone cannot express.
+	for _, want := range []string{"sweeps_since_boot=", "guests_evaluated=2", "currently_stopped=1"} {
+		if !strings.Contains(out, want) {
+			t.Errorf("the observable omits %q — it proves the sweep ran but not what it found:\n%s", want, out)
+		}
+	}
+}
+
+// It must be a summary, not a line per sweep: at 60 s that would be 1440 lines/day, which is the
+// pressure that made silence attractive in the first place.
+//
+// RED-PROOF: change the guard to `sweeps%1 != 0` → this fails with
+// "emitted 30 observables across 30 sweeps — that is the flood that made silence attractive".
+func TestGuestPowerSweep_IsASummaryNotAFlood(t *testing.T) {
+	var buf bytes.Buffer
+	lg := slog.New(slog.NewTextHandler(&buf, nil))
+
+	const sweeps = 30
+	for i := 1; i <= sweeps; i++ {
+		noteGuestPowerSweep(lg, i, 1, 0)
+	}
+
+	got := strings.Count(buf.String(), "watchdog alive")
+	want := sweeps / guestPowerHeartbeatEvery
+	if got == sweeps {
+		t.Fatalf("emitted %d observables across %d sweeps — that is the flood that made silence attractive", got, sweeps)
+	}
+	if got != want {
+		t.Errorf("emitted %d observables across %d sweeps, want %d", got, sweeps, want)
+	}
+}
+
+// The heartbeat period must stay short enough that a STALLED sweep is obvious well inside the outage
+// window this watchdog exists to close (the finding's incident was 9m47s of total appliance
+// downtime). If someone widens the cadence to hours the observable stops being a liveness signal.
+func TestGuestPowerHeartbeat_StaysUsefulAsALivenessSignal(t *testing.T) {
+	period := guestPowerHeartbeatEvery * int(guestPowerInterval.Seconds())
+	if period > 15*60 {
+		t.Errorf("heartbeat period is %ds (>15min) — too sparse to notice a stalled watchdog", period)
+	}
+	if guestPowerHeartbeatEvery < 2 {
+		t.Errorf("heartbeat every %d sweeps is a per-sweep flood", guestPowerHeartbeatEvery)
+	}
+}
+
+// Off-cadence sweeps stay quiet; a nil logger must not panic (the ticker goroutine has no recovery).
+func TestGuestPowerSweep_QuietOffCadenceAndNilSafe(t *testing.T) {
+	var buf bytes.Buffer
+	lg := slog.New(slog.NewTextHandler(&buf, nil))
+	noteGuestPowerSweep(lg, guestPowerHeartbeatEvery-1, 1, 0)
+	if buf.Len() != 0 {
+		t.Errorf("emitted off-cadence:\n%s", buf.String())
+	}
+	noteGuestPowerSweep(nil, guestPowerHeartbeatEvery, 1, 0) // must not panic
+}
+
+// A sweep that ABORTED on unproven ownership must NOT count as a healthy sweep — otherwise the
+// heartbeat would report liveness for a watchdog that is examining nothing, which is a worse lie than
+// silence.
+//
+// RED-PROOF: move the s.guestPowerSweeps++ above the Guests() error return → this fails with
+// "an aborted sweep was counted as healthy".
+func TestGuestPowerSweep_AbortedSweepIsNotCounted(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{guestsErr: errors.New("pool read failed")}
+	s := gpServer(t, ctl, nil)
+	for i := 0; i < guestPowerHeartbeatEvery*2; i++ {
+		s.GuestPowerTick(context.Background())
+	}
+	if s.guestPowerSweeps != 0 {
+		t.Errorf("an aborted sweep was counted as healthy (sweeps=%d) — the heartbeat would claim liveness for a watchdog examining nothing",
+			s.guestPowerSweeps)
+	}
+}

internal/localapi/guestpower_test.go

@@ -0,0 +1,239 @@
+package localapi
+
+import (
+	"context"
+	"errors"
+	"log/slog"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+// F-REBOOT (Campaign 8 fault 11): a guest rebooted mid-backup never came back — stopped, unlocked,
+// nothing retrying, 9m47s of total appliance outage.
+//
+// Scenario A (a crashed guest is restarted), B (a deliberately stopped guest is left alone) and
+// C (bounded retry, then escalate). B and C are what make A safe.
+
+type fakeGuestPowerCtl struct {
+	guests    []proxmox.Guest
+	guestsErr error
+	locks     map[int]string // vmid -> lock ("" = unlocked)
+	onboot    map[int]bool
+	backupRun map[int]bool
+	backupErr error
+	started   []int
+	startErr  error
+}
+
+func (f *fakeGuestPowerCtl) Guests(context.Context) ([]proxmox.Guest, error) {
+	return f.guests, f.guestsErr
+}
+func (f *fakeGuestPowerCtl) Lock(_ context.Context, vmid int) (string, bool, error) {
+	return f.locks[vmid], f.onboot[vmid], nil
+}
+func (f *fakeGuestPowerCtl) BackupRunning(_ context.Context, vmid int) (bool, error) {
+	return f.backupRun[vmid], f.backupErr
+}
+func (f *fakeGuestPowerCtl) HasVzdumpSnapshot(context.Context, int) (bool, error) { return false, nil }
+func (f *fakeGuestPowerCtl) Unlock(context.Context, int) error                    { return nil }
+func (f *fakeGuestPowerCtl) DeleteVzdumpSnapshot(context.Context, int) error      { return nil }
+func (f *fakeGuestPowerCtl) Start(_ context.Context, vmid int) error {
+	f.started = append(f.started, vmid)
+	return f.startErr
+}
+
+func gpServer(t *testing.T, ctl StaleLockController, now func() time.Time) *Server {
+	t.Helper()
+	s := &Server{staleLock: ctl, logger: slog.New(slog.NewTextHandler(discardW{}, nil))}
+	if now != nil {
+		s.now = now
+	} else {
+		s.now = func() time.Time { return time.Now().UTC() }
+	}
+	return s
+}
+
+type discardW struct{}
+
+func (discardW) Write(p []byte) (int, error) { return len(p), nil }
+
+// Scenario A — a guest that should be running (onboot) and is stopped-and-unlocked IS started.
+//
+// RED-PROOF: delete the `s.staleLock.Start(...)` call in recoverOneStoppedGuest (or make the whole
+// function return before it) → started is empty and this fails with
+// "guest 9201 was NOT started — this is F-REBOOT".
+func TestGuestPower_StoppedOnbootGuestIsStarted(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{
+		guests: []proxmox.Guest{{VMID: 9201, Status: "stopped"}},
+		locks:  map[int]string{9201: ""},
+		onboot: map[int]bool{9201: true},
+	}
+	s := gpServer(t, ctl, nil)
+
+	s.GuestPowerTick(context.Background())
+
+	if len(ctl.started) != 1 || ctl.started[0] != 9201 {
+		t.Fatalf("guest 9201 was NOT started — this is F-REBOOT (started=%v)", ctl.started)
+	}
+}
+
+// Scenario B — a DELIBERATELY stopped guest (onboot:0) is never started. This is the trap: fighting
+// the operator makes maintenance impossible and is worse than the outage being fixed.
+//
+// RED-PROOF: remove the `if !onboot { return }` guard → the golden/scratch guest is started and this
+// fails with "a deliberately stopped guest (onboot:0) was started".
+func TestGuestPower_DeliberatelyStoppedGuestIsLeftAlone(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{
+		guests: []proxmox.Guest{{VMID: 9100, Status: "stopped"}, {VMID: 990000, Status: "stopped"}},
+		locks:  map[int]string{9100: "", 990000: ""},
+		onboot: map[int]bool{9100: false, 990000: false}, // golden + scratch
+	}
+	s := gpServer(t, ctl, nil)
+
+	s.GuestPowerTick(context.Background())
+
+	if len(ctl.started) != 0 {
+		t.Errorf("a deliberately stopped guest (onboot:0) was started: %v", ctl.started)
+	}
+}
+
+// A LOCKED stopped guest belongs to the stale-lock path, which knows how to prove a lock is stale.
+// This watchdog must not race it.
+func TestGuestPower_LockedGuestIsLeftToTheStaleLockPath(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{
+		guests: []proxmox.Guest{{VMID: 9201, Status: "stopped"}},
+		locks:  map[int]string{9201: "snapshot-delete"},
+		onboot: map[int]bool{9201: true},
+	}
+	s := gpServer(t, ctl, nil)
+	s.GuestPowerTick(context.Background())
+	if len(ctl.started) != 0 {
+		t.Errorf("started a LOCKED guest: %v — that races the stale-lock recovery", ctl.started)
+	}
+}
+
+// A guest whose vzdump is genuinely in flight must be left stopped — a stop-mode backup stops the
+// guest ON PURPOSE, and starting it underneath would corrupt the backup.
+func TestGuestPower_InFlightBackupIsNotDisturbed(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{
+		guests:    []proxmox.Guest{{VMID: 9201, Status: "stopped"}},
+		locks:     map[int]string{9201: ""},
+		onboot:    map[int]bool{9201: true},
+		backupRun: map[int]bool{9201: true},
+	}
+	s := gpServer(t, ctl, nil)
+	s.GuestPowerTick(context.Background())
+	if len(ctl.started) != 0 {
+		t.Errorf("started a guest with a vzdump in flight: %v", ctl.started)
+	}
+}
+
+// Fail-safe: if we cannot confirm no backup is running, do NOT start.
+func TestGuestPower_UnconfirmableBackupFailsSafe(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{
+		guests:    []proxmox.Guest{{VMID: 9201, Status: "stopped"}},
+		locks:     map[int]string{9201: ""},
+		onboot:    map[int]bool{9201: true},
+		backupErr: errors.New("task list unavailable"),
+	}
+	s := gpServer(t, ctl, nil)
+	s.GuestPowerTick(context.Background())
+	if len(ctl.started) != 0 {
+		t.Errorf("started despite being unable to confirm no backup is running: %v", ctl.started)
+	}
+}
+
+// Unknown ownership ⇒ act on nothing. Starting a co-tenant's guest is worse than leaving ours down.
+func TestGuestPower_UnprovenOwnershipActsOnNothing(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{guestsErr: errors.New("pool read failed")}
+	s := gpServer(t, ctl, nil)
+	s.GuestPowerTick(context.Background())
+	if len(ctl.started) != 0 {
+		t.Errorf("acted with unproven ownership: %v", ctl.started)
+	}
+}
+
+// Scenario C — bounded retry, then escalate. A guest that will not start must NOT be retried forever.
+//
+// RED-PROOF: remove the `if st.attempts >= guestPowerMaxAttempts` branch → the sweep keeps starting
+// on every tick and this fails with "start attempted N times, want at most 3 — infinite retry loop".
+func TestGuestPower_RetryIsBoundedThenEscalates(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{
+		guests:   []proxmox.Guest{{VMID: 9201, Status: "stopped"}},
+		locks:    map[int]string{9201: ""},
+		onboot:   map[int]bool{9201: true},
+		startErr: errors.New("cannot start: storage offline"),
+	}
+	base := time.Now().UTC()
+	now := base
+	s := gpServer(t, ctl, func() time.Time { return now })
+
+	// Drive many ticks, advancing well past every backoff each time.
+	for i := 0; i < 12; i++ {
+		s.GuestPowerTick(context.Background())
+		now = now.Add(10 * time.Minute)
+	}
+
+	if len(ctl.started) > guestPowerMaxAttempts {
+		t.Errorf("start attempted %d times, want at most %d — this is an infinite retry loop",
+			len(ctl.started), guestPowerMaxAttempts)
+	}
+	if len(ctl.started) != guestPowerMaxAttempts {
+		t.Errorf("start attempted %d times, want exactly %d before giving up", len(ctl.started), guestPowerMaxAttempts)
+	}
+}
+
+// The backoff must actually hold a retry back — otherwise "bounded" is only bounded by luck.
+func TestGuestPower_BackoffDefersTheNextAttempt(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{
+		guests:   []proxmox.Guest{{VMID: 9201, Status: "stopped"}},
+		locks:    map[int]string{9201: ""},
+		onboot:   map[int]bool{9201: true},
+		startErr: errors.New("boom"),
+	}
+	base := time.Now().UTC()
+	now := base
+	s := gpServer(t, ctl, func() time.Time { return now })
+
+	s.GuestPowerTick(context.Background()) // attempt 1, arms a 1m backoff
+	if len(ctl.started) != 1 {
+		t.Fatalf("precondition: want 1 attempt, got %d", len(ctl.started))
+	}
+	now = base.Add(30 * time.Second) // still inside the 1m backoff
+	s.GuestPowerTick(context.Background())
+	if len(ctl.started) != 1 {
+		t.Errorf("retried inside the backoff window (%d attempts) — the bound is not being honoured", len(ctl.started))
+	}
+	now = base.Add(90 * time.Second) // past it
+	s.GuestPowerTick(context.Background())
+	if len(ctl.started) != 2 {
+		t.Errorf("did not retry after the backoff lapsed (%d attempts)", len(ctl.started))
+	}
+}
+
+// A guest that comes back healthy must lose its attempt history, so it does not carry old failures
+// into the next incident.
+func TestGuestPower_RunningGuestClearsAttemptHistory(t *testing.T) {
+	ctl := &fakeGuestPowerCtl{
+		guests:   []proxmox.Guest{{VMID: 9201, Status: "stopped"}},
+		locks:    map[int]string{9201: ""},
+		onboot:   map[int]bool{9201: true},
+		startErr: errors.New("boom"),
+	}
+	base := time.Now().UTC()
+	now := base
+	s := gpServer(t, ctl, func() time.Time { return now })
+	s.GuestPowerTick(context.Background())
+
+	if _, due := s.guestPowerDue(9201); due {
+		t.Error("precondition: a backoff should be armed after a failed start")
+	}
+	// it comes back up
+	ctl.guests = []proxmox.Guest{{VMID: 9201, Status: "running"}}
+	s.GuestPowerTick(context.Background())
+	if _, due := s.guestPowerDue(9201); !due {
+		t.Error("attempt history survived the guest coming back healthy")
+	}
+}

internal/localapi/intermediary.go

@@ -233,7 +233,33 @@ func (b *GuestBinder) AttachDrive(ctx context.Context, vmid int, where string) (
 	// makes this converge a double-bind to one (the old umount-one+mount-one never did).
 	n := countHostMounts(stable)
 	if n == 1 && b.GuestSeesMount(ctx, vmid, stable) {
-		return stable, nil // exactly one bind + guest-visible → fully live, no-op
+		// R-117: "one bind + the guest sees it" is NOT liveness. Both of those are path-presence tests, so
+		// this early return declared a namespace that EIO'd on every call "fully live" and defeated the
+		// three call sites that already invoke this repair — the 20 s reconcile ticker, agent startup, and
+		// the controller's Return branch BEFORE it restarts the apps (spike §8.2). The verdict decides:
+		switch lv := bindLiveness(stable, where); lv {
+		case BindStaleDevice:
+			// Case (a). The raw mount has healed onto the returning device; re-binding this stale shell
+			// onto it REPAIRS the namespace live, with no guest restart (proven, spike §8.1). Fall through
+			// to the normalize+rebind below. WARN not INFO-per-tick: this fires once, then it is fixed.
+			b.logger.Warn("guest-attach: bind is STALE — it names a different device than the raw mount; re-binding",
+				"vmid", vmid, "where", where, "stable", stable, "verdict", lv.String())
+		case BindAborted:
+			// Case (b), the Q7 steady-state case. The raw mount is the SAME aborted superblock, so a
+			// re-bind produces a fresh bind to a still-dead filesystem — and because this runs every 20 s
+			// it would be an infinite silent retry: exactly the silence Q7 found, with more CPU. Leave the
+			// mount alone and let the truth travel in BoundUnderParent, which now reads false, so the
+			// drive gate stops the apps and raises the alarm. Clearing an aborted filesystem needs a
+			// remount or a fsck — an operator decision, never an automatic one (R-117a).
+			//
+			// DEBUG, not WARN: this repeats every tick, and the operator-facing signal is the /disks
+			// payload plus the customer alarm. Per logging-conventions, INFO is for state changes.
+			b.logger.Debug("guest-attach: filesystem under the bind has ABORTED — not re-binding (a re-bind cannot clear it); reported not-live instead",
+				"vmid", vmid, "where", where, "stable", stable, "verdict", lv.String())
+			return stable, nil
+		default: // BindLive, BindUnknown — genuinely live, or we cannot tell. Unchanged behaviour.
+			return stable, nil
+		}
 	}
 	for i := 0; i < 16 && countHostMounts(stable) > 0; i++ {
 		if err := b.run(ctx, "umount", stable); err != nil {
@@ -250,23 +276,7 @@ func (b *GuestBinder) AttachDrive(ctx context.Context, vmid int, where string) (
 
 // countHostMounts returns how many times `path` appears as a mount target in /proc/self/mountinfo (i.e.
 // how many stacked binds are at it). 0 = not mounted; >1 = stacked duplicates. Used to normalize to one.
-func countHostMounts(path string) int {
-	f, err := os.Open("/proc/self/mountinfo")
-	if err != nil {
-		return 0
-	}
-	defer f.Close()
-	n := 0
-	sc := bufio.NewScanner(f)
-	sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
-	for sc.Scan() {
-		fields := strings.Fields(sc.Text())
-		if len(fields) >= 5 && fields[4] == path {
-			n++
-		}
-	}
-	return n
-}
+func countHostMounts(path string) int { return len(hostMountEntries(path)) }
 
 // GuestSeesMount reports whether vmid's guest currently has `path` as a mount target in ITS mount
 // namespace (read from /proc/<guest-init-pid>/mountinfo). This is the GUEST-side truth the host-side
@@ -278,7 +288,7 @@ func (b *GuestBinder) GuestSeesMount(ctx context.Context, vmid int, path string)
 	if pid == "" {
 		return false
 	}
-	data, err := os.ReadFile("/proc/" + pid + "/mountinfo")
+	data, err := os.ReadFile(procGuestMountinfo(pid))
 	if err != nil {
 		return false
 	}
@@ -391,20 +401,212 @@ func (b *GuestBinder) DetachDrive(ctx context.Context, where string) error {
 // isHostMountpoint reports whether path is currently a mount target in the host's mount table
 // (/proc/self/mountinfo). Pure read — used for idempotency (skip re-binding) and the BoundUnderParent
 // report. A read error → false (treat as not-mounted; AttachDrive then (re)binds, which is safe).
-func isHostMountpoint(path string) bool {
-	f, err := os.Open("/proc/self/mountinfo")
+func isHostMountpoint(path string) bool { return len(hostMountEntries(path)) > 0 }
+
+// procSelfMountinfo is the host mount table every predicate in this file reads. It is a package var
+// ONLY so a test can point the REAL parsers at a captured fixture — production never reassigns it, and a
+// test that does must restore it (t.Cleanup). Injecting the DATA rather than the verdict is what keeps
+// the R-117 tests non-hollow: the parser, the predicate and the /disks handler all run for real.
+var procSelfMountinfo = "/proc/self/mountinfo"
+
+// procGuestMountinfo resolves a guest's init PID to its mount-table path. A package var for the same
+// single reason as procSelfMountinfo: so a test can point the REAL GuestSeesMount at a captured guest
+// mount table. Production never reassigns it.
+var procGuestMountinfo = func(pid string) string { return "/proc/" + pid + "/mountinfo" }
+
+// mountEntry is the parsed subset of a mountinfo line the liveness predicate needs. Field numbers are
+// the kernel's 1-based numbering (proc(5) "/proc/<pid>/mountinfo"): 3 = major:minor, 4 = root within the
+// filesystem, 5 = mount point; after the " - " separator come fstype, source and the per-superblock
+// options. Mount points containing spaces are octal-escaped by the kernel, so strings.Fields is safe.
+type mountEntry struct {
+	// Devno is field 3, the backing device as major:minor. THIS is the field R-117 was lost for want of
+	// reading: it sat in the same parsed slice as the mount point and was discarded.
+	Devno string
+	// Root is field 4 — which subtree of the filesystem is mounted (e.g. /felhom-data for our binds).
+	Root string
+	// FSType is the filesystem driver, needed to know whether SuperOpts' vocabulary is one we can read.
+	FSType string
+	// SuperOpts is the per-superblock option list — where ext4 records that it has stopped serving I/O.
+	SuperOpts string
+}
+
+// hostMountEntries returns every entry in the host mount table whose mount point is `path`. There is
+// more than one when binds are stacked (the double-bind case AttachDrive normalizes). A read error
+// yields nil — callers treat that as "not mounted"/"cannot tell", never as a positive.
+//
+// Pure /proc read, NO BLOCK I/O, per CLAUDE.md's health-check rule: a probe that touches a wedged
+// device enters uninterruptible sleep and survives SIGKILL (measured, R-117 spike §6.3).
+func hostMountEntries(path string) []mountEntry {
+	f, err := os.Open(procSelfMountinfo)
 	if err != nil {
-		return false
+		return nil
 	}
 	defer f.Close()
+	var out []mountEntry
 	sc := bufio.NewScanner(f)
 	sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
 	for sc.Scan() {
-		// mountinfo field 5 (0-indexed 4) is the mount point.
 		fields := strings.Fields(sc.Text())
-		if len(fields) >= 5 && fields[4] == path {
-			return true
+		if len(fields) < 5 || fields[4] != path {
+			continue
+		}
+		e := mountEntry{Devno: fields[2], Root: fields[3]}
+		// The optional-fields run is variable-length; the " - " separator terminates it.
+		for i := 5; i < len(fields); i++ {
+			if fields[i] != "-" {
+				continue
+			}
+			if len(fields) > i+1 {
+				e.FSType = fields[i+1]
+			}
+			if len(fields) > i+3 {
+				e.SuperOpts = fields[i+3]
+			}
+			break
+		}
+		out = append(out, e)
+	}
+	return out
+}
+
+// BindLiveness is the THREE-state answer to "is the bind at the stable path actually usable?".
+//
+// Three states and not a bool, deliberately. The R-117 fix must be able to say "cannot tell", and the
+// cost of getting that wrong is asymmetric: reporting a live drive absent STOPS a customer's apps. The
+// workspace's false-invariant table records `newestArchiveOn` promising "errors degrade to unknown,
+// never to no-backup" over a (value, bool) signature that made it unrepresentable — the comment was a
+// wish. Read every verdict through Usable() and no caller can repeat that.
+type BindLiveness int
+
+const (
+	// BindUnknown — liveness could not be established (unreadable /proc, no raw mount to compare
+	// against, or a filesystem whose abort vocabulary we have not measured). TREATED AS PRESENT by
+	// Usable(), the same rule devicePresent applies to an empty path (disks.go).
+	BindUnknown BindLiveness = iota
+	// BindLive — the bind names the same device as the raw mount and its filesystem has not aborted.
+	BindLive
+	// BindStaleDevice — R-117 case (a), the detach/return case. The bind still references the superblock
+	// of the drive that went away, while the raw mount has healed onto the returning device via its
+	// fs-UUID-keyed unit. Every access through the bind fails. RE-BINDING REPAIRS THIS.
+	BindStaleDevice
+	// BindAborted — R-117 case (b), the Q7 steady-state case. The filesystem under the bind has given up:
+	// ext4 sets `shutdown` when its device vanished, `emergency_ro` when errors=remount-ro fired in place.
+	// The raw mount is the SAME aborted superblock, so RE-BINDING CANNOT REPAIR THIS — it must surface as
+	// not-live so the drive gate stops the apps and alarms. See AttachDrive's switch.
+	BindAborted
+)
+
+// Usable is the ONLY sanctioned way to turn a verdict into a yes/no, so the unknown-is-present rule
+// lives in exactly one place. Pinned by TestBindLiveness_UnknownIsTreatedAsPresent.
+func (l BindLiveness) Usable() bool { return l == BindLive || l == BindUnknown }
+
+func (l BindLiveness) String() string {
+	switch l {
+	case BindLive:
+		return "live"
+	case BindStaleDevice:
+		return "stale-device"
+	case BindAborted:
+		return "filesystem-aborted"
+	default:
+		return "unknown"
+	}
+}
+
+// abortTokensByFS maps a filesystem driver to the per-superblock option tokens it sets when it has
+// stopped serving I/O. BOTH ext4 tokens are load-bearing and BOTH were measured (R-117 spike §4):
+// `shutdown` when the device was removed, `emergency_ro` when errors=remount-ro fired with the device
+// still present. A check for only `shutdown` passes the ENTIRE Q7 state, which is the silent half.
+//
+// ext2/ext3 are served by the ext4 driver on this kernel, so they emit the same tokens. Anything else is
+// a customer-supplied filesystem whose vocabulary we have not measured — it yields UNKNOWN, never LIVE
+// (the agent itself only ever formats ext4).
+var abortTokensByFS = map[string][]string{
+	"ext4": {"shutdown", "emergency_ro"},
+	"ext3": {"shutdown", "emergency_ro"},
+	"ext2": {"shutdown", "emergency_ro"},
+}
+
+// fsAborted reports whether the entry's filesystem has aborted, and whether we could tell at all.
+// `known` false means the fstype is not in abortTokensByFS — the caller must degrade to BindUnknown
+// rather than infer health from the absence of a token it does not know how to look for.
+func fsAborted(e mountEntry) (aborted, known bool) {
+	toks, ok := abortTokensByFS[e.FSType]
+	if !ok {
+		return false, false
+	}
+	for _, opt := range strings.Split(e.SuperOpts, ",") {
+		for _, t := range toks {
+			if opt == t {
+				return true, true
+			}
 		}
 	}
-	return false
+	return false, true
+}
+
+// bindLiveness is the R-117 predicate: does the bind at `stable` actually work? `raw` is the drive's RAW
+// host mount (/mnt/<name>). Reads /proc only — NO block I/O, per CLAUDE.md's health-check rule.
+//
+// WHY THIS EXISTS. GuestSeesMount and isHostMountpoint both compare only field 5 (the mount point) of a
+// mountinfo line, so both answer "does a mount by that name exist" and neither can see that the bind and
+// the raw mount name DIFFERENT devices. Measured live: raw on 8:32 /dev/sdc while the bind read
+// 8:16 /dev/sdb with `shutdown`, BoundUnderParent true, EIO on every read and write, and the gate
+// restarting the customer's apps onto it (R-117 spike §5.2).
+//
+// HOST-SIDE ONLY, deliberately: the host bind and the guest's view of it are the same mount in one
+// propagation peer group and carry identical devno and super options (measured, spike §5.2), so this
+// needs no lxc-info fork. Guest VISIBILITY is a different question and stays with GuestSeesMount.
+func bindLiveness(stable, raw string) BindLiveness {
+	if stable == "" || raw == "" {
+		return BindUnknown // nothing to compare — never claim absent
+	}
+	binds := hostMountEntries(stable)
+	if len(binds) == 0 {
+		return BindUnknown // nothing bound here; that is isHostMountpoint's question, not this one
+	}
+	rawEntries := hostMountEntries(raw)
+	if len(rawEntries) == 0 {
+		return BindUnknown // the raw mount is gone — devicePresent already reports that as absent
+	}
+	// The two cases are distinguished by WHETHER THE DEVICES AGREE, and the abort flag is read off a
+	// DIFFERENT entry in each. Getting this backwards is a live trap, caught here by
+	// TestBindLiveness_Verdicts: in the real return state the stale bind carries `shutdown` AND names a
+	// different device, so an abort-first rule classifies it BindAborted — which reports correctly but
+	// refuses the re-bind that actually repairs it. The question a verdict must answer for AttachDrive is
+	// not "is something aborted" but "would a re-bind help".
+	rawEntry := rawEntries[0]
+	for _, b := range binds {
+		if b.Devno != rawEntry.Devno {
+			// P1 — case (a). The bind references a superblock that is NOT the one the raw mount now has:
+			// the drive went away and came back, and the raw mount healed onto it via its fs-UUID-keyed
+			// unit. Sound rather than heuristic — a stale bind pins the dead superblock, which keeps the
+			// old device index allocated, which FORCES the returning device onto a different number
+			// (measured both ways, including the control test where releasing the bind let the letter be
+			// reused, spike §3.4).
+			//
+			// Whether a re-bind repairs it depends on the RAW mount, which is what a re-bind would point
+			// at — not on the stale bind's own abort flag.
+			if aborted, known := fsAborted(rawEntry); known && aborted {
+				return BindAborted // re-binding would land on another dead filesystem
+			}
+			return BindStaleDevice // re-binding lands on the healthy returning device: repairable
+		}
+	}
+	// Same superblock on both sides, so a re-bind is a no-op by construction. Only the filesystem's own
+	// abort state can tell us anything — and this is R-117's steady-state half (spike §9), where the
+	// device NEVER LEFT so the devnos above agree and P1 alone reads healthy.
+	for _, b := range binds {
+		if aborted, known := fsAborted(b); known && aborted {
+			return BindAborted
+		}
+	}
+	// Devices agree and nothing aborted. If we cannot read this filesystem's abort vocabulary we must not
+	// call it live — say unknown, which Usable() treats as present.
+	for _, b := range binds {
+		if _, known := fsAborted(b); !known {
+			return BindUnknown
+		}
+	}
+	return BindLive
 }

internal/localapi/intermediary_attach_liveness_test.go

@@ -0,0 +1,187 @@
+package localapi
+
+import (
+	"context"
+	"io"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// R-117 §2.2 — WHAT AttachDrive DOES with each verdict, which is a RULING and not a detail.
+//
+// The two dead states have DIFFERENT repairs, and the existing self-heal only fits one:
+//
+//   - STALE DEVICE (the detach/return case). The raw mount has healed onto the returning device via its
+//     fs-UUID-keyed unit, so umount + re-bind lands the namespace on a HEALTHY superblock. Repair is
+//     correct, and it happens live with no guest restart (proven on hardware, spike §8.1). It MUST run:
+//     three call sites already invoke it — the 20 s reconcile ticker, agent startup, and the controller's
+//     Return branch BEFORE it restarts the apps (spike §8.2) — and before v0.117.0 all three were
+//     short-circuited by `if n == 1 && GuestSeesMount(...)` declaring the dead namespace "fully live".
+//
+//   - ABORTED FILESYSTEM (the steady-state case, R-117a). The raw mount is the SAME aborted superblock,
+//     so a re-bind produces a fresh bind to a still-dead filesystem. It MUST NOT run: AttachDrive is
+//     called every 20 s, so re-binding here is an infinite silent retry — the exact silence R-117a
+//     found, with more CPU — and it would mask the state instead of surfacing it. The truth travels in
+//     BoundUnderParent (now false), so the drive gate stops the apps and alarms. Clearing an aborted
+//     filesystem needs a remount or a fsck; that is an operator decision, never an automatic one.
+//
+// These tests assert the CONSEQUENCE — which privileged commands were issued — not that a verdict was
+// computed. RED-PROOF: make the BindAborted arm fall through to the re-bind instead of returning, and
+// TestAttachDrive_AbortedFilesystem_DoesNotRebind fails on the recorded umount/mount calls.
+
+// attachRecorder is a proxmox.Runner that records every privileged call and answers `lxc-info -p` with a
+// fixed PID, so the REAL GuestSeesMount runs against a captured guest mount table.
+type attachRecorder struct {
+	calls [][]string
+	pid   string
+}
+
+func (r *attachRecorder) Run(_ context.Context, name string, args ...string) ([]byte, []byte, error) {
+	r.calls = append(r.calls, append([]string{name}, args...))
+	if name == "lxc-info" {
+		return []byte(r.pid + "\n"), nil, nil
+	}
+	return nil, nil, nil
+}
+
+func (r *attachRecorder) RunStdin(ctx context.Context, _ io.Reader, name string, args ...string) ([]byte, []byte, error) {
+	return r.Run(ctx, name, args...)
+}
+
+// mountOps returns just the mount-table-mutating calls — the ones that constitute "a repair ran".
+func (r *attachRecorder) mountOps() []string {
+	var out []string
+	for _, c := range r.calls {
+		switch c[0] {
+		case "umount", "mount":
+			out = append(out, strings.Join(c, " "))
+		}
+	}
+	return out
+}
+
+// attachFixture points BOTH mount tables at fixtures: the host one (procSelfMountinfo, which
+// countHostMounts and bindLiveness read) and the guest one (procGuestMountinfo, which the REAL
+// GuestSeesMount reads). Only the data is injected — every predicate runs for real.
+func attachFixture(t *testing.T, hostBody, guestBody string) *attachRecorder {
+	t.Helper()
+	dir := t.TempDir()
+	hp := filepath.Join(dir, "host-mountinfo")
+	gp := filepath.Join(dir, "guest-mountinfo")
+	if err := os.WriteFile(hp, []byte(hostBody), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(gp, []byte(guestBody), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	prevHost, prevGuest := procSelfMountinfo, procGuestMountinfo
+	procSelfMountinfo = hp
+	procGuestMountinfo = func(string) string { return gp }
+	t.Cleanup(func() { procSelfMountinfo, procGuestMountinfo = prevHost, prevGuest })
+	return &attachRecorder{pid: "9301"}
+}
+
+// guestSeesStale / guestSeesHealthy / guestSeesAborted are the GUEST-side captures — note `master:450`,
+// the slave-of-the-shared-parent tag that proves propagation was wired (spike §3.1). The guest carries the
+// same devno and super options as the host bind, because it IS the same mount.
+const guestSeesStale = `759 1176 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:459 master:450 - ext4 /dev/sdb rw,stripe=512,shutdown
+`
+const guestSeesHealthy = `759 1176 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:459 master:450 - ext4 /dev/sdb rw,stripe=512
+`
+const guestSeesAborted = `759 1176 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:459 master:450 - ext4 /dev/sdb rw,stripe=512,emergency_ro
+`
+
+func attachBinder(rec *attachRecorder) *GuestBinder {
+	return NewGuestBinder(rec, slog.New(slog.NewTextHandler(io.Discard, nil)))
+}
+
+// TestAttachDrive_StaleBind_Rebinds is Q6's consequence: the repair that was short-circuited for three
+// releases now runs. Exactly the state measured on hardware — bind on 8:16, raw healed onto 8:32.
+func TestAttachDrive_StaleBind_Rebinds(t *testing.T) {
+	rec := attachFixture(t, mountinfoStaleBind, guestSeesStale)
+	b := attachBinder(rec)
+
+	got, err := b.AttachDrive(context.Background(), 9301, "/mnt/r117sd")
+	if err != nil {
+		t.Fatalf("AttachDrive: %v", err)
+	}
+	if got != "/mnt/felhom-drives/r117sd" {
+		t.Errorf("stable path = %q", got)
+	}
+	ops := rec.mountOps()
+	if len(ops) == 0 {
+		t.Fatal("NO repair ran over a stale bind — this is the R-117 short-circuit: `n == 1 && " +
+			"GuestSeesMount` declared an EIO namespace \"fully live\", so the 20 s ticker, agent startup " +
+			"and the controller's pre-restart re-attach all did nothing")
+	}
+	var sawUmount, sawBind bool
+	for _, o := range ops {
+		if strings.HasPrefix(o, "umount /mnt/felhom-drives/r117sd") {
+			sawUmount = true
+		}
+		if o == "mount --bind /mnt/r117sd/felhom-data /mnt/felhom-drives/r117sd" {
+			sawBind = true
+		}
+	}
+	if !sawUmount || !sawBind {
+		t.Errorf("repair did not umount-then-rebind; ops=%v", ops)
+	}
+}
+
+// TestAttachDrive_AbortedFilesystem_DoesNotRebind is the §2.2 ruling. A re-bind here cannot repair
+// anything (the raw mount is the same aborted superblock) and AttachDrive runs every 20 s, so re-binding
+// would be an infinite silent retry that also masks the state.
+func TestAttachDrive_AbortedFilesystem_DoesNotRebind(t *testing.T) {
+	rec := attachFixture(t, mountinfoAborted, guestSeesAborted)
+	b := attachBinder(rec)
+
+	got, err := b.AttachDrive(context.Background(), 9301, "/mnt/r117sd")
+	if err != nil {
+		t.Fatalf("AttachDrive returned an error for an aborted filesystem: %v — it must be a quiet no-op; "+
+			"an error here would log `reconcile: AttachDrive failed` every 20 s", err)
+	}
+	if got != "/mnt/felhom-drives/r117sd" {
+		t.Errorf("stable path = %q", got)
+	}
+	if ops := rec.mountOps(); len(ops) != 0 {
+		t.Errorf("AttachDrive re-bound an ABORTED filesystem: %v\n"+
+			"A re-bind lands on the SAME dead superblock, and this runs every 20 s — an infinite silent "+
+			"retry, which is R-117a's silence with more CPU. The state must SURFACE via "+
+			"BoundUnderParent=false so the gate stops the apps and alarms.", ops)
+	}
+}
+
+// TestAttachDrive_Healthy_IsStillANoOp — the idempotency the reconcile ticker depends on. If this
+// regressed, every tick would umount and re-bind a working drive, re-firing propagation into the guest
+// 4320 times a day.
+func TestAttachDrive_Healthy_IsStillANoOp(t *testing.T) {
+	rec := attachFixture(t, mountinfoHealthy, guestSeesHealthy)
+	b := attachBinder(rec)
+
+	if _, err := b.AttachDrive(context.Background(), 9301, "/mnt/r117sd"); err != nil {
+		t.Fatalf("AttachDrive: %v", err)
+	}
+	if ops := rec.mountOps(); len(ops) != 0 {
+		t.Errorf("a healthy bind was disturbed: %v — the 20 s reconcile must stay a no-op", ops)
+	}
+}
+
+// TestAttachDrive_UnknownLiveness_IsANoOp — cannot-tell must not trigger churn either. An unreadable
+// mount table making the agent umount and re-bind every 20 s would be a self-inflicted outage.
+func TestAttachDrive_UnknownLiveness_IsANoOp(t *testing.T) {
+	// Host table healthy (so n == 1) but on a filesystem whose abort vocabulary we cannot read.
+	rec := attachFixture(t, mountinfoUnknownFS,
+		`759 1176 8:16 /felhom-data /mnt/felhom-drives/r117sd rw,relatime shared:459 master:450 - btrfs /dev/sdb rw
+`)
+	b := attachBinder(rec)
+
+	if _, err := b.AttachDrive(context.Background(), 9301, "/mnt/r117sd"); err != nil {
+		t.Fatalf("AttachDrive: %v", err)
+	}
+	if ops := rec.mountOps(); len(ops) != 0 {
+		t.Errorf("an UNKNOWN verdict caused a re-bind: %v — cannot-tell must never churn a live mount", ops)
+	}
+}

internal/localapi/intermediary_test.go

@@ -73,6 +73,11 @@ func TestDisks_GuestPathAndBoundUnderParent(t *testing.T) {
 	srv.baseCtx = context.Background()
 	// felhom-usb is bound under the parent; felhom-flash is not.
 	srv.boundCheck = func(p string) bool { return p == "/mnt/felhom-drives/felhom-usb" }
+	// R-113 (v0.114.0): BoundUnderParent is now `bound && device present`. This test's subject is the
+	// BIND half, so hold the device half constant at present — otherwise the fixture would be asserting
+	// a drive that is bound with no raw mount underneath it, which is the absent state, not this test's
+	// case. Device presence has its own tests (TestDisks_DevicePresence*).
+	srv.deviceCheck = func(string) bool { return true }
 
 	disks := decodeDisks(t, do(t, srv.Handler(), "GET", "/disks", "A", "").Body.Bytes())
 	byMount := map[string]DiskInfo{}

internal/localapi/netstorage.go

@@ -41,12 +41,12 @@ type netStorageAddRequest struct {
 	Name           string `json:"name"`
 	Protocol       string `json:"protocol"` // "nfs" | "smb"
 	Server         string `json:"server"`
-	Export         string `json:"export"`              // NFS export path | SMB share name
-	MappedUID      int    `json:"mapped_uid"`          // container uid (e.g. 1000)
-	MappedGID      int    `json:"mapped_gid"`          // container gid
-	IdleTimeoutSec int    `json:"idle_timeout_sec"`    // automount idle-unmount window; 0 → default
-	Username       string `json:"username,omitempty"`  // SMB only (secret — written to creds file)
-	Password       string `json:"password,omitempty"`  // SMB only (secret — written to creds file)
+	Export         string `json:"export"`             // NFS export path | SMB share name
+	MappedUID      int    `json:"mapped_uid"`         // container uid (e.g. 1000)
+	MappedGID      int    `json:"mapped_gid"`         // container gid
+	IdleTimeoutSec int    `json:"idle_timeout_sec"`   // automount idle-unmount window; 0 → default
+	Username       string `json:"username,omitempty"` // SMB only (secret — written to creds file)
+	Password       string `json:"password,omitempty"` // SMB only (secret — written to creds file)
 }
 
 // handleNetStorageAdd installs a NAS share host-side — verify-before-commit (SPIKE-nas-verify).