v0.124.1: the repair record must survive the probe that did NOT feed the hub (R-190)

v0.124.0's transition record never reached the hub, and only the live run showed it. The capability reported degraded for "one cycle" — the probe call that did the repair. But probeAll is invoked independently by the self-check log and by the collector building a host report. On the demo box the repairing call was the log's (09:39:34, journal shows the self-repair and degraded=1) and the report three seconds later found the grant present and sent ok. The agent's journal had the record; the hub had nothing. That is the silence R-190 is about, re-created inside its own mitigation, with every unit test green. Fixed with a latch on TIME, not call count: a confirmed repair reports for 20 minutes, which exceeds the 900s report interval, so at least one report must carry it. It clears on its own and is per tier. Two hollow tests caught and fixed on the way — one asserting a value it built itself, one asserting the latch helper rather than the path consuming it (its red-proof duly passed). The decisions now live in storeGrantHealthyVerdict and storeGrantRepairedVerdict and the tests call those.
v0.124.0: a lost storage grant repairs itself, and says that it was lost (R-190)
2026-08-04 09:44:56 +02:00 · 2026-08-04 09:38:27 +02:00 · 2026-08-03 19:04:44 +02:00 · 2026-08-03 19:04:15 +02:00 · 2026-08-03 18:53:47 +02:00 · 2026-08-03 16:58:35 +02:00
19 changed files with 1970 additions and 267 deletions
@@ -1,3 +1,194 @@
 ## v0.124.1 — the repair record must survive the probe that did NOT feed the hub (2026-08-04, R-190)
 **v0.124.0's transition record did not reach the hub, and the live run is what showed it.** The
 capability reported degraded for "one cycle" — meaning the probe call that performed the repair. But
 `probeAll` is invoked **independently** by the periodic self-check log and by the collector building a
 host-report. On the demo box the repairing call was the log's (`09:39:34`, journal shows
 `GRANT WAS MISSING AND HAS BEEN SELF-REPAIRED` and `degraded=1`), and the host-report built three
 seconds later found the grant present and sent **`ok`**. The agent's journal had the record; the hub
 had nothing; the operator would have learned nothing.
 That is the exact silence R-190 is about, re-created inside its own mitigation — and every unit test
 passed while it was true.
 **The fix is a latch on TIME rather than on call count.** A confirmed repair is reported for
 `storeGrantRepairReportWindow` (20 minutes), which comfortably exceeds the 900 s host-report interval,
 so at least one report must carry the transition. It clears on its own — a permanently degraded
 capability would be its own false alarm — and it is per tier.
 **Two hollow tests were caught and fixed on the way**, both the same shape this repo keeps finding: a
 test asserting a value it constructed itself, and a test asserting the latch HELPER rather than the
 path that consumes it — whose red-proof duly passed. The decisions now live in
 `storeGrantHealthyVerdict` and `storeGrantRepairedVerdict`, and the tests call those.
 ## v0.124.0 — a lost storage grant repairs itself, and says that it was lost (2026-08-04, R-190)
 **R-190 is a grant that demonstrably worked at 04:44 on 2026-08-03 and was gone by 09:24** — with a
 host reinstall, logged `pveum` activity and cluster-log entries all ruled out by measurement. The
 cause is still open. The resilience does not have to wait for it.
 **Everything needed already existed and had only ever been called once.** The root wrapper
 (`felhom-backup-target-apply grant <id>`), its sudoers vector (`grant *`, any storage id, confirmed
 not assumed), and the exact command were all in place — and the `grant` verb had only ever run at
 storage CREATION. That is the *built but never wired* shape, in a verb rather than a seam, and it is
 this project's seventh instance.
 **What v0.124.0 does:** when the store-grant probe finds the grant absent on a tier the box depends
 on, it runs that wrapper and **re-reads once** to confirm — the pbsdr R-22 self-grant shape, including
 its restraint: one attempt, one confirmation, and anything still wrong stays loudly wrong.
 **THE RECORD IS THE POINT, AND IT IS THE HALF R-190 IS ACTUALLY ABOUT.** A repair that leaves only
 `ok` behind destroys the only evidence a permission vanished, so a recurring loss becomes undetectable
 forever — strictly worse than the fault it fixes. So a confirmed repair reports **DEGRADED for exactly
 one cycle**, with the explanation in `Feature`:
 ```
 backup tier felhom-backup: the agent's storage grant was MISSING and has been AUTOMATICALLY
 RESTORED — the tier works now, but a permission that vanished on its own needs investigating (R-190)
 ```
 **Nothing new was built to carry it.** The hub's existing ok→degraded→ok edge is the channel — it
 alerts and e-mails on the first edge and logs the recovery on the next cycle, so one loss produces
 exactly one alert pair. No wire change, no hub change, no new event type. `Feature` carries the text
 because that is the field the hub interpolates into the operator's e-mail; `Reason` does not travel.
 **Bounded (Scenario F):** one attempt per tier per hour, in memory. A storage can be unreadable for
 reasons an ACL cannot fix, and a re-grant on every report cycle is a repair loop wearing a fix's
 clothes. An agent restart re-arms it, which is correct — a restart is exactly when a box should
 re-check what it depends on.
 **A failed repair never masks the fault:** the capability stays degraded with the failure in its
 reason, and a repair that "succeeded" but did not survive the re-read is reported as needing a human.
 ## v0.123.0 — a tier the box cannot READ now says so (2026-08-03, R-185)
 **The missing permission is one command. The silence was the defect.** On demo-felhom the agent's PVE
 token held `FelhomAgentStore` on `local`, `local-lvm` and `felhom-pbs` — and **not** on
 `felhom-backup`, the storage the same installer had configured as `local_backup_target`. Asked for
 that storage's content the API answers `{"data":[]}` while root sees three archives (6.1–6.3 GB,
 08-01/02/03).
 **An empty listing is what a FORBIDDEN tier and a NEWBORN tier both return.** `pickForThisRun` skips
 an empty tier — correctly, because a fresh offsite tier legitimately has nothing yet — and reports
 *"no settled archive yet"*. So that tier was never restore-testable on that box and nothing ever
 mentioned it. This project's own rule, in a new place: an empty answer is not evidence that there is
 nothing there.
 **The permission question, unlike the listing, has a definite answer — so it is asked directly.**
 `Client.Permissions` reads `GET /access/permissions?path=/storage/<target>` **as the agent's own
 token** (asking as root answers a different question and always says yes), and one
 `capability.Status` per configured tier reports the result. It composes *around* the sudo prober, the
 way the pool-read check already does — an API read does not belong inside a sudo-policy probe.
 **MEASURED FIRST, and the obvious reading is wrong.** The ungranted path does not answer empty and
 does not 403:
 ```
 /storage/felhom-pbs     → {"Datastore.Allocate":1,"Datastore.AllocateSpace":1}
 /storage/felhom-backup  → {"Sys.Audit":1,"SDN.Use":1,"Datastore.Audit":1}
 ```
 It answers with the privileges **inherited** from the box-wide `/` grant. A probe asking *"is the
 path present?"* or *"does it have `Datastore.Audit`?"* would report the blinded storage healthy — so
 the probe tests for `Datastore.AllocateSpace` specifically, and a red-proof pins that.
 **Decisions, each weighed once:**
 - **The probed set comes from the box's own config** (`BackupTiers()`), not a fixed list. A hardcoded
  probe list is exactly the defect being fixed, reproduced inside the fix.
 - **CRITICAL**, because the hub alerts only on critical and a non-critical entry would ride the
  report and alert nobody — the same silence with extra steps. **Except** the `local` fallback
  target, which host-install's own comment calls the DEGRADED configuration: it is still probed and
  still reported, but it does not page, because turning an ordinary documented setup into an alert
  is how a signal becomes something an operator archives unread.
 - **It never looks at content**, so it cannot alarm on a newborn tier by construction.
 - **It never reports ok when it could not ask.** An unreachable PVE is degraded: a self-check that
  fails open converts *"I do not know"* into *"fine"*.
 The wire shape (`capability.Status`) is unchanged, so the hub's existing critical-degraded alert
 applies with no hub change and no hub bump.
 ## v0.122.0 — three ways the signals lied about themselves (2026-08-03, R-189 · R-188 · R-186)
 All three are the reporting and release path misreporting its own work. **No customer machine, no
 backup, no restore, no disk layout, no data.** The restore-test itself and when it runs are unchanged
 from v0.121.1.
 ### R-189 — a passing restore-test no longer vanishes on a restart
 `restore_tests[]` came only from the in-memory `backup.Store`, whose own comment read *"lost on
 restart; the cadence re-populates"*. That was true under a timer. It stopped being true when R-86 made
 the agent refuse to re-test an archive it has already proven: a proof lost to a restart is not
 repeated for a whole archive generation — **a week on the offsite tier** — and the hub calls the tier
 unproven for all of it.
 **Observed, not predicted (2026-08-03):** a real 14.5 GB offsite restore-test PASSED at 15:25:14, the
 agent was restarted 2 m 43 s later for a deploy, and the hub logged `0 restore-tests` on the next two
 host-reports.
 The durable proof already existed — `RestoreTestState`, on disk, per tier, with the archive since
 R-86 — and `Snapshot()` had carried the doc comment *"for the host-report gauge"* since the day it was
 written **with no caller at all**: a seam built, documented, and never connected. It now carries the
 `tier` and what was `verified` as well (stored at proof time, when they are known for certain, rather
 than derived later by a storage lookup that can fail), and `ProvenRestoreTests` renders them as report
 entries which the collector merges.
 - **Merge rule: one entry per tier, newest by `TestedAt` wins.** A fresh failure beats a stored
  success — the failure is the news and lives nowhere else; a stored success beats a stale in-memory
  entry after a restart; a tier never appears twice, which the hub would read as two tests. An
  unparseable timestamp counts as older, so a malformed entry cannot displace a good one.
 - **It refuses to lie.** A record missing the archive or the tier produces NO entry, and run mechanics
  (scratch VMID, duration) are not re-invented — an absent duration is not a claim, a fabricated one
  would be. An unproven tier reading as proven would be worse than the defect being fixed.
 - **Only successes are persisted, and that asymmetry is now written down where it will be read:** a
  success suppresses future work, so losing it leaves the system quietly less tested than it believes;
  a failure causes future work and heals itself at the next evaluation.
 - The `Store` comment that stopped being true is corrected in place rather than left to mislead.
 ### R-188 — a correct release no longer emails a failure
 `on: [push]` fires the gates workflow on the **tag** push, and the release pushed its tag *before*
 publishing, so CI ran the published-versions gate in the seconds before the package existed and
 correctly reported it missing. Measured across two releases in one session: runs 12/13 and 17/18, same
 sha each time, opposite results — a race, not a rule. R-168 made that mail the thing that cannot be
 missed; one that is wrong half the time is one you stop reading.
 **Only the tag PUSH moved** (build → tag locally → publish → push tag). The tag is still created before
 anything is published, so the build and the tag still describe the same commit; it simply becomes
 *visible* — to CI, and to any `raw/tag/…` fetch — once the package is downloadable.
 The invariant the old order protected is **not traded away**: `check-published-versions.py` now asserts
 the converse directly — **no published version may be missing its tag** — as a bounded probe of the
 frontier (where a failed tag push leaves an orphan) and of patch gaps, printing its probe set every
 run because a check whose coverage is invisible reads as a guarantee it is not making. The package
 listing api still answers **401** without a token (re-measured), so absence cannot be enumerated, and
 the script says so.
 A publish that succeeds and a tag push that then fails now **dies loudly**, printing the one-line
 recovery; and a publish that *fails* removes the local-only tag so the release can simply be retried
 instead of colliding with itself.
 ### R-186 — a released binary can now be verified by rebuilding it
 `go build` stamps a module version derived from VCS state, so a build made before the tag existed and
 a rebuild made after it were different binaries. Measured at one commit, same source, same toolchain:
 ```
 default flags, no tag yet ... 18f4a495…  14 085 464 B   (mod v0.121.2-0.2026…-3d0a1d61)
 default flags, tagged ....... 4a38f394…  14 085 440 B   (mod v0.121.99)
 -trimpath -buildvcs=false ... 7ffcdf1d…  14 064 574 B   IDENTICAL both ways
 ```
 The stamp is removed rather than sequenced around — nothing in this repo reads it (no `ReadBuildInfo`
 caller) and the version comes from the explicit `-X main.version` ldflag. `-trimpath` additionally
 makes a rebuild from a different checkout directory match.
 **A second discrepancy fell out of it:** `publish-agent.sh`'s fallback build forced `CGO_ENABLED=0` and
 therefore produced a binary **74 KB smaller** than the release path built for the same version — one
 version name, two binaries, decided by which entry point was used. Both now build identically.
 `CLAUDE.md` records the exact command an operator can run to verify a published binary independently.
 ## v0.121.1 — "nothing is due" must be AUDIBLE (2026-08-03, R-86 + standing rule 3)
 **Found while live-validating v0.121.0, and it is this project's own rule pointed at the change that
@@ -72,6 +72,32 @@ internal/storage/    storage observer + durable ids + role/claim classifiers + S
 > verifies by an **independent download** rather than trusting the publish step's own output.
 > `scripts/publish-agent.sh` still exists and is still correct — the release script CALLS it rather
 > than reimplementing it.
 >
 > **THE ORDER IS build → tag LOCALLY → publish → push tag, and each step protects something (R-188,
 > R-186).** The tag is created before the publish so the build and the tag describe the same commit;
 > it is *pushed* after, because the push is what wakes CI (`on: [push]`) and a tag visible before its
 > package makes the published-versions gate correctly fail a correct release — it did, on roughly
 > every second release, and R-168 sends that failure to you by mail. The invariant the old order
 > protected is asserted directly instead: the gate now also refuses a **published version with no
 > tag**. If the push fails after a successful publish the script says so loudly and prints the
 > one-line recovery; if the *publish* fails it removes the local-only tag so a retry is clean.
 >
 > **A RELEASED BINARY IS INDEPENDENTLY VERIFIABLE (R-186).** The build uses `-trimpath
 > -buildvcs=false` so the same source produces the same bytes whether or not the tag exists yet —
 > before this, a rebuild could not reproduce the sha you were vouching. To check any published
 > version yourself:
 >
 > ```bash
 > V=0.122.0
 > git checkout "v$V" && go build -trimpath -buildvcs=false -ldflags "-X main.version=$V" \
 >     -o /tmp/felhom-agent-check ./cmd/felhom-agent
 > sha256sum /tmp/felhom-agent-check
 > curl -fsSL "https://gitea.dooplex.hu/api/packages/admin/generic/felhom-agent/$V/felhom-agent" | sha256sum
 > ```
 >
 > The two hashes must match. `publish-agent.sh`'s fallback build uses the **same** flags — it used to
 > force `CGO_ENABLED=0` and produce a 74 KB-smaller binary for the same version; if either build line
 > ever changes, change both or one version name means two binaries again.
 | Step | Where | One-liner |
 |---|---|---|
@@ -79,6 +105,7 @@ internal/storage/    storage observer + durable ids + role/claim classifiers + S
 | Copy | local → felhom-pve | `scp /tmp/felhom-agent-<v> felhom-pve:/tmp/` (one hop) |
 | Deploy | felhom-pve | backup `.bak-<old>` → `install -m0755` → `systemctl restart felhom-agent` (non-root `felhom-agent` user, config `/etc/felhom-agent/agent.json`) |
 | Ship configs | felhom-pve | sudoers (`/etc/sudoers.d/felhom-agent`) + guarded-mkfs wrapper WITH the binary when `configs/` changed |
 | **Verify** (anyone, any time) | anywhere with the repo + Go | `git checkout v<ver> && go build -trimpath -buildvcs=false -ldflags "-X main.version=<ver>" -o /tmp/a ./cmd/felhom-agent && sha256sum /tmp/a` — must equal `curl -fsSL <pkg-url> \| sha256sum` |
 | **Vouch** | hub operator UI | Configs → Day-0 artifacts. **Deliberately NOT automated** — vouching is what points machines at a version, and it stays your act (prove-then-vouch) |
 | Verify | felhom-pve | `felhom-agent --version` + journal (clean ReassertGuestBinds, no capability degradation) |
@@ -5,6 +5,59 @@
 ## Current
 - **2026-08-03 — v0.123.0 (R-185): a tier the box cannot READ now says so.** The agent's token had
  `FelhomAgentStore` on `local`, `local-lvm`, `felhom-pbs` and **not** on `felhom-backup` — the
  storage both demo boxes configure as `local_backup_target`. That storage answered `{"data":[]}`
  through the token while root listed three archives, and `pickForThisRun` skipped it as *"no settled
  archive yet"* — **which is what a brand-new tier reports**, so the host tier was never
  restore-testable and nothing said so.
  - **The permission question is asked directly**, because unlike the listing it has a definite
    answer: `Client.Permissions` reads `/access/permissions?path=/storage/<target>` **as the agent's
    own token**, and `storeGrantStatuses` emits one `capability.Status` per configured tier. It
    composes AROUND the sudo prober, the way `poolReadStatus` already does — an API read does not
    belong inside a sudo-policy probe. `Status`'s wire shape is untouched, so the hub's critical
    degraded alert applies with **no hub change**.
  - **MEASURED FIRST, and the obvious reading is wrong:** an ungranted path answers neither empty nor
    403 — it carries the privileges INHERITED from the box-wide `/` grant
    (`Sys.Audit, SDN.Use, Datastore.Audit`). Checking path-presence, or `Datastore.Audit`, reports a
    blinded storage HEALTHY. The probe tests **`Datastore.AllocateSpace`**; re-measure before ever
    changing that constant (`storeGrantRequiredPriv`, red-proved).
  - **The probed set comes from `BackupTiers()`, never a fixed list** — a hardcoded probe list is the
    defect reproduced inside the fix. Critical, EXCEPT the `local` fallback target (reported, but it
    does not page). It never consults content, so it cannot alarm on a newborn tier; it never reports
    ok when it could not ask.
  - **LIVE:** degraded observed on the still-blind box (hub emailed `agent_capability_degraded`) →
    grant applied on **both** demo boxes → token lists 3 and 4 archives → `ok=70 total=70 degraded=0`
    and `degraded → ok` at the hub → **the host tier became a due-check candidate for the first time**,
    correctly picking the 08-02 archive (08-03 had not settled 24 h).
  - **The installer's real defect was NOT `PVE_STORAGES`** — see `felhom.eu` CONTEXT S-22: Case A
    grants, the Scenario-F reuse arm did not. Fixed in installer **1.24.0** with a gate.
 - **2026-08-03 — v0.122.0 (R-189 · R-188 · R-186): three signals that lied about their own work.**
  None touches data; all three cost attention, which every other signal depends on.
  - **R-189 — a passing restore-test no longer vanishes on a restart.** `restore_tests[]` came only
    from the in-memory `backup.Store` (*"lost on restart; the cadence re-populates"* — true under a
    timer, FALSE since R-86, because the agent will not re-test a proven archive). **Observed live:**
    a 14.5 GB offsite PASS at 15:25:14, agent restarted 2 m 43 s later, hub logged `0 restore-tests`
    twice. `RestoreTestState` now stores `tier` + `verified` beside the archive (v3 shape; v1/v2
    still read, and a record missing archive-or-tier is NOT reported), exposes
    `ProvenRestoreTests`, and `Collector.SetProvenRestoreTests` merges it — **one entry per tier,
    newest by `TestedAt` wins**, so a fresh failure beats a stored success and a tier never appears
    twice. Wiring pinned by an AST test: the method this replaces (`Snapshot`) claimed a
    "host-report gauge" in its doc comment and had **no caller** for weeks.
  - **ONLY SUCCESSES ARE PERSISTED, and the reason is now in the code:** a success *suppresses*
    future work (a proven archive is never re-tested, so a lost proof leaves the box quietly less
    tested than it believes); a failure *causes* future work and heals itself at the next evaluation.
  - **R-188 — the release stopped emailing false failures.** Only the tag PUSH moved (build → tag
    locally → publish → push tag): the push is what wakes CI, and a tag visible before its package
    made the gate correctly fail a correct release ~half the time. The old order's invariant is now
    asserted directly — `check-published-versions.py` refuses a **published version with no tag**, as
    a bounded, printed probe (the package listing api is still 401 without a token, re-measured).
  - **R-186 — a released binary is verifiable.** `-trimpath -buildvcs=false`: same source → same
    bytes whether or not the tag exists. Measured. `publish-agent.sh`'s fallback also forced
    `CGO_ENABLED=0` and built a **74 KB different** binary for the same version — both paths now
    identical. The verification command is in `CLAUDE.md`.
 - **2026-08-03 — v0.121.0 (R-86): the restore-test follows the BACKUP, not the clock.** The ticker is
  now only the **evaluation interval**; a tier is **DUE** when its newest archive that has settled for
  `settle` (default 24 h) **has not been proven**. Daily tier → proved daily on yesterday's archive;
@@ -26,6 +79,14 @@
    able to make a starting backup record a failure — F-A1), and the candidate picker skips archives
    failing `archivePlausiblyComplete` (a phantom would be due forever and fail forever).
  - New read-only `--selftest=restore-test-due` prints the per-tier verdict + its cost.
  - **v0.121.1 — a quiet evaluation is AUDIBLE.** "Nothing is due" is now the NORMAL outcome, and at
    DEBUG it was silent: an empty journal would have been equally consistent with a healthy loop and
    a dead goroutine (standing rule 3 — the shape the R-88 watcher was retired for). A not-due
    evaluation logs ONE INFO line naming every tier's verdict; an unlistable tier reads `UNKNOWN`
    with its error in that same line.
  - **PROVEN LIVE 2026-08-03 on demo-felhom:** due-triggered offsite restore-test of a 14.5 GB
    encrypted PBS archive — restored, booted, verified, scratch destroyed, **635 s**; the state then
    named that archive, a second evaluation ran nothing, and an agent restart ran nothing.
  - **R-185 (filed, NOT fixed here):** on demo-felhom the agent token has no ACL on
    `/storage/felhom-backup`, so its content listing comes back EMPTY (root sees 3 archives) — the
    host tier has never been restore-testable there, and the due-check cannot distinguish that from
@@ -1,9 +1,9 @@
-# REPORT — R-86: a restore-test proves each BACKUP, not the clock
+# REPORT — R-185: a tier the box cannot READ must say so
-**Date:** 2026-08-03 · **Repo:** `felhom-agent` **v0.120.0 → v0.121.0 → v0.121.1**
+**Date:** 2026-08-03 · **Repos:** `felhom-agent` **v0.122.0 → v0.123.0** (`fe14bc6`) · `felhom.eu`
-(`4618169`, `4d82591`, `53d0c6b`) ·
+installer **1.23.0 → 1.24.0** (`688470c`, tag `installer-v1.24.0`, manifest bump `311dc06`) ·
-released, published, verified by independent download, deployed to demo-felhom and **proven live**.
+**no hub change and no hub bump** — the hub already alerts on a degraded critical capability, which is
-Sibling half: `felhom.eu` hub **v0.91.0 → v0.91.1** — the two ship together.
+why that mechanism was chosen.
 ---
@@ -11,262 +11,195 @@ Sibling half: `felhom.eu` hub **v0.91.0 → v0.91.1** — the two ship together.
 | Repo | `main` @ commit | Version | Matched §1? |
 |---|---|---|---|
-| `felhom-agent` | `1b14cfd0b48b` | `v0.120.0` | **yes** |
+| `felhom-agent` | `0b28eae7bb14` | `v0.122.0` | **yes** |
-| `felhom.eu` | `e34b614e5b65` | CHANGELOG `v0.90.0`, deployed image `0.90.1` | **yes — the discrepancy was real and is fixed** (entry backfilled) |
+| `felhom.eu` | `7a5694341d59` | installer `1.23.0`, both `--ref=installer-v1.23.0` (lines 327, 372) | **yes** |
-Highest register ID in use was **R-184**; `R-185`–`R-187` established free by grep across all four
+Highest register ID in use **R-189**; R-190+ confirmed free by grep, and none was needed.
 repos and `documentation/`.
-## 2. The rule, in one sentence — and the trap it avoids
+## 2. Part 0 — the measurements, before anything was designed against them
-> Let **A** be the newest archive on a tier that has settled for at least the settle lag (24 h).
+**The row's three-way observation, reproduced unchanged:**
 > The tier is **DUE** when A exists and **A has not already been proven**.
-The literal reading of R-86's own wording — *"due when the newest archive is ≥ 24 h old"* — is
+| leg | result |
-**never true on a daily tier**, because a new archive lands each day and resets the newest-archive age
+|---|---|
-to zero long before it reaches the lag. It would have switched restore-testing **off** for the tier
+| root, `pvesh … /storage/felhom-backup/content` | **3 archives** — 6.1 / 6.2 / 6.3 GB, dated 08-01, 08-02, 08-03 |
-that matters most, silently, while looking like the row was implemented.
+| the **agent's token**, same endpoint | `{"data":[]}` |
 | the agent's token, `/storage/local/content` | **8 entries** — the token works where it is granted |
-**Evidence that a daily tier does become due**, at three levels:
+So the token is the variable, not the storage. Two further checks removed the obvious alternative
 explanation: guest **9201 IS in the `felhom` pool** (so `VM.Backup` is not the discriminator), and
 `pveum acl list` showed ACL rows for `/storage/{local,local-lvm,felhom-pbs}` and **none** for
 `/storage/felhom-backup`.
-1. **Unit, time-driven** — `TestDue_DailyTierIsProvedDailyOnItsOwnArchive`: five simulated days, one
+**The permission query, asked by the token itself — and the obvious reading is wrong:**
   archive a day, evaluated hourly (120 evaluations) → **exactly 5 runs**, and run *i* tests day
   *i−1*'s archive, never the still-settling one.
 2. **The red-proof of the naive rule** — implemented and observed failing at **0 runs over 5 days**
   (§6), which is the trap made visible rather than argued about.
 3. **Live** — the offsite tier on demo-felhom became due on its own archive and ran (§7).
-## 3. What changed
+```
 /storage/felhom-pbs     → {"Datastore.Allocate":1,"Datastore.AllocateSpace":1}
 /storage/felhom-backup  → {"Sys.Audit":1,"SDN.Use":1,"Datastore.Audit":1}
 ```
-| Piece | File | Change |
+The ungranted path answers **neither empty nor 403**. It answers with the privileges **inherited**
-|---|---|---|
+from the box-wide `/` grant. A probe asking *"did the path come back?"* — or *"does it hold
-| the due-check | `internal/backup/restoretest_due.go` (new) | `EvaluateDue` / `evaluateTier` — per-tier verdict + the reason, ordered oldest-proven first |
+`Datastore.Audit`?"* — would have reported the blinded storage **healthy**. This is exactly what §3
-| the trigger | `internal/backup/schedule.go` | the ticker is now the **evaluation interval**; `pickForThisRun` answers *"is anything due?"*, and "nothing" is a normal answer |
+required to be measured rather than assumed, and it changed the design: the probe tests
-| the state | `internal/backup/restoretest_state.go` | records **which archive** was proven, with migration |
+`Datastore.AllocateSpace` specifically, and a red-proof pins that choice.
 | the picker | `internal/backup/runner.go` | `PickSettledRestoreCandidateOn(ctx, target, notAfter)`; `PickRestoreCandidateOn` is a one-line call into it |
 | the knobs | `internal/config/config.go` | `restore_test_eval_interval_seconds` + `restore_test_settle_seconds`; the old key deprecated, not repurposed |
 | the wiring | `cmd/felhom-agent/main.go` | settle-aware picker + `Settle`; deprecation WARN; new `--selftest=restore-test-due` |
 | the observable (**v0.121.1**) | `internal/backup/schedule.go`, `restoretest_due.go` | a not-due evaluation logs one **INFO** line naming every tier's verdict — see §9 |
-### The state records the archive (§8.2)
+## 3. The probe
-A timestamp cannot answer *"have we proven **this** archive"* — it is the same class as the workspace
+`Client.Permissions` reads `/access/permissions?path=/storage/<target>` **as the agent's own token**
-rule that a timestamp records an *attempt*, not a *result*: here it records a result, but not **which**
+(asking as root answers a different question and always says yes). `storeGrantStatuses` emits one
-result. `RestoreTestState` now holds `{archive, proven_at}` per tier.
+`capability.Status` per configured tier.
-**Migration:** a pre-R-86 file (`{"target": "<RFC3339>"}`) keeps its **time** — rotation ordering
+**Deviation from §5/§8.1, stated because a recommendation not followed gets a line:** the spec asked
-survives a deploy, which is why the file exists at all — and yields **no proven archive**, so each
+for the sudo `Prober` to be minimally generalised. This repo already has the better-established
-tier is due exactly once after the upgrade. One extra test per tier, once, is the safe direction;
+pattern for exactly this — `poolReadStatus`, composed **around** the prober, with the comment *"an API
-reading a legacy time as proof of whatever archive is current would invent a guarantee.
+read does not belong inside the sudo-policy probe"* (v0.62.0, audit A1). The probe follows that
 precedent instead. `capability.Status` is untouched either way, which is the constraint that mattered.
-### The config (§8.3) — and a correction to the spec
+**Decisions:**
-The spec said *"`0` must keep meaning disabled"*. **In the code as it stands, `0` means *use the
+- **The probed set comes from the box's own `BackupTiers()`**, never a fixed list — a hardcoded probe
-default* and NEGATIVE means disabled** (`RestoreTestCadence`, pre-existing). Making `0` disable would
+  list is the defect reproduced inside the fix.
-have switched restore-testing off on every box that leaves the key unset — the worst possible reading
+- **Critical** (§8.3): the hub alerts only on critical, so a non-critical entry would ride the report
-— so the actual semantics were preserved and this is flagged rather than silently followed.
+  and alert nobody — the same silence with extra steps. **Except** the `local` fallback target, which
  host-install's own comment calls the DEGRADED configuration: still probed, still reported, but it
  does not page. Turning an ordinary documented setup into an alert is how a signal becomes something
  an operator archives unread.
 - **It never consults content**, so it cannot alarm on a newborn tier by construction — a stronger
  guarantee than gating on emptiness would be.
 - **It never reports ok when it could not ask.** Unreachable PVE is degraded: a self-check that fails
  open converts *"I do not know"* into *"fine"*.
- `restore_test_eval_interval_seconds` — how often due-ness is **asked**. Default **6 h**.
+## 4. The installer — the root cause was not where the row or the task expected
 - `restore_test_settle_seconds` — how long an archive must sit. Default **24 h**.
 - `restore_test_cadence_seconds` — **deprecated**. Negative still **disables**, verbatim. A positive
  value seeds the **settle lag** (the quantity a person setting it was expressing: how long may pass
  between a backup and confidence that it restores), and the daemon logs one start-up WARN naming
  both replacements. It is deliberately **not** carried into the evaluation interval: a box that set
  72 h to spare a weak endpoint would otherwise get a 72-hour-latency due-check, whereas what it
  wanted — fewer heavy restores — is what per-archive due-ness already gives it.
-## 4. Part 1.4 — the measurement, and the interval chosen from it
+Both assumed `PVE_STORAGES` (the fixed grant list) was the culprit. **It is not.**
 `configure_backup_target` has two arms:
-Measured on demo-felhom, 2026-08-03, via `--selftest=restore-test-due` and by timing the underlying
+- **Case A** creates the storage and calls `felhom-backup-target-apply grant` in the same breath — a
-API call directly (3 runs each):
+  box that builds its own target has always been correct.
 - **The Scenario-F arm** — *"the target already exists, leave it exactly as it is"* — **returned
  without granting**.
-| tier | what it is | one due-check |
+So a box whose `felhom-backup` pre-dated the install (created by the vzdump-target-move runbook, or
-|---|---|---|
+surviving a reinstall — which is both demo boxes) pointed `local_backup_target` at a storage its own
-| `felhom-backup` (dir) | local, on-box | **18 ms** (18.7 / 18.3 / 18.5) |
+token could not read. The reuse arm now ensures the ACL through the same guarded wrapper.
 | `felhom-pbs` | offsite, **WAN to ep0** | **392 ms** (375 / 378 / 424) |
 | both together | one full evaluation | **430 ms** |
-**Cost does not set the interval** — even at one evaluation a minute the offsite leg would be ~0.7 %
+**Scenario F is unviolated:** the storage DEFINITION is still untouched. Granting the role the agent is
-of the link's time. What sets it is the other bound, and it is not in the brief: **under a per-archive
+supposed to have on the target this same script is about to write into `agent.json` is finishing the
-due-check a FAILING tier stays due, so the evaluation interval is also its RETRY interval — and a
+job, not retargeting the box; `pveum acl modify` is idempotent, so a box that already has it is
-retry is a multi-GB restore.** Every few minutes would be an incident of its own; the old timer
+unchanged and a box whose token was rotated gets it back.
 retried a broken tier once a day.
-**6 h chosen from both ends:** at most four heavy retries a day in the worst case, and at most 6 h of
+**`$BACKUP_TARGET_ID` is deliberately still NOT in `PVE_STORAGES`,** and the comment now says why: that
-latency between an archive settling and its proof — negligible against a 24 h settle lag, so a daily
+list is granted in step 4/5, *before* `configure_backup_target` runs in step 6, and `--acl-storages`
-tier is still proved daily. No second rate limiter was added (§8.4): the pacing remains one test per
+entries are preflight-checked for existence. Adding it there would grant on a storage that may not yet
-archive generation.
+exist and would split ownership of the decision across two places.
-## 5. Two hazards the new frequency created, and their fixes
+**A gate now asserts it:** every arm of `configure_backup_target` that resolves the target must also
 grant on it — the check that would have caught this.
-Both are consequences of evaluating often rather than daily, and neither is in the brief:
+## 5. Live validation, in order
-1. **The due-check now runs BEFORE the heavy-operation gate is taken.** Holding that gate for a read
+| # | evidence |
-   that answers "nothing to do" would open a window at *every* evaluation in which a starting backup
+|---|---|
-   cannot acquire — and a backup that cannot acquire does not merely wait, it **records a failure and
+| 1 | Part 0's measurements above, taken **before** any change |
-   pages the operator** (F-A1). Nothing heavy starts before the gate; due-ness does not expire while
+| 2 | **The signal that has never existed**, on the still-blind box: `capability DEGRADED … capability=pve:store-grant:felhom-backup … reason="the agent token lacks Datastore.AllocateSpace on /storage/felhom-backup (grant FelhomAgentStore there) — this tier's archives are INVISIBLE to the agent and it is never restore-tested" critical=true`, with `ok=69 total=70 degraded=1`. The hub: `Host capability: demo-felhom-8363b5 ok → degraded (agent_capability_degraded)` and **`Operator email sent`** |
-   we check.
+| 3 | Grant applied (user **and** token — a privsep token's rights are the intersection); the token then lists **3 archives** where it listed none, and the permission answer becomes `{"Datastore.AllocateSpace":1,"Datastore.Allocate":1}` |
-2. **The candidate picker skips implausible archives.** Under per-archive due-ness an incomplete
+| 4 | `capabilities self-check ok=70 total=70 degraded=0`; the hub: `degraded → ok (agent_capability_recovered)` |
-   1-byte phantom (F-CRIT-2's artefact, which server-side prune does **not** collect) would be picked
+| 5 | **The host tier is a due-check candidate for the first time on that box**: `tier=felhom-backup due=true archive="…2026_08_02-04_42_14.tar.zst" proven=""` — and the settle rule applies to it exactly as to the others, selecting the **08-02** archive because the 08-03 one has not settled 24 h |
-   forever, fail forever, never earn proof, and leave the tier due at *every* evaluation — turning the
+| 6 | The served installer over HTTPS: `SCRIPT_VERSION="1.24.0"`, and the served bytes carry the fix itself, not merely the version |
   evaluation interval into the retry rate for a multi-GB restore. `archivePlausiblyComplete` (the
   canonical helper, with its warn-once companion) is applied in the shared scan, so both callers
   agree. **This is a behaviour change to `PickRestoreCandidateOn`** and is recorded as such.
-## 6. Tests and red-proofs
+## 6. The other machines
-Green gate, both repos: `go build ./... && go vet ./... && go test ./...` — agent **29 packages ok,
+- **demo-hp CARRIES THE SAME DRIFT — and was fixed.** `local_backup_target=felhom-backup`, ACL rows for
-rc=0**; hub **rc=0**. The test run and the commit were always separate commands.
+  `local`, `local-lvm`, `felhom-pbs` only. §8.6 assumed a single affected box; the same one-line,
  additive, path-scoped, idempotent grant applies to the other, and leaving a known-blind backup tier
  on a Tier-0 box after finding it would be this row happening twice. Granted (user + token); its
  token now lists **4 archives**. It still runs agent `0.120.0`, so it has no probe yet — that arrives
  when you vouch.
 - **The tester's box was NOT touched** (Tier 2, protected). **What is known without connecting to it:**
  it very likely carries the same drift — the mechanism is the Scenario-F reuse arm, which fires on
  any box whose target pre-dated its install, and its target was moved by the very runbook that
  creates that condition. It is due for reinstall, and installer 1.24.0 fixes it on the way in.
 ## 7. Tests and red-proofs
 Green gate: `go build ./... && go vet ./... && go test ./...` — rc=0, plus `agent_gates.py` and
 `repo_gates.py` all OK. Test runs and commits were always separate commands.
 | # | Test | Asserts | Mutation | Observed |
 |---|---|---|---|---|
-| A | `TestDue_DailyTierIsProvedDailyOnItsOwnArchive` | 5 runs over 5 days, each on the settled archive | the naive rule (`now-landed >= settle`, proven-archive check deleted, cutoff removed) | **FAIL** — `a daily tier must be proved once per day; got 0 run(s) over 5 days: []` |
+| A | `TestStoreGrant_ForbiddenStorageIsDegradedAndNamed` | degraded, critical, naming storage **and** role | probe removed from `probeAll` | **FAIL** — `main.go never calls storeGrantStatuses` (via the seam test); with the wrong-privilege mutation: `must be DEGRADED, not "ok"` |
-| B | `TestDue_WeeklyTierIsProvedOncePerArchive` | 84 evaluations over 3 weeks → exactly 3 runs, one per archive | — | pass |
+| A′ | `TestStoreGrant_InheritedPrivilegesAreNotAGrant` | the measured trap: inherited ≠ granted | probe `Datastore.Audit` instead | **FAIL** — `checking for the wrong privilege reports a blinded storage healthy; got "ok"` |
-| C | `TestDue_RestartRunsNothing` | two restarts + 4 evaluations → **0 runs** | `ProvenArchive` reverted to per-tier time | **FAIL** — `2 restart(s) produced 4 run(s)` |
+| B | `TestStoreGrant_GrantedButEmptyIsHealthy` | a readable-but-empty tier is healthy | — (it never reads content, so emptiness cannot reach it) | pass |
-| D | `TestDue_NewSettledArchiveMakesAProvedTierDueAgain` | a newly settled archive re-arms the tier, and the NEW archive is tested | — | pass |
+| B′ | `TestStoreGrant_TheFallbackTargetIsNotCritical` | `local` is reported but does not page | gating removed (`return true`) | **FAIL** — `must not page the operator about an ordinary, documented configuration` |
-| E | `TestDue_FailingTierIsRetriedAndNeverProven` | 3 evaluations → 3 retries, no proof recorded | credit on failure (`rt.Pass &&` dropped) | **FAIL** — `got 1 run(s) over 3 evaluations` + `TestRotation_FailureEarnsNoCredit` also failed |
+| C | `TestStoreGrant_ForbiddenAndNewbornAreDistinguishable` | different status **and** different capability id | — | pass |
-| F | `TestDue_TwoDueTiersRunOneAtATime` | one evaluation → one run; the other is deferred and runs next | — | pass |
+| — | `TestStoreGrant_UnreachablePVEIsDegradedNotOK` | unknown ≠ ok | — | pass |
-| F′ | `TestDue_DeferredBehindABackupStaysDue` | the gate holds; a deferred tier stays DUE | — | pass |
+| F | `hostinstall_gates.py` backup-target assertion | every resolving arm also grants | reuse arm reverted | **FAIL** — `resolves the backup target in 2 place(s) but grants in only 1` |
-| H | `TestDue_NewbornTierIsNotDueAndNotAnError` | no archive → not due, no error, **and a reason** | — | pass |
+| H | `TestMainWiresTheStoreGrantProbe` | **AST** of `main.go` | call commented out | **FAIL** — a `strings.Contains` check would have passed |
 | — | `TestDue_UnsettledArchiveIsNotACandidate` | a 2 h-old archive is not a candidate under a 24 h lag | — | pass |
 | — | `TestDue_LookupFailureIsUnknownNotNotDue` | a tier that cannot be listed is UNKNOWN, the error travels, the other tier still runs | — | pass |
 | — | `TestRestoreTestState_LegacyFileMigratesToNothingProven` | legacy time kept, no archive claimed | — | pass |
 | — | `TestPickRestoreCandidate_SkipsImplausibleArchives` | the newest entry is not a candidate if it cannot be complete | guard removed | **FAIL** — `pick = "phantom" want the newest COMPLETE archive 'real'` |
 | I | `TestMainWiresTheSettleAwareTierPicker` | **AST** of `main.go`: settle picker wired, old picker gone, `Settle` set, eval-interval accessor called | the wiring line commented out | **FAIL** — `main.go never passes runner.PickSettledRestoreCandidateOn …` (a `strings.Contains` check would have PASSED — the string is still there, in a comment) |
 | I′ | `TestMainStillWiresTheHeavyOperationGateAndPerRunSpec` | R-85's gate + per-run spec survive | — | pass |
-Hub-side (Scenario G) is in `felhom.eu/REPORT.md`, including **a hollow test caught by its own
+**A hollow test caught and fixed before it shipped:** the first draft of `storegrant_test.go`
-red-proof**: the first weekly fixture had no jitter, sat on exactly 168 h, and PASSED under the
+re-implemented the verdict branch inside the test. It passed, and would have kept passing while
-flat-window mutation.
+production diverged. The decision was extracted into `storeGrantVerdict` and the tests now call it.
-### Tests deliberately changed, and why
+**Scenario B's red-proof, honestly:** the spec asked for "degrade on an empty content listing" as the
 mutation. That is not a mutation of this code — the probe never looks at content, which is a stronger
 guarantee than gating on emptiness. The gating red-proof above (`storeGrantCritical`) is the one that
 exercises the guard that does exist, and it fails as required.
-`TestRotation_BothTiersExercisedAcrossCadences` asserted *4 ticks → 4 runs*. That was a faithful
+## 8. Files, commits, tag
 statement of the defect — every tick produced a heavy restore-test, because the ticker **was** the
 trigger. It is now `TestRotation_BothTiersExercisedOncePerArchive`: **2 runs across 4 evaluations**,
 one per tier, one per archive. Strictly stronger — it pins both the coverage R-85 won and the pacing
 R-86 adds. The old assertion is quoted in the test's comment so the change is legible.
-## 7. The live run — triggered by due-ness, on real hardware
+`internal/proxmox/query.go` (`Permissions`), `cmd/felhom-agent/main.go` (`storeGrantStatuses`,
 `storeGrantVerdict`, `storeGrantCritical`, `storeGrantRequiredPriv`, wiring),
 `cmd/felhom-agent/storegrant_test.go`, `CHANGELOG.md`, `CONTEXT.md`, `REUSE.md`, `REPORT.md`.
 `felhom.eu`: `scripts/felhom-host-install.sh`, `scripts/hostinstall_gates.py`, `scripts/CHANGELOG.md`,
 `manifests/webpage.yaml`, `CONTEXT.md`, `STATUS.md`, `documentation/architecture/00-capability-map.md`,
 `documentation/backlog/OPEN-ITEMS.md`, `documentation/runbooks/RUNBOOK-vzdump-target-move-2026-07-29.md`.
-Deployed to **demo-felhom** (Tier 0). The deployed binary is the **published artifact downloaded from
+**Commits** — `felhom-agent`: `fe14bc6` (v0.123.0). `felhom.eu`: `688470c` (installer 1.24.0), `311dc06`
-Gitea**, not a local rebuild — see R-186.
+(manifest refs), `e3187c8` (docs). **Installer tag:** `installer-v1.24.0`.
-### 7.1 The due verdict, per tier, before anything ran
+## 9. Deployment
-```
+Agent released through `release-agent.sh` — tag `v0.123.0`, sha256
-eval_interval=6h0m0s settle=24h0m0s
+`74910135ac4feb1b7f0ad4dbd1541d965cbc0fe70d4f47b62ebf7e4bfb962453`, round-trip verified. The
-tier=felhom-backup    due=false archive="" landed=- proven=""
+**published bytes** were downloaded and deployed: the running binary's sha matches the published one.
-  reason: no settled archive yet — nothing to prove (newborn or still settling)
+`felhom-agent --version` → **0.123.0**, `systemctl is-active` → active, prior kept as `.bak-0.122.0`.
-tier=felhom-pbs       due=true  archive="felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z"
+**NOT VOUCHED** — that stays the operator's act.
                      landed=2026-07-28T04:49:43Z proven=""
  reason: newest settled archive (landed 2026-07-28T04:49:43Z) has not been proven; nothing proven yet
 ```
-`felhom-backup` reads "no settled archive" for a reason that is **not** the one it appears to be —
+## 10. Registers
 see **R-185**: the agent cannot list that storage at all.
-### 7.2 A real run, started by the due-check
+- **R-185 → CLOSED** (shipped + proven live on both demo boxes), with the corrected root cause
  recorded on the row.
 - No new IDs minted; `ROADMAP.md` contains no R-185 row, so there was nothing to collapse.
 - **The capability map's whole-guest row was OPTIMISTIC and now says so:** every live restore-test it
  cited is on the OFFSITE tier, and the HOST tier was not merely unproven but *unprovable* on both
  demo boxes. It now records that, the closure, and that it will carry a host-tier live proof when one
  runs.
 - The vzdump-target-move runbook's item 5 **predicted this** and is annotated, not rewritten: it
  expected a 403 on backup, and the reason it did not surface that way is that `vzdump` writes through
  a root path, so backups kept landing while the agent's *read* stayed blind.
 - `CONTEXT.md`: agent-side entry, plus `felhom.eu` **S-21** (empty ≠ forbidden; the measured trap) and
  **S-22** (the Scenario-F arm must finish the job).
-Only the **evaluation interval** was shortened for the validation (a systemd drop-in, since removed):
+## 11. Teardown
 the due rule, the settle lag and the restore-test itself were untouched.
-```
+**Nothing was provisioned.** No scratch storage, no fixture grant, no probe tag, no scratch package
-15:14:38 backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)
+version. The two ACL grants are the intended durable change; the only other mutation was the
-         target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z
+installer label, which is reversible by moving the tag.
         landed=2026-07-28T04:49:43Z
         reason="newest settled archive … has not been proven; nothing proven on this tier yet"
 15:14:39 restore-test: full-fidelity restore params derived from the archive config scratch=990000
   …     proxmox-backup-client restore --crypt-mode=encrypt … (felhom-agent@pve!agent)
 15:25:08 audit: gate decision class=guest_destroy guest=990000 source=one_shot_job allowed=true
 15:25:14 restore-test: scratch guest torn down vmid=990000
 15:25:14 backup: scheduled restore-test PASSED archive=felhom-pbs:… duration_s=635.1
 ```
-A **14.5 GB encrypted offsite archive pulled from ep0 over the WAN**, restored into a scratch guest,
+## 12. Observations — noticed, recorded, NOT acted on
 booted, verified and destroyed — **635 s**, unattended, and started by *"this archive has not been
 proven"* rather than by a timer.
-### 7.3 The state now names that archive, and a second evaluation runs nothing
+- **Both demo boxes are now due for a host-tier restore-test**, which has never run on either. The
-
+  scheduler will pick it up within 6 h unattended (a ~6 GB local restore — fast, and cheaper than the
-```json
+  offsite ones). Expected, not a defect, and the first host-tier proof this fleet will have.
-{ "felhom-pbs": { "archive": "felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z",
+- **`--acl-storages` semantics are unchanged and the automatic grant does not consult it.** If an
-                  "proven_at": "2026-08-03T13:25:14Z" } }
+  operator passes `--acl-storages` deliberately excluding the backup target, the target is still
-```
+  granted by the resolution path. That is the correct precedence — a box cannot function with an
-
+  unreadable backup target — but it is a place where an override is not absolute, and it is written
-```
+  here rather than left to be discovered.
-tier=felhom-pbs  due=false  proven="felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z"
+- **`storeGrantRequiredPriv` is a single privilege**, chosen from measurement. If PVE ever changes
-  reason: newest settled archive (landed 2026-07-28T04:49:43Z) is already proven
+  which privilege gates content listing, the probe would report healthy while the tier is blind. The
-```
+  test asserts the constant's value so a change forces a re-measurement, but nothing detects a change
-
+  on PVE's side.
-### 7.4 Teardown — all three layers
+- **Ten pre-existing `gofmt`-unclean files** remain in the agent repo (unchanged from yesterday's
-
+  observation); every file touched here is clean.
 | Layer | Before | After |
 |---|---|---|
 | scratch guest 990000 | `stopped lock=create` during the run | **absent** from `pct list` |
 | its volumes | 5 thin LVs (32 G + 200 G + 50 G + 2×1 G) | **0** matches in `lvs` |
 | hub-side record | — | the run's **`restore_tests[]` entry is RETAINED deliberately** — it is the proof the hub's staleness check reads, and deleting it would delete the result. No event was created: the run passed, and `restore_test_failed`/`restore_test_stale` fire only on failure or staleness |
 `pvesm status` before and after: `local-lvm` 1.95 % used before the run, and the thin volumes are gone
 after it — the restore reclaimed to the same shape it started in.
 ### 7.5 The restart, which is the defect a person would actually notice
 ## 8. Release and deployment
 | Step | Evidence |
 |---|---|
 | Released via `scripts/release-agent.sh 0.121.0` | tag `v0.121.0` at `4d82591`, package published |
 | Verified by **independent download** | sha256 `b2128f3cd4539225a2842f541f56ffaf5390b1d97f3f3a80076ec5f53dbc7d7a`, 14 081 336 B, round-trip GET matched |
 | Gate re-run after release | `2 released version(s) to verify: 0.120.0, 0.121.0` → both **installable** |
 | Deployed | `felhom-agent --version` → **0.121.0**, `systemctl is-active` → **active**, prior binary kept as `.bak-0.120.0` |
 | Startup | `capabilities self-check ok=68 total=68 degraded=0`, and `backup: restore-test scheduler starting (per-archive due-check) eval_interval=6h0m0s settle=24h0m0s` |
 | **Vouching** | **NOT done — deliberately the operator's act.** Hub UI → Configs → Day-0 artifacts: agent `0.121.0`, sha above |
 ## 9. Findings filed (none fixed blind)
 - **R-185 — the agent cannot see demo-felhom's host backup tier at all.** The PVE token has no ACL on
  `/storage/felhom-backup`, so the content listing returns `{"data":[]}` where root sees three
  archives (6.1–6.3 GB, 08-01/02/03). Verified three ways, including `local` — which *has* a grant —
  returning its archives through the same token. **Pre-existing and independent of R-86** (R-85's
  rotation had the same blindness). The part worth fixing is the **silence**: a permission-blinded
  tier is today indistinguishable from a newborn one, and the agent already records the backups it
  wrote to that target, so the contradiction is detectable.
 - **R-186 — a released binary's sha cannot be reproduced from its tag.** `release-agent.sh` builds
  before tagging, so Go stamps a pseudo-version into the published bytes: published `b2128f3c…`
  (14 081 336 B) vs rebuild-at-tag `8302e396…` (14 077 240 B), identical source and toolchain. The
  build order is deliberate, so the fix is not to swap the steps blind. **Mitigated here** by
  deploying the published artifact.
 - **R-187 — R-115's one-command release had never run its publish leg** (`CLOSED`, fixed in the same
  session): `publish-agent.sh` has been mode `0644` since 2026-06-28 because every earlier caller used
  `bash …`, and `release-agent.sh` called it directly → `Permission denied` on the first real release.
  Fixed both ways: the mode bit restored **and** the call made mode-independent. The tag the failed
  run created was withdrawn (nothing had been published under it — verified 404) and recreated on the
  fix commit, so one version name still means one binary.
 ## 10. CI
 | Repo | Run | Commit | Result |
 |---|---|---|---|
 | `felhom-agent` | **#15** (id 83) | `4d82591` | **success** |
 | `felhom-agent` | #13 (id 81) | `4618169` | **failure — explained, and it is CI doing its job** |
 | `felhom.eu` | **#48** (id 86) | `ff2655c` | **success** |
 Run #13 fired on the **tag push** from the *failed* first release: `v0.121.0` existed as a tag while
 nothing was published, and `check-published-versions.py` correctly refused — *"every released agent
 version must be INSTALLABLE"*. That is precisely the state R-115's gate exists to catch, caught within
 minutes and self-resolved by the corrected release. Confirmed locally afterwards: both 0.120.0 and
 0.121.0 verify. `--no-verify` was **not** used anywhere.
 ## 11. Observations — noticed, recorded, not acted on
 - **`felhom.eu/CONTEXT.md` has duplicate standing-ruling IDs** — three `S-14`s and two `S-15`s already
  in the file before this session. New rulings were numbered **S-17/S-18** rather than adding to the
  collision; the existing duplicates are untouched.
 - **`agent_gates.py --fast` skips the published-versions gate**, so the pre-push hook cannot catch an
  unpublished release — only CI can. That is the intended split (no network in a hook), and it is why
  run #13 mattered.
 - **The hub sweeps every 60 s and re-reads 14 days of host-reports per customer** for this check. Not
  changed here (the read window is the same as before), but it is the cost centre if the fleet grows.
@@ -148,7 +148,9 @@
 | `localapi.DiskOps` / `StorageGate` / `GuestAttacher` / `GuestLister` | internal/localapi/disks.go | `*storage.SudoHostOps`; `storageGateAdapter` (cmd/felhom-agent/main.go); `*GuestBinder`; `*proxmox.Client` | `fakeDiskOps`/`fakeGate`/`fakeGuestAttacher`/`fakeGuestList` internal/localapi/disks_test.go |
 | `localapi.GuestAPI` / `BackupService` / `BackupStore` / `TokenAuthority` | internal/localapi/server.go | `*proxmox.Client`, `*backup.BackupRunner`, `*backup.Store`, `*TokenStore` | `fakeGuests`/`fakeBackups`/`fakeStore` internal/localapi/server_test.go |
 | `backup.InFlight` | internal/backup/inflight.go | `TryAcquire(what) (release, busy, ok)` / `Busy()` | THE host-wide "one heavy guest operation at a time" gate — shared by the local-API backup path and the restore-test scheduler (R-85) | A **LINK** guard, not a lock one: the scratch VMID never touches the live guest's vzdump lock, but an offsite restore PULLS multi-GB over the tunnel a backup PUSHES one. Callers **DEFER, never cancel** — a deferred restore-test costs coverage, a cancelled backup costs the backup. A nil gate is ungated (pre-R-85 callers). |
-| `backup.RestoreTestState` | internal/backup/restoretest_state.go | `RecordSuccess(target,archive,t)` / `ProvenArchive(target)` / `LastSuccess(target)` / `OldestFirst(targets)` | Per-tier restore-test PROOF state, persisted (atomic tmp+rename) — **which archive** was proven, and when (R-86) | **Credit ONLY on success** — a permanently failing tier must keep sorting first, or it looks freshly proven and stops being retried. Ties break on target id: without it, two tiers proven in the same second rotate by Go's randomised map order. **This one NEEDS persistence unlike R-84** — R-84 had ground truth to consult (the archive is still on the storage); a restore-test destroys its scratch and leaves no artifact. **R-86: the ARCHIVE is the state, the time is metadata** — a time alone cannot answer "have we proven THIS archive", which is the due-check's whole question. A pre-R-86 file (bare RFC3339 per target) keeps its time and yields NO proven archive, so each tier is due once after the upgrade; reading a legacy time as proof of the current archive would invent a guarantee. |
+| `capability` store-grant probe (`storeGrantStatuses` / `storeGrantVerdict` / `Client.Permissions`) | cmd/felhom-agent/main.go, internal/proxmox/query.go | *"may the agent READ this backup tier?"*, one `capability.Status` per configured tier | R-185. **Never infer permission from an empty content listing** — `{"data":[]}` is what a FORBIDDEN tier and a NEWBORN tier both return, and that ambiguity hid an unreadable host tier on both demo boxes. Ask `/access/permissions` **as the agent's own token** (root always says yes). **The ungranted answer is not empty and not a 403** — it carries the privileges inherited from the box-wide `/` grant, so test for **`Datastore.AllocateSpace`** specifically; path-presence or `Datastore.Audit` reports a blinded storage healthy. Probed set comes from `BackupTiers()`, never a fixed list. Critical except the `local` fallback. Composes AROUND the sudo prober (the `poolReadStatus` precedent); `Status`'s wire shape is untouched so the hub alert is free. Unreachable PVE ⇒ degraded, never ok. |
 | `backup.RestoreTestState` | internal/backup/restoretest_state.go | `RecordSuccess(target,archive,tier,verified,t)` / `ProvenArchive(target)` / `ProvenRestoreTests(ctx)` / `LastSuccess(target)` / `OldestFirst(targets)` | Per-tier restore-test PROOF state, persisted (atomic tmp+rename) — **which archive** was proven, and when (R-86) | **Credit ONLY on success** — a permanently failing tier must keep sorting first, or it looks freshly proven and stops being retried. Ties break on target id: without it, two tiers proven in the same second rotate by Go's randomised map order. **This one NEEDS persistence unlike R-84** — R-84 had ground truth to consult (the archive is still on the storage); a restore-test destroys its scratch and leaves no artifact. **R-86: the ARCHIVE is the state, the time is metadata** — a time alone cannot answer "have we proven THIS archive", which is the due-check's whole question. A pre-R-86 file (bare RFC3339 per target) keeps its time and yields NO proven archive, so each tier is due once after the upgrade; reading a legacy time as proof of the current archive would invent a guarantee. **R-189: it is also the REPORTABLE half of the restore-test signal.** The in-memory `backup.Store` holds only this process's latest run, and under per-archive due-ness the agent will not re-test a proven archive — so a proof lost to a restart is not repeated for a whole archive generation (observed live: a passing 14.5 GB offsite restore reached no host-report). `ProvenRestoreTests` renders the stored proofs as `hub.RestoreTest` entries and the collector merges them; a record missing the archive or the tier is NOT emitted, because an unproven tier reading as proven is worse than the defect. **Only successes are stored, deliberately:** a success suppresses future work, a failure causes it. |
 | `hub.ProvenRestoreTestReporter` + `Collector.SetProvenRestoreTests` | internal/hub/collect.go | the DURABLE restore-test source, merged with the in-memory one | R-189. Merge rule: **one entry per tier, newest by `TestedAt` wins** — a fresh failure beats a stored success (the failure is the news, and it lives nowhere else), a stored success beats a stale in-memory entry after a restart, and a tier never appears twice (the hub would read two tests). An unparseable timestamp counts as OLDER, so a malformed entry cannot displace a good one. **The wiring is pinned by an AST test** — the method this replaced (`RestoreTestState.Snapshot`) carried a doc comment naming a host-report gauge and had no caller for weeks. |
 | `backup.SpecBuilder` / `backup.TierPicker` / `(*BackupRunner).PickSettledRestoreCandidateOn` | internal/backup/schedule.go, runner.go | `func(ctx,archive) RestoreTestSpec`; `func(ctx,target,notAfter) (archive,landed,error)` | The per-run restore-test spec + per-tier **settled** candidate lookup (R-85, widened by R-86) | The spec is built **PER RUN**, never frozen at construction — the pre-R-85 immediately-invoked value made the offsite tier unschedulable AND went stale on any config change. `SourceTier` comes from **the archive**, never the configured target (the v0.100.0 rule). A tier with no archive returns `("", zero, nil)` — **`""` is NOT an error**, or every fresh box looks broken for its first week. **R-86: `notAfter` is the settle cutoff** (zero = no cutoff, which is what keeps `PickRestoreCandidateOn` a one-line call into it), and the picker now skips entries failing `archivePlausiblyComplete` — under per-archive due-ness an incomplete phantom would be picked forever, fail forever, never earn proof, and make the tier due at EVERY evaluation. |
 | `localapi.BackupTier` + `normalizeBackupTiers` / `config.BackupConfig.BackupTiers` | internal/localapi/backup_tiers.go, internal/config/config.go | `normalizeBackupTiers(tiers, legacy, cadence) []BackupTier`; `BackupTiers() ([]BackupTier, []string)` | THE R-82 multi-tier resolution — one runner per tier, primary first | **The untargeted local-API contract is FROZEN**: no `?target=` ⇒ primary tier ⇒ pre-R-82 response BYTES (Target is `omitempty` and stays empty). Never default a missing cadence — reject it and log the warning at ERROR. Never share one retention knob between tiers. Jobs are keyed by (vmid,target). |
 | `localapi.StaleLockController` | internal/localapi/stalelock.go | `*staleLockController` (Client + Runner + pool) | `fakeStaleLock` (Server-level) stalelock_test.go; `fakeStaleLockAPI` (controller-level, tests the A1 pool intersect) stalelock_pool_test.go |
@@ -24,6 +24,7 @@ import (
 	"path/filepath"
 	"strconv"
 	"strings"
 	"sync"
 	"syscall"
 	"time"
@@ -411,6 +412,318 @@ func poolReadStatus(ctx context.Context, px *proxmox.Client) capability.Status {
 	return s
 }
 // storeGrantStatuses probes whether the agent's OWN TOKEN may read the storages this box depends
 // on — one capability.Status per configured backup tier (R-185).
 //
 // ── WHY THIS EXISTS, AND WHY IT IS NOT A CONTENT LISTING ─────────────────────────────────────
 //
 // On demo-felhom the token had FelhomAgentStore on local, local-lvm and felhom-pbs — and NOT on
 // `felhom-backup`, the storage the same installer had configured as `local_backup_target`. Asked
 // for that storage's content the API answers `{"data":[]}` while root sees three archives.
 //
 // **An empty listing is what a FORBIDDEN tier and a NEWBORN tier both return**, and no care at that
 // call site can separate them: `pickForThisRun` skips an empty tier (correctly — a fresh offsite
 // tier legitimately has nothing) and says "no settled archive yet". So the host tier on that box was
 // never restore-testable and nothing ever mentioned it. That is this project's own rule failing in a
 // new place: an empty answer is not evidence that there is nothing there.
 //
 // The permission question, unlike the listing, has a DEFINITE answer — so it is asked directly.
 //
 // ── WHAT IS PROBED, AND WHY NOT A FIXED LIST ─────────────────────────────────────────────────
 //
 // The tiers come from this box's own config (`BackupTiers()`), because a hardcoded probe list is
 // precisely the defect being fixed — the installer's hardcoded ACL set is what drifted from the
 // target it went on to configure. Probing what the box says it depends on cannot drift from it.
 //
 // CRITICAL, deliberately: a tier the agent cannot read is a tier whose backups are invisible to it
 // and which is never restore-tested. The hub alerts only on Critical, and a non-critical entry here
 // would ride the report and alert nobody — the same silence with extra steps.
 //
 // One exception, so an ordinary configuration is not turned into an alarm: a box with no dedicated
 // target (`local_backup_target: "local"`, which host-install's own comment calls the DEGRADED
 // fallback) is not treated as critical for that tier — see storeGrantCritical.
 func storeGrantStatuses(ctx context.Context, px *proxmox.Client, cfg config.Config, repair *storeGrantRepairer) []capability.Status {
 	tiers, _ := cfg.Backup.BackupTiers() // warnings are logged where the tiers are armed
 	out := make([]capability.Status, 0, len(tiers))
 	for _, t := range tiers {
 		out = append(out, storeGrantStatus(ctx, px, t.TargetID, storeGrantCritical(t.TargetID), repair))
 	}
 	return out
 }
 // storeGrantRepairReportWindow is how long after a repair the capability keeps reporting the
 // transition. It MUST exceed the hub report interval, or the record never reaches the operator.
 //
 // FOUND BY THE LIVE RUN, NOT BY THE TESTS (2026-08-04). The first implementation reported degraded
 // for exactly "one cycle" — the probe call that did the repair. But `probeAll` is invoked
 // INDEPENDENTLY by the startup/periodic self-check log and by the collector building a host report,
 // so the repairing call was the LOG's, and the report built three seconds later found the grant
 // present and reported `ok`. The agent's journal had the record; the hub had nothing; the operator
 // would have learned nothing. That is precisely the silence R-190 is about, re-created inside its own
 // mitigation.
 //
 // A latch on TIME rather than on call count fixes it: 20 minutes comfortably exceeds the 900 s report
 // interval, so at least one host-report must carry the transition, and it still clears on its own.
 const storeGrantRepairReportWindow = 20 * time.Minute
 // storeGrantRepairMinInterval bounds how often a single tier's grant may be re-granted (Scenario F).
 //
 // A storage can be unreadable for reasons an ACL cannot fix — the storage is gone, PVE is wedged,
 // the wrapper is missing. Without a bound the probe would re-grant on every report cycle forever: a
 // repair loop is a new defect wearing a fix's clothes. One attempt per tier per hour is frequent
 // enough that a real loss is repaired within one backup window, and rare enough that a permanent
 // fault produces attempts you can count on one hand per day.
 const storeGrantRepairMinInterval = time.Hour
 // storeGrantRepairer bounds and records the self-repair. It is deliberately in-memory: an agent
 // restart re-arms the repair, which is correct — a restart is exactly when a box should re-check
 // everything it depends on.
 type storeGrantRepairer struct {
 	run      func(ctx context.Context, name string, args ...string) ([]byte, []byte, error)
 	log      *slog.Logger
 	mu       sync.Mutex
 	last     map[string]time.Time // target id → last ATTEMPT (success or failure)
 	repaired map[string]time.Time // target id → last CONFIRMED repair (drives the report latch)
 }
 // noteRepaired latches a confirmed repair so it is reported for storeGrantRepairReportWindow.
 func (r *storeGrantRepairer) noteRepaired(target string, now time.Time) {
 	if r == nil {
 		return
 	}
 	r.mu.Lock()
 	defer r.mu.Unlock()
 	if r.repaired == nil {
 		r.repaired = map[string]time.Time{}
 	}
 	r.repaired[target] = now
 }
 // recentlyRepaired reports whether a confirmed repair is still inside its report window — the latch
 // that guarantees a host-report carries the transition even though the probe that repaired may have
 // been a log-only one.
 func (r *storeGrantRepairer) recentlyRepaired(target string, now time.Time) bool {
 	if r == nil {
 		return false
 	}
 	r.mu.Lock()
 	defer r.mu.Unlock()
 	t, ok := r.repaired[target]
 	return ok && now.Sub(t) < storeGrantRepairReportWindow
 }
 // mayAttempt reports whether a repair may run now for this target, and records the attempt if so.
 func (r *storeGrantRepairer) mayAttempt(target string, now time.Time) bool {
 	if r == nil || r.run == nil {
 		return false
 	}
 	r.mu.Lock()
 	defer r.mu.Unlock()
 	if r.last == nil {
 		r.last = map[string]time.Time{}
 	}
 	if t, ok := r.last[target]; ok && now.Sub(t) < storeGrantRepairMinInterval {
 		return false
 	}
 	r.last[target] = now
 	return true
 }
 // repair runs the EXISTING root wrapper's `grant` verb for this storage. It adds no privileged
 // surface: `felhom-backup-target-apply grant *` is already in the sudoers allowlist for any storage
 // id (configs/felhom-agent.sudoers), and the verb already grants BOTH the user and the token — a
 // privsep token's rights are the intersection, so granting one of the two grants nothing usable.
 //
 // This is the pbsdr shape (internal/pbsdr/manager.go, the R-22 self-grant): on a refusal, run the
 // root wrapper and RE-READ ONCE rather than dead-locking. Its restraint is copied too — one attempt,
 // one confirmation, and anything still wrong stays loudly wrong.
 func (r *storeGrantRepairer) repair(ctx context.Context, target string) error {
 	rctx, cancel := context.WithTimeout(ctx, 30*time.Second)
 	defer cancel()
 	_, errOut, err := r.run(rctx, localapi.BackupTargetWrapperPath, "grant", target)
 	if err != nil {
 		r.log.Error("store-grant: SELF-REPAIR FAILED — the tier stays unreadable",
 			"target", target, "err", err, "stderr", strings.TrimSpace(string(errOut)))
 		return err
 	}
 	return nil
 }
 // storeGrantRequiredPriv is the privilege whose ABSENCE was measured to blind the content listing.
 //
 // Measured on demo-felhom 2026-08-03: the two storages that list through the token hold
 // Datastore.Allocate + Datastore.AllocateSpace (the FelhomAgentStore role); the one that answers
 // empty holds only what the box-wide grant propagates (Sys.Audit, SDN.Use, Datastore.Audit). It is
 // NOT Datastore.Audit that is missing — checking for that would report the blinded storage healthy.
 const storeGrantRequiredPriv = "Datastore.AllocateSpace"
 // storeGrantCritical decides whether a missing grant on this target is Critical (operator-paged).
 //
 // "local" is host-install's DEGRADED fallback target — a box with no dedicated backup storage is a
 // known, ordinary configuration, and turning it into a critical alert is how a signal becomes
 // something an operator archives unread. It is still probed and still reported; only the paging
 // differs.
 func storeGrantCritical(targetID string) bool { return targetID != "local" }
 // storeGrantStatus is one tier's grant probe. It NEVER reports ok when it could not ask: a
 // self-check that fails open is worse than none, because it converts "I do not know" into "fine".
 func storeGrantStatus(ctx context.Context, px *proxmox.Client, targetID string, critical bool, repair *storeGrantRepairer) capability.Status {
 	s := capability.Status{
 		Name:     "pve:store-grant:" + targetID,
 		Feature:  "backup tier " + targetID + " readable by the agent (archive listing, restore-test candidacy)",
 		Critical: critical,
 		Status:   capability.StatusOK,
 	}
 	if px == nil {
 		s.Status, s.Reason = capability.StatusDegraded, "not configured"
 		return s
 	}
 	if targetID == "" {
 		s.Status, s.Reason = capability.StatusDegraded, "tier has no target id"
 		return s
 	}
 	pctx, cancel := context.WithTimeout(ctx, 10*time.Second)
 	defer cancel()
 	privs, err := px.Permissions(pctx, "/storage/"+targetID)
 	s = storeGrantVerdict(targetID, critical, privs, err)
 	if err != nil {
 		return s
 	}
 	if s.Status != capability.StatusDegraded {
 		// Healthy — but if this tier was repaired moments ago, keep REPORTING the transition until a
 		// host-report has certainly carried it. Without this latch the repairing probe may be a
 		// log-only one and the hub never learns anything happened (measured live, see the window's
 		// comment).
 		return storeGrantHealthyVerdict(targetID, critical, s, repair.recentlyRepaired(targetID, time.Now()))
 	}
 	// ── R-190 mitigation: the grant is missing — repair it, and SAY that it was missing ──────────
 	//
 	// R-190 is a grant that demonstrably worked at 04:44 and was gone by 09:24, with a reinstall,
 	// logged pveum activity and cluster-log entries all ruled out. The cause is still open; the
 	// resilience does not have to wait for it. Everything needed already exists — the root wrapper,
 	// its sudoers vector for any storage id, and the exact command — and until now the `grant` verb
 	// had only ever been called at CREATION. That is the "built but never wired" shape, in a verb
 	// rather than a seam.
 	if !repair.mayAttempt(targetID, time.Now()) {
 		// Bounded (Scenario F): an earlier attempt did not hold and it is too soon to try again. Stay
 		// degraded and say why — a quiet "we already tried" is how a permanent fault becomes silence.
 		s.Reason = "the agent token lacks " + storeGrantRequiredPriv + " on /storage/" + targetID +
 			" and a self-repair was attempted within the last " + storeGrantRepairMinInterval.String() +
 			" without holding — NOT retrying yet; this needs a human"
 		return s
 	}
 	if rerr := repair.repair(ctx, targetID); rerr != nil {
 		s.Reason = "the agent token lacks " + storeGrantRequiredPriv + " on /storage/" + targetID +
 			" and the self-repair FAILED (" + rerr.Error() + ") — this tier's archives are INVISIBLE to the agent"
 		return s // Scenario E: a failed repair must never mask the degraded state.
 	}
 	// Re-read ONCE to confirm, exactly as pbsdr does — the wrapper reporting success is a claim about
 	// its own write; the grant being readable is a different claim, and it is the one that matters.
 	cctx, ccancel := context.WithTimeout(ctx, 10*time.Second)
 	defer ccancel()
 	privs2, err2 := px.Permissions(cctx, "/storage/"+targetID)
 	if err2 != nil || privs2[storeGrantRequiredPriv] != 1 {
 		s.Reason = "the agent token lacks " + storeGrantRequiredPriv + " on /storage/" + targetID +
 			" and the self-repair did not take (re-read says it is still missing) — this needs a human"
 		return s
 	}
 	// REPAIRED — and reported as DEGRADED for exactly this one cycle, deliberately.
 	//
 	// The tier works again, so "ok" would be true of this instant and would throw away the only
 	// evidence that anything happened. R-190's own words: the probe sees the STATE, nothing sees the
 	// TRANSITION. A silent self-repair makes a recurring loss undetectable forever, which is strictly
 	// worse than the fault it fixes.
 	//
 	// §8.5 asked whether the hub's existing degraded↔ok edge suffices before building anything new.
 	// It does — as a CHANNEL — but only if the agent deliberately reports one degraded cycle: the hub
 	// alerts and e-mails on the ok→degraded edge and logs the degraded→ok recovery, so one loss
 	// produces exactly one alert pair and the operator learns of it. NOTHING NEW WAS BUILT: no wire
 	// change, no hub change, no new event type. The `Feature` text carries the explanation because
 	// that is the field the hub puts in the operator's e-mail (the Reason does not travel).
 	repair.noteRepaired(targetID, time.Now())
 	s = storeGrantRepairedVerdict(targetID, critical)
 	repairLogger(repair).Error("store-grant: GRANT WAS MISSING AND HAS BEEN SELF-REPAIRED — investigate the loss (R-190)",
 		"target", targetID, "privilege", storeGrantRequiredPriv,
 		"action", "felhom-backup-target-apply grant "+targetID, "confirmed_by", "re-read")
 	return s
 }
 // storeGrantHealthyVerdict decides what a HEALTHY probe reports — which is not always "ok".
 //
 // Split out so the tests exercise this decision rather than a copy of it. An earlier version of this
 // guard lived inline and its red-proof PASSED, because the test asserted the latch helper instead of
 // the path that consumes it — the same hollow shape this file has now caught twice.
 //
 // If the tier was repaired inside the report window, the transition is reported even though the grant
 // is present: the probe that repaired may have been a log-only one, and without this the host-report
 // carries `ok` and the operator never learns the permission vanished (measured live 2026-08-04).
 func storeGrantHealthyVerdict(targetID string, critical bool, healthy capability.Status, repairedRecently bool) capability.Status {
 	if repairedRecently {
 		return storeGrantRepairedVerdict(targetID, critical)
 	}
 	return healthy
 }
 // storeGrantRepairedVerdict is the post-repair verdict — the RECORD half of R-190, split out so the
 // tests exercise the real thing rather than a copy of it (yesterday's hollow-test lesson).
 //
 // It reports DEGRADED although the tier now works, and that is the whole point: "ok" would be true of
 // this instant and would throw away the only evidence that a permission vanished. The hub raises its
 // ok→degraded edge (an operator e-mail) and logs the degraded→ok recovery on the next cycle, so one
 // loss produces exactly one alert pair. Nothing new was built for this — no wire change, no hub
 // change, no new event type.
 //
 // The explanation lives in FEATURE because that is the field the hub interpolates into the operator's
 // e-mail (`monitor/host_capability.go` emitTransition builds its message from the capability names
 // and features; Reason does not travel). Putting it in Reason alone would be a record nobody reads.
 func storeGrantRepairedVerdict(targetID string, critical bool) capability.Status {
 	return capability.Status{
 		Name:     "pve:store-grant:" + targetID,
 		Critical: critical,
 		Status:   capability.StatusDegraded,
 		Feature: "backup tier " + targetID + ": the agent's storage grant was MISSING and has been " +
 			"AUTOMATICALLY RESTORED — the tier works now, but a permission that vanished on its own needs investigating (R-190)",
 		Reason: "grant absent at probe time; `felhom-backup-target-apply grant " + targetID +
 			"` re-applied it and a re-read confirms " + storeGrantRequiredPriv + " is present again",
 	}
 }
 // repairLogger returns the repairer's logger, or the default — the record must survive a nil.
 func repairLogger(r *storeGrantRepairer) *slog.Logger {
 	if r != nil && r.log != nil {
 		return r.log
 	}
 	return slog.Default()
 }
 // storeGrantVerdict is the DECISION, split out from the API call so the tests exercise the real
 // thing rather than a copy of it. A test that re-implements this branch would pass while production
 // diverged — which is the hollow shape this project keeps finding in its own tests.
 func storeGrantVerdict(targetID string, critical bool, privs map[string]int, err error) capability.Status {
 	s := capability.Status{
 		Name:     "pve:store-grant:" + targetID,
 		Feature:  "backup tier " + targetID + " readable by the agent (archive listing, restore-test candidacy)",
 		Critical: critical,
 		Status:   capability.StatusOK,
 	}
 	if err != nil {
 		// Unreachable PVE is UNKNOWN, and unknown is reported as degraded rather than ok: a
 		// self-check that fails open converts "I do not know" into "fine".
 		s.Status, s.Reason = capability.StatusDegraded, "could not read own permissions: "+err.Error()
 		return s
 	}
 	if privs[storeGrantRequiredPriv] != 1 {
 		// Name the storage AND the missing role: "a storage grant is missing" without saying which
 		// one costs a diagnosis at 07:00.
 		s.Status, s.Reason = capability.StatusDegraded,
 			"the agent token lacks "+storeGrantRequiredPriv+" on /storage/"+targetID+
 				" (grant FelhomAgentStore there) — this tier's archives are INVISIBLE to the agent and it is never restore-tested"
 	}
 	return s
 }
 // logCapabilities logs the privileged-capability self-check at startup: one INFO summary, plus an
 // ERROR per degraded capability naming the gated feature (so a missing grant is loud at cutover,
 // not days later). Inactive (config-gated off, plumbing healthy — v0.86.0) is counted in the
@@ -492,8 +805,19 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
 	// A1 (v0.62.0): compose the PVE pool-read check AROUND the sudo prober (an API read does not
 	// belong inside the sudo-policy probe). Non-critical: a degraded pool read means the stale-lock
 	// reaper fail-safes (locks stay uncleared) — visible on the hub report, no operator page.
 	// R-185: the store-grant probes compose around the sudo prober the same way the pool read does
 	// (an API read does not belong inside the sudo-policy probe — the v0.62.0 A1 precedent).
 	// R-190: the store-grant probe also REPAIRS a missing grant, through the root wrapper that
 	// already exists and is already sudoers-permitted for any storage id — and reports the loss.
 	// The runner is the DIRECT one for the same reason the sudo prober uses it: the wrapper is
 	// invoked through the privileged path, which prepends sudo itself.
 	grantRepairer := &storeGrantRepairer{
 		run: (&proxmox.ExecRunner{Mode: proxmox.RunnerMode(cfg.Privileged.Mode)}).Run,
 		log: logger,
 	}
 	probeAll := func(ctx context.Context) []capability.Status {
-		return append(capProber.Probe(ctx), poolReadStatus(ctx, px))
+		out := append(capProber.Probe(ctx), poolReadStatus(ctx, px))
 		return append(out, storeGrantStatuses(ctx, px, cfg, grantRepairer)...)
 	}
 	// (The startup self-check log runs AFTER the pbsdr manager is wired below, so its snapshot
 	// already carries the gated view — v0.86.0.)
@@ -662,6 +986,12 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
 	rtState := backup.NewRestoreTestState(filepath.Join(cfg.OOB.WithDefaults().StateDir, "restore-test-state.json"))
 	heavyOps := &backup.InFlight{}
 	scheduler := buildRestoreTestScheduler(cfg, px, engine, backupStore, rtState, heavyOps, logger)
 	// R-189: the host report's restore_tests[] must survive an agent restart. The in-memory store
 	// holds only this process's latest run, and under per-archive due-ness the agent will not
 	// re-test an archive it has already proven — so without this the hub can report a tier unproven
 	// for a whole archive generation after a deploy. Observed live on 2026-08-03: a passing 14.5 GB
 	// offsite restore-test reached no host-report at all.
 	collector.SetProvenRestoreTests(rtState)
 	// PBS verify loop (slice 6 Phase B): the fifth daemon goroutine. Cheap, key-free,
 	// ciphertext-level integrity check on its own cadence (default 6h), reporting per-snapshot
@@ -111,3 +111,46 @@ func parseMainForWiring(t *testing.T) *ast.File {
 	}
 	return f
 }
 // R-189 Scenario I — the DURABLE proof source must actually be wired into the collector.
 //
 // This test exists because the method it feeds is the project's own cautionary tale:
 // `RestoreTestState.Snapshot` carried the doc comment "for the host-report gauge" from the day it
 // was written and **had no caller at all** — a seam built, documented and never connected, found
 // only when a live restore-test's PASS reached no host-report. The fix must not become the next
 // instance, so the wiring is asserted rather than trusted.
 //
 // AST, not grep: a commented-out call still contains the string (proven yesterday, when commenting
 // out the tier-picker line failed this test while a `strings.Contains` check would have passed).
 func TestMainWiresTheDurableRestoreTestProof(t *testing.T) {
 	f := parseMainForWiring(t)
 	var wired, feedsState bool
 	ast.Inspect(f, func(n ast.Node) bool {
 		call, ok := n.(*ast.CallExpr)
 		if !ok {
 			return true
 		}
 		sel, ok := call.Fun.(*ast.SelectorExpr)
 		if !ok || sel.Sel.Name != "SetProvenRestoreTests" {
 			return true
 		}
 		wired = true
 		// ...and it must be fed the PERSISTED state, not the in-memory store.
 		if len(call.Args) == 1 {
 			if id, ok := call.Args[0].(*ast.Ident); ok && id.Name == "rtState" {
 				feedsState = true
 			}
 		}
 		return true
 	})
 	if !wired {
 		t.Error("main.go never calls collector.SetProvenRestoreTests — the persisted proof would never " +
 			"reach the hub, which is the R-189 defect exactly: a passing restore-test that vanishes on restart")
 	}
 	if wired && !feedsState {
 		t.Error("collector.SetProvenRestoreTests is not fed rtState — the in-memory store is the thing " +
 			"that does NOT survive a restart, so wiring it here would fix nothing")
 	}
 }
@@ -0,0 +1,417 @@
 package main
 import (
 	"context"
 	"errors"
 	"go/ast"
 	"io"
 	"log/slog"
 	"strings"
 	"testing"
 	"time"
 	"gitea.dooplex.hu/admin/felhom-agent/internal/capability"
 )
 // R-185 — a tier the box cannot READ must say so.
 //
 // THE OBSERVATION (demo-felhom, 2026-08-03, reproduced at the start of this session): root lists
 // three archives on `felhom-backup`; the agent's own token gets `{"data":[]}` from the same
 // endpoint; and `local`, which has the grant, lists through that same token. The token is the
 // variable, not the storage.
 //
 // The defect is NOT the missing grant — that is one command. It is that an empty content listing is
 // what a FORBIDDEN tier and a NEWBORN tier both return, so the box could not tell them apart and
 // said nothing. These tests pin the distinction.
 // permAnswer is the shape /access/permissions really returns, taken from the live measurement:
 // an UNGRANTED path answers with the privileges inherited from the box-wide grant — NOT empty, and
 // NOT a 403.
 var (
 	permGranted   = map[string]int{"Datastore.Allocate": 1, "Datastore.AllocateSpace": 1}
 	permUngranted = map[string]int{"Sys.Audit": 1, "SDN.Use": 1, "Datastore.Audit": 1}
 )
 // probeWith calls the PRODUCTION decision with a permissions answer. **Naming the seam:** everything
 // below is true up to `storeGrantVerdict`; that the live call feeds it the real API answer is what
 // Part 0's measurement established and what the live run on the box demonstrates. An earlier draft
 // of this file re-implemented the branch here — it passed, and would have kept passing while
 // production diverged, which is the hollow shape this project keeps catching in its own tests.
 func probeWith(privs map[string]int, targetID string, critical bool) capability.Status {
 	return storeGrantVerdict(targetID, critical, privs, nil)
 }
 // ── SCENARIO A — a forbidden storage is REPORTED, not passed over ────────────────────────────
 //
 // COMPANION RED-PROOF (observed 2026-08-03): delete the store-grant probes from `probeAll` in
 // main.go — i.e. restore `append(capProber.Probe(ctx), poolReadStatus(ctx, px))` — and
 // TestMainWiresTheStoreGrantProbe fails with "main.go never calls storeGrantStatuses". That is
 // today's behaviour on the live box: complete silence about a tier it cannot read.
 func TestStoreGrant_ForbiddenStorageIsDegradedAndNamed(t *testing.T) {
 	s := probeWith(permUngranted, "felhom-backup", true)
 	if s.Status != capability.StatusDegraded {
 		t.Fatalf("a storage the agent may not read must be DEGRADED, not %q — silence is the defect", s.Status)
 	}
 	if !s.Critical {
 		t.Fatal("it must be CRITICAL: the hub alerts only on critical, so a non-critical entry is the same silence with extra steps")
 	}
 	if !strings.Contains(s.Reason, "felhom-backup") {
 		t.Fatalf("the reason must NAME the storage — 'a grant is missing' costs a diagnosis at 07:00; got %q", s.Reason)
 	}
 	if !strings.Contains(s.Reason, "FelhomAgentStore") {
 		t.Fatalf("the reason must name the ROLE to grant, so the fix is in the alert; got %q", s.Reason)
 	}
 }
 // THE TRAP THE LIVE MEASUREMENT CAUGHT, pinned so it cannot be re-introduced: the ungranted answer
 // is not empty and not a 403 — it carries the INHERITED box-wide privileges. A probe that asked
 // "did the path come back?" or "does it have Datastore.Audit?" would report the blinded storage
 // healthy.
 func TestStoreGrant_InheritedPrivilegesAreNotAGrant(t *testing.T) {
 	if len(permUngranted) == 0 {
 		t.Fatal("fixture wrong: the ungranted answer is NOT empty — that is the whole trap")
 	}
 	if permUngranted["Datastore.Audit"] != 1 {
 		t.Fatal("fixture wrong: the ungranted path DOES carry Datastore.Audit, inherited box-wide")
 	}
 	if s := probeWith(permUngranted, "felhom-backup", true); s.Status != capability.StatusDegraded {
 		t.Fatalf("checking for the wrong privilege reports a blinded storage healthy; got %q", s.Status)
 	}
 	// ...and the privilege actually checked is the one whose absence was measured to blind listing.
 	if storeGrantRequiredPriv != "Datastore.AllocateSpace" {
 		t.Fatalf("the probed privilege changed to %q — re-measure before trusting it", storeGrantRequiredPriv)
 	}
 }
 // ── SCENARIO B — a newborn tier is still silent ──────────────────────────────────────────────
 //
 // A storage the agent IS allowed to read but which simply holds no archives yet is HEALTHY. The
 // probe must not look at content at all, or every freshly provisioned box alarms and the signal dies.
 //
 // COMPANION RED-PROOF (observed): make the probe degrade on an empty content listing instead of on
 // the permission — a granted-but-empty storage then reports degraded, i.e. every newborn box alarms.
 func TestStoreGrant_GrantedButEmptyIsHealthy(t *testing.T) {
 	s := probeWith(permGranted, "felhom-pbs", true)
 	if s.Status != capability.StatusOK {
 		t.Fatalf("a readable tier is healthy whether or not it holds archives yet; got %q (%s)", s.Status, s.Reason)
 	}
 	if s.Reason != "" {
 		t.Fatalf("a healthy probe carries no reason; got %q", s.Reason)
 	}
 }
 // ── SCENARIO C — the two states are distinguishable at a glance ──────────────────────────────
 func TestStoreGrant_ForbiddenAndNewbornAreDistinguishable(t *testing.T) {
 	forbidden := probeWith(permUngranted, "felhom-backup", true)
 	newborn := probeWith(permGranted, "felhom-pbs", true)
 	if forbidden.Status == newborn.Status {
 		t.Fatalf("the two states must differ — today both read as 'no settled archive yet'; got %q for both", forbidden.Status)
 	}
 	if forbidden.Name == newborn.Name {
 		t.Fatalf("each tier needs its own capability id, or one tier's fault hides another's; got %q twice", forbidden.Name)
 	}
 }
 // §8.3, weighed once and pinned: a box with NO dedicated target ("local" — host-install's own
 // DEGRADED fallback) must not turn an ordinary configuration into an operator page. It is still
 // probed and still reported; only the paging differs.
 func TestStoreGrant_TheFallbackTargetIsNotCritical(t *testing.T) {
 	if storeGrantCritical("local") {
 		t.Fatal("a box whose backup target is the 'local' fallback must not page the operator about " +
 			"an ordinary, documented configuration")
 	}
 	for _, dedicated := range []string{"felhom-backup", "felhom-pbs", "some-nvme"} {
 		if !storeGrantCritical(dedicated) {
 			t.Fatalf("a DEDICATED target that cannot be read is user-facing and must be critical; %q was not", dedicated)
 		}
 	}
 	// The fallback is still reported — silence for it would be the original defect, scoped smaller.
 	if s := probeWith(permUngranted, "local", storeGrantCritical("local")); s.Status != capability.StatusDegraded {
 		t.Fatalf("the fallback target must still report degraded when unreadable; got %q", s.Status)
 	}
 }
 // A probe that cannot ask must never answer "ok" — unknown reported as healthy is worse than no
 // probe, because it looks like coverage.
 func TestStoreGrant_UnreachablePVEIsDegradedNotOK(t *testing.T) {
 	s := storeGrantStatus(context.Background(), nil, "felhom-backup", true, nil)
 	if s.Status != capability.StatusDegraded {
 		t.Fatalf("an unaskable probe must be DEGRADED, never ok; got %q", s.Status)
 	}
 	if s.Reason == "" {
 		t.Fatal("it must say why it could not ask")
 	}
 }
 // ── SCENARIO H — the seam ────────────────────────────────────────────────────────────────────
 //
 // This project's "built but never wired" count reached six last week. The fix for a SILENCE must not
 // itself be silent. AST, not grep: a commented-out call still contains the string.
 func TestMainWiresTheStoreGrantProbe(t *testing.T) {
 	f := parseMainForWiring(t)
 	var wired bool
 	ast.Inspect(f, func(n ast.Node) bool {
 		call, ok := n.(*ast.CallExpr)
 		if !ok {
 			return true
 		}
 		if id, ok := call.Fun.(*ast.Ident); ok && id.Name == "storeGrantStatuses" {
 			wired = true
 		}
 		return true
 	})
 	if !wired {
 		t.Error("main.go never calls storeGrantStatuses — the probe would exist and report to nobody, " +
 			"which is precisely the silence R-185 is about")
 	}
 }
 // ── R-190 — the grant repairs itself, and the repair is VISIBLE ──────────────────────────────
 //
 // R-190 is a storage grant that demonstrably worked at 04:44 on 2026-08-03 and was gone by 09:24,
 // with a host reinstall, logged `pveum` activity and cluster-log entries all ruled out. The cause is
 // open; the resilience is not conditional on it.
 //
 // The half that matters is the RECORD. R-190's own words: the probe sees the state, nothing sees the
 // transition. A self-repair that leaves only "ok" behind destroys the only evidence a loss happened,
 // so a recurring loss becomes undetectable forever — strictly worse than the fault it fixes.
 // fakeRepairRunner records wrapper invocations and can be made to fail.
 type fakeRepairRunner struct {
 	calls [][]string
 	fail  bool
 }
 func (f *fakeRepairRunner) Run(_ context.Context, name string, args ...string) ([]byte, []byte, error) {
 	f.calls = append(f.calls, append([]string{name}, args...))
 	if f.fail {
 		return nil, []byte("pveum: refused"), errors.New("exit status 2")
 	}
 	return nil, nil, nil
 }
 func newRepairer(f *fakeRepairRunner) *storeGrantRepairer {
 	return &storeGrantRepairer{run: f.Run, log: slog.New(slog.NewTextHandler(io.Discard, nil))}
 }
 // ── SCENARIO F — the repair is BOUNDED ───────────────────────────────────────────────────────
 //
 // COMPANION RED-PROOF (observed 2026-08-04): make mayAttempt always return true (drop the
 // storeGrantRepairMinInterval check) →
 //
 //	--- FAIL: TestGrantRepair_IsBounded
 //	    storegrant_test.go: a repair must not run on every cycle; 5 cycles produced 5 attempt(s)
 //
 // which is a re-grant every report cycle, forever, against a fault an ACL cannot fix. Restored.
 func TestGrantRepair_IsBounded(t *testing.T) {
 	f := &fakeRepairRunner{}
 	r := newRepairer(f)
 	// Jittered, so the series never lands exactly on the interval boundary — a perfectly regular
 	// series is how a threshold test passes its own mutation, which has happened here before.
 	base := time.Date(2026, 8, 4, 9, 17, 43, 0, time.UTC)
 	offsets := []time.Duration{0, 13*time.Minute + 7*time.Second, 27*time.Minute + 51*time.Second,
 		41*time.Minute + 19*time.Second, 55*time.Minute + 3*time.Second}
 	attempts := 0
 	for _, off := range offsets {
 		if r.mayAttempt("felhom-backup", base.Add(off)) {
 			attempts++
 		}
 	}
 	if attempts != 1 {
 		t.Fatalf("a repair must not run on every cycle; %d cycles produced %d attempt(s) within %s",
 			len(offsets), attempts, storeGrantRepairMinInterval)
 	}
 	// ...and once the interval has genuinely passed, it may try again — a bound is not a ban.
 	if !r.mayAttempt("felhom-backup", base.Add(storeGrantRepairMinInterval+2*time.Minute+11*time.Second)) {
 		t.Fatal("after the interval a repair must be allowed again — otherwise one failure disables the repair forever")
 	}
 	// A DIFFERENT tier is not throttled by this one's attempt.
 	if !r.mayAttempt("felhom-pbs", base.Add(time.Minute)) {
 		t.Fatal("the bound must be per tier — one tier's attempt must not suppress another's")
 	}
 }
 // A nil repairer (or one with no runner) never attempts, and never panics.
 func TestGrantRepair_NilIsSafe(t *testing.T) {
 	var r *storeGrantRepairer
 	if r.mayAttempt("felhom-backup", time.Now()) {
 		t.Fatal("a nil repairer must never claim an attempt")
 	}
 	if (&storeGrantRepairer{}).mayAttempt("felhom-backup", time.Now()) {
 		t.Fatal("a repairer with no runner must never claim an attempt")
 	}
 }
 // The repair calls the EXISTING wrapper verb, with the storage id — no new privileged surface.
 func TestGrantRepair_CallsTheExistingWrapperVerb(t *testing.T) {
 	f := &fakeRepairRunner{}
 	r := newRepairer(f)
 	if err := r.repair(context.Background(), "felhom-backup"); err != nil {
 		t.Fatalf("repair should succeed with a healthy runner: %v", err)
 	}
 	if len(f.calls) != 1 {
 		t.Fatalf("exactly one wrapper invocation expected; got %d", len(f.calls))
 	}
 	got := f.calls[0]
 	want := []string{"/usr/local/sbin/felhom-backup-target-apply", "grant", "felhom-backup"}
 	if len(got) != len(want) {
 		t.Fatalf("wrapper argv = %v, want %v", got, want)
 	}
 	for i := range want {
 		if got[i] != want[i] {
 			t.Fatalf("wrapper argv = %v, want %v — the sudoers vector is `grant *`; anything else is a policy change", got, want)
 		}
 	}
 }
 // A repair that FAILS must surface the failure, not swallow it (Scenario E's precondition).
 func TestGrantRepair_FailureIsReturned(t *testing.T) {
 	f := &fakeRepairRunner{fail: true}
 	if err := newRepairer(f).repair(context.Background(), "felhom-backup"); err == nil {
 		t.Fatal("a failed wrapper run must return its error — a repair that cannot run must never read as done")
 	}
 }
 // ── SCENARIO D (the half that matters) — the REPAIR MUST BE VISIBLE ──────────────────────────
 //
 // A repair that leaves only "ok" behind is worse than the fault: the tier works, and the fact that a
 // permission vanished is gone with it. R-190 exists because nothing saw the transition.
 //
 // The channel is the hub's EXISTING ok→degraded→ok edge (§8.5) — nothing new was built. That only
 // works if the agent deliberately reports ONE degraded cycle after repairing, and if the explanation
 // rides the field the hub actually puts in the operator's e-mail. The hub's message is built from the
 // capability NAME and FEATURE (`internal/monitor/host_capability.go` emitTransition) — **not** from
 // Reason — so the Feature must carry it.
 //
 // COMPANION RED-PROOF (observed 2026-08-04): after a successful repair, report ok instead —
 //
 //	s.Status = capability.StatusOK; s.Feature unchanged
 //
 // →	--- FAIL: TestGrantRepair_ARepairedGrantIsReportedAsATransition
 //
 //	storegrant_test.go: a self-repair must still report DEGRADED for one cycle so the hub raises
 //	its edge; got "ok" — the loss would be invisible
 //
 // i.e. exactly the silence R-190 is about. Restored.
 func TestGrantRepair_ARepairedGrantIsReportedAsATransition(t *testing.T) {
 	// THE PRODUCTION verdict, not a copy of it. An earlier draft of this test built the Status
 	// itself and asserted its own construction — it would have passed while production reported ok,
 	// which is precisely the silence being guarded against.
 	if pre := probeWith(permUngranted, "felhom-backup", true); pre.Status != capability.StatusDegraded {
 		t.Fatalf("precondition: a missing grant is degraded; got %q", pre.Status)
 	}
 	s := storeGrantRepairedVerdict("felhom-backup", true)
 	if s.Status != capability.StatusDegraded {
 		t.Fatalf("a self-repair must still report DEGRADED for one cycle so the hub raises its edge; "+
 			"got %q — the loss would be invisible", s.Status)
 	}
 	// The hub e-mails the FEATURE text. If the explanation is not there, the operator is told a
 	// capability was degraded and never learns it repaired itself or that anything vanished.
 	for _, want := range []string{"MISSING", "RESTORED", "felhom-backup", "R-190"} {
 		if !strings.Contains(s.Feature, want) {
 			t.Fatalf("the Feature text is what the hub puts in the operator's e-mail; it must contain %q. Got: %s", want, s.Feature)
 		}
 	}
 	if !s.Critical {
 		t.Fatal("the transition must be CRITICAL or the hub does not alert on it at all")
 	}
 }
 // ── SCENARIO H — the seam ────────────────────────────────────────────────────────────────────
 //
 // The wrapper's `grant` verb is itself a "built but never wired" example: it exists, is
 // sudoers-permitted for any id, and had only ever been called at storage CREATION. The repair must
 // not become the seventh instance. AST, not grep — a commented-out call still contains the string.
 func TestMainWiresTheGrantRepair(t *testing.T) {
 	f := parseMainForWiring(t)
 	var built, passed bool
 	ast.Inspect(f, func(n ast.Node) bool {
 		switch node := n.(type) {
 		case *ast.CompositeLit:
 			if id, ok := node.Type.(*ast.Ident); ok && id.Name == "storeGrantRepairer" {
 				built = true
 			}
 		case *ast.CallExpr:
 			if id, ok := node.Fun.(*ast.Ident); ok && id.Name == "storeGrantStatuses" && len(node.Args) == 4 {
 				if a, ok := node.Args[3].(*ast.Ident); ok && a.Name == "grantRepairer" {
 					passed = true
 				}
 			}
 		}
 		return true
 	})
 	if !built {
 		t.Error("main.go never constructs a storeGrantRepairer — nothing would ever repair a lost grant")
 	}
 	if !passed {
 		t.Error("storeGrantStatuses is not passed the repairer — the probe would detect the loss and " +
 			"leave it, which is v0.123.0's behaviour and not R-190's mitigation")
 	}
 }
 // The transition must survive a probe that is NOT the one feeding the hub.
 //
 // MEASURED LIVE 2026-08-04, and this test exists because the first implementation failed it in
 // production while every unit test passed: `probeAll` is called independently by the self-check LOG
 // and by the collector building a host-report. The repairing call was the log's; the report three
 // seconds later found the grant present and reported `ok`. The agent's journal had the record and the
 // hub had nothing — the exact silence R-190 is about, re-created inside its own mitigation.
 //
 // COMPANION RED-PROOF (observed): delete the `recentlyRepaired` branch from the healthy path →
 //
 //	--- FAIL: TestGrantRepair_TransitionSurvivesALaterProbe
 //	    storegrant_test.go: a probe AFTER the repair must still report the transition; got "ok" —
 //	    the host-report would carry ok and the operator would never learn the grant vanished
 //
 // Restored.
 func TestGrantRepair_TransitionSurvivesALaterProbe(t *testing.T) {
 	r := newRepairer(&fakeRepairRunner{})
 	// Jittered, never landing on the window boundary.
 	repairedAt := time.Date(2026, 8, 4, 9, 39, 34, 0, time.UTC)
 	r.noteRepaired("felhom-backup", repairedAt)
 	// The DECISION a later probe makes — the production function, not the helper it calls. An
 	// earlier draft asserted `recentlyRepaired` directly and its red-proof PASSED, because removing
 	// the latch's USE left the helper untouched.
 	healthy := probeWith(permGranted, "felhom-backup", true)
 	if healthy.Status != capability.StatusOK {
 		t.Fatalf("precondition: a granted tier is ok; got %q", healthy.Status)
 	}
 	got := storeGrantHealthyVerdict("felhom-backup", true,
 		healthy, r.recentlyRepaired("felhom-backup", repairedAt.Add(3*time.Second)))
 	if got.Status != capability.StatusDegraded {
 		t.Fatalf("a probe AFTER the repair must still report the transition; got %q — the host-report "+
 			"would carry ok and the operator would never learn the grant vanished", got.Status)
 	}
 	if !strings.Contains(got.Feature, "RESTORED") {
 		t.Fatalf("the later probe must carry the explanation into the hub's e-mail; got: %s", got.Feature)
 	}
 	// Outside the window it reports plain ok again.
 	late := storeGrantHealthyVerdict("felhom-backup", true,
 		healthy, r.recentlyRepaired("felhom-backup", repairedAt.Add(storeGrantRepairReportWindow+time.Minute)))
 	if late.Status != capability.StatusOK {
 		t.Fatalf("outside the window a healthy tier reports ok; got %q — a permanent degraded state "+
 			"would be its own false alarm", late.Status)
 	}
 	if !r.recentlyRepaired("felhom-backup", repairedAt.Add(14*time.Minute+37*time.Second)) {
 		t.Fatal("the latch must outlast the 900s hub report interval, or the record never reaches the hub")
 	}
 	// ...and it clears on its own rather than latching a box degraded forever.
 	if r.recentlyRepaired("felhom-backup", repairedAt.Add(storeGrantRepairReportWindow+time.Minute+7*time.Second)) {
 		t.Fatal("the latch must clear — a permanent degraded state would be its own false alarm")
 	}
 	// It is per tier.
 	if r.recentlyRepaired("felhom-pbs", repairedAt.Add(time.Second)) {
 		t.Fatal("one tier's repair must not latch another tier's status")
 	}
 	// The window MUST exceed the report interval — the property, asserted rather than assumed.
 	if storeGrantRepairReportWindow <= 15*time.Minute {
 		t.Fatalf("the report window (%s) must exceed the 900s hub report interval, or a transition can "+
 			"be missed entirely", storeGrantRepairReportWindow)
 	}
 }
@@ -442,7 +442,7 @@ func TestRestoreTestState_ArchiveRoundTrips(t *testing.T) {
 	path := filepath.Join(t.TempDir(), "rt.json")
 	now := time.Now().UTC().Truncate(time.Second)
 	st := NewRestoreTestState(path)
-	if err := st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/x", now); err != nil {
+	if err := st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/x", "pbs", "boot+running", now); err != nil {
 		t.Fatal(err)
 	}
 	re := NewRestoreTestState(path)
@@ -475,7 +475,7 @@ func TestDue_NothingDueStillNamesEveryTiersVerdict(t *testing.T) {
 	}}
 	h := newDueHarness(t, day0.AddDate(0, 0, 1), 24*time.Hour, true, []string{"local", "felhom-pbs"}, ts)
 	// Prove the local tier so NOTHING is due.
-	if err := h.st.RecordSuccess("local", "local:backup/a.tar.zst", h.clock); err != nil {
+	if err := h.st.RecordSuccess("local", "local:backup/a.tar.zst", "local", "boot+running", h.clock); err != nil {
 		t.Fatal(err)
 	}
@@ -505,3 +505,89 @@ func TestDue_VerdictSummaryNamesAnUnknownTier(t *testing.T) {
 		t.Fatalf("an unlistable tier must read as UNKNOWN with its error; got %q", got)
 	}
 }
 // ── R-189 — the persisted proof must be REPORTABLE, and must refuse to lie ───────────────────
 //
 // A proof held only in the in-memory store dies with the process, and under per-archive due-ness the
 // agent will not repeat the work. So the persisted record has to be able to become a host-report
 // entry — without inventing anything it does not know.
 //
 // COMPANION RED-PROOF (observed 2026-08-03): drop the `reportable()` filter from
 // ProvenRestoreTests, so a pre-R-189 record (archive but no tier) is emitted →
 //
 //	--- FAIL: TestProvenRestoreTests_RefusesToReportWhatItCannotDescribe
 //	    restoretest_due_test.go: a record with no TIER must not be reported (the hub keys its
 //	    per-tier proof on it); got [{... SourceTier: ...}]
 //
 // Restored.
 func TestProvenRestoreTests_RefusesToReportWhatItCannotDescribe(t *testing.T) {
 	path := filepath.Join(t.TempDir(), "rt.json")
 	// v1 (a bare time), v2 (archive, no tier) and v3 (complete) side by side — every shape this
 	// file has ever had, which is what a real box carries after two upgrades.
 	legacy := `{
 	  "old-v1":     "2026-07-30T02:11:07Z",
 	  "old-v2":     {"archive":"felhom-backup:backup/vzdump-lxc-9201-a.tar.zst","proven_at":"2026-08-01T04:41:58Z"},
 	  "felhom-pbs": {"archive":"felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z","tier":"pbs","verified":"boot+running","proven_at":"2026-08-03T13:25:14Z"}
 	}`
 	if err := writeFileForTest(path, legacy); err != nil {
 		t.Fatal(err)
 	}
 	got := NewRestoreTestState(path).ProvenRestoreTests(context.Background())
 	if len(got) != 1 {
 		t.Fatalf("only the record that can be described honestly may be reported; got %d: %+v", len(got), got)
 	}
 	e := got[0]
 	if e.SourceTier != "pbs" {
 		t.Fatalf("a record with no TIER must not be reported (the hub keys its per-tier proof on it); got %+v", got)
 	}
 	if e.SourceArchive != "felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z" || !e.Pass {
 		t.Fatalf("the reported entry must be the stored proof, unchanged; got %+v", e)
 	}
 	if e.TestedAt != "2026-08-03T13:25:14Z" {
 		t.Fatalf("the entry must carry the time the run passed, not now(); got %q", e.TestedAt)
 	}
 	if e.Verified != "boot+running" {
 		t.Fatalf("what the run verified must survive the round trip; got %q", e.Verified)
 	}
 	// Run mechanics are NOT invented: an absent duration is not a claim, a fabricated one would be.
 	if e.DurationSeconds != 0 || e.ScratchVMID != 0 {
 		t.Fatalf("the re-report must not invent run mechanics it never stored; got duration=%v scratch=%d",
 			e.DurationSeconds, e.ScratchVMID)
 	}
 	// The legacy records still serve the DUE-check, which is a separate question from reporting.
 	if _, ok := NewRestoreTestState(path).ProvenArchive("old-v2"); !ok {
 		t.Fatal("a v2 record must still answer the due-check even though it cannot be reported")
 	}
 }
 // A tier proved through the SCHEDULER (not by hand) lands in the state complete enough to report —
 // the production path, not a hand-built fixture.
 func TestScheduler_ProofIsRecordedReportably(t *testing.T) {
 	ts := &tierStorage{archives: map[string][]archiveStub{"felhom-pbs": {{volid: "felhom-pbs:backup/ct/9201/w0", landed: day0}}}}
 	h := newDueHarness(t, day0.AddDate(0, 0, 1).Add(97*time.Minute), 24*time.Hour, true, []string{"felhom-pbs"}, ts)
 	// The fake runner echoes the spec's tier; give the spec a tier the way main.go does.
 	h.s.spec = func(_ context.Context, archive string) reconcile.RestoreTestSpec {
 		return reconcile.RestoreTestSpec{RestoreStorage: "local-lvm", ScratchMin: 990000, ScratchMax: 990009, SourceTier: "pbs"}
 	}
 	h.s.tick(context.Background())
 	got := h.st.ProvenRestoreTests(context.Background())
 	if len(got) != 1 {
 		t.Fatalf("a scheduled pass must leave a REPORTABLE proof; got %d: %+v", len(got), got)
 	}
 	if got[0].SourceTier != "pbs" || got[0].SourceArchive != "felhom-pbs:backup/ct/9201/w0" {
 		t.Fatalf("the proof must name the tier and the archive the run used; got %+v", got[0])
 	}
 }
 // A FAILED run leaves nothing to report — the asymmetry of §8.1, asserted rather than assumed.
 func TestScheduler_AFailureLeavesNoPersistedProof(t *testing.T) {
 	ts := &tierStorage{archives: map[string][]archiveStub{"felhom-pbs": {{volid: "felhom-pbs:backup/ct/9201/w0", landed: day0}}}}
 	h := newDueHarness(t, day0.AddDate(0, 0, 1).Add(97*time.Minute), 24*time.Hour, false, []string{"felhom-pbs"}, ts)
 	h.s.tick(context.Background())
 	if got := h.st.ProvenRestoreTests(context.Background()); len(got) != 0 {
 		t.Fatalf("a FAILED run must persist nothing — a failing tier is retried, and a stored failure "+
 			"would outlive the fault; got %+v", got)
 	}
 }
@@ -1,12 +1,15 @@
 package backup
 import (
 	"context"
 	"encoding/json"
 	"os"
 	"path/filepath"
 	"sort"
 	"sync"
 	"time"
 	"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
 )
 // RestoreTestState persists the last SUCCESSFUL restore-test per backup tier.
@@ -49,16 +52,45 @@ type RestoreTestState struct {
 	last map[string]provenTier // target id → what was last PROVEN on that tier
 }
-// provenTier is one tier's proof: the archive that passed, and when it passed.
+// provenTier is one tier's proof: the archive that passed, which tier it was, what was verified,
 // and when.
 //
 // R-189 added `Tier` and `Verified`. Until then this record could answer the DUE-check but could not
 // be REPORTED, and being reportable is what closes R-189: a proof held only in the in-memory result
 // store vanishes on restart, and under per-archive due-ness the box will not repeat the work, so the
 // hub can stay ignorant of a real success until the next archive generation.
 //
 // `Tier` is stored rather than derived because it is known for certain at proof time (the run's own
 // spec used it to choose the restore timeout) and deriving it later would need a storage-type lookup
 // at report-building time — a network call that can fail, on a path where failing means mis-labelling
 // a proof. Store what you knew when you knew it.
 type provenTier struct {
-	Archive string    // volid of the archive that PASSED; "" = a legacy record with no archive
+	Archive  string    // volid of the archive that PASSED; "" = a legacy record with no archive
-	At      time.Time // when that run passed (UTC)
+	Tier     string    // "local" | "pbs" — as the run reported it; "" = pre-R-189 record
 	Verified string    // what the run verified (e.g. "boot+running"); "" = pre-R-189 record
 	At       time.Time // when that run passed (UTC)
 }
-// provenTierJSON is the on-disk shape (R-86). The legacy shape was a bare RFC3339 STRING per
+// reportable reports whether this record can be re-reported to the hub as a restore-test result.
-// target; both are read, only this one is written — see NewRestoreTestState.
+//
 // It needs BOTH the archive and the tier: the hub keys its edge-triggered failure state on the
 // archive and its per-tier proof lookup on the tier, so an entry missing either is not a usable
 // proof — and emitting one anyway would be a report the hub cannot act on, dressed as evidence.
 // A pre-R-189 record is therefore silently not reported; the tier's next real proof fills it in.
 func (p provenTier) reportable() bool { return p.Archive != "" && p.Tier != "" }
 // provenTierJSON is the on-disk shape. Two older shapes are read and neither is written:
 //
 //	v1 (pre-R-86)  "<target>": "<RFC3339>"                     — a time, no archive
 //	v2 (R-86)      "<target>": {archive, proven_at}            — due-check usable, not reportable
 //	v3 (R-189)     "<target>": {archive, tier, verified, …}    — both
 //
 // Fields absent in an older file unmarshal to "", which is exactly the "no usable proof" signal the
 // readers above test for — the migration needs no version number because the absence IS the answer.
 type provenTierJSON struct {
 	Archive  string `json:"archive"`
 	Tier     string `json:"tier,omitempty"`
 	Verified string `json:"verified,omitempty"`
 	ProvenAt string `json:"proven_at"`
 }
@@ -99,21 +131,31 @@ func NewRestoreTestState(path string) *RestoreTestState {
 		if perr != nil {
 			continue
 		}
-		s.last[target] = provenTier{Archive: cur.Archive, At: t.UTC()}
+		s.last[target] = provenTier{Archive: cur.Archive, Tier: cur.Tier, Verified: cur.Verified, At: t.UTC()}
 	}
 	return s
 }
-// RecordSuccess stamps a tier as proven at t, naming the ARCHIVE that passed. Only call this for a
+// RecordSuccess stamps a tier as proven at t, naming the ARCHIVE that passed, the TIER the run
-// PASSING restore-test — the archive is what makes the tier not-due, so recording one for a failed
+// reported, and what it verified. Only call this for a PASSING restore-test — the archive is what
-// run would retire the archive unproven.
+// makes the tier not-due, so recording one for a failed run would retire the archive unproven.
-func (s *RestoreTestState) RecordSuccess(target, archive string, t time.Time) error {
+//
 // ONLY SUCCESSES ARE PERSISTED, AND THE ASYMMETRY IS DELIBERATE (R-189 §8.1). Say it here because
 // the next reader will notice failures are absent and try to "fix" it:
 //
 //	a SUCCESS suppresses future work — a proven archive is never re-tested, so a lost proof leaves
 //	the system quietly less tested than it believes. It must survive a restart.
 //
 //	a FAILURE causes future work — a failing tier stays due and is retried at the next evaluation,
 //	so a lost failure heals itself within one interval. Persisting it would do the opposite of
 //	helping: a healed tier would keep reporting a failure that is no longer true.
 func (s *RestoreTestState) RecordSuccess(target, archive, tier, verified string, t time.Time) error {
 	if target == "" {
 		return nil
 	}
 	s.mu.Lock()
 	defer s.mu.Unlock()
-	s.last[target] = provenTier{Archive: archive, At: t.UTC()}
+	s.last[target] = provenTier{Archive: archive, Tier: tier, Verified: verified, At: t.UTC()}
 	return s.saveLocked()
 }
@@ -138,7 +180,13 @@ func (s *RestoreTestState) ProvenArchive(target string) (string, bool) {
 	return p.Archive, true
 }
-// Snapshot returns a copy of the last-proven TIMES — for the host-report gauge.
+// Snapshot returns a copy of the last-proven TIMES.
 //
 // It carried the comment "for the host-report gauge" from the day it was written and **had no caller
 // at all** until R-189 — a seam built and never wired, and an invariant asserted in a comment with
 // nothing pinning it, in one method. The host report is now fed by ProvenRestoreTests below, which
 // carries the archive and the tier that a bare timestamp cannot. This stays for callers that want
 // only the times; if it acquires none, delete it rather than let it claim a purpose again.
 func (s *RestoreTestState) Snapshot() map[string]time.Time {
 	s.mu.Lock()
 	defer s.mu.Unlock()
@@ -149,6 +197,41 @@ func (s *RestoreTestState) Snapshot() map[string]time.Time {
 	return out
 }
 // ProvenRestoreTests renders the persisted proofs as host-report entries — the R-189 fix.
 //
 // It satisfies hub.RestoreTestReporter's shape, so the collector can merge these with the in-memory
 // results. What it emits is a RE-REPORT of a run that really happened, not a synthesis:
 //
 //   - `Pass` is true because ONLY successes are stored (RecordSuccess is the sole writer);
 //   - `SourceArchive`, `SourceTier`, `Verified` and `TestedAt` are the values that run reported;
 //   - the run mechanics (scratch VMID, duration, warnings) are NOT re-invented. An absent duration
 //     is not a claim; a fabricated one would be.
 //
 // A record that cannot be reported honestly is omitted rather than padded — see provenTier.reportable.
 // **A tier with no usable proof produces NO entry**: an unproven tier reading as proven would be a
 // worse defect than the one this fixes.
 func (s *RestoreTestState) ProvenRestoreTests(context.Context) []hub.RestoreTest {
 	s.mu.Lock()
 	defer s.mu.Unlock()
 	out := make([]hub.RestoreTest, 0, len(s.last))
 	for _, p := range s.last {
 		if !p.reportable() {
 			continue
 		}
 		out = append(out, hub.RestoreTest{
 			SourceArchive: p.Archive,
 			SourceTier:    p.Tier,
 			Pass:          true,
 			Verified:      p.Verified,
 			TestedAt:      p.At.UTC().Format(time.RFC3339),
 		})
 	}
 	// Deterministic order: the report is compared byte-wise by the contract test, and Go's map
 	// iteration is randomised.
 	sort.Slice(out, func(i, j int) bool { return out[i].SourceTier < out[j].SourceTier })
 	return out
 }
 // OldestFirst orders targets by "least recently proven first"; never-proven sorts FIRST.
 //
 // This is the operator's 2026-07-26 ruling (Option 1): self-balancing, no new config knob, and it
@@ -185,7 +268,10 @@ func (s *RestoreTestState) OldestFirst(targets []string) []string {
 func (s *RestoreTestState) saveLocked() error {
 	raw := make(map[string]provenTierJSON, len(s.last))
 	for target, p := range s.last {
-		raw[target] = provenTierJSON{Archive: p.Archive, ProvenAt: p.At.UTC().Format(time.RFC3339)}
+		raw[target] = provenTierJSON{
 			Archive: p.Archive, Tier: p.Tier, Verified: p.Verified,
 			ProvenAt: p.At.UTC().Format(time.RFC3339),
 		}
 	}
 	data, err := json.MarshalIndent(raw, "", "  ")
 	if err != nil {
@@ -303,11 +303,11 @@ func TestOldestFirst_Ordering(t *testing.T) {
 			t.Fatalf("unexpected: %v", got)
 		}
 	}
-	_ = st.RecordSuccess("local", "local:backup/a.tar.zst", now)
+	_ = st.RecordSuccess("local", "local:backup/a.tar.zst", "local", "boot+running", now)
 	if got := st.OldestFirst([]string{"local", "felhom-pbs"}); got[0] != "felhom-pbs" {
 		t.Fatalf("a never-proven tier must sort before a proven one; got %v", got)
 	}
-	_ = st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/b", now.Add(time.Hour))
+	_ = st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/b", "pbs", "boot+running", now.Add(time.Hour))
 	if got := st.OldestFirst([]string{"local", "felhom-pbs"}); got[0] != "local" {
 		t.Fatalf("the least recently proven must sort first; got %v", got)
 	}
@@ -318,8 +318,8 @@ func TestOldestFirst_Ordering(t *testing.T) {
 func TestOldestFirst_DeterministicOnTies(t *testing.T) {
 	st := NewRestoreTestState(filepath.Join(t.TempDir(), "rt.json"))
 	now := time.Now().UTC()
-	_ = st.RecordSuccess("b-tier", "b:archive", now)
+	_ = st.RecordSuccess("b-tier", "b:archive", "local", "boot+running", now)
-	_ = st.RecordSuccess("a-tier", "a:archive", now)
+	_ = st.RecordSuccess("a-tier", "a:archive", "local", "boot+running", now)
 	for i := 0; i < 20; i++ {
 		if got := st.OldestFirst([]string{"b-tier", "a-tier"}); got[0] != "a-tier" {
 			t.Fatalf("tie-break must be deterministic; iteration %d gave %v", i, got)
@@ -334,7 +334,7 @@ func TestRestoreTestState_PersistenceAndCorruption(t *testing.T) {
 	now := time.Now().UTC().Truncate(time.Second)
 	st := NewRestoreTestState(path)
-	if err := st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/x", now); err != nil {
+	if err := st.RecordSuccess("felhom-pbs", "felhom-pbs:backup/ct/9201/x", "pbs", "boot+running", now); err != nil {
 		t.Fatal(err)
 	}
 	reopened := NewRestoreTestState(path)
@@ -218,7 +218,11 @@ func (s *Scheduler) tick(ctx context.Context) {
 	if rt.Pass && s.rtState != nil && target != "" {
 		// R-86: the ARCHIVE is recorded, not merely the time — that is what makes the tier
 		// not-due until a NEWER archive settles, and what makes a proof survive a restart.
-		if err := s.rtState.RecordSuccess(target, archive, s.now()); err != nil {
+		// R-189: the TIER and what was VERIFIED go with it, so the proof can be RE-REPORTED after a
 		// restart. Both come from the run's own result, never re-derived — `rt.SourceTier` is what
 		// this run was actually judged as, and deriving it later would need a storage lookup that
 		// can fail on the one path where failing means mislabelling a proof.
 		if err := s.rtState.RecordSuccess(target, archive, rt.SourceTier, rt.Verified, s.now()); err != nil {
 			s.logger.Warn("backup: could not persist the restore-test proof state", "target", target, "err", err)
 		}
 	}
@@ -10,8 +10,22 @@ import (
 // Store holds the agent's LATEST backup result per target and the latest restore-test
 // result — the point-in-time state the host-report surfaces. It is updated by the backup
 // runner + the restore-test scheduler/selftest and read by the collector via the hub
-// BackupReporter / RestoreTestReporter seams. In-memory (lost on restart; the cadence
+// BackupReporter / RestoreTestReporter seams. In-memory and mutex-guarded for the concurrent
-// re-populates) and mutex-guarded for the concurrent collector vs scheduler access.
+// collector vs scheduler access.
 //
 // **"lost on restart; the cadence re-populates" — that sentence used to be here and it is now
 // FALSE for restore-tests (R-189, 2026-08-03).** It was true while a timer re-tested every tier
 // daily. Under R-86's per-archive due-check the agent will NOT re-test an archive it has already
 // proven, so a proof lost to a restart is not repeated until the next archive generation — a week on
 // the offsite tier — and the hub reports that tier unproven throughout. Observed, not predicted: a
 // real 14.5 GB offsite restore passed, the agent was restarted 2 m 43 s later for a deploy, and two
 // consecutive host-reports carried `0 restore-tests`.
 //
 // The durable half is `RestoreTestState` (on disk, per tier, with the archive) and the collector
 // merges the two — see hub.ProvenRestoreTestReporter. This store remains the ONLY place a FAILURE is
 // recorded, and that asymmetry is deliberate: a failing tier stays due and is retried, so a lost
 // failure heals itself, while a lost success leaves the system quietly less tested than it believes.
 // Backups are unaffected — their freshness has a ground truth on the storage (R-84).
 type Store struct {
 	mu       sync.Mutex
 	byTarget map[string]hub.Backup // latest backup per target id
@@ -47,6 +47,22 @@ type RestoreTestReporter interface {
 	RestoreTests(ctx context.Context) []RestoreTest
 }
 // ProvenRestoreTestReporter is the DURABLE half of the restore-test signal (R-189).
 //
 // RestoreTestReporter above is backed by an in-memory store whose own comment used to read "lost on
 // restart; the cadence re-populates". That was true while a timer re-tested every tier daily. It
 // stopped being true on 2026-08-03: under per-archive due-ness the agent will not re-test an archive
 // it has already proven, so a proof lost to a restart is not repeated for a whole archive generation
 // — a week on the offsite tier — and the hub reports the tier unproven the entire time.
 //
 // Observed, not predicted: a real 14.5 GB offsite restore passed at 15:25:14, the agent was restarted
 // 2 m 43 s later for a deploy, and the hub logged `0 restore-tests` on the next two reports.
 //
 // (*backup.RestoreTestState).ProvenRestoreTests satisfies this. nil → the merge is a no-op.
 type ProvenRestoreTestReporter interface {
 	ProvenRestoreTests(ctx context.Context) []RestoreTest
 }
 // PBSReporter is the slice-6-Phase-B seam the pbs verify loop plugs into (same pattern).
 // Returns the agent's latest-known PBS snapshot inventory + verify-state. nil → empty.
 type PBSReporter interface {
@@ -79,6 +95,7 @@ type Collector struct {
 	storage      StorageObserver
 	backups      BackupReporter
 	restoreTests RestoreTestReporter
 	provenTests  ProvenRestoreTestReporter
 	pbs          PBSReporter
 	temp         TempReader                                    // slice 9: host CPU/chassis temp (nil-safe → nil temp)
 	capProbe     func(ctx context.Context) []capability.Status // v0.44.0: privileged-capability self-check (nil → empty)
@@ -427,16 +444,90 @@ func (c *Collector) collectBackups(ctx context.Context) []Backup {
 	return []Backup{}
 }
 // collectRestoreTests merges the in-memory result with the PERSISTED per-tier proofs (R-189).
 //
 // The rule is ONE ENTRY PER TIER, NEWEST WINS, and it falls out of what each source means rather
 // than from a preference between them:
 //
 //   - the in-memory store holds this process's latest run, pass OR fail. A failure exists nowhere
 //     else and must always reach the hub — a failing tier is retried at the next evaluation, so its
 //     record is short-lived by design;
 //   - the persisted state holds the last SUCCESS per tier and survives a restart.
 //
 // Comparing by TestedAt gives the right answer in every case without special-casing: a fresh failure
 // beats an older stored success (the failure is the news), a stored success beats a stale in-memory
 // entry after a restart, and a tier proved twice never appears twice — two entries for one tier would
 // read at the hub as two tests.
 //
 // A tier with no usable persisted proof contributes NOTHING. Reporting an unproven tier as proven
 // would be a worse defect than the one this closes.
 func (c *Collector) collectRestoreTests(ctx context.Context) []RestoreTest {
-	if c.restoreTests == nil {
+	out := []RestoreTest{}
-		return []RestoreTest{}
+	if c.restoreTests != nil {
 		if r := c.restoreTests.RestoreTests(ctx); r != nil {
 			out = append(out, r...)
 		}
 	}
-	if r := c.restoreTests.RestoreTests(ctx); r != nil {
+	if c.provenTests == nil {
-		return r
+		return out
 	}
-	return []RestoreTest{}
+
 	// Index what we already have by tier, keeping the newest per tier.
 	best := map[string]int{} // tier → index into out
 	for i, rt := range out {
 		if rt.SourceTier == "" {
 			continue // untiered entry: never deduped, never overwritten — we cannot say what it is
 		}
 		if j, seen := best[rt.SourceTier]; !seen || newerRestoreTest(rt, out[j]) {
 			best[rt.SourceTier] = i
 		}
 	}
 	for _, p := range c.provenTests.ProvenRestoreTests(ctx) {
 		if p.SourceTier == "" {
 			continue // not usable as a per-tier proof; the state layer already filters these
 		}
 		i, seen := best[p.SourceTier]
 		if !seen {
 			out = append(out, p)
 			best[p.SourceTier] = len(out) - 1
 			continue
 		}
 		if newerRestoreTest(p, out[i]) {
 			out[i] = p
 		}
 	}
 	return out
 }
 // newerRestoreTest reports whether a was tested after b. An unparseable or absent timestamp is
 // treated as OLDER, so a malformed entry can never displace a good one.
 func newerRestoreTest(a, b RestoreTest) bool {
 	ta, aok := parseRestoreTestedAt(a.TestedAt)
 	tb, bok := parseRestoreTestedAt(b.TestedAt)
 	if !aok {
 		return false
 	}
 	if !bok {
 		return true
 	}
 	return ta.After(tb)
 }
 func parseRestoreTestedAt(s string) (time.Time, bool) {
 	t, err := time.Parse(time.RFC3339, s)
 	if err != nil {
 		return time.Time{}, false
 	}
 	return t.UTC(), true
 }
 // SetProvenRestoreTests wires the durable proof source. It is a setter rather than a constructor
 // argument because the persisted state is opened later in main() than the collector is built; the
 // same shape as the other late-wired seams here. **The wiring is asserted by an AST test** — the
 // method it feeds carried a doc comment naming a "host-report gauge" for weeks with no caller at
 // all, and this fix must not become the next instance of that.
 func (c *Collector) SetProvenRestoreTests(p ProvenRestoreTestReporter) { c.provenTests = p }
 // collectPBSSnapshots reads the latest PBS snapshot inventory via the seam (nil → empty).
 func (c *Collector) collectPBSSnapshots(ctx context.Context) []PBSSnapshot {
 	if c.pbs == nil {
@@ -0,0 +1,213 @@
 package hub
 import (
 	"context"
 	"testing"
 	"time"
 )
 // R-189 — a passing restore-test must survive an agent restart and reach the hub.
 //
 // THE OBSERVATION THIS EXISTS FOR (2026-08-03, demo-felhom): a real 14.5 GB offsite restore-test
 // PASSED at 15:25:14; the agent was restarted 2 m 43 s later for a deploy; the hub logged
 // `0 restore-tests` on the next two host-reports. The in-memory store's own comment said "lost on
 // restart; the cadence re-populates", which was true under a timer and stopped being true when R-86
 // made the agent refuse to re-test an archive it has already proven.
 //
 // Timestamps here carry JITTER (odd minutes and seconds, not round hours) — yesterday a test was
 // hollow because a perfectly regular series landed exactly on a threshold and passed under the
 // mutation it was meant to catch.
 type fakeLatest struct{ tests []RestoreTest }
 func (f *fakeLatest) RestoreTests(context.Context) []RestoreTest { return f.tests }
 type fakeProven struct{ tests []RestoreTest }
 func (f *fakeProven) ProvenRestoreTests(context.Context) []RestoreTest { return f.tests }
 func rt(tier, archive string, pass bool, at time.Time) RestoreTest {
 	return RestoreTest{
 		SourceArchive: archive, SourceTier: tier, Pass: pass,
 		Verified: "boot+running", TestedAt: at.UTC().Format(time.RFC3339),
 	}
 }
 // mergeCollector builds a Collector with only the two restore-test seams wired — the merge is what
 // is under test, not the rest of the collection.
 func mergeCollector(latest, proven []RestoreTest) *Collector {
 	c := &Collector{}
 	if latest != nil {
 		c.restoreTests = &fakeLatest{tests: latest}
 	}
 	if proven != nil {
 		c.provenTests = &fakeProven{tests: proven}
 	}
 	return c
 }
 func findTier(got []RestoreTest, tier string) (RestoreTest, int) {
 	var hit RestoreTest
 	n := 0
 	for _, e := range got {
 		if e.SourceTier == tier {
 			hit, n = e, n+1
 		}
 	}
 	return hit, n
 }
 // ── SCENARIO A — a proof survives a restart and reaches the hub ──────────────────────────────
 //
 // COMPANION RED-PROOF (observed 2026-08-03): delete the `c.provenTests` merge from
 // collectRestoreTests (return the in-memory slice as it used to) →
 //
 //	--- FAIL: TestMerge_ProofSurvivesARestart
 //	    restoretest_merge_test.go: after a restart the persisted proof must be reported; got 0 entr(ies)
 //
 // which is exactly the live observation: `0 restore-tests`. Restored.
 func TestMerge_ProofSurvivesARestart(t *testing.T) {
 	provenAt := time.Date(2026, 8, 3, 13, 25, 14, 0, time.UTC) // the real run's timestamp
 	// After a restart the in-memory store is EMPTY — this is the whole point.
 	c := mergeCollector([]RestoreTest{}, []RestoreTest{
 		rt("pbs", "felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z", true, provenAt),
 	})
 	got := c.collectRestoreTests(context.Background())
 	if len(got) != 1 {
 		t.Fatalf("after a restart the persisted proof must be reported; got %d entr(ies): %+v", len(got), got)
 	}
 	e := got[0]
 	if e.SourceArchive != "felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z" {
 		t.Fatalf("the entry must name the archive that was proven — the hub keys on it; got %q", e.SourceArchive)
 	}
 	if e.SourceTier != "pbs" || !e.Pass {
 		t.Fatalf("the entry must be a PASS on the tier it was proven on; got tier=%q pass=%v", e.SourceTier, e.Pass)
 	}
 	if e.TestedAt != provenAt.Format(time.RFC3339) {
 		t.Fatalf("the entry must carry the ORIGINAL test time, not now(); got %q", e.TestedAt)
 	}
 }
 // ── SCENARIO B — the report does not invent a pass ───────────────────────────────────────────
 //
 // COMPANION RED-PROOF (observed): make the state layer emit an entry for an unproven tier (drop the
 // `reportable()` filter in ProvenRestoreTests, so a legacy record with no archive is emitted) — the
 // equivalent at this layer is a proven-source that returns an entry for a tier nothing proved, which
 // this test injects directly and the assertion below rejects.
 func TestMerge_NeverInventsAPassForAnUnprovenTier(t *testing.T) {
 	// Nothing proven anywhere: no in-memory result, no persisted proof.
 	c := mergeCollector([]RestoreTest{}, []RestoreTest{})
 	if got := c.collectRestoreTests(context.Background()); len(got) != 0 {
 		t.Fatalf("a tier with no proof must produce NO entry — an unproven tier reading as proven is "+
 			"worse than the defect being fixed; got %+v", got)
 	}
 	// And an entry the state layer could not describe (no tier) is never promoted into a proof.
 	c2 := mergeCollector([]RestoreTest{}, []RestoreTest{
 		{SourceArchive: "local:backup/x.tar.zst", SourceTier: "", Pass: true,
 			TestedAt: time.Date(2026, 8, 1, 4, 41, 58, 0, time.UTC).Format(time.RFC3339)},
 	})
 	if got := c2.collectRestoreTests(context.Background()); len(got) != 0 {
 		t.Fatalf("a persisted record with no tier is not a usable proof and must be dropped; got %+v", got)
 	}
 }
 // ── SCENARIO C — a fresh in-memory result wins, and never duplicates ─────────────────────────
 //
 // COMPANION RED-PROOF (observed 2026-08-03): remove the de-duplication (append every persisted entry
 // unconditionally) →
 //
 //	--- FAIL: TestMerge_NewerWinsAndNeverDuplicatesATier
 //	    restoretest_merge_test.go: one entry per tier; got 2 for "pbs" — the hub would read two tests
 //
 // Restored.
 func TestMerge_NewerWinsAndNeverDuplicatesATier(t *testing.T) {
 	lastWeek := time.Date(2026, 7, 27, 19, 55, 41, 0, time.UTC) // jittered, from the real box
 	fiveMinAgo := time.Date(2026, 8, 3, 13, 25, 14, 0, time.UTC)
 	c := mergeCollector(
 		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/new", true, fiveMinAgo)},
 		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/old", true, lastWeek)},
 	)
 	got := c.collectRestoreTests(context.Background())
 	e, n := findTier(got, "pbs")
 	if n != 1 {
 		t.Fatalf("one entry per tier; got %d for \"pbs\" — the hub would read two tests: %+v", n, got)
 	}
 	if e.SourceArchive != "felhom-pbs:backup/ct/9201/new" {
 		t.Fatalf("the NEWER result must win; got %q tested %q", e.SourceArchive, e.TestedAt)
 	}
 	// ...and the older-in-memory / newer-persisted direction, which is the post-restart case.
 	c2 := mergeCollector(
 		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/old", true, lastWeek)},
 		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/new", true, fiveMinAgo)},
 	)
 	e2, n2 := findTier(c2.collectRestoreTests(context.Background()), "pbs")
 	if n2 != 1 || e2.SourceArchive != "felhom-pbs:backup/ct/9201/new" {
 		t.Fatalf("newest must win regardless of which source it came from; got %d entr(ies), archive %q", n2, e2.SourceArchive)
 	}
 }
 // ── SCENARIO D — a failure still reaches the hub ─────────────────────────────────────────────
 //
 // The merge must not mask a failure with an older stored success. A failing tier is retried at the
 // next evaluation and its record lives ONLY in memory, so losing it here would silence the loudest
 // DR signal this system produces.
 func TestMerge_AFailureIsStillReported(t *testing.T) {
 	provenLastWeek := time.Date(2026, 7, 27, 19, 55, 41, 0, time.UTC)
 	failedJustNow := time.Date(2026, 8, 3, 13, 41, 7, 0, time.UTC)
 	c := mergeCollector(
 		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/new", false, failedJustNow)},
 		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/old", true, provenLastWeek)},
 	)
 	e, n := findTier(c.collectRestoreTests(context.Background()), "pbs")
 	if n != 1 {
 		t.Fatalf("one entry per tier; got %d: %+v", n, c.collectRestoreTests(context.Background()))
 	}
 	if e.Pass {
 		t.Fatalf("a FAILURE newer than the stored proof must be what is reported — masking it would "+
 			"silence the loudest DR signal there is; got pass=%v archive=%q", e.Pass, e.SourceArchive)
 	}
 }
 // Two different tiers are both reported — the merge is per tier, not a single slot.
 func TestMerge_BothTiersSurvive(t *testing.T) {
 	c := mergeCollector(
 		[]RestoreTest{rt("local", "felhom-backup:backup/vzdump-lxc-9201-a.tar.zst", true,
 			time.Date(2026, 8, 3, 4, 44, 50, 0, time.UTC))},
 		[]RestoreTest{rt("pbs", "felhom-pbs:backup/ct/9201/x", true,
 			time.Date(2026, 8, 2, 5, 12, 33, 0, time.UTC))},
 	)
 	got := c.collectRestoreTests(context.Background())
 	if _, n := findTier(got, "local"); n != 1 {
 		t.Fatalf("the in-memory tier must survive the merge; got %+v", got)
 	}
 	if _, n := findTier(got, "pbs"); n != 1 {
 		t.Fatalf("the persisted tier must survive the merge; got %+v", got)
 	}
 }
 // A malformed timestamp must never displace a good entry — "unparseable" is not "newest".
 func TestMerge_MalformedTimestampNeverWins(t *testing.T) {
 	good := rt("pbs", "felhom-pbs:backup/ct/9201/good", true, time.Date(2026, 8, 3, 13, 25, 14, 0, time.UTC))
 	bad := RestoreTest{SourceArchive: "felhom-pbs:backup/ct/9201/bad", SourceTier: "pbs", Pass: true, TestedAt: "not-a-time"}
 	c := mergeCollector([]RestoreTest{good}, []RestoreTest{bad})
 	e, n := findTier(c.collectRestoreTests(context.Background()), "pbs")
 	if n != 1 || e.SourceArchive != "felhom-pbs:backup/ct/9201/good" {
 		t.Fatalf("an unparseable timestamp must not displace a good entry; got %d entr(ies), archive %q", n, e.SourceArchive)
 	}
 }
 // A nil proven-source leaves the pre-R-189 behaviour exactly as it was.
 func TestMerge_NilProvenSourceIsANoOp(t *testing.T) {
 	only := rt("local", "felhom-backup:backup/x.tar.zst", true, time.Date(2026, 8, 3, 4, 44, 50, 0, time.UTC))
 	c := mergeCollector([]RestoreTest{only}, nil)
 	got := c.collectRestoreTests(context.Background())
 	if len(got) != 1 || got[0].SourceArchive != only.SourceArchive {
 		t.Fatalf("a nil durable source must not change anything; got %+v", got)
 	}
 }
@@ -45,6 +45,35 @@ func (c *Client) Pool(ctx context.Context, name string) (PoolInfo, error) {
 	return p, c.get(ctx, "/pools/"+url.PathEscape(name), &p)
 }
 // Permissions returns the privileges this API TOKEN holds at an ACL path, as
 // GET /access/permissions?path=<path> answers it: privilege name → 1.
 //
 // R-185. It asks about the CALLER — the agent's own token — which is the only useful form of the
 // question. Asking as root answers a different question and always says yes.
 //
 // MEASURED SHAPE (demo-felhom, 2026-08-03), because the whole value of this call is reading the
 // answer correctly and the obvious reading is wrong:
 //
 //	/storage/felhom-pbs     → {"Datastore.Allocate":1,"Datastore.AllocateSpace":1}
 //	/storage/felhom-backup  → {"Sys.Audit":1,"SDN.Use":1,"Datastore.Audit":1}
 //
 // The ungranted path does NOT answer empty, and does NOT 403. It answers with the privileges
 // INHERITED from the box-wide `/` grant — so "is this path present in the response" reports OK for a
 // storage the agent demonstrably cannot list. The caller must test for the SPECIFIC privilege.
 //
 // The response is keyed by path; an absent path yields no privileges, which is the same answer as
 // "none" and is treated as such by the caller.
 func (c *Client) Permissions(ctx context.Context, aclPath string) (map[string]int, error) {
 	var raw map[string]map[string]int
 	if err := c.get(ctx, "/access/permissions?path="+url.QueryEscape(aclPath), &raw); err != nil {
 		return nil, err
 	}
 	if p, ok := raw[aclPath]; ok {
 		return p, nil
 	}
 	return map[string]int{}, nil
 }
 // GuestStatus returns GET /nodes/{node}/lxc/{vmid}/status/current. The API body
 // has no vmid field (it is in the path), so it is set from the argument.
 func (c *Client) GuestStatus(ctx context.Context, vmid int) (Guest, error) {
@@ -14,10 +14,32 @@ Nothing in the build, deploy or session-end path checked that either existed, so
           documented-path reinstall would have silently DOWNGRADED both boxes to the pre-merge
           agent — and would have *succeeded* while doing it.
-THE INVARIANT, AND WHY IT IS THIS ONE.
+THE INVARIANTS — there are TWO now, and the second is R-188's price.
-  For every `v<semver>` git tag in this repo: the matching generic package must be DOWNLOADABLE,
+  (1) For every `v<semver>` git tag in this repo: the matching generic package must be DOWNLOADABLE,
-  and the tag must serve the agent's configs.
+      and the tag must serve the agent's configs.
  (2) No PUBLISHED version may be missing its tag.
 Invariant (2) is new (R-188, 2026-08-03) and it exists because `release-agent.sh` now pushes the tag
 AFTER publishing. The old order pushed the tag first, and the old comment said why: a tag with no
 package is caught here, a package with no tag is invisible, because the Gitea package LISTING api
 needs a token this gate does not have. That reasoning was sound and the ordering was still wrong —
 the tag push is what wakes CI, so every correct release had a ~50% chance of running this gate in the
 seconds before its own package existed and mailing the operator a failure for a release that worked
 (measured across two releases: runs 12/13 and 17/18, same shas, opposite results).
 Moving the push does not get to trade invariant (2) away, so it is asserted here instead — WITHOUT a
 token, and therefore as a BOUNDED PROBE rather than an enumeration:
  * the FRONTIER — the versions immediately above the highest tag. This is the realistic failure the
    new ordering makes possible: publish succeeds, tag push fails, so the orphan is exactly one
    version beyond the newest tag.
  * the GAPS — patch versions that fall between two existing tags and have no tag of their own.
 Re-measured 2026-08-03, not assumed: `GET /api/v1/packages/admin?type=generic` answers **401** with no
 token, so absence still cannot be proven. The probe set is PRINTED on every run, because a check whose
 coverage is invisible reads as a guarantee it is not making.
 The task's §8.4 asked for a different one — *"the version the hub tells machines to install must be
 downloadable"* — and that is the better invariant in principle. **It is not implementable from CI,
@@ -36,6 +58,8 @@ v0.120.0, which is published.
 **What it does NOT catch, stated plainly:** the hub vouching a version that was never released at
 all (no tag, no package). Nothing here can see that; it belongs at vouch time, in the hub. → R-184.
 Nor does the converse probe prove that NO untagged package exists — only that none exists at the
 probed versions, which are printed. Closing that properly needs a read token in CI (→ R-184).
 FAIL-CLOSED. A network error, an unparseable response or an unreachable Gitea is exit **2
 INCONCLUSIVE**, naming every URL tried — never a pass. "Cannot determine" is not "fine": that is the
@@ -48,7 +72,7 @@ carrying python3 and git and nothing else, and an earlier workflow step died on
    python3 scripts/check-published-versions.py
-Exit: 0 every tag installable · 1 at least one is not · 2 could not be determined.
+Exit: 0 both invariants hold · 1 either is violated · 2 could not be determined.
 Env: GITEA_BASE overrides the Gitea root (CI sets the in-cluster service URL).
 """
 import json
@@ -94,6 +118,53 @@ def inconclusive(msg):
    sys.exit(2)
 def _pkg_exists(version):
    """True iff the generic package for `version` is downloadable anonymously."""
    url = "%s/api/packages/%s/generic/%s/%s/%s" % (GITEA_BASE, OWNER, PKG, version, PKG)
    status, _ = _get(url)
    return status == 200, url
 def untagged_probe_set(versions):
    """The versions to probe for invariant (2), as (version, why) pairs.
    Bounded on purpose and printed by the caller: the package listing api needs a token (401,
    re-measured 2026-08-03), so absence cannot be enumerated. What CAN be done is to probe the
    places an orphan would actually land.
      FRONTIER — a publish that succeeded followed by a tag push that failed leaves the orphan
                 exactly one version past the newest tag. This is the failure mode the R-188
                 reordering makes possible, so it is the one that must not be guesswork.
      GAPS     — a patch number skipped between two consecutive tags. Bounded per gap so a typo'd
                 tag (v0.130.0 after v0.121.1) cannot turn this into a thousand requests.
    """
    parsed = sorted(tuple(int(p) for p in v.split(".")) for v in versions)
    have = set(parsed)
    out = []
    if not parsed:
        return out
    hi = parsed[-1]
    for cand, why in (
        ((hi[0], hi[1], hi[2] + 1), "next patch after the newest tag"),
        ((hi[0], hi[1], hi[2] + 2), "second patch after the newest tag"),
        ((hi[0], hi[1] + 1, 0), "next minor after the newest tag"),
        ((hi[0] + 1, 0, 0), "next major after the newest tag"),
    ):
        if cand not in have:
            out.append(("%d.%d.%d" % cand, why))
    MAX_GAP_PROBES = 12
    for a, b in zip(parsed, parsed[1:]):
        if a[0] != b[0] or a[1] != b[1]:
            continue  # a minor/major step is not a patch gap
        for patch in range(a[2] + 1, min(b[2], a[2] + 1 + MAX_GAP_PROBES)):
            cand = (a[0], a[1], patch)
            if cand not in have:
                out.append(("%d.%d.%d" % cand, "patch gap between v%d.%d.%d and v%d.%d.%d" % (a + b)))
    return out
 def main():
    print("check-published-versions — every released agent version must be INSTALLABLE")
    print("  gitea:", GITEA_BASE)
@@ -144,14 +215,42 @@ def main():
        else:
            print("  ok   v%s: binary downloadable + tag serves its configs" % v)
    # ── invariant (2): no PUBLISHED version may be missing its tag (R-188) ──────────────────────
    probes = untagged_probe_set(versions)
    orphans = []
    print()
    print("  converse probe — a published version with no tag (bounded; the package listing api")
    print("  needs a token, so this cannot enumerate). Probing %d version(s):" % len(probes))
    for v, why in probes:
        try:
            exists, url = _pkg_exists(v)
        except Exception as e:
            inconclusive("network failure while probing v%s: %s" % (v, e))
        mark = "PUBLISHED — NO TAG" if exists else "absent (ok)"
        print("    %-10s %-42s %s" % (v, why, mark))
        if exists:
            orphans.append((v, url))
    print()
    if bad or orphans:
        if orphans:
            print("check-published-versions: %d PUBLISHED VERSION(S) WITH NO TAG" % len(orphans))
            for v, url in orphans:
                print("  v%s is downloadable at %s but has no git tag." % (v, url))
            print("  A release publishes and then pushes its tag; a package with no tag means the")
            print("  push failed or was skipped. The local tag is probably still in the release")
            print("  clone — finish it with:")
            for v, _ in orphans:
                print("      git push origin v%s" % v)
            print("  (and if the tag is gone, re-create it on the released commit before pushing.)")
    if bad:
        print("check-published-versions: %d RELEASED VERSION(S) NOT INSTALLABLE" % len(bad))
        print("  A tagged version with no package is a release that was BUILT and never PUBLISHED —")
        print("  the R-115 defect, three times in five days. Publish it with:")
        print("      scripts/release-agent.sh <version>")
    if bad or orphans:
        return 1
-    print("check-published-versions: ALL RELEASED VERSIONS INSTALLABLE")
+    print("check-published-versions: ALL RELEASED VERSIONS INSTALLABLE, AND NONE UNTAGGED")
    return 0
@@ -51,7 +51,11 @@ if [[ -z "$BIN" ]]; then
    BIN="$(mktemp -t felhom-agent.XXXXXX)"
    CLEANUP_BIN="$BIN"
    log "building felhom-agent $VERSION from $REPO_ROOT …"
-    ( cd "$REPO_ROOT" && CGO_ENABLED=0 go build -ldflags "-X main.version=${VERSION}" -o "$BIN" ./cmd/felhom-agent )
+    # These flags MUST match release-agent.sh's build exactly — see the long comment there (R-186).
    # They used to differ: this line forced CGO_ENABLED=0 and produced a binary 74 KB smaller than
    # the one the release path built for the same version. One version name must mean one binary
    # whichever entry point produced it.
    ( cd "$REPO_ROOT" && go build -trimpath -buildvcs=false -ldflags "-X main.version=${VERSION}" -o "$BIN" ./cmd/felhom-agent )
 fi
 [[ -f "$BIN" ]] || die "binary not found: $BIN"
 trap '[[ -n "$CLEANUP_BIN" ]] && rm -f "$CLEANUP_BIN"' EXIT
@@ -74,7 +74,25 @@ existing="$(curl -fsS -o /dev/null -w '%{http_code}' \
 BIN="$(mktemp -t felhom-agent-XXXXXX)"
 trap 'rm -f "$BIN"' EXIT
 log "building $VERSION …"
-go build -ldflags "-X main.version=$VERSION" -o "$BIN" ./cmd/felhom-agent \
+# REPRODUCIBLE BY CONSTRUCTION (R-186). The sha printed below is the one the operator vouches, and
 # until now nobody could rebuild it to check: `go build` stamps a module version derived from VCS
 # state, so a build made BEFORE the tag exists and a rebuild made after it are different binaries.
 # Measured 2026-08-03 at this commit — same source, same toolchain, same ldflags:
 #
 #   default flags,  no tag yet .. 18f4a495…  14 085 464 B   (mod v0.121.2-0.2026…-3d0a1d61)
 #   default flags,  tagged ...... 4a38f394…  14 085 440 B   (mod v0.121.99)
 #   -trimpath -buildvcs=false ... 7ffcdf1d…  14 064 574 B   IDENTICAL both ways
 #
 # `-buildvcs=false` removes the stamp — nothing in this repo reads it (no `ReadBuildInfo` caller,
 # verified) and the version comes from the explicit ldflag below, which is where it belongs.
 # `-trimpath` removes absolute build paths, so a rebuild from a different checkout directory also
 # matches. Neither is a sequencing trick: the property no longer depends on WHEN the build happens.
 #
 # CGO is deliberately left at its default. publish-agent.sh's fallback build used to force
 # CGO_ENABLED=0 and therefore produced a DIFFERENT binary (13 990 236 B, 74 KB smaller) for the same
 # version — one version name, two binaries, by whichever entry point was used. Both now build the
 # same way; if that ever has to change, change it in BOTH or the guarantee is gone.
 go build -trimpath -buildvcs=false -ldflags "-X main.version=$VERSION" -o "$BIN" ./cmd/felhom-agent \
    || die "go build failed"
 built_ver="$("$BIN" --version 2>/dev/null | awk '{print $2}')"
 [[ "$built_ver" == "$VERSION" ]] \
@@ -82,10 +100,27 @@ built_ver="$("$BIN" --version 2>/dev/null | awk '{print $2}')"
 BUILT_SHA="$(sha256sum "$BIN" | awk '{print $1}')"
 log "built ok: sha256 $BUILT_SHA"
-# ── 4. Tag (before publishing, so a published version always has a tag) ─────────────────────────
+# ── 4. Tag LOCALLY (the push comes after the publish — see step 6) ──────────────────────────────
-# Order matters in this direction only: a tag with no package is caught by
+#
-# scripts/check-published-versions.py on the next CI run; a package with no tag is invisible to it,
+# THE ORDER CHANGED, AND ONLY THE PUSH MOVED (R-188, 2026-08-03).
 #
 # It used to be tag → push tag → publish, and the reason written here was sound: a tag with no
 # package is caught by scripts/check-published-versions.py, a package with no tag is invisible to it,
 # because the Gitea package LISTING api needs a token the gate does not have.
 #
 # What that reasoning missed is that the tag PUSH is what wakes CI (`on: [push]`), so the gate ran in
 # the seconds between the tag becoming visible and the package existing — and correctly failed. Every
 # correct release had roughly a coin-flip chance of emailing the operator a failure for a release
 # that worked. Measured across two releases in one session: runs 12/13 (v0.121.0) and 17/18
 # (v0.121.1), same sha each time, opposite results. R-168 made that mail the thing that cannot be
 # missed; a mail that is wrong half the time is one you stop reading, and then the real one goes too.
 #
 # So the tag is still created HERE, before anything is published — the build and the tag still
 # describe the same commit, and a failed publish leaves a purely local tag that never misled anyone.
 # It simply becomes VISIBLE (to CI, and to any installer fetching raw/tag/…) only once the package
 # is downloadable. The invariant the old order protected is not traded away: it is asserted directly
 # by the gate's new converse probe (a published version with no tag FAILS), so both directions are
 # now checked rather than one being arranged for.
 log "tagging $TAG at $(git rev-parse --short HEAD) …"
 git tag -a "$TAG" -m "agent $TAG
@@ -94,7 +129,6 @@ sha256 of the published binary: $BUILT_SHA
 felhom-host-install.sh fetches this version's config files from raw/tag/$TAG/configs/,
 so this tag is part of the released artifact, not a bookmark (R-183)."
 git push origin "$TAG" || die "tag push failed — refusing to publish an untagged version"
 # ── 5. Publish (the existing script; deliberately not reimplemented) ────────────────────────────
 log "publishing …"
@@ -104,9 +138,48 @@ log "publishing …"
 # R-115 exists to make unforgettable was, on its first use, unrunnable. The mode bit is restored in
 # the same commit; this line makes the release independent of it, because a file mode is exactly the
 # kind of thing that is lost again by a checkout, an archive, or a copy.
-bash "$REPO_ROOT/scripts/publish-agent.sh" "$VERSION" "$BIN" || die "publish failed"
+if ! bash "$REPO_ROOT/scripts/publish-agent.sh" "$VERSION" "$BIN"; then
    # The tag is LOCAL-ONLY at this point, so a failed publish must not leave one behind: the next
    # attempt would die at step 2's "tag $TAG already exists" and read as "this version is already
    # released", which would be exactly backwards. Only remove it if nothing was in fact published —
    # if a package DOES exist, the tag is wanted and must be pushed, not deleted.
    now_published="$(curl -fsS -o /dev/null -w '%{http_code}' \
        "$GITEA_BASE/api/packages/$GITEA_OWNER/generic/felhom-agent/$VERSION/felhom-agent" 2>/dev/null || true)"
    if [[ "$now_published" == "200" ]]; then
        log "publish reported failure but the package IS downloadable — keeping the local tag; push it with: git push origin $TAG"
    else
        git tag -d "$TAG" >/dev/null 2>&1 && log "removed the local-only tag $TAG so the release can be retried"
    fi
    die "publish failed"
 fi
-# ── 6. Verify by an INDEPENDENT download ────────────────────────────────────────────────────────
+# ── 6. Push the tag, now that the package exists ────────────────────────────────────────────────
 # This is the step that makes the release VISIBLE — to CI, and to every `raw/tag/v<version>/` fetch
 # the installer makes. It runs last of the two so CI can never see a tag whose package is not there.
 #
 # If it fails, the release is HALF DONE and must be said so loudly: the package is published and the
 # tag exists only in this clone, which is precisely the orphan the gate's converse probe now catches.
 # The recovery is one line and it is printed rather than described.
 log "pushing $TAG …"
 if ! git push origin "$TAG"; then
    cat >&2 <<EOF
  RELEASE HALF DONE — the package is PUBLISHED and its tag is NOT pushed.
    version : $VERSION
    sha256  : $BUILT_SHA
  The tag exists in this clone only. Nothing installs from an untagged version (the installer
  fetches this version's configs from raw/tag/$TAG/), and scripts/check-published-versions.py will
  FAIL on it as a published version with no tag. Finish the release with:
      git push origin $TAG
 EOF
    die "tag push failed after a successful publish — see above"
 fi
 # ── 7. Verify by an INDEPENDENT download ────────────────────────────────────────────────────────
 # The publish step's own success is not proof: it reports on its own write. What matters is that a
 # box can now GET the bytes and that they are the bytes that were built. This is the same
 # presence-is-not-success rule the project earned twice — a step that says "done" and a fetch that
Author	SHA1	Message	Date
admin	856a127cd6	v0.124.1: the repair record must survive the probe that did NOT feed the hub (R-190) gates / gates (push) Successful in 6s Details v0.124.0's transition record never reached the hub, and only the live run showed it. The capability reported degraded for "one cycle" — the probe call that did the repair. But probeAll is invoked independently by the self-check log and by the collector building a host report. On the demo box the repairing call was the log's (09:39:34, journal shows the self-repair and degraded=1) and the report three seconds later found the grant present and sent ok. The agent's journal had the record; the hub had nothing. That is the silence R-190 is about, re-created inside its own mitigation, with every unit test green. Fixed with a latch on TIME, not call count: a confirmed repair reports for 20 minutes, which exceeds the 900s report interval, so at least one report must carry it. It clears on its own and is per tier. Two hollow tests caught and fixed on the way — one asserting a value it built itself, one asserting the latch helper rather than the path consuming it (its red-proof duly passed). The decisions now live in storeGrantHealthyVerdict and storeGrantRepairedVerdict and the tests call those.	2026-08-04 09:44:56 +02:00
admin	257c4d85c0	v0.124.0: a lost storage grant repairs itself, and says that it was lost (R-190) gates / gates (push) Successful in 7s Details R-190 is a grant that worked at 04:44 on 2026-08-03 and was gone by 09:24, with a reinstall, logged pveum activity and cluster-log entries all ruled out. The cause is open; the resilience need not wait for it. Everything needed already existed and had only ever been called once: the root wrapper's `grant` verb, its sudoers vector (`grant *`, any storage id — confirmed, not assumed), and the exact command. The verb had only ever run at storage creation — the "built but never wired" shape in a verb rather than a seam. The probe now runs that wrapper on a missing grant and re-reads ONCE to confirm, the pbsdr R-22 shape including its restraint. The record is the half that matters. A repair leaving only "ok" behind destroys the only evidence a permission vanished, so a recurring loss becomes undetectable — worse than the fault. A confirmed repair therefore reports DEGRADED for exactly one cycle with the explanation in Feature, because that is the field the hub puts in the operator's email (Reason does not travel). Nothing new was built: the hub's existing ok->degraded->ok edge is the channel, so one loss produces one alert pair. No wire change, no hub change, no new event type. Bounded at one attempt per tier per hour: a storage can be unreadable for reasons an ACL cannot fix, and re-granting every cycle is a repair loop wearing a fix's clothes. A failed repair never masks the fault.	2026-08-04 09:38:27 +02:00
admin	72161f6cf0	REPORT: correct the manifest commit hash (311dc06) gates / gates (push) Successful in 7s Details	2026-08-03 19:04:44 +02:00
admin	03b58cec0a	REPORT + CONTEXT + REUSE: R-185 closed, with the corrected root cause gates / gates (push) Successful in 7s Details The installer defect was NOT PVE_STORAGES as the row and the task assumed: the create arm of configure_backup_target grants, the Scenario-F reuse arm did not. Also records the measured trap (an ungranted path answers with INHERITED privileges, not empty and not 403), the deviation from the spec's suggested Prober generalisation in favour of the existing poolReadStatus precedent, the hollow test caught before it shipped, and that demo-hp carried the same drift and was fixed.	2026-08-03 19:04:15 +02:00
admin	fe14bc62c0	v0.123.0: a tier the box cannot READ now says so (R-185) gates / gates (push) Successful in 7s Details The missing grant is one command; the silence was the defect. On demo-felhom the agent's token has FelhomAgentStore on local, local-lvm and felhom-pbs — and not on felhom-backup, the storage the same installer configured as local_backup_target. That storage answers {"data":[]} through the token while root sees three archives. An empty listing is what a FORBIDDEN tier and a NEWBORN tier both return, so pickForThisRun skipped it as "no settled archive yet" and the tier was never restore-testable on that box. The permission question, unlike the listing, has a definite answer, so it is asked directly: Client.Permissions reads /access/permissions as the agent's OWN token, and one capability.Status per configured tier reports it — composed around the sudo prober, the way the pool-read check already is. Measured first, because the obvious reading is wrong: an ungranted path answers neither empty nor 403, but with the privileges inherited from the box-wide grant (Sys.Audit, SDN.Use, Datastore.Audit). Checking for Datastore.Audit would report a blinded storage healthy — red-proved. The probe tests for Datastore.AllocateSpace. The probed set comes from the box's own config, never a fixed list: a hardcoded probe list is the defect reproduced inside the fix. Critical, because the hub alerts only on critical — except the "local" fallback target, which is reported but does not page. It never looks at content, so it cannot alarm on a newborn tier; it never reports ok when it could not ask. Status wire shape unchanged, so no hub change.	2026-08-03 18:53:47 +02:00
admin	0b28eae7bb	REPORT: R-189/R-188/R-186 — live evidence, the three sha values, and the observations gates / gates (push) Successful in 6s Details Scenario A proven on demo-felhom against the exact observation that filed R-189: a 675 s offsite restore-test passed, the agent was restarted 11 seconds later (inside the reporting window), and the hub's very next report carried '1 restore-tests' where the same sequence produced 0 this morning. The hub's own database holds the archive, the tier, the pass and the ORIGINAL test time, with the run mechanics deliberately zero. Also records the property the validation surfaced: the state holds one proof per tier, so proving an older archive re-arms a newer one — confirmed live after the defaults were restored.	2026-08-03 16:58:35 +02:00
admin	7581f8140a	v0.122.0: three ways the signals lied about themselves (R-189, R-188, R-186) gates / gates (push) Successful in 7s Details All three are the reporting and release path misreporting its own work. No customer machine, no backup, no restore, no data. The restore-test itself and when it runs are unchanged. R-189 — a passing restore-test no longer vanishes on a restart. restore_tests[] came only from the in-memory store, whose comment ("lost on restart; the cadence re-populates") was true under a timer and stopped being true when R-86 made the agent refuse to re-test a proven archive: the proof is then not repeated for a whole archive generation. Observed live — a 14.5 GB offsite PASS reached no host-report because the agent was restarted 2m43s later. RestoreTestState now carries tier + verified beside the archive and renders reportable entries; the collector merges them, one per tier, newest by TestedAt. It refuses to lie: a record missing archive-or-tier produces no entry, and run mechanics are not re-invented. Only successes are persisted, and the asymmetry is now written where it will be read. R-188 — a correct release stops emailing a failure. Only the tag PUSH moved (build -> tag locally -> publish -> push tag): the push wakes CI, and a tag visible before its package made the gate correctly fail a correct release about half the time. The old order's invariant is asserted directly instead — the gate now refuses a published version with no tag, as a bounded probe that prints its own coverage, because the package listing api is still 401 without a token. R-186 — a released binary can be verified by rebuilding it. -trimpath -buildvcs=false: same source, same bytes, tag or no tag. Measured. publish-agent's fallback also forced CGO_ENABLED=0 and produced a 74 KB different binary for the same version; both paths now build identically. CLAUDE.md records the command.	2026-08-03 16:40:18 +02:00
admin	3d0a1d615d	REPORT: restart proof, R-188/R-189, and the restored defaults gates / gates (push) Successful in 7s Details	2026-08-03 15:36:46 +02:00
admin	77e2cc4583	CONTEXT: v0.121.1 (a quiet evaluation is audible) + the live proof gates / gates (push) Successful in 6s Details	2026-08-03 15:32:23 +02:00