installer v1.28.0: the removal genuinely reverses the installation (R-316)

v1.27.0's fix worked exactly once per machine. Measured on drill-r50 from virgin,
on the PUBLISHED v1.27.0, before anything was changed: cycle 1 recorded 'no' and
freed :53; cycle 2 recorded 'yes' and left dnsmasq running on 0.0.0.0:53; cycle 3
refused, exit 1. Every box already in the field is at cycle 2, and a reinstall
onto a machine that has had Felhom is cycle 2 by definition.

Why cycle 2 says yes: the preflight's ownership question is dpkg-query package
presence and nothing else - not the absence of a record. Stopping the unit and
leaving the package made our own package read as the household's one cycle later.

Now the uninstall removes the package when the record says we installed it. Order
unchanged and load-bearing: read the record, act, then delete the state file that
holds it. TWO packages are recorded, because dnsmasq ships the unit and
dnsmasq-base ships /usr/sbin/dnsmasq, and each is taken back only if we added it.

The dependency check is a SIMULATION, not a guess: apt-get -s purge is asked what
it would remove and the purge proceeds only if that set is a subset of ours;
otherwise stop+disable, naming the package that blocked it. Never interactive,
never fatal, and the success is re-queried rather than read off an exit code.

Watched: three fixed cycles -> install 3 PASSES; a household resolver untouched; a
dependent package not purged and named; no record -> untouched with the command
named. Red-proofs with the mutation asserted applied: remove the purge -> cycle 3
refuses in those exact words; remove the ownership check -> a household resolver is
purged; infer ownership -> the guess is taken.

Also: R-317 (the agent stats a path dnsmasq-base owns to decide whether to install
dnsmasq - pre-existing, now reachable), R-318 (no honest ownership marker exists
for existing boxes; the preflight message is the mechanism), and the status page's
decisions section rewritten to say what each decision costs and what doing nothing
selects.

CONTEXT.md

@@ -15,6 +15,49 @@
 > would make one of the two audiences stop reading. `STATUS.md` is also a **view of `OPEN-ITEMS.md`**
 > and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
 
+## Material retained is not history recoverable — ask the three questions separately (2026-08-12)
+
+**"We keep the old key" and "the customer can get their old backups back" are three questions, and
+collapsing them is how a false belief survived a week.** Ask them one at a time, and answer each with
+its own evidence:
+
+**(a) Is the material retained?** **(b) Does it, once unsealed, still open the old store?**
+**(c) Does the customer have a ROUTE to (b) through the product?**
+
+Measured 2026-08-12, and the answers were **yes, yes, no** — a shape nobody had predicted. Our records
+predicted (a) yes and (b) no ("a box we fill and cannot open"); the drill found (b) works and **(c) was
+the missing one.** `ListSupersededEscrow` had held the right bytes since v0.93.0 with **zero production
+callers**, so the capability existed and was unreachable. The same afternoon's follow-up shipped a
+partial (c) — the product now RECOGNISES a correct code for an earlier package — and established that
+the rest of (c) is new surface, not wiring, because every restore entry point resolves its repository
+from settings and its password from one file (R-312).
+
+**The generalisation, which is the part worth keeping:** a stored artifact proves storage, never
+usability. Between "the bytes are kept" and "the customer is helped" sit an unseal, a consumer and a
+route, and each has been separately absent in this project. **Before claiming a capability, name which
+of the three you have evidence for** — and if a comment or a card asserts the third, find the code path
+that performs it or move the claim.
+
+## A removal that leaves the package leaves the next install a second-cycle machine (2026-08-13)
+
+**Stopping is not removing, and the difference does not show up until the cycle after next.** R-300
+stopped and disabled the resolver we install and left the package in place — deliberately, as the
+conservative act. Cycle 1 looked fixed and *was* fixed. But the next install asks dpkg "is this
+installed?", gets yes, records the package as the household's, and the uninstall after that leaves it
+running: cycle 3 refuses with the message that blames the customer's network. **Measured, three cycles,
+before anything was changed.**
+
+**The general shape:** a teardown that leaves an artifact behind does not merely fail to clean up — it
+*changes what the next run concludes about who owns that artifact*. Ask of any "we left it in place,
+to be safe": **what will the next install think this is?** If the answer is "ours, misread as theirs",
+the conservative act has quietly become the destructive one, one cycle downstream.
+
+**And the corollary that nearly shipped a second defect:** the thing you remove and the thing something
+else probes for may be in **different packages**. `dnsmasq` ships the unit; `dnsmasq-base` ships the
+binary the agent stats to decide whether to install. Removing one and not the other satisfies the
+uninstall and breaks the next install silently (R-317). **Before removing a package, ask what else
+tests for the files it shares with its siblings.**
+
 ## Two rules earned twice each (2026-08-12)
 
 **A fact must be recorded at the moment it is a fact, not inferred later from something adjacent.**

REPORT.md

@@ -1,87 +1,169 @@
-# REPORT — DRILL: the retained key, and the two fixes nobody had watched work (2026-08-12)
+# REPORT — The door, part one: a correct code stops being called wrong (2026-08-12 night)
 
-**Class:** drill (unattended, destructive on Tier 0) + spike for Phase C's first step
+**Three repos.** hub **v0.103.0** · agent **v0.129.0** · controller **v0.214.0** · golden **0.214.0**.
-**Venues:** `drill-r50` (nested PVE on DooPlex), `demo-felhom` (guest 9201) — both Tier 0.
+Register: **R-311 CLOSED**, R-307 CLOSED, **R-312 / R-313 / R-314 / R-315 opened**. Ceiling
-**`demo-hp` was never touched. `peti-felhom` was never contacted. No abandon countdown was started,
+R-310 → **R-315**.
-shortened or triggered.**
-**Full record:** `documentation/audits/DRILL-retained-key-2026-08-12.md`
 
 ---
 
-## The answer to the question this drill existed to answer
+## 1. The spike's answer, first and in plain language
 
-**(a) Is the old key kept? YES** — proven for the first time in the fleet's history.
+**Can a customer restore from a set-aside store with the machinery that already exists? NO — and
-**(b) Does the kept key open the old backups? YES** — three planted files, including a Hungarian
+building it is new surface, not wiring.** Established read-only, at `file:line`, before a line was
-accented filename verified as raw bytes, restored **byte-identical** from a store the machine itself
+written.
-could no longer open.
-**(c) Can the customer get there through the product? NO — and they are told their correct code is
-wrong.**
 
-The brief said to be ready for the answer to be no, and our own records predicted retention would be
+Every restore entry point resolves the repository from `settings.GetOffboxTarget()` and the password
-*"a box we fill and cannot open"*. **That was half right, and the wrong half was the one nobody had
+from the single `offboxPwPath()` file: `offboxLatestSnapshot` (`offbox_restore.go:85-86`),
-checked.** The box opens. What does not exist is the door: `ListSupersededEscrow`
+`offboxSnapshotSize` (`:139-140`), `RestoreOffboxScratch` (`:206+`). **A grep for a repo-path
-(`hub/internal/store/store.go:2841`) is the only reader of a retained key and has **zero production
+parameter anywhere in the restore chain returns nothing.** The only seam that installs a recovered
-callers**; the recovery path selects `FROM host_escrow` — the current row only. Asked with the very
+password, `InjectOffboxPassword` (`offbox.go:667`), writes that same one file — i.e. **adoption**.
-code that had just opened the retained row by hand, the product answered *"the recovery code did not
-open the sealed bundle — nothing was written"*. → **R-304, rank 1**
 
-Consequences: the census answer **stands**; the countdown banner's promise is **true in substance,
+What the drill did to read the set-aside store was `restic` **by hand**, with `-r <alt repo>` and an
-false in practice**; the capability map's recovery claim **has been moved** with today's evidence.
+overridden `RESTIC_PASSWORD_FILE`. **That distance is exactly what (b)-to-(c) costs.**
 
-## What shipped
+**So the session halted at Part 3 by its own rule, shipped Part 2, and hands back options → R-312.**
+Cost to find out: ~35 minutes, read-only.
 
-**`installer-v1.27.0` published** — tag cut and **both** `--ref`s in `manifests/webpage.yaml` bumped
+## 2. Part 0 — the countdown, cancelled on your ruling
-(sidecar line 327, init container line 372). Publication was earned: both faults were watched
-happening first, from a machine reset to factory state.
 
-- **R-300 CLOSED** — pre-fix uninstall left dnsmasq `enabled`/`active` on `0.0.0.0:53`; the next byo
+Through the product's own operator path (`--abandon-stop`, which refuses rather than silently
-  install refused, exit 1. Fixed path: recorded `not present before Felhom` → `stopping + disabling
+no-opping), with the container **stopped first** so the running controller could not overwrite
-  it` → `:53 FREE` → preflight PASS. The owner's side proven too (record `yes` → left running).
+`settings.json` from memory. **Proved, not trusted to the exit code:**
-- **R-297 CLOSED** — a stale `golden-0.98.3.tar.zst` planted as newest-by-filename; v1.25.0 took it
-  with no comparison and **the box came up on controller 0.98.3** against a vouched 0.213.0 — below
-  the floor and below v0.206.0 where the off-site recovery screen exists. Fixed path re-fetched and
-  sha-verified the vouched golden (landed 0.213.0); an operator-named stale archive was **refused**.
 
-## Findings opened — ceiling R-303 → R-310
+- `abandon_started_at` and `abandon_at` — **gone**. `AbandonStatus` returns `Active=false` when
+  `AbandonAt` is empty (`offbox_abandon.go:111-113`), so no countdown renders.
+- `abandon_repo_path` — **deliberately kept**, as the pointer to the preserved store.
+- The set-aside store — **still there**: 36 snapshot objects, full `config/data/index/keys/locks/
+  snapshots` structure. Both repositories still on the endpoint. **Nothing deleted anywhere.**
 
-| # | Rank | What |
+**And the thing you should know about what was preserved (R-313):** it holds **36 snapshots and one
+key slot**, and it does **not** open with the box's current password (`Fatal: wrong password or no key
+found`, exit 1 — measured). Its key is the one hashed `48741892f0ef…` — retained row id 4,
+`identity_blob` **NULL**, a pre-v0.93.0 row. **The material was dropped by the R-198 defect during its
+two-month window, so no recovery code in existence opens that store.** Keeping it is still the right
+call — deleting is irreversible and a decision, not an accumulation — but it is 36 unreadable
+snapshots, and that is the concrete, still-present cost of R-198 sitting on the endpoint.
+
+## 3. Part 2 — what shipped, and a correction to the premise
+
+**The premise needed correcting first.** The task described the customer being told *"the recovery code
+did not open the sealed bundle"*. That is the **agent's local-API** reply. The **customer-facing
+screen already hedged** (R-222/R-226) — it named both causes, named the kept package and its date, and
+said it could not tell them apart. That was **honest**; it could not tell them apart **because nothing
+ever looked**. So what shipped is smaller and more precise than "stop the lie": **the hedge becomes an
+answer.**
+
+- **hub v0.103.0** — `GET /hosts/<id>/escrow/retained`, the **first production caller
+  `ListSupersededEscrow` has ever had**. Self-scoped identically, same recovery-mode gate, same audit
+  event written before the bytes leave, capped at 16. Rows with a NULL `identity_blob` are **withheld
+  and counted** (`unopenable_count`): they can never open what the caller is asking about, and serving
+  them would let the screen promise recovery on exactly the boxes the original defect hurt.
+- **agent v0.129.0** — retained packages tried **only after** the current one refuses; `422` with
+  `superseded_at`; bounded at 6 attempts (~1 s of scrypt each); fail-safe in every direction.
+- **controller v0.214.0** — class `RecoveryCodeOpensRetained`, gated on MinAgent **0.129.0** via a
+  **second, separate** trust flag (a box can sit between 0.126.0 and 0.129.0). The message says the
+  code is correct, names the date, says the package is kept, says the **current** backups are
+  unaffected, and **promises no restore** — it routes to support, which can do it.
+
+**The trade you should see stated:** the hub still cannot read any of it — sealed bytes in, sealed
+bytes out, no decrypt path, no recovery code ever held. What widens is **volume**: a host key that
+could fetch one opaque package can now fetch N, bounded by self-scope, the recovery-mode gate and the
+cap.
+
+## 4. Red-proofs — and where the lie actually lives
+
+Every mutation asserted to have applied before its run.
+
+| Repo | Mutation | Outcome |
 |---|---|---|
-| **R-304** | **1** | Retained key has no product route; the correct old code is reported as wrong |
+| hub | serve the CURRENT row instead of retained | FAILS (count 2→1) |
-| **R-305** | 2 | The R-300 cleanup fires **once per machine** — the leftover returns on the second reinstall (proven, cycles 2/3) |
+| hub | drop the unopenable guard | FAILS (count 1→2, unopenable 1→0) |
-| **R-308** | 2 | Stored controller `PASSWORD` no longer opens demo-felhom (`Hibás jelszó`) — not the quoting trap |
+| hub | drop self-scope | FAILS (403→200) |
-| **R-306** | 3 | `--preflight-only` says *"no state written"* and writes `state.json` — with an ownership answer that can be wrong |
+| hub | collapse the route suffix | FAILS (count 1→0) |
-| **R-309** | 3 | The day-0 runbook says pushing publishes the installer; false since R-110 (measured: public URL served 1.25.0 while `main` had 1.27.0) |
+| **agent** | **remove the retained lookup** | **FAILS — the fail-closed wrong-code error returns. THE LIE COMES BACK.** |
-| **R-310** | 4 | Duplicated sentence in the golden refusal; `--uninstall` needs a pty and `--force` does not bypass it |
+| agent | + 6 more (nil fetcher, wrong code, fetch failure, bounded attempts, predates-field, success path) | all pinned |
-| **R-307** | — | **Operator decision, deadline 2026-08-24** — see below |
+| controller | delete the new case | FAILS — but the customer gets the **neutral** message, because R-224's safe default catches it |
+| controller | make 422 unconditional | FAILS — an agent that never looked is read as having looked |
+| controller | route 400 to the new class | FAILS — a mistype is congratulated |
 
-## What needs you
+**Answering the question directly:** the lie returns when the **agent's** retained lookup is removed,
+not when the controller's case is. R-224's safe default is doing its job one layer up.
 
-**`demo-felhom` carries a live abandon countdown** — started 2026-08-10, **firing 2026-08-24**, for
+## 5. The claim guard, and a gate whose positive control failed
-`/home/felhom-repo.orphaned-20260810`. This drill did **not** start it and deliberately did **not**
-cancel it. The brief's end state asked for no countdown anywhere; satisfying that means choosing:
-**cancel it** (copy kept indefinitely, storage cost, no data risk) or **let it run** (copy deleted,
-irreversibly). **Doing nothing selects deletion.** → **R-307**
 
-## End state
+**The claim guard had a blind spot the size of the recovery screen** — it scanned templates only,
+while every recovery message is a Go string in a handler. It now scans `recovery_handlers.go` too, and
+**on its first run convicted a pre-existing unregistered claim**. 8 → 10 registered claims.
 
-- **`demo-felhom`** — up, reporting, healthy, on the vouched pair; `repo_password` restored to the
+**The wire-contract gate: declared, and honestly weaker than it looks (R-315).** The hub response was
-  original (`sha c60c8bc737a6b7c6…`), escrow re-sealed and uploaded, off-site repo reachable
+made a **named type** so the gate could resolve it; the wire is declared as a fourth ROOT and the tag
-  (`restic snapshots` exit 0). Its recovery code was rotated by the final ceremony and
+count rose **174 → 182**, so the fields are inspected. But a positive control — renaming the
-  `R_DEMO-FELHOM` updated in place (prior file backed up alongside). Planted data removed; eight
+agent-side `superseded_at` tag — **still passed**, because the check is repo-wide name-presence and
-  secret-bearing files **shredded**.
+the string also occurs as a map key elsewhere. The gate documents this ("name-reachability is not
-- **`demo-hp`** — untouched, reporting.
+use"), so it is a known limit, not a regression — **but declaring this wire bought documentation, not
-- **`drill-r50`** — **reverted to snapshot `virgin`, powered off.**
+enforcement**, and saying otherwise would have been false.
-- **Hub** — two new retained rows (the P1 and P2 blobs), deliberately kept as the fixture proving the
-  retention works. `drill-r50-0a4f9a` re-used, not duplicated: no new scratch customer.
-- **Off-site** — only demo-felhom's own repository path touched, `backup` the only mutating verb used.
-  **No prune, no forget, no delete, no rename anywhere.** One snapshot added and deliberately left:
-  `6ea85413`, 66 KiB, tagged `drill-retained-key-20260812` — removable by ID if you want it gone.
 
-## Honest gaps
+## 6. Live state
 
-- **The Phase A logs did not survive** the intermediate revert to `virgin`. Every quotation in the
+| | |
-  audit is verbatim from the live run, but the raw files are gone. Procedural lesson, recorded.
+|---|---|
-- The planted data reached the store via `restic` directly, not the dashboard button, because of
+| controller | **0.214.0** on guest 9201, `Up … (healthy)` |
-  R-308 — so the app-backup→unit→offsite chain went unexercised. Not what this drill measured.
+| agent | **0.129.0** on `felhom-pve`, unit active, journal clean |
-- Wall clock **≈ 1 h 13 min** against a 4–5 h envelope. Nothing was dropped; Phase C ran concurrently
+| hub | **0.103.0** — see §7 |
-  with Phase B on a different machine.
+| golden | **0.214.0** baked + published |
+
+### Live proof on hardware — the 422, end to end
+
+```
+OLD code (opens retained row 11)   HTTP 422  opens_retained: True
+                                             superseded_at: 2026-08-12T15:18:55Z
+                                             retained_has_restic_pw: True
+                                             "the recovery code is correct, but it belongs to an
+                                              EARLIER sealed package (superseded …), not the one
+                                              currently held"
+WRONG code (negative control)      HTTP 400  "the recovery code did not open the sealed bundle"
+```
+
+The hub half measured directly too: `GET …/escrow/retained` → **200**, `count=2`,
+**`unopenable_count=1`** — that one being retained row id 4, the pre-v0.93.0 row whose material R-198
+destroyed. The withholding rule is doing exactly what it was written for, on real data.
+
+**What was NOT walked, and why.** The customer's rendered sentence was **not** produced end-to-end.
+`recoveryUnlockHandler` redirects to `/backups/remote` when `!recoveryOffer()`, and `demo-felhom`
+holds its own repository password again (restored yesterday), so it is correctly **not** in the
+offered state. Walking it would mean removing that password to fake a rebuilt box — destabilising a
+healthy machine to render a sentence whose logic is pinned by six handler tests and whose upstream 422
+is proven live. I did not. **Method stated: endpoint-level for the agent and hub, handler-level for the
+message.** What the customer DOES see on this box today is the orphan card, and it is honest:
+*„Megnyitni innen egyelőre nem lehet, és ez nem a kódodon múlik."*
+
+### A correction I have to make about my own last report — R-308 was wrong
+
+I reported that the stored controller password no longer opens `demo-felhom`. **It does.** I had
+stripped only DOUBLE quotes from the `~/.config/credentials` value; the values are wrapped in
+**SINGLE** quotes, so I was sending a literal `'` as part of the password. Unquoted correctly it is 13
+characters and logs in first try — **HTTP 302 with a session cookie**.
+
+The same bug then made this session's first live R-311 test read as a **failure** (HTTP 400) for
+twenty minutes, and I nearly filed the fix as broken. It is the **third** wrong "the credential is
+stale" verdict this project has produced from that one trap. R-308 is **withdrawn**; the real lesson
+is filed with it — never let a shell decide what a secret is.
+
+## 7. What was dropped, named plainly
+
+- **Part 3 (the route) — HALTED at the spike, by the task's own rule.** → R-312.
+- **§7's fixture walk was not re-run end-to-end.** Yesterday's drill already proved the byte-identical
+  restore from a set-aside store; today's change is upstream of it (which sentence is shown), the
+  dashboard is unreachable headlessly (R-308), and the restore route does not exist (R-312). What was
+  proved live is the 422 itself.
+- Explicitly out of scope and still open, so it does not read as forgotten: **R-305** (the removal fix
+  helps a machine once — the tester's second reinstall still hits it), the hub emails naming the
+  retired secret, **R-309** (the runbook's publication claim), the CI runs that fail with no log, the
+  twenty unread facts, the nine grey claims, **R-303**.
+
+## 8. Bypass, stated as required
+
+`git push --no-verify` was used **once**, on `felhom-agent`. The `release-complete` gate refuses a
+CHANGELOG entry whose tag and package do not exist; `release-agent.sh` refuses a tree that is not
+pushed. Circular by construction. The bypass was immediately followed by the real release
+(`release-agent.sh 0.129.0`), and the gates were re-run afterwards: **green**.

STATUS.md

@@ -1,6 +1,6 @@
 # STATUS — what works, what's broken, what's next
 
-**Updated 2026-08-12 (late evening — retained-key drill).**
+**Updated 2026-08-12 (night — the door, part one).**
 
 > **A view, not a source.** `documentation/backlog/OPEN-ITEMS.md` is the authority; this page restates
 > part of it in plain words, and **nothing may exist only here**. **Items, not paragraphs. One screen.**
@@ -8,23 +8,50 @@
 
 ## Waiting on you
 
-*(The golden-vouch and floor-raise asks that stood here are **done** — the hub reads golden **0.213.0**,
+*Three decisions. Each says what it would cost to leave alone, because two of them quietly choose an
-floor **0.213.0**, agent **0.128.0**. Checked live, not assumed.)*
+outcome if you do not answer. The register row is the detail, not the decision.*
 
-- **R-307 — a countdown is already running on `demo-felhom`**, started 10 Aug, **firing 24 August**,
+### Should a customer be able to get their old backups back themselves, or is that a phone call?
-  after which that set-aside off-site copy is deleted for good. Today's drill did not start it and
+
-  deliberately did not cancel it. Cancel it (copy kept, costs storage) or let it run (copy deleted).
+Since Tuesday a machine recognises an older recovery code and says so honestly, but it cannot hand the
-  **Doing nothing chooses deletion.**
+files over — it tells the person to write to us, and we can do it by hand.
-- **R-304 — decide what we promise about old backups.** The retention works, but only an operator can
+
-  use it; the customer is told their correct code is wrong. Either build the door, or stop promising.
+- **Build it:** the restore code has to accept a second location and password instead of only its own.
-- **R-303 — one coherence decision**, ranked low: a box can still raise the "we cannot open your new
+  Contained — three functions and a screen — plus one genuine design question: what a customer sees
-  backups" card while counting down to deleting the old ones. The two no longer contradict each other,
+  when they have several old sets and must pick one.
-  but the state is odd and the wrong fix would hide a real second fault.
+- **Leave it:** nothing breaks. Every customer in this position becomes a support conversation, and we
+  keep a promise we can only keep manually.
+
+**If you do nothing:** the honest message stays and the work never gets scheduled. Nobody is blocked;
+this is the one decision here with no deadline of any kind. *(register: R-312)*
+
+### The old copy on the demo machine cannot be opened by anyone. Keep paying to store it, or delete it?
+
+You told me to keep it, and I did. Then I found out what it is: 36 backups and a single key, and that
+key was destroyed by the bug we fixed on 4 August. **No recovery code in existence opens it.**
+
+- **Keep it:** pennies of storage, and it stays as the one physical example of what that bug cost.
+- **Delete it:** irreversible, and the example goes with it.
+
+**If you do nothing:** it stays forever and stops being a decision — which is how five scratch
+customers accumulated. Nobody is blocked. *(register: R-313)*
+
+### When a machine is in two kinds of trouble at once, should it say both things?
+
+A machine can count down to deleting its old backups while also reporting that it cannot open its new
+ones. Both cards are true; together they are bewildering.
+
+- **Leave it:** two true statements, confusing side by side. Nobody has been hurt by it.
+- **Hide the older card during a countdown:** tidier, and it risks hiding a real second failure — which
+  is why I have not done it.
+
+**If you do nothing:** both keep showing. Ranked low on purpose. *(register: R-303)*
 
 ## What works
 
-Both demo machines are home, healthy and reporting on the approved pair. Off-site is credentialed on
+Both demo machines are home, healthy and reporting on the approved pair — **controller 0.214.0, agent
-`demo-hp` and its repository still opens with the machine's own key. `drill-r50` is blocked, as intended.
+0.129.0**, delivered by the floor rather than by hand. Off-site is credentialed on `demo-hp` and its
+repository still opens with the machine's own key. `drill-r50` is reverted to `virgin`, powered off.
 
 ## Shipped
 
@@ -39,6 +66,16 @@ Both demo machines are home, healthy and reporting on the approved pair. Off-sit
   fifth deliberately left, because it is true where it renders.
 - **One name per secret, box side** (R-295): the dashboard code is „Beállító kód" everywhere;
   „Visszaállító kód" is retired. It collided with the escrow „Helyreállítási kód" and cost a real code.
+- **A correct recovery code is no longer called wrong** (R-311, three components). If a customer types
+  the code for an older set of backups, the machine now checks the packages we kept, recognises it, and
+  says so: *your code is correct, it belongs to an earlier package, we kept it, your current backups are
+  fine, write to us*. It deliberately promises no restore, because there is no button yet.
+- **The countdown on `demo-felhom` is cancelled** on your ruling (R-307). Nothing was deleted; the
+  24 August deadline is gone. See R-313 for what that copy turns out to be.
+- **Vouched and delivered 2026-08-12**: golden 0.214.0, agent 0.129.0, floor 0.214.0 — both machines
+  took it themselves. The version guard was watched working on the way: `demo-hp` was **held** back
+  while its agent was older, and updated 6 seconds after the agent caught up. That guard exists because
+  a machine once ran ahead of its agent and a customer was told a correct code was wrong; first sighting.
 - **Both installer fixes are now PUBLISHED** as `installer-v1.27.0` (R-297 + R-300). Each fault was
   watched happening first, on a machine reset to factory state: the old installer really did build a
   machine on a base image from July, and our own uninstall really did block our own next install.
@@ -50,11 +87,10 @@ Both demo machines are home, healthy and reporting on the approved pair. Off-sit
   everything on it is lost.** First act of the visit: copy the ~3.6 GB off before anything is
   reinstalled — it is currently the only copy in existence. Whether it stays parked is your call and is
   deliberately left open.
-- **Kept backups can be opened — but not by the customer** (R-304). We now know for certain the old key
+- **Kept backups can be opened — but still only by us** (R-304 partly closed, R-312 open). The machine
-  is kept and **does** open the old backups: three planted files came back byte-for-byte from a store the
+  now recognises an older code and says so plainly instead of hedging. What it still cannot do is hand
-  machine itself could no longer read. But nothing in the product reads a kept key, so it takes an
+  the customer their old files: that needs the restore code to accept a second location, which is real
-  operator with a shell. Worse, a customer who types their **correct** old code is told it *"did not open
+  work rather than wiring. Today the honest answer is "your code is right, write to us" — and we can.
-  the sealed bundle"*. The keeping is real; the promise that they can use it is not.
 - **The dnsmasq fix helps a machine once** (R-305). On a machine that never had Felhom it works. On the
   second reinstall the leftover comes back, because the package is never removed — so the machine looks,
   to our own installer, as if the household had installed it. Watched happening the same afternoon.
@@ -64,5 +100,6 @@ Both demo machines are home, healthy and reporting on the approved pair. Off-sit
 
 ## Working on next
 
-Your two decisions above (R-307's deadline is the hard one); then R-304's shape; then the hub naming;
+R-312's shape (the button, or deliberately no button); then R-305, because the tester's second
-then the 2026-08-09 batch (R-279 … R-292), still untriaged against everything since.
+reinstall still hits the dnsmasq wall; then the hub naming; then the 2026-08-09 batch
+(R-279 … R-292), still untriaged against everything since.

documentation/audits/DRILL-retained-key-2026-08-12.md

@@ -450,8 +450,8 @@ whose credentials reach nothing else — and the only mutating verb used was `ba
 | Phase A — removal leftover | 16:50 → 17:12 | **~22 min** (4 installs, 4 uninstalls, 3 preflights) |
 | Phase B — stale golden | 17:13 → 17:34 | **~21 min** (2 full provisions) |
 | Phase C — retained key | 17:14 → 17:23 | **~25 min** (overlapped Phase B; different machine) |
-| Record, register, publication, teardown | 17:35 → 17:55 | **~20 min** |
+| Record, register, publication, teardown | 17:34 → 17:45 | **~11 min** |
-| **Total** | **16:42 → 17:55** | **≈ 1 h 13 min** |
+| **Total** | **16:42 → 17:45** | **≈ 1 h 03 min** |
 
 Phase C ran concurrently with Phase B deliberately — different machines, no shared state. The
 4–5 hour envelope was not needed; nothing was hurried and nothing was dropped for time.

documentation/backlog/OPEN-ITEMS.md

@@ -441,7 +441,7 @@ builds the receiving struct by hand cannot see a field that never decodes, which
 |---|---|---|
 | **R-264** | **Twenty-one facts the boxes report that the hub can now decode nowhere, each allowlisted with a reason rather than silently skipped — and for these the reason is "no consumer today, and one is arguably owed".** Split out of R-260 on 2026-08-08 so that closing the CLASS (gated) and fixing its sharpest instance (`operator_key_configured`) could not be mistaken for having decided what the hub should do with the rest. **The list, grouped by what a consumer would be for.** **(a) Guest-network health — `guest_net` and its seven children** (`checked_at`, `has_route`, `dhclient_alive`, `heal_succeeded`, `heals_last_hour`, `last_heal_at`, `damped`). The R-54 watchdog reports per-guest network state and self-heal counts every cycle and the hub — the component that emails the operator — models none of it. There is a live incident in this project's own record where a killed `dhclient` took a tunnel down for 1 h 15 m (`audits/INCIDENT-guest-dhclient-killed-2026-07-20.md`); a recurring-heal signal is exactly what would have surfaced it. **This is the strongest candidate of the twenty-one.** **(b) `selfupdate_pending` + `selfupdate_pending_version`** — an agent that has flipped its binary and never committed reports pending on every heartbeat so that "the operator sees WHY the version isn't advancing", and no operator can see it. **(c) `mgmt_plane.healed_recently`** — bounded: the hub DOES alarm on the `privsep_healed_at` timestamp beside it, so the recurring-clobber signal is not lost, only this flag. **(d) `restore_tests.mount_parity` + `mount_inventory`** — R-262's subject; the verdict is not lost (a mismatch fails before `Pass` is set) but the hub cannot tell a full-fidelity pass from a boot-only one. **(e) `pbs_dr.applied_at`.** **(f) Controller-side: `config_hash`, `reporting_disabled`, `stacks`, `storage.migrated_to`, `backup.last_db_dump`, `backup.last_integrity_check`** — the last two are backup-integrity timestamps, which is the "presence is not success" neighbourhood. **For each the question is the same and is NOT answered here: is it wanted? If the hub should act on it, model it and name what consults it. If it should not, the honest end is that the emitter stops sending it** — a fact emitted forever and consumed nowhere is a future false green waiting for someone to write a check against it. **Deliberately not decided in the G-1 session**, whose scope was the gate plus the operator-access instance; unilaterally removing emitters would also break the byte-identical cross-repo host-report golden and is a coordinated two-repo change | **READY** — owner Viktor |
 
-| **R-265** | **A CI run can fail with NO LOG PERSISTED, and the alarm mail then points the operator at a log that does not exist.** Observed 2026-08-08 as run **264** (`650cc8a`, a **documentation-only** commit) sat between two green runs of identical gate code. **Measured rather than assumed — the shape is unmistakable:** every other run in the session took **18–34 s and has a log (HTTP 200)**; 264 took **834 s** (07:12:40 → 07:26:34 UTC) and `GET /actions/jobs/264/logs` returns **HTTP 500 — `actions_log/…/264.log.zst: file does not exist`**. The runner pod never restarted (`act-runner`, 0 restarts, 5 d 17 h uptime), so the runner did not die; the JOB hung and was reaped. **It is NOT a gate finding, and four independent facts say so:** the diff from the green run before it is Markdown only; the same content is green two commits later (run 265, `dd55a3f`, 33 s); the gate code is byte-identical across 263/264/265; and 260–262, which WERE real gate failures, all failed in under 35 s **with** logs. **THE CAUSE OF THE HANG IS UNDETERMINED and is deliberately not guessed at.** DooPlex was doing heavy work in that window (a 139 MB `kubectl cp`, and a `go run` compiling the whole hub module for the live-validation harness), which is a plausible contention story — but the box has 40 cores and sat at load ~5, so it is **not established** and is recorded as a hypothesis, not a cause. **THE FINDING THAT MATTERS IS THE SECOND-ORDER ONE, and it is this workflow's own stated purpose turned against it.** `gates.yml` exists because "a detector nobody hears is the defect R-29 filed, rebuilt one layer up", and its alarm mail says *"The failing gate names itself in the run log."* **Here there is no run log**, so an operator following that sentence finds nothing and cannot tell an infrastructure reap from a real conviction. Worse and **unverified**: the alarm step is `if: failure()`, and whether it even ran for a reaped job is unknown — if it did not, this was a red CI that alarmed nobody, which is exactly the shape the workflow was built to prevent. **Fix shape, not a decision:** (a) make the alarm mail state the run's DURATION and whether a log exists, so a log-less reap is self-identifying; (b) give the job an explicit `timeout-minutes` well under the reap so it fails fast, loudly and with a log; (c) establish whether the alarm fires at all on a reaped job — that is one deliberate test, and until it is run, "CI alarms on failure" is an assumption  **SECOND SIGHTING, 2026-08-10 — and I nearly missed it.** felhom.eu CI **run 293** (`67eced8`, the demo-felhom recovery commit) **FAILED**, and its log is **unretrievable**: `/actions/jobs/293/logs` returns **HTTP 500**, while the runs either side of it (292 and 294) both return HTTP 200 with 12 290 bytes. So the failure is real and its reason is **not establishable** — exactly the shape this row was filed for. **What can be said:** the same tree passes the FULL gate set locally (`GITEA_BASE=… python3 scripts/repo_gates.py`, all eight OK), and run **294** on the very next commit passed in CI, so nothing persisted. **What cannot:** why 293 failed. **And the process failure is mine** — the end-of-session checklist says confirm your own push's CI run by ID, and I pushed 67eced8 without doing so; it was found only because the NEXT session happened to list recent runs. A pull check that is skipped is not a check| **CLOSED 2026-08-08 — `timeout-minutes: 5` on the gates job, and the alarm mail now states elapsed seconds and qualifies its own "names itself in the run log" sentence.** ⚠ **The unknown is NOT closed and must not be read as closed:** whether the `if: failure()` alarm fires for a REAPED job is still unverified. The timeout makes the reap unreachable in practice; it does not answer what happens inside one |
+| **R-265** | **A CI run can fail with NO LOG PERSISTED, and the alarm mail then points the operator at a log that does not exist.** Observed 2026-08-08 as run **264** (`650cc8a`, a **documentation-only** commit) sat between two green runs of identical gate code. **Measured rather than assumed — the shape is unmistakable:** every other run in the session took **18–34 s and has a log (HTTP 200)**; 264 took **834 s** (07:12:40 → 07:26:34 UTC) and `GET /actions/jobs/264/logs` returns **HTTP 500 — `actions_log/…/264.log.zst: file does not exist`**. The runner pod never restarted (`act-runner`, 0 restarts, 5 d 17 h uptime), so the runner did not die; the JOB hung and was reaped. **It is NOT a gate finding, and four independent facts say so:** the diff from the green run before it is Markdown only; the same content is green two commits later (run 265, `dd55a3f`, 33 s); the gate code is byte-identical across 263/264/265; and 260–262, which WERE real gate failures, all failed in under 35 s **with** logs. **THE CAUSE OF THE HANG IS UNDETERMINED and is deliberately not guessed at.** DooPlex was doing heavy work in that window (a 139 MB `kubectl cp`, and a `go run` compiling the whole hub module for the live-validation harness), which is a plausible contention story — but the box has 40 cores and sat at load ~5, so it is **not established** and is recorded as a hypothesis, not a cause. **THE FINDING THAT MATTERS IS THE SECOND-ORDER ONE, and it is this workflow's own stated purpose turned against it.** `gates.yml` exists because "a detector nobody hears is the defect R-29 filed, rebuilt one layer up", and its alarm mail says *"The failing gate names itself in the run log."* **Here there is no run log**, so an operator following that sentence finds nothing and cannot tell an infrastructure reap from a real conviction. Worse and **unverified**: the alarm step is `if: failure()`, and whether it even ran for a reaped job is unknown — if it did not, this was a red CI that alarmed nobody, which is exactly the shape the workflow was built to prevent. **Fix shape, not a decision:** (a) make the alarm mail state the run's DURATION and whether a log exists, so a log-less reap is self-identifying; (b) give the job an explicit `timeout-minutes` well under the reap so it fails fast, loudly and with a log; (c) establish whether the alarm fires at all on a reaped job — that is one deliberate test, and until it is run, "CI alarms on failure" is an assumption  **SECOND SIGHTING, 2026-08-10 — and I nearly missed it.** felhom.eu CI **run 293** (`67eced8`, the demo-felhom recovery commit) **FAILED**, and its log is **unretrievable**: `/actions/jobs/293/logs` returns **HTTP 500**, while the runs either side of it (292 and 294) both return HTTP 200 with 12 290 bytes. So the failure is real and its reason is **not establishable** — exactly the shape this row was filed for. **What can be said:** the same tree passes the FULL gate set locally (`GITEA_BASE=… python3 scripts/repo_gates.py`, all eight OK), and run **294** on the very next commit passed in CI, so nothing persisted. **What cannot:** why 293 failed. **And the process failure is mine** — the end-of-session checklist says confirm your own push's CI run by ID, and I pushed 67eced8 without doing so; it was found only because the NEXT session happened to list recent runs. A pull check that is skipped is not a check| **CLOSED 2026-08-08 — `timeout-minutes: 5` on the gates job, and the alarm mail now states elapsed seconds and qualifies its own "names itself in the run log" sentence.** ⚠ **The unknown is NOT closed and must not be read as closed:** whether the `if: failure()` alarm fires for a REAPED job is still unverified. The timeout makes the reap unreachable in practice; it does not answer what happens inside one | **THIRD SIGHTING 2026-08-12 (controller run 324).** Failed after **11 minutes** where the previous run took 18 s; `/actions/runs/324/logs` and the web log endpoint both **404**. The jobs API *does* answer, and it names the failing step: **"Fetch the pushed commit and the sibling clone it needs"** — i.e. it died at the clone, before any gate ran, while a `felhom.eu` push was in flight. **Diagnosed rather than assumed:** the same sha passed all 11 gates locally on a clean tree, and an API **re-run of the same sha (326) succeeded** with no code change. So the jobs endpoint is a usable substitute for the missing log — it gives the failing STEP, which is most of what the operator needs — and `POST /actions/runs/<id>/rerun` (HTTP 201) is the cheap disambiguator between environmental and real.
 
 **Explicitly still open, untouched by this session:** R-246 (the wrong stale flag on `demo-hp` —
 clearing it is an operator act hub-side), R-255, R-256, R-257, R-258, R-259, R-261, R-262, R-263, and
@@ -605,9 +605,17 @@ class (an image `VOLUME` at an unmounted path) is still live — `immich-server`
 | **R-302** | **The abandon banner promised retrieval it could not see was still true — fixed by PINNING a fingerprint at the decision.** The countdown strip said *„Addig még visszaszerezheted őket a helyreállítási kóddal"* unconditionally, on every page, and R-301 established it can co-render with the orphan card correctly saying we cannot tell. **THE OBVIOUS CONDITION WAS REJECTED AND HERE IS WHY, so nobody re-proposes it:** the proxy *"does the hub hold a key different from the one I use?"* asks about the WRONG key — the set-aside copies were written under an OLDER key the box no longer has, which is why they were set aside — so on a twice-rebuilt box it answers "promise it" about copies no key on file can open. **Demonstrated, not argued:** under the proxy both Scenario B (package replaced) and Scenario D (legacy countdown) flip back to promising. Instead `startAbandonCountdown` pins `AbandonPinnedEscrowKeySHA256` — the hub's escrow key fingerprint AS CACHED AT THE DECISION — written once, never refreshed; the banner then asks only *"is the hub still holding that same package?"*. **⚠ THE PIN IS A RECORDED ASSUMPTION:** nothing on the box records which key wrote the set-aside copies, so it presumes the package held at the decision is that one — true in the ordinary rebuilt-box story, NOT provable, wrong on a twice-rebuilt box. Recorded in the field comment so it can be narrowed rather than hardening into a fact. Empty is not a match on either side; a countdown started before v0.213.0 carries no pin and takes the cautious branch (deliberately NOT backfilled). **A template sweep found a FOURTH instance** (the backups-page abandon block — same condition applied, since fixing the strip and not the page would leave them contradicting) **and a FIFTH** (the abandon confirmation screen, deliberately LEFT: it renders at the moment of the decision, where the promise is true by construction because that is the package about to be pinned) | **CLOSED — controller v0.213.0** | R-301 | — | CC |
 | **R-303** | **`markOrphaned` has no guard against an active abandon countdown — the co-render is made HARMLESS, not IMPOSSIBLE.** `ensureOffboxRepo` calls `markOrphaned()` for a claimed box (`offbox.go:804`) with no check on `AbandonAt`, so a later run finding the FRESH store unopenable re-raises the orphan card while the countdown runs. R-302 ensures the two surfaces no longer contradict each other in that state, but the state itself is still reachable and is arguably incoherent: a box counting down to deleting its old history while simultaneously reporting its NEW history is unopenable is in trouble in two ways at once and says so in two separate cards. **Ranked LOW deliberately** — it is a coherence question, not a correctness one, and the wrong fix (suppressing the orphan card during a countdown) would hide a real second fault | **READY (S) — NEW 2026-08-12** | R-302 | Decide whether a countdown should suppress, defer or annotate the orphan card — and beware hiding a genuine second failure | operator + CC |
 | **R-304** | **The retained escrow key works, and the customer is told their correct code is wrong.** DRILL 2026-08-12 answered the three questions separately, on `demo-felhom`, with planted data. **(a) retention: WORKS** — the first retained row in fleet history to carry material (`host_escrow_superseded` id 11, `identity_blob` 572 B), byte-identical (`sha256 a10032341c8584ed…`) to the pre-supersession `host_escrow` row. **(b) the material opens the old store: YES** — unsealed with the OLD recovery code it yielded a password byte-identical to the pre-change one (`sha c60c8bc737a6b7c6…`), and restored three planted files **byte-identical** from a store the box itself could no longer open (negative control first: `Fatal: wrong password or no key found`), **including a Hungarian accented filename verified as raw bytes**. **(c) the customer's route: DOES NOT EXIST, and misinforms.** `ListSupersededEscrow` (`store.go:2841`) is the only reader of a retained `identity_blob` and has **zero production callers** — five call sites, all `_test.go`; the product path (`POST /escrow/recover-offsite-password` → `FetchIdentityEscrow` → `GetHostDRBundle`, `store.go:3152`) selects `FROM host_escrow` — the CURRENT row only. Asked for the old password with the code that demonstrably opens the retained row, the product answered **"the recovery code did not open the sealed bundle — nothing was written"**. **This is the R-224 class again**: there an unreachable hub was reported as a bad code; here a VALID code for retained history is reported as a bad code, and the customer's attempt ends there. **Consequence:** the census answer stands (it was about retention); the countdown banner's promise is true in substance and false in practice; **any capability-map claim that the customer can recover the old history with their recovery code is false today and must move** | **READY (L) — NEW 2026-08-12, RANK 1** | R-198, R-199, R-224, R-241 | Decide the shape: serve retained rows on the recovery path (needs a "which package?" choice — a customer may have several), or stop promising retrieval anywhere the customer cannot perform it. **Until one of those, the honest position is that retention is an operator-only capability.** At minimum, the refusal must stop asserting the code is wrong when the hub simply never looked | operator + CC |
-| **R-305** | **The R-300 cleanup fires exactly once per machine, and the second reinstall hits the original wall.** PROVEN on `drill-r50` 2026-08-12, not deduced. The uninstall stops+disables dnsmasq but deliberately never **purges** the package, so on every later install `dpkg-query` reports it installed, preflight records `dnsmasq_preexisting=yes`, the agent's `EnsureDnsmasq` re-enables it, and the uninstall then only *restarts* it — leaving it `enabled`/`active` on `0.0.0.0:53` exactly as before the fix. Cycle 1: `no` → disabled → `:53 FREE` → next install PASSES. Cycle 2: `yes` → left running → **cycle-3 byo preflight FAILED, exit 1**. The improved refusal does fire and names the command, so the customer is not stranded — but they must run by hand a cleanup Felhom could have run itself, on a box Felhom does own | **READY (M) — NEW 2026-08-12, RANK 2** | R-300 | Distinguish "the package pre-dates Felhom" from "a previous **Felhom** install left it" — e.g. keep the ownership record outside `state.json` (which the uninstall deletes), or record Felhom-installed-ness where the agent installs it. **Do not fix by purging on hosts we may not own** | CC |
+| **R-305** | **The R-300 cleanup fires exactly once per machine, and the second reinstall hits the original wall.** PROVEN on `drill-r50` 2026-08-12, not deduced. The uninstall stops+disables dnsmasq but deliberately never **purges** the package, so on every later install `dpkg-query` reports it installed, preflight records `dnsmasq_preexisting=yes`, the agent's `EnsureDnsmasq` re-enables it, and the uninstall then only *restarts* it — leaving it `enabled`/`active` on `0.0.0.0:53` exactly as before the fix. Cycle 1: `no` → disabled → `:53 FREE` → next install PASSES. Cycle 2: `yes` → left running → **cycle-3 byo preflight FAILED, exit 1**. The improved refusal does fire and names the command, so the customer is not stranded — but they must run by hand a cleanup Felhom could have run itself, on a box Felhom does own **CLOSED 2026-08-13 by R-316** — reproduced on `drill-r50` from `virgin` on the published v1.27.0 (cycle 2 recorded `yes`, cycle 3 refused exit 1), then fixed by removing the package the record says we installed, and watched passing on cycle 3. | **CLOSED — superseded by R-316** | R-300 | Distinguish "the package pre-dates Felhom" from "a previous **Felhom** install left it" — e.g. keep the ownership record outside `state.json` (which the uninstall deletes), or record Felhom-installed-ness where the agent installs it. **Do not fix by purging on hosts we may not own** | CC |
 | **R-306** | **`--preflight-only` says "no state written" and writes state — with an answer that can be wrong.** `_state_put` short-circuits on `DRY_RUN` only (`felhom-host-install.sh:418`), so a preflight-only run creates `/var/lib/felhom-install/state.json`. Observed live: after a run whose banner read `PRE-FLIGHT PASS (mode=byo) — no state written, no install step executed`, the file existed containing `{"completed": [], "dnsmasq_preexisting": "yes"}`. Both the banner and the flag's own comment at line 226 assert the opposite. **The harm is not the file, it is the value**: the runbook recommends preflight-only first, then the same command without the flag, so on a box carrying a Felhom leftover the wrong ownership answer is baked in before the real install begins | **READY (S) — NEW 2026-08-12, RANK 3** | R-300, R-305 | Either make `_state_put` a no-op under `PREFLIGHT_ONLY` (and record ownership at install instead), or correct both claims. A comment asserting an invariant needs a test pinning it | CC |
-| **R-307** | **`demo-felhom` carries a LIVE abandon countdown that this drill did not start — and the end state says there should be none.** `settings.json`: `abandon_started_at 2026-08-10T08:06:31Z`, `abandon_at 2026-08-24T08:06:31Z`, `abandon_repo_path /home/felhom-repo.orphaned-20260810`. The drill's fence forbade starting, shortening or triggering a countdown, and none was; but its required end state was *"no abandon countdown anywhere"*, and one exists. **The two ways to satisfy that are not equivalent and the choice is the operator's:** cancel it (the orphaned repository is kept indefinitely — storage cost, no data risk) or let it run (on **2026-08-24** that repository is deleted, **irreversibly**). **Doing nothing selects deletion by default.** Deliberately not resolved by CC | **BLOCKED ON OPERATOR — NEW 2026-08-12** | R-301, R-302, R-303 | Decide before 2026-08-24 | operator |
+| **R-307** | **`demo-felhom` carries a LIVE abandon countdown that this drill did not start — and the end state says there should be none.** `settings.json`: `abandon_started_at 2026-08-10T08:06:31Z`, `abandon_at 2026-08-24T08:06:31Z`, `abandon_repo_path /home/felhom-repo.orphaned-20260810`. The drill's fence forbade starting, shortening or triggering a countdown, and none was; but its required end state was *"no abandon countdown anywhere"*, and one exists. **The two ways to satisfy that are not equivalent and the choice is the operator's:** cancel it (the orphaned repository is kept indefinitely — storage cost, no data risk) or let it run (on **2026-08-24** that repository is deleted, **irreversibly**). **Doing nothing selects deletion by default.** Deliberately not resolved by CC. **RULED 2026-08-12 (operator): KEEP the set-aside history — cancel the countdown.** Done the same evening through the product's own operator path (`--abandon-stop`, which refuses rather than silently no-opping), with the container stopped first so the running controller could not overwrite `settings.json` from memory. **Proved, not trusted to the exit code:** `abandon_started_at` and `abandon_at` are GONE (`AbandonStatus` returns `Active=false` when `AbandonAt` is empty — `offbox_abandon.go:111-113`, so no countdown renders); `abandon_repo_path` deliberately REMAINS as the pointer to the preserved store; the store still holds **36 snapshot objects** and its full `config/data/index/keys/locks/snapshots` structure; both repositories still present on the endpoint; **nothing deleted anywhere**. See R-313 — what was preserved cannot currently be opened by anyone | **CLOSED — countdown cancelled 2026-08-12 on the operator's ruling** | R-301, R-302, R-303, R-313 | — | CC |
-| **R-308** | **The stored controller password no longer opens `demo-felhom`.** `PASSWORD` in `~/.config/credentials` on DooPlex returns `Hibás jelszó` against `https://127.0.0.1:443/login` with `Host: felhom.demo-felhom.eu`. **NOT the known quoting trap** — the surrounding quotes were stripped before use and the stored value is unquoted (16 bytes for 15 chars), so this is a genuinely stale credential, not a mis-read. Cost during the drill: the dashboard was unreachable headlessly, so the offsite push had to be made with `restic` directly instead of the page's own button, which left the app-backup→unit→offsite chain unexercised. `R_DEMO-FELHOM` **was** correct and has since been rotated by this session's final ceremony (updated in place; prior file backed up alongside) | **READY (S) — NEW 2026-08-12, RANK 2** | — | Recover or reset the demo-felhom dashboard password and re-store it; note which boxes share one and which do not | operator |
+| **R-308** | **~~The stored controller password no longer opens `demo-felhom`~~ — WITHDRAWN 2026-08-12, this was MY BUG, not a defect.** The original diagnosis (`Hibás jelszó`) came from stripping only DOUBLE quotes off the `~/.config/credentials` value; the values are wrapped in **SINGLE** quotes, so a literal `'` was sent as part of the password. Correctly unquoted the value is 13 characters and logs in first try — **HTTP 302 with a `felhom_session` cookie**, measured. **This is the third time this project has produced a wrong 'the credential is stale' verdict from that exact trap**, and the standing memory about it is what caught it here — but only after the same mistake had also made the R-311 live test read as a FAILURE (HTTP 400) for twenty minutes. **The lesson is not 'strip quotes' but 'never let a shell decide what a secret is'**: parse the credentials file in Python, strip a matched pair of EITHER quote, and verify the length before use | **WITHDRAWN — not a defect (my error)** | — | Consider a tiny `read_credential.py` helper so no session hand-rolls the parse again | CC |
 | **R-309** | **The day-0 runbook says pushing the installer publishes it. It has not since R-110.** `documentation/runbooks/day0-install.md` §C.1: *"it is always the current `main` script. There is no release tag, no staging copy and no version selector; pushing `scripts/felhom-host-install.sh` publishes it."* R-110 moved `/scripts/` to the **installer tag's** tree — `manifests/webpage.yaml` carries **two** `--ref=installer-v…` pins (lines 327, 372) and `hostinstall_gates.py` gate 6 enforces that they name a tag. Confirmed live 2026-08-12: `https://felhom.eu/scripts/felhom-host-install.sh` served `SCRIPT_VERSION="1.25.0"` while `main` had `1.27.0`, three and a half hours after the push. **The runbook's version is the dangerous one to believe in both directions** — it would have an operator think a pushed fix is live when it is not, and think a pushed mistake is live when it is not | **READY (S) — NEW 2026-08-12, RANK 3** | R-110 | Correct §C.1 to name the tag-and-manifest publication step (as performed today) | CC |
 | **R-310** | **Two small edges on the installer, neither costing more than a moment.** (1) The R-297 operator-named refusal states the vouched version twice in consecutive sentences (*"…but the vouched golden is 0.213.0. The vouched golden is 0.213.0."*). (2) `--uninstall` reads its typed vmid confirmation from `/dev/tty` and `--force` deliberately does **not** bypass it, so teardown cannot be scripted without a pty — correct for an irreversible destroy, but undocumented; it surfaces as `line 891: /dev/tty: No such device or address` and an rc=1 that looks like a failure rather than a refusal to proceed unattended | **READY (S) — NEW 2026-08-12, RANK 4** | R-297 | Drop the duplicated sentence; add one runbook line naming the pty requirement | CC |
+| **R-311** | **A correct recovery code for a retained package stopped being reported as wrong.** The drill of 2026-08-12 proved the retained key WORKS (it opened a set-aside store and restored planted files byte-identical) while the product answered *"the recovery code did not open the sealed bundle"*. **What was already right:** the customer-facing screen did not bluntly accuse — R-222/R-226 hedged, naming both causes and the kept package. **What was wrong was the hedge's reason:** *„innen nem tudjuk megkülönböztetni őket"* was true only because NOTHING EVER LOOKED. **Shipped, three repos.** Hub v0.103.0: `GET /hosts/<id>/escrow/retained` — the first production caller `ListSupersededEscrow` has ever had — self-scoped, same recovery-mode gate, same audit event before the bytes leave, rows with a NULL `identity_blob` WITHHELD and returned as `unopenable_count` (they can never open anything, and serving them would let the screen promise recovery on exactly the boxes the original defect hurt), capped at 16. Agent v0.129.0: `FetchRetained` consulted ONLY after the current package refuses, `ErrCodeOpensRetained` + the supersession date, local API answers **422**, bounded at 6 attempts (~1 s of scrypt each), and fail-safe in every direction — a nil fetcher, a 404 from an older hub, a transport failure or a malformed package all leave the original refusal standing. Controller v0.214.0: class `RecoveryCodeOpensRetained` gated on `FeatureRetainedRecoveryClass` (MinAgent 0.129.0, SEPARATE from the R-224 gate because a box can sit between 0.126.0 and 0.129.0), and a message that says the code is correct, names the date, says the earlier package is kept, says the CURRENT backups are unaffected, and **deliberately promises no restore** — it routes to support, which can do it by hand. **The lie's true home was the agent:** removing the CONTROLLER case yields the neutral message (R-224's safe default catches it); removing the AGENT's retained lookup brings back the fail-closed wrong-code error in those exact words — both demonstrated, mutations asserted applied **DELIVERED FLEET-WIDE 2026-08-12 night.** Operator vouched golden 0.214.0 / agent 0.129.0 / min agent 0.129.0 and raised the floor to 0.214.0; both artifact shas in `hub_settings` match the bake and the release byte-for-byte (`3a40379c…`, `53a54f06…`). **The Part D / R-216 hold was OBSERVED FIRING for the first time** — with the floor at 0.214.0 and `demo-hp` still on agent 0.128.0 the hub logged *"managed floor HELD for demo-hp: agent 0.128.0 < MinAgent 0.129.0 (controller floor withheld)"*, and the box took the update **6 seconds** after its agent was brought to 0.129.0 (`auto-floor`, 19:03:18→19:03:24Z). That guard exists because a box once landed on a controller newer than its agent and its customer was told a correct recovery code was wrong; **the MinAgent declaration in the CHANGELOG header is what made it hold**, which is the whole argument for the three-part coupling rule. Both demo boxes now run controller 0.214.0 + agent 0.129.0, verified ON the boxes. | **CLOSED — shipped + delivered: hub v0.103.0 + agent v0.129.0 + controller v0.214.0** | R-199, R-222, R-224, R-226, R-304 | — | CC |
+| **R-312** | **There is no in-product route from the recovery screen to a set-aside store, and building one is not wiring — it is new surface.** Established read-only before any code was written (the session's §4 spike). Every restore entry point resolves the repository from `m.settings.GetOffboxTarget()` and the password from the single `offboxPwPath()` file: `offboxLatestSnapshot` (`offbox_restore.go:85-86`), `offboxSnapshotSize` (`:139-140`), `RestoreOffboxScratch` (`:206+`). **There is no repo-path parameter anywhere in the chain** — a grep for one returns nothing. The only existing seam that installs a recovered password, `InjectOffboxPassword` (`offbox.go:667`), writes that same one file, i.e. **ADOPTS** the set-aside store as the machine's current target. So the two options are (a) thread an alternative (repo, password) through three functions plus the UI, or (b) adopt — and adoption is a different product decision. **The session HALTED here by its own rule** and shipped R-311 alone. What the drill did to read the set-aside store was `restic` by hand with `-r <alt repo>` and an overridden `RESTIC_PASSWORD_FILE`; **that distance is exactly what (b)-to-(c) costs** | **READY (L) — NEW 2026-08-12** | R-304, R-311 | Decide (a) vs (b). If (a): the parameter is small and contained; the UI question ("which package?" when a customer has several) is the real design work. **Do not choose (b) without deciding what happens to the machine's current off-site history** | operator + CC |
+| **R-313** | **`demo-felhom`'s set-aside store is UNRECOVERABLE — 36 snapshots whose key we destroyed ourselves.** `/home/felhom-repo.orphaned-20260810` holds **36 snapshot objects and exactly one key slot**, and it does NOT open with the box's current password (`Fatal: wrong password or no key found`, exit 1 — measured). Its password is the one hashed `48741892f0ef4d59…`, which is retained row id 4 — **`identity_blob` NULL**, a pre-v0.93.0 row. So the material was dropped by the R-198 defect during its two-month window, and no recovery code in existence opens that store. **This is the concrete, still-present cost of R-198**, sitting on the endpoint rather than in a post-mortem. It also means the operator's ruling to KEEP it (R-307, countdown cancelled — see below) preserves bytes nobody can read: correct as a decision, and worth knowing as a fact | **READY (S) — NEW 2026-08-12** | R-198, R-307 | Decide whether to keep paying storage for 36 unreadable snapshots, or delete them deliberately and record why. **Not urgent and not automatic — but it should be a decision rather than an accumulation** | operator |
+| **R-314** | **`StopAbandon` has no web route — a customer who telephones is served by a command line.** `--abandon-stop` exists on the controller binary (`cmd/controller/main.go:86`) and refuses rather than silently no-opping, which is right. But there is no handler: the only in-product way to cancel a countdown is the customer finding their recovery code (Scenario G cancels it at the moment the code proves they still have it). **An operator who is telephoned instead has to reach a shell on the customer's machine.** Used this session on the operator's ruling, container stopped first so the running controller could not overwrite `settings.json` from memory — a sequencing subtlety that is itself an argument for a route | **READY (S) — NEW 2026-08-12, RANK 3** | R-241, R-307 | An operator-authenticated POST that calls the same `StopAbandon`, so the telephone path and the code path converge | CC |
+| **R-315** | **The wire-contract gate's positive control FAILS on the new wire: it checks name-presence, not decodability.** R-311 declared `hub -> agent (GET /escrow/retained)` as a fourth ROOT, and the gate's tag count rose 174 → 182, so the fields ARE inspected. But renaming the agent-side `superseded_at` json tag to `superseded_at_RENAMED` **still passed** — because the string `superseded_at` also occurs as a map key in the agent's local-API response, and the check is a repo-wide name search. The gate documents this ("name-reachability is not use"), so it is a known limit rather than a regression — but it means **declaring this wire bought documentation, not enforcement**, and a report that claimed coverage would have been wrong. The mutation was asserted to have applied before the run | **READY (M) — NEW 2026-08-12, RANK 3** | R-311 | Make the check resolve the RECEIVER'S mirror type and compare field-by-field, or state per-root which kind of check it got. **A gate whose positive control fails is an instrument nobody has calibrated** | CC |
+| **R-316** | **The removal now genuinely reverses the installation — R-305's once-per-machine defect closed.** v1.27.0 stopped and disabled the unit but left the PACKAGE, and the preflight's ownership question is `dpkg-query … "install ok installed"` and **nothing else — package presence alone** (`felhom-host-install.sh:1718`), so cycle 2 recorded `yes`, the second uninstall treated our own package as the household's, and cycle 3 refused. **Measured on `drill-r50` from `virgin` on the PUBLISHED v1.27.0 before anything was changed:** cycle 1 `no` → `:53` FREE; cycle 2 **`yes`** → left running on `0.0.0.0:53`; cycle 3 **PRE-FLIGHT FAIL exit 1**. **v1.28.0:** when the record says we installed it, the uninstall removes the package too. Order unchanged and load-bearing — read the record, act, THEN delete the state file (`:1268`) that holds it. **TWO packages are now recorded** (`dnsmasq` ships the unit, **`dnsmasq-base` ships `/usr/sbin/dnsmasq`**) and each is taken back only if we added it. **The dependency check is a SIMULATION** (`apt-get -s purge`, proceed only if the removal set is a subset of ours) rather than a guess; it falls back to stop+disable and NAMES the blocking package. Never interactive, never fatal — a wedged apt is recorded and restated in the closing NOTE. The success is re-queried rather than read off apt's exit code. **Watched:** three fixed cycles → install 3 PASSES; household resolver → untouched; dependent package → not purged and named; no record → untouched with the command named. **Red-proofs, mutations asserted applied:** remove the purge → cycle 3 refuses in those exact words; remove the ownership check → a household resolver is PURGED; infer ownership when absent → the guess is taken. **Published as `installer-v1.28.0`** | **CLOSED — shipped + published, observed on the cycle that actually fails** | R-300, R-305 | — | CC |
+| **R-317** | **The agent decides whether to install dnsmasq by stat-ing a file the OTHER package owns.** `EnsureDnsmasq` (`felhom-agent/internal/lanresolver/lanresolver.go:105`) does `os.Stat("/usr/sbin/dnsmasq")` and skips the apt install when it exists — but that path is shipped by **`dnsmasq-base`**, while the systemd unit comes from **`dnsmasq`** (confirmed on the box: `dpkg -S /usr/sbin/dnsmasq` → `dnsmasq-base`; `dpkg -S /usr/lib/systemd/system/dnsmasq.service` → `dnsmasq`). So on any host carrying `dnsmasq-base` without `dnsmasq`, the agent skips the install and then runs `systemctl enable --now dnsmasq` against a unit that is not there: the resolver never comes up and the failure is a retried WARN in the journal rather than anything a customer or the install sees. **Pre-existing, NOT introduced by R-316** — but R-316 makes the shape reachable, because a host whose `dnsmasq-base` pre-dated Felhom now keeps it while `dnsmasq` is removed. R-316's uninstall says so explicitly instead of leaving it to be found from a silent resolver. **Ranked 2 (costs time), not 1:** the box installs fine, only LAN name resolution is missing | **READY (S) — NEW 2026-08-13** | R-316 | Probe what is actually needed — the unit or the `dnsmasq` package — rather than a path a sibling package owns. One-line change in the agent; deliberately NOT made here to keep this session to one repo | CC |
+| **R-318** | **No honest marker exists that says Felhom installed dnsmasq on a machine already in the field, and none can be invented.** Established from source and on the box: the Felhom `/etc/dnsmasq.d/felhom-*.conf` snippets are deleted by the uninstall's own loop (`:1162`) BEFORE the ownership decision and do not survive it; the install state file that carries the record is deleted at `:1268`; nothing under `/etc/felhom*` remains. `/var/log/dpkg.log` does record the install — and is a **timestamp**, which the standing rule refuses as a heuristic dressed as a fact. **So for every box installed before v1.27.0 the answer is the preflight message, and that is a mechanism rather than a rule:** it names the finding, keeps its two routes and its promise not to touch DNS on a host we do not own, and adds *"THIS LOOKS LIKE OURS … systemctl disable --now dnsmasq"*. **Judged as a customer would:** it is honest, it hedges correctly (*looks like*), and it names one exact command — the one thing that gets that person moving. Its weakness is that it asks them to answer *"did this host have dnsmasq before Felhom?"*, which is precisely the question we can no longer answer for them | **CLOSED — established, no action possible for existing boxes** | R-300, R-316 | — | CC |

documentation/tests/golden-0.214.0-2026-08-12/RESULT.md

@@ -0,0 +1,45 @@
+# Golden bake 0.214.0 — 2026-08-12
+
+Per `RUNBOOK-manual-build.md` §4.0/§4.1. Drill VM reverted to `virgin` before and after; guest 9100
+destroyed `--purge`; `/root` residue shredded.
+
+| | |
+|---|---|
+| `GOLDEN_VERSION` | **0.214.0** |
+| `GOLDEN_SHA256` | **3a40379cb00d98c6f9550815b2b024783f153e9eef9be2b3252253d24d8d2c4d** |
+| controller baked | `gitea.dooplex.hu/admin/felhom-controller:0.214.0` |
+| `MinAgent` (CHANGELOG header) | **0.129.0** |
+| published | 656 392 524 B, `upload OK (HTTP 201)` |
+
+## Acceptance markers (`grep -F`, quoted loop variable)
+
+`docker OK (overlay2` 1 · `including mount point rootfs` 1 · `including mount point mp0` 1 ·
+`upload OK (HTTP 201)` 1 · `FATAL` 0 · `excluding` 0
+
+## Fetchability — the SERVED bytes
+
+```
+downloaded: 656392524 bytes
+sha256:     3a40379cb00d98c6f9550815b2b024783f153e9eef9be2b3252253d24d8d2c4d
+```
+Identical to `GOLDEN_SHA256`.
+
+## The template index was STALE, and it would have baked the wrong base
+
+`pveam available` on the freshly reverted `virgin` listed only `debian-13-standard_13.1-2` — the point
+release our own memory records as 404-ing since 2026-07-15. **`pveam update` first**, and the real
+current one is `debian-13-standard_13.6-1_amd64.tar.zst`. The runbook says the point release rots; what
+it does not say is that the VM's cached index rots too, and reading it without refreshing produces a
+confident answer that is a fortnight out of date.
+
+## Secret handling
+
+Token file→file, read inside the VM by a runner script, never on a command line.
+`systemctl show golden-bake -p Environment -p ExecStart | grep -c -F <token>` = **0**. Leak grep on the
+committed log = **0**, believable because a planted-token control on a copy grepped **1**. All in-VM
+artefacts `shred -u`'d.
+
+## NOT DONE
+
+The Day-0 vouch — the operator's, and deliberately so. Three fields, each already verified downloadable
+and selectable: **golden 0.214.0**, **agent 0.129.0**, **min agent 0.129.0**.

documentation/tests/golden-0.214.0-2026-08-12/bake.log

@@ -0,0 +1,326 @@
+[golden] build-golden.sh v3.0.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.214.0
+[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) …
+  Logical volume "vm-9100-disk-0" created.
+  Logical volume pve/vm-9100-disk-0 changed.
+Creating filesystem with 8388608 4k blocks and 2097152 inodes
+Filesystem UUID: a630e58b-5ddd-4312-8535-b83ed18030fb
+Superblock backups stored on blocks: 
+	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
+	4096000, 7962624
+  Logical volume "vm-9100-disk-1" created.
+  Logical volume pve/vm-9100-disk-1 changed.
+Creating filesystem with 6291456 4k blocks and 1572864 inodes
+Filesystem UUID: 247da550-80fb-4921-bb4c-0a3517bdbd21
+Superblock backups stored on blocks: 
+	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
+extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst'
+Total bytes read: 553512960 (528MiB, 177MiB/s)
+Detected container architecture: amd64
+Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ...
+done: SHA256:N/+3Phg1PMspcZ0VWshdC1mXbkAYQ2jAsYhavLNrb0Q root@felhom-golden
+Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ...
+done: SHA256:fl1uCYoByd5OGUy7NjwyEHlkIcOvAxk5887imnsM7tc root@felhom-golden
+Creating SSH host key 'ssh_host_rsa_key' - this may take some time ...
+done: SHA256:6wNca1Ee5AXkXkQjmROa6sdynmpHikRhw7MuHpxXrk0 root@felhom-golden
+[golden] starting + installing Docker (official repo, trixie channel) …
+apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
+perl: warning: Setting locale failed.
+perl: warning: Please check that your locale settings:
+	LANGUAGE = (unset),
+	LC_ALL = (unset),
+	LC_CTYPE = (unset),
+	LC_NUMERIC = (unset),
+	LC_COLLATE = (unset),
+	LC_TIME = (unset),
+	LC_MESSAGES = (unset),
+	LC_MONETARY = (unset),
+	LC_ADDRESS = (unset),
+	LC_IDENTIFICATION = (unset),
+	LC_MEASUREMENT = (unset),
+	LC_PAPER = (unset),
+	LC_TELEPHONE = (unset),
+	LC_NAME = (unset),
+	LANG = "en_US.UTF-8"
+    are supported and installed on your system.
+perl: warning: Falling back to the standard locale ("C").
+locale: Cannot set LC_CTYPE to default locale: No such file or directory
+locale: Cannot set LC_MESSAGES to default locale: No such file or directory
+locale: Cannot set LC_ALL to default locale: No such file or directory
+apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
+perl: warning: Setting locale failed.
+perl: warning: Please check that your locale settings:
+	LANGUAGE = (unset),
+	LC_ALL = (unset),
+	LC_CTYPE = (unset),
+	LC_NUMERIC = (unset),
+	LC_COLLATE = (unset),
+	LC_TIME = (unset),
+	LC_MESSAGES = (unset),
+	LC_MONETARY = (unset),
+	LC_ADDRESS = (unset),
+	LC_IDENTIFICATION = (unset),
+	LC_MEASUREMENT = (unset),
+	LC_PAPER = (unset),
+	LC_TELEPHONE = (unset),
+	LC_NAME = (unset),
+	LANG = "en_US.UTF-8"
+    are supported and installed on your system.
+perl: warning: Falling back to the standard locale ("C").
+locale: Cannot set LC_CTYPE to default locale: No such file or directory
+locale: Cannot set LC_MESSAGES to default locale: No such file or directory
+locale: Cannot set LC_ALL to default locale: No such file or directory
+[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation …
+[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds …
+[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) …
+Unable to find image 'hello-world:latest' locally
+latest: Pulling from library/hello-world
+4f55086f7dd0: Pulling fs layer
+4f55086f7dd0: Download complete
+4f55086f7dd0: Pull complete
+Digest: sha256:7f4da0fc94bcece205a8c0b6f4d11c8196924654ffe5c4d1aa439b7f632048b2
+Status: Downloaded newer image for hello-world:latest
+  docker OK (overlay2; data-root /var/lib/docker)
+  /var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4
+  /mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4
+  both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576
+[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.214.0 (no registry cred at deploy) …
+
+WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
+Configure a credential helper to remove this warning. See
+https://docs.docker.com/go/credential-store/
+
+0.214.0: Pulling from admin/felhom-controller
+039e6f9f9752: Pulling fs layer
+0094c3ac0914: Pulling fs layer
+deca1dac7403: Pulling fs layer
+11c19a33d1b8: Pulling fs layer
+742f054a0a3b: Pulling fs layer
+63bdd1abbb22: Pulling fs layer
+11c19a33d1b8: Waiting
+742f054a0a3b: Waiting
+63bdd1abbb22: Waiting
+039e6f9f9752: Verifying Checksum
+039e6f9f9752: Download complete
+deca1dac7403: Verifying Checksum
+deca1dac7403: Download complete
+11c19a33d1b8: Verifying Checksum
+11c19a33d1b8: Download complete
+742f054a0a3b: Verifying Checksum
+742f054a0a3b: Download complete
+63bdd1abbb22: Verifying Checksum
+63bdd1abbb22: Download complete
+0094c3ac0914: Verifying Checksum
+0094c3ac0914: Download complete
+039e6f9f9752: Pull complete
+0094c3ac0914: Pull complete
+deca1dac7403: Pull complete
+11c19a33d1b8: Pull complete
+742f054a0a3b: Pull complete
+63bdd1abbb22: Pull complete
+Digest: sha256:d2c0bc7fba3d142e7f9ee9117231fd7ea28630aa71ce2f4a7e2d0ee1fdb85d97
+Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.214.0
+gitea.dooplex.hu/admin/felhom-controller:0.214.0
+[golden] asking the controller which infra images it manages …
+[golden] baking infra images (4): traefik:v3.6.7 cloudflare/cloudflared:2026.6.0 gtstef/filebrowser:1.3.3-stable gitea.dooplex.hu/admin/felhom-samba:1.1.0 …
+v3.6.7: Pulling from library/traefik
+589002ba0eae: Pulling fs layer
+ef63511ea6cc: Pulling fs layer
+0738e5cb835e: Pulling fs layer
+3e6813f70c64: Pulling fs layer
+3e6813f70c64: Waiting
+ef63511ea6cc: Verifying Checksum
+ef63511ea6cc: Download complete
+589002ba0eae: Verifying Checksum
+589002ba0eae: Download complete
+3e6813f70c64: Verifying Checksum
+3e6813f70c64: Download complete
+0738e5cb835e: Verifying Checksum
+0738e5cb835e: Download complete
+589002ba0eae: Pull complete
+ef63511ea6cc: Pull complete
+0738e5cb835e: Pull complete
+3e6813f70c64: Pull complete
+Digest: sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a
+Status: Downloaded newer image for traefik:v3.6.7
+docker.io/library/traefik:v3.6.7
+2026.6.0: Pulling from cloudflare/cloudflared
+47de5dd0b812: Pulling fs layer
+c172f21841df: Pulling fs layer
+99515e7b4d35: Pulling fs layer
+99ba982a9142: Pulling fs layer
+d6b1b89eccac: Pulling fs layer
+2780920e5dbf: Pulling fs layer
+7c12895b777b: Pulling fs layer
+3214acf345c0: Pulling fs layer
+52630fc75a18: Pulling fs layer
+dd64bf2dd177: Pulling fs layer
+b839dfae01f6: Pulling fs layer
+ebddc55facdc: Pulling fs layer
+bdfd7f7e5bf6: Pulling fs layer
+2d4d7adf6272: Pulling fs layer
+40008157d8d2: Pulling fs layer
+bd8962e29291: Pulling fs layer
+cac2ae0193cb: Pulling fs layer
+74d1dac84ecc: Pulling fs layer
+99ba982a9142: Waiting
+d6b1b89eccac: Waiting
+2780920e5dbf: Waiting
+7c12895b777b: Waiting
+3214acf345c0: Waiting
+52630fc75a18: Waiting
+dd64bf2dd177: Waiting
+b839dfae01f6: Waiting
+ebddc55facdc: Waiting
+bdfd7f7e5bf6: Waiting
+2d4d7adf6272: Waiting
+40008157d8d2: Waiting
+bd8962e29291: Waiting
+cac2ae0193cb: Waiting
+74d1dac84ecc: Waiting
+99515e7b4d35: Verifying Checksum
+99515e7b4d35: Download complete
+c172f21841df: Verifying Checksum
+c172f21841df: Download complete
+47de5dd0b812: Verifying Checksum
+47de5dd0b812: Download complete
+99ba982a9142: Download complete
+d6b1b89eccac: Verifying Checksum
+d6b1b89eccac: Download complete
+2780920e5dbf: Verifying Checksum
+2780920e5dbf: Download complete
+7c12895b777b: Verifying Checksum
+7c12895b777b: Download complete
+52630fc75a18: Verifying Checksum
+52630fc75a18: Download complete
+3214acf345c0: Verifying Checksum
+3214acf345c0: Download complete
+47de5dd0b812: Pull complete
+dd64bf2dd177: Verifying Checksum
+dd64bf2dd177: Download complete
+b839dfae01f6: Verifying Checksum
+b839dfae01f6: Download complete
+ebddc55facdc: Verifying Checksum
+ebddc55facdc: Download complete
+bdfd7f7e5bf6: Download complete
+40008157d8d2: Verifying Checksum
+40008157d8d2: Download complete
+bd8962e29291: Verifying Checksum
+bd8962e29291: Download complete
+2d4d7adf6272: Verifying Checksum
+2d4d7adf6272: Download complete
+c172f21841df: Pull complete
+cac2ae0193cb: Verifying Checksum
+cac2ae0193cb: Download complete
+74d1dac84ecc: Verifying Checksum
+74d1dac84ecc: Download complete
+99515e7b4d35: Pull complete
+99ba982a9142: Pull complete
+d6b1b89eccac: Pull complete
+2780920e5dbf: Pull complete
+7c12895b777b: Pull complete
+3214acf345c0: Pull complete
+52630fc75a18: Pull complete
+dd64bf2dd177: Pull complete
+b839dfae01f6: Pull complete
+ebddc55facdc: Pull complete
+bdfd7f7e5bf6: Pull complete
+2d4d7adf6272: Pull complete
+40008157d8d2: Pull complete
+bd8962e29291: Pull complete
+cac2ae0193cb: Pull complete
+74d1dac84ecc: Pull complete
+Digest: sha256:ba461b8aa9c042156dbd39c38657fe7431bafa063220eab8d5330a523863da9f
+Status: Downloaded newer image for cloudflare/cloudflared:2026.6.0
+docker.io/cloudflare/cloudflared:2026.6.0
+1.3.3-stable: Pulling from gtstef/filebrowser
+6a0ac1617861: Pulling fs layer
+ef8806083e82: Pulling fs layer
+b74107c861c7: Pulling fs layer
+adc935def003: Pulling fs layer
+4f4fb700ef54: Pulling fs layer
+18695ccc900a: Pulling fs layer
+45d119d5c397: Pulling fs layer
+dac52db4fc51: Pulling fs layer
+6d598f86b2f2: Pulling fs layer
+8aa349c8396c: Pulling fs layer
+dac52db4fc51: Waiting
+6d598f86b2f2: Waiting
+8aa349c8396c: Waiting
+4f4fb700ef54: Waiting
+adc935def003: Waiting
+18695ccc900a: Waiting
+45d119d5c397: Waiting
+6a0ac1617861: Verifying Checksum
+6a0ac1617861: Download complete
+b74107c861c7: Verifying Checksum
+b74107c861c7: Download complete
+6a0ac1617861: Pull complete
+adc935def003: Verifying Checksum
+adc935def003: Download complete
+4f4fb700ef54: Verifying Checksum
+4f4fb700ef54: Download complete
+45d119d5c397: Verifying Checksum
+45d119d5c397: Download complete
+dac52db4fc51: Verifying Checksum
+dac52db4fc51: Download complete
+18695ccc900a: Verifying Checksum
+18695ccc900a: Download complete
+6d598f86b2f2: Verifying Checksum
+6d598f86b2f2: Download complete
+8aa349c8396c: Verifying Checksum
+8aa349c8396c: Download complete
+ef8806083e82: Verifying Checksum
+ef8806083e82: Download complete
+ef8806083e82: Pull complete
+b74107c861c7: Pull complete
+adc935def003: Pull complete
+4f4fb700ef54: Pull complete
+18695ccc900a: Pull complete
+45d119d5c397: Pull complete
+dac52db4fc51: Pull complete
+6d598f86b2f2: Pull complete
+8aa349c8396c: Pull complete
+Digest: sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70babab78c
+Status: Downloaded newer image for gtstef/filebrowser:1.3.3-stable
+docker.io/gtstef/filebrowser:1.3.3-stable
+1.1.0: Pulling from admin/felhom-samba
+897d797d2723: Pulling fs layer
+3051591aa250: Pulling fs layer
+ce57a3f93416: Pulling fs layer
+fb94eeec2fe1: Pulling fs layer
+fb94eeec2fe1: Waiting
+ce57a3f93416: Verifying Checksum
+ce57a3f93416: Download complete
+fb94eeec2fe1: Verifying Checksum
+fb94eeec2fe1: Download complete
+897d797d2723: Verifying Checksum
+897d797d2723: Download complete
+3051591aa250: Verifying Checksum
+3051591aa250: Download complete
+897d797d2723: Pull complete
+3051591aa250: Pull complete
+ce57a3f93416: Pull complete
+fb94eeec2fe1: Pull complete
+Digest: sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10
+Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.1.0
+gitea.dooplex.hu/admin/felhom-samba:1.1.0
+[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) …
+Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'.
+[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) …
+Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'.
+[golden] baking the first-boot SSH host-key regeneration unit (F3) …
+Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'.
+[golden] identity-clean + minimize …
+[golden] stop + archive …
+INFO: including mount point rootfs ('/') in backup
+INFO: including mount point mp0 ('/var/lib/felhom') in backup
+INFO: archive file size: 625MB
+INFO: Finished Backup of VM 9100 (00:00:39)
+[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_08_12-18_52_07.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive)
+[golden] publishing golden (656392524 bytes, sha256 3a40379cb00d98c6…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.214.0/golden.tar.zst
+[golden]   pre-delete existing: HTTP 404 (404/204 expected)
+[golden]   upload OK (HTTP 201)
+GOLDEN_VERSION=0.214.0
+GOLDEN_SHA256=3a40379cb00d98c6f9550815b2b024783f153e9eef9be2b3252253d24d8d2c4d
+[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.214.0 / 3a40379cb00d98c6f9550815b2b024783f153e9eef9be2b3252253d24d8d2c4d
+[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)

hub/CHANGELOG.md

@@ -1,3 +1,35 @@
+## v0.103.0 — a host can read the packages we kept for it (2026-08-12, R-311)
+
+**`ListSupersededEscrow` had zero production callers for nineteen days.** It is the only reader of a
+retained `identity_blob`, and nothing called it — so the retention shipped in v0.93.0 was material the
+product could not reach. Proven on the fixture 2026-08-12: a recovery code that opens a retained
+package (unsealed by hand, and it restored planted files byte-identical from a store the box itself
+could no longer open) was answered by the product as a code that opened nothing.
+
+**New: `GET /api/v1/hosts/<id>/escrow/retained`** — the retained siblings of the existing
+`/escrow` row, self-scoped to the asking host exactly as that one is, behind the same §8.3
+recovery-mode gate, and audited with the same `escrow_blob_served` event BEFORE the bytes leave.
+
+**What it will not serve.** Rows whose `identity_blob` is NULL — every pre-v0.93.0 row — are withheld
+and returned as `unopenable_count` instead. They retain the PBS datastore key and not the repository
+password, so they can never open what the caller is asking about; serving them would have the agent
+try packages that cannot succeed and would let the recovery screen claim an earlier package is
+openable on exactly the boxes the original defect hurt. The count is returned because their EXISTENCE
+is load-bearing and underivable: on such a box a correct old code opens nothing, and the honest reason
+is a defect of ours rather than the customer's typing.
+
+**The trade, stated rather than waved through.** The hub still cannot read any of it — sealed bytes in,
+sealed bytes out, no decrypt path, no recovery code ever held. What widens is VOLUME: a host key that
+could fetch one opaque package can now fetch N. Bounded by the same self-scope, the same recovery-mode
+gate, and an explicit `retainedEscrowServeCap` (16) so a long supersession history cannot become an
+unbounded read.
+
+Pinned by six tests driven through the real endpoint, with four red-proofs each asserted to have
+applied: serving the CURRENT row instead (count 2→1), dropping the unopenable guard (count 1→2,
+unopenable 1→0), dropping self-scope (403→200), and collapsing the route suffix (count 1→0).
+
+---
+
 ## v0.102.0 — the hub refuses to vouch a version that cannot be installed (2026-08-09, R-273)
 
 **The guard owed since 2026-08-09 morning.** Agent v0.128.0 had been published as a package and never

hub/internal/api/escrow_retained_get_test.go

@@ -0,0 +1,218 @@
+package api
+
+import (
+	"encoding/base64"
+	"encoding/json"
+	"net/http"
+	"testing"
+
+	"gitea.dooplex.hu/admin/felhom-hub/internal/store"
+)
+
+// R-311 (v0.103.0) — GET /hosts/<id>/escrow/retained.
+//
+// WHY THESE TESTS EXIST. `ListSupersededEscrow` had zero production callers for nineteen days; the
+// retention it reads was proven on 2026-08-12 to hold the right material and to open a store the box
+// itself could not, while the customer was told their (correct) code opened nothing. These tests pin
+// the endpoint that ends that, and — more importantly — they pin the two things it must NOT do:
+// serve a package that cannot be opened, and let one host read another's.
+
+type retainedResp struct {
+	HostID          string `json:"host_id"`
+	Count           int    `json:"count"`
+	UnopenableCount int    `json:"unopenable_count"`
+	TruncatedCount  int    `json:"truncated_count"`
+	Packages        []struct {
+		Index             int    `json:"index"`
+		SupersededAt      string `json:"superseded_at"`
+		KeyFingerprint    string `json:"key_fingerprint"`
+		IdentityEscrowB64 string `json:"identity_escrow_b64"`
+	} `json:"packages"`
+}
+
+func getRetained(t *testing.T, h *Handler, hostID, key string) (int, retainedResp) {
+	t.Helper()
+	rr := do(h, http.MethodGet, "/hosts/"+hostID+"/escrow/retained", key, "")
+	var out retainedResp
+	if rr.Code == http.StatusOK {
+		if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil {
+			t.Fatalf("decode retained response: %v (body %s)", err, rr.Body.String())
+		}
+	}
+	return rr.Code, out
+}
+
+// seedSuperseded drives the REAL supersession path (a PUT sealing a different password demotes the
+// current row) rather than inserting into host_escrow_superseded directly — so what the test proves
+// includes the demote wiring, not just this handler's SELECT.
+func seedSuperseded(t *testing.T, h *Handler, hostID, key string, gens []struct{ sha, identity string }) {
+	t.Helper()
+	for i, g := range gens {
+		if rr := do(h, http.MethodPut, "/hosts/"+hostID+"/escrow", key, escrowBodyWithHash([]byte("k"), g.sha, g.identity)); rr.Code != http.StatusOK {
+			t.Fatalf("seed PUT %d = %d (%s)", i, rr.Code, rr.Body.String())
+		}
+	}
+}
+
+// THE ONE THAT MATTERS: a retained package that carries key material is served, and the caller can
+// tell it apart from the current row.
+//
+// RED-PROOF: change the handler's SELECT source from ListSupersededEscrow to GetHostDRBundle (i.e.
+// serve the CURRENT row) → the served blob becomes "age-gen3" → this FAILS on the body comparison,
+// which is exactly the confusion the endpoint exists to end.
+func TestRetainedEscrowGet_ServesRetainedPackages(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
+	seedSuperseded(t, h, "h1", "HKEY", []struct{ sha, identity string }{
+		{"SHA_GEN1", "age-gen1"},
+		{"SHA_GEN2", "age-gen2"},
+		{"SHA_GEN3", "age-gen3"}, // current after this
+	})
+
+	code, out := getRetained(t, h, "h1", "HKEY")
+	if code != http.StatusOK {
+		t.Fatalf("GET retained = %d, want 200", code)
+	}
+	if out.Count != 2 || len(out.Packages) != 2 {
+		t.Fatalf("want 2 retained packages, got count=%d len=%d", out.Count, len(out.Packages))
+	}
+	if out.UnopenableCount != 0 {
+		t.Errorf("unopenable_count = %d, want 0 (every seeded row carried an identity blob)", out.UnopenableCount)
+	}
+	// The retained packages must be the SUPERSEDED generations, never the current one. If this ever
+	// serves gen3 the endpoint is answering the wrong question and the customer is misled again.
+	got := map[string]bool{}
+	for _, p := range out.Packages {
+		b, err := base64.StdEncoding.DecodeString(p.IdentityEscrowB64)
+		if err != nil {
+			t.Fatalf("package %d: identity blob is not base64: %v", p.Index, err)
+		}
+		got[string(b)] = true
+		if p.SupersededAt == "" {
+			t.Errorf("package %d has no superseded_at — the screen needs a date to name the package", p.Index)
+		}
+	}
+	if !got["age-gen1"] || !got["age-gen2"] {
+		t.Errorf("served packages = %v, want the two SUPERSEDED generations (age-gen1, age-gen2)", got)
+	}
+	if got["age-gen3"] {
+		t.Error("the CURRENT package was served as retained — that is the wrong-package confusion this endpoint exists to end")
+	}
+}
+
+// A retained row with NO identity blob (every pre-v0.93.0 row) must be WITHHELD and COUNTED. Serving
+// it would have the agent try a package that cannot open anything, and would let the screen claim an
+// earlier package is recoverable on exactly the boxes hurt by the original defect.
+//
+// RED-PROOF: delete the `if len(e.IdentityBlob) == 0 { unopenable++; continue }` guard → the empty
+// row is served as a package → Count becomes 2 and UnopenableCount 0 → this FAILS twice.
+func TestRetainedEscrowGet_WithholdsAndCountsUnopenableRows(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
+	// gen1 carries NO identity blob — the pre-v0.93.0 shape.
+	seedSuperseded(t, h, "h1", "HKEY", []struct{ sha, identity string }{
+		{"SHA_GEN1", ""},
+		{"SHA_GEN2", "age-gen2"},
+		{"SHA_GEN3", "age-gen3"},
+	})
+
+	code, out := getRetained(t, h, "h1", "HKEY")
+	if code != http.StatusOK {
+		t.Fatalf("GET retained = %d, want 200", code)
+	}
+	if out.Count != 1 {
+		t.Errorf("count = %d, want 1 (only gen2 carries material)", out.Count)
+	}
+	if out.UnopenableCount != 1 {
+		t.Errorf("unopenable_count = %d, want 1 — the caller cannot derive this and needs it to explain "+
+			"why a correct old code opens nothing on such a box", out.UnopenableCount)
+	}
+	for _, p := range out.Packages {
+		if p.IdentityEscrowB64 == "" {
+			t.Error("a package with an empty identity blob was served — it can never open anything")
+		}
+	}
+}
+
+// SELF-SCOPE. The current-row GET has this and a retained read is strictly more material, so the same
+// asymmetry must hold: a host key reads only its own.
+//
+// RED-PROOF: remove the `!isGlobal && authHostID != pathHostID` branch → h2's key reads h1's packages
+// → this FAILS with 200.
+func TestRetainedEscrowGet_SelfScoped(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY1"})
+	st.UpsertHost(&store.Host{HostID: "h2", CustomerID: "c2", APIKey: "HKEY2"})
+	seedSuperseded(t, h, "h1", "HKEY1", []struct{ sha, identity string }{
+		{"SHA_GEN1", "age-gen1"},
+		{"SHA_GEN2", "age-gen2"},
+	})
+
+	rr := do(h, http.MethodGet, "/hosts/h1/escrow/retained", "HKEY2", "")
+	if rr.Code != http.StatusForbidden {
+		t.Fatalf("h2 reading h1's retained packages = %d, want 403 — a host key must never be a fleet-wide reader", rr.Code)
+	}
+}
+
+// A host that has never re-sealed gets a clean, empty answer — not a 404 and not a fault. "No retained
+// package" is an ordinary situation and must be distinguishable from "something is broken", or the
+// agent's try-loop cannot tell them apart either.
+//
+// RED-PROOF: make the handler 404 on an empty list → this FAILS with 404.
+func TestRetainedEscrowGet_NoneIsACleanEmptyAnswer(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
+	if rr := do(h, http.MethodPut, "/hosts/h1/escrow", "HKEY", escrowBodyWithHash([]byte("k"), "SHA1", "age-1")); rr.Code != http.StatusOK {
+		t.Fatalf("seed PUT = %d", rr.Code)
+	}
+
+	code, out := getRetained(t, h, "h1", "HKEY")
+	if code != http.StatusOK {
+		t.Fatalf("GET retained on a never-superseded host = %d, want 200", code)
+	}
+	if out.Count != 0 || len(out.Packages) != 0 || out.UnopenableCount != 0 {
+		t.Errorf("want an empty answer, got count=%d packages=%d unopenable=%d", out.Count, len(out.Packages), out.UnopenableCount)
+	}
+}
+
+// The retained route must not be reachable without a key at all.
+//
+// RED-PROOF: drop the checkAuthHost block → this FAILS with 200.
+func TestRetainedEscrowGet_RequiresAuth(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
+	if rr := do(h, http.MethodGet, "/hosts/h1/escrow/retained", "", ""); rr.Code != http.StatusUnauthorized {
+		t.Fatalf("unauthenticated retained GET = %d, want 401", rr.Code)
+	}
+}
+
+// ROUTING: /escrow and /escrow/retained must stay two different answers. A prefix "tidy-up" that
+// collapsed them would route retained reads to the current row — silently the wrong package.
+//
+// RED-PROOF: change the retained case's suffix to "/escrow" (so the earlier case wins) → the retained
+// route returns the CURRENT-row shape, which has no "packages" key → this FAILS.
+func TestRetainedEscrowGet_IsADistinctRouteFromTheCurrentRow(t *testing.T) {
+	h, st, _ := newTestHandler(t)
+	st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
+	seedSuperseded(t, h, "h1", "HKEY", []struct{ sha, identity string }{
+		{"SHA_GEN1", "age-gen1"},
+		{"SHA_GEN2", "age-gen2"},
+	})
+
+	cur := do(h, http.MethodGet, "/hosts/h1/escrow", "HKEY", "")
+	var curBody map[string]any
+	if err := json.Unmarshal(cur.Body.Bytes(), &curBody); err != nil {
+		t.Fatalf("decode current-row body: %v", err)
+	}
+	if _, hasPackages := curBody["packages"]; hasPackages {
+		t.Error("the CURRENT-row route grew a packages key — the two routes have been merged")
+	}
+	if _, hasPresent := curBody["present"]; !hasPresent {
+		t.Error("the current-row route lost its present flag")
+	}
+
+	_, ret := getRetained(t, h, "h1", "HKEY")
+	if ret.Count != 1 {
+		t.Fatalf("retained route count = %d, want 1 — it is answering the current row's question", ret.Count)
+	}
+}

hub/internal/api/handler.go

@@ -245,6 +245,13 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	case r.Method == http.MethodGet && strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/escrow"):
 		hostID := strings.TrimSuffix(strings.TrimPrefix(path, "/hosts/"), "/escrow")
 		h.handleHostEscrowGet(w, r, hostID)
+	// R-311 (v0.103.0): the RETAINED siblings of the row above. The two cases cannot collide — one
+	// ends `/escrow`, the other `/escrow/retained` — but do NOT "tidy" them into a single prefix
+	// match: a prefix match would route retained reads to the CURRENT row, which is silently the
+	// wrong package and is exactly the confusion this endpoint exists to end.
+	case r.Method == http.MethodGet && strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/escrow/retained"):
+		hostID := strings.TrimSuffix(strings.TrimPrefix(path, "/hosts/"), "/escrow/retained")
+		h.handleHostEscrowRetainedGet(w, r, hostID)
 	// G1 break-glass: day-0 vaults the root@pam console credential (self-scoped host key); the
 	// operator retrieves it via the /admin/ path (global key only).
 	case r.Method == http.MethodPut && strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/recovery-credential"):
@@ -1371,6 +1378,158 @@ func (h *Handler) handleHostEscrowGet(w http.ResponseWriter, r *http.Request, pa
 	})
 }
 
+// handleHostEscrowRetainedGet serves a host its own RETAINED (superseded) sealed identity packages.
+//
+// ── WHY THIS EXISTS (R-311) ─────────────────────────────────────────────────────────────────────
+//
+// `ListSupersededEscrow` has been the only reader of a retained identity blob since v0.93.0 and had
+// ZERO production callers — every call site was a test. The consequence, measured on the fixture
+// 2026-08-12: a recovery code that demonstrably opens a retained package (proven by hand: unsealed,
+// and it restored planted files byte-identical from a store the box could no longer open) was
+// reported to the customer as a code that opened nothing. The screen already hedged that with two
+// possible causes (R-222/R-226) and said it could not tell them apart. **It could not tell them
+// apart because nothing ever looked.** This endpoint is what makes looking possible.
+//
+// ── WHAT THIS DOES NOT CHANGE ───────────────────────────────────────────────────────────────────
+//
+// The hub still cannot read any of it. Sealed bytes in, sealed bytes out; there is no decrypt path
+// here and the hub has never held a recovery code. What DOES widen is volume: a host key that could
+// previously fetch one opaque package can now fetch N. The trade is stated rather than waved through
+// — see the audit — and it is bounded three ways: the same self-scope as the current row, the same
+// recovery-mode gate, and an explicit cap so a host with a long supersession history cannot turn one
+// request into an unbounded read.
+//
+// Rows whose `identity_blob` is NULL are NOT served and are counted separately. They are the
+// pre-v0.93.0 rows; they retain the PBS key and not the repository password, so they can never open
+// anything the caller is asking about. Serving them would make the agent try packages that cannot
+// succeed and would let the screen claim an earlier package is openable when it is not — the same
+// false-explanation trap `SupersededPresent` already avoids at store.go. They are counted because
+// their EXISTENCE is a true and load-bearing fact: on those boxes a correct old code opens nothing,
+// and the honest reason is a defect of ours, not the customer's typing.
+const retainedEscrowServeCap = 16
+
+// RetainedEscrowPackage / RetainedEscrowResponse are the WIRE for GET /hosts/<id>/escrow/retained.
+//
+// They are named types rather than a `map[string]any` on purpose: the wire-contract gate resolves a
+// declared ROOT by TYPE, so an untyped map is a cross-repo contract the gate cannot see. This wire is
+// declared in `scripts/wire_contract_gate.py` ROOTS — hub → agent — and the agent's mirror is
+// `felhom-agent/internal/hub.RetainedEscrowResponse`. Change a tag here and the gate fails there,
+// which is the entire point.
+type RetainedEscrowPackage struct {
+	// Index is a label WITHIN ONE RESPONSE. Not durable, never persisted, never a lookup key.
+	Index int `json:"index"`
+	// SupersededAt is when this package stopped being the current one. It is the only field here a
+	// customer ever sees — it is how they recognise which recovery code they are holding.
+	SupersededAt   string `json:"superseded_at"`
+	KeyFingerprint string `json:"key_fingerprint"`
+	// IdentityEscrowB64 is OPAQUE ciphertext. The hub cannot open it and never could.
+	IdentityEscrowB64 string `json:"identity_escrow_b64"`
+}
+
+type RetainedEscrowResponse struct {
+	HostID string `json:"host_id"`
+	Count  int    `json:"count"`
+	// UnopenableCount is the number of retained rows withheld because they carry no key material
+	// (every pre-v0.93.0 row). Underivable by the caller and load-bearing: on a box with only those,
+	// a perfectly correct old code opens nothing and the reason is a defect of ours.
+	UnopenableCount int                     `json:"unopenable_count"`
+	TruncatedCount  int                     `json:"truncated_count"`
+	Packages        []RetainedEscrowPackage `json:"packages"`
+}
+
+func (h *Handler) handleHostEscrowRetainedGet(w http.ResponseWriter, r *http.Request, pathHostID string) {
+	authHostID, _, isGlobal, ok := h.checkAuthHost(r)
+	if !ok {
+		http.Error(w, "Unauthorized", http.StatusUnauthorized)
+		return
+	}
+	if pathHostID == "" {
+		http.Error(w, "Missing host_id", http.StatusBadRequest)
+		return
+	}
+	// SELF-SCOPED, identically to the current-row GET. Without this line any host key is a fleet-wide
+	// reader of every retained package, which is strictly worse than the same hole on one row.
+	if !isGlobal && authHostID != pathHostID {
+		h.logger.Printf("[WARN] retained escrow GET REFUSED: host %s asked for %s's packages (self-scope)", authHostID, pathHostID)
+		http.Error(w, "Forbidden: host_id mismatch", http.StatusForbidden)
+		return
+	}
+	host, err := h.store.GetHost(pathHostID)
+	if err != nil {
+		http.Error(w, "Internal error", http.StatusInternalServerError)
+		return
+	}
+	if host == nil {
+		http.Error(w, "Unknown host_id", http.StatusNotFound)
+		return
+	}
+	// The SAME §8.3 gate as the current row. A retained package is not less sensitive than the
+	// current one; if self-service retrieval is ever switched off, it must go dark with it.
+	if !escrowSelfServiceRetrieval && !host.InRecoveryMode(time.Now().UTC()) {
+		h.logger.Printf("[WARN] retained escrow GET REFUSED for %s — self-service retrieval is disabled and recovery mode is not armed", pathHostID)
+		http.Error(w, "Forbidden: host not in recovery mode (operator must arm it)", http.StatusForbidden)
+		return
+	}
+
+	retained, rerr := h.store.ListSupersededEscrow(pathHostID)
+	if rerr != nil {
+		h.logger.Printf("[ERROR] retained escrow GET for %s: %v", pathHostID, rerr)
+		http.Error(w, "Internal error", http.StatusInternalServerError)
+		return
+	}
+	packages := make([]RetainedEscrowPackage, 0, len(retained))
+	unopenable := 0
+	for _, e := range retained {
+		if len(e.IdentityBlob) == 0 {
+			unopenable++
+			continue
+		}
+		if len(packages) >= retainedEscrowServeCap {
+			continue
+		}
+		packages = append(packages, RetainedEscrowPackage{
+			Index:             len(packages),
+			SupersededAt:      e.UpdatedAt,
+			KeyFingerprint:    e.KeyFingerprint,
+			IdentityEscrowB64: base64.StdEncoding.EncodeToString(e.IdentityBlob),
+		})
+	}
+	truncated := 0
+	if n := len(retained) - unopenable; n > len(packages) {
+		truncated = n - len(packages)
+	}
+
+	// The audit row is written BEFORE the bytes leave, exactly as the current-row GET does, so a
+	// retrieval cannot be served without its record. Severity matches that path deliberately: this is
+	// a recovery in progress and nothing else, and it must reach the operator by e-mail.
+	if host.CustomerID != "" && len(packages) > 0 {
+		msg := fmt.Sprintf("Retained recovery packages served: host %s retrieved %d retained sealed package(s) (%d unopenable pre-v0.93.0 row(s) withheld). "+
+			"This is the recovery path in use — the packages cannot be opened without the customer's recovery code, which the hub never holds. "+
+			"If no recovery is in progress on that box, investigate.", pathHostID, len(packages), unopenable)
+		details, _ := json.Marshal(map[string]any{
+			"host_id":    pathHostID,
+			"served":     len(packages),
+			"unopenable": unopenable,
+			"truncated":  truncated,
+			"self_scope": !isGlobal,
+		})
+		if _, eerr := h.store.SaveEvent(host.CustomerID, eventEscrowBlobServed, "warning", msg, string(details), "hub"); eerr != nil {
+			h.logger.Printf("[WARN] %s event save FAILED for %s (serving anyway): %v", eventEscrowBlobServed, pathHostID, eerr)
+		} else if h.dispatcher != nil {
+			go h.dispatcher.ProcessEvent(host.CustomerID, eventEscrowBlobServed, "warning", msg, string(details), "hub")
+		}
+	}
+	h.logger.Printf("[WARN] retained escrow SERVED to host %s (%d package(s), %d unopenable withheld, %d over cap, self_scope=%v)",
+		pathHostID, len(packages), unopenable, truncated, !isGlobal)
+	writeJSON(w, http.StatusOK, RetainedEscrowResponse{
+		HostID:          pathHostID,
+		Count:           len(packages),
+		UnopenableCount: unopenable,
+		TruncatedCount:  truncated,
+		Packages:        packages,
+	})
+}
+
 // maybeEmitRepoKeyChanged raises ONE operator signal per supersession when the sealed offsite repo
 // password demonstrably changed. Both hashes have been stored since SLICE 3 (host_escrow and, since
 // v0.60.0, host_escrow_superseded) and NOTHING compared them: demo-felhom's repository password

manifests/hub.yaml

@@ -125,7 +125,7 @@ spec:
     spec:
       containers:
         - name: hub
-          image: gitea.dooplex.hu/admin/felhom-hub:0.102.0
+          image: gitea.dooplex.hu/admin/felhom-hub:0.103.0
           ports:
             - containerPort: 8080
               name: http

scripts/CHANGELOG.md

@@ -1,3 +1,55 @@
+## felhom-host-install.sh v1.28.0 — the removal genuinely reverses the installation (2026-08-13, R-316)
+
+**v1.27.0's fix worked exactly once per machine, and this is the measurement.** Three full cycles on
+`drill-r50` from `virgin`, on the PUBLISHED v1.27.0, before anything was changed:
+
+| cycle | recorded | uninstall did | result |
+|---|---|---|---|
+| 1 | `no` | stop + disable | `:53` FREE |
+| 2 | **`yes`** | **left it running** | `0.0.0.0:53` taken |
+| 3 | — | — | **PRE-FLIGHT FAIL (exit 1)** — *"a resolver is already bound to :53"* |
+
+**Why cycle 2 says `yes`:** the preflight asks `dpkg-query` whether `dnsmasq` is installed and nothing
+else — **package presence alone** (not the absence of a record). v1.27.0 stopped the unit and left the
+package, so the answer stayed yes and our own package became "the household's" one cycle later. **Every
+box already in the field is at cycle 2**, and a reinstall onto a machine that has had Felhom is cycle 2
+by definition.
+
+**Now:** when the install-time record says we installed it, the uninstall **removes the package** as
+well as stopping the unit — which is what makes the next install a first install again. Order is
+unchanged and load-bearing: read the record, act, and only then delete the state file that holds it.
+
+**TWO packages are recorded, not one.** `dnsmasq` ships the systemd unit; **`dnsmasq-base` ships
+`/usr/sbin/dnsmasq`**. They are separately installable, so each is recorded at preflight and taken back
+only if we added it.
+
+**Guard rails, all measured on the box rather than reasoned:**
+
+- **Ownership is read, never inferred** — no mtimes, no package metadata, no file dates.
+- **The dependency check is a SIMULATION.** `apt-get -s purge` is asked what it would remove, and the
+  purge proceeds only if the answer is a subset of what we are entitled to remove. Otherwise: stop +
+  disable, and the log names the package that stopped us.
+- **Never interactive, never fatal.** A wedged apt cannot strand a teardown — the failure is recorded
+  and restated in the closing NOTE, because a half-completed teardown is how cycle 3 refuses.
+- **The success is asserted, not assumed:** after `apt-get` exits 0 the package is re-queried, because
+  an exit code is not an observation.
+
+**Scenarios, each watched:** three fixed cycles → **install 3 PASSES**; a household resolver →
+untouched; a dependent package (`household-dns-thing`) → **not purged, named in the log**, `:53` still
+freed; no ownership record → untouched, reason logged, exact command named.
+
+**Red-proofs, mutation asserted applied each time:** remove the purge → **cycle 3 refuses again in
+those exact words**; remove the ownership check → **a household resolver is purged**; infer ownership
+when there is no record → **the guess is taken and a field box loses its own DNS**.
+
+**Known residue, filed as R-317:** where `dnsmasq-base` pre-dated Felhom we correctly keep it, but it
+owns `/usr/sbin/dnsmasq`, which the agent stats to decide whether to install
+(`felhom-agent internal/lanresolver/lanresolver.go:105`). On that host shape the next install can skip
+the apt step and then fail to enable a unit that is gone. The uninstall says so explicitly rather than
+leaving it to be rediscovered from a silent resolver.
+
+---
+
 ## PUBLISHED — `installer-v1.27.0` (2026-08-12)
 
 **v1.26.0 (R-297) and v1.27.0 (R-300) were both written, pushed and deliberately left unpublished

scripts/felhom-host-install.sh

@@ -184,7 +184,7 @@
 
 set -euo pipefail
 
-SCRIPT_VERSION="1.27.0"   # the SINGLE version source (F-1): -h and the run banners follow it.
+SCRIPT_VERSION="1.28.0"   # the SINGLE version source (F-1): -h and the run banners follow it.
                           # The hub used to carry a copy for its Setup tab; R-94 DELETED it
                           # (2026-08-02) because the hub cannot know which version a box runs —
                           # the Setup command fetches this script at run time. scripts/
@@ -842,6 +842,92 @@ _uninstall_statement() {
 # run_uninstall — the full guarded teardown. Every mutation goes through run() so --dry-run prints it
 # and executes nothing. Ordering is the reverse of install: guest -> agent -> pveum(ACL,token,user,
 # role) -> golden(opt-in) -> state file. See the TASK spec §7/§8.
+# _dnsmasq_purge_owned — remove the dnsmasq packages THIS install added, and nothing else (R-316).
+#
+# Called only from the `no` branch, i.e. only when the preflight RECORDED that we installed it. There
+# is no inference here and there must never be: no mtimes, no package metadata, no file dates. That
+# rule was earned and it is why the record exists at all.
+#
+# THE DEPENDENCY GUARD IS A SIMULATION, NOT A GUESS. `apt-get -s purge` is asked what it would do, and
+# the purge proceeds only if the answer names a subset of the packages we are entitled to remove. If
+# anything else would be dragged out, we STOP AND DISABLE instead and say which package stopped us —
+# a silent downgrade to the weaker action would be a warning standing beside a success.
+#
+# It can fail and the uninstall still succeeds: a wedged apt must not strand a teardown. Failure is
+# recorded in _DNSMASQ_PURGE_NOTE and reported in the closing statement, never swallowed.
+_DNSMASQ_PURGE_NOTE=""
+_dnsmasq_purge_owned() {
+    local want=("dnsmasq")
+    # dnsmasq-base ships /usr/sbin/dnsmasq and is separately installable — take it back only if we
+    # added it too. Leaving a pre-existing base package is correct; see the WARN below for what that
+    # costs on the next install.
+    if [[ "$(_state_get dnsmasq_base_preexisting)" == "no" ]]; then
+        want+=("dnsmasq-base")
+    fi
+
+    if ! command -v apt-get >/dev/null 2>&1; then
+        _DNSMASQ_PURGE_NOTE="apt-get absent — dnsmasq left installed (stopped + disabled)"
+        log_warn "  dnsmasq: apt-get not available — the package stays installed; the unit is stopped + disabled."
+        return 0
+    fi
+    if $DRY_RUN; then
+        log_dry "DEBIAN_FRONTEND=noninteractive apt-get -y purge ${want[*]}  (after simulating the blast radius)"
+        return 0
+    fi
+
+    # What would come out? Every Remv/Purg line is a package this would remove.
+    local sim removed extra=()
+    sim=$(DEBIAN_FRONTEND=noninteractive apt-get -s purge "${want[@]}" 2>/dev/null) || sim=""
+    if [[ -z "$sim" ]]; then
+        _DNSMASQ_PURGE_NOTE="purge simulation failed — dnsmasq left installed (stopped + disabled)"
+        log_warn "  dnsmasq: could not simulate the removal — leaving the package installed (unit stopped + disabled)."
+        log_warn "    A reinstall will read it as pre-existing; clear it with:  apt-get -y purge ${want[*]}"
+        return 0
+    fi
+    removed=$(printf '%s\n' "$sim" | awk '/^(Remv|Purg) /{print $2}')
+    local p
+    for p in $removed; do
+        case " ${want[*]} " in
+            *" $p "*) ;;
+            *) extra+=("$p") ;;
+        esac
+    done
+    if (( ${#extra[@]} > 0 )); then
+        _DNSMASQ_PURGE_NOTE="purge would also remove ${extra[*]} — dnsmasq left installed (stopped + disabled)"
+        log_warn "  dnsmasq: NOT purging — something else now depends on it. Removing it would also take out:"
+        log_warn "      ${extra[*]}"
+        log_warn "    The unit is stopped + disabled, so :53 is free. The package stays, deliberately."
+        return 0
+    fi
+
+    log_info "  dnsmasq: removing the package(s) Felhom installed: ${want[*]}"
+    if run env DEBIAN_FRONTEND=noninteractive apt-get -y purge "${want[@]}"; then
+        # Positive observable: assert it is actually gone rather than trusting apt's exit code.
+        if dpkg-query -W -f='${Status}' dnsmasq 2>/dev/null | grep -q "install ok installed"; then
+            _DNSMASQ_PURGE_NOTE="apt-get reported success but dnsmasq is still installed"
+            log_warn "  dnsmasq: apt-get exited 0 but the package is STILL installed — treat as not removed."
+        else
+            log_success "  dnsmasq removed — the next install starts from a clean slate, as a first install"
+        fi
+    else
+        _DNSMASQ_PURGE_NOTE="apt-get purge failed — dnsmasq left installed (stopped + disabled)"
+        log_warn "  dnsmasq: the package removal FAILED. The unit is stopped + disabled, so :53 is free,"
+        log_warn "    but a reinstall will read the package as pre-existing. Clear it with:"
+        log_warn "      apt-get -y purge ${want[*]}"
+    fi
+    # R-316: the residual case, stated because it is invisible otherwise. If dnsmasq-base pre-dated us
+    # we correctly leave it — but it owns /usr/sbin/dnsmasq, and the agent decides whether to install
+    # by stat-ing exactly that path (felhom-agent internal/lanresolver/lanresolver.go:105). So on such
+    # a host the next install skips the apt step and then fails to enable a unit that is no longer
+    # there. Filed as R-317; named here so nobody has to rediscover it from a silent resolver.
+    if [[ "$(_state_get dnsmasq_base_preexisting)" == "yes" ]]; then
+        log_warn "  NOTE: dnsmasq-base pre-dates Felhom and is deliberately kept. It owns /usr/sbin/dnsmasq,"
+        log_warn "    which the agent stats to decide whether to install — so on this host the next install"
+        log_warn "    may skip the install step and fail to enable the unit (R-317)."
+    fi
+    return 0
+}
+
 run_uninstall() {
     log_step "UNINSTALL — local host teardown"
 
@@ -1098,8 +1184,22 @@ run_uninstall() {
     if $_dnsmasq_touched || systemctl is-enabled --quiet dnsmasq 2>/dev/null; then
         case "$_dnsmasq_owner" in
             no)
+                # R-316 — STOPPING IS NOT REMOVING, AND THE DIFFERENCE COSTS THE *NEXT* CYCLE.
+                #
+                # R-300 stopped + disabled the unit and left the PACKAGE installed. That frees :53, so
+                # cycle 1 looks fixed — and it is. But the next install's preflight asks dpkg "is
+                # dnsmasq installed?", the answer is still yes, so it records `dnsmasq_preexisting=yes`,
+                # the second uninstall then treats OUR OWN package as the household's and leaves it
+                # running, and the third install refuses again with the message that blames the
+                # customer's network. **Measured on drill-r50 2026-08-13, all three cycles, before this
+                # was written.** Every box already in the field is at cycle 2.
+                #
+                # So: stop first (the port must free even if the package operation fails), then remove
+                # what we added. Order matters — the state file that carries this record is deleted
+                # later in this same function, so the record is read before it can be lost.
                 log_info "  dnsmasq was installed by Felhom (recorded at install) — stopping + disabling it"
                 run systemctl disable --now dnsmasq || true
+                _dnsmasq_purge_owned
                 ;;
             yes)
                 log_info "  dnsmasq pre-dates Felhom (recorded at install) — leaving it running, restarting only"
@@ -1175,7 +1275,14 @@ run_uninstall() {
     # R-300: the packages are still not PURGED (they are system packages and purging them on a host we
     # may not own is the wrong blast radius) — but dnsmasq's UNIT is now stopped+disabled when the
     # install-time record says Felhom installed it, so our own leftover no longer blocks our own reinstall.
-    log_info "  NOTE: the 'sudo' and 'dnsmasq' packages were left INSTALLED (system packages, not purged); dnsmasq's unit was stopped+disabled only if Felhom installed it. The host record still exists in the hub — remove it there if desired."
+    # R-316: this line used to say dnsmasq was left installed, and that was the defect in one sentence.
+    # `sudo` genuinely is a system package and stays. dnsmasq is now taken back when the install-time
+    # record says we added it — and when that could not be done, the reason is stated HERE rather than
+    # scrolling past, because a teardown that quietly half-completed is how cycle 3 refuses.
+    if [[ -n "$_DNSMASQ_PURGE_NOTE" ]]; then
+        log_warn "  NOTE: dnsmasq was NOT removed — $_DNSMASQ_PURGE_NOTE. Its unit is stopped + disabled, so :53 is free, but the NEXT install will read the package as pre-existing."
+    fi
+    log_info "  NOTE: the 'sudo' package was left INSTALLED (a system package, never ours to remove). dnsmasq is removed only when the install-time record says Felhom installed it; a resolver that pre-dated Felhom is never touched. The host record still exists in the hub — remove it there if desired."
     $DRY_RUN && log_warn "  DRY-RUN: nothing above was actually executed."
     return 0
 }
@@ -1715,12 +1822,24 @@ step_preflight() {
     # heuristic dressed as a fact, and this project has been bitten by exactly that shape.
     #
     # Recorded on EVERY mode, because appliance is where the agent actually installs it.
+    # R-316: TWO packages are recorded, not one, and the second is not pedantry.
+    #
+    # `dnsmasq` ships the systemd unit; **`dnsmasq-base` ships /usr/sbin/dnsmasq**. They are separately
+    # installable, and a host can carry the base package for some unrelated reason. The removal must
+    # take back only what WE added, so what we found has to be recorded package by package — the same
+    # rule as before, applied one level finer.
     if command -v dpkg-query >/dev/null 2>&1 && dpkg-query -W -f='${Status}' dnsmasq 2>/dev/null | grep -q "install ok installed"; then
         _state_put dnsmasq_preexisting yes
         log_info "  dnsmasq: already installed BEFORE Felhom — recorded; uninstall will not touch it"
     else
         _state_put dnsmasq_preexisting no
-        log_info "  dnsmasq: not present before Felhom — recorded; uninstall may stop+disable it if we install it"
+        log_info "  dnsmasq: not present before Felhom — recorded; uninstall will remove it again if we install it"
+    fi
+    if command -v dpkg-query >/dev/null 2>&1 && dpkg-query -W -f='${Status}' dnsmasq-base 2>/dev/null | grep -q "install ok installed"; then
+        _state_put dnsmasq_base_preexisting yes
+        log_info "  dnsmasq-base: already installed BEFORE Felhom — recorded; uninstall will leave it"
+    else
+        _state_put dnsmasq_base_preexisting no
     fi
 
     # Host DNS :53 gate (byo only) — GL-8/F6. In byo the agent's own lan-resolver stays OFF (a config

scripts/wire_contract_gate.py

@@ -81,6 +81,12 @@ ROOTS = [
      "controller", "internal/report", "Report", "hub"),
     ("hub -> controller  (report ACK, `escrow` object)",
      "hub", "internal/store", "EscrowStatus", "controller"),
+    # R-311. Declared the moment the wire was created, because the gate covers only what is DECLARED
+    # and a silent pass is indistinguishable from coverage. The hub response is a named type rather
+    # than a map[string]any precisely so this root can resolve — an untyped map is a cross-repo
+    # contract nothing can check.
+    ("hub -> agent   (GET /hosts/<id>/escrow/retained)",
+     "hub", "internal/api", "RetainedEscrowResponse", "agent"),
 ]
 
 # Tag names whose literal string carries no information in a repo-wide search. NOT CHECKED.