From 259f48aedc05e38c00a67674357efbce6e692e2f Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Wed, 5 Aug 2026 23:28:54 +0200 Subject: [PATCH] OPEN-ITEMS: CAMPAIGN-11 Phase 2 findings R-224..R-227, and what Phase 2 recorded against existing rows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit R-224 every non-code failure on the unlock path is reported as a statement about the customer's code. F3 (hub REJECTed) and F4 (agent stopped) both returned M4 for a CORRECT current code, in 0.056 s and 0.030 s — neither compatible with a real unseal (~1.0 s of scrypt). The agent's err field distinguishes fetch-failure from unseal-failure and both become one HTTP 400; the controller has no reach-failure branch. The vocabulary exists and is wired to agentClient() construction, not to the RPC. R-216's gate answers source=version and cannot see reachability. R-225 the store reports 0 snapshots / 0 GB when it cannot read it, above a card saying it holds backups. Measured over SFTP: snapshot f3d9cd67 and 12 535 KB are really there. snapshot_count/repo_size_bytes are ABSENT from settings.json after a rebuild — unknown renders as zero. R-226 M1, the only message that tells a customer to check their typing, is unreachable on any box that has re-escrowed (M4's branch precedes it). R-227 a restart mid-unlock returns a raw English "Bad Gateway" (I3). Recorded against existing rows: R-216 §4.1 now MEASURED twice (and a correction — SetFloor's DEBUG line can never reach the debug ring, so a restart alone would not have produced it); R-218's live half still NOT measured, deliberately, because the venue has a target; R-217 and R-215's fixes HELD under their faults. R-199's back-pointer was already present in the capability map — the brief's assumption that it was owed is stale. No product code changed. --- documentation/backlog/OPEN-ITEMS.md | 55 +++++++++++++++++++++++++++-- 1 file changed, 52 insertions(+), 3 deletions(-) diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index f07a096..9291162 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -58,9 +58,17 @@ drill needed — the fourth is R-193, and until it goes the recovery is still op ## CAMPAIGN 11 — the recovery journey, 2026-08-05 **The whole journey was walked end to end for the first time, on a throwaway appliance built from the -published ISO. The data came back byte-identical; the journey did not exist.** Nine findings, R-214 … -R-222. Six are fixed in controller **v0.201.0** + hub **v0.97.1**; **three are deliberately still -open** and each blocks a real flow. Evidence: `tests/campaign11-evidence-2026-08-05/journal.md`. +published ISO. The data came back byte-identical; the journey did not exist.** **Ten findings from +Phases 1 and 3, R-214 … R-223** (seven fixed in controller **v0.201.0** + hub **v0.97.0/0.97.1**; +**three deliberately still open**, each blocking a real flow), **plus four from Phase 2's injected +faults, R-224 … R-227.** Evidence: `tests/campaign11-evidence-2026-08-05/journal.md` (Phases 0/1/3) +and `journal-phase24.md` (Phases 2/4). Campaign document: +`audits/CAMPAIGN-11-recovery-journey-2026-08-05.md`. + +> **Phase 2's verdict in one line.** The cryptography, the retention and the transport all work and +> are now proven live. **What fails is being told the truth:** a mistyped code, a hub outage, a +> stopped agent and a correct code for a retained earlier package all produce **one** message, and +> three of the four are wrong. | ID | What | State | |---|---|---| @@ -75,6 +83,47 @@ open** and each blocks a real flow. Evidence: `tests/campaign11-evidence-2026-08 | **R-221** | **A rebuilt box cannot run the escrow ceremony at all.** The preflight refuses on `escrow.pbs_storage_id`, which the pbsdr bridge seeds into `agent.json` only via `finishConverged`. The convergence marker lives on the HOST and survives a guest rebuild; `agent.json` is rewritten by the installer. Unchanged descriptor → same hash → early return → the seed never runs into a config that no longer has it. **Red-proved**: moving only the marker aside seeded it instantly (`grep -c escrow`: 0 → 1). A real blocker for re-escrow, which is exactly what a rebuilt box must do | **OPEN — NOT FIXED** | | **R-223** | **The Day-0 manifest vouched agent 0.120.0 while the recovery feature needs 0.125.0 — and a reinstall DOWNGRADES a box that was fixed by hand.** Verbatim from the second reinstall: `agent (existing): felhom-agent 0.125.0` → `manifest: agent v0.120.0` → `installed /usr/local/bin/felhom-agent (felhom-agent 0.120.0)`. So every rebuild re-broke the recovery path — the one event that makes the feature necessary. **⚠ AND IT WAS NOT A DROPDOWN.** The first vouch attempt was REFUSED by R-120's gate (`configs.go:1162`): *"golden 0.192.0 is older than the newest controller the fleet reports (0.201.0)"*. **The artifacts form saves as a unit, so the agent could not be vouched while the golden was stale — and the golden had been stale since controller 0.193.0, meaning the Day-0 manifest had been effectively UNVOUCHABLE for days and nobody had cause to notice.** The real remedy was a golden rebake | **CLOSED 2026-08-05.** Golden **0.201.0** baked in the drill VM (658 165 766 B, sha `e730d7cab343eb35…f007654`, **round-trip verified from Gitea**), then manifest set in one save: `agent=0.125.0 golden=0.201.0 min_agent=0.125.0`. A fresh install now lands on current agent AND current controller | +### Phase 2 — the injected faults, 2026-08-05/06 (unattended) + +Eleven faults, each judged on **the message, not the outcome**, each with a positive control proving +the fault was real. Full observables: `tests/campaign11-evidence-2026-08-05/journal-phase24.md`. + +| ID | What | State | +|---|---|---| +| **R-224** | **Every non-code failure on the unlock path is reported to the customer as a statement about their code.** Two faults, one defect. **F3** — the hub REJECTed (control: `302` → `exit 7`) and the **correct current** code returned M4 (*„Ez a kód nem nyitja meg azt a csomagot…"*) in **0.0556 s**. **F4** — `felhom-agent` stopped (control: `:8443` gone) and the same correct code returned M4 in **0.0299 s**. Neither elapsed time is compatible with a real unseal (~1.0 s of scrypt; F1's wrong-code attempts took 1.194/1.004/1.014 s), so **no unseal was attempted in either case** — the failure was upstream and was attributed to the code anyway. **The discriminator exists and is thrown away**: the agent's own `err` reads `fetching the sealed bundle: hub: transport error: … no route to host` (F3) versus `the recovery code did not unwrap the identity escrow` (F1), but both return **HTTP 400** under one merged sentence (*"…or the bundle could not be fetched"*), and the controller's failure path has **no branch for "could not ask / could not reach"**. **⚠ The vocabulary already exists and is wired to the wrong condition** — „A gép házon belüli kapcsolata most nem elérhető" fires only when `s.agentClient()` fails to *construct*, never when the RPC fails. **And R-216's gate cannot catch it**: the box's own ring reads `recovery capability gate: offsite_key_recovery=yes (source=version)` — the gate discriminates the agent's **age**, not its **reachability**, so a dead agent of the right version sails through the guard whose own comment says *"An attempt that cannot succeed must never be made, because its failure is attributed to the code."* **This is Phase 1's headline finding relocated from the version channel to the transport.** Breaches **I6** | **OPEN — NOT FIXED.** Fix shape: classify `rerr` (transport/fetch vs unseal) at the agent's HTTP boundary and give the controller a reach-failure branch. Do NOT widen the capability gate into a liveness probe on the unlock path — a probe that must succeed before an unlock adds a second thing that can fail | +| **R-225** | **The remote store reports `0 pillanatkép · 0 / 50 GB` when the box cannot read it — directly above a card stating the store holds backups.** `/backups/remote` renders both at once. **Ground truth, measured over SFTP with the box's own credential (read-only, no decryption):** `/home/felhom-repo/snapshots` holds **`f3d9cd67…`** — the Phase 0 snapshot with all three sentinels, the customer's only surviving copy — and `du -s` returns **12 535 KB**, matching the journal's `repo_size_bytes 12 611 522`. **Mechanism, from the box's own state:** after the rebuild the `offbox` block carries **no `snapshot_count` and no `repo_size_bytes` key at all** — the values are *unknown* and unknown renders as the zero value. **This is R-217's defect class in a second location**, and `OffsiteInventory.Empty` exists precisely because *"len(Apps)==0 is also what a failed read looks like"*. **I5 checked and NOT breached** — the hub's `0.0 GB` is 12.5 MB rounded, and the pool total is consistent; only the box's snapshot **count**, an integer, is false. Breaches **I6** | **OPEN — NOT FIXED.** A count the box has never successfully taken must render as unknown („—"), never as `0` | +| **R-226** | **M1 — the only message that tells a customer to check their typing — is unreachable on any box that has re-escrowed.** The failure path tests M4's condition *before* M1's (`if present, at := s.recoverySuperseded(); present { …M4…; return }`), so on every box the hub keeps an earlier package for, a **genuinely mistyped code** produces M4. **Measured**: F1's three wrong-code attempts each returned M4 after a real ~1 s unseal. M4 is hedged (*„**Ha** egy korábbi kódot adtál meg"*) and states two true facts, but it offers no hint to re-check the ten words and routes the customer to support about *older* backups. **The population that has re-escrowed is exactly the population that has just been handed a new recovery code and is most likely to be typing one.** R-222's fix removed one conflation and introduced another on the same branch | **OPEN — NOT FIXED.** Fix shape: M4 should name both possibilities (a mistype OR an earlier code), since from the engine's side they genuinely are indistinguishable — which is the honest thing to say | +| **R-227** | **A controller restart mid-unlock returns a raw English `Bad Gateway`.** F8 restarted the container at T+0.7 s, inside the unseal window (control: `StartedAt` moved). The customer got **HTTP 502 / „Bad Gateway"** from traefik — a raw upstream error, in English, naming no reason and saying nothing about whether the key was installed. **The state half is clean**: the four `/data/offbox` files stayed byte-identical with mtimes frozen, and the controller returned healthy in 40 s. Breaches **I3** | **OPEN — NOT FIXED.** Low severity — the window is ~1 s wide. Recorded rather than inflated | + +**Recorded against existing rows by Phase 2:** + +- **R-216 — §4.1 is now MEASURED, not deduced.** The previous session could only offer two absences. + The box's own `/settings` renders „Minimális verzió (üzemeltető) **0.200.0**" (`GetFloor()`, whose + only writer is the report-ACK handler; **both hold branches serve `Floor=""`**, pinned by + `managed_floor_test.go:94`), and a **cold-started** controller logs + `settle-gate: GO — at/above floor 0.200.0 (we are 0.201.0)` against the same line reading + `floor still unknown after 1m30s` while the hold was in force. The hub's HELD lines ran every + 15 min to 22:12:06 and stopped, with a liveness control proving the hub kept logging. **The floor + is served.** +- **⚠ A correction to how that positive was to be taken.** `SetFloor`'s line is `u.dbg(...)`, gated on + `cfg.Logging.Level == "debug"` and written to the **logger** — it can **never** reach the logx debug + ring, so it cannot appear in `/api/debug/logs` at any level. A controller restart alone would not + have produced it. Confirmed with a level census on the ring first (1196 DEBUG / 2802 INFO / 2 WARN), + so the absence was known to be structural rather than evidential. +- **R-218 — the live half is STILL NOT MEASURED, deliberately.** The fix is present and correct + (`needsOffsiteCredential` now retires on the **target**, not the key), but the venue **has** a + target, so the box correctly does not declare; declaring here would be the bug. The state that + exercises it is shape (a), which the venue no longer holds. **Recorded as not measured rather than + inferred from the unit test.** +- **R-217 — its fix HELD under exactly its fault** (F5): with the store blocked after a successful + unlock, the page rendered M3 and **no listing block at all**; the three false-claim strings are + absent, verified in UTF-8 with accented positive controls present. +- **R-215 — its fix is present** and the `GET /recovery` gate consults the same predicate as the POST + sibling. +- **R-199's back-pointer in `architecture/00-capability-map.md` was already added** — the brief lists + it as owed; it is present on the escrow-recovery row, explicitly labelled as the omitted + back-pointer. **No action taken; the brief's assumption was stale.** + **Untouched by this session, stated so nothing is presumed closed by association:** **R-213** (putting files back — the half the recovery screen deliberately does not do) and **R-202** (the orphan card's unconditional promise, which CAMPAIGN-11 §7 step 7 measured the customer-facing cost of).