OPEN-ITEMS: CAMPAIGN-11 Phase 2 findings R-224..R-227, and what Phase 2 recorded against existing rows
gates / gates (push) Successful in 7s

R-224  every non-code failure on the unlock path is reported as a statement
       about the customer's code. F3 (hub REJECTed) and F4 (agent stopped) both
       returned M4 for a CORRECT current code, in 0.056 s and 0.030 s — neither
       compatible with a real unseal (~1.0 s of scrypt). The agent's err field
       distinguishes fetch-failure from unseal-failure and both become one HTTP
       400; the controller has no reach-failure branch. The vocabulary exists
       and is wired to agentClient() construction, not to the RPC. R-216's gate
       answers source=version and cannot see reachability.
R-225  the store reports 0 snapshots / 0 GB when it cannot read it, above a
       card saying it holds backups. Measured over SFTP: snapshot f3d9cd67 and
       12 535 KB are really there. snapshot_count/repo_size_bytes are ABSENT
       from settings.json after a rebuild — unknown renders as zero.
R-226  M1, the only message that tells a customer to check their typing, is
       unreachable on any box that has re-escrowed (M4's branch precedes it).
R-227  a restart mid-unlock returns a raw English "Bad Gateway" (I3).

Recorded against existing rows: R-216 §4.1 now MEASURED twice (and a correction
— SetFloor's DEBUG line can never reach the debug ring, so a restart alone
would not have produced it); R-218's live half still NOT measured, deliberately,
because the venue has a target; R-217 and R-215's fixes HELD under their faults.
R-199's back-pointer was already present in the capability map — the brief's
assumption that it was owed is stale.

No product code changed.
This commit is contained in:
2026-08-05 23:28:54 +02:00
parent 04ac465da6
commit 259f48aedc
+52 -3
View File
@@ -58,9 +58,17 @@ drill needed — the fourth is R-193, and until it goes the recovery is still op
## CAMPAIGN 11 — the recovery journey, 2026-08-05 ## CAMPAIGN 11 — the recovery journey, 2026-08-05
**The whole journey was walked end to end for the first time, on a throwaway appliance built from the **The whole journey was walked end to end for the first time, on a throwaway appliance built from the
published ISO. The data came back byte-identical; the journey did not exist.** Nine findings, R-214 … published ISO. The data came back byte-identical; the journey did not exist.** **Ten findings from
R-222. Six are fixed in controller **v0.201.0** + hub **v0.97.1**; **three are deliberately still Phases 1 and 3, R-214 … R-223** (seven fixed in controller **v0.201.0** + hub **v0.97.0/0.97.1**;
open** and each blocks a real flow. Evidence: `tests/campaign11-evidence-2026-08-05/journal.md`. **three deliberately still open**, each blocking a real flow), **plus four from Phase 2's injected
faults, R-224 … R-227.** Evidence: `tests/campaign11-evidence-2026-08-05/journal.md` (Phases 0/1/3)
and `journal-phase24.md` (Phases 2/4). Campaign document:
`audits/CAMPAIGN-11-recovery-journey-2026-08-05.md`.
> **Phase 2's verdict in one line.** The cryptography, the retention and the transport all work and
> are now proven live. **What fails is being told the truth:** a mistyped code, a hub outage, a
> stopped agent and a correct code for a retained earlier package all produce **one** message, and
> three of the four are wrong.
| ID | What | State | | ID | What | State |
|---|---|---| |---|---|---|
@@ -75,6 +83,47 @@ open** and each blocks a real flow. Evidence: `tests/campaign11-evidence-2026-08
| **R-221** | **A rebuilt box cannot run the escrow ceremony at all.** The preflight refuses on `escrow.pbs_storage_id`, which the pbsdr bridge seeds into `agent.json` only via `finishConverged`. The convergence marker lives on the HOST and survives a guest rebuild; `agent.json` is rewritten by the installer. Unchanged descriptor → same hash → early return → the seed never runs into a config that no longer has it. **Red-proved**: moving only the marker aside seeded it instantly (`grep -c escrow`: 0 → 1). A real blocker for re-escrow, which is exactly what a rebuilt box must do | **OPEN — NOT FIXED** | | **R-221** | **A rebuilt box cannot run the escrow ceremony at all.** The preflight refuses on `escrow.pbs_storage_id`, which the pbsdr bridge seeds into `agent.json` only via `finishConverged`. The convergence marker lives on the HOST and survives a guest rebuild; `agent.json` is rewritten by the installer. Unchanged descriptor → same hash → early return → the seed never runs into a config that no longer has it. **Red-proved**: moving only the marker aside seeded it instantly (`grep -c escrow`: 0 → 1). A real blocker for re-escrow, which is exactly what a rebuilt box must do | **OPEN — NOT FIXED** |
| **R-223** | **The Day-0 manifest vouched agent 0.120.0 while the recovery feature needs 0.125.0 — and a reinstall DOWNGRADES a box that was fixed by hand.** Verbatim from the second reinstall: `agent (existing): felhom-agent 0.125.0``manifest: agent v0.120.0``installed /usr/local/bin/felhom-agent (felhom-agent 0.120.0)`. So every rebuild re-broke the recovery path — the one event that makes the feature necessary. **⚠ AND IT WAS NOT A DROPDOWN.** The first vouch attempt was REFUSED by R-120's gate (`configs.go:1162`): *"golden 0.192.0 is older than the newest controller the fleet reports (0.201.0)"*. **The artifacts form saves as a unit, so the agent could not be vouched while the golden was stale — and the golden had been stale since controller 0.193.0, meaning the Day-0 manifest had been effectively UNVOUCHABLE for days and nobody had cause to notice.** The real remedy was a golden rebake | **CLOSED 2026-08-05.** Golden **0.201.0** baked in the drill VM (658 165 766 B, sha `e730d7cab343eb35…f007654`, **round-trip verified from Gitea**), then manifest set in one save: `agent=0.125.0 golden=0.201.0 min_agent=0.125.0`. A fresh install now lands on current agent AND current controller | | **R-223** | **The Day-0 manifest vouched agent 0.120.0 while the recovery feature needs 0.125.0 — and a reinstall DOWNGRADES a box that was fixed by hand.** Verbatim from the second reinstall: `agent (existing): felhom-agent 0.125.0``manifest: agent v0.120.0``installed /usr/local/bin/felhom-agent (felhom-agent 0.120.0)`. So every rebuild re-broke the recovery path — the one event that makes the feature necessary. **⚠ AND IT WAS NOT A DROPDOWN.** The first vouch attempt was REFUSED by R-120's gate (`configs.go:1162`): *"golden 0.192.0 is older than the newest controller the fleet reports (0.201.0)"*. **The artifacts form saves as a unit, so the agent could not be vouched while the golden was stale — and the golden had been stale since controller 0.193.0, meaning the Day-0 manifest had been effectively UNVOUCHABLE for days and nobody had cause to notice.** The real remedy was a golden rebake | **CLOSED 2026-08-05.** Golden **0.201.0** baked in the drill VM (658 165 766 B, sha `e730d7cab343eb35…f007654`, **round-trip verified from Gitea**), then manifest set in one save: `agent=0.125.0 golden=0.201.0 min_agent=0.125.0`. A fresh install now lands on current agent AND current controller |
### Phase 2 — the injected faults, 2026-08-05/06 (unattended)
Eleven faults, each judged on **the message, not the outcome**, each with a positive control proving
the fault was real. Full observables: `tests/campaign11-evidence-2026-08-05/journal-phase24.md`.
| ID | What | State |
|---|---|---|
| **R-224** | **Every non-code failure on the unlock path is reported to the customer as a statement about their code.** Two faults, one defect. **F3** — the hub REJECTed (control: `302``exit 7`) and the **correct current** code returned M4 (*„Ez a kód nem nyitja meg azt a csomagot…"*) in **0.0556 s**. **F4**`felhom-agent` stopped (control: `:8443` gone) and the same correct code returned M4 in **0.0299 s**. Neither elapsed time is compatible with a real unseal (~1.0 s of scrypt; F1's wrong-code attempts took 1.194/1.004/1.014 s), so **no unseal was attempted in either case** — the failure was upstream and was attributed to the code anyway. **The discriminator exists and is thrown away**: the agent's own `err` reads `fetching the sealed bundle: hub: transport error: … no route to host` (F3) versus `the recovery code did not unwrap the identity escrow` (F1), but both return **HTTP 400** under one merged sentence (*"…or the bundle could not be fetched"*), and the controller's failure path has **no branch for "could not ask / could not reach"**. **⚠ The vocabulary already exists and is wired to the wrong condition** — „A gép házon belüli kapcsolata most nem elérhető" fires only when `s.agentClient()` fails to *construct*, never when the RPC fails. **And R-216's gate cannot catch it**: the box's own ring reads `recovery capability gate: offsite_key_recovery=yes (source=version)` — the gate discriminates the agent's **age**, not its **reachability**, so a dead agent of the right version sails through the guard whose own comment says *"An attempt that cannot succeed must never be made, because its failure is attributed to the code."* **This is Phase 1's headline finding relocated from the version channel to the transport.** Breaches **I6** | **OPEN — NOT FIXED.** Fix shape: classify `rerr` (transport/fetch vs unseal) at the agent's HTTP boundary and give the controller a reach-failure branch. Do NOT widen the capability gate into a liveness probe on the unlock path — a probe that must succeed before an unlock adds a second thing that can fail |
| **R-225** | **The remote store reports `0 pillanatkép · 0 / 50 GB` when the box cannot read it — directly above a card stating the store holds backups.** `/backups/remote` renders both at once. **Ground truth, measured over SFTP with the box's own credential (read-only, no decryption):** `/home/felhom-repo/snapshots` holds **`f3d9cd67…`** — the Phase 0 snapshot with all three sentinels, the customer's only surviving copy — and `du -s` returns **12 535 KB**, matching the journal's `repo_size_bytes 12 611 522`. **Mechanism, from the box's own state:** after the rebuild the `offbox` block carries **no `snapshot_count` and no `repo_size_bytes` key at all** — the values are *unknown* and unknown renders as the zero value. **This is R-217's defect class in a second location**, and `OffsiteInventory.Empty` exists precisely because *"len(Apps)==0 is also what a failed read looks like"*. **I5 checked and NOT breached** — the hub's `0.0 GB` is 12.5 MB rounded, and the pool total is consistent; only the box's snapshot **count**, an integer, is false. Breaches **I6** | **OPEN — NOT FIXED.** A count the box has never successfully taken must render as unknown („—"), never as `0` |
| **R-226** | **M1 — the only message that tells a customer to check their typing — is unreachable on any box that has re-escrowed.** The failure path tests M4's condition *before* M1's (`if present, at := s.recoverySuperseded(); present { …M4…; return }`), so on every box the hub keeps an earlier package for, a **genuinely mistyped code** produces M4. **Measured**: F1's three wrong-code attempts each returned M4 after a real ~1 s unseal. M4 is hedged (*„**Ha** egy korábbi kódot adtál meg"*) and states two true facts, but it offers no hint to re-check the ten words and routes the customer to support about *older* backups. **The population that has re-escrowed is exactly the population that has just been handed a new recovery code and is most likely to be typing one.** R-222's fix removed one conflation and introduced another on the same branch | **OPEN — NOT FIXED.** Fix shape: M4 should name both possibilities (a mistype OR an earlier code), since from the engine's side they genuinely are indistinguishable — which is the honest thing to say |
| **R-227** | **A controller restart mid-unlock returns a raw English `Bad Gateway`.** F8 restarted the container at T+0.7 s, inside the unseal window (control: `StartedAt` moved). The customer got **HTTP 502 / „Bad Gateway"** from traefik — a raw upstream error, in English, naming no reason and saying nothing about whether the key was installed. **The state half is clean**: the four `/data/offbox` files stayed byte-identical with mtimes frozen, and the controller returned healthy in 40 s. Breaches **I3** | **OPEN — NOT FIXED.** Low severity — the window is ~1 s wide. Recorded rather than inflated |
**Recorded against existing rows by Phase 2:**
- **R-216 — §4.1 is now MEASURED, not deduced.** The previous session could only offer two absences.
The box's own `/settings` renders „Minimális verzió (üzemeltető) **0.200.0**" (`GetFloor()`, whose
only writer is the report-ACK handler; **both hold branches serve `Floor=""`**, pinned by
`managed_floor_test.go:94`), and a **cold-started** controller logs
`settle-gate: GO — at/above floor 0.200.0 (we are 0.201.0)` against the same line reading
`floor still unknown after 1m30s` while the hold was in force. The hub's HELD lines ran every
15 min to 22:12:06 and stopped, with a liveness control proving the hub kept logging. **The floor
is served.**
- **⚠ A correction to how that positive was to be taken.** `SetFloor`'s line is `u.dbg(...)`, gated on
`cfg.Logging.Level == "debug"` and written to the **logger** — it can **never** reach the logx debug
ring, so it cannot appear in `/api/debug/logs` at any level. A controller restart alone would not
have produced it. Confirmed with a level census on the ring first (1196 DEBUG / 2802 INFO / 2 WARN),
so the absence was known to be structural rather than evidential.
- **R-218 — the live half is STILL NOT MEASURED, deliberately.** The fix is present and correct
(`needsOffsiteCredential` now retires on the **target**, not the key), but the venue **has** a
target, so the box correctly does not declare; declaring here would be the bug. The state that
exercises it is shape (a), which the venue no longer holds. **Recorded as not measured rather than
inferred from the unit test.**
- **R-217 — its fix HELD under exactly its fault** (F5): with the store blocked after a successful
unlock, the page rendered M3 and **no listing block at all**; the three false-claim strings are
absent, verified in UTF-8 with accented positive controls present.
- **R-215 — its fix is present** and the `GET /recovery` gate consults the same predicate as the POST
sibling.
- **R-199's back-pointer in `architecture/00-capability-map.md` was already added** — the brief lists
it as owed; it is present on the escrow-recovery row, explicitly labelled as the omitted
back-pointer. **No action taken; the brief's assumption was stale.**
**Untouched by this session, stated so nothing is presumed closed by association:** **R-213** (putting **Untouched by this session, stated so nothing is presumed closed by association:** **R-213** (putting
files back — the half the recovery screen deliberately does not do) and **R-202** (the orphan card's files back — the half the recovery screen deliberately does not do) and **R-202** (the orphan card's
unconditional promise, which CAMPAIGN-11 §7 step 7 measured the customer-facing cost of). unconditional promise, which CAMPAIGN-11 §7 step 7 measured the customer-facing cost of).