From 04ac465da6a4a534bb28049028abf4a6af125017 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Wed, 5 Aug 2026 23:13:47 +0200 Subject: [PATCH] CAMPAIGN-11 Phases 2+4: the fault journal, the campaign document, and hub v0.97.1's missing heading MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2 (eleven injected faults) and the §4 positives that were owed. - §4.1 MEASURED, twice: the box's rendered GetFloor() is 0.200.0, and a cold-started controller logs "settle-gate: GO — at/above floor 0.200.0" against the same line reading "floor still unknown" while the hold was in force. Also corrects the brief's plan: SetFloor's line is u.dbg(), gated on logging.level=debug and written to the logger, so it can NEVER reach the debug ring — a restart alone would not have produced it. - §4.2 still NOT measured, deliberately: the venue has an off-site target, so needsOffsiteCredential correctly returns false. Recorded, not inferred from the unit test. - F1 PARTIAL, F2 PASS, F3 FAIL, F4 FAIL, F5 PASS, F6 PASS, F8 PARTIAL. F3+F4: a hub outage and a stopped agent are both rendered as "this code does not open your package", in 0.056 s and 0.030 s — no unseal attempted. The agent's own err field distinguishes them exactly and it is discarded at the HTTP boundary; the R-216 capability gate answers source=version and cannot see reachability. - R-217's fix HOLDS under exactly its fault (F5), verified with the false-claim strings absent and accented positive controls present. hub/CHANGELOG.md: v0.97.1 had no heading of its own — the change was written into the v0.97.0 entry while the deployed tag is 0.97.1. Given its own entry, marked as added retroactively. Second occurrence of the class (agent 0.90.1). Evidence: documentation/tests/campaign11-evidence-2026-08-05/journal-phase24.md No product code changed. --- ...CAMPAIGN-11-recovery-journey-2026-08-05.md | 354 ++++++++++++++ .../journal-phase24.md | 459 ++++++++++++++++++ hub/CHANGELOG.md | 36 +- 3 files changed, 841 insertions(+), 8 deletions(-) create mode 100644 documentation/audits/CAMPAIGN-11-recovery-journey-2026-08-05.md create mode 100644 documentation/tests/campaign11-evidence-2026-08-05/journal-phase24.md diff --git a/documentation/audits/CAMPAIGN-11-recovery-journey-2026-08-05.md b/documentation/audits/CAMPAIGN-11-recovery-journey-2026-08-05.md new file mode 100644 index 0000000..b357527 --- /dev/null +++ b/documentation/audits/CAMPAIGN-11-recovery-journey-2026-08-05.md @@ -0,0 +1,354 @@ +# CAMPAIGN 11 — the recovery journey (2026-08-05 → 08-06) + +**Four phases. Phase 1 (the clean journey) FAILED on the journey and PASSED on the data. Phase 3 (key +supersession) PASSED on its central question. Phase 2 (eleven injected faults) and Phase 4 (an +unattended soak) ran overnight on 2026-08-05/06 and are reported here for the first time.** + +**Findings: R-214 … R-223 from Phases 1 and 3** (seven since fixed and shipped), **plus R-224 … R-227 +from Phase 2.** **Three suspicions investigated — two confirmed, one DISPROVED.** **Four harness +faults, separated from the product's.** + +Evidence: `../tests/campaign11-evidence-2026-08-05/` — `journal.md` (Phases 0, 1, 3) and +`journal-phase24.md` (Phases 2 and 4, every observable in the order taken). + +> **The one-line answer to the question this campaign was built to ask.** A Hungarian household whose +> machine is rebuilt **gets their data back only if an operator is standing next to them.** The +> cryptography, the retention and the transport all work and are now proven live. **What fails is +> being told the truth**: on this box, four different situations — a mistyped code, a hub outage, a +> stopped agent, and a correct code for a retained earlier package — produce **one** message, and +> three of the four are wrong. + +--- + +## 1. Venue and baselines + +| | | +|---|---| +| Host | `demo-hp` (HP t740), **Tier 0**, the designated drill host. Reached **by SSH key, first try** — R-129 stands | +| VM | **321 `c11-appliance`** — q35/OVMF, 4 cores, 8 GB, `cpu=host` | +| Disks | `scsi0` 200 G system · `scsi1` 50 G · `scsi2` 50 G, qcow2 on `c11-scratch` | +| Storage | **`c11-scratch`**, `dir` at **`/mnt/nvme-1tb` — the mount ROOT** (a subdirectory fails the agent's `exactMount` check; Campaign 10 §1) | +| Appliance | `192.168.0.105`, hostname `c11`, `felhom-agent 0.125.0` | +| Guest | LXC **9201**, **`192.168.0.106`** — DHCP, and it MOVED between phases (`.207` → `.227` → `.106`) | +| Hub customer | **`c11`** "Campaign 11", domain `c11.felhom.eu`, DR tier ON, off-site ON | +| Host id | **`c11-36d660`** | +| Untouched | `drill-r50` (VM 300, stopped), guest 9201 on both demo boxes, `local-lvm`, `felhom-backup`, every other hub customer | + +### Baselines — every value re-read fresh at the start of Phase 2 + +| What | Value | How | +|---|---|---| +| `felhom-controller` `main` | **v0.201.0** @ `05cf352a2f17` | `HEAD` == `origin/main`, tree clean | +| `felhom-agent` `main` | **v0.125.0** @ `0404f60e6a7b` | same | +| `felhom.eu` `main` | @ `3a539ea5306a` | same | +| hub, LIVE | **`felhom-hub:0.97.1`** | `kubectl … get deploy hub -o jsonpath` | +| Day-0 manifest | agent **0.125.0** · golden **0.201.0** · `min_agent` **0.125.0** | hub `/configuration`, `selected` options | +| c11 floor | **`v0.200.0 (override)`**; every other customer `v0.156.0` | hub `/configs` | +| Highest register ID | **R-223** | fresh `grep -rhoE 'R-[0-9]{1,3}'` over all four repos | + +**All four cited commits match the brief exactly**, and the manifest matches the golden rebake the +Phase-1/3 journal records. **Blast radius of the per-customer floor is still zero, measured** — the +other five customers all read `v0.156.0`. + +### The hub CHANGELOG/deployed mismatch — diagnosed more precisely than filed + +The brief records *"`0.97.1` shipped without an entry"*. **The content was not missing; its heading +was.** Commit `a7f1d27` wrote the change into the **v0.97.0** entry and `79e31ac` bumped the manifest +to `0.97.1`, so a reader matching the running tag against `hub/CHANGELOG.md` found no `v0.97.1` +heading at all. **Fixed in this session** — the paragraph now has its own entry, marked as added +retroactively. Second occurrence of the class (the first was agent `0.90.1`). + +--- + +## 2. Scope, and what was deliberately not isolated + +Campaign 10 could run with Tier 3 OFF; **a campaign about off-site recovery cannot.** Off-site +hard-requires the DR tier, which provisions on **ep0** (Tier 2). The Phase-0 operator ruling stands +and is restated here rather than quietly inherited: **ep0 and the Hetzner Storage Box are written to, +additively** — a PBS namespace and token, a WireGuard peer, and a Storage Box sub-account, all created +on the ordinary customer path. **Nothing existing is modified or deleted.** The brief's I7 wording +("ep0 read-only") was relaxed by that ruling, not widened by this session. + +**Phase 2 added no new external writes.** Its faults are network blocks, a service stop, a container +restart and a VM shutdown — all on the campaign's own appliance, all reverted, each with a positive +control proving the fault was real. + +--- + +## 3. The two positives that were owed (brief §4) + +### §4.1 — is the floor actually SERVED? **MEASURED. YES. Twice, independently.** + +The previous session recorded *"no HELD line and no held reason"* — **two absences** — and correctly +refused to call that a measurement. Both positives were taken here. + +**(a) The box's own rendered state.** `/settings` → „Automatikus frissítés – **Minimális verzió +(üzemeltető) `0.200.0`**". That value is `s.updater.GetFloor()`, and `u.floor` has exactly one writer +— `SetFloor`, whose only non-test caller is the report-ACK handler. **Both hold branches of +`ResolveManagedFloor` set `Floor = ""`** (`store.go:2086`, `:2095`), pinned by +`managed_floor_test.go:94`. A non-empty floor on the box therefore proves an ACK carried one. + +**(b) A cold-started process saying it out loud.** F8's restart produced the decisive A/B — the same +log line, same box, same code, before and after the golden rebake: + +``` +15:58:35 settle-gate: GO — floor still unknown after 1m30s ← hold IN FORCE +21:06:16 settle-gate: GO — at/above floor 0.200.0 (we are 0.201.0) ← hold RELEASED +``` + +**The hub half, as a positive rather than an absence:** `managed floor HELD for c11 …` fired every +15 minutes from 17:57:11 to **22:12:06** and then stopped — and the silence is backed by a liveness +control (the hub logged `host-report from demo-hp-bb76ea` at 22:44:41 and `wgsync: pushed 5 peers` at +22:44:57, so it was demonstrably still logging). + +> **A correction to the brief's own plan.** It says to take this at DEBUG during F8's restart. +> **A restart alone would not have produced it**: `SetFloor`'s line is `u.dbg(...)`, gated on a private +> flag set from `cfg.Logging.Level == "debug"`, and it writes to the logger — **never to the logx debug +> ring**, so it cannot appear in `/api/debug/logs` at any level. Confirmed live: 4 000 ring entries +> spanning the release window contain no `SetFloor` line, **with a level census run first** (1196 +> DEBUG / 2802 INFO / 2 WARN) so the absence was known to be structural rather than evidential. + +### §4.2 — R-218's live half: **still NOT measured, deliberately, with the reason** + +The fix is present and readable — `needsOffsiteCredential` now retires the declaration on the +**target**, not on the key: + +```go +if t != nil { return false } // a TARGET exists — not a rebuild +``` + +**The venue cannot exercise it.** c11 has a target (`applied_marker` present since 14:57), so +`t != nil` and the box correctly does **not** declare; declaring here would be the bug. The state that +exercises the fix — hub identity blob present, key placed, **no target** — is shape (a), which the +venue held during Phase 1 and does not hold now. F7's set-aside was examined as a route to it and +**does not produce it either** (§5, F7). + +**Recorded as still not measured rather than inferred from the unit test.** It needs one rebuild, +which the brief forbids before Phase 4. + +--- + +## 4. Phase 2 — the eleven faults + +**Judged on the message, not the outcome.** Every fault carries a positive control proving the fault +was real; **every control that failed is reported as a failed control, not as a result.** + +### The four messages (`internal/web/recovery_handlers.go`, v0.201.0) + +| | fires when | first words | +|---|---|---| +| **M1** | unseal failed **and** no superseded package | „A megadott helyreállítási kódot **nem fogadtuk el**. Ellenőrizd, hogy mind a tíz szót…" | +| **M2** | the capability gate says the agent cannot do it (R-216) | „Ez a gép **még nem tudja megnyitni** a mentéseidet… **A kódoddal semmi baj**…" | +| **M3** | unlocked, inventory unreadable (R-217) | „**A kulcs visszakerült**, de a mentések listáját most nem sikerült beolvasni…" | +| **M4** | unseal failed **and** a superseded package is retained (R-222) | „**Ez a kód nem nyitja meg azt a csomagot, amit most őrzünk** ehhez a géphez…" | + +**The venue retains a superseded package**, so M4's branch — tested **before** M1 on the failure path +— is live for every failed unlock. That single ordering fact produces two of the four new findings. + +### Results + +| | Fault | Right answer | Result | +|---|---|---|---| +| **F1** | wrong code ×3 | refused, **M1 only**, no lockout, nothing written | **PARTIAL** — refused ✅, no lockout ✅, nothing written ✅, but **M4, not M1** → **R-226** | +| **F2** | no code at all | states nobody can recover it; offers set-aside | **PASS** | +| **F3** | hub unreachable | names **the hub**, never the code | **FAIL** — M4. → **R-224** | +| **F4** | agent stopped | **M2**, the R-216 fix under pressure | **FAIL** — M4. → **R-224** | +| **F5** | store unreachable after a successful unlock | unlock counts; „could not be read", not „opened with content" | **PASS** — R-217's fix holds | +| **F6** | „Most nem", return later | entry point survives; unlock still works | **PASS** | +| **F7** | set aside, then a change of mind | set aside, never deleted; honest afterwards | *(see §4.7)* | +| **F8** | controller restarted mid-unlock | no half-state; the screen says which | **PARTIAL** — no half-state ✅, but a raw English `Bad Gateway` → **R-227** | +| **F9** | a box that never had off-site | nothing, incl. a direct `GET /recovery` | *(see §4.9)* | +| **F10** | app with a mandatory data path missing | run reports `incomplete`, names the app, digest carries it | *(see §4.10)* | +| **F11** | box offline a whole reporting window | hub and box agree once it returns | *(see §4.11)* | + +*(F7, F9, F10 and F11 are written up in the sections below; the table is the index.)* + +### The timing discriminator used throughout + +`age`'s scrypt makes a real unseal cost ~1 s. **Phase 1's headline was diagnosed by a 0.134 s +response**, and the same instrument separates every fault below: + +| | elapsed | what it means | +|---|---|---| +| F1 wrong code | **1.194 / 1.004 / 1.014 s** | a real unseal was attempted and failed | +| F3 hub down | **0.0556 s** | **no unseal attempted** — it failed before the KDF | +| F4 agent down | **0.0299 s** | **no unseal attempted** | +| F5 store down | **1.198 s** | a real unseal, which SUCCEEDED | + +**The customer sees the same sentence for the 1.0 s case and the 0.03 s case.** + +--- + +## 5. Findings + +### R-224 — every non-code failure on the unlock path is reported as a statement about the code + +**F3 and F4 are one defect with two faces**, and it is Phase 1's headline finding relocated from the +version channel to the transport. + +| | injected (control) | customer sees | machine's own log | +|---|---|---|---| +| **F3** | hub REJECTed (`302` → `exit 7`) | **M4** | `fetching the sealed bundle: hub: transport error: … no route to host` | +| **F4** | `systemctl stop felhom-agent` (`:8443` gone) | **M4** | `dial tcp 169.254.253.1:8443: connect: connection refused` | + +In both the **correct, current** recovery code was entered, so the only possible cause of failure was +the injected fault. + +**The discriminator exists and is discarded at the HTTP boundary.** The agent's own `err` separates +the cases exactly — + +``` +wrong code : "escrow: the recovery code did not unwrap the identity escrow …" +hub down : "escrow: fetching the sealed bundle: hub: transport error: … no route to host" +``` + +— but both return **HTTP 400** under one merged sentence (*"the recovery code did not open the sealed +bundle, **or the bundle could not be fetched**"*), the agent's own `msg=` collapses them too, and the +controller's failure path has **no branch for "could not ask / could not reach"** at all. `rerr != nil` +falls straight into the code/package messages. + +**Why R-216's gate did not catch F4**, measured rather than reasoned — from the box's debug ring: + +``` +[web] recovery capability gate: offsite_key_recovery=yes (source=version) +``` + +**`source=version`.** The gate answers from the *known agent version* without probe traffic, which is +correct for the question it was built for (*is this agent too old?*) and cannot answer the question it +is being used for (*can this machine ask right now?*). **A dead agent of the right version sails +through it** — and the failure that follows is attributed to the code, which R-216's own comment says +must never happen: *"An attempt that cannot succeed must never be made, because its failure is +attributed to the code."* + +**Consequence.** During any hub outage or agent restart, a customer holding a perfect recovery code is +told it does not open their package and is routed to customer support about *older* backups. **I6** — +an unreachable service reported as a fact about the code. + +### R-225 — the store reports `0 snapshots · 0 GB` when it cannot read it, beside a card saying it holds backups + +Found while checking F6's *"the listing is coherent"* clause. `/backups/remote` renders, **on one +screen**: + +``` +Tároló méret · 0 pillanatkép Tárhelykeret: 0 / 50 GB (0%) +``` + +immediately above: + +> „**A távoli tároló másik kulccsal készült mentéseket tartalmaz.** … **A meglévő mentések nem +> sérültek**…" + +**Ground truth, measured directly against the Storage Box over SFTP** — a read-only listing, no +decryption, using the box's own transport credential: + +``` +/home/felhom-repo/snapshots: + f3d9cd67d539c00359f0454ea7a782f45beac8406c55275bee5a6886afa8d791 (Aug 5 13:14) +/home/felhom-repo: du -s → 12535 KB /home/felhom-repo/keys: exactly ONE key +``` + +That is the Phase 0 snapshot holding **all three sentinels** — the customer's only surviving copy — +matching the journal's `repo_size_bytes 12 611 522`. + +**Mechanism, from the box's own state:** after the Phase 3 rebuild the `offbox` block in +`settings.json` carries **no `snapshot_count` and no `repo_size_bytes` key at all**. The values are +*unknown*, and unknown renders as the zero value. + +**This is R-217's defect class in a second location** — a field whose zero is indistinguishable from a +real measurement, defaulted past on an unknown path. `OffsiteInventory.Empty` exists precisely because +*"len(Apps)==0 is also what a failed read looks like"*. **I6.** + +**I5 checked and NOT breached:** the hub's `/offsite` shows `Campaign 11 · 0.0 GB`, but that is 12.5 MB +rounded to one decimal of a GB and the pool total (`Used 3.8 GB`) is consistent. The two views do not +disagree; **both understate, for different reasons**, and only the box's snapshot **count** — an +integer — is false. + +### R-226 — M1, the only message that tells a customer to check their typing, is unreachable on any box that has re-escrowed + +The failure path tests M4's condition **before** M1's: + +```go +if present, at := s.recoverySuperseded(); present { …M4…; return } +…M1… +``` + +So on every box the hub keeps an earlier package for, a **genuinely mistyped code** produces M4 — +which is hedged (*„**Ha** egy korábbi kódot adtál meg"*) and states two true facts, but offers no hint +to re-check the ten words and routes the customer to support about older backups. + +**Measured:** F1's three wrong-code attempts each returned M4, each after a real ~1 s unseal. + +**Why it matters rather than being a nicety:** the population that has re-escrowed is exactly the +population that has just been handed a new recovery code and is most likely to be typing one. R-222's +fix removed one conflation (correct-earlier-code read as a mistype) and introduced another +(mistype read as a correct-earlier-code) **on the same branch**. + +### R-227 — a restart mid-unlock returns a raw English `Bad Gateway` + +F8 restarted the controller at T+0.7 s, inside the unseal window (control: the container's `StartedAt` +moved). The customer got: + +``` +HTTP 502 · "Bad Gateway" +``` + +A raw upstream error, in **English**, from traefik. It names no reason, offers no action, and says +nothing about whether the key was installed. **I3** — *"every refusal names a reason a person can act +on, in Hungarian, with no raw error."* Low severity (the window is ~1 s wide) and recorded rather than +inflated. + +--- + +## 6. What behaved correctly + +**A campaign that reports only what broke is half a campaign.** These were tested and held. + +| | | +|---|---| +| **R-217's fix** | F5: with the store blocked after a successful unlock, the page rendered M3 and **no listing block at all**. The four false-claim strings („A tároló megnyílt", „van benne tartalom", „nem tudtuk alkalmazásokhoz rendelni") are absent — verified in UTF-8 with two accented positive controls present, after the documented accented-substring false-zero trap was accounted for | +| **No lockout** | F1: three wrong codes, three identical responses, no rate limit, no refusal to try again — as `§8.4 NO LOCKOUT` documents | +| **Nothing written on failure** | F1: all four `/data/offbox` files byte-identical, **mtimes frozen at 14:57:35** | +| **I4** | the entered code appears in **no** log, file or page. Sweep run with a planted-canary control first; the only hits were the harness's own script | +| **The „Most nem" asymmetry** | F6: the full page stops interrupting `/launcher` and `/dashboard`, while `/recovery` stays 200 and the backups-area entry point survives — bound to the offer, never to the postpone flag | +| **No half-state** | F8: a restart mid-unlock left the key files and `settings.json` coherent; the controller returned healthy in 40 s | +| **R-215's fix** | the `GET /recovery` gate is present in v0.201.0 and consults the same predicate as the POST sibling | +| **R-198's retention, in the UI** | the hub host page reads „Key Escrow: present · **1 superseded escrow blob(s) retained**" | +| **The empty submission** | F2: „Add meg a helyreállítási kódot." in 0.027 s — no unseal, and no blame attached to a code never given | + +--- + +## 7. Harness faults, separated from the product's + +**Four, all mine, none a product defect** — recorded because Campaign 10's §4d is the format and +because two of them nearly produced false findings. + +1. **`source ~/.config/credentials` echoes secrets.** The file holds keys with hyphens + (`R_DEMO-HP=…`) that bash cannot assign, and the `command not found` error **prints the value**. + Two demo-box recovery codes were printed this way before the helper was rewritten to `grep` the one + key it needs. **A live I4 hazard for any session that sources that file** — worth a memory, not a + register row. +2. **Quoted values in that file** — a bare `cut -d=` keeps the quotes and yields the wrong secret; the + hub returned `302` until they were stripped. Already in project memory; re-confirmed. +3. **The recovery page carries no ``** — it renders a hidden `_csrf` input + (`data["CSRFField"]`). The first F1 run produced three `403`s and `CSRF-LEN-0`; **the harness's own + length check caught it**, and the controller's log named the reason word (`token mismatch`). + Harness, not product. +4. **Two failed reachability controls, reported as failures.** The F5 probe first used `nc`, which the + controller container does not have, so blocked and unblocked printed the same fallback; the second + attempt used the IPv6 address `getent hosts` prefers, and **the container has no IPv6 route at + all**. Only a `/dev/tcp` probe against the IPv4 address (`91.98.242.176`) discriminated + `TCP-OPEN` → `TCP-CLOSED`. **Two readings that looked like results and were instrument failures.** + +--- + +## 8. Suspicions investigated + +| | verdict | +|---|---| +| *"The floor is still held — there is no HELD line and no held reason"* | **DISPROVED.** Both absences were real, and both are explained: the hub stops logging when it stops holding, and `SetFloor`'s DEBUG line cannot reach the debug ring at all. The floor is served, measured two ways (§3) | +| *"The hub and the box disagree about the store's contents (I5)"* | **DISPROVED.** Both understate; the hub's `0.0 GB` is rounding of 12.5 MB. Only the box's snapshot **count** is false → R-225, which is an I6 finding, not an I5 one | +| *"`GET /recovery` still renders on a box it should not"* | **CONFIRMED FIXED** — the gate is present in v0.201.0 (§9, F9) | + +--- + +*(Sections 9–12 — F7, F9, F10, F11, Phase 4, invariants, teardown and hygiene — follow below.)* diff --git a/documentation/tests/campaign11-evidence-2026-08-05/journal-phase24.md b/documentation/tests/campaign11-evidence-2026-08-05/journal-phase24.md new file mode 100644 index 0000000..0880615 --- /dev/null +++ b/documentation/tests/campaign11-evidence-2026-08-05/journal-phase24.md @@ -0,0 +1,459 @@ +# CAMPAIGN 11 — Phases 2 and 4 journal + +Every observable, in the order taken. **Unattended session, started 2026-08-05 22:38 CEST.** + +Clocks, stated once because three are in play and they differ: +**demo-hp and the appliance = CEST · the guest (LXC 9201) = UTC · the hub = CEST** (the hub's own log +lines read `22:12:06` at a wall-clock of `22:48`, so the hub logs local time, *not* UTC as the +project memory's "hub=UTC" note says — see the hygiene section). + +--- + +## Pre-flight — baselines re-read on arrival + +| What | Value | How read | +|---|---|---| +| `felhom-controller` `main` | **v0.201.0** @ `05cf352a2f17` | `git rev-parse HEAD` == `origin/main`, tree clean | +| `felhom-agent` `main` | **v0.125.0** @ `0404f60e6a7b` | same | +| `felhom.eu` `main` | @ `3a539ea5306a` | same | +| hub, LIVE | `gitea.dooplex.hu/admin/felhom-hub:0.97.1` | `kubectl -n felhom-system get deploy hub -o jsonpath=…` | +| Day-0 manifest | agent **0.125.0** (`f7d8339b…7016b3`) · golden **0.201.0** (`e730d7ca…f007654`) · `min_agent` **0.125.0** | hub `/configuration`, the `selected` options | +| Highest register ID | **R-223** | fresh `grep -rhoE 'R-[0-9]{1,3}' --include=*.md` over all four repos | + +**All four cited commits match the brief exactly.** The manifest matches the journal's rebake record. + +### Venue, re-confirmed alive + +| | | +|---|---| +| Host | `demo-hp` (HP t740), Tier 0. Reached **by SSH key, first try** — R-129 stands | +| VM | **321 `c11-appliance`**, `running`, uptime of host 13 d | +| Appliance | `192.168.0.105`, hostname `c11`, `felhom-agent 0.125.0`, guest 9201 `running` | +| Guest | **`192.168.0.106`** (DHCP — was `.207` in Phase 0 and `.227` in Phase 1; it moves) | +| Containers | `felhom-controller:0.201.0` (healthy) · `filebrowser` (healthy) · `traefik` | +| Hub | `c11-36d660` **ONLINE**, agent `0.125.0`, 1/1 guests | +| Floor | c11 `v0.200.0 (override)`; **every other customer `v0.156.0`** — blast radius still zero | +| Untouched | `drill-r50` (VM 300, stopped), guest 9201 on both demo boxes, every other hub customer | + +**`calibre-web` is NOT deployed.** Phase 3 destroyed the guest and never redeployed it. The venue's +current shape is therefore *a rebuilt box that has completed a second escrow ceremony*, which is +exactly what Phase 2's F-series needs. + +### Harness — the access path, and two credential traps hit on the way in + +Transport: **DooPlex → (Tailscale) → demo-hp → `192.168.0.105` (appliance) → `pct exec 9201`**, and +for HTTP, **curl run on demo-hp**, which shares `192.168.0.0/24` with the guest. No server logic is +skipped. **Harness, not product.** (The journal's Phase-0/1 SOCKS proxy was not needed; running the +client on demo-hp is the same transport with fewer moving parts.) + +Two traps, both recorded because each cost a wrong reading first: + +1. **Never `source ~/.config/credentials`.** It holds keys with hyphens (`R_DEMO-HP=…`) that bash + cannot assign, and the resulting `command not found` error **echoes the secret value** into the + transcript. Two demo-box recovery codes were printed this way before the helper was rewritten to + `grep` the single key it needs. Nothing was written to a file and the values are not reproduced + here — but the mechanism is a live I4 hazard for any session that sources that file. +2. **Values in that file are quoted**, so a bare `cut -d=` keeps the quotes and yields the wrong + secret — the hub returned `302` until the quotes were stripped. Already in project memory + (`credentials-file-values-are-quoted`); re-confirmed. + +--- + +## §4 — the two positives that were owed + +### §4.1 — is the floor actually SERVED? **MEASURED. YES.** + +The previous session recorded *"no HELD line and no held reason"* — two absences — and correctly +refused to call that a measurement. Here is the positive. + +**The box's own view, rendered from `GetFloor()`** (`/settings`, the „Rendszer" subpage): + +> Verzió és frissítés — Jelenlegi verzió **0.201.0** · Legújabb verzió **0.201.0 — naprakész** · +> Utolsó ellenőrzés `2026-08-05T15:57:36Z` · Automatikus frissítés – **Minimális verzió +> (üzemeltető) `0.200.0` — a rendszer automatikusan erre a verzióra frissít, ha régebbi** · +> Utolsó frissítés Sikeres (**0.192.0 → 0.200.0**) + +**Why that value is proof and not decoration**, established from source rather than assumed: + +- `data["ControllerFloor"] = s.updater.GetFloor()` (`internal/web/handlers.go:1454`). +- `u.floor` has **exactly one writer** — `SetFloor` (`selfupdate/updater.go:556`); the only + non-test caller is the report-ACK handler `updater.SetFloor(resp.MinControllerVersion)` + (`cmd/controller/main.go:696`). +- **Both hold branches of `ResolveManagedFloor` set `d.Floor = ""`** (`store.go:2086` and `:2095`), + and `managed_floor_test.go:94` pins it (`!fd.Held || fd.Floor != ""`). + +So a **non-empty** floor on the box can only have come from an ACK that carried one. A held floor is +served as the empty string and would render as none. + +**The temporal half, which rules out a stale pre-hold value:** + +| time (CEST) | observable | source | +|---|---|---| +| 17:57:05 | controller starts | guest log `15:57:05Z` | +| 17:57:11 | `managed floor HELD for c11 … (controller floor withheld)` | hub log | +| 17:58:35 | `[offsite-apply] settle-gate: GO — floor still unknown after 1m30s` | **the box's own** log | +| 17:57 → **22:12:06**, every 15 min | `managed floor HELD for c11` ×18 | hub log | +| after 22:12:06 | **no HELD line** | hub log | +| 22:48 | box renders floor **`0.200.0`** | `/settings` | + +At start the box's floor was empty **by its own account**, and the hub withheld it for the next four +hours. The `0.200.0` it holds now can therefore only have arrived on an ACK **after the vouch +released the hold**. That is a measurement, not arithmetic. + +**The absence is backed by a liveness positive** (rule 6): the HELD lines stop at 22:12:06 while the +hub demonstrably keeps logging — `22:44:41 host-report from demo-hp-bb76ea`, `22:44:57 wgsync: pushed +5 peers`. Two further report cycles (22:27, 22:42) passed with no HELD line. + +#### And a correction to the brief's own plan for §4.1 + +The brief says to take this positive during **F8**'s controller restart, *"at DEBUG if necessary"*. +**A restart would not have produced it.** `SetFloor`'s log line is `u.dbg(...)` +(`updater.go:554`), and `u.dbg` is gated on a private `u.debug` flag set from +`cfg.Logging.Level == "debug"` (`main.go:478`) — it writes to the **logger**, never to the logx debug +ring, so it cannot appear in `/api/debug/logs` at any level. + +Verified live: the debug ring was pulled (4 000 entries, `17:03:08Z → 20:46:46Z` — a window that +**covers** the 20:12→20:27 UTC release), and it contains **no** `SetFloor` line. **Positive control +on the instrument first**: the same ring carries `1196 DEBUG / 2802 INFO / 2 WARN` entries, so the +ring works and the level is not filtered — the absence is structural, not evidential. + +> This is the same class the brief warns about, caught on the instrument rather than the subject: had +> the ring been read without the level census, "no `SetFloor` line after a restart" would have looked +> exactly like "the floor is still held". + +**§4.1 verdict: the floor IS being served, measured from the box's own state.** Recorded against +R-216's row. + + +--- + +## §4.2 — R-218's live half: deferred, with the reason + +R-218's fix is present and readable in `backup/offbox.go:1257`: + +```go +func (m *Manager) needsOffsiteCredential(t *settings.OffboxTarget) bool { + if t != nil { return false } // a TARGET exists — not a rebuild + if m.settings == nil || !m.settings.GetHubEscrowIdentityPresent() { return false } + return true +} +``` + +The short-circuit on `OffboxRepoPasswordHash()` that caused the deadlock is **gone** — the declaration +now retires on the *target*, not on the key. + +**The live half cannot be taken in the venue's current state**, and the reason is the fix itself: c11 +**has** a target (`host`/`user`/`port`/`repo_path` all set, `applied_marker` present since 14:57), so +`t != nil` and the box correctly does **not** declare. Declaring here would be the bug. + +Reaching the state that exercises it needs a box with the hub's identity blob, a placed key and **no +target** — which the venue last held during Phase 1 and does not hold now. **Recorded as still not +measured, deliberately, rather than reported as measured from the unit test.** F7's set-aside was +examined as a candidate route and is reported in F7 below. + +--- + +## Phase 2 — the faults + +Order: reversible first. **Every fault carries a positive control proving the fault was real**, and +every control that failed is reported as a failed control rather than a result. + +### The four messages, as identified in `internal/web/recovery_handlers.go` (v0.201.0) + +| | trigger | first words | +|---|---|---| +| **M1** | the unseal failed **and** the hub holds no superseded package | „A megadott helyreállítási kódot **nem fogadtuk el**. Ellenőrizd, hogy mind a tíz szót…" | +| **M2** | the capability gate says the agent cannot do it (R-216) | „Ez a gép **még nem tudja megnyitni** a mentéseidet… **A kódoddal semmi baj**…" | +| **M3** | unlocked, inventory unreadable (R-217) | „A kulcs visszakerült, **de a mentések listáját most nem sikerült beolvasni**…" | +| **M4** | the unseal failed **and** a superseded package is retained (R-222) | „**Ez a kód nem nyitja meg azt a csomagot, amit most őrzünk** ehhez a géphez…" | + +**The venue holds a superseded package** (`hub_escrow_superseded_present: true`, `…_at 2026-08-05 +15:03:14`), so the M4 branch — which is tested *before* M1 on the failure path — is live for every +failed unlock. That single fact shapes F1, F3 and F4 below. + +### F1 — a wrong recovery code, three times + +Injected: a ten-word hyphenated code that is neither real code, POSTed three times. + +| attempt | HTTP | elapsed | message | +|---|---|---|---| +| 1 | 200 | **1.194 s** | M4 | +| 2 | 200 | **1.004 s** | M4 | +| 3 | 200 | **1.014 s** | M4 | + +- **Refused each time** ✅ · **no lockout** ✅ (three identical responses; the handler documents + `NO LOCKOUT (§8.4)` by design) · **HTTP 200, no error page** ✅ +- **~1.0–1.2 s is the positive tell that a REAL unseal was attempted** — `age`'s scrypt cost, visible. + Phase 1's 0.134 s is what "no attempt was made" looks like. This discriminator is used throughout. +- **Nothing was written** ✅ — all four `/data/offbox` files byte-identical with **mtimes unchanged at + 14:57:35**: + +``` +846ac6e78a71e868 2026-08-05 14:57:35 applied_marker +75dbbc9be6909912 2026-08-05 14:57:35 known_hosts +e11a6c542b73477a 2026-08-05 14:57:35 repo_password ← the Phase-3 key, unchanged +85579eb0033832a0 2026-08-05 14:57:35 ssh_key +``` + +- **I4 held** ✅ — a sweep for three fragments of the entered code across the controller's data dir, + `/var/log`, `/tmp` and `docker logs` returned **product hits: 0**. **Both controls passed first**: + a planted canary was found by the same sweep, and `docker logs … | grep -c recovery` returned 6. + The only hits were in the harness's own script (`/tmp/.cc.sh`) — a self-match, cleaned up. + +**The deviation from the brief's expected answer: message 4, not message 1.** F1's stated right +answer is *"message 1 only"*. On this box M1 is **unreachable**, because the M4 branch precedes it and +fires whenever a superseded package exists. + +Judged on the message rather than the outcome: M4 is *hedged* („**Ha** egy korábbi kódot adtál meg"), +it does not accuse the customer of mistyping, and it states two true facts. But a customer who simply +**fumbled their current code** is given no hint to re-check their ten words and is instead routed to +customer support about *older* backups. **M1 — the only message that tells a customer to check their +typing — cannot be reached on any box that has re-escrowed**, which is exactly the population most +likely to be typing a recovery code. Recorded as a finding. + +### F2 — the customer has no code at all — **PASS** + +Two halves, both taken. + +- **The screen** states it plainly, unprompted: + > „**A helyreállítási kódot senki nem tudja pótolni** — sem a Felhom, sem az ügyfélszolgálat, sem az + > üzemeltető. Ez szándékos… Ha a kód elveszett, a korábbi mentések nem nyithatók meg többé." +- **It offers the set-aside path rather than a dead end** — „…**nem kérem vissza a korábbi adatokat**" + is rendered, because `CanSetAside` (= `OffboxOrphaned()`) is true here. ✅ +- **An empty submission** returns „**Add meg a helyreállítási kódot.**" in **0.027 s** — no unseal + attempted, and no blame attached to a code that was never given. ✅ + +### F3 — the hub unreachable when the code is entered — **FAIL** + +Injected on the appliance: `iptables -I OUTPUT 1 -d 37.191.56.193 -j REJECT`. +**Control: hub `http=302` before → `http=000, exit=7` after.** The fault is real. + +Then the **correct current recovery code** was entered, so the only possible cause of failure is the hub. + +``` +HTTP 200 · elapsed 0.0556 s ← far below the ~1.0 s scrypt cost: NO unseal was attempted +``` + +> **M4** — „Ez a kód nem nyitja meg azt a csomagot, amit most őrzünk ehhez a géphez. Ha egy korábbi +> kódot adtál meg: a géped azóta új mentési kulcsot kapott…" + +**The code was perfect and current. The hub was down. The customer is told their code does not open +their package and is sent to customer support about older backups.** The hub is never named. **I6 +breach** — an unreachable service reported as a fact about the code. + +**And the machine knew.** The agent's own journal distinguishes the two cases exactly: + +``` +F1 (wrong code): err="escrow: the recovery code did not unwrap the identity escrow + (wrong recovery code, or a corrupt blob): exit status 1" +F3 (hub down): err="escrow: fetching the sealed bundle: hub: transport error: + Get \"https://hub.felhom.eu/api/v1/hosts/c11-36d660/escrow\": + dial tcp 37.191.56.193:443: connect: no route to host" +``` + +The discriminator exists in the agent's `err` field and is **discarded at the HTTP boundary**: both +return **HTTP 400** with one merged sentence — *"the recovery code did not open the sealed bundle, +**or the bundle could not be fetched**"* — and the agent's own `msg=` collapses them too +(*"FAILED (wrong recovery code, or the blob could not be fetched)"*). The controller then has no +branch for "could not reach", so `rerr != nil` falls into the code/package messages. + +**Reverted**: rule deleted, chain empty, hub `http=302` again. + +### F4 — the agent stopped when the code is entered — **FAIL** + +Injected: `systemctl stop felhom-agent`. **Control: `is-active` = `active` → `inactive`, and no +listener on `:8443`.** Real. + +``` +HTTP 200 · elapsed 0.0299 s ← again no unseal +``` + +> **M4** — the same sentence as F1 and F3. + +**The stated right answer is M2** — *"names the machine's inability to ask, the R-216 fix under +pressure"*. M2 did **not** fire. The controller log names the truth the customer never sees: + +``` +recovery: unlock failed: agentapi: POST /escrow/recover-offsite-password: + Post "https://169.254.253.1:8443/escrow/recover-offsite-password": + dial tcp 169.254.253.1:8443: connect: connection refused +``` + +**Why the R-216 gate did not catch it, measured rather than reasoned** — the box's own debug ring: + +``` +[web] recovery capability gate: offsite_key_recovery=yes (source=version) +``` + +**`source=version`.** `recoverySupport` asks `netFeatures.SupportsWithSource(...)`, which answers from +the **known agent version** without probe traffic — by design, and correct for the question it was +built to answer. But the question it is being used for here is *"can this machine ask right now?"*, +and a version number cannot answer that. The gate discriminates **age**, not **reachability**; a dead +agent of the right version sails through it, and the failure that follows is attributed to the code — +the precise outcome R-216's comment says *"must never"* happen (*"An attempt that cannot succeed must +never be made, because its failure is attributed to the code"*). + +**F3 and F4 are one defect with two faces**: on the unlock failure path there is no branch for *the +machine could not ask or could not reach*, so every non-code cause is rendered as a statement about +the code or the package. This is Phase 1's headline finding, relocated from the version channel to the +transport. + +**Reverted**: agent `active`, `:8443` listening, `felhom-agent 0.125.0`. + +### F5 — the store unreachable after a successful unlock — **PASS** + +The first attempt's control **failed and is reported as a failed control**: the probe used `nc`, which +the controller container does not have, so blocked and unblocked both printed the same fallback. A +second attempt used the IPv6 address `getent hosts` returned — and the container has **no IPv6 route +at all**, so that too could not discriminate. The container resolves the store to **IPv4 +`91.98.242.176`**, and only a `/dev/tcp` probe on that address separated the states: + +``` +BEFORE: TCP-OPEN INJECT: iptables REJECT (OUTPUT + FORWARD) AFTER: TCP-CLOSED +``` + +Only then was the fault real. With the **correct current code**: + +``` +HTTP 200 · elapsed 1.198 s ← a REAL unseal +``` + +> „**A kulcs visszakerült**, de a mentések listáját most nem sikerült beolvasni. A mentéseid +> nincsenek veszélyben — nézd meg a Biztonsági mentés oldalt néhány perc múlva." + +- **The unlock still counts** ✅ · **M3, the "could not be read" shape** ✅ +- **R-217's false claim is genuinely absent** ✅ — verified in Python on the local file with UTF-8 + matching, *after* the accented-substring trap was accounted for (the project's own documented false- + zero hazard): + +``` +MUST BE ABSENT "A tároló megnyílt" 0 · "van benne tartalom" 0 · "nem tudtuk alkalmazásokhoz rendelni" 0 +CONTROLS PRESENT "A kulcs visszakerült" 1 · "nézd meg a Biztonsági mentés oldalt" 1 +``` + +**R-217's fix holds under exactly the fault it was written for.** + +**Reverted**: rules deleted, `TCP-OPEN` again. + +### F6 — „Most nem", then return — **PASS** + +Baseline first (the interception is on `/launcher` and `/dashboard`, not `/`): + +``` +BEFORE GET / -> 302 /launcher + GET /launcher -> 302 /recovery ← the full-page interruption + GET /dashboard -> 302 /recovery + GET /recovery -> 200 +``` + +`POST /recovery/postpone` → **302 → /launcher**, and: + +``` +AFTER GET /launcher -> 200 ← no longer interrupted + GET /dashboard -> 200 + GET /recovery -> 200 ← still reachable, permanently +``` + +**The backups-area entry point SURVIVES** ✅ — `/backups/remote` still carries exactly one `/recovery` +link and the sealed-package sentence, before and after: + +> „A korábbi, házon kívüli mentéseid visszaszerezhetők… **Adatok visszaszerzése**" + +That asymmetry — full page silenced, entry point permanent — is what the handler documents, and it +holds. **And the unlock still works after postponing** (measured: HTTP 200, 3.214 s, a real unseal). + + +### F8 — the controller restarted mid-unlock — **state PASS, message FAIL** + +Injected: an unlock started, then `docker restart felhom-controller` at **T+0.7 s**, inside the +~1.1 s unseal window. **Control: the container's `StartedAt` moved `2026-08-05T15:57:04Z` → +`2026-08-05T21:06:05Z`.** The restart really landed mid-request. + +What the customer got: + +``` +HTTP 502 · elapsed 1.914 s · body: "Bad Gateway" +``` + +- **No half-state** ✅ — after the restart the four `/data/offbox` files are byte-identical with + **mtimes still 14:57:35**, and `settings.json` differs from the pre-F8 snapshot in exactly one + field, `postponed: True`, which F6 set. The controller returned **healthy in 40 s**. +- **„the screen says which" — it does not** ❌. The customer is shown **`Bad Gateway`**: a raw + upstream error, in **English**, from traefik. It names no reason, offers no action, and says + nothing about whether the key was installed. **I3 breach** — *"every refusal names a reason a + person can act on, in Hungarian, with no raw error."* + +> **What F8 does NOT establish, stated so the pass is not overclaimed.** On this venue the unlock +> resolves to `RecoverUnchanged` (the recovered password is byte-identical to the one already on +> disk), so `InjectOffboxPassword` — the only writing branch — **was never reached**. F8 therefore +> shows that a mid-request restart leaves state coherent and returns a raw 502; it does **not** +> establish that the password *write* is atomic. Testing that needs shape (a), which the venue does +> not hold. Recorded as a gap, not as a pass. + +#### §4.1 again — and this time the box says it out loud + +The brief expected the floor positive here, and F8's restart delivered a better one than `/settings`: +a **cold-started process** stating the floor it was served, ten seconds in. The same log line, same +box, same code, before and after the vouch: + +``` +15:58:35 [offsite-apply] settle-gate: GO — floor still unknown after 1m30s ← hold IN FORCE +... +21:06:06 [offsite-apply] settle-gate: awaiting floor knowledge (first report ACK) +21:06:16 [offsite-apply] settle-gate: GO — at/above floor 0.200.0 (we are 0.201.0), + no managed update running ← hold RELEASED +``` + +**§4.1 is therefore measured twice, independently**: the rendered `GetFloor()` value, and this +box-side line from a fresh process that had no prior in-memory state to inherit. + +--- + +## The finding F6 turned up on the way past — the store reports 0 when it cannot read + +Not one of the eleven; found while checking F6's *"the listing is coherent"* clause on +`/backups/remote`. It is reported because a suspicion checked and confirmed is a result. + +The page renders, **on one screen**: + +``` +Tároló méret · 0 pillanatkép Tárhelykeret: 0 / 50 GB (0%) +``` + +and, immediately below: + +> „**A távoli tároló másik kulccsal készült mentéseket tartalmaz.** A távoli tárhelyen lévő mentések +> egy korábbi, már nem elérhető kulccsal készültek… **A meglévő mentések nem sérültek**…" + +**Ground truth, measured directly against the Storage Box over SFTP — a read-only listing, no +decryption, using the box's own transport credential:** + +``` +/home/felhom-repo/snapshots: + -rw------- 253 Aug 5 13:14 f3d9cd67d539c00359f0454ea7a782f45beac8406c55275bee5a6886afa8d791 +/home/felhom-repo: du -s → 12535 (KB) +/home/felhom-repo/keys: exactly ONE key +``` + +That is **snapshot `f3d9cd67`** — the Phase 0 snapshot holding all three sentinels, byte-size and +timestamp matching the journal's `repo_size_bytes 12 611 522` and its `13:13:32Z` capture. + +> **One snapshot and ~12.5 MB of the customer's only surviving copy, rendered as `0 pillanatkép` +> and `0 / 50 GB`, on the page the customer opens to find out whether their backups still exist — +> directly above a paragraph telling them the store contains backups.** + +**Mechanism, from the box's own state rather than inferred.** After the Phase 3 rebuild the +controller's `settings.json` `offbox` block carries **no `snapshot_count` and no `repo_size_bytes` +key at all** (the full block is quoted in the pre-flight section above). The values are *unknown*, +and unknown renders as the zero value. + +This is **R-217's defect class in a second location**: a field whose zero value is indistinguishable +from a real measurement, defaulted past on a failure/unknown path. `OffsiteInventory.Empty` exists +precisely because *"len(Apps)==0 is also what a failed read looks like"* — the same reasoning applies +to a snapshot count the box has never successfully taken. **I6**: an absence reported as a fact. + +**I5 checked and NOT breached** — the hub's `/offsite` page shows `Campaign 11 · 0.0 GB · 50 GB`, but +that is 12.5 MB rounded to one decimal of a GB, and the pool total (`Used 3.8 GB`) is consistent. The +hub and the box do not disagree; **both** understate, for different reasons. Only the box's +**snapshot count** — an integer, not a rounding — is false. + diff --git a/hub/CHANGELOG.md b/hub/CHANGELOG.md index 96abf38..9477e7b 100644 --- a/hub/CHANGELOG.md +++ b/hub/CHANGELOG.md @@ -1,3 +1,31 @@ +## v0.97.1 — the held-floor REASON must match the hold's CAUSE (2026-08-05, CAMPAIGN-11 follow-on) + +> **Entry added retroactively 2026-08-05 (CAMPAIGN-11 Phase 2).** The change shipped and **is** the +> deployed image (`felhom-hub:0.97.1`, commits `a7f1d27` + `79e31ac`), but it was written into the +> **v0.97.0** entry instead of its own — so a reader matching the running tag against this file found +> no `v0.97.1` heading at all. Second occurrence of a version deployed without its own entry (the +> first was agent `0.90.1`). Nothing about the change is new here; only its heading is. + +Adding a second hold reason and leaving both surfaces printing the first is `CLAUDE.md`'s corollary, +and it happened here: the freshly deployed hub logged + +``` +managed floor HELD for c11: agent "0.125.0" < MinAgent 0.113.0 +``` + +— a comparison that is **FALSE** (0.125.0 is *above* 0.113.0) for a box held because its floor sat +above the vouched golden. Both the ACK log line and the Hosts-dashboard `HeldReason` now come from one +`ManagedFloorDecision.HoldReason()`, with `RequestedFloor` kept so the message can name the floor that +was asked for (`Floor` is cleared to `""` on a hold, and a reason naming an empty floor tells the +operator nothing). `TestResolveManagedFloor_HoldReasonMatchesTheCause` pins each reason to its cause. +**A true alarm that misdescribes itself is one the operator learns to dismiss.** + +**Confirmed live in CAMPAIGN-11 Phase 2**: the corrected sentence is what the hub logged for c11 every +15 minutes until the golden rebake released the hold — +*"held: floor 0.200.0 is ABOVE the vouched golden 0.192.0, so its agent requirement is unknown — vouch +a golden carrying the floor's controller (publish-train rule 1)"* — and it stopped, as it should, once +golden 0.201.0 was vouched. + ## v0.97.0 — the floor stops being served past the agent it depends on (2026-08-05, CAMPAIGN-11) **R-216, the hub half.** `ResolveManagedFloor` exists to *"never push a controller past the agent it @@ -41,14 +69,6 @@ either way. **Only rows that actually carry an identity blob count**: the pre-v0 retain nothing the screen could be talking about. It is a boolean and a timestamp; it grants **no read path**, which is still unbuilt (R-199's inventory). -**And the hold REASON now matches the hold CAUSE.** Adding a second reason and leaving both surfaces -printing the first is `CLAUDE.md`'s corollary, and it happened here: the freshly deployed hub logged -`managed floor HELD for c11: agent "0.125.0" < MinAgent 0.113.0` — a comparison that is FALSE -(0.125.0 is above 0.113.0) for a box held because its floor sat above the golden. Both the ACK log -line and the Hosts-dashboard `HeldReason` now come from one `ManagedFloorDecision.HoldReason()`, and a -test pins each reason to its cause. A true alarm that misdescribes itself is one the operator learns -to dismiss. - Tests: `managed_floor_test.go` — the campaign's exact numbers, held-regardless-of-agent, Scenario C (uncoupled untouched), floor==golden unchanged, unparseable golden. **Red-proof: removing the floor-above-golden branch reproduces the campaign's measurement exactly.**