# REPORT — v0.117.0: R-117, the liveness signal now tests liveness (2026-07-30) **Overwritten** per the standing rule. Full evidence: `felhom.eu/documentation/audits/R117-v0117-2026-07-30.md`. Spike this builds on: `audits/SPIKE-r117-bind-liveness-2026-07-30.md`. | | | |---|---| | agent | 0.116.0 → **0.117.0** @ `966d8f4` | | published + vouched | **0.117.0**, sha `e6768a12eba1c9e0…` (anonymous GET verified, manifest read back) | | deployed | **demo-hp 0.117.0** (from **0.113.0** — see R-121) · felhom-pve still **0.115.0**, out of scope | | controller / golden / hub | 0.186.0 / 0.186.0 / 0.82.0 — **untouched** | | tests | **849 → 863**, 29/29 packages, `build`/`vet`/`test` each `rc=0` read separately | ## The defect `BoundUnderParent` reported a namespace returning `EIO` on every read and write as healthy, and the gate restarted the customer's apps onto it. Both existing terms — `GuestSeesMount` and `isHostMountpoint` — parse a mountinfo line and then test only `fields[4]`, the mount POINT. **Field 3, `major:minor`, sat in the same parsed slice and was discarded.** ## The fix — one predicate, both halves mandatory `bindLiveness` (`internal/localapi/intermediary.go`) becomes the third term of the conjunction at both `/disks` construction sites. `/proc` only, **no block I/O**: - **the bind must name the same device as the raw mount** — sound, not heuristic: a stale bind pins the dead superblock, which keeps the old device index allocated, which forces the returning device onto a different number; - **the filesystem must not have aborted** — `shutdown` or `emergency_ro`, both measured. **The second is not optional.** A device that fails *without disappearing* gives the identical all-signals-healthy state with the **devnos equal** and the drive never `Disconnected`, so the gate emits nothing at all, indefinitely (R-117a). A devno-only fix passes every payload test; red-proof RP3 exists for exactly that. **Three states, never a bool:** `{Unknown, Live, StaleDevice, Aborted}`, `Unknown` the zero value, and `Usable()` is the single place where unknown counts as **present** — absent stops a working customer's apps. ## The ruling (task §2.2) — and it is flagged for overrule `StaleDevice` ⇒ **re-bind** (the raw mount is a healthy new superblock; repairs live, guest never restarts). `Aborted` ⇒ **quiet no-op, and surface via `BoundUnderParent=false`** — a re-bind lands on the *same* dead superblock and this runs every 20 s, so repairing would be an infinite silent retry that also masks the state. **No operator decision taken quietly:** it does change customer-visible behaviour (apps stop and an alarm fires where today nothing happens), but it routes an already-broken state into the **existing** gate, event types and Hungarian copy — no new concept reaches the customer — and the alternative is applications writing documents into a filesystem that rejects every write. **No new recovery path.** `AttachDrive`'s normalize leg already did the repair and three call sites already invoked it (20 s ticker, agent startup, and the controller's `Return` branch **before** `restartStacks`). All three died on `if n == 1 && GuestSeesMount(...)` logging *"fully live, no-op"* about a dead namespace. **Ordering trap, caught by a test:** reading the abort flag before comparing devices classifies the real return state as aborted (its stale bind carries `shutdown` too) and refuses the repair **while still reporting correctly** — so the abort flag is read off the RAW mount in the stale case. ## Live validation, demo-hp | State | raw | bind | verdict | usable | |---|---|---|---|---| | healthy | `8:16` | `8:16` | `live` | true | | **R-117 return** | `8:32` `/dev/sdc` | `8:16 …,shutdown` | **`stale-device`** | **false** | | **Q7 in-place abort** | `252:11 …,emergency_ro` | `252:11 …,emergency_ro` | **`filesystem-aborted`** | **false** | 340–497 µs per call. **No block I/O proven by `strace`**: only `/proc/self/mountinfo`, **0** `statfs`. **No regression through the real pipeline:** `GET /disks` with the controller's own credential shows the live backup-target drive `bound_under_parent=True`, with 32 gate lines in 3 min as the positive observable and zero spurious transitions. ## Tests + red-proofs 14 new tests; the payload ones drive the real `/disks` handler and the `AttachDrive` ones record the actual privileged calls. Fixtures are the spike's captured mountinfo, injected as **data** via `procSelfMountinfo`/`procGuestMountinfo`, so the real parser, predicate, `AttachDrive` and handler all run. **6 red-proofs, each verified to have landed.** RP1/RP2 (term 3 removed per site), RP3 (**the P1-only fix**), RP4 (`emergency_ro` dropped), RP5 (unknown reported dead), RP6 (aborted arm falls through to a re-bind). **A hollow test was caught, by RP1 failing to fail.** The aborted fixture first used a `/dev/mapper` device, for which `RoleForStorage` derives `role="system"` — a system row never enters the block that computes `BoundUnderParent`, so the field stayed false **by default**, the assertion passed, and no mutation could ever fail it. Fixtures now assert the production row shape before asserting the field. ## Not yet done / not covered - **The stale-bind repair on hardware.** `StablePathForRaw` hardcodes the production `/mnt/felhom-drives`, so exercising `AttachDrive` for a scratch drive writes a submount into guest 9201's namespace, which the task fences. It happened once inadvertently, was reverted within a minute and 9201 verified unaffected, and was deliberately not repeated. Filed **R-117h**. - **Sustained-load behaviour of the predicate** — flagged unmeasured by the spike and **still unmeasured**. - **felhom-pve not upgraded** (0.115.0). The fleet is non-uniform. - **R-117g** — an aborted filesystem is never cleared automatically, by design, so it alarms until a human acts; there is no guided recovery. - **R-121** — a box's installed agent can sit releases behind the vouched one and nothing notices; the R-120 gate is golden-vs-fleet-controller and cannot see it. ## Observations (not acted on) Guest 9201 has 2 stacked entries at `/mnt/felhom-drives/nvme-1tb` while the host has 1 (pre-existing; `AttachDrive` normalizes the host count only). 14 stale `felhom-agent.bak-*` binaries on demo-hp (~200 MB), nothing prunes them. 11 files in this repo fail `gofmt -l`, all pre-existing and none in `internal/localapi/`.