felhom.eu/documentation/audits/SESSION-C-2026-07-29.md

SESSION-C-2026-07-29 — R-113, R-114, R-112 proven on a real box; C5 fails on a new defect

Run: RUNBOOK-session-c-2026-07-29.md, executed by CC on DooPlex, 2026-07-29. Session C of three. A → R-113 (agent v0.114.0) · B → R-114 + R-112 (controller v0.186.0) · C → this.

Claim	Fix	Verdict
C4 — the offer appears and moves the target when accepted	R-112	✅ PASS
C3 — with the target absent the customer is told the drive is missing, and offered nothing	R-114	✅ PASS
C5 — drive loss produces backup_target_absent (error) and backup_target_restored (info)	R-113	❌ FAIL — the alarm fires, but as the generic event; the recovery is the specific one. Unmatched pair → R-116

All three fixes are proven to work. R-113's gate fires, R-114's message is correct, R-112's banner reaches the customer. C5 fails on a fourth, separate defect that only became reachable because R-113 made the gate fire at all — it could not have been seen before.

---

1. Baselines, and what the drill box actually ran

Artifact	Vouched / used	How it got there
agent	0.114.0	from the Day-0 manifest — Artifact manifest served for sess-c (agent=0.114.0 golden=0.185.1). C5 therefore tested the shipped binary, closing R-115 observation 1 for R-113
golden	0.185.1	vouched; bakes controller 0.185.1
controller	0.186.0	hand-deployed after install (§3.1 ruling (a)) via the bootstrap path — 0.185.1 → 0.186.0, Up 21 seconds (healthy)
hub	0.81.0	unchanged
host-install	1.22.0	unchanged

Repos at run time: controller b331f18, agent b58d7bc, felhom.eu af518ba, all clean.

§3.1 ruling — (a), deploy 0.186.0 by hand. A bake is its own session and would have inserted one before any verdict; the install path is already proven by E-2d. Caveat, as required: C3/C4 prove the CODE, not the SHIPPED GOLDEN. The vouched golden lagging a controller release is R-115's shape and is filed there, not as a new ID.

§3.2 cadence — driveGateLoop registered server.go:224, defined intermediary.go:328, ticker 30 s (:339) ⇒ C5 budget 60 s. §3.3 — SCSI hot-detach, volume survives as unused0.

Fence re-measured before and after: local-lvm 38.78 % both times.

2. Timeline (VM 9301 sessionc, nested PVE on demo-hp)

UTC	Event
18:29	VM created — one disk, q35/OVMF SB-off, hotplug disk, outside the felhom pool
18:33:47	registered unclaimed, pairing code 2XT-3PV
18:34:08	bound to sess-c by CC — no operator STOP (E-2d retired it)
18:34:57	manifest served: agent 0.114.0, golden 0.185.1
18:38:03	controller_started (0.185.1) — install complete, bind→controller in 3 m 55 s
~18:40	controller 0.186.0 deployed; dashboard claimed with the emailed code
18:43:50	target drive hot-detached
18:43:54	gate fired — 4 seconds
18:47:01	reattached
18:47:24	gate RETURNED (23 s)
18:50:32	non-target drive detached (mirror)
18:50:54	gate fired (22 s)

3. C4 — PASS (R-112 + the assign flow)

The banner reached a customer's page for the first time. E-2d could only prove the JSON.

Never-configured box, before any second drive:

GET /backups http=200
  banner element        : 1
  never-configured copy : 1      <- "ugyanazon a lemezen van" IS in the HTML
  absent copy           : 0
  offer block           : 0      <- correct: no eligible drive registered yet

After attaching a 100 GB disk and bringing it in through the dashboard wizard:

  banner element : 1     offer block : 1     data-path="/mnt/felhom-drives/mentes"

Decline path — PROVEN. After registering and not accepting: target still local, no felhom-backup storage, agent.json local_backup_target = local. Registration confers no role.

Accept: {"assigned":"/mnt/felhom-drives/mentes","restart_required":true}. The agent did NOT self-restart — ActiveEnterTimestamp 20:35:15 CEST, unchanged at 20:42:20. In-flight check performed and recorded before restarting: 0 vzdump processes, 0 backup lines in the agent journal (5 min), 0 running PVE tasks. Wrapper created dir: felhom-backup, path /mnt/mentes, is_mountpoint 1 — the drive's own mountpoint.

After restart, healthy renders nothing — and it is proven POSITIVELY, not by an empty page:

  banner element 0 · offer block 0 · degraded copy 0 · absent copy 0
  /backup/tiers calls  before 6 · after idle 8s 6 (delta 0) · after page load 7 (delta +1)

That call has exactly one caller (resolveBackupTargetState), so the seam ran and chose silence.

4. C3 — PASS (R-114)

With the target drive detached, the customer's page:

  banner element        : 1
  ABSENT copy (correct) : 1     <- "vissza nem csatlakoztatod"
  system-disk copy (BAD): 0     <- E-2d saw THIS
  offer block (BAD)     : 0     <- E-2d offered the VANISHED drive

API: {"degraded":true,"target":"felhom-backup","message":"A rendszermentés meghajtója nem érhető el — amíg vissza nem csatlakoztatod, a teljes rendszermentés nem készül el."}

Both of E-2d's falsehoods are gone.

5. C5 — FAIL, and R-113 is nonetheless PROVEN

R-113 works. This is the headline. E-2d measured zero gate activity over 4½ minutes. Here:

18:43:54 [INFO] [settings] Storage path /mnt/felhom-drives/mentes disconnected=true
18:43:54 [WARN] [gate] drive ABSENT /mnt/felhom-drives/mentes — stopped+blocked 0 app(s): []

4 seconds, and SetDisconnected reached — the flag R-114 and the offer suppression depend on. And it fired on exactly the shape that defeated it before:

raw   /mnt/mentes               : NOT mounted        <- device-bound, died with the device
bind  /mnt/felhom-drives/mentes : /dev/sdb[/felhom-data]   <- the stale bind SURVIVED

The bind outlived the device, as always; the gate now sees past it.

What fails is which event is sent.

absent : Event pushed: storage_disconnected (error)      <- GENERIC
return : Event pushed: backup_target_restored (info)     <- SPECIFIC

C5 required backup_target_absent. It never fired — backup_target_absent count 0 across the run. Worse than a missing label: the alarm and its recovery are a mismatched pair, which is precisely what notifyDriveReturned's own comment forbids — "a target that alarmed as backup_target_absent has to recover as backup_target_restored, not as a generic reconnect the operator cannot match". Here it is inverted: generic alarm, specific recovery.

Root cause, established at source

notifyDriveAbsent (intermediary.go:635-646) branches on isTarget[path], built by driveTargetByPath (:602-616) as out[GuestPath] = d.BackupTarget / out[MountPath] = d.BackupTarget. The gate's a.Path is the guest path.

The drive is represented by two /disks rows, and the flag and the guest path sit on different ones:

Row	Source	BackupTarget	GuestPath
the felhom-backup storage	Observe path, felhom-agent/internal/localapi/disks.go:211 (t.Name == primaryTargetID)	true	set only when role is user-data
the registry drive	union path, disks.go:265-267 — never assigns BackupTarget at all	false	/mnt/felhom-drives/mentes

While the device is absent the target-flagged row loses the classification that gives it a guest path, so out["/mnt/felhom-drives/mentes"] is written false by the union row and the specific branch is skipped. On return the rows rejoin and the specific event fires — which is exactly the asymmetry observed. v0.184.1 fixed the keying (both keys are mapped); it did not fix the flag being absent from the row that carries the guest path.

(The precise role-classification step that drops the guest path while absent was not isolated — the two-row split and the observed absent/return asymmetry are the evidence.)

Filed as R-116. Not fixed: a defect found in a validation run is a finding.

6. Mirror scenario + over-correction guard — PASS, with a caveat

A non-target drive (adat, 70 GB, registered but never assigned) detached at 18:50:32:

18:50:54 [WARN] [gate] drive ABSENT /mnt/felhom-drives/adat — stopped+blocked 0 app(s)
18:50:54 [INFO] Event pushed: storage_disconnected (error) — Meghajtó váratlanul leválasztva: Adat meghajto
  backup_target_absent count : 0     storage_disconnected count : 1

Over-correction guard — the real risk of R-113 — PASSES. With both drives present the gate produced 0 ABSENT lines, and the target stayed degraded:false with banner element 0 throughout the whole mirror sequence. R-113's stricter presence did not make a healthy drive read absent.

Caveat, stated rather than glossed: because the target drive also produced the generic event, the mirror passes trivially. It confirms no over-correction; it cannot confirm specific-vs-generic discrimination, because nothing produced the specific event on absence at all.

7. Findings

1. R-116 (new) — the absent alarm is generic while the recovery is specific; the pair cannot be matched by an operator. Root cause §5. This defect was unreachable before R-113: the gate never ran, so the branch was never taken. Fixing one bug exposed the next, which is what a validation run is for.
2. The stale bind survives a return, still naming a dead device node. After the drive came back as /dev/sdc, findmnt /mnt/felhom-drives/mentes still read /dev/sdb[/felhom-data] while the state reported degraded:false. E-2d noticed this and did not chase it; it is now confirmed. Whether apps writing through that path reach the live device was not established — worth its own look.
3. The capability map still has no E-2 / backup-target rows at all, so no row could be moved to PROVEN-LIVE. Creating them is a design act, not a validation act; filed rather than invented (third session in a row this has been noted).

8. Teardown

VM 9301 destroyed --purge, sc-images storage removed, scratch dir removed. pvesm status after == before (local-lvm 38.78 %, felhom-backup 931059224 KiB available). Guest 9201 and drill-r50 untouched.

Customer ruling: DELETE (R-93's lesson — no half-real fixtures). Attempted and correctly refused: delete sess-c REFUSED: host sess-c-8158e4 is ONLINE. The VM is destroyed so no further reports come; the host ages OK → WARN (30 m) → DOWN (>1 h) and is then deletable. Command, unchanged from E-2d:

POST /configs/sess-c/delete  ack_hosts=1 ack_reset=1 ack_purge=1 confirm_id=sess-c expect_hosts=1

9. What did not happen

• backup_target_absent never fired, so its severity, its Hungarian customer copy and its hub routing are all still unexercised end-to-end. R-116 blocks them.
• The offer control was accepted via the endpoint it POSTs, not by a browser click — no browser automation exists on DooPlex. The rendered control, its data-path and its non-auto-submission were verified in the HTML.
• The stale-bind-after-return question (finding 2) was observed, not investigated.