# OPEN OBSERVATION — the `backup_target_*` pair stopped firing under rapid detach/reattach cycling

**Not filed as a finding yet.** The system was still being hammered when this was observed, and a
verdict taken on a system mid-abuse is worth little. The disposition below says exactly what would
settle it.

## What was observed (2026-08-02, run 2b, cycles ~30-32)

Three violations in ~5 minutes, all of the same shape — the expected event simply absent:

    04:36:55  I1-pair  mentes return -> got=None sev=None   (want backup_target_restored/info)
    04:39:27  I1       mentes absent -> got=None sev=None   (want backup_target_absent/error)
    04:41:42  I1-pair  mentes return -> got=None sev=None   (want backup_target_restored/info)

## It is NOT hub-side suppression, and NOT a truncated log

The hub pod has **0 restarts over 43 h**, so its log is intact, and the controller's own log matches
the hub's line for line — the events were **never emitted**, not emitted-and-dropped:

    controller  02:32:34  Event pushed: backup_target_absent (error) — … Mentes …
    controller  02:32:38  Event pushed: whole_guest_backup_failed (error) — … felhom-backup tier …
    controller  02:35:32  Event pushed: health_degraded (warning) …
    controller  02:42:08  Event pushed: storage_disconnected (error) — … Adatok      <-- generic path STILL WORKS
    (no backup_target_restored, and no second backup_target_absent, at any point after 02:32:34)

So this is controller-side emission, and it is **specific to the `backup_target_*` pair**: the generic
`storage_disconnected` / `storage_reconnected` pair for the *other* drive kept firing normally
throughout the same window (02:16:02, 02:18:21, 02:27:38, 02:29:35, 02:42:08).

## The internally inconsistent state alongside it

Sampled while it was happening:

    backup-target:  {"degraded": false, "known": true, "label": "Mentes", "target": "felhom-backup"}
    drives:         adatok  bound_under_parent=False  backup_target=None
                    mentes  bound_under_parent=False  backup_target=True

`mentes` is the backup target, reads **not bound**, and the target state simultaneously reports
**`degraded: false`**. Those two cannot both be right. If that survives quiescence it is the more
serious half of this observation — a **false healthy** on the backup target, which is I5/I6's
failure mode and worse than a missing event.

## What is NOT yet established

- Whether the pair recovers once the cycling stops. The harness detaches and reattaches roughly every
  two minutes; a real customer does not. Rapid cycling may be outside the design envelope, in which
  case this is a robustness note rather than a defect.
- Whether the controller restart at 02:25:37 is implicated. In this controller instance there was
  **1 `backup_target_absent` and 0 `backup_target_restored`**; the previous instance emitted complete
  pairs (02:13:21 restored). That is suggestive, not established.
- Whether the `degraded: false` reading was a transient mid-flux sample or a stuck state.

## Disposition — how to settle it

After run 2b ends, let the box **quiesce with both drives attached**, then:

1. confirm `bound_under_parent` returns to `True` for both drives and the target still reads healthy —
   if it does not, the false-healthy is real and this becomes a finding;
2. do **one** slow detach/reattach of `mentes` with several minutes either side, and check the pair
   fires. If it does, the observation is "the pair does not survive rapid cycling"; if it does not,
   the target alarm has silently stopped working, which is severe — an alarm that never fires again
   after its first firing is worse than no alarm.

Only then does it earn an `R-n` or a retraction.

---

# RESOLVED (2026-08-02) — transient, self-recovered. NOT filed.

Both halves of the disposition were run. Neither survived.

## 1. It self-recovered, mid-run, with no intervention

The violations are **clustered at cycles 31-33 and nowhere else** across 39 cycles:

    c28 I1 PASS   c28 I1-pair PASS
    c29 I1 PASS   c29 I1-pair PASS
    c30 I1 PASS   c30 I1-pair PASS
    c31 I1 PASS   c31 I1-pair VIOLATION
    c32 I1 VIOLATION   c32 I1-pair VIOLATION
    c33 I1 VIOLATION   c33 I1-pair VIOLATION
    c34 I1 PASS   c34 I1-pair PASS      <-- recovered on its own
    c35..c39 all PASS

Final tally **I1 37 PASS / 2 VIOLATION, I1-pair 36 PASS / 3 VIOLATION** — a ~3-cycle window inside a
13.5-hour run of sustained 2-minute detach/reattach cycling.

## 2. A slow cycle on the quiesced box works perfectly

With the run finished and the box quiet, one detach with 4 minutes either side:

    07:00:08  backup_target_absent  (error) — A rendszermentés meghajtója nem érhető el: Mentes
    07:04:42  backup_target_restored (info) — A rendszermentés meghajtója újra elérhető: Mentes
    07:06:37  health_recovered      (info) — Rendszer állapot helyreállt: ok (volt: warn)

## 3. The "false healthy" was a mid-flux sample, not a stuck state

The alarming pair (`mentes bound_under_parent=False` **and** `degraded: false`) does not survive
quiescence. Three minutes after the run ended:

    drives:  adatok bound=True          mentes bound=True  backup_target=True
    target:  {"degraded": false, "known": true, "label": "Mentes", "target": "felhom-backup"}

Consistent. I was sampling a system mid-detach and reading the two halves at different instants.

## Verdict

**No `R-n`.** The target alarm does not fire during a narrow window under detach/reattach cycling far
faster than any real box sees, and it resumes by itself. Recorded so a future session that sees the
same three-cycle cluster knows it was chased to the ground and what settled it.

---

# Also investigated and cleared: the hub's SQLITE_BUSY event drops

Chased during the same test and **also not a finding**, recorded because the log line is alarming:

    [ERROR] Failed to save event from c10-soak: database is locked (5) (SQLITE_BUSY)

7 occurrences in 24 h, **including one for the real customer `demo-felhom`** — so it is not a
campaign artifact. The hub returns HTTP 500 on the failed save (`handler.go:1715`) and dispatches
notifications only *after* a successful save, so a genuinely lost event would be a lost alarm.

**But the controller retries 3 times**, and **0 events exhausted their attempts** across the whole
run (`grep -c "Event push failed after 3 attempts"` = 0). The 07:04:39 drop landed 3 seconds later:

    07:04:39 [ERROR] Failed to save event … SQLITE_BUSY
    07:04:42 [INFO]  Event from c10-soak: backup_target_restored (info) …   <-- same event, retried

**Nothing was lost.** The one observation worth keeping is cosmetic: `[ERROR] Failed to save event`
reads like data loss and is not — a reader would reasonably conclude an alarm had been dropped. The
same contention also skipped monitor ticks (Host OOB, Offsite, restore-test, log-bundle purge), which
are periodic and simply run again.
