felhom.eu/REPORT-r101.md

# REPORT — R-101 + F-DIAG + F-OPS (2026-07-28)

Controller **v0.181.0 → v0.182.0**; `felhom.eu` gains the manual-restore runbook (F-OPS) and the
OPEN-ITEMS rows. Written as `REPORT-r101.md` so the shared `REPORT.md` is not clobbered.

## Baselines (reconfirmed, not copied)
`felhom-controller 3db8bfb`, `felhom.eu 6b7d516`, `felhom-agent d5c7691` — all = origin/main; the only
dirt in `felhom.eu` was a **foreign** `PROMPT-TEMPLATE.md`. Controller **0.181.0** live on both boxes,
hub `felhom-hub:0.80.0` ready 1/1.

---

## Phase 0

**The render sites — three dishonest, two already honest.** The spec listed `backups_apps.html:216`
as a defect site; it is in fact the one branch that *already* pairs its timestamp with a status badge.
The real third site is the `Tier2DestInactive` branch.

| site | rendered | honest? |
|---|---|---|
| `:231` **restore confirm dialog** | `Legutóbbi másolat: {{.Tier2LastRun}}` — raw RFC3339, no status | **NO** — the one that matters |
| `:195` `Tier2DestDisconnected` | `Utolsó: …`, no status | **NO** |
| `:206` `Tier2DestInactive` | `Utolsó: …`, no status | **NO** |
| `:217` main configured branch | `Utolsó: …` **+ status badge** | already honest |
| `sharing.html:181` | rendered **only** when status=="ok" | already honest |

`Tier2LastStatus` was already set unconditionally at `handlers.go:1182`, so this was a wording/anchor
problem, not a plumbing one. The restore button was gated on `{{if .Tier2LastRun}}`, so **Scenario C
was live-reachable**: a tier that had attempted and never succeeded offered a restore and a timestamp.

**`cd.LastRun` is written on failure** — `recordTier2Failure` (`tier2.go:573-574`) writes it alongside
`LastStatus:"error"`. Identical shape to R-100.

**Legacy state is universal, not an edge case.** All 7 Tier-2 rows across both boxes had `last_run` and
no anchor. Scenario E was therefore the *initial state of every customer*, which is what made the
legacy marker non-optional.

---

## Part 1 — the strings shipped

| case | string |
|---|---|
| dialog, normal | `… Legutóbbi sikeres másolat: 2026-07-28 16:43.` |
| dialog, newest attempt failed | `… Legutóbbi sikeres másolat: 2026-07-28 16:40. Figyelem: a legutóbbi mentési kísérlet nem sikerült, ezért a visszaállított fájlok ennél régebbiek lehetnek.` |
| card | `Utolsó sikeres: 2 perce` |
| never succeeded | `Még nincs sikeres másolat` + `Még nincs sikeres másolat, amiből vissza lehetne állítani.` (restore removed) |
| **legacy row** | `Utolsó: …` / `Legutóbbi másolat: …` — **today's wording, unchanged**, logged once per stack |

**Timestamp made human-readable** (agreed): new `fmtTimeStr` renders Budapest-local `2026-07-28 16:40`
instead of the raw UTC `2026-07-28T14:40:55Z` a customer was previously asked to reason about.

**`SuccessTracked` is what makes the legacy case possible at all.** Without it, "row predates the
anchor" and "row has an anchor and it is empty" are indistinguishable — both are `LastSuccess==""` —
and every existing row would have rendered as never-succeeded on deploy. Legacy rows migrate on first
touch: a row whose last known state was `ok` adopts that time (truthful — under the old code that run
did succeed); a row whose last state was `error` seeds **nothing**, because the old data evidences no
success.

## Part 2 — the copy-site hazard, and it was in the path

The three `record*` helpers each built a **whole `CrossDriveBackup` literal**, with a helper re-applying
exactly two fields; everything else was zeroed on every status write. Adding `LastSuccess` to that shape
would have had `recordTier2Failure` **clear** it — the mirror image of the defect, firing on the *first*
failure rather than lying dormant.

Replaced with **`tier2Update`**, which copies the existing row and overlays the outcome: **compile-safe
by construction** — a new field carries over unless deliberately overwritten, so nothing is preserved by
a list that can fall out of date. Callers now clear explicitly what a run invalidates, reproducing the
old behaviour exactly.

**Sweep of other rebuild sites:** `SetTier2Preference` mutates in place (safe); `SetCrossDriveConfig(name, nil)`
in `api/router.go:774` is a deliberate delete. No others.

## Part 3 — F-DIAG

| class | signal it maps to | message head |
|---|---|---|
| `quota` | the pre-run soft-quota gate | `A távoli mentés nem fért el a tárhelykereten belül` |
| `orphaned` | `ErrOffboxOrphaned` sentinel | `A távoli tárhely egy korábbi, már nem elérhető kulccsal készült` |
| `no_repo` | restic "unable to open config file" | `A távoli tárhelyen nincs mentési adattár` |
| `no_units` | "produced no snapshots" | `Nem volt mit menteni: egyetlen kijelölt alkalmazásnak sem található mentése` |
| `transport` | refused/reset/timeout/authn/host-key | `A távoli tárhely nem érhető el (hálózat vagy bejelentkezés)` |
| **`unknown`** | anything else | `A távoli mentés ismeretlen okból nem sikerült` |

The `unknown` class is deliberate: a cause that cannot be told apart where the error is produced is
reported as unknown rather than folded into a neighbour.

**Secrets — and this caught a bug in my own first attempt.** The old message was
`"…: " + err.Error()`, carrying the repo reference `sftp:<user>@<host>:<path>` off the box. My first
sanitiser regex-matched `sftp:…` and `user@host` and *looked* complete; its own test caught it leaking
on `ssh: connect to host <host> port 23: Connection refused` — a bare hostname in neither shape. It now
redacts the target's **actual** host/user/repo-path literally, with the regex kept only as a backstop.
Guessing at what a secret looks like fails exactly where it matters.

## Part 4 — F-OPS

`documentation/runbooks/RUNBOOK-manual-guest-restore.md`. Grounded in the real bind shape read off live
guest 9201, not written from memory. Covers: which `mpN` are storage volumes (restored) versus **host
binds** (taken as-is on the target); the `mp9` trap — it embeds the **source** VMID, so restoring to a
different VMID can bind **another guest's bootstrap credentials**; strip-and-re-add before first boot;
the hookscript check; and a positive pre-start verification that asserts every bind path exists rather
than accepting "no error". Docs only, by design.

---

## Red-proofs — all observed failing

| # | red-proof | observed failure |
|---|---|---|
| A | dialog back on the attempt clock | `the dialog does not name the last SUCCESSFUL copy` |
| C | gate the restore on `LastRun` again | `a tier that has NEVER succeeded still offers a restore — the dialog would promise a copy that does not exist` |
| D | make the caution unconditional | `a HEALTHY tier shows the failed-attempt caution ("nem sikerült")` |
| F | clear the anchor on failure | `a FAILED run wiped the success anchor (round 1) — one bad night would read as 'no copy has ever succeeded'` |
| + | raw sanitiser | `the repo reference reached the message ("sftp:" leaked)` |

**F exercises the real `recordTier2Success` → `recordTier2Failure` sequence**, not a modelled copy — the
R-100 lesson. The Scenario A/C/D tests **render the production template tree** and assert on the string
the customer reads; a test asserting a template variable would prove nothing about wording, which is
the defect.

`go build`, `go vet ./...`, `go test ./...` — 27 packages, `rc=0`; `template_id_gate.py` and
`emoji_gate.py` both OK. Run separately from every commit.

---

## LIVE on demo-hp — the rendered dialog, which is the deliverable

**Legacy state** (before any run under v0.182.0) — today's wording, no fright:
```
Legutóbbi másolat: 2026-07-28 03:30
```

Failure induced genuinely: the Tier-2 destination directory was **moved aside** and replaced by a file,
so `mkdir …/recovery-unit` fails. (`chmod` does not work — the controller runs as root, which bypasses
permission bits; `chattr +i` is refused, the unprivileged container lacks `CAP_LINUX_IMMUTABLE`. Both
were tried and reported rather than glossed.) The real data was only ever moved, never deleted.

```
status       = error
last_run     = 2026-07-28T14:42:18Z     ← ADVANCED
last_success = 2026-07-28T14:40:55Z     ← HELD
last_error   = mkdir …/paperless-ngx/recovery-unit: …
```

**The rendered dialog, failed state:**
```
Visszaállítja a hiányzó fájlokat a másodlagos másolatból? A meglévő fájlok NEM módosulnak és NEM
törlődnek. Az alkalmazás a művelet idejére leáll. Legutóbbi sikeres másolat: 2026-07-28 16:40.
Figyelem: a legutóbbi mentési kísérlet nem sikerült, ezért a visszaállított fájlok ennél régebbiek
lehetnek.
```

**The rendered dialog, healthy state** (after restoring the destination and a successful run) — no
caution, no tonal change:
```
Visszaállítja a hiányzó fájlokat a másodlagos másolatból? A meglévő fájlok NEM módosulnak és NEM
törlődnek. Az alkalmazás a művelet idejére leáll. Legutóbbi sikeres másolat: 2026-07-28 16:43.
```

Card lines: `Utolsó sikeres: 2 perce` → `Utolsó sikeres: most`.

**Everything restored:** destination is a directory again, 86 MB intact, mode 755, `.r101-aside` gone,
`status=ok`, `last_success=2026-07-28T14:43:23Z`.

**demo-felhom is the untouched control:** all 5 rows still `tracked=None` after the deploy, rendering
today's way, 15/15 containers up. Scenario E holding across a whole box nobody ran.

---

## NOT yet live-validated (carried forward)
- **F-DIAG's classes** — unit-proven only. No live offsite failure of each class was induced; the
  `transport` class is the only one this arc exercised indirectly.
- **Scenario C live** — the never-succeeded rendering is unit-proven; no fleet row is in that state
  (every row either migrated or has a real success), and manufacturing one would mean breaking a
  customer app's only Tier-2 history.
- **The Tier-2 restore itself** was not executed — this arc changed what the dialog *says*, not what the
  restore does.
- **R-100's 48h staleness threshold** — injected clock only.
- **Fault 4** (restic transport interruption), **R-99**, **F-HUB**, fault 12, the three-way concurrency
  overlap — next campaign's material, untouched here.