felhom.eu/REPORT-r101.md

REPORT — R-101 + F-DIAG + F-OPS (2026-07-28)

Controller v0.181.0 → v0.182.0; felhom.eu gains the manual-restore runbook (F-OPS) and the OPEN-ITEMS rows. Written as REPORT-r101.md so the shared REPORT.md is not clobbered.

Baselines (reconfirmed, not copied)

felhom-controller 3db8bfb, felhom.eu 6b7d516, felhom-agent d5c7691 — all = origin/main; the only dirt in felhom.eu was a foreign PROMPT-TEMPLATE.md. Controller 0.181.0 live on both boxes, hub felhom-hub:0.80.0 ready 1/1.

---

Phase 0

The render sites — three dishonest, two already honest. The spec listed backups_apps.html:216 as a defect site; it is in fact the one branch that already pairs its timestamp with a status badge. The real third site is the Tier2DestInactive branch.

site	rendered	honest?
:231 restore confirm dialog	Legutóbbi másolat: {{.Tier2LastRun}} — raw RFC3339, no status	NO — the one that matters
:195 Tier2DestDisconnected	Utolsó: …, no status	NO
:206 Tier2DestInactive	Utolsó: …, no status	NO
:217 main configured branch	Utolsó: … + status badge	already honest
sharing.html:181	rendered only when status=="ok"	already honest

Tier2LastStatus was already set unconditionally at handlers.go:1182, so this was a wording/anchor problem, not a plumbing one. The restore button was gated on {{if .Tier2LastRun}}, so Scenario C was live-reachable: a tier that had attempted and never succeeded offered a restore and a timestamp.

cd.LastRun is written on failure — recordTier2Failure (tier2.go:573-574) writes it alongside LastStatus:"error". Identical shape to R-100.

Legacy state is universal, not an edge case. All 7 Tier-2 rows across both boxes had last_run and no anchor. Scenario E was therefore the initial state of every customer, which is what made the legacy marker non-optional.

---

Part 1 — the strings shipped

case	string
dialog, normal	… Legutóbbi sikeres másolat: 2026-07-28 16:43.
dialog, newest attempt failed	… Legutóbbi sikeres másolat: 2026-07-28 16:40. Figyelem: a legutóbbi mentési kísérlet nem sikerült, ezért a visszaállított fájlok ennél régebbiek lehetnek.
card	Utolsó sikeres: 2 perce
never succeeded	Még nincs sikeres másolat + Még nincs sikeres másolat, amiből vissza lehetne állítani. (restore removed)
legacy row	Utolsó: … / Legutóbbi másolat: … — today's wording, unchanged, logged once per stack

Timestamp made human-readable (agreed): new fmtTimeStr renders Budapest-local 2026-07-28 16:40 instead of the raw UTC 2026-07-28T14:40:55Z a customer was previously asked to reason about.

SuccessTracked is what makes the legacy case possible at all. Without it, "row predates the anchor" and "row has an anchor and it is empty" are indistinguishable — both are LastSuccess=="" — and every existing row would have rendered as never-succeeded on deploy. Legacy rows migrate on first touch: a row whose last known state was ok adopts that time (truthful — under the old code that run did succeed); a row whose last state was error seeds nothing, because the old data evidences no success.

Part 2 — the copy-site hazard, and it was in the path

The three record* helpers each built a whole CrossDriveBackup literal, with a helper re-applying exactly two fields; everything else was zeroed on every status write. Adding LastSuccess to that shape would have had recordTier2Failure clear it — the mirror image of the defect, firing on the first failure rather than lying dormant.

Replaced with tier2Update, which copies the existing row and overlays the outcome: compile-safe by construction — a new field carries over unless deliberately overwritten, so nothing is preserved by a list that can fall out of date. Callers now clear explicitly what a run invalidates, reproducing the old behaviour exactly.

Sweep of other rebuild sites: SetTier2Preference mutates in place (safe); SetCrossDriveConfig(name, nil) in api/router.go:774 is a deliberate delete. No others.

Part 3 — F-DIAG

class	signal it maps to	message head
quota	the pre-run soft-quota gate	A távoli mentés nem fért el a tárhelykereten belül
orphaned	ErrOffboxOrphaned sentinel	A távoli tárhely egy korábbi, már nem elérhető kulccsal készült
no_repo	restic "unable to open config file"	A távoli tárhelyen nincs mentési adattár
no_units	"produced no snapshots"	Nem volt mit menteni: egyetlen kijelölt alkalmazásnak sem található mentése
transport	refused/reset/timeout/authn/host-key	A távoli tárhely nem érhető el (hálózat vagy bejelentkezés)
unknown	anything else	A távoli mentés ismeretlen okból nem sikerült

The unknown class is deliberate: a cause that cannot be told apart where the error is produced is reported as unknown rather than folded into a neighbour.

Secrets — and this caught a bug in my own first attempt. The old message was "…: " + err.Error(), carrying the repo reference sftp:<user>@<host>:<path> off the box. My first sanitiser regex-matched sftp:… and user@host and looked complete; its own test caught it leaking on ssh: connect to host <host> port 23: Connection refused — a bare hostname in neither shape. It now redacts the target's actual host/user/repo-path literally, with the regex kept only as a backstop. Guessing at what a secret looks like fails exactly where it matters.

Part 4 — F-OPS

documentation/runbooks/RUNBOOK-manual-guest-restore.md. Grounded in the real bind shape read off live guest 9201, not written from memory. Covers: which mpN are storage volumes (restored) versus host binds (taken as-is on the target); the mp9 trap — it embeds the source VMID, so restoring to a different VMID can bind another guest's bootstrap credentials; strip-and-re-add before first boot; the hookscript check; and a positive pre-start verification that asserts every bind path exists rather than accepting "no error". Docs only, by design.

---

Red-proofs — all observed failing

#	red-proof	observed failure
A	dialog back on the attempt clock	the dialog does not name the last SUCCESSFUL copy
C	gate the restore on LastRun again	a tier that has NEVER succeeded still offers a restore — the dialog would promise a copy that does not exist
D	make the caution unconditional	a HEALTHY tier shows the failed-attempt caution ("nem sikerült")
F	clear the anchor on failure	a FAILED run wiped the success anchor (round 1) — one bad night would read as 'no copy has ever succeeded'
+	raw sanitiser	the repo reference reached the message ("sftp:" leaked)

F exercises the real recordTier2Success → recordTier2Failure sequence, not a modelled copy — the R-100 lesson. The Scenario A/C/D tests render the production template tree and assert on the string the customer reads; a test asserting a template variable would prove nothing about wording, which is the defect.

go build, go vet ./..., go test ./... — 27 packages, rc=0; template_id_gate.py and emoji_gate.py both OK. Run separately from every commit.

---

LIVE on demo-hp — the rendered dialog, which is the deliverable

Legacy state (before any run under v0.182.0) — today's wording, no fright:

Legutóbbi másolat: 2026-07-28 03:30

Failure induced genuinely: the Tier-2 destination directory was moved aside and replaced by a file, so mkdir …/recovery-unit fails. (chmod does not work — the controller runs as root, which bypasses permission bits; chattr +i is refused, the unprivileged container lacks CAP_LINUX_IMMUTABLE. Both were tried and reported rather than glossed.) The real data was only ever moved, never deleted.

status       = error
last_run     = 2026-07-28T14:42:18Z     ← ADVANCED
last_success = 2026-07-28T14:40:55Z     ← HELD
last_error   = mkdir …/paperless-ngx/recovery-unit: …

The rendered dialog, failed state:

Visszaállítja a hiányzó fájlokat a másodlagos másolatból? A meglévő fájlok NEM módosulnak és NEM
törlődnek. Az alkalmazás a művelet idejére leáll. Legutóbbi sikeres másolat: 2026-07-28 16:40.
Figyelem: a legutóbbi mentési kísérlet nem sikerült, ezért a visszaállított fájlok ennél régebbiek
lehetnek.

The rendered dialog, healthy state (after restoring the destination and a successful run) — no caution, no tonal change:

Visszaállítja a hiányzó fájlokat a másodlagos másolatból? A meglévő fájlok NEM módosulnak és NEM
törlődnek. Az alkalmazás a művelet idejére leáll. Legutóbbi sikeres másolat: 2026-07-28 16:43.

Card lines: Utolsó sikeres: 2 perce → Utolsó sikeres: most.

Everything restored: destination is a directory again, 86 MB intact, mode 755, .r101-aside gone, status=ok, last_success=2026-07-28T14:43:23Z.

demo-felhom is the untouched control: all 5 rows still tracked=None after the deploy, rendering today's way, 15/15 containers up. Scenario E holding across a whole box nobody ran.

---

NOT yet live-validated (carried forward)

• F-DIAG's classes — unit-proven only. No live offsite failure of each class was induced; the transport class is the only one this arc exercised indirectly.
• Scenario C live — the never-succeeded rendering is unit-proven; no fleet row is in that state (every row either migrated or has a real success), and manufacturing one would mean breaking a customer app's only Tier-2 history.
• The Tier-2 restore itself was not executed — this arc changed what the dialog says, not what the restore does.
• R-100's 48h staleness threshold — injected clock only.
• Fault 4 (restic transport interruption), R-99, F-HUB, fault 12, the three-way concurrency overlap — next campaign's material, untouched here.