admin/felhom-controller
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

REPORT + CONTEXT: R-101 + F-DIAG (v0.182.0), rendered dialog proven live

Browse Source
This commit is contained in:
authentik Default Admin
2026-07-28 16:46:16 +02:00
parent 3f048e042b
commit d8b3279731
2 changed files with 77 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTEXT.md
+16 -1
View File
@@ -7,7 +7,22 @@
> >
> Ask Claude Code: "Please update CONTEXT.md with what we did today" > Ask Claude Code: "Please update CONTEXT.md with what we did today"
Last updated: 2026-07-28 (v0.181.0 — R-100: the offsite last-SUCCESS anchor) Last updated: 2026-07-28 (v0.182.0 — R-101 + F-DIAG: the restore dialog names the last SUCCESSFUL copy)
> **2026-07-28 — v0.182.0 (R-101 + F-DIAG).** `Tier2LastRun` is the ATTEMPT clock (written on failure)
> and was rendered as „Legutóbbi másolat" in the **restore confirm dialog** — misinformation at a
> decision point: the restore fills in MISSING files, so a customer with a failing Tier-2 restored and
> silently got OLDER files. New `CrossDriveBackup.LastSuccess` + **`SuccessTracked`**; the marker is
> load-bearing because **all 7 fleet rows were pre-anchor at deploy** — without it every customer sees
> „Még nincs sikeres másolat" at once. Legacy rows migrate on first touch (`ok` adopts its time,
> `error` seeds nothing). **PART 2 — the three `record*` helpers rebuilt the WHOLE struct with only 2
> fields carried over; the naive fix would have had `recordTier2Failure` CLEAR the anchor.** Replaced
> by `tier2Update` (copy-and-overlay = safe by construction). New `fmtTimeStr` → Budapest-local dates
> in the dialog instead of raw UTC RFC3339. **F-DIAG:** 6 classes incl. an honest `unknown`, and the
> notification no longer passes `err.Error()` through raw — **LESSON: my first sanitiser was regex-only
> and leaked a bare hostname; its own test caught it. Redact KNOWN values, don't guess at shapes.**
> Live on demo-hp: rendered dialog read in the failed, healthy AND legacy states. F-OPS documented at
> `felhom.eu/documentation/runbooks/RUNBOOK-manual-guest-restore.md`.
> **2026-07-28 — v0.181.0 (R-100).** `OffboxTarget.LastSuccess` + wire field `last_success`; the hub > **2026-07-28 — v0.181.0 (R-100).** `OffboxTarget.LastSuccess` + wire field `last_success`; the hub
> (v0.80.0) anchors offsite staleness on it. **`LastRun` is written unconditionally on every run > (v0.80.0) anchors offsite staleness on it. **`LastRun` is written unconditionally on every run
REPORT.md
+61 -62
View File
@@ -1,85 +1,84 @@
# REPORT — R-100: record the last SUCCESS, not just the last attempt (v0.181.0) (2026-07-28) # REPORT — R-101 + F-DIAG: the customer must not be told a failed backup is a copy (v0.182.0) (2026-07-28)
**Overwritten** per the standing rule. Controller **v0.180.0 → v0.181.0** — the **producer** half of **Overwritten** per the standing rule. Controller **v0.181.0 → v0.182.0**. Full arc incl. Phase 0, the
R-100, shipped and deployed BEFORE the hub so the consumer never reads a field nothing sends. F-OPS runbook and the rendered dialogs: `felhom.eu/REPORT-r101.md`.
Consumer: hub **v0.80.0**. Full arc, Phase 0 answers and the live proof: `felhom.eu/REPORT-r100.md`.
## Baselines (reconfirmed, not copied) ## Baselines (reconfirmed, not copied)
`felhom-controller 4056fec` = origin/main, clean; controller 0.180.0 live on both boxes; hub 0.79.0 `felhom-controller 3db8bfb` = origin/main, clean; 0.181.0 live on both boxes; hub 0.80.0 ready 1/1.
(manifest and running pod); agent 0.110.0.
## What changed here ## R-101 — misinformation at a decision point, not an alarm bug
`OffboxTarget` gains **`LastSuccess`** (RFC3339), carried to the hub on the offsite report as `Tier2LastRun` is the ATTEMPT clock (`recordTier2Failure` writes it too) and it was rendered as
`last_success`. `LastRun` is written **unconditionally** at the end of every run including failures — it „Legutóbbi másolat" in the **restore confirm dialog**. The restore it guards fills in MISSING files
records an **attempt**, so "how long since `LastRun`" answers "how long since we last TRIED", which is without touching existing ones — so a customer whose Tier-2 had been failing was told a copy existed
not the question the hub's freshness verdict was asking of it. from last night, restored, and silently received **older** files while believing they were recent. No
error, no signal, and the customer's own reason for restoring quietly defeated.
The rule is a pure function, called unconditionally beside the `LastRun` write: `CrossDriveBackup` gains `LastSuccess` (the offsite anchor's rule and shape, not a second
implementation) plus `SuccessTracked`, which distinguishes "predates the anchor" from "has one and it
is empty". That marker is not optional: **all 7 Tier-2 rows on the fleet** were pre-anchor at deploy,
so without it every customer would have been shown „Még nincs sikeres másolat" at once. Legacy rows
migrate truthfully on first touch — an `ok` row adopts its time, an `error` row seeds nothing.
```go Three dishonest render sites fixed; the main configured branch and the shares surface were already
func offboxAnchorAfterRun(prev, at string, runErr error) string { honest. The dialog also stops printing raw UTC RFC3339 — new `fmtTimeStr` gives Budapest-local
if runErr != nil { return prev } // failures neither advance nor clear the anchor `2026-07-28 16:40`.
return at
}
```
Both directions are distinct bugs and both are pinned: a failure must not **advance** it (the original ## Part 2 — the copy-site hazard, in the path of Part 1
defect) and must not **clear** it (one bad night making an established tier read as never-succeeded —
the mirror-image over-correction, and on the hub side the newborn-box path).
**Why not simply `LastStatus == "error" ⇒ stale`:** that is the tempting one-line fix and it pages on The three `record*` helpers each built a WHOLE struct literal with two fields re-applied; everything
every transient blip — the F-A1 noise mode that trains an operator to ignore the alarm. Anchoring on the else was zeroed on every status write. Adding `LastSuccess` to that shape would have had
last success tolerates one bad night and catches a persistent one, using the threshold that already `recordTier2Failure` **clear** it — the mirror image of the defect, firing on the first failure.
exists. No new knob. `tier2Update` now copies the existing row and overlays the outcome: **safe by construction**. Sweep:
`SetTier2Preference` mutates in place (safe), `SetCrossDriveConfig(name, nil)` is a deliberate delete.
## Two silent-wipe sites, found and closed ## F-DIAG — six classes, and a leak I caught in my own fix
Both are the "seam built but never wired" shape — the field exists, the writer sets it, and an unrelated `ClassifyOffsiteFailure` → `quota` / `orphaned` / `no_repo` / `no_units` / `transport` / **`unknown`**,
routine path zeroes it. Neither would have surfaced until the hub's verdict changed, days later. each with its own Hungarian message; unclassifiable says so rather than being folded into a neighbour.
- **`offboxConfigHandler`** rebuilds the target from the form and copies runtime status across field by The old message was a raw `err.Error()` passthrough carrying `sftp:<user>@<host>:<path>` off the box.
field, so an ordinary settings save (edit the host, edit the path) would have erased the anchor. My first sanitiser regex-matched `sftp:…` and `user@host` and looked complete — **its own test caught it
- **`ApplyOffsiteTarget`** does the same on a hub re-apply — an established tier reset to "never leaking** on `ssh: connect to host <host> port 23: Connection refused`, a bare hostname in neither
succeeded" every time the hub re-pushed its descriptor. shape. It now redacts the target's **actual** host/user/path literally, regex kept as a backstop.
## A hollow test of my own, caught by red-proofing it
The first version of `TestOffboxLastSuccess_OnlyAdvancesOnSuccess` re-implemented the rule in a local
closure. Mutating the production code left it **green** — it asserted my model, not the system. That is
what the extraction to `offboxAnchorAfterRun` is for: the test now calls the real rule and the red-proof
bites.
## Red-proofs — all observed failing ## Red-proofs — all observed failing
| red-proof | observed failure | | # | red-proof | observed failure |
|---|---| |---|---|---|
| drop the `runErr` guard | `a FAILED run advanced LastSuccess to "2026-07-21T02:15:00Z" — that is the R-100 defect in mirror image` | | A | dialog back on the attempt clock | `the dialog does not name the last SUCCESSFUL copy` |
| always return `prev` | `a successful run did not advance the anchor` | | C | gate the restore on `LastRun` | `a tier that has NEVER succeeded still offers a restore` |
| drop `LastSuccess` from the wire | `OffboxReportStatus dropped LastSuccess — the hub would degrade forever on a controller that has it` | | D | caution made unconditional | `a HEALTHY tier shows the failed-attempt caution ("nem sikerült")` |
| drop the handler preservation | `a settings save erased LastSuccess (got "") — the tier would read as never-succeeded` | | F | clear the anchor on failure | `a FAILED run wiped the success anchor (round 1)` |
| + | raw sanitiser | `the repo reference reached the message ("sftp:" leaked)` |
`go build`, `go vet ./...`, `go test ./...` — 27 packages, `rc=0`, run separately from the commit. F runs the **real** `recordTier2Success` → `recordTier2Failure` sequence; A/C/D **render the production
template tree** and assert the string the customer reads.
`go build`, `go vet ./...`, `go test ./...` — 27 packages `rc=0`; template-ID and emoji gates OK.
## LIVE on demo-hp — the rendered dialog
Failure induced by moving the Tier-2 destination aside and leaving a file in its place, so
`mkdir …/recovery-unit` fails. (`chmod` is useless — the controller is root; `chattr +i` is refused in
an unprivileged container. Both tried, both reported.) Data only moved, never deleted; restored intact.
## Live, on demo-hp and demo-felhom
``` ```
demo-hp success run → ok last_run=11:24:20Z last_success=11:24:20Z status=error last_run=14:42:18Z (ADVANCED) last_success=14:40:55Z (HELD)
demo-hp settings save (port 23→2) → last_success PRESERVED 11:24:20Z ← the wipe-site fix, live
demo-hp failing run → error last_run=11:25:48Z last_success=11:24:20Z ← ANCHOR HELD
demo-felhom healthy run→ ok last_run=11:29:22Z last_success=11:29:22Z ← advanced
``` ```
Both shapes reached the hub. The failure was induced with a **closed port** (creates nothing, touches no **failed state:** `… Legutóbbi sikeres másolat: 2026-07-28 16:40. Figyelem: a legutóbbi mentési
data) and the config was restored field by field. `peti-felhom` was never touched. kísérlet nem sikerült, ezért a visszaállított fájlok ennél régebbiek lehetnek.`
**healthy state:** `… Legutóbbi sikeres másolat: 2026-07-28 16:43.` — no caution, no tonal change.
**legacy state (pre-run):** `Legutóbbi másolat: 2026-07-28 03:30` — today's wording, unchanged.
**Not proven live:** the hub's 48h staleness threshold itself — that is unit-tested with an injected Restored: destination a directory again, 86 MB intact, mode 755, `status=ok`.
clock and would take days to observe live. **demo-felhom is the untouched control** — all 5 rows still legacy, rendering today's way, 15/15 up.
## Deployed ## Deployed
`felhom-controller` **0.181.0** on **demo-hp** and **demo-felhom**, both `Up (healthy)`. `felhom-controller` **0.182.0** on demo-hp and demo-felhom, both `Up (healthy)`.
## NOT yet live-validated (carried forward) ## NOT yet live-validated (carried forward)
- The 48h threshold, and with it an actually-observed `offsite_stale` firing from a stale anchor. - **F-DIAG's classes** — unit-proven; no live failure of each class was induced.
- **R-101** (filed today) — Tier-2 `LastRun` is also written on failure and three customer surfaces - **Scenario C live** — never-succeeded is unit-proven only; no fleet row is in that state and
render it without a status, incl. the restore-confirm dialog. manufacturing one means destroying an app's only Tier-2 history.
- **Fault 4** — restic transport interruption; the closed-port injection sidesteps it, not solves it. - **The Tier-2 restore itself** — this arc changed what the dialog says, not what the restore does.
- **R-99** — prune never removes phantom snapshots. - R-100's 48h threshold (injected clock), fault 4, R-99, F-HUB, fault 12, the concurrency overlap.