admin/felhom-controller
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

REPORT + CONTEXT: R-101 + F-DIAG (v0.182.0), rendered dialog proven live

Browse Source
This commit is contained in:
authentik Default Admin
2026-07-28 16:46:16 +02:00
parent 3f048e042b
commit d8b3279731
2 changed files with 77 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
REPORT.md
+61 -62
View File
@@ -1,85 +1,84 @@
# REPORT — R-100: record the last SUCCESS, not just the last attempt (v0.181.0) (2026-07-28)
# REPORT — R-101 + F-DIAG: the customer must not be told a failed backup is a copy (v0.182.0) (2026-07-28)
**Overwritten** per the standing rule. Controller **v0.180.0 → v0.181.0** — the **producer** half of
R-100, shipped and deployed BEFORE the hub so the consumer never reads a field nothing sends.
Consumer: hub **v0.80.0**. Full arc, Phase 0 answers and the live proof: `felhom.eu/REPORT-r100.md`.
**Overwritten** per the standing rule. Controller **v0.181.0 → v0.182.0**. Full arc incl. Phase 0, the
F-OPS runbook and the rendered dialogs: `felhom.eu/REPORT-r101.md`.
## Baselines (reconfirmed, not copied)
`felhom-controller 4056fec` = origin/main, clean; controller 0.180.0 live on both boxes; hub 0.79.0
(manifest and running pod); agent 0.110.0.
`felhom-controller 3db8bfb` = origin/main, clean; 0.181.0 live on both boxes; hub 0.80.0 ready 1/1.
## What changed here
## R-101 — misinformation at a decision point, not an alarm bug
`OffboxTarget` gains **`LastSuccess`** (RFC3339), carried to the hub on the offsite report as
`last_success`. `LastRun` is written **unconditionally** at the end of every run including failures — it
records an **attempt**, so "how long since `LastRun`" answers "how long since we last TRIED", which is
not the question the hub's freshness verdict was asking of it.
`Tier2LastRun` is the ATTEMPT clock (`recordTier2Failure` writes it too) and it was rendered as
„Legutóbbi másolat" in the **restore confirm dialog**. The restore it guards fills in MISSING files
without touching existing ones — so a customer whose Tier-2 had been failing was told a copy existed
from last night, restored, and silently received **older** files while believing they were recent. No
error, no signal, and the customer's own reason for restoring quietly defeated.
The rule is a pure function, called unconditionally beside the `LastRun` write:
`CrossDriveBackup` gains `LastSuccess` (the offsite anchor's rule and shape, not a second
implementation) plus `SuccessTracked`, which distinguishes "predates the anchor" from "has one and it
is empty". That marker is not optional: **all 7 Tier-2 rows on the fleet** were pre-anchor at deploy,
so without it every customer would have been shown „Még nincs sikeres másolat" at once. Legacy rows
migrate truthfully on first touch — an `ok` row adopts its time, an `error` row seeds nothing.
```go
func offboxAnchorAfterRun(prev, at string, runErr error) string {
if runErr != nil { return prev } // failures neither advance nor clear the anchor
return at
}
```
Three dishonest render sites fixed; the main configured branch and the shares surface were already
honest. The dialog also stops printing raw UTC RFC3339 — new `fmtTimeStr` gives Budapest-local
`2026-07-28 16:40`.
Both directions are distinct bugs and both are pinned: a failure must not **advance** it (the original
defect) and must not **clear** it (one bad night making an established tier read as never-succeeded —
the mirror-image over-correction, and on the hub side the newborn-box path).
## Part 2 — the copy-site hazard, in the path of Part 1
**Why not simply `LastStatus == "error" ⇒ stale`:** that is the tempting one-line fix and it pages on
every transient blip — the F-A1 noise mode that trains an operator to ignore the alarm. Anchoring on the
last success tolerates one bad night and catches a persistent one, using the threshold that already
exists. No new knob.
The three `record*` helpers each built a WHOLE struct literal with two fields re-applied; everything
else was zeroed on every status write. Adding `LastSuccess` to that shape would have had
`recordTier2Failure` **clear** it — the mirror image of the defect, firing on the first failure.
`tier2Update` now copies the existing row and overlays the outcome: **safe by construction**. Sweep:
`SetTier2Preference` mutates in place (safe), `SetCrossDriveConfig(name, nil)` is a deliberate delete.
## Two silent-wipe sites, found and closed
## F-DIAG — six classes, and a leak I caught in my own fix
Both are the "seam built but never wired" shape — the field exists, the writer sets it, and an unrelated
routine path zeroes it. Neither would have surfaced until the hub's verdict changed, days later.
`ClassifyOffsiteFailure` → `quota` / `orphaned` / `no_repo` / `no_units` / `transport` / **`unknown`**,
each with its own Hungarian message; unclassifiable says so rather than being folded into a neighbour.
- **`offboxConfigHandler`** rebuilds the target from the form and copies runtime status across field by
field, so an ordinary settings save (edit the host, edit the path) would have erased the anchor.
- **`ApplyOffsiteTarget`** does the same on a hub re-apply — an established tier reset to "never
succeeded" every time the hub re-pushed its descriptor.
## A hollow test of my own, caught by red-proofing it
The first version of `TestOffboxLastSuccess_OnlyAdvancesOnSuccess` re-implemented the rule in a local
closure. Mutating the production code left it **green** — it asserted my model, not the system. That is
what the extraction to `offboxAnchorAfterRun` is for: the test now calls the real rule and the red-proof
bites.
The old message was a raw `err.Error()` passthrough carrying `sftp:<user>@<host>:<path>` off the box.
My first sanitiser regex-matched `sftp:…` and `user@host` and looked complete — **its own test caught it
leaking** on `ssh: connect to host <host> port 23: Connection refused`, a bare hostname in neither
shape. It now redacts the target's **actual** host/user/path literally, regex kept as a backstop.
## Red-proofs — all observed failing
| red-proof | observed failure |
|---|---|
| drop the `runErr` guard | `a FAILED run advanced LastSuccess to "2026-07-21T02:15:00Z" — that is the R-100 defect in mirror image` |
| always return `prev` | `a successful run did not advance the anchor` |
| drop `LastSuccess` from the wire | `OffboxReportStatus dropped LastSuccess — the hub would degrade forever on a controller that has it` |
| drop the handler preservation | `a settings save erased LastSuccess (got "") — the tier would read as never-succeeded` |
| # | red-proof | observed failure |
|---|---|---|
| A | dialog back on the attempt clock | `the dialog does not name the last SUCCESSFUL copy` |
| C | gate the restore on `LastRun` | `a tier that has NEVER succeeded still offers a restore` |
| D | caution made unconditional | `a HEALTHY tier shows the failed-attempt caution ("nem sikerült")` |
| F | clear the anchor on failure | `a FAILED run wiped the success anchor (round 1)` |
| + | raw sanitiser | `the repo reference reached the message ("sftp:" leaked)` |
`go build`, `go vet ./...`, `go test ./...` — 27 packages, `rc=0`, run separately from the commit.
F runs the **real** `recordTier2Success` → `recordTier2Failure` sequence; A/C/D **render the production
template tree** and assert the string the customer reads.
`go build`, `go vet ./...`, `go test ./...` — 27 packages `rc=0`; template-ID and emoji gates OK.
## LIVE on demo-hp — the rendered dialog
Failure induced by moving the Tier-2 destination aside and leaving a file in its place, so
`mkdir …/recovery-unit` fails. (`chmod` is useless — the controller is root; `chattr +i` is refused in
an unprivileged container. Both tried, both reported.) Data only moved, never deleted; restored intact.
## Live, on demo-hp and demo-felhom
```
demo-hp success run → ok last_run=11:24:20Z last_success=11:24:20Z
demo-hp settings save (port 23→2) → last_success PRESERVED 11:24:20Z ← the wipe-site fix, live
demo-hp failing run → error last_run=11:25:48Z last_success=11:24:20Z ← ANCHOR HELD
demo-felhom healthy run→ ok last_run=11:29:22Z last_success=11:29:22Z ← advanced
status=error last_run=14:42:18Z (ADVANCED) last_success=14:40:55Z (HELD)
```
Both shapes reached the hub. The failure was induced with a **closed port** (creates nothing, touches no
data) and the config was restored field by field. `peti-felhom` was never touched.
**failed state:** `… Legutóbbi sikeres másolat: 2026-07-28 16:40. Figyelem: a legutóbbi mentési
kísérlet nem sikerült, ezért a visszaállított fájlok ennél régebbiek lehetnek.`
**healthy state:** `… Legutóbbi sikeres másolat: 2026-07-28 16:43.` — no caution, no tonal change.
**legacy state (pre-run):** `Legutóbbi másolat: 2026-07-28 03:30` — today's wording, unchanged.
**Not proven live:** the hub's 48h staleness threshold itself — that is unit-tested with an injected
clock and would take days to observe live.
Restored: destination a directory again, 86 MB intact, mode 755, `status=ok`.
**demo-felhom is the untouched control** — all 5 rows still legacy, rendering today's way, 15/15 up.
## Deployed
`felhom-controller` **0.181.0** on **demo-hp** and **demo-felhom**, both `Up (healthy)`.
`felhom-controller` **0.182.0** on demo-hp and demo-felhom, both `Up (healthy)`.
## NOT yet live-validated (carried forward)
- The 48h threshold, and with it an actually-observed `offsite_stale` firing from a stale anchor.
- **R-101** (filed today) — Tier-2 `LastRun` is also written on failure and three customer surfaces
render it without a status, incl. the restore-confirm dialog.
- **Fault 4** — restic transport interruption; the closed-port injection sidesteps it, not solves it.
- **R-99** — prune never removes phantom snapshots.
- **F-DIAG's classes** — unit-proven; no live failure of each class was induced.
- **Scenario C live** — never-succeeded is unit-proven only; no fleet row is in that state and
manufacturing one means destroying an app's only Tier-2 history.
- **The Tier-2 restore itself** — this arc changed what the dialog says, not what the restore does.
- R-100's 48h threshold (injected clock), fault 4, R-99, F-HUB, fault 12, the concurrency overlap.