R-385: make an UNRECORDED golden fail the currency gate; file R-386; own the alarm ladder
gates / gates (push) Successful in 17s
gates / gates (push) Successful in 17s
The gate failed only on `released > baked`, so it could catch a forgotten bake and nothing else. A golden AHEAD of the record passed silently - and that is how controller 0.221.1 was built, baked AND vouched while the newest CHANGELOG heading still read v0.221.0, with every gate green. Reproduced on the real history: newest released 0.221.0 / newest golden baked 0.221.1 -> exit 0. The gate now asks whether the version being shipped is WRITTEN DOWN: the baked version must have its own `## vX.Y.Z` heading anywhere in the CHANGELOG. Membership rather than `baked > released` deliberately - a comparison against the newest heading alone goes green the moment any later entry is written, leaving the unrecorded version permanently unrecorded. INCONCLUSIVE (exit 2) preserved; every refusal names a reason and a route. Red-proofed both directions: old gate/old record exit 0, new gate/old record exit 1, new gate/fixed record exit 0, absent clone exit 2, post-bake exit 0. 08-alarm-ladder.md is new, and its absence was itself the finding: no document owned "when does a broken app raise an alarm?". The rules lived as comments in four packages, each locally correct, with the ordering between them legible only by reading one function top to bottom - which is how R-384 survived review. R-383 and R-384 closed into CLOSED-ITEMS with their rules kept. R-385 filed closed. R-386 filed OPEN: a single-container app stopped out of band raises no alarm, and a comment claims the opposite - measured live, 9 scans, 0 events, against a positive control from the same box 17 minutes earlier. Not fixed here. Golden 0.222.0 baked and published; vouching is the operator's act.
This commit is contained in:
@@ -1,62 +1,97 @@
|
||||
# REPORT — R-361 and the two loose ends v0.220.2 left (2026-08-22 → 23)
|
||||
# REPORT — felhom.eu: the golden-currency gate could not see an unrecorded golden (R-385)
|
||||
|
||||
Companion to `felhom-controller` **v0.221.0 → v0.221.1**. Full record:
|
||||
`documentation/audits/DRILL-r361-2026-08-22/`.
|
||||
**Session 2026-08-23.** Companion to `felhom-controller` v0.222.0 (R-384, R-383) — see that repo's
|
||||
`REPORT.md` for the controller work and the full live walk.
|
||||
|
||||
## What this repo carried
|
||||
## What was wrong here
|
||||
|
||||
- **`documentation/architecture/07-backup-architecture.md`** — a dated **[FACT]** on R-361 (the
|
||||
comment that asserted an invariant the code did not have, and what it cost), and a **[DESIGN]** on
|
||||
the `db_dumps` decision **including the trap it created**: a stable list lets
|
||||
`CaptureRecoveryUnit`'s already-current early return fire, so anything that must happen on every
|
||||
capture has to sit above that check.
|
||||
- **`documentation/architecture/00-capability-map.md`** — the **negative** from Part 3, recorded so it
|
||||
is not re-derived: a HELD app does **not** raise the dead-app alarm, measured on the shipped build,
|
||||
and the reading that said it would was wrong and why.
|
||||
- **`STATUS.md`** — the outcome in plain words; the deciding section says what happens if nothing is
|
||||
done.
|
||||
- **`documentation/tests/golden-0.221.1-2026-08-23/`** — the golden bake.
|
||||
- **Register** — R-361 closed and compressed; **R-383** and **R-384** opened.
|
||||
`scripts/golden_currency_gate.py` asked ONE question — *is the golden BEHIND the record?* — and
|
||||
therefore could only ever catch a forgotten bake. **It said nothing when the golden was AHEAD of the
|
||||
record**, and that direction is not harmless: a golden ahead of every CHANGELOG heading was built
|
||||
from something never written down.
|
||||
|
||||
## Part 3 — a measurement that cancelled a Part, and that is a good outcome
|
||||
That is not hypothetical. Controller **0.221.1** was built, baked **and vouched** on 2026-08-23 while
|
||||
the newest heading in the controller CHANGELOG still read `v0.221.0`. Measured on the real history,
|
||||
with the old gate:
|
||||
|
||||
The runbook's reading was that a held app alarms as a dead app. **It does not.** On the shipped
|
||||
v0.220.2 a hold was created deliberately; `docmost` aggregated to `unhealthy`; `IsDownState` is
|
||||
`{stopped, exited, degraded}`; the dead-app heartbeat read **`0 currently down`** at scans 600 and
|
||||
620 with the scans demonstrably running over it. **Part 2 was dropped in full** and **no register row
|
||||
was opened**, exactly as the runbook directs.
|
||||
```
|
||||
newest released controller : 0.221.0 (## v0.221.0 — taking the undo copy destroyed …)
|
||||
newest golden baked : 0.221.1 (documentation/tests/golden-0.221.1-2026-08-23)
|
||||
golden currency gate OK …
|
||||
EXIT=0
|
||||
```
|
||||
|
||||
**The positive control took three attempts, and that is the second finding.** Two live attempts
|
||||
failed to produce a lasting down state at all — `privatebin` went `stopped` (whitelisted by design)
|
||||
and `bookstack` went `degraded` then `unhealthy`. An absent alarm from a detector never shown working
|
||||
proves nothing, so the control was moved to the layer the detector lives in: `classifyRunStates` is a
|
||||
pure function, and it raises the banner for `degraded`/`exited` while staying silent for the states
|
||||
measured live.
|
||||
Every gate was green while the fleet ran a version the record did not name.
|
||||
|
||||
## Findings opened
|
||||
## The fix, and why it is membership and not a comparison
|
||||
|
||||
- **R-383 (MEDIUM)** — the double-failure message tells the customer *"a korábbi állapot mentése
|
||||
megvan"* while naming the very file whose absence caused the failure. Observed on **both** v0.220.2
|
||||
and v0.221.1. **R-361's own class** — a sentence asserting a property the code does not check.
|
||||
- **R-384 (MEDIUM)** — an app whose **database** has died reads `unhealthy` and raises no dead-app
|
||||
banner and no customer e-mail, because `aggregateState` checks `unhealthy > 0` before the
|
||||
mixed-case degraded branch. Not invisible everywhere (the health report counts it), but it does not
|
||||
alarm.
|
||||
The gate now asks **"is the version we are shipping WRITTEN DOWN?"** — the baked version must have its
|
||||
own `## vX.Y.Z` heading **anywhere** in the controller CHANGELOG, not merely at the top (an entry may
|
||||
legitimately be overtaken by later ones; what may never happen is that it is absent).
|
||||
|
||||
**Register size: `OPEN-ITEMS.md` 325 236 → 327 266 bytes; `CLOSED-ITEMS.md` 66 777 → 68 464.**
|
||||
R-361's full text: `git show a8caa0fdde7c:documentation/backlog/OPEN-ITEMS.md`.
|
||||
**Membership, not `baked > released`, deliberately:** a comparison against the newest heading alone
|
||||
goes green the moment ANY later entry is written — which would have left 0.221.1 permanently
|
||||
unrecorded and the gate permanently silent about it.
|
||||
|
||||
## The golden was baked here, and why
|
||||
Preserved unchanged: **INCONCLUSIVE (exit 2)** for an absent clone or an unparseable CHANGELOG — *not
|
||||
knowing is never a pass, and never a conviction*. Every refusal names a reason **and** a route,
|
||||
including what to do if a bake was a throwaway that must never be delivered.
|
||||
|
||||
`golden-currency` refused the docs push: 0.221.1 released with no golden. **Not circular** — a golden
|
||||
needs the controller image, already pushed, not this commit — so the gate was satisfied by doing the
|
||||
work rather than bypassed. **No push in this session used `--no-verify`.**
|
||||
## Red-proofs — both directions, against the real history
|
||||
|
||||
Golden **0.221.1**, sha256 `1c8bf6cf08cadabeca6331f38360d905e867c235067cd10c2716915b6e6df089`,
|
||||
656 966 079 B, round-trip verified, all markers hit, both negative controls at zero, both token-leak
|
||||
greps proved able to convict before their zeros were accepted.
|
||||
| Run | Gate | CHANGELOG | Golden baked | Exit | |
|
||||
|---|---|---|---|---|---|
|
||||
| `gate-01` | **old** | v0.221.0 | 0.221.1 | **0** | the blindness, reproduced |
|
||||
| `gate-02` | **new** | v0.221.0 | 0.221.1 | **1** | convicted |
|
||||
| `gate-03` | new | v0.221.1 | 0.221.1 | **0** | Part 0's heading makes it pass |
|
||||
| `gate-04` | new | absent clone | — | **2** | INCONCLUSIVE preserved |
|
||||
| `gate-05` | new | v0.222.0 | 0.222.0 | **0** | post-bake |
|
||||
|
||||
## Operator follow-up
|
||||
Transcripts: `documentation/audits/DRILL-r384-dead-db-alarm-2026-08-23/evidence/gate-0*.txt`.
|
||||
|
||||
**Vouch** the golden — a **three-field** save: `golden_version` **0.221.1**, `agent_version`
|
||||
**0.130.0**, `min_agent` **0.129.0**. **Then** raise the floor to **0.221.1**, last, in its own save.
|
||||
## Files changed
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `scripts/golden_currency_gate.py` | the unrecorded-golden conviction; `newest_released` → `released_versions` returning the whole set; docstring records the second blindness |
|
||||
| `documentation/architecture/08-alarm-ladder.md` | **NEW.** The alarm ladder as a dated [DESIGN] |
|
||||
| `documentation/architecture/00-capability-map.md` | R-384 marked closed with its live evidence; points at the new doc |
|
||||
| `documentation/backlog/OPEN-ITEMS.md` | R-383/R-384 removed (closed); **R-385** (closed) and **R-386** (open) filed |
|
||||
| `documentation/backlog/CLOSED-ITEMS.md` | R-383 + R-384 compressed, each keeping its rules and naming `git show 1eb64bec5183:…` for the full text |
|
||||
| `documentation/tests/golden-0.222.0-2026-08-23/` | **NEW.** Bake evidence + log + the vouching instructions |
|
||||
| `STATUS.md` | the 0.222.0 vouch replaces the (now completed) 0.221.1 one; R-386 added in plain words |
|
||||
| `documentation/audits/DRILL-r384-dead-db-alarm-2026-08-23/` | **NEW.** The drill record and 29 evidence files |
|
||||
|
||||
## The alarm ladder had no owning document — that absence is a finding
|
||||
|
||||
Nothing in `documentation/architecture/` owned the question *"when does a customer's app being broken
|
||||
raise an alarm?"* The rules lived as comments across four packages, each locally correct, with the
|
||||
ordering between them legible only by reading `aggregateState` top to bottom. **That is precisely how
|
||||
R-384 survived review**, and three separate defects in this ladder (R-51, C9-F2, R-384) were each
|
||||
found on live hardware rather than by reading. `08-alarm-ladder.md` now owns it.
|
||||
|
||||
## Golden
|
||||
|
||||
**Baked and PUBLISHED: 0.222.0.** `GOLDEN_SHA256 = 19f5904f5379…`, `upload OK (HTTP 201)`, round-trip
|
||||
`HTTP 206` from the package URL, all acceptance markers counted.
|
||||
**VOUCHING IS THE OPERATOR'S ACT AND WAS NOT DONE HERE.**
|
||||
|
||||
**Deviation recorded:** the bake runbook's §4.1 is missing a `pveam update`. On the `virgin` snapshot
|
||||
the template index is stale, so the listed template cannot be downloaded and the failure presents as
|
||||
`400 Parameter verification failed. template: no such template` rather than as a stale index.
|
||||
|
||||
## Register size
|
||||
|
||||
| File | Before | After |
|
||||
|---|---|---|
|
||||
| `OPEN-ITEMS.md` | 327,266 B | **328,325 B** |
|
||||
| `CLOSED-ITEMS.md` | 68,464 B | **71,441 B** |
|
||||
|
||||
OPEN grew ~1 KB despite two closures, because R-386 is a substantial new finding. Recorded rather
|
||||
than smoothed over.
|
||||
|
||||
## Hub numbers as read at session start (live, `GET /configuration`)
|
||||
|
||||
`golden_version` **0.221.1** · `agent_version` **0.130.0** · `min_agent` **0.129.0** ·
|
||||
controller floor **0.221.1**. The task expected 0.220.2/0.220.2; the operator had already vouched.
|
||||
**The hub was READ ONLY this session** — nothing was written to it.
|
||||
|
||||
Reference in New Issue
Block a user