REPORT: installer golden gate and the 0.211.0 bake
gates / gates (push) Successful in 20s

This commit is contained in:
2026-08-10 14:20:47 +02:00
parent 999b0a35f8
commit 2ccbbb8da1
+36 -137
View File
@@ -1,150 +1,49 @@
# REPORT — the census, the receipts, and a promise we should stop making (2026-08-10)
# REPORT — installer golden gate + golden 0.211.0 bake (2026-08-10)
Documentation and one read-only census. **No code that runs on a customer's machine, no deploy, no
bake.** Both demo boxes are online and were not changed. **The tester's machine was not contacted**
only its hub-side record was read, and it has none.
Commits `eb60087` (R-297 installer) and `999b0a3` (bake evidence). CI run **299**`success`.
All eight `repo_gates.py` gates OK, including `golden-currency`, which this run took from RED to GREEN.
**`--no-verify` was not used at any point** — the currency gate refused an earlier push and that
refusal was respected until the bake made it true.
---
## R-297 — an install takes the golden you approved
## 1. Part 2's answer, first
`felhom-host-install.sh` v1.26.0. Step 7 short-circuited on ANY local archive: no version compare,
no digest, no warning, so the manifest's sha256 — which vouches from a different trust root than the
code host — was consulted only on the fetch path. Discovery is `sort | tail -1`: correct by recency,
never by verification.
> **No. A machine that is not ours cannot be in the state that cost demo-felhom its history — not
> today, and not for anything enrolled from now on.**
`golden_local_matches_manifest()` compares digest first (a match means it IS the vouched artifact),
then the controller tag baked at `/etc/felhom-controller-image`. Auto-discovered mismatch → ignore
and fetch the vouched one; operator-named (`--golden`) mismatch → REFUSE, never silently substitute;
unreadable manifest → refuse rather than pass.
The hub holds escrow for **three** hosts. Both demo boxes lost their pre-fix key in the same four-hour
window on 2026-08-04; that is the entire affected population and it is entirely ours. `peti-felhom`
has **no host row and no escrow at all**, and neither does `david`. Anything enrolled from here escrows
under the fixed code, which has been in force since 2026-08-04.
**Measured on real hardware:** three goldens sit on `demo-hp` carrying controller **0.153.0** and
**0.210.0**, and nothing compared either to the vouched version before use.
## 2. The census, and the control that came first
**NOT PUBLISHED.** The task asked that a fresh install be watched taking a stale local golden on
`drill-r50` BEFORE the fix. That was not done, so `installer-v1.26.0` is deliberately not cut and
`webpage.yaml` is untouched — pushing to `main` publishes nothing (R-110). Publishing an unobserved
change to the one artifact that runs as root on a virgin box is the failure this project keeps
writing rules about.
**Control (run before the census, and it had to pass or the census was worthless):**
## Golden 0.211.0 — baked, published, round-trip verified
```
demo-felhom-8363b5 host_escrow -> present (572 bytes) want present OK
demo-felhom-8363b5 host_escrow_superseded -> absent (NULL) want absent OK
control PASSED: the query distinguishes both states on known cases.
```
Full evidence: `documentation/tests/golden-0.211.0-2026-08-10/` (`bake.log` + `RESULT.md`).
| host | current | superseded | material | superseded_at | verdict |
|---|---|---|---|---|---|
| `demo-felhom-8363b5` | 572 B | id=4 | **ABSENT** | 2026-08-04 07:20:08 | old backups lost (before the fix) |
| `demo-hp-bb76ea` | 572 B | id=3 | **ABSENT** | 2026-08-04 07:15:36 | old backups lost (before the fix) |
| `drill-r50-0a4f9a` | none | 0 | | | no supersession has happened |
| `peti-felhom` | — | — | — | — | **no host record, no escrow** |
| `david` | — | — | — | — | **no host record, no escrow** |
- `GOLDEN_SHA256=8593516889eb93fe1691410d7306be8cb87ee835b8d2378740eb34022272f849`
- Served bytes re-downloaded and hashed → identical. 656 141 689 B.
- All six acceptance markers counted (two initially returned a false `0` through broken shell
quoting and were re-run with `grep -F` — a 0 from a broken instrument is not a measurement).
- Token never on a command line; leak grep on the committed log = 0, and that 0 was made believable
by first proving the grep finds a planted token.
- Drill VM reverted to `virgin` before and after; build guest destroyed `--purge`; `/root` residue empty.
An orphan check found no escrow row pointing at a host the hub does not know.
## Registers
**One correction I made to my own instrument.** The first run labelled both losses *"superseded AFTER
the fix — unexpected"*, because a date-only comparison puts `2026-08-04 07:20:08` after `2026-08-04`.
The in-force moment is pinned instead from the hub's own first post-fix escrow row — **11:11:37Z**
which independently agrees with the register's *"four hours before v0.93.0 fixed the retention"*. Both
losses are then correctly *before* the fix: explained, not anomalous.
R-295 (PARTIAL — hub half open), R-296, R-297, R-298 filed. Ceiling moved R-294 → **R-298**.
**What this does NOT claim:** that a *retained* blob has ever been unwrapped on a superseded row.
Retention is proven; recovery **from** a superseded row remains unexercised.
## NOT DONE
## 3. Part 1 — the deleter is ESTABLISHED, and it was on file all along
The locator was correct: the record is **inside the R-267 row** — *"Pruned to the newest 10 per package
on the operator's rule, with the live-vouched golden/agent/floor asserted into the KEEP set before a
single DELETE was issued; 33 deletions, all HTTP 204."*
Every corroboration checked and every one holds:
- **The arithmetic:** 23 agent + 7 golden = 30, plus three older agent versions (0.81.0/0.80.0/0.79.0)
that *"only became visible after the first 30 deletions moved them onto page one"* = **33**.
- **The live PAGINATED listing** — 14 pages, 653 package-versions — `felhom-agent` generic at **exactly
10**, `felhom-golden` generic at **exactly 10**. That is what a newest-10 prune leaves.
The midnight-cleanup candidate is retired, and **R-291's blocking condition is released**: the operator
was being asked to establish something already written down.
**My counter-argument yesterday was wrong, in precisely the way R-267 warns about.** I argued against
the prune because *"container packages hold 19 each"*. That came from an **unpaginated** query the API
caps at 50/page. Paginated, they hold **270** and **169** — they were never in the prune. R-267 records
the identical trap one paragraph above the sentence I could not find: *"An unpaginated listing is not
evidence of a total — this repo's own rule, walked into while measuring."*
## 4. Part 3 — the twelve
**Three restored. Nine honest greys.** The map already read PROVEN-LIVE for the three, so the dataset
was *behind* it — restoring follows the map rather than raising a status in the view.
| claim | outcome |
|---|---|
| `drives.enrol` | **RESTORED**`audits/SPIKE-raw-drive-enroll-2026-06-15.md`: a live drive walked scan → format → mount → PVE storage → one-click enrol, with the resulting `storage.cfg` entry and mount unit recorded |
| `backup.tier1` | **RESTORED**`audits/CAMPAIGN-8-backup-restore-2026-07-27.md`: adversarial, destructive, unattended, both boxes + ep0; A2 proven end to end |
| `fail.lost-recovery-code` | **RESTORED**`audits/REHEARSAL-byo-reinstall-2026-08-09.md`, and proven the hard way on 2026-08-10 |
| `backup.restore-proof` | **grey — THE DECAY RULE FIRED** (below) |
| `install.installer-by-tag` | grey — hits are ISO spikes; nothing walks a tag rollback |
| `use.lifecycle` | grey — two passing mentions, no walk |
| `drives.migrate` | grey — one or two mentions only |
| `backup.whole-machine` | grey — diagnostics and phase findings, no walk of the claim |
| `fault.selfheal` | grey — **the best hit argues the other way**: `INCIDENT-guest-dhclient-killed-2026-07-20.md` documents self-heal's *absence* through a 1 h 15 m outage |
| `fault.operator-email` | grey — source-verified, delivery never observed |
| `fail.drive-filling` | grey — weak hits only |
| `fail.hub-down` | grey — Campaign 11's hub-unreachable work produced a *finding* (R-224), not a pass |
## 5. Did the "code moved under the proof" rule fire? **Yes — for the first time.**
`backup.restore-proof` **has** a receipt: `architecture/_recovery-inventory-2026-07-28.md` carries live
journal lines for scheduled restore-tests on both boxes and both tiers. It is superseded anyway —
demo-hp logged `restore_test_failed` on 2026-08-05, and the box has since been wiped and reinstalled.
The claim is about a **continuing** scheduled behaviour, so a 2026-07-28 observation cannot carry it.
**So the rule can fire, and now has.** Two nights ago it fired zero times out of twelve because every
downgrade came from *missing* evidence, not decayed evidence — there was nothing for it to bite on.
**A consequence to note: the capability map still reads `PROVEN-LIVE (2026-08-03)` for that row, so the
map is now the thing out of step**, and it is the source. Recorded rather than silently rewritten.
## 6. Part 4 — the promise, and why the wording is what it is
**Surfaces:**
| file:line | |
|---|---|
| `controller/internal/web/templates/backups_remote.html:101` | **the false promise** |
| `controller/internal/web/templates/backups_remote.html:98` | the orphan explanation — accurate, keep |
| `controller/internal/web/templates/layout.html:143` | the 14-day abandon countdown — accurate, keep |
| `controller/internal/settings/settings.go:337-338` | `OrphanedRenamedTo` schema comment — accurate, keep |
*(The capability map was searched and makes no such claim — nothing to correct there.)*
**The three cases:** set aside **before** the fix → not recoverable by construction (the restic
password lives only in the identity bundle, `escrow/identity.go:39` read by `escrow/recover.go:91`, and
those rows are NULL); set aside **after** → recoverable in principle, never demonstrated; **today's
population is entirely the first case**.
**The deciding fact: neither the box nor the customer can tell which case they are in.** The hub holds
the discriminator (`host_escrow_superseded.identity_blob`); the box caches only
`HubEscrowIdentityPresent`, which is about the *current* escrow; and **no field on the report or ACK
wire carries superseded-blob retention**. The box renders the card. **A conditional promise the
renderer cannot evaluate is the same defect as an unconditional false one** — so the specified copy
stops promising, says plainly what happens, explains why it cannot promise, and names a route
(write to us).
**Specification:** `documentation/design/SPEC-orphan-card-copy-2026-08-10.md` — copy, surfaces, and the
render tests that should pin it, including a regression guard that the string `visszaállítható lehet`
never returns. **Not implemented, on purpose:** it lands in the controller, and a controller release is
undelivered until a golden carries it (R-242). It should ship with the next controller change so one
bake and one approval cover both, and the spec says so.
## 7. Register
**Ceiling R-292 → R-294.** Opened **R-293** (the census) and **R-294** (the promise + spec). **R-287
turned to ESTABLISHED** with my unpaginated-count error withdrawn. **R-291's blocking condition
released.** **R-288 gained a second measured cost**, and it is a different failure mode from the first:
prose rows make claims ambiguous; rows-about-other-things make facts unfindable.
## 8. Observations — noticed, not acted on
- **The capability map is now out of step in two directions** — behind the dataset for three claims it
already called PROVEN-LIVE, and ahead of it for `backup.restore-proof`. Both point at R-288.
- **`fault.selfheal`'s only real document argues against it.** Worth someone deciding whether the
capability is real and unwalked, or overstated.
- **Retention is proven; recovery from a superseded row is not.** The census proves blobs are now kept;
nobody has ever unwrapped one. That is the next thing worth a drill, and it needs no customer.
- The page now shows the three restored claims as **"moved twice"** rather than once, so an unsettled
status reads as unsettled.
- **The Day-0 vouch** — the operator's, deliberately not made here.
- **Part 2** (uninstall leaves `dnsmasq` bound to :53) — dropped first, per the task's drop order.
- **Part 4a** (hub half of the code naming, and its emails) — dropped second, filed as R-295 PARTIAL.