admin
2263245cf2
gates / gates (push) Successful in 17s
GOLDEN_SHA256 9287f7cef5f13166276e8406005e3f28004004510c5184f1c1c7377f7aafad2e, 657 873 700 B. Evidence documentation/tests/golden-0.230.0-2026-08-31/. WHY IT WAS OWED: the newest golden was 0.229.0, which IS the build R-403 says deletes a good copy. Every fresh install and the whole fleet floor still carried it. golden_currency_gate.py had been red acrossdddcc80,6e550ae,130f7a6and32a4c35. THREE INDEPENDENT READERS agreed before anything was vouched: the bake's own print, the round trip of the PUBLISHED bytes (HTTP 200, 657873700 B, same sha), and the hub's Day-0 dropdown reading Gitea on a different code path. And the delivered artifact names the controller it will start - ./etc/felhom-controller-image read OUT of the downloaded archive says felhom-controller:0.230.0, with 19382 entries under var/lib/felhom/docker/. BOTH PRE-GATES were shown able to see something before their zeroes were believed: the 404 pre-gate, and the token-leak grep which returns 0 on the committed log and 1 on a seeded throwaway copy. The transient unit's own properties were grepped for the token too - 0, with the same seeded positive control returning 1. Acceptance markers counted on the COMMITTED log: 1/1/1/1 present, 0/0 absent, and the zeroes are believable because the same including-mount-point pattern returns two real lines on that file. THE VOUCH IS A THREE-FIELD CHANGE and only one field moved, which is stated rather than left to look careless: golden_version 0.229.0 -> 0.230.0; agent_version 0.130.0 and min_agent 0.129.0 UNCHANGED because v0.230.0's CHANGELOG header says MinAgent 0.129.0 and 0.129.0 <= 0.130.0, so this is not the R-216 shape. The 303 flash was not treated as proof - the page was re-read and golden_behind_fleet confirmed absent. THE FLOOR is a separate setting and was raised on the operator's explicit answer: min_controller_version 0.229.0 -> 0.230.0. THE POSITIVE OBSERVABLE, from the agent's own journal on demo-felhom, which was still running the defective 0.229.0: 16:21:30 controller-swap: image file written, restarting bootstrap target=...0.230.0 16:21:40 controller-swap: new controller healthy target=...0.230.0 Both boxes now 0.230.0 healthy. Honest note: the polling loop's first read already said 0.230.0, so the transition was not seen by the loop - the journal is the evidence. R-410 FILED, found while the gate went green: golden_currency_gate.py is satisfied by a DIRECTORY NAME (EVIDENCE_RE against os.listdir, :89,:123). I created the evidence directory before the bake finished and the gate would have passed at that moment. It already declares that it does not check the vouch; it does not declare that the bake check is a filename check. Fix: read the GOLDEN_SHA256= line out of the directory's bake.log, with a red-proof on an empty directory. R-242 updated - seventh debt, paid the same day, twice in one day. Teardown: pct destroy 9100 --purge, shred -u AFTER the log was copied out, poweroff, qemu confirmed exited with ps -eo comm (not pgrep -f, which self-matches), disk reverted to virgin. All 13 gates green - the first push this session that needed no --no-verify. Ceiling R-409 -> R-410.
141 lines
7.5 KiB
Markdown
141 lines
7.5 KiB
Markdown
# Golden bake 0.230.0 — 2026-08-31
|
|
|
|
Baked, published, round-trip verified, **vouched**, and the fleet floor raised. `demo-felhom` picked
|
|
up the new controller **by itself, unattended** — the agent's own journal is the proof, not an absent
|
|
error.
|
|
|
|
**Why it was owed:** v0.230.0 is the build that stops a poorer copy deleting a richer one (R-403).
|
|
Until this bake the newest golden was **0.229.0 — the build that has the defect** — so every fresh
|
|
install and the whole fleet floor still carried it. `golden_currency_gate.py` had been RED since the
|
|
v0.230.0 release and was red at `dddcc80`, `6e550ae`, `130f7a6` and `32a4c35`.
|
|
|
|
## What was produced
|
|
|
|
| | |
|
|
|---|---|
|
|
| `GOLDEN_VERSION` | **0.230.0** |
|
|
| `GOLDEN_SHA256` | `9287f7cef5f13166276e8406005e3f28004004510c5184f1c1c7377f7aafad2e` |
|
|
| size | **657 873 700 B** |
|
|
| package URL | `…/api/packages/admin/generic/felhom-golden/0.230.0/golden.tar.zst` |
|
|
| baked controller | `gitea.dooplex.hu/admin/felhom-controller:0.230.0` |
|
|
| `MinAgent` | **0.129.0** — read from the controller `CHANGELOG.md` header, not assumed |
|
|
| script | `build-golden.sh v3.0.0`, sha256 `7b0fb5cf…73b6a1` on DooPlex |
|
|
| venue | the drill VM on DooPlex, reverted to `virgin` and **cold-booted** first (`pve-manager/9.2.2`) |
|
|
| template | `debian-13-standard_13.6-1_amd64.tar.zst`, after `pveam update` — the virgin snapshot's INDEX is stale too, and that failure reads as a bogus `400 no such template` |
|
|
| archive volid | `local:backup/vzdump-lxc-9100-2026_08_31-16_15_14.tar.zst` |
|
|
| bake wall clock | ~16:11 → 16:15 CEST, `vzdump` leg 00:00:41 |
|
|
|
|
**One thing the 0.229.0 bake did that this one did not**, said plainly rather than left as an
|
|
implied equivalence: the script's sha256 was **not** compared across the hop. It was `scp`'d
|
|
file→file and its DooPlex-side sha is recorded above; a corrupted copy would have failed the bake
|
|
rather than produced a wrong golden, but that is an argument, not a measurement.
|
|
|
|
## Acceptance markers — counted on the COMMITTED log, not eyeballed
|
|
|
|
```
|
|
docker OK (overlay2 : 1
|
|
including mount point rootfs : 1 (line 317)
|
|
including mount point mp0 : 1 (line 318) <- there is no mp1 since v3.0.0 (R-165/R-233)
|
|
upload OK (HTTP 201) : 1 (line 324)
|
|
--- must be ZERO ---
|
|
excluding : 0
|
|
FATAL : 0
|
|
```
|
|
|
|
**The zeroes are believable because the greps are shown to work on this file:** the same
|
|
`including mount point` pattern that returns 0 for `excluding` returns 2 real lines. An instrument
|
|
that can silently drop results is not a measurement.
|
|
|
|
## Three independent readers agreed before anything was vouched
|
|
|
|
1. **The bake itself** printed `GOLDEN_SHA256=9287f7ce…ad2e`.
|
|
2. **The round trip** — the published bytes downloaded back: `HTTP 200`, `657 873 700 B`,
|
|
sha256 `9287f7ce…ad2e`. The hash is of the **downloaded** bytes, never the local file.
|
|
3. **The hub's Day-0 dropdown**, a different code path, read `data-sha="9287f7cef5f13166276e…"`
|
|
straight from Gitea.
|
|
|
|
**And the delivered artifact names the controller it will start**, read out of the downloaded
|
|
archive itself:
|
|
|
|
```
|
|
$ tar --zstd -xOf golden.tar.zst ./etc/felhom-controller-image
|
|
gitea.dooplex.hu/admin/felhom-controller:0.230.0
|
|
```
|
|
|
|
with **19 382** entries under `var/lib/felhom/docker/` — the baked image store is in the archive, not
|
|
a promise that it will be pulled later.
|
|
|
|
## Both pre-gates were proven able to see something first
|
|
|
|
| gate | negative result | the positive control that makes it believable |
|
|
|---|---|---|
|
|
| 404 pre-gate on the package URL | `HTTP 404` for 0.230.0 before the bake | the same URL shape returns `HTTP 200` for the controller image manifest, and the bake's own pre-delete logged `HTTP 404 (404/204 expected)` |
|
|
| token-leak grep on the committed log | **0** in `06-bake.log` and `06-bake-clean.log` | the token appended to a throwaway copy of the same log greps **1**; the copy was then `shred -u`'d |
|
|
|
|
The token was also kept off every command line: the transient unit's own properties were grepped for
|
|
it — `systemctl show golden-bake -p Environment -p ExecStart` → **0**, with the same seeded positive
|
|
control returning **1**.
|
|
|
|
## The vouch — a THREE-field change, checked rather than assumed
|
|
|
|
| field | before | after | why |
|
|
|---|---|---|---|
|
|
| `golden_version` | 0.229.0 | **0.230.0** | the new bake |
|
|
| `agent_version` | 0.130.0 | 0.130.0 | **unchanged** — already ≥ MinAgent |
|
|
| `min_agent` | 0.129.0 | 0.129.0 | **unchanged** — v0.230.0's CHANGELOG header says `MinAgent: 0.129.0` |
|
|
|
|
`min_agent` (0.129.0) ≤ `agent_version` (0.130.0), so this is **not** the R-216 shape the hub holds
|
|
against. Only one field actually moved, and that is stated rather than left to look like a one-field
|
|
vouch performed carelessly.
|
|
|
|
`POST /configuration/artifacts` → `303 Location: /configuration?flash=artifacts_set`. **The flash was
|
|
not treated as proof:** the page was re-read and the selected options confirmed
|
|
(`golden_version` selected 0.230.0, sha `9287f7ce…`), and the R-120 refusal banner
|
|
(`golden_behind_fleet`) confirmed **absent**.
|
|
|
|
## The fleet floor, and the unattended proof that it worked
|
|
|
|
`POST /configuration/global-floor` `min_controller_version` **0.229.0 → 0.230.0**, re-read from the
|
|
page afterwards. This is a **separate setting from the Day-0 artifacts** — the page says so itself —
|
|
and it is the one that moves the existing fleet rather than fresh installs.
|
|
|
|
**`demo-felhom` was on controller 0.229.0 — the R-403 build — and moved itself.** From the agent's
|
|
journal on that host, which is a POSITIVE observable and not a missing error:
|
|
|
|
```
|
|
16:21:30 controller-swap: image file written, restarting bootstrap target=…felhom-controller:0.230.0
|
|
16:21:40 controller-swap: new controller healthy target=…felhom-controller:0.230.0
|
|
```
|
|
|
|
**Honest note on the watch:** the polling loop's *first* read already said 0.230.0, so the transition
|
|
was not observed by the loop. The evidence is the journal above plus the container reading
|
|
`Up 8 seconds (healthy)` at that first read. Both boxes now:
|
|
|
|
```
|
|
demo-felhom gitea.dooplex.hu/admin/felhom-controller:0.230.0 Up (healthy)
|
|
demo-hp gitea.dooplex.hu/admin/felhom-controller:0.230.0 Up (healthy)
|
|
```
|
|
|
|
## Teardown — all of it, and it is not "nothing was created"
|
|
|
|
`pct destroy 9100 --purge` (both logical volumes removed), then `shred -u` of the token, the runner
|
|
script, `build-golden.sh` and `bake.log` **after** the log was copied out for evidence — `ls | grep`
|
|
in `/root` returns nothing. Then `poweroff`, waited for the qemu process to actually exit (checked
|
|
with `ps -eo comm`, **not** `pgrep -f`, which self-matches), and `qemu-img snapshot -a virgin`. The
|
|
disk carries the single `virgin` snapshot and nothing else.
|
|
|
|
## Files here
|
|
|
|
| file | what |
|
|
|---|---|
|
|
| `01-preconditions.txt` | disk headroom, image present, MinAgent, hub state before |
|
|
| `02-pveam.txt`, `03-template.txt` | the index refresh and the template download |
|
|
| `04-staged.txt`, `05-bake-launch.txt` | staging + the token-leak check on the unit properties |
|
|
| `06-bake.log`, `06-bake-clean.log` | the bake, raw and with the locale noise stripped |
|
|
| `07-teardown-vm.txt` | destroy, shred, revert to virgin |
|
|
| `08-roundtrip.txt`, `16-archive-content-proof.txt` | the published bytes, and what is inside them |
|
|
| `00-`, `10-`, `13-hub-*.html` | the hub configuration page before, after the vouch, after the floor |
|
|
| `09-vouch-post.txt`, `11-vouch-verified.txt`, `14-floor-raise.txt` | the two POSTs and their re-reads |
|
|
| `12-golden-currency-gate.txt` | the gate that was red, now green |
|
|
| `15-demo-felhom-selfupdate.txt` | the unattended pickup |
|