admin
08b75e602e
gates / gates (push) Successful in 13s
Closes the delivery gap's build half. The vouched golden carried controller 0.203.0 while 0.205.0 was released, so a machine installed last night got neither R-237 (restore list keyed on the store) nor R-234 (skipped-app verdict). Both were measured from the customer's side on that box. version 0.205.0 sha256 8f49b2e8ccbc86a49df821fee9fb00c07293758811d3d0f0512dd0cf5fd54ee8 size 656,937,561 bytes (uncompressed 2,003,343,360) MinAgent 0.127.0 Round-trip verified rather than trusted: the published bytes were fetched back, re-hashed (match), zstd-tested, and ./etc/felhom-controller-image was read OUT of the downloaded archive -> felhom-controller:0.205.0. That last step is the one that matters, because GOLDEN_VERSION is derived from the tag argument and could have been right over stale content. Acceptance markers all pass; unit Result=success ExecMainStatus=0. Secret hygiene: token file->file, in-VM runner so it never reached a command line (unit-property grep 0), literal-value leak grep on the COMMITTED log 0 - with a positive control proving the grep works before the 0 was believed. Bake VM torn down: CT 9100 purged, secrets shredded, qemu observed gone via ps -eo comm, drill.qcow2 reverted to virgin. NOT VOUCHED - that is the operator's act. Only ONE field actually moves: golden_version 0.203.0 -> 0.205.0 (+ its derived sha). agent_version and min_agent both stay 0.127.0, because the new golden's MinAgent is also 0.127.0. The R-120 gate passes exactly: the newest controller the fleet reports is 0.205.0, so a 0.204.0 golden would have been refused.
119 lines
6.3 KiB
Markdown
119 lines
6.3 KiB
Markdown
# Golden 0.205.0 — bake evidence (2026-08-07)
|
|
|
|
Baked per `runbooks/RUNBOOK-manual-build.md` §4.0–§4.1, in the drill VM on DooPlex
|
|
(`/mnt/5_hdd/felhom.eu/drill/drill.qcow2`, internal snapshot `virgin`) — the sanctioned bake exception
|
|
in `runbooks/target-selection.md`. **R-239.**
|
|
|
|
**Baked, published and verified. NOT vouched** — the hub's Day-0 artifacts were read but not written.
|
|
That is deliberate: this part ends at a STOP, and vouching is the operator's act.
|
|
|
|
## The artifact
|
|
|
|
| | |
|
|
|---|---|
|
|
| version | **0.205.0** |
|
|
| sha256 | `8f49b2e8ccbc86a49df821fee9fb00c07293758811d3d0f0512dd0cf5fd54ee8` |
|
|
| size | **656,937,561** bytes (uncompressed 2,003,343,360) |
|
|
| controller baked in | `gitea.dooplex.hu/admin/felhom-controller:0.205.0` |
|
|
| MinAgent (from the controller `CHANGELOG.md` header) | **0.127.0** |
|
|
| published at | `…/api/packages/admin/generic/felhom-golden/0.205.0/golden.tar.zst` |
|
|
| golden archive volid (in-VM, destroyed with the CT) | `vzdump-lxc-9100-2026_08_07-09_00_40.tar.zst` |
|
|
| `build-golden.sh` version | 3.0.0 (R-165 — one data volume, **there is no `mp1`**) |
|
|
|
|
## Preconditions, checked rather than assumed
|
|
|
|
- **Clean-tree gate**, both source repos: `felhom-controller` HEAD == `origin/main` ==
|
|
`c6b69d888e2b`, tree clean; `felhom-agent` (which owns `build-golden.sh`) HEAD == `origin/main` ==
|
|
`703db166e762`, tree clean.
|
|
- Disk headroom `df -h /mnt/5_hdd /` → **24%** and **50%**. (Abort threshold is 90%.)
|
|
- `ps -eo comm | grep -c qemu-system-x86` → **0**. Not `pgrep -f`, which self-matches and reports a
|
|
false "still running".
|
|
- `qemu-img snapshot -a virgin` **succeeded**, which independently re-proves nothing held the qcow2.
|
|
- Template **listed, not assumed**: `pveam available --section system | grep debian-13` →
|
|
`debian-13-standard_13.6-1_amd64.tar.zst` (unchanged since 2026-07-31), and `pveam download` printed
|
|
`checksum verified`.
|
|
- CT 9100 **absent** on `virgin` before the run; `pct list` empty.
|
|
- Token verified against the Gitea API **before** starting (`/api/v1/user` → 200), so a credential
|
|
problem could not surface mid-bake. Token file `0600`.
|
|
- **404 pre-gate** on the real publish URL → **404** (the R-233 URL shape: `…/0.205.0/golden.tar.zst`,
|
|
not `felhom-golden-<VER>.tar.zst`).
|
|
- Controller image `0.205.0` manifest → **200**, so the tag being baked demonstrably exists.
|
|
- Drill VM read `pve-manager/9.2.2` — unchanged from the 0.203.0 bake.
|
|
|
|
## Acceptance markers
|
|
|
|
From `bake.log` in this directory. The runbook's original marker list was broken (R-233); these are
|
|
the real strings.
|
|
|
|
| marker | count | required |
|
|
|---|---|---|
|
|
| `docker OK (overlay2` | 1 | ≥1 |
|
|
| `including mount point` | 2 — `rootfs ('/')` and `mp0 ('/var/lib/felhom')` | rootfs + mp0 |
|
|
| `upload OK (HTTP 201)` | 1 | ≥1 |
|
|
| `excluding` | 0 | 0 |
|
|
| `FATAL` | 0 | 0 |
|
|
| `ERROR:` / `WARN:` | 0 / 0 | 0 |
|
|
| unit result | `Result=success`, `ExecMainStatus=0` | success / 0 |
|
|
|
|
Infra images baked (4, from the controller's own declaration): `traefik:v3.6.7`,
|
|
`cloudflare/cloudflared:2026.6.0`, `gtstef/filebrowser:1.3.3-stable`,
|
|
`gitea.dooplex.hu/admin/felhom-samba:1.1.0`.
|
|
|
|
## The published bytes were verified by round trip, not trusted
|
|
|
|
The bake **prints** a hash; that is the baker marking its own homework. Recomputed on DooPlex from an
|
|
independent download, and the content then read out of that download:
|
|
|
|
1. `curl` the package → **HTTP 200**, **656,937,561** bytes.
|
|
2. `sha256sum` → `8f49b2e8ccbc86a4…4ee8` — **matches** the baked value; byte count matches.
|
|
3. `zstd -t` → decompresses clean to **2,003,343,360** bytes: a real archive, not an error page.
|
|
4. `tar -xO ./etc/felhom-controller-image` **out of the published archive** →
|
|
`gitea.dooplex.hu/admin/felhom-controller:0.205.0`.
|
|
5. **19,358** entries under `var/lib/felhom/docker` — the baked images are on the data volume.
|
|
|
|
**Step 4 is the one that matters:** `GOLDEN_VERSION` is *derived from the tag argument*, so the label
|
|
could be right while the content was stale. It is not — the archive says so itself.
|
|
|
|
## Secret hygiene
|
|
|
|
- Token copied **file → file** (`scp`), never through a shell on either side.
|
|
- Invocation in an **in-VM runner script that reads the token itself**, so the value never reached a
|
|
command line. Verified: `systemctl show golden-bake -p Environment -p ExecStart | grep -c -F <token>`
|
|
→ **0**.
|
|
- Token-leak grep by **literal value** on the saved log → **0**. A broad `[a-f0-9]{40}` pattern is
|
|
wrong here: it false-hits image shas.
|
|
- **Positive control on that grep**, because an absence proves nothing until the instrument is shown to
|
|
work: the token was appended to a throwaway copy → **1 hit**; the copy was `shred -u`'d; the
|
|
committed log's **0** is therefore a measurement. Run against the copy **in this directory** — the
|
|
one that gets committed — not only the one in the VM.
|
|
|
|
## Teardown
|
|
|
|
`pct destroy 9100 --purge` (both logical volumes removed, `pct list` empty) · `shred -u` of the token,
|
|
`build-golden.sh`, the runner and `bake.log` **after** copying the log out · `poweroff` · qemu observed
|
|
gone via `ps -eo comm` · `qemu-img snapshot -a virgin` (confirmed: `virgin`, 0 B VM_SIZE). The drill VM
|
|
is back at its baseline and DooPlex is unchanged apart from this evidence directory.
|
|
|
|
## What is NOT done here — the STOP
|
|
|
|
**Vouching.** The hub still serves golden **0.203.0** (sha `3039c6ff…dc88`), agent **0.127.0**,
|
|
`min_agent` **0.127.0**.
|
|
|
|
**It is a three-field change, and this time only one field moves** — read from the live
|
|
`hub_settings`, not assumed:
|
|
|
|
| field | now | after | changes? |
|
|
|---|---|---|---|
|
|
| `artifact_golden_version` | `0.203.0` | `0.205.0` | **YES** |
|
|
| `artifact_golden_sha256` | `3039c6ff…dc88` | `8f49b2e8…4ee8` | **YES** (derived from the above) |
|
|
| `artifact_agent_version` | `0.127.0` | `0.127.0` | **no** — already ≥ the new golden's MinAgent |
|
|
| `artifact_min_agent` | `0.127.0` | `0.127.0` | **no** — the new golden's MinAgent is also 0.127.0 |
|
|
|
|
**The R-120 gate will pass, and only just.** It refuses a golden older than the newest controller the
|
|
fleet reports; the newest reported is **0.205.0** (demo-hp, 2026-08-07 06:29:52). A 0.204.0 golden
|
|
would have been **refused**.
|
|
|
|
**Vouching is reversible**: re-select the previous values and Save. A bake never deletes an older
|
|
golden's package (the publish step's pre-delete targets only its own version), so rolling back is a
|
|
form submission, not a rebuild.
|