admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

golden 0.203.0 baked + published; runbook acceptance markers fixed (R-233)
gates / gates (push) Successful in 13s
Details

Browse Source 
Bake evidence: documentation/tests/golden-0.203.0-2026-08-06/ (bake.log + README).
sha256 3039c6ffa7a5a8b2d959daddb2895c58b44de70f8d4f4a7e12ad4b1c0d61dc88, verified
by an independent round-trip download and by reading /etc/felhom-controller-image
out of the published archive itself. NOT vouched — the hub still serves 0.201.0.

R-233: RUNBOOK-manual-build.md §4.1 named three pass markers, two of which the
script cannot print (`overlay2 OK` does not exist; `mp1` stopped existing in
build-golden.sh v3.0.0 under R-165), and a 404 pre-gate URL with the wrong
filename, which would 404 for the wrong reason and pass even when the version
already existed. A grep for an impossible string reads 0 forever and 0 is
indistinguishable from failure. Markers re-captured from the real log; token
handling moved off the command line into an in-VM runner script; a positive
control is now required on the token-leak grep; the vouch step rewritten as the
three-field change it is (golden_version + agent_version + min_agent).
This commit is contained in:
authentik Default Admin
2026-08-06 13:23:22 +02:00
parent 0c4411e54b
commit 1c47e3b6fd
4 changed files with 468 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
documentation/tests/golden-0.203.0-2026-08-06/README.md
+93
View File
@@ -0,0 +1,93 @@
# Golden 0.203.0 — bake evidence (2026-08-06)
Baked per `runbooks/RUNBOOK-manual-build.md` §4.0–§4.1, in the drill VM on DooPlex
(`/mnt/5_hdd/felhom.eu/drill/drill.qcow2`, internal snapshot `virgin`) — the sanctioned bake
exception in `runbooks/target-selection.md`. Task 4 Part 3.
**Baked, published and verified. NOT vouched** — the hub's Day-0 artifacts were read but not written,
and that is deliberate: Part 3 ends at a STOP.
## The artifact
| | |
|---|---|
| version | **0.203.0** |
| sha256 | `3039c6ffa7a5a8b2d959daddb2895c58b44de70f8d4f4a7e12ad4b1c0d61dc88` |
| size | 658,292,861 bytes (uncompressed 2,008,145,920) |
| controller baked in | `gitea.dooplex.hu/admin/felhom-controller:0.203.0` |
| MinAgent (from the controller CHANGELOG header) | **0.127.0** |
| published at | `…/api/packages/admin/generic/felhom-golden/0.203.0/golden.tar.zst` |
## Preconditions, checked rather than assumed
- Disk headroom `df -h /mnt/5_hdd /` → 24% and 50%. (Abort threshold is 90%.)
- `ps -eo comm | grep -c qemu-system-x86` → **0**. Not `pgrep -f`, which self-matches and reports a
false "still running".
- `qemu-img snapshot -a virgin` **succeeded**, which independently re-proves nothing held the qcow2.
- Template **listed, not assumed**: `pveam available` → `debian-13-standard_13.6-1_amd64.tar.zst`
(unchanged since 2026-07-31), and `pveam download` printed `checksum verified`.
- CT 9100 absent on `virgin` before the run.
- Token verified against the Gitea API **before** starting, so a credential problem could not surface
mid-bake.
- 404 pre-gate on the **real** publish URL → 404. (The URL the runbook gave was wrong — R-233.)
- Controller image `0.203.0` manifest → 200, so the tag being baked demonstrably exists.
## Acceptance markers
The runbook's marker list was **broken** — see R-233; two of three strings could never match. These
are the real ones, from `bake.log` in this directory:
| marker | count | required |
|---|---|---|
| `docker OK (overlay2` | 1 | ≥1 |
| `including mount point` | 2 — `rootfs ('/')` and `mp0 ('/var/lib/felhom')` | rootfs + mp0 |
| `upload OK (HTTP 201)` | 1 | ≥1 |
| `excluding` | 0 | 0 |
| `FATAL` | 0 | 0 |
| `ERROR:` / `WARN:` | 0 / 0 | 0 |
| unit result | `Result=success`, `ExecMainStatus=0` | success / 0 |
There is no `mp1`: `build-golden.sh` v3.0.0 (R-165) collapsed the two data volumes into one.
## The published bytes were verified by round trip, not trusted
The bake **prints** a hash; that is the baker marking its own homework. The hash below was recomputed
on DooPlex from an independent download, and the content was then read out of that download:
1. `curl` the package → HTTP 200, 658,292,861 bytes.
2. `sha256sum` → `3039c6ff…dc88` — **matches** the baked value, byte count matches.
3. `zstd -t` → decompresses clean to 2,008,145,920 bytes: a real archive, not an error page.
4. `tar -xO ./etc/felhom-controller-image` **out of the published archive** →
`gitea.dooplex.hu/admin/felhom-controller:0.203.0`.
5. 19,358 entries under `var/lib/felhom/docker` — the baked images are on the data volume.
Step 4 is the one that matters: `GOLDEN_VERSION` is *derived from the tag argument*, so the label
could be right while the content was stale. It is not — the archive says so itself.
A third independent witness: the hub's own Day-0 dropdown now lists `0.203.0` with
`data-sha="3039c6ff…dc88"`.
## Secret hygiene
- Token copied **file → file** (`scp`), never through a shell on either side.
- Invocation moved into an in-VM runner script that reads the token itself. The runbook's
`--setenv=GITEA_TOKEN=$GT` form put the value on a command line and into the transient unit's
properties. Verified: `systemctl show golden-bake -p Environment -p ExecStart | grep -c -F <token>`
→ **0**.
- Token-leak grep by **literal value** on the saved log → **0**. A broad `[a-f0-9]{40}` pattern is
wrong here: it false-hits image shas.
- **Positive control on that grep**, because an absence proves nothing until the instrument is shown
to work: token appended to a throwaway copy → **1 hit**; copy `shred -u`'d; the real log's `0` is
therefore a measurement. Re-run against the committed copy in this directory → **0**.
## Teardown
`pct destroy 9100 --purge` (both logical volumes removed, `pct list` empty) · `shred -u` of the token,
`build-golden.sh`, the runner and `bake.log` **after** copying the log out · `poweroff` · qemu observed
gone in ~15 s via `ps -eo comm` · `qemu-img snapshot -a virgin`. The drill VM is back at its baseline
and DooPlex is unchanged apart from this evidence directory.
## What is NOT done here
Vouching. The hub still serves golden **0.201.0** and agent **0.125.0**, with `min_agent` 0.125.0.
See the session report for what changes on vouch and why it is a three-field change.
documentation/tests/golden-0.203.0-2026-08-06/bake.log
+325
View File
@@ -0,0 +1,325 @@
[golden] build-golden.sh v3.0.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.203.0
[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) …
Logical volume "vm-9100-disk-0" created.
Logical volume pve/vm-9100-disk-0 changed.
Creating filesystem with 8388608 4k blocks and 2097152 inodes
Filesystem UUID: e05e2970-4bab-420d-a32d-04bdd73fd5cd
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624
Logical volume "vm-9100-disk-1" created.
Logical volume pve/vm-9100-disk-1 changed.
Creating filesystem with 6291456 4k blocks and 1572864 inodes
Filesystem UUID: 82eacdcd-c966-4662-8fe5-d0036a44c7e4
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst'
Total bytes read: 553512960 (528MiB, 169MiB/s)
Detected container architecture: amd64
Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ...
done: SHA256:fS0llHyvLdZnXkBe4qOXJwHXZYqvlPr4hW88z838PJA root@felhom-golden
Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ...
done: SHA256:WEiExYNcAOEKEQ/cMYNA/IeFclQZrtf+YEunVDhjwF0 root@felhom-golden
Creating SSH host key 'ssh_host_rsa_key' - this may take some time ...
done: SHA256:wjJozAK6HvpRa+i8F/cdG+DPBY66vzfACKvi6Ex0jFk root@felhom-golden
[golden] starting + installing Docker (official repo, trixie channel) …
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation …
[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds …
[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) …
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
4f55086f7dd0: Pulling fs layer
4f55086f7dd0: Verifying Checksum
4f55086f7dd0: Download complete
4f55086f7dd0: Pull complete
Digest: sha256:7f4da0fc94bcece205a8c0b6f4d11c8196924654ffe5c4d1aa439b7f632048b2
Status: Downloaded newer image for hello-world:latest
docker OK (overlay2; data-root /var/lib/docker)
/var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4
/mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4
both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576
[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.203.0 (no registry cred at deploy) …
WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/
0.203.0: Pulling from admin/felhom-controller
039e6f9f9752: Pulling fs layer
80fbb63a1167: Pulling fs layer
9d6c7a1331cb: Pulling fs layer
3f0ff4614786: Pulling fs layer
e7855986fdc0: Pulling fs layer
a0fcfc238aba: Pulling fs layer
3f0ff4614786: Waiting
e7855986fdc0: Waiting
a0fcfc238aba: Waiting
9d6c7a1331cb: Verifying Checksum
9d6c7a1331cb: Download complete
039e6f9f9752: Verifying Checksum
039e6f9f9752: Download complete
3f0ff4614786: Verifying Checksum
3f0ff4614786: Download complete
e7855986fdc0: Verifying Checksum
e7855986fdc0: Download complete
a0fcfc238aba: Verifying Checksum
a0fcfc238aba: Download complete
80fbb63a1167: Verifying Checksum
80fbb63a1167: Download complete
039e6f9f9752: Pull complete
80fbb63a1167: Pull complete
9d6c7a1331cb: Pull complete
3f0ff4614786: Pull complete
e7855986fdc0: Pull complete
a0fcfc238aba: Pull complete
Digest: sha256:7fe128938fa20775dd1b82c18a50dfa3691d18137d6a0b49344c9f2df16683f6
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.203.0
gitea.dooplex.hu/admin/felhom-controller:0.203.0
[golden] asking the controller which infra images it manages …
[golden] baking infra images (4): traefik:v3.6.7 cloudflare/cloudflared:2026.6.0 gtstef/filebrowser:1.3.3-stable gitea.dooplex.hu/admin/felhom-samba:1.1.0 …
v3.6.7: Pulling from library/traefik
589002ba0eae: Pulling fs layer
ef63511ea6cc: Pulling fs layer
0738e5cb835e: Pulling fs layer
3e6813f70c64: Pulling fs layer
3e6813f70c64: Waiting
589002ba0eae: Download complete
ef63511ea6cc: Download complete
3e6813f70c64: Verifying Checksum
3e6813f70c64: Download complete
0738e5cb835e: Verifying Checksum
0738e5cb835e: Download complete
589002ba0eae: Pull complete
ef63511ea6cc: Pull complete
0738e5cb835e: Pull complete
3e6813f70c64: Pull complete
Digest: sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a
Status: Downloaded newer image for traefik:v3.6.7
docker.io/library/traefik:v3.6.7
2026.6.0: Pulling from cloudflare/cloudflared
47de5dd0b812: Pulling fs layer
c172f21841df: Pulling fs layer
99515e7b4d35: Pulling fs layer
99ba982a9142: Pulling fs layer
d6b1b89eccac: Pulling fs layer
2780920e5dbf: Pulling fs layer
7c12895b777b: Pulling fs layer
3214acf345c0: Pulling fs layer
52630fc75a18: Pulling fs layer
dd64bf2dd177: Pulling fs layer
b839dfae01f6: Pulling fs layer
ebddc55facdc: Pulling fs layer
bdfd7f7e5bf6: Pulling fs layer
2d4d7adf6272: Pulling fs layer
40008157d8d2: Pulling fs layer
bd8962e29291: Pulling fs layer
cac2ae0193cb: Pulling fs layer
74d1dac84ecc: Pulling fs layer
dd64bf2dd177: Waiting
b839dfae01f6: Waiting
ebddc55facdc: Waiting
bdfd7f7e5bf6: Waiting
2d4d7adf6272: Waiting
40008157d8d2: Waiting
bd8962e29291: Waiting
cac2ae0193cb: Waiting
99ba982a9142: Waiting
74d1dac84ecc: Waiting
d6b1b89eccac: Waiting
3214acf345c0: Waiting
52630fc75a18: Waiting
2780920e5dbf: Waiting
7c12895b777b: Waiting
47de5dd0b812: Verifying Checksum
47de5dd0b812: Download complete
c172f21841df: Download complete
99515e7b4d35: Verifying Checksum
99515e7b4d35: Download complete
47de5dd0b812: Pull complete
99ba982a9142: Verifying Checksum
99ba982a9142: Download complete
d6b1b89eccac: Verifying Checksum
d6b1b89eccac: Download complete
2780920e5dbf: Verifying Checksum
2780920e5dbf: Download complete
7c12895b777b: Verifying Checksum
7c12895b777b: Download complete
3214acf345c0: Verifying Checksum
3214acf345c0: Download complete
c172f21841df: Pull complete
52630fc75a18: Verifying Checksum
52630fc75a18: Download complete
dd64bf2dd177: Verifying Checksum
dd64bf2dd177: Download complete
b839dfae01f6: Verifying Checksum
b839dfae01f6: Download complete
ebddc55facdc: Verifying Checksum
ebddc55facdc: Download complete
bdfd7f7e5bf6: Verifying Checksum
bdfd7f7e5bf6: Download complete
2d4d7adf6272: Verifying Checksum
2d4d7adf6272: Download complete
bd8962e29291: Download complete
cac2ae0193cb: Verifying Checksum
cac2ae0193cb: Download complete
99515e7b4d35: Pull complete
40008157d8d2: Verifying Checksum
40008157d8d2: Download complete
74d1dac84ecc: Verifying Checksum
74d1dac84ecc: Download complete
99ba982a9142: Pull complete
d6b1b89eccac: Pull complete
2780920e5dbf: Pull complete
7c12895b777b: Pull complete
3214acf345c0: Pull complete
52630fc75a18: Pull complete
dd64bf2dd177: Pull complete
b839dfae01f6: Pull complete
ebddc55facdc: Pull complete
bdfd7f7e5bf6: Pull complete
2d4d7adf6272: Pull complete
40008157d8d2: Pull complete
bd8962e29291: Pull complete
cac2ae0193cb: Pull complete
74d1dac84ecc: Pull complete
Digest: sha256:ba461b8aa9c042156dbd39c38657fe7431bafa063220eab8d5330a523863da9f
Status: Downloaded newer image for cloudflare/cloudflared:2026.6.0
docker.io/cloudflare/cloudflared:2026.6.0
1.3.3-stable: Pulling from gtstef/filebrowser
6a0ac1617861: Pulling fs layer
ef8806083e82: Pulling fs layer
b74107c861c7: Pulling fs layer
adc935def003: Pulling fs layer
4f4fb700ef54: Pulling fs layer
18695ccc900a: Pulling fs layer
45d119d5c397: Pulling fs layer
dac52db4fc51: Pulling fs layer
6d598f86b2f2: Pulling fs layer
8aa349c8396c: Pulling fs layer
4f4fb700ef54: Waiting
18695ccc900a: Waiting
45d119d5c397: Waiting
dac52db4fc51: Waiting
6d598f86b2f2: Waiting
8aa349c8396c: Waiting
adc935def003: Waiting
6a0ac1617861: Verifying Checksum
6a0ac1617861: Download complete
adc935def003: Verifying Checksum
adc935def003: Download complete
4f4fb700ef54: Verifying Checksum
4f4fb700ef54: Download complete
6a0ac1617861: Pull complete
ef8806083e82: Verifying Checksum
ef8806083e82: Download complete
45d119d5c397: Verifying Checksum
45d119d5c397: Download complete
b74107c861c7: Verifying Checksum
b74107c861c7: Download complete
dac52db4fc51: Verifying Checksum
dac52db4fc51: Download complete
6d598f86b2f2: Verifying Checksum
6d598f86b2f2: Download complete
8aa349c8396c: Verifying Checksum
8aa349c8396c: Download complete
18695ccc900a: Verifying Checksum
18695ccc900a: Download complete
ef8806083e82: Pull complete
b74107c861c7: Pull complete
adc935def003: Pull complete
4f4fb700ef54: Pull complete
18695ccc900a: Pull complete
45d119d5c397: Pull complete
dac52db4fc51: Pull complete
6d598f86b2f2: Pull complete
8aa349c8396c: Pull complete
Digest: sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70babab78c
Status: Downloaded newer image for gtstef/filebrowser:1.3.3-stable
docker.io/gtstef/filebrowser:1.3.3-stable
1.1.0: Pulling from admin/felhom-samba
897d797d2723: Pulling fs layer
3051591aa250: Pulling fs layer
ce57a3f93416: Pulling fs layer
fb94eeec2fe1: Pulling fs layer
fb94eeec2fe1: Waiting
ce57a3f93416: Verifying Checksum
ce57a3f93416: Download complete
fb94eeec2fe1: Verifying Checksum
fb94eeec2fe1: Download complete
897d797d2723: Verifying Checksum
897d797d2723: Download complete
3051591aa250: Verifying Checksum
3051591aa250: Download complete
897d797d2723: Pull complete
3051591aa250: Pull complete
ce57a3f93416: Pull complete
fb94eeec2fe1: Pull complete
Digest: sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.1.0
gitea.dooplex.hu/admin/felhom-samba:1.1.0
[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'.
[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'.
[golden] baking the first-boot SSH host-key regeneration unit (F3) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'.
[golden] identity-clean + minimize …
[golden] stop + archive …
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/var/lib/felhom') in backup
INFO: archive file size: 627MB
INFO: Finished Backup of VM 9100 (00:00:30)
[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_08_06-13_16_07.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive)
[golden] publishing golden (658292861 bytes, sha256 3039c6ffa7a5a8b2…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.203.0/golden.tar.zst
[golden] pre-delete existing: HTTP 404 (404/204 expected)
[golden] upload OK (HTTP 201)
GOLDEN_VERSION=0.203.0
GOLDEN_SHA256=3039c6ffa7a5a8b2d959daddb2895c58b44de70f8d4f4a7e12ad4b1c0d61dc88
[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.203.0 / 3039c6ffa7a5a8b2d959daddb2895c58b44de70f8d4f4a7e12ad4b1c0d61dc88
[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)