diff --git a/STATUS.md b/STATUS.md index 98d981e..3a4c002 100644 --- a/STATUS.md +++ b/STATUS.md @@ -62,20 +62,21 @@ per-guest network health, which we already lost 1 h 15 m to once. ## What we're working on -- **Widening the check** so a fourth secret-in-a-page is caught by a machine, not by someone. *(R-255)* -- **Deciding the twenty-one** — for each: give it a reader, or stop sending it. *(R-264)* -- **Proving the hub really keeps the old sealed key** when a machine re-seals. *(R-198)* -- Still open from the overnight sweep, none urgent: *(R-256…R-259, R-261…R-263)* +- **Widening the check** so a fourth secret-in-a-page is caught by a machine. *(R-255)* · **Deciding + the twenty-one** — each gets a reader, or stops being sent. *(R-264)* +- **Proving the hub really keeps the old sealed key** when a machine re-seals. *(R-198)* · Still open + from the overnight sweep, none urgent: *(R-256…R-259, R-261…R-263)* ## Waiting on you -- **One approval: the new base image.** Tonight's release is baked, published and byte-checked, and - **installations still receive yesterday's version until you press Save.** Hub → Configuration → - Day-0 artifacts → Golden **0.208.0** → Save. One field moves; the other two are already right and - were checked. Reversible — re-select 0.207.0 and Save. *(R-242)* -- **Third time in three days, so worth a minute.** A check now catches the *baking* being forgotten; - **nothing catches the approval being forgotten** — tonight's bake proved it, going green before the - approval existed. Two ways to close it are written up, neither built. *(ROADMAP G-8)* +- **One approval, and now it is the only one: the new base image.** Baked, published, byte-checked — + and **installations receive an older version until you press Save.** Hub → Configuration → Day-0 + artifacts → Golden **0.209.0** → Save. One field moves, the other two were checked and are right, + and it is reversible. It supersedes last night's unvouched 0.208.0, so this is one decision, not + two. *(R-242)* +- **Fourth time in three days, so worth a minute.** A check now catches the *baking* being forgotten + — it caught it again today and refused the push until it was done. **Nothing catches the approval + being forgotten.** Two ways to close that are written up, neither built. *(ROADMAP G-8)* ## DooPlex infrastructure — separate from the product diff --git a/documentation/tests/golden-0.209.0-2026-08-08/README.md b/documentation/tests/golden-0.209.0-2026-08-08/README.md new file mode 100644 index 0000000..daf91bb --- /dev/null +++ b/documentation/tests/golden-0.209.0-2026-08-08/README.md @@ -0,0 +1,106 @@ +# Golden 0.209.0 — baked, published, round-trip verified, **NOT VOUCHED** (2026-08-08) + +**Why this bake exists, and why it supersedes last night's.** The G-1 session released controller +**v0.209.0** (R-247), which made `golden_currency_gate.py` correctly red and **refused the push** — +no golden carried the newest release. The honest answer to that gate is the bake it asks for, not +`--no-verify`; the gate's own docstring says the cost of a trip is "one bake, which is the operation +the project wants to be routine anyway". + +**This also consolidates the operator's approval into ONE.** Golden **0.208.0** was baked last night +and never vouched. It is superseded rather than wasted: 0.209.0 contains everything 0.208.0 did, plus +R-247. **Vouch 0.209.0 and 0.208.0 needs no separate decision.** + +## Where it ran + +The **drill VM on DooPlex** (`/mnt/5_hdd/felhom.eu/drill/drill.qcow2`, snapshot `virgin`) — the +accepted Tier-2 exception for bakes. Canonical §4.0 launch line, cold-booted, **restored to `virgin` +afterwards** and the snapshot list re-read. Liveness via `ps -eo comm | grep -c qemu-system-x86`, +never `pgrep -f`. + +## The inputs + +| | | +|---|---| +| host | `drill-pve`, `pve-manager/9.2.2` | +| template | **`debian-13-standard_13.6-1_amd64.tar.zst`** — listed with `pveam available` on the day; checksum verified on download | +| build script | `felhom-agent/configs/build-golden.sh` v3.0.0, clean tree at `6981450` (`git status --porcelain` empty) | +| controller baked | `gitea.dooplex.hu/admin/felhom-controller:0.209.0`, built and pushed from a clean tree at `fcffaf5` | + +## The token never crossed a shell + +`scp` file → file into a `0600` file, read by a runner script **inside** the VM. + +``` +systemctl show golden-bake -p Environment -p ExecStart | grep -c -F "$(cat /root/.gitea-token)" → 0 +``` + +## The 404 pre-gate, with a control + +``` +felhom-golden/0.208.0/golden.tar.zst → 200 ← the control: the URL SHAPE is right +felhom-golden/0.209.0/golden.tar.zst → 404 ← the pre-gate: nothing to overwrite +``` + +## Acceptance markers — grepped verbatim against this run's log + +| marker | count | verbatim | +|---|---|---| +| storage driver | **1** | ` docker OK (overlay2; data-root /var/lib/docker)` | +| mount points — rootfs **and mp0**, no mp1 | **2** | `INFO: including mount point rootfs ('/') in backup` · `INFO: including mount point mp0 ('/var/lib/felhom') in backup` | +| upload | **1** | `[golden] upload OK (HTTP 201)` | +| `excluding` | **0** | — | +| `FATAL` | **0** | — | +| `mp1` | **0** | — | + +`Result=success`, `ExecMainStatus=0`. Archive 626 MB. + +## The publish, and the round trip — the published BYTES + +``` +GOLDEN_VERSION=0.209.0 +GOLDEN_SHA256=c9c4bcd6d9f53dcaf3c7a56ffce1c401864a5564af4eca8ff5efd8d5eae818ff +``` + +| | | +|---|---| +| size | **656 697 956 B** — identical to what the bake reported | +| sha256 | **`c9c4bcd6…e818ff`** — hashed independently on DooPlex; identical | +| **`./etc/felhom-controller-image` read OUT of the downloaded archive** | **`gitea.dooplex.hu/admin/felhom-controller:0.209.0`** | + +The delivered artifact saying which controller it will start — not a claim about what was built. + +## The vouch, awaiting the operator + +`MinAgent` read from the golden's controller `CHANGELOG.md` header, not assumed: +`## v0.209.0 — … — MinAgent 0.127.0`. + +| field | now (live `hub_settings`) | after | why | +|---|---|---|---| +| `golden_version` | 0.207.0 | **0.209.0** | the only field that moves | +| `agent_version` | 0.127.0 | 0.127.0 | already ≥ `MinAgent 0.127.0` — checked | +| `min_agent` | 0.127.0 | 0.127.0 | equals the declared `MinAgent`; **not above `agent_version`** (the R-216 shape hub v0.97.0 holds) | + +Hub → Configuration → Day-0 artifacts → Golden `0.209.0` → Save. The R-120 gate sits on that save and +refuses a golden older than the newest controller the fleet reports. **Reversible** — re-select the +previous value and Save; no package is deleted by a bake. + +## Teardown + +`pct destroy 9100 --purge` · token, runner, build script and in-VM log `shred -u`'d, `/root` residue +clean · `poweroff` · qemu confirmed gone (**0**) · `qemu-img snapshot -a virgin` restored. + +**Token-leak grep on the log COMMITTED here, and a `0` is evidence only because the grep was shown to +work:** + +``` +grep -c -F "" bake.log → 0 +grep -c -F "" → 1 ← the control; copy then shredded +``` + +## What this does and does not change + +**Does:** `golden_currency_gate.py` is green; the push it was blocking can proceed. + +**Does NOT:** a fresh Day-0 install still lands on **0.207.0**. The gate checks the BAKE, not the +vouch — R-242's untouched half, demonstrated again here: the gate flips green on the presence of this +directory, with no vouch anywhere near it. diff --git a/documentation/tests/golden-0.209.0-2026-08-08/bake.log b/documentation/tests/golden-0.209.0-2026-08-08/bake.log new file mode 100644 index 0000000..32548be --- /dev/null +++ b/documentation/tests/golden-0.209.0-2026-08-08/bake.log @@ -0,0 +1,322 @@ +[golden] build-golden.sh v3.0.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.209.0 +[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) … + Logical volume "vm-9100-disk-0" created. + Logical volume pve/vm-9100-disk-0 changed. +Creating filesystem with 8388608 4k blocks and 2097152 inodes +Filesystem UUID: 8be84520-bf6e-476c-8458-896f4bfda3b7 +Superblock backups stored on blocks: + 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, + 4096000, 7962624 + Logical volume "vm-9100-disk-1" created. + Logical volume pve/vm-9100-disk-1 changed. +Creating filesystem with 6291456 4k blocks and 1572864 inodes +Filesystem UUID: 37a93e82-e4b5-4438-88f2-62852ece5637 +Superblock backups stored on blocks: + 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, +extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst' +Total bytes read: 553512960 (528MiB, 169MiB/s) +Detected container architecture: amd64 +Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ... +done: SHA256:L767URF8/qUvOJAfm2wOY9zOt54prZAv0Cr3sz2vVy4 root@felhom-golden +Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ... +done: SHA256:ovz3qiVwNN/wm2HZ3BqhaMG9hpyWgolZ9SY6BahHdl4 root@felhom-golden +Creating SSH host key 'ssh_host_rsa_key' - this may take some time ... +done: SHA256:Usm+AmoTIhzKMxrVcvIb4gKMC11TNBJsBSgREtv+klE root@felhom-golden +[golden] starting + installing Docker (official repo, trixie channel) … +apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct! +perl: warning: Setting locale failed. +perl: warning: Please check that your locale settings: + LANGUAGE = (unset), + LC_ALL = (unset), + LC_CTYPE = (unset), + LC_NUMERIC = (unset), + LC_COLLATE = (unset), + LC_TIME = (unset), + LC_MESSAGES = (unset), + LC_MONETARY = (unset), + LC_ADDRESS = (unset), + LC_IDENTIFICATION = (unset), + LC_MEASUREMENT = (unset), + LC_PAPER = (unset), + LC_TELEPHONE = (unset), + LC_NAME = (unset), + LANG = "en_US.UTF-8" + are supported and installed on your system. +perl: warning: Falling back to the standard locale ("C"). +locale: Cannot set LC_CTYPE to default locale: No such file or directory +locale: Cannot set LC_MESSAGES to default locale: No such file or directory +locale: Cannot set LC_ALL to default locale: No such file or directory +apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct! +perl: warning: Setting locale failed. +perl: warning: Please check that your locale settings: + LANGUAGE = (unset), + LC_ALL = (unset), + LC_CTYPE = (unset), + LC_NUMERIC = (unset), + LC_COLLATE = (unset), + LC_TIME = (unset), + LC_MESSAGES = (unset), + LC_MONETARY = (unset), + LC_ADDRESS = (unset), + LC_IDENTIFICATION = (unset), + LC_MEASUREMENT = (unset), + LC_PAPER = (unset), + LC_TELEPHONE = (unset), + LC_NAME = (unset), + LANG = "en_US.UTF-8" + are supported and installed on your system. +perl: warning: Falling back to the standard locale ("C"). +locale: Cannot set LC_CTYPE to default locale: No such file or directory +locale: Cannot set LC_MESSAGES to default locale: No such file or directory +locale: Cannot set LC_ALL to default locale: No such file or directory +[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation … +[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds … +[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) … +Unable to find image 'hello-world:latest' locally +latest: Pulling from library/hello-world +4f55086f7dd0: Pulling fs layer +4f55086f7dd0: Download complete +4f55086f7dd0: Pull complete +Digest: sha256:7f4da0fc94bcece205a8c0b6f4d11c8196924654ffe5c4d1aa439b7f632048b2 +Status: Downloaded newer image for hello-world:latest + docker OK (overlay2; data-root /var/lib/docker) + /var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4 + /mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4 + both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576 +[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.209.0 (no registry cred at deploy) … + +WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'. +Configure a credential helper to remove this warning. See +https://docs.docker.com/go/credential-store/ + +0.209.0: Pulling from admin/felhom-controller +039e6f9f9752: Pulling fs layer +80fbb63a1167: Pulling fs layer +9d6c7a1331cb: Pulling fs layer +3f0ff4614786: Pulling fs layer +6fcb0c3e9ec6: Pulling fs layer +743ddd729c34: Pulling fs layer +6fcb0c3e9ec6: Waiting +3f0ff4614786: Waiting +743ddd729c34: Waiting +039e6f9f9752: Verifying Checksum +039e6f9f9752: Download complete +3f0ff4614786: Verifying Checksum +3f0ff4614786: Download complete +9d6c7a1331cb: Verifying Checksum +9d6c7a1331cb: Download complete +6fcb0c3e9ec6: Verifying Checksum +6fcb0c3e9ec6: Download complete +743ddd729c34: Verifying Checksum +743ddd729c34: Download complete +80fbb63a1167: Verifying Checksum +80fbb63a1167: Download complete +039e6f9f9752: Pull complete +80fbb63a1167: Pull complete +9d6c7a1331cb: Pull complete +3f0ff4614786: Pull complete +6fcb0c3e9ec6: Pull complete +743ddd729c34: Pull complete +Digest: sha256:a4f5dc719077b0ee567533d6220787ab783b4f9af8f673b39353c5d8daae46c8 +Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.209.0 +gitea.dooplex.hu/admin/felhom-controller:0.209.0 +[golden] asking the controller which infra images it manages … +[golden] baking infra images (4): traefik:v3.6.7 cloudflare/cloudflared:2026.6.0 gtstef/filebrowser:1.3.3-stable gitea.dooplex.hu/admin/felhom-samba:1.1.0 … +v3.6.7: Pulling from library/traefik +589002ba0eae: Pulling fs layer +ef63511ea6cc: Pulling fs layer +0738e5cb835e: Pulling fs layer +3e6813f70c64: Pulling fs layer +3e6813f70c64: Waiting +ef63511ea6cc: Verifying Checksum +ef63511ea6cc: Download complete +589002ba0eae: Verifying Checksum +589002ba0eae: Download complete +3e6813f70c64: Verifying Checksum +3e6813f70c64: Download complete +0738e5cb835e: Verifying Checksum +0738e5cb835e: Download complete +589002ba0eae: Pull complete +ef63511ea6cc: Pull complete +0738e5cb835e: Pull complete +3e6813f70c64: Pull complete +Digest: sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a +Status: Downloaded newer image for traefik:v3.6.7 +docker.io/library/traefik:v3.6.7 +2026.6.0: Pulling from cloudflare/cloudflared +47de5dd0b812: Pulling fs layer +c172f21841df: Pulling fs layer +99515e7b4d35: Pulling fs layer +99ba982a9142: Pulling fs layer +d6b1b89eccac: Pulling fs layer +2780920e5dbf: Pulling fs layer +7c12895b777b: Pulling fs layer +3214acf345c0: Pulling fs layer +52630fc75a18: Pulling fs layer +dd64bf2dd177: Pulling fs layer +b839dfae01f6: Pulling fs layer +ebddc55facdc: Pulling fs layer +bdfd7f7e5bf6: Pulling fs layer +2d4d7adf6272: Pulling fs layer +40008157d8d2: Pulling fs layer +bd8962e29291: Pulling fs layer +cac2ae0193cb: Pulling fs layer +74d1dac84ecc: Pulling fs layer +dd64bf2dd177: Waiting +b839dfae01f6: Waiting +ebddc55facdc: Waiting +bdfd7f7e5bf6: Waiting +2d4d7adf6272: Waiting +40008157d8d2: Waiting +bd8962e29291: Waiting +cac2ae0193cb: Waiting +74d1dac84ecc: Waiting +99ba982a9142: Waiting +d6b1b89eccac: Waiting +2780920e5dbf: Waiting +7c12895b777b: Waiting +3214acf345c0: Waiting +52630fc75a18: Waiting +c172f21841df: Verifying Checksum +c172f21841df: Download complete +99515e7b4d35: Download complete +47de5dd0b812: Download complete +99ba982a9142: Verifying Checksum +99ba982a9142: Download complete +d6b1b89eccac: Verifying Checksum +d6b1b89eccac: Download complete +2780920e5dbf: Verifying Checksum +2780920e5dbf: Download complete +7c12895b777b: Verifying Checksum +7c12895b777b: Download complete +3214acf345c0: Download complete +52630fc75a18: Verifying Checksum +52630fc75a18: Download complete +dd64bf2dd177: Download complete +b839dfae01f6: Verifying Checksum +b839dfae01f6: Download complete +ebddc55facdc: Verifying Checksum +ebddc55facdc: Download complete +47de5dd0b812: Pull complete +bdfd7f7e5bf6: Download complete +2d4d7adf6272: Verifying Checksum +2d4d7adf6272: Download complete +bd8962e29291: Verifying Checksum +bd8962e29291: Download complete +40008157d8d2: Verifying Checksum +40008157d8d2: Download complete +cac2ae0193cb: Verifying Checksum +cac2ae0193cb: Download complete +c172f21841df: Pull complete +74d1dac84ecc: Verifying Checksum +74d1dac84ecc: Download complete +99515e7b4d35: Pull complete +99ba982a9142: Pull complete +d6b1b89eccac: Pull complete +2780920e5dbf: Pull complete +7c12895b777b: Pull complete +3214acf345c0: Pull complete +52630fc75a18: Pull complete +dd64bf2dd177: Pull complete +b839dfae01f6: Pull complete +ebddc55facdc: Pull complete +bdfd7f7e5bf6: Pull complete +2d4d7adf6272: Pull complete +40008157d8d2: Pull complete +bd8962e29291: Pull complete +cac2ae0193cb: Pull complete +74d1dac84ecc: Pull complete +Digest: sha256:ba461b8aa9c042156dbd39c38657fe7431bafa063220eab8d5330a523863da9f +Status: Downloaded newer image for cloudflare/cloudflared:2026.6.0 +docker.io/cloudflare/cloudflared:2026.6.0 +1.3.3-stable: Pulling from gtstef/filebrowser +6a0ac1617861: Pulling fs layer +ef8806083e82: Pulling fs layer +b74107c861c7: Pulling fs layer +adc935def003: Pulling fs layer +4f4fb700ef54: Pulling fs layer +18695ccc900a: Pulling fs layer +45d119d5c397: Pulling fs layer +dac52db4fc51: Pulling fs layer +6d598f86b2f2: Pulling fs layer +8aa349c8396c: Pulling fs layer +adc935def003: Waiting +4f4fb700ef54: Waiting +18695ccc900a: Waiting +45d119d5c397: Waiting +dac52db4fc51: Waiting +6d598f86b2f2: Waiting +8aa349c8396c: Waiting +6a0ac1617861: Verifying Checksum +6a0ac1617861: Download complete +adc935def003: Verifying Checksum +adc935def003: Download complete +b74107c861c7: Verifying Checksum +b74107c861c7: Download complete +4f4fb700ef54: Verifying Checksum +4f4fb700ef54: Download complete +45d119d5c397: Verifying Checksum +45d119d5c397: Download complete +dac52db4fc51: Verifying Checksum +dac52db4fc51: Download complete +6a0ac1617861: Pull complete +18695ccc900a: Verifying Checksum +18695ccc900a: Download complete +6d598f86b2f2: Verifying Checksum +6d598f86b2f2: Download complete +ef8806083e82: Download complete +8aa349c8396c: Verifying Checksum +8aa349c8396c: Download complete +ef8806083e82: Pull complete +b74107c861c7: Pull complete +adc935def003: Pull complete +4f4fb700ef54: Pull complete +18695ccc900a: Pull complete +45d119d5c397: Pull complete +dac52db4fc51: Pull complete +6d598f86b2f2: Pull complete +8aa349c8396c: Pull complete +Digest: sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70babab78c +Status: Downloaded newer image for gtstef/filebrowser:1.3.3-stable +docker.io/gtstef/filebrowser:1.3.3-stable +1.1.0: Pulling from admin/felhom-samba +897d797d2723: Pulling fs layer +3051591aa250: Pulling fs layer +ce57a3f93416: Pulling fs layer +fb94eeec2fe1: Pulling fs layer +fb94eeec2fe1: Waiting +ce57a3f93416: Verifying Checksum +ce57a3f93416: Download complete +897d797d2723: Verifying Checksum +897d797d2723: Download complete +fb94eeec2fe1: Verifying Checksum +fb94eeec2fe1: Download complete +897d797d2723: Pull complete +3051591aa250: Verifying Checksum +3051591aa250: Download complete +3051591aa250: Pull complete +ce57a3f93416: Pull complete +fb94eeec2fe1: Pull complete +Digest: sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10 +Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.1.0 +gitea.dooplex.hu/admin/felhom-samba:1.1.0 +[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'. +[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'. +[golden] baking the first-boot SSH host-key regeneration unit (F3) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'. +[golden] identity-clean + minimize … +[golden] stop + archive … +INFO: including mount point rootfs ('/') in backup +INFO: including mount point mp0 ('/var/lib/felhom') in backup +INFO: archive file size: 626MB +INFO: Finished Backup of VM 9100 (00:00:29) +[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_08_08-08_55_52.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive) +[golden] publishing golden (656697956 bytes, sha256 c9c4bcd6d9f53dca…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.209.0/golden.tar.zst +[golden] pre-delete existing: HTTP 404 (404/204 expected) +[golden] upload OK (HTTP 201) +GOLDEN_VERSION=0.209.0 +GOLDEN_SHA256=c9c4bcd6d9f53dcaf3c7a56ffce1c401864a5564af4eca8ff5efd8d5eae818ff +[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.209.0 / c9c4bcd6d9f53dcaf3c7a56ffce1c401864a5564af4eca8ff5efd8d5eae818ff +[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)