diff --git a/STATUS.md b/STATUS.md index b94d6af0..be22eb1c 100644 --- a/STATUS.md +++ b/STATUS.md @@ -1,8 +1,8 @@ # STATUS — what works, what's broken, what's next -**Updated 2026-09-01 — I ran the week's six releases against each other overnight. Three things -broke, none of them where I expected. The worst: the backup-content check I shipped yesterday -CANNOT RUN AT ALL on a machine set up like `demo-felhom`, and it fails silently every night.** +**Updated 2026-09-01 (second pass) — both faults from the overnight test are fixed, and the golden +is baked, vouched and delivered. Both machines are on 0.232.0 and moved themselves. NOTHING is +waiting on you.** **Earlier 2026-08-31 — I measured whether the box could test its own remote restore without you, then built the narrow version you picked. The measurement is why it is 3 seconds a night and @@ -18,33 +18,26 @@ not an evening's work.** *This section is allowed to be longer than one screen, and each item says what happens if you do nothing.* -1. **The new nightly backup check does not work on `demo-felhom`** (R-414). It ran for the first - time on its own last night and could not start: that machine has no registered data drive, so - there is nowhere to unpack a backup to look at it. It will fail the same way every night, and - the only trace is one line in a log — the hub cannot tell it apart from a machine that never - had the feature. **Your backups there are fine** — the backup itself ran normally. It is the - CHECK that cannot run. **If you do nothing:** that machine, and any like it, silently never - gets the check. I have not changed anything; the fix is a small decision about where it should - unpack to, and it is yours. +1. **Nothing is waiting on you.** Both problems the overnight test found are fixed and proven on + the real machines: + - the background job that could delete a live restore's lock now waits its turn — and the check + that finds the next one like it is a test, not a comment, so it cannot come back quietly; + - the nightly backup check now runs on `demo-felhom`. It said **pass** there this morning, on a + machine where it could not start at all yesterday. + The golden carrying both is baked, vouched, and the floor is raised. `demo-felhom` picked it up + **by itself in about 20 seconds**. -2. **A golden carrying 0.231.0 is owed.** Golden **0.230.0** is baked, vouched and on both - machines. **0.231.0 is the build with the new nightly backup-content check**, and it is on - `demo-hp` only. **If you do nothing:** the check stays on one machine; a newly installed box - does not get it, and neither does `demo-felhom`. Nothing breaks — this adds a check, it does not - fix a defect. Baking and vouching 0.231.0 and raising the floor closes it, the same three-field - change as before. - -3. **Whether to keep the test app `bentopdf` on `demo-hp`.** I deployed it last night because it +2. **Whether to keep the test app `bentopdf` on `demo-hp`.** I deployed it last night because it is the ONLY app of our 53 with neither a database nor stored files — which makes it the only way to prove the new check does not cry wolf on an app that legitimately has nothing. It passed silently, which is what we needed to see. **My pick: keep it**, as a permanent control. **If you do nothing:** it stays, using almost no space. Say the word and I remove it. -4. **Nothing else about this release.** Everything in 0.231.0 ships in the controller image plus +3. **Nothing else about this release.** Everything in 0.232.0 ships in the controller image plus two register lines in the hub (already live). No customer action, no data migration, no credential change. -5. **Whether a documents-only push should still be checked for a missing golden** (R-404). We have now +4. **Whether a documents-only push should still be checked for a missing golden** (R-404). We have now skipped that check **eight times**, each time for a written reason: it runs on every push to the website/documentation repository, including pushes that change nothing a machine installs. **A guard we correctly skip eight times is teaching us to skip it.** @@ -56,11 +49,11 @@ nothing.* **If you do nothing:** nothing breaks, the skipping stays routine, and the count keeps rising. I have NOT changed it; this is yours to decide and mine to build. -6. **Whether to change the hub password** (R-350). I printed it into my own session log on 20 August. +5. **Whether to change the hub password** (R-350). I printed it into my own session log on 20 August. Not in git, not in any saved file — in the log on this machine. **If you do nothing:** it stays as it is, at the risk you accept by leaving it. I can change it without ever showing you the new one. -7. **`demo-hp`'s network setup does not match our own notes** (R-338) — the machine works, the page is +6. **`demo-hp`'s network setup does not match our own notes** (R-338) — the machine works, the page is wrong, or the other way round. **If you do nothing:** the page keeps misleading the next session, as it misled one by an hour. diff --git a/documentation/tests/golden-0.232.0-2026-09-01/01-preconditions.txt b/documentation/tests/golden-0.232.0-2026-09-01/01-preconditions.txt new file mode 100644 index 00000000..dd4324a2 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/01-preconditions.txt @@ -0,0 +1,17 @@ +=== preconditions 2026-09-01T08:37:09Z === +/dev/sda1 9.1T 3.1T 5.5T 37% /mnt/5_hdd +/dev/sdb1 445G 217G 206G 52% / +felhom-controller clean=[] HEAD=62c6a8a origin=62c6a8a +felhom.eu clean=[?? documentation/tests/golden-0.232.0-2026-09-01/;] HEAD=f41a1a0 origin=f41a1a0 +MinAgent for v0.232.0: **MinAgent: 0.129.0** (unchanged) +controller image 0.232.0 in registry: HTTP 200 +404 pre-gate on the golden package: HTTP 404 + +=== drill VM === +Snapshot list: +ID TAG VM_SIZE DATE VM_CLOCK ICOUNT +1 virgin 0 B 2026-07-03 14:12:15 0000:00:00.000 0 +no qemu running + +=== bake script fingerprint on DooPlex (to be compared across the hop - the 0.230.0 bake skipped this) === +7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1 /mnt/5_hdd/felhom.eu/git/felhom-agent/configs/build-golden.sh diff --git a/documentation/tests/golden-0.232.0-2026-09-01/02-template.txt b/documentation/tests/golden-0.232.0-2026-09-01/02-template.txt new file mode 100644 index 00000000..f0baf3b7 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/02-template.txt @@ -0,0 +1,9 @@ +=== pveam update (the virgin snapshot's INDEX is stale too) === +update successful +system debian-13-standard_13.6-1_amd64.tar.zst +calculating checksum...OK, checksum verified +download of 'http://download.proxmox.com/images/system/debian-13-standard_13.6-1_amd64.tar.zst' to '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst' finished + +=== THE FINGERPRINT COMPARISON ACROSS THE HOP — the 0.230.0 bake skipped this and said so === + DooPlex : 7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1 + in the VM: 7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1 diff --git a/documentation/tests/golden-0.232.0-2026-09-01/03-bake-launch.txt b/documentation/tests/golden-0.232.0-2026-09-01/03-bake-launch.txt new file mode 100644 index 00000000..152fe2f8 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/03-bake-launch.txt @@ -0,0 +1,6 @@ +Running as unit: golden-bake.service; invocation ID: 4067665db1c6467ab5385dd87754cfa3 +=== token-leak check on the unit properties (must be 0) === +0 +=== and the grep is PROVEN to work (must be 1) === +1 +active diff --git a/documentation/tests/golden-0.232.0-2026-09-01/04-markers.txt b/documentation/tests/golden-0.232.0-2026-09-01/04-markers.txt new file mode 100644 index 00000000..33d318f8 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/04-markers.txt @@ -0,0 +1,17 @@ +=== acceptance markers, counted on the COMMITTED log === + docker OK (overlay2 : 1 + including mount point : + 310:INFO: including mount point rootfs ('/') in backup + 311:INFO: including mount point mp0 ('/var/lib/felhom') in backup + upload OK (HTTP 201) : 1 + MUST BE ZERO - excluding : 0 + MUST BE ZERO - FATAL : 0 + +=== the result === +GOLDEN_VERSION=0.232.0 +GOLDEN_SHA256=5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8 + archive volid: local:backup/vzdump-lxc-9100-2026_09_01-10_43_08.tar.zst + +=== token-leak grep on the COMMITTED log (must be 0), and the grep PROVEN to work === + bake.log: 0 + seeded throwaway copy (must be 1): 1 diff --git a/documentation/tests/golden-0.232.0-2026-09-01/05-teardown.txt b/documentation/tests/golden-0.232.0-2026-09-01/05-teardown.txt new file mode 100644 index 00000000..25cfdc13 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/05-teardown.txt @@ -0,0 +1,4 @@ + Logical volume "vm-9100-disk-1" successfully removed. +purging CT 9100 from related configurations.. + leftovers in the VM: 0 +1 virgin 0 B 2026-07-03 14:12:15 0000:00:00.000 0 diff --git a/documentation/tests/golden-0.232.0-2026-09-01/06-bake-clean.log b/documentation/tests/golden-0.232.0-2026-09-01/06-bake-clean.log new file mode 100644 index 00000000..01e1e3d9 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/06-bake-clean.log @@ -0,0 +1,315 @@ +[golden] build-golden.sh v3.0.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.232.0 +[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) … + Logical volume "vm-9100-disk-0" created. + Logical volume pve/vm-9100-disk-0 changed. +Creating filesystem with 8388608 4k blocks and 2097152 inodes +Filesystem UUID: 60543827-d756-4347-b65a-2d399dffa214 +Superblock backups stored on blocks: + 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, + 4096000, 7962624 + Logical volume "vm-9100-disk-1" created. + Logical volume pve/vm-9100-disk-1 changed. +Creating filesystem with 6291456 4k blocks and 1572864 inodes +Filesystem UUID: 6214059f-3782-4307-a844-45a978831072 +Superblock backups stored on blocks: + 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, +extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst' +Total bytes read: 553512960 (528MiB, 98MiB/s) +Detected container architecture: amd64 +Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ... +done: SHA256:Wmq2LMDE0vR5hKbvaGiv+/J2PEtk2Jx456YNlTDwNcw root@felhom-golden +Creating SSH host key 'ssh_host_rsa_key' - this may take some time ... +done: SHA256:T/Qo3J9usornV+MhvzdnqKwX0f84ysnnMfIplLSyYCQ root@felhom-golden +Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ... +done: SHA256:c76v869yvL57EFiXagl57F1hvUMkwdWAKHbrJsSKHfQ root@felhom-golden +[golden] starting + installing Docker (official repo, trixie channel) … +apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct! +perl: warning: Setting locale failed. +perl: warning: Please check that your locale settings: + LANGUAGE = (unset), + LC_ALL = (unset), + LC_CTYPE = (unset), + LC_NUMERIC = (unset), + LC_COLLATE = (unset), + LC_TIME = (unset), + LC_MESSAGES = (unset), + LC_MONETARY = (unset), + LC_ADDRESS = (unset), + LC_IDENTIFICATION = (unset), + LC_MEASUREMENT = (unset), + LC_PAPER = (unset), + LC_TELEPHONE = (unset), + LC_NAME = (unset), + LANG = "en_US.UTF-8" + are supported and installed on your system. +perl: warning: Falling back to the standard locale ("C"). +apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct! +perl: warning: Setting locale failed. +perl: warning: Please check that your locale settings: + LANGUAGE = (unset), + LC_ALL = (unset), + LC_CTYPE = (unset), + LC_NUMERIC = (unset), + LC_COLLATE = (unset), + LC_TIME = (unset), + LC_MESSAGES = (unset), + LC_MONETARY = (unset), + LC_ADDRESS = (unset), + LC_IDENTIFICATION = (unset), + LC_MEASUREMENT = (unset), + LC_PAPER = (unset), + LC_TELEPHONE = (unset), + LC_NAME = (unset), + LANG = "en_US.UTF-8" + are supported and installed on your system. +perl: warning: Falling back to the standard locale ("C"). +[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation … +[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds … +[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) … +Unable to find image 'hello-world:latest' locally +latest: Pulling from library/hello-world +4f55086f7dd0: Pulling fs layer +4f55086f7dd0: Download complete +4f55086f7dd0: Pull complete +Digest: sha256:5dd0d3e6e255913fc30f90b9f2b1d359cc2cbdb48090cc4b65f1676e203243cc +Status: Downloaded newer image for hello-world:latest + docker OK (overlay2; data-root /var/lib/docker) + /var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4 + /mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4 + both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576 +[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.232.0 (no registry cred at deploy) … + +WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'. +Configure a credential helper to remove this warning. See +https://docs.docker.com/go/credential-store/ + +0.232.0: Pulling from admin/felhom-controller +a8ac7f6c67ab: Pulling fs layer +bf30769d36e7: Pulling fs layer +044b66fbe46c: Pulling fs layer +b5c41a28e83f: Pulling fs layer +e39b6b4d6e0d: Pulling fs layer +7f86327acbd2: Pulling fs layer +b5c41a28e83f: Waiting +e39b6b4d6e0d: Waiting +7f86327acbd2: Waiting +a8ac7f6c67ab: Verifying Checksum +a8ac7f6c67ab: Download complete +b5c41a28e83f: Verifying Checksum +b5c41a28e83f: Download complete +e39b6b4d6e0d: Verifying Checksum +e39b6b4d6e0d: Download complete +7f86327acbd2: Verifying Checksum +7f86327acbd2: Download complete +044b66fbe46c: Verifying Checksum +044b66fbe46c: Download complete +bf30769d36e7: Verifying Checksum +bf30769d36e7: Download complete +a8ac7f6c67ab: Pull complete +bf30769d36e7: Pull complete +044b66fbe46c: Pull complete +b5c41a28e83f: Pull complete +e39b6b4d6e0d: Pull complete +7f86327acbd2: Pull complete +Digest: sha256:51b2403520e425e0a7d4bc00cfd395d3b06078d79cf7927bee1035d662814b8c +Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.232.0 +gitea.dooplex.hu/admin/felhom-controller:0.232.0 +[golden] asking the controller which infra images it manages … +[golden] baking infra images (4): traefik:v3.6.7 cloudflare/cloudflared:2026.6.0 gtstef/filebrowser:1.3.3-stable gitea.dooplex.hu/admin/felhom-samba:1.1.0 … +v3.6.7: Pulling from library/traefik +589002ba0eae: Pulling fs layer +ef63511ea6cc: Pulling fs layer +0738e5cb835e: Pulling fs layer +3e6813f70c64: Pulling fs layer +3e6813f70c64: Waiting +ef63511ea6cc: Download complete +589002ba0eae: Verifying Checksum +589002ba0eae: Download complete +3e6813f70c64: Verifying Checksum +3e6813f70c64: Download complete +0738e5cb835e: Verifying Checksum +0738e5cb835e: Download complete +589002ba0eae: Pull complete +ef63511ea6cc: Pull complete +0738e5cb835e: Pull complete +3e6813f70c64: Pull complete +Digest: sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a +Status: Downloaded newer image for traefik:v3.6.7 +docker.io/library/traefik:v3.6.7 +2026.6.0: Pulling from cloudflare/cloudflared +47de5dd0b812: Pulling fs layer +c172f21841df: Pulling fs layer +99515e7b4d35: Pulling fs layer +99ba982a9142: Pulling fs layer +d6b1b89eccac: Pulling fs layer +2780920e5dbf: Pulling fs layer +7c12895b777b: Pulling fs layer +3214acf345c0: Pulling fs layer +52630fc75a18: Pulling fs layer +dd64bf2dd177: Pulling fs layer +b839dfae01f6: Pulling fs layer +ebddc55facdc: Pulling fs layer +bdfd7f7e5bf6: Pulling fs layer +2d4d7adf6272: Pulling fs layer +40008157d8d2: Pulling fs layer +bd8962e29291: Pulling fs layer +cac2ae0193cb: Pulling fs layer +74d1dac84ecc: Pulling fs layer +dd64bf2dd177: Waiting +b839dfae01f6: Waiting +ebddc55facdc: Waiting +bdfd7f7e5bf6: Waiting +2d4d7adf6272: Waiting +40008157d8d2: Waiting +bd8962e29291: Waiting +cac2ae0193cb: Waiting +74d1dac84ecc: Waiting +2780920e5dbf: Waiting +7c12895b777b: Waiting +3214acf345c0: Waiting +52630fc75a18: Waiting +99ba982a9142: Waiting +d6b1b89eccac: Waiting +c172f21841df: Verifying Checksum +c172f21841df: Download complete +47de5dd0b812: Download complete +99515e7b4d35: Verifying Checksum +99515e7b4d35: Download complete +99ba982a9142: Verifying Checksum +99ba982a9142: Download complete +d6b1b89eccac: Download complete +2780920e5dbf: Verifying Checksum +2780920e5dbf: Download complete +7c12895b777b: Verifying Checksum +7c12895b777b: Download complete +3214acf345c0: Verifying Checksum +3214acf345c0: Download complete +47de5dd0b812: Pull complete +52630fc75a18: Verifying Checksum +52630fc75a18: Download complete +dd64bf2dd177: Verifying Checksum +dd64bf2dd177: Download complete +b839dfae01f6: Verifying Checksum +b839dfae01f6: Download complete +ebddc55facdc: Verifying Checksum +ebddc55facdc: Download complete +bdfd7f7e5bf6: Download complete +2d4d7adf6272: Verifying Checksum +2d4d7adf6272: Download complete +bd8962e29291: Download complete +c172f21841df: Pull complete +40008157d8d2: Download complete +cac2ae0193cb: Verifying Checksum +cac2ae0193cb: Download complete +74d1dac84ecc: Verifying Checksum +74d1dac84ecc: Download complete +99515e7b4d35: Pull complete +99ba982a9142: Pull complete +d6b1b89eccac: Pull complete +2780920e5dbf: Pull complete +7c12895b777b: Pull complete +3214acf345c0: Pull complete +52630fc75a18: Pull complete +dd64bf2dd177: Pull complete +b839dfae01f6: Pull complete +ebddc55facdc: Pull complete +bdfd7f7e5bf6: Pull complete +2d4d7adf6272: Pull complete +40008157d8d2: Pull complete +bd8962e29291: Pull complete +cac2ae0193cb: Pull complete +74d1dac84ecc: Pull complete +Digest: sha256:ba461b8aa9c042156dbd39c38657fe7431bafa063220eab8d5330a523863da9f +Status: Downloaded newer image for cloudflare/cloudflared:2026.6.0 +docker.io/cloudflare/cloudflared:2026.6.0 +1.3.3-stable: Pulling from gtstef/filebrowser +6a0ac1617861: Pulling fs layer +ef8806083e82: Pulling fs layer +b74107c861c7: Pulling fs layer +adc935def003: Pulling fs layer +4f4fb700ef54: Pulling fs layer +18695ccc900a: Pulling fs layer +45d119d5c397: Pulling fs layer +dac52db4fc51: Pulling fs layer +6d598f86b2f2: Pulling fs layer +8aa349c8396c: Pulling fs layer +adc935def003: Waiting +4f4fb700ef54: Waiting +18695ccc900a: Waiting +45d119d5c397: Waiting +dac52db4fc51: Waiting +6d598f86b2f2: Waiting +8aa349c8396c: Waiting +6a0ac1617861: Verifying Checksum +6a0ac1617861: Download complete +adc935def003: Verifying Checksum +adc935def003: Download complete +b74107c861c7: Verifying Checksum +b74107c861c7: Download complete +4f4fb700ef54: Verifying Checksum +4f4fb700ef54: Download complete +6a0ac1617861: Pull complete +45d119d5c397: Verifying Checksum +45d119d5c397: Download complete +18695ccc900a: Verifying Checksum +18695ccc900a: Download complete +dac52db4fc51: Verifying Checksum +dac52db4fc51: Download complete +6d598f86b2f2: Verifying Checksum +6d598f86b2f2: Download complete +8aa349c8396c: Verifying Checksum +8aa349c8396c: Download complete +ef8806083e82: Download complete +ef8806083e82: Pull complete +b74107c861c7: Pull complete +adc935def003: Pull complete +4f4fb700ef54: Pull complete +18695ccc900a: Pull complete +45d119d5c397: Pull complete +dac52db4fc51: Pull complete +6d598f86b2f2: Pull complete +8aa349c8396c: Pull complete +Digest: sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70babab78c +Status: Downloaded newer image for gtstef/filebrowser:1.3.3-stable +docker.io/gtstef/filebrowser:1.3.3-stable +1.1.0: Pulling from admin/felhom-samba +897d797d2723: Pulling fs layer +3051591aa250: Pulling fs layer +ce57a3f93416: Pulling fs layer +fb94eeec2fe1: Pulling fs layer +fb94eeec2fe1: Waiting +ce57a3f93416: Verifying Checksum +ce57a3f93416: Download complete +897d797d2723: Verifying Checksum +897d797d2723: Download complete +fb94eeec2fe1: Verifying Checksum +fb94eeec2fe1: Download complete +3051591aa250: Verifying Checksum +3051591aa250: Download complete +897d797d2723: Pull complete +3051591aa250: Pull complete +ce57a3f93416: Pull complete +fb94eeec2fe1: Pull complete +Digest: sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10 +Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.1.0 +gitea.dooplex.hu/admin/felhom-samba:1.1.0 +[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'. +[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'. +[golden] baking the first-boot SSH host-key regeneration unit (F3) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'. +[golden] identity-clean + minimize … +[golden] stop + archive … +INFO: including mount point rootfs ('/') in backup +INFO: including mount point mp0 ('/var/lib/felhom') in backup +INFO: archive file size: 627MB +INFO: Finished Backup of VM 9100 (00:00:30) +[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_09_01-10_43_08.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive) +[golden] publishing golden (657494489 bytes, sha256 5f8a53ed5b19a6cb…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.232.0/golden.tar.zst +[golden] pre-delete existing: HTTP 404 (404/204 expected) +[golden] upload OK (HTTP 201) +GOLDEN_VERSION=0.232.0 +GOLDEN_SHA256=5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8 +[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.232.0 / 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8 +[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge) diff --git a/documentation/tests/golden-0.232.0-2026-09-01/07-roundtrip.txt b/documentation/tests/golden-0.232.0-2026-09-01/07-roundtrip.txt new file mode 100644 index 00000000..39571f84 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/07-roundtrip.txt @@ -0,0 +1,8 @@ +=== READER 2: the ROUND TRIP — hash the DOWNLOADED bytes, not the bake's printout === + http=200 bytes=657494489 + downloaded sha256: 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8 + bake printed : 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8 + +=== THE DELIVERED ARTIFACT MUST NAME ITS OWN CONTROLLER === + gitea.dooplex.hu/admin/felhom-controller:0.232.0 + docker store entries in the archive: 19382 diff --git a/documentation/tests/golden-0.232.0-2026-09-01/08-hub-before-vouch.txt b/documentation/tests/golden-0.232.0-2026-09-01/08-hub-before-vouch.txt new file mode 100644 index 00000000..6372f58e --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/08-hub-before-vouch.txt @@ -0,0 +1,8 @@ +=== READER 3: the hub's Day-0 dropdown, reading Gitea on a different code path === + sha=... + 0.232.0 sha=5f8a53ed5b19a6cb2006298c... + 0.230.0 sha=9287f7cef5f13166276e8406... <-- currently SELECTED + 0.229.0 sha=39aa886df77b21757aef3b29... + has 0.232.0: ['0.232.0'] + current fleet floor: 0.230.0 + agent_version selected: 0.130.0 diff --git a/documentation/tests/golden-0.232.0-2026-09-01/09-vouch.txt b/documentation/tests/golden-0.232.0-2026-09-01/09-vouch.txt new file mode 100644 index 00000000..01a43b7c --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/09-vouch.txt @@ -0,0 +1,9 @@ +=== VOUCH: the THREE fields, moved together === + golden_version 0.230.0 -> 0.232.0 (this bake carries TWO releases: 0.231.0 was never baked) + agent_version 0.130.0 -> 0.130.0 UNCHANGED, already >= MinAgent + min_agent 0.129.0 -> 0.129.0 UNCHANGED, v0.232.0's CHANGELOG says MinAgent 0.129.0 + min_agent (0.129.0) <= agent_version (0.130.0): NOT the R-216 shape + +POST http=303 + Location: /configuration?flash=artifacts_set + (read from -D, never %{redirect_url} — R-132) diff --git a/documentation/tests/golden-0.232.0-2026-09-01/10-hub-after-vouch.html b/documentation/tests/golden-0.232.0-2026-09-01/10-hub-after-vouch.html new file mode 100644 index 00000000..298f947d --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/10-hub-after-vouch.html @@ -0,0 +1,286 @@ + + + + + + Configuration — Felhom Hub + + + + +
+
+

Felhom Hub

+ +
+ +

Configuration

+ + + + + + + + + + + + + + + + + + + + + + +
+

Managed updates — global floor

+

+ The minimum controller version every box auto-updates to (unless a per-customer override is set). + Saving takes effect immediately — boxes below the floor update on their next + report, no customer action. Blank = no global floor. This setting is independent of the Day-0 + artifact manifest below. +

+ +

+ Effective floor: + v0.230.0 + + — source: DB (hub_settings); env fallback would be v0.120.0 + +

+
+ + + + DB override: v0.230.0 +
+ + +
+ + +
+

Day-0 artifacts — agent & golden

+

+ The current agent binary + golden archive the host-bootstrap script fetches from Gitea and + verifies (sha256) before installing. The hub vouches for these checksums (a different trust + root than Gitea). Pick a version — the sha256 is read from Gitea automatically (no manual + copy). Choose — none — to clear an artifact. +

+
+ + + + + + + + + + + + + + The golden's controller CHANGELOG MinAgent:. The hub HOLDS the floor for any box whose agent is below this — blank = uncoupled release, no gating. + + + + sha256 of configs/felhom-pbs-apply (R-50b). Unlike the agent and golden this root-owned wrapper is installed from raw/branch/main — unversioned and unpinned. Recording it here does not fix the channel; it makes host drift visible: agents report the installed file's hash and a mismatch is surfaced on the host page. + + +
+ +
+ + +
+

Login password

+

+ The password for signing in to this hub UI. Changing it takes effect immediately + for the next sign-in — your current session stays logged in. Enter your current password to confirm. + If you ever lose it, the deployment ConfigMap (auth.password_hash) remains the reset path. +

+
+ + + + + + + + + + + + +
+ +
+ + +
+

Assets

+

+ App logos and screenshots served to controllers. Assets are seeded from the Docker image + and synced to controllers daily via the asset manifest API. +

+
+
+ Files in manifest + 211 +
+
+ Manifest generated + 2026-08-31T18:54:31Z +
+
+
+ + + +
+

+ Re-reads the baked-in asset seed directory and updates changed files. + Controllers will pick up changes on their next daily sync or manual trigger. +

+
+ + +
+ + diff --git a/documentation/tests/golden-0.232.0-2026-09-01/11-vouch-verified.txt b/documentation/tests/golden-0.232.0-2026-09-01/11-vouch-verified.txt new file mode 100644 index 00000000..1aaea6ee --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/11-vouch-verified.txt @@ -0,0 +1,11 @@ +=== RE-READ FROM THE PAGE — a 303 and a success flash are not proof === + agent_version SELECTED = 0.130.0 sha=a56a92a7bd68f5b46736eaec... + golden_version SELECTED = 0.232.0 sha=5f8a53ed5b19a6cb2006298c... + min_agent = 0.129.0 + golden_sha256 = 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8 + R-120 refusal banner present: False + +=== raise the fleet floor 0.230.0 -> 0.232.0 (Viktor agreed to this step) === + POST http=303 + Location: /configuration?flash=floor_set + after: name="min_controller_version" value="0.232.0" diff --git a/documentation/tests/golden-0.232.0-2026-09-01/12-selfupdate.txt b/documentation/tests/golden-0.232.0-2026-09-01/12-selfupdate.txt new file mode 100644 index 00000000..e2fed071 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/12-selfupdate.txt @@ -0,0 +1,20 @@ +=== both boxes were HAND-DEPLOYED during validation, so the floor had nothing to move. + Rolling demo-felhom back to 0.231.0 so the delivery chain is actually exercised. === +rolled-back-to-0.231.0 + now: gitea.dooplex.hu/admin/felhom-controller:0.232.0 + +=== waiting for the agent to pull it up to the floor === + MOVED ITSELF after ~20s + final: gitea.dooplex.hu/admin/felhom-controller:0.232.0 +=== demo-felhom's OWN agent journal — the self-update, in its words === + Sep 01 10:47:16 demo-felhom felhom-agent[2564828]: time=2026-09-01T10:47:16.761+02:00 level=INFO msg="controller-swap: image file written, restarting bootstrap" vmid=9201 target=gitea.dooplex.hu/admin/felhom-controller:0.232.0 + Sep 01 10:47:26 demo-felhom felhom-agent[2564828]: time=2026-09-01T10:47:26.188+02:00 level=INFO msg="controller-swap: new controller healthy" vmid=9201 target=gitea.dooplex.hu/admin/felhom-controller:0.232.0 + +=== both boxes, final === +hp: gitea.dooplex.hu/admin/felhom-controller:0.232.0 Up 17 minutes (healthy) +felhom-pve: gitea.dooplex.hu/admin/felhom-controller:0.232.0 Up 23 seconds (healthy) + +=== the golden gate — the first bake it actually gates (R-410) === + newest released controller : 0.232.0 (## v0.232.0 — one writer at a time, and a check that can actually run (2026-09-01, R-411/R) + newest golden baked : 0.232.0 (documentation/tests/golden-0.232.0-2026-09-01 [sha 5f8a53ed5b19…]) +golden currency gate OK — the newest released controller has a golden (NOTE: this checks the BAKE, not the vouch — see the module docstring) diff --git a/documentation/tests/golden-0.232.0-2026-09-01/README.md b/documentation/tests/golden-0.232.0-2026-09-01/README.md new file mode 100644 index 00000000..87dccea6 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/README.md @@ -0,0 +1,100 @@ +# Golden bake 0.232.0 — 2026-09-01 + +Baked, published, round-trip verified, **vouched**, and the fleet floor raised. +**This bake carries TWO releases: 0.231.0 was never baked**, so the fleet went 0.230.0 → 0.232.0. + +| | | +|---|---| +| `GOLDEN_SHA256` | `5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8` | +| size | **657 494 489 B** | +| baked controller | `gitea.dooplex.hu/admin/felhom-controller:0.232.0` | +| `MinAgent` | **0.129.0** — read from the controller CHANGELOG header, not assumed | +| script | `build-golden.sh v3.0.0`, sha `7b0fb5cf…73b6a1` | +| template | `debian-13-standard_13.6-1_amd64.tar.zst`, after `pveam update` | +| archive volid | `local:backup/vzdump-lxc-9100-2026_09_01-10_43_08.tar.zst` | + +## The check the 0.230.0 bake skipped, and this one did not + +**The bake script's fingerprint was compared ACROSS THE HOP:** + +``` +DooPlex : 7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1 +in the VM : 7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1 +``` + +The 0.230.0 bake recorded only the DooPlex-side hash and said so plainly. This one is a measurement. + +## Acceptance markers — counted on the COMMITTED log + +``` +docker OK (overlay2 : 1 +including mount point rootfs : 1 (line 310) +including mount point mp0 : 1 (line 311) <- no mp1 since v3.0.0 (R-165/R-233) +upload OK (HTTP 201) : 1 +--- must be ZERO --- +excluding : 0 +FATAL : 0 +``` + +## Three independent readers agreed before anything was vouched + +1. **The bake** printed `GOLDEN_SHA256=5f8a53ed…91b8`. +2. **The round trip** — the published bytes downloaded back: `HTTP 200`, **657 494 489 B**, sha + `5f8a53ed…91b8`. Hashed from the **downloaded** bytes, never the local file. +3. **The hub's Day-0 dropdown**, a different code path reading Gitea: `data-sha="5f8a53ed5b19a6cb…"`. + +**And the delivered artifact names the controller it will start**, read out of the downloaded archive: + +``` +$ tar --zstd -xOf golden.tar.zst ./etc/felhom-controller-image +gitea.dooplex.hu/admin/felhom-controller:0.232.0 +``` + +with **19 382** entries under `var/lib/felhom/docker/` — the image store is IN the archive. + +## Pre-gates, each proven able to see something first + +| gate | result | the control that makes it believable | +|---|---|---| +| 404 pre-gate | `HTTP 404` for 0.232.0 before the bake | the controller image manifest returns `HTTP 200` on the same host | +| token-leak grep on the committed log | **0** | the token appended to a throwaway copy greps **1**; the copy was `shred -u`'d | +| token off every command line | unit properties grep **0** | same seeded control returns **1** | + +## The vouch — three fields, and only one moved + +| field | before | after | why | +|---|---|---|---| +| `golden_version` | 0.230.0 | **0.232.0** | the new bake | +| `agent_version` | 0.130.0 | 0.130.0 | unchanged — already ≥ MinAgent | +| `min_agent` | 0.129.0 | 0.129.0 | unchanged — v0.232.0's header says `MinAgent: 0.129.0` | + +`min_agent` (0.129.0) ≤ `agent_version` (0.130.0), so **not** the R-216 shape. +`POST /configuration/artifacts` → `303 flash=artifacts_set`; **the flash was not treated as proof** — +the page was re-read (`golden_version` selected 0.232.0, sha `5f8a53ed…`) and the R-120 refusal banner +confirmed **absent**. + +## The floor, and the box moving itself + +`min_controller_version` **0.230.0 → 0.232.0**, re-read from the page afterwards. + +**Both boxes had been hand-deployed during validation, so the floor had nothing to move.** Rather than +claim delivery untested, `demo-felhom` was rolled back to 0.231.0 and the chain exercised for real. It +moved itself in **~20 s**, in its own words: + +``` +10:47:16 controller-swap: image file written, restarting bootstrap target=…felhom-controller:0.232.0 +10:47:26 controller-swap: new controller healthy target=…felhom-controller:0.232.0 +``` + +## Teardown + +`pct destroy 9100 --purge`, `shred -u` of the token, runner, script and log **after** the log was +copied out (`ls | grep` in the VM's `/root` → 0), `poweroff`, qemu confirmed exited with +`ps -eo comm` (**not** `pgrep -f`, which self-matches), and `qemu-img snapshot -a virgin`. The disk +carries the single `virgin` snapshot. + +## And this is the first bake the gate actually gates + +`golden_currency_gate.py` now reads `GOLDEN_SHA256=` out of this directory's `bake.log` rather than +matching the directory's name (R-410, shipped earlier the same day). Its verdict: +`newest golden baked : 0.232.0 (… [sha 5f8a53ed5b19…])`. diff --git a/documentation/tests/golden-0.232.0-2026-09-01/bake.log b/documentation/tests/golden-0.232.0-2026-09-01/bake.log new file mode 100644 index 00000000..6a3157b1 --- /dev/null +++ b/documentation/tests/golden-0.232.0-2026-09-01/bake.log @@ -0,0 +1,321 @@ +[golden] build-golden.sh v3.0.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.232.0 +[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) … + Logical volume "vm-9100-disk-0" created. + Logical volume pve/vm-9100-disk-0 changed. +Creating filesystem with 8388608 4k blocks and 2097152 inodes +Filesystem UUID: 60543827-d756-4347-b65a-2d399dffa214 +Superblock backups stored on blocks: + 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, + 4096000, 7962624 + Logical volume "vm-9100-disk-1" created. + Logical volume pve/vm-9100-disk-1 changed. +Creating filesystem with 6291456 4k blocks and 1572864 inodes +Filesystem UUID: 6214059f-3782-4307-a844-45a978831072 +Superblock backups stored on blocks: + 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, +extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst' +Total bytes read: 553512960 (528MiB, 98MiB/s) +Detected container architecture: amd64 +Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ... +done: SHA256:Wmq2LMDE0vR5hKbvaGiv+/J2PEtk2Jx456YNlTDwNcw root@felhom-golden +Creating SSH host key 'ssh_host_rsa_key' - this may take some time ... +done: SHA256:T/Qo3J9usornV+MhvzdnqKwX0f84ysnnMfIplLSyYCQ root@felhom-golden +Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ... +done: SHA256:c76v869yvL57EFiXagl57F1hvUMkwdWAKHbrJsSKHfQ root@felhom-golden +[golden] starting + installing Docker (official repo, trixie channel) … +apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct! +perl: warning: Setting locale failed. +perl: warning: Please check that your locale settings: + LANGUAGE = (unset), + LC_ALL = (unset), + LC_CTYPE = (unset), + LC_NUMERIC = (unset), + LC_COLLATE = (unset), + LC_TIME = (unset), + LC_MESSAGES = (unset), + LC_MONETARY = (unset), + LC_ADDRESS = (unset), + LC_IDENTIFICATION = (unset), + LC_MEASUREMENT = (unset), + LC_PAPER = (unset), + LC_TELEPHONE = (unset), + LC_NAME = (unset), + LANG = "en_US.UTF-8" + are supported and installed on your system. +perl: warning: Falling back to the standard locale ("C"). +locale: Cannot set LC_CTYPE to default locale: No such file or directory +locale: Cannot set LC_MESSAGES to default locale: No such file or directory +locale: Cannot set LC_ALL to default locale: No such file or directory +apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct! +perl: warning: Setting locale failed. +perl: warning: Please check that your locale settings: + LANGUAGE = (unset), + LC_ALL = (unset), + LC_CTYPE = (unset), + LC_NUMERIC = (unset), + LC_COLLATE = (unset), + LC_TIME = (unset), + LC_MESSAGES = (unset), + LC_MONETARY = (unset), + LC_ADDRESS = (unset), + LC_IDENTIFICATION = (unset), + LC_MEASUREMENT = (unset), + LC_PAPER = (unset), + LC_TELEPHONE = (unset), + LC_NAME = (unset), + LANG = "en_US.UTF-8" + are supported and installed on your system. +perl: warning: Falling back to the standard locale ("C"). +locale: Cannot set LC_CTYPE to default locale: No such file or directory +locale: Cannot set LC_MESSAGES to default locale: No such file or directory +locale: Cannot set LC_ALL to default locale: No such file or directory +[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation … +[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds … +[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) … +Unable to find image 'hello-world:latest' locally +latest: Pulling from library/hello-world +4f55086f7dd0: Pulling fs layer +4f55086f7dd0: Download complete +4f55086f7dd0: Pull complete +Digest: sha256:5dd0d3e6e255913fc30f90b9f2b1d359cc2cbdb48090cc4b65f1676e203243cc +Status: Downloaded newer image for hello-world:latest + docker OK (overlay2; data-root /var/lib/docker) + /var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4 + /mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4 + both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576 +[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.232.0 (no registry cred at deploy) … + +WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'. +Configure a credential helper to remove this warning. See +https://docs.docker.com/go/credential-store/ + +0.232.0: Pulling from admin/felhom-controller +a8ac7f6c67ab: Pulling fs layer +bf30769d36e7: Pulling fs layer +044b66fbe46c: Pulling fs layer +b5c41a28e83f: Pulling fs layer +e39b6b4d6e0d: Pulling fs layer +7f86327acbd2: Pulling fs layer +b5c41a28e83f: Waiting +e39b6b4d6e0d: Waiting +7f86327acbd2: Waiting +a8ac7f6c67ab: Verifying Checksum +a8ac7f6c67ab: Download complete +b5c41a28e83f: Verifying Checksum +b5c41a28e83f: Download complete +e39b6b4d6e0d: Verifying Checksum +e39b6b4d6e0d: Download complete +7f86327acbd2: Verifying Checksum +7f86327acbd2: Download complete +044b66fbe46c: Verifying Checksum +044b66fbe46c: Download complete +bf30769d36e7: Verifying Checksum +bf30769d36e7: Download complete +a8ac7f6c67ab: Pull complete +bf30769d36e7: Pull complete +044b66fbe46c: Pull complete +b5c41a28e83f: Pull complete +e39b6b4d6e0d: Pull complete +7f86327acbd2: Pull complete +Digest: sha256:51b2403520e425e0a7d4bc00cfd395d3b06078d79cf7927bee1035d662814b8c +Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.232.0 +gitea.dooplex.hu/admin/felhom-controller:0.232.0 +[golden] asking the controller which infra images it manages … +[golden] baking infra images (4): traefik:v3.6.7 cloudflare/cloudflared:2026.6.0 gtstef/filebrowser:1.3.3-stable gitea.dooplex.hu/admin/felhom-samba:1.1.0 … +v3.6.7: Pulling from library/traefik +589002ba0eae: Pulling fs layer +ef63511ea6cc: Pulling fs layer +0738e5cb835e: Pulling fs layer +3e6813f70c64: Pulling fs layer +3e6813f70c64: Waiting +ef63511ea6cc: Download complete +589002ba0eae: Verifying Checksum +589002ba0eae: Download complete +3e6813f70c64: Verifying Checksum +3e6813f70c64: Download complete +0738e5cb835e: Verifying Checksum +0738e5cb835e: Download complete +589002ba0eae: Pull complete +ef63511ea6cc: Pull complete +0738e5cb835e: Pull complete +3e6813f70c64: Pull complete +Digest: sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a +Status: Downloaded newer image for traefik:v3.6.7 +docker.io/library/traefik:v3.6.7 +2026.6.0: Pulling from cloudflare/cloudflared +47de5dd0b812: Pulling fs layer +c172f21841df: Pulling fs layer +99515e7b4d35: Pulling fs layer +99ba982a9142: Pulling fs layer +d6b1b89eccac: Pulling fs layer +2780920e5dbf: Pulling fs layer +7c12895b777b: Pulling fs layer +3214acf345c0: Pulling fs layer +52630fc75a18: Pulling fs layer +dd64bf2dd177: Pulling fs layer +b839dfae01f6: Pulling fs layer +ebddc55facdc: Pulling fs layer +bdfd7f7e5bf6: Pulling fs layer +2d4d7adf6272: Pulling fs layer +40008157d8d2: Pulling fs layer +bd8962e29291: Pulling fs layer +cac2ae0193cb: Pulling fs layer +74d1dac84ecc: Pulling fs layer +dd64bf2dd177: Waiting +b839dfae01f6: Waiting +ebddc55facdc: Waiting +bdfd7f7e5bf6: Waiting +2d4d7adf6272: Waiting +40008157d8d2: Waiting +bd8962e29291: Waiting +cac2ae0193cb: Waiting +74d1dac84ecc: Waiting +2780920e5dbf: Waiting +7c12895b777b: Waiting +3214acf345c0: Waiting +52630fc75a18: Waiting +99ba982a9142: Waiting +d6b1b89eccac: Waiting +c172f21841df: Verifying Checksum +c172f21841df: Download complete +47de5dd0b812: Download complete +99515e7b4d35: Verifying Checksum +99515e7b4d35: Download complete +99ba982a9142: Verifying Checksum +99ba982a9142: Download complete +d6b1b89eccac: Download complete +2780920e5dbf: Verifying Checksum +2780920e5dbf: Download complete +7c12895b777b: Verifying Checksum +7c12895b777b: Download complete +3214acf345c0: Verifying Checksum +3214acf345c0: Download complete +47de5dd0b812: Pull complete +52630fc75a18: Verifying Checksum +52630fc75a18: Download complete +dd64bf2dd177: Verifying Checksum +dd64bf2dd177: Download complete +b839dfae01f6: Verifying Checksum +b839dfae01f6: Download complete +ebddc55facdc: Verifying Checksum +ebddc55facdc: Download complete +bdfd7f7e5bf6: Download complete +2d4d7adf6272: Verifying Checksum +2d4d7adf6272: Download complete +bd8962e29291: Download complete +c172f21841df: Pull complete +40008157d8d2: Download complete +cac2ae0193cb: Verifying Checksum +cac2ae0193cb: Download complete +74d1dac84ecc: Verifying Checksum +74d1dac84ecc: Download complete +99515e7b4d35: Pull complete +99ba982a9142: Pull complete +d6b1b89eccac: Pull complete +2780920e5dbf: Pull complete +7c12895b777b: Pull complete +3214acf345c0: Pull complete +52630fc75a18: Pull complete +dd64bf2dd177: Pull complete +b839dfae01f6: Pull complete +ebddc55facdc: Pull complete +bdfd7f7e5bf6: Pull complete +2d4d7adf6272: Pull complete +40008157d8d2: Pull complete +bd8962e29291: Pull complete +cac2ae0193cb: Pull complete +74d1dac84ecc: Pull complete +Digest: sha256:ba461b8aa9c042156dbd39c38657fe7431bafa063220eab8d5330a523863da9f +Status: Downloaded newer image for cloudflare/cloudflared:2026.6.0 +docker.io/cloudflare/cloudflared:2026.6.0 +1.3.3-stable: Pulling from gtstef/filebrowser +6a0ac1617861: Pulling fs layer +ef8806083e82: Pulling fs layer +b74107c861c7: Pulling fs layer +adc935def003: Pulling fs layer +4f4fb700ef54: Pulling fs layer +18695ccc900a: Pulling fs layer +45d119d5c397: Pulling fs layer +dac52db4fc51: Pulling fs layer +6d598f86b2f2: Pulling fs layer +8aa349c8396c: Pulling fs layer +adc935def003: Waiting +4f4fb700ef54: Waiting +18695ccc900a: Waiting +45d119d5c397: Waiting +dac52db4fc51: Waiting +6d598f86b2f2: Waiting +8aa349c8396c: Waiting +6a0ac1617861: Verifying Checksum +6a0ac1617861: Download complete +adc935def003: Verifying Checksum +adc935def003: Download complete +b74107c861c7: Verifying Checksum +b74107c861c7: Download complete +4f4fb700ef54: Verifying Checksum +4f4fb700ef54: Download complete +6a0ac1617861: Pull complete +45d119d5c397: Verifying Checksum +45d119d5c397: Download complete +18695ccc900a: Verifying Checksum +18695ccc900a: Download complete +dac52db4fc51: Verifying Checksum +dac52db4fc51: Download complete +6d598f86b2f2: Verifying Checksum +6d598f86b2f2: Download complete +8aa349c8396c: Verifying Checksum +8aa349c8396c: Download complete +ef8806083e82: Download complete +ef8806083e82: Pull complete +b74107c861c7: Pull complete +adc935def003: Pull complete +4f4fb700ef54: Pull complete +18695ccc900a: Pull complete +45d119d5c397: Pull complete +dac52db4fc51: Pull complete +6d598f86b2f2: Pull complete +8aa349c8396c: Pull complete +Digest: sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70babab78c +Status: Downloaded newer image for gtstef/filebrowser:1.3.3-stable +docker.io/gtstef/filebrowser:1.3.3-stable +1.1.0: Pulling from admin/felhom-samba +897d797d2723: Pulling fs layer +3051591aa250: Pulling fs layer +ce57a3f93416: Pulling fs layer +fb94eeec2fe1: Pulling fs layer +fb94eeec2fe1: Waiting +ce57a3f93416: Verifying Checksum +ce57a3f93416: Download complete +897d797d2723: Verifying Checksum +897d797d2723: Download complete +fb94eeec2fe1: Verifying Checksum +fb94eeec2fe1: Download complete +3051591aa250: Verifying Checksum +3051591aa250: Download complete +897d797d2723: Pull complete +3051591aa250: Pull complete +ce57a3f93416: Pull complete +fb94eeec2fe1: Pull complete +Digest: sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10 +Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.1.0 +gitea.dooplex.hu/admin/felhom-samba:1.1.0 +[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'. +[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'. +[golden] baking the first-boot SSH host-key regeneration unit (F3) … +Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'. +[golden] identity-clean + minimize … +[golden] stop + archive … +INFO: including mount point rootfs ('/') in backup +INFO: including mount point mp0 ('/var/lib/felhom') in backup +INFO: archive file size: 627MB +INFO: Finished Backup of VM 9100 (00:00:30) +[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_09_01-10_43_08.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive) +[golden] publishing golden (657494489 bytes, sha256 5f8a53ed5b19a6cb…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.232.0/golden.tar.zst +[golden] pre-delete existing: HTTP 404 (404/204 expected) +[golden] upload OK (HTTP 201) +GOLDEN_VERSION=0.232.0 +GOLDEN_SHA256=5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8 +[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.232.0 / 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8 +[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)