golden 0.232.0 baked, vouched, floor raised — and it carries TWO releases
gates / gates (push) Successful in 17s

GOLDEN_SHA256 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8,
657 494 489 B. 0.231.0 was never baked, so the fleet went 0.230.0 -> 0.232.0.

THE CHECK THE 0.230.0 BAKE SKIPPED, AND THIS ONE DID NOT: the bake script's fingerprint was
compared ACROSS THE HOP - 7b0fb5cf...73b6a1 on DooPlex and inside the VM. The previous bake
recorded only the DooPlex-side hash and said so; this one is a measurement.

Three independent readers agreed before anything was vouched: the bake's own print, the round
trip of the published bytes (HTTP 200, 657494489 B, same sha, hashed from what was
downloaded), and the hub's Day-0 dropdown reading Gitea on a different code path. The
delivered artifact names its own controller - ./etc/felhom-controller-image reads
felhom-controller:0.232.0 - with 19382 entries under var/lib/felhom/docker/.

Both pre-gates were shown able to see something before their zeroes were believed, and the
manifest was RE-READ after vouching rather than trusted from the 303 flash.

DELIVERY WAS ACTUALLY EXERCISED. Both boxes had been hand-deployed during validation, so the
floor had nothing to move. Rather than report delivery untested, demo-felhom was rolled back
to 0.231.0 and the chain run for real - it moved itself in ~20s:
  10:47:16 controller-swap: image file written, restarting bootstrap  target=...0.232.0
  10:47:26 controller-swap: new controller healthy                    target=...0.232.0

And this is the first bake golden_currency_gate.py actually gates: it now reads the
GOLDEN_SHA256 line out of the bake log rather than matching a directory name (R-410, shipped
hours earlier the same day). All 13 felhom.eu gates are green, golden-currency included, for
the first time since v0.230.0 was released.
This commit is contained in:
2026-09-01 10:50:54 +02:00
parent f41a1a0ad8
commit 63eff21a5c
15 changed files with 1147 additions and 23 deletions
+16 -23
View File
@@ -1,8 +1,8 @@
# STATUS — what works, what's broken, what's next
**Updated 2026-09-01 — I ran the week's six releases against each other overnight. Three things
broke, none of them where I expected. The worst: the backup-content check I shipped yesterday
CANNOT RUN AT ALL on a machine set up like `demo-felhom`, and it fails silently every night.**
**Updated 2026-09-01 (second pass) — both faults from the overnight test are fixed, and the golden
is baked, vouched and delivered. Both machines are on 0.232.0 and moved themselves. NOTHING is
waiting on you.**
**Earlier 2026-08-31 — I measured whether the box could test its own remote restore without you,
then built the narrow version you picked. The measurement is why it is 3 seconds a night and
@@ -18,33 +18,26 @@ not an evening's work.**
*This section is allowed to be longer than one screen, and each item says what happens if you do
nothing.*
1. **The new nightly backup check does not work on `demo-felhom`** (R-414). It ran for the first
time on its own last night and could not start: that machine has no registered data drive, so
there is nowhere to unpack a backup to look at it. It will fail the same way every night, and
the only trace is one line in a log — the hub cannot tell it apart from a machine that never
had the feature. **Your backups there are fine** — the backup itself ran normally. It is the
CHECK that cannot run. **If you do nothing:** that machine, and any like it, silently never
gets the check. I have not changed anything; the fix is a small decision about where it should
unpack to, and it is yours.
1. **Nothing is waiting on you.** Both problems the overnight test found are fixed and proven on
the real machines:
- the background job that could delete a live restore's lock now waits its turn — and the check
that finds the next one like it is a test, not a comment, so it cannot come back quietly;
- the nightly backup check now runs on `demo-felhom`. It said **pass** there this morning, on a
machine where it could not start at all yesterday.
The golden carrying both is baked, vouched, and the floor is raised. `demo-felhom` picked it up
**by itself in about 20 seconds**.
2. **A golden carrying 0.231.0 is owed.** Golden **0.230.0** is baked, vouched and on both
machines. **0.231.0 is the build with the new nightly backup-content check**, and it is on
`demo-hp` only. **If you do nothing:** the check stays on one machine; a newly installed box
does not get it, and neither does `demo-felhom`. Nothing breaks — this adds a check, it does not
fix a defect. Baking and vouching 0.231.0 and raising the floor closes it, the same three-field
change as before.
3. **Whether to keep the test app `bentopdf` on `demo-hp`.** I deployed it last night because it
2. **Whether to keep the test app `bentopdf` on `demo-hp`.** I deployed it last night because it
is the ONLY app of our 53 with neither a database nor stored files — which makes it the only
way to prove the new check does not cry wolf on an app that legitimately has nothing. It
passed silently, which is what we needed to see. **My pick: keep it**, as a permanent control.
**If you do nothing:** it stays, using almost no space. Say the word and I remove it.
4. **Nothing else about this release.** Everything in 0.231.0 ships in the controller image plus
3. **Nothing else about this release.** Everything in 0.232.0 ships in the controller image plus
two register lines in the hub (already live). No customer action, no data migration, no
credential change.
5. **Whether a documents-only push should still be checked for a missing golden** (R-404). We have now
4. **Whether a documents-only push should still be checked for a missing golden** (R-404). We have now
skipped that check **eight times**, each time for a written reason: it runs on every push to the
website/documentation repository, including pushes that change nothing a machine installs.
**A guard we correctly skip eight times is teaching us to skip it.**
@@ -56,11 +49,11 @@ nothing.*
**If you do nothing:** nothing breaks, the skipping stays routine, and the count keeps rising.
I have NOT changed it; this is yours to decide and mine to build.
6. **Whether to change the hub password** (R-350). I printed it into my own session log on 20 August.
5. **Whether to change the hub password** (R-350). I printed it into my own session log on 20 August.
Not in git, not in any saved file — in the log on this machine. **If you do nothing:** it stays as
it is, at the risk you accept by leaving it. I can change it without ever showing you the new one.
7. **`demo-hp`'s network setup does not match our own notes** (R-338) — the machine works, the page is
6. **`demo-hp`'s network setup does not match our own notes** (R-338) — the machine works, the page is
wrong, or the other way round. **If you do nothing:** the page keeps misleading the next session,
as it misled one by an hour.
@@ -0,0 +1,17 @@
=== preconditions 2026-09-01T08:37:09Z ===
/dev/sda1 9.1T 3.1T 5.5T 37% /mnt/5_hdd
/dev/sdb1 445G 217G 206G 52% /
felhom-controller clean=[] HEAD=62c6a8a origin=62c6a8a
felhom.eu clean=[?? documentation/tests/golden-0.232.0-2026-09-01/;] HEAD=f41a1a0 origin=f41a1a0
MinAgent for v0.232.0: **MinAgent: 0.129.0** (unchanged)
controller image 0.232.0 in registry: HTTP 200
404 pre-gate on the golden package: HTTP 404
=== drill VM ===
Snapshot list:
ID TAG VM_SIZE DATE VM_CLOCK ICOUNT
1 virgin 0 B 2026-07-03 14:12:15 0000:00:00.000 0
no qemu running
=== bake script fingerprint on DooPlex (to be compared across the hop - the 0.230.0 bake skipped this) ===
7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1 /mnt/5_hdd/felhom.eu/git/felhom-agent/configs/build-golden.sh
@@ -0,0 +1,9 @@
=== pveam update (the virgin snapshot's INDEX is stale too) ===
update successful
system debian-13-standard_13.6-1_amd64.tar.zst
calculating checksum...OK, checksum verified
download of 'http://download.proxmox.com/images/system/debian-13-standard_13.6-1_amd64.tar.zst' to '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst' finished
=== THE FINGERPRINT COMPARISON ACROSS THE HOP — the 0.230.0 bake skipped this and said so ===
DooPlex : 7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1
in the VM: 7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1
@@ -0,0 +1,6 @@
Running as unit: golden-bake.service; invocation ID: 4067665db1c6467ab5385dd87754cfa3
=== token-leak check on the unit properties (must be 0) ===
0
=== and the grep is PROVEN to work (must be 1) ===
1
active
@@ -0,0 +1,17 @@
=== acceptance markers, counted on the COMMITTED log ===
docker OK (overlay2 : 1
including mount point :
310:INFO: including mount point rootfs ('/') in backup
311:INFO: including mount point mp0 ('/var/lib/felhom') in backup
upload OK (HTTP 201) : 1
MUST BE ZERO - excluding : 0
MUST BE ZERO - FATAL : 0
=== the result ===
GOLDEN_VERSION=0.232.0
GOLDEN_SHA256=5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8
archive volid: local:backup/vzdump-lxc-9100-2026_09_01-10_43_08.tar.zst
=== token-leak grep on the COMMITTED log (must be 0), and the grep PROVEN to work ===
bake.log: 0
seeded throwaway copy (must be 1): 1
@@ -0,0 +1,4 @@
Logical volume "vm-9100-disk-1" successfully removed.
purging CT 9100 from related configurations..
leftovers in the VM: 0
1 virgin 0 B 2026-07-03 14:12:15 0000:00:00.000 0
@@ -0,0 +1,315 @@
[golden] build-golden.sh v3.0.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.232.0
[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) …
Logical volume "vm-9100-disk-0" created.
Logical volume pve/vm-9100-disk-0 changed.
Creating filesystem with 8388608 4k blocks and 2097152 inodes
Filesystem UUID: 60543827-d756-4347-b65a-2d399dffa214
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624
Logical volume "vm-9100-disk-1" created.
Logical volume pve/vm-9100-disk-1 changed.
Creating filesystem with 6291456 4k blocks and 1572864 inodes
Filesystem UUID: 6214059f-3782-4307-a844-45a978831072
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst'
Total bytes read: 553512960 (528MiB, 98MiB/s)
Detected container architecture: amd64
Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ...
done: SHA256:Wmq2LMDE0vR5hKbvaGiv+/J2PEtk2Jx456YNlTDwNcw root@felhom-golden
Creating SSH host key 'ssh_host_rsa_key' - this may take some time ...
done: SHA256:T/Qo3J9usornV+MhvzdnqKwX0f84ysnnMfIplLSyYCQ root@felhom-golden
Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ...
done: SHA256:c76v869yvL57EFiXagl57F1hvUMkwdWAKHbrJsSKHfQ root@felhom-golden
[golden] starting + installing Docker (official repo, trixie channel) …
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation …
[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds …
[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) …
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
4f55086f7dd0: Pulling fs layer
4f55086f7dd0: Download complete
4f55086f7dd0: Pull complete
Digest: sha256:5dd0d3e6e255913fc30f90b9f2b1d359cc2cbdb48090cc4b65f1676e203243cc
Status: Downloaded newer image for hello-world:latest
docker OK (overlay2; data-root /var/lib/docker)
/var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4
/mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4
both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576
[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.232.0 (no registry cred at deploy) …
WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/
0.232.0: Pulling from admin/felhom-controller
a8ac7f6c67ab: Pulling fs layer
bf30769d36e7: Pulling fs layer
044b66fbe46c: Pulling fs layer
b5c41a28e83f: Pulling fs layer
e39b6b4d6e0d: Pulling fs layer
7f86327acbd2: Pulling fs layer
b5c41a28e83f: Waiting
e39b6b4d6e0d: Waiting
7f86327acbd2: Waiting
a8ac7f6c67ab: Verifying Checksum
a8ac7f6c67ab: Download complete
b5c41a28e83f: Verifying Checksum
b5c41a28e83f: Download complete
e39b6b4d6e0d: Verifying Checksum
e39b6b4d6e0d: Download complete
7f86327acbd2: Verifying Checksum
7f86327acbd2: Download complete
044b66fbe46c: Verifying Checksum
044b66fbe46c: Download complete
bf30769d36e7: Verifying Checksum
bf30769d36e7: Download complete
a8ac7f6c67ab: Pull complete
bf30769d36e7: Pull complete
044b66fbe46c: Pull complete
b5c41a28e83f: Pull complete
e39b6b4d6e0d: Pull complete
7f86327acbd2: Pull complete
Digest: sha256:51b2403520e425e0a7d4bc00cfd395d3b06078d79cf7927bee1035d662814b8c
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.232.0
gitea.dooplex.hu/admin/felhom-controller:0.232.0
[golden] asking the controller which infra images it manages …
[golden] baking infra images (4): traefik:v3.6.7 cloudflare/cloudflared:2026.6.0 gtstef/filebrowser:1.3.3-stable gitea.dooplex.hu/admin/felhom-samba:1.1.0 …
v3.6.7: Pulling from library/traefik
589002ba0eae: Pulling fs layer
ef63511ea6cc: Pulling fs layer
0738e5cb835e: Pulling fs layer
3e6813f70c64: Pulling fs layer
3e6813f70c64: Waiting
ef63511ea6cc: Download complete
589002ba0eae: Verifying Checksum
589002ba0eae: Download complete
3e6813f70c64: Verifying Checksum
3e6813f70c64: Download complete
0738e5cb835e: Verifying Checksum
0738e5cb835e: Download complete
589002ba0eae: Pull complete
ef63511ea6cc: Pull complete
0738e5cb835e: Pull complete
3e6813f70c64: Pull complete
Digest: sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a
Status: Downloaded newer image for traefik:v3.6.7
docker.io/library/traefik:v3.6.7
2026.6.0: Pulling from cloudflare/cloudflared
47de5dd0b812: Pulling fs layer
c172f21841df: Pulling fs layer
99515e7b4d35: Pulling fs layer
99ba982a9142: Pulling fs layer
d6b1b89eccac: Pulling fs layer
2780920e5dbf: Pulling fs layer
7c12895b777b: Pulling fs layer
3214acf345c0: Pulling fs layer
52630fc75a18: Pulling fs layer
dd64bf2dd177: Pulling fs layer
b839dfae01f6: Pulling fs layer
ebddc55facdc: Pulling fs layer
bdfd7f7e5bf6: Pulling fs layer
2d4d7adf6272: Pulling fs layer
40008157d8d2: Pulling fs layer
bd8962e29291: Pulling fs layer
cac2ae0193cb: Pulling fs layer
74d1dac84ecc: Pulling fs layer
dd64bf2dd177: Waiting
b839dfae01f6: Waiting
ebddc55facdc: Waiting
bdfd7f7e5bf6: Waiting
2d4d7adf6272: Waiting
40008157d8d2: Waiting
bd8962e29291: Waiting
cac2ae0193cb: Waiting
74d1dac84ecc: Waiting
2780920e5dbf: Waiting
7c12895b777b: Waiting
3214acf345c0: Waiting
52630fc75a18: Waiting
99ba982a9142: Waiting
d6b1b89eccac: Waiting
c172f21841df: Verifying Checksum
c172f21841df: Download complete
47de5dd0b812: Download complete
99515e7b4d35: Verifying Checksum
99515e7b4d35: Download complete
99ba982a9142: Verifying Checksum
99ba982a9142: Download complete
d6b1b89eccac: Download complete
2780920e5dbf: Verifying Checksum
2780920e5dbf: Download complete
7c12895b777b: Verifying Checksum
7c12895b777b: Download complete
3214acf345c0: Verifying Checksum
3214acf345c0: Download complete
47de5dd0b812: Pull complete
52630fc75a18: Verifying Checksum
52630fc75a18: Download complete
dd64bf2dd177: Verifying Checksum
dd64bf2dd177: Download complete
b839dfae01f6: Verifying Checksum
b839dfae01f6: Download complete
ebddc55facdc: Verifying Checksum
ebddc55facdc: Download complete
bdfd7f7e5bf6: Download complete
2d4d7adf6272: Verifying Checksum
2d4d7adf6272: Download complete
bd8962e29291: Download complete
c172f21841df: Pull complete
40008157d8d2: Download complete
cac2ae0193cb: Verifying Checksum
cac2ae0193cb: Download complete
74d1dac84ecc: Verifying Checksum
74d1dac84ecc: Download complete
99515e7b4d35: Pull complete
99ba982a9142: Pull complete
d6b1b89eccac: Pull complete
2780920e5dbf: Pull complete
7c12895b777b: Pull complete
3214acf345c0: Pull complete
52630fc75a18: Pull complete
dd64bf2dd177: Pull complete
b839dfae01f6: Pull complete
ebddc55facdc: Pull complete
bdfd7f7e5bf6: Pull complete
2d4d7adf6272: Pull complete
40008157d8d2: Pull complete
bd8962e29291: Pull complete
cac2ae0193cb: Pull complete
74d1dac84ecc: Pull complete
Digest: sha256:ba461b8aa9c042156dbd39c38657fe7431bafa063220eab8d5330a523863da9f
Status: Downloaded newer image for cloudflare/cloudflared:2026.6.0
docker.io/cloudflare/cloudflared:2026.6.0
1.3.3-stable: Pulling from gtstef/filebrowser
6a0ac1617861: Pulling fs layer
ef8806083e82: Pulling fs layer
b74107c861c7: Pulling fs layer
adc935def003: Pulling fs layer
4f4fb700ef54: Pulling fs layer
18695ccc900a: Pulling fs layer
45d119d5c397: Pulling fs layer
dac52db4fc51: Pulling fs layer
6d598f86b2f2: Pulling fs layer
8aa349c8396c: Pulling fs layer
adc935def003: Waiting
4f4fb700ef54: Waiting
18695ccc900a: Waiting
45d119d5c397: Waiting
dac52db4fc51: Waiting
6d598f86b2f2: Waiting
8aa349c8396c: Waiting
6a0ac1617861: Verifying Checksum
6a0ac1617861: Download complete
adc935def003: Verifying Checksum
adc935def003: Download complete
b74107c861c7: Verifying Checksum
b74107c861c7: Download complete
4f4fb700ef54: Verifying Checksum
4f4fb700ef54: Download complete
6a0ac1617861: Pull complete
45d119d5c397: Verifying Checksum
45d119d5c397: Download complete
18695ccc900a: Verifying Checksum
18695ccc900a: Download complete
dac52db4fc51: Verifying Checksum
dac52db4fc51: Download complete
6d598f86b2f2: Verifying Checksum
6d598f86b2f2: Download complete
8aa349c8396c: Verifying Checksum
8aa349c8396c: Download complete
ef8806083e82: Download complete
ef8806083e82: Pull complete
b74107c861c7: Pull complete
adc935def003: Pull complete
4f4fb700ef54: Pull complete
18695ccc900a: Pull complete
45d119d5c397: Pull complete
dac52db4fc51: Pull complete
6d598f86b2f2: Pull complete
8aa349c8396c: Pull complete
Digest: sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70babab78c
Status: Downloaded newer image for gtstef/filebrowser:1.3.3-stable
docker.io/gtstef/filebrowser:1.3.3-stable
1.1.0: Pulling from admin/felhom-samba
897d797d2723: Pulling fs layer
3051591aa250: Pulling fs layer
ce57a3f93416: Pulling fs layer
fb94eeec2fe1: Pulling fs layer
fb94eeec2fe1: Waiting
ce57a3f93416: Verifying Checksum
ce57a3f93416: Download complete
897d797d2723: Verifying Checksum
897d797d2723: Download complete
fb94eeec2fe1: Verifying Checksum
fb94eeec2fe1: Download complete
3051591aa250: Verifying Checksum
3051591aa250: Download complete
897d797d2723: Pull complete
3051591aa250: Pull complete
ce57a3f93416: Pull complete
fb94eeec2fe1: Pull complete
Digest: sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.1.0
gitea.dooplex.hu/admin/felhom-samba:1.1.0
[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'.
[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'.
[golden] baking the first-boot SSH host-key regeneration unit (F3) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'.
[golden] identity-clean + minimize …
[golden] stop + archive …
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/var/lib/felhom') in backup
INFO: archive file size: 627MB
INFO: Finished Backup of VM 9100 (00:00:30)
[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_09_01-10_43_08.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive)
[golden] publishing golden (657494489 bytes, sha256 5f8a53ed5b19a6cb…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.232.0/golden.tar.zst
[golden] pre-delete existing: HTTP 404 (404/204 expected)
[golden] upload OK (HTTP 201)
GOLDEN_VERSION=0.232.0
GOLDEN_SHA256=5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8
[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.232.0 / 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8
[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)
@@ -0,0 +1,8 @@
=== READER 2: the ROUND TRIP — hash the DOWNLOADED bytes, not the bake's printout ===
http=200 bytes=657494489
downloaded sha256: 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8
bake printed : 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8
=== THE DELIVERED ARTIFACT MUST NAME ITS OWN CONTROLLER ===
gitea.dooplex.hu/admin/felhom-controller:0.232.0
docker store entries in the archive: 19382
@@ -0,0 +1,8 @@
=== READER 3: the hub's Day-0 dropdown, reading Gitea on a different code path ===
sha=...
0.232.0 sha=5f8a53ed5b19a6cb2006298c...
0.230.0 sha=9287f7cef5f13166276e8406... <-- currently SELECTED
0.229.0 sha=39aa886df77b21757aef3b29...
has 0.232.0: ['0.232.0']
current fleet floor: 0.230.0
agent_version selected: 0.130.0
@@ -0,0 +1,9 @@
=== VOUCH: the THREE fields, moved together ===
golden_version 0.230.0 -> 0.232.0 (this bake carries TWO releases: 0.231.0 was never baked)
agent_version 0.130.0 -> 0.130.0 UNCHANGED, already >= MinAgent
min_agent 0.129.0 -> 0.129.0 UNCHANGED, v0.232.0's CHANGELOG says MinAgent 0.129.0
min_agent (0.129.0) <= agent_version (0.130.0): NOT the R-216 shape
POST http=303
Location: /configuration?flash=artifacts_set
(read from -D, never %{redirect_url} — R-132)
@@ -0,0 +1,286 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Configuration — Felhom Hub</title>
<link rel="stylesheet" href="/style.css?v=0.110.0">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
</svg>
<div class="container">
<header>
<h1>Felhom <span>Hub</span></h1>
<nav class="nav-links">
<a href="/" class="nav-link">Dashboard</a>
<a href="/configs" class="nav-link">Customers</a>
<a href="/apps" class="nav-link">Apps</a>
<a href="/hosts" class="nav-link">Hosts</a>
<a href="/offsite" class="nav-link">Offsite</a>
<a href="/configuration" class="nav-link active">Configuration</a>
</nav>
</header>
<h2 style="margin-bottom: 1rem;">Configuration</h2>
<section class="card">
<h3 style="margin-top: 0;">Managed updates — global floor</h3>
<p class="text-muted" style="margin: 0 0 0.75rem; font-size: 0.85em;">
The minimum controller version every box auto-updates to (unless a per-customer override is set).
<strong>Saving takes effect immediately</strong> — boxes below the floor update on their next
report, no customer action. Blank = no global floor. This setting is independent of the Day-0
artifact manifest below.
</p>
<p style="margin: 0 0 0.75rem; font-size: 0.85em;">
Effective floor:
<code>v0.230.0</code>
<span style="color: #cbd5e1;">— source: <strong>DB (hub_settings)</strong>; env fallback would be <code>v0.120.0</code></span>
</p>
<form id="global-floor-form" method="POST" action="/configuration/global-floor" style="display: flex; gap: 0.5rem; align-items: center; flex-wrap: wrap;">
<input type="hidden" name="_csrf" value="">
<input type="text" id="global-floor-input" name="min_controller_version" value="0.230.0" placeholder="e.g. 0.86.0 (blank = clear DB override)" style="padding: 0.3em 0.5em; width: 16em;">
<button class="btn btn-sm" type="button" onclick="confirmGlobalFloor()">Save global floor…</button>
<span style="font-size: 0.85em; color: #cbd5e1;">DB override: <code>v0.230.0</code></span>
</form>
<div id="global-floor-confirm" style="display: none; margin-top: 0.75rem; padding: 0.75rem; border: 1px solid #7c3f00; background: #241a0a; border-radius: 6px; max-width: 44em;">
<p id="global-floor-impact" style="margin: 0 0 0.5rem; font-size: 0.9em;">…</p>
<p style="margin: 0 0 0.5rem; font-size: 0.85em; color: #cbd5e1;">Type the version again to confirm (or <code>CLEAR</code> to remove the DB override):</p>
<input type="text" id="global-floor-confirm-input" placeholder="retype the version…" style="padding: 0.3em 0.5em; width: 16em;">
<button class="btn btn-sm" type="button" onclick="submitGlobalFloor()">Confirm &amp; apply</button>
<button class="btn btn-sm btn-ghost" type="button" onclick="document.getElementById('global-floor-confirm').style.display='none';">Cancel</button>
<p id="global-floor-confirm-err" style="margin: 0.4em 0 0; font-size: 0.8em; color: #f87171;"></p>
</div>
<script>
function confirmGlobalFloor() {
var v = document.getElementById('global-floor-input').value.trim();
var box = document.getElementById('global-floor-confirm');
var impact = document.getElementById('global-floor-impact');
document.getElementById('global-floor-confirm-input').value = '';
document.getElementById('global-floor-confirm-err').textContent = '';
box.style.display = 'block';
if (v === '') {
impact.textContent = 'This will CLEAR the DB floor override (the box falls back to the env default). Type CLEAR to confirm.';
return;
}
impact.textContent = 'Checking blast radius…';
fetch('/configuration/global-floor/impact?v=' + encodeURIComponent(v))
.then(function(r){ return r.json(); })
.then(function(d){
if (!d.valid) { impact.textContent = 'Invalid version — use X.Y.Z.'; return; }
impact.textContent = 'Saving the minimum version v' + d.version +
' takes effect immediately — currently ' + d.below +
' box(es) are below this version and would update on their next report.';
})
.catch(function(){ impact.textContent = 'Could not compute the blast radius; proceed with caution.'; });
}
function submitGlobalFloor() {
var v = document.getElementById('global-floor-input').value.trim();
var typed = document.getElementById('global-floor-confirm-input').value.trim();
var err = document.getElementById('global-floor-confirm-err');
var expected = (v === '') ? 'CLEAR' : v;
if (typed !== expected) { err.textContent = 'Confirmation does not match (' + expected + ').'; return; }
document.getElementById('global-floor-form').submit();
}
</script>
</section>
<section class="card">
<h3 style="margin-top: 0;">Day-0 artifacts — agent &amp; golden</h3>
<p class="text-muted" style="margin: 0 0 0.75rem; font-size: 0.85em;">
The current agent binary + golden archive the host-bootstrap script fetches from Gitea and
verifies (sha256) before installing. The hub vouches for these checksums (a different trust
root than Gitea). Pick a version — the sha256 is read from Gitea automatically (no manual
copy). Choose <em>— none —</em> to clear an artifact.
</p>
<form method="POST" action="/configuration/artifacts" style="display: grid; grid-template-columns: auto 12em 1fr; gap: 0.5rem; align-items: center; max-width: 56em;">
<input type="hidden" name="_csrf" value="">
<label style="font-size: 0.9em; color: #cbd5e1;">Agent</label>
<select name="agent_version" id="agent_version" onchange="syncArtifactSha('agent')" style="padding: 0.3em 0.5em;">
<option value="" data-sha="">— none —</option>
<option value="0.130.0" data-sha="a56a92a7bd68f5b46736eaec4806c3d26c16ccb35118c4ac0e3d8094eaefabc3" selected>0.130.0</option>
<option value="0.129.0" data-sha="53a54f0620afbd6d4a1b86607e2a84dfbe7a290f44ed38a9485d6d971eecde8d" >0.129.0</option>
<option value="0.128.0" data-sha="c6eba73bf9b9ad6980cfef57bfb3db31581abc9d643de2ff50d4254576fc1a59" >0.128.0</option>
<option value="0.127.0" data-sha="f0d2c89311f03fd2b9ab6ae242e09a724ff8ea1fd2214759bae247c5db7dbf72" >0.127.0</option>
<option value="0.126.0" data-sha="7ecf8e9cdba237bc2d81003440095eace6418d00c3a485f3ca77742a25e4a93b" >0.126.0</option>
<option value="0.125.0" data-sha="f7d8339b53d92a6c45be7eaf189469a041b6b00b758a64511c0479beae7016b3" >0.125.0</option>
<option value="0.124.1" data-sha="5c279bda64cdec8cbd76f8a800bcd602063bb17175f406b225775e7dece3a21c" >0.124.1</option>
<option value="0.124.0" data-sha="5e4179383bc838a7ad360efcfab5e8f02a2939ed8bcc4bc68ffd85ba02a5c9a4" >0.124.0</option>
<option value="0.123.0" data-sha="74910135ac4feb1b7f0ad4dbd1541d965cbc0fe70d4f47b62ebf7e4bfb962453" >0.123.0</option>
<option value="0.122.0" data-sha="d5f294e56c1ef59055e8e87fb9135aa477632dbbc56a5d4bff46bbd0466c1edf" >0.122.0</option>
<option value="0.121.1" data-sha="afaeeb509d1ed70d6e6bebac0393a3cd5be59d51e3db9ff96ef8524bd78546d7" >0.121.1</option>
<option value="0.121.0" data-sha="b2128f3cd4539225a2842f541f56ffaf5390b1d97f3f3a80076ec5f53dbc7d7a" >0.121.0</option>
</select>
<input type="text" name="agent_sha256" id="agent_sha256" value="a56a92a7bd68f5b46736eaec4806c3d26c16ccb35118c4ac0e3d8094eaefabc3" readonly placeholder="64-hex sha256 (blank = none)" style="padding: 0.3em 0.5em; font-family: monospace; opacity: 0.7;">
<label style="font-size: 0.9em; color: #cbd5e1;">Golden</label>
<select name="golden_version" id="golden_version" onchange="syncArtifactSha('golden')" style="padding: 0.3em 0.5em;">
<option value="" data-sha="">— none —</option>
<option value="0.232.0" data-sha="5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8" selected>0.232.0</option>
<option value="0.230.0" data-sha="9287f7cef5f13166276e8406005e3f28004004510c5184f1c1c7377f7aafad2e" >0.230.0</option>
<option value="0.229.0" data-sha="39aa886df77b21757aef3b298a389343dc0df5134bb0f14e8f92a451d7bdae87" >0.229.0</option>
<option value="0.228.0" data-sha="76a3a98b9e7cc23bf8ae51b38a6272f576df285cb34cd22235ac3f06a31e53ec" >0.228.0</option>
<option value="0.227.1" data-sha="66754491dc9bd0130ef8ded9562f63c53a5ffdcfd91baa551141e55fa083ea32" >0.227.1</option>
<option value="0.226.1" data-sha="70ed8e9377dec22a9b493e55f222b0e25a49d7f3caec8c506e0412fd6baefe69" >0.226.1</option>
<option value="0.223.0" data-sha="9eaf39ac39219b42ec9e6cbf890275febcdcc6f53325fe0c0f591d3431044f17" >0.223.0</option>
<option value="0.222.0" data-sha="19f5904f53792684f046ec0bc25426645cb87ad73d5cfc6c03639d9f82706037" >0.222.0</option>
<option value="0.221.1" data-sha="1c8bf6cf08cadabeca6331f38360d905e867c235067cd10c2716915b6e6df089" >0.221.1</option>
</select>
<input type="text" name="golden_sha256" id="golden_sha256" value="5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8" readonly placeholder="64-hex sha256 (blank = none)" style="padding: 0.3em 0.5em; font-family: monospace; opacity: 0.7;">
<label style="font-size: 0.9em; color: #cbd5e1;">Min agent</label>
<input type="text" name="min_agent" value="0.129.0" placeholder="e.g. 0.81.0 (blank = uncoupled)" style="padding: 0.3em 0.5em;">
<span style="font-size: 0.8em; color: #94a6bf;">The golden's controller CHANGELOG <code>MinAgent:</code>. The hub HOLDS the floor for any box whose agent is below this — blank = uncoupled release, no gating.</span>
<label style="font-size: 0.9em; color: #cbd5e1;">PBS wrapper</label>
<input type="text" name="wrapper_sha256" value="104db0a4401f65bbc476e82bfb1796433bcb36f8f8cce69efb3bb5c40fcb16b3" placeholder="64-hex sha256 (blank = not vouched)" style="grid-column: 2 / 4; padding: 0.3em 0.5em; font-family: monospace;">
<span></span>
<span style="grid-column: 2 / 4; font-size: 0.8em; color: #94a6bf;">sha256 of <code>configs/felhom-pbs-apply</code> (R-50b). Unlike the agent and golden this root-owned wrapper is installed from <code>raw/branch/main</code> — unversioned and unpinned. Recording it here does not fix the channel; it makes host drift <em>visible</em>: agents report the installed file's hash and a mismatch is surfaced on the host page.</span>
<span></span><span></span>
<button class="btn btn-sm" type="submit" style="justify-self: start;">Save artifact manifest</button>
</form>
<script>
function syncArtifactSha(kind) {
var sel = document.getElementById(kind + '_version');
var sha = document.getElementById(kind + '_sha256');
if (!sel || !sha) return;
var opt = sel.options[sel.selectedIndex];
sha.value = (opt && opt.getAttribute('data-sha')) || '';
}
</script>
</section>
<section class="card">
<h3 style="margin-top: 0;">Login password</h3>
<p class="text-muted" style="margin: 0 0 0.75rem; font-size: 0.85em;">
The password for signing in to this hub UI. <strong>Changing it takes effect immediately</strong>
for the next sign-in — your current session stays logged in. Enter your current password to confirm.
If you ever lose it, the deployment ConfigMap (<code>auth.password_hash</code>) remains the reset path.
</p>
<form method="POST" action="/configuration/password" style="display: grid; grid-template-columns: auto 20em; gap: 0.5rem; align-items: center; max-width: 40em;"
onsubmit="return felhomCheckNewPw(this);">
<input type="hidden" name="_csrf" value="">
<label style="font-size: 0.9em; color: #cbd5e1;">Current password</label>
<input type="password" name="current_password" autocomplete="current-password" required style="padding: 0.3em 0.5em;">
<label style="font-size: 0.9em; color: #cbd5e1;">New password</label>
<input type="password" id="new_password" name="new_password" autocomplete="new-password" minlength="8" maxlength="72" required style="padding: 0.3em 0.5em;">
<label style="font-size: 0.9em; color: #cbd5e1;">Confirm new password</label>
<input type="password" id="confirm_password" name="confirm_password" autocomplete="new-password" minlength="8" maxlength="72" required style="padding: 0.3em 0.5em;">
<span></span>
<span>
<button class="btn btn-sm" type="submit">Change password</button>
<span id="pw-client-err" style="margin-left: 0.6em; font-size: 0.8em; color: #f87171;"></span>
</span>
</form>
<script>
function felhomCheckNewPw(form) {
var a = form.new_password.value;
var b = form.confirm_password.value;
var err = document.getElementById('pw-client-err');
err.textContent = '';
if (a.length < 8) { err.textContent = 'New password must be at least 8 characters.'; return false; }
if (a !== b) { err.textContent = 'New password and confirmation do not match.'; return false; }
return true;
}
</script>
</section>
<section class="card">
<h3 style="margin-top: 0;">Assets</h3>
<p class="text-muted" style="margin-bottom: 1rem;">
App logos and screenshots served to controllers. Assets are seeded from the Docker image
and synced to controllers daily via the asset manifest API.
</p>
<div class="info-grid">
<div class="info-item">
<span class="label">Files in manifest</span>
<span class="value">211</span>
</div>
<div class="info-item">
<span class="label">Manifest generated</span>
<span class="value" style="font-family: var(--font-mono); font-size: 0.85em;">2026-08-31T18:54:31Z</span>
</div>
</div>
<form method="POST" action="/configuration" style="margin-top: 1rem;">
<input type="hidden" name="_csrf" value="">
<input type="hidden" name="action" value="refresh_assets">
<button type="submit" class="btn" onclick="this.disabled=true;this.textContent='Refreshing…';this.form.submit();">Refresh Assets from Image</button>
</form>
<p class="text-muted" style="margin-top: 0.75rem; font-size: 0.8rem;">
Re-reads the baked-in asset seed directory and updates changed files.
Controllers will pick up changes on their next daily sync or manual trigger.
</p>
</section>
<footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
Felhom Hub <span style="font-family: var(--font-mono)">0.110.0</span>
</footer>
</div>
</body>
</html>
@@ -0,0 +1,11 @@
=== RE-READ FROM THE PAGE — a 303 and a success flash are not proof ===
agent_version SELECTED = 0.130.0 sha=a56a92a7bd68f5b46736eaec...
golden_version SELECTED = 0.232.0 sha=5f8a53ed5b19a6cb2006298c...
min_agent = 0.129.0
golden_sha256 = 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8
R-120 refusal banner present: False
=== raise the fleet floor 0.230.0 -> 0.232.0 (Viktor agreed to this step) ===
POST http=303
Location: /configuration?flash=floor_set
after: name="min_controller_version" value="0.232.0"
@@ -0,0 +1,20 @@
=== both boxes were HAND-DEPLOYED during validation, so the floor had nothing to move.
Rolling demo-felhom back to 0.231.0 so the delivery chain is actually exercised. ===
rolled-back-to-0.231.0
now: gitea.dooplex.hu/admin/felhom-controller:0.232.0
=== waiting for the agent to pull it up to the floor ===
MOVED ITSELF after ~20s
final: gitea.dooplex.hu/admin/felhom-controller:0.232.0
=== demo-felhom's OWN agent journal — the self-update, in its words ===
Sep 01 10:47:16 demo-felhom felhom-agent[2564828]: time=2026-09-01T10:47:16.761+02:00 level=INFO msg="controller-swap: image file written, restarting bootstrap" vmid=9201 target=gitea.dooplex.hu/admin/felhom-controller:0.232.0
Sep 01 10:47:26 demo-felhom felhom-agent[2564828]: time=2026-09-01T10:47:26.188+02:00 level=INFO msg="controller-swap: new controller healthy" vmid=9201 target=gitea.dooplex.hu/admin/felhom-controller:0.232.0
=== both boxes, final ===
hp: gitea.dooplex.hu/admin/felhom-controller:0.232.0 Up 17 minutes (healthy)
felhom-pve: gitea.dooplex.hu/admin/felhom-controller:0.232.0 Up 23 seconds (healthy)
=== the golden gate — the first bake it actually gates (R-410) ===
newest released controller : 0.232.0 (## v0.232.0 — one writer at a time, and a check that can actually run (2026-09-01, R-411/R)
newest golden baked : 0.232.0 (documentation/tests/golden-0.232.0-2026-09-01 [sha 5f8a53ed5b19…])
golden currency gate OK — the newest released controller has a golden (NOTE: this checks the BAKE, not the vouch — see the module docstring)
@@ -0,0 +1,100 @@
# Golden bake 0.232.0 — 2026-09-01
Baked, published, round-trip verified, **vouched**, and the fleet floor raised.
**This bake carries TWO releases: 0.231.0 was never baked**, so the fleet went 0.230.0 → 0.232.0.
| | |
|---|---|
| `GOLDEN_SHA256` | `5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8` |
| size | **657 494 489 B** |
| baked controller | `gitea.dooplex.hu/admin/felhom-controller:0.232.0` |
| `MinAgent` | **0.129.0** — read from the controller CHANGELOG header, not assumed |
| script | `build-golden.sh v3.0.0`, sha `7b0fb5cf…73b6a1` |
| template | `debian-13-standard_13.6-1_amd64.tar.zst`, after `pveam update` |
| archive volid | `local:backup/vzdump-lxc-9100-2026_09_01-10_43_08.tar.zst` |
## The check the 0.230.0 bake skipped, and this one did not
**The bake script's fingerprint was compared ACROSS THE HOP:**
```
DooPlex : 7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1
in the VM : 7b0fb5cf082fa3301a45578d1624e375d205b4e538d602e7d198a015e573b6a1
```
The 0.230.0 bake recorded only the DooPlex-side hash and said so plainly. This one is a measurement.
## Acceptance markers — counted on the COMMITTED log
```
docker OK (overlay2 : 1
including mount point rootfs : 1 (line 310)
including mount point mp0 : 1 (line 311) <- no mp1 since v3.0.0 (R-165/R-233)
upload OK (HTTP 201) : 1
--- must be ZERO ---
excluding : 0
FATAL : 0
```
## Three independent readers agreed before anything was vouched
1. **The bake** printed `GOLDEN_SHA256=5f8a53ed…91b8`.
2. **The round trip** — the published bytes downloaded back: `HTTP 200`, **657 494 489 B**, sha
`5f8a53ed…91b8`. Hashed from the **downloaded** bytes, never the local file.
3. **The hub's Day-0 dropdown**, a different code path reading Gitea: `data-sha="5f8a53ed5b19a6cb…"`.
**And the delivered artifact names the controller it will start**, read out of the downloaded archive:
```
$ tar --zstd -xOf golden.tar.zst ./etc/felhom-controller-image
gitea.dooplex.hu/admin/felhom-controller:0.232.0
```
with **19 382** entries under `var/lib/felhom/docker/` — the image store is IN the archive.
## Pre-gates, each proven able to see something first
| gate | result | the control that makes it believable |
|---|---|---|
| 404 pre-gate | `HTTP 404` for 0.232.0 before the bake | the controller image manifest returns `HTTP 200` on the same host |
| token-leak grep on the committed log | **0** | the token appended to a throwaway copy greps **1**; the copy was `shred -u`'d |
| token off every command line | unit properties grep **0** | same seeded control returns **1** |
## The vouch — three fields, and only one moved
| field | before | after | why |
|---|---|---|---|
| `golden_version` | 0.230.0 | **0.232.0** | the new bake |
| `agent_version` | 0.130.0 | 0.130.0 | unchanged — already ≥ MinAgent |
| `min_agent` | 0.129.0 | 0.129.0 | unchanged — v0.232.0's header says `MinAgent: 0.129.0` |
`min_agent` (0.129.0) ≤ `agent_version` (0.130.0), so **not** the R-216 shape.
`POST /configuration/artifacts` → `303 flash=artifacts_set`; **the flash was not treated as proof** —
the page was re-read (`golden_version` selected 0.232.0, sha `5f8a53ed…`) and the R-120 refusal banner
confirmed **absent**.
## The floor, and the box moving itself
`min_controller_version` **0.230.0 → 0.232.0**, re-read from the page afterwards.
**Both boxes had been hand-deployed during validation, so the floor had nothing to move.** Rather than
claim delivery untested, `demo-felhom` was rolled back to 0.231.0 and the chain exercised for real. It
moved itself in **~20 s**, in its own words:
```
10:47:16 controller-swap: image file written, restarting bootstrap target=…felhom-controller:0.232.0
10:47:26 controller-swap: new controller healthy target=…felhom-controller:0.232.0
```
## Teardown
`pct destroy 9100 --purge`, `shred -u` of the token, runner, script and log **after** the log was
copied out (`ls | grep` in the VM's `/root` → 0), `poweroff`, qemu confirmed exited with
`ps -eo comm` (**not** `pgrep -f`, which self-matches), and `qemu-img snapshot -a virgin`. The disk
carries the single `virgin` snapshot.
## And this is the first bake the gate actually gates
`golden_currency_gate.py` now reads `GOLDEN_SHA256=` out of this directory's `bake.log` rather than
matching the directory's name (R-410, shipped earlier the same day). Its verdict:
`newest golden baked : 0.232.0 (… [sha 5f8a53ed5b19…])`.
@@ -0,0 +1,321 @@
[golden] build-golden.sh v3.0.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.232.0
[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) …
Logical volume "vm-9100-disk-0" created.
Logical volume pve/vm-9100-disk-0 changed.
Creating filesystem with 8388608 4k blocks and 2097152 inodes
Filesystem UUID: 60543827-d756-4347-b65a-2d399dffa214
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624
Logical volume "vm-9100-disk-1" created.
Logical volume pve/vm-9100-disk-1 changed.
Creating filesystem with 6291456 4k blocks and 1572864 inodes
Filesystem UUID: 6214059f-3782-4307-a844-45a978831072
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst'
Total bytes read: 553512960 (528MiB, 98MiB/s)
Detected container architecture: amd64
Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ...
done: SHA256:Wmq2LMDE0vR5hKbvaGiv+/J2PEtk2Jx456YNlTDwNcw root@felhom-golden
Creating SSH host key 'ssh_host_rsa_key' - this may take some time ...
done: SHA256:T/Qo3J9usornV+MhvzdnqKwX0f84ysnnMfIplLSyYCQ root@felhom-golden
Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ...
done: SHA256:c76v869yvL57EFiXagl57F1hvUMkwdWAKHbrJsSKHfQ root@felhom-golden
[golden] starting + installing Docker (official repo, trixie channel) …
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation …
[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds …
[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) …
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
4f55086f7dd0: Pulling fs layer
4f55086f7dd0: Download complete
4f55086f7dd0: Pull complete
Digest: sha256:5dd0d3e6e255913fc30f90b9f2b1d359cc2cbdb48090cc4b65f1676e203243cc
Status: Downloaded newer image for hello-world:latest
docker OK (overlay2; data-root /var/lib/docker)
/var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4
/mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4
both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576
[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.232.0 (no registry cred at deploy) …
WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/
0.232.0: Pulling from admin/felhom-controller
a8ac7f6c67ab: Pulling fs layer
bf30769d36e7: Pulling fs layer
044b66fbe46c: Pulling fs layer
b5c41a28e83f: Pulling fs layer
e39b6b4d6e0d: Pulling fs layer
7f86327acbd2: Pulling fs layer
b5c41a28e83f: Waiting
e39b6b4d6e0d: Waiting
7f86327acbd2: Waiting
a8ac7f6c67ab: Verifying Checksum
a8ac7f6c67ab: Download complete
b5c41a28e83f: Verifying Checksum
b5c41a28e83f: Download complete
e39b6b4d6e0d: Verifying Checksum
e39b6b4d6e0d: Download complete
7f86327acbd2: Verifying Checksum
7f86327acbd2: Download complete
044b66fbe46c: Verifying Checksum
044b66fbe46c: Download complete
bf30769d36e7: Verifying Checksum
bf30769d36e7: Download complete
a8ac7f6c67ab: Pull complete
bf30769d36e7: Pull complete
044b66fbe46c: Pull complete
b5c41a28e83f: Pull complete
e39b6b4d6e0d: Pull complete
7f86327acbd2: Pull complete
Digest: sha256:51b2403520e425e0a7d4bc00cfd395d3b06078d79cf7927bee1035d662814b8c
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.232.0
gitea.dooplex.hu/admin/felhom-controller:0.232.0
[golden] asking the controller which infra images it manages …
[golden] baking infra images (4): traefik:v3.6.7 cloudflare/cloudflared:2026.6.0 gtstef/filebrowser:1.3.3-stable gitea.dooplex.hu/admin/felhom-samba:1.1.0 …
v3.6.7: Pulling from library/traefik
589002ba0eae: Pulling fs layer
ef63511ea6cc: Pulling fs layer
0738e5cb835e: Pulling fs layer
3e6813f70c64: Pulling fs layer
3e6813f70c64: Waiting
ef63511ea6cc: Download complete
589002ba0eae: Verifying Checksum
589002ba0eae: Download complete
3e6813f70c64: Verifying Checksum
3e6813f70c64: Download complete
0738e5cb835e: Verifying Checksum
0738e5cb835e: Download complete
589002ba0eae: Pull complete
ef63511ea6cc: Pull complete
0738e5cb835e: Pull complete
3e6813f70c64: Pull complete
Digest: sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a
Status: Downloaded newer image for traefik:v3.6.7
docker.io/library/traefik:v3.6.7
2026.6.0: Pulling from cloudflare/cloudflared
47de5dd0b812: Pulling fs layer
c172f21841df: Pulling fs layer
99515e7b4d35: Pulling fs layer
99ba982a9142: Pulling fs layer
d6b1b89eccac: Pulling fs layer
2780920e5dbf: Pulling fs layer
7c12895b777b: Pulling fs layer
3214acf345c0: Pulling fs layer
52630fc75a18: Pulling fs layer
dd64bf2dd177: Pulling fs layer
b839dfae01f6: Pulling fs layer
ebddc55facdc: Pulling fs layer
bdfd7f7e5bf6: Pulling fs layer
2d4d7adf6272: Pulling fs layer
40008157d8d2: Pulling fs layer
bd8962e29291: Pulling fs layer
cac2ae0193cb: Pulling fs layer
74d1dac84ecc: Pulling fs layer
dd64bf2dd177: Waiting
b839dfae01f6: Waiting
ebddc55facdc: Waiting
bdfd7f7e5bf6: Waiting
2d4d7adf6272: Waiting
40008157d8d2: Waiting
bd8962e29291: Waiting
cac2ae0193cb: Waiting
74d1dac84ecc: Waiting
2780920e5dbf: Waiting
7c12895b777b: Waiting
3214acf345c0: Waiting
52630fc75a18: Waiting
99ba982a9142: Waiting
d6b1b89eccac: Waiting
c172f21841df: Verifying Checksum
c172f21841df: Download complete
47de5dd0b812: Download complete
99515e7b4d35: Verifying Checksum
99515e7b4d35: Download complete
99ba982a9142: Verifying Checksum
99ba982a9142: Download complete
d6b1b89eccac: Download complete
2780920e5dbf: Verifying Checksum
2780920e5dbf: Download complete
7c12895b777b: Verifying Checksum
7c12895b777b: Download complete
3214acf345c0: Verifying Checksum
3214acf345c0: Download complete
47de5dd0b812: Pull complete
52630fc75a18: Verifying Checksum
52630fc75a18: Download complete
dd64bf2dd177: Verifying Checksum
dd64bf2dd177: Download complete
b839dfae01f6: Verifying Checksum
b839dfae01f6: Download complete
ebddc55facdc: Verifying Checksum
ebddc55facdc: Download complete
bdfd7f7e5bf6: Download complete
2d4d7adf6272: Verifying Checksum
2d4d7adf6272: Download complete
bd8962e29291: Download complete
c172f21841df: Pull complete
40008157d8d2: Download complete
cac2ae0193cb: Verifying Checksum
cac2ae0193cb: Download complete
74d1dac84ecc: Verifying Checksum
74d1dac84ecc: Download complete
99515e7b4d35: Pull complete
99ba982a9142: Pull complete
d6b1b89eccac: Pull complete
2780920e5dbf: Pull complete
7c12895b777b: Pull complete
3214acf345c0: Pull complete
52630fc75a18: Pull complete
dd64bf2dd177: Pull complete
b839dfae01f6: Pull complete
ebddc55facdc: Pull complete
bdfd7f7e5bf6: Pull complete
2d4d7adf6272: Pull complete
40008157d8d2: Pull complete
bd8962e29291: Pull complete
cac2ae0193cb: Pull complete
74d1dac84ecc: Pull complete
Digest: sha256:ba461b8aa9c042156dbd39c38657fe7431bafa063220eab8d5330a523863da9f
Status: Downloaded newer image for cloudflare/cloudflared:2026.6.0
docker.io/cloudflare/cloudflared:2026.6.0
1.3.3-stable: Pulling from gtstef/filebrowser
6a0ac1617861: Pulling fs layer
ef8806083e82: Pulling fs layer
b74107c861c7: Pulling fs layer
adc935def003: Pulling fs layer
4f4fb700ef54: Pulling fs layer
18695ccc900a: Pulling fs layer
45d119d5c397: Pulling fs layer
dac52db4fc51: Pulling fs layer
6d598f86b2f2: Pulling fs layer
8aa349c8396c: Pulling fs layer
adc935def003: Waiting
4f4fb700ef54: Waiting
18695ccc900a: Waiting
45d119d5c397: Waiting
dac52db4fc51: Waiting
6d598f86b2f2: Waiting
8aa349c8396c: Waiting
6a0ac1617861: Verifying Checksum
6a0ac1617861: Download complete
adc935def003: Verifying Checksum
adc935def003: Download complete
b74107c861c7: Verifying Checksum
b74107c861c7: Download complete
4f4fb700ef54: Verifying Checksum
4f4fb700ef54: Download complete
6a0ac1617861: Pull complete
45d119d5c397: Verifying Checksum
45d119d5c397: Download complete
18695ccc900a: Verifying Checksum
18695ccc900a: Download complete
dac52db4fc51: Verifying Checksum
dac52db4fc51: Download complete
6d598f86b2f2: Verifying Checksum
6d598f86b2f2: Download complete
8aa349c8396c: Verifying Checksum
8aa349c8396c: Download complete
ef8806083e82: Download complete
ef8806083e82: Pull complete
b74107c861c7: Pull complete
adc935def003: Pull complete
4f4fb700ef54: Pull complete
18695ccc900a: Pull complete
45d119d5c397: Pull complete
dac52db4fc51: Pull complete
6d598f86b2f2: Pull complete
8aa349c8396c: Pull complete
Digest: sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70babab78c
Status: Downloaded newer image for gtstef/filebrowser:1.3.3-stable
docker.io/gtstef/filebrowser:1.3.3-stable
1.1.0: Pulling from admin/felhom-samba
897d797d2723: Pulling fs layer
3051591aa250: Pulling fs layer
ce57a3f93416: Pulling fs layer
fb94eeec2fe1: Pulling fs layer
fb94eeec2fe1: Waiting
ce57a3f93416: Verifying Checksum
ce57a3f93416: Download complete
897d797d2723: Verifying Checksum
897d797d2723: Download complete
fb94eeec2fe1: Verifying Checksum
fb94eeec2fe1: Download complete
3051591aa250: Verifying Checksum
3051591aa250: Download complete
897d797d2723: Pull complete
3051591aa250: Pull complete
ce57a3f93416: Pull complete
fb94eeec2fe1: Pull complete
Digest: sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.1.0
gitea.dooplex.hu/admin/felhom-samba:1.1.0
[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'.
[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'.
[golden] baking the first-boot SSH host-key regeneration unit (F3) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'.
[golden] identity-clean + minimize …
[golden] stop + archive …
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/var/lib/felhom') in backup
INFO: archive file size: 627MB
INFO: Finished Backup of VM 9100 (00:00:30)
[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_09_01-10_43_08.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive)
[golden] publishing golden (657494489 bytes, sha256 5f8a53ed5b19a6cb…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.232.0/golden.tar.zst
[golden] pre-delete existing: HTTP 404 (404/204 expected)
[golden] upload OK (HTTP 201)
GOLDEN_VERSION=0.232.0
GOLDEN_SHA256=5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8
[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.232.0 / 5f8a53ed5b19a6cb2006298ce6239f6fca2b990cc3ef6eada89f602801ca91b8
[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)