golden 0.203.0 baked + published; runbook acceptance markers fixed (R-233)
gates / gates (push) Successful in 13s

Bake evidence: documentation/tests/golden-0.203.0-2026-08-06/ (bake.log + README).
sha256 3039c6ffa7a5a8b2d959daddb2895c58b44de70f8d4f4a7e12ad4b1c0d61dc88, verified
by an independent round-trip download and by reading /etc/felhom-controller-image
out of the published archive itself. NOT vouched — the hub still serves 0.201.0.

R-233: RUNBOOK-manual-build.md §4.1 named three pass markers, two of which the
script cannot print (`overlay2 OK` does not exist; `mp1` stopped existing in
build-golden.sh v3.0.0 under R-165), and a 404 pre-gate URL with the wrong
filename, which would 404 for the wrong reason and pass even when the version
already existed. A grep for an impossible string reads 0 forever and 0 is
indistinguishable from failure. Markers re-captured from the real log; token
handling moved off the command line into an in-VM runner script; a positive
control is now required on the token-leak grep; the vouch step rewritten as the
three-field change it is (golden_version + agent_version + min_agent).
This commit is contained in:
2026-08-06 13:23:22 +02:00
parent 0c4411e54b
commit 1c47e3b6fd
4 changed files with 468 additions and 8 deletions
+1
View File
@@ -155,6 +155,7 @@ the fault was real. Full observables: `tests/campaign11-evidence-2026-08-05/jour
| **R-230** | **Three instruction/memory follow-ups deliberately left by the part-2 session (2026-08-06), each needing a decision rather than an implementation.** (a) **A ruling is owed on auto-written staleness.** The hand-written `CLAUDE.md` files are now clean of version literals and expired blocks — the gate enforces it — but `MEMORY.md`, which Claude writes and which is the LARGER half of what loads (8.4k tokens vs the root file's 6.6k), carries **21 lines with component version literals**, **5 with bare host addresses**, and an entry still reading *"demo boxes REMOTE till ~08-02"* — the same expired-TEMPORARY class the gate was built to kill, now surviving in the one file the gate's content rules do not cover. **Partly actioned 2026-08-06 (close-out), and the ruling is STILL OWED:** the **three statements that were actively false** were corrected — `R-193 decision open` (closed 2026-08-05), `demo boxes REMOTE till ~08-02` (the box answers on the home LAN), `OPEN R-25b` (shipped 2026-07-21) — and gate check 6 now **WARNs** on version literals, host addresses, expired statements and stale-open citations in the index. WARN, never FAIL: Claude writes that file between sessions, so a hard failure would refuse a human's push over a line no human typed, and the warning is read by the model that will next edit it. **The remaining 32 version literals and 4 host addresses were deliberately left** for that loop. What is still owed is the bulk-correction ruling. **Correcting the premise:** the earlier report's "three expired statements" were all FALSE POSITIVES — each matched an ISO date inside a markdown link target, i.e. a filename — while the one real expired claim carried no ISO date at all. (b) **CLOSED 2026-08-06 (close-out)** — the workspace-root `CLAUDE.md` **is now a relative symlink** to the versioned copy, so the divergence class is gone rather than policed. Check 5 learned two shapes: for a link it asserts the target resolves to a real file (**a dangling link is worse than a diverged copy — the instructions load NOTHING and there is no content left to notice is wrong**), for two files byte-identity as before, so a clone elsewhere is unaffected. **Proven, not assumed:** three fresh sessions logged `session_start` for the link path, and a fourth **with no tools at all** quoted standing rule 1 verbatim — the content reaches the model, not just the path. (c) **The spec-as-failing-test pilot**, approved in principle and not started (was R-229(d)). | **READY** — owner Viktor |
| **R-232** | **DooPlex's backup makes every copy inside the same box — and nothing tells anyone when it fails.** Surveyed read-only 2026-08-06 (`audits/RECON-dooplex-backup-2026-08-06.md`). **What works:** five sets, 14/14 successful runs in 14 days; a file was restored from the `data` repo and matched the live original **byte for byte**; every set except two is cross-disk; k3s is integrity-checked on every run. **What the matrix exposes, ranked:** (a) **`notify_failure` is a no-op** — `NOTIFY_ON_FAILURE=true` but `NOTIFY_WEBHOOK_URL` is commented out, so a failed backup notifies **nobody**; the project already has a working Resend path that CI uses. Cheapest item, and it makes every other failure visible. (b) **Nothing leaves the box** — no rclone, no remote repo, no off-site target anywhere; Longhorn's target is `nfs://192.168.0.180:` pointing at DooPlex itself, and the only outbound-looking cron pulls *inbound* from Hetzner for a different project. The machine that runs the hub managing the customers' off-site chain has no off-site copy of its own. (c) **The backup tree is a single writable path** and the restic repos are not append-only — one bad script or ransomware destroys every copy at once. (d) **Two same-disk sets**: `.claude-memory` and the PostgreSQL dumps, whose source directory sits *inside* the backup tree. (e) **Longhorn `retain=1`** — one generation per volume, so a corruption noticed a day late has no earlier copy. (f) **`/opt/backup/docs/BACKUP-RESTORE.md` does not exist** though the systemd unit advertises it. (g) **`secrets/restic-repo` has never held a snapshot** — `backup-secrets.sh` contains no `restic` call; the secrets are GPG files on `sda1` only. (h) **No restore has ever been run** beyond today's single-file probe — the matrix's "ever demonstrated?" column is otherwise entirely empty. **Not a finding:** the restic passphrase. The on-box copy is on `sdb1`, a different disk from the backups, and the **operator holds an offline copy out of band** — so a disk loss is recoverable. The narrow residual is that it is operator-held rather than system-held, unlike the customer case's hub-vaulted escrow, so it should be confirmed current and findable by someone else. **Nothing was changed by the recon.** | **READY** — owner Viktor |
| **R-231** | **`/opt/backup/scripts/` on DooPlex is unversioned host state** — found 2026-08-06 while adding the auto-memory store to the backup set. No repository tracks the scripts that protect the recovery chain, so the edit made that day (`CLAUDE_MEMORY_DIR` in `backup-config.sh`, multi-path restic call in `backup-data.sh`) exists only on the box. This is the same class the part-2 session was closing, found inside the fix for it; the change is transcribed in `felhom.eu/workspace/README.md` so it is at least *recorded*. **Two related facts, both understating current safety:** the backup destination (`/mnt/5_hdd/backup`) is on the **same physical disk** as the workspace it protects, and the DooPlex backup set has **no off-site leg** (`sync-hetzner-backups.sh` is jarrs.eu and pulls *from* Hetzner *to* DooPlex). Bringing a root-owned production backup script under version control, and deciding what installs it, is its own scoped change. | **READY** — owner Viktor |
| **R-233** | **The golden bake's acceptance checks were a list of strings the script does not print** — found 2026-08-06 while baking golden 0.203.0 by following `runbooks/RUNBOOK-manual-build.md` §4.1 verbatim. Two of the three named pass markers **cannot ever match**: `overlay2 OK` is not in `build-golden.sh` at all (the line it means is ` docker OK (overlay2; data-root /var/lib/docker)`), and `including mount point … mp1` refers to a volume that stopped existing in `build-golden.sh` **v3.0.0**, when R-165 collapsed the two data volumes into one. The 404 pre-gate's URL was also wrong — the published filename is `golden.tar.zst`, not `felhom-golden-<VER>.tar.zst`, so the pre-gate would 404 for the wrong reason and pass **even when the version already existed**. This is the *"an instrument that can silently drop results is not a measurement"* class landing on the bake's own acceptance check: a grep for an impossible string reads `0` forever, and `0` is indistinguishable from failure. **The bake was never actually unguarded** — the script's own `[ "$drv" = "overlay2" ] || { echo FATAL; exit 1; }` is fail-closed and the run exited 0 with no `FATAL`. The **document** was the broken part, which is why nothing had ever gone wrong and nobody had noticed. **FIXED in the same session:** all markers re-captured from the real log rather than paraphrased, the corrected pre-gate URL, the token handling moved off the command line into an in-VM runner script (the old `--setenv=GITEA_TOKEN=$GT` form put the value where `systemctl show` prints it), a required **positive control** on the token-leak grep, and the vouch step rewritten as the three-field change it actually is. **The general lesson:** a runbook's pass markers must be **copied from a captured log, never written from memory** — §4.0 of that same file already learned this for the qemu launch line and says so; §4.1 had not. | **CLOSED 2026-08-06** — fixed in `RUNBOOK-manual-build.md` |
**Recorded against existing rows by Phase 2:**
+49 -8
View File
@@ -139,27 +139,68 @@ qemu-system-x86_64 -enable-kvm -cpu host -smp 4 -m 8192 \
0600), then run it as a transient unit so it survives a session close, reading the token from the
file **inside** the VM so it never reaches a command line:
Put the invocation in a **runner script inside the VM** that reads the token itself, and launch
that. The older `--setenv=GITEA_TOKEN=$GT` form put the value on a command line and into the
transient unit's properties, where `systemctl show` prints it:
```bash
GT=$(cat /root/.gitea-token); systemd-run --unit=golden-bake --collect \
--setenv=GITEA_USER=admin --setenv=GITEA_TOKEN=$GT \
--setenv=REGISTRY_USER=admin --setenv=REGISTRY_TOKEN=$GT \
bash -c "/root/build-golden.sh 9100 local:vztmpl/<template> local-lvm local vmbr0 \
gitea.dooplex.hu/admin/felhom-controller:<VER> > /root/bake.log 2>&1"
cat > /root/bake-run.sh <<'EOS'
#!/bin/bash
GT=$(cat /root/.gitea-token)
export GITEA_USER=admin GITEA_TOKEN="$GT" REGISTRY_USER=admin REGISTRY_TOKEN="$GT"
exec /root/build-golden.sh 9100 local:vztmpl/<template> local-lvm local vmbr0 \
gitea.dooplex.hu/admin/felhom-controller:<VER>
EOS
chmod 0700 /root/bake-run.sh
systemd-run --unit=golden-bake --collect bash -c "/root/bake-run.sh > /root/bake.log 2>&1"
```
Confirm the value went nowhere: `systemctl show golden-bake -p Environment -p ExecStart |
grep -c -F "$(cat /root/.gitea-token)"` must be `0`. Copy the token in **file → file** (`scp`), so
it never crosses a shell on either side.
`CONTROLLER_IMAGE` is a **required** argument (a hand-bumped default rotted twice) and
`GOLDEN_VERSION` is derived from it — the golden's version IS the controller it bakes.
**The script is the publisher**: it uploads to Gitea and prints `GOLDEN_VERSION` + `GOLDEN_SHA256`.
Pass markers: `overlay2 OK`, `including mount point` for rootfs **and mp0 and mp1** with no
`excluding`/`FATAL`, `upload OK (HTTP 201)`. The 404 pre-gate applies to the package URL.
Pass markers, **each captured from a real log rather than paraphrased** — two of the three named
here until 2026-08-06 could not match anything the script prints (see the note below):
`docker OK (overlay2`, `including mount point` for rootfs **and mp0** — there is no mp1 — with no
`excluding`/`FATAL`, `upload OK (HTTP 201)`. The 404 pre-gate applies to the package URL, which is
`…/generic/felhom-golden/<VER>/golden.tar.zst` — the filename is `golden.tar.zst`, **not**
`felhom-golden-<VER>.tar.zst`.
<!--
2026-08-06, R-233: this line listed `overlay2 OK` and demanded `including mount point` for `mp1`.
The script prints neither. The storage-driver line it means is ` docker OK (overlay2; data-root
/var/lib/docker)`, and mp1 stopped existing in build-golden.sh v3.0.0 (R-165 collapsed the two data
volumes into one). A reader following this literally greps for a string that can never appear and
reads 0 — the "an instrument that can silently drop results is not a measurement" class, aimed at the
bake's own acceptance check. The real guard was never weak: the script's own
`[ "$drv" = "overlay2" ] || { echo FATAL; exit 1; }` is fail-closed. The DOCUMENT was the broken part.
-->
4. Teardown: `pct destroy 9100 --purge`, `shred -u` the token/script/log **after** copying the log out
for evidence, `poweroff`, wait for qemu to exit, `qemu-img snapshot -a virgin`. Token-leak grep on
the saved log = `grep -c -F "$(cat ~/.gitea-token)"` (the literal value — a broad `[a-f0-9]{40}`
pattern false-hits image shas).
pattern false-hits image shas). **A `0` is only evidence once the grep is shown to work**: append
the token to a throwaway copy of the log, grep that (must be `1`), `shred -u` the copy, and only
then believe the `0`. Grep the copy that gets **committed**, not just the one in the VM.
5. Hub → Configuration → Day-0 artifacts: pick the new Golden, Save. **The R-120 gate lives on this
save** (`hub/internal/web/configs.go:1165`) and REFUSES a golden older than the newest controller
the fleet reports. It does **not** run on a controller image deploy — it is not a general drift net.
**Vouching is a THREE-field change, not one.** `golden_version` alone ships a controller onto an
older agent than it declares it needs. Read the golden's controller `CHANGELOG.md` header — it
carries `MinAgent <X>` — and move all three together:
`golden_version` → the new golden, `agent_version` → ≥ that `MinAgent`, `min_agent` → that
`MinAgent`. The `min_agent` field is what the hub HOLDS a box's floor against; blank means an
uncoupled release with no gating. Setting `min_agent` **above** the vouched agent is the R-216
shape and hub v0.97.0 now HOLDS it rather than serving past it.
**Vouching is reversible**: re-select the previous values and Save. The old golden's package is
never deleted by a bake (the publish step's pre-delete targets only its own version), so rolling
back is a form submission, not a rebuild.
A golden is only needed when a publish train wants fresh installs current — demo deploys never need it.
The full 0.188.0 run, with the observables: `documentation/audits/tester-gate-golden-0.188.0-2026-07-31.md`.
@@ -0,0 +1,93 @@
# Golden 0.203.0 — bake evidence (2026-08-06)
Baked per `runbooks/RUNBOOK-manual-build.md` §4.0–§4.1, in the drill VM on DooPlex
(`/mnt/5_hdd/felhom.eu/drill/drill.qcow2`, internal snapshot `virgin`) — the sanctioned bake
exception in `runbooks/target-selection.md`. Task 4 Part 3.
**Baked, published and verified. NOT vouched** — the hub's Day-0 artifacts were read but not written,
and that is deliberate: Part 3 ends at a STOP.
## The artifact
| | |
|---|---|
| version | **0.203.0** |
| sha256 | `3039c6ffa7a5a8b2d959daddb2895c58b44de70f8d4f4a7e12ad4b1c0d61dc88` |
| size | 658,292,861 bytes (uncompressed 2,008,145,920) |
| controller baked in | `gitea.dooplex.hu/admin/felhom-controller:0.203.0` |
| MinAgent (from the controller CHANGELOG header) | **0.127.0** |
| published at | `…/api/packages/admin/generic/felhom-golden/0.203.0/golden.tar.zst` |
## Preconditions, checked rather than assumed
- Disk headroom `df -h /mnt/5_hdd /` → 24% and 50%. (Abort threshold is 90%.)
- `ps -eo comm | grep -c qemu-system-x86`**0**. Not `pgrep -f`, which self-matches and reports a
false "still running".
- `qemu-img snapshot -a virgin` **succeeded**, which independently re-proves nothing held the qcow2.
- Template **listed, not assumed**: `pveam available``debian-13-standard_13.6-1_amd64.tar.zst`
(unchanged since 2026-07-31), and `pveam download` printed `checksum verified`.
- CT 9100 absent on `virgin` before the run.
- Token verified against the Gitea API **before** starting, so a credential problem could not surface
mid-bake.
- 404 pre-gate on the **real** publish URL → 404. (The URL the runbook gave was wrong — R-233.)
- Controller image `0.203.0` manifest → 200, so the tag being baked demonstrably exists.
## Acceptance markers
The runbook's marker list was **broken** — see R-233; two of three strings could never match. These
are the real ones, from `bake.log` in this directory:
| marker | count | required |
|---|---|---|
| `docker OK (overlay2` | 1 | ≥1 |
| `including mount point` | 2 — `rootfs ('/')` and `mp0 ('/var/lib/felhom')` | rootfs + mp0 |
| `upload OK (HTTP 201)` | 1 | ≥1 |
| `excluding` | 0 | 0 |
| `FATAL` | 0 | 0 |
| `ERROR:` / `WARN:` | 0 / 0 | 0 |
| unit result | `Result=success`, `ExecMainStatus=0` | success / 0 |
There is no `mp1`: `build-golden.sh` v3.0.0 (R-165) collapsed the two data volumes into one.
## The published bytes were verified by round trip, not trusted
The bake **prints** a hash; that is the baker marking its own homework. The hash below was recomputed
on DooPlex from an independent download, and the content was then read out of that download:
1. `curl` the package → HTTP 200, 658,292,861 bytes.
2. `sha256sum``3039c6ff…dc88`**matches** the baked value, byte count matches.
3. `zstd -t` → decompresses clean to 2,008,145,920 bytes: a real archive, not an error page.
4. `tar -xO ./etc/felhom-controller-image` **out of the published archive**
`gitea.dooplex.hu/admin/felhom-controller:0.203.0`.
5. 19,358 entries under `var/lib/felhom/docker` — the baked images are on the data volume.
Step 4 is the one that matters: `GOLDEN_VERSION` is *derived from the tag argument*, so the label
could be right while the content was stale. It is not — the archive says so itself.
A third independent witness: the hub's own Day-0 dropdown now lists `0.203.0` with
`data-sha="3039c6ff…dc88"`.
## Secret hygiene
- Token copied **file → file** (`scp`), never through a shell on either side.
- Invocation moved into an in-VM runner script that reads the token itself. The runbook's
`--setenv=GITEA_TOKEN=$GT` form put the value on a command line and into the transient unit's
properties. Verified: `systemctl show golden-bake -p Environment -p ExecStart | grep -c -F <token>`
**0**.
- Token-leak grep by **literal value** on the saved log → **0**. A broad `[a-f0-9]{40}` pattern is
wrong here: it false-hits image shas.
- **Positive control on that grep**, because an absence proves nothing until the instrument is shown
to work: token appended to a throwaway copy → **1 hit**; copy `shred -u`'d; the real log's `0` is
therefore a measurement. Re-run against the committed copy in this directory → **0**.
## Teardown
`pct destroy 9100 --purge` (both logical volumes removed, `pct list` empty) · `shred -u` of the token,
`build-golden.sh`, the runner and `bake.log` **after** copying the log out · `poweroff` · qemu observed
gone in ~15 s via `ps -eo comm` · `qemu-img snapshot -a virgin`. The drill VM is back at its baseline
and DooPlex is unchanged apart from this evidence directory.
## What is NOT done here
Vouching. The hub still serves golden **0.201.0** and agent **0.125.0**, with `min_agent` 0.125.0.
See the session report for what changes on vouch and why it is a three-field change.
@@ -0,0 +1,325 @@
[golden] build-golden.sh v3.0.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.203.0
[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) …
Logical volume "vm-9100-disk-0" created.
Logical volume pve/vm-9100-disk-0 changed.
Creating filesystem with 8388608 4k blocks and 2097152 inodes
Filesystem UUID: e05e2970-4bab-420d-a32d-04bdd73fd5cd
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624
Logical volume "vm-9100-disk-1" created.
Logical volume pve/vm-9100-disk-1 changed.
Creating filesystem with 6291456 4k blocks and 1572864 inodes
Filesystem UUID: 82eacdcd-c966-4662-8fe5-d0036a44c7e4
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst'
Total bytes read: 553512960 (528MiB, 169MiB/s)
Detected container architecture: amd64
Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ...
done: SHA256:fS0llHyvLdZnXkBe4qOXJwHXZYqvlPr4hW88z838PJA root@felhom-golden
Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ...
done: SHA256:WEiExYNcAOEKEQ/cMYNA/IeFclQZrtf+YEunVDhjwF0 root@felhom-golden
Creating SSH host key 'ssh_host_rsa_key' - this may take some time ...
done: SHA256:wjJozAK6HvpRa+i8F/cdG+DPBY66vzfACKvi6Ex0jFk root@felhom-golden
[golden] starting + installing Docker (official repo, trixie channel) …
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation …
[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds …
[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) …
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
4f55086f7dd0: Pulling fs layer
4f55086f7dd0: Verifying Checksum
4f55086f7dd0: Download complete
4f55086f7dd0: Pull complete
Digest: sha256:7f4da0fc94bcece205a8c0b6f4d11c8196924654ffe5c4d1aa439b7f632048b2
Status: Downloaded newer image for hello-world:latest
docker OK (overlay2; data-root /var/lib/docker)
/var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4
/mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4
both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576
[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.203.0 (no registry cred at deploy) …
WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/
0.203.0: Pulling from admin/felhom-controller
039e6f9f9752: Pulling fs layer
80fbb63a1167: Pulling fs layer
9d6c7a1331cb: Pulling fs layer
3f0ff4614786: Pulling fs layer
e7855986fdc0: Pulling fs layer
a0fcfc238aba: Pulling fs layer
3f0ff4614786: Waiting
e7855986fdc0: Waiting
a0fcfc238aba: Waiting
9d6c7a1331cb: Verifying Checksum
9d6c7a1331cb: Download complete
039e6f9f9752: Verifying Checksum
039e6f9f9752: Download complete
3f0ff4614786: Verifying Checksum
3f0ff4614786: Download complete
e7855986fdc0: Verifying Checksum
e7855986fdc0: Download complete
a0fcfc238aba: Verifying Checksum
a0fcfc238aba: Download complete
80fbb63a1167: Verifying Checksum
80fbb63a1167: Download complete
039e6f9f9752: Pull complete
80fbb63a1167: Pull complete
9d6c7a1331cb: Pull complete
3f0ff4614786: Pull complete
e7855986fdc0: Pull complete
a0fcfc238aba: Pull complete
Digest: sha256:7fe128938fa20775dd1b82c18a50dfa3691d18137d6a0b49344c9f2df16683f6
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.203.0
gitea.dooplex.hu/admin/felhom-controller:0.203.0
[golden] asking the controller which infra images it manages …
[golden] baking infra images (4): traefik:v3.6.7 cloudflare/cloudflared:2026.6.0 gtstef/filebrowser:1.3.3-stable gitea.dooplex.hu/admin/felhom-samba:1.1.0 …
v3.6.7: Pulling from library/traefik
589002ba0eae: Pulling fs layer
ef63511ea6cc: Pulling fs layer
0738e5cb835e: Pulling fs layer
3e6813f70c64: Pulling fs layer
3e6813f70c64: Waiting
589002ba0eae: Download complete
ef63511ea6cc: Download complete
3e6813f70c64: Verifying Checksum
3e6813f70c64: Download complete
0738e5cb835e: Verifying Checksum
0738e5cb835e: Download complete
589002ba0eae: Pull complete
ef63511ea6cc: Pull complete
0738e5cb835e: Pull complete
3e6813f70c64: Pull complete
Digest: sha256:a9890c898f379c1905ee5b28342f6b408dc863f08db2dab20e46c267d1ff463a
Status: Downloaded newer image for traefik:v3.6.7
docker.io/library/traefik:v3.6.7
2026.6.0: Pulling from cloudflare/cloudflared
47de5dd0b812: Pulling fs layer
c172f21841df: Pulling fs layer
99515e7b4d35: Pulling fs layer
99ba982a9142: Pulling fs layer
d6b1b89eccac: Pulling fs layer
2780920e5dbf: Pulling fs layer
7c12895b777b: Pulling fs layer
3214acf345c0: Pulling fs layer
52630fc75a18: Pulling fs layer
dd64bf2dd177: Pulling fs layer
b839dfae01f6: Pulling fs layer
ebddc55facdc: Pulling fs layer
bdfd7f7e5bf6: Pulling fs layer
2d4d7adf6272: Pulling fs layer
40008157d8d2: Pulling fs layer
bd8962e29291: Pulling fs layer
cac2ae0193cb: Pulling fs layer
74d1dac84ecc: Pulling fs layer
dd64bf2dd177: Waiting
b839dfae01f6: Waiting
ebddc55facdc: Waiting
bdfd7f7e5bf6: Waiting
2d4d7adf6272: Waiting
40008157d8d2: Waiting
bd8962e29291: Waiting
cac2ae0193cb: Waiting
99ba982a9142: Waiting
74d1dac84ecc: Waiting
d6b1b89eccac: Waiting
3214acf345c0: Waiting
52630fc75a18: Waiting
2780920e5dbf: Waiting
7c12895b777b: Waiting
47de5dd0b812: Verifying Checksum
47de5dd0b812: Download complete
c172f21841df: Download complete
99515e7b4d35: Verifying Checksum
99515e7b4d35: Download complete
47de5dd0b812: Pull complete
99ba982a9142: Verifying Checksum
99ba982a9142: Download complete
d6b1b89eccac: Verifying Checksum
d6b1b89eccac: Download complete
2780920e5dbf: Verifying Checksum
2780920e5dbf: Download complete
7c12895b777b: Verifying Checksum
7c12895b777b: Download complete
3214acf345c0: Verifying Checksum
3214acf345c0: Download complete
c172f21841df: Pull complete
52630fc75a18: Verifying Checksum
52630fc75a18: Download complete
dd64bf2dd177: Verifying Checksum
dd64bf2dd177: Download complete
b839dfae01f6: Verifying Checksum
b839dfae01f6: Download complete
ebddc55facdc: Verifying Checksum
ebddc55facdc: Download complete
bdfd7f7e5bf6: Verifying Checksum
bdfd7f7e5bf6: Download complete
2d4d7adf6272: Verifying Checksum
2d4d7adf6272: Download complete
bd8962e29291: Download complete
cac2ae0193cb: Verifying Checksum
cac2ae0193cb: Download complete
99515e7b4d35: Pull complete
40008157d8d2: Verifying Checksum
40008157d8d2: Download complete
74d1dac84ecc: Verifying Checksum
74d1dac84ecc: Download complete
99ba982a9142: Pull complete
d6b1b89eccac: Pull complete
2780920e5dbf: Pull complete
7c12895b777b: Pull complete
3214acf345c0: Pull complete
52630fc75a18: Pull complete
dd64bf2dd177: Pull complete
b839dfae01f6: Pull complete
ebddc55facdc: Pull complete
bdfd7f7e5bf6: Pull complete
2d4d7adf6272: Pull complete
40008157d8d2: Pull complete
bd8962e29291: Pull complete
cac2ae0193cb: Pull complete
74d1dac84ecc: Pull complete
Digest: sha256:ba461b8aa9c042156dbd39c38657fe7431bafa063220eab8d5330a523863da9f
Status: Downloaded newer image for cloudflare/cloudflared:2026.6.0
docker.io/cloudflare/cloudflared:2026.6.0
1.3.3-stable: Pulling from gtstef/filebrowser
6a0ac1617861: Pulling fs layer
ef8806083e82: Pulling fs layer
b74107c861c7: Pulling fs layer
adc935def003: Pulling fs layer
4f4fb700ef54: Pulling fs layer
18695ccc900a: Pulling fs layer
45d119d5c397: Pulling fs layer
dac52db4fc51: Pulling fs layer
6d598f86b2f2: Pulling fs layer
8aa349c8396c: Pulling fs layer
4f4fb700ef54: Waiting
18695ccc900a: Waiting
45d119d5c397: Waiting
dac52db4fc51: Waiting
6d598f86b2f2: Waiting
8aa349c8396c: Waiting
adc935def003: Waiting
6a0ac1617861: Verifying Checksum
6a0ac1617861: Download complete
adc935def003: Verifying Checksum
adc935def003: Download complete
4f4fb700ef54: Verifying Checksum
4f4fb700ef54: Download complete
6a0ac1617861: Pull complete
ef8806083e82: Verifying Checksum
ef8806083e82: Download complete
45d119d5c397: Verifying Checksum
45d119d5c397: Download complete
b74107c861c7: Verifying Checksum
b74107c861c7: Download complete
dac52db4fc51: Verifying Checksum
dac52db4fc51: Download complete
6d598f86b2f2: Verifying Checksum
6d598f86b2f2: Download complete
8aa349c8396c: Verifying Checksum
8aa349c8396c: Download complete
18695ccc900a: Verifying Checksum
18695ccc900a: Download complete
ef8806083e82: Pull complete
b74107c861c7: Pull complete
adc935def003: Pull complete
4f4fb700ef54: Pull complete
18695ccc900a: Pull complete
45d119d5c397: Pull complete
dac52db4fc51: Pull complete
6d598f86b2f2: Pull complete
8aa349c8396c: Pull complete
Digest: sha256:eb3733681db8757412632c61a99ad656f0d94ed6781bb2ea114b4d70babab78c
Status: Downloaded newer image for gtstef/filebrowser:1.3.3-stable
docker.io/gtstef/filebrowser:1.3.3-stable
1.1.0: Pulling from admin/felhom-samba
897d797d2723: Pulling fs layer
3051591aa250: Pulling fs layer
ce57a3f93416: Pulling fs layer
fb94eeec2fe1: Pulling fs layer
fb94eeec2fe1: Waiting
ce57a3f93416: Verifying Checksum
ce57a3f93416: Download complete
fb94eeec2fe1: Verifying Checksum
fb94eeec2fe1: Download complete
897d797d2723: Verifying Checksum
897d797d2723: Download complete
3051591aa250: Verifying Checksum
3051591aa250: Download complete
897d797d2723: Pull complete
3051591aa250: Pull complete
ce57a3f93416: Pull complete
fb94eeec2fe1: Pull complete
Digest: sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.1.0
gitea.dooplex.hu/admin/felhom-samba:1.1.0
[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'.
[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'.
[golden] baking the first-boot SSH host-key regeneration unit (F3) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'.
[golden] identity-clean + minimize …
[golden] stop + archive …
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/var/lib/felhom') in backup
INFO: archive file size: 627MB
INFO: Finished Backup of VM 9100 (00:00:30)
[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_08_06-13_16_07.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive)
[golden] publishing golden (658292861 bytes, sha256 3039c6ffa7a5a8b2…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.203.0/golden.tar.zst
[golden] pre-delete existing: HTTP 404 (404/204 expected)
[golden] upload OK (HTTP 201)
GOLDEN_VERSION=0.203.0
GOLDEN_SHA256=3039c6ffa7a5a8b2d959daddb2895c58b44de70f8d4f4a7e12ad4b1c0d61dc88
[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.203.0 / 3039c6ffa7a5a8b2d959daddb2895c58b44de70f8d4f4a7e12ad4b1c0d61dc88
[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)