felhom-agent/REPORT.md

REPORT — v0.120.0 + golden 3.0.0: one data volume (R-165 · D-a · V-c), 2026-08-03

Overwritten per the standing rule. The prior contents (R-168, CI) have their durable record in CHANGELOG.md.

Outcome: the merge is BUILT, PROVEN AT THE BAKE, and NOT YET PROVEN ON A BOX. Everything through Phase 5 is shipped and green. Phases 6–7 — reinstalling the two demo boxes from the merged golden and proving one end to end — were NOT done, and the golden is deliberately left unvouched as a result. That is a scope reduction against what was approved at the STOP and it is stated here first rather than at the end. Remaining work: R-178.

---

1. Baselines — all three matched §1, re-read on arrival

Repo	§1	found	match
felhom-agent	587dbb43fe97, v0.119.0, script 2.1.0	same	yes
felhom-controller	d5be67b91363, v0.191.2	same	yes
felhom.eu	1806dfa8e9f6, hub v0.89.0	same	yes

Highest register ID R-177; R-178 grepped free before minting.

2. Phase 0 — the two probes (full detail: felhom.eu/documentation/audits/SPIKE-r165-phase0-2026-08-03.md)

P1 — PASS. A real pre-merge archive (mp0+mp1, confirmed from its own vzdump log, not assumed) restore-tested on demo-hp: pass: true, mount_parity: "ok", 84 s, inventory listing mp0=50G and mp1=20G. mountParity was not relaxed. Limit stated: it ran with the pre-merge agent because the merged one did not exist yet; the comparison is archive-vs-its-own-restore and never consults the host layout, so it carries — and it is one command to re-run against v0.120.0.

P2 — all three variants mechanically clean; the deciding fact was not in the task's table. Each built as a throwaway LXC with the golden's own daemon.json, then measured at first boot and after each of three reboots.

	V-a	V-b	V-c (chosen)
dockerd + docker run, reboots	3/3	3/3	3/3
ONE filesystem, ONE free figure	yes	yes	yes
container statfs("/") = merged volume	yes	yes	yes
container's view of /mnt	8.0K	17.9M — all of Docker	8.0K
customer data inside Docker's data-root	YES	no	no

The ordering risk that motivated the probe did not materialise — an /etc/fstab bind is ordered under local-fs.target, before docker.service. What separates them is which documented guarantee they break. V-c was probed because both named variants broke one, and it is offered/chosen as the shape that breaks neither. The operator ruled V-c at the STOP.

P3 — 8 checks, 8 passed. Every retargeted golden assertion was run against a deliberately wrong shape (a real split-layout guest and a real archive of it) and shown to reject it, and against a right shape and shown to accept it. Guest and archive torn down.

3. Files changed and commits

Repo	Commit	Subject
felhom-controller	4be6467	v0.192.0 — the capture floor replaces the bulkhead (B2)
felhom-agent	cd6e267	v0.120.0 — one data volume (V-c), golden 3.0.0
felhom.eu	7406ac7	audits: R-165 Phase 0
felhom.eu	e3525e6	host-install: one data volume, derived from the disk

Agent files: configs/build-golden.sh (2.1.0 → 3.0.0), internal/reconcile/bringup.go, internal/reconcile/bringup_test.go, cmd/felhom-agent/main.go, CHANGELOG.md.

4. The variant, and the golden bake as evidence

[golden] build-golden.sh v3.0.0 — baking controller …felhom-controller:0.192.0
[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds
  docker OK (overlay2; data-root /var/lib/docker)
  /var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4
  /mnt/sys_drive is a real mount:  /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4
  both paths are ONE filesystem:   /dev/mapper/pve-vm--9100--disk--1  23317576
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/var/lib/felhom') in backup      <-- and NO mp1 line at all
INFO: archive file size: 619MB

Published: registry HTTP 200, felhom-golden/0.192.0, sha 54e2a4c431daf580d2807b82d810be36…. Build guest 9100 destroyed; scratch scripts and credential file removed from the host.

Baked on felhom-pve, not DooPlex. The historical bake fixture is DooPlex's nested drill VM, and this task forbids DooPlex outright — so it ran on a Tier-0 demo host, which the task names as the target. demo-hp was avoided for the probes and bake because target-selection.md records its local-lvm as an over-subscribed thin pool backing a live guest, and it carries no template.

5. Provisioning — and a census that changed the answer

--sysdata-grow is FOLDED IN, not removed. The task's rule was to remove it unless something outside the repo passes it. My first census said nothing did — and it was wrong, because I truncated it with head -10 and the output was exactly 10 lines. That is §9.11's own trap ("an instrument that can silently drop results is not a measurement") and this project's own recorded lesson. Re-run without truncation: felhom.eu/scripts/felhom-host-install.sh:2621 passes it.

So: the flags stay accepted (removing them would fail every install on an unknown flag), and their GiB are added to the single volume's grow rather than dropped — dropping them would silently shrink every appliance an older installer builds by 42 of 250 GiB. The existing test was retargeted to pin the fold, and it caught the behaviour change before I did.

§8.2 — the single volume's size IS derived from the physical disk, and already was. The same census found felhom-host-install.sh:1776 step_grows reading the thin pool's real free space; the merge only collapsed its 80/20 split into one total (226 = the previous 184+42), so a standard appliance keeps exactly its 250 G. An unflagged install does not get the golden's 24 G. No row needed to be filed for this.

6. The floor's constants, and the proof it sits below the warning band

97% used or 1 GiB free, whichever binds first — fillwatch's two-term shape, which proved live on 2026-08-02 when the critical alert fired on the free-byte term at 91% used. Deliberately beyond fillwatch's critical band (95% / 2 GiB), so the chain is warn (85/5) → critical (95/2) → refuse (97/1) and the customer is always warned before a refusal. TestFloorSitsBelowTheCriticalWarningBand pins the ordering on both terms; a red-proof setting the floor equal to the critical band fails it.

1 GiB is a reserve, not a working budget: §7.5 measures a DB-backed unit at up to ~2× its data, so no fixed number guarantees a capture fits. What it guarantees is the bulkhead's actual job — that a capture cannot consume the last of the space the container runtime needs.

7. Tests and red-proofs

Controller 1184 → 1191; agent suite green; all gates green in all three repos.

#	mutated	result
C	the whole floor block removed	3 tests FAIL
D	headroom predicate → per-unit cap (UsedGB > 20)	2 tests FAIL
E	floor set equal to fillwatch's critical band	1 test FAILS
P3	4 retargeted shell assertions vs a wrong shape	8/8 rejected/accepted correctly

One fixture was hollow and was strengthened mid-red-proof. TestFloor_TheOld20GCeilingIsGone sat at exactly 20 GB, so it survived a literal UsedGB > 20 cap — it passed the very shape it forbids. Raised to 120 GB; the mutation now fails it. Reported rather than quietly fixed, per §9.10's rule that a mutation must remove every guard a test covers.

8. Deployed versions

what	where	version
controller	guest 9201 (felhom-pve)	0.192.0, healthy
agent	felhom-pve	0.120.0, active
agent	demo-hp	0.120.0, active
golden script	repo	3.0.0
golden archive	registry	0.192.0, HTTP 200 — UNVOUCHED on purpose

Prior agent binaries backed up in place as /root/felhom-agent.pre-0.120.0.bak on both hosts.

9. WHAT WAS NOT DONE — Part 4, and why

Neither demo box was reinstalled, and nothing was wiped. I stopped before the destructive phase because the remaining session budget could not confidently cover a wipe plus its end-to-end proof plus the second box, and a half-reinstalled box is worse than an untouched one. Both demo boxes are running normally on the split layout.

Consequently the golden is deliberately UNVOUCHED. Vouching is what makes a fresh install pick a golden up; vouching one no box has been proven from would put an unproven disk layout in front of the next install anywhere. Prove first, then vouch (CONTEXT.md S-14). The golden being published but unvouched is a safe, reversible state — nothing consumes it.

Nothing in the field is affected. No box has the merged layout, nothing assumes it at runtime (§8.5), and agent v0.120.0 folds the retired grow so an older installer still produces the same capacity. The work is filed as R-178 with the ordering (demo-hp first, proven, then demo-felhom).

10. Teardown — all three layers

layer	evidence
machine	probe guests 9401/9402/9403 destroyed; P3 guest 9410 destroyed; golden build guest 9100 destroyed; P1's scratch 990000 torn down by the restore-test itself. pct list on both hosts shows only their own 9201
host	felhom-pve local-lvm 112398205 → 107204406 KiB (5.19 GB returned) after the probes; P3's archive freed with pvesm free; /root/{build-golden.sh,.regcred.env,p2-probe.sh,p3-assertions.sh} and /tmp/p*.log removed
hub	none created, verified not assumed — no probe ran a controller or claimed a box. Registers hold the same 5 customers and 4 hosts as before. The reinstalls that would have produced NEW host records did not happen, so no old records needed disposing

11. Register rows

Opened: R-178. Closed: R-163 (the ceiling it records no longer exists), R-175 (§7.5 fixed in the same pass). Updated: R-165 → SHIPPED-not-yet-proven-live; R-176 → (a) answered by P1, (b) withdrawn because every node is reinstalled rather than migrated.

12. CI and --no-verify

--no-verify NOT used — every push ran .githooks/pre-push and passed. CI run ids are recorded in felhom.eu/REPORT.md.

13. Observations — noticed, NOT acted on

• I gated a wait on DONE. golden archive volid, which the bake prints BEFORE publishing, and briefly reported the publish as failed. It had not failed; a 649 MB upload was still running. No harm (the archive is on disk and the guest destroy did not touch it), but it is the same class as the truncated census: a marker that is not a completion marker. Worth a [golden] publish OK line at the true end of the script — not added here, out of scope.
• The felhom-golden package listing endpoint returns a shape my query did not expect; the direct artifact URL is the reliable check.