diff --git a/REPORT.md b/REPORT.md index 6fc8bd0..0f449dd 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,41 +1,189 @@ -# REPORT — CI runs the gate entry point on every push (R-168, 2026-08-02) +# REPORT — v0.120.0 + golden 3.0.0: one data volume (R-165 · D-a · V-c), 2026-08-03 -**Overwritten** per the standing rule. The prior contents (session 1's gate entry point, same day) have their durable record in `CHANGELOG.md`. +**Overwritten** per the standing rule. The prior contents (R-168, CI) have their durable record in +`CHANGELOG.md`. -**No version bump, no build, no deploy.** This adds `.gitea/workflows/gates.yml` and a CHANGELOG -entry. Nothing compiled changed. +**Outcome: the merge is BUILT, PROVEN AT THE BAKE, and NOT YET PROVEN ON A BOX.** Everything through +Phase 5 is shipped and green. **Phases 6–7 — reinstalling the two demo boxes from the merged golden +and proving one end to end — were NOT done**, and the golden is deliberately left **unvouched** as a +result. That is a scope reduction against what was approved at the STOP and it is stated here first +rather than at the end. Remaining work: **R-178**. -## What changed +--- -`.gitea/workflows/gates.yml` — on every push, a Gitea Actions runner obtains this repo at the -**exact pushed SHA** (shallow `git fetch`, no `uses:` step anywhere) and runs -`python3 scripts/agent_gates.py --fast` and nothing else. The exit code is the job's result: no `|| true`, no -pipe that could swallow it. +## 1. Baselines — all three matched §1, re-read on arrival -**It REPORTS, it cannot REFUSE**, and the workflow header says so in the pre-push hook's voice: this -repo pushes straight to `main` with no pull request, so there is no merge for a status check to stand -at. The refusing half is `.githooks/pre-push` (per-clone, `--no-verify`-able); this half notices when -that was skipped. Making CI blocking needs branch protection plus a PR workflow — an operator -decision, tracked as `felhom.eu` `OPEN-ITEMS.md` **R-169**. +| Repo | §1 | found | match | +|---|---|---|---| +| `felhom-agent` | `587dbb43fe97`, v0.119.0, script 2.1.0 | same | **yes** | +| `felhom-controller` | `d5be67b91363`, v0.191.2 | same | **yes** | +| `felhom.eu` | `1806dfa8e9f6`, hub v0.89.0 | same | **yes** | -**A failed run emails the operator.** Probe P5 measured that Gitea itself sends **nothing** on a -failed run — no mail, no notification row, no log line — so the workflow sends its own via Resend and -prints the provider's accepted id, making "a message left the machine" an observable. Demonstrated on -a real red run in `felhom.eu`: `RESEND-ACCEPTED id=5ff34766-c5f8-4588-8104-08296aeb45ab`. +Highest register ID **R-177**; `R-178` grepped free before minting. -**CI reproduces the workspace's SIBLING LAYOUT on purpose.** This repo's entry point invokes the -shared `reuse_refs_check.py` that lives in the `felhom.eu` clone next door and is deliberately never -copied here, and this repo's `REUSE.md` cites `hub/internal/store/dr_recipe.go`, which lives in the hub. The workflow clones -`felhom.eu` as a sibling; without it the gate fails **closed** with `gate is MISSING` — correctly, -but for the wrong reason. +## 2. Phase 0 — the two probes (full detail: `felhom.eu/documentation/audits/SPIKE-r165-phase0-2026-08-03.md`) -## Verification +**P1 — PASS.** A real pre-merge archive (`mp0+mp1`, confirmed from its own vzdump log, not assumed) +restore-tested on demo-hp: `pass: true`, `mount_parity: "ok"`, 84 s, inventory listing `mp0=50G` and +`mp1=20G`. **`mountParity` was not relaxed.** Limit stated: it ran with the pre-merge agent because the +merged one did not exist yet; the comparison is archive-vs-its-own-restore and never consults the host +layout, so it carries — and it is one command to re-run against v0.120.0. -First run: run #1, id=9, conclusion **success**, sha `eb991445`. +**P2 — all three variants mechanically clean; the deciding fact was not in the task's table.** Each +built as a throwaway LXC with the golden's own `daemon.json`, then measured at first boot and after +**each of three reboots**. -**CI and the local pre-push hook AGREE**, which is the check that mattered: `90 cited paths — exact 88, suffix 1, ambiguous 0, cross-repo 1, FAILED 0 (siblings searched: felhom.eu)` — identical to the local run. +| | V-a | V-b | **V-c (chosen)** | +|---|---|---|---| +| dockerd + `docker run`, reboots | 3/3 | 3/3 | **3/3** | +| ONE filesystem, ONE free figure | yes | yes | **yes** | +| container `statfs("/")` = merged volume | yes | yes | **yes** | +| container's view of `/mnt` | 8.0K | **17.9M — all of Docker** | **8.0K** | +| customer data inside Docker's data-root | **YES** | no | **no** | -The runner is unprivileged host-mode, shared with the other three repos on a single owner-scoped -registration (measured: all four repos' tasks claimed by `felhom-gates-runner`). Full probe detail, -the security posture and the teardown: `felhom.eu/documentation/audits/SPIKE-ci-runner-2026-08-02.md` -and `felhom.eu/REPORT.md`. +**The ordering risk that motivated the probe did not materialise** — an `/etc/fstab` bind is ordered +under `local-fs.target`, before `docker.service`. What separates them is which documented guarantee +they break. **V-c was probed because both named variants broke one**, and it is offered/chosen as the +shape that breaks neither. **The operator ruled V-c at the STOP.** + +**P3 — 8 checks, 8 passed.** Every retargeted golden assertion was run against a **deliberately wrong +shape** (a real split-layout guest and a real archive of it) and shown to reject it, and against a +right shape and shown to accept it. Guest and archive torn down. + +## 3. Files changed and commits + +| Repo | Commit | Subject | +|---|---|---| +| `felhom-controller` | `4be6467` | v0.192.0 — the capture floor replaces the bulkhead (B2) | +| `felhom-agent` | `cd6e267` | v0.120.0 — one data volume (V-c), golden 3.0.0 | +| `felhom.eu` | `7406ac7` | audits: R-165 Phase 0 | +| `felhom.eu` | `e3525e6` | host-install: one data volume, derived from the disk | + +Agent files: `configs/build-golden.sh` (2.1.0 → **3.0.0**), `internal/reconcile/bringup.go`, +`internal/reconcile/bringup_test.go`, `cmd/felhom-agent/main.go`, `CHANGELOG.md`. + +## 4. The variant, and the golden bake as evidence + +``` +[golden] build-golden.sh v3.0.0 — baking controller …felhom-controller:0.192.0 +[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds + docker OK (overlay2; data-root /var/lib/docker) + /var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4 + /mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4 + both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576 +INFO: including mount point rootfs ('/') in backup +INFO: including mount point mp0 ('/var/lib/felhom') in backup <-- and NO mp1 line at all +INFO: archive file size: 619MB +``` + +Published: registry **HTTP 200**, `felhom-golden/0.192.0`, sha `54e2a4c431daf580d2807b82d810be36…`. +Build guest 9100 destroyed; scratch scripts and credential file removed from the host. + +**Baked on `felhom-pve`, not DooPlex.** The historical bake fixture is DooPlex's nested drill VM, and +this task forbids DooPlex outright — so it ran on a Tier-0 demo host, which the task names as the +target. demo-hp was avoided for the probes and bake because `target-selection.md` records its +`local-lvm` as an over-subscribed thin pool backing a live guest, and it carries no template. + +## 5. Provisioning — and a census that changed the answer + +**`--sysdata-grow` is FOLDED IN, not removed.** The task's rule was to remove it unless something +outside the repo passes it. **My first census said nothing did — and it was wrong, because I truncated +it with `head -10` and the output was exactly 10 lines.** That is §9.11's own trap +("an instrument that can silently drop results is not a measurement") and this project's own recorded +lesson. Re-run without truncation: **`felhom.eu/scripts/felhom-host-install.sh:2621` passes it.** + +So: the flags stay accepted (removing them would fail every install on an unknown flag), and their GiB +are **added** to the single volume's grow rather than dropped — dropping them would silently shrink +every appliance an older installer builds by 42 of 250 GiB. The existing test was **retargeted** to pin +the fold, and it caught the behaviour change before I did. + +**§8.2 — the single volume's size IS derived from the physical disk, and already was.** The same +census found `felhom-host-install.sh:1776 step_grows` reading the thin pool's real free space; the +merge only collapsed its 80/20 split into one total (`226` = the previous `184+42`), so **a standard +appliance keeps exactly its 250 G**. An unflagged install does **not** get the golden's 24 G. No row +needed to be filed for this. + +## 6. The floor's constants, and the proof it sits below the warning band + +**97% used or 1 GiB free**, whichever binds first — `fillwatch`'s two-term shape, which proved live on +2026-08-02 when the critical alert fired on the free-byte term at 91% used. **Deliberately beyond +fillwatch's critical band (95% / 2 GiB)**, so the chain is warn (85/5) → critical (95/2) → refuse +(97/1) and the customer is always warned before a refusal. `TestFloorSitsBelowTheCriticalWarningBand` +pins the ordering on **both** terms; a red-proof setting the floor equal to the critical band fails it. + +1 GiB is a **reserve, not a working budget**: §7.5 measures a DB-backed unit at up to ~2× its data, so +no fixed number guarantees a capture fits. What it guarantees is the bulkhead's actual job — that a +capture cannot consume the last of the space the container runtime needs. + +## 7. Tests and red-proofs + +Controller **1184 → 1191**; agent suite green; all gates green in all three repos. + +| # | mutated | result | +|---|---|---| +| C | the **whole** floor block removed | 3 tests FAIL | +| D | headroom predicate → per-unit cap (`UsedGB > 20`) | 2 tests FAIL | +| E | floor set **equal** to fillwatch's critical band | 1 test FAILS | +| P3 | 4 retargeted shell assertions vs a wrong shape | **8/8 rejected/accepted correctly** | + +**One fixture was hollow and was strengthened mid-red-proof.** `TestFloor_TheOld20GCeilingIsGone` sat +at exactly 20 GB, so it **survived** a literal `UsedGB > 20` cap — it passed the very shape it forbids. +Raised to 120 GB; the mutation now fails it. Reported rather than quietly fixed, per §9.10's rule that +a mutation must remove every guard a test covers. + +## 8. Deployed versions + +| what | where | version | +|---|---|---| +| controller | guest 9201 (felhom-pve) | **0.192.0**, healthy | +| agent | felhom-pve | **0.120.0**, `active` | +| agent | demo-hp | **0.120.0**, `active` | +| golden script | repo | **3.0.0** | +| golden archive | registry | **0.192.0**, HTTP 200 — **UNVOUCHED on purpose** | + +Prior agent binaries backed up in place as `/root/felhom-agent.pre-0.120.0.bak` on both hosts. + +## 9. WHAT WAS NOT DONE — Part 4, and why + +**Neither demo box was reinstalled, and nothing was wiped.** I stopped before the destructive phase +because the remaining session budget could not confidently cover a wipe **plus** its end-to-end proof +**plus** the second box, and a half-reinstalled box is worse than an untouched one. Both demo boxes are +running normally on the split layout. + +**Consequently the golden is deliberately UNVOUCHED.** Vouching is what makes a fresh install pick a +golden up; vouching one no box has been proven from would put an unproven disk layout in front of the +next install anywhere. **Prove first, then vouch** (`CONTEXT.md` S-14). The golden being published but +unvouched is a safe, reversible state — nothing consumes it. + +**Nothing in the field is affected.** No box has the merged layout, nothing assumes it at runtime +(§8.5), and agent v0.120.0 folds the retired grow so an older installer still produces the same +capacity. **The work is filed as R-178** with the ordering (demo-hp first, proven, then demo-felhom). + +## 10. Teardown — all three layers + +| layer | evidence | +|---|---| +| **machine** | probe guests 9401/9402/9403 destroyed; P3 guest 9410 destroyed; golden build guest 9100 destroyed; P1's scratch 990000 torn down by the restore-test itself. `pct list` on both hosts shows only their own `9201` | +| **host** | felhom-pve `local-lvm` 112398205 → 107204406 KiB (**5.19 GB returned**) after the probes; P3's archive freed with `pvesm free`; `/root/{build-golden.sh,.regcred.env,p2-probe.sh,p3-assertions.sh}` and `/tmp/p*.log` removed | +| **hub** | **none created, verified not assumed** — no probe ran a controller or claimed a box. Registers hold the same 5 customers and 4 hosts as before. The reinstalls that would have produced NEW host records did not happen, so no old records needed disposing | + +## 11. Register rows + +**Opened:** R-178. **Closed:** R-163 (the ceiling it records no longer exists), R-175 (§7.5 fixed in +the same pass). **Updated:** R-165 → SHIPPED-not-yet-proven-live; R-176 → (a) answered by P1, (b) +**withdrawn** because every node is reinstalled rather than migrated. + +## 12. CI and `--no-verify` + +**`--no-verify` NOT used** — every push ran `.githooks/pre-push` and passed. CI run ids are recorded +in `felhom.eu/REPORT.md`. + +## 13. Observations — noticed, NOT acted on + +- **I gated a wait on `DONE. golden archive volid`, which the bake prints BEFORE publishing**, and + briefly reported the publish as failed. It had not failed; a 649 MB upload was still running. No harm + (the archive is on disk and the guest destroy did not touch it), but it is the same class as the + truncated census: **a marker that is not a completion marker.** Worth a `[golden] publish OK` line at + the true end of the script — not added here, out of scope. +- The `felhom-golden` package listing endpoint returns a shape my query did not expect; the direct + artifact URL is the reliable check.