Operator-present drill on demo-felhom guest 9201. Preflight refused with the key removed, went green after one 60s tick with the SAME MainPID (1993397 both sides, so no restart), and all 45 config keys came back identical. Positive control run before the change so the green afterwards is a measurement, not an artefact of the probe. Marker sha unchanged throughout.
10 KiB
REPORT — the seed that never ran twice, and three pictures that were not true (2026-08-08)
Four defects of one family, each with a source-verified mechanism, tests and red-proofs.
Agent v0.128.0 · controller v0.210.0 · gates.yml (workflow only). The hub was not touched,
not bumped and not deployed.
1. Part 1's live sequence — RUN, OPERATOR-PRESENT, PASSED
On demo-felhom guest 9201, agent 0.128.0. Backup taken first and verified byte-identical
(sha256 eff18437…), pbsdr marker sha recorded and unchanged at every step
(fd4c97b5…).
Step 3 — the preflight with the key removed (today's defect, live):
{"id": "pbs_storage_id", "ok": false, "detail": "escrow.pbs_storage_id not configured"}
Step 4 — one 60 s tick later, same daemon:
{"id": "pbs_storage_id", "ok": true, "detail": "felhom-pbs"}
No restart: agent MainPID 1993397 before and 1993397 after — the same process. The wizard would have refused at 18:04:16Z and would pass at 18:05:44Z, with nobody touching the box.
Step 5 — nothing else moved: 45 keys in the backup, 45 now, added none / removed none / changed none. The one key is back with its original value.
Positive control: the preflight was exercised BEFORE the change and reported ok: true on the
same row, so a green afterwards is a measurement and not an artefact of the probe.
2. The writer of agent.json — ESTABLISHED
step_agent_config, felhom.eu/scripts/felhom-host-install.sh:2396; the Python render at
:2449; the O_TRUNC write at :2579. PRESERVE_FROM defaults empty (:256) and is set
only by an explicit --preserve-from (:1246). The render never writes an escrow section at
all — grep over the whole heredoc: zero hits. The pbsdr marker is host-side
(<agent-state>/pbsdr/marker.json) and survives. R-221's attribution was correct.
A rebuild is only the case that was measured; the same hole opens for a hand-edited or restored config, which is the honest reason the fix is at the seam rather than in the installer.
3. Red-proofs — 6 of 6, each with the mutation asserted applied
| # | mutation | assertion it applied | outcome |
|---|---|---|---|
| 1 | Part 1 / Scenario A: remove the new seed call | marker MUTATED: the R-221 re-assert removed present |
RED — and yes, it failed against today's tree, with the intended message ✔ |
| 2 | Part 1: remove the early return as well | marker MUTATED: early return deleted present |
RED — the zero-Proxmox-calls assertion is load-bearing, not decorative ✔ |
| 3 | Part 3 / F: revert to status.LastDBDump.Success |
marker present | RED — app X's false green returns ✔ |
| 4 | Part 3 / G: map "no result" to ok |
marker present | RED — green-on-presence returns ✔ |
| 5 | Part 2 / D: ignore DiskKnown in the template |
marker present | RED — „0.0 GB / 0.0 GB (0%)" in the nominal colour returns ✔ |
| 6 | Part 2 / E: force the flag false | marker present | RED — a healthy box is shown losing its numbers ✔ |
All six restored and re-verified green. Answer to the question asked directly: the Part 1 test DID fail against today's tree.
4. The Hungarian strings as shipped
„A tárhely mérete most nem olvasható ki."— the disk caveat line„nem ismert"— the short label in the value slot„Erről a mentésről nincs eredményünk."— thetitleon the no-verdict backup mark
5. The §7.3 truth table as implemented
| this app's own most recent dump result | restore point | verdict |
|---|---|---|
| any of its databases failed | yes | error |
| all clean | yes | ok |
| none recorded | yes | no icon, time only, with the title above |
| any | no | no tier-1 row, unchanged |
Recency was left alone, deliberately: an age threshold means inventing a number, and the time is already printed beside the icon. Recorded as an observation.
6. The CI timeout, and what is still unknown
timeout-minutes: 5 — every honest run in the observed session finished in 18–34 s, so 5 min
is ~9× the slowest honest run and far under whatever reaped run 264 at 834 s. The alarm mail now
carries Elapsed (a start stamp in step 1 via $GITHUB_ENV; an absent stamp prints
unknown (no start stamp), never a bogus number), and its "names itself in the run log" sentence is
qualified so it cannot mislead when there is no log.
THE UNKNOWN IS NOT CLOSED. Whether the if: failure() alarm fires at all for a reaped job is
still unverified. The timeout makes the reap unreachable in practice; it does not answer what
happens inside one. Demonstrating it would mean deliberately hanging a run on main, which would
leave the branch red for a parallel session, so it was not done. Said in the workflow comment, the
changelog, R-265 and here — four places, none of them claiming it is answered.
7. Tests
| before | after | |
|---|---|---|
| controller | — | 1355 total (+7 this session: 4 verdict, 3 disk-meter) |
| agent | — | 947 total (+5 this session) |
go build ./... && go vet ./... && go test ./... green in both repos (controller 28 packages,
agent 29), run separately from every commit. controller_gates.py --fast all OK; agent_gates.py --fast all OK; repo_gates.py --fast all 8 OK.
The dashboard test extracts the meter block from the shipped template rather than copying it — a copied block drifts and then passes while the page it covers has changed.
8. Deploy
| version | evidence | |
|---|---|---|
| agent | 0.128.0 | felhom-agent --version on felhom-pve; service active; prior binary kept as felhom-agent.bak-0.127.0 |
| controller | 0.210.0 | docker ps on guest 9201: felhom-controller:0.210.0 Up (healthy) |
Endpoint-level validation of the controller UI was ATTEMPTED AND DID NOT SUCCEED — stated rather
than skipped. What was tried: login POST against the container IP 172.17.0.2:8080 with the
mandatory Host: header, first with curl's cookie jar and then with the Set-Cookie handled
explicitly (the known felhom_session jar trap). Login returned 302 and / returned 302 back to
login both times, so the dashboard was never rendered. Fallback observable, on the deployed
artifact rather than the source — grep -a inside the running container's binary:
SystemInfo.DiskKnown ×1, the disk caveat line ×6, the no-verdict title ×1, --version →
0.210.0 (commit c732fe1). That proves the shipped bytes carry both fixes; it does not prove the
rendered page, and the template-level tests are what stand for that.
9. The bake, and the three Day-0 values
documentation/tests/golden-0.210.0-2026-08-08/ — golden 0.210.0, 656 787 777 B, sha256
b9f701fa…4c0a00, round-trip verified, ./etc/felhom-controller-image read out of the downloaded
archive → felhom-controller:0.210.0. Markers all green, token grep 0 with a control returning 1,
bake VM destroyed, drill disk restored to virgin.
| field | set to | verified |
|---|---|---|
golden_version |
0.210.0 | package GET 200; hub dropdown offers it, data-sha matches the bake |
agent_version |
0.128.0 | package GET 200; hub dropdown offers it, data-sha matches the deployed binary |
min_agent |
0.127.0 (unchanged) | read from the CHANGELOG header written this session |
⚠ The agent was NOT published until this session checked, and it mattered. R-221's fix is in the
agent; the binary had been hand-deployed and never published, so agent_version 0.128.0 was not
selectable and a fresh install would have received 0.127.0 — the golden would have carried the
controller fixes and not the one the headline defect needed. Caught by checking each value was
fetchable instead of assuming. Published from the live-deployed bytes, sha-verified across the
hop first (c6eba73b… identical on both sides).
min_agent stays 0.127.0 deliberately: MinAgent declares what the controller requires, and
v0.210.0 requires nothing new from the agent. R-221 is delivered by agent_version, not by the floor.
Nothing was vouched. No hub setting was touched. The Save is the operator's.
10. Gate failures remaining
None. golden_currency_gate.py went red the moment the controller was bumped — correct, and
closed by the bake, not by --no-verify. No --no-verify anywhere in this session.
11. Register
Closed: R-221, R-259, R-258, R-265 (the last with its unknown explicitly still open).
Minted: R-266 — the failed root statfs still travels to the hub as a 0-of-0 disk; ranked
low because it is the quiet direction, and now a two-repo wire change governed by G-1's gate.
Highest ID moved R-265 → R-266. G-3 unblocked in ROADMAP.md; CONTEXT S-39 rules the
convention.
Still open, untouched: R-246, R-255, R-256, R-257, R-261, R-262, R-263, R-264, R-240, R-243, R-202, R-213, R-244, R-214/R-235, C7's test-comment half, and G-8's other half.
12. The capability-map row
00-capability-map.md:93 — "A failed per-app Tier-1 backup reaches the OPERATOR". Checked, and
it was NOT claiming something untrue: it is about the operator notification path and claims nothing
about what /backups/apps draws. But its narrative — "the page you open to ask whether ONE app is
backed up" — invites the wrong reading, and the adjacent thing WAS false: the page's tick was green
on presence until v0.210.0, so the two halves disagreed and only the operator half was true. The row
now records that.
13. Observations — noticed, NOT acted on
- Other collectors in
info_linux.goreturn silently on error —readLoadAvg,readMemInfoand the temperature read. Only the disk one was traced to a customer-visible surface, and the change was deliberately not widened into a refactor of that file. - The tick's recency weakness stands. A tick over a three-week-old restore point is still a tick. Adding an age threshold means inventing a number; the time is printed beside it.
- The controller UI could not be driven headlessly this session (§8). Worth one session to re-establish the documented headless login, because "invoke the endpoint the UI invokes" is this project's standard validation method and it is currently unavailable for the controller.
HDDKnownis wired but has no template consumer yet — the HDD path renders throughStorageBars, which has its ownDisconnectedstate. Adding the flag there is the natural next step of the S-39 conversion and is part of G-3's survey, not this session.