felhom.eu/REPORT.md

felhom.eu — task reports

Overwrite this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in hub/CHANGELOG.md; the scripts history lives in scripts/CHANGELOG.md.

---

REPORT — Golden rebuild 0.98.3 (drill B5 + B1) — docs half (2026-07-03)

Implementation half in felhom-agent/REPORT.md (build-golden.sh v2.0.0 @ ceca355). Full drill evidence: documentation/audits/DRILL-golden-098-2026-07-03.md (A–D transcripts, unit states, resolution-order + fetch/sha proofs, cleanup, observations). This repo's changes are docs-only.

Baselines

Repo	Base → head
felhom.eu	2e33a8b → this push (docs)
felhom-agent	c9f963d → ceca355 (script + CHANGELOG)
felhom-controller	untouched; 0.98.3 reconfirmed current + pullable, and is the baked tag

What shipped (system-level)

• Golden 0.98.3 — bakes controller 0.98.3 + the felhom-controller-bootstrap.path unit; published to Gitea (felhom-golden/0.98.3/golden.tar.zst, HTTP 201, round-trip sha b9a02ef1b6f02b9b58babc4c6aad9cf6c053ebdfba116c78c8e7830de757fd01) and operator-vouched in the Day-0 manifest (now: agent 0.63.0 + golden 0.98.3 — verified via /api/v1/artifacts).
• B5 dead: clean-room Day-0 install (Scenario C, local golden; Scenario D, vouched Gitea fetch + sha verify) lands controller 0.98.3 on first boot; selfupdate/check reports up-to-date → the box self-manages; agent selftest clean; bentopdf deploys + answers 200.
• B1 dead: isolated proof — service condition-failed + path active (waiting) on a mount-less boot; pct set -mp9 … against the RUNNING guest started the controller in ~1 s, no reboot (uptime -s unchanged). Installer v1.9.1 reboot retained as belt (removal = recorded cleanup).

Docs changed (this repo)

• documentation/runbooks/day0-install.md — D.1b retired to a one-line selfupdate/check verification; old procedure → Part F troubleshooting row keyed on "golden older than 0.86.0"; header versions line (script v1.9.1 / agent v0.63.0 / golden v0.98.3); A.3 drilled-known-good pair + vouch-≥0.98.3 note; A.4 floor text rewritten + raise-floor recommendation.
• documentation/audits/DRILL-day0-cleanroom-2026-07-03.md — ledger B1, B5 → FIXED; R6 belt-note.
• documentation/backlog/FOLLOWUP-golden-default-controller-tag.md + backlog/README.md — RESOLVED (M18/M19 convention: file kept + annotated, README entry marked FIXED; the note's :0.43.0 numbers were history — the live default had already rotted to :0.85.1, which is the form of the problem the mandatory arg kills).
• NEW documentation/audits/DRILL-golden-098-2026-07-03.md — the evidence doc.

Key proofs (short form; transcripts in the evidence doc)

Gate	Evidence
B5 red-proof	no-arg build-golden.sh dies with usage, exit 1, before any pct op (run on Windows + in the drill VM)
Scenario A	[golden] build-golden.sh v2.0.0 — baking controller …0.98.3; vzdump log: mp0 AND mp1 included; guest 9100 destroyed
Scenario B	before: ConditionPathExists … not met + path active (waiting); after mp9 hot-plug: service SUCCESS @ +1 s, container Up (healthy) 0.98.3, boot time unchanged
Scenario C	[SKIP] using local golden: …18_01_21.tar.zst (resolution order); first boot 0.98.3; update_available:false; hub rows agent 0.63.0 / controller 0.98.3; bentopdf 200
Publish	pre-delete 404 → PUT 201 → round-trip GET sha matches
Scenario D	fetching golden v0.98.3 from Gitea → verified sha256 b9a02ef1… matches the hub manifest → SUCCESS; first boot 0.98.3; up-to-date
Cleanup	all 8 drill-1 hub tables at count 0, demo-felhom + peti-felhom intact; drill VM reverted to virgin (kept); bake cred file removed

Secrets: registry read-cred via 0600 env file only; the bake script's in-guest docker logout + rm /root/.docker/config.json line is present and ran before archiving; publish used the build server's out-of-band Gitea admin credential; nothing committed.

Observations / operator follow-ups

1. SECURITY: the customer-config git.token (held by every customer box) is a Gitea admin token with package-WRITE — the bake proved it by successfully publishing with it. The manifest-sha chain protects installs from tampered artifacts, but the capability shouldn't exist customer-side: issue a scoped read-only account/token + rotate.
2. build-golden.sh's publish block auto-fires whenever REGISTRY_* is set (needed for the pull too) → it published BEFORE Scenario C; deleted (204) and re-published after the gate. Candidate cleanup: a GOLDEN_PUBLISH=1 opt-in flag.
3. The installer's post-provision reboot is now redundant (path unit wins first) — candidate removal in a future installer version; kept per the task rules.
4. Recommended: raise the global controller floor to 0.98.3 (UI, 1 min) for drift protection.
5. Drill-environment note: launching the drill VM with dhcpstart=10.0.2.30 (+ explicit hostfwd…-10.0.2.15:22) eliminates the prior drill's slirp DHCP/IP-collision quirk — worth using in every future drill.