felhom.eu/REPORT.md

REPORT — The removal that works once, and a status page that says what it is asking for (2026-08-13)

Shipped: installer-v1.28.0, published and verified against the live URL. Register: R-305 CLOSED (by R-316), R-316 / R-317 / R-318 opened; ceiling R-315 → R-318. Venue: drill-r50 only. Neither demo box reinstalled. peti-felhom not contacted. Nothing deleted.

---

1. Cycle 3, before and after

Before — on the PUBLISHED v1.27.0, from virgin, exit 1:

[ERROR]   a resolver is already bound to :53 on this host:
      udp UNCONN 0 0 0.0.0.0:53 … users:(("dnsmasq",pid=7076,fd=4)) …
[ERROR] a resolver is already bound to :53 on this host — Felhom needs the guest reachable by name on your LAN.
  Stop or reconfigure that resolver, OR point your LAN DNS at the guest's address, then re-run.
  (Felhom does NOT touch DNS services on a host it does not own — this is a refusal, not a change.)
  THIS LOOKS LIKE OURS. A previous Felhom install leaves the dnsmasq PACKAGE installed and its unit
  enabled (only our config snippet is removed), and unconstrained it binds 0.0.0.0:53 — which is what
  this gate is seeing. If this host had no dnsmasq before Felhom, clear it with:
      systemctl disable --now dnsmasq
  Then re-run this installer. If dnsmasq is YOURS, leave it and use one of the two routes above.
[ERROR] PRE-FLIGHT FAIL (exit 1) — fix the finding above and re-run

After — v1.28.0, three fresh cycles from virgin, exit 0:

:53 now: 0
CYCLE 3 rc=0
[INFO]   host DNS (:53): free
[OK] pre-flight passed
[OK] PRE-FLIGHT PASS (mode=byo) — no state written, no install step executed

The three cycles were run on bytes sha256-identical to what the live URL now serves (cca1dedd9b7c152c…, compared three ways: live URL, tested file, repo main).

2. The mechanism, at file:line

ownership recorded	felhom-host-install.sh:1718 — preflight, dpkg-query -W -f='${Status}' dnsmasq | grep -q "install ok installed"
ownership read	:1097 (_state_get dnsmasq_preexisting) at uninstall
state file deleted	:1268 — after the read. The order was already correct.
Felhom's dnsmasq snippets removed	:1162, in the loop just above the ownership decision

Why cycle 2 concludes "pre-existing": package presence alone. The preflight asks dpkg one question and nothing else — it does not consult the absence of a record, and it cannot, because the record was deleted with the state file. v1.27.0 stopped the unit and left the package, so the answer stayed yes and our own package became "the household's" one cycle later.

What the first uninstall did and did not remove: snippets — removed. Unit — stopped + disabled. State file (and with it the record) — removed. Package — left. That last one is the whole defect.

3. What v1.28.0 changes

When the record says we installed it, the uninstall removes the package as well as stopping the unit. Order unchanged: read the record → act → delete the state.

Two packages are recorded, not one. dnsmasq ships the systemd unit; dnsmasq-base ships /usr/sbin/dnsmasq (dpkg -S, measured on the box). Separately installable, so each is recorded at preflight and taken back only if we added it.

Guard rails: ownership read, never inferred; the dependency check is an apt-get -s purge simulation that proceeds only if the removal set is a subset of ours, else stop+disable naming the blocking package; never interactive; never fatal — a wedged apt is recorded and restated in the closing NOTE; and the success is re-queried rather than read off apt's exit code.

4. Scenarios and red-proofs

Scenario	Result
A three cycles, fixed	install 3 PASSES, :53 free at every uninstall
B resolver pre-dates Felhom	untouched — install ok installed, unit active, both packages recorded yes
C something depends on it	not purged; log named household-dns-thing; stop+disable; :53 free; dependent survived
D no ownership record	untouched, reason logged, exact command named

Red-proofs — mutation asserted applied before each run:

Mutation	Outcome
remove the purge call	cycle 2 records yes; cycle 3 refuses, exit 1, in those exact words
remove the ownership check	the household's resolver is PURGED (unknown ok not-installed)
infer ownership when no record	the guess is taken — a field box loses its own DNS

One honest note on red-proof B. Its first run appeared to pass for the wrong reason: apt failed with dpkg was interrupted (a broken state my own earlier manual package juggling left), so the resolver survived by accident rather than by the guard. I repaired dpkg and re-ran, and it then failed as required. Worth stating twice over: the mutation looked like a passing guard, and my own fix's apt-failure path was incidentally observed doing exactly what it should — logging, continuing, naming the command.

5. The machines already in the field

Measured, not reasoned. A field box is one with the package installed and no record — scenario D. Its uninstall leaves the resolver running with the reason logged, and its next install refuses with the message quoted in §1.

Is there an honest durable marker? No, and none can be invented. The Felhom /etc/dnsmasq.d/ felhom-*.conf snippets are deleted by the uninstall's own loop before the ownership decision; the state file carrying the record is deleted at :1268; nothing under /etc/felhom* survives. /var/log/dpkg.log does record the install and is a timestamp — refused by the standing rule as a heuristic dressed as a fact. → R-318

The preflight message, judged as a customer would: it states the finding, keeps its two routes and its written promise not to touch DNS on a host we do not own, and adds the one thing that gets a person moving — "THIS LOOKS LIKE OURS" plus one exact command. It hedges correctly (looks like). Its weakness is that it asks them "did this host have dnsmasq before Felhom?" — precisely the question we can no longer answer for them. That is a mechanism, not a rule: the command is on the screen at the moment it is needed.

6. The defect I nearly shipped

The agent decides whether to install dnsmasq with os.Stat("/usr/sbin/dnsmasq") (felhom-agent/internal/lanresolver/lanresolver.go:105) — but that path belongs to dnsmasq-base, while the unit comes from dnsmasq. Purging only dnsmasq would leave the binary, so the next install would skip the apt step and then fail to enable a unit that is gone — a silent resolver where today there is at least a visible refusal. That is why ownership is recorded per package.

It remains reachable where dnsmasq-base pre-dated Felhom (we correctly keep it). Pre-existing, not introduced here, filed as R-317, and the uninstall now says so out loud rather than leaving it to be found from a resolver that never came up. Fixing it is a one-line agent change, deliberately not made here to keep this session to one repo.

7. Publication

installer-v1.28.0 tagged; both --refs in manifests/webpage.yaml moved (sidecar 327, init 372); ArgoCD synced to 823cd29; live deployment carries the new ref.

live URL serves: SCRIPT_VERSION="1.28.0"       (was 1.27.0)
_dnsmasq_purge_owned present in the served file: 3

Pushing publishes nothing here — /scripts/ follows the tag. The runbook still says otherwise; R-309 remains open and is not forgotten, deliberately out of scope.

8. Teardown — four layers

Layer	State
machine	drill-r50: 0 guests, no /var/lib/felhom-install, dnsmasq not-installed at the end
host	drill-r50 reverted to snapshot virgin, powered off; qemu.pid removed. DooPlex was drill host only
hub	drill-r50-0a4f9a RETAINED — pre-existing since 2026-07-25, re-used not duplicated. No new host or customer row was created this session. Nothing deleted
off-site	Not contacted at all — no restic, sftp or endpoint call was made. demo-felhom: last_status=ok, escrow=escrowed, no abandon countdown

Both demo boxes untouched and healthy: controller 0.214.0, agent 0.129.0, Up … (healthy).

9. Evidence, and a repeated miss

Logs at documentation/audits/evidence-r316-2026-08-13/: the fixed cycles (F1, F2, F3), all four scenarios (B, C, D), and every red-proof (RPA*, RPB*, RPD*).

The Part 1 logs did not survive. They were on the drill VM's disk and were destroyed by the revert to virgin between Part 1 and Part 2 — the same mistake as Tuesday, in the same place. The §1 quotation is verbatim from the live run as read at the time, and RPA1/RPA2/RPA3.log are an independent reproduction of the identical three-cycle failure, retained. Recorded rather than glossed; the fix is procedural and I have now got it wrong twice.

10. Observations — noticed, not acted on

• dnsmasq-base is flagged "automatically installed and no longer required" after dnsmasq goes. We deliberately do not autoremove — that would be a blast radius nobody asked for.
• apt-get -s purge exits 0 even when it prints E: dpkg was interrupted. The simulation output is the signal, never the exit code — which is why the guard parses Remv/Purg lines rather than trusting $?. Another entry for the exit-codes-that-lie class.
• The drill VM's pveam index is stale on virgin and needs pveam update before listing templates — already recorded yesterday, hit again today in passing.