From 955a4f07b703bf842b436925fa4bc94eb463fe3f Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Thu, 13 Aug 2026 10:59:55 +0200 Subject: [PATCH] REPORT + CONTEXT: A4 reverses its own premise, the naming half, and the first R-264 reader MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A4's evidence contradicts the task's framing and the register now says so: peti-felhom is a real machine with 482 reports and a real person behind it; david -> tester-1 is a record that has never had a host, an escrow or a report. The risk is real; only the word that named it was wrong. CONTEXT gains three standing rulings: read the fact that carries the RISK (heals_last_hour, not state) and decode the one that decides the question (heal_succeeded — R-260's lesson); one secret in two situations keeps its name and changes its sentence, and the mail must name the page the machine actually shows; and evidence dies in the INTERMEDIATE revert — with the corollary that a durable citation may never point at a file whose contract is to be overwritten, which is why the R-316 report was moved to audits/ before this one was written. --- CONTEXT.md | 89 ++++++++++ REPORT.md | 513 ++++++++++++++++++++++++++++++++++++++--------------- 2 files changed, 457 insertions(+), 145 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 700c78c..d16b5cb 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -15,6 +15,95 @@ > would make one of the two audiences stop reading. `STATUS.md` is also a **view of `OPEN-ITEMS.md`** > and holds nothing of its own; this file does hold its own content, namely the standing rulings below. +## A fact the boxes send and the hub cannot read is a future false green — and one now has a reader (2026-08-13, R-319) + +**The first of R-264's twenty-one unconsumed facts is read.** The agent emitted `guest_net` on every +heartbeat for **twenty-three days** (agent v0.92.0, R-54, 2026-07-21) while the string occurred +**nowhere** in `felhom.eu/hub/` — stored as raw text in `report_json`, reaching no check, no alarm and +no screen. Hub v0.104.0 models it: `parseGuestNet`/`guestNet` in `web/hosts.go`, rendered as the +host-detail **Guest network** card. + +**The rule this instance teaches, and it generalises past guest networking:** + +- **Read the fact that carries the RISK, not the fact that is easiest to render.** Here that is + `heals_last_hour`, not `state`. A guest the watchdog keeps repairing is healthy at every instant + anyone looks and is nevertheless failing — rendering the state alone gives it a green tick, which is + the failed-disk-drawn-as-a-healthy-empty-disk shape wearing different clothes. +- **Decode the field that decides the question, not five of the eight.** `heal_succeeded` was added + mid-build for exactly this: six FAILED repairs is a guest that is down, six successful ones is a + nuisance, and the count alone cannot tell them apart. **R-260 is the standing warning** — a hub + decoder three fields short of the agent dropped the one its checker existed to answer. +- **Three absences, three sentences.** Agent too old / capable agent silent / guest state not asserted. + All render unknown; conflating them is how an operator starts ignoring a card. The state switch is an + **allow-list**, so adding a state to the agent can never silently paint it green here. +- **REMOVING an allowlist entry is what proves a reader is wired.** An allowlisted tag is *skipped* by + `wire_contract_gate.py`. Re-labelling the eight `guest_net` entries would have left the new reader's + own fields unchecked while the gate reported coverage. Deleting them moved checked tags **182 → 190** + and skipped **88 → 80** — the positive control that the wiring is real, independent of any test. +- **A card is not an alarm, and building the card first is deliberate.** No email was added: the + incident behind this (a killed `dhclient`, tunnel down 1 h 15 m, nobody told) was a **visibility** + failure, and with zero repairs ever recorded on either demo box any alarm threshold would be invented. + **The visible count is what will supply the threshold.** + +**Where the twenty stand** (R-264, counts measured from the allowlist rather than estimated): +**8 read · 5 deliberately not consumed · 1 redundant · 6 still owed a reader.** The gate grew a **third** +entry kind — `not consumed, deliberately` — because an undecided fact and a decided one must not read +alike, and *"arguably owed"* had been carried for five days as though it were a decision. + +**Two findings came out of deciding the last one rather than from building anything:** + +- **R-321** — `reporting_disabled` is *redundant* (`health.status = "disabled"` travels beside it, is + decoded, and IS rendered), **but `StalenessChecker.Check` is age-only** (`monitor/staleness.go:88+`, + sole skip `IsCustomerBlocked`), so a deliberately-silent box still alarms `node_stale` then + `node_down`. **Decoding the flag would have felt like progress and left the alarm firing** — the fix + belongs in the checker, which already holds the status it needs. +- **R-322** — `retrieval_promise_gate.py` has **never scanned the hub**, which composes every customer + e-mail, i.e. the copy read *before* any box screen. A hand scan with its own four stems returns zero, + so it is a scope gap rather than a live defect — recorded before someone writes the first hub-side + retrieval promise assuming the guard has them covered. + +## One secret in two situations keeps its NAME and changes its SENTENCE — and the mail must name the page the machine shows (2026-08-13, R-295 hub half) + +The box side shipped 2026-08-10; the hub half was dropped twice and was the last place „Visszaállító +kód" survived — in the **mails**, the one surface a customer reads *before* they see any screen. It is +now „**Beállító kód**" everywhere; the ten-word escrow code stays „**Helyreállítási kód**"; no +dashboard-code mail names both. + +**The mechanism worth carrying: the mail names a PAGE, and which page depends on state only the hub +knows.** `ReissueForReenroll` sent the *reset* mail, directing the customer to „Elfelejtett jelszó" — +but a rebuilt box has no password, so the controller computes `reset := s.authEnabled()` → false +(`felhom-controller/controller/internal/web/claim.go:279`), renders „**A szerver beállítása**", and +serves **no login page at all**. The named route was not on their screen. **The two situations are +distinguishable only at the hub, because the hub chose which call site fired** — so the fix is a new +`EmailKind` (`reenroll`), not a reworded shared template. A `default` branch falls back to the +first-setup mail, which is the safe direction and is pinned so a future reader knows it was chosen. + +**The re-enrol mail deliberately says nothing about apps or backups.** A clean-slate reinstall is +precisely where that reassurance could be false; `TestFormatClaimEmail_ReenrollPromisesNothingAboutTheData` +guards the CLAIM rather than one phrasing of it. + +**Naming is not function, and the pin says so:** `TestReenrollSplit_ChangesTheMailNotTheSecret` — the +generation still advances, the stored bcrypt hash still verifies the emailed code, no plaintext is +persisted. Box-side companion: the controller's `TestResetCode_StillAcceptedOnTheSetupPage`. + +## Evidence dies in the INTERMEDIATE revert, not the final one (2026-08-13, R-320) + +Twice in three days, on the same box at the same point, a phase's logs were destroyed by a revert to +`virgin` *between* phases (drill 2026-08-12 Phase A→B; R-316 2026-08-13 Part 1→2). Both times the +golden-bake runbook's existing *"scp the log OUT first"* was applied to the **final** teardown and not +the middle one. Both times the conclusions survived on luck. + +**Standing rule 5** now: evidence comes off at the end of the phase that produced it, before any revert +— **the pull is the LAST ACT OF THE PHASE**, not a step to remember later. Four homes: +`workspace-CLAUDE.md`, `runbooks/target-selection.md`, `RUNBOOK-rehearsal-v3.md`, `PROMPT-TEMPLATE.md` +report item 8. **And the already-gone case is documented rather than improvised:** say so plainly and +reproduce the finding independently. + +**A corollary this session hit immediately:** the second incident was recorded in `REPORT.md`, which +every session overwrites — writing tonight's report would have **destroyed the record of a destroyed +record**. It was preserved to `audits/REPORT-r316-installer-v1.28.0-2026-08-13.md` first. *A durable +citation may never point at a file whose contract is to be overwritten.* + ## Material retained is not history recoverable — ask the three questions separately (2026-08-12) **"We keep the old key" and "the customer can get their old backups back" are three questions, and diff --git a/REPORT.md b/REPORT.md index 0fbff0b..ac97310 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,170 +1,393 @@ -# REPORT — The removal that works once, and a status page that says what it is asking for (2026-08-13) +# REPORT — the small debts paid, and one unread fact given a reader (2026-08-13, evening) -**Shipped:** `installer-v1.28.0`, published and verified against the live URL. -**Register:** R-305 CLOSED (by R-316), **R-316 / R-317 / R-318 opened**; ceiling R-315 → **R-318**. -**Venue:** `drill-r50` only. Neither demo box reinstalled. `peti-felhom` not contacted. Nothing deleted. +**Shipped: hub v0.104.0, live and verified.** Parts A and B complete; **Part C dropped**, named in §12. +No controller change, no agent change, no wire change — **so there was no bake and no approval**, as the +task expected. --- -## 1. Cycle 3, before and after +## 1. A4 FIRST — what the fleet actually is, and the entry corrected -**Before — on the PUBLISHED v1.27.0, from `virgin`, exit 1:** +**The task's premise for A4 was wrong, and the evidence is not ambiguous.** It asked me to establish +the facts and not accept the framing; doing that reverses the conclusion. Two different records have +been called "the tester", and only one of them carries the risk. + +**Read from the hub's own store** (`hub.db` + `-wal` + `-shm`, copied together — the WAL rule): + +| | `peti-felhom` | `david` → `tester-1` | +|---|---|---| +| customer row | created 2026-02-27 | `david` created 2026-08-01, **deleted today 07:55:49**; `tester-1` created **07:56:47** | +| host row | `peti-felhom-86d37d`, deleted 2026-07-15 08:56:22 (`host_deletions` id 1) | **never, either name** | +| controller reports | **482**, 2026-02-27 → 2026-07-15 08:39:00 | **0** | +| host reports | present until deletion | **0** | +| escrow rows | none | none | +| events | staleness alarms after it went silent | **4 total** — three hub-side `expected_dbdump_missed` false alarms (R-195's subject) and its own deletion | + +**So:** + +- **"The tester" in this project's own vocabulary is Peti** — `documentation/pilot/PETI-tester-agreement.md` + says *"Operator: Viktor. Tester: Peti"* — and the machine is **a real 80-core Proxmox server + belonging to a real person**, running Felhom as a BYO guest. It reported for four and a half months. + `target-selection.md` puts it at **Tier 2 — protected**, *"because there is a real person behind it"*. + **The 3.6 GB with no key, no off-site copy and no local backup is REAL, and the entry keeps its rank.** +- **`tester-1` is a record with no machine** — no host, no escrow, no report, ever; and `david` before + it was the same. **A record with no machine behind it can lose nothing.** + +**Reconciling this with what the operator said** (*"there is no actual tester yet — only a pre-created +customer, now renamed"*): both statements are true of different things. The **pilot programme** never +began — the agreement was drafted 2026-07-09, the onboarding runbook stopped at P1 — and the +pre-created record is `tester-1`. **Meanwhile the hardware and the data have existed the whole time.** +Nothing about the risk changed; only the word that names it. + +**The corrected entry, as shipped** (`STATUS.md`, "Broken, or knowingly incomplete"): + +> - **Peti's machine has no recovery route at all** — see the `PETI` row. **This is a real machine +> belonging to a real person**, not one of ours and not a record: it reported to the hub for four and a +> half months and has been silent since 15 July, when its host record was deleted. There is no key, no +> off-site copy and no local backup. **If that drive fails, everything on it is lost.** First act of the +> visit: copy the ~3.6 GB off before anything is reinstalled — it is currently the only copy in +> existence. Whether it stays parked is your call and is deliberately left open. + +The `PETI` register row carries the full disambiguation with every count measured, and ends +*"Wherever a document says 'the tester's machine', read `peti-felhom`."* + +**What the fleet actually is, in one sentence, now on `STATUS.md`:** the hub holds **five customer +records and three machines** — `demo-felhom` and `demo-hp` (ours, disposable), `drill-r50` (a nested +drill VM, reverted and powered off); `peti-felhom` is a real machine we have not heard from since 15 +July and has no host record; `tester-1` is a record with no machine. + +--- + +## 2. The hub strings as shipped, with bytes confirmed + +Enumerated at `file:line` before editing. All written from explicit bytes and verified as hex — no +non-ASCII crossed a shell chain; the mojibake check (`Ã`/`Å`/`â€`) is clean in every touched file. + +| Where | Was | Is | Hex of the shipped name | +|---|---|---|---| +| `notify/templates.go:216` subject | „Jelszó-visszaállítási kód" | **„Beállító kód a jelszavad visszaállításához"** | `4265c3a16c6cc3ad74c3b3206bc3b364…` | +| `notify/templates.go:221` body | „Visszaállító kód: %s" | **„Beállító kód: %s"** | `4265c3a16c6cc3ad74c3b3206bc3b3643a202573` | +| `notify/templates.go` **new** `reenroll` | *(did not exist)* | subject **„Új beállító kód — újratelepült a szervered"**, body names **„A szerver beállítása"** | `224120737a6572766572206265c3a16c6cc3ad74c3a1736122` | +| `notify/templates.go:72` lockout | „beállító/visszaállító kód próbálkozás" | **„beállító kód próbálkozás"** | `…6265c3a16c6cc3ad74c3b3206bc3b364…` | +| `web/templates/customer_unified.html:476` | „Visszaállító kód küldése" | **„Beállító kód küldése"** | `4265c3a16c6cc3ad74c3b3206bc3b364206bc3bc6c64c3a97365` | + +**The page-naming fix, established at `file:line` rather than assumed.** `ReissueForReenroll` sent the +**reset** mail (`claim/engine.go:181`), which directs the customer to an „Elfelejtett jelszó" page. A +**rebuilt** box has no password, so the controller computes `reset := s.authEnabled()` → false +(`felhom-controller/controller/internal/web/claim.go:279`), renders **„A szerver beállítása"**, and +serves **no login page at all** — the named route is not on their screen. Only the hub can tell the two +situations apart, because the hub chose which call site fired, so the fix is a **new `EmailKind`**, not +a reworded shared template. Same secret, same name, different sentence. + +The re-enrol mail deliberately says **nothing** about apps or backups — +`TestFormatClaimEmail_ReenrollPromisesNothingAboutTheData` pins it. A clean-slate reinstall is +precisely where such a reassurance could be false, and this project has spent four register rows +removing promises it could not see were still true. + +**Live-verified** (endpoint-level, the exact URL the operator UI serves; ClusterIP + Basic auth): +`GET /customers/demo-felhom` → **„Beállító kód küldése" present, „Visszaállító kód" absent**. + +### Had the claim guard ever scanned the hub? **No — never.** + +`retrieval_promise_gate.py` lives in `felhom-controller/controller/scripts/`; its declared surfaces are +that repo's `internal/web/templates` plus **one** Go handler file, added on 2026-08-12 by R-311 on the +express ground that the highest-stakes customer copy *"had never been scanned"*. **The same sentence is +true one repo over** — the hub composes every customer e-mail, i.e. the copy a customer reads *before* +they see any box screen. + +**I scanned it by hand with the gate's own four stems** (`visszaállíthat`, `visszaszerezhet`, +`visszahozhat`, `visszanyit`) across every non-test `.go`/`.html` under `hub/internal/`: **zero +occurrences.** So nothing was hiding, and my new strings contain no stem and need no registration. +**It is a scope gap, not a live defect — filed as R-322 rather than fixed**, because pointing a +controller gate at a sibling repo makes a controller gate fail on a felhom.eu edit, and G-1 already +taught this project what a cross-repo gate costs. *(A recommendation not followed gets its line: I +recommend a hub-side sibling in `repo_gates.py` sharing ONE stem list — two copies of a word list is +how the plural got past the singular in R-299.)* + +--- + +## 3. The runbook correction, and where else the claim lived + +`runbooks/day0-install.md` §C.1 said *"there is no release tag… pushing `scripts/felhom-host-install.sh` +publishes it."* **It has published nothing since R-110 shipped on 2026-08-03.** §C.1 now opens by naming +its own former error and states the three acts that actually publish: + +1. bump `SCRIPT_VERSION` and push to `main` — **publishes nothing**; +2. cut and push the tag `installer-v`; +3. move **BOTH** `--ref=installer-v…` pins in `manifests/webpage.yaml` — the git-sync **sidecar** and the + **init container** (lines 327 and 372) — commit, sync. **The one-pin trap is named**: the running pod + keeps serving until it restarts, and a fresh pod seeded by a stale init container then serves the OLD + script with no error anywhere. + +**How to verify from outside**, since a push, a green sync and a correct-looking manifest are each +consistent with nothing having been published: ``` -[ERROR] a resolver is already bound to :53 on this host: - udp UNCONN 0 0 0.0.0.0:53 … users:(("dnsmasq",pid=7076,fd=4)) … -[ERROR] a resolver is already bound to :53 on this host — Felhom needs the guest reachable by name on your LAN. - Stop or reconfigure that resolver, OR point your LAN DNS at the guest's address, then re-run. - (Felhom does NOT touch DNS services on a host it does not own — this is a refusal, not a change.) - THIS LOOKS LIKE OURS. A previous Felhom install leaves the dnsmasq PACKAGE installed and its unit - enabled (only our config snippet is removed), and unconstrained it binds 0.0.0.0:53 — which is what - this gate is seeing. If this host had no dnsmasq before Felhom, clear it with: - systemctl disable --now dnsmasq - Then re-run this installer. If dnsmasq is YOURS, leave it and use one of the two routes above. -[ERROR] PRE-FLIGHT FAIL (exit 1) — fix the finding above and re-run +curl -fsS https://felhom.eu/scripts/felhom-host-install.sh | grep -m1 '^SCRIPT_VERSION' ``` -**After — v1.28.0, three fresh cycles from `virgin`, exit 0:** +**Measured while writing it: served `1.28.0`, `main` `1.28.0`, both pins `installer-v1.28.0`** — the +three agreeing is the observation; any one alone is not. -``` -:53 now: 0 -CYCLE 3 rc=0 -[INFO] host DNS (:53): free -[OK] pre-flight passed -[OK] PRE-FLIGHT PASS (mode=byo) — no state written, no install step executed -``` +**Copied elsewhere? Yes, once, and it was hunted.** +`audits/SPIKE-universal-iso-3-2026-07-31.md:184` says the same thing **and cites `day0-install.md` as +its source** — which is how it spread. It was **true on the day it was written**, so the dated finding +is kept verbatim and carries a SUPERSEDED note; falsifying a dated record to tidy it is its own defect. +Two other hits are correct in context: `hostinstall_gates.py:198` states the consequence of the +manifest *losing* its tag, and the 2026-08-12 drill record already names the sentence as false. -The three cycles were run on bytes **sha256-identical** to what the live URL now serves -(`cca1dedd9b7c152c…`, compared three ways: live URL, tested file, repo main). +--- -## 2. The mechanism, at `file:line` +## 4. The evidence rule, as written and where it lives + +> **Evidence is copied off the machine at the end of the phase that produced it — before any revert, +> snapshot restore or teardown. Not at the end of the session.** +> +> **The mechanism, because a rule without one is a wish:** the last act of a phase that ran on a machine +> is `scp`/`pct pull` of its logs into the evidence directory on DooPlex — the same act that ends the +> phase, not a separate step to remember later. +> +> **When a session notices the evidence is already gone: say so plainly in the report and REPRODUCE it +> independently.** That is the documented expectation, not an improvisation invented under pressure. + +**Four homes**, chosen so a session meets it before the revert rather than after: + +1. `runbooks/workspace-CLAUDE.md` — **standing rule 5**, so it loads in every session, with the + two-incident record in the R-96 rationale comment beneath it. +2. `runbooks/target-selection.md` — its own section, in the doc you read *before* picking a machine to + break. Tier 0 machines are disposable, which is exactly why nothing you need may be left on one. +3. `RUNBOOK-rehearsal-v3.md` — in the standing-rules block, flagged as applying to every phase boundary. +4. `PROMPT-TEMPLATE.md` — a new **report item 8**, so a session must state that it did this. + +**Provenance, both occurrences verified, not recalled:** 2026-08-12 the retained-key drill lost its +Phase A logs to the revert to `virgin` between Phase A and Phase B +(`audits/DRILL-retained-key-2026-08-12.md` §11.5); 2026-08-13 R-316 lost its Part 1 logs, **same box, +same revert, same point** (§9 — *"the same mistake as Tuesday, in the same place"*). Both times the +existing "scp the log OUT first" was applied to the **final** teardown and not the **intermediate** one. +Filed as **R-320**. + +**A side effect worth naming:** that second record lived in `REPORT.md`, which this report overwrites — +**writing tonight's report would have destroyed the record of a destroyed record.** It was preserved to +`audits/REPORT-r316-installer-v1.28.0-2026-08-13.md` first, and both citations now point there. + +--- + +## 5. The three rulings, recorded as decided + +**`STATUS.md`'s "Waiting on you" section is now empty**, and says so: *"Nothing. All three questions +that stood here were answered on 12–13 August and have moved to Decided below."* A new **Decided** +section carries them, and each register row's status field changed from `READY` to `DECIDED`. + +| Ruling | Trigger that reopens it | +|---|---| +| **R-312 — recovering an old backup stays a phone call.** Not built, deliberately; the operator path genuinely works (the 2026-08-12 drill restored planted files byte-identical by hand) | **A real need appearing — one request from a customer who is not us.** Until then R-304's honest position stands: retention is operator-only, and nothing may promise the customer can do it | +| **R-313 — the unopenable set-aside store on `demo-felhom` is KEPT, as a test fixture.** The specific advantage the operator accepted: it is the only state in existence where a set-aside store is present and **cannot be opened** — a case that cannot be manufactured without destroying another key on purpose | **Delete it when the work it is a fixture FOR ships, or is abandoned** — i.e. when R-312 is built, or when R-312's ruling is made permanent. On either event, delete deliberately and record why | +| **R-303 — a machine in two kinds of trouble says both things: left as it is.** Real-world likelihood unknown; the tidier fix risks hiding a genuine second failure | **An observation of the combined state occurring OUTSIDE a constructed test.** One sighting on a real machine reopens it | + +--- + +## 6. B0 — do the facts arrive, and is this hub-only? + +**Yes, and yes.** Both halves established before any code was written. + +- **On the wire and in the database.** `demo-felhom-8363b5`'s newest `host_reports` row carries + `guest_net.checked_at` plus per-guest `vmid/state/mode/ip/has_route/dhclient_alive/checked_at/message`. + The agent's wire type (`felhom-agent/internal/hub/report.go:139-160`) additionally declares `healed`, + `heal_succeeded`, `last_heal_at`, `heals_last_hour`, `damped` — absent from the live rows only because + they are `omitempty` on a box that has never needed a repair. +- **Read by nothing.** The string `guest_net` occurred **nowhere** in `felhom.eu/hub/` — no struct, no + template, no checker. Stored as raw text inside `report_json`. + +**So no wire change and no agent change were needed: hub-only.** The halt condition did not fire. + +--- + +## 7. Reclassification counts — measured, not estimated + +The register row said *twenty-one*; **the gate's allowlist held twenty.** Twenty is what the +dispositions account for, exactly: + +| State | Count | Which | +|---|---|---| +| **Read** (removed from the allowlist) | **8** | `guest_net` + its seven children | +| **Deliberately not consumed** (new third kind, with the ruling and its date) | **5** | `mgmt_plane.healed_recently`, `pbs_dr.applied_at`, `config_hash`, `stacks`, `storage.migrated_to` | +| **Redundant** (decided on its own merits) | **1** | `reporting_disabled` | +| **Still owed a reader** (R-264, open) | **6** | `selfupdate_pending`, `selfupdate_pending_version`, `restore_tests.mount_parity`, `restore_tests.mount_inventory`, `backup.last_db_dump`, `backup.last_integrity_check` | + +**The eight are REMOVED rather than re-labelled, and that is the point:** an allowlisted tag is +*skipped* by the gate, so leaving them would have meant the new reader's own fields were never checked +for reachability at all. + +**A thing the task told me to re-read from the register, which was not there.** The operator's +dispositions were made on 2026-08-12; **R-264 still read `READY — owner Viktor` and all twenty +allowlist entries still said "arguably owed"**. A session told to re-read them from the register would +have found none. They are written down now, which is the point of writing them down. + +**`reporting_disabled`, decided on its own merits — and the decision found a real defect the flag would +not have fixed.** The product does support the state: the controller sends one minimal report carrying +`reporting_disabled: true` **and** `health.status: "disabled"` (`cmd/controller/main.go:1246-1260`), +then goes quiet by design. The hub **already decodes** `health_status` and **already renders** that +customer as `disabled` (`web/rollup.go:25`) — so the flag is a second spelling of a fact already read: +**redundant**. But `StalenessChecker.Check` (`monitor/staleness.go:88+`) is **age-only**; its sole skip +is `IsCustomerBlocked`. **A deliberately-silent box still goes `node_stale` at 30 minutes and +`node_down` at 60** — exactly the false alarm B0 predicted. **Filed as R-321 and deliberately not fixed +here:** decoding the flag would have felt like progress and left the alarm firing. The fix belongs in +the checker, which already holds the status it needs. + +--- + +## 8. The reader as built, and the alarm judgement + +**`hub/internal/web/hosts.go`** — `parseGuestNet` / `guestNet` / `guestNetView`; rendered as a **Guest +network** card on the host-detail page, which is where a person looks at a machine. + +**The signal is the repair count, not the state.** A guest the watchdog keeps repairing is healthy at +every instant anyone looks and is nevertheless failing; rendering `state` alone would give it a green +tick — the exact shape of the failed-disk-drawn-as-a-healthy-empty-disk defect. `heals_last_hour` sits +**beside** the state and drives the summary badge. + +**`heal_succeeded` is decoded too, and that was a deliberate addition mid-build.** My first cut carried +the count and not the outcome — which is **R-260 exactly**: a hub decoder three fields short of the +agent, missing the one that decides the question. Six *failed* repairs is a guest that is down; six +successful ones is a nuisance. + +**The four scenarios, and what the card shows for each — all four verified by test, and three of them +observed on the live fleet:** + +| | Rendered | Live sighting | +|---|---|---| +| **A** healthy, no repairs | `healthy` badge, address, route, dhclient, `0` repairs | **`demo-felhom-8363b5`** — 9201 healthy, `192.168.0.149` (dhcp), swept `08:45:33Z`; **`demo-hp-bb76ea`** — 9201 healthy, `192.168.0.118`, swept `08:52:35Z` | +| **B** repaired repeatedly | `needs attention`; the **count** rendered as a warn badge with the last repair time; a sentence naming the 1 h 15 m incident | not yet on hardware — neither demo box has needed a repair since the watchdog shipped | +| **C** does not report it | **`unknown`, never healthy** — three absences, three different sentences | **`drill-r50-0a4f9a`** — *"A capable agent sent no guest-network stanza"* → `unknown`. **The unknown branch fired on real data, not only in a test** | +| **D** malformed | `unknown`, and the page returns **200** | covered by test; the decode error is swallowed on purpose so one box's bad field cannot break the page for the fleet | + +Two extra branches beyond the four: an unrecognised guest `state` renders unknown (the switch is an +allow-list, so adding a state to the agent can never silently paint it green), and an **empty guest +list with a fresh sweep is NOT silence** — the agent's contract says that means "the watchdog ran and +found nothing", which must stay distinguishable from "the watchdog is not wired", the shape the +v0.91.0 inert seam hid behind. + +### The alarm judgement: **no email, deliberately** + +I agree with the task's recommendation and did not add one. The reasoning, since it is a judgement: + +- **The incident was a visibility failure, not a paging failure.** Nobody could *see* the condition for + 1 h 15 m. A card fixes what actually broke. +- **A new alarm on a fleet of two demo machines is untested noise**, on a dispatcher whose severity + contract is exact-match lowercase and whose customer-message allow-list must move with it — a + two-place change that fires against no real population. +- **There is no calibrated threshold to alarm on.** Neither demo box has produced a single repair since + the watchdog shipped, so any number I picked would be invented. **The visible count is what will + supply the threshold** — which is the argument for building the card first and the alarm second. + +**Revisit when a third machine exists, or when a repair count is seen climbing on real hardware.** + +--- + +## 9. Red-proofs — every mutation asserted applied before its run + +Five in total: three mandated for B2, two for A2. Each mutation was **confirmed present by grep before +the test ran**, and confirmed **absent by grep after restoring**. + +| # | Mutation | Asserted applied | Outcome | +|---|---|---|---| +| **1** *(the one that matters)* | The unknown branches in the card's badge chain replaced by the healthy badge | `grep -c 'title="MUTATED">healthy'` → **2** | **RED — both C sub-cases.** A silent machine seen rendering as healthy: *"a silent box must SAY it is unknown"* and *"an old agent's silence must be named as an old agent's silence"* | +| **2** | `RepairCount: 0` in the decoder — the climbing-repairs signal dropped | `hosts.go:385 RepairCount: 0, … // MUTATED` | **RED — B**: *"a guest repaired 6 times in an hour is reported as fine — the whole point of the card"* | +| **3** | `Degraded()` forced `true` — the healthy branch broken | `hosts.go:348 func … { return true /* MUTATED */ }` | **RED — A**: *"a machine that is fine is being alarmed on"* — the guard is reachable in **both** directions | +| **4** | `ReissueForReenroll` routed back to `EmailReset` | `grep` → line 195 reads `EmailReset` | **RED — two tests**: the new `TestReenrollSplit_ChangesTheMailNotTheSecret` **and** the pre-existing `TestReissueForReenroll` | +| **5** | „Visszaállító kód: %s" restored in the reset mail body | `grep -n` → `templates.go:241` | **RED — `TestFormatClaimEmail_OneNamePerSecret`** | + +**After every restore the suite is green again**, and `grep -c MUTATED` returns **0** in both touched +files. + +**A positive control that the reader is WIRED and not merely written**, independent of the tests: the +wire-contract gate's checked-tag count rose **182 → 190** and its skipped count fell **88 → 80** as the +eight allowlist entries came out. The gate still passes — meaning those eight tags are now genuinely +reachable in the hub. + +--- + +## 10. Part C — **DROPPED**, and one part of it answered anyway + +**Dropped, as the task's own drop order specifies.** The honest reason is not only time: **the task +names "nine claims" that are grey, and I could not identify which nine.** The capability map holds +**22** rows marked `IMPLEMENTED`, five `PARTIAL` and four `MISSING`, and nothing in the tree enumerates +a set of nine. Picking nine myself and reporting on them would have produced a confident answer to a +question nobody asked — the A4 failure mode, one section later. + +**The one I can answer, because the task identified it precisely.** The self-heal claim is the +capability-map row *"Box survives an unattended app or guest-network failure… it is noticed, and where +safe it is repaired"*, and its best document does **not** argue against it. The 1 h 15 m outage +(`audits/INCIDENT-guest-dhclient-killed-2026-07-20.md`) is the row's **origin**, not its evidence: the +evidence is a **deliberate replay of that incident the next day** — `kill -9` at 12:43:18, detected on +process liveness **57 s** later while the lease was still live, healed at 12:45:18 with the incident's +verbatim invocation, and **the tunnel never dropped** (`cloudflared Up 29 hours`). That is a walk, and +the outage was prevented rather than merely observed. **The claim does not need to move down.** + +**What I would flag instead:** the row's *guest-network* leg was proven on the box and, until today, +**the hub could not see the condition at all** — which is the gap this session closed and which no +capability row expressed. My new row states its own status honestly as **IMPLEMENTED, not +PROVEN-LIVE**, because **no machine has ever been observed with a climbing repair count on this card.** + +--- + +## 11. Versions, commits, deploy, CI | | | |---|---| -| ownership **recorded** | `felhom-host-install.sh:1718` — preflight, `dpkg-query -W -f='${Status}' dnsmasq \| grep -q "install ok installed"` | -| ownership **read** | `:1097` (`_state_get dnsmasq_preexisting`) at uninstall | -| state file **deleted** | `:1268` — *after* the read. **The order was already correct.** | -| Felhom's dnsmasq snippets removed | `:1162`, in the loop just above the ownership decision | +| **Shipped** | **hub v0.104.0** (`gitea.dooplex.hu/admin/felhom-hub:0.104.0`, 25 MB) | +| **Unchanged** | controller **0.214.0**, agent **0.129.0**, installer **1.28.0**, golden 0.214.0, floor 0.214.0, MinAgent 0.129.0 | +| **Commits** | `4d6ec7c` the work · `7c97c94` the manifest bump | +| **Manifest** | `manifests/hub.yaml:128` → `felhom-hub:0.104.0`, GitOps only — **no `kubectl set image`** | +| **Deploy** | ArgoCD hard-refresh + sync → `Synced / Healthy`; pod `hub-5c4d8d4b49-864dc` **1/1 Running**; deployment image confirmed `0.104.0` | +| **CI, confirmed by run ID** | run **334** (`7c97c949f6`) **success** · run **333** (`4d6ec7c7bb`) **success** | +| **Gates** | `repo_gates.py --fast` — all **8** OK, and again in the pre-push hook on both pushes. **No `--no-verify`.** | +| **Green gate** | `hub/`: build rc=0, vet rc=0, **test rc=0 across 18 packages**. Run separately from every commit | -**Why cycle 2 concludes "pre-existing": package presence alone.** The preflight asks dpkg one question -and nothing else — it does **not** consult the absence of a record, and it cannot, because the record -was deleted with the state file. v1.27.0 stopped the unit and left the package, so the answer stayed -`yes` and our own package became "the household's" one cycle later. +**Live validation method — stated, as the fence requires:** endpoint-level. I invoked the exact URLs +the operator UI serves (`GET /hosts/`, `GET /customers/`) against the hub's ClusterIP with the +operator Basic-auth credential, so no server logic was skipped — only browser rendering. +`claude-in-chrome` is not available on DooPlex; a click-through remains the operator's. -**What the first uninstall did and did not remove:** snippets — removed. Unit — stopped + disabled. -State file (and with it the record) — removed. **Package — left.** That last one is the whole defect. +**Files:** `hub/internal/web/hosts.go`, `hub/internal/web/templates/host_detail_body.html`, +`hub/internal/claim/engine.go`, `hub/internal/notify/templates.go`, `hub/internal/web/configs.go`, +`hub/internal/web/templates/customer_unified.html`, `scripts/wire_contract_gate.py`, plus +`hub/internal/web/hosts_guestnet_test.go` (**new**, 7 tests) and `hub/internal/claim/naming_test.go` +(**new**, 6 tests). Tests **+13**. -## 3. What v1.28.0 changes +**Register:** R-319, R-320, R-321, R-322 minted (ceiling was R-318, grepped first). R-264, R-309, R-312, +R-313, R-303 and the `PETI` row updated. `STATUS.md` and the capability map updated. -When the record says we installed it, the uninstall **removes the package** as well as stopping the -unit. Order unchanged: read the record → act → delete the state. +--- -**Two packages are recorded, not one.** `dnsmasq` ships the systemd unit; **`dnsmasq-base` ships -`/usr/sbin/dnsmasq`** (`dpkg -S`, measured on the box). Separately installable, so each is recorded at -preflight and taken back only if we added it. +## 12. What was dropped, and observations -**Guard rails:** ownership **read, never inferred**; the dependency check is an **`apt-get -s purge` -simulation** that proceeds only if the removal set is a subset of ours, else stop+disable **naming the -blocking package**; never interactive; **never fatal** — a wedged apt is recorded and restated in the -closing NOTE; and the success is **re-queried** rather than read off apt's exit code. +**Dropped — named plainly, nothing silently shortened:** -## 4. Scenarios and red-proofs +- **Part C in full**, except the self-heal verdict in §10. Reason in §10: the nine were not identifiable, + and inventing them would have been worse than the gap. -| Scenario | Result | -|---|---| -| **A** three cycles, fixed | install 3 **PASSES**, `:53` free at every uninstall | -| **B** resolver pre-dates Felhom | **untouched** — `install ok installed`, unit active, both packages recorded `yes` | -| **C** something depends on it | **not purged**; log named `household-dns-thing`; stop+disable; `:53` free; dependent survived | -| **D** no ownership record | **untouched**, reason logged, exact command named | +**Nothing else was dropped.** Part A is complete (A1–A5), Part B is complete (B0, B1, and the one reader +B2 asked for — not four). -**Red-proofs — mutation asserted applied before each run:** +### Observations — noticed, not acted on -| Mutation | Outcome | -|---|---| -| remove the purge call | cycle 2 records `yes`; **cycle 3 refuses, exit 1, in those exact words** | -| remove the ownership check | **the household's resolver is PURGED** (`unknown ok not-installed`) | -| infer ownership when no record | **the guess is taken** — a field box loses its own DNS | - -**One honest note on red-proof B.** Its first run appeared to pass *for the wrong reason*: apt failed -with `dpkg was interrupted` (a broken state my own earlier manual package juggling left), so the -resolver survived by accident rather than by the guard. I repaired dpkg and re-ran, and it then failed -as required. Worth stating twice over: the mutation looked like a passing guard, and **my own fix's -apt-failure path was incidentally observed doing exactly what it should** — logging, continuing, naming -the command. - -## 5. The machines already in the field - -**Measured, not reasoned.** A field box is one with the package installed and **no record** — scenario -D. Its uninstall leaves the resolver running with the reason logged, and its next install refuses with -the message quoted in §1. - -**Is there an honest durable marker? No, and none can be invented.** The Felhom `/etc/dnsmasq.d/ -felhom-*.conf` snippets are deleted by the uninstall's own loop *before* the ownership decision; the -state file carrying the record is deleted at `:1268`; nothing under `/etc/felhom*` survives. -`/var/log/dpkg.log` does record the install and is a **timestamp** — refused by the standing rule as a -heuristic dressed as a fact. → **R-318** - -**The preflight message, judged as a customer would:** it states the finding, keeps its two routes and -its written promise not to touch DNS on a host we do not own, and adds the one thing that gets a person -moving — *"THIS LOOKS LIKE OURS"* plus one exact command. It hedges correctly (*looks like*). Its -weakness is that it asks them *"did this host have dnsmasq before Felhom?"* — precisely the question we -can no longer answer for them. **That is a mechanism, not a rule: the command is on the screen at the -moment it is needed.** - -## 6. The defect I nearly shipped - -The agent decides whether to install dnsmasq with `os.Stat("/usr/sbin/dnsmasq")` -(`felhom-agent/internal/lanresolver/lanresolver.go:105`) — but that path belongs to **`dnsmasq-base`**, -while the unit comes from **`dnsmasq`**. Purging only `dnsmasq` would leave the binary, so the next -install would skip the apt step and then fail to enable a unit that is gone — **a silent resolver where -today there is at least a visible refusal.** That is why ownership is recorded per package. - -It remains reachable where `dnsmasq-base` pre-dated Felhom (we correctly keep it). **Pre-existing, not -introduced here**, filed as **R-317**, and the uninstall now says so out loud rather than leaving it to -be found from a resolver that never came up. Fixing it is a one-line agent change, deliberately **not** -made here to keep this session to one repo. - -## 7. Publication - -`installer-v1.28.0` tagged; **both** `--ref`s in `manifests/webpage.yaml` moved (sidecar 327, init 372); -ArgoCD synced to `823cd29`; live deployment carries the new ref. - -``` -live URL serves: SCRIPT_VERSION="1.28.0" (was 1.27.0) -_dnsmasq_purge_owned present in the served file: 3 -``` - -**Pushing publishes nothing here** — `/scripts/` follows the tag. The runbook still says otherwise; -**R-309 remains open and is not forgotten**, deliberately out of scope. - -## 8. Teardown — four layers - -| Layer | State | -|---|---| -| **machine** | `drill-r50`: 0 guests, no `/var/lib/felhom-install`, dnsmasq `not-installed` at the end | -| **host** | `drill-r50` **reverted to snapshot `virgin`, powered off**; `qemu.pid` removed. DooPlex was drill host only | -| **hub** | `drill-r50-0a4f9a` **RETAINED** — pre-existing since 2026-07-25, re-used not duplicated. **No new host or customer row was created this session.** Nothing deleted | -| **off-site** | **Not contacted at all** — no restic, sftp or endpoint call was made. `demo-felhom`: `last_status=ok`, `escrow=escrowed`, **no abandon countdown** | - -Both demo boxes untouched and healthy: controller **0.214.0**, agent **0.129.0**, `Up … (healthy)`. - -## 9. Evidence, and a repeated miss - -Logs at `documentation/audits/evidence-r316-2026-08-13/`: the fixed cycles (`F1`, `F2`, `F3`), all four -scenarios (`B`, `C`, `D`), and every red-proof (`RPA*`, `RPB*`, `RPD*`). - -**The Part 1 logs did not survive.** They were on the drill VM's disk and were destroyed by the revert -to `virgin` between Part 1 and Part 2 — **the same mistake as Tuesday, in the same place.** The §1 -quotation is verbatim from the live run as read at the time, and `RPA1/RPA2/RPA3.log` are an -independent reproduction of the identical three-cycle failure, retained. Recorded rather than glossed; -the fix is procedural and I have now got it wrong twice. - -## 10. Observations — noticed, not acted on - -- **`dnsmasq-base` is flagged "automatically installed and no longer required"** after `dnsmasq` goes. - We deliberately do not `autoremove` — that would be a blast radius nobody asked for. -- **`apt-get -s purge` exits 0 even when it prints `E: dpkg was interrupted`.** The simulation output is - the signal, never the exit code — which is why the guard parses `Remv`/`Purg` lines rather than - trusting `$?`. Another entry for the exit-codes-that-lie class. -- The drill VM's `pveam` index is stale on `virgin` and needs `pveam update` before listing templates — - already recorded yesterday, hit again today in passing. +- **A "0 bytes" answer was nearly a wrong verdict, twice in one hour.** `demo-hp`'s host page came back + empty from a `kubectl run` curl pod, which reads exactly like "the card does not render". Retrying with + a clean pod name returned **HTTP 200** and a correct card. *An empty listing is not evidence of + emptiness* — the instrument was the fault, and the rule caught it. +- **R-264's count was wrong**: the row says twenty-one facts, the allowlist held twenty. The row is now + written against the measured number. +- **The wire gate does not detect a stale allowlist entry.** Its sibling `retrieval_promise_gate.py` + fails on one; this one silently skips. Had I re-labelled the eight `guest_net` entries instead of + deleting them, the gate would have reported a coverage it did not have and said nothing. Worth the + same treatment. +- **A third secret sits one homograph away from the two that collided.** The hub's self-bind flow calls + the **five-word** retrieval passphrase „**visszaállító jelszó**" (`web/selfbind.go:255,261`; + `notify/templates.go:285`) — *jelszó*, not *kód*, so it is outside R-295's ruling and I left it alone. + But „Visszaállító kód" was just retired for being a near-homograph of „Helyreállítási kód", and this + is a near-homograph of the name that was retired. **Three secrets, three names, and two of them still + begin with the same word.** A ruling for it is the operator's, not mine. +- **`agents.md`-style version drift in project memory**: `instructions_gate` emits 32 version-literal + warnings against `MEMORY.md`. Not a failure and not mine to fix tonight, but it is aimed at the next + model to edit that file — which is worth someone acting on before it is noise nobody reads.