Files
felhom.eu/documentation/audits/REPORT-small-debts-and-first-reader-2026-08-13.md
T
admin e0b56c976f
gates / gates (push) Successful in 15s
REPORT + CONTEXT: the third name, the second door, and a number that answered a different question
Three rules carried forward. A name must separate on the STEM, not the noun — naming
this secret after the act it is used in would have recreated the trap, because the
other factor on the same page is the „Párosító kód". A guard is worth what its positive
control is worth: this one's selftest convicted its own step-3 case and found a defect
in the guard itself. And a suppression must rest on the machine's own declaration, then
be checked for the SECOND door — recording the disabled state rather than deleting it
is what let the deadline check skip it too.

Yesterday's report is preserved to audits/ because it carries the only record of the
self-heal verdict (Part C was dropped, so that reasoning is in no register row) — the
rule written last night, applied to itself the first time it mattered.
2026-08-13 16:01:45 +02:00

27 KiB
Raw Blame History

REPORT — the small debts paid, and one unread fact given a reader (2026-08-13, evening)

Shipped: hub v0.104.0, live and verified. Parts A and B complete; Part C dropped, named in §12. No controller change, no agent change, no wire change — so there was no bake and no approval, as the task expected.


1. A4 FIRST — what the fleet actually is, and the entry corrected

The task's premise for A4 was wrong, and the evidence is not ambiguous. It asked me to establish the facts and not accept the framing; doing that reverses the conclusion. Two different records have been called "the tester", and only one of them carries the risk.

Read from the hub's own store (hub.db + -wal + -shm, copied together — the WAL rule):

peti-felhom davidtester-1
customer row created 2026-02-27 david created 2026-08-01, deleted today 07:55:49; tester-1 created 07:56:47
host row peti-felhom-86d37d, deleted 2026-07-15 08:56:22 (host_deletions id 1) never, either name
controller reports 482, 2026-02-27 → 2026-07-15 08:39:00 0
host reports present until deletion 0
escrow rows none none
events staleness alarms after it went silent 4 total — three hub-side expected_dbdump_missed false alarms (R-195's subject) and its own deletion

So:

  • "The tester" in this project's own vocabulary is Petidocumentation/pilot/PETI-tester-agreement.md says "Operator: Viktor. Tester: Peti" — and the machine is a real 80-core Proxmox server belonging to a real person, running Felhom as a BYO guest. It reported for four and a half months. target-selection.md puts it at Tier 2 — protected, "because there is a real person behind it". The 3.6 GB with no key, no off-site copy and no local backup is REAL, and the entry keeps its rank.
  • tester-1 is a record with no machine — no host, no escrow, no report, ever; and david before it was the same. A record with no machine behind it can lose nothing.

Reconciling this with what the operator said ("there is no actual tester yet — only a pre-created customer, now renamed"): both statements are true of different things. The pilot programme never began — the agreement was drafted 2026-07-09, the onboarding runbook stopped at P1 — and the pre-created record is tester-1. Meanwhile the hardware and the data have existed the whole time. Nothing about the risk changed; only the word that names it.

The corrected entry, as shipped (STATUS.md, "Broken, or knowingly incomplete"):

  • Peti's machine has no recovery route at all — see the PETI row. This is a real machine belonging to a real person, not one of ours and not a record: it reported to the hub for four and a half months and has been silent since 15 July, when its host record was deleted. There is no key, no off-site copy and no local backup. If that drive fails, everything on it is lost. First act of the visit: copy the ~3.6 GB off before anything is reinstalled — it is currently the only copy in existence. Whether it stays parked is your call and is deliberately left open.

The PETI register row carries the full disambiguation with every count measured, and ends "Wherever a document says 'the tester's machine', read peti-felhom."

What the fleet actually is, in one sentence, now on STATUS.md: the hub holds five customer records and three machinesdemo-felhom and demo-hp (ours, disposable), drill-r50 (a nested drill VM, reverted and powered off); peti-felhom is a real machine we have not heard from since 15 July and has no host record; tester-1 is a record with no machine.


2. The hub strings as shipped, with bytes confirmed

Enumerated at file:line before editing. All written from explicit bytes and verified as hex — no non-ASCII crossed a shell chain; the mojibake check (Ã/Å/â€) is clean in every touched file.

Where Was Is Hex of the shipped name
notify/templates.go:216 subject „Jelszó-visszaállítási kód" „Beállító kód a jelszavad visszaállításához" 4265c3a16c6cc3ad74c3b3206bc3b364…
notify/templates.go:221 body „Visszaállító kód: %s" „Beállító kód: %s" 4265c3a16c6cc3ad74c3b3206bc3b3643a202573
notify/templates.go new reenroll (did not exist) subject „Új beállító kód — újratelepült a szervered", body names „A szerver beállítása" 224120737a6572766572206265c3a16c6cc3ad74c3a1736122
notify/templates.go:72 lockout „beállító/visszaállító kód próbálkozás" „beállító kód próbálkozás" …6265c3a16c6cc3ad74c3b3206bc3b364…
web/templates/customer_unified.html:476 „Visszaállító kód küldése" „Beállító kód küldése" 4265c3a16c6cc3ad74c3b3206bc3b364206bc3bc6c64c3a97365

The page-naming fix, established at file:line rather than assumed. ReissueForReenroll sent the reset mail (claim/engine.go:181), which directs the customer to an „Elfelejtett jelszó" page. A rebuilt box has no password, so the controller computes reset := s.authEnabled() → false (felhom-controller/controller/internal/web/claim.go:279), renders „A szerver beállítása", and serves no login page at all — the named route is not on their screen. Only the hub can tell the two situations apart, because the hub chose which call site fired, so the fix is a new EmailKind, not a reworded shared template. Same secret, same name, different sentence.

The re-enrol mail deliberately says nothing about apps or backups — TestFormatClaimEmail_ReenrollPromisesNothingAboutTheData pins it. A clean-slate reinstall is precisely where such a reassurance could be false, and this project has spent four register rows removing promises it could not see were still true.

Live-verified (endpoint-level, the exact URL the operator UI serves; ClusterIP + Basic auth): GET /customers/demo-felhom„Beállító kód küldése" present, „Visszaállító kód" absent.

Had the claim guard ever scanned the hub? No — never.

retrieval_promise_gate.py lives in felhom-controller/controller/scripts/; its declared surfaces are that repo's internal/web/templates plus one Go handler file, added on 2026-08-12 by R-311 on the express ground that the highest-stakes customer copy "had never been scanned". The same sentence is true one repo over — the hub composes every customer e-mail, i.e. the copy a customer reads before they see any box screen.

I scanned it by hand with the gate's own four stems (visszaállíthat, visszaszerezhet, visszahozhat, visszanyit) across every non-test .go/.html under hub/internal/: zero occurrences. So nothing was hiding, and my new strings contain no stem and need no registration. It is a scope gap, not a live defect — filed as R-322 rather than fixed, because pointing a controller gate at a sibling repo makes a controller gate fail on a felhom.eu edit, and G-1 already taught this project what a cross-repo gate costs. (A recommendation not followed gets its line: I recommend a hub-side sibling in repo_gates.py sharing ONE stem list — two copies of a word list is how the plural got past the singular in R-299.)


3. The runbook correction, and where else the claim lived

runbooks/day0-install.md §C.1 said "there is no release tag… pushing scripts/felhom-host-install.sh publishes it." It has published nothing since R-110 shipped on 2026-08-03. §C.1 now opens by naming its own former error and states the three acts that actually publish:

  1. bump SCRIPT_VERSION and push to mainpublishes nothing;
  2. cut and push the tag installer-v<new SCRIPT_VERSION>;
  3. move BOTH --ref=installer-v… pins in manifests/webpage.yaml — the git-sync sidecar and the init container (lines 327 and 372) — commit, sync. The one-pin trap is named: the running pod keeps serving until it restarts, and a fresh pod seeded by a stale init container then serves the OLD script with no error anywhere.

How to verify from outside, since a push, a green sync and a correct-looking manifest are each consistent with nothing having been published:

curl -fsS https://felhom.eu/scripts/felhom-host-install.sh | grep -m1 '^SCRIPT_VERSION'

Measured while writing it: served 1.28.0, main 1.28.0, both pins installer-v1.28.0 — the three agreeing is the observation; any one alone is not.

Copied elsewhere? Yes, once, and it was hunted. audits/SPIKE-universal-iso-3-2026-07-31.md:184 says the same thing and cites day0-install.md as its source — which is how it spread. It was true on the day it was written, so the dated finding is kept verbatim and carries a SUPERSEDED note; falsifying a dated record to tidy it is its own defect. Two other hits are correct in context: hostinstall_gates.py:198 states the consequence of the manifest losing its tag, and the 2026-08-12 drill record already names the sentence as false.


4. The evidence rule, as written and where it lives

Evidence is copied off the machine at the end of the phase that produced it — before any revert, snapshot restore or teardown. Not at the end of the session.

The mechanism, because a rule without one is a wish: the last act of a phase that ran on a machine is scp/pct pull of its logs into the evidence directory on DooPlex — the same act that ends the phase, not a separate step to remember later.

When a session notices the evidence is already gone: say so plainly in the report and REPRODUCE it independently. That is the documented expectation, not an improvisation invented under pressure.

Four homes, chosen so a session meets it before the revert rather than after:

  1. runbooks/workspace-CLAUDE.mdstanding rule 5, so it loads in every session, with the two-incident record in the R-96 rationale comment beneath it.
  2. runbooks/target-selection.md — its own section, in the doc you read before picking a machine to break. Tier 0 machines are disposable, which is exactly why nothing you need may be left on one.
  3. RUNBOOK-rehearsal-v3.md — in the standing-rules block, flagged as applying to every phase boundary.
  4. PROMPT-TEMPLATE.md — a new report item 8, so a session must state that it did this.

Provenance, both occurrences verified, not recalled: 2026-08-12 the retained-key drill lost its Phase A logs to the revert to virgin between Phase A and Phase B (audits/DRILL-retained-key-2026-08-12.md §11.5); 2026-08-13 R-316 lost its Part 1 logs, same box, same revert, same point (§9 — "the same mistake as Tuesday, in the same place"). Both times the existing "scp the log OUT first" was applied to the final teardown and not the intermediate one. Filed as R-320.

A side effect worth naming: that second record lived in REPORT.md, which this report overwrites — writing tonight's report would have destroyed the record of a destroyed record. It was preserved to audits/REPORT-r316-installer-v1.28.0-2026-08-13.md first, and both citations now point there.


5. The three rulings, recorded as decided

STATUS.md's "Waiting on you" section is now empty, and says so: "Nothing. All three questions that stood here were answered on 1213 August and have moved to Decided below." A new Decided section carries them, and each register row's status field changed from READY to DECIDED.

Ruling Trigger that reopens it
R-312 — recovering an old backup stays a phone call. Not built, deliberately; the operator path genuinely works (the 2026-08-12 drill restored planted files byte-identical by hand) A real need appearing — one request from a customer who is not us. Until then R-304's honest position stands: retention is operator-only, and nothing may promise the customer can do it
R-313 — the unopenable set-aside store on demo-felhom is KEPT, as a test fixture. The specific advantage the operator accepted: it is the only state in existence where a set-aside store is present and cannot be opened — a case that cannot be manufactured without destroying another key on purpose Delete it when the work it is a fixture FOR ships, or is abandoned — i.e. when R-312 is built, or when R-312's ruling is made permanent. On either event, delete deliberately and record why
R-303 — a machine in two kinds of trouble says both things: left as it is. Real-world likelihood unknown; the tidier fix risks hiding a genuine second failure An observation of the combined state occurring OUTSIDE a constructed test. One sighting on a real machine reopens it

6. B0 — do the facts arrive, and is this hub-only?

Yes, and yes. Both halves established before any code was written.

  • On the wire and in the database. demo-felhom-8363b5's newest host_reports row carries guest_net.checked_at plus per-guest vmid/state/mode/ip/has_route/dhclient_alive/checked_at/message. The agent's wire type (felhom-agent/internal/hub/report.go:139-160) additionally declares healed, heal_succeeded, last_heal_at, heals_last_hour, damped — absent from the live rows only because they are omitempty on a box that has never needed a repair.
  • Read by nothing. The string guest_net occurred nowhere in felhom.eu/hub/ — no struct, no template, no checker. Stored as raw text inside report_json.

So no wire change and no agent change were needed: hub-only. The halt condition did not fire.


7. Reclassification counts — measured, not estimated

The register row said twenty-one; the gate's allowlist held twenty. Twenty is what the dispositions account for, exactly:

State Count Which
Read (removed from the allowlist) 8 guest_net + its seven children
Deliberately not consumed (new third kind, with the ruling and its date) 5 mgmt_plane.healed_recently, pbs_dr.applied_at, config_hash, stacks, storage.migrated_to
Redundant (decided on its own merits) 1 reporting_disabled
Still owed a reader (R-264, open) 6 selfupdate_pending, selfupdate_pending_version, restore_tests.mount_parity, restore_tests.mount_inventory, backup.last_db_dump, backup.last_integrity_check

The eight are REMOVED rather than re-labelled, and that is the point: an allowlisted tag is skipped by the gate, so leaving them would have meant the new reader's own fields were never checked for reachability at all.

A thing the task told me to re-read from the register, which was not there. The operator's dispositions were made on 2026-08-12; R-264 still read READY — owner Viktor and all twenty allowlist entries still said "arguably owed". A session told to re-read them from the register would have found none. They are written down now, which is the point of writing them down.

reporting_disabled, decided on its own merits — and the decision found a real defect the flag would not have fixed. The product does support the state: the controller sends one minimal report carrying reporting_disabled: true and health.status: "disabled" (cmd/controller/main.go:1246-1260), then goes quiet by design. The hub already decodes health_status and already renders that customer as disabled (web/rollup.go:25) — so the flag is a second spelling of a fact already read: redundant. But StalenessChecker.Check (monitor/staleness.go:88+) is age-only; its sole skip is IsCustomerBlocked. A deliberately-silent box still goes node_stale at 30 minutes and node_down at 60 — exactly the false alarm B0 predicted. Filed as R-321 and deliberately not fixed here: decoding the flag would have felt like progress and left the alarm firing. The fix belongs in the checker, which already holds the status it needs.


8. The reader as built, and the alarm judgement

hub/internal/web/hosts.goparseGuestNet / guestNet / guestNetView; rendered as a Guest network card on the host-detail page, which is where a person looks at a machine.

The signal is the repair count, not the state. A guest the watchdog keeps repairing is healthy at every instant anyone looks and is nevertheless failing; rendering state alone would give it a green tick — the exact shape of the failed-disk-drawn-as-a-healthy-empty-disk defect. heals_last_hour sits beside the state and drives the summary badge.

heal_succeeded is decoded too, and that was a deliberate addition mid-build. My first cut carried the count and not the outcome — which is R-260 exactly: a hub decoder three fields short of the agent, missing the one that decides the question. Six failed repairs is a guest that is down; six successful ones is a nuisance.

The four scenarios, and what the card shows for each — all four verified by test, and three of them observed on the live fleet:

Rendered Live sighting
A healthy, no repairs healthy badge, address, route, dhclient, 0 repairs demo-felhom-8363b5 — 9201 healthy, 192.168.0.149 (dhcp), swept 08:45:33Z; demo-hp-bb76ea — 9201 healthy, 192.168.0.118, swept 08:52:35Z
B repaired repeatedly needs attention; the count rendered as a warn badge with the last repair time; a sentence naming the 1 h 15 m incident not yet on hardware — neither demo box has needed a repair since the watchdog shipped
C does not report it unknown, never healthy — three absences, three different sentences drill-r50-0a4f9a"A capable agent sent no guest-network stanza"unknown. The unknown branch fired on real data, not only in a test
D malformed unknown, and the page returns 200 covered by test; the decode error is swallowed on purpose so one box's bad field cannot break the page for the fleet

Two extra branches beyond the four: an unrecognised guest state renders unknown (the switch is an allow-list, so adding a state to the agent can never silently paint it green), and an empty guest list with a fresh sweep is NOT silence — the agent's contract says that means "the watchdog ran and found nothing", which must stay distinguishable from "the watchdog is not wired", the shape the v0.91.0 inert seam hid behind.

The alarm judgement: no email, deliberately

I agree with the task's recommendation and did not add one. The reasoning, since it is a judgement:

  • The incident was a visibility failure, not a paging failure. Nobody could see the condition for 1 h 15 m. A card fixes what actually broke.
  • A new alarm on a fleet of two demo machines is untested noise, on a dispatcher whose severity contract is exact-match lowercase and whose customer-message allow-list must move with it — a two-place change that fires against no real population.
  • There is no calibrated threshold to alarm on. Neither demo box has produced a single repair since the watchdog shipped, so any number I picked would be invented. The visible count is what will supply the threshold — which is the argument for building the card first and the alarm second.

Revisit when a third machine exists, or when a repair count is seen climbing on real hardware.


9. Red-proofs — every mutation asserted applied before its run

Five in total: three mandated for B2, two for A2. Each mutation was confirmed present by grep before the test ran, and confirmed absent by grep after restoring.

# Mutation Asserted applied Outcome
1 (the one that matters) The unknown branches in the card's badge chain replaced by the healthy badge grep -c 'title="MUTATED">healthy'2 RED — both C sub-cases. A silent machine seen rendering as healthy: "a silent box must SAY it is unknown" and "an old agent's silence must be named as an old agent's silence"
2 RepairCount: 0 in the decoder — the climbing-repairs signal dropped hosts.go:385 RepairCount: 0, … // MUTATED RED — B: "a guest repaired 6 times in an hour is reported as fine — the whole point of the card"
3 Degraded() forced true — the healthy branch broken hosts.go:348 func … { return true /* MUTATED */ } RED — A: "a machine that is fine is being alarmed on" — the guard is reachable in both directions
4 ReissueForReenroll routed back to EmailReset grep → line 195 reads EmailReset RED — two tests: the new TestReenrollSplit_ChangesTheMailNotTheSecret and the pre-existing TestReissueForReenroll
5 „Visszaállító kód: %s" restored in the reset mail body grep -ntemplates.go:241 RED — TestFormatClaimEmail_OneNamePerSecret

After every restore the suite is green again, and grep -c MUTATED returns 0 in both touched files.

A positive control that the reader is WIRED and not merely written, independent of the tests: the wire-contract gate's checked-tag count rose 182 → 190 and its skipped count fell 88 → 80 as the eight allowlist entries came out. The gate still passes — meaning those eight tags are now genuinely reachable in the hub.


10. Part C — DROPPED, and one part of it answered anyway

Dropped, as the task's own drop order specifies. The honest reason is not only time: the task names "nine claims" that are grey, and I could not identify which nine. The capability map holds 22 rows marked IMPLEMENTED, five PARTIAL and four MISSING, and nothing in the tree enumerates a set of nine. Picking nine myself and reporting on them would have produced a confident answer to a question nobody asked — the A4 failure mode, one section later.

The one I can answer, because the task identified it precisely. The self-heal claim is the capability-map row "Box survives an unattended app or guest-network failure… it is noticed, and where safe it is repaired", and its best document does not argue against it. The 1 h 15 m outage (audits/INCIDENT-guest-dhclient-killed-2026-07-20.md) is the row's origin, not its evidence: the evidence is a deliberate replay of that incident the next daykill -9 at 12:43:18, detected on process liveness 57 s later while the lease was still live, healed at 12:45:18 with the incident's verbatim invocation, and the tunnel never dropped (cloudflared Up 29 hours). That is a walk, and the outage was prevented rather than merely observed. The claim does not need to move down.

What I would flag instead: the row's guest-network leg was proven on the box and, until today, the hub could not see the condition at all — which is the gap this session closed and which no capability row expressed. My new row states its own status honestly as IMPLEMENTED, not PROVEN-LIVE, because no machine has ever been observed with a climbing repair count on this card.


11. Versions, commits, deploy, CI

Shipped hub v0.104.0 (gitea.dooplex.hu/admin/felhom-hub:0.104.0, 25 MB)
Unchanged controller 0.214.0, agent 0.129.0, installer 1.28.0, golden 0.214.0, floor 0.214.0, MinAgent 0.129.0
Commits 4d6ec7c the work · 7c97c94 the manifest bump
Manifest manifests/hub.yaml:128felhom-hub:0.104.0, GitOps only — no kubectl set image
Deploy ArgoCD hard-refresh + sync → Synced / Healthy; pod hub-5c4d8d4b49-864dc 1/1 Running; deployment image confirmed 0.104.0
CI, confirmed by run ID run 334 (7c97c949f6) success · run 333 (4d6ec7c7bb) success
Gates repo_gates.py --fast — all 8 OK, and again in the pre-push hook on both pushes. No --no-verify.
Green gate hub/: build rc=0, vet rc=0, test rc=0 across 18 packages. Run separately from every commit

Live validation method — stated, as the fence requires: endpoint-level. I invoked the exact URLs the operator UI serves (GET /hosts/<id>, GET /customers/<id>) against the hub's ClusterIP with the operator Basic-auth credential, so no server logic was skipped — only browser rendering. claude-in-chrome is not available on DooPlex; a click-through remains the operator's.

Files: hub/internal/web/hosts.go, hub/internal/web/templates/host_detail_body.html, hub/internal/claim/engine.go, hub/internal/notify/templates.go, hub/internal/web/configs.go, hub/internal/web/templates/customer_unified.html, scripts/wire_contract_gate.py, plus hub/internal/web/hosts_guestnet_test.go (new, 7 tests) and hub/internal/claim/naming_test.go (new, 6 tests). Tests +13.

Register: R-319, R-320, R-321, R-322 minted (ceiling was R-318, grepped first). R-264, R-309, R-312, R-313, R-303 and the PETI row updated. STATUS.md and the capability map updated.


12. What was dropped, and observations

Dropped — named plainly, nothing silently shortened:

  • Part C in full, except the self-heal verdict in §10. Reason in §10: the nine were not identifiable, and inventing them would have been worse than the gap.

Nothing else was dropped. Part A is complete (A1A5), Part B is complete (B0, B1, and the one reader B2 asked for — not four).

Observations — noticed, not acted on

  • A "0 bytes" answer was nearly a wrong verdict, twice in one hour. demo-hp's host page came back empty from a kubectl run curl pod, which reads exactly like "the card does not render". Retrying with a clean pod name returned HTTP 200 and a correct card. An empty listing is not evidence of emptiness — the instrument was the fault, and the rule caught it.
  • R-264's count was wrong: the row says twenty-one facts, the allowlist held twenty. The row is now written against the measured number.
  • The wire gate does not detect a stale allowlist entry. Its sibling retrieval_promise_gate.py fails on one; this one silently skips. Had I re-labelled the eight guest_net entries instead of deleting them, the gate would have reported a coverage it did not have and said nothing. Worth the same treatment.
  • A third secret sits one homograph away from the two that collided. The hub's self-bind flow calls the five-word retrieval passphrase „visszaállító jelszó" (web/selfbind.go:255,261; notify/templates.go:285) — jelszó, not kód, so it is outside R-295's ruling and I left it alone. But „Visszaállító kód" was just retired for being a near-homograph of „Helyreállítási kód", and this is a near-homograph of the name that was retired. Three secrets, three names, and two of them still begin with the same word. A ruling for it is the operator's, not mine.
  • agents.md-style version drift in project memory: instructions_gate emits 32 version-literal warnings against MEMORY.md. Not a failure and not mine to fix tonight, but it is aimed at the next model to edit that file — which is worth someone acting on before it is noise nobody reads.