Files
felhom.eu/REPORT-wire-contract-gate.md
T
admin 650cc8a4a7
gates / gates (push) Failing after 13m54s
docs: record the CI episode — three instrument defects, and the two-homes rule (R-260)
The gate's third instrument defect and the reusable lesson underneath it: a gate needing a sibling
clone passes on a workstation and is INCONCLUSIVE in CI, and a gate must not shell out to a tool the
CI image may not have. Three red runs (260-262), each of which mailed the operator, before 263 went
green. CONTEXT S-38 gains the two-homes rule.
2026-08-08 09:12:29 +02:00

12 KiB

REPORT — G-1: a gate for the dropped field, then the fields it found (2026-08-08)

A non-overwritten REPORT-<topic>.md sibling, per CLAUDE.md:82-87 — a parallel session shares this clone and the shared REPORT.md was not touched.

1. The gate's output on today's tree — failing, before anything was fixed

This is the session. Captured verbatim in documentation/tests/wire-contract-gate-2026-08-08/BEFORE.md:

wire-contract gate — 210 tag(s) checked across 3 declared wire(s); 51 skipped
WIRE-CONTRACT GATE FAILED: 40 emitted field(s) cannot be received.

It named every one, with its emit path and its direction, and re-found escrow_stale (R-247) and every field R-260 listed. Had it been green, the gate would not work and that would have been the finding — which is not hypothetical: the night before, deadcode was rejected for the neighbouring C6 class for exactly that reason.

⚠ A count this session's prompt got wrong. The prompt said "465 emitted tags, eight unreachable". R-260's wording was "at least eight decision-bearing facts", never eight tags in total. Measured: 40 on the three declared wires. Checked against the repo, not quoted — the prompt's own rule 6, and the second prompt claim caught that way this week.

2. The forty, by disposition

# field(s) direction decision what changed
1 oob.operator_key_configured agent → hub receive and act decoded (pointer); oobDegraded fails on a missing key and the alert names it
2 oob.wg_handshake_age_s, oob.healed_at agent → hub receive, message only in HostOOBRow + the event payload; deliberately NOT in the predicate
3 escrow_stale hub → controller receive and act report.EscrowStatus.Stale; withheld-hash told apart from hash-less. R-247
4 12 host/system metric fields both → hub no consumer wanted — redundant allowlisted: the hub bands on the *_percent figures from the same stanzas
5 guests.spec.{disk_bytes,memory_bytes} agent → hub redundant sizing is hub-owned intent, not mirrored reality
6 storage_targets.smart.model_name agent → hub redundant a display label; smart.health + every banded counter ARE decoded
7 wireguard.last_handshake_age_s agent → hub redundant wgsync reconciles from its own state
8 21 fields (guest_net+7, selfupdate_pending+1, healed_recently, applied_at, mount_parity/_inventory, config_hash, reporting_disabled, stacks, migrated_to, last_db_dump, last_integrity_check) both → hub no consumer today, one arguably owed allowlisted against R-264, OPEN. Allowlisting is not deciding, and the entries say so

Full per-field reasons are in the gate's own ALLOWLIST, each a claim someone can re-check.

3. Scenario F — the choice, and why

Unknown is reported distinctly and is never ok. operator_key_configured decodes as a pointer: nil = the agent never said, which is not a value.

The version gate the prompt thought "probably right" was rejected on a measurement: the field and the oob stanza that carries it shipped in the same agent version (v0.72.0, 2026-07-05), so a stanza without the field cannot come from any released agent. The live fleet is 0.113.0 and 0.127.0; the vouched floor is 0.127.0. Building version-gating machinery the hub does not otherwise have, for a state no box can be in, is cost without cover. The case is still handled explicitly and pinned by a test, because "cannot happen" is a claim this project has been burned by.

4. R-247 — CLOSED

The field is received, and reconcileEscrowed tells a withheld hash from a hash-less one. Controller v0.209.0.

Deliberately not folded in, and said rather than skipped: the wrong flag on demo-hp is an operator act hub-side (R-246, still open), and the customer-facing Hungarian card copy is unchanged — that is UI work with its own review path.

5. The gate's blind spots, and its self-test

Published in the module docstring and in the gate's own output, because Campaign 12's C1 guard turned out blind to one of the three shapes it was written for:

  • generic tag names are not checked (name, state, status, …) — a repo-wide string test says nothing about them, so a drop of a generically-named field is missed; the gate under-reports rather than over-reports;
  • reachability of a NAME is not use of a VALUE;
  • only declared ROOTS are covered — the hub's desired-state (raw stored JSON, no typed emitter) and the agent's local API (no single root) are not;
  • it reads source, not traffic; test files and testdata/ are excluded on the receiving side deliberately (a tag present only in a fixture is not decodable — which is R-262 exactly).

--selftest plants an unreachable tag on a real root in a throwaway copy and asserts conviction: exit 1, planted tag named; unplanted tree exit 0.

THREE instrument defects this gate's own controls caught before it was trusted. None was found by review; each was found by making the gate prove something.

  1. A substring false negativegrep -F healed_at also matched privsep_healed_at. R-260 named healed_at, so its absence from the output was the tell. Whole-token now; 40, not 39.
  2. dr_recipe is not wholly opaque — its top-level section keys ARE decoded, through allow-lists that already swallowed offsite_restic for months (R-122). Now opaque only below depth 1.
  3. The search shelled out to grep and read its failure as a finding. CI convicted all 174 checked tags while the pre-push hook was green. The CI runner's image carries python3 and git and deliberately little else, and its grep does not support --include, so stdout was empty and empty was read as "absent". A gate that silently turns a tool failure into a finding is worse than no gate, and its green would have been as untrustworthy as its red. Removed the dependency rather than working around it: the search is pure Python now, one token index per receiving repo.

The BEFORE capture was RE-VERIFIED, not re-generated — the stronger claim. All 40 recorded fields were re-tested against the new implementation: agree=40, disagree=0, i.e. exactly the four this session fixed are now present and the other 36 still absent. The number stands under both implementations.

And the reusable half, which is about the gates and not about this gate. The pre-push hook runs on a workstation where every sibling repo is a real clone; CI checks out one repo, shallow. A gate that needs a sibling passes locally and is INCONCLUSIVE in CI — the two automated homes are not interchangeable, and a new gate must be checked in BOTH. The workflow's own alarm mail says a hook-versus-CI disagreement "outranks whatever the push was for"; it did. Fixed by fetching the agent clone in CI (.gitea/workflows/gates.yml), never by letting the gate skip when a sibling is absent — that is the fail-open shape and would leave it running in neither home (R-29).

Cost, stated plainly: three CI runs went red (260, 261, 262) and each sent the operator an alarm mail before run 263 went green. The alarm working is the system behaving correctly; the noise was mine.

6. What oobDegraded says when it fails

Host <id>: OPERATOR ACCESS DEGRADED — the operator's authorized_key is NOT installed —
felhom-sshd is up and answering, and nobody can log in through it. The break-glass net
(auto-heal + vaulted root@pam console) is still under the box.

and for the unreachable-but-handled unknown:

… — the agent reports operator access but is too old to say whether the operator key is
installed (pre-v0.72.0) — treat entry as UNPROVEN, not working. …

oobDegradedReason is now the single source for both the predicate and the text, so the message can never name a different fault from the one that fired. The old form derived it separately and had a vocabulary of two.

7. Tests and red-proofs

New: hub/internal/store/host_oob_decode_test.go (4 tests, raw JSON at the decode boundary), hub/internal/monitor/host_oob_operatorkey_test.go (6), plus two end-to-end tests in host_oob_test.go driving JSON → store → checker → event.

Red-proofs — 8 expected outcomes, 0 wrong, each with the mutation asserted applied:

mutation assertion it applied outcome
the gate on today's tree RED, naming all 40
planted unreachable tag (post-fix) self-test reports the planted tag by name RED on the plant, GREEN unplanted
drop operator_key_configured from the decoder json-tag occurrences in the decoder 2 → 1 RED — the false ok returns
make the check unconditional MUTATED unconditional degrade marker present RED — a healthy box alerts
treat unknown as ok MUTATED: unknown is silently ok again marker present RED — the silent pass returns
all three restored GREEN

The pre-existing fixture was part of the defect and was fixed too: oobReport() omitted operator_key_configured, so every earlier scenario ran against a report shape no released agent produces. Same family as R-262.

8. The capability-map row about operator access

Checked, and it was NOT claiming something untrue. 00-capability-map.md:127 claims OOB operator access is implemented, never that it is monitored, so no correction was owed. What was untrue sat one layer down — the hub's own health check could not see the key — and the row now records that, with the fix and the tests that pin it.

9. Gates, and what remains

python3 scripts/repo_gates.py --fastall 8 OK, including the new wire-contract and golden-currency. go build ./... && go vet ./... && go test ./... green in hub and controller (run separately from every commit). No --no-verify anywhere.

The one gate failure that remains is not a failure of this work: golden 0.208.0 is baked and byte-verified but still not vouched, so fresh installs receive 0.207.0. That is R-242's untouched half and one operator Save.

10. Register

R-260 CLOSED (class gated + sharpest instance fixed), R-247 CLOSED, G-1 CLOSED in ROADMAP.md. R-264 minted and OPEN — the twenty-one facts with no consumer, split out so that gating the class could not be mistaken for deciding them. Highest ID moved R-263 → R-264.

Explicitly still open: R-246, R-255, R-256, R-257, R-258, R-259, R-261, R-262, R-263, and C7's test-comment half, which Campaign 12 recorded as owed, not done.

11. Observations — noticed, NOT acted on

  1. stacks is the whole per-stack report object and the hub decodes none of it. The largest single unconsumed structure on the controller wire; folded into R-264 rather than sized here.
  2. The hub has no version-gating machinery for report fields at all. Not needed today (see §3), but the next additive field whose emitter and stanza do not ship together will need it, and there is no convention to reach for.
  3. backup.last_db_dump / last_integrity_check are backup-integrity timestamps the hub cannot see — the "presence is not success" neighbourhood, and worth ranking first inside R-264 after guest_net.
  4. The gate cannot cover the hub's desired-state wire because it is served as raw stored JSON. That is the one remaining hub→box direction with no contract check of any kind.