admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
felhom.eu/scripts/wire_contract_gate.py
T
admin 900e6d86e2 R-315: wire-contract gate checks mirrored roots field-by-field and prints each root's check kind 
The escrow/retained and escrow-ACK wires are now compared path-by-path against the receiver's named
mirror type, so the measured mutation (agent renames superseded_at, the old name still a local-API map
key) convicts. Decoy pair in test_gate_decoys.py, seen failing with the mirror removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 21:45:04 +02:00

730 lines
39 KiB
Python
Raw Permalink Blame History

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""wire_contract_gate.py — a field one side emits and the other cannot receive is a DEFECT (G-1).
Run from the repo root: python3 scripts/wire_contract_gate.py
Exit 0 clean · 1 convicted (an emitted field has no receiver) · 2 inconclusive (a root or a sibling
clone could not be resolved — never a silent pass).
Self-test: python3 scripts/wire_contract_gate.py --selftest
Plants an unreachable tag on a real root and asserts this gate convicts it, then asserts a clean
run is clean. A gate nobody has watched fail has not been shown to work.
WHY THIS EXISTS.
Campaign 12 (2026-08-08, `documentation/audits/CAMPAIGN-12-class-sweep-2026-08-08.md`) swept seven
recurring defect classes. This one — "a field one side sends and the other silently drops" — had
already been found by hand three times, most recently R-247: the hub sends `escrow_stale` in the
report ACK, the controller's `report.EscrowStatus` has no field for it, `encoding/json` discards it,
and the box then told a customer something false about their own recovery package.
The sweep found more of the same, and Part 4 ranked a gate for this class **first of eight
candidates** — cheapest, definitive, and `--fast`-eligible. The sharpest instance it found:
the agent emits `operator_key_configured` on every heartbeat (agent hub/report.go, commented
"operator authorized_key installed"); the hub's OOB decoder had no field for it; so
`oobDegraded` — the check that answers "can the operator get into this box right now" —
returned ok for a box with NO OPERATOR KEY INSTALLED.
The agent knew and said so. The hub threw it away on arrival.
THE TEST, AND WHY IT IS DEFINITIVE.
For every json tag reachable from a declared wire ROOT, ask whether that literal tag string occurs
ANYWHERE in the receiving repo's production Go or templates. **A tag that occurs nowhere cannot be
decoded by any struct, named or anonymous.** That last clause is the whole reason this test is used
instead of comparing struct to struct: the campaign's first attempt paired types by shape and
false-positived badly, because the hub decodes one report through SEVERAL ad-hoc anonymous structs.
A string test cannot be fooled that way.
⚠ WHAT THIS GATE DOES NOT SEE — stated here so nobody reads a green as coverage. Campaign 12's own
C1 guard turned out blind to one of the three shapes it was written for; the lesson taken is that a
gate must publish its holes.
1. GENERIC TAG NAMES ARE SKIPPED. `name`, `type`, `state`, `status` and the like occur in every
repo for unrelated reasons, so the string test cannot say anything about them. They are listed
in GENERIC below and are NOT checked — a genuine drop of a generically-named field is MISSED.
This makes the gate conservative: it under-reports and does not over-report.
2. IT PROVES REACHABILITY OF A NAME, NOT THAT ANYTHING ACTS ON THE VALUE. A tag mentioned once in
a struct nobody consults passes. It answers "can this be decoded at all", not "is it used".
R-315: a root listed in MIRRORS gets a FIELD-BY-FIELD check against the receiver's named mirror
type instead (a renamed receiver tag convicts even when the old name occurs elsewhere); every
run prints which check each root got. The other roots are still name-checked only.
3. ROOTS ARE DECLARED, NOT DISCOVERED. Only the wires in ROOTS are covered. The hub's
desired-state is served as raw stored JSON (`host.DesiredJSON`) with no typed emitter to walk,
and the agent's local API has no single root type — NEITHER IS COVERED.
4. IT READS SOURCE, NOT TRAFFIC. A field populated only at runtime under a key built by string
concatenation is invisible.
5. TEST FILES AND testdata/ ARE EXCLUDED ON THE RECEIVING SIDE, deliberately. A tag present only
in a fixture is not decodable by production — and R-262 is exactly that: the hub's
host-report golden carries `cpu_temp_c` and `loadavg`, which no hub struct reads.
"""
import os
import re
import shutil
import sys
import tempfile
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
WORKSPACE = os.path.dirname(ROOT)
REPOS = {
"hub": os.path.join(ROOT, "hub"),
"controller": os.path.join(WORKSPACE, "felhom-controller", "controller"),
"agent": os.path.join(WORKSPACE, "felhom-agent"),
}
# (label, emitter repo, package dir relative to that repo, root type, receiver repo)
# Each row is a CLAIM that this type's json is put on this wire. An unresolvable root is
# INCONCLUSIVE, never a pass.
ROOTS = [
("agent -> hub (POST /host-report)",
"agent", "internal/hub", "HostReport", "hub"),
("controller -> hub (POST /report)",
"controller", "internal/report", "Report", "hub"),
("hub -> controller (report ACK, `escrow` object)",
"hub", "internal/store", "EscrowStatus", "controller"),
# R-311. Declared the moment the wire was created, because the gate covers only what is DECLARED
# and a silent pass is indistinguishable from coverage. The hub response is a named type rather
# than a map[string]any precisely so this root can resolve — an untyped map is a cross-repo
# contract nothing can check.
("hub -> agent (GET /hosts/<id>/escrow/retained)",
"hub", "internal/api", "RetainedEscrowResponse", "agent"),
]
# R-315: a root whose receiver decodes it into ONE NAMED MIRROR TYPE gets the stronger check —
# field-by-field: every emitted dotted path must be a json path of the mirror type (generic names
# included, since a path comparison is exact). The name-reachability check below could not see a
# renamed receiver tag when the old name occurred anywhere else in the receiving repo (measured: the
# agent's `superseded_at` renamed still passed, because the local API used the same string as a map
# key). Roots NOT listed here keep the weaker name check, and the run prints which check each root
# got — a green on a name-checked root is not decodability. Keyed by root label:
# (receiver package dir, receiver type).
MIRRORS = {
"hub -> agent (GET /hosts/<id>/escrow/retained)": ("internal/hub", "RetainedEscrowResponse"),
"hub -> controller (report ACK, `escrow` object)": ("internal/report", "EscrowStatus"),
}
# Tag names whose literal string carries no information in a repo-wide search. NOT CHECKED.
# Listed rather than silently skipped: each one is a hole.
GENERIC = {
"name", "type", "state", "status", "error", "id", "path", "size_bytes", "enabled",
"message", "value", "version", "code", "label", "mode", "total_bytes", "used_bytes",
"avail_bytes", "reason", "detail", "where", "vmid", "target", "content", "apps", "app",
"phase", "warn", "count", "host", "guests", "kind", "url", "port", "user", "role",
"created_at", "updated_at", "started_at", "timestamp", "time", "percent", "used_fraction",
}
# A tag that is emitted and NOT consumed, with the reason it is acceptable. Every entry is a claim
# someone must be able to re-check later, so none of them is bare. A quiet exclusion would be a
# dropped field with paperwork, which is worse than the defect.
#
# THREE KINDS OF ENTRY, and the differences are deliberate:
# * "redundant" — the hub already decodes something that answers the same question. No consumer
# is wanted; the entry is the end of the matter.
# * "R-264" — a fact with no consumer that ARGUABLY should have one. The entry does NOT
# close the question; it records it against an OPEN register row so that
# allowlisting cannot be mistaken for deciding. R-260 is closed by the gate plus
# the operator-access fix; the leftover appetite is R-264.
# * "not consumed, deliberately" — RULED by the operator, with the date. The question IS closed:
# a reader was considered and declined on stated grounds. This kind was added on
# 2026-08-13 because "arguably owed" had been carried for five days as though it
# were a decision, and an undecided fact and a decided one must not read alike.
# THE EMITTER IS DELIBERATELY LEFT ALONE: removing it is a coordinated two-repo
# change and it also breaks the byte-identical cross-repo host-report golden. The
# honest end here is a recorded decision, not a deletion.
#
# Keyed by (wire label, dotted emit path).
_AH = "agent -> hub (POST /host-report)"
_CH = "controller -> hub (POST /report)"
_REDUNDANT_HOST_METRICS = (
"redundant: the hub decodes host.cpu_percent / memory_percent / disk_percent from the same "
"stanza and every host-health threshold is expressed on those. The absolute figure answers no "
"question the hub asks.")
_R264 = (
"no consumer today, and one is arguably owed — recorded against R-264 (OPEN) rather than "
"decided here. Allowlisting is not deciding.")
# RULED 2026-08-13 by the operator against R-264: these five were considered for a reader and
# declined. The reason each is declined is per-entry below — a bare "not wanted" would be the quiet
# exclusion this gate exists to prevent.
_NOREADER = (
"not consumed, DELIBERATELY — ruled 2026-08-13 (operator, against R-264). A reader was "
"considered and declined; the question is closed, not open. The emitter stays (removing it is a "
"two-repo change and breaks the host-report golden). ")
ALLOWLIST = {
# ---- redundant: the hub already decodes an equivalent ----
(_AH, "host.cpu_temp_c"): _REDUNDANT_HOST_METRICS,
(_AH, "host.loadavg"): _REDUNDANT_HOST_METRICS,
(_AH, "host.memory_total_bytes"): _REDUNDANT_HOST_METRICS,
(_AH, "host.memory_used_bytes"): _REDUNDANT_HOST_METRICS,
(_AH, "host.uptime_seconds"): _REDUNDANT_HOST_METRICS,
(_CH, "system.load_avg_1"): _REDUNDANT_HOST_METRICS,
(_CH, "system.load_avg_5"): _REDUNDANT_HOST_METRICS,
(_CH, "system.load_avg_15"): _REDUNDANT_HOST_METRICS,
(_CH, "system.memory_total_mb"): _REDUNDANT_HOST_METRICS,
(_CH, "system.memory_used_mb"): _REDUNDANT_HOST_METRICS,
(_CH, "system.temperature_celsius"): _REDUNDANT_HOST_METRICS,
(_CH, "system.uptime_seconds"): _REDUNDANT_HOST_METRICS,
(_AH, "guests.spec.disk_bytes"): (
"redundant: guest SIZING is hub-owned intent (the manifest), not box reality. The hub "
"decodes vmid/name/status/controller_version from the guest list and nothing more."),
(_AH, "guests.spec.memory_bytes"): (
"redundant: as guests.spec.disk_bytes — sizing is hub-owned intent, not mirrored reality."),
(_AH, "storage_targets.smart.model_name"): (
"redundant for a verdict: the hub decodes smart.health plus every counter it bands on. The "
"model string is a display label with no threshold attached to it."),
(_CH, "offsite.last_integrity_ok"): (
"R-359, controller v0.227.0: the off-site integrity VERDICT is published so a hub surface can "
"read it, and no hub surface reads it yet. That is deliberate and it is recorded rather than "
"quietly skipped. Building the display is a HUB change, and R-331 (2026-08-30) ruled exactly "
"that class a decision for the operator, not a thing to fold into a controller task: the "
"previous integrity display was REMOVED because it rendered `Integrity Unknown` for every "
"customer forever from fields nothing wrote. Publishing the value first, and consuming it when "
"someone decides what the screen should say, is the opposite order to the one that produced "
"that card. The sibling `offsite.last_integrity_check` is NOT allowlisted and passes on its "
"own — the string already occurs hub-side. **When a hub surface is built, delete this entry.**"),
# ---- controller v0.271.0 (`09` §6.4 part 7): the automatic update leg's nightly summary ----
(_CH, "update_leg"): (
"controller v0.271.0: the operator's ONE summary of the night's automatic update leg (done / undone / "
"held / failed / skipped with reasons) rides the report so it is in the hub's stored raw report; no hub "
"surface reads it yet — a fleet view is §6.3 slice 7 (decision 18), built when the fleet grows. The "
"household's events for an undone or held step are the existing app_update_undone / app_update_held. "
"**When a hub surface is built, delete this entry.**"),
(_CH, "update_leg.ended_at"): "controller v0.271.0: a field of update_leg above — same reason.",
# ---- R-87, controller v0.231.0: the nightly off-site PROOF's verdict ----
#
# Five fields, ONE reason, and it is the SAME reason as the two integrity siblings below: the
# verdict is published so a hub surface can read it, and no hub surface reads it yet. R-331 ruled
# that class a decision for the operator rather than something to fold into a controller task —
# the previous integrity card was REMOVED because it rendered a verdict for every customer
# forever from fields nothing wrote. Publishing first and consuming when someone decides what the
# screen should say is the opposite order to the one that produced that card.
#
# They are listed INDIVIDUALLY rather than as a prefix, so deleting them one at a time as a
# surface starts reading them is a reviewable diff. **Delete each entry when its field is read.**
# Filed with the integrity pair under R-402.
(_CH, "offsite.last_proof_result"): (
"R-87, controller v0.231.0: the nightly off-site content-proof VERDICT (pass / fail / "
"cannot_judge). Published for a hub surface that does not exist yet — same class and same "
"ruling as offsite.last_integrity_ok below (R-331, R-402). ABSENT means NOT RECORDED (a "
"controller older than v0.231.0), never 'failed', so any future reader must degrade to "
"unknown. **Delete this entry when a hub surface reads it.**"),
(_CH, "offsite.last_proof_reason"): (
"R-87: the machine reason code behind the verdict above (e.g. database_expected_none_captured). "
"Unconsumed for the identical reason. It matters because 'fail' alone cannot distinguish a "
"backup that is readable-and-empty from one whose manifest could not be read, and those are "
"different operator actions. **Delete with its siblings.**"),
(_CH, "offsite.last_proof_stack"): (
"R-87: WHICH app the newest verdict is about. One app is proved per night, so a verdict "
"without its app names nothing. Unconsumed for the same reason. **Delete with its siblings.**"),
(_CH, "offsite.last_proof_snapshot"): (
"R-87: WHICH SNAPSHOT was proved — the per-archive due-ness model (R-86, 07 §3) records the "
"snapshot and never a timestamp, and a hub surface showing the verdict without it cannot say "
"whether the newest backup was the one proved. Unconsumed for the same reason. **Delete with "
"its siblings.**"),
(_CH, "offsite.last_proof_run"): (
"R-87: when that verdict was reached. Unconsumed for the same reason. NOTE for whoever builds "
"the surface: this is an attempt-free stamp — it is written ONLY when a verdict was actually "
"reached, never on a skip, a missing snapshot or a restore error, so 'presence is not success' "
"does not bite here. **Delete with its siblings.**"),
(_CH, "offsite.last_integrity_depth"): (
"R-399, controller v0.228.0: the DEPTH the verdict was reached at, published beside "
"last_integrity_ok above and unconsumed for the identical reason. It matters because 'checked, "
"OK' means two different things at structure depth and at 100%, so a hub surface that shows the "
"verdict without the depth shows the same words for a check that re-read every byte and one "
"that only read the index. Absent = the box cannot answer (a controller older than v0.228.0), "
"never 'structure'. **Delete this entry with its sibling when a hub surface is built** — filed "
"as R-402 in OPEN-ITEMS.md."),
(_AH, "wireguard.last_handshake_age_s"): (
"redundant: hub-side wgsync reconciles peers from its own state, and the OOB path's own "
"wg_handshake_age_s IS now decoded (into HostOOBRow, for the alert text)."),
# ---- R-319: guest_net AND ALL SEVEN CHILDREN ARE GONE FROM THIS LIST ----
# They are no longer allowlisted because they are no longer unconsumed: the hub models the R-54
# watchdog stanza (web/hosts.go parseGuestNet → the host-detail Guest network card), including
# heals_last_hour and heal_succeeded. Removing the entries is the POINT — an allowlisted tag is
# SKIPPED by this gate, so leaving them here would mean the new reader's fields were never
# actually checked for reachability, and the gate would report a coverage it did not have.
# The first of R-264's readers; three groups remain owed one.
# ---- no consumer, and one is arguably owed: R-264, STILL OPEN ----
(_AH, "selfupdate_pending"): _R264 + (
" NOTE: the agent's own comment beside this field claimed 'the hub reads an absent field as "
"pending=false, the correct default'. The hub had no field at all, so it read nothing "
"either way. The comment was corrected in the same change as this entry."),
(_AH, "selfupdate_pending_version"): _R264,
(_AH, "mgmt_plane.healed_recently"): _NOREADER + (
"The recurring-clobber signal is NOT lost: the hub already alarms on "
"mgmt_plane.privsep_healed_at, the timestamp sitting beside this boolean. A second reader "
"for the flag would add a second way to say the same thing and no new fact."),
(_AH, "pbs_dr.applied_at"): _NOREADER + (
"A tier-applied TIMESTAMP, and the hub already decodes pbs_dr.state — which is the verdict. "
"Presence is not success: consuming the timestamp without the state is precisely the "
"attempt-read-as-result trap, and with the state it answers nothing further."),
(_AH, "restore_tests.mount_parity"): _R264 + (
" R-262: the hub's own comment claims this contract is mirrored field-for-field and that a "
"key-set test guards drift; it is two fields short and the fixture omits the same two."),
(_AH, "restore_tests.mount_inventory"): _R264 + " R-262, as mount_parity.",
(_CH, "config_hash"): _NOREADER + (
"A config FINGERPRINT. The hub authors the config and knows its own generation "
"(desired_generation, config_version), which is the convergence question it actually asks. "
"A hash it did not compute answers a question nobody has posed."),
(_CH, "reporting_disabled"): (
"redundant: the box sends health.status = \"disabled\" in the SAME minimal report "
"(controller cmd/controller/main.go:1254), the hub decodes it into reports.health_status, and "
"web/rollup.go:25 renders that customer as 'disabled'. The state IS visible; this flag is a "
"second spelling of a fact already read. ⚠ DECIDED ON ITS OWN MERITS 2026-08-13, and the "
"decision came with a REAL finding that a second decoded flag would NOT have fixed: "
"StalenessChecker.Check (monitor/staleness.go:88+) is age-only — it skips BLOCKED customers "
"and nothing else — so a deliberately-silent box still goes stale at 30 min and down at 60. "
"Filed as R-321. The fix belongs in the checker, which already has the health status it "
"needs; adding a field here would have felt like progress and left the alarm firing."),
(_CH, "stacks"): _NOREADER + (
"The whole per-stack report object. The hub's app view is built from app_telemetry, which is "
"a purpose-built wire with its own table; a second, differently-shaped source for the same "
"screen is how two answers to one question get shipped."),
(_CH, "storage.migrated_to"): _NOREADER + (
"A drive-migration marker: box-local bookkeeping about where data was moved ON that box. The "
"hub holds no drive-layout intent to reconcile it against, so it could only be displayed, "
"and a fact displayed with nothing to compare it to is decoration."),
(_CH, "backup.last_db_dump"): _R264,
(_CH, "backup.last_integrity_check"): _R264,
# ---- R-555 (2026-10-05): surfaced when comments stopped counting as receivers ----
#
# Each of these passed for months because its NAME occurred hub-side only in a comment. No hub struct
# decodes any of them. Listed so the drop is visible; none is known to be needed by a hub surface.
# **Delete an entry when a hub struct starts decoding the field.**
(_AH, "audit_tail"): (
"R-555: ignored on purpose — hub/internal/api/handler.go's hostReportPayload comment names it "
"among the fields the hub does not decode (backups/restore_tests/pbs_snapshots/audit_tail)."),
(_AH, "guests.spec"): (
"R-555: the parent of guests.spec.disk_bytes / memory_bytes above — sizing is hub-owned intent, "
"not mirrored reality, so the object is not decoded either."),
(_CH, "backup.integrity_ok"): (
"R-555 / R-331: no producer since disk-tier restic moved to the agent (slice 8C); the Backup card "
"dropped it rather than re-sourcing it (hub/internal/web/backup_card.go)."),
(_CH, "backup.repo_size_mb"): (
"R-555 / R-331: no producer since slice 8C; the Backup card reads offsite.repo_size_bytes "
"instead (hub/internal/web/backup_card.go)."),
(_CH, "stacks.deployed"): (
"R-555: surfaced 2026-10-05 — the hub decodes no deployed-app list from the report; no hub "
"surface needs it (R-888 closed by ruling, 09 §3 decision 130, 2026-10-05)."),
(_CH, "storage.decommissioned"): (
"R-555: surfaced 2026-10-05 — the hub decodes no per-drive decommissioned marker from the report; "
"no hub surface needs it (R-888 closed by ruling, 09 §3 decision 130, 2026-10-05)."),
}
# A node whose IMMEDIATE CHILDREN are still checked but whose DEEPER descendants are not, because the
# receiver passes the subtree through without decoding it. Keyed by (wire label, dotted path).
#
# ⚠ THE DEPTH IS THE WHOLE POINT AND IT IS NOT COSMETIC. The hub stores each DR-recipe half as
# json.RawMessage and re-emits nested shapes verbatim — so the LEAVES genuinely are not on this wire
# in the decoded sense. But the TOP-LEVEL SECTION KEYS of each half are decoded, by
# `hostHalfShape` / `appHalfShape` in hub/internal/store/dr_recipe.go, and those two structs are
# ALLOW-LISTS: a section an emitter adds is silently discarded until it is named in BOTH the shape
# struct and AssembledRecipe. That has already cost one shipped section — `offsite_restic` was
# emitted by the controller from fork-4, stored intact, and never appeared in a delivered recipe
# (R-122). Treating `dr_recipe` as wholly opaque would put exactly that defect back outside this
# gate's reach.
OPAQUE_BELOW = {
("agent -> hub (POST /host-report)", "dr_recipe"): (
"the hub stores the agent's half verbatim (`DRRecipe json.RawMessage`) and re-emits nested "
"shapes untouched; its TOP-LEVEL sections are decoded by hostHalfShape and ARE checked"),
("controller -> hub (POST /report)", "dr_recipe"): (
"the hub stores the controller's half verbatim (`SaveDRRecipeAppHalf`) and re-emits nested "
"shapes untouched; its TOP-LEVEL sections are decoded by appHalfShape and ARE checked"),
}
STRUCT_RE = re.compile(r"^type\s+(\w+)\s+struct\s*\{", re.M)
FIELD_RE = re.compile(
r"^\s*(?:(\w+)\s+)?([\[\]\*\w\.]+)\s+`[^`]*json:\"([^\"]*)\"[^`]*`")
INLINE_STRUCT_RE = re.compile(r"^\s*(\w+)\s+(\*?)struct\s*\{")
def die(msg, code=2):
print(msg, file=sys.stderr)
sys.exit(code)
def go_files(root, include_tests=False):
out = []
for dp, dn, fn in os.walk(root):
dn[:] = [d for d in dn if d not in (".git", "vendor", "node_modules")]
for f in fn:
if not f.endswith(".go"):
continue
if not include_tests and f.endswith("_test.go"):
continue
out.append(os.path.join(dp, f))
return out
def parse_structs(path):
"""name -> (body, dir). Anonymous inline structs are inlined into the parent's field list."""
src = open(path, encoding="utf-8", errors="replace").read()
out = {}
for m in STRUCT_RE.finditer(src):
i, depth = m.end(), 1
while i < len(src) and depth:
if src[i] == "{":
depth += 1
elif src[i] == "}":
depth -= 1
i += 1
out[m.group(1)] = src[m.end():i - 1]
return out
def build_index(repo_root):
"""(dir, name) -> body, plus name -> [(dir, body)] for unqualified fallback."""
by_dir, by_name = {}, {}
for p in go_files(repo_root):
d = os.path.relpath(os.path.dirname(p), repo_root)
for name, body in parse_structs(p).items():
by_dir[(d, name)] = body
by_name.setdefault(name, []).append((d, body))
return by_dir, by_name
def split_fields(body):
"""Yield (json_tag, go_type, inline_body_or_None) for one struct body, handling nested
anonymous structs by capturing their body."""
lines = body.split("\n")
i = 0
while i < len(lines):
line = lines[i]
m = INLINE_STRUCT_RE.match(line)
if m:
# capture the inline struct body, then read the tag off its closing line
depth, j, buf = 1, i + 1, []
while j < len(lines) and depth:
depth += line_delta(lines[j])
if depth:
buf.append(lines[j])
j += 1
closing = lines[j - 1] if j - 1 < len(lines) else ""
tm = re.search(r'json:"([^"]*)"', closing)
tag = tm.group(1).split(",")[0] if tm else None
yield (tag, "INLINE", "\n".join(buf))
i = j
continue
fm = FIELD_RE.match(line)
if fm:
tag = fm.group(3).split(",")[0]
if tag and tag != "-":
yield (tag, fm.group(2), None)
i += 1
def line_delta(line):
return line.count("{") - line.count("}")
def base_type(t):
return t.lstrip("*[]").split(".")[-1]
def qualifier(t):
parts = t.lstrip("*[]").split(".")
return parts[0] if len(parts) > 1 else None
def walk(by_dir, by_name, start_dir, start_type, seen=None, prefix=""):
"""Transitively collect (tag, dotted_path) reachable from a root type."""
if seen is None:
seen = set()
key = (start_dir, start_type)
if key in seen:
return []
seen.add(key)
body = by_dir.get(key)
if body is None:
cands = by_name.get(start_type, [])
if not cands:
return []
body = cands[0][1]
start_dir = cands[0][0]
out = []
for tag, gotype, inline in split_fields(body):
if tag is None:
continue
dotted = (prefix + "." + tag) if prefix else tag
out.append((tag, dotted))
# NOTE: the walk descends everywhere; OPAQUE_BELOW filtering happens in run(), so that the
# immediate children of an opaque node are still collected and checked. See OPAQUE_BELOW.
if inline is not None:
for t2, g2, i2 in split_fields(inline):
if t2 is None:
continue
out.append((t2, dotted + "." + t2))
bt = base_type(g2)
if i2 is None and (start_dir, bt) in by_dir or bt in by_name:
out += walk(by_dir, by_name, start_dir, bt, seen, dotted + "." + t2)
continue
bt = base_type(gotype)
if bt in ("string", "int", "int64", "bool", "float64", "byte", "any",
"RawMessage", "Time", "interface{}", "uint64", "int32", "uint32"):
continue
q = qualifier(gotype)
nd = start_dir
if q:
for (d, n) in by_dir:
if n == bt and os.path.basename(d) == q:
nd = d
break
if (nd, bt) in by_dir or bt in by_name:
out += walk(by_dir, by_name, nd, bt, seen, dotted)
return out
TOKEN_RE = re.compile(r"[A-Za-z0-9_]+")
TEMPLATE_COMMENT_RE = re.compile(r"\{\{-?\s*/\*.*?\*/\s*-?\}\}", re.S)
def strip_go_comments(src):
"""Go source with every // and /* */ comment removed; string, raw-string and rune literals kept.
R-555: a field whose name appeared hub-side only in a COMMENT counted as received — `language`
passed because configs.go said "this page's existing language". A comment is prose, not a decoder.
A small lexer rather than a regex, because a regex cannot tell `//` in "http://…" or a backtick
struct tag from a comment, and the struct tags are exactly what this gate reads.
"""
out, i, n = [], 0, len(src)
while i < n:
c = src[i]
if c == "/" and i + 1 < n and src[i + 1] == "/":
j = src.find("\n", i)
i = n if j < 0 else j # keep the newline
continue
if c == "/" and i + 1 < n and src[i + 1] == "*":
j = src.find("*/", i + 2)
i = n if j < 0 else j + 2
out.append(" ")
continue
if c in "\"'`":
j = i + 1
while j < n and src[j] != c:
if c != "`" and src[j] == "\\":
j += 1
elif c != "`" and src[j] == "\n":
break # an unterminated literal ends at the line
j += 1
out.append(src[i:j + 1])
i = j + 1
continue
out.append(c)
i += 1
return "".join(out)
def strip_template_comments(src):
"""An HTML template with every {{/* … */}} action removed (R-555)."""
return TEMPLATE_COMMENT_RE.sub(" ", src)
def receiver_tokens(repo_root):
"""Every `[A-Za-z0-9_]+` token in the receiver's PRODUCTION Go and templates, as a set.
WHOLE-TOKEN, NOT SUBSTRING. A substring test reports a genuinely dropped field as received:
`healed_at` occurs inside `privsep_healed_at`. That false negative was caught by this gate's own
control — R-260 named `healed_at`, so its absence from the first run's output was the tell.
PURE PYTHON, NO `grep`. The first version shelled out to `grep -rnE --include=...`, which works
on a workstation and returns NOTHING on the CI runner, whose image carries python3 and git and
deliberately little else — its grep does not support `--include`. Empty stdout was then read as
"the tag is absent", so the gate convicted ALL 174 checked tags and CI went red while the
pre-push hook was green. **A gate that silently reads a tool failure as a finding is worse than
no gate**, and swallowing that error is the exact thing this repo's rules forbid. Reading the
files here removes the dependency, is portable, and is one pass instead of ~350 subprocesses.
Test files and `testdata/` are excluded deliberately: a tag present only in a fixture is not
decodable by production — and that is R-262 exactly, where the cross-repo golden carries two
fields no hub struct reads.
"""
toks = set()
n = 0
for dp, dn, fn in os.walk(repo_root):
dn[:] = [d for d in dn if d not in (".git", "vendor", "node_modules", "testdata")]
for f in fn:
if f.endswith("_test.go"):
continue
if not (f.endswith(".go") or f.endswith(".html")):
continue
p = os.path.join(dp, f)
try:
with open(p, encoding="utf-8", errors="replace") as fh:
src = fh.read()
# R-555: comments are stripped first — a name in prose is not a decoder.
src = strip_go_comments(src) if f.endswith(".go") else strip_template_comments(src)
toks.update(TOKEN_RE.findall(src))
except OSError as e:
# never swallowed: an unreadable source file makes the answer unknown, not "absent"
die("wire-contract gate INCONCLUSIVE: cannot read %s: %s" % (p, e))
n += 1
if n == 0:
die("wire-contract gate INCONCLUSIVE: no production .go/.html found under %s — a receiver "
"with no source cannot be searched, and an empty search is not evidence of absence."
% repo_root)
return toks
def run(root_override=None, quiet=False):
"""Returns (exit_code, convictions)."""
repos = dict(REPOS)
if root_override:
repos.update(root_override)
for label, path in repos.items():
if not os.path.isdir(path):
die("wire-contract gate INCONCLUSIVE: sibling clone %r not found at %s\n"
" This gate compares two repositories; it cannot pass without both." % (label, path))
indexes = {k: build_index(v) for k, v in repos.items()}
# one pass per receiving repo, not one subprocess per tag
rtokens = {k: receiver_tokens(v) for k, v in repos.items()}
convictions = []
kinds = [] # R-315: (root label, which kind of check it got) — printed on every run
checked = skipped = 0
for label, emitter, pkgdir, rootname, receiver in ROOTS:
by_dir, by_name = indexes[emitter]
if (pkgdir, rootname) not in by_dir:
die("wire-contract gate INCONCLUSIVE: declared root %s.%s not found in %s/%s\n"
" A root that cannot be resolved is not a pass — fix the ROOTS table or the type."
% (emitter, rootname, emitter, pkgdir))
tags = walk(by_dir, by_name, pkgdir, rootname)
seen_tags = {}
for tag, dotted in tags:
seen_tags.setdefault(tag, dotted)
opaque_roots = [p for (lbl, p) in OPAQUE_BELOW if lbl == label]
missing = []
if label in MIRRORS:
mdir, mtype = MIRRORS[label]
rby_dir, rby_name = indexes[receiver]
if (mdir, mtype) not in rby_dir:
die("wire-contract gate INCONCLUSIVE: declared mirror %s.%s not found in %s/%s\n"
" A mirror that cannot be resolved is not a pass — fix MIRRORS or the type."
% (receiver, mtype, receiver, mdir))
mirror_paths = {d for _, d in walk(rby_dir, rby_name, mdir, mtype)}
emitted = sorted({d for _, d in tags})
n_root = 0
for dotted in emitted:
if (label, dotted) in ALLOWLIST or any(
dotted.startswith(o + ".") and dotted.count(".") > o.count(".") + 1
for o in opaque_roots):
skipped += 1
continue
checked += 1
n_root += 1
if dotted not in mirror_paths:
missing.append((dotted.split(".")[-1], dotted))
kinds.append((label, "FIELD-BY-FIELD against %s %s.%s (%d path(s))" % (receiver, mdir, mtype, n_root)))
if missing:
convictions.append((label, receiver, missing))
continue
kinds.append((label, "name-reachability only (a tag found ANYWHERE in %s passes)" % receiver))
for tag, dotted in sorted(seen_tags.items()):
if tag in GENERIC:
skipped += 1
continue
# opaque BELOW: the node itself and its immediate children are checked, deeper is not
if any(dotted.startswith(o + ".") and dotted.count(".") > o.count(".") + 1
for o in opaque_roots):
skipped += 1
continue
if (label, dotted) in ALLOWLIST:
skipped += 1
continue
checked += 1
if tag not in rtokens[receiver]:
missing.append((tag, dotted))
if missing:
convictions.append((label, receiver, missing))
if not quiet:
print("wire-contract gate — %d tag(s) checked across %d declared wire(s); "
"%d skipped (generic / opaque / allowlisted)" % (checked, len(ROOTS), skipped))
for label, kind in kinds:
print(" %-52s %s" % (label, kind))
if convictions:
if not quiet:
for label, receiver, missing in convictions:
print("\n %s" % label)
print(" the RECEIVER (%s) contains no occurrence of:" % receiver)
for tag, dotted in missing:
print(" %-32s (emitted at %s)" % (tag, dotted))
total = sum(len(m) for _, _, m in convictions)
print("\nWIRE-CONTRACT GATE FAILED: %d emitted field(s) cannot be received." % total)
print("A tag whose literal string occurs nowhere in the receiving repo cannot be decoded")
print("by any struct, named or anonymous — encoding/json discards it on arrival.")
print("Fix: model the field on the receiving side and say what consults it — or, if it is")
print("deliberately not consumed, add it to ALLOWLIST WITH A REASON. Never a quiet skip.")
print("\nBlind spots (a green is not full coverage — see the module docstring):")
print(" generic tag names are not checked; reachability of a NAME is not use of a VALUE;")
print(" only the declared ROOTS are covered (hub desired-state and the agent local API")
print(" are NOT).")
return 1, convictions
if not quiet:
print("wire-contract gate OK — every emitted field is at least decodable by its receiver")
print(" (BLIND SPOTS: generic tag names skipped; name-reachability is not use; only the")
print(" declared ROOTS are covered — hub desired-state and the agent local API are not.)")
return 0, []
def selftest():
"""Plant an unreachable tag on a real root in a THROWAWAY copy and assert conviction.
A gate nobody has watched fail has not been shown to work — and last night an off-the-shelf tool
for a neighbouring class was rejected because it was made to prove itself and could not.
"""
tmp = tempfile.mkdtemp(prefix="wirecontract-selftest-")
try:
agent = os.path.join(tmp, "agent")
shutil.copytree(REPOS["agent"], agent, symlinks=True,
ignore=shutil.ignore_patterns(".git"))
rp = os.path.join(agent, "internal", "hub", "report.go")
src = open(rp, encoding="utf-8").read()
marker = "type HostReport struct {"
if marker not in src:
die("SELFTEST INCONCLUSIVE: could not find %r to plant into" % marker)
planted = src.replace(
marker,
marker + "\n\tZZProbeUnreceivable string `json:\"zz_probe_unreceivable\"`", 1)
if planted == src:
die("SELFTEST INCONCLUSIVE: the plant did not apply")
open(rp, "w", encoding="utf-8").write(planted)
code, convictions = run({"agent": agent}, quiet=True)
found = any("zz_probe_unreceivable" == t
for _, _, miss in convictions for t, _ in miss)
print(" planted an unreachable tag on agent HostReport -> gate exit %d, "
"planted tag convicted: %s" % (code, found))
if code != 1 or not found:
print("SELFTEST FAILED: the gate did not convict a planted unreachable tag.")
return 1
code_clean, _ = run(quiet=True)
print(" unplanted tree -> gate exit %d" % code_clean)
if code_clean == 2:
print("SELFTEST FAILED: the unplanted run was INCONCLUSIVE.")
return 1
print("SELFTEST OK — the gate convicts a planted unreachable tag and the plant is the only "
"difference.")
return 0
finally:
shutil.rmtree(tmp, ignore_errors=True)
if __name__ == "__main__":
if "--selftest" in sys.argv:
sys.exit(selftest())
sys.exit(run()[0])
Reference in New Issue View Git Blame Copy Permalink