New shared scripts/instructions_gate.py, registered in controller_gates.py and agent_gates.py, never copied into a sibling repo (the reuse_refs_check.py precedent). 20 fixture tests, all asserting the effect: exit code AND that the message names the file and the reason. It is a consistency gate, not a budget gate, and the failure message says so. A /context reading measured the instruction files at 15k tokens against 869k free in a 1M window -- space is not the constraint, and a future reader must not re-derive the wrong reason. The 200-line ceiling is adherence guidance; a file nobody can hold in their head is where contradictions hide, and five were found here. Checks run against effective text (HTML comments stripped, because they are stripped before injection): the line ceiling; every .claude/rules/*.md declares paths: or an explicit unconditional: true; no component version literal; no TEMPORARY block carrying a past date; and the workspace-root CLAUDE.md is byte-identical to its versioned copy -- the live file sits outside any git repo, so that copy is its only version-controlled record. Two traps recorded so they are not reintroduced: a bare \d+\.\d+\.\d+ matches the first three octets of every IPv4 (the gate excludes dotted quads, or it fails on 192.168.0.180 in the agent's own file); and unconditional: true is NOT a Claude Code feature but this project's own marker. Workspace-root CLAUDE.md 208 -> 182 lines (142 effective), copy kept identical. The nine-instance invariant table moved into the felhom-testing skill, which triggers when writing or reviewing a test; all three directive bullets stayed in the core. felhom.eu/CLAUDE.md got surgical corrections only and is knowingly still over the ceiling at 227 effective lines -- closing it needs the restructure R-229 defers, said plainly rather than quietly absorbed. CONTEXT.md gains standing ruling S-35. OPEN-ITEMS.md gains R-229. Docs only -- no Go, no version bump, nothing built or deployed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JJc8sAGRWmavP3rMtdpkr2
This commit is contained in:
@@ -0,0 +1,238 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Instruction-file consistency gate — keeps CLAUDE.md files from silently regrowing.
|
||||
|
||||
Usage: python3 scripts/instructions_gate.py <repo-root> [<repo-root> ...]
|
||||
python3 scripts/instructions_gate.py --fast <repo-root> (identical: no network,
|
||||
no container runtime)
|
||||
|
||||
THIS IS A CONSISTENCY GATE, NOT A BUDGET GATE. Measured 2026-08-06 on a 1M-token window: the
|
||||
instruction files occupied 15k tokens against 869k free. Space is NOT the constraint here and no
|
||||
failure message may claim it is. The line ceiling exists because longer instruction files reduce
|
||||
ADHERENCE — Anthropic's guidance is to keep a CLAUDE.md under 200 lines — and because a file nobody
|
||||
can hold in their head is where contradictions hide. Four were found in this project on 2026-08-06,
|
||||
two of which decided where a destructive drill runs.
|
||||
|
||||
WHAT IS COUNTED. Every check runs against EFFECTIVE text: block-level HTML comments are stripped
|
||||
first, because they are stripped before injection and never reach the model. Verified empirically on
|
||||
Claude Code 2.1.222 with a control (two plain markers -> both seen) and a treatment (one marker
|
||||
inside <!-- -->, twice -> not seen). That is the whole point of the comment convention: earned
|
||||
rationale stays in the repo for human readers at zero cost to the instructions.
|
||||
|
||||
CHECKS (each names the file and the reason; a missing input is a FAILURE, never a skip):
|
||||
|
||||
1. <root>/CLAUDE.md is at most MAX_LINES effective lines.
|
||||
2. Every <root>/.claude/rules/*.md declares `paths:` in frontmatter, or `unconditional: true`.
|
||||
NOTE: `paths:` is a Claude Code feature — a rule carrying it loads only when a file matching
|
||||
one of its globs is read. `unconditional: true` is NOT a product feature; it is OUR marker,
|
||||
asserting that always-loading was deliberate. The product loads a rule with no `paths:` key
|
||||
unconditionally either way, so this check catches the ACCIDENT, not the product behaviour.
|
||||
3. No component version literal in effective text. Versions change several times a day and the
|
||||
fleet is not uniform, so a version in an instruction file is stale within a day (ask the hub's
|
||||
/hosts + /configs, or the box). Historical citations belong in an HTML comment beside the rule
|
||||
they justify, where they inform a human and cannot go stale in the model's view.
|
||||
IPv4 addresses are NOT versions — a bare \\d+\\.\\d+\\.\\d+ matches the first three octets of
|
||||
every one of them, which is how this check would otherwise fail on its own repo.
|
||||
4. No TEMPORARY block carrying a date already past. The block this gate was written for said
|
||||
"TEMPORARY - until ~2026-08-02 ... Delete this block on return" and was still being read as
|
||||
current fact on 2026-08-06, while a sibling repo's CLAUDE.md asserted the opposite.
|
||||
5. The workspace-root CLAUDE.md and its versioned copy at
|
||||
felhom.eu/documentation/runbooks/workspace-CLAUDE.md are byte-identical. The live file sits in
|
||||
a directory that is not a git repo, so the copy is the only version-controlled record of it;
|
||||
nothing but this check enforces that they agree.
|
||||
|
||||
THE POSITIVE OBSERVABLE. Every root prints a per-check tally with the measured numbers, not just a
|
||||
verdict. "0 failures" alone cannot tell a working gate from a blind one — if the effective line
|
||||
count suddenly reads 3, the stripper broke, and the count is where you see it.
|
||||
|
||||
The kill condition is pinned by scripts/test_instructions_gate.py: an over-length CLAUDE.md still
|
||||
FAILS, and a rule file with neither marker still FAILS.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
MAX_LINES = 200
|
||||
|
||||
# Block-level HTML comments: stripped before injection, so they cost nothing and are not counted.
|
||||
COMMENT_RE = re.compile(r"<!--.*?-->", re.S)
|
||||
|
||||
# A semver-ish literal that is NOT part of a dotted quad (IPv4) and not part of a longer run.
|
||||
VERSION_RE = re.compile(r"(?<![\d.])\d+\.\d+\.\d+(?![\d.])")
|
||||
|
||||
# "TEMPORARY" anywhere on a line, plus an ISO date somewhere in that block.
|
||||
TEMPORARY_RE = re.compile(r"TEMPORARY")
|
||||
ISO_DATE_RE = re.compile(r"(\d{4})-(\d{2})-(\d{2})")
|
||||
|
||||
WORKSPACE_COPY = os.path.join(
|
||||
"felhom.eu", "documentation", "runbooks", "workspace-CLAUDE.md"
|
||||
)
|
||||
|
||||
|
||||
def effective(text):
|
||||
"""The text the model actually receives: HTML comments removed."""
|
||||
return COMMENT_RE.sub("", text)
|
||||
|
||||
|
||||
def today_tuple():
|
||||
"""Local date as (y, m, d). Injectable via FELHOM_GATE_TODAY for the test suite."""
|
||||
override = os.environ.get("FELHOM_GATE_TODAY")
|
||||
if override:
|
||||
m = ISO_DATE_RE.match(override.strip())
|
||||
if m:
|
||||
return tuple(int(g) for g in m.groups())
|
||||
import datetime
|
||||
|
||||
d = datetime.date.today()
|
||||
return (d.year, d.month, d.day)
|
||||
|
||||
|
||||
def check_length(path, failures, tally):
|
||||
with io_open(path) as fh:
|
||||
eff = effective(fh.read())
|
||||
n = eff.count("\n")
|
||||
tally.append(" CLAUDE.md effective lines : %d (ceiling %d)" % (n, MAX_LINES))
|
||||
if n > MAX_LINES:
|
||||
failures.append(
|
||||
"%s: %d effective lines, ceiling %d. This is an ADHERENCE limit, not a space "
|
||||
"limit — long instruction files get followed less reliably and hide "
|
||||
"contradictions. Move path-bound guidance into .claude/rules/*.md with a `paths:` "
|
||||
"list, procedures into the skill that already covers them, and earned rationale "
|
||||
"into an HTML comment (free: stripped before injection)."
|
||||
% (path, n, MAX_LINES)
|
||||
)
|
||||
|
||||
|
||||
def check_rules(root, failures, tally):
|
||||
rules_dir = os.path.join(root, ".claude", "rules")
|
||||
if not os.path.isdir(rules_dir):
|
||||
tally.append(" rule files : none (no .claude/rules/)")
|
||||
return
|
||||
names = sorted(n for n in os.listdir(rules_dir) if n.endswith(".md"))
|
||||
scoped = 0
|
||||
for name in names:
|
||||
path = os.path.join(rules_dir, name)
|
||||
with io_open(path) as fh:
|
||||
head = fh.read(2048)
|
||||
has_paths = re.search(r"^paths:", head, re.M) is not None
|
||||
has_uncond = re.search(r"^unconditional:\s*true\s*$", head, re.M) is not None
|
||||
if has_paths:
|
||||
scoped += 1
|
||||
elif not has_uncond:
|
||||
failures.append(
|
||||
"%s: rule file has neither a `paths:` frontmatter list nor an explicit "
|
||||
"`unconditional: true`. Without `paths:` Claude Code loads it in EVERY session, "
|
||||
"which is rarely what a rule file is for — add the globs it applies to, or "
|
||||
"declare `unconditional: true` to say the always-loading is deliberate." % path
|
||||
)
|
||||
tally.append(
|
||||
" rule files : %d (%d path-scoped)" % (len(names), scoped)
|
||||
)
|
||||
|
||||
|
||||
def check_versions(path, failures, tally):
|
||||
with io_open(path) as fh:
|
||||
eff = effective(fh.read())
|
||||
hits = []
|
||||
for i, line in enumerate(eff.split("\n"), 1):
|
||||
for m in VERSION_RE.finditer(line):
|
||||
hits.append((i, m.group(), line.strip()[:90]))
|
||||
tally.append(" version literals : %d" % len(hits))
|
||||
for lineno, ver, ctx in hits:
|
||||
failures.append(
|
||||
"%s:%d: component version literal %r — versions change several times a day and the "
|
||||
"fleet is not uniform, so this is stale within a day. Ask the hub (/hosts, /configs) "
|
||||
"or the box. A historical citation belongs in an HTML comment beside the rule it "
|
||||
"justifies.\n %s" % (path, lineno, ver, ctx)
|
||||
)
|
||||
|
||||
|
||||
def check_temporary(path, failures, tally):
|
||||
with io_open(path) as fh:
|
||||
eff = effective(fh.read())
|
||||
lines = eff.split("\n")
|
||||
today = today_tuple()
|
||||
found = 0
|
||||
for i, line in enumerate(lines, 1):
|
||||
if not TEMPORARY_RE.search(line):
|
||||
continue
|
||||
found += 1
|
||||
window = "\n".join(lines[i - 1 : i + 6])
|
||||
for m in ISO_DATE_RE.finditer(window):
|
||||
when = tuple(int(g) for g in m.groups())
|
||||
if when < today:
|
||||
failures.append(
|
||||
"%s:%d: TEMPORARY block carrying the past date %s. A temporary block that "
|
||||
"outlives its own deadline is read as current fact — this gate exists "
|
||||
"because one did, for four days, while a sibling repo asserted the "
|
||||
"opposite. Delete it; the audit trail is the record."
|
||||
% (path, i, m.group())
|
||||
)
|
||||
break
|
||||
tally.append(" TEMPORARY blocks : %d" % found)
|
||||
|
||||
|
||||
def check_workspace_copy(workspace_root, failures, tally):
|
||||
live = os.path.join(workspace_root, "CLAUDE.md")
|
||||
copy = os.path.join(workspace_root, WORKSPACE_COPY)
|
||||
if not os.path.exists(live) or not os.path.exists(copy):
|
||||
tally.append(" workspace copy : n/a (not this workspace)")
|
||||
return
|
||||
with io_open(live) as a, io_open(copy) as b:
|
||||
same = a.read() == b.read()
|
||||
tally.append(" workspace copy identical : %s" % ("yes" if same else "NO"))
|
||||
if not same:
|
||||
failures.append(
|
||||
"%s and %s have diverged. The live workspace file sits in a directory that is not a "
|
||||
"git repo, so the copy is its only version-controlled record — nothing but this "
|
||||
"check enforces that they agree. Copy the live file over the versioned one."
|
||||
% (live, copy)
|
||||
)
|
||||
|
||||
|
||||
def io_open(path):
|
||||
return open(path, "r", encoding="utf-8")
|
||||
|
||||
|
||||
def main(argv):
|
||||
roots = [a for a in argv[1:] if a != "--fast"]
|
||||
if not roots:
|
||||
sys.stderr.write("usage: instructions_gate.py [--fast] <repo-root> ...\n")
|
||||
return 2
|
||||
|
||||
failures = []
|
||||
for root in roots:
|
||||
root = os.path.abspath(root)
|
||||
print("instructions_gate: %s" % root)
|
||||
tally = []
|
||||
|
||||
claude_md = os.path.join(root, "CLAUDE.md")
|
||||
if not os.path.exists(claude_md):
|
||||
failures.append(
|
||||
"%s: no CLAUDE.md. A missing input is a FAILURE, never a skip — a gate that "
|
||||
"silently passes on an absent file is how the check stops running." % root
|
||||
)
|
||||
else:
|
||||
check_length(claude_md, failures, tally)
|
||||
check_versions(claude_md, failures, tally)
|
||||
check_temporary(claude_md, failures, tally)
|
||||
|
||||
check_rules(root, failures, tally)
|
||||
check_workspace_copy(os.path.dirname(root), failures, tally)
|
||||
|
||||
for line in tally:
|
||||
print(line)
|
||||
|
||||
if failures:
|
||||
print("")
|
||||
print("instructions_gate: %d FAILURE(S)" % len(failures))
|
||||
for f in failures:
|
||||
print(" - %s" % f)
|
||||
return 1
|
||||
|
||||
print("")
|
||||
print("instructions_gate: OK")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
Reference in New Issue
Block a user