the decoy sweep: 29 gates read, 16 fooled, 10 fixed - and a gate that refuses the next one (R-421)
gates / gates (push) Failing after 17s
gates / gates (push) Failing after 17s
THE CLASS, now a row: an instrument that matches a LABEL rather than the fact it names. Five instances - R-410, R-400, R-378, R-419, R-94 - and EVERY ONE was found by accident, by someone looking at something else. The gates enforce every other rule in this project, including the rule that findings must be written down rather than left in prose. Nothing had ever checked the gates. METHOD, and it is the transferable part: for each gate, construct the label WITHOUT the fact - a directory with the right name and no bake log, a handler case that exists only in a comment, a note whose prose mentions the marker it lacks - run the gate, record what it says. No verdict was reached by reading. Reading is how all five hid. RESULT: 29 distinct scripts (35 registrations; three are shared across three runners). 19 sound, 4 holes left OPEN with rows, 6 that no plausible decoy could be built for and are named UNTESTED rather than called sound. A gate nobody tried to fool is UNKNOWN. SCOPE IS A FACT TOO - the largest single cause, and mundane. Eight gates decided what to look at with os.listdir, one level. Every one was green AND CORRECT today, and every one would have gone blind the moment anyone added a subdirectory. mojibake and docker-v already used os.walk, caught the identical planted file, and are the control that proves the cause was the listing and not the decoy. IN THIS REPO: hub-confirm and manifest-bearer now walk. observations_gate (R-419, CLOSED) requires a marker at a line start or after a sentence boundary and strips inline code spans - a note SAYING it carries no marker no longer satisfies the marker test. closed-register now CONVICTS on a row it cannot parse instead of warning: FOUR rows were in that state, TWO of them written by the session that closed them the day before, and every one was exempt from the only check that reads that file. The rows were repaired first and the conviction added second - registering a failing gate refuses every push. THE META-GATE: decoy_coverage_gate.py refuses a gate registered without a decoy or a named exemption. It convicted ITSELF the moment it was registered, which is how it came to have one. Coverage is a DECLARATION the gate AST-parses, never a grep - searching a test file for a gate's name would be the very shape this sweep exists to find. The 20 uncovered gates are listed by name (R-426). NOT FIXED, each with a row and a decoy asserting TODAY's behaviour so the fix must be deliberate: R-422 reuse-refs (only 7 extensions; a rotted .md citation is invisible), R-423 site (PAGES is a hardcoded list of 7), R-424 one-register (a defect parked as `idea`), R-425 offbox-rename (fixed FILES list). R-427: closed_register_gate checks ONE direction - twelve open rows carry a closed verdict and were NOT moved, because telling finished from partly-finished is a judgement and R-378 is the record of a machine getting it wrong. FIVE DECOYS WITHDRAWN AS ILLEGITIMATE, mine, named in the audit. A decoy nobody would write proves nothing, and manufacturing a finding to fill a row is worse than an honest NO. No product code. No version bump. No image. No golden owed. All four runners green. Register: OPEN 172 -> 178, CLOSED 160 -> 161.
This commit is contained in:
@@ -1,3 +1,32 @@
|
||||
## the decoy sweep — can this gate be fooled by a label? (2026-09-01, R-421) — NOT A RELEASE
|
||||
|
||||
**No product code, no version bump, no image, no golden.** A scripts change is not a release.
|
||||
|
||||
Four times in one week a gate turned out to match a NAME instead of the thing it named — R-410 (a
|
||||
`mkdir` turned the release gate green), R-400 (seven debug controls answering nothing), R-378 (a
|
||||
status word inside a sentence), R-419 (a phrase inside prose, including prose saying the marker was
|
||||
absent). **All four found by accident.** The gates enforce everything else here and were the one part
|
||||
nothing had checked.
|
||||
|
||||
**All 29 gate scripts read and decoyed. 16 were fooled.** 10 fixed here, 4 left with rows
|
||||
(R-422..R-425), 6 could not be given a plausible decoy and are named (R-426 group d).
|
||||
|
||||
**The largest single cause was mundane:** eight gates set their SCOPE with `os.listdir` (one level).
|
||||
Green and correct today; blind the moment anyone adds `templates/partials/`. `mojibake` and
|
||||
`docker-v` already used `os.walk`, caught the identical planted file, and are the control that
|
||||
proves the cause was the listing rather than the decoy.
|
||||
|
||||
Full survey table, and the five decoys withdrawn as illegitimate (mine, named):
|
||||
`documentation/audits/AUDIT-gate-decoys-2026-09-01.md`.
|
||||
|
||||
**In this repo:** `hub-confirm` and `manifest-bearer` now walk instead of listing one directory;
|
||||
`observations_gate` (R-419) requires a marker at a line start or after a sentence boundary and strips
|
||||
inline code spans, so a note SAYING it carries no marker no longer satisfies the marker test;
|
||||
`closed-register` now CONVICTS on a row it cannot parse instead of warning — four rows were in that
|
||||
state, two of them written by the session that closed them the day before, and every one was exempt
|
||||
from the only check that reads that file. New: `scripts/test_gate_decoys.py` (12 decoys) and
|
||||
`scripts/decoy_coverage_gate.py`, registered LAST, which convicted itself until given its own decoy.
|
||||
|
||||
## push_scope.py v1.0.0 + repo_gates.py --scope — block who can act, notify who cannot (2026-09-01, R-404/R-417)
|
||||
|
||||
**No product code, no version bump, no image, and deliberately NO GOLDEN** — creating one would be an
|
||||
|
||||
@@ -98,7 +98,18 @@ def main():
|
||||
convicted, warnings, checked = [], [], 0
|
||||
for n, rid, state, line in rows(CLOSED):
|
||||
if state is None:
|
||||
warnings.append((n, rid, "row is not four columns — no state cell to read"))
|
||||
# R-421 (2026-09-01): A ROW THIS GATE CANNOT READ IS A CONVICTION, NOT A WARNING.
|
||||
#
|
||||
# This was a warning, and the gate then printed OK. Four rows were in that state —
|
||||
# R-399 and R-400 for days, and R-404 and R-417 written malformed by the session that
|
||||
# closed them the day before this sweep. **Every one of them was exempt from the only
|
||||
# check that reads this file**, and the gate said OK each time. That is the sweep's own
|
||||
# shape one level up: an instrument that reports a pass over rows it never examined.
|
||||
#
|
||||
# The rows were repaired first and the conviction added second — registering a failing
|
||||
# gate refuses every push, which is the ordering instructions_gate learned the hard way.
|
||||
convictions.append((n, rid, "row is not four columns, so it has NO STATE CELL and this "
|
||||
"gate cannot judge it — an unreadable row is never a pass"))
|
||||
continue
|
||||
checked += 1
|
||||
verdict = leading_verdict(state)
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""decoy_coverage_gate.py — every registered gate ships with a decoy test (R-421).
|
||||
|
||||
Usage: python3 scripts/decoy_coverage_gate.py <repo-root> [<repo-root> ...]
|
||||
Exit 0 covered-or-registered · 1 a gate has neither a decoy nor an exemption · 2 inconclusive.
|
||||
|
||||
WHY THIS EXISTS. Four times in one week a gate turned out to be matching a NAME instead of the thing
|
||||
it named — R-410 (a `mkdir` turned the release gate green), R-400 (seven debug controls answering
|
||||
nothing), R-378 (a status word inside a sentence), R-419 (a phrase inside prose, including prose
|
||||
saying the marker was absent). All four were found by accident. **The gates are the machinery that
|
||||
enforces everything else here, and they were the one part nothing checked.** The 2026-09-01 sweep
|
||||
read all 29 and fooled 16 of them.
|
||||
|
||||
A survey fixes what it finds once. This makes the next one impossible to add quietly: **a NEW gate
|
||||
with no decoy fails immediately**, and the gates not yet covered are listed BY NAME below, each with
|
||||
its row, so the remaining debt is visible and shrinking rather than forgotten in a Python literal.
|
||||
|
||||
R-329's shape, deliberately, because this project already trusts it: walk everything, register the
|
||||
exceptions by name, and let a new one fail rather than slip through.
|
||||
|
||||
⚠ COVERAGE IS A DECLARATION, NOT A GREP. Each decoy suite exports `COVERS = {gate: why}` and this
|
||||
gate AST-parses that literal. Searching the test file for a gate's name would be exactly the
|
||||
substring-for-fact shape this whole sweep exists to find — the gate that checks for label-matching
|
||||
must not itself match on a label.
|
||||
"""
|
||||
import ast
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
|
||||
# ── EXEMPTIONS — each carries its row and one line saying why it is not covered YET ─────────────
|
||||
# Dated 2026-09-01. This list is DEBT, not a settled state: R-426 owns it and names every entry.
|
||||
EXEMPT = {
|
||||
# felhom.eu
|
||||
("felhom.eu", "site"):
|
||||
"R-423 — PAGES is a hardcoded list of 7 files; a new page is unscanned. The decoy passes "
|
||||
"TODAY, so a test asserting rejection would be a lie. Fix is to glob website/*.html.",
|
||||
("felhom.eu", "one-register"):
|
||||
"R-424 — a real defect parked under state `idea` is invisible. Declared in the gate's own "
|
||||
"docstring as residual hole 1; the decoy passes today.",
|
||||
("felhom.eu", "hostinstall"):
|
||||
"R-426 — no plausible decoy constructed yet. It asserts installer invariants against a "
|
||||
"shell script; a legitimate decoy needs a shape a real edit would produce.",
|
||||
("felhom.eu", "wire-contract"):
|
||||
"R-426 — 607 lines comparing emitted fields to receiver structs across two repos; a decoy "
|
||||
"needs a Go edit, which this sweep was forbidden from making.",
|
||||
("felhom.eu", "hub-copy"):
|
||||
"COVERED IN THE SWEEP, not yet in a suite: a retired name planted in a NEW hub template was "
|
||||
"REJECTED (it uses os.walk). R-426 tracks moving that decoy into the suite.",
|
||||
("felhom.eu", "due-checks"):
|
||||
"R-426 — no legitimate decoy constructed; the attempt in the sweep was a no-op and its "
|
||||
"verdict was withdrawn rather than reported.",
|
||||
("felhom.eu", "instructions"):
|
||||
"COVERED IN THE SWEEP, not yet in a suite: a version literal in effective text was REJECTED. "
|
||||
"R-426 tracks moving it in.",
|
||||
# felhom-controller
|
||||
("felhom-controller", "docker-v"):
|
||||
"COVERED IN THE SWEEP, not yet in a suite: an unallowlisted `-v` host path in a new Go file "
|
||||
"was REJECTED (it uses os.walk). R-426 tracks moving it in.",
|
||||
("felhom-controller", "offbox-rename"):
|
||||
"R-425 — the FILES list is fixed, so banned branding in a NEW offbox file is unscanned. The "
|
||||
"decoy passes today.",
|
||||
("felhom-controller", "reuse-refs"):
|
||||
"shared script; its decoy lives in felhom.eu/scripts/test_gate_decoys.py.",
|
||||
("felhom-controller", "instructions"):
|
||||
"shared script; see felhom.eu. R-426.",
|
||||
("felhom-controller", "observations"):
|
||||
"shared script; its decoy (R-419) lives in felhom.eu/scripts/test_gate_decoys.py.",
|
||||
# felhom-agent — no decoy suite yet
|
||||
("felhom-agent", "reuse-refs"): "shared script; decoy in felhom.eu. R-426.",
|
||||
("felhom-agent", "instructions"): "shared script; decoy in felhom.eu. R-426.",
|
||||
("felhom-agent", "observations"): "shared script; decoy in felhom.eu. R-426.",
|
||||
("felhom-agent", "published"):
|
||||
"R-426 — a network gate; a decoy needs a fake registry, which this sweep did not build.",
|
||||
("felhom-agent", "release-complete"):
|
||||
"R-426 — the sweep's decoy (a non-version heading on top of CHANGELOG.md) was WITHDRAWN: "
|
||||
"HEAD_RE.search scans the whole file, so the real release is still found and named. The "
|
||||
"gate looked sound; no legitimate decoy has been constructed yet.",
|
||||
# app-catalog
|
||||
("app-catalog-felhom.eu", "image-pins"):
|
||||
"COVERED IN THE SWEEP: a real untagged `image:` line was REJECTED, and the `x-image:` decoy "
|
||||
"was WITHDRAWN as illegitimate (x- fields are inert in Compose). R-426 tracks a suite.",
|
||||
("app-catalog-felhom.eu", "image-resolvable"):
|
||||
"R-426 — needs a container runtime and the network; not a --fast gate.",
|
||||
("app-catalog-felhom.eu", "volume-persistence"):
|
||||
"R-426 — 877 lines that actually run containers and diff them; not a --fast gate.",
|
||||
}
|
||||
|
||||
RUNNERS = {
|
||||
"felhom.eu": os.path.join("scripts", "repo_gates.py"),
|
||||
"felhom-controller": os.path.join("controller", "scripts", "controller_gates.py"),
|
||||
"felhom-agent": os.path.join("scripts", "agent_gates.py"),
|
||||
"app-catalog-felhom.eu": os.path.join("scripts", "catalog_gates.py"),
|
||||
}
|
||||
SUITES = {
|
||||
"felhom.eu": os.path.join("scripts", "test_gate_decoys.py"),
|
||||
"felhom-controller": os.path.join("controller", "scripts", "test_gate_decoys.py"),
|
||||
"felhom-agent": os.path.join("scripts", "test_gate_decoys.py"),
|
||||
"app-catalog-felhom.eu": os.path.join("scripts", "test_gate_decoys.py"),
|
||||
}
|
||||
|
||||
|
||||
def literal_from(path, name):
|
||||
"""AST-parse a module-level literal assignment. Never imports — importing a test RUNS it."""
|
||||
if not os.path.isfile(path):
|
||||
return None
|
||||
try:
|
||||
tree = ast.parse(io.open(path, encoding="utf-8").read())
|
||||
except SyntaxError as e:
|
||||
return ("ERROR", "%s does not parse: %s" % (path, e))
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.Assign):
|
||||
for t in node.targets:
|
||||
if isinstance(t, ast.Name) and t.id == name:
|
||||
try:
|
||||
return ast.literal_eval(node.value)
|
||||
except Exception:
|
||||
return ("ERROR", "%s in %s is not a literal" % (name, path))
|
||||
return None
|
||||
|
||||
|
||||
def gates_of(runner_path):
|
||||
"""The GATES table's LABELS, AST-read — the runner is never imported and never literal_eval'd.
|
||||
|
||||
Only the label is taken, deliberately. Three of the four runners build their script paths with
|
||||
`os.path.join(...)`, so the table as a whole is not a literal — but every label is a plain string
|
||||
constant, which is all this gate needs. Evaluating the whole row would make this gate fail on
|
||||
three repos for a reason that has nothing to do with coverage.
|
||||
"""
|
||||
if not os.path.isfile(runner_path):
|
||||
return None
|
||||
try:
|
||||
tree = ast.parse(io.open(runner_path, encoding="utf-8").read())
|
||||
except SyntaxError:
|
||||
return None
|
||||
for node in tree.body:
|
||||
if not isinstance(node, ast.Assign):
|
||||
continue
|
||||
if not any(isinstance(t, ast.Name) and t.id == "GATES" for t in node.targets):
|
||||
continue
|
||||
if not isinstance(node.value, (ast.List, ast.Tuple)):
|
||||
return None
|
||||
out = []
|
||||
for row in node.value.elts:
|
||||
if isinstance(row, (ast.Tuple, ast.List)) and row.elts:
|
||||
first = row.elts[0]
|
||||
if isinstance(first, ast.Constant) and isinstance(first.value, str):
|
||||
out.append(first.value)
|
||||
return out
|
||||
return None
|
||||
|
||||
|
||||
def main(argv):
|
||||
roots = argv[1:] or ["."]
|
||||
problems, inconclusive, lines = [], [], []
|
||||
total = covered = exempt = 0
|
||||
|
||||
for root in roots:
|
||||
root = os.path.abspath(root)
|
||||
name = os.path.basename(root)
|
||||
if name not in RUNNERS:
|
||||
inconclusive.append("unknown repo %r — no runner registered for it" % name)
|
||||
continue
|
||||
runner = os.path.join(root, RUNNERS[name])
|
||||
if not os.path.isfile(runner):
|
||||
inconclusive.append("%s: runner not found at %s" % (name, runner))
|
||||
continue
|
||||
labels = gates_of(runner)
|
||||
if labels is None:
|
||||
inconclusive.append("%s: could not read the GATES table from %s" % (name, runner))
|
||||
continue
|
||||
covers = literal_from(os.path.join(root, SUITES[name]), "COVERS") or {}
|
||||
if isinstance(covers, tuple):
|
||||
inconclusive.append("%s: %s" % (name, covers[1]))
|
||||
covers = {}
|
||||
|
||||
for label in labels:
|
||||
total += 1
|
||||
if label in covers:
|
||||
covered += 1
|
||||
lines.append(" COVERED %-22s %-20s %s" % (name, label, covers[label][:60]))
|
||||
elif (name, label) in EXEMPT:
|
||||
exempt += 1
|
||||
lines.append(" exempt %-22s %-20s %s" % (name, label, EXEMPT[(name, label)][:60]))
|
||||
else:
|
||||
problems.append((name, label))
|
||||
|
||||
print("decoy-coverage gate — %d registered gate(s): %d with a decoy, %d registered exempt, "
|
||||
"%d UNACCOUNTED" % (total, covered, exempt, len(problems)))
|
||||
for l in sorted(lines):
|
||||
print(l)
|
||||
|
||||
if inconclusive:
|
||||
print()
|
||||
for i in inconclusive:
|
||||
print(" INCONCLUSIVE: %s" % i)
|
||||
if not problems:
|
||||
print("\ndecoy-coverage gate INCONCLUSIVE — an undetermined result is never a pass")
|
||||
return 2
|
||||
|
||||
if problems:
|
||||
print()
|
||||
print("CONVICTED — these gates have neither a decoy test nor a registered exemption:")
|
||||
for repo, label in problems:
|
||||
print(" %s / %s" % (repo, label))
|
||||
print()
|
||||
print("A gate ships with a decoy test that has been SEEN TO FAIL. Construct the label")
|
||||
print("without the fact, run the gate, and assert it convicts — then name the gate in that")
|
||||
print("repo's scripts/test_gate_decoys.py COVERS map.")
|
||||
print("If no plausible decoy exists, say so: add it to EXEMPT here with a row number and one")
|
||||
print("line of reason. An honest exemption is a result; a silent gap is how R-410 happened.")
|
||||
return 1
|
||||
|
||||
print("\ndecoy-coverage gate OK — every registered gate has a decoy or a named exemption (R-426)")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -16,20 +16,35 @@ ROOT = os.path.join("hub", "internal", "web", "templates")
|
||||
NATIVE = re.compile(r"(?<![A-Za-z0-9_$])(?:confirm|prompt)\(\s*[^)\s]")
|
||||
|
||||
|
||||
def templates():
|
||||
"""Every .html under ROOT, AT ANY DEPTH.
|
||||
|
||||
Was `os.listdir(ROOT)`, which is one level only. There are no subdirectories today, so the gate
|
||||
was green and correct — and would have stayed green the moment anyone added `templates/partials/`,
|
||||
which is an ordinary thing to do. Measured 2026-09-01 by planting a template with a native
|
||||
`confirm()` in a new `partials/` directory: the gate passed it (AUDIT-gate-decoys, R-421).
|
||||
"""
|
||||
out = []
|
||||
for dirpath, _dirs, names in os.walk(ROOT):
|
||||
for fn in sorted(names):
|
||||
if fn.endswith(".html"):
|
||||
out.append(os.path.join(dirpath, fn))
|
||||
return sorted(out)
|
||||
|
||||
|
||||
def main():
|
||||
total = 0
|
||||
for fn in sorted(os.listdir(ROOT)):
|
||||
if not fn.endswith(".html"):
|
||||
continue
|
||||
path = os.path.join(ROOT, fn)
|
||||
for path in templates():
|
||||
rel = os.path.relpath(path, ROOT)
|
||||
for lineno, line in enumerate(io.open(path, encoding="utf-8"), 1):
|
||||
if NATIVE.search(line):
|
||||
total += 1
|
||||
print("%s:%d %s" % (fn, lineno, line.strip()[:120].encode('ascii', 'backslashreplace').decode()))
|
||||
print("%s:%d %s" % (rel, lineno, line.strip()[:120].encode('ascii', 'backslashreplace').decode()))
|
||||
if total:
|
||||
print("HUB CONFIRM GATE FAILED: %d native confirm()/prompt() call(s) remain" % total)
|
||||
sys.exit(1)
|
||||
print("hub confirm gate OK — no native confirm()/prompt() in hub templates")
|
||||
print("hub confirm gate OK — no native confirm()/prompt() in hub templates (%d scanned, any depth)"
|
||||
% len(templates()))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -30,10 +30,16 @@ KNOWN_BACKLOG = {"felhom.secret.yaml"}
|
||||
|
||||
def main():
|
||||
total = 0
|
||||
for fn in sorted(os.listdir(ROOT)):
|
||||
if not fn.endswith((".yaml", ".yml")):
|
||||
continue
|
||||
path = os.path.join(ROOT, fn)
|
||||
# AT ANY DEPTH. Was os.listdir(ROOT), one level only — measured 2026-09-01 to miss a
|
||||
# bearer-shaped literal in manifests/overlays/ (R-421). There are no subdirectories today; an
|
||||
# overlays/ or base/ directory is an ordinary thing to add, and the gate would have stayed green.
|
||||
paths = []
|
||||
for dirpath, _dirs, names in os.walk(ROOT):
|
||||
for fn in sorted(names):
|
||||
if fn.endswith((".yaml", ".yml")):
|
||||
paths.append(os.path.join(dirpath, fn))
|
||||
for path in sorted(paths):
|
||||
fn = os.path.basename(path)
|
||||
for lineno, line in enumerate(io.open(path, encoding="utf-8", errors="replace"), 1):
|
||||
for m in BEARER.finditer(line):
|
||||
masked = m.group(0)[:8] + "..." + m.group(0)[-4:]
|
||||
|
||||
@@ -76,8 +76,28 @@ ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
# `## 10. Observations — recorded, not acted on`, `### Observations`, `## Observations:` …
|
||||
HEADING_RE = re.compile(r"^(#+)\s*(?:\d+[.)]\s*)?observations\b", re.IGNORECASE)
|
||||
ITEM_RE = re.compile(r"^\s*(\d+)[.)]\s+(.*)$")
|
||||
FILED_RE = re.compile(r"\bFILED:\s*(R-\d+)", re.IGNORECASE)
|
||||
NOT_A_FINDING_RE = re.compile(r"\bNOT-A-FINDING:\s*(\S.*)$", re.IGNORECASE | re.MULTILINE)
|
||||
# ⚠ THE MARKER IS A MARKER, NOT A MENTION (R-419, fixed 2026-09-01).
|
||||
#
|
||||
# These were `\bFILED:` and `\bNOT-A-FINDING:` searched over the whole item body, so ANY occurrence
|
||||
# counted — including one inside a sentence ABOUT the markers. Measured, and by accident: an
|
||||
# observation reading *"it carries no `FILED:` and no `NOT-A-FINDING:` marker"* was reported
|
||||
# `OK 1. NOT-A-FINDING` and a real push of mine went green over an unfiled finding. **The gate's
|
||||
# whole job is to force an explicit choice, and a sentence disclaiming the choice counted as making
|
||||
# it.** That is the substring-for-field class this project has now shipped five times.
|
||||
#
|
||||
# A marker must now be the START of a line (after optional list/emphasis punctuation), which is
|
||||
# where a real marker is written and where a mention inside prose never is. `**FILED: R-417**` at
|
||||
# the end of a sentence is the common real shape, so a marker is also accepted after a sentence
|
||||
# boundary — but never mid-sentence and never inside backticks.
|
||||
_MARK = r"(?:^|(?<=[.!?)]\s)|(?<=[.!?)]\s\s))[\s>*_\-]*"
|
||||
FILED_RE = re.compile(_MARK + r"\*{0,2}FILED:\*{0,2}\s*(R-\d+)",
|
||||
re.IGNORECASE | re.MULTILINE)
|
||||
NOT_A_FINDING_RE = re.compile(_MARK + r"\*{0,2}NOT-A-FINDING:\*{0,2}\s*(\S.*)$",
|
||||
re.IGNORECASE | re.MULTILINE)
|
||||
|
||||
# A marker written inside backticks is being TALKED ABOUT, never used. Strip inline code spans
|
||||
# before matching — this is what makes the R-419 decoy fail.
|
||||
CODE_SPAN_RE = re.compile(r"`[^`]*`")
|
||||
|
||||
REGISTERS = [
|
||||
os.path.join(ROOT, "documentation", "backlog", "OPEN-ITEMS.md"),
|
||||
@@ -184,6 +204,7 @@ def main():
|
||||
convictions = []
|
||||
satisfied = []
|
||||
for num, body in items:
|
||||
body = CODE_SPAN_RE.sub("", body) # R-419: a marker inside backticks is a mention
|
||||
filed = FILED_RE.findall(body)
|
||||
declared = NOT_A_FINDING_RE.findall(body)
|
||||
first_line = body.split("\n")[0].strip()
|
||||
|
||||
@@ -139,6 +139,13 @@ GATES = [
|
||||
# R-389 — a live finding lived in a REPORT.md observations paragraph and nowhere else, and
|
||||
# REPORT.md is overwritten every session. Fast: stdlib file reads.
|
||||
("observations", os.path.join(SCRIPTS, "observations_gate.py"), [ROOT], True, False),
|
||||
# R-421 — every registered gate across all four repos ships with a decoy test, or is
|
||||
# named in the exemption list with its row. Registered LAST, after every runner was green:
|
||||
# a failing gate refuses every push, which is what instructions_gate learned the hard way.
|
||||
("decoy-coverage", os.path.join(SCRIPTS, "decoy_coverage_gate.py"),
|
||||
[ROOT, os.path.join(os.path.dirname(ROOT), "felhom-controller"),
|
||||
os.path.join(os.path.dirname(ROOT), "felhom-agent"),
|
||||
os.path.join(os.path.dirname(ROOT), "app-catalog-felhom.eu")], True, False),
|
||||
]
|
||||
|
||||
VERDICT = {0: "OK", 1: "FAILED", 2: "INCONCLUSIVE"}
|
||||
|
||||
@@ -0,0 +1,227 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""test_gate_decoys.py — can this gate be fooled by a LABEL? (R-421)
|
||||
|
||||
WHY THIS FILE EXISTS. Four times in one week a gate turned out to be matching a name instead of the
|
||||
thing it named — R-410 (a `mkdir` turned the release gate green), R-400 (seven debug controls that
|
||||
answered nothing), R-378 (a status word inside a sentence), R-419 (a phrase inside prose, including
|
||||
prose saying the marker was absent). **All four were found by accident.** The gates are the machinery
|
||||
that enforces everything else in this project, and they were the one part nothing checked.
|
||||
|
||||
A DECOY IS THE LABEL WITHOUT THE FACT. Each test below constructs one, runs the real gate, and
|
||||
asserts it CONVICTS. Where a decoy would pass, that is a live hole.
|
||||
|
||||
⚠ A DECOY MUST BE THE SHAPE A REAL SESSION WOULD PRODUCE. R-419 was not found by an absurd input —
|
||||
it was found by a genuine note explaining that it carried no marker. That is the standard. A decoy
|
||||
nobody would ever write proves nothing, and saying so is a result.
|
||||
|
||||
⚠ EVERY TEST ASSERTS BOTH DIRECTIONS where it can. A gate that rejects the decoy AND rejects the
|
||||
genuine article is worse than the hole it replaced.
|
||||
|
||||
Fixtures are planted in the real tree and removed in a `finally`. The suite asserts the tree is
|
||||
unchanged at the end.
|
||||
|
||||
Run from the repo root: python3 scripts/test_gate_decoys.py
|
||||
Exit 0 all decoys rejected · 1 a decoy passed (a live hole).
|
||||
"""
|
||||
import io
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
# ── WHAT THIS FILE COVERS ────────────────────────────────────────────────────────────────────────
|
||||
# Read by scripts/decoy_coverage_gate.py, which AST-parses this literal rather than grepping for
|
||||
# gate names — a substring search for coverage would be the very shape this sweep exists to find.
|
||||
# A gate named here MUST have a decoy below that has been seen to fail.
|
||||
COVERS = {
|
||||
"hub-confirm": "a native confirm() in templates/partials/ (scope was os.listdir)",
|
||||
"manifest-bearer": "a bearer literal in manifests/overlays/ (scope was os.listdir)",
|
||||
"observations": "R-419: prose SAYING it carries no marker, plus both genuine markers",
|
||||
"reuse-refs": "a cited .go path that does not exist; the .md hole is asserted as R-422",
|
||||
"golden-currency": "R-410: an empty directory with a perfect name, checked by what it COUNTED",
|
||||
"closed-register": "a verdict cell reading open, and a row with no state cell at all",
|
||||
"decoy-coverage": "a gate registered in a runner with no decoy and no exemption (its red-proof)",
|
||||
}
|
||||
fails = []
|
||||
ran = 0
|
||||
|
||||
|
||||
def gate(script, args=()):
|
||||
p = subprocess.run([sys.executable, os.path.join("scripts", script)] + list(args),
|
||||
cwd=ROOT, capture_output=True, text=True)
|
||||
return p.returncode, p.stdout + p.stderr
|
||||
|
||||
|
||||
def decoy(name, script, plant, args=(), expect="convict"):
|
||||
"""plant() is a callable returning a cleanup callable."""
|
||||
global ran
|
||||
ran += 1
|
||||
cleanup = plant()
|
||||
try:
|
||||
rc, out = gate(script, args)
|
||||
finally:
|
||||
cleanup()
|
||||
want_nonzero = (expect == "convict")
|
||||
if (rc != 0) != want_nonzero:
|
||||
fails.append("%s: decoy %s (rc=%d)\n%s" %
|
||||
(name, "PASSED - LIVE HOLE" if want_nonzero else "was wrongly convicted",
|
||||
rc, out[-700:]))
|
||||
else:
|
||||
print(" ok %-20s %s" % (name, "decoy rejected" if want_nonzero else "genuine accepted"))
|
||||
|
||||
|
||||
def plant_file(path, content):
|
||||
def _plant():
|
||||
made = []
|
||||
d = os.path.dirname(path)
|
||||
if d and not os.path.isdir(d):
|
||||
os.makedirs(d)
|
||||
made.append(d)
|
||||
io.open(path, "w", encoding="utf-8").write(content)
|
||||
|
||||
def _clean():
|
||||
if os.path.exists(path):
|
||||
os.remove(path)
|
||||
for m in reversed(made):
|
||||
if os.path.isdir(m) and not os.listdir(m):
|
||||
os.rmdir(m)
|
||||
return _clean
|
||||
return _plant
|
||||
|
||||
|
||||
def append_to(path, extra):
|
||||
def _plant():
|
||||
backup = io.open(path, encoding="utf-8").read()
|
||||
io.open(path, "w", encoding="utf-8").write(backup + extra)
|
||||
return lambda: io.open(path, "w", encoding="utf-8").write(backup)
|
||||
return _plant
|
||||
|
||||
|
||||
T = os.path.join(ROOT, "hub", "internal", "web", "templates")
|
||||
M = os.path.join(ROOT, "manifests")
|
||||
REP = os.path.join(ROOT, "REPORT.md")
|
||||
|
||||
print("decoys — felhom.eu")
|
||||
|
||||
# --- hub-confirm: a native confirm() one directory down (R-421) -------------------------------
|
||||
# Shape 1, name-for-fact: the gate used os.listdir, so its SCOPE was a directory listing rather
|
||||
# than the set of templates. There are no subdirectories today; adding templates/partials/ is an
|
||||
# ordinary act and the gate would have stayed green.
|
||||
decoy("hub-confirm/subdir", "hub_confirm_gate.py",
|
||||
plant_file(os.path.join(T, "partials", "decoy.html"),
|
||||
u'<button onclick="confirm(\'biztos?\')">x</button>\n'))
|
||||
|
||||
# --- manifest-bearer: a bearer literal one directory down --------------------------------------
|
||||
decoy("manifest-bearer/subdir", "manifest_bearer_gate.py",
|
||||
plant_file(os.path.join(M, "overlays", "decoy.yaml"),
|
||||
u"apiVersion: v1\ndata:\n token: %s\n" % ("a1b2c3d4" * 8)))
|
||||
|
||||
# --- observations: R-419 itself, and the genuine markers beside it -----------------------------
|
||||
# Shape 2, substring-for-field. THE decoy that found this class: an honest note SAYING it has no
|
||||
# marker satisfied the marker test.
|
||||
decoy("observations/R-419", "observations_gate.py",
|
||||
append_to(REP, u"\n## Observations\n\n1. **A real finding.** It carries no `FILED:` marker "
|
||||
u"and no `NOT-A-FINDING:` marker, deliberately.\n"), args=(ROOT,))
|
||||
decoy("observations/genuine-FILED", "observations_gate.py",
|
||||
append_to(REP, u"\n## Observations\n\n1. **A real finding.** Something broke. "
|
||||
u"**FILED: R-419**\n"), args=(ROOT,), expect="accept")
|
||||
decoy("observations/genuine-NAF", "observations_gate.py",
|
||||
append_to(REP, u"\n## Observations\n\n1. **A real finding.** Odd. **NOT-A-FINDING: my own "
|
||||
u"typo, corrected in the same minute.**\n"), args=(ROOT,), expect="accept")
|
||||
|
||||
# --- reuse-refs: a cited path that does not exist ----------------------------------------------
|
||||
# The .go case is REJECTED. The .md case is a KNOWN HOLE (R-422) and is asserted as such below, so
|
||||
# this file records the hole rather than pretending it is covered.
|
||||
decoy("reuse-refs/missing-go", "reuse_refs_check.py",
|
||||
append_to(os.path.join(ROOT, "REUSE.md"),
|
||||
u"\n- see `hub/internal/api/does_not_exist.go`\n"), args=(ROOT,))
|
||||
|
||||
# --- KNOWN HOLE, asserted so it cannot be forgotten (R-422) ------------------------------------
|
||||
# reuse_refs_check.py's PATH_RE matches only go|py|html|css|yml|yaml|sh. A rotted .md citation is
|
||||
# invisible. This asserts the CURRENT behaviour so the day it is fixed, this test fails and is
|
||||
# updated deliberately — a hole that nothing asserts is a hole nobody remembers.
|
||||
decoy("reuse-refs/missing-md (KNOWN HOLE R-422)", "reuse_refs_check.py",
|
||||
append_to(os.path.join(ROOT, "REUSE.md"),
|
||||
u"\n- see `documentation/architecture/99-does-not-exist.md`\n"),
|
||||
args=(ROOT,), expect="accept")
|
||||
|
||||
# --- golden-currency: R-410's own decoy, re-run here so the sweep owns it too ------------------
|
||||
def _mkdir_decoy():
|
||||
d = os.path.join(ROOT, "documentation", "tests", "golden-9.9.9-2026-01-01")
|
||||
os.makedirs(d)
|
||||
return lambda: os.path.isdir(d) and os.rmdir(d)
|
||||
|
||||
|
||||
def check_golden_names_the_fake():
|
||||
"""golden-currency exits 0 either way when currency is fine — the QUESTION is what it counted."""
|
||||
global ran
|
||||
ran += 1
|
||||
cleanup = _mkdir_decoy()
|
||||
try:
|
||||
_rc, out = gate("golden_currency_gate.py")
|
||||
finally:
|
||||
cleanup()
|
||||
if "newest golden baked : 9.9.9" in out:
|
||||
fails.append("golden-currency: an EMPTY directory was counted as a bake — R-410 has regressed")
|
||||
elif "NOT counted as bakes" not in out:
|
||||
fails.append("golden-currency: the empty directory was neither counted nor REPORTED; a "
|
||||
"half-finished bake must be visible, not silently ignored")
|
||||
else:
|
||||
print(" ok %-20s empty dir rejected AND named" % "golden-currency")
|
||||
|
||||
|
||||
check_golden_names_the_fake()
|
||||
|
||||
# --- closed-register: the verdict cell is the predicate, deliberately (R-378) -------------------
|
||||
# NOT a hole: R-378's whole lesson is that an open word ANYWHERE in a row convicts rows that are
|
||||
# genuinely closed. This asserts the deliberate behaviour so a future "fix" has to argue with it.
|
||||
decoy("closed-register/body-word (BY DESIGN)", "closed_register_gate.py",
|
||||
append_to(os.path.join(ROOT, "documentation", "backlog", "CLOSED-ITEMS.md"),
|
||||
u"\n| **R-903** | Work continues and it is still READY in the body. | CLOSED 2026-09-01 | none |\n"),
|
||||
expect="accept")
|
||||
# A 4-column row (| ID | Title | Shipped | Evidence |) whose VERDICT cell reads open.
|
||||
decoy("closed-register/verdict-word", "closed_register_gate.py",
|
||||
append_to(os.path.join(ROOT, "documentation", "backlog", "CLOSED-ITEMS.md"),
|
||||
u"\n| **R-904** | A finished thing. | READY - still being worked on | none |\n"))
|
||||
|
||||
# R-421: a row this gate cannot PARSE used to be a warning, and the gate then printed OK. Four rows
|
||||
# were in that state — two of them written by the session that closed them the day before this
|
||||
# sweep — so they were exempt from the only check that reads this file. An unreadable row is now a
|
||||
# conviction. This is the sweep's own shape one level up and it is why the decoy is kept.
|
||||
decoy("closed-register/unreadable-row", "closed_register_gate.py",
|
||||
append_to(os.path.join(ROOT, "documentation", "backlog", "CLOSED-ITEMS.md"),
|
||||
u"\n| **R-905** | A row with no state cell at all. |\n"))
|
||||
|
||||
# --- decoy-coverage: the meta-gate's own red-proof, kept as a test --------------------------------
|
||||
# It must convict a gate registered in a runner with no decoy and no exemption. Without this the
|
||||
# meta-gate is itself an unchecked instrument, which is the joke this whole sweep exists to avoid.
|
||||
# It also convicted ITSELF the moment it was registered, which is how this decoy came to be written.
|
||||
ran += 1
|
||||
_RUNNER = os.path.join(ROOT, "scripts", "repo_gates.py")
|
||||
_b = io.open(_RUNNER, encoding="utf-8").read()
|
||||
_anchor = ' ("observations", os.path.join(SCRIPTS, "observations_gate.py"), [ROOT], True, False),'
|
||||
try:
|
||||
assert _anchor in _b, "the runner's shape changed — this decoy can no longer be built"
|
||||
io.open(_RUNNER, "w", encoding="utf-8").write(_b.replace(
|
||||
_anchor, _anchor + '\n ("decoy-red-proof", os.path.join(SCRIPTS, "nope.py"), [], True, False),', 1))
|
||||
_rc, _out = gate("decoy_coverage_gate.py", (ROOT,))
|
||||
finally:
|
||||
io.open(_RUNNER, "w", encoding="utf-8").write(_b)
|
||||
if _rc == 0:
|
||||
fails.append("decoy-coverage: a NEW gate with no decoy and no exemption was ACCEPTED — the "
|
||||
"meta-gate cannot see the thing it exists for\n%s" % _out[-500:])
|
||||
elif "decoy-red-proof" not in _out:
|
||||
fails.append("decoy-coverage: it convicted, but did not NAME the uncovered gate")
|
||||
else:
|
||||
print(" ok %-20s a new gate with no decoy is convicted BY NAME" % "decoy-coverage")
|
||||
|
||||
print()
|
||||
if fails:
|
||||
for f in fails:
|
||||
print("FAIL: %s" % f)
|
||||
print("\n%d decoy(s) of %d exposed a hole" % (len(fails), ran))
|
||||
sys.exit(1)
|
||||
print("all %d felhom.eu decoys behaved — labels do not satisfy these gates" % ran)
|
||||
Reference in New Issue
Block a user