ci(R-1297 A): heavy suites by path on a branch push, all on a tag and nightly; no second full gate run
gates / gates (push) Successful in 3m23s

- script_tests_gate.py: test_gate_decoys / test_due_checks_gate / dooplex-offsite run on a CI branch push only
  when one of their paths changed (FELHOM_CHANGED_FILES); no list, an empty or unreadable list = every suite
- gates.yml: nightly schedule 03:20 UTC; the list is set only for push:refs/heads/*; a trimmed commits array
  (total_commits > listed) is an unknown list
- test_repo_gates.py and test_due_checks_gate.py group G drive the real runner with GATES narrowed to their
  labels, instead of the whole gate set a second time (130 s each in CI)
- red-proved: filter cases, workflow pins (old gates.yml), inert run_gate, unregistered due-checks
This commit is contained in:
admin committed 2026-10-26 18:14:23 +01:00
1 parent 8143a2c872
commit ecd82abab8
5 files changed
+194 -10

No files matched your search

+54 -1
View File
@@ -45,6 +45,51 @@ def timeout_for(rel):
return SUITE_TIMEOUT_S.get(rel.replace(os.sep, "/"), TIMEOUT_S)
SKIP_DIRS = {"__pycache__", ".git", "node_modules"}
# R-1297 (A, 2026-10-26): THREE HEAVY SUITES RUN ON A CI PUSH ONLY WHEN ONE OF THEIR OWN PATHS CHANGED. Measured in CI run
# 2366: the decoy suite 232 s, the due-checks suite 133 s, the DooPlex off-site suite 69 s — of a 13-minute run, on every
# push, documents included. The decoys test the GATE SCRIPTS (they plant a fixture and expect a conviction), so a push
# that touches no script cannot change their verdict; the other two test their own scripts only.
#
# A RUN WITHOUT A LIST RUNS EVERY SUITE. CI sets FELHOM_CHANGED_FILES only for a branch push whose file list is known
# (gates.yml); a release tag, the nightly schedule, a manual dispatch and the pre-push hook set nothing, so they run all
# of them. An unreadable or empty list is unknown, and unknown runs everything (fail-closed). What the filter may cost:
# a document change that breaks a decoy's FIXTURE (not a gate) is seen by the nightly run, within a day — said here so
# nobody reads the filter as free. Pinned by scripts/test_script_tests_gate.py (FILTER_CASES).
CHANGED_ENV = "FELHOM_CHANGED_FILES"
PATH_FILTERED = {
"scripts/test_gate_decoys.py": ("scripts/", ".gitea/", ".githooks/"),
"scripts/test_due_checks_gate.py": ("scripts/due_checks_gate.py", "scripts/test_due_checks_gate.py",
"scripts/repo_gates.py", "scripts/script_tests_gate.py"),
"scripts/dooplex-offsite/test_dooplex_offsite.py": ("scripts/dooplex-offsite/",),
}
def changed_paths():
"""The pushed paths, or None when no usable list exists (then every suite runs)."""
p = os.environ.get(CHANGED_ENV, "")
if not p:
return None
try:
with open(p, encoding="utf-8") as fh:
paths = [l.strip() for l in fh if l.strip()]
except OSError as e:
print("script-tests: %s=%s unreadable (%s) — every suite runs" % (CHANGED_ENV, p, e))
return None
if not paths:
print("script-tests: %s=%s is empty — every suite runs" % (CHANGED_ENV, p))
return None
return paths
def skip_reason(rel, changed):
"""Why a suite is skipped on this push, or "" when it runs."""
prefixes = PATH_FILTERED.get(rel.replace(os.sep, "/"))
if changed is None or not prefixes:
return ""
if any(c.startswith(pfx) for c in changed for pfx in prefixes):
return ""
return "none of its paths (%s) changed in this push — it runs on a release tag and nightly" % ", ".join(prefixes)
def find_suites(scripts_dir):
out = []
@@ -66,9 +111,16 @@ def main(argv):
print("script-tests: FAILED — no test_*.py found under %s/scripts (a gate that checked nothing is not green)" % root)
return 1
env = dict(os.environ, **{MARK: "1"})
changed = changed_paths()
failed = []
skipped = 0
for s in suites:
rel = os.path.relpath(s, root)
why = skip_reason(rel, changed)
if why:
print(" SKIP %-55s %s" % (rel, why))
skipped += 1
continue
t0 = time.time()
limit = timeout_for(rel)
try:
@@ -86,7 +138,8 @@ def main(argv):
if failed:
print("script-tests: FAILED — %d of %d suite(s): %s" % (len(failed), len(suites), ", ".join(failed)))
return 1
print("script-tests: OK — %d suite(s), every one exited 0" % len(suites))
print("script-tests: OK — %d suite(s) ran, every one exited 0%s" % (
len(suites) - skipped, (", %d skipped by path (R-1297)" % skipped) if skipped else ""))
return 0
+15 -3
View File
@@ -199,9 +199,21 @@ def test_g_gate_is_registered_in_the_runner():
repo_gates.py's source would also match a commented-out line, which is precisely the
failure mode being guarded against.
"""
p = subprocess.run([sys.executable, RUNNER, "--fast"],
capture_output=True, text=True, cwd=ROOT)
out = p.stdout + p.stderr
# R-1297 (A, 2026-10-26): the SAME runner (`main`, the real run_gate), its GATES table narrowed to the
# entries labelled due-checks — taken from the real table, so a commented-out registration still leaves
# nothing to run and every check below fails. It used to run the whole gate set again (~130 s in CI).
import io
sys.path.insert(0, HERE)
import repo_gates
saved, real, buf = repo_gates.GATES, sys.stdout, io.StringIO()
try:
repo_gates.GATES = [g for g in saved if g[0] == "due-checks"]
sys.stdout = buf
repo_gates.main(["--fast"])
finally:
sys.stdout = real
repo_gates.GATES = saved
out = buf.getvalue()
check("G: runner names the due-checks gate in its output", "due-checks" in out)
check("G: runner reports a verdict line for it",
any(l.strip().startswith("due-checks") and ("OK" in l or "FAILED" in l
+28 -4
View File
@@ -11,7 +11,15 @@ runner's actual effect.
Red-proofed 2026-08-02: replacing run_gate's body with `return 0` (the inert runner) turns
test_every_member_gate_actually_ran red while the summary still prints "all felhom.eu gates OK".
R-1297 (A, 2026-10-26): IT TESTS THE WIRING, NOT THE WHOLE GATE SET A SECOND TIME. It used to run `repo_gates.py --fast`
in full — every gate again, 130 s of every CI run, inside a run that had just executed the same gates. Now it drives the
SAME runner (`main`, the real `run_gate`, the real scripts) with the GATES table narrowed to the five fingerprinted
gates — taken FROM the real table by label, so a gate commented out of it fails here as before. That every gate in the
table runs is the entry point's own run on every push; this test only proves the runner calls what it lists.
Red-proved again 2026-10-26 with `return 0, ""` as run_gate's body.
"""
import io
import os
import subprocess
import sys
@@ -34,10 +42,26 @@ class RepoGatesTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
p = subprocess.run([sys.executable, ENTRY, "--fast"], cwd=ROOT,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
cls.rc = p.returncode
cls.out = p.stdout.decode("utf-8", "replace")
sys.path.insert(0, os.path.join(ROOT, "scripts"))
import repo_gates
cls.repo_gates = repo_gates
labels = [l for l, _f in FINGERPRINTS]
cls.picked = [g for g in repo_gates.GATES if g[0] in labels]
real, buf = sys.stdout, io.StringIO()
saved = repo_gates.GATES
try:
repo_gates.GATES = cls.picked
sys.stdout = buf
cls.rc = repo_gates.main(["--fast"])
finally:
sys.stdout = real
repo_gates.GATES = saved
cls.out = buf.getvalue()
def test_every_fingerprinted_gate_is_registered(self):
got = sorted(g[0] for g in self.picked)
self.assertEqual(got, sorted(l for l, _f in FINGERPRINTS),
"a fingerprinted gate is missing from repo_gates.GATES")
def test_exit_code_is_zero(self):
self.assertEqual(self.rc, 0, self.out)
+74 -2
View File
@@ -16,7 +16,8 @@ GATE = os.path.join(HERE, "script_tests_gate.py")
COVERS = {
"script-tests": ("a suite that PRINTS OK and exits 1; a failing suite three levels deep; an empty scripts/ "
"tree; plus the genuine article (must pass) and the nested-run mark (must step aside)"),
"tree; plus the genuine article (must pass) and the nested-run mark (must step aside); R-1297: a "
"failing heavy suite on a push that touched its own path, with no list, an empty or a missing list"),
}
@@ -54,8 +55,79 @@ def timeouts_ok():
return (g.timeout_for("scripts/test_gate_decoys.py") == 600 and g.timeout_for("scripts/test_other.py") == g.TIMEOUT_S == 300)
# R-1297 (A): a heavy suite runs on a CI push only when one of ITS paths changed; with no list (a tag, the nightly run,
# the pre-push hook) or an unreadable or empty list, every suite runs. Each case plants a FAILING heavy suite, so
# „it ran" is rc 1 and „it was skipped" is rc 0 — the verdict, not a log line, says which happened.
CHANGED = "FELHOM_CHANGED_FILES"
def gate_changed(root, changed):
"""changed: None (variable unset), "MISSING" (points at no file), or a list of pushed paths."""
env = {k: v for k, v in os.environ.items() if k not in ("FELHOM_SCRIPT_TESTS_GATE", CHANGED)}
if changed == "MISSING":
env[CHANGED] = os.path.join(root, "no-such-list.txt")
elif changed is not None:
lst = os.path.join(root, "pushed.txt")
open(lst, "w").write("".join(f + "\n" for f in changed))
env[CHANGED] = lst
p = subprocess.run([sys.executable, GATE, root], env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
return p.returncode, p.stdout.decode("utf-8", "replace")
HEAVY = {"test_ok.py": "pass\n", "test_gate_decoys.py": "raise SystemExit(1)\n",
"dooplex-offsite/test_dooplex_offsite.py": "raise SystemExit(1)\n",
"test_due_checks_gate.py": "raise SystemExit(1)\n"}
FILTER_CASES = [
# (name, files, changed, want rc, text the output must hold)
("filter: no list -> every suite runs", HEAVY, None, 1, "test_gate_decoys.py"),
("filter: docs-only push skips the 3", HEAVY, ["documentation/x.md", "STATUS.md"], 0, "SKIP"),
("filter: a gate script runs decoys", {"test_ok.py": "pass\n", "test_gate_decoys.py": "raise SystemExit(1)\n"},
["scripts/site_gates.py"], 1, "test_gate_decoys.py"),
("filter: the due gate runs its suite", {"test_ok.py": "pass\n", "test_due_checks_gate.py": "raise SystemExit(1)\n"},
["scripts/due_checks_gate.py"], 1, "test_due_checks_gate.py"),
("filter: offsite path runs offsite", {"test_ok.py": "pass\n", "dooplex-offsite/test_dooplex_offsite.py": "raise SystemExit(1)\n"},
["scripts/dooplex-offsite/dooplex-offsite.sh"], 1, "test_dooplex_offsite.py"),
("filter: hub push skips offsite", {"test_ok.py": "pass\n", "dooplex-offsite/test_dooplex_offsite.py": "raise SystemExit(1)\n"},
["hub/internal/web/hosts.go"], 0, "SKIP"),
("filter: empty list -> every suite", HEAVY, [], 1, "test_gate_decoys.py"),
("filter: missing list -> every suite", HEAVY, "MISSING", 1, "test_gate_decoys.py"),
("filter: other suites never skipped", {"test_other.py": "raise SystemExit(1)\n"}, ["documentation/x.md"], 1, "test_other.py"),
]
def workflow_pins():
"""R-1297: the filter is safe only while the nightly run exists and only a branch push sets the list. Read from the
real workflow and hook, so removing the schedule, or setting the list on every event, turns this red."""
root = os.path.dirname(HERE)
wf = open(os.path.join(root, ".gitea", "workflows", "gates.yml"), encoding="utf-8").read()
hook = open(os.path.join(root, ".githooks", "pre-push"), encoding="utf-8").read()
problems = []
if "schedule:" not in wf or "- cron:" not in wf:
problems.append("gates.yml has no nightly schedule — the filtered suites would run only when touched")
arm = wf.find("push:refs/heads/*)")
sets = [i for i in range(len(wf)) if wf.startswith('FELHOM_CHANGED_FILES=', i)]
if not sets or arm < 0 or any(i < arm or i > wf.find(";;", arm) for i in sets):
problems.append("gates.yml sets FELHOM_CHANGED_FILES outside the branch-push arm")
if "total_commits" not in wf:
problems.append("gates.yml does not treat a trimmed commits array as an unknown list")
if "FELHOM_CHANGED_FILES" in hook:
problems.append("the pre-push hook sets the suite filter — the hook must run every suite")
for p in problems:
print("FAIL workflow pin: " + p)
if not problems:
print("ok workflow pins (nightly run, branch-push-only list, trimmed payload, hook unfiltered)")
return not problems
def main():
bad = 0
bad += not workflow_pins()
for name, files, changed, want, text in FILTER_CASES:
got, out = gate_changed(fake_repo(files), changed)
ok = got == want and text in out
bad += not ok
print("%s %-38s want rc=%d got rc=%d%s" % ("ok " if ok else "FAIL", name, want, got,
"" if text in out else " (no %r in output)" % text))
t_ok = timeouts_ok()
bad += not t_ok
print("%s %-32s" % ("ok " if t_ok else "FAIL", "per-suite time limits (R-1209)"))
@@ -64,7 +136,7 @@ def main():
ok = got == want
bad += not ok
print("%s %-32s want rc=%d got rc=%d" % ("ok " if ok else "FAIL", name, want, got))
print("script-tests decoys: %d/%d" % (len(CASES) - bad, len(CASES)))
print("script-tests decoys: %d failed of %d" % (bad, len(CASES) + len(FILTER_CASES) + 1))
return 1 if bad else 0