R-885: script-tests gate runs every scripts/**/test_*.py on every push (decoys, red-proofs); closed (336 -> 335)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 16:48:12 +02:00
parent b018ca9092
commit ab2b304987
7 changed files with 203 additions and 1 deletions
@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""script_tests_gate.py — every Python test suite under scripts/ runs on every push (R-885).
+
+Usage:  python3 scripts/script_tests_gate.py [<repo-root>]
+Exit 0 every suite exited 0 · 1 a suite failed, timed out, or NO suite was found.
+
+WHY. `repo_gates.py` ran gates, not the `test_*.py` suites beside them, so a change that broke one — the 15 tests of
+the DooPlex hub-DB push/restore scripts, the 73 of the instructions gate — was caught only if someone ran it by hand.
+
+SCOPE IS A FACT (R-421): the suites are FOUND by walking scripts/ (os.walk, every depth), never listed by hand, so a new
+suite is covered the day it is added. Finding none is a FAILURE — a gate that checked nothing must not read as green.
+A suite's verdict is its EXIT CODE, never its output: a suite that prints "OK" and exits 1 has failed.
+
+NESTING. `test_repo_gates.py` runs `repo_gates.py`, which runs this gate, which would run `test_repo_gates.py` again.
+The gate marks its children with FELHOM_SCRIPT_TESTS_GATE=1 and, when it finds the mark, steps aside (exit 0, said
+out loud). Pinned by scripts/test_script_tests_gate.py (decoys).
+"""
+import os
+import subprocess
+import sys
+import time
+
+MARK = "FELHOM_SCRIPT_TESTS_GATE"
+TIMEOUT_S = 300
+SKIP_DIRS = {"__pycache__", ".git", "node_modules"}
+
+
+def find_suites(scripts_dir):
+    out = []
+    for dp, dns, fns in os.walk(scripts_dir):
+        dns[:] = sorted(d for d in dns if d not in SKIP_DIRS)
+        for f in sorted(fns):
+            if f.startswith("test_") and f.endswith(".py"):
+                out.append(os.path.join(dp, f))
+    return out
+
+
+def main(argv):
+    if os.environ.get(MARK):
+        print("script-tests: nested run (inside a suite this gate started) — stepping aside")
+        return 0
+    root = os.path.abspath(argv[1]) if len(argv) > 1 else os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+    suites = find_suites(os.path.join(root, "scripts"))
+    if not suites:
+        print("script-tests: FAILED — no test_*.py found under %s/scripts (a gate that checked nothing is not green)" % root)
+        return 1
+    env = dict(os.environ, **{MARK: "1"})
+    failed = []
+    for s in suites:
+        rel = os.path.relpath(s, root)
+        t0 = time.time()
+        try:
+            p = subprocess.run([sys.executable, "-W", "ignore", s], cwd=root, env=env,
+                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=TIMEOUT_S)
+            rc, tail = p.returncode, p.stdout.decode("utf-8", "replace").strip().splitlines()[-3:]
+        except subprocess.TimeoutExpired:
+            rc, tail = "timeout", ["(killed after %d s)" % TIMEOUT_S]
+        verdict = "OK  " if rc == 0 else "FAIL"
+        print("  %s %-55s %5.1fs rc=%s" % (verdict, rel, time.time() - t0, rc))
+        if rc != 0:
+            failed.append(rel)
+            for line in tail:
+                print("        | " + line[:200])
+    if failed:
+        print("script-tests: FAILED — %d of %d suite(s): %s" % (len(failed), len(suites), ", ".join(failed)))
+        return 1
+    print("script-tests: OK — %d suite(s), every one exited 0" % len(suites))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))