admin
5e8a82c3c4
gates / gates (push) Successful in 18s
New runbooks/nightly-rotation.md; observations_gate.py reads every section (R-471); target-selection.md names real paths (R-461); R-93 carries the fact that drill-r50 is gone. Register: R-473/R-474/R-466/R-471/R-453/R-461 and v0.240.0's R-477/R-478/R-480/R-482/R-484/R-485/R-486 closed; R-481, R-483, R-487, R-488, R-489 opened. 09 §6.1, 07 §6, CONTEXT, STATUS note. Evidence: audits/nightly-2026-09-13-adventurelog/, audits/v0240-2026-09-13/.
305 lines
15 KiB
Python
305 lines
15 KiB
Python
#!/usr/bin/env python3
|
|
# -*- coding: utf-8 -*-
|
|
"""Observations gate — an observation with no row behind it refuses the push.
|
|
|
|
Run from the repo root: python3 scripts/observations_gate.py [path/to/REPORT.md]
|
|
Exit 0 clean · 1 convicted (an observation is neither filed nor declared) · 2 inconclusive.
|
|
|
|
WHY THIS EXISTS, and what it cost to learn.
|
|
|
|
On 2026-08-23 a session measured, on live hardware, that **only the first broken app per hour reaches
|
|
the operator** — the notification cooldown keys on the event type and not on the app. It was real, it
|
|
was reproducible, and it was written in a numbered item under `## Observations` in `REPORT.md`. **It
|
|
was written nowhere else.** There was no register row. `REPORT.md` is overwritten every session by
|
|
this project's own convention, so the finding had a lifetime of exactly one session.
|
|
|
|
That is R-341's shape one surface over — a commitment recorded in prose that nothing enforces — and
|
|
it is why gate 10 exists. The same argument applies here and nothing was reading this section.
|
|
|
|
**The instruction invited it.** `documentation/PROMPT-TEMPLATE.md` §15 asked for "observations:
|
|
out-of-scope items noticed, documented, not acted on". "Documented" was satisfied by the paragraph.
|
|
The template was corrected in the same session that added this gate; **this gate is the mechanism
|
|
that correction points at**, because a rule without a mechanism is a wish.
|
|
|
|
── THE RULE ─────────────────────────────────────────────────────────────────────────────────
|
|
|
|
Every numbered item in a `REPORT.md` observations section must carry exactly one explicit marker:
|
|
|
|
* ``FILED: R-NNN`` — this observation is filed, as that row. The row MUST resolve in
|
|
`OPEN-ITEMS.md` or `CLOSED-ITEMS.md`.
|
|
* ``NOT-A-FINDING:`` — deliberately not filed, followed by a reason on the same item.
|
|
|
|
── WHY A MARKER AND NOT "MENTIONS AN R-NUMBER", WHICH IS WHAT WAS ASKED FOR ──────────────────
|
|
|
|
The specification for this gate said an item may "cite an R-NNN that resolves". **That rule would
|
|
have passed the exact item this gate was built to catch**, and the discrepancy is recorded here
|
|
rather than quietly resolved:
|
|
|
|
"This is R-182's known cooldown-key shape; it was harmless while app_start_failed was
|
|
undeliverable and is not any more. Not fixed here."
|
|
|
|
`R-182` resolves. It is cited as an **analogy** — the family the defect belongs to — not as the row
|
|
that files it. No parser can tell a citation-as-precedent from a citation-as-filing by reading prose,
|
|
and a gate that guesses would either miss this item or convict every item that mentions history.
|
|
|
|
So the marker is explicit and the burden is one token. **The cost is a format requirement on one
|
|
section of one file; the benefit is that the failure mode which produced this gate cannot recur
|
|
silently.** This is the same trade `due_checks_gate.py` made: dates moved out of prose and into a
|
|
machine-readable block, inside the register so no sidecar can drift.
|
|
|
|
── BOUNDARIES, stated so they are not silently re-decided ────────────────────────────────────
|
|
|
|
* **No observations section → PASS, quietly.** Most pushes do not touch `REPORT.md`, and a gate
|
|
that taxes every push is one that gets disabled within a week. Absence of the section is not
|
|
evidence of a hidden finding.
|
|
* **No `REPORT.md` at all → INCONCLUSIVE.** This gate is registered per-repo precisely because the
|
|
repo has one; if it has vanished, that is worth a word rather than a silent pass.
|
|
* **A section present but with no parseable items → INCONCLUSIVE, naming what it could not read.**
|
|
Fail closed on ambiguity in the RULE, fail open on ambiguity in the PARSE — but INCONCLUSIVE is
|
|
never a silent pass, and the runner reports 2 distinctly for exactly that reason.
|
|
* **REFUSES, does not warn.** Gate 10's reasoning applies unchanged: a warning is the thing that
|
|
gets scrolled past, and this repo has the census to prove it.
|
|
* **Both markers, or two of one, on a single item → conviction.** An item that is both filed and
|
|
declared-not-a-finding is not a parse problem, it is an undecided author.
|
|
* **A passing run still says what it looked at** (workspace standing rule 3): it prints the item
|
|
count and how each was satisfied, so a gate that silently examined nothing is visible.
|
|
|
|
Stdlib only; no network, no subprocess — `--fast`, so it runs in BOTH the pre-push hook and CI. A
|
|
non-fast gate would run in neither, which is the R-29 failure the runner ended.
|
|
"""
|
|
import os
|
|
import re
|
|
import sys
|
|
|
|
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
|
|
# `## 10. Observations — recorded, not acted on`, `### Observations`, `## Observations:` …
|
|
HEADING_RE = re.compile(r"^(#+)\s*(?:\d+[.)]\s*)?observations\b", re.IGNORECASE)
|
|
ITEM_RE = re.compile(r"^\s*(\d+)[.)]\s+(.*)$")
|
|
# ⚠ THE MARKER IS A MARKER, NOT A MENTION (R-419, fixed 2026-09-01).
|
|
#
|
|
# These were `\bFILED:` and `\bNOT-A-FINDING:` searched over the whole item body, so ANY occurrence
|
|
# counted — including one inside a sentence ABOUT the markers. Measured, and by accident: an
|
|
# observation reading *"it carries no `FILED:` and no `NOT-A-FINDING:` marker"* was reported
|
|
# `OK 1. NOT-A-FINDING` and a real push of mine went green over an unfiled finding. **The gate's
|
|
# whole job is to force an explicit choice, and a sentence disclaiming the choice counted as making
|
|
# it.** That is the substring-for-field class this project has now shipped five times.
|
|
#
|
|
# A marker must now be the START of a line (after optional list/emphasis punctuation), which is
|
|
# where a real marker is written and where a mention inside prose never is. `**FILED: R-417**` at
|
|
# the end of a sentence is the common real shape, so a marker is also accepted after a sentence
|
|
# boundary — but never mid-sentence and never inside backticks.
|
|
# Emphasis is NORMALISED AWAY before matching (see normalise_for_markers), so the anchor only has to
|
|
# describe where a marker sits in a SENTENCE: at the start of a line, or after a sentence ends. The
|
|
# common real shape is a bolded sentence followed by a bolded marker — `...them.** **FILED: R-427**` —
|
|
# and an anchor that did not allow the emphasis run between them rejected genuine markers. That was
|
|
# caught by this gate convicting the very report that documents it.
|
|
_MARK = r"(?:^|(?<=[.!?)]\s))\s*"
|
|
FILED_RE = re.compile(_MARK + r"FILED:\s*(R-\d+)", re.IGNORECASE | re.MULTILINE)
|
|
NOT_A_FINDING_RE = re.compile(_MARK + r"NOT-A-FINDING:\s*(\S.*)$", re.IGNORECASE | re.MULTILINE)
|
|
|
|
# A marker written inside backticks is being TALKED ABOUT, never used. Strip inline code spans
|
|
# before matching — this is what makes the R-419 decoy fail.
|
|
CODE_SPAN_RE = re.compile(r"`[^`]*`")
|
|
EMPHASIS_RE = re.compile(r"[*_]+")
|
|
|
|
|
|
def normalise_for_markers(body):
|
|
"""Strip inline code spans, then emphasis, before looking for a marker.
|
|
|
|
CODE SPANS GO FIRST AND THAT ORDER MATTERS: a marker inside backticks is being TALKED ABOUT,
|
|
never used, and removing it is what makes the R-419 decoy fail. Emphasis goes second so that
|
|
`**FILED: R-427**` and `FILED: R-427` are the same thing to the anchor below.
|
|
"""
|
|
return EMPHASIS_RE.sub("", CODE_SPAN_RE.sub("", body))
|
|
|
|
REGISTERS = [
|
|
os.path.join(ROOT, "documentation", "backlog", "OPEN-ITEMS.md"),
|
|
os.path.join(ROOT, "documentation", "backlog", "CLOSED-ITEMS.md"),
|
|
]
|
|
|
|
|
|
def find_registers():
|
|
"""Locate the registers. They live in felhom.eu; sibling repos reach across, as other gates do."""
|
|
found = [p for p in REGISTERS if os.path.isfile(p)]
|
|
if found:
|
|
return found
|
|
sibling = os.path.join(os.path.dirname(ROOT), "felhom.eu", "documentation", "backlog")
|
|
return [p for p in (os.path.join(sibling, "OPEN-ITEMS.md"),
|
|
os.path.join(sibling, "CLOSED-ITEMS.md")) if os.path.isfile(p)]
|
|
|
|
|
|
def known_rows(paths):
|
|
"""Every R-number that has a row in the registers."""
|
|
rows = set()
|
|
for p in paths:
|
|
with open(p, encoding="utf-8") as fh:
|
|
for line in fh:
|
|
m = re.match(r"^\|\s*\*\*(R-\d+)\*\*\s*\|", line)
|
|
if m:
|
|
rows.add(m.group(1).upper())
|
|
return rows
|
|
|
|
|
|
def observation_sections(text):
|
|
"""Every observations section in the file: [(heading_line, body_lines)], in document order.
|
|
|
|
R-471 (2026-09-13): this used to return the FIRST matching heading only, so a report carrying an
|
|
ordinary `## 11. Observations …` section and, appended below it, a second `## Observations` block
|
|
with an unmarked item PASSED — and the R-419 decoy, which plants exactly that shape, had been
|
|
reading LIVE HOLE at HEAD without anyone running the decoy suite. Every section is read now.
|
|
"""
|
|
lines = text.split("\n")
|
|
sections = []
|
|
i = 0
|
|
while i < len(lines):
|
|
m = HEADING_RE.match(lines[i])
|
|
if not m:
|
|
i += 1
|
|
continue
|
|
depth = len(m.group(1))
|
|
body = []
|
|
j = i + 1
|
|
while j < len(lines):
|
|
hm = re.match(r"^(#+)\s", lines[j])
|
|
if hm and len(hm.group(1)) <= depth:
|
|
break
|
|
body.append(lines[j])
|
|
j += 1
|
|
sections.append((lines[i].strip(), body))
|
|
i = j
|
|
return sections
|
|
|
|
|
|
def observation_items(text):
|
|
"""(items, heading) — each item is (number, its full text), over EVERY observations section.
|
|
heading is None when there is no section; several headings are joined with " + "."""
|
|
sections = observation_sections(text)
|
|
if not sections:
|
|
return None, None
|
|
|
|
items, cur = [], None
|
|
for _heading, body in sections:
|
|
for line in body:
|
|
m = ITEM_RE.match(line)
|
|
if m:
|
|
if cur:
|
|
items.append(cur)
|
|
cur = [m.group(1), m.group(2)]
|
|
elif cur is not None:
|
|
if line.strip() == "" and cur[1].endswith("\n\n"):
|
|
continue
|
|
cur[1] += "\n" + line
|
|
if cur:
|
|
items.append(cur)
|
|
cur = None
|
|
return items, " + ".join(h for h, _ in sections)
|
|
|
|
|
|
def main():
|
|
# The argument is a REPO ROOT (how the sibling runners invoke every shared gate) or, for tests
|
|
# and one-off checks, a REPORT.md path directly. Both are accepted so the controls in this
|
|
# session's evidence and the runner call the same code.
|
|
report = os.path.join(ROOT, "REPORT.md")
|
|
if len(sys.argv) > 1:
|
|
arg = sys.argv[1]
|
|
report = os.path.join(arg, "REPORT.md") if os.path.isdir(arg) else arg
|
|
|
|
if not os.path.isfile(report):
|
|
print("OBSERVATIONS GATE INCONCLUSIVE: no REPORT.md at %s" % report)
|
|
print(" This gate is registered here because this repo keeps one. If it was removed "
|
|
"deliberately, unregister the gate in the runner rather than leaving it unreadable.")
|
|
sys.exit(2)
|
|
|
|
with open(report, encoding="utf-8") as fh:
|
|
text = fh.read()
|
|
|
|
items, heading = observation_items(text)
|
|
|
|
if items is None:
|
|
print("observations gate OK — %s has no observations section (nothing to check)"
|
|
% os.path.basename(report))
|
|
sys.exit(0)
|
|
|
|
if not items:
|
|
print("OBSERVATIONS GATE INCONCLUSIVE: found the section %r but no numbered items under it."
|
|
% heading)
|
|
print(" Items must be a numbered list (`1.`, `2.` …). If the section is deliberately empty, "
|
|
"remove the heading — an empty section reads as coverage while providing none.")
|
|
sys.exit(2)
|
|
|
|
rows = known_rows(find_registers())
|
|
if not rows:
|
|
print("OBSERVATIONS GATE INCONCLUSIVE: could not read any register rows from "
|
|
"OPEN-ITEMS.md / CLOSED-ITEMS.md — cannot verify that a cited R-number resolves.")
|
|
sys.exit(2)
|
|
|
|
convictions = []
|
|
satisfied = []
|
|
for num, body in items:
|
|
body = normalise_for_markers(body) # R-419
|
|
filed = FILED_RE.findall(body)
|
|
declared = NOT_A_FINDING_RE.findall(body)
|
|
first_line = body.split("\n")[0].strip()
|
|
short = (first_line[:78] + "…") if len(first_line) > 78 else first_line
|
|
|
|
if filed and declared:
|
|
convictions.append((num, short,
|
|
"carries BOTH `FILED:` and `NOT-A-FINDING:` — decide which it is"))
|
|
continue
|
|
if len(filed) > 1:
|
|
convictions.append((num, short,
|
|
"carries %d `FILED:` markers (%s) — one observation, one row"
|
|
% (len(filed), ", ".join(filed))))
|
|
continue
|
|
if filed:
|
|
r = filed[0].upper()
|
|
if r not in rows:
|
|
convictions.append((num, short,
|
|
"`FILED: %s` does not resolve — no row for %s in OPEN-ITEMS.md "
|
|
"or CLOSED-ITEMS.md" % (r, r)))
|
|
else:
|
|
satisfied.append("%s. FILED %s" % (num, r))
|
|
continue
|
|
if declared:
|
|
reason = declared[0].strip()
|
|
if len(reason) < 12:
|
|
convictions.append((num, short,
|
|
"`NOT-A-FINDING:` carries no reason — the reason is the whole "
|
|
"point of the marker"))
|
|
else:
|
|
satisfied.append("%s. NOT-A-FINDING" % num)
|
|
continue
|
|
|
|
convictions.append((num, short, "neither `FILED: R-NNN` nor `NOT-A-FINDING: <reason>`"))
|
|
|
|
# Print the evidence unconditionally — a gate that only speaks when it fails teaches nobody what
|
|
# it is watching (workspace standing rule 3).
|
|
print(" report : %s" % report)
|
|
print(" section : %s" % heading)
|
|
print(" observation items : %d" % len(items))
|
|
for s in satisfied:
|
|
print(" OK %s" % s)
|
|
|
|
if convictions:
|
|
print("")
|
|
print("OBSERVATIONS GATE FAILED: %d observation(s) with nothing behind them." % len(convictions))
|
|
for num, short, why in convictions:
|
|
print("")
|
|
print(" item %s: %s" % (num, short))
|
|
print(" %s" % why)
|
|
print("")
|
|
print("An observation that lives only in REPORT.md has a lifetime of ONE SESSION — this file "
|
|
"is overwritten every time. That is how the cooldown-grain finding was lost on "
|
|
"2026-08-23 and had to be re-derived the next day.")
|
|
print("Fix: add `FILED: R-NNN` naming the row you opened for it, or `NOT-A-FINDING: <why "
|
|
"this is not worth a row>`. Opening the row is the default; declaring is the exception "
|
|
"and needs its reason stated.")
|
|
sys.exit(1)
|
|
|
|
print("observations gate OK — every observation is either filed or explicitly declared")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|