felhom.eu/scripts/closed_register_gate.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""closed_register_gate.py — CLOSED-ITEMS.md holds closed work ONLY (R-405, 2026-08-31).

WHAT IT CONVICTS ON (a FAIL, exit 1), two rules and nothing else:

  RULE 1 — no row in `CLOSED-ITEMS.md` may carry an OPEN state word (READY, OPEN, BLOCKED,
           WATCHING, WAITING-ON-OPERATOR) as the LEADING VERDICT of its state cell.
  RULE 2 — no `R-` identifier may have a table row in BOTH `CLOSED-ITEMS.md` and `OPEN-ITEMS.md`.

WHY IT EXISTS, and the cost that bought it. The 2026-08-22 compression sweep (commit `ef6ac6f`,
R-376..R-378) moved finished work out of `OPEN-ITEMS.md`. Its classifier matched a status word
ANYWHERE in the row, so rows that were not closed went with it. R-378 caught six of them in the same
session — R-123, R-190, R-214, R-264, R-295, R-352 — and restored them verbatim. **It missed a
seventh.** R-87 ("the restic tier is never restore-tested") went into the closed file with its own
state cell reading `READY — RE-RANKED UP 2026-08-03 (R-86 closed)`: a LEADING verdict of `READY`, and
the word `closed` later in the same cell describing a DIFFERENT row. It sat in the wrong file for
nine days while `OPEN-ITEMS.md`'s own ranking paragraph ranked it fourth and pointed at nothing.

READ THE LEADING VERDICT, NOT THE CELL, NOT THE ROW. This is R-378's whole lesson and this gate would
be wrong without it. Measured on `CLOSED-ITEMS.md` at 2026-08-31, matching an open word anywhere in
the state cell convicts THREE rows; two of them — R-224 and R-260 — are genuinely closed and merely
contain the words "open" and "OPEN" inside long prose verdicts. Matching the leading verdict convicts
exactly ONE, which is the real one. Matching the whole ROW convicts 144 of 151.

WHAT THIS GATE CANNOT SEE — the residual holes, named rather than implied:

  1. **A row whose body contains a `|` shifts its own cells and is read in the wrong place.** Two
     rows do this today (R-309, R-351: a pipe inside inline code). The gate cannot tell a shifted
     cell from a real one, so a mis-filed row that also contains a pipe escapes. It prints such rows
     as a WARNING when the leading verdict is not a verdict word AND an open word sits somewhere in
     the cell, which is the best signal available without a real Markdown parser. A leading verdict
     that is merely a version string is NOT warned on: this table's `Shipped` column legitimately
     holds `controller v0.227.0`, and ten warnings per run is how a gate gets ignored.
  2. **A row with no state cell at all escapes.** Two exist today (R-399, R-400 — two columns where
     the table declares four). An empty state cell is not an open word, so RULE 1 passes it. They are
     printed as a WARNING.
  3. **A closed-sounding verdict that is not true escapes.** `PARTLY CLOSED` leads with no open word.
     This gate checks where a row FILED, never whether the verdict is honest.
  4. **A duplicate id WITHIN one register escapes.** `OPEN-ITEMS.md` carries two unrelated findings
     both numbered R-133 (filed as R-406). Adding that rule would fail the gate on a pre-existing
     defect, and a registered-but-failing gate refuses every push, so it was deliberately left out.
  5. Nothing here reads audits, spikes or inventories. A finding that never reaches either register
     is invisible to this gate, as it is to `one_register_gate.py`.

Run:  python3 scripts/closed_register_gate.py [--fast]
"""
import io
import os
import re
import sys

ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CLOSED = os.path.join(ROOT, "documentation", "backlog", "CLOSED-ITEMS.md")
OPEN = os.path.join(ROOT, "documentation", "backlog", "OPEN-ITEMS.md")

# The id keeps its suffix letter: R-88a and R-88b are different rows, and R-209/R-209a are a closed
# row and its open successor. Dropping the letter invents duplicates that are not there.
ROW = re.compile(r"^\|\s*\*{0,2}(R-\d+[a-z]?)\b")

# A verdict ends at the first separator. `READY — RE-RANKED UP (R-86 closed)` leads with READY;
# `CLOSED 2026-08-06 — ... did not open the sealed bundle` leads with CLOSED.
SEPARATOR = re.compile(u"[—–(:,.]|\\s-\\s")
OPEN_WORD = re.compile(r"\b(READY|OPEN|BLOCKED|WATCHING|WAITING-ON-OPERATOR)\b", re.I)
DONE_WORD = re.compile(r"\b(CLOSED|SHIPPED|FIXED|KILLED|DONE|SUPERSEDED|OBSOLETE|WITHDRAWN|MERGED|"
                       r"DISCHARGED|RULED|MOVED|BANKED|PROVEN-LIVE|EXECUTED)\b", re.I)


def leading_verdict(cell):
    """The verdict word(s) before the first separator, with Markdown emphasis stripped."""
    text = cell.replace("*", "").replace("~", "").replace("`", "").strip()
    return SEPARATOR.split(text)[0].strip()


def rows(path):
    """Yield (line_no, id, state_cell_or_None, line) for every R- row in a pipe table.

    CLOSED-ITEMS.md declares `| ID | Title | Shipped | Evidence |` — four columns, so a well-formed
    row splits into six fields and the state cell is the third. A row of any other width has no
    state cell this function is willing to guess at, and says so with None.
    """
    for n, line in enumerate(io.open(path, encoding="utf-8"), 1):
        line = line.rstrip("\n")
        m = ROW.match(line)
        if not m:
            continue
        cells = line.split("|")
        state = cells[3] if len(cells) == 6 else None
        yield n, m.group(1), state, line


def main():
    for path in (CLOSED, OPEN):
        if not os.path.exists(path):
            print("closed-register gate: %s is missing — FAILURE, never a skip" % path)
            return 1

    convicted, warnings, checked = [], [], 0
    for n, rid, state, line in rows(CLOSED):
        if state is None:
            # R-421 (2026-09-01): A ROW THIS GATE CANNOT READ IS A CONVICTION, NOT A WARNING.
            #
            # This was a warning, and the gate then printed OK. Four rows were in that state —
            # R-399 and R-400 for days, and R-404 and R-417 written malformed by the session that
            # closed them the day before this sweep. **Every one of them was exempt from the only
            # check that reads this file**, and the gate said OK each time. That is the sweep's own
            # shape one level up: an instrument that reports a pass over rows it never examined.
            #
            # The rows were repaired first and the conviction added second — registering a failing
            # gate refuses every push, which is the ordering instructions_gate learned the hard way.
            convictions.append((n, rid, "row is not four columns, so it has NO STATE CELL and this "
                                        "gate cannot judge it — an unreadable row is never a pass"))
            continue
        checked += 1
        verdict = leading_verdict(state)
        if OPEN_WORD.search(verdict):
            convicted.append((n, rid, verdict, "open state word in the leading verdict"))
        elif not DONE_WORD.search(verdict) and OPEN_WORD.search(state):
            # Ambiguous and worth a human look: the leading verdict is not a verdict word at all
            # (so the cells may be shifted by a `|` in the body) AND an open word sits somewhere in
            # the cell. Deliberately NOT warned on a leading verdict that is merely a version
            # string — this table's `Shipped` column legitimately holds `controller v0.227.0`, and
            # ten such warnings per run is how a gate teaches people to stop reading it.
            warnings.append((n, rid, "leading verdict %r is not a verdict word and an open word "
                                     "sits in the cell — the row's cells may be shifted by a `|` "
                                     "in its body" % verdict[:60]))

    closed_ids = set(rid for _, rid, _, _ in rows(CLOSED))
    open_ids = {}
    for n, rid, _, _ in rows(OPEN):
        open_ids.setdefault(rid, n)
    for rid in sorted(closed_ids & set(open_ids), key=lambda r: (int(re.sub(r"\D", "", r)), r)):
        convicted.append((open_ids[rid], rid, "", "has a row in BOTH registers"))

    print("closed-register gate — %d closed rows with a readable state cell, %d open rows, "
          "%d convicted, %d warnings" % (checked, len(open_ids), len(convicted), len(warnings)))

    for n, rid, why in warnings:
        print("   WARNING  L%-5d %-7s %s" % (n, rid, why))

    if convicted:
        print()
        print("CONVICTED — CLOSED-ITEMS.md is for closed work only:")
        for n, rid, verdict, why in convicted:
            print("   L%-5d %-7s %s%s" % (n, rid, why, (" — verdict %r" % verdict) if verdict else ""))
        print()
        print("Move the row back into OPEN-ITEMS.md keeping its text, state, owner and rank, or —")
        print("if it really is finished — write the verdict that says so at the START of its state")
        print("cell. R-87 sat in the wrong file for nine days because nobody could see it there.")
        return 1

    print("closed-register gate OK — no open work filed as closed, no id in both registers.")
    return 0


sys.exit(main())