felhom.eu/scripts/register_shape_gate.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""register_shape_gate.py — OPEN-ITEMS.md is a TABLE, and a table has a shape (R-627, 2026-09-22).

WHAT IT CONVICTS ON (a FAIL, exit 1), seven rules (RULE 2 is reserved) and nothing else:

  RULE 1 — every register row (`| **R-nnn** | …`) ends with `|`, so it HAS a state cell.
  RULE 2 — reserved. **There is deliberately no cell-COUNT rule, and that is a measurement, not an
           omission.** The first draft counted cells and convicted 125 innocent rows: register cells
           carry literal `|` characters inside their prose and their shell snippets (`owner: CC |
           …`, `--format '{{.Names}}|{{.Image}}'`), so splitting a row on `|` does not yield its
           cells. A count that cannot be computed is not a check. RULE 1 catches the thing that
           actually breaks — a row whose state cell was eaten — because such a row stops ending
           with a pipe.
  RULE 3 — no `R-` identifier appears on more than one row.
  RULE 4 — no blank line sits between two register rows. A blank line ENDS a markdown table, so
           the rows after it render as a separate table — and, worse, a script that walks "the
           table" stops there.

  RULE 5 — (2026-10-03) every register row sits under a table header and splits — on pipes OUTSIDE
           backticks, `register_table.split_cells` — into exactly as many cells as that header declares.
           RULE 2's measurement still stands for a naive split; a backtick-aware split CAN count, and on
           2026-10-03 it did for 437 of 442 rows — the five it could not were real defects (a stray extra
           cell, rows written five columns wide), repaired before this rule was registered.
  RULE 6 — (2026-10-03) the `Category` cell is EXACTLY one of the eleven names in `CATEGORIES`.
  RULE 7 — (2026-10-03) the `Sev` cell is exactly `P1`, `P2`, `P3` or `P4`.
  RULE 8 — (2026-10-03) the `State` cell LEADS with one of `register_table.OPEN_STATES`. A finished word
           there is `closed_register_gate.py` RULE 3's conviction; any OTHER word is convicted here,
           because a state nobody defined ("PARTLY CLOSED", "PLUMBING COMPLETE") is how 18 rows became
           unreadable to every script before the 2026-10-03 triage.

WHY RULES 5–8 (2026-10-03, operator request). Every open row now carries one category and one severity
so the register can be read as a list of priorities, and a new row is filed into its category from the
start. A column that a script cannot read reliably is not a column — so the shape is enforced, by the
header's column NAMES, not by positions.

WHY IT EXISTS, and the cost that bought it. On 2026-09-21 the update night appended measured
results to eight existing rows with a regex that matched a row's trailing state cell. On two rows
(**R-446** and **R-458**) it consumed the state cell and did not put it back; the cell reappeared as
a stray FOURTH cell on a duplicated copy of a different row (**R-626** and **R-625**), and a blank
line was left between each pair. The register then reported **317 rows for 315 findings**, two rows
carried no state at all, and two findings existed twice with contradictory state cells.

**Nothing caught it.** `one_register_gate.py` compares OPEN-ITEMS against ROADMAP; `closed_register_gate.py`
forbids an id in BOTH OPEN and CLOSED — neither asks whether the file is a well-formed table, and
neither notices an id duplicated WITHIN OPEN-ITEMS. It was found the next morning by a person
reading the file. **A register that silently mis-states how many findings exist is the one file in
this project that must not be able to do that**, because every standing rule says "grep the register
before minting" and every count in every report is read off it.

WHAT THIS GATE CANNOT SEE — the residual holes, named rather than implied:

  1. **A row whose state cell is WRONG but present.** Shape is not meaning. A row that says READY
     when it is closed passes here and is `closed_register_gate.py`'s business.
  2. **A row whose text was truncated mid-sentence** but still ends with a valid state cell. Only a
     human or a diff against the previous commit sees that.
  3. **The DUE-CHECKS block and the other tables in this file.** This gate reads register rows only
     — lines that begin `| **R-`. The file's other tables are `due_checks_gate.py`'s business.

USAGE
  python3 scripts/register_shape_gate.py            # the real register
  python3 scripts/register_shape_gate.py <file>     # any file, for the red-proof
Exit: 0 clean · 1 convicted.
"""
import os
import re
import sys
from pathlib import Path

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import register_table  # noqa: E402 — the shared row reader (2026-10-03)

# A gate's own console must not decide its verdict. These scripts quote back Hungarian copy, file
# paths and arrow characters; a Windows console here is cp1250, and printing one of them raised
# UnicodeEncodeError *before the gate had decided anything* — reuse_refs_check.py died while printing
# a NOTE, which the runner then reported as a failure (2026-10-10). Same trap class as
# poster_facts_gate.py's, which was found by its own red-proof.
for _stream in (sys.stdout, sys.stderr):
    try:
        _stream.reconfigure(encoding="utf-8", errors="replace")
    except (AttributeError, ValueError, OSError):  # pragma: no cover - old Python, or a pipe
        pass


# The eleven categories (operator request 2026-10-03; the reviewer's default, operator may reverse).
# The same list, in the same order, heads OPEN-ITEMS.md's "How a row is filed" section.
CATEGORIES = (
    "Install & onboarding", "Apps & catalog", "App updates", "Backup & restore", "Storage & devices",
    "Security & access", "Box system & updates", "Monitoring & notifications", "Hub & operator",
    "Business & legal", "Process & tooling",
)
SEVERITIES = ("P1", "P2", "P3", "P4")
REQUIRED_COLUMNS = ("ID", "Category", "Sev", "What", "State", "Owner")

ROW = re.compile(r'^\| \*\*(R-\d+[a-z]?)\*\* \|')  # 2026-09-30: a letter suffix (R-88a, R-88b, R-209a) is a row too — it was skipped, 3 rows unchecked
REG = Path(__file__).resolve().parent.parent / "documentation" / "backlog" / "OPEN-ITEMS.md"


def cells(line):
    """The cells of a markdown table row, as a reader sees them.

    A row is `| a | b | c |`. Stripping the outer pipes and splitting on `|` gives the cells; a row
    that forgot its trailing pipe yields one fewer, which is exactly RULE 2's symptom, so RULE 1 and
    RULE 2 are reported separately rather than collapsed into one confusing message.
    """
    t = line.rstrip()
    if not t.endswith("|"):
        return None
    return t.strip("|").split("|")


SEP = re.compile(r'^\|\s*:?-{2,}:?\s*(\|\s*:?-{2,}:?\s*)*\|\s*$')


def check(path):
    lines = Path(path).read_text(encoding="utf-8").split("\n")
    bad = []
    seen = {}
    rows = 0

    for i, line in enumerate(lines, start=1):
        m = ROW.match(line)
        if not m:
            continue
        rows += 1
        rid = m.group(1)

        # RULE 3 first, and ALWAYS — a row that also breaks another rule is still a row, and
        # skipping it here would under-count the ids and report phantom duplicates. (Measured: the
        # first draft skipped them and claimed 5 duplicates where there were 2.)
        if rid in seen:
            bad.append((i, rid, f"RULE 3 — duplicate: {rid} already has a row at line {seen[rid]}"))
        else:
            seen[rid] = i

        # RULE 1 — the row must end with `|`, i.e. it must still HAVE a state cell
        if not line.rstrip().endswith("|"):
            bad.append((i, rid, "RULE 1 — the row does not end with `|`, so it has no state cell. "
                                "This is exactly what an eaten state cell looks like, and what "
                                "R-446, R-458 (2026-09-21) and R-254 (2026-08-08, unnoticed for 45 "
                                "days) each looked like."))

        # RULE 4 — look back over blanks to the previous non-blank line
        j = i - 2
        blanks = 0
        while j >= 0 and lines[j].strip() == "":
            blanks += 1
            j -= 1
        if blanks and j >= 0 and ROW.match(lines[j]):
            bad.append((i, rid, f"RULE 4 — {blanks} blank line(s) between this row and "
                                f"{ROW.match(lines[j]).group(1)} at line {j+1}. A blank line ends "
                                f"the markdown table."))

    # RULES 5–8 — by the table's own header (2026-10-03)
    for n, rid, columns, cells, _line in register_table.rows(path):
        if columns is None:
            bad.append((n, rid, "RULE 5 — the row sits under no `| ID | … |` table header, so no column "
                                "of it can be read by name"))
            continue
        missing = [c for c in REQUIRED_COLUMNS if c not in columns]
        if missing:
            bad.append((n, rid, "RULE 5 — its table header lacks the column(s) %s" % ", ".join(missing)))
            continue
        if cells is None:
            continue  # RULE 1 has already convicted it
        if len(cells) != len(columns):
            bad.append((n, rid, "RULE 5 — %d cells where the header declares %d. A literal `|` outside "
                                "backticks makes a cell; put it inside `code`." % (len(cells), len(columns))))
            continue
        cat = register_table.cell(columns, cells, "Category")
        if cat not in CATEGORIES:
            bad.append((n, rid, "RULE 6 — Category %r is not one of the eleven (%s)"
                                % (cat, " · ".join(CATEGORIES))))
        sev = register_table.cell(columns, cells, "Sev")
        if sev not in SEVERITIES:
            bad.append((n, rid, "RULE 7 — Sev %r is not P1, P2, P3 or P4" % sev))
        word = register_table.first_word(register_table.cell(columns, cells, "State") or "")
        if word not in register_table.OPEN_STATES and not register_table.CLOSED_FAMILY.match(word):
            bad.append((n, rid, "RULE 8 — the State leads with %r, which is not a defined state (%s)"
                                % (word, " · ".join(register_table.OPEN_STATES))))

    return rows, len(seen), bad


def main():
    path = sys.argv[1] if len(sys.argv) > 1 else REG
    rows, ids, bad = check(path)
    print(f"register-shape: {path}")
    print(f"register-shape: {rows} register row(s), {ids} distinct R- id(s)")
    if rows != ids:
        print(f"register-shape: {rows - ids} row(s) are DUPLICATES")
    if not bad:
        print("register-shape: OK — every row ends with a state cell, every id is unique, no blank "
              "line splits the table, and every row has its columns, a known category, a P1–P4 "
              "severity and a defined state")
        return 0
    print(f"register-shape: CONVICTED — {len(bad)} problem(s)")
    for line_no, rid, why in bad:
        print(f"  line {line_no}  {rid}: {why}")
    return 1


if __name__ == "__main__":
    sys.exit(main())