felhom.eu/scripts/golden_currency_gate.py

# -*- coding: utf-8 -*-
"""Golden-currency gate (R-242) — a controller release is not DELIVERED until a golden carries it.

Run from the repo root:  python3 scripts/golden_currency_gate.py
Exit 0 clean · 1 convicted (a released controller has no golden) · 2 inconclusive.

WHY THIS EXISTS, and why a rule was not enough.

R-242 was filed on 2026-08-07 as a mechanism-less rule: *a controller release that changes
customer-visible behaviour is not finished until a golden carries it, and nothing enforces that.*
It was deliberately recorded and not built. **It recurred the next day** — controller v0.206.0 shipped
the R-241 fixes while the vouched golden still carried 0.205.0, so a machine installed that morning
would have received neither. That is the second occurrence in two days (the first, R-239, went
unnoticed until a walk measured it from the customer's side), and it is what a rule without a
mechanism does.

The failure mode is FORGETTING, not lying — nobody ever decided to ship a stale golden. So this gate
is built to catch a missed step, and it is not, and does not pretend to be, an adversarial control.

⚠ WHAT IT CHECKS, AND WHAT IT DELIBERATELY DOES NOT — read this before trusting a green.

It checks that a golden has been **BAKED** for the newest released controller, by looking for that
version's bake-evidence directory in this repo. It does **NOT** check that the golden was **VOUCHED**,
because the vouched version lives ONLY in the hub's `hub_settings` table — there is no copy in git.

That limit is forced, not chosen, and the reasoning is recorded so nobody re-derives it:

  * Both the pre-push hook AND CI run `repo_gates.py --fast`, which by contract selects only gates
    that touch **no network**. A hub-reading gate could therefore be registered as non-fast and would
    then run in NEITHER place — a check that does not run where it applies is precisely the R-29
    census failure this repo's runner was built to end. A gate nobody runs is worse than no gate,
    because it reads as coverage.
  * Recording the vouched version in a tracked file instead would create a second source of truth
    that can drift from the hub, and a green gate over a false claim is the worst outcome available.

**So a bake without a vouch still passes this gate.** The bake is the step that happens in this repo
and is therefore the step this repo can see; the vouch is an operator act against the hub and needs a
different mechanism. That gap is real and is recorded as R-242's remaining half, NOT papered over
here. In practice the two are minutes apart in the same session, and the recurrence this gate is
built for was a missing BAKE. **That vouch half is STILL open after 2026-09-13** — the waiver below
covers a different thing, and this sentence is kept so the two are not confused.

WHY VERSION AND NOT BEHAVIOUR. It compares version numbers, so a controller release that changed
nothing a customer can see also trips it. That is accepted deliberately: deciding "customer-visible"
mechanically is not possible, judging it by hand is what already failed twice, and the cost of a
false trip is one bake — which is the operation the project wants to be routine anyway. **A gate that
cries wolf is one people learn to bypass, and `--no-verify` exists**, so the tolerance is stated
rather than assumed: if this ever fires on a release nobody wants a golden for, the honest fix is a
recorded waiver in the register, never a habit of bypassing. **That waiver is now BUILT — see
"THE WAIVER" below (2026-09-13).**

FAIL-CLOSED, BUT HONEST ABOUT NOT KNOWING. An absent controller clone, or a CHANGELOG whose top
header cannot be parsed, exits **2 (INCONCLUSIVE)** — never 0. The runner reports 2 distinctly for
exactly this reason: an undetermined result is not a pass, and it is not a conviction either.

── THE SECOND BLINDNESS, R-385 (added 2026-08-23) ────────────────────────────────────────────

Until this change the gate asked ONE question — *is the golden BEHIND the record?* — and so it could
only ever catch a forgotten bake. It said nothing when the golden was **AHEAD** of the record, and
that is not a harmless direction: a golden ahead of every CHANGELOG heading was built from something
**never written down**.

That is not a hypothetical. Controller **0.221.1** was built, baked AND vouched on 2026-08-23 while
the newest heading in the controller CHANGELOG still read v0.221.0 — the fix had been written inside
the v0.221.0 entry instead of getting its own. Every gate was green throughout, including this one,
measured: `newest released 0.221.0 / newest golden baked 0.221.1 → OK`. The fleet ran a version the
record did not name.

So the test is no longer "behind?" but "**is the version we are shipping WRITTEN DOWN?**". The gate
now looks for the baked version's own `## vX.Y.Z` heading anywhere in the CHANGELOG — not merely at
the top, because an entry may legitimately be overtaken by later ones; what may never happen is that
it is absent. An unrecorded golden is convicted (exit 1) exactly like a stale one.

**Why membership and not `baked > released`.** A comparison against the newest heading alone would go
green again the moment ANY later entry was written, leaving 0.221.1 permanently unrecorded and the
gate permanently silent about it. Membership cannot be satisfied by an unrelated later release.

── THE WAIVER — goldens on a CADENCE, not per release (added 2026-09-13, operator ruling) ────────

What happened between 2026-08-07 and 2026-09-01: **25 goldens in 26 days**, almost one per release,
because this gate trips on every release by design (see WHY VERSION AND NOT BEHAVIOUR) and the only
honest ways past it were a bake or a `--no-verify`. Thirteen bypasses were counted by 2026-09-01
(R-404/R-417). The operator ruled on 2026-09-13: **bake on a cadence — weekly, and always before any
drill or fresh install — not per release.** Every release still raises the FLOOR, so the fleet keeps
getting each release in ~20 s; only the golden, which protects a fresh install and nothing else,
moves to a cadence.

The mechanism is a small tracked file, `documentation/tests/golden-waiver.yml`:

    issued: 2026-09-13
    expires: 2026-09-27          # at most 14 days after issued, or the gate REFUSES the waiver
    reason: pre-customer development; goldens on a weekly cadence (operator ruling 2026-09-13)
    register_row: R-468          # must exist as a `**R-468**` row in OPEN-ITEMS.md

While the waiver is VALID, the "behind" conviction becomes a **loud ADVISORY** (exit 0) that names
the waiver, its expiry and how many releases the golden lags. When it runs out, the gate is red
again until someone bakes or renews. **A dated waiver cannot be forgotten — it just expires.** That
is what makes it different from R-242's original rule, which recurred the day after it was written:
renewal is a new commit with a diff, a deliberate act someone can see.

THE ASYMMETRY, stated because it is the whole design: **the waiver covers a golden that is BEHIND the
record. It never covers a golden that is UNRECORDED (R-385).** The first is a cadence choice; the
second is the fleet running something nobody wrote down, and no schedule makes that acceptable.

WHAT REFUSES THE WAIVER (exit 2, INCONCLUSIVE — never 0, never silently ignored): an expiry more
than 14 days after issue; an absent or unparseable date; a missing or empty reason; a register row
that is absent or whose `**R-n**` row does not exist. The 14-day cap is enforced HERE, not in the
runbook, because a cap in prose is the thing that failed. A file at the right path containing only
the word `expires` is the R-421 decoy and is refused like any other malformed waiver.

The register is read for exactly ONE fact — does the named row exist — never for meaning. Reading
prose for meaning is the R-421 class.

WHAT IT DOES NOT COVER: the vouch (still R-242's open half); an unrecorded golden (R-385); anything
after the first external install — the waiver's own row says it is a pre-customer arrangement.
"""
import datetime
import io
import os
import re
import sys

ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# The controller clone sits beside this one. The same sibling assumption reuse_refs_check.py and
# instructions_gate.py already make — an absent sibling is INCONCLUSIVE, never a silent pass.
#
# The GOLDEN_GATE_* environment variables are a TEST SEAM (2026-09-13): they move WHERE the gate
# reads, never WHAT it decides, so test_golden_currency_gate.py can build a "behind" or an
# "unrecorded" tree without depending on the real controller's release history. Unset in every real
# run — the hook and CI export nothing.
CONTROLLER_CHANGELOG = os.environ.get(
    "GOLDEN_GATE_CHANGELOG",
    os.path.join(os.path.dirname(ROOT), "felhom-controller", "CHANGELOG.md"))
EVIDENCE_DIR = os.environ.get("GOLDEN_GATE_EVIDENCE_DIR", os.path.join(ROOT, "documentation", "tests"))
WAIVER_PATH = os.environ.get("GOLDEN_GATE_WAIVER",
                             os.path.join(ROOT, "documentation", "tests", "golden-waiver.yml"))
REGISTER_PATH = os.environ.get("GOLDEN_GATE_REGISTER",
                               os.path.join(ROOT, "documentation", "backlog", "OPEN-ITEMS.md"))
# The HARD LIMIT on a waiver's life. Enforced here and nowhere else — see the module docstring.
WAIVER_MAX_DAYS = 14

# `## v0.206.0 — …` on the FIRST such line: the CHANGELOG is newest-first by convention.
RELEASED_RE = re.compile(r"^##\s+v(\d+)\.(\d+)\.(\d+)\b")
# `golden-0.205.0-2026-08-07/` — the bake-evidence directory the runbook's §4.1 produces.
EVIDENCE_RE = re.compile(r"^golden-(\d+)\.(\d+)\.(\d+)-\d{4}-\d{2}-\d{2}$")


def released_versions():
    """(newest_tuple, note, set_of_all_tuples) of the controller releases, or (None, reason, set()).

    R-385: the whole set is returned, not only the newest. The newest answers "is the golden behind?";
    membership answers "is the version we are shipping written down at all?" — and only the second
    question could have caught 0.221.1, whose heading was missing while a NEWER heading existed.
    """
    if not os.path.isfile(CONTROLLER_CHANGELOG):
        return None, "controller clone not found at %s" % CONTROLLER_CHANGELOG, set()
    newest = None
    note = ""
    every = set()
    with open(CONTROLLER_CHANGELOG, encoding="utf-8") as fh:
        for line in fh:
            m = RELEASED_RE.match(line)
            if m:
                v = tuple(int(g) for g in m.groups())
                every.add(v)
                if newest is None:
                    # newest-first by convention: the FIRST heading is the newest release.
                    newest, note = v, line.strip()[:90]
    if newest is None:
        return None, "no '## vX.Y.Z' header found in %s" % CONTROLLER_CHANGELOG, set()
    return newest, note, every


# GOLDEN_SHA_RE — the line `build-golden.sh` prints when it has actually published a bake.
# Reading THIS, rather than the directory's name, is R-410's whole fix.
GOLDEN_SHA_RE = re.compile(r"^GOLDEN_SHA256=([0-9a-f]{64})\s*$", re.MULTILINE)

# The bake log, as the runbook's §4.1 teardown copies it out. Two names are accepted because the
# 0.230.0 bake wrote `06-bake.log` alongside its README while 0.229.0 wrote `bake.log`; both are real
# bakes and neither should be called a fake.
BAKE_LOG_NAMES = ("bake.log", "06-bake.log", "bake-clean.log", "06-bake-clean.log")


def bake_sha_in(dirpath):
    """The GOLDEN_SHA256 a bake log in dirpath records, or None with the reason it could not be read.

    R-410. Until 2026-09-01 this gate matched EVIDENCE_RE against `os.listdir` and nothing else, so
    `mkdir documentation/tests/golden-9.9.9-2026-01-01` turned it green with no bake behind it —
    noticed while the 0.230.0 bake was running, when the evidence directory was created BEFORE the
    bake finished and the gate would have passed at that moment.

    A directory NAME is a label; `GOLDEN_SHA256=<64 hex>` is a fact only a completed publish produces.
    Reading it keeps the gate offline and `--fast`: it is one file read, no network.
    """
    for n in BAKE_LOG_NAMES:
        p = os.path.join(dirpath, n)
        if not os.path.isfile(p):
            continue
        try:
            with io.open(p, encoding="utf-8", errors="replace") as fh:
                m = GOLDEN_SHA_RE.search(fh.read())
        except OSError as e:
            return None, "bake log %s could not be read (%s)" % (n, e)
        if m:
            return m.group(1), n
        return None, "bake log %s carries no GOLDEN_SHA256= line" % n
    return None, "no bake log (looked for %s)" % ", ".join(BAKE_LOG_NAMES)


def newest_baked():
    """(tuple, str) of the newest golden bake recorded here, or (None, reason).

    A directory counts ONLY if it holds a bake log with a GOLDEN_SHA256 line (R-410). Directories that
    look right and hold nothing are reported by name, so a half-finished bake is visible rather than
    silently ignored.
    """
    if not os.path.isdir(EVIDENCE_DIR):
        return None, "bake-evidence directory not found at %s" % EVIDENCE_DIR
    found, rejected = [], []
    for name in os.listdir(EVIDENCE_DIR):
        m = EVIDENCE_RE.match(name)
        if not m:
            continue
        sha, why = bake_sha_in(os.path.join(EVIDENCE_DIR, name))
        if sha is None:
            rejected.append("%s (%s)" % (name, why))
            continue
        found.append((tuple(int(g) for g in m.groups()), "%s [sha %s…]" % (name, sha[:12])))
    if rejected:
        print("  NOT counted as bakes — a directory name is not a bake (R-410):")
        for r in sorted(rejected):
            print("    %s" % r)
    if not found:
        return None, ("no golden-<VER>-<DATE>/ directory under %s holds a bake log with a "
                      "GOLDEN_SHA256 line" % EVIDENCE_DIR)
    found.sort()
    return found[-1]


WAIVER_KEY_RE = re.compile(r"^\s*([a-z_]+)\s*:\s*(.*?)\s*$")
ROW_RE = re.compile(r"^R-\d+$")


def read_waiver(path=None, register=None, today=None):
    """(state, info) — state is 'absent', 'valid', 'expired' or 'malformed'.

    The file is four `key: value` lines; comments (`# …`) and blank lines are ignored. It is parsed by
    hand so the gate stays stdlib-only and `--fast`. Every way the file can be wrong returns
    'malformed' with the reason in info["why"] — the caller turns that into exit 2, never into 0 and
    never into a silent 'absent'. A decoy — the word `expires` with no date — lands here too.
    """
    path = path or WAIVER_PATH
    register = register or REGISTER_PATH
    today = today or datetime.datetime.now(datetime.timezone.utc).date()
    if not os.path.isfile(path):
        return "absent", {"path": path}
    try:
        text = io.open(path, encoding="utf-8").read()
    except OSError as e:
        return "malformed", {"path": path, "why": "cannot be read (%s)" % e}
    fields = {}
    for line in text.splitlines():
        line = line.split("#", 1)[0]
        m = WAIVER_KEY_RE.match(line)
        if m and m.group(2):
            fields[m.group(1)] = m.group(2).strip().strip("'\"")
    info = {"path": path, "fields": fields}

    def date_of(key):
        v = fields.get(key, "")
        try:
            return datetime.date.fromisoformat(v)
        except ValueError:
            return None

    issued, expires = date_of("issued"), date_of("expires")
    if issued is None:
        info["why"] = "`issued:` is absent or not a YYYY-MM-DD date (got %r)" % fields.get("issued", "")
        return "malformed", info
    if expires is None:
        info["why"] = "`expires:` is absent or not a YYYY-MM-DD date (got %r)" % fields.get("expires", "")
        return "malformed", info
    if expires <= issued:
        info["why"] = "`expires:` (%s) is not after `issued:` (%s)" % (expires, issued)
        return "malformed", info
    span = (expires - issued).days
    if span > WAIVER_MAX_DAYS:
        info["why"] = ("`expires:` is %d days after `issued:` — the hard limit is %d. A waiver that "
                       "tries to be permanent is refused; renew it with a new dated commit instead."
                       % (span, WAIVER_MAX_DAYS))
        return "malformed", info
    if not fields.get("reason"):
        info["why"] = "`reason:` is absent or empty"
        return "malformed", info
    row = fields.get("register_row", "")
    if not ROW_RE.match(row):
        info["why"] = "`register_row:` is absent or not of the form R-<n> (got %r)" % row
        return "malformed", info
    # The register is read for ONE fact — does the row exist — never for meaning (R-421).
    try:
        reg = io.open(register, encoding="utf-8").read()
    except OSError as e:
        info["why"] = "register %s cannot be read (%s)" % (register, e)
        return "malformed", info
    if ("**%s**" % row) not in reg:
        info["why"] = "`register_row: %s` names a row that does not exist in %s" % (
            row, os.path.relpath(register, ROOT) if register.startswith(ROOT) else register)
        return "malformed", info
    info.update({"issued": issued, "expires": expires, "row": row, "reason": fields["reason"],
                 "days_left": (expires - today).days})
    if today >= expires:
        return "expired", info
    return "valid", info


def vstr(v):
    return ".".join(str(p) for p in v)


def main():
    released, rel_note, every_released = released_versions()
    if released is None:
        print("GOLDEN CURRENCY GATE INCONCLUSIVE: %s" % rel_note)
        sys.exit(2)
    baked, bake_note = newest_baked()
    if baked is None:
        print("GOLDEN CURRENCY GATE INCONCLUSIVE: %s" % bake_note)
        sys.exit(2)

    # Print the evidence unconditionally — a gate that only speaks when it fails teaches nobody what
    # it is watching, and this one is watching the thing two releases already slipped through.
    print("  newest released controller : %s   (%s)" % (vstr(released), rel_note))
    print("  newest golden baked        : %s   (documentation/tests/%s)" % (vstr(baked), bake_note))

    # The waiver is read BEFORE any verdict, because a malformed one is a result of its own (exit 2)
    # whatever the golden's state — a file that says `expires` and means nothing must not lie there
    # looking like cover.
    wstate, winfo = read_waiver()
    if wstate == "malformed":
        print("")
        print("GOLDEN CURRENCY GATE INCONCLUSIVE: the waiver at %s is MALFORMED — %s"
              % (os.path.relpath(winfo["path"], ROOT), winfo["why"]))
        print("A malformed waiver is neither cover nor absence. Fix it (four lines: issued, expires "
              "<= %d days later, reason, register_row) or delete it." % WAIVER_MAX_DAYS)
        sys.exit(2)
    if wstate == "valid":
        print("  waiver                     : VALID until %s (%d day(s) left) — %s, reason: %s"
              % (winfo["expires"], winfo["days_left"], winfo["row"], winfo["reason"]))
    elif wstate == "expired":
        print("  waiver                     : EXPIRED on %s (%s)" % (winfo["expires"], winfo["row"]))

    # R-385 — UNRECORDED, checked before "behind". A golden whose version has no heading of its own
    # was built from something never written down, and that is a different (worse) fault than a
    # forgotten bake: there is nothing to read to find out what the fleet is running.
    if baked not in every_released:
        print("")
        print("GOLDEN CURRENCY GATE FAILED: golden %s is baked but UNRECORDED — the controller "
              "CHANGELOG has no '## v%s' heading." % (vstr(baked), vstr(baked)))
        if wstate == "valid":
            # THE ASYMMETRY. A waiver is a cadence choice about a golden that is BEHIND; it says
            # nothing about a golden nobody wrote down, and must not be read as if it did.
            print("A waiver exists (%s, until %s) and DOES NOT COVER THIS: it covers a golden that is "
                  "behind the record, never one that is unrecorded (R-385)."
                  % (winfo["row"], winfo["expires"]))
        print("The newest heading is %s. A golden ahead of the record was built from a version "
              "nobody wrote down, so no one can read what the fleet is running." % vstr(released))
        print("Fix: give v%s its own '## v%s — <what changed>' heading in "
              "felhom-controller/CHANGELOG.md, above the entries it supersedes. If its fix is "
              "currently described inside another version's entry, MOVE that text — do not "
              "duplicate it, and do not delete the reasoning." % (vstr(baked), vstr(baked)))
        print("If this bake was a throwaway that must never be delivered, delete its "
              "documentation/tests/golden-<VER>-<DATE>/ directory — never leave it to read as "
              "shipped.")
        sys.exit(1)

    if released > baked:
        lag = sorted(v for v in every_released if v > baked)
        if wstate == "valid":
            print("")
            print("GOLDEN CURRENCY GATE ADVISORY — WAIVED, NOT CLEAN: controller v%s is released and "
                  "NO golden carries it (newest bake is %s; %d release(s) behind: %s)."
                  % (vstr(released), vstr(baked), len(lag), ", ".join(vstr(v) for v in lag)))
            print("A machine installed right now would receive v%s and reach v%s by self-update."
                  % (vstr(baked), vstr(released)))
            print("Waived by %s until %s (%d day(s) left): %s"
                  % (winfo["row"], winfo["expires"], winfo["days_left"], winfo["reason"]))
            print("This is the operator's 2026-09-13 cadence ruling, not a pass: bake weekly and "
                  "before ANY drill or fresh install (RUNBOOK-manual-build.md §4.2). When the waiver "
                  "expires this gate is red again.")
            print("golden currency gate OK (WAIVED) — the newest released controller has NO golden; "
                  "a valid waiver covers it (NOTE: this checks the BAKE, not the vouch)")
            return
        print("")
        print("GOLDEN CURRENCY GATE FAILED: controller v%s is released and NO golden carries it "
              "(newest bake is %s; %d release(s) behind)." % (vstr(released), vstr(baked), len(lag)))
        if wstate == "expired":
            print("The waiver at documentation/tests/golden-waiver.yml EXPIRED on %s (%s). It ran "
                  "out, as a dated waiver is meant to: bake a golden, or renew it with a new dated "
                  "commit (at most %d days)." % (winfo["expires"], winfo["row"], WAIVER_MAX_DAYS))
        print("A machine installed right now would receive v%s — the release is written, tested and "
              "pushed, and NOT delivered." % vstr(baked))
        print("Fix: bake a golden per documentation/runbooks/RUNBOOK-manual-build.md §4.1, then vouch "
              "it (a THREE-field change: golden_version + agent_version + min_agent).")
        print("If the cadence ruling covers this release, issue a DATED waiver at "
              "documentation/tests/golden-waiver.yml (RUNBOOK-manual-build.md §4.2) — never a bypass.")
        sys.exit(1)

    if wstate == "expired":
        print("  NOTE: the waiver expired on %s and covers nothing today (the golden is current). "
              "Renew or delete it so it does not read as cover." % winfo["expires"])
    print("golden currency gate OK — the newest released controller has a golden "
          "(NOTE: this checks the BAKE, not the vouch — see the module docstring)")


if __name__ == "__main__":
    main()