app-catalog-felhom.eu/scripts/check-engine-major.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""check-engine-major.py — refuse a push that moves a DATABASE ENGINE across a MAJOR version.

Run from the repo root:
    python3 scripts/check-engine-major.py                      # diff origin/main..HEAD
    python3 scripts/check-engine-major.py --range <A>..<B>     # what .githooks/pre-push passes
Exit 0 no engine crosses a major · 1 REFUSED · 2 INCONCLUSIVE (no parent to diff against, or a pin
whose major cannot be read).

THE RULE THIS ENFORCES (app-catalog `CLAUDE.md`, operator ruling 2026-09-13, AMENDED 2026-09-21):

    A MariaDB image may cross a MAJOR version, but ONLY AS ITS OWN EDGE — never in the same commit
    as any other image move in that template. PostgreSQL and MySQL may NOT cross a major at all.

The MariaDB half was lifted by R-469 when Slice 4 shipped; the second clause is R-450's second half,
recorded while it was cheap: bookstack's `0b73e5e` moved the application 25.02.2 -> 26.05.2 AND
MariaDB 11.6 -> 12.3 in one commit — TWO migrations behind one edge, and an unreadable failure when
it breaks. One edge, one migration, so a failure names its own cause.

WHY IT EXISTS. On 2026-09-13 every `mariadb:` sidecar gained `MARIADB_AUTO_UPGRADE=1`, so the day a
MariaDB pin moves a major, the engine will CONVERT the customer's datadir on the next deliberate
Update (~7 s, own backup of the system tables first — SPIKE-r459-mariadb-upgrade-2026-09-06.md). That
is the right behaviour and it was ruled so. But the Update button still takes no backup of the app's
data (09-update-architecture.md §8.6), and PostgreSQL's image performs no conversion at all — it
REFUSES to start on an older major's datadir (R-463). Either way a cross-major pin is a customer-data
event, and until Slice 4 puts a verified backup in front of the button, the catalog must not offer one.

WHY A GATE AND NOT A SENTENCE IN CLAUDE.md. This project's most-repeated finding is that a rule with
no instrument is a wish. The rule's own expiry condition is the tell: "until Slice 4 ships" is exactly
the kind of clause nobody revisits. The gate carries the expiry in its refusal text, and removing the
gate is a diff someone reviews.

WHAT IT COMPARES. For every `templates/<app>/docker-compose.yml` changed between the two ends of the
range, the `image:` line of each SERVICE on both sides — per service, on that service's own line,
never a blind string replace (the same discipline as `upgrade-test.py`'s `render`). Only images whose
repository name is a database engine are judged (`ENGINES`); the app's own image may move as it
likes. The engine set is a NAME MATCH on the repository's last path component, so a registry prefix
(`docker.io/library/postgres:16`) or a digest suffix does not hide one. Scope is the glob, not a
hand-kept list of the four MariaDB and eleven PostgreSQL services — a list would need maintaining, and
the 2026-09-01 decoy sweep's rule is that scope is a fact too.

HONEST LIMIT — CI CANNOT RUN THIS YET. It needs a PARENT commit to diff against, and the CI runner
fetches at `--depth 1` (`.gitea/workflows/gates.yml`), which is the very gap R-452 recorded for the
`catalog_since` gate. So this runs in the pre-push hook, which has the full clone, and
`catalog_gates.py` SKIPS it — out loud — on a shallow clone rather than turning CI red on every push.
That is one gate short of enforcement, stated here so nobody mistakes the hook for CI. Fixing it is
R-452's fix (a deeper fetch), not a second row.

FAIL-CLOSED WHERE IT CAN BE. A range that cannot be resolved, a compose file at either end that cannot
be read, or an engine tag whose major cannot be parsed (`mariadb:lts`) is INCONCLUSIVE (2), never 0.
The pin gate already forbids floating tags, so an unparseable engine tag is a defect in its own right.
"""
import re
import subprocess
import sys

# Repository basenames that are database engines. A match here means "judge this service's major".
ENGINES = ("mariadb", "mysql", "postgres", "postgresql")

# R-469, 2026-09-21 — THE MARIADB HALF OF THE RULE IS LIFTED; THE POSTGRESQL HALF IS NOT.
#
# The rule's own condition was "until the Update button takes a verified backup as its precondition".
# Slice 4 SHIPPED on 2026-09-13 (controller v0.237.0/v0.238.0, any tier since v0.239.0), so the
# condition is met — for MariaDB. Every `mariadb:` sidecar carries MARIADB_AUTO_UPGRADE=1 (R-459) and
# the harness has WATCHED the conversion run on the E3/E3b edges, with the seeded data read back
# after. So a MariaDB major now has both halves it needs: a backup in front of it, and an engine that
# performs the conversion.
#
# PostgreSQL has neither the second half nor a procedure: its image performs no `pg_upgrade` and
# REFUSES to start on an older major's datadir, across ELEVEN templates (R-463). MySQL is in the same
# position with nothing measured at all. Both stay refused until R-463 produces a scripted
# `pg_upgrade` edge proven on all eleven.
LIFTED = ("mariadb",)

SERVICE_RE = re.compile(r"^  ([A-Za-z0-9_-]+):\s*$")
IMAGE_RE = re.compile(r"^\s+image:\s*[\"']?(\S+?)[\"']?\s*$")
TEMPLATE_RE = re.compile(r"^templates/[^/]+/docker-compose\.ya?ml$")
ZERO_SHA_RE = re.compile(r"^0{40}$")


def git(*args):
    p = subprocess.run(["git"] + list(args), capture_output=True, text=True)
    return p.returncode, p.stdout, p.stderr


def images_in(text):
    """{service: image} — per service, from that service's OWN `image:` line."""
    out, cur = {}, None
    for line in text.splitlines():
        m = SERVICE_RE.match(line)
        if m:
            cur = m.group(1)
            continue
        mi = IMAGE_RE.match(line)
        if mi and cur and cur not in out:
            out[cur] = mi.group(1)
    return out


def engine_of(ref):
    """(engine_name, tag) if the image's repository is a database engine, else None.

    `docker.io/library/postgres:16-alpine@sha256:…` -> ("postgres", "16-alpine").
    A registry with a port (`host:5000/postgres:16`) is handled by taking the LAST path component
    before splitting on ':'.
    """
    ref = ref.split("@", 1)[0]
    last = ref.rsplit("/", 1)[-1]
    if ":" in last:
        name, tag = last.rsplit(":", 1)
    else:
        name, tag = last, ""
    if name.lower() in ENGINES:
        return name.lower(), tag
    return None


def major_of(tag):
    m = re.match(r"^v?(\d+)", tag)
    return int(m.group(1)) if m else None


def resolve_range(spec):
    """'A..B' -> (A, B, note). An all-zero A (a new remote ref) falls back to origin/main."""
    if not spec or ".." not in spec:
        return None, None, "range must be <A>..<B> (got %r)" % spec
    a, b = spec.split("..", 1)
    if ZERO_SHA_RE.match(a):
        a = "origin/main"
    for r in (a, b):
        rc, _, err = git("rev-parse", "--verify", "-q", r + "^{commit}")
        if rc != 0:
            return None, None, "cannot resolve %r (%s)" % (r, err.strip() or "not a commit")
    return a, b, ""


def main(argv):
    spec = "origin/main..HEAD"
    for arg in argv:
        if arg.startswith("--range="):
            spec = arg[len("--range="):]
        elif arg == "--range" or arg.startswith("-"):
            pass
    if "--range" in argv:
        i = argv.index("--range")
        if i + 1 < len(argv):
            spec = argv[i + 1]

    rc, shallow, _ = git("rev-parse", "--is-shallow-repository")
    if rc == 0 and shallow.strip() == "true":
        print("ENGINE-MAJOR GATE INCONCLUSIVE: this clone is SHALLOW — there is no parent commit to "
              "diff an image: line against (the R-452 gap; the CI runner fetches at --depth 1). "
              "This gate is enforced by the pre-push hook, which has the full clone.")
        return 2

    a, b, why = resolve_range(spec)
    if a is None:
        print("ENGINE-MAJOR GATE INCONCLUSIVE: %s" % why)
        return 2

    rc, names, err = git("diff", "--name-only", a, b, "--", "templates")
    if rc != 0:
        print("ENGINE-MAJOR GATE INCONCLUSIVE: git diff %s %s failed: %s" % (a, b, err.strip()))
        return 2
    files = [n for n in names.split("\n") if TEMPLATE_RE.match(n)]

    compared, refused, bundled, allowed, unparseable = 0, [], [], [], []
    for path in files:
        rc_b, before_text, _ = git("show", "%s:%s" % (a, path))
        rc_a, after_text, _ = git("show", "%s:%s" % (b, path))
        if rc_a != 0:
            continue  # deleted at B: nothing is being offered
        before = images_in(before_text) if rc_b == 0 else {}
        after = images_in(after_text)

        # EVERY image move in this template, engine or not. It is the input to the "own edge" rule
        # (R-450): a MariaDB major is allowed only when it is the ONLY image this commit moves here.
        moves = [svc for svc, img in after.items() if svc in before and before[svc] != img]

        for svc, img_after in after.items():
            eng_after = engine_of(img_after)
            if eng_after is None or svc not in before:
                continue  # not an engine, or a NEW service (nothing to move across)
            eng_before = engine_of(before[svc])
            if eng_before is None:
                continue  # became an engine — there is no engine datadir to convert
            compared += 1
            if before[svc] == img_after:
                continue
            mb, ma = major_of(eng_before[1]), major_of(eng_after[1])
            if mb is None or ma is None:
                unparseable.append("%s %s: %s -> %s" % (path, svc, before[svc], img_after))
                continue
            if mb == ma:
                continue
            row = (path, svc, eng_after[0], mb, ma, before[svc], img_after)
            if eng_after[0] not in LIFTED:
                refused.append(row)
                continue
            # R-469 + R-450: lifted, but it must be the ONLY image this commit moves in this
            # template. `others` is named so the refusal can say WHAT it was bundled with.
            others = [o for o in moves if o != svc]
            if others:
                bundled.append(row + (sorted(others),))
                continue
            allowed.append(row)

    print("engine-major gate — range %s..%s: %d compose file(s) changed, %d engine pin(s) compared"
          % (a, b, len(files), compared))

    if refused or bundled:
        print("")
        for path, svc, eng, mb, ma, ib, ia in refused:
            print("ENGINE-MAJOR GATE FAILED: %s service %s moves %s %d -> %d (%s -> %s)."
                  % (path, svc, eng, mb, ma, ib, ia))
            print("  WHY: %s performs no datadir conversion of its own and REFUSES to start on an "
                  "older major's datadir (R-463, eleven templates). The Update takes a backup first "
                  "since Slice 4, but a backup is a route BACK, not a conversion — the app would "
                  "simply not come up. This half of the rule stands until R-463 has a scripted "
                  "pg_upgrade edge PROVEN on all eleven." % eng)
        for path, svc, eng, mb, ma, ib, ia, others in bundled:
            print("ENGINE-MAJOR GATE FAILED: %s service %s moves %s %d -> %d (%s -> %s) IN THE SAME "
                  "COMMIT as %s." % (path, svc, eng, mb, ma, ib, ia, ", ".join(others)))
            print("  WHY: an engine major gets its OWN EDGE (R-450). bookstack's 0b73e5e moved the "
                  "application AND MariaDB 11.6 -> 12.3 in one commit: two migrations behind one "
                  "edge, and an unreadable failure when it breaks. Split it into two commits — the "
                  "engine alone, then the app.")
        print("RULE (app-catalog CLAUDE.md, ruling 2026-09-13, amended by R-469 on 2026-09-21): "
              "MariaDB may cross a major AS ITS OWN EDGE; PostgreSQL and MySQL may not cross one at all.")
        return 1

    if unparseable:
        print("")
        for u in unparseable:
            print("ENGINE-MAJOR GATE INCONCLUSIVE: cannot read a MAJOR from %s" % u)
        print("An engine tag without a leading number cannot be judged and the pin gate already "
              "forbids floating tags — pin a numbered tag.")
        return 2

    for path, svc, eng, mb, ma, ib, ia in allowed:
        print("engine-major gate ALLOWED: %s service %s moves %s %d -> %d (%s -> %s) as its own edge "
              "— permitted since R-469 (Slice 4 shipped; MARIADB_AUTO_UPGRADE=1 converts the datadir)."
              % (path, svc, eng, mb, ma, ib, ia))
    print("engine-major gate OK — no forbidden engine major, and no engine major bundled with "
          "another image move (rule: CLAUDE.md, amended by R-469 2026-09-21)")
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))