admin
6a4a5f0cf7
gates / gates (push) Successful in 2s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
306 lines
16 KiB
Python
306 lines
16 KiB
Python
#!/usr/bin/env python3
|
|
# -*- coding: utf-8 -*-
|
|
"""check-engine-major.py — refuse a push that moves a DATABASE ENGINE across a MAJOR version.
|
|
|
|
Run from the repo root:
|
|
python3 scripts/check-engine-major.py # diff origin/main..HEAD
|
|
python3 scripts/check-engine-major.py --range <A>..<B> # what .githooks/pre-push passes
|
|
Exit 0 no engine crosses a major · 1 REFUSED · 2 INCONCLUSIVE (no parent to diff against, or a pin
|
|
whose major cannot be read).
|
|
|
|
THE RULE THIS ENFORCES (app-catalog `CLAUDE.md`, operator ruling 2026-09-13, AMENDED 2026-09-21):
|
|
|
|
A MariaDB image may cross a MAJOR version, but ONLY AS ITS OWN EDGE — never in the same commit
|
|
as any other image move in that template. PostgreSQL and MySQL may NOT cross a major at all.
|
|
|
|
The MariaDB half was lifted by R-469 when Slice 4 shipped; the second clause is R-450's second half,
|
|
recorded while it was cheap: bookstack's `0b73e5e` moved the application 25.02.2 -> 26.05.2 AND
|
|
MariaDB 11.6 -> 12.3 in one commit — TWO migrations behind one edge, and an unreadable failure when
|
|
it breaks. One edge, one migration, so a failure names its own cause.
|
|
|
|
WHY IT EXISTS. On 2026-09-13 every `mariadb:` sidecar gained `MARIADB_AUTO_UPGRADE=1`, so the day a
|
|
MariaDB pin moves a major, the engine will CONVERT the customer's datadir on the next deliberate
|
|
Update (~7 s, own backup of the system tables first — SPIKE-r459-mariadb-upgrade-2026-09-06.md). That
|
|
is the right behaviour and it was ruled so. But the Update button still takes no backup of the app's
|
|
data (09-update-architecture.md §8.6), and PostgreSQL's image performs no conversion at all — it
|
|
REFUSES to start on an older major's datadir (R-463). Either way a cross-major pin is a customer-data
|
|
event, and until Slice 4 puts a verified backup in front of the button, the catalog must not offer one.
|
|
|
|
WHY A GATE AND NOT A SENTENCE IN CLAUDE.md. This project's most-repeated finding is that a rule with
|
|
no instrument is a wish. The rule's own expiry condition is the tell: "until Slice 4 ships" is exactly
|
|
the kind of clause nobody revisits. The gate carries the expiry in its refusal text, and removing the
|
|
gate is a diff someone reviews.
|
|
|
|
WHAT IT COMPARES. For every `templates/<app>/docker-compose.yml` changed between the two ends of the
|
|
range, the `image:` line of each SERVICE on both sides — per service, on that service's own line,
|
|
never a blind string replace (the same discipline as `upgrade-test.py`'s `render`). Only images whose
|
|
repository name is a database engine are judged (`ENGINES`); the app's own image may move as it
|
|
likes. The engine set is a NAME MATCH on the repository's last path component, so a registry prefix
|
|
(`docker.io/library/postgres:16`) or a digest suffix does not hide one. Scope is the glob, not a
|
|
hand-kept list of the four MariaDB and eleven PostgreSQL services — a list would need maintaining, and
|
|
the 2026-09-01 decoy sweep's rule is that scope is a fact too.
|
|
|
|
HONEST LIMIT — CI CANNOT RUN THIS YET. It needs a PARENT commit to diff against, and the CI runner
|
|
fetches at `--depth 1` (`.gitea/workflows/gates.yml`), which is the very gap R-452 recorded for the
|
|
`catalog_since` gate. So this runs in the pre-push hook, which has the full clone, and
|
|
`catalog_gates.py` SKIPS it — out loud — on a shallow clone rather than turning CI red on every push.
|
|
That is one gate short of enforcement, stated here so nobody mistakes the hook for CI. Fixing it is
|
|
R-452's fix (a deeper fetch), not a second row.
|
|
|
|
FAIL-CLOSED WHERE IT CAN BE. A range that cannot be resolved, a compose file at either end that cannot
|
|
be read, or an engine tag whose major cannot be parsed (`mariadb:lts`) is INCONCLUSIVE (2), never 0.
|
|
The pin gate already forbids floating tags, so an unparseable engine tag is a defect in its own right.
|
|
"""
|
|
import re
|
|
import subprocess
|
|
import sys
|
|
|
|
# Repository basenames that are database engines. A match here means "judge this service's major".
|
|
ENGINES = ("mariadb", "mysql", "postgres", "postgresql", "postgis", "pgvector")
|
|
# The Postgres family, for the conversion clause below (postgis/postgis:16-3.5 is PostgreSQL 16; it was
|
|
# not judged at all before 2026-09-25 — adventurelog's engine could have crossed a major unseen).
|
|
PG_FAMILY = ("postgres", "postgresql", "postgis", "pgvector")
|
|
|
|
# R-469, 2026-09-21 — THE MARIADB HALF OF THE RULE IS LIFTED; THE POSTGRESQL HALF IS NOT.
|
|
#
|
|
# The rule's own condition was "until the Update button takes a verified backup as its precondition".
|
|
# Slice 4 SHIPPED on 2026-09-13 (controller v0.237.0/v0.238.0, any tier since v0.239.0), so the
|
|
# condition is met — for MariaDB. Every `mariadb:` sidecar carries MARIADB_AUTO_UPGRADE=1 (R-459) and
|
|
# the harness has WATCHED the conversion run on the E3/E3b edges, with the seeded data read back
|
|
# after. So a MariaDB major now has both halves it needs: a backup in front of it, and an engine that
|
|
# performs the conversion.
|
|
#
|
|
# PostgreSQL has neither the second half nor a procedure: its image performs no `pg_upgrade` and
|
|
# REFUSES to start on an older major's datadir, across ELEVEN templates (R-463). MySQL is in the same
|
|
# position with nothing measured at all. Both stay refused until R-463 produces a scripted
|
|
# `pg_upgrade` edge proven on all eleven.
|
|
LIFTED = ("mariadb",)
|
|
|
|
SERVICE_RE = re.compile(r"^ ([A-Za-z0-9_-]+):\s*$")
|
|
IMAGE_RE = re.compile(r"^\s+image:\s*[\"']?(\S+?)[\"']?\s*$")
|
|
TEMPLATE_RE = re.compile(r"^templates/[^/]+/docker-compose\.ya?ml$")
|
|
ZERO_SHA_RE = re.compile(r"^0{40}$")
|
|
|
|
|
|
def git(*args):
|
|
p = subprocess.run(["git"] + list(args), capture_output=True, text=True)
|
|
return p.returncode, p.stdout, p.stderr
|
|
|
|
|
|
def images_in(text):
|
|
"""{service: image} — per service, from that service's OWN `image:` line."""
|
|
out, cur = {}, None
|
|
for line in text.splitlines():
|
|
m = SERVICE_RE.match(line)
|
|
if m:
|
|
cur = m.group(1)
|
|
continue
|
|
mi = IMAGE_RE.match(line)
|
|
if mi and cur and cur not in out:
|
|
out[cur] = mi.group(1)
|
|
return out
|
|
|
|
|
|
def engine_of(ref):
|
|
"""(engine_name, tag) if the image's repository is a database engine, else None.
|
|
|
|
`docker.io/library/postgres:16-alpine@sha256:…` -> ("postgres", "16-alpine").
|
|
A registry with a port (`host:5000/postgres:16`) is handled by taking the LAST path component
|
|
before splitting on ':'.
|
|
"""
|
|
ref = ref.split("@", 1)[0]
|
|
last = ref.rsplit("/", 1)[-1]
|
|
if ":" in last:
|
|
name, tag = last.rsplit(":", 1)
|
|
else:
|
|
name, tag = last, ""
|
|
if name.lower() in ENGINES:
|
|
return name.lower(), tag
|
|
return None
|
|
|
|
|
|
def major_of(tag):
|
|
m = re.match(r"^(?:v|pg)?(\d+)", tag)
|
|
return int(m.group(1)) if m else None
|
|
|
|
|
|
def pg_conversion_proof(rev, compose_path, svc, mb, ma, after):
|
|
"""None when the template at `rev` carries a PROVEN two-venue ladder entry for exactly this step with
|
|
the matching engine_conversion mark; else the reason, as a sentence fragment."""
|
|
import json as _json
|
|
fy = compose_path.rsplit("/", 1)[0] + "/.felhom.yml"
|
|
rc, text, _ = git("show", "%s:%s" % (rev, fy))
|
|
if rc != 0:
|
|
return "%s cannot be read" % fy
|
|
entries = []
|
|
for line in text.splitlines():
|
|
m = re.match(r"^ - (\{.*\})\s*$", line)
|
|
if m:
|
|
try:
|
|
entries.append(_json.loads(m.group(1)))
|
|
except ValueError:
|
|
pass
|
|
step = [e for e in entries if e.get("to") == after]
|
|
if not step:
|
|
return "no ladder entry has `to` = this commit's images"
|
|
e = step[-1]
|
|
want = {"service": svc, "engine": "postgres", "from": mb, "to": ma}
|
|
if e.get("verdict") != "proven":
|
|
return "the step's entry is %r, not proven" % e.get("verdict")
|
|
if e.get("engine_conversion") != want:
|
|
return "the step's entry carries engine_conversion %r, want %r" % (e.get("engine_conversion"), want)
|
|
if not (e.get("evidence") and e.get("box_evidence")):
|
|
return "the step's entry does not cite BOTH venues' evidence (evidence + box_evidence)"
|
|
return None
|
|
|
|
|
|
def resolve_range(spec):
|
|
"""'A..B' -> (A, B, note). An all-zero A (a new remote ref) falls back to origin/main."""
|
|
if not spec or ".." not in spec:
|
|
return None, None, "range must be <A>..<B> (got %r)" % spec
|
|
a, b = spec.split("..", 1)
|
|
if ZERO_SHA_RE.match(a):
|
|
a = "origin/main"
|
|
for r in (a, b):
|
|
rc, _, err = git("rev-parse", "--verify", "-q", r + "^{commit}")
|
|
if rc != 0:
|
|
return None, None, "cannot resolve %r (%s)" % (r, err.strip() or "not a commit")
|
|
return a, b, ""
|
|
|
|
|
|
def main(argv):
|
|
spec = "origin/main..HEAD"
|
|
for arg in argv:
|
|
if arg.startswith("--range="):
|
|
spec = arg[len("--range="):]
|
|
elif arg == "--range" or arg.startswith("-"):
|
|
pass
|
|
if "--range" in argv:
|
|
i = argv.index("--range")
|
|
if i + 1 < len(argv):
|
|
spec = argv[i + 1]
|
|
|
|
rc, shallow, _ = git("rev-parse", "--is-shallow-repository")
|
|
if rc == 0 and shallow.strip() == "true":
|
|
print("ENGINE-MAJOR GATE INCONCLUSIVE: this clone is SHALLOW — there is no parent commit to "
|
|
"diff an image: line against (the R-452 gap; the CI runner fetches at --depth 1). "
|
|
"This gate is enforced by the pre-push hook, which has the full clone.")
|
|
return 2
|
|
|
|
a, b, why = resolve_range(spec)
|
|
if a is None:
|
|
print("ENGINE-MAJOR GATE INCONCLUSIVE: %s" % why)
|
|
return 2
|
|
|
|
rc, names, err = git("diff", "--name-only", a, b, "--", "templates")
|
|
if rc != 0:
|
|
print("ENGINE-MAJOR GATE INCONCLUSIVE: git diff %s %s failed: %s" % (a, b, err.strip()))
|
|
return 2
|
|
files = [n for n in names.split("\n") if TEMPLATE_RE.match(n)]
|
|
|
|
compared, refused, bundled, allowed, unparseable = 0, [], [], [], []
|
|
for path in files:
|
|
rc_b, before_text, _ = git("show", "%s:%s" % (a, path))
|
|
rc_a, after_text, _ = git("show", "%s:%s" % (b, path))
|
|
if rc_a != 0:
|
|
continue # deleted at B: nothing is being offered
|
|
before = images_in(before_text) if rc_b == 0 else {}
|
|
after = images_in(after_text)
|
|
|
|
# EVERY image move in this template, engine or not. It is the input to the "own edge" rule
|
|
# (R-450): a MariaDB major is allowed only when it is the ONLY image this commit moves here.
|
|
moves = [svc for svc, img in after.items() if svc in before and before[svc] != img]
|
|
|
|
for svc, img_after in after.items():
|
|
eng_after = engine_of(img_after)
|
|
if eng_after is None or svc not in before:
|
|
continue # not an engine, or a NEW service (nothing to move across)
|
|
eng_before = engine_of(before[svc])
|
|
if eng_before is None:
|
|
continue # became an engine — there is no engine datadir to convert
|
|
compared += 1
|
|
if before[svc] == img_after:
|
|
continue
|
|
mb, ma = major_of(eng_before[1]), major_of(eng_after[1])
|
|
if mb is None or ma is None:
|
|
unparseable.append("%s %s: %s -> %s" % (path, svc, before[svc], img_after))
|
|
continue
|
|
if mb == ma:
|
|
continue
|
|
row = (path, svc, eng_after[0], mb, ma, before[svc], img_after)
|
|
if eng_after[0] in PG_FAMILY:
|
|
# `09` §3 decision 35 (2026-09-25): a PostgreSQL major passes ONLY for a template whose ladder
|
|
# entry for THIS step is proven on both venues and carries the conversion mark, and only as
|
|
# the only image move in its commit (the MariaDB rule). Every other app stays refused.
|
|
why = pg_conversion_proof(b, path, svc, mb, ma, after)
|
|
others = [o for o in moves if o != svc]
|
|
if why is None and not others:
|
|
allowed.append(row)
|
|
continue
|
|
refused.append(row + ((why or "it is bundled with %s" % ", ".join(sorted(others))),))
|
|
continue
|
|
if eng_after[0] not in LIFTED:
|
|
refused.append(row + (None,))
|
|
continue
|
|
# R-469 + R-450: lifted, but it must be the ONLY image this commit moves in this
|
|
# template. `others` is named so the refusal can say WHAT it was bundled with.
|
|
others = [o for o in moves if o != svc]
|
|
if others:
|
|
bundled.append(row + (sorted(others),))
|
|
continue
|
|
allowed.append(row)
|
|
|
|
print("engine-major gate — range %s..%s: %d compose file(s) changed, %d engine pin(s) compared"
|
|
% (a, b, len(files), compared))
|
|
|
|
if refused or bundled:
|
|
print("")
|
|
for path, svc, eng, mb, ma, ib, ia, pgwhy in refused:
|
|
print("ENGINE-MAJOR GATE FAILED: %s service %s moves %s %d -> %d (%s -> %s)."
|
|
% (path, svc, eng, mb, ma, ib, ia))
|
|
if pgwhy:
|
|
print(" NOT PROVEN FOR THIS APP: %s. A PostgreSQL major passes only with a ladder entry for "
|
|
"this step that is proven on BOTH venues and carries engine_conversion {service, "
|
|
"engine: postgres, from, to} (written by upgrade-test.py --write-ladder), and only as "
|
|
"the only image move in its commit (`09` §3 decision 35)." % pgwhy)
|
|
print(" WHY: %s performs no datadir conversion of its own and REFUSES to start on an "
|
|
"older major's datadir (R-463, eleven templates). The Update takes a backup first "
|
|
"since Slice 4, but a backup is a route BACK, not a conversion — the app would "
|
|
"simply not come up. This half of the rule stands until R-463 has a scripted "
|
|
"pg_upgrade edge PROVEN on all eleven." % eng)
|
|
for path, svc, eng, mb, ma, ib, ia, others in bundled:
|
|
print("ENGINE-MAJOR GATE FAILED: %s service %s moves %s %d -> %d (%s -> %s) IN THE SAME "
|
|
"COMMIT as %s." % (path, svc, eng, mb, ma, ib, ia, ", ".join(others)))
|
|
print(" WHY: an engine major gets its OWN EDGE (R-450). bookstack's 0b73e5e moved the "
|
|
"application AND MariaDB 11.6 -> 12.3 in one commit: two migrations behind one "
|
|
"edge, and an unreadable failure when it breaks. Split it into two commits — the "
|
|
"engine alone, then the app.")
|
|
print("RULE (app-catalog CLAUDE.md, ruling 2026-09-13, amended by R-469 on 2026-09-21): "
|
|
"MariaDB may cross a major AS ITS OWN EDGE; PostgreSQL and MySQL may not cross one at all.")
|
|
return 1
|
|
|
|
if unparseable:
|
|
print("")
|
|
for u in unparseable:
|
|
print("ENGINE-MAJOR GATE INCONCLUSIVE: cannot read a MAJOR from %s" % u)
|
|
print("An engine tag without a leading number cannot be judged and the pin gate already "
|
|
"forbids floating tags — pin a numbered tag.")
|
|
return 2
|
|
|
|
for path, svc, eng, mb, ma, ib, ia in allowed:
|
|
if eng in PG_FAMILY:
|
|
print("engine-major gate ALLOWED: %s service %s moves %s %d -> %d (%s -> %s) as its own edge — "
|
|
"its ladder entry is proven on both venues and carries the conversion mark (`09` §3 "
|
|
"decision 35; the box converts it, controller v0.273.0)." % (path, svc, eng, mb, ma, ib, ia))
|
|
continue
|
|
print("engine-major gate ALLOWED: %s service %s moves %s %d -> %d (%s -> %s) as its own edge "
|
|
"— permitted since R-469 (Slice 4 shipped; MARIADB_AUTO_UPGRADE=1 converts the datadir)."
|
|
% (path, svc, eng, mb, ma, ib, ia))
|
|
print("engine-major gate OK — no forbidden engine major, and no engine major bundled with "
|
|
"another image move (rule: CLAUDE.md, amended by R-469 2026-09-21)")
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main(sys.argv[1:]))
|