admin/app-catalog-felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
22f598bb951358fa9c8666411db58fc00a402ca4
app-catalog-felhom.eu/scripts/test_gate_decoys.py
T
admin 6c690a1947
gates / gates (push) Failing after 2s
Details
gates: refuse a health probe the app does not answer (R-618) 
check-probe-matches-compose.py, a --fast gate so it bites in the hook and in CI.

The oracle was already in every template: the probed service's own compose healthcheck dials the
app on 127.0.0.1. The gate compares the .felhom.yml probe against it, statically.

Port mismatch REFUSES for every check type. Path mismatch REFUSES only where the probe can fail on
it (type api WITH expect) and WARNS otherwise, because probeHTTP calls any response healthy
otherwise - measured, not assumed. Six WARNs on the current catalog, each named in the CHANGELOG;
paperless-ngx is the loud one: no container matches the stack name, so no probe ever runs.

Four red-proofs and five decoys, suite now 51 cases. --root lets the suite judge its own clone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-09-22 10:51:34 +02:00

576 lines
34 KiB
Python
Raw Blame History

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""test_gate_decoys.py — can this repo's gates be fooled by a LABEL? (R-421)
The same instrument as `felhom.eu/scripts/test_gate_decoys.py`: a decoy is the LABEL without the
FACT, and a gate that convicts on the label alone — or fails to convict on the fact — is a live hole.
Every case asserts BOTH directions where it can: the genuine article must pass and the decoy must be
judged on what it IS, not on what it says.
Covered here (the `COVERS` literal is AST-read by `felhom.eu/scripts/decoy_coverage_gate.py`):
engine-major — `check-engine-major.py` refuses a database-engine pin that crosses a MAJOR.
Its label is the version string; its fact is the `image:` line of an engine SERVICE. Decoys a
real session would produce:
* the major moves in a COMMENT and in kimai's `serverVersion=11.6.2-MariaDB` env var, while
the image line stays — must PASS (nothing moved);
* the APP's own image crosses a major (kimai 2.57 -> 3.0) — must PASS (not an engine);
* a `mariadb:12.3` string lands in README.md — must PASS (not a template);
* the engine moves WITHIN its major (11.6 -> 11.8) — must PASS (the rule says MAJOR);
and the facts:
* `mariadb:11.6 -> mariadb:12.3` on `kimai-db` — must be REFUSED (exit 1), naming the rule
and its expiry (R-448);
* `postgres:16-alpine -> postgres:17-alpine` on `docmost-postgres` — must be REFUSED (the
eleven PostgreSQL services are covered by NAME MATCH, not by a list);
* `mariadb:11.6 -> mariadb:lts` — INCONCLUSIVE (exit 2), never 0: a major nobody can read is
not a pass.
HOW. The repo is cloned into a scratch directory; the WORKING-TREE gate is run inside the clone
(so the file under test is the one being edited, not HEAD's); each case is one commit on top of the
clone's HEAD and the gate is run with `--range HEAD~1..HEAD`. The real tree is never touched.
Run from the repo root: python3 scripts/test_gate_decoys.py
Exit 0 every decoy judged correctly · 1 a decoy passed or a genuine article was refused.
"""
import io
import os
import re
import shutil
import subprocess
import sys
import tempfile
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# ── WHAT THIS FILE COVERS ────────────────────────────────────────────────────────────────────────
# Read by felhom.eu/scripts/decoy_coverage_gate.py, which AST-parses this literal. A gate named here
# MUST have a decoy below that has been seen to fail.
COVERS = {
"engine-major": "the major moved in a comment/env var/README/app image, not on an engine's image: line",
"catalog-since": "the date bumped in a comment/README while .felhom.yml's field stayed; or only a comment/env moved, no image (R-452)",
"probe-matches-compose": "the port/path moved in a COMMENT, in traefik's loadbalancer label, in "
"`ports:`/`expose:`, or on a NON-probed service - none of which is where "
"the app listens; vs a real probe port/path that the app does not answer (R-618)",
"copy-i18n": "Hungarian edited in a COMMENT/README/display_name (label, not copy) vs a real frozen string changed; an English block that is not English, is not matched to a Hungarian twin, or rewrites a credential (R-560). Also the DEGRADED mode CI actually runs — PyYAML shadowed out, freeze only (R-595)",
}
fails = []
ran = 0
def sh(args, cwd):
return subprocess.run(args, cwd=cwd, capture_output=True, text=True)
def make_clone():
tmp = tempfile.mkdtemp(prefix="catalog-decoys-")
r = sh(["git", "clone", "-q", "file://" + ROOT, tmp], cwd=ROOT)
if r.returncode != 0:
raise SystemExit("clone failed: " + r.stderr)
sh(["git", "config", "user.email", "decoy@gate.invalid"], cwd=tmp)
sh(["git", "config", "user.name", "decoy"], cwd=tmp)
return tmp
def edit(clone, relpath, fn):
p = os.path.join(clone, relpath)
os.makedirs(os.path.dirname(p), exist_ok=True) # a case may ADD a file (a new app directory)
text = io.open(p, encoding="utf-8").read() if os.path.exists(p) else ""
new = fn(text)
if new == text:
raise SystemExit("case did not change %s — the case is broken, not the gate" % relpath)
with io.open(p, "w", encoding="utf-8") as fh:
fh.write(new)
def commit(clone, msg):
sh(["git", "add", "-A"], cwd=clone)
r = sh(["git", "commit", "-q", "-m", msg], cwd=clone)
if r.returncode != 0:
raise SystemExit("commit failed: " + r.stderr)
def reset(clone):
sh(["git", "reset", "-q", "--hard", "HEAD"], cwd=clone)
def case(name, clone, edits, expect_rc, must_contain=(), gate="check-engine-major.py"):
"""edits: list of (relpath, fn). Commits them, runs the gate on HEAD~1..HEAD, restores."""
global ran
ran += 1
base = sh(["git", "rev-parse", "HEAD"], cwd=clone).stdout.strip()
try:
for relpath, fn in edits:
edit(clone, relpath, fn)
commit(clone, name)
# the WORKING-TREE gate, run inside the clone (it reads git from its cwd)
r = sh([sys.executable, os.path.join(ROOT, "scripts", gate),
"--range", "HEAD~1..HEAD"], cwd=clone)
out = r.stdout + r.stderr
ok = r.returncode == expect_rc and all(m in out for m in must_contain)
if ok:
print(" ok %-52s rc=%d (expected %d)" % (name, r.returncode, expect_rc))
else:
fails.append("%s: rc=%d expected %d; missing %s\n%s" % (
name, r.returncode, expect_rc,
[m for m in must_contain if m not in out], out[-900:]))
return out
finally:
sh(["git", "reset", "-q", "--hard", base], cwd=clone)
def case_probe(name, clone, edits, expect_rc, must_contain=(), apps=("tandoor", "zipline", "wger",
"home-assistant")):
"""The probe gate reads FILES in a checkout, so its cases need `--root` and no commit.
Without `--root` the gate would read the REAL repo while the case edits the clone, every case
would see identical bytes, and every case would pass — the constant-for-measurement shape. The
app list is passed explicitly for the same reason: a whole-repo run is dominated by the three
genuine faults and would mask whether THIS case's edit changed anything.
"""
global ran
ran += 1
base = sh(["git", "rev-parse", "HEAD"], cwd=clone).stdout.strip()
try:
for relpath, fn in edits:
edit(clone, relpath, fn)
r = sh([sys.executable, os.path.join(ROOT, "scripts", "check-probe-matches-compose.py"),
"--root=" + clone] + list(apps), cwd=clone)
out = r.stdout + r.stderr
ok = r.returncode == expect_rc and all(m in out for m in must_contain)
if ok:
print(" ok %-52s rc=%d (expected %d)" % (name, r.returncode, expect_rc))
else:
fails.append("%s: rc=%d expected %d; missing %s\n%s" % (
name, r.returncode, expect_rc,
[m for m in must_contain if m not in out], out[-900:]))
return out
finally:
sh(["git", "checkout", "-q", "--", "."], cwd=clone)
sh(["git", "reset", "-q", "--hard", base], cwd=clone)
def case_copy(name, clone, edits, expect_rc, must_contain=(), extra_args=()):
"""The copy-i18n gate reads FILES, not commits, so its cases need neither a commit nor a range —
but they DO need --root, or the gate would read the real repo and judge files nobody edited.
That is the `constant-for-measurement` decoy shape, and it would make every case below pass."""
global ran
ran += 1
base = sh(["git", "rev-parse", "HEAD"], cwd=clone).stdout.strip()
try:
for relpath, fn in edits:
edit(clone, relpath, fn)
args = list(extra_args)
if "--expect-missing" not in args:
# MEASURE the clone's current coverage and hand it back as the ceiling. The ratchet is
# not what these cases test — they test the freeze, the structure and the language —
# and hard-coding a number here would make every case fail the day a batch lands.
# The ratchet has its own two cases below, in both directions.
probe = sh([sys.executable, os.path.join(ROOT, "scripts", "check-copy-i18n.py"),
"--root", clone, "--expect-missing", "-1"], cwd=clone)
m = re.search(r"(\d+) strings have no English", probe.stdout + probe.stderr)
if not m:
fails.append("%s: could not measure the clone's coverage — the case is broken, "
"not the gate" % name)
return ""
args += ["--expect-missing", m.group(1)]
r = sh([sys.executable, os.path.join(ROOT, "scripts", "check-copy-i18n.py"),
"--root", clone] + args, cwd=clone)
out = r.stdout + r.stderr
if r.returncode == expect_rc and all(m in out for m in must_contain):
print(" ok %-52s rc=%d (expected %d)" % (name, r.returncode, expect_rc))
else:
fails.append("%s: rc=%d expected %d; missing %s\n%s" % (
name, r.returncode, expect_rc,
[m for m in must_contain if m not in out], out[-900:]))
return out
finally:
sh(["git", "checkout", "-q", "--", "."], cwd=clone)
sh(["git", "clean", "-qfd"], cwd=clone)
sh(["git", "reset", "-q", "--hard", base], cwd=clone)
NOYAML = os.path.join(tempfile.gettempdir(), "felhom-decoy-noyaml")
def noyaml_dir():
"""A directory that shadows PyYAML with a module that refuses to import — the CI runner has
python3 and git and NOTHING else (.gitea/workflows/gates.yml), and the copy gate's first six
pushes each turned CI red because it imported yaml. These cases pin the degraded mode."""
os.makedirs(NOYAML, exist_ok=True)
with io.open(os.path.join(NOYAML, "yaml.py"), "w", encoding="utf-8") as fh:
fh.write('raise ImportError("no module named yaml (CI-runner simulation)")\n')
return NOYAML
def case_copy_noyaml(name, clone, edits, expect_rc, must_contain=()):
"""case_copy with PyYAML made unimportable — i.e. what CI actually runs."""
global ran
ran += 1
base = sh(["git", "rev-parse", "HEAD"], cwd=clone).stdout.strip()
env = dict(os.environ, PYTHONPATH=noyaml_dir())
try:
for relpath, fn in edits:
edit(clone, relpath, fn)
r = subprocess.run([sys.executable, os.path.join(ROOT, "scripts", "check-copy-i18n.py"),
"--root", clone], cwd=clone, capture_output=True, text=True, env=env)
out = r.stdout + r.stderr
if r.returncode == expect_rc and all(m in out for m in must_contain):
print(" ok %-52s rc=%d (expected %d)" % (name, r.returncode, expect_rc))
else:
fails.append("%s: rc=%d expected %d; missing %s\n%s" % (
name, r.returncode, expect_rc,
[m for m in must_contain if m not in out], out[-900:]))
return out
finally:
sh(["git", "checkout", "-q", "--", "."], cwd=clone)
sh(["git", "clean", "-qfd"], cwd=clone)
sh(["git", "reset", "-q", "--hard", base], cwd=clone)
def swap_image(service, frm, to):
"""Change ONLY the named service's own image: line — the same per-service discipline as the
gate, so the case moves the fact and nothing else."""
def _fn(text):
out, cur, done = [], None, False
for line in text.splitlines():
m = re.match(r"^ ([A-Za-z0-9_-]+):\s*$", line)
if m:
cur = m.group(1)
mi = re.match(r"^(\s+image:\s*)(\S+)\s*$", line)
if mi and cur == service and mi.group(2) == frm:
line = mi.group(1) + to
done = True
out.append(line)
if not done:
raise SystemExit("%s does not carry image %s — fixture drifted" % (service, frm))
return "\n".join(out) + "\n"
return _fn
def main():
gate = os.path.join(ROOT, "scripts", "check-engine-major.py")
if not os.path.isfile(gate):
print("FAIL: scripts/check-engine-major.py is missing — a failure, never a skip")
return 1
clone = make_clone()
try:
KIMAI = "templates/kimai/docker-compose.yml"
DOCMOST = "templates/docmost/docker-compose.yml"
# ── THE FACTS: these must be refused ─────────────────────────────────────────────────
out = case("FACT: docmost-postgres 16-alpine -> 17-alpine bundled with the app bump", clone,
[(DOCMOST, lambda t: swap_image("docmost-postgres", "postgres:16-alpine", "postgres:17-alpine")(t))],
expect_rc=1,
must_contain=("ENGINE-MAJOR GATE FAILED", "docmost-postgres", "postgres 16 -> 17"))
if "REFUSAL_TEXT" in os.environ:
print(out)
case("FACT: docmost-postgres postgres:16-alpine -> 17-alpine", clone,
[(DOCMOST, swap_image("docmost-postgres", "postgres:16-alpine", "postgres:17-alpine"))],
expect_rc=1, must_contain=("docmost-postgres", "postgres 16 -> 17", "R-463"))
# R-469 + R-450 (2026-09-21): a MariaDB major bundled with the app's own bump is the
# bookstack 0b73e5e shape — two migrations behind one edge — and stays refused.
case("FACT: kimai-db 11.6 -> 12.3 BUNDLED with the kimai app bump", clone,
[(KIMAI, lambda t: swap_image("kimai", "kimai/kimai2:apache-2.57.0", "kimai/kimai2:apache-2.58.0")(
swap_image("kimai-db", "mariadb:11.6", "mariadb:12.3")(t)))],
expect_rc=1, must_contain=("IN THE SAME COMMIT as kimai", "OWN EDGE", "R-450"))
case("FACT: kimai-db mariadb:11.6 -> mariadb:lts (major unreadable)", clone,
[(KIMAI, swap_image("kimai-db", "mariadb:11.6", "mariadb:lts"))],
expect_rc=2, must_contain=("INCONCLUSIVE",))
# ── THE GENUINE ARTICLES: these must pass ────────────────────────────────────────────
case("GENUINE: kimai-db mariadb:11.6 -> 11.8 (within major)", clone,
[(KIMAI, swap_image("kimai-db", "mariadb:11.6", "mariadb:11.8"))],
expect_rc=0, must_contain=("engine-major gate OK",))
# R-469: the LIFT itself. A MariaDB major ALONE in its template is now permitted, and the
# gate says so by name rather than passing in silence.
case("GENUINE: kimai-db mariadb:11.6 -> 12.3 ALONE (the R-469 lift)", clone,
[(KIMAI, swap_image("kimai-db", "mariadb:11.6", "mariadb:12.3"))],
expect_rc=0, must_contain=("ALLOWED", "kimai-db", "mariadb 11 -> 12", "R-469"))
# ── THE DECOYS: the label moves, the fact does not — these must pass ─────────────────
def comment_and_env(text):
# the version string moves in a COMMENT and in kimai's serverVersion env, image untouched
t = text.replace("serverVersion=11.6.2-MariaDB", "serverVersion=12.3.0-MariaDB")
return t.replace("# Database: mariadb", "# Database: mariadb (image: mariadb:12.3 soon)")
case("DECOY: major moves only in a comment + serverVersion env", clone,
[(KIMAI, comment_and_env)], expect_rc=0, must_contain=("engine-major gate OK",))
case("DECOY: the APP image crosses a major (kimai 2.57 -> 3.0)", clone,
[(KIMAI, swap_image("kimai", "kimai/kimai2:apache-2.57.0", "kimai/kimai2:apache-3.0.0"))],
expect_rc=0, must_contain=("engine-major gate OK",))
case("DECOY: 'mariadb:12.3' lands in README.md, not a template", clone,
[("README.md", lambda t: t + "\nDecoy: mariadb:11.6 -> mariadb:12.3 pending.\n")],
expect_rc=0, must_contain=("0 compose file(s) changed",))
# ── catalog-since (R-452): an image move must bump the app's catalog_since ───────────
import datetime
today = datetime.date.today().isoformat()
KIMAI_FY = "templates/kimai/.felhom.yml"
CS = "check-catalog-since.py"
def set_since(date):
def _fn(text):
new = re.sub(r'^catalog_since:\s*"?\d{4}-\d{2}-\d{2}"?', 'catalog_since: "%s"' % date, text, count=1, flags=re.M)
if new == text:
raise SystemExit("kimai's .felhom.yml carries no catalog_since — fixture drifted")
return new
return _fn
case("FACT: kimai image moves, catalog_since untouched", clone,
[(KIMAI, swap_image("kimai", "kimai/kimai2:apache-2.57.0", "kimai/kimai2:apache-2.58.0"))],
expect_rc=1, must_contain=("CATALOG-SINCE GATE FAILED", "kimai", "catalog_since is still"), gate=CS)
case("FACT: kimai image moves, catalog_since set to a FUTURE year", clone,
[(KIMAI, swap_image("kimai", "kimai/kimai2:apache-2.57.0", "kimai/kimai2:apache-2.58.0")),
(KIMAI_FY, set_since("2036-09-13"))],
expect_rc=1, must_contain=("in the future",), gate=CS)
case("GENUINE: kimai image moves AND catalog_since = today", clone,
[(KIMAI, swap_image("kimai", "kimai/kimai2:apache-2.57.0", "kimai/kimai2:apache-2.58.0")),
(KIMAI_FY, set_since(today))],
expect_rc=0, must_contain=("catalog-since gate OK", "1 image move(s) dated"), gate=CS)
case("DECOY: image moves; today's date lands in a COMMENT and README, the field stays", clone,
[(KIMAI, lambda t: swap_image("kimai", "kimai/kimai2:apache-2.57.0", "kimai/kimai2:apache-2.58.0")(t).replace("services:", "# catalog_since: %s\nservices:" % today, 1)),
("README.md", lambda t: t + "\ncatalog_since: %s (kimai)\n" % today)],
expect_rc=1, must_contain=("CATALOG-SINCE GATE FAILED",), gate=CS)
case("DECOY: only a comment + env line change, images untouched, date untouched", clone,
[(KIMAI, comment_and_env)], expect_rc=0, must_contain=("0 image move(s) dated", "catalog-since gate OK"), gate=CS)
# ── copy-i18n (R-560): Hungarian frozen, English sound ───────────────────────────────
PB = "templates/privatebin/.felhom.yml"
TOTAL = 1032 # every copy string in the catalog, measured on 94bc5febaca2
PB_EN = 14 # what the genuine block below translates
# A CORRECT English block for privatebin — the genuine article every decoy is a twist on.
GENUINE_EN = """
i18n:
en:
description: "Encrypted note and text sharing"
app_info:
tagline: "Encrypted text sharing - the server never sees the content"
use_cases:
- 'Share sensitive text safely'
- 'End-to-end encryption - the server cannot read the content'
- 'Choose how long it lasts (5 minutes to a year, or never)'
- 'Delete after reading, automatically'
- 'Password protection for extra safety'
first_steps:
- 'Open paste.DOMAIN in your browser'
- 'Type your text and select Send'
- 'Share the link you get - the encryption key is inside the URL'
deploy_fields:
- env_var: DOMAIN
label: "Domain"
description: "The server domain name"
- env_var: SUBDOMAIN
label: "Subdomain"
description: "The address this app answers on"
"""
def strip_en(t):
"""Remove an existing English block (and its comment header) — a case must behave the
same before and after that app's batch lands, or the suite rots on a future push."""
t = re.sub(r"\n# --- English copy.*\Z", "\n", t, flags=re.S)
return re.sub(r"\n^i18n:\n.*\Z", "\n", t, flags=re.S | re.M)
def add_en(block=GENUINE_EN):
return lambda t: strip_en(t).rstrip("\n") + "\n" + block
def en_with(old_, new_):
return add_en(GENUINE_EN.replace(old_, new_))
# THE FACTS — each must be refused.
case_copy("FACT: a Hungarian byte changed in a frozen string", clone,
[(PB, lambda t: t.replace("Titkosított jegyzet és szöveg megosztás",
"Titkosított jegyzet- és szövegmegosztás"))],
expect_rc=1, must_contain=("Hungarian CHANGED", "privatebin", "description"))
case_copy("FACT: a Hungarian first_step removed", clone,
[(PB, lambda t: t.replace(" - 'Oszd meg a generált linket - a titkosítási kulcs az URL-ben van'\n", ""))],
expect_rc=1, must_contain=("REMOVED", "first_steps"))
case_copy("FACT: a NEW app is not in the freeze", clone,
[("templates/decoyapp/.felhom.yml",
lambda t: 'display_name: "Decoy"\ndescription: "Uj alkalmazas"\nslug: decoyapp\n')],
expect_rc=1, must_contain=("not in the freeze", "--add-app"))
case_copy("FACT: an unknown key inside the English block", clone,
[(PB, en_with(' description: "Encrypted note and text sharing"',
' description: "Encrypted note and text sharing"\n docs_url: "https://example.invalid"'))],
expect_rc=1, must_contain=("unknown key",), extra_args=("--expect-missing", str(TOTAL - PB_EN)))
case_copy("FACT: an English deploy field with no Hungarian twin", clone,
[(PB, en_with(" - env_var: DOMAIN", " - env_var: NOSUCHFIELD"))],
expect_rc=1, must_contain=("no Hungarian twin", "NOSUCHFIELD"))
case_copy("FACT: an accented Hungarian letter left in the English", clone,
[(PB, en_with("Share sensitive text safely", "Érzékeny text sharing"))],
expect_rc=1, must_contain=("accented Hungarian letter",))
case_copy("FACT: ASCII-only Hungarian left in the English (no accent to find)", clone,
[(PB, en_with(' description: "The address this app answers on"',
' description: "Aldomain for the app"'))],
expect_rc=1, must_contain=("ASCII-only Hungarian", "aldomain"))
case_copy("FACT: the product begs (\"please\")", clone,
[(PB, en_with("Type your text and select Send", "Please type your text and select Send"))],
expect_rc=1, must_contain=("does not beg",))
case_copy("FACT: an English retrieval promise the Hungarian never made", clone,
[(PB, en_with("Password protection for extra safety",
"Deleted notes can still be restored later"))],
expect_rc=1, must_contain=("retrieval promise",))
# A credential is a LOGIN, not prose: gokapi's default_creds carries admin / adminadmin.
GK = "templates/gokapi/.felhom.yml"
case_copy("FACT: a credential token rewritten in translation", clone,
[(GK, lambda t: t.rstrip("\n") + """
i18n:
en:
app_info:
default_creds: "Sign in: administrator / hunter2"
""")],
expect_rc=1, must_contain=("credential token",))
case_copy("FACT: an i18n block for a language the controller does not render", clone,
[(PB, lambda t: t.rstrip("\n") + "\ni18n:\n de:\n description: \"Verschluesselte Notizen\"\n")],
expect_rc=1, must_contain=("renders en only",))
case_copy("FACT: an English list with a different number of steps", clone,
[(PB, en_with(" - 'Share the link you get - the encryption key is inside the URL'\n", ""))],
expect_rc=1, must_contain=("a list is replaced",))
# THE RATCHET — the one thing --expect-missing does not test for the cases above, so it is
# tested here explicitly, in BOTH directions. A ceiling that only convicts upwards can be
# left behind by a push that translated more than it recorded.
case_copy("FACT: ratchet — fewer strings translated than the ceiling records", clone,
[(PB, add_en())], expect_rc=1,
must_contain=("English coverage", "ABOVE"),
extra_args=("--expect-missing", "0"))
case_copy("FACT: ratchet — more translated than the ceiling records", clone,
[(PB, add_en())], expect_rc=1,
must_contain=("English coverage", "BELOW"),
extra_args=("--expect-missing", "999999"))
# THE GENUINE ARTICLE — must pass.
case_copy("GENUINE: a correct English block on privatebin", clone,
[(PB, add_en())], expect_rc=0,
must_contain=("copy-i18n: OK", "privatebin 14/14"))
# The ratchet is a fact about the CATALOG, not about how the gate was invoked. Naming one
# app must not change the count — the first version of this gate counted coverage only for
# the apps in scope, so `check-copy-i18n.py privatebin` reported 47 more missing strings
# than the same tree unscoped, and either number could have been made to "pass".
case_copy("GENUINE: naming an app does not change the coverage count", clone,
[(PB, add_en())], expect_rc=0,
must_contain=("copy-i18n: OK",),
extra_args=("privatebin",))
# DEGRADED MODE — what CI actually runs, because its runner has no PyYAML.
case_copy_noyaml("FACT(no-yaml): a Hungarian byte changed in a frozen string", clone,
[(PB, lambda t: t.replace("Titkosított jegyzet és szöveg megosztás",
"Titkosított jegyzet- és szövegmegosztás"))],
expect_rc=1, must_contain=("DEGRADED", "no longer in the file", "privatebin"))
case_copy_noyaml("FACT(no-yaml): a frozen Hungarian line deleted", clone,
[(PB, lambda t: t.replace(" - 'Oszd meg a generált linket - a titkosítási kulcs az URL-ben van'\n", ""))],
expect_rc=1, must_contain=("no longer in the file",))
case_copy_noyaml("FACT(no-yaml): a NEW app is not in the freeze", clone,
[("templates/decoyapp2/.felhom.yml",
lambda t: 'display_name: "Decoy"\ndescription: "Uj"\nslug: decoyapp2\n')],
expect_rc=1, must_contain=("not in the freeze",))
case_copy_noyaml("GENUINE(no-yaml): the untouched tree passes, and SAYS what it did not check",
clone, [("README.md", lambda t: t + "\n<!-- decoy: a harmless line -->\n")],
expect_rc=0,
must_contain=("DEGRADED", "OK (degraded", "NOT checked here"))
# The escaped-quote pair: romm's two help_texts whose YAML escapes an inner double quote.
# The degraded check must find them anyway — if it cannot, it convicts an honest tree.
case_copy_noyaml("GENUINE(no-yaml): romm's escaped-quote help_texts are still found", clone,
[("templates/romm/docker-compose.yml", lambda t: t + "\n# decoy comment\n")],
expect_rc=0, must_contain=("OK (degraded",))
# THE DECOYS — the LABEL moves, the FACT does not. Each must pass.
case_copy("DECOY: Hungarian rewritten inside a YAML COMMENT", clone,
[(PB, lambda t: t.replace("# --- App info (info page content) ---",
"# --- Alkalmazas informacio: Titkosított jegyzet MEGVALTOZOTT ---"))],
expect_rc=0, must_contain=("copy-i18n: OK",))
case_copy("DECOY: a frozen Hungarian sentence pasted into README.md", clone,
[("README.md", lambda t: t + "\nTitkositott jegyzet es szoveg megosztas (decoy)\n")],
expect_rc=0, must_contain=("copy-i18n: OK",))
case_copy("DECOY: display_name changed - a NAME, never copy", clone,
[(PB, lambda t: t.replace('display_name: "PrivateBin"', 'display_name: "PrivateBin 2"'))],
expect_rc=0, must_contain=("copy-i18n: OK",))
case_copy("DECOY: docs_url changed - configuration, never copy", clone,
[(PB, lambda t: t.replace("https://github.com/PrivateBin/PrivateBin/wiki",
"https://example.invalid/wiki"))],
expect_rc=0, must_contain=("copy-i18n: OK",))
case_copy("DECOY: Hungarian text added to a docker-compose.yml", clone,
[("templates/privatebin/docker-compose.yml",
lambda t: t.replace("services:", "# Titkosított jegyzet és szöveg megosztás\nservices:", 1))],
expect_rc=0, must_contain=("copy-i18n: OK",))
# ── probe-matches-compose (R-618) ────────────────────────────────────────────────────────
# The FACT is where the app LISTENS inside its container. The LABEL is every other number
# in the file that looks like a port: the traefik label, `ports:`, `expose:`, a comment,
# and a sidecar's own healthcheck. A gate that reads any of those would have passed the
# three templates that stopped a working app on 2026-09-21.
BS = "templates/bookstack/.felhom.yml"
BSC = "templates/bookstack/docker-compose.yml"
print("\n-- probe-matches-compose: the facts (each MUST be refused)")
# The three real faults, RE-INTRODUCED rather than read off the tree. Reading them off the
# tree passed only while the tree was broken; the case would have gone green for the wrong
# reason the moment R-618 was fixed, which is the `constant-for-measurement` shape again.
case_probe("FACT: tandoor's real R-618 port fault, re-introduced", clone,
[("templates/tandoor/.felhom.yml", lambda t: t.replace(" port: 80\n",
" port: 8080\n"))],
expect_rc=1, must_contain=("FAIL tandoor", "Nothing listens on 8080"),
apps=("tandoor",))
case_probe("FACT: wger's real R-618 port fault, re-introduced", clone,
[("templates/wger/.felhom.yml", lambda t: t.replace(" port: 8000\n",
" port: 80\n"))],
expect_rc=1, must_contain=("FAIL wger", "dials 8000 on loopback"),
apps=("wger",))
case_probe("FACT: zipline's real R-618 path fault, re-introduced", clone,
[("templates/zipline/.felhom.yml",
lambda t: t.replace('path: "/api/healthcheck"', 'path: "/api/health"'))],
expect_rc=1, must_contain=("FAIL zipline", "This probe CAN fail on it"),
apps=("zipline",))
case_probe("GENUINE: the fixed tree passes", clone, [], expect_rc=0,
must_contain=("probe-matches-compose: OK",),
apps=("tandoor", "zipline", "wger"))
case_probe("FACT: a clean app given a wrong probe port", clone,
[(BS, lambda t: t.replace("port: 80", "port: 8080"))],
expect_rc=1, must_contain=("FAIL bookstack", "dials 80 on loopback"),
apps=("bookstack",))
case_probe("FACT: api+expect probe given a path the app does not answer", clone,
[(BS, lambda t: t.replace(" - type: http\n port: 80",
" - type: api\n port: 80\n"
" path: /status\n expect:\n"
" status: 200"))],
expect_rc=1, must_contain=("FAIL bookstack", "This probe CAN fail on it"),
apps=("bookstack",))
print("-- probe-matches-compose: the decoys (the label moves, the fact does not)")
case_probe("DECOY: the port moves in a COMMENT in .felhom.yml", clone,
[(BS, lambda t: t.replace("healthcheck:",
"# the app listens on 8080 (decoy comment)\nhealthcheck:"))],
expect_rc=0, must_contain=("probe-matches-compose: OK",), apps=("bookstack",))
case_probe("DECOY: traefik loadbalancer.server.port changed", clone,
[(BSC, lambda t: t.replace("loadbalancer.server.port=80",
"loadbalancer.server.port=9999"))],
expect_rc=0, must_contain=("probe-matches-compose: OK",), apps=("bookstack",))
case_probe("DECOY: a published `ports:` mapping changed", clone,
[(BSC, lambda t: t.replace(" container_name: bookstack\n",
" container_name: bookstack\n"
" ports:\n - \"9999:80\"\n", 1))],
expect_rc=0, must_contain=("probe-matches-compose: OK",), apps=("bookstack",))
case_probe("DECOY: a NON-probed sidecar's healthcheck port changed", clone,
[(BSC, lambda t: t.replace(
'["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]',
'["CMD", "curl", "-f", "http://127.0.0.1:7777/"]'))],
expect_rc=0, must_contain=("probe-matches-compose: OK",), apps=("bookstack",))
case_probe("DECOY: path mismatch on a probe that CANNOT fail on it -> WARN, not FAIL", clone,
[], expect_rc=0,
must_contain=("WARN home-assistant", "Harmless TODAY"),
apps=("home-assistant",))
finally:
shutil.rmtree(clone, ignore_errors=True)
if fails:
print()
for f in fails:
print("FAIL: %s" % f)
return 1
print("\ncatalog gate decoys OK — %d case(s), every label judged on its fact (R-421)" % ran)
return 0
if __name__ == "__main__":
sys.exit(main())
Reference in New Issue View Git Blame Copy Permalink