admin
6faf432e22
gates / gates (push) Successful in 2s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
666 lines
35 KiB
Python
666 lines
35 KiB
Python
#!/usr/bin/env python3
|
||
# -*- coding: utf-8 -*-
|
||
"""check-copy-i18n.py — the catalog's copy is frozen in Hungarian and sound in English.
|
||
|
||
Run from the repo root (or through `scripts/catalog_gates.py`, which is the entry point):
|
||
|
||
python3 scripts/check-copy-i18n.py # every app
|
||
python3 scripts/check-copy-i18n.py privatebin # named apps only (FREEZE still runs on all)
|
||
python3 scripts/check-copy-i18n.py --capture-freeze # (re)write the freeze from disk
|
||
python3 scripts/check-copy-i18n.py --add-app NAME --reason "..." # admit a NEW app
|
||
|
||
Exit 0 clean · 1 convicted · 2 INCONCLUSIVE (freeze or templates missing) — never a pass.
|
||
|
||
────────────────────────────────────────────────────────────────────────────────────────────────
|
||
WHY THIS GATE EXISTS (localisation slice 5, R-560; design: felhom.eu/.../10-localisation.md §7).
|
||
|
||
The catalog carries 1 032 customer-facing strings across 53 apps — 832 of them with a Hungarian
|
||
letter in them. Slice 5 adds an English twin for each, as an `i18n: {en: …}` block inside the SAME
|
||
`.felhom.yml`. Two things can go wrong, and neither is visible by reading a diff:
|
||
|
||
1. **A Hungarian byte moves.** The product's first rule is that a household who never switches
|
||
language cannot tell a localisation release happened (§1). A translator "fixing a typo while
|
||
they are in there" breaks that silently — the Hungarian page renders, it just renders something
|
||
nobody signed off. CHECK 1 compares every Hungarian copy string against `copy_freeze/hu.json`,
|
||
captured before the first translation.
|
||
|
||
2. **The English is not English, or is not the same app.** An accented word left behind, a
|
||
„Jelentkezz be" that has no accents and therefore hides from every accent-based search, an
|
||
`env_var` invented in the translation, a credential rewritten into something that does not log
|
||
in, a retrieval promise the Hungarian never made. CHECKS 2-4.
|
||
|
||
WHAT IS *NOT* CHECKED, AND IS THE REVIEWER'S JOB: whether the English says the same thing as the
|
||
Hungarian, and whether an app's „first steps" match that app's real English screens. No gate can
|
||
answer either. The second is listed per app in the session REPORT as "unverified UI labels".
|
||
|
||
HUNGARIAN IS MATCHED BY ASCII-FOLDED STEMS (the workspace rule — an accented pattern returns a false
|
||
0 through an ssh/pct chain), and every run prints a POSITIVE and a NEGATIVE control for the matcher.
|
||
"""
|
||
import io
|
||
import json
|
||
import os
|
||
import re
|
||
import sys
|
||
import unicodedata
|
||
|
||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||
TEMPLATES = os.path.join(ROOT, "templates")
|
||
FREEZE_PATH = os.path.join(ROOT, "scripts", "copy_freeze", "hu.json")
|
||
|
||
|
||
def use_root(path):
|
||
"""Point the gate at another checkout. `scripts/test_gate_decoys.py` runs THIS script against a
|
||
scratch clone; without this it would read the real tree and judge the wrong files — the
|
||
"constant-for-measurement" decoy shape, committed by the gate itself."""
|
||
global ROOT, TEMPLATES, FREEZE_PATH
|
||
ROOT = os.path.abspath(path)
|
||
TEMPLATES = os.path.join(ROOT, "templates")
|
||
FREEZE_PATH = os.path.join(ROOT, "scripts", "copy_freeze", "hu.json")
|
||
|
||
SUPPORTED_LANGS = ("en",)
|
||
|
||
# ── The ratchet ──────────────────────────────────────────────────────────────────────────────────
|
||
#
|
||
# How many copy strings still have no English. It convicts ABOVE (a push that translated nothing it
|
||
# claimed to) *and* BELOW (a push that translated more than it lowered the ceiling for) — a ratchet
|
||
# that cannot be loosened by forgetting it, the same shape as the controller's EN_MISSING_CEILING
|
||
# and HU_FORMAL_CEILING.
|
||
#
|
||
# Measured on 94bc5febaca2, before any translation: 1 032 copy strings, none with English.
|
||
# 1032 → 943 (pilot) → 624 (batch 1) → 307 (batch 2) → 1 (batch 3, 16 apps) — ALL 53 APPS DONE
|
||
#
|
||
# THE FLOOR IS 1, NOT 0, UNTIL R-593 IS FIXED. papra's `deploy_fields[AUTH_SECRET].description`
|
||
# is a Hungarian DEFECT — it describes a session-signing key as „the app's subdomain". A
|
||
# localisation release may not change Hungarian bytes, and translating a wrong sentence faithfully
|
||
# would ship the error in a second language, so that ONE field is deliberately left untranslated
|
||
# and falls back to the Hungarian. Fixing R-593 is what lets this reach 0.
|
||
EN_MISSING_CEILING = 1
|
||
|
||
# ── What counts as COPY ──────────────────────────────────────────────────────────────────────────
|
||
#
|
||
# Measured, not assumed: a YAML walk of all 53 files on 94bc5febaca2 found copy in exactly these
|
||
# places and nowhere else. `display_name` is NOT copy — an app's name is not translated (operator
|
||
# ruling 7). `docs_url`, `help_url`, every `env_var`, `type`, `default`, `generate`, `path`, `role`,
|
||
# image, port and healthcheck field is configuration.
|
||
#
|
||
# NOTE against the task spec that commissioned this gate: it listed `deploy_fields[].placeholder`
|
||
# nowhere, and there are 13 of them (all `/mnt/felhom-drives/hdd_1`, no Hungarian). They are copy —
|
||
# a placeholder is shown to the customer — so they are frozen and translatable here.
|
||
|
||
|
||
def fold(s):
|
||
s = unicodedata.normalize("NFKD", s)
|
||
return "".join(c for c in s if not unicodedata.combining(c)).lower()
|
||
|
||
|
||
HU_LETTER = re.compile(r"[áéíóöőúüűÁÉÍÓÖŐÚÜŰ]")
|
||
|
||
# ASCII-only Hungarian. THIS LIST IS THE REASON THE GATE IS NOT JUST AN ACCENT SCAN: 200 of the
|
||
# catalog's 1 032 copy strings carry no accent at all, and ~120 of those are Hungarian —
|
||
# „Aldomain" (53×), „A szerver domain neve" (53×), „Jelentkezz be: …" (5×), „Magyar", „Angol",
|
||
# „Titkos kulcs", „Oszd meg a linket", „Csatlakoztasd …". An accent-only check passes every one of
|
||
# them in an English block. (The task spec said there were THREE such strings, „Igen"/„Nem"/„Nincs";
|
||
# measured, those three do not occur in the catalog at all and the real count is ~120.)
|
||
HU_ASCII_STEMS = [
|
||
"aldomain", "a szerver domain", "jelentkezz be", "oszd meg", "csatlakoztasd", "nyisd meg",
|
||
"ird be", "allitsd be", "kattints", "hozz letre", "valaszd", "masold", "regisztralj",
|
||
"magyar", "angol", "titkos kulcs", "felhasznalonev", "jelszo", "nyelv", "kompatibilis",
|
||
"beallitas", "alkalmazas", "szerver", "fajl", "mappa", "megosztas", "mentes",
|
||
]
|
||
|
||
# Second person, plain — no begging. Same rule and same regex as the controller's bundle gate.
|
||
EN_FORBIDDEN = re.compile(r"\b(please|kindly)\b", re.I)
|
||
|
||
# The retrieval promise, English half. Copied from felhom.eu/scripts/customer_copy_vocab.py
|
||
# (RETRIEVAL_STEMS_EN) rather than imported: this repo has no dependency on the sibling clone and a
|
||
# gate that is INCONCLUSIVE whenever a sibling is missing gets bypassed. DRIFT IS CHECKED — the
|
||
# gate reads the sibling when it is present and fails if the two lists disagree.
|
||
_ADV = r"(?:\s+\w+ly|\s+still|\s+always|\s+then)?"
|
||
RETRIEVAL_STEMS_EN = [
|
||
r"\b(?:can|could|will\s+be\s+able\s+to|are\s+able\s+to|is\s+able\s+to)" + _ADV +
|
||
r"(?:\s+be)?" + _ADV + r"\s+(?:restor|recover|retriev|un-?seal|open)",
|
||
r"\b(?:can|could|will)" + _ADV + r"(?:\s+be\s+able\s+to)?" + _ADV +
|
||
r"\s+(?:get|bring|have)\s+\w+\s+back",
|
||
r"\b(?:restorable|recoverable|retrievable)\b",
|
||
r"\b(?:are|is|remain|remains|stay|stays)" + _ADV + r"\s+(?:restor|recover|retriev)able",
|
||
]
|
||
RETRIEVAL_STEMS_HU = ["visszaállíthat", "visszaszerezhet", "visszahozhat", "visszanyit"]
|
||
|
||
# A credential token must survive translation byte for byte — the words around it are copy, the
|
||
# value is a login. Anything with an @, a slash, an underscore or a digit in it, plus the four
|
||
# account words that are values rather than prose.
|
||
CRED_TOKEN = re.compile(r"[^\s,;]*[@/_0-9][^\s,;]*")
|
||
CRED_WORDS = ("admin", "root", "password", "user")
|
||
|
||
|
||
# ── The walker ───────────────────────────────────────────────────────────────────────────────────
|
||
|
||
def copy_strings(meta):
|
||
"""Every customer-facing string of one parsed .felhom.yml, as {dotted path: value}.
|
||
|
||
The path is the KEY-MATCHED address the controller's overlay uses — `deploy_fields[SUBDOMAIN]`,
|
||
not `deploy_fields[1]` — so inserting a field above another does not renumber the freeze and
|
||
produce 40 spurious convictions.
|
||
"""
|
||
out = {}
|
||
|
||
def add(path, val):
|
||
if isinstance(val, str):
|
||
out[path] = val
|
||
|
||
add("description", meta.get("description"))
|
||
ai = meta.get("app_info") or {}
|
||
add("app_info.tagline", ai.get("tagline"))
|
||
add("app_info.default_creds", ai.get("default_creds"))
|
||
add("app_info.add_people", ai.get("add_people")) # decision 47 (controller >= 0.281.0): how to add a family member
|
||
for k in ("use_cases", "first_steps", "prerequisites"):
|
||
for i, v in enumerate(ai.get(k) or []):
|
||
add("app_info.%s[%d]" % (k, i), v)
|
||
for i, f in enumerate(meta.get("deploy_fields") or []):
|
||
key = f.get("env_var") or "#%d" % i
|
||
for fld in ("label", "description", "placeholder"):
|
||
add("deploy_fields[%s].%s" % (key, fld), f.get(fld))
|
||
for o in f.get("options") or []:
|
||
add("deploy_fields[%s].options[%s].label" % (key, o.get("value")), o.get("label"))
|
||
for g in meta.get("optional_config") or []:
|
||
gk = g.get("group") or "#"
|
||
add("optional_config[%s].group" % gk, g.get("group"))
|
||
add("optional_config[%s].description" % gk, g.get("description"))
|
||
for f in g.get("fields") or []:
|
||
key = f.get("env_var") or "#"
|
||
for fld in ("label", "help_text"):
|
||
add("optional_config[%s].fields[%s].%s" % (gk, key, fld), f.get(fld))
|
||
for ig in meta.get("integrations") or []:
|
||
t = ig.get("target") or "#"
|
||
add("integrations[%s].label" % t, ig.get("label"))
|
||
add("integrations[%s].description" % t, ig.get("description"))
|
||
for dp in meta.get("data_paths") or []:
|
||
p = dp.get("path") or "#"
|
||
add("data_paths[%s].label" % p, dp.get("label"))
|
||
ic = meta.get("initial_credentials") or {}
|
||
add("initial_credentials.note", ic.get("note"))
|
||
return out
|
||
|
||
|
||
def overlay_strings(ov, hu_meta):
|
||
"""Every string of one language's `i18n` block, addressed the SAME WAY as copy_strings.
|
||
|
||
Returned as (paths_to_values, structural_errors). A structural error is an entry the controller
|
||
could not match — an `env_var` with no Hungarian twin, an unknown key, a wrong type — and it is
|
||
a conviction rather than a warning: such an entry is silently inert on a box, which is the worst
|
||
of both worlds (the translator believes it shipped).
|
||
"""
|
||
errs = []
|
||
out = {}
|
||
if not isinstance(ov, dict):
|
||
return out, ["i18n block is not a mapping"]
|
||
|
||
hu_fields = {f.get("env_var"): f for f in (hu_meta.get("deploy_fields") or [])}
|
||
hu_groups = {g.get("group"): g for g in (hu_meta.get("optional_config") or [])}
|
||
hu_targets = {i.get("target") for i in (hu_meta.get("integrations") or [])}
|
||
hu_paths = {d.get("path") for d in (hu_meta.get("data_paths") or [])}
|
||
|
||
def scalar(path, val, where):
|
||
if val is None:
|
||
return
|
||
if not isinstance(val, str):
|
||
errs.append("%s must be a string, got %s" % (where, type(val).__name__))
|
||
return
|
||
out[path] = val
|
||
|
||
for k in ov:
|
||
if k not in ("description", "app_info", "deploy_fields", "optional_config",
|
||
"integrations", "data_paths", "initial_credentials"):
|
||
errs.append("unknown key %r in the English block — only copy fields may be translated" % k)
|
||
|
||
scalar("description", ov.get("description"), "description")
|
||
|
||
ai = ov.get("app_info")
|
||
if ai is not None:
|
||
if not isinstance(ai, dict):
|
||
errs.append("app_info must be a mapping")
|
||
else:
|
||
for k in ai:
|
||
if k not in ("tagline", "default_creds", "add_people", "use_cases", "first_steps", "prerequisites"):
|
||
errs.append("unknown key app_info.%s — `docs_url` and the rest are not copy" % k)
|
||
scalar("app_info.tagline", ai.get("tagline"), "app_info.tagline")
|
||
scalar("app_info.default_creds", ai.get("default_creds"), "app_info.default_creds")
|
||
scalar("app_info.add_people", ai.get("add_people"), "app_info.add_people")
|
||
hu_ai = hu_meta.get("app_info") or {}
|
||
for k in ("use_cases", "first_steps", "prerequisites"):
|
||
lst = ai.get(k)
|
||
if lst is None:
|
||
continue
|
||
if not isinstance(lst, list):
|
||
errs.append("app_info.%s must be a list" % k)
|
||
continue
|
||
for i, v in enumerate(lst):
|
||
scalar("app_info.%s[%d]" % (k, i), v, "app_info.%s[%d]" % (k, i))
|
||
hu_len = len(hu_ai.get(k) or [])
|
||
if len(lst) != hu_len:
|
||
errs.append("app_info.%s has %d entries, Hungarian has %d — a list is replaced "
|
||
"WHOLE, so the counts must match (or the block needs a preceding "
|
||
"`# en: %s differ because …` comment)" % (k, len(lst), hu_len, k))
|
||
|
||
df = ov.get("deploy_fields")
|
||
if df is not None:
|
||
if not isinstance(df, list):
|
||
errs.append("deploy_fields must be a list")
|
||
else:
|
||
for f in df:
|
||
if not isinstance(f, dict):
|
||
errs.append("a deploy_fields entry is not a mapping")
|
||
continue
|
||
key = f.get("env_var")
|
||
if not key:
|
||
errs.append("a deploy_fields entry has no env_var — entries are matched by key")
|
||
continue
|
||
if key not in hu_fields:
|
||
errs.append("deploy_fields[%s] has no Hungarian twin — it would be INERT" % key)
|
||
continue
|
||
for k in f:
|
||
if k not in ("env_var", "label", "description", "placeholder", "options"):
|
||
errs.append("unknown key deploy_fields[%s].%s — only copy is translatable" % (key, k))
|
||
for fld in ("label", "description", "placeholder"):
|
||
scalar("deploy_fields[%s].%s" % (key, fld), f.get(fld),
|
||
"deploy_fields[%s].%s" % (key, fld))
|
||
hu_opts = {o.get("value") for o in (hu_fields[key].get("options") or [])}
|
||
for o in f.get("options") or []:
|
||
if not isinstance(o, dict) or not o.get("value"):
|
||
errs.append("deploy_fields[%s]: an option has no value" % key)
|
||
continue
|
||
if o["value"] not in hu_opts:
|
||
errs.append("deploy_fields[%s].options[%s] has no Hungarian twin"
|
||
% (key, o["value"]))
|
||
continue
|
||
scalar("deploy_fields[%s].options[%s].label" % (key, o["value"]), o.get("label"),
|
||
"deploy_fields[%s].options[%s].label" % (key, o["value"]))
|
||
|
||
oc = ov.get("optional_config")
|
||
if oc is not None:
|
||
if not isinstance(oc, list):
|
||
errs.append("optional_config must be a list")
|
||
else:
|
||
for g in oc:
|
||
if not isinstance(g, dict):
|
||
errs.append("an optional_config entry is not a mapping")
|
||
continue
|
||
mg = g.get("match_group")
|
||
if not mg:
|
||
errs.append("an optional_config entry has no match_group — a group has no other "
|
||
"identity, so the Hungarian group name it translates must be named")
|
||
continue
|
||
if mg not in hu_groups:
|
||
errs.append("optional_config match_group %r has no Hungarian twin" % mg)
|
||
continue
|
||
for k in g:
|
||
if k not in ("match_group", "group", "description", "fields"):
|
||
errs.append("unknown key optional_config[%s].%s" % (mg, k))
|
||
scalar("optional_config[%s].group" % mg, g.get("group"), "optional_config group")
|
||
scalar("optional_config[%s].description" % mg, g.get("description"), "optional_config description")
|
||
hu_f = {f.get("env_var") for f in (hu_groups[mg].get("fields") or [])}
|
||
for f in g.get("fields") or []:
|
||
if not isinstance(f, dict) or not f.get("env_var"):
|
||
errs.append("optional_config[%s]: a field has no env_var" % mg)
|
||
continue
|
||
if f["env_var"] not in hu_f:
|
||
errs.append("optional_config[%s].fields[%s] has no Hungarian twin"
|
||
% (mg, f["env_var"]))
|
||
continue
|
||
for fld in ("label", "help_text"):
|
||
scalar("optional_config[%s].fields[%s].%s" % (mg, f["env_var"], fld),
|
||
f.get(fld), "optional_config field %s" % fld)
|
||
|
||
ints = ov.get("integrations")
|
||
if ints is not None:
|
||
for i in ints if isinstance(ints, list) else []:
|
||
if not isinstance(i, dict) or not i.get("target"):
|
||
errs.append("an integrations entry has no target")
|
||
continue
|
||
if i["target"] not in hu_targets:
|
||
errs.append("integrations[%s] has no Hungarian twin" % i["target"])
|
||
continue
|
||
for fld in ("label", "description"):
|
||
scalar("integrations[%s].%s" % (i["target"], fld), i.get(fld), "integration %s" % fld)
|
||
|
||
dps = ov.get("data_paths")
|
||
if dps is not None:
|
||
for d in dps if isinstance(dps, list) else []:
|
||
if not isinstance(d, dict) or not d.get("path"):
|
||
errs.append("a data_paths entry has no path")
|
||
continue
|
||
if d["path"] not in hu_paths:
|
||
errs.append("data_paths[%s] has no Hungarian twin" % d["path"])
|
||
continue
|
||
scalar("data_paths[%s].label" % d["path"], d.get("label"), "data_path label")
|
||
|
||
icv = ov.get("initial_credentials")
|
||
if icv is not None:
|
||
if not isinstance(icv, dict):
|
||
errs.append("initial_credentials must be a mapping")
|
||
else:
|
||
for k in icv:
|
||
if k != "note":
|
||
errs.append("unknown key initial_credentials.%s — only the note is copy" % k)
|
||
scalar("initial_credentials.note", icv.get("note"), "initial_credentials.note")
|
||
|
||
return out, errs
|
||
|
||
|
||
# ── Loading ──────────────────────────────────────────────────────────────────────────────────────
|
||
|
||
def have_yaml():
|
||
"""PyYAML is NOT on the CI runner, and that is deliberate, not an oversight.
|
||
|
||
`.gitea/workflows/gates.yml` says the runner is "a host-mode container with python3 and git and
|
||
nothing else", and every gate written before this one uses only the standard library. This gate
|
||
did not, so its first six pushes each turned CI red and sent an alarm mail while the local
|
||
pre-push hook was green — the exact disagreement that mail warns about, and it outranks whatever
|
||
the push was for. See DEGRADED MODE below for what runs instead."""
|
||
try:
|
||
import yaml # noqa: F401
|
||
return True
|
||
except ImportError:
|
||
return False
|
||
|
||
|
||
def load_yaml(path):
|
||
import yaml
|
||
with io.open(path, encoding="utf-8") as fh:
|
||
return yaml.safe_load(fh)
|
||
|
||
|
||
def freeze_only_check(freeze, apps, fails):
|
||
"""DEGRADED MODE — the freeze, checked without a YAML parser.
|
||
|
||
It asserts the ONE property that matters most and needs no parse: every frozen Hungarian string
|
||
still occurs VERBATIM in that app's `.felhom.yml` bytes. A changed, reworded or deleted Hungarian
|
||
string fails here exactly as it does in full mode.
|
||
|
||
What it CANNOT do, stated rather than implied: it cannot see WHERE a string sits, so a string
|
||
moved to another field passes; and it cannot read the English block at all, so structure,
|
||
language, credentials and the coverage ratchet are not checked. Those run in the pre-push hook,
|
||
which is where they bite anyway — the same division `catalog_gates.py` already uses for
|
||
engine-major on a shallow clone.
|
||
|
||
MEASURED 2026-09-20: 1 030 of the 1 032 frozen strings appear byte-for-byte in the raw file. The
|
||
other two are romm help_texts whose YAML source escapes an inner double quote, so the escaped
|
||
form is accepted too — with that, 1 032 of 1 032 are found. The variant is NOT a loosening: it
|
||
is the same characters, written the way YAML requires inside a double-quoted scalar.
|
||
"""
|
||
for app in apps:
|
||
frozen = (freeze.get("apps") or {}).get(app)
|
||
if frozen is None:
|
||
fails.append("%s: not in the freeze. A NEW app's Hungarian has never been reviewed — "
|
||
"run `python3 scripts/check-copy-i18n.py --add-app %s --reason \"…\"` in "
|
||
"the same commit." % (app, app))
|
||
continue
|
||
raw = io.open(os.path.join(TEMPLATES, app, ".felhom.yml"), encoding="utf-8").read()
|
||
for path, val in sorted(frozen.items()):
|
||
if val in raw or val.replace('"', '\\"') in raw:
|
||
continue
|
||
fails.append("%s: the frozen Hungarian at %s is no longer in the file: %r"
|
||
% (app, path, val))
|
||
|
||
|
||
def app_dirs():
|
||
return sorted(d for d in os.listdir(TEMPLATES)
|
||
if os.path.isdir(os.path.join(TEMPLATES, d))
|
||
and os.path.exists(os.path.join(TEMPLATES, d, ".felhom.yml")))
|
||
|
||
|
||
def read_freeze():
|
||
if not os.path.exists(FREEZE_PATH):
|
||
return None
|
||
with io.open(FREEZE_PATH, encoding="utf-8") as fh:
|
||
return json.load(fh)
|
||
|
||
|
||
def write_freeze(data):
|
||
os.makedirs(os.path.dirname(FREEZE_PATH), exist_ok=True)
|
||
with io.open(FREEZE_PATH, "w", encoding="utf-8") as fh:
|
||
json.dump(data, fh, ensure_ascii=False, indent=1, sort_keys=True)
|
||
fh.write("\n")
|
||
|
||
|
||
def capture(reasons=None):
|
||
apps = {}
|
||
for app in app_dirs():
|
||
apps[app] = copy_strings(load_yaml(os.path.join(TEMPLATES, app, ".felhom.yml")))
|
||
return {
|
||
"_what": "Every customer-facing Hungarian string in the catalog, frozen. See "
|
||
"scripts/check-copy-i18n.py. A value here may only change when the Hungarian "
|
||
"itself is deliberately rewritten — never by a translation.",
|
||
"_captured_from": "app-catalog-felhom.eu@94bc5febaca2 (2026-09-20), before localisation slice 5",
|
||
"reasons": reasons or {},
|
||
"apps": apps,
|
||
}
|
||
|
||
|
||
# ── The checks ───────────────────────────────────────────────────────────────────────────────────
|
||
|
||
def check_language(app, path, val, hu_val, display_name, fails):
|
||
where = "%s / i18n.en.%s" % (app, path)
|
||
if HU_LETTER.search(val):
|
||
fails.append("%s: an accented Hungarian letter in the English text: %r" % (where, val))
|
||
folded = fold(val)
|
||
for stem in HU_ASCII_STEMS:
|
||
# WORD BOUNDARIES. A bare substring test convicts honest English: „ird be" is inside
|
||
# "the third best", „angol" is inside "Angola". The stems are words, so they are matched
|
||
# as words — the negative control below is the case that found it.
|
||
if re.search(r"(?<![a-z0-9])%s(?![a-z0-9])" % re.escape(stem), folded):
|
||
fails.append("%s: ASCII-only Hungarian %r in the English text: %r" % (where, stem, val))
|
||
break
|
||
if EN_FORBIDDEN.search(val):
|
||
fails.append("%s: the product does not beg — no \"please\"/\"kindly\": %r" % (where, val))
|
||
for pat in RETRIEVAL_STEMS_EN:
|
||
if re.search(pat, val, re.I):
|
||
hu_promises = any(s in fold(hu_val) for s in [fold(x) for x in RETRIEVAL_STEMS_HU])
|
||
if not hu_promises:
|
||
fails.append("%s: an English retrieval promise the Hungarian does not make: %r"
|
||
% (where, val))
|
||
break
|
||
if "Felhom" in hu_val and "Felhom" not in val:
|
||
fails.append("%s: the product name „Felhom\" was dropped in translation: %r" % (where, val))
|
||
if display_name and display_name in hu_val and display_name not in val:
|
||
fails.append("%s: the app name %r was dropped or translated: %r" % (where, display_name, val))
|
||
# Credentials survive verbatim — the words around them are copy, the value is a login.
|
||
if path.endswith("default_creds") or path.endswith("initial_credentials.note"):
|
||
want = {t for t in CRED_TOKEN.findall(hu_val) if len(t) >= 3}
|
||
want |= {w for w in CRED_WORDS if re.search(r"\b%s\b" % w, hu_val, re.I)}
|
||
for tok in sorted(want):
|
||
# WORD BOUNDARIES, and they are load-bearing: the first version searched for the token
|
||
# as a bare substring, and „admin" -> "administrator" passed it. A login is the whole
|
||
# word or it is a different login.
|
||
if not re.search(r"(?<![A-Za-z0-9])%s(?![A-Za-z0-9])" % re.escape(tok), val, re.I):
|
||
fails.append("%s: the credential token %r is not in the English text: %r"
|
||
% (where, tok, val))
|
||
|
||
|
||
def main(argv):
|
||
if "--root" in argv:
|
||
use_root(argv[argv.index("--root") + 1])
|
||
# --expect-missing is for `scripts/test_gate_decoys.py` ONLY: a decoy case that adds a correct
|
||
# English block would otherwise trip the ratchet and look like a conviction. The gate run by
|
||
# `catalog_gates.py` never passes it, so the ratchet is untouched in every real run.
|
||
ceiling = EN_MISSING_CEILING
|
||
if "--expect-missing" in argv:
|
||
ceiling = int(argv[argv.index("--expect-missing") + 1])
|
||
|
||
if "--capture-freeze" in argv:
|
||
old = read_freeze() or {}
|
||
write_freeze(capture(old.get("reasons")))
|
||
print("copy-i18n: freeze written to %s (%d apps)" % (FREEZE_PATH, len(app_dirs())))
|
||
return 0
|
||
|
||
if "--add-app" in argv:
|
||
i = argv.index("--add-app")
|
||
name = argv[i + 1]
|
||
if "--reason" not in argv:
|
||
print("copy-i18n: --add-app needs --reason \"…\" — a new app's Hungarian has never been "
|
||
"reviewed, and the reason is the review")
|
||
return 2
|
||
reason = argv[argv.index("--reason") + 1]
|
||
fr = read_freeze() or capture()
|
||
fr.setdefault("reasons", {})[name] = reason
|
||
fr["apps"][name] = copy_strings(load_yaml(os.path.join(TEMPLATES, name, ".felhom.yml")))
|
||
write_freeze(fr)
|
||
print("copy-i18n: %s admitted to the freeze — %s" % (name, reason))
|
||
return 0
|
||
|
||
# Option VALUES are not app names. --root and --expect-missing each consume the argument
|
||
# after them, and a scope list that swallowed a path would silently skip every language check.
|
||
consumed = set()
|
||
for opt in ("--root", "--expect-missing", "--range"):
|
||
if opt in argv:
|
||
consumed.add(argv.index(opt) + 1)
|
||
scope = [a for i, a in enumerate(argv) if i >= 1 and not a.startswith("-") and i not in consumed]
|
||
|
||
# Controls for the ASCII-folded matcher, printed every run (workspace rule).
|
||
if "jelentkezz be" not in fold("Jelentkezz be: admin"):
|
||
print("copy-i18n INCONCLUSIVE: positive control failed — folding does not strip accents")
|
||
return 2
|
||
for control in ("Encrypted notes and text sharing",
|
||
"The third best option is in Angola", # „ird be" / „angol" as substrings
|
||
"These documents are kept on the server"): # „mentes" / „szerver" as substrings
|
||
if any(re.search(r"(?<![a-z0-9])%s(?![a-z0-9])" % re.escape(st), fold(control))
|
||
for st in HU_ASCII_STEMS):
|
||
print("copy-i18n INCONCLUSIVE: negative control failed on %r — a clean English "
|
||
"sentence convicted" % control)
|
||
return 2
|
||
print("copy-i18n: matcher controls OK (positive „Jelentkezz be…\" convicts, "
|
||
"negative „Encrypted notes…\" does not)")
|
||
|
||
freeze = read_freeze()
|
||
if freeze is None:
|
||
print("copy-i18n INCONCLUSIVE: no freeze at %s — run --capture-freeze first" % FREEZE_PATH)
|
||
return 2
|
||
|
||
# Drift check against the sibling's shared vocabulary, when the clone is present.
|
||
sib = os.path.join(os.path.dirname(ROOT), "felhom.eu", "scripts")
|
||
if os.path.exists(os.path.join(sib, "customer_copy_vocab.py")):
|
||
sys.path.insert(0, sib)
|
||
try:
|
||
import customer_copy_vocab as vocab
|
||
if list(vocab.RETRIEVAL_STEMS_EN) != RETRIEVAL_STEMS_EN:
|
||
print("copy-i18n: FAIL — RETRIEVAL_STEMS_EN has drifted from "
|
||
"felhom.eu/scripts/customer_copy_vocab.py; copy it across")
|
||
return 1
|
||
print("copy-i18n: retrieval-promise vocabulary matches the sibling clone")
|
||
except Exception as exc: # pragma: no cover
|
||
print("copy-i18n: note — sibling vocabulary unreadable (%s); using the local copy" % exc)
|
||
|
||
fails = []
|
||
apps = app_dirs()
|
||
|
||
# DEGRADED MODE. Without PyYAML the file cannot be parsed, so the structure, language,
|
||
# credential and coverage checks cannot run — but the FREEZE can, and it is the check that
|
||
# protects the customer-facing Hungarian. Running it and saying loudly what did not run beats
|
||
# both alternatives: exiting 2 would paint CI red on every push (and "inconclusive is never a
|
||
# pass" would make that permanent), and exiting 0 would be a gate that reports a pass it never
|
||
# measured.
|
||
if not have_yaml():
|
||
print("copy-i18n: DEGRADED - PyYAML is absent on this machine, so only the HUNGARIAN\n"
|
||
" FREEZE runs (%d apps). NOT checked here: the English block's\n"
|
||
" structure, its language, its credential tokens, and the coverage\n"
|
||
" ratchet - those run in the pre-push hook, which is where they\n"
|
||
" refuse a push anyway." % len(apps))
|
||
freeze_only_check(freeze, apps, fails)
|
||
if fails:
|
||
print("\ncopy-i18n: FAIL — %d finding(s)\n" % len(fails))
|
||
for f in fails:
|
||
print(" - %s" % f)
|
||
return 1
|
||
print("copy-i18n: OK (degraded — freeze only)")
|
||
return 0
|
||
|
||
en_missing_total = 0
|
||
per_app = []
|
||
|
||
for app in apps:
|
||
meta = load_yaml(os.path.join(TEMPLATES, app, ".felhom.yml"))
|
||
hu = copy_strings(meta)
|
||
|
||
# CHECK 1 — the Hungarian is frozen. Runs on EVERY app whatever the scope: a scoped push
|
||
# that quietly edits a neighbour is exactly the thing worth catching.
|
||
frozen = (freeze.get("apps") or {}).get(app)
|
||
if frozen is None:
|
||
fails.append("%s: not in the freeze. A NEW app's Hungarian has never been reviewed — "
|
||
"run `python3 scripts/check-copy-i18n.py --add-app %s --reason \"…\"` in "
|
||
"the same commit." % (app, app))
|
||
else:
|
||
for path, val in sorted(hu.items()):
|
||
if path not in frozen:
|
||
fails.append("%s: NEW Hungarian string %s = %r — the freeze does not know it"
|
||
% (app, path, val))
|
||
elif frozen[path] != val:
|
||
fails.append("%s: Hungarian CHANGED at %s\n frozen: %r\n on disk: %r"
|
||
% (app, path, frozen[path], val))
|
||
for path in sorted(frozen):
|
||
if path not in hu:
|
||
fails.append("%s: Hungarian string REMOVED at %s (%r)" % (app, path, frozen[path]))
|
||
|
||
# CHECKS 2-4 — the English block.
|
||
i18n = meta.get("i18n") or {}
|
||
en_paths = set()
|
||
if i18n:
|
||
if not isinstance(i18n, dict):
|
||
fails.append("%s: i18n is not a mapping" % app)
|
||
else:
|
||
for lang in i18n:
|
||
if lang not in SUPPORTED_LANGS:
|
||
fails.append("%s: i18n.%s — the controller renders %s only"
|
||
% (app, lang, "/".join(SUPPORTED_LANGS)))
|
||
for lang in SUPPORTED_LANGS:
|
||
if lang not in i18n:
|
||
continue
|
||
vals, errs = overlay_strings(i18n[lang], meta)
|
||
for e in errs:
|
||
fails.append("%s: %s" % (app, e))
|
||
# COVERAGE IS COUNTED FOR EVERY APP, scope or not: the ratchet is a fact
|
||
# about the whole catalog, and a scoped run that counted only the named apps
|
||
# would report a number that depends on how the gate was invoked — a ratchet
|
||
# anybody could loosen by naming one app. Only the LANGUAGE checks are scoped,
|
||
# because they are slow-ish and read only what the push touched.
|
||
for path, val in sorted(vals.items()):
|
||
en_paths.add(path)
|
||
if scope and app not in scope:
|
||
continue
|
||
check_language(app, path, val, hu.get(path, ""),
|
||
meta.get("display_name") or "", fails)
|
||
|
||
missing = [p for p in hu if p not in en_paths]
|
||
en_missing_total += len(missing)
|
||
per_app.append((app, len(hu) - len(missing), len(hu)))
|
||
|
||
print("copy-i18n: %d apps · %d Hungarian copy strings frozen · %d with English"
|
||
% (len(apps), sum(t for _a, _d, t in per_app),
|
||
sum(d for _a, d, _t in per_app)))
|
||
done = [p for p in per_app if p[1]]
|
||
if done:
|
||
print("copy-i18n: translated so far — " +
|
||
", ".join("%s %d/%d" % (a, d, t) for a, d, t in done))
|
||
|
||
# CHECK 5 — the ratchet.
|
||
if en_missing_total != ceiling:
|
||
direction = "ABOVE" if en_missing_total > ceiling else "BELOW"
|
||
fails.append(
|
||
"English coverage: %d strings have no English, ceiling is %d (%s).\n"
|
||
" %s" % (en_missing_total, ceiling, direction,
|
||
"Translate the rest, or lower EN_MISSING_CEILING in this file to %d in the "
|
||
"SAME commit — the ceiling records what was actually pushed."
|
||
% en_missing_total if direction == "BELOW" else
|
||
"A push claimed translations it did not make."))
|
||
|
||
if fails:
|
||
print("\ncopy-i18n: FAIL — %d finding(s)\n" % len(fails))
|
||
for f in fails:
|
||
print(" - %s" % f)
|
||
return 1
|
||
print("copy-i18n: OK")
|
||
return 0
|
||
|
||
|
||
if __name__ == "__main__":
|
||
sys.exit(main(sys.argv))
|