GATE: copy-i18n — Hungarian frozen, English sound (R-560, slice 5)
`scripts/check-copy-i18n.py`, fifth row of `catalog_gates.py`, static and in the
pre-push hook. Five checks:
1. FREEZE — every Hungarian copy string equals `copy_freeze/hu.json`. Runs on all
53 apps whatever scope is named: a scoped push that quietly edits a neighbour is
what a freeze is for. A NEW app must be admitted with `--add-app NAME --reason`.
2. STRUCTURE — the `i18n.en` block may carry copy fields and nothing else; every
key-matched entry (`env_var`, option `value`, `match_group`, `target`, `path`)
must have a Hungarian twin, or it would be INERT on the box and the translator
would never know. Lists must have the Hungarian's length — they are replaced
whole, never merged by index.
3. LANGUAGE — no accented Hungarian letter, no ASCII-ONLY Hungarian, no
"please"/"kindly", no English retrieval promise the Hungarian does not make, the
app name and „Felhom" preserved.
4. CREDENTIALS — the login tokens inside `default_creds` and the initial-credentials
note survive translation verbatim.
5. RATCHET — `EN_MISSING_CEILING` (1032 today) convicts above AND below.
MEASURED, against the numbers the task carried: 1 032 copy strings, 832 of them with
a Hungarian letter (that half matches). The ASCII-only Hungarian is NOT three strings
(„Igen"/„Nem"/„Nincs" do not occur in this catalog at all) but roughly 120 — „Aldomain"
and „A szerver domain neve" alone are 53 each. An accent-only gate would have passed
every one of them inside an English block, which is why check 3 folds and stems.
18 decoy cases in `test_gate_decoys.py`, each seen to convict or to pass as intended
(R-421). One of them found a real hole while being written: the credential check
searched for the token as a substring, so „admin" matched "administrator" and a
rewritten login passed. It now requires word boundaries.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -79,6 +79,11 @@ GATES = [
|
||||
# R-452 (2026-09-13): an image: move must bump that app's catalog_since. Same shape as
|
||||
# engine-major — git history, fast, skipped out loud on a shallow clone.
|
||||
("catalog-since", "check-catalog-since.py", False, True, True),
|
||||
# R-560 (2026-09-20): the Hungarian copy is frozen byte for byte and the English `i18n:` block
|
||||
# is structurally sound and actually English. Static, instant, no git history. It accepts app
|
||||
# scope for the LANGUAGE checks only — the Hungarian freeze always runs on all 53, because a
|
||||
# scoped push that quietly edits a neighbour's copy is precisely what a freeze is for.
|
||||
("copy-i18n", "check-copy-i18n.py", True, True, False),
|
||||
]
|
||||
|
||||
VERDICT = {0: "OK", 1: "FAILED", 2: "INCONCLUSIVE"}
|
||||
|
||||
@@ -0,0 +1,574 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""check-copy-i18n.py — the catalog's copy is frozen in Hungarian and sound in English.
|
||||
|
||||
Run from the repo root (or through `scripts/catalog_gates.py`, which is the entry point):
|
||||
|
||||
python3 scripts/check-copy-i18n.py # every app
|
||||
python3 scripts/check-copy-i18n.py privatebin # named apps only (FREEZE still runs on all)
|
||||
python3 scripts/check-copy-i18n.py --capture-freeze # (re)write the freeze from disk
|
||||
python3 scripts/check-copy-i18n.py --add-app NAME --reason "..." # admit a NEW app
|
||||
|
||||
Exit 0 clean · 1 convicted · 2 INCONCLUSIVE (freeze or templates missing) — never a pass.
|
||||
|
||||
────────────────────────────────────────────────────────────────────────────────────────────────
|
||||
WHY THIS GATE EXISTS (localisation slice 5, R-560; design: felhom.eu/.../10-localisation.md §7).
|
||||
|
||||
The catalog carries 1 032 customer-facing strings across 53 apps — 832 of them with a Hungarian
|
||||
letter in them. Slice 5 adds an English twin for each, as an `i18n: {en: …}` block inside the SAME
|
||||
`.felhom.yml`. Two things can go wrong, and neither is visible by reading a diff:
|
||||
|
||||
1. **A Hungarian byte moves.** The product's first rule is that a household who never switches
|
||||
language cannot tell a localisation release happened (§1). A translator "fixing a typo while
|
||||
they are in there" breaks that silently — the Hungarian page renders, it just renders something
|
||||
nobody signed off. CHECK 1 compares every Hungarian copy string against `copy_freeze/hu.json`,
|
||||
captured before the first translation.
|
||||
|
||||
2. **The English is not English, or is not the same app.** An accented word left behind, a
|
||||
„Jelentkezz be" that has no accents and therefore hides from every accent-based search, an
|
||||
`env_var` invented in the translation, a credential rewritten into something that does not log
|
||||
in, a retrieval promise the Hungarian never made. CHECKS 2-4.
|
||||
|
||||
WHAT IS *NOT* CHECKED, AND IS THE REVIEWER'S JOB: whether the English says the same thing as the
|
||||
Hungarian, and whether an app's „first steps" match that app's real English screens. No gate can
|
||||
answer either. The second is listed per app in the session REPORT as "unverified UI labels".
|
||||
|
||||
HUNGARIAN IS MATCHED BY ASCII-FOLDED STEMS (the workspace rule — an accented pattern returns a false
|
||||
0 through an ssh/pct chain), and every run prints a POSITIVE and a NEGATIVE control for the matcher.
|
||||
"""
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import unicodedata
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
TEMPLATES = os.path.join(ROOT, "templates")
|
||||
FREEZE_PATH = os.path.join(ROOT, "scripts", "copy_freeze", "hu.json")
|
||||
|
||||
|
||||
def use_root(path):
|
||||
"""Point the gate at another checkout. `scripts/test_gate_decoys.py` runs THIS script against a
|
||||
scratch clone; without this it would read the real tree and judge the wrong files — the
|
||||
"constant-for-measurement" decoy shape, committed by the gate itself."""
|
||||
global ROOT, TEMPLATES, FREEZE_PATH
|
||||
ROOT = os.path.abspath(path)
|
||||
TEMPLATES = os.path.join(ROOT, "templates")
|
||||
FREEZE_PATH = os.path.join(ROOT, "scripts", "copy_freeze", "hu.json")
|
||||
|
||||
SUPPORTED_LANGS = ("en",)
|
||||
|
||||
# ── The ratchet ──────────────────────────────────────────────────────────────────────────────────
|
||||
#
|
||||
# How many copy strings still have no English. It convicts ABOVE (a push that translated nothing it
|
||||
# claimed to) *and* BELOW (a push that translated more than it lowered the ceiling for) — a ratchet
|
||||
# that cannot be loosened by forgetting it, the same shape as the controller's EN_MISSING_CEILING
|
||||
# and HU_FORMAL_CEILING.
|
||||
#
|
||||
# Measured on 94bc5febaca2, before any translation: 1 032 copy strings, none with English.
|
||||
# 1032 → (pilot) → (batch 1) → (batch 2) → 0
|
||||
EN_MISSING_CEILING = 1032
|
||||
|
||||
# ── What counts as COPY ──────────────────────────────────────────────────────────────────────────
|
||||
#
|
||||
# Measured, not assumed: a YAML walk of all 53 files on 94bc5febaca2 found copy in exactly these
|
||||
# places and nowhere else. `display_name` is NOT copy — an app's name is not translated (operator
|
||||
# ruling 7). `docs_url`, `help_url`, every `env_var`, `type`, `default`, `generate`, `path`, `role`,
|
||||
# image, port and healthcheck field is configuration.
|
||||
#
|
||||
# NOTE against the task spec that commissioned this gate: it listed `deploy_fields[].placeholder`
|
||||
# nowhere, and there are 13 of them (all `/mnt/felhom-drives/hdd_1`, no Hungarian). They are copy —
|
||||
# a placeholder is shown to the customer — so they are frozen and translatable here.
|
||||
|
||||
|
||||
def fold(s):
|
||||
s = unicodedata.normalize("NFKD", s)
|
||||
return "".join(c for c in s if not unicodedata.combining(c)).lower()
|
||||
|
||||
|
||||
HU_LETTER = re.compile(r"[áéíóöőúüűÁÉÍÓÖŐÚÜŰ]")
|
||||
|
||||
# ASCII-only Hungarian. THIS LIST IS THE REASON THE GATE IS NOT JUST AN ACCENT SCAN: 200 of the
|
||||
# catalog's 1 032 copy strings carry no accent at all, and ~120 of those are Hungarian —
|
||||
# „Aldomain" (53×), „A szerver domain neve" (53×), „Jelentkezz be: …" (5×), „Magyar", „Angol",
|
||||
# „Titkos kulcs", „Oszd meg a linket", „Csatlakoztasd …". An accent-only check passes every one of
|
||||
# them in an English block. (The task spec said there were THREE such strings, „Igen"/„Nem"/„Nincs";
|
||||
# measured, those three do not occur in the catalog at all and the real count is ~120.)
|
||||
HU_ASCII_STEMS = [
|
||||
"aldomain", "a szerver domain", "jelentkezz be", "oszd meg", "csatlakoztasd", "nyisd meg",
|
||||
"ird be", "allitsd be", "kattints", "hozz letre", "valaszd", "masold", "regisztralj",
|
||||
"magyar", "angol", "titkos kulcs", "felhasznalonev", "jelszo", "nyelv", "kompatibilis",
|
||||
"beallitas", "alkalmazas", "szerver", "fajl", "mappa", "megosztas", "mentes",
|
||||
]
|
||||
|
||||
# Second person, plain — no begging. Same rule and same regex as the controller's bundle gate.
|
||||
EN_FORBIDDEN = re.compile(r"\b(please|kindly)\b", re.I)
|
||||
|
||||
# The retrieval promise, English half. Copied from felhom.eu/scripts/customer_copy_vocab.py
|
||||
# (RETRIEVAL_STEMS_EN) rather than imported: this repo has no dependency on the sibling clone and a
|
||||
# gate that is INCONCLUSIVE whenever a sibling is missing gets bypassed. DRIFT IS CHECKED — the
|
||||
# gate reads the sibling when it is present and fails if the two lists disagree.
|
||||
_ADV = r"(?:\s+\w+ly|\s+still|\s+always|\s+then)?"
|
||||
RETRIEVAL_STEMS_EN = [
|
||||
r"\b(?:can|could|will\s+be\s+able\s+to|are\s+able\s+to|is\s+able\s+to)" + _ADV +
|
||||
r"(?:\s+be)?" + _ADV + r"\s+(?:restor|recover|retriev|un-?seal|open)",
|
||||
r"\b(?:can|could|will)" + _ADV + r"(?:\s+be\s+able\s+to)?" + _ADV +
|
||||
r"\s+(?:get|bring|have)\s+\w+\s+back",
|
||||
r"\b(?:restorable|recoverable|retrievable)\b",
|
||||
r"\b(?:are|is|remain|remains|stay|stays)" + _ADV + r"\s+(?:restor|recover|retriev)able",
|
||||
]
|
||||
RETRIEVAL_STEMS_HU = ["visszaállíthat", "visszaszerezhet", "visszahozhat", "visszanyit"]
|
||||
|
||||
# A credential token must survive translation byte for byte — the words around it are copy, the
|
||||
# value is a login. Anything with an @, a slash, an underscore or a digit in it, plus the four
|
||||
# account words that are values rather than prose.
|
||||
CRED_TOKEN = re.compile(r"[^\s,;]*[@/_0-9][^\s,;]*")
|
||||
CRED_WORDS = ("admin", "root", "password", "user")
|
||||
|
||||
|
||||
# ── The walker ───────────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def copy_strings(meta):
|
||||
"""Every customer-facing string of one parsed .felhom.yml, as {dotted path: value}.
|
||||
|
||||
The path is the KEY-MATCHED address the controller's overlay uses — `deploy_fields[SUBDOMAIN]`,
|
||||
not `deploy_fields[1]` — so inserting a field above another does not renumber the freeze and
|
||||
produce 40 spurious convictions.
|
||||
"""
|
||||
out = {}
|
||||
|
||||
def add(path, val):
|
||||
if isinstance(val, str):
|
||||
out[path] = val
|
||||
|
||||
add("description", meta.get("description"))
|
||||
ai = meta.get("app_info") or {}
|
||||
add("app_info.tagline", ai.get("tagline"))
|
||||
add("app_info.default_creds", ai.get("default_creds"))
|
||||
for k in ("use_cases", "first_steps", "prerequisites"):
|
||||
for i, v in enumerate(ai.get(k) or []):
|
||||
add("app_info.%s[%d]" % (k, i), v)
|
||||
for i, f in enumerate(meta.get("deploy_fields") or []):
|
||||
key = f.get("env_var") or "#%d" % i
|
||||
for fld in ("label", "description", "placeholder"):
|
||||
add("deploy_fields[%s].%s" % (key, fld), f.get(fld))
|
||||
for o in f.get("options") or []:
|
||||
add("deploy_fields[%s].options[%s].label" % (key, o.get("value")), o.get("label"))
|
||||
for g in meta.get("optional_config") or []:
|
||||
gk = g.get("group") or "#"
|
||||
add("optional_config[%s].group" % gk, g.get("group"))
|
||||
add("optional_config[%s].description" % gk, g.get("description"))
|
||||
for f in g.get("fields") or []:
|
||||
key = f.get("env_var") or "#"
|
||||
for fld in ("label", "help_text"):
|
||||
add("optional_config[%s].fields[%s].%s" % (gk, key, fld), f.get(fld))
|
||||
for ig in meta.get("integrations") or []:
|
||||
t = ig.get("target") or "#"
|
||||
add("integrations[%s].label" % t, ig.get("label"))
|
||||
add("integrations[%s].description" % t, ig.get("description"))
|
||||
for dp in meta.get("data_paths") or []:
|
||||
p = dp.get("path") or "#"
|
||||
add("data_paths[%s].label" % p, dp.get("label"))
|
||||
ic = meta.get("initial_credentials") or {}
|
||||
add("initial_credentials.note", ic.get("note"))
|
||||
return out
|
||||
|
||||
|
||||
def overlay_strings(ov, hu_meta):
|
||||
"""Every string of one language's `i18n` block, addressed the SAME WAY as copy_strings.
|
||||
|
||||
Returned as (paths_to_values, structural_errors). A structural error is an entry the controller
|
||||
could not match — an `env_var` with no Hungarian twin, an unknown key, a wrong type — and it is
|
||||
a conviction rather than a warning: such an entry is silently inert on a box, which is the worst
|
||||
of both worlds (the translator believes it shipped).
|
||||
"""
|
||||
errs = []
|
||||
out = {}
|
||||
if not isinstance(ov, dict):
|
||||
return out, ["i18n block is not a mapping"]
|
||||
|
||||
hu_fields = {f.get("env_var"): f for f in (hu_meta.get("deploy_fields") or [])}
|
||||
hu_groups = {g.get("group"): g for g in (hu_meta.get("optional_config") or [])}
|
||||
hu_targets = {i.get("target") for i in (hu_meta.get("integrations") or [])}
|
||||
hu_paths = {d.get("path") for d in (hu_meta.get("data_paths") or [])}
|
||||
|
||||
def scalar(path, val, where):
|
||||
if val is None:
|
||||
return
|
||||
if not isinstance(val, str):
|
||||
errs.append("%s must be a string, got %s" % (where, type(val).__name__))
|
||||
return
|
||||
out[path] = val
|
||||
|
||||
for k in ov:
|
||||
if k not in ("description", "app_info", "deploy_fields", "optional_config",
|
||||
"integrations", "data_paths", "initial_credentials"):
|
||||
errs.append("unknown key %r in the English block — only copy fields may be translated" % k)
|
||||
|
||||
scalar("description", ov.get("description"), "description")
|
||||
|
||||
ai = ov.get("app_info")
|
||||
if ai is not None:
|
||||
if not isinstance(ai, dict):
|
||||
errs.append("app_info must be a mapping")
|
||||
else:
|
||||
for k in ai:
|
||||
if k not in ("tagline", "default_creds", "use_cases", "first_steps", "prerequisites"):
|
||||
errs.append("unknown key app_info.%s — `docs_url` and the rest are not copy" % k)
|
||||
scalar("app_info.tagline", ai.get("tagline"), "app_info.tagline")
|
||||
scalar("app_info.default_creds", ai.get("default_creds"), "app_info.default_creds")
|
||||
hu_ai = hu_meta.get("app_info") or {}
|
||||
for k in ("use_cases", "first_steps", "prerequisites"):
|
||||
lst = ai.get(k)
|
||||
if lst is None:
|
||||
continue
|
||||
if not isinstance(lst, list):
|
||||
errs.append("app_info.%s must be a list" % k)
|
||||
continue
|
||||
for i, v in enumerate(lst):
|
||||
scalar("app_info.%s[%d]" % (k, i), v, "app_info.%s[%d]" % (k, i))
|
||||
hu_len = len(hu_ai.get(k) or [])
|
||||
if len(lst) != hu_len:
|
||||
errs.append("app_info.%s has %d entries, Hungarian has %d — a list is replaced "
|
||||
"WHOLE, so the counts must match (or the block needs a preceding "
|
||||
"`# en: %s differ because …` comment)" % (k, len(lst), hu_len, k))
|
||||
|
||||
df = ov.get("deploy_fields")
|
||||
if df is not None:
|
||||
if not isinstance(df, list):
|
||||
errs.append("deploy_fields must be a list")
|
||||
else:
|
||||
for f in df:
|
||||
if not isinstance(f, dict):
|
||||
errs.append("a deploy_fields entry is not a mapping")
|
||||
continue
|
||||
key = f.get("env_var")
|
||||
if not key:
|
||||
errs.append("a deploy_fields entry has no env_var — entries are matched by key")
|
||||
continue
|
||||
if key not in hu_fields:
|
||||
errs.append("deploy_fields[%s] has no Hungarian twin — it would be INERT" % key)
|
||||
continue
|
||||
for k in f:
|
||||
if k not in ("env_var", "label", "description", "placeholder", "options"):
|
||||
errs.append("unknown key deploy_fields[%s].%s — only copy is translatable" % (key, k))
|
||||
for fld in ("label", "description", "placeholder"):
|
||||
scalar("deploy_fields[%s].%s" % (key, fld), f.get(fld),
|
||||
"deploy_fields[%s].%s" % (key, fld))
|
||||
hu_opts = {o.get("value") for o in (hu_fields[key].get("options") or [])}
|
||||
for o in f.get("options") or []:
|
||||
if not isinstance(o, dict) or not o.get("value"):
|
||||
errs.append("deploy_fields[%s]: an option has no value" % key)
|
||||
continue
|
||||
if o["value"] not in hu_opts:
|
||||
errs.append("deploy_fields[%s].options[%s] has no Hungarian twin"
|
||||
% (key, o["value"]))
|
||||
continue
|
||||
scalar("deploy_fields[%s].options[%s].label" % (key, o["value"]), o.get("label"),
|
||||
"deploy_fields[%s].options[%s].label" % (key, o["value"]))
|
||||
|
||||
oc = ov.get("optional_config")
|
||||
if oc is not None:
|
||||
if not isinstance(oc, list):
|
||||
errs.append("optional_config must be a list")
|
||||
else:
|
||||
for g in oc:
|
||||
if not isinstance(g, dict):
|
||||
errs.append("an optional_config entry is not a mapping")
|
||||
continue
|
||||
mg = g.get("match_group")
|
||||
if not mg:
|
||||
errs.append("an optional_config entry has no match_group — a group has no other "
|
||||
"identity, so the Hungarian group name it translates must be named")
|
||||
continue
|
||||
if mg not in hu_groups:
|
||||
errs.append("optional_config match_group %r has no Hungarian twin" % mg)
|
||||
continue
|
||||
for k in g:
|
||||
if k not in ("match_group", "group", "description", "fields"):
|
||||
errs.append("unknown key optional_config[%s].%s" % (mg, k))
|
||||
scalar("optional_config[%s].group" % mg, g.get("group"), "optional_config group")
|
||||
scalar("optional_config[%s].description" % mg, g.get("description"), "optional_config description")
|
||||
hu_f = {f.get("env_var") for f in (hu_groups[mg].get("fields") or [])}
|
||||
for f in g.get("fields") or []:
|
||||
if not isinstance(f, dict) or not f.get("env_var"):
|
||||
errs.append("optional_config[%s]: a field has no env_var" % mg)
|
||||
continue
|
||||
if f["env_var"] not in hu_f:
|
||||
errs.append("optional_config[%s].fields[%s] has no Hungarian twin"
|
||||
% (mg, f["env_var"]))
|
||||
continue
|
||||
for fld in ("label", "help_text"):
|
||||
scalar("optional_config[%s].fields[%s].%s" % (mg, f["env_var"], fld),
|
||||
f.get(fld), "optional_config field %s" % fld)
|
||||
|
||||
ints = ov.get("integrations")
|
||||
if ints is not None:
|
||||
for i in ints if isinstance(ints, list) else []:
|
||||
if not isinstance(i, dict) or not i.get("target"):
|
||||
errs.append("an integrations entry has no target")
|
||||
continue
|
||||
if i["target"] not in hu_targets:
|
||||
errs.append("integrations[%s] has no Hungarian twin" % i["target"])
|
||||
continue
|
||||
for fld in ("label", "description"):
|
||||
scalar("integrations[%s].%s" % (i["target"], fld), i.get(fld), "integration %s" % fld)
|
||||
|
||||
dps = ov.get("data_paths")
|
||||
if dps is not None:
|
||||
for d in dps if isinstance(dps, list) else []:
|
||||
if not isinstance(d, dict) or not d.get("path"):
|
||||
errs.append("a data_paths entry has no path")
|
||||
continue
|
||||
if d["path"] not in hu_paths:
|
||||
errs.append("data_paths[%s] has no Hungarian twin" % d["path"])
|
||||
continue
|
||||
scalar("data_paths[%s].label" % d["path"], d.get("label"), "data_path label")
|
||||
|
||||
icv = ov.get("initial_credentials")
|
||||
if icv is not None:
|
||||
if not isinstance(icv, dict):
|
||||
errs.append("initial_credentials must be a mapping")
|
||||
else:
|
||||
for k in icv:
|
||||
if k != "note":
|
||||
errs.append("unknown key initial_credentials.%s — only the note is copy" % k)
|
||||
scalar("initial_credentials.note", icv.get("note"), "initial_credentials.note")
|
||||
|
||||
return out, errs
|
||||
|
||||
|
||||
# ── Loading ──────────────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def load_yaml(path):
|
||||
import yaml
|
||||
with io.open(path, encoding="utf-8") as fh:
|
||||
return yaml.safe_load(fh)
|
||||
|
||||
|
||||
def app_dirs():
|
||||
return sorted(d for d in os.listdir(TEMPLATES)
|
||||
if os.path.isdir(os.path.join(TEMPLATES, d))
|
||||
and os.path.exists(os.path.join(TEMPLATES, d, ".felhom.yml")))
|
||||
|
||||
|
||||
def read_freeze():
|
||||
if not os.path.exists(FREEZE_PATH):
|
||||
return None
|
||||
with io.open(FREEZE_PATH, encoding="utf-8") as fh:
|
||||
return json.load(fh)
|
||||
|
||||
|
||||
def write_freeze(data):
|
||||
os.makedirs(os.path.dirname(FREEZE_PATH), exist_ok=True)
|
||||
with io.open(FREEZE_PATH, "w", encoding="utf-8") as fh:
|
||||
json.dump(data, fh, ensure_ascii=False, indent=1, sort_keys=True)
|
||||
fh.write("\n")
|
||||
|
||||
|
||||
def capture(reasons=None):
|
||||
apps = {}
|
||||
for app in app_dirs():
|
||||
apps[app] = copy_strings(load_yaml(os.path.join(TEMPLATES, app, ".felhom.yml")))
|
||||
return {
|
||||
"_what": "Every customer-facing Hungarian string in the catalog, frozen. See "
|
||||
"scripts/check-copy-i18n.py. A value here may only change when the Hungarian "
|
||||
"itself is deliberately rewritten — never by a translation.",
|
||||
"_captured_from": "app-catalog-felhom.eu@94bc5febaca2 (2026-09-20), before localisation slice 5",
|
||||
"reasons": reasons or {},
|
||||
"apps": apps,
|
||||
}
|
||||
|
||||
|
||||
# ── The checks ───────────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def check_language(app, path, val, hu_val, display_name, fails):
|
||||
where = "%s / i18n.en.%s" % (app, path)
|
||||
if HU_LETTER.search(val):
|
||||
fails.append("%s: an accented Hungarian letter in the English text: %r" % (where, val))
|
||||
folded = fold(val)
|
||||
for stem in HU_ASCII_STEMS:
|
||||
if stem in folded:
|
||||
fails.append("%s: ASCII-only Hungarian %r in the English text: %r" % (where, stem, val))
|
||||
break
|
||||
if EN_FORBIDDEN.search(val):
|
||||
fails.append("%s: the product does not beg — no \"please\"/\"kindly\": %r" % (where, val))
|
||||
for pat in RETRIEVAL_STEMS_EN:
|
||||
if re.search(pat, val, re.I):
|
||||
hu_promises = any(s in fold(hu_val) for s in [fold(x) for x in RETRIEVAL_STEMS_HU])
|
||||
if not hu_promises:
|
||||
fails.append("%s: an English retrieval promise the Hungarian does not make: %r"
|
||||
% (where, val))
|
||||
break
|
||||
if "Felhom" in hu_val and "Felhom" not in val:
|
||||
fails.append("%s: the product name „Felhom\" was dropped in translation: %r" % (where, val))
|
||||
if display_name and display_name in hu_val and display_name not in val:
|
||||
fails.append("%s: the app name %r was dropped or translated: %r" % (where, display_name, val))
|
||||
# Credentials survive verbatim — the words around them are copy, the value is a login.
|
||||
if path.endswith("default_creds") or path.endswith("initial_credentials.note"):
|
||||
want = {t for t in CRED_TOKEN.findall(hu_val) if len(t) >= 3}
|
||||
want |= {w for w in CRED_WORDS if re.search(r"\b%s\b" % w, hu_val, re.I)}
|
||||
for tok in sorted(want):
|
||||
# WORD BOUNDARIES, and they are load-bearing: the first version searched for the token
|
||||
# as a bare substring, and „admin" -> "administrator" passed it. A login is the whole
|
||||
# word or it is a different login.
|
||||
if not re.search(r"(?<![A-Za-z0-9])%s(?![A-Za-z0-9])" % re.escape(tok), val, re.I):
|
||||
fails.append("%s: the credential token %r is not in the English text: %r"
|
||||
% (where, tok, val))
|
||||
|
||||
|
||||
def main(argv):
|
||||
if "--root" in argv:
|
||||
use_root(argv[argv.index("--root") + 1])
|
||||
# --expect-missing is for `scripts/test_gate_decoys.py` ONLY: a decoy case that adds a correct
|
||||
# English block would otherwise trip the ratchet and look like a conviction. The gate run by
|
||||
# `catalog_gates.py` never passes it, so the ratchet is untouched in every real run.
|
||||
ceiling = EN_MISSING_CEILING
|
||||
if "--expect-missing" in argv:
|
||||
ceiling = int(argv[argv.index("--expect-missing") + 1])
|
||||
|
||||
if "--capture-freeze" in argv:
|
||||
old = read_freeze() or {}
|
||||
write_freeze(capture(old.get("reasons")))
|
||||
print("copy-i18n: freeze written to %s (%d apps)" % (FREEZE_PATH, len(app_dirs())))
|
||||
return 0
|
||||
|
||||
if "--add-app" in argv:
|
||||
i = argv.index("--add-app")
|
||||
name = argv[i + 1]
|
||||
if "--reason" not in argv:
|
||||
print("copy-i18n: --add-app needs --reason \"…\" — a new app's Hungarian has never been "
|
||||
"reviewed, and the reason is the review")
|
||||
return 2
|
||||
reason = argv[argv.index("--reason") + 1]
|
||||
fr = read_freeze() or capture()
|
||||
fr.setdefault("reasons", {})[name] = reason
|
||||
fr["apps"][name] = copy_strings(load_yaml(os.path.join(TEMPLATES, name, ".felhom.yml")))
|
||||
write_freeze(fr)
|
||||
print("copy-i18n: %s admitted to the freeze — %s" % (name, reason))
|
||||
return 0
|
||||
|
||||
# Option VALUES are not app names. --root and --expect-missing each consume the argument
|
||||
# after them, and a scope list that swallowed a path would silently skip every language check.
|
||||
consumed = set()
|
||||
for opt in ("--root", "--expect-missing", "--range"):
|
||||
if opt in argv:
|
||||
consumed.add(argv.index(opt) + 1)
|
||||
scope = [a for i, a in enumerate(argv) if i >= 1 and not a.startswith("-") and i not in consumed]
|
||||
|
||||
# Controls for the ASCII-folded matcher, printed every run (workspace rule).
|
||||
if "jelentkezz be" not in fold("Jelentkezz be: admin"):
|
||||
print("copy-i18n INCONCLUSIVE: positive control failed — folding does not strip accents")
|
||||
return 2
|
||||
if any(s in fold("Encrypted notes and text sharing") for s in HU_ASCII_STEMS):
|
||||
print("copy-i18n INCONCLUSIVE: negative control failed — a clean English sentence convicted")
|
||||
return 2
|
||||
print("copy-i18n: matcher controls OK (positive „Jelentkezz be…\" convicts, "
|
||||
"negative „Encrypted notes…\" does not)")
|
||||
|
||||
freeze = read_freeze()
|
||||
if freeze is None:
|
||||
print("copy-i18n INCONCLUSIVE: no freeze at %s — run --capture-freeze first" % FREEZE_PATH)
|
||||
return 2
|
||||
|
||||
# Drift check against the sibling's shared vocabulary, when the clone is present.
|
||||
sib = os.path.join(os.path.dirname(ROOT), "felhom.eu", "scripts")
|
||||
if os.path.exists(os.path.join(sib, "customer_copy_vocab.py")):
|
||||
sys.path.insert(0, sib)
|
||||
try:
|
||||
import customer_copy_vocab as vocab
|
||||
if list(vocab.RETRIEVAL_STEMS_EN) != RETRIEVAL_STEMS_EN:
|
||||
print("copy-i18n: FAIL — RETRIEVAL_STEMS_EN has drifted from "
|
||||
"felhom.eu/scripts/customer_copy_vocab.py; copy it across")
|
||||
return 1
|
||||
print("copy-i18n: retrieval-promise vocabulary matches the sibling clone")
|
||||
except Exception as exc: # pragma: no cover
|
||||
print("copy-i18n: note — sibling vocabulary unreadable (%s); using the local copy" % exc)
|
||||
|
||||
fails = []
|
||||
apps = app_dirs()
|
||||
en_missing_total = 0
|
||||
per_app = []
|
||||
|
||||
for app in apps:
|
||||
meta = load_yaml(os.path.join(TEMPLATES, app, ".felhom.yml"))
|
||||
hu = copy_strings(meta)
|
||||
|
||||
# CHECK 1 — the Hungarian is frozen. Runs on EVERY app whatever the scope: a scoped push
|
||||
# that quietly edits a neighbour is exactly the thing worth catching.
|
||||
frozen = (freeze.get("apps") or {}).get(app)
|
||||
if frozen is None:
|
||||
fails.append("%s: not in the freeze. A NEW app's Hungarian has never been reviewed — "
|
||||
"run `python3 scripts/check-copy-i18n.py --add-app %s --reason \"…\"` in "
|
||||
"the same commit." % (app, app))
|
||||
else:
|
||||
for path, val in sorted(hu.items()):
|
||||
if path not in frozen:
|
||||
fails.append("%s: NEW Hungarian string %s = %r — the freeze does not know it"
|
||||
% (app, path, val))
|
||||
elif frozen[path] != val:
|
||||
fails.append("%s: Hungarian CHANGED at %s\n frozen: %r\n on disk: %r"
|
||||
% (app, path, frozen[path], val))
|
||||
for path in sorted(frozen):
|
||||
if path not in hu:
|
||||
fails.append("%s: Hungarian string REMOVED at %s (%r)" % (app, path, frozen[path]))
|
||||
|
||||
# CHECKS 2-4 — the English block.
|
||||
i18n = meta.get("i18n") or {}
|
||||
en_paths = set()
|
||||
if i18n:
|
||||
if not isinstance(i18n, dict):
|
||||
fails.append("%s: i18n is not a mapping" % app)
|
||||
else:
|
||||
for lang in i18n:
|
||||
if lang not in SUPPORTED_LANGS:
|
||||
fails.append("%s: i18n.%s — the controller renders %s only"
|
||||
% (app, lang, "/".join(SUPPORTED_LANGS)))
|
||||
for lang in SUPPORTED_LANGS:
|
||||
if lang not in i18n:
|
||||
continue
|
||||
vals, errs = overlay_strings(i18n[lang], meta)
|
||||
for e in errs:
|
||||
fails.append("%s: %s" % (app, e))
|
||||
if scope and app not in scope:
|
||||
continue
|
||||
for path, val in sorted(vals.items()):
|
||||
en_paths.add(path)
|
||||
check_language(app, path, val, hu.get(path, ""),
|
||||
meta.get("display_name") or "", fails)
|
||||
|
||||
missing = [p for p in hu if p not in en_paths]
|
||||
en_missing_total += len(missing)
|
||||
per_app.append((app, len(hu) - len(missing), len(hu)))
|
||||
|
||||
print("copy-i18n: %d apps · %d Hungarian copy strings frozen · %d with English"
|
||||
% (len(apps), sum(t for _a, _d, t in per_app),
|
||||
sum(d for _a, d, _t in per_app)))
|
||||
done = [p for p in per_app if p[1]]
|
||||
if done:
|
||||
print("copy-i18n: translated so far — " +
|
||||
", ".join("%s %d/%d" % (a, d, t) for a, d, t in done))
|
||||
|
||||
# CHECK 5 — the ratchet.
|
||||
if en_missing_total != ceiling:
|
||||
direction = "ABOVE" if en_missing_total > ceiling else "BELOW"
|
||||
fails.append(
|
||||
"English coverage: %d strings have no English, ceiling is %d (%s).\n"
|
||||
" %s" % (en_missing_total, ceiling, direction,
|
||||
"Translate the rest, or lower EN_MISSING_CEILING in this file to %d in the "
|
||||
"SAME commit — the ceiling records what was actually pushed."
|
||||
% en_missing_total if direction == "BELOW" else
|
||||
"A push claimed translations it did not make."))
|
||||
|
||||
if fails:
|
||||
print("\ncopy-i18n: FAIL — %d finding(s)\n" % len(fails))
|
||||
for f in fails:
|
||||
print(" - %s" % f)
|
||||
return 1
|
||||
print("copy-i18n: OK")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -48,6 +48,7 @@ ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
COVERS = {
|
||||
"engine-major": "the major moved in a comment/env var/README/app image, not on an engine's image: line",
|
||||
"catalog-since": "the date bumped in a comment/README while .felhom.yml's field stayed; or only a comment/env moved, no image (R-452)",
|
||||
"copy-i18n": "Hungarian edited in a COMMENT/README/display_name (label, not copy) vs a real frozen string changed; an English block that is not English, is not matched to a Hungarian twin, or rewrites a credential (R-560)",
|
||||
}
|
||||
|
||||
fails = []
|
||||
@@ -70,6 +71,7 @@ def make_clone():
|
||||
|
||||
def edit(clone, relpath, fn):
|
||||
p = os.path.join(clone, relpath)
|
||||
os.makedirs(os.path.dirname(p), exist_ok=True) # a case may ADD a file (a new app directory)
|
||||
text = io.open(p, encoding="utf-8").read() if os.path.exists(p) else ""
|
||||
new = fn(text)
|
||||
if new == text:
|
||||
@@ -114,6 +116,32 @@ def case(name, clone, edits, expect_rc, must_contain=(), gate="check-engine-majo
|
||||
sh(["git", "reset", "-q", "--hard", base], cwd=clone)
|
||||
|
||||
|
||||
def case_copy(name, clone, edits, expect_rc, must_contain=(), extra_args=()):
|
||||
"""The copy-i18n gate reads FILES, not commits, so its cases need neither a commit nor a range —
|
||||
but they DO need --root, or the gate would read the real repo and judge files nobody edited.
|
||||
That is the `constant-for-measurement` decoy shape, and it would make every case below pass."""
|
||||
global ran
|
||||
ran += 1
|
||||
base = sh(["git", "rev-parse", "HEAD"], cwd=clone).stdout.strip()
|
||||
try:
|
||||
for relpath, fn in edits:
|
||||
edit(clone, relpath, fn)
|
||||
r = sh([sys.executable, os.path.join(ROOT, "scripts", "check-copy-i18n.py"),
|
||||
"--root", clone] + list(extra_args), cwd=clone)
|
||||
out = r.stdout + r.stderr
|
||||
if r.returncode == expect_rc and all(m in out for m in must_contain):
|
||||
print(" ok %-52s rc=%d (expected %d)" % (name, r.returncode, expect_rc))
|
||||
else:
|
||||
fails.append("%s: rc=%d expected %d; missing %s\n%s" % (
|
||||
name, r.returncode, expect_rc,
|
||||
[m for m in must_contain if m not in out], out[-900:]))
|
||||
return out
|
||||
finally:
|
||||
sh(["git", "checkout", "-q", "--", "."], cwd=clone)
|
||||
sh(["git", "clean", "-qfd"], cwd=clone)
|
||||
sh(["git", "reset", "-q", "--hard", base], cwd=clone)
|
||||
|
||||
|
||||
def swap_image(service, frm, to):
|
||||
"""Change ONLY the named service's own image: line — the same per-service discipline as the
|
||||
gate, so the case moves the fact and nothing else."""
|
||||
@@ -207,6 +235,125 @@ def main():
|
||||
expect_rc=1, must_contain=("CATALOG-SINCE GATE FAILED",), gate=CS)
|
||||
case("DECOY: only a comment + env line change, images untouched, date untouched", clone,
|
||||
[(KIMAI, comment_and_env)], expect_rc=0, must_contain=("0 image move(s) dated", "catalog-since gate OK"), gate=CS)
|
||||
|
||||
# ── copy-i18n (R-560): Hungarian frozen, English sound ───────────────────────────────
|
||||
PB = "templates/privatebin/.felhom.yml"
|
||||
TOTAL = 1032 # every copy string in the catalog, measured on 94bc5febaca2
|
||||
PB_EN = 14 # what the genuine block below translates
|
||||
|
||||
# A CORRECT English block for privatebin — the genuine article every decoy is a twist on.
|
||||
GENUINE_EN = """
|
||||
i18n:
|
||||
en:
|
||||
description: "Encrypted note and text sharing"
|
||||
app_info:
|
||||
tagline: "Encrypted text sharing - the server never sees the content"
|
||||
use_cases:
|
||||
- 'Share sensitive text safely'
|
||||
- 'End-to-end encryption - the server cannot read the content'
|
||||
- 'Choose how long it lasts (5 minutes to a year, or never)'
|
||||
- 'Delete after reading, automatically'
|
||||
- 'Password protection for extra safety'
|
||||
first_steps:
|
||||
- 'Open paste.DOMAIN in your browser'
|
||||
- 'Type your text and select Send'
|
||||
- 'Share the link you get - the encryption key is inside the URL'
|
||||
deploy_fields:
|
||||
- env_var: DOMAIN
|
||||
label: "Domain"
|
||||
description: "The server domain name"
|
||||
- env_var: SUBDOMAIN
|
||||
label: "Subdomain"
|
||||
description: "The address this app answers on"
|
||||
"""
|
||||
|
||||
def add_en(block=GENUINE_EN):
|
||||
return lambda t: t.rstrip("\n") + "\n" + block
|
||||
|
||||
def en_with(old_, new_):
|
||||
return add_en(GENUINE_EN.replace(old_, new_))
|
||||
|
||||
# THE FACTS — each must be refused.
|
||||
case_copy("FACT: a Hungarian byte changed in a frozen string", clone,
|
||||
[(PB, lambda t: t.replace("Titkosított jegyzet és szöveg megosztás",
|
||||
"Titkosított jegyzet- és szövegmegosztás"))],
|
||||
expect_rc=1, must_contain=("Hungarian CHANGED", "privatebin", "description"))
|
||||
case_copy("FACT: a Hungarian first_step removed", clone,
|
||||
[(PB, lambda t: t.replace(" - 'Oszd meg a generált linket - a titkosítási kulcs az URL-ben van'\n", ""))],
|
||||
expect_rc=1, must_contain=("REMOVED", "first_steps"))
|
||||
case_copy("FACT: a NEW app is not in the freeze", clone,
|
||||
[("templates/decoyapp/.felhom.yml",
|
||||
lambda t: 'display_name: "Decoy"\ndescription: "Uj alkalmazas"\nslug: decoyapp\n')],
|
||||
expect_rc=1, must_contain=("not in the freeze", "--add-app"))
|
||||
case_copy("FACT: an unknown key inside the English block", clone,
|
||||
[(PB, en_with(' description: "Encrypted note and text sharing"',
|
||||
' description: "Encrypted note and text sharing"\n docs_url: "https://example.invalid"'))],
|
||||
expect_rc=1, must_contain=("unknown key",), extra_args=("--expect-missing", str(TOTAL - PB_EN)))
|
||||
case_copy("FACT: an English deploy field with no Hungarian twin", clone,
|
||||
[(PB, en_with(" - env_var: DOMAIN", " - env_var: NOSUCHFIELD"))],
|
||||
expect_rc=1, must_contain=("no Hungarian twin", "NOSUCHFIELD"))
|
||||
case_copy("FACT: an accented Hungarian letter left in the English", clone,
|
||||
[(PB, en_with("Share sensitive text safely", "Érzékeny text sharing"))],
|
||||
expect_rc=1, must_contain=("accented Hungarian letter",),
|
||||
extra_args=("--expect-missing", str(TOTAL - PB_EN)))
|
||||
case_copy("FACT: ASCII-only Hungarian left in the English (no accent to find)", clone,
|
||||
[(PB, en_with(' description: "The address this app answers on"',
|
||||
' description: "Aldomain for the app"'))],
|
||||
expect_rc=1, must_contain=("ASCII-only Hungarian", "aldomain"),
|
||||
extra_args=("--expect-missing", str(TOTAL - PB_EN)))
|
||||
case_copy("FACT: the product begs (\"please\")", clone,
|
||||
[(PB, en_with("Type your text and select Send", "Please type your text and select Send"))],
|
||||
expect_rc=1, must_contain=("does not beg",),
|
||||
extra_args=("--expect-missing", str(TOTAL - PB_EN)))
|
||||
case_copy("FACT: an English retrieval promise the Hungarian never made", clone,
|
||||
[(PB, en_with("Password protection for extra safety",
|
||||
"Deleted notes can still be restored later"))],
|
||||
expect_rc=1, must_contain=("retrieval promise",),
|
||||
extra_args=("--expect-missing", str(TOTAL - PB_EN)))
|
||||
# A credential is a LOGIN, not prose: gokapi's default_creds carries admin / adminadmin.
|
||||
GK = "templates/gokapi/.felhom.yml"
|
||||
case_copy("FACT: a credential token rewritten in translation", clone,
|
||||
[(GK, lambda t: t.rstrip("\n") + """
|
||||
i18n:
|
||||
en:
|
||||
app_info:
|
||||
default_creds: "Sign in: administrator / hunter2"
|
||||
""")],
|
||||
expect_rc=1, must_contain=("credential token",),
|
||||
extra_args=("--expect-missing", str(TOTAL - 1)))
|
||||
case_copy("FACT: an i18n block for a language the controller does not render", clone,
|
||||
[(PB, lambda t: t.rstrip("\n") + "\ni18n:\n de:\n description: \"Verschluesselte Notizen\"\n")],
|
||||
expect_rc=1, must_contain=("renders en only",))
|
||||
case_copy("FACT: an English list with a different number of steps", clone,
|
||||
[(PB, en_with(" - 'Share the link you get - the encryption key is inside the URL'\n", ""))],
|
||||
expect_rc=1, must_contain=("a list is replaced",),
|
||||
extra_args=("--expect-missing", str(TOTAL - PB_EN + 1)))
|
||||
|
||||
# THE GENUINE ARTICLE — must pass.
|
||||
case_copy("GENUINE: a correct English block on privatebin", clone,
|
||||
[(PB, add_en())], expect_rc=0,
|
||||
must_contain=("copy-i18n: OK", "privatebin 14/14"),
|
||||
extra_args=("--expect-missing", str(TOTAL - PB_EN)))
|
||||
|
||||
# THE DECOYS — the LABEL moves, the FACT does not. Each must pass.
|
||||
case_copy("DECOY: Hungarian rewritten inside a YAML COMMENT", clone,
|
||||
[(PB, lambda t: t.replace("# --- App info (info page content) ---",
|
||||
"# --- Alkalmazas informacio: Titkosított jegyzet MEGVALTOZOTT ---"))],
|
||||
expect_rc=0, must_contain=("copy-i18n: OK",))
|
||||
case_copy("DECOY: a frozen Hungarian sentence pasted into README.md", clone,
|
||||
[("README.md", lambda t: t + "\nTitkositott jegyzet es szoveg megosztas (decoy)\n")],
|
||||
expect_rc=0, must_contain=("copy-i18n: OK",))
|
||||
case_copy("DECOY: display_name changed - a NAME, never copy", clone,
|
||||
[(PB, lambda t: t.replace('display_name: "PrivateBin"', 'display_name: "PrivateBin 2"'))],
|
||||
expect_rc=0, must_contain=("copy-i18n: OK",))
|
||||
case_copy("DECOY: docs_url changed - configuration, never copy", clone,
|
||||
[(PB, lambda t: t.replace("https://github.com/PrivateBin/PrivateBin/wiki",
|
||||
"https://example.invalid/wiki"))],
|
||||
expect_rc=0, must_contain=("copy-i18n: OK",))
|
||||
case_copy("DECOY: Hungarian text added to a docker-compose.yml", clone,
|
||||
[("templates/privatebin/docker-compose.yml",
|
||||
lambda t: t.replace("services:", "# Titkosított jegyzet és szöveg megosztás\nservices:", 1))],
|
||||
expect_rc=0, must_contain=("copy-i18n: OK",))
|
||||
finally:
|
||||
shutil.rmtree(clone, ignore_errors=True)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user