R-127 (a): four data-encrypting keys flagged data_key (n8n, calcom, wanderer, bookstack APP_KEY) + gate data-key with decoys

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
2026-10-05 21:38:29 +02:00
parent 84ced4478b
commit 66d1abb101
9 changed files with 214 additions and 2 deletions
+1 -1
View File
@@ -20,7 +20,7 @@ Templates are config; the few script helpers other scripts must REUSE, never re-
|---|---|---|
| **The one canonical example app** | `templates/paperless-ngx/` (both files) | Multi-container (app + postgres + redis), HDD + userdata mounts, full deploy_fields spectrum (domain/subdomain/secret/password/text/path/select). Copy this structure for any new app. |
| `.felhom.yml` required fields | `templates/paperless-ngx/.felhom.yml` | All 53 apps: `display_name`, `description` (Hungarian), `category`, `subdomain`, `slug`, `resources{mem_request, mem_limit, pi_compatible, needs_hdd}`, `deploy_fields`, `app_info{tagline, use_cases, first_steps, ...}`, `healthcheck`. Optional: `smtp_mapping` (email-capable apps), `open_path` (non-root landing page, e.g. ghost). |
| deploy_fields conventions | `templates/paperless-ngx/.felhom.yml` (`deploy_fields:` block) | Every app starts with `DOMAIN` (type `domain`) + `SUBDOMAIN` (type `subdomain`, `locked_after_deploy: true`). Secrets: `type: secret` + `generate:` — dominant generators `password:24` (DB passwords) and `hex:32` (app secret keys); `password:16` for shown admin passwords (`type: password`). HDD apps add `HDD_PATH` (`type: path`, placeholder `/mnt/felhom-drives/hdd_1`, locked). Labels/descriptions in Hungarian. |
| deploy_fields conventions | `templates/paperless-ngx/.felhom.yml` (`deploy_fields:` block) | Every app starts with `DOMAIN` (type `domain`) + `SUBDOMAIN` (type `subdomain`, `locked_after_deploy: true`). Secrets: `type: secret` + `generate:` — dominant generators `password:24` (DB passwords) and `hex:32` (app secret keys); `password:16` for shown admin passwords (`type: password`). HDD apps add `HDD_PATH` (`type: path`, placeholder `/mnt/felhom-drives/hdd_1`, locked). Labels/descriptions in Hungarian. **A key that ENCRYPTS STORED DATA gets `data_key: true`** + a comment saying why (the restore recovers it and refuses without it; the box never regenerates it) — gated by `scripts/check-data-key.py` (R-127): an env var naming `ENCRYPTION_KEY`/`PEPPER` must be flagged, any other data key goes in its `REGISTRY` with the reason, and an unexplained flag is refused. A key that only SIGNS sessions is not one, whatever its label says. |
| **Known default login → `after_install:`** (decision 45, controller ≥ 0.279.0) | `templates/bookstack/.felhom.yml` (`ADMIN_PASSWORD` field + `after_install:` block); `FIRST-ADMIN.md` for every app | An app that starts with a known admin login gets a generated `type: password` field (`generate: "password:24"`, `locked_after_deploy: true`) and ONE `after_install: {service, env: [ADMIN_PASSWORD], command: [...], success: "<marker the output must carry>"}` through the app's OWN CLI, run once after a FRESH install. Keep `app_info.default_creds` — the page hides it once the command succeeded and warns while it is in effect. **Prove on 9202 (drill catalog) before live: the default fails, the generated password works, a wrong one fails.** TRAPS: `success:` is required because a CLI can exit 0 on an error (claper's `rpc`); **pass the password as its own argument, never inside program code** (`sys.argv[1]` — mealie, wger; security review 2026-09-29); a special-character policy uses `generate: "password:24:special"` (controller ≥ 0.280.0, calibre-web); a Hungarian first-steps change needs `check-copy-i18n.py --capture-freeze`. |
| **Open first-run screen → `setup_gate:`** (decision 46, controller ≥ 0.280.0) | `templates/n8n/.felhom.yml` (probe), `templates/uptime-kuma/.felhom.yml` (no probe → the household's button); `FIRST-ADMIN.md` | `setup_gate: true` + optional `setup_done_probe: {url: http://<container_name>:<port>/<path>, field: <dotted.json.path>, done: "<text>"}` | TRAPS: the probe must FLIP on the setup — measure it before and after on 9202; an app with open sign-up after setup (R-711) is not closed by the gate; `url` is read on the docker network, so it names the container, not the subdomain. |
| **Open sign-up after the setup → `signup_block:`** (decision 47, controller ≥ 0.281.0) | `templates/opengist/.felhom.yml`, `templates/calcom/.felhom.yml` | `signup_block: "<traefik matcher>"` + `app_info.add_people` (hu) / `i18n.en.app_info.add_people` | TRAPS: block the app's API sign-up call, not only the page; an app's own invite link often uses the same address (the household's 15-minute window covers it); `add_people` is copy — `--capture-freeze`. |
+5
View File
@@ -27,6 +27,8 @@ Gates, in order (all must pass; **non-zero exit on any failure**):
record (NEW-APP-CHECKLIST.md; the 53 apps published before 2026-10-01 are exempt by name)
11. mem-limit-sum static, instant, whole repo — `.felhom.yml` resources.mem_limit equals the sum of every
service's deploy.resources.limits.memory, and every service has one (R-758)
12. data-key static, instant, whole repo — every data-encrypting key carries data_key: true (name rule
or REGISTRY), and no flag is unexplained (R-127)
4. engine-major static, needs GIT HISTORY — no database engine pin crosses a MAJOR version
(operator ruling 2026-09-13; expires when Slice 4 / R-448 ships). Runs in the
pre-push hook, which has the full clone; on a SHALLOW clone (CI fetches at
@@ -123,6 +125,9 @@ GATES = [
# templates were under it — the deploy screen and the box's overcommit warning read less than the app may take.
# Static, stdlib only (no PyYAML on CI), so --fast: the hook and CI.
("mem-limit-sum", "check-mem-limit-sum.py", True, True, False),
# R-127 leg (a) (2026-10-05): every data-encrypting key carries `data_key: true` (by its NAME, or by a registry
# entry with its reason) and every flag is accounted for — the restore's fail-closed gate reads only that flag.
("data-key", "check-data-key.py", True, True, False),
]
# 3 (R-605): the gate's HARNESS refused to run — its canary failed or it had nothing to judge — so NO app was
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""check-data-key.py — every data-encrypting key carries `data_key: true`, and every flag is accounted for (R-127 a).
WHAT WENT WRONG. `data_key: true` on a deploy field tells the controller the app ENCRYPTS STORED DATA with it: the
restore RECOVERS the value and refuses (fail-closed) when it cannot, and the box never generates a new one
(felhom-controller internal/backup/restore_unit.go missingDataKeys; internal/stacks/deploy.go GenerateSecretForField).
In 2026-08 only five fields carried it, while n8n's N8N_ENCRYPTION_KEY, calcom's CALENDSO_ENCRYPTION_KEY, wanderer's
POCKETBASE_ENCRYPTION_KEY and bookstack's APP_KEY (two-factor secrets) did not — so a restore missing one of them
would have proceeded onto data it cannot decrypt instead of refusing.
WHY NOT THE LABEL. The Hungarian label „Titkosítási kulcs" ("Encryption key") sits on 24 secrets, most of which only
SIGN sessions (Django SECRET_KEY, Phoenix SECRET_KEY_BASE, JWT secrets): regenerating those signs everyone out, it
loses no data. A label is copy, frozen byte for byte, and it is not the fact. The facts this gate reads:
1. NAME RULE — a field whose env var names an encryption key or a pepper (`ENCRYPTION_KEY`, `PEPPER`) is a data key
by what the app calls it. It must carry `data_key: true`.
2. REGISTRY — data keys whose name does not say so (bookstack APP_KEY encrypts two-factor secrets; …) are listed
below BY APP AND FIELD with the reason. Each must carry the flag (unflagging one is a regression), and an entry
whose field no longer exists is STALE (refused — a registry nobody prunes stops meaning anything).
3. AGREEMENT — a `data_key: true` that neither rule names is refused: add it to REGISTRY with its reason. Over-
flagging is not harmless either: the restore then REFUSES for a key that could have been regenerated.
stdlib only (the CI runner has no PyYAML). The flag is read only from a field's own block inside the top-level
`deploy_fields:` — never from a comment, never from the `i18n:` block, never from `steps/` files.
USAGE
python3 scripts/check-data-key.py [app …] [--root=<dir>] (`--all` accepted, a no-op: every directory is judged)
Exit: 0 agree · 1 convicted · 2 inconclusive.
Decoys: scripts/test_gate_decoys.py `data_key_cases` (COVERS "data-key").
"""
import io
import os
import re
import sys
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
NAME_RULE = re.compile(r"ENCRYPTION_KEY|PEPPER")
# (app, env_var) -> why it is a data key although its name does not say so.
REGISTRY = {
("adventurelog", "SECRET_KEY"): "the template's own comment: encrypts stored data; restore must recover it",
("bookstack", "APP_KEY"): "Laravel encrypt() on every member's two-factor secret (app/Access/Mfa/MfaValue.php)",
("dawarich", "SECRET_KEY_BASE"): "the template's own comment: stored data unreadable if it changes",
("papra", "AUTH_SECRET"): "the template's own comment: stored tokens invalid if it changes; keep the old key",
("sparkyfitness", "BETTER_AUTH_SECRET"): "signs sessions AND encrypts 2FA/TOTP secrets (template comment)",
}
FIELD_RE = re.compile(r"^ - env_var:\s*['\"]?([A-Za-z0-9_]+)['\"]?\s*(?:#.*)?$")
DATA_KEY_RE = re.compile(r"^ data_key:\s*(\S+?)\s*(?:#.*)?$")
TOP_KEY_RE = re.compile(r"^[A-Za-z0-9_]+:")
def fields(meta_text):
"""[(env_var, data_key_raw_or_None)] from the top-level deploy_fields: block only."""
out, cur, inside = [], None, False
for line in meta_text.split("\n"):
if TOP_KEY_RE.match(line):
inside = line.startswith("deploy_fields:")
cur = None
continue
if not inside:
continue
m = FIELD_RE.match(line)
if m:
cur = [m.group(1), None]
out.append(cur)
continue
m = DATA_KEY_RE.match(line)
if m and cur is not None:
cur[1] = m.group(1).strip("'\"")
return [(a, b) for a, b in out]
def main(argv):
root, apps = ROOT, []
for a in argv:
if a.startswith("--root="):
root = a.split("=", 1)[1]
elif a == "--all":
continue
elif a.startswith("-"):
print("unknown option: %s" % a)
return 2
else:
apps.append(a)
tdir = os.path.join(root, "templates")
if not os.path.isdir(tdir):
print("data-key: no templates/ under %s" % root)
return 2
every = sorted(n for n in os.listdir(tdir) if os.path.isfile(os.path.join(tdir, n, ".felhom.yml")))
judged = apps or every
unknown = [a for a in judged if a not in every]
if unknown:
print("data-key: no such template: %s" % ", ".join(unknown))
return 2
bad, undecided, flagged = [], [], 0
seen = set()
for app in judged:
text = io.open(os.path.join(tdir, app, ".felhom.yml"), encoding="utf-8").read()
for env, raw in fields(text):
seen.add((app, env))
if raw is not None and raw.lower() not in ("true", "false"):
undecided.append("%s/%s: data_key %r is not true/false" % (app, env, raw))
continue
on = raw is not None and raw.lower() == "true"
flagged += on
named = bool(NAME_RULE.search(env))
reg = (app, env) in REGISTRY
if (named or reg) and not on:
why = "its name says it encrypts" if named else "registered: " + REGISTRY[(app, env)]
bad.append("%s/%s is a data key (%s) but carries no `data_key: true` — a restore missing it would "
"proceed onto data it cannot decrypt" % (app, env, why))
elif on and not (named or reg):
bad.append("%s/%s carries `data_key: true` but neither its name nor REGISTRY says why — add it to "
"REGISTRY in scripts/check-data-key.py with the reason (or drop the flag)" % (app, env))
for (app, env), why in sorted(REGISTRY.items()):
if app in judged and (app, env) not in seen:
bad.append("REGISTRY entry %s/%s is STALE — no such deploy field (%s)" % (app, env, why))
for b in bad:
print("REFUSED: " + b)
for u in undecided:
print("INCONCLUSIVE: " + u)
if bad:
print("data-key: %d problem(s) in %d template(s)" % (len(bad), len(judged)))
return 1
if undecided:
return 2
print("data-key gate OK: %d template(s), %d data key(s), every flag agrees with the name rule and REGISTRY"
% (len(judged), flagged))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+3 -1
View File
@@ -65,7 +65,9 @@ class CatalogGatesFastTest(unittest.TestCase):
# 11 since 2026-10-01: onboarding (NEW-APP-CHECKLIST.md), fast and history-free, so it runs in CI too.
# 12 since family-gate joined — STALE AGAIN until 2026-10-05 (found by the burn-down, round 2).
# 13 since 2026-10-05: mem-limit-sum (R-758), fast and history-free, so it runs in CI too.
self.assertEqual(len(mod.GATES), 13)
# 14 since the same night: data-key (R-127 leg a).
self.assertEqual(len(mod.GATES), 14)
self.assertIn("data-key", [g[0] for g in mod.GATES if g[3] and not g[4]])
self.assertIn("mem-limit-sum", [g[0] for g in mod.GATES if g[3] and not g[4]])
self.assertIn("onboarding", [g[0] for g in mod.GATES if g[3] and not g[4]])
self.assertEqual([g[0] for g in mod.GATES if not g[3]], ["image-resolvable", "volume-persistence"])
+56
View File
@@ -58,6 +58,7 @@ COVERS = {
"family-gate": "family_gate written only in a COMMENT (not gated, no min_controller owed); min_controller only in a comment; a golden DIRECTORY named 0.287.0 with no bake log (the mkdir shape, R-410); a sibling with no golden (stated NOT CHECKED, never a pass of rule 3) - vs the facts: an unanchorable exception (regex, '/', '..'), an exception list with no gate, min_controller below 0.287.0, the newest baked golden below 0.287.0; and a genuine family app passes (decisions 63/64, finding F1)",
"onboarding": "a NEW template with no record; a record missing an id, or carrying it only inside an HTML comment; a `done` whose path does not exist, is an EMPTY directory (the mkdir shape, R-410) or names an absent sibling-repo file; an `n/a` with an empty or two-word reason; an `open` row; `opened:` backdated before the checklist; the template a new app copies lacking a new id - vs a complete record, an id added after `opened:`, and an exempt app's record with open rows (NEW-APP-CHECKLIST.md)",
"mem-limit-sum": "the right figure only in a COMMENT (the compose header's 'mem_limit: 640M', the .felhom.yml arithmetic) while the field is wrong; a `memory:` under reservations: (not a limit) making the sum come out right; a `mem_limit:` outside resources:; a steps/ file with the old figure (not judged) - vs the facts: a field under the sum, a service with no limit, an unreadable size (R-758)",
"data-key": "a data_key: true only in a COMMENT or inside the i18n: block (not the field); an 'Encryption key' LABEL on a signing secret (copy, not the fact - must pass unflagged) - vs the facts: an *_ENCRYPTION_KEY field unflagged, a REGISTERED field unflagged, an unexplained flag, a stale registry entry, a non-boolean flag (R-127)",
"copy-i18n": "Hungarian edited in a COMMENT/README/display_name (label, not copy) vs a real frozen string changed; an English block that is not English, is not matched to a Hungarian twin, or rewrites a credential (R-560). A retrieval promise REGISTERED in ALLOWLIST_EN passes only for its own app+path+sentence with a real reason; an entry for another app, a rewritten sentence, a stale entry or a two-word reason convicts (R-594). Also the DEGRADED mode CI actually runs — PyYAML shadowed out, freeze only (R-595)",
}
@@ -747,6 +748,60 @@ def mem_sum_cases():
shutil.rmtree(ws, ignore_errors=True)
def data_key_cases():
"""check-data-key.py reads templates/*/.felhom.yml via --root (R-127 leg a)."""
global ran
import tempfile
ws = tempfile.mkdtemp(prefix="catalog-datakey-")
try:
GOOD = ('display_name: "X"\n'
'deploy_fields:\n'
' - env_var: DOMAIN\n type: domain\n'
' - env_var: SECRET_KEY\n label: "Titkositasi kulcs"\n type: secret\n generate: "hex:32"\n'
' - env_var: APP_ENCRYPTION_KEY\n type: secret\n generate: "hex:16"\n'
' # why: encrypts stored data\n data_key: true\n'
'\ni18n:\n en:\n deploy_fields:\n - env_var: SECRET_KEY\n label: "Encryption key"\n')
BOOK = ('deploy_fields:\n - env_var: APP_KEY\n type: secret\n generate: "base64key:32"\n data_key: true\n')
def run(name, files, expect_rc, must=()):
global ran
cat = os.path.join(ws, "cat")
shutil.rmtree(cat, ignore_errors=True)
for app, text in files.items():
d = os.path.join(cat, "templates", app)
os.makedirs(d)
io.open(os.path.join(d, ".felhom.yml"), "w", encoding="utf-8").write(text)
r = sh([sys.executable, os.path.join(ROOT, "scripts", "check-data-key.py"), "--root=" + cat], ROOT)
out = r.stdout + r.stderr
ran += 1
ok = r.returncode == expect_rc and all(m in out for m in must)
print(" %s %-70s rc=%d (expected %d)" % ("ok" if ok else "XX", name, r.returncode, expect_rc))
if not ok:
fails.append("%s: rc=%d expected %d; missing %s\n%s" % (
name, r.returncode, expect_rc, [m for m in must if m not in out], out[-400:]))
print("\n-- data-key: genuine and decoys")
run("GENUINE: a flagged *_ENCRYPTION_KEY; a signing key labelled 'Encryption key' unflagged; bookstack registered",
{"demo": GOOD, "bookstack": BOOK}, 0, ("data-key gate OK", "2 data key(s)"))
print("-- data-key: the facts (each MUST be refused)")
run("FACT: the flag only in a COMMENT", {"demo": GOOD.replace(" data_key: true\n", " # data_key: true\n"),
"bookstack": BOOK}, 1, ("demo/APP_ENCRYPTION_KEY is a data key",))
run("FACT: the flag only inside the i18n: block", {"demo": GOOD.replace(" data_key: true\n", "") +
" data_key: true\n", "bookstack": BOOK}, 1,
("demo/APP_ENCRYPTION_KEY is a data key",))
run("FACT: a REGISTERED data key unflagged (bookstack APP_KEY)",
{"demo": GOOD, "bookstack": BOOK.replace(" data_key: true\n", "")}, 1, ("bookstack/APP_KEY is a data key", "registered"))
run("FACT: an unexplained flag (a signing key flagged)", {"demo": GOOD.replace(
' generate: "hex:32"\n', ' generate: "hex:32"\n data_key: true\n', 1), "bookstack": BOOK}, 1,
("demo/SECRET_KEY carries `data_key: true` but neither",))
run("FACT: a stale REGISTRY entry (bookstack has no APP_KEY any more)",
{"demo": GOOD, "bookstack": BOOK.replace("APP_KEY", "OTHER_KEY").replace(" data_key: true\n", "")}, 1, ("STALE",))
run("FACT: a non-boolean flag is INCONCLUSIVE", {"demo": GOOD.replace("data_key: true", "data_key: yes"),
"bookstack": BOOK}, 2, ("INCONCLUSIVE",))
finally:
shutil.rmtree(ws, ignore_errors=True)
def main():
gate = os.path.join(ROOT, "scripts", "check-engine-major.py")
if not os.path.isfile(gate):
@@ -1259,6 +1314,7 @@ i18n:
onboarding_cases()
family_gate_cases()
mem_sum_cases()
data_key_cases()
if fails:
print()
+4
View File
@@ -40,6 +40,10 @@ deploy_fields:
type: secret
generate: "base64key:32"
locked_after_deploy: true
# R-127: Laravel's APP_KEY encrypts each user's two-factor secret (app/Access/Mfa/MfaValue.php encrypt()).
# Regenerating it locks out every member who turned two-factor on, so a restore RECOVERS it and refuses rather
# than make a new one.
data_key: true
- env_var: DB_PASSWORD
label: "Adatbázis jelszó"
+3
View File
@@ -46,6 +46,9 @@ deploy_fields:
type: secret
generate: "hex:16"
locked_after_deploy: true
# R-127: encrypts the connected-app credentials (calendar and video integrations) stored in the database.
# Regenerating it breaks every connected calendar, so a restore RECOVERS it and refuses rather than make a new one.
data_key: true
- env_var: DB_PASSWORD
label: "Adatbázis jelszó"
+3
View File
@@ -40,6 +40,9 @@ deploy_fields:
type: secret
generate: "hex:16"
locked_after_deploy: true
# R-127: encrypts every saved credential in n8n's database (packages/core cipher). Regenerating it makes every
# stored credential unreadable, so a restore RECOVERS it and refuses rather than make a new one.
data_key: true
# --- The setup gate (controller >= 0.280.0, `09` §3 decision 46) ---
+3
View File
@@ -54,6 +54,9 @@ deploy_fields:
type: secret
generate: "hex:16"
locked_after_deploy: true
# R-127: PocketBase encrypts its stored settings (mail and sign-in provider secrets) with it. Regenerating it
# leaves them unreadable, so a restore RECOVERS it and refuses rather than make a new one.
data_key: true
# --- Sign-up (controller >= 0.282.0, `09` §3 decisions 47-48) ---
# NOT gated: its web server calls its own database through the public name, which a gate would refuse (measured on