diff --git a/README.md b/README.md index 47fb741..bff4794 100644 --- a/README.md +++ b/README.md @@ -308,13 +308,13 @@ block + the matching compose `${VAR}` lines. | App | DB Type | RAM (request / limit) | Pi | HDD Data | Subdomain | |-----|---------|----------------------|-----|----------|-----------| | ActualBudget | None (file) | 50M / 256M | yes | -- | budget.* | -| AdventureLog | PostgreSQL | 100M / 384M | yes | -- | travel.* | +| AdventureLog | PostgreSQL | 100M / 896M | yes | -- | travel.* | | Audiobookshelf | None (file) | 100M / 512M | yes | `${HDD_PATH}/media/audiobooks/` | audiobooks.* | | BentoPDF | None (file) | 100M / 384M | yes | -- | pdf.* | -| BookStack | MariaDB | 150M / 512M | yes | -- | wiki.* | -| Cal.com | PostgreSQL | 200M / 768M | no | -- | cal.* | +| BookStack | MariaDB | 150M / 768M | yes | -- | wiki.* | +| Cal.com | PostgreSQL | 200M / 1792M | no | -- | cal.* | | Calibre-Web Automated | None (file) | 200M / 768M | no | `${HDD_PATH}/media/books/` | books.* | -| Claper | PostgreSQL | 100M / 384M | yes | -- | present.* | +| Claper | PostgreSQL | 100M / 640M | yes | -- | present.* | | Code-Server | None (file) | 200M / 1024M | no | -- | code.* | | Crafty Controller | None (file) | 256M / 2048M | no | -- | minecraft.* | | Dawarich | PostgreSQL 17 (PostGIS) + Redis | 500M / 2688M | no | -- | timeline.* | @@ -334,16 +334,16 @@ block + the matching compose `${VAR}` lines. | Immich | PostgreSQL + Redis | 2048M / 4096M | no | `${HDD_PATH}/storage/immich/` | photos.* | | Jellyfin | None (file) | 512M / 2048M | no | `${HDD_PATH}/media/` | media.* | | Karakeep | SQLite + Meilisearch + Chrome | 700M / 2816M | no | -- | bookmarks.* | -| Kimai | MariaDB | 100M / 384M | yes | -- | time.* | +| Kimai | MariaDB | 100M / 640M | yes | -- | time.* | | Komga | None (file) | 200M / 512M | yes | `${HDD_PATH}/media/comics/` | comics.* | | Mealie | None (SQLite) | 200M / 1000M | yes | -- | recipes.* | | MeTube | None (file) | 128M / 768M | yes | `${USERDATA_PATH}/media/metube/` | video.* | | n8n | None (file) | 150M / 512M | no | -- | auto.* | | Navidrome | None (file) | 50M / 256M | yes | `${HDD_PATH}/media/music/` | music.* | -| Nextcloud | MariaDB + Redis | 256M / 1024M | no | `${HDD_PATH}/storage/nextcloud/` | cloud.* | +| Nextcloud | MariaDB + Redis | 256M / 1664M | no | `${HDD_PATH}/storage/nextcloud/` | cloud.* | | OnlyOffice | None (file) | 512M / 2048M | no | -- | office.* | | OpenGist | None (file) | 30M / 128M | yes | -- | gist.* | -| Outline | PostgreSQL + Redis | 200M / 768M | no | -- | kb.* | +| Outline | PostgreSQL + Redis | 200M / 1152M | no | -- | kb.* | | Paperless-ngx | PostgreSQL + Redis | 500M / 1152M | yes | `${HDD_PATH}/storage/paperless/` | paperless.* | | Papra | None (file) | 256M / 768M | yes | -- | papra.* | | Plant-it | None (file) | 50M / 256M | yes | -- | plants.* | @@ -364,7 +364,7 @@ block + the matching compose `${VAR}` lines. | Wanderer | None (file) | 100M / 384M | yes | -- | hike.* | | wger | SQLite | 100M / 384M | yes | -- | fitness.* | | Wishlist | None (file) | 30M / 128M | yes | -- | wishes.* | -| Zipline | PostgreSQL | 100M / 512M | no | -- | img.* | +| Zipline | PostgreSQL | 100M / 768M | no | -- | img.* | ### Variable types per app diff --git a/REUSE.md b/REUSE.md index 8ebec69..c8dcdc2 100644 --- a/REUSE.md +++ b/REUSE.md @@ -42,7 +42,7 @@ Templates are config; the few script helpers other scripts must REUSE, never re- | Docker healthcheck — Python images | `templates/mealie/docker-compose.yml` (~L47) | `test: ["CMD-SHELL", "python3 -c \"import socket; s=socket.create_connection(('localhost',),2); s.close()\""]` (mealie, crafty-controller). tandoor/wger use `urllib.request` variants for real HTTP checks. | | Docker healthcheck — DB/Redis sidecars | `templates/paperless-ngx/docker-compose.yml` (~L107, L129) | postgres: `pg_isready -U -d `; mariadb: `healthcheck.sh --connect --innodb_initialized`; redis: `redis-cli ping`. App container gets `depends_on: : condition: service_healthy`. | | MariaDB sidecar — `MARIADB_AUTO_UPGRADE=1` | `templates/bookstack/docker-compose.yml` (`bookstack-db` `environment:`), also kimai/nextcloud/romm | **Every `mariadb:` sidecar carries `MARIADB_AUTO_UPGRADE=1`** (operator ruling 2026-09-13, `felhom.eu/documentation/audits/SPIKE-r459-mariadb-upgrade-2026-09-06.md`). Without it a major engine move starts on the old datadir, logs that the conversion was **skipped**, and says `Check required!` on every start forever (R-459); with it the engine converts in ~7 s and backs its system tables up first (`system_mysql_backup_.sql.zst` left in the datadir). `MARIADB_DISABLE_UPGRADE_BACKUP` stays UNSET — that backup is the precaution. **Not an image change, so `catalog_since` does not move.** TRAP (R-464): the entrypoint prints `MariaDB upgrade not required` on an UNSUPPORTED downgrade too — ask `mariadb-upgrade --check-if-upgrade-is-needed` (exit 0 = needed, 1 = not), never the log line. PostgreSQL sidecars have NO equivalent (the image runs no `pg_upgrade`, R-463). **Until Slice 4 (R-448) ships, no template may move a `mariadb:`/`postgres:` pin across a MAJOR** — `scripts/check-engine-major.py` refuses it in the pre-push hook. | -| Memory convention | `templates/paperless-ngx/docker-compose.yml` (~L71) + `.felhom.yml resources:` | EVERY service has `deploy.resources.limits.memory` (compose is the enforcement). NO `reservations` anywhere. `.felhom.yml mem_limit` = SUM of all containers' limits (see paperless header comment: 768+256+128=1152M); `mem_request` = expected steady-state usage, display-only. | +| Memory convention | `templates/paperless-ngx/docker-compose.yml` (~L71) + `.felhom.yml resources:` | EVERY service has `deploy.resources.limits.memory` (compose is the enforcement). NO `reservations` anywhere. `.felhom.yml mem_limit` = SUM of all containers' limits (see paperless header comment: 768+256+128=1152M); `mem_request` = expected steady-state usage, display-only. **Gated since 2026-10-05 (R-758):** `scripts/check-mem-limit-sum.py` (`--fast`, stdlib only) refuses a `mem_limit` that is not the sum and a service with no limit; `steps/` files are not judged. | | Compose file skeleton | `templates/paperless-ngx/docker-compose.yml` (header) | Header comment (app, domain, DB type, RAM math, Pi), `restart: unless-stopped`, `TZ=Europe/Budapest`, explicit `container_name`, `traefik-public` external network + `-internal` for DBs, Traefik labels with ``Host(`${SUBDOMAIN}.${DOMAIN}`)``, named volumes for DB/config (NVMe), `${HDD_PATH}/appdata//...` for bulk data, `${USERDATA_PATH}/...` for customer-browsable content. | | App-email (SMTP shim) opt-in | `templates/vaultwarden/.felhom.yml` (`smtp_mapping:`) + README.md §smtp_mapping | `smtp_mapping` maps shim host/port/security/from to the app's own env names; compose MUST reference the mapped `${VAR:-}` keys with empty defaults. STARTTLS if the app can accept self-signed certs, else `security_value: "NONE"` plaintext (or the :2526 plaintext listener for STARTTLS-insistent clients — see calcom/nextcloud). TRAP: an image that treats defined-but-EMPTY mail vars as "set" (vaultwarden — campaign F1 2026-07-06) needs its own enable-flag gated `false` in compose and flipped `"true"` via `smtp_mapping.extra`; boot-prove a fresh email-off deploy for every new smtp-mapped app. | | Probe-container naming | `templates/vaultwarden/docker-compose.yml` (`container_name: vaultwarden`) + `templates/sparkyfitness/` | The controller-side `healthcheck.checks[]` probe dials the container whose **name equals the stack (directory) name exactly**; fallback = the FIRST running prefix-match, which in a multi-container stack can be the DB (verified: `felhom-controller/controller/internal/stacks/healthprobe.go` `findProbeContainer`). So the Traefik-exposed service's `container_name` must be exactly the stack name; sidecars `-db`, `-redis`, …. | diff --git a/onboarding/EXISTING-APPS-GAPS.md b/onboarding/EXISTING-APPS-GAPS.md index d0bf8b7..6a4246b 100644 --- a/onboarding/EXISTING-APPS-GAPS.md +++ b/onboarding/EXISTING-APPS-GAPS.md @@ -1,6 +1,6 @@ # EXISTING APPS — what the catalog already shows, per checklist group -> Generated by `scripts/onboarding_gaps.py` from committed files (catalog `b95f854`). **Do not edit by hand.** +> Generated by `scripts/onboarding_gaps.py` from committed files (catalog `6804f81`). **Do not edit by hand.** > Read only: nothing was re-tested. A cell is what a FILE says, not a measurement made today. The 53 apps > published before the checklist (2026-10-01) are exempt from the onboarding gate; this page is information, > not work (operator default 2026-10-01, may be reversed). @@ -14,7 +14,7 @@ | 2 Storage and backup | 36 / 53 | the 2026-10-02 persistence re-sweep (the fixed gate, R-801) read the app CLEAN, and an HDD app carries `backup:` classes — no restore round trip is recorded per app | | 3 Accounts and strangers | 39 / 53 | a FIRST-ADMIN.md row whose source is MEASURED on a box — lock-out (3.6) is recorded only for the R-752 apps | | 4 Health | 49 / 53 | every service has a compose healthcheck, a controller probe exists, the exposed container is named like the stack — no negative control is recorded | -| 5 Resources | 24 / 53 | every service limited, `mem_limit` = the sum, and a ladder entry carries a measured memory watch — no first-start-from-birth watch is recorded except immich's | +| 5 Resources | 32 / 53 | every service limited, `mem_limit` = the sum, and a ladder entry carries a measured memory watch — no first-start-from-birth watch is recorded except immich's | | 6 Updates | 26 / 53 | a proven (not backfilled) ladder step AND an upgrade fixture | | 7 Mail | — | not countable from files: whether an app WANTS mail is not recorded; the mapped count is below | | 8 Text and listing | 52 / 53 | English block, tagline + use_cases + first_steps, listed in README, a FIRST-ADMIN row | @@ -23,7 +23,7 @@ Mail: 7 app(s) carry `smtp_mapping`. ## Found while computing this page -- `mem_limit` differs from the sum of the compose limits (REUSE.md §2 says equal): 8 — adventurelog (384M vs 896), bookstack (512M vs 768), calcom (768M vs 1792), claper (384M vs 640), kimai (384M vs 640), nextcloud (1024M vs 1664), outline (768M vs 1152), zipline (512M vs 768). +- `mem_limit` differs from the sum of the compose limits (REUSE.md §2 says equal): 0 — none. - A service with no memory limit: none. - A MariaDB sidecar without `MARIADB_AUTO_UPGRADE=1`: none. - A PostgreSQL 18 data mount at the old path: none. @@ -33,13 +33,13 @@ Mail: 7 app(s) carry `smtp_mapping`. | app | 0 fit | 1 images/DB | 2 storage | 3 accounts | 4 health | 5 resources | 6 updates | 7 mail | 8 text | |---|---|---|---|---|---|---|---|---|---| | actualbudget | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate | yes | no watch | 0 proven step(s), fixture | — | yes | -| adventurelog | yes | engine rules hold | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | hc missing | watched 73%, mem_limit≠sum | 1 proven step(s), fixture | — | yes | +| adventurelog | yes | engine rules hold | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | hc missing | watched 73% | 1 proven step(s), fixture | — | yes | | audiobookshelf | yes | no DB sidecar | sweep clean, backup classes | class 4, measured + setup_gate | yes | watched 10% | 1 proven step(s), fixture | — | yes | | bentopdf | yes | no DB sidecar | sweep undetermined | class 5, read only | yes | no watch | 0 proven step(s), no fixture | — | yes | -| bookstack | yes | engine rules hold | sweep clean | class 3, measured + after_install | yes | watched 44%, mem_limit≠sum | 1 proven step(s), fixture | — | yes | -| calcom | yes | engine rules hold | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | yes | watched 62%, mem_limit≠sum | 1 proven step(s), fixture | smtp mapped | yes | +| bookstack | yes | engine rules hold | sweep clean | class 3, measured + after_install | yes | watched 44% | 1 proven step(s), fixture | — | yes | +| calcom | yes | engine rules hold | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | yes | watched 62% | 1 proven step(s), fixture | smtp mapped | yes | | calibre-web | yes | no DB sidecar | sweep clean, backup classes | class 3, measured + after_install | yes | watched 19% | 1 proven step(s), fixture | — | yes | -| claper | yes | engine rules hold | sweep undetermined | class 3 (+ open sign-up), measured + after_install | yes | watched 48%, mem_limit≠sum | 1 proven step(s), fixture | — | yes | +| claper | yes | engine rules hold | sweep undetermined | class 3 (+ open sign-up), measured + after_install | yes | watched 48% | 1 proven step(s), fixture | — | yes | | code-server | yes | no DB sidecar | sweep clean | class 1, read only | yes | no watch | 0 proven step(s), fixture | — | yes | | crafty-controller | yes | no DB sidecar | sweep undetermined | class 1, read only | yes | watched 3% | 1 proven step(s), fixture | — | yes | | docmost | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 80% | 1 proven step(s), fixture | — | yes | @@ -55,15 +55,15 @@ Mail: 7 app(s) carry `smtp_mapping`. | homepage | yes | no DB sidecar | sweep clean | class 5, read only | yes | no watch | 0 proven step(s), fixture | — | yes | | immich | yes | engine rules hold | sweep undetermined, backup classes | class 4, measured + setup_gate | probe container not in compose | watched 51% | 2 proven step(s), no fixture | — | yes | | jellyfin | yes | no DB sidecar | sweep clean, backup classes | class 4, measured + setup_gate | yes | no watch | 0 proven step(s), fixture | — | yes | -| kimai | yes | engine rules hold | sweep clean | class 1, read only | yes | watched 45%, mem_limit≠sum | 2 proven step(s), no fixture | — | yes | +| kimai | yes | engine rules hold | sweep clean | class 1, read only | yes | watched 45% | 2 proven step(s), no fixture | — | yes | | komga | yes | no DB sidecar | sweep clean, backup classes | class 4, measured + setup_gate | yes | watched 64% | 2 proven step(s), no fixture | — | yes | | mealie | yes | no DB sidecar | sweep clean | class 3, measured + after_install | yes | watched 23% | 1 proven step(s), fixture | smtp mapped | yes | | n8n | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate | yes | watched 23% | 3 proven step(s), fixture | — | yes | | navidrome | yes | no DB sidecar | sweep clean, backup classes | class 4, measured + setup_gate | yes | watched 7% | 2 proven step(s), fixture | — | yes | -| nextcloud | yes | engine rules hold | sweep clean, backup classes | class 1, measured | yes | watched 24%, mem_limit≠sum | 2 proven step(s), fixture | smtp mapped | yes | +| nextcloud | yes | engine rules hold | sweep clean, backup classes | class 1, measured | yes | watched 24% | 2 proven step(s), fixture | smtp mapped | yes | | onlyoffice | yes | no DB sidecar | sweep clean | class 5, read only | yes | no watch | 0 proven step(s), fixture | — | yes | | opengist | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate/signup_block | yes | watched 78% | 1 proven step(s), fixture | — | yes | -| outline | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 34%, mem_limit≠sum | 2 proven step(s), fixture | — | yes | +| outline | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 34% | 2 proven step(s), fixture | — | yes | | paperless-ngx | yes | engine rules hold | sweep clean, backup classes | class 1, read only | probe container not in compose | watched 40% | 1 proven step(s), fixture | — | yes | | papra | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | yes | no watch | 0 proven step(s), fixture | — | yes | | plant-it | — (abandoned) | no DB sidecar | sweep undetermined | class 4, read only + setup_gate | yes | no watch | 0 proven step(s), no fixture | — | yes | @@ -84,4 +84,4 @@ Mail: 7 app(s) carry `smtp_mapping`. | wanderer | yes | no DB sidecar | sweep undetermined | class 4, measured + signup_block/after_setup | yes | no watch | 0 proven step(s), no fixture | — | yes | | wger | — (hidden) | no DB sidecar | sweep undetermined | class 3, measured + after_install | yes | watched 50% | 1 proven step(s), fixture | smtp mapped | yes | | wishlist | yes | no DB sidecar | sweep undetermined | class 4, measured + setup_gate/signup_block | yes | watched 36% | 1 proven step(s), fixture | — | yes | -| zipline | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 34%, mem_limit≠sum | 2 proven step(s), fixture | — | yes | +| zipline | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 34% | 2 proven step(s), fixture | — | yes | diff --git a/scripts/catalog_gates.py b/scripts/catalog_gates.py index a8cd309..9d3f753 100644 --- a/scripts/catalog_gates.py +++ b/scripts/catalog_gates.py @@ -25,6 +25,8 @@ Gates, in order (all must pass; **non-zero exit on any failure**): ladder entry whose digests the registry still serves (hook; skipped on CI) 10. onboarding static, instant, whole repo — a NEW template directory carries a complete onboarding record (NEW-APP-CHECKLIST.md; the 53 apps published before 2026-10-01 are exempt by name) + 11. mem-limit-sum static, instant, whole repo — `.felhom.yml` resources.mem_limit equals the sum of every + service's deploy.resources.limits.memory, and every service has one (R-758) 4. engine-major static, needs GIT HISTORY — no database engine pin crosses a MAJOR version (operator ruling 2026-09-13; expires when Slice 4 / R-448 ships). Runs in the pre-push hook, which has the full clone; on a SHALLOW clone (CI fetches at @@ -117,6 +119,10 @@ GATES = [ # 2026-10-02 (`09` §3 decisions 63/64): a family-gated template's exceptions are literal prefixes, it declares # min_controller >= 0.287.0, and the newest baked golden knows the gate. Static, files only. ("family-gate", "check-family-gate.py", False, True, False), + # R-758 (2026-10-05): `.felhom.yml` resources.mem_limit is the SUM of the compose limits (REUSE.md §2). Eight + # templates were under it — the deploy screen and the box's overcommit warning read less than the app may take. + # Static, stdlib only (no PyYAML on CI), so --fast: the hook and CI. + ("mem-limit-sum", "check-mem-limit-sum.py", True, True, False), ] # 3 (R-605): the gate's HARNESS refused to run — its canary failed or it had nothing to judge — so NO app was diff --git a/scripts/check-mem-limit-sum.py b/scripts/check-mem-limit-sum.py new file mode 100644 index 0000000..6fc0251 --- /dev/null +++ b/scripts/check-mem-limit-sum.py @@ -0,0 +1,193 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +"""check-mem-limit-sum.py — `.felhom.yml` `resources.mem_limit` must equal the SUM of the compose limits (R-758). + +WHAT WENT WRONG. REUSE.md §2 says `mem_limit` = the sum of every service's +`deploy.resources.limits.memory` (paperless: 768+256+128=1152M). Nothing checked it, and on 2026-10-01 the new-app +checklist's gap page found eight templates UNDER their sum (calcom 768M declared, 1792M enforced). Docker enforces the +compose limits, so no app was starved; what was wrong is the figure the deploy screen shows the household and the +figure the box's overcommit warning adds up (controller `memoryVerdict`, deploy.go) — both read less than the app may +take. + +THE FACT, and the labels that are not it: + * The fact is a service's `memory:` key UNDER `deploy: resources: limits:` (Compose enforces that), and the + `mem_limit:` key UNDER `.felhom.yml`'s top-level `resources:`. + * NOT the fact: a `memory:` under `reservations:` (not a limit — and REUSE forbids reservations anyway), a figure in a + COMMENT (the compose header's "RAM: … (mem_limit: 384M)" line, the `.felhom.yml` arithmetic comment), a + `mem_limit:` anywhere but under `resources:`, the per-step files under `steps/` (a superseded step's own + definition, written by the ladder writer — not what a fresh install deploys). + * A service with NO limit is refused too (REUSE.md §2: every service has one): the sum would be a lie of omission. + +stdlib only, a line reader: the CI runner has no PyYAML (memory note "catalog CI has no PyYAML"), and a gate that +needs it would have to degrade — this one never needs it. + +USAGE + python3 scripts/check-mem-limit-sum.py # every template directory + python3 scripts/check-mem-limit-sum.py kimai calcom # only these + python3 scripts/check-mem-limit-sum.py --root= # judge another checkout (the decoy suite) + (`--all` is accepted and changes nothing: hidden and abandoned apps are always judged — a deployed one still runs.) +Exit: 0 every template's mem_limit is its sum · 1 convicted · 2 inconclusive (a figure nobody can read). +Decoys: scripts/test_gate_decoys.py `mem_sum_cases` (COVERS "mem-limit-sum"). +""" +import io +import os +import re +import sys + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + +SIZE_RE = re.compile(r"^\s*([0-9]+(?:\.[0-9]+)?)\s*([kKmMgG])(?:i?[bB])?\s*$") + + +def to_mb(value): + """'384M' / '1G' / '1.5g' / '512Mi' -> MB (int, rounded); None when unreadable.""" + m = SIZE_RE.match(str(value).strip().strip('"').strip("'")) + if not m: + return None + n, unit = float(m.group(1)), m.group(2).lower() + return int(round(n / 1024.0 if unit == "k" else n * 1024 if unit == "g" else n)) + + +def _strip_comment(line): + """Drop a trailing `# comment` that is not inside quotes (good enough for these files' scalars).""" + out, q = [], None + for i, ch in enumerate(line): + if q: + if ch == q: + q = None + elif ch in ("'", '"'): + q = ch + elif ch == "#" and (i == 0 or line[i - 1] in " \t"): + break + out.append(ch) + return "".join(out).rstrip() + + +def _walk(text): + """Yield (path_of_keys, key, value) for every `key: value` / `key:` line, by indentation. List items and + block scalars are skipped (no limit lives in one).""" + stack = [] # [(indent, key)] + block_indent = None # inside a `|` / `>` block scalar: skip lines deeper than this + for raw in text.split("\n"): + line = _strip_comment(raw) + if not line.strip(): + continue + indent = len(line) - len(line.lstrip(" ")) + if block_indent is not None: + if indent > block_indent: + continue + block_indent = None + s = line.strip() + if s.startswith("- "): + continue + m = re.match(r"^([A-Za-z0-9_.\-]+):(?:\s+(.*))?$", s) + if not m: + continue + while stack and stack[-1][0] >= indent: + stack.pop() + key, val = m.group(1), (m.group(2) or "").strip() + yield [k for _i, k in stack], key, val + if val in ("|", ">", "|-", ">-", "|+", ">+"): + block_indent = indent + elif val == "": + stack.append((indent, key)) + + +def compose_limits(text): + """{service: limit_mb or None (no limit) or 'bad:' (unreadable)}.""" + services = {} + for path, key, val in _walk(text): + if path == ["services"]: + services.setdefault(key, None) + elif len(path) == 5 and path[0] == "services" and path[2:] == ["deploy", "resources", "limits"] \ + and key == "memory": + mb = to_mb(val) + services[path[1]] = mb if mb is not None else "bad:" + val + return services + + +def declared_limit(text): + """The top-level `resources: mem_limit:` -> (mb or None, raw).""" + for path, key, val in _walk(text): + if path == ["resources"] and key == "mem_limit": + return to_mb(val), val + return None, None + + +def check_app(tdir, app): + """-> (verdict 0/1/2, message).""" + d = os.path.join(tdir, app) + try: + compose = io.open(os.path.join(d, "docker-compose.yml"), encoding="utf-8").read() + meta = io.open(os.path.join(d, ".felhom.yml"), encoding="utf-8").read() + except (IOError, OSError) as e: + return 2, "%s: unreadable template (%s)" % (app, e) + lims = compose_limits(compose) + if not lims: + return 2, "%s: no services found in docker-compose.yml" % app + bad = sorted(s for s, v in lims.items() if isinstance(v, str)) + if bad: + return 2, "%s: a memory limit nobody can read on %s (%s)" % (app, ", ".join(bad), + ", ".join(lims[s][4:] for s in bad)) + missing = sorted(s for s, v in lims.items() if v is None) + if missing: + return 1, "%s: service(s) with NO deploy.resources.limits.memory: %s (REUSE.md §2: every service has one)" % ( + app, ", ".join(missing)) + total = sum(lims.values()) + ml, raw = declared_limit(meta) + if raw is None: + return 1, "%s: .felhom.yml declares no resources.mem_limit (the sum is %dM)" % (app, total) + if ml is None: + return 2, "%s: .felhom.yml mem_limit %r is not a size" % (app, raw) + if ml != total: + parts = "+".join("%d" % lims[s] for s in lims) + return 1, "%s: .felhom.yml mem_limit %s is not the sum of the compose limits %s=%dM" % (app, raw, parts, total) + return 0, "%s: %dM = the sum" % (app, total) + + +def main(argv): + root, apps = ROOT, [] + for a in argv: + if a.startswith("--root="): + root = a.split("=", 1)[1] + elif a == "--all": + continue # catalog_gates.py passes it for hidden/abandoned apps; this gate judges every directory anyway + elif a.startswith("-"): + print("unknown option: %s" % a) + return 2 + else: + apps.append(a) + tdir = os.path.join(root, "templates") + if not os.path.isdir(tdir): + print("mem-limit-sum: no templates/ under %s" % root) + return 2 + every = sorted(n for n in os.listdir(tdir) if os.path.isdir(os.path.join(tdir, n))) + unknown = [a for a in apps if a not in every] + if unknown: + print("mem-limit-sum: no such template: %s" % ", ".join(unknown)) + return 2 + convicted, undecided = [], [] + for app in (apps or every): + v, msg = check_app(tdir, app) + if v == 1: + convicted.append(msg) + elif v == 2: + undecided.append(msg) + for m in convicted: + print("REFUSED: " + m) + for m in undecided: + print("INCONCLUSIVE: " + m) + n = len(apps or every) + if convicted: + print("mem-limit-sum: %d of %d template(s) refused — set .felhom.yml resources.mem_limit to the sum of the " + "compose limits (and show the arithmetic in a comment, like paperless-ngx)" % (len(convicted), n)) + return 1 + if undecided: + print("mem-limit-sum: INCONCLUSIVE on %d of %d template(s) — never a pass" % (len(undecided), n)) + return 2 + print("mem-limit-sum gate OK: %d template(s), every mem_limit is the sum of its compose limits" % n) + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv[1:])) diff --git a/scripts/test_catalog_gates.py b/scripts/test_catalog_gates.py index fae515b..d438b6b 100644 --- a/scripts/test_catalog_gates.py +++ b/scripts/test_catalog_gates.py @@ -64,7 +64,9 @@ class CatalogGatesFastTest(unittest.TestCase): # probe-measured joined; the test had been red on main (found by the more-night-apps session). # 11 since 2026-10-01: onboarding (NEW-APP-CHECKLIST.md), fast and history-free, so it runs in CI too. # 12 since family-gate joined — STALE AGAIN until 2026-10-05 (found by the burn-down, round 2). - self.assertEqual(len(mod.GATES), 12) + # 13 since 2026-10-05: mem-limit-sum (R-758), fast and history-free, so it runs in CI too. + self.assertEqual(len(mod.GATES), 13) + self.assertIn("mem-limit-sum", [g[0] for g in mod.GATES if g[3] and not g[4]]) self.assertIn("onboarding", [g[0] for g in mod.GATES if g[3] and not g[4]]) self.assertEqual([g[0] for g in mod.GATES if not g[3]], ["image-resolvable", "volume-persistence"]) self.assertIn("test-record", [g[0] for g in mod.GATES if g[3] and not g[4]]) # runs in CI too diff --git a/scripts/test_gate_decoys.py b/scripts/test_gate_decoys.py index 76c1c5b..61ddf50 100644 --- a/scripts/test_gate_decoys.py +++ b/scripts/test_gate_decoys.py @@ -57,6 +57,7 @@ COVERS = { "probe-measured": "the measurement written in the TAGLINE or another comment block, not directly above setup_done_probe:; a date with no before/after; before/after with no date; 'read upstream' instead of 'measured' - vs a genuine measured comment (R-715)", "family-gate": "family_gate written only in a COMMENT (not gated, no min_controller owed); min_controller only in a comment; a golden DIRECTORY named 0.287.0 with no bake log (the mkdir shape, R-410); a sibling with no golden (stated NOT CHECKED, never a pass of rule 3) - vs the facts: an unanchorable exception (regex, '/', '..'), an exception list with no gate, min_controller below 0.287.0, the newest baked golden below 0.287.0; and a genuine family app passes (decisions 63/64, finding F1)", "onboarding": "a NEW template with no record; a record missing an id, or carrying it only inside an HTML comment; a `done` whose path does not exist, is an EMPTY directory (the mkdir shape, R-410) or names an absent sibling-repo file; an `n/a` with an empty or two-word reason; an `open` row; `opened:` backdated before the checklist; the template a new app copies lacking a new id - vs a complete record, an id added after `opened:`, and an exempt app's record with open rows (NEW-APP-CHECKLIST.md)", + "mem-limit-sum": "the right figure only in a COMMENT (the compose header's 'mem_limit: 640M', the .felhom.yml arithmetic) while the field is wrong; a `memory:` under reservations: (not a limit) making the sum come out right; a `mem_limit:` outside resources:; a steps/ file with the old figure (not judged) - vs the facts: a field under the sum, a service with no limit, an unreadable size (R-758)", "copy-i18n": "Hungarian edited in a COMMENT/README/display_name (label, not copy) vs a real frozen string changed; an English block that is not English, is not matched to a Hungarian twin, or rewrites a credential (R-560). A retrieval promise REGISTERED in ALLOWLIST_EN passes only for its own app+path+sentence with a real reason; an entry for another app, a rewritten sentence, a stale entry or a two-word reason convicts (R-594). Also the DEGRADED mode CI actually runs — PyYAML shadowed out, freeze only (R-595)", } @@ -686,6 +687,66 @@ def family_gate_cases(): shutil.rmtree(ws, ignore_errors=True) +def mem_sum_cases(): + """check-mem-limit-sum.py reads FILES: templates/*/docker-compose.yml + .felhom.yml, via --root (R-758).""" + global ran + import tempfile + ws = tempfile.mkdtemp(prefix="catalog-memsum-") + try: + COMPOSE = ( + "# demo - header\n# RAM: ~100M (mem_limit: 640M)\n" + "services:\n" + " demo:\n image: demo/demo:1.0\n container_name: demo\n environment:\n - X=1\n" + " deploy:\n resources:\n limits:\n memory: 384M\n" + " demo-db:\n image: postgres:16-alpine\n command: |\n memory: 9999M\n" + " deploy:\n resources:\n limits:\n memory: 256M # the DB\n" + "volumes:\n demo_data:\n") + META = ('display_name: "Demo"\n# mem_limit: "999M" is not this line\n' + 'resources:\n mem_request: "100M"\n mem_limit: "640M" # 384+256\n pi_compatible: true\n') + + def run(name, compose, meta, expect_rc, must=(), extra=None): + global ran + cat = os.path.join(ws, "cat") + shutil.rmtree(cat, ignore_errors=True) + d = os.path.join(cat, "templates", "demo") + os.makedirs(d) + io.open(os.path.join(d, "docker-compose.yml"), "w", encoding="utf-8").write(compose) + io.open(os.path.join(d, ".felhom.yml"), "w", encoding="utf-8").write(meta) + if extra: + extra(d) + r = sh([sys.executable, os.path.join(ROOT, "scripts", "check-mem-limit-sum.py"), "--root=" + cat], ROOT) + out = r.stdout + r.stderr + ran += 1 + ok = r.returncode == expect_rc and all(m in out for m in must) + print(" %s %-70s rc=%d (expected %d)" % ("ok" if ok else "XX", name, r.returncode, expect_rc)) + if not ok: + fails.append("%s: rc=%d expected %d; missing %s\n%s" % ( + name, r.returncode, expect_rc, [m for m in must if m not in out], out[-400:])) + + def steps(d): + os.makedirs(os.path.join(d, "steps")) + io.open(os.path.join(d, "steps", "abc.felhom.yml"), "w").write('resources:\n mem_limit: "384M"\n') + + print("\n-- mem-limit-sum: genuine and decoys") + run("GENUINE: 384+256 = 640M, the field says 640M", COMPOSE, META, 0, ("mem-limit-sum gate OK",)) + run("GENUINE: 1G is 1024M (1G + 256M = 1280M)", COMPOSE.replace("memory: 384M", "memory: 1G"), + META.replace('"640M"', '"1280M"'), 0, ("gate OK",)) + run("DECOY: a steps/ file with an old figure is not judged", COMPOSE, META, 0, ("gate OK",), extra=steps) + print("-- mem-limit-sum: the facts (each MUST be refused)") + run("FACT: the field under the sum; the right figure only in comments", COMPOSE, + META.replace('mem_limit: "640M" # 384+256', 'mem_limit: "384M" # 384+256=640M'), 1, ("not the sum", "384+256=640M")) + run("FACT: reservations: memory makes nothing a limit (service w/o limit)", + COMPOSE.replace(" limits:\n memory: 256M", " reservations:\n memory: 256M"), + META.replace('"640M"', '"384M"'), 1, ("NO deploy.resources.limits.memory", "demo-db")) + run("FACT: mem_limit only OUTSIDE resources: (top level) is no declaration", COMPOSE, + META.replace(' mem_limit: "640M" # 384+256\n', '').replace('display_name: "Demo"\n', 'display_name: "Demo"\nmem_limit: "640M"\n'), + 1, ("declares no resources.mem_limit",)) + run("FACT: an unreadable size is INCONCLUSIVE, never a pass", COMPOSE.replace("memory: 384M", "memory: lots"), META, 2, + ("INCONCLUSIVE",)) + finally: + shutil.rmtree(ws, ignore_errors=True) + + def main(): gate = os.path.join(ROOT, "scripts", "check-engine-major.py") if not os.path.isfile(gate): @@ -1197,6 +1258,7 @@ i18n: onboarding_cases() family_gate_cases() + mem_sum_cases() if fails: print() diff --git a/templates/adventurelog/.felhom.yml b/templates/adventurelog/.felhom.yml index 5a164b8..bc5beff 100644 --- a/templates/adventurelog/.felhom.yml +++ b/templates/adventurelog/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-09-27" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "100M" - mem_limit: "384M" + mem_limit: "896M" # the sum of the compose limits (384+256+256) — R-758; was "384M" pi_compatible: true needs_hdd: false diff --git a/templates/bookstack/.felhom.yml b/templates/bookstack/.felhom.yml index c6ab3a5..1ae5d1a 100644 --- a/templates/bookstack/.felhom.yml +++ b/templates/bookstack/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-09-30" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "150M" - mem_limit: "512M" + mem_limit: "768M" # the sum of the compose limits (512+256) — R-758; was "512M" pi_compatible: true needs_hdd: false diff --git a/templates/calcom/.felhom.yml b/templates/calcom/.felhom.yml index 1f41caa..83bcf99 100644 --- a/templates/calcom/.felhom.yml +++ b/templates/calcom/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-09-28" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "200M" - mem_limit: "768M" + mem_limit: "1792M" # the sum of the compose limits (1536+256) — R-758; was "768M" pi_compatible: false needs_hdd: false diff --git a/templates/claper/.felhom.yml b/templates/claper/.felhom.yml index 4975770..95b1c11 100644 --- a/templates/claper/.felhom.yml +++ b/templates/claper/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-09-28" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "100M" - mem_limit: "384M" + mem_limit: "640M" # the sum of the compose limits (384+256) — R-758; was "384M" pi_compatible: true needs_hdd: false diff --git a/templates/kimai/.felhom.yml b/templates/kimai/.felhom.yml index 07c6c0a..41b8d35 100644 --- a/templates/kimai/.felhom.yml +++ b/templates/kimai/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-09-30" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "100M" - mem_limit: "384M" + mem_limit: "640M" # the sum of the compose limits (384+256) — R-758; was "384M" pi_compatible: true needs_hdd: false diff --git a/templates/nextcloud/.felhom.yml b/templates/nextcloud/.felhom.yml index 6438b85..8d87fc9 100644 --- a/templates/nextcloud/.felhom.yml +++ b/templates/nextcloud/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-10-01" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "256M" - mem_limit: "1024M" + mem_limit: "1664M" # the sum of the compose limits (1024+512+128) — R-758; was "1024M" pi_compatible: false needs_hdd: true diff --git a/templates/outline/.felhom.yml b/templates/outline/.felhom.yml index 6c61b87..3b8819a 100644 --- a/templates/outline/.felhom.yml +++ b/templates/outline/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-09-30" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "200M" - mem_limit: "768M" + mem_limit: "1152M" # the sum of the compose limits (768+256+128) — R-758; was "768M" pi_compatible: false needs_hdd: false diff --git a/templates/zipline/.felhom.yml b/templates/zipline/.felhom.yml index 0bca3b4..f8311b2 100644 --- a/templates/zipline/.felhom.yml +++ b/templates/zipline/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-09-30" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "100M" - mem_limit: "512M" + mem_limit: "768M" # the sum of the compose limits (512+256) — R-758; was "512M" pi_compatible: false needs_hdd: false