Website: the machine estimator (/gepigeny, /en/sizing), live but unlisted; SIZING.md §5 in three verdicts; R-997 filed
gates / gates (push) Successful in 6m42s

A household ticks apps and sees, for an 8, 16 and 32 GB box: fits well / fits, but
tight / does not fit, the totals, and the apps that burst on CPU. One rule in
assets/sizing.js, every number from assets/sizing-data.json (built by
scripts/build_sizing_web.py from the 2026-10-13 N100 measurement; test: 7 cases,
red-proofed). Unlisted: noindex, robots Disallow, not in the sitemap, linked from
nowhere — site_gates.py gate 25 enforces it (UNLISTED twin pairs keep every twin
check). SIZING.md §5 corrected: the 8 GB line contradicted its own x1.5 rule; guest
sizes 4.0/11.4/27.4 GB with the measured 612 MiB MemTotal shortfall. R-993 gains the
fact that the installer accepts a host below any minimum.
This commit is contained in:
2026-10-13 14:31:44 +02:00
parent fdca4971af
commit d5ef93b13b
14 changed files with 1813 additions and 15 deletions
+16
View File
@@ -1,3 +1,19 @@
## 2026-10-13 — `build_sizing_web.py` + `site_gates.py` UNLISTED pairs and gate 25
- **`build_sizing_web.py`** builds `website/assets/sizing-data.json` (the machine estimator's data) from
`documentation/audits/sizing-2026-10-13/sizing.json` and the two apps pages. Refuses (exit 1, writes nothing)
when a listed app is missing from `sizing.json` (FileBrowser excepted by name: built in), when a peak is below
its typical figure, or when the two apps pages disagree. `--check` compares the committed file with a fresh build.
- **`test_build_sizing_web.py`** (7 tests): fresh build = committed file; the four cases of SIZING.md §5 give the
expected verdicts; SIZING.md's verdict table agrees with the rule; `sizing.js` carries the rule's shape and none
of its numbers. Red-proofed: a hard-coded 470 in the JS, a changed headroom in the data, and a wrong table cell
each convicted. No JS runtime exists on DooPlex or in CI, so the rule is mirrored in Python on the same data.
- **`site_gates.py`**: `UNLISTED` twin pairs get every twin check (hreflang, globe, canonical, lang, no Hungarian
on the English page) and are excluded from the sitemap requirement; **gate 25** adds what unlisted means —
`noindex`, a `Disallow:` line per URL in `robots.txt`, absent from `sitemap.xml`, linked from no other page.
Red-proofed: each of the four, and a wrong twin alternate, convicted on its own; the first run caught the missing
`robots.txt` lines for real.
## 2026-10-12 (night) — make_hero_clip.py: scenes keep their own length
- **`--seconds` is no longer a fixed window.** Each scene now plays its full recorded length, read
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""build_sizing_web.py — website/assets/sizing-data.json for the machine estimator page.
python3 -I scripts/build_sizing_web.py write the file
python3 -I scripts/build_sizing_web.py --check exit 1 if the committed file differs from a fresh build
Inputs (read only):
documentation/audits/sizing-2026-10-13/sizing.json the 2026-10-13 N100 measurement
website/alkalmazasok.html, website/en/apps.html the apps the site lists, their names and categories
The page offers EXACTLY the apps the two apps pages list. Refusals (exit 1, nothing written):
- an app listed on a page but absent from sizing.json (FileBrowser excepted, BY NAME: it is built
into every box and its memory is inside the platform figure — SIZING.md §1)
- an app whose peak is below its typical figure
- the two pages listing different apps
The rule's numbers live HERE and in the data file only; website/assets/sizing.js reads them from the
file and has none of its own (scripts/test_build_sizing_web.py checks that).
"""
import hashlib, io, json, os, re, sys
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
SIZING = os.path.join(ROOT, "documentation", "audits", "sizing-2026-10-13", "sizing.json")
SOURCE_COMMIT = "c885b5ac" # the commit that added sizing.json; the sha256 below shows when it moved on
HU_PAGE = os.path.join(ROOT, "website", "alkalmazasok.html")
EN_PAGE = os.path.join(ROOT, "website", "en", "apps.html")
OUT = os.path.join(ROOT, "website", "assets", "sizing-data.json")
# SIZING.md §1/§5: platform inside the guest = its containers (255 MB) + the Docker daemons (≈ 220 MB RSS)
PLATFORM_MB = 470
HEADROOM = 1.5
# scripts/felhom-host-install.sh autosize_guest_caps(): guest = clamp(host-4096, min 4096, max host-2048),
# then the ceiling host-1024. `host` is MemTotal, which is below the label: the N100 sold as 16 GB reports
# 15,772 MiB (measured 2026-10-13), so a box of N GB is taken as N*1024 - 612 MiB.
GUEST_RULE = {"reserve_mib": 4096, "max_gap_mib": 2048, "min_mib": 4096, "ceiling_gap_mib": 1024,
"memtotal_shortfall_mib": 612}
BOXES_GB = [8, 16, 32]
BUILTIN = {"filebrowser"}
# One short line per medium / heavy-in-bursts app (the 9 of 2026-10-13). sizing.json's English notes, tidied.
CPU_NOTES = {
"immich": ("csak amíg új fotókat dolgoz fel (bélyegképek, arcok, keresés): 200 fotó kb. 6 perc",
"only while it processes new photos (thumbnails, faces, search): 200 photos took about 6 minutes"),
"jellyfin": ("csak amíg egy filmet átalakít egy olyan eszköznek, amely azt nem tudja lejátszani",
"only while it converts a film for a device that cannot play it"),
"paperless-ngx": ("csak amíg új beszkennelt dokumentumokat olvas be (szövegfelismerés, kb. 4 oldal percenként)",
"only while it reads new scanned documents (text recognition, about 4 pages a minute)"),
"calibre-web": ("csak amíg új könyveket vesz fel", "only while it imports new books"),
"romm": ("az első indításkor; a játékkönyvtár átnézését nem mértük", "at first start; a library scan was not measured"),
"onlyoffice": ("csak az első indításkor", "only at first start"),
"grimmory": ("csak az első indításkor", "only at first start"),
"tandoor": ("csak az első indításkor", "only at first start"),
"adventurelog": ("csak az első indításkor", "only at first start"),
}
def fail(msg):
print("build_sizing_web: REFUSED —", msg)
sys.exit(1)
def text(s):
return re.sub(r"\s+", " ", re.sub(r"<[^>]+>", "", s)).replace("&amp;", "&").strip()
def parse_apps_page(path):
t = io.open(path, encoding="utf-8-sig").read()
cats, apps = [], []
for sec in re.finditer(r'<div class="category-section" id="([^"]+)">(.*?)(?=<div class="category-section"|<section class="stats-section")', t, re.S):
cats.append((sec.group(1), text(re.search(r"<h2>(.*?)</h2>", sec.group(2), re.S).group(1))))
for c in re.finditer(r'<div class="app-card" id="([^"]+)">.*?<h3>(.*?)</h3>(?:<span class="app-tagline">(.*?)</span>)?', sec.group(2), re.S):
apps.append(dict(slug=c.group(1), category=len(cats) - 1, name=text(c.group(2)), tagline=text(c.group(3) or "")))
return cats, apps
def build():
raw = open(SIZING, "rb").read()
sz = json.loads(raw)
S = {a["slug"]: a for a in sz["apps"]}
hu_cats, hu = parse_apps_page(HU_PAGE)
en_cats, en = parse_apps_page(EN_PAGE)
if [a["slug"] for a in hu] != [a["slug"] for a in en] or len(hu_cats) != len(en_cats):
fail("the Hungarian and English apps pages do not list the same apps in the same order")
E = {a["slug"]: a for a in en}
apps = []
for a in hu:
slug = a["slug"]
rec = dict(slug=slug, name_hu=a["name"], name_en=E[slug]["name"], tagline_hu=a["tagline"],
tagline_en=E[slug]["tagline"], category=hu_cats[a["category"]][0])
if slug in BUILTIN:
rec.update(builtin=True, typical_mb=0, peak_mb=0, cpu_class="light", cpu_note_hu="", cpu_note_en="",
measured=True)
else:
if slug not in S:
fail("%s is listed on the apps pages but missing from sizing.json" % slug)
s = S[slug]
if s["mem_peak_mb"] is None or s["mem_typical_mb"] is None:
fail("%s has no memory figure in sizing.json" % slug)
if s["mem_peak_mb"] < s["mem_typical_mb"]:
fail("%s: peak %s < typical %s in sizing.json" % (slug, s["mem_peak_mb"], s["mem_typical_mb"]))
cls = s["cpu_class"]
note = CPU_NOTES.get(slug, ("", ""))
if cls in ("medium", "heavy-in-bursts") and not note[0]:
fail("%s is %s but has no CPU note here" % (slug, cls))
rec.update(typical_mb=s["mem_typical_mb"], peak_mb=s["mem_peak_mb"], cpu_class=cls,
cpu_note_hu=note[0] if cls != "light" else "", cpu_note_en=note[1] if cls != "light" else "",
measured=s["source"] != "unmeasured")
apps.append(rec)
return dict(
schema=1,
measured="2026-10-13",
measured_on="Intel N100 mini PC, 16 GB (demo-felhom)",
source=dict(path="documentation/audits/sizing-2026-10-13/sizing.json", commit=SOURCE_COMMIT,
sha256=hashlib.sha256(raw).hexdigest()),
platform_mb=PLATFORM_MB,
headroom=HEADROOM,
guest_rule=GUEST_RULE,
boxes_gb=BOXES_GB,
categories=[dict(id=h[0], id_en=e[0], name_hu=h[1], name_en=e[1]) for h, e in zip(hu_cats, en_cats)],
apps=apps,
)
def render(d):
return json.dumps(d, ensure_ascii=False, indent=1) + "\n"
if __name__ == "__main__":
out = render(build())
if "--check" in sys.argv:
cur = io.open(OUT, encoding="utf-8").read() if os.path.exists(OUT) else ""
if cur != out:
print("build_sizing_web: website/assets/sizing-data.json is NOT what the builder makes — rebuild it")
sys.exit(1)
print("build_sizing_web: sizing-data.json matches a fresh build")
else:
io.open(OUT, "w", encoding="utf-8", newline="\n").write(out)
print("wrote", os.path.relpath(OUT, ROOT), "-", len(json.loads(out)["apps"]), "apps")
+44 -6
View File
@@ -109,6 +109,15 @@ URL = {"index.html": "/", "alkalmazasok.html": "/alkalmazasok", "technologiak.ht
# English twin is written then, as a twin.
NO_TWIN = {"szolgaltatasok-nonpublic.html", "404.html", "adatkezeles.html", "feltetelek.html"}
# UNLISTED (2026-10-13, the machine estimator) — a twin pair that is LIVE but deliberately unlisted until the
# operator says "list it" (R-997). It gets every twin check (hreflang, globe, canonical, lang, no Hungarian on the
# English page), and gate 25 adds what unlisted means: `noindex` on both pages, a `Disallow:` line for each URL in
# robots.txt, absent from sitemap.xml, and linked from no other page. To LIST it: move the pair into TWINS, drop
# noindex and the Disallow lines, add both URLs to the sitemap and a link from the menu — in one commit.
UNLISTED = {"gepigeny.html": os.path.join("en", "sizing.html")}
URL.update({"gepigeny.html": "/gepigeny", os.path.join("en", "sizing.html"): "/en/sizing"})
ALL_TWINS = dict(TWINS, **UNLISTED)
MOCKUPS = set() # the design studies were deleted with website/preview/ at the M3 switch
# SHELL (2026-10-11, M3) — every public page now wears the controller's frame instead of the
@@ -138,10 +147,12 @@ SHELL = {
os.path.join("en", 'testing.html'),
os.path.join("en", 'contact.html'),
os.path.join("en", 'download.html'),
'gepigeny.html',
os.path.join("en", 'sizing.html'),
}
PAGES = list(TWINS) + list(TWINS.values()) + sorted(NO_TWIN) + sorted(MOCKUPS) + sorted(SHELL)
EN_PAGES = set(TWINS.values())
PAGES = list(ALL_TWINS) + list(ALL_TWINS.values()) + sorted(NO_TWIN) + sorted(MOCKUPS) + sorted(SHELL)
EN_PAGES = set(ALL_TWINS.values())
HU_PAGES = set(PAGES) - EN_PAGES - MOCKUPS - SHELL
ANALYTICS_EXEMPT = {"szolgaltatasok-nonpublic.html"} | MOCKUPS
@@ -425,7 +436,7 @@ def check_globe(p, lang, urls=None):
ALT_RE = re.compile(r'<link rel="alternate" hreflang="([^"]+)" href="([^"]+)">')
for hu, en in TWINS.items():
for hu, en in ALL_TWINS.items():
if hu not in pages or en not in pages:
fail("twin pair %s <-> %s: a page of the pair is missing" % (hu, en))
continue
@@ -438,7 +449,7 @@ for hu, en in TWINS.items():
canon = re.search(r'<link rel="canonical" href="([^"]+)">', pages[p])
if not canon or canon.group(1) != SITE + URL[p]:
fail("%s: canonical %r — it must point at the page itself (%s)" % (p, canon and canon.group(1), SITE + URL[p]))
print(" twins: %d pairs checked both ways (hreflang, globe, canonical)" % len(TWINS))
print(" twins: %d pairs checked both ways (hreflang, globe, canonical), %d of them unlisted" % (len(ALL_TWINS), len(UNLISTED)))
for p in sorted(NO_TWIN): # no twin: the globe's structure, current mark and label still hold
if p in pages:
check_globe(p, lang_of(p))
@@ -546,7 +557,7 @@ for p in sorted(x for x in pages if is_en(x)):
if BEG.search(t):
fail("%s: the site does not beg — no \"please\"/\"kindly\": %r" % (p, t[:120]))
# the retrieval promise: the English page may make it no more often than its Hungarian twin does
hu = next((h for h, e in TWINS.items() if e == p), None)
hu = next((h for h, e in ALL_TWINS.items() if e == p), None)
if hu and hu in pages:
en_n = sum(len(re.findall(rx, t, re.I)) for t in visible_strings(s) for rx in _RET_EN)
hu_n = sum(fold(t).count(fold(st)) for t in visible_strings(pages[hu]) for st in _RET_HU)
@@ -817,7 +828,8 @@ if not os.path.exists(_SITEMAP):
else:
_sm = io.open(_SITEMAP, encoding="utf-8-sig").read()
_locs = [re.sub(r"^https?://felhom\.eu", "", u) for u in re.findall(r"<loc>([^<]+)</loc>", _sm)]
_want = set(URL.values()) | _LEGAL_URLS
_unlisted_urls = {URL[_p] for _pair in UNLISTED.items() for _p in _pair}
_want = (set(URL.values()) - _unlisted_urls) | _LEGAL_URLS
_have = set(_locs)
for _u in sorted(_want - _have):
fail("sitemap.xml: %s is a public page and is NOT listed — search engines are told the site "
@@ -844,6 +856,32 @@ else:
if not [f for f in fails if "sitemap.xml" in f]:
print(" sitemap: %d URL(s), every public page once, every twin pair cross-named" % len(_locs))
# ---------------------------------------------------------------------------------------------
# gate 25 (2026-10-13): an UNLISTED pair stays unlisted until the operator lists it (R-997). Live and reachable by its
# link, but: noindex on the page, Disallow in robots.txt, absent from sitemap.xml, and no other page links to it —
# each of the four is what "unlisted" means, and losing any one of them quietly publishes the page.
_ROBOTS = os.path.join(W, "robots.txt")
_robots = io.open(_ROBOTS, encoding="utf-8-sig").read() if os.path.exists(_ROBOTS) else ""
_sm_locs = set(re.sub(r"^https?://felhom\.eu", "", u) for u in re.findall(r"<loc>([^<]+)</loc>", io.open(_SITEMAP, encoding="utf-8-sig").read())) if os.path.exists(_SITEMAP) else set()
_unl_pages = {p for _pair in UNLISTED.items() for p in _pair}
for _p in sorted(_unl_pages):
_s = pages.get(_p, "")
_u = URL[_p]
_m = re.search(r'<meta name="robots" content="([^"]*)">', _s)
if not _m or "noindex" not in [x.strip() for x in _m.group(1).split(",")]:
fail("%s: an UNLISTED page without <meta name=\"robots\" content=\"noindex\"> — search engines may show it" % _p)
if not re.search(r"(?m)^Disallow:\s*%s\s*$" % re.escape(_u), _robots):
fail("robots.txt: no 'Disallow: %s' — the UNLISTED page %s is open to crawlers" % (_u, _p))
if _u in _sm_locs:
fail("sitemap.xml: %s is listed — an UNLISTED page must not be in the sitemap" % _u)
_link = re.compile(r'href="(?:https?://felhom\.eu)?%s(?:[#?"])' % re.escape(_u))
for _q in sorted(pages):
if _q in _unl_pages:
continue # the pair's own globe, canonical and alternates point at each other
if _link.search(pages[_q]):
fail("%s links to the UNLISTED page %s — it is published by that link" % (_q, _u))
print(" unlisted: %d page(s) — noindex, robots Disallow, not in the sitemap, linked from no other page" % len(_unl_pages))
if fails:
print("\nSITE GATES FAILED: %d problem(s)" % len(fails))
sys.exit(1)
+114
View File
@@ -0,0 +1,114 @@
#!/usr/bin/env python3
"""Tests for the machine estimator's data and rule (website/assets/sizing-data.json, sizing.js).
No JavaScript runtime exists on DooPlex or in CI (python3 + git only), so the rule is checked here in Python, on
the SAME data file the page reads, and the JavaScript is held to having no numbers of its own:
1. the committed data file is exactly what scripts/build_sizing_web.py builds
2. the four cases of SIZING.md §5 give the expected verdicts by the rule (estimate() below mirrors sizing.js)
3. the verdict table in SIZING.md §5 says the same as the rule computes
4. sizing.js carries the rule's shape and none of its constants (they all come from the data file)
Run: python3 -I scripts/test_build_sizing_web.py
"""
import io, json, os, re, sys, unittest
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, os.path.join(ROOT, "scripts"))
import build_sizing_web as B
DATA = os.path.join(ROOT, "website", "assets", "sizing-data.json")
JS = os.path.join(ROOT, "website", "assets", "sizing.js")
SIZING_MD = os.path.join(ROOT, "documentation", "audits", "sizing-2026-10-13", "SIZING.md")
HOUSEHOLD = ["immich", "jellyfin", "nextcloud", "home-assistant", "vaultwarden", "paperless-ngx"]
WORDS = {"well": "fits well", "tight": "fits, but tight", "no": "does not fit"}
def guest_mib(d, gb): # mirrors sizing.js guestMib()
r = d["guest_rule"]
total = gb * 1024 - r["memtotal_shortfall_mib"]
mem = total - r["reserve_mib"]
if mem > total - r["max_gap_mib"]:
mem = total - r["max_gap_mib"]
if mem < r["min_mib"]:
mem = r["min_mib"]
if mem > total - r["ceiling_gap_mib"]:
mem = total - r["ceiling_gap_mib"]
return mem
def estimate(d, slugs): # mirrors sizing.js estimate()
by = {a["slug"]: a for a in d["apps"]}
peak = d["platform_mb"] + sum(by[s]["peak_mb"] for s in slugs if not by[s].get("builtin"))
out = []
for gb in d["boxes_gb"]:
g = guest_mib(d, gb)
out.append("well" if peak * d["headroom"] <= g else ("tight" if peak <= g else "no"))
return peak, out
class SizingWeb(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.d = json.load(io.open(DATA, encoding="utf-8"))
cls.all = [a["slug"] for a in cls.d["apps"] if not a.get("builtin")]
def test_committed_file_is_a_fresh_build(self):
self.assertEqual(io.open(DATA, encoding="utf-8").read(), B.render(B.build()),
"sizing-data.json differs from a fresh build — run scripts/build_sizing_web.py")
def test_offers_the_62_listed_apps_plus_builtin(self):
self.assertEqual(len(self.all), 62)
self.assertEqual([a["slug"] for a in self.d["apps"] if a.get("builtin")], ["filebrowser"])
def test_guest_sizes_follow_the_installer(self):
self.assertEqual([guest_mib(self.d, gb) for gb in (8, 16, 32)], [4096, 11676, 28060])
self.assertEqual(guest_mib(self.d, 4), 4 * 1024 - 612 - 1024) # a 4 GB host: the ceiling, not 4 GB
def test_the_four_cases(self):
cases = {
"household": (HOUSEHOLD, ["no", "well", "well"]),
"household without immich": ([s for s in HOUSEHOLD if s != "immich"], ["well", "well", "well"]),
"immich alone": (["immich"], ["tight", "well", "well"]),
"all 62": (self.all, ["no", "no", "tight"]),
"nothing": ([], ["well", "well", "well"]),
}
for name, (sel, want) in cases.items():
self.assertEqual(estimate(self.d, sel)[1], want, name)
def test_sizing_md_table_agrees_with_the_rule(self):
with io.open(SIZING_MD, encoding="utf-8") as f:
s = f.read()
block = re.search(r"<!-- sizing-verdicts:.*?-->(.*?)<!-- /sizing-verdicts -->", s, re.S)
self.assertIsNotNone(block, "SIZING.md has no sizing-verdicts table")
rows = {}
for line in block.group(1).splitlines():
cells = [c.strip() for c in line.strip().strip("|").split("|")]
if len(cells) == 5 and cells[0] not in ("Selection", "---"):
rows[cells[0]] = cells
sel = {"Household set": HOUSEHOLD, "Household set without Immich": [s for s in HOUSEHOLD if s != "immich"],
"Immich alone": ["immich"], "All 62 apps": self.all}
self.assertEqual(set(rows), set(sel))
for name, slugs in sel.items():
peak, verdicts = estimate(self.d, slugs)
self.assertEqual(rows[name][1], "%.1f GB" % (peak / 1024), name + " peak")
self.assertEqual(rows[name][2:], [WORDS[v] for v in verdicts], name)
def test_js_has_the_rule_and_none_of_its_numbers(self):
js = io.open(JS, encoding="utf-8").read()
for need in ("data.platform_mb", "data.headroom", "r.reserve_mib", "r.max_gap_mib", "r.min_mib",
"r.ceiling_gap_mib", "r.memtotal_shortfall_mib", "data.boxes_gb",
"peak * data.headroom <= guest", "peak <= guest"):
self.assertIn(need, js)
code = re.sub(r"/\*.*?\*/", "", js, flags=re.S)
code = re.sub(r"(?m)//.*$", "", code)
code = re.sub(r"'(?:[^'\\]|\\.)*'", "''", code)
nums = set(re.findall(r"(?<![\w.])\d+(?:\.\d+)?", code))
self.assertTrue(nums <= {"0", "1", "2", "1024"}, "sizing.js holds its own numbers: %s" % sorted(nums - {"0", "1", "2", "1024"}))
def test_data_has_no_peak_below_typical(self):
for a in self.d["apps"]:
self.assertGreaterEqual(a["peak_mb"], a["typical_mb"], a["slug"])
if __name__ == "__main__":
unittest.main(verbosity=2)