No image: line moved, so no catalog_since moved. Measured on demo-hp after this morning's promotion: OOMKilled true, 4530 gunicorn worker SIGKILLs in six hours, ~500% CPU in a permanent restart storm, host load 5.2 while otherwise idle. It ran clean for two hours first, which is why the walk on the scratch guest did not catch it. The update reported `done` and the app read `running` the whole time - nginx answers 200 while the workers behind it die. Nothing alarmed; the operator heard the fans. 768M is a measured first step, not a final answer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -1,3 +1,23 @@
|
||||
## romm gets 768M — 5.3.0 does not fit in 512M (2026-09-22, R-635)
|
||||
|
||||
**No `image:` line moved, so no `catalog_since` moved.** `romm` 512M -> **768M**; the header and
|
||||
`.felhom.yml` totals follow (1024M -> 1280M).
|
||||
|
||||
**Measured on demo-hp, not reasoned about.** `romm 5.0.0 -> 5.3.0` was promoted that morning after
|
||||
the edge was proven on the scratch guest, and the guarded Update on demo-hp reached `done` in 74.8 s.
|
||||
It ran clean for two hours. Then: `OOMKilled: true`, **4,530** `Worker … was sent SIGKILL! Perhaps
|
||||
out of memory?` in six hours, the container pinned at 457 MiB of its 512 MiB limit, **~500% CPU** in
|
||||
a permanent restart storm, host load average **5.2 while otherwise idle**.
|
||||
|
||||
**Two things this cost that a bigger number alone does not fix, both recorded in R-635.** The update
|
||||
read `done` and the app read `running` throughout, because nginx answers `GET /` with 200 while the
|
||||
gunicorn workers behind it are being killed — the probe is right, the port is right, and the green is
|
||||
still false. And nothing alarmed: OOM signals are invisible inside an LXC guest (R-528), and a
|
||||
running-but-thrashing app is not "down". **The operator heard the fans. That was the detector.**
|
||||
|
||||
768M is a first measured step, not a final answer — the soak that follows this entry is what settles
|
||||
whether it is enough.
|
||||
|
||||
## The probe gate now runs in CI's own degraded mode (2026-09-22, R-618)
|
||||
|
||||
**Caught by checking the push's CI run by job id, not by assuming it went green.** Job **877** on
|
||||
|
||||
@@ -15,7 +15,7 @@ catalog_since: "2026-09-22"
|
||||
# --- Resource hints (displayed on deploy screen) ---
|
||||
resources:
|
||||
mem_request: "300M"
|
||||
mem_limit: "1024M"
|
||||
mem_limit: "1280M"
|
||||
pi_compatible: false
|
||||
needs_hdd: true
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ROMM - ROM Manager for Game Libraries
|
||||
# Domain: ${SUBDOMAIN}.${DOMAIN}
|
||||
# Database: MariaDB + Redis
|
||||
# RAM: ~300MB (mem_limit: 1024M total — romm 512M + mariadb 384M + redis 128M) | Pi-compatible: No (MariaDB + heavy)
|
||||
# RAM: ~300MB (mem_limit: 1280M total — romm 768M + mariadb 384M + redis 128M) | Pi-compatible: No (MariaDB + heavy)
|
||||
#
|
||||
# Environment variables:
|
||||
# DOMAIN - Your domain (e.g., demo-felhom.eu)
|
||||
@@ -72,7 +72,11 @@ services:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 512M
|
||||
# 768M, not 512M: romm 5.3.0 does not fit in 512M. Measured on demo-hp 2026-09-22 —
|
||||
# OOMKilled true, 4,530 gunicorn worker SIGKILLs in six hours, ~500% CPU in a permanent
|
||||
# restart storm, while the web front end still answered 200 so the update read `done`
|
||||
# (R-635). 5.0.0 did fit; the version moved and the limit did not.
|
||||
memory: 768M
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:8080/"]
|
||||
interval: 30s
|
||||
|
||||
Reference in New Issue
Block a user