diff --git a/CHANGELOG.md b/CHANGELOG.md index 627d002..2481280 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,23 @@ +## romm gets 768M — 5.3.0 does not fit in 512M (2026-09-22, R-635) + +**No `image:` line moved, so no `catalog_since` moved.** `romm` 512M -> **768M**; the header and +`.felhom.yml` totals follow (1024M -> 1280M). + +**Measured on demo-hp, not reasoned about.** `romm 5.0.0 -> 5.3.0` was promoted that morning after +the edge was proven on the scratch guest, and the guarded Update on demo-hp reached `done` in 74.8 s. +It ran clean for two hours. Then: `OOMKilled: true`, **4,530** `Worker … was sent SIGKILL! Perhaps +out of memory?` in six hours, the container pinned at 457 MiB of its 512 MiB limit, **~500% CPU** in +a permanent restart storm, host load average **5.2 while otherwise idle**. + +**Two things this cost that a bigger number alone does not fix, both recorded in R-635.** The update +read `done` and the app read `running` throughout, because nginx answers `GET /` with 200 while the +gunicorn workers behind it are being killed — the probe is right, the port is right, and the green is +still false. And nothing alarmed: OOM signals are invisible inside an LXC guest (R-528), and a +running-but-thrashing app is not "down". **The operator heard the fans. That was the detector.** + +768M is a first measured step, not a final answer — the soak that follows this entry is what settles +whether it is enough. + ## The probe gate now runs in CI's own degraded mode (2026-09-22, R-618) **Caught by checking the push's CI run by job id, not by assuming it went green.** Job **877** on diff --git a/templates/romm/.felhom.yml b/templates/romm/.felhom.yml index 607e12f..cf4f22f 100644 --- a/templates/romm/.felhom.yml +++ b/templates/romm/.felhom.yml @@ -15,7 +15,7 @@ catalog_since: "2026-09-22" # --- Resource hints (displayed on deploy screen) --- resources: mem_request: "300M" - mem_limit: "1024M" + mem_limit: "1280M" pi_compatible: false needs_hdd: true diff --git a/templates/romm/docker-compose.yml b/templates/romm/docker-compose.yml index 97131cb..f09121f 100644 --- a/templates/romm/docker-compose.yml +++ b/templates/romm/docker-compose.yml @@ -1,7 +1,7 @@ # ROMM - ROM Manager for Game Libraries # Domain: ${SUBDOMAIN}.${DOMAIN} # Database: MariaDB + Redis -# RAM: ~300MB (mem_limit: 1024M total — romm 512M + mariadb 384M + redis 128M) | Pi-compatible: No (MariaDB + heavy) +# RAM: ~300MB (mem_limit: 1280M total — romm 768M + mariadb 384M + redis 128M) | Pi-compatible: No (MariaDB + heavy) # # Environment variables: # DOMAIN - Your domain (e.g., demo-felhom.eu) @@ -72,7 +72,11 @@ services: deploy: resources: limits: - memory: 512M + # 768M, not 512M: romm 5.3.0 does not fit in 512M. Measured on demo-hp 2026-09-22 — + # OOMKilled true, 4,530 gunicorn worker SIGKILLs in six hours, ~500% CPU in a permanent + # restart storm, while the web front end still answered 200 so the update read `done` + # (R-635). 5.0.0 did fit; the version moved and the limit did not. + memory: 768M healthcheck: test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:8080/"] interval: 30s