From 917a779cca6785a41f2d146449d72da015840490 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Fri, 2 Oct 2026 16:54:29 +0200 Subject: [PATCH] Persistence gate: empty declared volume = UNDETERMINED (R-788), the app's own fixture seed as the exercise; re-sweep verdicts (CLEAN 40 / UNDETERMINED 18 / BROKEN 0); papra 256M -> 768M (R-803); records 2.1, EXISTING-APPS-GAPS regenerated Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS --- CHANGELOG.md | 16 +++ CONTEXT.md | 3 + README.md | 2 +- REPORT.md | 24 ++--- REUSE.md | 2 +- .../persistence-sweep-2026-10-02/verdicts.txt | 60 ++++++++++++ onboarding/EXISTING-APPS-GAPS.md | 30 +++--- onboarding/dawarich.md | 2 +- onboarding/grimmory.md | 2 +- onboarding/karakeep.md | 2 +- onboarding/metube.md | 2 +- onboarding/radicale.md | 2 +- onboarding/sparkyfitness.md | 2 +- onboarding/wger.md | 2 +- scripts/check-volume-persistence.py | 98 +++++++++++++------ scripts/onboarding_gaps.py | 11 ++- scripts/test_check_volume_persistence.py | 33 ++++++- templates/papra/.felhom.yml | 4 +- templates/papra/docker-compose.yml | 4 +- 19 files changed, 224 insertions(+), 77 deletions(-) create mode 100644 audits/persistence-sweep-2026-10-02/verdicts.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f15d2e..97517f2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,19 @@ +## The persistence re-sweep (R-801) and its follow-ups; papra's memory limit (R-803) (2026-10-02, afternoon) + +- **`check-volume-persistence.py`:** an EMPTY declared volume after the exercise is UNDETERMINED, never CLEAN (R-788, + red-proofed); when a volume stays empty the gate calls the app's OWN upgrade-fixture seed once (`_exercise_fixture`, + through `upgrade_fixtures` / `upgrade_boxport`, exactly as `upgrade-test.py` picks it) — this replaces the morning's + MeTube-only `APP_EXERCISE`. Three tests changed with the rule (they pinned the old "empty uploads volume beside a full + database is CLEAN"). +- **The re-sweep of all 58 templates** with the fixed gate (bench, Docker Hub logged in): CLEAN 39 · UNDETERMINED 19 · + BROKEN 0 (August: 38 · 11 · 4 of 53, all decided by start-time writes — 0 of 53 probes had sent a request). The + fixture seed was called for 16 apps. Verdicts: `audits/persistence-sweep-2026-10-02/verdicts.txt`; + `onboarding_gaps.py` reads them; `EXISTING-APPS-GAPS.md` regenerated (storage 38 → 35 of 53). +- **papra: memory limit 256M → 768M (`mem_request` 50M → 256M)** — R-803: a fresh install was OOM-killed in its + migration and crash-looped. Measured from birth on the bench: peak anon 405 MiB (cgroup peak 483 MiB), ~255 MiB idle; + at 768M healthy in 31 s, 0 kills, the seed read back, persistence CLEAN. No image move. No box runs papra. +- Records 2.1 updated with the re-sweep (radicale, karakeep, dawarich, grimmory, metube; sparkyfitness and wger open). + ## The family gate in the catalog — Grimmory and MeTube published behind it (2026-10-02) - **New template fields** (controller ≥ 0.287.0, `09` §3 decisions 63/64): `family_gate: true`, `family_gate_except:` diff --git a/CONTEXT.md b/CONTEXT.md index 349e922..d19c235 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -2,6 +2,9 @@ > Created with the REUSE.md rollout (2026-07-03). History: `CHANGELOG.md`; format spec: `README.md`. +- **2026-10-02 (afternoon) — the persistence re-sweep:** the gate now sends requests (R-801), an empty declared volume is + UNDETERMINED (R-788), and it calls the app's own fixture seed. All 58: CLEAN 40 · UNDETERMINED 18 · BROKEN 0. papra 768M + (R-803). Remove keeps the household's userdata — `09` §3 decision 67 (controller v0.288.0 names it). - **2026-10-02 — THE FAMILY GATE in the catalog (controller v0.287.0, golden 0.287.0):** `family_gate:` / `family_gate_except:` / `min_controller:`; gate `family-gate`. Grimmory (exceptions OPDS/Kobo/KOReader/Komga) and MeTube (none) published with complete records. A family app's exceptions must be literal prefixes and each measured as a diff --git a/README.md b/README.md index b2ff7af..47fb741 100644 --- a/README.md +++ b/README.md @@ -345,7 +345,7 @@ block + the matching compose `${VAR}` lines. | OpenGist | None (file) | 30M / 128M | yes | -- | gist.* | | Outline | PostgreSQL + Redis | 200M / 768M | no | -- | kb.* | | Paperless-ngx | PostgreSQL + Redis | 500M / 1152M | yes | `${HDD_PATH}/storage/paperless/` | paperless.* | -| Papra | None (file) | 50M / 256M | yes | -- | papra.* | +| Papra | None (file) | 256M / 768M | yes | -- | papra.* | | Plant-it | None (file) | 50M / 256M | yes | -- | plants.* | | Plex | None (file) | 512M / 2048M | no | `${HDD_PATH}/media/` | plex.* | | PrivateBin | None (file) | 30M / 128M | yes | -- | paste.* | diff --git a/REPORT.md b/REPORT.md index bb94492..a725533 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,18 +1,10 @@ -# REPORT — the family gate in the catalog: Grimmory and MeTube published (2026-10-02) +# REPORT — the persistence re-sweep with the fixed gate (R-801); papra's memory limit (R-803) (2026-10-02, afternoon) -Session report: `felhom.eu/REPORT-family-gate-2026-10-02.md`. Evidence: `felhom.eu/documentation/audits/family-gate-2026-10-02/`. +Session report: `felhom.eu/REPORT-persistence-sweep-2026-10-02.md`. -- **Format:** `family_gate: true`, `family_gate_except:` (literal prefixes, anchored by the box), `min_controller:` - (README §family gate; REUSE row). **Gate `family-gate`** (`check-family-gate.py`): refuses a non-literal exception, an - exception without the gate, a missing/low `min_controller`, and a family app while the newest baked golden < 0.287.0. - Decoys seen red: `B/family-gate-decoys-red.txt`. -- **Grimmory** published (v3.5.0 + mariadb:11.4): setup gate for the first admin + the family gate for good; exceptions - OPDS v1/v2, Kobo, KOReader, Komga API — a stranger measured on each (the app's own 401), look-alikes stay gated. - Record `onboarding/grimmory.md`, all 61 ids. R-775's lock cannot be aimed from outside any more. -- **MeTube** published (2026.09.29): behind the family gate with no exception; the websocket passes the gate for a - member (101) and is refused for a stranger; downloads to `${USERDATA_PATH}/media/metube`; own-use sentence (hu, en). - Ladder 2026.09.28 → .29 (bench + 9202). Record `onboarding/metube.md`, all 61 ids. -- **Gates on the bench:** `catalog_gates.py grimmory` and `metube` exit 0, every gate OK, family-gate against golden 0.287.0. -- **Found and fixed here:** the volume-persistence gate never sent a request to any app (port read from label values; - R-801, `routed_ports()`, red-proofed) — a full re-sweep is owed; `APP_EXERCISE` gives MeTube its own write path - (R-788). `box_walk.py` no longer caches "not gated" from a moment without a router. +- **Gate** (`check-volume-persistence.py`): an empty declared volume after the exercise is UNDETERMINED (R-788, + red-proofed); the app's own upgrade-fixture seed is called when a volume stays empty (one call, never a copy). +- **All 58 templates on the bench:** CLEAN 40 · UNDETERMINED 18 · BROKEN 0 (papra counted CLEAN after its fix). No + data-loss finding. Verdicts: `audits/persistence-sweep-2026-10-02/verdicts.txt`; `EXISTING-APPS-GAPS.md` regenerated. +- **papra:** 256M → 768M (`mem_request` 256M) — a fresh install was OOM-killed in its migration (R-803); measured from birth. +- Open follow-ups: R-805 (empty binds), R-806 (https backends), R-807 (seeds that write only the database), R-804 (plant-it image gone). diff --git a/REUSE.md b/REUSE.md index 6d8850b..e251902 100644 --- a/REUSE.md +++ b/REUSE.md @@ -34,7 +34,7 @@ Templates are config; the few script helpers other scripts must REUSE, never re- | **Catalog gates — THE entry point** | `scripts/catalog_gates.py` | **Run `python3 scripts/catalog_gates.py ` after ANY template change** (mandated in `CLAUDE.md`). Runs all four gates below in order — image-pins, image-resolvable, volume-persistence, engine-major (2026-09-13; git-history diff, hook-only until CI fetches deeper, R-452) — and exits **non-zero if any fails**; **2 (UNDETERMINED) is reported distinctly and is never a pass**, 1 (convicted) outranks 2 in the summary. Naming app(s) scopes the two gates that accept scoping, which is the normal after-a-change run; with no names the RUNTIME gate deploys every template, so that form is **scratch host only**. **Why a runner** (operator ruling 2026-08-02, R-161): the only gates in this project that ever get run are the ones with a single entry point named in a CLAUDE.md — `felhom.eu/scripts/site_gates.py` is run, R-29's three orphans are named nowhere and have stopped nothing. Controller-side enforcement was rejected because a load-time check reads only the file and a static audit reports the catalog clean **including papra** — it would pass on the very defect it exists to catch; CI was rejected for now (neither repo has any, no users yet). Adding a fourth gate here means adding it to `GATES` in this file — nothing else. | | Image pinning | ALL `templates/*/docker-compose.yml` (`image:` line) | **Never `:latest` or untagged** (recovery-unit `ImagePins` pins the tag — `:latest` breaks restore fidelity). Pin a concrete version tag; an app deployed anywhere in the fleet pins to the digest it is RUNNING (pin ≠ upgrade); `@sha256:` digest pins also count. Gate: `python scripts/check-image-pins.py` after any compose change (swept 2026-07-12: 5 pins). TRAP: ghcr `tags/list` can be stale/partial — verify tag existence via `docker manifest inspect`, never the tag list. | | Image RESOLVABILITY (does the pin still exist?) | `scripts/check-image-resolvable.py` + `scripts/test_check_image_resolvable.py` | The complement to the pin gate, which is purely syntactic and cannot see rot. Run it at the START of every catalog campaign and before any publish train that vouches the catalog: `python3 scripts/check-image-resolvable.py [app …]`. Exit **0** all resolve, **1** the registry says an image is GONE, **2** INCONCLUSIVE/harness error. **Two traps it encodes, both live-observed:** (a) `docker manifest inspect` prints `toomanyrequests` and **still exits 0** — never trust the exit code alone (same shape as the ISO `validate-answer` trap); (b) the inverse — the first sweep called 24 of 65 pins dead, `postgres:16-alpine` among them, because Docker Hub throttled it partway. Ambiguity therefore resolves to INCONCLUSIVE, never to an accusation; a gate that cries wolf gets ignored. Unauthenticated Hub lookups WILL throttle on a full 65-pin sweep — `docker login` first, or expect exit 2. | -| Volume PERSISTENCE (does the app write where the template preserves?) | `scripts/check-volume-persistence.py` + `scripts/test_check_volume_persistence.py` | The third gate and the only RUNTIME one — **the two image gates are static and this class is invisible to static analysis**, which was measured, not assumed: a static audit of all 53 composes (every declared volume attached, no anonymous mounts, no stray host binds) reports the catalog clean AND reports papra clean. papra's compose is well-formed; it mounts `papra_data:/app/data` while the app writes `/app/app-data/db/db.sqlite` into the container's **writable layer** and cannot write `/app/data` at all — so `DumpAppVolumes` (`felhom-controller internal/backup/backup.go:543`) tars an empty directory and the backup verifies (R-156, Campaign 10). Run: `python3 scripts/check-volume-persistence.py [app …]` **on a scratch host, never a customer box**. Exit **0** all CLEAN, **1** REFUSED, **2** UNDETERMINED/prober untrustworthy. **Traps it encodes:** (a) `A` vs `C` in `docker diff` — a linuxserver.io entrypoint chowning its app tree produced 1305 `C` entries and called calibre-web BROKEN on the first pass, so DATA is decided from `A` only and a `C` on a DB file is adjudicated by comparing bytes against a pristine container of the same image; (b) no `docker exec` anywhere — Campaign 7 §1.1's OCI-error-to-stdout trap, so uid comes from `/proc//status` and writability from a host-side `stat`; (c) `base64key` secrets need the controller's `base64:` prefix (`deploy.go:904`) or bookstack serves 500s and the harness looks like an app defect; (d) it self-tests in BOTH directions against two canary templates before reporting anything — a detector that flags nothing turns an unexamined catalog into a documented-clean one. UNDETERMINED is **never** folded into CLEAN. | +| Volume PERSISTENCE (does the app write where the template preserves?) | `scripts/check-volume-persistence.py` + `scripts/test_check_volume_persistence.py` | The third gate and the only RUNTIME one — **the two image gates are static and this class is invisible to static analysis**, which was measured, not assumed: a static audit of all 53 composes (every declared volume attached, no anonymous mounts, no stray host binds) reports the catalog clean AND reports papra clean. papra's compose is well-formed; it mounts `papra_data:/app/data` while the app writes `/app/app-data/db/db.sqlite` into the container's **writable layer** and cannot write `/app/data` at all — so `DumpAppVolumes` (`felhom-controller internal/backup/backup.go:543`) tars an empty directory and the backup verifies (R-156, Campaign 10). Run: `python3 scripts/check-volume-persistence.py [app …]` **on a scratch host, never a customer box**. Exit **0** all CLEAN, **1** REFUSED, **2** UNDETERMINED/prober untrustworthy. **Traps it encodes:** (a) `A` vs `C` in `docker diff` — a linuxserver.io entrypoint chowning its app tree produced 1305 `C` entries and called calibre-web BROKEN on the first pass, so DATA is decided from `A` only and a `C` on a DB file is adjudicated by comparing bytes against a pristine container of the same image; (b) no `docker exec` anywhere — Campaign 7 §1.1's OCI-error-to-stdout trap, so uid comes from `/proc//status` and writability from a host-side `stat`; (c) `base64key` secrets need the controller's `base64:` prefix (`deploy.go:904`) or bookstack serves 500s and the harness looks like an app defect; (d) it self-tests in BOTH directions against two canary templates before reporting anything — a detector that flags nothing turns an unexamined catalog into a documented-clean one. UNDETERMINED is **never** folded into CLEAN. **Since 2026-10-02:** the routed port is read from the label NAME (`routed_ports()`, R-801 — before that the gate never sent a request); an EMPTY declared volume after the exercise is UNDETERMINED (R-788); when a volume stays empty the gate calls the app's OWN upgrade-fixture seed (`_exercise_fixture`, one call — never a copy). TRAPS: a seed that writes only to the database leaves upload/media volumes empty → UNDETERMINED (R-807); empty BIND mounts are not judged yet (R-805). | | Docker healthcheck host | ALL `templates/*/docker-compose.yml` (`healthcheck.test:`) | **Always `127.0.0.1`, never `localhost`.** BusyBox `wget` (and node/python/curl one-shots) resolve `localhost`→IPv6 `::1` with NO cross-address-family fallback; an app that binds IPv4-only then reads docker-`unhealthy` while fully serving (vaultwarden, re-run 2026-07-06 — swept all 48 templates). | | Docker healthcheck — BusyBox/wget images | `templates/vaultwarden/docker-compose.yml` (~L49) | `test: ["CMD", "wget", "--spider", "-q", "http://localhost:/"]`. Most common family (~20 apps, e.g. homebox, glance). | | Docker healthcheck — curl-capable images | `templates/paperless-ngx/docker-compose.yml` (~L76) | `test: ["CMD", "curl", "-f", "http://localhost:/"]` (~18 apps: jellyfin, immich, sonarr…). | diff --git a/audits/persistence-sweep-2026-10-02/verdicts.txt b/audits/persistence-sweep-2026-10-02/verdicts.txt new file mode 100644 index 0000000..3b6289a --- /dev/null +++ b/audits/persistence-sweep-2026-10-02/verdicts.txt @@ -0,0 +1,60 @@ +# check-volume-persistence.py --all, 2026-10-02 (R-801 port fix, R-788 empty-volume rule, the app's own fixture seed) — the verdict line per app, copied from felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/batch-0[1-8].txt; full table: A/sweep/TABLE.md +# papra: UNDETERMINED in the sweep (not running: OOM-killed at its old 256M limit, R-803); CLEAN at 768M from birth after the fix — felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/papra/gate-papra-768M.txt +actualbudget CLEAN +adventurelog CLEAN +audiobookshelf CLEAN +bentopdf UNDETERMINED +bookstack CLEAN +calcom CLEAN +calibre-web CLEAN +claper UNDETERMINED +code-server CLEAN +crafty-controller UNDETERMINED +dawarich UNDETERMINED +docmost UNDETERMINED +emby CLEAN +ghost CLEAN +gitea CLEAN +glance CLEAN +gokapi CLEAN +grafana CLEAN +gramps-web UNDETERMINED +grimmory CLEAN +home-assistant CLEAN +homebox CLEAN +homepage CLEAN +immich UNDETERMINED +jellyfin CLEAN +karakeep CLEAN +kimai CLEAN +komga CLEAN +mealie CLEAN +metube CLEAN +n8n CLEAN +navidrome CLEAN +nextcloud CLEAN +onlyoffice CLEAN +opengist CLEAN +outline UNDETERMINED +paperless-ngx CLEAN +papra CLEAN +plant-it UNDETERMINED +plex UNDETERMINED +privatebin CLEAN +radarr CLEAN +radicale CLEAN +rallly CLEAN +recipe-importer UNDETERMINED +romm CLEAN +seerr CLEAN +sonarr CLEAN +sparkyfitness UNDETERMINED +tandoor UNDETERMINED +termix CLEAN +uptime-kuma CLEAN +vaultwarden CLEAN +vikunja UNDETERMINED +wanderer UNDETERMINED +wger UNDETERMINED +wishlist UNDETERMINED +zipline UNDETERMINED diff --git a/onboarding/EXISTING-APPS-GAPS.md b/onboarding/EXISTING-APPS-GAPS.md index e0abaa4..fbe7e04 100644 --- a/onboarding/EXISTING-APPS-GAPS.md +++ b/onboarding/EXISTING-APPS-GAPS.md @@ -1,6 +1,6 @@ # EXISTING APPS — what the catalog already shows, per checklist group -> Generated by `scripts/onboarding_gaps.py` from committed files (catalog `6d72c09`). **Do not edit by hand.** +> Generated by `scripts/onboarding_gaps.py` from committed files (catalog `febf58c`). **Do not edit by hand.** > Read only: nothing was re-tested. A cell is what a FILE says, not a measurement made today. The 53 apps > published before the checklist (2026-10-01) are exempt from the onboarding gate; this page is information, > not work (operator default 2026-10-01, may be reversed). @@ -9,9 +9,9 @@ | group | covered | what "covered" means here (the signal read) | |---|---|---| -| 0 Fit | 52 / 53 | `lifecycle` available, `use_cases` and `pi_compatible` present — licence, telemetry, internet need and phone apps are recorded nowhere | +| 0 Fit | 51 / 53 | `lifecycle` available, `use_cases` and `pi_compatible` present — licence, telemetry, internet need and phone apps are recorded nowhere | | 1 Images, start command, DB | 53 / 53 | pins clean and the engine rules hold (MariaDB auto-upgrade, PG 18 mount) — entrypoint switches, the production server, migrations and secrets read (1.4–1.9) are recorded for NO app | -| 2 Storage and backup | 38 / 53 | the 2026-08-02 persistence sweep read the app CLEAN, and an HDD app carries `backup:` classes — no restore round trip is recorded per app | +| 2 Storage and backup | 36 / 53 | the 2026-10-02 persistence re-sweep (the fixed gate, R-801) read the app CLEAN, and an HDD app carries `backup:` classes — no restore round trip is recorded per app | | 3 Accounts and strangers | 39 / 53 | a FIRST-ADMIN.md row whose source is MEASURED on a box — lock-out (3.6) is recorded only for the R-752 apps | | 4 Health | 49 / 53 | every service has a compose healthcheck, a controller probe exists, the exposed container is named like the stack — no negative control is recorded | | 5 Resources | 24 / 53 | every service limited, `mem_limit` = the sum, and a ladder entry carries a measured memory watch — no first-start-from-birth watch is recorded except immich's | @@ -41,15 +41,15 @@ Mail: 6 app(s) carry `smtp_mapping`. | calibre-web | yes | no DB sidecar | sweep clean, backup classes | class 3, measured + after_install | yes | watched 19% | 1 proven step(s), fixture | — | yes | | claper | yes | engine rules hold | sweep undetermined | class 3 (+ open sign-up), measured + after_install | yes | watched 48%, mem_limit≠sum | 1 proven step(s), fixture | — | yes | | code-server | yes | no DB sidecar | sweep clean | class 1, read only | yes | no watch | 0 proven step(s), fixture | — | yes | -| crafty-controller | yes | no DB sidecar | sweep clean | class 1, read only | yes | watched 3% | 1 proven step(s), fixture | — | yes | +| crafty-controller | yes | no DB sidecar | sweep undetermined | class 1, read only | yes | watched 3% | 1 proven step(s), fixture | — | yes | | docmost | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 80% | 1 proven step(s), fixture | — | yes | | emby | yes | no DB sidecar | sweep clean, backup classes | class 4, measured + setup_gate | yes | watched 5% | 2 proven step(s), no fixture | — | yes | | ghost | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate | yes | watched 27% | 2 proven step(s), no fixture | — | yes | | gitea | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | yes | watched 27% | 1 proven step(s), fixture | smtp mapped | yes | -| glance | yes | no DB sidecar | sweep undetermined | class 5, read only | yes | no watch | 0 proven step(s), no fixture | — | yes | +| glance | yes | no DB sidecar | sweep clean | class 5, read only | yes | no watch | 0 proven step(s), no fixture | — | yes | | gokapi | yes | no DB sidecar | sweep clean | class 1, read only | yes | no watch | 0 proven step(s), no fixture | — | yes | | grafana | yes | no DB sidecar | sweep clean | class 1, read only | yes | watched 47% | 1 proven step(s), fixture | — | yes | -| gramps-web | yes | no DB sidecar | sweep broken | class 4, measured + setup_gate/signup_block/after_setup | yes | no watch | 0 proven step(s), fixture | — | yes | +| gramps-web | yes | no DB sidecar | sweep undetermined | class 4, measured + setup_gate/signup_block/after_setup | yes | no watch | 0 proven step(s), fixture | — | yes | | home-assistant | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate | yes | watched 32% | 1 proven step(s), fixture | — | yes | | homebox | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | yes | no watch | 0 proven step(s), no fixture | — | yes | | homepage | yes | no DB sidecar | sweep clean | class 5, read only | yes | no watch | 0 proven step(s), fixture | — | yes | @@ -63,12 +63,12 @@ Mail: 6 app(s) carry `smtp_mapping`. | nextcloud | yes | engine rules hold | sweep clean, backup classes | class 1, measured | yes | watched 24%, mem_limit≠sum | 2 proven step(s), fixture | smtp mapped | yes | | onlyoffice | yes | no DB sidecar | sweep clean | class 5, read only | yes | no watch | 0 proven step(s), fixture | — | yes | | opengist | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate/signup_block | yes | watched 78% | 1 proven step(s), fixture | — | yes | -| outline | yes | engine rules hold | sweep clean | class 4, measured + setup_gate | yes | watched 34%, mem_limit≠sum | 2 proven step(s), fixture | — | yes | +| outline | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 34%, mem_limit≠sum | 2 proven step(s), fixture | — | yes | | paperless-ngx | yes | engine rules hold | sweep clean, backup classes | class 1, read only | probe container not in compose | watched 40% | 1 proven step(s), fixture | — | yes | -| papra | yes | no DB sidecar | sweep broken | class 4, measured + setup_gate/signup_block/after_setup | yes | no watch | 0 proven step(s), fixture | — | yes | +| papra | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | yes | no watch | 0 proven step(s), fixture | — | yes | | plant-it | — (abandoned) | no DB sidecar | sweep undetermined | class 4, read only + setup_gate | yes | no watch | 0 proven step(s), no fixture | — | yes | -| plex | yes | no DB sidecar | sweep clean, backup classes | class 2, read only | yes | no watch | 0 proven step(s), fixture | — | yes | -| privatebin | yes | no DB sidecar | sweep broken | class 5, read only | yes | no watch | 0 proven step(s), fixture | — | yes | +| plex | yes | no DB sidecar | sweep undetermined, backup classes | class 2, read only | yes | no watch | 0 proven step(s), fixture | — | yes | +| privatebin | yes | no DB sidecar | sweep clean | class 5, read only | yes | no watch | 0 proven step(s), fixture | — | yes | | radarr | yes | no DB sidecar | sweep clean, backup classes | class 4, measured + setup_gate | yes | no watch | 0 proven step(s), fixture | — | yes | | rallly | yes | engine rules hold | sweep clean | class 4, measured + setup_gate | yes | watched 61% | 2 proven step(s), fixture | smtp mapped | yes | | recipe-importer | yes | no DB sidecar | sweep undetermined | class 4, measured + setup_gate | yes | no watch | 0 proven step(s), no fixture | — | not in README | @@ -76,12 +76,12 @@ Mail: 6 app(s) carry `smtp_mapping`. | seerr | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate | yes | no watch | 0 proven step(s), no fixture | — | yes | | sonarr | yes | no DB sidecar | sweep clean, backup classes | class 4, measured + setup_gate | yes | watched 14% | 1 proven step(s), no fixture | — | yes | | sparkyfitness | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate/signup_block/after_setup | yes | watched 31% | 1 proven step(s), fixture | — | yes | -| tandoor | yes | engine rules hold | sweep clean | class 4, measured + setup_gate | yes | watched 79% | 1 proven step(s), fixture | — | yes | +| tandoor | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 79% | 1 proven step(s), fixture | — | yes | | termix | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | yes | no watch | 0 proven step(s), fixture | — | yes | -| uptime-kuma | yes | no DB sidecar | sweep undetermined | class 4, measured + setup_gate | yes | watched 43% | 1 proven step(s), fixture | — | yes | +| uptime-kuma | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate | yes | watched 43% | 1 proven step(s), fixture | — | yes | | vaultwarden | yes | no DB sidecar | sweep clean | class 1, read only | yes | no watch | 0 proven step(s), fixture | smtp mapped | yes | -| vikunja | yes | no DB sidecar | sweep clean | class 4, measured + setup_gate/signup_block/after_setup | hc missing | no watch | 0 proven step(s), fixture | — | yes | +| vikunja | yes | no DB sidecar | sweep undetermined | class 4, measured + setup_gate/signup_block/after_setup | hc missing | no watch | 0 proven step(s), fixture | — | yes | | wanderer | yes | no DB sidecar | sweep undetermined | class 4, measured + signup_block/after_setup | yes | no watch | 0 proven step(s), no fixture | — | yes | -| wger | yes | no DB sidecar | sweep clean | class 3, measured + after_install | yes | watched 50% | 1 proven step(s), fixture | — | yes | -| wishlist | yes | no DB sidecar | sweep broken | class 4, measured + setup_gate/signup_block | yes | watched 36% | 1 proven step(s), fixture | — | yes | +| wger | — (hidden) | no DB sidecar | sweep undetermined | class 3, measured + after_install | yes | watched 50% | 1 proven step(s), fixture | — | yes | +| wishlist | yes | no DB sidecar | sweep undetermined | class 4, measured + setup_gate/signup_block | yes | watched 36% | 1 proven step(s), fixture | — | yes | | zipline | yes | engine rules hold | sweep undetermined | class 4, measured + setup_gate | yes | watched 34%, mem_limit≠sum | 2 proven step(s), fixture | — | yes | diff --git a/onboarding/dawarich.md b/onboarding/dawarich.md index e4d3676..1b3c7e3 100644 --- a/onboarding/dawarich.md +++ b/onboarding/dawarich.md @@ -28,7 +28,7 @@ after the step proofs (env only, no image); a mail-OFF boot was measured on the 1.7 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/S/S1-dawarich-reads.txt — the entrypoints (`web-entrypoint.sh`, `sidekiq-entrypoint.sh`) migrate and seed at every start; nothing to switch 1.8 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/box/dawarich/checks.txt — an unknown page is a plain 404 1.9 | done | app-catalog-felhom.eu/templates/dawarich/.felhom.yml — SECRET_KEY_BASE is `data_key: true`: Dawarich encrypts columns with keys derived from it; the restore recovered it (data_keys=1, box/dawarich/restore.txt) -2.1 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/G/cvp-dawarich-tail.txt — volume-persistence gate CLEAN +2.1 | done | felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/TABLE.md ; felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/evidence/dawarich/probe.json — re-sweep 2026-10-02 with the fixed gate: UNDETERMINED for ONE volume only, `dawarich-redis /data` (Redis writes its snapshot on its own schedule; a job queue, not the household's data — R-807); the app's own data (PostGIS) landed in its volume; nothing written outside a preserved folder (0 BROKEN). The 2026-10-01 CLEAN came from start-time writes alone 2.2 | done | app-catalog-felhom.eu/templates/dawarich/docker-compose.yml — named volumes (NVMe): database, public, storage, watched imports, redis 2.3 | n/a | needs_hdd false: no drive path to classify; the tier-1 unit holds the database dump and the volumes 2.4 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/box/dawarich/install.txt — the app migrated, seeded and stored the point on first start diff --git a/onboarding/grimmory.md b/onboarding/grimmory.md index 507f5fb..6030c75 100644 --- a/onboarding/grimmory.md +++ b/onboarding/grimmory.md @@ -32,7 +32,7 @@ registry reads at v3.5.0). The spike that measured the gate first: felhom.eu/doc 1.7 | done | felhom.eu/documentation/audits/family-gate-2026-10-02/B/grimmory-reads/upstream-v3.5.0.txt — the entrypoint makes the user (USER_ID/GROUP_ID 1000), chowns /app/data /books /bookdrop, drops to it with su-exec; JVM flags fixed in JAVA_TOOL_OPTIONS; migrations by Flyway inside the app; no switch left to decide 1.8 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/box/grimmory/checks.txt ; felhom.eu/documentation/audits/family-gate-2026-10-02/B/grimmory-reads/upstream-v3.5.0.txt — an unknown page answers the app's page (200), no stack trace; log level INFO; API docs off 1.9 | n/a | the token-signing key is generated by Grimmory inside its own database and travels with the database backup; no template secret encrypts data -2.1 | done | felhom.eu/documentation/audits/family-gate-2026-10-02/B/catalog-gates-grimmory.txt ; felhom.eu/documentation/audits/family-gate-2026-10-02/B/volume-persistence-after-R801.txt — volume-persistence CLEAN, also after the R-801 port fix (the gate now sends its requests) +2.1 | done | felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/TABLE.md ; felhom.eu/documentation/audits/family-gate-2026-10-02/B/volume-persistence-after-R801.txt — CLEAN with the fixed gate, twice (the state is in MariaDB; its empty /app/data BIND is not judged by the gate yet — R-805) 2.2 | done | app-catalog-felhom.eu/templates/grimmory/docker-compose.yml — the database in a named volume (NVMe); covers/app files in ${HDD_PATH}/appdata/grimmory/data; the books in ${USERDATA_PATH}/media/grimmory (the household browses them); the import inbox in ${IMPORT_PATH}/grimmory 2.3 | done | app-catalog-felhom.eu/templates/grimmory/.felhom.yml — books mandatory (the DB points at them), appdata mandatory, the import inbox excluded; tier 1 holds the database volume 2.4 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/S/S2-grimmory-bench-probe.txt ; felhom.eu/documentation/audits/family-gate-2026-10-02/B/box/grim-seed.txt — the folders are 1000:1000 from the first start; the uploaded book is written on the household's drive diff --git a/onboarding/karakeep.md b/onboarding/karakeep.md index 6d06feb..c7fb9ce 100644 --- a/onboarding/karakeep.md +++ b/onboarding/karakeep.md @@ -29,7 +29,7 @@ measurements (5.1, 5.2, 5.4); a limit that only rises cannot make a proven step 1.7 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/S/S1-karakeep-reads.txt — the entrypoint is s6 (`/init`) with fixed services; migrations run as its own `init-db-migration` service at every start; no switch to decide 1.8 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/box/karakeep/checks.txt — an unknown page redirects (308), no debug output 1.9 | n/a | NEXTAUTH_SECRET signs sessions only (sign in again), MEILI_MASTER_KEY guards a rebuildable index; neither encrypts stored data -2.1 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/G/cvp-karakeep-tail.txt — volume-persistence gate CLEAN +2.1 | done | felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/TABLE.md — CLEAN in the 2026-10-02 re-sweep with the fixed gate (R-801); the 2026-10-01 CLEAN (G/cvp-karakeep-tail.txt) came from start-time writes alone 2.2 | done | app-catalog-felhom.eu/templates/karakeep/docker-compose.yml — two named volumes (NVMe): the data (SQLite + assets) and the search index 2.3 | n/a | needs_hdd false: no drive path to classify; the tier-1 unit holds both volumes 2.4 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/box/karakeep/checks.txt — the app wrote its database and crawled assets on first start diff --git a/onboarding/metube.md b/onboarding/metube.md index 83fcdfe..6274a47 100644 --- a/onboarding/metube.md +++ b/onboarding/metube.md @@ -31,7 +31,7 @@ B/metube-reads/ (the entrypoint and every setting, read at the tag). 1.7 | done | felhom.eu/documentation/audits/family-gate-2026-10-02/B/metube-reads/reads-2026.09.29.txt — the entrypoint chowns the download and state folders to PUID:PGID, upgrades yt-dlp ONLY when YTDL_NIGHTLY_UPDATE_TIME is set (off), starts bgutil-pot and runs the app supervised; nothing else to decide 1.8 | done | felhom.eu/documentation/audits/family-gate-2026-10-02/B/box/metube-fresh.txt — an unknown page answers 404 with no trace; LOGLEVEL INFO by default 1.9 | n/a | MeTube has no secret at all, so nothing can lock the household out -2.1 | done | felhom.eu/documentation/audits/family-gate-2026-10-02/B/volume-persistence-after-R801.txt ; felhom.eu/documentation/audits/family-gate-2026-10-02/B/catalog-gates-metube.txt ; felhom.eu/documentation/audits/family-gate-2026-10-02/B/box/life.txt — CLEAN once the gate really exercised MeTube (a POST /add, APP_EXERCISE; before the R-801 port fix the gate sent no request at all and answered UNDETERMINED, felhom.eu/documentation/audits/family-gate-2026-10-02/B/catalog-gates-metube-run2.txt); also the history in /state read back after a recreate and after remove + restore +2.1 | done | felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/TABLE.md ; felhom.eu/documentation/audits/family-gate-2026-10-02/B/volume-persistence-after-R801.txt — CLEAN in the re-sweep: the gate called MeTube's own seed (a download) and both its folders received data 2.2 | done | app-catalog-felhom.eu/templates/metube/docker-compose.yml — the downloads in ${USERDATA_PATH}/media/metube (the household sees them in the file browser and the media player); the queue and history in the named volume metube_state (NVMe) 2.3 | done | app-catalog-felhom.eu/templates/metube/.felhom.yml — the downloads are class optional (they can be downloaded again; large); tier 1 holds metube_state 2.4 | done | felhom.eu/documentation/audits/family-gate-2026-10-02/B/box/metube-fresh.txt — the downloaded file is 1000:1000 0644 on the household's drive (PUID/PGID 1000) diff --git a/onboarding/radicale.md b/onboarding/radicale.md index c5496cc..cae3080 100644 --- a/onboarding/radicale.md +++ b/onboarding/radicale.md @@ -29,7 +29,7 @@ household out (2.5) — the template was changed, and every box and bench row be 1.7 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/S/S1-radicale-reads.txt — the image's entrypoint is `radicale --hosts …` with no switches; the template's own start command is the whole start-up 1.8 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/box/radicale/checks.txt — an unknown page answers 401 with no debug output 1.9 | done | app-catalog-felhom.eu/templates/radicale/docker-compose.yml — the password is NOT a data_key: the login file is written once and lives on the data volume, so a restore brings the old login back with the calendars (measured, 2.5); marking it would make a restore after a remove refuse -2.1 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/box/radicale/restore.txt — the seed came back from the volume backup alone after the app and its volume were removed (the persistence question answered by a restore) +2.1 | done | felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/TABLE.md ; felhom.eu/documentation/audits/new-apps-2026-10-01/box/radicale/restore.txt — CLEAN in the 2026-10-02 re-sweep with the fixed gate (R-801: it now really exercises the app); the seed came back from the volume backup alone after the app and its image were gone 2.2 | done | app-catalog-felhom.eu/templates/radicale/docker-compose.yml — one named volume (NVMe): `collections/` + the login file 2.3 | n/a | needs_hdd false: no drive path to classify; the tier-1 unit holds the one volume 2.4 | done | felhom.eu/documentation/audits/new-apps-2026-10-01/box/radicale/checks.txt — runs as uid 1000 `radicale`; the volume and the 0600 login file are its own diff --git a/onboarding/sparkyfitness.md b/onboarding/sparkyfitness.md index 9fb81d3..39f57da 100644 --- a/onboarding/sparkyfitness.md +++ b/onboarding/sparkyfitness.md @@ -29,7 +29,7 @@ catalog): felhom.eu/documentation/audits/visitors-2026-10-01/C/box/sparky-box.tx 1.7 | open | the image's entrypoint was not read 1.8 | done | felhom.eu/documentation/audits/visitors-2026-10-01/C/box/sparky-box.txt — an unknown page answers the app's own page (200, the SPA), an unknown API 401; no stack trace 1.9 | done | app-catalog-felhom.eu/templates/sparkyfitness/.felhom.yml — API_ENCRYPTION_KEY and BETTER_AUTH_SECRET carry data_key with the reason -2.1 | done | felhom.eu/documentation/audits/visitors-2026-10-01/C/bench/C2-volume-persistence-sparkyfitness.txt — volume-persistence gate CLEAN (it notes a benign write under /app/SparkyFitnessServer outside the mounts — not read further) +2.1 | open | re-sweep 2026-10-02 with the fixed gate (felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/TABLE.md): UNDETERMINED — the server's backup and uploads volumes stay empty after the seed (it writes only to the database); nothing written outside a preserved folder (R-807). The 2026-10-01 CLEAN came from start-time writes alone 2.2 | done | app-catalog-felhom.eu/templates/sparkyfitness/docker-compose.yml — three named volumes (NVMe): database, uploads, the app's own backup folder 2.3 | n/a | needs_hdd false: no drive path to classify; the tier-1 unit holds the three volumes 2.4 | done | felhom.eu/documentation/audits/visitors-2026-10-01/C/box/sparky-box.txt — the first start wrote its database and the seeded check-in through the app diff --git a/onboarding/wger.md b/onboarding/wger.md index 4848bc7..6dd8e8c 100644 --- a/onboarding/wger.md +++ b/onboarding/wger.md @@ -31,7 +31,7 @@ Measured on 9202 2026-10-01 from the drill catalog (e9f50b5, wger template ident 1.7 | open | two entrypoint switches are still undecided: `WGER_USE_GUNICORN` (R-755) and `DJANGO_DEBUG` — unset, so `collectstatic` never runs (R-762; C2, C8) 1.8 | done | felhom.eu/documentation/audits/new-app-checklist-2026-10-01/C/C2-static-reads.txt ; felhom.eu/documentation/audits/new-app-checklist-2026-10-01/C/C3-logins.txt — DEBUG False; an unknown page is a plain 404, no debug page 1.9 | n/a | SECRET_KEY signs sessions and reset links only; the JWT pair lives on the data volume and is backed up with it -2.1 | done | app-catalog-felhom.eu/audits/persistence-sweep-2026-08-02/state/gate.log — wger CLEAN (the two volumes are unchanged since) +2.1 | open | re-sweep 2026-10-02 with the fixed gate (felhom.eu/documentation/audits/persistence-sweep-2026-10-02/A/sweep/TABLE.md): UNDETERMINED — /home/wger/media stays empty after the seed (no upload in the seed); nothing written outside a preserved folder (R-807). The August CLEAN came from start-time writes alone 2.2 | done | app-catalog-felhom.eu/templates/wger/docker-compose.yml — two named volumes (NVMe), no drive path 2.3 | n/a | needs_hdd false: no drive path to classify; the tier-1 unit holds both named volumes 2.4 | done | felhom.eu/documentation/audits/new-app-checklist-2026-10-01/C/C8-signup-guest-media-static.txt — the app wrote its DB and a photo as user `wger` diff --git a/scripts/check-volume-persistence.py b/scripts/check-volume-persistence.py index 227339a..9b3bd69 100644 --- a/scripts/check-volume-persistence.py +++ b/scripts/check-volume-persistence.py @@ -42,6 +42,7 @@ honest. Every observation here comes from `docker inspect`, `docker diff`, `/pro filesystem, so a distroless or shell-less image is observed exactly like any other. """ import hashlib +import importlib.util import json import os import re @@ -276,7 +277,7 @@ def classify(probe: dict): CLEAN — something was written, all of it inside a declared named volume or a bind, nothing data-classified in any writable layer, every mount writable by its app uid. """ - broken, undet, notes, structural = [], [], [], [] + broken, undet, notes, structural, empty_vols = [], [], [], [], [] if probe.get("error"): return UNDETERMINED, [probe["error"]] containers = probe.get("containers") or [] @@ -339,7 +340,12 @@ def classify(probe: dict): broken.append(f"{nm}: mount {m['target']} is NOT writable by the app's own " f"uid={c.get('uid')} gid={c.get('gid')}") if m["class"] == "named-declared" and m.get("files", 0) == 0: - notes.append(f"{nm}: declared volume {m['target']} is EMPTY") + # R-788 (2026-10-02): an EMPTY declared volume after the exercise is a question not answered, never a + # pass — the app was not shown to write where the template preserves. MeTube's /state read CLEAN this + # way while its download path had never been exercised. Pinned by + # test_empty_declared_volume_is_undetermined_even_when_another_mount_has_data. + empty_vols.append(f"{nm}: declared volume {m['target']} is EMPTY after the exercise — the app was " + f"not shown to write where the template preserves it") for d in c.get("diff_data_dirs") or []: wrote_anything = True broken.append(f"{nm}: DATA in the writable layer at {d['dir']} " @@ -364,11 +370,13 @@ def classify(probe: dict): if broken: return BROKEN, broken + structural + notes if undet or structural: - return UNDETERMINED, undet + structural + notes + return UNDETERMINED, undet + structural + empty_vols + notes if not wrote_anything: return UNDETERMINED, ["nothing was written to any mount and nothing data-classified in any " "writable layer — the app produced no data to locate. Health is not " - "data: this is UNDETERMINED, not CLEAN"] + notes + "data: this is UNDETERMINED, not CLEAN"] + empty_vols + notes + if empty_vols: + return UNDETERMINED, empty_vols + notes return CLEAN, notes @@ -601,31 +609,49 @@ def routed_ports(resolved): return sorted(out) -# APP_EXERCISE — the app's OWN write path, for an app whose GET pages write nothing (R-788, 2026-10-02): MeTube writes -# only when it downloads, so a GET-only exercise leaves both of its mounts empty and the honest verdict is UNDETERMINED. -# Each entry is (method, path, json body); it is sent to every running container's routed port, like `_exercise`, and -# the gate then observes where the data landed as for any app. A request the app refuses writes nothing and the verdict -# stays UNDETERMINED — the exercise cannot turn a non-answer into a pass. -APP_EXERCISE = { - "metube": [("POST", "/add", {"url": "https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4", - "quality": "best", "format": "any", "download_type": "video", "auto_start": True})], -} +def _container_ip(name): + info = _inspect(name) or {} + for net in ((info.get("NetworkSettings") or {}).get("Networks") or {}).values(): + if net.get("IPAddress"): + return net["IPAddress"] + return None -def _exercise_app(app, cids, ports): - hits = [] - for method, path, body in APP_EXERCISE.get(app, []): - for cid in cids: - info = _inspect(cid) or {} - for net in ((info.get("NetworkSettings") or {}).get("Networks") or {}).values(): - ip = net.get("IPAddress") - for port in (ports if ip else []): - code = _sh(["curl", "-sS", "-o", "/dev/null", "-w", "%{http_code}", "--max-time", "30", "-X", method, - "-H", "Content-Type: application/json", "--data", json.dumps(body), - f"http://{ip}:{port}{path}"], timeout=60).stdout.strip() - if code and code != "000": - hits.append(f"{method} {ip}:{port}{path} -> {code} (the app's own write path)") - return hits +def _fixture_for(app, compose_text, env): + """The app's OWN front-door seed — the upgrade fixture (native bench set first, then the box walk's set through + upgrade_boxport, exactly as upgrade-test.py picks it). One call, never a copy of the fixture (R-801 brief, 2026-10-02). + None when the app has none, or the fixture modules are not beside this file (CI: no runtime gate runs there).""" + here = Path(__file__).resolve().parent + try: + sys.path.insert(0, str(here)) + spec = importlib.util.spec_from_file_location("upgrade_fixtures", str(here / "upgrade_fixtures.py")) + fx = importlib.util.module_from_spec(spec) + spec.loader.exec_module(fx) + f = fx.FIXTURES.get(app) + if f is not None: + return f + import upgrade_boxport + return upgrade_boxport.get(app, compose_text, env) + except Exception as e: # a broken fixture module must not crash the gate; the verdict stays honest + return None + + +def _exercise_fixture(app, compose_text, env): + """Seed the app through its own front door; return a line for the evidence, or None when there is no fixture. + A seed that fails writes nothing, so the verdict stays UNDETERMINED — the exercise cannot turn a non-answer into a + pass.""" + f = _fixture_for(app, compose_text, env) + if f is None: + return None + said = [] + try: + out = f.seed(_container_ip, lambda *a: said.append(" ".join(map(str, a)))) + except Exception as e: + return f"fixture {type(getattr(f, 'box', f)).__name__}: seed raised {type(e).__name__}: {str(e)[:160]}" + tried = getattr(f, "tried", None) + return (f"fixture {type(getattr(f, 'box', f)).__name__}: seed " + + ("OK" if out is not None else f"returned nothing (tried: {tried})") + + (f" — {said[-1][:160]}" if said else "")) def docker_prober(app: str, app_dir: Path, settle: int = 45, wait: int = 300) -> dict: @@ -683,8 +709,6 @@ def docker_prober(app: str, app_dir: Path, settle: int = 45, wait: int = 300) -> running = [c for c in cids if ((_inspect(c) or {}).get("State") or {}).get("Status") == "running"] hits = _exercise(running, ports) if (running and ports) else [] - if running and ports and app in APP_EXERCISE: - hits += _exercise_app(app, running, ports) time.sleep(settle) def observe(): @@ -724,6 +748,10 @@ def docker_prober(app: str, app_dir: Path, settle: int = 45, wait: int = 300) -> "diff_other_dirs": other}) return out + def empty_declared(cs): + return any(m["class"] == "named-declared" and m["files"] == 0 and not is_noise_dir(m.get("target")) + for c in cs if c.get("status") == "running" for m in c["mounts"]) + def nothing_written(cs): return not any(c["diff_data_dirs"] or c["diff_token_dirs"] or any(m["class"] != "tmpfs" and m["files"] > 0 for m in c["mounts"]) @@ -736,8 +764,18 @@ def docker_prober(app: str, app_dir: Path, settle: int = 45, wait: int = 300) -> hits += _exercise(running, ports, deep=True) time.sleep(90) containers = observe() + # Third: the app's own seed (its upgrade fixture), when it still wrote nothing or a declared volume is still + # empty — an app that writes only on a real action (R-788/R-801). Recorded so the table can say how many + # apps needed it. + fixture_used = None + if running and (nothing_written(containers) or empty_declared(containers)): + fixture_used = _exercise_fixture(app, compose_text, env) + if fixture_used is not None: + hits.append(fixture_used) + time.sleep(settle) + containers = observe() - return {"app": app, "ports": ports, "exercise": hits, + return {"app": app, "ports": ports, "exercise": hits, "fixture": fixture_used, "declared_volumes": sorted(declared), "containers": containers, "env_keys": sorted(env)} finally: diff --git a/scripts/onboarding_gaps.py b/scripts/onboarding_gaps.py index ac59138..1d43a6e 100644 --- a/scripts/onboarding_gaps.py +++ b/scripts/onboarding_gaps.py @@ -25,6 +25,9 @@ import yaml ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) OUT = os.path.join(ROOT, "onboarding", "EXISTING-APPS-GAPS.md") SWEEP = os.path.join(ROOT, "audits", "persistence-sweep-2026-08-02", "state", "gate.log") +# 2026-10-02: the re-sweep with the fixed gate (R-801 — the August run never sent a request; R-788). One `app VERDICT` line +# per app; when present it is the source, the August log only a fallback. +SWEEP2 = os.path.join(ROOT, "audits", "persistence-sweep-2026-10-02", "verdicts.txt") sys.path.insert(0, os.path.join(ROOT, "scripts")) @@ -53,6 +56,12 @@ def sweep_verdicts(): """The FIRST run's lists in the 2026-08-02 sweep (53 in scope). BROKEN and UNDETERMINED are named; the rest of the 53 were CLEAN.""" out = {} + if os.path.isfile(SWEEP2): + for line in io.open(SWEEP2, encoding="utf-8"): + m = re.match(r"^([a-z0-9-]+) (CLEAN|BROKEN|UNDETERMINED|INCONCLUSIVE)$", line.strip()) + if m and m.group(2) != "CLEAN": + out[m.group(1)] = m.group(2).lower() + return out if not os.path.isfile(SWEEP): return out text = io.open(SWEEP, encoding="utf-8").read() @@ -199,7 +208,7 @@ def main(argv): defs = [ ("0 Fit", "`lifecycle` available, `use_cases` and `pi_compatible` present — licence, telemetry, internet need and phone apps are recorded nowhere"), ("1 Images, start command, DB", "pins clean and the engine rules hold (MariaDB auto-upgrade, PG 18 mount) — entrypoint switches, the production server, migrations and secrets read (1.4–1.9) are recorded for NO app"), - ("2 Storage and backup", "the 2026-08-02 persistence sweep read the app CLEAN, and an HDD app carries `backup:` classes — no restore round trip is recorded per app"), + ("2 Storage and backup", "the 2026-10-02 persistence re-sweep (the fixed gate, R-801) read the app CLEAN, and an HDD app carries `backup:` classes — no restore round trip is recorded per app"), ("3 Accounts and strangers", "a FIRST-ADMIN.md row whose source is MEASURED on a box — lock-out (3.6) is recorded only for the R-752 apps"), ("4 Health", "every service has a compose healthcheck, a controller probe exists, the exposed container is named like the stack — no negative control is recorded"), ("5 Resources", "every service limited, `mem_limit` = the sum, and a ladder entry carries a measured memory watch — no first-start-from-birth watch is recorded except immich's"), diff --git a/scripts/test_check_volume_persistence.py b/scripts/test_check_volume_persistence.py index ccd5eb9..e3ebc58 100644 --- a/scripts/test_check_volume_persistence.py +++ b/scripts/test_check_volume_persistence.py @@ -159,7 +159,12 @@ class TestClassify(unittest.TestCase): c = _ctr("crafty", 0, 0, mounts=[_mount("/crafty/app/config", files=16), _mount("/crafty/backups", files=0)]) c["diff_other_dirs"] = [{"dir": "/crafty/app/classes", "added": ["x"]}] - self.assertEqual(cvp.classify({"app": "crafty", "containers": [c]})[0], cvp.CLEAN) + status, why = cvp.classify({"app": "crafty", "containers": [c]}) + # R-788 (2026-10-02): the empty /crafty/backups is no longer passed over — UNDETERMINED names it; the + # structural check still stays silent (that is what this test is about). + self.assertEqual(status, cvp.UNDETERMINED) + self.assertNotIn("NOTHING this app wrote landed", " ".join(why)) + self.assertIn("/crafty/backups is EMPTY", " ".join(why)) def test_a_SIBLING_container_holding_the_state_silences_it(self): """The measured docmost / immich / claper shape, and the reason the question is asked per @@ -171,7 +176,10 @@ class TestClassify(unittest.TestCase): db = _ctr("docmost-postgres", 0, 0, mounts=[_mount("/var/lib/postgresql/data", files=1540)]) status, why = cvp.classify({"app": "docmost", "containers": [app, db]}) - self.assertEqual(status, cvp.CLEAN) + # R-788: the uploads volume was never shown to receive an upload — UNDETERMINED, named; the + # STRUCTURAL check (this test's subject) must still stay silent. + self.assertEqual(status, cvp.UNDETERMINED) + self.assertIn("/app/data/storage is EMPTY", " ".join(why)) self.assertNotIn("NOTHING this app wrote landed", " ".join(why)) self.assertIn("benign when a sibling container holds the state", " ".join(why), "the per-container observation must still be reported, not dropped") @@ -479,5 +487,26 @@ class TestRoutedPorts(unittest.TestCase): def test_no_routed_port(self): self.assertEqual(cvp.routed_ports({"services": {"db": {"labels": {"x": "y"}}}}), []) + +class TestEmptyDeclaredVolume(unittest.TestCase): + def test_empty_declared_volume_is_undetermined_even_when_another_mount_has_data(self): + """R-788: one mount holds data, the DECLARED volume is empty — the old rule called this CLEAN.""" + probe = {"containers": [{"name": "app", "status": "running", "health": "healthy", "uid": 1000, "gid": 1000, + "mounts": [{"class": "bind", "target": "/downloads", "files": 1, "writable_by_app": "yes"}, + {"class": "named-declared", "target": "/state", "files": 0, "writable_by_app": "yes"}], + "diff_data_dirs": [], "diff_token_dirs": [], "diff_other_dirs": [], "diff_benign_db_touches": [], + "diff_unresolved": []}]} + st, why = cvp.classify(probe) + self.assertEqual(st, cvp.UNDETERMINED, why) + self.assertTrue(any("/state" in w and "EMPTY" in w for w in why), why) + + def test_both_written_is_clean(self): + probe = {"containers": [{"name": "app", "status": "running", "health": "healthy", "uid": 1000, "gid": 1000, + "mounts": [{"class": "bind", "target": "/downloads", "files": 1, "writable_by_app": "yes"}, + {"class": "named-declared", "target": "/state", "files": 2, "writable_by_app": "yes"}], + "diff_data_dirs": [], "diff_token_dirs": [], "diff_other_dirs": [], "diff_benign_db_touches": [], + "diff_unresolved": []}]} + self.assertEqual(cvp.classify(probe)[0], cvp.CLEAN) + if __name__ == "__main__": unittest.main(verbosity=2) diff --git a/templates/papra/.felhom.yml b/templates/papra/.felhom.yml index 15c568e..4764a90 100644 --- a/templates/papra/.felhom.yml +++ b/templates/papra/.felhom.yml @@ -14,8 +14,8 @@ catalog_since: "2026-09-22" # --- Resource hints (displayed on deploy screen) --- resources: - mem_request: "50M" - mem_limit: "256M" + mem_request: "256M" + mem_limit: "768M" pi_compatible: true needs_hdd: false diff --git a/templates/papra/docker-compose.yml b/templates/papra/docker-compose.yml index 44bd7c6..d79cacd 100644 --- a/templates/papra/docker-compose.yml +++ b/templates/papra/docker-compose.yml @@ -1,7 +1,7 @@ # Papra - Minimalista dokumentumtár és rendszerező # Domain: ${SUBDOMAIN}.${DOMAIN} # Database: None (file-based) -# RAM: ~50M (mem_limit: 256M) | Pi-compatible: Yes +# RAM: ~255M idle, ~405M anon at first-start migration (mem_limit: 768M, R-803) | Pi-compatible: Yes # # Environment variables: # DOMAIN - Your domain (e.g., demo-felhom.eu) @@ -38,7 +38,7 @@ services: deploy: resources: limits: - memory: 256M + memory: 768M healthcheck: # A papra képfájlban nincs wget és nincs curl — csak node. A korábbi wget # próba ezért soha nem futott le, a konténer véglegesen unhealthy maradt,