felhom.eu/documentation/audits/catalog-sweep-2026-10-24

Catalog sweep 2026-10-24 — every catalog app on a fresh box, the way a family uses it

Unattended validation sweep (Saturday 12:30Z → Sunday ~00:00Z). Brief: the operator's „every catalog app, one by one, on a fresh box" prompt. Venue: demo-hp (Tier 0). Method: endpoint-level — every step is the request the dashboard or the app's own page sends (no browser click-through except the screenshot harness; no docker exec to make state).

Safest readings taken (the brief allows no question)

1. ISO 1.30.0, not 1.36.0. The brief said „1.36.0 or the current one"; the current one on felhom.eu/letoltes was 1.30.0 (sha256 checked against the published .sha256). See _box/README.md.
2. 12 GiB for the VM, not 16. demo-hp had 14 GiB free beside 9201, 9202 and Tester 1; 12 is the brief's minimum. The guest itself is 7.7 GiB (the golden's size); the box's own memory admission decided what could run.
3. „Remove keeping data → install again with use my kept data" is a drive-folder question. By design (07 §6.5) the question is asked only for an app that keeps a folder on a drive (HDD_PATH). For the 40+ apps that keep their data in Docker volumes, „keep my data" keeps the recovery unit; the reinstall starts empty, and the item comes back by the unit restore („Visszaállítás"). So step 5 + 6 ran as: keep-remove → reinstall → read (empty, expected) → unit restore → read. That is a unit restore on every app with a seed, not only every fifth.
4. „Mentés most" is the local copy only (07 §8, by design); the second-drive and off-site copies run in the night. The sweep did not wait for a night, so „the right tiers ran" is checked as „the local unit holds the app".
5. Categories: the brief names „Apps & catalog" for bigger issues; a row whose harm is in backup/restore or storage is filed in that category instead (register_shape_gate has one category per row, and the owner reads it there).
6. Sev: the brief's scale (data at risk P1/P2, a broken app P2, a confusing step P3). R-1269, R-1271 are P2 by that rule (the data is not deleted in either; the household loses access to it).
7. The sweep box reads the LIVE catalog (main 5e196e1, then 227a544 — rules only, no template moved). The drill catalog was used only for the three apps whose fix needed proving; the box was pointed back after (_drill-repoint.txt).
8. Harness fixes live in the sweep's copy (tools/), not in the catalog's scripts; porting them is R-1277.

The box (Part 0)

_box/README.md: VM 9360 sweep1024-box (6 cores, 12 GiB, disks 256 + 200 + 150 GB on nvme-scratch), ISO 1.30.0, customer Tester4 (hub host Tester4-aa9466), guest 9201, domain tester4.test (no DNS: the browser LXC maps the names). Bind 12:42:18Z, day-0 SUCCESS 12:45:40Z, claim, recovery code escrowed (stored out-of-band), drives hdd_1 (210 GB) and hdd_2 (157 GB) plus the installer's internal place belso. Versions at the start: controller 0.342.0, agent 0.164.0, hub 0.169.0. Browser: LXC 9361 sweep1024-browser (Chromium + Playwright), 60 dashboard screens (15 pages × phone/computer × HU/EN) in _box/screens-dashboard/.

Method per app (Part 1)

tools/sweep.py <app> — one app at a time, small first, heavy last (tools/order.txt); the next starts after the last is removed with its data. Each <app>/ holds record.json (the measured facts), walk.log, the page texts (page-{install,guide,overview}-{hu,en}.txt) and screens/ (phone + computer, HU + EN, install page and the app after first login). Steps: install through POST /api/stacks/<app>/deploy (timed to the front door answering); a stranger request without the family cookie (must be refused where the template gates); one item through the app's own API (the catalog's fixture where one exists, a new one in tools/sweep_fixtures.py for seven more); „Mentés most"; remove keeping data; reinstall (the kept-data question answered „use" where it is asked); read; unit restore; read; remove with data; leftovers checked (containers, volumes, app.yaml, appdata, kept, the subdomain → 404). Memory: the peak of each container's anon memory from its cgroup (memory.stat; the file cache is not counted — project rule) against its compose limit; a * in the table means only docker stats was read for that app (it includes the cache). The anon reading joined mid-sweep, so 22 apps carry it and 40 carry the docker stats upper bound; of those 40 none passed 79 % of its limit (Tandoor 1009/1280) and none restarted. Above 80 % on anon: only OpenGist (F14).

Results

App	Verdict	Peak memory / limit (MiB)	Install → answering (s)	What we saw	Row / fix
actualbudget	works	49* / 256	30	the seeded item back after the restore
adventurelog	works with a problem	381 / 896	207	works; but the keep-backups → reinstall → restore path breaks it (DB password from the new install over the old data) — reproduced in its re-run	R-1269, R-1270
audiobookshelf	works	56* / 512	41	the seeded item back after the restore	drive names: R-1275
bentopdf	works	3* / 384	30	stateless (nothing to keep); installs, answers, removes clean
bookstack	works	167* / 768	71	the seeded item back after the restore
calcom	works	807 / 1792	278	the seeded item back after the restore; 2 start-up messages of the image itself
calibre-web	works	211 / 768	76	the book (its own upload) is listed again after the keep-data reinstall; the fixture's download check gets 401 (its sign-in for /download, not the app); the admin name is the box's generated one (re-run 2)	R-1277
claper	works	267* / 640	86	the seeded item back after the restore
code-server	works	81 / 1024	46	first install refused by the registry's rate limit, told as „exit code 1”; the re-run installed and worked (no seed: the editor has no API)	R-1273
crafty-controller	works	79* / 2048	76	the seeded item back after the restore
dawarich	works	835* / 2688	136	the seeded item back after the restore (every-5th: restore also before the reinstall)
docmost	works	528* / 1152	126	the seeded item back after the restore
emby	works	95 / 2048	41	installs, answers, removes clean; no seed (no fixture for its wizard)
ghost	works	167* / 512	121	the seeded item back after the restore
gitea	works	112 / 512	25	the seeded item back after the restore
glance	works	7* / 128	15	family gate holds a stranger; config-only (nothing to keep); the app's own headings are English
gokapi	works	6 / 128	15	installs, own login, removes clean; no seed (API key only in its UI)
grafana	works	383* / 768	61	the seeded item back after the restore
gramps-web	works	347* / 1024	206	installs, answers, removes clean; no seed fixture
grimmory	works with a problem	594 / 1408	96	first admin, library and an EPUB upload work; setup gate then family gate work as designed; the read-back through the family gate stayed inconclusive (the harness's gate logic, R-1277); remove-with-data leaves the empty parent folder	R-1276, R-1277
grocy	works	34* / 384	36	the seeded item back after the restore
home-assistant	works	311 / 1024	161	the seeded item back after the restore
homebox	works	205 / 256	20	the seeded item back after the restore
homepage	works with a problem	91* / 256	25	no gate at all: a stranger sees the box's CPU/RAM/disk (and later the family's links); still the upstream sample page	R-1272
immich	works with a problem	1845 / 6016	121	album (new fixture) in; „use my kept data” loaded the rehearsal's future-dated database and Immich never answered again (404, 20 min)	R-1271
jellyfin	works	110 / 2048	41	first user (new fixture, its wizard API) back after the restore
karakeep	works	649* / 2816	107	the seeded item back after the restore
kimai	works	303* / 640	111	own login; a Hungarian browser lands on /hu/ (curl without a language gets /en/)
komga	works	473* / 768	46	the seeded item back after the restore	drive names: R-1275
lubelogger	works	60* / 512	25	the seeded item back after the restore
mealie	works	322 / 1000	72	the seeded item back after the restore (the fixture raced after_install on the first run)
metube	works	90* / 768	61	family gate holds a stranger; download queue entry back
n8n	works with a problem	346 / 1536	151	the owner back after the restore; on the live template it tries to send usage data out ([Rudder] lines, 6 in the live re-run) — the drill fix: the variable in the running container, 0 error lines	drill a601950 (morning list)
navidrome	works	23* / 256	25	the seeded item back after the restore	drive names: R-1275
nextcloud	works with a problem	298 / 1664	122	user in; „use my kept data” loaded the rehearsal's future-dated database beside today's files — the user was not there	R-1271
nzbget	works	6* / 512	25	the seeded item back after the restore; „no news server” until the family adds one (the guide says so)
onlyoffice	works	464 / 2048	71	document server (no household data to keep); installs, answers, removes clean (every-5th restore)
opengist	works with a problem	125 / 128	15	the gist owner back after the restore; but 128M is too small: one sign-up + sign-in hit the limit 443 times and swapped 98 MiB (memprobe-128M.txt); at 256M (drill) anon 216 MiB	drill 405077f (morning list)
outline	works	327 / 1152	151	the seeded item back after the restore
paperless-ngx	works with a problem	669* / 1664	146	account in; „use my kept data” loaded the rehearsal's future-dated database — the account was not there	R-1271, R-1276
papra	works	320 / 768	76	the seeded item back after the restore (Papra 26.6 needs an Origin header — a browser sends it)
plant-it	not installed	–	–	lifecycle: abandoned — not offered for a new install (by design)
plex	works	223 / 2048	30	installs, answers, removes clean; no seed (a server is claimed with a plex.tv account)
privatebin	works	31* / 128	15	the seeded item back after the restore; D22 put-back PASS
prowlarr	works	114* / 512	21	the seeded item back after the restore
qbittorrent	works	77* / 1024	15	the seeded item back after the restore
radarr	works	100* / 512	41	the seeded item back after the restore
radicale	works	33* / 128	25	the seeded item back after the restore
rallly	works	346 / 1024	71	installs, answers, removes clean; no seed: Rallly 4.15 closed password sign-up (login by e-mail code); the guide's polls need no account	R-1277
recipe-importer	works	67* / 128	25	stateless helper; installs, answers, removes clean (every-5th restore)
romm	works with a problem	748* / 1536	116	item back through „use my kept data” (its own copy); remove-with-data left the empty parent folder	R-1276
sabnzbd	works	69* / 1024	25	the seeded item back after the restore
seerr	works	166 / 384	92	installs, answers (307 to its setup), removes clean; no seed (needs a media server)
sonarr	works	80* / 512	41	the seeded item back after the restore
sparkyfitness	works	417* / 1792	91	the seeded item back after the restore
tandoor	works with a problem	1052* / 1536	177	works; but the keep-backups → reinstall → restore path breaks it (DB password from the new install over the old data), and the restore still said success — reproduced on its own	R-1269, R-1270
termix	works	111* / 256	61	installs, answers, removes clean (every-5th restore); no seed fixture
transmission	works	6* / 512	20	the seeded item back after the restore
uptime-kuma	works	120* / 384	76	the seeded item back after the restore
vaultwarden	works	37 / 256	25	installs in 25 s, its own login page, sign-up closed (404); the box admin's invite works (run-live-2), but 1.37.4 has no route for the fixture's invited sign-up, so no item was seeded	R-1277
vikunja	works	37* / 256	10	the seeded item back after the restore; D22 put-back PASS
wanderer	not testable on this box	140* / 1024	605	not testable on this box: its web part calls its database at the PUBLIC name, which tester4.test cannot resolve (known, R-739) — unhealthy, 404	R-739
wger	not installed	–	–	lifecycle: hidden — not offered for a new install (by design)
wishlist	works	200* / 512	71	the seeded item back after the restore
zipline	works	233* / 768	81	the seeded item back after the restore

works: 52, works with a problem: 10, not installed: 2, not testable on this box: 1 (of 65)

Findings — each a row, a fix or NOT-A-FINDING

#	What	Disposition
F10	A reinstall's own capture rewrites the unit that „remove — keep backups" kept; restoring it pairs the NEW install's DB password with the OLD data — the app never starts, the item is gone (Tandoor ×2, AdventureLog)	R-1269 (P2)
F11	The unit restore reports ok: true „…az alkalmazás újraindult" while the controller logged „health check failed … did not reach running state"	R-1270 (P2)
F12	„Use my kept data" loaded the rehearsal box's future-dated off-site database (2026-10-28) for Paperless-ngx, Nextcloud and Immich — another drive's install, which 07 §6.5 says is refused; Immich then never answered	R-1271 (P2)
F1	Homepage has no gate: a stranger sees the box's CPU/RAM/disk (Glance, same class, is gated)	R-1272 (operator decision)
F7	An install refused by the registry's rate limit (toomanyrequests) tells the household only „exit code 1"	R-1273
F8	The restore refusal says „turn on remote backup, or connect a second drive" when both exist but have not copied yet	R-1274
F2	English pages show the drive names in Hungarian („Belső tárhely", „Külső tárhely (hdd_1)")	R-1275
F9	„Remove with data" leaves the empty parent folder appdata/<app> (Grimmory, RomM, Paperless-ngx)	R-1276
—	The catalog's box fixtures drifted from the apps (OpenGist, Calibre-Web, Mealie, Papra, Rallly; seven apps had none)	R-1277
F13	n8n sends usage data ([Rudder] lines; no N8N_DIAGNOSTICS_ENABLED=false)	fixed in the drill a601950, proven below; morning list
F14	OpenGist at 125.8 of 128 MiB anon after one sign-up + sign-in; at 128M it hits the limit 443 times and swaps 98 MiB	fixed in the drill 405077f + 492a17f (256M), proven below; morning list
—	52 anonymous Docker volumes stayed after every app was removed with data (PrivateBin, OnlyOffice, RomM, Immich: the images' own VOLUME paths; 4–72 KB, no household data). The sweep's own leftover check filters by the app's label and missed them; found in the end-state read (_box/anon-volumes.txt)	known: note on R-1267 (widened to OnlyOffice and RomM)
—	Wanderer unhealthy, 404: its web part calls its database by the public name, which a box without DNS cannot resolve	known: note on R-739
F3	The install page's default place is the internal place belso, not the 210 GB drive	NOT-A-FINDING: by design — 16 §5 registers the internal place as the default drive
F4	Guide labels in English (LubeLogger „Settings", NZBGet „Settings / News-Servers")	NOT-A-FINDING: they are the app's own on-screen labels, and the Hungarian catalog copy is frozen by the copy-i18n gate (measured on the drill)
F5	Kimai opens in English	NOT-A-FINDING: a Hungarian browser lands on /hu/; only a request with no language gets /en/
F6	AdventureLog backend 95 % of its limit	NOT-A-FINDING: that was docker stats with the cache; on anon 288.7/384 MiB (75 %), = the ladder's 73 %
—	Database error lines (SparkyFitness database "sparky" does not exist, Dawarich database "dawarich" already exists; SparkyFitness Invalid password = the harness's wrong-password control)	NOT-A-FINDING: the images' own first-start and restore-time checks; no restart, the item read back after the restore
—	Radicale redirects to an IP	NOT-A-FINDING: a curl artefact (no Host on the redirect probe); the browser lands right
—	NZBGet / Transmission screenshots show the browser's basic-auth prompt	NOT-A-FINDING: the app's own login, by design
—	code-server's first install failed (toomanyrequests)	the failed install was removed through the product (code-server.failed-remove.txt); the re-run worked; the message is R-1273

F10 proof — the kept unit is rewritten by the reinstall (_unitprobe/tandoor/)

tools/unitprobe.py tandoor: install, one account, „Mentés most" (unit 19:45:14–20), remove keeping data and backups, install again (a volume app: no kept-data question), wait for the controller's next capture, „Visszaállítás".

• 19:50:32 Recovery unit captured for tandoor — the unit's app.yaml checksum 0f366c97… → 4438c1c5…, created_at 19:45:26 → 19:50:32; the data files unchanged (unit-1/2/3*.txt).
• The restore: ok: true, „A(z) tandoor: 3 adatkötet és az adatbázis visszaállítva — az alkalmazás újraindult."
• After: state starting, the account not there, password authentication failed ×42 (app) and ×22 (postgres).
• The comment at felhom-controller/controller/internal/backup/recovery_unit.go:101-103 says the refresh never rewrites the definition away from its data; after a remove + reinstall it does.

Seen three times: the sweep's own Tandoor cycle (17:27:41 reinstall → 17:30:34 capture → 17:30:47 restore), this probe, and AdventureLog's re-run (20:03:18 → 20:05:32 → restore). AdventureLog's first run worked only because no capture fell between its reinstall and its restore.

D22 put-back — PASS on two apps (_putback/)

tools/putback.py <app>: the largest volume tar of the app's unit truncated to half, then „Visszaállítás". PrivateBin and Vikunja: the restore failed honestly („Nem változott semmi…"), the pre-restore data was put back, the app ran, and the seeded item read back through the front door.

Drill fixes, proven on the sweep box pointed at the drill (Part 2)

Drill catalog app-catalog-drill (never the live catalog): 742a354 (merge of live main), then:

• a601950 n8n — N8N_DIAGNOSTICS_ENABLED=false in the compose (usage data stays home).
• 405077f + 492a17f OpenGist — memory 128M → 256M (compose + .felhom.yml mem_limit), the comment carries the measured need.

How the box was pointed at the drill. The drill repo on Gitea is private (R-992) and the box holds no catalog credential, so the box reads a local mirror: a bare repository in the controller's data volume, loaded from a git bundle, repo_url: file:///opt/docker/felhom-controller/drill-catalog.git (the method of audits/burndown-3-2026-10-17/tools/repoint.py; loading the mirror is a docker exec git fetch into the controller — catalog plumbing, not app state). Only repo_url changes; tools/repoint.py live puts the saved controller.yaml back and checks it byte for byte (RESTORED-IDENTICAL, twice in _drill-repoint.txt). The box ended on the live catalog 227a544.

What went wrong first, and is kept as evidence. The first drill pass (chain 3, 21:18Z) set repo_url to the Gitea URL in the belief that the repo was public. The box's clone failed (fatal: could not read Username), the controller kept the templates it had, and n8n, OpenGist and Vaultwarden ran on the LIVE templates. The leftover [Rudder] lines in that n8n run showed it. Those three runs are kept as run-live-2/ (valid live re-runs). The second pass (chain 4) refused to install anything until the box's catalog cache read the drill's commit and the changed lines were in the box's copy of both templates: drill main fd23571, box cache fd23571: DRILL-CONFIRMED. (The drill also carries a six-line Plex ladder entry from an earlier session's test; Plex was not installed in this pass.)

n8n proof. The running container carried N8N_DIAGNOSTICS_ENABLED=false (read at 21:37:58, that line only); error lines 0 against 6 [Rudder] lines in the live re-run just before it (the control); the owner account read back after the unit restore.

OpenGist proof — and a wrong turn. At 256M (drill) OpenGist took 215.8 MiB anon. I first read that as „it grows into its limit" and reverted the fix (c420053); an automated security review flagged the revert as a lowered resource cap. A measurement settled it (tools/memprobe-guest.sh, opengist/memprobe-128M.txt): on the live 128M template one sign-up + sign-in hit memory.max 443 times and pushed 98 MiB into swap (anon 125 MiB, 0 oom_kill, 0 restarts). The real need is ~220 MiB; the fix went back in (405077f). Where the box has no swap room left, the same sign-up would end in a kill (not measured here).

Drill gates: catalog_gates.py --fast opengist n8n from the drill clone — all OK except security-scan, which looks for a felhom.eu clone beside the drill and found none; the shared scanner run by hand on the drill clone: 0 hits.

Teardown

Three layers, 2026-10-25 ~00:05Z:

• Machine (demo-hp): VM 9360 sweep1024-box and LXC 9361 sweep1024-browser — both created by this session — stopped and destroyed with --purge; no disk of either is left on nvme-scratch. Untouched: Tester 1 (VM 341), 9201, 9202, 9401, the thin pool, and the 1.30.0 ISO in local:iso (there since 2026-10-22). Before the destroy, the box's controller log, the agent journal, its end state and its anonymous-volume list were copied here (_box/).
• Host (DooPlex): the session's secret files (/run/user/1000/sweep-sc: the dashboard password copy, cookies, CSRF tokens) shredded. The drill catalog keeps its commits (a601950, 405077f, 492a17f, and the reverted turn c420053) for the morning list; nothing went to the live catalog.
• Hub: host Tester4-aa9466 stays for the operator to delete (CC holds no delete). Until then it reads as a box that stopped reporting.
• ep0: listed, nothing removed. The host's PBS link to ep0 (felhom-pbs) was set up at day-0 (pbsdr: converged, 14:44:48 CEST, _box/agent-journal.log.gz); no whole-box copy went to it (weekly cadence — only the local copy ran, 14:51). The app copies the box sent off-site went to the Tester4 restic repository on the storage box (unlocked with the rehearsal's recovery code; a new code escrowed — stored out-of-band), not to ep0.

Evidence map

• _box/ — Part 0 (VM, ISO, bind, day-0, claim, recovery, drives, dashboard screens).
• <app>/ — one per app; re-run apps keep the earlier run in first-run/, run-2/ or run-drill-pre/.
• <app>.console.txt, progress.log, batch.out, chain2.out, chain3.out — the runner's own output.
• _unitprobe/tandoor/ — F10. _putback/{privatebin,vikunja}/ — D22.
• tools/ — the harness (sweep copies of the catalog's box_walk.py and upgrade_fixtures_box.py, marked SWEEP COPY; sweep.py, sweep_fixtures.py, unitprobe.py, putback.py, repoint.py, table.py, the rehearsal's VM and browser helpers). Secrets were read by name through sec and never written here; the evidence was scanned for every known secret value and for token shapes (one app session token found and blanked: Actual Budget's), with a positive control.
• Screenshots were reduced with pngquant (lossy palette) to keep the repository small.