app-catalog-felhom.eu/CHANGELOG.md

# Changelog

## 2026-08-02 — one entry point for the catalog's gates (R-161 ruling)

`scripts/catalog_gates.py` runs all three gates — image-pins, image-resolvable, volume-persistence —
and exits non-zero if any fails. Mandated in `CLAUDE.md` the way `felhom.eu/scripts/site_gates.py` is:
**run it after any template change**, naming the app(s) you touched.

**Operator ruling, recorded because the alternatives were rejected for measured reasons.**
Controller-side enforcement at template load was rejected: such a check can only read the file, and a
static audit of all 53 templates reports the catalog clean **including papra** — it would pass on the
exact defect it exists to catch. CI was rejected for now: neither repo has any, and there are no users
yet. What was chosen copies the shape that demonstrably works here — of this project's gates, the only
ones that ever get run are the ones with a single entry point named in a CLAUDE.md; `site_gates.py` is
run, and R-29's three orphaned gates are named nowhere and have stopped nothing.

Behaviour: `0` all clean · `1` convicted · `2` UNDETERMINED, **never a pass**; a conviction outranks an
undetermined result in the summary so the reader knows which they have. Gate output is streamed, not
captured — a runner that swallows diagnostics makes a conviction unreadable. Scoping passes app names
through to the two gates that accept them; with no names the runtime gate deploys every template and
belongs on a scratch host.

**R-161 stays OPEN at reduced scope:** this is convention, run by a person. Real automatic enforcement
is owed when a second person touches templates.

Verified: `image-pins` passes standalone (53 templates, 0 unpinned); the unknown-option path exits 2;
the aggregation was unit-checked over five gate-code combinations. **The runtime leg was deliberately
NOT executed on DooPlex** — it deploys templates via `docker compose`, and DooPlex is the recovery
chain; it belongs on a scratch host.

## 2026-08-02 — persistence sweep: does every app's data land in a folder the template preserves?

Campaign 10's R-156 found papra writing its database into the container's writable layer while the
volume the template preserves stayed empty — so its backup completed, verified, and contained
nothing. papra was never the point: **nothing anywhere checked that the folder a template preserves
is the folder the app writes to**, across 53 templates. All 53 have now been measured live.

**Result: 43 CLEAN · 3 BROKEN · 7 UNDETERMINED.** Full report and per-app evidence:
`audits/persistence-sweep-2026-08-02/`.

**New gate — `scripts/check-volume-persistence.py`, the third and the only RUNTIME one.**
The two image gates are static, and **this defect class is invisible to static analysis** — measured,
not assumed: a static audit of all 53 composes (every declared volume attached, no anonymous mounts,
no stray host binds) reports the catalog clean *and reports papra clean*. papra's compose is
well-formed; only its behaviour is wrong. So the gate deploys each template, exercises it into
writing data, and compares where the data landed with what is mounted. Exit **0** all clean /
**1 REFUSED** / **2** undecided. `UNDETERMINED` is exit 2 and is never a pass.

It **refuses to report at all** unless it has just re-proven itself in both directions against two
canary templates built from a purpose-made image reproducing papra's ownership shape — the pair
differ only in which path the volume mounts at, so every run carries a live demonstration of R-156
and of its fix. A detector that flags nothing turns an unexamined catalog into a documented-clean one.

41 fixture tests (`scripts/test_check_volume_persistence.py`, no Docker) driving `check()` — the
function `__main__` calls — plus `rollup_diff`/`classify`. Every rule red-proofed.

**Two templates FIXED** (neither deployed anywhere in the fleet, so no data was stranded):

- **`gramps-web`** — mounted `/app/data`, `/app/media`, `/tmp`, and **`/app/data` is a path the
  application never writes**. Its accounts database (`GRAMPSWEB_USER_DB_URI` → `/app/users`) and
  **its family tree** (`GRAMPS_DATABASE_PATH` → `/root/.gramps/grampsdb`) both landed in the
  container's writable layer: destroyed by any redeploy, absent from every backup, while
  `gramps_data` was tarred nightly as an empty directory. Now persists the eight paths the image's
  own environment names, matching upstream's reference compose.
- **`wishlist`** — mounted `wishlist_data:/data`, another path the app never writes. `prod.db` went
  into the **anonymous** volume docker creates for the image's `VOLUME /usr/src/app/data` directive.
  Anonymous volumes are absent from `ResolveDockerVolumeNames`, so `DumpAppVolumes` never backs them
  up, and `compose down` + `up` orphans them — a store that survives a restart, loses on redeploy and
  is never in a backup. Now mounts `/usr/src/app/data` and `/usr/src/app/uploads` per upstream.

Every corrected path is confirmed by **two independent sources** — the shipped image's own
environment/`Config.Volumes`, and upstream's reference compose — never inferred from a directory name.

**`papra` is NOT fixed — referred to the operator.** The one-line fix is prepared and proven, but
papra is live on one box, and changing the mount target makes the next `compose up -d` recreate the
container and destroy the writable layer its documents currently live in. That data is already on
borrowed time, but the fix is what *schedules* the loss. See the report §6.1 for which box it is,
how far that was determined, and the two options. No migration was written.

**7 UNDETERMINED, counted separately and never folded into CLEAN** — `bentopdf` (stateless by
design), `uptime-kuma` / `privatebin` / `recipe-importer` (write nothing until a user completes
setup), `glance` (crash-loops for want of a seeded config — pre-existing, Campaign 7 §6.2),
`plant-it` (image does not resolve; `lifecycle: abandoned`), `wanderer` (unhealthy).

`CLAUDE.md` and `REUSE.md` updated with the gate and the traps it encodes.

## 2026-07-21 (later) — app lifecycle replaces the `retired/` directory move

**The `retired/` mechanism shipped earlier today was wrong and is withdrawn.** Moving a template out
of `templates/` does un-offer it — but it also makes the controller's orphan detector see the
template as GONE for anyone already running the app, flagging their working install `Elavult` and
offering a Törlés button. Withdrawing an app must never take a working app away from a customer.

Replaced by an optional top-level `lifecycle:` field in `.felhom.yml` (controller v0.158.0):

- `available` — default. Absent or empty means this, so all existing templates are unchanged.
- `hidden` — not offered for new installs; nothing shown to anyone already running it.
- `abandoned` — not offered for new installs, and every box already running it shows a permanent
  „Nem karbantartott" badge plus a notice that updates and security fixes will no longer arrive.

Deployed instances keep full function in every state; the controller refuses a deploy of a
non-available template server-side. An unknown value degrades to `available` with one WARN.

- **`plant-it` returns to `templates/`** with `lifecycle: abandoned` — the first user of the
  mechanism, and the case that motivated it. Its compose is deliberately unchanged: it pins
  `msdeluise/plant-it:0.10.0`, a repository that does not exist (the real one is `-server`), and the
  app is not installable, so rewriting it would imply it is. `retired/` is removed.
- **The resolvability gate is now lifecycle-aware.** Non-available apps are skipped by default and
  REPORTED, not silently dropped; `--all` includes them. An abandoned app's dead image is the
  expected end state, not a finding — counting it would leave the gate permanently red for something
  nobody intends to fix, and a gate that is always red is a gate nobody reads. 6 new fixture tests
  (19 total), including one asserting an all-skipped run is a pass rather than an error.

Catalog is back to **53 apps** (52 offered + plant-it abandoned).

## 2026-07-21 — catalog honesty: wanderer re-pinned, plant-it retired, and a standing rot gate (R-41 slice 1)

Campaign 7 left two apps sitting behind a working "Telepítés" button with images that did not
resolve at all, recorded as findings rather than fixed. Both are now diagnosed rather than hidden,
and the class of defect gets a gate so it cannot recur silently.

**wanderer — RE-PINNED. The project is alive; the template was pointing at a ghost.**
`ghcr.io/flomp/wanderer:0.16.0` does not resolve because upstream did three things at once: split
the app into two images, moved registry, and renamed the GitHub org (Flomp → open-wanderer). Current
shape, taken from upstream's own compose at tag v0.20.0 (2026-07-07):

- `flomp/wanderer-web:v0.20.0` — the SvelteKit web app, port 3000, `curl` on PATH.
- `flomp/wanderer-db:v0.20.0` — PocketBase, port 8090. Built FROM `scratch`: no shell, no package
  manager, a static curl baked in at `/curl` — hence the absolute-path healthcheck.
- `getmeili/meilisearch:v1.36.0` — still a required sidecar; both other services wait on its health.
  **Pinned DOWN from the v1.49 Campaign 7 had set**, per the R-42 ruling: a sidecar pin follows the
  app template's own proposed pin, never the newest tag independently.
- **New required volume** `/data/plugins` on the db — v0.20.0 moved the Strava/Komoot/Hammerhead
  integrations into a WASM plugin sandbox that lives there.
- **New: a second hostname** (`SUBDOMAIN_DB`, default `hike-db`). `PUBLIC_POCKETBASE_URL` is a
  browser-side variable — the user's browser talks to PocketBase directly, so it cannot be an
  internal address. Upstream's own proxy example uses two hostnames for the same reason.
- New generated secret `POCKETBASE_ENCRYPTION_KEY` (`hex:16` → exactly the 32 characters upstream
  requires). `mem_limit` 384M → 1024M, matching the sum of the three services.

**plant-it — RETIRED to `retired/plant-it/` (operator ruling 2026-07-21).** The pin was only
slightly wrong — the repository is `msdeluise/plant-it-server`, and `0.10.0` was the right version —
but correcting the name would have been the wrong fix. Upstream has **discontinued self-hosting**:
`backend/` and `deployment/` are deleted from `main`, the project is now an Android app on
F-Droid/Obtainium, and the last server image was pushed **2024-12-10** (a security-frozen Spring
Boot 3.4.0). It also requires **MySQL 8.0 + Redis**, which the template never had — its header
claimed "Database: None (file-based)", which was never true. Ruling: do not ship unmaintained
software to customers. Retirement is reversible (`git mv retired/plant-it templates/plant-it`);
nothing is deleted. Catalog is now **52 apps**.

**`scripts/check-image-resolvable.py` — R-41 slice 1: the standing rot gate.** `check-image-pins.py`
is syntactic and proves only that a template pins *something* concrete; it cannot see that the thing
is gone. This resolves every unique pin with `docker manifest inspect`, one image at a time, and
exits 0 / 1 (GONE) / 2 (inconclusive). Two traps are encoded in it, both observed live during this
change:

- `docker manifest inspect` prints `toomanyrequests: …` and **still exits 0** — the same
  exits-0-on-failure shape as the ISO tooling's `validate-answer`, so stderr is checked even on rc=0.
- The inverse, which the first full sweep actually did: it called **24 of 65 pins dead**, including
  `postgres:16-alpine` and `redis:7-alpine`, purely because Docker Hub throttled it partway through.
  Ambiguity now resolves to INCONCLUSIVE, never to an accusation — a gate that cries wolf gets
  ignored, and then it protects nothing.

14 fixture tests (`scripts/test_check_image_resolvable.py`), no network — the resolver is injected.

## 2026-07-19 — docs: workspace-root pointer follows the CC move to DooPlex

**Docs only, no template change.** Claude Code now runs on DooPlex (192.168.0.180, Debian 13)
instead of the Windows workstation. `CLAUDE.md`'s cross-repo pointer becomes
`/mnt/5_hdd/felhom.eu/git/CLAUDE.md`. This repo carried **no other** environment-specific content —
it was the only one of the four that needed nothing else.

## 2026-07-19 — CAMPAIGN 7: full catalog sweep (53/53 apps deployed + validated on the demo box)

Every app in the catalog was bumped to its newest stable upstream tag where one existed, then
**actually deployed** through the controller's real endpoints on the demo box (controller 0.146.0),
validated (all containers healthy, HTTP through the real Traefik ingress, log scan), and removed
again through the real delete flow. Full evidence + result matrix:
`felhom.eu/documentation/audits/CAMPAIGN-7-catalog-sweep-2026-07-19.md`.

**Result: 45 apps pass end-to-end, 4 do not, 1 is not automatable (plex needs a real PLEX_CLAIM).**

**Version bumps** — ~40 templates moved to current upstream, 15 of them across a major
(bookstack 25.02→26.05, immich v2→v3, calcom v4→v6, nextcloud 31→34, grafana 11→13, n8n 1→2,
outline 0.82→1.9, vikunja 0.24→2.3, tandoor 1→2, romm 4→5, radarr 5→6, privatebin 1→2,
onlyoffice 8→9, claper 1→2, gramps-web v24→v25). `uptime-kuma` moved off the floating `:2` tag
to `2.4.0`. **DB/cache sidecar majors were deliberately NOT bumped** — rationale in the campaign
doc §4 (a DB major is the application's decision, and `postgres:16-alpine` already tracks 16.x).

**13 template fixes, every one live-re-validated:**

- **7 broken healthchecks.** This is not cosmetic: Traefik will not route to an `unhealthy`
  container, so a probe that cannot run makes the app return **404 to the customer while it serves
  200 on its own port**. adventurelog (wget in a distroless image → Node-exec at an absolute path),
  emby (curl absent, BusyBox only), papra + wishlist (node-only images), homebox (`--spider` sends
  HEAD, endpoint answers 405 to HEAD / 200 to GET), zipline (v4 renamed `/api/health` →
  `/api/healthcheck`), tandoor (`start_period` too short for gunicorn).
- **5 apps that had NEVER been deployable** and were fixed: papra (missing required `AUTH_SECRET`,
  now a generated `data_key` secret), zipline (v4 `CORE_DATABASE_URL` → `DATABASE_URL`), wishlist
  (dead Docker Hub image → followed upstream to `ghcr.io/cmintey/wishlist:v0.66.0`), homebox
  (upstream dropped the `v` tag prefix + new required `HBOX_AUTH_API_KEY_PEPPER`), wger (2.6 needs
  the full `DJANGO_DB_*` set and listens on :8000, not :80 — the Traefik port was wrong too).
- **4 memory/OOM corrections proven by a live OOM:** gramps-web 384M→1024M, n8n 512M→1536M
  (V8 heap), rallly 256M→768M, tandoor 512M→1024M (+ its `mem_limit` sum was already wrong).
- **gokapi reverted v2.2.4 → v1.9.6**: v2 refuses to run against the seeded ConfigVersion-21
  config and demands an intermediate v2.0.0 pass, even on a fresh deploy. Shipping it would have
  broken every new gokapi deploy. Needs a dedicated v2 config-migration task.

**Still failing (recorded, not fixed):** `glance` (needs a seeded `glance.yml`; PROVEN pre-existing —
the pre-campaign v0.7.4 pin fails identically), `gokapi` (above), `plant-it` and `wanderer`
(their images do not resolve at all — neither the new tag nor the one the catalog already shipped).

## 2026-07-14 — backup classification `backup:` blocks for the 13 bind-bearing apps (controller v0.132.0)

Adds the referential-coupling `backup:` classification block to every catalog app that binds
`${HDD_PATH}`/`${USERDATA_PATH}` (13 apps: immich, paperless-ngx, nextcloud, calibre-web,
audiobookshelf, komga, navidrome, radarr, sonarr, emby, jellyfin, plex, romm). Each block lists its
`userdata:`/`hdd:` binds with a `class ∈ {mandatory, optional, excluded}` (COUPLED /
DECOUPLED-precious / DECOUPLED-bulk); classes are operator-ruled (Viktor, 2026-07-14) + spike SQ2.

Requires **controller v0.132.0**, which parses + validates these blocks (Task 2 of the
backup-classification-redesign arc, `felhom.eu/documentation/audits/SPIKE-backup-classification-2026-07-14.md`).
The classification is **INERT** — no backup tier changes behavior yet; Task 3 (tier policy engine)
and Task 4 (manual `.fab` UI) consume it. All 13 blocks were verified against the shipped controller
parser: parse-clean, every bind resolves `explicit` to its ruled class (zero validation errors).

Notes: audiobookshelf `media/audiobooks` = **optional** (consistency with komga/romm curated media;
the spike proposed excluded — PENDING a Viktor veto). radarr/sonarr `downloads` = excluded (transient
cross-app queue). emby/jellyfin/plex `media` = excluded (`:ro` readers; state in volumes). The 42
volume-only apps get no block (classification moot — state rides in the recovery unit's volume dumps).

## 2026-07-12 — image pinning sweep: `:latest` eliminated from all templates (5 pins) + standing gate

A catalog sweep found 5/53 templates with unpinned images. Beyond version discipline, `:latest`
breaks restore fidelity: the controller's recovery-unit `ImagePins` pins the *tag*, so restoring a
`:latest` app re-pulls whatever `:latest` means at restore time — potentially schema-incompatible
with the data being restored. Rule applied: a deployed app pins to the digest it is RUNNING
(pin ≠ upgrade); undeployed apps pin to the verified upstream stable. All five pins are
digest-identical to what `:latest` resolved to on 2026-07-12 — a pure no-op for running apps.

| App | Old | New | Evidence |
|-----|-----|-----|----------|
| bentopdf | `ghcr.io/alam00000/bentopdf:latest` | `:v2.8.6` | digest == latest (`eaeea1e4…`); undeployed |
| calibre-web | `crocodilestick/calibre-web-automated:latest` | `:v4.0.6` | digest == RUNNING image on demo 9201 (`c31a738b…`) |
| papra | `ghcr.io/papra-hq/papra:latest` | `:26.6.1-rootless` | latest == the -rootless variant (`a7a42e22…`); `-root` differs — variant preserved |
| recipe-importer | `gitea.dooplex.hu/admin/recipe-importer:latest` | `:v0.9.11` | tag pre-existed in registry, digest == latest (`f3cb617c…`) — no retag needed |
| termix | `ghcr.io/lukegus/termix:latest` | `:2.5.0` | digest == latest == release-2.5.0 (`4d337131…`); undeployed |

- New rerunnable gate `scripts/check-image-pins.py`: fails on `:latest`/`dev`/`nightly`/`edge`/
  `main`/`master` AND on untagged image refs (implicit :latest); `@sha256:` digests count as pinned.
  Red-proofed both shapes (revert→exit 1→restore).
- Standing rule added to `CLAUDE.md` (never :latest / untagged; deployed apps pin to running digest).
- `templates.json` carries no image strings (legacy metadata only) — untouched.
- Fleet caveat: non-deployment of bentopdf/papra/termix verified on demo 9201 only; felhotest
  unreachable + Peti's box offline at sweep time (operator approved proceeding — pins are
  digest-equal to latest, so worst case equals the status quo).

## 2026-07-06 — healthcheck sweep: `localhost` → `127.0.0.1` across all 48 templates

Escalation of the re-run vaultwarden observation
(`felhom.eu/documentation/audits/RERUN-p1p3-2026-07-06.md`) from an instance to a **class**: 48/53
templates used `localhost` in their docker healthcheck `test:` line. BusyBox `wget` (and the node /
python / curl one-shot forms, incl. mealie's `socket.create_connection`) resolve `localhost`→IPv6
`::1` with no cross-address-family fallback, so an IPv4-only-binding app reads docker-`unhealthy`
while fully serving. Mechanical sweep `localhost`→`127.0.0.1`, scoped strictly to the healthcheck
`test:` lines (diff-reviewed: no app env/config/label line changed; `.felhom.yml` files were already
clean). Industry practice — never `localhost` in container healthchecks. New REUSE.md convention row.

## 2026-07-06 — vaultwarden F1 fix: _ENABLE_SMTP boot-gate (campaign finding, pilot-blocking)

The no-mercy campaign (felhom.eu `audits/CAMPAIGN-nomercy-2026-07-06.md`, finding F1) proved that a
FRESH vaultwarden deploy with app-email off — the default state — crash-loops: the template always
defines `SMTP_HOST=${SMTP_HOST:-}` / `SMTP_FROM=${SMTP_FROM:-}`, and vaultwarden treats a
defined-but-EMPTY env var as "set", so its config validation (`smtp_host.is_some() ==
smtp_from.is_empty()`) errors out and the process exits. The old comment ("empty SMTP_HOST = mail
stays disabled") was wrong for this image. Empirically proven on the pinned
`vaultwarden/server:1.33.2-alpine` (probe P1: defined-empty pair → exact campaign error, exit 12;
P2: `_ENABLE_SMTP=false` + same empty pair → boots; P3: `_ENABLE_SMTP=true` + host+from → boots).

Fix: gate the whole SMTP group with vaultwarden's own `_ENABLE_SMTP` flag — compose default
`false` (validation skipped, mail off, clean boot), flipped to `"true"` by the app-email injection
via `smtp_mapping.extra` (no controller change needed — `extra` already rides `smtpEnv`). The ON
path is byte-identical to the previously send-tested state plus the flag.

Sweep note (no edits): the other five smtp-mapped templates (calcom, gitea, mealie, nextcloud,
rallly) are boot-proven tolerant of defined-empty mail env — all ran healthy as fresh email-off
deploys during the campaign; gitea's `GITEA__mailer__SMTP_ADDR=${...:-}` pattern likewise.
Vaultwarden was the only strict image. New REUSE.md trap row: strict images need an enable-flag
gated `false` in compose + `"true"` in `smtp_mapping.extra`; boot-prove fresh email-off deploys.

## 2026-07-03 — sparkyfitness FINALIZED + live-validated (both VERIFY markers resolved); REUSE probe-naming row

The first worked example of the new `felhom-app-catalog` skill (felhom.eu). Both
`VERIFY-BEFORE-FINALIZE` healthcheck guesses resolved by inspecting the real images on the demo box:
frontend (Alpine/nginx) HAS BusyBox wget → drafted `wget --spider :80/` probe confirmed + kept;
server HAS node v24.17.0 → node-exec `:3010/api/health` probe confirmed (path proven live:
`{"status":"UP"}`). Frontend `container_name` renamed → `sparkyfitness` (= the stack name): the
controller-side probe dials the exact-name container, fallback is the FIRST prefix match (could be
the DB) — new REUSE.md §2 "Probe-container naming" row records the convention (verified in
felhom-controller healthprobe.go). Mem-sum comment added (512+1024+256 = 1792M, value unchanged).
Live-validated on demo via the real dashboard UI (sync + Frissítés): 3/3 containers healthy,
controller probe `healthy: true` (http :80 → 200), `sparky.demo-felhom.eu` 200 via Traefik;
data_key secrets untouched (server/db containers not recreated). Kept deployed.

## 2026-07-03 — docs: CLAUDE.md light expansion

The minimal REUSE-rollout stub expanded to a proper (still ~30-line) CLAUDE.md: what the repo is
(one dir per app, two template files, Hungarian customer text), the push-to-main = deploy contract
(controller sync ≤15 min / manual trigger), legacy `templates.json` warning, and pointers
(REUSE.md, README format spec, the `felhom-build-deploy` skill). No template changes.

## 2026-07-03 — docs: REUSE.md introduced

Cross-repo reuse-map rollout (docs-only). New `REUSE.md`: catalog conventions verified against all
53 apps — the canonical example app (paperless-ngx), `.felhom.yml` required fields, healthcheck
family per image type (BusyBox wget / curl / Node / Python / DB sidecars), memory-limit convention,
new-app checklist, and traps (gokapi entrypoint hack, legacy templates.json). Known README drift
recorded in §6 (NOT fixed). Also a minimal `CLAUDE.md` carrying the REUSE.md pointer + maintenance
rule (full CLAUDE.md is a separate task).

## 2026-06-29 — App-email: calcom + nextcloud (tls_mode=plaintext :2526 + nextcloud split-From)
- **nextcloud** — `smtp_mapping` with `tls_mode: plaintext` (controller injects port 2526, the plaintext-only
  listener) + **split From** (`from_var=MAIL_FROM_ADDRESS` + `from_domain_var=MAIL_DOMAIN` → nextcloud@felhom.eu).
  Compose references the injected `${SMTP_*}`/`${MAIL_*}`. Live-confirmed: real password-reset delivered via
  plaintext :2526 (Symfony Mailer never attempted STARTTLS).
- **calcom** — `smtp_mapping` with `tls_mode: plaintext` (EMAIL_SERVER_HOST/PORT, EMAIL_FROM=calcom@felhom.eu).
  **Plus three pre-existing template fixes** (calcom never deployed before — the image pin was invalid):
  (1) image `v4.8.7`→`v4.6.9` (the pinned tag has no published image); (2) added required `DATABASE_DIRECT_URL`
  (Prisma `migrate deploy` fails without it → incomplete schema → 500s); (3) healthcheck `/api/health`→
  `/api/auth/providers` (the old path 404s in v4.x → container stayed unhealthy → Traefik wouldn't route).
- Both apps point at the controller's `:2526` plaintext-only listener because their SMTP clients
  opportunistically STARTTLS-upgrade and can't skip the self-signed cert — the listener simply doesn't offer
  STARTTLS, so they stay plaintext (accepted on the single-tenant app bridge).

## 2026-06-29 — App-email rollout: gitea + rallly (calcom/nextcloud/immich = findings)
- **gitea 1.23.4** — added `smtp_mapping` (STARTTLS via `GITEA__mailer__PROTOCOL=smtp+starttls` +
  `FORCE_TRUST_SERVER_CERT=true` to trust the shim's self-signed cert; single `GITEA__mailer__FROM`). Compose
  references the injected `GITEA__mailer__*` keys; env applied every boot.
- **rallly** — added `smtp_mapping` (Nodemailer STARTTLS, `SMTP_SECURE=false` + `SMTP_REJECT_UNAUTHORIZED=false`
  to accept the self-signed cert; single `NOREPLY_EMAIL`). **Also fixed three pre-existing template bugs** that
  made rallly undeployable (never caught because the bad pin never ran): (1) image pin `3.12.1` doesn't exist →
  `3.11.2`; (2) healthcheck used `wget`, absent from the rallly image (exit 127) → container unhealthy →
  **Traefik wouldn't route it** → replaced with a Node http check; (3) added required `SUPPORT_EMAIL` + a valid
  `NOREPLY_EMAIL` default (rallly refuses to boot without them).
- **Both gitea and rallly send-tested live** end-to-end (app → shim → hub → Resend): gitea password-reset
  (From `gitea@felhom.eu`) and rallly registration code (From `rallly@felhom.eu`) both delivered.
- **NOT wired — reported as findings** (`felhom.eu/documentation/audits/FINDING-app-email-rollout-2026-06-29.md`):
  - **cal.com v4.8.7** — hard-codes TLS `rejectUnauthorized:true` with no override; opportunistic STARTTLS
    against the self-signed shim fails. Needs a non-STARTTLS-advertising plaintext listener (mechanism change).
  - **nextcloud 31** — no cert-skip env (same opportunistic-STARTTLS gap) **and** a split From
    (`MAIL_FROM_ADDRESS`+`MAIL_DOMAIN`) the single-`from_var` mapping can't express.
  - **immich v2.5.5** — no SMTP env vars at all; config is admin-UI/DB or an `IMMICH_CONFIG_FILE` JSON. Does not
    fit env-injection; left for a future config-file-injection mechanism (or manual admin-UI setup).

## 2026-06-29 — App-email: smtp_mapping for Vaultwarden + Mealie
- Added the `smtp_mapping` block to `templates/vaultwarden/.felhom.yml` and `templates/mealie/.felhom.yml`,
  enabling managed outbound email (app → in-controller shim → hub → Resend) for the two spike-proven apps
  (`SPIKE-smtp-app-relay-2026-06-28`). The controller injects `SMTP_*` at deploy/redeploy when app-email is
  on (global + per-app); the From address is `<app>@felhom.eu`. SMTP auth creds are intentionally left unset
  (the shim accepts no-auth on the Docker network).
  - **Vaultwarden:** STARTTLS (`SMTP_SECURITY=starttls`) + `SMTP_ACCEPT_INVALID_CERTS/HOSTNAMES=true` to
    accept the shim's self-signed cert.
  - **Mealie:** plaintext (`SMTP_AUTH_STRATEGY=NONE`) on :2525 — Mealie has no accept-invalid-cert option, so
    STARTTLS to a self-signed shim would fail; plaintext to the Docker-network-only shim is the spike-validated
    mode.
- Both `docker-compose.yml` files now reference the injected `${SMTP_*}` keys (with harmless defaults) so the
  values reach the container; empty `SMTP_HOST` keeps mail disabled when the toggle is off.
- Documented the `smtp_mapping` pattern in `README.md` so further apps are easy adds.

## 2026-06-28 — Add SparkyFitness (v0.17.2) — nutrition/workout tracker
- New app `templates/sparkyfitness/{docker-compose.yml,.felhom.yml}`: a self-hosted nutrition/calorie +
  workout/weight tracker (alternative to wger). Three containers — nginx **frontend** (SPA :80, the sole
  Traefik ingress, proxies `/api`+`/uploads` internally) + Node **server** (:3010) + dedicated
  **postgres:15-alpine**. Server + DB stay on the internal network with no Traefik labels.
- **Native email/password auth** (no OIDC/Authentik — that's DooPlex-specific); subdomain `sparky`
  (deliberately ≠ wger's `fitness` to avoid a Host() collision). `pi_compatible: false`, `needs_hdd: false`.
- **Two DB roles**: `sparky` (POSTGRES superuser, runs init/migrations) + `sparkyapp` (limited app role the
  server auto-creates on first boot) — separate `DB_PASSWORD`/`APP_DB_PASSWORD`. `PGDATA` in a `pgdata`
  subdir of the named volume. Four auto-generated, `locked_after_deploy` secrets; `API_ENCRYPTION_KEY` +
  `BETTER_AUTH_SECRET` carry `data_key: true` (restore recovers, never regenerates — both are 64-char hex).
- Transcribed from the validated k3s manifest `homelab-manifests/workout-system/sparkyfitness.yaml`
  (pinned image tags, two-DB-role model, never-change crypto keys, `/api/health`, pg15 + PGDATA subdir).
- **Image-probe findings (build server, v0.17.2):** server keeps the `node -e` `/api/health` probe (node
  present); frontend keeps the `wget --spider` probe (both `wget` and `curl` present). No probe changes needed.
- **Live-validated on guest 9201 (controller v0.87.0):** synced via "Sablonok frissítése"; deployed through
  the real dashboard flow (Domain auto, Subdomain `sparky`, 4 secrets auto-gen). All 3 containers healthy;
  server log shows clean migrations + `sparkyapp` role created + RLS applied, no crash loop, no uploads
  EACCES; `GET /api/health` through the public edge returns `{"status":"UP"}`; login/register page serves
  over a valid TLS cert at `https://sparky.demo-felhom.eu`.

## 2026-06-26 — crafty-controller: image bump 4.4.8→4.10.7 + publish Java port range + connection guidance
- **Image bump** `crafty-4:4.4.8` → `4.10.7` (latest stable; 4.10.8/4.11.0 don't exist in the registry).
  6 minor versions of fixes incl. security CVEs. **Java 25 verified present** in 4.10.7
  (`/usr/lib/jvm/java-25-openjdk-amd64`, default `java -version` = openjdk 25.0.3; 8/11/17/21 also
  available) — so the latest-Minecraft (`26.x`, needs Java 25) blocker is resolved. Healthcheck + Traefik
  https-backend labels unchanged (Crafty still serves HTTPS on 8443).
- **Published the Java game-port range** `25565-25575:25565-25575` (TCP, 11 ports = up to 11 Java
  servers; first server 25565, rest 25566–25575). No `network_mode: host` (would break Traefik routing).
  Bedrock UDP 19132 intentionally out of scope.
- **App-page guidance** (`.felhom.yml` first_steps + prerequisites): how to set the server port within
  25565–25575, how to connect on the LAN (manual IP:port — "scan for LAN" won't auto-list), and that
  internet access needs operator port-forwarding. (Static text — can't show the live LAN IP.)
- **Live-verified on guest 9201:** 4.10.7 healthy; public URL 302; the guest's bridged LAN IP
  `192.168.0.121` reaches the real Crafty "test" server on `25565` (TCP OPEN + Minecraft SLP handshake
  returns JSON status); `:25575` reachable, `:25600` closed (negative control). In-place upgrade preserved
  the admin, the operator's configured MFA, and the test server.
- **Correction (earlier draft was wrong):** an earlier note here claimed the upgrade "locked out the
  admin (TOTP)." That was a misdiagnosis — the `totp_data` row + recovery codes were **operator-configured
  MFA**, so the 401 on a password-only login was correct behaviour, NOT an upgrade bug. There is **no
  upgrade regression**; the bump preserves data and MFA correctly.

## 2026-06-26 — crafty-controller: seed a felhom-generated admin password (replaces Crafty's ugly random one)
- **crafty-controller**: instead of reading Crafty's auto-generated (long, symbol-laden) random admin
  password, we now **seed** a clean felhom-generated one — same pattern as gokapi, so initial passwords are
  consistent across the catalog.
  - Crafty's image ships `app/config_original/default.json = {"username":"admin","password":"crafty"}`;
    "crafty" is 6 chars < Crafty's 8-char minimum, so Crafty rejected it and generated a random password.
  - New `CRAFTY_PASSWORD` deploy field (`type: password`, `generate: password:24`, locked after deploy —
    mirrors gokapi's `GOKAPI_PASSWORD`). The compose **entrypoint** overwrites the `default.json` template
    with this password before the launcher runs; on fresh install Crafty creates the `admin` user with it.
  - `initial_credentials.file` repointed `default-creds.txt` → `default.json` (same json/username/password
    keys), so the controller's app-page "Kezdeti belépési adatok" card shows the **seeded** password — the
    customer sees the same value at deploy time and on the app page.
  - Catalog-only change (reuses felhom-controller v0.84.0's initial_credentials reader + the gokapi-style
    seed). Requires a fresh install to take effect (the seed is only read on first run).

## 2026-06-26 — crafty-controller: surface the auto-generated initial admin password on the app page
- **crafty-controller**: Crafty writes a random admin password to `/crafty/app/config/default-creds.txt`
  at first boot (its built-in default is rejected as "too short"). Customers had to read the container
  logs to find it. Added an `initial_credentials` block (new general felhom-controller v0.84.0 mechanism):
  `file` + `format: json` + `username_key`/`password_key` + a `note`. The controller reads the file live
  from the container and shows username + password (masked, reveal/copy) on the app's page under "Kezdeti
  belépési adatok". Requires felhom-controller ≥ v0.84.0.
- Updated `first_steps` to point at the app page for the initial login instead of "find it in the logs".

## 2026-06-26 — crafty-controller: Traefik https backend + scoped skip-verify (fixes 502)
- **crafty-controller**: the healthcheck fix un-withheld the Traefik route, exposing a pre-existing
  **502** — Traefik proxied `http://…:8443` to Crafty's **HTTPS-only** self-signed backend (Crafty serves
  no plain-HTTP panel; `:8000` only redirects). Added two service labels:
  - `loadbalancer.server.scheme=https` — Traefik now speaks HTTPS to the backend.
  - `loadbalancer.serverstransport=insecure-skip-verify@file` — references the **named**
    serversTransport defined in the controller-managed Traefik dynamic config (felhom-controller v0.83.0),
    which skips verifying Crafty's per-container self-signed cert. Verification stays ON for every other
    backend (scoped Option B; no global `insecureSkipVerify`). The `@file` suffix is the cross-provider
    reference from the docker provider to the file-provider transport.
  - Requires felhom-controller ≥ v0.83.0 (which renders the `insecure-skip-verify` transport). `port=8443`
    and the router/tls labels are unchanged.

## 2026-06-26 — crafty-controller healthcheck fix (curl-absent + http-vs-TLS probe)
- **crafty-controller**: container was permanently `unhealthy` → route withheld (`routeUnpublished`).
  Two independent healthcheck root causes, both fixed in one change:
  - **Docker healthcheck** ran `curl -fk https://localhost:8443`, but the `crafty-4:4.4.8` image has
    **no `curl` and no `wget`** (`exec: "curl": not found`, FailingStreak 150). Replaced with a
    dependency-free **python3 TLS-socket** liveness probe (`/usr/bin/python3` is present): completes a
    TLS handshake to `127.0.0.1:8443` (unverified context mirrors the old `-k`; Crafty's cert is
    self-signed). `start_period` 30s → 60s for cold-boot headroom (cert gen + migrations).
  - **Controller-side probe** (`.felhom.yml healthcheck.checks`) was `type: http` against Crafty's
    **TLS-only** 8443 → `probeHTTP` sent plaintext HTTP, got a TLS record → `HealthProbe.Healthy=false`,
    which `manager.go` re-applies to override Docker's verdict back to `unhealthy`. Changed `http` →
    `tcp` (`probeTCP` dial succeeds against a TLS listener). Both layers had to change together.
  - Live-validated on guest 9201 (`demo-felhom`): synced → recreated via the update path → Docker
    `State.Health: healthy` (ExitCode 0), `health_probe.healthy: true` (tcp :8443, 5ms), http-vs-TLS
    WARNs stopped, stable green 3+ min, Traefik now **publishes** the route (`crafty-controller@docker`).
  - **Known follow-up (separate, out of this fix's scope):** the public URL still returns **502** — a
    distinct pre-existing bug the un-withheld route exposed: Traefik proxies `http://…:8443` to Crafty's
    HTTPS-only backend. Needs a Traefik HTTPS-backend + self-signed `serversTransport`
    (`insecureSkipVerify`) in the controller-generated Traefik config — tracked separately.

## 2026-06-23 — gokapi: index redirect + admin username display
- **gokapi**: seed `RedirectUrl` repointed from Gokapi's GitHub default → `https://${SUBDOMAIN}.${DOMAIN}/admin`.
  Gokapi's bare root `/` redirects to `RedirectUrl`; the controller's "Megnyitás" link is always the bare
  subdomain root, so it was landing on Gokapi's GitHub instead of the app. Now `/` → `/admin` → login.
  Applied to the live demo (config.json edit + restart) and the seed (future deploys).
- **gokapi**: added `app_info.default_creds` ("Felhasználó: admin · jelszó a Beállítások oldalon") so the
  app-info page shows the initial admin user like other apps; fixed `first_steps` (no more setup wizard).

## 2026-06-23 — gokapi reproducible headless setup (fixes public "maintenance mode")
- **gokapi**: was stuck in "maintenance mode" on the public URL since first deploy — Gokapi's one-time
  `/setup` wizard was never completed, and (verified against the docs + the v1.9.6 binary) **no Gokapi
  version supports env-var headless setup** for admin credentials. Worse, the unconfigured `/setup` was
  publicly reachable = an unauthenticated admin-takeover window.
  - Fix: the compose `entrypoint` now seeds a `config.json` on first boot (admin user, this app's public
    URL `https://${SUBDOMAIN}.${DOMAIN}/`, local storage, Encryption Level 0 so it restarts without a
    prompt) with `Password`/`SaltAdmin`/`SaltFiles` cleared, then runs Gokapi's documented
    `--deployment-password` one-shot to set the **felhom-generated** admin password **before** the
    server starts serving. The admin account is claimed at first boot → `/setup` is never exposed.
  - `.felhom.yml`: new `GOKAPI_PASSWORD` deploy field (`type: password`, `generate: password:24`,
    shown to the customer, locked after deploy). Admin username is `admin`.
  - Seed is pinned to Gokapi **v1.9.6** (`ConfigVersion 21`) — re-capture the seed if the image is bumped.
  - Live-validated on guest 9201: fresh remove+redeploy → headless auto-config, public login works, no
    maintenance page, admin claimed at first boot (browser-verified login).

## 2026-06-22 — gitea healthcheck fix (unattended test campaign)
- **gitea**: healthcheck probe repointed `/api/v1/version` → `/api/healthz` (docker HC + controller
  `.felhom.yml` probe), `start_period` 30s → 90s.
  - Surfaced during the Phase-2 deploy sweep: a fresh gitea reported `unhealthy` because
    `/api/v1/version` returns 404 until the install wizard / INSTALL_LOCK completes, while the
    container was serving fine on :3000 (`/api/healthz` → 200). Same class as the komga fix.

## 2026-06-22 — komga healthcheck fix (unattended test campaign)
- **komga**: healthcheck probe repointed `/api/v1/actuator/health` → `/actuator/health`.
  - Root cause: komga's Spring Boot actuator endpoint is served unauthenticated at `/actuator/health`
    (HTTP 200), while everything under the `/api/v1` prefix is auth-gated — so the old probe got
    HTTP 401, `curl -f` exited 22, and the container reported `unhealthy` despite serving normally
    on :25600. Diagnosed live on guest 9201 (probe matrix: `/`, `/actuator/health`,
    `/api/v1/oauth2/providers`, `/login` all 200; `/api/v1/actuator/health` → 401).
  - The `gotson/komga:1.20.0` image ships `curl` (verified), so the probe tool is unchanged.