docs: REPORT — v0.89.0 gap closure, calcom+nextcloud send-tested live

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

REPORT.md

@@ -1,105 +1,63 @@
-# REPORT — App-email SMTP relay (controller side), v0.88.0
+# REPORT — App-email gap closure: plaintext listener + split-From, v0.89.0
 
-**Task:** `TASK` — SMTP app-relay (apps → in-controller shim → hub → Resend). Implementation class.
-**Architecture (operator-confirmed):** **Shape 1** — the SMTP shim runs **in-process inside felhom-controller**,
-reusing the controller's existing hub client. (The alternative, a separate `felhom-smtp-shim` container, is the
-documented v2 path.)
+**Task:** Close the two relay gaps from `FINDING-app-email-rollout-2026-06-29.md` so opportunistic-STARTTLS
+clients (cal.com, nextcloud) can use the relay. Controller mechanism + two catalog mappings + live send-tests.
+**Locked posture:** plaintext on the single-tenant app Docker bridge is accepted (no TLS/CA work).
 
-## Confirmed baselines (verified against live repo)
-| Repo | `main` @ start | Live version | Target |
-|------|----------------|--------------|--------|
-| felhom-controller | `7cddb88` | v0.87.0 (CHANGELOG top v0.86.0; 0.87.0 was a version-only build) | **v0.88.0** |
-| felhom.eu (hub) | `4b97855` | **v0.17.0** (the Resend-key rotation already shipped 2026-06-29) | **v0.18.0** |
-| app-catalog | `5d42ca1` | unversioned | — |
+## Baselines (verified)
+- felhom-controller `main` @ `6692a2f`, **v0.88.0 → v0.89.0**. app-catalog touched. Hub untouched (N/A).
 
-> **Deviation from the prompt:** the prompt assumed the hub was at v0.16.0 → v0.17.0, but the prerequisite
-> Resend-key-rotation task already shipped as **v0.17.0** earlier today. So the hub leg targets **v0.18.0**.
-> The prerequisite is satisfied (key out-of-band in `Secret/resend-api`, hub reads `RESEND_API_KEY` env).
-
-## Files created / modified (controller)
-**Created**
-- `internal/mailrelay/policy.go` — From-header domain allowlist (validate-and-reject; fail-closed on parse error).
-- `internal/mailrelay/forward.go` — single-shot Bearer POST of raw MIME to hub `/api/v1/mail`.
-- `internal/mailrelay/server.go` — go-smtp server (two listeners, self-signed cert, accept-any AUTH incl. a
-  ~15-line LOGIN sasl server), `Data` = read raw → From-policy → forward → map hub status to SMTP reply.
-- `internal/mailrelay/lifecycle.go` — runtime start/stop so the global toggle applies without a restart.
-- `internal/mailrelay/mailrelay_test.go`, `internal/stacks/mailenv.go`, `internal/stacks/mailenv_test.go`.
-
-**Modified**
-- `internal/config/config.go` — `MailRelayConfig` (listeners, shim host, From allowlist, kill-switch) + defaults.
-- `internal/settings/settings.go` — global `AppEmail{Enabled,FromName}` toggle + accessors.
-- `internal/stacks/metadata.go` — `SMTPMapping` block parse + `HasSMTPMapping`.
-- `internal/stacks/deploy.go` — `AppConfig.EmailEnabled` (per-app toggle, persisted) + propagate in `SaveAppConfig`.
-- `internal/stacks/manager.go` — `stackEnv` injects the relay env (when global+per-app on, app has a mapping).
-- `internal/web/{server.go,handlers.go,tier2_config_handler.go}` — global toggle handler + route, per-app toggle
-  handler + route, page data.
-- `internal/web/templates/{settings.html,deploy.html}` — Hungarian "Alkalmazás-email" card + per-app toggle.
-- `cmd/controller/main.go` — wire the shim lifecycle (gated on hub configured + kill-switch), apply boot-time
-  toggle, close on shutdown, hand to the web server.
-- `go.mod`/`go.sum` — `github.com/emersion/go-smtp` v0.24.0 + `go-sasl`.
+## Files changed (controller)
+- `internal/mailrelay/server.go` — **Gap 1:** added a third listener `PlainNoTLSAddr` (default `:2526`),
+  plaintext, with `TLSConfig` left **nil** so go-smtp does NOT advertise STARTTLS. `:2525`/`:2465` unchanged.
+  `Start`/`Close`/addr-accessor updated for the third socket.
+- `internal/stacks/metadata.go` — **Gap 2:** `SMTPMapping` gains `TLSMode` (`yaml:"tls_mode"`) +
+  `FromDomainVar` (`yaml:"from_domain_var"`).
+- `internal/stacks/mailenv.go` — `smtpEnv` now picks the port via `shimPortForTLSMode` (`""`/`starttls`→2525;
+  `plaintext`→2526; `implicit-tls`→2465) instead of hardcoded 2525; split-From injection when `FromDomainVar`
+  is set (`FromVar=<local>` + `FromDomainVar=<domain>`), else the existing `<local>@<domain>`.
+- `internal/config/config.go` — `mail_relay.plain_no_tls_listen` (default `:2526`). `cmd/controller/main.go` —
+  pass `PlainNoTLSAddr` to the lifecycle Options.
 
 ## Green gate (local)
-`go build ./... && go vet ./... && go test ./...` — **PASS** (16 packages ok, 0 failures).
+`go build ./... && go vet ./... && go test ./...` — **PASS** (16 packages, 0 failures).
+- `TestSMTPEnv_PortByTLSMode` — plaintext→2526, starttls/empty→2525, implicit-tls→2465. **Companion**
+  `TestShimPortForTLSMode` — plaintext and starttls must map to DIFFERENT ports (fails if the switch is dropped).
+- `TestSMTPEnv_SplitFrom` — `from_domain_var` set → two keys (`MAIL_FROM_ADDRESS=nextcloud`+`MAIL_DOMAIN=felhom.eu`),
+  port 2526. **Companion** — a mapping without `from_domain_var` yields the single `vaultwarden@felhom.eu`.
+- `TestServer_PlainNoTLSListener_NoSTARTTLS` — `:2526` server has `TLSConfig==nil` & a real EHLO shows it does
+  **not** advertise STARTTLS while `:2525` does.
 
-### Tests & §10 companion red-proofs
-- **Passthrough not parse (§7 A / §10):** `TestData_HappyPath_ForwardsRawBytesUnchanged` asserts the forwarder
-  receives the **byte-equal** input (no parse/re-encode). PASS.
-- **From-reject before forward (§7 B / §10):** `TestData_FromOutsideAllowlist_Rejected_NoForward` — 550 +
-  **zero** forwarder calls. Companion `…CompanionProof_GateRemovedLetsItThrough` — allowing the domain lets the
-  same message reach the forwarder once (proves the gate is load-bearing). Both PASS.
-- **Single-shot on hub failure (§7 D / §10):** `TestData_HubUnreachable_SingleShot_Transient` — transient 4xx +
-  **exactly one** attempt (no retry loop). PASS.
-- **Status mapping (§6):** `TestMapStatusToSMTP` 200→nil, 429/4xx→451, 5xx→554. PASS.
-- **Toggles gate injection (§7 E / §8):** stacks `TestSMTPEnv_*` — both-on injects the full mapped env;
-  per-app-off / global-off / no-mapping inject nothing; Mealie-style mapping; household From-name override. PASS.
-- **Real socket (Q1/Q2):** `TestServer_EndToEnd_STARTTLS` drives a real `net/smtp` STARTTLS send through the
-  listener. PASS. Plus `TestLifecycle_StartStopIdempotent`.
+## Deployed v0.89.0 + listener
+Built `:0.89.0` on 180, deployed to guest 9201 (golden/bootstrap). `docker ps`: `…:0.89.0 Up (healthy)`. Shim
+started **three** listeners: `plaintext+STARTTLS on :2525`, `implicit-TLS on :2465`, and the new
+`plaintext-only (no STARTTLS) listener on :2526`.
 
-Test count: **+~17** controller test functions (mailrelay 9, stacks/mailenv 8); suite remains green.
-
-## Deployment & live validation — DONE (2026-06-29)
-- **Built** `felhom-controller:0.88.0` on 192.168.0.180; **deployed** to guest 9201 via the golden/bootstrap
-  mechanism (`docker pull` → `/etc/felhom-controller-image` → restart bootstrap). `docker ps`: `…:0.88.0 Up
-  (healthy)`. The controller container is **already on `traefik-public`** (and bridge), so apps resolve the
-  shim by `felhom-controller` — no network change needed.
-- **Global toggle via the REAL endpoint:** `POST /settings/app-email` (in-guest, the exact form the UI submits,
-  with session cookie + `_csrf`) → HTTP 200; controller logged `[mailrelay] app-email shim ON
-  (host=felhom-controller plain=[::]:2525 tls=[::]:2465)` and persisted `app_email.enabled=true` in
-  settings.json.
-- **End-to-end happy path (real pipeline, real Resend send):** a raw MIME message with an **inline CID image**,
-  From `vaultwarden@felhom.eu`, sent through the running shim from a throwaway container on `traefik-public`
-  (`app → shim → hub → Resend`). Result: `MAIL 250 / RCPT 250 / DATA 250 "OK: queued"`. Shim log:
-  `message relayed (from="vaultwarden@felhom.eu" rcpts=1 bytes=789 hub=200)`; hub log:
-  `/api/v1/mail: relayed for demo-felhom`. The DATA-250 is returned only after the hub→Resend leg accepted
-  (queued) the message. Recipient = the operator's own address; **inbox arrival is the operator's final
-  confirmation** (Resend returned 250/queued for the verified `felhom.eu` domain, which the spike established
-  delivers with inline image intact). The Resend key never touched the guest and is in no committed file.
-- **Security gate (live):** the same send From `evil@notfelhom.example` → `DATA 550 "5.7.1 sender address not
-  permitted"` and **no `/api/v1/mail` call reached the hub** (rejected before forward — scenario B confirmed live).
-- **Method note:** the relay pipeline (the new, previously-unproven code) is validated against the live Resend
-  account. The catalog `smtp_mapping` → per-app env injection is unit-tested (`TestSMTPEnv_*` incl. the
-  Vaultwarden mapping); deploying the Vaultwarden app and clicking its admin "send test email" would
-  additionally exercise that app's own SMTP client (already spike-proven) — recommended as the final manual
-  smoke once the catalog sync has the new templates.
-
-- **Inline CID image — confirmed rendering in the real client (Gmail):** a well-formed
-  `multipart/related; type="text/html"` message with a CID-referenced inline PNG, sent through the same
-  pipeline, **renders inline in the Gmail inbox** (Resend "Delivered", From `vaultwarden@felhom.eu`,
-  signed-by `felhom.eu`, TLS). NOTE: Resend's dashboard *preview pane* unreliably shows CID images as a broken
-  `attachment-0` even when the delivered mail renders correctly — validate against a real inbox, not the Resend
-  preview (as the spike did). An earlier synthetic test that used Python's `MIMEMultipart("related")` (which
-  omits the `type="text/html"` container param) was filed by Resend's parser as a loose attachment and broke —
-  that was a malformed *test message*, not a pipeline defect; real app mailers (e.g. Vaultwarden) emit a
-  well-formed related part and render correctly. No code change resulted.
-
-> **Demo box state:** global app-email left **ON** on guest 9201 (the validated working state). No customer apps
-> have the per-app toggle on, so nothing else changed.
+## Live send-tests
+- **nextcloud (Symfony Mailer) — Scenario B PROVEN:** toggle ON injected `SMTP_HOST=felhom-controller`,
+  `SMTP_PORT=2526`, `SMTP_SECURE=` (empty), and the **split From** `MAIL_FROM_ADDRESS=nextcloud` +
+  `MAIL_DOMAIN=felhom.eu`. **§8 env-consumption:** nextcloud applied it on redeploy (`occ config:system:get`
+  shows `mail_smtphost=felhom-controller mail_smtpport=2526 mail_smtpsecure= mail_from_address=nextcloud
+  mail_domain=felhom.eu`) — the official image reads these via `getenv()` every boot, **no occ step needed**. A
+  real password-reset relayed: shim `relayed (from="nextcloud@felhom.eu" rcpts=1 bytes=20430 hub=200)`. The
+  Symfony client sent **plaintext to :2526 and never attempted STARTTLS** → delivered. (Gap 1 + Gap 2 proven
+  end-to-end with a real opportunistic-STARTTLS client.)
+- **cal.com (Nodemailer) — Scenario A PROVEN:** toggle ON injected `EMAIL_SERVER_HOST=felhom-controller`,
+  `EMAIL_SERVER_PORT=2526`, `EMAIL_FROM=calcom@felhom.eu` (verified in-container). A real signup (HTTP 201
+  "Created user") triggered the verification email: shim `relayed (from="calcom@felhom.eu" rcpts=1 bytes=10260
+  hub=200)`. The Nodemailer client sent **plaintext to :2526 and never attempted STARTTLS** → delivered.
+  (calcom needed three pre-existing template fixes first — see Observations.)
+- **No regression (Scenario C):** gitea redeployed under v0.89.0 still injects `SMTP_PORT=2525` +
+  `smtp+starttls`; a real gitea forgot-password relayed `(from="gitea@felhom.eu" hub=200)` via the unchanged
+  :2525 STARTTLS path.
 
 ## Observations
-- **Mealie uses plaintext (NONE), not STARTTLS** — Mealie has no accept-invalid-cert option, so STARTTLS to the
-  self-signed shim would fail; the spike validated plaintext on :2525 (Docker-network only). Deliberate,
-  spike-faithful deviation from the prompt's "TLS for mealie" wording.
-- v1 is **single-shot, no spool** (per the Q6 decision). v2 = separate shim container + accept-and-spool retry +
-  `Resend-Idempotency-Key`.
-- Free-tier ceiling is **100 emails/day** across the fleet; further apps just need an `smtp_mapping` block.
-- No secrets are written to any committed file; the Resend key lives only hub-side (out-of-band).
+- **Three pre-existing cal.com template bugs** surfaced (never caught — the bad image pin meant calcom never
+  deployed): (1) image `v4.8.7` doesn't exist as a published image → `v4.6.9`; (2) missing `DATABASE_DIRECT_URL`
+  (Prisma `migrate deploy` fails → incomplete schema → 500s); (3) healthcheck path `/api/health` is 404 in
+  v4.x → `/api/auth/providers`. All fixed in the catalog.
+- **immich (Gap 3) remains unwired by decision** — it has no SMTP env (admin-UI/DB or `IMMICH_CONFIG_FILE`
+  JSON); the next email piece is a config-file render+mount mechanism.
+- **Other email-capable apps now wire trivially** as `tls_mode: plaintext` (ghost, outline, grafana, …) — no
+  per-app TLS determination needed under the accepted posture.
+- No secrets in any committed file; the Resend key stays hub-side.