hub v0.18.0: app-email passthrough POST /api/v1/mail → Resend SMTP

Raw-MIME passthrough (STARTTLS, AUTH LOGIN) — separate from the notify HTTP-API
alert path (which drops inline CID images). Per-customer token-bucket rate limit,
From-header allowlist backstop. Resend key stays hub-side. No new external dep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-29 08:45:22 +02:00
parent 4b97855cdd
commit fa3c4f2657
8 changed files with 695 additions and 72 deletions
+39 -72
View File
@@ -2,82 +2,49 @@
> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md).
---
## App-email passthrough — hub leg (`POST /api/v1/mail` → Resend SMTP), hub v0.18.0
# REPORT — Resend key rotation + de-git (2026-06-29)
**Task:** SMTP app-relay (apps → on-box shim → hub → Resend). Implements
`documentation/audits/SPIKE-smtp-app-relay-2026-06-28.md` (verdict READY).
**Task:** rotate the exposed (send-scoped) Resend API key and remove it from git without breaking any
consumer. **Model:** DECISION **A** (out-of-band `kubectl` + runbook) — operator-confirmed. **Status:**
S1S6 complete; **STOPPED at the §13 gate before S7** (operator deletes the old key). No secret value
appears in this repo, this report, or any changelog.
### Baseline (verified live)
- Hub `main` @ `4b97855`, version **v0.17.0** (the Resend-key rotation already shipped earlier today 2026-06-29)
→ target **v0.18.0**. (The prompt assumed v0.16.0→v0.17.0; the rotation took v0.17.0 first, so this leg is
v0.18.0.) Prerequisite satisfied: Resend key is out-of-band in `Secret/resend-api`, read via `RESEND_API_KEY`.
## S2 — consumer map (live cluster `felhom-system`)
### Files
- **Created** `internal/mailrelay/relay.go``ResendSMTP` (`Sender`): STARTTLS to `smtp.resend.com:587`,
`AUTH LOGIN resend/<key>` (small stdlib `net/smtp.Auth` LOGIN impl), raw `MAIL`/`RCPT`/`DATA` **passthrough**.
`FromDomain` (From-header parser). **No new external dependency.**
- **Created** `internal/api/mail.go``handleMail`: `checkAuthCustomer` → From-domain allowlist (403 backstop)
→ per-customer token-bucket rate limit (429) → passthrough to Resend (200 / 502). `SetMailRelay` wiring +
`mailRateLimiter`.
- **Modified** `internal/api/handler.go` — sender/limiter/allowlist fields + `POST /api/v1/mail` route.
- **Modified** `cmd/hub/main.go``MailConfig` (`per_customer_per_minute`, `from_domains`) + wire `ResendSMTP`
when a key is present (else 503). The `notify/dispatcher.go` HTTP-API alert path is **untouched**.
| Consumer | Source of key (before) | Live? | After |
|----------|------------------------|-------|-------|
| `hub` | ConfigMap `hub-config``resend_api_key` (plaintext in git) | **running** | env `RESEND_API_KEY``Secret/resend-api`; ConfigMap field now empty placeholder; hub `0.17.0` |
| `contact-mailer` | Secret `contact-mailer-config``RESEND_API_KEY` (plaintext in git) | **running** | `secretKeyRef` repointed to `Secret/resend-api`; old Secret deleted |
| `healthchecks` | Secret `healthchecks-config``EMAIL_HOST_PASSWORD` (plaintext in git) | **NOT deployed** (no workload cluster-wide) | value blanked + comment; to be wired to `resend-api` if/when deployed |
| Gmail "Send mail as" | external SMTP password | n/a | **operator-manual (S4) — pending** |
### Green gate (local)
`go build ./... && go vet ./... && go test ./...`**PASS** (6 packages ok, 0 failures).
`contact-mailer.yaml:16` held only a `re_xxxx` placeholder (not the real key). Both live consumers and
the live cluster confirmed to carry the **old** key before the change (hash match).
**Tests & §10 companion red-proofs**
- **Passthrough byte-equality (§7 A / §10):** `TestMail_HappyPath_PassthroughRawBytes` — the `Sender` receives
the raw bytes unchanged (not a parsed payload). PASS.
- **From-reject + companion (§7 B / §10):** `TestMail_FromOutsideAllowlist_Rejected_NoSend` (403, sender never
called) + `TestMail_FromReject_CompanionProof` (allowing the domain reaches the sender). PASS.
- **Per-box rate limit + isolation + companion (§7 C / §10):** `TestMail_RateLimit_PerCustomer` (429 on the 2nd
at 1/min; a different customer unaffected) + `TestMail_RateLimit_CompanionProof` (generous limit lets N+1
through). PASS.
- Send-failure→502, 401/503/400 paths, token-bucket unit (injected clock), LOGIN auth + From-domain parse. PASS.
## What was created / changed / rolled
### Deployment & live validation
- **Deploy:** build `felhom-hub:0.18.0` on 180 → bump `manifests/hub.yaml` image → ArgoCD sync (auto-sync off).
The `mail` config is optional (defaults 30/min, `felhom.eu`); the Resend key is already injected via
`Secret/resend-api` (`RESEND_API_KEY`), so the relay activates on deploy.
- **End-to-end (app → shim → hub → Resend → real inbox):** result recorded here after the live run; method
stated. The Resend key is supplied to the hub out-of-band — never on the guest, never committed.
- **Created** `Secret/resend-api` (`RESEND_API_KEY`) imperatively from the out-of-band `$RESEND_API` on
host 180, value piped via stdin into `kubectl apply` — **never echoed, never written to a file, never
committed**. Confirmed to hold the **new** key (hash ≠ old).
- **Hub code:** `cmd/hub/main.go` — new `RESEND_API_KEY` env override (mirrors `REGISTRY_TOKEN`). Built +
pushed `felhom-hub:0.17.0`; manifest bumped; ArgoCD-synced. Running pod confirmed to carry the new key.
- **contact-mailer:** `secretKeyRef``resend-api`; rolled. Old `Secret/contact-mailer-config` (held the
old key) **deleted** from the cluster.
- **Rolled:** `deploy/hub`, `deploy/contact-mailer` — both `Healthy`, ArgoCD `Synced`.
## De-git diff (paths only — no values)
- `manifests/hub.yaml` — ConfigMap `resend_api_key: "<key>"``""` + comment; Deployment gained
`RESEND_API_KEY` env from `Secret/resend-api`; image `0.16.0``0.17.0`.
- `manifests/felhom.secret.yaml``healthchecks-config` `EMAIL_HOST_PASSWORD: "<key>"``""` + comment;
`Secret/contact-mailer-config` block (held the key) **removed**.
- `manifests/contact-mailer.yaml``secretKeyRef` name `contact-mailer-config``resend-api`; the
setup-comment `re_xxxx` placeholder replaced with the runbook command.
- `documentation/runbooks/secrets.md`**new** out-of-band secret runbook (create/rotate/verify).
- `hub/cmd/hub/main.go`, `hub/CHANGELOG.md` — env override + changelog.
**Verification:** `grep -rIn 're_' manifests/` → none; `grep -rIn 're_XZZenCJs' .` (real key) → none.
## S6 — verify (before deleting old key)
| Check | Result |
|-------|--------|
| New key send-capable | ✅ Resend `POST /emails` from host 180 → **HTTP 200**, id `79df5b65…` |
| contact-mailer end-to-end | ✅ form POST → **HTTP 200**; pod log `[OK] Email sent: resend_id=c13d86ae…` (new key via `resend-api`) |
| hub | ✅ **end-to-end** `/notify` (event_type=`test`) → log `Notification email sent to nagyfenyvesi.viktor@gmail.com`, `{"sent":true}`. Sent via `Secret/resend-api` (new key). Used a throwaway prefs row `_resend-rotation-test`, then **blanked it** (follow-up notify → `sent:false, no_email`). |
| Gmail | ✅ **S4 done** — operator updated "Send mail as" SMTP password; confirmed test mail arrives. |
(contact-mailer/direct-key test mail went to `info@felhom.eu`; the hub test went to the operator's Gmail.
Operator confirmed arrival of both.)
## STOP — §13 gate (only S7 remains — operator action)
All consumers (hub, contact-mailer, Gmail) verified sending on the **new** key. The one remaining,
irreversible step is **operator-only**:
- **S7 — delete the OLD key in Resend** (dashboard → API Keys). Then re-verify one consumer still sends
(e.g. submit the contact form again) to prove nothing was left on the old key.
After S7 + re-verify, the rotation is fully closed (only the S8 history-scrub remains, deferred).
## S8 — history scrub (deferred)
The old key string is still in git **history** (`manifests/hub.yaml`, `felhom.secret.yaml`). Lower urgency
— rotation already invalidates it once S7 runs. Recommend scheduling a `git filter-repo`/BFG pass (also
scrubs the other still-committed secrets noted in the runbook). **Not done in this run.**
## Backlog surfaced
- `felhom.secret.yaml` still commits other plaintext secrets (`healthchecks-config` `SECRET_KEY` /
`SUPERUSER_PASSWORD`, `umami-config`, `gitea-creds`) — same hygiene gap, out of scope here; tracked in
the secrets runbook.
- `Secret/resend-api` is out-of-band: on a cluster rebuild it must be re-created per the runbook.
### Observations
- App-relay is a **separate** code path from the hub's own structured alerts (which keep using the Resend
**HTTP API**) — raw passthrough is required because the API path silently drops inline CID images (spike §4).
- v1: single-shot, no spool, no idempotency key. v2 would add accept-and-spool + `Resend-Idempotency-Key`.
- Fleet free-tier ceiling is 100 emails/day. No secrets in any committed file.