hub v0.18.0: app-email passthrough POST /api/v1/mail → Resend SMTP
Raw-MIME passthrough (STARTTLS, AUTH LOGIN) — separate from the notify HTTP-API alert path (which drops inline CID images). Per-customer token-bucket rate limit, From-header allowlist backstop. Resend key stays hub-side. No new external dep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -2,82 +2,49 @@
|
||||
|
||||
> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md).
|
||||
|
||||
---
|
||||
## App-email passthrough — hub leg (`POST /api/v1/mail` → Resend SMTP), hub v0.18.0
|
||||
|
||||
# REPORT — Resend key rotation + de-git (2026-06-29)
|
||||
**Task:** SMTP app-relay (apps → on-box shim → hub → Resend). Implements
|
||||
`documentation/audits/SPIKE-smtp-app-relay-2026-06-28.md` (verdict READY).
|
||||
|
||||
**Task:** rotate the exposed (send-scoped) Resend API key and remove it from git without breaking any
|
||||
consumer. **Model:** DECISION **A** (out-of-band `kubectl` + runbook) — operator-confirmed. **Status:**
|
||||
S1–S6 complete; **STOPPED at the §13 gate before S7** (operator deletes the old key). No secret value
|
||||
appears in this repo, this report, or any changelog.
|
||||
### Baseline (verified live)
|
||||
- Hub `main` @ `4b97855`, version **v0.17.0** (the Resend-key rotation already shipped earlier today 2026-06-29)
|
||||
→ target **v0.18.0**. (The prompt assumed v0.16.0→v0.17.0; the rotation took v0.17.0 first, so this leg is
|
||||
v0.18.0.) Prerequisite satisfied: Resend key is out-of-band in `Secret/resend-api`, read via `RESEND_API_KEY`.
|
||||
|
||||
## S2 — consumer map (live cluster `felhom-system`)
|
||||
### Files
|
||||
- **Created** `internal/mailrelay/relay.go` — `ResendSMTP` (`Sender`): STARTTLS to `smtp.resend.com:587`,
|
||||
`AUTH LOGIN resend/<key>` (small stdlib `net/smtp.Auth` LOGIN impl), raw `MAIL`/`RCPT`/`DATA` **passthrough**.
|
||||
`FromDomain` (From-header parser). **No new external dependency.**
|
||||
- **Created** `internal/api/mail.go` — `handleMail`: `checkAuthCustomer` → From-domain allowlist (403 backstop)
|
||||
→ per-customer token-bucket rate limit (429) → passthrough to Resend (200 / 502). `SetMailRelay` wiring +
|
||||
`mailRateLimiter`.
|
||||
- **Modified** `internal/api/handler.go` — sender/limiter/allowlist fields + `POST /api/v1/mail` route.
|
||||
- **Modified** `cmd/hub/main.go` — `MailConfig` (`per_customer_per_minute`, `from_domains`) + wire `ResendSMTP`
|
||||
when a key is present (else 503). The `notify/dispatcher.go` HTTP-API alert path is **untouched**.
|
||||
|
||||
| Consumer | Source of key (before) | Live? | After |
|
||||
|----------|------------------------|-------|-------|
|
||||
| `hub` | ConfigMap `hub-config` → `resend_api_key` (plaintext in git) | **running** | env `RESEND_API_KEY` ← `Secret/resend-api`; ConfigMap field now empty placeholder; hub `0.17.0` |
|
||||
| `contact-mailer` | Secret `contact-mailer-config` → `RESEND_API_KEY` (plaintext in git) | **running** | `secretKeyRef` repointed to `Secret/resend-api`; old Secret deleted |
|
||||
| `healthchecks` | Secret `healthchecks-config` → `EMAIL_HOST_PASSWORD` (plaintext in git) | **NOT deployed** (no workload cluster-wide) | value blanked + comment; to be wired to `resend-api` if/when deployed |
|
||||
| Gmail "Send mail as" | external SMTP password | n/a | **operator-manual (S4) — pending** |
|
||||
### Green gate (local)
|
||||
`go build ./... && go vet ./... && go test ./...` — **PASS** (6 packages ok, 0 failures).
|
||||
|
||||
`contact-mailer.yaml:16` held only a `re_xxxx` placeholder (not the real key). Both live consumers and
|
||||
the live cluster confirmed to carry the **old** key before the change (hash match).
|
||||
**Tests & §10 companion red-proofs**
|
||||
- **Passthrough byte-equality (§7 A / §10):** `TestMail_HappyPath_PassthroughRawBytes` — the `Sender` receives
|
||||
the raw bytes unchanged (not a parsed payload). PASS.
|
||||
- **From-reject + companion (§7 B / §10):** `TestMail_FromOutsideAllowlist_Rejected_NoSend` (403, sender never
|
||||
called) + `TestMail_FromReject_CompanionProof` (allowing the domain reaches the sender). PASS.
|
||||
- **Per-box rate limit + isolation + companion (§7 C / §10):** `TestMail_RateLimit_PerCustomer` (429 on the 2nd
|
||||
at 1/min; a different customer unaffected) + `TestMail_RateLimit_CompanionProof` (generous limit lets N+1
|
||||
through). PASS.
|
||||
- Send-failure→502, 401/503/400 paths, token-bucket unit (injected clock), LOGIN auth + From-domain parse. PASS.
|
||||
|
||||
## What was created / changed / rolled
|
||||
### Deployment & live validation
|
||||
- **Deploy:** build `felhom-hub:0.18.0` on 180 → bump `manifests/hub.yaml` image → ArgoCD sync (auto-sync off).
|
||||
The `mail` config is optional (defaults 30/min, `felhom.eu`); the Resend key is already injected via
|
||||
`Secret/resend-api` (`RESEND_API_KEY`), so the relay activates on deploy.
|
||||
- **End-to-end (app → shim → hub → Resend → real inbox):** result recorded here after the live run; method
|
||||
stated. The Resend key is supplied to the hub out-of-band — never on the guest, never committed.
|
||||
|
||||
- **Created** `Secret/resend-api` (`RESEND_API_KEY`) imperatively from the out-of-band `$RESEND_API` on
|
||||
host 180, value piped via stdin into `kubectl apply` — **never echoed, never written to a file, never
|
||||
committed**. Confirmed to hold the **new** key (hash ≠ old).
|
||||
- **Hub code:** `cmd/hub/main.go` — new `RESEND_API_KEY` env override (mirrors `REGISTRY_TOKEN`). Built +
|
||||
pushed `felhom-hub:0.17.0`; manifest bumped; ArgoCD-synced. Running pod confirmed to carry the new key.
|
||||
- **contact-mailer:** `secretKeyRef` → `resend-api`; rolled. Old `Secret/contact-mailer-config` (held the
|
||||
old key) **deleted** from the cluster.
|
||||
- **Rolled:** `deploy/hub`, `deploy/contact-mailer` — both `Healthy`, ArgoCD `Synced`.
|
||||
|
||||
## De-git diff (paths only — no values)
|
||||
|
||||
- `manifests/hub.yaml` — ConfigMap `resend_api_key: "<key>"` → `""` + comment; Deployment gained
|
||||
`RESEND_API_KEY` env from `Secret/resend-api`; image `0.16.0` → `0.17.0`.
|
||||
- `manifests/felhom.secret.yaml` — `healthchecks-config` `EMAIL_HOST_PASSWORD: "<key>"` → `""` + comment;
|
||||
`Secret/contact-mailer-config` block (held the key) **removed**.
|
||||
- `manifests/contact-mailer.yaml` — `secretKeyRef` name `contact-mailer-config` → `resend-api`; the
|
||||
setup-comment `re_xxxx` placeholder replaced with the runbook command.
|
||||
- `documentation/runbooks/secrets.md` — **new** out-of-band secret runbook (create/rotate/verify).
|
||||
- `hub/cmd/hub/main.go`, `hub/CHANGELOG.md` — env override + changelog.
|
||||
|
||||
**Verification:** `grep -rIn 're_' manifests/` → none; `grep -rIn 're_XZZenCJs' .` (real key) → none.
|
||||
|
||||
## S6 — verify (before deleting old key)
|
||||
|
||||
| Check | Result |
|
||||
|-------|--------|
|
||||
| New key send-capable | ✅ Resend `POST /emails` from host 180 → **HTTP 200**, id `79df5b65…` |
|
||||
| contact-mailer end-to-end | ✅ form POST → **HTTP 200**; pod log `[OK] Email sent: resend_id=c13d86ae…` (new key via `resend-api`) |
|
||||
| hub | ✅ **end-to-end** `/notify` (event_type=`test`) → log `Notification email sent to nagyfenyvesi.viktor@gmail.com`, `{"sent":true}`. Sent via `Secret/resend-api` (new key). Used a throwaway prefs row `_resend-rotation-test`, then **blanked it** (follow-up notify → `sent:false, no_email`). |
|
||||
| Gmail | ✅ **S4 done** — operator updated "Send mail as" SMTP password; confirmed test mail arrives. |
|
||||
|
||||
(contact-mailer/direct-key test mail went to `info@felhom.eu`; the hub test went to the operator's Gmail.
|
||||
Operator confirmed arrival of both.)
|
||||
|
||||
## STOP — §13 gate (only S7 remains — operator action)
|
||||
|
||||
All consumers (hub, contact-mailer, Gmail) verified sending on the **new** key. The one remaining,
|
||||
irreversible step is **operator-only**:
|
||||
|
||||
- **S7 — delete the OLD key in Resend** (dashboard → API Keys). Then re-verify one consumer still sends
|
||||
(e.g. submit the contact form again) to prove nothing was left on the old key.
|
||||
|
||||
After S7 + re-verify, the rotation is fully closed (only the S8 history-scrub remains, deferred).
|
||||
|
||||
## S8 — history scrub (deferred)
|
||||
|
||||
The old key string is still in git **history** (`manifests/hub.yaml`, `felhom.secret.yaml`). Lower urgency
|
||||
— rotation already invalidates it once S7 runs. Recommend scheduling a `git filter-repo`/BFG pass (also
|
||||
scrubs the other still-committed secrets noted in the runbook). **Not done in this run.**
|
||||
|
||||
## Backlog surfaced
|
||||
|
||||
- `felhom.secret.yaml` still commits other plaintext secrets (`healthchecks-config` `SECRET_KEY` /
|
||||
`SUPERUSER_PASSWORD`, `umami-config`, `gitea-creds`) — same hygiene gap, out of scope here; tracked in
|
||||
the secrets runbook.
|
||||
- `Secret/resend-api` is out-of-band: on a cluster rebuild it must be re-created per the runbook.
|
||||
### Observations
|
||||
- App-relay is a **separate** code path from the hub's own structured alerts (which keep using the Resend
|
||||
**HTTP API**) — raw passthrough is required because the API path silently drops inline CID images (spike §4).
|
||||
- v1: single-shot, no spool, no idempotency key. v2 would add accept-and-spool + `Resend-Idempotency-Key`.
|
||||
- Fleet free-tier ceiling is 100 emails/day. No secrets in any committed file.
|
||||
|
||||
Reference in New Issue
Block a user