felhom.eu/REPORT.md

felhom.eu — task reports

Overwrite this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in hub/CHANGELOG.md.

---

REPORT — Resend key rotation + de-git (2026-06-29)

Task: rotate the exposed (send-scoped) Resend API key and remove it from git without breaking any consumer. Model: DECISION A (out-of-band kubectl + runbook) — operator-confirmed. Status: S1–S6 complete; STOPPED at the §13 gate before S7 (operator deletes the old key). No secret value appears in this repo, this report, or any changelog.

S2 — consumer map (live cluster felhom-system)

Consumer	Source of key (before)	Live?	After
hub	ConfigMap hub-config → resend_api_key (plaintext in git)	running	env RESEND_API_KEY ← Secret/resend-api; ConfigMap field now empty placeholder; hub 0.17.0
contact-mailer	Secret contact-mailer-config → RESEND_API_KEY (plaintext in git)	running	secretKeyRef repointed to Secret/resend-api; old Secret deleted
healthchecks	Secret healthchecks-config → EMAIL_HOST_PASSWORD (plaintext in git)	NOT deployed (no workload cluster-wide)	value blanked + comment; to be wired to resend-api if/when deployed
Gmail "Send mail as"	external SMTP password	n/a	operator-manual (S4) — pending

contact-mailer.yaml:16 held only a re_xxxx placeholder (not the real key). Both live consumers and the live cluster confirmed to carry the old key before the change (hash match).

What was created / changed / rolled

• Created Secret/resend-api (RESEND_API_KEY) imperatively from the out-of-band $RESEND_API on host 180, value piped via stdin into kubectl apply — never echoed, never written to a file, never committed. Confirmed to hold the new key (hash ≠ old).
• Hub code: cmd/hub/main.go — new RESEND_API_KEY env override (mirrors REGISTRY_TOKEN). Built + pushed felhom-hub:0.17.0; manifest bumped; ArgoCD-synced. Running pod confirmed to carry the new key.
• contact-mailer: secretKeyRef → resend-api; rolled. Old Secret/contact-mailer-config (held the old key) deleted from the cluster.
• Rolled: deploy/hub, deploy/contact-mailer — both Healthy, ArgoCD Synced.

De-git diff (paths only — no values)

• manifests/hub.yaml — ConfigMap resend_api_key: "<key>" → "" + comment; Deployment gained RESEND_API_KEY env from Secret/resend-api; image 0.16.0 → 0.17.0.
• manifests/felhom.secret.yaml — healthchecks-config EMAIL_HOST_PASSWORD: "<key>" → "" + comment; Secret/contact-mailer-config block (held the key) removed.
• manifests/contact-mailer.yaml — secretKeyRef name contact-mailer-config → resend-api; the setup-comment re_xxxx placeholder replaced with the runbook command.
• documentation/runbooks/secrets.md — new out-of-band secret runbook (create/rotate/verify).
• hub/cmd/hub/main.go, hub/CHANGELOG.md — env override + changelog.

Verification: grep -rIn 're_' manifests/ → none; grep -rIn 're_XZZenCJs' . (real key) → none.

S6 — verify (before deleting old key)

Check	Result
New key send-capable	✅ Resend POST /emails from host 180 → HTTP 200, id 79df5b65…
contact-mailer end-to-end	✅ form POST → HTTP 200; pod log [OK] Email sent: resend_id=c13d86ae… (new key via resend-api)
hub	✅ running pod env RESEND_API_KEY = new key (hash match) + uses the identical, now-proven Resend send path; Healthy. A literal /notify was not fired — it emails a real customer in Hungarian and no operator-owned target was readable without the DB. Can run a live hub send on request.
Gmail	⏳ operator-manual (S4) — update "Send mail as" SMTP password to the new key

(All test mail was sent to info@felhom.eu from the verified felhom.eu domain. Operator confirms inbox arrival.)

STOP — §13 gate (operator action required before S7)

Do not delete the old key in Resend (S7) until:

1. Gmail "Send mail as" (S4) is updated to the new key — otherwise S7 breaks Gmail send. Gmail is the one remaining consumer still on the old key.
2. You've confirmed the two test emails above (+ a Gmail test send) arrived.

Then delete the old key in Resend and re-verify one consumer still sends (proves nothing was left on the old key). After that, this can be marked fully closed.

S8 — history scrub (deferred)

The old key string is still in git history (manifests/hub.yaml, felhom.secret.yaml). Lower urgency — rotation already invalidates it once S7 runs. Recommend scheduling a git filter-repo/BFG pass (also scrubs the other still-committed secrets noted in the runbook). Not done in this run.

Backlog surfaced

• felhom.secret.yaml still commits other plaintext secrets (healthchecks-config SECRET_KEY / SUPERUSER_PASSWORD, umami-config, gitea-creds) — same hygiene gap, out of scope here; tracked in the secrets runbook.
• Secret/resend-api is out-of-band: on a cluster rebuild it must be re-created per the runbook.