docs: REPORT + already-committed secrets runbook for Resend rotation

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

REPORT.md

@@ -2,68 +2,82 @@
 
 > **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md).
 
-## Website refresh — reflect the Proxmox-based platform (2026-06-28)
+---
 
-**Content/marketing task (not Go — no `go build/vet/test` applies).** Positioning **Option 3** (Proxmox =
-the honest foundation everywhere; Kubernetes/multi-node/rack relabelled "Üzleti / igény szerint").
+# REPORT — Resend key rotation + de-git (2026-06-29)
 
-- **Baseline:** felhom.eu `main` @ `6c285ba`.
-- **Files edited:** `website/technologiak.html`, `website/index.html`, `website/gyik.html`,
-  `website/biztonsagimentes.html`.
-- **Commit pushed:** `aa3d7c7` (deployed live via git-sync).
-- **No CHANGELOG** entry — the only CHANGELOG in-repo is `hub/CHANGELOG.md` (the Go app); there is no
-  website changelog. No version bump (the website is unversioned).
+**Task:** rotate the exposed (send-scoped) Resend API key and remove it from git without breaking any
+consumer. **Model:** DECISION **A** (out-of-band `kubectl` + runbook) — operator-confirmed. **Status:**
+S1–S6 complete; **STOPPED at the §13 gate before S7** (operator deletes the old key). No secret value
+appears in this repo, this report, or any changelog.
 
-### What changed
-- **technologiak.html:** removed the "A legtöbb otthoni felhasználónak nincs szüksége Proxmoxra" warning;
-  reframed the Proxmox section as **the foundation** ("Az alapréteg: Proxmox", "Az alap, nem opció",
-  "Mit ad neked a Proxmox alap?") with only grounded benefits (snapshot-before-change, whole-system
-  backup + restore to new hardware, isolation, fast/repeatable provisioning); advanced use-cases
-  demoted to a secondary "És ha többre van szükséged…" note. Rebuilt the stack diagram to **5 layers**
-  (Hardver → Proxmox VE → Izolált rendszerkörnyezet → Docker → Felhő Felügyelő + Alkalmazások),
-  dropping the old "OS — Linux (Ubuntu/Debian)" box. Reframed the controller section as **the
-  management layer on every box** ("A kezelőréteg: Felhő Felügyelő"), not a competing option.
-  Relabelled Kubernetes ("Üzleti és haladó igényekre, kérésre" + an Üzleti/igény-szerint callout, k3s
-  card softened) and the Rack/Multi-node hardware cards ("Üzleti / igény szerint"); Mini PC stays the
-  primary home config and "Kezdj kicsiben" stays.
-- **index.html:** Proxmox tile "Virtualizáció haladóknak" → **"Stabil virtualizációs alap"**; Kubernetes
-  tile "Magas rendelkezésre állás" → **"Üzleti szintű rendelkezésre állás"**.
-- **gyik.html:** rewrote "Milyen operációs rendszer fut a szerveren?" (Proxmox VE foundation + Docker in
-  an isolated environment); added a new "**Mi az a Proxmox, és miért használjátok?**" entry under
-  Technikai kérdések; added a snapshot clause to "Hogyan történik a szoftverfrissítés?". JSON-LD FAQ
-  schema updated to match (OS answer + new question + update clause) — validated as valid JSON
-  (41 questions).
-- **biztonsagimentes.html:** added one accurate line that the **whole system** (not just files/DBs) is
-  backup/restorable to a new machine (Proxmox-level, encrypted offsite). 3-2-1 story unchanged.
+## S2 — consumer map (live cluster `felhom-system`)
 
-### Scenario results (A–E) — all PASS, verified live in-browser (claude-in-chrome)
-- **A — Proxmox is the foundation, not optional:** PASS. Live `technologiak.html` shows no "don't need
-  Proxmox" copy; Proxmox is "Az alapréteg … Erre a stabil alapra építünk minden Felhom gépet" with the
-  grounded benefit list. `index.html` tile reads "Stabil virtualizációs alap".
-- **B — stack diagram shows the real layers:** PASS. Live diagram renders 5 bands
-  Felhő Felügyelő + Alkalmazások / Docker / Izolált rendszerkörnyezet / Proxmox VE / Hardver. Old
-  4-box OS-Linux diagram gone.
-- **C — FAQ OS answer correct:** PASS. Live `gyik.html` OS answer is the Proxmox VE one; the new
-  "Mi az a Proxmox" entry renders and expands with the three grounded benefits.
-- **D — Kubernetes/multi-node demoted:** PASS. Live: Kubernetes section + k3s card + Rack + Multi-node
-  cards + the index tile all read business/on-request; Mini PC + "Kezdj kicsiben" intact; nothing
-  deleted.
-- **E — nothing else breaks:** PASS. UTF-8 BOM preserved on all 4 files (`head -c3` = `ef bb bf`; `git
-  diff` first hunks start at lines 249/633/676/884 — line 1 untouched). `<div>` open/close balanced per
-  file; JSON-LD valid; nav/ToC/footer render; Hungarian correct, adult tone, no English/placeholder
-  leakage, no layout break.
+| Consumer | Source of key (before) | Live? | After |
+|----------|------------------------|-------|-------|
+| `hub` | ConfigMap `hub-config` → `resend_api_key` (plaintext in git) | **running** | env `RESEND_API_KEY` ← `Secret/resend-api`; ConfigMap field now empty placeholder; hub `0.17.0` |
+| `contact-mailer` | Secret `contact-mailer-config` → `RESEND_API_KEY` (plaintext in git) | **running** | `secretKeyRef` repointed to `Secret/resend-api`; old Secret deleted |
+| `healthchecks` | Secret `healthchecks-config` → `EMAIL_HOST_PASSWORD` (plaintext in git) | **NOT deployed** (no workload cluster-wide) | value blanked + comment; to be wired to `resend-api` if/when deployed |
+| Gmail "Send mail as" | external SMTP password | n/a | **operator-manual (S4) — pending** |
 
-### Verification method
-Live browser (claude-in-chrome) against the deployed site: loaded `felhom.eu/technologiak.html`,
-`/gyik.html`, `/index.html` after the git-sync deploy and read the rendered text. Source-level checks
-(BOM bytes, git-diff hunk offsets, div balance, JSON-LD parse, no-leftover-old-framing grep) run locally.
+`contact-mailer.yaml:16` held only a `re_xxxx` placeholder (not the real key). Both live consumers and
+the live cluster confirmed to carry the **old** key before the change (hash match).
 
-### Observations (NOT acted on — out of scope)
-- **"45+ alkalmazás" claim:** the live `alkalmazasok.html` has ~48 app cards (`app-title-area` count),
-  so "45+" / "több mint 45" is accurate (conservative). No change needed.
-- **Tailscale VPN** still named in the `technologiak.html` Hálózat section, while the roadmap is
-  self-hosted Headscale/WireGuard — left as-is per the task's AUTHOR NOTES (align when the Headscale
-  spike lands).
-- **gyik.html "Hardver & Telepítés" step 3** still says "Feltelepítjük az operációs rendszert, a
-  konténerkezelőt…" — generic and accurate, no Proxmox-optional implication; left unchanged (minimal-edit
-  rule).
+## What was created / changed / rolled
+
+- **Created** `Secret/resend-api` (`RESEND_API_KEY`) imperatively from the out-of-band `$RESEND_API` on
+  host 180, value piped via stdin into `kubectl apply` — **never echoed, never written to a file, never
+  committed**. Confirmed to hold the **new** key (hash ≠ old).
+- **Hub code:** `cmd/hub/main.go` — new `RESEND_API_KEY` env override (mirrors `REGISTRY_TOKEN`). Built +
+  pushed `felhom-hub:0.17.0`; manifest bumped; ArgoCD-synced. Running pod confirmed to carry the new key.
+- **contact-mailer:** `secretKeyRef` → `resend-api`; rolled. Old `Secret/contact-mailer-config` (held the
+  old key) **deleted** from the cluster.
+- **Rolled:** `deploy/hub`, `deploy/contact-mailer` — both `Healthy`, ArgoCD `Synced`.
+
+## De-git diff (paths only — no values)
+
+- `manifests/hub.yaml` — ConfigMap `resend_api_key: "<key>"` → `""` + comment; Deployment gained
+  `RESEND_API_KEY` env from `Secret/resend-api`; image `0.16.0` → `0.17.0`.
+- `manifests/felhom.secret.yaml` — `healthchecks-config` `EMAIL_HOST_PASSWORD: "<key>"` → `""` + comment;
+  `Secret/contact-mailer-config` block (held the key) **removed**.
+- `manifests/contact-mailer.yaml` — `secretKeyRef` name `contact-mailer-config` → `resend-api`; the
+  setup-comment `re_xxxx` placeholder replaced with the runbook command.
+- `documentation/runbooks/secrets.md` — **new** out-of-band secret runbook (create/rotate/verify).
+- `hub/cmd/hub/main.go`, `hub/CHANGELOG.md` — env override + changelog.
+
+**Verification:** `grep -rIn 're_' manifests/` → none; `grep -rIn 're_XZZenCJs' .` (real key) → none.
+
+## S6 — verify (before deleting old key)
+
+| Check | Result |
+|-------|--------|
+| New key send-capable | ✅ Resend `POST /emails` from host 180 → **HTTP 200**, id `79df5b65…` |
+| contact-mailer end-to-end | ✅ form POST → **HTTP 200**; pod log `[OK] Email sent: resend_id=c13d86ae…` (new key via `resend-api`) |
+| hub | ✅ running pod env `RESEND_API_KEY` = **new** key (hash match) + uses the identical, now-proven Resend send path; `Healthy`. A literal `/notify` was **not** fired — it emails a real customer in Hungarian and no operator-owned target was readable without the DB. Can run a live hub send on request. |
+| Gmail | ⏳ **operator-manual (S4)** — update "Send mail as" SMTP password to the new key |
+
+(All test mail was sent to `info@felhom.eu` from the verified `felhom.eu` domain. Operator confirms inbox
+arrival.)
+
+## STOP — §13 gate (operator action required before S7)
+
+**Do not delete the old key in Resend (S7) until:**
+1. **Gmail "Send mail as" (S4)** is updated to the new key — otherwise S7 breaks Gmail send. Gmail is the
+   one remaining consumer still on the old key.
+2. You've confirmed the two test emails above (+ a Gmail test send) arrived.
+
+Then delete the old key in Resend and re-verify one consumer still sends (proves nothing was left on the
+old key). After that, this can be marked fully closed.
+
+## S8 — history scrub (deferred)
+
+The old key string is still in git **history** (`manifests/hub.yaml`, `felhom.secret.yaml`). Lower urgency
+— rotation already invalidates it once S7 runs. Recommend scheduling a `git filter-repo`/BFG pass (also
+scrubs the other still-committed secrets noted in the runbook). **Not done in this run.**
+
+## Backlog surfaced
+
+- `felhom.secret.yaml` still commits other plaintext secrets (`healthchecks-config` `SECRET_KEY` /
+  `SUPERUSER_PASSWORD`, `umami-config`, `gitea-creds`) — same hygiene gap, out of scope here; tracked in
+  the secrets runbook.
+- `Secret/resend-api` is out-of-band: on a cluster rebuild it must be re-created per the runbook.