security: rotate the published secrets and de-git felhom.secret.yaml (R-925, P1)

WHY .gitignore "was not working": it was working. git never consults
.gitignore for a file it ALREADY TRACKS. The rule `*secret*` matched fine --
proved by dropping an untracked copy in and watching check-ignore name
`.gitignore:3:*secret*`. The file had been tracked since feea0606, which is
ironically the commit that de-gitted the Resend key.

WHAT THE EXPOSED VALUE ACTUALLY WAS. Not an analytics password: the GITEA
ADMIN ACCOUNT PASSWORD (is_admin true; /api/v1/admin/users answered 200), in
a repo gitea.dooplex.hu serves anonymously to the internet. That is push
access to every repo -- including the one whose website/ is git-synced live
and whose scripts/ is published by tag to every new box installer (R-110).
Re-ranked P2 -> P1 on that measurement; my first ranking had only measured
the analytics blast radius.

Every committed value was still live. Nothing had ever been rotated.

ROTATED (values never echoed; written to a 0600 file on DooPlex):
  umami-config        APP_SECRET + POSTGRES_PASSWORD. The password was
                      changed INSIDE postgres (ALTER USER) as well as in the
                      Secret -- the env var is only read at first init, so
                      patching the Secret alone would have changed nothing.
  healthchecks-config SECRET_KEY + SUPERUSER_PASSWORD (nothing consumes them,
                      there is no healthchecks Deployment).
  gitea-creds         no longer holds the admin password at all: a SCOPED
                      token (read:package + read:repository).
  gitea admin         new random password; gitea-system/gitea-admin updated.

VERIFIED, not assumed:
  - new admin password -> 200, OLD PUBLISHED PASSWORD -> 401 (the leak is dead)
  - umami: a real beacon returns 200 (so the app authenticates to postgres and
    writes) while a bogus site id still returns 400 (so the 200 means something)
  - hub: "Registry version check: latest = 0.304.0" AND "Template fetched
    (5881 bytes)", no auth failures
  - BOTH token scopes are load-bearing, and the second was found by breaking
    it: a package-only token made the hub log "Template fetch: unexpected
    status 403", because the template fetcher reads a raw file out of the
    felhom-controller repo, not the registry.

AN INCIDENT CAUSED BY THE FIX, recorded because it is the useful part: the
rollout restart needed to pick up the new umami secret put umami into
CrashLoopBackOff and took stats.felhom.eu down (503) for ~4 minutes. Not the
rotation -- at memory 512Mi that pod runs for months but CANNOT RESTART:
startup (Prisma + Next.js) peaks over the limit and is OOMKilled (exit 137).
Raised to 1Gi IN THE MANIFEST, not just live, per .claude/rules/manifests.md
("never bare kubectl set -- the next sync reverts it and the fix silently
disappears").

THE GATE: KNOWN_BACKLOG is removed from manifest_bearer_gate.py, as its own
comment instructed. Red-proofed with a decoy: exit 1 with it, exit 0 without.
An exemption kept this visible for three months and changed nothing.

WHAT REMAINS (operator, and it is bigger than what was fixed): the same
password is still the admin password in ~12 other namespaces -- nextcloud,
paperless, bookstack (a DATABASE ROOT password), tandoor, calibre,
adventurelog, gokapi, qbittorrent, servarr, homepage. Rotating Gitea does not
touch them. Also owed: a kisfenyo Gitea token sits in plaintext in the local
homelab-manifests remote URL and was printed to a session transcript during
this investigation, so it should be replaced regardless (R-580's shape).

NOT a finding: homelab-manifests is private (404 anonymously) and does not
contain the password; ArgoCD's repo credential is a separate token and was
untouched by the rotation.

manifests/felhom.secret.yaml

@@ -1,51 +0,0 @@
-apiVersion: v1
-kind: Secret
-metadata:
-  name: healthchecks-config
-  namespace: felhom-system
-type: Opaque
-stringData:
-  # === REQUIRED: Generate a random key ===
-  # python3 -c "import secrets; print(secrets.token_urlsafe(50))"
-  SECRET_KEY: "jumZn0XOcO1oDs77siMCgfkg0S2JGLHUiKBAxGleUF0KodBg6SHj-mcLNPxt29Wb6pk"
-
-  # === REQUIRED: Superuser for first login ===
-  SUPERUSER_EMAIL: "admin@felhom.eu"
-  SUPERUSER_PASSWORD: "doodooP4ssWD001!"
-
-  # === REQUIRED: SMTP via Resend.com ===
-  EMAIL_HOST: "smtp.resend.com"
-  EMAIL_PORT: "587"
-  EMAIL_HOST_USER: "resend"
-  # Resend API key — NOT committed. Healthchecks is not currently deployed; when it is, wire its
-  # EMAIL_HOST_PASSWORD to the out-of-band Secret/resend-api (key RESEND_API_KEY) via secretKeyRef
-  # instead of inlining a value here. See documentation/runbooks/secrets.md.
-  EMAIL_HOST_PASSWORD: ""
-  EMAIL_USE_TLS: "True"
-  EMAIL_USE_VERIFICATION: "False"
-  DEFAULT_FROM_EMAIL: "monitoring@felhom.eu"
----
-# NOTE: the Resend API key (formerly Secret/contact-mailer-config RESEND_API_KEY) is no longer
-# committed. It lives in the out-of-band Secret/resend-api (key RESEND_API_KEY), created imperatively
-# per documentation/runbooks/secrets.md. Both the hub and contact-mailer now read from resend-api.
----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: umami-config
-  namespace: felhom-system
-type: Opaque
-stringData:
-  APP_SECRET: "65cee3c4826b2478bcd304d81d3f2193983544d159a7b89dbf2d528479927a86"
-  POSTGRES_PASSWORD: "a764dc950f1f065d8b6f5e402d6420dd"
----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: gitea-creds
-  namespace: felhom-system
-type: Opaque
-stringData:
-  username: "admin"
-  password: "doodooP4ssWD001!"
----

manifests/umami.yaml

@@ -208,12 +208,17 @@ spec:
               value: "1"
             - name: TZ
               value: "Europe/Budapest"
+          # 1Gi, NOT 512Mi (raised 2026-10-09). MEASURED: at 512Mi this pod runs fine for months but
+          # CANNOT RESTART — startup (Prisma migrate + Next.js) peaks above the limit, so the container
+          # reaches "Ready", is OOMKilled (exit 137) and enters CrashLoopBackOff. Found the hard way
+          # during the R-925 secret rotation: a `rollout restart` took stats.felhom.eu down (503) until
+          # the limit was raised. Steady state is well under this; the headroom is for startup only.
           resources:
             requests:
-              memory: "128Mi"
+              memory: "256Mi"
               cpu: "50m"
             limits:
-              memory: "512Mi"
+              memory: "1Gi"
               cpu: "500m"
           livenessProbe:
             httpGet: