WHY .gitignore "was not working": it was working. git never consults
.gitignore for a file it ALREADY TRACKS. The rule `*secret*` matched fine --
proved by dropping an untracked copy in and watching check-ignore name
`.gitignore:3:*secret*`. The file had been tracked since feea0606, which is
ironically the commit that de-gitted the Resend key.
WHAT THE EXPOSED VALUE ACTUALLY WAS. Not an analytics password: the GITEA
ADMIN ACCOUNT PASSWORD (is_admin true; /api/v1/admin/users answered 200), in
a repo gitea.dooplex.hu serves anonymously to the internet. That is push
access to every repo -- including the one whose website/ is git-synced live
and whose scripts/ is published by tag to every new box installer (R-110).
Re-ranked P2 -> P1 on that measurement; my first ranking had only measured
the analytics blast radius.
Every committed value was still live. Nothing had ever been rotated.
ROTATED (values never echoed; written to a 0600 file on DooPlex):
umami-config APP_SECRET + POSTGRES_PASSWORD. The password was
changed INSIDE postgres (ALTER USER) as well as in the
Secret -- the env var is only read at first init, so
patching the Secret alone would have changed nothing.
healthchecks-config SECRET_KEY + SUPERUSER_PASSWORD (nothing consumes them,
there is no healthchecks Deployment).
gitea-creds no longer holds the admin password at all: a SCOPED
token (read:package + read:repository).
gitea admin new random password; gitea-system/gitea-admin updated.
VERIFIED, not assumed:
- new admin password -> 200, OLD PUBLISHED PASSWORD -> 401 (the leak is dead)
- umami: a real beacon returns 200 (so the app authenticates to postgres and
writes) while a bogus site id still returns 400 (so the 200 means something)
- hub: "Registry version check: latest = 0.304.0" AND "Template fetched
(5881 bytes)", no auth failures
- BOTH token scopes are load-bearing, and the second was found by breaking
it: a package-only token made the hub log "Template fetch: unexpected
status 403", because the template fetcher reads a raw file out of the
felhom-controller repo, not the registry.
AN INCIDENT CAUSED BY THE FIX, recorded because it is the useful part: the
rollout restart needed to pick up the new umami secret put umami into
CrashLoopBackOff and took stats.felhom.eu down (503) for ~4 minutes. Not the
rotation -- at memory 512Mi that pod runs for months but CANNOT RESTART:
startup (Prisma + Next.js) peaks over the limit and is OOMKilled (exit 137).
Raised to 1Gi IN THE MANIFEST, not just live, per .claude/rules/manifests.md
("never bare kubectl set -- the next sync reverts it and the fix silently
disappears").
THE GATE: KNOWN_BACKLOG is removed from manifest_bearer_gate.py, as its own
comment instructed. Red-proofed with a decoy: exit 1 with it, exit 0 without.
An exemption kept this visible for three months and changed nothing.
WHAT REMAINS (operator, and it is bigger than what was fixed): the same
password is still the admin password in ~12 other namespaces -- nextcloud,
paperless, bookstack (a DATABASE ROOT password), tandoor, calibre,
adventurelog, gokapi, qbittorrent, servarr, homepage. Rotating Gitea does not
touch them. Also owed: a kisfenyo Gitea token sits in plaintext in the local
homelab-manifests remote URL and was printed to a session transcript during
this investigation, so it should be replaced regardless (R-580's shape).
NOT a finding: homelab-manifests is private (404 anonymously) and does not
contain the password; ArgoCD's repo credential is a separate token and was
untouched by the rotation.
Found during v0.118.0's own live proof, which is the only reason it was found:
I went to press "send test notification" for the English demo box and read
sendTestEmail first.
It had its own hardcoded Hungarian subject and body and never went through
FormatCustomerEmail, so it was the one customer mail v0.118.0 did not localise
— and it is the only customer mail an operator can trigger on demand, which
makes it the one most likely to be used to check whether the localisation
works. Pressing the button for an English household would have answered that
question wrongly, and convincingly.
The two sentences are extracted byte-for-byte into the bundle, so the Hungarian
test mail is unchanged. Red-proofed against the hardcoded version.
The general form worth keeping: the surface you would use to CHECK a feature is
the one most worth checking first. A broken instrument that reports success is
worse than a broken feature.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
admin
d