Files
felhom.eu/REPORT-website-gitsync-2026-10-19.md
T
2026-10-19 12:02:31 +02:00

5.2 KiB
Raw Blame History

REPORT — the website keeps syncing during an internet outage (2026-10-19)

In plain words

  1. It worked. The website's copy helpers now read Gitea inside the cluster, with no internet hop, and they wait and try again instead of stopping. The new website pod runs with 0 restarts, and the site and the installer script both answer normally.
  2. Note for later: this ties the website to the DooPlex cluster. If it moves (for example to Hetzner), the address must go back to the public one — register row R-1164.

Part 1 — anonymous in-cluster fetch (before any commit)

Throwaway pod in a scratch namespace (ccscratch-gitsync, deleted after), no credentials: git ls-remote http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git main installer-v1.34.0

ref in-cluster public (anonymous)
refs/heads/main 5d3cb5c1… 5d3cb5c1… — equal
refs/tags/installer-v1.34.0 add58eec… (annotated tag object) add58eec… — equal; peels to 624bcdc3…

Evidence: audits/website-gitsync-incluster-2026-10-19/gitsync-part1-*.txt.

Part 2 — the change

Commit fd75003e (no Co-Authored-By, per the brief):

  • manifests/webpage.yaml: all four git-syncs --repo=http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git; the two sidecars --max-failures=-1 (not the --one-time inits); one comment above the website sidecar. Refs and nginx-config-rev unchanged.
  • scripts/hostinstall_gates.py gate 6: fails unless there are four --repo= values and all are identical. Existing ref checks unchanged.

Gates: python3 scripts/repo_gates.py → all felhom.eu gates OK; python3 scripts/hostinstall_gates.py → rc 0 (ok: manifest: all four git-syncs read the same repo (…)).

Part 4 — deploy and proof

Deploy: hard refresh showed only Deployment/felhom-webpage OutOfSync; synced that resource alone at fd75003e (the skill's Deployment-only sync). New pod felhom-webpage-869ff88bf7-fswbc ready at 09:42 UTC; the old pod terminated. ArgoCD: Synced, operation Succeeded. (The first rollout status returned at once — it ran before the new ReplicaSet existed; re-run after the sync it waited for the old replica and finished.)

Scenario A (gitsync-scenarioA.txt): restarts 0,0,0; both sidecars started with the in-cluster --repo and --max-failures=-1, then updated successfully (main → fd75003e, installer-v1.34.0 → 624bcdc3); both init containers updated successfully; https://felhom.eu/ 200, https://felhom.eu/scripts/felhom-host-install.sh 200, served SCRIPT_VERSION="1.34.0" = manifest --ref=installer-v1.34.0.

Scenarios B and C (gitsync-scenarioBC.txt): scratch namespace ccscratch-gitsync-bc (deleted after), git-sync:v4.4.0 --repo=http://192.0.2.1/x.git --period=10s:

  • B, with --max-failures=-1: error syncing repo, will retry (failCount:1), restarts 0.
  • C, without: too many failures, aborting, restarts 1.
  • The first read at 95 s showed nothing for either: one fetch to an address that never answers takes ~133 s to time out. The re-read at 09:48 recorded both results. Instead of 90 s, a later run of this test needs about 150 s.

Scenario D — gate red-proof: one --repo changed to the public address → gate 6 FAIL … the git-sync --repo values differ, rc 1; restored byte-identical → rc 0.

Part 3 — docs changed

  • documentation/architecture/01-topology-and-trust.md §7: a [DESIGN] paragraph — the website pulls website/ (main) and scripts/ (installer tag) from Gitea in-cluster over plain HTTP, sidecars retry forever, tied to the DooPlex cluster.
  • .claude/rules/manifests.md „The two git-syncs": the fact that all four containers read Gitea in-cluster, must name one repo (gate 6), and the sidecars carry --max-failures=-1.
  • scripts/CHANGELOG.md: one entry.

Register

Rows before 166, after 167. Closed R-1163 (finding + fix, with evidence). Opened R-1164 (P4, Hub & operator: move the website to Hetzner; then the git-syncs must use the public address again).

Observations

  • The 95-second wait in the brief is too short for an address that never answers (~133 s TCP timeout per fetch) — NOT-A-FINDING: recorded here as the correct wait for this test; nothing in the product depends on it.

  • The skill says a whole-app sync would push git's view over de-gitted Secrets; today only the webpage Deployment was OutOfSync — NOT-A-FINDING: the Deployment-only sync was used anyway, as the skill says.

  • Around 09:50–09:55 UTC, DooPlex itself briefly could not resolve names (Temporary failure in name resolution; curl https://felhom.eu/ → 000). By 09:56 DNS, the internet and the site (200) were back. The website pod kept restarts 0,0,0 and synced e2c35007 in-cluster at 09:54 with no error. NOT-A-FINDING: the same class of short outage that triggered this task; it is not proof that the fix worked (whether the blip reached the cluster is unknown).

  • CI: fd75003e job 2092 success, e2c35007 job 2094 success.

Teardown

Scratch namespaces ccscratch-gitsync and ccscratch-gitsync-bc deleted (kubectl get ns | grep -c ccscratch → 0). No hub, controller, agent, customer box, demo box or ep0 touched.