docs: v0.83.0 REPORT/CONTEXT/README — scoped Traefik serversTransport for self-signed HTTPS backends
This commit is contained in:
+14
-1
@@ -7,7 +7,20 @@
|
||||
>
|
||||
> Ask Claude Code: "Please update CONTEXT.md with what we did today"
|
||||
|
||||
Last updated: 2026-06-22 (v0.76.0 — campaign-#3 hardening: settings recovery, restore-name validation, quiesce-marker quarantine)
|
||||
Last updated: 2026-06-26 (v0.83.0 — Traefik scoped serversTransport for self-signed HTTPS backends; crafty 502 fix)
|
||||
|
||||
> **2026-06-26 — v0.83.0 (deployed on 9201): scoped Traefik backend transport for self-signed HTTPS apps.**
|
||||
> Crafty is the **first/only catalog app with an HTTPS backend** (self-signed TLS on `:8443`, no plain-HTTP
|
||||
> port). The crafty healthcheck fix un-withheld its route, exposing a pre-existing 502 (Traefik proxied
|
||||
> HTTP to the HTTPS backend). Since traefik v3 forbids `insecureSkipVerify` via Docker labels, the
|
||||
> controller now renders a file-provider dynamic file `dynamic/serverstransports.yml` defining a **named**
|
||||
> `insecure-skip-verify` transport (`infra.RenderServersTransports` + `stacks.ensureServersTransports`,
|
||||
> written from `EnsureBaseStack` outside the traefik-running guard, write-if-changed, hot-loaded). Apps opt
|
||||
> in per-service via catalog labels `scheme=https` + `serverstransport=insecure-skip-verify@file`.
|
||||
> **No global `insecureSkipVerify`** — verification stays ON for every other backend (scoped Option B).
|
||||
> Pattern to reuse for any future self-signed HTTPS backend. Live-verified: `minecraft.demo-felhom.eu`
|
||||
> 502→302; filebrowser (HTTP) unaffected. Backlog: a `.felhom.yml backend_scheme` hint so the catalog
|
||||
> convention sets these labels instead of hand-adding.
|
||||
|
||||
> **2026-06-22 — v0.76.0 (deployed on 9201): three campaign-#3 hardening fixes.**
|
||||
> - **S1**: corrupt `settings.json` no longer crash-loops — `save()` writes a last-known-good `.bak`
|
||||
|
||||
@@ -1,62 +1,67 @@
|
||||
# REPORT — FileBrowser no-op recreate fix (F2) + drop restic binary (F1)
|
||||
# REPORT — Traefik scoped serversTransport for self-signed HTTPS backends (crafty 502 fix)
|
||||
|
||||
**Repo:** `felhom-controller` · **Version:** `v0.81.0` → **`v0.82.0`** · **Date:** 2026-06-24
|
||||
**Baseline:** `main` @ `036a6078b` (CHANGELOG top `v0.81.0`), trunk-based, no branches.
|
||||
**Repo:** `felhom-controller` · **Version:** `v0.82.0` → **`v0.83.0`** · **Date:** 2026-06-26
|
||||
**Commit:** `2d6df0f` · **Paired catalog commit:** `app-catalog-felhom.eu` `1216a8a`
|
||||
**Deployed:** guest 9201 (`demo-felhom`) on `felhom-pve`, golden/bootstrap mechanism.
|
||||
|
||||
Two findings from `TEST-REPORT-stable-path-sysdrive-restart-2026-06-23.md`, both validated at source.
|
||||
## Problem
|
||||
The crafty-controller healthcheck fix (catalog `68ce009`) un-withheld its Traefik route, exposing a
|
||||
pre-existing **502**: Traefik proxied **HTTP** to Crafty's **HTTPS-only** self-signed backend on `:8443`.
|
||||
Crafty is the only catalog app with an HTTPS backend (all others serve plain HTTP, so Traefik's default
|
||||
HTTP transport works for them). In Traefik v3 `insecureSkipVerify` cannot be set via Docker labels — it
|
||||
must live in static/file config — so the fix necessarily touched the controller (which generates the
|
||||
Traefik config), not just the catalog.
|
||||
|
||||
## F2 — gate the FileBrowser recreate on an actual change
|
||||
## Fix (Option B — scoped; verification stays ON by default)
|
||||
A **named** serversTransport, opted into per-service — no global `insecureSkipVerify`.
|
||||
|
||||
**Root cause:** `syncFileBrowserMounts` (`internal/web/handlers.go`) ran
|
||||
`docker compose up -d --force-recreate --remove-orphans` **unconditionally**. The existing
|
||||
`sourcesChanged` flag gated only the restore-mode DB reset (`down -v`); the force-recreate fired on
|
||||
every controller restart and every storage sync even when `config.yaml`/compose were byte-identical —
|
||||
bouncing the customer's file-access UI, contradicting the "Vezérlő újraindítása → apps keep running"
|
||||
promise (the 3.4 reproduction in the test report).
|
||||
- `internal/infra/infra.go` — new pure `RenderServersTransports()` + exported const
|
||||
`ServersTransportInsecure = "insecure-skip-verify"`. Emits:
|
||||
```yaml
|
||||
http:
|
||||
serversTransports:
|
||||
insecure-skip-verify:
|
||||
insecureSkipVerify: true
|
||||
```
|
||||
- `internal/stacks/infra.go` — new `ensureServersTransports(traefikDir)` writes
|
||||
`dynamic/serverstransports.yml` (0644) idempotently (write-only-on-change, like `wireController`, so the
|
||||
traefik file-watcher doesn't reload each self-heal tick). Called from `EnsureBaseStack` **outside**
|
||||
`ensureTraefik` (which early-returns when traefik is already running) so an established node still
|
||||
materializes the file on the next self-heal tick / restart; the file-provider `watch: true` hot-loads it
|
||||
(no traefik restart).
|
||||
- Catalog (`1216a8a`): crafty service gains `loadbalancer.server.scheme=https` +
|
||||
`loadbalancer.serverstransport=insecure-skip-verify@file`.
|
||||
- Tests: `TestServersTransports` (named transport + insecureSkipVerify + valid YAML) and the new render
|
||||
added to the YAML-parse / no-`:latest` matrix. `go test ./internal/infra/... ./internal/stacks/...` green.
|
||||
|
||||
**Fix:**
|
||||
- Capture `oldConfig`/`oldCompose` from disk **before** the writes.
|
||||
- Re-read `finalConfig`/`finalCompose` **after** the writes — so the integrations'
|
||||
`ReapplyConfigForTarget("filebrowser")` edits to `config.yaml` are included in the comparison.
|
||||
- New pure helper `fbNeedsRecreate(oldCfg, newCfg, oldCompose, newCompose) bool` (byte-equality on
|
||||
both files) drives the decision. `changed` → `up -d --force-recreate --remove-orphans`; otherwise a
|
||||
plain `up -d --remove-orphans` (ensures running, no bounce).
|
||||
- Preserved: the restore-mode DB reset stays gated on `sourcesChanged && resetDBOnChange`; when it runs
|
||||
it sets `changed = true` (a `down -v` removed the container, so it must be recreated).
|
||||
- First-ever run (no old files → empty bytes) differs from generated content → `changed = true` → creates it.
|
||||
## Deploy
|
||||
Commit → build+push `felhom-controller:0.83.0` on 192.168.0.180 → guest 9201 (`docker pull` →
|
||||
`/etc/felhom-controller-image` → restart bootstrap svc). On controller startup, `EnsureBaseStack` wrote
|
||||
`dynamic/serverstransports.yml` automatically (logged: `[infra] wrote backend transports → … (insecure-skip-verify)`)
|
||||
— no manual file drop needed. Catalog sync (`POST /api/sync`) pulled the two labels; crafty recreated via
|
||||
`POST /api/stacks/crafty-controller/update`.
|
||||
|
||||
**Files:** `internal/web/handlers.go` (helper + gate; added `bytes` import).
|
||||
## Verification (live, guest 9201)
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Dynamic file present + loaded | `/opt/docker/stacks/traefik/dynamic/serverstransports.yml` written, no traefik parse error |
|
||||
| **A — public URL** | `https://minecraft.demo-felhom.eu` → **302** (was 502), via Cloudflare edge + in-guest Traefik |
|
||||
| **A — Traefik backend** | access log: `302 … "crafty-controller@docker" "https://172.18.0.6:8443"` (before: `502 … "http://172.18.0.6:8443"`) |
|
||||
| **B — other apps unaffected** | filebrowser (HTTP backend) still **200** via Traefik — scoped, no global change |
|
||||
| **C — scoping proof** | **no** `insecureSkipVerify` in static `traefik.yml`; present ONLY in `dynamic/serverstransports.yml` |
|
||||
| crafty health | `Up (healthy)`, dashboard state `running` |
|
||||
|
||||
## F1 — drop the unused restic binary from the image
|
||||
## Deployed versions (`docker ps`)
|
||||
```
|
||||
felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.83.0 Up (healthy)
|
||||
crafty-controller registry.gitlab.com/crafty-controller/crafty-4:4.4.8 Up (healthy)
|
||||
```
|
||||
|
||||
`controller/Dockerfile`: removed the `restic \` apt line and its `# - restic: …` comment. Disk-tier
|
||||
restic moved to the host agent; no controller code execs the binary. Left **untouched** (still live in
|
||||
the dashboard/UI): `ResticSchedule` config, `migrateResticToRsync` settings migration, and the
|
||||
`Method`/backup-dir-name string references.
|
||||
## Verification method
|
||||
curl to the Cloudflare public edge + in-guest Traefik probe (`--resolve …:127.0.0.1`) + Traefik access
|
||||
logs. (No browser tool used this run.)
|
||||
|
||||
## Tests
|
||||
|
||||
- Added `TestFbNeedsRecreate` (`internal/web/filebrowser_gate_test.go`): unchanged → **false** (no
|
||||
recreate); config differs → **true**; compose differs → **true**; first run (no old files) → **true**.
|
||||
- **Red-proof:** hard-wiring `fbNeedsRecreate` to always return `true` (the old unconditional behaviour)
|
||||
makes the "unchanged → no recreate" case fail; restoring the byte-equality gate turns it green.
|
||||
- Web package top-level tests: +1 (added `TestFbNeedsRecreate`).
|
||||
- Green gate: `go build ./...` ✓ · `go vet ./...` ✓ · `go test ./...` ✓ (all packages ok).
|
||||
|
||||
## Deploy & verify (guest 9201 / felhom-pve)
|
||||
|
||||
Deployed via the golden bootstrap mechanism: `docker pull …:0.82.0` into the guest, set
|
||||
`/etc/felhom-controller-image` → `0.82.0`, `systemctl restart felhom-controller-bootstrap.service`.
|
||||
|
||||
- `docker ps` → `gitea.dooplex.hu/admin/felhom-controller:0.82.0 Up … (healthy)` ✓
|
||||
- **F1:** `docker exec felhom-controller command -v restic` → `NO-RESTIC` ✓
|
||||
- **F2 (no-op):** deploying 0.82.0 restarted the controller; its startup sync logged
|
||||
`FileBrowser sync — no config/compose change, ensured running without recreate (3 storage path(s))`,
|
||||
and FileBrowser `StartedAt` stayed `2026-06-23T18:15:47` (UNCHANGED — not bounced) ✓
|
||||
- **F2 (change path):** dirtied the on-disk `config.yaml` + restarted the controller → log
|
||||
`FileBrowser mounts synced (recreated) — 3 storage path(s), config updated`, FileBrowser `StartedAt`
|
||||
advanced to `2026-06-24T09:05:53` (recreated on a real change) ✓. The sync regenerated `config.yaml`
|
||||
back to canonical (test line gone), FileBrowser + controller healthy — state restored.
|
||||
|
||||
Method: invoked the exact server-side sync (controller restart → `syncFileBrowserMounts`) on the live
|
||||
guest; no browser tool needed for this check (it's container-lifecycle, observed via `docker inspect`).
|
||||
## Observation (carry, not acted on)
|
||||
Generalize a `backend_scheme`/`backend_tls` hint in `.felhom.yml` so future HTTPS-backend apps get the
|
||||
`scheme=https` + `serverstransport=…@file` labels via the catalog convention rather than hand-added —
|
||||
own task.
|
||||
|
||||
@@ -240,6 +240,8 @@ The controller stands up its own base stack — **traefik** (reverse proxy), **c
|
||||
|
||||
cloudflared is only deployed when a tunnel token is configured. **Triggers**: a first-boot goroutine (after stack init) and an unconditional call on every `system-health` tick (self-heal — cheap when healthy thanks to the idempotency). `monitor.EffectiveProtected` mirrors the cloudflared condition so a LAN-only node (no tunnel token) doesn't report a perpetual "protected container not running" FAIL.
|
||||
|
||||
**Backend transports — self-signed HTTPS backends (`ensureServersTransports` → `RenderServersTransports`, v0.83.0).** Traefik talks **HTTP** to app backends by default, which is correct for every catalog app that serves plain HTTP. The exception is an app that serves its **own self-signed TLS** on the internal docker bridge (the first is Crafty, HTTPS-only on `:8443`): Traefik must speak `https` to it *and* skip verifying a per-container self-signed cert (no CA to verify against; the hop never leaves the host). `insecureSkipVerify` is **not settable via Docker labels** in traefik v3 — it must live in static/file config — so `EnsureBaseStack` writes a file-provider dynamic file `dynamic/serverstransports.yml` defining a **named** transport `insecure-skip-verify` (write-if-changed; hot-loaded by the file watcher). An app **opts in per-service** via two catalog labels — `loadbalancer.server.scheme=https` + `loadbalancer.serverstransport=insecure-skip-verify@file` (the `@file` suffix is the cross-provider reference). Backend verification stays the default (ON) for every other service — there is deliberately **no global `insecureSkipVerify`** in `traefik.yml`. This write runs **outside** `ensureTraefik` (which early-returns when traefik is already up) so an established node still materializes the file on a self-heal tick.
|
||||
|
||||
> **Mount prerequisite (Section-G):** the controller writes these stacks under `/opt/docker/stacks` *inside its container*, but `docker compose up` runs on the **guest** Docker daemon. The golden's controller-bootstrap (`felhom-agent` `build-golden.sh`) therefore bind-mounts that path **same-path** (`-v /opt/docker/stacks:/opt/docker/stacks`) so the daemon resolves every relative bind source — without it, all bind-mounted stacks (base infra and customer apps) silently break.
|
||||
|
||||
**Controller routing + the wildcard cert anchor (`wireController` → `RenderControllerRoute`, v0.41.1 / v0.42.1).** filebrowser self-registers with traefik via Docker labels + `traefik-public` membership baked into its compose; the controller can't (it's started by the golden bootstrap *before* `traefik-public` exists, and the v2 `bootstrap.json` carries no domain — that comes from the hub pull). So `EnsureBaseStack` wires the controller **post-pull**: it `docker network connect traefik-public felhom-controller` and writes a traefik file-provider route `dynamic/controller.yml` (`Host(felhom.<domain>) → http://felhom-controller:8080`, write-if-changed). When DNS-01 ACME is configured, that route is **also the wildcard-cert anchor**: its router-level `tls.domains: *.<domain>` makes traefik **proactively obtain the wildcard `*.<domain>` + apex via Cloudflare DNS-01 at startup** (an entrypoint-level `http.tls.domains` does *not* trigger issuance in traefik v3 — only a router-level `tls.domains` does). Every other router then serves that one real wildcard cert by SNI — no per-app `certresolver` labels. This is what lets a LAN client reach the box directly at `*.<domain>` with the real cert (the `felhom-agent` split-horizon resolver depends on it).
|
||||
|
||||
Reference in New Issue
Block a user