diff --git a/CHANGELOG.md b/CHANGELOG.md index ba07b46..3fe514c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,17 @@ +## v0.292.0 — cloudflared says whether the tunnel is CONNECTED (R-841) (2026-10-04) + +**MinAgent: 0.131.0** (unchanged). No new household string. Agent v0.141.0 reads the result; an older agent ignores it. + +- **R-841.** The cloudflared stack now runs `tunnel --metrics localhost:20241 run` and has a Docker health check, + `cloudflared tunnel --metrics localhost:20241 ready` (every 30 s, 3 retries, 30 s start period). `/ready` answers + 200 only with at least one connection to Cloudflare. Measured 2026-10-04: with a wrong token the container stays + `running` while `/ready` answers 503 — the container state alone said "up" for a dead tunnel. The host agent reads + `State.Health` through its existing `docker inspect` sudoers line and reports `running` / `not_running` / `unknown`. +- Delivery: the infra reconcile (v0.286.0) sees the changed compose and recreates cloudflared once on upgrade; the + tunnel reconnects in seconds. +- Tests: `TestRenderCloudflared_HasAReadinessHealthcheck`; `TestCloudflaredRender` expects the new command. + Red-proof: `felhom.eu/documentation/audits/os-host-lane-2026-10-04/partA/controller-redproof.txt`. + ## v0.291.0 — a returning household's first night makes an off-site copy (decision 78, R-726); the built-in images raised and a release now moves them (R-838) (2026-10-04) **MinAgent: 0.131.0** (unchanged). No new household string. diff --git a/REPORT.md b/REPORT.md index 62f524a..f0cd557 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,8 +1,9 @@ -# REPORT — v0.291.0: decision 78 (R-726) and the infrastructure images (R-838) — 2026-10-04 +# REPORT — v0.292.0: cloudflared readiness health check (R-841) — 2026-10-04 -Full session report: `felhom.eu/REPORT-os-guest-lane-2026-10-04.md`. +Full session report: `felhom.eu/REPORT-os-host-lane-2026-10-04.md`. -- **R-726 (decision 78):** a claimed box that never made an off-site copy sets an orphaned old copy aside on its first - night and starts a new one; nothing deleted. Red-proved both ways. -- **R-838:** traefik v3.7.13, cloudflared 2026.9.3, filebrowser 1.5.6-stable; a running file browser is now moved by a - release (only its image line changes). `scripts/check-infra-pins.py` for the monthly re-test. +- The cloudflared compose gains a Docker health check that asks cloudflared itself whether the tunnel is connected + (`/ready` on a fixed loopback metrics port). The host agent (v0.141.0) reads it and reports the tunnel as running, + not running or unknown; the hub (v0.131.0) alarms after two not-running reports. +- Tests: render test pins the health check and the command; red-proof recorded in the audit folder. +- Live: see the session report (both demo boxes, cloudflared recreated once, state `healthy`). diff --git a/controller/README.md b/controller/README.md index 34380ab..f381c88 100644 --- a/controller/README.md +++ b/controller/README.md @@ -983,7 +983,7 @@ The controller stands up its own base stack — **traefik** (reverse proxy), **c `Manager.EnsureBaseStack()` creates the `traefik-public` network and the `felhom-tunnel` network, writes the forwarded-header clean-up, then deploys traefik → cloudflared → filebrowser under `${stacks_dir}/`. It is: - **single-flight** (a `TryLock` guard — it's called from both first boot and every health tick, so overlapping runs must not race on the same stack dir), -- **idempotent, and it reconciles** — filebrowser is skipped when running (never overwriting its compose, preserving the storage mounts `SyncFileBrowserMounts` manages); **traefik and cloudflared are rewritten and recreated when their rendered files differ from the ones on disk** (v0.286.0 — a release that changes their template reaches every installed box; equal files → nothing). A traefik rewrite that would drop the running certificate resolver is refused and logged. +- **idempotent, and it reconciles** — filebrowser is skipped when running (never overwriting its compose, preserving the storage mounts `SyncFileBrowserMounts` manages); **traefik and cloudflared are rewritten and recreated when their rendered files differ from the ones on disk** (v0.286.0 — a release that changes their template reaches every installed box; equal files → nothing). cloudflared carries a Docker health check on its own `/ready` endpoint (v0.292.0, R-841): `healthy` only while the tunnel has a connection — the host agent reports that state to the hub. A traefik rewrite that would drop the running certificate resolver is refused and logged. - **non-fatal** (logs, never crashes the controller). cloudflared is only deployed when a tunnel token is configured. **Triggers**: a first-boot goroutine (after stack init) and an unconditional call on every `system-health` tick (self-heal — cheap when healthy thanks to the idempotency). `monitor.EffectiveProtected` mirrors the cloudflared condition so a LAN-only node (no tunnel token) doesn't report a perpetual "protected container not running" FAIL. diff --git a/controller/internal/infra/infra_test.go b/controller/internal/infra/infra_test.go index 547246f..e643d06 100644 --- a/controller/internal/infra/infra_test.go +++ b/controller/internal/infra/infra_test.go @@ -178,8 +178,8 @@ func TestCloudflaredRender(t *testing.T) { if !strings.Contains(compose, CloudflaredImage) { t.Errorf("expected pinned cloudflared image %q", CloudflaredImage) } - if !strings.Contains(compose, "command: tunnel run") { - t.Error("expected `command: tunnel run`") + if !strings.Contains(compose, "command: tunnel --metrics localhost:20241 run") { // R-841: fixed metrics address + t.Error("expected `command: tunnel --metrics localhost:20241 run`") } } diff --git a/controller/internal/infra/templates/cloudflared-compose.yml.tmpl b/controller/internal/infra/templates/cloudflared-compose.yml.tmpl index 97c05e2..682875d 100644 --- a/controller/internal/infra/templates/cloudflared-compose.yml.tmpl +++ b/controller/internal/infra/templates/cloudflared-compose.yml.tmpl @@ -6,7 +6,17 @@ services: image: {{.Image}} container_name: cloudflared restart: unless-stopped - command: tunnel run + # R-841: metrics on the container's own loopback at a FIXED port, and a health check that asks cloudflared itself + # whether the tunnel is CONNECTED (`/ready`: 200 with readyConnections > 0, else 503 — measured 2026-10-04: with a + # wrong token the container stays "running" while /ready is 503). The host agent reads the result with + # `docker inspect` (State.Health) and reports the tunnel as running | not_running | unknown. + command: tunnel --metrics localhost:20241 run + healthcheck: + test: ["CMD", "cloudflared", "tunnel", "--metrics", "localhost:20241", "ready"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 30s environment: - TUNNEL_TOKEN={{.CFTunnelToken}} dns: diff --git a/controller/internal/stacks/infra_tunnel_test.go b/controller/internal/stacks/infra_tunnel_test.go index 8411e2c..50f62fb 100644 --- a/controller/internal/stacks/infra_tunnel_test.go +++ b/controller/internal/stacks/infra_tunnel_test.go @@ -282,3 +282,23 @@ func TestReconcileFileBrowserImage_MovesOnlyTheImage(t *testing.T) { t.Fatalf("an up-to-date file browser was touched: err %v calls %+v", err, *calls) } } + +// R-841: the cloudflared compose carries a health check that asks cloudflared whether the tunnel is CONNECTED, and +// pins the metrics address it asks. Red-proof: drop the healthcheck block from the template and this fails. +func TestRenderCloudflared_HasAReadinessHealthcheck(t *testing.T) { + for _, tunnel := range []bool{false, true} { + f, err := infra.RenderCloudflared(infra.CloudflaredData{CFTunnelToken: "tok", Tunnel: tunnel}) + if err != nil { + t.Fatal(err) + } + c := f["docker-compose.yml"].Content + for _, want := range []string{ + "command: tunnel --metrics localhost:20241 run", + `test: ["CMD", "cloudflared", "tunnel", "--metrics", "localhost:20241", "ready"]`, + } { + if !strings.Contains(c, want) { + t.Fatalf("tunnel=%v: compose lacks %q:\n%s", tunnel, want, c) + } + } + } +}