# REPORT — controller v0.74.0: fix the controller→agent connection leak ## Baseline → target `felhom-controller` `main` v0.73.0 → **v0.74.0**. Controller-only change; agent/hub/firewall untouched. ## Problem `Server.agentClient()` built a fresh `agentapi.Client` (hence a fresh bare `http.Transport`, `IdleConnTimeout:0`) on **every** agent API call and discarded it without closing idle connections. The agent's keep-alive left one idle `ESTABLISHED` socket per call to `192.168.0.162:8443`; they accumulated (~5.8k/day) until the ephemeral source-port range for that tuple exhausted → `connect: cannot assign requested address` (EADDRNOTAVAIL), disabling storage UI, host-metrics, and whole-guest backup after ~5 days of controller uptime. (`:8006`/pveproxy was immune — the controller never dials it.) Diagnosis: `felhom.eu/documentation/tests/unattended-test-campaign-2026-06-22-8443-diagnosis.md`. ## Change (commit `2a5b88f`) - `internal/web/server.go` — `Server` gains `agentCli *agentapi.Client`, `agentCliErr error`, `agentCliOnce sync.Once` (+ the `agentapi` import). - `internal/web/agent_disk_handlers.go` — `agentClient()` memoizes the build via `agentCliOnce` and **returns one shared client** (cfg.LocalAPI is static per process; a config-apply self-restarts the controller). The empty-endpoint "not configured" guard stays OUTSIDE the `Once`. Signature and all 19 call sites unchanged; `*agentapi.Client`/`*http.Client` are concurrency-safe so no extra locking. - `internal/agentapi/client.go` — `New` Transport hardened: `MaxIdleConns:4`, `MaxIdleConnsPerHost:2`, `IdleConnTimeout:90s` (was bare, `IdleConnTimeout:0`). Added optional `Client.Close()` (CloseIdleConnections) hygiene helper. ## Tests (green; both red-proofed) - `go build ./... && go vet ./... && go test ./...` — all green. - **T1** `TestAgentClient_ReusesSameInstance` (web) — two `agentClient()` calls return the identical pointer; `TestAgentClient_UnconfiguredErrors` — empty endpoint still errors. Red-proof: reverting to per-call `agentapi.New` → pointers differ → FAIL (shown, reverted). - **T2** `TestNew_TransportIdlePoolBounded` (agentapi, white-box) — `IdleConnTimeout>0` AND `MaxIdleConnsPerHost>0`. Red-proof: bare Transport → `IdleConnTimeout==0` → FAIL (shown, reverted). ## Deploy Built+pushed `gitea.dooplex.hu/admin/felhom-controller:0.74.0` on 192.168.0.180 (digest `sha256:2e85376e…`), deployed to guest 9201 via the bootstrap path (`docker pull` → pin `/etc/felhom-controller-image` → restart `felhom-controller-bootstrap.service`). `docker inspect`: `image=:0.74.0 status=running health=healthy`. > Process note: the first build packaged stale source (the build server's `~/git/felhom-controller` > was at v0.73.0 — the required `git -C ~/git/felhom-controller pull` step had been skipped, so the > image was the old code mislabeled `:0.74.0`; the live test still leaked). Pulled the source to > `2a5b88f` and rebuilt — second image (`sha256:2e85376e…`) is the real fix. ## LIVE acceptance — leak is gone (the real proof) Burst of **120 agent calls** (alternating `/api/disks` + `/api/host-metrics`) against the controller in guest 9201, sampling idle `ESTABLISHED` sockets to `192.168.0.162:8443` from the controller's netns (`nsenter -t -n ss -tn state established dst 192.168.0.162:8443 | grep -c 192.168`): | stage | pre-fix image (stale build) | **fixed image (2e85376e)** | |---|---|---| | before | 8 | **1** | | after 40 calls | 48 | **2** | | after 80 calls | 90 | **2** | | after 120 calls | 132 | **2** | **Fixed: flat at 2 (= MaxIdleConnsPerHost), independent of call count.** Pre-fix grew ~1/call. Agent endpoints still return real data (`/api/disks` lists disks, `/api/host-metrics` returns host CPU/mem). ## Not touched / separate open item The agent, its bridge-IP `ListenAddr` bind (deliberate defense-in-depth), and all firewall rules were **not** changed (controller-only fix). **Separate open item (NOT addressed here):** the defense-in-depth host firewall rule scoping `:8443` to the guest bridge subnet is still absent (`pve-firewall` disabled, no 8443 rule) — close it independently of this fix.