R-344: restore the idle-connection timeout our hand-rolled transports lost
gates / gates (push) Successful in 7s

Every client here pins TLS, so none can use http.DefaultTransport and each
hand-rolls its own. A composite literal takes IdleConnTimeout ZERO, which
means retain idle connections forever -- not "use a sane default".
pbsTargetsFromPVE builds a fresh pbs.Client every cycle and drops the
previous one, and an abandoned http.Transport does not close its
connections. One stranded socket per cycle, on both sides, forever.

Measured: 388 established connections on ep0 over 46 h, 194 per box, zero
closed in a 31-minute window. pvestatd and proxmox-backup-client made
162,404 requests in the same window and leaked none.

New leaf package internal/httpx owns the default (90s, http.DefaultTransport's
own value) and NewTransport, which returns a FRESH transport per call and
treats <=0 as "use the default", never "no timeout". pbs.Config gains
IdleConnTimeout for tests only.

hub and proxmox carried the same missing default and are corrected here for
consistency. Neither contributed to the ep0 leak -- both are built once per
process and neither talks to ep0:8007.

Tests count connections SERVER-side and model the abandonment, so they pin
the consequence rather than the field. Two red-proofs, both seen failing:
removing the timeout -> "still holds 5 open connection(s), want 0";
DisableKeepAlives -> "3 sequential requests over 3 connection(s), want 1"
(the leak test PASSES under that one -- it is the worse-fix guard that
catches it).

Not released: hand-installed on demo-hp only so demo-felhom stays the
control. CHANGELOG heading stays UNRELEASED until the publish is authorised.
This commit is contained in:
2026-08-20 11:09:39 +02:00
parent f17ed11599
commit ede49b610d
9 changed files with 404 additions and 7 deletions
+14 -2
View File
@@ -10,6 +10,8 @@ import (
"net/url"
"strings"
"time"
"gitea.dooplex.hu/admin/felhom-agent/internal/httpx"
)
// Client is the PBS-API client for ONE PBS server. Construct with NewClient. It is pure (no
@@ -31,6 +33,11 @@ type Config struct {
Secret string // token secret (from <id>.pw)
Namespace string // PBS namespace (from storage.cfg `namespace`); "" = root. S4 per-customer tenancy.
Timeout time.Duration
// IdleConnTimeout bounds how long this client's idle keep-alive connections are retained.
// Zero means httpx.DefaultIdleConnTimeout (90s) — it does NOT mean "no timeout", which is the
// R-344 defect. Production leaves it unset; only tests set it, to avoid a 90-second wait.
IdleConnTimeout time.Duration
}
// NewClient builds a fingerprint-pinned, token-authed PBS client.
@@ -55,8 +62,13 @@ func NewClient(cfg Config) (*Client, error) {
authHeader: "PBSAPIToken=" + cfg.TokenID + ":" + cfg.Secret,
namespace: cfg.Namespace,
http: &http.Client{
Timeout: timeout,
Transport: &http.Transport{TLSClientConfig: tlsCfg},
Timeout: timeout,
// R-344: this transport MUST come from httpx. pbsTargetsFromPVE (cmd/felhom-agent/
// main.go) builds a fresh Client every cycle and drops the previous one, so a
// transport with no idle timeout strands one connection per cycle, forever, on both
// sides. That leaked 388 sockets onto ep0 in 46 hours. Pinned by
// TestAbandonedClientsReleaseTheirConnections — do not inline an http.Transport here.
Transport: httpx.NewTransport(tlsCfg, cfg.IdleConnTimeout),
},
}, nil
}