hub v0.64.0 — offsite pool-box aggregate: fill, oversubscription, per-customer bars, operator alert (R-5)

The operator sees the shared pool box's real state on the hub: total box fill vs
capacity, Σ(shared soft quotas) vs capacity (the oversubscription ratio), per-customer
usage/quota bars, and a box-level operator alert (fill % + oversub ratio) on the existing
dispatcher's operator channel. Per-customer fill alerts already existed; the box-level
aggregate was the gap. READ-ONLY against Hetzner (GET only).

Phase-0 probe (gate PASSED): the live pool box 611714 returns capacity via
storage_box_type.size (1 TiB / bx11) and usage via a stats object (size/size_data/
size_snapshots), all bytes; our token reads it (200).

- hetznerapi: additive StorageBoxType + StorageBoxStats on StorageBox (no existing field/
  method changed); fake carries them + a GetBoxCalls counter; golden decode test.
- monitor.OffsiteBoxChecker: OffsiteChecker-sibling for the box; fetch-throttled (1 GET/
  15min), cached BoxSnapshot, escalation-only + recovery re-arm. FILL (used/capacity 80/90)
  + OVERSUB (Σ shared+enabled quotas / capacity, 2.0x) — independent. Σ from the ConfigJSON
  Descriptor (offsite.ReadDescriptor, new), never the report echo; dedicated+disabled
  excluded. Scope "pool-box" -> operator channel only, no SaveEvent. Failed fetch keeps the
  last snapshot degraded; missing data never becomes 0% and never transitions a band.
- config: Alerting.OffsiteBoxFill{Warn,Crit}Percent + OffsiteOversubWarnRatio (80/90/2.0
  defaults; thresholds pending Viktor's ruling). Constructed in the HETZNER_TOKEN branch,
  60s sweep, snapshot handed to the web server.
- web: Offsite-tab panel (fill bar, Σ+ratio, per-customer usage/quota rows) + a compact
  dashboard tile; reads the cached snapshot only, never fetches; nil -> "not configured".

Tests: 10 new + 4 red-proofs (throttle, Σ filter, escalation-only, failed-fetch honesty),
all confirmed red then restored. go build/vet/test all pass; hub confirm gate OK.
This commit is contained in:
2026-07-17 20:15:34 +02:00
parent 85a14192e7
commit 4bb2df0dc4
17 changed files with 1034 additions and 17 deletions
+10 -2
View File
@@ -22,7 +22,15 @@
</nav>
</header>
{{if not .}}
{{with .OffsiteTile}}
<a href="/offsite" class="offsite-tile offsite-tile-{{.Band}}">
<span class="offsite-tile-label">Offsite pool</span>
<span class="offsite-tile-val">{{formatFloat .FillPercent}}% &middot; {{.RatioStr}}</span>
{{if .Degraded}}<span class="offsite-tile-stale">stale</span>{{end}}
</a>
{{end}}
{{if not .Customers}}
<div class="empty-state">
<p>No customer reports received yet.</p>
<p class="hint">Configure <code>hub.enabled: true</code> in customer controller.yaml to start receiving reports.</p>
@@ -44,7 +52,7 @@
</tr>
</thead>
<tbody>
{{range .}}
{{range .Customers}}
<tr class="status-{{.OverallStatus}}" onclick="window.location='/customers/{{.CustomerID}}'">
<td class="customer-name">
<span class="status-dot status-dot-{{statusColor .OverallStatus}}"></span>
+39
View File
@@ -34,6 +34,45 @@
<div class="flash flash-success">Endpoint deleted.</div>
{{end}}
<!-- R-5 (v0.64.0): shared pool-box aggregate — total fill, oversubscription, per-customer usage -->
<section class="card" style="margin-bottom: 1.5rem;">
<h3 style="margin: 0 0 0.75rem;">Offsite pool box</h3>
{{if not .OffsiteBox.Configured}}
<p class="text-muted" style="font-size: 0.9em;">Offsite pool metrics not configured (no Hetzner token / pool box id on this hub).</p>
{{else if .OffsiteBox.Pending}}
<p class="text-muted" style="font-size: 0.9em;">Pool metrics loading — the first refresh has not landed yet.</p>
{{else}}
<table class="detail-table">
<tr><th style="width: 12rem;">Box</th><td>{{.OffsiteBox.BoxType}}</td></tr>
<tr><th>Capacity</th><td>{{.OffsiteBox.CapacityStr}}</td></tr>
<tr><th>Used</th><td>{{.OffsiteBox.UsedStr}} &middot; {{formatFloat .OffsiteBox.FillPercent}}% full <span class="text-muted">(data {{.OffsiteBox.DataStr}} + snapshots {{.OffsiteBox.SnapshotStr}})</span></td></tr>
</table>
<div class="bar" style="margin: 0.4rem 0 0.9rem;"><div class="bar-fill bar-{{.OffsiteBox.FillBand}}" style="width: {{formatFloat .OffsiteBox.FillPercent}}%;"></div></div>
<table class="detail-table">
<tr><th style="width: 12rem;">&Sigma; shared soft quotas</th><td>{{.OffsiteBox.SumQuotaStr}}</td></tr>
<tr><th>Oversubscription</th><td><span class="status-badge status-badge-{{if eq .OffsiteBox.OversubBand "warning"}}warn{{else}}ok{{end}}">{{.OffsiteBox.RatioStr}}</span> <span class="text-muted">&Sigma;(shared quotas) &divide; capacity</span></td></tr>
<tr><th>Fetched</th><td>{{timeAgo .OffsiteBox.FetchedAt}}{{if .OffsiteBox.Degraded}} <span class="status-badge status-badge-warn">STALE — last refresh failed</span>{{end}}</td></tr>
</table>
{{if .OffsiteCusts}}
<h4 style="margin: 1rem 0 0.5rem; font-size: 0.85rem; color: var(--text-3); text-transform: uppercase; letter-spacing: 0.03em;">Per-customer usage</h4>
<table class="container-table">
<thead><tr><th>Customer</th><th>Usage</th><th>Quota</th><th style="width: 35%;"></th></tr></thead>
<tbody>
{{range .OffsiteCusts}}
<tr>
<td>{{.CustomerName}}{{if .Dedicated}} <span class="text-muted">(dedicated)</span>{{end}}</td>
<td>{{if .NoUsage}}<span class="text-muted">no usage reported yet</span>{{else}}{{.UsageStr}}{{end}}</td>
<td>{{if .Dedicated}}<span class="text-muted">dedicated</span>{{else if .QuotaStr}}{{.QuotaStr}}{{else}}<span class="text-muted"></span>{{end}}</td>
<td>{{if .HasBar}}<div class="bar"><div class="bar-fill bar-{{.UsageBand}}" style="width: {{formatFloat .UsagePercent}}%;"></div></div>{{end}}</td>
</tr>
{{end}}
</tbody>
</table>
{{end}}
{{end}}
</section>
{{if .HasEndpoints}}
{{range .Endpoints}}
<section class="card" style="margin-bottom: 1.5rem;"
+19
View File
@@ -298,6 +298,25 @@ header h1 {
border-radius: var(--radius);
transition: width 0.3s ease;
}
/* Fill bands (v0.64.0, R-5) — exception color: neutral when nominal, amber ≥ warn, red ≥ crit; no green. */
.bar-fill.bar-ok { background: var(--text-3); }
.bar-fill.bar-warning { background: var(--warn); }
.bar-fill.bar-critical { background: var(--crit); }
/* Offsite pool dashboard tile (v0.64.0, R-5) — compact, band-colored, links to /offsite. */
.offsite-tile {
display: inline-flex; align-items: baseline; gap: 0.5rem;
padding: 0.4rem 0.8rem; margin-bottom: 1rem;
border: 1px solid var(--line); border-radius: var(--radius);
text-decoration: none; font-size: 0.85rem;
}
.offsite-tile-label { color: var(--text-3); text-transform: uppercase; font-size: 0.72rem; letter-spacing: 0.03em; }
.offsite-tile-val { color: var(--text-1); font-weight: 600; }
.offsite-tile-stale { color: var(--warn); font-size: 0.72rem; }
.offsite-tile-warning { border-color: var(--warn); }
.offsite-tile-warning .offsite-tile-val { color: var(--warn); }
.offsite-tile-critical { border-color: var(--crit); }
.offsite-tile-critical .offsite-tile-val { color: var(--crit); }
/* Container table */
.container-table {