hub v0.64.0 — offsite pool-box aggregate: fill, oversubscription, per-customer bars, operator alert (R-5)
The operator sees the shared pool box's real state on the hub: total box fill vs
capacity, Σ(shared soft quotas) vs capacity (the oversubscription ratio), per-customer
usage/quota bars, and a box-level operator alert (fill % + oversub ratio) on the existing
dispatcher's operator channel. Per-customer fill alerts already existed; the box-level
aggregate was the gap. READ-ONLY against Hetzner (GET only).
Phase-0 probe (gate PASSED): the live pool box 611714 returns capacity via
storage_box_type.size (1 TiB / bx11) and usage via a stats object (size/size_data/
size_snapshots), all bytes; our token reads it (200).
- hetznerapi: additive StorageBoxType + StorageBoxStats on StorageBox (no existing field/
method changed); fake carries them + a GetBoxCalls counter; golden decode test.
- monitor.OffsiteBoxChecker: OffsiteChecker-sibling for the box; fetch-throttled (1 GET/
15min), cached BoxSnapshot, escalation-only + recovery re-arm. FILL (used/capacity 80/90)
+ OVERSUB (Σ shared+enabled quotas / capacity, 2.0x) — independent. Σ from the ConfigJSON
Descriptor (offsite.ReadDescriptor, new), never the report echo; dedicated+disabled
excluded. Scope "pool-box" -> operator channel only, no SaveEvent. Failed fetch keeps the
last snapshot degraded; missing data never becomes 0% and never transitions a band.
- config: Alerting.OffsiteBoxFill{Warn,Crit}Percent + OffsiteOversubWarnRatio (80/90/2.0
defaults; thresholds pending Viktor's ruling). Constructed in the HETZNER_TOKEN branch,
60s sweep, snapshot handed to the web server.
- web: Offsite-tab panel (fill bar, Σ+ratio, per-customer usage/quota rows) + a compact
dashboard tile; reads the cached snapshot only, never fetches; nil -> "not configured".
Tests: 10 new + 4 red-proofs (throttle, Σ filter, escalation-only, failed-fetch honesty),
all confirmed red then restored. go build/vet/test all pass; hub confirm gate OK.
This commit is contained in:
@@ -1,5 +1,48 @@
|
||||
# Felhom Hub — Changelog
|
||||
|
||||
## v0.64.0 — offsite pool-box aggregate: fill, oversubscription, per-customer bars, operator alert (R-5) (2026-07-17)
|
||||
|
||||
Ships **R-5**: the operator sees the shared pool box's real state on the hub — **total box fill vs
|
||||
capacity**, **Σ(shared soft quotas) vs capacity** (the oversubscription ratio), **per-customer usage/quota
|
||||
bars**, and a **box-level operator alert** (fill % + oversubscription ratio) riding the existing
|
||||
dispatcher's operator channel. Per-customer fill alerts already existed (OffsiteChecker, 90/95% of each
|
||||
quota); the box-level aggregate was the gap — the operator's early warning that the *pool itself* is
|
||||
filling, before any single customer breaches. All READ-ONLY against Hetzner (GET only). Green:
|
||||
`go build/vet/test` all pass; hub confirm gate OK.
|
||||
|
||||
- **Phase-0 probe (gate, PASSED):** one authenticated GET of the live pool box (611714) pinned the API
|
||||
shape — capacity is `storage_box_type.size` (1 TiB for bx11), usage is a `stats` object
|
||||
(`size`/`size_data`/`size_snapshots`), all bytes; our token reads it (200). The type extension mirrors it.
|
||||
- **`hetznerapi` (`hetznerapi.go`, `fake.go`):** additive `StorageBoxType` + `StorageBoxStats` sub-structs
|
||||
on `StorageBox` (no existing field/method changed); fake carries them + a `GetBoxCalls` counter. Golden
|
||||
decode test against the (redacted) probe capture.
|
||||
- **`monitor.OffsiteBoxChecker` (`offsite_box.go`, new):** the OffsiteChecker-sibling for the box as a
|
||||
whole — **fetch-throttled** (one Hetzner GET per 15 min; ≈4/hour, never per-sweep or per-page-load),
|
||||
cached `BoxSnapshot`, escalation-only emits with silent recovery re-arm. Two independent signals: FILL
|
||||
(used/capacity, warn 80% / crit 90%) and OVERSUBSCRIPTION (Σ shared+enabled quotas / capacity, warn
|
||||
2.0×) — both can fire, neither masks the other. Σ(quota) is read from the authoritative ConfigJSON
|
||||
`Descriptor` (`offsite.ReadDescriptor`, new), NEVER the report echo; dedicated + disabled customers
|
||||
excluded. Events carry the customer-less scope `"pool-box"` → operator channel ONLY (`processCustomer`
|
||||
no-ops on it) and are NOT SaveEvent'd (no customer row to key them). A failed fetch keeps the last
|
||||
snapshot marked degraded — missing data never becomes 0% and never drives a band transition.
|
||||
- **Config (`cmd/hub/main.go`):** `Alerting.OffsiteBoxFillWarnPercent` (80) / `OffsiteBoxFillCritPercent`
|
||||
(90) / `OffsiteOversubWarnRatio` (2.0), plumbed like `StorageFill*`. The checker is constructed inside
|
||||
the existing `HETZNER_TOKEN` branch (shares the client), registered in the 60 s sweep, and its snapshot
|
||||
handed to the web server. **Thresholds are Claude's encoding of the starter suggestion — Viktor's ruling
|
||||
pending; the named keys are the one-line flip.**
|
||||
- **Web surfaces (`web/offsite_box.go` new, `templates/offsite.html`, `dashboard.html`, `style.css`):** an
|
||||
Offsite-tab panel (capacity, used with data/snapshot split, fill bar, Σ quotas + ratio, fetched-at, +
|
||||
per-customer rows sorted by usage — shared with a usage/quota bar, dedicated listed without one, no-report
|
||||
customers show "no usage reported yet") and a compact Dashboard tile (`fill% · ratio`, band-colored,
|
||||
linking to /offsite). The web layer reads only the cached snapshot — it NEVER fetches. Nil provider → both
|
||||
render an honest "not configured". Exception color (neutral/amber/red, no green).
|
||||
- **Tests + red-proofs (run-fail-restore):** 10 new tests (throttle ≈4-not-≈60, Σ/ratio truth table,
|
||||
band transitions incl. in-band no-re-emit + recovery re-arm + oversub independence + pool-box operator-only
|
||||
scope, failed-fetch honesty, zero-capacity guard, golden decode, panel render × "not configured"/"with
|
||||
data"/"no usage reported yet"). Red-proofs: (i) drop the throttle → ≈60 calls; (ii) sum dedicated/disabled
|
||||
→ wrong Σ; (iii) drop the escalation-only guard → in-band re-emit; (iv) zero the snapshot on a failed fetch
|
||||
→ lost last-known. All confirmed red, then restored.
|
||||
|
||||
## v0.63.0 — system-initiated immediacy: wire the proven poke/bump notifiers into every mutation site that lacked one (2026-07-17)
|
||||
|
||||
The immediate-sync arc (Dir-1 trigger, Dir-2b wait channel, Dir-2a agent poke) covered only
|
||||
|
||||
Reference in New Issue
Block a user