hub v0.65.0 — PBS DR storage visibility (ep0 usage op) + Offsite tab split + dual dashboard gauges (R-5)
Makes PBS DR storage visible like the restic pool box (v0.64.0), differentiated. Scoping
correction: restic = subaccounts on the shared Hetzner Storage Box (Hetzner API); PBS DR =
the felhom-offsite PBS datastore on the ep0 endpoint VM (NO Hetzner API). Option A
(Viktor-ruled): a read-only `usage` op on the felhom-tenantsync ep0 forced command (twin of
fingerprint), polled by a new hub checker on the 15-min throttle. READ-ONLY throughout.
Phase-0 (gate PASSED): on ep0 (PBS 4.2.3), df -B1 --output=size,used,avail <datastore path>
yields bytes (39990112256/7627939840/... ~19%), read-only, existing sudo context, no admin token.
- scripts/felhom-tenantsync.sh -> v1.2.0: read-only `usage` short-circuit (df on the datastore
path), no customer_id, no admin token, NO mutation. + a bash harness proving zero mutation.
- tenantsync.Client.Usage() + BoxUsage; unknown-op -> typed ErrUsageUnsupported (graceful).
- monitor.PBSDRBoxChecker: OffsiteBoxChecker clone over a usageReader seam; 15-min throttle,
cached PBSBoxSnapshot, escalation-only pbsdr_box_fill on the "pbsdr-box" scope (operator only,
no SaveEvent), recovery re-arm. Fill only. THREE states: ok / unavailable (ep0 <=v1.1.0,
neutral no-alert) / degraded (exec failed, keep last).
- config: Alerting.PBSDRBoxFill{Warn,Crit}Percent (80/90); built with the tenantsync client,
60s sweep, SetPBSDRBox. Hub deploy INDEPENDENT of the ep0 update (graceful degradation).
- web: /offsite splits into Restic + PBS DR hash tabs (endpoint cards under PBS DR); PBS panel;
the single dashboard tile becomes two gauges (RESTIC pct.ratio, PBS DR pct / n/a).
- runbook offsite-endpoint.md 10: v1.2.0 update steps (no sudoers/authorized_keys change).
Tests: 10 Go + the harness; 3 red-proofs (usage mutation, escalation-only, unavailable-drives-band)
confirmed red then restored. go build/vet/test + bash -n + hub confirm gate all pass.
This commit is contained in:
@@ -1,5 +1,51 @@
|
||||
# Felhom Hub — Changelog
|
||||
|
||||
## v0.65.0 — PBS DR storage visibility (ep0 `usage` op) + Offsite tab split (Restic / PBS DR) + dual dashboard gauges (R-5) (2026-07-17)
|
||||
|
||||
Makes the **PBS DR** storage visible like the restic pool box already is (v0.64.0), the two clearly
|
||||
differentiated. The scoping correction: "the restic box" and "the PBS box" are NOT two Hetzner Storage
|
||||
Boxes — **restic** = subaccounts on the shared Hetzner Storage Box (Hetzner API, v0.64.0); **PBS DR** = a
|
||||
PBS datastore (`felhom-offsite`) on the ep0 endpoint VM (NO Hetzner API). This adds the hub's read of the
|
||||
PBS datastore fill via **Option A (Viktor-ruled):** a new read-only `usage` op on the `felhom-tenantsync`
|
||||
ep0 forced command (the structural twin of the existing `fingerprint` op), polled by a new hub checker on
|
||||
the same 15-min throttle; splits the Offsite page into **Restic / PBS DR tabs**; and puts **two dashboard
|
||||
gauges** (restic %, PBS %). READ-ONLY against ep0 and Hetzner. Green: `go build/vet/test` + `bash -n` +
|
||||
the script harness; hub confirm gate OK.
|
||||
|
||||
- **Phase-0 probe (gate, PASSED):** on ep0 (PBS 4.2.3), `df -B1 --output=size,used,avail <datastore path>`
|
||||
(path from `proxmox-backup-manager datastore list --output-format json`) yields the datastore
|
||||
total/used/avail in **bytes** (live: 39990112256 / 7627939840 / 30686326784 → ~19%), read-only, in the
|
||||
existing sudo context, no admin token. (PBS 4.2 has no native `datastore usage` command.)
|
||||
- **`scripts/felhom-tenantsync.sh` → v1.2.0:** a read-only `usage` short-circuit (before the admin-token
|
||||
generation, like `fingerprint`) → `{"status":"ok","total","used","avail"}`. **No customer_id, no admin
|
||||
token, NO mutation.**
|
||||
- **`tenantsync.Client.Usage()`** (`internal/tenantsync/client.go`): `BoxUsage{Total,Used,Avail}` + the
|
||||
op; an endpoint ≤ v1.1.0 answers `bad_request "unknown op"` → typed `ErrUsageUnsupported` (the
|
||||
graceful-degradation signal).
|
||||
- **`monitor.PBSDRBoxChecker`** (`internal/monitor/pbsdr_box.go`, new): clones OffsiteBoxChecker over a
|
||||
`usageReader` seam (the tenantsync client) — 15-min throttle, cached `PBSBoxSnapshot`, escalation-only
|
||||
`pbsdr_box_fill` on the customer-less `"pbsdr-box"` scope (operator channel only, no SaveEvent),
|
||||
recovery re-arm. FILL ONLY (PBS uses namespaces, not quotas — no oversubscription). **THREE states:**
|
||||
`ok` (bands drive), `unavailable` (ErrUsageUnsupported — expected pre-update, neutral, NO alert, logged
|
||||
once, the gauge shows n/a), `degraded` (exec failed — keep last snapshot, no band transition).
|
||||
- **Config + wiring:** `Alerting.PBSDRBoxFill{Warn,Crit}Percent` (default 80/90, independently tunable);
|
||||
the checker is built ONLY when the tenantsync client exists (shares it), registered in the 60 s sweep,
|
||||
snapshot handed to the web server. **Graceful degradation: the hub deploy is INDEPENDENT of the ep0
|
||||
update** — a hub v0.65.0 against an ep0 still on v1.1.0 shows the honest "n/a", lighting up on the next
|
||||
poll once ep0 is updated (no hub redeploy).
|
||||
- **Web (`internal/web/pbsdr_box.go` new, `offsite.go`, `templates/offsite.html`, `dashboard.html`,
|
||||
`style.css`):** the Offsite page splits into **Restic** (the v0.64.0 pool-box panel + per-customer rows)
|
||||
and **PBS DR** (a new datastore panel — capacity/used/fill bar with band + the endpoint cards, which
|
||||
belong here: the endpoint IS the PBS host) hash tabs (server-rendered, no JS dependency for the data).
|
||||
The single dashboard tile becomes two gauges — **RESTIC** (pct·ratio) and **PBS DR** (pct; "n/a" when
|
||||
unavailable) — each band-colored, each linking to its tab.
|
||||
- **Tests + red-proofs:** 10 Go tests (Usage parse + unknown-op→typed; checker throttle/bands/pbsdr-box
|
||||
operator-only/unavailable-no-alert/degraded-keeps-last; PBS panel render × ok/unavailable/not-configured)
|
||||
+ a bash script harness (usage JSON + exit 0 + **zero mutation** + provision regression). Red-proofs
|
||||
(run-fail-restore): the op emitting a mutation → the harness zero-mutation assertion fails; the
|
||||
escalation-only guard removed → in-band re-emit fails; unavailable driving a band → the no-alert test
|
||||
fails. All confirmed red, then restored.
|
||||
|
||||
## v0.64.0 — offsite pool-box aggregate: fill, oversubscription, per-customer bars, operator alert (R-5) (2026-07-17)
|
||||
|
||||
Ships **R-5**: the operator sees the shared pool box's real state on the hub — **total box fill vs
|
||||
|
||||
Reference in New Issue
Block a user