hub v0.65.0 — PBS DR storage visibility (ep0 usage op) + Offsite tab split + dual dashboard gauges (R-5)
Makes PBS DR storage visible like the restic pool box (v0.64.0), differentiated. Scoping
correction: restic = subaccounts on the shared Hetzner Storage Box (Hetzner API); PBS DR =
the felhom-offsite PBS datastore on the ep0 endpoint VM (NO Hetzner API). Option A
(Viktor-ruled): a read-only `usage` op on the felhom-tenantsync ep0 forced command (twin of
fingerprint), polled by a new hub checker on the 15-min throttle. READ-ONLY throughout.
Phase-0 (gate PASSED): on ep0 (PBS 4.2.3), df -B1 --output=size,used,avail <datastore path>
yields bytes (39990112256/7627939840/... ~19%), read-only, existing sudo context, no admin token.
- scripts/felhom-tenantsync.sh -> v1.2.0: read-only `usage` short-circuit (df on the datastore
path), no customer_id, no admin token, NO mutation. + a bash harness proving zero mutation.
- tenantsync.Client.Usage() + BoxUsage; unknown-op -> typed ErrUsageUnsupported (graceful).
- monitor.PBSDRBoxChecker: OffsiteBoxChecker clone over a usageReader seam; 15-min throttle,
cached PBSBoxSnapshot, escalation-only pbsdr_box_fill on the "pbsdr-box" scope (operator only,
no SaveEvent), recovery re-arm. Fill only. THREE states: ok / unavailable (ep0 <=v1.1.0,
neutral no-alert) / degraded (exec failed, keep last).
- config: Alerting.PBSDRBoxFill{Warn,Crit}Percent (80/90); built with the tenantsync client,
60s sweep, SetPBSDRBox. Hub deploy INDEPENDENT of the ep0 update (graceful degradation).
- web: /offsite splits into Restic + PBS DR hash tabs (endpoint cards under PBS DR); PBS panel;
the single dashboard tile becomes two gauges (RESTIC pct.ratio, PBS DR pct / n/a).
- runbook offsite-endpoint.md 10: v1.2.0 update steps (no sudoers/authorized_keys change).
Tests: 10 Go + the harness; 3 red-proofs (usage mutation, escalation-only, unavailable-drives-band)
confirmed red then restored. go build/vet/test + bash -n + hub confirm gate all pass.
This commit is contained in:
@@ -22,12 +22,23 @@
|
||||
</nav>
|
||||
</header>
|
||||
|
||||
{{with .OffsiteTile}}
|
||||
<a href="/offsite" class="offsite-tile offsite-tile-{{.Band}}">
|
||||
<span class="offsite-tile-label">Offsite pool</span>
|
||||
<span class="offsite-tile-val">{{formatFloat .FillPercent}}% · {{.RatioStr}}</span>
|
||||
{{if .Degraded}}<span class="offsite-tile-stale">stale</span>{{end}}
|
||||
</a>
|
||||
{{if or .OffsiteTile .PBSTile}}
|
||||
<div class="offsite-gauges">
|
||||
{{with .OffsiteTile}}
|
||||
<a href="/offsite#tab=restic" class="offsite-tile offsite-tile-{{.Band}}">
|
||||
<span class="offsite-tile-label">Restic</span>
|
||||
<span class="offsite-tile-val">{{formatFloat .FillPercent}}% · {{.RatioStr}}</span>
|
||||
{{if .Degraded}}<span class="offsite-tile-stale">stale</span>{{end}}
|
||||
</a>
|
||||
{{end}}
|
||||
{{with .PBSTile}}
|
||||
<a href="/offsite#tab=pbsdr" class="offsite-tile offsite-tile-{{.Band}}">
|
||||
<span class="offsite-tile-label">PBS DR</span>
|
||||
{{if .Unavailable}}<span class="offsite-tile-val" style="color:var(--text-3);">n/a</span>{{else}}<span class="offsite-tile-val">{{formatFloat .FillPercent}}%</span>{{end}}
|
||||
{{if .Degraded}}<span class="offsite-tile-stale">stale</span>{{end}}
|
||||
</a>
|
||||
{{end}}
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
{{if not .Customers}}
|
||||
|
||||
@@ -21,11 +21,14 @@
|
||||
</nav>
|
||||
</header>
|
||||
|
||||
<h2 style="margin-bottom: 0.5rem;">Offsite connectivity</h2>
|
||||
<p class="text-muted" style="margin: 0 0 1rem; font-size: 0.85em;">
|
||||
Peer allocation and endpoint sync currently use the lowest endpoint id (ep0).
|
||||
Per-endpoint allocation is a future work item.
|
||||
</p>
|
||||
<h2 style="margin-bottom: 0.75rem;">Offsite</h2>
|
||||
|
||||
<!-- Two offsite tiers, two distinct stores (v0.65.0, R-5): RESTIC = subaccounts on the shared
|
||||
Hetzner Storage Box; PBS DR = the felhom-offsite datastore on the ep0 endpoint VM. -->
|
||||
<nav class="tab-nav" id="tab-nav">
|
||||
<a href="#tab=restic" data-tab="restic" class="active">Restic (shared box)</a>
|
||||
<a href="#tab=pbsdr" data-tab="pbsdr">PBS DR</a>
|
||||
</nav>
|
||||
|
||||
{{if eq .Flash "endpoint_saved"}}
|
||||
<div class="flash flash-success">Endpoint saved.</div>
|
||||
@@ -34,6 +37,9 @@
|
||||
<div class="flash flash-success">Endpoint deleted.</div>
|
||||
{{end}}
|
||||
|
||||
<!-- ═══ Restic tab ═══ -->
|
||||
<div class="tab-panel tab-panel-active" data-tab="restic">
|
||||
|
||||
<!-- R-5 (v0.64.0): shared pool-box aggregate — total fill, oversubscription, per-customer usage -->
|
||||
<section class="card" style="margin-bottom: 1.5rem;">
|
||||
<h3 style="margin: 0 0 0.75rem;">Offsite pool box</h3>
|
||||
@@ -73,6 +79,40 @@
|
||||
{{end}}
|
||||
</section>
|
||||
|
||||
</div><!-- ═══ /Restic tab ═══ -->
|
||||
|
||||
<!-- ═══ PBS DR tab ═══ -->
|
||||
<div class="tab-panel" data-tab="pbsdr">
|
||||
|
||||
<!-- R-5 (v0.65.0): PBS DR datastore fill (felhom-offsite on ep0, read via the tenantsync usage op) -->
|
||||
<section class="card" style="margin-bottom: 1.5rem;">
|
||||
<h3 style="margin: 0 0 0.75rem;">PBS DR datastore</h3>
|
||||
{{if not .PBSBox.Configured}}
|
||||
<p class="text-muted" style="font-size: 0.9em;">PBS DR metrics not configured (no offsite endpoint / tenantsync key on this hub).</p>
|
||||
{{else if .PBSBox.Pending}}
|
||||
<p class="text-muted" style="font-size: 0.9em;">PBS DR metrics loading — the first poll has not landed yet.</p>
|
||||
{{else if .PBSBox.Unavailable}}
|
||||
<p class="text-muted" style="font-size: 0.9em;">PBS DR usage not available — the endpoint script update (felhom-tenantsync v1.2.0) is pending. The gauge lights up on the next poll once ep0 is updated (no hub redeploy).</p>
|
||||
{{else if .PBSBox.HasFill}}
|
||||
<table class="detail-table">
|
||||
<tr><th style="width: 12rem;">Datastore</th><td><code>felhom-offsite</code> (ep0)</td></tr>
|
||||
<tr><th>Capacity</th><td>{{.PBSBox.CapacityStr}}</td></tr>
|
||||
<tr><th>Used</th><td>{{.PBSBox.UsedStr}} · {{formatFloat .PBSBox.FillPercent}}% full</td></tr>
|
||||
</table>
|
||||
<div class="bar" style="margin: 0.4rem 0 0.9rem;"><div class="bar-fill bar-{{.PBSBox.FillBand}}" style="width: {{formatFloat .PBSBox.FillPercent}}%;"></div></div>
|
||||
<table class="detail-table">
|
||||
<tr><th style="width: 12rem;">Polled</th><td>{{timeAgo .PBSBox.FetchedAt}}{{if .PBSBox.Degraded}} <span class="status-badge status-badge-warn">STALE — last poll failed</span>{{end}}</td></tr>
|
||||
</table>
|
||||
{{else}}
|
||||
<p class="text-muted" style="font-size: 0.9em;">PBS DR usage temporarily unavailable (endpoint poll failed).</p>
|
||||
{{end}}
|
||||
</section>
|
||||
|
||||
<p class="text-muted" style="margin: 0 0 1rem; font-size: 0.85em;">
|
||||
The endpoint below IS the PBS DR host. Peer allocation and endpoint sync currently use the
|
||||
lowest endpoint id (ep0); per-endpoint allocation is a future work item.
|
||||
</p>
|
||||
|
||||
{{if .HasEndpoints}}
|
||||
{{range .Endpoints}}
|
||||
<section class="card" style="margin-bottom: 1.5rem;"
|
||||
@@ -185,11 +225,39 @@
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
</div><!-- ═══ /PBS DR tab ═══ -->
|
||||
|
||||
<footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
|
||||
Felhom Hub <span style="font-family: var(--font-mono)">{{hubVersion}}</span>
|
||||
</footer>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Hash tabs (v0.65.0), mirrored from customer_unified.html. Without JS this never runs — the body
|
||||
// never gets .js-tabs, so both panels stay visible and the page reads top-to-bottom (data is
|
||||
// server-rendered; JS only picks which panel is shown). Default tab = restic.
|
||||
(function() {
|
||||
var panels = document.querySelectorAll('.tab-panel');
|
||||
var links = document.querySelectorAll('#tab-nav a');
|
||||
if (!panels.length || !links.length) return;
|
||||
document.body.classList.add('js-tabs');
|
||||
var known = {};
|
||||
panels.forEach(function(p) { known[p.getAttribute('data-tab')] = true; });
|
||||
function currentTab() {
|
||||
var m = (location.hash || '').match(/^#tab=([a-z-]+)$/);
|
||||
var t = m ? m[1] : '';
|
||||
return known[t] ? t : 'restic';
|
||||
}
|
||||
function activate() {
|
||||
var tab = currentTab();
|
||||
panels.forEach(function(p) { p.classList.toggle('tab-panel-active', p.getAttribute('data-tab') === tab); });
|
||||
links.forEach(function(a) { a.classList.toggle('active', a.getAttribute('data-tab') === tab); });
|
||||
}
|
||||
window.addEventListener('hashchange', activate);
|
||||
activate();
|
||||
})();
|
||||
</script>
|
||||
|
||||
<script>
|
||||
// Endpoint management JS (v0.47.0). The server enforces every guard — this layer only
|
||||
// fills the edit form from a card's data attributes and adds the pubkey-change confirm.
|
||||
|
||||
@@ -303,6 +303,10 @@ header h1 {
|
||||
.bar-fill.bar-warning { background: var(--warn); }
|
||||
.bar-fill.bar-critical { background: var(--crit); }
|
||||
|
||||
/* Offsite dashboard gauges (v0.65.0, R-5) — two side-by-side tiles (Restic, PBS DR). */
|
||||
.offsite-gauges { display: flex; gap: 0.75rem; flex-wrap: wrap; margin-bottom: 1rem; }
|
||||
.offsite-gauges .offsite-tile { margin-bottom: 0; }
|
||||
|
||||
/* Offsite pool dashboard tile (v0.64.0, R-5) — compact, band-colored, links to /offsite. */
|
||||
.offsite-tile {
|
||||
display: inline-flex; align-items: baseline; gap: 0.5rem;
|
||||
|
||||
Reference in New Issue
Block a user