hub v0.65.0 — PBS DR storage visibility (ep0 usage op) + Offsite tab split + dual dashboard gauges (R-5)
Makes PBS DR storage visible like the restic pool box (v0.64.0), differentiated. Scoping
correction: restic = subaccounts on the shared Hetzner Storage Box (Hetzner API); PBS DR =
the felhom-offsite PBS datastore on the ep0 endpoint VM (NO Hetzner API). Option A
(Viktor-ruled): a read-only `usage` op on the felhom-tenantsync ep0 forced command (twin of
fingerprint), polled by a new hub checker on the 15-min throttle. READ-ONLY throughout.
Phase-0 (gate PASSED): on ep0 (PBS 4.2.3), df -B1 --output=size,used,avail <datastore path>
yields bytes (39990112256/7627939840/... ~19%), read-only, existing sudo context, no admin token.
- scripts/felhom-tenantsync.sh -> v1.2.0: read-only `usage` short-circuit (df on the datastore
path), no customer_id, no admin token, NO mutation. + a bash harness proving zero mutation.
- tenantsync.Client.Usage() + BoxUsage; unknown-op -> typed ErrUsageUnsupported (graceful).
- monitor.PBSDRBoxChecker: OffsiteBoxChecker clone over a usageReader seam; 15-min throttle,
cached PBSBoxSnapshot, escalation-only pbsdr_box_fill on the "pbsdr-box" scope (operator only,
no SaveEvent), recovery re-arm. Fill only. THREE states: ok / unavailable (ep0 <=v1.1.0,
neutral no-alert) / degraded (exec failed, keep last).
- config: Alerting.PBSDRBoxFill{Warn,Crit}Percent (80/90); built with the tenantsync client,
60s sweep, SetPBSDRBox. Hub deploy INDEPENDENT of the ep0 update (graceful degradation).
- web: /offsite splits into Restic + PBS DR hash tabs (endpoint cards under PBS DR); PBS panel;
the single dashboard tile becomes two gauges (RESTIC pct.ratio, PBS DR pct / n/a).
- runbook offsite-endpoint.md 10: v1.2.0 update steps (no sudoers/authorized_keys change).
Tests: 10 Go + the harness; 3 red-proofs (usage mutation, escalation-only, unavailable-drives-band)
confirmed red then restored. go build/vet/test + bash -n + hub confirm gate all pass.
This commit is contained in:
@@ -1,7 +1,12 @@
|
||||
#!/usr/bin/env bash
|
||||
# felhom-tenantsync v1.1.0 — the offsite endpoint's per-customer PBS tenancy surface (PBS DR tier
|
||||
# felhom-tenantsync v1.2.0 — the offsite endpoint's per-customer PBS tenancy surface (PBS DR tier
|
||||
# SLICE 1; spike SPIKE-pbs-tier-provisioning-2026-07-10 §3).
|
||||
#
|
||||
# v1.2.0 (2026-07-17, hub R-5 PBS visibility): + read-only `usage` op — the felhom-offsite datastore's
|
||||
# total/used/avail (bytes, via `df` on the datastore path). No customer_id, no admin token, NO
|
||||
# mutation. Structural twin of `fingerprint`. Backward-compatible: an older hub never sends it, and a
|
||||
# hub that does against an OLD script gets the existing `bad_request "unknown op"` (graceful).
|
||||
#
|
||||
# Runs as the SSH forced command for the hub's SECOND `felhom-peersync` key (via sudo — its own
|
||||
# single sudoers line; the peersync script/key are untouched: one script, one job). JSON on stdin,
|
||||
# JSON on stdout. Ops:
|
||||
@@ -23,6 +28,8 @@
|
||||
# This is the DELIBERATE, gated data-destruction the slice-1 note reserved — the operator RESET
|
||||
# confirm (typed customer-id + separate escrow-custody ack) is the human decision it demanded.
|
||||
# {"op":"fingerprint"} → {"status":"ok","fingerprint":"<PBS cert sha256>"}
|
||||
# {"op":"usage"} → {"status":"ok","total":<bytes>,"used":<bytes>,"avail":<bytes>}
|
||||
# READ-ONLY datastore fill (df on the felhom-offsite path). No customer_id, no admin token, no mutation.
|
||||
#
|
||||
# Secret hygiene (load-bearing):
|
||||
# - The token secret exists ONLY in memory and in the final stdout JSON — never a file, never
|
||||
@@ -68,6 +75,20 @@ if [ "$OP" = "fingerprint" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# usage (v1.2.0) — READ-ONLY box-level datastore fill. No customer_id, no admin token, NO mutation
|
||||
# (a pure `df` on the datastore path). The hub's PBS-DR checker polls this on its 15-min throttle.
|
||||
# Short-circuits BEFORE the admin-token generation below, exactly like fingerprint.
|
||||
if [ "$OP" = "usage" ]; then
|
||||
DS_PATH=$(proxmox-backup-manager datastore list --output-format json \
|
||||
| jq -r --arg ds "$DS" '.[] | select(.name == $ds) | .path')
|
||||
[ -n "$DS_PATH" ] || err_json internal "datastore $DS not found"
|
||||
# df -B1: 1-byte blocks. --output pins the columns; tail -1 skips the header row.
|
||||
read -r TOTAL USED AVAIL < <(df -B1 --output=size,used,avail "$DS_PATH" | tail -1)
|
||||
[ -n "$TOTAL" ] && [ -n "$USED" ] && [ -n "$AVAIL" ] || err_json internal "could not read datastore usage"
|
||||
printf '{"status":"ok","total":%s,"used":%s,"avail":%s}\n' "$TOTAL" "$USED" "$AVAIL"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
case "$OP" in provision|reissue|deprovision) ;; *) err_json bad_request "unknown op" ;; esac
|
||||
|
||||
# customer_id → the namespace AND the token name. Conservative charset (PBS ns + token grammar,
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
#!/usr/bin/env bash
|
||||
# tenantsync-usage-harness.sh (v0.65.0, R-5) — Group-A test for the felhom-tenantsync `usage` op.
|
||||
# Stubs proxmox-backup-manager + df on PATH (real jq), runs the op, and asserts:
|
||||
# 1. usage → {"status":"ok","total","used","avail"} + exit 0 + ZERO mutation calls.
|
||||
# 2. provision (regression) → reaches the mutation path (admin-token generate) — the usage op did
|
||||
# not disturb the case dispatch — and NEVER emits the usage JSON.
|
||||
# Non-hollow: the stub LOGS every proxmox-backup-manager invocation; the zero-mutation assertion reads it.
|
||||
set -uo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "$0")" && pwd)"
|
||||
SCRIPT="$HERE/felhom-tenantsync.sh"
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
STUB="$WORK/bin"; mkdir -p "$STUB"
|
||||
export MUTLOG="$WORK/mutations.log"; : > "$MUTLOG"
|
||||
export CALLLOG="$WORK/calls.log"; : > "$CALLLOG"
|
||||
DS_PATH="$WORK/ds"; mkdir -p "$DS_PATH"
|
||||
|
||||
# Stub proxmox-backup-manager: read-only ops answered; MUTATING subcommands logged to MUTLOG.
|
||||
cat > "$STUB/proxmox-backup-manager" <<'PBM'
|
||||
#!/usr/bin/env bash
|
||||
echo "pbm $*" >> "$CALLLOG"
|
||||
case "$1 $2" in
|
||||
"cert info") echo "Fingerprint (sha256): aa:bb:cc:dd" ;;
|
||||
"datastore list") printf '[{"name":"scratch","path":"/srv/pbs-scratch"},{"name":"felhom-offsite","path":"%s"}]\n' "$DS_PATH" ;;
|
||||
"user generate-token") echo "MUTATION: $*" >> "$MUTLOG"; echo '{"value":"FAKE-ADMIN-TOKEN"}' ;;
|
||||
"user delete-token") echo "MUTATION: $*" >> "$MUTLOG" ;;
|
||||
"acl update") echo "MUTATION: $*" >> "$MUTLOG" ;;
|
||||
"namespace "*) echo "MUTATION: $*" >> "$MUTLOG" ;;
|
||||
*) echo "MUTATION: $*" >> "$MUTLOG" ;; # any other subcommand is mutating here
|
||||
esac
|
||||
PBM
|
||||
chmod +x "$STUB/proxmox-backup-manager"
|
||||
|
||||
# Stub proxmox-backup-client (used only by the mutating ops; log if invoked).
|
||||
cat > "$STUB/proxmox-backup-client" <<'PBC'
|
||||
#!/usr/bin/env bash
|
||||
echo "MUTATION: pbc $*" >> "$MUTLOG"
|
||||
PBC
|
||||
chmod +x "$STUB/proxmox-backup-client"
|
||||
|
||||
# Stub df: fixed bytes (total 41943040, used 8388608, avail 33554432 ⇒ 20% full).
|
||||
cat > "$STUB/df" <<'DF'
|
||||
#!/usr/bin/env bash
|
||||
echo "1B-blocks Used Avail"
|
||||
echo "41943040 8388608 33554432"
|
||||
DF
|
||||
chmod +x "$STUB/df"
|
||||
|
||||
export DS_PATH CALLLOG MUTLOG
|
||||
RC=0
|
||||
fail() { echo "FAIL: $*"; RC=1; }
|
||||
|
||||
# --- Test 1: usage op ---
|
||||
OUT=$(echo '{"op":"usage"}' | PATH="$STUB:$PATH" bash "$SCRIPT" 2>/dev/null); EC=$?
|
||||
[ "$EC" -eq 0 ] || fail "usage exit $EC (want 0)"
|
||||
EXP='{"status":"ok","total":41943040,"used":8388608,"avail":33554432}'
|
||||
[ "$OUT" = "$EXP" ] || fail "usage stdout = $OUT (want $EXP)"
|
||||
if [ -s "$MUTLOG" ]; then fail "usage ran mutations: $(cat "$MUTLOG")"; else echo "PASS: usage emits totals, exit 0, ZERO mutations"; fi
|
||||
|
||||
# --- Test 2: provision regression (dispatch intact; usage op did not break the case) ---
|
||||
: > "$MUTLOG"
|
||||
POUT=$(echo '{"op":"provision","customer_id":"harness"}' | PATH="$STUB:$PATH" bash "$SCRIPT" 2>/dev/null || true)
|
||||
grep -q "generate-token" "$MUTLOG" || fail "provision did not reach the mutation path (case dispatch broken by the usage op?)"
|
||||
case "$POUT" in *'"total"'*) fail "provision leaked the usage shape: $POUT" ;; esac
|
||||
[ "$RC" -eq 0 ] && echo "PASS: provision still dispatches to the mutation path (regression)"
|
||||
|
||||
exit $RC
|
||||
Reference in New Issue
Block a user