docs: PBS DR slice-1 runbook §10 addendum + REPORT/CONTEXT (hub v0.44.0 live)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-10 21:03:51 +02:00
parent 32ca1f60a7
commit 3ea2b481da
3 changed files with 112 additions and 29 deletions
@@ -346,3 +346,50 @@ in `agent.json`. The agent renders the config, claims a port, writes `felhom-op`
from `oob_operator_ssh_key`, and fills the belt sets. Verify: from the operator peer,
`ssh -p <claimed-port> felhom-op@<box tunnel IP>`; the host belt drops any non-operator tunnel source.
Re-verify PBS (`pvesm status --storage felhom-offsite` on the box) after the forward change.
## 10. The felhom-tenantsync surface (PBS DR tier SLICE 1 — installed 2026-07-10)
The hub's SECOND forced-command surface on the endpoint: per-customer PBS tenancy ops (namespace +
privilege-separated token + dual-grant, the §4a dance automated). Same low-priv user as peersync, its
OWN keypair + script + sudoers drop-in — **the peersync files are untouched** (one script, one job;
peersync script + sudoers sha256 asserted identical before/after at install). Script source of truth:
`scripts/felhom-tenantsync.sh` (v1.0.0; JSON stdin/stdout, ops `provision`/`reissue`/`fingerprint`,
NO deprovision — data deletion stays a deliberate decision). Hub client: `hub/internal/tenantsync`.
On the build server (the hub's credential; mirror of step 6):
```sh
ssh-keygen -t ed25519 -f tenantsync-ssh -N "" -C hub-tenancy@felhom
sudo kubectl -n felhom-system create secret generic tenantsync --from-file=key=tenantsync-ssh
shred -u tenantsync-ssh tenantsync-ssh.pub # from here the key exists ONLY in the Secret
```
(The pinned host key + endpoint address are REUSED from the peersync env — no new hostkey entry.
`manifests/hub.yaml` mounts the Secret at `/etc/hub-secrets/tenantsync/key` via
`TENANTSYNC_SSH_KEY_FILE`; absent Secret → the hub logs "PBS DR tenantsync disabled".)
On the box, as root:
```sh
# script: stage → strip CRLF if via a Windows checkout → syntax-check → install → THEN activate
tr -d '\r' < felhom-tenantsync.sh > /tmp/fts && bash -n /tmp/fts
install -o root -g root -m 0755 /tmp/fts /usr/local/bin/felhom-tenantsync && rm /tmp/fts
cat > /etc/sudoers.d/felhom-tenantsync <<'EOF'
felhom-peersync ALL=(root) NOPASSWD: /usr/local/bin/felhom-tenantsync
EOF
chmod 0440 /etc/sudoers.d/felhom-tenantsync && visudo -cf /etc/sudoers.d/felhom-tenantsync
printf 'restrict,command="sudo /usr/local/bin/felhom-tenantsync" %s\n' "<tenantsync pubkey line>" \
>> /home/felhom-peersync/.ssh/authorized_keys
```
Verify (the install's smoke, run from the build server with the key before it is shredded — pin the
host key from step 6's on-box read, never keyscan):
- `{"op":"fingerprint"}``{"status":"ok","fingerprint":"<PBS cert sha256>"}`
- `{"op":"provision","customer_id":"spike-smoke"}` → ok-shape; **assert `token_secret` length only,
never print it**; a second provision → `{"code":"token_exists"}`; `reissue` → fresh ok-shape;
unknown op → `bad_request`. Then tear the throwaway tenant down (delete token, ns ACL, namespace
via a transient root admin token — §4a teardown shape).
Install-run facts (2026-07-10): smoke exposed that `proxmox-backup-client … --output-format json`
wraps output as `{"data":[…]}` (the MANAGER returns bare arrays) — script fixed to `(.data // .)`
before the hub ever used the surface. The §6-era orphan `root@pam!spike` token + its
`/datastore/scratch` DatastoreAdmin ACL were removed in the same session (spike flag #1); after
teardown the endpoint holds exactly the real `demo-felhom-01` tenancy. Hub v0.44.0 logs
`PBS DR tenantsync enabled (endpoint 167.233.158.164:22, user felhom-peersync)` on start.