# REPORT — Slice 6 Phase B: PBS offsite tier (v0.6.0) (2026-06-09)

> Overwrite-latest report (most recent significant work only). Cumulative history lives in [CHANGELOG.md](CHANGELOG.md).

## Outcome

**Slice 6 is complete and live-validated; pushed as `v0.6.0`.** Phase B adds the PBS offsite
tier (doc 03 §8). The PBS spike made it small: backup-to-PBS and restore-from-PBS reuse Phase
A **unchanged** (PBS is just a storage target + a volid), and the operator token needs no
widening. The only new agent code is **the verify capability + a small PBS-API client +
PBSSnapshot reporting**. Escrow + host-loss DR remain slices 7/10.

## What landed

- **`internal/pbs` — the PBS-API client** (the agent's SECOND privileged external surface,
  slice-1 discipline): TLS **fingerprint-pinned** to the PBS leaf cert (a wrong fingerprint →
  rejected — unit-tested against a fake TLS server), **token auth**
  (`PBSAPIToken=<id>:<secret>`; id from the storage `username`, secret read at runtime from
  `/etc/pve/priv/storage/<id>.pw`, **never logged** — unit-asserted on the verify loop), typed,
  no shell. `Verify` (POST, **`ignore-verified=false`** so it actually re-reads), `Snapshots`,
  `TaskStatus`/`WaitVerify` (**node from the UPID** — the spike B4 gotcha), `NodeFromUPID`.
- **The verify maintenance loop** (`pbs/verify.go`) — the cheap, key-free, ciphertext-level
  integrity check (§8) on its own cadence (default 6h, the 5th daemon goroutine). NOT
  gated/journaled — maintenance/reporting like the slice-5 watchdog. Verify → poll → re-list →
  record per-snapshot `verify_state`; a failed verify is logged loudly.
- **`PBSSnapshot` reporting** filled (namespace/type/id/time/size/owner/protected/`encrypted`
  (from `files[].crypt-mode`)/`verify_state`/`verify_upid`). `PBSReporter` collector seam +
  in-memory `SnapshotStore`. Cross-repo golden (byte-identical) + bidirectional key-set tests;
  hub `handler.go` parses `pbs_snapshots` + logs a **failed-verify `[WARN]`** (deployed v0.7.3).
- **Truthful backup mode** — `Backup.mode` now reports the ACTUAL vzdump mode from the task
  log (PVE downgrades snapshot→stop for a stopped guest). `proxmox.Storage.Username`. config
  `PBSVerifyCadence`/`PBSSecretDir`. **`--selftest=pbs-verify`**.

## Tests

`go test ./...` green; **`go test -race ./...` green on the build server** (the new verify
goroutine). PBS client: **fingerprint-pin enforcement** (wrong→rejected, right→connects),
token-header construction + **never-logged** (loop log captured, secret asserted absent),
`Verify` POST→UPID, `Snapshots`→`PBSSnapshot` (incl. crypt-mode→`encrypted`, absent
verification→`none`), `NodeFromUPID`, verify cadence (records ok/failed, disabled-by-negative),
mode-actual parse, cross-repo golden + hub ingest.

## Live PBS runbook (reusing the spike's DooPlex PBS)

- **A — backup → PBS**: `--selftest=backup -vmid 9001` → `felhom-pbs:backup/ct/9001/…`,
  success, and **`mode:"stop"`** — the truthful-mode parse caught PVE's downgrade live.
- **B — verify → ok**: `--selftest=pbs-verify` triggered the verify, polled the task (node
  `dooplex` from the UPID), re-listed → all snapshots `verify_state:"ok"`.
- **C — restore-test from PBS**: `--selftest=restore-test` picked the newest PBS snapshot
  (`source_tier:"pbs"` — derived from the storage type), restored → net-link-down → boot →
  verified `running` → torn down, pass, no leak.
- **D — forced failed verify**: corrupted a chunk in the datastore on DooPlex → first re-verify
  still showed `ok` because PBS's `ignore-verified=true` default **skips** already-verified
  snapshots. **Fix landed** (`Verify` now sends `ignore-verified=false`) → re-verify re-read the
  chunk, detected the corruption: all snapshots `verify_state:"failed"`, agent logged
  `ERROR … FAILED-verify snapshots`. Then cleaned up: forgot the corrupted snapshots, GC'd the
  chunk, took a fresh backup, final verify → **ok**. Datastore left healthy.

Two gaps the runbook surfaced were fixed live (committed): the `ignore-verified` default (else
corruption is never caught) and the hardcoded `source_tier` (now derived from the storage type).

## Not done (flagged)

- **End-to-end hub reflection of `pbs_snapshots`** (daemon → `hub.felhom.eu`): the hub side
  (v0.7.3) is deployed and its parse + failed-verify WARN are unit-tested, but the live
  daemon→hub round-trip needs the demo host **enrolled** on the operator hub (a per-host key) —
  the same one-time enrollment deferred in slices 5/6A. **Offered, not done unprompted.**

## End state / left in place

PBS server on DooPlex + the `felhom-spike` datastore (one healthy, verified snapshot) + the
N100's encrypted `felhom-pbs` storage — all left up. Agent on the N100 is **v0.6.0**; config
reverted (`backup.local_backup_target=local`, pbs verify cadence disabled). No leftover guests.
No secrets committed — the token secret + encryption key live only under
`/etc/pve/priv/storage/` (0600), referenced by location/fingerprint.

## Next

Slice 7 — provisioning + identity-reset + the golden base (§9): the unified bring-up
primitive; restore-overwrite + decommission executors the gate already guards; escrow +
host-loss DR.