felhom.eu/documentation/audits/SPIKE-restic-password-custody-2026-07-09.md

SPIKE — fork-4 restic-offsite password custody (escrow-under-R + controller→agent hand-off)

Date: 2026-07-09 · Class: spike (crypto/mechanism validation before the fork-4 impl) · Executor: Claude Code live, Viktor supervising the 🛑 real-password read. No production code shipped; all probes were throwaway, discarded. Validates the custody design for the restic-offsite repo password: it rides the existing customer-recovery-code (R) zero-knowledge escrow (age-under-R), the controller pushes it to the agent for escrow-create, and enabling offsite (re-)issues a fresh R.

---

1. VERDICT

Custody-under-R for the restic offsite repo password is VALIDATED. Anchored on Probe 3 (the money shot): a password recovered from an age-under-R escrow of the real repo password decrypted the real standing demo repo on the Storage Box — it listed the real immich + audiobookshelf snapshots, proving the recovered value is a working repo key, not merely a byte-match. The crypto round-trip is byte-perfect, a wrong R fails closed with zero plaintext, the controller→agent hand-off is additive on an already-proven transport, and the DR pre-place re-injection seam is honored. The fork-4 impl is now mechanical (blueprint §7).

---

2. Probe 1 — crypto round-trip + wrong-R fail-closed ✅

Throwaway go test in felhom-agent/internal/escrow (run on the build host, deleted; git clean), using the generic WrapIdentity/UnwrapIdentity on a throwaway struct that ADDS restic_repo_password — so nothing in the production IdentityBundle was touched:

• Round-trip: a synthetic 256-bit-hex restic password wrapped under a generated R and unwrapped came back byte-identical (len 64), other bundle fields intact. (~2.5 s — real scrypt.)
• Wrong R: UnwrapIdentity(blob, R+"-WRONG") errored and emitted ZERO plaintext bytes (fails closed at the scrypt KDF — never a plausible-but-wrong password); the right R still recovered the same blob.

3. Probe 2 — controller→agent hand-off shape (source review) ✅

The controller→agent local API already carries authenticated POST-with-JSON-body extensively — POST /backup, /disks/format ({device,fstype}), /controller/swap ({image}), /disks/decommission ({where}), /disks/eject, /disks/assign — all withGuest auth+scoped. A password stage endpoint is additive on this proven channel, not new transport. Staging-hygiene shape to implement (the new part): the agent receives the pushed password, holds it transient (in-memory or a 0600 scratch), feeds it to escrow-create once, then wipes it; the value is never logged (log the field NAME only, mirroring escrow.AttachWGKey's discipline). No live throwaway endpoint was needed — the risk is staging hygiene, a design property stated here, not a transport to prove.

4. Probe 3 — THE MONEY SHOT: recovered password decrypts the REAL repo ✅ 🛑

Non-destructive; the live password file and repo were only read. Flow (hashes only — no value ever printed):

1. 🛑 Real password P piped file-to-file from the container's offbox/repo_password (64 bytes) to a 0600 scratch on the build host (exercises the guest→host data flow the real hand-off will carry).
2. Throwaway harness (real escrow.WrapIdentity): escrow P under a fresh R → blob; simulate DR → UnwrapIdentity(blob, R) = P'. sha256(P) == sha256(P') = 24c44f49… (len 64).
3. P' moved to a 0600 container scratch; ran the exact offbox restic … snapshots (the real sftp.command: ssh u629193-sub1@…-p 23 …-i offbox/ssh_key -s sftp, repo sftp:…:/home/felhom-demo-repo) with RESTIC_PASSWORD_FILE=P':

   14a3bbba  audiobookshelf  /mnt/felhom-drives/felhom-usb/backups/primary/audiobookshelf
   7ba512bf  immich          /mnt/felhom-drives/felhom-usb/backups/primary/immich
   2 snapshots
   

   The recovered password opened the real repo. This is the acceptance criterion.

5. Probe 4 — DR re-injection seam ✅ (+ a gap to close in the impl)

Throwaway go test in felhom-controller/internal/backup: pre-place a known "recovered" password at offbox/repo_password, then call WriteOffboxSecrets(...) → the file was byte-unchanged (the os.Stat … IsNotExist guard means gen only runs when absent). So a DR flow can pre-place the recovered password and the controller uses it, never overwrites. Gap for the impl: the config endpoint (POST /backup/offbox/config) carries ssh_key + known_hosts but NOT the repo password (auto-gen or pre-place only). So DR-restore needs an explicit inject step that writes the 0600 password file before offbox config runs (or a small POST /backup/offbox/inject-password). Pre-placing is honored; there is just no endpoint for it today.

6. Probe 5 — hygiene / entropy ✅

Repo password is 256-bit (generateOffboxPassword = 32 rand bytes → 64 hex; confirmed 64-byte file). age -p uses scrypt → a wrong R is expensive (the wrong-R probe took ~3.8 s of KDF, not a cheap brute). No password value was logged anywhere in the spike — only sha256 hashes and field names.

7. Impl blueprint (for the fork-4 spec)

• Agent — add ResticRepoPassword string \json:"restic_repo_password,omitempty"`toIdentityBundle(rides the existingWrapIdentityBundle/UnwrapIdentityBundleage-under-R path — Probe 1 proves it carries). Add a **stage endpoint** on the local API (controller-push;withGuestauth; hold transient, feed escrow-create once, wipe, never log the value). Re-add the reservedrestic_repo_coord as **DRResticCoord** in dr_recipe.go— Host/User/Port/RepoPath/host-key-fingerprint, **coordinates ONLY** (per the never-a-secret boundary + the_NoSecretstest), mirroringDRPBSCoord`.
• Controller — at offsite-enable: push the repo password to the agent, then trigger escrow-create (fresh R). DR-restore path pre-places the recovered password before WriteOffboxSecrets (Probe 4), via the explicit inject step (Probe 5 gap).
• Orchestration — offsite-enable → controller-push → agent escrow-create → fresh R surfaced to the customer once (supersedes any prior R) → opaque R-blob to the hub (reuse the existing K-blob storage). Because R is never retained, the password MUST enter the escrow at this ceremony — no retro-fit (same constraint as the S3 WG key).
• Hub — stores only the opaque R-blob (zero-knowledge preserved) + the non-secret restic_repo_coord.
• DR recovery flow — recover R → UnwrapIdentityBundle → password; read restic_repo_coord; operator re-provisions SFTP access (fresh sub-account key — access is regenerable, deliberately NOT escrowed); pre-place password → configure offbox → restore.

8. NOT done / boundaries

• The SFTP key + sub-account credential are ACCESS (regenerable at DR via a fresh console sub-account + key install) → deliberately NOT escrowed; only the repo password (irreplaceable — it's the data key) and the non-secret coordinates are.
• The real PBS key K and proxmox-backup-client were not involved — the restic password rides the generic age-under-R path, no K needed.
• Multi-customer provisioning + soft-quota, and the hub-driven both-models config, are separate tasks.
• This spike does not build any of the blueprint — it validates the mechanisms the fork-4 impl spec will wire.

9. Teardown state (clean)

All throwaway probes deleted (agent internal/escrow, controller internal/backup — both git-clean, nothing committed). All scratch secrets shredded (P/P'/blob/R on the build host; P' in the container). The live offbox/repo_password is untouched (still 64 bytes, original timestamp — only ever read) and the demo repo is intact (2 snapshots, as listed in Probe 3). Only deliverable committed: this findings doc.