felhom.eu/REPORT-break-the-circle-2026-10-09.md

# REPORT — break the circle: the password manager off-site, every recovery key listed, one sheet to print (2026-10-09)

| Part | What | Result |
|---|---|---|
| A | Vaultwarden's data in the nightly off-site copy | DONE, LIVE (operator yes in chat) — run by hand, restore test, throwaway start |
| B | Recovery-key inventory + printable sheet + paper walk | DONE — `runbooks/total-loss-of-dooplex.md`, `runbooks/break-glass-sheet.md` |
| C | R-922 A, a quiet restored hub, R-921 | BUILT, committed, NOT released (hub + controller); R-921 partly |

**Register: before 135 · after 138 · opened 2 (R-923, R-924) · closed 0.** A parallel session added R-925 meanwhile.

**Rulings recorded first:** (1) the operator's finding → **R-923** (P2); "off DooPlex" corrected in the hub-DB runbook
(Step 0 and §3), `gitea-restore.md`, `secrets.md`, STATUS; (2) R-922 → option A, recorded in the row.

## Part A — Vaultwarden (`vaultwarden-system`, 1.37.3, 7.4 MB)

- **Method:** Vaultwarden's own `vaultwarden backup` (SQLite `VACUUM INTO`, consistent while it runs; its backup
  guidance), plus `rsa_key.pem` and — when present — `attachments/`, `sends/`, `config.json`. The one backup file it
  writes into `/data` is copied out and removed (also on refusal). Same archive, key and write-only token as Gitea.
- **Live:** first run 2026-10-09T10:02Z: `vaultwarden: integrity ok, 1 user(s)`, pushed in 13 s; `/data` clean after.
- **Restore test (Sunday job, run by hand):** 27 969 files match, 10 repos pass `git fsck`, `Vaultwarden 1 user(s) / 797 item(s)`.
- **Throwaway (bench 9401):** row counts only — users 1, ciphers 797, folders 5, **twofactor 0**, sso_users 1; a
  Vaultwarden 1.37.3 started on the copy, `/alive` 200, no route out. Deleted (0 containers, 0 volumes, bench stopped);
  DooPlex scratch shredded. No login, no item opened, the master password never asked for.
- Tests: 26 (6 new), green with GNU, BusyBox and the fake `sqlite3`; 4 red-proofs (`audits/dooplex-survival-2026-10-09/vaultwarden/`).
  The tests found a defect before the live run (the optional-file listing failed when the last file was absent).
- Seen, not changed: Vaultwarden warns its `ADMIN_TOKEN` is plain text (in R-923).

## Part B — what a total loss needs

18 secrets/logins listed with where each lives today. **Exists only on DooPlex (lost with it):** S1 the DooPlex
off-site key, S2 the hub-DB key (otherwise only in Vaultwarden, which is behind S1), **S6 the signing keys (in no
backup at all, no passphrase — R-924)**, S7 the restore token (re-mintable). **Circles:** S8 Hetzner and S11 Cloudflare
logins if kept only in Vaultwarden. Checked: the nightly Secrets export opens with the DooPlex restic passphrase (S5) and
holds all eight Secrets the hub uses (names only read). **Paper walk:** with a filled sheet and the master password,
no Felhom recovery step needs DooPlex; without it, S1, S6 and S8 block.

## Part C — code (waits for the next release)

- **Hub** (`d55c590c`, `e5b9151e`, CI 1605 and 1606 success): R-922 A (`email_cleared` deletes the notification address;
  F12 guard kept); **MAIL-HOLD** (`<data_dir>/MAIL-HOLD` → no mail on any path, dropped not queued, banner, release
  button) — now step 1 of the hub restore runbook; two log lines no longer print the address; **SECURITY:**
  `/preferences` and `/notify` accepted any box's key for any household — now 403 (red-proved).
- **Controller** (`ec9b997`, CI 1607 success): R-922 sends `email_cleared` after a household's clear; R-921 pre-check —
  no stop while another tier's job is in flight. **Not covered:** the agent's host-wide busy lock (the case measured on
  demo-hp) — no agent endpoint serves it; needs an agent field (R-921 LEFT).
- **Release order:** hub first, then the controller (the controller sends a field an older hub ignores).
- Built by two helpers under this brief's fences; reviewed, tested again and committed here. One stash of mine briefly
  held a helper's uncommitted work (about a minute; nothing lost).

## Teardown

Machine: bench 9401 — 0 containers, 0 volumes, image removed, stopped as it was. Host: DooPlex scratch dirs shredded.
Hub: provisioned nothing. Vaultwarden: its one test backup file removed by name.

## Done after the report: R-924 option A (operator, 2026-10-09)

The three signing key files and their public halves ride the nightly copy (felhom.eu `96f68f1b`, CI 1614 success). Live
run 12:57Z: `signing: 3 key(s) with their public halves`, pushed in 22 s; restore test: `3 signing key(s) match`
(`audits/dooplex-survival-2026-10-09/signing/`). 4 new tests, 3 red-proofs. R-924 narrowed: left is printing them.

## Decisions for the operator (as asked; 1 is answered — A)

1. **R-924 — the signing keys.** (A) Print both on the sheet AND add them to the nightly encrypted copy (CC changes the
   DooPlex job with your yes). (B) Print only the recovery key and keep it away from home. **Pick A.** **If you do
   nothing:** after a loss of DooPlex no box accepts a signed update, bundle or OS step until it is re-enrolled on site.
2. **Print the sheet** (`runbooks/break-glass-sheet.md`, commands at its top, run from your workstation). **If you do
   nothing:** the off-site copies exist but cannot be opened after a loss of DooPlex.