dooplex-offsite: the operator signing keys ride the nightly encrypted copy; restore test proves each key matches its public half (R-924, operator ruling option A)
gates / gates (push) Successful in 6m7s
gates / gates (push) Successful in 6m7s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -111,7 +111,9 @@ operational key until the first PAYING customer exists; testers do not count.**
|
||||
per box (the blob binds `host_id`), so one box moves at a time and a fenced box cannot be swept along.
|
||||
Proven twice: `demo-hp-bb76ea` 2026-09-15, `demo-felhom-8363b5` 2026-09-16. **Measured 2026-10-09 (R-924):** all three
|
||||
key files open with an EMPTY passphrase, and `/mnt/5_hdd/felhom.eu` is in no backup — so today neither the "passphrase-
|
||||
protected" nor the "kept offline" of §1 holds, and a loss of DooPlex loses both keys at once. **Revisit on the first
|
||||
protected" nor the "kept offline" of §1 holds, and a loss of DooPlex loses both keys at once. **Operator ruling 2026-10-09 (R-924, option A):** both keys now ride the
|
||||
nightly encrypted DooPlex→ep0 copy (behind the DooPlex off-site key) and go on the printed break-glass sheet; the
|
||||
recovery key is still not passphrase-protected. **Revisit on the first
|
||||
sale** — at that point the key belongs behind the operator (or a hardware key, §7), and a fleet
|
||||
rollout step still has to be designed (R-530).
|
||||
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
### RED: missing-key refusal removed
|
||||
FAIL: test_a_missing_signing_key_refuses (__main__.Push.test_a_missing_signing_key_refuses)
|
||||
Ran 1 test in 0.540s
|
||||
FAILED (failures=1)
|
||||
### RED: restore: key/public-half match removed
|
||||
FAIL: test_restore_refuses_a_signing_key_that_does_not_match (__main__.RestoreTest.test_restore_refuses_a_signing_key_that_does_not_match)
|
||||
Ran 1 test in 0.763s
|
||||
FAILED (failures=1)
|
||||
### RED: signing step skipped
|
||||
FAIL: test_signing_keys_ride_the_copy (__main__.Push.test_signing_keys_ride_the_copy)
|
||||
Ran 1 test in 0.521s
|
||||
FAILED (failures=1)
|
||||
### GREEN (GNU)
|
||||
Ran 30 tests in 43.420s
|
||||
OK
|
||||
### GREEN (BusyBox tools + fake sqlite3)
|
||||
Ran 30 tests in 44.500s
|
||||
OK
|
||||
@@ -196,7 +196,7 @@ stopping line that lies.
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| **R-924** | Security & access | P2 | **The operator signing keys exist only on DooPlex, with no passphrase and in no backup — a loss of DooPlex means no box ever accepts a signed agent update, config bundle or OS step again without an on-site re-enrollment.** MEASURED 2026-10-09: `/mnt/5_hdd/felhom.eu/felhom-op-operational`, `felhom-rec-recovery`, `felhom_op_ed25519` (0600 `kisfenyo`); `ssh-keygen -y -P ''` opens all three (no passphrase); `/mnt/5_hdd/felhom.eu` is in no backup set, the off-site job does not carry them. `04` §1 says the operational key is "passphrase-protected" and the recovery key "kept offline" — today neither holds (custody ruling 1 of 2026-09-16 put both on DooPlex for this phase; `04` §3.1). Both keys share one fate, so the cold key cannot do its one job (re-pin after losing the operational key). `runbooks/total-loss-of-dooplex.md` S6. | **WAITING-ON-OPERATOR** — custody: print both (the break-glass sheet has the lines and commands) and/or add them to the nightly encrypted copy (a DooPlex change), or move the recovery key off DooPlex entirely. | operator decision | Operator: pick; CC: the copy change if picked | operator |
|
||||
| **R-924** | Security & access | P2 | **The operator signing keys exist only on DooPlex, with no passphrase and in no backup — a loss of DooPlex means no box ever accepts a signed agent update, config bundle or OS step again without an on-site re-enrollment.** MEASURED 2026-10-09: `/mnt/5_hdd/felhom.eu/felhom-op-operational`, `felhom-rec-recovery`, `felhom_op_ed25519` (0600 `kisfenyo`); `ssh-keygen -y -P ''` opens all three (no passphrase); `/mnt/5_hdd/felhom.eu` is in no backup set, the off-site job does not carry them. `04` §1 says the operational key is "passphrase-protected" and the recovery key "kept offline" — today neither holds (custody ruling 1 of 2026-09-16 put both on DooPlex for this phase; `04` §3.1). Both keys share one fate, so the cold key cannot do its one job (re-pin after losing the operational key). `runbooks/total-loss-of-dooplex.md` S6. | **NARROWED 2026-10-09 — operator ruling: option A** (print both AND copy them). The three key files and their public halves ride the nightly encrypted DooPlex→ep0 copy (`scripts/dooplex-offsite/`, a missing key refuses the push); the Sunday restore test proves each private key still derives its own public half (never printed). **LEFT:** the operator prints S6a/S6b (`runbooks/break-glass-sheet.md`); the keys still have no passphrase (custody ruling 1 of 2026-09-16 stands until the first sale). | operator: print | Operator: print the two keys; then close | operator |
|
||||
| **R-132** | Security & access | P3 | **`curl -w '%{redirect_url}'` reconstructs the request URL WITH its basic-auth credential** — so a `-u ":$HUB_PW"` call that never put the password in a URL still printed it **Merged 2026-10-05 from R-350 (duplicate):** (1) 2026-08-20 occurrence: POST /configuration/artifacts answers 303; leak lives only in the CC transcript under ~/.claude/projects/ on DooPlex, not in git/evidence (checked then). (2) `-v` and `--libcurl` also re-render the credential, not only %{redirect_url}; confirm redirects with %{http_code} + follow-up GET. (3) Rotation path: hub /configuration form (current_password/new_password/confirm_password); DB override wins over ConfigMap (break-glass); CC can rotate file-to-file without printing (operator-present-one-time-secrets) if asked. | **WAITING-ON-OPERATOR** — **ACTION: rotate `HUB_PW`** **Folded R-580 2026-10-03** (the same `curl -w %{redirect_url}` credential echo, seen again 2026-09-18). | — | Happened on 2026-07-31 while red-proofing the R-120 gate: the hub operator password was written to the session transcript by the write-out format, not by the request. `-u` is safe; the *reporting* was not. Rule: read the redirect from `-D -` and grep `^Location:`, never `%{redirect_url}`, on any authenticated call. Rotate the hub password (`/configuration` → Login password; ConfigMap `auth.password_hash` is the reset path) and update `~/.config/credentials` | Viktor |
|
||||
| **R-137** | Security & access | P3 | **Cloudflare geo-WAF rules are zone-scoped and non-namespaced — four cross-tenant faults.** `globalRuleDesc = "[felhom-geo] Global"` (`waf.go:18`) is one literal description per ZONE; `appRuleDescPrefix` keys by app name with no customer (`waf.go:21`); `BuildGlobalExpression` has no positive hostname scoping (`waf.go:241`); `applyDiff` deletes every `[felhom-geo]` rule not in THIS box's desired set (`geosync.go:320`) | READY (M) — **blocks shared-zone onboarding** | — | With two customers in one zone: they overwrite each other's Global rule forever; one customer's country policy applies zone-wide; per-app rules collide by name; and disabling the feature for one (or the hub's `RemoveGeoRules`) wipes them all. Interim mitigation, no code: keep geo-restriction OFF for every shared-zone customer. Fix = namespace descriptions by `customer_id` + add `http.host ends_with "<domain>"` to both expressions — a TWO-REPO change (controller + hub `RemoveGeoRules`). Same audit §5.1 | CC |
|
||||
| **R-255** | Security & access | P3 | **The check that would catch a fourth secret-in-the-body covers 4 of 27 pages, and the cheap gate that covers all 36 templates is blind to the shape that actually shipped.** Filed 2026-08-08 while closing R-254, **because a partial guard reported as complete is worse than no guard — it stops the next person looking.** **Two nets, both measured.** **(1) `scripts/secret_in_markup_gate.py`** reads all 36 templates and convicts any `{{ … }}` naming a secret unless allowlisted with a reason. It catches `{{.RetrievalPassword}}` and `{{.InitialCreds.Password}}`, **and it catches a launder through a local variable** because the assignment itself names the secret (`{{$v := .InitialCreds.Password}}` is convicted — verified). **It is blind to a secret arriving under a NEUTRAL PAGE-DATA KEY** — `data["Tagline"] = creds.Password` then `{{.AppInfo.Tagline}}` passes it cleanly, also verified. **That is exactly the shape of R-254 site two** (`value="{{$val}}"` inside an `{{if eq .Type "secret"}}` branch), so the gate **would not have caught one of the three instances it was written for.** **(2) The runtime body assertion** — render the page and grep the response for a sentinel — catches every shape, including that one (demonstrated on the same planted leak the gate missed). But it needs each page's data to be constructible in a test, and **only 4 of 27 page templates have that today**: `settings_security`, `app_info`, `deploy`, `backups_restore` — the four that were touched by R-249/R-252/R-253/R-254 and therefore got their own tests. **The other 23 pages have no runtime coverage at all.** **What closing this needs, so the cost is not re-estimated:** a per-page data fixture for the remaining 23 (most need a wired `Server` — `stackMgr`, `backupMgr`, agent seams), then one table-driven test that renders each with a sentinel substituted for every string in its data and asserts the sentinel is absent. **That is real scaffolding, which is why it was NOT built inside R-254's session** rather than half-built and declared done. | **READY** — owner Viktor | — | — | operator |
|
||||
|
||||
@@ -45,7 +45,7 @@ from the printed `data` field (hub-DB runbook Step 0 note) — `key show` cannot
|
||||
| S2 | Hub-DB off-site key — opens the hub database (ep0 `operator`, `host/dooplex-hub`) | PBS key `b2:19:bf:36:3b:97:3d:6c…` | `data`: ______________________________________________ |
|
||||
| S4 | Hub seal key `OFFSITE_SECRET_KEY` (64 hex) | — | ______________________________________________________ |
|
||||
| S5 | DooPlex restic / Secrets-export passphrase | — | ______________________________________________________ |
|
||||
| S6a | Signing RECOVERY key `felhom-rec-recovery` | `SHA256:/ixgTesZqykAGJpFUUd4kLAiHFgKOkYFLNC3AQXWP+k` | (staple the printout) |
|
||||
| S6a | Signing RECOVERY key `felhom-rec-recovery` (also in the S1 copy since 2026-10-09; the paper is the copy that needs nothing else) | `SHA256:/ixgTesZqykAGJpFUUd4kLAiHFgKOkYFLNC3AQXWP+k` | (staple the printout) |
|
||||
| S6b | Signing OPERATIONAL key `felhom-op-operational` | `SHA256:7YqN4rXO08yixTeOO+UtQ8jHyIGycICuctQgRYVGnWw` | (staple the printout) |
|
||||
| S7 | ep0 read-only token `dooplex-hub@pbs!restore` | — | ______________________________________________________ |
|
||||
| S8 | Hetzner account: login e-mail · password · two-factor recovery codes | — | ______________________________________________________ |
|
||||
|
||||
@@ -20,8 +20,7 @@ build tree. **Assume all of it is gone, and the house with it.**
|
||||
| The operator's workstation (if it has them) | the WireGuard peer to ep0 (`10.77.0.250`), maybe one of ep0's two root SSH keys, maybe a Bitwarden client's offline vault cache | the operator |
|
||||
|
||||
**Lost with DooPlex and NOT in any copy** (measured 2026-10-09): the container registry (rebuild the images from
|
||||
code); **the operator signing keys** `felhom-op-operational`, `felhom-rec-recovery`, `felhom_op_ed25519`
|
||||
(`/mnt/5_hdd/felhom.eu/`, no passphrase, no backup — R-924); DooPlex's own SSH key (one of ep0's two root keys);
|
||||
code); *(the operator signing keys were here until 2026-10-09 afternoon; they now ride the nightly copy — R-924)*; DooPlex's own SSH key (one of ep0's two root keys);
|
||||
the build tree and drills (`/mnt/5_hdd/felhom.eu/{build,drills}`); `.claude-memory` (same-disk restic only); the
|
||||
local restic repos; Prometheus history; DooPlex's other homelab apps (not Felhom; Longhorn on DooPlex only).
|
||||
|
||||
@@ -38,7 +37,7 @@ local restic repos; Prometheus history; DooPlex's other homelab apps (not Felhom
|
||||
| **S3** | **Vaultwarden master password** (user count 1; **no two-factor rows**, SSO-linked to Authentik but password login allowed) | every item in the password manager | the operator's head | no — if remembered |
|
||||
| **S4** | **Hub seal key** `OFFSITE_SECRET_KEY` | the hub opening its console passwords and off-site passwords | DooPlex (`Secret/offsite-secret-key`); inside the Secrets export (behind S1 + S5); Vaultwarden | yes unless on paper or S1+S5 work |
|
||||
| **S5** | **DooPlex restic/GPG passphrase** (`/etc/backup/restic-password`) | opening the nightly Secrets export (`secrets/*.gpg`: every k8s Secret — Resend, report-api, Hetzner token, the hub's SSH keys, gitea-creds, TLS …) | DooPlex (`sdb1`); "an offline copy, out of band" (operator, recon 2026-08-06 — form unknown to CC). **Checked 2026-10-09:** the newest export opens with it (symmetric GPG, AES256) and holds 428 Secrets, among them all eight the hub uses (names only read) | depends on that offline copy |
|
||||
| **S6** | **Signing recovery key** `felhom-rec-recovery` (and the operational key `felhom-op-operational`) | signing agent updates / config bundles / OS steps for every box; the recovery key authorizes rotating the operational key | **DooPlex only**, no passphrase, in no backup | **YES** — then every box needs on-site re-enrollment to accept a new key (`04` §4) |
|
||||
| **S6** | **Signing recovery key** `felhom-rec-recovery` (and the operational key `felhom-op-operational`) | signing agent updates / config bundles / OS steps for every box; the recovery key authorizes rotating the operational key | DooPlex, no passphrase; **since 2026-10-09 (R-924 option A) also in the nightly copy `host/dooplex-gitea` (behind S1)** and, once printed, on the sheet | no, if S1 or the printout survives — otherwise every box needs on-site re-enrollment (`04` §4) |
|
||||
| **S7** | **ep0 read-only restore token** `dooplex-hub@pbs!restore` | reading both copies without ep0 root | DooPlex only (`/etc/felhom-hub-backup/token-restore`) | yes — but ep0 root can mint a new one (hub-DB runbook Step 2) |
|
||||
| **S8** | **Hetzner account login + its two-factor** | ep0's console (rescue, root reset), ep0 itself, Storage Boxes, the Hetzner API token | the operator; probably Vaultwarden | **circle**: if only in Vaultwarden, it is behind the copy it is needed to reach — paper or head |
|
||||
| S9 | ep0 root SSH | the PBS tunnel; minting tokens | ep0 `authorized_keys`: DooPlex's key (lost) + one more key, owner not recorded (operator workstation?) | partly — Hetzner console (S8) is the fallback |
|
||||
@@ -72,7 +71,7 @@ local restic repos; Prometheus history; DooPlex's other homelab apps (not Felhom
|
||||
(§3 step 1: the `MAIL-HOLD` marker, from the next hub release; until then: no network until the pending notices are understood).
|
||||
9. **DNS:** in Cloudflare (S11) point `hub.felhom.eu` (today a CNAME to `dooplex.hopto.org`) and `gitea.dooplex.hu` at
|
||||
the new place. The boxes reconnect by themselves: their API keys are in the restored hub DB.
|
||||
10. **Signing:** with S6 on paper, put the keys back and sign as before. Without it, no box accepts an agent update or a
|
||||
10. **Signing:** the keys are in the restored copy (`signing/`) and on the sheet (S6) — put them back and sign as before. Without it, no box accepts an agent update or a
|
||||
bundle until it is re-enrolled on site (`04` §4).
|
||||
11. **Backups again:** reinstall `scripts/hub-db-backup/` and `scripts/dooplex-offsite/` from the restored code, the
|
||||
tunnel and the tokens, and check the next night's copies.
|
||||
|
||||
@@ -1,3 +1,10 @@
|
||||
## 2026-10-09 (late afternoon) — dooplex-offsite carries the operator signing keys (R-924, operator ruling option A)
|
||||
|
||||
- `felhom-dooplex-offsite`: `felhom-op-operational`, `felhom-rec-recovery`, `felhom_op_ed25519` and their `.pub` (from
|
||||
`/mnt/5_hdd/felhom.eu/`) ride the nightly encrypted copy under `signing/`; a missing key or a link refuses the push.
|
||||
- `felhom-dooplex-offsite-restore-test`: each private key must derive its own public half (`ssh-keygen -y`), never printed.
|
||||
- 4 new tests (30 total), green with GNU and BusyBox tools; 3 red-proofs in `audits/dooplex-survival-2026-10-09/signing/`.
|
||||
|
||||
## 2026-10-09 (afternoon) — dooplex-offsite carries the password manager too (R-923)
|
||||
|
||||
- `felhom-dooplex-offsite`: a new step runs Vaultwarden's own `vaultwarden backup` (SQLite `VACUUM INTO`, consistent
|
||||
|
||||
@@ -18,6 +18,8 @@ STATE=${FELHOM_DXOFF_STATE:-/var/lib/felhom-dooplex-offsite}
|
||||
TEXTFILE_DIR=${FELHOM_DXOFF_TEXTFILE_DIR:-/var/lib/node_exporter/textfile_collector}
|
||||
DUMPS=${FELHOM_DXOFF_DUMPS:-/mnt/5_hdd/backup/postgresql/dumps}
|
||||
SECRETS=${FELHOM_DXOFF_SECRETS:-/mnt/5_hdd/backup/secrets/exports}
|
||||
SIGNING=${FELHOM_DXOFF_SIGNING:-/mnt/5_hdd/felhom.eu}
|
||||
SIGNING_KEYS="felhom-op-operational felhom-rec-recovery felhom_op_ed25519"
|
||||
DUMP_MAX_AGE_H=${FELHOM_DXOFF_DUMP_MAX_AGE_H:-7}
|
||||
SECRETS_MAX_AGE_H=${FELHOM_DXOFF_SECRETS_MAX_AGE_H:-30}
|
||||
NOW=${FELHOM_DXOFF_NOW:-$(date +%s)}
|
||||
@@ -32,10 +34,11 @@ V() { kubectl -n vaultwarden-system exec deploy/vaultwarden -- "$@"; }
|
||||
umask 077
|
||||
STAGE="$STATE/stage"
|
||||
mkdir -p "$STATE"; chmod 700 "$STATE"
|
||||
rm -rf "$STAGE"; mkdir -p "$STAGE/root/gitea" "$STAGE/root/db" "$STAGE/root/secrets" "$STAGE/root/vaultwarden"
|
||||
rm -rf "$STAGE"; mkdir -p "$STAGE/root/gitea" "$STAGE/root/db" "$STAGE/root/secrets" "$STAGE/root/vaultwarden" "$STAGE/root/signing"
|
||||
cleanup() {
|
||||
for f in "$STAGE/root/gitea/gitea/conf/app.ini" "$STAGE/root/db/gitea.dump" "$STAGE/root/db/globals.sql" \
|
||||
"$STAGE/root/vaultwarden/db.sqlite3" "$STAGE/root/vaultwarden/rsa_key.pem"; do
|
||||
"$STAGE/root/vaultwarden/db.sqlite3" "$STAGE/root/vaultwarden/rsa_key.pem" \
|
||||
"$STAGE/root/signing/felhom-op-operational" "$STAGE/root/signing/felhom-rec-recovery" "$STAGE/root/signing/felhom_op_ed25519"; do
|
||||
[ -f "$f" ] && [ ! -L "$f" ] && { shred -u "$f" 2>/dev/null || rm -f "$f"; }
|
||||
done
|
||||
rm -rf "$STAGE"
|
||||
@@ -113,6 +116,17 @@ VUSERS=$(sqlite3 -readonly "$STAGE/root/vaultwarden/db.sqlite3" 'SELECT COUNT(*)
|
||||
echo "$VUSERS" > "$STAGE/root/vaultwarden/USERS"
|
||||
log "vaultwarden: integrity ok, $VUSERS user(s), files: db.sqlite3 $(echo $VLIST)"
|
||||
|
||||
# 3c. the operator signing keys (R-924, operator ruling 2026-10-09: option A — on paper AND in this copy). Without them
|
||||
# no box accepts a signed agent update, config bundle or OS step after a loss of DooPlex. Each private key and its .pub;
|
||||
# refuses when one is missing or is a link (a missing key would otherwise go unnoticed until the day it is needed).
|
||||
for k in $SIGNING_KEYS; do
|
||||
for f in "$k" "$k.pub"; do
|
||||
[ -f "$SIGNING/$f" ] && [ ! -L "$SIGNING/$f" ] || die "signing key file $f missing (or a link) in $SIGNING"
|
||||
cp "$SIGNING/$f" "$STAGE/root/signing/$f"
|
||||
done
|
||||
done
|
||||
log "signing: $(echo $SIGNING_KEYS | wc -w | tr -d ' ') key(s) with their public halves"
|
||||
|
||||
# 4. the manifest the restore test checks, then the push
|
||||
echo "$GOT" > "$STAGE/root/REPOS"
|
||||
(cd "$STAGE/root" && find . -type f ! -name MANIFEST.sha256 -print0 | sort -z | xargs -0 sha256sum > MANIFEST.sha256) \
|
||||
|
||||
@@ -21,7 +21,7 @@ die() { echo "felhom-dooplex-offsite-restore-test: FAILED: $*" >&2; exit 1; }
|
||||
umask 077
|
||||
mkdir -p "$STATE"; chmod 700 "$STATE"
|
||||
T=$(mktemp -d "$STATE/restore.XXXXXX")
|
||||
trap 'for f in "$T"/out/gitea/gitea/conf/app.ini "$T"/out/db/gitea.dump "$T"/out/db/globals.sql "$T"/out/vaultwarden/db.sqlite3 "$T"/out/vaultwarden/rsa_key.pem; do [ -f "$f" ] && shred -u "$f" 2>/dev/null; done; rm -rf "$T"' EXIT
|
||||
trap 'for f in "$T"/out/gitea/gitea/conf/app.ini "$T"/out/db/gitea.dump "$T"/out/db/globals.sql "$T"/out/vaultwarden/db.sqlite3 "$T"/out/vaultwarden/rsa_key.pem "$T"/out/signing/felhom-op-operational "$T"/out/signing/felhom-rec-recovery "$T"/out/signing/felhom_op_ed25519; do [ -f "$f" ] && shred -u "$f" 2>/dev/null; done; rm -rf "$T"' EXIT
|
||||
export PBS_PASSWORD_FILE="$TOKENS/token-restore" PBS_FINGERPRINT
|
||||
|
||||
LIST=$(proxmox-backup-client snapshot list host/dooplex-gitea --ns operator --output-format json --repository "$PBS_REPOSITORY_RESTORE") \
|
||||
@@ -76,7 +76,15 @@ VUSERS=$(sqlite3 -readonly "$VDB" 'SELECT COUNT(*) FROM users;' 2>/dev/null) ||
|
||||
VITEMS=$(sqlite3 -readonly "$VDB" 'SELECT COUNT(*) FROM ciphers;' 2>/dev/null) || die "cannot count Vaultwarden items"
|
||||
[ "${VUSERS:-0}" -gt 0 ] && [ "$VUSERS" = "$(cat "$O/vaultwarden/USERS" 2>/dev/null)" ] || die "Vaultwarden users: $VUSERS, USERS says $(cat "$O/vaultwarden/USERS" 2>/dev/null)"
|
||||
[ "${VITEMS:-0}" -gt 0 ] || die "the Vaultwarden copy holds no item"
|
||||
log "checked: $FILES files match the manifest, $GOT repositories pass git fsck, gitea.dump readable, $(ls "$O"/secrets | wc -l | tr -d ' ') secrets file(s), Vaultwarden $VUSERS user(s) / $VITEMS item(s)"
|
||||
# The signing keys (R-924): each private key must still derive its own public half — a usable key, never printed.
|
||||
NSIGN=0
|
||||
for k in felhom-op-operational felhom-rec-recovery felhom_op_ed25519; do
|
||||
[ -s "$O/signing/$k" ] && [ -s "$O/signing/$k.pub" ] || die "signing key $k missing from the copy"
|
||||
DER=$(ssh-keygen -y -P '' -f "$O/signing/$k" 2>/dev/null | cut -d' ' -f1,2) || die "signing key $k does not open"
|
||||
[ -n "$DER" ] && [ "$DER" = "$(cut -d' ' -f1,2 "$O/signing/$k.pub")" ] || die "signing key $k does not match its public half"
|
||||
NSIGN=$((NSIGN + 1))
|
||||
done
|
||||
log "checked: $FILES files match the manifest, $GOT repositories pass git fsck, gitea.dump readable, $(ls "$O"/secrets | wc -l | tr -d ' ') secrets file(s), Vaultwarden $VUSERS user(s) / $VITEMS item(s), $NSIGN signing key(s) match"
|
||||
|
||||
TMP="$TEXTFILE_DIR/felhom_dooplex_offsite_restore.prom.$$"
|
||||
{
|
||||
|
||||
@@ -106,6 +106,24 @@ try:
|
||||
except Exception as e:
|
||||
print("Error: " + str(e), file=sys.stderr); sys.exit(1)
|
||||
'''
|
||||
# ssh-keygen stand-in (the CI runner may lack OpenSSH): `ssh-keygen -y -P '' -f <key>` prints the public line a
|
||||
# test key file carries after "PUB:" — the same contract as the real tool (derive the public half from the private).
|
||||
FAKE_SSH_KEYGEN = r'''#!/bin/sh
|
||||
[ "$1" = "-y" ] || exit 97
|
||||
f=""; while [ $# -gt 0 ]; do [ "$1" = "-f" ] && f=$2; shift; done
|
||||
sed -n 's/^PUB://p' "$f" | grep . || { echo "Load key: invalid format" >&2; exit 255; }
|
||||
'''
|
||||
SIGNING_KEYS = ("felhom-op-operational", "felhom-rec-recovery", "felhom_op_ed25519")
|
||||
|
||||
|
||||
def make_signing(d):
|
||||
os.makedirs(d, exist_ok=True)
|
||||
for i, k in enumerate(SIGNING_KEYS):
|
||||
pub = "ssh-ed25519 AAAATEST%d %s" % (i, k)
|
||||
open(os.path.join(d, k), "w").write("-----BEGIN TEST KEY-----\nPUB:%s\n" % pub)
|
||||
open(os.path.join(d, k + ".pub"), "w").write(pub + "\n")
|
||||
|
||||
|
||||
REAL_SQLITE3 = None if os.environ.get("FORCE_FAKE_SQLITE3") else shutil.which("sqlite3")
|
||||
|
||||
|
||||
@@ -159,7 +177,10 @@ class Base(unittest.TestCase):
|
||||
os.makedirs(self.vw)
|
||||
make_vw_db(os.path.join(self.vw, "db.sqlite3"))
|
||||
open(os.path.join(self.vw, "rsa_key.pem"), "w").write("-----TEST KEY-----\n")
|
||||
fakes = [("kubectl", FAKE_KUBECTL), ("proxmox-backup-client", FAKE_PBS), ("pg_restore", FAKE_PG_RESTORE)]
|
||||
self.signing = j("signing")
|
||||
make_signing(self.signing)
|
||||
fakes = [("kubectl", FAKE_KUBECTL), ("proxmox-backup-client", FAKE_PBS), ("pg_restore", FAKE_PG_RESTORE),
|
||||
("ssh-keygen", FAKE_SSH_KEYGEN)]
|
||||
if not REAL_SQLITE3:
|
||||
fakes.append(("sqlite3", FAKE_SQLITE3))
|
||||
for name, body in fakes:
|
||||
@@ -183,7 +204,7 @@ class Base(unittest.TestCase):
|
||||
|
||||
def env(self, **kw):
|
||||
e = dict(os.environ, PATH=self.bin + ":" + os.environ["PATH"], FAKE_POD_DATA=self.pod, FAKE_PBS_DIR=self.pbs,
|
||||
FAKE_STATE=self.t, FAKE_VW_DATA=self.vw, FELHOM_DXOFF_CONF=self.conf, FELHOM_DXOFF_TOKENS=self.tokens,
|
||||
FAKE_STATE=self.t, FAKE_VW_DATA=self.vw, FELHOM_DXOFF_SIGNING=self.signing, FELHOM_DXOFF_CONF=self.conf, FELHOM_DXOFF_TOKENS=self.tokens,
|
||||
FELHOM_DXOFF_STATE=self.state, FELHOM_DXOFF_TEXTFILE_DIR=self.text, FELHOM_DXOFF_DUMPS=self.dumps,
|
||||
FELHOM_DXOFF_SECRETS=self.secrets)
|
||||
e.update({k: str(v) for k, v in kw.items()})
|
||||
@@ -314,6 +335,21 @@ class Push(Base):
|
||||
self.assertIn("Vaultwarden integrity_check", r.stderr)
|
||||
self.assertEqual(self.pushed(), []); self.assertFalse(self.signal())
|
||||
|
||||
def test_signing_keys_ride_the_copy(self):
|
||||
r = self.push()
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
snap = os.path.join(self.pbs, "snaps", self.pushed()[0])
|
||||
for k in SIGNING_KEYS:
|
||||
self.assertTrue(os.path.isfile(os.path.join(snap, "signing", k)), k)
|
||||
self.assertTrue(os.path.isfile(os.path.join(snap, "signing", k + ".pub")), k)
|
||||
|
||||
def test_a_missing_signing_key_refuses(self):
|
||||
os.remove(os.path.join(self.signing, "felhom-rec-recovery"))
|
||||
r = self.push()
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
self.assertIn("signing key file felhom-rec-recovery missing", r.stderr)
|
||||
self.assertEqual(self.pushed(), []); self.assertFalse(self.signal())
|
||||
|
||||
def test_failed_push_writes_no_signal(self):
|
||||
r = self.push(FAKE_PBS_FAIL=1)
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
@@ -378,6 +414,25 @@ class RestoreTest(Base):
|
||||
self.assertIn("Vaultwarden 1 user(s) / 3 item(s)", r.stdout)
|
||||
self.assertNotIn("encrypted", r.stdout + r.stderr)
|
||||
|
||||
def test_restore_checks_every_signing_key_opens_and_matches(self):
|
||||
self.pushed_copy()
|
||||
r = self.restore()
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertIn("3 signing key(s) match", r.stdout)
|
||||
self.assertNotIn("BEGIN TEST KEY", r.stdout + r.stderr)
|
||||
|
||||
def test_restore_refuses_a_signing_key_that_does_not_match(self):
|
||||
snap = self.pushed_copy()
|
||||
p = os.path.join(snap, "signing", "felhom-op-operational.pub")
|
||||
open(p, "w").write("ssh-ed25519 AAAAOTHER someone-else\n")
|
||||
man = os.path.join(snap, "MANIFEST.sha256")
|
||||
kept = [l for l in open(man).readlines() if not l.rstrip().endswith("felhom-op-operational.pub")]
|
||||
open(man, "w").writelines(kept)
|
||||
r = self.restore()
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
self.assertIn("does not match its public half", r.stderr)
|
||||
self.assertFalse(self.signal("felhom_dooplex_offsite_restore.prom"))
|
||||
|
||||
def test_restore_without_vaultwarden_fails(self):
|
||||
snap = self.pushed_copy()
|
||||
os.remove(os.path.join(snap, "vaultwarden/db.sqlite3"))
|
||||
|
||||
Reference in New Issue
Block a user