91cabdde1b
gates / gates (push) Successful in 7s
R-198 — host_escrow_superseded shipped with `blob` (the K-escrow / PBS datastore key) and
identity_blob was added to host_escrow LATER, never here. The offsite restic REPOSITORY
password lives in identity_blob. So demoteCurrentEscrowTx -- whose own comment calls it "THE
ONE escrow row-copy routine" -- retained the whole-guest key and silently dropped the off-site
data key, which is the secret the retention was built to preserve. And because the copy happens
as the new blob overwrites the old, the destroying act was the ESCROW CEREMONY: the exact thing
a rebuilt box tells its customer to run, on a card promising in Hungarian that the old backups
stay recoverable. Both demo boxes crossed that line on 2026-08-04.
- identity_blob added to the table (CREATE + additive ALTER) and carried in the shared copy
routine, so BOTH callers are fixed at once: re-escrow and host-delete demotion.
- ListSupersededEscrow reads it back; store.HostEscrow gains IdentityBlob.
- CountCurrentEscrowWithIdentity is the census of who the fix protects.
- Nothing is backfillable: pre-v0.93.0 retained rows have no blob and their sources are gone.
- Tests assert the CONSEQUENCE (a retained row can still yield a repo password), which is why
the pre-existing retention test stayed green for two months asserting the mechanism.
R-197 — SaveHostEscrow returns the hash it replaced; the escrow PUT raises
offsite_repo_key_changed (warning, operator-only, edge-triggered) when both hashes are known and
differ. No hash value travels. Severity chosen for the world v0.93.0 creates: with the identity
blob retained, a changed key is "this history now depends on an older recovery code", not a loss.
R-192 (half) — the stuck alert now reports the two shapes it actually covers, burned and
regressed, each stating its own measurement; the regressed text withdraws the Re-issue
recommendation. Every self-heal refusal leaves a notification_log row with its reason. The
guard's logic is unchanged; its 500-oldest-reports scoping stays OPEN and the window is named in
the alert text so the limitation travels with the number. offsite_delivery_stuck and
offsite_credential_restaged are added to operatorOnlyEvents -- neither was registered and neither
has a customerMessages entry, which is not a block.
R-196 — five comments (not the three the spec expected) claimed ReissueCredentials rotates the
restic repo password. It resets the PROVIDER password and cannot touch the repo password, which
is generated on the box. All five corrected; the staleness mark documented as precautionary. The
BEHAVIOUR stays open.
Not in this release: R-199, R-200, R-201 remain open -- the chain that hands the key back is
still unassembled. Part 5 hit its gate; the orphan card is untouched (R-202).
169 lines
7.1 KiB
Go
169 lines
7.1 KiB
Go
package store
|
|
|
|
import (
|
|
"bytes"
|
|
"testing"
|
|
)
|
|
|
|
// R-198 (v0.93.0) — the retention must keep the OFF-SITE data key, not only the PBS key.
|
|
//
|
|
// WHAT WAS BROKEN, and why these tests are the ones that would have caught it: host_escrow_superseded
|
|
// shipped with `blob` (the K-escrow / PBS datastore key) and `identity_blob` was added to host_escrow
|
|
// by a later ALTER and never to the retained table. The restic REPOSITORY password lives inside
|
|
// identity_blob. So every supersession retained the whole-guest key and destroyed the off-site data
|
|
// key — the exact secret the retention exists to preserve — and the destroying act is the escrow
|
|
// ceremony a rebuilt box asks its customer to run.
|
|
//
|
|
// The pre-existing TestSaveHostEscrow_RetainsSuperseded was GREEN throughout, because it asserts the
|
|
// MECHANISM (a retained row exists, with the old K-blob) and not the CONSEQUENCE (the retained row can
|
|
// still yield a repository password). These assert the consequence.
|
|
|
|
// Scenario A — a re-escrow retains BOTH sealed keys.
|
|
// RED-PROOF: drop `identity_blob` from demoteCurrentEscrowTx's INSERT/SELECT (production behaviour up
|
|
// to v0.92.0) → the retained row's identity blob is nil → this FAILS.
|
|
func TestSaveHostEscrow_RetainsIdentityBlob(t *testing.T) {
|
|
st := newTestStore(t)
|
|
const h = "hid-1"
|
|
oldIdentity := []byte("age-blob-sealing-REPO-PASSWORD-OLD")
|
|
newIdentity := []byte("age-blob-sealing-REPO-PASSWORD-NEW")
|
|
|
|
// Generation 1: the K-escrow, then the identity blob — the real order the escrow PUT uses
|
|
// (SaveHostEscrow, then SaveHostDRBundle).
|
|
if _, _, err := st.SaveHostEscrow(h, []byte("k-blob-old"), "fp-old", "zk", "2026-07-09T00:00:00Z", "SHA_OLD"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err := st.SaveHostDRBundle(h, oldIdentity, `{"gen":1}`); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
// Generation 2 with a DIFFERENT sealed repo password → supersede.
|
|
sup, prev, err := st.SaveHostEscrow(h, []byte("k-blob-new"), "fp-new", "zk", "2026-07-16T00:00:00Z", "SHA_NEW")
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if !sup {
|
|
t.Fatal("a different-passphrase upload must supersede")
|
|
}
|
|
if prev != "SHA_OLD" {
|
|
t.Fatalf("prevResticPwSHA256 = %q, want SHA_OLD (R-197 needs the replaced hash)", prev)
|
|
}
|
|
if err := st.SaveHostDRBundle(h, newIdentity, `{"gen":2}`); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
retained, err := st.ListSupersededEscrow(h)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if len(retained) != 1 {
|
|
t.Fatalf("retained rows = %d, want 1", len(retained))
|
|
}
|
|
// THE ASSERTION THIS WHOLE ITEM IS ABOUT: the off-site data key survived the supersession.
|
|
if retained[0].IdentityBlob == nil {
|
|
t.Fatal("R-198: the retained row carries NO identity blob — the off-site repository password " +
|
|
"was destroyed by the ceremony that was supposed to preserve it")
|
|
}
|
|
// And it is the PREVIOUS generation's blob, not the one that replaced it. This pins the ordering
|
|
// dependency named on demoteCurrentEscrowTx: the identity blob is written AFTER SaveHostEscrow, so
|
|
// the demote sees the old one. If that order ever inverts, the retained bytes would silently be
|
|
// the new blob filed under the old hash — recoverable-looking and wrong.
|
|
if !bytes.Equal(retained[0].IdentityBlob, oldIdentity) {
|
|
t.Fatalf("retained identity blob is not the PREVIOUS generation (got %q) — the demote ran after the overwrite",
|
|
retained[0].IdentityBlob)
|
|
}
|
|
if retained[0].ResticPwSHA256 != "SHA_OLD" || string(retained[0].Blob) != "k-blob-old" {
|
|
t.Fatalf("retained row is not the old generation: %+v", retained[0])
|
|
}
|
|
// Current row unchanged in behaviour: the NEW generation, both blobs.
|
|
if bundle, berr := st.GetHostDRBundle(h); berr != nil || bundle == nil || !bytes.Equal(bundle.IdentityBlob, newIdentity) {
|
|
t.Fatalf("current identity blob is not the new one: %+v (%v)", bundle, berr)
|
|
}
|
|
}
|
|
|
|
// Scenario B — deleting a host demotes BOTH blobs too. demoteCurrentEscrowTx is shared by the
|
|
// re-escrow path and the host-delete path; a shared routine proven through one caller is how a fix
|
|
// gets believed on a path nobody exercised.
|
|
// RED-PROOF: fix only the re-escrow caller (e.g. carry the column in SaveHostEscrow's own SQL instead
|
|
// of in the shared routine) → this FAILS while Scenario A passes.
|
|
func TestDeleteHost_DemotesIdentityBlob(t *testing.T) {
|
|
s := newTestStore(t)
|
|
const hostID, cust = "hid-del", "cust-del"
|
|
identity := []byte("age-blob-sealing-REPO-PASSWORD")
|
|
if err := s.UpsertHost(&Host{HostID: hostID, CustomerID: cust, APIKey: "k"}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if _, _, err := s.SaveHostEscrow(hostID, []byte("k-blob"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA_A"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err := s.SaveHostDRBundle(hostID, identity, `{}`); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
if err := s.DeleteHost(hostID, true); err != nil {
|
|
t.Fatalf("DeleteHost: %v", err)
|
|
}
|
|
retained, err := s.ListSupersededEscrow(hostID)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if len(retained) != 1 {
|
|
t.Fatalf("demoted rows = %d, want 1", len(retained))
|
|
}
|
|
if !bytes.Equal(retained[0].IdentityBlob, identity) {
|
|
t.Fatalf("R-198: host delete demoted custody WITHOUT the identity blob (got %q) — the off-site "+
|
|
"repository password was destroyed by a host delete", retained[0].IdentityBlob)
|
|
}
|
|
}
|
|
|
|
// Scenario C — a legacy host whose current escrow has NO identity blob supersedes cleanly. The column
|
|
// is nullable on purpose: a NOT NULL constraint here would make the fix block a ceremony, which is a
|
|
// worse failure than the one it repairs.
|
|
func TestSaveHostEscrow_SupersedesWithoutIdentityBlob(t *testing.T) {
|
|
st := newTestStore(t)
|
|
const h = "hid-legacy"
|
|
if _, _, err := st.SaveHostEscrow(h, []byte("k-old"), "fp", "zk", "2026-07-09T00:00:00Z", "SHA_OLD"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
// No SaveHostDRBundle — a slice-7-era upload.
|
|
sup, prev, err := st.SaveHostEscrow(h, []byte("k-new"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA_NEW")
|
|
if err != nil {
|
|
t.Fatalf("a supersession of an identity-less escrow must not fail: %v", err)
|
|
}
|
|
if !sup || prev != "SHA_OLD" {
|
|
t.Fatalf("superseded=%v prev=%q, want true/SHA_OLD", sup, prev)
|
|
}
|
|
retained, err := st.ListSupersededEscrow(h)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if len(retained) != 1 {
|
|
t.Fatalf("retained rows = %d, want 1", len(retained))
|
|
}
|
|
if retained[0].IdentityBlob != nil {
|
|
t.Fatalf("a legacy row must retain a NULL identity blob, got %q", retained[0].IdentityBlob)
|
|
}
|
|
}
|
|
|
|
// CountCurrentEscrowWithIdentity is the census §8.1 asks for: which hosts hold an identity blob today
|
|
// and are therefore protected from the next ceremony by this fix. Asserted rather than eyeballed,
|
|
// because the report quotes its numbers.
|
|
func TestCountCurrentEscrowWithIdentity(t *testing.T) {
|
|
st := newTestStore(t)
|
|
if _, _, err := st.SaveHostEscrow("with-id", []byte("k"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA1"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err := st.SaveHostDRBundle("with-id", []byte("age-blob"), `{}`); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if _, _, err := st.SaveHostEscrow("without-id", []byte("k"), "fp", "zk", "2026-07-16T00:00:00Z", "SHA2"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
total, withIdentity, err := st.CountCurrentEscrowWithIdentity()
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if total != 2 || withIdentity != 1 {
|
|
t.Fatalf("census = %d/%d, want 2 total / 1 with identity", withIdentity, total)
|
|
}
|
|
}
|