hub: R-39 core — stamp a secret GENERATION into the pbs_dr descriptor

The fleet half of R-39. An ep0 credential re-issue re-keys the SECRET of an existing
token, so token_id, fingerprint, datastore and namespace all come back byte-identical.
The agent re-applies on the descriptor's CONTENT HASH, so a re-issue was invisible to a
converged box: it short-circuited, never consumed the fresh secret, and served a revoked
credential while reporting `applied` — the N100 failure of 2026-07-18.

host_pbs_secrets gains a monotonic per-host `generation`, advanced by every fresh MINT and
by nothing else, stamped into the descriptor as `secret_generation`. That is now the only
field a re-key moves, and it is what re-arms the agent.

DEVIATION FROM SPEC, deliberate: the brief said to return "the new row's id (int64) …
no schema change". There is no row id — host_pbs_secrets is keyed by host_id and UPSERTed
last-write-wins, so a new row never exists, and created_at collides for two mints in the
same second. An additive counter column is the only monotonic source; it uses the repo's
existing idempotent ALTER-TABLE idiom.

RestageHostPBSSecret deliberately does NOT advance it: a re-stage re-arms the SAME secret,
the descriptor content genuinely has not changed, and a bump would cause a pointless agent
refetch loop (that method's own contract says so).

Also corrects a comment that asserted the re-issue refreshes the descriptor "with the NEW
token_id/fingerprint". That is false for a re-key, and believing it is why the descriptor
was never expected to be identical in the first place.

omitempty is load-bearing: a zero generation must not start emitting a new key into every
pre-existing descriptor, which would itself be a fleet-wide spurious re-apply.

Compatibility: agents below 0.91.0 drop the unknown JSON key and behave exactly as today —
inert, not breaking (Scenario C).

Tests: store-level monotonicity + per-host isolation + restage-leaves-it-alone; descriptor
byte-change, omitempty, and sibling-key round-trip; and a FLOW-level test driving
ReissuePBSDR against a fake that models a real re-key. Red-proof run at the assertion
level (not the compiler): commenting out the stamp makes the flow test fail with both
byte-identical blocks printed.
This commit is contained in:
2026-07-21 09:52:04 +02:00
parent 11ead4be0e
commit c484aa204e
7 changed files with 371 additions and 32 deletions
+42 -14
View File
@@ -58,6 +58,19 @@ type pbsDRDescriptor struct {
Namespace string `json:"namespace,omitempty"`
TokenID string `json:"token_id,omitempty"`
Fingerprint string `json:"fingerprint,omitempty"`
// SecretGeneration is the monotonic per-host counter advanced by every fresh secret MINT
// (store.SaveHostPBSSecret). It carries no secret material — only the fact that one rotated.
//
// It exists because a re-key changes NOTHING else in this struct: an ep0 re-issue re-keys the
// secret of an EXISTING token, so token_id/fingerprint/datastore/namespace stay byte-identical.
// The agent re-applies on the descriptor's content hash, so without this field a re-issue is
// invisible to a converged box — it short-circuits, never consumes the fresh secret, and serves a
// revoked credential while reporting `applied` (R-39, N100 2026-07-18).
//
// MUST stay field-exact with felhom-agent internal/hub.WirePBSDR. Agents below 0.91.0 do not
// carry the field, drop the unknown JSON key, and behave exactly as they do today — the field is
// inert to them, not breaking (Scenario C); the re-arm guarantee needs agent >= 0.91.0.
SecretGeneration int64 `json:"secret_generation,omitempty"`
}
// defaultPBSStorageID is the storage-entry id the agent bridge creates on a customer box. The DEMO
@@ -254,17 +267,19 @@ func (s *Server) pbsdrProvisionAtom(ctx context.Context, customerID string, host
}
// The atom: secret first (consume-once custody), then descriptor+bump (the agent's signal).
if err := s.store.SaveHostPBSSecret(host.HostID, res.TokenSecret); err != nil {
secretGen, err := s.store.SaveHostPBSSecret(host.HostID, res.TokenSecret)
if err != nil {
return "", fmt.Errorf("pbsdr: store one-time token secret: %w", err)
}
desc := &pbsDRDescriptor{
Enabled: true,
StorageID: storageID,
PBSTunnelIP: ep.PBSTunnelIP,
Datastore: res.Datastore,
Namespace: res.Namespace,
TokenID: res.TokenID,
Fingerprint: res.Fingerprint,
Enabled: true,
StorageID: storageID,
PBSTunnelIP: ep.PBSTunnelIP,
Datastore: res.Datastore,
Namespace: res.Namespace,
TokenID: res.TokenID,
Fingerprint: res.Fingerprint,
SecretGeneration: secretGen,
}
merged, err := mergePBSDR(host.DesiredJSON, desc)
if err != nil {
@@ -316,8 +331,14 @@ func (s *Server) PBSDRAutoProvision(ctx context.Context, customerID string) {
// ReissuePBSDR re-keys the customer's ep0 PBS token and re-arms the agent — the non-HTTP core shared
// by the operator button (handlePBSDRReissue) and the pbsdrheal self-heal reconciler (the escalation
// path when no stored secret is re-stageable, or the agent burned one and reports consumed_failed).
// tenantsync reissue → fresh consume-once secret (SaveHostPBSSecret) → descriptor refresh with the
// NEW token_id/fingerprint + generation bump (the agent's re-consume signal). This reuses the same
// tenantsync reissue → fresh consume-once secret (SaveHostPBSSecret) → descriptor refresh + host
// generation bump (the agent's re-consume signal).
//
// CORRECTION (R-39, v0.68.0): this comment used to claim the refresh carried "the NEW
// token_id/fingerprint". That is FALSE for a re-key — ep0 rotates the SECRET of an existing token,
// so token_id and fingerprint come back byte-identical and the re-assignments below are no-ops. That
// false belief is the whole reason the descriptor never moved and the agent never re-consumed
// (N100, 2026-07-18). The thing that actually changes the descriptor is SecretGeneration. This reuses the same
// reissue op the handler does — it is NOT a re-run of pbsdrProvisionAtom (which refuses ErrTokenExists
// and would not re-key). The secret value is never logged. Keep this in lockstep with the tail of
// handlePBSDRReissue.
@@ -343,13 +364,18 @@ func (s *Server) ReissuePBSDR(ctx context.Context, customerID string) error {
if err != nil {
return fmt.Errorf("pbsdr reissue for %s: %w", customerID, err)
}
if err := s.store.SaveHostPBSSecret(host.HostID, res.TokenSecret); err != nil {
secretGen, err := s.store.SaveHostPBSSecret(host.HostID, res.TokenSecret)
if err != nil {
return fmt.Errorf("pbsdr reissue for %s: store secret: %w", customerID, err)
}
// These four are re-assigned for completeness but are byte-identical on a re-key (see the
// correction above). SecretGeneration is the field that actually moves the descriptor hash and
// therefore re-arms the agent.
cur.TokenID = res.TokenID
cur.Fingerprint = res.Fingerprint
cur.Datastore = res.Datastore
cur.Namespace = res.Namespace
cur.SecretGeneration = secretGen
merged, err := mergePBSDR(host.DesiredJSON, cur)
if err != nil {
return fmt.Errorf("pbsdr reissue for %s: merge descriptor: %w", customerID, err)
@@ -395,7 +421,8 @@ func (s *Server) handlePBSDRReissue(w http.ResponseWriter, r *http.Request, cust
http.Error(w, "PBS credential re-issue failed: "+err.Error(), http.StatusBadGateway)
return
}
if err := s.store.SaveHostPBSSecret(host.HostID, res.TokenSecret); err != nil {
secretGen, err := s.store.SaveHostPBSSecret(host.HostID, res.TokenSecret)
if err != nil {
s.logger.Printf("[ERROR] pbsdr reissue for %s: secret store: %v", customerID, err)
http.Error(w, "Re-issued on the endpoint but storing the secret failed — re-issue again", http.StatusInternalServerError)
return
@@ -404,6 +431,7 @@ func (s *Server) handlePBSDRReissue(w http.ResponseWriter, r *http.Request, cust
cur.Fingerprint = res.Fingerprint
cur.Datastore = res.Datastore
cur.Namespace = res.Namespace
cur.SecretGeneration = secretGen // the ONLY field a re-key actually moves — see ReissuePBSDR
merged, err := mergePBSDR(host.DesiredJSON, cur)
if err == nil {
_, err = s.store.SetHostDesired(host.HostID, []byte(merged))
@@ -422,8 +450,8 @@ func (s *Server) handlePBSDRReissue(w http.ResponseWriter, r *http.Request, cust
// cascade stages (host → WG peer → descriptor → escrow) — one flag, ordered rollout, honest
// intermediate states (scenario D).
type pbsDRView struct {
Supported bool // tenantsync configured on this hub
NoHost bool // no enrolled host for the customer (cascade stage 1 waiting)
Supported bool // tenantsync configured on this hub
NoHost bool // no enrolled host for the customer (cascade stage 1 waiting)
HostID string
DRTier bool // the stored per-customer flag (operator INTENT)
Enabled bool // descriptor enabled (host-side reality)