fix(backup): O4 — generate a replacement for unrecoverable resettable secrets on restore

The proceed-path for a missing RESETTABLE secret redeployed the app with the
secret blank (compose "Defaulting to a blank string" → exit 1, live-hit in the
2026-07-04 drill Phase 5). Now the restore generates a fresh credential instead:

- stacks.Manager.GenerateSecretForField: replacement value from the field's
  catalog generate spec via the deploy flow's generateValue (no logic copied);
  refuses data-keys (defense-in-depth), spec-less and non-secret fields.
- backup.Manager.SetSecretGenerator seam (wired in main.go), consulted in
  RestoreFromRecoveryUnit AFTER the untouched fail-closed gate, for missing
  names NOT in DataKeyEnvVars. The generated value rides fullEnv into
  RecreateStackFromUnit → RedeployFromEnv → SaveAppConfig, so it persists
  encrypted in the guest app.yaml and round-trips on the next backup/restore
  (no second write path). reconcileRestoreSecrets stays pure and untouched.
- WARNs now discriminate: "generated replacement for X (credential was reset)"
  vs "X unrecoverable and has no generator — app may fail to start". Values are
  never logged (asserted in test).
- Residual case (documented, not pretended away): if a restored volume tar
  carries the OLD internal credential hash, the app may still fail auth until a
  manual in-DB reset — generation fully fixes only the fresh-init case.

Companion red-proof: pre-fix behaviour (generation skipped) fails
TestRestoreGeneratesMissingResettableSecret on the non-empty DB_PASSWORD
assertion (verified, reverted). Data-key gate proven unreachable by generation
in TestRestoreGenerationNeverReachesDataKeys.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6

controller/internal/backup/restore_unit.go

@@ -114,9 +114,36 @@ func (m *Manager) RestoreFromRecoveryUnit(stackName string) error {
 		m.logger.Printf("[ERROR] [backup] Restore REFUSED for %s: %v", stackName, err)
 		return err
 	}
+	// O4: a missing RESETTABLE secret used to redeploy blank (compose "Defaulting to a blank
+	// string" → exit 1). Generate a replacement via the deploy flow's generator instead —
+	// RecreateStackFromUnit persists fullEnv through SaveAppConfig, so the new value lands
+	// encrypted in the guest app.yaml and round-trips on the next backup/restore. Data-keys are
+	// never generated: the fail-closed gate above already refused if one was missing, and the
+	// generator itself refuses data-key fields (defense-in-depth). Values are never logged.
 	if len(missing) > 0 {
-		m.logger.Printf("[WARN] [backup] Restore %s: %d resettable secret(s) unrecoverable %v — proceeding (may need a credential reset; no data-key affected)",
-			stackName, len(missing), missing)
+		dataKeySet := make(map[string]bool, len(manifest.DataKeyEnvVars))
+		for _, dk := range manifest.DataKeyEnvVars {
+			dataKeySet[dk] = true
+		}
+		var generated, unresolved []string
+		for _, name := range missing {
+			if !dataKeySet[name] && m.generateSecret != nil {
+				if v, ok := m.generateSecret(stackName, name); ok && v != "" {
+					fullEnv[name] = v
+					generated = append(generated, name)
+					continue
+				}
+			}
+			unresolved = append(unresolved, name)
+		}
+		if len(generated) > 0 {
+			m.logger.Printf("[WARN] [backup] Restore %s: generated replacement for %v — the credential was reset (old value unrecoverable); stored data is unaffected (no data-key involved)",
+				stackName, generated)
+		}
+		if len(unresolved) > 0 {
+			m.logger.Printf("[WARN] [backup] Restore %s: %d resettable secret(s) unrecoverable and have no generator %v — proceeding, but the app may fail to start until the credential is set manually",
+				stackName, len(unresolved), unresolved)
+		}
 	}
 	m.logger.Printf("[INFO] [backup] Restoring %s from recovery unit: images=%d, secrets recovered=%d/%d, data_keys=%d",
 		stackName, len(manifest.ImagePins), len(manifest.SecretEnvVars)-len(missing), len(manifest.SecretEnvVars), len(manifest.DataKeyEnvVars))