Files
felhom.eu/documentation/audits/SPIKE-cc-creds-2026-10-17.md
T
adminandClaude Opus 5.5 b9c4f73bfa
gates / gates (push) Failing after 6m36s
SPIKE cc-creds: secret broker + CC transcript guards measured with canaries (R-1118..R-1122, 168 -> 173)
Mask PASS, PreToolUse refuse PASS, PostToolUse replace PARTIAL (crash and
large-output disk leaks, both with a measured fix), SOPS+age store PARTIAL
(private repo could not be created), transcript sweep PASS. Fake secrets
only; ~/.config/credentials not read; no CC settings changed. Files use
"creds": .gitignore *secret* ignores the brief's names. STATUS D10: the
store choice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-17 11:57:24 +02:00

233 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SPIKE — a secret broker and transcript guards for CC (2026-10-17)
> **File name:** the brief named this `SPIKE-cc-secrets-2026-10-17.md`. This repo's `.gitignore` line 3 ignores
> every path matching `*secret*` (a safety net against committing secret files), so the spike files use `creds`
> instead. The net was not forced (`git add -f`) and not changed.
**What this answers:** can CC use a secret without a value reaching a session transcript, and which of
the five proposed mechanisms really work? Every test used fake secrets („canaries", `CANARY-<Qn>-` + 24 hex
chars). **`~/.config/credentials` was not read. `~/.claude/settings.json` and every project settings file
were not changed** — every hook ran in a throwaway headless child session loaded with `--settings`.
**Run on:** DooPlex, Claude Code **2.1.300**, 2026-10-17 11:30–11:50 CEST. Scratch folder
`~/spike-cc-secrets/` (outside every repo). Probes: `SPIKE-cc-creds-2026-10-17/probes/`. Evidence (canaries
replaced by `<CANARY-name>`): `SPIKE-cc-creds-2026-10-17/evidence/`.
**Architecture document for this area: none.** `architecture/12-agent-tooling.md` covers instruction files
only; `runbooks/secrets.md` covers cluster Secrets only. Recorded as a register row.
**Inputs that were missing.** The design file `DESIGN-cc-secrets-2026-10-17.md` was not on DooPlex (searched
`/` and `/mnt/5_hdd` by name), so `SPIKE-cc-creds-2026-10-17/DESIGN.md` is **not** in git; the task's summary
of the design was used. `credentials-file-key-names-2026-10-16.md` was not found either; Q6 used a name pattern
instead (`TOKEN|SECRET|PASS|PW|KEY|API|CRED|AUTH|_AK|_SK|R_DEMO|HUB_|S3_|RESEND`).
## Results
| Q | Mechanism | Verdict | Measured | Evidence |
|---|---|---|---|---|
| Q1 | Output mask (`sec run` shape) | **PASS** | 7/7 shapes seen unfiltered, 0/7 filtered, exit codes equal; 6 extra encodings 6/6 → 0/6 after one fix; exit 37 → 37, SIGTERM → 143; 100 MB: +0.7 s per secret | `evidence/q1/` |
| Q2 | PreToolUse refuse | **PASS** | 4/4 refused before running, control ran, crashing hook **blocked** with `onFailure: "block"`, ran without it | `evidence/q2/` |
| Q3 | PostToolUse replace | **PARTIAL** | Small output: answer 0, transcript 0, other files 0 (control 1 / 3 / 1). **Two disk leaks found:** large output, crashing hook. Both closed by a tested fix | `evidence/q3/` |
| Q4 | SOPS + age store | **PARTIAL** | No canary in the encrypted file or `git log -p`; decrypt median **21 ms**; put without printing works; revert restores. **The private repo could not be created** (403 on API, push-to-create off) | `evidence/q4-*` |
| Q5 | Transcript sweep | **PASS** | 1 398 files, 1 098 MB, 35.7 s (9 values) / 55.6 s (15 values); planted copy found, outside file not found, unused value 0 | `evidence/q5-*` |
| Q6 | Two facts | done | `RESEND_API` is in CC's own environment; `~/rotated-secrets-2026-10-09.txt` exists (632 B, 2026-10-09 13:08, 0600) | `evidence/q6-*` |
## Q1 — the mask
Probe `probes/mask.py`: puts each secret into the child's **environment**, filters stdout and stderr, replaces
raw, URL-encoded (`quote`, `quote_plus`), JSON-escaped and base64 / urlsafe-base64 forms at all three byte
alignments with `«NAME»`. A carry buffer holds back the tail between reads. The red-proof checker
(`probes/seen.py`) computes the forms with **coreutils `base64` and stdlib**, not with `mask.py`, so a bug in
the mask's table cannot hide a leak from it.
| shape | unfiltered | filtered | rc |
|---|---|---|---|
| 1 `printf` | 1 | 0 | 0 / 0 |
| 2 `bash -x` | 2 | 0 | 0 / 0 |
| 3 `curl -u ":$C" -w '%{redirect_url}'` → local 303 | 1 | 0 | 0 / 0 |
| 4 `curl -v -H "Authorization: Bearer $C"` | 1 | 0 | 0 / 0 |
| 5 `base64` | 1 | 0 | 0 / 0 |
| 6 one byte per `flush()` | 1 | 0 | 0 / 0 |
| 7 value fed on stdin (`bash -c 'read -r T; …'`) | 1 | 0 | 0 / 0 |
Shape 3 confirms the R-132 mechanism live: the filtered line is `http://:«C»@127.0.0.1:18765/next` — curl
rebuilt the URL with the password in it.
**Shape 7 used `bash -c`, not ssh:** `ssh -o BatchMode=yes localhost true` failed (`Host key verification
failed`), so ssh was not tested.
**Extra encodings** (`evidence/q1/q1b-results.txt`), with a canary that holds `@ / : + " & % \` so the URL and
JSON forms differ from raw: urlencode, quote_plus, JSON, base64 at offset 1, base64 at offset 2, base64 wrapped
at 76 columns — 6/6 seen unfiltered, 0/6 after the fix below.
**The red-proof found a real gap.** The first mask matched exact bytes. On `base64` output wrapped at 76
columns, **2 of 3 copies leaked** through the filter (each split by a line break). The first checker also
reported 0 for that shape — a hollow zero, because the mask's tokens broke the checker's decode; the leak was
found by a direct search of the unwrapped text. Fix: a line break may sit between any two bytes of a form; the
carry buffer is 3× the longest form. After the fix: 3 tokens, 0 forms left. A first attempt at the fix crashed
the pump threads; the run then showed **0 seen in the unfiltered capture too**, so the red-proof column caught
a broken run, not a pass.
**Residue:** the last partial base64 group is not masked (`«C»OA==`), so up to 2 bytes of value information
stay visible. A byte filter cannot catch hex, gzip, reversed or otherwise transformed values; the design must
not promise more.
**Cost** (`evidence/q1/q1-timing.txt`, 106 MB of base64 text to `/dev/null`): bare `cat` 0.03 s; mask
pass-through 0.13 s; filtered, 1 secret 0.73 s; 2 secrets 1.65 s. Linear in the number of secrets — about
14 s per 100 MB with 17 values loaded. A broker should load only the values the command names.
## Q2 — PreToolUse refuse
`probes/pre.py` on matcher `Bash`, `"onFailure": "block"`, `timeout: 10`. The child's settings also allow the
test commands, so that a refusal can only come from the hook. Each refusal is recorded twice: in the hook's own
log (`evidence/q2/pre-hook.log`, rule name only) and as the tool result in the child transcript (the
`SPIKE-GUARD[...]` reason text).
| child command | result | transcript |
|---|---|---|
| `source ~/spike-cc-secrets/canary-Q2.txt` | refused, `source-secret-file` | `…/12a00509-….jsonl` |
| `printenv` | refused, `dump-environment` | `…/ef5c605d-….jsonl` |
| `curl … -w '%{redirect_url}' http://127.0.0.1:18765/` | refused, `curl-url-format-var` | `…/0a81227b-….jsonl` |
| `kubectl get secret -n spike-none -o yaml` | refused, `kubectl-read-secret` — nothing ran | `…/8b57feee-….jsonl` |
| control `ls ~/spike-cc-secrets` | **ran** (hook: pass) | `…/d0146efa-….jsonl` |
| hook `exit 1`, `onFailure: "block"` | **blocked**: `PreToolUse:Bash hook error: [exit 1]: failed; blocking because onFailure is "block"` | `…/07e9ade2-….jsonl` |
| hook `exit 1`, no `onFailure` | ran — so `onFailure` is what blocks | `…/81e7b4d2-….jsonl` |
Transcripts are under `~/.claude/projects/-home-kisfenyo-spike-cc-secrets-work/`. The Q2 canary is in **none**
of them (Q5 sweep), so the refused `source` never ran.
**The first control run was blocked by CC itself**, not by the hook: `ls in '/home/kisfenyo/spike-cc-secrets'
was blocked … may only list files in the allowed working directories`. The controls were re-run with
`--add-dir ~/spike-cc-secrets`. Without that check the control would have read as „the hook blocks
everything".
**Safety of the test:** each child ran under `env -i` with only `HOME USER LOGNAME PATH LANG TERM SHELL`, so a
guard that failed to refuse `printenv` could not have put `RESEND_API` or a CC token into a transcript.
## Q3 — PostToolUse replace (the central question)
`probes/post.py` returns, only when it changed something:
```json
{"hookSpecificOutput": {"hookEventName": "PostToolUse",
"updatedToolOutput": {"stdout": "...«CANARY»...", "stderr": "", "interrupted": false,
"isImage": false, "noOutputExpected": false}}}
```
It copies the whole `tool_response` and rewrites only `stdout` and `stderr`. The Bash `tool_response` carried
the keys `interrupted, isImage, noOutputExpected, stderr, stdout` (the docs list four; `noOutputExpected` is
the fifth), and on large output also `persistedOutputPath, persistedOutputSize`.
Child: `cat ~/spike-cc-secrets/canary-…txt`, then „repeat exactly what the command printed". (c) searched
`~/.claude`, `~/.claude.json`, `~/.cache`, `/tmp/claude-1000`; every canary had 0 files before its run.
| run | (a) final answer | (b) transcript | (c) other files |
|---|---|---|---|
| **control**, no hook (`canary-Q3C`) | **1** | **3** | the transcript only |
| **hook** (`canary-Q3`) | **0** (answer `«CANARY»`) | **0** | none |
| edge: hook crashes, `onFailure: "block"` (`Q3X`) | 0 | **2** | the transcript |
| edge: 197 KB output, hook (`Q3L`) | 0 | 0 | **`<session>/tool-results/b8qzklpyz.txt`** |
| fix: crash wrapper `guard \|\| printf <withheld JSON>` (`Q3Y`) | 0 | 0 | none |
| fix: hook also scrubs `persistedOutputPath` (`Q3P`) | 0 | 0 | none |
**Answer to the central question: yes.** `updatedToolOutput` keeps the original output out of the transcript
`.jsonl` and out of every other file under `~/.claude` — for an ordinary-sized output, with a hook that does
not crash. The control proved the measurement can see the value (3 copies in its transcript).
**Two leaks the design must close:**
1. **A PostToolUse hook that fails lets the original output through, even with `onFailure: "block"`.** The tool
already ran; „block" does not withhold its output. The child saw the value and the transcript holds it twice.
**Fix measured:** the hook command is `guard || printf '<updatedToolOutput: "[output withheld: secret guard
failed]">'` (`probes/post-crash-fallback.sh`) — the failure path itself replaces the output. 0 / 0 / 0.
2. **Large output is saved raw to `tool-results/<id>.txt` before the hook runs.** The hook's `stdout` holds only
the preview, so the hook found nothing to replace, and the full original sat on disk. The child then read
that file with Bash `grep`, and those calls were masked — so the model was protected, the disk was not.
**Fix measured:** the hook also rewrites the file at `persistedOutputPath` in place (`probes/post-persist.py`).
0 / 0 / 0. A window of milliseconds remains in which the raw file exists.
**Not measured:** the docs warn that OpenTelemetry spans and analytics events capture the original output
before the hook; telemetry was not enabled here. Hooks on `Read`, `Grep` and MCP tools (other output shapes)
were not tested — the probes cover `Bash` only.
## Q4 — the store (option B, SOPS + age)
Binaries pinned into `~/spike-cc-secrets/bin/` (`evidence/q4-binaries.txt`): **sops 3.10.2**, sha256
`79b0f844…01739` — equals the release's own `sops-v3.10.2.checksums.txt`. **age v1.2.1**, tarball sha256
`7df45a6c…79d50`; age publishes no checksum file (a `.proof` file exists and was not verified).
| measure | result |
|---|---|
| canary in the encrypted file (`grep -c`) | 0 and 0 |
| canary in `git log -p --all` (3 values, 4 commits) | 0 / 0 / 0 |
| decrypt one key, 20 runs | min 18, **median 21**, max 25 ms |
| put a key from a file, value never printed | works: `sops decrypt \| put.py NAME FILE \| sops encrypt --filename-override … /dev/stdin` |
| delete a key, commit, `git revert` | key back; decrypts byte-equal to the original (`cmp`) |
Two facts for the build: **`sops set` in 3.10.2 takes the value only on the command line** (visible in
`/proc/*/cmdline`), so `sec put` must use the pipe, never `sops set`. **Key names stay in clear text** in the
encrypted file — names are not secret by the design, but the operator should know.
**The private repo `admin/cc-secrets-spike` was not created.** Tried, in order: (1) `POST /api/v1/user/repos`
with the `kisfenyo` credential → 403; (2) the same with the `admin` entry of `~/.git-credentials` → 403, both
`token does not have at least one of required scope(s), required=[write:user]` (scopes:
`write:misc,write:notification,write:package,write:issue,write:repository`); (3) push-to-create →
`Push to create is not enabled for users`. I did not go further (no Gitea admin CLI, no kubectl into the
Gitea pod — DooPlex infrastructure). **Nothing to delete.**
**Substitute privacy check** (`evidence/q4-anon-checks.txt`), on an existing private repo,
`admin/homelab-manifests`, from `env -i` with no netrc and no credential helper: page **404**, raw
`.claude/settings.json` **404** (the same URL with credentials: **200** — the control from the other channel),
API **404**, `info/refs` **401**; a public `admin/felhom.eu` file anonymously **200**. So Gitea does hide a
private repo's raw files from strangers. The spike repo itself still needs a token with `write:user`, or the
operator makes the empty repo.
## Q5 — the transcript sweep
`probes/sweep.py` searches `~/.claude/projects/**/*.jsonl` and `**/tool-results/*` for each value and its forms
(mask forms plus the JSON-in-JSON escape), and prints names and file counts only.
- **Positive control:** a copy of a child transcript with `canary-Q5` added inside a JSON line → found, 1 file.
- **Negative control:** the same value in `~/spike-cc-secrets/work/outside-q5.jsonl` → not found.
- Never-used value → 0. The real Q3 leaks were found too: `Q3C` (control), `Q3X` (crash), `Q3L` (tool-results).
- **This session's own project folder:** 321 files, 775 MB → **0 for all 15 canaries.**
- Size and time: 1 398 files, **1 098 MB**; 35.7 s for 9 values, 55.6 s for 15. Fine for a nightly job.
Not caught by the sweep: a value split by an escaped `\n` inside JSON (the mask's line-break rule matches a raw
line break only). No real secret was fed to the sweep.
## Q6 — names only
- **`RESEND_API`** is in the claude process's own environment (pid of the parent `claude`) and in every Bash
call's environment. It comes from `~/.bashrc` line 88 (`export RESEND_API=…`). Every Bash call CC makes can
print it.
- `CLAUDE_CODE_MESSAGING_TOKEN` is in the Bash tool's environment (set by CC for its children, not present in the
parent's own environment).
- **`~/rotated-secrets-2026-10-09.txt` still exists:** 632 bytes, mtime 2026-10-09 13:08:36 +0200, mode 600.
Not opened.
## What the design must change
**Q3 PARTIAL — PostToolUse replace is a second layer, not a guarantee.**
- The replace hook's command must be wrapped so its own failure **withholds** the output (`guard || printf
withheld-json`). `onFailure: "block"` does not do this for PostToolUse.
- The replace hook must also scrub the file at `tool_response.persistedOutputPath`.
- The hook must cover the tools whose output can hold a value (`Read`, `Grep`, MCP), each with its own output
shape — not measured yet.
- The primary defence stays the broker's own mask (Q1) at the source, plus PreToolUse refusal (Q2). The
replace hook catches what slips past; the nightly sweep (Q5) alarms on what slips past both.
**Q4 PARTIAL — the store works; the repo needs one operator act.** A Gitea token with `write:user` (or the
operator creating the empty private repo) is the only missing step. `sec put` must use the decrypt-merge-encrypt
pipe, never `sops set`.
**Q1 limits to write into the design:** the mask matches encodings, not transformations; base64 leaves ≤ 2 bytes
in the last group; the cost is linear in loaded values.
## Teardown
- **Machine:** DooPlex only. `~/spike-cc-secrets/` kept (scratch, 0700, canaries only) for the build phase;
the Gitea netrc copy in it deleted at the end. The child transcripts with fake canaries stay under
`~/.claude/projects/-home-kisfenyo-spike-cc-secrets-work/` (the sweep's positive controls).
- **Host:** no other host touched. **Hub:** not touched. **Gitea:** nothing created (the repo was refused).