gates / gates (push) Failing after 6m36s
Mask PASS, PreToolUse refuse PASS, PostToolUse replace PARTIAL (crash and large-output disk leaks, both with a measured fix), SOPS+age store PARTIAL (private repo could not be created), transcript sweep PASS. Fake secrets only; ~/.config/credentials not read; no CC settings changed. Files use "creds": .gitignore *secret* ignores the brief's names. STATUS D10: the store choice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
233 lines
15 KiB
Markdown
233 lines
15 KiB
Markdown
# SPIKE — a secret broker and transcript guards for CC (2026-10-17)
|
||
|
||
> **File name:** the brief named this `SPIKE-cc-secrets-2026-10-17.md`. This repo's `.gitignore` line 3 ignores
|
||
> every path matching `*secret*` (a safety net against committing secret files), so the spike files use `creds`
|
||
> instead. The net was not forced (`git add -f`) and not changed.
|
||
|
||
**What this answers:** can CC use a secret without a value reaching a session transcript, and which of
|
||
the five proposed mechanisms really work? Every test used fake secrets („canaries", `CANARY-<Qn>-` + 24 hex
|
||
chars). **`~/.config/credentials` was not read. `~/.claude/settings.json` and every project settings file
|
||
were not changed** — every hook ran in a throwaway headless child session loaded with `--settings`.
|
||
|
||
**Run on:** DooPlex, Claude Code **2.1.300**, 2026-10-17 11:30–11:50 CEST. Scratch folder
|
||
`~/spike-cc-secrets/` (outside every repo). Probes: `SPIKE-cc-creds-2026-10-17/probes/`. Evidence (canaries
|
||
replaced by `<CANARY-name>`): `SPIKE-cc-creds-2026-10-17/evidence/`.
|
||
|
||
**Architecture document for this area: none.** `architecture/12-agent-tooling.md` covers instruction files
|
||
only; `runbooks/secrets.md` covers cluster Secrets only. Recorded as a register row.
|
||
|
||
**Inputs that were missing.** The design file `DESIGN-cc-secrets-2026-10-17.md` was not on DooPlex (searched
|
||
`/` and `/mnt/5_hdd` by name), so `SPIKE-cc-creds-2026-10-17/DESIGN.md` is **not** in git; the task's summary
|
||
of the design was used. `credentials-file-key-names-2026-10-16.md` was not found either; Q6 used a name pattern
|
||
instead (`TOKEN|SECRET|PASS|PW|KEY|API|CRED|AUTH|_AK|_SK|R_DEMO|HUB_|S3_|RESEND`).
|
||
|
||
## Results
|
||
|
||
| Q | Mechanism | Verdict | Measured | Evidence |
|
||
|---|---|---|---|---|
|
||
| Q1 | Output mask (`sec run` shape) | **PASS** | 7/7 shapes seen unfiltered, 0/7 filtered, exit codes equal; 6 extra encodings 6/6 → 0/6 after one fix; exit 37 → 37, SIGTERM → 143; 100 MB: +0.7 s per secret | `evidence/q1/` |
|
||
| Q2 | PreToolUse refuse | **PASS** | 4/4 refused before running, control ran, crashing hook **blocked** with `onFailure: "block"`, ran without it | `evidence/q2/` |
|
||
| Q3 | PostToolUse replace | **PARTIAL** | Small output: answer 0, transcript 0, other files 0 (control 1 / 3 / 1). **Two disk leaks found:** large output, crashing hook. Both closed by a tested fix | `evidence/q3/` |
|
||
| Q4 | SOPS + age store | **PARTIAL** | No canary in the encrypted file or `git log -p`; decrypt median **21 ms**; put without printing works; revert restores. **The private repo could not be created** (403 on API, push-to-create off) | `evidence/q4-*` |
|
||
| Q5 | Transcript sweep | **PASS** | 1 398 files, 1 098 MB, 35.7 s (9 values) / 55.6 s (15 values); planted copy found, outside file not found, unused value 0 | `evidence/q5-*` |
|
||
| Q6 | Two facts | done | `RESEND_API` is in CC's own environment; `~/rotated-secrets-2026-10-09.txt` exists (632 B, 2026-10-09 13:08, 0600) | `evidence/q6-*` |
|
||
|
||
## Q1 — the mask
|
||
|
||
Probe `probes/mask.py`: puts each secret into the child's **environment**, filters stdout and stderr, replaces
|
||
raw, URL-encoded (`quote`, `quote_plus`), JSON-escaped and base64 / urlsafe-base64 forms at all three byte
|
||
alignments with `«NAME»`. A carry buffer holds back the tail between reads. The red-proof checker
|
||
(`probes/seen.py`) computes the forms with **coreutils `base64` and stdlib**, not with `mask.py`, so a bug in
|
||
the mask's table cannot hide a leak from it.
|
||
|
||
| shape | unfiltered | filtered | rc |
|
||
|---|---|---|---|
|
||
| 1 `printf` | 1 | 0 | 0 / 0 |
|
||
| 2 `bash -x` | 2 | 0 | 0 / 0 |
|
||
| 3 `curl -u ":$C" -w '%{redirect_url}'` → local 303 | 1 | 0 | 0 / 0 |
|
||
| 4 `curl -v -H "Authorization: Bearer $C"` | 1 | 0 | 0 / 0 |
|
||
| 5 `base64` | 1 | 0 | 0 / 0 |
|
||
| 6 one byte per `flush()` | 1 | 0 | 0 / 0 |
|
||
| 7 value fed on stdin (`bash -c 'read -r T; …'`) | 1 | 0 | 0 / 0 |
|
||
|
||
Shape 3 confirms the R-132 mechanism live: the filtered line is `http://:«C»@127.0.0.1:18765/next` — curl
|
||
rebuilt the URL with the password in it.
|
||
|
||
**Shape 7 used `bash -c`, not ssh:** `ssh -o BatchMode=yes localhost true` failed (`Host key verification
|
||
failed`), so ssh was not tested.
|
||
|
||
**Extra encodings** (`evidence/q1/q1b-results.txt`), with a canary that holds `@ / : + " & % \` so the URL and
|
||
JSON forms differ from raw: urlencode, quote_plus, JSON, base64 at offset 1, base64 at offset 2, base64 wrapped
|
||
at 76 columns — 6/6 seen unfiltered, 0/6 after the fix below.
|
||
|
||
**The red-proof found a real gap.** The first mask matched exact bytes. On `base64` output wrapped at 76
|
||
columns, **2 of 3 copies leaked** through the filter (each split by a line break). The first checker also
|
||
reported 0 for that shape — a hollow zero, because the mask's tokens broke the checker's decode; the leak was
|
||
found by a direct search of the unwrapped text. Fix: a line break may sit between any two bytes of a form; the
|
||
carry buffer is 3× the longest form. After the fix: 3 tokens, 0 forms left. A first attempt at the fix crashed
|
||
the pump threads; the run then showed **0 seen in the unfiltered capture too**, so the red-proof column caught
|
||
a broken run, not a pass.
|
||
|
||
**Residue:** the last partial base64 group is not masked (`«C»OA==`), so up to 2 bytes of value information
|
||
stay visible. A byte filter cannot catch hex, gzip, reversed or otherwise transformed values; the design must
|
||
not promise more.
|
||
|
||
**Cost** (`evidence/q1/q1-timing.txt`, 106 MB of base64 text to `/dev/null`): bare `cat` 0.03 s; mask
|
||
pass-through 0.13 s; filtered, 1 secret 0.73 s; 2 secrets 1.65 s. Linear in the number of secrets — about
|
||
14 s per 100 MB with 17 values loaded. A broker should load only the values the command names.
|
||
|
||
## Q2 — PreToolUse refuse
|
||
|
||
`probes/pre.py` on matcher `Bash`, `"onFailure": "block"`, `timeout: 10`. The child's settings also allow the
|
||
test commands, so that a refusal can only come from the hook. Each refusal is recorded twice: in the hook's own
|
||
log (`evidence/q2/pre-hook.log`, rule name only) and as the tool result in the child transcript (the
|
||
`SPIKE-GUARD[...]` reason text).
|
||
|
||
| child command | result | transcript |
|
||
|---|---|---|
|
||
| `source ~/spike-cc-secrets/canary-Q2.txt` | refused, `source-secret-file` | `…/12a00509-….jsonl` |
|
||
| `printenv` | refused, `dump-environment` | `…/ef5c605d-….jsonl` |
|
||
| `curl … -w '%{redirect_url}' http://127.0.0.1:18765/` | refused, `curl-url-format-var` | `…/0a81227b-….jsonl` |
|
||
| `kubectl get secret -n spike-none -o yaml` | refused, `kubectl-read-secret` — nothing ran | `…/8b57feee-….jsonl` |
|
||
| control `ls ~/spike-cc-secrets` | **ran** (hook: pass) | `…/d0146efa-….jsonl` |
|
||
| hook `exit 1`, `onFailure: "block"` | **blocked**: `PreToolUse:Bash hook error: [exit 1]: failed; blocking because onFailure is "block"` | `…/07e9ade2-….jsonl` |
|
||
| hook `exit 1`, no `onFailure` | ran — so `onFailure` is what blocks | `…/81e7b4d2-….jsonl` |
|
||
|
||
Transcripts are under `~/.claude/projects/-home-kisfenyo-spike-cc-secrets-work/`. The Q2 canary is in **none**
|
||
of them (Q5 sweep), so the refused `source` never ran.
|
||
|
||
**The first control run was blocked by CC itself**, not by the hook: `ls in '/home/kisfenyo/spike-cc-secrets'
|
||
was blocked … may only list files in the allowed working directories`. The controls were re-run with
|
||
`--add-dir ~/spike-cc-secrets`. Without that check the control would have read as „the hook blocks
|
||
everything".
|
||
|
||
**Safety of the test:** each child ran under `env -i` with only `HOME USER LOGNAME PATH LANG TERM SHELL`, so a
|
||
guard that failed to refuse `printenv` could not have put `RESEND_API` or a CC token into a transcript.
|
||
|
||
## Q3 — PostToolUse replace (the central question)
|
||
|
||
`probes/post.py` returns, only when it changed something:
|
||
|
||
```json
|
||
{"hookSpecificOutput": {"hookEventName": "PostToolUse",
|
||
"updatedToolOutput": {"stdout": "...«CANARY»...", "stderr": "", "interrupted": false,
|
||
"isImage": false, "noOutputExpected": false}}}
|
||
```
|
||
|
||
It copies the whole `tool_response` and rewrites only `stdout` and `stderr`. The Bash `tool_response` carried
|
||
the keys `interrupted, isImage, noOutputExpected, stderr, stdout` (the docs list four; `noOutputExpected` is
|
||
the fifth), and on large output also `persistedOutputPath, persistedOutputSize`.
|
||
|
||
Child: `cat ~/spike-cc-secrets/canary-…txt`, then „repeat exactly what the command printed". (c) searched
|
||
`~/.claude`, `~/.claude.json`, `~/.cache`, `/tmp/claude-1000`; every canary had 0 files before its run.
|
||
|
||
| run | (a) final answer | (b) transcript | (c) other files |
|
||
|---|---|---|---|
|
||
| **control**, no hook (`canary-Q3C`) | **1** | **3** | the transcript only |
|
||
| **hook** (`canary-Q3`) | **0** (answer `«CANARY»`) | **0** | none |
|
||
| edge: hook crashes, `onFailure: "block"` (`Q3X`) | 0 | **2** | the transcript |
|
||
| edge: 197 KB output, hook (`Q3L`) | 0 | 0 | **`<session>/tool-results/b8qzklpyz.txt`** |
|
||
| fix: crash wrapper `guard \|\| printf <withheld JSON>` (`Q3Y`) | 0 | 0 | none |
|
||
| fix: hook also scrubs `persistedOutputPath` (`Q3P`) | 0 | 0 | none |
|
||
|
||
**Answer to the central question: yes.** `updatedToolOutput` keeps the original output out of the transcript
|
||
`.jsonl` and out of every other file under `~/.claude` — for an ordinary-sized output, with a hook that does
|
||
not crash. The control proved the measurement can see the value (3 copies in its transcript).
|
||
|
||
**Two leaks the design must close:**
|
||
|
||
1. **A PostToolUse hook that fails lets the original output through, even with `onFailure: "block"`.** The tool
|
||
already ran; „block" does not withhold its output. The child saw the value and the transcript holds it twice.
|
||
**Fix measured:** the hook command is `guard || printf '<updatedToolOutput: "[output withheld: secret guard
|
||
failed]">'` (`probes/post-crash-fallback.sh`) — the failure path itself replaces the output. 0 / 0 / 0.
|
||
2. **Large output is saved raw to `tool-results/<id>.txt` before the hook runs.** The hook's `stdout` holds only
|
||
the preview, so the hook found nothing to replace, and the full original sat on disk. The child then read
|
||
that file with Bash `grep`, and those calls were masked — so the model was protected, the disk was not.
|
||
**Fix measured:** the hook also rewrites the file at `persistedOutputPath` in place (`probes/post-persist.py`).
|
||
0 / 0 / 0. A window of milliseconds remains in which the raw file exists.
|
||
|
||
**Not measured:** the docs warn that OpenTelemetry spans and analytics events capture the original output
|
||
before the hook; telemetry was not enabled here. Hooks on `Read`, `Grep` and MCP tools (other output shapes)
|
||
were not tested — the probes cover `Bash` only.
|
||
|
||
## Q4 — the store (option B, SOPS + age)
|
||
|
||
Binaries pinned into `~/spike-cc-secrets/bin/` (`evidence/q4-binaries.txt`): **sops 3.10.2**, sha256
|
||
`79b0f844…01739` — equals the release's own `sops-v3.10.2.checksums.txt`. **age v1.2.1**, tarball sha256
|
||
`7df45a6c…79d50`; age publishes no checksum file (a `.proof` file exists and was not verified).
|
||
|
||
| measure | result |
|
||
|---|---|
|
||
| canary in the encrypted file (`grep -c`) | 0 and 0 |
|
||
| canary in `git log -p --all` (3 values, 4 commits) | 0 / 0 / 0 |
|
||
| decrypt one key, 20 runs | min 18, **median 21**, max 25 ms |
|
||
| put a key from a file, value never printed | works: `sops decrypt \| put.py NAME FILE \| sops encrypt --filename-override … /dev/stdin` |
|
||
| delete a key, commit, `git revert` | key back; decrypts byte-equal to the original (`cmp`) |
|
||
|
||
Two facts for the build: **`sops set` in 3.10.2 takes the value only on the command line** (visible in
|
||
`/proc/*/cmdline`), so `sec put` must use the pipe, never `sops set`. **Key names stay in clear text** in the
|
||
encrypted file — names are not secret by the design, but the operator should know.
|
||
|
||
**The private repo `admin/cc-secrets-spike` was not created.** Tried, in order: (1) `POST /api/v1/user/repos`
|
||
with the `kisfenyo` credential → 403; (2) the same with the `admin` entry of `~/.git-credentials` → 403, both
|
||
`token does not have at least one of required scope(s), required=[write:user]` (scopes:
|
||
`write:misc,write:notification,write:package,write:issue,write:repository`); (3) push-to-create →
|
||
`Push to create is not enabled for users`. I did not go further (no Gitea admin CLI, no kubectl into the
|
||
Gitea pod — DooPlex infrastructure). **Nothing to delete.**
|
||
|
||
**Substitute privacy check** (`evidence/q4-anon-checks.txt`), on an existing private repo,
|
||
`admin/homelab-manifests`, from `env -i` with no netrc and no credential helper: page **404**, raw
|
||
`.claude/settings.json` **404** (the same URL with credentials: **200** — the control from the other channel),
|
||
API **404**, `info/refs` **401**; a public `admin/felhom.eu` file anonymously **200**. So Gitea does hide a
|
||
private repo's raw files from strangers. The spike repo itself still needs a token with `write:user`, or the
|
||
operator makes the empty repo.
|
||
|
||
## Q5 — the transcript sweep
|
||
|
||
`probes/sweep.py` searches `~/.claude/projects/**/*.jsonl` and `**/tool-results/*` for each value and its forms
|
||
(mask forms plus the JSON-in-JSON escape), and prints names and file counts only.
|
||
|
||
- **Positive control:** a copy of a child transcript with `canary-Q5` added inside a JSON line → found, 1 file.
|
||
- **Negative control:** the same value in `~/spike-cc-secrets/work/outside-q5.jsonl` → not found.
|
||
- Never-used value → 0. The real Q3 leaks were found too: `Q3C` (control), `Q3X` (crash), `Q3L` (tool-results).
|
||
- **This session's own project folder:** 321 files, 775 MB → **0 for all 15 canaries.**
|
||
- Size and time: 1 398 files, **1 098 MB**; 35.7 s for 9 values, 55.6 s for 15. Fine for a nightly job.
|
||
|
||
Not caught by the sweep: a value split by an escaped `\n` inside JSON (the mask's line-break rule matches a raw
|
||
line break only). No real secret was fed to the sweep.
|
||
|
||
## Q6 — names only
|
||
|
||
- **`RESEND_API`** is in the claude process's own environment (pid of the parent `claude`) and in every Bash
|
||
call's environment. It comes from `~/.bashrc` line 88 (`export RESEND_API=…`). Every Bash call CC makes can
|
||
print it.
|
||
- `CLAUDE_CODE_MESSAGING_TOKEN` is in the Bash tool's environment (set by CC for its children, not present in the
|
||
parent's own environment).
|
||
- **`~/rotated-secrets-2026-10-09.txt` still exists:** 632 bytes, mtime 2026-10-09 13:08:36 +0200, mode 600.
|
||
Not opened.
|
||
|
||
## What the design must change
|
||
|
||
**Q3 PARTIAL — PostToolUse replace is a second layer, not a guarantee.**
|
||
- The replace hook's command must be wrapped so its own failure **withholds** the output (`guard || printf
|
||
withheld-json`). `onFailure: "block"` does not do this for PostToolUse.
|
||
- The replace hook must also scrub the file at `tool_response.persistedOutputPath`.
|
||
- The hook must cover the tools whose output can hold a value (`Read`, `Grep`, MCP), each with its own output
|
||
shape — not measured yet.
|
||
- The primary defence stays the broker's own mask (Q1) at the source, plus PreToolUse refusal (Q2). The
|
||
replace hook catches what slips past; the nightly sweep (Q5) alarms on what slips past both.
|
||
|
||
**Q4 PARTIAL — the store works; the repo needs one operator act.** A Gitea token with `write:user` (or the
|
||
operator creating the empty private repo) is the only missing step. `sec put` must use the decrypt-merge-encrypt
|
||
pipe, never `sops set`.
|
||
|
||
**Q1 limits to write into the design:** the mask matches encodings, not transformations; base64 leaves ≤ 2 bytes
|
||
in the last group; the cost is linear in loaded values.
|
||
|
||
## Teardown
|
||
|
||
- **Machine:** DooPlex only. `~/spike-cc-secrets/` kept (scratch, 0700, canaries only) for the build phase;
|
||
the Gitea netrc copy in it deleted at the end. The child transcripts with fake canaries stay under
|
||
`~/.claude/projects/-home-kisfenyo-spike-cc-secrets-work/` (the sweep's positive controls).
|
||
- **Host:** no other host touched. **Hub:** not touched. **Gitea:** nothing created (the repo was refused).
|