v0.91.0 — the DR tier can no longer be applied and dead at the same time (R-39 + R-50b(a))

Closes the agent half of R-39's fleet fix. Requires hub >=0.68.0 for the re-arm signal;
that hub is safe for 0.90.0 agents (unknown key dropped), so it deploys first.

Three compounding defects let a box report `applied` while every PBS request 401'd:

1. The re-key was INVISIBLE. An ep0 re-issue rotates the secret of an existing token, so
   token_id/fingerprint/datastore/namespace come back byte-identical and the descriptor
   content hash never moved — the converged agent short-circuited and never consumed the
   fresh secret. WirePBSDR.SecretGeneration (field-exact with the hub) is what moves the
   hash now, because descriptorHash marshals this struct.

2. The agent could not READ its own credential. It writes /etc/pve/priv/storage/<id>.pw
   through the root wrapper, but that dir is 0700 root:www-data and the wrapper had no
   read verb — so the target resolver got "permission denied" every cycle, warned, and
   skipped. The one loop that could have caught the 401 was blind BY CONSTRUCTION. Adds a
   narrow `read` verb (+ exactly one sudoers line, + a pbsdr-read capability row): one
   secret to stdout, no network, no mutation, never in argv (sudo logs argv), traversal
   refused by the id grammar, the dir allowlist AND a resolved-path prefix assertion.

3. Nothing probed AUTHENTICATION. pbs.ProbeAuth (GET /version + an ErrUnauthorized
   sentinel) runs on the 15-minute collect path and its verdict becomes a loud
   `auth_failed` the hub escalates to a fresh mint. /version needs no datastore, namespace
   or privilege, so a 401 means the CREDENTIAL is bad; 403 is deliberately NOT treated as
   unauthorized, since re-keying a too-narrow token would mint forever without fixing
   anything. A transport error is UNKNOWN, never a rejection — otherwise every network
   blip burns a credential. Recovery self-clears.

R-50b(a): the report now carries the installed wrapper's sha256 so drift against the
vouched manifest value is answerable. Empty = unknown, never drift.

Three red-proofs, all at the assertion level. Removing SecretGeneration fails the re-arm
test with "consume calls=1, want 2". Swallowing the probe result leaves State:applied
AuthFailed:false — the July-18 shape exactly. Notably, deleting the wrapper's id charset
guard alone does NOT open a traversal hole (readlink + the prefix assertion still catch
it), so the isolating red-proof removes BOTH and shows the out-of-tree secret printed —
the layering is real, and a single-guard red-proof would have passed vacuously.
This commit is contained in:
2026-07-21 10:12:31 +02:00
parent 8c55ac7fda
commit b2ca63ee9f
14 changed files with 784 additions and 8 deletions
+58
View File
@@ -1,3 +1,61 @@
## v0.91.0 — the DR tier can no longer be `applied` and dead at the same time (R-39 fleet fix + R-50b(a)) (2026-07-21)
**Requires hub >= v0.68.0** for the re-arm signal. Hub v0.68.0 is safe for 0.90.0 agents (they drop
the unknown descriptor key), but the guarantees below need THIS agent. **MinAgent → 0.91.0 is an
operator manifest save, sequenced after the fleet has self-updated — not a code change.**
### What was broken
Three compounding defects let a box report `applied` while every PBS request 401'd:
1. **The re-key was invisible.** An ep0 re-issue rotates the SECRET of an existing token, so
`token_id`, `fingerprint`, `datastore` and `namespace` all come back byte-identical. The agent
re-applies on the descriptor's CONTENT HASH, so a converged box short-circuited and never consumed
the fresh secret. Proof from the N100: `consumed-failed.json` carried a hash byte-identical to the
`marker.json` written two minutes before the re-issue.
2. **The agent could not read its own credential.** It WRITES
`/etc/pve/priv/storage/<id>.pw` through the root wrapper, but that directory is `0700 root:www-data`
and the wrapper had no read verb — so `pbsTargetsFromPVE` got "permission denied" every cycle,
logged a Warn and skipped the datastore. The one loop that could have caught the 401 was blind **by
construction**.
3. **Nothing probed authentication.** A snapshot list that fails with 401 looked exactly like "PBS is
busy".
### The fix
- **`WirePBSDR.SecretGeneration`** — field-exact with the hub's descriptor. Because `descriptorHash`
marshals this struct, the hub's monotonic mint counter is what finally moves the hash and re-arms a
converged agent.
- **Wrapper `read` verb** (+ exactly ONE sudoers line, + a `pbsdr-read` capability row). Prints one
secret to STDOUT and nothing else: no network, no mutation, no logging of the value, and the secret
never rides argv (sudo logs argv). Traversal is refused three times over — the id grammar admits no
slash, `val_sdir` pins the directory, and the RESOLVED path is prefix-asserted.
- **`pbs.ProbeAuth`** — `GET /version`, the cheapest authenticated question, with a distinct
`ErrUnauthorized` sentinel. `/version` needs no datastore, namespace or privilege, so a 401 there
means the CREDENTIAL is bad — not that an ACL is narrow. **403 is deliberately NOT treated as
unauthorized**: re-keying a too-narrow token would mint credentials forever without fixing anything.
- **The probe runs on the 15-minute collect path** (not the 6 h verify cadence) and its verdict
becomes a LOUD `auth_failed` state the hub's pbsdrheal escalates to a fresh mint. **A transport
error is UNKNOWN, never a rejection** — otherwise every network blip would burn a credential.
Recovery is self-clearing.
- **`readPBSSecret`** now prefers a directly-readable file and falls back to the wrapper, so a box
with its own agent-owned secret dir needs no sudo at all.
- **R-50b(a):** the report carries the installed wrapper's sha256, so drift against the vouched
manifest value is finally answerable. Empty = unknown, never drift.
### Tests
Scenario A (a re-key re-arms a converged agent) with the hash mechanism asserted separately; the
unknown-field compat direction; auth_failed loud/ignored-for-other-storage/none-before-descriptor/
self-clearing; and the wrapper `read` verb executed under real bash — traversal refusals, secret to
stdout only, missing-file refusal, side-effect freedom.
**Three red-proofs run at the assertion level.** Removing `SecretGeneration` makes the re-arm test
fail with `consume calls=1, want 2`. Swallowing the probe result leaves `State:applied
AuthFailed:false` — the July-18 shape exactly. Deleting the id charset guard alone does **not** open
a traversal hole (readlink + the prefix assertion still catch it), so the isolating red-proof removes
the charset guard AND the prefix assertion and shows the out-of-tree secret printed.
## docs — the workflow moved to DooPlex-local execution (2026-07-19)
**Docs only, no version bump, no code change.** Claude Code now runs on DooPlex (192.168.0.180,