docs: felhom-agent CLAUDE.md becomes a core plus path-scoped rules (R-229 leg b)
gates / gates (push) Successful in 8s
gates / gates (push) Successful in 8s
175 -> 99 effective lines. New .claude/rules/{proxmox,localapi,backup,storage}.md alongside the
existing health-checks.md. The release section points at the felhom-build-deploy skill rather than
restating a table that drifts from the script; the layout section's per-package annotations moved
into the rule file for their area instead of being deleted.
Kept in the core because it is the only part re-injected after /compact: the root-CLI fence and its
three exceptions, the destructive-op gate, prove-ownership (audit A1), the gate entry point, the F9
live-validation fence, and the checklist.
health-checks.md overlaps localapi.md and storage.md on three globs -- deliberate, both load,
stated in each file. Go build/vet/test green and unchanged.
This commit is contained in:
@@ -0,0 +1,46 @@
|
|||||||
|
---
|
||||||
|
paths: ["internal/backup/**", "internal/pbs/**", "internal/pbsdr/**", "internal/dr/**"]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Backup, PBS and DR
|
||||||
|
|
||||||
|
`internal/backup/` is the vzdump runner, restore-test scheduler and report store. `internal/pbs/` is
|
||||||
|
the fingerprint-pinned PBS-API client plus the verify maintenance loop. `internal/pbsdr/` and
|
||||||
|
`internal/dr/` carry the DR tier and recipe halves.
|
||||||
|
|
||||||
|
## The three PBS laws
|
||||||
|
|
||||||
|
1. **Set-only.** `pvesm remove` **DELETES the encryption key**. Re-apply configuration; never remove
|
||||||
|
and re-add a PBS storage to change it.
|
||||||
|
2. **Secret on stdin.** A token secret is passed on stdin, never as an argv the process table shows.
|
||||||
|
3. **Verify the pin BEFORE consuming the secret.** A fingerprint check after the secret has been sent
|
||||||
|
protects nothing.
|
||||||
|
|
||||||
|
## Verify is server-side, and its default skips the work
|
||||||
|
|
||||||
|
The agent drives verification **remotely** via the PBS API; `proxmox-backup-client` has **no** verify
|
||||||
|
subcommand. `POST .../verify` defaults to **`ignore-verified=true`, which SKIPS already-verified
|
||||||
|
snapshots** — send `ignore-verified=false` to actually re-read and detect corruption. A verify that
|
||||||
|
skipped everything reports success.
|
||||||
|
|
||||||
|
## Presence is not success
|
||||||
|
|
||||||
|
A timestamp recording an **attempt** is not evidence of a **result**. Where a status field travels
|
||||||
|
beside a timestamp, the verdict must consult **both** — or the timestamp must record only successes.
|
||||||
|
Ask of any timestamp: *what exactly must have happened for this to be set?* If the answer is "we
|
||||||
|
tried", it cannot answer "did it work".
|
||||||
|
|
||||||
|
**Corollary:** when a verdict changes which field it counts from, the alarm text changes with it.
|
||||||
|
Leaving a message reading `last run 8h ago` while alarming on a six-day-old **success** turns a true
|
||||||
|
alarm into one the operator dismisses.
|
||||||
|
|
||||||
|
## Prune is server-side now
|
||||||
|
|
||||||
|
`DatastoreBackup` carries **no** `Datastore.Prune`. Boxes set `keep_last: 0` and the off-site endpoint
|
||||||
|
runs the prune jobs. **Box tokens stay write-only — never widen that grant** (R-89).
|
||||||
|
|
||||||
|
<!--
|
||||||
|
The ignore-verified default is the sharpest instance of the "absent log line" class in this repo: a
|
||||||
|
verify that silently skipped every snapshot completes fast, exits clean, and reports the same shape
|
||||||
|
as one that read every byte.
|
||||||
|
-->
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
---
|
||||||
|
paths: ["internal/localapi/**", "internal/authz/**", "internal/guesthook/**"]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Local API, authz and guest hooks — the per-guest blast radius
|
||||||
|
|
||||||
|
`internal/localapi/` is the narrow per-guest local API: token store, disks/format, guest binds,
|
||||||
|
controller swap, stale-lock recovery, pinned self-signed leaf. `internal/authz/` is the operator
|
||||||
|
signed-op verifier (SSHSIG) plus the durable nonce store. `internal/guesthook/` installs the
|
||||||
|
pre-start self-heal hookscript.
|
||||||
|
|
||||||
|
> **Overlap note:** `health-checks.md` also matches `internal/localapi/**` and
|
||||||
|
> `internal/guesthook/**`. That is deliberate — both rules apply there and both load. Neither
|
||||||
|
> supersedes the other.
|
||||||
|
|
||||||
|
## Scoping is the whole security property
|
||||||
|
|
||||||
|
This API is reachable **from inside a customer guest**. Every route must be scoped to the guest that
|
||||||
|
called it — a route that can name another guest's id has escaped its blast radius. Fail **safe to
|
||||||
|
protected**: an unrecognised or unresolvable caller gets less access, never more.
|
||||||
|
|
||||||
|
## Replay protection must survive a restart
|
||||||
|
|
||||||
|
**`authz.MemoryNonceStore` on a real host is a defect** — replay protection dies on restart. Use
|
||||||
|
`authz.FileNonceStore`. The memory store exists for tests.
|
||||||
|
|
||||||
|
## The token is a hash on disk, plaintext only at mint
|
||||||
|
|
||||||
|
The store keeps **hashes**. The plaintext token exists in exactly one place, `bootstrap.json` on the
|
||||||
|
PVE host — so a "read the token" step means reading that file, and a lost token is re-minted, never
|
||||||
|
recovered.
|
||||||
|
|
||||||
|
## Binds can brick guest boot
|
||||||
|
|
||||||
|
| Do not | Because | Use |
|
||||||
|
|---|---|---|
|
||||||
|
| `GuestBinder.AttachBind`/`DetachBind` (per-drive `pct set -mpN`) | legacy model; a missing bind source can **brick guest boot** (C1) | `AttachDrive`/`DetachDrive` (intermediary model) |
|
||||||
|
| `isHostMountpoint` to reconcile bind state | a boolean cannot converge stacked double-binds (the `/mnt` doubling bug) | `countHostMounts` normalization inside `AttachDrive` |
|
||||||
|
|
||||||
|
<!--
|
||||||
|
Why fail-safe-to-protected rather than fail-closed: this API also carries the recovery paths. A hard
|
||||||
|
refusal on an unresolvable caller would make a half-broken guest unrecoverable through the very
|
||||||
|
interface built to recover it. Less access, never none.
|
||||||
|
-->
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
---
|
||||||
|
paths: ["internal/proxmox/**", "internal/reconcile/**", "internal/signedjobs/**"]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Proxmox — the API contract, and how destructive work is gated
|
||||||
|
|
||||||
|
`internal/proxmox/` is the API-first `Client` plus the fenced root-CLI `Privileged`.
|
||||||
|
`internal/reconcile/` is the reconcile engine, reversibility gate, op journal and crash recovery.
|
||||||
|
`internal/signedjobs/` holds the operator-signed destructive executors (wipe, decommission).
|
||||||
|
|
||||||
|
## A 200 on the POST is not success
|
||||||
|
|
||||||
|
**Every mutating op is async**: it returns a **UPID**, and `WaitTask` must assert
|
||||||
|
`exitstatus == "OK"`. Authorization can fail at *task execution* long after the HTTP call returned
|
||||||
|
200. Treating the POST's status as the result is how a failed destroy reads as a successful one.
|
||||||
|
|
||||||
|
## The privsep token gotcha
|
||||||
|
|
||||||
|
A `--privsep 1` token's rights are the **intersection** of the backing user's permissions **and** the
|
||||||
|
token's own ACLs. The role must be granted on **both** or every call 403s. The same intersection rule
|
||||||
|
bites on PBS (`token ∩ user`).
|
||||||
|
|
||||||
|
## TLS
|
||||||
|
|
||||||
|
**SHA-256 leaf-cert pinning** against the self-signed host cert. **No insecure default**, ever. The
|
||||||
|
pin is the raw leaf-DER sha — the SAN is never checked, so a cert rotation changes the pin and the
|
||||||
|
agent must be re-pinned.
|
||||||
|
|
||||||
|
## The destructive path — never the direct call
|
||||||
|
|
||||||
|
| Do not | Because | Use |
|
||||||
|
|---|---|---|
|
||||||
|
| `Client.DestroyLXC` / `Vzdump` / `SetConfig` ad-hoc | skips classification, signature, per-guest serialization, crash recovery | `reconcile.Engine` paths / `RunSignedJob`; queue via `Queue.Submit` |
|
||||||
|
| add a method to `proxmox.Privileged` | breaks the 3-exception root-CLI fence (`routing_test.go`) | `proxmox.Runner` + a new sudoers `Cmnd_Alias` + `validate.go`-style checks |
|
||||||
|
| treat `ListLXC` output as "guests we own" | audit A1 — pre-v0.62.0 the stale-lock reaper did exactly this, contained only by the pool-scoped token | intersect with `Client.Pool` membership (`staleLockController.Guests()`); **fail safe on read failure** |
|
||||||
|
|
||||||
|
Full trap table: `REUSE.md` §3. Every guest joins the `felhom` pool — `VM.Audit` comes from the
|
||||||
|
`/pool` grant, not from a per-guest ACL.
|
||||||
|
|
||||||
|
<!--
|
||||||
|
The fence is not stylistic. It is what makes this component auditable: two types, one of which can
|
||||||
|
only speak HTTP and one of which can only shell out, with a test asserting neither crosses. A single
|
||||||
|
convenience method on Privileged that also makes an HTTP call would end that property silently.
|
||||||
|
-->
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
---
|
||||||
|
paths: ["internal/storage/**", "internal/escrow/**"]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Storage and escrow — format safety and zero-knowledge recovery
|
||||||
|
|
||||||
|
`internal/storage/` is the storage observer, durable IDs, role/claim classifiers, `SudoHostOps` and
|
||||||
|
the watchdog. `internal/escrow/` is the PBS-key escrow with its zero-knowledge recovery code.
|
||||||
|
|
||||||
|
> **Overlap note:** `health-checks.md` also matches `internal/storage/**`. Deliberate — both rules
|
||||||
|
> apply there and both load.
|
||||||
|
|
||||||
|
## Never format the device you inspected
|
||||||
|
|
||||||
|
**AGENT-001 is a TOCTOU:** acting on the caller's `req.Device` (or any remembered `/dev` path) after
|
||||||
|
inspection lets `/dev` re-enumeration retarget the node to a **different physical disk**. Format the
|
||||||
|
**re-resolved** device — `Server.reresolveWipe` / `reresolveBlank`.
|
||||||
|
|
||||||
|
**Never exec raw `mkfs.*`** (including `Binaries.MkfsExt4`/`MkfsXfs`): sudoers no longer allowlists
|
||||||
|
raw mkfs, and going direct bypasses the claim filter and the wrapper's re-checks. Use
|
||||||
|
`SudoHostOps.Format`, which routes through `felhom-mkfs-guarded`.
|
||||||
|
|
||||||
|
## The two durable-ID schemes refuse each other
|
||||||
|
|
||||||
|
They are not interchangeable, and each returns a `binding_mismatch` for the other's scheme:
|
||||||
|
|
||||||
|
| Purpose | Scheme | Resolver |
|
||||||
|
|---|---|---|
|
||||||
|
| wipe confirmation | `byid:` / `byuuid:` | `ResolveDurableDevice`, `DiskInfo.WipeDurableID` |
|
||||||
|
| enrolled-storage remount | `uuid:` | `ResolveStorageDevice` |
|
||||||
|
|
||||||
|
Using `DiskInfo.DurableID` (a `uuid:`) as a wipe-confirmation id is F20-BUG2.
|
||||||
|
|
||||||
|
## Drive data is never taken by force
|
||||||
|
|
||||||
|
Plain `umount` only — **never `-l`, never `-f`**, and never any format operation under
|
||||||
|
`/mnt/felhom-drives`.
|
||||||
|
|
||||||
|
## Escrow is zero-knowledge, and a fetch failure is not a wrong code
|
||||||
|
|
||||||
|
The server holds no client key; a no-key restore fails with `missing key`. **A fetch failure must
|
||||||
|
never be reported as a wrong recovery code** — that told a customer their correct code was bad, in
|
||||||
|
hundredths of a second, when checking a code actually takes about one. Distinguish "we could not
|
||||||
|
reach the store" from "the code did not match", always.
|
||||||
|
|
||||||
|
<!--
|
||||||
|
The escrow recovery-code "flake" was a REAL defect, not a flake. "Known flake, re-run" needs evidence
|
||||||
|
before it is said out loud — that phrase cost this project a real finding once.
|
||||||
|
-->
|
||||||
@@ -1,3 +1,29 @@
|
|||||||
|
## docs — CLAUDE.md becomes a core plus path-scoped rules (2026-08-06, R-229 leg (b)) — no version bump
|
||||||
|
|
||||||
|
**Documentation only. No Go changed, nothing built, nothing deployed.** `go build`/`vet`/`test` green
|
||||||
|
and unchanged.
|
||||||
|
|
||||||
|
**175 -> 99 effective lines** (207 -> 103 raw, 14,093 -> 6,267 bytes). The release/publish-train
|
||||||
|
section was the largest block and the `felhom-build-deploy` skill already carries the procedure, so
|
||||||
|
the core points at it instead of restating a table that drifts from the script. The package layout
|
||||||
|
went the same way as the controller's: `REUSE.md` and the tree are its home, and the per-package
|
||||||
|
annotations that were doing real work moved into the rule file for the area they describe rather than
|
||||||
|
being deleted.
|
||||||
|
|
||||||
|
**New:** `.claude/rules/{proxmox,localapi,backup,storage}.md`, all `paths:`-scoped, all <=46 effective
|
||||||
|
lines, joining the existing `health-checks.md`.
|
||||||
|
|
||||||
|
**Kept in the core deliberately** — it is the only part re-injected after `/compact`: the root-CLI
|
||||||
|
fence and its three named exceptions (breaching it is how this component stops being auditable), the
|
||||||
|
destructive-op gate, the prove-ownership rule from audit A1, the gate entry point, the F9
|
||||||
|
live-validation fence, trunk-based with its revert-and-report escape hatch, and the end-of-session
|
||||||
|
checklist.
|
||||||
|
|
||||||
|
**Glob overlap, stated rather than silently resolved:** `health-checks.md` matches
|
||||||
|
`internal/{localapi,guesthook}/**` and `internal/storage/**`, which `localapi.md` and `storage.md`
|
||||||
|
also match. Both rules load in those directories and neither supersedes the other; each new file says
|
||||||
|
so in its own text so a reader who sees two rules fire is not left guessing which wins.
|
||||||
|
|
||||||
## docs — the "CI is still owed" claim was stale; corrected (2026-08-06, R-229 part 2) — no version bump
|
## docs — the "CI is still owed" claim was stale; corrected (2026-08-06, R-229 part 2) — no version bump
|
||||||
|
|
||||||
**One sentence, no code.** This file asserted that continuous integration was still owed
|
**One sentence, no code.** This file asserted that continuous integration was still owed
|
||||||
|
|||||||
@@ -1,207 +1,103 @@
|
|||||||
# CLAUDE.md — `felhom-agent`
|
# CLAUDE.md — `felhom-agent`
|
||||||
|
|
||||||
> Loads when Claude Code touches this repo. Stable orientation only — **current state lives in
|
> Stable orientation only — **current state lives in `CONTEXT.md` and the top of `CHANGELOG.md`**,
|
||||||
> `CONTEXT.md` and the top of `CHANGELOG.md`**, never here. Cross-repo orientation: workspace-root
|
> never here. Cross-repo conventions (artifact taxonomy, access, clean-tree gate, secrets,
|
||||||
> `/mnt/5_hdd/felhom.eu/git/CLAUDE.md`.
|
> CHANGELOG/REPORT): workspace-root `/mnt/5_hdd/felhom.eu/git/CLAUDE.md`. Path-scoped detail:
|
||||||
|
> `.claude/rules/`.
|
||||||
|
|
||||||
## What this repo is
|
## What this repo is
|
||||||
|
|
||||||
`felhom-agent` is the operator-tier **host agent** that runs on each Proxmox host and owns **all**
|
The operator-tier **host agent**, one per Proxmox host, owning **all** Proxmox interaction:
|
||||||
Proxmox interaction: provision/restore guests, host storage, backup/restore orchestration, the hub
|
provision/restore guests, host storage, backup/restore orchestration, the hub control loop, and a
|
||||||
control loop, and a narrow per-guest local API. It is the **most privilege-sensitive** component.
|
narrow per-guest local API. It is the **most privilege-sensitive component in the system**.
|
||||||
|
|
||||||
- Renamed former `proxmox-controller` repo.
|
- Renamed from `proxmox-controller`.
|
||||||
- **Distinct from `felhom-controller`** — that is the *in-guest* controller (Docker-only, no Proxmox
|
- **Distinct from `felhom-controller`** — that is the *in-guest* controller, Docker-only, holding no
|
||||||
creds). Do not confuse them.
|
Proxmox credentials. Do not confuse them.
|
||||||
- Control plane, not data plane: if the agent dies, apps keep serving; only management degrades.
|
- **Control plane, not data plane:** if the agent dies, apps keep serving; only management degrades.
|
||||||
|
- Pure Go stdlib + `golang.org/x/crypto`. No web frameworks.
|
||||||
|
|
||||||
## Read before writing code
|
## Doing X → read Y
|
||||||
|
|
||||||
- **`REUSE.md`** — canonical helpers, format-safety guards, traps, seams. Check it first; update it
|
| Doing | Read |
|
||||||
in the same commit that changes a shared helper or pattern.
|
|---|---|
|
||||||
- `CONTEXT.md` (current state + open threads) and the top `CHANGELOG.md` entry (authoritative history).
|
| writing any new code | `REUSE.md` — helpers, format-safety guards, traps, seams |
|
||||||
- Design doc: `felhom.eu/documentation/architecture/03-host-agent.md` (locked). Platform facts:
|
| needing current state / open threads | `CONTEXT.md` + the top `CHANGELOG.md` entry |
|
||||||
`felhom.eu/documentation/proxmox-platform.md` + `tests/phase{0,1-2,3,4}-findings.md`.
|
| Proxmox, reconcile or signed jobs | loads itself: `.claude/rules/proxmox.md` |
|
||||||
|
| local API, authz or guest hooks | loads itself: `.claude/rules/localapi.md` |
|
||||||
|
| backup, PBS or DR | loads itself: `.claude/rules/backup.md` |
|
||||||
|
| storage or escrow | loads itself: `.claude/rules/storage.md` |
|
||||||
|
| writing a health check | loads itself: `.claude/rules/health-checks.md` |
|
||||||
|
| **release, build, publish, deploy, verify a version** | the **`felhom-build-deploy`** skill — **never hand-roll it** |
|
||||||
|
| writing or reviewing a test, fixing a bug | the **`felhom-testing`** skill |
|
||||||
|
| host addresses, break-glass, node facts | `felhom.eu/documentation/operations/nodes.md` — never restate them |
|
||||||
|
| which box may I break | `felhom.eu/documentation/runbooks/target-selection.md` |
|
||||||
|
| what version is live anywhere | ask the hub (`/hosts`, `/configs`) or the box — **never a doc** |
|
||||||
|
| the authoritative design | `felhom.eu/documentation/architecture/03-host-agent.md` (locked) |
|
||||||
|
|
||||||
## Layout (verified against the tree)
|
## The root-CLI fence — API-first, exactly three exceptions
|
||||||
|
|
||||||
```
|
This is in the core because breaching it is how this component stops being auditable.
|
||||||
cmd/felhom-agent/ main + flags + --selftest modes + the daemon entry
|
|
||||||
cmd/felhom-opsign/ offline operator signing CLI (SSHSIG)
|
|
||||||
internal/authz/ operator signed-op verifier (SSHSIG) + durable FileNonceStore
|
|
||||||
internal/backup/ vzdump backup runner + restore-test scheduler + report store
|
|
||||||
internal/capability/ live sudo-policy capability probe (degradation visibility)
|
|
||||||
internal/config/ JSON config + FELHOM_AGENT_* env overlay; secrets redacted (Redacted())
|
|
||||||
internal/desired/ hub desired-state syncer (envelope observer)
|
|
||||||
internal/escrow/ PBS-key escrow (zero-knowledge recovery code)
|
|
||||||
internal/guesthook/ pre-start self-heal hookscript install
|
|
||||||
internal/hub/ daemon: HostReport collector + Bearer client + resilient Loop
|
|
||||||
internal/lanresolver/ split-horizon DNS on guest IP change (dnsmasq RESTART, not reload)
|
|
||||||
internal/localapi/ per-guest local API: token store, disks/format, guest binds, controller swap,
|
|
||||||
stale-lock recovery, pinned self-signed leaf
|
|
||||||
internal/log/ slog setup
|
|
||||||
internal/pbs/ PBS-API client (fingerprint-pinned) + verify maintenance loop
|
|
||||||
internal/provision/ guest bootstrap back-half (token mint → bootstrap.json → pct bind)
|
|
||||||
internal/proxmox/ API-first Client + fenced root-CLI Privileged + UPID WaitTask
|
|
||||||
internal/reconcile/ reconcile engine + reversibility gate + op journal + crash recovery
|
|
||||||
internal/signedjobs/ operator-signed destructive executors (wipe, decommission)
|
|
||||||
internal/storage/ storage observer + durable ids + role/claim classifiers + SudoHostOps + watchdog
|
|
||||||
```
|
|
||||||
|
|
||||||
## Build / run
|
|
||||||
|
|
||||||
- Module `gitea.dooplex.hu/admin/felhom-agent`; binary `felhom-agent` (`cmd/felhom-agent/`).
|
|
||||||
- **Pure Go stdlib + `golang.org/x/crypto` only** — no web frameworks. The Go version is `go.mod`'s
|
|
||||||
business, not this file's. DooPlex (where CC runs) has the toolchain — build and run live tests
|
|
||||||
locally.
|
|
||||||
- Version via `-ldflags "-X main.version=<v>"`; `--version` flag. Bump on meaningful changes + CHANGELOG entry.
|
|
||||||
- **Full build/deploy/publish runbook: use the `felhom-build-deploy` skill.** Summary:
|
|
||||||
|
|
||||||
> **Clean-tree gate before any build:** `git status --porcelain` must be empty and
|
|
||||||
> `git rev-parse HEAD` must equal `git rev-parse origin/main` in the repo being built. An unpushed
|
|
||||||
> change does not exist — never build a dirty or unpushed tree. The `git pull` in the build step
|
|
||||||
> stays (it is a no-op when you work in this tree, and load-bearing if anything was pushed from
|
|
||||||
> elsewhere).
|
|
||||||
|
|
||||||
> **RELEASING IS ONE COMMAND, AND IT PUBLISHES (R-115).** Never hand-roll the build. The release
|
|
||||||
> script also creates the `v<version>` git TAG that `felhom-host-install.sh` fetches that version's
|
|
||||||
> sixteen config files from (R-183), and verifies by an **independent download** rather than trusting
|
|
||||||
> the publish step's own output. The order is **build → tag LOCALLY → publish → push tag**, and each
|
|
||||||
> step protects something (R-188, R-186). A released binary is independently verifiable: the build
|
|
||||||
> uses `-trimpath -buildvcs=false`, so the same source produces the same bytes.
|
|
||||||
>
|
|
||||||
> The ordering rationale, the verification recipe and the `publish-agent.sh` flag-parity trap are in
|
|
||||||
> the **`felhom-build-deploy`** skill.
|
|
||||||
|
|
||||||
<!--
|
|
||||||
RELEASE RATIONALE — history, not directives. Full procedure lives in the felhom-build-deploy skill.
|
|
||||||
|
|
||||||
R-115: there used to be a raw `go build` line here and a *separate* "Publish" row, so publishing was
|
|
||||||
a step someone had to remember — and it was forgotten three times in five days, the last leaving an
|
|
||||||
agent version deployed on both demo hosts and undownloadable, where a documented-path reinstall
|
|
||||||
would have silently downgraded them while reporting success. scripts/publish-agent.sh still exists
|
|
||||||
and is still correct — the release script CALLS it rather than reimplementing it.
|
|
||||||
|
|
||||||
R-188/R-186 ordering: the tag is created before the publish so the build and the tag describe the
|
|
||||||
same commit; it is *pushed* after, because the push is what wakes CI (on: [push]) and a tag visible
|
|
||||||
before its package makes the published-versions gate correctly fail a correct release — it did, on
|
|
||||||
roughly every second release, and R-168 sends that failure by mail. The invariant the old order
|
|
||||||
protected is asserted directly instead: the gate now also refuses a published version with no tag.
|
|
||||||
If the push fails after a successful publish the script says so loudly and prints the one-line
|
|
||||||
recovery; if the publish fails it removes the local-only tag so a retry is clean.
|
|
||||||
|
|
||||||
R-186 reproducibility: before -trimpath -buildvcs=false, a rebuild could not reproduce the sha you
|
|
||||||
were vouching. publish-agent.sh's fallback build uses the SAME flags — it used to force
|
|
||||||
CGO_ENABLED=0 and produce a 74 KB-smaller binary for the same version; if either build line ever
|
|
||||||
changes, change both, or one version name means two binaries again.
|
|
||||||
-->
|
|
||||||
|
|
||||||
|
|
||||||
| Step | Where | One-liner |
|
|
||||||
|---|---|---|
|
|
||||||
| **Release** (build + tag + publish + verify) | DooPlex (local) | `GITEA_USER=admin GITEA_TOKEN=<tok> scripts/release-agent.sh <ver>` — refuses a dirty/unpushed tree and refuses to re-release an existing version |
|
|
||||||
| Copy | local → felhom-pve | `scp /tmp/felhom-agent-<v> felhom-pve:/tmp/` (one hop) |
|
|
||||||
| Deploy | felhom-pve | backup `.bak-<old>` → `install -m0755` → `systemctl restart felhom-agent` (non-root `felhom-agent` user, config `/etc/felhom-agent/agent.json`) |
|
|
||||||
| Ship configs | felhom-pve | sudoers (`/etc/sudoers.d/felhom-agent`) + guarded-mkfs wrapper WITH the binary when `configs/` changed |
|
|
||||||
| **Verify** (anyone, any time) | anywhere with the repo + Go | `git checkout v<ver> && go build -trimpath -buildvcs=false -ldflags "-X main.version=<ver>" -o /tmp/a ./cmd/felhom-agent && sha256sum /tmp/a` — must equal `curl -fsSL <pkg-url> \| sha256sum` |
|
|
||||||
| **Vouch** | hub operator UI | Configs → Day-0 artifacts. **Deliberately NOT automated** — vouching is what points machines at a version, and it stays your act (prove-then-vouch) |
|
|
||||||
| Verify | felhom-pve | `felhom-agent --version` + journal (clean ReassertGuestBinds, no capability degradation) |
|
|
||||||
|
|
||||||
## Proxmox model (the load-bearing rules)
|
|
||||||
|
|
||||||
- **API-first** via a scoped `FelhomAgent` token. Raw root-CLI is **fenced to exactly 3 exceptions**:
|
- **API-first** via a scoped `FelhomAgent` token. Raw root-CLI is **fenced to exactly 3 exceptions**:
|
||||||
keyctl `pct create` (golden image), USB mount/fstab, SMART/sensors. `Client` never shells out;
|
keyctl `pct create` (golden image), USB mount/fstab, SMART/sensors.
|
||||||
`Privileged` never makes HTTP calls (asserted by `routing_test.go`). Keep that fence.
|
- **`Client` never shells out; `Privileged` never makes HTTP calls** — asserted by `routing_test.go`.
|
||||||
- **Every mutating op is async** → returns a UPID → `WaitTask` asserts `exitstatus == "OK"`. A 200 on
|
Adding a method to `proxmox.Privileged` breaks the fence; use `proxmox.Runner` plus a new sudoers
|
||||||
the POST is **not** success; authorization can fail at task execution.
|
`Cmnd_Alias` and `validate.go`-style checks (`REUSE.md` §3).
|
||||||
- **TLS:** SHA-256 leaf-cert pinning (self-signed host cert). No insecure default.
|
- **Destructive ops go through the reconcile gate / signed-jobs path.** Never call
|
||||||
- **Privsep token gotcha:** a `--privsep 1` token's rights = intersection of the backing user's perms
|
`Client.DestroyLXC` / `Vzdump` / `SetConfig` ad-hoc — that skips classification, signature,
|
||||||
AND the token's ACLs — the role must be granted on **both**, or every call 403s.
|
per-guest serialization and crash recovery.
|
||||||
- Destructive ops go through the reconcile gate / signed-jobs path — never call `Client.DestroyLXC`/
|
- **Ownership must be PROVEN, never assumed.** A raw `ListLXC` list is not "guests the agent owns";
|
||||||
`Vzdump`/`SetConfig` ad-hoc (REUSE.md §3).
|
intersect with `Client.Pool` membership and fail safe on a read failure (audit A1).
|
||||||
|
|
||||||
## Demo host (for live tests)
|
## Gates — ONE entry point
|
||||||
|
|
||||||
Two demo nodes: **`demo-felhom`** (N100, `ssh felhom-pve`) and **`demo-hp`** (HP t740, `ssh demo-hp`,
|
**Run `python3 scripts/agent_gates.py` from the repo root after ANY change here.** It runs this
|
||||||
no baked key — break-glass via the hub). **Addresses, routes, node names, break-glass recipe and what
|
repo's gates — `reuse_refs_check` and `instructions_gate`, both the **shared** copies in
|
||||||
is provisioned on each: `felhom.eu/documentation/operations/nodes.md`** — the single home; do not
|
`felhom.eu/scripts/`, never copied into this repo (a copy recreates the drift they detect; an absent
|
||||||
restate them here, and re-check an address rather than trusting one written down.
|
sibling clone FAILS). `--fast` selects the gates touching no network and no container runtime; today
|
||||||
|
that is all of them. **A missing gate is a FAILURE, never a skip.**
|
||||||
|
|
||||||
**Which box is safe to break, and what may be done to each:
|
**The pre-push hook** (`.githooks/pre-push`) runs it with `--fast` and refuses a failing push. It is
|
||||||
`felhom.eu/documentation/runbooks/target-selection.md`** — read it before any destructive test.
|
**per-clone** — switch it on once with `git config core.hooksPath .githooks`, and a manual run WARNS
|
||||||
Drill and build VMs belong on `demo-hp`, not DooPlex (operator ruling 2026-07-25).
|
when this clone is unarmed. `git push --no-verify` bypasses it deliberately; **say so in the session
|
||||||
|
report when you use it** — CI re-runs the same entry point on every push and **emails the operator on
|
||||||
The agent pins the served leaf cert — verify the fingerprint still matches before a live run.
|
failure**, so a bypass is noticed even though it is not blocked (R-168, CLOSED 2026-08-02).
|
||||||
Selftest modes (run locally on DooPlex, pointed at the demo API):
|
|
||||||
`--selftest[=read|task|hub|storage|backup|restore-test|pbs-verify]`; no flag = the daemon.
|
|
||||||
|
|
||||||
<!--
|
<!--
|
||||||
`localapi` does not bind a LAN literal: since the R-50 island migration (2026-07-25) it binds
|
WHY ONE ENTRY POINT (2026-08-02, R-29): a census of all gates across the four repos found every check
|
||||||
169.254.253.1:8443 on vmbr9, which is location-independent by design, and proxmox.endpoint is
|
a CLAUDE.md names was passing, and two of the four nobody is told to run were failing. This repo was
|
||||||
https://127.0.0.1:8006. That is why the N100 being travel-portable needs no config edit. The
|
the extreme case — nothing ran against it at all, and 90 cited paths were checked by no one.
|
||||||
2026-07-20..08-02 vacation window and its findings are recorded in
|
|
||||||
felhom.eu/documentation/audits/AUDIT-vacation-remote-ops-2026-07-20.md.
|
|
||||||
|
|
||||||
LEGACY: WINDOWS WORKSTATION — until 2026-07-19 CC ran on Windows 11; pct commands over SSH needed
|
|
||||||
export MSYS_NO_PATHCONV=1, and every remote command used SSH=/c/Windows/System32/OpenSSH/ssh.exe.
|
|
||||||
Agent deploy was a two-hop copy via the Windows box (cygpath -w for the local scp path; CRLF hazard
|
|
||||||
on config files). The workspace-root CLAUDE.md carries the full version.
|
|
||||||
-->
|
-->
|
||||||
|
|
||||||
|
## Live validation — the fence
|
||||||
|
|
||||||
|
Exercise the **SERVER-SIDE PIPELINE** a real user triggers, end-to-end. **The forbidden shortcut is
|
||||||
|
BYPASSING it** — the F9 episode was a raw guest-attach with hand-set state, and it proved nothing.
|
||||||
|
|
||||||
|
`claude-in-chrome` is NOT available on DooPlex. Invoking the exact endpoint the UI invokes is an
|
||||||
|
acceptable proxy — **say which method was used**. Low-level mechanism tests where the direct call IS
|
||||||
|
the mechanism are exempt.
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
### Trunk-based — no branches
|
- **Trunk-based — no branches.** All shippable work commits directly to `main`; `main` equals what is
|
||||||
|
deployed. Report-only artifacts (audits, findings, fixspecs) go to `felhom.eu/documentation/`.
|
||||||
All shippable work commits **directly to `main`**; `main` equals what is deployed.
|
- **Unattended escape hatch:** if a fix cannot be cleanly verified and shipped, **revert and report**
|
||||||
- Report-only artifacts (audits, findings, fixspecs) → `felhom.eu/documentation/` (`audits/`, `backlog/`).
|
— never park it on a branch.
|
||||||
- Risky/supervised fixes are spec'd, then implemented **during the supervised session, on `main`**.
|
- **Logging**: the slog logger fans out to journald (configured level) plus the always-DEBUG
|
||||||
- Unattended escape hatch: if a fix can't be cleanly verified/shipped, revert + report — never park on a branch.
|
`applog.Ring` (remote pulls). English, keys-never-values, durations on outcomes. Full rules:
|
||||||
|
|
||||||
> **In every repository where you make a change, update both files in that repo:**
|
|
||||||
> - **`CHANGELOG.md`** — cumulative log, newest on top.
|
|
||||||
> - **`REPORT.md`** — **overwrite** with the most recent implementation/validation summary only.
|
|
||||||
>
|
|
||||||
> **Never write secrets** into any committed file — reference them as "stored out-of-band".
|
|
||||||
|
|
||||||
- Code quality: verify generated code for bugs/edge cases; add debug logging; **ask rather than
|
|
||||||
guess** when you'd otherwise invent input/output.
|
|
||||||
- **A health check issues no block I/O** — full rule, measurement and scope:
|
|
||||||
`.claude/rules/health-checks.md`, which loads when you touch the packages that write them.
|
|
||||||
- Update `REUSE.md` if you added/changed/deprecated a shared helper or pattern (same commit).
|
|
||||||
- **Run `python3 scripts/agent_gates.py` from the repo root after ANY change in this repo.** It is
|
|
||||||
the ONE entry point for this repo's gates. Today it runs one — `reuse_refs_check` over this
|
|
||||||
repo's `REUSE.md` — and it exists at one gate on purpose: a census on 2026-08-02 found that every
|
|
||||||
check a `CLAUDE.md` names was passing and two of the four nobody is told to run were failing, and
|
|
||||||
this repo was the extreme case, with nothing running against it at all and 90 cited paths checked
|
|
||||||
by no one. It grows when the agent grows a second check. `--fast` selects the gates that touch no
|
|
||||||
network and no container runtime; today that is all of them. A missing gate is a FAILURE, never a
|
|
||||||
skip. **The shared `reuse_refs_check.py` lives in `felhom.eu/scripts/` and is never copied here**
|
|
||||||
— a copy would recreate the drift it detects; an absent sibling clone FAILS the gate.
|
|
||||||
**The pre-push hook** (`.githooks/pre-push`) runs it with `--fast` and refuses a failing push. It
|
|
||||||
is per-clone — switch it on once with `git config core.hooksPath .githooks`, and a manual run
|
|
||||||
WARNS when this clone is unarmed. `git push --no-verify` bypasses it deliberately; **say so in the
|
|
||||||
session report when you use it** — CI re-runs the same entry point on every push and **emails the
|
|
||||||
operator on failure**, so a bypass is noticed even though it is not blocked (R-168, CLOSED
|
|
||||||
2026-08-02). This file already said so at the release section; the two now agree.
|
|
||||||
- Testing doctrine (non-hollow tests, red-proofs, seams): use the `felhom-testing` skill.
|
|
||||||
- **Logging**: the slog logger fans out to journald (configured level) + the always-DEBUG `applog.Ring`
|
|
||||||
(remote pulls) — English, keys-never-values, durations on outcomes; full rules in
|
|
||||||
`felhom.eu/documentation/runbooks/logging-conventions.md`.
|
`felhom.eu/documentation/runbooks/logging-conventions.md`.
|
||||||
|
- Update `REUSE.md` in the same commit that adds, changes or deprecates a shared helper or pattern.
|
||||||
|
|
||||||
### Live validation
|
## End-of-session checklist
|
||||||
|
|
||||||
Exercise the SERVER-SIDE PIPELINE a real user triggers, end-to-end. The forbidden shortcut is
|
- **`CHANGELOG.md`** (cumulative, newest on top) and **`REPORT.md`** (overwritten with this run only)
|
||||||
BYPASSING it (the F9 episode: raw guest-attach + hand-set state). Invoking the exact endpoint the UI
|
— in every repo touched.
|
||||||
invokes is an acceptable proxy when a browser isn't available — say which method was used. Low-level
|
- **`CONTEXT.md`** — decisions, state, what is next.
|
||||||
mechanism tests where the direct call IS the mechanism are exempt.
|
- **`REUSE.md`** — if a shared helper or pattern moved.
|
||||||
|
- **A finding goes in `felhom.eu/documentation/backlog/OPEN-ITEMS.md` first**, never only in a report
|
||||||
## Workflow & artifacts
|
or an audit.
|
||||||
|
- **Confirm your own last push's CI run went green, by run ID** — CI mails on failure, which is a PUSH
|
||||||
- Implement **`TASK.md` / `TASK-*.md`** specs (when placed as `TASK.md` or told to), then push +
|
signal; this is the PULL check that catches a lost or unread mail. An unchecked green is an
|
||||||
CHANGELOG + REPORT.md.
|
assumption, not an observation.
|
||||||
- **`RUNBOOK-*.md`** — an operational procedure. CC executes the steps it has access and capability
|
|
||||||
for, including live validation on the demo Proxmox host (CC has root@felhom-pve SSH + the
|
|
||||||
felhom-agent token). Mark a step HUMAN only when it genuinely needs physical presence, a real-world
|
|
||||||
decision, or credentials CC truly lacks. Judgment still applies: confirm before irreversible ops on
|
|
||||||
real customer data — demo scratch guests are fair game.
|
|
||||||
|
|||||||
Reference in New Issue
Block a user