admin
af8a7a58a4
PROMPT-TEMPLATE: standard 'For the operator' plain-language section, mandatory for M+ tasks and anything with a STOP. ROADMAP rulings (operator, 2026-07-21): R-25b full-teardown cascade with three acks + typed name (M-sized, spec to follow, no longer blocks R-3); R-11 channel = direct Messenger, doc is the architect's; R-42 option (a), sidecars follow the app; R-17 delete the archive - spike-lite found NO tooling verb targets it, so it is an operator console action; R-4 complete (freemail.hu verified). R-55 + R-41 slice 1 marked shipped; new R-56 (app difficulty classification - the constructive half of the glance ruling). scripts/build-hub.sh v1.23.0: the hub build script was outside any repo. Adopted verbatim + versioned; the build-dir path is now a symlink to it. felhom-testing skill: the ~1/5 recovery-code 'known flake' is retired - it was a real defect the test was correctly detecting.
75 lines
4.5 KiB
Markdown
75 lines
4.5 KiB
Markdown
---
|
||
name: felhom-testing
|
||
description: Felhom testing doctrine — use when writing or reviewing ANY Go test in felhom-controller, felhom-agent, or the felhom.eu hub, and for EVERY correctness or security fix (the red-proof is mandatory there). Triggers - "write a test", "add tests", reviewing a diff that changes logic, fixing a bug, hardening a guard, or validating a fix live. Contains the non-hollow rules, the companion red-proof procedure, seam locations, and the green-gate command.
|
||
---
|
||
|
||
# Felhom testing doctrine
|
||
|
||
## Non-hollow rule (the cardinal one)
|
||
|
||
A test must assert the **effect**, not the absence of error. Wrong: HTTP 200 came back, `err == nil`,
|
||
"function ran". Right: the stored row has the expected value, the rendered HTML contains the badge,
|
||
the state file transitioned, the fake recorded the exact command. If deleting the fix wouldn't fail
|
||
the test, the test is hollow.
|
||
|
||
## Companion red-proof (mandatory for every correctness/security fix)
|
||
|
||
Prove the test detects the bug it guards against:
|
||
|
||
1. Temporarily restore the PRE-FIX shape (revert the fixed line, or model the old predicate inline).
|
||
2. Run the test → it must **FAIL**, with the wrong value visible in the failure message.
|
||
3. Restore the fix → test passes. `git diff` clean.
|
||
4. Record the red-proof outcome in `REPORT.md` (what failed, with what value).
|
||
|
||
In-tree exemplars (verified):
|
||
- `felhom.eu/hub/internal/notify/dispatcher_test.go` `TestSeverityNotifies` (~L27–49) — models the
|
||
pre-fix `warning||error` predicate inline and asserts the fix routes what it dropped.
|
||
- `felhom.eu/hub/internal/api/event_test.go` `TestHandleEvent_CriticalPreserved` — asserts the STORED
|
||
severity; its red-proof was run by reverting the one-line switch (stored `"info"` → FAIL).
|
||
|
||
## Seams over shell-outs
|
||
|
||
Never let a unit test touch docker/pct/real /dev. Every repo's **`REUSE.md` §4** lists its seams and
|
||
existing fakes — inject there:
|
||
- controller: `diskAgent` (`mockAgent`), `quiesce.Backend/Stacks`, `channelhealth.Probe/Sink`,
|
||
`selfupdate.AgentSwapper`, `offboxRunner`, `bootstrap.PullFunc`.
|
||
- agent: `proxmox.Runner` (`mockRunner`), `storage.HostOps/HostReader`, localapi `Options` fakes,
|
||
Server seam funcs (`reresolveWipe`, `deviceDurableID` — override in tests, no real /dev).
|
||
- hub: `Dispatcher.sendEmailFn`, `mailrelay.Sender`, `mailRateLimiter.now` (clock),
|
||
`monitor.EventNotifyFunc`, provider interfaces on the api Handler.
|
||
|
||
Test harness conventions: real store on `t.TempDir()` DB (`hub/internal/api/host_test.go`
|
||
`newTestHandler` pattern); `t.Cleanup` for teardown; table-driven where natural; pure
|
||
classifier functions get fixture tables (agent `classifyClaim` style).
|
||
|
||
## What every test suite should also cover
|
||
|
||
- **Negative cases:** the 400/401/403/refusal paths, not just the happy path (e.g. unknown
|
||
event_type → 400 AND nothing stored).
|
||
- **Idempotency:** re-running the op is a clean no-op where the contract says so (registry add,
|
||
intent set, mount ensure).
|
||
- **Fail-safe direction:** for guards, ambiguity must refuse (agent claim/role classifiers are the
|
||
canon — any read error ⇒ most-protected verdict).
|
||
|
||
## Green gate (run before every commit that touches Go code)
|
||
|
||
```bash
|
||
go build ./... && go vet ./... && go test ./...
|
||
```
|
||
Run it in the module dir: `felhom.eu/hub/`, `felhom-controller/controller/`, `felhom-agent/` root.
|
||
**No known flakes.** The long-standing agent `TestGenerateRecoveryCode_EntropyAndFormat` ~1/5 failure
|
||
was **fixed in agent v0.93.0 (2026-07-21)** — and it was never a flaky test. It was a real defect the
|
||
test was correctly detecting and everyone had been told to re-run past: the EFF wordlist contains four
|
||
hyphenated entries, so a recovery code could come out reading as 11 words instead of 10. If it fails
|
||
now, it is a regression. **The lesson generalises: "known flake, just re-run it" is a diagnosis, and
|
||
it needs the same evidence as any other one.** A test that fails at a stable, explainable rate is
|
||
usually telling the truth about a rare input, not misbehaving.
|
||
|
||
## Live validation doctrine (after unit-land)
|
||
|
||
Exercise the SERVER-SIDE PIPELINE a real user triggers, end-to-end — never hand-set state around it
|
||
(the F9 lesson). Invoking the exact endpoint the UI invokes is an acceptable proxy when a browser
|
||
isn't available; the residual is client-side rendering only — SAY which method was used. Test crash
|
||
behavior with kill -9 / OOM, never `docker kill` (containers' restart policy masks the difference).
|
||
Don't echo API keys/tokens into logs or REPORT — extract into shell vars, print lengths only.
|