From e6fe7319876d4e05539b957af26e5f26109444a0 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Tue, 6 Oct 2026 01:45:49 +0200 Subject: [PATCH] decoy coverage: 8 exemptions removed (agent + catalog suites, R-426; 4 left, controller) Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS --- .../night-burndown-2026-10-05/NIGHT-LOG.md | 1 + documentation/backlog/OPEN-ITEMS.md | 2 +- scripts/CHANGELOG.md | 4 ++++ scripts/decoy_coverage_gate.py | 19 +------------------ 4 files changed, 7 insertions(+), 19 deletions(-) diff --git a/documentation/audits/night-burndown-2026-10-05/NIGHT-LOG.md b/documentation/audits/night-burndown-2026-10-05/NIGHT-LOG.md index b0fa37e7..01cb7072 100644 --- a/documentation/audits/night-burndown-2026-10-05/NIGHT-LOG.md +++ b/documentation/audits/night-burndown-2026-10-05/NIGHT-LOG.md @@ -102,3 +102,4 @@ cherry-picks onto `main`, writes CHANGELOG, closes rows, pushes and watches CI. | rows | 25 closed as delivered (164 open after) | | fixed without a row | `test_golden_currency_gate.py` used the local date, the gate UTC — red between 00:00 and 02:00 CEST (red-proved) | | R-426 | felhom.eu side done: 16 → 12 exemptions; 3 foolable gates fixed (hostinstall R-185 count, age install, one-register leading verdict); decision 135 | 25 | felhom.eu night/gates ×4 | +| R-426 | agent + catalog decoys (8 exemptions removed; image-pins: 3 holes closed); 4 left (controller); agent go test red 21:25–01:55 from the installer change — fixed (test reads ISO first-boot files the installer only names) | 35 | agent `b2b82ae`, catalog `75f35842` | diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index a1468e6b..054602cc 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -317,7 +317,7 @@ stopping line that lies. | **R-392** | Process & tooling | P4 | **No architecture document covers the two-AI workflow.** `documentation/architecture/` holds eight documents and **all eight cover the product** — topology, host agent, control-plane authorization, hub, off-site connectivity, backup, controller modules, capability map. Nothing records how the Claude.ai / Claude Code split works, what each side owns, how skills and `.claude/rules/` are scoped, or why. **The absence was found by trying to fill the template field, not by a survey:** the task that added the five process skills (2026-08-25) had to name an owning architecture document and could not, and the template requires that be recorded rather than passed over. The exposure today is low — the split is stable and both sides work — but it lives entirely in the operator's head and in chat, which is precisely the shape of a commitment nothing enforces. | **OPEN — LOW** | — | Write one architecture document for the agent-tooling layer: which AI owns which artifact class (`TASK-*.md`, `RUNBOOK-*.md`, validation), how skills are scoped and installed, what belongs in a `CLAUDE.md` versus a skill versus a rules file, and the reasoning for each boundary. **The rules themselves already exist** in `skills/felhom-doc-authoring/SKILL.md`; what is missing is the map of who owns what. Do not restate the doc-authoring rules there — point at that skill. | CC | | **R-394** | Process & tooling | P4 | **`felhom-build-deploy/SKILL.md` is 179 lines, over the 150-line limit its own repo now enforces.** Found 2026-08-25 by `scripts/check_skills.py` on its first run — the over-length was discovered BY the new checker, on the day the limit was written down, which is the checker working as intended. **It is not edited and not trimmed here:** the task that introduced the limit explicitly scoped the four pre-existing skills out, and trimming a build-and-deploy skill without exercising its commands is how a wrong command ships to a live host. **It is a named single-entry exception in `GRANDFATHERED` in `scripts/check_skills.py`, printed as a WARN on every run**, so it cannot fade; a NEW skill over the limit is convicted normally, and growing the set requires editing that file in a commit with a row to name. **The rationale for the limit** — attention thins across the excess, so the lines that matter are not the ones that survive — is in `skills/felhom-doc-authoring/SKILL.md` §5. | **OPEN — LOW** | — | Trim `felhom-build-deploy/SKILL.md` under 150 lines in a session that can VERIFY the commands it keeps, then delete its `GRANDFATHERED` entry in the same commit. The likely trim is the per-artifact command blocks moving behind a pointer to the runbooks, keeping the gotchas inline — but that is a judgement for a session with a build to run, not a line-count exercise. | CC | | **R-421** | Process & tooling | P4 | **THE CLASS: an instrument that matches a LABEL rather than the fact it names — five instances, every one found by accident.** R-410 (a `mkdir` turned the release gate green), R-400 (seven debug controls answering nothing), R-378 (a status word inside a sentence), R-419 (a phrase inside prose, including prose saying the marker was ABSENT), R-94 (a test comparing a constant to itself). **The gates are the machinery that enforces everything else in this project, and they were the one part nothing had ever checked.** The 2026-09-01 decoy sweep read all 29 scripts and fooled **16**. Ten were fixed the same day; four remain with rows (R-422..R-425); six could not be given a plausible decoy and are named. **The shapes, so the next one is cheap to recognise:** (1) name-for-fact — it matches a path or directory NAME while the fact lives inside the file; (2) substring-for-field — it matches a token anywhere in a body instead of in the field that carries it; (3) declaration-for-reachability — it checks a thing is declared, not that it RESOLVES; (4) constant-for-measurement — it compares a value against itself. **The single largest cause was mundane:** eight gates set their SCOPE with `os.listdir` (one level), so every one was green and correct today and would have gone blind the moment anyone added a subdirectory. `decoy_coverage_gate.py` now refuses a new gate that ships without a decoy. | **OPEN — the class row; it stays open as the place the next instance is recorded** | — | — | CC | -| **R-426** | Process & tooling | P4 | **The decoy-coverage exemption list — 20 registered gates that ship WITHOUT a decoy test, each named.** `scripts/decoy_coverage_gate.py`'s `EXEMPT` map is debt, and this row owns it so it lives in the register and not only in a Python literal. **Four kinds:** (a) genuinely covered in the 2026-09-01 sweep but not yet moved into a suite — `hub-copy`, `instructions`, `docker-v`, `image-pins`; (b) blocked by an open hole and therefore un-assertable as rejecting — `site` (R-423), `one-register` (R-424), `offbox-rename` (R-425); (c) shared scripts whose decoy lives in `felhom.eu` and is counted there — `reuse-refs`, `instructions`, `observations` in the controller and agent runners; (d) **no plausible decoy constructed yet** — `hostinstall`, `wire-contract`, `due-checks`, `published`, `image-resolvable`, `volume-persistence`. Group (d) is the honest unknown: six gates whose soundness is UNTESTED, not established. **The list is green today and shrinks; a NEW gate with no decoy fails immediately.** | **OPEN — 20 names; group (d) is six untested gates** **2026-10-05 (burn-down night): PARTIAL** (`68df84bc`): the instructions gate's decoy is in the suite; the exemption list is **16** (was 20; wire-contract has decoys now). The row stays open for the 16. **2026-10-06 (burn-down night, later): the felhom.eu side is DONE** — hostinstall, due-checks and one-register have decoys (three gates could be fooled; all three fixed), a dead controller entry removed, a STALE EXEMPTION notice added; **12 exemptions left, all in the controller, agent and catalog runners** (decision 135). | — | — | CC | +| **R-426** | Process & tooling | P4 | **The decoy-coverage exemption list — 20 registered gates that ship WITHOUT a decoy test, each named.** `scripts/decoy_coverage_gate.py`'s `EXEMPT` map is debt, and this row owns it so it lives in the register and not only in a Python literal. **Four kinds:** (a) genuinely covered in the 2026-09-01 sweep but not yet moved into a suite — `hub-copy`, `instructions`, `docker-v`, `image-pins`; (b) blocked by an open hole and therefore un-assertable as rejecting — `site` (R-423), `one-register` (R-424), `offbox-rename` (R-425); (c) shared scripts whose decoy lives in `felhom.eu` and is counted there — `reuse-refs`, `instructions`, `observations` in the controller and agent runners; (d) **no plausible decoy constructed yet** — `hostinstall`, `wire-contract`, `due-checks`, `published`, `image-resolvable`, `volume-persistence`. Group (d) is the honest unknown: six gates whose soundness is UNTESTED, not established. **The list is green today and shrinks; a NEW gate with no decoy fails immediately.** | **OPEN — 20 names; group (d) is six untested gates** **2026-10-05 (burn-down night): PARTIAL** (`68df84bc`): the instructions gate's decoy is in the suite; the exemption list is **16** (was 20; wire-contract has decoys now). The row stays open for the 16. **2026-10-06 (burn-down night, later): the felhom.eu side is DONE** — hostinstall, due-checks and one-register have decoys (three gates could be fooled; all three fixed), a dead controller entry removed, a STALE EXEMPTION notice added; **12 exemptions left, all in the controller, agent and catalog runners** (decision 135). **2026-10-06 01:45: agent and catalog sides DONE** — new decoy suites (agent `scripts/test_gate_decoys.py`: published against a fake Gitea, release-complete, the shared reuse-refs/instructions/observations; catalog: image-pins — **three holes closed** (a quoted `"image":` key, `${APP_IMAGE:-x}`, a bare `@sha256:`), image-resolvable and volume-persistence through PATH-stub docker); 8 exemptions removed. **4 left, all felhom-controller** (docker-v, reuse-refs, instructions, observations). Seen, not changed: image-resolvable reads `no such host` as GONE, so a DNS outage would cry wolf (a judgement for the operator). | — | — | CC | | **R-502** | Process & tooling | P4 | **[P3-LOW] The bootstrap regression harness is run by NO gate and NO CI — and it had never exercised the pairing banner.** MEASURED 2026-09-14: `grep -rn bootstrap-modes scripts/*.py .gitea/workflows` → nothing; the harness is run by hand in `felhom-iso-assistant:trixie`. While adding the R-496 checks, its fake hub's `register` reply turned out to carry no `pairing_code`, so `print_pairing_banner` returned early in every run: the banner that a household reads first had no test at all until ISO v1.27.0. **Fix shape:** register the harness in `repo_gates.py` behind a container-available check that reports INCONCLUSIVE (never skip-as-pass) when docker is absent, with a decoy. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: test harness coverage; no household meets it directly.** **2026-10-05 (burn-down night): NEEDS THE OPERATOR.** The harness runs `docker run`; registered as a gate it would run Docker on DooPlex on every full gate run. Next: an operator word that a non-fast container gate may run there (INCONCLUSIVE without docker; never in --fast — CI has no docker). | — | — | CC | | **R-507** | Process & tooling | P4 | **[P3-LOW] The proof-install harness cannot drive the graphical installer, so a release's graphical entry is proven only up to its password screen.** MEASURED 2026-09-14 on VM 332 (ISO 1.27.0): `qm sendkey 332 tab` did not move focus (both password copies landed in one field), `mouse_move 1237 772` + `mouse_button 1` did not move the cursor or press Next, while `alt-n` did advance a page. The TUI entry is fully drivable. The gate's "proof install on BOTH menu entries" was met for 1.26.1 (by a person) and not for 1.27.x. **Fix shape:** measure QEMU `input-send-event` with absolute coordinates, or a VNC client on DooPlex; until then a release's graphical proof is an operator click-through. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: release-test tooling; an operator click-through covers it.** | — | — | CC | | **R-579** | Process & tooling | P4 | **[P3-LOW] Five page shells loaded `style.css` with NO cache-buster, so a browser holding an older copy kept being served CSS that did not know about the newest UI.** FOUND 2026-09-18 by the operator's screenshot of the v0.254.0 language globe: it rendered as a bare, unstyled `
` — a stray triangle and two plain words outside the card — because `login.html`, `claim.html`, `recovery.html`, `launcher_shared.html` and `launcher_share_password.html` requested `/static/style.css` with no `?v=`, while `layout.html` has used `?v={{.Version}}` since v0.166.0. **`.Version` was also absent from three of those five data maps.** FIXED AND CLOSED in the same session (controller v0.255.0): the parameter on all five, and `Version` set once in `executeTemplateLang` so a new shell cannot miss it; `TestGlobeOnAnonymousShells` now refuses an absent or EMPTY `?v=`. **The general form, which is the part worth keeping: a template that loads a versioned asset WITHOUT its version is invisible to every test that reads markup — the markup is correct and the browser fetches the wrong file.** A gate over "every stylesheet/script link in a template carries `?v=`" would catch the class; not built, because 8 first-boot-wizard templates would fail it and R-554 deletes them. | **DEFERRED** (2026-10-03 triage: the row's verdict was finished, but it names open work no other row carries — the gate "every stylesheet/script link carries `?v=`" was deliberately not built until R-554 deletes the old wizard; nothing else owns it) — **CLOSED 2026-09-18 - controller v0.255.0** | — | — | CC | diff --git a/scripts/CHANGELOG.md b/scripts/CHANGELOG.md index 38ec6ecb..7a03aede 100644 --- a/scripts/CHANGELOG.md +++ b/scripts/CHANGELOG.md @@ -1,3 +1,7 @@ +## gates — decoy coverage: 8 exemptions removed (agent + catalog suites exist now; R-426) (2026-10-06) + +- `decoy_coverage_gate.py` EXEMPT: the five felhom-agent and three app-catalog entries are gone — each repo's `scripts/test_gate_decoys.py` COVERS them. 4 left, all felhom-controller. + ## gates — R-426: the felhom.eu exemptions get decoys, and three gates that could be fooled are fixed (burn-down night, 2026-10-06) - R-426: hostinstall gate — the R-185 grant count now counts only real path-qualified invocations (the dry-run echo had stood in for a deleted grant) and the age-install check ignores commented-out lines; four decoys in test_gate_decoys.py, `felhom.eu/hostinstall` leaves the decoy-coverage EXEMPT list. diff --git a/scripts/decoy_coverage_gate.py b/scripts/decoy_coverage_gate.py index 0ee8a021..303e6a07 100644 --- a/scripts/decoy_coverage_gate.py +++ b/scripts/decoy_coverage_gate.py @@ -43,24 +43,7 @@ EXEMPT = { "shared script; see felhom.eu. R-426.", ("felhom-controller", "observations"): "shared script; its decoy (R-419) lives in felhom.eu/scripts/test_gate_decoys.py.", - # felhom-agent — no decoy suite yet - ("felhom-agent", "reuse-refs"): "shared script; decoy in felhom.eu. R-426.", - ("felhom-agent", "instructions"): "shared script; decoy in felhom.eu. R-426.", - ("felhom-agent", "observations"): "shared script; decoy in felhom.eu. R-426.", - ("felhom-agent", "published"): - "R-426 — a network gate; a decoy needs a fake registry, which this sweep did not build.", - ("felhom-agent", "release-complete"): - "R-426 — the sweep's decoy (a non-version heading on top of CHANGELOG.md) was WITHDRAWN: " - "HEAD_RE.search scans the whole file, so the real release is still found and named. The " - "gate looked sound; no legitimate decoy has been constructed yet.", - # app-catalog - ("app-catalog-felhom.eu", "image-pins"): - "COVERED IN THE SWEEP: a real untagged `image:` line was REJECTED, and the `x-image:` decoy " - "was WITHDRAWN as illegitimate (x- fields are inert in Compose). R-426 tracks a suite.", - ("app-catalog-felhom.eu", "image-resolvable"): - "R-426 — needs a container runtime and the network; not a --fast gate.", - ("app-catalog-felhom.eu", "volume-persistence"): - "R-426 — 877 lines that actually run containers and diff them; not a --fast gate.", + # felhom-agent and app-catalog: covered by their own scripts/test_gate_decoys.py since 2026-10-06 (R-426) } RUNNERS = {