The four existing skills cover the product; nothing covered how work is reported. Two rules this project has paid for — check the artifact rather than the report, and do not state a claim more firmly than the evidence allows — lived only in the operator's head and in chat, where Claude Code never read them. - felhom-evidence five confidence tiers, artifact-over-report - felhom-diagnosis no hypothesis until a command has been seen red - felhom-plain-language ASD-STE100, two options, the re-pitch - felhom-handoff the note goes to a FILE, not the conversation - felhom-doc-authoring the pointer decides whether material is reached scripts/check_skills.py asserts what decides whether a skill is EVER reached: frontmatter parses, name == directory, description and body non-empty, under 150 lines, installed copy still samefile()s into the repo. install_skills.py globs and never reads the file, so a missing description installs perfectly and then silently never loads. It convicted on its first run: felhom-build-deploy is 179 lines. NOT trimmed here (pre-existing skills are out of scope, and trimming a deploy skill without exercising its commands is how a wrong command reaches a live host) — a named single-entry GRANDFATHERED exception, WARNed every run, R-394. A new skill over the limit is convicted. Red-proof run and seen failing: description removed from felhom-evidence -> exit 1, "frontmatter field 'description' is missing or empty". Restored, tree clean. skills/SOURCES.md records both MIT upstreams, that these are adaptations not copies, and the six pieces deliberately EXCLUDED with reasons. Register: R-392 (no architecture doc covers the two-AI workflow), R-393 (decision-log skill deferred, with the reason), R-394.
This commit is contained in:
Executable
+155
@@ -0,0 +1,155 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""check_skills.py — assert every Felhom SKILL.md is well formed and live-linked.
|
||||
|
||||
Run from anywhere: python3 scripts/check_skills.py
|
||||
Exit 0 clean · 1 convicted (at least one skill is malformed).
|
||||
|
||||
WHY THIS EXISTS. `install_skills.py` discovers skills by globbing `skills/*/SKILL.md` and creates a
|
||||
symlink for each. It does not read the file. A SKILL.md whose frontmatter is missing a `description`
|
||||
therefore installs perfectly and then never loads, and NOTHING SAYS SO — the failure is silent at
|
||||
exactly the point a skill is supposed to fire. "The file exists" is a hollow check; this script
|
||||
asserts the properties that decide whether the material is ever reached.
|
||||
|
||||
It reports EVERY offending file and field, never stopping at the first — a checker that stops early
|
||||
turns one fix into several runs.
|
||||
|
||||
NO THIRD-PARTY DEPENDENCY, deliberately. The frontmatter here is two simple `key: value` lines; a
|
||||
YAML library would be a new install requirement on every machine that runs the gates, bought for
|
||||
nothing. If the frontmatter ever needs real YAML, that is the moment to reconsider — not before.
|
||||
|
||||
WHAT THIS CANNOT SEE, stated so it is not mistaken for coverage it does not give: whether the model
|
||||
actually REACHES a skill when it should. That is behavioural, it is decided by the wording of the
|
||||
`description`, and no mechanical check can settle it. A green run here means the skill is loadable,
|
||||
never that it fires.
|
||||
"""
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
SRC = os.path.join(REPO, "skills")
|
||||
INSTALLED = os.path.join(os.path.expanduser("~"), ".claude", "skills")
|
||||
|
||||
MAX_LINES = 150
|
||||
|
||||
# ── GRANDFATHERED, and named rather than silently exempted ───────────────────────────────────────
|
||||
# `felhom-build-deploy` was 179 lines on the day this checker was written (2026-08-25), which is how
|
||||
# the over-length was discovered at all. The task that added this script forbids editing the four
|
||||
# pre-existing skills, so trimming it here would have been out of scope AND would have hidden the
|
||||
# finding. It is therefore an EXPLICIT, single-entry exception carrying its register row, printed as
|
||||
# a WARN on every run so it cannot fade into the background.
|
||||
#
|
||||
# THE SET CANNOT GROW SILENTLY: a NEW skill over the limit is not in this dict and is convicted
|
||||
# normally. Adding an entry is an edit to this file, in a commit, with a row to name.
|
||||
GRANDFATHERED = {
|
||||
"felhom-build-deploy": "R-394 — 179 lines when the limit was introduced; trim is a scoped session",
|
||||
}
|
||||
|
||||
|
||||
def parse_frontmatter(lines):
|
||||
"""Return (fields, body_lines, error). Frontmatter is the block between the first two '---'."""
|
||||
if not lines or lines[0].strip() != "---":
|
||||
return None, None, "does not start with a '---' frontmatter block"
|
||||
close = None
|
||||
for i in range(1, len(lines)):
|
||||
if lines[i].strip() == "---":
|
||||
close = i
|
||||
break
|
||||
if close is None:
|
||||
return None, None, "frontmatter block is never closed by a second '---'"
|
||||
fields = {}
|
||||
for raw in lines[1:close]:
|
||||
if not raw.strip():
|
||||
continue
|
||||
if raw.startswith((" ", "\t")):
|
||||
continue # continuation of the previous value — the key is what we check
|
||||
if ":" not in raw:
|
||||
return None, None, "frontmatter line is not 'key: value': %r" % raw.strip()[:60]
|
||||
k, v = raw.split(":", 1)
|
||||
fields[k.strip()] = v.strip()
|
||||
return fields, lines[close + 1:], None
|
||||
|
||||
|
||||
def check(name, problems, warnings):
|
||||
path = os.path.join(SRC, name, "SKILL.md")
|
||||
with io.open(path, "r", encoding="utf-8") as fh:
|
||||
text = fh.read()
|
||||
lines = text.splitlines()
|
||||
|
||||
fields, body, err = parse_frontmatter(lines)
|
||||
if err:
|
||||
problems.append("%s: %s" % (path, err))
|
||||
return
|
||||
|
||||
if not fields.get("name"):
|
||||
problems.append("%s: frontmatter field 'name' is missing or empty" % path)
|
||||
elif fields["name"] != name:
|
||||
problems.append("%s: frontmatter 'name' is %r but the directory is %r — they must match"
|
||||
% (path, fields["name"], name))
|
||||
|
||||
if not fields.get("description"):
|
||||
problems.append("%s: frontmatter field 'description' is missing or empty" % path)
|
||||
|
||||
if body is not None and not "".join(body).strip():
|
||||
problems.append("%s: the body below the frontmatter is empty" % path)
|
||||
|
||||
n = len(lines)
|
||||
if n >= MAX_LINES:
|
||||
if name in GRANDFATHERED:
|
||||
warnings.append("%s: %d lines, OVER the %d-line limit — grandfathered (%s)"
|
||||
% (name, n, MAX_LINES, GRANDFATHERED[name]))
|
||||
else:
|
||||
problems.append("%s: %d lines — a SKILL.md must be UNDER %d" % (path, n, MAX_LINES))
|
||||
|
||||
# installed copy, if the installer has been run: it must resolve back into the repo
|
||||
inst = os.path.join(INSTALLED, name, "SKILL.md")
|
||||
link = "not installed"
|
||||
if os.path.exists(inst):
|
||||
try:
|
||||
if os.path.samefile(inst, path):
|
||||
link = "live-linked"
|
||||
else:
|
||||
problems.append("%s: installed copy %s does NOT resolve to the repo file — a stale "
|
||||
"copy will be read instead of your edits" % (path, inst))
|
||||
link = "STALE COPY"
|
||||
except OSError as e:
|
||||
problems.append("%s: cannot compare with installed copy %s (%s)" % (path, inst, e))
|
||||
link = "UNREADABLE"
|
||||
else:
|
||||
warnings.append("%s: not installed under %s — run scripts/install_skills.py" % (name, INSTALLED))
|
||||
|
||||
print("OK %-24s %3d lines %s" % (name, n, link))
|
||||
|
||||
|
||||
def main():
|
||||
if not os.path.isdir(SRC):
|
||||
print("FAIL: no skills/ dir at %s" % SRC)
|
||||
return 1
|
||||
names = sorted(d for d in os.listdir(SRC)
|
||||
if os.path.isfile(os.path.join(SRC, d, "SKILL.md")))
|
||||
if not names:
|
||||
print("FAIL: no skills found under %s" % SRC)
|
||||
return 1
|
||||
|
||||
problems, warnings = [], []
|
||||
for n in names:
|
||||
try:
|
||||
check(n, problems, warnings)
|
||||
except OSError as e:
|
||||
problems.append("%s: unreadable (%s)" % (n, e))
|
||||
print("FAIL %-24s unreadable" % n)
|
||||
|
||||
for w in warnings:
|
||||
print("WARN %s" % w)
|
||||
if problems:
|
||||
print("\nFAIL: %d problem(s) across %d skill(s):" % (len(problems), len(names)))
|
||||
for p in problems:
|
||||
print(" - %s" % p)
|
||||
return 1
|
||||
print("\nPASS: %d skill(s) well formed." % len(names))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user