3 Commits

Author SHA1 Message Date
admin 9bd1a54d71 gates: one entry point (scripts/repo_gates.py) + pre-push hook
A census of all thirteen gate scripts across the four felhom repos on 2026-08-02 found one
clean correlation: every check a CLAUDE.md tells a person to run was passing, and two of the
four nobody is told to run were failing — one since 14 July. Neither failure was harmful in
effect (checked line by line); nothing would have said so if they had been. The fix is not
more gates, it is one place to run them from.

repo_gates.py runs site + hostinstall + hub-confirm + manifest-bearer + reuse-refs, streams
each gate's own output, and exits worst-wins non-zero. A missing gate script is a FAILURE and
prints the path tried — fail-closed, because a runner that quietly skips a gate is the
inert-seam failure this project has shipped four times. It copies catalog_gates.py (R-161),
NOT site_gates.py, which is a gate and not a runner.

.githooks/pre-push runs it with --fast and refuses the push. Honest limits are written into
the hook itself: per-clone (core.hooksPath is local config), and --no-verify bypasses it on
purpose. Any manual run WARNS when the clone is unarmed. Measured on git 2.47.3: a relative
core.hooksPath resolves correctly and the hook's cwd is the repo root from any subdirectory.

test_repo_gates.py is a SEAM test — it asserts each member gate's own distinctive stdout, not
the runner's summary line, which an inert runner prints while calling nothing. Red-proofed:
replacing run_gate's body with 'return 0' still prints 'all felhom.eu gates OK' and exits 0,
and turns the seam test red.
2026-08-02 15:22:44 +02:00
admin 2137094799 scripts: reuse_refs_check resolves package shorthand and sibling repos
RED on all four repos with 13 findings, and a hand audit of all 13 on 2026-08-02 found
ZERO genuine drift: twelve were package shorthand whose file sits a couple of directories
deeper, and one (wgsync/reconciler.go, cited by the controller) lives in the hub. REUSE.md
cites by package shorthand and across repos on purpose; the tool was what was wrong.

Resolution order, first hit wins, every non-exact hit PRINTED so a weakening is visible:
exact / suffix / ambiguous (real citation, imprecise shorthand — not a failure) / sibling
repo (as-is or with the sibling's own name stripped from the token) / FAIL. A failure lists
every resolution attempted, so a 'not found' claim names what was tried. Per-root tallies
are the positive observable: '0 failures' alone cannot tell a working checker from a blind
one. Evidence trees (audits/, documentation/tests/) are excluded from the suffix index — a
copy of a file is not the file. An absent sibling is never a failure; an unreadable parent
says so and continues.

Result: 13/13 resolve, all four roots exit 0. felhom.eu 60 exact + 1 suffix; controller 126
exact + 6 suffix + 1 cross-repo; agent 88 + 1 + 1; catalog 17 exact + 3 cross-repo.

New scripts/test_reuse_refs_check.py: 13 fixture tests, one per resolution row plus the kill
condition. Red-proof: making resolve() return 'exact' for an unresolvable token turns 4 of
them red.
2026-08-02 15:16:11 +02:00
admin d319ae573e hub: delete the host-install version label (R-94) + invert hostinstall gate 1
The Setup tab said 'host-install 1.19.0' while the served script was 1.22.0, and had
been wrong since 2026-07-14. Deriving the number honestly is not possible: the Option-1
command downloads felhom-host-install.sh from the website at RUN TIME and the website
git-syncs main every 30s (R-110), so no build-time value in the hub can be true. R-94(a)
offered derive-or-delete; deleted, which removes the drift class instead of automating it.

- configs.go: hostInstallVersion const, pageData.ScriptVersion field and its assignment
  all removed; a NOTE in their place records why there is no constant here.
- customer_unified.html: the sentence now says the command always fetches the current
  installer, and renders no version.
- hostinstall_gates.py gate 1: the third assertion INVERTS — it used to require the hub
  const to equal SCRIPT_VERSION, it now asserts the hub carries no host-install version
  literal at all, matched in six code shapes across every .go/.html under hub/ (comments
  are deliberately not stripped: a // inside a URL literal would blind the scan).
- render_test.go: the assertion 'html contains hostInstallVersion' compared the constant
  to itself and passed at ANY value — demonstrated green with the const at 9.9.9 while the
  script was 1.22.0. Deleted, not replaced: there is no longer a version to assert.
- felhom-host-install.sh: COMMENT ONLY (SCRIPT_VERSION untouched) — it claimed the gate
  keeps the hub copy equal, an invariant that no longer exists.

Red-proofs: restoring the const fails the rewritten gate 1 (3 shapes hit); the old
render_test assertion passes at 9.9.9.
2026-08-02 15:16:01 +02:00
11 changed files with 690 additions and 61 deletions
+47
View File
@@ -0,0 +1,47 @@
#!/bin/sh
# pre-push — refuse a push that carries a broken gate. (2026-08-02, R-29 leg (b) first half.)
#
# Runs this repo's ONE gate entry point in --fast mode: only checks that touch no network and no
# container runtime, so a push stays a push and never pulls images or starts containers. The slow
# gates stay deliberate periodic runs; a hook that takes minutes gets bypassed within a week and
# the bypass becomes the habit.
#
# BOTH LINES BELOW ARE DELIBERATE. An absent log line is not evidence a hook ran — a silent pass is
# equally consistent with "gates green" and "hook never fired", so a passing push says so out loud.
#
# HONEST LIMITS, stated so this is not mistaken for enforcement it cannot provide:
# * per-clone — core.hooksPath is local config and a clone does not carry it. Arm a clone once:
# git config core.hooksPath .githooks
# Any manual entry-point run WARNS when the clone is unarmed.
# * skippable — `git push --no-verify` bypasses this entirely. That is on purpose: an escape
# hatch that cannot be reached is one that gets removed the first time it is
# inconvenient. USING IT MUST BE STATED IN THE SESSION REPORT.
# The half that is neither per-clone nor skippable is CI — felhom.eu OPEN-ITEMS.md R-168.
#
# Measured 2026-08-02 (git 2.47.3): a relative core.hooksPath resolves correctly and the hook's cwd
# is the repo root whether `git push` is issued from the root or from any subdirectory. The
# explicit rev-parse below does not depend on that.
set -u
root=$(git rev-parse --show-toplevel 2>/dev/null) || {
echo "pre-push: FAIL - cannot resolve the repo root (git rev-parse --show-toplevel)." >&2
exit 1
}
cd "$root" || exit 1
if ! command -v python3 >/dev/null 2>&1; then
echo "pre-push: FAIL - python3 not found, so the gates CANNOT run. This is a failure, never a" >&2
echo " pass by default. Install python3, or push with --no-verify and say so." >&2
exit 1
fi
echo "pre-push [felhom.eu]: running scripts/repo_gates.py --fast ..."
python3 "scripts/repo_gates.py" --fast
rc=$?
if [ "$rc" -ne 0 ]; then
echo "pre-push [felhom.eu]: PUSH REFUSED - gates exited $rc. Fix the finding above, or bypass with" >&2
echo " 'git push --no-verify' and state that you did in the session report." >&2
else
echo "pre-push [felhom.eu]: gates OK - push proceeding."
fi
exit $rc
+25 -3
View File
@@ -174,11 +174,33 @@ Steps: commit+push code → `cd /mnt/5_hdd/felhom.eu/build/felhom-hub && ./build
(local) → bump `manifests/hub.yaml` tag + push → ArgoCD hard-refresh + sync (kubectl-patch method in
the skill, now local `sudo kubectl`) → verify Synced/Healthy + rollout + image + startup log.
## Gates — ONE entry point
**Run `python3 scripts/repo_gates.py` after ANY change in this repo.** It is the one entry point
and runs every gate — `site_gates.py`, `hostinstall_gates.py`, `hub_confirm_gate.py`,
`manifest_bearer_gate.py` and `reuse_refs_check.py` on this root — streaming each gate's own output
and exiting non-zero if any fails. `--fast` selects only the gates that touch no network and no
container runtime; today that is all of them. A missing gate script is a FAILURE, never a skip.
**Why a runner and not five invocations** (2026-08-02, R-29): a census of all thirteen gates across
the four repos found that every check a `CLAUDE.md` names was passing, and two of the four nobody
is told to run were failing — one since 14 July. The single-entry-point shape is the only one that
demonstrably gets run here; `app-catalog-felhom.eu/scripts/catalog_gates.py` is the canonical
version of it (R-161) and `repo_gates.py` copies it. `site_gates.py` is a *gate*, not a runner —
do not model new work on it.
**The pre-push hook.** `.githooks/pre-push` runs `repo_gates.py --fast` and refuses the push if it
fails. It is **per-clone** and switched on once with `git config core.hooksPath .githooks` — a
clone does not carry it, and any manual `repo_gates.py` run WARNS when this clone is unarmed.
`git push --no-verify` bypasses it deliberately; **say so in the session report when you use it**.
Both facts are why continuous integration is still owed (`OPEN-ITEMS.md` R-168) — this hook is
local and skippable, and only CI is neither.
## Build & deploy — Website / Manifests
- **Website** auto-deploys via git-sync; just push to `main` (live in 12 min). **Run
`python3 scripts/site_gates.py` after ANY website change**; new pages go into its `PAGES` list.
Emergency edits: https://files.felhom.eu. All `website/` HTML is **UTF-8 with BOM** — preserve it.
- **Website** auto-deploys via git-sync; just push to `main` (live in 12 min). Website changes go
through `repo_gates.py` above (it runs `site_gates.py`); new pages go into that gate's `PAGES`
list. Emergency edits: https://files.felhom.eu. All `website/` HTML is **UTF-8 with BOM** — preserve it.
- **Manifests** are GitOps via the `felhom` app — commit to `main`, then deliberate sync.
## Key patterns
+8 -11
View File
@@ -20,12 +20,14 @@ import (
var validCustomerID = regexp.MustCompile(`^[a-zA-Z0-9.\-]+$`)
// hostInstallVersion is the felhom-host-install.sh version the customer page's install-command
// generator targets. Kept in sync with scripts/felhom-host-install.sh SCRIPT_VERSION — the generator
// only ever emits flags this version parses. Display-only (the Option-1 command downloads the served
// script, which is always current); bump when the generator's flag surface follows a new script.
// Drift is now gated: scripts/hostinstall_gates.py asserts this const == SCRIPT_VERSION (drill F-1).
const hostInstallVersion = "1.19.0"
// NOTE (R-94, 2026-08-02): there is deliberately NO host-install version constant here, and the
// Setup tab renders no version number. The hub cannot know which version a box will run: the
// Option-1 command downloads felhom-host-install.sh from the website at run time, and the website
// git-syncs `main` every 30s (R-110). Any build-time literal here is a guess wearing a version
// number's authority — the previous const said 1.19.0 while the served script was 1.22.0, and had
// been wrong since 2026-07-14. The single version source is scripts/felhom-host-install.sh's
// SCRIPT_VERSION; scripts/hostinstall_gates.py gate 1 now asserts this file's ABSENCE of any
// host-install version literal.
// validSemver matches a bare X.Y.Z controller version (the floor format). Empty is also accepted by
// the floor handlers (clears the override).
@@ -331,9 +333,6 @@ func (s *Server) handleCustomerUnified(w http.ResponseWriter, r *http.Request, c
CSRFField template.HTML
CSRFToken string
// ScriptVersion drives the install-command generator's header (GL-7). Display-only.
ScriptVersion string
// Hosts (v0.47.0): the customer's enrolled hosts for the Host tab — a LIST by design
// (1 today, N for a later HA cluster). Each entry is the hostDetailData view-model map
// the shared host_detail_body sub-template renders.
@@ -484,8 +483,6 @@ func (s *Server) handleCustomerUnified(w http.ResponseWriter, r *http.Request, c
CSRFField: s.csrfField(r),
CSRFToken: s.csrfToken(r),
ScriptVersion: hostInstallVersion,
Hosts: hostViews,
}
+5 -4
View File
@@ -215,10 +215,11 @@ func TestTemplates_InstallGenerator(t *testing.T) {
t.Errorf("generator control missing: %s", id)
}
}
// targets the right script version + carries the client-side customer id
if !strings.Contains(html, hostInstallVersion) {
t.Errorf("ScriptVersion %s not rendered", hostInstallVersion)
}
// carries the client-side customer id. There is deliberately NO version assertion here: R-94
// deleted the rendered host-install version, because the hub cannot know which version a box
// will run (the script is fetched at run time). The assertion that used to sit here compared
// hostInstallVersion to itself and passed at any value — it was demonstrated green with the
// const set to "9.9.9" while the served script was 1.22.0.
if !strings.Contains(html, `data-customer-id="peti-felhom"`) {
t.Errorf("generator missing data-customer-id")
}
@@ -491,9 +491,10 @@
<section class="card">
<h2>Setup Command</h2>
<p class="text-muted" style="margin-bottom: 1rem; font-size: 0.85rem;">
Day-0 host bootstrap for host-install <strong>{{.ScriptVersion}}</strong>. Run on a
freshly-PVE-installed Proxmox <strong>host</strong> as root (create the customer in the
hub first). It enrolls the host, installs + verifies the agent, and provisions the guest;
Day-0 host bootstrap. The command always fetches the <strong>current</strong>
felhom-host-install.sh from felhom.eu &mdash; there is no version to pick here. Run it
on a freshly-PVE-installed Proxmox <strong>host</strong> as root (create the customer in
the hub first). It enrolls the host, installs + verifies the agent, and provisions the guest;
the in-guest controller then pulls its own <code>controller.yaml</code>. The retrieval
passphrase is entered at the no-echo prompt — never on the command line.
</p>
+5 -3
View File
@@ -184,9 +184,11 @@
set -euo pipefail
SCRIPT_VERSION="1.22.0" # the SINGLE version source (F-1): -h, the run banners, and the hub
# Setup-tab copy (hub internal/web/configs.go hostInstallVersion —
# scripts/hostinstall_gates.py asserts the two stay equal) all follow it.
SCRIPT_VERSION="1.22.0" # the SINGLE version source (F-1): -h and the run banners follow it.
# The hub used to carry a copy for its Setup tab; R-94 DELETED it
# (2026-08-02) because the hub cannot know which version a box runs —
# the Setup command fetches this script at run time. scripts/
# hostinstall_gates.py now asserts the hub carries NO version literal.
# 1.16.0: the FELHOM_ESCROW sudoers alias (controller-driven escrow
# ceremony, agent v0.88.0) ships on every install/update — it rides the
# CANONICAL sudoers fetch below (configs/felhom-agent.sudoers from the
+53 -17
View File
@@ -5,9 +5,17 @@ drill-swept findings (DRILL-day0-vm-2026-07-12 F-1/F-7/F-9/F-10 + the ACL-narrow
Run from the repo root: python scripts/hostinstall_gates.py
Gates (all must pass; non-zero exit on any failure):
1. version exactly ONE version source: SCRIPT_VERSION exists, the header line carries
no version literal, and the hub Setup-tab const (hub internal/web/configs.go
hostInstallVersion) equals SCRIPT_VERSION (F-1 structural fix)
1. version exactly ONE version source: SCRIPT_VERSION exists, the header line carries no
version literal, and **the hub carries no host-install version literal at all**.
The third assertion inverted on 2026-08-02 (R-94): it used to require the hub's
`hostInstallVersion` const to EQUAL SCRIPT_VERSION, which is unachievable
honestly the Option-1 install command downloads felhom-host-install.sh from
the website at RUN TIME and the website git-syncs `main` every 30 seconds
(R-110), so the hub cannot know which version a given box will run. A
build-time literal there is a guess with a version number's authority, and the
real one drifted to 1.19.0-vs-1.22.0 and stayed wrong for 19 days. The label was
deleted rather than derived; this gate now pins its absence (F-1 structural fix,
second form).
2. age the `age` package is installed by the agent-install step (F-10)
3. pbs-apply configs/felhom-pbs-apply is fetched + installed to
/usr/local/sbin/felhom-pbs-apply (F-7), and the uninstall removes it
@@ -54,21 +62,49 @@ if re.search(r'felhom-host-install\.sh\s+v\d+\.\d+\.\d+', header):
else:
ok("header has no version literal")
# hub Setup-tab const must equal SCRIPT_VERSION (the copy the drill found at 1.12.0).
if os.path.exists(HUB_CONFIGS) and script_ver:
with io.open(HUB_CONFIGS, "r", encoding="utf-8") as f:
hub_src = f.read()
hm = re.search(r'hostInstallVersion\s*=\s*"(\d+\.\d+\.\d+)"', hub_src)
if not hm:
fail("hub hostInstallVersion const not found in internal/web/configs.go")
elif hm.group(1) != script_ver:
fail("hub Setup-tab hostInstallVersion=%s != SCRIPT_VERSION=%s (F-1: bump both together)"
% (hm.group(1), script_ver))
else:
ok("hub Setup-tab hostInstallVersion matches (%s)" % hm.group(1))
# The hub must carry NO host-install version literal at all (R-94, 2026-08-02). It cannot know
# which version a box will run — the Option-1 command fetches the script from the website at run
# time and the website git-syncs `main` every 30s. The const this replaced said 1.19.0 while the
# served script was 1.22.0, and had been wrong since 2026-07-14.
#
# Matched in CODE SHAPES, never as bare prose: the deleted declarations, the struct field, the
# assignment and the template action, plus a rename-proof generic form of each. Comments are
# deliberately NOT stripped (a `//` inside a URL string literal would truncate the scan and turn
# this gate blind); a comment that merely NAMES the identifier is allowed, and configs.go carries
# exactly such a note explaining the absence.
BANNED = [
(r'\bconst\s+hostInstallVersion\b', "const hostInstallVersion"),
(r'\bhostInstallVersion\s*=', "hostInstallVersion assignment"),
(r'(?i)\bconst\s+\w*hostinstall\w*version\b', "a renamed host-install version const"),
(r'\bScriptVersion\s+string\b', "ScriptVersion struct field"),
(r'\bScriptVersion\s*:', "ScriptVersion struct assignment"),
(r'\{\{\s*\.ScriptVersion\s*\}\}', "{{.ScriptVersion}} template action"),
]
HUB_DIR = os.path.join(ROOT, "hub")
if not os.path.isdir(HUB_DIR):
fail("hub/ not found at %s — cannot assert the absence of a host-install version literal" % HUB_DIR)
else:
if not os.path.exists(HUB_CONFIGS):
fail("hub/internal/web/configs.go not found — cannot cross-check the Setup-tab version")
scanned, hits = 0, 0
for dirpath, dirs, files in os.walk(HUB_DIR):
dirs[:] = [d for d in dirs if d not in (".git", "vendor", "node_modules")]
for fn in files:
if not (fn.endswith(".go") or fn.endswith(".html")):
continue
fp = os.path.join(dirpath, fn)
scanned += 1
with io.open(fp, "r", encoding="utf-8") as f:
for lineno, line in enumerate(f, 1):
for pat, what in BANNED:
if re.search(pat, line):
hits += 1
fail("%s:%d carries %s — the hub must render NO host-install version "
"(R-94: the served script is fetched at run time, so no build-time "
"value can be true). Single source: scripts/felhom-host-install.sh "
"SCRIPT_VERSION. Line: %s"
% (os.path.relpath(fp, ROOT), lineno, what, line.strip()[:120]))
if not hits:
ok("hub carries no host-install version literal (%d .go/.html files scanned, %d shapes checked)"
% (scanned, len(BANNED)))
# ── 2. age package (F-10) ───────────────────────────────────────────────────────
# must match the REAL install invocation, not the log_dry echo (red-proof-hardened twice:
+128
View File
@@ -0,0 +1,128 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""repo_gates.py — THE entry point for this repo's gates. Run from the repo root:
python3 scripts/repo_gates.py # every gate
python3 scripts/repo_gates.py --fast # only gates that touch no network and no container
# runtime (what .githooks/pre-push runs)
Gates, in order (all must pass; **non-zero exit on any failure**):
1. site website HTML: BOM, emoji, nav/footer, analytics, CDN, tokens, cache-busting
2. hostinstall felhom-host-install.sh's five drill-swept invariants (+ R-94's absent-version)
3. hub-confirm no native confirm()/prompt() in hub templates
4. manifest-bearer no bearer-shaped literal anywhere in manifests/
5. reuse-refs every path cited by this repo's REUSE.md still resolves
WHY THIS FILE EXISTS (2026-08-02, closing R-29 leg (a) and half of leg (b)).
A census of all thirteen gate scripts across the four felhom repos found one clean correlation:
**every check a CLAUDE.md tells a person to run was passing, and two of the four nobody is told
to run were failing** one since 14 July. Neither failure was harmful in effect, which was
checked line by line; nothing would have said so if they had been. The fix is not more gates, it
is one place to run them from. `app-catalog-felhom.eu/scripts/catalog_gates.py` is the canonical
shape (R-161) and this copies it deliberately rather than inventing a second one.
`site_gates.py` is a GATE eight assertions in one file and is NOT the model for this file. A
runner that invokes separate gates is the shape that survives; copying site_gates would just add
a ninth monolith.
FAIL-CLOSED. A gate script that is missing is a FAILURE, never a skip, and the exact path tried
is printed. A runner that quietly drops a gate is the inert-seam failure this project has shipped
four times.
EXIT CODES. Each gate returns 0 clean / 1 convicted / 2 inconclusive. This runner exits non-zero
if any gate is non-zero, and reports 2 distinctly as INCONCLUSIVE an undetermined result is
never a pass, but it is not a conviction either, and the operator needs to know which they have.
"""
import os
import subprocess
import sys
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
SCRIPTS = os.path.join(ROOT, "scripts")
# (label, absolute script path, args, fast)
GATES = [
("site", os.path.join(SCRIPTS, "site_gates.py"), [], True),
("hostinstall", os.path.join(SCRIPTS, "hostinstall_gates.py"), [], True),
("hub-confirm", os.path.join(SCRIPTS, "hub_confirm_gate.py"), [], True),
("manifest-bearer", os.path.join(SCRIPTS, "manifest_bearer_gate.py"), [], True),
("reuse-refs", os.path.join(SCRIPTS, "reuse_refs_check.py"), [ROOT], True),
]
VERDICT = {0: "OK", 1: "FAILED", 2: "INCONCLUSIVE"}
def hooks_armed_note(root):
"""Print a WARNING (never a failure) when this clone's pre-push hook is not switched on.
core.hooksPath is local config and a clone does not carry it, so an unarmed clone is silent
by construction this is the only place it becomes visible.
"""
try:
val = subprocess.check_output(["git", "config", "--get", "core.hooksPath"],
cwd=root, stderr=subprocess.DEVNULL).decode().strip()
except Exception:
val = ""
norm = val.replace("\\", "/").rstrip("/")
if norm == ".githooks" or norm.endswith("/.githooks"):
return
print("WARNING: this clone is UNARMED — core.hooksPath is %s, so the pre-push hook will not\n"
" run here. Switch it on once with: git config core.hooksPath .githooks"
% (("'" + val + "'") if val else "unset"))
def run_gate(label, path, args):
if not os.path.exists(path):
print("\nFAIL: gate '%s' is MISSING — tried %s" % (label, path))
print(" A missing gate is a failure, never a skip (fail-closed).")
return 1
print("\n" + "=" * 78)
print("== gate: %s (%s%s)" % (label, os.path.basename(path),
(" " + " ".join(args)) if args else ""))
print("=" * 78, flush=True)
# stream the gate's own output rather than capturing it — its diagnostics are the point,
# and a runner that swallows them makes a conviction unreadable.
return subprocess.call([sys.executable, path] + args, cwd=ROOT)
def main(argv):
fast = "--fast" in argv
unknown = [a for a in argv if a != "--fast"]
if unknown:
print("unknown argument(s): %s" % " ".join(unknown))
print("usage: python3 scripts/repo_gates.py [--fast]")
return 2
selected = [g for g in GATES if g[3] or not fast]
skipped = [g[0] for g in GATES if not (g[3] or not fast)]
print("repo_gates (felhom.eu) — %d gate(s)%s" % (len(selected), " [--fast]" if fast else ""))
if skipped:
print(" --fast SKIPPED (deliberate periodic runs, never in a hook): %s" % ", ".join(skipped))
hooks_armed_note(ROOT)
results = [(label, run_gate(label, path, args)) for label, path, args, _f in selected]
print("\n" + "=" * 78)
print("== summary")
print("=" * 78)
worst = 0
for label, rc in results:
print(" %-18s %-13s (exit %d)" % (label, VERDICT.get(rc, "ERROR"), rc))
if rc != 0:
worst = 1 if rc == 1 or worst == 1 else 2
if worst == 0:
print("\nall felhom.eu gates OK")
return 0
convicted = [l for l, rc in results if rc == 1]
undecided = [l for l, rc in results if rc not in (0, 1)]
if convicted:
print("\nCONVICTED: %s" % ", ".join(convicted))
if undecided:
print("UNDETERMINED (never a pass): %s" % ", ".join(undecided))
return worst
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+166 -20
View File
@@ -1,21 +1,142 @@
# -*- coding: utf-8 -*-
"""REUSE.md refs check — staleness defense for the per-repo reuse maps.
Usage: python scripts/reuse_refs_check.py <repo-root> [<repo-root> ...]
Usage: python3 scripts/reuse_refs_check.py <repo-root> [<repo-root> ...]
For each repo root given, parses <root>/REUSE.md, extracts every cited file path
(*.go, *.py, *.html, *.css, *.yml, *.yaml, *.sh) and verifies the file exists in
the tree. Only slash-containing (repo-relative) tokens are checked bare filenames
are conventions, not citations. Prints offending lines; exits non-zero if any cited
path is missing. Symbols are NOT checked here those are spot-verified by the
reviewer at file:line.
(*.go, *.py, *.html, *.css, *.yml, *.yaml, *.sh) and verifies the file EXISTS somewhere it
could honestly be. Only slash-containing tokens are checked bare filenames are conventions,
not citations. Symbols are NOT checked here; those are spot-verified by the reviewer at
file:line. Exits non-zero if any cited path resolves nowhere.
RESOLUTION ORDER (2026-08-02 operator ruling; first hit wins, and every non-exact hit is
PRINTED so a weakening of the check is visible rather than silent):
1. exact <root>/<token> exists no note
2. suffix exactly one indexed file under <root> ends with /<token>
3. ambiguous more than one does still OK: the citation is real, the shorthand is
imprecise. All matches are printed and marked AMBIGUOUS.
4. cross-repo resolved in an immediate SIBLING repo (a sibling dir containing .git), either
as-is or with the sibling's own name stripped from the front of the token
5. FAIL nowhere prints the file, line, token, and EVERY resolution attempted
WHY THIS SHAPE. Before this, the checker demanded repo-relative paths and was RED on all four
repos: 13 findings, and a hand audit of all 13 on 2026-08-02 found **zero** genuine drift.
Twelve were package shorthand whose file sits one or two directories deeper
(`appbackup/userdata.go` `controller/internal/appbackup/userdata.go`); one,
`wgsync/reconciler.go`, is cited by the controller's REUSE.md and lives in the HUB. REUSE.md
cites by package shorthand and across repos on purpose that convention is the useful one, and
the tool was what was wrong. Rejected alternatives, recorded so they are not revisited:
rewriting all four REUSE.md files to full paths (makes the docs worse to serve the tool), and
deleting the checker (REUSE drift across four repos is a live risk).
THE POSITIVE OBSERVABLE. Every root prints a per-rule tally. "0 failures" alone cannot tell a
working checker from a blind one a run that suddenly resolves everything by SUFFIX is telling
you something, and the counts are where you see it. The kill condition is pinned by
scripts/test_reuse_refs_check.py: a citation that exists nowhere still FAILS.
"""
import io, os, re, sys
# path-looking tokens ending in a checked extension; globs (*) are conventions, not refs
PATH_RE = re.compile(r'[A-Za-z0-9_][A-Za-z0-9_./\-]*/[A-Za-z0-9_./\-]*\.(?:go|py|html|css|yml|yaml|sh)\b')
# An EVIDENCE COPY of a file is not the file — never let an audit or a test-findings tree satisfy
# a citation. `.git`/`vendor`/`node_modules` are excluded as noise.
EXCLUDE_NAMES = {".git", "node_modules", "vendor", "audits"}
EXCLUDE_RELPATHS = {"documentation/tests"}
fails = 0
_index_cache = {}
class RepoIndex(object):
"""One walk per repo root, reused across every token and every sibling lookup."""
def __init__(self, root):
self.root = root
self.name = os.path.basename(root)
self.files = set() # posix-style relpaths
self.by_base = {} # basename -> [relpath, ...]
for dirpath, dirs, filenames in os.walk(root):
rel = os.path.relpath(dirpath, root).replace(os.sep, "/")
if rel == ".":
rel = ""
dirs[:] = [d for d in dirs
if d not in EXCLUDE_NAMES
and ((rel + "/" + d).lstrip("/") not in EXCLUDE_RELPATHS)]
for fn in filenames:
p = (rel + "/" + fn).lstrip("/")
self.files.add(p)
self.by_base.setdefault(fn, []).append(p)
def exact(self, token):
return token in self.files
def suffix_matches(self, token):
base = token.rsplit("/", 1)[-1]
return sorted(p for p in self.by_base.get(base, [])
if p != token and p.endswith("/" + token))
def index_for(root):
root = os.path.abspath(root)
if root not in _index_cache:
_index_cache[root] = RepoIndex(root)
return _index_cache[root]
def find_siblings(root):
"""Immediate sibling dirs of <root> that are themselves git working trees. One level only.
Returns (list_of_paths, error_message_or_None). A sibling repo that is simply absent is NEVER
a failure a clone in isolation must still be able to check itself.
"""
parent = os.path.dirname(os.path.abspath(root))
try:
entries = sorted(os.listdir(parent))
except OSError as e:
return [], "parent %s not readable (%s) — siblings were NOT searched" % (parent, e)
sibs = []
for e in entries:
p = os.path.join(parent, e)
if os.path.abspath(p) == os.path.abspath(root):
continue
if os.path.isdir(p) and os.path.exists(os.path.join(p, ".git")):
sibs.append(p)
return sibs, None
def resolve(token, idx, siblings):
"""(rule, note, tried) — rule is one of exact/suffix/ambiguous/cross-repo/None."""
tried = ["repo-relative %s/%s" % (idx.name, token)]
if idx.exact(token):
return "exact", "", tried
tried.append("suffix search over %d indexed files in %s" % (len(idx.files), idx.name))
m = idx.suffix_matches(token)
if len(m) == 1:
return "suffix", "resolved by suffix → %s" % m[0], tried
if len(m) > 1:
return "ambiguous", "AMBIGUOUS — %d matches: %s" % (len(m), ", ".join(m)), tried
for sib in siblings:
sidx = index_for(sib)
# a token may carry the sibling's own repo name on the front (app-catalog's REUSE.md
# cites `felhom.eu/scripts/site_gates.py` that way) — try both forms.
cands = [token]
if token.startswith(sidx.name + "/"):
cands.append(token[len(sidx.name) + 1:])
for cand in cands:
tried.append("sibling %s: %s" % (sidx.name, cand))
if sidx.exact(cand):
return "cross-repo", "cross-repo → %s/%s" % (sidx.name, cand), tried
sm = sidx.suffix_matches(cand)
if len(sm) == 1:
return "cross-repo", "cross-repo (suffix) → %s/%s" % (sidx.name, sm[0]), tried
if len(sm) > 1:
return "cross-repo", "cross-repo AMBIGUOUS in %s%d matches: %s" % (
sidx.name, len(sm), ", ".join(sm)), tried
return None, "", tried
def check_repo(root):
@@ -27,7 +148,14 @@ def check_repo(root):
print("FAIL [%s]: no REUSE.md at %s" % (name, reuse))
fails += 1
return
seen, missing = set(), 0
idx = index_for(root)
siblings, sib_err = find_siblings(root)
if sib_err:
# say so and continue — do NOT silently pretend siblings were searched
print("NOTE [%s]: %s" % (name, sib_err))
seen = set()
tally = {"exact": 0, "suffix": 0, "ambiguous": 0, "cross-repo": 0, "failed": 0}
with io.open(reuse, encoding="utf-8") as f:
for lineno, line in enumerate(f, 1):
for m in PATH_RE.finditer(line):
@@ -37,18 +165,36 @@ def check_repo(root):
if p in seen:
continue
seen.add(p)
if not os.path.isfile(os.path.join(root, p)):
print("FAIL [%s] line %d: cited path missing: %s" % (name, lineno, p))
missing += 1
if missing:
fails += missing
else:
print("OK [%s]: %d cited paths, all exist" % (name, len(seen)))
rule, note, tried = resolve(p, idx, siblings)
if rule is None:
tally["failed"] += 1
print("FAIL [%s] line %d: cited path resolves NOWHERE: %s" % (name, lineno, p))
for t in tried:
print(" tried: %s" % t)
if not siblings and not sib_err:
print(" tried: no sibling git repos found beside %s" % name)
else:
tally[rule] += 1
if note:
print("note [%s] line %d: %s (%s)" % (name, lineno, p, note))
print("%s [%s]: %d cited paths — exact %d, suffix %d, ambiguous %d, cross-repo %d, FAILED %d "
"(siblings searched: %s)" % (
"FAIL" if tally["failed"] else "OK ", name, len(seen),
tally["exact"], tally["suffix"], tally["ambiguous"], tally["cross-repo"],
tally["failed"],
", ".join(os.path.basename(s) for s in siblings) or "none"))
fails += tally["failed"]
if len(sys.argv) < 2:
print(__doc__)
sys.exit(2)
for r in sys.argv[1:]:
check_repo(r)
sys.exit(1 if fails else 0)
def main(argv):
if len(argv) < 1:
print(__doc__)
return 2
for r in argv:
check_repo(r)
return 1 if fails else 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+58
View File
@@ -0,0 +1,58 @@
# -*- coding: utf-8 -*-
"""Seam test for scripts/repo_gates.py.
Run: python3 scripts/test_repo_gates.py
WHY THIS EXISTS. An entry point is a seam by definition: a runner that LISTS a gate but never
executes it is inert and fully green, and this project has shipped an inert seam four times. So
the assertion is on each member gate's OWN distinctive stdout — never on the runner's summary
line, which the runner can print without ever calling anything plus the exit code, which is a
runner's actual effect.
Red-proofed 2026-08-02: replacing run_gate's body with `return 0` (the inert runner) turns
test_every_member_gate_actually_ran red while the summary still prints "all felhom.eu gates OK".
"""
import os
import subprocess
import sys
import unittest
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
ENTRY = os.path.join(ROOT, "scripts", "repo_gates.py")
# (label, a substring only THAT gate can print)
FINGERPRINTS = [
("site", "site gates OK"),
("hostinstall", "hostinstall gates: ALL PASS"),
("hub-confirm", "hub confirm gate"),
("manifest-bearer", "manifest bearer gate"),
("reuse-refs", "cited paths — exact"),
]
class RepoGatesTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
p = subprocess.run([sys.executable, ENTRY, "--fast"], cwd=ROOT,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
cls.rc = p.returncode
cls.out = p.stdout.decode("utf-8", "replace")
def test_exit_code_is_zero(self):
self.assertEqual(self.rc, 0, self.out)
def test_every_member_gate_actually_ran(self):
for label, fingerprint in FINGERPRINTS:
self.assertIn(fingerprint, self.out,
"gate %r is listed but its own output never appeared — an inert runner "
"prints the summary without calling anything:\n%s" % (label, self.out))
def test_unknown_argument_is_rejected(self):
p = subprocess.run([sys.executable, ENTRY, "--nope"], cwd=ROOT,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
self.assertEqual(p.returncode, 2, p.stdout.decode("utf-8", "replace"))
if __name__ == "__main__":
unittest.main(verbosity=2)
+191
View File
@@ -0,0 +1,191 @@
# -*- coding: utf-8 -*-
"""Fixture tests for scripts/reuse_refs_check.py — one per row of its resolution table, plus the
kill condition.
Run: python3 scripts/test_reuse_refs_check.py
THE ONE THAT MATTERS is test_absent_path_fails (Scenario E). The 2026-08-02 change taught the
checker to resolve suffixes and sibling repos, which turned 13 findings green in one step. A
checker made green by being made BLIND is a failure this project has shipped before, so the
ability to still fail is pinned here, and the assertion is on the EXIT CODE the effect not on
the summary text, which the checker can print without having decided anything.
"""
import importlib.util
import io
import os
import shutil
import sys
import tempfile
import unittest
SCRIPT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "reuse_refs_check.py")
def load_checker():
"""Fresh module per test — the checker keeps a global failure count and an index cache."""
spec = importlib.util.spec_from_file_location("reuse_refs_check_under_test", SCRIPT)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
def write(path, text=""):
d = os.path.dirname(path)
if d and not os.path.isdir(d):
os.makedirs(d)
with io.open(path, "w", encoding="utf-8") as f:
f.write(text)
class ReuseRefsCheckTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp(prefix="reuse-refs-")
self.root = os.path.join(self.tmp, "myrepo")
write(os.path.join(self.root, ".git"), "gitdir: elsewhere\n")
def tearDown(self):
shutil.rmtree(self.tmp, ignore_errors=True)
# ── harness ──────────────────────────────────────────────────────────────────
def run_check(self, *roots):
"""Returns (exit_code, stdout). Exit code is the assertion that counts."""
mod = load_checker()
buf = io.StringIO()
real = sys.stdout
sys.stdout = buf
try:
rc = mod.main(list(roots) or [self.root])
finally:
sys.stdout = real
return rc, buf.getvalue()
def reuse(self, body):
write(os.path.join(self.root, "REUSE.md"), body)
def sibling(self, name):
p = os.path.join(self.tmp, name)
write(os.path.join(p, ".git"), "gitdir: elsewhere\n")
return p
# ── row 1: exact ─────────────────────────────────────────────────────────────
def test_exact_match_is_silent_and_passes(self):
write(os.path.join(self.root, "a", "b.go"), "package a\n")
self.reuse("see `a/b.go` for the thing\n")
rc, out = self.run_check()
self.assertEqual(rc, 0, out)
self.assertIn("exact 1", out)
self.assertNotIn("note [", out) # an exact hit prints no note
# ── row 2: suffix ────────────────────────────────────────────────────────────
def test_package_shorthand_resolves_by_suffix(self):
write(os.path.join(self.root, "controller", "internal", "pkg", "x.go"), "package pkg\n")
self.reuse("see `pkg/x.go`\n")
rc, out = self.run_check()
self.assertEqual(rc, 0, out)
self.assertIn("resolved by suffix", out)
self.assertIn("controller/internal/pkg/x.go", out)
self.assertIn("suffix 1", out)
# ── row 3: ambiguous — real citation, imprecise shorthand; NOT a failure ─────
def test_two_suffix_matches_are_ambiguous_not_fatal(self):
write(os.path.join(self.root, "one", "pkg", "x.go"), "package pkg\n")
write(os.path.join(self.root, "two", "pkg", "x.go"), "package pkg\n")
self.reuse("see `pkg/x.go`\n")
rc, out = self.run_check()
self.assertEqual(rc, 0, out)
self.assertIn("AMBIGUOUS", out)
self.assertIn("one/pkg/x.go", out)
self.assertIn("two/pkg/x.go", out)
self.assertIn("ambiguous 1", out)
# ── row 4: cross-repo, by suffix in a sibling ────────────────────────────────
def test_sibling_repo_resolution(self):
sib = self.sibling("otherrepo")
write(os.path.join(sib, "hub", "internal", "wgsync", "reconciler.go"), "package wgsync\n")
self.reuse("see `wgsync/reconciler.go`\n")
rc, out = self.run_check()
self.assertEqual(rc, 0, out)
self.assertIn("cross-repo", out)
self.assertIn("otherrepo", out)
self.assertIn("cross-repo 1", out)
# ── row 4b: cross-repo where the token CARRIES the sibling's repo name ───────
def test_sibling_repo_name_prefixed_token(self):
sib = self.sibling("felhom.eu")
write(os.path.join(sib, "scripts", "site_gates.py"), "# gate\n")
self.reuse("run `felhom.eu/scripts/site_gates.py`\n")
rc, out = self.run_check()
self.assertEqual(rc, 0, out)
self.assertIn("cross-repo", out)
def test_non_git_sibling_is_not_searched(self):
plain = os.path.join(self.tmp, "notarepo") # no .git — not a repo, must not resolve
write(os.path.join(plain, "pkg", "x.go"), "package pkg\n")
self.reuse("see `pkg/x.go`\n")
rc, out = self.run_check()
self.assertNotEqual(rc, 0, out)
# ── row 5: THE KILL CONDITION (Scenario E) ──────────────────────────────────
def test_absent_path_fails(self):
write(os.path.join(self.root, "internal", "present.go"), "package internal\n")
self.reuse("line one\nsee `internal/definitely_absent_xyz.go`\n")
rc, out = self.run_check()
self.assertNotEqual(rc, 0, "a citation that exists NOWHERE must fail:\n" + out)
self.assertIn("definitely_absent_xyz.go", out)
self.assertIn("line 2", out) # names the line
self.assertIn("FAILED 1", out)
def test_failure_lists_every_resolution_attempted(self):
"""CLAUDE.md standing rule: a 'not found' claim must name what was tried."""
self.sibling("otherrepo")
self.reuse("see `internal/definitely_absent_xyz.go`\n")
rc, out = self.run_check()
self.assertNotEqual(rc, 0, out)
self.assertIn("tried: repo-relative myrepo/internal/definitely_absent_xyz.go", out)
self.assertIn("tried: suffix search over", out)
self.assertIn("tried: sibling otherrepo", out)
# ── evidence trees are not the file ─────────────────────────────────────────
def test_evidence_copy_does_not_satisfy_a_citation(self):
write(os.path.join(self.root, "documentation", "audits", "pkg", "x.go"), "package pkg\n")
write(os.path.join(self.root, "documentation", "tests", "pkg", "y.go"), "package pkg\n")
self.reuse("see `pkg/x.go` and `pkg/y.go`\n")
rc, out = self.run_check()
self.assertNotEqual(rc, 0, "an audits/ or documentation/tests/ copy must NOT resolve:\n" + out)
self.assertIn("FAILED 2", out)
# ── a clone in isolation must still check itself ────────────────────────────
def test_no_siblings_is_not_a_failure(self):
write(os.path.join(self.root, "a", "b.go"), "package a\n")
self.reuse("see `a/b.go`\n")
rc, out = self.run_check()
self.assertEqual(rc, 0, out)
self.assertIn("siblings searched: none", out)
def test_missing_reuse_md_fails(self):
rc, out = self.run_check()
self.assertNotEqual(rc, 0, out)
self.assertIn("no REUSE.md", out)
# ── globs stay conventions, not refs ────────────────────────────────────────
def test_glob_is_not_a_citation(self):
self.reuse("the gates are `scripts/*.py`\n")
rc, out = self.run_check()
self.assertEqual(rc, 0, out)
self.assertIn("0 cited paths", out)
# ── no args → usage, exit 2 ─────────────────────────────────────────────────
def test_no_args_is_usage_exit_2(self):
mod = load_checker()
buf, real = io.StringIO(), sys.stdout
sys.stdout = buf
try:
rc = mod.main([])
finally:
sys.stdout = real
self.assertEqual(rc, 2)
if __name__ == "__main__":
unittest.main(verbosity=2)