release path publishes, and an unreleasable version fails CI (R-115, R-183)
gates / gates (push) Failing after 7s

NO VERSION BUMP and nothing built: no Go code changed. The agent stays v0.120.0.

scripts/release-agent.sh — THE way to release. build -> tag -> publish -> verify
by INDEPENDENT download. Publishing was a separate remembered step and was
forgotten three times in five days (R-111's 17 stranded releases, 0.114.0, and
0.120.0 — deployed to both demo hosts and undownloadable, so a documented-path
reinstall would have silently downgraded them WHILE REPORTING SUCCESS). R-111's
own closing line named this leg and closed SHIPPED without it; it recurred the
same afternoon, which is the evidence that a note is not a mechanism.

It tags because felhom-host-install.sh now fetches the sixteen agent config
files from raw/tag/v<version>/ (R-183): a released version with no tag 404s a
box mid-install, as root, on a virgin machine. It verifies by downloading what
it just published and comparing the sha to what it built — the publish step's
own success is a report on its own write; a fetch returning the right bytes is
a different claim. It refuses a dirty/unpushed tree and refuses to re-release an
existing version. It does NOT vouch: that points machines at a version and stays
the operator's act.

scripts/check-published-versions.py — the gate. Every v<semver> tag must have a
downloadable package AND a tag tree serving the agent's configs. Registered as
NOT --fast (needs network; a push must not fail because Gitea blinked), and the
CI workflow now runs the FULL gate set instead of --fast — otherwise the gate
would have been registered and never run, the built-but-never-wired failure this
project has shipped four times.

The invariant is not the one specified, and the reason was measured, not assumed:
the hub artifact manifest is 401 without a per-customer passphrase and Gitea's
package LISTING api is 401 without a token, while the package DOWNLOAD url and
the git TAGS api are anonymous. So CI cannot ask "what is vouched" without an
operator credential — whose addition is the operator's call. The tag-based
invariant needs none and catches all three recorded instances. What it does not
catch (the hub vouching a version never released at all) is filed as R-184.
This commit is contained in:
2026-08-03 12:34:20 +02:00
parent 9dfd89cb94
commit dd2d1feb6e
6 changed files with 363 additions and 3 deletions
+159
View File
@@ -0,0 +1,159 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""check-published-versions.py — a released agent version must be INSTALLABLE, not merely built.
R-115. A box installs the agent from a Gitea generic package the hub vouches, never from git, and
since R-110/R-183 it also fetches the agent's sixteen config files from `raw/tag/v<version>/`.
Nothing in the build, deploy or session-end path checked that either existed, so "deployed" and
"installable" were independent states that drifted silently. **Three instances in five days:**
* R-111 (2026-07-29) 17 releases v0.97.0-v0.113.0 built and never published — a new customer
would have installed without the whole R-82 tiered-backup arc, F-CRIT-2 and F-REBOOT.
* 0.114.0 (same afternoon) built, deployed to felhom-pve, never published.
* 0.120.0 (2026-08-03) built, committed, deployed to BOTH demo hosts, never published. A
documented-path reinstall would have silently DOWNGRADED both boxes to the pre-merge
agent — and would have *succeeded* while doing it.
THE INVARIANT, AND WHY IT IS THIS ONE.
For every `v<semver>` git tag in this repo: the matching generic package must be DOWNLOADABLE,
and the tag must serve the agent's configs.
The task's §8.4 asked for a different one — *"the version the hub tells machines to install must be
downloadable"* — and that is the better invariant in principle. **It is not implementable from CI,
and that was measured rather than assumed:** the hub's artifact manifest
(`GET /api/v1/artifacts/<customer>`) answers **401** without a per-customer retrieval passphrase,
and the Gitea package LISTING api answers **401** without a token, while the package DOWNLOAD url
and the git TAGS api are both anonymous. So a credential-free gate can ask *"is this version
installable"* but not *"which version is vouched"*. Putting an operator credential into CI to close
that gap is the operator's call, not a gate author's — it is recorded as a limitation below and as a
backlog row rather than quietly assumed away.
**What this invariant does catch: all three instances above.** `release-agent.sh` creates the tag and
publishes in one act, so a release whose publish was skipped, failed, or was forgotten leaves a tag
with no package — which is exactly what this refuses. It needs no version floor: tags begin at
v0.120.0, which is published.
**What it does NOT catch, stated plainly:** the hub vouching a version that was never released at
all (no tag, no package). Nothing here can see that; it belongs at vouch time, in the hub. → R-184.
FAIL-CLOSED. A network error, an unparseable response or an unreachable Gitea is exit **2
INCONCLUSIVE**, naming every URL tried — never a pass. "Cannot determine" is not "fine": that is the
standing rule this project earned twice, and a gate that green-lights on its own blindness is worse
than no gate, because it looks like coverage.
Pure python3 + urllib, NO curl and no third-party module: the CI runner is a host-mode container
carrying python3 and git and nothing else, and an earlier workflow step died on
`curl: command not found`.
python3 scripts/check-published-versions.py
Exit: 0 every tag installable · 1 at least one is not · 2 could not be determined.
Env: GITEA_BASE overrides the Gitea root (CI sets the in-cluster service URL).
"""
import json
import os
import re
import sys
import urllib.error
import urllib.request
GITEA_BASE = os.environ.get("GITEA_BASE", "https://gitea.dooplex.hu").rstrip("/")
OWNER = "admin"
REPO = "felhom-agent"
PKG = "felhom-agent"
TIMEOUT = 25
# One config the installer fetches. Its presence proves the TAG's tree carries the configs the
# sixteen `fetch_raw` calls will ask for — a tag that exists but predates them would 404 a box
# mid-install, on a virgin machine, as root.
PROBE_CONFIG = "configs/felhom-agent.service"
TAG_RE = re.compile(r"^v(\d+\.\d+\.\d+)$")
tried = []
def _get(url, want_body=False):
"""GET a URL. Returns (status, body_or_None). Network failure raises."""
tried.append(url)
req = urllib.request.Request(url, method="GET")
try:
with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
body = r.read() if want_body else None
return r.status, body
except urllib.error.HTTPError as e:
return e.code, None
def inconclusive(msg):
print("INCONCLUSIVE:", msg)
print(" URLs tried (a 'no access' claim must name its attempts):")
for u in tried:
print(" ", u)
sys.exit(2)
def main():
print("check-published-versions — every released agent version must be INSTALLABLE")
print(" gitea:", GITEA_BASE)
tags_url = "%s/api/v1/repos/%s/%s/tags?limit=200" % (GITEA_BASE, OWNER, REPO)
try:
status, body = _get(tags_url, want_body=True)
except Exception as e:
inconclusive("cannot reach Gitea to list tags: %s" % e)
if status != 200 or not body:
inconclusive("tags api returned HTTP %s — cannot enumerate releases" % status)
try:
tags = [t["name"] for t in json.loads(body.decode("utf-8"))]
except Exception as e:
inconclusive("tags api response is not the expected JSON: %s" % e)
versions = sorted(m.group(1) for m in (TAG_RE.match(t) for t in tags) if m)
if not versions:
# Not a failure: a repo legitimately has no version tags before its first release. Say so
# loudly rather than reporting a vacuous pass — an empty check that prints OK is how an
# unexamined thing becomes a documented-clean one.
print(" no v<semver> tags in this repo yet — nothing to check, and nothing proven")
print("\ncheck-published-versions: NOTHING TO CHECK")
return 0
print(" %d released version(s) to verify: %s" % (len(versions), ", ".join(versions)))
bad = []
for v in versions:
pkg_url = "%s/api/packages/%s/generic/%s/%s/%s" % (GITEA_BASE, OWNER, PKG, v, PKG)
raw_url = "%s/%s/%s/raw/tag/v%s/%s" % (GITEA_BASE, OWNER, REPO, v, PROBE_CONFIG)
try:
pkg_status, _ = _get(pkg_url)
raw_status, _ = _get(raw_url)
except Exception as e:
inconclusive("network failure while checking v%s: %s" % (v, e))
problems = []
if pkg_status != 200:
problems.append("binary NOT downloadable (HTTP %s at %s)" % (pkg_status, pkg_url))
if raw_status != 200:
problems.append("tag does not serve %s (HTTP %s) — a box would 404 mid-install"
% (PROBE_CONFIG, raw_status))
if problems:
bad.append((v, problems))
print(" FAIL v%s:" % v)
for p in problems:
print(" -", p)
else:
print(" ok v%s: binary downloadable + tag serves its configs" % v)
print()
if bad:
print("check-published-versions: %d RELEASED VERSION(S) NOT INSTALLABLE" % len(bad))
print(" A tagged version with no package is a release that was BUILT and never PUBLISHED —")
print(" the R-115 defect, three times in five days. Publish it with:")
print(" scripts/release-agent.sh <version>")
return 1
print("check-published-versions: ALL RELEASED VERSIONS INSTALLABLE")
return 0
if __name__ == "__main__":
sys.exit(main())