release path publishes, and an unreleasable version fails CI (R-115, R-183)
gates / gates (push) Failing after 7s
gates / gates (push) Failing after 7s
NO VERSION BUMP and nothing built: no Go code changed. The agent stays v0.120.0. scripts/release-agent.sh — THE way to release. build -> tag -> publish -> verify by INDEPENDENT download. Publishing was a separate remembered step and was forgotten three times in five days (R-111's 17 stranded releases, 0.114.0, and 0.120.0 — deployed to both demo hosts and undownloadable, so a documented-path reinstall would have silently downgraded them WHILE REPORTING SUCCESS). R-111's own closing line named this leg and closed SHIPPED without it; it recurred the same afternoon, which is the evidence that a note is not a mechanism. It tags because felhom-host-install.sh now fetches the sixteen agent config files from raw/tag/v<version>/ (R-183): a released version with no tag 404s a box mid-install, as root, on a virgin machine. It verifies by downloading what it just published and comparing the sha to what it built — the publish step's own success is a report on its own write; a fetch returning the right bytes is a different claim. It refuses a dirty/unpushed tree and refuses to re-release an existing version. It does NOT vouch: that points machines at a version and stays the operator's act. scripts/check-published-versions.py — the gate. Every v<semver> tag must have a downloadable package AND a tag tree serving the agent's configs. Registered as NOT --fast (needs network; a push must not fail because Gitea blinked), and the CI workflow now runs the FULL gate set instead of --fast — otherwise the gate would have been registered and never run, the built-but-never-wired failure this project has shipped four times. The invariant is not the one specified, and the reason was measured, not assumed: the hub artifact manifest is 401 without a per-customer passphrase and Gitea's package LISTING api is 401 without a token, while the package DOWNLOAD url and the git TAGS api are anonymous. So CI cannot ask "what is vouched" without an operator credential — whose addition is the operator's call. The tag-based invariant needs none and catches all three recorded instances. What it does not catch (the hub vouching a version never released at all) is filed as R-184.
This commit is contained in:
@@ -0,0 +1,159 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""check-published-versions.py — a released agent version must be INSTALLABLE, not merely built.
|
||||
|
||||
R-115. A box installs the agent from a Gitea generic package the hub vouches, never from git, and
|
||||
since R-110/R-183 it also fetches the agent's sixteen config files from `raw/tag/v<version>/`.
|
||||
Nothing in the build, deploy or session-end path checked that either existed, so "deployed" and
|
||||
"installable" were independent states that drifted silently. **Three instances in five days:**
|
||||
|
||||
* R-111 (2026-07-29) 17 releases v0.97.0-v0.113.0 built and never published — a new customer
|
||||
would have installed without the whole R-82 tiered-backup arc, F-CRIT-2 and F-REBOOT.
|
||||
* 0.114.0 (same afternoon) built, deployed to felhom-pve, never published.
|
||||
* 0.120.0 (2026-08-03) built, committed, deployed to BOTH demo hosts, never published. A
|
||||
documented-path reinstall would have silently DOWNGRADED both boxes to the pre-merge
|
||||
agent — and would have *succeeded* while doing it.
|
||||
|
||||
THE INVARIANT, AND WHY IT IS THIS ONE.
|
||||
|
||||
For every `v<semver>` git tag in this repo: the matching generic package must be DOWNLOADABLE,
|
||||
and the tag must serve the agent's configs.
|
||||
|
||||
The task's §8.4 asked for a different one — *"the version the hub tells machines to install must be
|
||||
downloadable"* — and that is the better invariant in principle. **It is not implementable from CI,
|
||||
and that was measured rather than assumed:** the hub's artifact manifest
|
||||
(`GET /api/v1/artifacts/<customer>`) answers **401** without a per-customer retrieval passphrase,
|
||||
and the Gitea package LISTING api answers **401** without a token, while the package DOWNLOAD url
|
||||
and the git TAGS api are both anonymous. So a credential-free gate can ask *"is this version
|
||||
installable"* but not *"which version is vouched"*. Putting an operator credential into CI to close
|
||||
that gap is the operator's call, not a gate author's — it is recorded as a limitation below and as a
|
||||
backlog row rather than quietly assumed away.
|
||||
|
||||
**What this invariant does catch: all three instances above.** `release-agent.sh` creates the tag and
|
||||
publishes in one act, so a release whose publish was skipped, failed, or was forgotten leaves a tag
|
||||
with no package — which is exactly what this refuses. It needs no version floor: tags begin at
|
||||
v0.120.0, which is published.
|
||||
|
||||
**What it does NOT catch, stated plainly:** the hub vouching a version that was never released at
|
||||
all (no tag, no package). Nothing here can see that; it belongs at vouch time, in the hub. → R-184.
|
||||
|
||||
FAIL-CLOSED. A network error, an unparseable response or an unreachable Gitea is exit **2
|
||||
INCONCLUSIVE**, naming every URL tried — never a pass. "Cannot determine" is not "fine": that is the
|
||||
standing rule this project earned twice, and a gate that green-lights on its own blindness is worse
|
||||
than no gate, because it looks like coverage.
|
||||
|
||||
Pure python3 + urllib, NO curl and no third-party module: the CI runner is a host-mode container
|
||||
carrying python3 and git and nothing else, and an earlier workflow step died on
|
||||
`curl: command not found`.
|
||||
|
||||
python3 scripts/check-published-versions.py
|
||||
|
||||
Exit: 0 every tag installable · 1 at least one is not · 2 could not be determined.
|
||||
Env: GITEA_BASE overrides the Gitea root (CI sets the in-cluster service URL).
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
GITEA_BASE = os.environ.get("GITEA_BASE", "https://gitea.dooplex.hu").rstrip("/")
|
||||
OWNER = "admin"
|
||||
REPO = "felhom-agent"
|
||||
PKG = "felhom-agent"
|
||||
TIMEOUT = 25
|
||||
|
||||
# One config the installer fetches. Its presence proves the TAG's tree carries the configs the
|
||||
# sixteen `fetch_raw` calls will ask for — a tag that exists but predates them would 404 a box
|
||||
# mid-install, on a virgin machine, as root.
|
||||
PROBE_CONFIG = "configs/felhom-agent.service"
|
||||
|
||||
TAG_RE = re.compile(r"^v(\d+\.\d+\.\d+)$")
|
||||
|
||||
tried = []
|
||||
|
||||
|
||||
def _get(url, want_body=False):
|
||||
"""GET a URL. Returns (status, body_or_None). Network failure raises."""
|
||||
tried.append(url)
|
||||
req = urllib.request.Request(url, method="GET")
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
|
||||
body = r.read() if want_body else None
|
||||
return r.status, body
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, None
|
||||
|
||||
|
||||
def inconclusive(msg):
|
||||
print("INCONCLUSIVE:", msg)
|
||||
print(" URLs tried (a 'no access' claim must name its attempts):")
|
||||
for u in tried:
|
||||
print(" ", u)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def main():
|
||||
print("check-published-versions — every released agent version must be INSTALLABLE")
|
||||
print(" gitea:", GITEA_BASE)
|
||||
|
||||
tags_url = "%s/api/v1/repos/%s/%s/tags?limit=200" % (GITEA_BASE, OWNER, REPO)
|
||||
try:
|
||||
status, body = _get(tags_url, want_body=True)
|
||||
except Exception as e:
|
||||
inconclusive("cannot reach Gitea to list tags: %s" % e)
|
||||
if status != 200 or not body:
|
||||
inconclusive("tags api returned HTTP %s — cannot enumerate releases" % status)
|
||||
try:
|
||||
tags = [t["name"] for t in json.loads(body.decode("utf-8"))]
|
||||
except Exception as e:
|
||||
inconclusive("tags api response is not the expected JSON: %s" % e)
|
||||
|
||||
versions = sorted(m.group(1) for m in (TAG_RE.match(t) for t in tags) if m)
|
||||
if not versions:
|
||||
# Not a failure: a repo legitimately has no version tags before its first release. Say so
|
||||
# loudly rather than reporting a vacuous pass — an empty check that prints OK is how an
|
||||
# unexamined thing becomes a documented-clean one.
|
||||
print(" no v<semver> tags in this repo yet — nothing to check, and nothing proven")
|
||||
print("\ncheck-published-versions: NOTHING TO CHECK")
|
||||
return 0
|
||||
print(" %d released version(s) to verify: %s" % (len(versions), ", ".join(versions)))
|
||||
|
||||
bad = []
|
||||
for v in versions:
|
||||
pkg_url = "%s/api/packages/%s/generic/%s/%s/%s" % (GITEA_BASE, OWNER, PKG, v, PKG)
|
||||
raw_url = "%s/%s/%s/raw/tag/v%s/%s" % (GITEA_BASE, OWNER, REPO, v, PROBE_CONFIG)
|
||||
try:
|
||||
pkg_status, _ = _get(pkg_url)
|
||||
raw_status, _ = _get(raw_url)
|
||||
except Exception as e:
|
||||
inconclusive("network failure while checking v%s: %s" % (v, e))
|
||||
|
||||
problems = []
|
||||
if pkg_status != 200:
|
||||
problems.append("binary NOT downloadable (HTTP %s at %s)" % (pkg_status, pkg_url))
|
||||
if raw_status != 200:
|
||||
problems.append("tag does not serve %s (HTTP %s) — a box would 404 mid-install"
|
||||
% (PROBE_CONFIG, raw_status))
|
||||
if problems:
|
||||
bad.append((v, problems))
|
||||
print(" FAIL v%s:" % v)
|
||||
for p in problems:
|
||||
print(" -", p)
|
||||
else:
|
||||
print(" ok v%s: binary downloadable + tag serves its configs" % v)
|
||||
|
||||
print()
|
||||
if bad:
|
||||
print("check-published-versions: %d RELEASED VERSION(S) NOT INSTALLABLE" % len(bad))
|
||||
print(" A tagged version with no package is a release that was BUILT and never PUBLISHED —")
|
||||
print(" the R-115 defect, three times in five days. Publish it with:")
|
||||
print(" scripts/release-agent.sh <version>")
|
||||
return 1
|
||||
print("check-published-versions: ALL RELEASED VERSIONS INSTALLABLE")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user