catalog: re-pin wanderer to the current upstream shape, retire plant-it, add the resolvability gate
wanderer: ghcr.io/flomp/wanderer:0.16.0 is a ghost - upstream split the app into web+db images, moved registry and renamed the org. Restructured to upstream's own v0.20.0 compose (3 services, new /data/plugins volume, second public hostname for PocketBase, meilisearch pinned DOWN to upstream's v1.36.0 per the R-42 ruling). plant-it: retired. The repo name was wrong (plant-it-server) but upstream has DELETED self-hosting; last server image is 2024-12-10 and it needs MySQL+Redis the template never had. Moved to retired/ rather than deleted - reversible. R-41 slice 1: check-image-resolvable.py. Encodes two traps - manifest inspect exits 0 while printing toomanyrequests, and the inverse, where the first sweep called 24 of 65 pins dead because Hub throttled it. Ambiguity is INCONCLUSIVE, never an accusation.
This commit is contained in:
@@ -0,0 +1,177 @@
|
||||
#!/usr/bin/env python3
|
||||
"""check-image-resolvable.py — catalog gate: every pinned image must still EXIST upstream.
|
||||
|
||||
The companion to `check-image-pins.py`, which is purely syntactic: it proves a template pins a
|
||||
concrete tag, never that the tag is still there. That gap is how `plant-it` and `wanderer` sat behind
|
||||
a working "Telepítés" button for months with images that did not resolve at all — the templates were
|
||||
perfectly well-formed and pointed at nothing (Campaign 7, §6.2). **Silent rot is the real risk**
|
||||
(ROADMAP R-41): an upstream rename, a repo split, or a pruned tag breaks a template without touching
|
||||
this repo, so nothing in a change-triggered gate would ever notice.
|
||||
|
||||
This resolves each unique `image:` pin against its registry with
|
||||
`docker manifest inspect <ref>` and exits non-zero listing everything that did not resolve.
|
||||
|
||||
python3 scripts/check-image-resolvable.py # whole catalog
|
||||
python3 scripts/check-image-resolvable.py wanderer … # only these app dirs
|
||||
|
||||
THE TRAP THIS SCRIPT IS BUILT AROUND: gate on EACH `docker manifest inspect`'s OWN exit code, one
|
||||
image at a time. Never pipe the run through anything that summarises (`| grep`, `| tee`, `&&` chains,
|
||||
a wrapping shell) and then read the exit code of THAT — you get the pipeline's status, which is the
|
||||
last element's, and unresolvable images sail straight through reporting success. This is the same
|
||||
class of defect as the `validate-answer` trap in the ISO tooling (exits 0 on failure, so the build
|
||||
gates on its OUTPUT text instead).
|
||||
|
||||
Requires network + a working `docker`. It is therefore a PERIODIC/manual gate, not a pre-commit one
|
||||
— run it at the start of every catalog campaign, and before any publish train that vouches the
|
||||
catalog. Unit tests inject `resolver` and never touch the network.
|
||||
"""
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
IMAGE_RE = re.compile(r"^\s*image:\s*[\"']?([^\s\"'#]+)") # same shape as check-image-pins.py
|
||||
|
||||
# A ref that must never resolve, for self-testing the resolver end of the gate. `.invalid` is
|
||||
# reserved by RFC 2606 and can never be a real registry.
|
||||
CANARY_REF = "felhom-nonexistent.invalid/no/such:image"
|
||||
|
||||
|
||||
def collect_images(root: Path, only: list[str] | None = None) -> dict[str, list[str]]:
|
||||
"""Map each unique image ref -> the ['app:line'] sites that pin it. Pure; no network."""
|
||||
sites: dict[str, list[str]] = {}
|
||||
for f in sorted(root.glob("templates/*/docker-compose.yml")):
|
||||
app = f.parent.name
|
||||
if only and app not in only:
|
||||
continue
|
||||
for lineno, line in enumerate(f.read_text(encoding="utf-8").splitlines(), 1):
|
||||
m = IMAGE_RE.match(line)
|
||||
if m:
|
||||
sites.setdefault(m.group(1), []).append(f"{app}:{lineno}")
|
||||
return sites
|
||||
|
||||
|
||||
OK, ABSENT, INCONCLUSIVE = "ok", "absent", "inconclusive"
|
||||
|
||||
# Substrings that mean the registry positively answered "that image is not here". ONLY these
|
||||
# justify failing the gate.
|
||||
ABSENT_MARKERS = (
|
||||
"manifest unknown", "not found", "no such manifest", "does not exist",
|
||||
"repository name not known", "unknown: unknown", "manifest_unknown",
|
||||
"name unknown", "no such host", "unsupported protocol scheme",
|
||||
)
|
||||
# Substrings that mean "we could not find out" — a throttle, an auth wall, a network fault. These
|
||||
# must NEVER be reported as a dead image.
|
||||
INCONCLUSIVE_MARKERS = (
|
||||
"toomanyrequests", "rate limit", "too many requests",
|
||||
"unauthorized", "authentication required", "denied",
|
||||
"timeout", "timed out", "temporary failure", "connection refused",
|
||||
"i/o timeout", "tls handshake", "service unavailable", "500 internal",
|
||||
)
|
||||
|
||||
|
||||
def classify(returncode: int, err: str) -> str:
|
||||
"""Turn one `docker manifest inspect` result into ok / absent / inconclusive.
|
||||
|
||||
TWO TRAPS, BOTH LIVE-OBSERVED, BOTH LOAD-BEARING:
|
||||
|
||||
1. `docker manifest inspect` prints `toomanyrequests: You have reached your unauthenticated
|
||||
pull rate limit` and **still exits 0**. Same shape as the ISO tooling's `validate-answer`.
|
||||
So a non-empty stderr is checked even on rc=0, or a throttled run reports a clean bill of
|
||||
health for images it never actually resolved.
|
||||
2. The inverse, which is what a naive gate does: treating that throttle as a failure. On
|
||||
2026-07-21 the first full sweep called 24 of 65 pins dead — including `postgres:16-alpine`
|
||||
and `redis:7-alpine` — purely because Docker Hub started throttling partway through. A gate
|
||||
that cries wolf gets ignored, and then it protects nothing.
|
||||
|
||||
Ambiguity therefore resolves to INCONCLUSIVE, never to a failure: this gate may only accuse an
|
||||
image when the registry positively said it is gone.
|
||||
"""
|
||||
low = err.lower()
|
||||
if any(m in low for m in INCONCLUSIVE_MARKERS):
|
||||
return INCONCLUSIVE
|
||||
if returncode == 0:
|
||||
# rc=0 WITH error text is trap 1 — do not trust it as success.
|
||||
return OK if not low.strip() else INCONCLUSIVE
|
||||
if any(m in low for m in ABSENT_MARKERS):
|
||||
return ABSENT
|
||||
return INCONCLUSIVE # an unrecognised failure is not evidence of absence
|
||||
|
||||
|
||||
def docker_resolver(ref: str) -> tuple[str, str]:
|
||||
"""(status, detail) from `docker manifest inspect ref`.
|
||||
|
||||
LOAD-BEARING: the decision comes from THIS call, for THIS one image — never from a piped or
|
||||
aggregated summary, whose exit status is the last element's.
|
||||
"""
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["docker", "manifest", "inspect", ref],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, timeout=120, text=True,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
return INCONCLUSIVE, "timed out after 120s"
|
||||
except OSError as e:
|
||||
return INCONCLUSIVE, f"could not run docker: {e}"
|
||||
err = (r.stderr or "").strip()
|
||||
return classify(r.returncode, err), err.splitlines()[0] if err else ""
|
||||
|
||||
|
||||
def check_images(sites: dict[str, list[str]], resolver) -> tuple[list[str], list[tuple[str, str]]]:
|
||||
"""Resolve every ref once. Returns (absent_refs, [(inconclusive_ref, why)]). No I/O of its own."""
|
||||
absent, inconclusive = [], []
|
||||
for ref in sorted(sites):
|
||||
status, detail = resolver(ref)
|
||||
if status == ABSENT:
|
||||
absent.append(ref)
|
||||
elif status != OK:
|
||||
inconclusive.append((ref, detail))
|
||||
return absent, inconclusive
|
||||
|
||||
|
||||
def check(root: Path, only: list[str] | None = None, resolver=docker_resolver) -> int:
|
||||
sites = collect_images(root, only)
|
||||
if not sites:
|
||||
print(f"ERROR: no images found under {root}/templates/", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
# Self-test the resolver before trusting a green result: if it says a ref that CANNOT exist
|
||||
# resolves, it is broken (or something is intercepting the registry) and a clean run would be a
|
||||
# false all-clear — the exact failure this gate exists to prevent.
|
||||
if resolver(CANARY_REF)[0] == OK:
|
||||
print(f"ERROR: resolver returned success for {CANARY_REF} — it is not trustworthy; "
|
||||
"refusing to report a result", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
print(f"resolving {len(sites)} unique image pin(s)…")
|
||||
absent, inconclusive = check_images(sites, resolver)
|
||||
|
||||
if absent:
|
||||
print("\nUNRESOLVABLE IMAGES (the registry says these are GONE):")
|
||||
for ref in absent:
|
||||
print(f" {ref}")
|
||||
for site in sites[ref]:
|
||||
print(f" pinned at templates/{site}")
|
||||
|
||||
if inconclusive:
|
||||
print("\nINCONCLUSIVE (could NOT be checked — this is not an accusation):")
|
||||
for ref, why in inconclusive:
|
||||
print(f" {ref} [{why}]")
|
||||
print("\n Docker Hub throttles unauthenticated manifest lookups, and a large sweep will hit")
|
||||
print(" the ceiling partway through. Re-run after `docker login`, or wait out the window —")
|
||||
print(" the result above is NOT evidence that these images are missing.")
|
||||
|
||||
if absent:
|
||||
print(f"\n{len(absent)} of {len(sites)} image pin(s) are GONE"
|
||||
f"{f'; {len(inconclusive)} could not be checked' if inconclusive else ''}.")
|
||||
return 1
|
||||
if inconclusive:
|
||||
print(f"\nINCOMPLETE: {len(sites) - len(inconclusive)} of {len(sites)} pins verified, "
|
||||
f"{len(inconclusive)} unchecked. No dead images among those checked.")
|
||||
return 2
|
||||
print(f"image-resolvability gate OK — {len(sites)} unique pins, all resolve")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(check(Path(__file__).resolve().parent.parent, only=sys.argv[1:] or None))
|
||||
@@ -0,0 +1,148 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fixture tests for check-image-resolvable.py. NO NETWORK — the resolver is injected.
|
||||
|
||||
Run: python3 scripts/test_check_image_resolvable.py
|
||||
"""
|
||||
import importlib.util
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
_spec = importlib.util.spec_from_file_location(
|
||||
"cir", Path(__file__).resolve().parent / "check-image-resolvable.py")
|
||||
cir = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(cir)
|
||||
|
||||
# One ref that a fake registry "has", one that is knowingly invalid — `.invalid` is RFC-2606
|
||||
# reserved, so this can never accidentally succeed even if someone runs the suite online.
|
||||
GOOD = "getmeili/meilisearch:v1.49"
|
||||
DEAD = "flomp-nonexistent.invalid/wanderer:0.16.0"
|
||||
|
||||
|
||||
def fake_resolver(ref: str):
|
||||
"""Resolves exactly one ref. Everything else — including the canary — is reported GONE."""
|
||||
return (cir.OK, "") if ref == GOOD else (cir.ABSENT, "manifest unknown")
|
||||
|
||||
|
||||
def throttled_resolver(ref: str):
|
||||
"""Docker Hub throttling: rc=0 AND an error on stderr — the live 2026-07-21 failure."""
|
||||
return (cir.INCONCLUSIVE, "toomanyrequests: You have reached your unauthenticated pull rate limit")
|
||||
|
||||
|
||||
def make_catalog(tmp: Path, apps: dict[str, list[str]]) -> Path:
|
||||
for app, images in apps.items():
|
||||
d = tmp / "templates" / app
|
||||
d.mkdir(parents=True)
|
||||
body = "services:\n" + "".join(
|
||||
f" svc{i}:\n image: {img}\n" for i, img in enumerate(images))
|
||||
(d / "docker-compose.yml").write_text(body, encoding="utf-8")
|
||||
return tmp
|
||||
|
||||
|
||||
class TestResolvabilityGate(unittest.TestCase):
|
||||
def test_dead_ref_fails_and_is_named(self):
|
||||
"""The whole point: one dead pin must fail the gate, and say which app pins it."""
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
root = make_catalog(Path(td), {"alive": [GOOD], "rotten": [DEAD]})
|
||||
sites = cir.collect_images(root)
|
||||
absent, inconclusive = cir.check_images(sites, fake_resolver)
|
||||
self.assertEqual(absent, [DEAD], "a dead image pin MUST fail the gate")
|
||||
self.assertEqual(inconclusive, [])
|
||||
self.assertEqual(sites[DEAD], ["rotten:3"],
|
||||
"the failure must point at the app+line that pins it")
|
||||
|
||||
def test_all_resolvable_passes(self):
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
root = make_catalog(Path(td), {"alive": [GOOD], "also": [GOOD]})
|
||||
self.assertEqual(cir.check_images(cir.collect_images(root), fake_resolver), ([], []))
|
||||
|
||||
def test_same_ref_in_two_apps_is_resolved_once_but_both_sites_reported(self):
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
root = make_catalog(Path(td), {"a": [DEAD], "b": [DEAD]})
|
||||
sites = cir.collect_images(root)
|
||||
calls = []
|
||||
|
||||
def counting(ref):
|
||||
calls.append(ref)
|
||||
return fake_resolver(ref)
|
||||
|
||||
absent, _ = cir.check_images(sites, counting)
|
||||
self.assertEqual(calls, [DEAD], "each unique ref must be resolved exactly once")
|
||||
self.assertEqual(sorted(sites[DEAD]), ["a:3", "b:3"])
|
||||
self.assertEqual(absent, [DEAD])
|
||||
|
||||
def test_multi_service_app_collects_every_image(self):
|
||||
"""A repo split (wanderer web+db) means one app pins several images — miss one and the
|
||||
gate would pass an app that cannot start."""
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
root = make_catalog(Path(td), {"wanderer": [GOOD, DEAD]})
|
||||
sites = cir.collect_images(root)
|
||||
self.assertEqual(sorted(sites), sorted([GOOD, DEAD]))
|
||||
absent, _ = cir.check_images(sites, fake_resolver)
|
||||
self.assertEqual(absent, [DEAD])
|
||||
|
||||
def test_untrustworthy_resolver_refuses_to_report(self):
|
||||
"""RED-PROOF companion for the false-all-clear mode: a resolver that says yes to
|
||||
everything (broken docker, an intercepting proxy) must NOT yield a clean bill of health."""
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
root = make_catalog(Path(td), {"rotten": [DEAD]})
|
||||
rc = cir.check(root, resolver=lambda ref: (cir.OK, ""))
|
||||
self.assertEqual(rc, 2, "a resolver that resolves the canary must abort, not pass")
|
||||
|
||||
def test_only_filter_restricts_to_named_apps(self):
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
root = make_catalog(Path(td), {"a": [GOOD], "b": [DEAD]})
|
||||
self.assertEqual(sorted(cir.collect_images(root, only=["a"])), [GOOD])
|
||||
|
||||
def test_check_returns_1_end_to_end_on_a_dead_pin(self):
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
root = make_catalog(Path(td), {"alive": [GOOD], "rotten": [DEAD]})
|
||||
self.assertEqual(cir.check(root, resolver=fake_resolver), 1)
|
||||
|
||||
def test_empty_catalog_is_an_error_not_a_pass(self):
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
(Path(td) / "templates").mkdir()
|
||||
self.assertEqual(cir.check(Path(td), resolver=fake_resolver), 2)
|
||||
|
||||
|
||||
class TestClassifyGuardsAgainstFalseAlarms(unittest.TestCase):
|
||||
"""RED-PROOF for the defect the first live sweep actually had: Docker Hub throttling was
|
||||
reported as 24 dead images (postgres:16-alpine, redis:7-alpine among them). A gate that
|
||||
accuses healthy images gets ignored, so ambiguity must resolve to INCONCLUSIVE."""
|
||||
|
||||
def test_rate_limit_on_rc0_is_inconclusive_not_ok(self):
|
||||
"""The live shape: rc=0 WITH an error on stderr. Trusting rc alone reports a false PASS."""
|
||||
self.assertEqual(
|
||||
cir.classify(0, "toomanyrequests: You have reached your unauthenticated pull rate limit"),
|
||||
cir.INCONCLUSIVE)
|
||||
|
||||
def test_rate_limit_is_never_absent(self):
|
||||
for rc in (0, 1):
|
||||
self.assertEqual(
|
||||
cir.classify(rc, "toomanyrequests: rate limit exceeded"), cir.INCONCLUSIVE,
|
||||
"a throttle must NEVER be reported as a missing image")
|
||||
|
||||
def test_genuine_absence_is_absent(self):
|
||||
self.assertEqual(cir.classify(1, "manifest unknown"), cir.ABSENT)
|
||||
self.assertEqual(cir.classify(1, "errors:\n denied: requested access to the resource is denied"),
|
||||
cir.INCONCLUSIVE, "an auth wall is not proof of absence")
|
||||
|
||||
def test_clean_success_is_ok(self):
|
||||
self.assertEqual(cir.classify(0, ""), cir.OK)
|
||||
|
||||
def test_unrecognised_failure_is_inconclusive(self):
|
||||
self.assertEqual(cir.classify(7, "something nobody has seen before"), cir.INCONCLUSIVE)
|
||||
|
||||
def test_throttled_sweep_reports_incomplete_not_failure(self):
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
root = make_catalog(Path(td), {"a": [GOOD], "b": [DEAD]})
|
||||
rc = cir.check(root, resolver=lambda ref: (
|
||||
(cir.ABSENT, "manifest unknown") if ref == cir.CANARY_REF
|
||||
else throttled_resolver(ref)))
|
||||
self.assertEqual(rc, 2, "a fully-throttled sweep is INCOMPLETE (2), not a failure (1) "
|
||||
"and not a pass (0)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
Reference in New Issue
Block a user