Files
felhom.eu/documentation/audits/the-28-2026-09-22/teardown.py
T
admin 186546d562
gates / gates (push) Successful in 27s
THE TWENTY-EIGHT: every app no drill had touched, walked in one night
All 28 walked on scratch guest 9202 against the private drill catalog. 26 deployed, 6 proven,
5 inconclusive, 14 with no upstream edge, 1 failed honestly (outline 1.9.1->1.10.1, HELD with the
right sentence), 2 undeployable - one (plant-it) by design, refused by the lifecycle gate, proven
live for the first time. Each app also got the half the update night skipped: a restore from its own
copy with the seed read back again - 21 restored, 2 correctly REFUSED per 07 6.2.

R-630 RAISED TO P1 by measurement: a stack with NO probe container does not skip verifying - it
waits out the full health timeout and HOLDS, stopping an app whose three containers read healthy.
The controller's own words: "not healthy within 5m0s (last: no probe container)".

R-633 opened: a remove sent during a restore reports success and leaves a container restarting with
a live public route. The product already refuses that clash for update and for restore, naming the
blocker; remove has no such guard.

R-634 opened: an app can be running, healthy and serving while recorded as deployed=false, and is
then unremovable. Reproducible alone on sparkyfitness; concurrency-linked on two others.

R-631 and R-632 CLOSED. Register 321 -> 323. Seven interventions, six of them my own harness -
named, with what each cost. No product code. The live catalog's image: lines are byte-identical to
the start of the night.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-09-22 16:00:56 +02:00

65 lines
4.4 KiB
Python

#!/usr/bin/env python3
"""Teardown, three layers plus Gitea — and every claim READ BACK, never assumed."""
import json, os, subprocess, sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(os.path.dirname(HERE), "update-night-2026-09-21"))
import walk as w # noqa: E402
LIVE = "1ad1f34b6e51843851839bdd18154a6603c6e590"
rec = {}
w.login()
# ── layer 1: the machine ─────────────────────────────────────────────────────────────────────────
print("== 9202 back on the LIVE catalog")
subprocess.run([sys.executable, os.path.join(os.path.dirname(HERE),
"update-night-2026-09-21", "repoint_drill.py"), "live"],
cwd=os.path.join(os.path.dirname(HERE), "update-night-2026-09-21"))
rec["machine"] = w.guest('''set +e
echo "== controller.yaml git block (token redacted by the reader, not by us):"
grep -A6 '^git:' /var/lib/docker/volumes/felhom-controller-data/_data/controller.yaml
echo "== the CACHE is what decides which remote is followed (R-615):"
C=/var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache
git -C $C remote -v | sed 's#://[^@]*@#://#' | head -1
git -C $C log --oneline -1
echo "== containers still running (expect exactly three):"
docker ps --format '{{.Names}}'
echo "== any container from tonight that should be gone:"
docker ps -a --format '{{.Names}}|{{.Status}}' | grep -vE '^(felhom-controller|filebrowser|traefik)\\|' || echo NONE
echo "== any app.yaml left (expect none):"
ls /opt/docker/stacks/*/app.yaml 2>/dev/null || echo NONE
echo "== drill images (expect none):"
docker images --format '{{.Repository}}:{{.Tag}}' | grep -E '^(localhost:5000|drill/)' || echo NONE
echo "== disk:"
df -h / | tail -1''')
print(rec["machine"])
# ── layer 2: the host ────────────────────────────────────────────────────────────────────────────
print("\n== demo-hp host")
rec["host"] = subprocess.run(["ssh", "-o", "ConnectTimeout=15", "demo-hp",
"LC_ALL=C pct list; LC_ALL=C pvesm status"],
capture_output=True, text=True).stdout
print(rec["host"])
# ── layer 3: the hub ─────────────────────────────────────────────────────────────────────────────
rec["hub"] = "nothing provisioned, nothing changed — 9202 runs hub.enabled: false (R-620)"
# ── Gitea ────────────────────────────────────────────────────────────────────────────────────────
print("\n== Gitea")
subprocess.run(["git", "-C", "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill", "fetch", "-q", "live", "main"])
subprocess.run(["git", "-C", "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill", "reset", "-q", "--hard", "live/main"])
subprocess.run(["git", "-C", "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill", "push", "-qf", "origin", "main"])
rec["drill_head"] = subprocess.run(["git", "-C", "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill",
"rev-parse", "HEAD"], capture_output=True, text=True).stdout.strip()
# THE DIFF THE METHOD REQUIRES: every image: line on the live catalog identical to the baseline
d = subprocess.run(["git", "-C", "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu", "diff",
LIVE, "origin/main", "--", "templates/"], capture_output=True, text=True).stdout
rec["live_template_diff_lines"] = len([l for l in d.split("\n") if l.startswith(("+", "-"))
and not l.startswith(("+++", "---"))])
rec["live_image_line_diff"] = [l for l in d.split("\n")
if l.startswith(("+", "-")) and "image:" in l]
print("live catalog templates/ diff vs baseline:", rec["live_template_diff_lines"], "lines;",
"image: lines changed:", rec["live_image_line_diff"] or "NONE")
json.dump(rec, open(os.path.join(HERE, "teardown", "teardown.json"), "w"),
ensure_ascii=False, indent=2)
print("\nwritten teardown/teardown.json")