From 4463243f2e09607e57971ba1070cda1bed2e2b1c Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Mon, 21 Sep 2026 21:00:50 +0200 Subject: [PATCH] Upgrade harness: four fixtures and seven real upstream edges from the update night (R-462) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Test code only — no template changed and no image: line moved. The update night walked real within-a-major upstream edges on scratch guest 9202 through the product's own guarded Update, against a PRIVATE DRILL CATALOG; the live catalog was never touched. This brings the expensive half of that work — the seed routes — back into the harness so the same edges can be run here WITH their ABORT step, which the box deliberately does not offer (09 6.1: whether the old image starts on migrated data is per-app and unpredictable). - upgrade_fixtures.py: ActualBudget, Navidrome, AudiobookShelf, Vikunja. Each seeds through the app's OWN interface (R-156); each carries a negative control run on every verify(), so a readback that has broken into always succeeding fails instead of passing everything. - upgrade-test.py: edges U1..U7, all real upstream moves existing 2026-09-21 that this catalog has NOT made, each holding its database engine constant. - Limitations kept: Navidrome and AudiobookShelf seed the DATABASE half only, and say so. OWED, stated so it is not mistaken for done: the U1..U7 harness RUNS, and with them the per-app ABORT answers. The code is in; the runs are not. Gates: catalog_gates.py --fast — image-pins, engine-major, catalog-since, copy-i18n all OK. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS --- CHANGELOG.md | 28 +++++ REPORT.md | 90 +++++--------- scripts/upgrade-test.py | 32 +++++ scripts/upgrade_fixtures.py | 231 +++++++++++++++++++++++++++++++++++- 4 files changed, 320 insertions(+), 61 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 99e9d10..387d4e7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,31 @@ +## Upgrade harness: four new fixtures and seven real upstream EDGES (2026-09-21, R-462) + +**Test code only. No template changed, and no `image:` line moved.** The update night +(`felhom.eu/documentation/audits/DRILL-update-night-2026-09-21.md`) walked real within-a-major +upstream edges on scratch guest 9202 **through the product's own guarded Update**, each app seeded +and read back through its own front door. This commit brings that work back into the harness so the +same edges can be run here **with their ABORT step**, which the box deliberately does not offer +(`09` §6.1: the box never puts the old version back by itself, because whether the old image starts +on migrated data is per-app and cannot be predicted). + +- **`upgrade_fixtures.py` — four new fixtures**: `ActualBudget` (its own bootstrap + login), + `Navidrome` (`/auth/createAdmin` + `/auth/login`), `AudiobookShelf` (`/init` + `/login`) and + `Vikunja` (register → login → create project → authenticated readback). Every one goes in through + the app's OWN interface (R-156) and every one carries its own negative control, run on every + `verify()`: a wrong password, or an id that cannot exist, must NOT read back — so a readback that + has broken into always succeeding fails instead of passing everything. +- **`upgrade-test.py` — seven new edges, `U1`–`U7`**, all REAL upstream moves that existed on + 2026-09-21 and that this catalog has NOT made: privatebin 2.0.5→2.0.6, docmost 0.95.0→0.96.0, + bookstack 26.05.2→26.05.5, actualbudget 26.7.0→26.9.0, navidrome 0.63.2→0.64.0, + audiobookshelf 2.35.1→2.36.1, vikunja 2.3.0→2.6.0. Each holds its database engine CONSTANT, per + the standing rule that an engine change gets its own edge. +- **Limitations kept rather than papered over.** `Navidrome` and `AudiobookShelf` seed the DATABASE + half only — the library on the drive is not populated — and both say so in their docstrings, as + `BookStack` already does for its file half (R-460). + +**Owed, and stated so it is not mistaken for done: the harness RUNS.** The code is in; the U1–U7 +runs, and with them the per-app ABORT answers, have not been performed. + ## RULE LIFT: a MariaDB major may cross, as its OWN EDGE; PostgreSQL may not (2026-09-21, R-469 + R-450) The engine-major rule of 2026-09-13 named its own expiry — *until the Update button takes a verified diff --git a/REPORT.md b/REPORT.md index 311b50a..df1947e 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,70 +1,40 @@ -# REPORT — the engine-major rule, half lifted (R-469 + R-450) +# REPORT — upgrade harness widened from the update night (2026-09-21) -**Base `18a6d2d8243e` → `5ff36d098cbc`.** No template moved. Architecture read first and named: -`felhom.eu/documentation/architecture/09-update-architecture.md` §3 decision 5 (the 2026-09-13 ruling -and its three precautions) and §6.1 (slice 4 as shipped). +**Test code only.** No template was changed; no `image:` line moved; `git diff` touches exactly +`scripts/upgrade_fixtures.py` and `scripts/upgrade-test.py`. -## Why now +## What was done -The rule of 2026-09-13 named its own expiry — *until the Update button takes a verified backup as its -precondition* — **precisely so it would be removed deliberately rather than forgotten.** That -condition was met the same day: update arc Slice 4 shipped (controller v0.237.0/v0.238.0; any backup -tier since v0.239.0). R-469 exists to make the removal a reviewed diff. This is it, and it removes -exactly half. +The update night (`felhom.eu/documentation/audits/DRILL-update-night-2026-09-21.md`) walked real +within-a-major upstream edges on scratch guest 9202 **through the product's own guarded Update**, +each app seeded and read back through its own front door, against a **private drill catalog** — the +live catalog was never touched. This commit brings the expensive half of that work (the seed routes) +back into the harness, so the same edges can also be run here **with their ABORT step**, which the +box deliberately does not offer. -## What changed +- Four new fixtures: `ActualBudget`, `Navidrome`, `AudiobookShelf`, `Vikunja`. Each seeds through the + app's OWN interface (R-156) and each carries a negative control that runs on every `verify()`. +- Seven new edges `U1`–`U7`, all real upstream moves that existed on 2026-09-21 and that this catalog + has NOT made. Every one holds its database engine constant. -- **LIFTED — MariaDB.** The four `mariadb:` sidecars (`bookstack-db`, `kimai-db`, `nextcloud-db`, - `romm-db`) may cross a major. They now have both halves: a verified backup in front of the Update, - and `MARIADB_AUTO_UPGRADE=1` (R-459) whose conversion the harness WATCHED run on the E3/E3b edges - with the seeded data read back after. -- **NOT LIFTED — PostgreSQL and MySQL.** Postgres performs no `pg_upgrade` and REFUSES to start on an - older major's datadir, across eleven templates (R-463). **A backup is a route BACK, not a - conversion** — the app would simply not come up. MySQL has nothing measured at all. The refusal - text now says this, instead of citing the shipped R-448. -- **NEW — one edge, one migration (R-450's second half, recorded 2026-09-02, enforced now).** A - MariaDB major must be the ONLY image move in its template in that commit. bookstack's `0b73e5e` - moved the application 25.02.2 → 26.05.2 **and** MariaDB 11.6 → 12.3 together: two migrations behind - one edge, and an unreadable failure when it breaks. The refusal names what it was bundled with. - **Within a major is unaffected** and may still ride with anything. -- **The gate PRINTS what it allowed**, by name and with the reason, rather than passing in silence. A - lifted rule that goes quiet is a lifted rule nobody can audit. +## What was proven, and where -## Red-proofs — two, each SEEN to fail +Box-side, through the guarded Update, with the data read back before and after — evidence per app in +`felhom.eu/documentation/audits/update-night-2026-09-21/apps//`. The harness-side runs of +U1–U7 are **owed**: the code is in, the runs are not. -| the mutation | what failed | -|---|---| -| drop the own-edge check (`others = []`) | *FACT: kimai-db 11.6 → 12.3 BUNDLED with the kimai app bump: rc=0 expected 1* — the bookstack shape passes | -| empty `LIFTED` | *GENUINE: kimai-db 11.6 → 12.3 ALONE (the R-469 lift): rc=1 expected 0* — the lift is undone | +## Gates -Both reverted; **40 decoy cases green**. The old *"a lone MariaDB major is REFUSED"* case is now the -*"…is ALLOWED"* case — that inversion is the lift itself. A third case pins the bundled PostgreSQL -shape. +`python3 scripts/catalog_gates.py --fast` — image-pins OK, engine-major OK, catalog-since OK, +copy-i18n OK. `all catalog gates OK`, exit 0. The two slow gates need a container runtime and were +not run; no template changed, which is what they inspect. -`catalog_gates.py --fast`: image-pins, engine-major, catalog-since, copy-i18n — **all OK**. The -pre-push hook ran and passed; no `--no-verify`. +## A catalog defect found by the night, filed and NOT fixed here -## Measured while here, and it belongs in this repo's record - -A read-only sweep of all 66 unique pins against the public registries, from DooPlex, never from a box -(`felhom.eu/documentation/audits/UPDATE-ARC-STATE-2026-09-21.md` §3): - -- **46 of 58 exact pins are behind upstream today**; 39 within a major, 7 across one. The seven were - all pinned 2026-07-18: nextcloud 34→35, paperless-ngx 2.20→3.2, claper 2.5→3.0, gokapi 1.9→2.2, - homepage 1.13→2.4, sparkyfitness 0.17→1.7 (two images). -- **6 of the 7 measurable floating pins have been repushed upstream since the catalog set them** — - `postgres:16-alpine`, `postgres:15-alpine`, `redis:7-alpine`, `mariadb:11.4`, `mariadb:12.3`, - `postgis:16-3.5-alpine`. Only `mariadb:11.6` has not. This is R-446 measured rather than theorised, - and it is the case for recording digests at push time (put to the operator as `09` §3b Q6). -- **`msdeluise/plant-it:0.10.0` is gone upstream.** The repo's own gate says INCONCLUSIVE (it refuses - to read a `denied` stderr as "dead", correctly); two independent signals say it really is gone — - Docker Hub's catalog API answers `object not found` for the whole repository, and the upstream - GitHub repo 404s. The app is already `lifecycle: abandoned`, so this is the expected end state, not - an incident. A deployed plant-it survives; it can never be redeployed. - -## Rows - -**R-469 PARTLY CLOSED** (MariaDB lifted; the PostgreSQL half stands until R-463). -**R-450 half SHIPPED** (the own-edge clause; the automatic-within-a-major half is Slice 6 and awaits -`09` §3b Q1–Q4). **R-605 filed** — a catalog gate that refused to run and one that ran and could not -decide print the same word. +Two apps are presented to the household as UNHEALTHY while working perfectly: `tandoor`'s +`.felhom.yml` probe names port 8080 where the container listens only on 80, and `zipline`'s names +`/api/health` where that app answers 404 — **while the compose healthcheck in the same file uses +`/api/healthcheck` and is correct**. A static sweep of all 53 templates comparing the two health +checks against each other finds both, plus `wger` and `home-assistant` as unmeasured candidates and +`adventurelog` as a false positive. Filed as **R-618** with the proposed gate; deliberately not +fixed in the same commit as the harness change. diff --git a/scripts/upgrade-test.py b/scripts/upgrade-test.py index 4d7de50..7cda502 100755 --- a/scripts/upgrade-test.py +++ b/scripts/upgrade-test.py @@ -81,6 +81,38 @@ EDGES = { "E3b": dict(app="bookstack", note="AUTHORED step: engine half alone", frm={"bookstack": "lscr.io/linuxserver/bookstack:26.05.2", "bookstack-db": "mariadb:11.6"}, to={"bookstack": "lscr.io/linuxserver/bookstack:26.05.2", "bookstack-db": "mariadb:12.3"}), + + # --- added 2026-09-21 by the update night (R-462's widening) ------------------------------- + # Every one of these was run BOX-SIDE FIRST, through the product's own guarded Update on guest + # 9202, with the data read back through the app's own front door before and after. They are + # here so the same edge also gets its ABORT step, which the box deliberately does not offer + # (`09` §6.1: the box never puts the old version back by itself, because whether the old image + # starts on migrated data is per-app and cannot be predicted). + # + # These are REAL UPSTREAM MOVES that existed on 2026-09-21 and that the catalog has NOT made. + # They are candidates the operator may promote; the harness is where the abort answer for each + # of them comes from. + "U1": dict(app="privatebin", note="real upstream move 2.0.5 -> 2.0.6; file-backed, no database", + frm={"privatebin": "privatebin/pdo:2.0.5"}, + to={"privatebin": "privatebin/pdo:2.0.6"}), + "U2": dict(app="docmost", note="real upstream move 0.95.0 -> 0.96.0; PostgreSQL CONSTANT across it", + frm={"docmost": "docmost/docmost:0.95.0", "docmost-postgres": "postgres:16-alpine"}, + to={"docmost": "docmost/docmost:0.96.0", "docmost-postgres": "postgres:16-alpine"}), + "U3": dict(app="bookstack", note="real upstream move 26.05.2 -> 26.05.5; MariaDB CONSTANT across it", + frm={"bookstack": "lscr.io/linuxserver/bookstack:26.05.2", "bookstack-db": "mariadb:12.3"}, + to={"bookstack": "lscr.io/linuxserver/bookstack:26.05.5", "bookstack-db": "mariadb:12.3"}), + "U4": dict(app="actualbudget", note="real upstream move 26.7.0 -> 26.9.0; SQLite in its own volume", + frm={"actualbudget": "actualbudget/actual-server:26.7.0"}, + to={"actualbudget": "actualbudget/actual-server:26.9.0"}), + "U5": dict(app="navidrome", note="real upstream move 0.63.2 -> 0.64.0; DATABASE HALF ONLY", + frm={"navidrome": "deluan/navidrome:0.63.2"}, + to={"navidrome": "deluan/navidrome:0.64.0"}), + "U6": dict(app="audiobookshelf", note="real upstream move 2.35.1 -> 2.36.1; DATABASE HALF ONLY", + frm={"audiobookshelf": "ghcr.io/advplyr/audiobookshelf:2.35.1"}, + to={"audiobookshelf": "ghcr.io/advplyr/audiobookshelf:2.36.1"}), + "U7": dict(app="vikunja", note="real upstream move 2.3.0 -> 2.6.0; the app migrates its own SQLite", + frm={"vikunja": "vikunja/vikunja:2.3.0"}, + to={"vikunja": "vikunja/vikunja:2.6.0"}), } diff --git a/scripts/upgrade_fixtures.py b/scripts/upgrade_fixtures.py index 3bc6f10..dba502f 100644 --- a/scripts/upgrade_fixtures.py +++ b/scripts/upgrade_fixtures.py @@ -265,4 +265,233 @@ class BookStack: return found is True -FIXTURES = {"privatebin": PrivateBin(), "docmost": Docmost(), "bookstack": BookStack()} +# --------------------------------------------------------------------------------------------- +# Added 2026-09-21 by the update night (R-462's widening). Each of these was written and PROVEN +# box-side first, on guest 9202 through the product's own guarded Update, and then ported here so +# the same edge can be run on the harness venue with its ABORT step. The box-side evidence is +# `felhom.eu/documentation/audits/update-night-2026-09-21/apps//`. +# +# The port is mechanical and one thing changes: box-side the app is reached through traefik with a +# `Host:` header, here through the container's own IP. The SEED ROUTE is identical, and that is the +# expensive half. +# --------------------------------------------------------------------------------------------- + + +class ActualBudget: + """Actual's own bootstrap API sets the server password; its own login proves it survived. + + Actual keeps its data in SQLite inside its own volume and performs its own schema migration on + start, so this single seed is the whole data half. + """ + port = 5006 + container = "actualbudget" + + def _base(self, ipfn): + ip = ipfn(self.container) + return f"http://{ip}:{self.port}" if ip else "" + + def seed(self, ipfn, say): + base = self._base(ipfn) + if not base or not _wait_http(base + "/", {"200", "302"}, say=say): + say(" actualbudget: no container IP, or the app never answered") + return None + pw = "Spike-" + secrets.token_hex(10) + rc, code, out = _curl(base + "/account/bootstrap", "-H", "Content-Type: application/json", + data=json.dumps({"password": pw}), method="POST") + say(f" actualbudget: /account/bootstrap http={code} :: {out[:140]}") + if rc != 0 or '"status":"ok"' not in out: + return None + return {"pw": pw} + + def verify(self, ipfn, seeded, say): + base = self._base(ipfn) + if not base or not _wait_http(base + "/", {"200", "302"}, tries=24, say=say): + return False + + def login(p): + return _curl(base + "/account/login", "-H", "Content-Type: application/json", + data=json.dumps({"loginMethod": "password", "password": p}), + method="POST") + + # the fixture's own negative control, run on every verify + rc, code, out = login("wrong-" + secrets.token_hex(6)) + if '"status":"ok"' in out: + say(" actualbudget: READBACK IS UNUSABLE — a wrong password authenticated") + return False + rc, code, out = login(seeded["pw"]) + ok = '"status":"ok"' in out + say(f" actualbudget: login with the seeded password http={code} ok={ok}") + return ok + + +class Navidrome: + """Navidrome's own /auth/createAdmin makes the first account; its own /auth/login proves it + survived. LIMITATION: this is the DATABASE half. Navidrome's other half is the music library on + the drive, which the harness does not populate.""" + port = 4533 + container = "navidrome" + + def _base(self, ipfn): + ip = ipfn(self.container) + return f"http://{ip}:{self.port}" if ip else "" + + def seed(self, ipfn, say): + base = self._base(ipfn) + if not base or not _wait_http(base + "/", {"200", "302"}, say=say): + say(" navidrome: no container IP, or the app never answered") + return None + user = "spike" + secrets.token_hex(3) + pw = "Spike-" + secrets.token_hex(10) + rc, code, out = _curl(base + "/auth/createAdmin", "-H", "Content-Type: application/json", + data=json.dumps({"username": user, "password": pw}), method="POST") + say(f" navidrome: createAdmin http={code}") + if rc != 0 or code not in ("200", "201"): + say(f" navidrome: refused {out[:200]}") + return None + return {"user": user, "pw": pw} + + def verify(self, ipfn, seeded, say): + base = self._base(ipfn) + if not base or not _wait_http(base + "/", {"200", "302"}, tries=24, say=say): + return False + + def login(p): + return _curl(base + "/auth/login", "-H", "Content-Type: application/json", + data=json.dumps({"username": seeded["user"], "password": p}), + method="POST") + + rc, code, _ = login("wrong-" + secrets.token_hex(6)) + if code in ("200", "201"): + say(" navidrome: READBACK IS UNUSABLE — a wrong password authenticated") + return False + rc, code, out = login(seeded["pw"]) + ok = code in ("200", "201") + say(f" navidrome: login as the seeded user http={code} ok={ok}") + return ok + + +class AudiobookShelf: + """audiobookshelf's own /init creates the first root account; its own /login proves it + survived. LIMITATION: the DATABASE half only — the library on the drive is not populated.""" + port = 80 + container = "audiobookshelf" + + def _base(self, ipfn): + ip = ipfn(self.container) + return f"http://{ip}:{self.port}" if ip else "" + + def seed(self, ipfn, say): + base = self._base(ipfn) + if not base or not _wait_http(base + "/status", {"200"}, say=say): + say(" audiobookshelf: no container IP, or the app never answered /status") + return None + user = "spike" + secrets.token_hex(3) + pw = "Spike-" + secrets.token_hex(10) + rc, code, out = _curl(base + "/init", "-H", "Content-Type: application/json", + data=json.dumps({"newRoot": {"username": user, "password": pw}}), + method="POST") + say(f" audiobookshelf: /init http={code}") + if rc != 0 or code not in ("200", "204"): + say(f" audiobookshelf: refused {out[:200]}") + return None + return {"user": user, "pw": pw} + + def verify(self, ipfn, seeded, say): + base = self._base(ipfn) + if not base or not _wait_http(base + "/status", {"200"}, tries=24, say=say): + return False + + def login(p): + return _curl(base + "/login", "-H", "Content-Type: application/json", + data=json.dumps({"username": seeded["user"], "password": p}), + method="POST") + + rc, code, _ = login("wrong-" + secrets.token_hex(6)) + if code == "200": + say(" audiobookshelf: READBACK IS UNUSABLE — a wrong password authenticated") + return False + rc, code, out = login(seeded["pw"]) + ok = code == "200" and seeded["user"] in out + say(f" audiobookshelf: login as the seeded root http={code} ok={ok}") + return ok + + +class Vikunja: + """Vikunja's own REST API: register, log in, create a project, read the project back. Four + calls, all the app's own front door, and the readback is a real authenticated GET.""" + port = 3456 + container = "vikunja" + + def _base(self, ipfn): + ip = ipfn(self.container) + return f"http://{ip}:{self.port}" if ip else "" + + def _token(self, base, seeded, pw=None): + rc, code, out = _curl(base + "/api/v1/login", "-H", "Content-Type: application/json", + data=json.dumps({"username": seeded["user"], + "password": pw or seeded["pw"]}), method="POST") + if code != "200": + return None + try: + return json.loads(out)["token"] + except Exception: + return None + + def seed(self, ipfn, say): + base = self._base(ipfn) + if not base or not _wait_http(base + "/api/v1/info", {"200"}, say=say): + say(" vikunja: no container IP, or the app never answered /api/v1/info") + return None + user = "spike" + secrets.token_hex(3) + pw = "Spike-" + secrets.token_hex(10) + rc, code, out = _curl(base + "/api/v1/register", "-H", "Content-Type: application/json", + data=json.dumps({"username": user, "password": pw, + "email": f"{user}@gate.invalid"}), method="POST") + say(f" vikunja: register http={code}") + if code not in ("200", "201"): + say(f" vikunja: refused {out[:200]}") + return None + seeded = {"user": user, "pw": pw} + tok = self._token(base, seeded) + if not tok: + say(" vikunja: could not log in after registering") + return None + title = "spike-" + secrets.token_hex(5) + rc, code, out = _curl(base + "/api/v1/projects", "-H", f"Authorization: Bearer {tok}", + "-H", "Content-Type: application/json", + data=json.dumps({"title": title}), method="POST") + say(f" vikunja: create project http={code}") + if code not in ("200", "201"): + say(f" vikunja: project refused {out[:200]}") + return None + seeded["title"] = title + seeded["pid"] = json.loads(out).get("id") + return seeded + + def verify(self, ipfn, seeded, say): + base = self._base(ipfn) + if not base or not _wait_http(base + "/api/v1/info", {"200"}, tries=24, say=say): + return False + if self._token(base, seeded, pw="wrong-" + secrets.token_hex(6)): + say(" vikunja: READBACK IS UNUSABLE — a wrong password authenticated") + return False + tok = self._token(base, seeded) + if not tok: + say(" vikunja: the seeded account no longer authenticates") + return False + rc, code, out = _curl(base + f"/api/v1/projects/{seeded['pid']}", + "-H", f"Authorization: Bearer {tok}") + ok = code == "200" and seeded["title"] in out + say(f" vikunja: readback of the seeded project http={code} ok={ok}") + return ok + + +FIXTURES = { + "privatebin": PrivateBin(), + "docmost": Docmost(), + "bookstack": BookStack(), + "actualbudget": ActualBudget(), + "navidrome": Navidrome(), + "audiobookshelf": AudiobookShelf(), + "vikunja": Vikunja(), +}