Files
felhom.eu/documentation/audits/update-night-2026-09-21/reproduce_pg_refusal.sh
T
admin 9c69b3ff07
gates / gates (push) Successful in 27s
Update night: Phases 2-4 evidence — both engines, the unattended HOLD, and five new findings
Evidence off the machine at the end of the phases that produced it (R-320). Teardown follows.

PHASE 2 — the two database engines, through the REAL Update button:
- MariaDB 11.6 -> 12.3 on nextcloud: PROVEN, and pressed through the button for the first time.
  All four SPIKE-r459 observables: the datadir's own record moved 11.6.2 -> 12.3.3; the engine
  itself says "already upgraded ... no need to run mariadb-upgrade again"; the entrypoint says
  "Major version upgrade detected ... Check required!" and then STARTED and FINISHED it (not the
  `skipped due to $MARIADB_AUTO_UPGRADE` line R-459 feared); and the engine took its own
  pre-upgrade backup, 631 905 B. The seeded Nextcloud account read back.
- PostgreSQL 16 -> 17 on docmost: FAILED exactly as R-463 predicted and nobody had measured.
  5.1 s to held; the pin named 17 while nothing ran; the restore brought it back in 29.1 s.
  The engine's REFUSAL LINE was destroyed by failAndHold before any probe could read it, so it
  was REPRODUCED INDEPENDENTLY with a control on every step (R-320).

PHASE 3 — the bad days. B1 produced THE UNATTENDED HOLD, which this project has never had: the
caller pressed once with nobody watching, the app held after 312.9 s, and passes 2 and 3 pressed
nothing. B2 put the pin back on a pull failure in 1.0 s. B3 refused `busy` six times. B4 showed
there is NO single-flight — 5 of 5 updates ran at once and all ended honest. B5 cut the power in
`backing-up` and the box recovered itself and said so. B7 refused under the 2 GB floor. B9 found
R-458's risk narrower than the row states.

PHASE 4 — every badge on the box is TRUE, and the held app answers all four of Q4's questions.

FINDINGS, five new and three corrections to existing rows. The one that matters: R-618 is P1 —
two templates name a health probe the app does not answer, and because the guarded update waits
on that same probe, a SUCCESSFUL update ends by STOPPING a working app. Measured: tandoor served
HTTP 200 on the new version at four samples across five minutes and was then stopped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-09-21 22:13:57 +02:00

70 lines
4.0 KiB
Bash
Executable File

#!/bin/bash
# Reproduce, independently of the product, the thing the product destroyed: PostgreSQL 17's own
# words when it meets a datadir written by 16.
#
# EVERY STEP HAS A CONTROL, because the first attempt did not and produced a confident wrong
# answer: the volume lookup returned empty, so the copy was empty, so postgres:17 initialised a
# FRESH datadir and started happily — and the run reported "running=true" as though the refusal had
# not happened. It printed an EMPTY PG_VERSION one line earlier and nothing stopped. An instrument
# that reports nothing must halt the step, not continue it.
set -u
OUT=/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/update-night-2026-09-21/18-postgres-refusal-reproduced.txt
cat > /tmp/pgr2.sh <<'SCRIPT'
set -u
fail() { echo "STOP: $*"; exit 3; }
echo "=== 0. find the live docmost 16 datadir, and PROVE it is 16 before touching anything"
SRC=$(docker inspect docmost-postgres --format '{{range .Mounts}}{{println .Type .Name .Source .Destination}}{{end}}' 2>/dev/null)
echo "mounts of docmost-postgres:"; echo "$SRC"
V=$(docker inspect docmost-postgres --format '{{range .Mounts}}{{if eq .Destination "/var/lib/postgresql/data"}}{{.Name}}{{end}}{{end}}' 2>/dev/null)
[ -n "$V" ] || fail "no volume is mounted at /var/lib/postgresql/data — cannot copy what cannot be found"
echo "volume: $V"
SRCVER=$(docker run --rm -v "$V":/d:ro alpine:3.20 sh -c 'cat /d/PG_VERSION 2>/dev/null')
echo "PG_VERSION of the SOURCE: '${SRCVER}'"
[ "$SRCVER" = "16" ] || fail "the source datadir is '${SRCVER}', not 16 — the reproduction would measure the wrong thing"
echo
echo "=== 1. copy it, and PROVE the copy is 16 too"
docker rm -f DRILL-pg17-refuse DRILL-pg16-ok >/dev/null 2>&1
docker volume rm DRILL-pg16-copy >/dev/null 2>&1
docker volume create DRILL-pg16-copy >/dev/null
docker run --rm -v "$V":/src:ro -v DRILL-pg16-copy:/dst alpine:3.20 sh -c 'cp -a /src/. /dst/ && echo copied'
CPVER=$(docker run --rm -v DRILL-pg16-copy:/d alpine:3.20 sh -c 'cat /d/PG_VERSION 2>/dev/null')
echo "PG_VERSION of the COPY: '${CPVER}'"
[ "$CPVER" = "16" ] || fail "the copy is '${CPVER}' — the copy step did not work"
echo "size of the copy: $(docker run --rm -v DRILL-pg16-copy:/d alpine:3.20 du -sh /d | cut -f1)"
echo
echo "=== 2. THE MEASUREMENT: postgres:17-alpine on that 16 datadir — the household's exact case"
docker run -d --name DRILL-pg17-refuse -e POSTGRES_PASSWORD=drill \
-v DRILL-pg16-copy:/var/lib/postgresql/data postgres:17-alpine >/dev/null
sleep 10
echo "container: $(docker inspect DRILL-pg17-refuse --format 'running={{.State.Running}} exit={{.State.ExitCode}} restarts={{.RestartCount}}')"
echo "--- POSTGRESQL 17's OWN WORDS, VERBATIM:"
docker logs DRILL-pg17-refuse 2>&1 | tail -10
echo "--- the copy's PG_VERSION AFTER 17 refused it (must still be 16 — nothing was migrated):"
docker run --rm -v DRILL-pg16-copy:/d alpine:3.20 sh -c 'cat /d/PG_VERSION'
echo
echo "=== 3. POSITIVE CONTROL: the same copy under postgres:16-alpine must start AND hold the data"
docker run -d --name DRILL-pg16-ok -e POSTGRES_PASSWORD=drill \
-v DRILL-pg16-copy:/var/lib/postgresql/data postgres:16-alpine >/dev/null
for i in $(seq 1 30); do docker exec DRILL-pg16-ok pg_isready -U docmost >/dev/null 2>&1 && break; sleep 2; done
echo "container: $(docker inspect DRILL-pg16-ok --format 'running={{.State.Running}} exit={{.State.ExitCode}}')"
docker logs DRILL-pg16-ok 2>&1 | tail -3
echo "--- the DATA, asked of the engine itself:"
docker exec DRILL-pg16-ok psql -U docmost -d docmost -tAc \
"select count(*) from information_schema.tables where table_schema='public'" 2>&1 | tail -1 \
| sed 's/^/ tables in the public schema: /'
echo
echo "=== 4. teardown, BY NAME"
docker rm -f DRILL-pg17-refuse DRILL-pg16-ok >/dev/null 2>&1
docker volume rm DRILL-pg16-copy >/dev/null 2>&1
echo "removed: DRILL-pg17-refuse, DRILL-pg16-ok, volume DRILL-pg16-copy"
SCRIPT
cat /tmp/pgr2.sh | ssh -o ConnectTimeout=30 demo-hp \
'cat > /tmp/pgr2.sh; pct push 9202 /tmp/pgr2.sh /tmp/pgr2.sh >/dev/null 2>&1; pct exec 9202 -- bash /tmp/pgr2.sh; rm -f /tmp/pgr2.sh' 2>/dev/null | tee "$OUT"