# RUNBOOK — the monthly re-test of same-name security fixes (`09` §3 decisions 52, 54, 55; R-740, R-743)

**What it does.** An image such as `postgres:18-alpine` or `redis:7-alpine` gets security fixes under the SAME name.
A box takes such a fix at night only when the catalog has re-tested the tag at the new digest on both venues and written
it as a ladder step. This runbook is that re-test, once a month. **One command does the work**; the steps around it set
up the two venues and tear them down.

**Scope (decision 55):** every app with a proven ladder — not only the database and redis lines. The web apps face the
internet; the databases do not. `--engines-only` is the narrow switch, not the default.

**Who runs it (decision 54):** a CC session the operator starts once a month with the standing brief
`claude/MONTHLY-security-retest.md` (in the planning project), from DooPlex. STATUS carries "Monthly security re-test:
last run <date>, next due <date>" — update it at the end of every run.
**Why not a cron job (measured 2026-09-30):** it needs a fresh bench LXC on demo-hp, scratch guest 9202 pointed at the
drill catalog, a drill reset (a force-push — the permission check refused it once and the operator allowed it), and pushes
to the LIVE catalog. None of that should happen with nobody watching.

## 1. Look first (read-only, 1 minute)

```bash
cd /mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu && git pull -q
python3 scripts/retest-floating.py --dry-run                  # every app with a proven ladder (decision 55)
```

`nothing to re-test today` ends the month. Otherwise go on.

## 2. The bench (LXC 9401 on demo-hp)

The recipe in `audits/rulings-2026-10-01/A/A1-bench-create.txt` (60 GB disk on `nvme-scratch` — 40 GB filled up on
2026-09-30), swap 0 (the stricter venue, R-733); `pveam download` the Debian 13 template first if it is gone.
`retest-floating.py` syncs the catalog's scripts and templates to it itself (it checks only docker + python3 first, R-749).

## 3. The box (scratch guest 9202)

1. **Reset the drill** to the live catalog: `git -C /mnt/5_hdd/felhom.eu/drill/app-catalog-drill fetch live && git reset --hard
   live/main && git push -f origin main` (force-push: ask if the permission check refuses).
2. **Point 9202 at the drill** (`09` §6.5): the `repoint.py drill` of the latest audit's `tools/` (saves `controller.yaml`,
   sets the drill URL + credentials, removes the catalog cache, restarts). Quote `repo_url` read back.
3. `export SC=<a 0600 scratch dir>` holding `.ctlpw` (9202's dashboard password — never committed).

## 4. The run

```bash
python3 scripts/retest-floating.py --push \
    --evidence $SC/evidence --evidence-rel felhom.eu/documentation/audits/retest-<YYYY-MM>
```

Per app: bench (the full method, 10-minute memory watch), box (fresh install at the OLD tested digest, seed, the re-test
entry in the drill, the guarded Update, read-back, the running digest must be the NEW one), then the writer, the catalog
gates and one commit (pushed with `--push`; the pre-push gates run). A failure stops that app and never the list; the
summary names each app DONE or STOPPED with its reason. Order: database/redis lines first, then the rest alphabetically
(nextcloud — internet-facing, holds data — comes before the linuxserver apps). Copy `$SC/evidence/<date>/*` to
`felhom.eu/documentation/audits/retest-<YYYY-MM>/` (the ladder entries cite `retest-<YYYY-MM>/<app>/{bench,box}`).

**Monthly cost (measured 2026-10-01):** see "What it cost" below. linuxserver images (bookstack, radarr, sonarr,
code-server) are rebuilt upstream weekly under the same tag, so most months they come up.

## 4a. Infrastructure pins (R-838, 2026-10-04)

The box's three built-in containers — traefik, cloudflared, filebrowser — are pinned in
`felhom-controller/controller/internal/infra/infra.go` (`TraefikImage`, `CloudflaredImage`, `FileBrowserImage`). They are
**not** catalog templates, so `retest-floating.py` never sees them. Each month:

```bash
cd felhom-controller/controller && python3 scripts/check-infra-pins.py   # exit 1 = at least one BEHIND (report only)
```

For each BEHIND pin: read the upstream release notes between the pinned and the newest version (same channel:
traefik v3.x, cloudflared YYYY.M.P, filebrowser N.N.N-stable — never a beta), name any breaking change against what we
configure, raise the constant, release the controller, prove it on 9202 then on both demo boxes (the containers are
recreated within ~20 s of the new controller starting: traefik/cloudflared by the base-infra bring-up, filebrowser by
the start-up mount sync), and time the public gap through the tunnel. Measured 2026-10-04: ≤ 19.6 s on demo-hp, ≤ 14.7 s
on demo-felhom, counting the controller's own restart (`audits/os-guest-lane-2026-10-04/partF/`).

## 5. Teardown (three layers, stated)

Machine: 9202 back on the live catalog (`repoint.py restore`), apps the run installed are removed by it. Host: `pct
destroy 9401 --purge`. Hub: nothing touched. Reset the drill again (§3.1).

## What proves it works (2026-09-30)

`audits/night-rulings-2026-09-30/A/e2e/`: docmost at an OLDER `redis:7-alpine` digest on 9202, the re-test on the bench,
"run tonight's chain now" → the leg's `docmost: step pressed … step ended done after 95.0 s`, the new digest running, the
data read back, the badge back to "Naprakész".