hub v0.176.1: manifest bump (Restart controller for every guest; token presses named) — on the operator's yes 2026-10-29

Tests red before the fix: TestR1255_TheCarryTakesTheNewestPackageThatHasAHash,
TestR1255_APurgeNeverDeletesAPackageAReusedPathStillNeeds, TestR1255_TwoSetAsidesOnOneDayGetTwoNames.
Commit only; rides the next hub release.

.gitea/workflows/gates.yml

@@ -18,8 +18,13 @@
 name: gates
 # R-1129: `workflow_dispatch` is the manual PROOF that the failure mail can be sent. A push run is
 # unchanged by it: the proof step below is skipped on a push, and `force_alarm` is then empty.
+# R-1297 (A, 2026-10-26): the nightly schedule is the run that still executes the three path-filtered suites when no
+# push touched their files (scripts/script_tests_gate.py PATH_FILTERED). It sets no file list, so every suite runs —
+# like a release tag (a tag push) and a manual dispatch. 03:20 UTC: after the 01:40 release copy and the night chain.
 on:
   push:
+  schedule:
+    - cron: "20 3 * * *"
   workflow_dispatch:
     inputs:
       force_alarm:
@@ -149,6 +154,12 @@ jobs:
           if not commits:
               sys.stderr.write("the payload carries no commits array - unknown\n")
               raise SystemExit(0)
+          # R-1297: Gitea trims the payload's commits array (the newest few only); a push of more commits than it
+          # lists has files nobody named here. That list is UNKNOWN, never short - so nothing is printed.
+          total = ev.get("total_commits")
+          if isinstance(total, int) and total > len(commits):
+              sys.stderr.write("the payload lists %d of %d commits - the file list is unknown\n" % (len(commits), total))
+              raise SystemExit(0)
           seen = []
           for c in commits:
               for key in ("added", "modified", "removed"):
@@ -172,6 +183,18 @@ jobs:
           [ "$SCOPE" = "docs" ] || SCOPE=code
           echo "PUSH_SCOPE=$SCOPE" >> "$GITHUB_ENV"
           echo "scope: $SCOPE"
+          # R-1297 (A): the heavy suites read this list and skip themselves when none of their paths changed. ONLY a
+          # branch push with a known list sets it; a tag, the nightly schedule and a dispatch run every suite.
+          case "${GITHUB_EVENT_NAME:-}:${GITHUB_REF:-}" in
+            push:refs/heads/*)
+              if [ -s /tmp/pushed-files.txt ]; then
+                echo "FELHOM_CHANGED_FILES=/tmp/pushed-files.txt" >> "$GITHUB_ENV"
+                echo "suite filter: ON (a branch push with a known file list)"
+              else
+                echo "suite filter: OFF (no usable file list) - every suite runs"
+              fi ;;
+            *) echo "suite filter: OFF (${GITHUB_EVENT_NAME:-?} ${GITHUB_REF:-?}) - every suite runs" ;;
+          esac
 
       - name: Run the gate entry point
         # The ONLY thing CI runs. No go build, no go test, no linting, no deploy — those are either
@@ -188,6 +211,9 @@ jobs:
           # R-986: the security-scan gate fetches the PINNED gitleaks here (sha256-checked); no Go, so govulncheck says
           # NOT CHECKED in CI, out loud. Warn-first either way: the scan lists, it never fails this job.
           FELHOM_SCAN_FETCH: "1"
+          # R-1327: test_image_runtime.py reads a zstd layer with the PINNED zstandard wheel (sha256-checked); with this
+          # set, a missing decoder FAILS the suite instead of saying NOT CHECKED.
+          FELHOM_WHEEL_FETCH: "1"
         run: python3 scripts/repo_gates.py --fast --scope="${PUSH_SCOPE:-code}"
 
       - name: Manual proof - alarm key present, optional forced failure (dispatch only, R-1129)

.gitea/workflows/security-watch.yml

@@ -0,0 +1,128 @@
+# security-watch — every night, read the security notices of everything Felhom ships, and mail the operator
+# (admin@felhom.eu, never a household) what affects a pinned version (R-1303). The logic, its classes and its mails:
+# scripts/security_watch.py; the list of pins: scripts/security_watch_inventory.py; why these sources:
+# documentation/audits/security-watch-2026-10-26/SPIKE.md.
+#
+# THE MEMORY is documentation/security-watch/state.json on `main`, committed back by this job with its own Actions
+# token. Why there and not elsewhere (2026-10-26): the two runners each have their OWN disk (act-runner-data,
+# act-runner-2-data — a job lands on either), so a runner file would be two memories; Actions cache needs a node
+# `uses:` step, which this python-and-git runner cannot run; anything on the DooPlex host is a host change. A file in
+# the repo is one memory, readable by a person, and its history says when each advisory was first mailed.
+# If the push of the memory fails, the job fails and the alarm below mails — the next night may then mail an advisory
+# twice, which is the safe direction.
+#
+# Manual run (`workflow_dispatch`, mode):
+#   run        a normal night, now
+#   seed       record every advisory open now as known and mail nothing new (the first run; prints what it WOULD mail)
+#   dry        no mail, no memory change — read the result table in the log
+#   test-mail  send ONE clearly marked test advisory to the operator; touches nothing else
+name: security-watch
+on:
+  schedule:
+    - cron: "40 3 * * *"   # 03:40 UTC: after the 03:20 gates run
+  workflow_dispatch:
+    inputs:
+      mode:
+        description: "run | seed | dry | test-mail"
+        default: "dry"
+
+jobs:
+  watch:
+    runs-on: felhom-gates
+    timeout-minutes: 20
+    permissions:
+      contents: write
+    steps:
+      - name: Fetch main (latest, for the memory) and the two sibling repos
+        run: |
+          echo "WATCH_STARTED_AT=$(date +%s)" >> "$GITHUB_ENV"
+          base=http://gitea.gitea-system.svc.cluster.local:3000/admin
+          git init -q . && git remote add origin $base/felhom.eu.git
+          git fetch -q --depth 1 origin main && git checkout -q -B main FETCH_HEAD
+          echo "felhom.eu main at $(git rev-parse --short=12 HEAD)"
+          for r in felhom-controller app-catalog-felhom.eu; do
+            git init -q ../$r && git -C ../$r remote add origin $base/$r.git
+            git -C ../$r fetch -q --depth 1 origin main && git -C ../$r checkout -q FETCH_HEAD
+            echo "$r at $(git -C ../$r rev-parse --short=12 HEAD)"
+          done
+
+      - name: Run the watch
+        env:
+          RESEND_API_KEY: ${{ secrets.RESEND_API_KEY }}
+          GH_ADVISORY_TOKEN: ${{ secrets.GH_ADVISORY_TOKEN }}
+          MODE: ${{ github.event.inputs.mode }}
+          # R-1327: zstd layers (Home Assistant and others) are read with the PINNED zstandard wheel, sha256-checked
+          # (scripts/image_runtime.py); without it they are NOT CHECKED, out loud.
+          FELHOM_WHEEL_FETCH: "1"
+        run: |
+          set -u
+          mkdir -p documentation/security-watch
+          mode="${MODE:-run}"
+          [ "${GITHUB_EVENT_NAME:-}" = "schedule" ] && mode=run
+          echo "mode: $mode"
+          case "$mode" in
+            test-mail) python3 scripts/security_watch.py --test-mail; exit $? ;;
+            dry)  extra="--no-mail" ; state=/tmp/state-dry.json
+                  cp documentation/security-watch/state.json "$state" 2>/dev/null || true ;;
+            seed) extra="--seed" ; state=documentation/security-watch/state.json ;;
+            run)  extra="" ; state=documentation/security-watch/state.json ;;
+            *) echo "unknown mode $mode"; exit 2 ;;
+          esac
+          python3 scripts/security_watch.py --state="$state" --catalog=../app-catalog-felhom.eu \
+            --controller=../felhom-controller --json=/tmp/watch-results.json $extra
+
+      - name: Commit the memory (only when it changed)
+        if: github.event.inputs.mode != 'dry' && github.event.inputs.mode != 'test-mail'
+        env:
+          PUSH_TOKEN: ${{ secrets.GITEA_TOKEN }}
+        run: |
+          set -u
+          f=documentation/security-watch/state.json
+          if [ ! -f "$f" ]; then
+            echo "no memory file was written - nothing to commit"; exit 0
+          fi
+          if git ls-files --error-unmatch "$f" >/dev/null 2>&1 && git diff --quiet -- "$f"; then
+            echo "memory unchanged - nothing to commit"; exit 0
+          fi
+          cp "$f" /tmp/state.json
+          auth=$(python3 -c 'import base64,os;print(base64.b64encode(("x:"+os.environ["PUSH_TOKEN"]).encode()).decode())')
+          for attempt in 1 2 3; do
+            git fetch -q --depth 1 origin main && git checkout -q -B main FETCH_HEAD
+            mkdir -p "$(dirname "$f")" && cp /tmp/state.json "$f"
+            git add "$f"
+            git -c user.name="felhom security watch" -c user.email=monitoring@felhom.eu \
+              commit -q -m "security-watch: memory of $(date -u +%F) (R-1303). Docs only." || { echo "nothing to commit"; exit 0; }
+            if git -c http.extraHeader="Authorization: Basic $auth" push -q origin HEAD:main; then
+              echo "memory pushed: $(git rev-parse --short=12 HEAD)"; exit 0
+            fi
+            echo "push refused (attempt $attempt) - re-fetching main"
+            sleep 5
+          done
+          echo "MEMORY NOT SAVED after 3 attempts"; exit 1
+
+      - name: Alarm on failure
+        if: failure()
+        env:
+          RESEND_API_KEY: ${{ secrets.RESEND_API_KEY }}
+        run: |
+          python3 - <<'PY'
+          import json, os, sys, time, urllib.request
+          key = os.environ.get("RESEND_API_KEY", "")
+          if not key:
+              sys.exit("ALARM FAILED: RESEND_API_KEY is empty")
+          run = os.environ.get("GITHUB_RUN_ID", "?")
+          link = "https://gitea.dooplex.hu/%s/actions/runs/%s" % (os.environ.get("GITHUB_REPOSITORY", "?"), run)
+          started = os.environ.get("WATCH_STARTED_AT", "")
+          elapsed = "%d s" % (int(time.time()) - int(started)) if started.isdigit() else "unknown"
+          print("ALARM-LINK " + link)
+          body = json.dumps({"from": "Felhom CI <monitoring@felhom.eu>", "to": ["admin@felhom.eu"],
+              "subject": "[felhom CI] security watch FAILED",
+              "text": ("The nightly security watch did not finish (R-1303).\n\nRun     : %s\nElapsed : %s\n\n"
+                       "Until it runs again, silence from the watch does NOT mean that nothing affects us.\n"
+                       "The run log names the step: the watch itself, a mail it could not send, or its memory "
+                       "it could not save.\n") % (link, elapsed)}).encode()
+          req = urllib.request.Request("https://api.resend.com/emails", data=body, method="POST", headers={
+              "Authorization": "Bearer %s" % key, "Content-Type": "application/json", "User-Agent": "felhom-ci/1.0"})
+          with urllib.request.urlopen(req, timeout=30) as r:
+              print("RESEND-ACCEPTED id=%s" % json.load(r)["id"])
+          PY

CONTEXT.md

@@ -16,6 +16,194 @@
 > and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
 
 
+> **2026-10-29 (attended release run) — RELEASED: hub v0.175.0, agent v0.168.0 (VOUCHED), controller v0.347.0 (global
+> floor), catalog `313eaf6`. Operator rulings D62 + D63 (`09` §3 decisions 281/282, `07` §6.3).** D62: an unused restore
+> copy stays 30 days (was the night's 14, decided by CC — superseded); the kept-copies list shows each restore copy (date,
+> size, „automatikusan törlődik") with „Törlés most" (typed name; a held/waiting copy never; 409 while a backup/restore
+> runs or the app updates; one log line + operator-only event `restore_copy_deleted`, hub `19afefab`); controller
+> `e3fce09`. D63: R-1256 H4 + H5's failed stop proven live on Tester 1 with a `-tags faultinject` test build
+> (`cmd/controller/faultpoint_{on,off}.go`; the release image has no switch, `grep -c "FAULT INJECTED"` = 0) — nothing
+> reverted; D62 proven end to end on the release build (hub stored the event). Hub 0.175.0 (`sha256:642e94b6…`, from
+> `bf69bb91`, CI 2487; manifest `d6352cd8`, Deployment/hub only): one ceremony on Tester 1 end to end, 30 min clean. Agent
+> 0.168.0 (`1e70196c…`, bundle `3415c034…`, tag `v0.168.0` at `91bd96c`): signed `agent_update` to the three test boxes,
+> each committed, self-check 76/76; vouched 10:21:56 CET; `pinned.day0_agent` = 0.168.0 (`6b0b093`). Controller 0.347.0
+> (`sha256:c79b06e6…`, from `fd75c7c`, CI 2491; rc.1 = `e3fce09` on 9202): per-box floors 09:57 CET, walk green on all
+> three + https-only, a local restore on Tester 1, the OOM line (one `app_oom`, one mail), global floor 10:31 CET
+> (MinAgent 0.158.1). Catalog: the night's two patches + CHANGELOG (`313eaf6`), Tester 1 read it. Tester 1's recovery code
+> re-minted and stored as `TESTER1_RECOVERY_CODE_2026_10_29` (the 10-25 one opens only older packages; removal is the
+> operator's, D60). Filed R-1340 (the mixed-hold mail tells the option-C story), R-1341 (the OOM WARN every 30 s); R-1342 (a copy deletable during the stable-start watch; a removed app alarmed — fixed on main, not released); closed R-1238, R-1338, R-1256, R-1255; R-528 WATCHING to its 11-12 check. Open for
+> the operator: Radicale's hold record on Tester 1. Report: `REPORT-release-2026-10-29.md`.
+
+> **2026-10-28 night (unattended) — restore edges, escrow edges, the host OOM count on main; NOTHING released.**
+> **Decided by CC unattended — operator may reverse:** (1) an unused restore copy goes after 14 days, swept every 6 h
+> (`07` §6.3 [DESIGN] 2026-10-28; options 14 d / for ever / 3 d); (2) R-1256's H4 and H5's failed-stop branch stay on main
+> with test proofs only — the brief wants a live proof or a revert; H4 runs only on an off-site restore (9202 has no
+> target) and both branches fail safe (hold or keep, never pour); the attended live proof or a revert of those two branches
+> of controller `342a394` is owed; (3) the catalog changes (R-1338 measurement, Plex fixtures, the Plex ladder entry) are a
+> prepared patch, `audits/night-2026-10-28/catalog-morning/`, not a push. Controller `342a394` (R-1256), `e33cffa`
+> (R-1254 controller half), `eed23e7` (R-528), `20c5877` (unknown app → 404); agent `5ddd124` (R-1254), `6f933d6` (R-528,
+> `GET /guest/oom-kills`); hub `1ee6d5a5` (R-1255). Live: H5 busy volume + H8 sweep on 9202; R-528 on Tester 1 (one
+> `app_oom` „seen by docker,cgroup"); R-1238 Plex: the unclaimed route works from Plex's own container (its front door answers 401 to the
+> proxy), bench + box 9202 proven, the ladder entry in the prepared patch (`catalog_since` kept at 2026-10-23: the pin did
+> not move, the writer had set today). Claims walked: fail.customer-self-restore (route proven, stays partial:
+> no non-operator yet), fault.selfheal (built → walked). R-1339 filed (a false „not mounted" log line).
+
+> **2026-10-28 (evening, attended) — Gitea 28.2.0; Wanderer's search memory measured, not changed.** Gitea 1.27.3 → 28.2.0
+> live `0baf8f9` (bench + box + undo; 1.27.3 refuses a 28-migrated database, the undo copy restores it). D57 → decision
+> 280: imports only from github.com/codeberg.org/gitlab.com (`[migrations] EGRESS_MODE=strict` + `ALLOWED_HOST_LIST`, the
+> 28.x names — 1.x's ALLOWED_DOMAINS is gone; the security review's „wrong key" notice is pre-28). The Gitea fixture posts
+> every installer field (28 renamed the form). R-1338: `MEILI_MAX_INDEXING_MEMORY`=128 MiB helped on the bench (17.7 %)
+> but NOT on 9202 (95.7 %, steady) → not pushed; the ladder has no entry type for a same-digest re-measure.
+
+> **2026-10-28 (afternoon, attended at the STOPs) — Docker Hub stops refusing the watch; the four big steps.** R-1334:
+> measured first — the 429s were the BURST (211 tag-list requests in ~2 min from six workers), not the count; `fe677bae`
+> spaces Docker Hub requests (1/s), retries a 429 (Retry-After; NOT CHECKED after the 4th try) and caches tag lists 24 h
+> per registry path on the runner (two runners = two caches); `8644450d` retries a registry 429 in the image reader. Dry
+> runs 2458/2459: 0 refused — **no Docker Hub login needed** (the operator's key was not asked for). Big steps, each a
+> NORMAL ladder step (`09` §3 [DESIGN] of today: no pin move without a step, also when no box runs the app), each with
+> bench + box 9202 + the UNDO proven on 9202 (`audits/dockerhub-bigsteps-2026-10-28/undo_proof2.py`, probe on port 1):
+> Wanderer v0.21.2 `1825247` (PocketBase internal, no `SUBDOMAIN_DB`, inbound federation off — no
+> `POCKETBASE_PROXY_SECRET`; Meilisearch 384M, still `memory_tight` → R-1338; offered again, D59 ended, decision 279);
+> Termix 2.8.0 → 2.9.2 `e0b9594` → 26.10.5 `589c957` (two pushes: `check-test-record-move` judges the whole push range
+> as ONE move); Homepage v2.7.0 `1a254d4` (fixture reads its first-start config). New tools: `box_walk.py
+> --to-definition`, fixtures Wanderer/Homepage/Seerr (Seerr brings a throwaway Jellyfin, `felhom-drill-jellyfin`).
+> Termix X-Forwarded-For on 9202: the 6th wrong login is 429 on 2.8.0, 2.9.2 and 26.10.5 from the LAN (traefik
+> replaces the header) and through the tunnel's address with Cloudflare's appended real address.
+
+> **2026-10-28 (day, attended at the STOPs) — HTTP2Shell lane batches; the runtime cache fills itself in CI.** **Operator:**
+> D61 = keep RomM offered (`09` §3 decision 277; the first RomM with Node ≥ 24.22.1 is a lane step). **Decided by CC (the
+> brief's choice):** zstd in CI by a hash-pinned `zstandard` 0.25.0 wheel, not `zstd` in the runner image (decision 278) —
+> this REPLACES the 10-27 night's „cache filled on DooPlex" (2) above. **Live on the operator's yes, CI green by hash:**
+> catalog `b5b78f6` (HA 2026.10.4, Wanderer hidden), `0158b50` (Immich v3.3.5, repair re-proven), `abf3db3` (ghost,
+> metube, zipline), `71e3fdf` (karakeep, tandoor), `651a991` (papra, dawarich), `0829994` (adventurelog v0.13.1 + health
+> `start_period` 300 s — its backend needs ~2 min 45 s after an update). Box-walk tool fixes (catalog `box_walk.py`):
+> `move_images` moves two services sharing one image; the walk loads the drill into the box mirror before the install.
+> felhom.eu: `a3247d9f` (zstd), `73c91eac` (decoder named in words), `29e9cd36` (the watch's memory keyed per group —
+> `dooplex:<image>|<id>`, old keys copied once; W13). Corrected: Calibre-Web DOES load Brotli (`/proc/<pid>/maps` from the
+> guest; in-container maps were unreadable). Rows: R-1313, R-1322, R-1327, R-1332 closed; R-1334–R-1337 opened (229 → 229).
+> 9202 ends on the LIVE catalog; the drill is back at FROM for all lane apps (`5c51ddc`). Trap: a `git pull --autostash`
+> in felhom.eu emptied a walk's `walk.txt` that was being written in the clone — walk output goes to the scratchpad now.
+
+> **2026-10-27 night (attended to 22:40, then alone) — D60, the runtime watch, HTTP2Shell.** **Decided by CC unattended
+> — operator may reverse:** (1) Home Assistant's monthly release is the same main version (one comparator) → a LANE step
+> with CVE-2026-111497 (`09` §3 decision 276); (2) the watch's runtime cache is filled on DooPlex (zstd, the stored Docker
+> Hub login through `sec run`) and committed in `state.json`, instead of a zstd dependency in CI (`08` §11); (3) no
+> Paperless step (it decodes only replies to its own calls, R-1331); (4) RomM kept (STATUS D61, R-1330); (5) the live
+> catalog changes stay PATCHES/drill commits at night — the push waits for the operator. **Done:** D60 enforced
+> (cc-secrets 0.4.0 `092e282` + two review hardenings `27fd806`/`e99f937`; proven in two new CC sessions); D56–D59
+> recorded (decisions 271–275); hub `startupProbe` 180 s live on the operator's yes (`a69ddbde`, one start, 0 restarts;
+> R-1326 closed); `scripts/image_runtime.py` + the watch's „runtime" source and „dooplex" group (`f5a4995d`; 154 images
+> cached; positive controls HA + RomM, negative audiobookshelf); ghcr.io tag paging fixed; lane walks PROVEN on 9202 and
+> written to the DRILL: Home Assistant 2026.9.4 → 2026.10.4 (`3c8ebb5`), Immich v3.2.4 → v3.3.5 (`34b14d5`); Wanderer
+> hidden in the drill (`ee18bff`). Rows R-1327–R-1332 opened, R-1326 closed. Morning: the catalog push (writer on the
+> live catalog), the homelab list to the homelab session.
+
+> **2026-10-27 evening — R-1299 the clean-up window button (hub v0.174.0 + v0.174.1); cc-secrets 0.3.0 `sec rm`.** The
+> customer page's „Backup & DR" tab has the „Off-site clean-up window" card (`internal/web/offsite_window.go`): the
+> one-shot grant through a confirm step, same route and rule (`/offsite/window-grant/<id>`, `wr(grpOffsite)`), cancel,
+> a limit bounded by the store's snapshot count (ceiling 500). Every grant/cancel is a STORED operator-only info event
+> `offsite_window_grant` — 0.174.0 only called the dispatcher (which writes no row) and its test asserted the call; the
+> operator's live click exposed it, 0.174.1 writes the row (`store.SaveEvent`) and the test reads rows. Hub builds came
+> from a clean clone (`FELHOM_HUB_REPO_DIR`) — the shared clone held another session's uncommitted installer edit.
+> Start-up 31–36 s vs a liveness kill at ~65 s: R-1326. `sec rm NAME` (cc-secrets `dc82cf9`); 46 dead names removed on
+> the operator's yes (92 → 46). Agent 0.167.0 vouched by the operator; `pinned.day0_agent` = 0.167.0 (agent `3055bc5`).
+
+> **2026-10-27 (attended release block) — RELEASED: hub v0.173.0, agent v0.167.0, controller v0.346.0.** Hub 0.173.0
+> deployed 09:35 CET (image `sha256:458bfb03…`, from `29134c52`, ArgoCD sync of Deployment/hub only, 30-min watch clean).
+> Agent 0.167.0 by `release-agent.sh` from a clean clone (agent `28bdd282…`, bundle `9d034416…`; tag `v0.167.0`), signed
+> `agent_update` to the three test boxes (configs/ unchanged → no bundle job); **NOT vouched yet** — after the vouch set
+> `pinned.day0_agent` in `felhom-agent/scripts/retention-policy.json` (D50). Controller 0.346.0 from `b1b9be3` (image
+> `sha256:3d910b03…`, CI 2423): per-box floors 15:09 CET, box walk green, held-apps lists match (0 = 0), global floor
+> 15:37 CET (MinAgent 0.158.1). **R-1250 fixed the same day (`09` §3 decision 269):** the hold failed four times
+> („not quiet" — qBittorrent sends ~14 s after its limit falls; the agent reads 5–15 s after the call); `710ceac`
+> `upwatch.waitDrained` asks only after the held app reports its own upload near zero; live on `0.346.0-rc.1` on 9201:
+> 94.5 Mbit/s, the limit back exactly (`audits/linetest-2026-10-27/`). **R-1218's live proof waived (decision 270):** the
+> hub page's off-site switch did not reach Tester 1's saved target (R-1312). demo-hp's slot is minute 95 (11:35 CET),
+> Tester 1's 213, demo-felhom's 20 — each host's own. D55's window ran 03:18Z: 223 removed, 373 → 150. R-1174 closed
+> (the status gate refuses since today). Closed 11 rows; opened R-1312.
+
+> **2026-10-26 night (alone) — the data-safety fixes, on `main`, NOT released.** Controller: R-1220 `7e15a37`
+> (`liveDriveBinds(drives)`: an unreadable app.yaml = installed), R-1218/R-1219 `1170b48` (stage when the password FILE
+> exists; no ceremony during a recovery offer), R-1187 `dfd0f8e`…`90a2530` (.fab: links as links, FIFO refused, `os.Root`
+> writes, downward-only links, owners/modes, busy and bad-path refusals; the import's compose gets `USERDATA_PATH`/
+> `IMPORT_PATH`; the DB loads through `appbackup.ImportDump` — the import's `mysqladmin`/`mysql` broke every MariaDB 11+
+> import), R-1298 `b156e45`, Go 1.27.3 `2560cf1`. Agent: R-1232 `3911cc6` (own-key archives only in the due-check),
+> Go 1.27.3 `fbbb168`. Hub: R-1218 `09b5f02f` (409 on a key-less package over a keyed one), Go 1.27.3 `d625a143`.
+> Live: `audits/night-fixes-2026-10-26/` (R-1220, R-1187, R-1298 on 9202 with local test images; R-1232 on a BYO Tester4
+> VM, uninstalled and destroyed). R-1218's live shape needs the operator (off-site off on Tester 1) — else reverted
+> before the release. R-528: design in the row, not built. New: R-1299, R-1302.
+
+> **2026-10-26 (night) — D55 = 260 (`09` §3 decision 268); the grant is SET.** CC used the operator's one-purpose token
+> `HUB_TOKEN_D55_WINDOW` („cc-d55-window", Off-site write only) once: `POST /offsite/window-grant/demo-hp` `max_remove=260`
+> → 200, 19:10:31Z; read back `offsite_window_grant:demo-hp = max:260` (hub DB with WAL) and the hub log line „granted ONE
+> clean-up window … 260". The grant is consumed at demo-hp's next window request (after its next nightly off-site run).
+> The token is the operator's to revoke after that. The hub has no page button for the grant (R-1299).
+
+> **2026-10-26 (evening run) — R-1297 A built (`ecd82aba`, `e33ec821`); D53 built, commit only (controller `5bd03ad`, hub
+> `7e57e824`); D54 blocked on a scope.** CI: `script_tests_gate.py` `PATH_FILTERED` (decoys / due-checks / dooplex-offsite
+> run on a branch push only when their paths change; `FELHOM_CHANGED_FILES` set only for `push:refs/heads/*` in
+> `gates.yml`; nightly 03:20 UTC + tags run all); `test_repo_gates.py` / due-checks group G narrow `repo_gates.GATES` by
+> label (the wiring, not a second full run). The decoy suite's register-shape cases now assert the script's exit code
+> (an inert `main()` passed them). D53: `internal/holdreport` (controller) → `app_held` / `app_hold_cleared` (info) →
+> hub `held_apps` + `HeldAppsChecker` (`app_held_reminder`, operator-only, per-app cooldown; grace 6 h / 1 d / 3 d, then
+> weekly; `alerting.held_app_grace_override` test-only). Live on 9202 against a TEST hub process (no hub deploy):
+> `audits/d53-held-apps-2026-10-26/`. D54: CC's hub token lacks `offsite:write` (403) — STATUS D55 (the cap: the plan
+> grows ~14 a night, 220 would be refused). New R-1298 (9202's orphan holds would be reminded weekly after D53 ships).
+
+> **2026-10-26 (evening) — operator rulings by starting the evening brief: D53 = operator-only held-app reminders first
+> (`09` §3 decision 265; build `08` §10); D54 = one clean-up window for demo-hp with cap 220 through the hub's one-shot
+> grant `POST /offsite/window-grant/<customer>` (decision 266; the cap rule unchanged); R-1297 (B) DONE by the operator — a
+> second CI runner, both runners 2 CPUs (decision 267). CI trim (R-1297 A) is CC's: every removed run keeps a path filter,
+> a tag run or a nightly run.
+
+> **2026-10-26 (afternoon run) — R-1295 fixed at the cause and released; R-1292 released; R-1294 read; R-1285 committed;
+> R-1291 designed.** R-1295's cause, measured on a throwaway PBS 4.2.5: the PBS proxy caches permissions up to 5 s and an
+> ACL-only change does not clear it, so under concurrent requests a new token is refused for ~5 s (v1.3.0 11/30 failed,
+> v1.4.0 0/30). Fixed: felhom-tenantsync v1.4.0 INSTALLED on ep0 (operator yes), hub v0.172.0 RELEASED (retry + operator
+> mail `pbsdr_provision_failed`); live: a fresh Tester4 install made its first whole-guest copy. Controller v0.345.0
+> RELEASED (global floor 13:27:04Z): the 50 % rule owns qBittorrent's NORMAL slot (live on Tester 1). R-1294: demo-hp's
+> plan is honest but can never catch up with the cap (D54). R-1285: agent `b1bc63d`, the pass after the commit, not
+> released (live needs a box from 0.164.0). R-1291: `08` §10 design (D53). New R-1296, R-1297 (CI speed: operator „both").
+
+> **2026-10-26 (attended release run) — released: hub 0.171.0, agent 0.166.0 (vouched 10:04:55 CET), controller 0.344.0
+> (global floor 09:07:47 UTC, MinAgent 0.158.1), catalog `73c165d`; golden 0.344.0 baked and vouched (10:26:23 CET).**
+> Live on the release builds (`audits/release-monday-2026-10-26/`): R-1287 (the set-aside mails „⚠️ … nothing failed"),
+> R-1289 (the vouch line names 0.344.0), R-1280 (a 25 h job REJECTED on demo-hp, a 30 min one ran), R-1281 (forged header
+> 000 with a working control; the household's files 200; OnlyOffice fetches the document over `felhom-files`), R-1282,
+> R-1278, R-1286, R-1283 — closed. **R-1285 back to OPEN:** in the normal two-job delivery the bundle runs in the OLD
+> agent process, so the policy pass waits for the next bundle. R-1261 stays VERIFY until Tuesday's live line test (R-1250).
+> New: R-1294 (demo-hp's weekly off-site clean-up refused by its own cap, 209 > 179). The golden waiver file is removed —
+> the gate is green on the new golden alone. D50 pin `day0_agent` → 0.166.0.
+
+> **2026-10-26 (attended release run) — D51 = Start; D52 = bake the golden today on v0.344.0.** (`09` §3 decisions 263,
+> 264.) D51: the operator pressed Start on demo-hp's Kimai (07:57 local); the crash-loop hold lifted, both containers
+> healthy, 0 restarts (R-1286). D52: the Sunday 2026-10-25 bake was missed (not in that night's brief); the golden waiver
+> is renewed for 2026-10-26 only (`expires: 2026-10-27`) so the release commits can be pushed; the golden is baked on
+> v0.344.0 the same day and the waiver is removed after the vouch. The weekly Sunday bake stays the rule; every
+> Sunday-night brief names it.
+
+> **2026-10-25 night (unattended, commit only) — the night's fixes, each red-proved; proven live on a BYO test VM (demo-hp
+> VM 9372, customer Tester4, agent 0.165.0, controller rc.1–rc.3).** Controller (`a79a292`…`7ea942a`, rides v0.344.0):
+> R-1261 the line hold holds EVERY running seeder, drops a removed app's record, names qBittorrent's slot (`up_limit` /
+> `alt_up_limit` via `app/preferences` — `transfer/uploadLimit` follows the current speed mode, measured); R-1288 the
+> restore of a set-aside unit sets the replaced unit aside AT the restore, no alarm (the brief's safest reading, `07`
+> §8.2d changed); R-1282 the „later" media restart stored and resumed; R-1281 FileBrowser served on its own
+> `felhom-files` network (traefik + OnlyOffice while linked), off traefik-public — the forged header from an app went
+> 200 → no answer; R-1286 every hold reason says what it is (Kimai's was a crash-loop stop, not a failed restore). Hub
+> (`f6328486`, v0.171.0): R-1287 a set-aside-only digest is a warning „nothing failed"; R-1289 suffixed versions never
+> „the fleet's newest". Agent (`d912307`, `abcc1e6`, next agent release + vouch): R-1280 a signed window ≤ 24 h; R-1285
+> the agent runs the wrapper's `bundle-policy` pass after every bundle install. Drill catalog `6733764`: R-1283 MeTube
+> `optional` (a named ruling in the download-binds gate). New rows R-1291 (a held app has no lasting signal), R-1292
+> (the 50 % rule follows the speed mode).
+
+> **2026-10-25 (evening, attended) — D50 = A; installer 1.36.1 published.** (`09` §3 decision 262.) The registry keeps the
+> newest 5 agent versions + the MinAgent floor + the day-0 agent; the agent CI asserts exactly these
+> (`retention-policy.json` `pinned`, copied from hub state by the vouching session — CI holds no hub secret). The operator
+> also pruned controller, golden and hub images by hand; every artifact still named (manifest agent/bundle/golden, the
+> floor agent, tags v0.158.1+, controller 0.343.0/0.342.0/0.331.0, samba 1.2.0, the ISO) downloads
+> (`audits/night-2026-10-25/part1-artifacts.txt`). Installer 1.36.1 (`installer-v1.36.1` → `79ee3e33`, served sha256
+> `e18cdd17…`): the preflight words 401/404/409 again (R-1284).
+
 > **2026-10-25 (attended release run) — D49 = A, D44/D45 = all picks, D46 = yes; released: hub 0.170.0, agent 0.165.0,
 > controller 0.343.0, catalog `ebbf1f4`** (`09` §3 decisions 258–261; `REPORT-release-2026-10-26.md`). **D49 = A:** a
 > restore whose app does not start keeps both copies and never pours back by itself (`07` §8.2e). **Fix A** (controller

REPORT-afternoon-2026-10-26.md

@@ -0,0 +1,72 @@
+# REPORT — 2026-10-26 afternoon: R-1295 fixed at the cause, R-1294 read, R-1285 / R-1292, R-1291 designed, releases
+
+**Plain words first.**
+1. **R-1295 cause:** PBS's proxy keeps permissions in a 5-second cache, and a new access key's grant does not clear it.
+   When other boxes are talking to ep0 at that moment, the new key is refused for a few seconds, and the old script
+   gave up after one quick retry. **Fixed at the cause on ep0** (the script now waits up to 15 s) **and in the hub**
+   (retries, and **a failure now mails you** — yes). Live: a fresh Tester4 install set up its whole-box off-site copy.
+2. **Tester 2's reinstall is safe** from this failure: the fix is on ep0 and in the hub, and if anything still fails you
+   get a mail that says what to do.
+3. **R-1294:** demo-hp's clean-up plan is **honest** (old copies only, every app keeps its copies), but it can never get
+   back under the safety cap by itself. **Your pick D54:** a one-time window with a raised cap (my pick).
+4. **R-1292 fixed and released; R-1285 fixed, not released** (its live proof needs a box coming from 0.164.0).
+5. **Released:** hub 0.172.0, controller 0.345.0 (every live box), felhom-tenantsync v1.4.0 on ep0. Not released: agent.
+
+Safest readings taken alone (said first): R-1292's live test ran on Tester 1, not 9202 (9202 has no tested line, so the
+rule never acts there); R-1285 has no live test today (a box coming from 0.164.0 needs a signed downgrade chain); the
+fresh-install proof ran after the hub rc deploy, both on your yes.
+
+## Each fix, before → after
+
+- **R-1295 (hub v0.172.0 + tenantsync v1.4.0):** a new access key refused by the PBS cache → rolled back, no off-site copy,
+  only a log line → the script waits up to 15 s (the admin key too); the hub retries 3×, 7 s apart; a final failure mails
+  you („… could not be set up — save the customer config to retry"). Measured 11/30 → 0/30 under load; red-proved.
+- **R-1292 (controller v0.345.0):** with alternative speed on, the 50 % rule changed the alternative limit or gave up →
+  it sets only the normal limit. Live on Tester 1; red-proved.
+- **R-1285 (agent `b1bc63d`, not released):** the old agent installed the bundle, so the new policy pass waited a release
+  → the new agent runs it once after its update commits. Tested in the real order; red-proved.
+- **R-1294 (read only):** no change; D54.
+- **R-1291 (design only):** `08` §10; D53.
+
+## Versions, CI by commit
+
+| What | Version / digest | Commit → CI |
+|---|---|---|
+| hub | 0.172.0, `sha256:7c8d129a…` (rc.1 `f6cb8f57…` for the proof) | `722013da` 2362 ✓, `232f42d2` 2366 ✓; manifests `ee38dee2` 2364 ✓ |
+| ep0 | felhom-tenantsync v1.4.0, sha256 `751b3c26…` (v1.3.0 kept beside it) | `722013da` |
+| controller | 0.345.0, `sha256:2dc2c0a5…` (rc.1 `1378e4e2…`) | `2619191` 2365 ✓, `6bfb784` 2368 ✓ |
+| agent | — (not released) | `b1bc63d` 2363 ✓ |
+
+Global floor 0.345.0 at 13:27:04Z (MinAgent 0.158.1), read back from the hub DB with its WAL and the hub log; the
+per-box floors cleared. Every live box reports 0.345.0.
+
+## Rows
+
+Before **212**, after **212**; opened **2** (R-1296 the hub drops ep0's refusal reason; R-1297 CI speed — your „both"),
+closed **2** (R-1295, R-1292). R-1285 VERIFY; R-1294 and R-1291 wait for D54 / D53.
+
+## Observations
+
+- FILED: R-1296 — PBS's real refusal reason never reaches the hub log (found only by reading ep0's journal).
+- FILED: R-1297 — felhom.eu CI ~13 min, one runner; ruled „both".
+- NOT-A-FINDING: the `backup_tier_skipped` 6 min after the fresh install — before the box applied its descriptor; the hub
+  held the mail by design (box enrolled < 1 h); 15 min later the box took its key and made its first copy.
+- NOT-A-FINDING: 9202's FileBrowser WARN on the move from 0.343.0 (twice now) — healed in 3 s, log only, every live box is
+  past that move.
+- NOT-A-FINDING: R-1292 run 1 „FAIL" — my probe's quoting broke its reads (kept as evidence); run 2 with a fail-fast
+  instrument check passed.
+- My mistake, fixed: removing Tester 1's test record the first time raced a path unit that restarted the controller; the
+  second time I stopped the path unit first.
+
+## Teardown
+
+- **Machines:** VM 9373 (the R-1295 fresh install) and LXC 9374 (the throwaway PBS) destroyed with their disks; Tester 1's
+  two throwaway qBittorrents removed through the product, its rule record removed (it had none before), helper files
+  gone; 9202 back on 0.343.0 exactly as found (FileBrowser on `traefik-public`, no `felhom-files` network, the rc image
+  deleted — 9202 is not bound to the hub, so no floor moves it). Nothing of mine runs on demo-hp into Tuesday 10:00.
+- **Product uninstall of VM 9373: not run** — tried root SSH with the original password (refused: day-0 replaced it with
+  the vaulted one) and DooPlex's key (never installed); the vault read is yours. The VM was destroyed instead.
+- **Hub:** delete host `Tester4-b5ec39` (STATUS). Its first whole-guest copy stays in Tester4's ep0 namespace (a host
+  delete keeps the tenancy, by design).
+- **ep0:** only the script swap you approved; v1.3.0 kept as `/usr/local/bin/felhom-tenantsync.v1.3.0`.
+- **Secrets:** evidence scanned (the sec values + the bind token and pairing code): 0 hits.

REPORT-cloudflared-2026-10-26.md

@@ -0,0 +1,76 @@
+# REPORT — cloudflared 2026.10.3 on every box (R-1300), 2026-10-26 evening, attended
+
+> A sibling of `REPORT.md`: the night session (`TASK-night-burndown-and-tuesday-2026-10-26.md`) owns `REPORT.md`
+> tonight.
+
+## In plain words
+
+1. **We run 3 cloudflared, one in each box's guest** (Tester 1, demo-hp, demo-felhom). None on DooPlex. Tester4 has no
+   tunnel; Tester 2 has no box.
+2. **The https-origin trap does not apply.** Each tunnel sends to `https://traefik` with „No TLS Verify" on, and traefik
+   shows its own self-signed certificate — not an eMudhra one.
+3. **0.345.1 is on every box: 4 of 4 reporting controllers** (the three above + Tester4); none left behind.
+4. **Longest tunnel gap: about 11 seconds** (demo-hp). Tester 1 about 2 s, demo-felhom about 4 s.
+
+Decisions: the operator said yes to the build-gate exception (build from a pushed tag) and to the global floor.
+
+## Versions, digests, CI
+
+| What | Value |
+|---|---|
+| controller tag `v0.345.1` | commit `f3204024` = v0.345.0 `6bfb784` + one commit; CI job 2393 success (attempt 1) |
+| controller `main` | `a1c65b3d` (same bump + `build.sh` tag path); CI job 2392 success (attempt 1) |
+| controller image `0.345.1` / `:latest` | `sha256:7cfdf7e1f835628de9c7d536e0a60c06bad5bbe19e8133765cc0a6bb732dbff6` |
+| cloudflared | `cloudflare/cloudflared:2026.10.3`, index `sha256:faa8e0a1475f137c910be6025f43c8e4092fdd35461b31e19fd1390657a1ff39` |
+| MinAgent | 0.158.1 (unchanged, declared with the floor) |
+| global floor | 0.345.1 at 20:08:39Z (`303 floor_set`; read back from the running hub: „Effective floor: v0.345.1 — source: DB") |
+| per-box floors | set 20:05:08Z (Tester 1), 20:06:04Z (demo-hp, demo-felhom); cleared 20:39:54Z |
+
+Tests and gates: controller gates `--fast` green on both commits; `go test ./...` green on the tag commit; the new
+reconcile test and the four build-gate cases red-proved. felhom.eu gates green on push. **Methods:** box state by
+`docker ps`/`docker logs` in each guest; tunnel state by the hub's Hosts page and by a public name with no app
+(`cfprobe.<domain>`: 404 = tunnel up, 530 = down), probed every 0.5–1 s from DooPlex.
+
+## Register
+
+Rows before **213**, after **214**, opened **2** (R-1300, R-1301), closed **1** (R-1300).
+
+## Observations
+
+- The N100 host's clock is 14 s slow; chrony fails at every boot — **FILED: R-1301** (and `lived-in/n100.md`).
+- I put a test run and a commit in one command on the detached tag clone (standing rule 1). The commit was local; the
+  test result was then read on its own (green) before anything was pushed — **NOT-A-FINDING: my slip, caught before
+  push, recorded here.**
+- `noTLSVerify` on every tunnel route (R-510) — **NOT-A-FINDING: a leg that never leaves the guest** (cloudflared and
+  traefik alone on the /29 tunnel network); now written in `01` as a recorded exception, with the rule „never turn off
+  a certificate check to make an update work".
+- cloudflared 2026.10.4 (today, VULN-176088: metrics debug endpoints allowlist) — **NOT-A-FINDING: our metrics bind to
+  the container's own loopback**; not taken because it is under a day old and the notice names 2026.10.3.
+- 0.345.1 is built with 0.345.0's Go builder; `main` moved to Go 1.27.3 tonight (R-972) — **NOT-A-FINDING: R-972 owns
+  it; the next release from `main` carries it.**
+- Single-sample `000` answers at 20:13:37 (both Tester 1 probes at once) and 20:17:13 (demo-hp) — **NOT-A-FINDING: a
+  client-side timeout, no 530; the hub and the boxes showed the tunnel up throughout.**
+- `internal/stacks` tests took 450 s under the night session's load (each R-442 remove test 26 s) — **NOT-A-FINDING: CI
+  green; slow only under load.**
+- Tester4's agent reads `0.167.0-night1 floor held` on the Hosts page — the night session's agent; not touched.
+
+## What changed in documents
+
+- `architecture/01-topology-and-trust.md` „Tunnel placement": the version, the rule, the recorded exception.
+- `architecture/11-os-updates.md` C8: the new pin.
+- `STATUS.md`: one line (released; `main` and Sunday's golden carry it). The golden 0.344.0 still has 2026.9.3; a new
+  box moves to the floor in its first minutes. No extra bake (operator did not ask).
+- System poster: none of the three needed — no fact on the poster changed (cloudflared still runs in the guest).
+- Controller: `build.sh` may push from the pushed tag `v<VERSION>` (operator's yes). Controller report:
+  `felhom-controller/REPORT-cloudflared-0.345.1.md`.
+
+## For the operator
+
+- **Nothing to do.** If you want a second look: Cloudflare dashboard → Networking → Tunnels shows each connector's
+  version (CC holds no Cloudflare token).
+
+## Teardown
+
+Machine: the image pulled on scratch guest 9202 for the version check was removed. Host: nothing. Hub: the three
+per-box floors cleared; the global floor stays at 0.345.1 by design. Clones `felhom-controller-sec`,
+`felhom-controller-mainsec` and `felhom.eu-cfsec` on DooPlex are my work copies (all pushed); removed at the end.

REPORT-dockerhub-bigsteps-2026-10-28.md

@@ -0,0 +1,162 @@
+# REPORT — 2026-10-28 (from midday, attended at the STOPs): the watch and Docker Hub; the four big security updates
+
+Architecture read: `08-alarm-ladder.md` §11 (the watch); `09-update-architecture.md` §3 (decisions 255, 274, 275) and
+§6.4 (the ladder); the catalog's `CLAUDE.md` (the test record) and its move gates.
+
+## In plain words
+
+1. **Why Docker Hub refused the watch:** speed, not volume. The night run asked 211 times in about 2 minutes from six
+   workers and got 15 refusals. Now the requests are spaced, and a refusal is retried. **No Docker Hub login was needed.**
+2. **One CI night with 0 refusals:** not yet. Two daytime test runs had 0 refusals. Tonight's 03:40 run is the check
+   (R-1334 stays open until then).
+3. **The four apps — all live, each a normal ladder step** (bench + box + undo proven):
+   - Wanderer v0.21.2
+   - Termix 26.10.5 (two steps: 2.8.0 → 2.9.2 → 26.10.5)
+   - Homepage v2.7.0
+   - Seerr v3.5.1 (a definition step: new image name, new start command)
+4. **Wanderer is offered again: yes** (your word; D59 ended). Federation from other servers stays off (your word).
+5. **What you do:** nothing today. Tomorrow morning I check that the night run had 0 Docker Hub refusals and then
+   close R-1334.
+
+## Decisions (yours, at the STOPs)
+
+- **Pushed, one STOP each:** Wanderer, Termix, Homepage, Seerr.
+- **Wanderer offered again**, and **its inbound federation stays off** (`09` §3 decision 279).
+- **The Docker Hub key was not asked for.** The fix without a login measured clean.
+
+## Part 1 — R-1334 (evidence: `documentation/audits/dockerhub-bigsteps-2026-10-28/part1/`)
+
+**The cause, measured.** Docker Hub's tag-list API (`hub.docker.com`) refused requests that came in a burst:
+
+| Run | Requests | Time | Refused |
+|---|---|---|---|
+| Night 2446 | 211 | ~2 min | 15 |
+| Dry 2450 | 224 | ~3.5 min | 0 |
+
+No scheduled job on DooPlex calls Docker Hub at that hour (k3s cron jobs and CI schedules, read only: Renovate runs
+on Saturdays; the others are Longhorn backups and Postgres dumps). Other traffic from the same address was not measured.
+
+**The fix.**
+- `fe677bae`:
+  - Docker Hub requests are spaced: one per second across the workers.
+  - A 429 is retried after its `Retry-After`. The image is NOT CHECKED only after the 4th try.
+  - Tag lists are kept for 24 hours per registry path, on the CI runner. Two names of one image fetch once.
+  - Test W14: 7 checks, red first.
+- `8644450d`: the image reader also retries a registry 429. ghcr.io had refused two blob reads when two dry runs came
+  15 minutes apart. Its retry checks were red first.
+
+**Measured after.** Dry runs 2458 and 2459:
+- 0 Docker Hub 429s and 0 retries. 210 requests over about 7 minutes.
+- The only NOT CHECKED images are 6 private DooPlex ones (by design) and `minio/minio:latest`, which Docker Hub no
+  longer serves (noted on R-1328 for the homelab session).
+- The two CI runners each keep their own tag cache. So the cache helps a re-run on the same runner. The spacing is
+  what fixes the night.
+
+## Part 2 — the four big steps (evidence: `documentation/audits/dockerhub-bigsteps-2026-10-28/<app>/`)
+
+**The path, by the rules.** No box runs any of the four apps (checked on all three boxes and on 9202). There is still
+no pin move without a step: the catalog's test record and its move gates require it. `09` did not say so, so it now
+has a `[DESIGN]` note. So each app is a normal ladder step. In every case below the undo was proven on 9202: the new
+version, with its checks broken on purpose, ended `undone`, and the data read back on the old version.
+
+### Wanderer v0.20.0 → v0.21.2 — catalog `1825247`
+
+- **Template, changed at v0.20.0 first:**
+  - PocketBase is internal (no public address, no database-subdomain question at install). Upstream says the db must
+    not be reachable from outside.
+  - The sign-up lock now also covers `PUT /api/v1/user`. v0.20.0 already had that route, and the old lock missed it.
+  - No `POCKETBASE_PROXY_SECRET`, so inbound federation is rejected. There is no federation switch, and this is the
+    only way to close it.
+- **Fixture:** a user and a private list.
+- **Proof:**
+  - Bench: `proven`.
+  - Box 9202: update in 78 s.
+  - Undo: back to v0.20.0 in 28 s.
+- **Memory:** Meilisearch fills whatever limit it gets: 256 → 384 MB, still 88.8 %. That is filed as R-1338.
+
+### Termix 2.8.0 → 2.9.2 → 26.10.5 — catalog `e0b9594`, `589c957`
+
+- **Two pushes**, because 2.9 must start once before 26.x.
+- **Fixture:** the existing one.
+- **Proof, per hop:** bench, box and undo all proven.
+- **D58, measured on 9202.** Seven wrong logins, each with a different forged `X-Forwarded-For`:
+  - **From the LAN:** Traefik replaces the header. The 6th try got HTTP 429 on 2.8.0, 2.9.2 and 26.10.5.
+  - **Through the tunnel's own address** (a test container at cloudflared's address, with the chain Cloudflare
+    delivers): the 6th try got 429 on all three.
+  - So the bypass was not reachable on our boxes, before the update either. 26.10.5 adds the app's own guard
+    (`trust proxy: loopback`).
+- **A loopback-only login route in 26.10.5:** off in our image. `NODE_ENV=production`, and nginx sets `X-Real-IP` on
+  every request.
+
+### Homepage v1.13.2 → v2.7.0 — catalog `1a254d4`
+
+- **Fixture:** a new one. It reads the dashboard's own services and bookmarks (written at first start) and plants
+  nothing.
+- **Proof:** bench, box and undo all proven. The family gate still answers 302.
+
+### Seerr: Jellyseerr 2.7.3 → Seerr v3.5.1 — catalog `9ef06a7`
+
+- **A definition step:**
+  - The new image, plus `init: true`.
+  - The container starts as root only to hand `/app/config` to UID 1000. Then it runs Seerr as `node` (smoke-tested
+    first).
+- **Fixture:** a new one. It brings a throwaway Jellyfin for Seerr's first sign-in and stores an application title.
+- **Proof:**
+  - Bench: `proven`. Its bare abort loses the seed on the converted data, as Jellyfin did yesterday.
+  - Box 9202: `proven`.
+  - Undo: Seerr ran its migration, failed its check, and the box's undo copy brought the data back.
+
+**Tools added in the catalog.**
+- `box_walk.py --to-definition`, with 8 checks in `test_box_walk_drill_move.py`.
+- The Wanderer, Homepage and Seerr fixtures.
+- The bench's Wanderer override was removed, because the template does it now.
+- The undo tool: `audits/dockerhub-bigsteps-2026-10-28/undo_proof2.py`. Its broken probe now uses port 1, because a
+  single-page app answers 200 on any path.
+
+## Commits and CI (all green by hash)
+
+| Repo | Commit | What | CI job |
+|---|---|---|---|
+| felhom.eu | `fe677bae` | Docker Hub spacing, retry, tag cache | gates 2457; dry runs 2458 and 2459 |
+| felhom.eu | `8644450d`, `3c0a33c1` | image-reader retry; evidence | 2460 |
+| catalog | `1825247` | Wanderer | 2461 |
+| catalog | `e0b9594` | Termix hop 1 | 2462 |
+| catalog | `589c957` | Termix hop 2 | 2463 |
+| catalog | `1a254d4` | Homepage | 2464 |
+| catalog | `9ef06a7` | Seerr | 2467 |
+| felhom.eu | this close-out | docs, register, evidence | in the final check |
+
+## Register — 229 before, 226 after
+
+| Opened (1) | Closed (4) |
+|---|---|
+| R-1338 — Meilisearch memory | R-1335 — Wanderer |
+| | R-1311 — Termix |
+| | R-1320 — Homepage |
+| | R-1317 — Seerr |
+
+R-1334 is VERIFY; it closes after tonight. R-1328 has a note added (minio).
+
+## Observations
+
+| Observation | Result |
+|---|---|
+| Meilisearch fills its memory limit | FILED: R-1338 |
+| `minio/minio:latest` gives 401 on Docker Hub | FILED: noted on R-1328 (the homelab list) |
+| Three security-review notices: „image downgrade" in the drill | NOT-A-FINDING: the drill is the test copy and must stand at FROM before a walk; the live catalog is at the new versions |
+| The update accepts a container whose Docker check is still „starting" | NOT-A-FINDING: by design the declared probe decides (`09` §3); the undo tool now breaks the probe on port 1 |
+| `test-record-move` judges a whole push as one move, so two hops refused together | NOT-A-FINDING: the rule works as built; Termix went out as two pushes |
+| The `memory_tight` mark needs a raised limit and a new measurement | NOT-A-FINDING: the gate did its job; done for Wanderer |
+| The Termix header test, first try: no gate cookie, and the shell expanded `$x` too early | NOT-A-FINDING: my test tool; fixed and re-run, and every try is kept |
+| The Termix header test with a forged-only chain skipped the limit on every version | NOT-A-FINDING: Cloudflare always appends the real address; with the real chain every version locks |
+| Termix 2.9.2's route dropped for a while right after the first install (502, then 404) | NOT-A-FINDING: the test now waits for health; 2.9.2 is only a stop on the way to 26.10.5 |
+| Termix 26.10.5's loopback auto-login route | NOT-A-FINDING: off in production, and nginx sets `X-Real-IP` |
+| Secret scan of the evidence found `"password": "wrong-"` | NOT-A-FINDING: the deliberately wrong test password, a code literal |
+
+## Teardown
+
+- **Machine:** 9202 is on the LIVE catalog (`repoint-9202-live.txt`, cache `9ef06a7`). It runs only its standing apps.
+  Every test app was removed through the product. The throwaway Jellyfin containers were removed on the bench and
+  on 9202.
+- **Bench:** 9401 is stopped, with 0 containers.
+- **Host and hub:** nothing provisioned, no change.

REPORT-night-2026-10-25.md

@@ -0,0 +1,49 @@
+# REPORT — installer 1.36.1 published; the night's fixes (2026-10-25 evening → 2026-10-26 morning)
+
+**Plain words first.**
+1. **Every package that is still named is there.** You pruned old controller, agent and golden packages by hand; nothing a
+   box or the installer can still ask for is missing (the agent 0.165.0 + bundle, the 0.158.1 floor agent, golden
+   0.331.0, the agent tags from 0.158.1 up, controller 0.343.0 / 0.342.0 / 0.331.0, samba 1.2.0, the ISO).
+2. **Installer 1.36.1 is live, and its three messages were seen:** a wrong customer → „not found (404)", a wrong
+   passphrase → „passphrase REJECTED (401)", a customer whose box is live → „already has a live box (409)" (R-1167's
+   sentence, which could never show before).
+3. **The agent CI is green with D50 = A** (the newest 5 versions + the MinAgent floor + the day-0 agent).
+4. **R-1261 is fixed and proven before Tuesday** — on a test VM with two seeders: both held, 94.8 Mbit/s measured, both
+   limits back; a speed-mode switch mid-test never left 1 KiB/s. It reaches households with Monday's controller release.
+5. **You decide:** Kimai on demo-hp — press Start, or remove it (STATUS, D51). And the Monday release list (STATUS).
+
+## Each fix, before → after
+
+- **R-1261 (controller `a79a292`):** one seeder held, a removed app's record kept forever, qBittorrent's limit put back
+  in whatever slot its mode showed → every seeder held, the record dropped, the held slot named. Live on the rig.
+- **R-1287 (hub `f6328486`, v0.171.0):** a set-aside mailed „🔴 1 of 1 apps failed to back up" → „⚠️ … a backup of an
+  earlier install was set aside — nothing failed", a warning. Test on the real dispatch path.
+- **R-1288 (controller `b5863a9`):** a set-aside restore's replaced unit was set aside at the NEXT backup, with a second
+  alarm → set aside at the restore, no alarm. Live on the rig (one set-aside mail only).
+- **R-1281 (controller `7ea942a`):** an app container signed in to FileBrowser with a forged header (200) → no answer;
+  the household's file manager still 200; a FileBrowser left on traefik-public is moved off by the next round. Live.
+- **R-1289 (hub `f6328486`):** a dev build no box ran was „the fleet's newest" → test builds never count.
+- **R-1282 (controller `b5863a9`):** the 03:30 media restart was forgotten by a controller restart → stored, resumed once.
+- **R-1280 (agent `d912307`):** a signed window of any length → at most 24 h (verifier, root wrapper, felhom-opsign).
+- **R-1285 (agent `abcc1e6`):** a BYO host lost our SSH file one update late → the agent runs the new wrapper's policy
+  pass after every bundle install.
+- **R-1283 (drill catalog `6733764`):** MeTube's downloads `excluded` → `optional` (a named ruling in the gate).
+- **R-1286 (controller `838c834`):** Kimai's crash-loop hold read „a failed restore whose rollback also failed" → every
+  hold reason says what it is. The cause: the crash-loop guard during the 10-14 power drill.
+
+## Teardown — three layers
+
+- **Machine (rig VM 9372 on demo-hp):** the product's own `--uninstall` ran first (rc 0, „UNINSTALL complete — removed:
+  guest 9201, the felhom-agent …", `audits/night-2026-10-25/rig/uninstall-console.txt`); the host journal was copied off
+  before (`rig/rig-host-journal.txt`). Then `qm destroy 9372 --purge 1`: `qm list` shows no 9372, its folder on
+  `nvme-scratch` is gone, the plain-PVE ISO is removed from demo-hp, the ISO work folder on DooPlex is removed, the old
+  host key is removed from `known_hosts`.
+- **Host (demo-hp):** guest 9202's test tracker, downloader container, test files and image are removed (0 containers,
+  0 files, port 6969 closed). `nvme-scratch` is a standing storage (it holds 341, 9202, 9401) — this session did not add
+  it, so it stays. `local-lvm` (the thin pool) was never used: 48 %.
+- **Hub:** host `Tester4-84bbe7` stays on the Hosts page — a host delete is the operator's (STATUS, „Also Monday"). The
+  R-1285 test package was deleted from the registry (204, then 404).
+- **Secrets:** the scratch session, CSRF and header files, the dashboard password file, the rig root password file, the
+  test bundle and the test agent build are shredded. The evidence was scanned against the four known values (through
+  `sec`) and four token shapes: 0 hits (one shape hit is code that reads a file). The `sec` entries
+  `RIG_2026_10_25_DASH_PW` and `RIG_2026_10_25_SETUP_CODE` remain; `sec` has no delete, and the box they opened is gone.

REPORT-night-2026-10-28.md

@@ -0,0 +1,155 @@
+# REPORT — night 2026-10-28 → 29 (unattended)
+
+## Safest readings taken (decided by CC unattended — operator may reverse)
+
+1. **An unused restore copy is deleted after 14 days** (R-1256 H8; `07` §6.3 [DESIGN] 2026-10-28; CONTEXT). Options:
+   14 days, for ever (as before), or 3 days. A held or waiting copy is never touched.
+2. **H4 and H5's failed-stop branch stay on main with test proofs only.** Rule 1 asks for a live proof by morning or a
+   revert.
+   - H4 runs only on an off-site restore, and 9202 has no off-site target. A fault injected there races the write-back.
+     It can leave an operator-only hold (R-379), which is not ours to clear.
+   - H5's failed stop needs a `docker stop` that fails on demand.
+   - Both fixes fail safe: they hold or keep, and never pour.
+   - Reading: keep them, and say so first. The live proof is owed to an attended session. Or the operator reverts those
+     two branches of controller `342a394`.
+3. **The catalog changes are two prepared patches, not a push** (the brief: drill only, no live catalog push).
+   `documentation/audits/night-2026-10-28/catalog-morning/`. The catalog working tree is clean again.
+4. **Plex's `catalog_since` stays 2026-10-23** in the prepared patch. The ladder writer set it to the day it ran. But the
+   date means "the pin last moved", and the pin did not move tonight (it moved on 10-23, the one `--no-verify` push).
+   Put back by hand, and written in the patch README.
+5. **STATUS was at its 50-line limit.** Yesterday's one-item do-list (the homelab list, R-1328) became item 5 of the
+   morning list. Its words did not change.
+
+## Five lines
+
+1. Restore edges: 4 of 4 fixed (H4, H5, H6, H8), each test red first. Proven live: 2 (H5's busy volume, H8's sweep).
+2. Escrow edges: R-1255 (hub: the carry, the set-aside name, the purge) and R-1254 (the ceremony half, agent +
+   controller) fixed on main, tests red first. No live proof: that needs a hub deploy, and no hub deploy runs at night.
+3. OOM proven: **yes** — on Tester 1, one `app_oom` "seen by docker,cgroup" and one mail.
+4. Plex route: **works** — unclaimed, from Plex's own container. Proven on the bench and on box 9202 (1.41.4 → 1.43.5,
+   the section read back with the same uuid).
+5. Claims walked: 2 walked, 2 proven, 0 failed. Not walked: 31 of 55 (was 32).
+
+## Morning release list (each one is yours; nothing was released)
+
+1. **Hub vX:** R-1255 (felhom.eu `1ee6d5a5`).
+2. **Agent vY + your vouch:** R-1254 (`5ddd124`) and R-528 (`6f933d6`). Then `pinned.day0_agent` (D50).
+3. **Controller vZ:** R-1256 (`342a394`), R-1254's controller half (`e33cffa`), R-528 (`eed23e7`), and an unknown app
+   answers 404 (`20c5877`). MinAgent unchanged (0.158.1): an older agent answers 404, and the controller asks again
+   after 10 minutes.
+4. **Catalog:** the two prepared patches. R-1338 (the memory figure; Wanderer re-measured at 27.7 %, not tight), and
+   the Plex fixtures + the Plex ladder entry (R-1238, proven on both venues).
+5. Owed to an attended session: the live proof of H4 and H5's failed stop, or a revert of those two branches.
+
+If nothing is done: the fixes stay on main only. No box changes.
+
+## [DESIGN] lines
+
+- R-1256: `documentation/architecture/07-backup-architecture.md` §6.3, "[DESIGN] 2026-10-28 night — four restore
+  edges", plus two [FACT] paragraphs (H5, H8 live).
+- R-1255: `03-host-agent.md`, "[DESIGN] 2026-10-28 night — WHICH retained row is carried".
+- R-1254: `03-host-agent.md`, "[DESIGN] 2026-10-28 night — a lost or late answer".
+- R-528: `08-alarm-ladder.md`, "[DESIGN] 2026-10-28 night — the host's counter beside Docker's flag", with its [FACT]
+  of the Tester 1 proof.
+
+## What was proven live
+
+- **H5, scratch 9202, 01:33 UTC:** Vikunja's first volume was held by a created-only container. The household's
+  restore was pressed through the page. The result was "Failed to remove volume vikunja_vikunja_data before the restore
+  … (not written into)". The restore failed, and the put-back poured the live copy of both volumes. The app ran, with
+  no hold. Projects from before and after the restore point read back.
+- **H8, scratch 9202, 03:29 UTC:** three copies were planted by hand. The first sweep removed the two older than 14
+  days and kept the young one (8 ms). Then the young one was removed by hand.
+- **R-528, Tester 1, 2026-10-28 23:27 UTC:** a Radicale capped at 128 MB. The host counter went 0 → 1, with one
+  `app_oom` "seen by docker,cgroup" and one operator mail.
+- **R-1238 Plex box walk, 9202, 03:45–03:59 UTC:**
+  - Deployed through the setup gate at 1.41.4.
+  - A library section was seeded from Plex's own container. Its front door answers 401 to the proxy.
+  - The household's Update ran in 41 s.
+  - The section read back on 1.43.5 with the same uuid.
+  - The 10-minute soak peaked at 10.6 %, with no kill. The verdict was `proven`.
+
+## Claims walked (household first; front doors only)
+
+| Claim | Result | Evidence |
+|---|---|---|
+| fail.customer-self-restore | **PROVEN-LIVE** (the route). Stays `partial`: no one other than the operator has done it yet. | `audits/night-2026-10-28/part5-claims/selfrestore-walk.log` |
+| fault.selfheal | **PROVEN-LIVE**, built → walked. A crashed redis ran again in 1 s, and the probe read the app healthy. A guest reboot brought everything back in under 2 minutes. No event, no mail. | `…/part5-claims/selfheal/WALK.md` |
+
+Not walkable tonight, with the reason for each:
+
+- backup.sikeres (the run half): needs an off-site target, and 9202 has none.
+- backup.fill-warning: 9202's drive is a shared 938 GB host disk, which cannot be filled safely.
+- The app-crash mail and fault.operator-email: 9202 has no hub, and a hub deploy is not a night's work.
+- backup.restore-proof: needs every tier on both machines. Its block is R-366. CC's hub token reads no event list.
+
+## Commits and CI (by commit)
+
+| Repo | Commit | CI run id / # | Result |
+|---|---|---|---|
+| felhom.eu (hub) | `1ee6d5a5` R-1255 | 2470 / #1331 | green |
+| felhom.eu | `8a3df79f` design notes | 2476 / #1332 | green |
+| felhom.eu | `d8752de9` R-1338 evidence | 2477 / #1333 | green |
+| felhom.eu | `0abed37f` + `6a6dd94d` H5, the self-restore walk, row states | 2478 / #1334 | green |
+| felhom.eu | `7a6b9ff1` 07 [DESIGN] + facts, fault.selfheal walked | 2482 / #1337 | green |
+| felhom.eu | close-out (this report, STATUS, CONTEXT, rows, Plex box evidence, patches) | see the end of the final report | — |
+| controller | `342a394` R-1256 | 2469 / #532 | green |
+| controller | `e33cffa` R-1254 | 2472 / #533 | green |
+| controller | `eed23e7` R-528 | 2474 / #534 | green |
+| controller | `20c5877` unknown app → 404 | 2475 / #535 | green |
+| controller | `a815b37` docs | 2483 / #536 | green |
+| agent | `5ddd124` R-1254 | 2471 / #283 | green |
+| agent | `6f933d6` R-528 | 2473 / #284 | green |
+| agent | `91bd96c` docs | 2479 / #285 | green |
+| catalog | none pushed (2 prepared patches) | — | — |
+| drill | `41b37ad`, `37736f0`, then their reverts `8aaf198` + `3b66d35` (the tree equals its start) | — | — |
+
+## Teardown (three layers)
+
+- **Machine — scratch 9202 (04:14 UTC):**
+  - The controller is back on 0.346.1 and healthy. The test image `0.347.0-rc.n1028` is removed (a local tag, never
+    pushed).
+  - The live catalog is back; the config is byte-identical to the saved copy.
+  - The drill mirror is back at its start (`320cff5d`).
+  - No throwaway app, no planted restore copy, no volume made tonight. The containers are as found: the controller,
+    the file manager, paperless (3) and traefik.
+- **Machine — Tester 1 (23:30 UTC):** controller 0.346.1, agent 0.167.0 (sha256 `28bdd282…`), the test app removed.
+- **Host — demo-hp:** the bench 9401 is stopped. No new guest, and nothing on the thin pool.
+- **Hub:** nothing provisioned.
+
+## Register
+
+Before: 225. After: 226. Opened: 1 (R-1339). Closed: 0.
+
+## Observations
+
+- FILED: R-1339 — at boot the controller logs "drive …/userdata/<app> not mounted", but the drive IS mounted. No data
+  is lost; the log line misleads a diagnosis.
+- NOT-A-FINDING: the Plex bench soak got 401s. An unclaimed Plex refuses the harness's request. There were 0 kills, and
+  the peak was 12 % of 2 GiB.
+- NOT-A-FINDING: Plex's front door answers 401 to the box walk. Plex does not take the proxy's address as local. The
+  fixture asks from Plex's own container, which is the measured route (R-1238).
+- NOT-A-FINDING: 9202 drops `app_start_failed` and `controller_started`, because it has no hub. This is by design on a
+  scratch box.
+- NOT-A-FINDING: the ladder writer sets `catalog_since` to the day it runs. It is made for a pin move, and this was the
+  one entry written after a move made by hand (R-1238's history). Put back in the patch.
+- NOT-A-FINDING: the security review flagged the drill's Plex downgrade. It was on purpose: drill only, scratch 9202,
+  and reverted.
+- NOT-A-FINDING: the 03:45 push was refused once. The nightly security watch pushed its own state during the gate run.
+  A rebase, then the push went in.
+- NOT-A-FINDING: the security watch's daily list has a new Brotli decoder hole in many images. That is its normal job,
+  with its own mail. A catalog image move would be a live catalog push, which the night fences.
+- NOT-A-FINDING: 9202 holds 359 Docker volumes, all made before 2026-10-28 16:00. None is from tonight. This is scratch
+  hygiene from earlier sessions, not a product defect.
+- Fixed (small): two dead bench Wanderer tokens were redacted from committed evidence. They stay in the history of
+  `d8752de9`; the throwaway app is gone.
+- Fixed (small): a stale R-356 sentence in the self-restore claim's note.
+
+## Other
+
+- Instruction files: none edited.
+- System poster: no listed fact changed. The restore copy is not on the poster, and nothing was released.
+- Architecture read: `07-backup-architecture.md` (restore), `03-host-agent.md` (escrow), `08-alarm-ladder.md` (OOM),
+  `09-update-architecture.md` §6.4 (the ladder).
+- Secrets: the night's passwords stayed in the scratchpad. Every evidence folder was scanned; the only hits are setting
+  names (`cf_tunnel_token`, `PLEX_CLAIM`, "password from settings.json"), never values.

REPORT-release-2026-10-27.md

@@ -0,0 +1,70 @@
+# REPORT — 2026-10-26 night → 2026-10-27 afternoon: the night's fixes, the line test, one release block
+
+**Safest reading taken alone:** none needed today — every release step had the operator's yes in the session.
+
+## Plain words first
+
+1. **The night's fixes:** all released today. Proven live: kept data (R-1220), the reinstalled box's off-site tier
+   (R-1232), app-file export/import (R-1187 a, b, g), held-app reports (R-1298, D53). Not live: R-1219 (handler test, as
+   the brief allowed) and R-1218 (the operator waived its live proof — no product path reaches the case on a hub box).
+2. **The line test:** ran — but only after a same-day fix. As built, the hold was refused four times (qBittorrent keeps
+   sending ~14 s after it is slowed). With the fix: **94.5 Mbit/s**, the seeder's own limit back **exactly**.
+3. **Released:** hub 0.173.0, agent 0.167.0 (not vouched yet), controller 0.346.0 (global floor 15:37 CET).
+4. **D55 grant seen:** yes — the window ran 03:18 UTC, 223 of 223 removed (373 → 150), no alarm.
+5. **The operator still does:** vouch agent 0.167.0; revoke the token `cc-d55-window`; delete hub host
+   `Tester4-b5dbbd`.
+
+## Versions, digests, CI by commit
+
+| | Version | Image / sha256 | From | CI |
+|---|---|---|---|---|
+| hub | 0.173.0 | `sha256:458bfb03e614db4b7abd6442e6af15c652ba4b9b4509fe2b0efe00ac979b8f52` | `29134c52` (manifest `bbc3acb8`) | 2418, 2419 ✓ |
+| agent | 0.167.0 | agent `28bdd2826f45f3767880fcff4d3105afb7247c09c0ced99d78b3a978d8d826f0`, bundle `9d0344164b22634159984bfbcdaf77c53e6d2423812973ed8b4f2135a7ed1268` | `54d3a10` (tag `v0.167.0`) | 2400, 2420 ✓ |
+| controller | 0.346.0 | `sha256:3d910b039fc52f08366c657774d9664320d6913fdebdd3deb2bbbb01d7eb9dd3` | `b1b9be3` (fix `710ceac`) | 2421, 2423 ✓ |
+| controller rc | 0.346.0-rc.1 | `sha256:bfa091e9…` | `710ceac` | 2421 ✓ |
+
+## Each fix, before → after
+
+- **R-1250** (`710ceac`): four holds refused „not quiet" (4.0–4.2 Mbit/s) → held, slowed after 22 s, 94.5 Mbit/s, limit
+  5898240 back exactly; the 50 % rule kept the normal slot (0.15 % move). `audits/linetest-2026-10-27/`.
+- **R-1232**: `due:false` (a foreign archive counted) → `due:true / absent`, first whole-box copy at once (night).
+- **R-1220 / R-1187 / R-1298**: see `audits/night-fixes-2026-10-26/`.
+- **R-1218** hub: a key-less upload over a keyed package → 409 (red-proved); controller: stage when the FILE exists
+  (red-proved). Live proof waived (`09` §3 decision 270).
+- **R-1285**: the bundle pass now runs right after the commit in the normal delivery (all three boxes) — the BYO removal
+  itself is still unproven (stays VERIFY).
+- **R-1174** (dated check): the status gate refuses since today — a hand run convicted a copy, rc 1.
+
+## Release steps
+
+Hub: yes 09:30 CET → build, manifest, sync of Deployment/hub only, Synced/Healthy, the digest matches, the „Held apps"
+list renders, 30 min: 0 events, 0 WARN, reports flowing, no alarm mail. Agent: yes → release-agent.sh from a clean clone,
+signed updates (demo-felhom 10:26, Tester 1 10:32, demo-hp 13:49 after its test). Controller: rc.1 on 9201 and Tester 1
+(9202 was session 2's) → 0.346.0 → per-box floors 15:09 → all three „update successful" → box walk green (start logs 0
+errors, every app healthy, https-only green, held apps 0 = 0) → yes → global floor 15:37 → 30-min watch clean → per-box
+floors cleared 16:09 (DB reads empty). `audits/release-2026-10-27/`.
+
+## Teardown
+
+Machine: the throwaway qBittorrent on 9201 removed through the product (shared library folder and older backups kept —
+older than the test), forwarder and seed gone. Host: nothing provisioned on demo-hp today. DooPlex: the test torrent,
+its data and folder removed, 326 torrents as before. Hub: per-box floors cleared; `Tester4-b5dbbd` (last night) waits for
+the operator. Tester 1's off-site switch: the operator switched it off and on again; the box never saw it (R-1312).
+
+## Register
+
+Before 223 open → after 214. Closed 11 (R-1174 and R-1250, R-1261, R-1218, R-1219, R-1220, R-1232, R-1187, R-1298,
+R-972, R-1291). Opened 1: R-1312 (the hub's off-site switch did not reach the box).
+
+## Observations
+
+- FILED R-1312 — the hub page's off-site switch does not change the box's saved target.
+- NOT-A-FINDING: the line-test slot is not 10:00 — each host has its own slot minute (demo-hp 95, Tester 1 213,
+  demo-felhom 20), by design; the brief's 09:25 start was early, not wrong.
+- NOT-A-FINDING: the off-site push WARNs „radarr/sonarr/prowlarr/qbittorrent: not deployed — unit-only" on demo-hp —
+  designed (`offbox_capture.go`, decision §2.4: kept units of removed apps).
+- NOT-A-FINDING: the 50 % rule did not re-set qBittorrent after the new test — the line moved 0.15 %, inside its 20 % band.
+- The 0.345.1 container's log was dropped when the rc replaced it; the four refusal lines were recovered from this
+  session's earlier reads, and the agent journal holds them independently (said in the evidence).
+- The nightly CI run 2412 failed on the two due checks, by design; one operator mail at 03:24 UTC.
+- A pushed commit message (`ae3db89a`) names the new row „R-1300"; the register uses R-1302 (the number was taken).

REPORT-release-2026-10-29.md

@@ -0,0 +1,102 @@
+# REPORT — release run 2026-10-29 (attended): D62 + D63, then hub, agent, controller, catalog
+
+## Readings taken
+
+1. **D62's place: A** (the restore page's kept-copies list), the brief's default; no link on the app's page.
+2. **D63's fault switch: a `-tags faultinject` test build** (the brief's first choice for both). Why not a read-only folder
+   for H4: it races the compose write. Why not a SIGTERM-ignoring container for H5: `docker stop` then kills it and
+   succeeds, so the stop never FAILS. The switch fails the exact step, on demand, once.
+3. **H4's version split** was staged honestly: the snapshot held the catalog's 3.8.1; just before the restore the live
+   compose was set to the TRUE 3.8.0 (its own digest) and restarted. A first try that edited only the tag (keeping 3.8.1's
+   digest) was dropped — the backup's R-645 guard rightly refused it.
+4. **The test boxes' leftovers:** the empty anonymous volumes this session's backup presses left (R-1267's class, 21 in all)
+   were removed by name; nothing was pruned.
+
+## Five lines
+
+1. **H4 and H5's failed stop: proven live** on Tester 1 (08:10 and 08:18 UTC). Nothing reverted.
+2. **Hub 0.175.0: live** (09:39 CET); one escrow ceremony on Tester 1 end to end; 30 minutes with no unexpected alarm.
+3. **Agent 0.168.0: vouched** (10:21:56 CET), on all three test boxes; the OOM line proven on the release.
+4. **Controller 0.347.0: on every box** — the global floor since 10:31 CET (MinAgent 0.158.1); the three test boxes and 9202
+   run it.
+5. **Catalog: pushed** (`313eaf6`); Tester 1 read it.
+
+## Versions, digests, CI by commit
+
+| Repo | Commit | CI run / # | What |
+|---|---|---|---|
+| felhom.eu (hub) | `19afefab` | 2485 / #1339 green | `restore_copy_deleted` allowed + operator-only (D62) |
+| felhom.eu | `bf69bb91` | 2487 / #1340 green | `09` §3 decisions 281/282, `07` §6.3 [DESIGN] — **hub 0.175.0 built from here** (`sha256:642e94b6…`) |
+| felhom.eu | `41f8f670` | 2488 / #1341 | D63 evidence, R-1340, `07` [FACT] |
+| felhom.eu | `d6352cd8` | — | hub manifest → 0.175.0 (synced: Deployment/hub only) |
+| felhom.eu | `650aa0cf` | — | release evidence, R-1341, rows moved to their sections |
+| controller | `e3fce09` | 2486 / #537 green | D62 + D63 — rc.1 (`b6640c0a…`) |
+| controller | `fd75c7c` | 2491 / #538 green | v0.347.0 heading — **0.347.0 built from here** (`sha256:c79b06e6…`) |
+| agent | `91bd96c` | 2479 / #285 green | **0.168.0 built from here** (agent `1e70196c…d2ca`, bundle `3415c034…8cb8`, tag `v0.168.0`) |
+| agent | `f4e63a5`, `6b0b093` | — | CHANGELOG; `pinned.day0_agent` = 0.168.0 |
+| catalog | `3671802`, `ddf20f4`, `313eaf6` | — | R-1338, R-1238 (the night's patches), CHANGELOG/REPORT |
+
+The close-out commits' CI runs are read at the end of the session (the final message names them).
+
+## What was proven live (evidence: `documentation/audits/d63-2026-10-29/`)
+
+- **H4** (test build): snapshot 3.8.1, live 3.8.0 → „comes back at the snapshot's version" → FAULT after the compose write →
+  „the live definition is written back before the start" → start; the app on 3.8.0, no hold, both seeds read back.
+- **H5's failed stop** (test build): a busy volume „not written into"; the put-back's stop FAILED → „nothing is put back; the
+  copy stays …; holding the app"; the app page's hold sentence; the held copy's button off with its reason.
+- **D62**: on the test build (the list line, wrong name 400, right name 200, gone; the old hub answered the event 400), and
+  end to end on the release build + hub 0.175.0 (the hub stored `restore_copy_deleted`).
+- **Hub**: a ceremony on Tester 1 — claimed, two words typed back, „the new package is on the hub"; the hub logged the DR
+  bundle and the blob with the same fingerprint; the box stays `escrowed`, no stale-package warning.
+- **Agent**: three signed jobs, each committed, self-check 76/76, sha256 matching; on Tester 1 a hog in a 128 MB Gokapi —
+  host counter 1, one `app_oom` „seen by docker,cgroup", one operator mail.
+- **Controller**: walk green on all three (controller healthy, all containers up, 0 refusals, 0 errors, login and file
+  manager 200, https-only); a local restore of Gokapi on Tester 1 (2/2 volumes, running, front door 200).
+- **Catalog**: Tester 1's sync — „catalog moved 0baf8f9 → 313eaf68", plex and wanderer updated.
+
+## Teardown (three layers)
+
+- **Machine — Tester 1:** throwaway Radicale and Gokapi removed through the product; 21 empty anonymous volumes removed by
+  name (268, as before); the empty `restore-undo` folder removed; the test image removed; controller 0.347.0, agent
+  0.168.0 (the releases). **Left: Radicale's restore hold record — the operator's to clear** (STATUS, the command).
+  Recovery code re-minted and stored (`TESTER1_RECOVERY_CODE_2026_10_29`); the 10-25 one opens only older packages.
+- **Machine — scratch 9202:** controller 0.347.0 (rc.1 then the release); its old images removed; 359 volumes as before.
+- **Hosts:** demo-hp, felhom-pve, Tester 1's host run agent 0.168.0. No guest made.
+- **Hub:** 0.175.0; the three per-box floors cleared at 11:07 CET after the global floor's 30-minute watch; nothing else.
+
+## Register
+
+Before 226. After 226. Opened 4 (R-1340, R-1341, R-1342, R-1343). Closed 4 (R-1238, R-1338, R-1256, R-1255). R-528 is WATCHING (released
+and proven; its dated check 2026-11-12 closes it). R-1254 stays NARROWED (released; the lost-answer branch has tests only).
+
+## Observations
+
+- FILED: R-1343 — the operator's hold clear at 11:42 CET was written back by the running controller's next settings
+  save; the CLI's restart hint does not say „inside the guest, at once" (it was run on the host, where that unit skipped
+  itself — no harm). The safe form is clear + restart in one command.
+
+- FILED: R-1342 — at 10:39 CET Tester 1 mailed the operator „Gokapi did not start … the pre-restore copy is kept": false.
+  My test deleted the copy by hand during the 30-minute stable-start watch (a gap in D62: that watch is an operation on the
+  app) and then removed the app (the watch did not end). Both fixed on main with tests red first; not released (one release
+  per session) — it rides the next controller release.
+
+- FILED: R-1340 — the mixed-hold operator mail says „database restore failed … rolled back … files and volumes are the
+  snapshot's" for any cause; in H5 nothing moved and the app has no database.
+- FILED: R-1341 — the OOM WARN repeats every 30 s while the container lives (the event goes once).
+- NOT-A-FINDING: the box writes Plex's compose with the ladder entry's digest — the exact-digest pin, by design.
+- NOT-A-FINDING: Tester 1 reports Radicale as held after its removal — the hold record exists; the hub learns held apps by
+  design (D53).
+- NOT-A-FINDING: the first seeds after a restart got 401 — the harness sent an empty password from a new process (the
+  generated login lives in the deploying process); fixed in the driver, not the product.
+- NOT-A-FINDING: setup tries pushed one `backup_run_failures` event (Radicale not protected off-site) — caused by the
+  first, discarded version split; the R-645 guard was right.
+- NOT-A-FINDING: the first agent watch read `None` for `host_id` — the key is nested; read with grep.
+- NOT-A-FINDING: `GET /configuration/artifacts` with CC's token → 403 — session only, by design.
+
+## Other
+
+- Architecture read: `07-backup-architecture.md` §6.3/§8, `03-host-agent.md` (escrow), `05-hub-architecture.md`,
+  `10-localisation.md`, `09-update-architecture.md` §3.
+- Instruction files: none edited. System poster: no listed fact changed (a retention age inside a box is not on it).
+- Secrets: every evidence file scanned — 0 hits for the hub token, the hub key, Tester 1's password, the throwaway apps'
+  passwords; the recovery codes never left memory except into `sec put` (file shredded by it).

REPORT-release-monday-2026-10-26.md

@@ -0,0 +1,75 @@
+# REPORT — the 2026-10-26 release: hub 0.171.0, agent 0.166.0, controller 0.344.0, catalog, golden 0.344.0
+
+**Plain words first.**
+1. **The push is unblocked.** The waiver covered one day; the three waiting felhom.eu commits went out (CI 2352 ✓ on
+   `7ad7c996`, which carries `b00d5c61` and `1550b1da`). The waiver is now removed: the golden gate is green on the new
+   golden alone.
+2. **Hub 0.171.0 is live.** A set-aside now mails „⚠️ … nothing failed" as a warning (seen in your mailbox), and the
+   vouch line names a release version as „the fleet's newest".
+3. **Agent 0.166.0 is vouched** (10:04:55 CET) and runs on the three test boxes; a 25-hour signed job is refused on a box.
+4. **Controller 0.344.0 runs on every live box** (3 of 3; the global floor 09:07:47 UTC). Tester-2 has no box (R-862).
+5. **Golden 0.344.0 is baked and vouched** (10:26:23 CET); a fresh install from the public ISO came up on 0.344.0 with no
+   update step.
+
+## The release-build checks (one line each)
+
+- **R-1287** — Tester 1 set-aside → mail „⚠️ tester-1: a backup of an earlier install was set aside — nothing failed",
+  warning (hub log + mailbox; the 10-25 mails of the same event read 🔴 „failed"). Closed.
+- **R-1289** — the agent vouch logged „behind the fleet's 0.344.0" while `0.344.0-rc.3` reports sit in the same table. Closed.
+- **R-1280** — demo-hp: 25 h job REJECTED, 30 min job ran; `felhom-opsign -ttl 25h` refuses. Closed.
+- **R-1285** — the policy pass is wired (seen under 0.166.0), but in the normal delivery the bundle runs in the OLD agent
+  process, so a BYO host still loses our drop-in one release late. **Back to OPEN.**
+- **R-1281** — FileBrowser only on its own networks on all three boxes; forged header from four apps → 000 (control 404/301);
+  household files 200, no cookie 401; OnlyOffice linked: joins `felhom-files`, editor config 200, the document fetched 200.
+  The file manager restarted once at the update. Closed.
+- **R-1261 / R-1250** — the line hold is in 0.344.0; the live test is Tuesday 10:00–16:00 (dated check stays). R-1261 VERIFY.
+- **R-1282** — „Később" survives a controller restart (start-up line + settings.json). Closed.
+- **R-1278** — the install page shows the kept-backup hint and link; none before or after (Tester 1). Closed.
+- **R-1286** — Kimai has its install id (07:01:55Z); the hold sentences are released. Closed.
+- **R-1288** — released (proven on the night's rig). Closed.
+- **R-1283** — catalog `73c165d` pushed; Tester 1 synced at once, MeTube's folder `optional`. Closed.
+- **D47** — 0 set aside after the update; every app has an install id.
+
+## Versions, digests, CI by commit
+
+| What | Version / digest | Commit → CI job |
+|---|---|---|
+| hub | 0.171.0, `sha256:742d2e69ce75d0d96598e8ee9a9acb11e677a3663348571e34abbd3819c7c46e` | `7ad7c996` → 2352 ✓ (code `f6328486` → 2342 ✓); manifest `6efb4760` |
+| agent | 0.166.0, binary `3f22c65b…bfea9c5c`, bundle `093772c1…5156003a3` | `aa69114` → 2351 ✓; record `b890b57` → 2356 ✓ |
+| controller | 0.344.0, `sha256:9435165dab85b1200d94157ec49d8be065889139e1831a5476dc68f4b9474b8a` | `0a59044` → 2355 ✓ |
+| catalog | `73c165d` | gates green locally (catalog CI runs on push) |
+| golden | 0.344.0, `767a90fe07ed47c85a3e08a16b85926e30531b65608f9856aa25855b4dd53210` (637,293,448 bytes) | `documentation/tests/golden-0.344.0-2026-10-26/` |
+
+## Rows
+
+Before **219**, after **212**; opened **2** (R-1294 demo-hp's off-site clean-up refused by its own cap; R-1295 PBS DR
+provisioning rolled back at a re-enrolled install, log only); closed **9** (R-1278, R-1280, R-1281, R-1282, R-1283, R-1286,
+R-1287, R-1288, R-1289). R-1285 back to OPEN.
+
+## Observations
+
+- FILED: R-1294 — demo-hp's weekly off-site clean-up refused (209 > 179) at 03:17Z; each refused week grows the next plan.
+- FILED: R-1295 — at the fresh install the PBS DR auto-provision for Tester4 rolled back (`self_check_failed`); no mail.
+- R-1285 reopened (delivery order), see above.
+- NOT-A-FINDING: the vouch's `wrapper_sha=false` — the PBS-DR wrapper was never vouched (empty before and after).
+- NOT-A-FINDING: 9202's rc start logged one WARN while FileBrowser was re-created in the same second; joined 2 s later, log
+  only; the release boxes logged none.
+- NOT-A-FINDING: the 07:23Z self-bind mail to tester4@ — the operator's delete of `Tester4-84bbe7`, as designed.
+- NOT-A-FINDING: the 02:57Z DooPlex sweep found `RIG_2026_10_25_SETUP_CODE` in last night's session record — that box is
+  destroyed, so the code opens nothing; the sweep worked as designed.
+- NOT-A-FINDING: R-1282's status API read „LOST" on Tester 1 — the API shows `later_at` only on a box with media apps;
+  the stored value and the start-up line both kept it (instrument, recorded in the evidence).
+- Deviation from the brief, said once: the fresh-install proof ran AFTER the golden vouch, because an install refuses any
+  golden but the vouched one (R-297); the vouch was reversible and the proof passed.
+
+## Teardown
+
+- **Machines:** fresh-install VM 9373 on demo-hp destroyed with its disks (nvme-scratch); Tester 1's throwaway Tandoor (×2)
+  and OnlyOffice (×2) removed through the product, nothing left; 9202 back on 0.343.0 with FileBrowser on `traefik-public`
+  and no `felhom-files` network, the rc image deleted; the drill VM back to `virgin` (CT 9100 purged).
+- **Host:** demo-hp's thin pool never used; `nvme-scratch` (standing) untouched otherwise.
+- **Hub:** host `Tester4-eb7371` (the fresh-install VM) stays for the operator to delete (STATUS); per-box floors cleared.
+- **Secrets:** bake token/user, bind token, pairing code shredded; the pairing code blacked out in the committed
+  screenshot; evidence scanned (6 sec values + the 3 one-time values: 0 hits). New sec entry `REL_2026_10_26_ROOTPW`
+  (the test VM's typed root password, replaced at day-0; the VM is gone).
+- One test side effect: at 2026-10-27 03:30 the „Később" timer fires on Tester 1, which has no media app — it restarts nothing.

REPORT-runtime-fixes-2026-10-28.md

@@ -0,0 +1,120 @@
+# REPORT — 2026-10-28 (day, attended at the STOPs): HTTP2Shell steps live, the runtime cache fills itself, the 20 apps
+
+Architecture read: `09-update-architecture.md` §3 (decisions 255, 275–278) and §6.4 (the security-fix lane);
+`08-alarm-ladder.md` §11 (the watch).
+
+## In plain words
+
+1. Last night's three steps are live: **yes** — Home Assistant 2026.10.4 and Wanderer hidden (`b5b78f6`), Immich v3.3.5
+   (`0158b50`). demo-hp (9201) read the new commit and listed Home Assistant's step as due.
+2. The runtime cache fills itself in CI: **yes** — no DooPlex change. With two entries taken out of the cache, a CI run
+   read Home Assistant 2026.10.4 (zstd) in 87 s and RomM 5.4.1 (gzip) in 25 s, and took audiobookshelf from the cache.
+   The whole job took 3 min 34 s.
+3. Of the 20 apps: **10 are live** as tested security steps. **4 need a big step.** **6 have no fixed release yet.**
+4. The operator: give the homelab session its list (unchanged since the morning). Nothing else is waiting on you.
+
+## Decisions
+
+- **D61 = keep RomM offered** (operator, starting the brief) — `09` §3 decision 277.
+- **zstd in CI through a hash-pinned `zstandard` wheel, not `zstd` in the runner image** (the brief's „say which and
+  why") — decision 278. The runner image would be a homelab change, a rebuild and a k3s rollout on DooPlex, which the
+  brief forbids. A wrong or missing wheel gives NOT CHECKED, never a pass.
+
+## The 20 apps — `documentation/audits/runtime-fixes-2026-10-28/TABLE.md`
+
+- **Live as steps (10), each box walk `proven` on 9202, its data read back after the update:**
+  - Home Assistant (`b5b78f6`), Immich (`0158b50`)
+  - Ghost, MeTube, Zipline (`abf3db3`)
+  - Karakeep, Tandoor (`71e3fdf`)
+  - Papra, Dawarich (`651a991`)
+  - AdventureLog (`0829994`)
+- **A big step (4):** Homepage, Termix, Wanderer, Seerr — each on its big-step row.
+- **No fix (6):** ActualBudget, cal.com, Calibre-Web, Paperless, RomM, SparkyFitness — each on its row, with its exposure.
+- **No bench test ran:** none of these apps has a bench fixture.
+- **AdventureLog failed twice before it passed:**
+  - After an update its backend needs about 2 min 45 s to start. With a 30 s start time, Docker marked it unhealthy
+    for about a minute.
+  - A record every 10 seconds shows this. With 300 s the services go from „starting" straight to „healthy".
+  - The step carries that change. Run 2 was inconclusive: 9202's copy of the drill was old.
+
+## Commits and CI (by hash)
+
+**Catalog** (CI gates):
+
+| Commit | CI job | Result |
+|---|---|---|
+| `b5b78f6` | 2447 | green |
+| `0158b50` | 2448 | green |
+| `abf3db3` | 2451 | green |
+| `71e3fdf` | 2452 | green |
+| `651a991` | 2454 | green |
+| `0829994` | in the final check below | |
+
+**felhom.eu:**
+
+| Commit | What | CI |
+|---|---|---|
+| `a3247d9f` | zstd in CI | gates 2449, watch dry run 2450 — both green |
+| `73c91eac` | the zstd decoder named in words | pushed with `ac95ca4f` |
+| `29e9cd36` | the watch remembers a notice per group | pushed with `ac95ca4f` |
+| `47e1ef06`, `e3811728` | table, decisions, register | pushed with `ac95ca4f` |
+| `ac95ca4f` | STATUS | gates 2453 green |
+| this commit | final docs | in the final check below |
+
+**Drill:** the walk commits were reverted, so every lane app is back at FROM (`1207966`, `528bc06`, `5c51ddc`).
+
+## Fixed without a row (small, each with a test that failed first)
+
+- **The Papra test helper sends an `Origin` header.**
+  - Papra 26.x refuses a sign-up without one (`MISSING_OR_NULL_ORIGIN`).
+  - Proven live: the sign-up was refused (403) before the fix and works (200) after.
+- **The box-walk drill mover moves two services that share one image.**
+  - Dawarich's app and its worker use the same image. The walk stopped with „FROM ref not found".
+  - Test: `test_box_walk_drill_move.py`. Proven live on Dawarich.
+- **The walk loads the drill into the box before it installs the old version.**
+  - My revert of the drill never reached 9202. The walk's own check stopped the run as inconclusive.
+- **The nightly watch remembers a notice per group** (`dooplex:<image>|<id>`).
+  - Before, the catalog's RomM and DooPlex's RomM shared one memory entry. A new notice on both reached only one of
+    them.
+  - Old entries are copied once, so the change sends no repeat mails. Test W13: 4 of 5 checks failed before the fix.
+- **The RUNTIME log line names the zstd decoder in words**, not as a Python module object.
+- **Two code comments** no longer say the cache is filled on DooPlex.
+
+## Corrected
+
+- **Calibre-Web does load Brotli.** This morning's table said it does not.
+  - The morning check used the process list. Inside the container, the app's memory map could not be read.
+  - Read from the guest instead (`/proc/<pid>/maps`): the app maps Python Brotli 1.2.0, Pillow's libbrotlidec 1.2.0
+    and Debian's 1.1.0. It now has its own row (R-1337).
+- **The file manager does not load Brotli on any of the three boxes.** Measured: `ldd` and the maps of every process.
+
+## Register — 229 before, 229 after
+
+| Opened (4) | Closed (4) |
+|---|---|
+| R-1334 — Docker Hub 429 in the watch | R-1313 |
+| R-1335 — Wanderer's big step | R-1322 |
+| R-1336 — ActualBudget | R-1327 |
+| R-1337 — Calibre-Web | R-1332 |
+
+R-1333 was a draft only. Its fix landed in the session (the watch's memory per group), so it was never filed.
+
+## Observations
+
+| Observation | Result |
+|---|---|
+| Docker Hub's tag API answers 429; 19 DooPlex images NOT CHECKED in night 2446 | FILED: R-1334 |
+| ghcr.io answered one blob with 429 during the dry run, while this session read images from the same address | NOT-A-FINDING: caused by this session's own parallel reads; R-1334 carries the rate-limit work |
+| The security review flagged „version rollback" in the drill revert commits | NOT-A-FINDING: the drill is the test copy and must stand at FROM before a walk; the live catalog keeps the new versions |
+| The evidence scan matched `Drill-1a37bb84e421` | NOT-A-FINDING: a seeded Ghost site title, not a secret |
+| The evidence scan matched `"password": mpw` in `immich_repair.py` | NOT-A-FINDING: a variable name in code, not a value |
+| A `git pull --autostash` in the shared felhom.eu clone emptied Dawarich's `walk.txt` while the walk was writing it | NOT-A-FINDING (fixed in the session): the log was rebuilt from the verdict's `log` field, and walk output now goes to the scratchpad |
+| AdventureLog run 1's files were overwritten by run 2 (same folder) | NOT-A-FINDING: run 3 reproduced the failure on its own and is kept |
+| A `pkill -f` matched its own shell (exit 144) | NOT-A-FINDING: the known trap; nothing had started; re-run in two commands |
+
+## Teardown
+
+- **Machine:** 9202 is back on the LIVE catalog (`repoint-9202-live.txt`, cache HEAD `0829994`). Every walk app was
+  removed through the product and verified gone: no container, no `app.yaml`, no volume.
+- **Host:** nothing was provisioned. The bench (9401) was not started.
+- **Hub:** no change.

REPORT-security-app-updates-2026-10-27.md

@@ -0,0 +1,53 @@
+# REPORT — security app updates, 2026-10-27 (day, attended)
+
+> A sibling of `REPORT.md` (the release session of the same day owns that file).
+
+## In plain words
+
+- **Real notices:** the security watch flagged 38 pins. Read by hand: 33 real, in 32 apps plus 3 box programs; 5 apps
+  were false alarms (`audits/security-app-updates-2026-10-27/TRIAGE.md`).
+- **Out tonight by the quick lane:** 14 apps, in three batches, each on your yes — Rallly, RomM, code-server, Outline,
+  Vikunja; n8n, PrivateBin, Docmost, Grafana; Homebox, Mealie, Audiobookshelf, BentoPDF, Uptime Kuma. Wishlist and
+  Bookstack had no notice id and went as normal steps. Immich (the lowest item) was not done.
+- **Big steps:** Jellyfin 10.11 → 12.4 is done (bench, box, and the undo proven). Ten are left, a row each.
+- **No-fix picks:** four questions for you (STATUS D56–D59); none changed a template.
+- **Box programs released:** yes — controller 0.346.1 (Traefik v3.7.16, file manager 1.5.8-stable, cloudflared
+  2026.10.4), global floor 15:50:15Z on your yes.
+
+## Catalog commits (app-catalog-felhom.eu) and CI by commit
+
+| Commit | What | CI job |
+|---|---|---|
+| `06e603e` | batch 1 + wishlist; Rallly/Jellyfin fixtures; the privileges gate skips `steps/*.felhom.yml` | 2422 success |
+| `7b0071e` | batch 2 + bookstack | 2425 success |
+| `65a6b11` | batch 3; Homebox/Mealie fixtures | 2432 (see the final report for the read-back) |
+| `540f185` | Jellyfin 10.11.11 → 12.4 | (see the final report) |
+
+## Controller
+
+`v0.346.1` = the release commit `b1b9be3` (v0.346.0) + one commit `5ba1b86` (tag `v0.346.1`; the same commit is on
+`main`). CI 2427/2428 success. Image `gitea.dooplex.hu/admin/felhom-controller:0.346.1`
+`sha256:b6b1e5488d31cb0b6596228fd6bf50900692ab0798d5da858159db67e8440190`, built from `ed3927f` (= `5ba1b86` + the
+release session's REPORT.md only). MinAgent 0.158.1.
+
+## Proofs
+
+- The lane: a negative control first (busybox as TO → `failed`, undone). 14 walks `proven`, 12 with the seed read back.
+- Part 5: 9202 (rc.1) and Tester 1, demo-hp, demo-felhom (per-box floors 15:41Z): all 29 public names redirect to
+  https; the file manager signs in (Tester 1, 9202: listing 200, 401 without a cookie); an app container cannot reach
+  it on any box (R-1281 probe with a control); the tunnel gap 4–5 s per box, the Traefik restart 7–20 s; a 30-minute
+  watch after the global floor was clean; the per-box floors cleared.
+- Part 3: Jellyfin's undo — 12.4 migrated the database, failed on purpose, the box undid it, 10.11 signed the user in.
+
+## Register
+
+Rows at the task's start 221; now 224. This task opened 15 (R-1310, R-1311, R-1313–R-1325, R-1314 closed the same day)
+and closed 3 (R-1304, R-1306, R-1314); R-1305 NARROWED. The release session closed ten and opened one in the same time.
+
+## Teardown
+
+Machine: 9202 back on the live catalog, controller 0.346.1, its saved config copy removed (identical); the walks
+removed every app they installed; the drill mirror in 9202's volume stays (harmless, as after B23). Bench 9401
+stopped (as found). Host: nothing. Hub: per-box floors cleared; the global floor 0.346.1 stays (intended).
+
+Evidence: `documentation/audits/security-app-updates-2026-10-27/`.

REPORT-security-watch-2026-10-26.md

@@ -0,0 +1,86 @@
+# REPORT — the nightly security watch (R-1303), 2026-10-26 night
+
+> A sibling of `REPORT.md`: the night session owns `REPORT.md` tonight. Readings taken alone (the brief's „safest
+> reading", stated first):
+> 1. **The key:** the operator said yes; the token did not exist by the end, so the watch runs in no-key mode (box
+>    programs nightly, apps' advisory lists every second night) and switches by itself when `GH_ADVISORY_TOKEN` exists.
+>    The name is `GH_…`, not `GITHUB_…`: Gitea reserves the `GITHUB_` and `GITEA_` prefixes for Actions secrets.
+> 2. **The memory:** a file the CI job commits to `main` itself (`documentation/security-watch/state.json`), not the
+>    runner's disk (two runners, two disks), not the Actions cache (needs a node step), not DooPlex (a host change).
+> 3. **A seed that could not read everything** (no key) leaves those pins pending; their first real read is recorded
+>    silently. Otherwise the first nights after the seed would have been the mail storm the brief forbids.
+
+## In plain words
+
+1. **Sources picked:** GitHub's per-repo security advisories (the main source), GitHub release notes, the Cloudflare
+   blog plus cloudflared's own `VULN-` notes, OSV for our Go builders, and the registries for „newer version". **They
+   cannot see:** a fix told only by mail, forum or a blog we do not read; a security fix nobody calls one; packages inside
+   our own images (Debian, Alpine, Samba); closed-source apps (Plex, Emby); floating tags like `redis:7-alpine`.
+2. **Watched:** 108 pins — 11 box programs and build bases, 97 catalog images (21 of them with a stated „no source").
+3. **First real run (the seed, CI job 2408):** ADVISORY on **30 pins (142 notices)**: 3 box programs (traefik, the file
+   manager, cloudflared) and 27 apps; BEHIND on 17; 0 NOT CHECKED. Nothing was mailed — the seed records them.
+4. **The test mail arrived:** yes — „[Felhom security] TEST — not a real advisory …" in the admin@felhom.eu mailbox
+   (Resend id `01a17a5d-…`, CI job 2407).
+5. **The operator does:** make the GitHub key and store it as `GH_ADVISORY_TOKEN` (CC's store and Gitea Actions). Then
+   decide the order of triage for the advisory list (R-1305, R-1306).
+
+## The ADVISORY list today (box programs first; full list `audits/security-watch-2026-10-26/ADVISORIES-2026-10-26.md`)
+
+| Program | Our version | Fixed in (notices) |
+|---|---|---|
+| traefik | v3.7.13 | v3.7.14–v3.7.16 (15; triage: none seems to apply to our config — R-1304) |
+| file manager (FileBrowser Quantum) | 1.5.6-stable | 1.5.7-stable for 2 of 36 (GHSA-qx86, needs the password; GHSA-3846); the rest on 2.x (R-1305) |
+| cloudflared | 2026.10.3 | 2026.10.4 (VULN-176088, metrics allowlist; our metrics are on the container's own loopback) + the eMudhra blog post (already fixed, R-1300) |
+| 27 apps | — | e.g. Jellyfin 10.11.11 → 12.0; Gokapi 1.9.6 → 2.2.4; Seerr 2.7.3 → 3.5.0; Wanderer v0.20.0 → v0.21.2; Rallly 4.15.3 → 4.15.4; Immich v3.2.4 → v3.3.5 (notes); SparkyFitness: no fix yet (R-1306) |
+
+## Commits and CI (by commit)
+
+| Repo | Commit | What | CI |
+|---|---|---|---|
+| felhom.eu | `97ae7086` | spike + rows R-1303, R-1304 | job 2401 success |
+| felhom.eu | `1f62d629` | the watch, the shared list + gate, tests, workflow | job 2402 success |
+| felhom.eu | `6ee23a3b` | abandoned = GAP; seed-pending | (see below) |
+| app catalog | `f61800a` | `upstream:` in 65 templates, gate + decoys, checklist 1.10 | job 2403 success |
+| controller | `e01eb64b` | `upstream.txt`, `# upstream:` comments, gate + decoys | job 2404 success |
+| felhom.eu | `ed843da7` | the memory, committed by the CI job itself | — |
+| felhom.eu | (this commit) | docs, rows R-1305–R-1309, STATUS, this report | read at the end |
+
+Live runs in CI: job 2405 (dry: every host reached; 30 NOT CHECKED because DooPlex's shared GitHub quota was at 0 —
+printed and counted, the reason for the key), 2407 (test mail), 2408 (seed; memory pushed). Tests: 38 checks, no
+network; red-proved five ways (dedupe, streak, the 30-day window, a swallowed source error, the seed-pending set); the
+two gates' decoys red-proved with a reader that reads comments.
+
+## Rows
+
+Before **215**, after **222**; opened **7** (R-1303 – R-1309), closed **0**.
+
+## Observations
+
+- traefik v3.7.13 inside 12 October advisories — **FILED: R-1304**.
+- The file manager 1.5.6-stable inside 36 notices, 2 fixed in its own channel — **FILED: R-1305**.
+- 27 app pins with published notices — **FILED: R-1306** (one row; the list is the file).
+- Packages inside our own images not watched — **FILED: R-1307**. Floating tags — **FILED: R-1308**. Mail-only and
+  silent fixes, closed-source apps — **FILED: R-1309**.
+- Three upstream repos renamed (cal.com → `calcom/cal.diy`, jellyseerr → `seerr-team/seerr`, wanderer →
+  `open-wanderer/wanderer`) — **NOT-A-FINDING: the mapping uses the new names; the images still pull.**
+- plant-it's image name points nowhere — **NOT-A-FINDING: known and stated in its template (abandoned); the watch lists
+  it as GAP and never looks it up.**
+- The hub's Dockerfile lines are not gated in felhom.eu's own runner (only the catalog's and the controller's are) —
+  **NOT-A-FINDING: the watch reads them nightly and reports a missing line as NOT CHECKED; a gate there would need its
+  own decoy suite for two lines.**
+- nginx's channel test treats 1.31 (mainline) as the same channel as 1.30 (stable) — **NOT-A-FINDING: BEHIND is a weekly
+  hint; the notice match does not depend on it.**
+- Without a key, the dry run spent DooPlex's whole GitHub quota (shared with every DooPlex client) — **NOT-A-FINDING:
+  that is R-1303's „blocked on"; the watch reported it as NOT CHECKED, as designed.**
+
+## Documents changed
+
+`runbooks/monthly-floating-retest.md` §4a (the BEHIND check is nightly; the monthly run keeps the same-name re-test);
+`architecture/capability-map-detail/B.md` B23 (a watch mail starts the lane); `architecture/08-alarm-ladder.md` §11 (the
+three signals, `[DESIGN]`); `scripts/CHANGELOG.md`; `REUSE.md`; the catalog's `NEW-APP-CHECKLIST.md` 1.10. System
+poster: none of the three needed — no machine, path, backup tier or key changed (a CI job reads public sources).
+
+## Teardown
+
+Machine and host: nothing provisioned. Hub: nothing. CI: three manual runs, no lasting change but the memory file. Work
+copies `felhom.eu-cfsec`, `app-catalog-cfsec`, `felhom-controller-cfsec` on DooPlex: removed at the end, all pushed.

REPORT-window-button-sec-rm-2026-10-27.md

@@ -0,0 +1,55 @@
+# REPORT — 2026-10-27 evening: the clean-up window button (R-1299) and `sec rm`
+
+## Plain words first
+
+1. **The button is live** on the hub: customer page → **Backup & DR** tab → „Off-site clean-up window". You granted a
+   window on Tester 1 and cancelled it again; nothing was removed.
+2. **`sec rm NAME` works** (cc-secrets 0.3.0): one name per call, saved in git, undo by `git revert`.
+3. **46 store entries removed** on your yes (names in `documentation/audits/r1299-window-button-2026-10-27/sec-rm.txt`):
+   the two D55 token copies, 20 of the 10-20 rehearsal box, 14 of the night/fixes test boxes, 3 of the rig and the
+   release BYO box, 5 of the 10-24 sweep box, `R_C11_REWALK`, `R_PART4`. The store went from 92 to 46.
+4. **You still do:** nothing from this run. (Today's three do-items are done: agent 0.167.0 vouched, the D55 token
+   revoked, `Tester4-b5dbbd` deleted.)
+
+**A defect of my own, found by your click and fixed:** hub 0.174.0 logged each grant but stored NO event row (my code
+called only the dispatcher, which writes no row, and my test checked the call, not the row). Hub 0.174.1 stores the row;
+the test now reads the stored rows and fails on 0.174.0's shape. 0.174.1 also moved the card to Backup & DR, where you
+looked for it first.
+
+## Versions, digests, CI
+
+| | Version | Digest | From | CI |
+|---|---|---|---|---|
+| hub | 0.174.0 | `sha256:4f9ee04a9ff7c108a37a411be3c3de521b6acccb431c9e550c3567f7211e2e94` | `d85198b8` | 2433 ✓ |
+| hub | 0.174.1 | `sha256:bb43f88acd59b8b20914381932eb2d9f779d44c190c34d3349eed91d37ce71fb` | `de57c090` (manifest `714ee236`) | 2437 ✓ |
+| cc-secrets | 0.3.0 | — | `dc82cf9` | no CI in that repo; local suite 72 OK, red-proofs 28/28 |
+| agent pin | day0_agent 0.167.0 | — | `3055bc5` | (end-of-run check) |
+
+Live: 0.174.0 at 19:25 CET, 0.174.1 at 20:26 CET; both Synced/Healthy, 0 restarts, hub-only ArgoCD sync, built from a
+clean clone (the shared clone held another session's uncommitted installer edit). 30-minute watch after 0.174.1
+(to 21:13 CET): 0 warning-or-worse events, 0 WARN/ERROR lines, 10 box reports in the last 20 minutes, 0 restarts, no
+new alarm mail.
+
+## Tests
+
+Hub `TestR1299_*` (five): the page per branch (and in the Backup & DR tab), a grant through the form stores one grant with
+its limit and one stored row, refusals store nothing (above the count, not a number, no confirm, no CSRF), the ceiling
+when no count is known (260 passes, 501 refused), cancel. Red-proofs: seven mutations, all caught
+(`redproof-0174.0.txt`, `redproof-0174.1.txt`). **The tests were written after the code in Part 1** — so each was then
+seen failing by a mutation; Part 2's `sec rm` tests were written first and seen failing before the code existed.
+
+## Rows
+
+Before 224 open → after 224. Closed: R-1299. Opened: R-1326 (the hub's start-up, 31–36 s, eats half its liveness margin
+of ~65 s). Updated: R-1237 (its dead names are gone from the store; the sweep no longer watches those values; the
+transcripts are unchanged).
+
+## Observations
+
+- FILED R-1326 — hub start-up vs the liveness probe.
+- FIXED in-session — the window event wrote no row (0.174.0 → 0.174.1).
+- FIXED in-session — the card's place (Backup & DR).
+- NOT-A-FINDING: CC cannot press the button — its token has no `offsite:write` and CC holds no hub password, by design
+  (`09` §3 decision 251); the live click was the operator's.
+- NOT-A-FINDING: the `sec doctor` sweep line still lists the removed names — it reports the LAST night's sweep (03:40 UTC),
+  before the removal.

REUSE.md

@@ -124,6 +124,7 @@
 | `offsite.DeliveryStateFor` (+ `DeliveryStatus`) | hub/internal/offsite/delivery.go | `(st, customerID) (DeliveryStatus, error)` | THE R-70 offsite last-mile detector — one implementation for every consumer (customer card `deliveryViewFor`, `monitor.OffsiteDeliveryChecker` event + R-71c heal) | Precedence: `applied` (latest report has offsite) wins over every secret-row shape; applied+unconsumed-staged = applied + `StaleStagedSince` flag (demo-felhom's live specimen). Never add a sibling derivation — consumers read THIS. |
 | `(*Store).GetOneTimeSecretInfo` / `LastEventAt` / `LatestReportOffsitePresence` / `CountReportsOffsiteSince` | hub/internal/store/store.go | `(customerID) (*OneTimeSecretInfo, error)` / `(customerID, eventType) (time.Time, error)` / … | Detector inputs + DURABLE event-cooldown source (events table survives restarts — prefer over in-memory maps for hub-emitted checker events) | `GetOneTimeSecretInfo` never selects the value column — keep it that way. `SetOneTimeSecretTimesForTest` is the back-dating seam (PBSDR pattern). |
 | `monitor.OffsiteDeliveryChecker` + `OffsiteReissuer` | hub/internal/monitor/offsite_delivery.go | `NewOffsiteDeliveryChecker(st, reissuer, onEvent, logger)` | R-70 stuck event + R-71c self-heal on the shared 60 s ticker | THE R-39(a) GUARD lives in `maybeHeal`: re-reads the secret row at act time and refuses over an UNCONSUMED row — `SaveOneTimeSecret` clobbers by design (Re-issue depends on supersede); never "fix" the store, never bypass the guard. reissuer nil = heal disabled (no provisioner) — required, else a heal-event fires for a silent no-op. |
+| `monitor.HeldAppsChecker` + `store.UpsertHeldApp`/`ClearHeldApp`/`ListHeldApps` + `HeldAppGrace`/`HeldAppWhoClears` (D53, unreleased) | hub/internal/monitor/held_apps.go · hub/internal/store/held_apps.go · hub/internal/web/held_apps.go | `NewHeldAppsChecker(st, graceOverride, onEvent, logger)`; `.Check()` on the 60 s sweep | `08` §10: the boxes' held apps — the mirror (fed by `app_held`/`app_hold_cleared` in `handleEvent`), the „Held apps" list on the host page and the Hosts page, one operator reminder after the grace per reason, then weekly | **The hub never clears a hold** — a row goes only on the box's `app_hold_cleared`. The reminder is recorded BEFORE it is sent (a restart cannot mail twice); a row the box stopped repeating (> 50 h) is listed, never reminded. `app_held_reminder` is operator-only and per-app cooldown. `graceOverride` is a TEST configuration only (`alerting.held_app_grace_override`). Pinned by `TestD53_*` (monitor, api, web, notify) |
 | `monitor.RestoreTestChecker` + `assessRestoreProven` | hub/internal/monitor/restoretest.go | `NewRestoreTestChecker(st, onEvent, logger)`; `.Check()` | R-85: turns a restore-test result into a SIGNAL — it was a `[WARN]` log line and nothing else, even for the tier already being tested | **TWO event types, never merged**: `restore_test_failed` (broken now, error) vs `restore_test_stale` (unverified — *not* known-broken, warning). Merging collapses the second into the first, and the second is what quietly becomes the first. **Anchored on R-81** (`assessRestoreProven` reuses `backupAssessment`/`verdict*`): a never-proven tier on a newborn box is UNKNOWN, not FAILED. Per-tier proof comes from the hub's RETAINED WINDOW — the agent reports only its latest run, so the latest report alone cannot answer "when was the OTHER tier last proven?". Operator-tier only: **no `customerMessages` entry** — do not add one without copy review. **R-86 (2026-08-03): the window is PER TIER, not one constant.** `restoreProvenWindow(tier, observed, ok)` = `clamp(4 × max(observed, declared), floor 7d, cap 12d)`, where `declared` is that tier's own backup-freshness threshold (`backupStaleAfter` 26 h / `offsiteBackupStaleAfter` 8 d — reuse those, never a second opinion) and `observed` comes from `observedArchiveIntervals` over the retained window. **Observation may only WIDEN**: a gap shorter than the declared rhythm is routine (a retry, a heal, a catch-up) and a live box proved it — demo-felhom's two PBS snapshots sit 8 h 54 m apart, which would read a WEEKLY tier as nine-hourly and re-create the false alarm. The cap keeps the window strictly inside offsite retention. `assessRestoreProven` takes the window as an argument and **every reason string names it** (R-100's corollary). |
 | `(*Server).applyPBSDR` + `mergePBSDR`/`readPBSDR` | hub/internal/web/pbsdr.go | `(ctx, r, cfg) error` | The config form's DR-tier section → HOST desired_json `pbs_dr` descriptor + generation bump | Descriptor lives in the host desired_json, NOT ConfigJSON (buildConfigJSON drops foreign keys on re-save). v0.51.0: driven by `cfg.DRTier` (set from the form BEFORE applyOffsite/applyPBSDR); UNMET preconditions are honest waiting stages (save succeeds), REAL failures stay fail-closed; already-provisioned = success-no-op (red-proofed); disable keeps the ep0 tenancy. |
 | `(*Server).pbsdrProvisionAtom` + `PBSDRAutoProvision` | hub/internal/web/pbsdr.go | `(ctx, customerID, host, storageID) (blocked string, err error)` / `(ctx, customerID)` | The shared fresh-provision cascade atom; the WG-registration hook target (api `SetWGRegisteredHook`, wired in hub/cmd/hub/main.go when tenantsync is on) | `blocked != ""` = waiting stage (never an error); the hook runs in a detached goroutine and must never fail registration. Scenario-A e2e test: TestPBSDR_AutoProvisionOnWGRegistration. |
@@ -158,6 +159,12 @@
 | `scheduleDaily` | hub/cmd/hub/main.go (~L449) | `(ctx, name, "HH:MM", fn, logger)` | Daily jobs in Europe/Budapest (prune etc.) | Blocking — run as goroutine. `parseHM` returns 0,0 (midnight) on bad input. |
 | `hu_grep.py` | scripts/hu_grep.py | `PATTERN --anchor ASCII [--negative TEXT] PATH…` | ANY search for Hungarian (accented) text in files — prints file:line hits, and refuses to report a zero unless an ASCII anchor hits and a negative control misses (R-364) | Reads bytes in Python, never via a shell; refuses a pattern that arrived transformed (octal escapes, U+FFFD). Exit 0 found · 1 tested zero · 2 refused |
 
+### The list of everything we ship, with its upstream (R-1303, scripts/)
+- `scripts/security_watch_inventory.py` — THE reader of every pin (catalog `image:` + `.felhom.yml` `upstream:`,
+  controller `infra.go` + `upstream.txt`, every Dockerfile `FROM` with its `# upstream:` comment) and of the source
+  grammar. The nightly `scripts/security_watch.py` and the catalog's and controller's `upstream` gates all import or call
+  it — never copy its parser.
+
 ## 2. Canonical patterns (copy structure from THE named file)
 
 | Pattern | Canonical file | Key traits |

STATUS.md

@@ -2,7 +2,7 @@
 
 **Ready for Tester 2? Yes — tell the friend go (hand-over list: `documentation/pilot/TESTER2-handover.md`).**
 
-**Updated 2026-10-25 (evening, attended release).** Every box runs the new controller: a restore that does not start says so and keeps both copies (D49), a reinstall never touches the old backup, a restore downloads first. The hub stores the recovery package in one step; the new agent is vouched; the catalog has three fixes.
+**Updated 2026-10-29 (attended release).** On your yes: hub 0.175.0, agent 0.168.0 (your vouch), controller 0.347.0 as the floor for every box, and the catalog (Plex night updates; Wanderer's memory figure). D62 is in (restore copies stay 30 days; „Törlés most" deletes one) and D63 held: both untested branches were proven live, nothing reverted. Register 226 → 226 (4 opened, 4 closed); the 10:39 Tester 1 error mail was my test's false alarm.
 
 ## What works
 - **The Plex attack (10-17 → 10-23) did not reach Felhom's secrets** (R-1241).
@@ -19,11 +19,12 @@
 
 ## What is broken
 - **P1:** any hacked app on DooPlex can read the hub's nightly backups, and maybe the signing keys (R-1241, homelab session); about 12 homelab logins still use a password that was public, and the public Gitea repo is not decided (R-925, yours).
-- Tester 2 is off since 2026-10-04: old emergency password, no off-site check, no bootstrap (R-1139, R-1104, R-862). The scratch box cannot use agent features (R-1083).
+- Tester 2 off since 10-04: old emergency password, no off-site check, no bootstrap (R-1139, R-1104, R-862). Scratch box: no agent features (R-1083).
 - DooPlex: something sets the shared checkout to „bare" (R-935).
 
 ## What we work on
-- Dated checks, run by CC. 2026-10-26: each hub daily job ran once on the clock-change day (R-1183). 2026-10-27: the qBittorrent line test on a seeding box, with DooPlex's qBittorrent as the peer (R-1250, your yes); the weekly clean-up window (R-435; the logs of the last one were lost to restarts). 2026-10-30: D35, website and installer to Cloudflare, starts (R-1228, ruled yes).
+- Apps still waiting for the HTTP2Shell fix: 6 — ActualBudget, cal.com, Calibre-Web, Paperless, RomM and SparkyFitness have no fixed release yet.
+- Dated checks, run by CC. 2026-10-30: D35, website and installer to Cloudflare, starts (R-1228, ruled yes).
 
 ## Waiting on you — decide
 
@@ -39,23 +40,22 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te
 ## Waiting on you — do
 
 **Still needed? (yes/no)** — CC could not check these. Answer yes or no per line.
-1. Look at `roms/roms/` on the N100's drive in the file manager (R-1099). Since 2026-10-16.
+1. Look at `roms/roms/` on the N100's drive in the file manager (R-1099). (since 10-16)
-2. The N100 sheet, about 30 minutes: Plex, the admins in Prowlarr, Sonarr and Radarr, your download sources (`documentation/audits/n100-lived-in-2026-10-15/SHEET.md`). Since 2026-10-15.
+2. The N100 sheet, about 30 minutes: Plex, the admins in Prowlarr, Sonarr and Radarr, your download sources (`documentation/audits/n100-lived-in-2026-10-15/SHEET.md`). (since 10-15)
-3. Look at these demo-hp pages when you have time: Calibre-Web, the file manager, AdventureLog, Plex, the app page, the Plex and Sonarr guides. Since 2026-10-16.
+3. Look at these demo-hp pages when you have time: Calibre-Web, the file manager, AdventureLog, Plex, the app page, the Plex and Sonarr guides. (since 10-16)
-4. Remove the test server „plex-k6" from your Plex account (plex.tv → Settings → Authorized Devices). Since 2026-10-15.
+4. Remove the test server „plex-k6" from your Plex account (plex.tv → Settings → Authorized Devices). (since 10-15)
-5. Ask Google to index `https://felhom.eu/sajat-felho-tarhely` in Search Console (R-1035). Since 2026-10-14.
+5. Ask Google to index `https://felhom.eu/sajat-felho-tarhely` in Search Console (R-1035). (since 10-14)
-6. BIOS: on the N100, read „After Power Loss"; on the HP, set „Bypass F1 Prompt on Configuration Changes" to Enable. Since 2026-10-14.
+6. BIOS: on the N100, read „After Power Loss"; on the HP, set „Bypass F1 Prompt on Configuration Changes" to Enable. (since 10-14)
-7. Pin the first Facebook post (R-917), and look at the new cover in the Facebook phone app (R-919). Since 2026-10-12.
+7. Pin the first Facebook post (R-917), and look at the new cover in the Facebook phone app (R-919). (since 10-12)
 
-**Day, 2026-10-23 — your do list.**
+**2026-10-29 (release run).** 1. Clear test app Radicale's hold on Tester 1, in ONE command (R-1343): `ssh root@192.168.0.154 "pct exec 9201 -- bash -c 'docker exec felhom-controller felhom-controller --clear-restore-hold radicale && systemctl restart felhom-controller-bootstrap.service'"`. If not: Radicale stays held there. 2. (since 10-28) Give the homelab session `audits/http2shell-2026-10-27/HOMELAB-LIST.md` (19 images, R-1328). If not: DooPlex keeps the bad runtimes.
+**2026-10-23 do list.**
 1. Print the break-glass sheet (`documentation/runbooks/break-glass-sheet.md`, the part from „The sheet”, one A4), fill it by hand, keep it outside the home (D32; R-923, R-924). If not: after a DooPlex loss no off-site copy opens.
-2. On the hub's Hosts page: Discard appliance 49, and delete host Tester4-f967d4 (the BYO test host, removed 2026-10-25). If not: dead entries alarm.
+2. On the hub's Hosts page: Discard appliance 49. If not: a dead entry alarms.
 3. Give the homelab session R-1241, R-1242, the lists at the end of `documentation/audits/plex2shell-felhom-impact-2026-10-23/README.md` and `documentation/audits/dooplex-alarms-2026-10-24/README.md`, and an upload alarm on DooPlex's `eno1` (~100 Mbit/s line; the flood peaked at 905). If not: the next hacked app on DooPlex can read the hub's backups.
 **The rest**
 - System poster needs a refresh: the new 01:40 „release copy” arrow from DooPlex to ep0 (D33); the hub-database label already says every 6 hours. If not: the drawing misses one copy.
-- Tell CC what removed four old agent packages from Gitea on 2026-10-25 (a clean-up rule?): the agent's CI is red until the gate and the rule agree (R-1290).
+- Rotate the reused homelab logins and decide the Gitea repo's visibility (R-925, P1). (since 10-09)
-- Publish installer 1.36.1 (tag + webpage sync): a wrong passphrase then reads „passphrase REJECTED", not a doubled code (R-1284).
+- Two DooPlex leftovers: `leftovers/2026-10-14/first-post/` (keep or delete); the `deny`-list edit in homelab-manifests' `.claude/settings.json` (commit or drop). (since 10-14)
-- Rotate the reused homelab logins and decide the Gitea repo's visibility (R-925, P1). Since 2026-10-09.
-- Two DooPlex leftovers: `leftovers/2026-10-14/first-post/` (keep or delete); the `deny`-list edit in homelab-manifests' `.claude/settings.json` (commit or drop). Since 2026-10-14.
 - Before the first paying customer: Tandoor's permission (R-789); a lawyer for the licence list and the terms (R-802, R-813); SparkyFitness by e-mail (R-784).
-- Later (yours, not urgent): R-1250 (a seeding qBittorrent is never tested), R-1248 (a hard upload cap per app), R-1229 (ArgoCD's sync settings into git), R-862 (Tester 2's bootstrap, when it is back), R-504, R-1071, R-1073, R-1077, R-1042, R-132, R-908, R-779, R-904, R-232, R-433, R-933, R-213, R-832, R-388, R-882, R-883, R-814, R-1056, R-1164, R-930, R-920, R-902.
+- Later (yours, not urgent): R-1248 (a hard upload cap per app), R-1229 (ArgoCD's sync settings into git), R-862 (Tester 2's bootstrap, when it is back), R-504, R-1071, R-1073, R-1077, R-1042, R-132, R-908, R-779, R-904, R-232, R-433, R-933, R-213, R-832, R-388, R-882, R-883, R-814, R-1056, R-1164, R-930, R-920, R-902.

documentation/architecture/00-capability-map.md

@@ -103,7 +103,7 @@
 | ID | Capability | Component | Status | Scope | Evidence |
 |---|---|---|---|---|---|
 | <a id="b1"></a>[B1](capability-map-detail/B.md#b1) | Deploy an app from the catalog (env config, memory guard, health-aware progress) | controller, catalog (65 apps, images pinned) | **PROVEN-LIVE** | **re-checked 2026-10-24** on controller 0.342.0 (after 6d90374 + 2fe6341): all 65 apps through the deploy request on a fresh box — 62 installed and answered, Wanderer installed but cannot run without DNS (R-739), 2 not offered by lifecycle; the memory guard did not fire (one app at a time) | `audits/catalog-sweep-2026-10-24/README.md` (2026-10-24); `tests/CAMPAIGN-2-2026-07-07.md` (2026-07-07); `audits/RERUN-p1p3-2026-07-06.md` (2026-07-06) |
-| <a id="b2"></a>[B2](capability-map-detail/B.md#b2) | App lifecycle: start/stop/restart/update/logs/remove/redeploy | controller | **PROVEN-LIVE** | the actions; remove-with-data re-proven 2026-09-13; **re-check (2026-10-24):** mechanism changed after its evidence — 8ae9178 (removal deletes labelled volumes, R-1117) | `audits/R442-2026-09-13/` (2026-09-13); `tests/CAMPAIGN-2-2026-07-07.md` (2026-07-07) |
+| <a id="b2"></a>[B2](capability-map-detail/B.md#b2) | App lifecycle: start/stop/restart/update/logs/remove/redeploy | controller | **PROVEN-LIVE** | the actions; remove-with-data re-proven 2026-09-13; **re-checked 2026-10-28 night on 9202** after 8ae9178 (removal deletes labelled volumes, R-1117): install, stop, start, restart, logs, the protected stacks refuse a stop (403), remove with data → its labelled volume and container gone (`audits/night-2026-10-28/part5-claims/lifecycle-walk.*`) | `audits/R442-2026-09-13/` (2026-09-13); `tests/CAMPAIGN-2-2026-07-07.md` (2026-07-07) |
 | <a id="b3"></a>[B3](capability-map-detail/B.md#b3) | Update is GUARDED: it refuses without a restorable backup, backs up first when the copy is stale, and HOLDS an app that does not come up — on ANY backup tier, and the release itself arrives by the managed floor | controller v0.237.0 + v0.238.0 + v0.238.1 + v0.239.0, hub v0.112.0 | **PROVEN-LIVE** | — | `audits/slice4-2026-09-13/` (2026-09-13); `audits/rulings-r472-r475-2026-09-13/` (2026-09-13) |
 | <a id="b4"></a>[B4](capability-map-detail/B.md#b4) | A failed update is UNDONE by the box itself — the previous version back with its data from seconds before the update; the app is held only if that undo fails too | controller v0.263.2 (the undo), v0.264.0 + hub v0.120.0 (the mail) | **PROVEN-LIVE** | scratch 9202 | `audits/undo-live-2026-09-23/README.md` (2026-09-23) |
 | <a id="b5"></a>[B5](capability-map-detail/B.md#b5) | The catalog holds only TESTED steps — an image move without a proven test record (bench + box, digests the registry still serves) is refused at push time | catalog `6db08a5` (test-record gate, `ladder.py`) | **PROVEN-LIVE** | written PROVEN: the catalog gate against the live registry | `audits/DRILL-night-2026-09-23.md` (2026-09-23) |
@@ -124,7 +124,7 @@
 | <a id="b20"></a>[B20](capability-map-detail/B.md#b20) | App crashes → customer notified (one event per transition, no flapping spam) | controller v0.120, hub v0.48 | **IMPLEMENTED** | crash→customer mail never live-confirmed | `audits/CAMPAIGN-3-2026-07-11.md` (2026-07-11) |
 | <a id="b21"></a>[B21](capability-map-detail/B.md#b21) | Post-deploy optional config (API keys etc.) with restart | controller, catalog `.felhom.yml` | **IMPLEMENTED** | config-save + restart in no campaign | `tests/CAMPAIGN-2-2026-07-07.md` (2026-07-07) |
 | <a id="b22"></a>[B22](capability-map-detail/B.md#b22) | Backup classification: 13 bind-bearing apps carry mandatory/optional/excluded classes | catalog, controller v0.132–133 | **PROVEN-LIVE** | — | `audits/SPIKE-backup-classification-2026-07-14.md` (2026-07-14); `audits/CAMPAIGN-6D-2026-07-15.md` (2026-07-15) |
-| <a id="b23"></a>[B23](capability-map-detail/B.md#b23) | An urgent security fix reaches the boxes at night — a `security_fix` ladder step (box walk required, bench optional, a CVE/GHSA notice), taken by the night update (`09` §3 decision 255, D39 = B) | catalog `3898593`, controller v0.341.0 | **PROVEN-LIVE** | scratch 9202 (drill catalog), Plex 1.43.4 → 1.43.5; an app with no box fixture is checked by health, version and a soak only (`seed: none`) | `audits/security-lane-2026-10-23/` (2026-10-23) |
+| <a id="b23"></a>[B23](capability-map-detail/B.md#b23) | An urgent security fix reaches the boxes at night — a `security_fix` ladder step (box walk required, bench optional, a CVE/GHSA notice), taken by the night update (`09` §3 decision 255, D39 = B) | catalog `3898593`, controller v0.341.0 | **PROVEN-LIVE** | scratch 9202 (drill catalog), Plex 1.43.4 → 1.43.5; an app with no box fixture is checked by health, version and a soak only (`seed: none`) | `audits/security-lane-2026-10-23/` (2026-10-23); batch use: `audits/security-app-updates-2026-10-27/` (2026-10-27, 14 steps live) |
 
 ## C. Protection & recovery (the product promise)
 
@@ -195,8 +195,8 @@
 | <a id="e5"></a>[E5](capability-map-detail/E.md#e5) | Files from Windows Explorer / Mac Finder (SMB server) | controller v0.145.0 + `felhom-samba:1.0.0` | **PROVEN-LIVE** | — | *(evidence not on file — not re-verifiable)*: the row cites `felhom-controller` REPORT.md (v0.144.0, outside `documentation/`), the design `controller/sharing.md` and the transport verdict `audits/SPIKE-lan-discovery-2026-07-18.md` — none is a run of the shipped server |
 | <a id="e6"></a>[E6](capability-map-detail/E.md#e6) | Media to TV via DLNA | — | **MISSING** | — | R-6; R-8 |
 | <a id="e7"></a>[E7](capability-map-detail/E.md#e7) | File access via browser | FileBrowser (infra app, auto-mount sync) | **IMPLEMENTED** | browse/download through FileBrowser in no doc | `audits/storage-lifecycle-acceptance-2026-06-15.md` (2026-06-15); R-75 |
-| <a id="e8"></a>[E8](capability-map-detail/E.md#e8) | Indítópult (app launcher) — one-tap grid of the household's openable apps | controller v0.163.0 | **IMPLEMENTED** | live click-through pending | R-72 |
+| <a id="e8"></a>[E8](capability-map-detail/E.md#e8) | Indítópult (app launcher) — one-tap grid of the household's openable apps | controller v0.163.0 | **PROVEN-LIVE** (2026-10-28 night, 9202, endpoint-level — the household's page lists a tile per openable app, each linking to its public address) | `audits/night-2026-10-28/part5-claims/launcher-walk.*` | R-72 |
-| <a id="e9"></a>[E9](capability-map-detail/E.md#e9) | Indítópult megosztás (vendég link) — capability URL `/s/<token>` serves a standalone read-only guest launcher (no account, no admin session); optional per-share password; QR | controller v0.165.0 | **IMPLEMENTED** | — | R-15 |
+| <a id="e9"></a>[E9](capability-map-detail/E.md#e9) | Indítópult megosztás (vendég link) — capability URL `/s/<token>` serves a standalone read-only guest launcher (no account, no admin session); optional per-share password; QR | controller v0.165.0 | **PROVEN-LIVE** (2026-10-28 night, 9202, endpoint-level: switched on, opened with NO session — the apps, no admin link; a wrong token 404; rotate → the old link 404, the new 200; switched off → 404; the box left as found) | `audits/night-2026-10-28/part5-claims/launcher-walk.*` | R-15 |
 | <a id="e10"></a>[E10](capability-map-detail/E.md#e10) | Forgot dashboard password → instant reset code | controller v0.123, hub | **PROVEN-LIVE** | — | `audits/DRILL-day0-take2-2026-07-12.md` (2026-07-12) |
 | <a id="e11"></a>[E11](capability-map-detail/E.md#e11) | The dashboard in English — the household picks its language; Hungarian unchanged | controller v0.247.0 (spike) → v0.250.0 (slice 1, R-556 CLOSED) | **PARTIAL** | every dashboard template live; other surfaces open | `audits/i18n-slice1-2026-09-17/C/live/` (2026-09-17); `audits/i18n-2026-09-17/live/` (2026-09-17) |
 | <a id="e12"></a>[E12](capability-map-detail/E.md#e12) | The box tells visitors apart — a stranger's wrong passwords lock only the stranger (dashboard, setup gate, apps that read the visitor from the right) | controller v0.286.1, catalog router resets | **PARTIAL** | — | `audits/visitors-2026-10-01/A/L2-demo-hp-real-tunnel.txt` (2026-10-01); R-779 |

documentation/architecture/01-topology-and-trust.md

@@ -281,6 +281,13 @@ app. Controller down → the gated app answers an error, never the app. Measured
   tunnel. Since controller v0.286.0 it sits ALONE on the `felhom-tunnel` network at the fixed address `172.16.253.2`
   (traefik at `.3`), so traefik can believe forwarded headers from it and from nothing else (§5). Geo-restriction WAF
   is **hub-enforced** (the hub holds the CF API token; the controller only reports geo desired-state).
+  **Version and rule (2026-10-26, R-1300):** the image is `cloudflare/cloudflared:2026.10.3` since controller v0.345.1
+  (`infra.CloudflaredImage`). **cloudflared follows Cloudflare's security notices** — a notice is a patch release of
+  the controller, and a controller update recreates the container on the new image by itself
+  (`TestEnsureCloudflared_ANewImageRecreatesTheTunnel`). **Never turn off a certificate check to make an update
+  work.** The one check that is off today is the route's own: `*.<domain>` → `https://traefik` with `noTLSVerify`
+  (R-510; traefik shows its self-signed default certificate on a leg that never leaves the guest) — a recorded
+  exception, not a pattern to copy.
 - **[DESIGN] The website's content comes from Gitea IN-CLUSTER** (2026-10-19). The `felhom-webpage` pod pulls
   `website/` (from `main`) and `scripts/` (from the `installer-v…` tag, R-110) with four git-syncs that read
   `http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git` — plain HTTP, safe only because it never

documentation/architecture/03-host-agent.md

@@ -510,7 +510,7 @@ per Part 1: **snapshot** (LVM-thin, transient, whole-guest rollback — not a ba
   backup is due (`GET /backup/due`, §6) → **quiesces** (stops its app stacks) → `POST /backup` →
   polls `GET /backup/status` to `done` → **unquiesces** (restarts exactly the stacks it stopped).
   Implemented in `felhom-controller` v0.36.0 (`internal/quiesce`) + `felhom-agent` v0.11.0 (the
-  `/backup/due` cadence policy + `/backup/status` phases). **An agent-initiated vzdump is
+  `/backup/due` cadence policy + `/backup/status` phases). *[DESIGN] R-1232 (2026-10-26 night, agent main `3911cc6`, next agent release): on an encrypted tier the due-check counts only archives written with THIS storage's key (the R-727 discriminator — an archive carries no host id), so a reinstalled box does not read the old box's archive as its own last backup.* **An agent-initiated vzdump is
   crash-consistent only** (there is no inbound-to-guest channel to trigger a quiesce — §3/§5); the
   controller stopping its stacks first is what makes the captured state **clean-shutdown-consistent**
   (validated live: a quiesced postgres restore comes up clean — "database system was shut down" — vs
@@ -569,6 +569,16 @@ never opens, re-seals or logs it; the audit (`escrow_carried`) carries host ids
 hosts → no carry, `escrow_carry_refused` to the operator. The box then sees „identity present", its mint guard (R-241)
 holds, and it offers the recovery screen. Proven live 2026-10-21 (`audits/fixes-2026-10-21/P1-live/`: the right code
 opened the household's history, Nextcloud's 3 031 files came back byte-identical, the next off-site run appended).
+**[DESIGN] 2026-10-28 night — WHICH retained row is carried (R-1255 H3; felhom.eu `1ee6d5a5`, rides the next hub
+release):** the newest retained row that has an identity blob AND a repository-password hash; only when no row has a
+hash, the newest row as before (a legacy package whose hash was never recorded may still seal the password — never fewer
+ways back). Before, the newest row alone was taken: a newer package without the password was carried while the
+household's real one stayed under the deleted host. **And the set-aside purge (R-1255 H4):** a set-aside is named
+`<repo>.orphaned-<YYYYMMDDTHHMMSSZ>` (unique per run; was the date, which a same-day abandonment freed for the next
+set-aside), and a retained row is purged only when it was tagged with the deleted path no later than that path's
+deletion (`repo_tagged_at`; a row tagged before the column existed uses `superseded_at`) — a reused path's later copy is
+still stored, and its packages stay. Store tests `TestR1255_*`, red before the fix; no live proof (a hub deploy is not
+tonight's).
 
 #### The separation principle (the rule that governs every posture)
 Reading customer data needs **BOTH** the encrypted chunks **AND** a usable key. **Zero-knowledge
@@ -652,6 +662,15 @@ nothing waiting 404, hub failure 502 (still waiting). A controller that has not
 R-978 order. Proven on demo-hp 2026-10-19: ceremony 1 typed back → code 1 opens the hub's package, code 0 is a kept
 earlier one (422); ceremony 2 shown and abandoned → code 1 still opens it, code 2 opens nothing
 (`audits/day-2026-10-19/P1/escrow-*.txt`).
+**[DESIGN] 2026-10-28 night — a lost or late answer (R-1254 H2; agent `5ddd124` + controller `e33cffa`, unreleased):**
+the upload at the claim or the confirm can lose its answer AFTER the hub stored the package, and the household was then
+told „the previous code still works" — false. Now, after a failed upload, the agent asks the hub which package it holds
+(`GET /api/v1/hosts/<id>/escrow`, the identity blob compared byte for byte, ciphertext only, 15 s): THIS package → a
+success; another package → 502, the old sentence (now true); no answer → **504**, a sentence that promises nothing about
+either code, the wait goes on. The controller words 504 and its own failed call as „Nem tudjuk biztosan… Őrizd meg
+mindkét kódot…" (`api.escrow.confirm_store_unknown`), and waits up to 60 s for the claim and the confirm (was the
+client's 15 s — shorter than the agent's 30 s upload, so an abandoned claim could store a package whose code reached
+nobody). Tests `TestR1254_*` in both repos, red first. Live proof with the agent release (the vouch).
 
 #### Custody across generations — what the hub retains when a blob is superseded (hub v0.60.0; **contract corrected hub v0.93.0, R-198**)
 

documentation/architecture/04-control-plane-authorization.md

@@ -80,7 +80,9 @@ nonce (DoS-safe). Each layer is mandatory and was proven to reject independently
   the root wrapper (`configs/felhom-os-apply`) alike (corrected 2026-10-24). **[DESIGN — the operator picked a code change
   2026-10-25 (D44); row R-1280]:** neither caps the window's LENGTH (`expires_at − issued_at`); `felhom-opsign` defaults to 30 min
   (`cmd/felhom-opsign/main.go:57`), and the nonce store keeps a record until its expiry. The pick: the agent and the
-  root wrapper refuse a window longer than 24 h. Not built yet;
+  root wrapper refuse a window longer than 24 h. **Built 2026-10-25 night (agent `d912307`, unreleased — rides the next
+  agent release):** `authz.MaxWindow = 24h` (`ErrWindowTooLong`), the wrapper's `MAX_WINDOW_S`, and `felhom-opsign -ttl`
+  refuses more; exactly 24 h is accepted;
 - **nonce** — unseen within the window (the nonce store **must be persistent across agent restarts**
   and expiry-pruned; a non-persistent store reopens the replay window after a restart).
 
@@ -181,6 +183,18 @@ the deletion is the hub's), `run_job` (`fill-watch`, `offsite-integrity`, `offsi
 Anything else is refused at the hub's POST and again on the box. No action deletes data, starts a countdown or
 shortens one (`TestOpActions_ClosedList`, `TestOperatorActions_ClosedList`). Unanswered after 24 h: expired.
 
+**`[DESIGN]` 2026-10-29 — D64 = B, and the agent's own list (operator ruling 12:10; `09` §3 decision 283).** The
+controller's list gains **`clear_hold <app>`** for every hold reason, with no signature: the RUNNING controller clears the
+hold in memory and saves at once (no later settings save can write it back — R-1343). The operator accepted the cost in
+words: a compromised hub could release a data-protecting hold and start an app on half-restored data; the hub's confirm
+step shows the hold's reason and, for a restore or put-back hold, that warning. **R-379 („only a person clears a hold")
+now reads „a person — on the box, or the operator on the hub".** The hub refuses `clear_hold` for an app the box does not
+report held now. **The AGENT has its own closed list on the host report's reply — `restart_controller <vmid>` only**
+(`store.ValidateHostOperatorAction` / agent `localapi.OperatorActionNames`; never a shared verb, `TestHostOperatorActions_ClosedList`):
+the supervisor's restart with its guards, the 3-in-15-minutes brake counting operator restarts too (then 30 minutes
+refused), the result in the next report (requested at once), a vmid that is not a guest of that host refused at the hub.
+Neither verb deletes data, starts a countdown or shortens one. A guest reboot from the hub is NOT on either list.
+
 ### 6.2 Who may press the operator's buttons: session, Basic, or a scoped token [hub v0.152.0, `09` §3 decision 251]
 
 The hub's operator surface has three ways in: a browser session, Basic auth with the operator password (scripts,

documentation/architecture/05-hub-architecture.md

@@ -139,7 +139,9 @@ walk is green; the golden is baked weekly and vouched by the operator; a new box
 itself to the floor (R-1052: not yet proven end to end). Setting the global floor today needs the operator page with the
 operator's password — there is no scoped route for CC (R-1051). **An agent can be vouched mid-week; the golden stays
 weekly** (hub v0.158.0): the R-120 golden guard judges only a CHANGED golden selection, so a Day-0 save that keeps the
-vouched golden (same version and sha) saves a new agent even while that golden is behind the fleet. **The agent list
+vouched golden (same version and sha) saves a new agent even while that golden is behind the fleet. **„The fleet" is the
+newest RELEASED controller any box reported** — a test build (`-rc.N`, `-dev…`) never counts (R-1289, hub v0.171.0,
+unreleased: the semver compare could not read a suffix and kept a dev build no box ran as the fleet's newest). **The agent list
 offers real agent packages only (R-1011, hub 0.162.0, deployed since; corrected 2026-10-24):** the sha lookup asks for exactly the
 `felhom-agent` file (the golden: `golden.tar.zst`) and never falls back to another file, so a step bundle
 (`0.156.1-step1`, only the config bundle) is left out of the list, and a vouch of one is refused
@@ -298,7 +300,7 @@ What remains:
   **`host_escrow.blob`** (and the identity blob beside it).
   **[DESIGN] 2026-10-24 (R-1254; committed felhom.eu `dcf4f84c`, hub next v0.170.0 — NOT deployed):** the escrow PUT
   stores the blob, its sealed-password hash and the identity blob in ONE transaction (`Store.SaveHostEscrowWithIdentity`;
-  the superseded copy is made inside it). A failure leaves all of the old package or all of the new — never the new hash
+  the superseded copy is made inside it). *[DESIGN] R-1218 (2026-10-26 night, hub main `09b5f02f`, next hub release): a package that seals the off-site repository password is never replaced by one that seals none — the PUT answers 409 and the good package stays current (it used to be demoted to operator-only custody, so the household's code no longer opened its off-site copy).* A failure leaves all of the old package or all of the new — never the new hash
   beside the old identity package, which a box would read as „escrowed" while the written code opened nothing. A
   malformed identity blob is refused before anything is stored. (The agent ceremony's „the previous code still works"
   after a lost answer, R-1254's other half, stays open.)
@@ -389,6 +391,17 @@ the explicit Re-issue action") and the re-issue itself then refused with 400 —
 tier. **Not built:** releasing ONLY the token on host delete. The endpoint's only removal op
 (`deprovision`) destroys the backups too, so a token-only release needs a new endpoint operation.
 
+**[DESIGN — R-1295, hub v0.172.0 (rc.1 deployed for the proof) + tenantsync v1.4.0 (committed, not on ep0)] A new
+token's self-check waits out PBS's permission cache, and a failure is never silent.** PBS's proxy caches permissions for
+up to 5 s and an ACL-only change does not clear that cache; a request from another client between the token create and
+its grant leaves the token refused (403) for up to ~5 s, and the script's one quick retry fell inside the window
+(Tester4, 2026-10-26: rolled back, the box got no whole-guest off-site tier, only a log line). Measured on a throwaway PBS
+4.2.5 under load: v1.3.0 failed 11 of 30, v1.4.0 0 of 30 (`audits/r1295-pbsdr-selfcheck-2026-10-26/`). The endpoint now
+re-checks the same token (and its own admin token) for up to 15 s; the hub repeats a `self_check_failed` rollback (3
+attempts, 7 s apart, a log line each) in the provision and the F-14 re-issue paths; a final failure of the WG hook raises
+the operator-only `pbsdr_provision_failed` („… save the customer config to retry"). Live 2026-10-26 on hub rc.1: a fresh
+Tester4 install provisioned the tier on the first try.
+
 ## 15. Customer e-mails — what the hub writes to a household [hub v0.118.0, R-558]
 
 **The section this document did not have.** The hub composes every sentence a household reads before
@@ -746,7 +759,7 @@ method (405), so a token gets 403.
 | `/configs/{id}/edit` | `customers:read` | **session only** | Edit can change the household's e-mail — and a new e-mail auto-mails the self-bind link (ruling 13:20). |
 | `/offsite/remove-unpinned/{id}` | — | **session only** |  |
 | `/offsite/abandon-cancel/{id}` | — | **session only** | Cancelling a pending set-aside deletion: no group names it; until the operator places it, session only. |
-| `/offsite/window-grant/{id}` | — | `offsite:write` |  |
+| `/offsite/window-grant/{id}` | — | `offsite:write` | The one-shot clean-up window: the script address AND the customer page's „Backup & DR" card (R-1299, hub v0.174.0): `from=page` + `confirm=1` from the form; `cancel=1` removes a waiting grant; a limit above the store's snapshot count is refused. |
 | `/offsite/windows-enabled` | — | `offsite:write` |  |
 | `/os/{id}` | `os:read` | `os:write` |  |
 | `/offsite/key-audit` | — | `offsite:read` | Runs the daily key check now: it reads keys and reports, it changes none. |

documentation/architecture/07-backup-architecture.md

@@ -255,6 +255,21 @@ the identity bundle's shape is `{tunnel_token, pbs_token, wg_private_key, restic
 
 **[DESIGN] Off-site deletion custody (decisions 68–69, 2026-10-03) — BUILT hub v0.127.0 / controller v0.289.1, live on both demo boxes.** The restic repository password stays on the box only. The box's off-site key is append-only (pinned in `authorized_keys`, written by the hub — the key registrar); the box never receives the sub-account password, which the hub stores encrypted at rest. Old snapshots are pruned by the box itself, only in a weekly window the hub opens, behind a fake-snapshot guard (R-822); the window has run live on both demo boxes since 2026-10-05 (R-95, closed). The guard's residual — past-dated fakes steering the keeps, and a window check that trusts the box's own counts — is R-822 and R-895. **Hub v0.158.0 (built 2026-10-18; deployed the same day on the operator's yes — corrected 2026-10-24) adds the hub's own check:** the 07:10 key audit compares the repository's `snapshots/` listing, over the hub's own login, with yesterday's — files gone that no clean-up window explains raise `offsite_snapshots_vanished` (error), files no reported box run explains raise `offsite_snapshots_unexplained` (warning); both operator-only (R-895 slice 2). A Felhom-side pruner holding repository passwords is rejected.
 
+**[FINDING — R-1294, 2026-10-26, read only; a rule change waits for the operator (STATUS D54)] A store that once goes
+over the window's cap never comes back by itself.** The cap is half the store (hub `offsitekeys.MaxRemove`); each refused
+window adds a week of snapshots and removes none, so the plan grows faster than the cap. demo-hp: one test day
+(2026-10-16, 152 snapshots) aged past 7 days and the 10-26 window refused 209 > 179 — an HONEST plan (every removal 7–30
+days old, every app keeps ≥ 9 or its last copies; `audits/r1294-offsite-cap-2026-10-26/`). The existing way out is the
+operator's one-shot raised cap (R-833, `GrantOffsiteWindowOnceMax`).
+**[DESIGN — R-1299, built hub v0.174.0/v0.174.1 2026-10-27, `d85198b8` + `de57c090`] The one-shot window has a button.**
+The customer page's „Backup & DR" tab carries the „Off-site clean-up window" card: the weekly switch, the normal cap,
+a waiting grant and the last window; an optional limit, a confirm step that names the customer and the limit, then the
+same route and rule as before (`POST /offsite/window-grant/<customer>`, `offsite:write` or a session). A raised limit
+above the store's measured snapshot count is refused (a window cannot remove more than exist; the 500 ceiling stays). A
+waiting grant can be cancelled before the box uses it. Every grant and cancel is a stored operator-only info event
+`offsite_window_grant` (who, customer, limit). Live: granted and cancelled on tester-1 by the operator, nothing removed
+(`audits/r1299-window-button-2026-10-27/`).
+
 **[DESIGN, built hub 2026-10-07 — R-32 option A, `09` §3 decision 167] What RESET does to the household's off-site
 copy.** On the shared pool box a sub-account is a LOGIN, not the data: deleting it leaves its folder (measured
 2026-07-21: 1.4 GB in two `.orphaned-*` folders after a RESET), and a re-enabled customer lands on the same folder
@@ -396,6 +411,8 @@ never moved by an update or an undo, and the unit restore accepts it.
 > copy protected the app's files. Since v0.245.0 every dashboard page carries the reminder (the R-241
 > bar, second instance) and the Tier-1 sentence renders by state — „védené … szünetel" while paused.
 > Measured on a fresh box 2026-09-16: zero snapshots, and the page said the files were protected.
+>
+> *[DESIGN] R-1219 / R-1218 (2026-10-26 night, controller main `1170b48`, next release): during a recovery offer the bar does NOT urge the ceremony (a new code would replace the key that opens the old off-site copy) and the start answers 409; a ceremony stages the off-site password whenever its FILE is on the box, also with off-site switched off or cleared.*
 
 > **R-191 (2026-08-04) — this row was RIGHT and the configuration disagreed with it, weekly, for as long as R-89 has been in force.** The contract has not changed: offsite retention is ep0's, the box's token is write-only, and the box cannot delete its own history. What had not followed was the installer's `keep_last: 2` on the offsite tier, so every weekly run uploaded its snapshot successfully and then failed the whole JOB on a prune the token is refused — `whole_guest_backup_failed` in the operator's inbox about a backup that had already succeeded. Fixed in installer **1.25.0** (`keep_last: 0`) and on both live boxes; a gate now asserts it. **Verified before changing it:** ep0's two prune jobs have run every day since 2026-07-27, 18 tasks, all OK. A doc that states the contract does not enforce it — the gate does.
 
@@ -804,6 +821,58 @@ household's newest data („gen2") and its download on the drive were there, and
 nothing. Run 1 of the same proof found the journal left as `undo.json.tmp` (the rename lived only in the page cache) —
 the app was held for support, never started on the cut data; the folder sync above fixed it, run 2 passed.
 
+**`[DESIGN]` 2026-10-28 night — four restore edges (R-1256, review round 2 HUNT-1 H4/H5/H6/H8; controller `342a394`,
+unreleased — the morning list).** (H4) An off-site restore whose definition write fails after the compose was written
+writes the LIVE definition back before it starts the app; a write-back that fails too HOLDS the app (`restore_mixed`) —
+never a start on the snapshot's definition over the live data. (H5) A put-back whose stop fails pours nothing, keeps the
+copy and holds the app (off-site, local unit, and a restore cut by a power loss); a failed definition write-back keeps the
+copy (D20); a `docker volume rm -f` that fails (the volume is in use) fails that volume — the `volume create` after it was
+a no-op and the tar was unpacked ON TOP of the old content. (H6) A restore op begins in ONE step with the check for every
+other operation (a restore, the backup run, a second-drive copy — under the lock the mirror asks under); every restore
+button and the kept loads begin that way, and the install-time „use my kept data" asks about the second-drive copy too.
+(H8) An unused restore copy goes after 14 days (every 6 h): only when the app has no restore hold of any kind, the
+app-stop marker is readable and does not name the app, and no restore runs. **Decided by CC unattended 2026-10-28 —
+operator may reverse:** *How long does an unused restore copy stay?* (a) 14 days; (b) for ever (as before); (c) 3 days.
+(a) holds up to two weeks of disk per restore and gives support two weeks; (b) grows without end; (c) is shorter than a
+support reply. A held or waiting copy is never touched under any option. Tests `TestR1256_*`, each red first.
+**`[DESIGN]` 2026-10-29 — D62 replaces the age, and the household can delete a copy (operator ruling 08:17; `09` §3
+decision 281).** An unused copy stays **30 days**. The restore page's kept-copies list („Megőrzött másolatok") shows each
+restore copy: the app, the date of the restore it protects, its size and „automatikusan törlődik: <date>". „Törlés most"
+deletes one earlier after the warning („Ez a(z) <app> adatainak másolata a <dátum>-i visszaállítás előttről. Ha törlöd, a
+visszaállítás többé nem vonható vissza, és ez a másolat nem hozható vissza.") and the app's name typed. Never deletable: a
+copy a hold or a pending put-back needs (the same rule as the sweep; the button is off with the reason, and the handler
+refuses it), any copy while a backup or restore runs or the app updates (409), and — R-1342, controller `652a690`,
+unreleased — a copy the 30-minute stable-start watch still holds (it reads „checking"; an app removed during that watch
+ends it quietly instead of alarming). The check and the removal are one step
+under the backup manager's lock. One `[WARN]` log line and one operator-only event `restore_copy_deleted` (who, app,
+restore date, size; hub `19afefab`) per delete. Controller `e3fce09`, tests `TestD62_*`. **D63** (`09` §3 decision 282):
+H4 and H5's failed stop ship only with a live proof, staged with a fault switch that exists only in a `-tags faultinject`
+test build.
+**`[FACT]` 2026-10-29, Tester 1, test build `0.347.0-rc.1-d63fault` (`audits/d63-2026-10-29/`):** (H4, 08:10 UTC) a
+throwaway Radicale ran the true 3.8.0 (digest `0d60d544…`) while its newest off-site snapshot held 3.8.1's data; the full
+restore said „it comes back at the snapshot's version", the fault failed the definition write after the compose was
+written, the log then read „the live definition is written back before the start" and only then „Starting stack"; the app
+ran 3.8.0 again, no hold, and an event made before the snapshot and one made after it both read back. (H5, 08:18 UTC) a
+created-only container held the data volume („Failed to remove volume … (not written into)"); the restore's own stop
+passed, the put-back's stop failed — „nothing is put back; the copy stays at …; holding the app"; the app page showed the
+hold sentence; on the restore page the held copy's „Törlés most" was off with „Nem törölhető: az alkalmazás le van
+állítva, és ez a másolat az út vissza." (D62, 08:15 UTC) a copy kept because a second restore began inside the first one's
+stable-start window was listed with „visszaállítás: 2026. 10. 29. 09:11 · 37,3 KB · automatikusan törlődik: 2026. 11.
+28."; a wrong typed name → 400, the right one → 200, the copy gone, one `[WARN]` line; the event got HTTP 400 from hub
+0.174.1, which does not know the type yet (hub 0.175.0 does). The release image carries no fault switch (`grep -c "FAULT
+INJECTED"` = 0 on `0.347.0-rc.1`, 2 on the test image).
+**`[FACT]` 2026-10-29 night, scratch 9202 (`audits/night-2026-10-28/r1256-restore/h5-live/`):** vikunja's first volume
+held by a created-only container, the household's restore pressed through the page: „Failed to remove volume
+vikunja_vikunja_data before the restore … (not written into)", the restore failed, the put-back poured the live copy of
+both volumes, the app ran, no hold, and a project made before the restore point AND one made after it both read back. H4
+(off-site only, and a fault there races the write-back) and H5's failed stop are proven by tests only; their live proof
+is owed to an attended session.
+
+**`[FACT]` 2026-10-29 03:29 UTC, scratch 9202 (`audits/night-2026-10-28/r1256-restore/h8-live/`):** three restore
+copies planted by hand (dated 2026-10-01, 2026-10-05 and 2026-10-28); the first six-hour sweep after the controller start
+removed the two older than 14 days, one of them under the standing app paperless, and kept the young one; the job took
+8 ms. The young copy was then removed by hand.
+
 **[DESIGN] 2026-08-22 — the failure ladder of a database restore: replay → rollback → hold.**
 Recorded here rather than only in a closed register row, because a decision that survives only inside
 a closed work item is a decision nobody will find.
@@ -916,6 +985,8 @@ run makes it longer.
 
 *Second-drive copies that are KEPT (D25) and the rule that a rejected backup block never deletes one: §8.2a (R-1252, 2026-10-24).*
 
+*[DESIGN] R-1220 (2026-10-26 night, controller main `7e15a37`, ships with the next controller release): an app whose `app.yaml` EXISTS but does not parse counts as INSTALLED — every folder its compose binds, on every registered drive, is live and never listed as kept data (fail closed; one WARN names the app).*
+
 **What it is.** "Remove the app, keep my data" leaves the app's drive folder (`<drive>/appdata/<app>`, or the folder
 its definition binds through `${HDD_PATH}`) in place. A reinstall over it no longer runs silently into the old files
 (R-657): the install asks „A megőrzött adataimat használom" / "Use my kept data" (the database from the newest copy of
@@ -1073,6 +1144,8 @@ second drive's whole restore (its `CheckOnly` pass carries the pull, so its file
 load from a unit and from off-site, and a set-aside unit (D47) — the off-site restore, the second drive's file restore
 and the no-unit fallback. Not covered: the `.fab` import — it removes the running app's volumes before its definition is pulled (R-1279).
 
+*[DESIGN] The `.fab` route, R-1187 (2026-10-26 night, controller main, next release): the export carries a symlink AS A LINK and stops on a FIFO, socket or device naming its path; a drive folder that cannot be packed fails the export (never left out). The import writes only through `os.Root`, keeps owners and modes, restores a link only when it points DOWNWARD inside the folder (no absolute target, no „..”, no link on the way — others refused and listed), refuses while another operation holds the box and refuses an empty or relative `HDD_PATH`; its compose runs get the controller's own `USERDATA_PATH`/`IMPORT_PATH`; the database loads through the backup side (`appbackup.ImportDump`: mariadb / mariadb-admin, psql ON_ERROR_STOP) — the import's own `mysqladmin`/`mysql` calls failed on every MariaDB 11+ image. Live: `audits/night-fixes-2026-10-26/`.*
+
 ### 6.7 Why an off-site run failed — the failure classifier (R-571)
 
 **[FACT, read from source 2026-10-05, felhom-controller `114ff27`]** When an off-site (restic) run fails, the
@@ -1720,7 +1793,7 @@ names the app and both identities). Nothing moves a unit while that app is being
 telepítés mentése") with **„Visszaállítás"** — the one unit-restore body, with the unit's own database password — and
 „Eltávolítás". Only those two, and the app's remove with „delete its backups too", ever remove it (the §8.2a / R-1252 H9
 rule). After a restore from it, the app is that unit's install; the app's own unit of the install it replaced is then
-set aside at the next backup — never lost.
+set aside ~~at the next backup~~ **at the restore itself** — never lost. **[DESIGN — R-1288, 2026-10-25 night, the brief's safest reading; controller `b5863a9`, unreleased]:** the restore of a set-aside unit moves the replaced install's unit aside whole right after it (success, or „not started yet" — the data stays, D49 = A), one log line „replaced by the restored install", **no operator alarm** — the household chose this restore; before, the next backup's writer set it aside with a second alarm and a second kept copy (measured on Tester 1, 0.343.0). **[FACT]** on the night's rig (rc.2): the line, the account back, the next backup set nothing aside, one set-aside mail only (`audits/night-2026-10-25/rig/r1288-live.txt`). A set-aside's operator mail (R-1287, hub v0.171.0, unreleased) reads „⚠️ … a backup of an earlier install was set aside — nothing failed", a warning.
 
 **Known limit, stated.** A box that reinstalled and refreshed BEFORE this release already holds a rewritten unit; its
 old password is gone and nothing can bring it back. A unit from before D47 of an app with no carried secret has no
@@ -1728,7 +1801,7 @@ signal: it is treated as this install's (the old behaviour) and gets the identit
 **Why a volume app's reinstall asks no kept-data question:** the question (§6.5, decision 36) is about the app's drive
 FOLDER (`${HDD_PATH}` binds under `appdata/`); a volume app keeps nothing there — „remove, keep my data" removes its named
 volumes and keeps only its backups — so the install starts empty, and the kept backup is now the set-aside unit on the
-restore page. (The missing offer at install is its own row, R-1278.)
+restore page. **[DESIGN — R-1278, controller `fbec5cb`, unreleased]:** the install page now says so for an app not installed — „the new install starts empty; load the backup afterwards on the Restore page" — with a link (set-aside and removed-app units counted).
 
 ### 8.2e A restore whose app does not start is never „success" [DESIGN, R-1270, 2026-10-25 — controller `b680db8` + fix A; D49 = A (operator 2026-10-25); released in controller 0.343.0]
 

documentation/architecture/08-alarm-ladder.md

@@ -251,6 +251,19 @@ today's engine, and its build (read the counter of every container; „probably
 before any code, by the brief's measure-first rule — `09` §3 decision 155 stands for the operator to confirm or drop.
 Cost measured for the record: one `docker exec … cat memory.events` across demo-hp's 21 containers takes 1.6 s wall
 (0.47 s user, 0.44 s sys); cloudflared has no `cat` and reads as unknown. `audits/design-build-2026-10-06/C/`.
+**[DESIGN] 2026-10-28 night — the host's counter beside Docker's flag (R-528, as designed in the row 2026-10-26; agent
+`6f933d6` + controller `eed23e7`, unreleased):** the agent serves `GET /guest/oom-kills` — the kernel's `oom_kill` of
+each of the asking guest's containers, read on the host from `/sys/fs/cgroup/lxc/<vmid>/ns/system.slice/docker-<id>.scope/memory.events`
+(0444; the non-root agent reads it — no sudo); the 30-s scan asks once per pass (5 s cap; an agent without the route
+404s and is asked again after 10 minutes; no agent = Docker's flag only). A running container whose CURRENT run counts a
+kill (its scope is new at each start) raises the same `app_oom` (once per run), whatever Docker's flag says; the
+event's details carry `source` (`docker`, `cgroup` or both) and the host's count is the run's kill count — so a
+container with a false flag now also reaches the storm rung. **[FACT] live 2026-10-28 23:27 on Tester 1** (test builds
+of both, put back afterwards): a throwaway Radicale capped at 128 MB, `tail /dev/zero` inside it → the hog killed
+(exit 137), the container kept running, the host counter 0 → 1; the controller logged the OOM line naming Radicale,
+„seen by docker,cgroup", and pushed ONE `app_oom` (none in the next two minutes); the operator's mailbox (a second
+channel) held exactly one mail, details `"source":"docker,cgroup"`. Docker 29.8.2 reported this kill too, so the
+cgroup-only path is proven by the tests, not live. `audits/night-2026-10-28/r528-oom/`.
 
 **The box STOPS a crash loop or an out-of-memory storm (decision 28 of `09` §3, R-667, controller v0.269.0 /
 hub v0.123.0, 2026-09-24).** Alarming was not enough: gokapi crash-looped for hours at 385 → 546 restarts
@@ -591,6 +604,22 @@ normal use; work it does not explain is the sign of a hacked app. Numbers and me
    start-up check puts it back; the 50 % rule waits while a record exists. Measured on demo-hp with a real seed (10
    Mbit/s to DooPlex's qBittorrent): refused `not-quiet` without the hold; with it the uplink fell to 0.02–0.16 Mbit/s,
    94.2 Mbit/s measured, the app held 56.8 s and back to 1 249 280 B/s exactly (`audits/dooplex-alarms-2026-10-24/`).
+   **[DESIGN — R-1250 drain wait, 2026-10-27; controller `710ceac`, released v0.346.0]:** between the hold and the call
+   the controller waits until every held app REPORTS its own upload ≤ 16 KiB/s on two readings 2 s apart (at most 90 s;
+   an unreadable speed waits 20 s) — qBittorrent 5.2.4 kept sending at full speed ~14 s after its limit fell, inside the
+   agent's 5–15 s quiet read, and four tries in an hour were refused; with the wait the test ran (94.5 Mbit/s, the limit
+   back exactly; `audits/linetest-2026-10-27/`; `09` §3 decision 269).
+   **[DESIGN — R-1261, 2026-10-25 night; controller `a79a292`, unreleased]:** EVERY running seeder is held (qBittorrent
+   and Transmission — one left seeding kept the test refused `not-quiet` every time); all records first
+   (`upwatch.line_test_holds`), then the apps; a record whose app was removed while held is dropped (it used to stop the
+   50 % rule for good); qBittorrent's hold names its slot (`up_limit` / `alt_up_limit` through `app/preferences` —
+   `transfer/uploadLimit` follows the CURRENT speed mode, measured on 5.2.4), so the household's switch to the
+   alternative speed during the test never leaves 1 KiB/s behind. **[DESIGN — R-1292, controller `2619191`, unreleased]:**
+   the 50 % rule owns the NORMAL slot only (`up_limit` through `app/preferences`); the household's alternative limit is
+   never read or written by it, and a mode switch never reads as „the household changed it". **[FACT]** the night's rig (BYO VM, agent 0.165.0,
+   rc.1, two seeders to 9202's qBittorrent): with a mode switch mid-test both limits came back, the test refused
+   (the switch let qBittorrent upload); without, both held (qBittorrent in its alternative slot), 94.8 Mbit/s measured,
+   both limits back exactly (`audits/night-2026-10-25/rig/hold-watch*.txt`).
    While the box owns it, qBittorrent's page says: „A qBittorrent
    feltöltése legfeljebb N Mbit/s, hogy a családi internet gyors maradjon. A qBittorrentben átállíthatod."
 6. **Limits, stated:** *(added 2026-10-24, review round 2: ANY viewer explains any amount of upload, so a hacked app can send freely while one real playback runs — `upwatch/watch.go:275,299`; R-1262)*; an attacker who knows the check can make the app's API lie — a per-container ceiling (`tc` from
@@ -598,3 +627,100 @@ normal use; work it does not explain is the sign of a hacked app. Numbers and me
    whatever the API says; not built. The box cannot tell LAN from internet bytes per app; a host-network container
    (Samba) has no counter; the push floor is a lower bound, the weekly test is close to the line.
 7. **Release:** released 2026-10-24 on the operator's yes (controller v0.342.0, hub v0.169.0; R-1244) — corrected 2026-10-24.
+
+## 10. A held app the operator can see [DESIGN — built 2026-10-26 (commit only): controller `5bd03ad`, hub `7e57e824`; D53 = operator-only, `09` §3 decision 265; R-1291]
+
+**Why.** A held app is stopped on purpose and stays stopped until a person acts — and today nobody is told after the
+first minute. demo-hp's Kimai sat held for 11 days (2026-10-14 → 10-26) and was found by accident (R-1286). The
+controller sends one event when the crash-loop guard stops an app (`app_stopped_unhealthy`); no other hold sends
+anything, and the hub has no held-app state at all. On a household box the same silence would hide a failed restore.
+
+**The holds that exist** (controller `settings.RestoreHold.Reason`, the sentences of `holdWhy`, R-1286):
+
+| Reason | What put it there | Who may clear it |
+|---|---|---|
+| `restore_failed` | a restore whose rollback also failed | **a person only** (R-379: support, `--clear-restore-hold`) |
+| `restore_mixed` | an off-site restore whose database replay failed | a person (the same) |
+| `update_failed` | a failed update | the household, by restoring the app from its backup |
+| `unhealthy_stop` | the crash-loop guard | the household's Start (one more try, `09` §3 decision 28) |
+| `putback_pending` | a restore waiting for its pre-restore copy | nobody — it clears itself when the drive comes back |
+
+**What the controller reports.** The legacy report is frozen (hub rule), so the event channel carries it:
+`app_held` when a hold is written (stack, reason, since — the hold's own `At`) and `app_hold_cleared` when it goes.
+Because an event can be lost, the controller re-sends `app_held` for every hold still in place **once a day**; the hub
+treats it as idempotent. Both types enter `allowedEventTypes` AND `customerMessages` (hu + en) in the same hub commit
+(the coupling rule); `app_held` is operator-only in v1.
+
+**Where the hub shows it.** The host page (and the customer page) get a „Held apps" list: app, reason in one line
+(the `holdWhy` sentence), since, and who may clear it. Built from the open `app_held` events per customer and stack.
+
+**When it mails.** One operator mail per hold after a grace that follows the reason: `putback_pending` 6 hours (it
+should have cleared itself — if not, something is stuck); `restore_failed` / `restore_mixed` 1 day (only a person can
+act); `update_failed` and `unhealthy_stop` 3 days (the household can act first). Then one reminder a week while it
+lasts, through the dispatcher's existing per-type cooldown (§6.2). A cleared hold sends nothing.
+
+**Who clears it.** Unchanged: R-379 holds only a person; the household's Start or restore for its own two. **The hub
+never clears a hold** — it only shows and reminds.
+
+**Cost.** One controller release (the two events + the daily re-send, ~½ day with tests) and one hub release (allow-list,
+the i18n keys, the list on two pages, a reminder checker; ~1 day). No agent change.
+
+**Ruled (D53, operator 2026-10-26, `09` §3 decision 265):** v1 is operator-only. Household mails for the two holds it can
+clear itself may follow once the operator list has run.
+
+**As built (2026-10-26, commit only — releases ride an attended run).** The wire reason is the controller's own constant:
+`putback_pending` (one word), and the empty legacy reason is sent as `restore_failed`. The controller's reporter is ONE
+observer of the hold store, ticked every minute (`internal/holdreport`), never a send inside a hold writer; the set it
+last reported is kept in settings, so a hold cleared while the controller was down is still reported cleared. The hub
+mirrors the events in `held_apps`, sends `app_held_reminder` (warning, operator-only, per-app cooldown, the subject
+names the app) once after the grace and then weekly, records the reminder before it sends it, and does not remind about
+a row the box stopped repeating for 50 h (the list still shows it, marked). `alerting.held_app_grace_override` replaces
+every grace — a test configuration only. Live proof: `audits/d53-held-apps-2026-10-26/` (a crash-looped throwaway app on
+9202 against a test hub with a 3-minute grace: listed on both pages, one operator mail, Start cleared it, the row went,
+every household leg `skipped operator_only`).
+
+*[DESIGN] R-1298 (2026-10-26 night, controller main `b156e45`): an install's hold (`unhealthy_stop`, `update_failed`, `putback_pending`) of an app that is NOT installed (no `app.yaml`) is not reported — it would remind weekly about an app that does not exist; a restore hold is always reported (R-379, a person clears it).*
+
+## 11. The security watch — three operator-only signals [DESIGN — built 2026-10-26: felhom.eu `1f62d629`, `6ee23a3b`; R-1303]
+
+The nightly CI job `security-watch` (03:40 UTC, `scripts/security_watch.py`) reads where each shipped program publishes
+its security notices (sources and their limits: `audits/security-watch-2026-10-26/SPIKE.md`) and mails **admin@felhom.eu
+only** — no household, no hub event:
+
+| Signal | When | Once? |
+|---|---|---|
+| **ADVISORY** — „[Felhom security] <programs, box first> — N new advisories" | a notice affects the version we pin | once per (program, notice); a reminder every 7 days while it stays open; a vendor-blog notice never repeats |
+| **BEHIND** — „[Felhom security] weekly: …" | Monday (or the last summary is 8+ days old) | one summary; it also lists GAP, FLOATING and tonight's NOT CHECKED |
+| **NOT CHECKED** — „[Felhom security] the watch could not check …" | the same source failed two nights in a row | once per streak; a good night resets it |
+
+The memory is `documentation/security-watch/state.json`, committed by the job itself; a missing memory re-seeds AND mails
+that it did. A job that does not finish mails „[felhom CI] security watch FAILED". The mail starts the security-fix lane
+(capability map B23).
+
+**[DESIGN — R-1327/R-1328, built 2026-10-27 night, commit only] A fourth source, „runtime", and a third group,
+„DooPlex".** HTTP2Shell (CVE-2026-111497) sat in the Node.js and Python Brotli INSIDE app images, where no app notice
+mentions it. The watch now reads each pin's image (`scripts/image_runtime.py`: the registry config, then only where
+needed the layers — Alpine/Debian package databases, Node's `node_version.h`, Python `.dist-info` — never a pull) and
+matches it against `scripts/security_watch_runtimes.json`; a hit is an ADVISORY with source „runtime", a read that
+fails is NOT CHECKED. Results are cached by the tag's digest in `state.json` (`runtime_cache`) and the nightly lookup is
+a HEAD request (Docker Hub does not count it against the anonymous pull limit — the first run's GETs hit HTTP 429).
+**The cache fills itself in CI (2026-10-28, R-1327; replaces the 2026-10-27 night's DooPlex filling run).** Images with
+zstd layers (Home Assistant) are read with the `zstandard` package, version 0.25.0, as a wheel pinned by the sha256 PyPI
+publishes and checked every time it is opened; the workflows fetch it (`FELHOM_WHEEL_FETCH=1`) into the runner's cache.
+Not the `zstd` program in the CI runner image: that image is built in the homelab repo, and changing it is a DooPlex
+change. A missing or wrong wheel makes the zstd layers NOT CHECKED, said in the log's `RUNTIME` line — never a pass. A
+new image is read on the night its tag first appears; the log's `RUNTIME read=… (… s) from-cache=…` line says how much
+was read and how long it took. The DooPlex group (`documentation/security-watch/dooplex-workloads.txt`,
+from homelab-manifests) is read the same way and mailed in its own block; Felhom changes nothing on DooPlex.
+
+**Docker Hub's tag API is asked gently (2026-10-28, R-1334).** The 2026-10-28 night run was refused 15 times (HTTP 429)
+when it sent 211 tag-list requests to `hub.docker.com` in about 2 minutes from six workers; a 3.5-minute dry run that
+morning sent 224 and was refused none. So: the requests to Docker Hub are spaced (one per second across the workers);
+a 429 waits (its `Retry-After`, else 10/30/60 s) and tries again, and the image is NOT CHECKED only after the fourth
+try; a tag list is kept for 24 hours per registry path in the CI runner's cache (`~/.cache/felhom-watch-tags.json`, not
+committed), so two names of one path (`lscr.io/linuxserver/x`, `linuxserver/x`) and a same-day re-run fetch nothing.
+The log's `TAGS fetched=… from-cache=… retried-429=…` line counts it. No Docker Hub login is used. Two CI runners take turns, each with its own
+cache, so the cache helps a same-day re-run on one runner, not the nightly; the spacing is what fixes the night (two
+dry runs 2026-10-28: 0 refused). The image reader (`image_runtime.py`) also waits out a 429 and tries again (3 tries):
+ghcr.io refused two blob reads when two dry runs read the same 14 new images 15 minutes apart.
+

documentation/architecture/09-update-architecture.md

@@ -370,6 +370,17 @@ builds them (`audits/update-rulings-2026-09-23/`).
     both options. **Why (b):** decision 13's mark exists for "does not fit the memory" (RomM, R-635), and
     only the app's own memory decides that. The cgroup peak stays in every record (`memory_cgroup_peak_pct`
     in the ladder entry). R-652.
+    **Refined 2026-10-28 night (R-1338, the night brief's „fix the measurement — the gate reads the right figure"):** the
+    mark reads the LIVE anon — `anon` without the pages the app already gave back. An allocator that frees with
+    MADV_FREE leaves the pages mapped (`anon` keeps counting them) on the INACTIVE FILE list, and the kernel takes them
+    back first. Measured: Wanderer's Meilisearch read `anon` 83–96 % of its 384 MiB limit on the bench AND on 9202 (with
+    or without swap, with or without `MEILI_MAX_INDEXING_MEMORY`), 0 kills, while `file` was ~1 MB and `inactive_file`
+    ~290 MB; it used ~100 MB. Figure (one home: catalog `scripts/memfigure.py`): the larger of `anon − max(0,
+    active_file + inactive_file − file)` and `active_anon + inactive_anon − shmem`, capped at `anon` — it errs high.
+    Wanderer v0.20.0 → v0.21.2 re-run on the bench: `proven`, no marks (search 83.3 % anon, **27.7 % live**); 9202:
+    95.2 % anon, **25.1 % live**. The evening's „17.7 % with the cap" was a different container (the one after the
+    swap, started over existing data), not the cap's effect. The catalog change (and Wanderer's entry without
+    `memory_tight`) is prepared, not pushed — the morning list. `audits/night-2026-10-28/r1338-meili/`.
 
 23. **The push-time test-record gate asks the registry — for moved refs only** — *decided by CC
     unattended 2026-09-23 night — operator may reverse.* *One sentence:* may a `--fast` (hook) gate use
@@ -1098,6 +1109,192 @@ its length, and both fixes cost something the household would notice — operato
      night leg logged `plex: skipped — not_proven` and a hand press took the step; on 0.341.0-rc.1 the night leg logged
      `plex: security fix CVE-2026-96652,CVE-2026-96654,CVE-2026-96655,CVE-2026-96656 taken` and ended `done` in 30.1 s
      after the update's second-drive copy and undo copy. Released as controller 0.341.0, global floor 11:24 UTC.
+     **Batch use, 2026-10-27 (the lane's first live use, `audits/security-app-updates-2026-10-27/`):** the security
+     watch (R-1303) flagged 38 pins; a hand triage put 15 apps in the lane. One negative control first (TO
+     `busybox:1.36.1` → `failed`, undone), then walks on 9202 three at a time, each written to the drill, then **a STOP
+     per batch of about five** before the live catalog: batch 1 `06e603e` (rallly, romm, code-server, outline, vikunja),
+     batch 2 `7b0071e` (n8n, privatebin, docmost, grafana), batch 3 `65a6b11` (homebox, mealie, audiobookshelf,
+     bentopdf, uptime-kuma) — 14 steps, 12 with the seed read back after the update, 2 `seed: none` (code-server: no
+     script route; bentopdf: no data). Two notices with no CVE/GHSA id (wishlist's critical account takeover,
+     bookstack's two leaks) could not use the lane and went as NORMAL steps (bench 9401 + box) in the same batches.
+     **What the batch use taught:** (a) a walk can be `inconclusive` because the FIXTURE is stale, not the step — four
+     were (Rallly 4.15's code-only sign-in, Jellyfin 10.11's wizard, Homebox 0.26's entities API, Mealie's generated
+     admin password); each was fixed and the walk run again; (b) three walks at a time on 9202 (25 GiB) kept every soak
+     clean; (c) a one-step-per-app lane stops below a fix that needs a NEW main version — those stay big steps with a
+     row each (R-1311, R-1313–R-1322).
+
+### 2026-10-26 (night brief) — D55: demo-hp's one clean-up window gets a cap of 260
+
+268. **demo-hp's one-shot clean-up window is granted with a raised cap of 260, not 220 (D55 = 260).** *Operator,
+     2026-10-26 night.* Options: A) 260 — cost: one window may remove up to 260 old copies on a test box (the box still
+     removes only what its normal keep rule selects, and the fake-copy guard still runs); B) 220 as first ruled — cost:
+     the plan grows ~14 a night (209 on 2026-10-26, ~223 at the next run), so it is refused again. Why A: the cap must sit
+     above the honest plan on the night it is used. CC's token lacks `offsite:write`, so the operator made a one-purpose
+     token `cc-d55-window` (Off-site write only); CC used it once, 2026-10-26 19:10:31Z (`max:260` stored, read back).
+     **The window ran** 2026-10-27 03:18:15–03:18:41Z (window 17): 223 of 223 planned removed, 373 → 150, closed by the
+     box, no alarm (hub row + the box's own log agree). The operator revokes the token.
+
+### 2026-10-27 (evening, operator 21:33) — D60: CC never deletes from the secret store; D56–D59 = CC's picks
+
+271. **CC never deletes from the `sec` store, with or without the operator's yes (D60).** *Operator, 2026-10-27 21:33.*
+     `sec rm` is the operator's command, run in his own terminal. Why: one purpose of the store is that CC cannot delete
+     by mistake (the old credentials file was wiped once). Made mechanical the same evening: cc-secrets 0.4.0 (`092e282`,
+     hardened `27fd806`) refuses `sec rm` in every form and any removal or overwrite of the store file or the age key
+     (Bash and Write/Edit guards), proven in a new CC session. The 46 names removed earlier that day by CC (on the
+     operator's yes, before this ruling) can come back by `git revert` of their `rm NAME` commits — the operator's act.
+
+272. **D56 = keep SparkyFitness offered** (R-785) while its two flaws have no fix — both need a family account or a leaked
+     photo link; the fix rides its big step. *Operator, 2026-10-27 21:33 (CC's pick).*
+
+273. **D57 = keep Gitea's repository import on** (R-1315) — sign-up is closed; judged again in its big step. *Operator,
+     2026-10-27 21:33 (CC's pick).*
+
+274. **D58 = Termix waits for its big step** (R-1311); the forged-address header is NOT stripped now — stripping it lets
+     a stranger lock the family out (R-775). *Operator, 2026-10-27 21:33 (CC's pick).*
+
+275. **D59 = hide Wanderer from new installs until its big step is done** (R-1313) — any server on the internet can
+     overwrite its records with no login; no box runs it; an installed copy is not touched. *Operator, 2026-10-27 21:33
+     (CC's pick).*
+
+276. **A Home Assistant monthly release (2026.9 → 2026.10) is the SAME main version for the ladder, so a security fix
+     may ride the lane.** *Decided by CC unattended 2026-10-27 — operator may reverse.* Options: A) the lane — the one
+     comparator (`stacks.CompareImageRefs`, Q3) parses „2026" as the major, a notice id exists (CVE-2026-111497, Python
+     Brotli 1.2.0 → 1.2.1 inside the image); cost: a monthly HA release carries feature changes into the night lane;
+     B) a normal step — cost: a second rule beside the comparator for one app's version shape (Q3 warns that is how a
+     major gets automated by accident). Why A: the brief („a normal step if it counts as a main version in our ladder
+     rules, else the lane") and the one-comparator rule; the box walk + soak is the guard either way.
+
+### 2026-10-28 (evening, attended) — D57 judged again in Gitea's big step: imports from three hosts only
+
+280. **Gitea's repository import is limited to github.com, codeberg.org and gitlab.com** (`[migrations] EGRESS_MODE=strict`,
+     `ALLOWED_HOST_LIST`, Gitea 28's own egress rules) — this replaces D57's „keep import on, unrestricted" (decision 273).
+     Measured on 9202 with 28.2.0: the default (`lax`) already refuses home-network and Docker addresses (422); the
+     allow-list also refuses every other public host (gitea.com 422) and lets GitHub import (201). Options: the allow-list
+     (cost: an import from a self-hosted forge or Bitbucket is refused) or 28's default (cost: the box fetches any public
+     address a family member names). *Operator, 2026-10-28, at the Gitea STOP; CC's pick.*
+
+### 2026-10-28 (afternoon, attended) — the four big steps; D59 ends; Wanderer's inbound federation stays off
+
+279. **D59 ends: Wanderer is offered again** after its big step v0.20.0 → v0.21.2 went live (catalog `1825247`), and
+     **its inbound federation stays OFF** — the template sets no `POCKETBASE_PROXY_SECRET`, so v0.21 rejects every
+     inbound ActivityPub delivery (fail closed, upstream's docs). Options: off (a family instance takes nothing from
+     strangers' servers; the route GHSA-fprh abused stays shut) or on (a generated shared secret; other servers deliver
+     follows, comments, updates). *Operator, 2026-10-28, at the Wanderer STOP; CC's pick both times.*
+
+**[DESIGN] A pin move needs its step — also when no box runs the app (2026-10-28, stated here because the brief asked
+for „the rule for a pin change when no box runs the old version" and this document did not say it).** The rule lives in
+the catalog: `app-catalog-felhom.eu/CLAUDE.md` („An `image:` move needs its TEST RECORD") and its gates —
+`check-test-record.py` (the newest ladder entry IS the compose) and `check-test-record-move.py` (a moved ref must add a
+proven entry). There is no exception for an app no box runs: the writer (`upgrade-test.py --write-ladder`) refuses
+unless the bench AND the box walk say `proven`, so every big step is a normal ladder step. A big step's UNDO is proven
+on 9202 as well (the 2026-10-27 Jellyfin method, generalised in `audits/dockerhub-bigsteps-2026-10-28/undo_proof2.py`):
+TO with failing checks must end `undone` and the seed must read back on FROM. **The broken probe must fail on any app:**
+a single-page app answers 200 on every path, so a „no such path" probe passed and Termix's update ended `done`; the
+tool now probes port 1. `test-record-move` also refuses a `memory_tight` step whose range does not raise the limit —
+Wanderer's Meilisearch went 256 → 384 MB and was measured again (it sizes itself to its limit: 88.8 %).
+
+### 2026-10-29 (morning, attended) — D62: an unused restore copy stays 30 days, and the household may delete it; D63: prove or revert
+
+281. **D62 = an unused restore copy is kept 30 days, and the household can delete it earlier** with one button
+     („Törlés most") in the kept-copies list on the restore page (place A), after a warning that names the app and the
+     restore's date and the app's name typed. A copy a hold or a pending put-back needs is never deleted — not by the
+     sweep, not by the button (it is off, with the reason in one line); a delete while a backup or restore runs is refused.
+     One log line and one operator event (`restore_copy_deleted`: who, app, date, size) per delete. Replaces the night's
+     14 days (decided by CC unattended). *Operator, 2026-10-29 08:17.* `07` §6.3.
+282. **D63 = R-1256 H4 and H5's failed-stop branch are released only with a live proof; else they are reverted** (their
+     tests stay, skipped with the reason) before the controller tag. The proof uses a fault switch that exists only in a
+     `-tags faultinject` test build — never in the release image (`TestD63_TheReleaseBuildHasNoFaultSwitch`). *Operator,
+     2026-10-29 08:17.* `07` §6.3.
+
+### 2026-10-29 (12:10, operator) — D64 = B: the hub may clear any hold
+
+283. **D64 = B — the hub's „Clear hold" works for every hold reason, with no operator signature.** The operator accepted
+     that a compromised hub could clear a data-protecting hold (an app could start on half-restored data); the confirm
+     step shows the hold's reason and, for a restore or put-back hold, that warning before the press. R-379 now reads
+     „a person — on the box, or the operator on the hub". Built as the controller's `clear_hold` operator action
+     (`04` §6.1), cleared by the running controller (R-1343). *Operator, 2026-10-29 12:10.*
+
+### 2026-10-28 (day, attended) — RomM kept; the runtime cache fills itself in CI
+
+277. **D61 = keep RomM offered** (R-1330) while no RomM release carries Node ≥ 24.22.1; the first one that does is taken
+     as a lane step (CVE-2026-111497). No Node process runs in RomM (gunicorn, rq and nginx — `ps` on demo-hp 9201) and the
+     setup gate is in front; the watch flags it every night until then. *Operator, 2026-10-28 (starting the day brief),
+     CC's pick.*
+
+278. **The watch reads zstd layers in CI with a hash-pinned `zstandard` wheel, not with `zstd` in the CI runner image.**
+     *CC, 2026-10-28, the brief's choice („say which and why").* Options: A) the wheel (0.25.0, PyPI's sha256 per
+     Python/libc, checked on every open, fetched into the runner's cache) — cost: a download from PyPI when the cache is
+     empty, and a pin to renew when the runner's Python moves; B) `zstd` in `felhom-act-runner` — cost: a homelab-repo
+     change, an image rebuild and a k3s rollout on DooPlex, which the brief forbids, and every CI job depends on that
+     image. Why A: no DooPlex change, and a wrong or missing wheel is NOT CHECKED out loud, never a pass (`08` §11).
+
+**The lane batch of 2026-10-28 (R-1332), how it ran — the method, not a new rule.** Twenty apps carried a bad runtime
+(the watch's first full read). For each pin the registry's tags in the pinned tag's own shape were listed, and the newest
+one in the same main version and the newest overall were READ FROM THE IMAGE (`image_runtime.py`), never from release
+notes (`audits/runtime-fixes-2026-10-28/TABLE.md`). Three states: a fixed release in the same main version → a
+security-fix step; a fix only in a new main version → the app's big-step row carries it; no fixed release → a row with
+the exposure. The main version is the tag's first number, and for a `0.x` app the minor (Homebox 0.26 → 0.27 on
+2026-10-27; Karakeep 0.33 → 0.34 today). Batches of about five, each walked on 9202 (`box_walk.py --security`, soak
+600 s, the seeded data read back), the writer run on the live catalog, `catalog_gates.py --fast`, and a STOP before each
+push. None of the eight apps has a bench fixture, so no bench ran (§6.4, the lane's stated trade). Two tool gaps found
+on the way were fixed in the session: the drill mover moved two services that share one image only half (Dawarich), and
+a drill commit made outside a walk (the revert to FROM) never reached the box's mirror before the install.
+
+### 2026-10-27 (attended release block) — the line-test hold is fixed the same day; R-1218's live proof is waived
+
+269. **The line-test hold waits until the held app has really slowed down — fixed in the controller the same day.**
+     *Operator, 2026-10-27 12:30 CET.* Options: A) fix in the controller now, prove it on an rc on demo-hp's 9201 while the
+     seed runs, ride today's 0.346.0 — cost: ~1.5 h, the release later in the afternoon; B) file it and fix later — cost:
+     a box that seeds all day keeps getting no weekly test. Why A: measured four refusals in an hour (qBittorrent sends
+     ~14 s after its limit falls; the agent reads 5–15 s after the call). Built `710ceac`; live on `0.346.0-rc.1`: 94.5
+     Mbit/s, the limit back exactly (`audits/linetest-2026-10-27/`).
+
+270. **R-1218 is released without its live proof (rule 2 waived for this row).** *Operator, 2026-10-27 15:15 CET.*
+     Options: A) keep both halves, waive the live proof — cost: released key code proven by red tests only; a box that
+     lost its key file entirely now gets a 409 on its ceremony instead of a silent demotion; B) revert both halves — cost:
+     a hub 0.173.1 and a second deploy, and the demotion bug stays. Why A: no product path makes the box's saved off-site
+     target „off" on a hub box (the hub page's switch did not reach Tester 1's target; the household's clear is refused on
+     a hub tier), so no live run can tell the old code from the new; the change only widens a safety step and adds a
+     refusal. R-1312 asks what the hub switch does on the box.
+
+### 2026-10-26 (evening brief) — D53: held apps reach the operator; D54: demo-hp's stuck clean-up; R-1297 (B): a second CI runner
+
+265. **A held app reaches the OPERATOR first, never the household (D53 = operator-only reminders first).** *Operator,
+     2026-10-26 evening (starting the brief is the yes).* Options: A) operator only — cost: a household whose app is held
+     still learns only from the app's page; B) the operator and, for the two holds a household can clear, the household
+     too — cost: two more customer mails to write and translate, about a stop they may not understand. Why A: the
+     operator can act on every reason; household mails can follow once the operator list has run. Built from `08` §10.
+266. **demo-hp's off-site store gets ONE clean-up window with a raised cap of 220, through the hub's existing one-shot
+     grant; the cap rule itself is unchanged (D54 = prune once).** *Operator, 2026-10-26 evening.* Options: A) a one-shot
+     grant — cost: one window prunes 209 snapshots at once on a test box (the plan was read honest, R-1294); B) change the
+     cap rule — cost: hub and controller code, and a weaker guard against a mass deletion on every box. Why A: the guard
+     stays as strong as it is; the grant expires after one use.
+267. **CI gets a second runner on DooPlex; both runners have 2 CPUs (R-1297 B).** *Done by the operator, 2026-10-26.*
+     Options: A) a second runner — cost: DooPlex CPU while two jobs run; B) only trim the suites — cost: every repo's job
+     still queues behind felhom.eu's run. Why both (the operator's „both"): the trim makes one run short, the runner stops
+     the queue. The trim (A) is CC's and keeps every check (path filters, a tag run and a nightly run).
+
+### 2026-10-26 (attended release run) — D51: Kimai's hold; D52: the missed weekly golden
+
+263. **demo-hp's Kimai, held by the crash-loop guard since the 2026-10-14 power drill, is started again, not removed
+     (D51 = Start).** *Operator, 2026-10-26 07:57.* Options: A) Start — cost: if it crash-loops again it is held again;
+     B) remove it through the product — cost: the test app and its data are gone. Why A: the household's Start is the
+     designed way to lift this kind of hold (decision 28), and it is a test box. Result: hold lifted, healthy (R-1286).
+264. **The missed Sunday bake is made up the same day, on the new controller, and the waiver covers that one day only
+     (D52 = bake today on v0.344.0).** *Operator, 2026-10-26.* Options: A) renew the waiver to the next Sunday — cost: a
+     week of new installs on a golden 13 releases behind; B) bake today — cost: about an hour of the release day. Why B:
+     the gate goes green by a fresh golden, not by a waiver. The weekly Sunday bake (`runbooks/nightly-rotation.md`)
+     stays the rule; every Sunday-night brief names it.
+
+### 2026-10-25 (evening, attended) — D50: which agent packages the registry keeps
+
+262. **The registry keeps the newest 5 released agent versions, plus the hub's MinAgent floor, plus the version the hub's
+     day-0 manifest names; CI asserts exactly these (D50 = A).** *Operator ruling 2026-10-25 evening.* Options: A) this
+     — cost: a box on a version outside the set cannot fetch its own binary again (none is, on the day); B) the newest 10,
+     never pruned by hand — cost: today's four are gone, so CI stays red until ten new versions exist. Why A: it is what
+     the operator prunes to, and the two pinned versions are the only old ones anything still asks for. Built: agent
+     `scripts/retention-policy.json` (`generic_versions_kept` 5, `pinned`) + `check-published-versions.py`; the pins are
+     copies of hub state, updated by the session that records a vouch or a new MinAgent (R-1290).
 
 ### 2026-10-25 (attended release run) — D49: a restore that does not start keeps both copies; D44–D46: the review picks
 

documentation/architecture/11-os-updates.md

@@ -79,11 +79,19 @@ boxes installed. Nobody keeps a hand-written list.
   **[DESIGN, operator ruling D43 = A, 2026-10-24 — committed agent `c8d3c17`, unreleased]** The host's root-key-only sshd
   drop-in (D4, R-982) is an APPLIANCE-only file of the config bundle: written only when the root-owned install record
   says `appliance`; on a `byo` record nothing is written and a copy the bundle wrote earlier (this path, our first line)
-  is removed; a box whose record proves nothing is left as it is (R-1257).
+  is removed; a box whose record proves nothing is left as it is (R-1257). **[FACT] 2026-10-25 (BYO test host, agent
+  0.164.0 → 0.165.0):** the removal came one bundle LATE — the bundle that brings the new wrapper is planned by the OLD
+  one (R-1285). **[DESIGN — R-1285, agent `abcc1e6`, unreleased]:** right after every bundle install the agent asks the
+  wrapper now on disk for a policy pass (`mode: "bundle-policy"`), which removes our appliance-only files from a BYO
+  record (no signature: it only removes a file we wrote; a failing `sshd -t` puts it back).
+  **[DESIGN — R-1285 follow-up, agent `b1bc63d`, unreleased]:** in the normal delivery the bundle job runs in the OLD
+  process (measured on demo-hp 2026-10-26: the bundle 09:03:07–09:03:09, the restart into 0.166.0 at 09:03:10), so the
+  new agent also runs the pass once right after its self-update commits (`selfupdate.Manager.SetOnCommitted`). Only a
+  box coming from 0.164.0 can show it: 0.165.0's wrapper already applies D43 at plan time.
 - **The Proxmox MAJOR upgrade** (PVE 9 → 10). It is a later step of its own, drilled first (R-808 item 5).
 - **`cloudflared` and the other infrastructure images** (traefik, filebrowser). **[FACT] (C8)** They are containers in
   the guest, pinned in the controller (`internal/infra/infra.go:26`: `cloudflare/cloudflared:2026.6.0`, since
-  2026-06-11 — *`2026.9.3` today, corrected 2026-10-24*) and baked into the golden; upstream was `2026.9.3` on 2026-10-04. They move only by a controller release
+  2026-06-11 — *`2026.9.3` from 2026-10-24; `2026.10.3` since controller v0.345.1, 2026-10-26 — Cloudflare's eMudhra notice, R-1300*) and baked into the golden; upstream was `2026.9.3` on 2026-10-04. They move only by a controller release
   — the app-image question (`09`), not an OS package. The gap is R-838.
 
 ---

documentation/architecture/12-agent-tooling.md

@@ -72,4 +72,7 @@ secret rule or a review step, or remove a rule — that is the operator's. The f
 that uses a value by NAME and never prints it, and Claude Code hooks in `~/.claude/settings.json` — so every CC
 session on DooPlex — that refuse the known leak shapes, mask stored values in tool output, and a nightly sweep of old
 session records that mails NAMES only. Built 2026-10-17 (R-1118); evidence `audits/cc-creds-build-2026-10-17/`.
+**D60 (operator 2026-10-27 21:33, `09` §3 decision 271): CC never deletes from the store.** `sec rm` is the operator's
+command; the hooks refuse it in every form, and any removal or overwrite of the store file or the age key (Bash and a
+`pre-write` Write/Edit hook, cc-secrets ≥ 0.4.0).
 `runbooks/secrets.md` stays the home of the CLUSTER Secrets.

documentation/architecture/13-threat-model.md

@@ -84,7 +84,13 @@ directly; the only guard is that the header NAME is a per-box secret, which trae
 client request and sets only for a browser holding the dashboard's files pass
 (`felhom-controller/controller/internal/infra/filebrowser_sso.go:15-22`, `internal/web/files_sso.go`). The name file's
 creation race was fixed by review round 2 (controller `b01bb4b`). The pick: FileBrowser joins only the network traefik uses, so no app
-container reaches it directly. Not built yet.
+container reaches it directly. **Built 2026-10-25 night (controller `7ea942a`, unreleased):** a network of its own for
+the route, `felhom-files` — traefik is on it, the CHECKED FileBrowser joins it by id after R-1126's default-login check,
+and FileBrowser leaves traefik-public; OnlyOffice joins it while — and only while — its FileBrowser link is on (it
+fetches and saves documents at `http://filebrowser:80`), the one stated exception. **[FACT] measured on the night's rig:**
+before, from the qBittorrent and Transmission containers the forged header as the household user got the drive's
+listing (200); after, no answer (`000`); the household's „Fájlok ↗" walk still gets 200 through traefik
+(`audits/night-2026-10-25/rig/r1281-*.txt`).
 
 ## 6. What we accept (do not re-report)
 

documentation/architecture/15-media-library.md

@@ -169,8 +169,10 @@ download (R-1019). *(Fixed since: catalog `c50adfc` marks the download and watch
 `app-catalog-felhom.eu/templates/transmission/.felhom.yml:46-51`; noted 2026-10-24.)* **[DESIGN — the operator picked a code change 2026-10-25 (D44); row R-1283]:** the same commit also
 marks the download folders of NZBGet, SABnzbd and MeTube `excluded` (`templates/{nzbget,sabnzbd,metube}/.felhom.yml`).
 NZBGet and SABnzbd stay excluded (their files can be fetched again). The pick for MeTube: `optional`, so the
-second-drive copy carries it and the off-site copy does not — a MeTube video can be gone from its source site. Not
+second-drive copy carries it and the off-site copy does not — a MeTube video can be gone from its source site.
-built yet. **[DESIGN]** The per-drive media bind is declared `excluded` for every media app — the family's
+**Built in the DRILL catalog 2026-10-25 night (`app-catalog-drill` `6733764`, not live):** `class: optional`, and the
+download-binds gate allows `optional` only for an app named with its ruling (`OPTIONAL_BY_RULING`); mandatory and
+unlisted stay refused. The live push is the operator's (the Monday list). **[DESIGN]** The per-drive media bind is declared `excluded` for every media app — the family's
 media is no single app's data; whether the library gets its own household-level backup is a separate decision.
 
 ## 8. Removal and export assume one drive per app
@@ -275,7 +277,9 @@ returns** (G7f).
 **[DESIGN — the operator picked a code change 2026-10-25 (D44); row R-1282]:** the „Később"
 03:30 media restart is an IN-MEMORY timer — a controller restart forgets it and only the notice stays
 (`internal/web/media_events.go` `handleMediaRestart`, `time.AfterFunc`). The pick: the controller stores the due
-time and runs the restart after a restart of its own. Not built yet.
+time and runs the restart after a restart of its own. **Built 2026-10-25 night (controller `b5863a9`, unreleased):**
+`settings.media_restart_later_at`; at start `ResumeMediaLater` re-arms it, runs it at once when it passed while the
+controller was down, and clears it once it ran.
 
 **[DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265)]:** the Sonarr/Radarr API key is READ from
 `/config/config.xml`; the „set at install" option §6 calls cleaner was not built (`internal/mediawire/mediawire.go:336-345`).

documentation/architecture/capability-map-detail/B.md

@@ -392,6 +392,10 @@ four version observables at TO, a 600 s memory soak), and the only writer turns
 ladder step. Every box on controller >= 0.341.0 takes it at night with the usual guards (backup first, undo on a
 failed health check, held / stopped / needs_person / W+5h respected).
 
+**Trigger (2026-10-26, R-1303):** a mail from the nightly security watch („[Felhom security] …", `08` §11) starts the
+lane: it names the program, our version and the fixed version. An app notice goes this lane; a box program's (traefik,
+cloudflared, the file manager) is a controller release — the cloudflared 2026.10.3 run (R-1300) is the worked example.
+
 **Components:** catalog `3898593` (`ladder.py check_security`, the two test-record gates, `upgrade-test.py --write-ladder
 --security`, `box_walk.py --security`); controller v0.341.0 (`legCandidate`).
 
@@ -405,4 +409,10 @@ a hand press took the step; on 0.341.0-rc.1 the night leg logged `plex: security
 **Gap:** an app with no box fixture (Plex today, R-1238) is checked only by health, version and the soak — no data is
 read back. `files_may_change` is a bench measurement and reads `false` when no bench ran (decision 255 item 7).
 
+**Batch use (2026-10-27, `audits/security-app-updates-2026-10-27/`):** 14 security-fix steps in three batches, each
+pushed live after the operator's yes at a batch STOP (catalog `06e603e`, `7b0071e`, `65a6b11`); 12 read the seed back
+after the update, 2 are `seed: none`. demo-hp's 9201 read `06e603e` at once and holds RomM's step for its night leg
+(`live-readback-batch1-9201.txt`). Notices with no CVE/GHSA id go as normal steps in the same batches (wishlist,
+bookstack). Four stale fixtures were found and fixed by the walks' `inconclusive` C1 (`09` decision 255, batch note).
+
 ---

documentation/architecture/capability-map-detail/E.md

@@ -142,7 +142,9 @@
 
 **Evidence (as written):** New FIRST sidebar page `/launcher`: colored tiles (deterministic slug color or `.felhom.yml` `brand_color`) + white glyph/monogram, one per openable app (tile ⟺ „Megnyitás" — subdomain presence is the single criterion; controller excluded). Operational → `<a target=_blank>` to the public URL; stopped → greyed + state badge, no link. `/` stays the Vezérlőpult. Endpoint-level + render-test verified; `felhom-controller/REPORT.md` (2026-07-24)
 
-**Gap / roadmap (as written):** Live operator click-through of a real tile → app pending (browser automation not available on DooPlex). Follow-up: curate `brand_color` for top catalog apps (R-72). Sharing the launcher outside the household is now the capability-URL guest link — see the row below
+**Walked 2026-10-28 night (9202, endpoint-level — the page and links the browser receives):** `/launcher` answered 200 with one tile per openable app (`hike`, `paperless`, `tasks`), each a link to the app's public address; the controller has no tile. `audits/night-2026-10-28/part5-claims/launcher-walk.*`.
+
+**Gap / roadmap (as written):** Live operator click-through of a real tile → app pending (browser automation not available on DooPlex; the endpoint walk above stands in). Follow-up: curate `brand_color` for top catalog apps (R-72). Sharing the launcher outside the household is now the capability-URL guest link — see the row below
 
 ---
 
@@ -157,7 +159,7 @@
 
 **Status (as written):** **IMPLEMENTED**
 
-**Evidence and gap (as written — this row's text carries a `|` outside code, so its last cells are kept together as they stood):** 160-bit `crypto/rand` token, constant-time match (empty stored = disabled = byte-identical to the mux default 404); guest headers `noindex`/`no-referrer`/`no-store`; optional SEPARATE bcrypt share password + its own per-IP attempt map; signed cookie `HMAC(token`\ | `passwordHash)` keyed with `session_secret` (rotate-token OR change-password invalidates all cookies); token redacted in logs (`/s/<redacted>`). Groups A–G (14 tests) + 3 red-proofs; **§13 endpoint-level live validation on 9201 all-pass** (`felhom-controller/REPORT.md` 2026-07-24). **Design ruling: member accounts SUPERSEDED by this capability-URL model; per-member tile visibility parked under the SSO arc (R-15).** | Full operator browser click-through + a validation doc pending → then PROVEN-LIVE. Accepted residuals: link-preview crawlers fetch once (noindex prevents indexing); reverse-proxy/CF access logs hold the path (ops-tier); the modal link carries the request Host (LAN-IP admin ⇒ LAN-IP link)
+**Evidence and gap (as written — this row's text carries a `|` outside code, so its last cells are kept together as they stood):** 160-bit `crypto/rand` token, constant-time match (empty stored = disabled = byte-identical to the mux default 404); guest headers `noindex`/`no-referrer`/`no-store`; optional SEPARATE bcrypt share password + its own per-IP attempt map; signed cookie `HMAC(token`\ | `passwordHash)` keyed with `session_secret` (rotate-token OR change-password invalidates all cookies); token redacted in logs (`/s/<redacted>`). Groups A–G (14 tests) + 3 red-proofs; **§13 endpoint-level live validation on 9201 all-pass** (`felhom-controller/REPORT.md` 2026-07-24). **Design ruling: member accounts SUPERSEDED by this capability-URL model; per-member tile visibility parked under the SSO arc (R-15).** | **Walked 2026-10-28 night on 9202 (endpoint-level, `audits/night-2026-10-28/part5-claims/launcher-walk.*`):** switched on from the launcher (303 `flash.share.enabled`); the link opened with NO session → 200, the household's app tiles, no admin link; a wrong token → 404; rotate → the old link 404, the new one 200; switched off → 404 (it was off before the walk). NOT-A-FINDING: an unknown non-`/s/` route answers an anonymous visitor 302 to the login page, so a wrong token is told apart from an unknown ROUTE (404 vs 302) — what the design hides is a wrong token from a disabled one, which both answer 404. → PROVEN-LIVE. Accepted residuals: link-preview crawlers fetch once (noindex prevents indexing); reverse-proxy/CF access logs hold the path (ops-tier); the modal link carries the request Host (LAN-IP admin ⇒ LAN-IP link)
 
 ---
 

documentation/architecture/where-felhom-stands.html

@@ -6,17 +6,17 @@
 </style></head><body><div class="wrap">
 <h1>Where Felhom stands</h1>
 <div style="color:#8fa0b8;font-size:12.5px;margin:7px 0 4px;max-width:1100px;line-height:1.5">What we built, what happens when things go wrong, and what is still missing. <b style="color:#cbd7e8">Generated from <code>where-felhom-stands.yaml</code></b>, which cites the capability map, register row or evidence document behind every claim — and which is checked by <code>scripts/check_stands.py</code>.</div>
-<div style="color:#6b7a91;font-size:11.5px;margin-bottom:16px">Verified 2026-08-22 against felhom-agent <code>40d857b527</code>, felhom-controller <code>2da259af38</code>, hub <code>877fcd2a38</code>. <b style="color:#fbbf24">15 claim(s) carry a recorded status move</b> (8 re-checked in this pass) — each is marked on the page.</div>
+<div style="color:#6b7a91;font-size:11.5px;margin-bottom:16px">Verified 2026-08-22 against felhom-agent <code>40d857b527</code>, felhom-controller <code>2da259af38</code>, hub <code>877fcd2a38</code>. <b style="color:#fbbf24">17 claim(s) carry a recorded status move</b> (7 re-checked in this pass) — each is marked on the page.</div>
-<div style="display:flex;gap:22px;flex-wrap:wrap;background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:11px 14px;margin-bottom:18px"><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Walked</b> <span style="color:#7c8aa3">(20)</span><span style="display:block;color:#6b7a91;font-size:10.5px">done end to end on real hardware, evidence on file</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Built</b> <span style="color:#7c8aa3">(15)</span><span style="display:block;color:#6b7a91;font-size:10.5px">shipped and tested, the real path never walked</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Partial</b> <span style="color:#7c8aa3">(16)</span><span style="display:block;color:#6b7a91;font-size:10.5px">some walked, some not — the note says which</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Missing</b> <span style="color:#7c8aa3">(4)</span><span style="display:block;color:#6b7a91;font-size:10.5px">does not exist</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="font-size:11.5px"><b style="color:#7dd3fc">needs-hardware</b><span style="display:block;color:#6b7a91;font-size:10.5px">only a running box could settle it</span></span></div></div>
+<div style="display:flex;gap:22px;flex-wrap:wrap;background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:11px 14px;margin-bottom:18px"><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Walked</b> <span style="color:#7c8aa3">(24)</span><span style="display:block;color:#6b7a91;font-size:10.5px">done end to end on real hardware, evidence on file</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Built</b> <span style="color:#7c8aa3">(12)</span><span style="display:block;color:#6b7a91;font-size:10.5px">shipped and tested, the real path never walked</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Partial</b> <span style="color:#7c8aa3">(15)</span><span style="display:block;color:#6b7a91;font-size:10.5px">some walked, some not — the note says which</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Missing</b> <span style="color:#7c8aa3">(4)</span><span style="display:block;color:#6b7a91;font-size:10.5px">does not exist</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="font-size:11.5px"><b style="color:#7dd3fc">needs-hardware</b><span style="display:block;color:#6b7a91;font-size:10.5px">only a running box could settle it</span></span></div></div>
 <div style="font-size:15px;font-weight:600;margin:0 0 9px">The journey</div>
 <div style="color:#6b7a91;font-size:11.5px;margin-bottom:10px">Seven stages, left to right, as they happen to a person.</div>
 <div class="grid7">
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">1</span> &nbsp;Getting a box</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A blank machine installs itself from our own boot image and registers itself as unclaimed — proven on two different boards<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Two boards: N100 2026-07-18, HP t740 2026-07-21.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Bare-metal Felhom ISO (blank hardware → zero-touch auto-ins… · evidence tests/VALIDATION-n100-rehearsal-2026-07-18.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The installer is published rather than pushed: rolling it back is one act<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Gate 6 of hostinstall_gates asserts the manifest names an installer-v tag; ran green tonight.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: gate 6 asserts the manifest names an installer tag, but no walk of a rollback is on file</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map The installer is PUBLISHED, not pushed · register R-110</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Installing onto hardware the customer already owns — the path exists, the first real one has not happened<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">UPGRADED: a real --mode byo install completed on demo-hp 2026-08-09 (Day-0 provision SUCCESS, 3m49s). Still not a customer's own hardware, so not 'walked' — but 'has not happened' is now false.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A machine that ends up on the wrong network port explains itself on screen and finds its way back — proven in a virtual drill, never on metal<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Map says the same: PROVEN-LIVE (nested drill — nested != metal).</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Box survives a wrong-NIC install · evidence audits/SPIKE-firstboot-nic-sweep-2026-07-22.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reinstalling a machine we previously installed refuses, because our own removal leaves a name service holding the port our own installer checks<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. Confirmed live 2026-08-09; dnsmasq restarted by --uninstall seizes :53.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-272 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">2</span> &nbsp;Making it theirs</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An emailed one-time code; the customer sets their own password and the operator never sees it<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Exercised live 2026-08-09: reset code accepted at /claim, 302 + session.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Customer claim: one-time emailed code → customer sets own p… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The customer can bind their own machine from a link, with no operator present — done once, for real<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">attempts=0, locked=0, source customer_selfbind, 2026-07-18.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Customer binds their own appliance (self-service) · evidence tests/VALIDATION-n100-rehearsal-2026-07-18.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">It has never been done by a person who is not the operator<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Map records MISSING (as evidence). Still true after 2026-08-09.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A customer (not the operator) performs a restore via UI alone</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Each of the two secrets has one name on every surface, and the re-enrol mail points at a page a rebuilt machine shows<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Shipped (controller v0.211.0, hub v0.104.0/v0.105.0); no customer has typed the passphrase yet. Operator ruling 2026-10-12 (S13). The old defect cost a real code on 2026-08-09.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-282 · register R-295 · register R-323 · register R-327</span></span></div></div>
-<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">3</span> &nbsp;Using it</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Apps installed from a catalog of about 52, with a memory guard and health-aware progress<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">53 templates listed on the rebuilt box 2026-08-09; deploy driven live.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Deploy an app from the catalog (env config, memory guard, h… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Start, stop, restart, update, logs, remove — and the parts that must not be stopped cannot be<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited by the map row or anywhere else</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App lifecycle: start/stop/restart/update/logs/remove/redeploy · register R-108</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reachable from anywhere through a tunnel, per-app addresses<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Verified tonight: the rebuilt box answered on its public URL from outside.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Remote access via Cloudflare Tunnel + Traefik (per-app subd… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reachable on the home network when the internet is down<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#7dd3fc;font-size:10px;white-space:nowrap">needs-hardware</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Only an observation on a running box with WAN pulled could settle it. Boxes are off.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map LAN access when internet is down (lan_resolver)</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Phone photos, documents with text recognition, files from Windows Explorer or a Mac<span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Files from Windows Explorer / Mac Finder (SMB server) · evidence audits/SPIKE-lan-discovery-2026-07-18.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A one-tap launcher, and a read-only guest link for visitors<span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Indítópult (app launcher) — one-tap grid of the household's…</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-heigLine truncated
+<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">3</span> &nbsp;Using it</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Apps installed from a catalog of about 52, with a memory guard and health-aware progress<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">53 templates listed on the rebuilt box 2026-08-09; deploy driven live.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Deploy an app from the catalog (env config, memory guard, h… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Start, stop, restart, update, logs, remove — and the parts that must not be stopped cannot be<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was built</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Walked 2026-10-28 night on scratch 9202: install, stop, start, restart, logs; traefik and cloudflared refuse a stop (403); remove with data leaves no volume and no container. The update half: the same day's guarded updates on 9202 (bench + box + undo walks).</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: a walk document is now cited (2026-10-28 night)</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App lifecycle: start/stop/restart/update/logs/remove/redeploy · register R-108 · evidence audits/night-2026-10-28/part5-claims/lifecycle-walk.log · evidence audits/dockerhub-bigsteps-2026-10-28</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reachable from anywhere through a tunnel, per-app addresses<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Verified tonight: the rebuilt box answered on its public URL from outside.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Remote access via Cloudflare Tunnel + Traefik (per-app subd… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reachable on the home network when the internet is down<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#7dd3fc;font-size:10px;white-space:nowrap">needs-hardware</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Only an observation on a running box with WAN pulled could settle it. Boxes are off.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map LAN access when internet is down (lan_resolver)</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Phone photos, documents with text recognition, files from Windows Explorer or a Mac<span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Files from Windows Explorer / Mac Finder (SMB server) · evidence audits/SPIKE-lan-discovery-2026-07-18.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A one-tap launcher, and a read-only guest link for visitors<sLine truncated
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">4</span> &nbsp;Drives</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A new drive is found, offered, formatted, mounted and enrolled — including on awkward older boot layouts<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">moved twice: 2026-08-09 walked -&gt; built, then 2026-08-10 back to walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Applies to a NEW drive. Re-attaching an existing one after a reinstall is R-280 and fails.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: receipt found 2026-08-10: a live throwaway drive on felhom-pve walked scan -&gt; format -&gt; mount -&gt; PVE dir storage -&gt; one-click Regisztralas, with the resulting storage.cfg entry and systemd mount unit recorded. The map already read PROVEN-LIVE; the dataset was behind it. Scope unchanged: this is a NEW drive. Re-attaching an existing one after a reinstall is R-280 and still fails.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Drive wizard: scan/format/mount/enroll, incl. legacy-boot L… · register R-220 · evidence audits/SPIKE-raw-drive-enroll-2026-06-15.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Moving data between drives, crash-safe; removing a drive safely; unplug detected<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Data migration between drives (all / per-app), crash-safe</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A network drive can be browsed and hold bulk media, but may not hold an app's data — enforced<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">RefuseAsAppNamespace is the fail-closed predicate.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Network storage (NAS) is browse + bulk-media only · register R-108 · evidence audits/R108-network-app-namespace-2026-07-30.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">After a reinstall the data drive cannot be re-attached through any dashboard route<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. /api/disks/candidates returns empty; the restore page promises two clicks.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-280 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">5</span> &nbsp;Backing up</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">App data on the machine, nightly database dumps, a copy on a second drive<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">moved twice: 2026-08-09 walked -&gt; built, then 2026-08-10 back to walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">True for every app but one, and that one was found on 2026-08-21: paperless-ngx's 72-table PostgreSQL was dumped nightly and written into a folder named after an app that does not exist, so it never entered the recovery unit, the second-drive copy or the off-site copy. Fixed and proven live 2026-08-22 (R-355) — the dump is now in the app's own unit and in the off-site snapshot for the first time. A catalogue-wide sweep, itself proven able to convict a planted case, says 1 of 53 was affected.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: receipt found 2026-08-10: an adversarial, destructive, unattended overnight campaign across both boxes and ep0, with A2 (one quiesce, two tiers) proven end to end. Map already read PROVEN-LIVE.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Tier-2 secondary-drive copy: class-driven legs · register R-355 · evidence audits/CAMPAIGN-8-backup-restore-2026-07-27.md · evidence audits/DRILL-backup-truth-2026-08-21/evidence/phase4-paperl…</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A whole-machine archive that lands off the guest's own disk — a single-drive machine is recorded as degraded rather than pretending<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Whole-guest backup lands OFF the guest's own physical device · register R-165</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An encrypted off-site copy, sealed with a key the operator cannot read<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The COPY is real and the key is still unreadable to us — what failed is DAILY and UNBROKEN. The 2026-08-09 note said '18 snapshots, daily, unbroken'; the next snapshot after 2026-08-09 08:30 was 2026-08-21 22:17, put there by hand during the drill. Twelve days, no alarm. Two causes in series: the 2026-08-21 rebuild lost the off-box target (R-193's shape), and after the self-heal restored it EVERY per-app switch was still off, so the first run logged 'backup OK: 0 app(s) backed up, 14s'.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the claim is about a CONTINUING daily copy; a 12-day silent gap was found on 2026-08-21 and nothing reported it</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Offsite (restic → Hetzner Storage Box) · register R-199 · register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The backups prove themselves: a restore is actually performed, uLine truncated
-<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">6</span> &nbsp;When something goes wrong</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The machine watches itself and repairs some faults without telling anyone it had to<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">R-264 records that the self-heal counters reach the hub and are decoded nowhere.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Box survives an unattended app or guest-network failure · register R-264</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the operator by email, one mail per run, every failing app named<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED against source: backup_run_failures digest is allowlisted (api/handler.go:1837), operator-only (dispatcher.go:423), templated (templates.go:48); recovery_unit_capture_failed is record-only (dispatcher.go:376). NOTE: R-182's register row still describes the PRE-FIX behaviour — see R-289.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the digest is wired and source-verified, but no run of it has been observed delivering</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A failed per-app Tier-1 backup reaches the OPERATOR — EVERY… · register R-182</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the customer<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The customer leg is built; today's log shows customer-channel rows skipped as operator_only.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App crashes → customer notified (one event per transition, …</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An already-paired box is still told to pair itself<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-214 · register R-235</span></span></div></div>
+<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">6</span> &nbsp;When something goes wrong</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The machine watches itself and repairs some faults without telling anyone it had to<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was built</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Walked 2026-10-29 night on scratch 9202: a crashed app part (kill -9 of paperless-redis) ran again in 1 s and the health probe read the app healthy; a guest reboot brought every app back in under 2 minutes, the boot sweep found nothing to start; no event, no banner, no mail. Docker's restart policy did the repair in both legs; the controller's own boot repair had nothing to do (its last proof: 6 of 6 hard resets, 2026-08-02). R-264 records that the self-heal counters reach the hub and are decoded nowhere.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: walked 2026-10-29 on scratch 9202 (a walk document is now cited)</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Box survives an unattended app or guest-network failure · register R-264 · evidence audits/night-2026-10-28/part5-claims/selfheal/WALK.md · evidence audits/DIAG-bootrecon-drive-absent-2026-08-02.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the operator by email, one mail per run, every failing app named<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED against source: backup_run_failures digest is allowlisted (api/handler.go:1837), operator-only (dispatcher.go:423), templated (templates.go:48); recovery_unit_capture_failed is record-only (dispatcher.go:376). NOTE: R-182's register row still describes the PRE-FIX behaviour — see R-289.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the digest is wired and source-verified, but no run of it has been observed delivering</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A failed per-app Tier-1 backup reaches the OPERATOR — EVERY… · register R-182</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the customer<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The customer leg is built; today's log shows customer-channel rows skipped as operator_only.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App crashes → customer notified (one event per transition, …</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An already-paired box no longer tells anyone to pair it — its console says there is nothing to do<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was partial</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Was a warning card (the console kept showing the pairing code). Fixed by ISO 1.28.0 (R-535; R-214, R-235 closed). Walked 2026-10-29 00:04 on Tester 1 (paired, claimed): the console reads, in Hungarian and English, that there is nothing to do on this machine and no need to log in — no pairing code.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the defect is closLine truncated
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">7</span> &nbsp;Getting everything back</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A rebuilt machine shows a full-page recovery screen without anyone looking for it, and says plainly that nobody can replace a lost recovery code<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Seen unsought on the rebuilt demo-hp 2026-08-09, seal date matching host_escrow.created_at.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The customer's code opens the sealed package and the data returns byte for byte — including accented Hungarian filenames, verified as raw bytes<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Still true, and re-proven 2026-08-22 (5/5 byte-identical, both accented names as raw bytes) — but the SCOPE is narrower than the sentence sounds and was silently narrower still until v0.218.0. The drill found the off-site restore had NO named-volume leg at all: the archive sat in the unit, the snapshot and the checking folder and was never replayed, under a success message (R-354, fixed and proven 2026-08-22). AND 40 of the 53 catalogue apps STILL cannot run this route at all — it refuses first, saying a running app is not installed (R-356, open). So: proven for an app that declares a data drive; unproven and currently unreachable for the class whose entire dataset is a named volume.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · register R-354 · register R-356 · evidence tests/walk5-r201-2026-08-07/journal.md · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Walked end to end with no command line inside the machine (2026-08-07)<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#a78bfa;font-size:10px;white-space:nowrap">contested</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONTESTED-RESOLVED: not a contradiction. The walk proves the ROUTE needs no guest shell; the map's MISSING row is about a NON-OPERATOR doing it, which has still never happened. Two questions, one word 'customer'.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · evidence tests/walk5-r201-2026-08-07/journal.md · capability-map A customer (not the operator) performs a restore via UI alone</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Putting restored files back where they belong is still manual<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. Confirmed 2026-08-09: the restore lands in a verification folder and says so.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-213</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The tripwire that says someone is opening this customer's backups does fire<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED tonight from the hub store: escrow_blob_served 2026-08-09 10:19:41Z = 12:19 CEST in the operator's mailbox. R-281's original claim of silence is WITHDRAWN.</span><span style="display:block;color:#4b5bLine truncated
 </div>
 <div style="font-size:15px;font-weight:600;margin:22px 0 9px">When things go wrong</div>
@@ -40,7 +40,7 @@
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">A fresh install picks up an old image</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A fresh install picks up an old image<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Narrowed 2026-08-09: the resume path fetched the vouched golden; a fresh install still takes the newest LOCAL archive with no manifest comparison.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-274</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The off-site provider account is deleted</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The off-site provider account is deleted<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The credential that holds the customer's documents can still delete.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-95</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The machine is switched off for an afternoon</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The machine is switched off for an afternoon — no notion of expected downtime<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Confirmed 2026-08-09: eight operator mails for deliberate, attended work.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-285</span></span></div></div>
-<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">A customer restores their own data with no help</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A customer restores their own data with no help<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#a78bfa;font-size:10px;white-space:nowrap">contested</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONTESTED-RESOLVED: the map's MISSING is about a NON-OPERATOR performing it; the walks prove the route, not the person. Neither record was wrong. SINCE 2026-08-21 there is a second, harder blocker and it is not about the person at all: for the 40 of 53 apps that declare no data drive the off-site restore REFUSES before it starts, telling the customer a running app 'nincs telepítve' and to reinstall it to the same place — which those apps give them no way to choose (R-356). Those are exactly the apps whose whole dataset is a named volume. Until that is fixed, most customers cannot self-restore off-site even in principle.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A customer (not the operator) performs a restore via UI alone · register R-201 · register R-356 · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div></div>
+<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">A customer restores their own data with no help</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A customer restores their own data with no help<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Walked 2026-10-29 night on scratch 9202 through the household's restore button (the page's own POST): vikunja from the 02:30 local point in 79 s — the project made before the point came back, the one made after it is gone, the app runs, no hold; a restore that meets a busy volume puts the live data back (R-1256 H5). The route needs no help. Still open: no person who is not the operator has done it (claim.never-by-non-operator). The old off-site blocker (R-356) shipped in controller v0.219.0.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A customer (not the operator) performs a restore via UI alone · register R-201 · register R-356 · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md · evidence audits/night-2026-10-28/part5-claims/selfrestore-walk.log</span></span></div></div>
 </div>
 <div style="margin-top:26px;padding-top:12px;border-top:1px solid #1f2c44;color:#4b5b76;font-size:10.5px;line-height:1.6">Generated by <code>scripts/render_stands.py</code> from <code>where-felhom-stands.yaml</code>. Do not hand-edit this file — edit the data and regenerate. The 2026-08-09 React bundle is kept as <code>where-felhom-stands-2026-08-09-snapshot.html</code> and is not maintained.</div>
 </div></body></html>

documentation/architecture/where-felhom-stands.yaml

@@ -155,17 +155,20 @@ claims:
     band: journey
     stage: 3
     title: "Start, stop, restart, update, logs, remove — and the parts that must not be stopped cannot be"
-    status: built
+    status: walked
+    note: "Walked 2026-10-28 night on scratch 9202: install, stop, start, restart, logs; traefik and cloudflared refuse a stop (403); remove with data leaves no volume and no container. The update half: the same day's guarded updates on 9202 (bench + box + undo walks)."
     sources:
       - capability-map: "App lifecycle: start/stop/restart/update/logs/remove/redeploy"
       - register: "R-108"
+      - evidence: "audits/night-2026-10-28/part5-claims/lifecycle-walk.log"
+      - evidence: "audits/dockerhub-bigsteps-2026-10-28"
     changed:
-      from: walked
+      from: built
-      reason: "no walk document cited by the map row or anywhere else"
+      reason: "a walk document is now cited (2026-10-28 night)"
     verified:
-      date: 2026-08-09
+      date: 2026-10-28
-      verdict: downgraded
+      verdict: upgraded
-      depth: register+map
+      depth: source-read
   - id: use.tunnel
     band: journey
     stage: 3
@@ -207,13 +210,18 @@ claims:
     band: journey
     stage: 3
     title: "A one-tap launcher, and a read-only guest link for visitors"
-    status: built
+    status: walked
+    note: "Walked 2026-10-28 night on scratch 9202 through the pages the browser receives: the launcher lists each openable app; the guest link opens with no login and no admin link, a wrong token is 404, rotate and switch-off kill the old link."
     sources:
       - capability-map: "Indítópult (app launcher) — one-tap grid of the household's openable apps"
+      - evidence: "audits/night-2026-10-28/part5-claims/launcher-walk.log"
+    changed:
+      from: built
+      reason: "walked live 2026-10-28 night (map E8, E9 → PROVEN-LIVE)"
     verified:
-      date: 2026-08-09
+      date: 2026-10-28
-      verdict: confirmed
+      verdict: upgraded
-      depth: register+map
+      depth: source-read
   - id: use.dlna
     band: journey
     stage: 3
@@ -410,17 +418,19 @@ claims:
     band: journey
     stage: 6
     title: "The machine watches itself and repairs some faults without telling anyone it had to"
-    status: built
+    status: walked
-    note: "R-264 records that the self-heal counters reach the hub and are decoded nowhere."
+    note: "Walked 2026-10-29 night on scratch 9202: a crashed app part (kill -9 of paperless-redis) ran again in 1 s and the health probe read the app healthy; a guest reboot brought every app back in under 2 minutes, the boot sweep found nothing to start; no event, no banner, no mail. Docker's restart policy did the repair in both legs; the controller's own boot repair had nothing to do (its last proof: 6 of 6 hard resets, 2026-08-02). R-264 records that the self-heal counters reach the hub and are decoded nowhere."
     sources:
       - capability-map: "Box survives an unattended app or guest-network failure"
       - register: "R-264"
+      - evidence: "audits/night-2026-10-28/part5-claims/selfheal/WALK.md"
+      - evidence: "audits/DIAG-bootrecon-drive-absent-2026-08-02.md"
     changed:
-      from: walked
+      from: built
-      reason: "no walk document cited"
+      reason: "walked 2026-10-29 on scratch 9202 (a walk document is now cited)"
     verified:
-      date: 2026-08-09
+      date: 2026-10-29
-      verdict: downgraded
+      verdict: upgraded
       depth: source-read
   - id: fault.operator-email
     band: journey
@@ -453,16 +463,20 @@ claims:
   - id: fault.already-paired
     band: journey
     stage: 6
-    title: "An already-paired box is still told to pair itself"
+    title: "An already-paired box no longer tells anyone to pair it — its console says there is nothing to do"
-    status: partial
+    status: walked
-    note: "Warning card."
+    note: "Was a warning card (the console kept showing the pairing code). Fixed by ISO 1.28.0 (R-535; R-214, R-235 closed). Walked 2026-10-29 00:04 on Tester 1 (paired, claimed): the console reads, in Hungarian and English, that there is nothing to do on this machine and no need to log in — no pairing code."
     sources:
       - register: "R-214"
       - register: "R-235"
+      - evidence: "audits/night-2026-10-28/part5-claims/tester1-console-2026-10-29.png"
+    changed:
+      from: partial
+      reason: "the defect is closed and the console was read live on a paired box (2026-10-29)"
     verified:
-      date: 2026-08-09
+      date: 2026-10-29
-      verdict: confirmed
+      verdict: upgraded
-      depth: register+map
+      depth: source-read
   - id: recover.screen
     band: journey
     stage: 7
@@ -757,13 +771,14 @@ claims:
     band: failures
     title: "A customer restores their own data with no help"
     status: partial
-    note: "CONTESTED-RESOLVED: the map's MISSING is about a NON-OPERATOR performing it; the walks prove the route, not the person. Neither record was wrong. SINCE 2026-08-21 there is a second, harder blocker and it is not about the person at all: for the 40 of 53 apps that declare no data drive the off-site restore REFUSES before it starts, telling the customer a running app 'nincs telepítve' and to reinstall it to the same place — which those apps give them no way to choose (R-356). Those are exactly the apps whose whole dataset is a named volume. Until that is fixed, most customers cannot self-restore off-site even in principle."
+    note: "Walked 2026-10-29 night on scratch 9202 through the household's restore button (the page's own POST): vikunja from the 02:30 local point in 79 s — the project made before the point came back, the one made after it is gone, the app runs, no hold; a restore that meets a busy volume puts the live data back (R-1256 H5). The route needs no help. Still open: no person who is not the operator has done it (claim.never-by-non-operator). The old off-site blocker (R-356) shipped in controller v0.219.0."
     sources:
       - capability-map: "A customer (not the operator) performs a restore via UI alone"
       - register: "R-201"
       - register: "R-356"
       - evidence: "audits/REPORT-DRILL-backup-truth-2026-08-21.md"
+      - evidence: "audits/night-2026-10-28/part5-claims/selfrestore-walk.log"
     verified:
-      date: 2026-08-22
+      date: 2026-10-29
-      verdict: contested
+      verdict: confirmed
       depth: source-read

documentation/audits/cloudflared-2026.10.3-2026-10-26/NOTES.md

@@ -0,0 +1,27 @@
+# cloudflared 2026.10.3 — the eMudhra notice (R-1300), 2026-10-26 evening
+
+**The notice.** Cloudflare, 2026-10-20 (blog „8 eMudhra certificates are compromised"): eight eMudhra (emSign)
+intermediate CA keys were weak (recoverable from the public certificates; six TLS, two code-signing). cloudflared
+before 2026.10.2 trusts them. Docker users take the fixed image. Cloudflare: „do not resolve these errors by disabling
+verification". Release notes: 2026.10.2 = Go 1.27.3 + new distroless base (the trust store); 2026.10.3 = newer
+debian base + Go bump. 2026.10.4 (2026-10-26, today) adds a metrics-endpoint allowlist (VULN-176088) — not taken:
+under a day old, and the notice names 2026.10.3.
+
+**Image.** `cloudflare/cloudflared:2026.10.3` index digest
+`sha256:faa8e0a1475f137c910be6025f43c8e4092fdd35461b31e19fd1390657a1ff39` (Docker Hub API and a pull on 9202 agree;
+amd64 `sha256:c3f24df7…`); `--version` = `cloudflared version 2026.10.3 (built 2026-10-21-13:24 UTC)`.
+(`part3-image-check.txt`)
+
+**Part 1 — the list.** Three boxes report to the hub (demo-felhom, demo-hp, Tester 1); each guest 9201 runs
+`cloudflare/cloudflared:2026.9.3` (`part1-boxes.txt`, `part1-hub-hosts.txt`). 9202 (scratch) has no tunnel. Tester 2: no box.
+DooPlex: no cloudflared in k3s, Docker, systemd, dpkg or processes; the same queries find traefik/the hub (control)
+(`part1-dooplex.txt`). Cloudflare account: CC holds no Cloudflare token (`sec list`); the connector versions are the
+operator's look in Networking → Tunnels.
+
+**Part 2 — the https-origin trap.** Every tunnel's remote route (cloudflared's own „Updated to new configuration" log
+line, all three boxes): `*.<domain>` → `https://traefik`, `originRequest.noTLSVerify: true`, then `http_status:404`
+(`part2-remote-config.txt`). traefik answers SNI `traefik` with `CN=TRAEFIK DEFAULT CERT`, self-signed
+(`part2-origin-cert.txt`). No eMudhra chain → the new trust store refuses nothing on the origin leg: **the trap does
+not apply.** The `noTLSVerify` is the day-0 recipe (R-510) for a leg that never leaves the guest (cloudflared and
+traefik alone on the /29 `felhom-tunnel`); not changed tonight (the brief forbids touching certificate checks, and
+turning verification ON needs traefik to present a cert cloudflared can verify — a design question, not tonight).

documentation/audits/cloudflared-2026.10.3-2026-10-26/obs-n100-clock.txt

@@ -0,0 +1,57 @@
+System clock synchronized: no
+              NTP service: inactive
+failed
+inactive
+× chrony.service - chrony, an NTP client/server
+     Loaded: loaded (/usr/lib/systemd/system/chrony.service; enabled; preset: enabled)
+     Active: failed (Result: exit-code) since Wed 2026-10-21 05:23:50 CEST; 5 days ago
+ Invocation: 135c8cdc305140c8ac2bf943db811094
+       Docs: man:chronyd(8)
+             man:chronyc(1)
+Oct 16 05:09:25 demo-felhom systemd[1]: Failed to start chrony.service - chrony, an NTP client/server.
+-- Boot 56db71938b304a57bdedf30f89860e1f --
+Oct 21 05:23:50 demo-felhom systemd[1]: Starting chrony.service - chrony, an NTP client/server...
+Oct 21 05:23:50 demo-felhom chronyd[676]: chronyd version 4.8 starting (+CMDMON +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +NTS +SECHASH +IPV6 -DEBUG)
+Oct 21 05:23:50 demo-felhom chronyd[676]: Loaded 0 symmetric keys
+Oct 21 05:23:50 demo-felhom chronyd[676]: Using leap second list /usr/share/zoneinfo/leap-seconds.list
+Oct 21 05:23:50 demo-felhom chronyd[676]: Frequency -2.666 +/- 0.016 ppm read from /var/lib/chrony/chrony.drift
+Oct 21 05:23:50 demo-felhom chronyd[676]: Loaded seccomp filter (level 1)
+Oct 21 05:23:50 demo-felhom chronyd[676]: Fatal error : Could not send notification to $NOTIFY_SOCKET
+Oct 21 05:23:50 demo-felhom chronyd[676]: Fatal error : Could not send notification to $NOTIFY_SOCKET
+Oct 21 05:23:50 demo-felhom systemd[1]: chrony.service: Main process exited, code=exited, status=1/FAILURE
+Oct 21 05:23:50 demo-felhom systemd[1]: chrony.service: Failed with result 'exit-code'.
+Oct 21 05:23:50 demo-felhom systemd[1]: Failed to start chrony.service - chrony, an NTP client/server.
+2026-10-21 05:23:45
+[Unit]
+Description=chrony, an NTP client/server
+Documentation=man:chronyd(8) man:chronyc(1) man:chrony.conf(5)
+Conflicts=openntpd.service ntp.service ntpsec.service
+ConditionVirtualization=|!container
+ConditionVirtualization=|wsl
+ConditionCapability=CAP_SYS_TIME
+[Service]
+Type=notify
+PIDFile=/run/chrony/chronyd.pid
+EnvironmentFile=-/etc/default/chrony
+User=_chrony
+ExecStart=!/usr/sbin/chronyd -n $DAEMON_OPTS
+CapabilityBoundingSet=~CAP_AUDIT_CONTROL CAP_AUDIT_READ CAP_AUDIT_WRITE
+CapabilityBoundingSet=~CAP_BLOCK_SUSPEND CAP_KILL CAP_LEASE CAP_LINUX_IMMUTABLE
+CapabilityBoundingSet=~CAP_MAC_ADMIN CAP_MAC_OVERRIDE CAP_MKNOD CAP_SYS_ADMIN
+CapabilityBoundingSet=~CAP_SYS_BOOT CAP_SYS_CHROOT CAP_SYS_MODULE CAP_SYS_PACCT
+CapabilityBoundingSet=~CAP_SYS_PTRACE CAP_SYS_RAWIO CAP_SYS_TTY_CONFIG CAP_WAKE_ALARM
+DeviceAllow=char-pps rw
+DeviceAllow=char-ptp rw
+DeviceAllow=char-rtc rw
+DevicePolicy=closed
+LockPersonality=yes
+MemoryDenyWriteExecute=yes
+NoNewPrivileges=yes
+PrivateTmp=yes
+ProcSubset=pid
+ProtectControlGroups=yes
+ProtectHome=yes
+ProtectHostname=yes
+
+DAEMON_OPTS="-F 1"
+ii  chrony         4.8-4~bpo13+2 amd64        Versatile implementation of the Network Time Protocol

documentation/audits/cloudflared-2026.10.3-2026-10-26/part1-boxes.txt

@@ -0,0 +1,26 @@
+== felhom-pve
+demo-felhom
+VMID       Status     Lock         Name                
+9201       running                 demo-felhom         
+-- ct 9201
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.345.0 Up 6 hours (healthy)
+cloudflared cloudflare/cloudflared:2026.9.3 Up 5 days (healthy)
+== demo-hp
+demo-hp
+VMID       Status     Lock         Name                
+9201       running                 demo-hp             
+9202       running                 demo-hp-scratch     
+9401       stopped                 upgrade-harness     
+-- ct 9201
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.345.0 Up 6 hours (healthy)
+cloudflared cloudflare/cloudflared:2026.9.3 Up 3 days (healthy)
+-- ct 9202
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.343.0 Up 2 hours (healthy)
+       341 night1004-tester1    running    8192             200.00 1364274   
+== root@192.168.0.154
+felhom
+VMID       Status     Lock         Name                
+9201       running                 tester-1            
+-- ct 9201
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.345.0 Up 6 hours (healthy)
+cloudflared cloudflare/cloudflared:2026.9.3 Up 2 days (healthy)

documentation/audits/cloudflared-2026.10.3-2026-10-26/part1-dooplex.txt

@@ -0,0 +1,15 @@
+== k3s pods
+== k3s images any
+== docker
+== systemd
+== dpkg
+== processes
+2667091 /bin/bash -c source /home/kisfenyo/.claude/shell-snapshots/snapshot-bash-1793042538348-zvqq4e.sh 2>/dev/null || true && shopt -u extglob 2>/dev/null || true && { \builtin unalias -- 'unsetenv'
+2667092 /bin/bash -c source /home/kisfenyo/.claude/shell-snapshots/snapshot-bash-1793042538348-zvqq4e.sh 2>/dev/null || true && shopt -u extglob 2>/dev/null || true && { \builtin unalias -- 'unsetenv'
+2667753 /bin/bash -c source /home/kisfenyo/.claude/shell-snapshots/snapshot-bash-1793042538348-zvqq4e.sh 2>/dev/null || true && shopt -u extglob 2>/dev/null || true && { \builtin unalias -- 'unsetenv'
+== positive control: same queries find traefik/hub
+1
+2
+1
+== docker images cached
+== homelab-manifests mentions

documentation/audits/cloudflared-2026.10.3-2026-10-26/part1-hub-hosts.txt

@@ -0,0 +1,4 @@
+Host | Customer | Agent | Proxmox / kernel | Status | Guests | CPU | Mem | Disk | Cloudflared | Worst Storage
+demo-felhom-8363b5 | Demo Ügyfél | 0.166.0 floor: declared MinAgent | 9.2.21 7.0.14-25-pve | ONLINE | 1/2 | 0% | 20% | 35% | running (connected) | 35% local
+demo-hp-bb76ea | Demo HP | 0.166.0 floor: declared MinAgent | 9.2.21 7.0.14-25-pve | ONLINE | 1/2 | 11% | 45% | 61% | running (connected) | 61% local
+tester-1-d70be4 | Tester 1 | 0.166.0 floor: declared MinAgent | 9.2.2 7.0.14-22-pve | ONLINE | 1/2 | 7% | 33% | 27% | running (connected) | 27% local

documentation/audits/cloudflared-2026.10.3-2026-10-26/part2-origin-cert.txt

@@ -0,0 +1,21 @@
+== felhom-pve
+traefik=172.20.0.2
+/bin/openssl
+ 0 s:CN=TRAEFIK DEFAULT CERT
+   i:CN=TRAEFIK DEFAULT CERT
+Verify return code: 18 (self-signed certificate)
+    Verify return code: 18 (self-signed certificate)
+== demo-hp
+traefik=172.26.0.2
+/bin/openssl
+ 0 s:CN=TRAEFIK DEFAULT CERT
+   i:CN=TRAEFIK DEFAULT CERT
+Verify return code: 18 (self-signed certificate)
+    Verify return code: 18 (self-signed certificate)
+== root@192.168.0.154
+traefik=172.22.0.2
+/bin/openssl
+ 0 s:CN=TRAEFIK DEFAULT CERT
+   i:CN=TRAEFIK DEFAULT CERT
+Verify return code: 18 (self-signed certificate)
+    Verify return code: 18 (self-signed certificate)

documentation/audits/cloudflared-2026.10.3-2026-10-26/part2-remote-config.txt

@@ -0,0 +1,18 @@
+== felhom-pve
+2026-10-21T03:24:06Z INF Starting tunnel tunnelID=8b4edf48-1846-4484-870e-68b48cf43483
+2026-10-21T03:24:06Z INF Version 2026.9.3 (Checksum af73b73c18b1fe8c50293af01630a59789ddd9eb1edb56472fe5559b3a4a340c)
+2026-10-21T03:24:06Z INF GOOS: linux, GOVersion: go1.26.8, GoArch: amd64
+2026-10-21T03:24:07Z INF Updated to new configuration config="{\"ingress\":[{\"hostname\":\"*.demo-felhom.eu\", \"originRequest\":{\"noTLSVerify\":true}, \"service\":\"https://traefik\"}, {\"service\":\"http_status:404\"}], \"warp-routing\":{\"enabled\":false}}" version=1
+cloudflared version 2026.9.3 (built 2026-09-24-16:19 UTC)
+== demo-hp
+2026-10-23T15:34:00Z INF Starting tunnel tunnelID=d7a3c848-d754-46ce-a1c1-ca6bc7a81775
+2026-10-23T15:34:00Z INF Version 2026.9.3 (Checksum af73b73c18b1fe8c50293af01630a59789ddd9eb1edb56472fe5559b3a4a340c)
+2026-10-23T15:34:00Z INF GOOS: linux, GOVersion: go1.26.8, GoArch: amd64
+2026-10-23T15:34:01Z INF Updated to new configuration config="{\"ingress\":[{\"hostname\":\"*.enkisfelhom.hu\", \"originRequest\":{\"noTLSVerify\":true}, \"service\":\"https://traefik\"}, {\"service\":\"http_status:404\"}], \"warp-routing\":{\"enabled\":false}}" version=1
+cloudflared version 2026.9.3 (built 2026-09-24-16:19 UTC)
+== root@192.168.0.154
+2026-10-24T17:47:21Z INF Starting tunnel tunnelID=52d0c214-b833-477c-86d8-f787d9b550dd
+2026-10-24T17:47:21Z INF Version 2026.9.3 (Checksum af73b73c18b1fe8c50293af01630a59789ddd9eb1edb56472fe5559b3a4a340c)
+2026-10-24T17:47:21Z INF GOOS: linux, GOVersion: go1.26.8, GoArch: amd64
+2026-10-24T17:47:22Z INF Updated to new configuration config="{\"ingress\":[{\"hostname\":\"*.enkicsifelhom.hu\", \"originRequest\":{\"noTLSVerify\":true}, \"service\":\"https://traefik\"}, {\"service\":\"http_status:404\"}], \"warp-routing\":{\"enabled\":false}}" version=2
+cloudflared version 2026.9.3 (built 2026-09-24-16:19 UTC)

documentation/audits/cloudflared-2026.10.3-2026-10-26/part3-image-check.txt

@@ -0,0 +1,4 @@
+docker.io/cloudflare/cloudflared:2026.10.3
+cloudflare/cloudflared@sha256:faa8e0a1475f137c910be6025f43c8e4092fdd35461b31e19fd1390657a1ff39 2026-10-21T13:26:38.565206178Z
+cloudflared version 2026.10.3 (built 2026-10-21-13:24 UTC)
+removed

documentation/audits/cloudflared-2026.10.3-2026-10-26/part4-build-0.345.1.log

@@ -0,0 +1,181 @@
+
+[INFO]  ╔══════════════════════════════════════╗
+[INFO]  ║  felhom-controller image builder     ║
+[INFO]  ╚══════════════════════════════════════╝
+[INFO]  Version:    0.345.1
+[INFO]  Commit:     f3204024
+[INFO]  Repo:       /mnt/5_hdd/felhom.eu/git/felhom-controller-sec
+[INFO]  Source:     /mnt/5_hdd/felhom.eu/git/felhom-controller-sec/controller
+[INFO]  Build dir:  /mnt/5_hdd/felhom.eu/git/felhom-controller-mainsec/controller/workspace
+[INFO]  Image:      gitea.dooplex.hu/admin/felhom-controller:0.345.1
+
+[STEP]  0/4 — Checking the registry: is 0.345.1 already published?
+[INFO]  0.345.1 is not present in the registry — proceeding.
+[INFO]  Source tree is clean and at the pushed tag v0.345.1 (not origin/main).
+[STEP]  1/4 — Preparing build workspace...
+[INFO]  Source copied: 1079 Go files
+[STEP]  2/4 — Syncing app assets...
+[INFO]  Synced 263 assets (52 SVG + 16 PNG logos, 195 screenshots)
+[STEP]  3/4 — Preparing Go modules...
+[INFO]  Running go mod tidy...
+[STEP]  4/4 — Building Docker image...
+[INFO]  Building for current platform + pushing...
+#0 building with "default" instance using docker driver
+
+#1 [internal] load build definition from Dockerfile
+#1 transferring dockerfile: 6.94kB done
+#1 DONE 0.0s
+
+#2 [internal] load metadata for docker.io/library/golang:1.27.2-trixie@sha256:47667934874913a449b96faa7ef4777fa285d9ec29126ee80ed61c45b7cccf0e
+#2 ...
+
+#3 [auth] library/golang:pull token for registry-1.docker.io
+#3 DONE 0.0s
+
+#4 [auth] library/debian:pull token for registry-1.docker.io
+#4 DONE 0.0s
+
+#5 [internal] load metadata for docker.io/library/debian:13.7-slim@sha256:a29215f6a35e51e22adffa17f89e9d2ef06214e64a2bad10d765c46aea49f11f
+#5 ...
+
+#2 [internal] load metadata for docker.io/library/golang:1.27.2-trixie@sha256:47667934874913a449b96faa7ef4777fa285d9ec29126ee80ed61c45b7cccf0e
+#2 DONE 1.0s
+
+#5 [internal] load metadata for docker.io/library/debian:13.7-slim@sha256:a29215f6a35e51e22adffa17f89e9d2ef06214e64a2bad10d765c46aea49f11f
+#5 DONE 1.1s
+
+#6 [internal] load .dockerignore
+#6 transferring context: 2B done
+#6 DONE 0.0s
+
+#7 [restic 1/3] FROM docker.io/library/debian:13.7-slim@sha256:a29215f6a35e51e22adffa17f89e9d2ef06214e64a2bad10d765c46aea49f11f
+#7 resolve docker.io/library/debian:13.7-slim@sha256:a29215f6a35e51e22adffa17f89e9d2ef06214e64a2bad10d765c46aea49f11f 0.0s done
+#7 DONE 0.0s
+
+#8 [builder 1/7] FROM docker.io/library/golang:1.27.2-trixie@sha256:47667934874913a449b96faa7ef4777fa285d9ec29126ee80ed61c45b7cccf0e
+#8 resolve docker.io/library/golang:1.27.2-trixie@sha256:47667934874913a449b96faa7ef4777fa285d9ec29126ee80ed61c45b7cccf0e 0.0s done
+#8 DONE 0.0s
+
+#9 [internal] load build context
+#9 transferring context: 36.99MB 0.4s done
+#9 DONE 0.4s
+
+#10 [builder 3/7] COPY go.mod ./
+#10 CACHED
+
+#11 [builder 2/7] WORKDIR /build
+#11 CACHED
+
+#12 [builder 4/7] RUN go mod download || true
+#12 CACHED
+
+#13 [builder 5/7] COPY . .
+#13 DONE 0.6s
+
+#14 [builder 6/7] RUN go mod tidy
+#14 0.532 go: downloading gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
+#14 0.619 go: downloading github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e
+#14 0.619 go: downloading modernc.org/fileutil v1.3.40
+#14 0.698 go: downloading golang.org/x/term v0.47.0
+#14 1.291 go: downloading modernc.org/goabi0 v0.2.0
+#14 1.291 go: downloading modernc.org/ccgo/v4 v4.30.1
+#14 1.291 go: downloading modernc.org/cc/v4 v4.27.1
+#14 1.291 go: downloading golang.org/x/tools v0.38.0
+#14 1.422 go: downloading modernc.org/opt v0.1.4
+#14 1.422 go: downloading modernc.org/token v1.1.0
+#14 1.422 go: downloading modernc.org/sortutil v1.2.1
+#14 1.422 go: downloading modernc.org/strutil v1.2.1
+#14 1.598 go: downloading modernc.org/gc/v2 v2.6.5
+#14 1.598 go: downloading golang.org/x/mod v0.29.0
+#14 1.598 go: downloading modernc.org/gc/v3 v3.1.1
+#14 1.820 go: downloading github.com/hashicorp/golang-lru/v2 v2.0.7
+#14 2.038 go: downloading golang.org/x/sync v0.17.0
+#14 DONE 2.4s
+
+#15 [builder 7/7] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build     -ldflags="-s -w         -X main.Version=0.345.1         -X main.BuildTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)         -X main.GitCommit=f3204024"     -o /build/felhom-controller     ./cmd/controller/
+#15 DONE 22.3s
+
+#16 [stage-2 3/7] COPY --from=restic /usr/local/bin/restic /usr/local/bin/restic
+#16 CACHED
+
+#17 [stage-2 4/7] RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker.gpg     && . /etc/os-release     && echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker.gpg]        https://download.docker.com/linux/debian ${VERSION_CODENAME} stable" > /etc/apt/sources.list.d/docker.list     && apt-get update     && apt-get install -y --no-install-recommends docker-ce-cli docker-compose-plugin     && rm -rf /var/lib/apt/lists/*
+#17 CACHED
+
+#18 [stage-2 2/7] RUN apt-get update && apt-get upgrade -y --no-install-recommends && apt-get install -y --no-install-recommends     ca-certificates     curl     gnupg     git     util-linux     rsync     openssh-client     && rm -rf /var/lib/apt/lists/*
+#18 CACHED
+
+#19 [restic 2/3] COPY third_party/restic/restic-release-key.asc /tmp/restic-release-key.asc
+#19 CACHED
+
+#20 [restic 3/3] RUN set -eu;     apt-get update && apt-get install -y --no-install-recommends ca-certificates curl gnupg gpgv bzip2;     case "amd64" in       amd64) want="f415415624dcc452f2a02b8c33641791a8c6d6d3b65bbb3543fcf9a25151585c" ;;       arm64) want="a5f64aaab53d51e311fa3829124c5b703f2d14cf187d8640b6be3b2b49376465" ;;       *) echo "no pinned restic checksum for architecture 'amd64'" >&2; exit 1 ;;     esac;     f="restic_0.19.1_linux_amd64.bz2";     url="https://github.com/restic/restic/releases/download/v0.19.1";     cd /tmp;     curl -fsSL -o "$f" "$url/$f"; curl -fsSL -o SHA256SUMS "$url/SHA256SUMS"; curl -fsSL -o SHA256SUMS.asc "$url/SHA256SUMS.asc";     gpg --dearmor < /tmp/restic-release-key.asc > /tmp/restic-release-key.gpg;     gpgv --keyring /tmp/restic-release-key.gpg SHA256SUMS.asc SHA256SUMS;     grep " ${f}\$" SHA256SUMS | sha256sum -c -;     echo "${want}  ${f}" | sha256sum -c -;     bunzip2 -c "$f" > /usr/local/bin/restic; chmod 0755 /usr/local/bin/restic;     /usr/local/bin/restic version | grep -q "^restic 0.19.1 "
+#20 CACHED
+
+#21 [stage-2 5/7] RUN mkdir -p /opt/docker/felhom-controller/data
+#21 CACHED
+
+#22 [stage-2 6/7] COPY --from=builder /build/felhom-controller /usr/local/bin/felhom-controller
+#22 DONE 0.1s
+
+#23 [stage-2 7/7] COPY assets/ /usr/share/felhom/assets/
+#23 DONE 0.1s
+
+#24 exporting to image
+#24 exporting layers
+#24 exporting layers 2.6s done
+#24 exporting manifest sha256:c64bc088431b2398ec209fe5b2971bfd33ead8bb1ba219dedf601f6b556ebb63
+#24 exporting manifest sha256:c64bc088431b2398ec209fe5b2971bfd33ead8bb1ba219dedf601f6b556ebb63 0.3s done
+#24 exporting config sha256:94b98adfce1d5f7610e683f3f8a6855a7fb2a4bd5c668d19cf1415d6ba0327fa
+#24 exporting config sha256:94b98adfce1d5f7610e683f3f8a6855a7fb2a4bd5c668d19cf1415d6ba0327fa 0.2s done
+#24 exporting attestation manifest sha256:7368ba4d08db112ea8eb69a8e35f247298306710e6f9220005ce3220638bb891
+#24 exporting attestation manifest sha256:7368ba4d08db112ea8eb69a8e35f247298306710e6f9220005ce3220638bb891 0.3s done
+#24 exporting manifest list sha256:7cfdf7e1f835628de9c7d536e0a60c06bad5bbe19e8133765cc0a6bb732dbff6
+#24 exporting manifest list sha256:7cfdf7e1f835628de9c7d536e0a60c06bad5bbe19e8133765cc0a6bb732dbff6 0.2s done
+#24 naming to gitea.dooplex.hu/admin/felhom-controller:0.345.1
+#24 naming to gitea.dooplex.hu/admin/felhom-controller:0.345.1 0.0s done
+#24 unpacking to gitea.dooplex.hu/admin/felhom-controller:0.345.1
+#24 unpacking to gitea.dooplex.hu/admin/felhom-controller:0.345.1 0.6s done
+#24 naming to gitea.dooplex.hu/admin/felhom-controller:latest 0.0s done
+#24 unpacking to gitea.dooplex.hu/admin/felhom-controller:latest 0.1s done
+#24 DONE 4.5s
+[INFO]  Pushing...
+The push refers to repository [gitea.dooplex.hu/admin/felhom-controller]
+c3f83389ce1b: Waiting
+44136fa355b3: Waiting
+b2b4e3e31fd1: Waiting
+ecc510c1e359: Waiting
+c3f83389ce1b: Layer already exists
+44136fa355b3: Already exists
+b2b4e3e31fd1: Waiting
+ecc510c1e359: Layer already exists
+e81c1fce9826: Layer already exists
+63a1406b1961: Waiting
+d747c9b6922d: Waiting
+8c5bc1ea4981: Waiting
+63a1406b1961: Layer already exists
+d747c9b6922d: Layer already exists
+b2b4e3e31fd1: Pushed
+8c5bc1ea4981: Pushed
+bd4abb283803: Pushed
+0.345.1: digest: sha256:7cfdf7e1f835628de9c7d536e0a60c06bad5bbe19e8133765cc0a6bb732dbff6 size: 856
+The push refers to repository [gitea.dooplex.hu/admin/felhom-controller]
+b2b4e3e31fd1: Waiting
+44136fa355b3: Waiting
+c3f83389ce1b: Waiting
+ecc510c1e359: Waiting
+bd4abb283803: Layer already exists
+44136fa355b3: Already exists
+ecc510c1e359: Layer already exists
+b2b4e3e31fd1: Already exists
+d747c9b6922d: Layer already exists
+c3f83389ce1b: Layer already exists
+e81c1fce9826: Layer already exists
+63a1406b1961: Layer already exists
+8c5bc1ea4981: Layer already exists
+latest: digest: sha256:7cfdf7e1f835628de9c7d536e0a60c06bad5bbe19e8133765cc0a6bb732dbff6 size: 856
+
+[INFO]  ╔══════════════════════════════════════╗
+[INFO]  ║  Build complete ✓                    ║
+[INFO]  ╚══════════════════════════════════════╝
+[INFO]  Image: gitea.dooplex.hu/admin/felhom-controller:0.345.1
+[INFO]  Size:  486M
+

documentation/audits/cloudflared-2026.10.3-2026-10-26/part5-hp-n100-box.txt

@@ -0,0 +1,26 @@
+== demo-hp
+cloudflared cloudflare/cloudflared:2026.10.3 Up 11 seconds (healthy)
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.345.1 Up 27 seconds (healthy)
+started=2026-10-26T20:06:31.171324386Z
+2026-10-26T20:06:31Z INF Version 2026.10.3 (Checksum 919cd4640a5984b57ed420f1dd61e4155ee60abb52ac5fe2adb5f8dc71d0c2e5)
+2026-10-26T20:06:31Z INF GOOS: linux, GOVersion: go1.27.3, GoArch: amd64
+2026-10-26T20:06:31Z INF Registered tunnel connection connIndex=0 connection=843ee444-05e8-4aec-9b3e-339cd75a0243 event=0 ip=198.41.200.113 location=vie06 protocol=quic
+2026-10-26T20:06:31Z INF Updated to new configuration config="{\"ingress\":[{\"hostname\":\"*.enkisfelhom.hu\", \"originRequest\":{\"noTLSVerify\":true}, \"service\":\"https://traefik\"}, {\"service\":\"http_status:404\"}], \"warp-routing\":{\"enabled\":false}}" version=1
+2026-10-26T20:06:32Z INF Registered tunnel connection connIndex=1 connection=f4340b8a-6bb4-47b2-a781-d649bf2f8f58 event=0 ip=198.41.192.47 location=bud01 protocol=quic
+2026-10-26T20:06:33Z INF Registered tunnel connection connIndex=2 connection=fc7d8940-2277-4687-8942-c9a84142d9c9 event=0 ip=198.41.192.77 location=bud01 protocol=quic
+cloudflared version 2026.10.3 (built 2026-10-21-13:24 UTC)
+2026/10/26 20:06:16 updater.go:855: [INFO] [selfupdate] Post-update startup: update successful (0.345.0 → 0.345.1)
+2026/10/26 20:06:17 infra.go:188: [INFO] [infra] cloudflared compose changed (tunnel network true) — recreating cloudflared (the tunnel reconnects in seconds)
+== felhom-pve
+cloudflared cloudflare/cloudflared:2026.10.3 Up 25 seconds (healthy)
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.345.1 Up 30 seconds (healthy)
+started=2026-10-26T20:06:04.399090267Z
+2026-10-26T20:06:04Z INF Version 2026.10.3 (Checksum 919cd4640a5984b57ed420f1dd61e4155ee60abb52ac5fe2adb5f8dc71d0c2e5)
+2026-10-26T20:06:04Z INF GOOS: linux, GOVersion: go1.27.3, GoArch: amd64
+2026-10-26T20:06:05Z INF Registered tunnel connection connIndex=0 connection=592e0ea3-9acc-4ed6-a5e6-cb68d2855353 event=0 ip=198.41.192.167 location=bud01 protocol=quic
+2026-10-26T20:06:05Z INF Updated to new configuration config="{\"ingress\":[{\"hostname\":\"*.demo-felhom.eu\", \"originRequest\":{\"noTLSVerify\":true}, \"service\":\"https://traefik\"}, {\"service\":\"http_status:404\"}], \"warp-routing\":{\"enabled\":false}}" version=1
+2026-10-26T20:06:05Z INF Registered tunnel connection connIndex=1 connection=da046196-a6fa-492c-8f47-65e607f005f8 event=0 ip=198.41.200.73 location=vie06 protocol=quic
+2026-10-26T20:06:06Z INF Registered tunnel connection connIndex=2 connection=d183c0b2-19c3-4bd4-8779-e769170cf504 event=0 ip=198.41.200.23 location=vie05 protocol=quic
+cloudflared version 2026.10.3 (built 2026-10-21-13:24 UTC)
+2026/10/26 20:06:00 [INFO] [selfupdate] Post-update startup: update successful (0.345.0 → 0.345.1)
+2026/10/26 20:06:00 [INFO] [infra] cloudflared compose changed (tunnel network true) — recreating cloudflared (the tunnel reconnects in seconds)

documentation/audits/cloudflared-2026.10.3-2026-10-26/part5-hp-n100-floor.txt

@@ -0,0 +1,8 @@
+2026-10-26T20:06:04Z
+customer demo-hp
+HTTP/1.1 303 See Other
+Location: /customers/demo-hp?flash=floor_set
+2026-10-26T20:06:04Z
+customer demo-felhom
+HTTP/1.1 303 See Other
+Location: /customers/demo-felhom?flash=floor_set

documentation/audits/cloudflared-2026.10.3-2026-10-26/part5-https-only.txt

@@ -0,0 +1,34 @@
+control NEGATIVE (a stub answering 200 over plain http): convicted, as it must (200 -)
+== tester-1 (enkicsifelhom.hu): 6 public name(s)
+   OK   control POSITIVE http://felhom.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkicsifelhom.hu/r1033-check?q=1
+   OK   http://files.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://files.enkicsifelhom.hu/r1033-check?q=1
+   OK   http://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1
+   OK   http://paperless.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paperless.enkicsifelhom.hu/r1033-check?q=1
+   OK   http://paste.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paste.enkicsifelhom.hu/r1033-check?q=1
+   OK   http://wiki.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkicsifelhom.hu/r1033-check?q=1
+== demo-hp (enkisfelhom.hu): 15 public name(s)
+   OK   http://arcade.enkisfelhom.hu/r1033-check?q=1 -> 301 https://arcade.enkisfelhom.hu/r1033-check?q=1
+   OK   http://books.enkisfelhom.hu/r1033-check?q=1 -> 301 https://books.enkisfelhom.hu/r1033-check?q=1
+   OK   http://docs.enkisfelhom.hu/r1033-check?q=1 -> 301 https://docs.enkisfelhom.hu/r1033-check?q=1
+   OK   http://felhom.enkisfelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkisfelhom.hu/r1033-check?q=1
+   OK   http://files.enkisfelhom.hu/r1033-check?q=1 -> 301 https://files.enkisfelhom.hu/r1033-check?q=1
+   OK   http://gist.enkisfelhom.hu/r1033-check?q=1 -> 301 https://gist.enkisfelhom.hu/r1033-check?q=1
+   OK   http://ha.enkisfelhom.hu/r1033-check?q=1 -> 301 https://ha.enkisfelhom.hu/r1033-check?q=1
+   OK   http://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1
+   OK   http://papir.enkisfelhom.hu/r1033-check?q=1 -> 301 https://papir.enkisfelhom.hu/r1033-check?q=1
+   OK   http://paste.enkisfelhom.hu/r1033-check?q=1 -> 301 https://paste.enkisfelhom.hu/r1033-check?q=1
+   OK   http://pdf.enkisfelhom.hu/r1033-check?q=1 -> 301 https://pdf.enkisfelhom.hu/r1033-check?q=1
+   OK   http://plex.enkisfelhom.hu/r1033-check?q=1 -> 301 https://plex.enkisfelhom.hu/r1033-check?q=1
+   OK   http://time.enkisfelhom.hu/r1033-check?q=1 -> 301 https://time.enkisfelhom.hu/r1033-check?q=1
+   OK   http://travel.enkisfelhom.hu/r1033-check?q=1 -> 301 https://travel.enkisfelhom.hu/r1033-check?q=1
+   OK   http://wiki.enkisfelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkisfelhom.hu/r1033-check?q=1
+== demo-felhom (demo-felhom.eu): 8 public name(s)
+   OK   http://felhom.demo-felhom.eu/r1033-check?q=1 -> 301 https://felhom.demo-felhom.eu/r1033-check?q=1
+   OK   http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1
+   OK   http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1
+   OK   http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1
+   OK   http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1
+   OK   http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1
+   OK   http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1
+   OK   http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
+== every public name redirects http -> https

documentation/audits/cloudflared-2026.10.3-2026-10-26/part5-hub-hosts-after.txt

@@ -0,0 +1,5 @@
+Host | Status | Cloudflared
+Tester4-b5dbbd | ONLINE | unknown (no running customer guest)
+demo-felhom-8363b5 | ONLINE | running (connected)
+demo-hp-bb76ea | ONLINE | running (connected)
+tester-1-d70be4 | ONLINE | running (connected)

documentation/audits/cloudflared-2026.10.3-2026-10-26/part5-tester1-box.txt

@@ -0,0 +1,14 @@
+2026-10-26T20:05:25Z INF Version 2026.10.3 (Checksum 919cd4640a5984b57ed420f1dd61e4155ee60abb52ac5fe2adb5f8dc71d0c2e5)
+2026-10-26T20:05:25Z INF GOOS: linux, GOVersion: go1.27.3, GoArch: amd64
+2026-10-26T20:05:25Z INF Registered tunnel connection connIndex=0 connection=3cc5adb1-aea4-4d4f-8f8e-d63fefd414b2 event=0 ip=198.41.192.37 location=bud01 protocol=quic
+2026-10-26T20:05:25Z INF Updated to new configuration config="{\"ingress\":[{\"hostname\":\"*.enkicsifelhom.hu\", \"originRequest\":{\"noTLSVerify\":true}, \"service\":\"https://traefik\"}, {\"service\":\"http_status:404\"}], \"warp-routing\":{\"enabled\":false}}" version=2
+2026-10-26T20:05:25Z INF Registered tunnel connection connIndex=1 connection=27f1251e-375d-4774-a0b9-54d8bfd1641a event=0 ip=198.41.200.33 location=vie05 protocol=quic
+2026-10-26T20:05:27Z INF Registered tunnel connection connIndex=2 connection=be88ef22-3c33-4da0-ab61-cb14ec33e5ef event=0 ip=198.41.192.227 location=bud01 protocol=quic
+2026-10-26T20:05:28Z INF Registered tunnel connection connIndex=3 connection=4e5fa7a2-4c94-4c0e-93d0-395fb2b0957d event=0 ip=198.41.200.63 location=vie06 protocol=quic
+cloudflared version 2026.10.3 (built 2026-10-21-13:24 UTC)
+2026/10/26 20:05:19 [INFO] [selfupdate] Post-update startup: update successful (0.345.0 → 0.345.1)
+2026/10/26 20:05:19 [INFO] Self-update enabled (check every 6h, auto-update: false, auto-update time: 04:30)
+2026/10/26 20:05:19 [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
+2026/10/26 20:05:19 [INFO] [offsite-apply] settle-gate: awaiting floor knowledge (first report ACK) before offsite apply
+2026/10/26 20:05:20 [INFO] [infra] cloudflared compose changed (tunnel network true) — recreating cloudflared (the tunnel reconnects in seconds)
+2026/10/26 20:05:29 [INFO] [offsite-apply] settle-gate: GO — at/above floor 0.345.1 (we are 0.345.1), no managed update running

documentation/audits/cloudflared-2026.10.3-2026-10-26/part5-tester1-floor.txt

@@ -0,0 +1,3 @@
+2026-10-26T20:05:08Z
+HTTP/1.1 303 See Other
+Location: /customers/tester-1?flash=floor_set

documentation/audits/cloudflared-2026.10.3-2026-10-26/part6-clear-perbox.txt

@@ -0,0 +1,6 @@
+20:39:54 clear tester-1
+Location: /customers/tester-1?flash=floor_set
+20:39:54 clear demo-hp
+Location: /customers/demo-hp?flash=floor_set
+20:39:54 clear demo-felhom
+Location: /customers/demo-felhom?flash=floor_set

documentation/audits/cloudflared-2026.10.3-2026-10-26/part6-floor-readback.txt

@@ -0,0 +1,7 @@
+er action. Blank = no global floor. This setting is independent of the Day-0 artifact manifest below. Effective floor: v0.345.1 — source: DB (hub_settings) ; env fallback would be v0.120.0 Save global floor… DB override: v0.345.1 … Type the versio
+ (no manual copy). Choose — none — to clear an artifact. Agent — none — 0.166.0 0.165.0 0.164.0 0.163.0 0.162.0 0.161.0 0.158.1 Golden — none — 0.344.0 0.331.0 0.329.0 0.318.0 0.313.0 0.312.0 0.311.0 Min agent The golden's controller CHANGELOG Min
+2026/10/26 21:05:08 [INFO] Customer tester-1 controller-version floor override set to "0.345.1" (declared MinAgent "0.158.1")
+2026/10/26 21:05:08 [INFO] access token "cc-releases-2": POST /customers/tester-1/floor → 303
+2026/10/26 21:05:11 [INFO] managed floor SERVED for tester-1: floor 0.345.1, agent requirement "0.158.1" from declared (golden 0.344.0)
+2026/10/26 21:06:04 [INFO] Customer demo-hp controller-version floor override set to "0.345.1" (declared MinAgent "0.158.1")
+2026/10/26 21:06:04 [INFO] access token "cc-releases-2": POST /customers/demo-hp/floor → 303

documentation/audits/cloudflared-2026.10.3-2026-10-26/part6-global-floor.txt

@@ -0,0 +1,3 @@
+2026-10-26T20:08:39Z
+HTTP/1.1 303 See Other
+Location: /configuration?flash=floor_set

documentation/audits/cloudflared-2026.10.3-2026-10-26/part6-hub-final.txt

@@ -0,0 +1,11 @@
+Customer ID | Name | Domain | Status | Version | Floor | Config
+Tester-2 | Tester 2 | sajatfelhom.hu | DOWN | 0.292.0 | v0.345.1 ● | MANAGED
+Tester4 | Tester4 | tester4.test | OK | 0.345.1 | v0.345.1 | MANAGED
+demo-felhom | Demo Ügyfél | demo-felhom.eu | OK | 0.345.1 | v0.345.1 | MANAGED
+demo-hp | Demo HP | enkisfelhom.hu | OK | 0.345.1 | v0.345.1 | MANAGED
+tester-1 | Tester 1 | enkicsifelhom.hu | OK | 0.345.1 | v0.345.1 | MANAGED
+Host | Customer | Agent | Proxmox / kernel | Status | Guests | CPU | Mem | Disk | Cloudflared | Worst Storage
+Tester4-b5dbbd | Tester4 | 0.167.0-night1 floor held | 9.2.2 7.0.2-6-pve | ONLINE | 1/1 | 8% | 27% | 18% | absent (no cloudflared container in the guest (no tunnel set up on this box)) | 18% local
+demo-felhom-8363b5 | Demo Ügyfél | 0.166.0 floor: declared MinAgent | 9.2.21 7.0.14-25-pve | ONLINE | 1/2 | 5% | 20% | 35% | running (connected) | 35% local
+demo-hp-bb76ea | Demo HP | 0.166.0 floor: declared MinAgent | 9.2.21 7.0.14-25-pve | ONLINE | 1/2 | 19% | 66% | 65% | running (connected) | 65% local
+tester-1-d70be4 | Tester 1 | 0.166.0 floor: declared MinAgent | 9.2.2 7.0.14-22-pve | ONLINE | 1/2 | 0% | 34% | 27% | running (connected) | 27% local

documentation/audits/cloudflared-2026.10.3-2026-10-26/part6-watch.txt

@@ -0,0 +1,28 @@
+== 20:09:04
+felhom-pve: cloudflared=cloudflare/cloudflared:2026.10.3(Up 2 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 2 minutes (healthy)) 
+demo-hp: cloudflared=cloudflare/cloudflared:2026.10.3(Up 2 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 2 minutes (healthy)) 
+root@192.168.0.154: cloudflared=cloudflare/cloudflared:2026.10.3(Up 3 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 3 minutes (healthy)) 
+== 20:14:08
+felhom-pve: cloudflared=cloudflare/cloudflared:2026.10.3(Up 7 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 7 minutes (healthy)) 
+demo-hp: cloudflared=cloudflare/cloudflared:2026.10.3(Up 7 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 7 minutes (healthy)) 
+root@192.168.0.154: cloudflared=cloudflare/cloudflared:2026.10.3(Up 8 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 8 minutes (healthy)) 
+== 20:19:14
+felhom-pve: cloudflared=cloudflare/cloudflared:2026.10.3(Up 12 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 13 minutes (healthy)) 
+demo-hp: cloudflared=cloudflare/cloudflared:2026.10.3(Up 12 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 13 minutes (healthy)) 
+root@192.168.0.154: cloudflared=cloudflare/cloudflared:2026.10.3(Up 13 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 14 minutes (healthy)) 
+== 20:24:19
+felhom-pve: cloudflared=cloudflare/cloudflared:2026.10.3(Up 18 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 18 minutes (healthy)) 
+demo-hp: cloudflared=cloudflare/cloudflared:2026.10.3(Up 17 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 18 minutes (healthy)) 
+root@192.168.0.154: cloudflared=cloudflare/cloudflared:2026.10.3(Up 18 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 19 minutes (healthy)) 
+== 20:29:23
+felhom-pve: cloudflared=cloudflare/cloudflared:2026.10.3(Up 23 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 23 minutes (healthy)) 
+demo-hp: cloudflared=cloudflare/cloudflared:2026.10.3(Up 22 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 23 minutes (healthy)) 
+root@192.168.0.154: cloudflared=cloudflare/cloudflared:2026.10.3(Up 24 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 24 minutes (healthy)) 
+== 20:34:27
+felhom-pve: cloudflared=cloudflare/cloudflared:2026.10.3(Up 28 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 28 minutes (healthy)) 
+demo-hp: cloudflared=cloudflare/cloudflared:2026.10.3(Up 27 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 28 minutes (healthy)) 
+root@192.168.0.154: cloudflared=cloudflare/cloudflared:2026.10.3(Up 29 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 29 minutes (healthy)) 
+== 20:39:31
+felhom-pve: cloudflared=cloudflare/cloudflared:2026.10.3(Up 33 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 33 minutes (healthy)) 
+demo-hp: cloudflared=cloudflare/cloudflared:2026.10.3(Up 33 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 33 minutes (healthy)) 
+root@192.168.0.154: cloudflared=cloudflare/cloudflared:2026.10.3(Up 34 minutes (healthy)) felhom-controller=gitea.dooplex.hu/admin/felhom-controller:0.345.1(Up 34 minutes (healthy)) 

documentation/audits/d53-held-apps-2026-10-26/README.md

@@ -0,0 +1,62 @@
+# D53 — a held app reaches the operator: build and live proof (2026-10-26 evening)
+
+**Design:** `architecture/08-alarm-ladder.md` §10. **Ruling:** D53 = operator-only reminders first (`09` §3 decision 265).
+**Code (commit only, not released):** controller `5bd03ad` (CI 2377 ✓), hub `7e57e824` (CI 2378 ✓).
+
+## Red proofs
+
+- Controller `internal/holdreport`: with an inert `Tick`, all six behaviour tests failed; with the scheduler line
+  commented out, the wiring test failed.
+- Hub: six mutations, each turned its test red — `redproof-hub.txt` (one first attempt was only a compile error; it was
+  redone with a compiling mutation, the last line).
+
+## Live proof — method
+
+No hub deploy is allowed in this run, so the hub side ran as a **test hub process** on DooPlex (`192.168.0.180:18090`),
+built from a clean clone at `7e57e824`, scratch database, a throwaway login, its own random global key, operator mail to
+`d53-test@felhom.eu` (the catch-all), and `alerting.held_app_grace_override: 3m` (the test configuration). Scratch guest
+9202 ran a **local-only** controller image built from `5bd03ad` (`0.346.0-d53test`, `docker save`/`load`, never pushed to
+the registry), pointed at the test hub for the test. Pages were read through the test hub's own login (the endpoints
+the operator's browser uses); the mail was read in the mailbox (a different channel from the hub log).
+
+## Live proof — what happened (UTC)
+
+| Time | What | Evidence |
+|---|---|---|
+| 17:28 | Both lists empty: „No app is held on this box." / „… on any box." | `live/0-before_*.html` |
+| 17:29:52 | Test controller up; „Notifier enabled (hub: …:18090)", job `hold-report` every 1m | `live/9202-controller-log.txt` |
+| 17:30:52 | The first tick reported **four old holds 9202 has carried since 2026-09-24** (`chaoscrash`, `chaosoom`, `chaosoomb`, `gokapi`) | `live/test-hub.log` |
+| 17:33:35 | Four operator reminders (one per app, per-app cooldown) | mailbox, `live/notification_log.txt` |
+| 17:35:41 | Throwaway Radicale installed through the product | — |
+| 17:36:18–17:37:48 | Seven `kill -9` of its main process, 15 s apart | — |
+| 17:37:52 | The guard stopped it: „crash_loop — 7 in 10m0s; STOPPING it"; hold `unhealthy_stop` | `live/9202-controller-log.txt` |
+| 17:38:52 | `app_held` radicale reached the hub (next tick) | `live/test-hub.log` |
+| 17:41:34 | One reminder after the 3-minute grace; mail „[Felhom] ⚠️ demo-hp: the app radicale is still held — a person must act" in the inbox at 17:41:35 | mailbox |
+| 17:42 | Both pages list 5 held apps, Radicale with its reason, sentence, and who may clear it | `live/1-held_*.html` |
+| 17:45:37 | The household's Start (`POST /api/stacks/radicale/start`) lifted the hold | controller log |
+| 17:45:52 | `app_hold_cleared` → the Radicale row went from both pages; no reminder after it | `live/2-cleared_*.html`, `live/held_apps_final.txt` |
+
+**No household mail:** every reminder's customer leg is `skipped operator_only` (`live/notification_log.txt`).
+
+## Teardown
+
+- **Machine (9202):** Radicale removed through the product (no container, volume, `app.yaml`, data or subdomain left);
+  config and image file put back byte-identical to the saved copies; the test's two settings traces (`reported_holds`, the
+  Radicale stop time) removed while the controller was stopped; the test image removed; controller 0.343.0 healthy, hub off,
+  exactly as found. The four old holds and an old PrivateBin record (below) were there before and stay.
+- **Host (demo-hp):** nothing provisioned; the transfer file removed.
+- **Hub:** the real hub was not touched. The test hub process stopped; its database, login, key and binary deleted. The
+  local test image on DooPlex removed.
+
+## Observations
+
+- FILED: R-1298 — 9202 carries leftovers made outside the product: four crash-loop holds since 2026-09-24 for apps that
+  are not installed, and a PrivateBin `app.yaml` (2026-10-25) with no container, so the product reads it as installed.
+  Once D53 is released, such orphan holds would be reminded weekly about apps that do not exist.
+- NOT-A-FINDING: the test hub's log looked cut mid-line at 18:45:52 local — the `sec` wrapper holds back an unfinished
+  part of a long-running child's output; the full line was there when the process ended.
+- NOT-A-FINDING: my first switch of 9202 used `systemctl start` on the bootstrap oneshot, which was still active, so the
+  controller was briefly not running (about one minute); `restart` is the right verb. Fixed in the run.
+
+Secrets: the evidence was scanned for the Resend key, the hub token, the report key and 9202's password — 0 hits; the
+test hub's CSRF tokens are redacted.

documentation/audits/d53-held-apps-2026-10-26/live/0-before_hosts.html

@@ -0,0 +1,126 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Hosts — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.173.0-d53test">
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    
+<script>
+
+
+function felhomConfirm(el,question,onYes){
+    if(!el||el.dataset.fcOpen)return;
+    el.dataset.fcOpen='1';
+    var wrap=document.createElement('span');wrap.className='inline-confirm';
+    var q=document.createElement('span');q.className='inline-confirm-q';q.textContent=question;
+    var yes=document.createElement('button');yes.type='button';yes.className='btn btn-sm btn-danger';yes.textContent='Igen';
+    var no=document.createElement('button');no.type='button';no.className='btn btn-sm btn-outline';no.textContent='Mégse';
+    wrap.appendChild(q);wrap.appendChild(yes);wrap.appendChild(no);
+    el.style.display='none';el.parentNode.insertBefore(wrap,el.nextSibling);
+    function close(){wrap.remove();el.style.display='';delete el.dataset.fcOpen;}
+    no.addEventListener('click',close);
+    yes.addEventListener('click',function(){close();onYes();});
+}
+document.addEventListener('click',function(e){
+    var btn=e.target.closest?e.target.closest('[data-confirm]'):null;
+    if(!btn)return;
+    e.preventDefault();
+    felhomConfirm(btn,btn.getAttribute('data-confirm'),function(){
+        var form=btn.closest('form');
+        if(form){if(form.requestSubmit)form.requestSubmit(btn);else form.submit();}
+    });
+});
+</script>
+
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link active">Hosts</a>
+                <a href="/system" class="nav-link">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+        
+
+        <h2 style="margin-bottom: 1rem;">Hosts</h2>
+
+        
+
+        
+        <section class="card" id="held-apps" style="margin-bottom: 1.5rem; padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem; margin-top: 0;">Held apps</h2>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No app is held on any box.</p>
+            </div>
+            
+        </section>
+
+        
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <table class="data-table">
+                <thead>
+                    <tr>
+                        <th>Host</th>
+                        <th>Customer</th>
+                        <th>Agent</th>
+                        <th>Proxmox / kernel</th>
+                        <th>Status</th>
+                        <th>Guests</th>
+                        <th>CPU</th>
+                        <th>Mem</th>
+                        <th>Disk</th>
+                        <th>Cloudflared</th>
+                        <th>Worst Storage</th>
+                    </tr>
+                </thead>
+                <tbody>
+                    
+                    <tr onclick="window.location='/hosts/d53-test-9202'" style="cursor: pointer;">
+                        <td><a href="/hosts/d53-test-9202">d53-test-9202</a></td>
+                        <td>demo-hp</td>
+                        <td>—</td>
+                        <td style="font-size: 0.85em;">unknown<br><span class="text-muted">unknown</span></td>
+                        <td><span class="status-badge status-badge-pending">NO REPORT</span></td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+        </section>
+        
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.173.0-d53test</span>
+        </footer>
+    </div>
+</body>
+</html>

documentation/audits/d53-held-apps-2026-10-26/live/0-before_hosts_d53-test-9202.html

@@ -0,0 +1,347 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>d53-test-9202 — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.173.0-d53test">
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link active">Hosts</a>
+                <a href="/system" class="nav-link">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+        
+
+        <a href="/hosts" class="back-link">&larr; Hosts</a>
+
+        
+        
+        <section class="card">
+            <div style="display: flex; justify-content: space-between; align-items: center; flex-wrap: wrap; gap: 0.5rem;">
+                <h2 style="margin: 0;">d53-test-9202</h2>
+                <span class="status-badge status-badge-pending">NO REPORT</span>
+            </div>
+            <div class="info-grid" style="margin-top: 1rem;">
+                <div class="info-item">
+                    <span class="label">Host ID</span>
+                    <span class="value" style="font-family: var(--font-mono)">d53-test-9202</span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Customer</span>
+                    <span class="value"><a href="/customers/demo-hp">demo-hp</a></span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Agent Version</span>
+                    <span class="value">—</span>
+                </div>
+                
+                
+                <div class="info-item">
+                    <span class="label">Enrolled</span>
+                    <span class="value">just now</span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Last Report</span>
+                    <span class="value">waiting for first report</span>
+                </div>
+                
+                
+                <div class="info-item">
+                    <span class="label">Box connection</span>
+                    <span class="value" data-box-connection="unknown">unknown (the hub restarted recently, or the host has no customer)</span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Desired Generation</span>
+                    <span class="value">0</span>
+                </div>
+                
+            </div>
+        </section>
+
+        
+        <section class="card">
+            <div class="empty-state">
+                <p>Waiting for first report.</p>
+                <p class="hint">Vitals, guests and storage appear once this host's agent sends a host-report.</p>
+            </div>
+        </section>
+        
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Guests</h2>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No guests reported on this host.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" id="held-apps" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Held apps</h2>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No app is held on this box.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Storage Targets</h2>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No storage targets reported.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Capabilities</h2>
+            
+            
+            <div class="empty-state" style="border: none;">
+                <p>No capability report yet (waiting for the first host report).</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Diagnostics — Log Bundles</h2>
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                Pull-based: the box ships its debug ring on its own next cycle — controller &le; one report interval (~15 min),
+                agent &asymp; one heartbeat. The pull is recorded in the box's own log (customer-visible). Bundles expire after 72 h.
+            </p>
+            <div style="display: flex; gap: 0.5rem; margin: 0.75rem 0;">
+                <form method="POST" action="/hosts/d53-test-9202/request-logs" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="component" value="controller">
+                    <button type="submit" class="btn btn-sm">Request controller logs</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/request-logs" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="component" value="agent">
+                    <button type="submit" class="btn btn-sm">Request agent logs</button>
+                </form>
+            </div>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No log bundles. Use the request buttons above — the box delivers on its next cycle.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" id="operator-actions">
+            <h2>Operator Actions</h2>
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                The box acts on its next report reply (usually seconds, at most one report interval) and answers on the
+                report after that. Each press runs once. Recorded in the box's own log and in Events. Unanswered after a day: expired.
+            </p>
+            <div style="display: flex; flex-wrap: wrap; gap: 0.5rem; margin: 0.75rem 0;">
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="offsite_backup_now">
+                    <button type="submit" class="btn btn-sm">Run off-site backup now</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="run_job">
+                    <select name="arg"><option value="disk-health-check">disk-health-check</option><option value="fill-watch">fill-watch</option><option value="offsite-integrity">offsite-integrity</option><option value="offsite-proof">offsite-proof</option></select>
+                    <button type="submit" class="btn btn-sm">Run check now</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="abandon_stop">
+                    <button type="submit" class="btn btn-sm btn-outline">Stop deletion countdown</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="abandon_extend">
+                    <input type="number" name="arg" min="1" max="30" value="7" style="width: 4.5em; padding: 0.3em 0.5em;" aria-label="days">
+                    <button type="submit" class="btn btn-sm btn-outline">Extend countdown (days from now)</button>
+                </form>
+            </div>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No operator actions for this box yet.</p>
+            </div>
+            
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Network</h2>
+            <div class="info-grid">
+                <div class="info-item">
+                    <span class="label">WireGuard</span>
+                    <span class="value">
+                        <span class="text-muted">no peer allocated</span>
+                    </span>
+                </div>
+            </div>
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem; margin-top: 0.75rem;">
+                Waiting for the first host report.
+            </p>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Guest network
+                <span class="badge badge-neutral" title="A capable agent sent no guest-network stanza">unknown</span>
+                
+            </h2>
+
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                This host reported no guest-network state, so it is <strong>unknown</strong> &mdash; not healthy.
+                The watchdog is default-on; a silent capable agent means it was switched off
+                (<code>guest_net.disable</code>), or this report predates it on this box.
+            </p>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>DR / Backup</h2>
+            <div class="info-grid">
+                <div class="info-item">
+                    <span class="label">DR Recipe</span>
+                    <span class="value"><span class="text-muted">none</span></span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Key Escrow</span>
+                    <span class="value"><span class="text-muted">none</span></span>
+                </div>
+                <div class="info-item" id="offsite-integrity">
+                    <span class="label">Last full check (off-site)</span>
+                    <span class="value">not reported (no off-site status in the box&#39;s report)</span>
+                </div>
+            </div>
+        </section>
+
+        
+        <section class="card">
+            <h2>Console access</h2>
+            
+            <p><span class="badge badge-neutral">not vaulted</span></p>
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                No console credential is vaulted for this host. Expected for a byo host — the owner manages root@pam. Otherwise the installer's step 4b did not run; re-run felhom-host-install.sh, or set and vault one per the break-glass runbook &sect;5.
+            </p>
+            
+        </section>
+        
+
+        
+        
+        <section class="card" style="border-color: var(--crit);">
+            <h2>Danger zone</h2>
+            <p class="text-muted" style="font-size: 0.85rem;">
+                Removing this host deletes its reports, guests, log bundles and WireGuard peer, and
+                permanently invalidates its API key — a still-running agent would receive 401s.
+                Re-enrollment requires the Day-0 passphrase flow.
+            </p>
+            <button type="button" class="btn btn-danger btn-sm" onclick="hostDeleteConfirm('d53-test-9202')">Remove host&hellip;</button>
+            <div id="host-delete-confirm-d53-test-9202" style="display: none; margin-top: 0.75rem; padding: 0.75rem; border: 1px solid var(--crit); background: var(--crit-dim); border-radius: var(--radius); max-width: 44em;">
+                <p id="host-delete-impact-d53-test-9202" style="margin: 0 0 0.5rem; font-size: 0.9em;">&hellip;</p>
+                <label id="host-delete-escrow-row-d53-test-9202" style="display: none; margin: 0 0 0.5rem; font-size: 0.85em;">
+                    <input type="checkbox" id="host-delete-escrow-d53-test-9202">
+                    Move key escrow to retained custody (required when escrow present) + remove DR bundle for this host
+                </label>
+                
+                <p style="margin: 0 0 0.5rem; font-size: 0.85em; color: var(--text-2);">Type the host id to confirm:</p>
+                <form method="POST" action="/hosts/d53-test-9202/delete" id="host-delete-form-d53-test-9202" style="display: flex; gap: 0.5rem; align-items: center; flex-wrap: wrap;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="confirm_host_id" id="host-delete-confirm-hidden-d53-test-9202" value="">
+                    <input type="hidden" name="delete_escrow" id="host-delete-escrow-hidden-d53-test-9202" value="">
+                    
+                    <input type="text" id="host-delete-confirm-input-d53-test-9202" placeholder="retype the host id&hellip;" style="padding: 0.3em 0.5em; width: 16em;">
+                    <button type="button" class="btn btn-danger btn-sm" onclick="hostDeleteSubmit('d53-test-9202')">Confirm &amp; remove</button>
+                    <button type="button" class="btn btn-sm btn-outline" onclick="document.getElementById('host-delete-confirm-d53-test-9202').style.display='none';">Cancel</button>
+                </form>
+                <p id="host-delete-err-d53-test-9202" style="margin: 0.4em 0 0; font-size: 0.8em; color: var(--crit);"></p>
+            </div>
+        </section>
+        <script>
+        function hostDeleteConfirm(hostID) {
+            var box = document.getElementById('host-delete-confirm-' + hostID);
+            var impact = document.getElementById('host-delete-impact-' + hostID);
+            document.getElementById('host-delete-confirm-input-' + hostID).value = '';
+            document.getElementById('host-delete-err-' + hostID).textContent = '';
+            box.style.display = 'block';
+            impact.textContent = 'Checking impact…';
+            fetch('/hosts/' + encodeURIComponent(hostID) + '/delete-impact')
+                .then(function(r){ return r.json(); })
+                .then(function(d){
+                    var parts = ['Deleting ' + hostID + ' removes ' + d.guests + ' guest row(s), ' +
+                        d.reports + ' host report(s), ' + d.log_bundles + ' agent log bundle(s)'];
+                    if (d.wg_peer_bound) parts.push('the bound WireGuard peer');
+                    if (d.pbs_secret_present) parts.push('the staged PBS secret');
+                    if (d.recovery_present) parts.push('the break-glass recovery credential');
+                    impact.textContent = parts.join(', ') + '. Host status: ' + d.status + '.' +
+                        (d.deletable ? (d.off_tick_required ? ' No box connection for 6+ minutes — the "box is off" tick is required.' : '') : ' Host is ONLINE — deletion will be refused.');
+                    document.getElementById('host-delete-escrow-row-' + hostID).style.display =
+                        d.escrow_present ? 'block' : 'none';
+                })
+                .catch(function(){ impact.textContent = 'Could not compute the impact — the server will still enforce every gate.'; });
+        }
+        function hostDeleteSubmit(hostID) {
+            var typed = document.getElementById('host-delete-confirm-input-' + hostID).value.trim();
+            var err = document.getElementById('host-delete-err-' + hostID);
+            if (typed !== hostID) { err.textContent = 'Confirmation does not match the host id.'; return; }
+            var escrowRow = document.getElementById('host-delete-escrow-row-' + hostID);
+            var escrowCb = document.getElementById('host-delete-escrow-' + hostID);
+            if (escrowRow.style.display !== 'none' && !escrowCb.checked) {
+                err.textContent = 'This host has a key escrow — tick the acknowledgement to delete it too.';
+                return;
+            }
+            var offCb = document.getElementById('host-delete-off-' + hostID);
+            if (offCb && !offCb.checked) {
+                err.textContent = 'Tick "I checked: the box is off" first.';
+                return;
+            }
+            if (offCb) { document.getElementById('host-delete-off-hidden-' + hostID).value = '1'; }
+            document.getElementById('host-delete-confirm-hidden-' + hostID).value = typed;
+            document.getElementById('host-delete-escrow-hidden-' + hostID).value = escrowCb.checked ? '1' : '';
+            document.getElementById('host-delete-form-' + hostID).submit();
+        }
+        </script>
+        
+
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.173.0-d53test</span>
+        </footer>
+    </div>
+</body>
+</html>

documentation/audits/d53-held-apps-2026-10-26/live/1-held_hosts.html

@@ -0,0 +1,177 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Hosts — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.173.0-d53test">
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    
+<script>
+
+
+function felhomConfirm(el,question,onYes){
+    if(!el||el.dataset.fcOpen)return;
+    el.dataset.fcOpen='1';
+    var wrap=document.createElement('span');wrap.className='inline-confirm';
+    var q=document.createElement('span');q.className='inline-confirm-q';q.textContent=question;
+    var yes=document.createElement('button');yes.type='button';yes.className='btn btn-sm btn-danger';yes.textContent='Igen';
+    var no=document.createElement('button');no.type='button';no.className='btn btn-sm btn-outline';no.textContent='Mégse';
+    wrap.appendChild(q);wrap.appendChild(yes);wrap.appendChild(no);
+    el.style.display='none';el.parentNode.insertBefore(wrap,el.nextSibling);
+    function close(){wrap.remove();el.style.display='';delete el.dataset.fcOpen;}
+    no.addEventListener('click',close);
+    yes.addEventListener('click',function(){close();onYes();});
+}
+document.addEventListener('click',function(e){
+    var btn=e.target.closest?e.target.closest('[data-confirm]'):null;
+    if(!btn)return;
+    e.preventDefault();
+    felhomConfirm(btn,btn.getAttribute('data-confirm'),function(){
+        var form=btn.closest('form');
+        if(form){if(form.requestSubmit)form.requestSubmit(btn);else form.submit();}
+    });
+});
+</script>
+
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link active">Hosts</a>
+                <a href="/system" class="nav-link">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+        
+
+        <h2 style="margin-bottom: 1rem;">Hosts</h2>
+
+        
+
+        
+        <section class="card" id="held-apps" style="margin-bottom: 1.5rem; padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem; margin-top: 0;">Held apps</h2>
+            
+            <table class="data-table">
+                <thead>
+                    <tr>
+                        <th>Customer</th>
+                        <th>App</th>
+                        <th>Reason</th>
+                        <th>Since</th>
+                        <th>Who may clear it</th>
+                    </tr>
+                </thead>
+                <tbody>
+                    
+                    <tr data-held-app="demo-hp/gokapi">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>gokapi</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                    <tr data-held-app="demo-hp/chaosoom">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>chaosoom</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                    <tr data-held-app="demo-hp/chaoscrash">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>chaoscrash</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                    <tr data-held-app="demo-hp/chaosoomb">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>chaosoomb</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                    <tr data-held-app="demo-hp/radicale">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>radicale</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>7 min ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+            
+        </section>
+
+        
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <table class="data-table">
+                <thead>
+                    <tr>
+                        <th>Host</th>
+                        <th>Customer</th>
+                        <th>Agent</th>
+                        <th>Proxmox / kernel</th>
+                        <th>Status</th>
+                        <th>Guests</th>
+                        <th>CPU</th>
+                        <th>Mem</th>
+                        <th>Disk</th>
+                        <th>Cloudflared</th>
+                        <th>Worst Storage</th>
+                    </tr>
+                </thead>
+                <tbody>
+                    
+                    <tr onclick="window.location='/hosts/d53-test-9202'" style="cursor: pointer;">
+                        <td><a href="/hosts/d53-test-9202">d53-test-9202</a></td>
+                        <td>Demo HP (scratch guest 9202)</td>
+                        <td>—</td>
+                        <td style="font-size: 0.85em;">unknown<br><span class="text-muted">unknown</span></td>
+                        <td><span class="status-badge status-badge-pending">NO REPORT</span></td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+        </section>
+        
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.173.0-d53test</span>
+        </footer>
+    </div>
+</body>
+</html>

documentation/audits/d53-held-apps-2026-10-26/live/1-held_hosts_d53-test-9202.html

@@ -0,0 +1,399 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>d53-test-9202 — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.173.0-d53test">
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link active">Hosts</a>
+                <a href="/system" class="nav-link">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+        
+
+        <a href="/hosts" class="back-link">&larr; Hosts</a>
+
+        
+        
+        <section class="card">
+            <div style="display: flex; justify-content: space-between; align-items: center; flex-wrap: wrap; gap: 0.5rem;">
+                <h2 style="margin: 0;">d53-test-9202</h2>
+                <span class="status-badge status-badge-pending">NO REPORT</span>
+            </div>
+            <div class="info-grid" style="margin-top: 1rem;">
+                <div class="info-item">
+                    <span class="label">Host ID</span>
+                    <span class="value" style="font-family: var(--font-mono)">d53-test-9202</span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Customer</span>
+                    <span class="value"><a href="/customers/demo-hp">Demo HP (scratch guest 9202)</a></span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Agent Version</span>
+                    <span class="value">—</span>
+                </div>
+                
+                
+                <div class="info-item">
+                    <span class="label">Enrolled</span>
+                    <span class="value">16 min ago</span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Last Report</span>
+                    <span class="value">waiting for first report</span>
+                </div>
+                
+                
+                <div class="info-item">
+                    <span class="label">Box connection</span>
+                    <span class="value" data-box-connection="not_connected" style="color: var(--warn)">not connected since the hub started (2026-10-26 17:28 UTC, 17 min ago)</span>
+                    
+                </div>
+                <div class="info-item">
+                    <span class="label">Desired Generation</span>
+                    <span class="value">0</span>
+                </div>
+                
+            </div>
+        </section>
+
+        
+        <section class="card">
+            <div class="empty-state">
+                <p>Waiting for first report.</p>
+                <p class="hint">Vitals, guests and storage appear once this host's agent sends a host-report.</p>
+            </div>
+        </section>
+        
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Guests</h2>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No guests reported on this host.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" id="held-apps" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Held apps</h2>
+            
+            <table class="data-table">
+                <thead>
+                    <tr>
+                        <th>App</th>
+                        <th>Reason</th>
+                        <th>Since</th>
+                        <th>Why</th>
+                        <th>Who may clear it</th>
+                    </tr>
+                </thead>
+                <tbody>
+                    
+                    <tr data-held-app="gokapi">
+                        <td><code>gokapi</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>stopped by the box for a crash loop (2026-09-24T10:37:48Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                    <tr data-held-app="chaosoom">
+                        <td><code>chaosoom</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>stopped by the box for a oom storm (2026-09-24T11:48:01Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                    <tr data-held-app="chaoscrash">
+                        <td><code>chaoscrash</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>stopped by the box for a crash loop (2026-09-24T12:16:24Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                    <tr data-held-app="chaosoomb">
+                        <td><code>chaosoomb</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>stopped by the box for a oom storm (2026-09-24T12:39:51Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                    <tr data-held-app="radicale">
+                        <td><code>radicale</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>7 min ago</td>
+                        <td>stopped by the box for a crash loop (2026-10-26T17:37:52Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+            
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Storage Targets</h2>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No storage targets reported.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Capabilities</h2>
+            
+            
+            <div class="empty-state" style="border: none;">
+                <p>No capability report yet (waiting for the first host report).</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Diagnostics — Log Bundles</h2>
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                Pull-based: the box ships its debug ring on its own next cycle — controller &le; one report interval (~15 min),
+                agent &asymp; one heartbeat. The pull is recorded in the box's own log (customer-visible). Bundles expire after 72 h.
+            </p>
+            <div style="display: flex; gap: 0.5rem; margin: 0.75rem 0;">
+                <form method="POST" action="/hosts/d53-test-9202/request-logs" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="component" value="controller">
+                    <button type="submit" class="btn btn-sm">Request controller logs</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/request-logs" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="component" value="agent">
+                    <button type="submit" class="btn btn-sm">Request agent logs</button>
+                </form>
+            </div>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No log bundles. Use the request buttons above — the box delivers on its next cycle.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" id="operator-actions">
+            <h2>Operator Actions</h2>
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                The box acts on its next report reply (usually seconds, at most one report interval) and answers on the
+                report after that. Each press runs once. Recorded in the box's own log and in Events. Unanswered after a day: expired.
+            </p>
+            <div style="display: flex; flex-wrap: wrap; gap: 0.5rem; margin: 0.75rem 0;">
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="offsite_backup_now">
+                    <button type="submit" class="btn btn-sm">Run off-site backup now</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="run_job">
+                    <select name="arg"><option value="disk-health-check">disk-health-check</option><option value="fill-watch">fill-watch</option><option value="offsite-integrity">offsite-integrity</option><option value="offsite-proof">offsite-proof</option></select>
+                    <button type="submit" class="btn btn-sm">Run check now</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="abandon_stop">
+                    <button type="submit" class="btn btn-sm btn-outline">Stop deletion countdown</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="abandon_extend">
+                    <input type="number" name="arg" min="1" max="30" value="7" style="width: 4.5em; padding: 0.3em 0.5em;" aria-label="days">
+                    <button type="submit" class="btn btn-sm btn-outline">Extend countdown (days from now)</button>
+                </form>
+            </div>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No operator actions for this box yet.</p>
+            </div>
+            
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Network</h2>
+            <div class="info-grid">
+                <div class="info-item">
+                    <span class="label">WireGuard</span>
+                    <span class="value">
+                        <span class="text-muted">no peer allocated</span>
+                    </span>
+                </div>
+            </div>
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem; margin-top: 0.75rem;">
+                Waiting for the first host report.
+            </p>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Guest network
+                <span class="badge badge-neutral" title="A capable agent sent no guest-network stanza">unknown</span>
+                
+            </h2>
+
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                This host reported no guest-network state, so it is <strong>unknown</strong> &mdash; not healthy.
+                The watchdog is default-on; a silent capable agent means it was switched off
+                (<code>guest_net.disable</code>), or this report predates it on this box.
+            </p>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>DR / Backup</h2>
+            <div class="info-grid">
+                <div class="info-item">
+                    <span class="label">DR Recipe</span>
+                    <span class="value"><span class="text-muted">none</span></span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Key Escrow</span>
+                    <span class="value"><span class="text-muted">none</span></span>
+                </div>
+                <div class="info-item" id="offsite-integrity">
+                    <span class="label">Last full check (off-site)</span>
+                    <span class="value">not reported (no off-site status in the box&#39;s report)</span>
+                </div>
+            </div>
+        </section>
+
+        
+        <section class="card">
+            <h2>Console access</h2>
+            
+            <p><span class="badge badge-neutral">not vaulted</span></p>
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                No console credential is vaulted for this host. Expected for a byo host — the owner manages root@pam. Otherwise the installer's step 4b did not run; re-run felhom-host-install.sh, or set and vault one per the break-glass runbook &sect;5.
+            </p>
+            
+        </section>
+        
+
+        
+        
+        <section class="card" style="border-color: var(--crit);">
+            <h2>Danger zone</h2>
+            <p class="text-muted" style="font-size: 0.85rem;">
+                Removing this host deletes its reports, guests, log bundles and WireGuard peer, and
+                permanently invalidates its API key — a still-running agent would receive 401s.
+                Re-enrollment requires the Day-0 passphrase flow.
+            </p>
+            <button type="button" class="btn btn-danger btn-sm" onclick="hostDeleteConfirm('d53-test-9202')">Remove host&hellip;</button>
+            <div id="host-delete-confirm-d53-test-9202" style="display: none; margin-top: 0.75rem; padding: 0.75rem; border: 1px solid var(--crit); background: var(--crit-dim); border-radius: var(--radius); max-width: 44em;">
+                <p id="host-delete-impact-d53-test-9202" style="margin: 0 0 0.5rem; font-size: 0.9em;">&hellip;</p>
+                <label id="host-delete-escrow-row-d53-test-9202" style="display: none; margin: 0 0 0.5rem; font-size: 0.85em;">
+                    <input type="checkbox" id="host-delete-escrow-d53-test-9202">
+                    Move key escrow to retained custody (required when escrow present) + remove DR bundle for this host
+                </label>
+                
+                <p style="margin: 0 0 0.5rem; font-size: 0.85em; color: var(--text-2);">Type the host id to confirm:</p>
+                <form method="POST" action="/hosts/d53-test-9202/delete" id="host-delete-form-d53-test-9202" style="display: flex; gap: 0.5rem; align-items: center; flex-wrap: wrap;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="confirm_host_id" id="host-delete-confirm-hidden-d53-test-9202" value="">
+                    <input type="hidden" name="delete_escrow" id="host-delete-escrow-hidden-d53-test-9202" value="">
+                    
+                    <input type="text" id="host-delete-confirm-input-d53-test-9202" placeholder="retype the host id&hellip;" style="padding: 0.3em 0.5em; width: 16em;">
+                    <button type="button" class="btn btn-danger btn-sm" onclick="hostDeleteSubmit('d53-test-9202')">Confirm &amp; remove</button>
+                    <button type="button" class="btn btn-sm btn-outline" onclick="document.getElementById('host-delete-confirm-d53-test-9202').style.display='none';">Cancel</button>
+                </form>
+                <p id="host-delete-err-d53-test-9202" style="margin: 0.4em 0 0; font-size: 0.8em; color: var(--crit);"></p>
+            </div>
+        </section>
+        <script>
+        function hostDeleteConfirm(hostID) {
+            var box = document.getElementById('host-delete-confirm-' + hostID);
+            var impact = document.getElementById('host-delete-impact-' + hostID);
+            document.getElementById('host-delete-confirm-input-' + hostID).value = '';
+            document.getElementById('host-delete-err-' + hostID).textContent = '';
+            box.style.display = 'block';
+            impact.textContent = 'Checking impact…';
+            fetch('/hosts/' + encodeURIComponent(hostID) + '/delete-impact')
+                .then(function(r){ return r.json(); })
+                .then(function(d){
+                    var parts = ['Deleting ' + hostID + ' removes ' + d.guests + ' guest row(s), ' +
+                        d.reports + ' host report(s), ' + d.log_bundles + ' agent log bundle(s)'];
+                    if (d.wg_peer_bound) parts.push('the bound WireGuard peer');
+                    if (d.pbs_secret_present) parts.push('the staged PBS secret');
+                    if (d.recovery_present) parts.push('the break-glass recovery credential');
+                    impact.textContent = parts.join(', ') + '. Host status: ' + d.status + '.' +
+                        (d.deletable ? (d.off_tick_required ? ' No box connection for 6+ minutes — the "box is off" tick is required.' : '') : ' Host is ONLINE — deletion will be refused.');
+                    document.getElementById('host-delete-escrow-row-' + hostID).style.display =
+                        d.escrow_present ? 'block' : 'none';
+                })
+                .catch(function(){ impact.textContent = 'Could not compute the impact — the server will still enforce every gate.'; });
+        }
+        function hostDeleteSubmit(hostID) {
+            var typed = document.getElementById('host-delete-confirm-input-' + hostID).value.trim();
+            var err = document.getElementById('host-delete-err-' + hostID);
+            if (typed !== hostID) { err.textContent = 'Confirmation does not match the host id.'; return; }
+            var escrowRow = document.getElementById('host-delete-escrow-row-' + hostID);
+            var escrowCb = document.getElementById('host-delete-escrow-' + hostID);
+            if (escrowRow.style.display !== 'none' && !escrowCb.checked) {
+                err.textContent = 'This host has a key escrow — tick the acknowledgement to delete it too.';
+                return;
+            }
+            var offCb = document.getElementById('host-delete-off-' + hostID);
+            if (offCb && !offCb.checked) {
+                err.textContent = 'Tick "I checked: the box is off" first.';
+                return;
+            }
+            if (offCb) { document.getElementById('host-delete-off-hidden-' + hostID).value = '1'; }
+            document.getElementById('host-delete-confirm-hidden-' + hostID).value = typed;
+            document.getElementById('host-delete-escrow-hidden-' + hostID).value = escrowCb.checked ? '1' : '';
+            document.getElementById('host-delete-form-' + hostID).submit();
+        }
+        </script>
+        
+
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.173.0-d53test</span>
+        </footer>
+    </div>
+</body>
+</html>

documentation/audits/d53-held-apps-2026-10-26/live/2-cleared_hosts.html

@@ -0,0 +1,169 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Hosts — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.173.0-d53test">
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    
+<script>
+
+
+function felhomConfirm(el,question,onYes){
+    if(!el||el.dataset.fcOpen)return;
+    el.dataset.fcOpen='1';
+    var wrap=document.createElement('span');wrap.className='inline-confirm';
+    var q=document.createElement('span');q.className='inline-confirm-q';q.textContent=question;
+    var yes=document.createElement('button');yes.type='button';yes.className='btn btn-sm btn-danger';yes.textContent='Igen';
+    var no=document.createElement('button');no.type='button';no.className='btn btn-sm btn-outline';no.textContent='Mégse';
+    wrap.appendChild(q);wrap.appendChild(yes);wrap.appendChild(no);
+    el.style.display='none';el.parentNode.insertBefore(wrap,el.nextSibling);
+    function close(){wrap.remove();el.style.display='';delete el.dataset.fcOpen;}
+    no.addEventListener('click',close);
+    yes.addEventListener('click',function(){close();onYes();});
+}
+document.addEventListener('click',function(e){
+    var btn=e.target.closest?e.target.closest('[data-confirm]'):null;
+    if(!btn)return;
+    e.preventDefault();
+    felhomConfirm(btn,btn.getAttribute('data-confirm'),function(){
+        var form=btn.closest('form');
+        if(form){if(form.requestSubmit)form.requestSubmit(btn);else form.submit();}
+    });
+});
+</script>
+
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link active">Hosts</a>
+                <a href="/system" class="nav-link">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+        
+
+        <h2 style="margin-bottom: 1rem;">Hosts</h2>
+
+        
+
+        
+        <section class="card" id="held-apps" style="margin-bottom: 1.5rem; padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem; margin-top: 0;">Held apps</h2>
+            
+            <table class="data-table">
+                <thead>
+                    <tr>
+                        <th>Customer</th>
+                        <th>App</th>
+                        <th>Reason</th>
+                        <th>Since</th>
+                        <th>Who may clear it</th>
+                    </tr>
+                </thead>
+                <tbody>
+                    
+                    <tr data-held-app="demo-hp/gokapi">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>gokapi</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                    <tr data-held-app="demo-hp/chaosoom">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>chaosoom</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                    <tr data-held-app="demo-hp/chaoscrash">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>chaoscrash</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                    <tr data-held-app="demo-hp/chaosoomb">
+                        <td><a href="/hosts/d53-test-9202#held-apps">demo-hp</a></td>
+                        <td><code>chaosoomb</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>the household&#39;s Start (one more try)</td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+            
+        </section>
+
+        
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <table class="data-table">
+                <thead>
+                    <tr>
+                        <th>Host</th>
+                        <th>Customer</th>
+                        <th>Agent</th>
+                        <th>Proxmox / kernel</th>
+                        <th>Status</th>
+                        <th>Guests</th>
+                        <th>CPU</th>
+                        <th>Mem</th>
+                        <th>Disk</th>
+                        <th>Cloudflared</th>
+                        <th>Worst Storage</th>
+                    </tr>
+                </thead>
+                <tbody>
+                    
+                    <tr onclick="window.location='/hosts/d53-test-9202'" style="cursor: pointer;">
+                        <td><a href="/hosts/d53-test-9202">d53-test-9202</a></td>
+                        <td>Demo HP (scratch guest 9202)</td>
+                        <td>—</td>
+                        <td style="font-size: 0.85em;">unknown<br><span class="text-muted">unknown</span></td>
+                        <td><span class="status-badge status-badge-pending">NO REPORT</span></td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                        <td>—</td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+        </section>
+        
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.173.0-d53test</span>
+        </footer>
+    </div>
+</body>
+</html>

documentation/audits/d53-held-apps-2026-10-26/live/2-cleared_hosts_d53-test-9202.html

@@ -0,0 +1,391 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>d53-test-9202 — Felhom Hub</title>
+    <link rel="stylesheet" href="/style.css?v=0.173.0-d53test">
+</head>
+<body>
+    <svg xmlns="http://www.w3.org/2000/svg" style="display:none" aria-hidden="true">
+<symbol id="i-triangle-alert" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3" /> <path d="M12 9v4" /> <path d="M12 17h.01" /></symbol>
+<symbol id="i-check" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 6 9 17l-5-5" /></symbol>
+<symbol id="i-server" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2" ry="2" /> <rect width="20" height="8" x="2" y="14" rx="2" ry="2" /> <line x1="6" x2="6.01" y1="6" y2="6" /> <line x1="6" x2="6.01" y1="18" y2="18" /></symbol>
+<symbol id="i-settings" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9.671 4.136a2.34 2.34 0 0 1 4.659 0 2.34 2.34 0 0 0 3.319 1.915 2.34 2.34 0 0 1 2.33 4.033 2.34 2.34 0 0 0 0 3.831 2.34 2.34 0 0 1-2.33 4.033 2.34 2.34 0 0 0-3.319 1.915 2.34 2.34 0 0 1-4.659 0 2.34 2.34 0 0 0-3.32-1.915 2.34 2.34 0 0 1-2.33-4.033 2.34 2.34 0 0 0 0-3.831A2.34 2.34 0 0 1 6.35 6.051a2.34 2.34 0 0 0 3.319-1.915" /> <circle cx="12" cy="12" r="3" /></symbol>
+<symbol id="i-x" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 6 6 18" /> <path d="m6 6 12 12" /></symbol>
+<symbol id="i-info" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 16v-4" /> <path d="M12 8h.01" /></symbol>
+<symbol id="i-hard-drive" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 16h.01" /> <path d="M2.212 11.577a2 2 0 0 0-.212.896V18a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-5.527a2 2 0 0 0-.212-.896L18.55 5.11A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z" /> <path d="M21.946 12.013H2.054" /> <path d="M6 16h.01" /></symbol>
+<symbol id="i-cpu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20v2" /> <path d="M12 2v2" /> <path d="M17 20v2" /> <path d="M17 2v2" /> <path d="M2 12h2" /> <path d="M2 17h2" /> <path d="M2 7h2" /> <path d="M20 12h2" /> <path d="M20 17h2" /> <path d="M20 7h2" /> <path d="M7 20v2" /> <path d="M7 2v2" /> <rect x="4" y="4" width="16" height="16" rx="2" /> <rect x="8" y="8" width="8" height="8" rx="1" /></symbol>
+<symbol id="i-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10" /> <path d="M12 6v6l4 2" /></symbol>
+<symbol id="i-boxes" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.97 12.92A2 2 0 0 0 2 14.63v3.24a2 2 0 0 0 .97 1.71l3 1.8a2 2 0 0 0 2.06 0L12 19v-5.5l-5-3-4.03 2.42Z" /> <path d="m7 16.5-4.74-2.85" /> <path d="m7 16.5 5-3" /> <path d="M7 16.5v5.17" /> <path d="M12 13.5V19l3.97 2.38a2 2 0 0 0 2.06 0l3-1.8a2 2 0 0 0 .97-1.71v-3.24a2 2 0 0 0-.97-1.71L17 10.5l-5 3Z" /> <path d="m17 16.5-5-3" /> <path d="m17 16.5 4.74-2.85" /> <path d="M17 16.5v5.17" /> <path d="M7.97 4.42A2 2 0 0 0 7 6.13v4.37l5 3 5-3V6.13a2 2 0 0 0-.97-1.71l-3-1.8a2 2 0 0 0-2.06 0l-3 1.8Z" /> <path d="M12 8 7.26 5.15" /> <path d="m12 8 4.74-2.85" /> <path d="M12 13.5V8" /></symbol>
+<symbol id="i-users" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2" /> <path d="M16 3.128a4 4 0 0 1 0 7.744" /> <path d="M22 21v-2a4 4 0 0 0-3-3.87" /> <circle cx="9" cy="7" r="4" /></symbol>
+</svg>
+    <div class="container">
+        <header>
+            <h1>Felhom <span>Hub</span></h1>
+            <nav class="nav-links">
+                <a href="/" class="nav-link">Dashboard</a>
+                <a href="/configs" class="nav-link">Customers</a>
+                <a href="/apps" class="nav-link">Apps</a>
+                <a href="/hosts" class="nav-link active">Hosts</a>
+                <a href="/system" class="nav-link">System</a>
+                <a href="/offsite" class="nav-link">Offsite</a>
+                <a href="/configuration" class="nav-link">Configuration</a>
+            </nav>
+        </header>
+        
+
+        <a href="/hosts" class="back-link">&larr; Hosts</a>
+
+        
+        
+        <section class="card">
+            <div style="display: flex; justify-content: space-between; align-items: center; flex-wrap: wrap; gap: 0.5rem;">
+                <h2 style="margin: 0;">d53-test-9202</h2>
+                <span class="status-badge status-badge-pending">NO REPORT</span>
+            </div>
+            <div class="info-grid" style="margin-top: 1rem;">
+                <div class="info-item">
+                    <span class="label">Host ID</span>
+                    <span class="value" style="font-family: var(--font-mono)">d53-test-9202</span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Customer</span>
+                    <span class="value"><a href="/customers/demo-hp">Demo HP (scratch guest 9202)</a></span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Agent Version</span>
+                    <span class="value">—</span>
+                </div>
+                
+                
+                <div class="info-item">
+                    <span class="label">Enrolled</span>
+                    <span class="value">22 min ago</span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Last Report</span>
+                    <span class="value">waiting for first report</span>
+                </div>
+                
+                
+                <div class="info-item">
+                    <span class="label">Box connection</span>
+                    <span class="value" data-box-connection="not_connected" style="color: var(--warn)">not connected since the hub started (2026-10-26 17:28 UTC, 23 min ago)</span>
+                    
+                </div>
+                <div class="info-item">
+                    <span class="label">Desired Generation</span>
+                    <span class="value">0</span>
+                </div>
+                
+            </div>
+        </section>
+
+        
+        <section class="card">
+            <div class="empty-state">
+                <p>Waiting for first report.</p>
+                <p class="hint">Vitals, guests and storage appear once this host's agent sends a host-report.</p>
+            </div>
+        </section>
+        
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Guests</h2>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No guests reported on this host.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" id="held-apps" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Held apps</h2>
+            
+            <table class="data-table">
+                <thead>
+                    <tr>
+                        <th>App</th>
+                        <th>Reason</th>
+                        <th>Since</th>
+                        <th>Why</th>
+                        <th>Who may clear it</th>
+                    </tr>
+                </thead>
+                <tbody>
+                    
+                    <tr data-held-app="gokapi">
+                        <td><code>gokapi</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>stopped by the box for a crash loop (2026-09-24T10:37:48Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                    <tr data-held-app="chaosoom">
+                        <td><code>chaosoom</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>stopped by the box for a oom storm (2026-09-24T11:48:01Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                    <tr data-held-app="chaoscrash">
+                        <td><code>chaoscrash</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>stopped by the box for a crash loop (2026-09-24T12:16:24Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                    <tr data-held-app="chaosoomb">
+                        <td><code>chaosoomb</code></td>
+                        <td><span class="severity-badge severity-warning">unhealthy_stop</span></td>
+                        <td>32d ago</td>
+                        <td>stopped by the box for a oom storm (2026-09-24T12:39:51Z) — the household&#39;s Start gives it one more try</td>
+                        <td>the household&#39;s Start (one more try) <span class="text-muted">(operator reminded)</span></td>
+                    </tr>
+                    
+                </tbody>
+            </table>
+            
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Storage Targets</h2>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No storage targets reported.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" style="padding: 0; overflow: hidden;">
+            <h2 style="padding: 1.25rem 1.25rem 0.5rem;">Capabilities</h2>
+            
+            
+            <div class="empty-state" style="border: none;">
+                <p>No capability report yet (waiting for the first host report).</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Diagnostics — Log Bundles</h2>
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                Pull-based: the box ships its debug ring on its own next cycle — controller &le; one report interval (~15 min),
+                agent &asymp; one heartbeat. The pull is recorded in the box's own log (customer-visible). Bundles expire after 72 h.
+            </p>
+            <div style="display: flex; gap: 0.5rem; margin: 0.75rem 0;">
+                <form method="POST" action="/hosts/d53-test-9202/request-logs" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="component" value="controller">
+                    <button type="submit" class="btn btn-sm">Request controller logs</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/request-logs" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="component" value="agent">
+                    <button type="submit" class="btn btn-sm">Request agent logs</button>
+                </form>
+            </div>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No log bundles. Use the request buttons above — the box delivers on its next cycle.</p>
+            </div>
+            
+        </section>
+
+        
+        <section class="card" id="operator-actions">
+            <h2>Operator Actions</h2>
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                The box acts on its next report reply (usually seconds, at most one report interval) and answers on the
+                report after that. Each press runs once. Recorded in the box's own log and in Events. Unanswered after a day: expired.
+            </p>
+            <div style="display: flex; flex-wrap: wrap; gap: 0.5rem; margin: 0.75rem 0;">
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="offsite_backup_now">
+                    <button type="submit" class="btn btn-sm">Run off-site backup now</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="run_job">
+                    <select name="arg"><option value="disk-health-check">disk-health-check</option><option value="fill-watch">fill-watch</option><option value="offsite-integrity">offsite-integrity</option><option value="offsite-proof">offsite-proof</option></select>
+                    <button type="submit" class="btn btn-sm">Run check now</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="abandon_stop">
+                    <button type="submit" class="btn btn-sm btn-outline">Stop deletion countdown</button>
+                </form>
+                <form method="POST" action="/hosts/d53-test-9202/operator-action" style="display: inline;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="action" value="abandon_extend">
+                    <input type="number" name="arg" min="1" max="30" value="7" style="width: 4.5em; padding: 0.3em 0.5em;" aria-label="days">
+                    <button type="submit" class="btn btn-sm btn-outline">Extend countdown (days from now)</button>
+                </form>
+            </div>
+            
+            <div class="empty-state" style="border: none;">
+                <p>No operator actions for this box yet.</p>
+            </div>
+            
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Network</h2>
+            <div class="info-grid">
+                <div class="info-item">
+                    <span class="label">WireGuard</span>
+                    <span class="value">
+                        <span class="text-muted">no peer allocated</span>
+                    </span>
+                </div>
+            </div>
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem; margin-top: 0.75rem;">
+                Waiting for the first host report.
+            </p>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>Guest network
+                <span class="badge badge-neutral" title="A capable agent sent no guest-network stanza">unknown</span>
+                
+            </h2>
+
+            
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                This host reported no guest-network state, so it is <strong>unknown</strong> &mdash; not healthy.
+                The watchdog is default-on; a silent capable agent means it was switched off
+                (<code>guest_net.disable</code>), or this report predates it on this box.
+            </p>
+            
+        </section>
+
+        
+        <section class="card">
+            <h2>DR / Backup</h2>
+            <div class="info-grid">
+                <div class="info-item">
+                    <span class="label">DR Recipe</span>
+                    <span class="value"><span class="text-muted">none</span></span>
+                </div>
+                <div class="info-item">
+                    <span class="label">Key Escrow</span>
+                    <span class="value"><span class="text-muted">none</span></span>
+                </div>
+                <div class="info-item" id="offsite-integrity">
+                    <span class="label">Last full check (off-site)</span>
+                    <span class="value">not reported (no off-site status in the box&#39;s report)</span>
+                </div>
+            </div>
+        </section>
+
+        
+        <section class="card">
+            <h2>Console access</h2>
+            
+            <p><span class="badge badge-neutral">not vaulted</span></p>
+            <p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
+                No console credential is vaulted for this host. Expected for a byo host — the owner manages root@pam. Otherwise the installer's step 4b did not run; re-run felhom-host-install.sh, or set and vault one per the break-glass runbook &sect;5.
+            </p>
+            
+        </section>
+        
+
+        
+        
+        <section class="card" style="border-color: var(--crit);">
+            <h2>Danger zone</h2>
+            <p class="text-muted" style="font-size: 0.85rem;">
+                Removing this host deletes its reports, guests, log bundles and WireGuard peer, and
+                permanently invalidates its API key — a still-running agent would receive 401s.
+                Re-enrollment requires the Day-0 passphrase flow.
+            </p>
+            <button type="button" class="btn btn-danger btn-sm" onclick="hostDeleteConfirm('d53-test-9202')">Remove host&hellip;</button>
+            <div id="host-delete-confirm-d53-test-9202" style="display: none; margin-top: 0.75rem; padding: 0.75rem; border: 1px solid var(--crit); background: var(--crit-dim); border-radius: var(--radius); max-width: 44em;">
+                <p id="host-delete-impact-d53-test-9202" style="margin: 0 0 0.5rem; font-size: 0.9em;">&hellip;</p>
+                <label id="host-delete-escrow-row-d53-test-9202" style="display: none; margin: 0 0 0.5rem; font-size: 0.85em;">
+                    <input type="checkbox" id="host-delete-escrow-d53-test-9202">
+                    Move key escrow to retained custody (required when escrow present) + remove DR bundle for this host
+                </label>
+                
+                <p style="margin: 0 0 0.5rem; font-size: 0.85em; color: var(--text-2);">Type the host id to confirm:</p>
+                <form method="POST" action="/hosts/d53-test-9202/delete" id="host-delete-form-d53-test-9202" style="display: flex; gap: 0.5rem; align-items: center; flex-wrap: wrap;">
+                    <input type="hidden" name="_csrf" value="<redacted-test-csrf>">
+                    <input type="hidden" name="confirm_host_id" id="host-delete-confirm-hidden-d53-test-9202" value="">
+                    <input type="hidden" name="delete_escrow" id="host-delete-escrow-hidden-d53-test-9202" value="">
+                    
+                    <input type="text" id="host-delete-confirm-input-d53-test-9202" placeholder="retype the host id&hellip;" style="padding: 0.3em 0.5em; width: 16em;">
+                    <button type="button" class="btn btn-danger btn-sm" onclick="hostDeleteSubmit('d53-test-9202')">Confirm &amp; remove</button>
+                    <button type="button" class="btn btn-sm btn-outline" onclick="document.getElementById('host-delete-confirm-d53-test-9202').style.display='none';">Cancel</button>
+                </form>
+                <p id="host-delete-err-d53-test-9202" style="margin: 0.4em 0 0; font-size: 0.8em; color: var(--crit);"></p>
+            </div>
+        </section>
+        <script>
+        function hostDeleteConfirm(hostID) {
+            var box = document.getElementById('host-delete-confirm-' + hostID);
+            var impact = document.getElementById('host-delete-impact-' + hostID);
+            document.getElementById('host-delete-confirm-input-' + hostID).value = '';
+            document.getElementById('host-delete-err-' + hostID).textContent = '';
+            box.style.display = 'block';
+            impact.textContent = 'Checking impact…';
+            fetch('/hosts/' + encodeURIComponent(hostID) + '/delete-impact')
+                .then(function(r){ return r.json(); })
+                .then(function(d){
+                    var parts = ['Deleting ' + hostID + ' removes ' + d.guests + ' guest row(s), ' +
+                        d.reports + ' host report(s), ' + d.log_bundles + ' agent log bundle(s)'];
+                    if (d.wg_peer_bound) parts.push('the bound WireGuard peer');
+                    if (d.pbs_secret_present) parts.push('the staged PBS secret');
+                    if (d.recovery_present) parts.push('the break-glass recovery credential');
+                    impact.textContent = parts.join(', ') + '. Host status: ' + d.status + '.' +
+                        (d.deletable ? (d.off_tick_required ? ' No box connection for 6+ minutes — the "box is off" tick is required.' : '') : ' Host is ONLINE — deletion will be refused.');
+                    document.getElementById('host-delete-escrow-row-' + hostID).style.display =
+                        d.escrow_present ? 'block' : 'none';
+                })
+                .catch(function(){ impact.textContent = 'Could not compute the impact — the server will still enforce every gate.'; });
+        }
+        function hostDeleteSubmit(hostID) {
+            var typed = document.getElementById('host-delete-confirm-input-' + hostID).value.trim();
+            var err = document.getElementById('host-delete-err-' + hostID);
+            if (typed !== hostID) { err.textContent = 'Confirmation does not match the host id.'; return; }
+            var escrowRow = document.getElementById('host-delete-escrow-row-' + hostID);
+            var escrowCb = document.getElementById('host-delete-escrow-' + hostID);
+            if (escrowRow.style.display !== 'none' && !escrowCb.checked) {
+                err.textContent = 'This host has a key escrow — tick the acknowledgement to delete it too.';
+                return;
+            }
+            var offCb = document.getElementById('host-delete-off-' + hostID);
+            if (offCb && !offCb.checked) {
+                err.textContent = 'Tick "I checked: the box is off" first.';
+                return;
+            }
+            if (offCb) { document.getElementById('host-delete-off-hidden-' + hostID).value = '1'; }
+            document.getElementById('host-delete-confirm-hidden-' + hostID).value = typed;
+            document.getElementById('host-delete-escrow-hidden-' + hostID).value = escrowCb.checked ? '1' : '';
+            document.getElementById('host-delete-form-' + hostID).submit();
+        }
+        </script>
+        
+
+
+        <footer style="margin-top: 2rem; color: var(--text-muted); font-size: 0.8rem; text-align: center;">
+            Felhom Hub <span style="font-family: var(--font-mono)">0.173.0-d53test</span>
+        </footer>
+    </div>
+</body>
+</html>

documentation/audits/d53-held-apps-2026-10-26/live/9202-controller-log.txt

@@ -0,0 +1,253 @@
+2026/10/26 17:29:52 notifier.go:81: [INFO] Notifier enabled (hub: http://192.168.0.180:18090)
+2026/10/26 17:29:52 scheduler.go:120: [INFO] [scheduler] Registered periodic job: hold-report (every 1m0s)
+2026/10/26 17:29:52 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="hold-report" interval=1m0s totalJobs=8
+2026/10/26 17:29:52 notifier.go:1145: [DEBUG] SyncPreferences: url=http://192.168.0.180:18090/api/v1/preferences source=default email= events=[backup_failed db_dump_failed disk_warning disk_critical storage_disconnected node_down health_critical expected_backup_missed expected_dbdump_missed offbox_enlarge_blocked app_update_undone app_update_held app_stopped_unhealthy storage_reconnected backup_integrity_failed crossdrive_failed health_recovered] cooldown=6h
+2026/10/26 17:29:52 notifier.go:1176: [INFO] Notification preferences synced to hub: source=default status=ok email=, events=[backup_failed db_dump_failed disk_warning disk_critical storage_disconnected node_down health_critical expected_backup_missed expected_dbdump_missed offbox_enlarge_blocked app_update_undone app_update_held app_stopped_unhealthy storage_reconnected backup_integrity_failed crossdrive_failed health_recovered], cooldown=6h, email_cleared=false
+2026/10/26 17:29:52 sync.go:559: [DEBUG] [sync] radicale/docker-compose.yml: hash match, skipped
+2026/10/26 17:29:52 sync.go:559: [DEBUG] [sync] radicale/.felhom.yml: hash match, skipped
+2026/10/26 17:30:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:30:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:30:52 holdreport.go:137: [INFO] [holdreport] chaoscrash held (unhealthy_stop since 2026-09-24T12:16:24Z) — reported to the hub
+2026/10/26 17:30:52 holdreport.go:137: [INFO] [holdreport] chaosoom held (unhealthy_stop since 2026-09-24T11:48:01Z) — reported to the hub
+2026/10/26 17:30:52 holdreport.go:137: [INFO] [holdreport] chaosoomb held (unhealthy_stop since 2026-09-24T12:39:51Z) — reported to the hub
+2026/10/26 17:30:52 holdreport.go:137: [INFO] [holdreport] gokapi held (unhealthy_stop since 2026-09-24T10:37:48Z) — reported to the hub
+2026/10/26 17:30:52 holdreport.go:137: [DEBUG] [holdreport] full re-send: 4 hold(s) sent, 0 cleared
+2026/10/26 17:30:52 notifier.go:317: [DEBUG] PushEvent: type=app_held severity=info url=http://192.168.0.180:18090/api/v1/event
+2026/10/26 17:30:52 notifier.go:317: [DEBUG] PushEvent: type=app_held severity=info url=http://192.168.0.180:18090/api/v1/event
+2026/10/26 17:30:52 notifier.go:317: [DEBUG] PushEvent: type=app_held severity=info url=http://192.168.0.180:18090/api/v1/event
+2026/10/26 17:30:52 notifier.go:317: [DEBUG] PushEvent: type=app_held severity=info url=http://192.168.0.180:18090/api/v1/event
+2026/10/26 17:30:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 1ms)
+2026/10/26 17:30:52 notifier.go:343: [DEBUG] PushEvent: app_held pushed OK (HTTP 200)
+2026/10/26 17:30:52 notifier.go:355: [INFO] Event pushed: app_held (info) [hu-only] — App chaosoom is held: stopped by the box for a oom storm (2026-09-24T11:48:01Z) — the household's Start gives it one more try
+2026/10/26 17:30:52 notifier.go:343: [DEBUG] PushEvent: app_held pushed OK (HTTP 200)
+2026/10/26 17:30:52 notifier.go:355: [INFO] Event pushed: app_held (info) [hu-only] — App gokapi is held: stopped by the box for a crash loop (2026-09-24T10:37:48Z) — the household's Start gives it one more try
+2026/10/26 17:30:52 notifier.go:343: [DEBUG] PushEvent: app_held pushed OK (HTTP 200)
+2026/10/26 17:30:52 notifier.go:355: [INFO] Event pushed: app_held (info) [hu-only] — App chaoscrash is held: stopped by the box for a crash loop (2026-09-24T12:16:24Z) — the household's Start gives it one more try
+2026/10/26 17:30:52 notifier.go:343: [DEBUG] PushEvent: app_held pushed OK (HTTP 200)
+2026/10/26 17:30:52 notifier.go:355: [INFO] Event pushed: app_held (info) [hu-only] — App chaosoomb is held: stopped by the box for a oom storm (2026-09-24T12:39:51Z) — the household's Start gives it one more try
+2026/10/26 17:31:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:31:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:31:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:32:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:32:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:32:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:33:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:33:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:33:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:34:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:34:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:34:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:35:41 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale
+2026/10/26 17:35:41 router.go:78: [DEBUG] [api] GET /api/stacks/radicale (path=/stacks/radicale)
+2026/10/26 17:35:41 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale/deploy-fields
+2026/10/26 17:35:41 router.go:78: [DEBUG] [api] GET /api/stacks/radicale/deploy-fields (path=/stacks/radicale/deploy-fields)
+2026/10/26 17:35:41 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/radicale/deploy
+2026/10/26 17:35:41 router.go:78: [DEBUG] [api] POST /api/stacks/radicale/deploy (path=/stacks/radicale/deploy)
+2026/10/26 17:35:41 router.go:442: [INFO] [api] Deploy requested for stack: radicale
+2026/10/26 17:35:41 router.go:78: [DEBUG] [api] deployStack: name=radicale contentLength=145
+2026/10/26 17:35:41 deploy.go:338: [DEBUG] Deploy radicale: received 4 user values
+2026/10/26 17:35:41 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 1 encrypted, 1 sensitive fields
+2026/10/26 17:35:41 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/26 17:35:41 deploy.go:574: [INFO] [stacks] Deploying stack radicale with 4 env vars: [SUBDOMAIN, RADICALE_USER, RADICALE_PASSWORD, DOMAIN]
+2026/10/26 17:35:41 manager.go:1856: [INFO] [stacks] Deploying stack radicale — checking 1 images...
+2026/10/26 17:35:41 manager.go:1860: [DEBUG]   ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c — not found locally, will pull
+2026/10/26 17:35:41 lifecycle.go:70: [DEBUG] [integrations] OnStackStart: stack=radicale, waiting 5s for state refresh
+2026/10/26 17:35:41 manager.go:1701: [DEBUG] Running: docker compose up -d (in /opt/docker/stacks/radicale)
+2026/10/26 17:35:46 lifecycle.go:86: [DEBUG] [integrations] OnStackStart: stack=radicale integrationsFound=0
+2026/10/26 17:35:46 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale
+2026/10/26 17:35:46 router.go:78: [DEBUG] [api] GET /api/stacks/radicale (path=/stacks/radicale)
+2026/10/26 17:35:50 deploy.go:664: [INFO] [stacks] Stack radicale deployed successfully (took 9.0s)
+2026/10/26 17:35:50 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 1 encrypted, 1 sensitive fields
+2026/10/26 17:35:50 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/26 17:35:50 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 0 encrypted, 1 sensitive fields
+2026/10/26 17:35:50 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/26 17:35:50 installed.go:420: [INFO] [stacks] installed-images radicale: recorded 1 service(s) (radicale=ghcr.io/kozea/radicale:3.8.1 (sha256:54d9406cd30f…))
+2026/10/26 17:35:50 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 0 encrypted, 1 sensitive fields
+2026/10/26 17:35:50 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/26 17:35:50 pin.go:93: [INFO] [stacks] pin radicale: radicale=ghcr.io/kozea/radicale:3.8.1
+2026/10/26 17:35:50 [DEBUG] [backup] radicale: off-site default not applied: no off-site target
+2026/10/26 17:35:51 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale
+2026/10/26 17:35:51 router.go:78: [DEBUG] [api] GET /api/stacks/radicale (path=/stacks/radicale)
+2026/10/26 17:35:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:35:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:35:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:35:53 manager.go:1701: [DEBUG] Running: docker compose ps -a --format table {{.Name}}	{{.Image}}	{{.State}}	{{.Status}} (in /opt/docker/stacks/radicale)
+2026/10/26 17:35:53 manager.go:1817: [INFO] [stacks] Stack radicale post-start status:
+2026/10/26 17:35:53 manager.go:1820: [INFO] [stacks]   radicale   ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c   running   Up 3 seconds (health: starting)
+2026/10/26 17:35:56 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale
+2026/10/26 17:35:56 router.go:78: [DEBUG] [api] GET /api/stacks/radicale (path=/stacks/radicale)
+2026/10/26 17:36:01 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale
+2026/10/26 17:36:01 router.go:78: [DEBUG] [api] GET /api/stacks/radicale (path=/stacks/radicale)
+2026/10/26 17:36:06 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale
+2026/10/26 17:36:06 router.go:78: [DEBUG] [api] GET /api/stacks/radicale (path=/stacks/radicale)
+2026/10/26 17:36:12 healthprobe.go:187: [DEBUG] Health probe radicale: API GET :5232/.web/ → 200 (4ms)
+2026/10/26 17:36:22 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 10s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:36:42 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 30s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:36:52 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 40s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:36:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:36:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:36:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:37:02 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 50s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:37:12 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m0s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:37:32 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m20s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:37:42 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m30s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:37:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:37:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:37:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:37:52 main.go:4311: [WARN] [deadapp] radicale: crash_loop — 7 in 10m0s; STOPPING it (decision 28)
+2026/10/26 17:37:52 manager.go:1443: [DEBUG] [stacks] StopStack radicale: current state=starting deployed=true containers=1
+2026/10/26 17:37:52 manager.go:1446: [INFO] [stacks] Stopping stack: radicale
+2026/10/26 17:37:52 manager.go:1701: [DEBUG] Running: docker compose down (in /opt/docker/stacks/radicale)
+2026/10/26 17:37:52 manager.go:1455: [INFO] [stacks] Stack radicale stopped successfully (took 0.3s)
+2026/10/26 17:37:52 settings.go:2043: [WARN] [settings] restore hold SET for radicale — the app stays stopped until it is cleared
+2026/10/26 17:37:52 update_guard.go:873: [WARN] [backup] radicale is STOPPED by the box: crash_loop (trip 1 within 24h0m0s) — Start gives it one more try (decision 28)
+2026/10/26 17:37:52 notifier.go:317: [DEBUG] PushEvent: type=app_stopped_unhealthy severity=warning url=http://192.168.0.180:18090/api/v1/event
+2026/10/26 17:37:52 notifier.go:343: [DEBUG] PushEvent: app_stopped_unhealthy pushed OK (HTTP 200)
+2026/10/26 17:37:52 notifier.go:355: [INFO] Event pushed: app_stopped_unhealthy (warning) [hu-only] — A(z) radicale alkalmazást leállítottuk, mert újra és újra összeomlott (vagy elfogyott a memóriája). Az Indítás gombbal újra megpróbálhatod.
+2026/10/26 17:38:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:38:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:38:52 holdreport.go:137: [INFO] [holdreport] radicale held (unhealthy_stop since 2026-10-26T17:37:52Z) — reported to the hub
+2026/10/26 17:38:52 notifier.go:317: [DEBUG] PushEvent: type=app_held severity=info url=http://192.168.0.180:18090/api/v1/event
+2026/10/26 17:38:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 1ms)
+2026/10/26 17:38:52 notifier.go:343: [DEBUG] PushEvent: app_held pushed OK (HTTP 200)
+2026/10/26 17:38:52 notifier.go:355: [INFO] Event pushed: app_held (info) [hu-only] — App radicale is held: stopped by the box for a crash loop (2026-10-26T17:37:52Z) — the household's Start gives it one more try
+2026/10/26 17:39:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:39:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:39:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:39:52 backup.go:540: [DEBUG] groupStacksByDrive: /mnt/sys_drive → [privatebin, radicale]
+2026/10/26 17:40:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:40:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:40:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:41:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:41:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:41:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:42:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:42:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:42:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:43:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:43:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:43:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:44:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:44:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:44:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:44:52 backup.go:540: [DEBUG] groupStacksByDrive: /mnt/sys_drive → [privatebin, radicale]
+2026/10/26 17:44:52 sync.go:559: [DEBUG] [sync] radicale/docker-compose.yml: hash match, skipped
+2026/10/26 17:44:52 sync.go:559: [DEBUG] [sync] radicale/.felhom.yml: hash match, skipped
+2026/10/26 17:45:37 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/radicale/start
+2026/10/26 17:45:37 router.go:78: [DEBUG] [api] POST /api/stacks/radicale/start (path=/stacks/radicale/start)
+2026/10/26 17:45:37 router.go:663: [INFO] [api] start requested for stack: radicale
+2026/10/26 17:45:37 router.go:78: [DEBUG] [api] actionStack: action=start name=radicale
+2026/10/26 17:45:37 settings.go:2960: [INFO] [settings] unhealthy-stop hold LIFTED for radicale (Start)
+2026/10/26 17:45:37 router.go:687: [INFO] [api] radicale: the unhealthy stop is LIFTED by Start — one more try (decision 28)
+2026/10/26 17:45:37 manager.go:1354: [DEBUG] [stacks] StartStack radicale: current state=stopped deployed=true
+2026/10/26 17:45:37 manager.go:1357: [INFO] [stacks] Starting stack: radicale
+2026/10/26 17:45:37 manager.go:1364: [DEBUG] [stacks] StartStack radicale: prepared 10 env vars for compose
+2026/10/26 17:45:37 manager.go:1701: [DEBUG] Running: docker compose up -d (in /opt/docker/stacks/radicale)
+2026/10/26 17:45:37 manager.go:1372: [INFO] [stacks] Stack radicale started successfully (took 0.4s)
+2026/10/26 17:45:38 installed.go:409: [DEBUG] [stacks] installed-images radicale: unchanged (1 service(s)) — app.yaml not rewritten
+2026/10/26 17:45:38 lifecycle.go:70: [DEBUG] [integrations] OnStackStart: stack=radicale, waiting 5s for state refresh
+2026/10/26 17:45:41 manager.go:1701: [DEBUG] Running: docker compose ps -a --format table {{.Name}}	{{.Image}}	{{.State}}	{{.Status}} (in /opt/docker/stacks/radicale)
+2026/10/26 17:45:41 manager.go:1817: [INFO] [stacks] Stack radicale post-start status:
+2026/10/26 17:45:41 manager.go:1820: [INFO] [stacks]   radicale   ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c   running   Up 3 seconds (health: starting)
+2026/10/26 17:45:43 lifecycle.go:86: [DEBUG] [integrations] OnStackStart: stack=radicale integrationsFound=0
+2026/10/26 17:45:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:45:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:45:52 holdreport.go:137: [INFO] [holdreport] radicale hold cleared — reported to the hub
+2026/10/26 17:45:52 notifier.go:317: [DEBUG] PushEvent: type=app_hold_cleared severity=info url=http://192.168.0.180:18090/api/v1/event
+2026/10/26 17:45:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 1ms)
+2026/10/26 17:45:52 notifier.go:343: [DEBUG] PushEvent: app_hold_cleared pushed OK (HTTP 200)
+2026/10/26 17:45:52 notifier.go:355: [INFO] Event pushed: app_hold_cleared (info) [hu-only] — App radicale is no longer held
+2026/10/26 17:45:52 healthprobe.go:187: [DEBUG] Health probe radicale: API GET :5232/.web/ → 200 (3ms)
+2026/10/26 17:46:02 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 10s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:46:12 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 20s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:46:22 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 30s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:46:32 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 40s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:46:42 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 50s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:46:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:46:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:46:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:46:52 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m0s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:47:02 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m10s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:47:12 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m20s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:47:22 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m30s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:47:32 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m40s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:47:42 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 1m50s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:47:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:47:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:47:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:47:52 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 2m0s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:48:02 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 2m10s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:48:12 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 2m20s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:48:22 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:48:32 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 2m40s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:48:42 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:48:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:48:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:48:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:48:52 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:49:02 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:49:12 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 3m20s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:49:22 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 3m30s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:49:32 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 3m40s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:49:42 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 3m50s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:49:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:49:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:49:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:49:52 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 4m0s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:49:52 backup.go:540: [DEBUG] groupStacksByDrive: /mnt/sys_drive → [privatebin, radicale]
+2026/10/26 17:49:52 dbdump.go:149: [DEBUG] DiscoverDatabases: docker ps output: 9fa4e86f3613	radicale	radicale	ghcr.io/kozea/radicale:3.8.1
+2026/10/26 17:49:52 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container radicale (image=ghcr.io/kozea/radicale:3.8.1, not a database)
+2026/10/26 17:49:52 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for radicale → /mnt/sys_drive/felhom-data/backups/primary/radicale (images=1, secrets-referenced=1, data_keys=0, portable-carried=0/0, withheld=1)
+2026/10/26 17:50:02 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 4m10s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:50:12 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 4m20s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:50:22 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 4m30s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:50:32 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 4m40s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:50:42 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 4m50s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:50:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:50:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:50:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:50:52 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 5m0s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:51:02 healthprobe.go:187: [DEBUG] Health probe radicale: API GET :5232/.web/ → 200 (3ms)
+2026/10/26 17:51:12 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 10s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:51:22 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 20s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:51:32 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping radicale — last check 30s ago, effective interval 5m0s, healthy=true
+2026/10/26 17:51:34 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/radicale/stop
+2026/10/26 17:51:34 router.go:78: [DEBUG] [api] POST /api/stacks/radicale/stop (path=/stacks/radicale/stop)
+2026/10/26 17:51:34 router.go:663: [INFO] [api] stop requested for stack: radicale
+2026/10/26 17:51:34 router.go:78: [DEBUG] [api] actionStack: action=stop name=radicale
+2026/10/26 17:51:34 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 0 encrypted, 1 sensitive fields
+2026/10/26 17:51:34 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/26 17:51:34 desiredstate.go:81: [INFO] [stacks] desired state for radicale recorded as "stopped" (was "running")
+2026/10/26 17:51:34 manager.go:1443: [DEBUG] [stacks] StopStack radicale: current state=running deployed=true containers=1
+2026/10/26 17:51:34 manager.go:1446: [INFO] [stacks] Stopping stack: radicale
+2026/10/26 17:51:34 manager.go:1701: [DEBUG] Running: docker compose down (in /opt/docker/stacks/radicale)
+2026/10/26 17:51:35 manager.go:1455: [INFO] [stacks] Stack radicale stopped successfully (took 0.4s)
+2026/10/26 17:51:35 lifecycle.go:24: [DEBUG] [integrations] OnStackStop: stack=radicale integrationsFound=0
+2026/10/26 17:51:40 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale
+2026/10/26 17:51:40 router.go:78: [DEBUG] [api] GET /api/stacks/radicale (path=/stacks/radicale)
+2026/10/26 17:51:40 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/radicale/remove
+2026/10/26 17:51:40 router.go:78: [DEBUG] [api] POST /api/stacks/radicale/remove (path=/stacks/radicale/remove)
+2026/10/26 17:51:40 router.go:1025: [INFO] [api] Remove requested for stack: radicale
+2026/10/26 17:51:40 router.go:78: [DEBUG] [api] removeStack: name=radicale
+2026/10/26 17:51:40 router.go:78: [DEBUG] [api] removeStack: name=radicale removeHDDData=true removeBackups=true
+2026/10/26 17:51:40 delete.go:626: [DEBUG] [stacks] RemoveStack called: name="radicale", removeHDDData=true, backupPathsToRemove=3
+2026/10/26 17:51:40 delete.go:640: [DEBUG] [stacks] RemoveStack radicale: state=stopped, deployed=true, orphaned=false, deploying=false
+2026/10/26 17:51:40 delete.go:697: [INFO] Removing deployed stack: radicale (removeHDDData=true, hddDeclared=false, backupPaths=3)
+2026/10/26 17:51:40 delete.go:710: [DEBUG] [stacks] RemoveStack radicale: found 0 HDD mounts from compose file
+2026/10/26 17:51:40 manager.go:1701: [DEBUG] Running: docker compose down --volumes (in /opt/docker/stacks/radicale)
+2026/10/26 17:51:40 delete.go:736: [DEBUG] [stacks] RemoveStack radicale: compose down output: 
+2026/10/26 17:51:52 scheduler.go:450: [INFO] [scheduler] Running job: hold-report
+2026/10/26 17:51:52 scheduler.go:85: [DEBUG] [scheduler] job hold-report: execution starting
+2026/10/26 17:51:52 scheduler.go:467: [INFO] [scheduler] Job hold-report completed (took 0s)
+2026/10/26 17:52:06 delete.go:774: [INFO] [stacks] RemoveStack radicale: removed volume(s) [radicale_radicale_data]
+2026/10/26 17:52:06 delete.go:837: [DEBUG] [stacks] RemoveStack radicale: processing 3 backup paths for removal (base=/mnt/sys_drive/felhom-data/backups)
+2026/10/26 17:52:06 delete.go:854: [INFO] Removed backup data: /mnt/sys_drive/felhom-data/backups/primary/radicale (36K)
+2026/10/26 17:52:06 delete.go:862: [DEBUG] [stacks] RemoveStack radicale: removing app.yaml at /opt/docker/stacks/radicale/app.yaml
+2026/10/26 17:52:06 delete.go:879: [INFO] Stack radicale removed successfully (took 26.8s)
+2026/10/26 17:52:07 lifecycle.go:176: [DEBUG] [integrations] OnStackRemove: stack=radicale integrationsFound=0
+2026/10/26 17:52:07 notifier.go:355: [INFO] Event pushed: app_removed (info) [hu-only] — Alkalmazás eltávolítva: radicale
+2026/10/26 17:52:07 image_retention.go:265: [INFO] [stacks] image retention (remove of radicale): deleted [ghcr.io/kozea/radicale:<none>] (0139a3fcd1ed, 96.6MB) — no container, installed app or undo names it (decision 53)
+2026/10/26 17:52:07 image_retention.go:238: [INFO] [stacks] image retention (remove of radicale): pass over 40 image(s) of [ghcr.io/kozea/radicale] — 1 candidate(s), 1 deleted, the rest kept
+2026/10/26 17:52:12 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/radicale
+2026/10/26 17:52:12 router.go:78: [DEBUG] [api] GET /api/stacks/radicale (path=/stacks/radicale)
+privatebin

documentation/audits/d53-held-apps-2026-10-26/live/held_apps_final.txt

@@ -0,0 +1 @@
+held_apps after: [('chaosoom', 'unhealthy_stop'), ('gokapi', 'unhealthy_stop'), ('chaoscrash', 'unhealthy_stop'), ('chaosoomb', 'unhealthy_stop')]

documentation/audits/d53-held-apps-2026-10-26/live/notification_log.txt

@@ -0,0 +1,14 @@
+['id', 'customer_id', 'event_type', 'severity', 'message', 'status', 'error_message', 'created_at', 'channel']
+['1', 'demo-hp', 'app_definition_refused', 'warning', 'immich: the box REFUSED a catalog definition — it asks for c', 'sent', '', '2026-10-26 17:29:52', 'operator']
+['2', 'demo-hp', 'app_definition_refused', 'warning', 'immich: the box REFUSED a catalog definition — it asks for c', 'skipped', 'operator_only', '2026-10-26 17:29:52', 'customer']
+['3', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app gokapi has been held for 32 days (', 'sent', '', '2026-10-26 17:33:35', 'operator']
+['4', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app gokapi has been held for 32 days (', 'skipped', 'operator_only', '2026-10-26 17:33:35', 'customer']
+['5', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app chaosoom has been held for 32 days', 'sent', '', '2026-10-26 17:33:35', 'operator']
+['6', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app chaosoom has been held for 32 days', 'skipped', 'operator_only', '2026-10-26 17:33:35', 'customer']
+['7', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app chaoscrash has been held for 32 da', 'sent', '', '2026-10-26 17:33:35', 'operator']
+['8', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app chaoscrash has been held for 32 da', 'skipped', 'operator_only', '2026-10-26 17:33:35', 'customer']
+['9', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app chaosoomb has been held for 32 day', 'sent', '', '2026-10-26 17:33:35', 'operator']
+['10', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app chaosoomb has been held for 32 day', 'skipped', 'operator_only', '2026-10-26 17:33:35', 'customer']
+['11', 'demo-hp', 'app_stopped_unhealthy', 'warning', 'A(z) radicale alkalmazást leállítottuk, mert újra és újra ös', 'sent', '', '2026-10-26 17:37:53', 'operator']
+['12', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app radicale has been held for 4m0s (u', 'sent', '', '2026-10-26 17:41:35', 'operator']
+['13', 'demo-hp', 'app_held_reminder', 'warning', 'Customer demo-hp: the app radicale has been held for 4m0s (u', 'skipped', 'operator_only', '2026-10-26 17:41:35', 'customer']

documentation/audits/d53-held-apps-2026-10-26/live/test-hub.log

@@ -0,0 +1,82 @@
+2026/10/26 18:28:34 [INFO] felhom-hub 0.173.0-d53test starting
+2026/10/26 18:28:34 [INFO] [store] app-update event types added to 0 household(s)' notification prefs (one-time, add-only): []
+2026/10/26 18:28:34 [INFO] [store] app_stopped_unhealthy added to 0 household(s)' notification prefs (one-time, add-only): []
+2026/10/26 18:28:34 [ERROR] OFFSITE_SECRET_KEY unset — off-site passwords cannot be sealed; provisioning and the key registrar are DISABLED
+2026/10/26 18:28:34 [INFO] Database opened at /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad/testhub/data/hub.db
+2026/10/26 18:28:34 [WARN] Cannot read seed dir /usr/share/felhom/assets-seed: open /usr/share/felhom/assets-seed: no such file or directory (no assets will be available until uploaded)
+2026/10/26 18:28:34 [INFO] Asset manifest built: 0 files
+2026/10/26 18:28:34 [INFO] App-email relay enabled (limit 30/min/customer, From domains [felhom.eu])
+2026/10/26 18:28:34 [INFO] osupdates: approval rule = every ring-0 box healthy for 24h0m0s and 1 night run(s)
+2026/10/26 18:28:34 [INFO] osupdates: the Docker engine set is approved only by the operator, after 2 healthy ring-0 night(s)
+2026/10/26 18:28:34 [INFO] osupdates: alarms = no OS leg 168h0m0s, reboot needed 336h0m0s, ring 0 stalled 168h0m0s, not covered 336h0m0s
+2026/10/26 18:28:34 [INFO] access log: /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad/testhub/data/access-log, one file per UTC day, kept 90 days
+2026/10/26 18:28:34 [INFO] Registry version checker disabled (no credentials configured)
+2026/10/26 18:28:34 [INFO] WG peer-sync disabled (endpoint not configured)
+2026/10/26 18:28:34 [INFO] PBS DR tenantsync disabled (key or endpoint not configured)
+2026/10/26 18:28:34 [INFO] agent-plane poke disabled (key or endpoint not configured)
+2026/10/26 18:28:34 [INFO] PBS-DR self-heal reconciler started (interval 5m)
+2026/10/26 18:28:34 [INFO] offsite credential self-heal reconciler started (interval 5m, debounce 2 reports)
+2026/10/26 18:28:34 [INFO] Staleness checker initialized: 0 ok, 0 stale, 0 down (node_stale after 45m0s, node_down after 1h30m0s)
+2026/10/26 18:28:34 [INFO] prune: next run at 2026-10-27 04:30 CET (in 10h1m25s)
+2026/10/26 18:28:34 [INFO] Host staleness checker initialized: 0 ok, 0 already alarmed (seeded silently, R-1001), 0 stale, 0 down (left unseeded → first Check emits; host_stale after 45m0s, host_down after 1h30m0s)
+2026/10/26 18:28:34 [INFO] Host capability checker initialized: 0 ok, 0 degraded, 0 with a refused whole-guest backup (degraded/refused left unseeded → first Check emits)
+2026/10/26 18:28:34 [INFO] Host leaf checker initialized: 0 host fingerprint(s) seeded
+2026/10/26 18:28:34 [INFO] Host disk checker initialized: warn=90% crit=95%, 0 ok seeded, 0 already-breached left unseeded (first Check emits)
+2026/10/26 18:28:34 [INFO] Storage fill checker initialized: warn=90% crit=95%, 0 ok seeded, 0 already-breached left unseeded, 0 root-backed excluded
+2026/10/26 18:28:34 [INFO] Host mgmt-plane checker initialized: 0 host heal-state(s) seeded
+2026/10/26 18:28:34 [INFO] Host OOB checker initialized: 0 host(s) seeded degraded
+2026/10/26 18:28:34 [INFO] Offsite checker initialized: fill warn=90% crit=95%, stale after 48h0m0s, 0 ok-seeded
+2026/10/26 18:28:34 [WARN] Held apps: the reminder grace is OVERRIDDEN to 3m0s for every reason — a test configuration
+2026/10/26 18:28:34 [WARN] Golden vouch check OFF: no registry credentials, so the hub cannot list published goldens
+2026/10/26 18:28:34 [INFO] db-snapshot 08:00: next run at 2026-10-27 08:00 CET (in 13h31m25s)
+2026/10/26 18:28:34 [INFO] db-snapshot 20:00: next run at 2026-10-26 20:00 CET (in 1h31m25s)
+2026/10/26 18:28:34 [INFO] deadline-check: next run at 2026-10-27 05:00 CET (in 10h31m25s)
+2026/10/26 18:28:34 [INFO] Listening on 192.168.0.180:18090
+2026/10/26 18:28:34 [INFO] db-snapshot 14:00: next run at 2026-10-27 14:00 CET (in 19h31m25s)
+2026/10/26 18:28:34 [INFO] db-snapshot 02:00: next run at 2026-10-27 02:00 CET (in 7h31m25s)
+2026/10/26 18:28:34 [INFO] db snapshot written: hub-20261026T172834Z.db (409600 bytes, 13ms); pruned 0, keeping 2
+2026/10/26 18:29:34 [INFO] osupdates: guest candidate  (0 packages, first seen 0001-01-01T00:00:00Z): waiting — no ring-0 box
+2026/10/26 18:29:34 [INFO] osupdates: host candidate  (0 packages, first seen 0001-01-01T00:00:00Z): waiting — no ring-0 box
+2026/10/26 18:29:34 [INFO] osupdates: alarm check done in 4ms, 0 alarm(s) sent []
+2026/10/26 18:29:52 [DEBUG] Notification prefs read for demo-hp: exists=false household_saved=false
+2026/10/26 18:29:52 [INFO] Notification preferences updated for demo-hp: address set=false, household=false, events=[backup_failed db_dump_failed disk_warning disk_critical storage_disconnected node_down health_critical expected_backup_missed expected_dbdump_missed offbox_enlarge_blocked app_update_undone app_update_held app_stopped_unhealthy storage_reconnected backup_integrity_failed crossdrive_failed health_recovered]
+2026/10/26 18:29:52 [INFO] Event from demo-hp: app_definition_refused (warning) — immich: the box REFUSED a catalog definition — it asks for container rights not on the box's allow-list. Nothing was written; the app keeps its last good compose. Check who pushed to the catalog. the catalog definition of immich asks for container rights the box does not allow: immich-server: bind /mnt/sizing-bench/appdata/immich -> /usr/src/app/upload rw; immich-server: bind /mnt/sizing-bench/userdata/media/photos -> /external/photos ro
+2026/10/26 18:29:52 [INFO] Operator email sent for demo-hp/app_definition_refused
+2026/10/26 18:29:57 [INFO] Event from demo-hp: controller_started (info) — Controller elindult (0.346.0-d53test)
+2026/10/26 18:29:58 [INFO] DR-recipe app-half stored for customer demo-hp (v1)
+2026/10/26 18:29:58 [INFO] Received report from demo-hp (8171 bytes)
+2026/10/26 18:30:52 [INFO] Event from demo-hp: app_held (info) — App chaosoom is held: stopped by the box for a oom storm (2026-09-24T11:48:01Z) — the household's Start gives it one more try
+2026/10/26 18:30:52 [DEBUG] [held-apps] demo-hp/chaosoom held (unhealthy_stop since 2026-09-24T11:48:01Z)
+2026/10/26 18:30:52 [INFO] Event from demo-hp: app_held (info) — App gokapi is held: stopped by the box for a crash loop (2026-09-24T10:37:48Z) — the household's Start gives it one more try
+2026/10/26 18:30:52 [DEBUG] [held-apps] demo-hp/gokapi held (unhealthy_stop since 2026-09-24T10:37:48Z)
+2026/10/26 18:30:52 [INFO] Event from demo-hp: app_held (info) — App chaoscrash is held: stopped by the box for a crash loop (2026-09-24T12:16:24Z) — the household's Start gives it one more try
+2026/10/26 18:30:52 [DEBUG] [held-apps] demo-hp/chaoscrash held (unhealthy_stop since 2026-09-24T12:16:24Z)
+2026/10/26 18:30:52 [INFO] Event from demo-hp: app_held (info) — App chaosoomb is held: stopped by the box for a oom storm (2026-09-24T12:39:51Z) — the household's Start gives it one more try
+2026/10/26 18:30:52 [DEBUG] [held-apps] demo-hp/chaosoomb held (unhealthy_stop since 2026-09-24T12:39:51Z)
+2026/10/26 18:33:34 [INFO] Held apps: reminder for demo-hp/gokapi (unhealthy_stop, held 32 days, repeat=false)
+2026/10/26 18:33:35 [INFO] Operator email sent for demo-hp/app_held_reminder
+2026/10/26 18:33:35 [INFO] Held apps: reminder for demo-hp/chaosoom (unhealthy_stop, held 32 days, repeat=false)
+2026/10/26 18:33:35 [INFO] Operator email sent for demo-hp/app_held_reminder
+2026/10/26 18:33:35 [INFO] Held apps: reminder for demo-hp/chaoscrash (unhealthy_stop, held 32 days, repeat=false)
+2026/10/26 18:33:35 [INFO] Operator email sent for demo-hp/app_held_reminder
+2026/10/26 18:33:35 [INFO] Held apps: reminder for demo-hp/chaosoomb (unhealthy_stop, held 32 days, repeat=false)
+2026/10/26 18:33:35 [INFO] Operator email sent for demo-hp/app_held_reminder
+2026/10/26 18:35:41 [INFO] Event from demo-hp: app_deploy_started (info) — Alkalmazás telepítése elindult: Radicale
+2026/10/26 18:35:44 [INFO] DR-recipe app-half stored for customer demo-hp (v1)
+2026/10/26 18:35:44 [INFO] Received report from demo-hp (8070 bytes)
+2026/10/26 18:35:50 [INFO] Event from demo-hp: app_deployed (info) — Alkalmazás telepítve: Radicale
+2026/10/26 18:37:52 [INFO] Event from demo-hp: app_stopped_unhealthy (warning) — A(z) radicale alkalmazást leállítottuk, mert újra és újra összeomlott (vagy elfogyott a memóriája). Az Indítás gombbal újra megpróbálhatod.
+2026/10/26 18:37:53 [INFO] Operator email sent for demo-hp/app_stopped_unhealthy
+2026/10/26 18:38:52 [INFO] Event from demo-hp: app_held (info) — App radicale is held: stopped by the box for a crash loop (2026-10-26T17:37:52Z) — the household's Start gives it one more try
+2026/10/26 18:38:52 [DEBUG] [held-apps] demo-hp/radicale held (unhealthy_stop since 2026-10-26T17:37:52Z)
+2026/10/26 18:41:34 [INFO] Held apps: reminder for demo-hp/radicale (unhealthy_stop, held 4m0s, repeat=false)
+2026/10/26 18:41:35 [INFO] Operator email sent for demo-hp/app_held_reminder
+2026/10/26 18:44:52 [INFO] DR-recipe app-half stored for customer demo-hp (v1)
+2026/10/26 18:44:52 [INFO] Received report from demo-hp (7771 bytes)
+2026/10/26 18:45:52 [INFO] Event from demo-hp: app_hold_cleared (info) — App radicale is no longer held
+2026/10/26 18:45:52 [INFO] [held-apps] demo-hp/radicale no longer held (a row existed: true)
+2026/10/26 18:52:07 [INFO] Event from demo-hp: app_removed (info) — Alkalmazás eltávolítva: radicale
+2026/10/26 18:52:09 [INFO] DR-recipe app-half stored for customer demo-hp (v1)
+2026/10/26 18:52:09 [INFO] Received report from demo-hp (7122 bytes)
+2026/10/26 18:52:49 [INFO] Received signal terminated, shutting down...
+2026/10/26 18:52:49 [INFO] felhom-hub stopped

documentation/audits/d53-held-apps-2026-10-26/redproof-hub.txt

@@ -0,0 +1,7 @@
+RED  internal/notify/dispatcher.go | "app_held_reminder": true, }  // IsOperatorOnly | ['--- FAIL: TestD53_TheHeldAppReminderMailsTheOperatorOnly (0.04s)']
+RED  internal/api/handler.go | h.recordHeldApp(payload.CustomerID, payload.EventType, paylo | ['--- FAIL: TestD53_TheBoxsHeldAppsAreMirrored (0.05s)']
+RED  internal/monitor/held_apps.go | if age < hc.grace(h.Reason) { 		return 	} | ['--- FAIL: TestD53_OneReminderAfterTheGraceThenWeekly (0.04s)', '--- FAIL: TestD53_TheGraceFollowsTheReason (0.17s)']
+RED  internal/monitor/held_apps.go | if !h.RemindedAt.IsZero() && now.Sub(h.RemindedAt) < heldApp | ['--- FAIL: TestD53_OneReminderAfterTheGraceThenWeekly (0.03s)']
+RED  internal/web/hosts.go | "HeldApps":       heldApps, | []
+RED  internal/web/templates/host_detail_body.html | <td>{{.WhoClears}}{{if .Reminded}} | ['--- FAIL: TestD53_HeldAppsListOnTheHostPageAndTheHostsPage (0.05s)']
+RED  internal/web/hosts.go | HeldApps emptied (compiling mutation) | TestD53_HeldAppsListOnTheHostPageAndTheHostsPage

documentation/audits/d63-2026-10-29/9202-after.txt

@@ -0,0 +1,9 @@
+gitea.dooplex.hu/admin/felhom-controller:0.347.0
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.347.0 Up 50 seconds (healthy)
+filebrowser gtstef/filebrowser:1.5.8-stable Up 6 hours (healthy)
+paperless-postgres postgres:18-alpine Up 6 hours (healthy)
+paperless-redis redis:7-alpine Up 6 hours (healthy)
+paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 Up 6 hours (healthy)
+traefik traefik:v3.7.16 Up 6 hours
+4
+359

documentation/audits/d63-2026-10-29/9202-before.txt

@@ -0,0 +1,8 @@
+gitea.dooplex.hu/admin/felhom-controller:0.346.1
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.346.1 Up 5 hours (healthy)
+filebrowser gtstef/filebrowser:1.5.8-stable Up 5 hours (healthy)
+paperless-postgres postgres:18-alpine Up 5 hours (healthy)
+paperless-redis redis:7-alpine Up 5 hours (healthy)
+paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 Up 5 hours (healthy)
+traefik traefik:v3.7.16 Up 5 hours
+359

documentation/audits/d63-2026-10-29/9202-rc.txt

@@ -0,0 +1,8 @@
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1 Up 50 seconds (healthy)
+sha256:2a73dd06cb0edba44e83e2a423738ef03ad1f6110faeeab57da9304509592140
+2026/10/29 08:53:39 undo.go:676: [INFO] [stacks] applied-meta backfill (R-646): recorded 0 []; skipped 0 [] — not current with the catalog, their pinned version's .felhom.yml is no longer on the box
+2026/10/29 08:53:39 scheduler.go:120: [INFO] [scheduler] Registered periodic job: conversion-copy-release (every 1h0m0s)
+2026/10/29 08:53:39 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="conversion-copy-release" interval=1h0m0s totalJobs=14
+2026/10/29 08:53:39 scheduler.go:120: [INFO] [scheduler] Registered periodic job: restore-copy-sweep (every 6h0m0s)
+2026/10/29 08:53:39 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="restore-copy-sweep" interval=6h0m0s totalJobs=15

documentation/audits/d63-2026-10-29/README.md

@@ -0,0 +1,18 @@
+# D63 + D62 live proofs — Tester 1, 2026-10-29 (attended)
+
+Test build `0.347.0-rc.1-d63fault` = controller `e3fce09` built with `-tags faultinject` on top of the local release
+candidate `0.347.0-rc.1` (never pushed; removed from the box after). The release image carries no fault switch
+(`grep -c "FAULT INJECTED"`: 0 on `0.347.0-rc.1`, 2 on the test image). Driver: `d63_live.py` (box_walk helpers).
+
+| File | What it shows |
+|---|---|
+| `run-setup.log`, `run-setup2.log` | first setup tries. A hand-edited compose at "3.8.0" kept 3.8.1's digest, and the backup refused the app (R-645: it ran a version other than its pin) — so no unit. Both reverted. These runs pushed one `backup_run_failures` event (Radicale not protected off-site) — caused by this setup. |
+| `run-setup3.log` | the real setup: Radicale backed up at its catalog 3.8.1 (local unit + off-site snapshot `78002a51`). |
+| `run-pull.log` | the off-site snapshot pulled to the restore scratch (the wizard's two steps). |
+| `run-h4.log`, `run-h4b.log`, `run-h4c.log` | failed seeds: 401 from Radicale. Cause: the harness, not the app — the generated login lives only in the deploying process, a new process sent an empty password. NOT 3.8.0 (an early guess, withdrawn). |
+| `run-h4d.log` | **H4 proven**: live 3.8.0 (true digest `0d60d544…`), snapshot 3.8.1; fault after the compose write; the live definition written back before the start; app on 3.8.0, no hold, T1 + T2 read back. |
+| `run-d62.log` | first D62 try: the script typed the wrong name (a parsing bug in the script) → 400; its "copy gone" line came from a finder that could not see `/mnt/sys_drive` (wrong). |
+| `run-d62b.log` | **D62 proven**: the list line with the date and „automatikusan törlődik", the warning text, wrong name 400, right name 200, the copy gone, the log line; the hub event 400 (hub 0.174.1 does not know the type). |
+| `run-h5.log` | **H5's failed stop proven**: busy volume not written into; the put-back's stop failed; nothing put back; copy kept; app held; the held copy's button off. Its operator mail text is wrong for this case → R-1340. |
+| `run-teardown.log`, `tester1-after.txt` | the app removed; controller back on 0.346.1; test image removed; 15 anonymous volumes (R-1267's PrivateBin class, made by this session's five backup runs) removed by name — 268 volumes, as before. The Radicale **restore hold record stays** (clearing a hold is the operator's). |
+| `controller-full.log` | the controller's whole log 07:50 → teardown. |

documentation/audits/d63-2026-10-29/agent-delivery-journal.txt

@@ -0,0 +1,28 @@
+09:04:08
+== root@192.168.0.154 felhom-agent 0.168.0
+Oct 29 09:56:27 felhom felhom-agent[1161]: time=2026-10-29T09:56:27.781+01:00 level=WARN msg="agent_update: handing the signed job to the root wrapper (felhom-os-apply agent_update)" staged=/var/lib/felhom-agent/selfupda
+Oct 29 09:56:28 felhom felhom-agent[1161]: time=2026-10-29T09:56:28.301+01:00 level=WARN msg="agent_update: signed update verified as root and handed off; restart scheduled" version=0.168.0 report="{\"agent_update\": {\"
+Oct 29 09:56:28 felhom felhom-agent[1161]: time=2026-10-29T09:56:28.302+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=fdb69d53f8afa8d3 op=agent_update
+Oct 29 09:56:31 felhom felhom-agent[424223]: time=2026-10-29T09:56:31.398+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
+Oct 29 09:56:32 felhom felhom-agent[424223]: time=2026-10-29T09:56:32.081+01:00 level=WARN msg="selfupdate: new version running — dwelling before commit" version=0.168.0 prev=0.167.0 dwell=1m0s
+Oct 29 09:57:32 felhom sudo[426210]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/local/sbin/felhom-selfupdate-guarded commit
+Oct 29 09:57:32 felhom felhom-selfupdate-guarded[426218]: committed (pending cleared, .prev retained)
+Oct 29 09:57:32 felhom felhom-agent[424223]: time=2026-10-29T09:57:32.122+01:00 level=WARN msg="selfupdate: update committed" version=0.168.0 wrapper=""
+== hp felhom-agent 0.168.0
+Oct 29 09:55:33 demo-hp felhom-agent[1265]: time=2026-10-29T09:55:33.564+01:00 level=WARN msg="agent_update: handing the signed job to the root wrapper (felhom-os-apply agent_update)" staged=/var/lib/felhom-agent/selfupd
+Oct 29 09:55:33 demo-hp felhom-agent[1265]: time=2026-10-29T09:55:33.953+01:00 level=WARN msg="agent_update: signed update verified as root and handed off; restart scheduled" version=0.168.0 report="{\"agent_update\": {\
+Oct 29 09:55:33 demo-hp felhom-agent[1265]: time=2026-10-29T09:55:33.954+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=63ed679a243bcac2 op=agent_update
+Oct 29 09:55:37 demo-hp felhom-agent[961306]: time=2026-10-29T09:55:37.136+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
+Oct 29 09:55:38 demo-hp felhom-agent[961306]: time=2026-10-29T09:55:38.063+01:00 level=WARN msg="selfupdate: new version running — dwelling before commit" version=0.168.0 prev=0.167.0 dwell=1m0s
+Oct 29 09:56:38 demo-hp sudo[966235]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/local/sbin/felhom-selfupdate-guarded commit
+Oct 29 09:56:38 demo-hp felhom-selfupdate-guarded[966243]: committed (pending cleared, .prev retained)
+Oct 29 09:56:38 demo-hp felhom-agent[961306]: time=2026-10-29T09:56:38.091+01:00 level=WARN msg="selfupdate: update committed" version=0.168.0 wrapper=""
+== felhom-pve felhom-agent 0.168.0
+Oct 29 10:03:40 demo-felhom felhom-agent[1194]: time=2026-10-29T10:03:40.145+01:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=95d5b5018bde3b37 op=agent_update key_id=felhom-op-1 nonce=7655701562a
+Oct 29 10:03:40 demo-felhom felhom-agent[1194]: time=2026-10-29T10:03:40.429+01:00 level=WARN msg="agent_update: handing the signed job to the root wrapper (felhom-os-apply agent_update)" staged=/var/lib/felhom-agent/sel
+Oct 29 10:03:40 demo-felhom felhom-agent[1194]: time=2026-10-29T10:03:40.698+01:00 level=WARN msg="agent_update: signed update verified as root and handed off; restart scheduled" version=0.168.0 report="{\"agent_update\"
+Oct 29 10:03:40 demo-felhom felhom-agent[1194]: time=2026-10-29T10:03:40.698+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=95d5b5018bde3b37 op=agent_update
+Oct 29 10:03:43 demo-felhom felhom-agent[357627]: time=2026-10-29T10:03:43.578+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
+Oct 29 10:03:44 demo-felhom felhom-agent[357627]: time=2026-10-29T10:03:44.857+01:00 level=WARN msg="selfupdate: new version running — dwelling before commit" version=0.168.0 prev=0.167.0 dwell=1m0s
+
+[exited with code 0]

documentation/audits/d63-2026-10-29/agent-delivery.txt

@@ -0,0 +1,28 @@
+== root@192.168.0.154 felhom-agent 0.168.0
+Oct 29 09:56:27 felhom felhom-agent[1161]: time=2026-10-29T09:56:27.781+01:00 level=WARN msg="agent_update: handing the signed job to the root wrapper (felhom-os-apply agent_update)" staged=/var/lib/felhom-agent/selfupda
+Oct 29 09:56:28 felhom felhom-agent[1161]: time=2026-10-29T09:56:28.301+01:00 level=WARN msg="agent_update: signed update verified as root and handed off; restart scheduled" version=0.168.0 report="{\"agent_update\": {\"
+Oct 29 09:56:28 felhom felhom-agent[1161]: time=2026-10-29T09:56:28.302+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=fdb69d53f8afa8d3 op=agent_update
+Oct 29 09:56:31 felhom felhom-agent[424223]: time=2026-10-29T09:56:31.398+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
+Oct 29 09:56:32 felhom felhom-agent[424223]: time=2026-10-29T09:56:32.081+01:00 level=WARN msg="selfupdate: new version running — dwelling before commit" version=0.168.0 prev=0.167.0 dwell=1m0s
+Oct 29 09:57:32 felhom sudo[426210]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/local/sbin/felhom-selfupdate-guarded commit
+Oct 29 09:57:32 felhom felhom-selfupdate-guarded[426218]: committed (pending cleared, .prev retained)
+Oct 29 09:57:32 felhom felhom-agent[424223]: time=2026-10-29T09:57:32.122+01:00 level=WARN msg="selfupdate: update committed" version=0.168.0 wrapper=""
+== hp felhom-agent 0.168.0
+Oct 29 09:55:33 demo-hp felhom-agent[1265]: time=2026-10-29T09:55:33.564+01:00 level=WARN msg="agent_update: handing the signed job to the root wrapper (felhom-os-apply agent_update)" staged=/var/lib/felhom-agent/selfupd
+Oct 29 09:55:33 demo-hp felhom-agent[1265]: time=2026-10-29T09:55:33.953+01:00 level=WARN msg="agent_update: signed update verified as root and handed off; restart scheduled" version=0.168.0 report="{\"agent_update\": {\
+Oct 29 09:55:33 demo-hp felhom-agent[1265]: time=2026-10-29T09:55:33.954+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=63ed679a243bcac2 op=agent_update
+Oct 29 09:55:37 demo-hp felhom-agent[961306]: time=2026-10-29T09:55:37.136+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
+Oct 29 09:55:38 demo-hp felhom-agent[961306]: time=2026-10-29T09:55:38.063+01:00 level=WARN msg="selfupdate: new version running — dwelling before commit" version=0.168.0 prev=0.167.0 dwell=1m0s
+Oct 29 09:56:38 demo-hp sudo[966235]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/local/sbin/felhom-selfupdate-guarded commit
+Oct 29 09:56:38 demo-hp felhom-selfupdate-guarded[966243]: committed (pending cleared, .prev retained)
+Oct 29 09:56:38 demo-hp felhom-agent[961306]: time=2026-10-29T09:56:38.091+01:00 level=WARN msg="selfupdate: update committed" version=0.168.0 wrapper=""
+== felhom-pve felhom-agent 0.168.0
+Oct 29 10:03:40 demo-felhom felhom-agent[1194]: time=2026-10-29T10:03:40.145+01:00 level=WARN msg="signedjobs: AUTHORIZED signed op — executing" job=95d5b5018bde3b37 op=agent_update key_id=felhom-op-1 nonce=7655701562a
+Oct 29 10:03:40 demo-felhom felhom-agent[1194]: time=2026-10-29T10:03:40.429+01:00 level=WARN msg="agent_update: handing the signed job to the root wrapper (felhom-os-apply agent_update)" staged=/var/lib/felhom-agent/sel
+Oct 29 10:03:40 demo-felhom felhom-agent[1194]: time=2026-10-29T10:03:40.698+01:00 level=WARN msg="agent_update: signed update verified as root and handed off; restart scheduled" version=0.168.0 report="{\"agent_update\"
+Oct 29 10:03:40 demo-felhom felhom-agent[1194]: time=2026-10-29T10:03:40.698+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=95d5b5018bde3b37 op=agent_update
+Oct 29 10:03:43 demo-felhom felhom-agent[357627]: time=2026-10-29T10:03:43.578+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
+Oct 29 10:03:44 demo-felhom felhom-agent[357627]: time=2026-10-29T10:03:44.857+01:00 level=WARN msg="selfupdate: new version running — dwelling before commit" version=0.168.0 prev=0.167.0 dwell=1m0s
+Oct 29 10:04:44 demo-felhom felhom-agent[357627]: time=2026-10-29T10:04:44.877+01:00 level=WARN msg="selfupdate: update committed" version=0.168.0 wrapper=""
+felhom-agent 0.168.0
+1e70196c51346cf10e3f04f51975de5899b5dc6baa5a45ec1d17bc248830d2ca  /usr/local/bin/felhom-agent

documentation/audits/d63-2026-10-29/catalog-sync-tester1.txt

@@ -0,0 +1,11 @@
+sync -> 200 {'ok': True, 'data': {'ok': True, 'updated': ['plex', 'wanderer'], 'message': 'Sablonok frissítve — frissítve: plex, wanderer', 'catalog_moved': True}
+2026/10/29 09:32:20 [INFO] [sync] Updated plex/docker-compose.yml
+2026/10/29 09:32:20 [INFO] [sync] Updated plex/.felhom.yml
+2026/10/29 09:32:20 [INFO] [sync] Updated wanderer/.felhom.yml
+2026/10/29 09:32:20 [INFO] [sync] catalog moved 0baf8f948575 -> 313eaf68d01f (0 new, 2 stack dir(s) changed)
+2026/10/29 09:32:20 [INFO] [sync] Catalog sync complete
+2026/10/29 09:32:20 [INFO] [api] Catalog sync completed: Sablonok frissítve — frissítve: plex, wanderer
+
+2
+stacks
+

documentation/audits/d63-2026-10-29/ceremony.log

@@ -0,0 +1,17 @@
+09:40:09 preflight → 200 {"data": {"agent_supported": true, "escrow_state": "escrowed", "items": [{"id": "pbs_storage_id", "ok": true, "detail": "felhom-pbs"}, {"id": "dr_tier", "ok": true, "detail": "DR tier applied"}, {"id": "age_binary", "ok": true, "detail": "/usr/bin/age"}, {"id": "hub_upload", "ok": true, "detail": "hub upload target configured"}, {"id": "staged_secret", "ok": false, "detail": "no staged secret (informational \u2014 the controller re-stages when offsite is configured)"}, {"id": "sudo_grant", "ok": true, "detail": "sudo grant listed (list-mode)"}], "offbox_configured": true, "ok": true, "stale": 
+09:40:09 start → 200 {"data": {"job_id": "escrow-1793263209404621564", "phase": "running"}, "error": "", "ok": true}
+09:40:12 status → 200 {"data": {"claim_expires_in_sec": 599, "claimable": true, "claimed": false, "detail": "", "entropy_bits": 129.24070185585344, "job_id": "escrow-1793263209404621564", "key_fingerprint": "82:c6:61:ba:ae:36:47:af:0f:dc:17:0b:0d:5c:5d:63:cd:e7:47:e6:fe:9a:14:fe:66:c1:f0:e6:8f:a1:44:3b", "phase": "done", "restic_pw_sealed": true, "uploaded": false}, "error": "", "ok": true}
+09:40:12 claim → 200; code received: True (10 words); confirm_required=True positions=[3, 4]
+09:40:12 confirm → 200 {"data": {"uploaded": true}, "error": "", "ok": true}
+09:52:46 controller escrow lines:
+2026/10/29 08:40:09 [INFO] [web] escrow readiness: READY (agent preflight ok) — the recovery-code reminder is shown
+2026/10/29 08:40:09 [INFO] [web] escrow ceremony started via wizard (job escrow-1793263209404621564)
+2026/10/29 08:40:12 [INFO] [web] escrow recovery code claimed (one-shot; not logged) — its package waits for the typed-back words
+2026/10/29 08:40:12 [INFO] [web] escrow recovery code typed back — the new package is on the hub (R-1151)
+
+stale warning (nem fedi): False | control word Helyre present: False
+escrow_state: escrowed
+2026/10/29 08:40:09 [INFO] [web] escrow ceremony started via wizard (job escrow-1793263209404621564)
+2026/10/29 08:40:12 [INFO] [web] escrow recovery code claimed (one-shot; not logged) — its package waits for the typed-back words
+2026/10/29 08:40:12 [INFO] [web] escrow recovery code typed back — the new package is on the hub (R-1151)
+

documentation/audits/d63-2026-10-29/ceremony.py

@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+"""ceremony.py — one escrow ceremony on Tester 1 through the wizard's own endpoints (hub 0.175.0 check, 2026-10-29).
+The recovery code lives in this process's memory only: never printed, never written, dropped at the end. The dashboard
+password is read from the 0600 file and handed to curl on stdin (never on a command line)."""
+import json, os, subprocess, sys, time
+
+sys.path.insert(0, "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts")
+SCR = "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad"
+os.environ.setdefault("SC", f"{SCR}/rt-tester1")
+os.environ.setdefault("TARGET", "tester-1")
+import box_walk as w
+
+w.login()
+sess = open(f"{w.SC}/sess{os.getpid()}.txt").read().strip()
+csrf = open(f"{w.SC}/csrf{os.getpid()}.txt").read().strip()
+BASE_ARGS = ["curl", "-sk", "-H", w.HOSTHDR, "-H", f"Cookie: {sess}", "-H", f"X-CSRF-Token: {csrf}"]
+
+
+def call(method, path, form_stdin=None, body=None):
+    args = BASE_ARGS + ["-X", method, "-w", "\n%{http_code}"]
+    inp = None
+    if form_stdin is not None:
+        args += ["--data-urlencode", "password@-"]
+        inp = form_stdin
+    if body is not None:
+        args += ["-H", "Content-Type: application/json", "--data-binary", "@-"]
+        inp = json.dumps(body)
+    r = subprocess.run(args + [f"{w.BASE}{path}"], input=inp, capture_output=True, text=True, timeout=120)
+    out, _, code = r.stdout.rpartition("\n")
+    try:
+        return code, json.loads(out)
+    except Exception:
+        return code, {"raw": out[:200]}
+
+
+def safe(d):
+    """A copy with the code removed — the only form that is printed."""
+    d = json.loads(json.dumps(d))
+    if isinstance(d.get("data"), dict):
+        d["data"].pop("recovery_code", None)
+    return d
+
+
+t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+code, d = call("GET", "/api/escrow/preflight")
+w.say(f"preflight → {code} {json.dumps(d)[:600]}")
+pw = open(f"{w.SC}/.ctlpw").read().strip()
+code, d = call("POST", "/api/escrow/start", form_stdin=pw)
+pw = None
+w.say(f"start → {code} {json.dumps(safe(d))[:300]}")
+if code != "200":
+    sys.exit(1)
+for _ in range(120):
+    time.sleep(3)
+    code, d = call("GET", "/api/escrow/status")
+    ph = (d.get("data") or {}).get("phase")
+    if ph not in ("running", "queued", "starting", None) or (d.get("data") or {}).get("claimable"):
+        break
+w.say(f"status → {code} {json.dumps(safe(d))[:400]}")
+code, d = call("POST", "/api/escrow/claim")
+rc = (d.get("data") or {}).get("recovery_code") or ""
+pos = (d.get("data") or {}).get("verify_positions") or []
+w.say(f"claim → {code}; code received: {bool(rc)} ({len(rc.split('-'))} words); confirm_required={(d.get('data') or {}).get('confirm_required')} positions={pos}")
+if rc and pos:
+    words = rc.split("-")
+    picked = [words[pos[0] - 1], words[pos[1] - 1]]
+    rc, words = None, None
+    code, d = call("POST", "/api/escrow/confirm", body={"words": picked})
+    picked = None
+    w.say(f"confirm → {code} {json.dumps(safe(d))[:400]}")
+rc = None
+for _ in range(40):
+    time.sleep(15)
+    st = w.guest("docker logs --since " + t0 + " felhom-controller 2>&1 | grep -iE 'escrow' | grep -viE 'recovery_code' | tail -12")
+    if "escrowed" in st or "auto-confirm" in st.lower():
+        break
+w.say("controller escrow lines:\n" + st)
+open(f"{SCR}/d63/ev/ceremony.log", "a").write("\n".join(w.LOG) + "\n")

documentation/audits/d63-2026-10-29/ctl-switch.txt

@@ -0,0 +1 @@
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault Up 50 seconds (healthy)

documentation/audits/d63-2026-10-29/d63_live.py

@@ -0,0 +1,260 @@
+#!/usr/bin/env python3
+"""d63_live.py <phase> — D63 live proofs on Tester 1 (2026-10-29, attended), test build with `-tags faultinject`.
+
+Phases:
+  setup     deploy a throwaway Radicale, seed T1 through its own front door, run it at 3.8.0 (the stack folder's
+            compose edited, the household's Restart), switch it into the off-site set and press „Mentés most" for
+            off-site, then put the compose back at 3.8.1 and Restart — so the newest off-site snapshot holds 3.8.0's
+            data under 3.8.1's live definition.
+  pull      pull the newest off-site snapshot of radicale to the restore scratch (the wizard's two steps).
+  h4        seed T2; arm defwrite-after-compose "0 1"; press the full restore. Want: the fault, then the live definition
+            written back BEFORE the start; the app runs 3.8.1; no hold; T1 and T2 read back.
+  d62       a full restore with no fault; the copy is listed on the restore page with its date; a wrong name is refused;
+            the right name deletes it; it is gone.
+  h5        a created-only container holds radicale's data volume (the restore cannot replace it → the put-back);
+            arm stop-radicale "1 1" (the restore's own stop passes, the put-back's stop fails). Want: nothing is put
+            back, the copy stays, the app is held with the D20 sentence; the page shows the held copy's button OFF.
+  teardown  the blocker and the fault files go; the app is removed through the product.
+State (passwords) stays in the scratchpad, never printed."""
+import json, os, re, subprocess, sys, time
+
+sys.path.insert(0, "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts")
+SCR = "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad"
+os.environ.setdefault("SC", f"{SCR}/rt-tester1")
+os.environ.setdefault("TARGET", "tester-1")
+import box_walk as w
+import upgrade_fixtures_box as fb
+
+EV = f"{SCR}/d63/ev"
+os.makedirs(EV, exist_ok=True)
+STATE = f"{SCR}/d63/.state.json"
+APP, SUB = "radicale", "radicale"
+fx = fb.FIXTURES[APP]
+FAULTS = "/var/lib/docker/volumes/felhom-controller-data/_data/faults"
+COMPOSE = f"/opt/docker/stacks/{APP}/docker-compose.yml"
+OLD, NEW = "ghcr.io/kozea/radicale:3.8.0", "ghcr.io/kozea/radicale:3.8.1"
+OLD_FULL = "ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468"
+NEW_FULL = "ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c"
+
+
+def state():
+    return json.load(open(STATE)) if os.path.exists(STATE) else {}
+
+
+def save(st):
+    json.dump(st, open(STATE, "w"))
+    os.chmod(STATE, 0o600)
+
+
+def form(path, fields, accept_json=False):
+    """A dashboard form post with the session and its CSRF token — what the page's button sends."""
+    sess = open(f"{w.SC}/sess{os.getpid()}.txt").read().strip()
+    csrf = open(f"{w.SC}/csrf{os.getpid()}.txt").read().strip()
+    args = ["curl", "-sk", "-D", "-", "-H", w.HOSTHDR, "-H", f"Cookie: {sess}", "-H", f"X-CSRF-Token: {csrf}", "-X", "POST",
+            "--data-urlencode", f"_csrf={csrf}"]
+    if accept_json:
+        args += ["-H", "Accept: application/json"]
+    for k, v in fields.items():
+        args += ["--data-urlencode", f"{k}={v}"]
+    bodyf = f"{SCR}/d63/.body"
+    r = w.sh(args + ["-o", bodyf, f"{w.BASE}{path}"], timeout=180)
+    out = r.stdout or ""
+    head = out.split("\n")[0].strip()
+    loc = [l.strip() for l in out.split("\n") if l.lower().startswith("location:")]
+    body = open(bodyf).read() if os.path.exists(bodyf) else ""
+    return head, loc[:1], body
+
+
+def wait_op(label, cap=1800):
+    t0, last = time.time(), None
+    while time.time() - t0 < cap:
+        code, d = w.ctl("GET", "/api/backup/restore-status")
+        dd = d.get("data") or {}
+        cur = (dd.get("running"), dd.get("op"), dd.get("ok"), (dd.get("message") or "")[:300])
+        if cur != last:
+            w.say(f"      +{round(time.time()-t0):>5}s {label}: {cur}")
+            last = cur
+        if not dd.get("running") and time.time() - t0 > 5:
+            return dd
+        time.sleep(3)
+    return {}
+
+
+def ctl_log(since, pat):
+    return w.guest(f"docker logs --since {since} felhom-controller 2>&1 | grep -E '{pat}' | tail -60")
+
+
+def now():
+    return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+
+
+def look():
+    st = w.stack(APP)
+    w.say(f"  app: state={st.get('state')} hold={st.get('hold_reason')!r} deployed={st.get('deployed')}")
+    w.say("  compose image: " + w.guest(f"grep -E '^ +image:' {COMPOSE}").strip())
+    w.say("  container: " + w.guest(f"docker ps -a --filter name=^/{APP}$ --format '{{{{.Image}}}} {{{{.Status}}}}'").strip())
+    w.say("  copies: " + (w.guest("find /mnt/*/felhom-data/backups/restore-undo /opt/docker/felhom-data/backups/restore-undo -mindepth 2 -maxdepth 2 -type d 2>/dev/null | head").strip() or "(none)"))
+    return st
+
+
+def set_image(img, label, full=False):
+    if full:
+        w.guest(f"sed -i -E 's#^( +image: )ghcr.io/kozea/radicale:[^ ]+$#\\1{img}#' {COMPOSE}")
+        img = img.split("@")[0]
+    else:
+        w.guest(f"sed -i -E 's#^( +image: )ghcr.io/kozea/radicale:[0-9.]+#\\1{img}#' {COMPOSE}")
+    c, d = w.ctl("POST", f"/api/stacks/{APP}/restart")
+    w.say(f"  [{label}] compose → {img}; Restart → {c} {str(d)[:120]}")
+    for _ in range(40):
+        time.sleep(3)
+        if img in w.guest(f"docker ps --filter name=^/{APP}$ --format '{{{{.Image}}}}'"):
+            break
+    look()
+
+
+w.login()
+phase = sys.argv[1]
+st = state()
+if st.get("t1"):  # the deploy's generated login lives only in the deploying process; the seed reads w.GENERATED
+    w.GENERATED[APP] = {"RADICALE_USER": st["t1"]["user"], "RADICALE_PASSWORD": st["t1"]["pw"]}
+t_start = now()
+w.say(f"=== phase {phase} at {t_start}")
+w.say("controller image: " + w.guest("cat /etc/felhom-controller-image; docker ps --filter name=felhom-controller --format '{{.Image}} {{.Status}}'").strip())
+
+if phase == "setup":
+    assert w.deploy(APP, SUB), "deploy failed"
+    t1 = fx.seed(w, SUB, w.say)
+    assert t1, "seed T1 failed"
+    st["t1"] = t1
+    save(st)
+    set_image(OLD, "setup")
+    w.say(f"  T1 readback at 3.8.0: {fx.verify(w, SUB, t1, w.say)}")
+    w.say("  off-site toggle: " + str(form("/backup/offbox/toggle", {"app": APP, "enabled": "on"})[:2]))
+    w.say("  off-site run: " + str(form("/backup/offbox/run", {})[:2]))
+    t0 = time.time()
+    while time.time() - t0 < 3600:
+        time.sleep(15)
+        if "manual off-box backup" in ctl_log(t_start, "off-box backup|offbox.*(completed|finished|failed)") or \
+                re.search(r"(?i)off-?box (run|backup) (completed|finished|done)", ctl_log(t_start, "offbox|off-box")):
+            break
+    w.say("  off-site run log: " + ctl_log(t_start, "offbox|off-box|radicale").strip()[-3000:])
+    set_image(NEW, "setup")
+    w.say(f"  T1 readback at 3.8.1: {fx.verify(w, SUB, t1, w.say)}")
+
+elif phase in ("setup2", "setup3"):
+    # The off-site run pushes recovery units; a fresh app has none until a backup run made one ("skipped (no unit)").
+    # The household's „Mentés most" (whole box, the only backup press there is — R-648) makes radicale's unit at 3.8.0.
+    look()
+    c, d = w.ctl("POST", "/api/backup/run")
+    w.say(f"  local backup press → {c} {str(d)[:200]}")
+    t0 = time.time()
+    while time.time() - t0 < 2400:
+        time.sleep(10)
+        lg = ctl_log(t_start, r"App-data backup completed|\[backup\].*(failed|FAILED)|backup run (finished|complete)")
+        if "App-data backup completed" in lg:
+            break
+    w.say("  local backup log: " + ctl_log(t_start, r"App-data backup completed|radicale").strip()[-2500:])
+    w.say("  off-site run: " + str(form("/backup/offbox/run", {})[:2]))
+    t1 = now()
+    t0 = time.time()
+    while time.time() - t0 < 2400:
+        time.sleep(10)
+        if re.search(r"\[offbox\] backup (OK|failed)", ctl_log(t1, r"\[offbox\] backup")):
+            break
+    w.say("  off-site run log: " + ctl_log(t1, r"\[offbox\] backup|radicale").strip()[-2500:])
+    if phase == "setup2":
+        set_image(NEW, "setup2")
+    w.say(f"  T1 readback: {fx.verify(w, SUB, st['t1'], w.say)}")
+    w.say("  radicale's off-site snapshots: " + w.guest("ls -la /mnt/sys_drive/felhom-data/backups/primary/radicale 2>&1 | head -8"))
+
+elif phase == "pull":
+    w.say("  prepare: " + str(form("/backup/offbox/restore", {"app": APP, "mode": "full"})[:2]))
+    w.say("  pull: " + str(form("/backup/offbox/restore", {"app": APP, "mode": "full", "confirm": "1"})[:2]))
+    wait_op("pull")
+
+elif phase == "h4":
+    # the live definition moves to the TRUE 3.8.0 (its own digest) right before the restore — within one catalog-sync
+    # tick — so the snapshot (3.8.1) and the live definition differ and the restore writes the snapshot's definition
+    set_image(NEW_FULL, "h4", full=True)  # T2 is seeded on 3.8.1 (09:05-09:08: the 401s were the harness's empty password in a new process, not 3.8.0)
+    t2 = None
+    for _ in range(6):
+        time.sleep(10)
+        t2 = fx.seed(w, SUB, w.say)
+        if t2:
+            break
+    assert t2, "seed T2 failed"
+    st["t2"] = t2
+    save(st)
+    set_image(OLD_FULL, "h4", full=True)
+    time.sleep(15)
+    look()
+    w.guest(f"mkdir -p {FAULTS} && echo '0 1' > {FAULTS}/defwrite-after-compose && cat {FAULTS}/defwrite-after-compose")
+    w.say("  armed: defwrite-after-compose = 0 1")
+    w.say("  restore: " + str(form("/backup/offbox/reconstitute", {"app": APP, "confirm": "1"})[:2]))
+    rec = wait_op("h4 restore")
+    w.say(f"  restore record: {json.dumps(rec)[:1200]}")
+    w.say("  controller log:\n" + ctl_log(t_start, "faultinject|radicale|R-1256|definition"))
+    look()
+    w.say(f"  T1 readback: {fx.verify(w, SUB, st['t1'], w.say)}")
+    w.say(f"  T2 readback: {fx.verify(w, SUB, t2, w.say)}")
+    w.say("  fault file left: " + (w.guest(f"ls {FAULTS} 2>/dev/null").strip() or "(none)"))
+
+elif phase == "d62":
+    look()
+    w.say("  restore (no fault): " + str(form("/backup/offbox/reconstitute", {"app": APP, "confirm": "1"})[:2]))
+    rec = wait_op("d62 restore")
+    # at once: the stable-start check releases the copy about a minute after a good restore (R-1270)
+    page = w.page("/backups/restore")
+    t_page = now()
+    rows = re.findall(r'<details class="kept-delete" data-restore-copy="([^"]+)">(.*?)</details>', page, re.S)
+    w.say(f"  page at {t_page}: {len(rows)} deletable restore copy row(s); restore ok={(rec.get('last') or {}).get('ok')}")
+    m = re.search(r'data-restore-copies="true">(.*?)</div>\s*<script>', page, re.S)
+    if m:
+        w.say("  list text: " + re.sub(r"\s+", " ", re.sub(r"<[^>]+>", " ", m.group(1))).strip()[:800])
+    if rows:
+        app_stamp = rows[0][0]
+        app, stamp = app_stamp.split("/", 1)
+        names = re.findall(r'<span class="app-row-name">([^<]+)</span>', page[:page.find('data-restore-copy="' + app_stamp)])
+        typed = names[-1].strip() if names else app
+        h, l, b = form("/backup/restore-copy/delete", {"app": app, "stamp": stamp, "confirm": typed + "x"}, accept_json=True)
+        w.say(f"  wrong name ({typed + 'x'!r}) → {h} {b.strip()[:300]}")
+        h, l, b = form("/backup/restore-copy/delete", {"app": app, "stamp": stamp, "confirm": typed}, accept_json=True)
+        w.say(f"  right name ({typed!r}) → {h} {b.strip()[:300]}")
+        look()
+        page2 = w.page("/backups/restore")
+        w.say(f"  page after: rows={page2.count('data-restore-copy=')}")
+        w.say("  controller log: " + ctl_log(t_start, "restore copy DELETED|restore-copy delete|restore_copy_deleted|Event push|released"))
+    else:
+        w.say("  NO deletable copy listed — the D62 page proof did not happen")
+    rec2 = wait_op("settle", cap=5)
+    w.say(f"  T1 readback: {fx.verify(w, SUB, st['t1'], w.say)}")
+
+elif phase == "h5":
+    look()
+    vols = w.guest(f"docker volume ls --format '{{{{.Name}}}}' | grep -E '^{APP}' | sort").split()
+    w.say(f"  volumes: {vols}")
+    datav = vols[0]
+    w.say("  blocker: " + w.guest(f"docker rm -f d63-blocker >/dev/null 2>&1; docker create --name d63-blocker -v {datav}:/x alpine true"))
+    w.say("  premise (rm -f with the blocker): " + w.guest(f"docker volume rm -f {datav} 2>&1; echo rc=$?").strip())
+    w.guest(f"mkdir -p {FAULTS} && echo '1 1' > {FAULTS}/stop-{APP}")
+    w.say(f"  armed: stop-{APP} = 1 1")
+    w.say("  restore: " + str(form("/backup/offbox/reconstitute", {"app": APP, "confirm": "1"})[:2]))
+    rec = wait_op("h5 restore")
+    w.say(f"  restore record: {json.dumps(rec)[:1500]}")
+    w.say("  controller log:\n" + ctl_log(t_start, "faultinject|radicale|R-1256|put-back|put back|Failed to remove|hold"))
+    st2 = look()
+    app_page = w.page(f"/apps/{APP}")
+    held = re.search(r'data-held="true">(.*?)</div>', app_page, re.S)
+    w.say("  app page hold sentence: " + (re.sub(r"<[^>]+>", " ", held.group(1)).strip()[:600] if held else "(none)"))
+    page = w.page("/backups/restore")
+    w.say(f"  restore page: button off={'data-restore-copy-off' in page}, needed line="
+          f"{re.findall(r'data-restore-copy-needed=.true.>([^<]+)<', page)[:2]}, deletable rows={page.count('data-restore-copy=')}")
+    w.say("  fault file left: " + (w.guest(f"ls {FAULTS} 2>/dev/null").strip() or "(none)"))
+
+elif phase == "teardown":
+    w.guest(f"docker rm -f d63-blocker >/dev/null 2>&1; rm -rf {FAULTS}")
+    look()
+    w.remove(APP)
+    look()
+
+open(os.path.join(EV, f"{phase}.log"), "a").write("\n".join(w.LOG) + "\n")

documentation/audits/d63-2026-10-29/floor.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# floor.sh <token NAME> <path> <min_controller_version or ""> <min_agent or ""> — one floor save at the hub with CC's
+# scoped token (decision 251). The token arrives through `sec run` under the NAME; it goes to curl through a header file
+# in $XDG_RUNTIME_DIR (printf is a builtin, so the value is never on a command line) and is never printed.
+# Prints the status line and the Location header only: 303 to ?flash=floor_set is success; any other flash is a refusal.
+set -euo pipefail
+NAME=$1; P=$2; VER=$3; MA=$4
+H=$(mktemp -p "${XDG_RUNTIME_DIR:-/tmp}")
+printf "Authorization: Bearer %s\n" "${!NAME}" > "$H"
+curl -s -o /dev/null -D - -H @"$H" --data-urlencode "min_controller_version=$VER" --data-urlencode "min_agent=$MA" \
+  "http://10.43.52.34:8080$P" | grep -iE "^HTTP|^location" || true
+: > "$H"; rm -f "$H"

documentation/audits/d63-2026-10-29/global-floor-readback.txt

@@ -0,0 +1,22 @@
+ Managed updates — global floor 
+ Saving takes effect immediately — boxes below the floor update on their next
+ report, no customer action. Blank = no global floor. This setting is independent of the Day-0
+ Effective floor:
+ v0.347.0 
+ Save global floor… 
+ DB override: v0.347.0 
+ function confirmGlobalFloor() {
+ var v = document.getElementById('global-floor-input').value.trim();
+ var box = document.getElementById('global-floor-confirm');
+ var impact = document.getElementById('global-floor-impact');
+ document.getElementById('global-floor-confirm-input').value = '';
+ document.getElementById('global-floor-confirm-err').textContent = '';
+ impact.textContent = 'This will CLEAR the DB floor override (the box falls back to the env default). Type CLEAR to confirm.';
+ fetch('/configuration/global-floor/impact?v=' + encodeURIComponent(v))
+ function submitGlobalFloor() {
+ var v = document.getElementById('global-floor-input').value.trim();
+ var typed = document.getElementById('global-floor-confirm-input').value.trim();
+ var err = document.getElementById('global-floor-confirm-err');
+ document.getElementById('global-floor-form').submit();
+2026/10/29 10:31:31 [INFO] Global controller-version floor set to "0.347.0" (declared MinAgent "0.158.1")
+2026/10/29 10:31:31 [INFO] access token "cc-releases-2": POST /configuration/global-floor → 303

documentation/audits/d63-2026-10-29/global-floor.txt

@@ -0,0 +1,2 @@
+HTTP/1.1 303 See Other
+Location: /configuration?flash=floor_set

documentation/audits/d63-2026-10-29/hub-events-release.log

@@ -0,0 +1,39 @@
+2026/10/29 09:38:27 [INFO] Default controller-version floor: 0.120.0
+2026/10/29 09:40:15 [INFO] managed floor SERVED for tester-1: floor 0.346.1, agent requirement "0.158.1" from declared (golden 0.344.0)
+2026/10/29 09:40:52 [INFO] managed floor SERVED for demo-hp: floor 0.346.1, agent requirement "0.158.1" from declared (golden 0.344.0)
+2026/10/29 09:48:58 [INFO] managed floor SERVED for demo-felhom: floor 0.346.1, agent requirement "0.158.1" from declared (golden 0.344.0)
+2026/10/29 09:57:50 [INFO] Customer tester-1 controller-version floor override set to "0.347.0" (declared MinAgent "0.158.1")
+2026/10/29 09:57:50 [INFO] access token "cc-releases-2": POST /customers/tester-1/floor → 303
+2026/10/29 09:57:50 [INFO] Customer demo-hp controller-version floor override set to "0.347.0" (declared MinAgent "0.158.1")
+2026/10/29 09:57:50 [INFO] access token "cc-releases-2": POST /customers/demo-hp/floor → 303
+2026/10/29 09:57:50 [INFO] Customer demo-felhom controller-version floor override set to "0.347.0" (declared MinAgent "0.158.1")
+2026/10/29 09:57:50 [INFO] access token "cc-releases-2": POST /customers/demo-felhom/floor → 303
+2026/10/29 09:57:53 [INFO] managed floor SERVED for tester-1: floor 0.347.0, agent requirement "0.158.1" from declared (golden 0.344.0)
+2026/10/29 09:57:54 [INFO] managed floor SERVED for demo-felhom: floor 0.347.0, agent requirement "0.158.1" from declared (golden 0.344.0)
+2026/10/29 09:57:54 [INFO] managed floor SERVED for demo-hp: floor 0.347.0, agent requirement "0.158.1" from declared (golden 0.344.0)
+2026/10/29 09:58:00 [INFO] Event from demo-felhom: controller_updated (info) — Controller frissítve: 0.346.1 → 0.347.0
+2026/10/29 09:58:01 [INFO] Event from tester-1: controller_updated (info) — Controller frissítve: 0.346.1 → 0.347.0
+2026/10/29 09:58:03 [INFO] Event from demo-hp: controller_updated (info) — Controller frissítve: 0.346.1 → 0.347.0
+2026/10/29 09:58:05 [INFO] Event from demo-felhom: controller_started (info) — Controller elindult (0.347.0)
+2026/10/29 09:58:06 [INFO] Event from tester-1: controller_started (info) — Controller elindult (0.347.0)
+2026/10/29 09:58:08 [INFO] Event from demo-hp: controller_started (info) — Controller elindult (0.347.0)
+2026/10/29 09:59:01 [INFO] Event from tester-1: app_held (info) — App radicale is held: held after an off-site restore failed its database replay (2026-10-29T08:18:38Z) — clear the hold to start it
+2026/10/29 09:59:16 [INFO] Event from tester-1: app_deploy_started (info) — Alkalmazás telepítése elindult: Gokapi
+2026/10/29 09:59:20 [INFO] Event from tester-1: app_deployed (info) — Alkalmazás telepítve: Gokapi
+2026/10/29 10:02:32 [INFO] Event from tester-1: app_oom (warning) — Alkalmazás memóriája elfogyott: gokapi (gokapi) — egy folyamatát a memóriakorlát leállította
+2026/10/29 10:02:32 [INFO] Operator email sent for tester-1/app_oom
+2026/10/29 10:06:03 [INFO] Event from tester-1: app_removed (info) — Alkalmazás eltávolítva: gokapi
+2026/10/29 10:07:18 [INFO] Event from tester-1: app_deploy_started (info) — Alkalmazás telepítése elindult: Gokapi
+2026/10/29 10:07:23 [INFO] Event from tester-1: app_deployed (info) — Alkalmazás telepítve: Gokapi
+2026/10/29 10:10:31 [INFO] Event from tester-1: restore_copy_deleted (info) — gokapi: the restore copy from before the restore of 2026-10-29T09:08:36Z (61264 bytes) was deleted by hand — the household (dashboard sign-in) from 192.168.0.180. That restore can no longer be undone from this box.
+2026/10/29 10:11:37 [INFO] Event from tester-1: app_removed (info) — Alkalmazás eltávolítva: gokapi
+2026/10/29 10:31:31 [INFO] Global controller-version floor set to "0.347.0" (declared MinAgent "0.158.1")
+2026/10/29 10:31:31 [INFO] access token "cc-releases-2": POST /configuration/global-floor → 303
+2026/10/29 10:39:45 [INFO] Event from tester-1: backup_run_failures (error) — App "gokapi": its unit restore (not stable after its start) put the data back, but the app did not start in time (R-1270). The household was told it has not started yet; the pre-restore copy is kept. If it does not start by itself within 30 minutes, it needs a look. Error: no stable 3m0s within 30m0s
+2026/10/29 10:39:45 [INFO] Operator email sent for tester-1/backup_run_failures
+2026/10/29 11:07:36 [INFO] Customer tester-1 controller-version floor override set to "" (declared MinAgent "")
+2026/10/29 11:07:36 [INFO] access token "cc-releases-2": POST /customers/tester-1/floor → 303
+2026/10/29 11:07:37 [INFO] Customer demo-hp controller-version floor override set to "" (declared MinAgent "")
+2026/10/29 11:07:37 [INFO] access token "cc-releases-2": POST /customers/demo-hp/floor → 303
+2026/10/29 11:07:37 [INFO] Customer demo-felhom controller-version floor override set to "" (declared MinAgent "")
+2026/10/29 11:07:37 [INFO] access token "cc-releases-2": POST /customers/demo-felhom/floor → 303

documentation/audits/d63-2026-10-29/opsign-agent.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# opsign-agent.sh <hub-key NAME> <version> <sha256> <host_id>... — one signed agent_update job per box, queued at the hub.
+# The hub key arrives through `sec run` under the NAME given; its value is never printed.
+set -euo pipefail
+KEYNAME=$1; VER=$2; SHA=$3; shift 3
+HUBKEY=${!KEYNAME}
+D=/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad/d63
+for H in "$@"; do
+  echo "== $H"
+  "$D/felhom-opsign" -op agent_update -host "$H" -key-id felhom-op-1 -key /mnt/5_hdd/felhom.eu/felhom-op-operational \
+    -agent-version "$VER" -sha256 "$SHA" -ttl 45m -upload http://10.43.52.34:8080 -hub-key "$HUBKEY" 2>&1 | tail -3
+done

documentation/audits/d63-2026-10-29/opsign-agent.txt

@@ -0,0 +1,12 @@
+== tester-1-d70be4
+signed: op=agent_update host=tester-1-d70be4 guest="" key_id=felhom-op-1 nonce=d853bac716c5c4fbfb2b566436885b14 expires=2026-10-29T09:37:05Z
+{"op_blob_b64":"eyJleHBpcmVzX2F0IjoiMjAyNi0xMC0yOVQwOTozNzowNVoiLCJpc3N1ZWRfYXQiOiIyMDI2LTEwLTI5VDA4OjUyOjA1WiIsImtleV9pZCI6ImZlbGhvbS1vcC0xIiwibm9uY2UiOiJkODUzYmFjNzE2YzVjNGZiZmIyYjU2NjQzNjg4NWIxNCIsIm9wIjoiYWdlbnRfdXBkYXRlIiwicGFyYW1zIjp7InNoYTI1NiI6IjFlNzAxOTZjNTEzNDZjZjEwZTNmMDRmNTE5NzVkZTU4OTliNWRjNmJhYTVhNDVlYzFkMTdiYzI0ODgzMGQyY2EiLCJ2ZXJzaW9uIjoiMC4xNjguMCJ9LCJ0YXJnZXQiOnsiZ3Vlc3RfaWQiOiIiLCJob3N0X2lkIjoidGVzdGVyLTEtZDcwYmU0In19","sig_armored":"-----BEGIN SSH SIGNATURE-----\nU1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgvzPSoI2ADfHbHEAHRCPmujzBoW\nMZnxNr+xYHre2UvD4AAAAMZmVsaG9tLW9wLXYxAAAAAAAAAAZzaGE1MTIAAABTAAAAC3Nz\naC1lZDI1NTE5AAAAQJsggYiigisd0vNWgbqm+IqEMkSJuWqcdR/HvrB1CFdYTALfaT7S8n\nsX2+4JNk/Qn95q08vRge98k/xgoqtuxQ8=\n-----END SSH SIGNATURE-----\n"}
+uploaded signed op to the hub jobs queue
+== demo-hp-bb76ea
+signed: op=agent_update host=demo-hp-bb76ea guest="" key_id=felhom-op-1 nonce=4806149bb90d075b876a866744da475f expires=2026-10-29T09:37:05Z
+{"op_blob_b64":"eyJleHBpcmVzX2F0IjoiMjAyNi0xMC0yOVQwOTozNzowNVoiLCJpc3N1ZWRfYXQiOiIyMDI2LTEwLTI5VDA4OjUyOjA1WiIsImtleV9pZCI6ImZlbGhvbS1vcC0xIiwibm9uY2UiOiI0ODA2MTQ5YmI5MGQwNzViODc2YTg2Njc0NGRhNDc1ZiIsIm9wIjoiYWdlbnRfdXBkYXRlIiwicGFyYW1zIjp7InNoYTI1NiI6IjFlNzAxOTZjNTEzNDZjZjEwZTNmMDRmNTE5NzVkZTU4OTliNWRjNmJhYTVhNDVlYzFkMTdiYzI0ODgzMGQyY2EiLCJ2ZXJzaW9uIjoiMC4xNjguMCJ9LCJ0YXJnZXQiOnsiZ3Vlc3RfaWQiOiIiLCJob3N0X2lkIjoiZGVtby1ocC1iYjc2ZWEifX0=","sig_armored":"-----BEGIN SSH SIGNATURE-----\nU1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgvzPSoI2ADfHbHEAHRCPmujzBoW\nMZnxNr+xYHre2UvD4AAAAMZmVsaG9tLW9wLXYxAAAAAAAAAAZzaGE1MTIAAABTAAAAC3Nz\naC1lZDI1NTE5AAAAQCTkF8R1B2TQsTUJwD1Wp8Bvut39A3AA1IOUiZY6kkUMJYU78i4Voa\nxYPt4TtoRtNKJH2lgX9+8ej2ndtarNpAI=\n-----END SSH SIGNATURE-----\n"}
+uploaded signed op to the hub jobs queue
+== demo-felhom-8363b5
+signed: op=agent_update host=demo-felhom-8363b5 guest="" key_id=felhom-op-1 nonce=7655701562aaeb40205877356e345792 expires=2026-10-29T09:37:05Z
+{"op_blob_b64":"eyJleHBpcmVzX2F0IjoiMjAyNi0xMC0yOVQwOTozNzowNVoiLCJpc3N1ZWRfYXQiOiIyMDI2LTEwLTI5VDA4OjUyOjA1WiIsImtleV9pZCI6ImZlbGhvbS1vcC0xIiwibm9uY2UiOiI3NjU1NzAxNTYyYWFlYjQwMjA1ODc3MzU2ZTM0NTc5MiIsIm9wIjoiYWdlbnRfdXBkYXRlIiwicGFyYW1zIjp7InNoYTI1NiI6IjFlNzAxOTZjNTEzNDZjZjEwZTNmMDRmNTE5NzVkZTU4OTliNWRjNmJhYTVhNDVlYzFkMTdiYzI0ODgzMGQyY2EiLCJ2ZXJzaW9uIjoiMC4xNjguMCJ9LCJ0YXJnZXQiOnsiZ3Vlc3RfaWQiOiIiLCJob3N0X2lkIjoiZGVtby1mZWxob20tODM2M2I1In19","sig_armored":"-----BEGIN SSH SIGNATURE-----\nU1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgvzPSoI2ADfHbHEAHRCPmujzBoW\nMZnxNr+xYHre2UvD4AAAAMZmVsaG9tLW9wLXYxAAAAAAAAAAZzaGE1MTIAAABTAAAAC3Nz\naC1lZDI1NTE5AAAAQKk9J7puhHscGOLV/msgTtOaX+E/XGHlettw5r9xiu7MnvJ/wWXlUD\no2vrV5I0XtvqFInZY+0hfp8jnSjqtHXAI=\n-----END SSH SIGNATURE-----\n"}
+uploaded signed op to the hub jobs queue

documentation/audits/d63-2026-10-29/p4_live.py

@@ -0,0 +1,108 @@
+#!/usr/bin/env python3
+"""p4_live.py <phase> — Tester 1 on the release build (controller 0.347.0, agent 0.168.0), 2026-10-29, attended.
+  deploy    a throwaway Gokapi (catalog cap 128 MB).
+  backup    the household's „Mentés most" (whole box; the only backup press there is, R-648) — makes Gokapi's unit.
+  restore   one LOCAL restore of Gokapi from its unit through the restore page's form; the app must run after.
+  oom       a memory hog inside Gokapi (`tail /dev/zero`, a child): the host counter rises; the controller logs the OOM
+            line ONCE with the app and the source, and pushes app_oom once.
+  teardown  the app removed through the product."""
+import json, os, sys, time
+
+sys.path.insert(0, "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts")
+SCR = "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad"
+os.environ.setdefault("SC", f"{SCR}/rt-tester1")
+os.environ.setdefault("TARGET", "tester-1")
+import box_walk as w
+
+EV = f"{SCR}/d63/ev"
+APP, SUB = "gokapi", "gokapi"
+
+
+def now():
+    return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+
+
+def look():
+    st = w.stack(APP)
+    w.say(f"  app: state={st.get('state')} hold={st.get('hold_reason')!r} deployed={st.get('deployed')}")
+    w.say("  container: " + w.guest(f"docker ps -a --filter name=^/{APP}$ --format '{{{{.Image}}}} {{{{.Status}}}}'").strip())
+    return st
+
+
+w.login()
+phase = sys.argv[1]
+t0 = now()
+w.say(f"=== phase {phase} at {t0}")
+w.say("controller: " + w.guest("docker ps --filter name=^/felhom-controller$ --format '{{.Image}} {{.Status}}'").strip())
+
+if phase == "deploy":
+    assert w.deploy(APP, SUB), "deploy failed"
+    w.say("  cap: " + w.guest(f"docker inspect -f '{{{{.HostConfig.Memory}}}}' {APP}; docker exec {APP} sh -c 'command -v tail; echo ok'").strip())
+    look()
+
+elif phase == "backup":
+    c, d = w.ctl("POST", "/api/backup/run")
+    w.say(f"  backup press → {c} {str(d)[:160]}")
+    for _ in range(240):
+        time.sleep(10)
+        lg = w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -E 'App-data backup completed|Recovery unit (captured|NOT captured) for {APP}' | tail -3")
+        if "App-data backup completed" in lg:
+            break
+    w.say("  log: " + w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -E 'App-data backup completed|{APP}' | tail -8"))
+    w.say(f"  restore points: {json.dumps(w.snapshots(APP))[:300]}")
+
+elif phase == "restore":
+    look()
+    res = w.restore(APP)
+    w.say(f"  result: {json.dumps(res)[:600]}")
+    code, d = w.ctl("GET", "/api/backup/restore-status")
+    w.say(f"  restore record: {json.dumps(d)[:600]}")
+    w.say("  log: " + w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -E '{APP}|Restore-from-unit|R-1270|restore copy' | tail -15"))
+    look()
+    w.say("  front door: " + str(w.app_curl(SUB, "/")[1]))
+
+elif phase == "oom":
+    cid = w.guest(f"docker inspect -f '{{{{.Id}}}}' {APP}").strip()
+    hostf = f"/sys/fs/cgroup/lxc/9201/ns/system.slice/docker-{cid}.scope/memory.events"
+    w.say("  host counter before: " + w.sh(w.ssh_args() + ["root@192.168.0.154", f"grep oom_kill {hostf}"]).stdout.strip())
+    w.say("  hog: " + w.guest(f"docker exec {APP} sh -c 'tail /dev/zero' ; echo rc=$?", timeout=300).strip())
+    w.say("  after: " + w.guest(f"docker inspect -f 'running={{{{.State.Running}}}} oomkilled={{{{.State.OOMKilled}}}} restarts={{{{.RestartCount}}}}' {APP}"))
+    w.say("  host counter after: " + w.sh(w.ssh_args() + ["root@192.168.0.154", f"grep oom_kill {hostf}"]).stdout.strip())
+    time.sleep(75)
+    w.say("  OOM lines (first 2 scans): " + w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -iE 'oom' | head -10"))
+    time.sleep(65)
+    w.say("  app_oom events pushed: " + w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -cE 'Event pushed: app_oom'"))
+    w.say("  OOM lines (after 4 scans): " + w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -iE 'oom' | head -10"))
+
+elif phase == "d62":
+    # D62 end to end on the release build and hub 0.175.0: restore, the copy listed, deleted through the form at once
+    import re, subprocess
+    res = w.restore(APP)
+    w.say(f"  restore ok={res.get('state_after')} hold={res.get('hold_after')}")
+    page = w.page("/backups/restore")
+    rows = re.findall(r'<details class="kept-delete" data-restore-copy="([^"]+)">', page)
+    w.say(f"  rows: {rows}")
+    m = re.search(r'data-restore-copies="true">(.*?)</div>\s*<script>', page, re.S)
+    if m:
+        w.say("  list text: " + re.sub(r"\s+", " ", re.sub(r"<[^>]+>", " ", m.group(1))).strip()[:400])
+    if rows:
+        app, stamp = rows[0].split("/", 1)
+        names = re.findall(r'<span class="app-row-name">([^<]+)</span>', page[:page.find('data-restore-copy="' + rows[0])])
+        typed = names[-1].strip() if names else app
+        sess = open(f"{w.SC}/sess{os.getpid()}.txt").read().strip()
+        csrf = open(f"{w.SC}/csrf{os.getpid()}.txt").read().strip()
+        r = subprocess.run(["curl", "-sk", "-w", "\n%{http_code}", "-H", w.HOSTHDR, "-H", f"Cookie: {sess}", "-H", f"X-CSRF-Token: {csrf}",
+                            "-H", "Accept: application/json", "-X", "POST", "--data-urlencode", f"_csrf={csrf}",
+                            "--data-urlencode", f"app={app}", "--data-urlencode", f"stamp={stamp}", "--data-urlencode", f"confirm={typed}",
+                            f"{w.BASE}/backup/restore-copy/delete"], capture_output=True, text=True, timeout=60)
+        w.say(f"  delete ({typed!r}) → {r.stdout.strip()[-300:]}")
+        time.sleep(8)
+        w.say("  log: " + w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -E 'restore copy DELETED|restore_copy_deleted|Event push' | tail -4"))
+        w.say("  copies left: " + (w.guest("find /mnt/sys_drive/felhom-data/backups/restore-undo -mindepth 2 -maxdepth 2 2>/dev/null").strip() or "(none)"))
+
+elif phase == "teardown":
+    look()
+    w.remove(APP)
+    look()
+
+open(os.path.join(EV, f"p4-{phase}.log"), "a").write("\n".join(w.LOG) + "\n")

documentation/audits/d63-2026-10-29/perbox-floors-cleared.txt

@@ -0,0 +1,9 @@
+== tester-1
+HTTP/1.1 303 See Other
+Location: /customers/tester-1?flash=floor_set
+== demo-hp
+HTTP/1.1 303 See Other
+Location: /customers/demo-hp?flash=floor_set
+== demo-felhom
+HTTP/1.1 303 See Other
+Location: /customers/demo-felhom?flash=floor_set

documentation/audits/d63-2026-10-29/perbox-floors.txt

@@ -0,0 +1,9 @@
+== tester-1
+HTTP/1.1 303 See Other
+Location: /customers/tester-1?flash=floor_set
+== demo-hp
+HTTP/1.1 303 See Other
+Location: /customers/demo-hp?flash=floor_set
+== demo-felhom
+HTTP/1.1 303 See Other
+Location: /customers/demo-felhom?flash=floor_set

documentation/audits/d63-2026-10-29/run-ceremony.log

@@ -0,0 +1,11 @@
+09:40:09 preflight → 200 {"data": {"agent_supported": true, "escrow_state": "escrowed", "items": [{"id": "pbs_storage_id", "ok": true, "detail": "felhom-pbs"}, {"id": "dr_tier", "ok": true, "detail": "DR tier applied"}, {"id": "age_binary", "ok": true, "detail": "/usr/bin/age"}, {"id": "hub_upload", "ok": true, "detail": "hub upload target configured"}, {"id": "staged_secret", "ok": false, "detail": "no staged secret (informational \u2014 the controller re-stages when offsite is configured)"}, {"id": "sudo_grant", "ok": true, "detail": "sudo grant listed (list-mode)"}], "offbox_configured": true, "ok": true, "stale": 
+09:40:09 start → 200 {"data": {"job_id": "escrow-1793263209404621564", "phase": "running"}, "error": "", "ok": true}
+09:40:12 status → 200 {"data": {"claim_expires_in_sec": 599, "claimable": true, "claimed": false, "detail": "", "entropy_bits": 129.24070185585344, "job_id": "escrow-1793263209404621564", "key_fingerprint": "82:c6:61:ba:ae:36:47:af:0f:dc:17:0b:0d:5c:5d:63:cd:e7:47:e6:fe:9a:14:fe:66:c1:f0:e6:8f:a1:44:3b", "phase": "done", "restic_pw_sealed": true, "uploaded": false}, "error": "", "ok": true}
+09:40:12 claim → 200; code received: True (10 words); confirm_required=True positions=[3, 4]
+09:40:12 confirm → 200 {"data": {"uploaded": true}, "error": "", "ok": true}
+09:52:46 controller escrow lines:
+2026/10/29 08:40:09 [INFO] [web] escrow readiness: READY (agent preflight ok) — the recovery-code reminder is shown
+2026/10/29 08:40:09 [INFO] [web] escrow ceremony started via wizard (job escrow-1793263209404621564)
+2026/10/29 08:40:12 [INFO] [web] escrow recovery code claimed (one-shot; not logged) — its package waits for the typed-back words
+2026/10/29 08:40:12 [INFO] [web] escrow recovery code typed back — the new package is on the hub (R-1151)
+

documentation/audits/d63-2026-10-29/run-ceremony2.log

@@ -0,0 +1,12 @@
+10:34:17 preflight → 200 {"data": {"agent_supported": true, "escrow_state": "escrowed", "items": [{"id": "pbs_storage_id", "ok": true, "detail": "felhom-pbs"}, {"id": "dr_tier", "ok": true, "detail": "DR tier applied"}, {"id": "age_binary", "ok": true, "detail": "/usr/bin/age"}, {"id": "hub_upload", "ok": true, "detail": "hub upload target configured"}, {"id": "staged_secret", "ok": false, "detail": "no staged secret (informational \u2014 the controller re-stages when offsite is configured)"}, {"id": "sudo_grant", "ok": true, "detail": "sudo grant listed (list-mode)"}], "offbox_configured": true, "ok": true, "stale": 
+10:34:17 start → 200 {"data": {"job_id": "escrow-1793266457881359663", "phase": "running"}, "error": "", "ok": true}
+10:34:20 status → 200 {"data": {"claim_expires_in_sec": 599, "claimable": true, "claimed": false, "detail": "", "entropy_bits": 129.24070185585344, "job_id": "escrow-1793266457881359663", "key_fingerprint": "82:c6:61:ba:ae:36:47:af:0f:dc:17:0b:0d:5c:5d:63:cd:e7:47:e6:fe:9a:14:fe:66:c1:f0:e6:8f:a1:44:3b", "phase": "done", "restic_pw_sealed": true, "uploaded": false}, "error": "", "ok": true}
+10:34:20 claim → 200; code received: True (10 words); confirm_required=True positions=[3, 7]
+10:34:22 sec put TESTER1_RECOVERY_CODE_2026_10_29 → rc=0 (file shredded by sec: True)
+10:34:22 confirm → 200 {"data": {"uploaded": true}, "error": "", "ok": true}
+10:46:52 controller escrow lines:
+2026/10/29 09:34:17 [INFO] [web] escrow readiness: READY (agent preflight ok) — the recovery-code reminder is shown
+2026/10/29 09:34:17 [INFO] [web] escrow ceremony started via wizard (job escrow-1793266457881359663)
+2026/10/29 09:34:20 [INFO] [web] escrow recovery code claimed (one-shot; not logged) — its package waits for the typed-back words
+2026/10/29 09:34:22 [INFO] [web] escrow recovery code typed back — the new package is on the hub (R-1151)
+

documentation/audits/d63-2026-10-29/run-d62.log

@@ -0,0 +1,24 @@
+09:10:56 === phase d62 at 2026-10-29T08:10:56Z
+09:10:59 controller image: gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault Up 18 minutes (healthy)
+09:10:59   app: state=running hold=None deployed=True
+09:11:03   compose image: image: ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468
+09:11:06   container: ghcr.io/kozea/radicale:3.8.0 Up 45 seconds (healthy)
+09:11:09   copies: (none)
+09:11:09   restore (no fault): ('HTTP/2 302', ['location: /backups/restore/app?name=radicale&flash=flash.offbox.full_restore_started'])
+09:11:09       +    0s d62 restore: (True, 'offbox-reconstitute', None, '')
+09:12:12       +   64s d62 restore: (False, 'offbox-reconstitute', None, '')
+09:12:12   restore record: {"running": false, "op": "offbox-reconstitute", "stack": "radicale", "started_at": "2026-10-29T08:11:09.437135355Z", "last": {"op": "offbox-reconstitute", "stack": "radicale", "ok": true, "message": "A(z) radicale: 0 f\u00e1jl \u00e9s 1 adatk\u00f6tet vissza\u00e1ll\u00edtva (ment\u00e9s: 2026. 10. 29. 09:02) \u2014 az alkalmaz\u00e1s \u00fajraindult. Ennek az alkalmaz\u00e1snak nincs adatb\u00e1zisa.", "finished_at": "2026-10-29T08:12:11.98337803Z"}, "last_recent": true}
+09:12:13   app: state=running hold=None deployed=True
+09:12:16   compose image: image: ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c
+09:12:19   container: ghcr.io/kozea/radicale:3.8.1 Up About a minute (healthy)
+09:12:22   copies: (none)
+09:12:58   page: 1 deletable restore copy row(s)
+09:12:58   list text: Radicale visszaállítás: 2026. 10. 29. 09:11 · 37,3 KB · automatikusan törlődik: 2026. 11. 28. Törlés most Ez a(z) Radicale adatainak másolata a 2026. 10. 29-i visszaállítás előttről. Ha törlöd, a visszaállítás többé nem vonható vissza, és ez a másolat nem hozható vissza. A törléshez írd be: Radicale Törlés most
+09:12:58   warning shown: Ez a(z) Radicale adatainak másolata a 2026. 10. 29-i visszaállítás előttről. Ha törlöd, a visszaállítás többé nem vonható vissza, és ez a másolat nem hozható vissza.
+09:12:58   wrong name → HTTP/2 400 
+09:12:58   right name ('sonarr') → HTTP/2 400 
+09:13:02   copy after: (gone)
+09:13:05   controller log: 2026/10/29 08:12:58 [WARN] [web] restore-copy delete REFUSED for /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081111Z: the typed confirmation did not match
+2026/10/29 08:12:58 [WARN] [web] restore-copy delete REFUSED for /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081111Z: the typed confirmation did not match
+

documentation/audits/d63-2026-10-29/run-d62b.log

@@ -0,0 +1,26 @@
+09:13:34 === phase d62 at 2026-10-29T08:13:34Z
+09:13:37 controller image: gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault Up 20 minutes (healthy)
+09:13:37   app: state=running hold=None deployed=True
+09:13:40   compose image: image: ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c
+09:13:43   container: ghcr.io/kozea/radicale:3.8.1 Up 2 minutes (healthy)
+09:13:46   copies: /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081111Z
+09:13:46   restore (no fault): ('HTTP/2 302', ['location: /backups/restore/app?name=radicale&flash=flash.offbox.full_restore_started'])
+09:13:46       +    0s d62 restore: (True, 'offbox-reconstitute', None, '')
+09:14:50       +   64s d62 restore: (False, 'offbox-reconstitute', None, '')
+09:15:32   page at 2026-10-29T08:15:32Z: 2 deletable restore copy row(s); restore ok=True
+09:15:32   list text: Radicale visszaállítás: 2026. 10. 29. 09:11 · 37,3 KB · automatikusan törlődik: 2026. 11. 28. Törlés most Ez a(z) Radicale adatainak másolata a 2026. 10. 29-i visszaállítás előttről. Ha törlöd, a visszaállítás többé nem vonható vissza, és ez a másolat nem hozható vissza. A törléshez írd be: Radicale Törlés most Radicale visszaállítás: 2026. 10. 29. 09:13 · 29,3 KB · automatikusan törlődik: 2026. 11. 28. Törlés most Ez a(z) Radicale adatainak másolata a 2026. 10. 29-i visszaállítás előttről. Ha törlöd, a visszaállítás többé nem vonható vissza, és ez a másolat nem hozható vissza. A törléshez írd be: Radicale Törlés most
+09:15:32   wrong name ('Radicalex') → HTTP/2 400 {"error":"A beírt név nem egyezik (Radicale), ezért nem töröltünk semmit.","ok":false,"redirect":""}
+09:15:32   right name ('Radicale') → HTTP/2 200 {"error":"","ok":true,"redirect":"/backups/restore?fa=Radicale\u0026flash=flash.restore_copy.deleted"}
+09:15:32   app: state=running hold=None deployed=True
+09:15:35   compose image: image: ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c
+09:15:38   container: ghcr.io/kozea/radicale:3.8.1 Up About a minute (healthy)
+09:15:41   copies: /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081348Z
+09:16:38   page after: rows=1
+09:16:41   controller log: 2026/10/29 08:15:32 [WARN] [web] restore-copy delete REFUSED for /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081111Z: the typed confirmation did not match
+2026/10/29 08:15:32 [WARN] [web] restore copy DELETED by hand (D62): app=radicale restore_at=2026-10-29T08:11:11Z size=38243 path=/mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081111Z who=the household (dashboard sign-in) from 192.168.0.180
+2026/10/29 08:15:38 [WARN] Event push failed after 3 attempts (restore_copy_deleted/info): HTTP 400
+
+09:16:41       +    0s settle: (False, 'offbox-reconstitute', None, '')
+09:16:50     radicale: readback http=200 found=True (controls: no auth 401, wrong 401, absent 404)
+09:16:50   T1 readback: True

documentation/audits/d63-2026-10-29/run-h4.log

@@ -0,0 +1,14 @@
+09:04:12 === phase h4 at 2026-10-29T08:04:12Z
+09:04:15 controller image: gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault Up 11 minutes (healthy)
+09:04:27   [h4] compose → ghcr.io/kozea/radicale:3.8.0; Restart → 200 {'ok': True, 'data': {'state': 'starting'}, 'message': 'Stack radicale restart requested — state now: starting'}
+09:04:34   app: state=starting hold=None deployed=True
+09:04:37   compose image: image: ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468
+09:04:40   container: ghcr.io/kozea/radicale:3.8.0 Up 12 seconds (healthy)
+09:04:43   copies: (none)
+09:04:45     radicale: MKCALENDAR /csalad/felhom-a718ed00/ http=401
+Traceback (most recent call last):
+  File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad/d63/d63_live.py", line 176, in <module>
+    assert t2, "seed T2 failed"
+           ^^
+AssertionError: seed T2 failed

documentation/audits/d63-2026-10-29/run-h4b.log

@@ -0,0 +1,19 @@
+09:05:09 === phase h4 at 2026-10-29T08:05:09Z
+09:05:12 controller image: gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault Up 12 minutes (healthy)
+09:05:17   [h4] compose → ghcr.io/kozea/radicale:3.8.0; Restart → 200 {'ok': True, 'data': {'state': 'starting'}, 'message': 'Stack radicale restart requested — state now: starting'}
+09:05:23   app: state=starting hold=None deployed=True
+09:05:26   compose image: image: ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468
+09:05:29   container: ghcr.io/kozea/radicale:3.8.0 Up 12 seconds (healthy)
+09:05:33   copies: (none)
+09:05:44     radicale: MKCALENDAR /csalad/felhom-5424c15e/ http=401
+09:05:56     radicale: MKCALENDAR /csalad/felhom-e4dc5c62/ http=401
+09:06:07     radicale: MKCALENDAR /csalad/felhom-561ce5bf/ http=401
+09:06:19     radicale: MKCALENDAR /csalad/felhom-f49d45b0/ http=401
+09:06:30     radicale: MKCALENDAR /csalad/felhom-258756e5/ http=401
+09:06:41     radicale: MKCALENDAR /csalad/felhom-dc019255/ http=401
+Traceback (most recent call last):
+  File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad/d63/d63_live.py", line 181, in <module>
+    assert t2, "seed T2 failed"
+           ^^
+AssertionError: seed T2 failed

documentation/audits/d63-2026-10-29/run-h4c.log

@@ -0,0 +1,19 @@
+09:07:01 === phase h4 at 2026-10-29T08:07:01Z
+09:07:04 controller image: gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault Up 14 minutes (healthy)
+09:07:08   [h4] compose → ghcr.io/kozea/radicale:3.8.1; Restart → 200 {'ok': True, 'data': {'state': 'starting'}, 'message': 'Stack radicale restart requested — state now: starting'}
+09:07:14   app: state=starting hold=None deployed=True
+09:07:17   compose image: image: ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c
+09:07:21   container: ghcr.io/kozea/radicale:3.8.1 Up 11 seconds (healthy)
+09:07:24   copies: (none)
+09:07:35     radicale: MKCALENDAR /csalad/felhom-4d229933/ http=401
+09:07:46     radicale: MKCALENDAR /csalad/felhom-091d495a/ http=401
+09:07:57     radicale: MKCALENDAR /csalad/felhom-a07bdb98/ http=401
+09:08:08     radicale: MKCALENDAR /csalad/felhom-f69aaadf/ http=401
+09:08:20     radicale: MKCALENDAR /csalad/felhom-576f2139/ http=401
+09:08:32     radicale: MKCALENDAR /csalad/felhom-a26a7f40/ http=401
+Traceback (most recent call last):
+  File "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/0b76c4d7-9b39-4e4e-869d-d0d2c6006331/scratchpad/d63/d63_live.py", line 182, in <module>
+    assert t2, "seed T2 failed"
+           ^^
+AssertionError: seed T2 failed

documentation/audits/d63-2026-10-29/run-h4d.log

@@ -0,0 +1,71 @@
+09:08:54 === phase h4 at 2026-10-29T08:08:54Z
+09:08:58 controller image: gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault Up 15 minutes (healthy)
+09:09:02   [h4] compose → ghcr.io/kozea/radicale:3.8.1; Restart → 200 {'ok': True, 'data': {'state': 'starting'}, 'message': 'Stack radicale restart requested — state now: starting'}
+09:09:08   app: state=starting hold=None deployed=True
+09:09:11   compose image: image: ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c
+09:09:14   container: ghcr.io/kozea/radicale:3.8.1 Up 11 seconds (healthy)
+09:09:17   copies: (none)
+09:09:28     radicale: MKCALENDAR /csalad/felhom-749ed888/ http=201
+09:09:28     radicale: PUT event http=201
+09:09:33   [h4] compose → ghcr.io/kozea/radicale:3.8.0; Restart → 200 {'ok': True, 'data': {'state': 'starting'}, 'message': 'Stack radicale restart requested — state now: starting'}
+09:09:39   app: state=running hold=None deployed=True
+09:09:42   compose image: image: ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468
+09:09:45   container: ghcr.io/kozea/radicale:3.8.0 Up 11 seconds (healthy)
+09:09:48   copies: (none)
+09:10:03   app: state=running hold=None deployed=True
+09:10:07   compose image: image: ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468
+09:10:10   container: ghcr.io/kozea/radicale:3.8.0 Up 36 seconds (healthy)
+09:10:13   copies: (none)
+09:10:16   armed: defwrite-after-compose = 0 1
+09:10:16   restore: ('HTTP/2 302', ['location: /backups/restore/app?name=radicale&flash=flash.offbox.full_restore_started'])
+09:10:16       +    0s h4 restore: (True, 'offbox-reconstitute', None, '')
+09:10:22       +    6s h4 restore: (False, 'offbox-reconstitute', None, '')
+09:10:22   restore record: {"running": false, "op": "offbox-reconstitute", "stack": "radicale", "started_at": "2026-10-29T08:10:16.87791986Z", "last": {"op": "offbox-reconstitute", "stack": "radicale", "ok": false, "message": "A teljes vissza\u00e1ll\u00edt\u00e1s sikertelen: restoring radicale: writing the snapshot's definition failed: FAULT INJECTED at defwrite-after-compose (D63 test build)", "finished_at": "2026-10-29T08:10:20.32238992Z"}, "last_recent": true}
+09:10:26   controller log:
+2026/10/29 08:09:01 [INFO] [api] restart requested for stack: radicale
+2026/10/29 08:09:01 [INFO] [stacks] Restarting stack: radicale
+2026/10/29 08:09:02 [INFO] [stacks] Stack radicale: started again: radicale 2026-10-29T08:07:08.330110078Z → 2026-10-29T08:09:01.852817156Z
+2026/10/29 08:09:02 [INFO] [stacks] Stack radicale restarted successfully (took 1.1s)
+2026/10/29 08:09:05 [INFO] [stacks] Stack radicale post-start status:
+2026/10/29 08:09:05 [INFO] [stacks]   radicale   ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c   running   Up 3 seconds (health: starting)
+2026/10/29 08:09:32 [INFO] [api] restart requested for stack: radicale
+2026/10/29 08:09:32 [INFO] [stacks] Restarting stack: radicale
+2026/10/29 08:09:33 [INFO] [stacks] Stack radicale: started again: radicale 2026-10-29T08:09:01.852817156Z → 2026-10-29T08:09:32.950147579Z
+2026/10/29 08:09:33 [INFO] [stacks] Stack radicale restarted successfully (took 0.8s)
+2026/10/29 08:09:33 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 0 encrypted, 1 sensitive fields
+2026/10/29 08:09:33 [INFO] [stacks] installed-images radicale: recorded 1 service(s) (radicale=ghcr.io/kozea/radicale:3.8.0 (sha256:0d60d544710b…))
+2026/10/29 08:09:33 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/29 08:09:36 [INFO] [stacks] Stack radicale post-start status:
+2026/10/29 08:09:36 [INFO] [stacks]   radicale   ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468   running   Up 3 seconds (health: starting)
+2026/10/29 08:10:18 [INFO] [offbox] radicale: snapshot 78002a51 holds data of [ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c] (written 2026-10-29T08:03:01Z); the app runs [ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468] — it comes back at the snapshot's version
+2026/10/29 08:10:19 [INFO] [stacks] Stopping stack: radicale
+2026/10/29 08:10:19 [INFO] [stacks] Stack radicale stopped successfully (took 0.3s)
+2026/10/29 08:10:19 [INFO] [offbox] radicale: the live volumes are copied before the restore (1 copied, 0 new from the snapshot) in 332ms — /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081019Z (R-893)
+2026/10/29 08:10:19 [INFO] [backup] radicale: the restore copy is recorded — a power cut from here on puts the app back on start (R-1195)
+2026/10/29 08:10:19 [ERROR] [faultinject] defwrite-after-compose: FAULT INJECTED (fail left 0)
+2026/10/29 08:10:19 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 1 encrypted, 1 sensitive fields
+2026/10/29 08:10:19 [INFO] [stacks] Redeploying radicale from recovery unit with 4 env vars
+2026/10/29 08:10:19 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/29 08:10:19 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 0 encrypted, 1 sensitive fields
+2026/10/29 08:10:19 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/29 08:10:19 [INFO] [stacks] pin radicale: radicale=ghcr.io/kozea/radicale:3.8.0
+2026/10/29 08:10:19 [INFO] [offbox] radicale: the definition write failed (FAULT INJECTED at defwrite-after-compose (D63 test build)); the live definition is written back before the start
+2026/10/29 08:10:19 [INFO] [stacks] Starting stack: radicale
+2026/10/29 08:10:20 [INFO] [stacks] Stack radicale started successfully (took 0.3s)
+2026/10/29 08:10:20 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/radicale — 4 env vars, 0 encrypted, 1 sensitive fields
+2026/10/29 08:10:20 [INFO] [stacks] installed-images radicale: recorded 1 service(s) (radicale=ghcr.io/kozea/radicale:3.8.0 (sha256:0d60d544710b…))
+2026/10/29 08:10:20 [INFO] [stacks] SaveAppConfig: saved config for radicale
+2026/10/29 08:10:20 [ERROR] [web] off-box reconstitute radicale (async): restoring radicale: writing the snapshot's definition failed: FAULT INJECTED at defwrite-after-compose (D63 test build)
+2026/10/29 08:10:23 [INFO] [stacks] Stack radicale post-start status:
+2026/10/29 08:10:23 [INFO] [stacks]   radicale   ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468   running   Up 3 seconds (health: starting)
+
+09:10:26   app: state=starting hold=None deployed=True
+09:10:29   compose image: image: ghcr.io/kozea/radicale:3.8.0@sha256:0d60d544710bda0fa83bfdea685db731b6cbf74e12d7f22b4997e9cff3a50468
+09:10:33   container: ghcr.io/kozea/radicale:3.8.0 Up 11 seconds (healthy)
+09:10:36   copies: (none)
+09:10:38     radicale: readback http=200 found=True (controls: no auth 401, wrong 401, absent 404)
+09:10:38   T1 readback: True
+09:10:42     radicale: readback http=200 found=True (controls: no auth 401, wrong 401, absent 404)
+09:10:42   T2 readback: True
+09:10:45   fault file left: (none)

documentation/audits/d63-2026-10-29/run-h5.log

@@ -0,0 +1,42 @@
+09:18:10 === phase h5 at 2026-10-29T08:18:10Z
+09:18:13 controller image: gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault
+gitea.dooplex.hu/admin/felhom-controller:0.347.0-rc.1-d63fault Up 25 minutes (healthy)
+09:18:13   app: state=running hold=None deployed=True
+09:18:16   compose image: image: ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c
+09:18:19   container: ghcr.io/kozea/radicale:3.8.1 Up 4 minutes (healthy)
+09:18:23   copies: (none)
+09:18:26   volumes: ['radicale_radicale_data']
+09:18:29   blocker: bcea23576117066b1fbd5e4f1c0ce85c9ce3784f26e361ea1c8454c3f0bb24b7
+
+09:18:32   premise (rm -f with the blocker): Error response from daemon: remove radicale_radicale_data: volume is in use - [68fba54d540a68f2b3b45988d919d05a8b974388f4accd6825129acfec7f4e6d, bcea23576117066b1fbd5e4f1c0ce85c9ce3784f26e361ea1c8454c3f0bb24b7]
+rc=1
+09:18:35   armed: stop-radicale = 1 1
+09:18:35   restore: ('HTTP/2 302', ['location: /backups/restore/app?name=radicale&flash=flash.offbox.full_restore_started'])
+09:18:35       +    0s h5 restore: (True, 'offbox-reconstitute', None, '')
+09:18:41       +    6s h5 restore: (False, 'offbox-reconstitute', None, '')
+09:18:41   restore record: {"running": false, "op": "offbox-reconstitute", "stack": "radicale", "started_at": "2026-10-29T08:18:35.733794285Z", "last": {"op": "offbox-reconstitute", "stack": "radicale", "ok": false, "message": "A teljes vissza\u00e1ll\u00edt\u00e1s sikertelen: a(z) radicale vissza\u00e1ll\u00edt\u00e1sa f\u00e9l\u00faton megszakadt. Az alkalmaz\u00e1s egyes f\u00e1jljai vagy adatai m\u00e1r a ment\u00e9sb\u0151l sz\u00e1rmaznak, m\u00e1sok nem, ez\u00e9rt nem illenek \u00f6ssze. Az alkalmaz\u00e1st biztons\u00e1gi okb\u00f3l LE\u00c1LL\u00cdTVA hagytuk. Ehhez seg\u00edts\u00e9g kell: vedd fel vel\u00fcnk a kapcsolatot", "finished_at": "2026-10-29T08:18:38.916515438Z"}, "last_recent": true}
+09:18:44   controller log:
+2026/10/29 08:18:38 [WARN] [faultinject] stop-radicale: passed (skip left 0, fail left 1)
+2026/10/29 08:18:38 [INFO] [stacks] Stopping stack: radicale
+2026/10/29 08:18:38 [INFO] [stacks] Stack radicale stopped successfully (took 0.3s)
+2026/10/29 08:18:38 [INFO] [offbox] radicale: the live volumes are copied before the restore (1 copied, 0 new from the snapshot) in 345ms — /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081838Z (R-893)
+2026/10/29 08:18:38 [INFO] [backup] radicale: the restore copy is recorded — a power cut from here on puts the app back on start (R-1195)
+2026/10/29 08:18:38 [INFO] [backup] Restoring Docker volume radicale_radicale_data for radicale
+2026/10/29 08:18:38 [ERROR] [backup] Failed to remove volume radicale_radicale_data before the restore: Error response from daemon: remove radicale_radicale_data: volume is in use - [bcea23576117066b1fbd5e4f1c0ce85c9ce3784f26e361ea1c8454c3f0bb24b7] — exit status 1 (not written into)
+2026/10/29 08:18:38 [ERROR] [offbox] radicale: stopping the app before the put-back FAILED: FAULT INJECTED at stop-radicale (D63 test build) — nothing is put back; the copy stays at /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081838Z; holding the app
+2026/10/29 08:18:38 [WARN] [settings] restore hold SET for radicale — the app stays stopped until it is cleared
+2026/10/29 08:18:38 [ERROR] [faultinject] stop-radicale: FAULT INJECTED (fail left 0)
+2026/10/29 08:18:38 [INFO] [stacks] Stopping stack: radicale
+2026/10/29 08:18:38 [INFO] [stacks] Stack radicale stopped successfully (took 0.1s)
+2026/10/29 08:18:38 [ERROR] [offbox] radicale: the restore stopped half-way after files/volumes/version had moved — HOLDING the app stopped for support (R-893); error was: failed to restore 1 volume(s): [radicale_radicale_data]
+2026/10/29 08:18:38 [WARN] [offbox] radicale: the restore's volume copy is KEPT for support at /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081838Z
+2026/10/29 08:18:38 [ERROR] [web] off-box reconstitute radicale (async): a(z) radicale visszaállítása félúton megszakadt. Az alkalmazás egyes fájljai vagy adatai már a mentésből származnak, mások nem, ezért nem illenek össze. Az alkalmazást biztonsági okból LEÁLLÍTVA hagytuk. Ehhez segítség kell: vedd fel velünk a kapcsolatot
+2026/10/29 08:18:39 [INFO] Event pushed: backup_run_failures (error) [hu-only] — App "radicale" is HELD STOPPED for support: its off-site database restore failed. The database was rolled back to the pre-restore copy and the live definition written back, but its files and named volumes are the snapshot's — a mixed state (R-893). The app will not start from any path until the hold is cleared. Replay error: failed to restore 1 volume(s): [radicale_radicale_data]
+
+09:18:44   app: state=stopped hold='a(z) radicale visszaállítása 2026-10-29 08:18-kor félúton megszakadt: az adatai részben a mentésből származnak, részben nem. Az alkalmazás biztonsági okból leállítva marad. Ehhez segítség kell: vedd fel velünk a kapcsolatot' deployed=True
+09:18:47   compose image: image: ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c
+09:18:51   container: 
+09:18:54   copies: /mnt/sys_drive/felhom-data/backups/restore-undo/radicale/20261029T081838Z
+09:18:54   app page hold sentence: a(z) radicale visszaállítása 2026-10-29 08:18-kor félúton megszakadt: az adatai részben a mentésből származnak, részben nem. Az alkalmazás biztonsági okból leállítva marad. Ehhez segítség kell: vedd fel velünk a kapcsolatot  Mentések
+09:19:02   restore page: button off=True, needed line=['Nem törölhető: az alkalmazás le van állítva, és ez a másolat az út vissza. Az ügyfélszolgálat dönt róla.'], deletable rows=0
+09:19:05   fault file left: (none)

documentation/audits/d63-2026-10-29/run-p4-backup.log

@@ -0,0 +1,13 @@
+09:59:44 === phase backup at 2026-10-29T08:59:44Z
+09:59:47 controller: gitea.dooplex.hu/admin/felhom-controller:0.347.0 Up About a minute (healthy)
+09:59:47   backup press → 200 {'ok': True, 'message': 'Mentés elindítva'}
+10:00:32   log: 2026/10/29 09:00:00 [INFO] [backup] Volume dump: gokapi/gokapi_gokapi_data → 43.0 KB
+2026/10/29 09:00:00 [INFO] [backup] Restarting gokapi after volume dump
+2026/10/29 09:00:00 [INFO] [stacks] Starting stack: gokapi
+2026/10/29 09:00:01 [INFO] [stacks] Stack gokapi started successfully (took 0.4s)
+2026/10/29 09:00:04 [INFO] [stacks] Stack gokapi post-start status:
+2026/10/29 09:00:04 [INFO] [stacks]   gokapi    f0rc3/gokapi:v1.9.6   running   Up 3 seconds (health: starting)
+2026/10/29 09:00:23 [INFO] [backup] App-data backup completed: 2 databases (534.9 KB total), 4 volume dump(s) (35.888s)
+2026/10/29 09:00:23 [INFO] [backup] Recovery unit captured for gokapi → /mnt/sys_drive/felhom-data/backups/primary/gokapi (images=1, secrets-referenced=1, data_keys=0, portable-carried=0/0, withheld=1)
+
+10:00:32   restore points: [{"time": "2026-10-29T09:00:00Z", "short_id": "helyi", "tier": 1, "drive_label": "Bels\u0151 SSD (rendszer)"}]

documentation/audits/d63-2026-10-29/run-p4-deploy.log

@@ -0,0 +1,10 @@
+09:59:13 === phase deploy at 2026-10-29T08:59:13Z
+09:59:16 controller: gitea.dooplex.hu/admin/felhom-controller:0.347.0 Up About a minute (healthy)
+09:59:16   [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['GOKAPI_PASSWORD']
+09:59:16   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+09:59:36   [1] deployed, controller state=running, pinned={'gokapi': 'f0rc3/gokapi:v1.9.6'}
+09:59:39   cap: 134217728
+/usr/bin/tail
+ok
+09:59:39   app: state=running hold=None deployed=True
+09:59:42   container: f0rc3/gokapi:v1.9.6 Up 21 seconds (healthy)