2026-09-24: 09 decisions 24-25, part 5 shipped; the whole-copy truth table; fourth suppression; rows; STATUS; evidence

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-09-24 08:52:42 +02:00
parent 500488cad6
commit 86c4b9a0d8
60 changed files with 4066 additions and 57 deletions
@@ -14,6 +14,27 @@
 > language, one screen, no identifiers in the prose. Same subjects, different readers; merging them
 > would make one of the two audiences stop reading. `STATUS.md` is also a **view of `OPEN-ITEMS.md`**
 > and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
+## 2026-09-24 (morning) — the fleet on 0.267.0 then 0.268.0; the undo after a restore; option A; the ladder on the box
+
+**Operator rulings (`09` §3 decisions 24–25).** 24 — the fleet takes 0.267.0 although the chaos hour's stop rule
+fired (both faults predate it). 25 — R-659 option A: a held app's page names only a copy that brings the app back
+WHOLE; with none, it says so + support is informed + `app_hold_no_whole_copy` (critical, operator-only); option B
+(a database-only restore under existing files) is NOT built.
+
+**Shipped.** Controller **v0.268.0** (`206b035`, MinAgent 0.131.0): R-658 (undo selects volumes from the compose
+definition; the restore labels what it creates; the remove counts unlabelled volumes), R-659 (`backup.WholeOnTier`
+= the restores' own refusals; `HoldAfterFailedUpdateWhole`), R-660 (fourth suppression, `UpdateHeldStacks`), R-651,
+and `09` §6.4 **part 5** (one press = one tested step; `stacks.StepKey` = catalog `ladder.step_key`). Hub
+**v0.122.0** (`app_hold_no_whole_copy` allow-listed + operator-only + per-app cooldown). Catalog **`5ed599c`**:
+`steps/<key>.yml` for every intermediate step (8 backfilled from the NEWEST commit naming the step's images),
+gate rule 4 with decoys, the writer keeps the superseded step, R-653/R-656 in the bench, two suites un-drifted (R-663).
+Floors: 0.267.0 at 05:12Z, **0.268.0 at 06:47Z**, both with declared MinAgent 0.131.0; both demo boxes arrived
+healthy within ~15 s each time. Evidence and the claims the brief got wrong: `documentation/audits/ladder-2026-09-24/README.md`.
+
+**Measured, and now the design:** for an app with declared drive files, neither the own-unit nor the second-drive
+unit restore brings it back (R-538's guard is in the shared function) — only off-site does (`07` §6 table; R-661 is
+the second-drive gap). The box's `--depth 1` clone carries a template's `steps/` folder (whole tree, one commit).
+
 ## 2026-09-24 (night shift 2026-09-23) — tests off DooPlex's Docker, the test record, 12 published steps, the chaos hour

 **Operator word, `09` §3 decision 21:** tonight the catalog may move every app whose within-a-major edge is
@@ -1,34 +1,39 @@
-# REPORT — night shift 2026-09-23: DooPlex guarded, the test record, 12 published steps, the chaos hour
+# REPORT — 2026-09-24: the fleet to 0.267.0 and 0.268.0; the undo after a restore; option A; the ladder

-Full record: `documentation/audits/DRILL-night-2026-09-23.md`; step log `documentation/audits/night-2026-09-23/PROGRESS.md`.
-Architecture read first: `09-update-architecture.md` (§3 decisions 11–20, §6.1a, §6.4), `07` §6, `08`.
+Full record (not done first, claims checked, red-proofs, live proofs, teardown):
+`documentation/audits/ladder-2026-09-24/README.md`.

 ## Not done, or changed
- **The floor stays 0.266.0** (the brief's rule: Part D found R-658 and R-659). Recommended to raise — the two
-  P1s are pre-existing; STATUS carries the question.
- adventurelog not moved (R-655); gitea inconclusive (R-624); 11 across-major and 12 no-route/no-fixture apps listed.
- Chaos rounds 2–5 landed their accidents outside the action (the runner waited on the wrong file; fixed from
-  round 5/6). The event column was rebuilt from the controller's log after the debug ring missed two holds.
- The stop rule fired at round 11 (R-659) after all twelve rounds had run.

-## What shipped
- felhom-controller **v0.267.0** `80e6ad8c4772` (CI 919): R-650, R-640, R-499, R-518; deployed to 9202 only.
- app-catalog-felhom.eu: the test record + gates `6db08a5` (CI 920), R-612/R-613 `a5a729a`, harness fixes
-  `1263773`, 12 steps: romm ×2, ghost, wishlist, opengist, navidrome, komga (+768M), n8n, emby, kimai-db,
-  immich-ML, nextcloud (CI 922–937).
- felhom.eu: evidence, `09` §3 decisions 21–23 and §6.4 parts 4/6, rows R-651..R-660, four closed, capability
-  map, nightly rotation, STATUS.
+Nothing left undone. Changed: the operator's English sentence without „please"; the hold names one (the newest)
+whole copy; Part A's second failing update was not a failing edge (R-665); R-653/R-656 unit-tested only.

-## Red-proofs (each seen failing)
-Controller 8 (see `felhom-controller/REPORT.md`); catalog gate 3 (`B2-gate-redproofs.txt`); R-612/R-613
-before/after on 9202 through the product.
+## Baselines → ends

-## Gates
-controller: `go test -count=1 ./...` rc 0, `controller_gates.py` OK. catalog: `catalog_gates.py --fast` OK,
-`test_gate_decoys.py` 80 OK, `test_ladder_writer.py` OK, `test_catalog_gates.py` OK. felhom.eu: `repo_gates.py`
-(run at commit). `unproven.py --summary`: not walked 35 of 55 (unchanged).
+| repo | start | end |
+|---|---|---|
+| felhom-controller | `80e6ad8c4772` v0.267.0 | `206b035` v0.268.0 |
+| felhom.eu | `3e58c184f62e` hub v0.121.0 | hub v0.122.0 (`e4d45a8`, manifest `500488c`) + docs |
+| app-catalog-felhom.eu | `585a7cba22cf` | `5ed599c` |
+| felhom-agent | `d9864a94bf62` | untouched |

-## Teardown — three layers
-Machine: 9202 apps removed through the product (0 containers/volumes/copies), kept drive folders removed by name,
-config identical to the saved copy, live catalog; bench 9401 destroyed; 9201 24 containers before/after.
-Host: `pct list` as before; template removed. Hub: not touched (no floor change).
+## Hub v0.122.0
+
+`app_hold_no_whole_copy` in `allowedEventTypes` + `operatorOnlyEvents` + `perAppCooldownEvents`; test pinning both
+registers, red-proofed. Rolled out by ArgoCD (Synced, image read back from the pod, startup line `felhom-hub 0.122.0
+starting`).
+
+## Floors
+
+0.267.0 (05:12:14Z) and 0.268.0 (06:47:13Z), each with declared MinAgent 0.131.0, read back from the hub; both demo
+boxes healthy within 30 s. `drill-r50` stays held (agent 0.129.0).
+
+## Documents
+
+`09` §3 decisions 24–25 + §6.4 part 5 SHIPPED + §6.1a note; `07` §6 whole-copy truth table; `08` §5 fourth
+suppression + the crash-loop measurement; capability map row; `CONTEXT.md`; `STATUS.md`.
+
+## Rows
+
+Closed R-658, R-659, R-660, R-651, R-653, R-656, R-40, R-663. Opened R-661, R-662, R-664, R-665, R-666, R-667.
+Narrowed R-450. Register 336 → 335 rows, 683,233 → 679,393 bytes.
@@ -1,25 +1,30 @@
 # STATUS — what works, what's broken, what's next

-**Updated 2026-09-24 (morning) — the night shift. Eleven apps moved to newer versions, each with a written test result. The chaos hour found two serious faults in the undo's way back. The new controller is NOT yet on the demo boxes: one question for you below.**
+**Updated 2026-09-24 (morning session). The fleet is on controller 0.268.0. The undo works after a restore. A stopped app's page tells the truth. A box that is several updates behind now climbs one tested step per press.**

-**Decisions I took on my own (you may reverse them):**
-1. The memory test now counts only the app's own memory, not the kernel's file cache. The cache made two healthy apps read "100 % full" with no memory kills, and would have forced bigger memory limits for no reason. The old figure is still recorded beside the new one.
-2. The catalog's push check now asks the image registry, but only for images a push moves. Without it, a tested image could change before it reaches a box. Other pushes make no network call.
+**Decisions I took on my own:** none. Two small changes to your wording, both below.

-**What I exercised.** Tests can no longer run real Docker commands on your own machine by accident. A restore now refuses a cut-off database copy before it touches anything (proven with the real restore button). Two page texts now tell the truth: where this box's full backup really goes, and how long the backup button stops the apps (about 8 minutes). Every catalog version move now needs a written test result, and a check refuses a move without one. I moved 11 apps (12 steps), each tested twice: on a test bench with 10 minutes of memory watching, and on the scratch machine through the real Update button. The HP demo box then updated three of its own apps through the real button: all three finished and answer.
+**What I did, and it worked.**
+- **The demo boxes got 0.267.0**, as you said yes to. Then, after every live test passed, they got **0.268.0**. Both boxes arrived healthy about 15 seconds after each change.
+- **The undo after a restore.** A restore used to rebuild an app's storage without the tag the undo looked for, so the undo copied nothing. Now the undo finds the storage by its name. A restore also puts the tag back. Tested on the scratch machine with an app restored by the OLD version, the way customer boxes are today: the failed update was undone, and data written before and after the restore came back.
+- **A stopped app's page (your option A).** When no copy on the box can bring an app back with its files, the page now says so, says support is informed, and shows no restore button. You get an urgent alarm. Tested by repeating last night's exact case, in Hungarian and English.
+- **Small fixes.** A stopped app no longer raises a second, false alarm. A removed app leaves no old files behind. The test bench now marks a memory test with no real load as "not proven", and it starts every run with an empty drive folder.
+- **The update ladder.** One press now does one tested step, never a jump. Tested with RomM on the scratch machine: first press moved only the app, second press moved only the database engine. The data came back after each press. The page says how many steps remain.

-**What broke, and whether it healed.**
- **After a restore, the automatic undo has nothing to put back.** A restore rebuilds an app's storage without the tag the undo uses to find it. The next failed update is then "undone" with no data copy. The app I tested happened to survive this. Not fixed tonight (one controller release per night). Serious.
- **A stopped app can be pointed at a restore that refuses it.** After a failed update and a failed undo, the page names a backup to restore. For an app with files on a drive, the restore refuses that backup, and on a box with no off-site copy nothing else brings the app back. The stop rule fired here; all chaos rounds had already run. Serious.
- Adventurelog's new version was not moved: our own template checks it the wrong way, and the new version needs an internet download at every start.
- Reinstalling Nextcloud over kept files never finishes, and the box only says "unhealthy".
- Smaller: a stopped app raises a second, extra alarm; the memory test once ran with no real load (fixed and re-run).
+**Your wording, changed in two small ways.**
+- English: I removed the word "please". The house rule for English copy forbids it.
+- Hungarian: I kept your text exactly. It uses the formal "Ön" form; the rest of the screens use the informal form. That mismatch is already on the list.

-**Rows.** Ten opened, four closed. The list went from 330 to 336. Wishlist and uptime-kuma were fixed in the catalog; their rows stay open, smaller.
+**What I found (new, not fixed).**
+- **The second drive cannot bring back an app with files, even though it holds everything.** Its restore button refuses such apps, and the file restore brings back only files, not the database. So for Nextcloud, Immich, Paperless and Calibre, only the off-site copy counts. This needs your decision.
+- **A long crash loop never raises an alarm** if the app looks "up" for a moment between crashes. Seen on the scratch machine: 385 restarts, zero alarms.
+- The stopped-app sentence says "do not remove the app", but the Remove button is still there. This needs your word.
+- Smaller: a ladder step has no health check of its own yet; right after a restore, an update may be judged with the restored health check; one restore option ("database only") is described in the code but offered nowhere.

-**What needs you — one question.** Should the demo boxes get controller 0.267.0 now?
- **Yes, raise the floor (my recommendation).** The two serious faults are also in the version the boxes run today. 0.267.0 does not cause them, and it makes restores safer.
- **No, wait.** Nothing changes on the boxes. The fixes above stay on the scratch machine only, until you say so.
-If you do nothing, the demo boxes stay on 0.266.0.
+**Rows.** Seven closed, six opened. The list went from 336 to 335.

-**Nothing on Peti's machine, the off-site box or your own machine (beyond normal pushes) was touched. The test bench was deleted. The scratch machine is back on the real catalogue with its standing apps.**
+**What needs you — two questions.**
+1. **The second drive and apps with files.** (a) Build a "files plus database" restore for the second drive (my recommendation — then the second drive is a real way back, as customers would expect). (b) Rule that the second drive is files-only for these apps. If you do nothing, the page keeps naming only the off-site copy for these apps, which is true.
+2. **The Remove button on a stranded app.** (a) Hide it while support is informed (my recommendation — a removal destroys what support needs). (b) Keep it and soften the sentence. If you do nothing, both stay as they are.
+
+**Not done.** No automatic updates yet (plan part 7). No customer box touched by hand. Nothing on Peti's machine, the off-site box or your own machine was touched, beyond the normal hub update. The scratch machine is back on the real catalog with its standing apps.
@@ -307,6 +307,18 @@ the unit, the Tier-2 mirror AND the Tier-2 record, so „Teljes visszaállítás
 (R-486); „Mentési adatok törlése" deletes the unit, every mirror and the app's backup preferences, and
 never touches off-site snapshots (R-474). A removed app whose unit was kept is listed on both local backup pages with its restore since controller v0.242.0 (R-487): **the local lists are keyed on the drives, not on what is deployed** — the rule R-237 set for the off-site list — and the restore opens the unit where it sits, a data drive included. **For a bind-data app the update's tier order is second drive → off-site → own unit (R-479, v0.241.0), because the unit does not hold the files.**

+**[FACT, measured from source 2026-09-24, controller v0.267.0/v0.268.0] Which copy brings an app back WHOLE
+— read from the restores' own refusals, not from what the tier stores** (`09` §3 decision 25, R-659):
+
+| app | own unit (Tier 1) | second drive (Tier 2) | off-site (Tier 3) |
+|---|---|---|---|
+| no declared drive files (class B, 45 apps) | whole — the unit restore | whole — „Teljes visszaállítás" (the unit restore) | whole — the full restore |
+| declared drive files (`DeclaredDriveFileLegs`: calibre-web, immich, nextcloud, paperless-ngx) | **not** — refused (R-538) | **not** — its unit restore is refused by the same guard; the file restore only ADDS missing files (R-661) | whole — „Teljes visszaállítás (fájlok + adatbázis)" |
+
+`backup.WholeOnTier` is this table; `TestR659_TruthTableAgreesWithTheRestoresRefusal` pins that it cannot drift
+from the refusal. An app with only OPTIONAL legs (audiobookshelf, komga, romm) is class-B here: its files are
+never moved by an update or an undo, and the unit restore accepts it.
+
 ### 6.1 The four tiers, as configured on the live fleet

 | Tier | Location | Captures | Cadence (LIVE) | Retention (LIVE) | Encrypted |
@@ -80,7 +80,7 @@ is allowed; every down member then reads as supervised.
 | `stopped`, `exited` | **yes** | not running, will not recover alone |
 | `degraded` | **yes** | [DESIGN, R-51] a dead supervised member is as unreachable as a single app that exited — immich-server sat Exited 18 h with the app 100 % dead and no alert |
 | `unhealthy` | **NO** | [DESIGN] a running container whose healthcheck is failing. Folding it in reintroduces the flapping fix-3 was written to stop. **R-384 did not change this** — it asks a prior question instead |
-| `restarting` | **NO**, until sustained | [DESIGN, C9-F2] `restarting` is on the normal deploy path, so folding it in would alarm fleet-wide on every update. Becomes down after **5 min** (`crashLoopAfter`) |
+| `restarting` | **NO**, until sustained | [DESIGN, C9-F2] `restarting` is on the normal deploy path, so folding it in would alarm fleet-wide on every update. Becomes down after **5 min** (`crashLoopAfter`) of UNINTERRUPTED `restarting` — **measured 2026-09-24: a container that reads `running` for a moment between restarts resets the clock and never gets there (gokapi, 385 restarts, „0 currently down"; R-667)** |
 | `starting`, `deploying` | no | mid-start |
 | `paused` | no | a deliberate user action |
 | `unknown` | no | [DESIGN] fail-OPEN — never manufacture a dead-app alert from an inconclusive read |
@@ -91,13 +91,14 @@ member is dead; only the excuse is missing). Both are recorded at their sites.

 ---

-## 5. The three suppressions, all at `classifyRunStates`
+## 5. The four suppressions, all at `classifyRunStates`

 | Suppression | Rule | Expires? |
 |---|---|---|
 | **deliberate user stop** | `StateStopped` is not down **unless** the quiesce loop reports it failed to restart that stack | n/a — lifted by `failedRestart` |
 | **quiesce cycle** | a stack this backup cycle stopped is exempt | **yes**, 180 s after unquiescing |
 | **boot grace** | no evaluation for 90 s after controller start | **yes** |
+| **update hold** (v0.268.0, R-660) | an app HELD after a failed update is stopped by the product and has its own event (`app_update_held`, and `app_hold_no_whole_copy` when no copy brings it back whole); it is not "down" | **yes** — lifted with the hold (a restore, or the operator). A RESTORE hold (R-379) is deliberately not in the set: it has no event of its own |

 **None of them latch.** [DESIGN, R-97b + R-88 Scenario D] A permanent suppression trades a loud false
 alarm for a silent real one, which is the same error as an over-eager alarm, in the opposite
@@ -311,6 +311,24 @@ builds them (`audits/update-rulings-2026-09-23/`).
    decision 17 says the box pulls the recorded digest; recording one the registry no longer serves makes
    every box's pull of that step fail (Scenario E). `app-catalog-felhom.eu/scripts/check-test-record-move.py`.

+### 2026-09-24 — two operator rulings
+
+24. **The fleet takes controller v0.267.0 although the chaos hour's stop rule fired** — both faults the
+    rule caught (R-658, R-659) predate that release; v0.267.0 causes neither and adds R-640's protection to
+    the same restore path. Floor saved with MinAgent 0.131.0 at 05:12:14Z; both demo boxes on v0.267.0,
+    healthy, 10 s later (`audits/ladder-2026-09-24/part0/`).
+
+25. **R-659, option A: a held app's page names only a copy that can bring the app back WHOLE; when this box
+    has none, the page says so and that support is informed, and the operator gets an urgent event.** The
+    database-only restore under existing files (option B) is NOT built. *As built (v0.268.0 + hub
+    v0.122.0):* "whole" is read from the restores' OWN refusals, not from what a tier stores — measured from
+    source, an app with declared drive files (`DeclaredDriveFileLegs`) is refused by both the own-unit and the
+    second-drive unit restore (R-538's guard sits in the shared function), so for those apps only the
+    off-site copy counts (the second-drive gap is R-661). The hold names the newest whole copy with what it
+    holds; with none, `hold.update.no_whole_copy`, no Mentések button, `app_hold_no_whole_copy` (critical,
+    operator-only). The operator's English was used with one word dropped („please" — the house rule,
+    `i18n_missing_gate.py`); the Hungarian verbatim, its formal register recorded against R-516.
+
 **RomM follow-ups, operator-agreed the same day:** the test bench watches memory after an update
 (`upgrade-test.py`, 2026-09-23); a version move checks the memory limit (gate or checklist — §6.4);
 R-636's louder repeated alarm.
@@ -842,7 +860,9 @@ an app pinned before v0.263.2 has no such record until its next pin. **Two more,
 2026-09-23 night (`audits/DRILL-night-2026-09-23.md` Part D):** the undo finds an app's volumes by their compose
 label, and a restore recreates them WITHOUT it — so after any restore the undo copies nothing (R-658, P1); and a
 held file-leg app is pointed at a restore that refuses a database-only copy, with no route left on a box without
-an off-site tier (R-659, P1). A controller kill during `verifying` resumed and undid correctly (round 9).
+an off-site tier (R-659, P1). A controller kill during `verifying` resumed and undid correctly (round 9). **Both fixed in v0.268.0 (2026-09-24):** the undo selects volumes from the app's compose definition and the restore
+labels what it creates (R-658); the hold names only a copy that brings the app back whole, else says support is
+informed (R-659, decision 25). Proven live on 9202, `audits/ladder-2026-09-24/`.

 The spike, as it was run by hand before any build:

@@ -1081,7 +1101,7 @@ what the part can do to a household's data if it is wrong, not how likely that i
 | **2** | **SHIPPED — controller v0.264.0 + hub v0.120.0, proven live on 9202 and 9201 2026-09-23** (`audits/undo-fleet-2026-09-23/`). **The update sentences in the household's language** (R-606) and a mail when an automatic update is undone or held: events `app_update_undone` (warning) and `app_update_held` (error), one per app per outcome, on by default, mailed in the household's language with the app named in the subject; per-app cooldown on both legs. Leftovers: R-647. | R-606, 15 | **1** | — | none |
 | **3** | **SHIPPED — controller v0.264.0, proven live on 9202 2026-09-23.** **A disabled notifier says so** (R-620), so the mail of part 2 can be measured on a scratch box at all. | R-620 | **0.5** | — | none |
 | **4** | **SHIPPED — catalog `6db08a5`, night 2026-09-23** (`audits/DRILL-night-2026-09-23.md`): `update_ladder:` in `.felhom.yml`, one JSON entry per line (spiked on controller v0.266.0 and v0.267.0 first — the controller ignores the key); `check-test-record.py` (static, CI) + `check-test-record-move.py` (history + the registry for moved refs only, decision 23); the only writer `upgrade-test.py --write-ladder`; the 21 moves of 2026-09-22 backfilled from their records (21 proven). Not built: `steps/<to>.yml` (part 5 needs it once an app has two steps); `CompareImageRefs` did NOT move to the gate — the gate asks for a proven test instead, which is decision 13's own test. **The test record + the catalog gate + the memory check.** The harness writes the ladder entry (below) from its verdict record, including the memory watch's peak and marks; the gate refuses an image move with no entry, an entry with a `failed` verdict, or one with no memory watch; `CompareImageRefs`' rule moves here as the push-time safety net. **Backfill:** one entry per current pin — the 21 proven moves from their records, every other pin `needs_person: "never tested"`, which is honest and keeps them manual. A version move re-checks `mem_limit` against the watch's peak (the RomM follow-up: gate, not checklist, because the watch now produces the number). | 13, R-635 follow-up | **2.5** | the memory watch (shipped 2026-09-23) | none on a box — catalog-side only |
-| **5** | **The ladder on the box.** Read `update_ladder:` from the clone, find the installed step, apply ONE step with its OWN definition (`steps/<to>.yml`, the last step the current template), repeat next night; a failed step stops the ladder for that app. `CatalogOrder` compares refs with the digest stripped (see part 7). | 14 | **2.5** | 1, 4 | medium — each step is the guarded update + undo; the new risk is rendering the wrong step's definition, pinned by a test per step shape |
+| **5** | **SHIPPED — controller v0.268.0 (`206b035`) + catalog `5ed599c`, proven live on 9202 2026-09-24** (`audits/ladder-2026-09-24/partD/`): romm 5.3.0/11.4 → 5.3.1/11.4 (its app step, from `steps/90dd9d68258286ef.yml`) → 5.3.1/11.8 (the engine step; `mariadb-upgrade` ran) in two presses, the seeded account read back after each, the page's „Hátralévő frissítési lépések" 2 → 1 → none. Step files: `templates/<app>/steps/<StepKey(to)>.yml` (sha256 of `to` as canonical JSON, 16 hex; `stacks.StepKey` = `ladder.step_key`), refused absent or wrong by `check-test-record.py` rule 4, written by `--write-ladder` when a step is superseded, 8 backfilled from the NEWEST commit naming each step's images (romm's from `f4eb94f`, not the OOM-looping `15f9ebf`). The box reads them from its `--depth 1` clone (the whole tree is there — measured). One press = one step; a missing step file refuses before anything moves; an installed version matching no entry jumps, logged by name (measured live: vikunja 2.5.0). Limitation: a step has no `.felhom.yml` of its own (R-664). **The ladder on the box.** Read `update_ladder:` from the clone, find the installed step, apply ONE step with its OWN definition (`steps/<to>.yml`, the last step the current template), repeat next night; a failed step stops the ladder for that app. `CatalogOrder` compares refs with the digest stripped (see part 7). | 14 | **2.5** | 1, 4 | medium — each step is the guarded update + undo; the new risk is rendering the wrong step's definition, pinned by a test per step shape |
 | **6** | **CATALOG HALF SHIPPED — night 2026-09-23:** every ladder entry carries the digest per `to` ref (`scripts/image_digest.py`, stdlib; equals Docker's `RepoDigests` on a box), and the move gate refuses a digest the registry no longer serves. **The box half (compare, render `name:tag@sha256`) is not built.** **Digests.** The catalog records `sha256` per pin at push time (`check-image-resolvable.py` already resolves it); the box compares it for the badge and renders `name:tag@sha256:…` when present. **Measured 2026-09-23 on 9202:** Docker and Compose both pull and run `redis:7-alpine@sha256:858f…`, and refuse a digest that does not exist (`audits/update-rulings-2026-09-23/70-…`). **Build trap, read from source:** `splitImageRef` returns "unorderable" for ANY ref containing `@` (`updateorder.go:134`), so the digest must be split off before ordering or every digest-pinned app reads Unknown. A digest gone upstream fails the PULL — Scenario E, pin back, nothing ran. | 17, R-446 | **2** | 4 (the entry carries the digest) | low |
 | **7** | **The update leg in the chain + the automatic caller + the switch.** A leg that starts when the off-site leg has FINISHED (legs are clock-scheduled today, not chained — a completion signal is new), one app at a time (there is no single-flight, §3b Q4), `app_update.unattended` default ON, `stacks.update_window` removed, reads `UpdateRefusal.Reason`, remembers a failed step so it never re-presses it. **See the one open point below.** | 11, 12 | **3** | 1, 2, 5 | medium — the only part that acts with nobody watching; everything above is what makes it safe |
 | **8** | **SHIPPED — controller v0.265.0 + hub v0.121.0, proven live on 9202 2026-09-23** (`audits/cleanup-2026-09-23/`; a kernel `oom_kill` counter, not the sticky flag — `08` §6.2). **R-636** — the same OOM key re-firing escalates instead of staying one `warning` for six hours. | R-636 | **1** | — | none |
@@ -0,0 +1,106 @@
+# 2026-09-24 — the fleet to 0.267.0 and 0.268.0; the undo after a restore; option A; the ladder on the box
+
+Architecture read first and named: `architecture/09-update-architecture.md` (§3 decisions 13–25, §6.1, §6.1a,
+§6.4 parts 4–5), `07-backup-architecture.md` §6, `08-alarm-ladder.md` §5. **Method: endpoint-level** — every
+product act is the endpoint the UI invokes; pages fetched as HTML in both languages (`?lang=`); no browser.
+Scratch guest **9202** on demo-hp (Tier 0), drill catalog for every failing edge; the live catalog carried only
+the step files and gates.
+
+## Not done, or changed
+
+- **Nothing not done.** Parts 0, A, B, C, D and E all ran; the interim "one step per app per day" gate was not
+  needed (Part D shipped).
+- **The operator's English sentence lost one word, „please"** — the house rule (`i18n_missing_gate.py`,
+  `EN_FORBIDDEN`) refuses it. The Hungarian is verbatim; its formal register raised the R-516 ceiling 18 → 19 with
+  the reason written beside it.
+- **The hold names ONE copy (the newest whole one), not a list.** A list would need new sentence copy; the existing
+  sentence names one copy with what it holds.
+- **Part A's second failing update (g) did not fail:** right after a restore the stack dir's `.felhom.yml` was the
+  restored one (real probe port), so that press was a good update. The proof the brief asks for is (e) — an app
+  restored under v0.267.0, then a failing update on v0.268.0 — and it passed. (g) still shows the v0.268.0 restore's
+  volumes copied with no "unlabelled" warning. Filed as R-665 (not diagnosed).
+- **R-653 and R-656 are unit-tested and red-proofed, not run on a bench** — no bench was provisioned this session.
+- **Two catalog test suites were already red at the start** (`test_gate_decoys.py`, `test_ladder_writer.py` — last
+  night's pin moves under literals). Fixed and filed as R-663 (closed).
+- **One of the first R-660 control's two alarm events is not attributable** (the dropped-event line names no app);
+  the repeat control produced exactly one. Likely gokapi (R-667), stated as inference.
+- An older wiring test (`TestR475_AdapterReadsEveryTier`) asserted the adapter hands the hold the precondition copy
+  — the thing R-659 changes on purpose. Its assertion was replaced, with the reason in the test.
+
+## Claims in the brief, checked
+
+| the brief's claim | what was found |
+|---|---|
+| compose resolves volume names as `<project>_<volume>` for every template (some set `name:`) | **No template sets `name:`**, top-level or per volume (all 53 read). Supported anyway, and tested. |
+| the syncer can carry a `steps/` folder | **It does not need to.** The syncer copies two files into the stack dir; the box reads `steps/` from its catalog CLONE, which (depth 1) holds the whole tree. Measured on 9202: `steps/spike-B.yml` present in the clone after one sync (`partD/01`). |
+| „saját meghajtó" never holds files for a file-leg app (from `07` §6) | **Held, and it goes further:** read from source, the SECOND-DRIVE unit restore refuses such an app too (the R-538 guard is in the shared function), and the second drive's file restore brings back no database. So only off-site brings a file app back whole (R-661). An app with only OPTIONAL legs (romm, komga, audiobookshelf) is accepted by the unit restore. |
+| one press = one step is safe for an engine step whose datadir converts | **Held for MariaDB, measured once:** romm press 2 ran `mariadb-upgrade` 11.4.13 → 11.8.9 and the account read back. The undo copy covers the pre-conversion volume if the step fails (not exercised on an engine step this session). PostgreSQL majors stay refused (part 10). |
+| today's press jumps A → C | **Held** on v0.267.0: vikunja 2.4.0 → 2.6.0 in one press, 2.5.0 never ran (`partD/00-spike.json`). |
+
+## Part 0 / Part E — the floor
+
+| save (UTC) | floor / MinAgent | read back | demo-felhom | demo-hp |
+|---|---|---|---|---|
+| 05:12:14 | 0.267.0 / 0.131.0 | `min_controller_version=0.267.0`, `min_agent=0.131.0`; hub `SERVED … from declared` | 0.267.0 healthy at +28 s | same |
+| 06:47:13 | 0.268.0 / 0.131.0 | same shape | 0.268.0 healthy at +30 s | same |
+
+Hub 0.122.0 rolled out first (ArgoCD, manifest `500488c`, pod image read back). **Stays below:** `drill-r50`
+(DOWN, agent 0.129.0 — held by MinAgent, a Tier-0 leftover). Evidence `part0/`.
+
+## Red-proofs (each seen failing, then restored)
+
+| row | mutation | failed at | file |
+|---|---|---|---|
+| R-658 undo | `appVolumes` returns the label selector | `copied 0 of 2 declared volumes` | `redproofs/A-r658-undo.txt` |
+| R-658 restore | create without labels | `created vikunja_files WITHOUT the project label` | `A-r658-restore.txt` |
+| R-658 remove report | `appVolumeSet` = labelled only | `volumes_removed = []` | `C-r651-and-A-remove-report.txt` |
+| R-651 | skip deleting the applied files | both files survive; undo would probe the REMOVED install's file | same |
+| R-659 hold | `WholeOnTier` always true | round-11 case names „saját meghajtó" | `B-r659-hold.txt` |
+| R-659 page | gate dropped from `stacks.html` | Mentések button beside a no-whole-copy hold | `B-r659-page.txt` |
+| R-659 hub | dropped from `operatorOnlyEvents` | "must be operator-only" | `B-hub-operator-only.txt` |
+| R-660 | `updateHeldSet` returns nil / union line removed | "held app reported DOWN" / wiring | `C-r660.txt` |
+| ladder (box) | `nextLadderStep` always the template | press 1 pinned C; C brought up after B failed; missing step file → `done` | `D-ladder.txt` |
+| ladder (catalog gate) | rule 4 removed | 3 decoys pass wrongly | `D-catalog-gate.txt` |
+| ladder (writer) | STEP block off | "has no definition" / file absent | `D-catalog-writer.txt` |
+| R-653 / R-656 | `load_verdict` always reached / no removal | ghost's all-`err` watch reads reached; last run's config survives | `C-r653-r656.txt` |
+
+Green: controller `go build/vet/test` rc 0 (31 packages); `controller_gates.py` all OK; hub `go test` OK +
+`repo_gates.py` OK; catalog `catalog_gates.py` OK, decoys 84 OK, writer 6 OK, bench 5 OK.
+
+## Live proofs on 9202
+
+- **A (R-658)** — v0.267.0: vikunja installed, a pull-failing update took its own backup, restore from „helyi" →
+  both volumes' labels `null` (`partA/01`). v0.268.0: failing update (2.5.0 → 2.6.0, probe 8999) → log
+  *„carry no compose label … copied by name"*, *„will hold 2 named volume(s)"*, undone in 7 s, seeds A (before) and
+  B (after the restore) read back. v0.268.0 restore → labels `project`/`volume`/`version`, no compose warning
+  (`partA/02`). Remove → `volumes_removed` names both.
+- **B (R-659)** — nextcloud installed at ladder step 1, seeded; failing update with the undo copy's marker removed
+  during `verifying` → HELD, `hold_no_whole_copy: true`; the sentence in hu and en exactly as ruled (ASCII
+  fragments, positive and negative control); no Mentések button on the app page or the list, both languages; log
+  *„NO copy on this box brings it back whole (seen: tier 2 …, tier 1 …; drive files declared: true)"* and
+  *„DROPPED event app_hold_no_whole_copy (severity critical)"* (R-620's line). Steps line „… lépések: 1"
+  (`partB/01`).
+- **C (R-660, R-651)** — dead-app heartbeat 2 s after the hold: *„0 currently down"*; no `app_start_failed` for
+  the held app in ~12 min; a throwaway stopped out of band raised one (`partB/03`, `06`). Remove: `applied-compose.yml`
+  and `applied-meta/` present before, gone after; every undo copy gone (`partB/07`).
+- **D (the ladder)** — romm installed at 5.3.0/11.4, seeded; the page read „Hátralévő frissítési lépések: 2" /
+  "Update steps remaining: 2". Press 1 → *„ladder — step 2 of 3 … from 90dd9d68258286ef.yml"*, pin 5.3.1/11.4,
+  applied == that step file, MariaDB „upgrade not required", account read back, count 1. Press 2 → *„the last step
+  (3 of 3)"*, pin 5.3.1/11.8, `mariadb-upgrade` 11.4.13 → 11.8.9, account read back, count gone (`partD/10`). A
+  second seed after press 1 was refused by RomM (403 for a non-admin) — the readback rests on seed A.
+
+## Teardown — three layers
+
+- **Machine (9202):** vikunja, nextcloud, romm, actualbudget ×2 removed THROUGH THE PRODUCT; 0 volumes, 0 undo
+  copies; drive folders nextcloud/romm removed by name (R-442 keeps them on 9202); 11 test images removed by name
+  (`/var/lib/docker` 27 % → 18 %), no prune. `controller.yaml` restored from `controller.yaml.pre-ladder0924`, git
+  URL read back as the live catalog, cache at live `5ed599c`. **9202 stays on v0.268.0.** Standing apps as found
+  (gokapi was crash-looping before the session — R-644/R-667).
+- **Host (demo-hp):** no guest created; nothing else touched.
+- **Hub:** no customer or appliance record created. Floor 0.268.0 is the intended end state.
+- **Drill repo:** reset to live `main` `5ed599c`, `has_actions: False`, private — read back.
+
+## Rows
+
+Closed: R-658, R-659, R-660, R-651, R-653, R-656, R-40 (superseded), R-663 (filed and fixed). Opened: R-661, R-662,
+R-664, R-665, R-666, R-667. Narrowed: R-450 (part 5 shipped). Register **336 → 335 rows, 683,233 → 679,393 bytes**.
@@ -0,0 +1,10 @@
+2026/09/24 07:29:33 [INFO] host-report from demo-hp-bb76ea (1 guests, 4 storage targets, 2 backups, 2 restore-tests, 2 pbs-snapshots, 14808 bytes)
+2026/09/24 07:29:33 [INFO] DR-recipe host-half stored for customer demo-hp (host demo-hp-bb76ea, v1)
+2026/09/24 07:31:48 [INFO] Offsite pool-box refreshed: 0.3% full (3.0 GB of 1.00 TB), Σ shared quota 250 GB, oversub 0.24x
+2026/09/24 07:32:46 [INFO] PBS-DR box refreshed: 16.1% full (15.7 GB of 97.9 GB)
+2026/09/24 07:33:27 [INFO] wgsync: pushed 4 peers to 167.233.158.164:22
+2026/09/24 07:34:43 [INFO] felhom-hub 0.122.0 starting
+2026/09/24 07:34:43 [INFO] Default controller-version floor: 0.120.0
+2026/09/24 07:34:45 [INFO] Gitea artifact browser enabled (Day-0 version dropdowns) via http://gitea.gitea-system.svc.cluster.local:3000
+2026/09/24 07:34:45 [INFO] Registry version checker started (every 6h)
+2026/09/24 07:34:45 [DEBUG] Registry version check: latest = 0.267.0
@@ -0,0 +1,15 @@
+privatebin privatebin/pdo:2.0.6 Up 5 hours (healthy)
+paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 Up 5 hours (healthy)
+paperless-postgres postgres:16-alpine Up 5 hours (healthy)
+paperless-redis redis:7-alpine Up 5 hours (healthy)
+gokapi f0rc3/gokapi:v1.9.6 Restarting (1) 45 seconds ago
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.267.0 Up 7 hours (healthy)
+filebrowser gtstef/filebrowser:1.3.3-stable Up 9 hours (healthy)
+traefik traefik:v3.6.7 Up 9 hours
+/dev/loop1       69G   18G   48G  27% /var/lib/docker
+git:
+    branch: main
+    repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
+    sync_interval: 15m
+    token: <redacted>
+actualbudget adventurelog audiobookshelf bentopdf bookstack calcom calibre-web claper code-server crafty-controller docmost emby filebrowser ghost gitea glance gokapi grafana gramps-web home-assistant homebox homepage immich jellyfin kimai komga mealie n8n navidrome nextcloud onlyoffice opengist outline paperless-ngx papra plant-it plex privatebin radarr rallly recipe-importer romm seerr sonarr sparkyfitness tandoor termix traefik uptime-kuma vaultwarden vikunja wanderer wger wishlist zipline 
@@ -0,0 +1,8 @@
+gitea.dooplex.hu/admin/felhom-controller:0.268.0
+gitea.dooplex.hu/admin/felhom-controller:0.268.0 Up 6 seconds (healthy)
+2026/09/24 06:16:41 pin.go:290: [INFO] [stacks] pin adoption: 0 pinned, 4 already pinned, 0 left unpinned (0 not completely observed, 0 running something the template no longer offers)
+2026/09/24 06:16:41 undo.go:656: [INFO] [stacks] applied-meta backfill (R-646): recorded 0 []; skipped 0 [] — not current with the catalog, their pinned version's .felhom.yml is no longer on the box
+2026/09/24 06:16:41 main.go:527: [INFO] Metrics collector started (60s interval)
+2026/09/24 06:16:41 scheduler.go:67: [DEBUG] [scheduler] scheduler started: periodic=8 daily=8
+2026/09/24 06:16:46 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event controller_started (severity info) — further controller_started events are logged at DEBUG only
+
@@ -0,0 +1,6 @@
+gitea.dooplex.hu/admin/felhom-hub:0.122.0
+2026-09-24T06:47:13Z
+HTTP/1.1 303 See Other
+Location: /configuration?flash=floor_set
+name="min_agent" value="0.131.0"
+name="min_controller_version" value="0.268.0"
@@ -0,0 +1,6 @@
+06:47:26 felhom-pve=gitea.dooplex.hu/admin/felhom-controller:0.268.0|Up 3 seconds (health: starting) demo-hp=gitea.dooplex.hu/admin/felhom-controller:0.268.0|Up 3 seconds (health: starting)
+06:47:43 felhom-pve=gitea.dooplex.hu/admin/felhom-controller:0.268.0|Up 20 seconds (healthy) demo-hp=gitea.dooplex.hu/admin/felhom-controller:0.268.0|Up 20 seconds (healthy)
+both-arrived
+2026/09/24 08:47:14 [INFO] Global controller-version floor set to "0.268.0" (declared MinAgent "0.131.0")
+2026/09/24 08:47:16 [INFO] managed floor SERVED for demo-felhom: floor 0.268.0, agent requirement "0.131.0" from declared (golden 0.258.0)
+2026/09/24 08:47:17 [INFO] managed floor SERVED for demo-hp: floor 0.268.0, agent requirement "0.131.0" from declared (golden 0.258.0)
@@ -0,0 +1,100 @@
+{
+  "controller": "gitea.dooplex.hu/admin/felhom-controller:0.267.0",
+  "drill_A": "58c7c938bc7c",
+  "deployed": true,
+  "seed_A": true,
+  "labels_after_install": "vikunja_vikunja_data {\"com.docker.compose.config-hash\":\"fe3fc6629444c7799d20fe7f96a201fcf52bd22fe1dcea326288a2a8963b48e0\",\"com.docker.compose.project\":\"vikunja\",\"com.docker.compose.version\":\"5.5.0\",\"com.docker.compose.volume\":\"vikunja_data\"}\nvikunja_vikunja_db {\"com.docker.compose.config-hash\":\"9c995441cdf40d38f9a48997f7376d7d20f88e50557ea8bca04237eeb3b2c9d1\",\"com.docker.compose.project\":\"vikunja\",\"com.docker.compose.version\":\"5.5.0\",\"com.docker.compose.volume\":\"vikunja_db\"}\n",
+  "drill_E": "ef42b4f34455",
+  "badge_wait": 22.8,
+  "press_backup": {
+    "accepted": true,
+    "http": "202",
+    "phases": [
+      {
+        "t": 0.0,
+        "phase": "backing-up",
+        "label": "Biztonsági mentés készül a frissítés előtt…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 2.1,
+        "phase": "safety-dump",
+        "label": "Adatbázis pillanatkép…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 3.1,
+        "phase": "pulling",
+        "label": "Új verzió letöltése…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 4.1,
+        "phase": "failed",
+        "label": "A frissítés nem sikerült",
+        "updating": false,
+        "error": "Az új verzió letöltése nem sikerült, ezért a frissítés elmaradt. Az alkalmazás a korábbi verzióval fut tovább.",
+        "hold": null
+      }
+    ],
+    "duration_s": 4.1,
+    "final_phase": "failed",
+    "update_error": "Az új verzió letöltése nem sikerült, ezért a frissítés elmaradt. Az alkalmazás a korábbi verzióval fut tovább.",
+    "hold_reason": null,
+    "state": "running"
+  },
+  "snapshots": [
+    {
+      "time": "2026-09-24T06:11:05Z",
+      "short_id": "helyi",
+      "tier": 1,
+      "drive_label": "Belső SSD (rendszer)"
+    }
+  ],
+  "drill_back": "087b3edb7475",
+  "restore": {
+    "ok": true,
+    "snapshot_id": "helyi",
+    "snapshots": [
+      {
+        "time": "2026-09-24T06:11:05Z",
+        "short_id": "helyi",
+        "tier": 1,
+        "drive_label": "Belső SSD (rendszer)"
+      }
+    ],
+    "http": "HTTP/2 302",
+    "location": [
+      "location: /backups/restore?flash=flash.restore.started"
+    ],
+    "seconds": 6.1,
+    "state_after": "running",
+    "hold_after": null,
+    "observables_after": {
+      "pinned_images": {
+        "vikunja": "vikunja/vikunja:2.5.0"
+      },
+      "installed_images": {
+        "vikunja": "vikunja/vikunja:2.5.0"
+      },
+      "catalog_images": {
+        "vikunja": "vikunja/vikunja:2.5.99-notag"
+      },
+      "live_compose_image_lines": [
+        "image: vikunja/vikunja:2.5.0"
+      ],
+      "docker_inspect": [
+        "vikunja  vikunja/vikunja:2.5.0  running=true  restarts=0"
+      ]
+    }
+  },
+  "labels_after_restore_v0267": "vikunja_vikunja_data null\nvikunja_vikunja_db null\n",
+  "A_after_restore": true,
+  "seed_B_after_restore": true
+}
@@ -0,0 +1,30 @@
+08:10:26 drill 58c7c938bc7c: vikunja 2.5.0 (the install version) push rc=0
+08:10:30   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+08:10:35   [1] deployed, controller state=running, pinned={'vikunja': 'vikunja/vikunja:2.5.0'}
+08:10:36     vikunja: register http=200
+08:10:36     vikunja: create project http=201
+08:10:39 labels after install:
+vikunja_vikunja_data {"com.docker.compose.config-hash":"fe3fc6629444c7799d20fe7f96a201fcf52bd22fe1dcea326288a2a8963b48e0","com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_data"}
+vikunja_vikunja_db {"com.docker.compose.config-hash":"9c995441cdf40d38f9a48997f7376d7d20f88e50557ea8bca04237eeb3b2c9d1","com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_db"}
+
+08:10:40 drill ef42b4f34455: vikunja 2.5.99-notag (a tag that does not exist: backing-up runs, the pull fails, nothing moves) push rc=0
+08:11:03   [sync] the badge needed 22.8s and 4 sync+rescan rounds to catch up to vikunja/vikunja:2.5.99-notag — R-607's window, measured
+08:11:03   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:11:03       +   0.0s  phase=backing-up  label=Biztonsági mentés készül a frissítés előtt…  err=None  hold=None
+08:11:05       +   2.1s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:11:06       +   3.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:11:07       +   4.1s  phase=failed  label=A frissítés nem sikerült  err=Az új verzió letöltése nem sikerült, ezért a frissítés elmaradt. Az alkalmazás a korábbi verzióval fut tovább.  hold=None
+08:11:07 snapshots offered: [{"time": "2026-09-24T06:11:05Z", "short_id": "helyi", "tier": 1, "drive_label": "Bels\u0151 SSD (rendszer)"}]
+08:11:08 drill 087b3edb7475: vikunja 2.5.0 (back to the install version) push rc=0
+08:11:13   [R] restoring vikunja from snapshot 'helyi' (of 1 offered)
+08:11:13   [R] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
+08:11:13       +   0.0s restore (True, None, None)
+08:11:19       +   6.1s restore (False, None, None)
+08:11:19   [R] after restore: state=running hold=None phase=failed
+08:11:25 labels after the v0.267.0 restore:
+vikunja_vikunja_data null
+vikunja_vikunja_db null
+
+08:11:26     vikunja: readback of the seeded project http=200 ok=True
+08:11:26     vikunja: register http=200
+08:11:26     vikunja: create project http=201
@@ -0,0 +1,30 @@
+08:10:26 drill 58c7c938bc7c: vikunja 2.5.0 (the install version) push rc=0
+08:10:30   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+08:10:35   [1] deployed, controller state=running, pinned={'vikunja': 'vikunja/vikunja:2.5.0'}
+08:10:36     vikunja: register http=200
+08:10:36     vikunja: create project http=201
+08:10:39 labels after install:
+vikunja_vikunja_data {"com.docker.compose.config-hash":"fe3fc6629444c7799d20fe7f96a201fcf52bd22fe1dcea326288a2a8963b48e0","com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_data"}
+vikunja_vikunja_db {"com.docker.compose.config-hash":"9c995441cdf40d38f9a48997f7376d7d20f88e50557ea8bca04237eeb3b2c9d1","com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_db"}
+
+08:10:40 drill ef42b4f34455: vikunja 2.5.99-notag (a tag that does not exist: backing-up runs, the pull fails, nothing moves) push rc=0
+08:11:03   [sync] the badge needed 22.8s and 4 sync+rescan rounds to catch up to vikunja/vikunja:2.5.99-notag — R-607's window, measured
+08:11:03   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:11:03       +   0.0s  phase=backing-up  label=Biztonsági mentés készül a frissítés előtt…  err=None  hold=None
+08:11:05       +   2.1s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:11:06       +   3.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:11:07       +   4.1s  phase=failed  label=A frissítés nem sikerült  err=Az új verzió letöltése nem sikerült, ezért a frissítés elmaradt. Az alkalmazás a korábbi verzióval fut tovább.  hold=None
+08:11:07 snapshots offered: [{"time": "2026-09-24T06:11:05Z", "short_id": "helyi", "tier": 1, "drive_label": "Bels\u0151 SSD (rendszer)"}]
+08:11:08 drill 087b3edb7475: vikunja 2.5.0 (back to the install version) push rc=0
+08:11:13   [R] restoring vikunja from snapshot 'helyi' (of 1 offered)
+08:11:13   [R] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
+08:11:13       +   0.0s restore (True, None, None)
+08:11:19       +   6.1s restore (False, None, None)
+08:11:19   [R] after restore: state=running hold=None phase=failed
+08:11:25 labels after the v0.267.0 restore:
+vikunja_vikunja_data null
+vikunja_vikunja_db null
+
+08:11:26     vikunja: readback of the seeded project http=200 ok=True
+08:11:26     vikunja: register http=200
+08:11:26     vikunja: create project http=201
@@ -0,0 +1,205 @@
+{
+  "controller": "gitea.dooplex.hu/admin/felhom-controller:0.268.0",
+  "probe_port_real": 3456,
+  "labels_before": "vikunja_vikunja_data null\nvikunja_vikunja_db null\n",
+  "drill_e": "17a5e25f99bc",
+  "badge_e": 4.5,
+  "press_e": {
+    "accepted": true,
+    "http": "202",
+    "phases": [
+      {
+        "t": 0.0,
+        "phase": "safety-dump",
+        "label": "Adatbázis pillanatkép…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 1.1,
+        "phase": "pulling",
+        "label": "Új verzió letöltése…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 3.1,
+        "phase": "copying",
+        "label": "Az adatok másolása a frissítés előtt…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 4.1,
+        "phase": "verifying",
+        "label": "Működés ellenőrzése…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 94.3,
+        "phase": "undoing",
+        "label": "Visszaállítás az előző változatra…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 102.5,
+        "phase": "undone",
+        "label": "Visszaállítva az előző változatra",
+        "updating": false,
+        "error": null,
+        "hold": null
+      }
+    ],
+    "duration_s": 102.5,
+    "final_phase": "undone",
+    "update_error": null,
+    "hold_reason": null,
+    "state": "running"
+  },
+  "log_e": "2026/09/24 06:17:27 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition\n2026/09/24 06:17:27 undo.go:387: [WARN] [stacks] update vikunja: volume(s) [vikunja_vikunja_data vikunja_vikunja_db] carry no compose label (recreated by a restore before v0.268.0 — R-658) — copied by name\n2026/09/24 06:17:28 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB\n2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T061729Z in 431ms\n2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T061729Z in 428ms\n2026/09/24 06:19:01 undo.go:501: [WARN] [stacks] update vikunja: UNDO — putting back the previous version and its 2 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))\n2026/09/24 06:19:09 undo.go:549: [INFO] [stacks] update vikunja: UNDONE in 7s — the previous version is running on the data from before the update (the app's health check passed)\n",
+  "read_e": {
+    "A": true,
+    "B": true
+  },
+  "obs_e": {
+    "pinned_images": {
+      "vikunja": "vikunja/vikunja:2.5.0"
+    },
+    "installed_images": {
+      "vikunja": "vikunja/vikunja:2.5.0"
+    },
+    "catalog_images": {
+      "vikunja": "vikunja/vikunja:2.6.0"
+    },
+    "live_compose_image_lines": [
+      "image: vikunja/vikunja:2.5.0"
+    ],
+    "docker_inspect": [
+      "vikunja  vikunja/vikunja:2.5.0  running=true  restarts=0"
+    ]
+  },
+  "restore_f": {
+    "ok": true,
+    "snapshot_id": "helyi",
+    "snapshots": [
+      {
+        "time": "2026-09-24T06:16:42Z",
+        "short_id": "helyi",
+        "tier": 1,
+        "drive_label": "Belső SSD (rendszer)"
+      }
+    ],
+    "http": "HTTP/2 302",
+    "location": [
+      "location: /backups/restore?flash=flash.restore.started"
+    ],
+    "seconds": 97.2,
+    "state_after": "unhealthy",
+    "hold_after": null,
+    "observables_after": {
+      "pinned_images": {
+        "vikunja": "vikunja/vikunja:2.5.0"
+      },
+      "installed_images": {
+        "vikunja": "vikunja/vikunja:2.5.0"
+      },
+      "catalog_images": {
+        "vikunja": "vikunja/vikunja:2.6.0"
+      },
+      "live_compose_image_lines": [
+        "image: vikunja/vikunja:2.5.0"
+      ],
+      "docker_inspect": [
+        "vikunja  vikunja/vikunja:2.5.0  running=true  restarts=0"
+      ]
+    }
+  },
+  "labels_after_restore_v0268": "vikunja_vikunja_data {\"com.docker.compose.project\":\"vikunja\",\"com.docker.compose.version\":\"5.5.0\",\"com.docker.compose.volume\":\"vikunja_data\"}\nvikunja_vikunja_db {\"com.docker.compose.project\":\"vikunja\",\"com.docker.compose.version\":\"5.5.0\",\"com.docker.compose.volume\":\"vikunja_db\"}\n",
+  "log_f": "2026/09/24 06:19:17 restore.go:152: [INFO] [backup] Restoring Docker volume vikunja_vikunja_data for vikunja\n2026/09/24 06:19:18 restore.go:152: [INFO] [backup] Restoring Docker volume vikunja_vikunja_db for vikunja\n",
+  "compose_warn_f": "",
+  "read_f": {
+    "A": true
+  },
+  "press_g": {
+    "accepted": true,
+    "http": "202",
+    "phases": [
+      {
+        "t": 0.0,
+        "phase": "backing-up",
+        "label": "Biztonsági mentés készül a frissítés előtt…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 2.1,
+        "phase": "safety-dump",
+        "label": "Adatbázis pillanatkép…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 3.1,
+        "phase": "pulling",
+        "label": "Új verzió letöltése…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 4.1,
+        "phase": "copying",
+        "label": "Az adatok másolása a frissítés előtt…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 5.2,
+        "phase": "starting",
+        "label": "Indítás az új verzióval…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 6.2,
+        "phase": "verifying",
+        "label": "Működés ellenőrzése…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 11.3,
+        "phase": "done",
+        "label": "Frissítve",
+        "updating": false,
+        "error": null,
+        "hold": null
+      }
+    ],
+    "duration_s": 11.3,
+    "final_phase": "done",
+    "update_error": null,
+    "hold_reason": null,
+    "state": "running"
+  },
+  "log_g": "2026/09/24 06:21:07 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition\n2026/09/24 06:21:09 tier2.go:425: [INFO] [backup] Tier 2 copied vikunja → /mnt/felhom-drives/scratch_hdd/backups/secondary/vikunja (2.6 MB, 0 leg(s), 0s)\n2026/09/24 06:21:10 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB\n2026/09/24 06:21:12 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T062112Z in 481ms\n2026/09/24 06:21:13 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T062112Z in 462ms\n",
+  "read_g": {
+    "A": true,
+    "C": true
+  },
+  "drill_h": "418e8ed2aeb2",
+  "remove_h": "200",
+  "leftovers_h": "total 24\ndrwxr-xr-x  3 root root 4096 Sep 24 06:21 .\ndrwxr-xr-x 57 root root 4096 Sep 13 20:22 ..\n-rw-r--r--  1 root root 4419 Sep 24 06:19 .felhom.yml\n-rw-r--r--  1 root root 1280 Sep 24 06:21 docker-compose.yml\ndrwxr-xr-x 12 root root 4096 Sep 24 06:19 hold-logs\nno vikunja volumes\n"
+}
@@ -0,0 +1,69 @@
+08:17:21 labels before (unlabelled after the v0.267.0 restore):
+vikunja_vikunja_data null
+vikunja_vikunja_db null
+
+08:17:22 drill 17a5e25f99bc: vikunja 2.6.0 + probe 8999 (the failing edge) push rc=0
+08:17:27   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:17:27       +   0.0s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:17:28       +   1.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:17:30       +   3.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+08:17:31       +   4.1s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+08:19:01       +  94.3s  phase=undoing  label=Visszaállítás az előző változatra…  err=None  hold=None
+08:19:10       + 102.5s  phase=undone  label=Visszaállítva az előző változatra  err=None  hold=None
+08:19:13 controller log (e):
+2026/09/24 06:17:27 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition
+2026/09/24 06:17:27 undo.go:387: [WARN] [stacks] update vikunja: volume(s) [vikunja_vikunja_data vikunja_vikunja_db] carry no compose label (recreated by a restore before v0.268.0 — R-658) — copied by name
+2026/09/24 06:17:28 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB
+2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T061729Z in 431ms
+2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T061729Z in 428ms
+2026/09/24 06:19:01 undo.go:501: [WARN] [stacks] update vikunja: UNDO — putting back the previous version and its 2 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))
+2026/09/24 06:19:09 undo.go:549: [INFO] [stacks] update vikunja: UNDONE in 7s — the previous version is running on the data from before the update (the app's health check passed)
+
+08:19:13     vikunja: readback of the seeded project http=200 ok=True
+08:19:14     vikunja: readback of the seeded project http=200 ok=True
+08:19:14 READBACK after the undo (e): {'A': True, 'B': True}
+08:19:17   [R] restoring vikunja from snapshot 'helyi' (of 1 offered)
+08:19:17   [R] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
+08:19:17       +   0.0s restore (True, None, None)
+08:20:54       +  97.2s restore (False, None, None)
+08:20:54   [R] after restore: state=unhealthy hold=None phase=undone
+08:21:00 labels after the v0.268.0 restore:
+vikunja_vikunja_data {"com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_data"}
+vikunja_vikunja_db {"com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_db"}
+
+08:21:06 compose warnings after the restore (want none): (none)
+08:21:07     vikunja: readback of the seeded project http=200 ok=True
+08:21:07 READBACK after the v0.268.0 restore (f): {'A': True}
+08:21:07     vikunja: register http=200
+08:21:07     vikunja: create project http=201
+08:21:07   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:21:08       +   0.0s  phase=backing-up  label=Biztonsági mentés készül a frissítés előtt…  err=None  hold=None
+08:21:10       +   2.1s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:21:11       +   3.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:21:12       +   4.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+08:21:13       +   5.2s  phase=starting  label=Indítás az új verzióval…  err=None  hold=None
+08:21:14       +   6.2s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+08:21:19       +  11.3s  phase=done  label=Frissítve  err=None  hold=None
+08:21:22 controller log (g):
+2026/09/24 06:21:07 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition
+2026/09/24 06:21:09 tier2.go:425: [INFO] [backup] Tier 2 copied vikunja → /mnt/felhom-drives/scratch_hdd/backups/secondary/vikunja (2.6 MB, 0 leg(s), 0s)
+2026/09/24 06:21:10 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB
+2026/09/24 06:21:12 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T062112Z in 481ms
+2026/09/24 06:21:13 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T062112Z in 462ms
+
+08:21:23     vikunja: readback of the seeded project http=200 ok=True
+08:21:23     vikunja: readback of the seeded project http=200 ok=True
+08:21:23 READBACK after the second undo (g): {'A': True, 'C': True}
+08:21:24 drill 418e8ed2aeb2: vikunja back to 2.5.0 + its real probe push rc=0
+08:21:24   [X] stop -> 200 {'ok': True, 'message': 'Stack vikunja stop completed'}
+08:21:56   [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'vikunja', 'volumes_removed': ['vikunja_vikunja_data', 'vikunja_vikunja_db'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [
+08:22:04   [X] after remove: deployed=False leftovers='/opt/docker/stacks/vikunja'
+08:22:07 after remove:
+total 24
+drwxr-xr-x  3 root root 4096 Sep 24 06:21 .
+drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
+-rw-r--r--  1 root root 4419 Sep 24 06:19 .felhom.yml
+-rw-r--r--  1 root root 1280 Sep 24 06:21 docker-compose.yml
+drwxr-xr-x 12 root root 4096 Sep 24 06:19 hold-logs
+no vikunja volumes
+
@@ -0,0 +1,69 @@
+08:17:21 labels before (unlabelled after the v0.267.0 restore):
+vikunja_vikunja_data null
+vikunja_vikunja_db null
+
+08:17:22 drill 17a5e25f99bc: vikunja 2.6.0 + probe 8999 (the failing edge) push rc=0
+08:17:27   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:17:27       +   0.0s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:17:28       +   1.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:17:30       +   3.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+08:17:31       +   4.1s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+08:19:01       +  94.3s  phase=undoing  label=Visszaállítás az előző változatra…  err=None  hold=None
+08:19:10       + 102.5s  phase=undone  label=Visszaállítva az előző változatra  err=None  hold=None
+08:19:13 controller log (e):
+2026/09/24 06:17:27 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition
+2026/09/24 06:17:27 undo.go:387: [WARN] [stacks] update vikunja: volume(s) [vikunja_vikunja_data vikunja_vikunja_db] carry no compose label (recreated by a restore before v0.268.0 — R-658) — copied by name
+2026/09/24 06:17:28 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB
+2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T061729Z in 431ms
+2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T061729Z in 428ms
+2026/09/24 06:19:01 undo.go:501: [WARN] [stacks] update vikunja: UNDO — putting back the previous version and its 2 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))
+2026/09/24 06:19:09 undo.go:549: [INFO] [stacks] update vikunja: UNDONE in 7s — the previous version is running on the data from before the update (the app's health check passed)
+
+08:19:13     vikunja: readback of the seeded project http=200 ok=True
+08:19:14     vikunja: readback of the seeded project http=200 ok=True
+08:19:14 READBACK after the undo (e): {'A': True, 'B': True}
+08:19:17   [R] restoring vikunja from snapshot 'helyi' (of 1 offered)
+08:19:17   [R] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
+08:19:17       +   0.0s restore (True, None, None)
+08:20:54       +  97.2s restore (False, None, None)
+08:20:54   [R] after restore: state=unhealthy hold=None phase=undone
+08:21:00 labels after the v0.268.0 restore:
+vikunja_vikunja_data {"com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_data"}
+vikunja_vikunja_db {"com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_db"}
+
+08:21:06 compose warnings after the restore (want none): (none)
+08:21:07     vikunja: readback of the seeded project http=200 ok=True
+08:21:07 READBACK after the v0.268.0 restore (f): {'A': True}
+08:21:07     vikunja: register http=200
+08:21:07     vikunja: create project http=201
+08:21:07   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:21:08       +   0.0s  phase=backing-up  label=Biztonsági mentés készül a frissítés előtt…  err=None  hold=None
+08:21:10       +   2.1s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:21:11       +   3.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:21:12       +   4.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+08:21:13       +   5.2s  phase=starting  label=Indítás az új verzióval…  err=None  hold=None
+08:21:14       +   6.2s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+08:21:19       +  11.3s  phase=done  label=Frissítve  err=None  hold=None
+08:21:22 controller log (g):
+2026/09/24 06:21:07 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition
+2026/09/24 06:21:09 tier2.go:425: [INFO] [backup] Tier 2 copied vikunja → /mnt/felhom-drives/scratch_hdd/backups/secondary/vikunja (2.6 MB, 0 leg(s), 0s)
+2026/09/24 06:21:10 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB
+2026/09/24 06:21:12 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T062112Z in 481ms
+2026/09/24 06:21:13 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T062112Z in 462ms
+
+08:21:23     vikunja: readback of the seeded project http=200 ok=True
+08:21:23     vikunja: readback of the seeded project http=200 ok=True
+08:21:23 READBACK after the second undo (g): {'A': True, 'C': True}
+08:21:24 drill 418e8ed2aeb2: vikunja back to 2.5.0 + its real probe push rc=0
+08:21:24   [X] stop -> 200 {'ok': True, 'message': 'Stack vikunja stop completed'}
+08:21:56   [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'vikunja', 'volumes_removed': ['vikunja_vikunja_data', 'vikunja_vikunja_db'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [
+08:22:04   [X] after remove: deployed=False leftovers='/opt/docker/stacks/vikunja'
+08:22:07 after remove:
+total 24
+drwxr-xr-x  3 root root 4096 Sep 24 06:21 .
+drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
+-rw-r--r--  1 root root 4419 Sep 24 06:19 .felhom.yml
+-rw-r--r--  1 root root 1280 Sep 24 06:21 docker-compose.yml
+drwxr-xr-x 12 root root 4096 Sep 24 06:19 hold-logs
+no vikunja volumes
+
@@ -0,0 +1,31 @@
+2026/09/24 06:19:17 handlers.go:1701: [WARN] [web] Restore requested (async): stack=vikunja, snapshot=helyi from 172.18.0.9:57016
+2026/09/24 06:19:17 restore_unit.go:379: [INFO] [backup] Restoring vikunja from recovery unit /mnt/sys_drive/felhom-data/backups/primary/vikunja: images=1, secrets recovered=1/1, data_keys=0
+2026/09/24 06:19:17 manager.go:1265: [INFO] [stacks] Stopping stack: vikunja
+2026/09/24 06:19:17 restore.go:152: [INFO] [backup] Restoring Docker volume vikunja_vikunja_data for vikunja
+2026/09/24 06:19:18 restore.go:152: [INFO] [backup] Restoring Docker volume vikunja_vikunja_db for vikunja
+2026/09/24 06:19:18 restore.go:195: [INFO] [backup] Restored 2 Docker volume(s) for vikunja
+2026/09/24 06:19:18 deploy.go:645: [INFO] [stacks] Redeploying vikunja from recovery unit with 3 env vars
+2026/09/24 06:19:18 pin.go:93: [INFO] [stacks] pin vikunja: vikunja=vikunja/vikunja:2.5.0
+2026/09/24 06:19:21 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
+2026/09/24 06:19:31 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
+2026/09/24 06:19:41 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
+2026/09/24 06:19:51 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
+2026/09/24 06:20:01 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
+2026/09/24 06:20:21 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
+2026/09/24 06:20:41 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
+2026/09/24 06:20:52 restore_unit.go:458: [WARN] [backup] vikunja restored but health check failed: stack vikunja did not reach running state within 1m30s after restore
+2026/09/24 06:20:52 restore_unit.go:464: [INFO] [backup] Restore-from-unit completed: vikunja — 2 volume(s) of 2 listed, 0 database(s) of 0 listed
+2026/09/24 06:20:52 handlers.go:1713: [INFO] [web] Restore completed (async): stack=vikunja in 1m35.21599499s (volumes 2/2, dbs 0/0)
+2026/09/24 06:21:07 update.go:733: [INFO] [stacks] update vikunja: no usable copy on any tier — younger than 24h0m0s and not older than this install's deploy (2026-09-24T06:19:18Z) (found: Tier 2 (second drive) at 2026-09-24T06:11:05Z (10m0s old); Tier 1 (own recovery unit) at 2026-09-24T06:16:42Z (4m0s old)) — backing up first
+2026/09/24 06:21:07 update_guard.go:360: [INFO] [backup] update pre-backup for vikunja: starting (DB dump → volume dump → unit capture → Tier 2)
+2026/09/24 06:21:08 backup.go:918: [INFO] [backup] Stopping vikunja for safe volume dump
+2026/09/24 06:21:08 manager.go:1265: [INFO] [stacks] Stopping stack: vikunja
+2026/09/24 06:21:09 update_guard.go:416: [INFO] [backup] update pre-backup for vikunja: recovery unit captured (0 database dump(s))
+2026/09/24 06:21:09 update.go:763: [INFO] [stacks] update vikunja: safety dump done (0 file(s)) []
+2026/09/24 06:21:10 update.go:1213: [INFO] [stacks] update vikunja: phase pinning
+2026/09/24 06:21:10 pin.go:370: [INFO] [stacks] update vikunja: pin advanced to the catalog's current definition (vikunja=vikunja/vikunja:2.6.0)
+2026/09/24 06:21:13 healthprobe.go:190: [WARN] Health probe vikunja: API GET :3456/api/v1/info → Get "http://vikunja:3456/api/v1/info": dial tcp 172.18.0.5:3456: connect: connection refused
+2026/09/24 06:21:18 update.go:884: [INFO] [stacks] update vikunja: healthy after 5s (the app's health check passed)
+2026/09/24 06:21:18 update.go:892: [INFO] [stacks] update vikunja: DONE in 11s
+2026/09/24 06:21:24 manager.go:1265: [INFO] [stacks] Stopping stack: vikunja
+
@@ -0,0 +1,15 @@
+{
+  "A": {
+    "user": "drill9ba9d9",
+    "pw": "<redacted \u2014 a throwaway account on a removed app>",
+    "title": "drill-d60cb3a1c9",
+    "pid": 2
+  },
+  "B": {
+    "user": "drilld26feb",
+    "pw": "<redacted \u2014 a throwaway account on a removed app>",
+    "title": "drill-4821c514ac",
+    "pid": 4
+  },
+  "sub": "vikunja-la"
+}
@@ -0,0 +1,75 @@
+{
+  "controller": "gitea.dooplex.hu/admin/felhom-controller:0.268.0",
+  "drill_1": "6a41e807228f",
+  "deployed": true,
+  "seed_A": true,
+  "drill_2": "a6d49ca9eabc",
+  "badge_wait": 4.4,
+  "steps_left_line_hu": [
+    [
+      "1",
+      "Hátralévő frissítési lépések: 1"
+    ]
+  ],
+  "steps_left_line_en": [
+    [
+      "1",
+      "Update steps remaining: 1"
+    ]
+  ],
+  "phases": [
+    [
+      0.0,
+      "backing-up"
+    ],
+    [
+      18.9,
+      "safety-dump"
+    ],
+    [
+      21.0,
+      "pulling"
+    ],
+    [
+      22.6,
+      "copying"
+    ],
+    [
+      36.8,
+      "starting"
+    ],
+    [
+      43.1,
+      "verifying"
+    ],
+    [
+      133.4,
+      "undoing"
+    ],
+    [
+      136.0,
+      "failed"
+    ]
+  ],
+  "cutoff": "copy: nextcloud_nextcloud_db_data.pre-update-20260924T062548Z\ndata\n",
+  "api": {
+    "state": "stopped",
+    "update_phase": "failed",
+    "hold_reason": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.",
+    "hold_no_whole_copy": true,
+    "update_error": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást."
+  },
+  "app_page_hu": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.",
+  "app_page_hu_restore_button": false,
+  "stacks_page_hu_restore_button": false,
+  "app_page_en": "The update of nextcloud did not work, and neither did the automatic undo. This box has no copy that can bring the app back together with its files. Felhom support has been told — until then, do not restart or remove the app.",
+  "app_page_en_restore_button": false,
+  "stacks_page_en_restore_button": false,
+  "ascii_checks": {
+    "hu has 'gyfelszolgalat' (positive)": true,
+    "hu has 'Visszaallithato a Mentesek' (must be absent)": false,
+    "en has 'Felhom support has been told' (positive)": true
+  },
+  "log": "2026/09/24 06:25:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event crossdrive_completed (severity info)\n2026/09/24 06:26:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event health_change (severity warning)\n2026/09/24 06:27:37 undo.go:501: [WARN] [stacks] update nextcloud: UNDO — putting back the previous version and its 3 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))\n2026/09/24 06:27:40 update.go:937: [ERROR] [stacks] update nextcloud: the UNDO failed too (untouched) — HOLDING the app; the undo copies are kept: [{nextcloud_nextcloud_db_data nextcloud_nextcloud_db_data.pre-update-20260924T062548Z} {nextcloud_nextcloud_html nextcloud_nextcloud_html.pre-update-20260924T062548Z} {nextcloud_nextcloud_redis_data nextcloud_nextcloud_redis_data.pre-update-20260924T062548Z}]\n2026/09/24 06:27:40 update_guard.go:705: [ERROR] [backup] nextcloud is HELD STOPPED after a failed update and NO copy on this box brings it back whole (seen: [tier 2 at 2026-09-24T06:25:42Z tier 1 at 2026-09-24T06:25:43Z]; drive files declared: true; undo: \"untouched\") — support must act (R-659)\n2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_update_held (severity error) — further app_update_held events are logged at DEBUG only\n2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_hold_no_whole_copy (severity critical) — further app_hold_no_whole_copy events are logged at DEBUG only\n",
+  "debug_ring_events": []
+}
@@ -0,0 +1,37 @@
+08:23:32 drill 6a41e807228f: nextcloud template = ladder step 1's own definition (34.0.1 / mariadb 12.3) push rc=0
+08:23:40   [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/nextcloud']
+08:23:40   [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['NEXTCLOUD_ADMIN_PASSWORD', 'HDD_PATH']
+08:23:41   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+08:25:06   [1] deployed, controller state=running, pinned={'nextcloud': 'nextcloud:34.0.1-apache', 'nextcloud-db': 'mariadb:12.3', 'nextcloud-redis': 'redis:7-alpine'}
+08:25:15     nextcloud: occ user:add :: The account "drill10a42a" was created successfully Display name set to "drill10a42a"
+08:25:18     nextcloud: seeded user drill10a42a
+08:25:19 drill a6d49ca9eabc: nextcloud back to the head (34.0.4) + probe port 8999 push rc=0
+08:25:24 steps-left line hu=[('1', 'Hátralévő frissítési lépések: 1')] en=[('1', 'Update steps remaining: 1')]
+08:25:24 Update -> 202
+08:25:24   +   0.0s phase=backing-up
+08:25:43   +  18.9s phase=safety-dump
+08:25:45   +  21.0s phase=pulling
+08:25:47   +  22.6s phase=copying
+08:26:01   +  36.8s phase=starting
+08:26:07   +  43.1s phase=verifying
+08:26:13   >>> finished-marker removed from one undo copy:
+copy: nextcloud_nextcloud_db_data.pre-update-20260924T062548Z
+data
+
+08:27:37   + 133.4s phase=undoing
+08:27:40   + 136.0s phase=failed
+08:27:40 API: {"state": "stopped", "update_phase": "failed", "hold_reason": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.", "hold_no_whole_copy": true, "update_error": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást."}
+08:27:40 [hu] app page held block: 'A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.'  restore-button app=False list=False
+08:27:40 [en] app page held block: 'The update of nextcloud did not work, and neither did the automatic undo. This box has no copy that can bring the app back together with its files. Felhom support has been told — until then, do not restart or remove the app.'  restore-button app=False list=False
+08:27:40 checks: {"hu has 'gyfelszolgalat' (positive)": true, "hu has 'Visszaallithato a Mentesek' (must be absent)": false, "en has 'Felhom support has been told' (positive)": true}
+08:30:13 controller log:
+2026/09/24 06:25:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event crossdrive_completed (severity info)
+2026/09/24 06:26:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event health_change (severity warning)
+2026/09/24 06:27:37 undo.go:501: [WARN] [stacks] update nextcloud: UNDO — putting back the previous version and its 3 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))
+2026/09/24 06:27:40 update.go:937: [ERROR] [stacks] update nextcloud: the UNDO failed too (untouched) — HOLDING the app; the undo copies are kept: [{nextcloud_nextcloud_db_data nextcloud_nextcloud_db_data.pre-update-20260924T062548Z} {nextcloud_nextcloud_html nextcloud_nextcloud_html.pre-update-20260924T062548Z} {nextcloud_nextcloud_redis_data nextcloud_nextcloud_redis_data.pre-update-20260924T062548Z}]
+2026/09/24 06:27:40 update_guard.go:705: [ERROR] [backup] nextcloud is HELD STOPPED after a failed update and NO copy on this box brings it back whole (seen: [tier 2 at 2026-09-24T06:25:42Z tier 1 at 2026-09-24T06:25:43Z]; drive files declared: true; undo: "untouched") — support must act (R-659)
+2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_update_held (severity error) — further app_update_held events are logged at DEBUG only
+2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_hold_no_whole_copy (severity critical) — further app_hold_no_whole_copy events are logged at DEBUG only
+
+08:30:14 debug ring (events since the press):
+  
@@ -0,0 +1,37 @@
+08:23:32 drill 6a41e807228f: nextcloud template = ladder step 1's own definition (34.0.1 / mariadb 12.3) push rc=0
+08:23:40   [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/nextcloud']
+08:23:40   [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['NEXTCLOUD_ADMIN_PASSWORD', 'HDD_PATH']
+08:23:41   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+08:25:06   [1] deployed, controller state=running, pinned={'nextcloud': 'nextcloud:34.0.1-apache', 'nextcloud-db': 'mariadb:12.3', 'nextcloud-redis': 'redis:7-alpine'}
+08:25:15     nextcloud: occ user:add :: The account "drill10a42a" was created successfully Display name set to "drill10a42a"
+08:25:18     nextcloud: seeded user drill10a42a
+08:25:19 drill a6d49ca9eabc: nextcloud back to the head (34.0.4) + probe port 8999 push rc=0
+08:25:24 steps-left line hu=[('1', 'Hátralévő frissítési lépések: 1')] en=[('1', 'Update steps remaining: 1')]
+08:25:24 Update -> 202
+08:25:24   +   0.0s phase=backing-up
+08:25:43   +  18.9s phase=safety-dump
+08:25:45   +  21.0s phase=pulling
+08:25:47   +  22.6s phase=copying
+08:26:01   +  36.8s phase=starting
+08:26:07   +  43.1s phase=verifying
+08:26:13   >>> finished-marker removed from one undo copy:
+copy: nextcloud_nextcloud_db_data.pre-update-20260924T062548Z
+data
+
+08:27:37   + 133.4s phase=undoing
+08:27:40   + 136.0s phase=failed
+08:27:40 API: {"state": "stopped", "update_phase": "failed", "hold_reason": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.", "hold_no_whole_copy": true, "update_error": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást."}
+08:27:40 [hu] app page held block: 'A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.'  restore-button app=False list=False
+08:27:40 [en] app page held block: 'The update of nextcloud did not work, and neither did the automatic undo. This box has no copy that can bring the app back together with its files. Felhom support has been told — until then, do not restart or remove the app.'  restore-button app=False list=False
+08:27:40 checks: {"hu has 'gyfelszolgalat' (positive)": true, "hu has 'Visszaallithato a Mentesek' (must be absent)": false, "en has 'Felhom support has been told' (positive)": true}
+08:30:13 controller log:
+2026/09/24 06:25:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event crossdrive_completed (severity info)
+2026/09/24 06:26:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event health_change (severity warning)
+2026/09/24 06:27:37 undo.go:501: [WARN] [stacks] update nextcloud: UNDO — putting back the previous version and its 3 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))
+2026/09/24 06:27:40 update.go:937: [ERROR] [stacks] update nextcloud: the UNDO failed too (untouched) — HOLDING the app; the undo copies are kept: [{nextcloud_nextcloud_db_data nextcloud_nextcloud_db_data.pre-update-20260924T062548Z} {nextcloud_nextcloud_html nextcloud_nextcloud_html.pre-update-20260924T062548Z} {nextcloud_nextcloud_redis_data nextcloud_nextcloud_redis_data.pre-update-20260924T062548Z}]
+2026/09/24 06:27:40 update_guard.go:705: [ERROR] [backup] nextcloud is HELD STOPPED after a failed update and NO copy on this box brings it back whole (seen: [tier 2 at 2026-09-24T06:25:42Z tier 1 at 2026-09-24T06:25:43Z]; drive files declared: true; undo: "untouched") — support must act (R-659)
+2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_update_held (severity error) — further app_update_held events are logged at DEBUG only
+2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_hold_no_whole_copy (severity critical) — further app_hold_no_whole_copy events are logged at DEBUG only
+
+08:30:14 debug ring (events since the press):
+  
@@ -0,0 +1,27 @@
+2026/09/24 06:23:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:24:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:24:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:25:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:25:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:26:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:26:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:27:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:27:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:27:42 main.go:1951: [INFO] [deadapp] check alive: 20 scans since boot, 4 deployed app(s) evaluated, 0 currently down
+2026/09/24 06:28:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:28:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:29:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:29:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:30:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+
+200 ['entries', 'total']
+200 {'timestamp': '2026-09-24T06:27:41Z', 'level': 'DEBUG', 'message': '[stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container)', 'source': 'healthprobe.go:104'}
+{"timestamp": "2026-09-24T06:27:42Z", "level": "INFO", "message": "[deadapp] check alive: 20 scans since boot, 4 deployed app(s) evaluated, 0 currently down", "source": "main.go:1951"}
+{"timestamp": "2026-09-24T06:28:11Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
+{"timestamp": "2026-09-24T06:28:41Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
+{"timestamp": "2026-09-24T06:29:11Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
+{"timestamp": "2026-09-24T06:29:41Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
+{"timestamp": "2026-09-24T06:30:11Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
+2026/09/24 06:27:42 main.go:1951: [INFO] [deadapp] check alive: 20 scans since boot, 4 deployed app(s) evaluated, 0 currently down
+gokapi Restarting (1) 28 seconds ago
+
@@ -0,0 +1,8 @@
+{
+  "events": [
+    "2026-09-24T06:31:41Z notifier disabled: dropped event app_start_failed (severity warning)",
+    "2026-09-24T06:31:41Z notifier disabled: dropped event app_start_failed (severity warning)"
+  ],
+  "log": "2026/09/24 06:31:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting\n2026/09/24 06:31:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting\n2026/09/24 06:31:41 notifier.go:1259: [DEBUG] notifier disabled: dropped event app_start_failed (severity warning)\n2026/09/24 06:31:41 notifier.go:1259: [DEBUG] notifier disabled: dropped event app_start_failed (severity warning)\n",
+  "nextcloud_state": "stopped"
+}
@@ -0,0 +1,16 @@
+08:31:02   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+08:31:12   [1] deployed, controller state=running, pinned={'actualbudget': 'actualbudget/actual-server:26.9.0'}
+08:31:16 stopping the throwaway's container out of band (not through the product): actualbudget
+08:31:46 events/heartbeats since the control started:
+  2026-09-24T06:31:41Z notifier disabled: dropped event app_start_failed (severity warning)
+  2026-09-24T06:31:41Z notifier disabled: dropped event app_start_failed (severity warning)
+08:31:49 controller log lines:
+2026/09/24 06:31:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:31:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/09/24 06:31:41 notifier.go:1259: [DEBUG] notifier disabled: dropped event app_start_failed (severity warning)
+2026/09/24 06:31:41 notifier.go:1259: [DEBUG] notifier disabled: dropped event app_start_failed (severity warning)
+
+08:31:49 nextcloud meanwhile: state=stopped held=True
+08:31:49   [X] stop -> 200 {'ok': True, 'message': 'Stack actualbudget stop completed'}
+08:32:21   [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'actualbudget', 'volumes_removed': ['actualbudget_actualbudget_data'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd
+08:32:30   [X] after remove: deployed=False leftovers='/opt/docker/stacks/actualbudget'
@@ -0,0 +1,5 @@
+gokapi restarting  running
+nextcloud stopped held running
+paperless-ngx running  running
+privatebin running  stopped
+[]
@@ -0,0 +1,2 @@
+2026-09-24T06:34:42Z [stacks] ScanStacks: found stack "gokapi" deployed=true composePath=/opt/docker/stacks/gokapi/docker-compose.yml
+2026-09-24T06:34:42Z [stacks] ScanStacks: found stack "nextcloud" deployed=true composePath=/opt/docker/stacks/nextcloud/docker-compose.yml
@@ -0,0 +1,128 @@
+[
+  {
+    "t": 10,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 15,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 20,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 25,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 30,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 35,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 40,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 45,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 50,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 55,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 60,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 65,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 70,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 75,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 80,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 85,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 90,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  },
+  {
+    "t": 95,
+    "banner": [],
+    "events": [
+      "2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
+    ]
+  }
+]
@@ -0,0 +1,30 @@
+08:39:36 before remove:
+nextcloud_nextcloud_db_data
+nextcloud_nextcloud_db_data.pre-update-20260924T062548Z
+nextcloud_nextcloud_html
+nextcloud_nextcloud_html.pre-update-20260924T062548Z
+nextcloud_nextcloud_redis_data
+nextcloud_nextcloud_redis_data.pre-update-20260924T062548Z
+app.yaml
+applied-compose.yml
+applied-meta
+docker-compose.yml
+hold-logs
+
+08:39:36   [X] stop -> 200 {'ok': True, 'message': 'Stack nextcloud stop completed'}
+08:39:41   [X] remove (with drive data) -> 409 {'ok': False, 'error': 'A(z) /mnt/felhom-drives/scratch_hdd/userdata/nextcloud tárhely jelenleg nem elérhető — az alkalmazás nem távolítható el, amíg a meghajtó
+08:39:41   [X] refused because the drive path cannot be resolved (R-442, fail-closed and right) — removing the app and KEEPING the drive data instead
+08:40:10   [X] remove (keeping drive data) -> 200 {'ok': True, 'data': {'removed': 'nextcloud', 'volumes_removed': ['nextcloud_nextcloud_db_data', 'nextcloud_nextcloud_html', 'nextcloud_nextcloud_redis_data'], 
+08:40:18   [X] after remove: deployed=False leftovers='/opt/docker/stacks/nextcloud'
+08:40:21 after remove:
+no nextcloud volumes (incl. undo copies)
+total 32
+drwxr-xr-x  3 root root 4096 Sep 24 06:40 .
+drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
+-rw-r--r--  1 root root 9182 Sep 24 06:25 .felhom.yml
+-rw-r--r--  1 root root 4593 Sep 24 06:25 docker-compose.yml
+drwxr-xr-x  4 root root 4096 Sep 24 06:27 hold-logs
+ls: cannot access '/mnt/felhom-drives/scratch_hdd/appdata/nextcloud': No such file or directory
+/mnt/felhom-drives/scratch_hdd/userdata/nextcloud
+
+08:40:24 drive folders removed by name (R-442 keeps them on 9202): ls: cannot access '/mnt/felhom-drives/scratch_hdd/userdata/nextcloud': No such file or directory
@@ -0,0 +1,10 @@
+git:
+    branch: main
+    repo_url: https://gitea.dooplex.hu/admin/app-catalog-drill.git
+    sync_interval: 15m
+    token: <redacted>
+    username: "admin"
+hub:
+update:
+    health_timeout: 90s
+
@@ -0,0 +1,110 @@
+{
+  "deployed_A": true,
+  "A": {
+    "pinned_images": {
+      "vikunja": "vikunja/vikunja:2.4.0"
+    },
+    "installed_images": {
+      "vikunja": "vikunja/vikunja:2.4.0"
+    },
+    "catalog_images": null,
+    "live_compose_image_lines": [
+      "image: vikunja/vikunja:2.4.0"
+    ],
+    "docker_inspect": [
+      "vikunja  vikunja/vikunja:2.4.0  running=true  restarts=0"
+    ]
+  },
+  "badge_wait_s": 10.5,
+  "cache_steps": "ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/': No such file or directory\nls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/steps/': No such file or directory\napp.yaml\napplied-compose.yml\napplied-meta\ndocker-compose.yml\nhold-logs\n",
+  "badges": {
+    "hu": [
+      {
+        "title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
+        "text": "Frissítés elérhető — 2 napja"
+      }
+    ],
+    "en": [
+      {
+        "title": "A newer version of this app is available. Select the Update button to start it.",
+        "text": "Update available — 2 days ago"
+      }
+    ]
+  },
+  "press": {
+    "accepted": true,
+    "http": "202",
+    "phases": [
+      {
+        "t": 0.0,
+        "phase": "backing-up",
+        "label": "Biztonsági mentés készül a frissítés előtt…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 2.1,
+        "phase": "safety-dump",
+        "label": "Adatbázis pillanatkép…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 3.1,
+        "phase": "pulling",
+        "label": "Új verzió letöltése…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 4.1,
+        "phase": "copying",
+        "label": "Az adatok másolása a frissítés előtt…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 12.3,
+        "phase": "verifying",
+        "label": "Működés ellenőrzése…",
+        "updating": true,
+        "error": null,
+        "hold": null
+      },
+      {
+        "t": 32.8,
+        "phase": "done",
+        "label": "Frissítve",
+        "updating": false,
+        "error": null,
+        "hold": null
+      }
+    ],
+    "duration_s": 32.9,
+    "final_phase": "done",
+    "update_error": null,
+    "hold_reason": null,
+    "state": "running"
+  },
+  "after": {
+    "pinned_images": {
+      "vikunja": "vikunja/vikunja:2.6.0"
+    },
+    "installed_images": {
+      "vikunja": "vikunja/vikunja:2.6.0"
+    },
+    "catalog_images": {
+      "vikunja": "vikunja/vikunja:2.6.0"
+    },
+    "live_compose_image_lines": [
+      "image: vikunja/vikunja:2.6.0"
+    ],
+    "docker_inspect": [
+      "vikunja  vikunja/vikunja:2.6.0  running=true  restarts=0"
+    ]
+  }
+}
@@ -0,0 +1,27 @@
+07:44:29 drill commit 15a4107ef0c0: SPIKE vikunja A=2.4.0 (push rc=0)
+07:45:48   [sync] the badge NEVER caught up to vikunja/vikunja:2.4.0 in 78.9s — catalog_images = None
+07:45:48   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+07:45:58   [1] deployed, controller state=running, pinned={'vikunja': 'vikunja/vikunja:2.4.0'}
+07:46:03 drill commit 4d557385d146: SPIKE vikunja B=2.5.0 + steps/spike-B.yml (push rc=0)
+07:46:03 drill commit 1cb96ca718dd: SPIKE vikunja C=2.6.0 (push rc=0)
+07:46:14   [sync] the badge needed 10.5s and 2 sync+rescan rounds to catch up to vikunja/vikunja:2.6.0 — R-607's window, measured
+07:46:18 cache:
+ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/': No such file or directory
+ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/steps/': No such file or directory
+app.yaml
+applied-compose.yml
+applied-meta
+docker-compose.yml
+hold-logs
+
+07:46:18   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+07:46:18       +   0.0s  phase=backing-up  label=Biztonsági mentés készül a frissítés előtt…  err=None  hold=None
+07:46:20       +   2.1s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+07:46:21       +   3.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+07:46:22       +   4.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+07:46:30       +  12.3s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+07:46:51       +  32.8s  phase=done  label=Frissítve  err=None  hold=None
+07:46:54 after: {"pinned_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "installed_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "catalog_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "live_compose_image_lines": ["image: vikunja/vikunja:2.6.0"], "docker_inspect": ["vikunja  vikunja/vikunja:2.6.0  running=true  restarts=0"]}
+07:46:55   [X] stop -> 200 {'ok': True, 'message': 'Stack vikunja stop completed'}
+07:47:26   [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'vikunja', 'volumes_removed': ['vikunja_vikunja_data', 'vikunja_vikunja_db'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [
+07:47:35   [X] after remove: deployed=False leftovers='/opt/docker/stacks/vikunja'
@@ -0,0 +1,27 @@
+07:44:29 drill commit 15a4107ef0c0: SPIKE vikunja A=2.4.0 (push rc=0)
+07:45:48   [sync] the badge NEVER caught up to vikunja/vikunja:2.4.0 in 78.9s — catalog_images = None
+07:45:48   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+07:45:58   [1] deployed, controller state=running, pinned={'vikunja': 'vikunja/vikunja:2.4.0'}
+07:46:03 drill commit 4d557385d146: SPIKE vikunja B=2.5.0 + steps/spike-B.yml (push rc=0)
+07:46:03 drill commit 1cb96ca718dd: SPIKE vikunja C=2.6.0 (push rc=0)
+07:46:14   [sync] the badge needed 10.5s and 2 sync+rescan rounds to catch up to vikunja/vikunja:2.6.0 — R-607's window, measured
+07:46:18 cache:
+ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/': No such file or directory
+ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/steps/': No such file or directory
+app.yaml
+applied-compose.yml
+applied-meta
+docker-compose.yml
+hold-logs
+
+07:46:18   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+07:46:18       +   0.0s  phase=backing-up  label=Biztonsági mentés készül a frissítés előtt…  err=None  hold=None
+07:46:20       +   2.1s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+07:46:21       +   3.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+07:46:22       +   4.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+07:46:30       +  12.3s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+07:46:51       +  32.8s  phase=done  label=Frissítve  err=None  hold=None
+07:46:54 after: {"pinned_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "installed_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "catalog_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "live_compose_image_lines": ["image: vikunja/vikunja:2.6.0"], "docker_inspect": ["vikunja  vikunja/vikunja:2.6.0  running=true  restarts=0"]}
+07:46:55   [X] stop -> 200 {'ok': True, 'message': 'Stack vikunja stop completed'}
+07:47:26   [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'vikunja', 'volumes_removed': ['vikunja_vikunja_data', 'vikunja_vikunja_db'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [
+07:47:35   [X] after remove: deployed=False leftovers='/opt/docker/stacks/vikunja'
@@ -0,0 +1,25 @@
+1cb96ca718dd
+1
+true
+/var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache/templates/vikunja/:
+total 24
+drwxr-xr-x  3 root root 4096 Sep 24 05:46 .
+drwxr-xr-x 55 root root 4096 Sep 24 05:44 ..
+-rw-r--r--  1 root root 4419 Sep 24 05:44 .felhom.yml
+-rw-r--r--  1 root root 1280 Sep 24 05:46 docker-compose.yml
+drwxr-xr-x  2 root root 4096 Sep 24 05:46 steps
+
+/var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache/templates/vikunja/steps/:
+total 12
+drwxr-xr-x 2 root root 4096 Sep 24 05:46 .
+drwxr-xr-x 3 root root 4096 Sep 24 05:46 ..
+-rw-r--r-- 1 root root 1280 Sep 24 05:46 spike-B.yml
+total 32
+drwxr-xr-x  4 root root 4096 Sep 24 05:47 .
+drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
+-rw-r--r--  1 root root 4419 Sep 23 22:15 .felhom.yml
+-rw-r--r--  1 root root 1280 Sep 24 05:46 applied-compose.yml
+drwxr-xr-x  2 root root 4096 Sep 24 05:46 applied-meta
+-rw-r--r--  1 root root 1280 Sep 24 05:46 docker-compose.yml
+drwxr-xr-x 11 root root 4096 Sep 23 21:05 hold-logs
+
@@ -0,0 +1,295 @@
+{
+  "controller": "gitea.dooplex.hu/admin/felhom-controller:0.268.0",
+  "drill_install": "95d83b7ab698",
+  "deployed": true,
+  "seed_A": true,
+  "drill_head": "0e4e12aed1f2",
+  "badge_wait": 4.4,
+  "before": {
+    "obs": {
+      "pinned_images": {
+        "romm": "rommapp/romm:5.3.0",
+        "romm-db": "mariadb:11.4",
+        "romm-redis": "redis:7-alpine"
+      },
+      "installed_images": {
+        "romm": "rommapp/romm:5.3.0",
+        "romm-db": "mariadb:11.4",
+        "romm-redis": "redis:7-alpine"
+      },
+      "catalog_images": {
+        "romm": "rommapp/romm:5.3.1",
+        "romm-db": "mariadb:11.8",
+        "romm-redis": "redis:7-alpine"
+      },
+      "live_compose_image_lines": [
+        "image: rommapp/romm:5.3.0",
+        "image: mariadb:11.4",
+        "image: redis:7-alpine"
+      ],
+      "docker_inspect": [
+        "romm  rommapp/romm:5.3.0  running=true  restarts=0",
+        "romm-redis  redis:7-alpine  running=true  restarts=0",
+        "romm-db  mariadb:11.4  running=true  restarts=0"
+      ]
+    },
+    "steps_line": {
+      "hu": [
+        [
+          "2",
+          "Hátralévő frissítési lépések: 2"
+        ]
+      ],
+      "en": [
+        [
+          "2",
+          "Update steps remaining: 2"
+        ]
+      ]
+    },
+    "badges": {
+      "hu": [
+        {
+          "title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
+          "text": "Frissítés elérhető — 1 napja"
+        }
+      ],
+      "en": [
+        {
+          "title": "A newer version of this app is available. Select the Update button to start it.",
+          "text": "Update available — 1 day ago"
+        }
+      ]
+    }
+  },
+  "press1": {
+    "press": {
+      "accepted": true,
+      "http": "202",
+      "phases": [
+        {
+          "t": 0.0,
+          "phase": "safety-dump",
+          "label": "Adatbázis pillanatkép…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 2.1,
+          "phase": "pulling",
+          "label": "Új verzió letöltése…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 3.1,
+          "phase": "copying",
+          "label": "Az adatok másolása a frissítés előtt…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 10.3,
+          "phase": "starting",
+          "label": "Indítás az új verzióval…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 21.6,
+          "phase": "verifying",
+          "label": "Működés ellenőrzése…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 57.4,
+          "phase": "done",
+          "label": "Frissítve",
+          "updating": false,
+          "error": null,
+          "hold": null
+        }
+      ],
+      "duration_s": 57.5,
+      "final_phase": "done",
+      "update_error": null,
+      "hold_reason": null,
+      "state": "running"
+    },
+    "obs": {
+      "pinned_images": {
+        "romm": "rommapp/romm:5.3.1",
+        "romm-db": "mariadb:11.4",
+        "romm-redis": "redis:7-alpine"
+      },
+      "installed_images": {
+        "romm": "rommapp/romm:5.3.1",
+        "romm-db": "mariadb:11.4",
+        "romm-redis": "redis:7-alpine"
+      },
+      "catalog_images": {
+        "romm": "rommapp/romm:5.3.1",
+        "romm-db": "mariadb:11.8",
+        "romm-redis": "redis:7-alpine"
+      },
+      "live_compose_image_lines": [
+        "image: rommapp/romm:5.3.1",
+        "image: mariadb:11.4",
+        "image: redis:7-alpine"
+      ],
+      "docker_inspect": [
+        "romm  rommapp/romm:5.3.1  running=true  restarts=0",
+        "romm-redis  redis:7-alpine  running=true  restarts=0",
+        "romm-db  mariadb:11.4  running=true  restarts=0"
+      ]
+    },
+    "readback_A": true,
+    "log": "2026/09/24 06:42:31 update.go:722: [INFO] [stacks] update romm: ladder — step 2 of 3: romm=rommapp/romm:5.3.0, romm-db=mariadb:11.4, romm-redis=redis:7-alpine → romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine, from 90dd9d68258286ef.yml\n2026/09/24 06:42:33 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine)\n2026/09/24 06:43:28 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)\n2026/09/24 06:43:28 update.go:892: [INFO] [stacks] update romm: DONE in 57s\n",
+    "db_log": "2026-09-24 08:42:42+02:00 [Note] [Entrypoint]: MariaDB upgrade not required\nVersion: '11.4.13-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution\n",
+    "files": "26:    image: rommapp/romm:5.3.1\n86:          memory: 768M\n102:    image: mariadb:11.4\n123:          memory: 384M\n132:    image: redis:7-alpine\n145:          memory: 128M\n---\napplied == steps/90dd9d68258286ef.yml\napplied != template\n",
+    "steps_line": {
+      "hu": [
+        [
+          "1",
+          "Hátralévő frissítési lépések: 1"
+        ]
+      ],
+      "en": [
+        [
+          "1",
+          "Update steps remaining: 1"
+        ]
+      ]
+    },
+    "badges": {
+      "hu": [
+        {
+          "title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
+          "text": "Frissítés elérhető — 1 napja"
+        }
+      ],
+      "en": [
+        {
+          "title": "A newer version of this app is available. Select the Update button to start it.",
+          "text": "Update available — 1 day ago"
+        }
+      ]
+    }
+  },
+  "press2": {
+    "press": {
+      "accepted": true,
+      "http": "202",
+      "phases": [
+        {
+          "t": 0.0,
+          "phase": "safety-dump",
+          "label": "Adatbázis pillanatkép…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 2.1,
+          "phase": "pulling",
+          "label": "Új verzió letöltése…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 3.1,
+          "phase": "copying",
+          "label": "Az adatok másolása a frissítés előtt…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 15.4,
+          "phase": "starting",
+          "label": "Indítás az új verzióval…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 31.8,
+          "phase": "verifying",
+          "label": "Működés ellenőrzése…",
+          "updating": true,
+          "error": null,
+          "hold": null
+        },
+        {
+          "t": 67.7,
+          "phase": "done",
+          "label": "Frissítve",
+          "updating": false,
+          "error": null,
+          "hold": null
+        }
+      ],
+      "duration_s": 67.7,
+      "final_phase": "done",
+      "update_error": null,
+      "hold_reason": null,
+      "state": "running"
+    },
+    "obs": {
+      "pinned_images": {
+        "romm": "rommapp/romm:5.3.1",
+        "romm-db": "mariadb:11.8",
+        "romm-redis": "redis:7-alpine"
+      },
+      "installed_images": {
+        "romm": "rommapp/romm:5.3.1",
+        "romm-db": "mariadb:11.8",
+        "romm-redis": "redis:7-alpine"
+      },
+      "catalog_images": {
+        "romm": "rommapp/romm:5.3.1",
+        "romm-db": "mariadb:11.8",
+        "romm-redis": "redis:7-alpine"
+      },
+      "live_compose_image_lines": [
+        "image: rommapp/romm:5.3.1",
+        "image: mariadb:11.8",
+        "image: redis:7-alpine"
+      ],
+      "docker_inspect": [
+        "romm-db  mariadb:11.8  running=true  restarts=0",
+        "romm  rommapp/romm:5.3.1  running=true  restarts=0",
+        "romm-redis  redis:7-alpine  running=true  restarts=0"
+      ]
+    },
+    "readback_A": true,
+    "log": "2026/09/24 06:43:46 update.go:722: [INFO] [stacks] update romm: ladder — the last step (3 of 3) — the catalog's current definition\n2026/09/24 06:43:48 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.8, romm-redis=redis:7-alpine)\n2026/09/24 06:44:53 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)\n2026/09/24 06:44:53 update.go:892: [INFO] [stacks] update romm: DONE in 1m7s\n",
+    "db_log": "Version: '11.8.9-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution\n2026-09-24 08:44:05+02:00 [Note] [Entrypoint]: Starting mariadb-upgrade\nThe --upgrade-system-tables option was used, user tables won't be touched.\nMajor version upgrade detected from 11.4.13-MariaDB to 11.8.9-MariaDB. Check required!\n2026-09-24 08:44:12+02:00 [Note] [Entrypoint]: Finished mariadb-upgrade\nVersion: '11.8.9-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution\n",
+    "files": "26:    image: rommapp/romm:5.3.1\n86:          memory: 768M\n102:    image: mariadb:11.8\n123:          memory: 384M\n132:    image: redis:7-alpine\n145:          memory: 128M\n---\napplied != steps/90dd…\napplied == template\n",
+    "steps_line": {
+      "hu": [],
+      "en": []
+    },
+    "badges": {
+      "hu": [
+        {
+          "title": "Ez az alkalmazás a legfrissebb elérhető változatot futtatja.",
+          "text": "Naprakész"
+        }
+      ],
+      "en": [
+        {
+          "title": "This app is running the newest version available.",
+          "text": "Up to date"
+        }
+      ]
+    }
+  }
+}
@@ -0,0 +1,70 @@
+08:41:05 drill 95d83b7ab698: romm template = step 1's own definition (5.3.0 / mariadb 11.4) for the install push rc=0
+08:41:12   [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/romm']
+08:41:12   [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH']
+08:41:12   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+08:42:13   [1] deployed, controller state=running, pinned={'romm': 'rommapp/romm:5.3.0', 'romm-db': 'mariadb:11.4', 'romm-redis': 'redis:7-alpine'}
+08:42:22     romm: POST /api/users http=201
+08:42:23 drill 0e4e12aed1f2: romm back to the head (5.3.1 / mariadb 11.8) push rc=0
+08:42:31 before: steps line {"hu": [["2", "Hátralévő frissítési lépések: 2"]], "en": [["2", "Update steps remaining: 2"]]} badges {"hu": [{"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.", "text": "Frissítés elérhető — 1 napja"}], "en": [{"title": "A newer version of this app is available. Select the Update button to start it.", "text": "Update available — 1 day 
+08:42:31   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:42:31       +   0.0s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:42:33       +   2.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:42:34       +   3.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+08:42:41       +  10.3s  phase=starting  label=Indítás az új verzióval…  err=None  hold=None
+08:42:53       +  21.6s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+08:43:28       +  57.4s  phase=done  label=Frissítve  err=None  hold=None
+08:43:37     romm: login as the seeded user http=200 ok=True
+08:43:43     romm: POST /api/users http=403
+08:43:43     romm: refused {"detail":"Forbidden"}
+08:43:46 PRESS 1: phase=done pinned={'romm': 'rommapp/romm:5.3.1', 'romm-db': 'mariadb:11.4', 'romm-redis': 'redis:7-alpine'} readback A=True
+2026/09/24 06:42:31 update.go:722: [INFO] [stacks] update romm: ladder — step 2 of 3: romm=rommapp/romm:5.3.0, romm-db=mariadb:11.4, romm-redis=redis:7-alpine → romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine, from 90dd9d68258286ef.yml
+2026/09/24 06:42:33 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine)
+2026/09/24 06:43:28 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)
+2026/09/24 06:43:28 update.go:892: [INFO] [stacks] update romm: DONE in 57s
+
+DB: 2026-09-24 08:42:42+02:00 [Note] [Entrypoint]: MariaDB upgrade not required
+Version: '11.4.13-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
+
+files: 26:    image: rommapp/romm:5.3.1
+86:          memory: 768M
+102:    image: mariadb:11.4
+123:          memory: 384M
+132:    image: redis:7-alpine
+145:          memory: 128M
+---
+applied == steps/90dd9d68258286ef.yml
+applied != template
+
+steps line: {'hu': [('1', 'Hátralévő frissítési lépések: 1')], 'en': [('1', 'Update steps remaining: 1')]}
+08:43:46   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:43:46       +   0.0s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:43:48       +   2.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:43:50       +   3.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+08:44:02       +  15.4s  phase=starting  label=Indítás az új verzióval…  err=None  hold=None
+08:44:18       +  31.8s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+08:44:54       +  67.7s  phase=done  label=Frissítve  err=None  hold=None
+08:45:02     romm: login as the seeded user http=200 ok=True
+08:45:11 PRESS 2: phase=done pinned={'romm': 'rommapp/romm:5.3.1', 'romm-db': 'mariadb:11.8', 'romm-redis': 'redis:7-alpine'} readback A=True
+2026/09/24 06:43:46 update.go:722: [INFO] [stacks] update romm: ladder — the last step (3 of 3) — the catalog's current definition
+2026/09/24 06:43:48 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.8, romm-redis=redis:7-alpine)
+2026/09/24 06:44:53 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)
+2026/09/24 06:44:53 update.go:892: [INFO] [stacks] update romm: DONE in 1m7s
+
+DB: Version: '11.8.9-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
+2026-09-24 08:44:05+02:00 [Note] [Entrypoint]: Starting mariadb-upgrade
+The --upgrade-system-tables option was used, user tables won't be touched.
+Major version upgrade detected from 11.4.13-MariaDB to 11.8.9-MariaDB. Check required!
+2026-09-24 08:44:12+02:00 [Note] [Entrypoint]: Finished mariadb-upgrade
+Version: '11.8.9-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
+
+files: 26:    image: rommapp/romm:5.3.1
+86:          memory: 768M
+102:    image: mariadb:11.8
+123:          memory: 384M
+132:    image: redis:7-alpine
+145:          memory: 128M
+---
+applied != steps/90dd…
+applied == template
+
+steps line: {'hu': [], 'en': []}
@@ -0,0 +1,70 @@
+08:41:05 drill 95d83b7ab698: romm template = step 1's own definition (5.3.0 / mariadb 11.4) for the install push rc=0
+08:41:12   [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/romm']
+08:41:12   [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH']
+08:41:12   [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
+08:42:13   [1] deployed, controller state=running, pinned={'romm': 'rommapp/romm:5.3.0', 'romm-db': 'mariadb:11.4', 'romm-redis': 'redis:7-alpine'}
+08:42:22     romm: POST /api/users http=201
+08:42:23 drill 0e4e12aed1f2: romm back to the head (5.3.1 / mariadb 11.8) push rc=0
+08:42:31 before: steps line {"hu": [["2", "Hátralévő frissítési lépések: 2"]], "en": [["2", "Update steps remaining: 2"]]} badges {"hu": [{"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.", "text": "Frissítés elérhető — 1 napja"}], "en": [{"title": "A newer version of this app is available. Select the Update button to start it.", "text": "Update available — 1 day 
+08:42:31   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:42:31       +   0.0s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:42:33       +   2.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:42:34       +   3.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+08:42:41       +  10.3s  phase=starting  label=Indítás az új verzióval…  err=None  hold=None
+08:42:53       +  21.6s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+08:43:28       +  57.4s  phase=done  label=Frissítve  err=None  hold=None
+08:43:37     romm: login as the seeded user http=200 ok=True
+08:43:43     romm: POST /api/users http=403
+08:43:43     romm: refused {"detail":"Forbidden"}
+08:43:46 PRESS 1: phase=done pinned={'romm': 'rommapp/romm:5.3.1', 'romm-db': 'mariadb:11.4', 'romm-redis': 'redis:7-alpine'} readback A=True
+2026/09/24 06:42:31 update.go:722: [INFO] [stacks] update romm: ladder — step 2 of 3: romm=rommapp/romm:5.3.0, romm-db=mariadb:11.4, romm-redis=redis:7-alpine → romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine, from 90dd9d68258286ef.yml
+2026/09/24 06:42:33 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine)
+2026/09/24 06:43:28 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)
+2026/09/24 06:43:28 update.go:892: [INFO] [stacks] update romm: DONE in 57s
+
+DB: 2026-09-24 08:42:42+02:00 [Note] [Entrypoint]: MariaDB upgrade not required
+Version: '11.4.13-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
+
+files: 26:    image: rommapp/romm:5.3.1
+86:          memory: 768M
+102:    image: mariadb:11.4
+123:          memory: 384M
+132:    image: redis:7-alpine
+145:          memory: 128M
+---
+applied == steps/90dd9d68258286ef.yml
+applied != template
+
+steps line: {'hu': [('1', 'Hátralévő frissítési lépések: 1')], 'en': [('1', 'Update steps remaining: 1')]}
+08:43:46   [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
+08:43:46       +   0.0s  phase=safety-dump  label=Adatbázis pillanatkép…  err=None  hold=None
+08:43:48       +   2.1s  phase=pulling  label=Új verzió letöltése…  err=None  hold=None
+08:43:50       +   3.1s  phase=copying  label=Az adatok másolása a frissítés előtt…  err=None  hold=None
+08:44:02       +  15.4s  phase=starting  label=Indítás az új verzióval…  err=None  hold=None
+08:44:18       +  31.8s  phase=verifying  label=Működés ellenőrzése…  err=None  hold=None
+08:44:54       +  67.7s  phase=done  label=Frissítve  err=None  hold=None
+08:45:02     romm: login as the seeded user http=200 ok=True
+08:45:11 PRESS 2: phase=done pinned={'romm': 'rommapp/romm:5.3.1', 'romm-db': 'mariadb:11.8', 'romm-redis': 'redis:7-alpine'} readback A=True
+2026/09/24 06:43:46 update.go:722: [INFO] [stacks] update romm: ladder — the last step (3 of 3) — the catalog's current definition
+2026/09/24 06:43:48 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.8, romm-redis=redis:7-alpine)
+2026/09/24 06:44:53 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)
+2026/09/24 06:44:53 update.go:892: [INFO] [stacks] update romm: DONE in 1m7s
+
+DB: Version: '11.8.9-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
+2026-09-24 08:44:05+02:00 [Note] [Entrypoint]: Starting mariadb-upgrade
+The --upgrade-system-tables option was used, user tables won't be touched.
+Major version upgrade detected from 11.4.13-MariaDB to 11.8.9-MariaDB. Check required!
+2026-09-24 08:44:12+02:00 [Note] [Entrypoint]: Finished mariadb-upgrade
+Version: '11.8.9-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
+
+files: 26:    image: rommapp/romm:5.3.1
+86:          memory: 768M
+102:    image: mariadb:11.8
+123:          memory: 384M
+132:    image: redis:7-alpine
+145:          memory: 128M
+---
+applied != steps/90dd…
+applied == template
+
+steps line: {'hu': [], 'en': []}
@@ -0,0 +1,27 @@
+08:45:32   [X] stop -> 200 {'ok': True, 'message': 'Stack romm stop completed'}
+08:45:37   [X] remove (with drive data) -> 409 {'ok': False, 'error': 'A(z) /mnt/felhom-drives/scratch_hdd/userdata/romm tárhely jelenleg nem elérhető — az alkalmazás nem távolítható el, amíg a meghajtó viss
+08:45:37   [X] refused because the drive path cannot be resolved (R-442, fail-closed and right) — removing the app and KEEPING the drive data instead
+08:46:04   [X] remove (keeping drive data) -> 200 {'ok': True, 'data': {'removed': 'romm', 'volumes_removed': ['romm_romm_config', 'romm_romm_db_data', 'romm_romm_redis_data'], 'hdd_paths_removed': [], 'hdd_pat
+08:46:12   [X] after remove: deployed=False leftovers='/opt/docker/stacks/romm'
+08:45:32   [X] stop -> 200 {'ok': True, 'message': 'Stack romm stop completed'}
+08:45:37   [X] remove (with drive data) -> 409 {'ok': False, 'error': 'A(z) /mnt/felhom-drives/scratch_hdd/userdata/romm tárhely jelenleg nem elérhető — az alkalmazás nem távolítható el, amíg a meghajtó viss
+08:45:37   [X] refused because the drive path cannot be resolved (R-442, fail-closed and right) — removing the app and KEEPING the drive data instead
+08:46:04   [X] remove (keeping drive data) -> 200 {'ok': True, 'data': {'removed': 'romm', 'volumes_removed': ['romm_romm_config', 'romm_romm_db_data', 'romm_romm_redis_data'], 'hdd_paths_removed': [], 'hdd_pat
+08:46:12   [X] after remove: deployed=False leftovers='/opt/docker/stacks/romm'
+no-romm-volumes
+total 36
+drwxr-xr-x  3 root root  4096 Sep 24 06:46 .
+drwxr-xr-x 57 root root  4096 Sep 13 20:22 ..
+-rw-r--r--  1 root root 12798 Sep 23 22:15 .felhom.yml
+-rw-r--r--  1 root root  5771 Sep 24 06:43 docker-compose.yml
+drwxr-xr-x 11 root root  4096 Sep 23 10:13 hold-logs
+ls: cannot access '/mnt/felhom-drives/scratch_hdd/userdata/romm': No such file or directory
+felhom-controller Up 29 minutes (healthy)
+privatebin Up 6 hours (healthy)
+paperless-webserver Up 6 hours (healthy)
+paperless-postgres Up 6 hours (healthy)
+paperless-redis Up 6 hours (healthy)
+gokapi Restarting (1) 58 seconds ago
+filebrowser Up 10 hours (healthy)
+traefik Up 10 hours
+
@@ -0,0 +1,28 @@
+git:
+    branch: main
+    repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
+    sync_interval: 15m
+    token: <redacted>
+    username: ""
+hub:
+0
+
+/dev/loop1       69G   18G   48G  27% /var/lib/docker
+0
+removed vikunja/vikunja:2.4.0
+removed vikunja/vikunja:2.5.0
+removed vikunja/vikunja:2.6.0
+removed nextcloud:34.0.1-apache
+removed nextcloud:34.0.4-apache
+removed mariadb:12.3
+removed rommapp/romm:5.3.0
+removed rommapp/romm:5.3.1
+removed mariadb:11.4
+removed mariadb:11.8
+removed actualbudget/actual-server:26.9.0
+/dev/loop1       69G   12G   54G  18% /var/lib/docker
+ref: refs/heads/main
+5ed599cd5f50
+
+drill=5ed599cd5f50 live=5ed599cd5f50
+drill has_actions: False private: True
@@ -0,0 +1,9 @@
+--- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy (0.01s)
+    --- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy/files_/_unit_only_(round_11) (0.00s)
+        r659_whole_copy_test.go:95: no whole copy exists, yet the hold names "A frissítés nem sikerült, és az automatikus visszaállítás sem. Az adatok az új változat által hagyott állapotban vannak. A(z) nextcloud frissítése 2026-09-23 23:42-kor nem sikerült, és az alkalmazás nem indult el az új verzióval. Az alkalmazás biztonsági okból leállítva marad, hogy az adatai ne sérüljenek. Visszaállítható a Mentések oldalon ebből a biztonsági mentésből: saját meghajtó, 2026-09-23 21:42 — ez a másolat csak a beállításokat és az adatbázist tartalmazza, a fájlokat nem." (none=false)
+    --- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy/files_/_second_drive_only (0.00s)
+        r659_whole_copy_test.go:95: no whole copy exists, yet the hold names "A frissítés nem sikerült, és az automatikus visszaállítás sem. Az adatok az új változat által hagyott állapotban vannak. A(z) nextcloud frissítése 2026-09-23 23:42-kor nem sikerült, és az alkalmazás nem indult el az új verzióval. Az alkalmazás biztonsági okból leállítva marad, hogy az adatai ne sérüljenek. Visszaállítható a Mentések oldalon ebből a biztonsági mentésből: második meghajtó, 2026-09-23 03:42 — ez a másolat a beállításokat, az adatbázist és a fájlokat tartalmazza." (none=false)
+    --- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy/files_/_unit_+_second_drive (0.00s)
+        r659_whole_copy_test.go:95: no whole copy exists, yet the hold names "A frissítés nem sikerült, és az automatikus visszaállítás sem. Az adatok az új változat által hagyott állapotban vannak. A(z) nextcloud frissítése 2026-09-23 23:42-kor nem sikerült, és az alkalmazás nem indult el az új verzióval. Az alkalmazás biztonsági okból leállítva marad, hogy az adatai ne sérüljenek. Visszaállítható a Mentések oldalon ebből a biztonsági mentésből: saját meghajtó, 2026-09-23 21:42 — ez a másolat csak a beállításokat és az adatbázist tartalmazza, a fájlokat nem." (none=false)
+    --- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy/files_/_off-site_+_unit (0.00s)
+ok  	gitea.dooplex.hu/admin/felhom-controller/internal/backup	0.019s
@@ -0,0 +1,6 @@
+--- FAIL: TestR659_HeldPage_NoWholeCopyOffersNoRestoreButton (0.24s)
+    r659_held_page_test.go:33: stacks: a hold naming no whole copy still offers the Mentések button — the restore there would refuse
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-controller/internal/web	0.252s
+FAIL
+ok  	gitea.dooplex.hu/admin/felhom-controller/internal/web	0.283s
@@ -0,0 +1,9 @@
+FAIL
+FAIL: test_the_apps_own_folders_are_cleared_and_said (__main__.ClearScratch.test_the_apps_own_folders_are_cleared_and_said)
+AssertionError: True is not false : the last run's files are still there
+FAIL: test_every_request_errored_is_inconclusive (__main__.LoadVerdict.test_every_request_errored_is_inconclusive)
+AssertionError: 'reached' != 'inconclusive'
+FAIL: test_under_half_is_inconclusive_and_none_is_inconclusive (__main__.LoadVerdict.test_under_half_is_inconclusive_and_none_is_inconclusive)
+AssertionError: 'reached' != 'inconclusive'
+FAILED (failures=3)
+OK
@@ -0,0 +1,5 @@
+-- probe-matches-compose: the decoys (the label moves, the fact does not)
+  XX  FACT: a two-step ladder with NO steps/ file for the first step rc=0 (expected 1)
+  XX  DECOY: the steps/ file has the right NAME and names the head's image rc=0 (expected 1)
+  XX  DECOY: the step's definition sits beside the template under another name rc=0 (expected 1)
+test-record gate — 53 template(s) read, 25 carry a ladder, 0 convicted
@@ -0,0 +1,5 @@
+FAIL
+FAIL: test_a_second_step_keeps_the_first_steps_definition (__main__.WriterTest.test_a_second_step_keeps_the_first_steps_definition)
+AssertionError: False is not true : WROTE navidrome: {'navidrome': 'deluan/navidrome:0.64.1-next'} -> {'navidrome': 'deluan/navidrome:0.64.1-after'} peak 41.0% marks {'files_may_change': False, 'needs_person': None, 'memory_tight': False}
+FAILED (failures=1)
+OK
@@ -0,0 +1,7 @@
+--- FAIL: TestLadder_TwoPressesTwoSteps (0.01s)
+    ladder_test.go:83: press 1 pinned nextcloud:34.0.1-apache, want the tested step B nextcloud:33.0.0-apache — one press must be one step
+--- FAIL: TestLadder_FailedStepStopsTheLadder (0.01s)
+    ladder_test.go:152: C was brought up after B failed: ups=[nextcloud:34.0.1-apache nextcloud:31.0.14-apache]
+--- FAIL: TestLadder_MissingStepFileRefusesBeforeAnythingMoves (0.01s)
+    ladder_test.go:169: phase="done" key="", want failed/pin_failed
+ok  	gitea.dooplex.hu/admin/felhom-controller/internal/stacks	0.051s
@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+"""Part A, first half — on controller v0.267.0 (the field's version): vikunja installed, seeded (A),
+backed up by the product (the update's own `backing-up` leg, then a pull that fails: nothing moves),
+RESTORED from its own unit — which recreates its volumes WITHOUT compose labels (R-658) — and seeded
+again (B, after the restore). Evidence only; every act is a product endpoint."""
+import json, re, sys, time
+sys.path.insert(0, ".")
+import walk as w
+import fixtures as fx
+
+ST = "../partA/state.json"
+F = fx.Vikunja()
+SUB = "vikunja-la"
+out = {"controller": w.guest("cat /etc/felhom-controller-image").strip()}
+w.login()
+
+
+def drill_set(ref, msg):
+    import fcntl
+    with open(f"{w.SC}/drill.lock", "w") as lk:
+        fcntl.flock(lk, fcntl.LOCK_EX)
+        w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
+        p = f"{w.DRILL}/templates/vikunja/docker-compose.yml"
+        s = open(p).read()
+        s = re.sub(r"image: vikunja/vikunja:\S+", f"image: vikunja/vikunja:{ref}", s)
+        open(p, "w").write(s)
+        w.sh(["git", "-C", w.DRILL, "commit", "-qam", f"LIVE-A vikunja {ref}: {msg}"])
+        r = w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
+        h = w.sh(["git", "-C", w.DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
+        w.say(f"drill {h}: vikunja {ref} ({msg}) push rc={r.returncode}")
+        return h
+
+
+def labels():
+    return w.guest("for v in $(docker volume ls -q | grep '^vikunja_'); do echo \"$v $(docker volume inspect $v --format '{{json .Labels}}')\"; done")
+
+
+out["drill_A"] = drill_set("2.5.0", "the install version")
+w.sync_rescan()
+out["deployed"] = w.deploy("vikunja", SUB)
+w.wait_app(SUB, "/", tries=60, delay=3)
+A = F.seed(w, SUB, w.say)
+out["seed_A"] = bool(A)
+out["labels_after_install"] = labels()
+w.say("labels after install:\n" + out["labels_after_install"])
+
+out["drill_E"] = drill_set("2.5.99-notag", "a tag that does not exist: backing-up runs, the pull fails, nothing moves")
+out["badge_wait"] = w.sync_rescan("vikunja", "vikunja/vikunja:2.5.99-notag")
+out["press_backup"] = w.press_update("vikunja")
+out["snapshots"] = w.snapshots("vikunja")
+w.say("snapshots offered: " + json.dumps(out["snapshots"])[:400])
+out["drill_back"] = drill_set("2.5.0", "back to the install version")
+w.sync_rescan()
+
+out["restore"] = w.restore("vikunja")
+out["labels_after_restore_v0267"] = labels()
+w.say("labels after the v0.267.0 restore:\n" + out["labels_after_restore_v0267"])
+w.wait_app(SUB, "/", tries=60, delay=3)
+out["A_after_restore"] = bool(F.verify(w, SUB, A, w.say))
+B = F.seed(w, SUB, w.say)
+out["seed_B_after_restore"] = bool(B)
+json.dump({"A": A, "B": B, "sub": SUB}, open(ST, "w"))
+json.dump(out, open("../partA/01-v0267-install-backup-restore.json", "w"), indent=2, ensure_ascii=False, default=str)
+open("../partA/01-v0267-install-backup-restore.log", "w").write("\n".join(w.LOG) + "\n")
@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+"""Part A, second half — on controller v0.268.0. vikunja's volumes are UNLABELLED (restored under v0.267.0,
+liveA1). (e) a failing update (real move 2.5.0 -> 2.6.0 + a probe port the app does not answer): the undo
+must copy BOTH volumes and seeds A and B must read back. (f) a restore under v0.268.0: the recreated
+volumes carry compose's labels. (g) seed C, the same failing update again: undone, both copied, A B C read
+back. (h) remove: volumes_removed names both, no applied-compose.yml / applied-meta/ left (R-651)."""
+import json, re, sys, time
+sys.path.insert(0, ".")
+import walk as w
+import fixtures as fx
+
+S = json.load(open("../partA/state.json"))
+F, SUB = fx.Vikunja(), S["sub"]
+out = {"controller": w.guest("cat /etc/felhom-controller-image").strip()}
+w.login()
+
+
+def drill(edit, msg):
+    import fcntl
+    with open(f"{w.SC}/drill.lock", "w") as lk:
+        fcntl.flock(lk, fcntl.LOCK_EX)
+        w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
+        edit()
+        w.sh(["git", "-C", w.DRILL, "commit", "-qam", "LIVE-A " + msg])
+        r = w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
+        h = w.sh(["git", "-C", w.DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
+        w.say(f"drill {h}: {msg} push rc={r.returncode}")
+        return h
+
+
+def set_vik(ref, port):
+    p = f"{w.DRILL}/templates/vikunja/docker-compose.yml"
+    s = re.sub(r"image: vikunja/vikunja:\S+", f"image: vikunja/vikunja:{ref}", open(p).read())
+    open(p, "w").write(s)
+    fy = f"{w.DRILL}/templates/vikunja/.felhom.yml"
+    f = open(fy).read()
+    f2 = re.sub(r"(healthcheck:\n(?:.*\n){0,8}?\s+port: )\d+", lambda m: m.group(1) + str(port), f, count=1)
+    assert f2 != f or str(port) in f, "probe port not found"
+    open(fy, "w").write(f2)
+
+
+def labels():
+    return w.guest("for v in $(docker volume ls -q | grep '^vikunja_'); do echo \"$v $(docker volume inspect $v --format '{{json .Labels}}')\"; done")
+
+
+def ctl_log(since):
+    return w.guest(f"docker logs --since {since} felhom-controller 2>&1 | grep -E 'vikunja' | grep -E 'undo copy will hold|carry no compose label|ladder|copied|UNDONE|UNDO|RestoreStack|Restoring Docker volume|was not created|volume .* already exists|created WITHOUT' | tail -30")
+
+
+def readall(tag, seeds):
+    w.wait_app(SUB, "/", tries=60, delay=3)
+    r = {k: bool(F.verify(w, SUB, v, w.say)) for k, v in seeds.items()}
+    w.say(f"READBACK {tag}: {r}")
+    return r
+
+
+PORT_REAL = int(re.search(r"port: (\d+)", open(f"{w.DRILL}/templates/vikunja/.felhom.yml").read().split("healthcheck:")[1]).group(1))
+out["probe_port_real"] = PORT_REAL
+out["labels_before"] = labels()
+w.say("labels before (unlabelled after the v0.267.0 restore):\n" + out["labels_before"])
+
+# (e)
+t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+out["drill_e"] = drill(lambda: set_vik("2.6.0", 8999), "vikunja 2.6.0 + probe 8999 (the failing edge)")
+out["badge_e"] = w.sync_rescan("vikunja", "vikunja/vikunja:2.6.0")
+out["press_e"] = w.press_update("vikunja")
+out["log_e"] = ctl_log(t0)
+w.say("controller log (e):\n" + out["log_e"])
+out["read_e"] = readall("after the undo (e)", {"A": S["A"], "B": S["B"]})
+out["obs_e"] = w.observables("vikunja")
+
+# (f)
+t1 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+out["restore_f"] = w.restore("vikunja")
+out["labels_after_restore_v0268"] = labels()
+w.say("labels after the v0.268.0 restore:\n" + out["labels_after_restore_v0268"])
+out["log_f"] = ctl_log(t1)
+out["compose_warn_f"] = w.guest(f"docker logs --since {t1} felhom-controller 2>&1 | grep -iE 'not created by Docker Compose|already exists|Recreate' | tail -5")
+w.say("compose warnings after the restore (want none): " + (out["compose_warn_f"].strip() or "(none)"))
+out["read_f"] = readall("after the v0.268.0 restore (f)", {"A": S["A"]})
+
+# (g)
+C = F.seed(w, SUB, w.say)
+t2 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+out["press_g"] = w.press_update("vikunja")
+out["log_g"] = ctl_log(t2)
+w.say("controller log (g):\n" + out["log_g"])
+out["read_g"] = readall("after the second undo (g)", {"A": S["A"], "C": C})
+
+# (h)
+out["drill_h"] = drill(lambda: set_vik("2.5.0", PORT_REAL), "vikunja back to 2.5.0 + its real probe")
+out["remove_h"] = w.remove("vikunja")
+out["leftovers_h"] = w.guest("ls -la /opt/docker/stacks/vikunja/; docker volume ls -q | grep -E '^vikunja' || echo 'no vikunja volumes'")
+w.say("after remove:\n" + out["leftovers_h"])
+json.dump(out, open("../partA/02-v0268-undo-after-restore.json", "w"), indent=2, ensure_ascii=False, default=str)
+open("../partA/02-v0268-undo-after-restore.log", "w").write("\n".join(w.LOG) + "\n")
@@ -0,0 +1,117 @@
+#!/usr/bin/env python3
+"""Part B — R-659 on controller v0.268.0: chaos round 11 reproduced. nextcloud (declared drive files),
+a failing update (34.0.1 -> 34.0.4 + a probe port it does not answer), the undo copy CUT OFF during
+`verifying` (its finished-marker removed — `09` §6.1a's measured case) -> HOLD. No off-site tier on 9202.
+Expected: the no-whole-copy sentence in hu and en, no Mentések button beside it, the operator event
+(DROPPED at WARN on this hub-less box — R-620), and NO app_start_failed after the hold (R-660)."""
+import html as H, json, re, sys, time
+sys.path.insert(0, ".")
+import walk as w
+import fixtures as fx
+
+F, SUB, APP = fx.Nextcloud(), "cloud-lb", "nextcloud"
+CAT = f"{w.DRILL}/templates/{APP}"
+out = {"controller": w.guest("cat /etc/felhom-controller-image").strip()}
+w.login()
+
+
+def drill(edit, msg):
+    import fcntl
+    with open(f"{w.SC}/drill.lock", "w") as lk:
+        fcntl.flock(lk, fcntl.LOCK_EX)
+        w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
+        edit()
+        w.sh(["git", "-C", w.DRILL, "commit", "-qam", "LIVE-B " + msg])
+        r = w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
+        h = w.sh(["git", "-C", w.DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
+        w.say(f"drill {h}: {msg} push rc={r.returncode}")
+        return h
+
+
+HEAD = open(f"{CAT}/docker-compose.yml").read()
+STEP1 = open(f"{CAT}/steps/eef2e4afe1218021.yml").read()
+FY = open(f"{CAT}/.felhom.yml").read()
+
+
+def set_compose(body):
+    return lambda: open(f"{CAT}/docker-compose.yml", "w").write(body)
+
+
+def set_probe(port):
+    def _e():
+        f = open(f"{CAT}/.felhom.yml").read()
+        open(f"{CAT}/.felhom.yml", "w").write(re.sub(r"(healthcheck:\n\s+checks:\n\s+- type: api\n\s+port: )\d+", lambda m: m.group(1) + str(port), f, count=1))
+    return _e
+
+
+out["drill_1"] = drill(set_compose(STEP1), "nextcloud template = ladder step 1's own definition (34.0.1 / mariadb 12.3)")
+w.sync_rescan()
+out["deployed"] = w.deploy(APP, SUB)
+A = F.seed(w, SUB, w.say)
+out["seed_A"] = bool(A)
+out["drill_2"] = drill(lambda: (set_compose(HEAD)(), set_probe(8999)()), "nextcloud back to the head (34.0.4) + probe port 8999")
+out["badge_wait"] = w.sync_rescan(APP, "nextcloud:34.0.4-apache", tries=60)
+h = H.unescape(w.page(f"/apps/{APP}"))
+out["steps_left_line_hu"] = re.findall(r'data-ladder-steps="(\d+)">([^<]*)<', h)
+out["steps_left_line_en"] = re.findall(r'data-ladder-steps="(\d+)">([^<]*)<', H.unescape(w.page(f"/apps/{APP}?lang=en")))
+w.say(f"steps-left line hu={out['steps_left_line_hu']} en={out['steps_left_line_en']}")
+
+cut = {}
+
+
+def on_phase(ph):
+    if ph == "verifying" and not cut:
+        cut["out"] = w.guest(f"""c=$(docker volume ls -q --filter label=felhom.undo-copy-of={APP} | head -1); echo "copy: $c"
+docker run --rm -v $c:/c alpine sh -c 'rm -f /c/felhom-undo-complete; ls /c | head -5'""")
+        w.say("  >>> finished-marker removed from one undo copy:\n" + cut["out"])
+
+
+t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+code, d = w.ctl("POST", f"/api/stacks/{APP}/update")
+w.say(f"Update -> {code}")
+phases, seen, tp = [], None, time.time()
+while time.time() - tp < 1500:
+    st = w.stack(APP)
+    ph = st.get("update_phase")
+    if ph != seen and ph:
+        seen = ph
+        phases.append((round(time.time() - tp, 1), ph))
+        w.say(f"  +{phases[-1][0]:6.1f}s phase={ph}")
+        on_phase(ph)
+    if not st.get("updating") and ph in ("done", "failed", "undone") and time.time() - tp > 3:
+        break
+    time.sleep(0.5)
+held_at = time.time()
+out["phases"], out["cutoff"] = phases, cut.get("out")
+st = w.stack(APP)
+out["api"] = {k: st.get(k) for k in ("state", "update_phase", "hold_reason", "hold_no_whole_copy", "update_error")}
+w.say("API: " + json.dumps(out["api"], ensure_ascii=False))
+for lang in ("hu", "en"):
+    page = H.unescape(w.page(f"/apps/{APP}?lang={lang}"))
+    m = re.search(r'data-held="true">(.*?)</div>', page, re.S)
+    out[f"app_page_{lang}"] = m.group(1).strip() if m else None
+    out[f"app_page_{lang}_restore_button"] = bool(m and 'href="/backups/apps"' in m.group(1))
+    lst = H.unescape(w.page(f"/stacks?lang={lang}"))
+    m2 = re.search(r'data-held="true">(.*?)</div>', lst, re.S)
+    out[f"stacks_page_{lang}_restore_button"] = bool(m2 and 'href="/backups/apps"' in m2.group(1))
+    w.say(f"[{lang}] app page held block: {out[f'app_page_{lang}']!r}  restore-button app={out[f'app_page_{lang}_restore_button']} list={out[f'stacks_page_{lang}_restore_button']}")
+# ASCII-fragment checks, with controls (the ui-hungarian rule)
+hu = out["app_page_hu"] or ""
+out["ascii_checks"] = {"hu has 'gyfelszolgalat' (positive)": "gyfélszolgálat" in hu,
+                       "hu has 'Visszaallithato a Mentesek' (must be absent)": "Visszaállítható a Mentések" in hu,
+                       "en has 'Felhom support has been told' (positive)": "Felhom support has been told" in (out["app_page_en"] or "")}
+w.say("checks: " + json.dumps(out["ascii_checks"]))
+
+# R-660: two and a half minutes of the dead-app scan after the hold
+while time.time() - held_at < 150:
+    time.sleep(10)
+out["log"] = w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -E 'nextcloud|DROPPED|dropped event' | grep -E 'NO copy|DROPPED|dropped event|HELD|UNDO|app_start_failed|no_whole' | tail -30")
+w.say("controller log:\n" + out["log"])
+c2, dl = w.ctl("GET", "/api/debug/logs?level=DEBUG&lines=4000")
+ents = ((dl.get("data") or {}).get("entries") or []) if isinstance(dl, dict) else []
+out["debug_ring_events"] = [e.get("timestamp", "") + " " + e.get("message", "") for e in ents
+                            if ("app_start_failed" in e.get("message", "") or "app_hold_no_whole_copy" in e.get("message", "")
+                                or "app_update_held" in e.get("message", "")) and e.get("timestamp", "") >= t0]
+w.say("debug ring (events since the press):\n  " + "\n  ".join(out["debug_ring_events"]))
+json.dump(out, open("../partB/01-round11-reproduced.json", "w"), indent=2, ensure_ascii=False, default=str)
+open("../partB/01-round11-reproduced.log", "w").write("\n".join(w.LOG) + "\n")
@@ -0,0 +1,29 @@
+#!/usr/bin/env python3
+"""R-660 positive control: a throwaway app (actualbudget) stopped OUT OF BAND must raise app_start_failed
+while the held nextcloud (held since 06:27:40Z) must not. Then the throwaway is removed through the product."""
+import json, sys, time
+sys.path.insert(0, ".")
+import walk as w
+w.login()
+t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+ok = w.deploy("actualbudget", "budget-c660")
+w.wait_app("budget-c660", "/", tries=40, delay=3)
+w.say("stopping the throwaway's container out of band (not through the product): " + w.guest("docker stop actualbudget 2>&1").strip())
+found = None
+for i in range(24):
+    time.sleep(10)
+    c, d = w.ctl("GET", "/api/debug/logs?level=DEBUG&lines=6000")
+    ents = ((d.get("data") or {}).get("entries") or []) if isinstance(d, dict) else []
+    evs = [e["timestamp"] + " " + e["message"] for e in ents if e.get("timestamp", "") >= t0 and
+           ("app_start_failed" in e.get("message", "") or "currently down" in e.get("message", ""))]
+    if any("app_start_failed" in x for x in evs):
+        found = evs
+        break
+w.say("events/heartbeats since the control started:\n  " + "\n  ".join(found or evs))
+names = w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -iE 'app_start_failed|not running|DeadApp|nem fut' | tail -8")
+w.say("controller log lines:\n" + names)
+st = w.stack("nextcloud")
+w.say(f"nextcloud meanwhile: state={st.get('state')} held={bool(st.get('hold_reason'))}")
+w.remove("actualbudget")
+json.dump({"events": found or evs, "log": names, "nextcloud_state": st.get("state")}, open("../partB/03-r660-control.json", "w"), indent=2)
+open("../partB/03-r660-control.log", "w").write("\n".join(w.LOG) + "\n")
@@ -0,0 +1,26 @@
+import sys, time, re, html as H, json
+sys.path.insert(0, ".")
+import walk as w
+w.login()
+w.deploy("actualbudget", "budget-c660b")
+w.wait_app("budget-c660b", "/", tries=40, delay=3)
+t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+w.say("out-of-band stop: " + w.guest("docker stop actualbudget 2>&1").strip())
+seen = []
+for i in range(20):
+    time.sleep(5)
+    h = H.unescape(w.page("/"))
+    txt = re.sub(r"\s+", " ", re.sub("<[^>]+>", " ", h))
+    for name in ("actualbudget", "Actual", "nextcloud", "Nextcloud", "gokapi", "Gokapi"):
+        pass
+    m = re.findall(r"(nem fut[^.]{0,200})", txt)
+    c, d = w.ctl("GET", "/api/debug/logs?level=DEBUG&lines=6000")
+    ents = ((d.get("data") or {}).get("entries") or [])
+    ev = [e["timestamp"] + " " + e["message"] for e in ents if e.get("timestamp", "") >= t0 and "app_start_failed" in e["message"]]
+    if m or ev:
+        seen.append({"t": i * 5, "banner": m[:3], "events": ev})
+        w.say(f"+{i*5}s banner={m[:3]} events={len(ev)}")
+    if ev and m:
+        break
+json.dump(seen, open("../partB/06-r660-control-banner.json", "w"), indent=2, ensure_ascii=False)
+w.remove("actualbudget")
@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+"""Part D live — the ladder on controller v0.268.0. romm installed at 5.3.0 / mariadb 11.4 (the ladder's
+step-1 definition, steps/7f9c6a74b5891f8e.yml, served as the drill template for the install only); the
+drill template then back to the head (5.3.1 / 11.8). Two presses: press 1 must apply ONLY the app step
+(5.3.1 / 11.4, from steps/90dd9d68258286ef.yml), press 2 the engine step (11.4 -> 11.8, the template).
+Seeded data read back after each press; the page's steps-left line read in both languages each time."""
+import html as H, json, re, sys, time
+sys.path.insert(0, ".")
+import walk as w
+import fixtures as fx
+
+F, SUB, APP = fx.Romm(), "arcade-ld", "romm"
+CAT = f"{w.DRILL}/templates/{APP}"
+out = {"controller": w.guest("cat /etc/felhom-controller-image").strip()}
+w.login()
+
+
+def drill(body, msg):
+    import fcntl
+    with open(f"{w.SC}/drill.lock", "w") as lk:
+        fcntl.flock(lk, fcntl.LOCK_EX)
+        w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
+        open(f"{CAT}/docker-compose.yml", "w").write(body)
+        w.sh(["git", "-C", w.DRILL, "commit", "-qam", "LIVE-D " + msg])
+        r = w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
+        h = w.sh(["git", "-C", w.DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
+        w.say(f"drill {h}: {msg} push rc={r.returncode}")
+        return h
+
+
+def steps_line():
+    r = {}
+    for lang, q in (("hu", ""), ("en", "?lang=en")):
+        r[lang] = re.findall(r'data-ladder-steps="(\d+)">([^<]*)<', H.unescape(w.page(f"/apps/{APP}{q}")))
+    return r
+
+
+def box_files():
+    return w.guest(f"grep -nE 'image:|memory:' /opt/docker/stacks/{APP}/docker-compose.yml; echo ---; cmp -s /opt/docker/stacks/{APP}/applied-compose.yml /var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache/templates/{APP}/steps/90dd9d68258286ef.yml && echo 'applied == steps/90dd9d68258286ef.yml' || echo 'applied != steps/90dd…'; cmp -s /opt/docker/stacks/{APP}/applied-compose.yml /var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache/templates/{APP}/docker-compose.yml && echo 'applied == template' || echo 'applied != template'")
+
+
+HEAD = open(f"{CAT}/docker-compose.yml").read()
+STEP1 = open(f"{CAT}/steps/7f9c6a74b5891f8e.yml").read()
+out["drill_install"] = drill(STEP1, "romm template = step 1's own definition (5.3.0 / mariadb 11.4) for the install")
+w.sync_rescan()
+out["deployed"] = w.deploy(APP, SUB)
+A = F.seed(w, SUB, w.say)
+out["seed_A"] = bool(A)
+out["drill_head"] = drill(HEAD, "romm back to the head (5.3.1 / mariadb 11.8)")
+out["badge_wait"] = w.sync_rescan(APP, "mariadb:11.8", tries=60)
+out["before"] = {"obs": w.observables(APP), "steps_line": steps_line(), "badges": w.badges(APP)}
+w.say("before: steps line " + json.dumps(out["before"]["steps_line"], ensure_ascii=False) + " badges " + json.dumps(out["before"]["badges"], ensure_ascii=False)[:300])
+
+for n in (1, 2):
+    t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+    p = w.press_update(APP)
+    obs = w.observables(APP)
+    w.wait_app(SUB, "/api/heartbeat", want=("200",), tries=60, delay=5)
+    rb = bool(F.verify(w, SUB, A, w.say))
+    log = w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -E 'update romm' | grep -E 'ladder|pin advanced|DONE|UNDO|healthy after' | tail -8")
+    dblog = w.guest(f"docker logs --since {t0} romm-db 2>&1 | grep -iE 'upgrade|mariadb-upgrade|Version:' | tail -6")
+    B = F.seed(w, SUB, w.say) if n == 1 else None
+    out[f"press{n}"] = {"press": p, "obs": obs, "readback_A": rb, "log": log, "db_log": dblog,
+                        "files": box_files(), "steps_line": steps_line(), "badges": w.badges(APP)}
+    if B:
+        out["seed_B"] = B
+    w.say(f"PRESS {n}: phase={p.get('final_phase')} pinned={obs.get('pinned_images')} readback A={rb}\n{log}\nDB: {dblog}\nfiles: {out[f'press{n}']['files']}\nsteps line: {out[f'press{n}']['steps_line']}")
+if out.get("seed_B"):
+    out["readback_B_after_press2"] = bool(F.verify(w, SUB, out["seed_B"], w.say))
+json.dump(out, open("../partD/10-romm-two-steps.json", "w"), indent=2, ensure_ascii=False, default=str)
+open("../partD/10-romm-two-steps.log", "w").write("\n".join(w.LOG) + "\n")
@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+"""Point guest 9202 at the drill catalog (and a 90 s health timeout), or restore the saved config.
+
+`09` §6.5: `git.repo_url` alone is INERT (R-615) — the cache dir must go too. The saved copy is
+`controller.yaml.pre-ladder0924` (NOT the older `.pre-28`, which a restore must never pick up).
+"""
+import re, sys, io
+sys.path.insert(0, '.')
+import walk as w
+
+VOL = "/var/lib/docker/volumes/felhom-controller-data/_data"
+DRILL_REPO = "https://gitea.dooplex.hu/admin/app-catalog-drill.git"
+
+
+def creds():
+    for l in io.open("/home/kisfenyo/.git-credentials").read().strip().split("\n"):
+        m = re.match(r'https://(admin):([^@]+)@gitea\.dooplex\.hu', l)
+        if m:
+            return m.group(1), m.group(2)
+    raise SystemExit("no admin credential")
+
+
+def to_drill():
+    u, t = creds()
+    print(w.guest(f"""
+set -e
+test -f {VOL}/controller.yaml.pre-ladder0924 || cp -p {VOL}/controller.yaml {VOL}/controller.yaml.pre-ladder0924
+python3 - <<'PY'
+import re
+p = "{VOL}/controller.yaml"
+s = open(p).read()
+s = re.sub(r'(^\\s+repo_url: ).*$', r'\\g<1>{DRILL_REPO}', s, count=1, flags=re.M)
+s = re.sub(r'(^git:(?:\\n\\s+.*)*?\\n\\s+token: ).*$', r'\\g<1>"{t}"', s, count=1, flags=re.M)
+s = re.sub(r'(^git:(?:\\n\\s+.*)*?\\n\\s+username: ).*$', r'\\g<1>"{u}"', s, count=1, flags=re.M)
+if not re.search(r'^update:', s, re.M):
+    s += "update:\\n    health_timeout: 90s\\n"
+open(p, "w").write(s)
+PY
+rm -rf {VOL}/catalog-cache {VOL}/data/catalog-cache
+docker restart felhom-controller >/dev/null
+sleep 15
+grep -A6 '^git:' {VOL}/controller.yaml | sed 's/token:.*/token: <redacted>/'
+grep -A2 '^update:' {VOL}/controller.yaml
+"""))
+
+
+def restore():
+    print(w.guest(f"""
+set -e
+cp -p {VOL}/controller.yaml.pre-ladder0924 {VOL}/controller.yaml
+rm -rf {VOL}/catalog-cache {VOL}/data/catalog-cache
+docker restart felhom-controller >/dev/null
+sleep 15
+grep -A6 '^git:' {VOL}/controller.yaml | sed 's/token:.*/token: <redacted>/'
+grep -c '^update:' {VOL}/controller.yaml || true
+"""))
+
+
+if __name__ == "__main__":
+    to_drill() if sys.argv[1] == "drill" else restore()
@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+"""Part D spike (30 min) on guest 9202, controller v0.267.0, drill catalog.
+
+1. an app two steps behind (vikunja A=2.4.0 -> B=2.5.0 -> C=2.6.0): does ONE press jump A -> C today?
+2. does a `templates/<app>/steps/` folder reach the box? (it is read from the catalog clone)
+Evidence only; every act is a product endpoint."""
+import json, os, sys, time
+sys.path.insert(0, ".")
+import walk as w
+
+DRILL = w.DRILL
+CACHE = "/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache"
+
+
+def commit(msg, edit):
+    edit()
+    w.sh(["git", "-C", DRILL, "add", "-A"])
+    w.sh(["git", "-C", DRILL, "commit", "-q", "-m", msg])
+    r = w.sh(["git", "-C", DRILL, "push", "-q", "origin", "main"], timeout=120)
+    h = w.sh(["git", "-C", DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
+    w.say(f"drill commit {h}: {msg} (push rc={r.returncode})")
+    return h
+
+
+def set_img(ref):
+    p = f"{DRILL}/templates/vikunja/docker-compose.yml"
+    s = open(p).read()
+    import re
+    s = re.sub(r"image: vikunja/vikunja:\S+", f"image: vikunja/vikunja:{ref}", s)
+    open(p, "w").write(s)
+
+
+out = {}
+commit("SPIKE vikunja A=2.4.0", lambda: set_img("2.4.0"))
+w.login()
+w.sync_rescan("vikunja", "vikunja/vikunja:2.4.0")
+ok = w.deploy("vikunja", "vikunja-sp")
+out["deployed_A"] = ok
+out["A"] = w.observables("vikunja")
+
+
+def step_b():
+    set_img("2.5.0")
+    os.makedirs(f"{DRILL}/templates/vikunja/steps", exist_ok=True)
+    open(f"{DRILL}/templates/vikunja/steps/spike-B.yml", "w").write(
+        open(f"{DRILL}/templates/vikunja/docker-compose.yml").read())
+
+
+commit("SPIKE vikunja B=2.5.0 + steps/spike-B.yml", step_b)
+commit("SPIKE vikunja C=2.6.0", lambda: set_img("2.6.0"))
+out["badge_wait_s"] = w.sync_rescan("vikunja", "vikunja/vikunja:2.6.0")
+out["cache_steps"] = w.guest(f"git -C {CACHE} rev-parse --short=12 HEAD; git -C {CACHE} rev-list --count HEAD; ls -la {CACHE}/templates/vikunja/ {CACHE}/templates/vikunja/steps/ 2>&1; ls /opt/docker/stacks/vikunja/")
+w.say("cache:\n" + out["cache_steps"])
+out["badges"] = w.badges("vikunja")
+out["press"] = w.press_update("vikunja")
+out["after"] = w.observables("vikunja")
+w.say("after: " + json.dumps(out["after"]))
+w.remove("vikunja")
+json.dump(out, open("../partD/00-spike.json", "w"), indent=2, ensure_ascii=False)
+open("../partD/00-spike.log", "w").write("\n".join(w.LOG) + "\n")
@@ -0,0 +1,21 @@
+import sys, json, re
+sys.path.insert(0, ".")
+import walk as w
+w.login()
+import fcntl
+CAT = f"{w.DRILL}/templates/nextcloud"
+with open(f"{w.SC}/drill.lock", "w") as lk:
+    fcntl.flock(lk, fcntl.LOCK_EX)
+    w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
+    f = open(f"{CAT}/.felhom.yml").read()
+    open(f"{CAT}/.felhom.yml", "w").write(re.sub(r"(healthcheck:\n\s+checks:\n\s+- type: api\n\s+port: )8999", r"\g<1>80", f, count=1))
+    w.sh(["git", "-C", w.DRILL, "commit", "-qam", "LIVE-B teardown: nextcloud real probe back"])
+    w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
+before = w.guest("docker volume ls -q | grep -E '^nextcloud' ; ls /opt/docker/stacks/nextcloud/")
+w.say("before remove:\n" + before)
+code = w.remove("nextcloud")
+after = w.guest("docker volume ls -q | grep -E '^nextcloud' || echo 'no nextcloud volumes (incl. undo copies)'; ls -la /opt/docker/stacks/nextcloud/; ls -d /mnt/felhom-drives/scratch_hdd/userdata/nextcloud /mnt/felhom-drives/scratch_hdd/appdata/nextcloud 2>&1")
+w.say("after remove:\n" + after)
+rm = w.guest("rm -rf /mnt/felhom-drives/scratch_hdd/userdata/nextcloud /mnt/felhom-drives/scratch_hdd/appdata/nextcloud && ls -d /mnt/felhom-drives/scratch_hdd/userdata/nextcloud 2>&1")
+w.say("drive folders removed by name (R-442 keeps them on 9202): " + rm.strip())
+open("../partB/07-teardown.log", "w").write("\n".join(w.LOG) + "\n")
@@ -0,0 +1,507 @@
+#!/usr/bin/env python3
+"""walk.py — ONE app's full update walk on guest 9202, through the product's own endpoints.
+
+EVIDENCE, NOT PRODUCT. It presses exactly the buttons a person presses:
+  POST /api/stacks/<n>/deploy · POST /api/sync · POST /api/stacks/rescan
+  POST /api/stacks/<n>/update · POST /api/stacks/<n>/remove
+and reads GET /api/stacks/<n>. No controller code exists for it.
+
+The walk, per `09` §6.4 and the update-night brief §4:
+  1 deploy from the DRILL catalog at the LIVE pin
+  2 seed through the app's OWN front door (R-156: never a volume, never SQL)
+  3 read the seed back            <- control C1; a fixture that cannot prove itself proves nothing
+  4 „Mentés most"
+  5 commit the real one-step bump to the DRILL repo, sync, rescan, read the badge in BOTH languages
+  6 press the guarded Update, record every phase with timestamps
+  7 read the seed back through the front door
+  8 the four version observables side by side
+  9 write the verdict record in `09`'s JSON shape
+
+`inconclusive` is a first-class verdict and is NEVER collapsed into `failed`.
+"""
+import argparse, json, os, re, subprocess, sys, time
+from datetime import datetime, timezone
+
+SC = "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/25947a06-40b7-43dd-8334-2519e02142da/scratchpad"
+EV = "/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/ladder-2026-09-24"
+DRILL = "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill"
+# GUEST=9201 selects demo-hp's hub-enabled guest (the mail proof); default 9202, the scratch guest.
+GUEST = os.environ.get("GUEST", "9202")
+BASE = {"9202": "https://192.168.0.114", "9201": "https://192.168.0.138"}[GUEST]
+DOMAIN = os.environ.get("DOMAIN", "enkisfelhom.hu")
+HOSTHDR = f"Host: felhom.{DOMAIN}"
+HP = "demo-hp"
+
+LOG = []
+
+
+def say(*a):
+    line = " ".join(str(x) for x in a)
+    ts = datetime.now().strftime("%H:%M:%S")
+    print(f"{ts} {line}", flush=True)
+    LOG.append(f"{ts} {line}")
+
+
+def sh(args, timeout=300, inp=None):
+    try:
+        return subprocess.run(args, capture_output=True, text=True, timeout=timeout, input=inp)
+    except (subprocess.TimeoutExpired, OSError) as e:
+        return subprocess.CompletedProcess(args, 124, "", f"{e}")
+
+
+def guest(script, timeout=600):
+    """Run a bash script inside guest 9202. Piped as a file — never as an argument (quoting)."""
+    # ONE TEMP FILE PER CALL (night 2026-09-23): the shared /tmp/w<guest>.sh swapped scripts under
+    # two concurrent walks (memory: guest-helper-shares-one-tmp-file).
+    import secrets as _s
+    t = f"/tmp/w{GUEST}-{os.getpid()}-{_s.token_hex(4)}.sh"
+    r = sh(["ssh", "-o", "ConnectTimeout=20", "-o", "StrictHostKeyChecking=accept-new", HP,
+            f"export LC_ALL=C; cat > {t}; pct push {GUEST} {t} {t} >/dev/null 2>&1; "
+            f"pct exec {GUEST} -- bash {t}; pct exec {GUEST} -- rm -f {t}; rm -f {t}"],
+           timeout=timeout, inp=script)
+    return r.stdout or ""
+
+
+def login():
+    pw = open(f"{SC}/.ctlpw").read().strip()
+    sh(["curl", "-sk", "-D", f"{SC}/hdr{os.getpid()}.txt", "-o", "/dev/null", "-H", HOSTHDR,
+        "-X", "POST", "--data-urlencode", f"password={pw}", f"{BASE}/login"])
+    h = open(f"{SC}/hdr{os.getpid()}.txt").read()
+    m = re.search(r"felhom_session=[A-Za-z0-9._-]+", h, re.I)
+    if not m:
+        sys.exit("login failed: no session cookie")
+    open(f"{SC}/sess{os.getpid()}.txt", "w").write(m.group(0))
+    r = sh(["curl", "-sk", "-L", "-H", HOSTHDR, "-H", f"Cookie: {m.group(0)}", f"{BASE}/"])
+    c = re.search(r'<meta name="csrf-token" content="([^"]+)"', r.stdout or "")
+    if not c:
+        sys.exit("login failed: no csrf token")
+    open(f"{SC}/csrf{os.getpid()}.txt", "w").write(c.group(1))
+
+
+def ctl(method, path, data=None, raw=False, tries=2):
+    """One controller API call. Re-logs in once on a 302/401 — the controller's session store is
+    in memory, so any controller restart during the night invalidates it silently."""
+    for attempt in range(tries):
+        sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip()
+        csrf = open(f"{SC}/csrf{os.getpid()}.txt").read().strip()
+        args = ["curl", "-sk", "-H", HOSTHDR, "-H", f"Cookie: {sess}", "-w", "\n%{http_code}"]
+        if method != "GET":
+            args += ["-H", f"X-CSRF-Token: {csrf}", "-H", "Content-Type: application/json",
+                     "-X", method]
+            if data is not None:
+                args += ["--data", json.dumps(data)]
+        args.append(f"{BASE}{path}")
+        r = sh(args)
+        body, _, code = (r.stdout or "").rpartition("\n")
+        if code.strip() in ("302", "401") and attempt + 1 < tries:
+            login()
+            continue
+        if raw:
+            return code.strip(), body
+        try:
+            return code.strip(), json.loads(body)
+        except Exception:
+            return code.strip(), {"_raw": body[:600]}
+    return code.strip(), {"_raw": body[:600]}
+
+
+def page(path):
+    sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip()
+    r = sh(["curl", "-sk", "-H", HOSTHDR, "-H", f"Cookie: {sess}", f"{BASE}{path}"])
+    return r.stdout or ""
+
+
+def app_curl(sub, path, *extra, method=None, data=None, timeout=45):
+    """A call to the APP's own front door on 9202 — the household's route, not ours."""
+    args = ["curl", "-sSk", "--max-time", str(timeout), "-H", f"Host: {sub}.{DOMAIN}",
+            "-w", "\n%{http_code}"]
+    if method:
+        args += ["-X", method]
+    if data is not None:
+        args += ["--data-binary", "@-"]
+    args += list(extra) + [f"{BASE}{path}"]
+    r = sh(args, timeout=timeout + 30, inp=data)
+    body, _, code = (r.stdout or "").rpartition("\n")
+    return r.returncode, code.strip(), body
+
+
+def stack(name):
+    _, d = ctl("GET", f"/api/stacks/{name}")
+    return (d.get("data") or {}) if isinstance(d, dict) else {}
+
+
+def wait_app(sub, path="/", want=("200", "302", "303", "401", "403"), tries=60, delay=5):
+    """Settling says the container runs; this says the APP answers. Not the same thing."""
+    last = None
+    for _ in range(tries):
+        rc, code, _ = app_curl(sub, path, timeout=15)
+        last = (rc, code)
+        if rc == 0 and code in want:
+            return True
+        time.sleep(delay)
+    say(f"    app never answered on {sub}{path} (last rc={last[0]} code={last[1]})")
+    return False
+
+
+# ------------------------------------------------------------------ the walk
+
+
+DRIVE = "/mnt/felhom-drives/scratch_hdd/userdata"
+
+# What THIS run generated for a deploy, per app. Deploy secrets are ENCRYPTED AT REST in
+# `app.yaml` (`ENC:…`), which is right and which means a fixture cannot read an app's admin
+# password back off the box — the household sees it once. So the value the harness itself
+# generated is kept here for the life of the run, and nowhere else.
+GENERATED = {}
+
+
+def deploy_values(name, sub):
+    """Fill EVERY required deploy field the way the wizard would, by asking the box what this app
+    asks for — `GET /api/stacks/<n>/deploy-fields` — instead of assuming DOMAIN+SUBDOMAIN.
+
+    Measured 2026-09-21: three apps in one batch refused at the deploy with a correct 400 because
+    a required field was absent — `HDD_PATH` (navidrome, audiobookshelf) and an admin password
+    (grafana). The refusals happen BEFORE anything is created (`deploy.go:324`), which is the only
+    reason this was safe to discover by running it (live-probes rule).
+
+    A `path` field must name a directory that ALREADY EXISTS (`deploy.go:330`), so one is made on
+    the scratch drive first — the same act the drive browser performs for a household.
+    """
+    code, d = ctl("GET", f"/api/stacks/{name}/deploy-fields")
+    fields = (((d.get("data") or {}).get("metadata") or {}).get("deploy_fields")) or []
+    values = {"DOMAIN": DOMAIN, "SUBDOMAIN": sub}
+    made = []
+    for f in fields:
+        ev, ty = f.get("env_var"), f.get("type")
+        if ev in values:
+            continue
+        # `type: password` is MANDATORY whatever `required` says — `deploy.go:305-312` refuses
+        # when the caller sends none, deliberately ("the user needs to know their password"),
+        # while `.felhom.yml` declares `required: false` and the API serves that verbatim. A
+        # caller that trusts the contract gets a 400. Measured tonight on grafana; filed.
+        if not f.get("required") and ty != "password":
+            continue                      # the controller generates the optional secrets itself
+        if ty == "path":
+            p = f"{DRIVE}/{name}"
+            values[ev] = p
+            made.append(p)
+        elif ty in ("secret", "password"):
+            import secrets as _s
+            values[ev] = "Drill-" + _s.token_hex(12)
+            GENERATED.setdefault(name, {})[ev] = values[ev]
+        elif f.get("default"):
+            values[ev] = f["default"]
+        else:
+            values[ev] = f"drill-{name}"
+    if made:
+        guest("mkdir -p " + " ".join(made) + "; ls -ld " + " ".join(made))
+        say(f"  [1] made the drive paths this app requires: {made}")
+    extra = [k for k in values if k not in ("DOMAIN", "SUBDOMAIN")]
+    if extra:
+        say(f"  [1] required fields filled beyond DOMAIN/SUBDOMAIN: {extra}")
+    return values
+
+
+def deploy(name, sub, extra_values=None):
+    st = stack(name)
+    if st.get("deployed"):
+        say(f"  [1] {name} already deployed — reusing")
+        return True
+    values = deploy_values(name, sub)
+    if extra_values:
+        values.update(extra_values)
+    code, d = ctl("POST", f"/api/stacks/{name}/deploy", {"values": values})
+    say(f"  [1] deploy -> {code} {str(d)[:120]}")
+    if code != "202":
+        return False
+    # WAIT FOR `deployed`, NOT FOR `running`. Measured 2026-09-21 on tandoor: docker reported the
+    # container `healthy` while the controller's own state read `unhealthy` — a gate on `running`
+    # alone therefore times out on an app that is up. The state is RECORDED rather than required;
+    # the real gate is the fixture's own `wait_app`, which asks whether the APP answers.
+    seen = None
+    for _ in range(90):
+        time.sleep(5)
+        st = stack(name)
+        seen = st.get("state")
+        # `deployed` alone is NOT enough and `state` alone is NOT right. Measured 2026-09-21:
+        # tandoor reads `unhealthy` while serving (R-618), so gating on "running" hangs; and romm
+        # read `deployed=True, state=degraded, pinned_images=None` twenty seconds in, i.e. the
+        # deploy had not finished writing app.yaml. The PIN is the deploy's own completion mark
+        # (`runComposeDeploy` writes it), so that is what to wait for.
+        pins = (st.get("app_config") or {}).get("pinned_images")
+        if st.get("deployed") and pins and seen in ("running", "unhealthy", "degraded"):
+            say(f"  [1] deployed, controller state={seen}, "
+                f"pinned={(st.get('app_config') or {}).get('pinned_images')}")
+            if seen != "running":
+                say(f"  [1] NOTE: the controller's own state is {seen!r}, not 'running' — recorded, "
+                    f"not treated as a failure; the fixture's front-door wait is the real gate")
+            return True
+    say(f"  [1] never became deployed (last controller state={seen!r})")
+    return False
+
+
+def backup_now(name):
+    """R-648 (2026-09-23): NO whole-box „Mentés most" from a drill, ever.
+
+    `POST /api/backup/run` is the only backup endpoint and it is WHOLE-BOX: on 9201 it stopped and
+    restarted 9 of 10 standing apps twice, and on 9202 it broke a deploy in flight (R-634). The product
+    has NO per-app backup endpoint (router.go: /backup/run, /backup/tier2 only); the per-app backup
+    exists only inside the guarded update, whose `backing-up` phase calls RunAppBackupNow for the one
+    app. So this presses nothing: the update takes the throwaway app's own backup, and says so in its
+    phase list. A seed written "after the backup" is therefore written before the update's own backup
+    — the undo's last-second copy is still the one that must bring it back."""
+    say(f"  [4] backup press SKIPPED for {name} (R-648: whole-box only; the update's backing-up phase backs up {name} alone)")
+    return None
+
+def drill_bump(app, frm, to, service_hint=None):
+    """Serialised across concurrent walks: one git working tree, one lock."""
+    import fcntl
+    with open(f"{SC}/drill.lock", "w") as lk:
+        fcntl.flock(lk, fcntl.LOCK_EX)
+        sh(["git", "-C", DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
+        return _drill_bump(app, frm, to, service_hint)
+
+
+def _drill_bump(app, frm, to, service_hint=None):
+    """Commit the edge to the DRILL repo. catalog_since set by hand (the drill repo has no gates).
+
+    `frm`/`to` may be comma-separated lists of the SAME length: an app whose own version lives in
+    two images (adventurelog's backend and frontend) moves both in one edge, while its engine
+    sidecar stays where it is — `09` §3b Q3's rule is per SERVICE, and an app-half edge must move
+    every service that carries the app's own version and no others.
+    """
+    comp = f"{DRILL}/templates/{app}/docker-compose.yml"
+    fy = f"{DRILL}/templates/{app}/.felhom.yml"
+    s = open(comp).read()
+    froms = [x.strip() for x in frm.split(",") if x.strip()]
+    tos = [x.strip() for x in to.split(",") if x.strip()]
+    if len(froms) != len(tos):
+        say(f"  [5] from/to lists differ in length: {froms} vs {tos}")
+        return None
+    for f1, t1 in zip(froms, tos):
+        if f"image: {f1}" not in s:
+            say(f"  [5] FROM ref not found in compose: {f1}")
+            return None
+        s = s.replace(f"image: {f1}", f"image: {t1}")
+    open(comp, "w").write(s)
+    f = open(fy).read()
+    today = datetime.now().strftime("%Y-%m-%d")
+    f = re.sub(r'^catalog_since:.*$', f'catalog_since: "{today}"', f, count=1, flags=re.M)
+    open(fy, "w").write(f)
+    sh(["git", "-C", DRILL, "add", "-A"])
+    sh(["git", "-C", DRILL, "commit", "-q", "-m", f"DRILL {app}: {frm} -> {to}"])
+    r = sh(["git", "-C", DRILL, "push", "-q", "origin", "main"], timeout=120)
+    h = sh(["git", "-C", DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
+    say(f"  [5] drill commit {h}: {app} {frm} -> {to} (push rc={r.returncode})")
+    return h
+
+
+def sync_rescan(expect_app=None, expect_ref=None, tries=12, delay=5):
+    """Sync, rescan, and — when told what to expect — WAIT FOR THE BADGE TO CATCH UP.
+
+    R-607: `POST /api/sync` answers "nincs valtozas" while the catalog HAS moved, and
+    `catalog_images` stays stale until a separate rescan. Tonight showed the rescan alone is not
+    enough either: mealie's badge read "Naprakesz" seconds after its bump was pushed, and the
+    Update that followed moved nothing and still reported "Frissitve". So when the caller knows
+    which reference should appear, this polls for it and SAYS HOW LONG IT TOOK — which is the
+    NUMBER R-607 asks for and has never had.
+    """
+    t0 = time.time()
+    ctl("POST", "/api/sync")
+    time.sleep(2)
+    ctl("POST", "/api/stacks/rescan")
+    time.sleep(2)
+    if not expect_app or not expect_ref:
+        return None
+    for i in range(tries):
+        cat = stack(expect_app).get("catalog_images") or {}
+        if expect_ref in cat.values():
+            waited = round(time.time() - t0, 1)
+            if i:
+                say(f"  [sync] the badge needed {waited}s and {i+1} sync+rescan rounds to catch up "
+                    f"to {expect_ref} — R-607's window, measured")
+            return waited
+        time.sleep(delay)
+        ctl("POST", "/api/sync")
+        time.sleep(1)
+        ctl("POST", "/api/stacks/rescan")
+    say(f"  [sync] the badge NEVER caught up to {expect_ref} in {round(time.time()-t0,1)}s — "
+        f"catalog_images = {stack(expect_app).get('catalog_images')}")
+    return None
+
+
+def badges(name):
+    out = {}
+    for lang, suffix in (("hu", ""), ("en", "?lang=en")):
+        h = page(f"/apps/{name}{suffix}")
+        m = re.findall(r'<span class="tag tag-[^"]*"[^>]*title="([^"]*)"[^>]*>([^<]*)<', h)
+        out[lang] = [{"title": a.strip(), "text": b.strip()} for a, b in m][:3]
+    return out
+
+
+def press_update(name, poll=1.0, cap_s=1800):
+    code, d = ctl("POST", f"/api/stacks/{name}/update")
+    say(f"  [6] Update -> {code} {str(d)[:220]}")
+    if code not in ("202", "200"):
+        return {"accepted": False, "http": code, "refusal": d, "phases": [], "duration_s": 0}
+    phases, seen, t0 = [], None, time.time()
+    while time.time() - t0 < cap_s:
+        st = stack(name)
+        ph = st.get("update_phase")
+        if ph != seen:
+            seen = ph
+            rec = {"t": round(time.time() - t0, 1), "phase": ph,
+                   "label": st.get("update_phase_label"), "updating": st.get("updating"),
+                   "error": st.get("update_error"), "hold": st.get("hold_reason")}
+            phases.append(rec)
+            say(f"      +{rec['t']:>6.1f}s  phase={ph}  label={rec['label']}  "
+                f"err={rec['error']}  hold={rec['hold']}")
+        if not st.get("updating") and ph in ("done", "failed", "undone", None) and time.time() - t0 > 3:
+            break
+        time.sleep(poll)
+    st = stack(name)
+    return {"accepted": True, "http": code, "phases": phases,
+            "duration_s": round(time.time() - t0, 1),
+            "final_phase": st.get("update_phase"), "update_error": st.get("update_error"),
+            "hold_reason": st.get("hold_reason"), "state": st.get("state")}
+
+
+def observables(name):
+    st = stack(name)
+    ac = st.get("app_config") or {}
+    live = guest(f"""
+grep -E '^\\s+image:' /opt/docker/stacks/{name}/docker-compose.yml 2>/dev/null | sed 's/^ *//'
+echo '---inspect---'
+for c in $(docker ps -a --filter label=com.docker.compose.project={name} --format '{{{{.Names}}}}'); do
+  echo -n "$c  "; docker inspect "$c" --format '{{{{.Config.Image}}}}  running={{{{.State.Running}}}}  restarts={{{{.RestartCount}}}}'
+done
+""")
+    a, _, b = live.partition("---inspect---")
+    return {
+        "pinned_images": ac.get("pinned_images"),
+        "installed_images": {k: (v.get("ref") if isinstance(v, dict) else v)
+                             for k, v in (ac.get("installed_images") or {}).items()},
+        "catalog_images": st.get("catalog_images"),
+        "live_compose_image_lines": [x for x in a.strip().splitlines() if x.strip()],
+        "docker_inspect": [x for x in b.strip().splitlines() if x.strip()],
+    }
+
+
+def app_logs(name, lines=400):
+    """The app's own container log, DECODED. The endpoint answers a JSON envelope whose `logs` is
+    one string with escaped newlines — a scan over the envelope sees a single enormous line and
+    finds nothing, which reads exactly like "the app printed no migration line" and is not. R-96
+    rule 3 in a new place: an absent line is not evidence when the instrument cannot see lines."""
+    code, d = ctl("GET", f"/api/stacks/{name}/logs?lines={lines}")
+    if isinstance(d, dict):
+        data = d.get("data")
+        if isinstance(data, dict) and isinstance(data.get("logs"), str):
+            return data["logs"]
+        if isinstance(d.get("_raw"), str):
+            return d["_raw"]
+    return str(d)
+
+
+def write_verdict(rec, appdir):
+    os.makedirs(appdir, exist_ok=True)
+    p = os.path.join(appdir, "verdict.json")
+    json.dump(rec, open(p, "w"), indent=2, ensure_ascii=False)
+    say(f"  [9] verdict {rec['verdict']}  -> {p}")
+
+
+def remove(name):
+    """Remove through the PRODUCT, never `docker rm` (live-probes rule). The remove endpoint
+    refuses a running stack — `409 still running` — so the stop is part of the act, not a tidy-up."""
+    c1, d1 = ctl("POST", f"/api/stacks/{name}/stop")
+    say(f"  [X] stop -> {c1} {str(d1)[:100]}")
+    for _ in range(24):
+        time.sleep(5)
+        if stack(name).get("state") != "running":
+            break
+    code, d = ctl("POST", f"/api/stacks/{name}/remove",
+                  {"remove_hdd_data": True, "remove_backups": True})
+    say(f"  [X] remove (with drive data) -> {code} {str(d)[:160]}")
+    if code == "409":
+        # R-442's fail-closed guard: when the storage subsystem cannot RESOLVE the app's drive
+        # path, the removal is REFUSED and the app is kept rather than half-deleted. On guest 9202
+        # `/api/disks` answers `agent not configured`, so every app deployed with an HDD_PATH hits
+        # this. The household's other choice — remove the app, KEEP the data — is accepted, and the
+        # harness takes it, then tidies its own directory by name at teardown.
+        say("  [X] refused because the drive path cannot be resolved (R-442, fail-closed and right)"
+            " — removing the app and KEEPING the drive data instead")
+        code, d = ctl("POST", f"/api/stacks/{name}/remove",
+                      {"remove_hdd_data": False, "remove_backups": True})
+        say(f"  [X] remove (keeping drive data) -> {code} {str(d)[:160]}")
+    time.sleep(5)
+    st = stack(name)
+    left = guest(f"ls -d /opt/docker/stacks/{name} 2>/dev/null; "
+                 f"docker ps -a --filter label=com.docker.compose.project={name} --format '{{{{.Names}}}}'")
+    say(f"  [X] after remove: deployed={st.get('deployed')} leftovers={left.strip()!r}")
+    return code
+
+
+def app_env(name, key):
+    """Read one deploy value the CUSTOMER was given (e.g. the generated admin password) from the
+    app's own `app.yaml`. This is not seeding — it is how the household logs in; the controller
+    shows them the same value. Data still goes in through the app's own front door."""
+    out = guest(f"grep -E '^\\s*{key}:' /opt/docker/stacks/{name}/app.yaml 2>/dev/null | head -1")
+    if ":" in out:
+        return out.split(":", 1)[1].strip().strip('"').strip("'")
+    return ""
+
+
+def snapshots(name):
+    """The restorable copies the backups page offers for this app."""
+    code, d = ctl("GET", f"/api/backup/snapshots?stack={name}")
+    data = d.get("data") if isinstance(d, dict) else None
+    if isinstance(data, dict):
+        for k in ("snapshots", "items", "restore_points"):
+            if isinstance(data.get(k), list):
+                return data[k]
+    return data if isinstance(data, list) else []
+
+
+def restore(name, snapshot_id=None, wait_s=1200):
+    """The household's own way out: the „Visszaállítás a mentésből" button on the backups page.
+
+    A FORM post, not an API call — `POST /backup/restore` with `_csrf`, `stack_name`,
+    `snapshot_id` — because that is the button the sentence tells them to press.
+    """
+    snaps = snapshots(name)
+    if snapshot_id is None:
+        if not snaps:
+            say(f"  [R] no restorable copy offered for {name}")
+            return {"ok": False, "why": "no snapshot offered", "snapshots": snaps}
+        first = snaps[0]
+        snapshot_id = first.get("id") or first.get("snapshot_id") or first.get("short_id")
+    say(f"  [R] restoring {name} from snapshot {snapshot_id!r} (of {len(snaps)} offered)")
+    sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip()
+    csrf = open(f"{SC}/csrf{os.getpid()}.txt").read().strip()
+    r = sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", HOSTHDR, "-H", f"Cookie: {sess}",
+            "-X", "POST",
+            "--data-urlencode", f"_csrf={csrf}",
+            "--data-urlencode", f"stack_name={name}",
+            "--data-urlencode", f"snapshot_id={snapshot_id}",
+            f"{BASE}/backup/restore"], timeout=180)
+    head = (r.stdout or "").split("\n")[0].strip()
+    loc = [l for l in (r.stdout or "").split("\n") if l.lower().startswith("location:")]
+    say(f"  [R] POST /backup/restore -> {head} {loc[:1]}")
+    t0 = time.time()
+    last = None
+    while time.time() - t0 < wait_s:
+        code, d = ctl("GET", "/api/backup/restore-status")
+        dd = d.get("data") or {}
+        cur = (dd.get("running"), dd.get("phase") or dd.get("state"), dd.get("message"))
+        if cur != last:
+            say(f"      +{round(time.time()-t0,1):>6.1f}s restore {cur}")
+            last = cur
+        if not dd.get("running", False) and time.time() - t0 > 5:
+            break
+        time.sleep(2)
+    st = stack(name)
+    say(f"  [R] after restore: state={st.get('state')} hold={st.get('hold_reason')!r} "
+        f"phase={st.get('update_phase')}")
+    return {"ok": True, "snapshot_id": snapshot_id, "snapshots": snaps,
+            "http": head, "location": loc[:1], "seconds": round(time.time() - t0, 1),
+            "state_after": st.get("state"), "hold_after": st.get("hold_reason"),
+            "observables_after": observables(name)}
@@ -362,3 +362,17 @@ Compressed here to title, shipping version, evidence, and the sentences that sta
 | **R-640** | **A truncated PostgreSQL copy loaded with rc 0 into an empty database (P2, narrowed to the restore paths).** Closed in **v0.267.0**: `appbackup.CheckDumpComplete` (the engine's end marker); the unit restore and the off-site restore refuse before the first mutation, every replay checks again before any load. **Rule:** a dump is judged by its END — the header and a `CREATE TABLE` say nothing about whether it finished. | **CLOSED 2026-09-23 — three red-proofs + PROVEN-LIVE on 9202 (the household's restore button refused a half-length docmost copy; containers untouched; the whole copy then restored)** | `audits/night-2026-09-23/A2-*`, `E1-r640-live.*` |
 | **R-499** | **Every driveless app was told its data was „already in the full system backup (PBS)" (P2).** Closed in **v0.267.0**: the Tier-2 page's sentence has four branches from the box's own whole-system backup target (own drive / same disk / drive gone / cannot ask); „(PBS)" and „nincs külön teendő" only where true. | **CLOSED 2026-09-23 — two red-proofs; live on 9202 (the `unknown` branch, hu + en, matching `/api/storage/backup-target`)** | `audits/night-2026-09-23/A4-*`, `A7-*` |
 | **R-626** | **A removed app came back (P2).** Measured on v0.266.0, NOT reproduced: navidrome removed through the product, 390 s of `docker events` (the remove's destroy seen, no create), a controller restart at +150 s, a guest reboot after → no container, no volume. The two known creators (the restore/remove race R-633, the backup/deploy race R-634) are fixed. **Rule kept from the row:** a check that runs once, immediately, cannot see a thing created just after it — watch a window. Leftover found and filed: R-651. | **CLOSED 2026-09-23 — by measurement (a positive observable: the destroy events)** | `audits/night-2026-09-23/A3-*` |
+
+## 2026-09-24 — the undo after a restore, the held app's page, the ladder (controller v0.268.0, hub v0.122.0, catalog `5ed599c`)
+
+| Row | What | Closed | Full text |
+|---|---|---|---|
+| **R-658** | **After a restore, the undo copied NOTHING (P1).** v0.268.0 (`206b035`): the undo selects volumes from the rendered compose file (`DeclaredVolumeNames`: `name:` else `<project>_<key>`, each checked to exist); the label is a logged cross-check. The unit restore creates volumes WITH compose's project/volume/version labels (never a guessed config-hash); the remove counts unlabelled declared volumes. Live on 9202: restored under v0.267.0 → labels `null`; on v0.268.0 a failing update copied both by name and seeds A and B (B written after the restore) read back; a v0.268.0 restore → labels present, no compose warning. **Rule:** never select an app's volumes by the compose label. | v0.268.0, 2026-09-24 | `git show 500488cad673:documentation/backlog/OPEN-ITEMS.md` |
+| **R-659** | **A held app's page named a way back the restore refused (P1).** Operator ruling 2026-09-24 (`09` §3 decision 25, option A). v0.268.0 + hub v0.122.0: the hold names the newest WHOLE copy (`WholeOnTier` asks the refusal's own predicate); with none, `hold.update.no_whole_copy` in the household's language, no Mentések button, and `app_hold_no_whole_copy` (critical, operator-only). Live on 9202: round 11 reproduced (nextcloud, cut-off undo copy) — both languages, no button on either page, the event's R-620 line. **Rule:** a sentence may name a copy as a way back only when the restore for that copy would accept it. Left open beside it: R-661, R-666. | v0.268.0 / hub v0.122.0, 2026-09-24 | `git show 500488cad673:documentation/backlog/OPEN-ITEMS.md` |
+| **R-660** | **A held app also raised `app_start_failed` (P3).** v0.268.0: a fourth suppression set at `classifyRunStates` — the update-held apps (`backup.UpdateHeldStacks`; a RESTORE hold is not in it). Live: after the hold no `app_start_failed` in ~12 min of scans, while a throwaway stopped out of band raised exactly one. | v0.268.0, 2026-09-24 | `git show 500488cad673:documentation/backlog/OPEN-ITEMS.md` |
+| **R-651** | **A removed app left `applied-compose.yml` and `applied-meta/` (P3).** v0.268.0: remove deletes both (the catalog mirror and `hold-logs/` stay — the latter is evidence). Live: present before, gone after, on nextcloud and vikunja. | v0.268.0, 2026-09-24 | `git show 500488cad673:documentation/backlog/OPEN-ITEMS.md` |
+| **R-653** | **The memory watch wrote `proven` over a watch whose load never reached the app (P3).** Catalog `5ed599c`: `load_verdict` — `reached` only when at least half the requests got an HTTP answer, else the edge is `inconclusive`. Unit-tested and red-proofed; no bench run this session. | catalog, 2026-09-24 | `git show 500488cad673:documentation/backlog/OPEN-ITEMS.md` |
+| **R-656** | **The bench re-used an app's scratch drive folder (P3).** Catalog `5ed599c`: `clear_scratch_folders` removes the app's own `${HDD_PATH}`/`${USERDATA_PATH}`/`${IMPORT_PATH}` bind folders before FROM and says so; never a bare root, never outside. Unit-tested and red-proofed; no bench run this session. | catalog, 2026-09-24 | `git show 500488cad673:documentation/backlog/OPEN-ITEMS.md` |
+| **R-40** | **The update path could not express a multi-hop upgrade (P2).** Superseded by `09` §3 decisions 13–14 and shipped as §6.4 part 5 (v0.268.0): the catalog records every tested step with its own definition, and one press climbs one. **Rule kept:** a hop exists for a box only when the catalog holds it as a TESTED step — a >1-major catalog move without its intermediate steps is still a jump for a box below it. | v0.268.0, 2026-09-24 | `git show 500488cad673:documentation/backlog/OPEN-ITEMS.md` |
+| **R-663** | **Two catalog test suites had been red since the night of 2026-09-23 (P3).** Filed and fixed the same session: `test_gate_decoys.py` (kimai-db 11.6 → 11.8) and `test_ladder_writer.py` (navidrome 0.64.0 → 0.64.1) typed the live pins as literals; they now READ them. 84 decoy cases OK. **Rule:** a fixture that names a live pin reads it; the catalog moves under it every night. | catalog `5ed599c`, 2026-09-24 | this entry |
+