From e3b785f33a5b0dc9d2c4b5dd604b5ed4b6490e7f Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Wed, 14 Oct 2026 19:28:13 +0200 Subject: [PATCH] Immich repair on every box: rollout read-back, R-1021 closed, R-1007 dated line, F12 fleet delivery, STATUS, report --- REPORT-immich-rollout-2026-10-14.md | 49 +++++++++++++++++ STATUS.md | 12 ++--- .../architecture/00-capability-map.md | 2 +- .../architecture/capability-map-detail/F.md | 4 ++ .../E/rollout-readback.txt | 52 +++++++++++++++++++ .../audits/immich-repair-2026-10-14/README.md | 12 +++++ documentation/backlog/CLOSED-ITEMS.md | 8 ++- documentation/backlog/OPEN-ITEMS.md | 1 - 8 files changed, 131 insertions(+), 9 deletions(-) create mode 100644 REPORT-immich-rollout-2026-10-14.md create mode 100644 documentation/audits/immich-repair-2026-10-14/E/rollout-readback.txt diff --git a/REPORT-immich-rollout-2026-10-14.md b/REPORT-immich-rollout-2026-10-14.md new file mode 100644 index 00000000..500c6aef --- /dev/null +++ b/REPORT-immich-rollout-2026-10-14.md @@ -0,0 +1,49 @@ +# REPORT — 2026-10-14 evening: the Immich repair on every box; an Immich bump cannot switch it off; the shared checkout cleaned + +Times UTC. Read first: `09-update-architecture.md` (golden, floor, MinAgent), `runbooks/RUNBOOK-manual-build.md` §4, +`audits/immich-repair-2026-10-14/`. Baselines: felhom.eu `e62fc00e` (moved past the prompt's `6e0643f6` by other +sessions), controller `9b95e8e0`, catalog `74c8e842`, agent `9b749ee7`. No controller, agent or hub release. No power cut. +No key, token or password in any file. + +## 1. Every box + +| Box | Controller | Agent | +|---|---|---| +| demo-felhom (N100) | 0.311.0 | 0.156.1 | +| demo-hp | 0.311.0 | 0.156.1 | +| Tester 1 | 0.311.0 | 0.156.1 | +| Tester 2 | off (0.292.0 last seen) — gets the floor when it next reports | 0.142.0 | + +Read on the hub AND on each box (`docker ps`, `felhom-agent --version`): `audits/immich-repair-2026-10-14/E/rollout-readback.txt`. +Hub: golden 0.311.0 + agent 0.156.1 (not `-step1`) + min_agent 0.155.0; global floor 0.311.0 / 0.155.0. **demo-hp's own +floor 0.311.0 is still set** (the clear did not save) — harmless today, it would hold demo-hp at the next raise. + +## 2. The gate + +Catalog `unclean-boot-images` refuses a compose (or `steps/` file) whose repair service runs an image the +`after_unclean_boot.images` list does not hold, and a block it cannot match. Decoys pass: the new image in a comment, on +another service, under a commented block, no block. Five mutations each turned a case red. Catalog `8f5ad8a`. + +## 3. The shared checkout + +Moved (nothing deleted) to `/mnt/5_hdd/felhom.eu/leftovers/2026-10-14/`: `walk.py` (+ patch; identical to main's +version), `sizing-2026-10-13/` (an older copy, = commit `c885b5ac`), `usenet-2026-10-12/` (identical but main's +README), Facebook `first-post/` (NOT in git; not opened). Clean at `origin/main` after. `build-hub.sh` v1.24.0 refuses a +stale or dirty tree: 6 of 6 test cases; the old script 5 of 6 wrong. + +## 4. Claims in the prompt that were wrong + +- The Facebook folder did not block the pull (main has no `first-post/`); it was moved so the tree could be clean. +- The modified `walk.py` was not a local change: main already holds exactly that text (`52615d91`). +- „Each demo customer had its own floor 0.309.0": by today only demo-hp had one (the other two were cleared earlier). +- Mine: one command ran the catalog runner test right after the commit (`&&`), against the rule „never combine a test + run and a commit" — the commit itself followed a separate green gate run; the test passed. + +## 5. Record + +- Commits: catalog `8f5ad8a`; felhom.eu `621783b3` (build script), `be4d35f5` (golden record), and this one. +- Register: **156 before → 155 after; opened 0; closed 1** (R-1021). R-1007 got a dated line. +- Capability map F12: fleet delivery done. STATUS: the floor/golden and checkout „do" lines replaced by what is left. +- System poster: no listed fact changed. Fixed without a row: none. +- Teardown: drill VM — CT 9100 purged, token/script/log shredded, VM off, disk on `virgin`; hosts — nothing made; hub — CC + read only (the operator saved the manifest and floor). diff --git a/STATUS.md b/STATUS.md index 9f7a3e7a..ffce33ef 100644 --- a/STATUS.md +++ b/STATUS.md @@ -2,12 +2,12 @@ **Ready for the first real tester (Tester-2): yes. Tester 2 (a laptop) is off; nothing was sent to it.** -**Updated 2026-10-14 (evening).** Agent 0.156.1 + its bundle on all three boxes; controller 0.311.0 on demo-hp (its own floor), 0.310.0 on the other two; hub 0.150.0; golden 0.310.0. Open items: see `documentation/backlog/OPEN-ITEMS.md`. Older notes, day by day: -`documentation/backlog/STATUS-HISTORY.md`. Reports: `REPORT-seo-felho-tarhely-2026-10-14.md`, `REPORT-immich-repair-2026-10-14.md`, `REPORT-power-cut-fixes-2026-10-14.md`, `REPORT-night-power-2026-10-13.md`, `REPORT-poweroff.md`. +**Updated 2026-10-14 (evening).** Agent 0.156.1 + its bundle and controller 0.311.0 on all three boxes; hub 0.150.0; golden 0.311.0 (vouched, global floor). Open items: see `documentation/backlog/OPEN-ITEMS.md`. Older notes, day by day: +`documentation/backlog/STATUS-HISTORY.md`. Reports: `REPORT-immich-rollout-2026-10-14.md`, `REPORT-seo-felho-tarhely-2026-10-14.md`, `REPORT-immich-repair-2026-10-14.md`, `REPORT-power-cut-fixes-2026-10-14.md`, `REPORT-night-power-2026-10-13.md`, `REPORT-poweroff.md`. ## What works - A power cut in the middle of a backup no longer blocks later backups: the box removes the half-written backup at start-up (proven with a real cut on demo-hp, on all three boxes now). -- After a power cut, Immich repairs the photos the cut left broken, by itself, and the app page lists any lost photo (proven with a real cut on demo-hp: 41 of 41 repaired; on demo-hp only until the floor moves). +- After a power cut, Immich repairs the photos the cut left broken, by itself, and the app page lists any lost photo (proven with a real cut on demo-hp: 41 of 41 repaired; on all three boxes now). An Immich update cannot switch it off quietly: the catalog refuses an untested Immich version. - A household can switch the machine off; the hub holds alarms for 14 days and clears the state by itself (N100). - A real power cut on demo-hp: it started by itself after all 8 cuts of the night (with the BIOS set), every app back in ~4 min; stale and recovered mails true and paired. - A page for people searching „felhő tárhely" is live (/sajat-felho-tarhely, /en/own-cloud-storage): what cloud storage is, what to do when Google or iCloud fills up, a five-year cost table with prices read 2026-10-14, and who should NOT buy this. @@ -22,7 +22,7 @@ - Tester 2 has never made an off-site check (R-862). The scratch box cannot read the drill catalog (R-992). ## What we work on -- The power-cut findings still open (R-1004, R-1005, R-1010, R-1012, R-1013); the hub build must refuse a stale checkout (R-1021). +- The power-cut findings still open (R-1004, R-1005, R-1010, R-1012, R-1013). - Media apps on every drive (R-1023): the layout is proven on Tester 1 — a download reaches the library as one file, no copy. The live templates get fixed next (R-1014). ## Waiting on you — decide @@ -40,8 +40,8 @@ | D9 | R-1023 | Media apps: bind the whole `media` folder (download apps can then write into Photos and Music), or move films and shows to their own folder first? | **Own folder** (`15-media-library.md` §2) | Whole `media`: no move, but the internet-facing apps reach the family photos | the build waits for this answer | the build | ## Waiting on you — do -- **Controller 0.311.0 to the N100 and Tester 1:** bake golden 0.311.0 (CC, ~30 min), vouch it, then the global floor 0.311.0 (MinAgent 0.155.0) and clear demo-hp's own floor. If not: only demo-hp repairs Immich after a power cut; demo-hp's own floor stays above the golden. -- **The shared `git/felhom.eu` checkout** is behind and its pull fails on untracked leftovers (R-1021): say whose they are, or let CC move them aside. If not: the hub build script builds stale code. +- **Clear demo-hp's own floor** (hub → Demo HP → floor: empty → Save floor). It still reads 0.311.0. If not: the next global raise passes demo-hp by. +- **Leftovers moved out of the shared checkout** to `/mnt/5_hdd/felhom.eu/leftovers/2026-10-14/` (nothing deleted): the Facebook `first-post/` folder is not in git — keep, commit or delete it. The rest is already in git. If not: it just stays there. - **The media test on Tester 1** (about one hour): `documentation/audits/media-all-drives-2026-10-14/TEST-SHEET.md`, then write „done". Then the teardown word when you want (R-1022). - **Ask Google to index the new page:** Search Console → URL-ellenőrzés → `https://felhom.eu/sajat-felho-tarhely` → „Indexelés kérése". If not: the page may sit unseen, and the 11 November / 9 December checks (R-1035) read zero for a reason that is not the page. - **N100 BIOS:** read „After Power Loss" (the HP's „Previous State" made it start after 8 of 8 cuts). **HP BIOS:** „Bypass F1 Prompt on Configuration Changes" → Enable. diff --git a/documentation/architecture/00-capability-map.md b/documentation/architecture/00-capability-map.md index b13b9e9a..4b17799e 100644 --- a/documentation/architecture/00-capability-map.md +++ b/documentation/architecture/00-capability-map.md @@ -217,7 +217,7 @@ | [F7](capability-map-detail/F.md#f7) | Always-on debug rings + on-demand log-bundle pulls with TTL/custody | controller v0.116, agent v0.83, hub v0.46 | **PROVEN-LIVE** | — | `audits/CAMPAIGN-3-2026-07-11.md` (2026-07-11) | | [F8](capability-map-detail/F.md#f8) | Operator alerting (Healthchecks → monitoring@felhom.eu) | k3s, Resend | **IMPLEMENTED** | no corpus validation doc | operator infra; no evidence document | | [F9](capability-map-detail/F.md#f9) | Backup-deadline alerting (`expected_backup_missed`) is ANCHORED — absence of signal is UNKNOWN, not failure | hub v0.75.0 | **IMPLEMENTED** | — | `audits/DIAG-backup-missed-2026-07-26.md` (2026-07-26); R-82 | -| [F12](capability-map-detail/F.md#f12) | After a power cut, an app rebuilds the files the cut left empty, and the household sees any lost photo on the app page | controller v0.311.0, hub v0.150.0, catalog Immich block | **PROVEN-LIVE** | demo-hp 2026-10-14: a real cut mid-upload, 41 of 41 broken photos repaired, 120 controls untouched; the lost-photo note not exercised live (0 lost; unit- and 9202-tested) | `audits/immich-repair-2026-10-14/README.md` (2026-10-14); `09` §3 decision 236; R-1007 | +| [F12](capability-map-detail/F.md#f12) | After a power cut, an app rebuilds the files the cut left empty, and the household sees any lost photo on the app page | controller v0.311.0, hub v0.150.0, catalog Immich block | **PROVEN-LIVE** | demo-hp 2026-10-14: a real cut mid-upload, 41 of 41 broken photos repaired, 120 controls untouched; the lost-photo note not exercised live (0 lost; unit- and 9202-tested). **Fleet delivery done 2026-10-14:** golden 0.311.0 + global floor; all three demo boxes on 0.311.0; catalog gate `unclean-boot-images` | `audits/immich-repair-2026-10-14/README.md` (2026-10-14); `09` §3 decision 236; R-1007 | | [F11](capability-map-detail/F.md#f11) | The household turns the whole machine off („A gép kikapcsolása"); the hub holds every silence alarm for 14 days and shows KIKAPCSOLVA | agent v0.155.0, hub v0.149.0, controller v0.310.0 | **PROVEN-LIVE** | proven on the N100 2026-10-13; fleet delivery waits on the floor | `audits/poweroff-2026-10-13/README.md` (2026-10-13); `architecture/14-host-power-off.md`; R-1000, R-1002 | | [F10](capability-map-detail/F.md#f10) | A box that is not always on (off at its backup time) makes the missed night up once when it comes back; the household sees a banner; the operator gets the missed-backup alarm, down or not | controller v0.295.0, hub v0.134.0, agent v0.145.0 | **PARTIAL** | catch-up and banner live; R-872 alarm by tests; suspend not measured | `audits/catchup-2026-10-05/partA/` (2026-10-05); R-872 | diff --git a/documentation/architecture/capability-map-detail/F.md b/documentation/architecture/capability-map-detail/F.md index 88877c67..d6dfe453 100644 --- a/documentation/architecture/capability-map-detail/F.md +++ b/documentation/architecture/capability-map-detail/F.md @@ -212,6 +212,10 @@ counts. (`app_repaired_after_crash`, operator-only), catalog `templates/immich/.felhom.yml` `after_unclean_boot` (proven on `immich-server:v3.2.4`). Decision 236. +**Fleet delivery (2026-10-14):** golden 0.311.0 vouched and the global floor 0.311.0 (MinAgent 0.155.0); N100, demo-hp and +Tester 1 run controller 0.311.0. An Immich image bump the repair was not proven on is refused at push by the catalog gate +`unclean-boot-images` (`scripts/check-unclean-boot-images.py`) — the box would otherwise only skip the repair. + **Evidence:** `audits/immich-repair-2026-10-14/` — spike A on 9202 (the five candidates), the scripts on real damage (9202), the live cut on demo-hp 9201 (D/). **Not exercised live:** a lost original (the cut lost none); pinned by unit tests and the 9202 run. diff --git a/documentation/audits/immich-repair-2026-10-14/E/rollout-readback.txt b/documentation/audits/immich-repair-2026-10-14/E/rollout-readback.txt new file mode 100644 index 00000000..4cd60b6e --- /dev/null +++ b/documentation/audits/immich-repair-2026-10-14/E/rollout-readback.txt @@ -0,0 +1,52 @@ +# rollout read-back 2026-10-14T17:24:50Z +hub /configuration: golden 0.311.0 (efe5f02e…) selected, agent 0.156.1 (83c7da27…, not -step1) selected, min_agent 0.155.0; global floor 0.311.0 / 0.155.0 +hub /system per-customer floors: demo-hp 0.311.0 (set 2026-10-14, still present); demo-felhom none; tester-1 none +hub /configs: demo-felhom 0.311.0 OK; demo-hp 0.311.0 (override); tester-1 0.311.0 OK; Tester-2 DOWN 0.292.0 (floor v0.311.0) +hub /hosts agents: demo-felhom 0.156.1, demo-hp 0.156.1, tester-1 0.156.1, Tester-2 0.142.0 (down) +== box felhom-pve +felhom-agent 0.156.1 +cloudflared cloudflare/cloudflared:2026.9.3 Up 20 hours (healthy) +felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.311.0 Up About a minute (healthy) +filebrowser gtstef/filebrowser:1.5.6-stable Up About a minute (healthy) +opengist ghcr.io/thomiceli/opengist:1.15 Up 15 hours (healthy) +traefik traefik:v3.7.13 Up 20 hours +== box demo-hp +felhom-agent 0.156.1 +adventurelog ghcr.io/seanmorley15/adventurelog-backend:v0.13.0 Up 2 hours (healthy) +adventurelog-frontend ghcr.io/seanmorley15/adventurelog-frontend:v0.13.0 Up 2 hours (healthy) +adventurelog-postgres postgis/postgis:16-3.5-alpine Up 2 hours (healthy) +bentopdf ghcr.io/alam00000/bentopdf:v2.8.6 Up 2 hours (healthy) +bookstack lscr.io/linuxserver/bookstack:26.09.1 Up 2 hours (healthy) +bookstack-db mariadb:12.3 Up 2 hours (healthy) +calibre-web crocodilestick/calibre-web-automated:v4.0.8 Up 2 hours (healthy) +cloudflared cloudflare/cloudflared:2026.9.3 Up 2 hours (healthy) +docmost docmost/docmost:0.96.0 Up 2 hours (healthy) +docmost-postgres postgres:18-alpine Up 2 hours (healthy) +docmost-redis redis:7-alpine Up 2 hours (healthy) +felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.311.0 Up 2 hours (healthy) +filebrowser gtstef/filebrowser:1.5.6-stable Up 2 hours (healthy) +home-assistant ghcr.io/home-assistant/home-assistant:2026.9.4 Up 2 hours (healthy) +opengist ghcr.io/thomiceli/opengist:1.15 Up 2 hours (healthy) +paperless-postgres postgres:18-alpine Up 2 hours (healthy) +paperless-redis redis:7-alpine Up 2 hours (healthy) +paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 Up 2 hours (healthy) +privatebin privatebin/pdo:2.0.6 Up 2 hours (healthy) +traefik traefik:v3.7.13 Up 2 hours +== box root@192.168.0.154 +felhom-agent 0.156.1 +bookstack lscr.io/linuxserver/bookstack:26.09.1 Up 2 hours (healthy) +bookstack-db mariadb:12.3 Up 2 hours (healthy) +cloudflared cloudflare/cloudflared:2026.9.3 Up 2 hours (healthy) +felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.311.0 Up About a minute (healthy) +filebrowser gtstef/filebrowser:1.5.6-stable Up 2 hours (healthy) +paperless-postgres postgres:18-alpine Up 2 hours (healthy) +paperless-redis redis:7-alpine Up 2 hours (healthy) +paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 Up 2 hours (healthy) +plex plexinc/pms-docker:1.41.4.9463-630c9f557 Up About an hour (healthy) +privatebin privatebin/pdo:2.0.6 Up 2 hours (healthy) +prowlarr lscr.io/linuxserver/prowlarr:version-2.6.5.5623 Up 59 minutes (healthy) +qbittorrent qbittorrentofficial/qbittorrent-nox:5.2.4-1 Up 2 hours (healthy) +radarr lscr.io/linuxserver/radarr:6.4.4 Up 2 hours (healthy) +sonarr lscr.io/linuxserver/sonarr:4.0.20 Up 2 hours (healthy) +traefik traefik:v3.7.13 Up 2 hours +transmission lscr.io/linuxserver/transmission:version-4.1.3-r0 Up 2 hours (healthy) diff --git a/documentation/audits/immich-repair-2026-10-14/README.md b/documentation/audits/immich-repair-2026-10-14/README.md index c3539669..5fe348be 100644 --- a/documentation/audits/immich-repair-2026-10-14/README.md +++ b/documentation/audits/immich-repair-2026-10-14/README.md @@ -92,3 +92,15 @@ the CONTROLS before the cut; uploads 61–200 ran into the cut. Findings: the event's and the log's duration read 0 (R-1020); the hub build script builds a stale checkout when its pull fails (R-1021). Not this session: Plex was removed on Tester 1 at 15:41 and its controller restarted at 15:42 — another session's work; this session's cut also restarted Tester 1 (VM 341 on the HP), and that session was told. + +## E — Rollout to every box, the image gate, the shared checkout (2026-10-14 evening) + +- **Gate:** catalog `scripts/check-unclean-boot-images.py` (`unclean-boot-images`, `--fast`: hook and CI). 11 decoy cases + (`E/catalog-decoys.txt`); five mutations of the gate each turned a case red (`E/redproof-unclean-boot.txt`). Catalog `8f5ad8a`. +- **Golden 0.311.0:** baked in the drill VM, sha `efe5f02e…`, round trip equal (`documentation/tests/golden-0.311.0-2026-10-14/`). + Vouched by the operator with agent 0.156.1 and min_agent 0.155.0; global floor 0.311.0 / 0.155.0. +- **Read-back** (`E/rollout-readback.txt`; the hub pages AND each box's own `docker ps`): N100, demo-hp and Tester 1 run + controller 0.311.0 and agent 0.156.1; app containers kept running (not restarted by the update). demo-hp's own floor + 0.311.0 is still set. Tester 2 is off: it gets the floor when it next reports. +- **Shared checkout (R-1021):** leftovers moved to `/mnt/5_hdd/felhom.eu/leftovers/2026-10-14/` (README there); clean at + `origin/main`. `build-hub.sh` v1.24.0 refuses a stale or dirty tree (`E/bh-new.txt`; the old script `E/bh-old.txt`). diff --git a/documentation/backlog/CLOSED-ITEMS.md b/documentation/backlog/CLOSED-ITEMS.md index 7b9a052d..28cd193b 100644 --- a/documentation/backlog/CLOSED-ITEMS.md +++ b/documentation/backlog/CLOSED-ITEMS.md @@ -26,11 +26,17 @@ --- +## 2026-10-14 — the Immich repair on every box; the shared checkout cleaned + +| Row | What | Closed | Evidence | +|---|---|---|---| +| **R-1021** | **`build-hub.sh` built STALE hub code when its `git pull` failed — it only warned** (P3) | CLOSED 2026-10-14 — FIXED: `build-hub.sh` v1.24.0 fetches, pulls `--ff-only`, and refuses (exit 1) unless the tree is clean and `HEAD == origin/main`; the commit label is read after the pull; no override (no runbook needs one); `FELHOM_HUB_REPO_DIR` builds from another clone under the same rule. `test_build_hub_tree_gate.sh` (stub docker): 6 cases right; the old script gets 5 wrong. The shared checkout's leftovers MOVED (operator ruling „move", nothing deleted) to `/mnt/5_hdd/felhom.eu/leftovers/2026-10-14/` (its README: all but one already in git; the Facebook `first-post/` folder is not in git and is the operator's to decide); the checkout is clean at `origin/main` | felhom.eu `621783b3`; `audits/immich-repair-2026-10-14/E/bh-new.txt`, `bh-old.txt` | + ## 2026-10-14 — Immich repairs its broken photos after a power cut | Row | What | Closed | Evidence | |---|---|---|---| -| **R-1007** | **After a power cut Immich shows photos as done that are broken, and nothing repairs them** (P3) | CLOSED 2026-10-14 — decision 236 (operator): the box asks Immich through Immich's OWN job queue (no key into the household's account; every user's photos), for the asset ids of the zero-byte thumbnail/preview files only, then reads Immich's own missing-files report with a read-only session. Controller v0.311.0 (`after_unclean_boot`, once per start after an UNCLEAN boot, after the 15-min crash grace), hub v0.150.0 (`app_repaired_after_crash`, operator-only, counts only), catalog Immich block (proven on `immich-server:v3.2.4` only). **Proven live on demo-hp 9201:** a real cut mid-upload (15:25:04 UTC) left 41 photos with empty files; 41 of 41 repaired in 10 s; 120 control files untouched; 0 originals lost, so the household's lost-photo note was not exercised live (unit- and 9202-tested). **Reasoning kept:** a household-made Immich API key reaches only the admin's photos — a family member's photo was refused (measured); Immich's nightly „missing thumbnails" reads only its database, never the disk | controller `8d862e4` + `3b85a0c`, hub `822d0393`, catalog `6a1bb04`; `audits/immich-repair-2026-10-14/` | +| **R-1007** | **After a power cut Immich shows photos as done that are broken, and nothing repairs them** (P3) | CLOSED 2026-10-14 — decision 236 (operator): the box asks Immich through Immich's OWN job queue (no key into the household's account; every user's photos), for the asset ids of the zero-byte thumbnail/preview files only, then reads Immich's own missing-files report with a read-only session. Controller v0.311.0 (`after_unclean_boot`, once per start after an UNCLEAN boot, after the 15-min crash grace), hub v0.150.0 (`app_repaired_after_crash`, operator-only, counts only), catalog Immich block (proven on `immich-server:v3.2.4` only). **Proven live on demo-hp 9201:** a real cut mid-upload (15:25:04 UTC) left 41 photos with empty files; 41 of 41 repaired in 10 s; 120 control files untouched; 0 originals lost, so the household's lost-photo note was not exercised live (unit- and 9202-tested). **Reasoning kept:** a household-made Immich API key reaches only the admin's photos — a family member's photo was refused (measured); Immich's nightly „missing thumbnails" reads only its database, never the disk **2026-10-14 (later): rolled out to all demo boxes** — golden 0.311.0 vouched, global floor 0.311.0 / MinAgent 0.155.0; N100, demo-hp and Tester 1 run controller 0.311.0 + agent 0.156.1; **gate added:** catalog `unclean-boot-images` refuses an image bump the repair was not proven on (catalog `8f5ad8a`). | controller `8d862e4` + `3b85a0c`, hub `822d0393`, catalog `6a1bb04`; `audits/immich-repair-2026-10-14/` | ## 2026-10-14 — the estimator takes the second machine's figures, raise only diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index 625510dc..19a9014e 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -300,7 +300,6 @@ stopping line that lies. | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| -| **R-1021** | Process & tooling | P3 | **`build-hub.sh` builds STALE hub code when its `git pull` fails — it only warns.** 2026-10-14: the shared checkout `git/felhom.eu` was behind `origin/main` (`52b6599d` vs `822d0393`) and its pull aborted on untracked leftovers (`audits/sizing-2026-10-13/`, `audits/usenet-2026-10-12/`, a Facebook folder, a modified `walk.py`) — the script would have shipped hub 0.149-era code labelled 0.150.0. This session built from its own clean clone instead (a copy of the script with only `REPO_DIR` changed). | **OPEN** — filed 2026-10-14 | operator: whose are the leftovers in the shared checkout | CC: make the build refuse unless the source tree is clean and `HEAD == origin/main` (the clean-tree gate, in the script); the operator or their owner cleans the shared checkout | CC | | **R-992** | Process & tooling | P3 | **The drill-catalog route that `09` §6.5 documents cannot be followed: `admin/app-catalog-drill` is PRIVATE and no box has catalog credentials.** Measured 2026-10-12 while publishing the Usenet pair: the box's clone fails with `fatal: could not read Username for 'https://gitea.dooplex.hu'` (exit 128), the Gitea API reports the drill repo `private=true` and the live catalog `private=false`, and NEITHER the current controller.yaml on 9202 NOR any of its 30 saved copies has ever carried `git.username` or `git.token` (the controller supports both — `internal/config/config.go` GitConfig). The route worked for the campaigns of 2026-10-10 and the torrent pair earlier the same day, so the repository's visibility changed between then and now. **Worked around, not fixed:** that campaign staged its two templates in the box's own catalog cache (`audits/usenet-2026-10-12/tools/stage_in_cache.py`), which is where the sync copies a catalog change anyway, with the sync interval raised to 12h for the duration and rescan used in place of sync. The real fix is an operator decision: either a read-only catalog credential on the scratch box (a secret on a Tier-0 guest) or making the drill repository readable. Until then every new-app campaign hits this. | WAITING-ON-OPERATOR | the operator's choice between a box credential and a readable drill repo | Decide; then either set `git.username`/`git.token` on 9202 from the credentials store, or flip the drill repo, and correct `09` §6.5 either way | operator | | **R-1017** | Process & tooling | P3 | **A comment names a guard test that does not exist.** `felhom-controller/controller/internal/appbackup/paths.go:58` says „`TestNoBareDrivePathToUserdataDir` is the guard that keeps the count from growing”; no test of that name (or `NoBareDrive*`) exists in the repo (grep 2026-10-14; positive control: `TestRemoveStack_R800_UserdataKeptIsNamed` is found). A comment asserting an invariant with no test pinning it — the class in the `felhom-testing` skill. | **READY** — filed 2026-10-14 | — | Write the guard (a test that fails when a bare drive path reaches `UserdataDir`), red-proof it, or correct the comment | CC | | **R-1022** | Process & tooling | P3 | **Tester 1 follows the drill catalog and carries two test drives — teardown owed (operator ruling (e), 2026-10-14).** Steps, only on the operator's word: (1) remove the test apps (Sonarr, Radarr, qBittorrent, Transmission, Plex, Jellyfin) through the dashboard WITHOUT „delete data” (R-1027); (2) decommission Media A and Media B through Tester 1's dashboard; (3) `repoint.py restore` — `controller.yaml` from `controller.yaml.pre-media-2026-10-14`, read back byte-identical, `repo_url` the live catalog; remove `drill-catalog.git` from the controller volume; (4) on demo-hp: `qm set 341 --delete scsi2,scsi3`, then delete the two volumes BY NAME (`nvme-scratch:341/vm-341-disk-3.raw`, `-disk-4.raw`) — never `qm destroy`; (5) VM memory unchanged (8192, not raised); (6) drill repo reset to live `main`. Tools: `audits/media-all-drives-2026-10-14/``tools/`. | **WAITING-ON-OPERATOR** — filed 2026-10-14 | the operator's teardown word | Run the six steps; quote each read-back | CC |