Controller v0.314.0 released under decision 250: floor 0.314.0 raised by CC's token at 13:11 UTC after the three test boxes
gates / gates (push) Successful in 6m45s
gates / gates (push) Successful in 6m45s
- 09 §3 decision 253: a restore puts the setup gate back as it was (R-1057 P2); 07 (what a per-app backup carries); 15 §6 (Prowlarr links itself; Plex by address). - Evidence audits/gate-restore-2026-10-15/: 9202 live proofs (R-1057, R-1058), the release (per-customer floors, Tester 1 rc.1 -> 0.314.0 by itself, box walk, global floor, read-back), Plex fake-code check on 9201, red-proofs. - Register: R-1023, R-1053, R-1057, R-1058 closed; R-1061 filed (P4); R-1060 narrowed; R-1052 blocked (customer creation is session only). STATUS, CONTEXT, the build skill's release steps, Prowlarr's website card. Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -26,6 +26,15 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-10-15 (evening) — controller v0.314.0, the first release under decision 250 (the floor raised by CC's token)
|
||||
|
||||
| Row | What | Closed | Evidence |
|
||||
|---|---|---|---|
|
||||
| **R-1057** | **A restore of a REMOVED gated app came back without its setup gate, even from a backup made before the setup.** (P2 — raised from P3 by the reviewer 2026-10-15) | CLOSED 2026-10-15 — **controller v0.314.0** (`09` §3 decision 253): the per-app backup records the gate; a restore puts it back as it was (closed then → closed now, file first; open then → open; no record → closed until the probe or „Kész"). Unit tests red-proved by ten mutations. **Live on 9202** (rc.1): Prowlarr installed, no account, the Update's per-app backup, remove keeping backups, restore → a stranger got `/ping` 401, `/` 302 to the gate, `/initialize.json` 401; the household 200; the box's log: „restored with its setup gate CLOSED — it was closed when the backup was taken (R-1057)"; „Kész" then opened it. Before the fix the same walk gave the stranger 200. Plex: the same rule, proven by a unit test with Plex's gate shape (no live Plex restore). | `audits/gate-restore-2026-10-15/9202/restore.txt`, `02-controller-log.txt` |
|
||||
| **R-1053** | **A box on an `-rc` tag was never moved by the floor.** (P3) | CLOSED 2026-10-15 — **controller v0.314.0**: `util.ParseVersionPre` puts `X.Y.Z-rc.N` below `X.Y.Z` for the floor only; the button never offers a test build; an unreadable version WARNs once per change. **Live:** Tester 1 put on 0.314.0-rc.1 by hand (rc.4 runs the old check and could never leave by itself), then the floor 0.314.0 moved it: „Post-update startup: update successful (0.314.0-rc.1 → 0.314.0)" at 13:09:38 UTC. | `audits/gate-restore-2026-10-15/release/01-tester1-rc1.txt`, `03-boxes-move.txt` |
|
||||
| **R-1023** | **Build: media apps see every drive, and the box says when they must restart.** (P2) | CLOSED 2026-10-15 — delivered by controller v0.313.0 and the floor; its last step was Tester 1 off the test build (R-1053): Tester 1 runs the release 0.314.0 since 13:09:38 UTC, moved by the floor by itself. | `audits/gate-restore-2026-10-15/release/03-boxes-move.txt`, `audits/media-build-2026-10-15/` |
|
||||
| **R-1058** | **Prowlarr installed AFTER Sonarr/Radarr was not linked until „Bekötés most".** (P3) | CLOSED 2026-10-15 — **controller v0.314.0**: Prowlarr's own start runs the media wiring (`linkOnlyApps`). **Live on 9202** (rc.1): Sonarr first, then Prowlarr → „a media app came up: prowlarr", Sonarr in Prowlarr's Apps ~40 s later, no button; a second run „unchanged"; Prowlarr's connection test 200. Catalog `e67aad4` and the website card drop the button sentence. | `audits/gate-restore-2026-10-15/9202/link.txt` |
|
||||
|
||||
## 2026-10-15 (evening) — Prowlarr in the live catalog (catalog 73b45a6)
|
||||
|
||||
| Row | What | Closed | Evidence |
|
||||
|
||||
@@ -124,7 +124,7 @@ stopping line that lies.
|
||||
| **R-494** | Install & onboarding | P4 | **NARROWED 2026-09-14 by operator ruling → [P3-LOW] the hub COULD create the tunnel at customer creation, for a domain already on Cloudflare. Not blocking: every customer has their own domain and the operator creates the tunnel per day-0 A.1 (`architecture/01-topology-and-trust.md`).** *Original finding, kept:* **[P1-HIGH] A new customer's dashboard has NO reachable address unless the operator hand-makes a Cloudflare tunnel — the link in the setup-code mail is dead.** MEASURED 2026-09-14 on a fresh install from the public ISO (drill intervention **I1**): the claim mail points at `https://felhom.drill0242.felhom.eu`; that name has **no A and no AAAA** record (`dig @1.1.1.1`, control `felhom.enkisfelhom.hu` resolves); the hub has **no tunnel- or DNS-creation code** (`hub/internal/cloudflare/` holds only geo-rule removal; `cf_tunnel_token` is a pasted, optional form field, `configs.go:1478`) — day-0 runbook A.1 makes it a manual Cloudflare-dashboard step that nothing on the customer-create page asks for; the box's own split-horizon resolver on the appliance LAN IP answered `google.com` but not the dashboard name at 13:27:39Z; the agent applied the record at **13:27:44Z** (`lanresolver: applied split-horizon record … ip=192.168.0.158`, 3 m 46 s after the controller started), so the box CAN answer the name — **but only to a device that uses the box as its DNS server, and no document, screen or mail tells a household to do that**; the router and the installer-offered DNS answer nothing. The page was reachable only at the guest's LAN address with the name forced (`curl --resolve …:443:192.168.0.158`). **A volunteer could not have done that.** **What it needs:** an operator ruling — the hub creates the tunnel and DNS at customer creation, or the product gives a household a LAN address that works with no DNS change. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: operator-side automation; the operator creates the tunnel by hand per the day-0 runbook.** | — | — | CC |
|
||||
| **R-504** | Install & onboarding | P4 | **[P3-LOW] `iso.felhom.eu` cannot show an index page on its own — its root returns 404, and the download page lives on the website instead.** MEASURED 2026-09-14: `https://iso.felhom.eu/` and `/index.html` → 404; only named objects answer. The host is an R2 bucket behind a custom domain; whether R2 would serve an uploaded `index.html` at `/` was **not measured** (uploading anything to the public bucket is a publication). The ISO v1.27.0 task puts the Hungarian download page at `felhom.eu/letoltes` (published with the ISO, after the operator's yes). **Remaining:** a redirect from `iso.felhom.eu/` to that page needs a Cloudflare rule the session has no credential for. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (Cloudflare rule)** **Re-ranked 2026-10-03: P3→P4: households are sent to the website's download page; the bare address is cosmetic.** | — | — | operator |
|
||||
|
||||
## Apps & catalog — 15 rows (P2 2, P3 8, P4 5)
|
||||
## Apps & catalog — 14 rows (P2 2, P3 7, P4 5)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -134,10 +134,9 @@ stopping line that lies.
|
||||
| **R-994** | Apps & catalog | P3 | **Seven apps peak above 80 % of a memory limit, and 23 apps' typical use is above their catalog `mem_request`.** MEASURED 2026-10-13 (`documentation/audits/sizing-2026-10-13/SIZING.md` §3, `evidence/catalog-check.json`). Near a limit — risk: the container is stopped for using too much memory: docmost (container 475 of 512 MB), romm (703 of 768), termix (117 of 128), uptime-kuma (227 of 256, tester-1 telemetry), grafana (435 of 512), immich (`immich-server` 1,665 of 2,048 during a 200-photo import), tandoor (827 of 1,024). No container was killed during the bench. Under-stated request (typical ÷ request): calcom ×4.0, adventurelog ×3.6, termix ×3.4, claper ×2.5, grafana ×2.2, komga ×2.2, docmost ×2.1, recipe-importer ×2.1, uptime-kuma ×2.1, zipline ×2.0, and 13 more ≤ ×1.8. Templates NOT changed (read-only run). | **READY** — filed 2026-10-13 | — | Raise the seven limits and the 23 requests in their templates, each through the catalog's harness; re-measure Immich with a larger library first | CC |
|
||||
| **R-1018** | Apps & catalog | P3 | **The OnlyOffice links (FileBrowser, Nextcloud) have no recorded live proof since the move to Proxmox.** `controller/internal/integrations/` holds two handlers; no audit after the Proxmox move shows a document opened from FileBrowser or Nextcloud in OnlyOffice (searched 2026-10-14). The media-app links of `15` §6 will copy this code, including its revoke-on-stop rule (`lifecycle.go:10-13`). | **OPEN** — filed 2026-10-14 | — | One live walk on a Tier-0 box: install OnlyOffice + FileBrowser, open a document, save, read it back; then stop one app and record what the link does | CC |
|
||||
| **R-1028** | Apps & catalog | P3 | **Plex's claim code can run out during the install, and the box cannot recover it.** plex.tv's code lives 4 minutes from the moment the household copies it; the clock covers the wizard, the image pull (`plexinc/pms-docker` is 351 MB, measured) and the first start. Plex reads `PLEX_CLAIM` only on its first run, and the field is `locked_after_deploy` — so a late code leaves an unclaimed server and the only way back is remove + install again. Reported by the operator during the 2026-10-14 test; on Tester 1's line the pull+start took 8.5 s and the claim worked (`audits/media-all-drives-2026-10-14/``G-auth/G3-plex-claim.txt`), so it bites on a slow line or a slow hand. **K6, 2026-10-14 night (`audits/media-drive-events-2026-10-14/``K6-1-unclaimed-plex.txt`):** an unclaimed Plex starts with no code at all, and `POST /myplex/claim?token=` on the RUNNING server reaches plex.tv (a wrong code → 500 after plex.tv's 401). So the box can pull + start first and ask for the code after: no race. A real claim needs the household's code — the morning step on the throwaway `plex-k6` on Tester 1. Narrowed 2026-10-14 night. **PROVEN 2026-10-15 morning (`K6-2-claim-after-start.txt`):** the operator's real code, given to the RUNNING unclaimed `plex-k6` → 200, `claimed` 0→1, `PlexOnlineToken` written, and the open door closed (no token → 401). `ADVERTISE_IP` was unset and the claim did not need it; whether clients find the server without it was not measurable (the throwaway had no published port). **BUILT 2026-10-15 (v0.313.0):** `after_start` field — not asked at install; the Plex page asks for the code once Plex runs and claims the running server; the gate opens only when Plex says claimed. Proven up to the code on 9201 (G5). | **NARROWED** — the dashboard claim with a real code is the last step | a household Plex code | Claim through the Plex page with a real code (G5c); then close | CC |
|
||||
| **R-1060** | Apps & catalog | P2 | **On controller 0.313.0 (the floor) the Plex claim button can never work: Plex refuses the code with 401 because the box calls it as `http://plex:32400`.** Plex's own log: „Request came in with unrecognized domain / IP 'plex' in header Host; treating as non-local" → `401 POST /myplex/claim` in 0 ms (plex.tv never asked). Measured 2026-10-15 on demo-hp 9201, two codes, the second used ~1 minute after it was made (`audits/media-build-2026-10-15/G7/G5c-claim.txt`, `G5d-claim.txt`). The K6 spike on Tester 1 called the container by its IP address and got 200 (`audits/media-drive-events-2026-10-14/K6-2-claim-after-start.txt`); the product's `plexClaim` (`felhom-controller` `internal/web/media_events.go`) uses the name, and its test replaces the call with a seam, so no test ever met a real Plex. Since 0.313.0 does not ask the code in the wizard (`after_start`), a new Plex on any box at the floor stays unclaimed and behind its setup gate — safe, but unusable. | **OPEN** — filed 2026-10-15 | — | Dial Plex by the container's IP (or send a Host header Plex accepts); a test against a real Plex, red first; then claim 9201's waiting Plex with a fresh code | CC |
|
||||
| **R-1060** | Apps & catalog | P2 | **On controller 0.313.0 (the floor) the Plex claim button can never work: Plex refuses the code with 401 because the box calls it as `http://plex:32400`.** Plex's own log: „Request came in with unrecognized domain / IP 'plex' in header Host; treating as non-local" → `401 POST /myplex/claim` in 0 ms (plex.tv never asked). Measured 2026-10-15 on demo-hp 9201, two codes, the second used ~1 minute after it was made (`audits/media-build-2026-10-15/G7/G5c-claim.txt`, `G5d-claim.txt`). The K6 spike on Tester 1 called the container by its IP address and got 200 (`audits/media-drive-events-2026-10-14/K6-2-claim-after-start.txt`); the product's `plexClaim` (`felhom-controller` `internal/web/media_events.go`) uses the name, and its test replaces the call with a seam, so no test ever met a real Plex. Since 0.313.0 does not ask the code in the wizard (`after_start`), a new Plex on any box at the floor stays unclaimed and behind its setup gate — safe, but unusable. | **NARROWED** — filed 2026-10-15; the fix shipped in controller v0.314.0 (the floor since 13:11 UTC): all Plex calls by address (`mediawire.PlexURL`), red-proved against a fake Plex that refuses the name. Live on 9201 with a FAKE code through the dashboard's claim call: Plex took it as local and asked plex.tv (500 after 938 ms; before: 401 in 0 ms) — `audits/gate-restore-2026-10-15/plex9201/01-fake-code.txt`. A fifth call by name remains — the generic health probe's `GET :32400/identity` — and is harmless: `/identity` answers anyone (200 measured) | the operator's claim code | Claim 9201's waiting Plex through the dashboard with a fresh code (a watcher waits for `~/.felhom-demohp/plex-claim`); prove claimed, gate open, a stranger meets Plex's own login, one library wiring run; then close | CC |
|
||||
| **R-1029** | Apps & catalog | P3 | **Plex's folder picker shows the drives as identical „media” folders.** With one bind per drive at the same path (`/mnt/felhom-drives/<drive>/userdata/media`), the leaf of every library folder is `media`; the drive's name sits two levels up, so in Plex's picker the household sees several folders all called „media” (reported by the operator during the 2026-10-14 test on Tester 1). It conflicts with `15` §2 reason 2 (the same path everywhere). | **READY** — filed 2026-10-14 | the layout decision (STATUS D9) | Decide in `15` §2: the players (`:ro`, never linked) may get a readable path per drive (e.g. `/media/<drive label>`) without breaking the hardlink rule, which binds only the download/library apps | CC |
|
||||
| **R-1036** | Apps & catalog | P3 | **The apps page lags the catalog by two apps.** `app-catalog-felhom.eu/templates/` holds **64** apps; `/alkalmazasok` and `/en/apps` render **63** cards, one of which (`filebrowser`) is built in and not a catalog template — so **62 catalog apps are listed**. Missing: **`plant-it`** and **`wger`**, both in the catalog since 2026-10-06/08. Their website assets are ALREADY present (`plant-it-logo.svg` + 3 screenshots, `wger-logo.png` + 3 screenshots, renamed to the catalog slug by `a3507761`, 2026-10-08) — only the cards were never written, so this is not a new-asset job. Found 2026-10-14 while reconciling the app count for the „felhő tárhely” page, where four places said **60** and the rest said **62**. **Done in that session:** every stated count on the site set to **62** (`index.html`, `szolgaltatasok.html`, `en/services.html` were 60), which is what the apps page actually shows — so no visitor can count the cards and find the number wrong. **62 is still not the catalog's number.** **To close:** write the two cards on both apps pages (gate 15 keeps the two pages' app sets equal, so they move together), then raise every stated count 62 → 64 in the same commit. **The operator chose this split at the 2026-10-14 STOP** over doing it inside the SEO change. **2026-10-15 (R-1048):** Prowlarr's card added on both pages and every stated count 62 → 63 (the media group 19 → 20), so the site still states what it shows; the two owed cards would make it 65. **Note before writing them:** `wger` is `lifecycle: hidden` and `plant-it` `lifecycle: abandoned` — neither is offered for a new install, so a card would advertise an app the wizard refuses. | **OPEN** | — | write the `plant-it` + `wger` cards on both apps pages, then 62 → 64 everywhere, one commit | CC |
|
||||
| **R-1058** | Apps & catalog | P3 | **Prowlarr installed AFTER Sonarr/Radarr is not linked to them until the household presses „Bekötés most" (or a media app restarts).** The box's wiring runs when an app with the `media_library` marker comes up (`felhom-controller` `internal/stacks/manager.go` `onMediaUp`); Prowlarr has no marker (it uses no drive), so its own install starts no run. Measured 2026-10-15 on demo-hp 9201, live catalog, controller 0.313.0 (`audits/prowlarr-2026-10-15/box9201/link.txt`): Sonarr first → its run; Prowlarr after → no run in 150 s; „Bekötés most" wrote `application Sonarr`; Radarr after Prowlarr → linked by itself ~60 s later. The template copy and the website card tell the household this today. | **OPEN** — filed 2026-10-15 by the Prowlarr read-back | — | Start the wiring when Prowlarr comes up too (a list of link-only apps beside the marker, no drive binds); red-proof with the 9201 order (Sonarr, then Prowlarr); then drop the button sentence from the copy | CC |
|
||||
| **R-76** | Apps & catalog | P4 | **FileBrowser-created folders break the setgid chain, and a drop-zone's mode is not stable** **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-26, size S, roadmap state `idea (surfaced by the R-75 spike, 2026-07-26)`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **2026-10-06 night: not a catalog fix** — the setgid chain is set by the controller's FileBrowser setup; a controller row. | — | Two related findings from `audits/SPIKE-catalog-data-paths-2026-07-26.md` P3/P5, both **pre-existing** and deliberately left alone by that spike. **(a)** FileBrowser Quantum 1.3.3 creates files `0644` and folders `0755` and does **not** propagate the setgid bit — even though the entrypoint wrapper's `umask 002` really is in effect (`/proc/1/status` `Umask: 0002`). Group inheritance itself works (a file uploaded into a 2775 group-100 dir landed group 100, not the process gid 1000), so the convention's *group* half holds and only its *mode* half is lost. The consequence is proven with a control: inside a UI-created `0755` folder a gid-1000 process's file landed group **1000**, while the identical write into the 2775 parent landed group **100**. So **any folder a customer creates through FileBrowser breaks the shared-group chain one level down.** Latent today — every userdata-touching catalog app that declares an identity declares uid/gid **1000**, the same uid FileBrowser runs as, so owner permissions mask it; it bites the day a content app runs as a different non-root uid with gid 1000. The comment at `infra/infra.go:156` is right that the image ignores `-e UMASK` but does not say t | CC |
|
||||
| **R-707** | Apps & catalog | P4 | **[P2] 37 apps still start with a login a stranger can take (`09` §3 decision 45).** Audit of all 53 apps: `app-catalog-felhom.eu/FIRST-ADMIN.md` (class, fix route, status, measured or read). Open: **3 hard-coded defaults** — calibre-web (`admin / admin123`, measured working on demo-hp and 9202; its own `cps.py -s` route needs a generated password WITH a special character — our generator is letters+digits, a controller change), mealie (`changeme@example.com / MyPassword`), wger (`admin / adminadmin`); **34 open first-run screens** (the first visitor creates the admin: actualbudget, adventurelog, audiobookshelf, calcom, docmost, emby, ghost, gitea, gramps-web, home-assistant, homebox, immich, jellyfin, komga, n8n, navidrome, opengist, outline, papra, plant-it, radarr, rallly, recipe-importer, romm, seerr, sonarr, sparkyfitness, tandoor, termix, uptime-kuma, vikunja, wanderer, wishlist, zipline). **Stale notes:** romm's `default_creds` `admin / admin` answers 401 on demo-hp (like a wrong password) — the page now warns with a login that does not exist; zipline's looks stale too. **Measured on demo-hp 2026-09-28 (read-only):** bookstack's default still logs in on the INSTALLED app (the fix is for new installs; the page now warns). Each fix: route (a) env or (b) the app's own CLI/API via `after_install:`, proven on 9202 with the default failing and the generated password working; route (c) a page sentence. Several sessions (operator, 2026-09-28). **2026-09-29 (controller v0.280.0, catalog `d0e7e2e`):** every class-3 app fixed — mealie, wger, calibre-web by `after_install` (calibre-web with the new `password:24:special`), proven on 9202 fresh installs (`audits/login-gate-2026-09-29/D/`); the setup gate (decision 46, spike PASSED) built and live on immich, n8n, audiobookshelf (probes measured) and uptime-kuma (button) (`…/C/`); romm's and zipline's stale notes removed. **Left: 30 class-4 apps** — gate each (probe measured on 9202 where one exists — 11 upstream candidates listed in `…/B/B-VERDICT.md` §3; the button otherwise). **2026-09-29 afternoon (controller v0.281.0, catalog `6faf432`):** 28 more class-4 apps gated — 32 of 34 — each proven on 9202 (`audits/gate-rollout-2026-09-29/`B): stranger → gate page / 401, household reached the first-setup screen, the gate opened (9 by a measured probe, the rest by the press), the app answered after. seerr, outline, rallly: gated, their opening needs a media server / e-mail (not proven). **Left:** wanderer (R-714); plant-it is not installable. | **NARROWED** (2026-10-03 triage: the row's verdict was finished, but it names open work no other row carries — seerr, outline and rallly are gated, but the gate OPENING is not proven (needs a media server / e-mail)) — **CLOSED — 2026-09-29 (the rest → R-714)** | — | — | CC |
|
||||
| **R-905** | Apps & catalog | P4 | **wger's collected style files (283 MB) go into every wger backup, though the app rebuilds them at every start.** Since R-762's fix (`DJANGO_DEBUG=False` + `collectstatic`, catalog `cf1ed43`) the static files sit in a named volume that the backup legs copy like data (measured 283 MB, `audits/design-build-2026-10-06/`); `collectstatic` regenerates them, so they are rebuildable bytes in every Tier-1 unit, Tier-2 copy and off-site snapshot. wger is `lifecycle: hidden` and no box runs it (hub read 2026-10-08). Options to weigh: a non-backed-up volume class for regenerable data, or an anonymous volume. Found by the R-762 helpers 2026-10-08. | **OPEN — needs a design before wger is shown** | — | Design the „regenerable volume" exclusion (catalog + controller backup legs) | CC |
|
||||
@@ -189,16 +188,15 @@ stopping line that lies.
|
||||
| **R-878** | Backup & restore | P4 | **A catch-up (R-871) runs the database-dump leg in the DAY, and that leg stops an app with a volume for its copy — the household may notice the stop, and a large volume makes it longer.** MEASURED 2026-10-05 on demo-felhom: the catch-up at 08:25:02 stopped opengist, copied 182.5 KB, started it again — about 1 s, then a few seconds of `health: starting`; the night does exactly the same, unseen. Nothing measured for a large volume. Fix direction (if it matters): skip the volume copy of a running app in a DAYTIME catch-up and leave it to the next night, or warn. `audits/catchup-2026-10-05/partA/live-demo-felhom.txt` | **READY — owner: CC** **2026-10-05 (burn-down night): NEEDS A LIVE MEASUREMENT** — a timed catch-up on 9202 for an app with a multi-GB volume, then the skip-or-warn choice. | — | measure a large volume first | CC |
|
||||
| **R-32** | Backup & restore | P4 | **[P2-HIGH] RESET must purge the customer base dir; the orphan card must stay honest; unattributed bytes must be visible.** The rehearsal's S7 said in advance that an orphan card would BE a finding — and one appeared (16:58:14). Cause: RESET's `"hetzner":"ok"` leg destroys the sub-account, but **a Hetzner sub-account is an access-control object, not a data object** — its directory survives, so re-enabling offsite recreated an account over the previous lifecycle's ciphertext, encrypted under a key that same RESET had destroyed. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-21, size M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **2026-10-06 night: design written** (`audits/night-burndown-2026-10-06/design-R-32.md`) — pick A: purge the repo + set-aside copies through the sub-account's own password login BEFORE deleting it (no main-account credential); the „repo data destroyed" comment at `hub/internal/offsite/offsite.go:276-280` is false for the shared tier; part (3) waits on a read-only measurement (`du` on port 23). Operator: agree to route A. **2026-10-07 07:58: `09` §3 decision 167 — option A yes, and delete the old RESET leftovers once.** **2026-10-07 (Part D step 2, read only):** the hub shows bx11 holding 3.3 GB of data while the four live customers account for ~0.6 GB → **~2.7 GB belongs to no live customer**; no live customer shows a set-aside folder, so the leftovers are in homes whose sub-account is gone, reachable only by the pool box's MAIN account. **The one-time clean-up is STOPPED:** no main-account login for bx11 is held on DooPlex (the API token in the credentials file sees no box). Needs: the main-account login, or the operator lists and deletes in the Hetzner console (`audits/day-2026-10-07/D/D2-listing-readonly.txt`). **2026-10-07: option A DELIVERED (hub 0.142.0):** RESET purges the repo and every `.orphaned-*` through the sub-account's own login before deleting it, and keeps the sub-account if the purge fails (red-proved, `audits/day-2026-10-07/D/`). Not yet seen live (no scratch RESET today). **The one-time clean-up of the ~2.7 GB of leftovers is STOPPED:** it needs the pool box's main-account login (see above). **2026-10-07 14:43 — `09` §3 decision 173: leave the ~2.7 GB of old leftovers for now; re-ranked P2 → P4.** Option A itself is delivered (hub 0.142.0). What stays open is ONLY the leftovers: they were computed (box total minus live customers), never listed, and deleting them needs the pool box's MAIN login, which reaches every customer's data. **Blocker: a safe main-account path.** | — | **Ruling from the run (three parts, deliberately separate):** (1) because RESET destroys custody, the ciphertext it leaves behind is unrecoverable **BY DESIGN** → RESET gains a **main-account purge of the customer base dir** (the existing operator ack already covers it); (2) the **move-aside guard STAYS** for reinstall-*without*-RESET — there custody survives and the card's "history recoverable" promise is true (R-26 depends on exactly that); (3) the operator **Restic tab shows per-customer directory bytes vs attributed snapshot bytes**, so dead data cannot hide. Measured on the pool box that night: **49 M attributed** (2 snapshots, 48.717 MiB) against **1.4 G + 3.0 M unattributed** across TWO `.orphaned-*` dirs. Evidence `restic-and-pool.txt` | CC |
|
||||
|
||||
## Storage & devices — 4 rows (P2 1, P3 2, P4 1)
|
||||
## Storage & devices — 3 rows (P3 2, P4 1)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| **R-1023** | Storage & devices | P2 | **Build: media apps see every drive, and the box says when they must restart (operator rulings 2026-10-14, `09` §3).** Scope from `architecture/15-media-library.md`: one bind per drive of `userdata/media/<library folders>` at the same path; the drive-absent gate per bind (today it keys on the app's one `HDD_PATH`, and the boot gate waits only for THAT drive — measured after the 17:25 power cut); the restart notice after a drive is added, removed or comes back, with „restart now” / „later” and a per-drive „apps restarted since” mark; removal, export and backup classes taught that a shared library is no app's own (R-1027, R-1019). **S5/S6 measured 2026-10-14:** a pulled drive stops none of the media apps (they key on another drive), they keep a dead view; on return nothing restarts them — so the restart notice MUST also cover „a drive came back”, the restart must be stop+start, and Sonarr/Radarr's own health page says nothing about an unreachable root folder, so the box must say it. **Night 2026-10-14 measured every event × app (`15` §10, `audits/media-drive-events-2026-10-14/`):** nothing is lost while a drive is gone (K1), so the build needs NO scan/unmonitor/torrent guard; it DOES need: (1) one bind per PRESENT drive only — a bind to an absent path makes the app unstartable (K3-4: 500 „error while creating mount source path … permission denied”); (2) the media apps recreated without the drive BEFORE a safe removal unmounts it (R-1044); (3) a returned drive restarts every container that binds it, FileBrowser included (K5: 2–10 s per app); (4) „restart now” cuts a playing stream (K5-4: cut, back in 22 s) — check Plex/Jellyfin sessions first or offer „later”; (5) the stale view is found with NO disk I/O: each container's `/proc/<pid>/mountinfo` shows the old device with `shutdown` (K4). Narrowed 2026-10-14 night. **BUILT 2026-10-15 — controller v0.313.0 + catalog `0188430` (`15` §11; `audits/media-build-2026-10-15/`): proven on Tester 1 (G1–G4, G6) and demo-hp 9201 (G7).** What is left is delivery. **2026-10-15 11:29: the operator vouched golden 0.313.0 and raised the global floor to 0.313.0 (read back from the hub DB).** demo-felhom moved to 0.313.0 by itself through the floor; demo-hp was put back on 0.313.0 by hand (it ran the test tag rc.4 and has its own floor override 0.311.0); Tester 1 still runs rc.4, which the floor never moves (R-1053) — fenced in the 2026-10-15 afternoon task. | **NARROWED** — delivered by the floor; Tester 1 waits on R-1053's step | R-1053 (Tester 1 back on a release tag) | Put Tester 1 on the 0.313.0 tag (or let R-1053's fix move it); read it back; close | CC |
|
||||
| **R-330** | Storage & devices | P3 | **Disk health Phase 2 — the three SMART attributes the wire does not carry.** The failing drive's most telling counter was **187 `Reported_Uncorrect`**, sitting at normalized **1** against threshold **0** with a raw count of **1001** — one point from failing and structurally unable to get there. Also wanted: **199 `UDMA_CRC_Error_Count`** (cabling) and **188 `Command_Timeout`**. None are on the agent→controller wire today, so v0.215.0's ladder could not use them. Phase 2 also persists periodic SMART **samples** (the right home is `metrics.MetricsStore`, NOT the Phase-1 state file, which is one record per disk and must stay that way). **This is a declared WIRE change, so under the G-1 gate the hub must model the new fields in the SAME session** — that is precisely why it was kept out of Phase 1, where it would have turned a one-word severity fix into a three-repo change | **READY** — **2026-10-12: the wire half is CLOSED in this row** — released in agent v0.150.0 (boxes run 0.154.0), controller and hub decode 187/188/199. **Only open work:** the verdict slice (use the three counters in the disk verdict and the household's words) and persisting samples. **READY (M) — NEW 2026-08-14** **2026-10-06 night: the wire half BUILT on main (unreleased):** the agent sends SMART 187 `reported_uncorrect`, 188 `command_timeout` (raw, vendor-packed on some drives — carried as reported), 199 `udma_crc_errors` (pointer + omitempty: unknown is absent, never 0); the controller decodes them and the hub's host-report mirror models them (G-1 wire gate green; it convicted the agent half alone). **Carried only** — no verdict, banner, mail or alarm reads them (`TestR330_CountersChangeNoVerdictYet`); using them changes what a household is told and is the next slice, with persisting samples. Red-proofs `audits/night-burndown-2026-10-06/r330/`. Ships with agent v0.150.0 and the next controller and hub releases. | R-328 (closed) | The verdict slice: 187/188/199 into the disk verdict with tests; then persist samples | CC |
|
||||
| **R-332** | Storage & devices | P3 | **The new Hiba-from-counters path has never fired on real hardware.** v0.215.0's whole point is a verdict the product could not previously reach, and it is proven only against the committed fixture's values in unit tests (12 scenario groups, 11 of 12 red-proofs failing as required). The live validation on demo-hp proved the **negative** — three healthy disks still read Rendben across the deploy, no false alert — and the **severity wire** end to end, but no live disk has actually reached Hiba. **This is the honest gap and it must not be closed by pointing at the fixture tests**: the drive that produced the fixture is in DooPlex, which is Tier 2 and never a drill target, and the demo boxes are all-flash and healthy | **WATCHING — NEW 2026-08-14, NARROWED same day.** One item originally in this gap is now PROVEN LIVE: the **persisted state surviving a controller restart**. The v0.215.0→v0.216.0 redeploy destroyed and rebuilt the container, and the new one read back a `changed_at` written by the PREVIOUS version (`2026-08-14T07:23:14.640216851Z`, still intact at 09:31:35Z) instead of re-baselining — Scenario L on real hardware, not just the production-path unit test. **What remains unproven is the verdict itself, plus the stronger restart half: an already-ALERTED disk not re-alerting** | a real degrading disk, or an injection harness | **Closing condition:** a live disk reaching Hiba from counters, OR a deliberate injection through the REAL pipeline (agent `/disks` → controller check → hub event), not a hand-set verdict | CC |
|
||||
| **R-331** | Storage & devices | P4 | **Disk health Phase 3 — growth-rate detection, and retiring the static 64.** The v0.215.0 count backstop (64 unreadable sectors → Hiba) is **a judgement from ONE drive**: the observed benign excursion peaked at 16 and cleared inside an hour, and the terminal run passed 64 at 13 Aug 11:28 and never came back. It is deliberately a backstop BEHIND the sustain rule, not the primary signal, but it is still a magic number tuned on a single sample and it will be wrong for some drive. With Phase 2's history the box can ask the question that actually matters — *is this count climbing, and how fast* — which distinguishes a drive with eight stable aging sectors from one adding forty a day, something no static threshold can do. Revisit 64 when that exists | **READY (M) — NEW 2026-08-14** | R-330 | Growth-rate rule over persisted samples; re-derive or delete the static 64 | CC |
|
||||
|
||||
## Security & access — 22 rows (P1 2, P2 3, P3 11, P4 6)
|
||||
## Security & access — 21 rows (P1 2, P2 3, P3 10, P4 6)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -223,9 +221,8 @@ stopping line that lies.
|
||||
| **R-1040** | Security & access | P4 | **Until the household makes Sonarr's or Radarr's first admin, any container on the box can read the app's API key.** Measured on the bench 2026-10-14 with the R-1030 seed in place: `AuthenticationRequired=Enabled`, `AuthenticationMethod=None` — `/initialize.json` from another container answers 200 with `apiKey`. Outsiders are blocked by the box's setup gate in that window; a compromised other app is not. | **OPEN** — filed 2026-10-14 | — | Seed the API key at install (`SONARR__AUTH__APIKEY`, measured to work) so the box knows it, and close the window by telling the household in the first steps; or seed a Forms user | CC |
|
||||
| **R-1050** | Security & access | P4 | **Raise HSTS to one year on or after 2026-11-15, if no https trouble was reported.** `09` §3 decision 249 (D2 = A, operator 2026-10-15): HSTS stays `max-age=86400` now; after a month with no https trouble it becomes one year (`31536000`); never preload, never `includeSubDomains`. The value is `HSTSMaxAgeSeconds` in `felhom-controller/controller/internal/infra/infra.go`, pinned by `TestRenderForwardedHeaders_HTTPSOnly`. „Trouble” = any report of a name that will not open, or a box answering http after the floor. | **WATCHING** — filed 2026-10-15 | the date | On or after 2026-11-15: read the register and the operator's mail for https trouble since 2026-10-15; none → a controller release with one year (test updated), bake → vouch → floor | CC |
|
||||
| **R-1054** | Security & access | P3 | **The hub's operator/global API key is still one bearer that can do everything on `/api/v1`, including reading every box's break-glass root password (`GET /admin/hosts/{id}/recovery-credential`).** Hub v0.152.0 gave the web surface scoped tokens (`05` §17) but tokens are refused on `/api/v1` by design. Who uses the global key today is not listed in one place. Fix: list its consumers; move each operator use to a scoped token on the web side (or a token class on the API), and keep the global key only where a box path needs it — or retire it. | OPEN — filed 2026-10-15 by the token build | — | List the global key's consumers (pod env, scripts, runbooks) | CC |
|
||||
| **R-1057** | Security & access | P3 | **A restore of a REMOVED gated app comes back without its setup gate, even when the backup was taken before the household's setup — so the app returns open, with no account.** Measured 2026-10-15 on 9202 with Prowlarr (`audits/prowlarr-2026-10-15/box/run3/restore.txt`): installed (gated, no account made), the guarded Update's per-app backup, remove keeping the backups, the household's restore button — afterwards a stranger's `/ping` through traefik answered 200, where before the remove it answered 401 (the gate). This follows the documented rule (`felhom-controller` `internal/stacks/life_records.go` `carryLifeRecords`: no prior record = no gate, „the data comes back with its admin"); the rule's assumption fails for a backup made before the setup. For a first-visitor app (Prowlarr, n8n, …) the first stranger sets the sign-in; for Prowlarr `/initialize.json` also hands out the API key (seen before the setup in the install window — NOT measured after this restore). Needs: the household to install, let a backup run before finishing the setup, remove, restore. | **OPEN** — filed 2026-10-15 by the Prowlarr walk | — | Record the gate's state in the per-app backup and re-close it on restore when it was closed then; red-proof with this walk (`tools/walk_restore.py`) | CC |
|
||||
|
||||
## Box system & updates — 13 rows (P2 1, P3 9, P4 3)
|
||||
## Box system & updates — 13 rows (P2 1, P3 8, P4 4)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -240,8 +237,8 @@ stopping line that lies.
|
||||
| **R-1004** | Box system & updates | P4 | **After an unclean stop every guest ext4 volume waits ~43 s in multi-mount protection before it mounts, so each restart after a power cut takes ~1.5 min longer.** Kernel: „EXT4-fs warning (device dm-6): ext4_multi_mount_protect: MMP interval 42 higher than expected, please wait" on dm-6/dm-7 and loop0/loop1, demo-hp and Tester 1, every round of `audits/night-power-2026-10-13/` (R1 `repairs.txt`). | **READY** — filed 2026-10-14 | — | CC: find where MMP is switched on for the guest volumes (Proxmox `pct` create, or our own mkfs), whether it protects anything on a one-host disk, and measure the restart without it on a scratch guest | CC |
|
||||
| **R-1013** | Box system & updates | P4 | **After a power cut the agent's start-up stale-lock pass logs an ERROR „pct unlock failed … no lock found" when Proxmox's own start-all removed the lock in the same second.** The outcome is the goal (no lock), so the line is a false alarm in the log. Seen on demo-hp 2026-10-14 12:07:44–45 UTC (`audits/power-cut-fixes-2026-10-14/E/after-boot.txt`; Proxmox: „removed left over backup lock from '9201'!"). Small, but the agent's release for this session is spent (v0.156.1). | **OPEN** — filed 2026-10-14 | — | CC: treat „no lock found" from `pct unlock` as already clear (INFO), with a test; ride the next agent release | CC |
|
||||
| **R-1010** | Box system & updates | P3 | **The agent's stale-lock janitor cannot remove a half-removed `vzdump` snapshot: Proxmox's delete fails on the first missing volume („Failed to find logical volume pve/snap_vm-9201-disk-0_vzdump") and the janitor retries the same call every 10 min forever, leaving the other volume's snapshot and the snapshot entry.** Seen 2026-10-14 after a backup ran out of space on a full host root (LVM could not write its metadata; `audits/power-cut-fixes-2026-10-14/README.md` C4). Fixed by hand with `pct delsnapshot 9201 vzdump --force`. | **READY** — filed 2026-10-14 | — | CC: on that error, retry once with `--force` (a sudoers line or the API's `force=1`), with a test that replays the error text; next agent release | CC |
|
||||
| **R-1052** | Box system & updates | P3 | **The new-box path is not proven: a fresh install from a golden older than the floor reaching the floor by itself at first boot.** `09` §3 decision 250 relies on it (the golden is weekly, releases ride the floor). Proof owed at the next controller release: a throwaway VM on demo-hp installed from the ISO (the vouched golden one release behind the floor); time from claim to the floor; the controller's self-update log lines; then the VM and its hub records removed. | **OPEN** — filed 2026-10-15 | the next controller release | Run it at the next release; quote the times | CC |
|
||||
| **R-1053** | Box system & updates | P3 | **A box running a controller test tag (`0.313.0-rc.N`) is never moved by the floor, and says so only at DEBUG.** `util.ParseVersion` wants X.Y.Z, so `maybeAutoUpdate` logs `current … not parseable (dev?) — skip` and the box stays on the test build forever (measured 2026-10-15 on demo-hp and Tester 1 after the floor reached 0.313.0; demo-felhom, on a release tag, moved by itself). The same parser keeps an rc tag off the household's update button — that half is wanted. Today's guard is procedural (the `felhom-build-deploy` skill: write the release tag back after testing). Tester 1 still runs rc.4. | **READY** — filed 2026-10-15 | — | A controller fix: treat `X.Y.Z-rc.N` as a pre-release BELOW `X.Y.Z` for the floor (and keep it off the button), log the skip at WARN; a test per case. Meanwhile put Tester 1 on 0.313.0 | CC |
|
||||
| **R-1052** | Box system & updates | P3 | **The new-box path is not proven: a fresh install from a golden older than the floor reaching the floor by itself at first boot.** `09` §3 decision 250 relies on it (the golden is weekly, releases ride the floor). Proof owed at the next controller release: a throwaway VM on demo-hp installed from the ISO (the vouched golden one release behind the floor); time from claim to the floor; the controller's self-update log lines; then the VM and its hub records removed. | **BLOCKED** — filed 2026-10-15; not run with the 0.314.0 release: the proof needs a throwaway customer, and creating a customer is session only since hub 0.153.0 (`09` §3 decision 252) — CC's token is refused by design | the operator: create a throwaway customer (or name one to reuse) in the browser | Then: a VM on demo-hp from the current ISO, claimed as that customer; time from claim to the floor; remove the VM; the operator deletes the customer (session only) | CC |
|
||||
| **R-1061** | Box system & updates | P4 | **A per-customer floor reached Tester 1 only at its next 15-minute report, not in seconds.** Set 2026-10-15 12:55:51 UTC for tester-1, demo-hp and demo-felhom together (hub 0.153.0, CC's token): demo-hp and demo-felhom ran 0.314.0 within 13 s (the hub's wake); Tester 1 moved at 13:09:35, its regular report (`audits/gate-restore-2026-10-15/release/03-boxes-move.txt`). It had restarted at 12:54:29 onto rc.1 and logged „hub wait channel active". Why the wake did not reach it is not read (the wake line is DEBUG). No harm: the floor arrived. | **OPEN** — filed 2026-10-15 | — | Read Tester 1's DEBUG ring at the next floor change; compare with the hub's bump log | CC |
|
||||
|
||||
|
||||
## Monitoring & notifications — 10 rows (P2 1, P3 6, P4 3)
|
||||
|
||||
Reference in New Issue
Block a user