diff --git a/STATUS.md b/STATUS.md index bc48b05..9006f8f 100644 --- a/STATUS.md +++ b/STATUS.md @@ -91,6 +91,11 @@ being readable.* it writes PASS/FAIL to `/var/log/felhom-store-postboot-check.log`. On PASS, 34 GB comes back. *(R-209a)* - **Backup scripts on DooPlex are unversioned host state** *(R-231)*, and the instruction-file follow-ups each need a decision rather than an edit *(R-229, R-230)*. +- **The Configuration page: 26 s → about 10 s, and not finished.** It was never hashing anything — + the hashes are already stored and just read. It was making 42 calls one after another. What is + left is a single slow Gitea lookup per dropdown; making it instant means holding a short-lived + copy, which trades freshness for speed and is your call. **Numbers were taken on a busy box and + swing 3×** — worth a re-check when it is idle. *(R-267)* - **Our build-check alarm has one gap left.** A run that hangs is now cut off after five minutes and the mail says how long it took — but **whether the alarm fires at all when the machinery kills a run outright is still unverified**, and we have not claimed otherwise. *(R-265)* diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index 526e20c..3bc741c 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -494,6 +494,8 @@ applied.** The one that matters: Scenario A **fails against today's tree** with |---|---|---| | **R-266** | **A failed root `statfs` still reaches the hub as a 0-of-0 disk, and the hub cannot tell that from an empty one.** Split out of R-259 on 2026-08-08 so that fixing the CUSTOMER-facing half could not be mistaken for fixing the wire. `report/builder.go:93-95` copies `sysInfo.DiskTotalGB` / `DiskUsedGB` / `DiskPercent` into `r.Storage[0]` (`Mount: "/"`), and those are exactly the zeros a failed `statfs` leaves behind — the controller now KNOWS the measurement failed (`SystemInfo.DiskKnown`, controller v0.210.0) and the report still does not carry it. **Deliberately not fixed here, for a reason that is now structural rather than a preference:** adding a field to that report is a change to a declared wire, which since G-1 means the receiving side must model it in the same session (`scripts/wire_contract_gate.py` refuses otherwise) — a two-repo change with a hub bump, and this session deliberately touched no hub code. **RANKED LOW, and the reason is that the consequence is bounded:** the hub bands host storage on `disk_percent`, so a failed read presents as 0% used — the *quiet* direction. It cannot raise a false "nearly full" alarm; it can only fail to raise a true one, and only while the root filesystem is unreadable, which is a state with louder symptoms of its own. **Fix shape when it is taken:** carry `disk_known` on the storage entry and have the hub's fill checker skip an unknown reading rather than band it — never treat absent as 0 | **READY** — owner Viktor | +| **R-267** | **The Configuration page is 2.6× faster and is still ~10 s, and the remaining cost is ONE Gitea call whose latency swings 20× with load.** Reported by the operator as *"almost minutes to load"*, with the reasonable guess that it hashes artifacts on page load. **THAT GUESS DOES NOT HOLD and the code already said so** — Gitea stores each file's sha256 and `gitea.FileSHA256` reads it as metadata (*"the artifact bytes are never downloaded"*). The cost was latency × count. **Fixed in hub v0.100.0–0.100.2, three legs, each found by refusing to accept a number that did not match the arithmetic:** (1) the per-version sha lookups were **serial** — 2 packages × (1 search + 20 lookups) = 42 sequential calls; now concurrent, bounded 8. (2) The two dropdowns resolved **one after the other**; now side by side. (3) The client used `http.DefaultTransport`, whose **`MaxIdleConnsPerHost` is 2**, so under a 16-way fan-out nearly every call paid a fresh TCP setup *and* a fresh authentication — and authentication is the expensive half (`/api/v1/version` unauthenticated **0.03 s** vs an authenticated package call **0.24 s**). **Measured: 26.2 s → mean 9.85 s over 8 samples (min 5.13, max 18.13).** **THE REMAINING COST IS THE PACKAGE SEARCH** `/api/v1/packages/admin?type=generic&q=…&limit=100`, one per dropdown: **0.20–3.8 s each depending on load**, and running the two concurrently does not help (3.68 s for both together vs 3.82 s for one alone — Gitea appears to serialise them). By contrast **16 concurrent file-metadata calls take 0.58 s**. **⚠ EVERY NUMBER HERE IS CONTAMINATED and that is stated rather than hidden:** they were taken on DooPlex at load average 7–11 while this same session was building images, running two Go suites and baking a golden. The same search measured **3.8 s** in-cluster and **0.44 s** from the host ninety seconds later. **Re-measure on an idle box before deciding anything.** **The operator's two proposals, answered on the measurement:** *"reduce the number of artifacts"* — only **50 generic versions exist in total** (33 agent + 17 golden) and `limit=25` costs 0.20 s against `limit=100`'s 0.44 s, so pruning helps **somewhat and sub-linearly**; it is worth doing for its own sake (it pairs with R-210) but it is not the lever. *"hash on creation, store in the DB"* — **NOT RECOMMENDED, and the reason is a rule this project already holds**: Gitea IS the store, and a copy in `hub_settings` would be a second source of truth that can drift from the registry it describes, while the operator reads exactly that value to confirm what they are about to vouch. `golden_currency_gate.py` records the same reasoning for the vouched version. **The lever that would actually work, and the trade-off that makes it a DECISION rather than an implementation:** cache the version list + shas **in memory** with a short TTL and refresh in the background — the page becomes instant and bounded-stale, at the cost that a just-published artifact does not appear for up to the TTL. That is an operator call about how fresh the dropdown must be, and it is why this row is open rather than closed. **Cheaper interim, also a decision:** drop the dropdown cap from 20 to ~8, which cuts the fan-out but hides older versions from a rollback | **READY** — owner Viktor | + **Explicitly still open, untouched by this session:** R-246, R-255, R-256, R-257, R-261, R-262, R-263, **R-264** (the twenty-one undecided facts — a design session of its own), R-240, R-243, R-202, R-213, R-244, R-214/R-235, and **C7's test-comment half**, which Campaign 12 recorded as