From 36bcd12543cef74247fe21ddd1271b9756042718 Mon Sep 17 00:00:00 2001
From: kisfenyo <nagyfenyvesi.viktor@gmail.com>
Date: Sun, 9 Aug 2026 19:15:17 +0200
Subject: [PATCH] Deploy hub v0.102.0, and record that the package deleter is
 still not established

manifests/hub.yaml 0.101.0 -> 0.102.0. Image built and pushed, and verified
served by the registry before the bump rather than after.

R-287 corrected on two counts. My own sentence "no DELETE on the packages API
appears in 48h of Gitea router logs" is WITHDRAWN: kubectl logs on the Gitea pod
now returns nothing older than 2026-08-09 16:35 and contains zero api/packages
lines even for requests I made myself, so the log never covered the window and
its silence was never evidence.

A second attempt to attribute the deletion also failed and the deleter remains
NOT ESTABLISHED. Sources exhausted: no register row records a package prune
(R-210 is WAITING-ON-OPERATOR, says "Nothing was deleted; this is a list, not an
action", and concerns local Docker images); package_version has no soft-delete
column so a deletion leaves no row; Gitea's action feed carries no package
operation at all in the window; and a uniform newest-ten cap is not visible --
felhom-agent generic holds 10 but the container packages hold 19 each. It may be
unestablishable from this side: Gitea keeps no package-deletion trail.
---
 documentation/backlog/OPEN-ITEMS.md | 2 +-
 manifests/hub.yaml                  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md
index b2f8ea4..2a7f425 100644
--- a/documentation/backlog/OPEN-ITEMS.md
+++ b/documentation/backlog/OPEN-ITEMS.md
@@ -515,7 +515,7 @@ applied.** The one that matters: Scenario A **fails against today's tree** with
 | **R-284** | **„A kiválasztott tárhely majdnem megtelt." on a store that is 93 % FREE — an apparent inverted threshold.** Calibre-Web's deploy page rendered `<option value="/mnt/sys_drive" data-free-percent="93">` alongside „Tárhely (sys_drive) — **64.2 GB szabad**" and the warning „A kiválasztott tárhely majdnem megtelt." 93 % free read as 93 % used is the obvious candidate, and `checkStorageSpace(this)` is the function to look at. **Not confirmed by reading the code** — reported as measured output only. A capacity warning that cries wolf on an empty disk is one a customer learns to click past | **READY (XS) — NEW 2026-08-09** | — | Check `checkStorageSpace`'s comparison against `data-free-percent`; add a render test per branch | CC |
 | **R-285** | **A planned, supervised reinstall pages the operator as if the machine had died — there is no notion of expected downtime anywhere.** During the 2026-08-09 rehearsal the hub sent, all `status: sent` to the operator channel: `host_stale` 08:58 UTC, `node_stale` 09:00, **`host_down` 09:28 (error)**, **`node_down` 09:30 (error)**, `host_leaf_changed` 09:31, `host_recovered` 09:31, `node_recovered` 09:34, `offsite_delivery_stuck` 09:34 — eight operator mails for work that was deliberate, attended and announced. **This is the OPPOSITE gap from the one R-281 filed:** the alarms are not missing, they are indiscriminate. `host_stale` at 30 min and `host_down` at 60 min (`monitor/host_staleness.go:22-23`, `downAfter = 2 * threshold`) cannot distinguish a wiped-on-purpose box from a dead one, and `host_leaf_changed` firing on a reinstall is correct-but-expected. **Note the interaction with the mute used on 2026-08-09 evening:** blocking a customer silences everything, so today the only two settings are *page me for planned work* and *tell me nothing at all*. **What is owed is a middle:** a maintenance window, or an operator-set expected-downtime flag, that suppresses staleness and leaf-change while leaving genuine faults audible | **READY (M) — NEW 2026-08-09** | — | The evidence is the operator's mailbox plus `events`/`notification_log` for 2026-08-09 | CC |
 | **R-286** | **A control drawn from the same channel as the measurement cannot detect a defect in that channel — and this one passed while the measurement was wrong.** The P7 check asked *"did the hub record anything?"* against a stale snapshot, got "no", and then validated itself with *"is the hub recording ANY events today, for anyone?"* — **against the same stale snapshot**. It answered "2 events all day", which was internally consistent and entirely false. The standing rule (*an absent log line is not evidence*) was followed in form: a positive control WAS run. **It was the wrong kind of control**, and nothing in the rule as written says so. **The independent channel existed and was available the whole time: the operator's mailbox.** One glance at it would have shown eight alarms in the window. **The durable lesson, to be added where the standing rules live:** a control must come from a DIFFERENT channel than the measurement — same query, same snapshot, same API, same clock all fail this. **Concrete follow-through owed:** (a) add this to the standing rules in `runbooks/workspace-CLAUDE.md`; (b) any hub-state check in a runbook must copy `-wal` or query the pod directly, never `cat hub.db` alone — the trap `operations/nodes.md` already documents | **READY (S) — NEW 2026-08-09** | — | Parent: R-281 (withdrawn) | CC |
-| **R-287** | **`felhom-agent` CI is red for a TRUE reason, and the diagnosis it was filed under is wrong in every particular.** The task premise was *"the gate is sensitive to being run against a tag ref rather than a branch"*. **It is not.** `check-published-versions.py` enumerates releases from the **Gitea tags API** (`/api/v1/repos/admin/felhom-agent/tags?limit=200`, `main()`), so the checked-out ref is irrelevant; and the two previous tag pushes **passed** (run 190 `v0.126.0`, run 216 `v0.127.0`). **What is actually true:** run 267 (main, `28ba8593b8`, 2026-08-08 14:29 UTC) printed `ok v0.120.0: binary downloadable`; run 284 (tag, **the same commit**, 2026-08-09 09:30 UTC) printed `FAIL v0.120.0 — binary NOT downloadable (HTTP 404)`. **A published release became uninstallable between those two runs.** The registry now holds exactly the ten newest versions (0.121.0…0.128.0); `0.128.0` was published **2026-08-08 16:47 CEST = 14:47 UTC, eighteen minutes after run 267**, and `0.120.0` is gone. **WHO REMOVED IT IS NOT ESTABLISHED, and that is stated rather than guessed:** `package_cleanup_rule` is **empty** (queried in Postgres), `app.ini` sets no package limit, `publish-agent.sh` only pre-deletes the version it is publishing (`:77`), the Gitea pod has **53 days uptime and 0 restarts** so `RUN_AT_START` did not fire, and **no `DELETE` on the packages API appears in 48 h of Gitea router logs**. The leading candidate is the internal `[cron.cleanup_packages]` `@midnight` job, which falls inside the window and would leave no router log line — **leading candidate is not established.** **THEREFORE NO GATE WAS SILENCED AND NO WORKFLOW WAS CHANGED.** Silencing it would hide a released-but-uninstallable version, which is the exact R-115 defect the gate exists to catch. **It will recur:** if the ten-version window is real, the next publish evicts `0.121.0`. **Two honest fixes, both out of tonight's scope:** bound the gate to versions at or above the vouched `min_agent` floor (0.127.0 today — nothing installs 0.120.0 and nothing can), or retire ancient tags when their packages go. **Also measured, and good news:** the failure alarm DID send — `RESEND-ACCEPTED id=fa1a7a83-714f-4357-b0ca-d3c4bb7ae73f` | **READY (M) — NEW 2026-08-09** | — | Establish the deleter first; do not raise the retention until it is known | Viktor |
+| **R-287** | **`felhom-agent` CI is red for a TRUE reason, and the diagnosis it was filed under is wrong in every particular.** The task premise was *"the gate is sensitive to being run against a tag ref rather than a branch"*. **It is not.** `check-published-versions.py` enumerates releases from the **Gitea tags API** (`/api/v1/repos/admin/felhom-agent/tags?limit=200`, `main()`), so the checked-out ref is irrelevant; and the two previous tag pushes **passed** (run 190 `v0.126.0`, run 216 `v0.127.0`). **What is actually true:** run 267 (main, `28ba8593b8`, 2026-08-08 14:29 UTC) printed `ok v0.120.0: binary downloadable`; run 284 (tag, **the same commit**, 2026-08-09 09:30 UTC) printed `FAIL v0.120.0 — binary NOT downloadable (HTTP 404)`. **A published release became uninstallable between those two runs.** The registry now holds exactly the ten newest versions (0.121.0…0.128.0); `0.128.0` was published **2026-08-08 16:47 CEST = 14:47 UTC, eighteen minutes after run 267**, and `0.120.0` is gone. **WHO REMOVED IT IS NOT ESTABLISHED, and that is stated rather than guessed:** `package_cleanup_rule` is **empty** (queried in Postgres), `app.ini` sets no package limit, `publish-agent.sh` only pre-deletes the version it is publishing (`:77`), the Gitea pod has **53 days uptime and 0 restarts** so `RUN_AT_START` did not fire, and I wrote that *"no DELETE on the packages API appears in 48 h of Gitea router logs"*. **THAT SENTENCE WAS AN OVER-CLAIM AND IS WITHDRAWN 2026-08-09 (evening).** Re-checked: `kubectl logs --since=72h` on the Gitea pod returns nothing older than **2026-08-09 16:35** — the container log has rotated, and it contains **zero** `api/packages` lines even for requests I made myself. **The log does not cover the window, so its silence was never evidence** — the exact rule this project keeps re-learning. **A SECOND ATTEMPT TO ATTRIBUTE THE DELETION ALSO FAILED, and the deleter remains NOT ESTABLISHED.** Sources exhausted: (1) no register row records a package prune — **R-210** is the only prune-adjacent row, it is `WAITING-ON-OPERATOR`, it says in terms *"Nothing was deleted; this is a list, not an action"*, and it concerns **local Docker images on DooPlex, not the Gitea registry**; (2) `package_version` has **no soft-delete column** (`created_unix, creator_id, download_count, id, is_internal, lower_version, metadata_json, package_id, version`) so a deletion leaves no row; (3) Gitea's `action` feed carries **no package operation at all** in 2026-08-08 → 2026-08-09, and **nothing whatever on the evening of 2026-08-08**; (4) a uniform *"newest ten per package"* cap is **not visible in the current state** — `felhom-agent` generic holds 10, but `felhom-controller` and `felhom-hub` container packages hold **19 each**. **It may be unestablishable from this side: Gitea keeps no package-deletion trail.** What remains established is unchanged — v0.120.0 downloadable 2026-08-08 14:29 UTC, 404 by 2026-08-09 09:30 UTC, and the generic package now holding exactly the newest ten. **THEREFORE NO GATE WAS SILENCED AND NO WORKFLOW WAS CHANGED.** Silencing it would hide a released-but-uninstallable version, which is the exact R-115 defect the gate exists to catch. **It will recur:** if the ten-version window is real, the next publish evicts `0.121.0`. **Two honest fixes, both out of tonight's scope:** bound the gate to versions at or above the vouched `min_agent` floor (0.127.0 today — nothing installs 0.120.0 and nothing can), or retire ancient tags when their packages go. **Also measured, and good news:** the failure alarm DID send — `RESEND-ACCEPTED id=fa1a7a83-714f-4357-b0ca-d3c4bb7ae73f` | **READY (M) — NEW 2026-08-09** | — | Establish the deleter first; do not raise the retention until it is known | Viktor |
 | **R-288** | **The capability map is too long to be read, and that is why it stops being true.** `architecture/00-capability-map.md` is **134 642 bytes / 19 456 words across 99 table rows in only 159 lines** — because the rows ARE the length. Measured, longest first: the unaided-recovery-journey row is **3 024 words**, the offsite-password-recovery row **1 087**, the unattended-restore-proof row **971**, the app/guest-network-failure row **904**. That single longest row is a novella of nested corrections, each appended rather than resolved. Its own verification stamp reads **2026-07-16 against evidence corpus @ felhom.eu tip `4b18cc5`** (line 23) — three weeks stale, which is the measurable consequence: nobody re-reads a row they cannot finish. **This is the project's memory, so restructuring it is surgery and wants daylight** — filed, deliberately not attempted in the 2026-08-09 session. **What the shape should probably be:** one line of status per capability plus a dated evidence pointer, with the argument moved to the audit it came from | **READY (M) — NEW 2026-08-09** | — | Do not fold this into another session; it needs its own | Viktor |
 | **R-289** | **R-182's register row describes a defect the code no longer has — an OPEN row that is a false alarm.** The row reads *"A full disk tells the operator about ONE app and silently swallows every other app's refusal for an hour"*, cited at `hub/internal/notify/dispatcher.go:268`. **Read against live source 2026-08-09, that is fixed:** the per-run digest `backup_run_failures` is allowlisted (`hub/internal/api/handler.go:1837`), operator-only (`dispatcher.go:423`) and templated (`notify/templates.go:48`); `recovery_unit_capture_failed` is now a **record-only** event (`dispatcher.go:376`) whose notification IS the digest, listing every failed app in one mail; and a cooldown drop now writes a `suppressed` row instead of vanishing (`dispatcher.go:314-330`). The capability map already records the fixed shape (*"EVERY failing app, in ONE mail per run"*). **So the register is behind the code, which is the mirror of the decay this session was looking for** — the session expected stale PROOFS and found a stale DEFECT. **Not closed here, deliberately:** the digest's *delivery* has never been observed end to end (the page's own "an app crashes — the email leg has never been confirmed" card), so the honest move is to re-scope R-182 to that residue rather than tick it | **READY (XS) — NEW 2026-08-09** | — | Re-scope R-182 to "the digest has never been seen delivering", or close it and open that | CC |
 | **R-290** | **Most capability-map rows that back a green dot cite no evidence document at all — measured, 20 of 28 probed.** The page's *Walked* means *"done end to end on real hardware, evidence on file"*. Extracting the evidence column for the 28 rows behind the page's claims found a `tests/` or `audits/` path in **8**; the other 20 carry prose only. **Consequence, applied this session:** of 32 claims the page drew as Walked, **12 were downgraded to Built** because no walk document exists for them — `install.installer-by-tag`, `use.lifecycle`, `drives.enrol`, `drives.migrate`, `backup.tier1`, `backup.whole-machine`, `backup.restore-proof`, `fault.selfheal`, `fault.operator-email`, `fail.drive-filling`, `fail.lost-recovery-code`, `fail.hub-down`. **This is not a claim that those twelve are false** — several are near-certainly fine — it is a claim that nothing on file distinguishes them from an opinion, which is exactly what the status word promises. **The gate now enforces it going forward:** `scripts/check_stands.py` fails on `status: walked` with no `evidence:` source. **What is owed:** either a walk document per row, or an honest demotion in the map itself (the map is the source; the dataset only follows it) | **READY (M) — NEW 2026-08-09** | R-288 | The dataset was corrected; **the capability map itself still says PROVEN-LIVE for these rows** and is the thing to fix | Viktor |
diff --git a/manifests/hub.yaml b/manifests/hub.yaml
index 12469b2..cc9bd02 100644
--- a/manifests/hub.yaml
+++ b/manifests/hub.yaml
@@ -125,7 +125,7 @@ spec:
     spec:
       containers:
         - name: hub
-          image: gitea.dooplex.hu/admin/felhom-hub:0.101.0
+          image: gitea.dooplex.hu/admin/felhom-hub:0.102.0
           ports:
             - containerPort: 8080
               name: http