From 3d0a1d615d11d46f2a34ec5d29dc67c3ac962e67 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Mon, 3 Aug 2026 15:36:46 +0200 Subject: [PATCH] REPORT: restart proof, R-188/R-189, and the restored defaults --- REPORT.md | 47 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/REPORT.md b/REPORT.md index c2c3866..5e8e9cb 100644 --- a/REPORT.md +++ b/REPORT.md @@ -212,6 +212,25 @@ after it — the restore reclaimed to the same shape it started in. ### 7.5 The restart, which is the defect a person would actually notice +Under the old scheduler every agent deploy restarted the ticker, so a restore-test ran one interval +after each deploy regardless of what had been proven. The proof of the fix cannot be *"nothing +appeared in the log"* — that is the absent-line trap this project has a standing rule about — so +v0.121.1 makes a quiet evaluation say what it decided, and the evaluation interval was shortened to +2 min for the validation so evaluations are **observable**, not assumed: + +``` +15:32:10 felhom-agent daemon starting version=0.121.1 ← the restart +15:32:11 backup: restore-test scheduler starting (per-archive due-check) eval_interval=2m0s settle=24h0m0s +15:34:12 backup: restore-test evaluated — nothing due + verdicts="felhom-backup: no settled archive yet — nothing to prove (newborn or still settling); + felhom-pbs: newest settled archive (landed 2026-07-28T04:49:43Z) is already proven" +15:36:12 backup: restore-test evaluated — nothing due (same verdicts) + +runs since the restart: 0 +``` + +Evaluations demonstrably **happened** and demonstrably **decided**; nothing ran. The 6 h default was +restored afterwards (§8). ## 8. Release and deployment @@ -223,7 +242,12 @@ after it — the restore reclaimed to the same shape it started in. | Gate re-run after release | `2 released version(s) to verify: 0.120.0, 0.121.0` → both **installable** | | Deployed | `felhom-agent --version` → **0.121.0**, `systemctl is-active` → **active**, prior binary kept as `.bak-0.120.0` | | Startup | `capabilities self-check ok=68 total=68 degraded=0`, and `backup: restore-test scheduler starting (per-archive due-check) eval_interval=6h0m0s settle=24h0m0s` | -| **Vouching** | **NOT done — deliberately the operator's act.** Hub UI → Configs → Day-0 artifacts: agent `0.121.0`, sha above | +| Second release, same path | **v0.121.1** — tag `v0.121.1`, sha256 `afaeeb509d1ed70d6e6bebac0393a3cd5be59d51e3db9ff96ef8524bd78546d7`, round-trip verified | +| Deployed (published bytes again) | `felhom-agent --version` → **0.121.1**, `active`; prior kept as `.bak-0.121.0` | +| Validation config removed | the 2-min drop-in deleted; the daemon back on **`eval_interval=6h0m0s settle=24h0m0s`** | +| Fleet | demo-hp still runs **0.120.0** — deliberate: pointing machines at a version is what **vouching** does | +| **Vouching** | **NOT done — deliberately the operator's act.** Hub UI → Configs → Day-0 artifacts: agent **`0.121.1`**, sha `afaeeb50…` (0.121.0 also published, sha `b2128f3c…`) | +| Config compatibility checked on both boxes | `restore_test_cadence_seconds = 0` on demo-felhom **and** demo-hp, and the installer writes `0` — so no box is on the deprecation path, and a fresh install gets the new defaults with no installer change | ## 9. Findings filed (none fixed blind) @@ -246,6 +270,27 @@ after it — the restore reclaimed to the same shape it started in. run created was withdrawn (nothing had been published under it — verified 404) and recreated on the fix commit, so one version name still means one binary. +- **R-188 — a correct agent release emails a CI failure about half the time.** `on: [push]` fires the + gates workflow on the **tag** push too, and `release-agent.sh` pushes the tag before publishing + (deliberately). CI can therefore run the published-versions gate inside the window where the tag + exists and the package does not, and correctly report *"every released agent version must be + INSTALLABLE"* for a release that completes seconds later. **Measured across two releases in one + session:** v0.121.0 → runs #12 success / #13 failure on the same sha; v0.121.1 → #17 failure / #18 + success on the same sha; and one pair both green — a race, not a rule. It matters because R-168 + made CI email on failure so a red gate cannot be missed; a signal that cries wolf on every second + correct release is how that mail becomes something you archive unread. +- **R-189 — a passing restore-test can be invisible to the hub, and R-86 widened that window.** + Observed live: **the 15:25:14 PASS reached no host-report at all**. `restore_tests[]` comes from an + **in-memory** store (*"lost on restart; the cadence re-populates"*) and the report interval is + 900 s; the agent was restarted 2 m 43 s after the run for the v0.121.1 deploy. That used to + self-heal within 24 h because the next cadence re-tested the tier — **under per-archive due-ness + the agent will not re-test a proven archive**, so the hub can stay ignorant until the next archive + generation. The persisted proof already exists: `RestoreTestState.Snapshot()` is documented *"for + the host-report gauge"* and has **no production caller** — a seam built and never wired, and an + invariant asserted only in a comment, in one method. Bounded, not over-ranked: the hub scans its + retained window and the offsite tier's window (12 d) is wider than its archive rhythm (7 d), so one + lost report is tolerated. Filed, not fixed — it is a report-contract change. + ## 10. CI | Repo | Run | Commit | Result |