D53 built + live-proven (08 §10 [DESIGN — built]); R-1297 decoys: register-shape decoys assert the script's exit
gates / gates (push) Successful in 3m25s

- 08 §10: built (controller 5bd03ad, hub 7e57e824, commit only); putback_pending as on the wire; as-built notes
- audits/d53-held-apps-2026-10-26: live proof on 9202 against a test hub (3-min grace), red-proofs, teardown
- R-1291 BUILT (not released); R-1298 filed (9202's orphan holds + PrivateBin leftover)
- test_gate_decoys.py: the register-shape cases also run the script and assert its exit code — R-1297's decoy 2
  showed an inert main() passed every case (9 cases red with it now)
- audits/r1297-ci-trim-2026-10-26: runners, times, decoys
This commit is contained in:
admin committed 2026-10-26 19:03:07 +01:00
1 parent 7e57e824a2
commit e33ec8210a
23 files changed
+2464 -7

No files matched your search

+2 -1
View File
@@ -305,7 +305,7 @@ stopping line that lies.
| **R-1223** | Monitoring & notifications | P3 | **After a reboot, a box whose tunnel ep0 refuses is invisible: the future stamp is gone, so neither the D30 re-key nor the hub's alarm sees it.** Fresh review 2026-10-23 night (`REPORT-night-2026-10-23.md`, A8 + H1): the kernel forgets `latest-handshake` when the interface restarts (0 = never); the agent then sends neither `last_handshake_age_s` nor `handshake_future_s`, and `checkWGTunnel` (`hub/internal/api/host_tunnel.go`) does not judge a registered, active tunnel with no handshake at all. Shape: the clock ran ahead, was corrected, the box rebooted (some households switch off at night), ep0 still holds the future stamp and starts nothing. | OPEN | — | Agent: report `tunnel_up_s` (how long the unit has been active) with no handshake; hub: a registered, active tunnel with no handshake for > 30 min while reporting → `wg_tunnel_stale`; the agent re-keys the same way after 30 min with no handshake. Tests both sides; rides an agent release + a hub deploy | CC |
| **R-1224** | Monitoring & notifications | P4 | **The operator's 1-hour mail cooldown for `wg_tunnel_stale` is per customer, not per host, and `wg_tunnel_ok` does not end it.** Fresh review 2026-10-23 night (`REPORT-night-2026-10-23.md`, H3): `notify/dispatcher.go:620` keys the cooldown `customerID:eventType`. Two hosts of one household going stale within an hour, or stale → ok → stale, store a second event row but mail once (the suppression is in `notification_log`). | OPEN | — | Key the cooldown by `host_id` from the details for host-scoped operator events, and let the matching ok event end it; a dispatcher test with two hosts; rides a hub deploy | CC |
| **R-1233** | Monitoring & notifications | P4 | **`[ERROR] [stacks] execCommand failed: exit status 1` names no command.** Seen once a night (2026-10-21 00:30:49 and 2026-10-23 02:17:14 UTC, each right after an app stopped for its dump); the command and stderr go only into the returned error, which the caller may drop (`internal/stacks/manager.go:1757`). An ERROR nobody can judge. | OPEN | — | Log the command's first two words (`docker inspect`, never the arguments — `docker exec … psql` can carry a password) and the stderr's first line; find the caller of the nightly one | CC |
| **R-1291** | Monitoring & notifications | P3 | **A held app gives the operator no lasting signal: one event at the stop, then nothing — demo-hp's Kimai sat held 11 days unseen.** Found 2026-10-25 night (R-1286). No held-app state reaches the hub (the report carries none; the hub shows none); the crash-loop stop sends `app_stopped_unhealthy` once. On a household box an app held after a crash loop, a failed update or a failed restore stays stopped until someone opens its page. | **OPEN** — ruled 2026-10-26 evening: **D53 = operator-only reminders first** (`09` §3 decision 265); CC builds `08` §10, commit only. **Earlier:** WAITING-ON-OPERATOR — 2026-10-26: the design is written (`08` §10 [DESIGN — proposed]: `app_held` / `app_hold_cleared` events + a daily re-send, a „Held apps" list on the host page, an operator reminder after a grace per reason, the hub never clears a hold; ~½ day controller + ~1 day hub). The pick is STATUS D53. **Earlier:** OPEN | — | a two-repo design: the controller reports its holds (stack, reason, since), the hub shows them on the host and mails a reminder after N days (the D-level of the reason); one row's worth of design first | CC |
| **R-1291** | Monitoring & notifications | P3 | **A held app gives the operator no lasting signal: one event at the stop, then nothing — demo-hp's Kimai sat held 11 days unseen.** Found 2026-10-25 night (R-1286). No held-app state reaches the hub (the report carries none; the hub shows none); the crash-loop stop sends `app_stopped_unhealthy` once. On a household box an app held after a crash loop, a failed update or a failed restore stays stopped until someone opens its page. | **OPEN — BUILT, commit only, NOT released** — 2026-10-26 evening: controller `5bd03ad` (`internal/holdreport`) + hub `7e57e824` (`held_apps`, `HeldAppsChecker`, the „Held apps" list on the host page and the Hosts page); red-proved; live on 9202 against a test hub with a 3-min grace (listed, one operator mail, Start cleared it, the row went, household leg `skipped operator_only`) — `audits/d53-held-apps-2026-10-26/`. Closes when both release. Ruled D53 = operator-only reminders first (`09` §3 decision 265). **Earlier:** WAITING-ON-OPERATOR — 2026-10-26: the design is written (`08` §10 [DESIGN — proposed]: `app_held` / `app_hold_cleared` events + a daily re-send, a „Held apps" list on the host page, an operator reminder after a grace per reason, the hub never clears a hold; ~½ day controller + ~1 day hub). The pick is STATUS D53. **Earlier:** OPEN | — | a two-repo design: the controller reports its holds (stack, reason, since), the hub shows them on the host and mails a reminder after N days (the D-level of the reason); one row's worth of design first | CC |
| **R-1296** | Monitoring & notifications | P4 | **When ep0's tenancy script refuses, the hub log shows only its one-line error — PBS's own reason (the 403 `missing Datastore.Audit|Datastore.Backup on …`) stays in the script's stderr, which the hub drops on a refusal.** Found 2026-10-26 diagnosing R-1295: the cause was visible only by reading ep0's proxy journal over root SSH. `hub/internal/tenantsync/client.go` `parseResponse` returns the JSON error and ignores stderr; the script keeps secrets off stderr by design (its header), so the tail is safe to log. | OPEN | — | append the last ~3 stderr lines to the refusal error (bounded, never stdout); a test that a secret on stdout never appears | CC |
## Hub & operator — 12 rows (P3 3, P4 9)
@@ -400,3 +400,4 @@ stopping line that lies.
| R-1035 | 2026-12-09 | the same three numbers 8 weeks after publishing, compared with the 4-week read; a null at 4 weeks is not a verdict, both reads are needed; detail in the R-1035 row |
<!-- DUE-CHECKS-END -->
| **R-1297** | Process & tooling | P3 | **CI waits dominate a working day: felhom.eu's CI run takes ~13 min and the ONE runner makes every repo's job wait behind it (up to 10–17 min queued).** Measured 2026-10-26 (operator: „these CI waiting times are getting out of hand"): since 10-25 felhom.eu runs median 793 s, the controller/agent/catalog 57–102 s; queue median 5 s, max 548–1023 s. In run 2366, ~10 of 13 minutes are four suites — `test_gate_decoys.py` 232 s, `test_due_checks_gate.py` 133 s, `test_repo_gates.py` 132 s (it runs the whole gate set a SECOND time), `dooplex-offsite/test_dooplex_offsite.py` 69 s — plus `hub-gofmt` ~110 s (a cold Go cache per run). **Ruled 2026-10-26 by the operator: BOTH** — (A) trim felhom.eu CI: the heavy suites only when their own files change, no duplicate full run, a Go cache for hub-gofmt; (B) a second CI runner on DooPlex. | OPEN — (B) DONE by the operator 2026-10-26: a second runner, both runners 2 CPUs (`09` §3 decision 267); (A) CC, 2026-10-26 evening | — | (A) CC, a run of its own (a CI scope change, ruled); (B) a DooPlex change — a brief or the homelab session; and CC waits on CI only before a release or at the end of a run | CC |
| **R-1298** | Monitoring & notifications | P3 | **Scratch guest 9202 carries leftovers made outside the product, and D53 would remind about them every week once released: four crash-loop holds since 2026-09-24 for apps that are not installed (`chaoscrash`, `chaosoom`, `chaosoomb`, `gokapi`), and a PrivateBin `app.yaml` (2026-10-25) with no container that the product reads as „installed".** Found 2026-10-26 evening in the D53 live proof: the new reporter sent the four holds at its first tick and the test hub mailed four reminders („held 32 days"). The product's removal clears an install's hold (R-704); these apps were removed by hand in a chaos drill. On a household box the same shape would mail the operator weekly about an app that does not exist. | OPEN | — | decide whether the reporter (or the hub) skips an `unhealthy_stop` / `update_failed` / `putback_pending` hold whose app is not installed (a restore hold stays reported — operator-cleared by design, R-379); then clear 9202's four holds with `-clear-restore-hold` and remove PrivateBin through the product | CC |